Bandwidthd issues?

phil.davis

2.2-BETA (amd64)
built on Sat Sep 27 14:17:44 CDT 2014
FreeBSD 10.1-PRERELEASE

I had this in system.log

Sep 28 09:46:05 apu22 php-fpm[54340]: /rc.newwanip: Creating rrd update script
Sep 28 09:46:05 apu22 bandwidthd: Opening re0
Sep 28 09:46:05 apu22 bandwidthd: Opening re0
Sep 28 09:46:05 apu22 bandwidthd: Packet Encoding: Ethernet
Sep 28 09:46:05 apu22 bandwidthd: Packet Encoding: Ethernet
Sep 28 09:46:06 apu22 kernel: done.
Sep 28 09:46:07 apu22 php-fpm[54340]: /rc.newwanip: pfSense package system has detected an ip change 0.0.0.0 ->  192.168.111.100 ... Restarting packages.
Sep 28 09:46:07 apu22 check_reload_status: Starting packages
Sep 28 09:46:07 apu22 check_reload_status: Reloading filter
Sep 28 09:46:08 apu22 php: rc.bootup: ROUTING: setting default route to 10.49.0.250
Sep 28 09:46:08 apu22 kernel: done.
Sep 28 09:46:08 apu22 php-fpm[54340]: /rc.start_packages: Restarting/Starting all packages.
Sep 28 09:46:08 apu22 kernel: done.
Sep 28 09:46:08 apu22 php-fpm[54340]: /rc.start_packages: You should specify an interface for bandwidthd to listen on. Exiting.
Sep 28 09:46:08 apu22 check_reload_status: Updating all dyndns
Sep 28 09:46:08 apu22 kernel: .done.
Sep 28 09:46:09 apu22 check_reload_status: Could not connect to /var/run/php-fpm.socket
Sep 28 09:46:10 apu22 check_reload_status: Could not connect to /var/run/php-fpm.socket
... could not connect message then repeats a few times a second...

It seems that something in starting (or even in this case not starting) bandwidthd has resulted in php-fpm going AWOL.
There were no php-fpm processes on the system.
I found this useful command in /etc/rc and ran it from the console:

/usr/local/sbin/php-fpm -c /usr/local/lib/php.ini -y /usr/local/lib/php-fpm.conf -RD

Now I got php-fpm processes like this:

[2.2-BETA][root@apu22.localdomain]/var/run(25): ps aux | grep php
root   20815   0.0  1.4 216392 28572  -  Ss   10:13AM  0:00.01 php-fpm: master
root   20854   0.0  2.3 220488 45824  -  S    10:13AM  0:00.44 php-fpm: pool li
root   38102   0.0  0.1  18816  2300 u0  S+   10:14AM  0:00.00 grep php

and files in /var/run like:

[2.2-BETA][root@apu22.localdomain]/var/run(26): ls -l php*
-rw-r--r--  1 root  wheel   5 Sep 28 10:13 php-fpm.pid
srw-------  1 root  wheel   0 Sep 28 10:13 php-fpm.socket
-rw-r--r--  1 root  wheel  58 Sep 28 09:45 php_modules_load_errors.txt

The webGUI now works. I removed package bandwidthd and rebooted. It all came up fine.

At least this might help someone get their webGUI running again so they can use it to easily remove bandwidthd.

I wonder what in bandwidthd is causing php-fpm to disappear?

swooper

Being a N00b with new hardware, I wasn't sure what i'd done wrong. I can confirm that this was the issue for me too.

Once I installed Bandwithd, the firewall would work, but i'd get a 503 tryign to access the WebGUI. Tried and tested everything else out. I'll have to be really slow and carefull to add packages now (on the 18-10-2014 snapshot)

charliem

Hmm, I can't reproduce this problem. I've upgraded several times, with the bandwidthd package installed, as well as remove & re-install the package, no problem. Currently running:

bandwidthd 2.0.1_6 pkg v.0.5, and 
pfsense 2.2-BETA (amd64) built on Thu Oct 16 18:20:50 CDT 2014 FreeBSD 10.1-RC2

arrmo

Very odd - definitely having this issue here. In fact, very odd - I manually removed BandwidthD, so I can get back to the GUI. It says it's not installed, but it's running (and storing data to my PostgreSQL database, as configured earlier).

Any things I can help check?

Thanks!

charliem

Does anything show up in /tmp/php_errors.txt?

This line, from Phil's log below, looks strange. Can you verify you have configured an interface for bandwidthd?

Sep 28 09:46:08 apu22 php-fpm[54340]: /rc.start_packages: You should specify an interface for bandwidthd to listen on. Exiting.

arrmo

Hi,

Nothing here at least … but my BandwidthD is running, even though it says it's not installed ... :(.

Do I need to reinstall? I can, to check this ... but then I lose the GUI, and it seems to be a very manual process to go back?

Thanks!

arrmo

Hi,

OK, some very odd behavior with BandwidthD … :(. A couple items, below,

I updated my install, but pfSense still says BandwdithD is not installed - even though I now have 2 copies of bandwidthd running it seems. Not sure how / why. Is there a way to fully / cleanly uninstall BandwidthD, to try again?
I reinstalled BandwidthD from the GUI -> yep, my GUI is broken again (clearly the BandwidthD install breaks it!). Nothing in /tmp/php_errors.txt though. Any other places to look?
When I try to manually run /usr/pbi/bandwidthd-amd64/local/bandwidthd/bandwidthd now, I get the error message: Shared object "libpq.so.5" not found, required by "bandwidthd"

Suggestions?

Thanks!

arrmo

And a couple more observations,
4) The command above works to get the GUI going again (/usr/local/sbin/php-fpm -c /usr/local/lib/php.ini -y /usr/local/lib/php-fpm.conf -RD). Thanks!
5) I then removed and reinstalled BandwidthD … now it seems to be broken (no GUI to configure it, unable to restart it).

Thoughts?

Thanks!

arrmo

Sorry for the barrage of postings, but trying to help debug this. A couple more observations …
6) I updated pfSense, and then BandwidthD gets reinstalled (as I have it installed now) -> does not work from the GUI, and I have to execute the command above to get the GUI restarted.
7) I can manually start bandwidthd after reboot, by executing /usr/local/etc/rc.d/bandwidthd.sh start

Thanks!

lshantz

The sad truth is, this install from the Pfsense library has been borked for TWO years!!!! Does the author not maintain it, or how does that work? Very unfortunate.

Unfortunately I am a little confused about this "patch". I didn't see where it can be downloaded, or how to install it to make it work. Do I need to manually make the code change at the terminal?

charliem

The patch mentioned near the start of this thread has been applied already, you don't need to edit anything. It fixed bandwidthd so that it would start up correctly with a full install version of pfSense. (Prior to that, it only started on embedded versions).

Problems mentioned later in the thread, such as bandwidthd causing php-fpm (and thus the webgui) to fail, have not been addressed, tracked down or patched. Patches welcome.

Note that the later issues do not apply universally; ie, I continue to use bandwidthd without issue on 64bit 2.2Beta

arrmo

Hi,

I admit, still a bit confused the fact that some installations see this, others don't. Any thoughts on what to check?

Thanks!

cmb

I split part of the old thread into this thread, leaving here only the part referencing something that might still be an issue for some.

@arrmo:

I admit, still a bit confused the fact that some installations see this, others don't. Any thoughts on what to check?

I'm not seeing this either, with bandwidthd running on a handful of different systems on latest 2.2 snapshot.

If you're still seeing this, make sure you're on the latest 2.2 snapshot and can still replicate, and please provide the following info:

which platform, full or nano? (not sure? If you have a Diagnostics>Nanobsd menu, it's nano. otherwise it's full)
32 or 64 bit? (not sure? look for i386 or amd64 on dashboard)
what other packages do you have installed?
have you installed anything manually at the command line via 'pkg'?
is there anything about your setup or configuration that's atypical? anything really out of the ordinary you're doing

cmb

@lshantz:

The sad truth is, this install from the Pfsense library has been borked for TWO years!!!!

That's not the truth, it's one of the most widely-used packages. I'm not aware of any issues in versions prior to 2.2, the things discussed in this thread and the one where this was originally posted were specific to 2.2 back in the alpha days. It's working fine in 2.2 now as well.

arrmo

Hi,

Still trying to get this working. It sort of works … but breaks the GUI (I can work around that), but also creates two copies of bandwidthd on upgrade (that's more painful). Is there a way to fully de-install, so I can try a re-install? It could be my machine, as others don't see this, but not sure what all to remove to clean it up.

Thanks!

arrmo

Hi,

FYI - did a complete uninstall of bandwidthd, then reinstalled. All looked good, then I rebooted … :(. Again I had two copies of bandwidthd running, and it broke my GUI.

So the reboot after seems critical (to demonstrate the problem).

Thanks.

cmb

Anyone who can replicate the issue with bandwidthd breaking the web interface, is there anything unusual about your setup? Seems it's easy for some to replicate, but most of us, including myself with at least a handful of 2.2 systems running bandwidthd, have never been able to replicate. Could someone share a config backup from a system that exhibits the issue?

arrmo

Yep, I can get that to you - but is there an easy way to remove any sensitive information?

Thanks!

reggie14

For what its worth, installing bandwidthd seemed to bork my webGUI too. More specifically, after installation I started configuring it (i.e., selecting the LAN interface and identifying the subnet), I hit save and that's when the webGUI went down.

I restarted the pfsense box and managed to get back in. bandwidthd isn't running, and I don't plan to enable it for fear of not being able to get back in next time it goes down.

My setup isn't hugely special. I'm running 2.2-RC (amd64), Jan 02 build, with packages apinger, darkstat and snort.

Brrm

@reggie14:

For what its worth, installing bandwidthd seemed to bork my webGUI too. More specifically, after installation I started configuring it (i.e., selecting the LAN interface and identifying the subnet), I hit save and that's when the webGUI went down.

I restarted the pfsense box and managed to get back in. bandwidthd isn't running, and I don't plan to enable it for fear of not being able to get back in next time it goes down.

My setup isn't hugely special. I'm running 2.2-RC (amd64), Jan 02 build, with packages apinger, darkstat and snort.

Same thing happened to me. But after a reboot it's running without issues.