2.2 upgrade- Do we really need 200 sysctl's running for thermal sensor widget?
-
so far 3 of the boxes I've upgraded, have all broke on widgets.
But this one is exceptionally fun. 200+ sysctl processes running, trying to feed the thermal sensor widget, and the
webconfigurator is stalled.I know I have a nice 8 core atom, but I'm thinking this is excessive?
root 32715 0.0 0.0 17144 2596 - I 8:44PM 0:00.00 sh -c /sbin/sysctl -a | grep temperature root 32914 0.0 0.1 12420 5392 - R 8:44PM 0:00.10 /sbin/sysctl -aps auxwww | grep sysctl | wc -l 245 -
Either something is somehow running getThermalSensorsData hundreds of times simultaneously (highly unlikely), seems sysctl output must be stalling on your system for some reason.
What output do you get if you run "sysctl -a" at a command prompt, does it hang at some point, or finish after spitting out a bunch of stuff?
-
good call
via ssh: sysctl -astalls after:
dev.cpu.0.%desc: ACPI CPU dev.cpu.0.%driver: cpu dev.cpu.0.%location: handle=\_PR_.CPU0 dev.cpu.0.%pnpinfo: _HID=none _UID=0 dev.cpu.0.%parent: acpi0 dev.cpu.0.freq: 2400 dev.cpu.0.freq_levels: 2400/10000 2300/9250 2200/8500 2100/7750 2000/7000 1900/6250 1800/5500 1700/4750 1600/4000 1500/3250 1400/2500 1300/1750 1200/1000 1050/875 900/750 750/625 600/500 450/375 300/250 150/125 dev.cpu.0.cx_supported: C1/1/1 C2/2/41 dev.cpu.0.cx_lowest: C1 dev.cpu.0.cx_usage: 100.00% 0.00% last 978us dev.cpu.0.coretemp.delta: 72 dev.cpu.0.coretemp.resolution: 1 dev.cpu.0.coretemp.tjmax: 98.0C dev.cpu.0.coretemp.throttle_log: 0 dev.cpu.0.temperature: 26.0C dev.cpu.1.%desc: ACPI CPU dev.cpu.1.%driver: cpu dev.cpu.1.%location: handle=\_PR_.CPU1 dev.cpu.1.%pnpinfo: _HID=none _UID=0 dev.cpu.1.%parent: acpi0 dev.cpu.1.cx_supported: C1/1/1 C2/2/41 dev.cpu.1.cx_lowest: C1 dev.cpu.1.cx_usage: 100.00% 0.00% last 286us dev.cpu.1.coretemp.delta: 72 dev.cpu.1.coretemp.resolution: 1 dev.cpu.1.coretemp.tjmax: 98.0C dev.cpu.1.coretemp.throttle_log: 0 dev.cpu.1.temperature: 26.0C dev.cpu.2.%desc: ACPI CPU dev.cpu.2.%driver: cpu dev.cpu.2.%location: handle=\_PR_.CPU2 dev.cpu.2.%pnpinfo: _HID=none _UID=0 dev.cpu.2.%parent: acpi0 dev.cpu.2.cx_supported: C1/1/1 C2/2/41 dev.cpu.2.cx_lowest: C1 dev.cpu.2.cx_usage: 100.00% 0.00% last 64219uscan't CTRL-C out of it, can't kill -9 it.
-
Same exact issue here with 2.2. Have Intel Atom C2758. If the temp widget is in dashboard, dashboard stalls and I have to reboot the box through IPMI/console. When I run the shell command, stalls and ctrl+C doesn't get out, have to reboot the box using the web interface.
Another box I have running 2.2 has the temp widget in the dashboard, but the cpu doesn't support it and doesn't display any relevant information. However, it doesn't lock up the dashboard.
-
What are the specifics of the hardware both of you are using? I checked multiple of the C2758s we sell that are in production to make sure we didn't miss anything in testing, and none exhibit this issue.
Same exact issue here with 2.2. Have Intel Atom C2758. If the temp widget is in dashboard, dashboard stalls and I have to reboot the box through IPMI/console. When I run the shell command, stalls and ctrl+C doesn't get out, have to reboot the box using the web interface.
Does it stall at the same spot that Sn3ak's output shows his stalling?
-
Exact model here: http://www.supermicro.com/products/system/1U/5018/SYS-5018A-MHN4.cfm
Yes, my shell command stalls out at the same spot.
Never had the issue just before the 2.2 upgrade.
I should also mention, the temperature meter in the system information widget works fine, its the thermal sensors widget that has the problem.
-
I have the A1SRi-2758F. 8GB (2x4GB Samsung) ram, 120 Samsung Evo. Mini-box m350 case / 80w PicoPSU,
2 40mm fans.Nothing hooked up other than WAN/LAN/IPMI interface. bios 1.0c / bmc v1.38.
Looks like there's updates on both. I'll get them updated and report back.
Edited: Clarified what I have / firmware revisions
-
its the thermal sensors widget that has the problem.
That's the only one that dumps all the sysctls and parses them, which is why.
Looks like you're both using even the exact same motherboard as the systems I tried that have no such issue. BIOS settings all at defaults? They are on the ones I checked. Not sure what to suggest, if you find further info on that circumstance, let us know.
-
His board is slightly different than mine.
His: A1SRi-2758F
Mine: A1SRM-2758FBut I think they are the same chipset.
The widget was working before the 2.2 update so the current BIOS settings should be OK. My thinking is a freeBSD driver is missing or bugged from the new build to support our particular boards? But you said you had the thermal widget working with the same board type. My BIOS settings are nothing crazy, is there a particular setting to check? I have a 50GB SSD with 8gb ram. Ever since I have removed the thermal widget, system has been running great.
Ideas?
-
@cmb:
That's the only one that dumps all the sysctls and parses them, which is why.
Looks like you're both using even the exact same motherboard as the systems I tried that have no such issue. BIOS settings all at defaults? They are on the ones I checked. Not sure what to suggest, if you find further info on that circumstance, let us know.
With bios reset to defaults, exact same issue. Running bios 1.0c, do you know what yours are running? anything new seems to ship with 1.0c.
I've also found out, it doesn't hang at the same place. it randomly hangs between cpu2 and cpu7, never before, never after (so far).
I mucked around with a bunch of things, such as turning off WHEA support in bios, no change.
My BMC was out of date, and is now running the latest 1.76?. No difference.I've ordered another identical board for another system. I will report back if anything differs.
BTW: I assumed powerd may have been contributing, turning it off made no change either.
I'm really curious to see if a fresh install solves the problem, but couldn't do so on the other rig, as it would take too much time.
-
I did a fresh install for someone else who has the exact same server (I got two when I purchased), but did the install on my server (removed my hhd and installed his). I loaded from scratch 2.2, smashed next for wizard, enabled cpu temp in advanced, and added widget, worked. Loaded his config from 2.1.2 which added all of his packages and settings back, etc, widget froze just like on my configuration (which is nearly identical). Its a software/configuration issue. I didn't go through all of the packages and uninstall/reinstall them to see if it was a package. However, I did remove pfBlocker and added pfBlockerNG, no change to widget functionality (still locks up pfSense webConfigurator). Also, the reset webConfigurator option from console does not make the webConfigurator accessible, have to reboot entire system.
I have the following packages installed on both servers (latest version of each):
bandwidthd
File Manager
Filer
LCDproc
Notes
OpenVPN Client Export Utility
pfBlockerNG
RRD Summary
Service Watchdog
snortIdeas?
-
I concur. I too have found on a new system, and transferring the SSD from it to mine that a fresh install works.
On my existing config I had uninstalled all packages prior to the upgrade, and removed all widgets except the default
system info. It still caused breakage.What I haven't been able to do yet, is diff the filesystems to see if something didn't upgrade right (I did confirm the kernel, and sysctl md5 hash). Or to selectively try restoring parts of my config over a new working one.
That part is annoying, as I just did a fresh start over for 2.1.? since my original config was from an early 1.something.
I never even got the chance to finish porting over the config. Figures. -
I'm not sure it was the upgrade process. Since after loading a 2.1.2 config into 2.2 fresh install borks it as well. Now, it could be that the config for 2.1.2 is not compatible with 2.2 in someway. Any insight on this?
Also wondering if a config reset of 2.2 upgraded from 2.1.2 would bring back a functional widget. May just have to reset everything back up manually? :'(
-
It's a pretty low-level failure, the root cause being that 'sysctl -a' hangs. I can't imagine what could be in the config, or even installed for that matter, which would impact that. Redoing the config from scratch, if you end up with the same resulting configuration, almost certainly won't do anything differently and likely isn't worth the time relative to other things that are more likely to narrow down the cause.
See if you can bisect to narrow down where the issue starts. First restore your config with no packages at all. If that works, restore it with half the packages. If that works, restore it with all the packages. That should help to narrow down the specific part of the config that's an issue.
-
Run "kldstat" and "ps xuawww" when it's in the problem state and post the results here.
That would almost have to be a BIOS or kernel module interaction stopping there.
-
kldstat:
Id Refs Address Size Name 1 18 0xffffffff80200000 22c57e8 kernel 2 1 0xffffffff82611000 5475 aesni.ko 3 1 0xffffffff82617000 cf4 coretemp.ko 4 1 0xffffffff82618000 2200 ichsmb.ko 5 2 0xffffffff8261b000 822 smbus.ko 6 1 0xffffffff8261c000 681c ipmi.konote: I've tried with all, except maybe aesni removed. no difference.
ps:
USER PID %CPU %MEM VSZ RSS TT STAT STARTED TIME COMMAND root 11 604.7 0.0 0 128 - RL 1Feb15 100149:36.31 [idle] root 9 100.0 0.0 0 16 - RL 1Feb15 15530:21.46 [idlepoll] root 95524 73.7 0.4 228284 31332 - R 11:40AM 0:11.89 /usr/local/bin/php -f /usr/local/sbin/prefixes.php dhcpd 31330 1.5 0.1 20852 10276 - Ss 1Feb15 231:18.58 /usr/local/sbin/dhcpd -6 -user dhcpd -group _dhcp -chroot /var/dhcpd -cf /etc/dhcpdv6.conf -pf /var/run/dhcpdv6.pid igb0 root 32948 0.7 0.0 14668 2472 - Ss 1Feb15 183:51.11 /usr/sbin/syslogd -s -c -c -l /var/dhcpd/var/run/log -P /var/run/syslog.pid -f /var/etc/syslog.conf -b 10.0.0.254 root 0 0.0 0.0 0 672 - DLs 1Feb15 0:47.62 [kernel] root 1 0.0 0.0 9472 828 - ILs 1Feb15 0:00.05 /sbin/init -- root 2 0.0 0.0 0 16 - DL 1Feb15 0:00.00 [crypto] root 3 0.0 0.0 0 16 - DL 1Feb15 0:00.00 [crypto returns] root 4 0.0 0.0 0 48 - DL 1Feb15 0:00.00 [cam] root 5 0.0 0.0 0 16 - DL 1Feb15 2:46.75 [pf purge] root 6 0.0 0.0 0 16 - DL 1Feb15 0:00.00 [sctp_iterator] root 7 0.0 0.0 0 16 - DL 1Feb15 0:18.70 [pagedaemon] root 8 0.0 0.0 0 16 - DL 1Feb15 0:00.00 [vmdaemon] root 10 0.0 0.0 0 16 - DL 1Feb15 0:00.00 [audit] root 12 0.0 0.0 0 1024 - WL 1Feb15 99:54.48 [intr] root 13 0.0 0.0 0 128 - DL 1Feb15 0:00.00 [ng_queue] root 14 0.0 0.0 0 48 - DL 1Feb15 0:00.06 [geom] root 15 0.0 0.0 0 16 - DL 1Feb15 8:18.18 [rand_harvestq] root 16 0.0 0.0 0 128 - DL 1Feb15 2:43.08 [usb] root 17 0.0 0.0 0 16 - DL 1Feb15 0:00.02 [pagezero] root 18 0.0 0.0 0 16 - DL 1Feb15 0:04.15 [bufdaemon] root 19 0.0 0.0 0 16 - DL 1Feb15 0:04.19 [vnlru] root 20 0.0 0.0 0 16 - DL 1Feb15 3:03.59 [syncer] root 46 0.0 0.0 0 16 - DL 1Feb15 0:23.52 [md0] root 51 0.0 0.0 0 16 - DL 1Feb15 1:16.99 [md1] root 277 0.0 0.3 224288 23432 - Ss 1Feb15 0:25.96 php-fpm: master process (/usr/local/lib/php-fpm.conf) (php-fpm) root 292 0.0 0.0 19032 2536 - INs 1Feb15 0:00.07 /usr/local/sbin/check_reload_status root 294 0.0 0.0 19032 2356 - IN 1Feb15 0:00.00 check_reload_status: Monitoring daemon of check_reload_status root 307 0.0 0.1 13164 4408 - Is 1Feb15 0:00.44 /sbin/devd -q root 10737 0.0 0.0 16812 2304 - Ss 1Feb15 1:41.59 /usr/local/sbin/filterlog -i pflog0 -p /var/run/filterlog.pid root 11765 0.0 0.1 32428 5132 - Is 1Feb15 0:00.01 /usr/sbin/sshd root 11865 0.0 0.0 14756 2184 - Is 1Feb15 0:00.02 /usr/local/sbin/sshlockout_pf 15 root 12245 0.0 0.0 16788 2244 - S 11:40AM 0:00.00 ping -c 3 -S 10.0.0.254 192.168.11.1 root 12389 0.0 0.0 18788 2288 - Is 1Feb15 0:00.03 /usr/sbin/inetd -wW -R 0 -a 127.0.0.1 /var/etc/inetd.conf root 14686 0.0 0.5 224288 40036 - I 11:37AM 0:00.24 php-fpm: pool lighty (php-fpm) root 15749 0.0 0.0 12464 2140 - Ss 1Feb15 3:46.85 /usr/local/sbin/apinger -c /var/etc/apinger.conf root 15933 0.0 0.0 28332 2948 - I 1Feb15 0:12.19 /usr/local/bin/rrdtool - root 23101 0.0 0.1 58988 7676 - S 1Feb15 0:34.31 /usr/local/sbin/lighttpd -f /var/etc/lighty-webConfigurator.conf root 24026 0.0 0.1 60552 10144 - Ss Tue10PM 0:23.68 /usr/local/libexec/nut/snmp-ups -a SmartUPS1500 root 27277 0.0 0.0 22948 2564 - Ss Tue10PM 0:06.89 /usr/local/sbin/upsd -u root root 30835 0.0 0.0 17144 2668 - S 1Feb15 0:25.13 /bin/sh /usr/local/pkg/sqpmon.sh root 31786 0.0 0.0 12412 1964 - Ss 1Feb15 0:11.41 /usr/local/sbin/dhcpleases6 -c /usr/local/bin/php -f /usr/local/sbin/prefixes.php|/bin/sh -l /var/dhcpd/var/db/dhcpd6.leases root 31881 0.0 0.0 18840 2540 - Is Tue10PM 0:00.00 /usr/local/sbin/upsmon SmartUPS1500@localhost uucp 32022 0.0 0.0 18840 2584 - S Tue10PM 0:05.36 /usr/local/sbin/upsmon SmartUPS1500@localhost root 32374 0.0 0.0 14540 2236 - S 1Feb15 0:15.80 /usr/local/sbin/radvd -p /var/run/radvd.pid -C /var/etc/radvd.conf -m syslog root 32835 0.0 0.1 55632 6180 - Ss 11:37AM 0:00.13 sshd: admin@pts/0 (sshd) root 35324 0.0 0.1 55632 6212 - Is 11:38AM 0:00.13 sshd: admin@pts/1 (sshd) root 37614 0.0 0.0 25480 2968 - Is 1Feb15 0:00.03 /usr/local/libexec/ipsec/starter --daemon charon root 37833 0.0 4.4 618236 363560 - Ss 1Feb15 7:08.16 /usr/local/libexec/ipsec/charon --use-syslog root 38025 0.0 0.0 23224 2664 - Is 1Feb15 0:18.21 /usr/local/sbin/filterdns -p /var/run/filterdns-ipsec.pid -i 60 -c /var/etc/ipsec/filterdns-ipsec.hosts -d 1 root 39019 0.0 0.2 67076 19296 - Ss 1Feb15 2:20.22 /usr/sbin/bsnmpd -c /var/etc/snmpd.conf -p /var/run/snmpd.pid root 49751 0.0 0.0 14548 2024 - Ss 1Feb15 1:45.10 /usr/sbin/powerd -b hadp -a hadp -n hadp root 58823 0.0 0.0 0 16 - DL 11:39AM 0:00.01 [ipmi0: kcs] root 59806 0.0 0.1 47888 6800 - Is 1Feb15 0:00.04 /usr/local/sbin/mpd4 -b -d /var/etc/l2tp-vpn -p /var/run/l2tp-vpn.pid -s l2tps l2tps root 64246 0.0 0.0 14756 2320 - Is 4Feb15 0:00.01 /usr/local/sbin/sshlockout_pf 15 root 69501 0.0 0.2 73644 13912 - Is 1Feb15 0:00.00 /usr/local/sbin/squid -f /usr/pbi/squid-amd64/local/etc/squid/squid.conf proxy 70004 0.0 17.0 1609644 1414836 - S 1Feb15 7:28.76 (squid-1) -f /usr/pbi/squid-amd64/local/etc/squid/squid.conf (squid) root 70462 0.0 0.0 12412 1936 - Is 1Feb15 0:00.00 /usr/local/bin/minicron 240 /var/run/ping_hosts.pid /usr/local/bin/ping_hosts.sh root 70575 0.0 0.0 12412 1948 - S 1Feb15 0:00.41 minicron: helper /usr/local/bin/ping_hosts.sh (minicron) root 70691 0.0 0.0 12412 1936 - Is 1Feb15 0:00.00 /usr/local/bin/minicron 3600 /var/run/expire_accounts.pid /usr/local/sbin/fcgicli -f /etc/rc.expireaccounts root 71301 0.0 0.0 12412 1948 - I 1Feb15 0:00.03 minicron: helper /usr/local/sbin/fcgicli -f /etc/rc.expireaccounts (minicron) root 71569 0.0 0.0 12412 1936 - Is 1Feb15 0:00.00 /usr/local/bin/minicron 86400 /var/run/update_alias_url_data.pid /usr/local/sbin/fcgicli -f /etc/rc.update_alias_url_data root 71960 0.0 0.0 12412 1948 - I 1Feb15 0:00.00 minicron: helper /usr/local/sbin/fcgicli -f /etc/rc.update_alias_url_data (minicron) root 77423 0.0 0.0 14756 2248 - Is 1Feb15 0:00.02 /usr/local/sbin/sshlockout_pf 15 root 77929 0.0 0.0 23492 3096 - Ss Tue10PM 0:01.06 ladvd: master [priv] (ladvd) ladvd 78134 0.0 0.0 23492 3076 - S Tue10PM 0:01.18 ladvd: child (ladvd) root 85967 0.0 0.1 18900 5128 - S 1Feb15 0:22.76 /usr/local/sbin/arpwatch -d -f /var/log/arp.dat -i igb0 root 89552 0.0 0.2 28172 18060 - Ss Mon11PM 4:07.87 /usr/local/sbin/ntpd -g -c /var/etc/ntpd.conf -p /var/run/ntpd.pid root 94712 0.0 0.0 16672 2420 - Is Tue10PM 0:00.30 /usr/sbin/cron -s root 94912 0.0 0.0 8312 1964 - SN 11:40AM 0:00.00 sleep 60 root 95257 0.0 0.0 17144 2656 - S 11:40AM 0:00.00 sh -c /usr/local/bin/php -f /usr/local/sbin/prefixes.php|/bin/sh root 95863 0.0 0.0 17144 2644 - S 11:40AM 0:00.00 /bin/sh root 96816 0.0 0.1 18900 4992 - S Tue10PM 0:03.31 /usr/local/sbin/arpwatch -d -f /var/log/arp.dat -i igb0 root 99779 0.0 0.0 8312 1964 - S 11:40AM 0:00.00 sleep 55 root 99978 0.0 0.0 17144 2684 - S 11:40AM 0:00.02 /bin/sh /usr/local/bin/ping_hosts.sh root 46612 0.0 0.0 17144 2436 v0- SN 1Feb15 2:52.70 /bin/sh /var/db/rrd/updaterrd.sh root 77253 0.0 0.0 43576 2692 v0 Is 1Feb15 0:00.02 login [pam] (login) root 77735 0.0 0.0 17144 2720 v0 I 1Feb15 0:00.01 -sh (sh) root 78072 0.0 0.0 17144 2604 v0 I+ 1Feb15 0:00.01 /bin/sh /etc/rc.initial root 12530 0.0 0.0 18816 2420 0 R+ 11:40AM 0:00.00 ps xuawww root 33245 0.0 0.0 17144 2680 0 Is 11:37AM 0:00.01 /bin/sh /etc/rc.initial root 34614 0.0 0.0 17484 3868 0 S 11:37AM 0:00.03 /bin/tcsh root 37139 0.0 0.0 17144 2680 1 Is 11:38AM 0:00.01 /bin/sh /etc/rc.initial root 38597 0.0 0.0 17484 3776 1 I 11:38AM 0:00.01 /bin/tcsh root 60305 0.0 0.1 12420 5476 1 R+ 11:39AM 0:00.22 sysctl -anote: you will see packages running, during the initial failure, all packages had been removed prior to the 2.2 upgrade when problem was
first found. Packages weren't re-added until a week or so later, when I gave up on a quick resolution. also powerd was disabled during some of my tests. I do believe I disabled ntpd during the test as well, but not 100% sure. -
and Thank you Jimp.
You lead me to the solution/workaround.
Notice idlepoll eating 100% of one cpu. I turned off device polling in advanced -> networking, the moment I hit save, sysctl finished.
I ran it again, and got the full sysctl output as expected.I turned device polling back on, reran the test. and it hung up like previously. the moment I turned it off / hit save. it spit out the remaining info.
So the problem is something related to device polling.
ps after turning it off, notice /usr/local/bin/php -f /usr/local/sbin/prefixes.php/usr/local/bin/php -f /usr/local/sbin/prefixes.php stopped eatting 80% cpu as well, which seemed to be running solid until I turned off device polling, though without more testing, that could be a coincidence?!
USER PID %CPU %MEM VSZ RSS TT STAT STARTED TIME COMMAND root 11 799.2 0.0 0 128 - RL 1Feb15 100249:17.64 [idle] root 55095 1.6 0.4 224288 34184 - S 11:56AM 0:00.28 php-fpm: pool lighty (php-fpm) dhcpd 31330 0.4 0.1 20852 10276 - Ss 1Feb15 231:33.66 /usr/local/sbin/dhcpd -6 -user dhcpd -group _dhcp -chroot /var/dhcpd -cf /etc/dhcpdv6.conf -pf /var/run/dhcpdv6.pid igb0 root 32948 0.3 0.0 14668 2472 - Ss 1Feb15 184:03.13 /usr/sbin/syslogd -s -c -c -l /var/dhcpd/var/run/log -P /var/run/syslog.pid -f /var/etc/syslog.conf -b 10.0.0.254 root 0 0.0 0.0 0 672 - DLs 1Feb15 0:47.65 [kernel] root 1 0.0 0.0 9472 828 - ILs 1Feb15 0:00.05 /sbin/init -- root 2 0.0 0.0 0 16 - DL 1Feb15 0:00.00 [crypto] root 3 0.0 0.0 0 16 - DL 1Feb15 0:00.00 [crypto returns] root 4 0.0 0.0 0 48 - DL 1Feb15 0:00.00 [cam] root 5 0.0 0.0 0 16 - DL 1Feb15 2:46.96 [pf purge] root 6 0.0 0.0 0 16 - DL 1Feb15 0:00.00 [sctp_iterator] root 7 0.0 0.0 0 16 - DL 1Feb15 0:18.72 [pagedaemon] root 8 0.0 0.0 0 16 - DL 1Feb15 0:00.00 [vmdaemon] root 9 0.0 0.0 0 16 - DL 1Feb15 15540:06.93 [idlepoll] root 10 0.0 0.0 0 16 - DL 1Feb15 0:00.00 [audit] root 12 0.0 0.0 0 1024 - WL 1Feb15 100:01.99 [intr] root 13 0.0 0.0 0 128 - DL 1Feb15 0:00.00 [ng_queue] root 14 0.0 0.0 0 48 - DL 1Feb15 0:00.07 [geom] root 15 0.0 0.0 0 16 - DL 1Feb15 8:18.63 [rand_harvestq] root 16 0.0 0.0 0 128 - DL 1Feb15 2:43.25 [usb] root 17 0.0 0.0 0 16 - DL 1Feb15 0:00.02 [pagezero] root 18 0.0 0.0 0 16 - DL 1Feb15 0:04.15 [bufdaemon] root 19 0.0 0.0 0 16 - DL 1Feb15 0:04.20 [vnlru] root 20 0.0 0.0 0 16 - DL 1Feb15 3:03.79 [syncer] root 46 0.0 0.0 0 16 - DL 1Feb15 0:23.54 [md0] root 51 0.0 0.0 0 16 - DL 1Feb15 1:17.08 [md1] root 277 0.0 0.3 224288 23432 - Ss 1Feb15 0:26.01 php-fpm: master process (/usr/local/lib/php-fpm.conf) (php-fpm) root 292 0.0 0.0 19032 2536 - INs 1Feb15 0:00.07 /usr/local/sbin/check_reload_status root 294 0.0 0.0 19032 2356 - IN 1Feb15 0:00.00 check_reload_status: Monitoring daemon of check_reload_status root 307 0.0 0.1 13164 4408 - Ss 1Feb15 0:00.44 /sbin/devd -q root 10737 0.0 0.0 16812 2304 - Ss 1Feb15 1:41.68 /usr/local/sbin/filterlog -i pflog0 -p /var/run/filterlog.pid root 11765 0.0 0.1 32428 5132 - Is 1Feb15 0:00.01 /usr/sbin/sshd root 11865 0.0 0.0 14756 2184 - Is 1Feb15 0:00.02 /usr/local/sbin/sshlockout_pf 15 root 12389 0.0 0.0 18788 2288 - Is 1Feb15 0:00.03 /usr/sbin/inetd -wW -R 0 -a 127.0.0.1 /var/etc/inetd.conf root 15749 0.0 0.0 12464 2140 - Ss 1Feb15 3:47.10 /usr/local/sbin/apinger -c /var/etc/apinger.conf root 15933 0.0 0.0 28332 2948 - S 1Feb15 0:12.20 /usr/local/bin/rrdtool - root 23101 0.0 0.1 58988 7676 - S 1Feb15 0:34.51 /usr/local/sbin/lighttpd -f /var/etc/lighty-webConfigurator.conf root 24026 0.0 0.1 60552 10144 - Ss Tue10PM 0:23.88 /usr/local/libexec/nut/snmp-ups -a SmartUPS1500 root 27277 0.0 0.0 22948 2564 - Ss Tue10PM 0:06.95 /usr/local/sbin/upsd -u root root 30835 0.0 0.0 17144 2668 - S 1Feb15 0:25.16 /bin/sh /usr/local/pkg/sqpmon.sh root 31786 0.0 0.0 12412 1964 - Is 1Feb15 0:11.41 /usr/local/sbin/dhcpleases6 -c /usr/local/bin/php -f /usr/local/sbin/prefixes.php|/bin/sh -l /var/dhcpd/var/db/dhcpd6.leases root 31881 0.0 0.0 18840 2540 - Is Tue10PM 0:00.00 /usr/local/sbin/upsmon SmartUPS1500@localhost uucp 32022 0.0 0.0 18840 2584 - S Tue10PM 0:05.40 /usr/local/sbin/upsmon SmartUPS1500@localhost root 32374 0.0 0.0 14540 2236 - S 1Feb15 0:15.82 /usr/local/sbin/radvd -p /var/run/radvd.pid -C /var/etc/radvd.conf -m syslog root 32835 0.0 0.1 55632 6188 - Ss 11:37AM 0:00.25 sshd: admin@pts/0 (sshd) root 35324 0.0 0.1 55632 6256 - Is 11:38AM 0:00.47 sshd: admin@pts/1 (sshd) root 37614 0.0 0.0 25480 2968 - Is 1Feb15 0:00.03 /usr/local/libexec/ipsec/starter --daemon charon root 37833 0.0 4.4 618236 363872 - Ss 1Feb15 7:08.52 /usr/local/libexec/ipsec/charon --use-syslog root 38025 0.0 0.0 23224 2664 - Is 1Feb15 0:18.23 /usr/local/sbin/filterdns -p /var/run/filterdns-ipsec.pid -i 60 -c /var/etc/ipsec/filterdns-ipsec.hosts -d 1 root 39019 0.0 0.2 67076 19296 - Ss 1Feb15 2:20.38 /usr/sbin/bsnmpd -c /var/etc/snmpd.conf -p /var/run/snmpd.pid root 45981 0.0 0.0 8312 1964 - IN 11:55AM 0:00.00 sleep 60 root 49751 0.0 0.0 14548 2024 - Ss 1Feb15 1:45.26 /usr/sbin/powerd -b hadp -a hadp -n hadp root 58823 0.0 0.0 0 16 - DL 11:39AM 0:00.01 [ipmi0: kcs] root 59806 0.0 0.1 47888 6800 - Is 1Feb15 0:00.04 /usr/local/sbin/mpd4 -b -d /var/etc/l2tp-vpn -p /var/run/l2tp-vpn.pid -s l2tps l2tps root 64246 0.0 0.0 14756 2320 - Is 4Feb15 0:00.01 /usr/local/sbin/sshlockout_pf 15 root 67245 0.0 0.0 8312 1964 - S 11:56AM 0:00.00 sleep 55 root 69501 0.0 0.2 73644 13912 - Is 1Feb15 0:00.00 /usr/local/sbin/squid -f /usr/pbi/squid-amd64/local/etc/squid/squid.conf proxy 70004 0.0 17.0 1609644 1414836 - S 1Feb15 7:28.85 (squid-1) -f /usr/pbi/squid-amd64/local/etc/squid/squid.conf (squid) root 70462 0.0 0.0 12412 1936 - Is 1Feb15 0:00.00 /usr/local/bin/minicron 240 /var/run/ping_hosts.pid /usr/local/bin/ping_hosts.sh root 70575 0.0 0.0 12412 1948 - I 1Feb15 0:00.41 minicron: helper /usr/local/bin/ping_hosts.sh (minicron) root 70691 0.0 0.0 12412 1936 - Is 1Feb15 0:00.00 /usr/local/bin/minicron 3600 /var/run/expire_accounts.pid /usr/local/sbin/fcgicli -f /etc/rc.expireaccounts root 71301 0.0 0.0 12412 1948 - I 1Feb15 0:00.03 minicron: helper /usr/local/sbin/fcgicli -f /etc/rc.expireaccounts (minicron) root 71569 0.0 0.0 12412 1936 - Is 1Feb15 0:00.00 /usr/local/bin/minicron 86400 /var/run/update_alias_url_data.pid /usr/local/sbin/fcgicli -f /etc/rc.update_alias_url_data root 71960 0.0 0.0 12412 1948 - I 1Feb15 0:00.00 minicron: helper /usr/local/sbin/fcgicli -f /etc/rc.update_alias_url_data (minicron) root 77423 0.0 0.0 14756 2248 - Is 1Feb15 0:00.02 /usr/local/sbin/sshlockout_pf 15 root 77929 0.0 0.0 23492 3096 - Ss Tue10PM 0:01.07 ladvd: master [priv] (ladvd) ladvd 78134 0.0 0.0 23492 3076 - S Tue10PM 0:01.19 ladvd: child (ladvd) root 85967 0.0 0.1 18900 5128 - S 1Feb15 0:22.78 /usr/local/sbin/arpwatch -d -f /var/log/arp.dat -i igb0 root 89552 0.0 0.2 28172 18060 - Ss Mon11PM 4:08.97 /usr/local/sbin/ntpd -g -c /var/etc/ntpd.conf -p /var/run/ntpd.pid root 94712 0.0 0.0 16672 2420 - Ss Tue10PM 0:00.30 /usr/sbin/cron -s root 96816 0.0 0.1 18900 4992 - S Tue10PM 0:03.33 /usr/local/sbin/arpwatch -d -f /var/log/arp.dat -i igb0 root 46612 0.0 0.0 17144 2436 v0- IN 1Feb15 2:52.87 /bin/sh /var/db/rrd/updaterrd.sh root 77253 0.0 0.0 43576 2692 v0 Is 1Feb15 0:00.02 login [pam] (login) root 77735 0.0 0.0 17144 2720 v0 I 1Feb15 0:00.01 -sh (sh) root 78072 0.0 0.0 17144 2604 v0 I+ 1Feb15 0:00.01 /bin/sh /etc/rc.initial root 33245 0.0 0.0 17144 2680 0 Is 11:37AM 0:00.01 /bin/sh /etc/rc.initial root 34614 0.0 0.0 17484 3868 0 S 11:37AM 0:00.03 /bin/tcsh root 67436 0.0 0.0 18816 2412 0 R+ 11:56AM 0:00.00 ps xuawww root 37139 0.0 0.0 17144 2680 1 Is 11:38AM 0:00.01 /bin/sh /etc/rc.initial root 38597 0.0 0.0 17484 3776 1 I+ 11:38AM 0:00.01 /bin/tcsh -
I turned device polling back on, reran the test. and it hung up like previously. the moment I turned it off / hit save. it spit out the remaining info.
So the problem is something related to device polling.This thing is still there in the GUI? :o ??? Consider yourself lucky that the box was still accessible. :D My experience: https://forum.pfsense.org/index.php?topic=65302.0
-
Ah, yes, polling. Yuck.
It's a horrible option for many people but there are still rare occasions when not only does it work, it can perform better. Usually better to keep it off though.
-
I can confirm that turning off device polling fixed it, thermal widget workes as expected now. Explains why loading a config showed the issue, device polling is off be default, config kept turning it back on.
Also a positive, my CPU would hover idle right around 13-15% load, all the time when device polling was on. It is now idle at 0% with device polling off.
Weird though, device polling didn't have an issue running in 2.1.x…
