XG-7100 > Netgate 8200 sonewconn Errors
-
Hi,
after migrating my XG-7100 configuration to a brand new Netgate 8200 running pfSense 24.11, I'm seeing the following errors in the system logs:
Mar 26 13:03:34 kernel sonewconn: pcb 0xfffff8001559fb00 (local:/var/etc/openvpn/server51/sock): Listen queue overflow: 2 already in queue awaiting acceptance (2 occurrences), euid 0, rgid 0, jail 0 Mar 26 13:03:19 kernel sonewconn: pcb 0xfffff80146108200 (local:/var/etc/openvpn/server32/sock): Listen queue overflow: 2 already in queue awaiting acceptance (1 occurrences), euid 0, rgid 0, jail 0 Mar 26 13:00:11 kernel sonewconn: pcb 0xfffff80146612b00 (local:/var/etc/openvpn/server44/sock): Listen queue overflow: 2 already in queue awaiting acceptance (1 occurrences), euid 0, rgid 0, jail 0 Mar 26 12:59:32 kernel sonewconn: pcb 0xfffff8001559fb00 (local:/var/etc/openvpn/server51/sock): Listen queue overflow: 2 already in queue awaiting acceptance (4 occurrences), euid 0, rgid 0, jail 0 Mar 26 12:37:12 kernel sonewconn: pcb 0xfffff8006402d200 (local:/var/etc/openvpn/server34/sock): Listen queue overflow: 2 already in queue awaiting acceptance (1 occurrences), euid 0, rgid 0, jail 0 Mar 26 12:37:06 kernel sonewconn: pcb 0xfffff80064196b00 (local:/var/etc/openvpn/server20/sock): Listen queue overflow: 2 already in queue awaiting acceptance (3 occurrences), euid 0, rgid 0, jail 0 Mar 26 12:36:44 kernel sonewconn: pcb 0xfffff8001559fb00 (local:/var/etc/openvpn/server51/sock): Listen queue overflow: 2 already in queue awaiting acceptance (2 occurrences), euid 0, rgid 0, jail 0Sometimes I can manually trigger these entries by opening the pfSense dashboard with the OpenVPN widget enabled or by navigating to Status > OpenVPN.
Is this a known issue with version 24.11?Thanks!
-Rico
-
I'm also seeing high CPU usage from the php-fpm process, even though the web interface is closed:
[24.11-RELEASE][admin@pfsense.office.lan]/root: top -aSH | grep php-fpm 72050 root 61 0 162M 73M accept 3 1:27 8.98% php-fpm: pool nginx (php-fpm) 25449 root 63 0 162M 72M accept 2 1:06 8.06% php-fpm: pool nginx (php-fpm) 68799 root 68 0 133M 68M accept 2 1:15 7.76% php-fpm: pool nginx (php-fpm) 1039 root 68 0 133M 69M accept 7 1:13 1.66% php-fpm: pool nginx (php-fpm)Maybe the issues are related?
Only 4 Packages installed:
Netgate_Firmware_Upgrade
openvpn-client-Export
System_Patches
WireGuard-Rico
-
For further testing, I imported the config from my 8200 to a 24.11 6100 test machine.
The same thing happens there: I constantly see between 4 and 7php-fpm: pool nginx (php-fpm)processes running, even when no one is logged into the web interface. However, they show no load at all. On the live 8200 with traffic, the processes are constantly under load. Can this be caused by OpenVPN?Additionally, I can trigger the
sonewconnmessage simply by opening the dashboard.
I installed all the recommended 24.11 patches on the test 6100—no success.
Then I tested by upgrading to 25.03 BETA—still no success.At my remote sites, I have several 6100s running 24.11, and I don't see any running
php-fpmprocesses unless someone is logged into the web interface.
So it must be related to my XG-7100 config?Out of ideas here...
-Rico
-
@Rico FWIW there's a fix for the dashboard widgets and a patch in the redmine.
https://docs.netgate.com/pfsense/en/latest/releases/25-03.html#dashboardBut that should be in the beta I expect.
If you run a "ps aux |grep php" what does it show is running PHP? (note it doesn't word wrap at least on my putty)
-
WebIF closed:
[24.11-RELEASE][admin@pfsense.office.lan]/root: ps aux | grep php root 94392 7.0 0.4 135828 69968 - S 06:46 9:14.10 php-fpm: pool nginx (php-fpm) root 76458 3.3 0.4 135828 69988 - R 08:28 3:09.27 php-fpm: pool nginx (php-fpm) root 25 0.0 0.0 12944 2464 - Is Wed10 0:00.00 /usr/local/bin/minicron 300 /var/run/ipsec_keepalive.pid /usr/local/bin/ipsec_keepalive.php root 612 0.0 0.0 12944 2484 - I Wed10 0:00.04 minicron: helper /usr/local/bin/ipsec_keepalive.php (minicron) root 1381 0.0 0.4 135828 69972 - I 21:39 36:08.94 php-fpm: pool nginx (php-fpm) root 34044 0.0 0.4 135828 67920 - S 06:51 8:50.09 php-fpm: pool nginx (php-fpm) root 39634 0.0 0.4 135828 70140 - I 21:38 36:08.26 php-fpm: pool nginx (php-fpm) root 46377 0.0 0.4 135828 70008 - I 23:14 31:16.23 php-fpm: pool nginx (php-fpm) root 50081 0.0 0.0 12972 2576 - S 09:22 0:00.00 /usr/local/sbin/fcgicli -f /etc/inc/openvpn.tls-verify.php -d servercn=xxx root 72010 0.0 0.2 110164 38312 - Ss Wed15 0:03.12 php-fpm: master process (/usr/local/lib/php-fpm.conf) (php-fpm) root 90782 0.0 0.4 135828 67908 - S 08:33 2:54.43 php-fpm: pool nginx (php-fpm) root 53879 0.0 0.0 13040 2656 0 S+ 09:22 0:00.00 grep phpWebIF open:
[24.11-RELEASE][admin@pfsense.office.lan]/root: ps aux | grep php root 1381 16.4 0.4 131816 68176 - R 21:39 36:15.77 php-fpm: pool nginx (php-fpm) root 94392 15.2 0.4 161880 71576 - S 06:46 9:19.52 php-fpm: pool nginx (php-fpm) root 46377 11.3 0.4 135912 72312 - S 23:14 31:22.64 php-fpm: pool nginx (php-fpm) root 39634 9.4 0.5 165976 76820 - S 21:38 36:14.16 php-fpm: pool nginx (php-fpm) root 76458 8.6 0.4 135912 72400 - S 08:28 3:15.91 php-fpm: pool nginx (php-fpm) root 34044 7.5 0.4 135912 72304 - R 06:51 8:55.16 php-fpm: pool nginx (php-fpm) root 90782 6.4 0.4 135828 72096 - S 08:33 2:58.88 php-fpm: pool nginx (php-fpm) root 25 0.0 0.0 12944 2464 - Is Wed10 0:00.00 /usr/local/bin/minicron 300 /var/run/ipsec_keepalive.pid /usr/local/bin/ipsec_keepalive.php root 612 0.0 0.0 12944 2484 - I Wed10 0:00.04 minicron: helper /usr/local/bin/ipsec_keepalive.php (minicron) root 72010 0.0 0.2 110164 38312 - Ss Wed15 0:03.12 php-fpm: master process (/usr/local/lib/php-fpm.conf) (php-fpm) root 76536 0.0 0.0 12972 2572 - S 09:23 0:00.00 /usr/local/sbin/fcgicli -f /etc/inc/openvpn.tls-verify.php -d servercn=xxx root 82125 0.0 0.0 13040 2668 0 S+ 09:23 0:00.00 grep php-Rico
-
@Rico FWIW I pulled up a 24.11 router via SSH and it has that many PHP-FPM processes too. Not sure why a bunch instead of 1-2 but typically they'd be idle unless processing PHP code. I don't recall the count on previous versions.
-
You also see a high load with the WebIF closed?
90886 root 56 0 162M 74M CPU7 7 7:15 12.60% php-fpm: pool nginx (php-fpm) 95613 root 68 0 133M 71M accept 2 7:15 10.60% php-fpm: pool nginx (php-fpm) 28968 root 68 0 162M 74M accept 7 8:10 10.50% php-fpm: pool nginx (php-fpm) 75399 root 68 0 162M 75M accept 5 7:52 7.76% php-fpm: pool nginx (php-fpm) 41216 root 68 0 160M 73M accept 0 7:29 4.49% php-fpm: pool nginx (php-fpm)Also sonewconn messages in the system logs?
-Rico
-
@Rico No, and no log entries though we don't have OpenVPN set up.
PID USERNAME THR PRI NICE SIZE RES STATE C TIME WCPU COMMAND 77412 root 10 20 0 798M 677M nanslp 0 937:11 1.18% suricata 86659 root 10 20 0 850M 721M nanslp 0 547:29 0.65% suricata 99674 unbound 4 20 0 113M 87M kqread 2 7:30 0.23% unbound 19999 root 1 20 0 14M 4100K CPU0 0 0:00 0.08% top 69725 root 5 68 0 17M 3168K uwait 0 3:54 0.02% dpinger 69646 root 5 68 0 21M 3260K uwait 0 4:25 0.01% dpinger 71901 root 1 20 0 13M 2768K kqread 0 2:12 0.01% tail_pfb
