pfSense Plus and SG-3100

mcury

@nokkief said in pfSense Plus and SG-3100:

I sadly was wrong, just did not pay well enough attention (and probably was hoping it was fixed). I still see the errors every 15 minutes.

Strange, I'm not getting those here.. More than a week running pfblocker, in several vlans..

15 minutes? Do you have a cron set to run every 15 minutes?

bmeeks

@biggy823 said in pfSense Plus and SG-3100:

@nocling Is Snort fixed in this version?

No. Snort and Suricata both are still victims of the PHP bug on ARM 32-bit hardware such as the SG-3100.

NokkieF

@mcury Not that I am aware of. Where can I see the cronjobs listed for the pfSense? crontab -l shows no cronjobs for root
cat of /etc/crontab shows
*/1 * * * * root /usr/sbin/newsyslog

mcury

@nokkief said in pfSense Plus and SG-3100:

@mcury Not that I am aware of. Where can I see the cronjobs listed for the pfSense? crontab -l shows no cronjobs for root
cat of /etc/crontab shows
*/1 * * * * root /usr/sbin/newsyslog

There is a package in package manager, cron
Install cron and check what is running every 15 minutes.. It will show all cron tasks
Did you set pfblockerng to update the lists every 15 minutes?

NokkieF

@mcury No, once a day. I will instal cron and see what it does

NokkieF

Looks like it's doing this
0,15,30,45 * * * * root /etc/rc.filter_configure_sync

mcury

@nokkief said in pfSense Plus and SG-3100:

Looks like it's doing this
0,15,30,45 * * * * root /etc/rc.filter_configure_sync

hm.. are you on 21.02 p2, right?
Because the problem with filter reload was happening only in 21.02 p1

NokkieF

@mcury Yeah, I am
21.02.2-RELEASE (arm)
built on Mon Apr 12 07:50:07 EDT 2021
FreeBSD 12.2-STABLE

The system is on the latest version.
Version information updated at Mon Apr 26 9:02:50 CEST 2021

mcury

@nokkief said in pfSense Plus and SG-3100:

@mcury Yeah, I am
21.02.2-RELEASE (arm)
built on Mon Apr 12 07:50:07 EDT 2021
FreeBSD 12.2-STABLE

The system is on the latest version.
Version information updated at Mon Apr 26 9:02:50 CEST 2021

hm, it is strange..
now I'm not sure if there is something particular about your setup that is triggering it, or if I'm not triggering it by luck and this bug is waiting to happen here, hidden..

Edit: Did you enable these options in DNS Resolver?

NokkieF

@mcury No, but I did enable the python module. But with or without is not making a difference.

s0m3f00l

@bmeeks said in pfSense Plus and SG-3100:

PHP bug on ARM 32-bit

Has anyone tried rebuilding php from source for the 3100? I am still stuck on 2.4.5p1 because we use suricata. If I wanted just a router firewall without any NGFW stuff I could have build an openwrt box.

artooro

@stephenw10 we've had issues with several SG-3100s upgrading them to pfSense 21.02.2, just working on one right now. I just tried clearing all the files in /var/db/aliastables to ensure that it wasn't a large table causing the issue but after a reboot I am still seeing kernel: pid 404 (php-cgi), jid 0, uid 0: exited on signal 11 (core dumped)

The two symptoms I've seen after multiple SG-3100 upgrades is that

The default gateway is missing
When connecting via console the menu interface doesn't load and it jumps straight to a shell.

Am I correct in there being no workaround at this time for these SG-3100 issues?

artooro

I just found that commenting out the following lines from /etc/rc.bootup resolved the php-cgi crash issue and on reboot both the default route and console are working correctly.
I figured it must be OpenVPN because of when the error shows up in system.log.

//echo "Syncing OpenVPN settings...";
//openvpn_resync_all();
//echo "done.\n";

Still need to determine what about OpenVPN would be triggering this.

NokkieF

@artooro Thanks, I will give it a try and see if it 'fixes' the error for me too.

artooro

Doesn't look like it helped anything. After another reboot php-cgi started to core dump when simply syncing the firewall. Now getting Configuring firewall.Segmentation fault (core dumped)

So far we've had to re-install the OS on quite a few SG-3100 appliances and currently have a policy of keeping them on 2.4.5-p1 until there is a more stable pfSense release.

NokkieF

@artooro Sadly I had the same result, the firewall was worse off than before. I decided to just roll back the whole thing to factory settings. I am now running pfBlocker with a bunch of lists without any issues. I no longer see the php errors.
I will add haproxy and the openvpn later to see if it screws it up again.

Luketa

@artooro @NokkieF ,
I am researching a lot before updating 2.4.5-p1, and I came to the conclusion that I am not going to update, my client is a 24 hour clinic and with this many problems I can get complicated.

NokkieF

So after I went back to the factory settings and set it up again I had no problems with pfBlocker installed. Recently I set up an OpenVPN client and I am seeing the php cor edumps again. So maybe it is related to openvpn?

bmeeks

The problem is within PHP itself on 32-bit ARM platforms. It will manifest itself in different packages at somewhat randomly differing places. Almost every installed package in pfSense makes use of PHP GUI code at some point. Depending on exactly what parts and pieces of PHP are in operation at any given moment, the bug may surface- or it may not.

In the case of the Snort and Suricata packages, the bug is triggered by successive calls to the preg_match() regex function of PHP.

So trying to find a particular package to blame is pointless, unless you pin it where the actual problem lives -- in PHP itself. Adding or removing particular packages, or reinstalling them, or reconfiguring them may appear to give temporary relief, but it's not a real fix. The only fix is within PHP itself.

If you remove all packages and run a plain-vanilla pfSense setup on your SG-3100, it will probably run fine. That's because the base pfSense code, for the most part, appears to not be using the PHP functions that cause the problem on 32-bit ARM platforms. But a number of the add-on packages will call those troublesome PHP functions and thus trigger the bug.

artooro

Looks like pfSense Plus 21.05 just came out, and this issue is still not fixed. And no word from anyone at Netgate either in this forum or the redmine issue.