Why is my pfSense Firewall Lagging and Giving 504 Gateway Timeout Errors?
-
What happened 900s (15m) before that error though? Something triggered a script that stalled.
-
@stephenw10 before 18:12:26, what I see is mostly /rc.newipsecdns: Gateway, none 'available' for inet6, use the first one configured. 'OPT87_VPNV6' and sshguard Exiting on signal and Now monitoring attacks. messages repeating from 18:03:09 to 18:12:26, but, on 18:02:53 and 18:03:06 I have the following errors
-
@stephenw10 here are logs at the time of around 15m mark
-
What was the interface you added there at 17:58:48?
If you manually run a Filter Reload from Status > Filter Reload do you see any errors? Do you get another php crash 900s after the reload?
-
@stephenw10 after 17:58:55, i have many 60676 /rc.filter_configure_sync: dpinger: No dpinger session running for gateway and /vpn_ipsec.php: dpinger: No dpinger session running for gateway messages, starting from 17:58:58 and finishing at the same second. after that at 17:59:00 gateway alarm 100% loses of tunnel VTI, the only different things i've seen that are not usual are these, at time 17:59:22 and 17:59:43.
-
@stephenw10 the interfaces I'm adding are mostly IPsec VTI tunnels that i've created and then giving those interfaces a gateway and static route
-
@stephenw10 i've did a filter reload and no errors in system logs and it was completed successfully.
-
Hmm. The log entry that looks closest to it is for rc.openvpn. Do you have any openvpn incidences defined? Do they have dynamic gateways set?
-
@stephenw10 I have checked my OpenVPN configuration, and it is set to use the WAN interface, which has a static IPv4 address and a static upstream gateway.
-
How many tunnels/gateways do you have?
-
@stephenw10 one default WANGW and 69 for tunnels
-
But are those all VTI tunnels with assigned interfaces that create gateways?
Because if so that is lot of gateways for anything that triggers a script when it bounces.
-
@stephenw10 sorry again for late response, yeah, they all have assigned interfaces. What script is being triggered ?
-
They are seen by pfSense as a WAN interface because they have a gateway. So you get the gateway and WAN IP scripts run for each one.
You might try disabling the gateway monitoring action for those gateways to reduce the churn.
-
@stephenw10 Ah, so that's why disabling dpinger helps. But should pinging 60 interfaces really be this difficult for pfSense?
-
It's not the pinging that causes the issue it's the scripts that get run when it sees the gateway as changing state. You can disable dpinger entirely or you can just disable the 'monitor action' in the gateway settings. Disabling the action keeps the ping data logging but stops it running scripts if/when the gateway goes down. Does that is usually preferable to disabling it entirely.
-
@Mushvan said in Why is my pfSense Firewall Lagging and Giving 504 Gateway Timeout Errors?:
But should pinging 60 interfaces really be this difficult for pfSense?
Wow .. 60.
A small script file that send a ping packet every ... not sure, 250 ms or so, and even 60 of them, that's no big deal.
But when one, or more of them get triggered because the interface and/or isn't there anymore ... dpinger will take action : it will reset (like pull down == destroy and pull up == recreate) the connection.
And now for the fun part : this will have a cascade effect on other processes, like nginx and unbound, just to name two of them, that will also get restarted. I've this 'feeling' that the 'mess' this creates goes up exponentially.
Your router is lagging, spikes to 100 % core usage etc ? I'm not very surprised.I don't have the hands on experience, as I'm just a "2 WAN and 4 LAN ports guy", but If I had to 60 interconnections , 60 interfaces to manage, I wouldn't take that "Swiss Army Knife" firewall router called pfSense, but something more bare bone like TNSR ?
