CARP / HA Logging for inactive WAN
-
Hello,
We have several HA PFsenses setup but a reoccurring issue on the PASSIVE node in relation to logging. It appears log files are building up and consuming all the disk space because of a reoccurring error:
kernel: arpresolve: can't allocate llinfo for EXTERNAL IP DEFAULT GATEWAY on vmx0
To reserve public IPs for the WAN, we use an internal IP for CARP VIPs and then Public address moves between nodes.
Now the passive node shows CARP status of BACKUP and because it's backup, it cannot access the internet or contact the default gateway until it becomes the MASTER node.
Is this expected and how would we prevent these messages from filling up the logs?
-
@Matt_Sharpe said in CARP / HA Logging for inactive WAN:
passive node shows CARP status of BACKUP and because it's backup, it cannot access the internet or contact the default gateway until it becomes the MASTER
Can you detail your setup? For instance we have a client with Comcast who has an HA setup. The pfSense WAN on both is in the Comcast private range 10.1.10.x, gateway 10.1.10.1, while the shared WAN IP is the public IP. Both routers have access to the Internet. Default route/gateway is 10.1.10.1 which works in this case.
If that can't work I suggest getting a larger public block so both can have a public IP. If router2 can access the Internet then it becomes easy to upgrade...upgrade router2, flip router1 to maintenance, upgrade router1, flip back.
-
@SteveITS It's essentially the same setup, but our DFG on the WAN interface is the WAN address required for routing. Meaning it will only work from the MASTER node.
If the DFG was a private address, how would that route to the internet. I imagine the 10.1.10.1 in your example exists also on the router for internet access ?
-
M Matt_Sharpe referenced this topic on
-
@Matt_Sharpe said in CARP / HA Logging for inactive WAN:
the 10.1.10.1 in your example exists also on the router
Comcast business routers provide NAT even when in bridge mode. I suspect it's so a customer can easily plug in a laptop to bypass the customer router, to test. But it also lets "odd" configurations like this work.
Without something like that a /29 is required for three usable public IPs.
https://docs.netgate.com/pfsense/en/latest/highavailability/#ip-address-requirements-for-carp -
Does the secondary node still show that public IP as the default gateway? Does it have any other gateways set? I assume the vmx0 NIC itself is still UP?
-
@stephenw10 yes the secondary node has the public IP set as DG so in case of failover its ready to go :) No other gateways. VMX0 is 'up' and in PASSIVE mode for CARP.
-
The first thing I'd do then is disable gateway monitoring since that cannot work on the backup node.
You might try setting the gateway as 'non-local' in the advanced gateway settings since it's outside any local subnet. That's why it's throwing that llinfo error.
-
You could also try turning down the ARP logging level:
[2.7.2-RELEASE][admin@t70.stevew.lan]/root: sysctl -d net.link.ether.arp.log_level net.link.ether.arp.log_level: Minimum log(9) level for recording rate limited arp log messages. The higher will be log more (emerg=0, info=6 (default), debug=7). -
@stephenw10 I think these options are exactly what I've been looking for!
I've set only the 'non-local' option for now as the DFG is technically non-local to the IP we use for WAN. This appears to have stopped the log entries for arp. I'll keep an eye over the weekend.
Thanks!!!
-
@stephenw10 This setting appears to have stopped the log filling up.
Another query would be, is it possible to move the /var/log partition to a dedicated disk in PFsenses?
-
Not from the gui or in any way that officially supported.
You can specify the log storage location in the syslog-ng package so use that to store it. You still need to forward logs to it from the normal syslogs though. And mounting a different disk for that requires some custom script.
