Backup pfSense can't ping it's own interface gateway but Master can
I'm still struggling with my CARP cluster… Backup pfSense can't ping the default gateway on an interface but the Master can.
Each pfSense box has 5 nics; 2 WAN connections via adsl modem/routers, 1 WLAN via wifi router, LAN and pfSync with crossover:
192.168.0.x - LAN
192.168.2.x - WAN1 - 192.168.2.1 WAN1 router
192.168.1.x - WAN2 - 192.168.1.254 WAN2 router
192.168.3.x - WLAN - 192.168.3.1 WLAN router
192.168.5.x - pfSync
Master works fine for everything, syncs with Backup, Backup vips show as backup correctly. If master goes down, Backup takes over, backup vips all become master until master comes back online which then takes over.
The problem I'm getting is if I try and ping the WLAN router from the Backup it fails, but on the master it works. Apinger leaves a log alarm saying 192.168.3.1 is down.
If I do a traceroute to 192.168.3.1 from the Backup it hits 192.168.0.1 first.
If I do a traceroute to 192.168.3.1 from the Master it hits 192.168.3.1 first.
netstat -rn on backup does not have an entry for 192.168.3.0/24, but the master does.
If I force and entry (from the static routes page) it still doesn't ping, but I get 192.168.3.0/24 - 192.168.3.1 in the route table.
If I then remove the static route (from gui) it removes the route from the table but then traceroute and ping starts working!
If I try and add the gateway back in the Gateways tab (so apinger can monitor it), it fails and marks it down.
The Master works fine, pfSync works and rules, aliases, nat, load balancer, virtual ips are all set to sync on the master.
I am totally confused as to what is going on!
Can some kind soul please help?
Thanks in advance,
It sounds like that LAN in misconfigured on the backup machine. If you have an interface in a particular network, a route is create automatically.
Thanks for that clue podilarius, after looking at the state table I noticed ICMP packets from pfsense to the 192.168.3.1 WLAN router were going via the LAN 192.168.0.1 vip.
This led me to the Manual Outbound NAT rules and I had a rule there saying:
"WLAN 192.168.3.0/24 * * * 192.168.0.1 * NO" (WLAN to vip1 LAN)
Removing this fixed it!
I had to remove stale states from the state table manually too for changes to take effect immediately as the gateway status still showed the 192.168.3.1 router as being down (through apinger).
I also didn't have a default gateway set on the backup so setting that fixed the routing tables.
And on a slightly different note, my 192.168.2.1 adsl modem/router didn't pass multicast over it's switch (they're just getting too clever and locked down these days!) so I had to put in another unmanaged switch inbetween to allow vip2 interfaces to switch from master->backup properly, (as stated in the sticky, but I had to read that more than a few times before it sank in!)
Looks like it's ok now, thanks for the help!