Troubleshooting VPN Connectivity to a High Availability Secondary Node
-
@stephenw10 thanks again for helping me with https://forum.netgate.com/topic/137947/nic-broadcom-57412-10gb-not-recognized, but it seems I have more issues with my HA firewall setup.
Was wondering if you can please help?
-
I am unable to ping the secondary FW from other other side of my VPN tunnel. I even configured NAT as described in https://docs.netgate.com/pfsense/en/latest/highavailability/troubleshooting-vpn-connectivity-to-a-high-availability-secondary-node.html. I am only able to ping the primary unless I VPNclient connect to the secondary, then I can ping the VIP and the secondary IP. any other suggestions?
-
When I fail over CARP VIPs to secondary FW, I am able to ping across data centers(all internal networks behind NAT) (VPN IPsec tunnel), and ssh, but my all hosts behind my zabbix proxy talking to the zabbix main server in the other data center (across the VPN IPsec tunnel) report to the main zabbix server as unreachable. It's very odd because I can ping and ssh to those hosts that the zabbix proxy is reporting back to the main zabbix server as unreachable. The hosts that report unreachable are able to reach the proxy and the main zabbix server. They can also ping the FW CARP VIP and the secondary FW when it is set at the master. Is this a known issue? All traffic is allowed on my LAN interface and IPsec interface. Do I have to flush routes/restart networking on the zabbix proxy to start talking to the new primary FW that owns the LAN VIP? I'm out of ideas...
Attached is my network design to help with what I'm trying to explain.
image url)Thanks!
-Rich
-
-
@rivest1000 said in Troubleshooting VPN Connectivity to a High Availability Secondary Node:
@stephenw10 thanks again for helping me with https://forum.netgate.com/topic/137947/nic-broadcom-57412-10gb-not-recognized, but it seems I have more issues with my HA firewall setup.
Was wondering if you can please help?
- I am unable to ping the secondary FW from other other side of my VPN tunnel. I even configured NAT as described in https://docs.netgate.com/pfsense/en/latest/highavailability/troubleshooting-vpn-connectivity-to-a-high-availability-secondary-node.html. I am only able to ping the primary unless I VPNclient connect to the secondary, then I can ping the VIP and the secondary IP. any other suggestions?
This technique works. You'll need to let us know what address on the secondary and post the VPN firewall rules from the primary, the outbound NAT on the appropriate interface, and the firewall rules on the secondary for the interface you are connecting to so we can see what the mistake is.
- When I fail over CARP VIPs to secondary FW, I am able to ping across data centers(all internal networks behind NAT) (VPN IPsec tunnel), and ssh, but my all hosts behind my zabbix proxy talking to the zabbix main server in the other data center (across the VPN IPsec tunnel) report to the main zabbix server as unreachable. It's very odd because I can ping and ssh to those hosts that the zabbix proxy is reporting back to the main zabbix server as unreachable. The hosts that report unreachable are able to reach the proxy and the main zabbix server. They can also ping the FW CARP VIP and the secondary FW when it is set at the master. Is this a known issue? All traffic is allowed on my LAN interface and IPsec interface. Do I have to flush routes/restart networking on the zabbix proxy to start talking to the new primary FW that owns the LAN VIP? I'm out of ideas...
Sounds like they are set to talk to interface addresses and not CARP VIPs.
-
@Derelict thanks.
Attached is an updated image with VIP and primary/secondary LAN/INTERNAL IPs and I also added the IPsec /30 network to the digram as that is configured in my outbound NAT settings :
Also Here is the configurations for the Primary FW :
Also Here is the configurations for the Secondary FW :
Thanks!
-Rich
-
The source address at that point will not be 192.168.199.24/30 it will be a source address from the other side of the VPN. Probably 10.2.x.x.
-
@Derelict ok I was going off of this link where it says source being the VPN subnet ...https://docs.netgate.com/pfsense/en/latest/highavailability/troubleshooting-vpn-connectivity-to-a-high-availability-secondary-node.html
-
@Derelict Yes you are right, it is working now. I'm still trying to understand my #2 question. Every server in Data center A is pointed at 10.8.0.1, but when I fail over to the .3 (secondary), my zabbix server reports all hosts on the remote(proxy server) side as unreachable, even though I can ssh and ping everything across the VPN tunnel.
Interesting. thanks!
-Rich
-
@rivest1000 said in Troubleshooting VPN Connectivity to a High Availability Secondary Node:
@Derelict ok I was going off of this link where it says source being the VPN subnet ...https://docs.netgate.com/pfsense/en/latest/highavailability/troubleshooting-vpn-connectivity-to-a-high-availability-secondary-node.html
That link is not wrong. In most cases the addresses you initially used would be outside public addresses, which would not be confused with the VPN subnet.
-
@rivest1000 said in Troubleshooting VPN Connectivity to a High Availability Secondary Node:
@Derelict Yes you are right, it is working now. I'm still trying to understand my #2 question. Every server in Data center A is pointed at 10.8.0.1, but when I fail over to the .3 (secondary), my zabbix server reports all hosts on the remote(proxy server) side as unreachable, even though I can ssh and ping everything across the VPN tunnel.
Interesting. thanks!
-Rich
Then Zabbix itself must not be using the CARP VIP. Hard to say based on what has been provided.
-
@derelict I’m troubleshooting this issue again and digging into tcpdumps between 10.2.x.x and 10.8.x.x. Traffic seems to drop tcp FIN and SYN flags in both directions while failed over the secondary PFsense. I just see ACK flags going back and forth while on the secondary PFsense firewall (carp in maint mode on primary). I tried enabling a few settings(bypass firewall, conservative, etc...), but as soon as the primary takes back the CARP VIPs then all traffic goes back to normal. Any idea why only the secondary node is not passing all traffic(ssh and other ports work just fine) across the IPSec tunnel? I have check and sync is working with all boxes checked.
Thanks!
-Rich
-
@rivest1000 Need a better description of exactly what you are saying. You should also probably start a new thread. It doesn't sound like you are talking about the well-known need to NAT from the primary when talking to the secondary over a VPN.
