2 pfSense VMs - Gateways monitoring ~40% Packet Loss
-
Hi.
I've been using pfSense for a while and I've always used only 1 instance for production and tried a few times to use 2 instances as a lab environment.
In the LAB environment one thing always puzzled me: once I get both pfSesne1 and pfSense2 boxes up and HA setup, both boxes show Gateway monitoring with around 40% packet loss.The Hypervisor is a Linux CentOS box, with Link Aggregation + bridge + vlan. All works fine.
I have 2 "fake" uplinks ( 2 Router appliances connected to the same WAN link, providing 2 VLANs to simulate 2 WAN connections ). I've always setup WAN0 to monitor 8.8.8.8 and WAN1 to monitor 8.8.4.4 on both boxes:
pfSense1: WAN0 = 10.10.10.2 / WAN1 = 10.10.11.2
pfSense2: WAN0 = 10.10.10.3 / WAN1 = 10.10.11.3
WAN0 CARP = 10.10.10.1
WAN1 CARP = 10.10.11.1redundancy/HA works fine.
I'll call this "BAD CONFIG" scenario.
After a bit of tricking with some configs, I decided to leave box2 to monitor the router appliances IPs: 10.10.10.254 and 10.10.11.254.
All of a sudden everything worked fine! Gateways packet loss is at 0%, GREAT!
I'll call this "GOOD CONFIG" scenario.So, i tried setting pfSense1 WAN1 monitor ip to 8.8.8.8 ( the same used for WAN0 monitoring ), and pfSense error poped up:
The following input errors were detected: The monitor IP address "8.8.8.8" is already in use. A different monitor IP must be chosen.
It's also worth mentioning that under "GOOD CONFIG", pinging 8.8.8.8 and 8.8.4.4 works fine. When using the "BAD CONFIG", I get the ~40% packet loss results.
So... I'd like to gently ask for directions here:
What happens?
What am I supposed to read to understand this behavior ?In this scenario of 2 boxes with 2 WAN interfaces each, should I use 4 different external IPs for monitoring ?
Thanks!