WAN Acting Up With Multiple PF VMs



  • Hey guys, we currently have 3 pfSense VMs on 2 different esxi 5.5 machines. PF1 is our main installation (2.2.3, 3 WAN Connections + 14 VLANs). PF2 and PF3 are PF 2.3 VMs that we want to replace our main one with (HA + CARP, 3 WANS + 14 VLANS). PF2 and PF3 have all the same settings as PF1 except DHCP is turned off and the WAN IPs are completely different. We're having an issue that whenever PF2 and PF3 are connected to WAN2 for an extended period of time, PF1 WAN2 is flooded with DHCP requests from VLAN1. My question is why is DHCP traffic appearing on the WAN side at all? All the WAN interfaces are statically assigned completely different IPs (including CARP VIPs).

    Current Setup:

    esxi host 1:
    pfSense 1 (2.2.3)
          WAN1: x.x.x.202
          WAN2: x.x.x.98
          WAN3: x.x.x.108
          VLAN1: 10.0.0.1
    pfSense 3 (2.3 HA+CARP, Backup)
          WAN1: x.x.x.203 (VIP: x.x.x.204)
          WAN2: x.x.x.99 (VIP: x.x.x.100)
          WAN3: x.x.x.109 (VIP: x.x.x.110)
          VLAN1: 10.0.0.5 (VIP: 10.0.0.6)

    esxi host 2:
    pfSense 2 (2.3 HA+CARP, Master)
          WAN1: x.x.x.205 (VIP: x.x.x.204)
          WAN2: x.x.x.101 (VIP: x.x.x.100)
          WAN3: x.x.x.111 (VIP: x.x.x.110)
          VLAN1: 10.0.0.2 (VIP: 10.0.0.6)

    VLAN1 DHCP and DNS is handled by Windows 2012 Domain Controller

    HA+CARP Sync Interface is an ethernet cable going from esxi host 1 to host 2 (no switch in between, only VM on said interface)
    Promiscuous mode, MAC Changes, and Forged Transmits enabled on all interfaces and vswitches
    Block Bogon and Local Traffic is enabled on all WAN interfaces

    Packet Capture of WAN2 on PF1 shows 10.0.1.94(Random Android Client) sending DHCP requests to 10.0.1.3 (Domain/DHCP server). When this happens packet loss jumps to ~70%
    We currently have PF2 and PF3 shutdown and WAN2 has not been acting up since.

    Thanks in advanced for any help  :)

    Edit: Forgot to mention there's only one client attached to PF2 and PF3 as a gateway (me, testing purposes) so there's not much, if any, traffic going through it.



  • Do you have conflicting VHIDs between them? Those must be unique across pairs, if one pair has VHID 1, a different pair cannot use VHID 1. Otherwise you have a MAC address conflict. That shouldn't result in DHCP requests showing up on the wrong network, but I guess it's possible in theory that a duplicate MAC could screw up something on the network in such a way that unexpected things happen.



  • The first time I set it up I accidentally set up two IPs on VHID 1 but I realized my mistake and changed it but it's still acting up even after fixing it  :(



  • FreeBSD 10 is NOT certified/supported on ESXi 5.1

    Refer to:

    https://forum.pfsense.org/index.php?topic=113220.0


Log in to reply