WAN Gateway Issues with CARP IP enabled.



  • Good morning

    I'm stuck with a problem I've been trying to work through off and on for for about two weeks now.
    I'm hoping you can help shed some light on this strange and frustrating issue I'm seeing in my HomeLab.

    So for my lab. I have two virtual instances of pfSense running on VMware 6.0.
    There are three vNICSs on each instance. WAN, LAN, SYNC, each on there own vlan
    There is one vSwitch, trunking across 4x physical interfaces to a Cisco 3750.
    The cable modem is connected to a port on the 3750G which is assigned to the same vlan as the wan interfaces in pfSense.
    I have business class service providing a /29 from WoW.

    Each FW works as expected when using just the WAN IP address with no CARP IP setup.
    When I add the WAN CARP IP address. I get gateway loss from the WAN interface of which ever firewall is the master.
    The slave does not see this issue, until it becomes the master and is holding the CARP IP.

    The CARP IP addresses and NAT appear to be working fine, I'm not seeing any issues there.
    I'm looked at ARP and I do see that the mac addresses are unique. I checked, because I was hoping if the mac were not unique that would explain that I was seeing.

    I checked the cable modem and cleared the ARP cache there, and I'm still seeing the same thing.
    I've ran packet captures, nothing that stood out as unusual. Ping tested from the WAN interface to the gateway and verified it wasn't just a visual monitor issue on pfSense.

    I'm unsure where to go from here.

    I've attached some screenshots. Any help or direction would be greatly appreciated. Let me know what else I can provide

    Thanks and regards,
    -Anthony-

    ![Screen Shot 2017-10-13 at 5.04.06 PM.png](/public/imported_attachments/1/Screen Shot 2017-10-13 at 5.04.06 PM.png)
    ![Screen Shot 2017-10-13 at 5.04.06 PM.png_thumb](/public/imported_attachments/1/Screen Shot 2017-10-13 at 5.04.06 PM.png_thumb)
    ![Screen Shot 2017-10-13 at 5.05.12 PM.png](/public/imported_attachments/1/Screen Shot 2017-10-13 at 5.05.12 PM.png)
    ![Screen Shot 2017-10-13 at 5.05.12 PM.png_thumb](/public/imported_attachments/1/Screen Shot 2017-10-13 at 5.05.12 PM.png_thumb)
    ![Screen Shot 2017-10-13 at 5.05.38 PM.png](/public/imported_attachments/1/Screen Shot 2017-10-13 at 5.05.38 PM.png)
    ![Screen Shot 2017-10-13 at 5.05.38 PM.png_thumb](/public/imported_attachments/1/Screen Shot 2017-10-13 at 5.05.38 PM.png_thumb)



  • There was a response on here a few minutes ago and I'm unsure what happened, but the post is no longer here.
    I see it was a first post.
    To the dude who posted, the solution worked for me and everything is showing normal now.

    The solution was to add a NAT rule from the WAN interface for this firewall to the CARP VIP IP.
    If I could buy you a beer I would, this resolved my issue.



  • Hi AnthonyW,

    I'm sorry, i reply with this solutions but deleted the post because i thought it was in the wrong topic :) But now I see you and another topic needs the same solution  8)



  • Thanks for the help. I was really stuck.
    If you private message me your paypal. I'll send you a beer.



  • can you show what exactly rule you created?

    as from you text I understand to

    • make a NAT rule
    • on WAN interface
    • with destination = WAN CARP IP
    • with destination port range = any
    • with redirect target IP = ??? (LAN CARP IP??)

    it claims I would not have specify a destination port from
    so I used "custom" with range from 1 to 65535
    at redirect target port I only set 1

    edit: I think the problem may is that on WAN MAC Addresses must be enabled by WAN Administrator.
    And since CARP is calculating a random MAC for CARP this MAC probably is not granted on WAN.

    Is there a way to hardly set this CARP MAC? Else I never will we able to use CARP on WAN, or what idea do you have else?

    it did not work.
    Still no accessibility to WAN CARP IP from WAN.

    I tried changing redirect target IP to pfSense original IP. Does not work.

    I tried a firewall rule (!= NAT)
    with any to any on WAN, did not work also.



  • as I now found out I was wrong that CARP MAC is randonly calculated every reboot.
    The CARD MAC always is "00:00:5E:00:01:<vhid>"

    We are going to check with global admin whether we can get a static VHID on WAN and therefore register the resulting CARP MAC.</vhid>