Help for CARP configuration with a single FO IP


  • Hello,

    I would like to know if it is possible to configure high availability with a single failover IP?

    Here is what I currently have as configuration on my PfSense:
    WAN => Public_IP / 32
    LAN => 192.168.0.254
    OPT10 => 192.168.10.254 (VLAN on the LAN card)

    And what I would like:
    PfSense_MASTER
    WAN => 192.168.100.1
    VIP_WAN => Public_IP / 32
    LAN => 192.168.0.254
    OPT10 => 192.168.10.250
    VIP_OPT => 192.168.10.254
    SYNC => 192.168.150.1

    PfSense_BACKUP
    WAN => 192.168.100.2
    VIP_WAN => Public_IP / 32
    LAN => 192.168.0.254
    OPT10 => 192.168.10.251
    VIP_OPT => 192.168.10.254
    SYNC => 192.168.150.2

    I did a test, but it doesn't work for now. Synchronization does not take place.
    From the VM behind OPT10, I ping 192.168.10.254 and Public_IP. But not beyond (8.8.8.8 for example).


  • I solved the problem of synchronization between the 2 PfSense. the synchronization problem between the 2 PfSense.
    On the 2nd, I added the 3rd network card before creating the OPT10.
    As the interfaces must be aligned, the sync did not work.

    However, I still can't access the internet. Neither from the parent company, nor from the VM.
    I also do not access the vm (web server) from the outside.


  • Further info, when I ping 8.8.8.8, the response is:
    ping.PNG

    The client's gateway is the VIP address (192.168.10.254).
    the answer (192.168.10.250), is the "real" address of the interface (OPT10).

    Here are the rules of the OPT10 interface, WAN and the NAT rules
    opt10Rule.PNG

    wanRule.PNG

    natRule.PNG


  • What is the CARP status page showing?
    Status > CARP


  • @viragomann

    Hello,

    Here is a capture of the CARP status
    Primary:
    carp state.PNG
    Backup:
    carp state backup.PNG

    As I said, from a client behind the firewall, when I ping 8.8.8.8, the response comes from 192.168.10.250.
    If I turn off the primary firewall, the response comes from Backup (192.168.10.251).
    Does this sound like a routing problem?
    So far everything is ok.

    On this tutorial (in French), he talks about creating a default gateway.
    Is that related?

    Cordially.


  • I think, you're are missing the outbound NAT rule. You must set the Outbound NAT to use the CARP IP instead of the interface IP for packets sending out to WAN.


  • @viragomann
    Here is the NAT rule for the concerned VLAN.
    NAT detail.PNG

    I have already put the VIP address instead of the interface address.


  • Here is some additional info. I tested the progress of a request (from the web server to google). the results are as follows (The previous captures already show the pind from the server to the OPT10 VIP):

    OPT10 VIP to OPT10 interface = OK:
    OPT10 VIP vers OPT10 interface.PNG

    OPT10 interface to WAN interface = OK:
    OPT10 interface vers WAN interface.PNG

    WAN interface to WAN VIP = OK:
    WAN interface vers WAN VIP.PNG

    WAN VIP to 8.8.8.8 = KO:
    WAN VIP vers goole.PNG

    OPT10 VIP to WAN VIP = OK:
    OPT10 VIP vers WAN VIP.PNG

    OPT10 to 8.8.8.8 = KO:
    OPT10 interface vers google.PNG

    I have a default route (via shellcmd):
    shellcmd.PNG


  • One more info, when I do a traceroute on my production (only 1 Pfsense), the 1st line corresponds to the gateway of my Proxmox host (In reality, the exact gateway, the one I added on the way via shellcmd , finished in .254. I don't know why the tracert returns .252, but it works) :
    tracert prod.PNG

    On the other hand, from the server on which I am performing my tests, it does not work:
    tracert test.PNG

    The problem seems to come from there.


  • That now seems to be a bit more complex with your default gateway on Proxmox. Why do you need this?

    Can you provide a drawing of your (virtual) network?

    And why do you set the routes via shellcmd? I can't see anything which can't be done natively in pfSense?


  • @viragomann

    Hello @viragomann

    The Proxmox gateway is, to my knowledge, necessary, because it is a failover IP used by the Pfsense WAN. I added his MAC address in his Proxmox configuration.
    Here is a network diagram:
    network.png

    Regarding shellcmd, I followed a tutorial. this is to apply the rules automatically each time the VM is restarted.

    I continue to do checks, I realized, the shellcmd command does not work since the implementation of the HA.
    As we see in the capture below, the road is missing add.

    Route on the production:
    route prod.PNG

    Route on the Test:
    route test.PNG

    While waiting for a response, I will try to add the missing routes.


  • @mourad13 said in Help for CARP configuration with a single FO IP:

    The Proxmox gateway is, to my knowledge, necessary, because it is a failover IP used by the Pfsense WAN.

    So this seems to be your upstream gateway.

    Does Proxmox also have an IP in your WAN subnet?

    Do a packet capture on the WAN interface with ICMP filter, while you try a ping to verify the outbound NAT is working.


  • @viragomann
    I had hidden the IPs, but it doesn't matter. This is a server that will soon be terminated and which is only used for these tests currently.

    The IP of the Proxmox server is 37.59.49.15
    The Proxmox gateway is 37.59.49.254

    The Failover IP (WAN VIP) is 146.59.130.46/32

    I tried to capture the packets, but I have no results (except the 2 OVH IPs for monitoring).
    I did a ping from Pfsense and from a client behind OPT10.
    I only have results when the capture is on OPT10.

    WAN :
    capture conf.PNG
    Capture.PNG

    OPT10 :
    Capture OPT10.PNG

    However, when I ping the VIP WAN IP from the client, the ping is OK, but no capture.

    I managed to reproduce the same routes as on my production, but only in cmd. I can't get from the interface (because of the mac address in the gateway and the / 32).


  • @mourad13 said in Help for CARP configuration with a single FO IP:

    The Proxmox gateway is 37.59.49.254
    The Failover IP (WAN VIP) is 146.59.130.46/32
    WAN => 192.168.100.1

    The gateway you're trying to access does not reside within any of the pfSense subnets.
    So I assume now, the route commands in shellcmd should provide a kind of PPP. Otherwise the gateway can't be reachable from pfSense.
    However, I'm not familiar with such configuration and cannot say if you have done it correctly. Also I'm not sure if that can work from the view of the gateway. I think the gateway has to support PPP as well.


  • @viragomann

    Thank you anyway for your help. This allowed me to identify the problem and better diagnose a routing problem.

    For my part, I carried out some test and I saw the change.
    In the routing table, without a gateway, the "use" column remains at 0.

    When I put the Proxmox gateway, some traffic seems to be detecting.
    I think it's a routing or NAT problem.

    Gateway :
    gw.PNG

    Route :
    route ok.PNG

    According to this tutorial (in French), it should however work.
    The only difference is, potentially, the / 32 mask.
    https://voiprovider.wordpress.com/2017/03/26/la-ha-avec-pfsense-et-1-seule-ip-wan/

    I will probably create another post in the "routing" category with a link to this post.