Help for CARP configuration with a single FO IP

[mourad13]

Hello,

I would like to know if it is possible to configure high availability with a single failover IP?

Here is what I currently have as configuration on my PfSense:
WAN => Public_IP / 32
LAN => 192.168.0.254
OPT10 => 192.168.10.254 (VLAN on the LAN card)

And what I would like:
PfSense_MASTER
WAN => 192.168.100.1
VIP_WAN => Public_IP / 32
LAN => 192.168.0.254
OPT10 => 192.168.10.250
VIP_OPT => 192.168.10.254
SYNC => 192.168.150.1

PfSense_BACKUP
WAN => 192.168.100.2
VIP_WAN => Public_IP / 32
LAN => 192.168.0.254
OPT10 => 192.168.10.251
VIP_OPT => 192.168.10.254
SYNC => 192.168.150.2

I did a test, but it doesn't work for now. Synchronization does not take place.
From the VM behind OPT10, I ping 192.168.10.254 and Public_IP. But not beyond (8.8.8.8 for example).

[mourad13]

I solved the problem of synchronization between the 2 PfSense. the synchronization problem between the 2 PfSense.
On the 2nd, I added the 3rd network card before creating the OPT10.
As the interfaces must be aligned, the sync did not work.

However, I still can't access the internet. Neither from the parent company, nor from the VM.
I also do not access the vm (web server) from the outside.

[mourad13]

Further info, when I ping 8.8.8.8, the response is:

The client's gateway is the VIP address (192.168.10.254).
the answer (192.168.10.250), is the "real" address of the interface (OPT10).

Here are the rules of the OPT10 interface, WAN and the NAT rules

[viragomann]

What is the CARP status page showing?
Status > CARP

[mourad13]

@viragomann

Hello,

Here is a capture of the CARP status
Primary:

Backup:

As I said, from a client behind the firewall, when I ping 8.8.8.8, the response comes from 192.168.10.250.
If I turn off the primary firewall, the response comes from Backup (192.168.10.251).
Does this sound like a routing problem?
So far everything is ok.

On this tutorial (in French), he talks about creating a default gateway.
Is that related?

Cordially.

[viragomann]

I think, you're are missing the outbound NAT rule. You must set the Outbound NAT to use the CARP IP instead of the interface IP for packets sending out to WAN.

[mourad13]

@viragomann
Here is the NAT rule for the concerned VLAN.

I have already put the VIP address instead of the interface address.

[mourad13]

Here is some additional info. I tested the progress of a request (from the web server to google). the results are as follows (The previous captures already show the pind from the server to the OPT10 VIP):

OPT10 VIP to OPT10 interface = OK:

OPT10 interface to WAN interface = OK:

WAN interface to WAN VIP = OK:

WAN VIP to 8.8.8.8 = KO:

OPT10 VIP to WAN VIP = OK:

OPT10 to 8.8.8.8 = KO:

I have a default route (via shellcmd):

[mourad13]

One more info, when I do a traceroute on my production (only 1 Pfsense), the 1st line corresponds to the gateway of my Proxmox host (In reality, the exact gateway, the one I added on the way via shellcmd , finished in .254. I don't know why the tracert returns .252, but it works) :

On the other hand, from the server on which I am performing my tests, it does not work:

The problem seems to come from there.

[viragomann]

That now seems to be a bit more complex with your default gateway on Proxmox. Why do you need this?

Can you provide a drawing of your (virtual) network?

And why do you set the routes via shellcmd? I can't see anything which can't be done natively in pfSense?

[mourad13]

@viragomann

Hello @viragomann

The Proxmox gateway is, to my knowledge, necessary, because it is a failover IP used by the Pfsense WAN. I added his MAC address in his Proxmox configuration.
Here is a network diagram:

Regarding shellcmd, I followed a tutorial. this is to apply the rules automatically each time the VM is restarted.

I continue to do checks, I realized, the shellcmd command does not work since the implementation of the HA.
As we see in the capture below, the road is missing add.

Route on the production:

Route on the Test:

While waiting for a response, I will try to add the missing routes.

[viragomann]

@mourad13 said in Help for CARP configuration with a single FO IP:

The Proxmox gateway is, to my knowledge, necessary, because it is a failover IP used by the Pfsense WAN.

So this seems to be your upstream gateway.

Does Proxmox also have an IP in your WAN subnet?

Do a packet capture on the WAN interface with ICMP filter, while you try a ping to verify the outbound NAT is working.

[mourad13]

@viragomann
I had hidden the IPs, but it doesn't matter. This is a server that will soon be terminated and which is only used for these tests currently.

The IP of the Proxmox server is 37.59.49.15
The Proxmox gateway is 37.59.49.254

The Failover IP (WAN VIP) is 146.59.130.46/32

I tried to capture the packets, but I have no results (except the 2 OVH IPs for monitoring).
I did a ping from Pfsense and from a client behind OPT10.
I only have results when the capture is on OPT10.

WAN :

OPT10 :

However, when I ping the VIP WAN IP from the client, the ping is OK, but no capture.

I managed to reproduce the same routes as on my production, but only in cmd. I can't get from the interface (because of the mac address in the gateway and the / 32).

[viragomann]

@mourad13 said in Help for CARP configuration with a single FO IP:

The Proxmox gateway is 37.59.49.254
The Failover IP (WAN VIP) is 146.59.130.46/32
WAN => 192.168.100.1

The gateway you're trying to access does not reside within any of the pfSense subnets.
So I assume now, the route commands in shellcmd should provide a kind of PPP. Otherwise the gateway can't be reachable from pfSense.
However, I'm not familiar with such configuration and cannot say if you have done it correctly. Also I'm not sure if that can work from the view of the gateway. I think the gateway has to support PPP as well.

[mourad13]

@viragomann

Thank you anyway for your help. This allowed me to identify the problem and better diagnose a routing problem.

For my part, I carried out some test and I saw the change.
In the routing table, without a gateway, the "use" column remains at 0.

When I put the Proxmox gateway, some traffic seems to be detecting.
I think it's a routing or NAT problem.

Gateway :

Route :

According to this tutorial (in French), it should however work.
The only difference is, potentially, the / 32 mask.
https://voiprovider.wordpress.com/2017/03/26/la-ha-avec-pfsense-et-1-seule-ip-wan/

I will probably create another post in the "routing" category with a link to this post.