There's some odd quirks in the Broadcom Trident II ASIC on the Arcticas that make it impractical to SPAN on the switches (they insert dotq tags on SPANned packets originating on untagged ports but destined for tagged ports - you wind up with a mix of tagged and untagged packets coming out your SPAN port).

We do have a workaround for the moment: running a script on the firewalls that periodically does an arping (using the CARP mac) if that firewall is currently the carp master, and a trigger on a CARP state change that does an immediate arping on a BACKUP->MASTER transition.

Hi,
thanks for the commands, I'll have a look into this later this week.

Regards
Marco

You can use a single public IP, with some restrictions.
Here are some notes I took, not definitive, but they worked for me.

Put private ips on the WAN interfaces of the primary and secondary firewalls.
I used the public ips with a 10. for the first octet and the correct subnet mask
If it's a /30 you may have to use .1 and .2 or something. It probably doesn't matter.
Leave the gateway blank for now. Un-check the block private option.

Make sure you are cabled in correctly, you may want to put the secondary in carp maintenance mode

Add a CARP vip on the interface with the public IP.

Add the gateway

add an outbound nat rule, something like this-
WAN 'this firewall' * * * (CARP IP) * NO

Restart dpinger after adding the rule.

Update interface with gateway.

Gateway status should show up on primary, but will be down on secondary.

Add port forwards and outbound nat as usual, using public carp. (Not interface address)

Proxy Server: General SettingsGeneral -> Advanced Features -> Integrations

Add the option:

http_port 192.168.0.237:3128

192.168.0.237 is the virtual IP.

try this,

after the system comes back up, and pfsense master fails back then goes down, try to reboot your modem.
It's possible it has something to do with the modem and its arp table.

Just reporting back my success.

I successfully brought up a temporary node to the same version as my other node. Moved the slave VM over and tested the fail over. Both way worked.

I later had a look at the event logs and I saw the incompatibility of the integration software on VM on my host.

All these trouble and it was because the VM on the node didn't have the right version of integration software….

I hope this can help others too... If you are running pfSense on a VM, make sure you check the integration software and have the correct version installed. Sometimes when you migrate back and forth, you lose track on the software version and it may not be compatible with the host's version!

Thank you.

Thanks for your tips! Thats almost exactly the way I do it right now because of this strange behavior.

Hindsight is 20/20  :-[

I'm taking a short outage on the weekend to update the primary properly. Lesson learned.

I'm not quite sure I followed but I think we have a similar setup in our data center.  Our WAN IP is in a /29 along with its gateway (a data center router).  A /25 is routed to our WAN IP.  pfSense's LAN IP is in the /25 (x.x.x.1) so is the gateway for the "LAN's" public IP addresses.

If you want a second device in the "outside" /29 you need to set it up in parallel with your pfSense not behind it.  A router won't pass "WAN subnet" traffic back through into the LAN since that's not where it is supposed to go.

Alright, I have a new issue now that I have used the tcp_outgoing_address command to specify my VIP for all outgoing HTTP traffic.

Nothing in my setup has changed except for enabling the clamAV engine in squid.  Since doing so, pages load slowly or not at all.

If I remove the tcp_outgoing_address command from my custom options, the problem goes away.

Files from eicar.com are caught by clamAV and there is no impact to performance.

As soon as I re-enter the tcp_outgoing_address into my squid custom options everything goes in the crapper.

Any ideas anyone?

Limiters and pfsync still won't work properly together.
You may disable state sync.
https://forum.pfsense.org/index.php?topic=108815.0

Good morning and once again thank you for your effort.

I set /24 because I thought that it has to be the same mask as the mask of the WAN IP. I also tried /32 though…

What do you mean with "private subnet"? 10.0.0.0/8 172.16.0.0/12 etc.? That's not the case...

I hope I find the time today logging/monitoring.

Is it possible that other NAT rules somehow interfere?

Hi all,

It's work!
Thank

Its now working!

Forgot to add 51.148.46.xx/29 to the Cisco router and set the interface. (WHAT A NOOB)

On pfSense all what is needed is to add the IP's to "Virtual IP Addresses" and set them up on "Firewall: NAT: 1:1"

I deleted Gateway51 from the gateway list as its not needed

Found the issue, PEBKAC. The LAN interfaces had inconsistent IPv6 settings (one was set to DHCP6 and the other to None). After setting them both to None the CARP failover works as expected.

Hi,

it's work, set rule for firewall, thank.

On your LAN side, if you have, say, a LAN and DMZ, you need rules to pass from LAN to DMZ without a gateway set. Under that, you can have a rule from LAN to any with a gateway set for whatever Multi-WAN scenario you setup (LB, failover, etc).

Hi,

I managed to resolve the issue for our case in the end.

The two servers we're using as our pfSense boxes are Dell PowerEdge R210II servers, each came loaded with 2 on board Gigabit Ethernet ports (one being used as the WAN interface and the other for the LAN interface).

In the first instance I had setup the pfSync to use the LAN interface, which I'm led to believe is a big no no, so I then set up a separate VLAN for the pfSync to use, but as this was still using the physical adaptor shared by the LAN interface, it made no difference.

In the end I bought and fitted an additional PCIe Gigabit Ethernet card in each of the servers, set up a VLAN to use the new physical adaptor (not being used by anything else) and set the pfSync to use the new VLAN and since then I have seen no issues with the sync slowing down or the Backup box becoming unresponsive whilst adding users.

I have now put the new pair into production and we've seen no problems.

Thanks everyone for their help and suggestions.

Hopefully this will help somebody else encountering similar issues.

Cheers,
Jan