I found the answer. It was caching. After the server was rebooted, NAT didn't work anymore and I had to recreate the Proxy Arp entry in the pfSense.

No, not yet, but I was planning on it:  I dug a little deeper in previous posts and found a similar problem which was fixed by rebooting a router.  After reading your reply, I'm confident that doing so (in conjunction with having the right type of VIPs) will take care of the problems I was having.  I can't try again for a few days because another project got elevated priority.  Thanks very much for your response.

Is your outbound nat set up to use this VIP?

Are you trying to reach more redundancy by using two subnets or you really need to connect two subnets to LAN?

You can explicitly disable multicast/broadcast and put these rules on the top.
Or alternatively - do not allow ANY protocol to ANY address but allow only traffic you really need to allow.
First approach is more efficient as multicast/broadcast packets are droped without going through all rules before being dropped by 'default deny all'.

What does that mean please?
@cpliu903:

when R1 sync to R2, R2 change status to master,  not backup.

ok..thanx..that was my guess too.

I believe some people have gotten this to work by adding 5 vlan interfaces and setting them all for DHCP. I'd do a forum/docs search going down that route and see if you can turn up better results. I seem to recall it coming up within the past two weeks.

I have not given all the details:

I have configured squid to bind on the LAN if.

I have a real IP on the WAN (ip of class 88.xx.xx.12/24)
and several VIP (ips of  class 93.xx.xx.0/24)

I need the squid users to come out with one of the  VIP, by adding
directive tcp_outgoing_address 93.xx.xx.10  in squid conf.

How can I do  to do that?
(I think I need carpdev to have VIP on virtual interface CARP)

kind regards

You could put a dumb off-the-shelf router in NAT mode in front of the pfSense.
Or if the modem supports that, reconfigure the modem.

Basically put a static, private subnet in front of the pfSense.

@GruensFroeschli:

Are the additional IPs routed to your WAN?

They are routed to my PPPoE address.  They are not configured on an adapter other than as virtual IP's.

@GruensFroeschli:

Are they all in use right now?

The 8 IP Block is fully utilised via 1:1 NAT and are all working fine
The 16 IP block is utilising 2 IP's at present.  1 is NAT, the other is configured as the live IP on the virtual linux system

@GruensFroeschli:

Do you have the public IP directly on the server?

The only live IP configured is the PTP ip for my ADSL connection

@GruensFroeschli:

There are several ways to go at such a problem.

Bridging VIPs –> The public IP on the pfSense, the internal servers have private IPs, traffic from the public IP forwarded to the private IP. routing
You cannot use VIPs if you have the public IP directly on the server itself.

Presently, VIP's and routing are accomplishing things for all the other IP's.
The IP that is causing the issue is the virtual machine configured with the real world ip (no internal IP address allocated)

I just experienced something similar (or same) while setting up two new 1.2.3 based embedded routers.  pfsync is working fine (over a dedicated interface)
I created 3 vips (LAN,LAN2,WAN) in that order, vhid's: 1,2,3.  The interfaces would fail over separately - I unplug WAN, it fails over to router2, but LAN stays on router1, and obviously does not provide upstream connectivity.

In desperation, I removed LAN2 and WAN vips, and recreated just WAN vip (LAN vhid:1, WAN vhid:2).  Now LAN+WAN seem to fail over together when WAN cable is pulled.  In fact, it all seems to work OK (except DHCP which I'll start a separate topic on) except that when we fail-back to router1, WAN VIP shows as "master" on both machines!  I have to do some more checking as to whether this is affecting anything, but it seems pretty strange.

Thanks for your prompt answer.
It is what I needed.

Regards.