DHCPD LOG.txt

@decibel83 A problem at Layer 2 is the most common cause.

FYI- You can disable NAT and route without also disabling the firewall.

Firewall > NAT, Outbound tab, set it to Disable Outbound NAT and save/apply.

@james92 Yes a dumb switch is fine.

@davidredekop Interesting. I have never had to change anything in proxmox for CARP.

As an aside, while fc00::/7 is the ULA network space, fc00::/8 is currently undefined. fd00::/8 is proper ULA addressing. Recommend implementing RFC 4193 and randomly selecting a /48 for ULA usage.

i was able to track down a bit of a solution
we had disabled hardware offloads , this is now turned back on which make xmlrpc sync much quicker and lower load and cpu.

also

we have two wans, on each wan we had two openvpn servers listening for different purposes, 7-8 years ago we were told that its best to listen on localhost with each vpn server, then nat port forward each external port so that each wan can listen on the same server, it appears if we do this now, each time an xmlproc sync occurs it causes pfctl and the reload scripts to thrash and loop 3 or more times.
we this this occuring over and over with localhost
php-fpm[6973]: /rc.openvpn: OpenVPN: One or more OpenVPN tunnel endpoints may have changed its IP. Reloading endpoints that may use VPN

the solution now is listening on a single carp ip, this means were not able to openvpn in the backup wan, but atleast vpn works on master and backup servers, just not the backup wan

all xmlrpc sync is re-enabled and no CARP timeouts so far.......

@nocternal Yup, I'll be doing just that, super nice we can do "micro" patches like this.

Thanks again!!

Just to add for anyone else coming across this issue.

Adding a vlan and therefore triggering a configuration reload and mini failover, caused exactly the same issue. Which was not fixable with restoring a configuration backup or even a restart of both firewalls.

Applying this patch:
Fix automatic firewall rules for HA DHCP server failover (Requires reboot or filter reload to activate, Redmine #13965)

Fixed the issue with the DHCP server.

The issue showed in Status / DHCP Leases a permanent status of My State - 'Recover', as well as previously mentioned 'communication-interrrupted'

@derelict At the moment, I only have one firewall.

I will add the other one later when I have more WAN addresses.

@tony-soprano
Any switch should support CARP. The protocol simply sends out mulitcasts to talk to the other node. So both has only to be within the same L2 network.

@viragomann
I have read the Netgate documentation regarding Single Address CARP. This brief paragraph states there are significant challenges with updating, etc. I tried to look around for more information on this type of setup. What info I did encounter seemed to be people who were spoofing the WAN MAC Address on the secondary unit and using scripts to determine the interface state(ifup/ifdown) in an attempt to avoid collision.
I would be hesitant to put any real trust in hacks of this nature in a corporate production environment.

Edit: PPPoE is not being used.

@jakub_ Yes. The advertisements are sourced from the interface IP address and CARP MAC.

Not sure why you are seen advertisements from both the primary (advskew 0) and secondary (advskew 100) there.

@viragomann said in Different MAC addresses for virtual ip and WAN interface:

nce both need to communicate with the gateway, you need to state the correct mask for both. That's not gonna to work on different interfaces.

Yes, I am trying that in but I think it goes wrong somewhere it the MAC addresses because I set it VirtualIP but it does not work.

@philippe-richard Hi.

Yes. After converting a few CARP to IP alias, the problem disappear.
Note we have applied the patch #12827.

Hi again.

Enabled promiscuous mode in esxi and now it works.

@louis2

I have been trying 'to fix' the issue 'that the sftp-server sees the proxy address' and not the 'client address'

Since a proxy is forwarding a package, it is not strange that the server at the destination side normally sees the address of the proxy and not that of the client.

Luckily there are protocols which allows the proxy to forward the client address.

So the big question is how:

to enforce HA-proxy to forward the client IP and how to enforce the SSH-server to use the (added) client address

The config is as follows:

pfsense 2.7 actual build ha-proxy haproxy-develop in ssl /https(TCP-mode frontend listening to WAN-address (4/6) port 22 IPV4-frontend and an IPV6-frontend IPV4 and an IPV6 back-end bitvise (advanced) ssh-server (on windows)

That works, no problem apart of the 'lack of client address issue'.

I did a lot of searching on the internet and found options like:

‘option forwardfor’ (usable for the front and/or the back-end) options like 'send-proxy' and 'send-proxy-v2' and for bitvise 'proxy protocol'(disabled or required (default disabled)) and 'Enable UPnP gateway forwarding (on/off (default off))

After reading the links below I decided to try almost all possible options ...... nothing worked.

So if someone has a working config, I would love to know how 😊

I did google a lot. Here some links, which might or might not help (enough)

https://www.haproxy.com/documentation/hapee/latest/load-balancing/client-ip-preservation/add-x-forward-for-header/

https://www.reddit.com/r/PFSENSE/comments/108siet/forwarding_source_ip_from_haproxy/

https://forum.netgate.com/topic/159562/solved-haproxy-forward-client-ip

@louis2 said in HA-proxy: How to forward url-A to url-B?:

In my actual case port 80 and/or port 443 should be forwarded to the corresponding back-end, where the certificate is at the server, not on HA-proxy/pfSense.
(HA-proxy should be transparent for the SSL)

For access control I choose ^Host ends with^ ^.example.com^
PS I assume that 'http-request redirect' can also handle https !?

Sure, it can. But I'm in doubt that HAproxy can read the host header of an encrypted request without having the server certificate and private key for it.
It can read the SNI at its best. But this might only work in TCP mode and then not for unencrypted traffic.

So maybe you can configure a separate HTTP frontend listening on port 80 for unencrypted request. Here you can use a host header ACL doing the http-request redirect.
And configure an additonal SSL/HTTPS TCP mode frontend for port 443, encrypted traffic with an SNI ACL and redirect it to whatever you want.

Well,

I'm trying to replicate the issue in a test environment with a single pfSense box using CARP IPs and a spare Aruba CX switch. Of course I can't 😠

I guess my next step is to actually setup a HA setup and then remove the secondary and see what happens...