@viragomann Thanks for your reply. Currently, I can't reach CARP IPs, I don't know where I'm wrong, CARP IPs of LAN is 172.16.100.4. I only can ping CARP IPs of WAN 10.84.100.4

and if I create master 10.84.3.2, slave 10.84.3.3 with VLAN 3. After set up that you can add 10.84.3.1 as CARP VIP on the master. I cannot ping as well.

@prk
You can do that all with Firewall > NAT > Outbound. Switch it into hybrid mod, then you can add rules to override the default behaviour (masquerading).

If you strict want to forward a public IP to a certain internal and have this internal IP use that public, you can use NAT 1:1 rules.

However, before you have to assign each IP out of the additional /29 subnet in Firewall > virtual IPs as type "IP Alias" to your WAN.

Still an issue with 2.5 by the way...

@jimp said in Can I find out the status of the CARP interface (BACKUP / MASTER) through a command?:

ifconfig -a | grep 'carp:'

This solution looks perfect, I only made one adjustment to get only the MASTER / BACKUP:

UserParameter = pfsense.carp.state, ifconfig -a | grep 'carp:' | cut -d '' -f2 | sed -n 1p

Sed is for taking only one CARP interface, it is very rare for one interface to be BACKUP while the others are MASTER, and vice versa.

PS: I don't know if I should close this post as resolved or how to do it if I should.

@lucazio

Hi,
what you want is net.inet.carp.preempt.

The preempt shold be enabled. That means if one interface is failing on a pfSense then ALL Interface do a failover not only one.

Also bare in mind I have seen some complications with carp and multicast on the esxi and the security settings of the protgroup / swtich. (Multicat - promismode / ARP address Change)

Once I was able to properly google for things I already know I didn't know I found this.

https://forum.netgate.com/topic/35849/accessing-wan-s-public-ip-from-the-lan-not-working-please-help/6

Split DNS worked like a charm for me!

Might need to enable reflection in the future but for now it the DNS method works fine.

@free4 said in Alert "XMLRPC method captive_portal_sync" in 2.5:

@jimp Oh ok

But wait...What's the point to backport the fix into RELENG_2_5_0 then ?

So it will be included in the next patch release, whenever that may be.

@mr_jinx
You can configure a failover group with the WAN gateway and the others box LAN interface.
So on the secondary you have to add the primarys LAN address as a gateway first. Then add a gateway failover group where you set the WAN GW as tier 1 and the pirmarys LAN IP as tier 2.
So now if the WAN GW is not accessible (cause the primary owns the WAN CARP) it goes out over the primary.

You can do the same on the primary with the secondarys LAN IP to retrieve updates when it's in CARP maintenance mode.

@viragomann
I already tried to build a chain, but dont like this aproach.
If one is temporary not reachable the update gets lost for all others in the chain and you'll not notice it.

@chrisnz
Hello, being of two distinct networks which, I think, should not be able to communicate with each other, the solution is to add an interface to the pfSense router, in your case not physical.
Since your switch is web managed the best thing you can do is to create a VLAN dedicated to the Guest network and use the switch for all your private connectivity. And only for those!
You will find everything you need in the pfSense and Netgear documentation, in the respective sections that talk about VLANs.
Googling I found this which looks a lot like the recommended solution:
pfSense router-on-a-stick VLAN configuration with a Netgear GS108E
I hope it will be useful to you.

@bimpe said in HA/CARP, with DHCP error:

https://forum.netgate.com/topic/106394/dhcp-not-working-properly-solved

The XMLRPC process will automatically add +100 to each skew when synchronizing the VIPs to the secondary node.

skew on second server with DHCP is more than 20 by ifconfig | grep carp ?

@koby-peleg-hen said in Another XMLRPC communication error:

ALL I want to achieve is 2 nodes on Heztner Cloud that can be sync between them for easy management

Sync is always primary to standby, never "to each other" or "between them". So I'd be careful with that. If you just want the config to be synced but no HA why sync at all? Just to have the same Aliases? If you don't run HA you commonly have other NICs/Interfaces or additional Interfaces and rules, syncing that to another node with a whole different setup makes no real sense to me?

@derelict said in VHIDs with two CARP HAs in the same LAN network?:

The CARP MAC address is derived from the VHID. This also applies to VRRP on the same segment.
You must use unique VHIDs on the same broadcast domain or you will experience MAC address collisions.

Hi, i will try it with unique VHIDs and let you know my results! Thanks for your fast help, regards Norbert!

Yes. consumer router between ISP modem and both pfsense. set the carp WAN IP as DMZ so you don't run into double nat scenario, and if you wish set the consumer router to hand out the same IP each time to each pfsense box. works like a charm. Yes, single point of failure in the consumer router, but with no rules or anything on it it's easy to swap out if you have a failure. perfect for home use or work.

Hey friend. This seems a little late coming, but I thought I'd leave a reply here as I ran into the same "issue" myself just today.

By the nature of an IPSec tunnel, they do not truly get "started" or "stopped" they only come up when traffic that is being routed through them is detected. So in this case, basically as soon as there is traffic on the IPSec tunnel from the secondary node after your primary fails, it will connect perfectly fine with the new tunnel.

To test this yourself, you can add an address in your P2 entry at the bottom to automatically ping. This ping will occur every few seconds and as soon as the first ping is sent from the secondary node after the fail, the tunnel will reconnect and allow traffic to pass.

Hopefully this helps you and any others pursuing this topic in the future!

@saymeeeow This is as official as it gets.

Redmine says its scheduled after version 2.5
So its gonna take a while.

I'm also experiencing issues with openvpn
Used as a client, when secondary node restarts, even though isn't master, openvpn client starts, causing havoc to main instance.

Straight forward to replicate.

@filosofixit
Now it's clear.

The radios have to be configured to use pfSense (192.168.0.1) as default gateway.

On pfSense you have to remove the check at "Block private networks" in the WAN interface settings.
Then add a firewall rule to WAN allowing traffic from the radios subnet (maybe 192.168.0.0/24) to whatever you need.

For accessing the radios from LAN you should disabling the NAT on these connections. Go into the outbound NAT settings. If it's in automatic mode, select the hybrid mode and save it.
Than add a rule:
Do not NAT: checked
Interface: WAN
source: LAN network (or an alias for all your internal networks or RFC1918)
destination: the radios subnet (192.168.0.0/24?)

@sipriuspt said in Question about switchs to be used between WAN CARP and ISP's:

Seems like this model with 4 SFP+ ports, as a 8 port model

if you want a specifically "SFP" switch I recommend this: https://www.cisco.com/c/en/us/support/switches/sg350-10sfp-10-port-gigabit-managed-sfp-switch/model.html
(stable reliable but no 10Gig option)

and you can use such RJ45 copper modules too:
https://www.fs.com/de-en/products/23681.html
(there are plenty of SFP manf. codes available)

5a61dd0b-dc90-4d9e-add4-91af54dd1d3c-image.png

I can still recommend for 10Gig:

https://www.ui.com/unifi-switching/unifi-switch-16-xg/
(here you have to reach into your wallet, deep down)
😉

@daddygo I've updated to the latest version. I still do get issues with the carp IP syncs. I don't get why it's doing a splitbrain master config all the time. Additionaly it does shift the IPs in the list. I don't get why. Is there any way, I can dig deeper log level wise to figure out what actually might be the reason?

@maartenv
FOUND IT! My hostname was wrong. I forgot to set the right hostname in the "General Setup"

@clumbo
The docker didnt listen yet? ( https://forum.netgate.com/topic/159331/haproxy-and-websockets?_=1608750100359 ) or different issue here?

@viragomann Good luck with that.

It might work for some things but if you do something like create a new CARP VIP with an advskew of 100 (the default on the secondary) I would expect it will not sync to the primary correctly.

@derelict it is a physical setup, not virtual.

I have the core network setup with BGP routing working via FRR.
The 2 BGP legs are on separate subnets, so CARP multicast isn't working...

I've been working hard on this setup and have (via a lot of reading and fustration over the CARP functionality not beinging configurable to something else eg simple ping's and then sync the state over the SYNC interface) have ended up with something like the following:

The two BGP routes are connected to two switches (which are stacked for failover and LAGG/LACP) and then setup LACP on the WAN interfaces on the XG-7100's so they are connected to each BGP and then have a network specialist (I'm dumb when it comes to network equipment) configure the switches for me so LAGG/LACP is isolated to 3 ports and multicast is kept on the ports the WAN ports are located in to avoid network spam.

I hope this will work, but I need to read up on the XG-7100 to setup WAN LACP and find a way to test it without the actual switch(es) to avoid downtime. For some reason Netgate likes to use switches and obscure non ethX naming schemes for everything and not expose information about the physical layer before actually configured PFSense makeing i difficult for a (PFSense/FreeBSD) noob to get up 2 speed.

There is a first time for everything.

Thanks for your updates and sorry for the very long delay in the update - I had to get moving on this project faster than expected and have just finised to the above state where WAN LACP and stacking switch configuration is needed (even our hosting partner had issues with BGP due to PFSense not being Cisco and for some reason FRR had issues with the BGP password causing weired issues and one of the BGP CPE's fail on ARP refresh from time to time, so it has been a fustrating and slow process).

@piba

Thankyou

This is just what I have been looking for I have been trying to workout how to do this with bitwarden_rs

Any chance you could share your a more full screenshot of the backend part too?

Thanks

@rivest1000 That should be fine. Sounds like you need to simultaneously capture an interesting connection on all three inside interfaces and see what there is to see. Sorry but it's something unique to your environment based on what I have so far. Are the missing FIN/SYN packets being sent to the primary while the secondary is MASTER?

You're POSITIVE the zabbix hosts have the correct default gateways for the necessary traffic?