2 HA pfsense boxes with 1 public IP working but…
-
For starters, stop unplugging cables. Yes, you should test eventually but start with basic functionality.
All of your CARP VIPs should be MASTER on primary and BACKUP on secondary. If not, fix that.
Use the Temporarily Disable CARP button on Status > CARP on the primary. All of your CARP VIPs should go MASTER on the secondary (Again, Status > CARP on the secondary). If they do not, fix that.
What documentation did you follow in setting up your HA cluster?
-
I used this document:
https://doc.pfsense.org/index.php/Configuring_pfSense_Hardware_Redundancy_%28CARP%29The IP, CARP and HA part works; just not 100% of the time. All CARP related settings, rules etc., copy from master to slave.
I’m unplugging cables to test fault tolerance. What if the WAN connection goes down for 3 seconds and comes back up? If the master doesn’t come back online properly, then everything going through the firewalls will go down. Is unplugging the WAN not a valid test? This behavior that I am seeing with pfSense HA something acceptable?
You can see what I’m talking about in this video https://youtu.be/pepq4VLOUHE
This is the current setup:
“ISP” SIDE CONFIGURATION
66.66.77.10/24 Windows Server
“ISP“ Router
66.66.77.1/24 Public DMZ Interface
66.66.66.1/30 WAN InterfaceMY SIDE CONFIGURATION
PFS1
10.10.10.1/30 WAN Interface Dummy IP
66.66.66.2/30 WAN CARP IP
77.77.77.2/24 Public DMZ Interface
77.77.77.1/24 Public DMZ Interface CARP IPPFS2
10.11.11.1/30 WAN Interface Dummy IP
66.66.66.2/30 WAN CARP IP
77.77.77.3/24 Public DMZ Interface
77.77.77.1/24 Public DMZ Interface CARP IP77.77.77.40/24 Windows Server
-
What is a "Dummy IP?"
So you are trying to do HA without the required 3 public IP addresses (minimum /29) on WAN?
You are still not providing the necessary details to properly help you diagnose your problem. Run through the connectivity troubleshooting steps and say what actually fails and where.
-
MY SIDE CONFIGURATION
PFS1
10.10.10.1/30 WAN Interface Dummy IPPFS2
10.11.11.1/30 WAN Interface Dummy IPI pointed this out before, but you need to have them in the same subnet. Why not try 10.10.10.1/30 and 10.10.10.2/30 ??
-
What is a "Dummy IP?"
So you are trying to do HA without the required 3 public IP addresses (minimum /29) on WAN?
You are still not providing the necessary details to properly help you diagnose your problem. Run through the connectivity troubleshooting steps and say what actually fails and where.
Yes, I am trying to do HA without 3 public IP addresses on WAN. What other info do you need? Dummy IP = any IP placed in the IPv4 Address field of the WAN Interface that is not going to conflict with any other interface, host etc. The "Dummy IP" is required because PFS requires you to enter an IP address in that field. And, as I said before, I do NOT have the option of using more than 1 public IP address and overall seems to work.
MY SIDE CONFIGURATION
PFS1
10.10.10.1/30 WAN Interface Dummy IPPFS2
10.11.11.1/30 WAN Interface Dummy IPI pointed this out before, but you need to have them in the same subnet. Why not try 10.10.10.1/30 and 10.10.10.2/30 ??
Good question. I tried it before and it didn't make a difference. I will try it again.
-
MY SIDE CONFIGURATION
PFS1
10.10.10.1/30 WAN Interface Dummy IPPFS2
10.11.11.1/30 WAN Interface Dummy IPI pointed this out before, but you need to have them in the same subnet. Why not try 10.10.10.1/30 and 10.10.10.2/30 ??
Just tried it again and same problem.
-
Again - pretty sparse with the details.
-
Again - pretty sparse with the details.
Again, what more details do you need??? I asked what details you need before. I'll provide the details needed.
-
You are still not providing the necessary details to properly help you diagnose your problem. Run through the connectivity troubleshooting steps and say what actually fails and where.
Help us help you.
-
You are still not providing the necessary details to properly help you diagnose your problem. Run through the connectivity troubleshooting steps and say what actually fails and where.
Help us help you.
I have no idea what details you are looking for. All I know right now is this:
- I start ping command on server on both ends
- Unplug Master Firewall WAN connection
- I see Slave Firewall takes over when a packet is dropped and ping resumes
- Right after that I plug the Master Firewall WAN connection
- Master Firewall takes over but all pings fail
-
Details like what are you pinging from where when you're testing. Specifics, like interfaces and IP addresses.
-
I am pinging 77.77.77.40 from 66.66.77.10. I am also pinging 66.66.77.10 from 77.77.77.40.
Diagram of setup is attached. FYI, this is a closed lab setup.
-
What address is 77.77.77.1/X routed to on 66.66.66 ?
-
-
How do the 77.77.77.X addresses get from the ISP to you? They have to be routed to you somehow.
-
How do the 77.77.77.X addresses get from the ISP to you? They have to be routed to you somehow.
If you are asking me where the 77.77.77.0/24 addresses come from, that does not matter because this is a lab designed for testing only with no real connection to the Internet. It is only for testing PFS HA functionality.
If you are asking what am I using to route IP traffic between 66.66.66.0/30 and 77.77.77.0/24, I am using a separate installation of PFS.
-
OK it doesn't matter if it is routed to the CARP VIP or not. I'm done.
-
OK it doesn't matter if it is routed to the CARP VIP or not. I'm done.
That's fine. You weren't reading the details, exaggerating information needed and making things more difficult than they really are.
-
That's fine. You weren't reading the details, exaggerating information needed and making things more difficult than they really are.
No, he was trying to point out that you could have a dozen different things wrong with your lab setup which no one can easily sort out. e.g. something on the WAN side by default can't ping your LAN, so the fact that 77 whatever can't ping 66 whatever is probably irrelevant; the 'isp router' config is unknown, etc… I'm not sure what you are trying to test with your methodology either, someone unplugging the WAN on the master seems an unlikely event. The HA failures I've dealt with usually involve failed hardware. If I was going to test, I'd pull power on the master and see what happens. Anyway, you are seeking free assistance from strangers on the Internet. If you don't want to work with someone who steps up, fine, but don't be offended if no one else wants to spend time trying to figure out what's wrong with your setup.
