2 HA pfsense boxes with 1 public IP working but…
-
You should have your WAN interfaces on the same bogus network, your setup shows them in separate /24s. When I did it, I used a private /30 as the public IP was a /30. That's probably not needed.
-
@Derelict: I was pinging various valid IP addresses on the internet.
@dotdash: I used a /24 subnet because that is what the ISP is giving here in the test lab. I final setup supposed to be a /30.
Today, I changed the setup. I took the whole thing off the internet and setup "my own internet" with public IP addresses and set up pfSense to route public IP addresses from the WAN to a DMZ. I got it to work but displays the same behavior as the previous setup.
Basically, if I unplug the master firewall's WAN cable, routing switches to the slave firewall properly within a second. When I see the ping timeout and then resumes receiving replies I plug the WAN cable back in to the master firewall. Sometimes it switches back to the master without a problem but sometimes it does not.
When it does not come back successfully, I unplug the master firewall's WAN cable again and the slave takes over successfully. It seems like I have to unplug all of the cables and plug them in an order where the WAN is plugged in first and then the rest.
-
For starters, stop unplugging cables. Yes, you should test eventually but start with basic functionality.
All of your CARP VIPs should be MASTER on primary and BACKUP on secondary. If not, fix that.
Use the Temporarily Disable CARP button on Status > CARP on the primary. All of your CARP VIPs should go MASTER on the secondary (Again, Status > CARP on the secondary). If they do not, fix that.
What documentation did you follow in setting up your HA cluster?
-
I used this document:
https://doc.pfsense.org/index.php/Configuring_pfSense_Hardware_Redundancy_%28CARP%29The IP, CARP and HA part works; just not 100% of the time. All CARP related settings, rules etc., copy from master to slave.
I’m unplugging cables to test fault tolerance. What if the WAN connection goes down for 3 seconds and comes back up? If the master doesn’t come back online properly, then everything going through the firewalls will go down. Is unplugging the WAN not a valid test? This behavior that I am seeing with pfSense HA something acceptable?
You can see what I’m talking about in this video https://youtu.be/pepq4VLOUHE
This is the current setup:
“ISP” SIDE CONFIGURATION
66.66.77.10/24 Windows Server
“ISP“ Router
66.66.77.1/24 Public DMZ Interface
66.66.66.1/30 WAN InterfaceMY SIDE CONFIGURATION
PFS1
10.10.10.1/30 WAN Interface Dummy IP
66.66.66.2/30 WAN CARP IP
77.77.77.2/24 Public DMZ Interface
77.77.77.1/24 Public DMZ Interface CARP IPPFS2
10.11.11.1/30 WAN Interface Dummy IP
66.66.66.2/30 WAN CARP IP
77.77.77.3/24 Public DMZ Interface
77.77.77.1/24 Public DMZ Interface CARP IP77.77.77.40/24 Windows Server
-
What is a "Dummy IP?"
So you are trying to do HA without the required 3 public IP addresses (minimum /29) on WAN?
You are still not providing the necessary details to properly help you diagnose your problem. Run through the connectivity troubleshooting steps and say what actually fails and where.
-
MY SIDE CONFIGURATION
PFS1
10.10.10.1/30 WAN Interface Dummy IPPFS2
10.11.11.1/30 WAN Interface Dummy IPI pointed this out before, but you need to have them in the same subnet. Why not try 10.10.10.1/30 and 10.10.10.2/30 ??
-
What is a "Dummy IP?"
So you are trying to do HA without the required 3 public IP addresses (minimum /29) on WAN?
You are still not providing the necessary details to properly help you diagnose your problem. Run through the connectivity troubleshooting steps and say what actually fails and where.
Yes, I am trying to do HA without 3 public IP addresses on WAN. What other info do you need? Dummy IP = any IP placed in the IPv4 Address field of the WAN Interface that is not going to conflict with any other interface, host etc. The "Dummy IP" is required because PFS requires you to enter an IP address in that field. And, as I said before, I do NOT have the option of using more than 1 public IP address and overall seems to work.
MY SIDE CONFIGURATION
PFS1
10.10.10.1/30 WAN Interface Dummy IPPFS2
10.11.11.1/30 WAN Interface Dummy IPI pointed this out before, but you need to have them in the same subnet. Why not try 10.10.10.1/30 and 10.10.10.2/30 ??
Good question. I tried it before and it didn't make a difference. I will try it again.
-
MY SIDE CONFIGURATION
PFS1
10.10.10.1/30 WAN Interface Dummy IPPFS2
10.11.11.1/30 WAN Interface Dummy IPI pointed this out before, but you need to have them in the same subnet. Why not try 10.10.10.1/30 and 10.10.10.2/30 ??
Just tried it again and same problem.
-
Again - pretty sparse with the details.
-
Again - pretty sparse with the details.
Again, what more details do you need??? I asked what details you need before. I'll provide the details needed.
-
You are still not providing the necessary details to properly help you diagnose your problem. Run through the connectivity troubleshooting steps and say what actually fails and where.
Help us help you.
-
You are still not providing the necessary details to properly help you diagnose your problem. Run through the connectivity troubleshooting steps and say what actually fails and where.
Help us help you.
I have no idea what details you are looking for. All I know right now is this:
- I start ping command on server on both ends
- Unplug Master Firewall WAN connection
- I see Slave Firewall takes over when a packet is dropped and ping resumes
- Right after that I plug the Master Firewall WAN connection
- Master Firewall takes over but all pings fail
-
Details like what are you pinging from where when you're testing. Specifics, like interfaces and IP addresses.
-
I am pinging 77.77.77.40 from 66.66.77.10. I am also pinging 66.66.77.10 from 77.77.77.40.
Diagram of setup is attached. FYI, this is a closed lab setup.
-
What address is 77.77.77.1/X routed to on 66.66.66 ?
-
-
How do the 77.77.77.X addresses get from the ISP to you? They have to be routed to you somehow.
-
How do the 77.77.77.X addresses get from the ISP to you? They have to be routed to you somehow.
If you are asking me where the 77.77.77.0/24 addresses come from, that does not matter because this is a lab designed for testing only with no real connection to the Internet. It is only for testing PFS HA functionality.
If you are asking what am I using to route IP traffic between 66.66.66.0/30 and 77.77.77.0/24, I am using a separate installation of PFS.
-
OK it doesn't matter if it is routed to the CARP VIP or not. I'm done.
-
OK it doesn't matter if it is routed to the CARP VIP or not. I'm done.
That's fine. You weren't reading the details, exaggerating information needed and making things more difficult than they really are.
