Pfsense HA with vm and physical box setup help
-
Hello
I have a netgate sg3100 which has been working for me. However, I have a homelab box with better specs. I am looking to setup a proxmox vm for pfsense and use the sg3100 in a HA back mode. I am confused about how to go about the order of things:
-
setup the vm as the master and replicate all config / rules there first? what is the fastest way to do this ?
-
then, wipe the sg3100 and setup that up as the backup? Do i need to wipe the back up before setting it up in HA mode?
-
use the default linux bridge or use ovs bridge? which one is better?
Any thing else I should look out for? Thanks
-
-
hello
Here is a diagram with a bit more detail about the equipment specs. Can somehow help with how to connect the virtualized pfsense instance on the homelab as master and the physical pfsense box as the redundant machine? I still cant figure out the connections.
-
Read through https://docs.netgate.com/pfsense/en/latest/book/highavailability/index.html and https://docs.netgate.com/pfsense/en/latest/book/highavailability/example-redundant-configuration.html along with the rest of the HA section. The second link has a diagram.
Configuration syncing will copy all changes from the master to the backup.
State syncing is only going to work if the interface assignments are the same: https://docs.netgate.com/pfsense/en/latest/book/highavailability/pfsync-overview.html#pfsync-and-physical-interfaces
-
Hello - I have read through all those links a couple times. I am still not sure how to connect the above items. For instance, in the diagrams provided on the links, they show the internet connection coming in through the isp router, then a wan switch , before hitting the primary pfsense box. I am looking to use one of the ports on the home lab server as my WAN port, then on from there to the other network devices. In my current connection, I have the pfsense box in front of the isp router. ( fios )
-
Do you have 3 public IPs? Not sure how you think your going to setup HA when you don't have 3 public IPs... If your going to set it up behind a nat - which you can do..
What in your drawing is going to do the nat? Where you internet comes in is that natted by some isp device?
And your going to need switch where you can put the wans of both firewalls on the same L2..
I don't see the point of setting up HA, other than letting you reboot your VM host and not loose internet.. So why not just not run your firewall on your esxi host? And just use the 3100?
-
Thanks for the reply. I don't have 3 public ips. I came across a few posts which seem to indicate that it was possible to use a single public ip with some caveats - e.g having the slave without an internet connection.
To answer your question about nat - In my current setup, I have my pfsense box in front of my isp router - the pfsense box does all the nat ( a cat5 directly from the ont to the pfsense box). The isp router is in bridge mode - no double natting. It gets a lan ip address downstream from the pfsense.
so in my drawing, I was going to use my homelab server, with the master pfsense vm as my nat device. I also have two switches I can use for this setup.
But all this is a moot point if I can't do this with a single public ip.
I am looking to do this because, I didnt want the sg3100 just sitting there after moving my pfsense duties to the vm on the lab server. Seems its a great learning experience as well to see if I can set it up as a hot spare.
Can you elaborate on your note. .. "run your firewall on your esxi host ...and just use the 3100" . What does that mean? Thanks
-
@vacquah said in Pfsense HA with vm and physical box setup help:
e.g having the slave without an internet connection.
What would be the point - where did you read this?
If you no longer want to use your 3100, leave it on the shelf as a spare.. Or sell it.
You can use rfc1918 addresses and the carp is the only public one.. If you really want to... But I don't really see what you think your going to get out of this other than the ability to reboot your vm host and still have internet.. So the spare would have to be connected to the wan L2 network..
If you have your isp device with only 1 port, that would have to be connected to a switch, so you could put the wans of both your pfsense installs on the same wan L2, even if only the carp is public IP.
-
@vacquah said in Pfsense HA with vm and physical box setup help:
possible to use a single public ip with some caveats
It is, as John mentioned, but yes caveats. For instance when Comcast bridges their router NAT still works for their default 10.1.10.x subnet, so we've set up at least one client I know of with WAN IPs of 10.1.10.2 on one, 10.1.10.3 on the other, and the CARP WAN aliases are their public IPs, while both have 192.168.0.1 as their CARP LAN alias and gateway.
You might also be able to set the ISP router's DMZ as an RFC1918 CARP alias IP, and then use a different private subnet for the LAN. I don't think we've tried that.
-
@teamits thanks for your input. As I said, it is a learning experience for me - better than just putting the sg3100 on the shelf. I'd appreciate if you can help me setup the physical connections, then I will dive in to experiment, per your suggestions. Many thanks.
-
The physical setup has been gone over an over.. There is plenty of documentation how to setup HA..
Not sure what else you want - this is meant to be a learning experience you say... How is someone holding your hand saying click this now that going to help you learn anything..
What specific questions do you have about how HA is setup.. What have you tried to do to set it up as described in the docs.. If you don't know how to do something on your VM software of choice - you better off asking their forums on how you bridge and interface or setup a vlan port group or pass tags, etc.
HAs setup is not rocket science or anything - there is a wan L2, there is a pfSync network L2 and there is a Lan L2.. The interfaces for your 2 pfsense will be in each L2..
-
@vacquah said in Pfsense HA with vm and physical box setup help:
For instance, in the diagrams provided on the links, they show the internet connection coming in through the isp router, then a wan switch , before hitting the primary pfsense box. I am looking to use one of the ports on the home lab server as my WAN port, then on from there to the other network devices. In my current connection, I have the pfsense box in front of the isp router. ( fios )
fyi
-
The WAN port from the VM needs to connect to the ISP router.
The WAN port from the 3100 needs to connect to the ISP router.
If the ISP router has only one port you'll need a switch.The LAN ports from both connect to your LAN switch.
Once you have the configuration sync set up any change on the master will copy everything to the backup.
-
@teamits
Hello @teamits - does this assume the isp router is in front of both pfsense boxes ? So , Ont —> isp router —> wan interface of both pfsense machines ?
I was looking to put the isp router behind the pfsense box. I am actually thinking of doing away with it entirely and replace it with a moca adapter.
I have a separate ethernet line from the ont going straight to my pfsense box in my current setup.
