My Adventures of a Working(ish) Azure HA pfSense Deployment

io

Hi, just wanting to share my experiences/setup in case it helps someone down the road. I'll format better if there is any actual interest in this/discussion. Would be great if anyone has additional insight on why my additional UDR step works and if anyone had success with utilizing HA Ports Load Balancing on Azure.

Some disclaimers:

I am self taught, so some of this may not be (most likely) valid/correct (please feel free to pitch in on suggestions/comments/etc.)
Working(ish) = active/active with replication between the two pfSense instances working (looks like state synchronization works too… which is surprising to me). However, I was not able to get Azure internal load balancer (preview - HA Ports) to work to get a true active active setup :( ... meaning all traffic hits one pfSense box only currently =P... but at least my other one is synced lol)
Overall setup is 2 pfSense servers {2.4.1} with 4 nic deployment. Both servers are running open vpn.

================================================

The Beginning
I started with this project prior to the official Azure deployment availability on marketplace. I download the pfSense ISO and booted up and installed utilizing hyper-v. Virtual Box would not work for me as NIC's need to be named hn0 for Azure to behave properly. I configured only a WAN adapter. After basic configuration, I uploaded the vhd to Azure and created a vm from this image (adding four nics via PowerShell). As a note, these are deployed to an availability set.

As time went by, newer versions of pfSense came out, and I updated via the GUI.

Moving from single nic to four nics via the UI
pfSense gui detected all the nics I created without a hitch. They are set to unassigned state initially. One important lesson I learned is after assigning my nics, I would end up getting locked out of the web gui. Early stages, simply set a pass all TCP rule on all interfaces (tweak later).

NICs are set to DHCP to pull azure ip. Remember to set all nic's to static.

my four nics are:
172.20.2.10 (ngfwwan) | box 2 is 2.11
172.20.3.10 (ngfwdmz) ***not utilizing currently | box 2 is 3.11
172.20.4.10 (ngfwsync) | box 2 is 4.11
172.20.5.10 (ngfwlan) | box 2 is 5.11

OpenVPN
I set OpenVPN to run on LAN. Port forward 1194 on WAN to localhost. Azure NSG needs to allow 1194 UDP to the WAN adapter private ip. Client pool is 172.18.0.0/24

Alias
I created an internet alias – I imported the IP range for the internet from Azure's listing.

Outbound Nat (both pfSense)
I set to manual outbound NAT.
127.0.0.1 , , to WAN
172.18.0.0/24, *, * to WAN,
172.20.6.0/24 INTERNET to WAN
172.20.7.0/24 INTERNET to WAN

Static Routes (both pfSense)
create a new gateway called lan_gw

set the ip to 172.20.5.1 , turn off monitoring

for your static routes 172.20.6.0/24 , utilize LAN gw & 172.20.7.0/24 utilize LAN gw.

Vnet
NGFW-WAN : 172.20.2.0/24
NGFW-DMZ : 172.20.3.0/24
NGFW-SYNC : 172.20.4.0/24
NGFW-LAN : 172.20.5.0/24

DMZ : 172.20.6.0/24
WEB : 172.20.7.0/24

UDR
DMZ_UDR (apply to DMZ subnet):
172.20.6.0/24 next hop virtual appliance 172.20.5.10
0.0.0.0/0 next hop virtual appliance 172.20.5.10

WEB_UDR (apply to WEBsubnet):
172.20.7.0/24 next hop virtual appliance 172.20.5.10
0.0.0.0/0 next hop virtual appliance 172.20.5.10

LAN_UDR (apply to ngfw-lan subnet):
0.0.0.0/0 next hop virtual appliance 172.20.2.10
****Note, none of the tutorials I found or posts I found did this with UDR… I'm not sure why this worked for me, but this is the only way I can get the VM's behind pfSense to actually be able to get out to do apt-get updates etc.

setup ha
both firewall
sync adapter firewall rules (both firewalls): tcp source sync net * * * *

box 1, system ha

check mark sync status, select sync interface
peer ip = 172.20.4.11

sync config to ip ( 172.20.4.11 ) , admin , password of pfSense2 box
check mark what makes sense ( i did first 6 + openvpn)

box 2, system ha,
check mark sync status, select sync interface
peer ip = 172.20.4.10

do not fill out anything else

future
What would be ideal is to have a HA Port load balancer balancing both lan nics (172.20.5.10 & 5.11). dmz udr would instead hop to the load balancer ip.

I need to figure out why the 0.0.0.0/0 hop to my wan adapter is the only way I can get this thing to work.

bwlinux

Thanks for the post. I have to implement something like this and well…. I haven't even created my first 2 Nic pfSense instance from the az CLI yet.

Just glad to know what I want to do is reasonably possible.

iamthewall 0

Hi io,

Hope you can reach me even that this post its quite old. I ran on the same “adventures” as you did. Even that I got the ha configuration on both firewalls I have not been able to show them as a cluster where they share a unique public ip. I have tried to set up an azure load balancer to do so and al the traffic is managed by one of the nodes but when I turn that node off, the ipsec set ups that I have to onpremise are not working. Have you tried to do this set up as well?