My Adventures of a Working(ish) Azure HA pfSense Deployment



  • Hi, just wanting to share my experiences/setup in case it helps someone down the road.  I'll format better if there is any actual interest in this/discussion.  Would be great if anyone has additional insight on why my additional UDR step works and if anyone had success with utilizing HA Ports Load Balancing on Azure.

    Some disclaimers:

    1. I am self taught, so some of this may not be (most likely) valid/correct (please feel free to pitch in on suggestions/comments/etc.)
    2. Working(ish) = active/active with replication between the two pfSense instances working (looks like state synchronization works too… which is surprising to me).  However, I was not able to get Azure internal load balancer (preview - HA Ports) to work to get a true active active setup :( ... meaning all traffic hits one pfSense box only currently =P... but at least my other one is synced lol)
    3. Overall setup is 2 pfSense servers {2.4.1} with 4 nic deployment. Both servers are running open vpn.

    ================================================

    The Beginning
    I started with this project prior to the official Azure deployment availability on marketplace. I download the pfSense ISO and booted up and installed utilizing hyper-v.  Virtual Box would not work for me as NIC's need to be named hn0 for Azure to behave properly. I configured only a WAN adapter.  After basic configuration, I uploaded the vhd to Azure and created a vm from this image (adding four nics via PowerShell).  As a note, these are deployed to an availability set.

    As time went by, newer versions of pfSense came out, and I updated via the GUI.

    Moving from single nic to four nics via the UI
    pfSense gui detected all the nics I created without a hitch.  They are set to unassigned state initially.  One important lesson I learned is after assigning my nics, I would end up getting locked out of the web gui.  Early stages, simply set a pass all TCP rule on all interfaces (tweak later).

    NICs are set to DHCP to pull azure ip.  Remember to set all nic's to static.

    my four nics are:
    172.20.2.10 (ngfwwan)  | box 2 is 2.11
    172.20.3.10 (ngfwdmz) ***not utilizing currently  | box 2 is 3.11
    172.20.4.10 (ngfwsync)  | box 2 is 4.11
    172.20.5.10 (ngfwlan)  | box 2 is 5.11

    OpenVPN
    I set OpenVPN to run on LAN. Port forward 1194 on WAN to localhost.  Azure NSG needs to allow 1194 UDP to the WAN adapter private ip. Client pool is 172.18.0.0/24

    Alias
    I created an internet alias – I imported the IP range for the internet from Azure's listing.

    Outbound Nat (both pfSense)
    I set to manual outbound NAT.
    127.0.0.1 , , to WAN
    172.18.0.0/24, *, * to WAN,
    172.20.6.0/24 INTERNET to WAN
    172.20.7.0/24 INTERNET to WAN

    Static Routes (both pfSense)
    create a new gateway called lan_gw

    set the ip to 172.20.5.1 , turn off monitoring

    for your static routes 172.20.6.0/24 , utilize LAN gw & 172.20.7.0/24 utilize LAN gw.

    Vnet
    NGFW-WAN : 172.20.2.0/24
    NGFW-DMZ : 172.20.3.0/24
    NGFW-SYNC : 172.20.4.0/24
    NGFW-LAN : 172.20.5.0/24

    DMZ : 172.20.6.0/24
    WEB : 172.20.7.0/24

    UDR
    DMZ_UDR (apply to DMZ subnet):
    172.20.6.0/24 next hop virtual appliance 172.20.5.10
    0.0.0.0/0 next hop virtual appliance 172.20.5.10

    WEB_UDR (apply to WEBsubnet):
    172.20.7.0/24 next hop virtual appliance 172.20.5.10
    0.0.0.0/0 next hop virtual appliance 172.20.5.10

    LAN_UDR (apply to ngfw-lan subnet):
    0.0.0.0/0 next hop virtual appliance 172.20.2.10
    ****Note, none of the tutorials I found or posts I found did this with UDR… I'm not sure why this worked for me, but this is the only way I can get the VM's behind pfSense to actually be able to get out to do apt-get updates etc.

    setup ha
    both firewall
    sync adapter firewall rules (both firewalls): tcp source sync net * * * *

    box 1, system ha

    check mark sync status, select sync interface
    peer ip = 172.20.4.11

    sync config to ip ( 172.20.4.11 ) , admin , password of pfSense2 box
    check mark what makes sense ( i did first 6 + openvpn)

    box 2, system ha,
    check mark sync status, select sync interface
    peer ip = 172.20.4.10

    do not fill out anything else

    future
    What would be ideal is to have a HA Port load balancer balancing both lan nics (172.20.5.10 & 5.11).  dmz udr would instead hop to the load balancer ip.

    I need to figure out why the 0.0.0.0/0 hop to my wan adapter is the only way I can get this thing to work.



  • Thanks for the post.  I have to implement something like this and well…. I haven't even created my first 2 Nic pfSense instance from the az CLI yet.

    Just glad to know what I want to do is reasonably possible.



  • Hi io,

    Hope you can reach me even that this post its quite old. I ran on the same “adventures” as you did. Even that I got the ha configuration on both firewalls I have not been able to show them as a cluster where they share a unique public ip. I have tried to set up an azure load balancer to do so and al the traffic is managed by one of the nodes but when I turn that node off, the ipsec set ups that I have to onpremise are not working. Have you tried to do this set up as well?


 

© Copyright 2002 - 2018 Rubicon Communications, LLC | Privacy Policy