Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    pfSense HA LAN Interfaces Only

    Scheduled Pinned Locked Moved
    HA/CARP/VIPs
    2
    91
    16.6k
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • V
      viragomann @CloudNode
      last edited by

      @iptvcld said in pfSense HA LAN Interfaces Only:

      Secondary Node: Created a GW Group as PPPOE WAN Tier1, Above noted GW as Tier 2 and VPN GW as Tier 3

      The VPN GW has to be set to "never", so that it is no member of this gateway group.

      All other settings should be correct.

      When you run a ping with 3 requests to 1.1.1.1 on the secondary the packets should go out to the masters LAN and be forwarded to the internet, since the gw monitoring has set the static route for this IP.

      When you sniff the traffic filter for ICMP and 1.1.1.1, you should see 3 ICMP request packets (and 3 responds if it works) on

      • both masters and secondarys LAN: 192.168.2.81 > 1.1.1.1
      • on the masters WAN: WAN IP > 1.1.1.1

      If you look into the masters state table you should see

      • one entry for LAN: 192.168.2.81 > 1.1.1.1
      • and one for WAN: WAN IP > 1.1.1.1

      Both should show 3/3 in the packets column.

      Not clear, why it doesn't behave this way on your setup.

      C 2 Replies Last reply Reply Quote 0
      • C
        CloudNode @viragomann
        last edited by

        @viragomann I removed all the settings we added, rebooted 2nd node and re-added all the settings as per above and still no go.. I am not able to see the ping req from master packet cap now even when sending a ping from 2nd node. But on the master wan, i can see the 1/1 request which shows that in the ping log on the 2nd node as well 1/3 goes

        1 Reply Last reply Reply Quote 0
        • C
          CloudNode @viragomann
          last edited by CloudNode

          @viragomann I think were on to something... I just did this and i was able to ping 1.1.1.1 3/3 and my 2nd node now has internet..

          As a test, i change the Outbound NAT rule from Source 127.0.0.0/8 to any..

          e5aec209-4082-46ee-8e3d-2bf1a108c88d-image.png

          ad265028-ad59-4de9-a151-3f24b3ee2b60-image.png

          e5913b5b-8467-477e-babe-efb55555e395-image.png bolded text

          But.. I know this is not the correct way to leave it; what do you think the issue was with 127.0.0.0/8 as the source?

          V 1 Reply Last reply Reply Quote 0
          • V
            viragomann @CloudNode
            last edited by

            @iptvcld said in pfSense HA LAN Interfaces Only:

            I think were on to something... I just did this and i was able to ping 1.1.1.1 3/3 and my 2nd node now has internet..

            Interestingly. 🙂

            I know this is not the correct way to leave it; what do you think the issue was with 127.0.0.0/8 as the source?

            It's not really a good idea to have any source natted, at least if you have incoming connections.
            Imagine you have a port forwarding to your web server. So packets form the IP 1.2.3.4 is forwarded to the internal server. However, since pfSense translated the source into its LAN address, the web server sees the packets coming from pfSense and you're not able to determine the real source address.

            Don't know, why 127.0.0.0/8 doesn't work here as source. But as a workaround you can try to set the destination to non-RFC1918 networks (add a proper alias first). So you rule will only be applied to packets which go to the internet.

            C 1 Reply Last reply Reply Quote 0
            • C
              CloudNode @viragomann
              last edited by

              @viragomann
              This is very odd for sure!
              Even when i change the source to This Firewall - the internet works..
              70b291c9-25bb-472d-8769-5f035ed3675c-image.png

              So for your workaround you suggest to put the source back to 127.0.0.0/8 and set the destination to non-RFC1918 via Alias?

              Would this still be a good solution if it works? As in, would it cause the issues you noted above?

              V 1 Reply Last reply Reply Quote 0
              • V
                viragomann @CloudNode
                last edited by

                @iptvcld said in pfSense HA LAN Interfaces Only:

                Even when i change the source to This Firewall - the internet works..

                Nice! That's all you need at all.
                Didn't think of this option.

                C 2 Replies Last reply Reply Quote 0
                • C
                  CloudNode @viragomann
                  last edited by CloudNode

                  @viragomann
                  Amazing Sir... I left it as This Firewall for the source and internet is up and running on backup node!!!

                  Later tonight, I will try swinging the wan over to make sure the tier 1 GW Group takes over as i currently have the group action trigger set to Link Down.

                  1 Reply Last reply Reply Quote 0
                  • C
                    CloudNode @viragomann
                    last edited by

                    @viragomann
                    I just tested this evening and everything works as i visioned!

                    LAN interfaces are using CARP and what i did was shutdown master node and my backup node became master for the carp LAN interfaces.. Great.. I then saw the GW from master go offline (which is good). I then swang the WAN cable over from master to backup node and right away WAN interface went UP and my GW changed to the PPPOE WAN Tier 1!! which was great.

                    I then powered on master node and saw my carp LAN interfaces switch back to the master node and once i swang my wan cable back to the master pf node, link went up there and also noticed on my backup node, the GW from master went ONLINE and internet access was flowing!!

                    One thing i noticed which may be normal after my CARP was active on my backup pf node for a short period before my master taking over again, but I noticed some of devices are showing the DHCP server IP of my backup node when i run ipconfig /all and if i do a release and renew, it will jump back to my master pf node. Is this normal and does this effect anything?

                    Thank you for all your great work, I have learned ALOT!!!

                    V 1 Reply Last reply Reply Quote 0
                    • V
                      viragomann @CloudNode
                      last edited by

                      @iptvcld
                      Did you set the "Failover peer IP" in the DHCP settings on both nodes?

                      The DHCP leases are bound to the hardware interfaces. So the interfaces on both have to have the same name. If you have different hardware or one bare metal installation and one virtual you can work around this with LAGG interfaces to abstract the hardware.

                      You can find some hints in the pfSense docs:
                      Modifying the DHCP Server
                      Troubleshooting High Availability DHCP Failover

                      C 1 Reply Last reply Reply Quote 0
                      • C
                        CloudNode @viragomann
                        last edited by

                        @viragomann
                        Morning..
                        Yes my enabled DHCP Server settings have my CARP VIP for the DNS, Gateway and for the Failover peer IP, i have it set to the interface IP of the backup node. (In the case my LAN interface IP) When i saved this, my backup automatically picked these settings up and applied the peer IP back to the master node IP.

                        As for the hardware, I have the same setup on both units.

                        Master Node Interfaces
                        LAN Interface (lan, lagg0)
                        IOT Interface (opt1, lagg0.10)
                        NOT Interface (opt2, lagg0.20)
                        SECURITYCAM Interface (opt3, lagg0.30)
                        VPN Interface (opt4, ovpns1)
                        SYNC Interface (opt5, em0)

                        Backup Node Interfaces
                        LAN Interface (lan, lagg0)
                        IOT Interface (opt1, lagg0.10)
                        NOT Interface (opt2, lagg0.20)
                        SECURITYCAM Interface (opt3, lagg0.30)
                        VPN Interface (opt4, ovpns1)
                        SYNC Interface (opt5, em0)

                        Status -> DHCP Leases on my nodes show My State: normal and Peer State: normal

                        I checked Leases on my backup node under status - DHCP Leases and i see a bunch that say online active next to them - and these are also online active showing on the master node.

                        Then i also see a bunch as offline / active on my backup service and online / active on my master

                        Not sure if this is normal operation for the dhcp leases to be hand on hand with each other.

                        V 1 Reply Last reply Reply Quote 0
                        • V
                          viragomann @CloudNode
                          last edited by

                          @iptvcld
                          Your settings look well and should work this way.

                          Then i also see a bunch as offline / active on my backup service and online / active on my master

                          I don't use any DHCP on CARP set ups at this time, so I cannot verify.
                          However, "offline" means that the machine holding the lease is actually not present in the pfSense ARP table. This seems normal as the machines doesn't communicate with the backup node in normal usage.
                          After you try to access the backup from one of the offline machines (e.g. ping its IP) it's state should become online.

                          C 1 Reply Last reply Reply Quote 0
                          • C
                            CloudNode @viragomann
                            last edited by

                            @viragomann
                            Not a problem Sir! Might be normal operation as right now i just noticed my cell phone showing online/active on both master and backup nodes (same showing) Check my phone network info and i can see it has a dhcp IP of my backup node. Again not sure if this is the way it should be working.

                            But i will post a new thread if i can not find anything on other forums as well.

                            V 1 Reply Last reply Reply Quote 0
                            • V
                              viragomann @CloudNode
                              last edited by

                              @iptvcld
                              I think, the important part is the lease state. Both nodes must be aware of all active DHCP leases. So that used IPs cannot be assigned a second time by the other machine. And that's the case as you mentioned.
                              Then it doesn't matter which node has issued the lease.

                              C 2 Replies Last reply Reply Quote 0
                              • C
                                CloudNode @viragomann
                                last edited by

                                @viragomann
                                Yes seems like this is the case
                                Primary
                                84d2b583-a504-455d-8f7b-b30fe75343b8-image.png

                                Secondary
                                88eca05d-4f4e-444e-8989-6c3048b43ec0-image.png

                                Primary
                                0255158e-bff6-4847-8e9c-d8dadd61d720-image.png

                                Secondary
                                78589834-9df8-4e91-bed0-1728db0bdc09-image.png

                                V 1 Reply Last reply Reply Quote 0
                                • V
                                  viragomann @CloudNode
                                  last edited by

                                  @iptvcld
                                  Out of curiosity, which devices are belonging to your NOT subnet?

                                  C 1 Reply Last reply Reply Quote 0
                                  • C
                                    CloudNode @viragomann
                                    last edited by

                                    @viragomann
                                    That is my Network of Things VLAN which I have Smart switch devices such as Tasmota bulbs/switches that i dont want them to reach out to the internet or other devices on my lan. They are all internal controlled/accessed devices.

                                    V 1 Reply Last reply Reply Quote 0
                                    • V
                                      viragomann @CloudNode
                                      last edited by

                                      @iptvcld said in pfSense HA LAN Interfaces Only:

                                      That is my Network of Things VLAN which I have Smart switch devices such as Tasmota bulbs/switches that i dont want them to reach out to the internet or other devices on my lan.

                                      Ahh. I don't have such devices in my network. All I have want to access at least internet.

                                      C 1 Reply Last reply Reply Quote 0
                                      • C
                                        CloudNode @viragomann
                                        last edited by

                                        @viragomann
                                        For that i have the Internet of Things VLAN which those devices have internet access but cannot talk to other vlans/networks on my LAN (inter-chatter)

                                        V 1 Reply Last reply Reply Quote 0
                                        • V
                                          viragomann @CloudNode
                                          last edited by

                                          @iptvcld
                                          Yes, I have an IOT subnet as well. On this only access to none-RFC1918 is allowed.

                                          C 1 Reply Last reply Reply Quote 0
                                          • C
                                            CloudNode @viragomann
                                            last edited by

                                            @viragomann said in pfSense HA LAN Interfaces Only:

                                            none-RFC1918

                                            I have this - i guess pretty much the same; IOT can talk to each other on the same vlan but cannot chat to others outside of IOT including the firewall it self

                                            9b9c3513-ee03-4e2f-b86f-521cd34dad85-image.png

                                            19b9c008-0ad3-4393-9710-8b72afd5e845-image.png

                                            V 1 Reply Last reply Reply Quote 0
                                            • First post
                                              Last post