Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    pfSense HA LAN Interfaces Only

    Scheduled Pinned Locked Moved
    HA/CARP/VIPs
    2
    91
    16.6k
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • C
      CloudNode @viragomann
      last edited by

      @viragomann I removed all the settings we added, rebooted 2nd node and re-added all the settings as per above and still no go.. I am not able to see the ping req from master packet cap now even when sending a ping from 2nd node. But on the master wan, i can see the 1/1 request which shows that in the ping log on the 2nd node as well 1/3 goes

      1 Reply Last reply Reply Quote 0
      • C
        CloudNode @viragomann
        last edited by CloudNode

        @viragomann I think were on to something... I just did this and i was able to ping 1.1.1.1 3/3 and my 2nd node now has internet..

        As a test, i change the Outbound NAT rule from Source 127.0.0.0/8 to any..

        e5aec209-4082-46ee-8e3d-2bf1a108c88d-image.png

        ad265028-ad59-4de9-a151-3f24b3ee2b60-image.png

        e5913b5b-8467-477e-babe-efb55555e395-image.png bolded text

        But.. I know this is not the correct way to leave it; what do you think the issue was with 127.0.0.0/8 as the source?

        V 1 Reply Last reply Reply Quote 0
        • V
          viragomann @CloudNode
          last edited by

          @iptvcld said in pfSense HA LAN Interfaces Only:

          I think were on to something... I just did this and i was able to ping 1.1.1.1 3/3 and my 2nd node now has internet..

          Interestingly. 🙂

          I know this is not the correct way to leave it; what do you think the issue was with 127.0.0.0/8 as the source?

          It's not really a good idea to have any source natted, at least if you have incoming connections.
          Imagine you have a port forwarding to your web server. So packets form the IP 1.2.3.4 is forwarded to the internal server. However, since pfSense translated the source into its LAN address, the web server sees the packets coming from pfSense and you're not able to determine the real source address.

          Don't know, why 127.0.0.0/8 doesn't work here as source. But as a workaround you can try to set the destination to non-RFC1918 networks (add a proper alias first). So you rule will only be applied to packets which go to the internet.

          C 1 Reply Last reply Reply Quote 0
          • C
            CloudNode @viragomann
            last edited by

            @viragomann
            This is very odd for sure!
            Even when i change the source to This Firewall - the internet works..
            70b291c9-25bb-472d-8769-5f035ed3675c-image.png

            So for your workaround you suggest to put the source back to 127.0.0.0/8 and set the destination to non-RFC1918 via Alias?

            Would this still be a good solution if it works? As in, would it cause the issues you noted above?

            V 1 Reply Last reply Reply Quote 0
            • V
              viragomann @CloudNode
              last edited by

              @iptvcld said in pfSense HA LAN Interfaces Only:

              Even when i change the source to This Firewall - the internet works..

              Nice! That's all you need at all.
              Didn't think of this option.

              C 2 Replies Last reply Reply Quote 0
              • C
                CloudNode @viragomann
                last edited by CloudNode

                @viragomann
                Amazing Sir... I left it as This Firewall for the source and internet is up and running on backup node!!!

                Later tonight, I will try swinging the wan over to make sure the tier 1 GW Group takes over as i currently have the group action trigger set to Link Down.

                1 Reply Last reply Reply Quote 0
                • C
                  CloudNode @viragomann
                  last edited by

                  @viragomann
                  I just tested this evening and everything works as i visioned!

                  LAN interfaces are using CARP and what i did was shutdown master node and my backup node became master for the carp LAN interfaces.. Great.. I then saw the GW from master go offline (which is good). I then swang the WAN cable over from master to backup node and right away WAN interface went UP and my GW changed to the PPPOE WAN Tier 1!! which was great.

                  I then powered on master node and saw my carp LAN interfaces switch back to the master node and once i swang my wan cable back to the master pf node, link went up there and also noticed on my backup node, the GW from master went ONLINE and internet access was flowing!!

                  One thing i noticed which may be normal after my CARP was active on my backup pf node for a short period before my master taking over again, but I noticed some of devices are showing the DHCP server IP of my backup node when i run ipconfig /all and if i do a release and renew, it will jump back to my master pf node. Is this normal and does this effect anything?

                  Thank you for all your great work, I have learned ALOT!!!

                  V 1 Reply Last reply Reply Quote 0
                  • V
                    viragomann @CloudNode
                    last edited by

                    @iptvcld
                    Did you set the "Failover peer IP" in the DHCP settings on both nodes?

                    The DHCP leases are bound to the hardware interfaces. So the interfaces on both have to have the same name. If you have different hardware or one bare metal installation and one virtual you can work around this with LAGG interfaces to abstract the hardware.

                    You can find some hints in the pfSense docs:
                    Modifying the DHCP Server
                    Troubleshooting High Availability DHCP Failover

                    C 1 Reply Last reply Reply Quote 0
                    • C
                      CloudNode @viragomann
                      last edited by

                      @viragomann
                      Morning..
                      Yes my enabled DHCP Server settings have my CARP VIP for the DNS, Gateway and for the Failover peer IP, i have it set to the interface IP of the backup node. (In the case my LAN interface IP) When i saved this, my backup automatically picked these settings up and applied the peer IP back to the master node IP.

                      As for the hardware, I have the same setup on both units.

                      Master Node Interfaces
                      LAN Interface (lan, lagg0)
                      IOT Interface (opt1, lagg0.10)
                      NOT Interface (opt2, lagg0.20)
                      SECURITYCAM Interface (opt3, lagg0.30)
                      VPN Interface (opt4, ovpns1)
                      SYNC Interface (opt5, em0)

                      Backup Node Interfaces
                      LAN Interface (lan, lagg0)
                      IOT Interface (opt1, lagg0.10)
                      NOT Interface (opt2, lagg0.20)
                      SECURITYCAM Interface (opt3, lagg0.30)
                      VPN Interface (opt4, ovpns1)
                      SYNC Interface (opt5, em0)

                      Status -> DHCP Leases on my nodes show My State: normal and Peer State: normal

                      I checked Leases on my backup node under status - DHCP Leases and i see a bunch that say online active next to them - and these are also online active showing on the master node.

                      Then i also see a bunch as offline / active on my backup service and online / active on my master

                      Not sure if this is normal operation for the dhcp leases to be hand on hand with each other.

                      V 1 Reply Last reply Reply Quote 0
                      • V
                        viragomann @CloudNode
                        last edited by

                        @iptvcld
                        Your settings look well and should work this way.

                        Then i also see a bunch as offline / active on my backup service and online / active on my master

                        I don't use any DHCP on CARP set ups at this time, so I cannot verify.
                        However, "offline" means that the machine holding the lease is actually not present in the pfSense ARP table. This seems normal as the machines doesn't communicate with the backup node in normal usage.
                        After you try to access the backup from one of the offline machines (e.g. ping its IP) it's state should become online.

                        C 1 Reply Last reply Reply Quote 0
                        • C
                          CloudNode @viragomann
                          last edited by

                          @viragomann
                          Not a problem Sir! Might be normal operation as right now i just noticed my cell phone showing online/active on both master and backup nodes (same showing) Check my phone network info and i can see it has a dhcp IP of my backup node. Again not sure if this is the way it should be working.

                          But i will post a new thread if i can not find anything on other forums as well.

                          V 1 Reply Last reply Reply Quote 0
                          • V
                            viragomann @CloudNode
                            last edited by

                            @iptvcld
                            I think, the important part is the lease state. Both nodes must be aware of all active DHCP leases. So that used IPs cannot be assigned a second time by the other machine. And that's the case as you mentioned.
                            Then it doesn't matter which node has issued the lease.

                            C 2 Replies Last reply Reply Quote 0
                            • C
                              CloudNode @viragomann
                              last edited by

                              @viragomann
                              Yes seems like this is the case
                              Primary
                              84d2b583-a504-455d-8f7b-b30fe75343b8-image.png

                              Secondary
                              88eca05d-4f4e-444e-8989-6c3048b43ec0-image.png

                              Primary
                              0255158e-bff6-4847-8e9c-d8dadd61d720-image.png

                              Secondary
                              78589834-9df8-4e91-bed0-1728db0bdc09-image.png

                              V 1 Reply Last reply Reply Quote 0
                              • V
                                viragomann @CloudNode
                                last edited by

                                @iptvcld
                                Out of curiosity, which devices are belonging to your NOT subnet?

                                C 1 Reply Last reply Reply Quote 0
                                • C
                                  CloudNode @viragomann
                                  last edited by

                                  @viragomann
                                  That is my Network of Things VLAN which I have Smart switch devices such as Tasmota bulbs/switches that i dont want them to reach out to the internet or other devices on my lan. They are all internal controlled/accessed devices.

                                  V 1 Reply Last reply Reply Quote 0
                                  • V
                                    viragomann @CloudNode
                                    last edited by

                                    @iptvcld said in pfSense HA LAN Interfaces Only:

                                    That is my Network of Things VLAN which I have Smart switch devices such as Tasmota bulbs/switches that i dont want them to reach out to the internet or other devices on my lan.

                                    Ahh. I don't have such devices in my network. All I have want to access at least internet.

                                    C 1 Reply Last reply Reply Quote 0
                                    • C
                                      CloudNode @viragomann
                                      last edited by

                                      @viragomann
                                      For that i have the Internet of Things VLAN which those devices have internet access but cannot talk to other vlans/networks on my LAN (inter-chatter)

                                      V 1 Reply Last reply Reply Quote 0
                                      • V
                                        viragomann @CloudNode
                                        last edited by

                                        @iptvcld
                                        Yes, I have an IOT subnet as well. On this only access to none-RFC1918 is allowed.

                                        C 1 Reply Last reply Reply Quote 0
                                        • C
                                          CloudNode @viragomann
                                          last edited by

                                          @viragomann said in pfSense HA LAN Interfaces Only:

                                          none-RFC1918

                                          I have this - i guess pretty much the same; IOT can talk to each other on the same vlan but cannot chat to others outside of IOT including the firewall it self

                                          9b9c3513-ee03-4e2f-b86f-521cd34dad85-image.png

                                          19b9c008-0ad3-4393-9710-8b72afd5e845-image.png

                                          V 1 Reply Last reply Reply Quote 0
                                          • V
                                            viragomann @CloudNode
                                            last edited by

                                            @iptvcld
                                            I use an RFC1918 alias on pfSense which simply includes all private networks (10.0.0.0/8, 172.16.0.0/12, 192.168.0.0/16).
                                            So I'm still save, when I add or change a subnet without the need of modifying the alias.

                                            C 1 Reply Last reply Reply Quote 1
                                            • First post
                                              Last post