Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    pfSense HA LAN Interfaces Only

    Scheduled Pinned Locked Moved
    HA/CARP/VIPs
    2
    91
    16.6k
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • V
      viragomann @CloudNode
      last edited by

      @iptvcld said in pfSense HA LAN Interfaces Only:

      I ran a packet cap again and i am seeing those monitoring requests come in as well:
      11:14:31.073869 IP 8.8.8.8.853 > 192.168.2.81.17018: tcp 0
      11:14:31.073887 IP 76.64.x.x.11762 > 8.8.8.8.853: tcp 0

      So you're using 8.8.8.8 for DNS resolution as well.
      Possibly that's an issue when you've stated the WAN gateway in System > General Setup.
      However, when using this IP for gateway monitoring, pfSense will automatically add a static route and point it to the respective gateway. So there might be a conflict.

      As mentioned above, you can use any other public IP for gateway monitoring which is responding to pings. It might be a good idea, to use another IP.

      Also you should have a check at System > Advanced > Miscellaneous > State Killing on Gateway Failure.

      For further testing you can set the WAN gateway as down manually on the backup.

      C 1 Reply Last reply Reply Quote 0
      • C
        CloudNode @viragomann
        last edited by

        @viragomann Correct; i am using 8.8.8.8 and 8.8.4.4 as DNS Res via system > gateway.

        I have now changed my GW monitoring IP to 1.1.1.1 on the secondary pf. Also on the secondary pf i have placed a CHECK under System > Advanced > Miscellaneous > State Killing on Gateway Failure.

        When i try to ping 8.8.8.8 from the secondary; i get this timeout

        PING 8.8.8.8 (8.8.8.8): 56 data bytes
92 bytes from 127.0.0.1: Time to live exceeded
Vr HL TOS  Len   ID Flg  off TTL Pro  cks      Src      Dst
 4  5  00 0054 263f   0 0000  01  01 0000 127.0.0.1  8.8.8.8 

92 bytes from 127.0.0.1: Time to live exceeded
Vr HL TOS  Len   ID Flg  off TTL Pro  cks      Src      Dst
 4  5  00 0054 b74f   0 0000  01  01 0000 127.0.0.1  8.8.8.8 

92 bytes from 127.0.0.1: Time to live exceeded
Vr HL TOS  Len   ID Flg  off TTL Pro  cks      Src      Dst
 4  5  00 0054 66ba   0 0000  01  01 0000 127.0.0.1  8.8.8.8 


--- 8.8.8.8 ping statistics ---
3 packets transmitted, 0 packets received, 100.0% packet loss

        I have marked my PPPOE WAN as down on the 2nd pf as well.
        This is so bizarre..
        I also have just an open rule for LAN
        6de4178c-6b2a-4d7d-ad19-2afa3c0c36dc-image.png

        V 1 Reply Last reply Reply Quote 0
        • V
          viragomann @CloudNode
          last edited by

          @iptvcld said in pfSense HA LAN Interfaces Only:

          When i try to ping 8.8.8.8 from the secondary; i get this timeout

          You cannot use 8.8.8.8 for troubleshooting, since you direct it to WAN gateway.
          However, for getting the DNS resolution work on the secondary when the WAN cable is connected to the primary, you should set the gateway to "none" for all DNS servers in System > General Setup.

          I have now changed my GW monitoring IP to 1.1.1.1 on the secondary pf.

          So try to ping 1.1.1.1 while taking a capture on the LAN interface.

          C 1 Reply Last reply Reply Quote 0
          • C
            CloudNode @viragomann
            last edited by

            @viragomann My master pf DNS settings like like this
            c34d0b73-4620-42e5-a695-24d1dc4b5a81-image.png

            and secondary pf is the same
            f3bcfe38-b3c2-43f3-9bcc-59509a4ac5dd-image.png

            When i ping 1.1.1.1 in seconday the packet capture just gave this

            12:57:15.477198 IP 192.168.2.81 > 1.1.1.1: ICMP echo request, id 18433, seq 797, length 9
            V 1 Reply Last reply Reply Quote 0
            • V
              viragomann @CloudNode
              last edited by

              @iptvcld said in pfSense HA LAN Interfaces Only:

              When i ping 1.1.1.1 in seconday the packet capture just gave this
              12:57:15.477198 IP 192.168.2.81 > 1.1.1.1: ICMP echo request, id 18433, seq 797, length 9

              You should see the same on the masters LAN. Check that out, please.

              C 1 Reply Last reply Reply Quote 0
              • C
                CloudNode @viragomann
                last edited by

                @viragomann thats correct; i just ran a packet cap on master LAN and then sent a 1.1.1.1 ping from the secondary pf

                13:05:44.735976 IP 192.168.2.81 > 1.1.1.1: ICMP echo request, id 18433, seq 1785, length 9
                V 1 Reply Last reply Reply Quote 0
                • V
                  viragomann @CloudNode
                  last edited by

                  @iptvcld
                  Well, so 1.1.1.1 is routed to the masters LAN address.
                  Now, if you take a capture on the masters WAN you should also see the ICMP packets to 1.1.1.1, but coming from the WAN IP.

                  If that's not the case, either the firewall rule or the outbound NAT on the master might failing anyhow.

                  C 3 Replies Last reply Reply Quote 0
                  • C
                    CloudNode @viragomann
                    last edited by

                    @viragomann Ran a packet cap on master and i see this request 3 times

                    13:10:40.370864 IP 192.168.2.81 > 1.1.1.1: ICMP echo request, id 18433, seq 2360, length 9

                    seems to be coming from the secondary pf LAN IP still and not the WAN IP

                    Outbound NAT on master is this:
                    525a045c-3692-4aaa-a82a-e04d3aa486ca-image.png

                    And LAN Rules
                    a368e097-3a10-4ad5-9bd4-8c50a2666839-image.png

                    1 Reply Last reply Reply Quote 0
                    • C
                      CloudNode @viragomann
                      last edited by

                      @viragomann Ran it again and i see this

                      13:17:05.527815 IP 76.64.x.x > 1.1.1.1: ICMP echo request, id 55322, seq 0, length 64
13:17:05.542715 IP 1.1.1.1 > 76.64.x.x: ICMP echo reply, id 55322, seq 0, length 64

                      then

                      13:17:05.562166 IP 192.168.2.81 > 1.1.1.1: ICMP echo request, id 34782, seq 78, length 9

                      PING 1.1.1.1 (1.1.1.1): 56 data bytes
                      64 bytes from 1.1.1.1: icmp_seq=0 ttl=55 time=15.187 ms

                      --- 1.1.1.1 ping statistics ---
                      3 packets transmitted, 1 packets received, 66.7% packet loss
                      round-trip min/avg/max/stddev = 15.187/15.187/15.187/0.000 ms

                      V 1 Reply Last reply Reply Quote 0
                      • C
                        CloudNode @viragomann
                        last edited by

                        @viragomann Does this need to be unchecked under my WAN interface?
                        Block private networks and loopback addresses

                        fc8b8c33-e5c7-4253-ac68-050b04b1b2a9-image.png

                        V 1 Reply Last reply Reply Quote 0
                        • V
                          viragomann @CloudNode
                          last edited by

                          @iptvcld said in pfSense HA LAN Interfaces Only:

                          Ran it again and i see this
                          13:17:05.527815 IP 76.64.x.x > 1.1.1.1: ICMP echo request, id 55322, seq 0, length 64
                          13:17:05.542715 IP 1.1.1.1 > 76.64.x.x: ICMP echo reply, id 55322, seq 0, length 64

                          then
                          13:17:05.562166 IP 192.168.2.81 > 1.1.1.1: ICMP echo request, id 34782, seq 78, length 9

                          PING 1.1.1.1 (1.1.1.1): 56 data bytes

                          Strange!

                          Check the masters state table and filter for 1.1.1.1 after pinging from backup.

                          C 3 Replies Last reply Reply Quote 0
                          • V
                            viragomann @CloudNode
                            last edited by

                            @iptvcld said in pfSense HA LAN Interfaces Only:

                            @viragomann Does this need to be unchecked under my WAN interface?
                            Block private networks and loopback addresses

                            fc8b8c33-e5c7-4253-ac68-050b04b1b2a9-image.png

                            No, this is only for incoming packets. There is no need to allow private addresses on WAN.

                            1 Reply Last reply Reply Quote 0
                            • C
                              CloudNode @viragomann
                              last edited by

                              This post is deleted!
                              1 Reply Last reply Reply Quote 0
                              • C
                                CloudNode @viragomann
                                last edited by

                                @viragomann This is the state table after pinging from 2nd
                                428da5d7-7b48-472e-b40e-1b54110340db-image.png

                                V 1 Reply Last reply Reply Quote 0
                                • C
                                  CloudNode @viragomann
                                  last edited by CloudNode

                                  @viragomann When try to ping 1.0.0.1 from 2nd; i get all fails

                                  PING 1.0.0.1 (1.0.0.1): 56 data bytes
92 bytes from 127.0.0.1: Time to live exceeded
Vr HL TOS  Len   ID Flg  off TTL Pro  cks      Src      Dst
 4  5  00 0054 ee77   0 0000  01  01 0000 127.0.0.1  1.0.0.1 

92 bytes from 127.0.0.1: Time to live exceeded
Vr HL TOS  Len   ID Flg  off TTL Pro  cks      Src      Dst
 4  5  00 0054 4453   0 0000  01  01 0000 127.0.0.1  1.0.0.1 

92 bytes from 127.0.0.1: Time to live exceeded
Vr HL TOS  Len   ID Flg  off TTL Pro  cks      Src      Dst
 4  5  00 0054 071e   0 0000  01  01 0000 127.0.0.1  1.0.0.1 


--- 1.0.0.1 ping statistics ---
3 packets transmitted, 0 packets received, 100.0% packet loss

                                  And when i run this ping while doing packet cap on master LAN; i dont see the entries come over

                                  V 1 Reply Last reply Reply Quote 0
                                  • V
                                    viragomann @CloudNode
                                    last edited by

                                    @iptvcld said in pfSense HA LAN Interfaces Only:

                                    This is the state table after pinging from 2nd

                                    The last two lines look well for only one ping packet. One state entry for LAN and one for WAN.
                                    But the other lines are showing faults at all. The packets column shows the packets for each direction (request, respond). So it shows many requests, but few responds.

                                    Not clear, what's going wrong here, now.
                                    Maybe there are some hints in the system log?

                                    1 Reply Last reply Reply Quote 0
                                    • V
                                      viragomann @CloudNode
                                      last edited by

                                      @iptvcld said in pfSense HA LAN Interfaces Only:

                                      When try to ping 1.0.0.1 from 2nd; i get all fails

                                      This IP might be routed out to WAN, since there is no special route for it. But WAN is not connected, hence it will fail.

                                      C 2 Replies Last reply Reply Quote 0
                                      • C
                                        CloudNode @viragomann
                                        last edited by

                                        @viragomann This makes sense..

                                        Its just so odd that 1 get 1/3 ping request to pass when doing a ping from 2nd pf. It should almost be all or none..

                                        1 Reply Last reply Reply Quote 0
                                        • C
                                          CloudNode @viragomann
                                          last edited by

                                          @viragomann So just an over view of what I have done to try to get this going

                                          • Master Node: Disabled Static Route configuration under HA Sync setting

                                          • Master Node: Outbound NAT changed to Hybrid and added a mapping for source 127.0.0.0/8 to ANY and NAT Address is LAN address (this synced over to Secondary node)

                                          • Secondary Node: System-Advanced-Miscellaneous; enabled State Killing on Gateway Failure

                                          • Secondary Node: System-Routing; Created a new GW using LAN interface with the IP of LAN interface from Master node as the Gateway and 1.1.1.1 for the monitoring IP.

                                          • Secondary Node: Created a GW Group as PPPOE WAN Tier1, Above noted GW as Tier 2 and VPN GW as Tier 3

                                          • Secondary Node: The new GW group was set to default under IPv4

                                          V 1 Reply Last reply Reply Quote 0
                                          • V
                                            viragomann @CloudNode
                                            last edited by

                                            @iptvcld said in pfSense HA LAN Interfaces Only:

                                            Secondary Node: Created a GW Group as PPPOE WAN Tier1, Above noted GW as Tier 2 and VPN GW as Tier 3

                                            The VPN GW has to be set to "never", so that it is no member of this gateway group.

                                            All other settings should be correct.

                                            When you run a ping with 3 requests to 1.1.1.1 on the secondary the packets should go out to the masters LAN and be forwarded to the internet, since the gw monitoring has set the static route for this IP.

                                            When you sniff the traffic filter for ICMP and 1.1.1.1, you should see 3 ICMP request packets (and 3 responds if it works) on

                                            • both masters and secondarys LAN: 192.168.2.81 > 1.1.1.1
                                            • on the masters WAN: WAN IP > 1.1.1.1

                                            If you look into the masters state table you should see

                                            • one entry for LAN: 192.168.2.81 > 1.1.1.1
                                            • and one for WAN: WAN IP > 1.1.1.1

                                            Both should show 3/3 in the packets column.

                                            Not clear, why it doesn't behave this way on your setup.

                                            C 2 Replies Last reply Reply Quote 0
                                            • First post
                                              Last post