Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    pfSense WAN keeps disconnecting out every 10 mins

    Scheduled Pinned Locked Moved General pfSense Questions
    13 Posts 3 Posters 4.3k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • DerelictD
      Derelict LAYER 8 Netgate
      last edited by

      What is happening when those ARP resolve messages start? You showed the end, what about the beginning? Is the MTU showing anything strange on the interface when it is not working?

      dpinger is trying to ping the gateway address but it cannot because it is not receiving an ARP response for it on WAN. Then it miraculously does for some reason.

      If it were me I'd packet capture for ARP on WAN and see what is happening. I'd just set interface WAN protocol ARP and a packet count of 100000 or 1000000 and let it run. Then get the times of the start and end of the can't allocate llinfo logs and see what's happening there in wireshark.

      Chattanooga, Tennessee, USA
      A comprehensive network diagram is worth 10,000 words and 15 conference calls.
      DO NOT set a source address/port in a port forward or firewall rule unless you KNOW you need it!
      Do Not Chat For Help! NO_WAN_EGRESS(TM)

      C 1 Reply Last reply Reply Quote 0
      • C
        carbontitan @Derelict
        last edited by

        @derelict There isn't anything in the log before the arpresolve.
        There is a minute or so of nothing in the logs, before the arpresolve messages.

        I will run the packet capture, as you suggested, and see if anything jumps out to indicate the problem.
        I will post back here when I know more.

        Many thanks for the help.

        1 Reply Last reply Reply Quote 0
        • C
          carbontitan
          last edited by

          So I have run a packet capture of the ARP packets during one of the outages.
          Not many packets were actually captured and I can't see anything obvious.
          I have masked my IP and my MAC address is being spoofed as the original TT router, as I was testing to see of that had anything to do with it, as it does with other ISPs.

          18:05:42.642541 04:c0:6f:3c:f0:88 > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 42: Ethernet (len 6), IPv4 (len 4), Request who-has 62.64.160.1 tell 62.64.167.xx, length 28
18:05:42.650730 28:8a:1c:ed:5c:53 > 04:c0:6f:3c:f0:88, ethertype ARP (0x0806), length 60: Ethernet (len 6), IPv4 (len 4), Reply 62.64.160.1 is-at 28:8a:1c:ed:5c:53, length 46
18:08:27.713590 04:c0:6f:3c:f0:88 > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 42: Ethernet (len 6), IPv4 (len 4), Request who-has 62.64.167.xx tell 62.64.167.xx, length 28
18:08:27.734947 04:c0:6f:3c:f0:88 > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 42: Ethernet (len 6), IPv4 (len 4), Request who-has 62.64.160.1 tell 62.64.167.xx, length 28
18:08:27.740788 28:8a:1c:ed:5c:53 > 04:c0:6f:3c:f0:88, ethertype ARP (0x0806), length 60: Ethernet (len 6), IPv4 (len 4), Reply 62.64.160.1 is-at 28:8a:1c:ed:5c:53, length 46
18:13:38.449603 ac:1f:6b:4b:db:44 > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 60: Ethernet (len 6), IPv4 (len 4), Request who-has 92.1.32.1 tell 92.1.39.xx, length 46
18:19:15.670626 ac:1f:6b:4b:db:44 > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 60: Ethernet (len 6), IPv4 (len 4), Request who-has 92.1.32.1 tell 92.1.39.xx, length 46
18:21:10.750994 ac:1f:6b:4b:db:44 > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 60: Ethernet (len 6), IPv4 (len 4), Request who-has 92.23.240.1 tell 92.23.246.xx, length 46
18:28:23.847441 04:c0:6f:3c:f0:88 > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 42: Ethernet (len 6), IPv4 (len 4), Request who-has 62.64.160.1 tell 62.64.167.xx, length 28
18:28:23.855219 28:8a:1c:ed:5c:53 > 04:c0:6f:3c:f0:88, ethertype ARP (0x0806), length 60: Ethernet (len 6), IPv4 (len 4), Reply 62.64.160.1 is-at 28:8a:1c:ed:5c:53, length 46
18:31:06.363902 04:c0:6f:3c:f0:88 > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 42: Ethernet (len 6), IPv4 (len 4), Request who-has 62.64.167.xx tell 62.64.167.xx, length 28
18:31:06.534937 04:c0:6f:3c:f0:88 > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 42: Ethernet (len 6), IPv4 (len 4), Request who-has 62.64.160.1 tell 62.64.167.xx, length 28
18:31:06.540914 28:8a:1c:ed:5c:53 > 04:c0:6f:3c:f0:88, ethertype ARP (0x0806), length 60: Ethernet (len 6), IPv4 (len 4), Reply 62.64.160.1 is-at 28:8a:1c:ed:5c:53, length 46
          1 Reply Last reply Reply Quote 0
          • DerelictD
            Derelict LAYER 8 Netgate
            last edited by

            You will have to explain the relationship between those numbering schemes and your WAN. One (62.64.160.1) is being answered. The others (92.1.32.1) is not.

            Please just download the pcap and upload it to the link I sent in chat. I can't tell anything with all that obfuscation.

            Chattanooga, Tennessee, USA
            A comprehensive network diagram is worth 10,000 words and 15 conference calls.
            DO NOT set a source address/port in a port forward or firewall rule unless you KNOW you need it!
            Do Not Chat For Help! NO_WAN_EGRESS(TM)

            1 Reply Last reply Reply Quote 0
            • DerelictD
              Derelict LAYER 8 Netgate
              last edited by

              Are you saying you have the same MAC address spoofed in the upstream router and pfSense WAN?

              Chattanooga, Tennessee, USA
              A comprehensive network diagram is worth 10,000 words and 15 conference calls.
              DO NOT set a source address/port in a port forward or firewall rule unless you KNOW you need it!
              Do Not Chat For Help! NO_WAN_EGRESS(TM)

              C 1 Reply Last reply Reply Quote 0
              • C
                carbontitan @Derelict
                last edited by

                @derelict I have the MAC set in pfSense, to the MAC that is of the TalkTalk router that they supplied when I purchased the FTTC package from TalkTalk.
                The setting of the MAC is irrelevant, as either if it is spoofed or not, it gives the same problem.

                1 Reply Last reply Reply Quote 0
                • DerelictD
                  Derelict LAYER 8 Netgate
                  last edited by

                  And how about a similar capture of DHCP traffic. I'd love to see a capture with both ARP and DHCP but you can't do that in the web gui.

                  Have you called TalkTalk? What do they have to say?

                  Chattanooga, Tennessee, USA
                  A comprehensive network diagram is worth 10,000 words and 15 conference calls.
                  DO NOT set a source address/port in a port forward or firewall rule unless you KNOW you need it!
                  Do Not Chat For Help! NO_WAN_EGRESS(TM)

                  C 1 Reply Last reply Reply Quote 0
                  • C
                    carbontitan @Derelict
                    last edited by

                    @derelict I will get a capture of the DHCP traffic also for you.

                    I haven't talked to TalkTalk yet, as it will be a real pain trying to get someone useful to talk to.
                    However, I put their supplied router back in for a little while and it seemed stable, however, I will need to test for longer with it.

                    1 Reply Last reply Reply Quote 0
                    • DerelictD
                      Derelict LAYER 8 Netgate
                      last edited by

                      They have extremely short DHCP lease time of 15 minutes.

                      When pfSense decides it is time to renew at about half that, it tries every few seconds and there is no response.

                      The lease eventually expires and is removed from the pfSense interface. You can then no longer ping their gateway address because the WAN is now unnumbered.

                      pfSense dhclient then sends a request to the broadcast and gets no answer.

                      pfSense dhclient then reverts to DHCPDISCOVER and, after several seconds, finally gets an answer.

                      This looks like it results in roughly 90 seconds of having no interface address on WAN in the capture you sent.

                      Is your modem in pure bridge mode or whatever you guys call it over there?

                      In my opinion the DHCP server should be honoring the renewal requests and responding with an ACK. It looks to pfSense like the DHCP server disappeared.

                      Chattanooga, Tennessee, USA
                      A comprehensive network diagram is worth 10,000 words and 15 conference calls.
                      DO NOT set a source address/port in a port forward or firewall rule unless you KNOW you need it!
                      Do Not Chat For Help! NO_WAN_EGRESS(TM)

                      1 Reply Last reply Reply Quote 0
                      • DerelictD
                        Derelict LAYER 8 Netgate
                        last edited by

                        It looks to me like your WAN is doing exactly what is described here:

                        https://en.wikipedia.org/wiki/Dynamic_Host_Configuration_Protocol#Reliability

                        The failure is in the lack of response from upstream (either the modem itself or the ISP.)

                        Chattanooga, Tennessee, USA
                        A comprehensive network diagram is worth 10,000 words and 15 conference calls.
                        DO NOT set a source address/port in a port forward or firewall rule unless you KNOW you need it!
                        Do Not Chat For Help! NO_WAN_EGRESS(TM)

                        1 Reply Last reply Reply Quote 0
                        • C
                          carbontitan
                          last edited by

                          Thank you for digging into the problem.
                          Yes I believe my modem is set to pure bridge mode. I have not changed anything on the modem. Also I have tested with a modem that I know works on another TalkTalk line.

                          I have started a conversation with TalkTalk on this problem and will see where I get with it.

                          If anyone happens to know of a way to resolve this problem, I would love to know.

                          1 Reply Last reply Reply Quote 0
                          • stephenw10S
                            stephenw10 Netgate Administrator
                            last edited by

                            I don't think it will make any difference here but just to confirm the modem you have is an actual Openreach device? Is it the Huawei or ECI 'modem'?
                            As far as I know there is no difference between them in practical terms and they are not configured any differently for Talktalk.

                            Steve

                            1 Reply Last reply Reply Quote 0
                            • First post
                              Last post
                            Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.