Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    WAN interface keeps dropping after upgrade to 1.2.1

    Scheduled Pinned Locked Moved General pfSense Questions
    43 Posts 6 Posters 21.5k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • C
      CoccoBill
      last edited by

      Thanks for your response. On reboot dhclient logs this:

      Jan 4 12:06:09 dhclient[257]: bound to [address removed] – renewal in 21600 seconds.
      Jan 4 12:06:09 dhclient[257]: DHCPACK from [address removed]
      Jan 4 12:06:09 dhclient[257]: DHCPREQUEST on fxp0 to 255.255.255.255 port 67
      Jan 4 12:06:07 dhclient[257]: DHCPOFFER from [address removed]
      Jan 4 12:06:06 dhclient[257]: DHCPDISCOVER on fxp0 to 255.255.255.255 port 67 interval 1
      Jan 4 12:05:50 dhclient[257]: DHCPREQUEST on fxp0 to 255.255.255.255 port 67

      21600 seconds is 6 hours, add an hour of unsuccessful renewal attempts and the pattern matches exactly with the connection dropouts. I haven't been able to see what's going on since the webgui only shows last 50 messages and only saves to ramdisk, guess I'll have to wait for the next drop and check the syslog file. I'm guessing it's under /var/log/?

      As of the dhcp server I have no clue, it's my ISP's server for my broadband connection at home.

      1 Reply Last reply Reply Quote 0
      • W
        wallabybob
        last edited by

        You'll find more log details than the web GUI provides in the file /var/log/system.log

        1 Reply Last reply Reply Quote 0
        • C
          CoccoBill
          last edited by

          Unfortunately there's nothing in the logs, after the 6 hour lease time dhclient tries to renew the lease but never gets a response. Last night this happened at 5AM and it kept on trying until at 11AM it successfully renewed the lease without me touching anything. I suppose it's a possibility there's a problem on the ISP's end, but the fact that the problems started immediately after upgrading to 1.2.1, and the fact that the problem is always fixed with a reboot suggests otherwise imo. Any ideas or should I just downgrade back to 1.2?

          1 Reply Last reply Reply Quote 0
          • W
            wallabybob
            last edited by

            Did you try to get a tcpdump trace of the DHCP interactions? It could be helpful to know if you are getting a DHCP response from the ISP. If there is no response at all then you should probably discuss this with your ISP. If you are getting a response then we can look further into why dhclient is apparently not seeing it.

            Did you look in the firewall logs to see if a DHCP response was blocked by the firewall?

            Regarding downgrade to 1.2 - I guess that depends on how much time you are prepared to put into investigating this problem, whether or not 1.2.1 has any new features that are important to you and how much pain the problem is causing. As time passes, its more and more likely that if you report another problem people will encourage you to upgrade to 1.2.1 (or later).

            1 Reply Last reply Reply Quote 0
            • C
              CoccoBill
              last edited by

              Just had another dropout so took a trace and sure enough, no responses from the DHCP server, and they're not being blocked by the firewall. I do see the outgoing requests though.

              A wild stab in the dark, it looks like my isp's server always immediately answers to a DHCPDISCOVER broadcast and the following DHCPREQUEST, but not to the DHCPREQUEST renewal. I'm probably way off base here but could it be that it requires the initial broadcast "handshake"?

              I guess I'll have a chat with my ISP's tech support tomorrow, thanks for your help.

              1 Reply Last reply Reply Quote 0
              • W
                wallabybob
                last edited by

                @CoccoBill:

                A wild stab in the dark, it looks like my isp's server always immediately answers to a DHCPDISCOVER broadcast and the following DHCPREQUEST, but not to the DHCPREQUEST renewal. I'm probably way off base here but could it be that it requires the initial broadcast "handshake"?

                If it does its broken!

                The tcpdump trace shows packets before they go to firewall.

                1 Reply Last reply Reply Quote 0
                • C
                  CoccoBill
                  last edited by

                  Opened a ticket at my isp's support, from their end the line seems functional and they're not aware of any problems with their DHCP service, as in no other complaints.

                  I wonder if FreeBSD7 doesn't like my Intel 82557 Pro/100.

                  1 Reply Last reply Reply Quote 0
                  • W
                    wallabybob
                    last edited by

                    @CoccoBill:

                    Opened a ticket at my isp's support, from their end the line seems functional and they're not aware of any problems with their DHCP service, as in no other complaints.

                    That's different from confirming they are actually sending DHCP responses back to you.

                    I wonder if FreeBSD7 doesn't like my Intel 82557 Pro/100.

                    The fxp has been around for a long time and gigabit interfaces have been available quite cheaply for some years so I suppose its possible the fxp driver has suffered some sort of regression that has hasn't been picked up because it hasn't been exercised in the necessary way. But I find it hard to imagine what kind of regression (apparently) affects DHCP responses and only DHCP responses and causes them to be discarded before they are traced.

                    Have you checked the interface error counters (# netstat -i ) for input errors on your WAN interface?

                    1 Reply Last reply Reply Quote 0
                    • C
                      CoccoBill
                      last edited by

                      Currently seems to be 298 Ierrs out of 12635041 Ipkts since the last reboot (so this period doesn't include dhcp problems), this sounds a bit high?

                      Update: Had another connection drop, no additional Ierrs and nothing in the logs.

                      1 Reply Last reply Reply Quote 0
                      • W
                        wallabybob
                        last edited by

                        I presume your first hop from the pfsense box to the ISP is to a modem of some sort. If so, your error count is probably OK.

                        Suggestion: Next time pfSense attempts to renew its DHCP lease make sure there is no other traffic on the WAN link no FTPs, no web browsing etc. Then do a

                        tcpdump -i fxp0

                        for as long as it takes to see a few DHCP renewal requests go to the ISP. This will show ALL traffic on that hop. You're looking to see if there is any response at all. Check the interface counters before and after to see if there were any received packets or any errors. Perhaps you are getting a DHCP response that is not accepted by dhclient but is accepted by some other systems (e.g. Windows). If you see a consistent response try again with the command

                        tcpdump -p -i fxp0

                        Do you see a different pattern?

                        If you can, capture the traces and post them here.

                        1 Reply Last reply Reply Quote 0
                        • C
                          CoccoBill
                          last edited by

                          Ok I'll try to do a trace of all the trafiic. And no modems involved, just direct ethernet connection (fibre-optic).

                          1 Reply Last reply Reply Quote 0
                          • W
                            wallabybob
                            last edited by

                            My gut feel, based on reported characteristics of fibre links, is that about 300 errors in about 12,000,000 packets is a "high" error rate. I doubt your system has been up long enough for the packet counter to wrap :-)

                            1 Reply Last reply Reply Quote 0
                            • C
                              CoccoBill
                              last edited by

                              Ok I did the trace and still no response. After that I downgraded to 1.2 and now for the first time in a week the connection has been up for 16 hours straight, the renewals work again. I would say there's something funny with 1.2.1 and my setup, but unfortunately I don't have the time nor energy to look into it further at the moment.

                              Thanks for your help!

                              1 Reply Last reply Reply Quote 0
                              • R
                                rigius
                                last edited by

                                Apparently I'm having the same problem after upgrading from 1.2.RELEASE to 1.2.2.

                                The affected interface is rl1 (WAN2), which gets its external IP address from a Zyxel P-660R-D1 working in half-bridge mode.

                                Here is the relevant log:

                                
Jan 15 14:40:02 	dhclient[307]: SENDING DIRECT
Jan 15 14:40:02 	dhclient[307]: DHCPREQUEST on rl1 to 192.168.1.1 port 67
Jan 15 14:39:33 	dhclient[307]: SENDING DIRECT
Jan 15 14:39:33 	dhclient[307]: DHCPREQUEST on rl1 to 192.168.1.1 port 67
Jan 15 14:39:20 	dhclient[307]: SENDING DIRECT
Jan 15 14:39:20 	dhclient[307]: DHCPREQUEST on rl1 to 192.168.1.1 port 67
Jan 15 14:39:12 	dhclient[307]: SENDING DIRECT
Jan 15 14:39:12 	dhclient[307]: DHCPREQUEST on rl1 to 192.168.1.1 port 67
Jan 15 14:39:04 	dhclient[307]: SENDING DIRECT
Jan 15 14:39:04 	dhclient[307]: DHCPREQUEST on rl1 to 192.168.1.1 port 67
Jan 15 14:39:01 	dhclient[307]: SENDING DIRECT
Jan 15 14:39:01 	dhclient[307]: DHCPREQUEST on rl1 to 192.168.1.1 port 67
Jan 15 14:38:58 	dhclient[307]: SENDING DIRECT
Jan 15 14:38:58 	dhclient[307]: DHCPREQUEST on rl1 to 192.168.1.1 port 67
Jan 15 14:38:56 	dhclient[307]: SENDING DIRECT
Jan 15 14:38:56 	dhclient[307]: DHCPREQUEST on rl1 to 192.168.1.1 port 67
Jan 15 14:11:12 	dnsmasq[848]: reading /var/dhcpd/var/db/dhcpd.leases
Jan 15 14:08:56 	dhclient[307]: bound to [EXTERNAL_IPADDRESS] -- renewal in 1800 seconds.
Jan 15 14:08:56 	dhclient[307]: DHCPACK from 192.168.1.1
Jan 15 14:08:56 	dhclient[307]: DHCPREQUEST on rl1 to 255.255.255.255 port 67
Jan 15 14:08:16 	dnsmasq[848]: reading /var/dhcpd/var/db/dhcpd.leases
Jan 15 14:05:58 	dnsmasq[848]: reading /var/dhcpd/var/db/dhcpd.leases
Jan 15 14:03:57 	dnsmasq[848]: reading /var/dhcpd/var/db/dhcpd.leases
Jan 15 14:02:06 	dnsmasq[848]: reading /var/dhcpd/var/db/dhcpd.leases
Jan 15 14:01:11 	dhclient[307]: SENDING DIRECT
Jan 15 14:01:11 	dhclient[307]: DHCPREQUEST on rl1 to 192.168.1.1 port 67
Jan 15 13:59:33 	dnsmasq[848]: reading /var/dhcpd/var/db/dhcpd.leases
Jan 15 13:57:37 	dhclient[307]: SENDING DIRECT
Jan 15 13:57:37 	dhclient[307]: DHCPREQUEST on rl1 to 192.168.1.1 port 67
Jan 15 13:56:13 	dnsmasq[848]: reading /var/dhcpd/var/db/dhcpd.leases
Jan 15 13:53:07 	dhclient[307]: SENDING DIRECT
Jan 15 13:53:07 	dhclient[307]: DHCPREQUEST on rl1 to 192.168.1.1 port 67
Jan 15 13:51:14 	dhclient[307]: SENDING DIRECT
[...]
                                1 Reply Last reply Reply Quote 0
                                • C
                                  CoccoBill
                                  last edited by

                                  Looks familiar, is it fixed with a reboot and starts again when your dhcp lease expires? Which network card does the interface have?

                                  1 Reply Last reply Reply Quote 0
                                  • R
                                    rigius
                                    last edited by

                                    The firewall is a Linitx FX5620 (Fabiatech) with one GB RTL8110S network card and five RTL8100C. It used to appear on the pfsense recommended hardware list. I'm having the problem on one of the RTL8100C ports.

                                    The connection has been cut only four or five times since the upgrade. It may take place when the dhcp lease expires, but not every time. It seems to restablish itself after some time (not yet sure of how much time neither). And yes, a reboot "fixes" temporarily the problem.

                                    After rebooting the ADSL router and the pfsense box, the connection has been stable for the last 20 hours or so. Here goes a commented log:

                                    
[...]
Jan 16 06:49:29 pfsense dhclient[307]: DHCPREQUEST on rl1 to 255.255.255.255 port 67
Jan 16 06:49:29 pfsense dhclient[307]: DHCPACK from 192.168.1.1
Jan 16 06:49:29 pfsense dhclient[307]: bound to EXTERNAL_IPADDRESS -- renewal in 1800 seconds.  #(gets address)
Jan 16 07:19:29 pfsense dhclient[307]: DHCPREQUEST on rl1 to 192.168.1.1 port 67                            #(at the end of the lease, asks for renewal)
Jan 16 07:19:29 pfsense dhclient[307]: SENDING DIRECT
Jan 16 07:19:30 pfsense dhclient[307]: DHCPREQUEST on rl1 to 192.168.1.1 port 67                            #(it doesn't get it and keeps asking)
Jan 16 07:19:30 pfsense dhclient[307]: SENDING DIRECT
Jan 16 07:19:32 pfsense dhclient[307]: DHCPREQUEST on rl1 to 192.168.1.1 port 67
Jan 16 07:19:32 pfsense dhclient[307]: SENDING DIRECT
Jan 16 07:19:34 pfsense dhclient[307]: DHCPREQUEST on rl1 to 192.168.1.1 port 67
Jan 16 07:19:34 pfsense dhclient[307]: SENDING DIRECT
Jan 16 07:19:36 pfsense dhclient[307]: DHCPREQUEST on rl1 to 192.168.1.1 port 67
[...]
Jan 16 07:47:43 pfsense dhclient[307]: DHCPREQUEST on rl1 to 255.255.255.255 port 67
Jan 16 07:47:43 pfsense dhclient[307]: DHCPACK from 192.168.1.1
Jan 16 07:47:43 pfsense dhclient[307]: bound to EXTERNAL_IPADDRESS -- renewal in 1800 seconds.  #(finally it gets a new lease, after about 24 minutes)
Jan 16 08:17:43 pfsense dhclient[307]: DHCPREQUEST on rl1 to 192.168.1.1 port 67                            #(and the cycle starts again)
Jan 16 08:17:43 pfsense dhclient[307]: SENDING DIRECT
Jan 16 08:17:45 pfsense dhclient[307]: DHCPREQUEST on rl1 to 192.168.1.1 port 67
Jan 16 08:17:45 pfsense dhclient[307]: SENDING DIRECT
[...]

                                    Could the DHCP renewal problem be provoked by the fact that dhclient is sending the renewal requests directly to the address of the DHCP server instead of the broadcast address? Is there a way to ask dhclient to avoid sending "direct" renewal requests?

                                    1 Reply Last reply Reply Quote 0
                                    • R
                                      rigius
                                      last edited by

                                      The OPT1 (rl1) connection is falling more or less once every day and now I'm also getting this messages:

                                      
Jan 20 12:38:45 	kernel: arpresolve: can't allocate route for OPT1_EXTERNAL_IPADDRESS
Jan 20 12:38:45 	kernel: arplookup OPT1_EXTERNAL_IPADDRESS failed: host is not on local network

                                      They seem to be related to the load balancing monitor IP address of the OPT1 interface (wich, to make a long story short, is OPT1_EXTERNAL_IPADDRESS).

                                      I'm beginning to get into trouble at work, so I think that I will have to downgrade to 1.2 very soon. Has anyone done a successful downgrade from 1.2.2 to 1.2 with the firmware update option?

                                      1 Reply Last reply Reply Quote 0
                                      • W
                                        wallabybob
                                        last edited by

                                        Could the DHCP renewal problem be provoked by the fact that dhclient is sending the renewal requests directly to the address of the DHCP server instead of the broadcast address?

                                        That's the way it is supposed to work: when you ask for lease renewal you ask the party that granted the lease.

                                        Is there a way to ask dhclient to avoid sending "direct" renewal requests?

                                        I suggest you lookup the FreeBSD man pages: go to http://www.freebsd.org/cgi/man.cgi and type dhclient in the Man page or Keyword Search box and click Submit.

                                        1 Reply Last reply Reply Quote 0
                                        • R
                                          rigius
                                          last edited by

                                          @wallabybob:

                                          Could the DHCP renewal problem be provoked by the fact that dhclient is sending the renewal requests directly to the address of the DHCP server instead of the broadcast address?

                                          That's the way it is supposed to work: when you ask for lease renewal you ask the party that granted the lease.

                                          Yes. You are right. After looking more carefully at the logs, I have found that:

                                          -DHCLIENT is not able to get its lease extended at renewal time.

                                          -DHCLIENT is getting its lease extended at rebinding time.

                                          So everything seems normal.

                                          Is there a way to ask dhclient to avoid sending "direct" renewal requests?

                                          I suggest you lookup the FreeBSD man pages: go to http://www.freebsd.org/cgi/man.cgi and type dhclient in the Man page or Keyword Search box and click Submit.

                                          This page has also helped me understand the process: http://www.tcpipguide.com/free/t_DHCPLeaseRenewalandRebindingProcesses.htm

                                          But I'm still getting this connection drops every day or so…

                                          1 Reply Last reply Reply Quote 0
                                          • W
                                            wallabybob
                                            last edited by

                                            So everything seems normal.

                                            Except that the lease extension requests are apparently ignored.

                                            Are the lease extension requests getting out on the wire? (May be hard to test without specialised equipment.)

                                            If so, why aren't they provoking a response? (Your ISP will probably have to help with than one.)

                                            Are there any link status change reports on rl1 at these times? (Does pulling out the rl1 cable generate a link status change on rl1 report?)

                                            1 Reply Last reply Reply Quote 0
                                            • First post
                                              Last post
                                            Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.