Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    dhclient exiting on WAN

    Scheduled Pinned Locked Moved
    General pfSense Questions
    3
    19
    2.1k
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • stephenw10S
      stephenw10 Netgate Administrator
      last edited by

      Is WAN_DHCP set as the default gateway in System > Routing?

      Do you have more than one IPv4 gateway shown there?

      T 1 Reply Last reply Reply Quote 0
      • T
        tomashk @stephenw10
        last edited by

        @stephenw10
        Now it is like that:
        6ed687ab-5677-4a07-ab6c-4d0a1f3c3784-image.png
        Previously it has been set as Automatic but I have just switched it to WAN_DHPC and it behaves exactly the same (or at least I don't see any changes). Only after pressing "Save" on System/Routing/Gateways page I got default route.

        Also as you can see IPv6 gateway is set to none if it matters but I don't use IPv6 so I don't think that it means anything.

        1 Reply Last reply Reply Quote 0
        • T
          tomashk
          last edited by

          Also I looked in the logs to see how it behaves when I used renew button for interface (scenario 1) to get IP (when dhclient did not work for me on the start of modem) and how it behaves when I get properly working internet and I just unplug ethernet cable from modem and plug it back (scenario 2).

          • scenario 1
            9deb256f-4317-4271-ba9b-7bfbd777db7e-image.png
          • scenario 2
            78983ee8-0bf8-4cd7-8a1e-8e078f483ae9-image.png

          As we can see first one doesn't add default route. Unfortunately I have not analysed it enough to know if it is bad or not. I just show my observations

          1 Reply Last reply Reply Quote 0
          • stephenw10S
            stephenw10 Netgate Administrator
            last edited by

            Hmm, so in scenario 1 you end up without a default route even though you do have a gateway?

            T 1 Reply Last reply Reply Quote 0
            • T
              tomashk @stephenw10
              last edited by

              @stephenw10
              Yes. Now with working internet I have situation from the picture below. With scenario 1 I don't have the first line starting with default
              c2f90571-aca3-4786-ad98-1d93e34bc984-image.png

              I will test a little bit more soon because I want to rule any situation that those problems are caused by my mistakes with configuration. I believe those problems started with ver 2.6/22.01 but I can't be sure - I haven't noticed it previously because I had long uptime and there was no need for turning it off.

              1 Reply Last reply Reply Quote 0
              • stephenw10S
                stephenw10 Netgate Administrator
                last edited by

                Mmm, there are a number of threads reporting similar issues. It look to be real IMO but it's not something we can easily replicate. It seems to be timing related.

                Steve

                GertjanG 1 Reply Last reply Reply Quote 0
                • GertjanG
                  Gertjan @stephenw10
                  last edited by

                  Scenario 1: from bottom to top :

                  a1ff4742-8be0-4df9-9cc5-400288de725f-image.png

                  Scenario 2: from bottom to top :

                  f5c7842c-78f0-40af-904f-0b073ff4bae4-image.png

                  So, if scenario 2 is what should happen = adding the 'default' route, it is not logged and thus not happening in scenario 1.
                  Maybe the IP was the same so the "add route default" wasn't needed ?
                  As far as know, dhclient uses shell scripts to do this. I'll have a look into that this weekend.

                  No "help me" PM's please. Use the forum, the community will thank you.
                  Edit : and where are the logs ??

                  T 1 Reply Last reply Reply Quote 0
                  • T
                    tomashk @Gertjan
                    last edited by

                    Sorry for late response - I was away for a few days.

                    @gertjan said in dhclient exiting on WAN:

                    Maybe the IP was the same so the "add route default" wasn't needed ?

                    Probably IP was the same. Unfortunately I've lost those logs but currently whenever I renew I get the same WAN IP.

                    PS. For test I installed pfsense on another device and restored my configuration. Of course it works properly every time. Probably it will take some time to find how it differs when compared with "wrong logs". The main difference I see from the start is that when it works (on new installation) dhclient is able to start properly. So there is nothing like

                    rc.bootup: The command '/sbin/dhclient -c /var/etc/dhclient_wan.conf igb0 > /tmp/igb0_output 2> /tmp/igb0_error_output' returned exit code '1', the output was ''

                    in the logs. And later I can see standard DHCPDISCOVER (or others) or dhclient's messages about link state like

                    igb0 link state up -> down
igb0 link state down -> up

                    I guess I'll try to find out what might stop dhclient from starting and throwing

                    igb0: no link .............. giving up

                    into /tmp/igb0_output. Of course I'll try to find other differences, but this one looks like the main one.

                    GertjanG stephenw10S 2 Replies Last reply Reply Quote 0
                    • GertjanG
                      Gertjan @tomashk
                      last edited by

                      @tomashk said in dhclient exiting on WAN:

                      I guess I'll try to find out what might stop dhclient from starting and throwing

                      igb0: no link .............. giving up

                      into /tmp/igb0_output

                      You've answered your own question.

                      The dhclient aborts when there is no WAN interface any more.
                      It stops executing, as the interface WAN isn't listed as a system resource any more.

                      Your issue isn't dhclient , that does exacly what it is designed to do.

                      Focus on the WAN interface : make it stopping going down - up - down - up - etc.

                      The thing is, you know already who brings the interface down : the other device in front of it.
                      Modems do so very often. It's their way to 'signal' that they created a "soft link" with the upstream ISP.
                      Or : this scenario : it powers up. The internal web GUI comes up, and the modem brings up it's LAN == the wan of pfSense. When the dhclient starts now, and send a DHCPDISCOVER, it will obtain an RFC1918 from the DHCP server of the modem. This RFC1918 will permit you to connect to the GUI of the modem to change it settings.
                      Moments later, the modem finished building the soft link with the ISP.
                      It brings down it's LAN, as it has to as this is the only way to reset the current connection, and brings it again. Now, when the dhclient of pfSense send a DHCPDISCOVER, the modem will be transparent, and the DHCPDISCOVER will reach the ISP. This time, the DHCP server of the ISP will reply, and give you a 'real' WAN IP.
                      This is how most modems work. For decades now.

                      Things become even more interesting if initially the modem has a hard time syncing with the ISP. Every time it looses the connection, because, for example, the initial speed was to high, it aborts, (bring its LAN down), and tries a connection with a lower speed.

                      dhclient, on the pfSense side, looses track.

                      Your mission, as a modem user, is to find the right timings settings.
                      You know it('s possible, because most often, when you hook up a PC to your modem and use that pppoe connection setting, your PC will connect.
                      Your PC also uses a dhcp client like software.

                      No "help me" PM's please. Use the forum, the community will thank you.
                      Edit : and where are the logs ??

                      1 Reply Last reply Reply Quote 0
                      • stephenw10S
                        stephenw10 Netgate Administrator @tomashk
                        last edited by

                        @tomashk said in dhclient exiting on WAN:

                        For test I installed pfsense on another device and restored my configuration. Of course it works properly every time.

                        What's the difference between those devices?

                        The is probably a timing issue. If the upstream device resets the link it may only be down for a short time and that must coincide with the dhclient script running in order to fail.
                        Running on hardware that is significantly faster or slower could well change the timing enough to prevent it happening.

                        It should not be possible to hit that but it may also be easy to work around by adding a boot delay in pfSense.

                        Steve

                        T 1 Reply Last reply Reply Quote 0
                        • T
                          tomashk @stephenw10
                          last edited by

                          @stephenw10 The one failing is based on small Asrock motherboard with Intel Celeron J4005 and working one is hp t620 plus thin client. I believe the second one is taking longer to start.

                          As you said this is probably a timing issue. The idea with boot delay sounds like good quick fix once I'll try to go back to J4005. For now I'll have to leave it because work doesn't leave me much time to experiment. For a moment I have that temporary new device which works. But in a week or two I am going to get back to it and try to understand it better.

                          By understand I mean for example that long time ago I looked into https://github.com/pfsense/FreeBSD-src/blob/devel-12/sbin/dhclient/dhclient.c (the file is important - I just took random branch from github as example) and I found the code that gives up (line 469)

                          aa71c72a-21e2-4c0f-bfdd-16a7a1ee9f69-image.png

                          Due to value 10 in line 468 and sleep from line 472 dhlient will try to wait for WAN interface only for 10 seconds. I was thinking that maybe it would be good to change value 10 to environment variable (with default value 10) so user can set this, but there are two problems with my current poor understanding:

                          • I'm not able to predict what would be broken in completely different part of the system if dhclient starts too long (maybe nothing but maybe some kind of the butterfly effect :) )
                          • there might already exist some system script that should be used for such situations
                          1 Reply Last reply Reply Quote 0
                          • stephenw10S
                            stephenw10 Netgate Administrator
                            last edited by

                            Mmm, I agree increasing that initially seems to be a good idea but could easily cause problems on systems with multiple disconnected WANs.

                            It would be nice to have that value configurable.

                            As a simple test you could just set the line:

                            autoboot_delay="30"

                            In /boot/loader.conf. That's normally 3s. If that works you might put it in loader.conf.local instead to prevent it getting overwritten.

                            Steve

                            1 Reply Last reply Reply Quote 0
                            • T
                              tomashk
                              last edited by

                              Quick update. Because I wanted to test the release candidate for 22.05 I took the old device, that had problems described above. After updating to the RC version it still existed. But I used suggested autoboot_delay and it fixed this timing problem. For me 10 seconds were enough to make it work.

                              1 Reply Last reply Reply Quote 1
                              • stephenw10S
                                stephenw10 Netgate Administrator
                                last edited by

                                Anyone still hitting this who can test there is a patch now available here:
                                https://redmine.pfsense.org/issues/13671#note-4
                                We need feedback there to confirm it.

                                Steve

                                1 Reply Last reply Reply Quote 0
                                • First post
                                  Last post