Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    Wireguard client Gateway disabled after reboot - service not starting

    Plus 23.09 Development Snapshots (Retired)
    3
    21
    1.4k
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • R
      rpotter28 @stephenw10
      last edited by

      @stephenw10 Sorry, but I lost patience with wireguard and uninstalled it. Switched to OpenVPN client with DCO and all is good.

      More reboots than usual because of the Beta track, is what did me in!
      _
      Richard

      1 Reply Last reply Reply Quote 1
      • 4
        4o4rh @stephenw10
        last edited by

        @stephenw10 i added wireguard to the service watchdog which seems to have fixed it for me. so should be an order of execution or a delay needed for pppoe to come up first.

        1 Reply Last reply Reply Quote 0
        • R
          rpotter28 @stephenw10
          last edited by

          @stephenw10 Steve, I felt bad that I didn't have wireguard installed when you replied, so I changed back from OpenVPN yesterday.

          Today, after the upgrade to RC built on Thu Oct 26 21:51:00 EDT 2023, Wireguard started right up, no issues. Has this been worked on, the pppoe part?

          Here are the log snippets, which shows wg0 going up after pppoe:

          Oct 27 13:12:42 php-cgi 504 rc.bootup: Default gateway setting WAN Gateway PPPoE as default.
          Oct 27 13:12:41 php-fpm 408 /rc.newwanip: rc.newwanip: on (IP address: xxx.xxx.xxx.xxx) (interface: 0_WAN[wan]) (real interface: pppoe0).
          Oct 27 13:12:41 php-fpm 408 /rc.newwanip: rc.newwanip: Info: starting on pppoe0.
          Oct 27 13:12:40 ppp 75227 [wan] IFACE: Rename interface ng0 to pppoe0
          Oct 27 13:12:40 check_reload_status 448 rc.newwanip starting pppoe0
          Oct 27 13:12:36 ppp 75227 [wan_link0] PPPoE: connection successful
          Oct 27 13:12:36 ppp 75227 PPPoE: rec'd ACNAME "KGTNON0881W"
          Oct 27 13:12:36 ppp 75227 [wan_link0] PPPoE: Connecting to ''
          Oct 27 13:12:36 kernel ng0: changing name to 'pppoe0

          Oct 27 13:13:00 kernel tun_wg0: link state changed to UP
          Oct 27 13:13:00 kernel wg0: changing name to 'tun_wg0'
          Oct 27 13:12:58 kernel tun_wg0: link state changed to DOWN
          Oct 27 13:12:35 kernel tun_wg0: link state changed to UP
          Oct 27 13:12:35 kernel wg0: changing name to 'tun_wg0'
          Oct 27 13:11:38 kernel tun_wg0: link state changed to DOWN

          Thanks!
          Richard

          1 Reply Last reply Reply Quote 1
          • stephenw10S
            stephenw10 Netgate Administrator
            last edited by

            Yes there have been some changes to the system aliases used there. I wonder if it was trying to start with something undefined.

            Thanks for the feedback!

            R 2 Replies Last reply Reply Quote 1
            • R
              rpotter28 @stephenw10
              last edited by

              @stephenw10 Thank you for seeing my initial post and giving it some traction. There has been only this one reboot. I am old school, never invite trouble by rebooting something that's working.

              I will follow up if anything changes after the next reboot, possibly another RC update?
              Thanks!

              1 Reply Last reply Reply Quote 0
              • stephenw10S
                stephenw10 Netgate Administrator
                last edited by

                Unless we find something show-stopping this will be the last build before release.

                R 1 Reply Last reply Reply Quote 0
                • R
                  rpotter28 @stephenw10
                  last edited by

                  @stephenw10 That makes sense, thanks. This would not be a show stopper anyways, and it looks good IMHO.
                  Thanks for your efforts!

                  1 Reply Last reply Reply Quote 1
                  • R
                    rpotter28 @stephenw10
                    last edited by

                    @stephenw10 I have been running the BETA as a vm on Hyper-v, but this morning I had the opportunity to switch over to a bare metal white box, to get it up to RC.

                    Guess what, wireguard didn't start. There is a difference in the logs:

                    Oct 30 10:46:03 php-fpm 411 /rc.newwanip: Default gateway setting WAN Gateway PPPoE as default.
                    Oct 30 10:46:02 php-fpm 411 /rc.newwanip: rc.newwanip: on (IP address: x.x.x.x) (interface: 0_WAN[wan]) (real interface: pppoe0).
                    Oct 30 10:46:02 php-fpm 411 /rc.newwanip: rc.newwanip: Info: starting on pppoe0.
                    Oct 30 10:46:01 ppp 40734 [wan] IFACE: Rename interface ng0 to pppoe0
                    Oct 30 10:46:01 check_reload_status 443 rc.newwanip starting pppoe0
                    Oct 30 10:45:57 ppp 40734 [wan_link0] PPPoE: connection successful
                    Oct 30 10:45:57 ppp 40734 PPPoE: rec'd ACNAME "KGTNON0881W"
                    Oct 30 10:45:55 ppp 40734 [wan_link0] PPPoE: Connecting to ''
                    Oct 30 10:45:55 kernel ng0: changing name to 'pppoe0'
                    Oct 30 10:45:55 php-cgi 478 rc.bootup: The command '/sbin/ifconfig 'pppoe0' inet6 -ifdisabled' returned exit code '1', the output was 'ifconfig: interface pppoe0 does not exist'
                    Oct 30 10:45:55 php-cgi 478 rc.bootup: The command '/sbin/ifconfig 'pppoe0' inet6 fe80::baca:3aff:fe8d:70b2%em0.35 delete' returned exit code '1', the output was 'ifconfig: interface pppoe0 does not exist'
                    Oct 30 10:45:55 php-cgi 478 rc.bootup: The command '/sbin/ifconfig 'pppoe0' inet6 ifdisabled' returned exit code '1', the output was 'ifconfig: interface pppoe0 does not exist

                    Do you have any idea where those first three rc.bootup lines are coming from? They are not there on the vm, and I don't have ipv6 enabled on any interfaces. I think I have same configs on both, but apparently not.

                    1 Reply Last reply Reply Quote 0
                    • stephenw10S
                      stephenw10 Netgate Administrator
                      last edited by

                      Hmm, looks like it's disabling the interface in order to remove a V6 address but failing because pppoe0 doesn't exist yet.

                      Is em0.35 the VLAN pppoe0 is on?

                      What is shown just before those lines?

                      R 2 Replies Last reply Reply Quote 0
                      • R
                        rpotter28 @stephenw10
                        last edited by

                        @stephenw10 said in Wireguard client Gateway disabled after reboot - service not starting:

                        Is em0.35 the VLAN pppoe0 is on?

                        Yes,

                        I will log at the logs,

                        1 Reply Last reply Reply Quote 0
                        • R
                          rpotter28 @stephenw10
                          last edited by

                          @stephenw10 This box has a 10GB LAGG, ix0 and ix1, with 7 vlans and 2 wg tunnels. I rebooted, so different log here.

                          In the logs I also see this: which is vlan 90 and I have no ipv6 config on any interfaces.

                          php-cgi 477 rc.bootup: The command '/sbin/ifconfig 'lagg0.90' inet6 delete' returned exit code '1', the output was 'ifconfig: ioctl (SIOCDIFADDR): Can't assign requested address'

                          And this:
                          Oct 30 12:58:46 kernel lagg0: IPv6 addresses on ix1 have been removed before adding it as a member to prevent IPv6 address scope violation.

                          Oct 30 12:58:45 kernel lagg0: IPv6 addresses on ix0 have been removed before adding it as a member to prevent IPv6 address scope violation.

                          All my vlans and interfaces are working as normal, just wireguard doesn't start on boot. Disables the gateways,

                          1 Reply Last reply Reply Quote 0
                          • stephenw10S
                            stephenw10 Netgate Administrator
                            last edited by

                            Hmm, maybe unrelated then. 🤔

                            I see similar lines for interfaces of that type where no IPv6 address is defined:

                            Oct 30 17:38:13 	kernel 		vlan0: changing name to 'lagg0.100'
Oct 30 17:38:13 	kernel 		lagg0: IPv6 addresses on igc1 have been removed before adding it as a member to prevent IPv6 address scope violation.
Oct 30 17:38:13 	php-cgi 	575 	rc.bootup: The command '/sbin/ifconfig 'lagg0.100' inet6 delete' returned exit code '1', the output was 'ifconfig: ioctl (SIOCDIFADDR): Can't assign requested address'

                            That shouldn't itself be an issue.

                            R 1 Reply Last reply Reply Quote 0
                            • R
                              rpotter28 @stephenw10
                              last edited by

                              @stephenw10 OK thanks.

                              So, I am back to these 3 lines (in bold) do not show-up on the vm, but they do on the bare metal:

                              Oct 30 10:46:03 php-fpm 411 /rc.newwanip: Default gateway setting WAN Gateway PPPoE as default.
                              Oct 30 10:46:02 php-fpm 411 /rc.newwanip: rc.newwanip: on (IP address: x.x.x.x) (interface: 0_WAN[wan]) (real interface: pppoe0).
                              Oct 30 10:46:02 php-fpm 411 /rc.newwanip: rc.newwanip: Info: starting on pppoe0.
                              Oct 30 10:46:01 ppp 40734 [wan] IFACE: Rename interface ng0 to pppoe0
                              Oct 30 10:46:01 check_reload_status 443 rc.newwanip starting pppoe0
                              Oct 30 10:45:57 ppp 40734 [wan_link0] PPPoE: connection successful
                              Oct 30 10:45:57 ppp 40734 PPPoE: rec'd ACNAME "KGTNON0881W"
                              Oct 30 10:45:55 ppp 40734 [wan_link0] PPPoE: Connecting to ''
                              Oct 30 10:45:55 kernel ng0: changing name to 'pppoe0'
                              Oct 30 10:45:55 php-cgi 478 rc.bootup: The command '/sbin/ifconfig 'pppoe0' inet6 -ifdisabled' returned exit code '1', the output was 'ifconfig: interface pppoe0 does not exist'
                              Oct 30 10:45:55 php-cgi 478 rc.bootup: The command '/sbin/ifconfig 'pppoe0' inet6 fe80::baca:3aff:fe8d:70b2%em0.35 delete' returned exit code '1', the output was 'ifconfig: interface pppoe0 does not exist'
                              Oct 30 10:45:55 php-cgi 478 rc.bootup: The command '/sbin/ifconfig 'pppoe0' inet6 ifdisabled' returned exit code '1', the output was 'ifconfig: interface pppoe0 does not exist

                              And I have no idea why that is :-)

                              R 1 Reply Last reply Reply Quote 0
                              • stephenw10S
                                stephenw10 Netgate Administrator
                                last edited by

                                Do you have a bridge configured on the bare metal box only?

                                R 1 Reply Last reply Reply Quote 0
                                • R
                                  rpotter28 @stephenw10
                                  last edited by

                                  @stephenw10 said in Wireguard client Gateway disabled after reboot - service not starting:

                                  Do you have a bridge configured on the bare metal box only?

                                  I am not bridging... It's a LACP LAGG.
                                  And no, I don't have a LAGG on the vm, no need to.

                                  1 Reply Last reply Reply Quote 0
                                  • stephenw10S
                                    stephenw10 Netgate Administrator
                                    last edited by

                                    Hmm, could be the lagg. That message is the system removing v6 addresses so they don't appear in the same layer 2. That could be a bridge or I guess a lagg. pfSense doesn't allow that for lagg interfaces but in FreeBSD it could be an issue. Either way that shouldn't be an issue.
                                    However I'm not sure why that would be trying to do it to a PPPoE interface. I imagine it may have inherited that from the interface it's on in some way Is em0 is use for something else?

                                    R 2 Replies Last reply Reply Quote 0
                                    • R
                                      rpotter28 @stephenw10
                                      last edited by

                                      @stephenw10 said in Wireguard client Gateway disabled after reboot - service not starting:

                                      em0 is use for something else?

                                      No sir, em0 just has vlan 35 for the pppoe connection. ISP requirement.

                                      All the vlans which includes the LAN are on the LAGG.

                                      This is similar to the vm, which also has 2 interfaces. WAN-vlan35-pppoe on one, and the other trunked for all vlans. Not a lagg, just one trunked hyper-v virtual nic.

                                      1 Reply Last reply Reply Quote 0
                                      • R
                                        rpotter28 @stephenw10
                                        last edited by rpotter28

                                        @stephenw10 said in Wireguard client Gateway disabled after reboot - service not starting:

                                        However I'm not sure why that would be trying to do it to a PPPoE interface

                                        I have given up, spent too much time on this. I am very sure I tripled checked everything, comparing the working vm to the bare metal settings in the GUI. I can find no rhyme or reason why.

                                        Following that thought, I analyzed and diffed the configs, still nothing stands out. So one has to conclude the issue in my bare metal install. My problem nobody else has I guess, but I can't find it.

                                        I thought of eliminating the em0 interface and just do it all on the LAGG, but that doesn't really make sense either for this issue?

                                        And I can't reinstall to test because it's on a HL licence. However, the vm is working perfectly, after 5 reboots now :-) So that proves it does work, and I am embarrassed that I can't make it work on my bare metal install.

                                        Richard

                                        1 Reply Last reply Reply Quote 0
                                        • R
                                          rpotter28 @rpotter28
                                          last edited by

                                          Today I removed vlan 35 from pfSense, and tagged it on the switch with a port pvid 35 back to pfSense. That got rid of the below errors, but wg gateways were still disabled. I find that interesting, but it did not fix the problem.

                                          Restarting dpinger with cron at boot does fix it, as posted in a thread here somewhere.
                                          @reboot root sleep 30 && /usr/local/sbin/pfSsh.php playback svc restart dpinger

                                          So, I am back to these 3 lines (in bold) do not show-up on the vm, but they do on the bare metal:
                                          Oct 30 10:45:55 php-cgi 478 rc.bootup: The command '/sbin/ifconfig 'pppoe0' inet6 -ifdisabled' returned exit code '1', the output was 'ifconfig: interface pppoe0 does not exist'
                                          Oct 30 10:45:55 php-cgi 478 rc.bootup: The command '/sbin/ifconfig 'pppoe0' inet6 fe80::baca:3aff:fe8d:70b2%em0.35 delete' returned exit code '1', the output was 'ifconfig: interface pppoe0 does not exist'
                                          Oct 30 10:45:55 php-cgi 478 rc.bootup: The command '/sbin/ifconfig 'pppoe0' inet6 ifdisabled' returned exit code '1', the output was 'ifconfig: interface pppoe0 does not exist

                                          1 Reply Last reply Reply Quote 0
                                          • First post
                                            Last post