WAN only connects on unplug and plug back in
-
@randombits said in WAN only connects on unplug and plug back in:
and gets a WAN IP address
How's your WAN setup ?
DHCP for IPv4, or something else ?
Did you set up the IPv6 part, and if so, how ?Typically, you should have this - the most common setup :
(disregard the VPNS line)
Getting WAN IP doesn't make the connection work. It's just a part of what is needed.
A remote (ISP) gateway is as important as the IPv4 WAN.
Because you use pfSense, the DNS IPs - if you got any from your ISP - can be thrown in the waste-bucket or discarded. pfSense uses a resolver. -
@randombits Correct.
-
This post is deleted! -
Ok Logs - now redacted
The first is after pfsense reboot then toggled WAN UP/DOWN WAN in Proxmox is the second part.
Aug 19 14:27:56 php-fpm 397 /index.php: The command '/usr/local/etc/rc.d/vmware-kmod status' returned exit code '1', the output was 'Checking vmmemctl kernel module: not loaded Checking vmblock kernel module: not loaded' Aug 19 14:27:56 php-fpm 397 /index.php: The command '/usr/local/etc/rc.d/vmware-kmod status' returned exit code '1', the output was 'Checking vmmemctl kernel module: not loaded Checking vmblock kernel module: not loaded' Aug 19 14:27:56 php-fpm 397 /index.php: The command '/usr/local/etc/rc.d/vmware-kmod status' returned exit code '1', the output was 'Checking vmmemctl kernel module: not loaded Checking vmblock kernel module: not loaded' Aug 19 14:27:18 arpwatch 35925 bogon 0.0.0.0 fc:b4:67:d4:db:b0 Aug 19 14:27:18 arpwatch 35925 bogon 0.0.0.0 fc:b4:67:d4:db:b0 Aug 19 14:27:18 arpwatch 35925 bogon 0.0.0.0 fc:b4:67:d4:db:b0 Aug 19 14:27:15 check_reload_status 428 Reloading filter Aug 19 14:27:15 php_wg 59854 /usr/local/pkg/wireguard/includes/wg_service.inc: The command '/sbin/route -n6 get 'default' 2>/dev/null | /usr/bin/egrep 'flags: <.*PROTO.*>'' returned exit code '1', the output was '' Aug 19 14:27:15 php_wg 59854 /usr/local/pkg/wireguard/includes/wg_service.inc: Default gateway setting Interface WAN_DHCP Gateway as default. Aug 19 14:27:15 vnstatd 48634 Interface "tun_wg1" enabled. Aug 19 14:27:14 php_wg 59854 /usr/local/pkg/wireguard/includes/wg_service.inc: Configuration Change: (system): [pfSense-pkg-WireGuard] Applied package default settings as necessary. Aug 19 14:27:14 check_reload_status 428 Syncing firewall Aug 19 14:27:14 php_wg 59854 /usr/local/pkg/wireguard/includes/wg_service.inc: Configuration Change: (system): [pfSense-pkg-WireGuard] Installed Unbound ACL group (WireGuard). Aug 19 14:27:14 check_reload_status 428 Syncing firewall Aug 19 14:27:13 php_wg 59854 /usr/local/pkg/wireguard/includes/wg_service.inc: Configuration Change: (system): [pfSense-pkg-WireGuard] De-installed Unbound ACL group (WireGuard). Aug 19 14:27:13 check_reload_status 428 Syncing firewall Aug 19 14:27:13 php_wg 59854 /usr/local/pkg/wireguard/includes/wg_service.inc: Configuration Change: (system): [pfSense-pkg-WireGuard] Installed interface group (WireGuard). Aug 19 14:27:13 php_wg 59854 /usr/local/pkg/wireguard/includes/wg_service.inc: Configuration Change: (system): [pfSense-pkg-WireGuard] De-installed interface group (WireGuard). Aug 19 14:27:12 check_reload_status 428 Syncing firewall Aug 19 14:27:12 php_wg 59854 /usr/local/pkg/wireguard/includes/wg_service.inc: Configuration Change: (system): [pfSense-pkg-WireGuard] Installed earlyshellcmd(s). Aug 19 14:27:11 check_reload_status 428 Syncing firewall Aug 19 14:27:11 php_wg 59854 /usr/local/pkg/wireguard/includes/wg_service.inc: Configuration Change: (system): [pfSense-pkg-WireGuard] De-installed earlyshellcmd(s). Aug 19 14:27:10 kernel tun_wg1: link state changed to UP Aug 19 14:27:10 kernel wg1: changing name to 'tun_wg1' Aug 19 14:27:10 php_wg 59854 /usr/local/pkg/wireguard/includes/wg_service.inc: Configuration Change: (system): [pfSense-pkg-WireGuard] Applied package default settings as necessary. Aug 19 14:27:10 php_wg 59854 /usr/local/pkg/wireguard/includes/wg_service.inc: Configuration Change: (system): [pfSense-pkg-WireGuard] Installed Unbound ACL group (WireGuard). Aug 19 14:27:10 vnstatd 48634 Interface "tun_wg0" enabled. Aug 19 14:27:10 php_wg 59854 /usr/local/pkg/wireguard/includes/wg_service.inc: Configuration Change: (system): [pfSense-pkg-WireGuard] De-installed Unbound ACL group (WireGuard). Aug 19 14:27:09 php_wg 59854 /usr/local/pkg/wireguard/includes/wg_service.inc: Configuration Change: (system): [pfSense-pkg-WireGuard] Installed interface group (WireGuard). Aug 19 14:27:09 php_wg 59854 /usr/local/pkg/wireguard/includes/wg_service.inc: Configuration Change: (system): [pfSense-pkg-WireGuard] De-installed interface group (WireGuard). Aug 19 14:27:09 php_wg 59854 /usr/local/pkg/wireguard/includes/wg_service.inc: Configuration Change: (system): [pfSense-pkg-WireGuard] Installed earlyshellcmd(s). Aug 19 14:27:09 check_reload_status 428 Syncing firewall Aug 19 14:27:09 php_wg 59854 /usr/local/pkg/wireguard/includes/wg_service.inc: Configuration Change: (system): [pfSense-pkg-WireGuard] De-installed earlyshellcmd(s). Aug 19 14:27:09 kernel tun_wg0: link state changed to UP Aug 19 14:27:09 kernel wg0: changing name to 'tun_wg0' Aug 19 14:27:08 check_reload_status 428 Syncing firewall Aug 19 14:27:08 php_wg 59854 /usr/local/pkg/wireguard/includes/wg_service.inc: Configuration Change: (system): [pfSense-pkg-WireGuard] Applied package default settings as necessary. Aug 19 14:27:07 php_wg 59854 /usr/local/pkg/wireguard/includes/wg_service.inc: Configuration Change: (system): [pfSense-pkg-WireGuard] Installed Unbound ACL group (WireGuard). Aug 19 14:27:07 php_wg 59854 /usr/local/pkg/wireguard/includes/wg_service.inc: Configuration Change: (system): [pfSense-pkg-WireGuard] De-installed Unbound ACL group (WireGuard). Aug 19 14:27:07 php_wg 59854 /usr/local/pkg/wireguard/includes/wg_service.inc: Configuration Change: (system): [pfSense-pkg-WireGuard] Installed interface group (WireGuard). Aug 19 14:27:07 php_wg 59854 /usr/local/pkg/wireguard/includes/wg_service.inc: Configuration Change: (system): [pfSense-pkg-WireGuard] De-installed interface group (WireGuard). Aug 19 14:27:07 login 85707 login on ttyv0 as root Aug 19 14:27:07 check_reload_status 428 Syncing firewall Aug 19 14:27:06 php_wg 59854 /usr/local/pkg/wireguard/includes/wg_service.inc: Configuration Change: (system): [pfSense-pkg-WireGuard] Installed earlyshellcmd(s). Aug 19 14:27:06 check_reload_status 428 Syncing firewall Aug 19 14:27:06 php_wg 59854 /usr/local/pkg/wireguard/includes/wg_service.inc: Configuration Change: (system): [pfSense-pkg-WireGuard] De-installed earlyshellcmd(s). Aug 19 14:27:06 php 77093 [pfBlockerNG] DNSBL parser daemon started Aug 19 14:27:05 php_pfb 70160 [pfBlockerNG] filterlog daemon started Aug 19 14:27:05 root 72945 Bootup complete Aug 19 14:27:05 tail_pfb 69910 [pfBlockerNG] Firewall Filter Service started Aug 19 14:27:05 lighttpd_pfb 67801 [pfBlockerNG] DNSBL Webserver started Aug 19 14:27:05 vnstatd 48634 Interface "tun_wg0" disabled. Aug 19 14:27:05 vnstatd 48634 Interface "tun_wg1" disabled. Aug 19 14:27:05 php_pfb 65170 [pfBlockerNG] filterlog daemon stopped Aug 19 14:27:05 lighttpd_pfb 64162 [pfBlockerNG] DNSBL Webserver stopped Aug 19 14:27:05 tail_pfb 63685 [pfBlockerNG] Firewall Filter Service stopped Aug 19 14:27:04 vnstatd 62273 Error: pidfile "/var/run/vnstat/vnstat.pid" lock failed (Resource temporarily unavailable), exiting. Aug 19 14:27:04 kernel tun_wg1: link state changed to DOWN Aug 19 14:27:04 kernel tun_wg0: link state changed to DOWN Aug 19 14:27:04 php-fpm 397 /rc.start_packages: Configuration Change: (system): [pfSense-pkg-WireGuard] Applied package default settings as necessary. Aug 19 14:27:04 php-fpm 397 /rc.start_packages: Configuration Change: (system): [pfSense-pkg-WireGuard] Installed Unbound ACL group (WireGuard). Aug 19 14:27:04 php-fpm 397 /rc.start_packages: Configuration Change: (system): [pfSense-pkg-WireGuard] De-installed Unbound ACL group (WireGuard). Aug 19 14:27:04 php-fpm 397 /rc.start_packages: Configuration Change: (system): [pfSense-pkg-WireGuard] Installed interface group (WireGuard). Aug 19 14:27:03 php-fpm 397 /rc.start_packages: Configuration Change: (system): [pfSense-pkg-WireGuard] De-installed interface group (WireGuard). Aug 19 14:27:03 check_reload_status 428 Syncing firewall Aug 19 14:27:03 check_reload_status 428 Syncing firewall Aug 19 14:27:03 php-fpm 397 /rc.start_packages: Configuration Change: (system): [pfSense-pkg-WireGuard] Installed earlyshellcmd(s). Aug 19 14:26:58 php-fpm 397 /rc.start_packages: Configuration Change: (system): [pfSense-pkg-WireGuard] De-installed earlyshellcmd(s). Aug 19 14:26:58 vnstatd 48634 Interface "em0" disabled. Aug 19 14:26:58 vnstatd 48634 Interface "em0.10" disabled. Aug 19 14:26:58 vnstatd 48634 Interface "em0.20" disabled. Aug 19 14:26:58 vnstatd 48634 Interface "em0.30" disabled. Aug 19 14:26:58 vnstatd 48634 Interface "em0.40" disabled. Aug 19 14:26:58 vnstatd 48634 Interface "em0.999" disabled. Aug 19 14:26:58 vnstatd 48634 Interface "em1" disabled. Aug 19 14:26:58 vnstatd 48634 Interface "pppoe0" disabled. Aug 19 14:26:58 vnstatd 48634 Monitoring (22): vtnet1 (10000 Mbit) vtnet0.999 (10000 Mbit) vtnet0.40 (10000 Mbit) vtnet0.30 (10000 Mbit) vtnet0.20 (10000 Mbit) vtnet0.10 (10000 Mbit) vtnet0 (10000 Mbit) tun_wg1 (1000 Mbit) tun_wg0 (1000 Mbit) pppoe0 (1000 Mbit) pfsync0 (1000 Mbit) pflog0 (1000 Mbit) ovpns1 (1000 Mbit) ovpnc2 (1000 Mbit) enc0 (1000 Mbit) em1 (1000 Mbit) em0.999 (1000 Mbit) em0.40 (1000 Mbit) em0.30 (1000 Mbit) em0.20 (1000 Mbit) em0.10 (1000 Mbit) em0 (1000 Mbit) Aug 19 14:26:58 vnstatd 48634 Data retention: 48 5MinuteHours, 4 HourlyDays, 62 DailyDays, 25 MonthlyMonths, -1 YearlyYears, 20 TopDayEntries Aug 19 14:26:58 vnstatd 48634 vnStat daemon 2.11 (pid:48634 uid:0 gid:0, SQLite 3.43.1) Aug 19 14:26:57 vnstatd 34674 SIGTERM received, exiting. Aug 19 14:26:53 php-fpm 56866 /index.php: The command '/usr/local/etc/rc.d/vmware-kmod status' returned exit code '1', the output was 'Checking vmmemctl kernel module: not loaded Checking vmblock kernel module: not loaded' Aug 19 14:26:52 php-fpm 56866 /index.php: The command '/usr/local/etc/rc.d/vmware-kmod status' returned exit code '1', the output was 'Checking vmmemctl kernel module: not loaded Checking vmblock kernel module: not loaded' Aug 19 14:26:52 php-fpm 56866 /index.php: The command '/usr/local/etc/rc.d/vmware-kmod status' returned exit code '1', the output was 'Checking vmmemctl kernel module: not loaded Checking vmblock kernel module: not loaded' Aug 19 14:26:51 sshguard 39798 Now monitoring attacks. Aug 19 14:26:51 php-fpm 56866 /index.php: Successful login for user 'admin' from: 192.168.2.21 (Local Database) Aug 19 14:26:50 kernel ovpnc2: promiscuous mode disabled Aug 19 14:26:50 kernel ovpnc2: promiscuous mode enabled Aug 19 14:26:50 arpwatch 37518 Link layer type 0 not ethernet or fddi Aug 19 14:26:50 kernel vtnet1: promiscuous mode enabled Aug 19 14:26:50 arpwatch 38684 listening on vtnet1 Aug 19 14:26:50 arpwatch 38300 pcap open : : No such device exists (No such device exists) Aug 19 14:26:50 kernel vtnet0.999: promiscuous mode enabled Aug 19 14:26:50 arpwatch 37953 listening on vtnet0.999 Aug 19 14:26:50 arpwatch 37332 listening on vtnet0.30 Aug 19 14:26:50 kernel vtnet0.30: promiscuous mode enabled Aug 19 14:26:50 kernel vtnet0.20: promiscuous mode enabled Aug 19 14:26:50 arpwatch 36727 listening on vtnet0.20 Aug 19 14:26:50 kernel vtnet0.10: promiscuous mode enabled Aug 19 14:26:50 arpwatch 36085 listening on vtnet0.10 Aug 19 14:26:50 arpwatch 35925 listening on vtnet0 Aug 19 14:31:05 php-fpm 58327 /rc.newwanip: rc.newwanip: on (IP address: xx.xx.xx.xx) (interface: PROTONVPN_NL[opt4]) (real interface: ovpnc2). Aug 19 14:31:05 php-fpm 58327 /rc.newwanip: rc.newwanip: Info: starting on ovpnc2. Aug 19 14:31:04 check_reload_status 428 rc.newwanip starting ovpnc2 Aug 19 14:31:03 kernel ovpnc2: link state changed to UP Aug 19 14:31:00 php-cgi 52317 servicewatchdog_cron.php: Service Watchdog detected service dpinger stopped. Restarting dpinger (Gateway Monitoring Daemon) Aug 19 14:30:48 php-fpm 56866 /rc.newwanip: rc.newwanip: on (IP address: xxx.xxx.xxx.xxx) (interface: WAN[wan]) (real interface: vtnet1). Aug 19 14:30:48 php-fpm 56866 /rc.newwanip: rc.newwanip: Info: starting on vtnet1. Aug 19 14:30:48 check_reload_status 428 Restarting IPsec tunnels Aug 19 14:30:48 php-fpm 396 /rc.linkup: The command '/sbin/route -n6 get 'default' 2>/dev/null | /usr/bin/egrep 'flags: <.*PROTO.*>'' returned exit code '1', the output was '' Aug 19 14:30:48 arpwatch 38684 bogon xxx.xxx.xxx.xxx d4:5a:3f:3c:db:09 Aug 19 14:30:47 check_reload_status 428 rc.newwanip starting vtnet1 Aug 19 14:30:47 php-fpm 396 /rc.linkup: The command '/sbin/ifconfig 'vtnet1' media 'autoselect'' returned exit code '1', the output was 'ifconfig: SIOCSIFMEDIA (media): Operation not supported' Aug 19 14:30:47 php-fpm 396 /rc.linkup: HOTPLUG: Configuring interface wan Aug 19 14:30:47 php-fpm 396 /rc.linkup: DEVD Ethernet attached event for wan Aug 19 14:30:47 php-fpm 396 /rc.linkup: Hotplug event detected for WAN(wan) dynamic IP address (4: dhcp) Aug 19 14:30:47 check_reload_status 428 Reloading filter Aug 19 14:30:41 kernel vtnet1: link state changed to UP Aug 19 14:30:41 check_reload_status 428 Linkup starting vtnet1 Aug 19 14:30:39 php-fpm 58327 /rc.openvpn: OpenVPN: One or more OpenVPN tunnel endpoints may have changed IP addresses. Reloading endpoints that may use WAN_DHCP. Aug 19 14:30:39 php-fpm 58327 /rc.openvpn: The command '/sbin/route -n6 get 'default' 2>/dev/null | /usr/bin/egrep 'flags: <.*PROTO.*>'' returned exit code '1', the output was '' Aug 19 14:30:38 check_reload_status 428 Reloading filter Aug 19 14:30:38 check_reload_status 428 Restarting OpenVPN tunnels/interfaces Aug 19 14:30:38 check_reload_status 428 Restarting IPsec tunnels Aug 19 14:30:38 check_reload_status 428 updating dyndns WAN_DHCP Aug 19 14:30:38 rc.gateway_alarm 81356 >>> Gateway alarm: WAN_DHCP (Addr:xxx.xxx.xxx.xxx Alarm:down RTT:0ms RTTsd:0ms Loss:100%) Aug 19 14:30:37 php-fpm 397 /rc.linkup: DEVD Ethernet detached event for wan Aug 19 14:30:37 php-fpm 397 /rc.linkup: Hotplug event detected for WAN(wan) dynamic IP address (4: dhcp) Aug 19 14:30:35 check_reload_status 428 Linkup starting vtnet1 Aug 19 14:30:35 kernel vtnet1: link state changed to DOWN
The WAN is set like this
My ISP doesn't supply any DNS and I have a fixed IP
It's all very standard no IPv6 although my ISP does supply it - a /48 block I think.
-
@randombits said in WAN only connects on unplug and plug back in:
It's all very standard no IPv6 although my ISP does supply it - a /48 block I think.
As soon as your get your connection issues sorted out, you know what will be next on the list
-
Before you replug the WAN is there a valid default route present in Diag > Routes?
-
@Gertjan I know, I'm scared of IPv6 ...
-
@stephenw10 No, vtnet1 doesn't appear to after I toggle the WAN. Scrub that I see my IP listed further down with the lo0 interface
After reboot
And after toggle
And the Gateway tab
-
This post is deleted! -
Hmm, OK well no default gateway is what's causing the problem. To confirm that try resaving the gateways page rather then reconnecting the WAN and make sure that also fixes it.
But what is causing that missing gateway... Any errors in the gateway or routing log after reboot?
-
@stephenw10 Nothing in the routing log and repeated below in the gateway log
Aug 19 17:42:31 dpinger 64161 WAN_DHCP XXX.XXX.XXX.XXX: sendto error: 65
until I toggle the WAN.
(I wonder if anything in Proxmox is the issue ? I can't see why. Although I do split the LAN/WAN in the switch as the VM only has one ethernet port, this has never been an issue before)
-
I would first try disabling some of those packages as a test. You seem to have a bunch of references to old interfaces that are no longer present. And it's also trying to load vmware tools in Proxmox.
This looks to be the issue but it's not clear what the cause is:
Aug 19 14:27:15 php_wg 59854 /usr/local/pkg/wireguard/includes/wg_service.inc: The command '/sbin/route -n6 get 'default' 2>/dev/null | /usr/bin/egrep 'flags: <.*PROTO.*>'' returned exit code '1', the output was '' Aug 19 14:27:15 php_wg 59854 /usr/local/pkg/wireguard/includes/wg_service.inc: Default gateway setting Interface WAN_DHCP Gateway as default.
I'd guess it's a timing issue due to one of those previous errors. Disabling some packages to prevent that may allow it to comes up with a default route and if so we can dig into that.
-
@stephenw10 Morning, I've removed Vmtools and removed Wireguard that made no difference. Where do I remove the those orphaned interfaces em0, pppoe, em0.xx etc ?
I've just noticed watching the boot screen I see
Configuring WAN interface...[fib_algo] Inet.0 (bsearch4#28) rebuild_fb_flm: switching algo to radix4_lockless
-
That message at boot is fine, not an issue.
The old interfaces look to be in vmstat.
-
@stephenw10 What I meant was where are they located in the cli (putty) so I remove them if that's the right thing to do. I did use pppoe with my old ADSL ISP, I just swapped from pppoe over to DHCP for the WAN
-
They are in the config file (/conf/config.xml) but you should be able to remove them in the gui by re-configuring vmstat.
-
@stephenw10 I was the right place I assumed, but a bit daunted pruning the xml. What do you mean by re-configuring the xmstat ?
The interfaces look like this
Does pfSense assume the WAN is vtnet0 or vtnet1 ? I vaguely remember em0 and em1 would get mixed up
-
@randombits said in WAN only connects on unplug and plug back in:
Does pfSense assume the WAN is vtnet0 or vtnet1 ?
It has no preset config for vtnet, the user has to assign those interfaces manually.
If you have em NICs it will initially use em0 as WAN unless you set it differently.
vmstat is used by a package you have installed so probably darkstat or Banwidthd. If you resave the config in that package it should remove those old NIC references.
-
@stephenw10 I removed both darkstat & bandwidthd and reinstalled them but the orphaned interfaces are still in the log.
I just looked in config.xml and there's no reference to emX.XXX etc only vtnet's
Just spotted this, but not sure how relevant it is
Aug 20 18:28:30 php-cgi 453 rc.bootup: Default gateway setting Interface WAN_DHCP Gateway as default. Aug 20 18:28:30 php-cgi 453 rc.bootup: The command '/sbin/route -n6 get 'default' 2>/dev/null | /usr/bin/egrep 'flags: <.*PROTO.*>'' returned exit code '1', the output was '' Aug 20 18:28:31 kernel done. Aug 20 18:28:34 php-cgi 453 rc.bootup: sync unbound done. Aug 20 18:28:35 kernel done.
One thing, many thanks for bearing with me - I do very much appreciate it !
-
Yes, that's what I pointed out above. It may or may not be the issue but it's certainly suspicious.
What interface references are you seeing now?
What packages do you still have installed?