welcome screen stuck on "booting..." and losing connection on WAN randomly



  • Hello,

    I'm not sure if these 2 items are related but it is the only "new" things I have seem in the past few weeks since the issue started.

    so my internet connection keep randomly dropping during the day, the only way to restore is to Disable and Re-enable WAN interface. When I check the PFsense box I see the following, there is not response to any keyboard strokes. Also the web interface eventually start up and I can access pfsense web interface without issues. (Console issue resolved* see second post)

    0_1541255689618_Screen Shot 2018-11-02 at 1.11.55 PM.png

    System log do not show anything out of the ordinary (that I can see). but it does show a lot of entries from Avahi.

    Nov 3 10:28:08	avahi-daemon	27509	Found user 'avahi' (UID 558) and group 'avahi' (GID 558).
    Nov 3 10:28:08	avahi-daemon	27509	Successfully dropped root privileges.
    Nov 3 10:28:08	avahi-daemon	27509	avahi-daemon 0.7 starting up.
    Nov 3 10:28:08	avahi-daemon	27509	WARNING: No NSS support for mDNS detected, consider installing nss-mdns!
    Nov 3 10:28:08	avahi-daemon	27509	Loading service file /usr/local/etc/avahi/services/sftp-ssh.service.
    Nov 3 10:28:08	avahi-daemon	27509	Loading service file /usr/local/etc/avahi/services/ssh.service.
    Nov 3 10:28:08	avahi-daemon	27509	Joining mDNS multicast group on interface ovpns1.IPv6 with address fe80::6a05:caff:fe3a:78de.
    Nov 3 10:28:08	avahi-daemon	27509	New relevant interface ovpns1.IPv6 for mDNS.
    Nov 3 10:28:08	avahi-daemon	27509	Joining mDNS multicast group on interface ovpns1.IPv4 with address 192.168.2.1.
    Nov 3 10:28:08	avahi-daemon	27509	New relevant interface ovpns1.IPv4 for mDNS.
    Nov 3 10:28:08	avahi-daemon	27509	Joining mDNS multicast group on interface igb1.10.IPv6 with address fe80::ae1f:6bff:fe10:d135.
    

    Any clues here where should I start looking?

    My hardware:
    SUPERMICRO MBD-X11SBA-LN4F-O
    SanDisk SSD PLUS 2.5" 120GB SATA III Internal Solid State Drive (SSD) SDSSDA-120G-G25
    Intel EXPI9301CTBLK 10/100/1000Mbps PCI-Express Network Adapter (WAN)
    Sandisk 16GB USB Drive
    Pfsense 2.4.4-RELEASE (amd64)
    Intel(R) Pentium(R) CPU N3700 @ 1.60GHz
    4 CPUs: 1 package(s) x 4 core(s)
    AES-NI CPU Crypto: Yes (active)

    Thanks


  • Netgate Administrator

    For the stuck at 'booting' issue do this:

    Intel Atom systems containing HD Graphics chipsets may experience console problems after the update. Affected systems will boot successfully, but fail to display console output after the boot menu. To fix the problem, add the following line to /boot/loader.conf.local to use the syscons console type:
    kern.vty=sc

    See: https://www.netgate.com/docs/pfsense/install/upgrade-guide.html#upgrading-from-versions-older-than-pfsense-2-4-4

    Steve



  • @stephenw10 said in welcome screen stuck on "booting..." and losing connection on WAN randomly:

    For the stuck at 'booting' issue do this:

    Intel Atom systems containing HD Graphics chipsets may experience console problems after the update. Affected systems will boot successfully, but fail to display console output after the boot menu. To fix the problem, add the following line to /boot/loader.conf.local to use the syscons console type:
    kern.vty=sc
    h
    See: https://www.netgate.com/docs/pfsense/install/upgrade-guide.html#upgrading-from-versions-older-than-pfsense-2-4-4

    Steve

    Steve, thank you, this definitely solved my condole issue. I guess this was not related to my WAN connection dropping randomly.


  • Netgate Administrator

    Indeed probably un-related.

    Those Avahi logs looks like symptoms of the WAN failure rather than a cause. You might run with Avahi disabled for a while just to try to catch some other log entries though.
    Also check the gateways log and the dhcp log (if your WAN is DHCP).
    You can also use the monitoring graphs to look for packet loss or latency increases in the run up to the disconnect. And/or increased CPU load or traffic throughput etc.

    Steve



  • Thank you sir, I had disable avahi and I can see some other error now.

    It looks like I am having some type of WAN failure, maybe my PCIe card is going bad?

    "php-fpm /rc.newwanip: rc.newwanip: Info: starting on igb2." keeps coming up

    </>
    ov 9 16:54:26 check_reload_status Restarting ipsec tunnels
    Nov 9 16:54:26 check_reload_status Restarting OpenVPN tunnels/interfaces
    Nov 9 16:54:26 check_reload_status Reloading filter
    Nov 9 16:54:27 php-fpm 29912 /rc.openvpn: Gateway, none 'available' for inet6, use the first one configured. 'WAN_DHCP6'
    Nov 9 16:54:27 php-fpm 29912 /rc.openvpn: OpenVPN: One or more OpenVPN tunnel endpoints may have changed its IP. Reloading endpoints that may use WAN_DHCP6.
    Nov 9 16:54:31 rc.gateway_alarm 51718 >>> Gateway alarm: WAN_DHCP (Addr:73.54.196.1 Alarm:0 RTT:9.495ms RTTsd:4.633ms Loss:16%)
    Nov 9 16:54:31 check_reload_status updating dyndns WAN_DHCP
    Nov 9 16:54:31 check_reload_status Restarting ipsec tunnels
    Nov 9 16:54:31 check_reload_status Restarting OpenVPN tunnels/interfaces
    Nov 9 16:54:31 check_reload_status Reloading filter
    Nov 9 16:54:32 php-fpm /rc.openvpn: Gateway, none 'available' for inet6, use the first one configured. 'WAN_DHCP6'
    Nov 9 16:54:32 php-fpm /rc.openvpn: OpenVPN: One or more OpenVPN tunnel endpoints may have changed its IP. Reloading endpoints that may use WAN_DHCP.
    Nov 9 17:00:44 check_reload_status rc.newwanip starting igb2
    Nov 9 17:00:45 php-fpm /rc.newwanip: rc.newwanip: Info: starting on igb2.
    Nov 9 17:00:45 php-fpm /rc.newwanip: rc.newwanip: on (IP address: 192.168.1.183) (interface: OPT1[opt1]) (real interface: igb2).
    Nov 9 17:00:46 php-fpm /rc.newwanip: Removing static route for monitor fe80::201:5cff:fe7e:6046 and adding a new route through fe80::201:5cff:fe7e:6046%em0
    Nov 9 17:00:47 php-fpm /rc.newwanip: Default gateway setting Interface WAN_DHCP Gateway as default.
    Nov 9 17:00:52 php-fpm /rc.newwanip: Resyncing OpenVPN instances for interface OPT1.
    Nov 9 17:00:52 php-fpm /rc.newwanip: Creating rrd update script
    Nov 9 17:00:54 php-fpm /rc.newwanip: pfSense package system has detected an IP change or dynamic WAN reconnection - 192.168.1.183 -> 192.168.1.183 - Restarting packages.
    Nov 9 17:00:54 check_reload_status Starting packages
    Nov 9 17:00:55 php-fpm /rc.start_packages: Restarting/Starting all packages.
    Nov 9 18:40:37 check_reload_status rc.newwanip starting igb2
    Nov 9 18:40:38 php-fpm /rc.newwanip: rc.newwanip: Info: starting on igb2.
    Nov 9 18:40:38 php-fpm /rc.newwanip: rc.newwanip: on (IP address: 192.168.1.183) (interface: OPT1[opt1]) (real interface: igb2).
    Nov 9 18:40:40 php-fpm /rc.newwanip: Removing static route for monitor fe80::201:5cff:fe7e:6046 and adding a new route through fe80::201:5cff:fe7e:6046%em0
    Nov 9 18:40:46 php-fpm /rc.newwanip: Resyncing OpenVPN instances for interface OPT1.
    Nov 9 18:40:46 php-fpm /rc.newwanip: Creating rrd update script
    Nov 9 18:40:48 php-fpm /rc.newwanip: pfSense package system has detected an IP change or dynamic WAN reconnection - 192.168.1.183 -> 192.168.1.183 - Restarting packages.
    Nov 9 18:40:48 check_reload_status Starting packages
    Nov 9 18:40:49 php-fpm 29912 /rc.start_packages: Restarting/Starting all packages.
    Nov 9 20:22:39 check_reload_status rc.newwanip starting igb2
    Nov 9 20:22:40 php-fpm 29912 /rc.newwanip: rc.newwanip: Info: starting on igb2.
    Nov 9 20:22:40 php-fpm 29912 /rc.newwanip: rc.newwanip: on (IP address: 192.168.1.183) (interface: OPT1[opt1]) (real interface: igb2).
    Nov 9 20:22:42 php-fpm 29912 /rc.newwanip: Removing static route for monitor fe80::201:5cff:fe7e:6046 and adding a new route through fe80::201:5cff:fe7e:6046%em0
    Nov 9 20:22:48 php-fpm 29912 /rc.newwanip: Resyncing OpenVPN instances for interface OPT1.
    Nov 9 20:22:48 php-fpm 29912 /rc.newwanip: Creating rrd update script
    Nov 9 20:22:50 php-fpm 29912 /rc.newwanip: pfSense package system has detected an IP change or dynamic WAN reconnection - 192.168.1.183 -> 192.168.1.183 - Restarting packages.
    Nov 9 20:22:50 check_reload_status Starting packages
    Nov 9 20:22:51 php-fpm 13931 /rc.start_packages: Restarting/Starting all packages.
    Nov 9 22:03:23 check_reload_status rc.newwanip starting igb2
    Nov 9 22:03:24 php-fpm /rc.newwanip: rc.newwanip: Info: starting on igb2.
    Nov 9 22:03:24 php-fpm /rc.newwanip: rc.newwanip: on (IP address: 192.168.1.183) (interface: OPT1[opt1]) (real interface: igb2).
    Nov 9 22:03:26 php-fpm /rc.newwanip: Removing static route for monitor fe80::201:5cff:fe7e:6046 and adding a new route through fe80::201:5cff:fe7e:6046%em0
    Nov 9 22:03:32 php-fpm /rc.newwanip: Resyncing OpenVPN instances for interface OPT1.
    Nov 9 22:03:32 php-fpm /rc.newwanip: Creating rrd update script
    Nov 9 22:03:34 php-fpm /rc.newwanip: pfSense package system has detected an IP change or dynamic WAN reconnection - 192.168.1.183 -> 192.168.1.183 - Restarting packages.
    Nov 9 22:03:34 check_reload_status Starting packages
    Nov 9 22:03:35 php-fpm 86079 /rc.start_packages: Restarting/Starting all packages.
    Nov 9 22:08:29 php-fpm 86079 /index.php: Successful login for user 'admin' from: 192.168.1.154 (Local Database)</>


  • Netgate Administrator

    What is igb2? OPT1 doesn't look like a WAN. Maybe it has a gateway on it incorrectly?

    Steve



  • Sir, You are correct, igb2 is not the WAN.

    Let me check and see what I assigned it to and post back. You may be correct and I had a gateway configure wrong. I had a MOBO failure about 2 month ago and I had to replaced it and re-imported my configuration. I may have a old config that not longer apply.



  • @maturola said in welcome screen stuck on "booting..." and losing connection on WAN randomly:

    Sir, You are correct, igb2 is not the WAN.

    Let me check and see what I assigned it to and post back. You may be correct and I had a gateway configure wrong. I had a MOBO failure about 2 month ago and I had to replaced it and re-imported my configuration. I may have an old config that not longer apply.

    Thank you for the help, it looks like that was in fact the issue, I believe that was a left over from importing the configuration from the failed MOBO, I believe I had OPT1 assigned to the old OpenVPN config.

    all errors are gone now and so far no interruption on connection for the past 12 hours.


  • Netgate Administrator

    Great. Thanks for the follow up.

    Steve