Navigation

    Netgate Discussion Forum
    • Register
    • Login
    • Search
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search

    Solid Stable [2.4.5-RELEASE] - Dropping LAN, Requires Reboot

    Hardware
    2
    4
    125
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • I
      Imburr last edited by Imburr

      My pfSense has been up an running over a year. I keep it updated, backed up, etc. After making no changes, last week I lost internet. Traced it down to the basement, and rebooted a switch. Ping to the firewall LAN gateway came up, but still no internet. Rebooted the firewall and we were back in business.

      This happened 5 or 6 more times over 3 days. I thought it might be the switch. I ordered and installed a brand new one. After 2 days, the problem happened again. When I lose network, I reboot the switch, and I can ping LAN devices but no internet. I then reboot the firewall, and internet comes back up.

      Prior to rebooting the firewall I cannot access the GUI or ping any IP on the device.

      What can I do to troubleshoot this?

      This is the logged events from when I rebooted the switch:

      Jan 22 13:42:33	php-fpm	59678	/rc.linkup: Hotplug event detected for VLAN101(opt3) static IP (192.168.101.1 )
      Jan 22 13:42:32	check_reload_status		Linkup starting igb1.101
      Jan 22 13:42:32	check_reload_status		Linkup starting igb1
      Jan 22 13:42:32	kernel		igb1.101: link state changed to UP
      Jan 22 13:42:32	kernel		igb1: link state changed to UP
      Jan 22 13:42:27	check_reload_status		Reloading filter
      Jan 22 13:42:27	check_reload_status		Reloading filter
      Jan 22 13:42:27	avahi-daemon	13670	Interface igb1.IPv4 no longer relevant for mDNS.
      Jan 22 13:42:27	avahi-daemon	13670	Leaving mDNS multicast group on interface igb1.IPv4 with address 10.10.10.1.
      Jan 22 13:42:27	avahi-daemon	13670	Joining mDNS multicast group on interface igb1.IPv4 with address 10.10.10.1.
      Jan 22 13:42:27	avahi-daemon	13670	Leaving mDNS multicast group on interface igb1.IPv4 with address 192.168.2.1.
      Jan 22 13:42:27	php-fpm	352	/rc.linkup: Hotplug event detected for VLAN101(opt3) static IP (192.168.101.1 )
      Jan 22 13:42:27	php-fpm	59678	/rc.linkup: DEVD Ethernet detached event for lan
      Jan 22 13:42:26	check_reload_status		Linkup starting igb1.101
      Jan 22 13:42:26	kernel		igb1.101: link state changed to DOWN
      Jan 22 13:42:26	kernel		igb1: link state changed to DOWN
      Jan 22 13:42:26	check_reload_status		Linkup starting igb1
      Jan 22 12:30:45	php-cgi		rc.update_urltables: /etc/rc.update_urltables: pfB_DNSBLIP_v4 does not need updating.
      Jan 22 12:30:45	php-cgi		rc.update_urltables: /etc/rc.update_urltables: pfB_Whitelist_v4 does not need updating.
      Jan 22 12:30:45	php-cgi		rc.update_urltables: /etc/rc.update_urltables: pfB_Level4_v4 does not need updating.
      

      And here is the log from when I reboot the pfSense:

      Jan 22 13:57:59	kernel		Origin="GenuineIntel" Id=0x40651 Family=0x6 Model=0x45 Stepping=1
      Jan 22 13:57:59	kernel		CPU: Intel(R) Core(TM) i3-4030U CPU @ 1.90GHz (1895.65-MHz K8-class CPU)
      Jan 22 13:57:59	kernel		VT(vga): resolution 640x480
      Jan 22 13:57:59	kernel		FreeBSD clang version 8.0.1 (tags/RELEASE_801/final 366581) (based on LLVM 8.0.1)
      Jan 22 13:57:59	kernel		root@buildbot1-nyi.netgate.com:/build/ce-crossbuild-245/obj/amd64/YNx4Qq3j/build/ce-crossbuild-245/sources/FreeBSD-src/sys/pfSense amd64
      Jan 22 13:57:59	kernel		FreeBSD 11.3-STABLE #243 abf8cba50ce(RELENG_2_4_5): Tue Jun 2 17:53:37 EDT 2020
      Jan 22 13:57:59	kernel		FreeBSD is a registered trademark of The FreeBSD Foundation.
      Jan 22 13:57:59	kernel		The Regents of the University of California. All rights reserved.
      Jan 22 13:57:59	kernel		Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
      Jan 22 13:57:59	kernel		Copyright (c) 1992-2020 The FreeBSD Project.
      Jan 22 13:57:59	syslogd		kernel boot file is /boot/kernel/kernel
      Jan 22 13:57:01	syslogd		exiting on signal 15
      Jan 22 13:43:16	check_reload_status		Reloading filter
      Jan 22 13:43:16	php-fpm	351	/rc.newwanipv6: Removing static route for monitor fe80::201:5cff:fead:ec46 and adding a new route through fe80::201:5cff:fead:ec46%igb0
      Jan 22 13:42:41	php-fpm	351	/rc.newwanipv6: rc.newwanipv6: on (IP address: 2001:XXX:d81:3528:645c) (interface: wan) (real interface: igb0).
      Jan 22 13:42:41	php-fpm	351	/rc.newwanipv6: rc.newwanipv6: Info: starting on igb0.
      Jan 22 13:42:40	check_reload_status		Reloading filter
      Jan 22 13:42:40	check_reload_status		updating dyndns lan
      Jan 22 13:42:40	rtsold		Received RA specifying route fe80::201:5cff:fead:ec46 for interface wan(igb0)
      Jan 22 13:42:34	php-fpm	59678	/rc.newwanip: rc.newwanip: on (IP address: 192.168.101.1) (interface: VLAN101[opt3]) (real interface: igb1.101).
      Jan 22 13:42:34	php-fpm	59678	/rc.newwanip: rc.newwanip: Info: starting on igb1.101.
      Jan 22 13:42:33	check_reload_status		Restarting ipsec tunnels
      Jan 22 13:42:33	check_reload_status		Reloading filter
      Jan 22 13:42:33	check_reload_status		rc.newwanip starting igb1.101
      
      I 1 Reply Last reply Reply Quote 0
      • I
        Imburr @Imburr last edited by

        I also see this crazy gap in the log, though the internet was up throughout this gap.

        Jan 22 12:30:45	php-cgi		rc.update_urltables: /etc/rc.update_urltables: pfB_Top_v4 does not need updating.
        Jan 22 12:30:45	php-cgi		rc.update_urltables: /etc/rc.update_urltables: Starting URL table alias updates
        Jan 22 12:30:00	php-cgi		rc.update_urltables: /etc/rc.update_urltables: Sleeping for 45 seconds.
        Jan 22 12:30:00	php-cgi		rc.update_urltables: /etc/rc.update_urltables: Starting up.
        Jan 22 04:00:19	php		/usr/local/sbin/execacb.php: End of configuration backup to https://acb.netgate.com/save (success).
        Jan 22 04:00:00	php		/usr/local/sbin/execacb.php: Beginning configuration backup to https://acb.netgate.com/save
        Jan 22 04:00:00	check_reload_status		Syncing firewall
        Jan 22 01:01:05	php-cgi		rc.dyndns.update: phpDynDNS (plex.domain.com): No change in my IP address and/or 25 days has not passed. Not updating dynamic DNS entry.
        Jan 22 01:01:04	php-cgi		rc.dyndns.update: phpDynDNS (terraria.domain.com): No change in my IP address and/or 25 days has not passed. Not updating dynamic DNS entry.
        Jan 22 01:01:02	php-cgi		rc.dyndns.update: phpDynDNS (pfsense.domain.com): No change in my IP address and/or 25 days has not passed. Not updating dynamic DNS entry.
        
        1 Reply Last reply Reply Quote 0
        • I
          Imburr last edited by

          Also found these around the same time.

          Jan 22 13:58:14	dpinger		send_interval 500ms loss_interval 2000ms time_period 60000ms report_interval 0ms data_len 1 alert_interval 1000ms latency_alarm 500ms loss_alarm 20% dest_addr 192.168.10.1 bind_addr 192.168.10.1 identifier "OVPN_VPNV4 "
          Jan 22 13:58:14	dpinger		send_interval 500ms loss_interval 2000ms time_period 60000ms report_interval 0ms data_len 1 alert_interval 1000ms latency_alarm 500ms loss_alarm 20% dest_addr fe80::201:XXX:ec46%igb0 bind_addr fe80::XXX:78f%igb0 identifier "WAN1_DHCP6 "
          Jan 22 13:58:14	dpinger		send_interval 500ms loss_interval 2000ms time_period 60000ms report_interval 0ms data_len 1 alert_interval 1000ms latency_alarm 500ms loss_alarm 20% dest_addr 73.XXX.XXX.1 bind_addr 73.XXX.XXX.185 identifier "WAN1_DHCP "
          Jan 22 13:58:08	dpinger		send_interval 500ms loss_interval 2000ms time_period 60000ms report_interval 0ms data_len 1 alert_interval 1000ms latency_alarm 500ms loss_alarm 20% dest_addr 192.168.10.1 bind_addr 192.168.10.1 identifier "OVPN_VPNV4 "
          Jan 22 13:58:08	dpinger		send_interval 500ms loss_interval 2000ms time_period 60000ms report_interval 0ms data_len 1 alert_interval 1000ms latency_alarm 500ms loss_alarm 20% dest_addr 73.XXX.XXX.1 bind_addr 73.XXX.XXX.185 identifier "WAN1_DHCP "
          Jan 22 13:58:07	dpinger		send_interval 500ms loss_interval 2000ms time_period 60000ms report_interval 0ms data_len 1 alert_interval 1000ms latency_alarm 500ms loss_alarm 20% dest_addr 192.168.10.1 bind_addr 192.168.10.1 identifier "OVPN_VPNV4 "
          Jan 22 13:58:07	dpinger		send_interval 500ms loss_interval 2000ms time_period 60000ms report_interval 0ms data_len 1 alert_interval 1000ms latency_alarm 500ms loss_alarm 20% dest_addr 73.XXX.XXX.1 bind_addr 73.XXX.XXX.185 identifier "WAN1_DHCP "
          Jan 22 13:58:05	dpinger		send_interval 500ms loss_interval 2000ms time_period 60000ms report_interval 0ms data_len 1 alert_interval 1000ms latency_alarm 500ms loss_alarm 20% dest_addr 192.168.10.1 bind_addr 192.168.10.1 identifier "OVPN_VPNV4 "
          Jan 22 13:58:05	dpinger		send_interval 500ms loss_interval 2000ms time_period 60000ms report_interval 0ms data_len 1 alert_interval 1000ms latency_alarm 500ms loss_alarm 20% dest_addr 73.XXX.XXX.1 bind_addr 73.XXX.XXX.185 identifier "WAN1_DHCP "
          Jan 22 13:43:16	dpinger		send_interval 500ms loss_interval 2000ms time_period 60000ms report_interval 0ms data_len 1 alert_interval 1000ms latency_alarm 500ms loss_alarm 20% dest_addr 192.168.10.1 bind_addr 192.168.10.1 identifier "OVPN_VPNV4 "
          Jan 22 13:43:16	dpinger		send_interval 500ms loss_interval 2000ms time_period 60000ms report_interval 0ms data_len 1 alert_interval 1000ms latency_alarm 500ms loss_alarm 20% dest_addr fe80::201:XXX:ec46%igb0 bind_addr fe80::XXX:78f%igb0 identifier "WAN1_DHCP6 "
          Jan 22 13:43:16	dpinger		send_interval 500ms loss_interval 2000ms time_period 60000ms report_interval 0ms data_len 1 alert_interval 1000ms latency_alarm 500ms loss_alarm 20% dest_addr 73.XXX.XXX.1 bind_addr 73.XXX.XXX.185 identifier "WAN1_DHCP "
          
          1 Reply Last reply Reply Quote 0
          • stephenw10
            stephenw10 Netgate Administrator last edited by

            If nothing is happening then nothing gets logged. A low rate system log is often a sign of good stability not a problem.

            Those last logs are all dpinger starting which happens if an interface goes down and then comes back up or you apply a change to it etc.

            I see in there that you have an OpenVPN gateway. Do you have the WAN gateway set as default in System > Routing > Gateways? If not it might have switched to a different default route when your WAN went down resulting in no connectivity.

            I don't see that logged specifically but those logs are incomplete.

            Steve

            1 Reply Last reply Reply Quote 0
            • First post
              Last post

            Products

            • Platform Overview
            • TNSR
            • pfSense
            • Appliances

            Services

            • Training
            • Professional Services

            Support

            • Subscription Plans
            • Contact Support
            • Product Lifecycle
            • Documentation

            News

            • Media Coverage
            • Press
            • Events

            Resources

            • Blog
            • FAQ
            • Find a Partner
            • Resource Library
            • Security Information

            Company

            • About Us
            • Careers
            • Partners
            • Contact Us
            • Legal
            Our Mission

            We provide leading-edge network security at a fair price - regardless of organizational size or network sophistication. We believe that an open-source security model offers disruptive pricing along with the agility required to quickly address emerging threats.

            Subscribe to our Newsletter

            Product information, software announcements, and special offers. See our newsletter archive to sign up for future newsletters and to read past announcements.

            © 2021 Rubicon Communications, LLC | Privacy Policy