Odd issues started lately
-
Hello all,
I am running pfSense latest version on a Lenovo SFF PC in the living room with a quad port intel NIC. It's been running well for several months, now all of a sudden we are experiencing lots of trouble.
I will be doing stuff on the network, and all of a sudden I have lost connection. Usually when this happens, I cannot ping the gateway from OPT1 or LAN network until I hot-plug.
The system log gives me no information about what happened, or why it has stopped working. I have had this happen quite often over the last few days and I have had enough. I lose internet connection on the LAN networks and sometimes even the WAN will act up until I hot-plug it. This usually requires me to reset my modem to re-lease the IP address from my ISP.
I am wondering if this could be an issue with the drivers, or something to do with the card and going to sleep? What information do you need to help me look into this problem, and where is the best place to find information on these problems.
-
If it actually loses connection I'd be amazed if nothing is logged at all, in any log.
Does it show the WAN gateway go down?
Steve
-
@stephenw10 The only thing on the WAN gateway I see is this:
send_interval 500ms loss_interval 2000ms time_period 60000ms report_interval 0ms data_len 1 alert_interval 1000ms latency_alarm 500ms loss_alarm 20% dest_addr 8.8.8.8 bind_addr 174.109.12.*** identifier "WAN_DHCP "
every once in awhile. This actually happened after I began the hot plug of the other LAN ethernet cables.
I shouldn't be losing access to the router even if the WAN is down though, right? I lose all access to the router, but I can still ping some other devices in the subnet. I cant figure anything but power saving issues or else the card may be going bad... but all 4 ports at once?!
-
That log entry is dpinger, the gateway monitoring service, starting. You would normally expect to see that when the WAN is connected or the WAN settings are re-saved etc.
It is not an alarm log showing it went down.If you connect to the pfSense console can you ping out from there when this happens?
If you run 'ifconfig -a' do the interfaces still show as up and linked?
There may be an error shown at the console which isn't logged.
Steve
-
@stephenw10 I haven't tried from the console because I couldn't SSH into it. Next time this happens I will try to hookup a monitor and do it from the console. When I do the hot-plug of the OPT1 network it shows up in the log as unplugged, and re-enabled. There weren't any information or logs before then though to state it was down; that is what I am confused by.
-
Can we see the actual logs shown?
-
@stephenw10 The issue kind of resolved for a little while, or I haven't noticed it. Then yesterday while I am at work my wife began complaining again. This time when I got home I realized the issue was only on one LAN port.
My upstairs is where I run the small "lab" and the OPT1 port first goes into a PowerConnect 2284 managed switch; that switch is LAG'd to a PowerConnect 5548p for POE devices. My desktop was hooked up to the PowerConnect 2284 and I was able to connect to the switch still, but could NOT ping the pfSense router.
I had to reboot the switch, or unplug and replug the cable for that interface in order for the router to be accessible again. The other LAN port was still working fine.
Going to look into getting the logs and removing any information I need to, stand-by.
-
Hello all,
My issue went away for quite awhile. Now lately I upgraded to newest version and after that reboot I have issues again.
The internet on the OPT1 is dying intermittently and that feeds my upstairs managed switches. I cannot figure it out, but when the issue occurs it cannot reach the router. Everything on the switch is fine and can be reached.
Once I hot plug the wire from the router to the managed switch the network comes back up. What is this doing that isn't happening automatically in the router? There isn't ANYthing in the logs until I hotplug the ethernet cable.
-
@justinseo But you dont provide any logs so helping is difficult.
Make sure no one else runs the IP of the interface of pfsense.
-
@cool_corona I am writing from a mobile device but I will upload logs when I am home.. I didn't see anything helpful there.
I made some changes to the Spanning Tree and LACP settings in my Power connect 2824 which is LACP to a 5854p this morning to see if it improves.
-
@cool_corona Here are some logs.
You will see where I hot-plug the OPT1 interface to get it back online.
Apr 17 19:25:12 php-fpm 329 /index.php: Successful login for user 'admin' from: 192.168.1.210 (Local Database)
Apr 17 19:25:07 sshguard 91220 Attack from "192.168.1.210" on service unknown service with danger 10.
Apr 17 19:25:07 php-fpm 329 /index.php: webConfigurator authentication error for user 'admin' from: 192.168.1.210
Apr 17 19:23:14 kernel arp: 192.168.1.10 moved from 00:26:55:df:4b:d1 to 84:2b:2b:ae:74:30 on em0
Apr 17 19:23:09 kernel arp: 192.168.1.10 moved from 84:2b:2b:ae:74:30 to 00:26:55:df:4b:d1 on em0
Apr 17 18:57:57 kernel arp: 192.168.1.232 moved from 78:2b:cb:57:b3:95 to 78:2b:cb:57:b3:94 on em0
Apr 17 18:57:52 kernel arp: 192.168.1.232 moved from 78:2b:cb:57:b3:94 to 78:2b:cb:57:b3:95 on em0
Apr 17 18:23:44 kernel arp: 192.168.1.232 moved from 78:2b:cb:57:b3:95 to 78:2b:cb:57:b3:94 on em0
Apr 17 17:49:36 kernel arp: 192.168.1.232 moved from 78:2b:cb:57:b3:95 to 78:2b:cb:57:b3:94 on em0
Apr 17 17:15:27 kernel arp: 192.168.1.232 moved from 78:2b:cb:57:b3:95 to 78:2b:cb:57:b3:94 on em0
Apr 17 16:41:19 kernel arp: 192.168.1.232 moved from 78:2b:cb:57:b3:95 to 78:2b:cb:57:b3:94 on em0
Apr 17 16:07:11 kernel arp: 192.168.1.232 moved from 78:2b:cb:57:b3:95 to 78:2b:cb:57:b3:94 on em0
Apr 17 15:33:03 kernel arp: 192.168.1.232 moved from 78:2b:cb:57:b3:95 to 78:2b:cb:57:b3:94 on em0
Apr 17 14:58:54 kernel arp: 192.168.1.232 moved from 78:2b:cb:57:b3:95 to 78:2b:cb:57:b3:94 on em0
Apr 17 14:24:46 kernel arp: 192.168.1.232 moved from 78:2b:cb:57:b3:95 to 78:2b:cb:57:b3:94 on em0
Apr 17 13:50:38 kernel arp: 192.168.1.232 moved from 78:2b:cb:57:b3:95 to 78:2b:cb:57:b3:94 on em0
Apr 17 13:16:30 kernel arp: 192.168.1.232 moved from 78:2b:cb:57:b3:95 to 78:2b:cb:57:b3:94 on em0
Apr 17 12:42:21 kernel arp: 192.168.1.232 moved from 78:2b:cb:57:b3:95 to 78:2b:cb:57:b3:94 on em0
Apr 17 12:08:13 kernel arp: 192.168.1.232 moved from 78:2b:cb:57:b3:95 to 78:2b:cb:57:b3:94 on em0
Apr 17 11:34:10 kernel arp: 192.168.1.232 moved from 78:2b:cb:57:b3:95 to 78:2b:cb:57:b3:94 on em0
Apr 17 11:34:05 kernel arp: 192.168.1.232 moved from 78:2b:cb:57:b3:94 to 78:2b:cb:57:b3:95 on em0
Apr 17 10:59:57 kernel arp: 192.168.1.232 moved from 78:2b:cb:57:b3:95 to 78:2b:cb:57:b3:94 on em0
Apr 17 10:25:48 kernel arp: 192.168.1.232 moved from 78:2b:cb:57:b3:95 to 78:2b:cb:57:b3:94 on em0
Apr 17 09:52:06 php-fpm 67374 /index.php: Successful login for user 'admin' from: 10.0.3.2 (Local Database)
Apr 17 09:51:08 kernel arp: 192.168.1.232 moved from 78:2b:cb:57:b3:95 to 78:2b:cb:57:b3:94 on em0
Apr 17 09:16:27 kernel arp: 192.168.1.232 moved from 78:2b:cb:57:b3:95 to 78:2b:cb:57:b3:94 on em0
Apr 17 08:42:19 kernel arp: 192.168.1.232 moved from 78:2b:cb:57:b3:95 to 78:2b:cb:57:b3:94 on em0
Apr 17 08:08:16 kernel arp: 192.168.1.232 moved from 78:2b:cb:57:b3:95 to 78:2b:cb:57:b3:94 on em0
Apr 17 08:08:11 kernel arp: 192.168.1.232 moved from 78:2b:cb:57:b3:95 to 78:2b:cb:57:b3:94 on em0
Apr 17 07:43:44 kernel arp: 192.168.1.247 moved from 00:26:55:df:4b:d0 to 00:26:55:df:4b:d1 on em0
Apr 17 07:43:39 kernel arp: 192.168.1.247 moved from 00:26:55:df:4b:d1 to 84:2b:2b:ae:74:30 on em0
Apr 17 07:33:58 kernel arp: 192.168.1.232 moved from 78:2b:cb:57:b3:94 to 78:2b:cb:57:b3:95 on em0
Apr 17 06:35:49 kernel arp: 192.168.1.10 moved from 00:26:55:df:4b:d0 to 84:2b:2b:ae:74:30 on em0
Apr 17 06:35:02 kernel arp: 192.168.1.10 moved from 84:2b:2b:ae:74:30 to 00:26:55:df:4b:d1 on em0
Apr 17 06:34:31 check_reload_status 366 Reloading filter
Apr 17 06:34:31 php-fpm 329 /rc.newwanip: rc.newwanip: on (IP address: 192.168.1.1) (interface: OPT1[opt1]) (real interface: em0).
Apr 17 06:34:31 php-fpm 329 /rc.newwanip: rc.newwanip: Info: starting on em0.
Apr 17 06:34:30 check_reload_status 366 Reloading filter
Apr 17 06:34:30 check_reload_status 366 rc.newwanip starting em0
Apr 17 06:34:30 php-fpm 329 /rc.linkup: Hotplug event detected for OPT1(opt1) static IP (192.168.1.1 )
Apr 17 06:34:29 kernel em0: link state changed to UP
Apr 17 06:34:29 check_reload_status 366 Linkup starting em0
Apr 17 06:34:11 check_reload_status 366 Reloading filter
Apr 17 06:34:11 php-fpm 13052 /rc.linkup: Hotplug event detected for OPT1(opt1) static IP (192.168.1.1 )
Apr 17 06:34:10 kernel em0: link state changed to DOWN
Apr 17 06:34:10 check_reload_status 366 Linkup starting em0
Apr 17 06:31:23 kernel arp: 192.168.1.10 moved from 00:26:55:df:4b:d0 to 84:2b:2b:ae:74:30 on em0
Apr 17 06:30:56 kernel arp: 192.168.1.247 moved from 00:26:55:df:4b:d1 to 84:2b:2b:ae:74:30 on em0
Apr 17 06:30:23 check_reload_status 366 Reloading filter
Apr 17 06:30:23 php-fpm 67374 /rc.newwanip: rc.newwanip: on (IP address: 192.168.1.1) (interface: OPT1[opt1]) (real interface: em0).These are from yesterday, not relating to todays issue.
Apr 16 18:36:03 dpinger 11216 WAN_DHCP 8.8.8.8: Clear latency 20273us stddev 14251us loss 5%
Apr 16 18:35:05 dpinger 11216 WAN_DHCP 8.8.8.8: sendto error: 65
Apr 16 18:35:04 dpinger 11216 WAN_DHCP 8.8.8.8: sendto error: 65
Apr 16 18:35:04 dpinger 11216 WAN_DHCP 8.8.8.8: sendto error: 65
Apr 16 18:35:03 dpinger 11216 WAN_DHCP 8.8.8.8: sendto error: 65
Apr 16 18:35:03 dpinger 11216 WAN_DHCP 8.8.8.8: sendto error: 65
Apr 16 18:35:02 dpinger 11216 WAN_DHCP 8.8.8.8: sendto error: 65
Apr 16 18:35:02 dpinger 11216 WAN_DHCP 8.8.8.8: sendto error: 65
Apr 16 18:35:01 dpinger 11216 WAN_DHCP 8.8.8.8: sendto error: 65
Apr 16 18:35:01 dpinger 11216 WAN_DHCP 8.8.8.8: sendto error: 65
Apr 16 18:35:00 dpinger 11216 WAN_DHCP 8.8.8.8: sendto error: 65
Apr 16 18:35:00 dpinger 11216 WAN_DHCP 8.8.8.8: sendto error: 65
Apr 16 18:34:59 dpinger 11216 WAN_DHCP 8.8.8.8: sendto error: 65
Apr 16 18:34:59 dpinger 11216 WAN_DHCP 8.8.8.8: sendto error: 65
Apr 16 18:34:58 dpinger 11216 WAN_DHCP 8.8.8.8: sendto error: 65
Apr 16 18:34:58 dpinger 11216 WAN_DHCP 8.8.8.8: sendto error: 65
Apr 16 18:34:57 dpinger 11216 WAN_DHCP 8.8.8.8: sendto error: 65
Apr 16 18:34:57 dpinger 11216 WAN_DHCP 8.8.8.8: sendto error: 65
Apr 16 18:34:56 dpinger 11216 WAN_DHCP 8.8.8.8: sendto error: 65
Apr 16 18:34:56 dpinger 11216 WAN_DHCP 8.8.8.8: sendto error: 65
Apr 16 18:34:55 dpinger 11216 WAN_DHCP 8.8.8.8: sendto error: 65
Apr 16 18:34:55 dpinger 11216 WAN_DHCP 8.8.8.8: sendto error: 65
Apr 16 18:34:54 dpinger 11216 WAN_DHCP 8.8.8.8: sendto error: 65
Apr 16 18:34:54 dpinger 11216 WAN_DHCP 8.8.8.8: sendto error: 65
Apr 16 18:34:53 dpinger 11216 WAN_DHCP 8.8.8.8: sendto error: 65
Apr 16 18:34:53 dpinger 11216 WAN_DHCP 8.8.8.8: sendto error: 65
Apr 16 18:34:52 dpinger 11216 WAN_DHCP 8.8.8.8: sendto error: 65
Apr 16 18:34:52 dpinger 11216 WAN_DHCP 8.8.8.8: sendto error: 65
Apr 16 18:34:51 dpinger 11216 WAN_DHCP 8.8.8.8: sendto error: 65
Apr 16 18:34:51 dpinger 11216 WAN_DHCP 8.8.8.8: sendto error: 65
Apr 16 18:34:50 dpinger 11216 WAN_DHCP 8.8.8.8: sendto error: 65
Apr 16 18:34:50 dpinger 11216 WAN_DHCP 8.8.8.8: sendto error: 65
Apr 16 18:34:49 dpinger 11216 WAN_DHCP 8.8.8.8: sendto error: 65
Apr 16 18:34:48 dpinger 11216 WAN_DHCP 8.8.8.8: sendto error: 65
Apr 16 18:34:48 dpinger 11216 WAN_DHCP 8.8.8.8: sendto error: 65
Apr 16 18:34:47 dpinger 11216 WAN_DHCP 8.8.8.8: sendto error: 65
Apr 16 18:34:47 dpinger 11216 WAN_DHCP 8.8.8.8: sendto error: 65
Apr 16 18:34:46 dpinger 11216 WAN_DHCP 8.8.8.8: sendto error: 65
Apr 16 18:34:46 dpinger 11216 WAN_DHCP 8.8.8.8: sendto error: 65
Apr 16 18:34:45 dpinger 11216 WAN_DHCP 8.8.8.8: sendto error: 65
Apr 16 18:34:45 dpinger 11216 WAN_DHCP 8.8.8.8: sendto error: 65
Apr 16 18:34:44 dpinger 11216 WAN_DHCP 8.8.8.8: sendto error: 65
Apr 16 18:34:44 dpinger 11216 WAN_DHCP 8.8.8.8: sendto error: 65
Apr 16 18:34:43 dpinger 11216 WAN_DHCP 8.8.8.8: sendto error: 65
Apr 16 18:34:43 dpinger 11216 WAN_DHCP 8.8.8.8: sendto error: 65
Apr 16 18:34:42 dpinger 11216 WAN_DHCP 8.8.8.8: sendto error: 65
Apr 16 18:34:42 dpinger 11216 WAN_DHCP 8.8.8.8: sendto error: 65
Apr 16 18:34:41 dpinger 11216 WAN_DHCP 8.8.8.8: sendto error: 65
Apr 16 18:34:41 dpinger 11216 WAN_DHCP 8.8.8.8: sendto error: 65
Apr 16 18:34:40 dpinger 11216 WAN_DHCP 8.8.8.8: sendto error: 65
Apr 17 06:34:31 dnsmasq 77190 using 10 more local addresses
Apr 17 06:34:31 dnsmasq 77190 using nameserver 1.0.0.1#53
Apr 17 06:34:31 dnsmasq 77190 using nameserver 1.1.1.1#53
Apr 17 06:34:31 dnsmasq 77190 using nameserver 192.168.1.81#53
Apr 17 06:34:31 dnsmasq 77190 using only locally-known addresses for domain 24.172.in-addr.arpa
Apr 17 06:34:31 dnsmasq 77190 using only locally-known addresses for domain 25.172.in-addr.arpa
Apr 17 06:34:31 dnsmasq 77190 using only locally-known addresses for domain 26.172.in-addr.arpa
Apr 17 06:34:31 dnsmasq 77190 using only locally-known addresses for domain 27.172.in-addr.arpa
Apr 17 06:34:31 dnsmasq 77190 using only locally-known addresses for domain 28.172.in-addr.arpa
Apr 17 06:34:31 dnsmasq 77190 using only locally-known addresses for domain 29.172.in-addr.arpa
Apr 17 06:34:31 dnsmasq 77190 using only locally-known addresses for domain 30.172.in-addr.arpa
Apr 17 06:34:31 dnsmasq 77190 using only locally-known addresses for domain 31.172.in-addr.arpa
Apr 17 06:34:31 dnsmasq 77190 reading /etc/resolv.conf
Apr 17 06:30:24 dnsmasq 77190 using 10 more local addresses
Apr 17 06:30:24 dnsmasq 77190 using nameserver 1.0.0.1#53
Apr 17 06:30:24 dnsmasq 77190 using nameserver 1.1.1.1#53
Apr 17 06:30:24 dnsmasq 77190 using nameserver 192.168.1.81#53
Apr 17 06:30:24 dnsmasq 77190 using only locally-known addresses for domain 24.172.in-addr.arpa
Apr 17 06:30:24 dnsmasq 77190 using only locally-known addresses for domain 25.172.in-addr.arpa
Apr 17 06:30:24 dnsmasq 77190 using only locally-known addresses for domain 26.172.in-addr.arpa
Apr 17 06:30:24 dnsmasq 77190 using only locally-known addresses for domain 27.172.in-addr.arpa
Apr 17 06:30:24 dnsmasq 77190 using only locally-known addresses for domain 28.172.in-addr.arpa
Apr 17 06:30:24 dnsmasq 77190 using only locally-known addresses for domain 29.172.in-addr.arpa
Apr 17 06:30:24 dnsmasq 77190 using only locally-known addresses for domain 30.172.in-addr.arpa
Apr 17 06:30:24 dnsmasq 77190 using only locally-known addresses for domain 31.172.in-addr.arpa
Apr 17 06:30:24 dnsmasq 77190 reading /etc/resolv.conf
Apr 16 21:24:55 dnsmasq 77190 using 10 more local addresses
Apr 16 21:24:55 dnsmasq 77190 using nameserver 1.0.0.1#53
Apr 16 21:24:55 dnsmasq 77190 using nameserver 1.1.1.1#53
Apr 16 21:24:55 dnsmasq 77190 using nameserver 192.168.1.81#53
Apr 16 21:24:55 dnsmasq 77190 using only locally-known addresses for domain 24.172.in-addr.arpa
Apr 16 21:24:55 dnsmasq 77190 using only locally-known addresses for domain 25.172.in-addr.arpa
Apr 16 21:24:55 dnsmasq 77190 using only locally-known addresses for domain 26.172.in-addr.arpa
Apr 16 21:24:55 dnsmasq 77190 using only locally-known addresses for domain 27.172.in-addr.arpa
Apr 16 21:24:55 dnsmasq 77190 using only locally-known addresses for domain 28.172.in-addr.arpa
Apr 16 21:24:55 dnsmasq 77190 using only locally-known addresses for domain 29.172.in-addr.arpa
Apr 16 21:24:55 dnsmasq 77190 using only locally-known addresses for domain 30.172.in-addr.arpa
Apr 16 21:24:55 dnsmasq 77190 using only locally-known addresses for domain 31.172.in-addr.arpa
Apr 16 21:24:55 dnsmasq 77190 reading /etc/resolv.conf
Apr 16 20:55:24 dnsmasq 77190 using 10 more local addresses
Apr 16 20:55:24 dnsmasq 77190 using nameserver 1.0.0.1#53
Apr 16 20:55:24 dnsmasq 77190 using nameserver 1.1.1.1#53
Apr 16 20:55:24 dnsmasq 77190 using nameserver 192.168.1.81#53
Apr 16 20:55:24 dnsmasq 77190 using only locally-known addresses for domain 24.172.in-addr.arpa
Apr 16 20:55:24 dnsmasq 77190 using only locally-known addresses for domain 25.172.in-addr.arpa
Apr 16 20:55:24 dnsmasq 77190 using only locally-known addresses for domain 26.172.in-addr.arpa
Apr 16 20:55:24 dnsmasq 77190 using only locally-known addresses for domain 27.172.in-addr.arpa
Apr 16 20:55:24 dnsmasq 77190 using only locally-known addresses for domain 28.172.in-addr.arpa
Apr 16 20:55:24 dnsmasq 77190 using only locally-known addresses for domain 29.172.in-addr.arpa
Apr 16 20:55:24 dnsmasq 77190 using only locally-known addresses for domain 30.172.in-addr.arpa -
Those ARP movements are expected?
If you pcap on the WAN when the connection fails do you see it trying to connect? Like ARP requests for the gateway with no replies perhaps?
Steve
-
@stephenw10 I will try it if it happens again. Last night I found that when I upgraded firmware on the 5548p switch the LAG which connects it to my 2428 switch was no longer a LAG. I redesignated those ports and reloaded the router.
Maybe there was a network loop? I will know more of it happens again as I won't need the internet back up immediately and can investigate more.