PfSense Locking Up With Large Downloads
-
Hello there,
I've been using pFsense for a few months now and it's been fantastic - Still very much a new user but learning as I go. Over the past week or so I've noticed a problem that seems to occur when downloading large files. I don't often do large downloads so I can't say if this is a recent issue - I am using 2.3.3-RELEASE-p1). It appears the pfsense box locks up until the download is finished. Sometimes when this happens the box doesn't spring back to life and I have to shut it down and reboot.
I tested this out today using a torrent application and downloaded a 9Gb torrent. I am on a 200Mbps connection and I get near enough 200 when I've tested with speedtests and such.
The torrent I tested was 9GB and I was getting about 25.5MiB/s after a few moments of starting it. Browser worked fine as I navigated pages and such. Then the torrent download dropped to about 5MiB/s and continued to slowly drop. I could no longer use my browser, any internet applications etc.
I have spent some time reading the forums here for a solution and some mentioned state tables could be an issue but currently mine sit at 778/396000 and don't climb that much above this even when the problem occurs.
Others mentioned memory / cpu usage being a probable cause - I have 4GB RAM and a N3150 @ 1.60GHz machine (custom box I bought from Amazon a while ago just for pfSense). According to the system information which I've watched as the problem occurs, memory is far from being fully consumed and CPU doesn't exceed about 70%. The box does use Realtek NIC.
The following is the log, I've copied from the exact moment it starts until the moment the pfsense box became completely unresponsive. It mentiones hotplug issues - There is no problem with the cables I use, this problem only arises when large files are being downloaded. The log also mentions "detected an IP change or dynamic WAN reconnection" - There is no IP change, and it's showing the same IP there twice.
I do use SNORT and pfBlockerNG but I could not see anything in their logs relating to this problem.
I'd really appreciate if someone could help me out with this problem.
Jul 9 23:24:25 kernel re1: promiscuous mode enabled
Jul 9 23:24:22 php-fpm 26305 /rc.newwanip: rc.newwanip: on (IP address: 86...) (interface: WAN[wan]) (real interface: re1).
Jul 9 23:24:22 php-fpm 26305 /rc.newwanip: rc.newwanip: Info: starting on re1.
Jul 9 23:24:17 php-fpm 56117 [pfBlockerNG] Starting cron process.
Jul 9 23:24:16 SnortStartup 68549 Ignoring additional START command since Snort is already starting…
Jul 9 23:24:16 php-fpm 56117 /rc.start_packages: Restarting/Starting all packages.
Jul 9 23:24:16 check_reload_status Reloading filter
Jul 9 23:24:16 check_reload_status updating dyndns wan
Jul 9 23:24:14 check_reload_status Restarting ipsec tunnels
Jul 9 23:24:14 php-fpm 56117 /rc.linkup: ROUTING: setting default route to 86...
Jul 9 23:24:14 check_reload_status rc.newwanip starting re1
Jul 9 23:24:14 php-fpm 56117 /rc.linkup: HOTPLUG: Configuring interface wan
Jul 9 23:24:14 php-fpm 56117 /rc.linkup: DEVD Ethernet attached event for wan
Jul 9 23:24:14 check_reload_status Starting packages
Jul 9 23:24:14 php-fpm 56117 /rc.newwanip: pfSense package system has detected an IP change or dynamic WAN reconnection - 86... -> 86... - Restarting packages.
Jul 9 23:24:12 check_reload_status Linkup starting re1
Jul 9 23:24:12 kernel re1: link state changed to UP
Jul 9 23:24:12 php-fpm 56117 /rc.newwanip: Creating rrd update script
Jul 9 23:24:12 php-fpm 56117 /rc.newwanip: Resyncing OpenVPN instances for interface WAN.
Jul 9 23:24:11 kernel arpresolve: can't allocate llinfo for 86... on re1
Jul 9 23:24:11 check_reload_status Reloading filter
Jul 9 23:24:11 check_reload_status Restarting OpenVPN tunnels/interfaces
Jul 9 23:24:11 check_reload_status Restarting ipsec tunnels
Jul 9 23:24:11 check_reload_status updating dyndns WAN_DHCP
Jul 9 23:24:10 check_reload_status Reloading filter
Jul 9 23:24:09 php-fpm 34117 /rc.linkup: DEVD Ethernet detached event for wan
Jul 9 23:24:08 check_reload_status Linkup starting re1
Jul 9 23:24:08 kernel re1: link state changed to DOWN
Jul 9 23:24:08 kernel re1: watchdog timeout
Jul 9 23:24:08 php-fpm 56117 /rc.newwanip: ROUTING: setting default route to 86...
Jul 9 23:23:45 php-fpm 34117 [pfBlockerNG] Starting cron process.
Jul 9 23:23:45 kernel re1: promiscuous mode disabled
Jul 9 23:23:45 kernel pid 58728 (snort), uid 0: exited on signal 11
Jul 9 23:23:45 SnortStartup 46574 Snort START for WANSNORT(33214_re1)…
Jul 9 23:23:45 php-fpm 34117 /rc.start_packages: Restarting/Starting all packages.
Jul 9 23:23:39 check_reload_status Starting packages
Jul 9 23:23:39 php-fpm 24556 /rc.newwanip: pfSense package system has detected an IP change or dynamic WAN reconnection - 86... -> 86... - Restarting packages.
Jul 9 23:23:37 php-fpm 24556 /rc.newwanip: Creating rrd update script
Jul 9 23:23:37 php-fpm 24556 /rc.newwanip: Resyncing OpenVPN instances for interface WAN.
Jul 9 23:23:34 php-fpm 34117 /rc.filter_configure_sync: dpinger: No dpinger session running for gateway WAN_DHCP
Jul 9 23:23:34 php-fpm 24556 /rc.newwanip: ROUTING: setting default route to 86...
Jul 9 23:23:33 php-fpm 56117 /rc.newwanip: rc.newwanip: on (IP address: 86...) (interface: WAN[wan]) (real interface: re1).
Jul 9 23:23:33 php-fpm 56117 /rc.newwanip: rc.newwanip: Info: starting on re1.
Jul 9 23:23:32 check_reload_status Reloading filter
Jul 9 23:23:32 check_reload_status updating dyndns wan
Jul 9 23:23:30 check_reload_status Restarting ipsec tunnels
Jul 9 23:23:30 php-fpm 56117 /rc.linkup: ROUTING: setting default route to 86...
Jul 9 23:23:30 check_reload_status rc.newwanip starting re1
Jul 9 23:23:29 php-fpm 56117 /rc.linkup: HOTPLUG: Configuring interface wan
Jul 9 23:23:29 php-fpm 56117 /rc.linkup: DEVD Ethernet attached event for wan
Jul 9 23:23:28 kernel re1: link state changed to UP
Jul 9 23:23:28 check_reload_status Linkup starting re1
Jul 9 23:23:26 check_reload_status Reloading filter
Jul 9 23:23:25 check_reload_status Reloading filter
Jul 9 23:23:25 check_reload_status Restarting OpenVPN tunnels/interfaces
Jul 9 23:23:25 check_reload_status Restarting ipsec tunnels
Jul 9 23:23:25 check_reload_status updating dyndns WAN_DHCP
Jul 9 23:23:25 php-fpm 54612 /rc.linkup: DEVD Ethernet detached event for wan
Jul 9 23:23:24 check_reload_status Linkup starting re1
Jul 9 23:23:24 kernel re1: link state changed to DOWN
Jul 9 23:23:24 kernel re1: watchdog timeout
Jul 9 23:23:21 check_reload_status Reloading filter
Jul 9 23:23:21 check_reload_status updating dyndns wan
Jul 9 23:23:19 php-fpm 24556 /rc.newwanip: rc.newwanip: on (IP address: 86...) (interface: WAN[wan]) (real interface: re1).
Jul 9 23:23:19 php-fpm 24556 /rc.newwanip: rc.newwanip: Info: starting on re1.
Jul 9 23:23:18 check_reload_status Restarting ipsec tunnels
Jul 9 23:23:18 php-fpm 91521 /rc.linkup: ROUTING: setting default route to 86...
Jul 9 23:23:18 check_reload_status rc.newwanip starting re1
Jul 9 23:23:18 php-fpm 91521 /rc.linkup: HOTPLUG: Configuring interface wan
Jul 9 23:23:18 php-fpm 91521 /rc.linkup: DEVD Ethernet attached event for wan
Jul 9 23:23:17 kernel re1: link state changed to UP
Jul 9 23:23:17 check_reload_status Linkup starting re1
Jul 9 23:23:15 check_reload_status Reloading filter
Jul 9 23:23:14 php-fpm 91521 /rc.linkup: DEVD Ethernet detached event for wan
Jul 9 23:23:13 check_reload_status Linkup starting re1
Jul 9 23:23:13 kernel re1: link state changed to DOWN
Jul 9 23:23:13 kernel re1: watchdog timeout
Jul 9 23:23:07 check_reload_status Reloading filter
Jul 9 23:23:07 php-fpm 87963 /rc.newwanip: rc.newwanip: on (IP address: 192... ) (interface: LAN[lan]) (real interface: re0).
Jul 9 23:23:07 php-fpm 87963 /rc.newwanip: rc.newwanip: Info: starting on re0.
Jul 9 23:23:06 check_reload_status Reloading filter
Jul 9 23:23:06 check_reload_status rc.newwanip starting re0
Jul 9 23:23:06 php-fpm 87963 /rc.linkup: Hotplug event detected for LAN(lan) static IP (192... )
Jul 9 23:23:05 kernel re0: link state changed to UP
Jul 9 23:23:05 check_reload_status Linkup starting re0
Jul 9 23:23:02 check_reload_status Reloading filter
Jul 9 23:23:02 php-fpm 48922 /rc.linkup: Hotplug event detected for LAN(lan) static IP (192... )
Jul 9 23:23:01 kernel re0: link state changed to DOWN
Jul 9 23:23:01 kernel re0: watchdog timeout
Jul 9 23:23:01 check_reload_status Linkup starting re0 -
The following is the log, I've copied from the exact moment it starts until the moment the pfsense box became completely unresponsive. It mentiones hotplug issues - There is no problem with the cables I use, this problem only arises when large files are being downloaded. The log also mentions "detected an IP change or dynamic WAN reconnection" - There is no IP change, and it's showing the same IP there twice.
As said : Your pfSense NIC detects cable removing.
This means : some one is ripping out the WAN cable - or the NIC (Realtek => may day …. ) is bad or the NIC on the other side is bad. -
Sounds like a realtec NIC crapping out under load.
-
As said : Your pfSense NIC detects cable removing.
This means : some one is ripping out the WAN cable - or the NIC (Realtek => may day …. ) is bad or the NIC on the other side is bad.Sounds like a realtec NIC crapping out under load.
Hi there,
Thanks very much for the response both!
As I said, the cable is fine - It only happens when large downloads are going on. I've even replaced the cable to rule out a cable issue too.
So it may be the card can't handle the load then. Is there any way I can confirm this? Some test I can perform?
I guess I could just replace the card with an Intel card but before I do that it'd be good to check this is the problem.
-
As said : Your pfSense NIC detects cable removing.
This means : some one is ripping out the WAN cable - or the NIC (Realtek => may day …. ) is bad or the NIC on the other side is bad.Sounds like a realtec NIC crapping out under load.
Hi there,
Thanks very much for the response both!
As I said, the cable is fine - It only happens when large downloads are going on. I've even replaced the cable to rule out a cable issue too.
So it may be the card can't handle the load then. Is there any way I can confirm this? Some test I can perform?
I guess I could just replace the card with an Intel card but before I do that it'd be good to check this is the problem.
Use iperf between your pfSense and another LAN host.
