Heavy Load makes interface drop

wiz561

Hi,

I've run into a strange problem. I'm a penetration tester and recently upgraded to 2.0 beta. We'll kick off a nmap or nessus scan on one of our boxes and just let it go. Nmap seems to work fine, but it's nessus where it kills the pfsense box. In the system logs, I get the following over and over again….

Feb 24 07:52:33 php: : Hotplug event detected for lan but ignoring since interface is configured with static IP (192.168.1.1)
Feb 24 07:52:33 check_reload_status: Linkup starting bge1
Feb 24 07:52:33 kernel: bge1_vlan120: link state changed to UP
Feb 24 07:52:33 kernel: bge1_vlan130: link state changed to UP
Feb 24 07:52:33 kernel: bge1_vlan140: link state changed to UP
Feb 24 07:52:33 kernel: bge1_vlan101: link state changed to UP
Feb 24 07:52:33 kernel: bge1: link state changed to UP
Feb 24 07:52:30 php: : Hotplug event detected for lan but ignoring since interface is configured with static IP (192.168.1.1)
Feb 24 07:52:30 check_reload_status: Linkup starting bge1
Feb 24 07:52:30 kernel: bge1_vlan120: link state changed to DOWN
Feb 24 07:52:30 kernel: bge1_vlan130: link state changed to DOWN
Feb 24 07:52:30 kernel: bge1_vlan140: link stat changed to DOWN
Feb 24 07:52:30 kernel: bge1_vlan101: link state changed to DOWN
Feb 24 07:52:30 kernel: bge1: link state changed to DOWN
Feb 24 07:52:30 kernel: bge1: watchdog timeout -- resetting

It pretty much stops all internal network traffic since it's dropping the interface where the vlan's are routed out of. After a few seconds, it will bring everything back up and everything is fine. One related entry is the firewall. I keep getting these as well...

@1 scrub in on bge1 all fragment reassemble
@1 block drop in log inet all label "Default deny rule IPv4"

I've created firewall rules to allow the IP to get out to wherever it needs to go and whatever port and protocol. Pretty much "*'s" across except for the IP. I had a suggestion in the firewall forum to add a floating rule that changes the state to 'none'. This pretty much blocked all traffic in both directions on that IP, so I figured that was bad.

I'm not sure if it's a 2.0 beta issue or another issue. If anybody has any suggestions, that would be great.

Thanks!

jimp

Hotplug/link state event means link loss, as if the cable were unplugged.

Check the cables, the switch, etc, etc. That's usually something physical.

wiz561

Thanks for the response. Nothing was disconnected or unplugged. The only thing was the heavy vulnerability scanning. Once I stopped the scans, everything returned to normal.

I only have a small window to do these scans so I have to get them done. I downgraded to 1.2.3 and no more problems with the interface dropping.

I also don't know if it was related, but I was trying different settings out. When the firewall optimization was set to 'conservative', it seemed like it would be a lot worse. The box should be powerful enough; it's a HP server with 4 gigs of ram, dual core, and running the 64 bit kernel.

thanks!

jimp

The latter problem sounds like you were running out of states.

But that wouldn't cause it to think it lost the physical link

wiz561

I was monitoring the 'states table size' to make sure it wasn't going over. I can't really explain in depth what it means, but I have a feeling that it's how many open connections there are, similar to the results of a netstat. Nonetheless, I just thought that if the number starts to reach the ceiling, you should increase it if you have the memory and processor power.

So, I raised it gradually all the way up until 550000 and still the same results. The active usually stayed around 475k to 500k, so I just guessed a max of 550k was good. Plus, I had only about 7% memory usage and 0 to 15% processor.

Meanwhile, it seemed like whatever I changed the max state size to, it would still drop the connection.