PfSense randomly loses connection, and reboot is only solution.

stephenw10

Ok so the gateway device is some box at your ISP?
When the gateway is marked down there will be something in the system logs. Often it will report the reason for marking it down as either excessive packet loss or delay. If it does not it usually means the connection has gone down. Is the WAN interface still showing as UP?
It may be the remote box doesn't like being pinged continuously. You could try altering the ping interval (1s be default) or disabling apinger entirely in System: Routing: Gateways: (edit gateway - advanced section for apinger tuning).

Steve

dreamslacker

@rocketdog:

Not sure about the BIOS, but could take a look..Been thinking about things getting overheated, but then why would just the GW drop?

If you do a Dmesg after the GW drops, does it show the link flapping on the ue0 nic?

If so, you either have a failing NIC or just general instability with the USB NIC (these aren't exactly what I would consider to be stable).

If the System Logs do not show Apinger alarm, restarting the NIC and followed by a filter reload, then the problem probably lies with the NIC (I've had this with a failing NIC before).

rocketdog

@dreamslacker:

If you do a Dmesg after the GW drops, does it show the link flapping on the ue0 nic?

If so, you either have a failing NIC or just general instability with the USB NIC (these aren't exactly what I would consider to be stable).

If the System Logs do not show Apinger alarm, restarting the NIC and followed by a filter reload, then the problem probably lies with the NIC (I've had this with a failing NIC before).

Yeah, tons of ups and downs. The FW has worked as a charm for the last few days…I tuned some stuff on a server (did some ifconfig eth0 RX downtune IIRC), and the TOR-server went down about 20%...Before, and when the FW-problems occured, the tor-relay was full throttle.. So I guess it's just this damn USB NIC. Too much traffic and it goes bananas..

How do you reload filters?

rocketdog

Looking at dmesg, I've lost connections several times…probably while I've not been using internet etc. And I have no idea for how long the ue0 is down, and why it comes back up again. Ideas?

ukphy0: <generic ieee="" 802.3u="" media="" interface=""> PHY 16 on miibus1
ukphy0:  none, 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto, auto-flow
ue0: <usb ethernet=""> on axe0
ZFS WARNING: Recommended minimum kmem_size is 512MB; expect unstable behavior.
             Consider tuning vm.kmem_size and vm.kmem_size_max
             in /boot/loader.conf.
ZFS filesystem version 5
ZFS storage pool version 28
ue0: link state changed to DOWN
bge0: link state changed to DOWN
pflog0: promiscuous mode enabled
ue0: link state changed to UP
bge0: link state changed to UP
ue0: link state changed to DOWN
ue0: link state changed to UP
ue0: link state changed to DOWN
ue0: link state changed to UP
ue0: promiscuous mode enabled
ue0: link state changed to DOWN
ue0: link state changed to UP
ue0: link state changed to DOWN
ue0: link state changed to UP
ue0: link state changed to DOWN
ue0: link state changed to UP
ue0: link state changed to DOWN
ue0: link state changed to UP</usb></generic>

Found this! What could cause this? 1-5 minutes downtime, but no "uplink" messages?

Edit: Missed to get these lines in the screenshot.

Feb 10 19:35:41 	apinger: Starting Alarm Pinger, apinger(17013)
Feb 10 19:35:51 	apinger: ALARM: WANGW(188.133.122.1) *** down ***
Feb 10 21:20:37 	apinger: ALARM: GW_WAN(188.122.133.1) *** down ***
Feb 11 00:39:54 	apinger: Starting Alarm Pinger, apinger(13674)
Feb 11 00:40:05 	apinger: ALARM: WANGW(188.133.122.1) *** down ***
Feb 11 03:03:19 	apinger: ALARM: GW_WAN(188.122.133.1) *** down ***
Feb 11 03:31:54 	apinger: Starting Alarm Pinger, apinger(15720)
Feb 11 03:32:04 	apinger: ALARM: WANGW(188.133.122.1) *** down ***
Feb 13 00:17:28 	apinger: ALARM: GW_WAN(188.122.133.1) *** loss ***
Feb 13 00:47:10 	apinger: alarm canceled: GW_WAN(188.122.133.1) *** loss ***

"Starting Alarm Pinger", then 10 seconds later I'm offline. Where can I find options about this thingy?

stephenw10

How far apart are these up down events?
Why is it using promiscuous mode? Is it bridged?

Steve

rocketdog

@stephenw10:

How far apart are these up down events?
Why is it using promiscuous mode? Is it bridged?

Steve

Promiscuous mode? I have no idea. I've never seen options like that. It is no bridged.

stephenw10

Usually a NIC would only need to use promiscuous mode if it has to be able to process frames addressed to other MACs. This is the case if it is part of a bridge or has been used for packet capturing among others.

Steve

rocketdog

@stephenw10:

Usually a NIC would only need to use promiscuous mode if it has to be able to process frames addressed to other MACs. This is the case if it is part of a bridge or has been used for packet capturing among others.

Steve

According to "Diagnostics > Packet Capture" promiscuous mode is disabled. By the way did, did you see my edit on my previous post?

stephenw10

You can tune apinger or disable it completely in System: Routing: Gateways: edit gateway, advanced.
That's not going to help if your usb nic really is flaky though.

Steve

dreamslacker

@rocketdog:

So I guess it's just this damn USB NIC. Too much traffic and it goes bananas..

How do you reload filters?

Seems like the case. USB NICs aren't exactly stellar performers. I seriously recommend getting a cheap VLAN capable switch (Netgear GS108T or HP Procurve 1810-8G or Mikrotik RB260GS) and using that with your BGE nic to create the WAN & LAN interfaces via VLANs instead. It's far more stable than trying to work with the USB NIC.

rocketdog

As soon as I turn on squid, the USB drops within hours. Since I disabled it, it hasnt dropped once. :) Too bad tho, since I really liked Squid. I guess I'll start using it the day I find relaying onion traffic boring. 8)

rocketdog

Sorry for keeping this thread alive, but..since I figured out, the reason the USB-NIC overloads is because of Squid, is there any certain tweaks I could do to Squid towards the hardware?

bryan.paradis

@rocketdog:

Sorry for keeping this thread alive, but..since I figured out, the reason the USB-NIC overloads is because of Squid, is there any certain tweaks I could do to Squid towards the hardware?

What is squid doing when the USB nic drops? Line up the logs. Look to make the logs more verbose.

Have you tried a different set of USB headers? Does it correspond with high load on the PC?

rocketdog

@bryan.paradis:

@rocketdog:

Sorry for keeping this thread alive, but..since I figured out, the reason the USB-NIC overloads is because of Squid, is there any certain tweaks I could do to Squid towards the hardware?

What is squid doing when the USB nic drops? Line up the logs. Look to make the logs more verbose.

Have you tried a different set of USB headers? Does it correspond with high load on the PC?

After all, it's just not Squid, it is traffic itself. USB-NIC/GW still drops, just not as frequent as with Squid enabled.

Attached some RRD graphs. If you want RRD graphs of CPU or other system-related stuff, just give me a shout.

week.png_thumb

day.png_thumb

8h.png_thumb

pubmsu

Hi there,

We used to use USB NICs and can confirm this flapping behavior - UP/DOWN within seconds. The only solution is NOT to use USB NICs with pfSense/FreeBSD. Go for VLAN switches and non-USB NICs.

Thanks,
msu