Firebox Marvel ports locking up (CORE-E SERIES)
-
Yeah, I set my ports at 100 base and full duplex. I think the speed of the ports was causing the issue. It has been up solid for over 24 hours now.
Did you have it set to 1000T full duplex before that or "Auto Detect"?
-
Sorry, it was on auto detect from install. Pfsense even says it should be set to auto detect.
-
Sorry, it was on auto detect from install. Pfsense even says it should be set to auto detect.
I had read from a few places that setting to auto detect would cause issues similar to yours (and mine) and that setting it to a fixed speed would fix it. I set mine to 1000T Full Duplex yesterday morning in hopes it would correct the issue. Now just wait and see.
-
Yeah, mine seems stable, I don't really need 1000Base on mine, it handles internet traffic only. Seeing as the ports on the modem are 100Mbps max, I see no reason to crank it higher at this time.
-
I have mine set at auto negotiate and it seems to be stable though I still get the random lockup but is now going days between lockups. The other day I tried to install Snort back on my box and it locked up within an hour of the install. I now have snot off for the last 4-5 days and it only locked up on Saturday once and that was with snort removed from the box. One thing I did do was to increase the size of the /tmp file system which resides in ram. I have pleanty of ram to spare as I am running 2gb of ram in my box. I bumped it up to 128MB just as a test and knock on wood has been running since Saturday evening eastern time.
-
My box has been up and running without issue now for 2 days and 21 hours, Snort has been back on it over 2 days now. Still no issues… Fingers crossed.
-
Mine locked up around 30 minutes ago. I went to the console and it was working. Before I could go downstairs to unplug and replug the LAN, it came up with this message (added attachment). After that, Internet would partially work. Had to go in and disable IPV6 to get it all working. Note, the first time it locked up (last week), IPV6 was disabled so I'm not sure it had anything to do with IPV6. The watchdog seems to be the key here. Any thoughts?
-
I am going to quote this again, i know we are not talking about realtec cards, but the symptoms are almost identical.
Known Issues
The Realtek NICs in this box are known to suffer a lock-up condition under certain circumstances. Despite repeated efforts it has not been possible to either cure the problem or ascertain exactly what triggers it. When the problem is triggered the system log will show watchdog timeout and refer to the interface causing it. Fortunately this doesn't affect all users and even then only under some circumstances.
It would seem to be related to packet fragmentation and hardware off loading. Some users have reportedly solved the problem by disabling all hardware offloading and/or using a better switch that can reassemble packets correctly. -
My box is still doing it, had to reset twice yesterday. Might slow the ports down to 10Mbps and see what happens.
-
My box is still doing it, had to reset twice yesterday. Might slow the ports down to 10Mbps and see what happens.
Sorry if you've tried this but if not, could you try the MSI/MSIX tweak(s) posted here and see if it helps:
https://doc.pfsense.org/index.php/Tuning_and_Troubleshooting_Network_Cards
Seems to help some…others, not so much.
Also seems that this is something released in the 2.2.x series as people go back to 2.1.x and problem goes away for the most part.
-
Thanks, I will try that. The weird thing is, the network was fine, as soon as I tried to login to the server, the port crashed/locked up/died. It does this not all the time, but seems to be triggered by it. Very odd if you ask me…
-
Reading some more, I found this thread (https://forum.pfsense.org/index.php?topic=96325.0) which is Intel related but I'm not so sure it doesn't go beyond that. Seem FreeBSD 10.1 has an issue that needs to be corrected at the OS level that 'could' be causing this.
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=199174
States "In Progress" under status.
Not sure if this effects Realtek cards or not but reading though the freebsd.org link, the symptoms sound exactly like mine and yours too.
-
Seems this issue (at least with Intel NIC's) has been around for some time: https://forums.freebsd.org/threads/workaround-freebsd-10-1-sudden-network-down.49264/
No clue if FreeBSD 10.2 fixes it or not. However, pfsense 2.1.5 has an older FreeBSD and I'll try that if it's not fixed soon.
-
What ever it is, as said before, was exacerbated with the 2.2.x upgrade in my opinion. I don't recall seeing it in past releases, maybe some have seen it, but personally I have not been using the great software near as long as some have.
I hope there is a fix for it soon, I hate to move to something else, but it is becoming too unreliable for myself to continue use.
-
What ever it is, as said before, was exacerbated with the 2.2.x upgrade in my opinion. I don't recall seeing it in past releases, maybe some have seen it, but personally I have not been using the great software near as long as some have.
I hope there is a fix for it soon, I hate to move to something else, but it is becoming too unreliable for myself to continue use.
2.2.x was the start of FreeBSD 10.1 with pfsense…as far as I can tell.
-
Interesting… I am hoping that one of the devs may chime in, maybe have some insight into this.
-
Had first LAN lockup with watchdog timeout in three days. Just turned off all hardware and checksum offloading, saved and rebooted. Now to start over monitoring.
If this doesn't work, I will probably go to 2.1.5 until (if that works) until it's resolved.
Edit: After reading around more, I'm more inclined to believe that the error is in FreeBSD and that a commit to fix the same error for the older Intel cards (em) has already been committed. https://reviews.freebsd.org/D3192
As to when it appears, not sure. Seems to have been sent to the Intel Networking for review too so time will tell.
As for the OP's issue…if this is a FreeBSD issue (seems to appear on lots of different brands of hardware), hopefully, the FreeBSD patches will fix it. Don't know enough about that to guess at this point though.
-
2.1.5 works.
-
Not sure if it matters or not but I just noticed that even though I checked to disable TSO under Advanced, Networking, the system tunable net.inet.tcp.tso was at 1. Changed it to 0, saved and then added net.inet.tcp.tso=0 to /boot/loader.conf.local just to make sure.
I guess I'll reboot now instead of waiting a few days because I had the disable TSO option checked before and it made no difference.
-
Keep us informed on how it works out, I have been up for a couple of days without issue so far. I did make the changes you mentioned earlier in the topic, it could be more stable from the changes, but this thing is temperamental and will flake out when it feels like it.