Firebox Marvel ports locking up (CORE-E SERIES)
-
I have a firebox, it has become very unstable to say the least. It is running 2.2.4 Nano 4G.
What is happening, every now and then network traffic just stops. I have 2 WAN ports configured and in use, I also have 2 LAN ports configured and in use. I can't tell with the WAN ports, but both LAN ports stop traffic. I have no idea what is affecting this or why it might be doing it, I can't test anything because I can't access it via web gui or via telnet on either of my LAN IPs.
I have tried serial and can access it that way, but with limited understanding of Linux I can't figure out a damn thing. lol.
So, if there is any logs I can push to help, please let me know. The only addons I run are LCDproc and Snort.
Any ideas on how to proceed with this?
-
It turns out, Snort was doing it. I uninstalled it last night and it has been fine since. Wonder what is up with that!
-
There are actually logs and alerts visible in Snort package. Perhaps use them and disable rules that are blocking yourself? Or, disable the blocking feature altogether until you tune the thing? Snort is not an install-and-forget package.
-
Snort seems to contribute to the problem, but i since removed snort and it has ran good since my last comment. Today it did it again, both my lan ports will not allow access to the gui, even though the system says they are up and i see my wan lights flashing so i know they are active, did not think to look at the lan leds.
It has me stumped, i did read somewhere these fireboxes have issues with the lan ports but this has not really done this in the past.
Some history as to what went on recently. One of my wan ports quit working, i found it had a weird name or type on the network connection and i could not change it. I ultimately ended up locking myself out of the box, so i had to go in via serial and reset the network ports.
Once i managed to get in, i reloaded the config backup to restore my settings. This worked but for two days it was saying it was installing and configuring snort!. I ended that process and rebooted and all was gunky dory until i noticed snort was not complete. I removed snort and reinstalled snort, after this all was back to normal.
Will i thought it was, then i started getting the constant port locking up on me, removing snort seemed to fix that issue, or so i thought.
Apart from a reinstall from scratch, i am stuck. I don't want to reload as is a serious pain in the ass with these boxes.
-
I am having this same issue where the ports on my Firebox X550e keep locking up. When they lock up I am getting the message kernel: arpresolve: can't allocate llinfo in the logs. This is not just on one interface. Last night both my lan and my wan interface locked up. If I unplug the ethernet cable and plug it back in things will return to normal.
-
I am having this same issue where the ports on my Firebox X550e keep locking up. When they lock up I am getting the message kernel: arpresolve: can't allocate llinfo in the logs. This is not just on one interface. Last night both my lan and my wan interface locked up. If I unplug the ethernet cable and plug it back in things will return to normal.
I am glad it's not just me, not glad it's happening to you… you know what I mean.
I have never tried unplugging the cables, I guess unplugging them changes the state that PFSense sees them and must reset upon connection. I shall try this next time it happens, I could not tell you if my WANs lock up, I normally just shut the unit down and boot it back up again.
Has to be buck with the driver for the network ports, I can't think of anything else it could be, right now I am off to research this as I know I have read about it somewhere.
-
And here it is…
Known Issues
The Realtek NICs in this box are known to suffer a lock-up condition under certain circumstances. Despite repeated efforts it has not been possible to either cure the problem or ascertain exactly what triggers it. When the problem is triggered the system log will show watchdog timeout and refer to the interface causing it. Fortunately this doesn't affect all users and even then only under some circumstances.
It would seem to be related to packet fragmentation and hardware off loading. Some users have reportedly solved the problem by disabling all hardware offloading and/or using a better switch that can reassemble packets correctly.As found here….
https://doc.pfsense.org/index.php/PfSense_on_Watchguard_FireboxForgive me for saying this, but this is kind of a dumb statement to make
or using a better switch that can reassemble packets correctly.
seeing as the ports are built into the Firebox, how does one use a better switch? Defeats the purpose of the Firebox does it not?
-
That is for the firebox core series not the core e series which have Marvel based nics
-
That is for the firebox core series not the core e series which have Marvel based nics
You are correct, my mistake but very similar to our problem.
-
How often are you seeing this? Do you mainly see it with heavy traffic? Just trying to compare and to see if there is something you're running that might be contributing to the issue. Are you running any add-ons?
-
I was only running LADVD, LCDPRocDEV and was still having the problem. I could only get around a hour or so out of the box before it would lock up. I thought it was Snot that was contributing to the issue but I had it disabled and still had the problem. The only thing I can think of is the config I restored came from a completely different box so I have now wiped my Card and did a re-install with an upgrade to the latest bios 8.1 we will see if that helps. I am currently running a base config that I did by hand with only LCDProc Dev installed. I will see if it lasts the night and let you know. I also had IPV6 with Prefix Delegation running but I have that turned of on my new install now. I will post back in the morning on how things are going.
Update everything was still running this morning Thursday after I woke up so things are going ok so far. I am going to let it run the rest of the day while I am at work. If it is still stable then I will start adding things back on one at a time starting with my IPV6 and so on.
-
I am still getting this lockup issue, same as yours, if I remove the lan cable and plug it back in, the port must reset and it works again. So it seeing the state as up to down and back to up, must reset something in a sense.
This is getting kinda old, I am considering pitching this Firebox as the hardware is not very reliable in this new build. When I say hardware, it has to be a driver issue for the Marvel Network ports.
-
I am still getting this lockup issue, same as yours, if I remove the lan cable and plug it back in, the port must reset and it works again. So it seeing the state as up to down and back to up, must reset something in a sense.
This is getting kinda old, I am considering pitching this Firebox as the hardware is not very reliable in this new build. When I say hardware, it has to be a driver issue for the Marvel Network ports.
Yep. Glad I'm not the only one seeing this issue. My x750e does the same thing with a fresh 2.2.4. install. The disconnects in 2.2.4 with Firebox are unacceptable. v2.1.5 is solid and that's what I reverted back to from my backup. I'm ordering pieces and parts to build a new faster system for PFsense and then am going to test it out throughly before actually making the switch to new hardware in the network.
-
Wish I could find a way to roll back, I upgraded from the GUI, so it has been overwritten. I find, the more time I spend in the GUI, the more often it will lock the port up. I also find, using IE in the GUI is less harsh than running Chrome to access it.
-
Last night it was lock up after lock up, something I have done to the box is to drop the LAN port speeds to 100base Full Duplex. I do not use the ports for subnet routing, both ports are on the same subnet, one is for access in case of a lock out (got to love headless boxes).
Since I have done this, the throughput seems better, the box seems to be more responsive and it has not locked out…. YET.
I shall update as to how this box is now working out.
-
one is for access in case of a lock out (got to love headless boxes).
At the IPMI port if one is there it would be ok, but if not using the IPMI port for this action
you could be creating a network loop! And then this could be the problem for locking out!!! -
Slowing my port speed down seems to have worked.
-
I'm not using a Firebox but I'm having similar lockups….can't access anything on LAN (and Internet stops) but console still working fine (monitored through IPMI port). It was suggested in other threads around the web to not set "Autodetect" for the port speed. Have you by chance set it to 1000T full duplex instead of Autodetect to see if that helps? (if you have done so, my apologies...just curious).
I'm going to follow this thread as it seems very much like what's happening to my SuperMicro N3700 setup. If my system locks up again, I'm going to do the same (unplug the LAN and replug to see if it comes back to life. I really hate to lower my port speed down - even though it wouldn't effect my network as my ISP is well below 100Meg).
Good luck.
-
Yeah, I set my ports at 100 base and full duplex. I think the speed of the ports was causing the issue. It has been up solid for over 24 hours now.
-
Yeah, I set my ports at 100 base and full duplex. I think the speed of the ports was causing the issue. It has been up solid for over 24 hours now.
Did you have it set to 1000T full duplex before that or "Auto Detect"?