-
The system log is in /var/log/system.log
ntopng can use significant CPU. If it's maxing out any cpu core it will cause latency issues.
Steve
-
So a few things were happening, I think.
I don't work Fridays and someone decided to plug a cable into the realtek port. I guess this was causing the crashing?
I ended up reinstalling everything and my usage now is always less than 50% and is normally at 2-5%.
Is it normal for the realtek port to cause hard crashes like this without any logs or reports?
But no crashes since, fingers crossed
-
@cgi2099 said in HP T730 help please:
Is it normal for the realtek port to cause hard crashes like this without any logs or reports?
No. Usually the Realtek hardware/driver stop talking to each other and you see the watchdog timeout errors logged but it doesn't affect any other part of the firewall. Many systems only have Realtek NICs though and in that situation you lose connection. The console should still work though.
Steve
-
Well, I spoke too soon; everything went well pretty much all day, ~8 hours. But then it kind of crashed this time. Not like before though.
This time everything lost internet. I could still access the console and web GUI this time. But nothing had internet. Also no errors in the console or log.
I had to reboot to get internet back. Any ideas?
-
What errors were shown when you tried to access anything?
Did you test from the pfSense command line, trying to ping out for example?
If nothing at all was logged it's hard to say...
-
I was able to ping internally but could not ping google or cloudflare.
Nothing logged though, from what I can tell, to me it was like the wan port stopped working.
I will dig another in winscp and see if I can find any more logs somewhere.
If it was having RAM errors or SSD errors would they get logged somewhere?
-
Bad RAM usually results in random crashes. A bad SSD can result in odd failures where services fail over some hours and logging obviously isn't possible. But you would see errors at the console.
When you tried to ping google.com what error was shown?
-
It just timed out when I pinged 8.8.8.8.
It's there a mem test built in?
Logging i believe is working, well it is for ntopng, netdata and adguard.
The only errors I am getting on the console are still:
ixl1: Malicious Driver Detection event 1 on RX queue 771, pf number 0 (PF-1) ixl1: Malicious Driver Detection event 1 on RX queue 769, pf number 0 (PF-1) ixl1: Malicious Driver Detection event 1 on RX queue 769, pf number 0 (PF-1)Same for ixl0
-
Ah, yes. Hitting that issue will prevent traffic but should not crash the OS which sounds like what you are now seeing?
Previously it looked like the entire firewall was crashing which would not have been that. -
For the above error I read somewhere to try the newer drivers and or increase buffer size. But I am unsure how to do either.
-
You are running 2.6?
You could go to 22.05 or try a 2.7 snapshot. The drivers are slightly newer there though there are no changes I'm aware of that would affect this.
I'm not aware of any workaround for that currently. Which buffer was referenced?
Steve
-
Yes on the 2.6.
I think I might go to 22.05, are there any things I need to be aware of switching from CE to Plus?
I guess the buffer doesn't apply to me since I don't have a bridge:
"Most of the issue reports have been from those running a bridge interface with ixl0 and ixl1. However, there have been multiple reports without using bridges as well.
Increasing the buffer size on the bridge reduced the frequency of the events (went from once a day to taking 5 days before it reoccurred)."I have ordered an Intel i350 to replace the x710, when the card comes in I'll swap out the RAM as well.
-
Nope the upgrade to 22.05 should be relatively painless.
Yes, with no bridges defined the bridge buffer does nothing. I'm not sure it actually affects this even when there are bridges in play.
Steve
-
Haven't upgraded yet. But maybe have some more useful info.
Today it has been "crashing" like before. But this time the console still works. When it "crashes" now I lose access to the web gui and all internet traffic goes down.
Only errors are the ones from before, nothing else on the console.
-
All your interfaces are ixl now though? If so that's expected. You might add the Realtek NIC as a management interface as that will remain up if/when you hit the ixl bug.
-
So going through my graphs, everything "crashes" when the ixl0 and ixl1 starts having huge packet loss.
Does this confirm it is an issue with the x710-t2l? Could this nic be getting hot and causing this? Or do you think it is more related to the bug?
This would suck because that thing was hella expensive.
-
How hot? Can you add a fan there as a test?
I'm not aware of heat being an issue with that bug but if it is that would be a very interesting discovery.
Steve
-
I haven't put my heat gun to it yet so not sure if it is getting hot. I was just wondering if that was a known problem. I think I'll redo the thermal compound as well
I will take some readings here in a bit and install a noctua 40mm on it just in case.
Are the x550-t2 decent cards? Any known issues?
-
@cgi2099 said in HP T730 help please:
Are the x550-t2 decent cards?
Yes. The x500 series are what I would recommend currently. In terms of price, performance and stability they are in the sweet spot IMO. That may change.
Steve
-
S stephenw10 moved this topic from General pfSense Questions on
-
Awesome, maybe I can find one of those reasonable somewhere.
I did see intel released new firmware for the x710 cards on the 8th but idk if that will fix any issues.
Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.