Chelsio T520-SO-CR stops running after a short time and console logs “firmware reports adapter error: insufficient airflow”
-
After running for a short time, the 10 GbE stops, and, if you warm reboot, you get this error:
“firmware reports adapter error: insufficient airflow”The server is a pfSense C2758 and the additional 40mm case fan is fitted and working correctly and moving plenty of air.
I have upgraded the BIOS of the C2758 to the latest v2.1 (from 1.0b) and fresh installed pfSense 2.4.4-p3, but I get the same error after running for a short while.The second identical pfSense server C2758 (setup is a HA cluster) also with a Chelsio T520-SO-CR and SFP+ Chelsio SM10G-SR is working fine.
Do you think it’s faulty;
• T520-SO-CR adapter card
• SFP+ Chelsio SM10G-SR
• C2758 mainboard fault?Is there anything else I can check?
-
If you add the 'Thermal Sensors' widget to the dashboard it will show you the NIC temperature.
Ignore the fact it will show as a red bar that is expecting a CPU temp and the Chelsio Cards run hot in normal conditions. The sensor is in the NIC processor and it's not unusual to see >90C. But if it's shutting down it may be even higher.
It should have an additional fan in the C2758 specifically to keep the NIC cool, it may have failed.
Steve
-
Thanks for your reply.
I will do that, the temperature monitor, but, it does have the additional 40mm fan, and, it's working well.
-
Hmm, at what point do you actually see that error? Is it possible to grab a picture of it?
Steve
-
This picture is following a warm reboot once the 10GbE adapter stops working.
-
@nzkiwi68 said in Chelsio T520-SO-CR stops running after a short time and console logs “firmware reports adapter error: insufficient airflow”:
firmware reports adapter error: insufficient airflow
Hmm well it certainly seems to be from the cxgbe driver so check the reported temps first.
https://github.com/pfsense/FreeBSD-src/blob/dd2966ccebbf44dee773537c980f53b16045c1c3/sys/dev/cxgbe/common/t4_hw.c#L207Steve
-
Yes, it seems to be overheating, despite the correct and working additional high airflow 40mm cooling fan.
So, if it's overheating, what do think, faulty adapter?
-
Just out of curiosity, what is the ambient temperature and environment like?
-
@spoggle Air conditioned server room, a/c set for 20 deg C
-
Hard to say, I don't think we've seen that before. Have to see what the temps it's actually reporting are. Compare that with the NIC in the other node.
Steve
-
if it's the temperature and if it isn't under warranty i will check the heatsink if it's well sitted and/or i will try to change the thermal paste
-
@kiokoman Thanks!
That's a very good tip. I'll be sure to try that too.
-
@stephenw10 Thanks for your help btw.
I have ordered a replacement NIC card (even if it just ends up as a spare) so I'll post once the new card arrives and we figure out the fault.
-
Were you able to get any temperature readings from the card? You can read it directly from the command line via the sysctls if that's easier for you.
Steve
-
@stephenw10 That C2758 is currently shut down and the customer is running on the backup HA C2758.
I've ordered a new T520 and when that arrives next week, one evening I'll have a look the temperature etc outside of production hours.
I will post my findings.
Thanks again for all your help!
-
Replaced the Chelsio T520 card and it's now stable.
Was too busy to and time constrained during the shutdown window to try the old card, measure the temperature etc, but, the old card is certainly faulty. The heatsink seems well attached, etc.
Anyway, the replacement card solved the problem, the C2758 is now stable.
Thanks everyone for your helpful posts.
