half network speed after upgrading

skogs

@high_bounce perhaps this is old school and things don't work this way anymore but generally the second you actually SET a speed and duplex on one side...the link goes to hell. Because it is no longer auto-negotiating properly, the other side will drop to ultra slow and half duplex.
If you set anything at all manually...you need to be hyper vigilant at setting everything manually. ... and most consumer switches and modems aren't even capable of being set manually....so guaranteed failure.

stephenw10

Yeah 8 queues each way, no problem.

You could look at the off-load options that are enabled in: ifconfig -vvvm cxl0

High_bounce

stephenw10

Hmm, I'd try disabling TSO4 and LRO as a first step there. Those are both normally disabled by default in Sys > Adv > Networking.

High_bounce

both are already checked in the Adv settings. should i try unchecking then rechecking in the UI?

stephenw10

Yes, try that. Though that may require you to reboot.

Otherwise you should be able to manually disable them using ifconfig like:

ifconfig cxl0 -tso4

Unfortunately the syntax there can be confusing and I don't have anything setup right now to check.

High_bounce

no changes with the radio buttons or ifconfig. ifconfig gets reset after reboot so maybe needs to be set in a config file

stephenw10

Disabling those options using ifconfig doesn't work? Or it doesn't make any difference to throughput?

High_bounce

it changes the options for the interface but no change in throughput. also i changed boot environment back to 22.05 to check for differences i found a few i have no idea if they make a difference:

boot logs
23.01 8 txq/rxq, 22 MSI-X, 46eq, 21iq
22.05 4 txq/rxq, 14 MSI-X, 30eq, 13iq

ifconfig looks all the same but 23.01 has an option called "NOMAP" i am not sure what it means and cant find anything online about it

stephenw10

NOMAP is to allow unmapped mbufs. I don't expect it to make any difference but I can't find any way to disable it! It looks like you should be able to ifconfig cxl0 -nomap but that doesn't work for me.

You can almost certainly tune the queues back to 4x in the sysctls/tunables but it's hard to imagine that would help. 22 MSI-X interrupts.... I couldn't say but again it's hard to see how more would be worse.

If you run top -HaSP during a test do you see the loading spread across the cores correctly?

High_bounce

no change in cpu utilisation im still at a lost as to cause are you able to replicate on your end?

stephenw10

I don't have a Chelsio 10GBase-T NIC to test with unfortunately. But I haven't seen anything like that on any other NICs.

High_bounce

no worries i just switch to an old intel x540-t2 should be fine until i upgrade the rest of my network to multi-gig thank you for all your help. i hope it gets fixed at some point i have always herd that Chelsio nics were the best with bsd maybe that is changing now