Modify Intel X540-T2 NIC To Work At 2.5GbE ?

panzerscope

@stephenw10 said in Modify Intel X540-T2 NIC To Work At 2.5GbE ?:

i226

Thanks for the responses guys. I have made the changes as noted in my prior post. I have an Intel i7 8700 (6 Core/12 Threads) which I am going to go ahead and pop in the firewall as this will give me a higher core clock and hyperthreading.

One question however regarding these variables:

hw.ix.num_queues=6 (Should be set to the number of physical CPU cores you have from what I understand, I set mine to 6 as that is what I have)
machdep.hyperthreading_intr_allowed=1 (Enables the use of logical cores, essentially hyperthreading)

The above variables would sort of work in tandem together, so I want to make sure I have them set correctly. Should "hw.ix.num_queues" be set to the value of physical cores ONLY (ignoring logical cores) ? and the "machdep.hyperthreading_intr_allowed" when enabled would make use of any available logical cores.

On another sidenote, I am sourcing a genuine Intel X550-T2. The one I have now is a Lenovo SKU. I am sure the change will make not make any particular difference, but Intel releases its NVM updates more frequently that the sub vendors.

Thanks
P

w0w

@panzerscope
You probably don't need to set those variables. These cards should work at 2 Gbit out of the box. As I already said, try flow control setting.This option might affect something.

panzerscope

@w0w said in Modify Intel X540-T2 NIC To Work At 2.5GbE ?:

@panzerscope
You probably don't need to set those variables. These cards should work at 2 Gbit out of the box. As I already said, try flow control setting.This option might affect something.

Possibly, but I would like to experiment with them all the same :). I did implement the flow control variable, unfortunately there was no improvement, but will leave it enabled anyway as it seems sensible to.

panzerscope

Hey all.

So a little update after installing the Intel X550-T2 and updating to the latest NVM (3.7), replacing the Lenovo SKU of the X550-T2 I originally had installed.

I did some further testing for sanity check and to eliminate other devices being the issue.

I hooked up my Main PC using a 10Gbe NIC (X540-T2) to Pfsense using a brand new CAT6 SFTP cable to remove my switch as the middleman and maybe a bad cable. No discernible difference upon multiple speed testing. So I can remove the switch as the issue.

I wondered if it was the cable going from my ONT to Pfsense causing the issue. Again, used a new CAT6 SFTP cable, again no discernible difference.

The only thing I did witness is when swapping out cables, causing a new connection to the ONT, was that when running a speed test immediately afterwards, my upload was at the full 2GB, but would drop off below that after subsequent tests.

Sitting here for the moment, it is either Pfsense/X550-T2 or the ONT that is being difficult.

Worth noting that I am still forcing 10Gbe link speed with the ONT. Auto negotiation ends up with the WAN going up and down, error in the logs show as "arpresolve: can't allocate llinfo for WAN IP HERE on ix0" This is a Kernel level message.

I am now going to disable some of the pfsense services to see if it is that, though I highly doubt it.

panzerscope

This post is deleted!

w0w

@panzerscope
And what about putting your PC directly, without pfsense and testing speeds again?

panzerscope

@w0w

I did that a while ago, should have reiterated that going from PC direct to ISP Router gives the full expected speeds.

w0w

@panzerscope
Hmm... Ok
Just to make sure it's not some hardware-specific glitch, I'd probably try installing IPFire or OpenWRT for x86 instead of pfSense—if you have enough time for such experiments, of course. The main thing is, don't forget to back up your configuration.

stephenw10

If you set the advertised speeds for auto-neg to only one speed in the sysctls does it still flap?

[25.03-BETA][root@m470-3.stevew.lan]/root: sysctl -d dev.ix.0.advertise_speed
dev.ix.0.advertise_speed: 
Control advertised link speed using these flags:
        0x1 - advertise 100M
        0x2 - advertise 1G
        0x4 - advertise 10G
        0x8 - advertise 10M

        0x10  - advertise 2.5G
        0x20  - advertise 5G

        100M and 10M are only supported on certain adapters.

panzerscope

@w0w said in Modify Intel X540-T2 NIC To Work At 2.5GbE ?:

@panzerscope
Hmm... Ok
Just to make sure it's not some hardware-specific glitch, I'd probably try installing IPFire or OpenWRT for x86 instead of pfSense—if you have enough time for such experiments, of course. The main thing is, don't forget to back up your configuration.

I see the logic of doing that, for sure. However I am not willing right now to go that deep with it.

@stephenw10 said in Modify Intel X540-T2 NIC To Work At 2.5GbE ?:

If you set the advertised speeds for auto-neg to only one speed in the sysctls does it still flap?

[25.03-BETA][root@m470-3.stevew.lan]/root: sysctl -d dev.ix.0.advertise_speed
dev.ix.0.advertise_speed: 
Control advertised link speed using these flags:
        0x1 - advertise 100M
        0x2 - advertise 1G
        0x4 - advertise 10G
        0x8 - advertise 10M

        0x10  - advertise 2.5G
        0x20  - advertise 5G

        100M and 10M are only supported on certain adapters.

If I set a link speed then it will be fine. Auto-Neg is a no go for me otherwise the WAN will go up and down all over the place. So I set it to 10G.

stephenw10

So it still flaps even if you expose only one possible negotiation speed?

panzerscope

@stephenw10 said in Modify Intel X540-T2 NIC To Work At 2.5GbE ?:

So it still flaps even if you expose only one possible negotiation speed?

What would the tuneable be for 10g only? Sorry I looked at what you put but don't fully understand the value/flag I need to input for sysctl -d dev.ix.0.advertise_speed

stephenw10

It would be like:

[25.03-BETA][root@m470-3.stevew.lan]/root: sysctl dev.ix.2.advertise_speed=4
dev.ix.2.advertise_speed: 6 -> 4

But obviously ix0 or whichever NIC you're using there

panzerscope

@stephenw10 said in Modify Intel X540-T2 NIC To Work At 2.5GbE ?:

It would be like:
[25.03-BETA][root@m470-3.stevew.lan]/root: sysctl dev.ix.2.advertise_speed=4
dev.ix.2.advertise_speed: 6 -> 4
But obviously ix0 or whichever NIC you're using there

Thanks. I have popped that in. Does that mean I should set my WAN to Autoselect now or leave it set at 10G? I had it manually set to 10G anyway, so seemingly nothing has changed.

stephenw10

Yes, when you set it to autoselect it will try to negotiate a link with the other side but will advertise only 10G so it can't try to connect at any other speed.

It may still flap since it still runs negotiation. But many newer NICs/drivers actually require that to work correctly.

panzerscope

@stephenw10 said in Modify Intel X540-T2 NIC To Work At 2.5GbE ?:

Yes, when you set it to autoselect it will try to negotiate a link with the other side but will advertise only 10G so it can't try to connect at any other speed.

It may still flap since it still runs negotiation. But many newer NICs/drivers actually require that to work correctly.

Thanks. I gave that a go, had to reboot for it work but autoselect now works :) However does not seem to be as reliable as just setting WAN to 10g manually. If I save a setting that causes WAN to restart, it will flop about again. I was not getting that when manually forcing 10g.

I will go back to manually setting it. Manually setting the link speed persisted through reboots anyway.