Low routing performance new build, N3160, i211-AT

pfBasic

That being said, my box gets gigabit throughput any way i run iperf. From it, to it, across it. I only saw ~5% on top while it was running.

I don't think the problem is your hardware though. It doesn't take much to just route at gigabit speeds and that's a pretty modern processor.

Capture.JPG_thumb

pfBasic

Are you using a sideloaded version of iperf or the iperf package?

iphonelover

I used both, the perf package, which is version 2, and installed iperf3 from the shell.

So I just finished testing this setup-

client1 running iperf3 -s connected to router on the wan port
client2 running iperf3 -c connected to router on the lan port

I didn't have my routes set up correctly, so I couldn't make the full connection, but I was able to iperf from the router lan interface to client1 at full gig speed of about 940 Mbps.

While not 100% conclusive it does lead me to believe that I can route at full gig speed.

Also I found that the Intel speed step was disabled in the BIOS, so I enabled that, and checked the PowerD settings in Pfsense, and cranked everything up to 11 and I can perf at 600 Mbps, where as before it was 350 Mbps.

pfBasic

yeah that hardware should have no issues routing full gig without any tweaking.

tirsojrp

Do not run iperf on the pfsense box.

VioletDragon

This can be fixed by enabling PowerD and set it to Hiadaptive and applying the following tweaks for network stack. once applied reboot and test.

Disable hardware checksum offload

Disable hardware TCP segmentation offload

Disable hardware large receive offload

kern.ipc.nmbclusters="1000000"
hw.igb.rxd="4096"
hw.igb.txd="4096"
net.inet.tcp.syncache.hashsize=1024
net.inet.tcp.syncache.bucketlimit=100
net.isr.defaultqlimit=4096
net.link.ifqmaxlen=10240
hw.igb.rx_process_limit="-1"
hw.igb.num_queues=2
dev.igb.0.fc=0
dev.igb.1.fc=0

A Former User

@VioletDragon said in Low routing performance new build, N3160, i211-AT:

This can be fixed by enabling PowerD and set it to Hiadaptive and applying the following tweaks for network stack. once applied reboot and test.

Disable hardware checksum offload

Disable hardware TCP segmentation offload

Disable hardware large receive offload

kern.ipc.nmbclusters="1000000"
hw.igb.rxd="4096"
hw.igb.txd="4096"
net.inet.tcp.syncache.hashsize=1024
net.inet.tcp.syncache.bucketlimit=100
net.isr.defaultqlimit=4096
net.link.ifqmaxlen=10240
hw.igb.rx_process_limit="-1"
hw.igb.num_queues=2
dev.igb.0.fc=0
dev.igb.1.fc=0

Thanks! This got me from 144Mbits/sec -> 204Mbits/sec -> 940Mbits/sec on my Odyssey X86J4105800.

VioletDragon

That is great. Glad it helped :D

BuiltOnSelfSuccess

@violetdragon

I already had:
PowerD enabled and set to Hiadaptive ,
Disable hardware large receive offload
Disable hardware checksum offload

So I went and made your suggested changes:
Disable hardware TCP segmentation offload
and I added all of the network stack tweaks that you suggested but I haven't seen an improvement in speed, are there ay other tweaks that you can suggest or different values for the tweaks that you'll already suggested that I can try please?

Thanks in advance.

BuiltOnSelfSuccess

I found another post (https://hardforum.com/threads/pfsense-2-5-new-configurations-for-igb-performance.2008210/#post-1044942639) and made some additional changes (only to the tunables as I'm not confident and knowledgeable enough to modify the loader.conf) which seemed to improve throughput, which one of these did the trick I have no idea but on my 200/20 speed I was previously getting around 90/12 I'm now achieving around 150/16 but I know that my device has previously been capable of 180/19 so I'll keep playing until I can improve the speed further but I'd be very grateful for any additional ideas/changes/tweaks etc...