Issue with throughput with X710-DA2 and PFsense 2.6

[stephenw10]

And can I assume a client connected directly to the modem gets a public IP and can see the full bandwidth?

[bawoodruff]

yup

[bawoodruff]

Wonder if I backup my current config, and get re-install would make a difference.... Seen some posts involving throughout issues being resolved by this. although this is a rather new install to begin with tbh.

[stephenw10]

It's an easy enough test but I wouldn't expect it to make any difference at all.

I would want to test against a WAN side local iperf server to confirm that is capable of passing 1G.
Testing against a remote speedtest server there are a lot of variables.

Steve

[bawoodruff]

Off to try and locate a decent (local to me) iperf server, unless someone knows of any in the Indianapolis area, lol

[bawoodruff]

While SSH'ed into the pfSense box and running several tests using"

iperf3 -c speedtest.serverius.net -p 5002 -P 10 -4 -R

shows a bitrate of 280ish Mbps

With the processor cores showing about 20% utilization

[stephenw10]

When I say local I mean actually in your location. When I do this I just use another pfSense box to server as a gateway for the unit under test and run the iperf server on that. That's not ideal but as long as the gateway device is sufficiently powerful it will expose any restrictions on the test device.

Steve

[bawoodruff]

So, I removed all the past settings I had in place and out these in place and reboot.

\boot\loader.conf.local

legal.intel_iwi.license_ack=1
legal.intel_ipw.license_ack=1
net.link.ifqmaxlen=2048
net.isr.defaultqlimit=2048
net.inet.tcp.soreceive_stream=1
net.inet.tcp.syncache.hashsize=1024
net.inet.tcp.syncache.bucketlimit=100
net.pf.source_nodes_hashsize=1048576

System Tunables

ev.igb.0.eee_control=0
dev.igb.1.eee_control=0
dev.igb.2.eee_control=0
dev.igb.3.eee_control=0
dev.igb.0.fc=0
dev.igb.1.fc=0
dev.igb.2.fc=0
dev.igb.3.fc=0
net.inet.ip.redirect=0
net.inet6.ip6.redirect=0
net.route.netisr_maxqlen=2048
net.inet.ip.intr_queue_maxlen=2048
net.inet.udp.maxdgram=16384
net.inet.tcp.syncookies=0
hw.intr_storm_threshold=0
kern.ipc.soacceptqueue=2048

And with FQ_Codel queues and Suricata disabled, I now seem to be hitting 900+Mbps
When I run :iperf3 -c proof.ovh.net -p 5201 -P 40 -R
Had to increase the number of streams, whereas before anything past 10 made no difference.

And with FQ_Codel and Suricata enabled, I now seem to be hitting 850+Mbps

[stephenw10]

Is that testing from pfSense directly?

You usually don't see any improvement with parallel streams beyond the number of NIC queues. iperf itself is still (deliberately) single threaded so it hits a single core limit there.

Steve

[bawoodruff]

Your right, all I was doing was increasing the number of retries generated beyond a certain number of streams. So I lowered the number of streams till the retry counts were at 0.

This is running on a LAN side Linux client to the internet with Suricata and FQ_Codel queues disabled: iperf3 -c proof.ovh.net -p 5201 -P 6 -R -t 30

The throughput is slightly lower today, but I'm attributing that to slightly higher ISP congestion considering the time of day, relative to when I last tested.

[stephenw10]

Hmm, well that looks pretty good. It at least proves the NIC is good.

I believe speedtest uses 4 connections though it could vary by server. You might test at 2 and 4 parallel streams in iperf to get a comparable result.

If you are able to test against your own server locally that will also be good test.

Steve