i226-V not getting expected speed

mcarson75

I recently upgraded my hardware on pfSense 2.7.2, and upgraded my ISP (Google Fiber) to 2 gig. I have not seen anywhere close to that, and have been troubleshooting why. It seems to be down to the pfsense box itself. Hardware specs for it are:

Intel N100
8 gb RAM
(4) Intel i-226V

3@pci0:4:0:0:        class=0x020000 rev=0x04 hdr=0x00 vendor=0x8086 device=0x125c subvendor=0x8086 subdevice=0x0000
    vendor     = 'Intel Corporation'
    device     = 'Ethernet Controller I226-V'
    class      = network
    subclass   = ethernet

I have two Unifi USW-8-Enterprise-POE switches connected via 10-gig fiber link. One server and pfsense are connected to one switch, and another server is connected to the far switch. If I configure iperf server on the near-side server (same switch as pfsense) and run iperf from the far server I get close to 2.5 gigabit, so I know the internal network is capable:

Connecting to host 192.168.150.16, port 5201
[  5] local 192.168.150.15 port 37474 connected to 192.168.150.16 port 5201
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec   249 MBytes  2.09 Gbits/sec  112    270 KBytes
[  5]   1.00-2.00   sec   281 MBytes  2.35 Gbits/sec  411    293 KBytes
[  5]   2.00-3.00   sec   279 MBytes  2.34 Gbits/sec  345    264 KBytes
[  5]   3.00-4.00   sec   281 MBytes  2.35 Gbits/sec  307    272 KBytes
[  5]   4.00-5.00   sec   281 MBytes  2.35 Gbits/sec  319    269 KBytes
[  5]   5.00-6.00   sec   281 MBytes  2.35 Gbits/sec  298    264 KBytes
[  5]   6.00-7.00   sec   280 MBytes  2.35 Gbits/sec  350    259 KBytes
[  5]   7.00-8.00   sec   281 MBytes  2.36 Gbits/sec  292    269 KBytes
[  5]   8.00-9.00   sec   280 MBytes  2.35 Gbits/sec  397    259 KBytes
[  5]   9.00-10.00  sec   281 MBytes  2.36 Gbits/sec  363    266 KBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec  2.71 GBytes  2.33 Gbits/sec  3194             sender
[  5]   0.00-10.04  sec  2.71 GBytes  2.31 Gbits/sec                  receiver

However, if I connect to the same server from pfsense I am only getting about 500 megabit:

Connecting to host 192.168.150.16, port 5201
[  5] local 192.168.150.1 port 55254 connected to 192.168.150.16 port 5201
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec  65.1 MBytes   546 Mbits/sec    0   3.00 MBytes       
[  5]   1.00-2.00   sec  62.1 MBytes   521 Mbits/sec    0   3.00 MBytes       
[  5]   2.00-3.00   sec  64.3 MBytes   539 Mbits/sec    0   3.00 MBytes       
[  5]   3.00-4.00   sec  62.2 MBytes   522 Mbits/sec    0   3.00 MBytes       
[  5]   4.00-5.00   sec  62.2 MBytes   521 Mbits/sec    0   3.00 MBytes       
[  5]   5.00-6.00   sec  61.1 MBytes   513 Mbits/sec    0   3.00 MBytes       
[  5]   6.00-7.00   sec  62.0 MBytes   520 Mbits/sec    0   3.00 MBytes       
[  5]   7.00-8.00   sec  61.5 MBytes   516 Mbits/sec    0   3.00 MBytes       
[  5]   8.00-9.00   sec  61.2 MBytes   513 Mbits/sec    0   3.00 MBytes       
[  5]   9.00-10.00  sec  62.4 MBytes   523 Mbits/sec    0   3.00 MBytes       
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec   624 MBytes   523 Mbits/sec    0             sender
[  5]   0.00-20.01  sec   624 MBytes   262 Mbits/sec                  receiver

I installed the same pfsense config from my older machine (i5-6700, 8 gb RAM). Are there some settings that need to be modified? Or is there something else with the hardware that is a problem?

Some (perhaps) relevant settings:

Dobby_

@mcarson75 said in i226-V not getting expected speed:

I installed the same pfsense config from my older machine (i5-6700, 8 gb RAM). Are there some settings that need to be modified? Or is there something else with the hardware that is a problem?

What are the throughput with this hardware? Also low like ~500 MBit/s? If not it will be the new pfSense hardware I would guess.

mcarson75

I didn't do this specific iperf speed test with the old hardware, but I could get 800-900 mbps through the internet with it (on a gigabit connection). I still have it here, so I need to do some testing I suppose.

stephenw10

Check the CPU core speeds are not stuck at some very low value. Some of those cheap N100 boxes have default BIOS values that are far from optimal.

See: https://forum.netgate.com/topic/186104/topton-n100-reporting-402-mhz

mcarson75

@stephenw10 Seems relevant since I have the exact same Topton hardware :) I'll read through the thread, but I have also ordered a Beelink EQ12 to test out.

Dobby_

@mcarson75 said in i226-V not getting expected speed:

However, if I connect to the same server from pfsense I am only getting about 500 megabit:

In normal it would make more sense in my eyes, if you connect the sender
(server 1) and the receiver (server 2) to the pfSense without any switches
between the set up and then you make the test once more again.

If you will see then something between 2,3x GBit/s and count the TCP/IP
overhead on top you may be ending with nearly 2,5 GBit/s the pfSense
were able to route, because it was running pure through the pfSense
box.