10gb routing not even close
-
I will start by saying how thankful I am pfsense exists. It has been a core piece of my infra for many years.
Now to my issue, which I just can’t seems to shake out.
I have a c2758 based machine that I am trying to get 10gb of throughput on. However it doesn’t seem to be anywhere near that. I started with a chelsio t4 card and thought that was the issue. So I got an intel x520-sr2 and a chelsio t5. Results from the three all net closely the same performance.
iperf tests from server to server on my network only yields 1.12gb at best when routing.
When testing iperf from the same machines on the same l2 I get around 7gb which is a issue all its own.
Overall I would be happy with 5-6gb but I am a far cry from that.
I have tried all of the googleable tuning guides to no end.
I also thought it could be pfsense, so I gave openbsd a spin… with similar results.
Is this server even capable of 10gb. It’s nearly the same as what you buy from netgate(should have just bought it from them).
Any ideas... any at all
-
Mod, this should probably be moved to hardware
-
Hmm very interesting. I have ordered two Chelsio T3 cards off eBay with CX4 Interfaces. Just waiting for them to arrive. While not exactly the same check this video out it could give you some tips on how to optimize your setup. I plan on going through a switch I will report back when I have everything up and running my wan link is 1Gb symmetric.
My PfSense firewall in a :
Intel(R) Core(TM) i5-3470 CPU @ 3.20GHz
Current: 3200 MHz, Max: 3201 MHz
4 CPUs: 1 package(s) x 4 core(s)
AES-NI CPU Crypto: Yes (active)with 4GB of ram. I PCIe slot I believe is gen 2 maybe 3 but I play on putting the card in the x16 slot.
My Server is a Dual Xeon E2690 with 128 GB RAM 1 TB SSD Boot Drive / 32 (8x4) TB HDD running storage spaces with 512 GB SSD Cache. The server is running Windows server 2012 R2. First I will check switch speed then I will check routing performance I would be shocked if I can get filtering performance anywhere near 10Gb but if I can get it above 2-3 Gbps I would feel like my money was well spent.
-
I have 5 of these servers
One running fedora 26 - mellanox connect x
two centos 7 - mellanox connect x
one pfsense - Chelsio T520
one openbsd - Intel X520
One freenas - Chelsio S320All connected via fiber to an arista 10GB switch
Everything expect the pfsense and openbsd boxes will get nearly 10G (8ish)
It makes no sense why these two machines want to be weirdos about their line rate, but it gets the full line rate of a 1GB link.Things I have tried so far
Swapping drives from known 10GB working machines to the pfsense machine
swapping SFP+ modules
new fiber cables
swapping fibre cables
moving to known working 10G ports on the switchI am totally at a loss here.. I just can't make sense of it.
-
moved to https://forum.pfsense.org/index.php?topic=139588.0