Load updated Intel IX module to get 10Gbps
-
@stephenw10
So to get 10Gbps we would need much bigger hardware then?
I thought the 6100 could be an option, looking at this:
-
That is a total throughput value though. You won't see that in an iperf3 test across an single link.
Yes you need significant CPU power if you want to pass 10Gbps using a single TCP stream like downloading a file.
-
@stephenw10 what appliance would you suggest here?
Not sure if budget allows me to get another PFsense, but I could ask
Otherwise, building something could be an option?
What specs would you recommend to have at least? -
For a single TCP stream like that I would look for fewer cores with a higher frequency. And something newer is obviously more efficient so some relatively recent i3 or i5 for example.
There a few threads discussing specific options.
-
@stephenw10 So you are saying that basically I could take an old desktop I have laying around and add some SFP+ card and that's it?
Just because of the higher CPU clock speed it would be faster?Netgate 8200 for example has only 2.4GHz compared to our current Intel Xeon CPU D-1518 with 2.20GHz.
There won't be much of a difference then?
Also, can we be sure this is actually the limiting factor?
Eg. if I do an Iperf3 with 8 streams, this is not getting faster either? I am missing something conceptional here?
-
iperf itself is deliberately single threaded so you may be limited there. You might need to run multiple iperf processes.
No there's not much difference between the D1518 and C3758:
https://www.cpubenchmark.net/compare/2799vs3696vs4746vs3280/Intel-Xeon-D-1518-vs-Intel-Atom-C3758-vs-Intel-i3-12300-vs-Intel-i3-8300 -
@stephenw10
So in theory I could use an old desktop PC with a CPU with 4+GHz and 16GB RAM, add a Intel X520-DA2 card and should be getting better bandwidth routed?thanks :)
-
If you have one I would certainly try it. Be aware some of those old systems can be very power hungry though.
-
@stephenw10
in the office it does not matter to have even 100W more running all time here, then I would first tell people to not leave their PCs running
I'll buy a card, install PFsense on a PC, backup current instance, restore on new machine, re-assign network interfaces where needed, test. Sounds like a plan? -
@stephenw10
Ok, seems that after all I shall not build a system myself, rather get an official appliance or something with warranty and possibly support.
What appliance would manage to give us the 10Gb here? -
@ogghi said in Load updated Intel IX module to get 10Gbps:
@stephenw10
So to get 10Gbps we would need much bigger hardware then?
I thought the 6100 could be an optionWhat traffic mix is the operational requirement? More specifically
- WAN type
- VPN connections and load
- typical number of concurrent connections, especially high bandwidth connections.
The reason I ask is I wonder how well the test results correlate with the operational performance under likely load conditions.
-
@Patch
Hi there.WAN Type is Init7 10/10Gbps fiber.
VPN connections (speed not relevant here) maximum of 20 ppl on OpenVPN, 3 WireGuard tunnels.pfTop: Up State 1-100/14500
Are those concurrent connections?High bandwidth usually would be the backup server sending backups to S3 storage outside...otherwise not too much high bandwidth things happening.
-
@Patch any idea?
-
The fastest one you can get! If you need to pass a single stream TCP connection through it at close to 10Gbps at least. Like sending backups to S3.
-
@stephenw10 What do you mean with the fastest?
One with the fastest possible CPU?Also I am wondering if the problem is really due to CPU clock / single stream? If I run multiple tests / transfers at the same time to different hosts, those tests will share those ~5Gbit...
? -
Well in a test like that I'd expect to see 7-8Gbps through the D1541 so seeing 5Gbps with a D1518 is not wildly low.
But as I said we have seen reports of dramatically higher throughput using other NICs. I've not tested that myself to confirm though.
-
@stephenw10
I guess I figured out what HW we have here:
Must be one of those guys, except we don't have the 4 port Ethernet card in:
https://www.newegg.com/supermicro-sys-5018d-fn8t-intel-xeon-processor-d-1518-2-2-ghz-cpu-tdp-support-35w-fcbga-1667/p/370-0003-000G9Also now I am thinking: If the Mellanox cards might give us better throughput, I could get a MCX4121A-ACAT which as of this list: https://www.freebsd.org/releases/12.1R/hardware/#support
is supported officially by FreeBSD?
I found that card for ~230 bucks :)You think it's worth a try?
Best regards!
-
@ogghi said in Load updated Intel IX module to get 10Gbps:
MCX4121A-ACAT
We have seen a number of reports of issues with that card specifically so, no, I wouldn't get that one. The user who reported getting close to 25Gbps was using a ConnectX-5 NIC.
-
@stephenw10
All right, I'll try and get a Mellanox MCX512A-ACAT aka ConnectX-5 EN and report back :)Thanks!
-
I will say that I've never tested that myself. It would be good to get a second test with it though as that first result was very surprising.
