Load updated Intel IX module to get 10Gbps
-
iperf itself is deliberately single threaded so you may be limited there. You might need to run multiple iperf processes.
No there's not much difference between the D1518 and C3758:
https://www.cpubenchmark.net/compare/2799vs3696vs4746vs3280/Intel-Xeon-D-1518-vs-Intel-Atom-C3758-vs-Intel-i3-12300-vs-Intel-i3-8300 -
@stephenw10
So in theory I could use an old desktop PC with a CPU with 4+GHz and 16GB RAM, add a Intel X520-DA2 card and should be getting better bandwidth routed?thanks :)
-
If you have one I would certainly try it. Be aware some of those old systems can be very power hungry though.
-
@stephenw10
in the office it does not matter to have even 100W more running all time here, then I would first tell people to not leave their PCs running
I'll buy a card, install PFsense on a PC, backup current instance, restore on new machine, re-assign network interfaces where needed, test. Sounds like a plan? -
@stephenw10
Ok, seems that after all I shall not build a system myself, rather get an official appliance or something with warranty and possibly support.
What appliance would manage to give us the 10Gb here? -
@ogghi said in Load updated Intel IX module to get 10Gbps:
@stephenw10
So to get 10Gbps we would need much bigger hardware then?
I thought the 6100 could be an optionWhat traffic mix is the operational requirement? More specifically
- WAN type
- VPN connections and load
- typical number of concurrent connections, especially high bandwidth connections.
The reason I ask is I wonder how well the test results correlate with the operational performance under likely load conditions.
-
@Patch
Hi there.WAN Type is Init7 10/10Gbps fiber.
VPN connections (speed not relevant here) maximum of 20 ppl on OpenVPN, 3 WireGuard tunnels.pfTop: Up State 1-100/14500
Are those concurrent connections?High bandwidth usually would be the backup server sending backups to S3 storage outside...otherwise not too much high bandwidth things happening.
-
@Patch any idea?
-
The fastest one you can get! If you need to pass a single stream TCP connection through it at close to 10Gbps at least. Like sending backups to S3.
-
@stephenw10 What do you mean with the fastest?
One with the fastest possible CPU?Also I am wondering if the problem is really due to CPU clock / single stream? If I run multiple tests / transfers at the same time to different hosts, those tests will share those ~5Gbit...
? -
Well in a test like that I'd expect to see 7-8Gbps through the D1541 so seeing 5Gbps with a D1518 is not wildly low.
But as I said we have seen reports of dramatically higher throughput using other NICs. I've not tested that myself to confirm though.
-
@stephenw10
I guess I figured out what HW we have here:
Must be one of those guys, except we don't have the 4 port Ethernet card in:
https://www.newegg.com/supermicro-sys-5018d-fn8t-intel-xeon-processor-d-1518-2-2-ghz-cpu-tdp-support-35w-fcbga-1667/p/370-0003-000G9Also now I am thinking: If the Mellanox cards might give us better throughput, I could get a MCX4121A-ACAT which as of this list: https://www.freebsd.org/releases/12.1R/hardware/#support
is supported officially by FreeBSD?
I found that card for ~230 bucks :)You think it's worth a try?
Best regards!
-
@ogghi said in Load updated Intel IX module to get 10Gbps:
MCX4121A-ACAT
We have seen a number of reports of issues with that card specifically so, no, I wouldn't get that one. The user who reported getting close to 25Gbps was using a ConnectX-5 NIC.
-
@stephenw10
All right, I'll try and get a Mellanox MCX512A-ACAT aka ConnectX-5 EN and report back :)Thanks!
-
I will say that I've never tested that myself. It would be good to get a second test with it though as that first result was very surprising.
-
I ran a Xeon D-1518 based pfSense system up until recently and the best performance I saw, if I recall correctly, was around ~6-7Gbit/s routing traffic between two different internal network segments (no NAT, no IDS/IPS) via an iperf3 test (single stream). I imagine with NAT in the picture, performance through WAN would have been a bit lower than that.
-
With the included ix NICs I assume? That's about what I'd expect. Which is why the reports of 25Gbps with Mellanox NICs are so surprising.
-
@stephenw10
hi there, I got the card.
How would I go to install it actually?
Install the card, change LAN setting to one of the new ports, apply, switch over cables?
Then do the same for WAN? -
Yes, pretty much exactly that. Just switch the interface assignments to the new NICs.
-
@stephenw10
Hi there!
I was not successful here this morning, had to revert and restore backup. Seems the routes were not updating, same for firewall rules.
All back running. What I saw as warning output above in the webUI made me think the issue with the limitation might be something else:Filter Reload There were error(s) loading the rules: pfctl: interface ix0 bandwidth limited to 4294967295 bps because selected scheduler is 32-bit limited - The line in question reads [0]: @ 2024-03-07 07:38:45So I tried disabling traffic shaper completely.
No change though.But most important, what did go wrong with interface change? I tried in UI to re-assign, also from cmd locally...
