Not getting 1gb routing between VLANS. I350 Intel NIC, HP DL380e Gen8 dual CPU
-
Good evening. I have pfSense 2.4.5-release-p1 installed on a Gen8 HP DL380e server with dual Xeon E5-2450 CPU's, 40 GB of ECC RAM and an intel I350 4 port NIC. I have the NIC configured with a 1 member LACP LAGG to a Cisco 3750x stack. I have noticed that I am not getting 1gb transfers across network segments (vlans). If I put the PC on the same VLAN and rely only on local switching I saturate a 1GB connection. As soon as pfSense has to route, the same transfer hovers between 65-75 MB/s. There are no interface errors in the cisco switch. I looked through the NIC tuning guide and set nmbclusers to 1 million but it made no difference. I also performed the same transfer with snort disabled and got the same results. Any advice on the best way to approach troubleshooting this? I feel like my hardware should be able to do gigabit routing. It's not like I'm using a raspberry pi. Thanks for any help!
-
So you are routing between VLANs over the same link? Router-on-stick style?
How are you testing? Any test that has any significant reply traffic will reduce that.
Steve
-
@stephenw10 said in Not getting 1gb routing between VLANS. I350 Intel NIC, HP DL380e Gen8 dual CPU:
So you are routing between VLANs over the same link? Router-on-stick style?
How are you testing? Any test that has any significant reply traffic will reduce that.
Steve
Yes I guess that's a fairly accurate way to describe it. I have a typo in my OP it is a 2 member 1gbps LACP LAGG total LAGG bandwidth of 2gbps. I guess I thought with a 2gbps port channel uplink to pfSense I should be able to get 1gbps transfers between VLANs. If this is a bad assumption I can try changing out to a single 10gbps link to the stack.
Was that a bad assumption?
-
It depends on the hashing used to identify the connections but between two hosts I would expect a single link to be used. You need multiple connections between multiple hosts to really balance the load usefully.
You should still close to line rate on a UDP test though. With no reply traffic you can send and receive at ~1Gbps simultaneously over a single link.Steve
-
How many vlans do you have on this lagg?
As mentioned you really have little control over which physical interface will be used. When both vlans are on the same lagg. Its quite possible for you to get a hairpin using the same physical interface for intervlan traffic.
If you want to make sure that is not part of the problem - I would move the vlans you want to test routing throughput on different physical interfaces.
Do you see full bandwidth then?
