Mixing different NIC Speeds (1Gb & 10Gb) Performance Problem Question
-
@stephenw10 Using a switch with larger buffers only mask the symptoms of broken TCP Flow Control. This isn't a pfSense issue. It's a DOCSIS issue when the WAN speed is greater than a LAN client, ergo 2.35Gbps WAN to 940Mbps LAN
-
My LAN MTU is now 1460 in PF, so far no change is certain speedtests like fast.com when PF lan @ 10Gb. Speeds still slowly tapering down to 500Mbps. However with MTU at 1460 and PF Lan at 1Gb performance is still solid.
-
@ngr2001 Yes what NIC are you using on the pfSense?
-
Intel X550-T2
When that same LAN NIC is moved to 1Gb is reports (rxpause & txpause) when I move the patch cable over to 10Gb these values disappear in the GUI.
-
@ngr2001 Can you do a
show int 10GigabitEthernet1/2/8? -
Ah, that seems like a pretty big clue....
-
The 7250 doesn't have any 10GbaseT ports though so I assume that's a different Intel NIC?
-
@ngr2001 said in Mixing different NIC Speeds (1Gb & 10Gb) Performance Problem Question:
My LAN MTU is now 1460 in PF, so far no change is certain speedtests
Did you set MSS like I suggested though? MTU does nothing to pf.
-
@stephenw10 Copper 10G-BaseT SFP+ modules most likely being used
-
Yes I have SPF+ RJ45 adapters in the 7250, thus I can easily move the CAT8 from one jack to the over for this test.
-
@ngr2001 Can you do a show int 10GigabitEthernet1/2/8?
-
I did not, but I just corrected that. I reset the states, waited a few mins and so far there may be a small boost in performance. I seem to be averaging now 700Mpbs instead of 500Mbps on fast.com, however if I switch ports we get the full 900+. So better but not perfect.
-
1/2/8 - My 10Gb uplink to PF LAN Nic.
I just noticed that 2 of my main clients currently at 1Gb on the switch have a lot of dropped packets. However right now I am running speedtests and I am not seeing the dropped packet counter increasing, so who know under which setup all these packets were dropped.
1Gb Win 11 Client
1Gb Win 11 Client
-
Should I disable flow control on my WAN NIC as a long term production setup being I am on Cable Internet ?
-
@ngr2001 I am not sure you are catching what I have said a few times across the two threads. Ethernet Flow Control is done on L2, which is what you have been configuring. TCP Flow Control is done at L4 which seems to not function when pfSense WAN is connected to DOCSIS cable modem. If you want to use Ethernet Flow control, which IMO is ugly, then you should leave it enabled.
With my setup, I don't use ethernet flow control.
-
Mmm, try with the MTU at the default 1500 so it matches the switch and client.
200k input errors is pretty significant though.
-
Just tried, same diff, speeds quickly taper down to 5-600Mbps.
I clear my stats on switch, right now at least the dropped packet count is not increasing.
-
Just to be clear you are running the switch at layer2 only right?
It's hard to see how adding the switch, or just changing the port speed, can have any effect on the tcp dircetly. It could be that is some how broken upstream such that when introducing some lower lever issue it's very badly effected though.
-
I am not using any layer 3 features that I am aware of right now, thus I guess it should be in layer 2. I do have flow control enabled on the switch though.
SSH@romulus#show run
Current configuration:
!
ver 08.0.95pT213
!
stack unit 1
module 1 icx7250-48-port-management-module
module 2 icx7250-sfp-plus-8port-80g-module
stack-port 1/2/1
stack-port 1/2/3
!vlan 1 name DEFAULT-VLAN by port
router-interface ve 1
!
!
symmetrical-flow-control enable
!
optical-monitor
optical-monitor non-ruckus-optic-enable
aaa authentication web-server default local
aaa authentication login default local
enable aaa console
hostname romulus
ip dhcp-client disable
ip dns server-address 10.0.0.1
ip route 0.0.0.0/0 10.0.0.1
!
no telnet server
!clock timezone us Eastern
!
!
ntp
disable serve
server time.cloudflare.com
!
!
no web-management http
!
manager disable
!
!
manager port-list 987
!
!
interface ethernet 1/1/4
flow-control neg-on
!
interface ethernet 1/1/8
flow-control neg-on
!
interface ethernet 1/1/48
flow-control neg-on
!
interface ethernet 1/2/1
flow-control neg-on
!
interface ethernet 1/2/8
flow-control neg-on
!
interface ve 1
ip address 10.0.0.3 255.255.255.0
!ip ssh idle-time 120
!
end
SSH@romulus# -
