Low throughput and packloss with pfsense and cable modem in bridge mode.
-
I trust you know that TCP flow control requires some packet loss to work.
-
I am not talking about some minor packetloss, inherent in TCP, but very high icmp packetloss, very slow downloadspeed, or total loss of connectivity during load.
I am sorry if i did not make that clear.Sep 5 11:18:25 rc.gateway_alarm 43569 >>> Gateway alarm: WAN_DHCP (Addr:37.205.xxx.xxx Alarm:1 RTT:6.308ms RTTsd:1.357ms Loss:22%) Sep 8 10:45:43 rc.gateway_alarm 12133 >>> Gateway alarm: WAN_DHCP (Addr:37.205.xxx.xxx Alarm:1 RTT:122.669ms RTTsd:91.533ms Loss:21%) Sep 8 10:48:52 rc.gateway_alarm 86454 >>> Gateway alarm: WAN_DHCP (Addr:37.205.xxx.xxx Alarm:0 RTT:82.858ms RTTsd:92.843ms Loss:15%) Sep 8 16:10:02 rc.gateway_alarm 6178 >>> Gateway alarm: WAN_DHCP (Addr:37.205.xxx.xxx Alarm:1 RTT:39.456ms RTTsd:150.982ms Loss:22%) Sep 8 16:10:45 rc.gateway_alarm 54332 >>> Gateway alarm: WAN_DHCP (Addr:37.205.xxx.xxx Alarm:1 RTT:576.200ms RTTsd:324.412ms Loss:95%) Sep 8 16:10:56 rc.gateway_alarm 97633 >>> Gateway alarm: WAN_DHCP (Addr:37.205.xxx.xxx Alarm:1 RTT:0.000ms RTTsd:0.000ms Loss:100%) -
The WAN connection is DHCP? Not PPPoE?
-
Yeah, no PPPoE, just straight up DHCP.
-
Hmm, what sort of CPU usage do you see when testing this? How does it spread across the cores? The output of
top -aSHwill show that.What you describe is exactly what I'd expect if it's somehow using 1 CPU core and that is pegged at 100%.
Steve
-
@stephenw10 said in Low throughput and packloss with pfsense and cable modem in bridge mode.:
Hmm, what sort of CPU usage do you see when testing this? How does it spread across the cores? The output of
top -aSHwill show that.What you describe is exactly what I'd expect if it's somehow using 1 CPU core and that is pegged at 100%.
Steve
I'll have a look when I have opportunity. The CPU usage on the dashboard does not peak out at 100%, though I realize this might now show the full picture.
However, the router is capable of routing at fullspeed when modem is in router mode, or when I did a local test through the router - Where i had a iperf client on the LAN side of the interface, and an iperf client on the WAN side - it had full throughput.
-
This is from double NAT setup, but should still give a look into cpu usage. I don't know why that whould change, with a public ip on the wan interface, instead of an internal. I can do the other test, I just requires more time and planning, since the network will down.
-
Mmm, no problem there. That's exactly what I expect to see, all 4 cores still show some idle time.
Hard to say what's different when connected direct. It's obviously physically it's connected to a different NIC so that would be where I'd first look. It does 'feel' like a flow control issue though you said you tried that.
Steve
-
@stephenw10 said in Low throughput and packloss with pfsense and cable modem in bridge mode.:
Mmm, no problem there. That's exactly what I expect to see, all 4 cores still show some idle time.
Hard to say what's different when connected direct. It's obviously physically it's connected to a different NIC so that would be where I'd first look. It does 'feel' like a flow control issue though you said you tried that.
Steve
Actually, the physical setup remains the same. Its still just connected directly to one of the switch ports of the Arris modem, no matter if the modem is bridged or not. Maybe the modem does something different with its ethernet ports when bridged.
I've also tried putting on a "mediator" switch between the modem and the pfsense - both a dumb switch and a managed hp one.
-
Hmm, weird.
Is is possible the modem is doing something like using a VLAN upstream which it doesn't do in bridge mode? Or priority tagging maybe?
Though anything like that would apply to the laptop connected to the modem equally.
I guess at a minimum I'd compare the verbose interface status in each situation (ifconfig -vvv) and take some pcaps on WAN to see what is actually missing.
Steve
-
So an update.
I now booted OpenWRT on a usb stick, on the exact same hardware setup, and everything runs flawlessly.
I don't know - I've spend quite some time troubleshooting here, but it definately seems like some wierd issue with pfsense. For now, I'll settle for using something other than pfsense, though it would prefer not to. Thanks the input. @stephenw10
-
Hmm, does the parent NIC link exactly the same way in OpenWRT?
-
I did not note it down, and compare thoroughly, but in general yes - Same IP / subnet, gateway, mtu / linkspeed. I havent had the opportunity to reconnect pfsense and check the difference, as i have a narrow window in the evening, defined by how tired I am that night :)
I tried booting from an opnsense USB, and I had the exact same issue. I then broadened my search to include FreeBSD, and I found a few posts with similar issues for the first time. https://forum.opnsense.org/index.php?topic=11015.0 for instance.
