Peculiar throughput problem pfSense to pfSense

stephenw10

Yes, you could create some outbound Limiters and use floating rules to capture that. Or just add an altq based shaper queue as default on WAN with a limit on it less than 360Mbps.

It would be a good test either way.

Or try testing from a wired client behind the 2100 which should easily hit the issue if it exists.

keyser

@stephenw10 I will run a test with a wired client later tonight.
Does the 2100 mvneta interace support ALTQ? I cannot seem to locate it in the supported interfaces list.

stephenw10

Oh good point!.... Yes it is, so you should be good to test either shaper type.

It is in the list in Plus but not CE.

keyser

@stephenw10 Right.... Sooo, the plot thickens :-(

I cannot replicate the issue - nor the packet loss - from internal clients. Any internal client on Site B that connects to the public IP of the Site A 6100 (no IPsec) transfers the file without packet loss. The speed of the wired client starts out faster than wireless clients, but very quickly tapers and settles at 7 MB/s throughput which is similar to the wireless client.

So now I'm at a complete loss... This seem to suggest that something the pfSense itself does on heavy WAN access causes the interface or GPON to drop packets that pfSense believes it has transmitted. But the same thing does not happen to packets it forwards....

Incidentally - the wired client showed that my WAN speed is actually 450 Mbps symmetrically. I can consistently get those numbers in different tests from a wired client. The 360 Mbps I reported is obviously capped by Wireless then.

This is just baffling.....

Should I try to implement ALTQ with Codel to see if makes a difference?

stephenw10

Yes I would try using codel.

But it still 'feels' like a TCP issue from the 2100 directly. Especially since that would also apply to traffic going over the VPN.

You might try setting one of the other congestion control algorithms like:

[25.03-BETA][admin@2100-3.stevew.lan]/root: kldload cc_vegas
[25.03-BETA][admin@2100-3.stevew.lan]/root: sysctl net.inet.tcp.cc
net.inet.tcp.cc.vegas.beta: 3
net.inet.tcp.cc.vegas.alpha: 1
net.inet.tcp.cc.abe_frlossreduce: 0
net.inet.tcp.cc.abe: 0
net.inet.tcp.cc.hystartplusplus.bblogs: 0
net.inet.tcp.cc.hystartplusplus.css_rounds: 5
net.inet.tcp.cc.hystartplusplus.css_growth_div: 4
net.inet.tcp.cc.hystartplusplus.n_rttsamples: 8
net.inet.tcp.cc.hystartplusplus.maxrtt_thresh: 16000
net.inet.tcp.cc.hystartplusplus.minrtt_thresh: 4000
net.inet.tcp.cc.available: 
CCmod           D PCB count
cubic           * 30
vegas             0

net.inet.tcp.cc.algorithm: cubic

[25.03-BETA][admin@2100-3.stevew.lan]/root: sysctl net.inet.tcp.cc.algorithm=vegas
net.inet.tcp.cc.algorithm: cubic -> vegas

If that makes any difference at all it would be a good clue.

keyser

@stephenw10 I’m leaving the site now, so this might be a tad to experimental to enable when it will be months before I’m back (in case it all goes south). Since I’m not really transferring data in/out of the PfSense itself, this is not a major issue right now. I’ll have a further look when I return

keyser

@stephenw10 but THANK YOU for your invaluable knowledge and desire to help. You really are indirectly one of the invaluable qualities that makes pfSense such a fantastic product.