Slow WAN Good LAN
-
Hello all,
Vital stats below, with 48 GB of RAM with 1 Gb ports, and typically under 1 on the 1 minute load even when testing. 1000/50 external connection
I've run tests via both IPERF to public servers from the firewall itself as well as locally in house, and to a couple of the common speed test sites. On average I'm pulling around 60 Mb/s out the WAN and less over the VPN of course.
Tested with a computer direct to the modem and got expected speeds.
Internally speeds are the expected near 1 Gb/s.
Have swapped the WAN cable.
Have switched the WAN with which previously tested to expected speeds internally.
I even spoofed the MAC of the interface to something from an entirely different vendor to take out the notion of the ISP throttling my firewall.Any further ideas what could be causing such a loss of throughput?
PowerEdge R620
CPU Type Intel(R) Xeon(R) CPU E5-2609 0 @ 2.40GHz
8 CPUs: 2 package(s) x 4 core(s)
AES-NI CPU Crypto: Yes (active)
QAT Crypto: No
-
Quick update, did a test with the speedtest-cli package and got some more promising results. However, that makes me question the iperf tests from that package and wonder why throughput across the box would be slow still.
Retrieving speedtest.net server list...
Selecting best server based on ping...
Hosted by East Central Energy (Braham, MN) [132.26 km]: 14.704 ms
Testing download speed................................................................................
Download: 801.63 Mbit/s
Testing upload speed......................................................................................................
Upload: 59.24 Mbit/s -
How exactly were you testing with iperf? Between which devices?
Have you tested the WAN speed directly, without pfSense routing the traffic?
What NICs are you usuing?
Steve
-
NIC: Intel(R) GbE 4P I350-t rNDC
I was using the iperf package from the repo against different public servers on https://iperf3serverlist.net/
Tests internally against various servers got good results, so the package seems to be working fine.
I'm currently eying my switch. I restarted it to no avail but taking one of the internal FW ports to plug my desktop directly through the FW gave good results to a public speed test. I have a LACP pair for the link to the FW and as mentioned it gets good tests internally, a router-on-a-stick type setup so it "shouldn't" be an issue there.
Any ideas are welcome, will keep poking the gremlins and update if something comes up.
-
Think I'm going to blame this one on OpenVPN. I have a VPN server set up as my means of accessing the infrastructure gear as an extra security step. After poking at several scenarios if I disable routing through the outbound tunnels the LAN net performs as expected.
I may be misunderstanding IPERF on a whole, the measures it gives back are appropriate for the upload speed but my main concern was in the loss of the download speed.
Now it becomes a matter of figuring why both the VPN server and client side connections seem to be crushing the link even more than they normally do. Historically I've gotten around 400 Mb/s over them, not ideal but tolerable reduction, but 60 Mb/s is just an absurd drop from 1 Gb/s
-
So 60Mbps over OpenVPN? Where is it connecting to?
-
Minnesota to Chicago (USA), so not a stretch by any means. Tested against several of the offered servers with similar results. Interestingly even when I routed traffic over the WAN but I was connected to the FW via my in house VPN it did the same, so it seems to be with the OpenVPN daemon somehow.
No notable changes to the system recently, but I also don't particularly know when the slowdown started. Mostly noted when pulling some large files in the other day and was getting far less than I would expect off places like Github.
In either case, I managed to work around it at least on the client vpn side of things by switching to WireGuard. The in house VPN is still using OVPN so it'd be good to get that sorted up, but it's not as big of a deal since it's just for local admin purposes.
-
Oh so some commercial VPN provider then?
@RegalMonkey said in Slow WAN Good LAN:
Interestingly even when I routed traffic over the WAN but I was connected to the FW via my in house VPN it did the same
So over OpenVPN just on the LAN side? Or coming into the hose from some remote location? If it's remote then you'd be limited by the upload speed on your WAN.
-
Both really.
My infrastructure segment is inaccessible unless you can either get on that vlan through a physical port on the switch, or via a VPN that the FW originates as the server to get on an administrative network.
There are also client mode VPN connections to a commercial provider.
Regardless of if the traffic is coming in via the admin VPN and then out WAN, or on the local segment and then routed over the client VPN out to the web it takes a big hit to throughput. It would be difficult to pin down if it affects traffic both ways given the huge imbalance in the down/up speeds.
It does seem to be limited to traffic routed externally that has the issue though. Running a speed test from the admin net to a local server works as expected despite going through a vpn tunnel to get to that network. But anything either from the admin vpn or going over the external commercial vpn to an external site is heavily limited.
