J1900 dual wan performace

4o4rh

I have a Quotom J1900 4 port and the igb devices (not the older version with eth devices)

1 port is connected via a fritzbox to fibre router 100Mg Tier 1 - Weight 2
1 port is connected to cable modem 120Mg Tier 1 - Weight 1
I used the weighting because the RTT on the cable is 3x longer than on the fibre.

2 ports connected via lagg to netgear switch downstairs, which has a 2 port lagg connection upstairs to another netgear switch.

From a linux pc upstairs,

With speed test, I am getting a 30% loss on download throughput before the VPN is even considered. The upload seems to be about the same as compared to before pfsense.

If I connect directly to the fritzbox and measure, the download throughput is the full 100Mg.

What can i do to get a better throughput.

stephenw10

More testing

You are load balancing between those WANs. What happens if you policy route a client via just one gateway? Do you still see reduced thoughput?

Unless you have it loaded with packages or traffic shaping etc a J1900 should be easily capable of filling both those with unencrypted traffic.

Steve

4o4rh

@stephenw10 said in J1900 dual wan performace:

More testing

You are load balancing between those WANs. What happens if you policy route a client via just one gateway? Do you still see reduced thoughput?

Unless you have it loaded with packages or traffic shaping etc a J1900 should be easily capable of filling both those with unencrypted traffic.

Steve

Hi Steve, so what is the solution? How would you optimize to get the best through put

local provider (100Mbps fibre) PPPoE to Fritzbox, 1Gbps to pfsense

91Mbps down / 46Mbps up
RTT 3.5ms RTTsd 1.4ms monitor 1.1.1.1

vodafone (120Mbps coax) modem to pfsense 1 Gbps

126Mbps down / 6Mps up
RTT 34.4ms RTTsd 1.3ms monitor 1.1.1.2

I have
1 x VOIP (direct to fibre GW)
1 x VLAN media. netflix, kodi, etc.
2 x VLANs WiFi (guest, home)
1 x VLAN IoT
1 x VLAN windows / linux wired connections
1 x VLAN wired work PCs (vpn connected) - bridged to guest wifi vlan

All above use VPN GW and VPN is using WAN GW
except work laptop which use WAN GW and have their own vpn client.

thanks

** so I tested with a dedicated rule for my PC
1 - WAN1 GW
2 - WAN1 FAILOVER GW
3 - WAN2 GW
4 - WAN2 FAILOVER GW
5 - LOADBALANCE

All results came back as they should be. As soon as a removed the dedicated rule for my PC, I was back to losing 30%

*** so the standard rule I have which sweeps up speedtest is;
protocol: udp/typ
source: any
source port: any
destination: alias (vpn bypass)
destination port: alias

I change the gateway per the previous test, and go back to losing 30% if when using the WANx GW is used.

It appears using an alias has a significant performance impact.

4o4rh

@stephenw10 silly me..... I had only speedtest.net bypass the vpn.
So the servers being selected was occurring over the vpn.

Once I added those to the bypass alias, I am now recording the full bandwidth of 100Mbps with balancing.

But I still come back to my original question to you Steve,
How would you optimize to get the best through put based on below.

local provider (100Mbps fibre) PPPoE to Fritzbox, 1Gbps to pfsense - weight 2
91Mbps down / 46Mbps up
RTT 3.5ms RTTsd 1.4ms monitor 1.1.1.1

vodafone (120Mbps coax) modem to pfsense 1 Gbps  - weight 1
126Mbps down / 6Mps up
RTT 34.4ms RTTsd 1.3ms monitor 1.1.1.2

Do you consider 30% in throughput with expressvpn above normal?
Obviously I expect some reduction.
p.s. why don't i see the total of the balanced connection. i seem to only see the bandwidth of one connection at any given time

stephenw10

The speedtest.net client is usually pretty good at testing a balanced connection, especially at relatively low speeds like this. I would expect it to show close to the full speed from both WANs combined if you're really routing that client correctly.

Policy routing by destination as an FQDN is never going to work correctly for something like speedtest.net that could resolve to thousands of IPs. You need to route by the source IP of your test host to see that.

OpenVPN will open one connection on one WAN, all traffic will go over it.
If you want to use both WANs you need to define 2 OpenVPN clients, one on each WAN, and then policy route to those as a group.

Steve

4o4rh

@stephenw10 I have
WAN1 Tier 1
WAN2 Tier 1 GW-WAN

VPN1 Tier1 GW-VPN
VPN2 Tier2

Both VPNs have GW-WAN as their interface.
Are you saying this is wrong?

stephenw10

Yes, you should put each VPN client on a specific gateway otherwise they may easily end up using the same WAN. There is only ever one connection from each VPN client so they cannot load-balance each tunnel.

Steve

4o4rh

@stephenw10 but i am using two ExpressVPN access points in a failover. Are you saying the issue is with using openvpn with such a failover, or with openvpn using the wan with loadbalancing. Sorry dude, treat me like a baby.

stephenw10

We can't see which interfaces or gateway groups the VPN clients themselves are using.

However since you look to be running the two VPN clients in a failover group only one will be carrying any traffic. And since that one client is a single connection it can only ever use one WAN at a time.

If you want to get the highest VPN throughput you need the two VPN clients in a load-balance gateway. And you need the clients themselves to be set to a specific, WAN gateway so both are used.

Steve

4o4rh

@stephenw10 so i reconfig'd to

default router -> WAN1_Failover
(was load balanced - I read this was not a valid config, next time I should RTFM)
VPN1 -> WAN1_failover
VPN2 -> WAN2_failover

VPN_GW -> VPN1 T1, VPN2 T2

But the bottom line is, VPN_GW doesn't really do me any good then, because I don't get to maximize bandwidth. I am better to assign the profile routing to VPN1 or 2 depending on whether the service is more upload or download hungry to utilize the two wan connections the best.

Thanks. I think my knowledge has increased from catastrophic, to only dangerous

stephenw10

To use both VPN tunnels you need to set the VPN_GW group as load-balance, both VPNs as the same tier. Right now it's failover so clients routed to it will only use one tunnel, even if there are a lot of clients and a lot of connections.

Steve

4o4rh

@stephenw10 last time i tried that, expressvpn didn't seem to like going through two different access points at the same time. I will try again and if still an issue, probably get a backup vpn provider

stephenw10

Mmm, they have to allow you connect with two clients at the same time. I have no idea if they do. I know some providers allow that.

4o4rh

@stephenw10 the problem is not getting two connections, that I have as a hot failover.

If i set the VPN GW services in load balancing, i start to get the below errors.

There were error(s) loading the rules: /tmp/rules.debug:534: sticky-address cannot be redefined - The line in question reads [534]: pass in quick on $VLAN_27_MEDIA $GWGW_GRP_WAN inet proto { tcp udp } from any to ! $LAN_LOCAL_ALL port $PORTS_WAN_GAMING tracker 1572529018 keep state label "USER_RULE: Pass WAN_PSN Ports"

The error goes away if I revert to failover. I did ask once, and they said it is not supported to have the two connections in load balance. Although they let you have 5 connections in total.

I get a 30% performance hit with them anyway, so I am thinking to try protonvpn anyhow.

stephenw10

That's a pfSense error though, nothing to do with ExpressVPN. If you are NATing out of those VPN gateways ExpressVPN wouldn't have idea you were load-balancing anyway.

What is GW_GRP_WAN?

It looks like that traffic would be not via the VPN no?

Steve

4o4rh

@stephenw10 that was WAN_loadBalance. I changed it to the VPN_loadbalance and error is gone and vpn loadbalance appears to to be working.

so i have
VPN_Balance / VPN1_failover -> WAN1_Failover
\ VPN2_failover -> WAN2_Failover

WAN_Balance / WAN1_failover
\ WAN2_failover

default_route -> WAN1_failover

all seems to be working now.

Thanks for all the help

stephenw10

Ok, great, that's what I would expect to need there.