Policy routing through separate gateway painfully slow
-
I've been trying to separate the OpenVPN connections from my main pfSense box out to a separate virtual and it isn't going well. If I policy route through the OpenVPN connection on the main box, everything performs well. If I connect to the vLAN on the virtual and policy route through the OpenVPN connection there, everything performs well. If I connect to the main box, route through a gateway on a vLAN connected to the virtual, and then policy route through the OpenVPN connection there, I get 85-90% of the normal downstream and literally a couple kbit/s upstream.
Here's a rough idea of how things are setup.
Main pfSense
LAN - vLAN 1, 192.168.218.1/24, no gateway (but a few static routes to my L3 switch for other vLANs which don't need to be filtered)
WAN - DHCP, Verizon FiOS
TRAN_101 - vLAN 101, 10.21.101.2/29, gateway of 10.21.101.1, this vLAN is not routed on the L3 switch
VPNNEWARK - OpenVPN connection to a server I control in Newark, NJVPN pfSense
WAN - 10.21.96.22/24, gateway of 10.21.96.254 (this is the L3 switch mentioned above), web admin accessible
TRAN_101 - 10.21.101.1/29, no gateway, this vLAN is not routed on the L3 switch
VPNNEWARK - OpenVPN connection to a server I control in Newark, NJThere are a few other VPN connections I also want to move in this manner (each getting a vLAN for transit network, with a single rule on the second pfSense to policy route through the VPN tunnel) but they're not listed above since I haven't even gotten the first working correctly.
Any thoughts?
-
Ok, If I Double NAT the performance issue goes away. I guess that means it is an asymmetric routing problem. I explicitly added a gateway to the VPNNEWARK rule, forcing traffic back to 10.21.101.2, but it didn't make a difference and everything still didn't work.