[SOLVED] OpenVPN client with public IP, don't receive inbound traffic on failover WAN

kribor · Mar 13, 2020, 2:50 PM

I have pfsense with WAN failover set up and working (WAN [default], WAN2)

Now, I want to maintain my public IP regardless of active WAN connection so I got a VPN with a public IP. Should take care things, right?

Almost. I got to a poiint where the VPN reconnects when I disconnect WAN and on outbound traffic it's coming from the public IP of my VPN. Most things fail over to WAN2

The inbound doesn't work without WAN connected and I can't figure out why. I have a ping running toward the public IP and it stops working when the WAN fails over to WAN2. I've tried playing around with the VPN settings and I realized that even though I select WAN2 (not failover group) as interface for the VPN client. It doesn't work until I reconnect WAN cable. Strange, right?

Any ideas on what might be going on here?

Also, I assigned an interface to the vpn client tun/tap device and then allow the ping in through an inbound rule on that interface's IP.

UPD: Solved as described in depth below. In short problem was:

Firewall -> Rules -> OpenVPN tab: Had an allow-all rule from openvpn server wizard (unrelated to this guide). Removed all rules since they break "reply-to" tags. Thanks @stephenw10
When setting up Gateway group for wan failover I had not changed the default gateway of Pfsense itself to the gateway group (System -> Routing)

stephenw10 · Mar 12, 2020, 1:50 AM

@kribor said in OpenVPN client with public IP, don't receive inbound traffic on failover WAN:

I assigned an interface to the vpn client tun/tap device and then allow the ping in through an inbound rule on that interface's IP.

You would need to do that for inbound traffic so that replys go back over the VPN. Make sure you don't have a rule on the OpenVPN tab passing that, it must be passed on the assigned interface tab.

What hardware is this? If it's something with switch ports make sure the WAN2 interface isn't set to reflect the WAN1 port state.

Steve

kribor · Mar 12, 2020, 6:13 AM

@stephenw10 said in OpenVPN client with public IP, don't receive inbound traffic on failover WAN:

You would need to do that for inbound traffic so that replys go back over the VPN. Make sure you don't have a rule on the OpenVPN tab passing that, it must be passed on the assigned interface tab.

I think that's what I've done if I'm understanding you correctly. The inbound pass rule for allowing ping is on the VPN interface OPENVPN_PUB1. Since you talk of OpenVPN tab (which I didn't notice before), there is an allow all rule there, you mean that could be interfering here?

I'm attaching some screenshots here for clarity and will test again when I get back to the fw physically.

Allow ping on interface for public IP:

The (apparently default) config under Openvpn tab (of Firewall rules):

Note: This rule is probably coming from the openvpn Server I also have running and therefore potentially unrelated.

What hardware is this? If it's something with switch ports make sure the WAN2 interface isn't set to reflect the WAN1 port state.

It's a HP360 server, no switch ports as far as I know.

stephenw10 · Mar 12, 2020, 12:41 PM

Yes, the OpenVPN tab is parsed first and includes all OpenVPN connects. Traffic passed there will not get reply-to tags and replies will go back out of the WAN (assuming that's the default route) breaking TCP, among other things.

You can see on those rules all the states and data are on that tab and non on the assigned interface.

You should remove that pass-all rule on the OpenVPN tab. That will apply to all VPN connections but do you need incoming traffic allowed on the other OpenVPN connections? Since those are effectively your WAN connections you almost certainly don't want to be passing all there anyway.

Steve

kribor · Mar 13, 2020, 7:31 AM

Thanks a lot @stephenw10, I removed the rule under Openvpn tab and that took care of the most pressing and frustrating issue for sure. I feel it's very close to being 100% so I will just keep asking until I get there.

What now works:

Pinging the openvpn public IP (lets say it's 200.1.1.1) from internet
Port forwarding from openvpn public IP to a host on LAN - assuming I do SNAT before it hits the LAN. I.e. source IP becomes 10.10.30.1 (FW IP on the LAN).

However, what I would expect to also work but doesn't - when WAN is disconnected - (packets get in, to host and back on LAN interface of pfsense but it is never routed out via VPN (or any interface from what I can tell))

Port forwarding from openvpn public IP to a host on LAN - assuming that LAN host has internet routed through that same IP.
Port forwarding from openvpn public IP to a host on LAN, even if the host that had a port forward to it, is routed out through the normal gateway group.

Are there any other rules that are parsed before that could be preventing the reply-to to be set? It works while WAN is connected but not while in failover

Attaching screenshots for clarity:
Port forward on port 8080:

Disabled SNAT rule with which it works (but hides source IP which I would prefer avoiding)

Rules on LAN (I change order of rule 3 and 4 here to route 10.10.30.2's internet traffic through WAN failover OR via vpn public IP):

Rules on vpn Public IP interface:

Openvpn tab now has no rules

kribor · Mar 13, 2020, 2:44 PM

Solved the final piece!

When setting up WAN fail-over I followed this guide: https://www.cyberciti.biz/faq/howto-configure-dual-wan-load-balance-failover-pfsense-router/

What the guide fails to mention is that you should also set pfsense's default gw to the gateway group. I still had it set to WAN. After fixing looks like:

stephenw10 · Mar 13, 2020, 3:09 PM

Yeah that's a really old guide. You couldn't set a gateway group as the default gateway before 2.4 which didn't exist in 2016.

You should not have to source NAT traffic out of the LAN. The reply-to tag should take care of sending replies back out of the OpenVPN gateway. The only reason you might have to is if the target server is not configured to allow access from outside it's own subnet.

Steve