Multiple OpenVPN clients non-functional

Derelict · Sep 6, 2017, 2:51 AM

I have three. All work perfectly.

johnpoz · Sep 6, 2017, 2:55 AM

Well now I am going to have to fire up 2 more just so I have 4 ;) hehehee.. Couldn't let you have more vpns than me.. ROFL…

douglasg14b · Sep 6, 2017, 3:01 AM

First, thank you for your willingness to help thus far, I really do appreciate it.

@Derelict:

Yes, but we all (or at least many, many, many of us) have multiple OpenVPN clients configured and they work just fine.

Connect to one, and post the Diagnostics > Routes (or netstat -rnfinet)

Connect to another, and post the Diagnostics > Routes (or netstat -rnfinet)

That's my train of thought, I'm obviously not the only person trying to do this, and very few other people seem to have issues. A config issue or something is likely the cause, though I have dug in for a few days before posting here to try and rule out PEBKAC.

Please remember I only enable one client at a time, I'm not running multiple at once for these scenarios.

Here are the routing tables:

Note: Column 1 for VPN IP is the Remote Host, Column 2 is the Virtual Address

IPv4 Routes with Client 1 Connected:


Destination	Gateway		Flags	Use	Mtu	Netif	
default		[WAN IP]	UGS	695883	1500	bce1	
8.8.4.4		[WAN IP]	UGHS	0	1500	bce1	
8.8.8.8		[WAN IP]	UGHS	1	1500	bce1	
10.35.0.9	link#7	UH	11157	1500	ovpnc1	
10.35.0.10	link#7	UHS	0	16384	lo0	
[WAN IP]/23	link#2	U	365731	1500	bce1	
[WAN IP]	link#2	UHS	0	16384	lo0	
[VPN I{]	[VPN IP]	UGHS	751	1500	ovpnc1	
127.0.0.1	link#6	UH	17630565	16384	lo0	
192.168.2.0/24	link#1	U	73771556	1500	bce0	
192.168.2.1	link#1	UHS	204	16384	lo0

IPv4 Routes with Client 2 Connected:


Destination	Gateway		Flags	Use	Mtu	Netif	
default		[WAN IP]	UGS	716552	1500	bce1	
8.8.4.4		[WAN IP]	UGHS	0	1500	bce1	
8.8.8.8		[WAN IP]	UGHS	1	1500	bce1	
10.35.0.13	link#8	UH	125	1500	ovpnc2	
10.35.0.14	link#8	UHS	0	16384	lo0	
[WAN IP]/23	link#2	U	368396	1500	bce1	
[WAN IP]	link#2	UHS	0	16384	lo0	
[VPN IP]	[VPN IP]	UGHS	2	1500	ovpnc2	
127.0.0.1	link#6	UH	18127255	16384	lo0	
192.168.2.0/24	link#1	U	73864363	1500	bce0	
192.168.2.1	link#1	UHS	204	16384	lo0

johnpoz · Sep 6, 2017, 3:21 AM

Why are you hiding this?

[VPN I{] [VPN IP] UGHS 751 1500 ovpnc1

It wouldn't be public..
Note: Column 1 for VPN IP is the Remote Host, Column 2 is the Virtual Address

My conf files are client3.x and client4.x there are not client1.x in the var/etc/openvpn dir..

douglasg14b · Sep 6, 2017, 3:42 AM

@johnpoz:

Why are you hiding this?

[VPN I{] [VPN IP] UGHS 751 1500 ovpnc1

It wouldn't be public..
Note: Column 1 for VPN IP is the Remote Host, Column 2 is the Virtual Address

My conf files are client3.x and client4.x there are not client1.x in the var/etc/openvpn dir..

Oh, here then:

VPN 1:

VPN 2 (non working):

Derelict · Sep 6, 2017, 5:54 AM

Do you see how the tunnel addresses are in 10.35.0.X/?? for both ovpnc1 and ovpnc2? That certainly could be part of the problem, depending on the netmasks involved. It looks like they are using net30 which should be OK.

You also have 8.8.8.8 and 8.8.4.4 apparently assigned to the OpenVPN gateways and those change. That could also be part of the problem, depending on what is set to use those DNS server addresses, including the firewall itself.

I don't know what those routes for the 104.223.91 addresses are, but they are also host routes assigned to individual OpenVPN clients.

You do realize that you have to completely stop and start an OpenVPN client after assigning an interface to it, right?

Not sure what you're doing wrong…

If I …. all connections time out.

What, exactly, does that mean. All connections from where? To where? What find of connection? Do DNS lookups work? pings? is it just web browsing? Everything? Have you sent any traceroutes? where do they stop? Sorry but this is going to require some sleuthing.

douglasg14b · Sep 6, 2017, 5:25 PM

@Derelict:

Do you see how the tunnel addresses are in 10.35.0.X/?? for both ovpnc1 and ovpnc2? That certainly could be part of the problem, depending on the netmasks involved. It looks like they are using net30 which should be OK.

You also have 8.8.8.8 and 8.8.4.4 apparently assigned to the OpenVPN gateways and those change. That could also be part of the problem, depending on what is set to use those DNS server addresses, including the firewall itself.

I don't know what those routes for the 104.223.91 addresses are, but they are also host routes assigned to individual OpenVPN clients.

You do realize that you have to completely stop and start an OpenVPN client after assigning an interface to it, right?

Not sure what you're doing wrong…

If I …. all connections time out.

What, exactly, does that mean. All connections from where? To where? What find of connection? Do DNS lookups work? pings? is it just web browsing? Everything? Have you sent any traceroutes? where do they stop? Sorry but this is going to require some sleuthing.

I stop the clients before changing or assigning the interface, yes. As for timing out, I can't ping any addresses I've tried, DNS lookups fail (If I clear my local DNS cache), traceroutes timeout at every hop.

I've removed all clients, interfaces, and gateways and setup most of these things from scratch again to try and simplify things and remove the other "junk" that was a result of me testing. Here are my routes now (issue still exists)

Client 1 (working)

Client 2 (timeouts)

Derelict · Sep 6, 2017, 6:35 PM

What are your policy routing rules that send traffic over the openvpn circuits?

douglasg14b · Sep 8, 2017, 6:40 PM

@Derelict:

What are your policy routing rules that send traffic over the openvpn circuits?

The first two rules are evaluated first to allow certain devices or sites to use the WAN gateway instead of the VPN. This is mostly due to certain services restricting or blocking access. These work as expected.

The highlighted rule is the VPN rule, it should be a catch-all.

The next two rules are to try and prevent torrents from leaking over my WAN if for some reason the VPN is down.

The final rule is the WAN catch-all for IPv4.

When I try and use my 2nd VPN client I just go into the highlighted rule and switch it to the gateway for the 2nd VPN client for testing purpose. I can switch the interfaces for each client around in the OpenVPN config and the first one will always work and the 2nd not, regardless of interface & gateway.

Derelict · Sep 8, 2017, 8:34 PM

Sorry. I know that would work if I tried to duplicate it. No idea what you're doing wrong. But you're doing something wrong somewhere.

douglasg14b · Sep 9, 2017, 12:34 AM

@Derelict:

Sorry. I know that would work if I tried to duplicate it. No idea what you're doing wrong. But you're doing something wrong somewhere.

I've checked and referenced to the point where I'm going crazy, and others on here don't seem to know at this point either as my setup looks correct.

100% sure there isn't a bug in PFSense specific to my setup? By process of elimination how much is left? Any way I can try and diagnose this further to rule out additional items, or rule in others? Multiple VPN connections is a must-have for me, so I'm at a loss here…

Edit: Could this have anything to do with the difficult I have with assigning an interface to the OpenVPN client? When I try it would always tell me:

An IPv4 protocol was selected, but the selected interface has no IPv4 address.

Which always seemed like a catch-22 since it can't have an IPv4 address till I start the OpenVPN client up. This can be found on https://github.com/pfsense/pfsense/blob/master/src/usr/local/www/vpn_openvpn_server.php#L277 .

Derelict · Sep 9, 2017, 1:08 AM

You don't put an address on an OpenVPN assigned interface. They get Enable checked, a name, then None / None selected there. That's it.

I have never seen that message when assigning an interface. What, exactly, are you doing when it happens?

douglasg14b · Sep 9, 2017, 1:24 AM

@Derelict:

You don't put an address on an OpenVPN assigned interface. They get Enable checked, a name, then None / None selected there. That's it.

I have never seen that message when assigning an interface. What, exactly, are you doing when it happens?

I get this error when creating or editing the OpenVPN client when I have an interface like the one below selected. I can just create a new client, enter any random gibberish into it and select an interface and that error will pop up. I get around it by commenting out that line of code.

Here is an example OpenVPN client: (not embedded because long) http://i.imgur.com/znt5J08.png

Interface is setup with none/none:

Derelict · Sep 9, 2017, 5:47 AM

Aha. I knew it had to be something.

The interface you select there is not the assigned interface, it is the interface used to establish the VPN connection - usually WAN. Or maybe a gateway group if you are running multi-wan and you want it to switch if your tier 1 fails.

So all of your OpenVPN client configurations will have WAN selected there because that is the interface used to ESTABLISH the VPN connection.

johnpoz · Sep 9, 2017, 10:20 AM

Good catch Derelict.. Glad that is settled…

douglasg14b · Sep 9, 2017, 5:43 PM

@Derelict:

Aha. I knew it had to be something.

The interface you select there is not the assigned interface, it is the interface used to establish the VPN connection - usually WAN. Or maybe a gateway group if you are running multi-wan and you want it to switch if your tier 1 fails.

So all of your OpenVPN client configurations will have WAN selected there because that is the interface used to ESTABLISH the VPN connection.

Sorry for being dull, but I'm gonna parrot what you said to make sure I understand.

The WAN interface is used to establish the VPN connection, so regardless of what interface I have selected there (unless I have multi-wan) it will use the WAN interface?

If that's the case, how do I set up routing rules for the VPN clients if they will all use the WAN interface? Do I always select WAN in my OpenVPN client setup?

Derelict · Sep 9, 2017, 7:15 PM

You have to specify a real interface there. In the example you posted to imgur, when your firewall needs to connect to xxx.vpnaccess.com on server port xxxx it will use the interface specified there to source from for the TUNNEL PACKETS - the Outer tunnel packets. It has nothing to do with what traffic is sent through the tunnel itself. It has to be a real interface. What you have there is nonsense.

Create all three OpenVPN clients. Tell them all to use the WAN interface.

Go into Interfaces > (assign). Assign an interface to each ovpncx instance.

Edit each interface, enable it, name it, leave the IPv4 and IPv6 configurations as None.

Make sure you bounce each OpenVPN as this is a required step after initial interface assignment. Another Edit/Save with no changes of each OpenVPN client will do this, as will a Stop/Start of each service in Status > Services.

Make sure outbound NAT is set for each new interface so traffic leaving that interface gets source translated to that particular tunnel address.

You can then policy route whatever traffic you want over each OpenVPN at any time using the policy routing rules on the source network interface. You will have three OpenVPN gateways to choose from. One on each assigned interface.

douglasg14b · Sep 10, 2017, 7:59 PM

@Derelict:

You have to specify a real interface there. In the example you posted to imgur, when your firewall needs to connect to xxx.vpnaccess.com on server port xxxx it will use the interface specified there to source from for the TUNNEL PACKETS - the Outer tunnel packets. It has nothing to do with what traffic is sent through the tunnel itself. It has to be a real interface. What you have there is nonsense.

Create all three OpenVPN clients. Tell them all to use the WAN interface.

Go into Interfaces > (assign). Assign an interface to each ovpncx instance.

Edit each interface, enable it, name it, leave the IPv4 and IPv6 configurations as None.

Make sure you bounce each OpenVPN as this is a required step after initial interface assignment. Another Edit/Save with no changes of each OpenVPN client will do this, as will a Stop/Start of each service in Status > Services.

Make sure outbound NAT is set for each new interface so traffic leaving that interface gets source translated to that particular tunnel address.

You can then policy route whatever traffic you want over each OpenVPN at any time using the policy routing rules on the source network interface. You will have three OpenVPN gateways to choose from. One on each assigned interface.

And we're working!

Thanks a ton for sticking with me and helping me sort this out.

Derelict · Sep 10, 2017, 8:37 PM

Excellent to hear. Glad it's working.

Derelict · Nov 9, 2017, 10:00 PM

It is absolutely NOT possible to run 2 concurrent OpenVPN clients in 2.4.

Absolute nonsense.