Multiple OpenVPN clients non-functional

douglasg14b · Sep 6, 2017, 2:31 AM

"I am connecting to the same provider, but entirely different servers (One server is dedicated for streaming traffic)."

And your vpn service provides service to multiple connections from the same IP? I find that unlikely..

They do, I pay for it specifically. If I enable multiple clients they all connect and receive IPs from the provider.

I feel like we're getting hung up on issues that are not the root of the problem, and the main body of my posts isn't being read or understood…

The above is a side-issue, I don't need multiple active clients for this problem to occur. I mentioned that I can create two or more clients, all disabled upon creation. If I enable any client except for the first one they do not work, all traffic times out. I only have ONE active VPN connection during this scenario.

Edit: What do you need from me?

If I delete the first, working, client. All other clients still continue to be non-functional when enabled, all clients will have their original config names. If I create an additional client, it will have a config named client1.*, and it will work. This seems to point towards some issue in how PFsense or OpenVPN is handling additional clients?

Derelict · Sep 6, 2017, 2:29 AM

Yes, but we all (or at least many, many, many of us) have multiple OpenVPN clients configured and they work just fine. OpenVPN is routed using the routing table. One client being started should not affect other clients or other traffic unless there is something misconfigured.

You will need to provide more information.

You want to make sure that don't pull routes is enabled on ALL your client configs for now.

Connect to one, and post the Diagnostics > Routes (or netstat -rnfinet)

Connect to another, and post the Diagnostics > Routes (or netstat -rnfinet)

You'll probably want to sanitize public IPs. Everybody does. Please don't sanitize private, RFC1918 addresses. It is possible the VPN provider is screwing this up and we'll need to see that.

johnpoz · Sep 6, 2017, 2:49 AM

Here I just fired up my 2nd vpn client on pfsense - took all of couple of minutes to setup pointing to one of my other vps I have running openvpn-accessserver

As derelict says your going to have to provide more info.. I am quite sure there are many many people with multiple vpn clients setup on pfsense..

Notice my IP I get from vpn - I made sure the openvpn-as were using different networks for vpn clients. Or there would be overlap.. Maybe this what is happening to you. Since maybe they hand out the same networks to all clients because they don't think the same client will be making 2 connections?

Derelict · Sep 6, 2017, 2:51 AM

I have three. All work perfectly.

johnpoz · Sep 6, 2017, 2:55 AM

Well now I am going to have to fire up 2 more just so I have 4 ;) hehehee.. Couldn't let you have more vpns than me.. ROFL…

douglasg14b · Sep 6, 2017, 3:01 AM

First, thank you for your willingness to help thus far, I really do appreciate it.

@Derelict:

Yes, but we all (or at least many, many, many of us) have multiple OpenVPN clients configured and they work just fine.

Connect to one, and post the Diagnostics > Routes (or netstat -rnfinet)

Connect to another, and post the Diagnostics > Routes (or netstat -rnfinet)

That's my train of thought, I'm obviously not the only person trying to do this, and very few other people seem to have issues. A config issue or something is likely the cause, though I have dug in for a few days before posting here to try and rule out PEBKAC.

Please remember I only enable one client at a time, I'm not running multiple at once for these scenarios.

Here are the routing tables:

Note: Column 1 for VPN IP is the Remote Host, Column 2 is the Virtual Address

IPv4 Routes with Client 1 Connected:


Destination	Gateway		Flags	Use	Mtu	Netif	
default		[WAN IP]	UGS	695883	1500	bce1	
8.8.4.4		[WAN IP]	UGHS	0	1500	bce1	
8.8.8.8		[WAN IP]	UGHS	1	1500	bce1	
10.35.0.9	link#7	UH	11157	1500	ovpnc1	
10.35.0.10	link#7	UHS	0	16384	lo0	
[WAN IP]/23	link#2	U	365731	1500	bce1	
[WAN IP]	link#2	UHS	0	16384	lo0	
[VPN I{]	[VPN IP]	UGHS	751	1500	ovpnc1	
127.0.0.1	link#6	UH	17630565	16384	lo0	
192.168.2.0/24	link#1	U	73771556	1500	bce0	
192.168.2.1	link#1	UHS	204	16384	lo0

IPv4 Routes with Client 2 Connected:


Destination	Gateway		Flags	Use	Mtu	Netif	
default		[WAN IP]	UGS	716552	1500	bce1	
8.8.4.4		[WAN IP]	UGHS	0	1500	bce1	
8.8.8.8		[WAN IP]	UGHS	1	1500	bce1	
10.35.0.13	link#8	UH	125	1500	ovpnc2	
10.35.0.14	link#8	UHS	0	16384	lo0	
[WAN IP]/23	link#2	U	368396	1500	bce1	
[WAN IP]	link#2	UHS	0	16384	lo0	
[VPN IP]	[VPN IP]	UGHS	2	1500	ovpnc2	
127.0.0.1	link#6	UH	18127255	16384	lo0	
192.168.2.0/24	link#1	U	73864363	1500	bce0	
192.168.2.1	link#1	UHS	204	16384	lo0

johnpoz · Sep 6, 2017, 3:21 AM

Why are you hiding this?

[VPN I{] [VPN IP] UGHS 751 1500 ovpnc1

It wouldn't be public..
Note: Column 1 for VPN IP is the Remote Host, Column 2 is the Virtual Address

My conf files are client3.x and client4.x there are not client1.x in the var/etc/openvpn dir..

douglasg14b · Sep 6, 2017, 3:42 AM

@johnpoz:

Why are you hiding this?

[VPN I{] [VPN IP] UGHS 751 1500 ovpnc1

It wouldn't be public..
Note: Column 1 for VPN IP is the Remote Host, Column 2 is the Virtual Address

My conf files are client3.x and client4.x there are not client1.x in the var/etc/openvpn dir..

Oh, here then:

VPN 1:

VPN 2 (non working):

Derelict · Sep 6, 2017, 5:54 AM

Do you see how the tunnel addresses are in 10.35.0.X/?? for both ovpnc1 and ovpnc2? That certainly could be part of the problem, depending on the netmasks involved. It looks like they are using net30 which should be OK.

You also have 8.8.8.8 and 8.8.4.4 apparently assigned to the OpenVPN gateways and those change. That could also be part of the problem, depending on what is set to use those DNS server addresses, including the firewall itself.

I don't know what those routes for the 104.223.91 addresses are, but they are also host routes assigned to individual OpenVPN clients.

You do realize that you have to completely stop and start an OpenVPN client after assigning an interface to it, right?

Not sure what you're doing wrong…

If I …. all connections time out.

What, exactly, does that mean. All connections from where? To where? What find of connection? Do DNS lookups work? pings? is it just web browsing? Everything? Have you sent any traceroutes? where do they stop? Sorry but this is going to require some sleuthing.

douglasg14b · Sep 6, 2017, 5:25 PM

@Derelict:

Do you see how the tunnel addresses are in 10.35.0.X/?? for both ovpnc1 and ovpnc2? That certainly could be part of the problem, depending on the netmasks involved. It looks like they are using net30 which should be OK.

You also have 8.8.8.8 and 8.8.4.4 apparently assigned to the OpenVPN gateways and those change. That could also be part of the problem, depending on what is set to use those DNS server addresses, including the firewall itself.

I don't know what those routes for the 104.223.91 addresses are, but they are also host routes assigned to individual OpenVPN clients.

You do realize that you have to completely stop and start an OpenVPN client after assigning an interface to it, right?

Not sure what you're doing wrong…

If I …. all connections time out.

What, exactly, does that mean. All connections from where? To where? What find of connection? Do DNS lookups work? pings? is it just web browsing? Everything? Have you sent any traceroutes? where do they stop? Sorry but this is going to require some sleuthing.

I stop the clients before changing or assigning the interface, yes. As for timing out, I can't ping any addresses I've tried, DNS lookups fail (If I clear my local DNS cache), traceroutes timeout at every hop.

I've removed all clients, interfaces, and gateways and setup most of these things from scratch again to try and simplify things and remove the other "junk" that was a result of me testing. Here are my routes now (issue still exists)

Client 1 (working)

Client 2 (timeouts)

Derelict · Sep 6, 2017, 6:35 PM

What are your policy routing rules that send traffic over the openvpn circuits?

douglasg14b · Sep 8, 2017, 6:40 PM

@Derelict:

What are your policy routing rules that send traffic over the openvpn circuits?

The first two rules are evaluated first to allow certain devices or sites to use the WAN gateway instead of the VPN. This is mostly due to certain services restricting or blocking access. These work as expected.

The highlighted rule is the VPN rule, it should be a catch-all.

The next two rules are to try and prevent torrents from leaking over my WAN if for some reason the VPN is down.

The final rule is the WAN catch-all for IPv4.

When I try and use my 2nd VPN client I just go into the highlighted rule and switch it to the gateway for the 2nd VPN client for testing purpose. I can switch the interfaces for each client around in the OpenVPN config and the first one will always work and the 2nd not, regardless of interface & gateway.

Derelict · Sep 8, 2017, 8:34 PM

Sorry. I know that would work if I tried to duplicate it. No idea what you're doing wrong. But you're doing something wrong somewhere.

douglasg14b · Sep 9, 2017, 12:34 AM

@Derelict:

Sorry. I know that would work if I tried to duplicate it. No idea what you're doing wrong. But you're doing something wrong somewhere.

I've checked and referenced to the point where I'm going crazy, and others on here don't seem to know at this point either as my setup looks correct.

100% sure there isn't a bug in PFSense specific to my setup? By process of elimination how much is left? Any way I can try and diagnose this further to rule out additional items, or rule in others? Multiple VPN connections is a must-have for me, so I'm at a loss here…

Edit: Could this have anything to do with the difficult I have with assigning an interface to the OpenVPN client? When I try it would always tell me:

An IPv4 protocol was selected, but the selected interface has no IPv4 address.

Which always seemed like a catch-22 since it can't have an IPv4 address till I start the OpenVPN client up. This can be found on https://github.com/pfsense/pfsense/blob/master/src/usr/local/www/vpn_openvpn_server.php#L277 .

Derelict · Sep 9, 2017, 1:08 AM

You don't put an address on an OpenVPN assigned interface. They get Enable checked, a name, then None / None selected there. That's it.

I have never seen that message when assigning an interface. What, exactly, are you doing when it happens?

douglasg14b · Sep 9, 2017, 1:24 AM

@Derelict:

You don't put an address on an OpenVPN assigned interface. They get Enable checked, a name, then None / None selected there. That's it.

I have never seen that message when assigning an interface. What, exactly, are you doing when it happens?

I get this error when creating or editing the OpenVPN client when I have an interface like the one below selected. I can just create a new client, enter any random gibberish into it and select an interface and that error will pop up. I get around it by commenting out that line of code.

Here is an example OpenVPN client: (not embedded because long) http://i.imgur.com/znt5J08.png

Interface is setup with none/none:

Derelict · Sep 9, 2017, 5:47 AM

Aha. I knew it had to be something.

The interface you select there is not the assigned interface, it is the interface used to establish the VPN connection - usually WAN. Or maybe a gateway group if you are running multi-wan and you want it to switch if your tier 1 fails.

So all of your OpenVPN client configurations will have WAN selected there because that is the interface used to ESTABLISH the VPN connection.

johnpoz · Sep 9, 2017, 10:20 AM

Good catch Derelict.. Glad that is settled…

douglasg14b · Sep 9, 2017, 5:43 PM

@Derelict:

Aha. I knew it had to be something.

The interface you select there is not the assigned interface, it is the interface used to establish the VPN connection - usually WAN. Or maybe a gateway group if you are running multi-wan and you want it to switch if your tier 1 fails.

So all of your OpenVPN client configurations will have WAN selected there because that is the interface used to ESTABLISH the VPN connection.

Sorry for being dull, but I'm gonna parrot what you said to make sure I understand.

The WAN interface is used to establish the VPN connection, so regardless of what interface I have selected there (unless I have multi-wan) it will use the WAN interface?

If that's the case, how do I set up routing rules for the VPN clients if they will all use the WAN interface? Do I always select WAN in my OpenVPN client setup?

Derelict · Sep 9, 2017, 7:15 PM

You have to specify a real interface there. In the example you posted to imgur, when your firewall needs to connect to xxx.vpnaccess.com on server port xxxx it will use the interface specified there to source from for the TUNNEL PACKETS - the Outer tunnel packets. It has nothing to do with what traffic is sent through the tunnel itself. It has to be a real interface. What you have there is nonsense.

Create all three OpenVPN clients. Tell them all to use the WAN interface.

Go into Interfaces > (assign). Assign an interface to each ovpncx instance.

Edit each interface, enable it, name it, leave the IPv4 and IPv6 configurations as None.

Make sure you bounce each OpenVPN as this is a required step after initial interface assignment. Another Edit/Save with no changes of each OpenVPN client will do this, as will a Stop/Start of each service in Status > Services.

Make sure outbound NAT is set for each new interface so traffic leaving that interface gets source translated to that particular tunnel address.

You can then policy route whatever traffic you want over each OpenVPN at any time using the policy routing rules on the source network interface. You will have three OpenVPN gateways to choose from. One on each assigned interface.