OpenVPN and two pfSense

[squirreldip]

I've been using pfSense for a number of years now and it's fantastic.  My primary is running in am ESXi VM and it works quite well.

I've begun moving to a non-virtualized setup as I want to do more with the pfSense box.  So currently I have two gateways, one VM and one not (my ISP gives two IP's).  I've attached an image which lays out my current home setup (Home Network.jpg).

Before I shutdown the VM I want to make sure all is working on the new system.  Most everything is working but I can't get the OpenVPN server working on the new system.

Originally, I set up the OpenVPN Server on the VM using the wizard (and some web assistance  ;)) but the new I've attempted to do everything manually.  I can connect to either pfSense box via OpenVPN and pass internal traffic but can only browse internet when on the VM.  I believe the issue is within the NAT and/or Rules settings but I believe I have everything duplicated…

Both my OpenVPN clients work.  These clients were also defined and working on the VM but I deleted once I confirmed working on the new box.

Any ideas would be appreciated...

[viragomann]

The rules should permit traffic to WAN for both machines. You haven't mentioned which is the physical and which the virtual.
Have you restarted the box? The outbound NAT sometimes need this to work.

Is the server configured to set the client to redirect whole traffic through the tunnel (Redirect Gateway)?

BTW: I don't understand the LAN rules for OPT3. OPT3 is the OpenVPN server, so why you want to access the server address from LAN? Should it be "OPT3 net"?

[squirreldip]

Sorry - Server 1 is the VM (working) and 2 is the box.

I've attached the Server configs…

Also - Changing from OPT3 Address to OPT3 Net causes the server on the VM to stop working (and still not work on the box).


[viragomann]

Try to remove the Advanced options in server settings. These are deprecated as far as I know

Instead check the "Redirect Gateway" option and "DNS Servers" and enter your DNS servers there. Ensure that you have activated the DNS forwarder or the resolver and that it is listening at LAN interface if you use that IP.

@squirreldip:

Also - Changing from OPT3 Address to OPT3 Net causes the server on the VM to stop working (and still not work on the box).

That's just a firewall rule and OPT3 address is part of OPT3 net. What's the rule good for?

[doktornotor]

@viragomann:

Try to remove the Advanced options in server settings. These are deprecated as far as I know

Yeah, when you look there there's a checkbox for redirect gateway in Tunnel Settings and DNS Servers in Client Settings.

I don't really understand the OPT3 LAN rule at all. VM shut down or not.

[squirreldip]

Thanks for the cleanup suggestions - Changed the server settings and removed the LAN rule…

Connections are much faster...

Still working as before:  VM passes net traffic box does not.

Note about internal traffic:  I can only ping devices that have the same gateway as the OpenVPN device with the exception of the internal server - I get ping response from the server (192.168.2.2) regardless of which VPN I'm attached to.

[doktornotor]

I seriously do not get the point of the exercise here…

-Shut down the redundant whatever that you don't intend to use.

• Plug the "box" firewall in place.
• Job done.

Really cannot meaningfully test anything like this.

[squirreldip]

@doktornotor:

I seriously do not get the point of the exercise here…

-Shut down the redundant whatever that you don't intend to use.

• Plug the "box" firewall in place.
• Job done.

Really cannot meaningfully test anything like this.

Point taken…  I'll do this and re-test when I get home (can only do so much from a remote desktop).

[squirreldip]

@squirreldip:

@doktornotor:

I seriously do not get the point of the exercise here…

-Shut down the redundant whatever that you don't intend to use.

• Plug the "box" firewall in place.
• Job done.

Really cannot meaningfully test anything like this.

Point taken…  I'll do this and re-test when I get home (can only do so much from a remote desktop).

Okay…  Shutdown the VM and changed the IP of the 'box' - can connect to OpenVPN but it still will not pass internet traffic.

[squirreldip]

Hopefully this may spark something:

If I disable either of the OpenVPN client interfaces (either OPT1 or OPT2) the OPT3 (Server) will now pass Internet traffic.  As soon as I re-enable (and make sure all are up) then the VPN will stop passing traffic.  I've made no other changes other than disabling one of the client VPNs.

Are there known issues running multiple OpenVPN connections on a single pfSense system?

Update:  I've attempted to remove all the interfaces, disable all the VPNs then reboot and recreate all.  Same problem with no internet being passed by the OpenVPN Server (OPT3)

[squirreldip]

Another interesting thing:

If I connect to the OPT3 (server) and then stop and restart either of the other two (clients) using "Status:OpenVPN" then internet traffic will pass.

Once I disconnect from OPT3 then reconnect traffic will no longer pass (unless I stop/restart again).

[doktornotor]

I do not get what's the point of assigning the OPT3 interface at all.

[2chemlud]

…here is a tutorial promoting an OPT interface for openVPN setup  if you want to route all traffic through your tunnel

https://forum.pfsense.org/index.php?topic=76015.0

:-)

[squirreldip]

@doktornotor:

I do not get what's the point of assigning the OPT3 interface at all.

I thought I required an interface defined to the OpenVPN server - I removed it…

Same problem persists - with both the clients up I cannot see internet traffic on the OpenVPN client.  Once one is disabled the traffic passes.

[2chemlud]

From my experience: Debugging VPN at this level is PITA. Start from scratch and do only the absolute necessary (preferably without wizzard for site-to-site) or import the config from the working pfSense. Everything else is usually a waste of time…

[doktornotor]

@squirreldip:

Problem persists - with both the clients up I cannot see internet traffic on the OpenVPN client.  Once one is disabled the traffic passes.

What do you mean? Are you trying to push all traffic via two different VPNs like some load ballancing?! Yeah obviously it will cause huge amount of trouble!

[squirreldip]

@2chemlud:

From my experience: Debugging VPN at this level is PITA. Start from scratch and do only the absolute necessary (preferably without wizzard for site-to-site) or import the config from the working pfSense. Everything else is usually a waste of time…

The issues are actually on both boxes.  Once both clients are connected then the server does not pass internet traffic.  This occurs on both systems (VM or box)

[2chemlud]

Once both clients are connected then the server does not pass internet traffic.  This occurs on both systems (VM or box)

"does not pass internet traffic" from where? LAN? VPN clients? I don't even get what is not working in your setup…

[doktornotor]

@2chemlud:

I don't even get what is not working in your setup…

No wonder, with terminology like "see internet traffic on client". Why should some OpenVPN client "see internet traffic"?