pfSense Open VPN LAN Side issues....
-
Hi, everyone. I'm fairly new to pfSense an definitley not a networking prfessional. I know enough to be dangerous, and generally don't need to ask for help. So, this is a rarity for me to be asking for some assistance.
Before I post the issue, yes, I have read dozens of other articles and posts and tried the offered solutions, but none have worked.
So, thanks in advance for reading what is probably yet another post on this topic.
So, here is the situation:
I have a virtualized environment running on a commercial host, with 11 different servers.It's VMWare, and no I do not have access to VMWare config.
I recently stood up a pfSense VM so that I can shut off public access to things like RDP and whatnot, and only access those through the pfSense OpenVPN connection.
I have installed and configured pfSense, and OpenVPN, and am able to connect to OpenVPN via UDP on port 1194 successfully.
I am also able to, once connected, access the pfSense web interface on the LAN side ip address.However, I cannot for the life of me access other LAN side resources.
Here is a summary of the network config:
All servers have randomly assigned IP Addresses on subnet 255.255.255.0.
All servers have the same gateway on subnet 255.255.255.255 with IP different from the wan subnet IP.All servers have internal LAN IP's of 192.168.0.X and can communicate with eachother no problem.
pfSense firewall has LAN IP of 192.168.0.250
OpenVPN tunnel network is 10.80.60.0/24
OpenVPN local network is 192.168.0.0/24
Server has the push "route 192.168.0.0 255.255.255.0" custom option added.
All firewall rules are default from OpenVPN wizard EXCEPT: I modified the LAN rule to allow any source, any destination, any port, IPV4*.
No NAT rules at all.
According to my research, this should work.
But, it doesn't.Any help would be appreciated. Also happy to post any additional information that might be needed to assist.
-
Post your server1.conf located here:
/var/etc/openvpn -
I blanked out the public IP. All else is unchanged.
dev ovpns1
verb 1
dev-type tun
dev-node /dev/tun1
writepid /var/run/openvpn_server1.pid
#user nobody
#group nobody
script-security 3
daemon
keepalive 10 60
ping-timer-rem
persist-tun
persist-key
proto udp4
cipher AES-128-CBC
auth SHA256
up /usr/local/sbin/ovpn-linkup
down /usr/local/sbin/ovpn-linkdown
client-connect /usr/local/sbin/openvpn.attributes.sh
client-disconnect /usr/local/sbin/openvpn.attributes.sh
local <PUBLIC IP>
tls-server
server 10.80.60.0 255.255.255.0
client-config-dir /var/etc/openvpn-csc/server1
username-as-common-name
plugin /usr/local/lib/openvpn/plugins/openvpn-plugin-auth-script.so /usr/local/sbin/ovpn_auth_verify_async user TG9jYWwgRGF0YWJhc2U= false server1 1194
tls-verify "/usr/local/sbin/ovpn_auth_verify tls 'Crossentric+OpenVPN+Server+Cert' 1"
lport 1194
management /var/etc/openvpn/server1.sock unix
push "route 192.168.0.0 255.255.255.0"
push "dhcp-option DNS 8.8.8.8"
push "dhcp-option DNS 8.8.4.4"
ca /var/etc/openvpn/server1.ca
cert /var/etc/openvpn/server1.cert
key /var/etc/openvpn/server1.key
dh /etc/dh-parameters.2048
tls-auth /var/etc/openvpn/server1.tls-auth 0
ncp-ciphers AES-128-GCM
persist-remote-ip
float
topology subnet
push "route 192.168.0.0 255.255.255.0" -
I also forgot to mention that the pfSense is able to successfully ping my servers on their internal IP addresses.
So, the issue seems isolated to traffic over the tunnel being able to reach the servers on internal LAN IP's. -
All servers have randomly assigned IP Addresses on subnet 255.255.255.0.
I'm assuming you meant to say 192.168.0.0/24 since 255.255.255.0 is not a "subnet".
All servers have the same gateway on subnet 255.255.255.255 with IP different from the wan subnet IP.
255.255.255.255 is a broadcast address, so I'm not sure what you meant here. Please elaborate with specifics.
Everything looks ok from a config standpoint. clients connecting to your VPN should have a route to the LAN on 192.168.0.0/24. Here's what I check:
-
What subnet is the client connecting from? Since you are using a routed tunnel, all subnets need to be unique on both sides. In other words, the client's LAN subnet cannot overlap the server-side LAN or it will break the routing.
-
Verify the devices on your LAN are using PFsense as the default gateway. Which according to your OP, should be 192.168.0.250.
-
How are you trying to access your LAN resources? By hostname, IP or FQDN?
-
If the resources you're trying to access are windows based, the windows firewall blocks traffic outside of its local subnet by default. So, I would disable the windows firewall until basic IP communication is established.
-
Not that it's part of your issue, but I'm curious why you're pushing external DNS servers to your clients on a split tunnel solution.
-
The last line of your config is redundant, the command gets auto-generated from what's defined in the IPv4 Local network(s) box. So, you can remove the redundant directive from the advanced configuration section.
-
-
I really appreciate the response and the time taken, so thank you for that.
Let me clarify the configuration because I probably incorrectly communicated something.
I have 11 servers (the number really is irrelevant actually).
Each of them has an EXTERNAL IP that is provider assigned, and is used for all application traffic with the world.Example:
Address: <provider assigned IP address>
Subnet: 255.255.255.0All 11 servers have the following gateway:
IP: <common gateway IP>
Subnet; 255.255.255.255Also,
All 11 servers have an INTERNAL IP assigned by me, which is used for application internal communications and management only.All Internal IP's are 192.168.0.X
All of my servers are currently working, communicating just fine.
So, the purpose of the pfSense systemis to give me access to the private network form the outside world, and I chose OpenVPN/pfSense to do it.
The objective is not to route any external traffic through PFSense at all.My PFSense VM also has one of the provider assigned external IP's, and one of the internal IP's.
I am able to connect via VPN to the pfSense server via it's external IP address.
Once connected, however, I cannot reach any of the 192.168.0.X addresses OTHER THAN pfSense web interface itself on 192.168.0.250.
It's not a windows issue because I can't even ping the internal addresses when on OpenVPN.
However, the pfSense/OpenVPN server CAN ping the internal IP addresses.I am able to ping all my internal server IPs from the pfSense Server (from both console and web interface).
But, I am not able to ping the internal server IP's from the OpvenVPN client.Additionally:
Currently, the firewall on my external IP's allows access to RDP, SSH, etc.I want to disable those protocols, and only be able to use them when connected to the internal network via OpenVPN hosted on pfSense.
I hope that helps explain what I'm doing a bit better.
Finally, to answer you question, we use external DNS servers because all communication with my application uses external DNS hosts. I don't have internal DNS hosts.
All internal communication is purely IP based, no DNS at all.
I also noticed that the last line of the config is redunant and I will remove it.
But, I'm fairly certain that won't fix the problem. -
@crossentric Its very simple
Network 10.80.60.0 is not known to your windows hosts to be behind 192.168.0.250, so the replies go to your default gateway and discarded
Add a specific route to your windows machines and it will work
route add -p 10.80.60.0 mask 255.255.255.0 192.168.0.250 -
I agree the network isn't that simple, and I assumed it was a routing issue.
And, most of all, THANKS!
I added that route to one of the windows hosts and it worked!
What is the equivalent in Linux (Debian)?
I have 3 windows servers and 8 Linux servers.
Thanks again!!!
-
@crossentric ip route add 10.80.60.0/24 via 192.168.0.250
You have to find how to make this permanent between reboots for your specific debian installationAnd compared to other things.. yes its simple and straight forward
-
@crossentric said in pfSense Open VPN LAN Side issues....:
I agree the network isn't that simple, and I assumed it was a routing issue.
And, most of all, THANKS!
I added that route to one of the windows hosts and it worked!
What is the equivalent in Linux (Debian)?
I have 3 windows servers and 8 Linux servers.
Thanks again!!!
So, PFsense is neither the edge device for your network nor are your servers using PFsense as the default gateway. That little tidbit of info should've been in your OP ;)
While a route on the servers technically works, it's not ideal and will be a management nightmare as you scale. Adding that route on the edge firewall is a cleaner solution.
I also noticed that the last line of the config is redunant and I will remove it.
But, I'm fairly certain that won't fix the problem.
Correct, it wasn't going to solve the problem...it was just a cleanup item.
Finally, to answer you question, we use external DNS servers because all communication with my application uses external DNS hosts. I don't have internal DNS hosts.
I get that your applications use external DNS... that wasn't why I was asking... I was asking because I was curious about hearing a special use case for pushing external DNS to your VPN clients. Because wherever the client is making its connection from has already provided DNS servers for name resolution. So, there's no real reason to push external DNS servers to your clients... other than maybe bypassing DNS filtering or personally preferring certain DNS servers.
-
@marvosa In a hosted environment usually there is no access to the edge router, so putting a route there isn't always feasible. And I doubt it will scale to more than a dozen too.
As for pussing dns to split tunnel clients, well, if there are dns serving internal ip's like here pushing these dns will enable internal name resolution for vpn clients too.
(as a special case) -
-
Ok...so maybe it didn't work.
It seemed to be working, however, I kept getting a ton of disconnects.
So, I have a different question.
Can I just make the tunnel and remote networks the same?
So, when I connect to OpenVPN, I get a 192.168.0.X network, same as the internal network.I've tried this and I still can't reach the hosts on the internal network.
Just wondering why that would be.
Thanks
-
What you are describing is a tap (compared to a tun) setup.
However, it won't solve you the disconnects issue. (what kind? logs?)
This has to be investigated on its own. -
@netblues adding routes to windows machines is never a good idea.
can you do it? sure. there's a better way. you could have had him set up virtual ips or aliases... would've worked too and been a lot easier.
-
@mcc85 What are we talking about?
Static routes not working on windows? Really? Who said that? On what grounds?
Now, in this situation there is no control on the router serving default gw.
So no routing options there.
We are talking about 7 hosts. (and a single route), with no probability to grow to 70.
And as for aliases and vips, please expand your thoughts. In networking usually there are more than one solutions.Now, for the disconnection issues, we need more info.
What gets disconnected? The tunnel? Maybe just browsing windows shares? Name resolution issues? Broadcasts? -
Well I got into the conversation because I’m doing something similar so I figured I would take a look at the first couple comments and I saw someone failingly recommend to open up static routes. That’s old school. People don’t do that anymore unless you’re hardcoding an old database or legacy software so that it’ll work. In this day and age you wanna use more dynamic approach, less hands on. Sure you could make a script that would replicate with a batch or shell script.. but think about how the internet evolved from tcp and dns.
What most people also can’t tell you is that if something is not working, then open up every possible port you can to eliminate the issue. Turn off firewalls. Open up the Dmz. Test the ports internally with software that you know can scan those ports. I can’t give you specific suggestions as to the nature of your issue, I’m not the most knowledgeable person there is but I know when some advice is less good than others.
It sounds like you have a ttl or port closing issue. Check it internally. If it stays running locally then the answer is that the port is getting disconnected between your routers. I spent weeks trying to get a home brew postfix server running only to find out that my isp was blocking my smtp relays and wouldn’t set me up with a reverse ptr record. Huge waste of time. Probably would’ve been better off having a vpn to a clean business grade Ip that didn’t have smtp blocked but meh. You live and learn.
The problem you’re having is not that dissimilar. Use some online ping tools or port checkers if you don’t see an issue between the sites, perhaps it’s a bad ocable? It happens. And it’s a pain in the ass to have done all that work to find a silly reason it wouldn’t work.
-
alright maybe I should’ve read into it a little more I just glanced over - I’ll take my foot to the mouth.. I’ve had plenty of time with VMware and virtubox though.
Maybe you can’t use routing, and that absolutely sucks because if the issue is external then He’s got no real way to troubleshoot it...
What you can do is come up with an overlay and focus on one machine till it’s working. Don’t focus on all 7. Just one. There are a lot of good variables that were suggested.
However... VMware does have some smart bridging finagling you can do. Doesn’t mean some of my advice doesn’t apply though. Disconnections could be something as stupid as a usb/network driver disconnecting... I’m trying to read a little more bear with me
-
@mcc85 Can we stay on topic please?
Generic solutions won't solve specific problems.
This isn't a greenfield design and not all options are open.
7 hosts don't even need a script.
One off route add -p.
And static routing is not old school. Policy routing on pf is mainly static too.
So, do you suggest ospf? Is-is ? Maybe bgp?
On windows?
And how exactly the internet evolved from tcp and dns? Shall we switch off tcp and dns? -
@crossentric said in pfSense Open VPN LAN Side issues....:
So, here is the situation:
I have a virtualized environment running on a commercial host, with 11 different servers.It's VMWare, and no I do not have access to VMWare config.
I recently stood up a pfSense VM so that I can shut off public access to things like RDP and whatnot, and only access those through the pfSense OpenVPN connection.