pfSense Open VPN LAN Side issues....



  • Hi, everyone. I'm fairly new to pfSense an definitley not a networking prfessional. I know enough to be dangerous, and generally don't need to ask for help. So, this is a rarity for me to be asking for some assistance.

    Before I post the issue, yes, I have read dozens of other articles and posts and tried the offered solutions, but none have worked.

    So, thanks in advance for reading what is probably yet another post on this topic.

    So, here is the situation:
    I have a virtualized environment running on a commercial host, with 11 different servers.

    It's VMWare, and no I do not have access to VMWare config.

    I recently stood up a pfSense VM so that I can shut off public access to things like RDP and whatnot, and only access those through the pfSense OpenVPN connection.

    I have installed and configured pfSense, and OpenVPN, and am able to connect to OpenVPN via UDP on port 1194 successfully.
    I am also able to, once connected, access the pfSense web interface on the LAN side ip address.

    However, I cannot for the life of me access other LAN side resources.

    Here is a summary of the network config:
    All servers have randomly assigned IP Addresses on subnet 255.255.255.0.
    All servers have the same gateway on subnet 255.255.255.255 with IP different from the wan subnet IP.

    All servers have internal LAN IP's of 192.168.0.X and can communicate with eachother no problem.

    pfSense firewall has LAN IP of 192.168.0.250

    OpenVPN tunnel network is 10.80.60.0/24

    OpenVPN local network is 192.168.0.0/24

    Server has the push "route 192.168.0.0 255.255.255.0" custom option added.

    All firewall rules are default from OpenVPN wizard EXCEPT: I modified the LAN rule to allow any source, any destination, any port, IPV4*.

    No NAT rules at all.

    According to my research, this should work.
    But, it doesn't.

    Any help would be appreciated. Also happy to post any additional information that might be needed to assist.



  • Post your server1.conf located here:

    /var/etc/openvpn
    


  • I blanked out the public IP. All else is unchanged.

    dev ovpns1
    verb 1
    dev-type tun
    dev-node /dev/tun1
    writepid /var/run/openvpn_server1.pid
    #user nobody
    #group nobody
    script-security 3
    daemon
    keepalive 10 60
    ping-timer-rem
    persist-tun
    persist-key
    proto udp4
    cipher AES-128-CBC
    auth SHA256
    up /usr/local/sbin/ovpn-linkup
    down /usr/local/sbin/ovpn-linkdown
    client-connect /usr/local/sbin/openvpn.attributes.sh
    client-disconnect /usr/local/sbin/openvpn.attributes.sh
    local <PUBLIC IP>
    tls-server
    server 10.80.60.0 255.255.255.0
    client-config-dir /var/etc/openvpn-csc/server1
    username-as-common-name
    plugin /usr/local/lib/openvpn/plugins/openvpn-plugin-auth-script.so /usr/local/sbin/ovpn_auth_verify_async user TG9jYWwgRGF0YWJhc2U= false server1 1194
    tls-verify "/usr/local/sbin/ovpn_auth_verify tls 'Crossentric+OpenVPN+Server+Cert' 1"
    lport 1194
    management /var/etc/openvpn/server1.sock unix
    push "route 192.168.0.0 255.255.255.0"
    push "dhcp-option DNS 8.8.8.8"
    push "dhcp-option DNS 8.8.4.4"
    ca /var/etc/openvpn/server1.ca
    cert /var/etc/openvpn/server1.cert
    key /var/etc/openvpn/server1.key
    dh /etc/dh-parameters.2048
    tls-auth /var/etc/openvpn/server1.tls-auth 0
    ncp-ciphers AES-128-GCM
    persist-remote-ip
    float
    topology subnet
    push "route 192.168.0.0 255.255.255.0"



  • I also forgot to mention that the pfSense is able to successfully ping my servers on their internal IP addresses.
    So, the issue seems isolated to traffic over the tunnel being able to reach the servers on internal LAN IP's.



  • All servers have randomly assigned IP Addresses on subnet 255.255.255.0.

    I'm assuming you meant to say 192.168.0.0/24 since 255.255.255.0 is not a "subnet".

    All servers have the same gateway on subnet 255.255.255.255 with IP different from the wan subnet IP.

    255.255.255.255 is a broadcast address, so I'm not sure what you meant here. Please elaborate with specifics.

    Everything looks ok from a config standpoint. clients connecting to your VPN should have a route to the LAN on 192.168.0.0/24. Here's what I check:

    • What subnet is the client connecting from? Since you are using a routed tunnel, all subnets need to be unique on both sides. In other words, the client's LAN subnet cannot overlap the server-side LAN or it will break the routing.

    • Verify the devices on your LAN are using PFsense as the default gateway. Which according to your OP, should be 192.168.0.250.

    • How are you trying to access your LAN resources? By hostname, IP or FQDN?

    • If the resources you're trying to access are windows based, the windows firewall blocks traffic outside of its local subnet by default. So, I would disable the windows firewall until basic IP communication is established.

    • Not that it's part of your issue, but I'm curious why you're pushing external DNS servers to your clients on a split tunnel solution.

    • The last line of your config is redundant, the command gets auto-generated from what's defined in the IPv4 Local network(s) box. So, you can remove the redundant directive from the advanced configuration section.



  • I really appreciate the response and the time taken, so thank you for that.

    Let me clarify the configuration because I probably incorrectly communicated something.

    I have 11 servers (the number really is irrelevant actually).
    Each of them has an EXTERNAL IP that is provider assigned, and is used for all application traffic with the world.

    Example:
    Address: <provider assigned IP address>
    Subnet: 255.255.255.0

    All 11 servers have the following gateway:
    IP: <common gateway IP>
    Subnet; 255.255.255.255

    Also,
    All 11 servers have an INTERNAL IP assigned by me, which is used for application internal communications and management only.

    All Internal IP's are 192.168.0.X

    All of my servers are currently working, communicating just fine.

    So, the purpose of the pfSense systemis to give me access to the private network form the outside world, and I chose OpenVPN/pfSense to do it.
    The objective is not to route any external traffic through PFSense at all.

    My PFSense VM also has one of the provider assigned external IP's, and one of the internal IP's.

    I am able to connect via VPN to the pfSense server via it's external IP address.
    Once connected, however, I cannot reach any of the 192.168.0.X addresses OTHER THAN pfSense web interface itself on 192.168.0.250.
    It's not a windows issue because I can't even ping the internal addresses when on OpenVPN.
    However, the pfSense/OpenVPN server CAN ping the internal IP addresses.

    I am able to ping all my internal server IPs from the pfSense Server (from both console and web interface).
    But, I am not able to ping the internal server IP's from the OpvenVPN client.

    Additionally:
    Currently, the firewall on my external IP's allows access to RDP, SSH, etc.

    I want to disable those protocols, and only be able to use them when connected to the internal network via OpenVPN hosted on pfSense.

    I hope that helps explain what I'm doing a bit better.

    Finally, to answer you question, we use external DNS servers because all communication with my application uses external DNS hosts. I don't have internal DNS hosts.

    All internal communication is purely IP based, no DNS at all.

    I also noticed that the last line of the config is redunant and I will remove it.
    But, I'm fairly certain that won't fix the problem.



  • @crossentric Its very simple
    Network 10.80.60.0 is not known to your windows hosts to be behind 192.168.0.250, so the replies go to your default gateway and discarded
    Add a specific route to your windows machines and it will work
    route add -p 10.80.60.0 mask 255.255.255.0 192.168.0.250



  • I agree the network isn't that simple, and I assumed it was a routing issue.

    And, most of all, THANKS!

    I added that route to one of the windows hosts and it worked!

    What is the equivalent in Linux (Debian)?

    I have 3 windows servers and 8 Linux servers.

    Thanks again!!!



  • @crossentric ip route add 10.80.60.0/24 via 192.168.0.250
    You have to find how to make this permanent between reboots for your specific debian installation

    And compared to other things.. yes its simple and straight forward



  • @crossentric said in pfSense Open VPN LAN Side issues....:

    I agree the network isn't that simple, and I assumed it was a routing issue.

    And, most of all, THANKS!

    I added that route to one of the windows hosts and it worked!

    What is the equivalent in Linux (Debian)?

    I have 3 windows servers and 8 Linux servers.

    Thanks again!!!

    So, PFsense is neither the edge device for your network nor are your servers using PFsense as the default gateway. That little tidbit of info should've been in your OP ;)

    While a route on the servers technically works, it's not ideal and will be a management nightmare as you scale. Adding that route on the edge firewall is a cleaner solution.

    I also noticed that the last line of the config is redunant and I will remove it.

    But, I'm fairly certain that won't fix the problem.

    Correct, it wasn't going to solve the problem...it was just a cleanup item.

    Finally, to answer you question, we use external DNS servers because all communication with my application uses external DNS hosts. I don't have internal DNS hosts.

    I get that your applications use external DNS... that wasn't why I was asking... I was asking because I was curious about hearing a special use case for pushing external DNS to your VPN clients. Because wherever the client is making its connection from has already provided DNS servers for name resolution. So, there's no real reason to push external DNS servers to your clients... other than maybe bypassing DNS filtering or personally preferring certain DNS servers.



  • @marvosa In a hosted environment usually there is no access to the edge router, so putting a route there isn't always feasible. And I doubt it will scale to more than a dozen too.
    As for pussing dns to split tunnel clients, well, if there are dns serving internal ip's like here pushing these dns will enable internal name resolution for vpn clients too.
    (as a special case)



  • I really appreciate everyone's help.

    And, this pfSense solution is to solve one specific problem.

    The balance of the solution will scale just fine.

    @netblues ... no problem making this persistent...I can do that no problem.

    @marvosa ... thanks for the assist.



  • Ok...so maybe it didn't work.

    It seemed to be working, however, I kept getting a ton of disconnects.

    So, I have a different question.

    Can I just make the tunnel and remote networks the same?
    So, when I connect to OpenVPN, I get a 192.168.0.X network, same as the internal network.

    I've tried this and I still can't reach the hosts on the internal network.

    Just wondering why that would be.

    Thanks



  • What you are describing is a tap (compared to a tun) setup.
    However, it won't solve you the disconnects issue. (what kind? logs?)
    This has to be investigated on its own.



  • @netblues adding routes to windows machines is never a good idea.

    can you do it? sure. there's a better way. you could have had him set up virtual ips or aliases... would've worked too and been a lot easier.



  • @mcc85 What are we talking about?
    Static routes not working on windows? Really? Who said that? On what grounds?
    Now, in this situation there is no control on the router serving default gw.
    So no routing options there.
    We are talking about 7 hosts. (and a single route), with no probability to grow to 70.
    And as for aliases and vips, please expand your thoughts. In networking usually there are more than one solutions.

    Now, for the disconnection issues, we need more info.
    What gets disconnected? The tunnel? Maybe just browsing windows shares? Name resolution issues? Broadcasts?



  • Well I got into the conversation because I’m doing something similar so I figured I would take a look at the first couple comments and I saw someone failingly recommend to open up static routes. That’s old school. People don’t do that anymore unless you’re hardcoding an old database or legacy software so that it’ll work. In this day and age you wanna use more dynamic approach, less hands on. Sure you could make a script that would replicate with a batch or shell script.. but think about how the internet evolved from tcp and dns.

    What most people also can’t tell you is that if something is not working, then open up every possible port you can to eliminate the issue. Turn off firewalls. Open up the Dmz. Test the ports internally with software that you know can scan those ports. I can’t give you specific suggestions as to the nature of your issue, I’m not the most knowledgeable person there is but I know when some advice is less good than others.

    It sounds like you have a ttl or port closing issue. Check it internally. If it stays running locally then the answer is that the port is getting disconnected between your routers. I spent weeks trying to get a home brew postfix server running only to find out that my isp was blocking my smtp relays and wouldn’t set me up with a reverse ptr record. Huge waste of time. Probably would’ve been better off having a vpn to a clean business grade Ip that didn’t have smtp blocked but meh. You live and learn.

    The problem you’re having is not that dissimilar. Use some online ping tools or port checkers if you don’t see an issue between the sites, perhaps it’s a bad ocable? It happens. And it’s a pain in the ass to have done all that work to find a silly reason it wouldn’t work.



  • alright maybe I should’ve read into it a little more I just glanced over - I’ll take my foot to the mouth.. I’ve had plenty of time with VMware and virtubox though.

    Maybe you can’t use routing, and that absolutely sucks because if the issue is external then He’s got no real way to troubleshoot it...

    What you can do is come up with an overlay and focus on one machine till it’s working. Don’t focus on all 7. Just one. There are a lot of good variables that were suggested.

    However... VMware does have some smart bridging finagling you can do. Doesn’t mean some of my advice doesn’t apply though. Disconnections could be something as stupid as a usb/network driver disconnecting... I’m trying to read a little more bear with me



  • @mcc85 Can we stay on topic please?
    Generic solutions won't solve specific problems.
    This isn't a greenfield design and not all options are open.
    7 hosts don't even need a script.
    One off route add -p.
    And static routing is not old school. Policy routing on pf is mainly static too.
    So, do you suggest ospf? Is-is ? Maybe bgp?
    On windows?
    And how exactly the internet evolved from tcp and dns? Shall we switch off tcp and dns?



  • @crossentric said in pfSense Open VPN LAN Side issues....:

    So, here is the situation:
    I have a virtualized environment running on a commercial host, with 11 different servers.

    It's VMWare, and no I do not have access to VMWare config.

    I recently stood up a pfSense VM so that I can shut off public access to things like RDP and whatnot, and only access those through the pfSense OpenVPN connection.



  • @netblues

    Static routing “on” windows hosts, that’s definitely old school. Like lmhosts and netbios. I’ve got a better response though still trying to theorize

    Lol and only ospf makes any sense if you’re talking about wan routes... static mappings on the router is what I was saying. Someone mentioned adding the routes to the hosts files or something that’s what I was suggesting against. Perfectly fine for routing.



  • Maybe a map ought to help isolate the problem. He’s got applications transmitting data to and fro externals, he’s trying to harden permissions and ports before he’s got it all working...

    It sounds like trying to change the oil on an engine that’s still running. He’s gotta escalate to more control otherwise he’s gonna lose his mind and bring anyone trying to help along with him.

    And no.... Just no.... You don’t use static mappings unless you absolutely have to, but given the circumstances maybe you made the right call. TCP and DNS are dynamic that was the metaphor. But Let’s not get into that. We can split hairs on that some other time, the point was just to find a way that involves less work and the router would’ve been the best way.

    “However, I cannot for the life of me access other LAN side resources.” // he says it’s all Ip based though, so if he’s able to ping then it should access the resources fine. I wanna say it’s probably an accounting or permissions issue but it could just be a network file sharing issue or even a missing windows credential. Sometimes you can get one or two clients to connect to a share but if that’s all that’s set on the windows machine then you’re not going to get any other connections. It could also be Samba 1.0 CIFS support depending on what version of Windows, like they pulled it from 1709 and on, so the only option there is to install and reboot on the machine he’s accessing on the far side... if they’re up to date or not, but that’d depend on how he’s accessing the files...

    He also said he hooked up one machine and it connected fine but it’s not staying alive... ?

    , gedit /etc/hosts will modify most Unix host files but if you’re looking for back and forth then you also need to set the hostname for the machine... firewall-cmd or whichever equivalent of it is there, it’d be nice to know if the windows machines are workin while the Linux machines aren’t cause then the suggestion about getting the static routes to stay over reboot might help... but makes me think it really might just be a samba cifs 1.0 issue

    I’d take a snapshot of the way it is set and focus on something far lan side.

    Doesn’t sound like it’s just a routing issue if all these suggestions were made



  • All, many thanks for the help and the insight.

    This honestly wasn't supposed to be difficult.

    I've decided to get rid of pfSense altogether and use the facilities my commercial host has. It's not ideal, but it does work.

    All I was trying to do was access my private network remotely as I've done numerous times before with a variety of products.

    This has just cost me too much time as it is.

    Thanks again


Log in to reply