Connects, but no comms between VPN and LAN2



  • Good afternoon all!

    I am a noob, but not completely noobish to PFSense. I've used it for about 2 years now at home and at work, but havent ever run into a problem. Till now.

    My issue is this. I have a PFSense 2.1 box, with Two physically different NICs for LAN1, and LAN2 (obviously excluding WAN, I figured that was a given). And I have two VPNs, one going to each LAN. LAN1 and VPN1 have worked for a long time, and still are working with absolutely no problems. LAN2 and VPN2 which are new to my network as of 3 days ago, are not working properly. I am able to successfully connect to the VPN2, and get a Tunnel address, but am not getting a LAN2 address, nor am I able to connect, ping, scan, anything on the LAN2 network. And have finally come to a point where I may start pulling my hair out.

    The Scenario is that LAN2 is a VM lab. Its connected to a switch, and some machines that are running VMs, and eventually some Cisco gear. LAN2 has no access to the internet, however I can turn that on when needed to allow for updates/downloads. The purpose for LAN2's VM network is to create a Pentesting environment. SO with the types of vulnerabilities on that network, I dont want to create any problems (so the internet is off). In fact, I also have LAN1 and LAN2 completely severed between each other. Which is why I need VPN2 access to LAN2.

    Some details about the setup, etc. (I'm at work, so I'm doing as much of this as I can by memory)

    LAN1 = 192.168.168.0/24 255.255.255.0
    VPN1 TUN = 10.0.8.0/24

    LAN2 = 192.168.1.0/24 255.255.255.0
    VPN2 TUN = 10.1.0.0/24

    I have verified that I can Ping on LAN2 when connected directly in it. I have also done some NMAP scans with no problems there. Its just once I connect to the VPN2 (successfully I might add) that nothing happens. When I do an ipconfig, or ifconfig I can see my local IP, and the Tunnel IP, but no LAN2 IP (192.168.1.0./24). Again, everything works fine with LAN1 and VPN1. I connect with both PC and Mac (Soon Linux as well). I setup VPN2 the same way I setup VPN1, but with the different CA, and Certs.

    I just cant figure out what is wrong, so I'm looking to you fine folks for assistance. I'm sure there is something I am missing…...

    Help, please... And thank you!

    Regards,
    -Chrisso


  • LAYER 8 Global Moderator

    "LAN2 has no access to the internet, however I can turn that on when needed to allow for updates/downloads."

    How do you accomplish this?  So you need to allow for firewall rules in your lan2 to be able to talk to your vpn2 network.  And you need to allow for rules on your vpn2 network to talk to lan2

    Are you passing your routing info..  What network is your vpn client on, with lan2 being a 192.168.1.0/24 network - you can run into problems if your client is on a 192.168.1.0/24 already.



  • John, Thanks for the reply. Unfortunately, the Clients are in some cases on the same 192.168.1.0/24 network. However, this is just a personal thing, so I have only been testing this from my 192.168.168.0/24 LAN1 network. I am aware of the shortcomings of the .1.0/24 nonsense. I will be changing that once I get a little bit further in the VM environment, and start changing up some of the premade VMs (DeIce 1.100, etc. disks, in case anyone is aware of them).

    You may have nailed it on the head with the Rules. I believe I have setup that 10.1.0.0/24 has access to 192.168.1.0/24, but I dont think I have setup the other end of that. I will double check. When I was looking at my settings (a few hundred times it felt like) yesterday, I tried to match them up as best I could with LAN1 and VPN1 (with the obvious corrections). So I will double check that again, and see if that may be the problem. It sounds like that would be it, but we'll double check.

    So far the way I have had the no internet access on LAN2 is by disabling the only rule created on LAN2. I cant remember the details, but will get a shot of it when I get home. And just to rule out any possibility of that being the problem, I did turn it on and try everything with the same problems. So I have since then turned it off.

    Again, Thank you for the quick reply, John.
    -Chrisso



  • Sorry I am just now getting to this. I was running a fever yesterday, so when I finally got home, I threw back some NyQuil and called it a night.

    Home today, and looking at the settings, I made a mistake in my description for VPN2 Tunnel. The tunnel address is actually "10.1.0.0/24", so please be aware of that when reading the following. I corrected the posts above to reflect what is correct.

    I did verify that the OpenVPN rules allowed flow back, with the following:
    Proto:IPv4, Source:, port:, Destination:, port:, Gateway:*, Queue:none

    I also want to post the LAN2 to VPN2 rules that I created:
    Proto:IPv4, Source: 10.1.0.0/24, Port:, Destination:192.168.1.0/24, Port:, Gateway:*, Queue:none.

    The following rule is what I use to disable internet access to the LAN2 subnet. I just disable it to kill inet access.
    LAN2, Proto:IPv4, Source:Lan2 net, Port:, Destination:, Port:, Gateway:, Queue:none.

    Hopefully all of that makes sense. Again, I'm on PFSense 2.1-Release (amd64). Any and all suggestions are greatly appreciated.

    Regards,
    -Chrisso



  • On VPN2 server, what have you got in Local Network/s?
    That is how the client connecting to VPN2 is told what network/s can be reached across the OpenVPN. You will need to put LAN2 subnet there. (and connect in from a place that is not already using the LAN2 subnet addresses.
    Post server and client conf files and then we can see what routes the OpenVPN link is being told about…



  • On the VPN2 server setup, I have the Tunnel setup with 10.1.0.0/24, and the return LAN setup as 192.168.1.0/24 (which is LAN2).

    The Client Config file is as follows, but with a different DYNDNS address:

    dev tun
    persist-tun
    persist-key
    cipher AES-128-CBC
    auth SHA1
    tls-client
    client
    resolv-retry infinite
    remote Random.DYNDNS.org 34447 udp
    lport 0
    verify-x509-name "rnsuser" name
    auth-user-pass
    pkcs12 pfSense-udp-34447-rnsuser.p12
    tls-auth pfSense-udp-34447-rnsuser-tls.key 1
    comp-lzo
    
    

    And now, I hate to be ultra noob and ask this, but where can I get the server conf files? Googling shows their under /var/etc, and while there is a bunch of stuff there, I do not see any config files?

    Thanks for the help Phil!

    -Chrisso


  • LAYER 8 Global Moderator

    they are in the openvpn folder under there

    [2.1-RELEASE][root@pfsense.local.lan]/var/etc/openvpn(4): ls
    server1.ca            server1.sock          server2.interface
    server1.cert          server1.tls-auth      server2.key
    server1.conf          server1.tls-verify.php server2.sock
    server1.crl-verify    server2.ca            server2.tls-auth
    server1.interface      server2.cert          server2.tls-verify.php
    server1.key            server2.conf

    See where I have server1 and server2.conf – I run one on tcp 443, and other on the default 1194 upd.. Issue is 1194 udp is not always a open port depending on where your located.. While if internet is there, 443 is normally always open.



  • Thanks John.

    Here is the server2.conf file. If something else is needed, please let me know. I changed the static IP info to "PublicIP". Rather not paste that out here.

    dev ovpns2
    dev-type tun
    tun-ipv6
    dev-node /dev/tun2
    writepid /var/run/openvpn_server2.pid
    #user nobody
    #group nobody
    script-security 3
    daemon
    keepalive 10 60
    ping-timer-rem
    persist-tun
    persist-key
    proto udp
    cipher AES-128-CBC
    up /usr/local/sbin/ovpn-linkup
    down /usr/local/sbin/ovpn-linkdown
    client-connect /usr/local/sbin/openvpn.attributes.sh
    client-disconnect /usr/local/sbin/openvpn.attributes.sh
    local PUBLICIP
    tls-server
    server 10.1.0.0 255.255.255.0
    client-config-dir /var/etc/openvpn-csc
    username-as-common-name
    auth-user-pass-verify /var/etc/openvpn/server2.php via-env
    tls-verify /var/etc/openvpn/server2.tls-verify.php
    lport 34447
    management /var/etc/openvpn/server2.sock unix
    max-clients 20
    push "route 192.168.1.0 255.255.255.0"
    client-to-client
    duplicate-cn
    ca /var/etc/openvpn/server2.ca 
    cert /var/etc/openvpn/server2.cert 
    key /var/etc/openvpn/server2.key 
    dh /etc/dh-parameters.1024
    tls-auth /var/etc/openvpn/server2.tls-auth 0
    comp-lzo
    persist-remote-ip
    float
    

    Regards,
    -Chrisso


  • LAYER 8 Global Moderator

    Keep in mind if this rule is not enabled PASS

    LAN2, Proto:IPv4, Source:Lan2 net, Port:, Destination:, Port:, Gateway:, Queue:none.

    You would not be able to talk back out the open vpn connection.  If you want to block internet but allow vpn you would need to have a rule that allows lan2 to talk to your vpn2 network.

    Also your float tab is empty?  If you have rules in there they can block traffic on any interface.

    Can you post your route table so we can see your vpn networks.  Example here are mine.

    Don't mind that 172.15 network - something for another thread, need to clean that up.




  • Thanks John. I Went ahead and re-enabled that rule. Eventually, I'll have to write out something custom to exclude any outbound traffic from LAN2 to the internet.  With this being a "Vulnerable Network", I'd like to avoid any and all "Official Notices", lol.

    -Chrisso


  • LAYER 8 Global Moderator

    well shows your local networks and your openvpn networks - so routing should work.

    you say you have your firewall rules that allow the traffic.. Have you tried restarting the vpn server..  I have had now and then, not very often.  But issue where I would get connected and just not able to talk to my lans.. restart the vpn server and bing bang zoom everything working.

    btw you have validated your client is getting the route to the lan2 network.. So for example here is my work box - currently connected to my home network.

    C:>ipconfig
    Ethernet adapter vpn:

    Connection-specific DNS Suffix  . : local.lan
      IPv4 Address. . . . . . . . . . . : 10.0.200.6
      Subnet Mask . . . . . . . . . . . : 255.255.255.252
      Default Gateway . . . . . . . . . :

    here is traceroute

    C:>tracert -d 192.168.1.100

    Tracing route to 192.168.1.100 over a maximum of 30 hops

    1  212 ms  181 ms  149 ms  10.0.200.1
      2  162 ms  202 ms  263 ms  192.168.1.100

    And attached you see the routes on the box that tell it where to go for my 192.168.1 and .2 networks.

    BTW the times really suck because I have to bounce off a proxy in Jax, FL to get to the internet from work here in Chicago.. So all the way from chicago to FL, then to Chicago again for my pfsense box at home - then back again - doesn't make for really good latency ;)  20 Miles apart and seeing response times like I was going to China ;)




  • I would like to say that I've done exactly that. But maybe I've rebooted the entire PFSense box. (Which I hope would do the same thing?)

    The kid is home with the flu today, so he's streaming "Thomas and Friends". I'll restart the VPN2 server service, and give that a go. Once he runs somewhere else or goes to sleep, I'll reboot the box again.

    If I didnt do that, and it starts working, then…..... I'm drinking tonight.

    Will post an update here soon.

    THanks again John,
    -Chris


  • LAYER 8 Global Moderator

    So question for you - can you actually access your pfsense web gui interface from the vpn2?

    Also I did a bit of edits on my last post, check your routes on your client, etc.



  • No access to the PFS-webgui from VPN2. Not really hitting anything when connected to VPN2. Just get the tunnel, and thats it.

    Screen shot of "ifconfig":

    Screenshot of the "traceroute", and then CTRL+C cause nothing was happening for a long long long time.:

    -Chrisso



  • For the record. Unless I'm thinking or looking at this incorrectly, but here is a screenshot of what I believe to be proper Rules.

    -Chrisso


  • LAYER 8 Global Moderator

    Well your rules should work - but rules are only ingress from outside pfsense – think of arrows pointing to pfsense from outside..

    So inbound into openvpn rule would never have a source of your lan networks.  So those 2 rules above with source of your lans don't come into play.. And then you have duplicate rules.

    Then on your lan2 interface that source of vpnnetwork does not come into play..

    So you have this

    pc ----> vpn pfsense lan ---> pc

    or

    pc ---> lan pfsense vpn ---> pc

    Only the rules were the arrows point to pfsense matter.  You don't do any outbound rules on pfsense.

    So you need vpn rules that allow vpn clients to talk to dest you want to allow.  so your any any rule allow that.

    then you lan rule inbound to lan with lan2 net as source would allow it to talk to vpn.

    So seems from your trace your sending packets to pfsense -- but what does pfsense do with it then?  Does it pass it through and the client is having an issue answering??  Do you have a HOST firewall on say 192.168.1.100 your tracing too?

    Maybe your getting there just fine but the host your trying to talk to has firewall that drops your packets?

    I would prob try and ping your lan2 box from vpn client and sniff on pfsense for that traffic on the lan2 interface - give me a minute and show you an example.

    edit:
    So see here sniffing on pfsense on my lan interface for icmp to or from my lan host 192.168.1.100, you can see the request from my vpn client IP 200.6 to my lan pc 1.100.. Then you see the replies.

    If you do not see the request, then pfsense never sent them for some reason.. If you see the request but not the reply then host either never got them or he doesn't want to reply to something outside of his local network.. Or he sent the reply to the wrong place and not pfsense, etc.

    edit2:  You might have issues getting to pfsense gui, since your only pushing routes to lan2.. You would have to hit the pfsense gui on its lan2 interface IP.. 192.168.1.X in your case.




  • John,

    I can try the ICMP capture, but… When I do a ping request while on VPN2, it times out with no response. I have not turned on WBGui access on LAN2 as I want to aavoid any access to the PFSense box from that LAN2. The idea is to have some buddies get on and pentest the network. Ideally, I'd keep the PFSense box totally out of reach, so there's no cheating, etc.

    I'm starting to think I need to install a previous version of PFSense and start over. I'll post up when I do the ICMP capture here in a few minutes.

    Thanks,
    -Chris


  • LAYER 8 Global Moderator

    Well time out doesn't tell us where the packet stops..  You need to figure out if pfsense is actually sending out the ping on the lan2 network..  If it does then we need to look elsewhere to why you don't get an answer.

    If you don't see the icmp request even go out the lan2 interface of pfsense – then we need to look to pfsense or before pfsense.

    If you see answers on the sniff, then that tells us something in pfsense rules are prob blocking the return to your client.

    As to blocking access - where is the rule for that?  I do believe the webgui runs on all interfaces by default??

    yeah I listen on 80.. just http and simple sockstat shows its listening on all addreresses

    USER    COMMAND    PID  FD PROTO  LOCAL ADDRESS        FOREIGN ADDRESS
    root    lighttpd  48617 9  tcp4  *:80                  :

    So if you don't have a firewall rule to block access from a specific interface to pfsense IP on port your listening on..  You would have access.



  • Here's what I got when I connect to VPN2, and ping 192.168.1.100, and PFbox sniffing it with the same settings you had.

    21:44:14.539641 IP 10.1.0.6 > 192.168.1.100: ICMP echo request, id 62491, seq 0, length 64
    21:44:15.541259 IP 10.1.0.6 > 192.168.1.100: ICMP echo request, id 62491, seq 1, length 64
    21:44:16.542836 IP 10.1.0.6 > 192.168.1.100: ICMP echo request, id 62491, seq 2, length 64
    21:44:17.544443 IP 10.1.0.6 > 192.168.1.100: ICMP echo request, id 62491, seq 3, length 64
    21:44:18.546023 IP 10.1.0.6 > 192.168.1.100: ICMP echo request, id 62491, seq 4, length 64
    21:44:19.547636 IP 10.1.0.6 > 192.168.1.100: ICMP echo request, id 62491, seq 5, length 64
    21:44:20.548851 IP 10.1.0.6 > 192.168.1.100: ICMP echo request, id 62491, seq 6, length 64
    21:44:21.550831 IP 10.1.0.6 > 192.168.1.100: ICMP echo request, id 62491, seq 7, length 64
    21:44:22.552424 IP 10.1.0.6 > 192.168.1.100: ICMP echo request, id 62491, seq 8, length 64
    



  • Boy this is driving me nuts… I've wiped and reinstalled the PF box with 2.0 R3, as well as 2.0.3, and of course the latest 2.1, and set everything up from scratch, and it still wont work. SO I am missing something.

    I'm just going to rest on it for a while, and hopefully something will click when I'm in the shower or something.

    I'm reloading the backup from before I wiped everything, but I'm not against starting from scratch again if someone suggests it.

    Still open for anything.

    Thanks,
    -Chrisso

    PS. I'm starting to see something screwy with VMWare ESXI.... I didnt see it yesterday when I was testing pings etc., but I'm seeing it now. Soo... I need to fix that part first, as we arent seeing anything wrong with my setup here. I'll keep this posted when able.


  • LAYER 8 Global Moderator

    So you see the ping request leave the lan2 interface of pfsense - but not get a response.. So how is pfsense at fault for your timeout?  Pfsense never got a reply, so how could it send it down the vpn tunnel to you.

    Your issue is elsewhere.

    For one a host firewall that your trying to ping blocking the traffic.  So your host never responds.



  • Basically after the wipe and reinstalls, I changed my focus elsewhere. I tested this all before I even considered PFSense as a problem. But later last night, I found problems with the networking side of my ESXI box, and the hosted machines on it.

    Where a premade VM (DeIce 1.100) is supposed to be on address 192.168.1.100, it in fact was not. It was a few days ago, but as of last night, it was actually pulling .1.20. When I found that out (through DHCP leases) and tried to ping it, I still couldnt, however I could ping other VMs on that 192.168.1.0/24 network (which I couldnt before).

    So long story short, I have a problem on my ESXI host within networking.

    I am very sorry, and I feel like a complete jackass….. Honestly, this was tested before hand when everything was working right. Somewhere in that tiny bit of time, I broke something on the ESXI box, and expected it to come back like normal. But it did not, and I did not know that.

    John, thank you very much for your time, and I truly am sorry for wasting so much of it as I did, however I did learn some additional things while troubleshooting all of this.

    Regards,
    -Chrisso


  • LAYER 8 Global Moderator

    troubleshooting a problem is never a waste of time, even if we spent time looking to what the problem was not.. Once we ruled those out as not the problem you get to what the goal was - find the source of the problem.

    And you get the added bonus which is always good!
    "I did learn some additional things while troubleshooting all of this. "

    Let us know how it works out - and I run esxi 5.5 and my pfsense is VM..  With multiple segments on my esxi, etc.  So if you need any help in that area even though its not pfsense directly let me know - glad to help.


Log in to reply