Connects, but no comms between VPN and LAN2

chrisso

Good afternoon all!

I am a noob, but not completely noobish to PFSense. I've used it for about 2 years now at home and at work, but havent ever run into a problem. Till now.

My issue is this. I have a PFSense 2.1 box, with Two physically different NICs for LAN1, and LAN2 (obviously excluding WAN, I figured that was a given). And I have two VPNs, one going to each LAN. LAN1 and VPN1 have worked for a long time, and still are working with absolutely no problems. LAN2 and VPN2 which are new to my network as of 3 days ago, are not working properly. I am able to successfully connect to the VPN2, and get a Tunnel address, but am not getting a LAN2 address, nor am I able to connect, ping, scan, anything on the LAN2 network. And have finally come to a point where I may start pulling my hair out.

The Scenario is that LAN2 is a VM lab. Its connected to a switch, and some machines that are running VMs, and eventually some Cisco gear. LAN2 has no access to the internet, however I can turn that on when needed to allow for updates/downloads. The purpose for LAN2's VM network is to create a Pentesting environment. SO with the types of vulnerabilities on that network, I dont want to create any problems (so the internet is off). In fact, I also have LAN1 and LAN2 completely severed between each other. Which is why I need VPN2 access to LAN2.

Some details about the setup, etc. (I'm at work, so I'm doing as much of this as I can by memory)

LAN1 = 192.168.168.0/24 255.255.255.0
VPN1 TUN = 10.0.8.0/24

LAN2 = 192.168.1.0/24 255.255.255.0
VPN2 TUN = 10.1.0.0/24

I have verified that I can Ping on LAN2 when connected directly in it. I have also done some NMAP scans with no problems there. Its just once I connect to the VPN2 (successfully I might add) that nothing happens. When I do an ipconfig, or ifconfig I can see my local IP, and the Tunnel IP, but no LAN2 IP (192.168.1.0./24). Again, everything works fine with LAN1 and VPN1. I connect with both PC and Mac (Soon Linux as well). I setup VPN2 the same way I setup VPN1, but with the different CA, and Certs.

I just cant figure out what is wrong, so I'm looking to you fine folks for assistance. I'm sure there is something I am missing…...

Help, please... And thank you!

Regards,
-Chrisso

johnpoz

"LAN2 has no access to the internet, however I can turn that on when needed to allow for updates/downloads."

How do you accomplish this? So you need to allow for firewall rules in your lan2 to be able to talk to your vpn2 network. And you need to allow for rules on your vpn2 network to talk to lan2

Are you passing your routing info.. What network is your vpn client on, with lan2 being a 192.168.1.0/24 network - you can run into problems if your client is on a 192.168.1.0/24 already.

chrisso

John, Thanks for the reply. Unfortunately, the Clients are in some cases on the same 192.168.1.0/24 network. However, this is just a personal thing, so I have only been testing this from my 192.168.168.0/24 LAN1 network. I am aware of the shortcomings of the .1.0/24 nonsense. I will be changing that once I get a little bit further in the VM environment, and start changing up some of the premade VMs (DeIce 1.100, etc. disks, in case anyone is aware of them).

You may have nailed it on the head with the Rules. I believe I have setup that 10.1.0.0/24 has access to 192.168.1.0/24, but I dont think I have setup the other end of that. I will double check. When I was looking at my settings (a few hundred times it felt like) yesterday, I tried to match them up as best I could with LAN1 and VPN1 (with the obvious corrections). So I will double check that again, and see if that may be the problem. It sounds like that would be it, but we'll double check.

So far the way I have had the no internet access on LAN2 is by disabling the only rule created on LAN2. I cant remember the details, but will get a shot of it when I get home. And just to rule out any possibility of that being the problem, I did turn it on and try everything with the same problems. So I have since then turned it off.

Again, Thank you for the quick reply, John.
-Chrisso

chrisso

Sorry I am just now getting to this. I was running a fever yesterday, so when I finally got home, I threw back some NyQuil and called it a night.

Home today, and looking at the settings, I made a mistake in my description for VPN2 Tunnel. The tunnel address is actually "10.1.0.0/24", so please be aware of that when reading the following. I corrected the posts above to reflect what is correct.

I did verify that the OpenVPN rules allowed flow back, with the following:
Proto:IPv4, Source:, port:, Destination:, port:, Gateway:*, Queue:none

I also want to post the LAN2 to VPN2 rules that I created:
Proto:IPv4, Source: 10.1.0.0/24, Port:, Destination:192.168.1.0/24, Port:, Gateway:*, Queue:none.

The following rule is what I use to disable internet access to the LAN2 subnet. I just disable it to kill inet access.
LAN2, Proto:IPv4, Source:Lan2 net, Port:, Destination:, Port:, Gateway:, Queue:none.

Hopefully all of that makes sense. Again, I'm on PFSense 2.1-Release (amd64). Any and all suggestions are greatly appreciated.

Regards,
-Chrisso

phil.davis

On VPN2 server, what have you got in Local Network/s?
That is how the client connecting to VPN2 is told what network/s can be reached across the OpenVPN. You will need to put LAN2 subnet there. (and connect in from a place that is not already using the LAN2 subnet addresses.
Post server and client conf files and then we can see what routes the OpenVPN link is being told about…

chrisso

On the VPN2 server setup, I have the Tunnel setup with 10.1.0.0/24, and the return LAN setup as 192.168.1.0/24 (which is LAN2).

The Client Config file is as follows, but with a different DYNDNS address:

dev tun
persist-tun
persist-key
cipher AES-128-CBC
auth SHA1
tls-client
client
resolv-retry infinite
remote Random.DYNDNS.org 34447 udp
lport 0
verify-x509-name "rnsuser" name
auth-user-pass
pkcs12 pfSense-udp-34447-rnsuser.p12
tls-auth pfSense-udp-34447-rnsuser-tls.key 1
comp-lzo

And now, I hate to be ultra noob and ask this, but where can I get the server conf files? Googling shows their under /var/etc, and while there is a bunch of stuff there, I do not see any config files?

Thanks for the help Phil!

-Chrisso

johnpoz

they are in the openvpn folder under there

[2.1-RELEASE][root@pfsense.local.lan]/var/etc/openvpn(4): ls
server1.ca server1.sock server2.interface
server1.cert server1.tls-auth server2.key
server1.conf server1.tls-verify.php server2.sock
server1.crl-verify server2.ca server2.tls-auth
server1.interface server2.cert server2.tls-verify.php
server1.key server2.conf

See where I have server1 and server2.conf – I run one on tcp 443, and other on the default 1194 upd.. Issue is 1194 udp is not always a open port depending on where your located.. While if internet is there, 443 is normally always open.

chrisso

Thanks John.

Here is the server2.conf file. If something else is needed, please let me know. I changed the static IP info to "PublicIP". Rather not paste that out here.

dev ovpns2
dev-type tun
tun-ipv6
dev-node /dev/tun2
writepid /var/run/openvpn_server2.pid
#user nobody
#group nobody
script-security 3
daemon
keepalive 10 60
ping-timer-rem
persist-tun
persist-key
proto udp
cipher AES-128-CBC
up /usr/local/sbin/ovpn-linkup
down /usr/local/sbin/ovpn-linkdown
client-connect /usr/local/sbin/openvpn.attributes.sh
client-disconnect /usr/local/sbin/openvpn.attributes.sh
local PUBLICIP
tls-server
server 10.1.0.0 255.255.255.0
client-config-dir /var/etc/openvpn-csc
username-as-common-name
auth-user-pass-verify /var/etc/openvpn/server2.php via-env
tls-verify /var/etc/openvpn/server2.tls-verify.php
lport 34447
management /var/etc/openvpn/server2.sock unix
max-clients 20
push "route 192.168.1.0 255.255.255.0"
client-to-client
duplicate-cn
ca /var/etc/openvpn/server2.ca 
cert /var/etc/openvpn/server2.cert 
key /var/etc/openvpn/server2.key 
dh /etc/dh-parameters.1024
tls-auth /var/etc/openvpn/server2.tls-auth 0
comp-lzo
persist-remote-ip
float

Regards,
-Chrisso

johnpoz

Keep in mind if this rule is not enabled PASS

LAN2, Proto:IPv4, Source:Lan2 net, Port:, Destination:, Port:, Gateway:, Queue:none.

You would not be able to talk back out the open vpn connection. If you want to block internet but allow vpn you would need to have a rule that allows lan2 to talk to your vpn2 network.

Also your float tab is empty? If you have rules in there they can block traffic on any interface.

Can you post your route table so we can see your vpn networks. Example here are mine.

Don't mind that 172.15 network - something for another thread, need to clean that up.

routetable1.png_thumb

chrisso

Thanks John. I Went ahead and re-enabled that rule. Eventually, I'll have to write out something custom to exclude any outbound traffic from LAN2 to the internet. With this being a "Vulnerable Network", I'd like to avoid any and all "Official Notices", lol.

-Chrisso

johnpoz

well shows your local networks and your openvpn networks - so routing should work.

you say you have your firewall rules that allow the traffic.. Have you tried restarting the vpn server.. I have had now and then, not very often. But issue where I would get connected and just not able to talk to my lans.. restart the vpn server and bing bang zoom everything working.

btw you have validated your client is getting the route to the lan2 network.. So for example here is my work box - currently connected to my home network.

C:>ipconfig
Ethernet adapter vpn:

Connection-specific DNS Suffix . : local.lan
IPv4 Address. . . . . . . . . . . : 10.0.200.6
Subnet Mask . . . . . . . . . . . : 255.255.255.252
Default Gateway . . . . . . . . . :

here is traceroute

C:>tracert -d 192.168.1.100

Tracing route to 192.168.1.100 over a maximum of 30 hops

1 212 ms 181 ms 149 ms 10.0.200.1
2 162 ms 202 ms 263 ms 192.168.1.100

And attached you see the routes on the box that tell it where to go for my 192.168.1 and .2 networks.

BTW the times really suck because I have to bounce off a proxy in Jax, FL to get to the internet from work here in Chicago.. So all the way from chicago to FL, then to Chicago again for my pfsense box at home - then back again - doesn't make for really good latency ;) 20 Miles apart and seeing response times like I was going to China ;)

routesvpn.png_thumb

chrisso

I would like to say that I've done exactly that. But maybe I've rebooted the entire PFSense box. (Which I hope would do the same thing?)

The kid is home with the flu today, so he's streaming "Thomas and Friends". I'll restart the VPN2 server service, and give that a go. Once he runs somewhere else or goes to sleep, I'll reboot the box again.

If I didnt do that, and it starts working, then…..... I'm drinking tonight.

Will post an update here soon.

THanks again John,
-Chris

johnpoz

So question for you - can you actually access your pfsense web gui interface from the vpn2?

Also I did a bit of edits on my last post, check your routes on your client, etc.

chrisso

No access to the PFS-webgui from VPN2. Not really hitting anything when connected to VPN2. Just get the tunnel, and thats it.

Screen shot of "ifconfig":

Screenshot of the "traceroute", and then CTRL+C cause nothing was happening for a long long long time.:

-Chrisso

chrisso

For the record. Unless I'm thinking or looking at this incorrectly, but here is a screenshot of what I believe to be proper Rules.

-Chrisso

johnpoz

Well your rules should work - but rules are only ingress from outside pfsense – think of arrows pointing to pfsense from outside..

So inbound into openvpn rule would never have a source of your lan networks. So those 2 rules above with source of your lans don't come into play.. And then you have duplicate rules.

Then on your lan2 interface that source of vpnnetwork does not come into play..

So you have this

pc ----> vpn pfsense lan ---> pc

or

pc ---> lan pfsense vpn ---> pc

Only the rules were the arrows point to pfsense matter. You don't do any outbound rules on pfsense.

So you need vpn rules that allow vpn clients to talk to dest you want to allow. so your any any rule allow that.

then you lan rule inbound to lan with lan2 net as source would allow it to talk to vpn.

So seems from your trace your sending packets to pfsense -- but what does pfsense do with it then? Does it pass it through and the client is having an issue answering?? Do you have a HOST firewall on say 192.168.1.100 your tracing too?

Maybe your getting there just fine but the host your trying to talk to has firewall that drops your packets?

I would prob try and ping your lan2 box from vpn client and sniff on pfsense for that traffic on the lan2 interface - give me a minute and show you an example.

edit:
So see here sniffing on pfsense on my lan interface for icmp to or from my lan host 192.168.1.100, you can see the request from my vpn client IP 200.6 to my lan pc 1.100.. Then you see the replies.

If you do not see the request, then pfsense never sent them for some reason.. If you see the request but not the reply then host either never got them or he doesn't want to reply to something outside of his local network.. Or he sent the reply to the wrong place and not pfsense, etc.

edit2: You might have issues getting to pfsense gui, since your only pushing routes to lan2.. You would have to hit the pfsense gui on its lan2 interface IP.. 192.168.1.X in your case.

pingtrace.png_thumb

chrisso

John,

I can try the ICMP capture, but… When I do a ping request while on VPN2, it times out with no response. I have not turned on WBGui access on LAN2 as I want to aavoid any access to the PFSense box from that LAN2. The idea is to have some buddies get on and pentest the network. Ideally, I'd keep the PFSense box totally out of reach, so there's no cheating, etc.

I'm starting to think I need to install a previous version of PFSense and start over. I'll post up when I do the ICMP capture here in a few minutes.

Thanks,
-Chris

johnpoz

Well time out doesn't tell us where the packet stops.. You need to figure out if pfsense is actually sending out the ping on the lan2 network.. If it does then we need to look elsewhere to why you don't get an answer.

If you don't see the icmp request even go out the lan2 interface of pfsense – then we need to look to pfsense or before pfsense.

If you see answers on the sniff, then that tells us something in pfsense rules are prob blocking the return to your client.

As to blocking access - where is the rule for that? I do believe the webgui runs on all interfaces by default??

yeah I listen on 80.. just http and simple sockstat shows its listening on all addreresses

USER COMMAND PID FD PROTO LOCAL ADDRESS FOREIGN ADDRESS
root lighttpd 48617 9 tcp4 *:80 :

So if you don't have a firewall rule to block access from a specific interface to pfsense IP on port your listening on.. You would have access.

chrisso

Here's what I got when I connect to VPN2, and ping 192.168.1.100, and PFbox sniffing it with the same settings you had.

21:44:14.539641 IP 10.1.0.6 > 192.168.1.100: ICMP echo request, id 62491, seq 0, length 64
21:44:15.541259 IP 10.1.0.6 > 192.168.1.100: ICMP echo request, id 62491, seq 1, length 64
21:44:16.542836 IP 10.1.0.6 > 192.168.1.100: ICMP echo request, id 62491, seq 2, length 64
21:44:17.544443 IP 10.1.0.6 > 192.168.1.100: ICMP echo request, id 62491, seq 3, length 64
21:44:18.546023 IP 10.1.0.6 > 192.168.1.100: ICMP echo request, id 62491, seq 4, length 64
21:44:19.547636 IP 10.1.0.6 > 192.168.1.100: ICMP echo request, id 62491, seq 5, length 64
21:44:20.548851 IP 10.1.0.6 > 192.168.1.100: ICMP echo request, id 62491, seq 6, length 64
21:44:21.550831 IP 10.1.0.6 > 192.168.1.100: ICMP echo request, id 62491, seq 7, length 64
21:44:22.552424 IP 10.1.0.6 > 192.168.1.100: ICMP echo request, id 62491, seq 8, length 64

chrisso

Boy this is driving me nuts… I've wiped and reinstalled the PF box with 2.0 R3, as well as 2.0.3, and of course the latest 2.1, and set everything up from scratch, and it still wont work. SO I am missing something.

I'm just going to rest on it for a while, and hopefully something will click when I'm in the shower or something.

I'm reloading the backup from before I wiped everything, but I'm not against starting from scratch again if someone suggests it.

Still open for anything.

Thanks,
-Chrisso

PS. I'm starting to see something screwy with VMWare ESXI.... I didnt see it yesterday when I was testing pings etc., but I'm seeing it now. Soo... I need to fix that part first, as we arent seeing anything wrong with my setup here. I'll keep this posted when able.