Connects, but no comms between VPN and LAN2

chrisso

Sorry I am just now getting to this. I was running a fever yesterday, so when I finally got home, I threw back some NyQuil and called it a night.

Home today, and looking at the settings, I made a mistake in my description for VPN2 Tunnel. The tunnel address is actually "10.1.0.0/24", so please be aware of that when reading the following. I corrected the posts above to reflect what is correct.

I did verify that the OpenVPN rules allowed flow back, with the following:
Proto:IPv4, Source:, port:, Destination:, port:, Gateway:*, Queue:none

I also want to post the LAN2 to VPN2 rules that I created:
Proto:IPv4, Source: 10.1.0.0/24, Port:, Destination:192.168.1.0/24, Port:, Gateway:*, Queue:none.

The following rule is what I use to disable internet access to the LAN2 subnet. I just disable it to kill inet access.
LAN2, Proto:IPv4, Source:Lan2 net, Port:, Destination:, Port:, Gateway:, Queue:none.

Hopefully all of that makes sense. Again, I'm on PFSense 2.1-Release (amd64). Any and all suggestions are greatly appreciated.

Regards,
-Chrisso

phil.davis

On VPN2 server, what have you got in Local Network/s?
That is how the client connecting to VPN2 is told what network/s can be reached across the OpenVPN. You will need to put LAN2 subnet there. (and connect in from a place that is not already using the LAN2 subnet addresses.
Post server and client conf files and then we can see what routes the OpenVPN link is being told about…

chrisso

On the VPN2 server setup, I have the Tunnel setup with 10.1.0.0/24, and the return LAN setup as 192.168.1.0/24 (which is LAN2).

The Client Config file is as follows, but with a different DYNDNS address:

dev tun
persist-tun
persist-key
cipher AES-128-CBC
auth SHA1
tls-client
client
resolv-retry infinite
remote Random.DYNDNS.org 34447 udp
lport 0
verify-x509-name "rnsuser" name
auth-user-pass
pkcs12 pfSense-udp-34447-rnsuser.p12
tls-auth pfSense-udp-34447-rnsuser-tls.key 1
comp-lzo

And now, I hate to be ultra noob and ask this, but where can I get the server conf files? Googling shows their under /var/etc, and while there is a bunch of stuff there, I do not see any config files?

Thanks for the help Phil!

-Chrisso

johnpoz

they are in the openvpn folder under there

[2.1-RELEASE][root@pfsense.local.lan]/var/etc/openvpn(4): ls
server1.ca server1.sock server2.interface
server1.cert server1.tls-auth server2.key
server1.conf server1.tls-verify.php server2.sock
server1.crl-verify server2.ca server2.tls-auth
server1.interface server2.cert server2.tls-verify.php
server1.key server2.conf

See where I have server1 and server2.conf – I run one on tcp 443, and other on the default 1194 upd.. Issue is 1194 udp is not always a open port depending on where your located.. While if internet is there, 443 is normally always open.

chrisso

Thanks John.

Here is the server2.conf file. If something else is needed, please let me know. I changed the static IP info to "PublicIP". Rather not paste that out here.

dev ovpns2
dev-type tun
tun-ipv6
dev-node /dev/tun2
writepid /var/run/openvpn_server2.pid
#user nobody
#group nobody
script-security 3
daemon
keepalive 10 60
ping-timer-rem
persist-tun
persist-key
proto udp
cipher AES-128-CBC
up /usr/local/sbin/ovpn-linkup
down /usr/local/sbin/ovpn-linkdown
client-connect /usr/local/sbin/openvpn.attributes.sh
client-disconnect /usr/local/sbin/openvpn.attributes.sh
local PUBLICIP
tls-server
server 10.1.0.0 255.255.255.0
client-config-dir /var/etc/openvpn-csc
username-as-common-name
auth-user-pass-verify /var/etc/openvpn/server2.php via-env
tls-verify /var/etc/openvpn/server2.tls-verify.php
lport 34447
management /var/etc/openvpn/server2.sock unix
max-clients 20
push "route 192.168.1.0 255.255.255.0"
client-to-client
duplicate-cn
ca /var/etc/openvpn/server2.ca 
cert /var/etc/openvpn/server2.cert 
key /var/etc/openvpn/server2.key 
dh /etc/dh-parameters.1024
tls-auth /var/etc/openvpn/server2.tls-auth 0
comp-lzo
persist-remote-ip
float

Regards,
-Chrisso

johnpoz

Keep in mind if this rule is not enabled PASS

LAN2, Proto:IPv4, Source:Lan2 net, Port:, Destination:, Port:, Gateway:, Queue:none.

You would not be able to talk back out the open vpn connection. If you want to block internet but allow vpn you would need to have a rule that allows lan2 to talk to your vpn2 network.

Also your float tab is empty? If you have rules in there they can block traffic on any interface.

Can you post your route table so we can see your vpn networks. Example here are mine.

Don't mind that 172.15 network - something for another thread, need to clean that up.

routetable1.png_thumb

chrisso

Thanks John. I Went ahead and re-enabled that rule. Eventually, I'll have to write out something custom to exclude any outbound traffic from LAN2 to the internet. With this being a "Vulnerable Network", I'd like to avoid any and all "Official Notices", lol.

-Chrisso

johnpoz

well shows your local networks and your openvpn networks - so routing should work.

you say you have your firewall rules that allow the traffic.. Have you tried restarting the vpn server.. I have had now and then, not very often. But issue where I would get connected and just not able to talk to my lans.. restart the vpn server and bing bang zoom everything working.

btw you have validated your client is getting the route to the lan2 network.. So for example here is my work box - currently connected to my home network.

C:>ipconfig
Ethernet adapter vpn:

Connection-specific DNS Suffix . : local.lan
IPv4 Address. . . . . . . . . . . : 10.0.200.6
Subnet Mask . . . . . . . . . . . : 255.255.255.252
Default Gateway . . . . . . . . . :

here is traceroute

C:>tracert -d 192.168.1.100

Tracing route to 192.168.1.100 over a maximum of 30 hops

1 212 ms 181 ms 149 ms 10.0.200.1
2 162 ms 202 ms 263 ms 192.168.1.100

And attached you see the routes on the box that tell it where to go for my 192.168.1 and .2 networks.

BTW the times really suck because I have to bounce off a proxy in Jax, FL to get to the internet from work here in Chicago.. So all the way from chicago to FL, then to Chicago again for my pfsense box at home - then back again - doesn't make for really good latency ;) 20 Miles apart and seeing response times like I was going to China ;)

routesvpn.png_thumb

chrisso

I would like to say that I've done exactly that. But maybe I've rebooted the entire PFSense box. (Which I hope would do the same thing?)

The kid is home with the flu today, so he's streaming "Thomas and Friends". I'll restart the VPN2 server service, and give that a go. Once he runs somewhere else or goes to sleep, I'll reboot the box again.

If I didnt do that, and it starts working, then…..... I'm drinking tonight.

Will post an update here soon.

THanks again John,
-Chris

johnpoz

So question for you - can you actually access your pfsense web gui interface from the vpn2?

Also I did a bit of edits on my last post, check your routes on your client, etc.

chrisso

No access to the PFS-webgui from VPN2. Not really hitting anything when connected to VPN2. Just get the tunnel, and thats it.

Screen shot of "ifconfig":

Screenshot of the "traceroute", and then CTRL+C cause nothing was happening for a long long long time.:

-Chrisso

chrisso

For the record. Unless I'm thinking or looking at this incorrectly, but here is a screenshot of what I believe to be proper Rules.

-Chrisso

johnpoz

Well your rules should work - but rules are only ingress from outside pfsense – think of arrows pointing to pfsense from outside..

So inbound into openvpn rule would never have a source of your lan networks. So those 2 rules above with source of your lans don't come into play.. And then you have duplicate rules.

Then on your lan2 interface that source of vpnnetwork does not come into play..

So you have this

pc ----> vpn pfsense lan ---> pc

or

pc ---> lan pfsense vpn ---> pc

Only the rules were the arrows point to pfsense matter. You don't do any outbound rules on pfsense.

So you need vpn rules that allow vpn clients to talk to dest you want to allow. so your any any rule allow that.

then you lan rule inbound to lan with lan2 net as source would allow it to talk to vpn.

So seems from your trace your sending packets to pfsense -- but what does pfsense do with it then? Does it pass it through and the client is having an issue answering?? Do you have a HOST firewall on say 192.168.1.100 your tracing too?

Maybe your getting there just fine but the host your trying to talk to has firewall that drops your packets?

I would prob try and ping your lan2 box from vpn client and sniff on pfsense for that traffic on the lan2 interface - give me a minute and show you an example.

edit:
So see here sniffing on pfsense on my lan interface for icmp to or from my lan host 192.168.1.100, you can see the request from my vpn client IP 200.6 to my lan pc 1.100.. Then you see the replies.

If you do not see the request, then pfsense never sent them for some reason.. If you see the request but not the reply then host either never got them or he doesn't want to reply to something outside of his local network.. Or he sent the reply to the wrong place and not pfsense, etc.

edit2: You might have issues getting to pfsense gui, since your only pushing routes to lan2.. You would have to hit the pfsense gui on its lan2 interface IP.. 192.168.1.X in your case.

pingtrace.png_thumb

chrisso

John,

I can try the ICMP capture, but… When I do a ping request while on VPN2, it times out with no response. I have not turned on WBGui access on LAN2 as I want to aavoid any access to the PFSense box from that LAN2. The idea is to have some buddies get on and pentest the network. Ideally, I'd keep the PFSense box totally out of reach, so there's no cheating, etc.

I'm starting to think I need to install a previous version of PFSense and start over. I'll post up when I do the ICMP capture here in a few minutes.

Thanks,
-Chris

johnpoz

Well time out doesn't tell us where the packet stops.. You need to figure out if pfsense is actually sending out the ping on the lan2 network.. If it does then we need to look elsewhere to why you don't get an answer.

If you don't see the icmp request even go out the lan2 interface of pfsense – then we need to look to pfsense or before pfsense.

If you see answers on the sniff, then that tells us something in pfsense rules are prob blocking the return to your client.

As to blocking access - where is the rule for that? I do believe the webgui runs on all interfaces by default??

yeah I listen on 80.. just http and simple sockstat shows its listening on all addreresses

USER COMMAND PID FD PROTO LOCAL ADDRESS FOREIGN ADDRESS
root lighttpd 48617 9 tcp4 *:80 :

So if you don't have a firewall rule to block access from a specific interface to pfsense IP on port your listening on.. You would have access.

chrisso

Here's what I got when I connect to VPN2, and ping 192.168.1.100, and PFbox sniffing it with the same settings you had.

21:44:14.539641 IP 10.1.0.6 > 192.168.1.100: ICMP echo request, id 62491, seq 0, length 64
21:44:15.541259 IP 10.1.0.6 > 192.168.1.100: ICMP echo request, id 62491, seq 1, length 64
21:44:16.542836 IP 10.1.0.6 > 192.168.1.100: ICMP echo request, id 62491, seq 2, length 64
21:44:17.544443 IP 10.1.0.6 > 192.168.1.100: ICMP echo request, id 62491, seq 3, length 64
21:44:18.546023 IP 10.1.0.6 > 192.168.1.100: ICMP echo request, id 62491, seq 4, length 64
21:44:19.547636 IP 10.1.0.6 > 192.168.1.100: ICMP echo request, id 62491, seq 5, length 64
21:44:20.548851 IP 10.1.0.6 > 192.168.1.100: ICMP echo request, id 62491, seq 6, length 64
21:44:21.550831 IP 10.1.0.6 > 192.168.1.100: ICMP echo request, id 62491, seq 7, length 64
21:44:22.552424 IP 10.1.0.6 > 192.168.1.100: ICMP echo request, id 62491, seq 8, length 64

chrisso

Boy this is driving me nuts… I've wiped and reinstalled the PF box with 2.0 R3, as well as 2.0.3, and of course the latest 2.1, and set everything up from scratch, and it still wont work. SO I am missing something.

I'm just going to rest on it for a while, and hopefully something will click when I'm in the shower or something.

I'm reloading the backup from before I wiped everything, but I'm not against starting from scratch again if someone suggests it.

Still open for anything.

Thanks,
-Chrisso

PS. I'm starting to see something screwy with VMWare ESXI.... I didnt see it yesterday when I was testing pings etc., but I'm seeing it now. Soo... I need to fix that part first, as we arent seeing anything wrong with my setup here. I'll keep this posted when able.

johnpoz

So you see the ping request leave the lan2 interface of pfsense - but not get a response.. So how is pfsense at fault for your timeout? Pfsense never got a reply, so how could it send it down the vpn tunnel to you.

Your issue is elsewhere.

For one a host firewall that your trying to ping blocking the traffic. So your host never responds.

chrisso

Basically after the wipe and reinstalls, I changed my focus elsewhere. I tested this all before I even considered PFSense as a problem. But later last night, I found problems with the networking side of my ESXI box, and the hosted machines on it.

Where a premade VM (DeIce 1.100) is supposed to be on address 192.168.1.100, it in fact was not. It was a few days ago, but as of last night, it was actually pulling .1.20. When I found that out (through DHCP leases) and tried to ping it, I still couldnt, however I could ping other VMs on that 192.168.1.0/24 network (which I couldnt before).

So long story short, I have a problem on my ESXI host within networking.

I am very sorry, and I feel like a complete jackass….. Honestly, this was tested before hand when everything was working right. Somewhere in that tiny bit of time, I broke something on the ESXI box, and expected it to come back like normal. But it did not, and I did not know that.

John, thank you very much for your time, and I truly am sorry for wasting so much of it as I did, however I did learn some additional things while troubleshooting all of this.

Regards,
-Chrisso

johnpoz

troubleshooting a problem is never a waste of time, even if we spent time looking to what the problem was not.. Once we ruled those out as not the problem you get to what the goal was - find the source of the problem.

And you get the added bonus which is always good!
"I did learn some additional things while troubleshooting all of this. "

Let us know how it works out - and I run esxi 5.5 and my pfsense is VM.. With multiple segments on my esxi, etc. So if you need any help in that area even though its not pfsense directly let me know - glad to help.