IPSEC tunnel up but cant access anything across tunnel

[jhabers]

I have set up dozens of IPSEC tunnels with the PFSense platform so this really has me stumped.

I am running 2.1.5 on both endpoints

Endpoint A LAN network is 10.0.10.0
Endpoint B LAN network is 172.16.88.0

The tunnel syncs up and the IPSEC status says its good (green)

I have put rules on both ends on the IPsec tab to allow all traffic

Problem is that I cannot ping, access the firewall or anything on the other network at all….from either side

I am stumped as to where to start troubleshooting. I have already

rebooted both firewalls
deleted and recreated the VPN tunnels
deleted and recreated the rules

could the problem lay with the 172. network? I thought I read somewhere that some 172 addresses were being made public.

Can anyone give lend a hand? Any logs you would like to see?

Thanks a ton!

Jonathan

[Derelict]

Not sure about your problem, but you're fine on the 172.16 network.  172.16.0.0 - 172.31.255.255 (/12) are fine.

Since you're familiar with the process all I can suggest is post up some screen shots to get more eyes on them.  Must be looking past something.

[anomaly0617]

@jhabers:

I have set up dozens of IPSEC tunnels with the PFSense platform so this really has me stumped.

I am running 2.1.5 on both endpoints

Endpoint A LAN network is 10.0.10.0
Endpoint B LAN network is 172.16.88.0

The tunnel syncs up and the IPSEC status says its good (green)

I have put rules on both ends on the IPsec tab to allow all traffic

Problem is that I cannot ping, access the firewall or anything on the other network at all….from either side

I am stumped as to where to start troubleshooting. I have already

rebooted both firewalls
deleted and recreated the VPN tunnels
deleted and recreated the rules

could the problem lay with the 172. network? I thought I read somewhere that some 172 addresses were being made public.

Can anyone give lend a hand? Any logs you would like to see?

Thanks a ton!

Jonathan

These are suggestions from experience:

• This sounds dumb, but create a rule on each side under Firewall -> Rules -> WAN that looks like this:
  Proto=ICMP Echo Request
  Source=Any
  SPort=Any
  Dest=Any
  DPort=Any
  Gateway=Any
  Queue=None
  Sched=None/Blank
  Desc=WAN Echo

  Do this on both sides, and apply the config.

  Under Diagnostics -> Ping make sure you can ping the opposite host using Source Address = WAN. No pings = No traffic.

  If there's no traffic, go to Diagnostics -> Traceroute and try to figure out why.

  I can't tell you the number of times I get no ping response because some pud at AT&T/Time-Warner/Comcast/[Insert inept ISP here] gave me a router with firewall rules turned on, NATting turned on, or some other dumb issue.

  Once done and your VPN tunnel is working, you can delete or disable the echo request rule. :-)

• Make sure that you have Phase 2's set up on both ends, and that they complement each other. I've been known to pull up the local firewall on one side of my screen and the remote side of the firewall (through a remote service like RDP, TeamViewer, ScreenConnect, LogMeIn, etc.) on the other side of the screen, and make absolutely sure the two configurations match and the network complement each other. No joke, this has been the solution a half dozen times in the past, and each time, I facepalm myself and move on.

• Once the above is done… check your VPN logs under Status -> IPSec -> Logs. Yes, the output of raccoon is hard to decipher but between error messages in raccoon and copy/pasting them to Google, I've found the solution to many IPSec problems.

• Once the tunnel is alive and well on both ends (Green status indicators on both sides of the tunnel in Status -> IPSec) I like to start out with an "Allow Any Any" rule on both sides to verify connectivity. In
  Firewall -> Rules -> IPSec:
  Proto: IPv4
  Source: *
  SPort: *
  Destination: *
  DPort: *
  Gateway: *
  Queue: None
  Schedule: None/Blank
  Desc: Allow IPSec VPN traffic
  Make the rule the highest in the list.

  Once set and applied on both ends, ping away and see if you get anything.

  Once you get some traffic through, you can start limiting the rules down if you wish.

• Restart the raccoon service on both firewalls. Again with the "hands in the air, I have no idea why this works, but…" it's solved more IPSec problems than I can count.

• Restart each firewall, if this is an option.

• Turn on logging under the rules on both sides and look for Block entries.

Hope this helps!

[unsichtbarre]

I know you stated that, not only did you create firewall rule(s) for IPsec, but you deleted and re-created them.

From personal experience, I can tell you that the first time I set up an IPsec VPN, I created the firewall rule on the LAN tab and selected IPsec as the interface (I even knew better!)

I know this sounds stupid, but make sure that the rule you created is on the: Firewall > Rules > IPsec (tab).

I would suggest starting wide open: Action -Pass, Interface - IPsec, TCP/IP Version - IPv4, Protocol - any, Source - any, Destination - any

Beyond that, the devil is always in the details.

-j

[snm777]

@jhabers:

I have set up dozens of IPSEC tunnels with the PFSense platform so this really has me stumped.

I am running 2.1.5 on both endpoints

Endpoint A LAN network is 10.0.10.0
Endpoint B LAN network is 172.16.88.0

The tunnel syncs up and the IPSEC status says its good (green)

I have put rules on both ends on the IPsec tab to allow all traffic

Problem is that I cannot ping, access the firewall or anything on the other network at all….from either side

I am stumped as to where to start troubleshooting. I have already

rebooted both firewalls
deleted and recreated the VPN tunnels
deleted and recreated the rules

could the problem lay with the 172. network? I thought I read somewhere that some 172 addresses were being made public.

Can anyone give lend a hand? Any logs you would like to see?

Thanks a ton!

Jonathan

I have a very similar situation- NOTHING was crossing the tunnel until I did a few things.  I am running pfsense 2.1.5 on vmware at both ends.  My tunnel is up and green. 
I have an ubuntu server on the local LAN subent at each end, and if I follow these instructions to allow the pfsense boxes to ping each other's local LAN interface across the tunnel, I have the exact scenario you describe. https://doc.pfsense.org/index.php/Why_can%27t_I_query_SNMP,_use_syslog,_NTP,_or_other_services_initiated_by_the_firewall_itself_over_IPsec_VPN

Now, this is where it gets odd.  I have an ubuntu server at either end of my tunnel, each on the same network as the LAN interface of the respective firewall.  If I have routes created as described in the link above, the first packet I try to send from one ubuntu server to the other is seen by the firewall, but NOTHING after that, no matter what I try.  The reason becomes apparent when I do an arp -a on the ubuntu servers - they have an arp entry for the far end IP I just tried to reach, and it is incomplete.
Well, because it is incomplete, until it receives a VALID arp for that address, no traffic will leave the server!

I'm going to assume you have already configured something like an any/any rule under the ipsec tab as part of your troubleshooting - if not, do so.  The directions below are what fixed my situation.

Here is hat I had to do to fix mine:
1. Remove the routes  perscribed by https://doc.pfsense.org/index.php/Why_can%27t_I_query_SNMP,_use_syslog,_NTP,_or_other_services_initiated_by_the_firewall_itself_over_IPsec_VPN  - they allow the firewalls to communicate, but seem to break local subnet communication for other hosts, at least on 2.1.5. If this is working for anyone with the rules it, I'd like to compare configs to see what I'm doing wrong.

2. Restart racoon (ipsec) on both ends after the routes are gone.

3. Restart the tunnel.  I go to Status -> Ipsec and click the gear next to the "Status" field on one gateway to do this. I assume OP knows this from his description, but others might not.

4. Go to the ubuntu hosts that have the <incomplete>arp entries.  Fun fact about arp -d - it doesn't really delete an ARP entry, it sets it to imcomplete - soooo, since we already know that incomplete ARP entries won't let traffic leave the box, in order to get rid of the exiting arp entries you can do one of two things:
    a: Reboot both servers.  This is the simplest way I've found and it works great.
    b: Down and up the interface that faces the pfsense on both servers.  in my case, that's
    sudo ifdown eth0 && sudo ifup eth0

5. From the ubuntu server on one end, ping the other ubuntu server, and it should be working.

I have racoon running in debug mode, and it was telling me everything was fine with the tunnel.  It seems that with ubuntu (and maybe other linux hosts), the incomplete arp entry that was being added to the table with the routes were int eh pfsense simply borked any future traffic.  I can't speak to a Windows or OSx station at either end, I don't have those to test with, but I suspect that the incomplete arp entry would affect them too, to one degree or another.  Good luck, I hope this helps!</incomplete>

[jhabers]

Sorry for such a delay in response. We sidelined this project for a while and I just got back on it today. I did some more troubleshooting and determined that the 10.0.10.0 endpoint was the one with the problem. I figured this out by creating IPSEC tunnels from my office to each of 172.16.88.0 and 10.0.10.0. Both Tunnels established but I was not able to pass trafffic between my office and 10.0.10.0 netowrk in either direction.

I look over every setting on that firewall again and it all looked good. The only other thing that I though of was that they have DSL at this location and maybe the "DSL Modem" was blocking some tracffic. I logged into the modem even though it is bridged and saw a lot of this in the log:

2015/02/25 22:14:49 EST WRN | kernel          | logInboundBlocked:IN=br1 OUT=ppp0 PHYSIN=eth0 SRC=70.15.110.34 DST=96.10.24.214 LEN=120 TOS=0x00 PREC=0x00 TTL=63 ID=22826 PROTO=ESP SPI=0xc10fb172
2015/02/25 22:14:44 EST WRN | kernel          | logInboundBlocked:IN=br1 OUT=ppp0 PHYSIN=eth0 SRC=70.15.110.34 DST=96.10.24.214 LEN=120 TOS=0x00 PREC=0x00 TTL=63 ID=1003 PROTO=ESP SPI=0xc10fb172
2015/02/25 22:14:39 EST WRN | kernel          | logInboundBlocked:IN=br1 OUT=ppp0 PHYSIN=eth0 SRC=70.15.110.34 DST=96.10.24.214 LEN=120 TOS=0x00 PREC=0x00 TTL=63 ID=7795 PROTO=ESP SPI=0xc10fb172
2015/02/25 22:14:35 EST WRN | kernel          | logInboundBlocked:IN=br1 OUT=ppp0 PHYSIN=eth0 SRC=70.15.110.34 DST=96.10.24.214 LEN=120 TOS=0x00 PREC=0x00 TTL=63 ID=49042 PROTO=ESP SPI=0xc10fb172

Those IPs are the 2 endpoints to the Tunnels:

Unfortunately I am at home and when I try to get into the settings of the modem it asks for the "access code" which is printed on the bottom of the modem. I will have to wait until friday at earliest to get that code to see what is set on this thing that is blocking traffic. (Snow Storm here tonight)

I will make sure to follow up in  this thread with what I find out

Thanks!