SG-3100 - routing all internet access over IPSEC tunnel
-
I have configured a working IPSEC v2 tunnel between a remote location and my location. The far end user can see servers and PCs on my local network as desired.
However I would like the far end to access all external internet connections through the local gateway here, rather than the SG-3100 gateway at the far end, is this possible?
Thanks
D.
-
Yes. Screenshots here are outdated but the settings are the same:
https://docs.netgate.com/pfsense/en/latest/vpn/ipsec/routing-internet-traffic-through-a-site-to-site-ipsec-vpn.htmlSteve
-
Aha that's great, thanks. Just need to change the one Phase 2 setting to 0.0.0.0/0 on the existing tunnel it appears.
Since we have a Barracuda F-series on the near end I'll need to translate those Netgate DEST NAT settings over, I imagine that should be do-able.
Cheers.
D.
-
@dhb said in SG-3100 - routing all internet access over IPSEC tunnel:
Cheers.
I have been trying to achieve this in the routed ipsec, but it isn't working, any ideas?
-
Where is it failing? Where does traffic reach in a packet capture?
There is an issue with VTI IPSec and NAT which you might be hitting here.
Steve
-
Not really sure how to do a packet capture.
What is the issue with VTI IPSEC and NAT?
-
As I understand it pf cannot apply NAT to a VTI interface. That may apply here. I have a feeling we tried something similar and it failed.
Start a continuous ping at the remote site to some external IP. Run a packet capture on the IPSec interface at the remote location to make sure that is leaving over the VPN. Run a packet capture at the SG-3100 end on the IPSec interface to make sure it is arriving then run it on the WAN to make sure it is leaving there and has been NAT'd.
Steve
-
@ccb056 said in SG-3100 - routing all internet access over IPSEC tunnel:
Not really sure how to do a packet capture.
What is the issue with VTI IPSEC and NAT?
Outbound NAT on a VTI interface does not work.
-
Is there a bug in redmine for this?
-
Has to be fixed upstream in freebsd.
-
If you're configured as expected though the outbound NAT is on the SG-3100 WAN not actually on the VTI interface. But I have a feeling this also hits it somehow. Try it and see.
You can run a packet capture from the webgui in Diag > Packet Capture. Filter by protocol ICMP and whatever the external IP is you're pinging.
Steve
-
This post is deleted!
-
Is there a FreeBSD bug report?
I can't seem to find one:
https://bugs.freebsd.org/bugzilla/buglist.cgi?order=Importance&query_format=advanced&short_desc=ipsec nat&short_desc_type=allwordssubstrI did find this pfsense bug:
https://redmine.pfsense.org/issues/9230And this opnsense thread:
https://github.com/opnsense/core/issues/440
-
That pfSense bug report is probably the closest thing. That is NATing over a VTI interface.
The Opnsense bug is refering to policy based IPSec which also has NAT limitations but is when you're hitting here specifically.
Were you able to run that test and get the pcaps?
Steve
-
I worked with a guy doing exactly this. He first tried this:
LAN <-> VTI (OB NAT) <-> VTI <-> WAN (OB NAT) <-> Internet
He changed it to this because of this limitation on NAT on the VTI interface:
LAN <-> VTI <-> VTI <-> WAN (OB NAT) <-> Internet
Setting the Outbound NAT on the WAN to NAT the LAN source addresses.
All works fine. I think he said he's getting 500Mbps or more over it.
You just have to make sure the WAN side has a gateway and a route back to the LAN network over the VTI.
I believe he used policy routing to send the internet traffic from certain local hosts over the VTI, not a 0.0.0.0/0 route. Both methods should work, but if I was going to try the route method I would probably adopt the OpenVPN strategy and add these instead of 0.0.0.0/0: 0.0.0.0/1 128.0.0.0/1 so your existing default route stays unmolested.
-
Ah nice!
-
Both my laptop and my phone can ping google.com through the ipsec tunnel, packet captures below
Here is the traceroute from my laptop without going through the tunnel:
Here is the traceroute from my laptop going through the tunnel, but using local dns:
And this is what it looks like going through the tunnel and using remote dns:
Here are some photos of the configs on both routers:
Note, the browser on the laptop seems to load the google.com webpage, but the phone cannot browse to any webpage
What do I need to change to get this working properly?
-
None of that really gives us anything to go on.
How are you selecting traffic to go over the tunnel? Policy routing or the routing table? How is that set up?
How are the two devices configured for both IP routing and DNS?
Not sure why you're showing us traceroutes and mentioning DNS. Use DNS tools to troubleshoot DNS and traceroute to troubleshoot routing.
It looks like you are routing through the tunnel and out the charter WAN so everything including outbound NAT there is working.
-
@ccb056 No idea why you're making it so complicated. Just policy route everything from hosts_tunneled out the VTI interface, get that working, then worry about whatever it is you're trying to do with DNS.
-
I am selecting traffic to go over the tunnel using this LAN rule on the local router:
Here are the routing tables on the local and remote routers:
Not everything is working.
The laptop is very spotty when accessing webpages, it can access google right now, but not yahoo, msn, etc.The phone cannot access anything.
-
So you have three sites connected all connected together there? A quick diagram would be very useful.
Could there be some route asymmetry here?
Do you see blocked TCP traffic in the firewall logs at either site when you try to open a failing page?There is no reply-to added to traffic from VTI interfaces so the system routing is always used.
Steve
-
Here is a diagram of the topology:
And here is a youtube video showing what I'm experiencing:
Youtube Video
-
Ok, do you see any blocked TCP traffic in the logs at either site though?
The video stream is probably UDP and pings which you said were working are ICMP so...
Steve
-
Here's a packet capture log from the interfaces on both the local and remote routers during an sftp transfer (tcp/22)
-
Hmm, so some of those seem OK, the local VTI cap for example.
How exactly were those taken? Not all at the same time I assume? And did the transfer fail every time?
Steve
-
One large download to the laptop was started over sftp.
Then the packet captures were taken sequentially, first on the local router, and then on the remote router, during the single file transfer.
After the packet captures were taken, the transfer was manually stopped.
What I found interesting was there was traffic on the wan side of the local router that appeared to be going directly to the sftp server. I expected this to all be inside the IPSEC tunnel.
-
What I found interesting was there was traffic on the wan side of the local router that appeared to be going directly to the sftp server. I expected this to all be inside the IPSEC tunnel.
If that is the case (I have not looked at the captures) you are not routing/policy routing the traffic into the IPsec correctly.
-
@Derelict I'm having this very issue with two SG-5100s using routed IPSEC. Would you be able to explain your solution a little bit more so that I can understand what I need to do on my end.
// I worked with a guy doing exactly this. He first tried this:
LAN <-> VTI (OB NAT) <-> VTI <-> WAN (OB NAT) <-> Internet
He changed it to this because of this limitation on NAT on the VTI interface:
LAN <-> VTI <-> VTI <-> WAN (OB NAT) <-> Internet
Setting the Outbound NAT on the WAN to NAT the LAN source addresses.
All works fine. I think he said he's getting 500Mbps or more over it.
You just have to make sure the WAN side has a gateway and a route back to the LAN network over the VTI.//I'm self taught and could use a slight hand holding on this one. Was super disappointed to encounter the issue using the routed IPSEC option but your info suggests current limitations can be overcome until the fix is made in FreeBSD. Appreciate any help you might be willing to provide.
-
What exactly do you have configured now? What works? What doesn't work?
The VTI interfaces have some limitations compared to other interface types, there is no reply-to feature due to where firewall rules are applied. Some NAT also cannot work because of that.
As long as you avoid those it should work.Steve
-
@stephenw10 appreciate the response. This thread (see link below) sums up my issues and Derelict later responded referring me back to this thread. I’m just trying to understand the solution he provides. Just looking for some explanation of the steps he outlines.
Original thread I started:
-
Does this help?
ETA: better resolution and firewall rule box.
-
@Derelict Most certainly. That's exactly how I have mine set up but am very glad to know this is how people smarter than I would do things. The biggest issue I suffer from is the 2xx ms latency due to distance between endpoints. Really do appreciate the extra help on this. Time is valuable and you let me have a little of yours.
-
@ngoehring123 Yeah. Can't help you with the latency. Glad it helped.
