Possible IPSec routing issue
-
I'm a long time user of PFsense and the go-to-guy for a lot of installations. Usually whenever there's a problem, I know it's something I overlooked or misunderstood, because there is rarely any real issues with PFsense. However, this time I'm not so sure, and because I'm pretty sure I tested everything, I just want to use the help of those more experienced than me just in case I did overlook something. Humbleness first, right! :)
Trying to connect a customer with one of our networks for using Veeam over SMB to backup some data through IPSec. I can ping from us to customer server, but not the other way around. I can ping our PFsense from customer server but anything beyond that is considered internet (as far as I can tell).
Customer side:
PFsense 2.4.4 p3: 192.168.75.1/24
Firewall rule for IF IPSec: Any **** (for troubleshooting purposes)
IPSec tunnel is up and running
Veeam server (Win 2019): 192.168.75.254/24 (Win FW turned off)Our side:
PFsense 2.4.2 p1: 10.10.0.251/24 (not default gw)
Firewall rule for IF IPSec: Any **** (for troubleshooting purposes)
IPSec tunnel is up and running
Backup repository (Win 2016): 10.10.0.2/24 (Win FW turned off)
On this side Win server has permanent route: IP:192.168.75.0 Mask:255.255.255.0 GW:10.10.0.251 Metric:1Tracert from 192.168.75.254:
Tracing route to 10.10.0.251 over a maximum of 30 hops
1 <1 ms <1 ms <1 ms 192.168.75.1
2 26 ms 26 ms 27 ms 10.10.0.251Tracing route to 10.10.0.2 over a maximum of 30 hops
1 <1 ms <1 ms <1 ms 192.168.75.1
2 * * * Request timed out.
3 * * * Request timed out.
4 * * * Request timed out.Like I said, from the other side all is well:
Tracing route to 192.168.75.254 over a maximum of 30 hops
1 <1 ms <1 ms <1 ms 10.10.0.251
2 * * * Request timed out.
3 27 ms 26 ms 26 ms 192.168.75.254I'm at a loss. If I'm being stupid, just lay it on me.
-
Bump.
-
Here is routing table on customer Windows machine:
C:\Windows\system32>route print =========================================================================== Interface List 7...00 0c 29 d0 a5 79 ......Intel(R) 82574L Gigabit Network Connection 1...........................Software Loopback Interface 1 =========================================================================== IPv4 Route Table =========================================================================== Active Routes: Network Destination Netmask Gateway Interface Metric 0.0.0.0 0.0.0.0 192.168.75.1 192.168.75.254 281 10.10.0.0 255.255.255.0 10.10.0.251 192.168.75.254 26 127.0.0.0 255.0.0.0 On-link 127.0.0.1 331 127.0.0.1 255.255.255.255 On-link 127.0.0.1 331 127.255.255.255 255.255.255.255 On-link 127.0.0.1 331 192.168.75.0 255.255.255.0 On-link 192.168.75.254 281 192.168.75.254 255.255.255.255 On-link 192.168.75.254 281 192.168.75.255 255.255.255.255 On-link 192.168.75.254 281 224.0.0.0 240.0.0.0 On-link 127.0.0.1 331 224.0.0.0 240.0.0.0 On-link 192.168.75.254 281 255.255.255.255 255.255.255.255 On-link 127.0.0.1 331 255.255.255.255 255.255.255.255 On-link 192.168.75.254 281 =========================================================================== Persistent Routes: Network Address Netmask Gateway Address Metric 0.0.0.0 0.0.0.0 192.168.75.1 Default =========================================================================== IPv6 Route Table =========================================================================== Active Routes: If Metric Network Destination Gateway 1 331 ::1/128 On-link 7 281 fe80::/64 On-link 7 281 fe80::a86a:5e7e:5151:3a69/128 On-link 1 331 ff00::/8 On-link 7 281 ff00::/8 On-link =========================================================================== Persistent Routes: None
And routing table on customer PFsense:
IPv4 Routes Destination Gateway Flags Use Mtu Netif Expire default *wan-ip* UGS 124614232 1500 igb0 10.135.3.0/24 10.135.3.2 UGS 4845 1500 ovpns1 10.135.3.1 link#10 UHS 0 16384 lo0 10.135.3.2 link#10 UH 587101 1500 ovpns1 *our-gw-ip* *wan-ip* UGHS 670368 1500 igb0 127.0.0.1 link#6 UH 35967849 16384 lo0 192.168.1.0/24 link#2 U 21674 1500 igb1 192.168.1.1 link#2 UHS 0 16384 lo0 192.168.75.0/24 link#9 U 169108234 1500 bridge0 192.168.75.1 link#9 UHS 0 16384 lo0 *wan-nw* link#1 U 9251379 1500 igb0 *wan-gw* link#1 UHS 0 16384 lo0
And interface list of customer PFsense:
WAN up 100baseTX <full-duplex> *wan-ip* IF1 up 1000baseTX <full-duplex> 192.168.1.1 IF2 down autoselect n/a IF3 up 1000baseTX <full-duplex> n/a LAN up <bridge IF1-3> 192.168.75.1
-
@jimp or anyone?
-
@Phatsta said in Possible IPSec routing issue:
Tracing route to 10.10.0.2 over a maximum of 30 hops
Keep in mind that only 10.10.0.251 has a dedicated route to the .75.0 /24 network and most likely 10.10.0.2 has NOT !
Thats why the pings to .2 end up in nirwana.This is a general problem when you put static routes to end devices where they should NOT be !!
Those static routes belong to the router ! Routers should route in a network and not end devices !So its more intelligent to place the static route NOT on 10.10.0.251 here, but on the default gateway both .251 and .2 devices (and probably all in the 10.10.0.0 segment) have configured.
This is most likely a layer 3 routing switch or the default router on the 10.10.0.0 /24 segment.
Here, on this gateway you need to set the static route and this will cover than ALL 10.10.0.0 devices who have this gateway as the default gateway !
Thats the right way to set this up and will fix your problem instantly ! -
@lfoerster I would love to place the static routes in the routers, but it is not possible to put a static route without adding a gateway to route through, and it's not possible to add a gateway from a not-connected network in PFsense. This is why I've had no choice but to put the static routes at the end device as I see it?
I'll have another look at this and consider what you said. Thank you for your input.
-
@lfoerster said in Possible IPSec routing issue:
So its more intelligent to place the static route NOT on 10.10.0.251 here, but on the default gateway both .251 and .2 devices (and probably all in the 10.10.0.0 segment) have configured.
Of course you were correct. I put a static route in the default GW on our side and it started to work immediately.
I do have to admit it gets me a little confused though, since I've been using static routes on clients before. And, while it says it's a headache when administering multiple clients (which we don't), this article says it should work:
https://docs.netgate.com/pfsense/en/latest/book/ipsec/site-to-site.htmlAnyway, thanks a million!