How to setup a second local network for an IPSec connection?

CodeNinja · Jul 6, 2020, 8:54 AM

We have currently 2 IPSec connections to customer sites (will be many more in nearby future).

At our site we have 2 networks:
10.128.10.0/24 (workstation network) and 10.128.11.0/24 (server network) which are "managed" by pfSense
at the customers site we use:
10.130.x.0/24 (Ubiquity Edgerouter X)

In first case we thought that we only need a connection between our workstation network (10.128.10.x/24) and the customers network for maintenance. For this i created a 1 Phase 1 and 1 Phase 2 for each customer. This is up and running and works fine.

Since last Friday one of our servers needs also access to the customers network to push some date into their server(s) every minute.

I have thought to simply add a second Phase 2 to a IPSec connection in pfSense and also create a Phase 2 with the same configurations at the Ubiquiti Edgerouter X and thought it will work but nope, it doesn't.

Only 1 (the first, 10.128.10.0/24) Phase 2 connects. The other one is simply not displayed in Status -> ipSec at pfSense. I also cant ping between devices in the 10.128.11.0/24 and 10.130.x.0/24 network though I can between devices in the 10.128.10.0/24 and 10.130.x.0/24 network.

pfSense config: http://snippi.com/s/0xrzfty (Could not post it here because of "Askimet")

I have no idea how i can solve my issue. Maybe adding a second Phase 2 is not the solution at all and maybe i need to add a route somewhere. Unfortunately i was not able to find something that helped me out on Google, maybe someone here can help me out?

Thanks in advance!

JeGr · Jul 6, 2020, 9:53 AM

@CodeNinja said in How to setup a second local network for an IPSec connection?:

Only 1 (the first, 10.128.10.0/24) Phase 2 connects. The other one is simply not displayed in Status -> ipSec at pfSense

Are you sure? If you configure 2 phase 2 entries for 2 local networks to the same remote network, pfsense can "merge" them and show just "one" phase 2 with 2 networks under the local column.

Just a random VPN tunnel we have at a customer site:

login-to-view

as you can see, it seems it's only 1 phase 2 entry, but it actually has 4 P2 entries merged into one status as there are 4 P2 with different remote subnets defined to the same local subnet.

But if it is indeed the case, that the second P2 isn't showing up/starting, check your logs or the edgerouter's and stop and start (stop/start is different then restart) the IPsec service on pfsense to check if both P2s are visible then.

You configured both P2s on the edgerouter, too? Does it have any log to show?

Also if the edgerouter has a problem with more than one P2 (pfsense has none I know of), you can simply delete the second P2 entry and just modifiy your first one to 10.128.10.0/23 which will include both 10.0/24 and 11.0/24 - et voila - you have achieved the same as with two P2 entries

Greets,
\jens

CodeNinja · Jul 6, 2020, 1:15 PM

@JeGr
Thanks for your reply. I'm sure that i only see 1 network:

login-to-view

I configured 2 Phase 2 networks for the concerning connection. They are identical to each other except for the "Local subnet", this is different.

I understand the "supernet" solution but i prefer to configure it the "clean" way...
Is there something i need to be aware of during configuring more than 1 Phase 2 for one connection?

UPDATE:
For some reason, the second connection is sometimes initiated (a random one of the 3 i configured for that connection) but sometimes it isn't. Its simply unreliable, how can i check the cause ?

JeGr · Jul 6, 2020, 4:42 PM

@CodeNinja said in How to setup a second local network for an IPSec connection?:

I understand the "supernet" solution but i prefer to configure it the "clean" way...

That actually IS the clean way as it simplifies the traffic selection by only introducing a single route instead of two (which could be combined) ;)

@CodeNinja said in How to setup a second local network for an IPSec connection?:

For some reason, the second connection is sometimes initiated (a random one of the 3 i configured for that connection) but sometimes it isn't. Its simply unreliable, how can i check the cause ?

Perhaps no traffic hits it and it doesn't get triggered to open? Perhaps the other side (edgerouter) has a problem and won't respond but initiate it instead? Only thing you could do would be checking the logs and perhaps setting a host to ping in your P2 settings so it would be forced up to ping that host.

CodeNinja · Jul 7, 2020, 8:48 AM

@JeGr said in How to setup a second local network for an IPSec connection?:

Perhaps no traffic hits it and it doesn't get triggered to open? Perhaps the other side (edgerouter) has a problem and won't respond but initiate it instead? Only thing you could do would be checking the logs and perhaps setting a host to ping in your P2 settings so it would be forced up to ping that host.

I noticed that that is exactly thats is what happens. When i ping from the EdgeRouter to the pfSense network, the connection is initiated within less than a minute. The other way round the connection is not initiated.

Actually its intended that the Edgerouter sets up the connection to our network and not visa versa so i don't need to open ports at the customer side (even if this is currently not working like that for some reason, port forwarding is still required).

Is it an idea/good practice (or maybe thats what you mean) to let the Edgerouter periodically ping the networks it should connect to?

JeGr · Jul 7, 2020, 9:31 AM

@CodeNinja said in How to setup a second local network for an IPSec connection?:

Is it an idea/good practice (or maybe thats what you mean) to let the Edgerouter periodically ping the networks it should connect to?

It's a possibility. It certainly isn't uncommon to keep a P2 open to let it ping a host on the other side of the network. Otherweise it wouldn't be configurable in pfsense, too ;)

The only thing that throws me off a little is, that it seems that pfsense can't initiate the connection/tunnels but the edgerouter does. That reminds me of setups, where the other side throws around multiple connection settings (in terms of cipher/hash/etc.) whereas pfsense only has one set of ciphers, hashes and credentials configured so if one side initiates the connection it always works, if the other side initiates it fails a few times until the correct set is established. But that's because I wrote one should check the logs if there's a phase 2 error or if the pfsense side simply has no reason to open the channel (without traffic). So if it works if one side pings through the tunnel - I'd do so :)

CodeNinja · Jul 7, 2020, 10:17 AM

@JeGr

Thanks for the use full info and replies, i will struggle around a bit with it this and next week and will update my post than with the solution i implemented.

CodeNinja · Jul 8, 2020, 8:10 AM

Pinging a specific pfSense network from the customer network (EdgeRouter) mostly initiates a tunnel between the Edgerouter and the concerning network, sometimes it doesn't for some reason.

I also noticed this morning that one of the connection had 8 tunnels where i expected only 4. 5 are duplicates from eachother and 1 is missing..

I expected a tunnel between 10.130.2.0/24 and:

10.128.10.0/24 <- duplicated, exist only once before i left the office yesterday
10.128.11.0/24 <- is missing (worked yesterday before i left the office)
10.128.20.0/24
10.128.30.0/24

login-to-view

How is this possible?

I'm also curious if its preferred/best practice to use "supernet" or this "multiple tunnel" construction like i currently do.

JeGr · Jul 8, 2020, 9:22 AM

@CodeNinja said in How to setup a second local network for an IPSec connection?:

I'm also curious if its preferred/best practice to use "supernet" or this "multiple tunnel" construction like i currently do.

In many bigger scenarios, I see "supernets" or bigger CIDR masks to simplify tunnel deployments. Especially in centralized structures with one or two "main" sites with big uplinks and many small/branch offices network design often tends to do sth. along these lines:

Roll out big network structure on main(1) -> e.g. multiple 172.19.x.0/24 networks for security segmentation
Dial Up / RAS VPN uses IP ranges either from an upper 172.19.x segment or another IP range altogether (e.g. 192.168.vvv.0/24)
Branch offices use separate range -> e.g. 10.10.bbb.0/24 for office 1, 10.20.bbb.0/24 for office 2 (or 10.11.bbb.0 if you have a whole lot of branch offices).

With that setup, you can easily do tunnels from "main" to "site a" with <172.19.0.0/16> <-> <10.10.0.0/16> and have no problem whatsoever to grow in either space. If you have the need for new networks on site or on in the main location - just add another VLAN with /24 and as your tunnel is set up with /16 it already includes the new networks.

So yeah, pretty common to use CIDR ranges bigger than your local network to add some "space to grow" lateron.

I also noticed this morning that one of the connection had 8 tunnels where i expected only 4. 5 are duplicates from eachother and 1 is missing..

That seems strange. A duplicate can (and will) happen at times, when rekeying gets near and the lifetime is about to expire. Then it's pretty normal to sometimes see every phase with a second entry as the old one gets "disabled" (but not disconnected) and the new one takes over so the rekey/lifetime turnaround goes smooth. You then see new traffic accumulate on the newer P2 and the old one won't get any more and after expiry should vanish a few seconds/minutes later. But having the same phase 5 times is strange. And some were brought up only seconds after another. Weird. I'd disconnect the whole bunch and reestablish the tunnel and check if that happens again. Perhaps something with the edgerouter on the other site? Maybe setting the split option in P1 of the connection could help if pfsense tries to group the connection but the edgerouter doesn't support it (fully) - but that's just a guess.