Routed IPSec (VTI) and Google Cloud
I was able to successfully setup a IPSec Policy based VPN tunnel between my pfSense box and Google Cloud (GCP) today using Google's Cloud VPN:
Tonight I tried to see whether I could get Routed IPSec to work since pfSense now supports it in 2.4.4 and Google supports creating route based VPN's:
However, I'm not quite sure how to set this up properly to get it to work. Where I'm stuck in particular is how to create the transit network. It seems I can't just arbitrarily pick a unused local subnet, it needs to be the same and supported on both sides (local and destination). In fact, I tried this at first, e.g. picking a somewhat arbitrary subnet like 192.168.77.1/30 and 220.127.116.11 for the Phase 2 parameters of routed IPSec. However, I was never able to pass any traffic to the Google compute instances unless first setup a static route using the 192.168.77.2 Gateway to the subnet the compute instances reside on. I also had to setup routes on the Google side for the compute instances to communicate with machines on subnets behind my pfSense box. All the while 18.104.22.168 as a gateway was not pingable.
My guess, is all this work isn't necessary if things are setup properly :). However, all I have to work with on the Google side is the VPC network for the compute instances, e.g. let's say that network 10.1.0.0/20. Does anyone have any idea how to properly setup a transit network with GCP? Do I need to setup a separate VPC network or leverage the 10.1.0.0/20 network somehow? Or, is it not possible to setup at this time?
Thanks in advance for any insight you can provide, I really appreciate it.
Well, I tried this morning to get dynamic routing to work using the pfSense OpenBGPD package and by setting up a GCP cloud router attached to the VPN endpoint on the GCP side. This also was a success and traffic an pass without any issues. I just adapted steps from this guide to work with GCP instead of AWS:
It would still be great if I could still get routed IPSec (VTI) to work somehow, but otherwise I'll stick with dynamic routing for now since it's a better option than using a policy based VPN or just static routing.
If you are doing any new deployments, use the FRR package, not OpenBGPd.
Thanks @Derelict. I tried getting the same setup to work tonight using FRR and could never get a connection to the peer established. With OpenBGPD everything worked fine.
Google Cloud forces one to use 169.254.x.x. addresses to setup the BGP session. So, let's assume I have the following for the BGP route:
169.254.40.1 -- GCP
169.254.40.2 - pfSense
I added 169.254.40.1 as the neighbor IP along with the private ASN I chose under BGP Neighbors settings. Then I added 169.254.40.2 in the Router ID under Global Settings. Finally, I added my local subnets under Networks to Distribute in the BGP section along with the local private ASN I chose. Then I started up FRR and BGP.
I have working IPSec tunnel to GCP, but for some reason I am not able to pass traffic between 169.254.40.1 and 169.254.40.2 to create the necessary routing table entries.
I feel like I'm missing something obvious, but I can't quite put my finger on it. Are there any log files I can take a look and if so do you know where they are located?
Thanks in advance for your help.
Well, it was something simple: I forgot to configure the "Update Source" under BGP Neighbor to be the virtual IP of the local end of the tunnel (e.g. 169.254.40.2). Once that was setup everything worked like a charm!
Of course, had I had watched Jim's video until the end in the first place vs. stopping after the BGP configuration somewhere in the middle, I would have seen the AWS VPC configuration bonus slide around minute 66 and saved myself 1.5 hours of frustration last night :).
Thanks again for the info on FRR. It has a ton of options to configure, which makes it look daunting, but thankfully a basic setup doesn't require all that much configuration.
Excellent. Glad you got it going. FRR is definitely the path forward.
I just wanted to follow up on this thread quick and mention that I get did routed IPSec (VTI) to work with Google Cloud Platform using dynamic routing. For the P2 IP addresses, one just has to to use the link-local IP's provided for the BGP session (e.g. 169.254.40.1 and 169.254.40.2 in my example) and things will work fine and routes get exchanged between Google Cloud and pfSense. This article provided me with the hint: