Google Cloud to pfSense VPN with BGP Dynamic Routing



  • So I'm trying to setup a vpn connection between our office (pfsense netgate router) and our google cloud vpc. The goal is to set it up as a dynamic routing (bgp) tunnel so we don't have to manually update our pfsense vpn every time we add a new subnet in Google. After googling and following some others guides on this, I've the tunnel established and the bgp routers are trading routing information between them. However I can't ping and servers on the google network. Doing a tracert to an IP in google from the local LAN, it doesn't even get past the pfsense it appears.

    I'm curious if anybody else has gone thru this process or could provide any input that might help. Attached are a bunch of screenshots of my pfsense setup. Thanks.

    Virtual IP Settings.png Phase2TunnelSettingsPart2.png Phase2TunnelSettingsPart1.png Phase1TunnelSettingsPart2.png Phase1TunnelSettingsPart1.png FRR Global Settings.png BGP Settings.png BGP Neighbor Settings.png BGP Status Part 4.png BGP Status Part 3.png BGP Status Part 2.png BGP Status Part 1.png



  • So after going back and looking thru others documentation (mainly here https://forum.netgate.com/topic/136509/routed-ipsec-vti-and-google-cloud). Looks like you still have to make additional P2 tunnels on the pfsense side for each network you want to connect (i.e. LAN to 10.0.128.0/21, etc.). Not knowing much about this to begin with, I was thinking this would be automatically handled, but I guess not. However it's still an improvement over having to manage static routes and tearing down the tunnel each time we add a new subnet in google.

    Other steps or screenshots I didn't show in the original post are:

    • Add pfsense firewall rule under ipsec to allow any source to any destination on any protocol (may look at restricting this in your case)
    • On Google's side of things, just follow their instructions (https://cloud.google.com/vpn/docs/how-to/creating-vpn-dynamic-routes)
    • Be sure to setup firewall rules on Google's side to allow ingress and egress from your pfsense networks


  • Hello,

    I have followed your screenshots, however I can't establish the connection. In IPSEC status I can see p1 and p2 connected fine. Do you have any idea why would that happen? Where should I check?

    IPv4 Unicast Summary:
    BGP router identifier 169.254.0.2, local AS number 65001 vrf-id 0
    BGP table version 2
    RIB entries 3, using 480 bytes of memory
    Peers 1, using 13 KiB of memory
    
    Neighbor        V         AS MsgRcvd MsgSent   TblVer  InQ OutQ  Up/Down State/PfxRcd
    169.254.0.1     4      65000       0       0        0    0    0    never       Active
    
    Total number of neighbors 1
    

    Thanks



  • You could "protect" a large network throught the tunnel, as an aggregate, and add smaller subnets, without having to manipulate vpn parameters
    Apart from that, bgp? Really??
    Ospf suits much better for the task. The overcomplexity if bgp serves no real purpose.



  • Since we're using Google's VPN service, it only appears to have BGP as an option and not OSPF. I'm sure we could setup our own pfsense router in Google and use OSPF that way, but that's not the case here. I'm no networking guru, so maybe I'm wrong. Agree with the large network option though, will play around with that.

    In response to ozbahceliler:
    Maybe check firewall rules. Under ipsec I created a rule to allow any port from any source to any destination. Also my screenshots are a little bit out of order. Here's everything I did in text format if you want to double-check against this. Hope it helps.

    Create New Tunnel in Google Cloud under vpn-us-central-1
    Name = vpn-us-central1-bgp-stl
    Description = blah blah blah
    Remote Peer IP = Public IP of pfsense
    IKE version = IKEv2
    Shared Secret = whatever password you want
    Routing Options = Dynamic (BGP)
    Cloud Router = grouter-01
    BGP Session = Create New
    Name = gcp-bgp-stl
    Peer ASN = ASN # of STL PFsense (64600)
    Cloud Router BGP IP = 169.254.10.1
    BGP Peer IP = BGP IP of STL Pfsense (169.254.10.2)

    PFSense Setup
    Firewall -> Virtual IPs -> Add
    Type = IP Alias
    Interface = Localhost
    Address = 169.254.10.2/30
    Description = blah blah blah

    VPN -> IPSec
    Add P1
    Key Exchange Version = IKEv2
    Internet Protocol = IPv4
    Interface = WAN
    Remote Gateway = Google VPN IP
    Description = gcp-central-vpn
    Authentication Method = Mutual PSK
    Pre-Shared Key = Same as shared secret used on google's side
    Encryption Algorithm = AES; 256 bits; SHA1; 24(2048(sub 56) bit)
    Lifetime = 36000
    Dead Peer Detection = Enabled
    Delay = 10
    Max failures = 5

    Add P2 underneath this P1 Tunnel
    Local Network = Network and 169.254.10.2/30
    Remote Network = Network and 169.254.10.1/30
    Description = BGP Tunnel
    Protocol = ESP
    Encryption Algorithms = AES128-GCM; 128 bits
    Hash Algorithms = SHA1
    PFS Key Group = 16 (4096 bit)
    Lifetime = 10800

    Copy this tunnel and add a second P2 tunnel for LAN to remote google network
    Need a P2 tunnel for each set of networks you want to connect with eachother
    Or possibly setup a P2 tunnel from LAN to a network that captures all networks on Google's side

    Firewall -> Rules -> IPsec
    Rule to allow any protocol from any source to any destination
    May need to restrict this later

    FRR Global Settings
    Enable FRR
    Master Password = QFx9IBnp
    Router ID = 169.254.10.2

    BGP
    Enable BGP
    Logging = Unchecked
    Local AS = 64600
    Router ID = 169.254.10.2
    Redistribute connected networks = Yes

    • this will advertise all local routes on pfsense

    Goto Neightbors under BGP
    Add
    Address = 169.254.10.1
    Description = gcp-central-neighbor
    Remote AS = 65005 (AS of Google router)
    Update Source = Virtual IP of 169.254.10.2

    Check Status to see if it shows routes from Google and neighbor status is ESTABLISHED.
    Zebra Status to check B routes, they came from BGP. C is for connected network, our local networks.
    Also check on Google side under VPC Network -> Routes -> Dynamic tab to see if your pfsense routes show up there.
    Update ingress and egress firewall rules on google side to allow access from pfsense networks



  • I recently had to deal with this myself and I made a guide about it. I hope it helps someone else!



  • Thanks Pablo. Good to have in case we ever move to an HA setup with Google VPN. For anyone else that reads this, my posts were for the Classic Google VPN setup (non HA).

    One note I wanted to add, in the BGP settings in my instructions above, don't change the setting for "Redistribute connected networks" to Yes. When set to Yes this advertised our WAN network to Google and caused issues with hitting public facing servers we had in Google. Since we only have a few networks locally, I just manually defined those along with the BGP network 169.254.10.0/30 in the fields below that setting.

    The other option may be to change the setting to Yes and somehow mark it to ignore the WAN network, but I haven't looked into that.