Help with google cloud, two separate offices, and telecommuters
My current setup is as follows:
My business is using AWS as our cloud provider with two physical locations (Virginia, and Washington State), and a handful of telecommuters. I've got pfsense boxes at both physical locations (Virginia is running 2.2.3, and Washington is running a newer version that I don't know off the top of my head). I'm got static routes over IPsec from each office to the AWS VPC, and an OpenVPN connection between the offices. Lastly telecommuters connect over Ipsec/ikev2 using the built-in windows client.
This AWS and the two physical locations to access internal resources from all the "sites". However my telecommuters cannot access internal AWS resources, and I'm not sure if they can access resources from the other physical location (our Washington location is more of a satellite, so they don't really have any of their own resources).
We're in the progress of migrating to Google for our cloud provider and I'm using that opportunity to fix the telecommuter problem. Here's my requirements and goals:
Windows users use the built-in windows VPN client to telecommute. The two offices do not route their traffic inter-office traffic over Google. There's a bandwidth cost for this, and until we've quantified that cost we don't want to pay it. We may later change our minds here, but only once we know what the cost is. Telecommuters and people from both offices can access resources from both offices and GCP.
What I've tried, and the hurdles I've run into:
We setup interconnect at GCP with dynamic routing and a bgp session. My pfsense is configured for ipsec and openbgp. The ipsec session connects, BGP never does. If I let the pfsense bgp session run for a while it will stop saying it is in the "Connect" state and start saying it is in the "Active" state. Google's end never identifies that the BGP session has been established. On the pfsense my firewall seems to indicate that it is blocking traffic on my IPSec network between my neighbors (169.x.y.z) on port 179. All traffic is allowed on my IPSec network, and I've specifically added a rule for port 179 between the neighbors. In general OpenBGP seems like a great bit pile of crap on the pfSense. My very first config had malformed config out of the box, it doesn't seem to start or restart properly, and the logging and status seems sketchy (if not outright wrong) at best.
Is BGP even the right solution for this (routing from my telecommuters, to my office, to google and back again)?
Does anybody have experience with BGP in general on the pfSense? I'd be happy to share some more details about my configuration if it would be helpful.
Has anybody successfully set this up with GCP?