Azure IPsec & BGP Woes
-
Hi all,
I've successfully set up a dynamic routing IPsec tunnel to Microsoft Azure, and life's been good. However, I'd like to enable BGP at this point, as Microsoft recently enabled BGP on VPN connections, and I'm having serious issues getting it working properly. Could use some help from the community! Please note: I am not seeing the bug where the IPsec tunnel drops when using BGP. My tunnel is 100% stable - verified from Azure and PfSense.
It's a simple setup, but for whatever reason, traffic won't flow across the tunnel when BGP is set up. When I run a tcpdump in PfSense, I can see the bgp trafflic flowing between the BGP peer addresses just fine, and Azure's backend fabric sees the BGP routes as expected. When I ping a running Linux VM in Azure (10.1.0.200, for example) from a local machine (10.0.1.101, in this case - my laptop), the request times out. Traceroutes do the same after the first hop at 10.0.1.1 (router).
Can anyone tell me what I'm doing wrong here?
Details:
Azure BGP Peer Address: 10.1.254.254
Azure ASN: 65515
Azure Subnet01: 10.1.0.0/16
Azure Subnet02: 10.2.0.0/16
Local PfSense box: 10.0.1.1
Local BGP Peer Address: 10.0.1.1
Local ASN: 65501
Local LAN Subnet: 10.0.1.0/24
OpenVPN subnet - running on PfSense box (not relevant here, but it shows up here and there so FYI): 10.8.0.0/24OpenBGP Raw Data: http://imgur.com/AfvMrgN
Relevant Firewall Rules: Floating rule to pass LAN/IPsec in any direction and any protocol
IPsec rule to pass any IPv4 traffic on any source/dest/port etc.Other Relevent Info: tcpdump logs taken on pfsense IPsec interface while pinging across tunnel from a host on the local subnet shows bgp traffic only, no ICMP at all: http://imgur.com/dRtX3RF
OpenBGP Statuses
See Imgur links below. Each set of three blocked out IPs are:- Azure Gateway Public IP (/32)
- ISP-given local public IPs (/23)
- ISP-given local public IP (/32)
http://imgur.com/L9ZeNjP
http://imgur.com/KCHSVLh
http://imgur.com/UiL40Nq
http://imgur.com/1znh0Sc
Routes in Pfsense.
There doesn't appear to be any routes set for the Azure subnets (10.1.0.0/16 and 10.2.0.0/16). In this image, the blacked out gateways are both the ISP's gateway, then the destinations are defined the same as the above imgur links - azure public IP /32, local public IP /23, and local IP /32:
http://imgur.com/LxufjdWMisc Info: When I ping from PfSense to the BGP gateway, you can see all the ICMP traffic very clearly. Yet when I ping the Azure BGP address from a host on the 10.0.1.0/24 subnet, it times out. The issue seems to lie with the fact that routes in PfSense aren't populating (see "routes in pfsense" above).
http://imgur.com/22jtIMRChanges I've made since the above post:
I went ahead and took a friend's advice (a networking guy, not necessarily pfSense), and added a gateway on another physical interface on the box (em2, in this case). This interface became my IPsec tunnel interface, and the following changes were made:
System > Routing > Gateways
- Add GW for IPsec using 10.0.0.1 address
System > Routing > Static Routes
-
Destination Network: 10.1.254.254
-
Gateway: IPSec GW listed above (10.0.0.1)
VPN > IPsec > Tunnels > Phase 2
- Local Subnet: 10.0.0.1/32
Package > OpenBGPD > Raw Config
- Change all 10.0.1.1 references to 10.0.0.1. See specifics: http://imgur.com/ISrO6S4
The IPsec tunnel successfully connects, and BGP traffic flows over the enc0 IPsec interface (See: http://imgur.com/BO29aj8). However, nothing else does. If you look at the em2 interface we created above, you can see all the goodies that SHOULD be going over the IPsec tunnel (See: http://imgur.com/ZQ0vyT1)
Oh, and one final thing: routes are now showing in pfsense for the 10.1.0.0/16 and 10.2.0.0/16 subnets from Azure, and they are set to go over the 10.0.0.1 gateway. These routes were created by BGP, not me.
-
Hi Ryan,
were you able to get VPN-BGP fully working with pfSense ?
rgds,
Agentsmith73 -
Sorry, I never got this working. I ended up moving to an Ubiquiti EdgeRouter Lite 3 which works like a charm.
Best of luck!
-
From this i'm guessing it's related to the IPSEC and openbgpd issue that's ongoing.
https://redmine.pfsense.org/issues/6223