Multi Site Multi WAN Multi VPN - Help Please
Please excuse my ignorance as I am new to PfSense and the multitude of configurations capable within it's framework but I would like to enlist advice/help from the many users on this forum.....
I have 3 sites, all with PPPoE dual-wan environments from 2 different providers...
Predominantly, they all use an equal-weighted Uplink group for outgoing traffic, balancing fairly well as they are almost equivalent speeds/capacities (Same at all 3 sites).
I had tried setting up multi interface IPSEC with routing between each site (wanting to have both provider VPN connections connected at the same time - so no failover should an interface/vpn drop), but was unable to create the static entries on each interface at the same time for a route statement. I played around with FRR trying to facilitate the weighted routes that way but also ran into problems.
I then opted to try to create a framework whereby each firewall ran a server - e.g. Site 1 ran a server that site 3 connected to, Site 2 ran a server that Site 1 connected to and Site 3 had a server that Site 2 connected to on a series of on the one provider. Then did the same again with the second provider, just as a client instead of a server .....
I ended up with 2 incoming and 2 outgoing on each firewall, all connected without issue.
IP subnet for Site 1 was 172.16.1.0/24. Site 2 was 2.0/24 and Site 3 was 3.0/24 .....
I setup FRR which is aware that for each subnet there are 2 possible paths..... I set weighting for primary as 10 and secondary as 20.
From any site I was able to connect to the other subnet without issue - 1 could get 2 and 3, 2 could get 1 and 3 and 3 could get 2 and 1 etc.
Unplugging the primary interface I lose all connection to the other sites, so I know I probably have something monumentally screwed up somewhere, and I'm sure there are better ways of doing this, but if someone could please point me in the right direction I would be very grateful. Remote access can be arranged if you want to have a browse around and tell me how bad I had it all configured, lol!
Any help would be appreciated as I'm driving myself mad.