Help with MPLS/Multi-Site to DC using pfSense
-
I'm sure I'm over-thinking this one, but it's a whole new territory for me with pfSense.
We're about to light a dozen branches with a "MPLS" fiber product, basically just a straight layer 2 link between them and their cabinet at our DC replacing site-to-site VPN's on broadband connections. Right now all the branches have the little C2358 based netgate machines running the latest 2.2, and their cabinet has a 2758 running 2.1.6. Currently they have the site-to-site tunnel over their broadband connection, and get their Internet access via the local broadband (split tunnel, obviously).
When the MPLS is lit we're aiming (or going to) have them come back to their cabinet for local, and have their internet access come out of there. Internet to the cabinet is already there, comes in from our core router to a Cisco 4000 series switch in that cab, then to their C2758. All dozen MPLS links will come via once piece of fiber into the DC, then into their cabinet either to the switch or their C2758. I also have a second brand-new C2758 ready to go in for that business unit, for redunancy or for something else.
Now my question - how best to accomplish this? We're doing similar with our three "major" branches, but using Cisco hardware. My first thought was just to setup another interface on the core 2758(s) on a unused subnet, say 10.100.0.0/16 and have it be the "wan" for all the branch pfsense boxes (i.e. - cabinet 10.100.1.0/24, branch 1 10.100.2.0/24, branch 2 10.100.3.0/24, etc etc) then just add rules allowing traffic between the "LAN" subnets to/back from the cabinet.
Second though was to use VLANs and pull the branch pfSense boxes out of the mix, but that will get ugly, fast. I'd (ideally) like to keep the branch pfsense boxes in place if we decide to add another WAN connection for some, etc.
So - those who have done it - how would you accomplish it? Again the connectivity is just straight layer 2 between the branch and the cabinet for them. All 12 will be coming in on one piece of fiber or copper into the cabinet.
I'd really like to make this work as smooth as possible, it's basically a proof of concept for my three main sites which right now are doing the same thing, but all Cisco. When we move to 10 Gig links between those main sites and the DC, I don't even want to fathom what Cisco is going to charge us - I'd rather take that opportunity to move everything to pfSense.
Finally, I'm not above reaching out to support and paying to get this done (I had the free incidents with the C2758, I just never registered them.. oops!).