Potential Alert for Hijack
not intended Please excuse me, I have been awake working on these problems without proper rest or care. Not ideal!
I figured since I jotted this much down that it would be also… Not ideal.. to discontinue posting in this timely thread. So forgive me! When I wake up I'll probably want to rewrite it anyway.
SO,
As the OP, lacking in my understanding and practice with such a series of changes from the usual -
I am presently dealing with the same scenario and difficulties in deploying MPLS. No matter what I've tried so far, There seems to be random disconnect issues. Or other hiccups.
My setup is virtually like the original post,
Complicated by the fact I am not sure how to best tackle this without issues. I am used to using PFSENSE as a direct-to-wan device and controlling my filtering and such, But with this MPLS deployment we decided to get rid of some old network structure which is where all lan clients are migrating to. (Servers already in both networks, Printers to follow clients.)
One big consideration: If you had 20 or 30 MPLS sites up - How would you run your MPLS/pfsense Interface? On a /16 private subnet of the entire CE range, a non-related /24 with routing, or within the CE? I do like routing but not having control over any of the endpoints is a hindrance, and its not as easy to make changes and see if there is a misconfiguration on equipment I do not control not to mention do not own or know inside and out.
The existing network of IPSEC Tunnels was very stable, and I intend for this to be better.
I resolved the immediate issues with migration and random disconnections (Via Telnet, RDP) by assigning second NICs native to the MPLS LAN Range for the critical services, But this defeats the effort I put into VLAN Trunking the old LAN and new networks - We rewired the building and there are literally two separate networks both at the desk and in the network rack hooked up via a router Trunk (Cisco 1841) in between MPLS and PFSENSE. It creates two completely different networks, with the MPLS and Vlan tagging to match on both the Interconnect and the MPLS Edge.
I am thinking that we are struggling with what MPLS Gateway should be - PE or CE (Customer Edge and Premises Edge) - Or which one we should be routing to as a default gateway on the new optional interface. (MPLS on OPT1, PFSENSE plugged into it feeding it DHCP and allowing clients on the old network range to utilize the MPLS via NAT for the time being.) Currently I have the Customer Edge as the PFSENSE Gateway. I have traffic passing between the networks rules, and even bypass traffic on same interface, Yet still issues.
Like the original post, I am stuck understanding how I am going to allow Public IP's inside to allow email or webservers to sit with the MPLS Private IP's as my "WAN" Endpoints. After reading about 15 threads relating to MPLS on these forums, seeing a variety of issues people have had with very mixed results - so here I am hoping to gain some further best practices and insight here. I want to make sure PFSENSE is setup right to allow for this as well.
So ignore the rest of my DERAILING post - I am seeking clarification on the original posters issues as well as the community's experiences and woes.
It comes down to simply finding out how to BEST get PFSENSE to handle the traffic. I do not want to bridge interfaces, I want to move everyone and eventually the LAN itself.
I am stuck with any hosts on existing network using PFSENSE as their default gateway dropping packets to hosts connecting through the trunk/interconnect regardless of the gateways. It seems to happen between networks and what has been described as an async nat or routing: It is Intermittent about every 5 to 10 minutes or so. As soon as I use another gateway on the Lan segment, NOT PFSENSE, a LinksysCisco Router for example with the same static routes - everything is okay. I can connect to the hosts just fine - Using the Customer Premises as the default gateway. I will serve this out via DHCP if I have to but I would like to understand what I am doing wrong, and what I could be doing better in this scenario of Private MPLS.
However, anything can communicate perfectly across to hosts sitting in the MPLS OPTLAN Subnet, for example, printing. Its just as soon as it hits Pfsense Interface IP on either LAN or MPLSOPT, something isn't going.
For now I have added secondary gateways to the problematic hosts but this is obviously a patch solution.
Before getting to modifying NAT rules (Do Not NAT for OLDNET to MPLSNET and Vice Versa) I couldn't even ping the hosts with PFSENSE as their Gateway, from the MPLS CE Router and new 10. network range. But again, anything using the old "Default gateway" on the lan, we had no issues at all communicating in the exact same round of tests.
All the issues (And NAT) go away if I disable filtering. I'm curious to know if PFSENSE is stripping the MPLS traffic and somehow dropping the VLAN tags, or simply NATTING where it should just be handing traffic off and out. Perhaps the solution is not to provide a workaround but to just completely migrate the entire network. IE: Disabling NAT. I want to prepare PFSENSE, regardless, for hosting with this MPLS setup and I am concerned that QOS and other nice features are being dropped by the way I am doing things with PFSENSE.
Perhaps I am missing something with the rules, or otherwise.
For the record I am using a BETA SNAPSHOT. Feb 18th. 2.1-BETA1
I am using ALIASES with networks defined as Allow (I am not sure how well this works -in these scenarios- Time will tell.)
I will continue to review the forums and look back here.
I am a supporter and strong pfsense lover,
I am SURE it can do what I want it to.
Could it be that POLLING is causing my issues?
There are so many variables - Literally dozens.
I do not mean to hijack,
(This post is WITHOUT INTENT for technical expectation for a resolution - I would obviously have to attach a couple drawings or post MUCH more detail, I am seeking to inform as well as hoping to stumble upon something someone may have come across - I have sure read a lot of like-minded issues on this.)
As an afterthought,
One of the members in another MPLS post mentioned he gave the Cisco Router between the MPLS and PFSENSE its own IP and subnet to resolve what sounded alot like what I'm seeing. I'm just stuck in my approach, I suppose.
http://forum.pfsense.org/index.php?topic=35906.0
http://forum.pfsense.org/index.php?topic=43938.0
http://forum.pfsense.org/index.php?topic=50910.0
http://forum.pfsense.org/index.php?topic=26228.0 - Older 2010 - But a spot on thread I would like to share and ask a bit more about - So adding a gateway to an OPT turns it into a NATTED wan Like interface, but removing manual rules erases that. Ideally this is the best, if possible to provide alongside a functional way for old clients to use the new CE MPLS gateway amidst migration.
and specifically: http://forum.pfsense.org/index.php/topic,24405.msg126788.html#msg126788
Curious, to think about asking the provider to cut up their MPLS services as mentioned above - I didn't think they could or would do that, though it would be lovely. How else would it be done beyond 1to1 nat. Cannot visualize how it would be with an MPLS/PFSENSE setup without major headache.
Hopefully some of these threads regarding MPLS are helpful for others as well. l )
Best,
Me.
