Trouble Routing between our Old LAN and our New psSense VLANs

  • Guys,

    Remember that I am not a networking guy... but a programmer now thrown to the wolves.

    Painting the Scene (optional reading):

    • My company has 4 geographic sites, all interconnected by site-to-site IPSEC tunnels using either Cisco or SonicWall firewalls.

    • Each site is one big, flat broadcast domain (one big Class "B" /16 subnet), no VLANs, etc.

    • Even though multiple fiber service providers are present at each site, no failover or load balancing of the WAN is configured.

    • The 4 old LANs are all 172.something/16.

    • In an effort to improve our circumstance, I have purchased 5 NetGate XG-1700-1U units with the expansion NIC installed (I love these things!).

    • I have contracted with AT&T to install VMWare's VeloCloud SD-WAN at each of our sites, in the near future.

    • While they will eventually handle our Site-to-site tunneling, I will be upgrading each of our sites to have 6 Class "C" internal subnets/VLANs in advance of that work.
      I have successfully stood up the Netgate/pfSense solution at each site with 10.whatever/24 VLANs. And so long as I don't try to communicate with my legacy network, it's working great.

    • I have site-to-site IPSEC tunnels up and working, VLANs routing, all that.

    What is not working (please help):

    • I can't get the legacy network to talk with our the new subnets in the following regard...

    • When I try to route traffic between my new network on Site A, it works great with Legacy Site A...

    • But my new network on Site A cannot communicate with the legacy network on Site B, C, or D, only the legacy network on Site A (so some routing is occurring).

    • I am controlling all access from the new to old network at each site by adding an extra interface on the Cisco Firewalls that connects to one of my pfSense VLANs, and building static routes between the two devices.

    • In fact, I am static routing everything.

    • We have a very distributed workforce, and I need to leave the legacy configuration in place, until a smoooooth migration can be completed, lots of folks VPNing into work, etc.

    • I suspect rules or configuration on the Netgate or Cisco (or both) devices is preventing me from communicating, but I am lost.

    • Any suggestions on a great write-up on doing this, or a point in the right direction about how to configure, where to look, etc. would be appreciated.

    • We will keep the pfSense firewalls in place even after our SD-WAN architecture is up, though they will then serve more for LAN routing, packet inspection, etc.


  • OK, I have figured this out, so I am sharing here in the hopes it helps others. I was doing a few significant things wrong.

    A much-simplified Example:

    New Site Network:

    • Netgate/pfSense firewall/router
    • LAN
    • Actually VLAN'd, half dozen subnets, etc.

    Old Site Network

    • Cisco Firepower firewall/router
    • No VLANs, subnets, etc.

    I needed to make the old network play nice with the new one until resources could be fully migrated over time.

    • I tried defining an interface on the new network that used an IP address from the old network, setting up routing and rules between the two, etc. That ended badly.

    • Maybe I had it backwards: I tried defining an interface on the old network that used an IP address from the new network, setting up routing and rules between the two, etc. That ended badly.

    • I tried setting up a new simple, dedicated subnet solely for the purpose of interconnecting the two routers to manage transferring data between the two networks, static routes, etc. That did work.

    I call it a transport network, but I bet you networking guys who know what you are doing actually already have a name for it (I'd be curious what that is).

    Where making the new network at one geographic site talk to the old network at a different site is concerned, I discovered that adding P2s on the IPSEC S2S tunnels was the trick (and not setting static routes, which I had tried).

    Problem solved. Thanks.

Log in to reply