Using HA XG-7100 1U's in a multi-homed environment



  • Hi Guys,

    Hope someone can shed some light on the following, its part software and part hardware.

    Current Setup

    I currently have a pair of XG-7100's in an HA config. I'm using the SFP ports in LACP mode connecting to a 2 stack switch for my lan connections.

    1 of the 4 port expansion cards on each device is used for CARP.

    I'm using another pair from same adapter (LACP) to connect to another stacked switch that handles WAN side connections.

    We have a /22 set of Public IP's and are currently using half of them /23

    (I'm using 10.X.X.X as public IP's in this example)

    We take IP transit from 2 Tier 1 providers using BGP. The connections are handled by a pair of Cisco ISR's with HRSP failover. They have IP's of 10.10.168.10 and 10.10.169.10. If one ASR link fails the working one will get a VIP of the failed one.

    Some of our servers are on the 168 segment, and others on the 169. What's we've done prior to deploying pfsense is to set the gateways on each server to be the corresponding .10 address.

    Problem/question

    The plan is that we will be using 1:1 NAT for all the servers behind the firewall thereby removing the need for individual firewalls on each server and the external NIC's and associated cabling.

    My question is, how do we route outbound traffic through the appropriate ASR/Gateway based on on the .168.X or 169.X segment?

    We only have one spare port on the devices so LACP is not possible and as I understand it the 8 port switch is redundant.

    My only thought at this stage is to make the gateway IP's /23.

    Any help would be greatly appreciated

    regards



  • Further to the above post, I'm guessing I could set the WAN ip address of the XG to be of type /23 and then it could talk to both gateways, but how to route outbound traffic via the correct ASR?

    regards


  • Netgate Administrator

    Were you able to get this setup?

    You could policy route outbouds traffic to the appropriate gateway based on the source IP.
    https://docs.netgate.com/pfsense/en/latest/book/multiwan/policy-routing-configuration.html

    However I think your problem here may be with inbound traffic because both gateways are on the same interface. The reply-to tags that apply to inbound traffic only apply to the interface gateway so all replies will likely go back via the same route, whatever is default. That will create an asymmetric route will all that implies.

    It would be better IMO to have two WAN interfaces with a /23 on each. Or apparent one upsteam router with the full /22 coming from it.

    Steve



  • @stephenw10 said in Using HA XG-7100 1U's in a multi-homed environment:

    https://docs.netgate.com/pfsense/en/latest/book/multiwan/policy-routing-configuration.html

    Yes I did in a manner of speaking. I followed your suggestion and used a /23 on the WAN and it works just fine.

    The issue I have is that I'm now not using the transit from the second provider save that it will act as a backup should the prime fail.

    Given that I'm using LACP on the 7100's for both Lan and Wan, I've no spare ports for a second Wan and I'm starting to think I've not bought suitable hardware which is a pain
    as I need 4 for a dual Wan, 2 for LAN and 1 for Sync and these devices only give me 5 usable ports even with the 4 port expansion card.

    regards


  • Netgate Administrator

    You could use one of the switch ports for the sync connection. Or just use a VLAN on the LAN lagg for sync.

    The Sync connection does not have a CARP VIP on it so the switched ports are fine for that.

    That would leave all 4 ports on the expansion card for two LACP LAGGs, one for each WAN.

    Unless I've misunderstood something.

    Steve



  • @stephenw10
    Thanks for the reply, however I need 2 ports for LACP on the LAN as well as the 4 for dual WAN.

    I'm using the fibre ports for sync at the moment. So I'm basically 2 ports short

    regards


  • Netgate Administrator

    You are using both SFP ports just for the sync connection?

    The bandwidth required for sync is not huge and it doesn't have a CARP VIP. Thus it can be a VLAN over the existing LAN LACP or one of the on-board switch connections.

    That leaves the two SFP ports as the LAN LACP and the 4 ports on the expansion card for WANs.

    Steve



  • excellent, thank you

    regards



  • I've been thinking on how to migrate what is in fact a production system without loosing connectivity. Can I do as suggested but make use of the sfp ports as WAN2 instead of LAN?

    If so, will I still be able to use one of the 8 port switch sockets as my sync?

    (I should add I will be in the datacenter incase it goes horribly wrong)

    regards


  • Netgate Administrator

    Yes, you could do that. A lagg of the two 10G SFP ports is obviously higher bandwidth, potentially at least.

    If you don't need that on the LAN then it makes no difference.

    Steve


Log in to reply