XG-7100 redundant connections to external switches



  • I am in the middle of deploying a set of XG-7100 firewalls and am having some difficulty creating redundant connections to external switches. On previous pfSense firewalls without integrated switches I would setup a LAGG (either failover or LACP depending on the situation) but I don't believe that is an option on the XG-7100 units without using the SFP+ ports (which I plan on reserving for future 10G upgrades and don't currently have modules for anyways).

    At first I thought the lack of ability to create an externally facing LAGG wouldn't be an issue because I would simply add redundant connections from my external switches and let STP sort it out. But after making multiple attempts at this I'm coming to the conclusion that the XG-7100 units don't support STP on the internal switches. Bear in mind that I haven't done a packet capture so I haven't been able to confirm.

    Do the XG-7100 switches support STP? If so, do they support rSTP and what are the other details behind their default configuration? Are configuration changes (priority, port mode, etc) relating to STP support possible?

    Without using STP, SFP+ modules, or other expansion cards is there any other way to create redundant connections to an external switch on the XG-7100?


  • LAYER 8 Netgate

    This post is deleted!

  • Netgate Administrator

    You can configure a LAGG to the onboard switch but it, currently, only supports load-balance mode. That's how the internal LAGG is configured but additional LAGGs to external ports can be set.

    Steve



  • Is there any documentation on how to setup additional load-balance LAGGs going out of the internal switch?

    I reviewed the video guide on the switch and it is mentioned but I don't believe it is covered in detail and it doesn't seem obvious how one would configure it. As far as I can tell you can only create LAGGs using unassigned physical interfaces, not the internal switch ports.


  • Netgate Administrator

    You just have to add the ports to a new lagg group on the Ports tab i the switches config:

    0_1551807910377_Selection_592.png

    So here ports 7 and 8 are a lagg to another switch or another pfSense box carrying the LAN subnet.

    Steve



  • That make sense.

    Any plans or ability to provide STP or LACP support on the internal switch in the future? I'm not sure if this is a driver or hardware limitation.


  • Netgate Administrator

    Both are being looked at. We would love to have them on the 7100 switch.
    I have not heard anything definite though.

    Steve



  • @stephenw10 actually you give wrong instruction. If you will do change LAGG on ETHn Ports from "-" to anything else yours pfSense WILL STUCK. I tried many cases how configure LAGG on XG-7100 Switch lagg0 and in end: ☠ I loose my network connectivity. So please could you explain how it can work?


  • Netgate Administrator

    Not sure I understand.

    You definitely can configure additional lagg groups on the switch as I showed there.

    They can't be lagg group 0 as that is already used on the internal ports 9 and 10 to connect to ix2 and ix3.

    The ports you set that on will be inaccessible until you connect them to a load-balance lagg so don't change it on ports you're using. to access the gui.

    Steve



  • @stephenw10 I changed "-" to 1 on two disconnected ports, click "save" and after this all connectivity to xg-7100 brokes. If connect to com port and run ifconfig no second lagg1 will be showed. Only restore to previous configuration with reboot fix issue. I have one xg-7100 not in production and have one cisco sb200fe 48ports to tests.



  • Could you bee so kind and share with me working XG-7100 pfSense configuration backup with factory defaults and configured switch ports:
    untagged port 3 and 4 to lagg1 on vlan 4091
    untagged port 5 and 6 to lagg2 on vlan 4091
    I want connect 3-4 port to #1 cisco switch on lagg. And 5-6 to #2 cisco switch on lagg.


  • Netgate Administrator

    Configuring and additional lagg group in the switch does not create a new lagg in pfSense Interfaces. It's a lagg only between the switch and whatever is on those ports.

    The internal switch can only do load-balance lagg. The Cisco switch must also support that.

    Steve



  • @stephenw10 yes I know that only LB can be done. No LACP. But still, when I have default hardware and clean OS/config changing "-" to something like 1 on port and click "save" destroy connectivity. So I don't know... Help pls.


  • Netgate Administrator

    Hmm, I see. That was an old config I have on that box.

    Try running at the console after making the change: etherswitchcfg

    That will show the current switch settings. Do you see, for example?:

    laggroup0:
            members 5,6,9,10
    laggroup1:
            members 5,6
    

    There I added ports 5 and 6 to lagg group 1 but they have been added to group 0 also.
    If so as a workaround you can set that back to just 9 and 10 using:
    etherswitchcfg laggroup0 members 9,10

    Steve



  • @stephenw10 ok, thanks. I will try maybe on monday, and will reply. but if I remove 5,6 ports from lagg0 this will be mean that pfSense can't reach them and clients on 5,6 will doesn't reach pfSense. This will be isolated switch, no?
    And if not remove 5,6 from lagg0 before add to lagg1 it will be collizion? That's why my pfSense was sticking previously...



  • @stephenw10 about that you said:

    etherswitchcfg laggroup0 members 9,10

    I see that this is a BIG 🐛 in pfSense Switch WebConfigurator.
    Configuring laggs over shell works fine.
    I created bugreport about it - hope they will fix it soon, before this I will use shell for it 😒


  • Netgate Administrator

    It is already fixed in 2.5 snapshots if you're able to try those. It was in fact fixed in 2.4.5 snaps too.

    Steve



  • https://redmine.pfsense.org/issues/9447 :( wasted time. Ok. Thanks. I better wait for Stable release. I'm now on Latest Base 2.4.4_2. There is known date of 2.4.5 or 2.5 release?


  • Netgate Administrator

    Not for 2.5 release. I'm not aware of any particular issue with running 2.5 on it currently but things are changing there everyday.

    You can set that command to run via the shellcmd package if you wish do you don't have to manually run it boot.
    https://docs.netgate.com/pfsense/en/latest/development/executing-commands-at-boot-time.html

    Steve



  • @stephenw10 you mean that etherswitchcfg laggroup1 members 5,6 is not permanent? Ok.


  • Netgate Administrator

    The config is actually correct. The bug is how it gets applied to the switch so when you reboot and that gets applied....

    Let me test that quickly...



  • @stephenw10 yes I test it, on reboot is same as like configuration from Web:
    laggroup0:
    members 3,4,5,6,9,10
    laggroup1:
    members 3,4
    laggroup2:
    members 3,4,5,6
    I will wait stable release thanks, topic done


  • Netgate Administrator

    Sorry I was called away. That does work though:

    Selection_604.png

    I'll see if we can get a patch you can apply directly to 2.4.4p2 via the patches package. That would be cleaner.

    Steve


  • Netgate Administrator

    Ok here's a patch you apply against 2.4.4p2. You can use the system patches package to do it which removes most of the scope for error.

    Selection_605.png

    switch_lagg_fix.diff

    Steve



  • Hmm I not getting running a trunk. (I have applied the patch above on 2.4.4P2)

    I have configure on the HPE/Aruba switch side two interfaces:

    interface 1/45
       untagged vlan 1
       trunk trk3 trunk
    
    interface 1/46
       untagged vlan 1
       trunk trk3 trunk
    

    and this on the PFSense:
    RIM-002-c.jpg

    If I connect Eth5 on one of that interfaces it works.
    If i connect Eth4 also to the switches, it still works but as soon as I unplug Eth5 my connection to the PF is lost.

    etherswtchcfg seems ok:

    laggroup0:
    	members 9,10
    laggroup1:
    	members 3,4,5
    

  • Netgate Administrator

    How is the trunk configured in the HP switch? It must be as load balancing as the 7100 on-board switch is.

    Steve



  • Hi I've just created a normal trunk without LACP

    trunk e 17/18 trk3 trunk
    

  • Netgate Administrator

    And there is no setting for failover, load-balance etc?

    What happens if you connect Eth4 first in your current setup? Or just Eth3?

    Do you get traffic over the first connected link that then fails when you disconnect it?

    Steve



  • @stephenw10
    Nope there is only an option if you wish an LACP Trunk or without LACP.
    The Trunk and interfaces are always up on the switch and the PF.
    But only one interface on the PF is working. Doesn't matter which one I connect first.

    If i configure eth 3-5 on the PF in the same LAG, only port 5 is working.
    If i configure eth 3-4 on the PF in the same LANG, only port 3 is working.
    really strange.

    If I ping the firewall i see on the packets counter that the traffic goes and comes from different interfaces in the LAG.



  • Hi all, do you know if the fix is now part of the current version 2.4.4p3? I'd like to configure something similar but I don't want to mess with the terminal. Thanks.


  • Netgate Administrator

    The fix allowing additional lagg groups to be added should be in p3 yes. If that's what you're referring to.

    Steve



  • hi @stephenw10 , i'm very new to pfsense and negate so sorry if I ask you to repeat it, but starting from the default configuration could you explain to me how I should proceed to create a lagg between 4 ports of the internal switch? Just to give you the full picture, this is what I want to achieve:

    ETH1 --> WAN

    ETH2, ETH3, ETH4 --> not used for now

    ETH5, ETH6, ETH7, ETH8 --> LAGG with interface OPT3

    Can I do that with the UI?

    Thank you.


  • Netgate Administrator

    You can do that but only using a load-balance type lagg. What ever you are connecting it to has to support that.

    You can't access that as a different interface in pfSense. Traffic using that still has to be sent to the internal switch via lagg0.

    You should start a different thread for this.

    Steve



  • Do you mean I should manually type "1" in the LAGG column for the ports I want to join in a lagg? and that's it? Like you showed in this post:

    @stephenw10 said in XG-7100 redundant connections to external switches:

    You just have to add the ports to a new lagg group on the Ports tab i the switches config:

    0_1551807910377_Selection_592.png

    So here ports 7 and 8 are a lagg to another switch or another pfSense box carrying the LAN subnet.

    Steve


  • Netgate Administrator

    Yes, if you want an additional lagg group of switch ports from the internal switch to some external device (that also support load-balance lagg).

    Steve


Log in to reply