Any benefit to going 10Gig on LAN interface with Gigabit WAN?



  • Hey all,

    My current pfSense custom build is based on an Asrock H270M-ITX/ac board (wifi card removed) which has dual Intel on board NIC's (i291V and i211AT)

    I recently upgraded the main switch in my network to 10Gig SFP+ in order to handle higher speed internal connections (primarily between my server, workstation and my NAS)

    I'm all done setting it up at this point, when I realized I have a spare Intel x520 10Gig adapter I am not using. I could pop it in the pfSense box and hook up a spare DAC cable between the pfSense box and the main switch.

    I guess my question to this is, would there be any benefit at all from doing this, or would it just be a waste of power consumption?

    My theory is that theoretically there might be some minor latency improvements by allowing the switch to pass 10gig packets straight through to the pfSense box, but that in practice this would be completely negligible.

    I'm curious what those of you with more networking experience than I have on here think.

    Thanks,
    Matt


  • LAYER 8 Netgate

    Unless you are routinely pushing several hundred megabits/second through the gigabit interface I doubt you will see any perceptible improvement.


  • LAYER 8 Global Moderator

    The only reason I could see doing it is if pfsense was routing vlans for you.. That were flowing over this current 1gig uplink.



  • @johnpoz said in Any benefit to going 10Gig on LAN interface with Gigabit WAN?:

    The only reason I could see doing it is if pfsense was routing vlans for you.. That were flowing over this current 1gig uplink.

    I do do some routing between internal VLAN's but it is minor (mostly occasional SSH traffic for setup and ICMP packets for testing purposes) and likely will not have any noticible impact.

    I was thinking more from a perspective of switching efficiency.

    As I understand it, when transmitting packets over dissimilar links a switch has to receive the full packet into it's buffer before retransmitting it at a different speed. With same speed links many switches support passing the packet directly from one port to another before the entire packet is received, thus reducing latency.

    I was thinking having the link to my pfSense box at the same speed as the main switch would eliminate one buffer in the chain that leads to the WAN.

    Now I know, we are probably talking microseconds here, so it is probably not going to be noticible to anyone on the network. Or is it? Because sometimes small latencies have a way of snowballing until their effect becomes much bigger than you'd think.

    Also, I wonder if this will allow the pfSense QOS settings to be more effective as when you have bandwidth restrictions they are now occurring in pfSense, not on the switch.



  • I would love to know if your x520 is recognized correctly by pfsense if you should try it..

    Looking forward new cable modems slated for market have 2.5gbps interfaces on them in anticipation that cable ISP's will eventually offer speeds in excess of 1gbps offerings out now.

    Nothing wrong with trying stuff out.



  • @chpalmer said in Any benefit to going 10Gig on LAN interface with Gigabit WAN?:

    I would love to know if your x520 is recognized correctly by pfsense if you should try it..

    Looking forward new cable modems slated for market have 2.5gbps interfaces on them in anticipation that cable ISP's will eventually offer speeds in excess of 1gbps offerings out now.

    Nothing wrong with trying stuff out.

    Interesting. I hadn't even considered it wouldn't be. Under Linux it is handled by the ixgbe kernel driver.

    Of course FreeBSD is a little different. What under linux is handled by the e1000 driver is called em in FreeBSD. the IGB driver seems the same though.

    I think FreeBSD's ixgbe equivalent is just called ix, from what I can tell googling. Not sure though.

    I would find it difficult to believe FreeBSD wouldn't support the x520, given how common of an enterprise network adapter it has been for years, and how FreeBSD widely used in enterprise servers.

    I will post back when I find out.



  • @chpalmer said in Any benefit to going 10Gig on LAN interface with Gigabit WAN?:

    I would love to know if your x520 is recognized correctly by pfsense if you should try it..

    Looking forward new cable modems slated for market have 2.5gbps interfaces on them in anticipation that cable ISP's will eventually offer speeds in excess of 1gbps offerings out now.

    Nothing wrong with trying stuff out.

    Some further research.

    Intel's driver website indicates FreeBSD support for the 82599ES chipset used in the x520:

    https://www.intel.com/content/www/us/en/support/articles/000055236/network-and-i-o/ethernet-products.html

    They also appear to have driver downloads...

    https://downloadcenter.intel.com/product/41282/Intel-82599ES-10-Gigabit-Ethernet-Controller

    I've never downloaded a driver for Linux or Unix though, so I don't know how that works. Loadable module?

    Hopefully it a driver is shipped with FreeBSD. the ixgbe driver reportedly supports the 82599ES chipset, and that driver is available in FreeBSD, but the chipset itself does not appear to be explicitly listed in the compatibility list here:

    https://www.freebsd.org/releases/11.3R/hardware.html#ethernet

    I'll give it a shot and report back, but probably won't get around to it until later in the month.


  • Netgate Administrator

    I've been running this card for a while:

    ix0@pci0:2:0:0:	class=0x020000 card=0x17d3103c chip=0x10fb8086 rev=0x01 hdr=0x00
        vendor     = 'Intel Corporation'
        device     = '82599ES 10-Gigabit SFI/SFP+ Network Connection'
        class      = network
        subclass   = ethernet
    

    It causes reboot problems but that's almost certainly a BIOS issue in that particular box. It's an HP-560SFP+ card.

    x520 in general should definitely be supported, it's been in the ixgbe driver for years.

    Steve


Log in to reply