Chelsio T420-CR Integrated Switch

  • Hi All,
      New to pfSense and the forum! Excited about the software!
      I am trying to integrate a couple of Proxmox servers with 10Gb SFP+ NICs into a small gigabit network. Transfers between these two hosts are the only thing that requires substantial bandwidth (e.g. backups and migrations), but I want to integrate them into the same subnet as the couple other computers on the 1Gb LAN (cluster quorum agents and such). In trying to balance the cost-noise-power-space equation, I was very interested to learn of the switch integral to Chelsio's T4 and T5 series NICs. From their website:

    "T420-CR integrates a high performance packet switch, which allows switching traffic from any of the input ports to any of the output ports (wire-to-wire) and from any of the output ports to any of the input ports (host-to-host)."

    This reads to me like the big data transfers occurring between the two 10Gb ports would be confined to the card and only the small amount of traffic to the rest of the network would encounter PCIe bus and CPU limitations. Sounds like I could eliminate an "expensive", noisy, power-hungry switch with 10Gb "uplinks".
      Does anyone have experience configuring this functionality in pfSense? Just create a bridge in pfSense that includes 10Gb LAN ports and 1Gb LAN port(s)? Anyone have experience with the performance of such a setup?

  • Based on my laymens understanding of how Intel does this, the internal NIC will check if an outgoing frame is assigned to the MAC address of one of the other virtual NICs interfaces, and will instead forward over the "internal switch". This was done as a way to speed up vm-guest to vm-guest network communications. I assume this feature should transparently work as long as the OS can assign a MAC address to a virtual NIC device and not just promiscuity handle MAC in the software.  This feature may be related to SR-IOV.

  • Hi Harvey, thanks for your response. Your layman's understanding is still way above my head! As far as I can tell, you are discussing the virtual network communications between guests OSes on a single host. Apologies if my questions were unclear or if I'm just not understanding your response.
    My question is regarding pfSense's ability to utilize the Chelsio's onboard switch between it's two ports and pfSense's ability to bridge these ports to one or more ports on a separate PCIe (in this case Intel) 1Gb ethernet card. I've proceeded to order the Chelsio. If anyone has suggestions on configuration to keep the traffic between the two 10Gb hosts off the PCIe bus and away from the CPU (plus adding a bus/CPU bound 1Gb bridge), this newb could use any advice you have.