Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    Chelsio Advanced Configuration

    Hardware
    chelsio configuration shell driver
    3
    12
    2.9k
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • kklouzalK
      kklouzal
      last edited by kklouzal

      I am trying to load the t4_tom module, as stated in the userguide, and properly enable TOE for this Chelsio t422-CR adapter, is this possible?

      The Chelsio-UnifiedWire-FreeBSD-UserGuide.pdf has steps on how to do this however I'm receiving some errors.

      Enable TCP offload on a T4 interface using ifconfig. An error usually indicates that t4_tom is not
      loaded.
      [root@host~]# ifconfig cxgbeX toe
      
      If successful, TOE4 and TOE6 will be displayed on the list of the enabled options.
      

      "ifconfig cxgbe0 toe" unfortunately results in the following error:
      ifconfig: toe: Resource temporarily unavailable

      "ifconfig cxgbe0" does NOT show TOE3 and TOE6 present in the listed options

      Sub-Section 3.4 "Connection Offload Policies (COP)" step 1 explains how to load t4_tom:

      If not completed already, load NIC and TOE drivers:
      [root@host~]# kldload if_cxgbe
      [root@host~]# kldload t4_tom
      

      Which also results in the following errors:
      kldload: can't load if_cxgbe: No such file or directory
      kldload: can't load t4_tom: No such file or directory

      How can I load t4_tom and enable some of the advanced features of my Chelsio card?
      Specifically I'm trying to ensure the full TCP/UDP offload engine is enabled.

      Thank you very much for your time and have a wonderful day!

      1 Reply Last reply Reply Quote 0
      • stephenw10S
        stephenw10 Netgate Administrator
        last edited by

        cxgbe is already in kernel. t4_tom is not included, you would need to move it across from a FreeBSD source.

        However TCP offload is unlikely to make much difference on a firewall which is mostly only forwarding packets. Even if you did get it working.

        Steve

        1 Reply Last reply Reply Quote 2
        • kklouzalK
          kklouzal
          last edited by

          Thank you for the reply. Our use case is latency sensitive and any improvement, no matter how small, is time well spent. I do however understand that PfSense is mostly only forwarding packets so at the very least it would be interesting to benchmark the results after enabling these features.
          These cards are quite expensive and even sold on the netgate store, that being said they are essentially being endorsed by the company as a preferred adapter. I see no reason why all the available features should not be readily accessible within PfSense by default? Is this a feature request that can be made somewhere?

          1 Reply Last reply Reply Quote 0
          • stephenw10S
            stephenw10 Netgate Administrator
            last edited by

            Sure you can open a feature request here:
            https://redmine.pfsense.org

            Steve

            1 Reply Last reply Reply Quote 1
            • kklouzalK
              kklouzal
              last edited by

              For anyone interested, a feature request has been added to the bug tracker here: https://redmine.pfsense.org/issues/9091

              More information on offloads from https://service.chelsio.com/beta/drivers/ChelsioUwire-FBSD-3.3.0.1/README.txt

              " When a TOE capable interface is the outbound path to a destination to which a
              new connection is being established, the connection should be offloaded
              automatically.
              "

              It would stand to reason that if your system was setup to use one or more Chelsio adapters, either exclusively or as an adapter that saw heavy traffic (WAN facing adapter), then every packet PfSense routed/forwarded over one such adapter would benefit from offloading.

              1 Reply Last reply Reply Quote 1
              • kklouzalK
                kklouzal
                last edited by

                Checking in 15 months later to see if any new information on this is available.

                1 Reply Last reply Reply Quote 0
                • stephenw10S
                  stephenw10 Netgate Administrator
                  last edited by

                  Did you ever try loading the module from FreeBSD?

                  Did it work? Help at all?

                  Steve

                  1 Reply Last reply Reply Quote 1
                  • kklouzalK
                    kklouzal
                    last edited by

                    I'm not sure how to do that exactly. Previous attempts lead me to believe that I would need to build PfSense from source to include the module.

                    1 Reply Last reply Reply Quote 0
                    • stephenw10S
                      stephenw10 Netgate Administrator
                      last edited by

                      What I would expect is you copy the t4_tom.ko module from a suitable FreeBSD version and kldload it. That's the first thing to try here.

                      Steve

                      1 Reply Last reply Reply Quote 1
                      • L
                        laser22
                        last edited by

                        https://calomel.org/ has some great info on chelsio

                        kklouzalK 1 Reply Last reply Reply Quote 1
                        • kklouzalK
                          kklouzal @laser22
                          last edited by kklouzal

                          @laser22 said in Chelsio Advanced Configuration:

                          https://calomel.org/ has some great info on chelsio

                          Indeed, very useful information, this website is an invaluable resource.

                          I would be willing to test the module in my system if anyone were able to provide it.

                          Hope this can be added to the baseline PfSense one day..

                          As far as firewall/packet forwarding is concerned I don't know if it will make much of a difference but there are plenty of packages that could take advantage of this, squid, maybe pfblocker, among others and with no change to their source required. TCP/UDP offload from the CPU directly onto the Chelsio T4/T5/T6 ASIC freeing up resources for the rest of the system to do normal packet forwarding/firewalling.

                          The issue on redmine had been assigned to someone at one point but it was eventually unassigned for some reason...

                          stephenw10S 1 Reply Last reply Reply Quote 0
                          • stephenw10S
                            stephenw10 Netgate Administrator @kklouzal
                            last edited by

                            Try these. I can't test them against hardware but they load fine in 2.5.2:

                            [2.5.2-RELEASE][admin@252dev.stevew.lan]/boot/modules: ls
                            bwi_v3_ucode.ko if_wg.ko        linker.hints    t4_tom.ko       toecore.ko
                            [2.5.2-RELEASE][admin@252dev.stevew.lan]/boot/modules: kldload t4_tom.ko 
                            [2.5.2-RELEASE][admin@252dev.stevew.lan]/boot/modules: kldstat
                            Id Refs Address                Size Name
                             1   15 0xffffffff80200000  3aea720 kernel
                             2    1 0xffffffff83f19000     1000 cpuctl.ko
                             3    1 0xffffffff83f1a000     2698 intpm.ko
                             4    1 0xffffffff83f1d000      b40 smbus.ko
                             5    1 0xffffffff83f1e000    344d8 if_wg.ko
                             6    1 0xffffffff83f53000    137b0 t4_tom.ko
                             7    1 0xffffffff83f67000      c7e toecore.ko
                            

                            t4_tom.ko.txt
                            toecore.ko.txt

                            Remove the .txt extension.

                            Steve

                            1 Reply Last reply Reply Quote 0
                            • First post
                              Last post
                            Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.