Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    Mellanox connectX4 interface card needs special settings. How !?

    Scheduled Pinned Locked Moved Hardware
    4 Posts 2 Posters 382 Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • L
      louis2
      last edited by louis2

      In my pfSense system I have multiple interface cards. Two are Mellanox cards an ConnectX3 and a ConnectX4.

      In the pfSense logging I find the following alarms related to the ConnectX4 card.

      2025-06-04 14:45:52.000 pfSense 6 kernel kernel: mce0: ERR: mlx5e_ioctl:3618:(pid 675): enable txcsum first.
      2025-06-04 14:45:52.000 pfSense 6 kernel kernel: mce1: ERR: mlx5e_ioctl:3618:(pid 675): enable txcsum first.

      Does any one know how to enter the necessary settings!?
      Preferable using a method officially supported by Netgate

      Sincerely,

      Louis

      1 Reply Last reply Reply Quote 0
      • stephenw10S
        stephenw10 Netgate Administrator
        last edited by

        Do you have hardware checksum offloading disabled in Sys > Adv > Networking?

        Are you seeing that during boot? Continually?

        Unfortunately I don't have either of those to test with myself.

        L 1 Reply Last reply Reply Quote 0
        • L
          louis2 @stephenw10
          last edited by louis2

          @stephenw10

          No, I have / leave enabled as much offloading as possible!

          Just ALTQ support was enabled. I do not know why it was on. I turned it off.
          Enable the ALTQ support for hn NICs. Checking this option will enable the ALTQ support for hn NICs. The ALTQ support disables the multiqueue API and may reduce the system capability to handle traffic. This will take effect after a machine reboot.

          Back to the subject.
          The normal driver for ConnectX4 seems to be ^mlx5en^, the normal driver for ConnectX3 ^mlx4en^.

          according to kldstat both are not loaded, however both cards are working.

          mlx5en[25.03-BETA][admin@pfSense.lan]/root: kldstat
          Id Refs Address Size Name
          1 32 0xffffffff80200000 2dcd238 kernel
          2 1 0xffffffff82fce000 1e2a8 opensolaris.ko
          3 1 0xffffffff82fed000 60b278 zfs.ko
          4 1 0xffffffff835f9000 8808 cryptodev.ko
          5 1 0xffffffff83fe5000 5a80 udf.ko
          6 1 0xffffffff83feb000 23a0 cpuctl.ko
          7 1 0xffffffff83fee000 4250 ichsmb.ko
          8 1 0xffffffff83ff3000 2178 smbus.ko
          9 1 0xffffffff83ff6000 9290 aesni.ko
          10 1 0xffffffff84000000 20f0 coretemp.ko
          11 1 0xffffffff84003000 114f4 if_pppoe.ko

          I did find suggestions to place the following lines in loader.conf(5):
          mlx4en_load=“YES”
          mlx5en_load="YES"
          Or perhaps better to put in tunables
          mlx4en_load, value = YES
          mlx5en_load, value = YES
          (I do not know if that advise is still valid / a good advice for the actual FreeBSD version, however that are the Mellanox drivers)

          Looking a bit deeper. For that I used ^ifconfig -vvm <nic>^ which suggest that not all options available are turned on for the nic-cards I am using. So next to question if the ^optimal driver^ is loaded, perhaps additionally some driver related settings have to be added or changed. Perhaps best also via the tunables (or ideally via the gui)

          See the picture below what I found using ifconfig -vvm
          ( I use a picture so that the option fields are all in one row)

          To answer, your question related if the alarms occur all the time, I assume so, but I can not say it for sure. Recent days I was tuning my GrayLog SetUp which made things like this visible (normally you are not aware of the alarms!)

          bdce00e7-0af3-432e-aea6-c9b3823b80d9-image.png

          1 Reply Last reply Reply Quote 0
          • stephenw10S
            stephenw10 Netgate Administrator
            last edited by

            You shouldn't need to load a driver for those, it's in kernel:

            [25.03-BETA][admin@4200.stevew.lan]/root: kldstat -v | grep mlx
            		586 mlx4en
            		585 mlx4
            		587 pci/lkpi_mlx5_core_pci_table
            		584 pci/lkpi_mlx4_pci_table
            		233 pci/mlx
            		232 mlx/mlxd
            

            BTW that ALTQ setting only applies to hn(4) NICs, so only in Hyper-V or Azure.

            1 Reply Last reply Reply Quote 0
            • First post
              Last post
            Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.