Intel X553 NICs (on C3758 SoC) not detected by pfSense 2.4/2.4.1



  • Hi!

    I have a Supermicro 5019A-FTN4 server, which has a A2SDi-8C-HLN4F mainboard with a C3758 SoC. It has 4 GbE ports which are powered by Intel X553 NICs.

    I was originally planning to run it as a VM in Proxmox VE but I'm having some issues with throughput in that setup, so I'm trying to narrow down the cause of that issue, which has brought me to attempting to install pfSense bare metal. However, when installing pfSense (I've tried both 2.4 and 2.4.1) on the server, no interfaces are detected.

    From everything I can see, the new FreeBSD 11.0 base in pfSense 2.4+ should support the X553 NICs out of the box. Is that not correct?

    I've been attempting to load the drivers into pfSense manually, but I'm not very proficient in FreeBSD so while I could probably succeed at some point, it would take a long time, and given that FreeBSD 11.0 should support the X553 anyways, I'm not even sure that this would fix the issue.

    Does anyone have any tips or suggestions?



  • Having same issue so I am following



  • Same issue same board.
    I thought the driver was added to FreeBSD in March.
    Tried to install FreeBSD separately to double check but not included as well.

    Seems to work in debian/ubuntu with the latest drivers from intel https://www.servethehome.com/day-0-with-intel-atom-c3000-getting-nics-working/

    Intel's latest driver includes X553 in their list of compatible devices.
    https://downloadmirror.intel.com/14688/eng/readme.txt

    Probably relevant as well:
    https://lists.freebsd.org/pipermail/freebsd-stable/2017-July/087388.html

    –---

    Another thread on the same topic:
    https://forum.pfsense.org/index.php?topic=137182.0



  • I also found this

    [–]gonzopanchoNetgate 3 points 2 months ago
    They don't work yet, but I canceled a family vacation (they went without me) to work on C3000.
    permalinkembedreport
    [–]4GettableOne 3 points 2 months ago
    That's hardcore thnx! Someone get this guy some gold and a beer.
    permalinkparentreport
    [–]pixel_of_moral_decay 1 point 2 months ago
    I think /u/gonzopancho is shitting out gold at this point ;)
    permalinkparentreport
    [–]SuddenWeatherReportCCNP R&S 3 points 2 months ago
    Dude you should have just went
    permalinkparentreport
    [–]ericmotordu 1 point 1 month ago
    /u/gonzopancho, can you indicate which part does not work? Thank you, Eric



  • I have a Supermicro 5019A-FTN4 server, which has a A2SDi-8C-HLN4F mainboard with a C3758 SoC. It has 4 GbE ports which are powered by Intel X553 NICs.

    Both the X553 (1 GBit/s) and X557 (10 GbE) ports are still not working yet due to the lag of support for the entire
    C3000 SoC, this ports are integrated in the SoC. If you draw in an Intel i340-T4, or i350-T4 or Intel Pro 1000 PT
    adapter it should run out of the box because these adapters are fully supported.

    I was originally planning to run it as a VM in Proxmox VE but I'm having some issues with throughput in that setup, so I'm trying to narrow down the cause of that issue, which has brought me to attempting to install pfSense bare metal. However, when installing pfSense (I've tried both 2.4 and 2.4.1) on the server, no interfaces are detected.

    Link 1 OpenSSL benchmark
    Link 2 Intel Atom C3958 16-Core Top End Embedded QAT Linux Benchmarks and Review

    From everything I can see, the new FreeBSD 11.0 base in pfSense 2.4+ should support the X553 NICs out of the box. Is that not correct?

    Now we are on FreeBSD 11.1 and there were much things that was fast changing between the version 2.4.0 and
    version 2.4.1 and there fore I personally think this will be not all worked out, and there will be perhaps the primary view
    on at this time as I see it right, and it rumors too that Netgate the company behind pfSense will be launch their own
    C3000 based hardware and due to that date they perhaps are actually working on it and they are not fully finished up
    all things according to that platform.

    I've been attempting to load the drivers into pfSense manually, but I'm not very proficient in FreeBSD so while I could probably succeed at some point, it would take a long time, and given that FreeBSD 11.0 should support the X553 anyways, I'm not even sure that this would fix the issue.

    Fairly said if this will be only an Intel NIC chip that is soldered alone on the board I would say try it out, but while this
    NICs or LAN ports are integrated into that C3000 SoC I will say try it out but on your own risk.

    Does anyone have any tips or suggestions?

    if you are willing to try it out really, install a fresh and fulla FreeBSD 11.1 OS on that system and see what
    is going on then! And then you might be able to get the driver out from there and put it on the pfSense 2.4.1
    system you must install again on that platform (hardware) then you can see what happens really, all other things
    are pure guess work until one of us were just doing and report it!



  • The driver exists, was just not added in PFSense.
    Working fine in FreeNAS beta, just need to find out how to export the driver…

    https://bugs.freenas.org/issues/26292



  • Ok, so I finally got it to work but still need to find some better way to do it, limited freebsd knowledge over here.

    I installed freebsd 11.1 RELEASE, same as pfsense 2.4.1, which doesn't have the updated driver (STABLE already includes it).
    Downloaded the 10Gb freebsd driver from intel website:
    https://downloadcenter.intel.com/download/14688/Intel-Network-Adapters-Driver-for-PCIe-10-Gigabit-Network-Connections-Under-FreeBSD-

    Compiled the driver and saved to an usb stick.

    Then installed pfsense 2.4.1, copied the driver to /boot/modules/ but got an error trying to load it

    module_register: cannot register pci/ix from if_ix_updated.ko; already loaded from if_ix.ko
    

    Then I ran

    kldunload /boot/kernel/if_ix.ko
    

    Followed by the load command again

    kldload -v /boot/modules/if_ix_updated.ko
    

    I know this is far from ideal but how can I make this work in boot?

    Do I need to compile a new kernel with the driver built in or is there a way to "tell" the kernel to load this module instead?
    I guess replacing it in /boot/kernel/ folder is not a good idea?

    Thanks.



  • I know this is far from ideal but how can I make this work in boot?

    Try out to run a script that is unloading and loading this for you after booting.

    Do I need to compile a new kernel with the driver built in or is there a way to "tell" the kernel to load this module instead?

    This would be not so easy as you could think about.
    /boot/modules is the right position here.

    At least only to be sure, now the NIC is running fine and able to use for you?
    A small thread about that and worth the time to read it too



  • Turn off VT-d support in the A2SDI BIOS and pfSense will find all  the NIC:s



  • Well, that would be interesting but not working here…
    Can you be more specific regarding the options to disable?



  • Can you be more specific regarding the options to disable?

    @Mullet Scandinavia AB said:

    Turn off VT-d support in the A2SDI BIOS and pfSense will find all  the NIC:s



  • wow, thanks…
    But of course I did that already and even reinstalled pfsense.
    NICs are not there, the only way I can make it work is with the newer driver...



  • This is weird. I had it up an running this morning with VT-d off but now it don't work again!

    And there are so few tools in the pfsense image so it's hard to debug no lspci and no network to fetch it either…

    Have tried new beta BIOS und updated LAN EEPROM but still not working. Strange that it works sometimes.



  • Guys any news on X553 driver?
    It's still not working on 2.4.2_1 and it'a January already.
    I've bought 6 pcs SYS-E200-9A with A2SDi-4C-HLN4F boards.
    It's perfect all in all even Windows 2016 server works perfectly there, but not pfsense.
    Any updates of manually putting FreeBSD drivers in there?
    disabling virtualization or VT-d in BIOS does nothing for pfsense and NIC detection.


  • Netgate Administrator

    The drivers were not in 11.1 release which is what we build on so until they are included in our base we may not get them.

    You can build the kernel module yourself as explained by EditioN above. To load it at boot time just create the file /boot/loader.conf.local and then put in that file:

    if_ix_load="YES"
    

    Steve



  • Thanks for the answer ,

    Will it survive the updates then? I mean how frequent PFsense updates modules in /boot? or it's more like image updates for the whole partition?
    So for now we may use that solution, but if will need to put it manually to /boot on every update i'm not sure it's usable as those will be in remote offices with hardly proper IT personnel avail.


  • Netgate Administrator

    The imported kernel module will be specific to that version. It probably won't work in 11.2 for example. We won't have tested anything against it so there's no guarantees that any pfSense update won't break it.

    Really you should wait for support to be included in the release. Certainly before deploying it in any critical application in my opinion.

    Steve



  • Managed to get it working somehow.
    Anyhow, we have bought those appliances already for all the offices, sadly before checking if it's supported yet! :)
    I failed to find a proper way to disable existing module, so i just rename it and load a module manually compiled from Intel sources.
    After that don't forget to fix permissions chmod 555 /boot/kernel/if_ix.ko and u'r good to go. (reboot)

    So yeah, it works just fine after that, but don't survive upgrades as far as i understand pfsense updates kernel also, along with modules apparently.

    Instructions:

    on 1st boot after install press any key on PFsense logo

    boot in Single User mode
    press enter for /bin/sh shell

    #connect your USB stick (Fat32 formatted) with driver
    #check what's it's name
    ls /dev/da*
    #mount usb
    mount_msdosfs /dev/da0s1 /media
    #unload existing if_ix module - check it's ID if necessary /sbin/kldstat )
    /sbin/kldunload -i 2
    #mount root as RW
    mount -rw /
    #rename existing if_ix.ko module
    mv /boot/kernel/if_ix.ko /boot/kernel/if_ix.ko.blacklisted
    #copy new module from USB
    cp /media/if_ix.ko /boot/modules/
    #fix permissions
    chmod 555 /boot/modules/if_ix.ko
    #make loader.conf.local
    echo 'if_ix_load="YES"' >> /boot/loader.conf.local
    #enjoy
    reboot

    In case somebody needs it - here's yandex.cloud direct link (compiled for FreeBSD 11.1 release) https://yadi.sk/d/F92wgHSy3RNDS9



  • You sir, have saved my bacon.  Your instructions and file has worked perfectly for me.

    Cheers.

    @MarvinFS:

    Managed to get it working somehow.
    Anyhow, we have bought those appliances already for all the offices, sadly before checking if it's supported yet! :)
    I failed to find a proper way to disable existing module, so i just rename it and load a module manually compiled from Intel sources.
    After that don't forget to fix permissions chmod 555 /boot/kernel/if_ix.ko and u'r good to go. (reboot)

    So yeah, it works just fine after that, but don't survive upgrades as far as i understand pfsense updates kernel also, along with modules apparently.

    Instructions:

    on 1st boot after install press any key on PFsense logo

    boot in Single User mode
    press enter for /bin/sh shell

    #connect your USB stick (Fat32 formatted) with driver
    #check what's it's name
    ls /dev/da*
    #mount usb
    mount_msdosfs /dev/da0s1 /media
    #unload existing if_ix module - check it's ID if necessary /sbin/kldstat )
    /sbin/kldunload -i 2
    #mount root as RW
    mount -rw /
    #rename existing if_ix.ko module
    mv /boot/kernel/if_ix.ko /boot/kernel/if_ix.ko.blacklisted
    #copy new module from USB
    cp /media/if_ix.ko /boot/modules/
    #fix permissions
    chmod 555 /boot/modules/if_ix.ko
    #make loader.conf.local
    echo 'if_ix_load="YES"' >> /boot/loader.conf.local
    #enjoy
    reboot

    In case somebody needs it - here's yandex.cloud direct link (compiled for FreeBSD 11.1 release) https://yadi.sk/d/F92wgHSy3RNDS9



  • @daddygrant:

    You sir, have saved my bacon.  Your instructions and file has worked perfectly for me.

    Cheers.

    Glad it worked and happy to help! Guess we'd wait for official support now and upgrade afterwards…



  • @MarvinFS:

    Managed to get it working somehow.
    Anyhow, we have bought those appliances already for all the offices, sadly before checking if it's supported yet! :)
    I failed to find a proper way to disable existing module, so i just rename it and load a module manually compiled from Intel sources.
    After that don't forget to fix permissions chmod 555 /boot/kernel/if_ix.ko and u'r good to go. (reboot)

    So yeah, it works just fine after that, but don't survive upgrades as far as i understand pfsense updates kernel also, along with modules apparently.

    Instructions:

    on 1st boot after install press any key on PFsense logo

    boot in Single User mode
    press enter for /bin/sh shell

    #connect your USB stick (Fat32 formatted) with driver
    #check what's it's name
    ls /dev/da*
    #mount usb
    mount_msdosfs /dev/da0s1 /media
    #unload existing if_ix module - check it's ID if necessary /sbin/kldstat )
    /sbin/kldunload -i 2
    #mount root as RW
    mount -rw /
    #rename existing if_ix.ko module
    mv /boot/kernel/if_ix.ko /boot/kernel/if_ix.ko.blacklisted
    #copy new module from USB
    cp /media/if_ix.ko /boot/modules/
    #fix permissions
    chmod 555 /boot/modules/if_ix.ko
    #make loader.conf.local
    echo 'if_ix_load="YES"' >> /boot/loader.conf.local
    #enjoy
    reboot

    In case somebody needs it - here's yandex.cloud direct link (compiled for FreeBSD 11.1 release) https://yadi.sk/d/F92wgHSy3RNDS9

    This is  very clear thanks !!!

    Do we know when this is going to be in the live and stable builds ? Its working for now but don't want to put it into production until its part of the default kernel.

    Hopefully its soon before the power supply in the old Dell R300 changes its mind about not being on fire :)



  • @stephenw10:

    The drivers were not in 11.1 release which is what we build on so until they are included in our base we may not get them.

    You can build the kernel module yourself as explained by EditioN above. To load it at boot time just create the file /boot/loader.conf.local and then put in that file:

    if_ix_load="YES"
    

    Steve

    or just i beleve it was ```
    if_ix_updated_load=yes



  • Hey guys,

    The new XG-7100 model includes the Atom C3000.
    I guess this is a good sign that the drivers will be built in soon?

    I must say that I have no issues so far with the driver I compiled but official support would be quite good to have.



  • @MarvinFS:

    Managed to get it working somehow.
    Anyhow, we have bought those appliances already for all the offices, sadly before checking if it's supported yet! :)
    I failed to find a proper way to disable existing module, so i just rename it and load a module manually compiled from Intel sources.
    After that don't forget to fix permissions chmod 555 /boot/kernel/if_ix.ko and u'r good to go. (reboot)

    So yeah, it works just fine after that, but don't survive upgrades as far as i understand pfsense updates kernel also, along with modules apparently.

    Instructions:

    on 1st boot after install press any key on PFsense logo

    boot in Single User mode
    press enter for /bin/sh shell

    #connect your USB stick (Fat32 formatted) with driver
    #check what's it's name
    ls /dev/da*
    #mount usb
    mount_msdosfs /dev/da0s1 /media
    #unload existing if_ix module - check it's ID if necessary /sbin/kldstat )
    /sbin/kldunload -i 2
    #mount root as RW
    mount -rw /
    #rename existing if_ix.ko module
    mv /boot/kernel/if_ix.ko /boot/kernel/if_ix.ko.blacklisted
    #copy new module from USB
    cp /media/if_ix.ko /boot/modules/
    #fix permissions
    chmod 555 /boot/modules/if_ix.ko
    #make loader.conf.local
    echo 'if_ix_load="YES"' >> /boot/loader.conf.local
    #enjoy
    reboot

    In case somebody needs it - here's yandex.cloud direct link (compiled for FreeBSD 11.1 release) https://yadi.sk/d/F92wgHSy3RNDS9

    Hello Everybody

    I would like to chime in and report aswell. I have e SuperMicro SuperServer E200-9A with the new C3558 in it. It also has four X553 NICs which I cannot get running under pfSense. I tried your instructions but they didn't work. Seems like I need to give myself permission somewhere (see attached picture). I set the system up with ZFS but I tried it with UFS and it didn't work aswell. Maybe I'm just missing something?

    Can anybody give me a tip here? Any help would be appreciated.


  • Netgate Administrator

    You don't need unload ix as it's not running as a module.

    It failed to remount / as read-write because it doesn't recognise the file system, you will need to specify it. Though ufs should have worked there. Try:

    mount -rw -t zfs /
    

    It would still be better to wait for a pfSense build based on a FreeBSD version that includes the driver though.

    Steve



  • @stephenw10:

    You don't need unload ix as it's not running as a module.

    It failed to remount / as read-write because it doesn't recognise the file system, you will need to specify it. Though ufs should have worked there. Try:

    mount -rw -t zfs /
    

    Thank you very much for the quick response. Although the command you provided didn't work either. I tried for fat32 and zfs… (see picture)
    Just to clarify: The OS drive on which pfSense is installed is ZFS. The USB-Stick with the if_ix.ko file on it is fat32, like in the instructions.

    @stephenw10:

    It would still be better to wait for a pfSense build based on a FreeBSD version that includes the driver though.

    I hope they are working on this. Although in the bugtracker there is no sign of it. :-\


  • Netgate Administrator

    Ah, yes. ZFS requires more hoop jumping. It's probably easier to reinstall with UFS and try there instead.

    I assume you're doing this because you don't have any functioning NICs? And you can't put one in temporarily?

    Steve



  • Okay. I will try reinstalling with UFS tomorrow, thank you.

    Yes, no NICs detected. I really hope the devs will include support for x553 NICs soon…
    By putting a NIC in temporarily, do you mean something like an USB-NIC? No I don't have any, but i could probably get one easily... Bad thing is, my system doesn't have an external USB 3.0 port. Good thing is, I'm not in a hurry: my current router is still running...

    I will post as soon as i try with UFS install again. BTW, is there any benefits to going with UFS vs. ZFS?

    Ben


  • Netgate Administrator

    UFS is far more widely tested in pfSense, it's still the default file system type for everything except mirrored disk installs. ZFS may well become the default at some point but there's no schedule for that I'm aware of.

    When you face odd issues it's best to keep everything as vanilla as possible for maximum chance someone else will have hit it before you.  ;) At least until things are working as expected.

    Yes a USB NIC or anything else you can put in temporarily just to get fully booted to make file transfers way easier.

    Steve



  • So there was a release for 2.4.3 that contained this information but sadly it doesn't work on CE builds  (factory installations only) mean some custom config they do with Netgate?  Why wouldnt they share this with the Community Edition it just seems dumb…

    https://doc.pfsense.org/index.php/2.4.3_New_Features_and_Changes#Interfaces_.2F_VIPs
    Hardware support for the XG-7100, including:
    C3000 NIC support (factory installations only)
    C3000 SoC support (factory installations only)
    Marvell 88E6190 switch support (factory installations only)

    I tested the 2.4.3 build against my C3558 Motherboard no dice on the X553 driver.


  • Galactic Empire Netgate

    We backported the C3000 support for hardware we're selling, and it's the only hardware that we have properly vetted the code with. FreeBSD is adding C3000 support to 11.2, so it will be more widely tested there on other hardware by FreeBSD users. So, not dumb.



  • Hi Iovr

    Is it stable in the factory installations?


  • Netgate Administrator

    We have not seen any stability issues on the particular hardware in the XG-7100, which is where we've tested it.

    Steve



  • @bendroid I have a ZFS filesystem as well. I had to do

    zfs set readonly=off zroot/ROOT/default
    

    and was able to mv files around. After making the changes, the property can be reset to default using,

    zfs inherit readonly  zroot/ROOT/default
    

    The pool properties out of the box are as follows (for reference),

    NAME   PROPERTY              VALUE                  SOURCE
    zroot  type                  filesystem             -
    zroot  mounted               yes                    -
    zroot  quota                 none                   default
    zroot  reservation           none                   default
    zroot  recordsize            128K                   default
    zroot  mountpoint            /zroot                 local
    zroot  sharenfs              off                    default
    zroot  checksum              on                     default
    zroot  compression           lz4                    local
    zroot  atime                 off                    local
    zroot  devices               on                     default
    zroot  exec                  on                     default
    zroot  setuid                on                     default
    zroot  readonly              off                    default
    zroot  jailed                off                    default
    zroot  snapdir               hidden                 default
    zroot  aclmode               discard                default
    zroot  aclinherit            restricted             default
    zroot  canmount              on                     default
    zroot  xattr                 off                    temporary
    zroot  copies                1                      default
    zroot  version               5                      -
    zroot  utf8only              off                    -
    zroot  normalization         none                   -
    zroot  casesensitivity       sensitive              -
    zroot  vscan                 off                    default
    zroot  nbmand                off                    default
    zroot  sharesmb              off                    default
    zroot  refquota              none                   default
    zroot  refreservation        none                   default
    zroot  primarycache          all                    default
    zroot  secondarycache        all                    default
    zroot  usedbysnapshots       0                      -
    zroot  usedbydataset         88K                    -
    zroot  usedbychildren        399M                   -
    zroot  usedbyrefreservation  0                      -
    zroot  logbias               latency                default
    zroot  dedup                 off                    default
    zroot  mlslabel                                     -
    zroot  sync                  standard               default
    zroot  refcompressratio      1.00x                  -
    zroot  volmode               default                default
    zroot  filesystem_limit      none                   default
    zroot  snapshot_limit        none                   default
    zroot  filesystem_count      none                   default
    zroot  snapshot_count        none                   default
    zroot  redundant_metadata    all                    default
    


  • @trumee Thank you alot, but I circumvented that issue with using developer builds. It's stable anough for my home and you get to try new fancy features. If you don't havea productive environment, I can only recommend going with the developer builds.



  • Did 2.4.4-snapshot already have the driver for X553 NICs of C3000 series from FreeBSD 11-stable/11.2-Release?


  • Netgate Administrator

    Current 2.4.4 snapshots are built on FreeBSD 11.2-RC3. You should test one if you can The more testing they see the faster we can find any issues such as drivers not working with your hardware.

    Steve



  • @abcnew said in Intel X553 NICs (on C3758 SoC) not detected by pfSense 2.4/2.4.1:

    Did 2.4.4-snapshot already have the driver for X553 NICs of C3000 series from FreeBSD 11-stable/11.2-Release?

    Yes, I have just installed the latest Devel Version (pfSense-CE-memstick-2.4.4-DEVELOPMENT-amd64-latest) on a SuperServer 5019A-FTN4 and the network interfaces are working.



  • Is there a safe upgrade path from 2.4.4-snapshot to release version?


  • Netgate Administrator

    Yes, it should be a direct upgrade from a snap to stable once 2.4.4 is released. Just set the repo back to 'latest stable'.

    However if you cannot afford the downtime of reinstalling you perhaps shouldn't be running a snapshot.

    Steve