Pfsense 2.4.0 incompatibility with Mellanox Connectx3
So, just found this out the hard way at work:
2.4.0, more specifically FreeBSD 11.1 - does not support drivers for Mellanox Connectx3, and Mellanox has no plans to provide upgraded drivers to support FreeBSD 11. Rather their support informed us that Connectx4 NICs ARE compatible with FreeBSD 11. No where was this information readily available until after digging into forums after upgrading. One of those "Have to Pass it to see whats in it" type of situations…
This all started with installing Zabbix Agent on a device. Looking at other 2.3.4p1 pFsense devices on our network, nearly all the installed packages, from quagga routing, to syslog and zabbix, are stating they require an update...an update that cannot be performed without first updating pfsense to 2.4.0...which will require swapping out dozens of expensive 10G fiber NICs...
So to the community - think long and hard about your upgrades
To pfsense and the FreedBSD project: C'mon man, really?!
You can configure the system/update/settings branch to stay on 2.3, and update the packages.
As for no longer supporting the required driver on newer FreeBSD OS i, cant help there.. :-\
We have never claimed to support those cards, that's all up to FreeBSD and/or Mellanox, not the pfSense project. We've actively discouraged people from using them for that very reason.
As for packages/updates, go to System > Update, Update Settings tab. Set it to "Security/Errata Only" and then you can stay on 2.3.x.
If you still can't update or see packages after switching to Security/Errata Only, run:
pkg-static install -f pkg pkg install -f pfSense-repo pfSense-upgrade
Several of the packages we needed we zabbix agents and quagga ospf, which it would not allow without the major OS upgrade. We've since moved past. Thanks for the responses.
2.4.0, more specifically FreeBSD 11.1 - does not support drivers for Mellanox Connectx3, and Mellanox has no plans to provide upgraded drivers to support FreeBSD 11.
If the new cards are available and running what is now the real problem? If it is urgent you could buy this cards
and all will be fine for you and your company, or? So this are Infiniband adapters, and this is more or less well
known and used for any kind of SAN or storage networks, for sure in some rarely cases this can be interesting
for admins to get the hands on and together with their switches able to serve 40/56 GBit/s or 25/50 or
50/100 or 100/200 GBit/s it could be nice to get this working well or together with different OS. But
perhaps also the money is rare and a driver not given, you could try out to install pfSense inside of a
VM to get the benefits of that adapters and being able to use pfSense more then other distributions.
Rather their support informed us that Connectx4 NICs ARE compatible with FreeBSD 11.
For sure a generic kernel driver from them would be fixing many things, but for perhaps some rare interested
users they don´t hire a driver coder I really think.