Can't assign NIC [Aquantia Corp. AQC107]

madbrain · Mar 20, 2023, 1:35 AM

I'm a newbie to pfSense & FreeBSD.

I realize this is an old topic, but I am having a hell of a time trying to use my two AQN-107 NICs with pfSense 2.6 .
I downloaded the driver from https://pkg.freebsd.org/FreeBSD:12:amd64/latest/All/aquantia-atlantic-kmod-0.0.5_1.pkg .
I installed it with pkg install -f -y .
I loaded it manually with kldload .
I have 2 major problems :

the module doesn't automatically reload upon reboot. How can I force it to do so ?
my clients (off the LAN side, using the pfSense DHCP server) don't have DNS when I use the Aquantia NIC.
If I use other NICs (Realtek 1 Gbps USB, Intel I211 PCIe on the motherboard), I don't have any DNS problem.
This is a really strange one. I have not done any manual configuration for DNS and certainly am not doing anything different when I switch NIC (same IP address for LAN side, same subnet, etc).
Any idea about this one ?
I'd like to make sure I'm using the best possible driver. Is there another build I should be using ? I'm on x64 (AMD Ryzen 2700).

Gertjan · Mar 20, 2023, 6:18 AM

@madbrain said in Can't assign NIC [Aquantia Corp. AQC107]:

the module doesn't automatically reload upon reboot. How can I force it to do

Can answer one : as it has been answered already above :

@rico said in Can't assign NIC [Aquantia Corp. AQC107]:

How the Installation should work (dry run / not tested, I don't own this NIC):
Unzip the file and place if_atlantic.ko into the /boot/kernel/ folder.
Change permissions via console or SSH: chmod 555 /boot/kernel/if_atlantic.ko
Edit /boot/loader.conf.local and add to the end of the file: if_atlantic_load="YES"
Reboot pfSense and see what happens. ;-)

madbrain · Mar 20, 2023, 7:54 AM

@gertjan Thank you ! The file to edit was actually /boot/loader.conf . Not /boot/loader.conf.local .
I copied the driver to /boot/kernel, but not sure if it's really required - it was already copied in /boot/modules , with the correct permission, from the package installer .

In any case, I was able to persist the changes for the WAN & LAN interfaces, and now the second problem (DNS issue) is gone after rebooting.

Gertjan · Mar 20, 2023, 8:14 AM

@madbrain said in Can't assign NIC [Aquantia Corp. AQC107]:

The file to edit was actually /boot/loader.conf . Not /boot/loader.conf.local

Noop.
The file /boot/loader.conf can be rewritten 'at any moment' by pfSense itself, discarding your edits.
Your own additions should be placed in /boot/loader.conf.local, which will not be touched.
If needed, you have to create this file.
This file will also get parsed at kernel boot time.

See, for example Hardware Tuning and Troubleshooting

stephenw10 · Mar 20, 2023, 7:21 PM

Not sure why you wouldn't see DNS there though. Is the service starting correctly?

madbrain · Mar 20, 2023, 7:17 PM

@gertjan said in Can't assign NIC [Aquantia Corp. AQC107]:

@madbrain said in Can't assign NIC [Aquantia Corp. AQC107]:

The file to edit was actually /boot/loader.conf . Not /boot/loader.conf.local

Noop.
The file /boot/loader.conf can be rewritten 'at any moment' by pfSense itself, discarding your edits.
Your own additions should be placed in /boot/loader.conf.local, which will not be touched.
If needed, you have to create this file.
This file will also get parsed at kernel boot time.

See, for example Hardware Tuning and Troubleshooting

Thanks. I stand corrected. I didn't know I was supposed to create the .local file. I just did so and moved the if_atlantic_load declaration to it.

What about the directory for the module ? Is it really supposed to be /boot/kernel ? The package installed it in /boot/modules originally. I made a copy in /boot/kernel, but wondering if it really needs to be in two places.

madbrain · Mar 20, 2023, 7:21 PM

@stephenw10 said in Can't assign NIC [Aquantia Corp. AQC107]:

Not sure why you wouldn't see DNS there though. Is the service starting correctly?

I believe the DNS service was starting properly, yes. In fact, at the console in the shell, DNS worked fine - I could ping internet hosts.
But my client box, connected to the LAN NIC, didn't have DNS.
This happened when was doing a manual kldload on the Aquantia module.

Once I made it persistent and rebooted, the problem with DNS on the client went away, somehow. I don't see a good explanation for this, but it's resolved now.

stephenw10 · Mar 20, 2023, 7:27 PM

Ah, OK. That would almost certainly have been because the interface was down or didn't exist when Unbound started.

madbrain · Mar 20, 2023, 7:34 PM

@stephenw10 said in Can't assign NIC [Aquantia Corp. AQC107]:

Ah, OK. That would almost certainly have been because the interface was down or didn't exist when Unbound started.

That makes sense. Indeed the interface wouldn't have existed at boot time. But after the kldload, I had gone through the menu to reconfigure the interfaces and IP addresses, and still DNS didn't work. Anyway, it's taken care of now, thank you !

onoseto · Apr 25, 2024, 4:34 PM

This was a deep rabbit hole but eventually rewarding!

Attaching the Aquantia driver for latest pfSense (2.7.2) – not that it's a good idea to download random binaries from strangers. ;)

Steps to reproduce successful compilation of net/aquantia-atlantic-kmod for pfSense.

Install Poudriere

git clone https://github.com/freebsd/poudriere.git
cd poudriere
./configure
make
make install
cp /usr/local/etc/poudriere.conf.sample /usr/local/etc/poudriere.conf (check & update)

Download & compile FreeBSD source code at correct version

pkg install git
rm -rf /usr/src
git clone -o freebsd https://git.FreeBSD.org/src.git /usr/src
cd /usr/src
git checkout 0c783a37d5d5
make buildworld -j number of CPU cores

Create jail for compilation

poudriere jail -c -j pf272amd64 -a amd64 -m src=/usr/src

Compile

cd
cat "net/aquantia-atlantic-kmod" > pkglist.txt
poudriere bulk -f pkglist.txt -j pf272amd64

Profit

find your binary at /usr/local/poudriere/data/packages/pf272amd64-default/All/aquantia-atlantic-kmod-0.0.5_3.pkg

aquantia-atlantic-kmod-0.0.5_3_freebsd14.0.zip

onoseto · Apr 25, 2024, 4:41 PM

Oh yeah, also wanted to thank to @Rico who gave me the idea that this might be possible and actually dig it. So happy, that I can now use the sweet 10g NIC, and learned a bunch along the way!

stephenw10 · Apr 25, 2024, 4:48 PM

Nice work!

madbrain · Apr 26, 2024, 10:47 AM

@onoseto Thanks for this great work !

My two AQN-107 NICs are in a drawer in the garage, currently. I have been using an Intel X550-T2 dual-port NIC with pfSense for a year, for both 2.5 Gbps WAN (Comcast) and 10 Gbps LAN.

The X550-T2 is a little bit faster than the Aquantia AQN-107, in addition to having two ports. It lacks WOL support, however, which doesn't matter for pfSense, but does for a desktop. I discovered this when I bought a second X550-T2 for my main Windows desktop to get last tiny bit of performance (max iperf3 on LAN with one stream). WOL no longer worked. I had to enable the motherboard Intel I211 NIC, and turn on WOL on it. I didn't bind any protocol to it (no TCP/IP). It does require an extra ethernet cable, and port on a 1Gb switch, but those are cheap enough.

Anyway, the Aquantia seems to be great with Linux as they have been supported out of the box in the kernel for several years with Ubuntu. Not sure about other distributions. My NAS runs Linux and uses an Aquantia AQN-107 and is able to max out the connection, unlike under Windows.

For pfSense, the AQN-107 driver is regrettably not part of the official distribution. Is there any chance that it will ever make it to the official pfSense distribution ? That would be really welcome for disaster recovery scenarios (drive failure) to be able to just upload the settings to restore, without the need to figure out how to rebuild/download and reinstall the driver.
There are of course workarounds, such as temporarily using the motherboard built-in NIC for either WAN or LAN.

stephenw10 · Apr 26, 2024, 12:49 PM

Likely when the driver is added the FreeBSD tree directly rather than as kmod pkg.

stephenw10 · Jun 13, 2024, 2:17 PM

@onoseto Are you able to re-attach that? Unfortunately attachments to the forum were lost for a short period and that was included.

kouled · Jan 21, 2025, 2:51 PM

Great topic and subject

I was not able to reproduce --> compile the source as there is no C compiler on the pfsense
First we need to install git on the pfsense
pkg install git (this works)
then the ./configure I get an error message :
configure: error: no acceptable C compiler found in $PATH

Any advice to move foward?

Thank you

stephenw10 · Jan 21, 2025, 4:06 PM

Don't try to build anything in pfSense. Instead build in FreeBSD. However for something like a kernel module this is non-trivial as you must build against the same kernel pfSense is using.

kouled · Jan 21, 2025, 4:29 PM

@stephenw10 Thank you I just try in a VM I create with exactly the same version and setting of my pfsense...

My goal is to make my 10G network card working on the pfsense

I have the chance (now, was not the case before) to have 10G internet access

So I 'll try with a freebsd with the same kernel and see

Gertjan · Jan 22, 2025, 6:51 AM

@kouled said in Can't assign NIC [Aquantia Corp. AQC107]:

My goal is to make my 10G network card working on the pfsense

Get a device, or NICs, that is known to work at that speed with pfSense is probably way easier.

kouled · Jan 22, 2025, 10:41 AM

I end up with error <=> I can't compile the driver...

/usr/src/aqtion-freebsd-0.0.5/aq_main.c:197:57: error: too many arguments provided to function-like macro invocation
197 | DRIVER_MODULE(atlantic, pci, aq_driver, aq_devclass, 0, 0);
| ^
/usr/include/sys/bus.h:854:9: note: macro 'DRIVER_MODULE' defined here
854 | #define DRIVER_MODULE(name, busname, driver, evh, arg)
| ^
fatal error: too many errors emitted, stopping now [-ferror-limit=]

In the main the function put 5 arguments in the bus.h that function is only expecting 5 arguments.. mismatch

the code is not working properly

I have no idea how I can run my 10g network card on my pfsense :-(