pfSense Newbie Needing Solarflare Card Help

stephenw10

It still might require their own driver or similar...

stephenw10

Supported devices seem to be these:
https://github.com/pfsense/FreeBSD-src/blob/RELENG_2_6_0/sys/dev/sfxge/common/efx.h#L90

By the sfxge(4) driver anyway.

MacWarrior

@stephenw10 -
pciconf -lv

none2@pci0:59:0:0:	class=0x020000 card=0x800f1924 chip=0x09231924 rev=0x01 hdr=0x00
    vendor     = 'Solarflare Communications'
    device     = 'SFC9140 10/40G Ethernet Controller'
    class      = network
    subclass   = ethernet
none3@pci0:59:0:1:	class=0x020000 card=0x800f1924 chip=0x09231924 rev=0x01 hdr=0x00
    vendor     = 'Solarflare Communications'
    device     = 'SFC9140 10/40G Ethernet Controller'
    class      = network
    subclass   = ethernet
none4@pci0:59:0:2:	class=0x020000 card=0x800f1924 chip=0x09231924 rev=0x01 hdr=0x00
    vendor     = 'Solarflare Communications'
    device     = 'SFC9140 10/40G Ethernet Controller'
    class      = network
    subclass   = ethernet
none5@pci0:59:0:3:	class=0x020000 card=0x800f1924 chip=0x09231924 rev=0x01 hdr=0x00
    vendor     = 'Solarflare Communications'
    device     = 'SFC9140 10/40G Ethernet Controller'
    class      = network
    subclass   = ethernet

ifconfig -vva

igc0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
	description: WAN
	options=e120bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,WOL_MAGIC,VLAN_HWFILTER,RXCSUM_IPV6,TXCSUM_IPV6>
	ether f0:************:77
	inet6 fe80::*************:2f77%igc0 prefixlen 64 scopeid 0x1
	inet 100.*********** netmask 0xffffe000 broadcast 100.************
	media: Ethernet autoselect (1000baseT <full-duplex>)
	status: active
	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
igc1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
	options=e120bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,WOL_MAGIC,VLAN_HWFILTER,RXCSUM_IPV6,TXCSUM_IPV6>
	ether f0:****************:76
	inet6 fe80::***************:2f76%igc1 prefixlen 64 scopeid 0x2
	inet6 fe80::******%igc1 prefixlen 64 scopeid 0x2
	inet 10.********** netmask 0xffffff00 broadcast 10.***********
	media: Ethernet autoselect (1000baseT <full-duplex>)
	status: active
	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
enc0: flags=0<> metric 0 mtu 1536
	groups: enc
	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
	options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
	inet6 ::1 prefixlen 128
	inet6 fe80::1%lo0 prefixlen 64 scopeid 0x4
	inet 127.0.0.1 netmask 0xff000000
	groups: lo
	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
pflog0: flags=100<PROMISC> metric 0 mtu 33160
	groups: pflog
pfsync0: flags=0<> metric 0 mtu 1500
	groups: pfsync

MacWarrior

MacWarrior

@luckman212 No problem. I understand.

MacWarrior

MacWarrior

@stephenw10 - Here is the download site I was at:

https://support-nic.xilinx.com/wp/drivers#

MacWarrior

MacWarrior

@stephenw10 - Looks like a new bootable just came out today.

MW

stephenw10

Hmm well 0923:1924 looks like it should be supported by the sfxge driver in 2.5.2:
https://github.com/pfsense/FreeBSD-src/blob/RELENG_2_5_0/sys/dev/sfxge/common/efx.h#L100
So I would be checking the boot log for errors to see why it doesn't attach.

I don't see any FreeBSD drivers at the Xilinx site and if there were they would need to be compiled and transferred. Much better to use the native drivers if you can.

Steve

MacWarrior

@stephenw10 -

Here are the only errors I'm seeing in the OS Boot Log:

module_register_init: MOD_LOAD (ipw_bss_fw, 0xffffffff8073dd40, 0) error 1
ipw_ibss: You need to read the LICENSE file in /usr/share/doc/legal/intel_ipw.LICENSE.
ipw_ibss: If you agree with the license, set legal.intel_ipw.license_ack=1 in /boot/loader.conf.
module_register_init: MOD_LOAD (ipw_ibss_fw, 0xffffffff8073ddf0, 0) error 1
ipw_monitor: You need to read the LICENSE file in /usr/share/doc/legal/intel_ipw.LICENSE.
ipw_monitor: If you agree with the license, set legal.intel_ipw.license_ack=1 in /boot/loader.conf.
module_register_init: MOD_LOAD (ipw_monitor_fw, 0xffffffff8073dea0, 0) error 1
iwi_bss: You need to read the LICENSE file in /usr/share/doc/legal/intel_iwi.LICENSE.
iwi_bss: If you agree with the license, set legal.intel_iwi.license_ack=1 in /boot/loader.conf.
module_register_init: MOD_LOAD (iwi_bss_fw, 0xffffffff80765730, 0) error 1
iwi_ibss: You need to read the LICENSE file in /usr/share/doc/legal/intel_iwi.LICENSE.
iwi_ibss: If you agree with the license, set legal.intel_iwi.license_ack=1 in /boot/loader.conf.
module_register_init: MOD_LOAD (iwi_ibss_fw, 0xffffffff807657e0, 0) error 1
iwi_monitor: You need to read the LICENSE file in /usr/share/doc/legal/intel_iwi.LICENSE.
iwi_monitor: If you agree with the license, set legal.intel_iwi.license_ack=1 in /boot/loader.conf.
module_register_init: MOD_LOAD (iwi_monitor_fw, 0xffffffff80765890, 0) error 1
wlan: mac acl policy registered
random: entropy device external interface
module_register_init: MOD_LOAD (vesa, 0xffffffff8140c3e0, 0) error 19

And when I search for Solarflare in the log, it doesn't show anything.

Also, I looked at the link you gave me. I have some computer knowledge but not enough to know how to load this onto my pfSense computer. Is it as simple as copying & pasting into the Shell or something else? Thank you for your help.

MacWarrior

stephenw10

Ah, I had thought this was in-kernel already but it looks like it's built as a module. So do this:

[2.5.2-RELEASE][admin@t70.stevew.lan]/root: kldload sfxge

Then check it's loaded with:

[2.5.2-RELEASE][admin@t70.stevew.lan]/root: kldstat 
Id Refs Address                Size Name
 1   18 0xffffffff80200000  3aea720 kernel
 2    1 0xffffffff83ceb000     2e60 smb.ko
 3    3 0xffffffff83cee000     2ef0 smbus.ko
 4    1 0xffffffff83cf2000     61c0 ichsmb.ko
 5    1 0xffffffff83cf9000     1ae8 mdio.ko
 6    1 0xffffffff83f21000     1000 cpuctl.ko
 7    1 0xffffffff83f22000     8cb0 aesni.ko
 8    1 0xffffffff83f2b000      b28 coretemp.ko
 9    1 0xffffffff83f2c000    7d0c0 sfxge.ko

You'll probably see a bunch of other messages as it attaches to the NICs.

If they then appear in ifconfig you can add it as a loader variable so it loads at every boot.

Steve

MacWarrior

@stephenw10 - I believe that was the fix. It is now showing up in my interfaces. I am going to do a few tests over the next couple of days and will let you know for sure.

2 other things I noticed problems with are: the Dashboard under Version always says "Unable to check for updates" and when I log in or select an item from the menu bar through the GUI it takes a very long time for it to log in and/or go to the selection from the menu bar. Sometimes as long as 45-60+ seconds. When I watch it through the terminal, it shows right away that I have logged in though. It happens with all 3 of my web browsers. Chrome, Firefox, Safari.

Thank you,
MacWarrior

stephenw10

Both those things usually indicate some lack of connectivity from pfSense itself.
Make sure it can ping from Diag > Ping but both IP and FQDN.

Steve

MacWarrior

@stephenw10 Hi Steve, I can ping internally but when I ping outside the network, I get "Host "apple.com" did not respond or could not be resolved." Which doesn't make sense to me when I can search and get onto websites.

MW

stephenw10

@macwarrior said in pfSense Newbie Needing Solarflare Card Help:

I can search and get onto websites.

That's from a client behind pfSense not pfSense itself though.

Go to Diag > DNS Lookup and make sure all the configured DNS servers are responding.

Steve

MacWarrior

@stephenw10 Hi Steve, 1st - in reference to adding "kldload sfxge" as a loader variable so it loads at every boot, how do I add that? I searched in the forums and docs and couldn't find how to accomplish that without screwing up my computer.

2nd - here is the strange part to me about the DNS Lookup. I used a 2nd computer I had and loaded it with pfSense with a Spectrum connection directly to the WAN and the 4 DNS Lookups work great. However, when I try it with my main 1st computer with pfSense unit hooked up to a MetroNet connection directly to my WAN, it says "No Response" to all 4 DNS Lookups. The reason I'm scratching my head is I checked every setting from my 2nd computer and compared it to the 1st computer, and it is identical to each other but I get the "No Response" on the 1st computer. Not sure what I am missing here. Thx.

MW

stephenw10

To load it at boot create the file /boot/loader.conf.local then add to it the line:

if_sfxge_load=YES

Try a lookup from the command line directly. First make sure it can ping out:

22.05-DEVELOPMENT][admin@apu.stevew.lan]/root: ping -c 2 8.8.8.8
PING 8.8.8.8 (8.8.8.8): 56 data bytes
64 bytes from 8.8.8.8: icmp_seq=0 ttl=116 time=7.352 ms
64 bytes from 8.8.8.8: icmp_seq=1 ttl=116 time=7.288 ms

--- 8.8.8.8 ping statistics ---
2 packets transmitted, 2 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 7.288/7.320/7.352/0.032 ms

Then try to query it:

[22.05-DEVELOPMENT][admin@apu.stevew.lan]/root: dig @8.8.8.8 google.com

; <<>> DiG 9.16.23 <<>> @8.8.8.8 google.com
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 36555
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 512
;; QUESTION SECTION:
;google.com.			IN	A

;; ANSWER SECTION:
google.com.		67	IN	A	172.217.16.238

;; Query time: 7 msec
;; SERVER: 8.8.8.8#53(8.8.8.8)
;; WHEN: Tue Feb 15 13:01:47 GMT 2022
;; MSG SIZE  rcvd: 55

If that works what errors do you see trying to query the configured servers?

Steve

MacWarrior

@stephenw10 HI Steve, results:

[2.5.2-RELEASE][admin@Birds-of-Prey.macwarriors.lan]/root: ping -c 2 8.8.8.8
PING 8.8.8.8 (8.8.8.8): 56 data bytes
64 bytes from 8.8.8.8: icmp_seq=0 ttl=61 time=13.298 ms
64 bytes from 8.8.8.8: icmp_seq=1 ttl=61 time=14.437 ms

--- 8.8.8.8 ping statistics ---
2 packets transmitted, 2 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 13.298/13.867/14.437/0.569 ms
[2.5.2-RELEASE][admin@Birds-of-Prey.macwarriors.lan]/root: dig @8.8.8.8 google.com

; <<>> DiG 9.16.16 <<>> @8.8.8.8 google.com
; (1 server found)
;; global options: +cmd
;; connection timed out; no servers could be reached

[2.5.2-RELEASE][admin@Birds-of-Prey.macwarriors.lan]/root: ping -c 2 1.1.1.1
PING 1.1.1.1 (1.1.1.1): 56 data bytes
64 bytes from 1.1.1.1: icmp_seq=0 ttl=60 time=34.296 ms
64 bytes from 1.1.1.1: icmp_seq=1 ttl=60 time=32.973 ms

--- 1.1.1.1 ping statistics ---
2 packets transmitted, 2 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 32.973/33.634/34.296/0.662 ms
[2.5.2-RELEASE][admin@Birds-of-Prey.macwarriors.lan]/root: dig @1.1.1.1 cloudflare.com

; <<>> DiG 9.16.16 <<>> @1.1.1.1 cloudflare.com
; (1 server found)
;; global options: +cmd
;; connection timed out; no servers could be reached

MW

MacWarrior

@stephenw10 said in pfSense Newbie Needing Solarflare Card Help:

To load it at boot create the file /boot/loader.conf.local then add to it the line:

Hi again Steve,
I followed your directions above but they still don't load.

MW

stephenw10

@macwarrior

Ah, I'm sorry it's not shown as an interface type for some reason. Change that line to:

sfxge_load=YES

Hmm, so you can ping but not connect with UDP. What firewall rules do you have on that interfaces? Can we see a screenshot?

Steve

MacWarrior

@stephenw10 Hi Steve, I hope you are monitoring right now. After changing the boot/loader file, it has stopped booting up. I have done a fresh install of 2.6 and created the boot/loader file again with the new parameter you gave me and it does the same thing by not booting up. Hmmm.
???

MW

MacWarrior

@stephenw10 I am currently at the Manual Config window after a clean install of 2.6 asking if I want to make any final manual mods.