HP NC523SFP issue

XKali

Hello,

I am currently changing my old system which is running on an x86 device to an x64.
I bought some hardware and did a fresh install of pfsense with version 2.5.1.

I have a lot of problems with a 10G SFP + card: HP NC523SFP
All the information found on the internet talks about the "qlxgb" driver, but they say it is included since 2.4.5p1.

I see the card, it is present in my web interface and I can configure it. As soon as I activate the card, the system displays an impressive number of "failed" lines and takes a long time to load.

On the other side of my fiber, when the server is turned on, it goes to UP but as soon as pfsense is started (it takes a long time because it cannot configure the 10G link), it goes back to status DOWN.

I know this card is working fine because I have tested with Debian and Windows without problems. I admit that I totally block on this point and that I would like help from anyone who has already encountered this problem with this card.

Thank you!

XKali

kiokoman

@xkali
post the log here, there is no way to understand what's happening without it

XKali

@kiokoman

I will send you logs.
As I can see on Github, they added it in pfSense but still not working :
https://github.com/pfsense/FreeBSD-src/pull/28

#ifconfig -vvvm ql0

ql0: flags=8803<UP,BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 9000
        options=80028<VLAN_MTU,JUMBO_MTU,LINKSTATE>
        capabilities=8013b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,TSO4,LINKSTATE>
        ether c4:34:6b:xx:xx:xx
        inet6 fe80::c634:6bff:fecc:c030%ql0 prefixlen 64 tentative scopeid 0x1
        inet 192.168.1.254 netmask 0xffffff00 broadcast 192.168.1.255
        media: Ethernet 10Gbase-SR <full-duplex> (autoselect)
        status: no carrier
        supported media:
                media autoselect
                media 10Gbase-SR mediaopt full-duplex
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>

For exemple with the ql0, an extract ( #dmesg | grep ql0 ) :

ql0: qla_issue_cmd: cmd[0x001b2218] = 0x80000002
ql0: qla_issue_cmd: exit (ret = 0xffffffff)
ql0: qla_get_max_rds: Q8_CMD_RD_MAX_RDS_PER_CNTXT failed
ql0: qla_issue_cmd: cmd[0x001b2218] = 0x80000003
ql0: qla_issue_cmd: exit (ret = 0xffffffff)
ql0: qla_get_max_sds: Q8_CMD_RD_MAX_RDS_PER_CNTXT failed
ql0: qla_issue_cmd: cmd[0x001b2218] = 0x80000004
ql0: qla_issue_cmd: exit (ret = 0xffffffff)
ql0: qla_get_max_rules: Q8_CMD_RD_MAX_RULES_PER_CNTXT failed
ql0: qla_issue_cmd: cmd[0x001b2218] = 0x80000005
ql0: qla_issue_cmd: exit (ret = 0xffffffff)
ql0: qla_get_max_rcv_cntxts: Q8_CMD_RD_MAX_RX_CNTXT failed
ql0: qla_issue_cmd: cmd[0x001b2218] = 0x80000006
ql0: qla_issue_cmd: exit (ret = 0xffffffff)
ql0: qla_get_max_tx_cntxts: Q8_CMD_RD_MAX_TX_CNTXT failed
ql0: qla_issue_cmd: cmd[0x001b2218] = 0x80000018
ql0: qla_issue_cmd: exit (ret = 0xffffffff)
ql0: qla_get_max_mtu: Q8_CMD_RD_MAX_MTU failed
ql0: qla_issue_cmd: cmd[0x001b2218] = 0x80000019
ql0: qla_issue_cmd: exit (ret = 0xffffffff)
ql0: qla_get_max_lro: Q8_CMD_RD_MAX_LRO failed
ql0: qla_issue_cmd: cmd[0x001b2218] = 0x80000016
ql0: qla_issue_cmd: exit (ret = 0xffffffff)
ql0: qla_get_flow_control: Q8_CMD_GET_FLOW_CNTRL failed
ql0: qla_issue_cmd: cmd[0x001b2218] = 0x80000007
ql0: qla_issue_cmd: exit (ret = 0xffffffff)
ql0: qla_init_rcv_cntxt: Q8_CMD_CREATE_RX_CNTXT failed
ql0: qla_fw_cmd: xmit queue full

kiokoman

@xkali
maybe you need to upgrade the firmware
check with

sysctl dev.ql

firmware can be found here

https://support.hpe.com/hpesc/public/km/product/4345884/Product#t=DriversandSoftware&sort=%40hpescuniversaldate%20descending&layout=table&numberOfResults=25&f:@kmswsoftwaretypekey=[swt8000029]

XKali

@kiokoman

#sysctl dev.ql

dev.ql.1.num_sds_rings: 4
dev.ql.1.num_rds_rings: 2
dev.ql.1.free_pkt_thres: 1024
dev.ql.1.snd_pkt_thres: 16
dev.ql.1.rcv_pkt_thres_d: 32
dev.ql.1.rcv_pkt_thres: 128
dev.ql.1.jumbo_replenish: 2
dev.ql.1.std_replenish: 8
dev.ql.1.debug: 0
dev.ql.1.fw_version: 4.20.1.1429931003
dev.ql.1.stats: 0
dev.ql.1.%parent: pci1
dev.ql.1.%pnpinfo: vendor=0x1077 device=0x8020 subvendor=0x103c subdevice=0x3733 class=0x020000
dev.ql.1.%location: slot=0 function=1 dbsf=pci0:1:0:1
dev.ql.1.%driver: ql
dev.ql.1.%desc: Qlogic ISP 80xx PCI CNA Adapter-Ethernet Function v1.1.36
dev.ql.0.wake: 0
dev.ql.0.num_sds_rings: 4
dev.ql.0.num_rds_rings: 2
dev.ql.0.free_pkt_thres: 1024
dev.ql.0.snd_pkt_thres: 16
dev.ql.0.rcv_pkt_thres_d: 32
dev.ql.0.rcv_pkt_thres: 128
dev.ql.0.jumbo_replenish: 2
dev.ql.0.std_replenish: 8
dev.ql.0.debug: 0
dev.ql.0.fw_version: 4.20.1.1429931003
dev.ql.0.stats: 0
dev.ql.0.%parent: pci1
dev.ql.0.%pnpinfo: vendor=0x1077 device=0x8020 subvendor=0x103c subdevice=0x3733 class=0x020000
dev.ql.0.%location: slot=0 function=0 dbsf=pci0:1:0:0 handle=\_SB_.PCI0.RP01.PXSX
dev.ql.0.%driver: ql
dev.ql.0.%desc: Qlogic ISP 80xx PCI CNA Adapter-Ethernet Function v1.1.36
dev.ql.%parent:

I've no idea how to update the firmware.
I don't really understand why a lot of people have this kind of issue and the implementation of driver didn't work with the 2.4.5p1.

Could you may be give tips for firmware update?
The firmware can't be find for FreeBSD on HP website

stephenw10

Update the firmware from Windows or Linux to save hassle. If there is a firmware update available that is. 4.20.1 appears to be current in the files at HPE.

XKali

@stephenw10 said in HP NC523SFP issue:

Update the firmware from Windows or Linux to save hassle. If there is a firmware update available that is. 4.20.1 appears to be current in the files at HPE

I've already the last firmware on this card but I'm not sure about the driver.

kiokoman

@xkali
i see there are some open bugs at freebsd, the workaround is to issue a

ifconfig ql0 MTU 9000

did you try?

stephenw10

Yes definitely worth trying that. The errors don't look quite the same but might be related.

XKali

@kiokoman @stephenw10
I've just tried the command but havec this issue :
#ifconfig ql0 MTU 9000

ifconfig: MTU: bad value

With lowercaps, i've no result in shell, but still not working.
I've tried down / up interface.

Currently, I bought an Intel X520-DA2 for make another test.
I think this is a problem with FreeBSD driver ? Because the qlxgb driver is new ?

Thank's a lot for your help!

XKali

Just for information, I've tried "opnsense" for check if I've the same issue and still not working.
I replace the SSD to get back pfsense (better from my opinion) and the link down just after pfsense load drivers (I think).

kiokoman

@xkali
try to set Speed and Duplex under interfaces to a static value instead of autoselect or default

XKali

@kiokoman
Same result, I'm really lost with this issue...

stephenw10

Does it work with the mtu at 1500?

ifconfig ql0 mtu 1500

XKali

@stephenw10
Same problem unfortunately

XKali

With the Intel X520-DA2, it's work fine and at the first boot with same install.
Maybe, it's a driver issue on pfSense for the HP card.

stephenw10

Yes, probably. Though it will be with the FreeBSD driver. pfSense doesn't have different drivers for most things.

If you can try verifying the behavior in FreeBSD directly.

Steve

XKali

@stephenw10 I receive 2 news servers on Monday, I will do test with FreeBSD.

banting

to those having similar issues, please reinstall pfsense without the network card then copy qlxgb.ko from freebsd set the if_qlxgb_load="YES" in the loader.conf or create your own loader.conf.local and add the line into it, power off then install your network card. that worked for me after having identical issue. still don't understand why it sees the card but will not work my guess is that whatever pfsense does with the card may be broken or not enough to fully activate the card. this is happening in version 2.5.2. anyway hope it helps anyone struggling with this.

stephenw10

So a newer qlxgb driver worked? Which FreeBSD version did you load that from?