HP Qlogic NC523SFP Not Functional By Default
-
Hello,
I have installed an HP NC523SFP card into my pfsense box.
With a fresh install of the latest version (
2.4.5-RELEASE-p1 (amd64)
built on Tue Jun 02 17:51:17 EDT 2020
FreeBSD 11.3-STABLE
), initial config took a very long time, as each and every error produced by the qLogic card hangs the input to the terminal.Eventually, I got the default setup and I have a copper 1G WAN and LAN functional.
As soon as I enable to the ql0 and/or ql1 interface, I see system errors like this:
Jul 26 08:54:39 kernel arg3 = 0xffffffff
Jul 26 08:54:39 kernel ql0: qla_get_max_mtu: Q8_CMD_RD_MAX_MTU failed
Jul 26 08:54:43 kernel ql0: qla_issue_cmd: cmd[0x001b2218] = 0x80000019
<TRUNCATED>
Jul 26 08:54:43 kernel ql0: qla_issue_cmd: exit (ret = 0xffffffff)
<TRUNCATED>
Jul 26 08:54:43 kernel ql0: qla_get_max_lro: Q8_CMD_RD_MAX_LRO failed
Jul 26 08:54:47 kernel ql0: qla_issue_cmd: cmd[0x001b2218] = 0x80000016
<TRUNCATED>
Jul 26 08:54:47 kernel ql0: qla_issue_cmd: exit (ret = 0xffffffff)
<TRUNCATED>
Jul 26 08:54:47 kernel ql0: qla_get_flow_control: Q8_CMD_GET_FLOW_CNTRL failed
Jul 26 08:54:52 kernel ql0: qla_issue_cmd: cmd[0x001b2218] = 0x80000007
<TRUNCATED>
Jul 26 08:54:52 kernel ql0: qla_issue_cmd: exit (ret = 0xffffffff)
<TRUNCATED>
Jul 26 08:54:52 kernel ql0: qla_init_rcv_cntxt: Q8_CMD_CREATE_RX_CNTXT failed
Jul 26 08:54:56 kernel ql0: qla_issue_cmd: cmd[0x001b2218] = 0x80000012
<TRUNCATED>
Jul 26 08:54:56 kernel ql0: qla_issue_cmd: exit (ret = 0xffffffff)
<TRUNCATED>
Jul 26 08:54:56 kernel ql0: qla_set_max_mtu: Q8_CMD_RD_MAX_MTU failedI have tried to set the MTU manually for that interface prior to enabling it, using the web gui, to 9000. This has no effect, but I figured I'd try it as I saw it on another (old) post about this driver.
Does anyone know how to get the qlxgb driver to work properly?
As of this release, it's supposed to be built in with support in the kernel.Thanks,
David -
https://forum.netgate.com/topic/139931/hp-qlogic-nc523sfp-driver-install-freebsd-11-pfsense-kernal-recompile
-Rico
-
@Rico
Thanks for the reply.The thread you linked to is the one I was referring to in my post. I have read through it a few times but I did not see anything that would help there.
Did I miss something? With the driver being integrated into the latest release, I thought it was no longer necessary to copy the freebsd ko file.
I have tried to add that driver to the autoload config just in case that matters, although the driver was obviously loaded because the interfaces came up and attempted to form a link. (I don't remember at the moment what the file name is, but it was in that thread.) I'm responding from my phone right now but I can double check later.
-
Update:
I revisited that other thread and realized that while I was asleep there was some activity.
However, my problem has gone away, as I'll explain:
I discovered that the XFP module on the other end was 1310nm rather than 850 (my SFP+ is 850nm). Obviously this is an issue. I think that while the card bios was booting up, it got upset about this mismatch. It either doesn't like booting without a link, or it gets mad when you hook up the wrong light to the SFP+. Either way, I fixed that problem and tested it in another machine with a correct 850nm source, and the link formed right away. There were no errors while booting up, and the card appeared to work in pfsense. I was not able to do any testing on it beyond that, but I will when it goes into the final machine.
I will come back and update this thread if I learn anything else.
-
Some final updates before I leave this thread alone...
The thread linked to by Rico has been undergoing lots of activity and is very useful (readers of this thread should check it out).
I have had further problems with the qlogic driver, but they are now gone. Basically, once you configure the MTU manually on the interface, the issue goes away.
The first time you do it, you have to wait for multi-user mode to happen, then open up the second virtual console (Ctrl+alt+f2), login, and go to a shell (option 8). In the shell, bring the interface down:
ifconfig ql0 down
Then set the MTU. You can set it to 1500 or 9000, depending on your preferences. Most likely you'll want it at 9000 eventually.
ifconfig ql0 mtu 9000
ifconfig ql0 upThis should get rid of the console going crazy and allow the interface to work. Then you can get into the web interface and configure the MTU for that interface via the web gui. Don't forget to do this step.
Unfortunately, I can't get anywhere near 10gbps performance on this card. The best I've been able to do after lots of tuning (sysctl, tunables, loader.conf.local) is 4gbps, which only worked for a bit and then it went back. It's difficult to figure out why; it's not a card issue probably. It's most likely the hardware and the config (as well as the other end of the connection). I've got it connected to a 10G module in a Brocade FLS624.
There is a known freeBSD bug that causes the MTU issue - I expect it will get patched in pfsense before long. For now, the above fix should work.
