Pdsbm-ln2 1U server Lan Ports fail in RC canidate. Using Intel 82573 nic

[Topper727]

In looking around maybe this has something to do with it.

https://downloadcenter.intel.com/Detail_Desc.aspx?agr=Y&DwnldID=17509

They say the last driver dated 4 months ago only supports the 9.x kernals of Freebsd.  Maybe this is why it breaks in 10.1?

[Topper727]

I did not find any DMA stuff in bios.. As far as ACPI I turned that off and pfsense would not boot then.. I turned off shadowing and all kinds of stuff nothing makes a difference.


[Topper727]

Updated to the 2.2 Dec 10th RC version.  Still same, I tried to put in a wifi PCI card in the system and it did same as the dual nic card bv intel I tried also

Seems what ever is messing up em1 from working anything else in the system will not run except USB -Network

The wifi Card I tried  was the wmp54gs pci… It failed and locked like the intel dual nic PCI card did showing this message (see picture)


[stephenw10]

That card should be supported by the bwn or bwi drivers, depending which version you have, but you need the firmware for it and I've seen it give others trouble before. Still shouldn't be hanging there though.  :-
Do you have the ports on the Intel NIC connected and up when booting? I've read some posts saying that makes a difference.

Steve

[Topper727]

Ok did some testing.  The 2 built on the pdsbm motherboard the Intel 82573 will only work with 1 of the 2.. either one but only 1..  And when you do this then nothing WIFI wmp54gs  or dual nic pci card I put in will boot, all lock up.  When I disabled the 2 built on intel 82573 and then booted with  the wifi card it worked..

I then tried the dual 82546 Intel PCI card with the onboard ports off and it worked fine also.

So I guess the newer Intel dual nic chip 82573 has some bug in the driver that the Intel 82546 does not.  I can't put the pci 82546 and wifi in as I don't have enough PCI slots

And pulling all cables from the net ports did not change a thing. Still same results.  If either 1 or both are enabled on the motherboard the PCI cards will only lock up .. wifi and dual pci nic.  .


[Topper727]

I can setup ssh for someone to look around if needed.. I hope to get this working please..

[heper]

have you tried resetting your bios to defaults ? using the jumper on the motherboard to be sure ?
be sure to have the cards in place before you do so.

years ago ( i think pfsense 1.2.3) i've wasted half a day trying to get it to work; after being desperate and giving up i decided to reset the bios and suddenly it all worked
it's a long shot, but can't hurt to try

it looks like an IRQ problem from the old DOS days to me … shouldn't be possible with modern x64 ACPI shitz but have you tried ALL different combination of pci(e) slots ? moving the nics to another slot might just do the trick

[Topper727]

I have updated and reset the bios from 2007 version to a 2010 version.  There is no PCI cards in the system when I test the two on board network ports.  When those are enabled with jumpers nothing in PCI slots work.  Not wifi card or the dual nic I have.  But disable the 2 on board and then I can use PCI slots.  Funny thing is that the two on board are newer chips but still using same driver as the dual nic I have and the dual nic will work fine if both on board are turned off.

[Topper727]

I would even buy gold support from PFSense if someone gets this working for me.

[stephenw10]

Did you try booting in verbose mode to get more info?

I'm guessing this is caused by something which is actually better supported in FreeBSD 10 than it was in 8.3. Some feature, possible something ACPI, is being seen and causing some problem which was previously ignored. You say it didn't boot at all when ACPI was disabled completely. You might try booting with various ACPI disabled instead.

Searching for 'Setup of Shared code failed' is not very helpful, there seem to be many different errors that result in this.

pcib1: failed to allocate initial I/O port window: 0-0xfff
pcib1: failed to allocate initial memory window: 0-0xfffff
pcib1: failed to allocate initial prefetch window: 0-0xfffff

This is obviously bad but it's not obvious quite how it relates to the issue. I would suggest that the BIOS is passing bad values for the PCI device setup which is why I suggested disabling ACPI.

Steve

[stephenw10]

Since you have the NICs that are specifically mentioned in the this bug list (old though), you should probably check the firmware:
@https://wiki.freebsd.org/BugBusting/Commonly_reported_issues:

em(4) watchdog timeouts on Intel 82573 NICs

Reference: http://e1000.sourceforge.net/doku.php?id=known_issues#v_l_e_tx_unit_hang_messages

Fix: DOS-based EEPROM patcher from Intel

Fix: Boot Linux, install ethtool(8), and follow this procedure

Troubleshooting: when using em(4) driver 6.7.3 or later, you can dump the contents of the EEPROM by doing sysctl dev.em.X.debug=2 (0=em0, 1=em1, etc.). You'll see something like this from the kernel:

Interface EEPROM Dump:
        Offset
        0x0000  3000 8f48 d903 0d20 f746 0057 ffff ffff
        0x0010  ffff ffff 026b 109a 15d9 109a 8086 80df
        0x0020  0000 2000 7e54 0000 1000 00da 0004 2700
        0x0030  6cc9 3150 0732 040b 2984 0000 c000 0706

Since the above code prints things in little-endian-stored words, you need to look at the value at offset 0x1f (and not 0x1e like Linux ethtool!), which in the above example is value 0xdf. Bit 0 should be set (1), which disables the power-saving feature that causes the problem. In the above example, the bit is set, which means that the above NIC is not susceptible to the bug

Hmm, this doesn't work for me using a different em interface but the same driver.  :-\

Edit: Hmm, not sure that debug level patch ever made it into the driver.

Steve

[stephenw10]

Ah, has just changed in more recent drivers. You have to run:

[2.2-RC][root@xtm5.localdomain]/root: sysctl dev.em.5.nvm=1
dev.em.5.nvm: -1 -> -1

Then check the output in dmesg (or the system log):

Interface EEPROM Dump:
Offset
0x0000  9000 877f 7adc 0420 f746 1090 ffff ffff 
0x0010  ffff ffff 026b 0000 8086 10d3 ffff 8758 
0x0020  0000 2001 7e74 ffff 1000 00c8 0000 2704 
0x0030  6cc9 3150 073e 460b 2d84 0140 f000 0706

My NIC has address 0x1f, bit 1 set to 0 but it's a different NIC so that's not relevant.

Steve

[stephenw10]

Also you might try the suggestion from https://www.freebsd.org/doc/handbook/acpi-overview.html for acpi related system hangs.
Add:

hint.apic.0.disabled="1"

to /boot/loader.conf.local
See if that allows you to boot with the other Intel card in the box.

Also you could try setting:

hw.acpi.host_mem_start="0xf0000000"

also in /boot/loader.conf.local
That might help with the pci brige allocation issues.

Try those separately though.

Steve

[Topper727]

@stephenw10:

Also you might try the suggestion from https://www.freebsd.org/doc/handbook/acpi-overview.html for acpi related system hangs.
Add:

hint.apic.0.disabled="1"

to /boot/loader.conf.local
See if that allows you to boot with the other Intel card in the box.

Also you could try setting:

hw.acpi.host_mem_start="0xf0000000"

also in /boot/loader.conf.local
That might help with the pci brige allocation issues.

Try those separately though.

Steve

Well careful with the ACPI in loader.conf. I just had done that and system failed to boot like when I changed in bios. I had to reinstall it all.  In doing that I ran out of time before work so the second part can not be done.


[stephenw10]

@Topper727:

Well careful with the ACPI in loader.conf.

Do you mean ACPI or APIC here? Did you try both?

Steve

[Topper727]

hint.apic.0.disabled="1"

Is what I tried. Now I will try the second command you mentioned.. Hope that don't crash me and make me reinstall again..

But not sure why ACPI said it crashed when apic is the command I used. Intersting. Look at the picture..

[Topper727]

hw.acpi.host_mem_start="0xf0000000" done in both loader.conf and loader.conf.local.. I had to create a new loader.conf.local non existed.  But when I did in both the same lock up happened. I did not pull the pci Card out with 2 nics and just try the on board dual nic but I don't think any thing would be different

I really don't want to switch to another firewall..I really like pfsense.

[stephenw10]

Did you try booting verbose yet?
Did you dump the nic eprom?

To avoid re-installing you can enter loader tunables by interrupting the boot loader and entering them manually. Of course they are then lost on the next boot.

Steve

[Topper727]

Verbose mode no
debug mode no

My work schedule keeps me busy.  I will have more time tomorrow as that is a day off for me. Then I will try those.  Really not sure how to do the debug part yet

Also breaking the bootloader to put in variables? Can this be done with USB keyboard? I can't use the keyboard after bios till firewall almost booted.. it need PS2 keyboard to be able to use during the part of first loading and first menu of pfsense

AND if I but in debug mode do I want to turn on the internal nic's and pull the pci I am using now or how you want this test?
and will it keep the logs if you want the log of when it will crash.

[Topper727]

https://bugs.pcbsd.org/issues/4212