Pdsbm-ln2 1U server Lan Ports fail in RC canidate. Using Intel 82573 nic

[Topper727]

Still not giving up, if one works and the driver is in Freebsd then there is some way to get the second port of the Intel 82546 dual nic built on the motherboard.  Below is the command pciconf.  I believe I see the device listed there that is missing

[2.2-BETA][admin@pfSense.localdomain]/root: pciconf -lv
hostb0@pci0:0:0:0:      class=0x060000 card=0xb58015d9 chip=0x29708086 rev=0x02 hdr=0x00
    class      = bridge
    subclass  = HOST-PCI
vgapci0@pci0:0:2:0:    class=0x030000 card=0xb58015d9 chip=0x29728086 rev=0x02 hdr=0x00
    class      = display
    subclass  = VGA
pcib1@pci0:0:28:0:      class=0x060400 card=0xb58015d9 chip=0x27d08086 rev=0x01 hdr=0x01
    class      = bridge
    subclass  = PCI-PCI
pcib2@pci0:0:28:4:      class=0x060400 card=0xb58015d9 chip=0x27e08086 rev=0x01 hdr=0x01
    class      = bridge
    subclass  = PCI-PCI
pcib3@pci0:0:28:5:      class=0x060400 card=0xb58015d9 chip=0x27e28086 rev=0x01 hdr=0x01
    class      = bridge
    subclass  = PCI-PCI
uhci0@pci0:0:29:0:      class=0x0c0300 card=0xb58015d9 chip=0x27c88086 rev=0x01 hdr=0x00
    class      = serial bus
    subclass  = USB
uhci1@pci0:0:29:1:      class=0x0c0300 card=0xb58015d9 chip=0x27c98086 rev=0x01 hdr=0x00
    class      = serial bus
    subclass  = USB
uhci2@pci0:0:29:2:      class=0x0c0300 card=0xb58015d9 chip=0x27ca8086 rev=0x01 hdr=0x00
    class      = serial bus
    subclass  = USB
uhci3@pci0:0:29:3:      class=0x0c0300 card=0xb58015d9 chip=0x27cb8086 rev=0x01 hdr=0x00
    class      = serial bus
    subclass  = USB
ehci0@pci0:0:29:7:      class=0x0c0320 card=0xb58015d9 chip=0x27cc8086 rev=0x01 hdr=0x00
    class      = serial bus
    subclass  = USB
pcib4@pci0:0:30:0:      class=0x060401 card=0xb58015d9 chip=0x244e8086 rev=0xe1 hdr=0x01
    class      = bridge
    subclass  = PCI-PCI
isab0@pci0:0:31:0:      class=0x060100 card=0xb58015d9 chip=0x27b88086 rev=0x01 hdr=0x00
    class      = bridge
    subclass  = PCI-ISA
atapci0@pci0:0:31:1:    class=0x01018a card=0xb58015d9 chip=0x27df8086 rev=0x01 hdr=0x00
    class      = mass storage
    subclass  = ATA
atapci1@pci0:0:31:2:    class=0x01018f card=0xb58015d9 chip=0x27c08086 rev=0x01 hdr=0x00
    class      = mass storage
    subclass  = ATA
none0@pci0:0:31:3:      class=0x0c0500 card=0xb58015d9 chip=0x27da8086 rev=0x01 hdr=0x00
    class      = serial bus
    subclass  = SMBus
none1@pci0:6:0:0:      class=0x020000 card=0x109a15d9 chip=0x109a8086 rev=0x00 hdr=0x00
    class      = network
    subclass  = ethernet
em0@pci0:7:0:0: class=0x020000 card=0x109a15d9 chip=0x109a8086 rev=0x00 hdr=0x00
    class      = network
    subclass  = ethernet

[Topper727]

I have updated the bios from 2007 version to a 2009 latest one they had.  I even played with jumpers to disable the working one and then the other worked.. So both will work but only 1 at a time.

[stephenw10]

Yep that's it but it doesn't help much. Actually reading back through the boot log most of the errors are not directly about the NICs but the buses/bridges the are attached to.
The thing with the most errors is pcib1 ,PCI bridge 1. That appears to be connecting PCI bus 0 to PCI bus 2. These are not necessarily PCI buses as you might think of them, they could be PCIe or AGP or some thing else.
PCI bridge 2 connects PCI bus 0 to PCI bus 6. The first NIC that fails to attach to a driver is on PCI bus 6.
PCI bridge 3 connects PCI bus 0 to PCI bus 7. The NIC that works is on bus 7.
Both these PCI bridges are showing the same allocation errors yet one is functioning just fine.

Those errors, though not good, may be not relevent. Do you see anything like that under 2.1.5?

You could try booting in verbose mode to see if any further errors are given.
I was sure there was on option to do this in 2.2  but I can't find it now  ??? so just add boot_verbose="1" to loader.conf.local.

Steve

[stephenw10]

This looks to have appeared in 9.1:
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=176073

Steve

[stephenw10]

Long shot but try this:

If additionally DMA-Support for VT-d is disabled, it also works with FreeBSD (9.1).

I assume in the BIOS.

Steve

[Topper727]

In looking around maybe this has something to do with it.

https://downloadcenter.intel.com/Detail_Desc.aspx?agr=Y&DwnldID=17509

They say the last driver dated 4 months ago only supports the 9.x kernals of Freebsd.  Maybe this is why it breaks in 10.1?

[Topper727]

I did not find any DMA stuff in bios.. As far as ACPI I turned that off and pfsense would not boot then.. I turned off shadowing and all kinds of stuff nothing makes a difference.


[Topper727]

Updated to the 2.2 Dec 10th RC version.  Still same, I tried to put in a wifi PCI card in the system and it did same as the dual nic card bv intel I tried also

Seems what ever is messing up em1 from working anything else in the system will not run except USB -Network

The wifi Card I tried  was the wmp54gs pci… It failed and locked like the intel dual nic PCI card did showing this message (see picture)


[stephenw10]

That card should be supported by the bwn or bwi drivers, depending which version you have, but you need the firmware for it and I've seen it give others trouble before. Still shouldn't be hanging there though.  :-
Do you have the ports on the Intel NIC connected and up when booting? I've read some posts saying that makes a difference.

Steve

[Topper727]

Ok did some testing.  The 2 built on the pdsbm motherboard the Intel 82573 will only work with 1 of the 2.. either one but only 1..  And when you do this then nothing WIFI wmp54gs  or dual nic pci card I put in will boot, all lock up.  When I disabled the 2 built on intel 82573 and then booted with  the wifi card it worked..

I then tried the dual 82546 Intel PCI card with the onboard ports off and it worked fine also.

So I guess the newer Intel dual nic chip 82573 has some bug in the driver that the Intel 82546 does not.  I can't put the pci 82546 and wifi in as I don't have enough PCI slots

And pulling all cables from the net ports did not change a thing. Still same results.  If either 1 or both are enabled on the motherboard the PCI cards will only lock up .. wifi and dual pci nic.  .


[Topper727]

I can setup ssh for someone to look around if needed.. I hope to get this working please..

[heper]

have you tried resetting your bios to defaults ? using the jumper on the motherboard to be sure ?
be sure to have the cards in place before you do so.

years ago ( i think pfsense 1.2.3) i've wasted half a day trying to get it to work; after being desperate and giving up i decided to reset the bios and suddenly it all worked
it's a long shot, but can't hurt to try

it looks like an IRQ problem from the old DOS days to me … shouldn't be possible with modern x64 ACPI shitz but have you tried ALL different combination of pci(e) slots ? moving the nics to another slot might just do the trick

[Topper727]

I have updated and reset the bios from 2007 version to a 2010 version.  There is no PCI cards in the system when I test the two on board network ports.  When those are enabled with jumpers nothing in PCI slots work.  Not wifi card or the dual nic I have.  But disable the 2 on board and then I can use PCI slots.  Funny thing is that the two on board are newer chips but still using same driver as the dual nic I have and the dual nic will work fine if both on board are turned off.

[Topper727]

I would even buy gold support from PFSense if someone gets this working for me.

[stephenw10]

Did you try booting in verbose mode to get more info?

I'm guessing this is caused by something which is actually better supported in FreeBSD 10 than it was in 8.3. Some feature, possible something ACPI, is being seen and causing some problem which was previously ignored. You say it didn't boot at all when ACPI was disabled completely. You might try booting with various ACPI disabled instead.

Searching for 'Setup of Shared code failed' is not very helpful, there seem to be many different errors that result in this.

pcib1: failed to allocate initial I/O port window: 0-0xfff
pcib1: failed to allocate initial memory window: 0-0xfffff
pcib1: failed to allocate initial prefetch window: 0-0xfffff

This is obviously bad but it's not obvious quite how it relates to the issue. I would suggest that the BIOS is passing bad values for the PCI device setup which is why I suggested disabling ACPI.

Steve

[stephenw10]

Since you have the NICs that are specifically mentioned in the this bug list (old though), you should probably check the firmware:
@https://wiki.freebsd.org/BugBusting/Commonly_reported_issues:

em(4) watchdog timeouts on Intel 82573 NICs

Reference: http://e1000.sourceforge.net/doku.php?id=known_issues#v_l_e_tx_unit_hang_messages

Fix: DOS-based EEPROM patcher from Intel

Fix: Boot Linux, install ethtool(8), and follow this procedure

Troubleshooting: when using em(4) driver 6.7.3 or later, you can dump the contents of the EEPROM by doing sysctl dev.em.X.debug=2 (0=em0, 1=em1, etc.). You'll see something like this from the kernel:

Interface EEPROM Dump:
        Offset
        0x0000  3000 8f48 d903 0d20 f746 0057 ffff ffff
        0x0010  ffff ffff 026b 109a 15d9 109a 8086 80df
        0x0020  0000 2000 7e54 0000 1000 00da 0004 2700
        0x0030  6cc9 3150 0732 040b 2984 0000 c000 0706

Since the above code prints things in little-endian-stored words, you need to look at the value at offset 0x1f (and not 0x1e like Linux ethtool!), which in the above example is value 0xdf. Bit 0 should be set (1), which disables the power-saving feature that causes the problem. In the above example, the bit is set, which means that the above NIC is not susceptible to the bug

Hmm, this doesn't work for me using a different em interface but the same driver.  :-\

Edit: Hmm, not sure that debug level patch ever made it into the driver.

Steve

[stephenw10]

Ah, has just changed in more recent drivers. You have to run:

[2.2-RC][root@xtm5.localdomain]/root: sysctl dev.em.5.nvm=1
dev.em.5.nvm: -1 -> -1

Then check the output in dmesg (or the system log):

Interface EEPROM Dump:
Offset
0x0000  9000 877f 7adc 0420 f746 1090 ffff ffff 
0x0010  ffff ffff 026b 0000 8086 10d3 ffff 8758 
0x0020  0000 2001 7e74 ffff 1000 00c8 0000 2704 
0x0030  6cc9 3150 073e 460b 2d84 0140 f000 0706

My NIC has address 0x1f, bit 1 set to 0 but it's a different NIC so that's not relevant.

Steve

[stephenw10]

Also you might try the suggestion from https://www.freebsd.org/doc/handbook/acpi-overview.html for acpi related system hangs.
Add:

hint.apic.0.disabled="1"

to /boot/loader.conf.local
See if that allows you to boot with the other Intel card in the box.

Also you could try setting:

hw.acpi.host_mem_start="0xf0000000"

also in /boot/loader.conf.local
That might help with the pci brige allocation issues.

Try those separately though.

Steve

[Topper727]

@stephenw10:

Also you might try the suggestion from https://www.freebsd.org/doc/handbook/acpi-overview.html for acpi related system hangs.
Add:

hint.apic.0.disabled="1"

to /boot/loader.conf.local
See if that allows you to boot with the other Intel card in the box.

Also you could try setting:

hw.acpi.host_mem_start="0xf0000000"

also in /boot/loader.conf.local
That might help with the pci brige allocation issues.

Try those separately though.

Steve

Well careful with the ACPI in loader.conf. I just had done that and system failed to boot like when I changed in bios. I had to reinstall it all.  In doing that I ran out of time before work so the second part can not be done.


[stephenw10]

@Topper727:

Well careful with the ACPI in loader.conf.

Do you mean ACPI or APIC here? Did you try both?

Steve