Pdsbm-ln2 1U server Lan Ports fail in RC canidate. Using Intel 82573 nic

[Topper727]

Posted by: stephenw10
« on: Today at 07:46:29 am » Insert Quote
Try disabling msi/msi-x:
https://doc.pfsense.org/index.php/Tuning_and_Troubleshooting_Network_Cards#MSI.2FMSIX

Tried this and nothing changed at all.. still only 1 built on lan port works.  This is not too much of problem right at moment cause I am using not fast internet so my USB - LAN 100 works ok for the slow speed.  But I plan to get much faster and I need the 1gig built on to work

[stephenw10]

Did you have that enabled during the above boot? I still see:

em0: Using an MSI interrupt

Steve

[Topper727]

yes, I had that still the default on and after sending that log then I changed it in crossed fingers it might work.  but did not

[stephenw10]

@charliem:

Not sure if that patch ever made it into 9.3 or 10.1

Looks like it's in 10.1:
https://svnweb.freebsd.org/base/release/10.1.0/sys/dev/acpica/acpi_pcib_acpi.c?revision=274417&view=markup#l210

Steve

[Topper727]

So wonder how to get this dual nic to show 2 nics not just 1 in pfsense 2.2?

[stephenw10]

Hard to believe you're the first person to run into this. More Googling required.
Since this appears to be a resource allocation problem it could be a bios bug. You could try disabling acpi or/and disabling any devices you're not using, sound card, parallel port etc.

Steve

[Topper727]

Still not giving up, if one works and the driver is in Freebsd then there is some way to get the second port of the Intel 82546 dual nic built on the motherboard.  Below is the command pciconf.  I believe I see the device listed there that is missing

[2.2-BETA][admin@pfSense.localdomain]/root: pciconf -lv
hostb0@pci0:0:0:0:      class=0x060000 card=0xb58015d9 chip=0x29708086 rev=0x02 hdr=0x00
    class      = bridge
    subclass  = HOST-PCI
vgapci0@pci0:0:2:0:    class=0x030000 card=0xb58015d9 chip=0x29728086 rev=0x02 hdr=0x00
    class      = display
    subclass  = VGA
pcib1@pci0:0:28:0:      class=0x060400 card=0xb58015d9 chip=0x27d08086 rev=0x01 hdr=0x01
    class      = bridge
    subclass  = PCI-PCI
pcib2@pci0:0:28:4:      class=0x060400 card=0xb58015d9 chip=0x27e08086 rev=0x01 hdr=0x01
    class      = bridge
    subclass  = PCI-PCI
pcib3@pci0:0:28:5:      class=0x060400 card=0xb58015d9 chip=0x27e28086 rev=0x01 hdr=0x01
    class      = bridge
    subclass  = PCI-PCI
uhci0@pci0:0:29:0:      class=0x0c0300 card=0xb58015d9 chip=0x27c88086 rev=0x01 hdr=0x00
    class      = serial bus
    subclass  = USB
uhci1@pci0:0:29:1:      class=0x0c0300 card=0xb58015d9 chip=0x27c98086 rev=0x01 hdr=0x00
    class      = serial bus
    subclass  = USB
uhci2@pci0:0:29:2:      class=0x0c0300 card=0xb58015d9 chip=0x27ca8086 rev=0x01 hdr=0x00
    class      = serial bus
    subclass  = USB
uhci3@pci0:0:29:3:      class=0x0c0300 card=0xb58015d9 chip=0x27cb8086 rev=0x01 hdr=0x00
    class      = serial bus
    subclass  = USB
ehci0@pci0:0:29:7:      class=0x0c0320 card=0xb58015d9 chip=0x27cc8086 rev=0x01 hdr=0x00
    class      = serial bus
    subclass  = USB
pcib4@pci0:0:30:0:      class=0x060401 card=0xb58015d9 chip=0x244e8086 rev=0xe1 hdr=0x01
    class      = bridge
    subclass  = PCI-PCI
isab0@pci0:0:31:0:      class=0x060100 card=0xb58015d9 chip=0x27b88086 rev=0x01 hdr=0x00
    class      = bridge
    subclass  = PCI-ISA
atapci0@pci0:0:31:1:    class=0x01018a card=0xb58015d9 chip=0x27df8086 rev=0x01 hdr=0x00
    class      = mass storage
    subclass  = ATA
atapci1@pci0:0:31:2:    class=0x01018f card=0xb58015d9 chip=0x27c08086 rev=0x01 hdr=0x00
    class      = mass storage
    subclass  = ATA
none0@pci0:0:31:3:      class=0x0c0500 card=0xb58015d9 chip=0x27da8086 rev=0x01 hdr=0x00
    class      = serial bus
    subclass  = SMBus
none1@pci0:6:0:0:      class=0x020000 card=0x109a15d9 chip=0x109a8086 rev=0x00 hdr=0x00
    class      = network
    subclass  = ethernet
em0@pci0:7:0:0: class=0x020000 card=0x109a15d9 chip=0x109a8086 rev=0x00 hdr=0x00
    class      = network
    subclass  = ethernet

[Topper727]

I have updated the bios from 2007 version to a 2009 latest one they had.  I even played with jumpers to disable the working one and then the other worked.. So both will work but only 1 at a time.

[stephenw10]

Yep that's it but it doesn't help much. Actually reading back through the boot log most of the errors are not directly about the NICs but the buses/bridges the are attached to.
The thing with the most errors is pcib1 ,PCI bridge 1. That appears to be connecting PCI bus 0 to PCI bus 2. These are not necessarily PCI buses as you might think of them, they could be PCIe or AGP or some thing else.
PCI bridge 2 connects PCI bus 0 to PCI bus 6. The first NIC that fails to attach to a driver is on PCI bus 6.
PCI bridge 3 connects PCI bus 0 to PCI bus 7. The NIC that works is on bus 7.
Both these PCI bridges are showing the same allocation errors yet one is functioning just fine.

Those errors, though not good, may be not relevent. Do you see anything like that under 2.1.5?

You could try booting in verbose mode to see if any further errors are given.
I was sure there was on option to do this in 2.2  but I can't find it now  ??? so just add boot_verbose="1" to loader.conf.local.

Steve

[stephenw10]

This looks to have appeared in 9.1:
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=176073

Steve

[stephenw10]

Long shot but try this:

If additionally DMA-Support for VT-d is disabled, it also works with FreeBSD (9.1).

I assume in the BIOS.

Steve

[Topper727]

In looking around maybe this has something to do with it.

https://downloadcenter.intel.com/Detail_Desc.aspx?agr=Y&DwnldID=17509

They say the last driver dated 4 months ago only supports the 9.x kernals of Freebsd.  Maybe this is why it breaks in 10.1?

[Topper727]

I did not find any DMA stuff in bios.. As far as ACPI I turned that off and pfsense would not boot then.. I turned off shadowing and all kinds of stuff nothing makes a difference.


[Topper727]

Updated to the 2.2 Dec 10th RC version.  Still same, I tried to put in a wifi PCI card in the system and it did same as the dual nic card bv intel I tried also

Seems what ever is messing up em1 from working anything else in the system will not run except USB -Network

The wifi Card I tried  was the wmp54gs pci… It failed and locked like the intel dual nic PCI card did showing this message (see picture)


[stephenw10]

That card should be supported by the bwn or bwi drivers, depending which version you have, but you need the firmware for it and I've seen it give others trouble before. Still shouldn't be hanging there though.  :-
Do you have the ports on the Intel NIC connected and up when booting? I've read some posts saying that makes a difference.

Steve

[Topper727]

Ok did some testing.  The 2 built on the pdsbm motherboard the Intel 82573 will only work with 1 of the 2.. either one but only 1..  And when you do this then nothing WIFI wmp54gs  or dual nic pci card I put in will boot, all lock up.  When I disabled the 2 built on intel 82573 and then booted with  the wifi card it worked..

I then tried the dual 82546 Intel PCI card with the onboard ports off and it worked fine also.

So I guess the newer Intel dual nic chip 82573 has some bug in the driver that the Intel 82546 does not.  I can't put the pci 82546 and wifi in as I don't have enough PCI slots

And pulling all cables from the net ports did not change a thing. Still same results.  If either 1 or both are enabled on the motherboard the PCI cards will only lock up .. wifi and dual pci nic.  .


[Topper727]

I can setup ssh for someone to look around if needed.. I hope to get this working please..

[heper]

have you tried resetting your bios to defaults ? using the jumper on the motherboard to be sure ?
be sure to have the cards in place before you do so.

years ago ( i think pfsense 1.2.3) i've wasted half a day trying to get it to work; after being desperate and giving up i decided to reset the bios and suddenly it all worked
it's a long shot, but can't hurt to try

it looks like an IRQ problem from the old DOS days to me … shouldn't be possible with modern x64 ACPI shitz but have you tried ALL different combination of pci(e) slots ? moving the nics to another slot might just do the trick

[Topper727]

I have updated and reset the bios from 2007 version to a 2010 version.  There is no PCI cards in the system when I test the two on board network ports.  When those are enabled with jumpers nothing in PCI slots work.  Not wifi card or the dual nic I have.  But disable the 2 on board and then I can use PCI slots.  Funny thing is that the two on board are newer chips but still using same driver as the dual nic I have and the dual nic will work fine if both on board are turned off.

[Topper727]

I would even buy gold support from PFSense if someone gets this working for me.