Intel Pro 1000 MT Quad issue (MagicJack ports???)



  • Having an issue with an Intel Pro 1000 MT Quad NIC I just bought - well I bought a lot of them (4 boxes of 5 NICs each brand new so total of 20).

    I am having one of two problems - either my interface assignments are thrown off by adding in the new card (which doesn't happen with any other types of cards (like Intel Pro 1000 PT dual) or there is an issue with this card.

    I have not yet reset or re-assigned the interfaces but I'll try that tomorrow.

    Are there any known issues with these Network cards? It is an Intel Pro 1000 MT Quad NIC. The ports say "MagicJack" on them, not Foxcon like I expected.

    I searched… didn't find anything.

    Pretty much what happens is that when I plug it in and turn it on the machine boots and I don't see any issues - but I can't get a connection to the LAN (or WAN) (or any interface). If I remove the card and reboot there are no issues.



  • @pf2.0nyc:

    I am having one of two problems - either my interface assignments are thrown off by adding in the new card

    Possible. Suppose you have interfaces em0 and em1 in the machine. You add a four port card. Suppose the scan of the PCI device tree finds the nics on the new card before it find the NICs on the card that was in the machine. Then the NICs on the new card become em0, em1, em2 and em3 while the NICs on the old card become em4 and em5. pfSense is configured so LAN and WAN interfaces are em0 and em1 but em0 and em1 are on the new card and not connected anywhere.



  • @wallabybob:

    @pf2.0nyc:

    I am having one of two problems - either my interface assignments are thrown off by adding in the new card

    Possible. Suppose you have interfaces em0 and em1 in the machine. You add a four port card. Suppose the scan of the PCI device tree finds the nics on the new card before it find the NICs on the card that was in the machine. Then the NICs on the new card become em0, em1, em2 and em3 while the NICs on the old card become em4 and em5. pfSense is configured so LAN and WAN interfaces are em0 and em1 but em0 and em1 are on the new card and not connected anywhere.

    Yes I agree, either the PCI scan is locking me out of the devices or the NIC is not compatable.

    Is there any documentation as to the scan of the PCI device tree? When I look at the Assign Interfaces tab the dropdown menu does not match the motherboard's listing of the PCI/PCIe slots. Is it supposed to?

    http://support.dell.com/support/edocs/systems/ws490/en/ug/about.htm

    ^^ motherboard documentation (shows numbered order of slots)

    In the Assign Interfaces dropdown it reads:
    Slot1 - em0, em1 - (dual PCIex8 NIC)
    Slot6 - em2, em3 - (top PCI-X slot which is where i'm trying to install quad-card)
    Slot5 - em4, em5 - (second PCI-X - LAN is em5)
    Slot3 - em6, em7 - (dual PCIex8 NIC)
    Motherboard
    Slot4 - ath0 - (PCI slot with wifi card installed)

    Here is what is installed and type of slot:

    Slot1 - PCIe x8 - Intel Pro 1000 PT Dual
    Slot2 - PCIe x16 - Video
    Slot3 - PCIe x8 - Intel Pro 1000 PT Dual
    Slot4 - PCI - Cisco WiFi card
    Slot5 - PCI-X - Intel Pro 1000 MT Dual
    Slot6 - PCI-X - Intel Pro 1000 MT Dual (Trying to install MT quad here)

    I keep changing variables which doesn't help out my troubleshooting very much and there seems to be no easy way to view interfaces or status from the (monitor) console (have not yet tried to hook up serial cable and/or SSH). I am trying to run this "headless" meaning configure the machine and then pull the video card.

    It looks like its scanning the first PCIe slot (pciex8) then going to the first PCI slot (pci-x) then going down the rest of the PCI slots, then finishing up the PCIe slots, then motherboard (and somehow the wireless or only PCI slot is last).

    Is it:
    Scan slot 1, found interface, assign… scan slot 2, find video card, skip over to PCI-X slots, scan PCI slot #1 (which happens to be mobo slot6 and PCI-X #1), found interface, assign... scan PCI slot #2 (mobo slot5 & PCI-X #2), find interface, assign... scan PCI slot #3 (mobo slot4 & only PCI slot), find wireless/ath0 skip assignment or assign ath0... scan the remaining PCIe slots??

    To the above order, does the presence of a video card (or other such as wifi card) change the order that pfsense scans slots and hardware?

    Is there a way to tell pfsense to scan slots in a certain order? (scan slots 1-6 in that order, motherboard last (or first))

    Thx for help/feedback. This is my home box I'm screwing around with but I need to upgrade a bunch of production boxes with these quad Intel Pro MT NICs if I can get this sorted out.

    EDIT:

    So I changed the LAN over to em0 and was able to verify that motherboard slot1 is scanned and assigned first. I was able to get that quad NIC working which is also good. The only thing that still puzzles me is the order of assignment/scanning.

    When I pulled the video card it scanned slot1 first (so my LAN stayed on the assigned em0) but then it still jumped over the (now empty) slot2 to the PCI slots however this time it scanned slot5 (PCI-X #2) first whereas if I put the video card back into slot2 it'll skip to slot6 (PCI-X #1) first...

    Is there any type of method to this madness?

    Thx.



  • It is an unfortunate fact of life that there is (apparently) no standard for assigning PCI bus numbers in a way that has some relationship with motherboard slot numbers. Hence what a motherboard BIOS does in this regard can vary even from BIOS release to release. Consequently you just have to put up with the fact that FreeBSD device names can change when devices are added to or removed from the system.

    PCI devices in a typical PC are uniquely identified by three numbers: function, device and bus. FreeBSD scans the PCI device tree by increasing function number, then device number, then bus number. Function number is usually fixed on the chip, device number by where a device is connected to its parent bridge, and bus number usually assigned by BIOS. Suppose you have em0 on bus 3, device 5, function 0 and you add another em on bus 3, device 4, function 0, then the new device becomes em0 and the old em0 becomes em1 (if you added just a single port card).



  • What I don't understand is the order - and if there is any logic behind it.

    I'm using the exact same motherboard and firmware (and chipset) on each system - essentially identical hardware across multiple boxes. It appears that even when I transfer the same exact cards from machine to machine they don't pull the same interface assignments.



  • @pf2.0nyc:

    What I don't understand is the order - and if there is any logic behind it.

    The logic is as I have described earlier. As long as you keep looking for something different you'll be disappointed - at least until someone changes the kernel code that walks the PCI device tree.

    @pf2.0nyc:

    I'm using the exact same motherboard and firmware (and chipset) on each system - essentially identical hardware across multiple boxes.

    identical hardware even down to PCB revisions and wiring patches?

    @pf2.0nyc:

    It appears that even when I transfer the same exact cards from machine to machine they don't pull the same interface assignments.

    I don't know what you are expecting - can you give an example?



  • regarding the PCB and wiring patches - never looked & never checked so I have to assume not.

    Regarding transferring the exact cards, I have two machines with identical hardware (motherboards have same production date from Dell, have same BIOS) When I transfer the same exact hardware (like only one set of NICs, HDD, mouse, keyboard, monitor, video card, etc. from one to the next (leaving the old box "barebones") pfsense does not pull the same devices in the same order.

    I use Western Digital HDD utility to do full erase (write zeros to every single LBA) and then place that HDD into machine #1 with configuration above (same hardware same NICs). Install/Build pfsense… write down the hardware assignments...

    Remove EVERYTHING from box #1 (HDD, RAM, CPUs (yes plural), CD/DVD drive, NICs, Video card, Wireless Card)

    Go back and zero-out the HDD (wipe it clean so no trace of prior install)

    Plug everything into box #2

    Build pfsense on box #2 - and I get different HW assignments... WTF??

    Could there really be that much of a difference between two motherboards where the tree would be different? The production dates are idential on both MoBos and the firmware is identical...



  • I guess what I'm getting at is if I knew the order that pfsense would assign devices I could edit a config file prior to changing out the hardware so everything would just "work"…

    Am I wrong?



  • I would expect that if you have two identical motherboards with the same BIOS version, you number ports on multi port cards the same way (e.g. always increasing port number as you go away from the mother board) and you move the cards from one motherboard to another such that the card in slot "n" goes to slot "n" in the new motherboard then the FreeBSD device name of a port on a card would be unchanged.

    As I think I explained earlier, if you add or remove cards the unit numbers of existing ports is likely to change.

    If you have a number of identical systems (same motherboard, same BIOs etc) and you configure them with the same types of cards in the slots (e.g. all mother boards have a 2 port PCI card in slot 2 etc) then I would expect all ports in the same position to have same name across all systems (e.g. port 3 in slot 2 is called em4 on all systems). However this is one caveat: As best I know ALL Intel GigE PCI chips are handled by the em driver. I believe SOME Intel GigE PCIe chips are handled by the em driver and others by the igb driver. Its possible some PCIe GigE cards have a different chips than other PCIe cards with the same number of ports so it may be a particular PCIe 2 port card provides em <n>and em <n+1>while another PCIe 2 port card might provide igb <n>and igb <n+1>and that can affect the naming of ports on other cards.

    If you think you have found a configuration that violates what I have written I'm happy to take a look but I would need at least the output of pfSense shell command pciconf -l -v from both systems and a very clear indication of the difference (e.g. motherboard LAN port is em0 in the first configuration and em9 in the second).</n+1></n></n+1></n>


Log in to reply