New 1u build - NIC not working?



  • Hey folks,
    My previous PFsense install was in a virtual machine running on ESXi. It has performed flawlessly but I have been looking for a new project so I ordered the parts for a 1u build to put in my rack at home. If anyone is interested I can provide a complete parts list and pictures. For now though, I have an issue with the Intel dual port server NIC I bought off of Ebay. Because this is a 1u build I had to run a solid PCIe riser card to get the intel NIC in the case. This is the piece I used: http://www.newegg.ca/Product/Product.aspx?Item=N82E16816129055CVF.
    I can get PFsense loaded and everything seems to be working but I can't get a connection on the intel NIC (either port). The system has 3 NICS (dual port intel and one integrated into the motherboard) but the only one that works is the one that is integrated into the motherboard. I can connect the integrated NIC into either the WAN or LAN side and get a working connection but I can't even ping the Intel NICS.

    I am thinking this is because:
    The NIC is shot
    I am using a 16x PCIe riser card with a PCIe 4x NIC.
    The riser card is bad

    Has anyone had any experience with this sort of thing?
    FWIW: The system is using a 2.5" 250GB SATA HDD, 4GB DDR3 1066 RAM, Celeron 550 SB, & Foxconn mini itx motherboard.



  • What version of pfSense? (Perhaps your NICs are too new for the software.)

    What NICs?

    Perhaps the NICs are broken, cable broken, not correctly plugged in etc etc.

    please post output of pfSense shell shell commands```

    pciconf -l -v

    ifconfig -a



  • I was able to connect my PC to the integrated NIC and access the web console and enable SSH. The same cable won't work on the Dell X3959 PRO/1000 Dual Port Server NIC. Here are the results of the commands. I manually set the IP addresses so I could test the cables.

    em0@pci0:1:0:0: class=0x020000 card=0x135e8086 chip=0x105e8086 rev=0x06 hdr=0x00
        class      = network
        subclass   = ethernet
    em1@pci0:1:0:1: class=0x020000 card=0x135e8086 chip=0x105e8086 rev=0x06 hdr=0x00
        class      = network
        subclass   = ethernet
    re0@pci0:3:0:0: class=0x020000 card=0x0d7a105b chip=0x816810ec rev=0x02 hdr=0x00
        class      = network
        subclass   = ethernet
    [2.0.1-RELEASE][admin@pfSense.localdomain]/root(2): ifconfig -a
    em0: flags=8843 <up,broadcast,running,simplex,multicast>metric 0 mtu 1500
            options=9b <rxcsum,txcsum,vlan_mtu,vlan_hwtagging,vlan_hwcsum>ether 00:15:17:76:38:c4
            inet6 fe80::215:17ff:fe76:38c4%em0 prefixlen 64 scopeid 0x1
            inet 10.0.0.60 netmask 0xffffffff broadcast 10.0.0.60
            nd6 options=3 <performnud,accept_rtadv>media: Ethernet autoselect
            status: no carrier
    em1: flags=8843 <up,broadcast,running,simplex,multicast>metric 0 mtu 1500
            options=19b <rxcsum,txcsum,vlan_mtu,vlan_hwtagging,vlan_hwcsum,tso4>ether 00:15:17:76:38:c5
            inet6 fe80::215:17ff:fe76:38c5%em1 prefixlen 64 scopeid 0x2
            nd6 options=3 <performnud,accept_rtadv>media: Ethernet autoselect
            status: no carrier
    re0: flags=8843 <up,broadcast,running,simplex,multicast>metric 0 mtu 1500
            options=389b <rxcsum,txcsum,vlan_mtu,vlan_hwtagging,vlan_hwcsum,wol_ucast,wol_mcast,wol_magic>ether d0:27:88:d2:93:0e
            inet6 fe80::d227:88ff:fed2:930e%re0 prefixlen 64 scopeid 0x3
            inet 10.0.0.50 netmask 0xffffff00 broadcast 10.0.0.255
            nd6 options=3 <performnud,accept_rtadv>media: Ethernet autoselect (1000baseT <full-duplex>)
            status: active
    pflog0: flags=100 <promisc>metric 0 mtu 33664
    pfsync0: flags=0<> metric 0 mtu 1460
            syncpeer: 224.0.0.240 maxupd: 128 syncok: 1
    enc0: flags=0<> metric 0 mtu 1536
    lo0: flags=8049 <up,loopback,running,multicast>metric 0 mtu 16384
            options=3 <rxcsum,txcsum>inet 127.0.0.1 netmask 0xff000000
            inet6 ::1 prefixlen 128
            inet6 fe80::1%lo0 prefixlen 64 scopeid 0x7
            nd6 options=3 <performnud,accept_rtadv>re0_vlan35: flags=8843 <up,broadcast,running,simplex,multicast>metric 0 mtu 1500
            options=3 <rxcsum,txcsum>ether d0:27:88:d2:93:0e
            inet6 fe80::215:17ff:fe76:38c4%re0_vlan35 prefixlen 64 scopeid 0x8
            nd6 options=3 <performnud,accept_rtadv>media: Ethernet autoselect (1000baseT <full-duplex>)
            status: active
            vlan: 35 parent interface: re0</full-duplex></performnud,accept_rtadv></rxcsum,txcsum></up,broadcast,running,simplex,multicast></performnud,accept_rtadv></rxcsum,txcsum></up,loopback,running,multicast></promisc></full-duplex></performnud,accept_rtadv></rxcsum,txcsum,vlan_mtu,vlan_hwtagging,vlan_hwcsum,wol_ucast,wol_mcast,wol_magic></up,broadcast,running,simplex,multicast></performnud,accept_rtadv></rxcsum,txcsum,vlan_mtu,vlan_hwtagging,vlan_hwcsum,tso4></up,broadcast,running,simplex,multicast></performnud,accept_rtadv></rxcsum,txcsum,vlan_mtu,vlan_hwtagging,vlan_hwcsum></up,broadcast,running,simplex,multicast> 
    

  • Netgate Administrator

    Your subnet mask on em0 is /32 so it cannot reply to pings. It's also in the same subnet as re0 so that's gonna screw with routing.
    Change it to, for example, 10.0.10.60/24.

    When you ran those commands what did you have connected to each NIC?

    Steve



  • I had a cable going from my PC to the re0 port on the new box and nothing connected to the ports on the Intel NIC. I'll change the subnet mask and try again. Both NICs being on the same subnet was just a test. I have tried under other configurations.



  • I changed the subnet mask to /24 and still no luck. The next steps are to test the NIC in another machine and test another NIC in this machine. Hopefully I'll come up with something.


  • Netgate Administrator

    Are getting any link/speed leds on the Intel NICs?
    A good test would be to connect them both up to a switch or something and run ifconfig again. Look to see if the status is active and that the media type has negotiated correctly.

    Steve



  • Both indicator lights on the ports are active with a cable plugged in. The NIC is set to auto negotiate but it looks like they aren't negotiating properly and they don't activate. I will continue testing.



  • If you only changed the mask, you didn't fix the big underlying problem there - you can't have two NICs on the same subnet. That's at least part of your problem if not all of it.



  • @KM:

    it looks like they aren't negotiating properly and they don't activate.

    I don't know what you mean by "don't activate". status shown by ifconfig doesn't go to Active?

    You won't get any communication through the NIC until the NIC status changes to active.



  • I changed the IP addresses so they are all on different subnets. What I mean by they don't activate is that the status in ifconfig doesn't change to active. I'm going to try a known working NIC in the box to verify but I think the NIC is just bad. I'll let you know what I find out.



  • I tested another NIC and it worked so I ordered another dual port server nic. Thanks for all the help folks!



  • So, I ordered another dual port server NIC (Intel Pro/1000 PCIe x4) and tried to get it working to no avail. Now I have two dual port NICs (not a problem because I can always find a use for these things) that aren't working in this box but a single port Intel desktop NIC that will. I would really like this box to have two LAN interfaces so I can do some DMZ stuff but at this point I may have to run a single port.

    Any suggestions?



  • @KM:

    I . . .  tried to get it working to no avail.

    What did you try and what was the outcome?



  • I purchased a new one and plugged it the same way as the first one. The cards are both detected but there is no connection.



  • Also, I should mention that I have since been able to get the card working with Ubuntu server which can detect and use all three interfaces (built in motherboard NIC).



  • @KM:

    Also, I should mention that I have since been able to get the card working with Ubuntu server which can detect and use all three interfaces (built in motherboard NIC).

    Please post the output of pfSense shell commands```

    ifconfig ; /etc/rc.banner

    
    I presume you have fixed the configuration problem previously discussed.
    
    Please give more information than "there is no connection". What did you attempt? What was reported? What did you expect to be reported?


  • Sorry I haven't replied sooner.. work has been pushing for crazy overtime. I did fix the previous config issue but in the end a reinstall of pfSense fixed the issue. I have no idea why but I have a working box now. I'll post pics of the build when a get a minute to breath.

    Thanks for all your help everyone!


Locked