New Netgate 6100 Appliance strange problem

brians

@stephenw10 Yes only on LAN2... I tried another laptop and did not recognize. I adjusted settings in pfsense on LAN2 to force 1Gbps still nothing. The other laptop was also an HP so maybe another brand with another NIC may work.

Considering the OP already did an RMA with 6100 and same issue, this may be some more widespread issue.

stephenw10

I think the same unit was returned as we were unable to duplicate the issue.
Really odd. I can't think why it would only effect one port...

VisionSoul

@brians Not HP related, i mean, mine was with a Lenovo T430 and T480, and both with same behaviour. Only works connected to a Switch. Other ports are just fine. My unit was sent to RMA, but was returned "with no problem" because they connected to a switch and was fine. However connected to a computer does not work ( they did not tested this method, but instead plugged directly to a switch ). So for them the unit is fine for all intents and purposes.

Best Regards!

cdturri

@visionsoul Another data point. New 6100. Connected my MacBook Pro directly to the 6100 using an Anker USB type ethernet adapter. It worked fine on all ports including LAN2.

VisionSoul

@cdturri Hi,

Nice to ear. I may try that too, because i only tried connected directly to ethernet port. I'll try via HUB asap.

Thanks for your feedback!

Best Regards

mOrbo

Hi there,

I got the same problem now.

It is a new 6100. But in my case, it's LAN1 (igc0) which is not working.

Connection a Laptop directly to LAN1 does not work. There is no Link at all (no flashing lights at PFSense and no Link-Status at the Laptop).
Connection a Laptop to LAN2 (igc1) does indeed work. After configuring an IP-Address on Port 2 (igc1), I'm also able to open the Webinterface.

My first guess was, LAN1 is broken. I testet this with multiple Laptops. But after connecting LAN1 to my Switch, it immediately gets a LINK.

You can see this if I do a "ifconfig -vvvma"

Laptop connected to Port1:

igc0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        description: OPT1
        options=e120bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,WOL_MAGIC,VLAN_HWFILTER,RXCSUM_IPV6,TXCSUM_IPV6>
        capabilities=f53fbb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,WOL_UCAST,WOL_MCAST,WOL_MAGIC,VLAN_HWFILTER,VLAN_HWTSO,NETMAP,RXCSUM_IPV6,TXCSUM_IPV6>
        ether 90:ec:77:35:ea:ae
        inet6 fe80::92ec:77ff:fe35:eaae%igc0 prefixlen 64 scopeid 0x1
        media: Ethernet autoselect
        status: no carrier
        supported media:
                media autoselect
                media 2500Base-T
                media 1000baseT
                media 1000baseT mediaopt full-duplex
                media 100baseTX mediaopt full-duplex
                media 100baseTX
                media 10baseT/UTP mediaopt full-duplex
                media 10baseT/UTP
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>

Switch connected to Port1:

igc0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        description: OPT1
        options=e120bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,WOL_MAGIC,VLAN_HWFILTER,RXCSUM_IPV6,TXCSUM_IPV6>
        capabilities=f53fbb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,WOL_UCAST,WOL_MCAST,WOL_MAGIC,VLAN_HWFILTER,VLAN_HWTSO,NETMAP,RXCSUM_IPV6,TXCSUM_IPV6>
        ether 90:ec:77:35:ea:ae
        inet6 fe80::92ec:77ff:fe35:eaae%igc0 prefixlen 64 scopeid 0x1
        media: Ethernet autoselect (1000baseT <full-duplex>)
        status: active
        supported media:
                media autoselect
                media 2500Base-T
                media 1000baseT
                media 1000baseT mediaopt full-duplex
                media 100baseTX mediaopt full-duplex
                media 100baseTX
                media 10baseT/UTP mediaopt full-duplex
                media 10baseT/UTP
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>

Nothing on Port1 and Laptop on Port2:

igc0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        description: OPT1
        options=e120bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,WOL_MAGIC,VLAN_HWFILTER,RXCSUM_IPV6,TXCSUM_IPV6>
        capabilities=f53fbb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,WOL_UCAST,WOL_MCAST,WOL_MAGIC,VLAN_HWFILTER,VLAN_HWTSO,NETMAP,RXCSUM_IPV6,TXCSUM_IPV6>
        ether 90:ec:77:35:ea:ae
        inet6 fe80::92ec:77ff:fe35:eaae%igc0 prefixlen 64 scopeid 0x1
        media: Ethernet autoselect
        status: no carrier
        supported media:
                media autoselect
                media 2500Base-T
                media 1000baseT
                media 1000baseT mediaopt full-duplex
                media 100baseTX mediaopt full-duplex
                media 100baseTX
                media 10baseT/UTP mediaopt full-duplex
                media 10baseT/UTP
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
		
igc1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        description: LAN2
        options=e120bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,WOL_MAGIC,VLAN_HWFILTER,RXCSUM_IPV6,TXCSUM_IPV6>
        capabilities=f53fbb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,WOL_UCAST,WOL_MCAST,WOL_MAGIC,VLAN_HWFILTER,VLAN_HWTSO,NETMAP,RXCSUM_IPV6,TXCSUM_IPV6>
        ether 90:ec:77:35:ea:af
        inet6 fe80::92ec:77ff:fe35:eaaf%igc1 prefixlen 64 scopeid 0x2
        inet 192.168.1.1 netmask 0xffffff00 broadcast 192.168.1.255
        media: Ethernet autoselect (1000baseT <full-duplex>)
        status: active
        supported media:
                media autoselect
                media 2500Base-T
                media 1000baseT
                media 1000baseT mediaopt full-duplex
                media 100baseTX mediaopt full-duplex
                media 100baseTX
                media 10baseT/UTP mediaopt full-duplex
                media 10baseT/UTP
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>

Very strange?

VisionSoul

@morbo Hi,

If you connect switch to port 1 and your laptop to switch do you get connection on the laptop? Mine solved that way.... Nevertheless, doesn't make very sense....

mOrbo

@visionsoul Yes this is working.

If there is a Switch in between, Port1 (igc0) is working as expected:

igc0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        description: LAN1
        options=e120bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,WOL_MAGIC,VLAN_HWFILTER,RXCSUM_IPV6,TXCSUM_IPV6>
        capabilities=f53fbb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,WOL_UCAST,WOL_MCAST,WOL_MAGIC,VLAN_HWFILTER,VLAN_HWTSO,NETMAP,RXCSUM_IPV6,TXCSUM_IPV6>
        ether 90:ec:77:35:ea:ae
        inet6 fe80::92ec:77ff:fe35:eaae%igc0 prefixlen 64 scopeid 0x1
        inet 192.168.2.1 netmask 0xffffff00 broadcast 192.168.2.255
        media: Ethernet autoselect (1000baseT <full-duplex>)
        status: active
        supported media:
                media autoselect
                media 2500Base-T
                media 1000baseT
                media 1000baseT mediaopt full-duplex
                media 100baseTX mediaopt full-duplex
                media 100baseTX
                media 10baseT/UTP mediaopt full-duplex
                media 10baseT/UTP
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>

Ping wird ausgeführt für 192.168.2.1 mit 32 Bytes Daten:
Antwort von 192.168.2.1: Bytes=32 Zeit<1ms TTL=64
Antwort von 192.168.2.1: Bytes=32 Zeit<1ms TTL=64
Antwort von 192.168.2.1: Bytes=32 Zeit<1ms TTL=64
Antwort von 192.168.2.1: Bytes=32 Zeit<1ms TTL=64

But this behavior is not normal.

VisionSoul

@morbo Yep, that's what i thought! Yes it's a strange behavior. Maybe someone from netgate can explain this? Who knows... They better than anyone should be able to explain why is that, since they design and developed the product.

Best Regards!

mer

If you look at the status line of the ifconfig output, when laptop is connected it says "no carrier", but goes to "active" when the switch is plugged in.

"no carrier" typically means the hardware that does all the autonegotiation isn't seeing any signal on the wire.

I'm assuming that the same cable used to connect the laptop was used to connect to the switch and that different cables were tried?

Are the igc0 and igc1 ports both part of the "4 discrete 2.5Gbs"?
Is there any configuration on the laptop that may put the ethernet into a low power mode?

I'm just really speculating, but perhaps something is causing a low power out of the laptop, that the switch can overcome. Toss in if the igc0 and igc1 are discrete components, perhaps a wrong value resistor on the igc0 path (PHY, etc) is causing the problem.

Something like that could tie into seeing the problem on a 4100 too, resistors are tiny little buggers now and a 1K vs a 10K is hard to tell.

mOrbo

@mer Yes I tried different cables, different laptops and even different switches (from high quality dell switches to really cheap 5-port unmanged switches). It's always the same. With Switch = Link, without Switch = no Link on Port-1.

And Port-2 is always working. So I don't think it has something to do with a config on the laptop.

I also assume, that the 4 LAN-Ports in the 6100 with the names igc0 - igc3 are on one physical network card, because they are one unit (see: https://docs.netgate.com/pfsense/en/latest/solutions/netgate-6100/io-ports.html).

stephenw10

The igc ports are using discrete ICs as oppose to the ix ports which are part of the C3558 SoC.

Were the test clients you used all attempting to link at the same speed?

Was the switch you added 1G or 2.5G/

Are you able to test connecting using a cross-over cable?
The behaviour described (link to a switch but not a client) is exactly what would happen if auto MDI/X was not working. In previous cases that has not been seen but it's worth testing here.

Also if this is a new unit you should open a ticket with us if you have not already done so:
https://www.netgate.com/tac-support-request

Steve

VisionSoul

@stephenw10 Hi

@morbo is having same issue as i. I had sent the unit back to my reseller and after testing they returned the unit saying that is perfectly functioning! They asked if i've tried to connect a switch to the port. By the time i said no, because never cross my mind that the port only works connected to a switch. Since all other ports and connecting just fine ( same laptop, same cables, everything equal.) In my case was port 2 (igc1)if i remember correctly. (i'm not using pfsense in the moment). But hey, nevertheless this is very strange all port are working just fine and only one is not! Or is this a problem with the units (since does not afect the same port (mine was igc1 and @morbo is igc0. So i only can conclude that is random).

For me even if i was using netgate, was not much big of a deal, because i would be using the 10gb ports to the main switch, but hey, nevertheless this is strange, And i paid almost 900€ for the unit!

Best Regards!

mOrbo

@stephenw10 Yes, all tested clients have a 1G nic. Standard HP or Dell Laptops.

Both Switches that I have tested are 1G Switches. 1 Dell N1548 and 1 TP-Link 5-Port "Desktop" Switch.

Sorry, but we don't have any cross-over cabels anymore. So I can't test this. Is it possible to set the NIC to a specific speed with ssh? I can do this also on the Switch side, so a specific Link-Speed can be tested without auto-negotiation. I've seen such a problem only once with a very old "german postal service" hardware.

The unit is new, but also bought from the German reseller "volatech". So I don't think I can open a case at netgate.

VisionSoul

@morbo Same reseller i bought mine from. Mine was sent to them and returned with "no problem"! Because there, they tested the port connected to a switch, and guess what? it worked...so unit was just fine and not defective, and they returned it as is.

It is what it is...

stephenw10

You can still open a ticket with us or with them.

The igc NIC/driver only supports auto-negotiation for the link speed/duplex. Setting a single speed in the webgui for it only restricts the speeds it can negotiate to that one value. so it won't link at all to anything that's set to a fixed link speed.
That applies to all the igc ports though so it's not the cause here.

Steve

VisionSoul

@stephenw10 Hi, if i may ask,

Do you have other reports on this issue? On this specific model? or maybe the 4100 since is almost the same.....without the sfp+ ports....?

Thanks in advance!

stephenw10

We have seen some and the results when we tested were inconclusive. As I said it behaves as though auto MDI/X is not working but in everything we have tested that was not the case.

Steve

mOrbo

@stephenw10 What do you think is the best thing to do now?

Opening a ticket with volatech should result in the same answer as for @VisionSoul, because Port-1 works on a Switch.

Opening a ticket with netgate does also not help, because netgate didn't sell the unit to me?

stephenw10

Open a ticket with us to discuss options: https://www.netgate.com/tac-support-request

If you reference this thread the support guys can ask me internally.

Steve