For future reference, when you reinstall it sets the boot env to the chosen install media. It is possible to set that manually from the uboot prompt though. For example:

setenv bootcmd 'run setLED; run emmcboot;' saveenv reset

You would only ever need to do that if for some reason you need to change boot media without reinstalling.

Steve

Hey everyone, thanks for your kind suggestions. At the same time I was working with Netgate directly as I personally became convinced pretty quickly that this was a hardware issue.

After 2 days of installing new firmware from console, resetting settings, and turning things off, Netgate eventually agreed it was hardware and gave me an RMA. Now I'm in the waiting game between shipping the device back and getting a new one. I have to say, having to pay return shipping for a dead on arrival device kinda blows. So does 2 days of hours spent troubleshooting (a few times with techs who clearly were convinced the fault was me misconfiguring things).

New customer here, not sure if I'll be a repeat one after this experience. What's worse: Ubiquiti where they lie about being hacked (which is why I picked netgate over the EdgeRouter) or this?

But I can say the community (y'all who replied above) seem pretty great!

The advantage currently is the increased resilience to filesystem damage from a power loss.

There is not (yet) any integration for things like snapshots or boot environments but that is on the cards.

ZFS will install and run fine on that box with the default settings. I've been running it here on numerous things for a long time.

Steve

Additional insights:

Version is 21.05.1 BIOS is 1.2c no NAT done, pfSense is internal firewall/router in front of their core switches ix0/ix1 are combined to lagg0 and connected to a core switch each (no crossover cabling) lagg0 is running normally (both channels active) rules on test VLAN interfaces are simple and to test were pass alls

so nothing out of the ordinary that would hinder traffic flow or performance

@lralvarez what @bmeeks said - but why would you want 21.02 vs 21.05.1 which is current? Running 21.05.1 on my sg4860..

They normally respond with link to image and instructions within a few minutes.

Ok, the best thing here is probably going to be to open a ticket with our sales guys. You can just email sales@netgate.com. They should be able to help you out with whatever numbers you need.

Steve

@stephenw10 OK, thanks for the info

@stephenw10 said in 1541 and 6100 ALTQ limitations with 10Gbe:

FQ_CoDel as a buffer bloat mitigation is usually applied via Limiters which can work on any interface.

In fact AltQ traffic shaping can now be used on ix NICs directly. Those docs are being updated:
https://github.com/pfsense/pfsense/blob/RELENG_2_5_2/src/etc/inc/interfaces.inc#L7006

Steve

Excellent - thank you.

Were you able to see any improvement here?

That's between New York and New Jersey? What latency do you see across the tunnel?

I would certainly expect to see more than 85Mbps provided the WAN connections at each end allow it.

Steve

Hmm, that is odd. The switch chip in the 1100 connects to all the ports and is auto MDI/MDIX.

Do you see link with a straight through cable to the unmanaged external switch? It shouldn't make any difference.

Do you see a link with the laptop connected to the LAN or OPT ports? Those should also be identical.

Steve

The only thing that could present a difference here is the hardware crypto in the safexcel driver. But you said you tried using a cipher that does not effect (blowfish) so it can't be that directly.

So I'm left trying to think of something you might have had set in the old device that's somehow incompatible with the SG-1100. I can't see what that could be though.

The fact setting the tunnel to use ports 600/4600 allowed it to come up implies something in the path blocking the standard ports. The crypto hardware doesn't care what ports are in use for example.

It really 'feels' like the upstream device trying to do something clever with IPSec traffic.

Are we able to review the config you are importing to the 1100? If you open a ticket with us and reference this thread the guys will make sure I see it.

It's hard to see how this could be a hardware issue. If we swapped it out I would expect another device to do exactly the same thing given the same config.

Steve

Yes, it's still safe to buy. 😉

We have no plans to stop selling it I'm aware of and even when we do it will still be supported with updates for a long while after that. (years)

Steve

If you only have one WAN they can be the same. Or should be able to.

The important thing to realise is that both of those settings create a static route for that IP via the gateway it's assigned to. So if you have multiple gateways and you try to use the same IP for monitoring and DNS on different gateways that creates a routing conflict.

Steve

That's possible, though that particular thread deals with a real USB Ethernet NIC.

Really it depends what is actually happening here. If the modem requires some manipulation before it appears as an Ethernet device. If it's just timing you can probably add a delay to prevent it. It's possible to just exclude USB Ethernet devices from the interface check at boot but doing so risks unknown behaviour in the event it's actually disconnected.

Steve

Mmm, that sort of throttling looks like a speed/duplex mismatch but I can't see where it would be.

The switch on the WAN side test rules that out. And that would affect all clients.

It has 8 NICs with no switch.
4x igc NICs at up to 2.5G
4x ix NICs where 2 are 10G SFP+ and 2 are 1G combo ports.

[21.09-RC][admin@6100.stevew.lan]/root: ifconfig -am igc0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500 options=8120b8<VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,WOL_MAGIC,VLAN_HWFILTER> capabilities=f53fbb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,WOL_UCAST,WOL_MCAST,WOL_MAGIC,VLAN_HWFILTER,VLAN_HWTSO,NETMAP,RXCSUM_IPV6,TXCSUM_IPV6> ether 00:08:a2:12:17:7a media: Ethernet autoselect (2500Base-T <full-duplex>) status: active supported media: media autoselect media 2500Base-T media 1000baseT media 1000baseT mediaopt full-duplex media 100baseTX mediaopt full-duplex media 100baseTX media 10baseT/UTP mediaopt full-duplex media 10baseT/UTP nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL> igc1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500 options=8120b8<VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,WOL_MAGIC,VLAN_HWFILTER> capabilities=f53fbb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,WOL_UCAST,WOL_MCAST,WOL_MAGIC,VLAN_HWFILTER,VLAN_HWTSO,NETMAP,RXCSUM_IPV6,TXCSUM_IPV6> ether 00:08:a2:12:17:7a hwaddr 00:08:a2:12:17:7b media: Ethernet autoselect (2500Base-T <full-duplex>) status: active supported media: media autoselect media 2500Base-T media 1000baseT media 1000baseT mediaopt full-duplex media 100baseTX mediaopt full-duplex media 100baseTX media 10baseT/UTP mediaopt full-duplex media 10baseT/UTP nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL> igc2: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500 description: OPT3 options=8120b8<VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,WOL_MAGIC,VLAN_HWFILTER> capabilities=f53fbb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,WOL_UCAST,WOL_MCAST,WOL_MAGIC,VLAN_HWFILTER,VLAN_HWTSO,NETMAP,RXCSUM_IPV6,TXCSUM_IPV6> ether 00:08:a2:12:17:7c inet6 fe80::208:a2ff:fe12:177c%igc2 prefixlen 64 scopeid 0x3 media: Ethernet 10baseT/UTP <full-duplex> status: active supported media: media autoselect media 2500Base-T media 1000baseT media 1000baseT mediaopt full-duplex media 100baseTX mediaopt full-duplex media 100baseTX media 10baseT/UTP mediaopt full-duplex media 10baseT/UTP nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL> igc3: flags=28943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST,PPROMISC> metric 0 mtu 1500 description: OPT4 options=8120b8<VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,WOL_MAGIC,VLAN_HWFILTER> capabilities=f53fbb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,WOL_UCAST,WOL_MCAST,WOL_MAGIC,VLAN_HWFILTER,VLAN_HWTSO,NETMAP,RXCSUM_IPV6,TXCSUM_IPV6> ether 00:08:a2:12:17:7d inet6 fe80::208:a2ff:fe12:177d%igc3 prefixlen 64 scopeid 0x4 inet 192.168.78.1 netmask 0xffffff00 broadcast 192.168.78.255 media: Ethernet autoselect status: no carrier supported media: media autoselect media 2500Base-T media 1000baseT media 1000baseT mediaopt full-duplex media 100baseTX mediaopt full-duplex media 100baseTX media 10baseT/UTP mediaopt full-duplex media 10baseT/UTP nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL> ix0: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500 options=8138b8<VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,WOL_UCAST,WOL_MCAST,WOL_MAGIC,VLAN_HWFILTER> capabilities=f53fbb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,WOL_UCAST,WOL_MCAST,WOL_MAGIC,VLAN_HWFILTER,VLAN_HWTSO,NETMAP,RXCSUM_IPV6,TXCSUM_IPV6> ether 00:08:a2:12:17:7e media: Ethernet autoselect status: no carrier supported media: media autoselect nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL> ix1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500 description: IX1 options=8138b8<VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,WOL_UCAST,WOL_MCAST,WOL_MAGIC,VLAN_HWFILTER> capabilities=f53fbb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,WOL_UCAST,WOL_MCAST,WOL_MAGIC,VLAN_HWFILTER,VLAN_HWTSO,NETMAP,RXCSUM_IPV6,TXCSUM_IPV6> ether 00:08:a2:12:17:7f inet6 fe80::208:a2ff:fe12:177f%ix1 prefixlen 64 scopeid 0x6 inet 192.168.79.2 netmask 0xffffff00 broadcast 192.168.79.255 media: Ethernet autoselect status: no carrier supported media: media autoselect nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL> ix2: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500 description: LAN options=8138b8<VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,WOL_UCAST,WOL_MCAST,WOL_MAGIC,VLAN_HWFILTER> capabilities=f53fbb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,WOL_UCAST,WOL_MCAST,WOL_MAGIC,VLAN_HWFILTER,VLAN_HWTSO,NETMAP,RXCSUM_IPV6,TXCSUM_IPV6> ether 00:08:a2:12:17:80 inet6 fe80::208:a2ff:fe12:1780%ix2 prefixlen 64 scopeid 0x7 inet 192.168.241.1 netmask 0xffffff00 broadcast 192.168.241.255 media: Ethernet autoselect status: no carrier supported media: media autoselect media 10baseT/UTP media 100baseTX media 1000baseT nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL> ix3: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500 description: WAN options=8138b8<VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,WOL_UCAST,WOL_MCAST,WOL_MAGIC,VLAN_HWFILTER> capabilities=f53fbb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,WOL_UCAST,WOL_MCAST,WOL_MAGIC,VLAN_HWFILTER,VLAN_HWTSO,NETMAP,RXCSUM_IPV6,TXCSUM_IPV6> ether 00:08:a2:12:17:81 inet6 fe80::208:a2ff:fe12:1781%ix3 prefixlen 64 tentative scopeid 0x8 inet 172.21.16.246 netmask 0xffffff00 broadcast 172.21.16.255 inet 172.21.16.247 netmask 0xffffffff broadcast 172.21.16.247 vhid 5 carp: MASTER vhid 5 advbase 1 advskew 0 media: Ethernet autoselect (1000baseT <full-duplex,rxpause,txpause>) status: active supported media: media autoselect media 10baseT/UTP media 100baseTX media 1000baseT nd6 options=2b<PERFORMNUD,ACCEPT_RTADV,IFDISABLED,AUTO_LINKLOCAL>

Steve

@johnpoz thank you! This was extremely helpful.

Just applied the patch afcc0e9c97c1993ae6b95f886665fcb4375d26c7 and rebooted.
It started with the service disabled, thanks.

Device and firmware: 21.05.1 / SG-3100.

fbafb584-8916-4c50-b990-7817727697a3-image.png

Yup, that will cover it.