[RESOLVED] pfSense with CenturyLink - working but only 90Mbps on 940Mbps service

rcoleman-netgate

@jhg-goow I would swap interfaces, cables, switch ports, whatever you can to verify it's not a failure that's causing the slowness.

stephenw10

Yup, check re1 is not linked at 100M.

If you have access to the Zyxel router check the WAN config there to be sure.

Steve

jhg-goow

If you have access to the Zyxel router check the WAN config there to be sure.

@stephenw10 I do, and the connection was definitely operating at 1Gbps (getting about 900Mbps) prior to being replaced with pfSense.

At this point, if it does not turn out to be a bad cable, I'll be forced to abandon pfSense and go back to the Zyxel router. I can't afford to buy several different mini-pcs with different ethernet adapters to troubleshoot this (non-profit, shoestring budget).

At home I have a Zotac CI323 (RTL8111 ethernet) running CentOS Stream 8 as a router/firewall, and it comfortably does 950Mbps on Comcast in bridge mode. But Comcast doesn't use PPPoE, it's a raw Ethernet connection directly to the Internet. CL's insistence on such an antiquated system seems counterproductive.

stephenw10

And you can see the VLAN setup on the Zyxel is not using priority tags?

jhg-goow

@stephenw10 said in pfSense with CenturyLink - working but only 90Mbps on 940Mbps service:

And you can see the VLAN setup on the Zyxel is not using priority tags?

There's nothing in the Zyxel UI that mentions priority tags other than on the screenshot I posted earlier. My pfSense config matches @rcoleman-netgate and he says he sees the full 900Mbps+ speed.

rcoleman-netgate

@jhg-goow said in pfSense with CenturyLink - working but only 90Mbps on 940Mbps service:

and he says he sees the full 900Mbps+ speed.

On intel drivers.

I highly suspect it's the Realtek chipset.

stephenw10

@jhg-goow said in pfSense with CenturyLink - working but only 90Mbps on 940Mbps service:

There's nothing in the Zyxel UI that mentions priority tags other than on the screenshot I posted earlier.

Not seeing any screenshots from the Zyxel but if it shows nothing about tagging that won't help.

jhg-goow

@stephenw10 After all the wailing and gnashing of teeth, the problem turned out to be a combination of a bad cable, lying hardware and lying software.

The bad cable was the one between the ONT and the WAN port on the router.
The lights on the router's WAN ethernet port were both lit, supposedly indicating connection at 1000Mbps (the ONT doesn't have a speed indicating LED).
In the pfSense shell, ifConfig re0 claimed the adapter was connected at 1000Mbps.

As soon as I swapped in a different cable, Internet speed went up to 930Mbps.

Lessons learned:

ALWAYS check the simplest things first, including cables.
Ethernet hardware and software can lie to you about connection speed in some cases.
A Celeron J4125 @ 2.00GHz with RTL8111 ethernet is capable of running pfSense and handling PPPoE at 1Gbps quite comfortably. At one point early in the troubleshooting I did set net.isr.dispatch=deferred, which may be necessary.
Pay attention to the numbers. It took me embarrassingly long to realize 90Mbps speed was telling me one of the adapters was running at 100Mbps instead of 1G.

It is a big relief to have this working (and prove my hardware and FreeBSD weren't the fundamental limitations), and I really appreciate all the suggestions and help I got on this forum.

Thank you all.

stephenw10

Nice. Yeah the interesting thing is that the Ethernet negotiation only requires 2 pairs but Gigabit requires all 4. So it's possible for both ends to negotiate 1G when the cable cannot support it. Some NICs will detect that and prevent it. Not Realtek apparently.

jhg-goow

@stephenw10 @stephenw10 It turns out the problem wasn't quite resolved with the new cable. After a few hours I noticed the speed had dropped again to 100Mbps.

I remembered that there was an Eero WiFi access point in the same general area, and it was quite close to the firewall box. I unplugged the AP and the connection immediately reverted to 1Gbps.

I have never heard of WiFi (2.4GHz, 5-6Ghz) interfering with wired Ethernet (125MHz), but I suppose the Eero device could be poorly designed and leak significantly at other frequencies at close range.

I moved the WiFi AP several feet away and also replaced the ONT<--> pfSense and AP<-->switch cables with CAT6a shielded twisted pair for good measure. The connection has been stable at 1GBps for several hours now, so I believe (hope:-) the problem is finally resolved.

stephenw10

Potentially an incorrectly wired cable? If the wires from two pairs were swapped at both ends it could appear correct but use pairs that are not physically in a twisted pair. In that situation the common mode rejection is dramatically reduced and hence is far more susceptible to interference.

Steve