New pfsense install runs at only 610Mb/s on a gigabit fibre connection

Eddles

Hi all, just installed my first pfsense firewall. The hardware is a Dell PowerEdge R210 with a Intel Xeon CPU X3430 @ 2.40GHz, the motherboard has a 2-port Broadcom 5716 and there is an add-in 4-port Broadcom 5709 PCI-E card. I plugged the WAN cable into one of the ports of the 5709 card, and the LAN cable into one of the ports of the on-board 5716. I have a gigabit FTTP Internet connection provided by Truespeed (www.truespeed.com)

So, I run speedtest on my file server, and it shows:

 Speedtest by Ookla

    Server: Truespeed Communications - Bath (id = 17970)
    ISP: TrueSpeed Communications Limited
    Latency:     0.75 ms   (0.07 ms jitter)
    Download:    94.34 Mbps (data used: 42.5 MB )                               
    Upload:    94.11 Mbps (data used: 42.5 MB )                               
    Packet Loss: Not available.

Seems to be running at Fast Ethernet speeds, not Gigabit Ethernet speeds. So, I decided to try and find the bottleneck, so started from the beginning. Thus, I tried speedtest on the pfsense box and I get:

 Retrieving speedtest.net configuration...
 Testing from TrueSpeed Communications Limited (45.152.29.197)...
 Retrieving speedtest.net server list...
 Selecting best server based on ping...
 Hosted by Country Connect Ltd (Cardiff) [47.98 km]: 12.398 ms
 Testing download speed................................................................................
 Download: 613.20 Mbit/s
 Testing upload speed......................................................................................................
 Upload: 510.16 Mbit/s

What am I doing wrong here, and how do I fix/improve this, please? Note that the supplied router/modem is completely and utterly inaccessible, not even a status screen. Thank you!

Eddles

Here's the output from status > interfaces:

WAN Interface (wan, bce0)
Status up 
MAC Address xx
IPv4 Address 192.168.0.10 
Subnet mask IPv4 255.255.255.0 
Gateway IPv4 192.168.0.1 
IPv6 Link Local fe80::210:18ff:fe9c:1c08%bce0 
MTU 1500 
Media 1000baseT <full-duplex,master> 
In/out packets 658342/739210 (679.65 MiB/358.99 MiB) 
In/out packets (pass) 658342/739210 (679.65 MiB/358.99 MiB) 
In/out packets (block) 279/0 (21 KiB/0 B) 
In/out errors 0/0 
Collisions 0 

LAN Interface (lan, bce4)
Status up 
MAC Address xx
IPv4 Address 192.168.1.1 
Subnet mask IPv4 255.255.255.0 
IPv6 Link Local fe80::1:1%bce4 
MTU 1500 
Media 1000baseT <full-duplex> 
In/out packets 287994/324174 (191.32 MiB/268.32 MiB) 
In/out packets (pass) 287994/324174 (191.32 MiB/268.32 MiB) 
In/out packets (block) 4036/0 (705 KiB/0 B) 
In/out errors 2/0 
Collisions 0

SteveITS

@eddles As Netgate has posted here before if you run the speed test on the router that uses up CPU cycles used for routing.

That said if the test is faster not going through the LAN, did you try using one of the other three ports as LAN?

AndyRH

Make sure your connection for the file server and the pfSense LAN port are running at 1Gb. Your speed tests shows that pfSense is connected at 1Gb on the WAN even if the tested speed is lower.

stephenw10

@eddles said in New pfsense install runs at only 610Mb/s on a gigabit fibre connection:

Download:    94.34 Mbps (data used: 42.5 MB )                               
Upload:    94.11 Mbps (data used: 42.5 MB )

This is waaay to close to 100M to be a coincidence. It's not going to apply to traffic from the firewall itself but something in the path to that server is linked at 100M.

Steve

Eddles

Hi all!

Speedtest running slow on the pfsense box now makes sense. I was aware the server was running at Fast Ethernet (i.e. 100Mb/s). Spotted that I used a CAT 5E cable to connect the server to the switch, I did think CAT 5E was capable of gigabit Ethernet, but clearly not. Ordered a few CAT6 cables, replaced the server cable and tried the speedtest again:

   Speedtest by Ookla

     Server: Truespeed Communications - Bath (id = 17970)
        ISP: TrueSpeed Communications Limited
    Latency:     0.68 ms   (0.09 ms jitter)
   Download:   936.84 Mbps (data used: 481.3 MB )                               
     Upload:   938.01 Mbps (data used: 469.2 MB )                               
Packet Loss: Not available.

I'm fully satisfied, thank you everyone.

netblues

@eddles said in New pfsense install runs at only 610Mb/s on a gigabit fibre connection:

Hi all!

Speedtest running slow on the pfsense box now makes sense. I was aware the server was running at Fast Ethernet (i.e. 100Mb/s). Spotted that I used a CAT 5E cable to connect the server to the switch, I did think CAT 5E was capable of gigabit Ethernet, but clearly not. Ordered a few CAT6 cables, replaced the server cable and tried the speedtest again:
   Speedtest by Ookla

     Server: Truespeed Communications - Bath (id = 17970)
        ISP: TrueSpeed Communications Limited
    Latency:     0.68 ms   (0.09 ms jitter)
   Download:   936.84 Mbps (data used: 481.3 MB )                               
     Upload:   938.01 Mbps (data used: 469.2 MB )                               
Packet Loss: Not available.
I'm fully satisfied, thank you everyone.

Your thinking is correct. cat5e cable is fully certified and capable of doing 1Gbit speeds up to 100 meters.
However all wires have to be connected or it falls back to100Mbit.
It was just a faulty cable.

Eddles

@netblues not sure about that... when we moved here 2 years ago, I wired my house with CAT5e, and I've recently discovered all those wires are running at only 100Mb/s. I've got a few prefabricated CAT5e patch cables that are also giving me 100Mb/s for devices, and when I replaced them with the new CAT6 cables, they ran at gigabit.

The server was plugged into the switch with a one-metre long, prefabricated CAT5e cable, and it was connected at 100Mb/s. As I mentioned before, changing that with a one-metre long, prefabricated CAT6 cable, and it works at gigabit. The switch the server and pfsense boxes are plugged into is a Cisco SG200-26P, so it must have been something I've done wrong.

stephenw10

Weird. Cat5e should certainly be Gigabit capable.

rcoleman-netgate

@eddles said in New pfsense install runs at only 610Mb/s on a gigabit fibre connection:

I've got a few prefabricated CAT5e patch cables that are also giving me 100Mb/s for devices, and when I replaced them with the new CAT6 cables, they ran at gigabit.

Dumb questions...

Are those patch cables going into the wall and the same runs that are reading as 100M or directly into the switch?
Did you verify you terminated both ends either A or B?
Are you using custom made cables from the jack to the devices/switch?

My experience is usually one of two things has happened when you're reading wrong speeds: Either A) you terminated the ends differently/poorly or B) you have damaged the wires along the way.

I'd get, at the very least, a cable tester and test the whole run. Personally I have a tool that does full testing (and not a Fluke device) that pairs with my cell phone and gives me full reports on CAT3/5/5e/6/6a/6e cables.

netblues

Cat5 is definitely capable for 1G

A cheap tester is a starting point
e.g.

https://www.amazon.com/Docooler-Network-Cable-Tester-Remote/dp/B00DQH4XPW/ref=sr_1_21?c=ts&keywords=Network+%26+Cable+Testers&qid=1650264072&s=industrial&sr=1-21&ts_id=7701919011

This is even better on the cheap side
https://www.amazon.com/Noyafa-SC8108-Network-Tester-Mapper/dp/B00T7HE698/ref=sr_1_20?c=ts&keywords=Network+%26+Cable+Testers&qid=1650264192&s=industrial&sr=1-20&ts_id=7701919011

If cost is not an issue, then..

https://www.amazon.com/Fluke-Networks-MicroScanner-Verifier-Identifies/dp/B07NJMKG9L/ref=sr_1_6?c=ts&keywords=Network+%26+Cable+Testers&qid=1650264284&s=industrial&sr=1-6&ts_id=7701919011

rcoleman-netgate

@netblues Well, fine, I'll make my recommendation:

https://pockethernet.com/

$225 or so. Had a few issues with the first release -- they replaced it three times in the first 18 months -- but since then it's been rock solid.

JKnott

@stephenw10 said in New pfsense install runs at only 610Mb/s on a gigabit fibre connection:

Weird. Cat5e should certainly be Gigabit capable.

Yep, unless they're defective. 1 Gb requires all 4 pairs, but 100 Mb only 2.

Eddles

@stephenw10 @rcoleman-netgate @netblues @jknott Odd, I got a number of wildly different cables, of different vintages, nearly all pre-made and CAT5e, and using with different devices, but they all ran at Fast Ethernet speeds (i.e. 100MB/s). I double-checked all my homemade CAT5e cables, all OK, but still Fast Ethernet speeds. I already have a simple cable tester, and it passed them all.

Changing out the cables for homemade CAT6 obtained gigabit speeds. I really don't know why this is happening, but as long as I get gigabit speeds with CAT6, I'm happy.

Thanks everyone!

rcoleman-netgate

@eddles said in New pfsense install runs at only 610Mb/s on a gigabit fibre connection:

@stephenw10 @rcoleman-netgate @netblues @jknott Odd, I got a number of wildly different cables, of different vintages, nearly all pre-made and CAT5e, and using with different devices, but they all ran at Fast Ethernet speeds (i.e. 100MB/s). I double-checked all my homemade CAT5e cables, all OK, but still Fast Ethernet speeds. I already have a simple cable tester, and it passed them all.

Are you using a patch panel at all? Or is one of your switches old(er)? Old infrastructure cable? bad splices?

Those are the only reasons I can think of why you'd have 100FD connections

Edit... I see you resolved it, but I'm still curious. Hopefully it helps someone else in the future.

marvosa

I have CAT5e jacks throughout the house and get gig speeds on every device.

Possibilities I can see... misbehaving NIC on the endpoint, damaged cabling, bent pins (or corrosion) on the jack, bad termination on the jack (or patch panel), misbehaving switchport, misconfigured NIC, could also be running CAT5 instead of CAT5e, amongst other things.

You can actually get up to 5 Gbit over CAT5e using mGig ports, so there shouldn't be any issues getting 1 Gbit speeds under normal circumstances. If you're not getting 1 Gbit over CAT5e, then there's an issue with something... somewhere... however subtle it may be. I feel like the CAT6 cable may be masking the underlying issue... but glad it's working.