Pfsense limiting WAN speeds?

ub3rz4ch

Sorry If I post this in the wrong area.

So I am on a completely fresh install of pfsense all default settings, I am testing the WAN speeds on Speedtest.net but also have tried downloading
a 1GB file from Thinkbroadband and I was getting 120 megabits per second (15megabytes per second) which I noticed is a little slower than I usually get

Before I was using pfsense direct from my ISP modem I was getting 160 megabits per second (20megabytes).

All the hardware is 1000baseT <full-duplex>, I see in pfsense WAN and LAN is 1000baseT <full-duplex>, and I am using a netgear managed switch, which also reads that all the ports are running at GB speeds, and both the PCS that I am using to test the speed also read 1Gbps on the network adapter.

so as a test I turned my modem back into its normal mode (to see if it is the "modem mode" that you can activate which disables WIFI and enables only one port for DHCP) and I can confirm it is not that as I have now got the same slow speed through Pfsense using the modem in its normal mode.

so connecting to the modem in the normal mode over WIFI (AC) I get my usual 160mbps. Testing the speed direct to pfsense and through the GB switch (with the modem in normal mode) I get slow speeds about 10/20 megabits per second slower…

I noticed Hardware TCP Segmentation Offloading and Hardware Large Receive Offloading state that sometimes this can cause poor performance on some NICS but I tired it and it made no difference.

can only help me figure this out, I am quite a noob with pfsense :P

I might add that I tested all my cables and there all working fine at GB speeds, I get the 20megabytes per second speeds if I go direct to the modem using any of the ethernet cables I am using, so its not the cables that are at fault.</full-duplex></full-duplex>

1st
The modem is not doing SPI/NAT and working through firewall rules and there for you might be getting not the same
throughput out with the pfsense if it is not really powerful enough. Don´t think a small firewall is able to get the same
out as your ISP given router that is ASIC/FPGA based and pimped! There is all done in silicon but pfSense is a software
firewall and so you may need to spend more money for getting the same throughput out.

What is your real hardware you are using?
What CPU and board or RAM?

marvosa

Multiple things to check here. Building on what BlueKobold started:

What is your bandwidth supposed to be? i.e. what service are you paying for from your ISP?
What kind of hardware you are using?
- Desktop or Server?
- CPU?
- RAM?
- PCI or PCIe NICs?
- Hardware or Software based NIC's?
PFsense virtualized or on bare metal?
Are your cables custom made or pre-fab'd?
Have you tested your cables with a cable tester?
Have you checked the duplex on all your interfaces? Noted in OP.
Are there any in/out errors on any of your interfaces?
If you switch is managed, are there any interface errors on your switch ports? Smh… "netgear managed switch".... that might be your issue right there ;)
Does your CPU max out when you're doing a speed test?
Are you running any packages? e.g. Snort, Suricata, Squid, AV, etc
Are you double NAT'd? i.e. is your modem in bridge or router mode?
Is traffic shaping configured?
Are any limiters configured?

ub3rz4ch

@marvosa:

Multiple things to check here. Building on what BlueKobold started:

What is your bandwidth supposed to be? i.e. what service are you paying for from your ISP?

What kind of hardware you are using?

Desktop or Server?

CPU?

RAM?

PCI or PCIe NICs?

Hardware or Software based NIC's?

PFsense virtualized or on bare metal?

Are your cables custom made or pre-fab'd?

Have you tested your cables with a cable tester?

Have you checked the duplex on all your interfaces? Noted in OP.

Are there any in/out errors on any of your interfaces?

If you switch is managed, are there any interface errors on your switch ports? Smh… "netgear managed switch".... that might be your issue right there ;)

Does your CPU max out when you're doing a speed test?

Are you running any packages? e.g. Snort, Suricata, Squid, AV, etc

Are you double NAT'd? i.e. is your modem in bridge or router mode?

Is traffic shaping configured?

Are any limiters configured?

Okay I'll try answer these for you.

My isp gives me 150mbps I usually get 160mbps and I tested the speeds not at peak times.
Right after I got slow speed with pfsense I tried with the normal setup using the isp router and got my fast speeds again.
It's a server.. it's a machine solely responsible for the pfsense. So its baremetal
It's specs are way overkill for a router/ firewall at home.
The CPU dual core 3.5GHz "AMD Athlon 64 Processor 3500+"
2.3.2-RELEASE (i386)
There is 2GB ram
PCI physical NIC a HP NC7170
Some cables are pre made some are self made cat6
It's not the switch or the cables as both give me full speeds from my isp router. And yes I tested every cable with are cable tester.
Yes all interfaces are at 1000baseT full duplex
No in out errors
No errors on the switch as I said before is working fine outside of pfsense.
Cpu Ive only noticed gets to about 20% if that.
Fresh install of pfsense no packages installed.
The modem is in what my isp calls modem mode which just uses one port to send to your own router.

"Enabling modem mode allows you to connect your own WiFi kit to the new Super Hub, effectively turning the Super Hub into a stand alone DOCSIS3 cable modem." Aka your own all in one router/wifi "When active, Modem Mode disables the wireless and routing functions of your Super Hub so you can use your own wireless router. For more information please visit" As its a coax "fibre" line I need it as I dont have a NIC that has coax.

No limiters or shaping set as it's a fresh install.
I might add I am getting 130mbps today but still I get 160mbps without pfsense.
I am using googles DNS but I dont think that makes a difference as I use googles dns before I had pfsense and had the right speeds
also I tired without googles DNS and it changed nothing.

Also I will say that I tried the "modem mode" on my Tplink ac1750 (Not going through pfsense to see if
its the "modem mode" that is the problem) and it was 160mbps. So its not the "modem mode" thats the issue either.

marvosa

It's a server.. it's a machine solely responsible for the pfsense. So its baremetal
It's specs are way overkill for a router/ firewall at home.
The CPU dual core 3.5GHz "AMD Athlon 64 Processor 3500+"
2.3.2-RELEASE (i386)
There is 2GB ram
PCI physical NIC a HP NC7170

Are you calling it a server because it's dedicated to PFsense? Because I can tell you that:

The Athlon 3500+ is an AMD desktop CPU. If you had a true server, it would have an Opteron.
The 3500+ actually runs at 2.2 Ghz (not 3.5 Ghz).
2 GB of ram is generally not enough for a server
As far as the HP NC7170 NIC, this is somewhat conflicting to me because the NC7170 is a PCI-X dual port server NIC and I highly doubt the motherboard you have running your Athlon 3500+ has PCI-X slots. If your MB doesn't have PCI-X slots, and you are using an NC7170, that tells me you're running it in a 32 bit PCI slot, which isn't what you want.

It's not the switch or the cables as both give me full speeds from my isp router. And yes I tested every cable with are cable tester.

Didn't necessarily think it was, but had to ask to make sure it wasn't over overlooked

No errors on the switch as I said before is working fine outside of pfsense.

While I believe you that the switch probably isn't the source of the problem, the statement of "working fine" isn't exactly a technical assessment :) Did you actually check the interface stats of each switch port? Just want to make sure we're thorough and not making assumptions with potential points of failure.

I am using googles DNS but I dont think that makes a difference as I use googles dns before I had pfsense and had the right speeds
also I tired without googles DNS and it changed nothing.

You're right, DNS wouldn't have anything to do with throughput once the connection is made.

Having said all that, considering your hardware, you may have to manage some of your expectations regarding throughput and overall performance.

The Athlon 3500+ is a desktop CPU that was released in 2004, which dates the architecture of the entire machine back 13 years. Given the bandwidth you're paying for, and the hardware requirements to push that bandwidth, I'd say you are in need of a hardware upgrade to consistently sustain 150+ Mbps throughput. Take a look at https://www.pfsense.org/hardware, In order to sustain 101-500 Mbps, their assessment says you need:

No less than a modern Intel or AMD CPU clocked at 2.0 GHz. Server class hardware with PCI-e network adapters, or newer desktop hardware with PCI-e network adapters.

IMO, here's what needs to be done at a minimum:

Upgrade your hardware
- If you're going the desktop route, while there are multiple CPU options that will work, I'd go with a minimum of a 3+ Ghz i3 if not an i5 or i7 if budget allows.
- 4 GB of RAM minimum, but I'd recommend 8 GB
- Purchase PCI-e Intel NIC's
Install the 64 bit version of PFsense, so the OS can take advantage of the extra ram and newer architecture.

ub3rz4ch

Okay

"I am quite a noob with pfsense :P"

I did say I wasn't exactly the best with Pfsense just want to say that firstly.

When I called it a "server" I define server myself personally as a machine dedicated to one job, not being used as a desktop pc at the same time
I should have been more specific and say that it wasnt server hardware, Ie server motherboard ram cpu etc.

I actually did not know the clockspeed and at the time I just saw the 3500+ on the main screen of pfsense and just guessed that meant 3.5ghz so
Yeah youre right that is pretty slow.

I figured 2GB would be enough, And I probably should have researched more into it.

The NIC youre right is plugged into a normal PCI slot, but as I understood that is fine (or so my friend who gave me the card said ) and it does read as 1GB speeds in pfsense?

However
I just looked up the speed of PCI slots which I did not know
"Speed 133 MB/s (32-bit at 33 MHz – the standard configuration)
266 MB/s (32-bit at 66 MHz or 64-bit at 33 MHz)
533 MB/s (64-bit at 66 MHz)"

I also saw that although the PCI-x card works fine in a normal slot its at reduced speed so would that be the problem, I obviously did not give it much thought
when testing the speeds. But correct me if I am wrong isnt 133 Megabytes/s 1064 Megabits/s ? so my speed I want of 160megabits/s would be 20megabytes/s

As for the cables I also was just clarifying I tested the cables. I know youre just trying to help so you had to ask.

I did check every port on the switch albeit using there own managed interface. (But as I said previously I get my usuall throughput without pfsense using the switch so I assumed its not the switch that is at fault)

So the answer to this topic is that my hardware isnt enough to keep up with the speeds I wish to achive, Its only a pc I had lying around I figured Id build it into a pfsense box I have done it before and had good success out of it, but this was before I had real quick internet.

I guess I should have researched more, I just figured someone on the forum could immediately point out the issue maybe it was obvious to someone on here but not me as I am not very proficient in networking. I guess I will go upgrade the machines hardware before I try get full speeds

At the time I set the box up I thought it would be way better spec than the modem the ISP gave me but I guess that isnt true.

I knew pfsense would run well on my hardware but I didnt know Id need better hardware for greater than 100mbps speeds I should have looked more into the hardware requirements

Also the PCI-X thing probably isnt helping…

In environments where extremely high throughput through several interfaces is required, especially with gigabit interfaces, PCI bus speed must be taken into account. When using multiple interfaces in the same system, the bandwidth of the PCI bus can easily become a bottleneck.

GentleJoe

The PC might not be able to fully push data at that speed over the PCI connected Ethernet card.

Have you tried running the 'iperf' test as client or server with a PC?

I use an Intel(R) Atom(TM) CPU D2550 @ 1.86GHz, even though my ISP speed is only 20/2, I can reach iperf tests on my LAN above 400 Mbps.
So you don't need a power hungry system these days to perform well.

ub3rz4ch

@Gentle:

The PC might not be able to fully push data at that speed over the PCI connected Ethernet card.

Have you tried running the 'iperf' test as client or server with a PC?

I use an Intel(R) Atom(TM) CPU D2550 @ 1.86GHz, even though my ISP speed is only 20/2, I can reach iperf tests on my LAN above 400 Mbps.
So you don't need a power hungry system these days to perform well.

I figured as much. I just installed iperf package and it returns a black page am I doing something wrong ?

GentleJoe

You just started the client there.

I would run there iperf server on pfsense, then connect with a client.

You can use clients on windows/linux/android.

https://www.sd-wan-experts.com/iperf-bandwidth-testing/