Slow upload speed [SOLVED]

[joda]

Hi,

I'm testing out pfsense 2.2.1 on my fit-pc2i (dual core atom and dual realtek GB-nic).

I'm on a 100Mbit line, but when I run the speed test (from a client behind the pfsense) I only reach 22-25Mbit upload.
The download speed maxes out at 100Mbit, so everything fine there.
With linux+iptables I max out at 100Mbit up and down, so the hardware is up for it.

Is NAT taking that big a hit on the upload performance?

The install is pretty vanilla, just regular NAT on outgoing traffic.
Load seem to land at 0.4 - 0.5 when doing the speed tests.

I've tried toggling (and rebooting)

• Hardware Checksum Offloading
• Hardware TCP Segmentation Offloading
• Hardware Large Receive Offloading

without any difference in performance.

Any help appreciated!

[heper]

nat shouldn't make much of a difference between up or downstream traffic. so its probably something else.

anything useful in logs?
rrd graphs with lots of packetloss?

how does status–>interfaces-->wan look ? in/out errors  or "media" not at  1000baseT <full-duplex>(or whatever it should be in your situation)
slow upload speed could also be related to a duplex mismatch (could be bad cable, or bad driver or whatever reason the autonegotiate is failing).</full-duplex>

[joda]

Thanks for the reply!

Both interfaces running at 1000baseT <full-duplex>No collisions or errors on the interfaces.

The RRD graphs for quality shows only a packetloss a few days ago, so nothing really.

I'm using the same cabling (and hardware) for testing with previous software(s).

Anything else worth investigating?</full-duplex>

[joda]

Some more testing done.

Replaced the fit-pc2i with an Intense PC2(i5, 8GB ram etc) and it pushes 100/100Mbit with pfsense 2.2.2.
Same cabling, same switches, same ports and so on.

Back on the fit-pc2i:
Tried toggling hyperthreading on/off without any noticable difference.
I even borrowed another fit-pc2i set it up with stock pfsense 2.2.2 and gets exactly the same throughput, 100Mbit down and 20-25 upstream.

Anyone else with fit-pc2i that have any performance numbers for comparison?

Next step is to try 2.1.5 to see if there's any difference.

Anything else I should look for?

[Harvy66]

Have you looked under System Activity to see what the load is like and what's causing the load? Many times low performance is because something is consume a lot of CPU, but not all of the time.

[almabes]

Enable powerd and set it for maximum or hiadaptive.
See if that helps.  Maybe your CPU isn't ramping up the MHz as the load increases.  I know it is an issue on the netgate rangely platforms.

YMMV

[joda]

Hi,

Some more testing.

Have turned on PowerD and set it to maximum.

Here's a dump from System Activity in the middle of a running speedtest

last pid: 15237;  load averages:  0.30,  0.18,  0.10  up 3+19:26:50    17:29:22
106 processes: 3 running, 83 sleeping, 19 waiting, 1 lock

Mem: 10M Active, 49M Inact, 48M Wired, 50M Buf, 857M Free
Swap: 2048M Total, 2048M Free

PID USERNAME PRI NICE  SIZE    RES STATE    TIME    WCPU COMMAND
  11 root    155 ki31    0K    8K RUN    87.3H  58.59% [idle]
  12 root    -92    -    0K  152K WAIT    15:34  26.66% [intr{irq256: re0}]
  12 root    -92    -    0K  152K WAIT    10:35  14.06% [intr{irq257: re1}]

Not really that high load, 0,30 - 0,40.
Is the amount of interrupts on the NICs normal when speedtesting?

[heper]

normal? not really … but realtek nics are famous for being horrible. (specially with freebsd)

you could attempt to enable 'device polling'  (system-->advanced-->networking). it's intended to stop/greatly reduce interrupts.

DISCLAIMER:
i have no idea if your nic is supported, so it could possibly go wrong and unable to access anything anymore.
enablling device polling is generally not recommanded in any normal situation, but who knows....

[Harvy66]

50% interrupt usage at 25Mb/s is really bad. My Intel NIC hangs around 5% at 1Gb.

[joda]

It didn't support polling so it kinda got stuck when I enabled it.

@harvy66: What boggles me is that I get 100Mbit downstream without any issues, but only 25 up. As if the amount of interrupts would differ(?)

[Harvy66]

Ahh yeah, missed that part. The only guess I can come up with is either the drivers are written that way or there is an interaction between the drivers and how FreeBSD handles moving data. I see no difference in interrupts for uploading or download, except if you include traffic shaping.

[joda]

After alot of desperate attempts, some of them being less scientific than others, I tried putting a 100Mbit switch between the WAN interface and the ISP termination. Turns out I'm now able to do 100/100.

I remove the switch and force the WAN interface to 100Mbit full-duplex (instead of the auto-neg 1Gbit), now 100/100 up/down works like a charm.
Interrupts on the NICs are down to just below 20%.

I guess this is another case of Realtek hardware/drivers. However the Linux kernel driver seems to be addressing this better.

Thanks for all the help and replys on the way!

[mbmondor]

drivers. set it up to vmx and all works fine :)

[JKnott]

@joda

Are you using WiFi? That will always provide poorer performance. For example, my connection is 500/20 Mb. When connected via Ethernet, I generally get 550 - 560 Mb down. On 802.11n WiFi, I get about 30 - 40 down. The Ethernet port for my WiFi AP is 100 Mb.

[mbmondor]

@JKnott it was drivers, i am using proxmox and the native network driver gave no upload speed lol. i am in full throttle on my 5ghz :) wifi and hard wired also :)

[JKnott]

@mbmondor

I just tried speedtest again and here's the result. I have seen a bit better. So, pfsense can definitely handle that sort of performance.

[mr-elamin2]

@joda We are now in 2023 and this solution has solved my my problem, I replaced the switch with an old 100 mbps switch and it has worked.
What I am thinking about right now is to confer SET (Switch Embedded Team) with one Team Member Port in Hyper-V, after that I think I could use my 1GB switch.

Thanks alot!