PfSense low throughput
-
1.3gb/s? wow… I tested with a couple of servers I have that are on 1gb lines and easily can do 900Mbit. Also from some of the testfiles from hosters but mostly via the cli speedtest.net script.
How are you speed testing? I don't have a 400mb+ connection with which to play with, but with iperf, I can easily reach the limit of my two test machine's 1.3gb/s with less than 5% cpu load, through firewall+NAT.
-
Exactly! especially because the D2700 hasnt shown any high load at all no matter what I throw at it..
I am using NAT but also 1:1 NAT with virtual IP's. (because of the multiple external IP's I have).
Maybe thats slowing things down in some way..The D525 will pass 500Mbps. Some have managed >600Mbps with some tuning. I'd be surprised if the D2700 couldn't manage 500Mbps. It won't do 1Gbps though and that's without any packages running.
Steve
-
It shouldn't be.
First off, are you or have you ever used limiters? Any type of traffic shaping? Captive portal?Set the Intel NIC as WAN and then run a test from the pfSense command line so you're only testing one side on the box. Is that what you did with the speedtest.net script? Otherwise try just fetching a large file from a known good source:
fetch -o /dev/null http://cachefly.cachefly.net/10mb.testOr maybe try something listed here: http://www.cloudtestfiles.net/
Steve
-
Hi Steve,
Thanks for you reply. I tried with the links you sent but I get lower speeds probably because I am located in NL not UK.I did once use a limiter for 1 IP but those I removed again, why do you mention this explicitly? Does that leave something that could impact througput even after removal?
( ooh and the Intel is on the WAN site. also most of the time my test speed on a server behind the pfSense is higher then the pfsense local tests :-\ )
[2.1.5-RELEASE][root@gateway]/root(1): fetch -o /dev/null http://cachefly.cachefly.net/10mb.test
/dev/null 100% of 10 MB 4230 kBps[2.1.5-RELEASE][root@gateway]/root(2): fetch -o /dev/null http://download.xs4all.nl/test/1GB.bin
/dev/null 100% of 953 MB 32 MBps 00m00sIt shouldn't be.
First off, are you or have you ever used limiters? Any type of traffic shaping? Captive portal?Set the Intel NIC as WAN and then run a test from the pfSense command line so you're only testing one side on the box. Is that what you did with the speedtest.net script? Otherwise try just fetching a large file from a known good source:
fetch -o /dev/null http://cachefly.cachefly.net/10mb.testOr maybe try something listed here: http://www.cloudtestfiles.net/
Steve
-
Ah, sorry. I assmed you were in the US. :-[ Cloudtestfiles have a site in Amsterdam.
What speeds do you see on the macbook directly connected using the same files?
Yes, I have seen odd traffic shaping where none should be before. If you've ever used it I would start from a clean install. Even if just as a test, you can always restore you config afterwards.
Is ifconfig showing the NICs connected at 'autoselect (1000baseT <full-duplex>)'?
Steve</full-duplex>
-
Aaah. I did reinstalled it, I only restored the config from the old setup so I should be good on that one. 8)
The WAN is a pppoe connection. everything else shows indeed 1000baseT <full-duplex>.
When I check with ifconfig the NIC who is used by the pppoe is also 1000baseT FD.
By the way, I tried to install iperf to do some other tests but for some reason I cannot install it.
I can install other packaged but this one throwes me an error:Beginning package installation for iperf .
Downloading package configuration file… done.
Saving updated package information... done.
Downloading iperf and its dependencies...
Checking for package installation...
Downloading https://files.pfsense.org/packages/amd64/8/All/iperf-2.0.5-amd64.pbi ... could not download from there or http://files.pfsense.org/packages/amd64/8/All//iperf-2.0.5-amd64.pbi.
of iperf-2.0.5-amd64 failed!Installation aborted.Removing package...
Starting package deletion for iperf-2.0.5-amd64...done.
Removing iperf components...
Tabs items... done.
Menu items... done.
Services... done.
Auxiliary files... done.
Package XML... done.
Configuration... done.
done.
Failed to install package.Installation halted.
By the looks of it the URL should be http://files.pfsense.org/packages/amd64/8/All/iperf-2.0.5-amd64.pbi. and not http://files.pfsense.org/packages/amd64/8/All//iperf-2.0.5-amd64.pbi.
So someone other then me made a typo somewhere.. :-[
[quote author=stephenw10 link=topic=84731.msg465211#msg465211 date=1417454403]
Ah, sorry. I assmed you were in the US. :-[ Cloudtestfiles have a site in Amsterdam.What speeds do you see on the macbook directly connected using the same files?
Yes, I have seen odd traffic shaping where none should be before. If you've ever used it I would start from a clean install. Even if just as a test, you can always restore you config afterwards.
Is ifconfig showing the NICs connected at 'autoselect (1000baseT <full-duplex>)'?
Steve
[/quote]</full-duplex></full-duplex> -
I would check BIOS settings. I've also read that Hyper-threading should also be turned off if you have it.
-
https://doc.pfsense.org/index.php/Tuning_and_Troubleshooting_Network_Cards
also check for errors on the wiring (just in case)
-
Is your Intel card PCI or PCI Express?
Also make your test using only the intel nics.
-
I wouldn't suspect hyperthreading (though it's an easy test). The general advise to disable it was based on much older hardware, netburst core, and no longer applies.
Even with two PCI NICs sharing the same bus I would expect ~500Mbps. Best to get that info though.
There are various power saving options that can kill performance. I forget exactly what it's called but there is a PCI power saving option that has been found causing problems on atoms before. There was a thread about it here somewhere, I should have book marked it. ::)
A clean install to which you restored your config may not be good enough. To rule out anything being carried across try just a very basic setup with nothing restored. Just run it live from an install CD or memstick install image.
Reading back, what hardware options did you set to reduce your throughout to 70Mbps?
Steve
Edit: Found it. This thread sees 600Mbps+ with a D2500: https://forum.pfsense.org/index.php?topic=67411.0
-
A lot of things to keep me busy here in this thread.. Much appreciated.
The speed went drastically down (70Mbps) when I checked the Hardware Checksum Offloading box. I also unticked for test the Hardware TCP Segmentation Offloading and Hardware Large Receive Offloading but there wasnt any noticable difference.
I dont have any power savings on in the BIOS and AFAIK also none in pfSense but I will search for powersettings for atoms maybe I indeed missed something there.
Next I will read the link you posted in your message.As mentioned somewhere along the road, I also tested the same setup in an i5 with an PCIe 4port Intel server card and got exactly the same throughput so my conclusion from that was that it is not likely to be the PCI bus and also not the Atom but still it doesnt harm to test further.
I heard nobody about the NAT 1:1 mapping so I guess thats never been a bottleneck?
Greetz,
OverrideI wouldn't suspect hyperthreading (though it's an easy test). The general advise to disable it was based on much older hardware, netburst core, and no longer applies.
Even with two PCI NICs sharing the same bus I would expect ~500Mbps. Best to get that info though.
There are various power saving options that can kill performance. I forget exactly what it's called but there is a PCI power saving option that has been found causing problems on atoms before. There was a thread about it here somewhere, I should have book marked it. ::)
A clean install to which you restored your config may not be good enough. To rule out anything being carried across try just a very basic setup with nothing restored. Just run it live from an install CD or memstick install image.
Reading back, what hardware options did you set to reduce your throughout to 70Mbps?
Steve
Edit: Found it. This thread sees 600Mbps+ with a D2500: https://forum.pfsense.org/index.php?topic=67411.0
-
Power saving options for Intel CPUs are C-states and SpeedStep.
Have you also tried resetting your modem to defaults and allowing it to pull down configs again?
-
I'm not talking about the CPU power saving, in the linked thread the offending option was PCI express ASPM which saves power by somehow sleeping/waking the PCIe bus. However the fact that you are seeing the identical speed on an i5 should be a big clue here. It's very unlikely both would have the same bad bios settings and even if they did the i5 is so much more powerful you'd expect to see at least some difference.
Both share the same upstream hardware connection, both use the same Intel driver. Some low level hardware incompatibility? Can you put a switch in between the WAN interface and the modem?
Steve
-
Actually reading back through the linked thread the PCIe ASPM only helped a little, the biggest performance increase came from using NICs on different buses. You would not have thought there would be any restrictions on a PCIe though. Also doesn't explain the i5 result.
Steve
