Gigabit troubleshooting
-
Update: iperf3 so far reporting <200/sec.
Not to beat a dead horse because it has been discussed several times before, but I'm going to bring it up again anyway.
I am in/around Des Moines, IA to start. I just got Centurylink 1gig hooked up today and they sent a tech out to do an 'install.' The first thing the tech says to me when he walks in the door is "So, they have you set up for the Gig, huh? Well, hate to break it to you, but we don't have any hardware that supports that speed." I inquire about the Technicolor, etc, at least to get me up and running at a higher speed - nope, they don't have it. Ok, so why even offer 1gig if you don't support it??
Anyway, long story short he says you are on your own, he only knows of 1 guy IN THE ENTIRETY OF IOWA that is hitting anywhere near gig speeds and there's nothing he can do and leaves. I'm sorry, I don't believe that at all.
SOOOO, here I am, with my brand spanking new Ferrari of an Intertube and I need some help squeaking out as much speed as possible.
First off, the basics-
Hardware
Jetway JC320U93W-2930-B Intel Celeron N2930 Dual Intel LAN Fanless NUC (Quad core) http://www.amazon.com/Jetway-JC320U93W-2930-B-Intel-Celeron-Fanless/dp/B00SHYW6US?ie=UTF8&psc=1&redirect=true&ref_=oh_aui_detailpage_o08_s00
Crucial 8Gb 1333 ram
250Gb SSDA little system information:
hw.machine: amd64
hw.model: Intel(R) Celeron(R) CPU N2930 @ 1.83GHz
hw.ncpu: 4
dev.em.0.%desc: Intel(R) PRO/1000 Network Connection 7.4.2
dev.em.0.%driver: em
dev.em.1.%desc: Intel(R) PRO/1000 Network Connection 7.4.2
dev.em.1.%driver: emThis is connected directly to the ONT by cat6. I am authenticating using PPPoE (most likely the issue) tagged VLAN 201.
My computers/AppleTv's, etc are connected to the router through a Netgear gigabit switch. At first I had them running through an Apple Airport Extreme, but that definitely is not stout enough for 1g.
I set the mbuf tunable to 1,000,000 per the pfsense documentation.
So far, the max I have been able to get is 587/572. Any other advice?
Thanks!–--A little more info. Tested that gigabit LAN is working. Pathetic speeds from the router to the iperf server.
Router to iMac (hardwired, testing gigabit LAN connection)
Accepted connection from 192.168.1.1, port 64195
[ 5] local 192.168.1.107 port 5201 connected to 192.168.1.1 port 61209
[ ID] Interval Transfer Bandwidth
[ 5] 0.00-1.00 sec 69.3 MBytes 582 Mbits/sec
[ 5] 1.00-2.00 sec 72.6 MBytes 609 Mbits/sec
[ 5] 2.00-3.00 sec 92.3 MBytes 774 Mbits/sec
[ 5] 3.00-4.00 sec 104 MBytes 872 Mbits/sec
[ 5] 4.00-5.00 sec 105 MBytes 880 Mbits/sec
[ 5] 5.00-6.00 sec 105 MBytes 880 Mbits/sec
[ 5] 6.00-7.00 sec 104 MBytes 876 Mbits/sec
[ 5] 7.00-8.00 sec 104 MBytes 876 Mbits/sec
[ 5] 8.00-9.00 sec 104 MBytes 874 Mbits/sec
[ 5] 9.00-10.00 sec 104 MBytes 875 Mbits/sec
[ 5] 10.00-10.00 sec 313 KBytes 843 Mbits/sec
[ ID] Interval Transfer Bandwidth Retr
[ 5] 0.00-10.00 sec 966 MBytes 810 Mbits/sec 0 sender
[ 5] 0.00-10.00 sec 966 MBytes 810 Mbits/sec receiveriperf.scottlinux.com to Router
Reverse mode, remote host iperf.scottlinux.com is sending
[ 4] local 63.224.182.9 port 26135 connected to 173.230.156.66 port 5201
[ ID] Interval Transfer Bandwidth
[ 4] 0.00-1.00 sec 2.39 MBytes 20.1 Mbits/sec
[ 4] 1.00-2.00 sec 6.56 MBytes 55.1 Mbits/sec
[ 4] 2.00-3.00 sec 11.2 MBytes 94.2 Mbits/sec
[ 4] 3.00-4.00 sec 15.6 MBytes 131 Mbits/sec
[ 4] 4.00-5.00 sec 20.1 MBytes 168 Mbits/sec
[ 4] 5.00-6.00 sec 25.0 MBytes 210 Mbits/sec
[ 4] 6.00-7.00 sec 29.2 MBytes 245 Mbits/sec
[ 4] 7.00-8.00 sec 27.6 MBytes 232 Mbits/sec
[ 4] 8.00-9.00 sec 18.3 MBytes 153 Mbits/sec
[ 4] 9.00-10.00 sec 19.7 MBytes 165 Mbits/sec
[ 4] 10.00-11.00 sec 20.3 MBytes 170 Mbits/sec
[ 4] 11.00-12.00 sec 20.9 MBytes 176 Mbits/sec
[ 4] 12.00-13.00 sec 21.3 MBytes 178 Mbits/sec
[ 4] 13.00-14.00 sec 21.3 MBytes 179 Mbits/sec
[ 4] 14.00-15.00 sec 21.7 MBytes 182 Mbits/sec
[ 4] 15.00-16.00 sec 21.6 MBytes 181 Mbits/sec
[ 4] 16.00-17.00 sec 21.3 MBytes 179 Mbits/sec
[ 4] 17.00-18.00 sec 21.5 MBytes 181 Mbits/sec
[ 4] 18.00-19.00 sec 18.2 MBytes 153 Mbits/sec
[ 4] 19.00-20.00 sec 17.3 MBytes 145 Mbits/sec
[ 4] 20.00-21.00 sec 18.8 MBytes 158 Mbits/sec
[ 4] 21.00-22.00 sec 19.6 MBytes 165 Mbits/sec
[ 4] 22.00-23.00 sec 21.0 MBytes 176 Mbits/sec
[ 4] 23.00-24.00 sec 21.2 MBytes 178 Mbits/sec
[ 4] 24.00-25.00 sec 21.4 MBytes 180 Mbits/sec
[ 4] 25.00-26.00 sec 22.2 MBytes 186 Mbits/sec
[ 4] 26.00-27.00 sec 21.7 MBytes 182 Mbits/sec
[ 4] 27.00-28.00 sec 21.8 MBytes 183 Mbits/sec
[ 4] 28.00-29.00 sec 22.1 MBytes 185 Mbits/sec
[ 4] 29.00-30.00 sec 16.6 MBytes 139 Mbits/sec
[ ID] Interval Transfer Bandwidth Retr
[ 4] 0.00-30.00 sec 591 MBytes 165 Mbits/sec 404 sender
[ 4] 0.00-30.00 sec 588 MBytes 164 Mbits/sec receiver
-
Wow, nothing? Ok. Can anyone give me any suggestions on how to test my router's performance (CPU load, throughput saturation, etc)? Maybe any tweaks? In the meantime I will continue searching the forums.
For what its worth, I am running 2.2.6-RELEASE (amd64).
Thanks!
-
Check out the hardware requirements page -> https://www.pfsense.org/hardware
In order to attain speeds approaching Gigabit, you need server class hardware with PCIe NICs:
501+ Mbit = "Multiple cores at > 2.0GHz are required. Server class hardware with PCI-e network adapters."
Unfortunately, you're looking at buying new hardware 'cause that tiny Celeron box is not going to cut it. Also, don't forget about your switch. Verify you can attain 900+ Mbit locally on your switch or it's not going to matter what you do with PFsense.
-
Unfortunately, you're looking at buying new hardware 'cause that tiny Celeron box is not going to cut it. Also, don't forget about your switch. Verify you can attain 900+ Mbit locally on your switch or it's not going to matter what you do with PFsense.
Thanks for the advice. I'll look at the hardware, but from what I have come across during my searches is that the n2930 chip should be able to handle it. It scores a 1665 on the Passmark where an i7-610 scored a 1900, so it's not a slouch of a processor. I have yet to see the processor go above 40-50%, and RAM usage has been minimal given I typically have 6Gb free at all times.
I've checked the wiring - from the ONT to the router is all cat6, maybe 50 feet. All tests have been ran directly connected to the router. I verified I was getting >800 from my iMac to my router through the netgear switch, so that isn't the issue.
Two things I'm curious about - how accurate is speedtest.net and Centurylink's speed test? For both of those I am getting close to 600. But when I test downloading a torrent, I've never gone about 10M/s.
Who knows, it may come down to hardware. I just need to ponder whether or not it is worth spending $$ on a new system when this one is only a month or so old, or do I want to continue with $120/month gigabit line that I am not utilizing. If I can prove that I'm running stable in the 600 range, I'll be happy with that until I decide to upgrade hardware.
-
Since you're in the US, I would recommend the speed test at www.dslreports.com/speedtest. It's not flash-based like speedtest.net (close other browser tabs to make sure the results aren't tainted by poor browser performance), and uses simultaneous connections to multiple geographically diverse servers (similar to how a torrent would likely be received). Speedtest.net uses multiple connections, but I believe they're all to the same server at the location selected.
-
Thanks for the advice. I checked dslreports and it came back with approximately the same results. However, this time I took some screenshots of what the system was doing.
top -P showed that yes, the processors are working, but I wouldn't call them incapable. CPU 3 was at 91%, the other 3 were hovering in the 30-40% range. Memory usage was negligible.
PFSense showed about 60% total utilization.
I'm 'OK' with 410 for now, but would still like to see more. I may look at bumping up the specs, but for now, are there any tweaks I should know about?
Thanks!
![CPU Usage.png](/public/imported_attachments/1/CPU Usage.png)
![CPU Usage.png_thumb](/public/imported_attachments/1/CPU Usage.png_thumb) -
you can start by disabling/removing squid.
if that doesn't help, you can try the new 2.3-rc snapshots. they are fairly stable & generally boost performance by some margin.
-
ROFL. yeah Squid is hosing your CPU. Hard to keep up with 1Gb. You may find you no longer need to cache data locally when you no longer have a sub-8Mb connection.
I laugh because that's a good problem to have.
-
Yeah, its a good problem to have! I may work on the caching settings, because I mainly use squid/sarg for reporting on my children's online activities/websites visited, etc. I don't feel it's necessary, but the wife instructed me to do so. So yeah, kinda stuck there lol. Unless anyone knows of another pkg that will report on actual page visits, not just the top level domain?
For example, sarg gives me this detail - www.dslreports.com/speedtest/3507765 as opposed to just www.dslreports.com
-
There are many different things that could be wrong in this case to archive 1 GBit/s at the WAN Port.
Not all CPUs or their cores could or should be comparable each against the others. And I am really sure
that the Intel Core i7 is blasting the Intel N2930 away, whatever was shown on a CPU comparison list,
because we are talking here about Layer3 routing and forwarding and not other things.So to be on the safe side you should using iPerf on two machines, one as the server and one as the client
and one in front of the WAN Port and the other on the LAN behind the WAN area. Speed tests over the
Internet are also measuring the Internet connection speed with all its bads and goods, and not purely
your pfSense hardware except the WAN routing performance.pfSense is a software firewall that is able to route network traffic also not a plain router likes DD-WRT or
OpenWRT or the most consumer home routers, they realize and work it out mostly done in silicon or by
the help of an ASIC/FPGA that would not being the same as a x86_64 based software firewall.If you really want to know what the Internet account and your pfSense hardware will be able to realize
it should be better to do a fresh install with a 64Bit version of pfSense 2.2.6 or 2.3RC, configuring the
WAN and LAN part and then do the measuring. No packets, no other services, no extra features or options
enabled, no VLANs, massively VPN, DPI or QoS tasks running beside of this set up.
Fresh install and plain configuration.Otherwise this would be not really matching the real world facts as I see it right, others may see this
different for sure, but together with Squid or SquidGuard or Snort or pfBlockerNG and other packets
each of them will eat some CPU power and narrows down the entire speed and throughput of your
pfSense box, for sure there are many CPUs that are really strong and powerful and they will route
1 GBit/s beside of any other installed packet likes the Intel 4 Core i3, i5, i7 or Xeon E3 or E5 CPUs
running @3,xGHz. But this is then not really electric power saving at all.This board here is running from 1,86GHz till 2,16GHz with the same CPU!
LinkSo I would suggest at first;
- enable PowerD (hi adaptive or adaptive)
lets scale the cpu frequency from its minimum to its maximum likes needed - enable TRIM support if a SSD or mSATA is in usage
also not a must be but I personally fell better with it - the mbuf size was set to 1000000 (not 1,000,000)
if it will be needed and not as a standard procedure - perhaps high up the RAM size for Squid
by default Squid is using only 256 MB of RAM
And if all of this will be not gain the entire throughput or WAN speed you really should have a look for
stronger hardware. Also Squid can be fine tuned, what to cache, how great the objects should be that
must be cached and the mode Squid is running on. - enable PowerD (hi adaptive or adaptive)
-
Thats a very good reply, thank you. I have some time this weekend that I can take the network down and check using iperf on the wan and lan ports of the router. I'm not running any packages other than squid, no VPN, no QoS, etc. Just a PPPoE VLAN to connect to CenturyLink.
This board here is running from 1,86GHz till 2,16GHz with the same CPU!
LinkI can't tell if you are saying thats a good or bad thing? Thats the same board I am running.
I have powerd set to hi-adaptive currently, TRIM support is enabled on my SSD, I upped the squid memory to 512 and verified mbuf is 1000000.
I'll respond more sometime this weekend after I conduct a little more testing. Thanks!