Gigabit Fiber, N550 vs. D2500 vs. N2930 vs. i3-3225, Intel vs. Realtek NICs
-
I recently switched to fiber broadband with gigabit up and down, and I thought I would share (also for my own record) my experience with my pfsense setup.
Before switching, I had cable broadband with 250/10 speed. After switching to fiber, I noticed my original setup was limiting the throughput.
Maximum bandwidth (plugged directly into modem): ~940/940
Setup #1 (original):
Hardware: Jetway NC9C-550, N550, Realtek NICs, 2MB RAM, SSD, pfsense 2.2.6
Throughput: ~350/350 (behind switches)Setup #2:
Hardware: Intel D2500CC, D2500, Intel NICs, 4MB RAM, SSD, pfsense 2.4.2
Throughput: ~500/940 (connected directly to pfsense)Setup #3:
Hardware: Jetway HBJC311U93W-2930-B, N2930, Intel NICs, 4MB RAM, SSD, pfsense 2.4.2
Throughput: ~850/940 (directly to pfsense), ~775/940 (behind switches)Setup #4:
Hardware: Intel DQ77, i3-3225, Intel NICs, 16MB RAM, SSD, pfsense 2.4.2
Throughput: ~940/940 (directly to pfsense)I settled on #3 because of it's small and fanless. There might be room to optimize, but it's (more than) enough for my needs.
This is just FYI. If you have gigabit broadband, it would be great if you can share your experience.
Cheers!
-
I recently switched to fiber broadband with gigabit up and down, and I thought I would share (also for my own record) my experience with my pfsense setup.
Before switching, I had cable broadband with 250/10 speed. After switching to fiber, I noticed my original setup was limiting the throughput.
Maximum bandwidth (plugged directly into modem): ~940/940
Setup #1 (original):
Hardware: Jetway NC9C-550, N550, Realtek NICs, 2MB RAM, SSD, pfsense 2.2.6
Throughput: ~350/350 (behind switches)Setup #2:
Hardware: Intel D2500CC, D2500, Intel NICs, 4MB RAM, SSD, pfsense 2.4.2
Throughput: ~500/940 (connected directly to pfsense)Setup #3:
Hardware: Jetway HBJC311U93W-2930-B, N2930, Intel NICs, 4MB RAM, SSD, pfsense 2.4.2
Throughput: ~850/940 (directly to pfsense), ~775/940 (behind switches)Setup #4:
Hardware: Intel DQ77, i3-3225, Intel NICs, 16MB RAM, SSD, pfsense 2.4.2
Throughput: ~940/940 (directly to pfsense)I settled on #3 because of it's small and fanless. There might be room to optimize, but it's (more than) enough for my needs.
This is just FYI. If you have gigabit broadband, it would be great if you can share your experience.
Cheers!
I found the netbook boxes not very reliable for the GFiber 1Gbps speed, IMHO. I ended up building a PC around the AMD FM2+ platform with the A10-8750K
https://forum.pfsense.org/index.php?topic=141740.0
-
I am so glad to find this post! I am a newbie to pfSense. I have Gig fiber to my home. I can get approx 940/920 Mb at my demark. I have tried pfSense on three different machines connected directly to the demark and measured at a switch on the LAN side. I see numbers varying in the mid 300s to just under 400 Mb each direction. The CPU usage on each machine remains under 2 or 3% so it does not appear to be CPU related. I am just starting to look at NIC hardware as a possible issue. It just floors me how much throughput is lost through the firewall. I would love to hear your thoughts on this.
Machine 1 is a Gigabyte EX58 MB (dial NICs) with:
Intel(R) Xeon(R) CPU W3680 @ 3.33GHz
12 CPUs: 1 package(s) x 6 core(s) x 2 hardware threads
AES-NI CPU Crypto: Yes (active)Machine 2 is a HP DL380 G5 (don't have this one up right now to copy CPU info)
Machine 3 is a HP DL380 G7 with:
Intel(R) Xeon(R) CPU X5675 @ 3.07GHz
24 CPUs: 2 package(s) x 6 core(s) x 2 hardware threads
AES-NI CPU Crypto: Yes (active)I didn't do a clean install on the G7. I moved the drive (SSD) from the G5 to the G7 for a quick comparison. I'll try a clean install on the G7 in the next few days to be certain of correct NIC drivers.
-
That sounds like you are not doing something your specific ISP requires such as putting your traffic on a specific VLAN or setting a VLAN priority. Or you're just connecting things wrong.
Those Xeons should easily be able to saturate gigabit.
What is the NIC hardware? 9Are they igb, re, rl, or ??)
-
Hi,
I will research the NIC hardware this weekend. I realized another variable I had introduced and then forgot about. When I made the previous good measurements I was not using the Verizon supplied Quantum Gateway router. I had the Verizon Optical Network Terminal (ONT) connected directly to a Cisco ASA 5550 with the 5550 doing the DHCP on the WAN. That is where I measured the better performance. Later I introduced the Quantum Gateway router directly connected to the ONT which is Verizon's preferred architecture. That facilitates running a Verizon speed test router to test server without other variables on the consumer end. All of my router to test server tests yield performance above 900 Mb/s in both directions.Since my post I have run several tests with different computers plugged into the Quantum Gateway router as are the pfSense systems. I am getting a little better than what I measure behind pfSense, but still in the 600 to 700 Mb/s range. It looks like the Quantum Gateway router is not able to service the provisioned rates. Verizon claims that it is capable. Perhaps it is in theory but that does not seem to hold up under measurement. The www.dslreports.com/speedtest site indicates a serious BufferBloat issue. Note: I am aware there are many external and internal variables with any speed test. I get the feel by running many tests to several different speed test sites and looking at where they tend to average.
One of the reasons I am evaluating pfSense is a candidate to replace my aging ASA-5550s that I originally picked up used on e-bay. I do not have a source for Cisco firmware updates so they become less secure over time. I would end up with more than one instance (I have several independent internal networks in my home) so I do want to keep a router up front directly behind the ONT with the separate pfSense instances behind that. I am currently looking at options in that area.
Sorry I got long winded here. Just letting you know that the numbers I reported are not fully attributable to the pfSense box. There may be a component caused by the NICs that I will address later, but I need to eliminate the impact of the Quantum Gateway router first so it isn't contributing to the issue.
Thanks!
David -
Update: My problem is solved.
It was NOT the NICs at all. I had turned in the Traffic Shaper and set just under the max provisioned rates. Apparently the default algorithm was in conflict in some way with the FIOS algorithm and it had a large negative impact on throughput.
I deleted the Traffic Shaper I now see around 900 Mb/s both directions with a maximum of 5% CPU utilization on my pfSense box.
