SG-1100 Throughput Test
-
We have seen crappy little "gigabit" switches that couldn't pass gig-e rates. Pull the switch and 942Mb.
-
but he said when he was testing with just his switch he was seeing 940.
-
Thank you for the fast feedback.
To further simplify the test setup I put the "Server" into the same vlan as the client. Since this was the same in both test it should not matter but anyway.
Now I have
- Server: is the same on both test - running iperf3 -s
- Cable Server -> Switch - is the same on both test
- Switch: 1GB - is the same on both test
- Cable Switch -> [Client | SG1100] - is the same on both tests
- SG1100 - Standard installation. no additional rules / nat /...
- Cable SG100 -> Client: Cat 6
- Client Command : iperf3 -c ip.address.server
Testsetup
Test 1 is: client -> gigabitswitch -> server
Test 2 is: client -> LANPORT sg1100 WANPORT - gigabitswitch - serverThe only difference I can see between the two test is the fact that in test 2 the sg1100 plus one cable is added to the flow between client and gigabit switch.
Test summary: The SG1100 "adds" 50MBits/sec to the result.
As i said. It is not that I am unhappy. I just would like to confirm that this is "expected" or if there are some additional tuning possible
Result test 1
Connecting to host "server ip", port 5201 [ 4] local "client-ip" port 40144 connected to "server-ip" port 5201 [ ID] Interval Transfer Bandwidth Retr Cwnd [ 4] 0.00-1.00 sec 112 MBytes 935 Mbits/sec 0 409 KBytes [ 4] 1.00-2.00 sec 111 MBytes 934 Mbits/sec 0 430 KBytes [ 4] 2.00-3.00 sec 112 MBytes 939 Mbits/sec 0 604 KBytes [ 4] 3.00-4.00 sec 111 MBytes 933 Mbits/sec 0 604 KBytes [ 4] 4.00-5.00 sec 111 MBytes 932 Mbits/sec 37 428 KBytes [ 4] 5.00-6.00 sec 108 MBytes 909 Mbits/sec 0 501 KBytes [ 4] 6.00-7.00 sec 109 MBytes 915 Mbits/sec 37 387 KBytes [ 4] 7.00-8.00 sec 111 MBytes 933 Mbits/sec 0 406 KBytes [ 4] 8.00-9.00 sec 110 MBytes 927 Mbits/sec 0 426 KBytes [ 4] 9.00-10.00 sec 110 MBytes 921 Mbits/sec 111 334 KBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bandwidth Retr [ 4] 0.00-10.00 sec 1.08 GBytes 928 Mbits/sec 185 sender [ 4] 0.00-10.00 sec 1.08 GBytes 925 Mbits/sec receiver
Result test 2
Connecting to host "server-ip", port 5201 [ 4] local "client ip" port 37190 connected to "server-ip" port 5201 [ ID] Interval Transfer Bandwidth Retr Cwnd [ 4] 0.00-1.00 sec 100 MBytes 840 Mbits/sec 25 549 KBytes [ 4] 1.00-2.00 sec 105 MBytes 879 Mbits/sec 0 682 KBytes [ 4] 2.00-3.00 sec 105 MBytes 878 Mbits/sec 0 789 KBytes [ 4] 3.00-4.00 sec 105 MBytes 881 Mbits/sec 2 667 KBytes [ 4] 4.00-5.00 sec 105 MBytes 882 Mbits/sec 0 776 KBytes [ 4] 5.00-6.00 sec 105 MBytes 883 Mbits/sec 5 655 KBytes [ 4] 6.00-7.00 sec 105 MBytes 883 Mbits/sec 0 766 KBytes [ 4] 7.00-8.00 sec 105 MBytes 884 Mbits/sec 9 639 KBytes [ 4] 8.00-9.00 sec 102 MBytes 860 Mbits/sec 0 752 KBytes [ 4] 9.00-10.00 sec 105 MBytes 880 Mbits/sec 6 625 KBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bandwidth Retr [ 4] 0.00-10.00 sec 1.02 GBytes 875 Mbits/sec 47 sender [ 4] 0.00-10.00 sec 1.02 GBytes 872 Mbits/sec receiver
-
Yeah.
The SG-1100 is designed with a single gigabit link to a switch chip. That switch chip breaks out to WAN, LAN, and OPT. Everyone wanted pfSense on an espresso.bin. This is how the espresso.bin is designed.
So it is essentially a router on a stick.
880Mbit/sec is pretty impressive in that context.
-
@derelict thank you. It may look so but I really dont want to water down our wine. I am really impressed.
I should have said "only adds 50MBits/sec". This device will serve every internet connection you can normally buy for decent money in Germany.
I believe from a price value point this is excellent given the fact that the powerful pfSense capabilities are within this small little device
-
I can't get more than 250 Mbps with a Vanilla install. Which version of pfSense are you running? Mine is 2.4.4
-
@JInx-IT plain vanilla pfsense latest version that was installed out of the box. 2.4.4-p2
-
You are correct. I have, however, heard that people running 2.3.* are getting gigabit speeds. I was wondering if it was a 2.4.* issue that was throttling me to around 250 Mpbs. If other people are getting Gigabit, or close, speeds in 2.4.*, I'd like to know what they are running and how it's configured. My gut says I have something misconfigured, but I don't have a clue what it would be. I was hoping I could compare between my setup and another person's who was getting at least closer to Gigabit speeds, with the same hardware.
-
@JInx-IT and what are your results with 2.4.4-p2 and what is your test environment?
-
Not sure what you're talking about since the SG-1100 was never supported by anything older than pfSense 2.4.4-p1.
-
Maybe he is confusing the SG-1100, with the 1000?
-
I've read posts from people claiming to be running an old Dell desktop with a couple of gigabit cards, running pfSense 2.3.*, getting in the high 990 Mbps. I can plug my laptop straight into the modem and get the same. High 990 Mbps, no problem. When I put the Netgate SG-1100 between my laptop and the modem, my speeds go down to 250 Mbps or lower.
-
@JInx-IT you installed 2.3 on a sg -1100?
-
@JInx-IT said in SG-1100 Throughput Test:
getting in the high 990 Mbps.
Nonsense - since not possible to get that on a gig interface... Do the math yourself..
-
That's with no traffic shaping or anything. It's just the basic setup.
-
Dude you have been show simple out of the box the SG-1100 doing high 800's if your only seeing 250ish you got something wrong with your testing method or your hardware in the path of your test.
You have yet to show your test method. Sorry but you aint going to see 990s on any gig interface. your going to be in the 940's as max..
Duplicate the testing that hbauer did above.
-
I never said I am running 2.3.*, I'm running what came on it, which is 2.4.4-p2. My question was, and still is, how to achieve Gigabit speeds with an SG-1100, when it looks like the hardware itself can't support anything more than 250 Mbps. I'm hoping I'm wrong, so I'm here looking for proof that I'm configuring something incorrectly. Why can my laptop to the modem get Gigabit speeds, but a basic config in the SG-1100 maxes out at 250 Mbps? The answer, I feel, is in looking at the differences between my configuration, and somebody's who has the correct config in their SG-1100, and by correct config, I mean somebody who is getting Gigabit speeds.
-
@JInx-IT I am just a netgate customer and I can confirm the results posted above. The hardware can do it without any changes to the standard config. I suggest to open another topic with more details about your modem. I suspect that there is a need for some changes.
-
Dude out of the box it will do gig speeds... You have been shown this and confirmed by multiple people..
Benchmarks are listed here as well
https://www.netgate.com/blog/netgates-new-sg-1100-punches-way-above-its-weight.html -
The modem is setup the way we have had things setup for the last 4 years worth of using a pfSense router, which we traditionally ran off of a SuperMicro system. Pretty much all of our clients are running one, and they aren't going much past 200 Mbps. When we went up to over 400 Mbps, we noticed the router was bottlenecking the speeds, because bypassing the router and running a laptop straight to the modem produced the advertised speeds. The same is holding true with the SG-1100. Doing a speedtest from the router maxes out at 250 Mbps, but taking the router out, and plugging the laptop directly into the modem, the laptop gets Gigabit speeds. What we need, is a router we can suggest to our clients that will get Gigabit speeds. The issue is, it looks like we will have to use some cast off desktops to get that going, instead of using a professional hardware package. What I'm hoping for is that I'm missing something in the config settings that is impeding the throughput. I would love to offer this box as a Gigabit solution.
-
@johnpoz So, are you saying that it's a defective SG-1100? Because, out of the box, it's not getting gig speeds. Not even close.
-
@JInx-IT Do you have a change to try my setup with iperf connected to a switch? If this is better then you know its a configuration thing. if not open a ticket at netgate
-
@JInx-IT said in SG-1100 Throughput Test:
it's not getting gig speeds. Not even close.
And you have yet to show you can actually do a valid test... Lets see your test method and showing that your client and server can actually talk doing gig, etc.
hbauer gave exacting details of testing done showing 880's
If after showing valid test methods and default config - then yeah open a support ticket.
Doing a speed test to some internet site with some client behind pfsense is NOT a valid test method.
-
What I use for speedtest from pfSense and any other Linux based platform is speedtest-cli. If that's not accurate enough, I am certainly open to new methods.
-
@JInx-IT If you are testing it from the firewall, that is not accurate. You need to test from a device behind the firewall to a public device on the other side.
-
I get close to she same results whetherI'm running it from the firewall or the laptop behind the firewall, where the big jump comes is if I remove the firewall and configure the laptop to run straight off the modem. What do you recommend I run instead of speedtest-cli?
-
You could try using iperf/iperf3 to test from local to remote. As noted your speeds are less then anticipated for the device; more than likely the configuration/testing methods are not right.
The more information use can share the better someone can assist you in resolving your problems.
-
@JInx-IT
Connect one device to WAN, and run iperf in server mode on it. Connect one device to LAN and let it run an iperf test to the server you just setup on the WAN device.Also make sure there are no limiters/traffic shaping configured, especially if you restored a config backup from a previous device. Make sure you are not testing over a VPN connection, that will reduce throughput considerably.
Also if your Internet connection is using PPPoE that might be a limiting factor too.
-
@hbauer said in SG-1100 Throughput Test:
iperf3
I'll look into iperf again. I skimmed over it and moved on because the closest servers it listed were on the other side of the country from me. I didn't see, or think of, an option to create your own test server on the WAN. I'll probably set that up tomorrow. I wasn't able to restore from backup to the sg-1100, so that's not a factor and I'm not using a VPN or PPPoE for any of this. I'll post my results after I get everything set up and run it both ways.
-
so did you test this? What were your results... I maintain a windows copy I compile myself for iperf3 if you want the latest and greatest version 3.6
-
I understand some time has passed on this thread, but I also see the slower speeds mentioned for this firewall.
Using iperf3 client-to-server through a cheap 1G switch I consistently get about 930 Mbits/sec. Using the same cables, client, and server through the SG-1100 LAN-to-WAN I consistently get about 445 Mbits/sec. The results are slightly slower if I use the built-in iperf3 package within pfSense as the server with the client on the LAN link. The netgate is running 2.4.4_3.
To specify the lab: a laptop is configured with a static IP in the WAN IP scope, is directly attached to the WAN port with a Cat6 cable, and listens with iperf3 as server. A laptop is configured via DHCP in the LAN IP scope, is attached to the LAN port with a Cat6 cable, and runs iperf3 as a client. Outside (WAN) laptop command is “iperf3 -p 5001 -s”. Inside (LAN) laptop command is “iperf3 -p 5001 -c <static IP of the WAN connected laptop>”.
This is a fresh install of pfSense with no special sauce. The LAN→WAN firewall rule is an IPv4* any any.
Now the really crazy part happens when I run the netgate built-in iperf3 package as a client. Running the same laptop as server on the LAN link, the netgate consistently gets about 865 Mbits/sec! Reverse this with the built-in iperf3 as server and the same laptop as client on the LAN link, back to mid 445 Mbits/sec. Huh?!?
-
@testgate said in SG-1100 Throughput Test:
I understand some time has passed on this thread, but I also see the slower speeds mentioned for this firewall.
Using iperf3 client-to-server through a cheap 1G switch I consistently get about 930 Mbits/sec. Using the same cables, client, and server through the SG-1100 LAN-to-WAN I consistently get about 445 Mbits/sec. The results are slightly slower if I use the built-in iperf3 package within pfSense as the server with the client on the LAN link. The netgate is running 2.4.4_3.
To specify the lab: a laptop is configured with a static IP in the WAN IP scope, is directly attached to the WAN port with a Cat6 cable, and listens with iperf3 as server. A laptop is configured via DHCP in the LAN IP scope, is attached to the LAN port with a Cat6 cable, and runs iperf3 as a client. Outside (WAN) laptop command is “iperf3 -p 5001 -s”. Inside (LAN) laptop command is “iperf3 -p 5001 -c <static IP of the WAN connected laptop>”.
This is a fresh install of pfSense with no special sauce. The LAN→WAN firewall rule is an IPv4* any any.
Now the really crazy part happens when I run the netgate built-in iperf3 package as a client. Running the same laptop as server on the LAN link, the netgate consistently gets about 865 Mbits/sec! Reverse this with the built-in iperf3 as server and the same laptop as client on the LAN link, back to mid 445 Mbits/sec. Huh?!?
-
Hmm, what happens if you keep the server on the SG-1100 but run the client with the reverse option
-R
?Or run the client on the SG-1100 with reverse?
The default in iperf is to have the client send traffic to the server (which always seemed an odd decision to me!). So in that test you are seeing nearly 900Mbps when it's sending but less than half that when it's receiving.
Running either client with -R reverses the traffic but keeps the states opening the same way. So swapping it determines if it's the way the firewall opens states or the traffic direction.
Steve
-
Thanks for your reply. Funny timing, I found that switch a few hours ago! Same results. It simply seems that the connection speed is asynchronous. The below is viewed from over the console cable to the SG-1100. The other testing device is a laptop connected to the LAN port. The text wrapped, but the second command had the -R.
I did a full factory reset earlier and had similar results.
Also of potential interest, I setup different scopes on LAN and OPT and tested across them as well as to/from WAN. Tests involving WAN from either LAN or OPT have the same asynchronous results. Tests between OPT and LAN were synchronous, but always below 500 Mbits/sec.
The only other thing I noticed strange is that it seems the netgate is tagging outbound packets with TOS 7. You can even see this activity with simply ping responses from the netgate. Strange.
I really wouldn’t ordinarily notice or care much, but this installation is going into a FIOS delivered service very close to 1Gbps. So the SG-1100 throughput will matter.
In the below capture, the laptop connected directly to LAN was running "iperf -p 5001 -s".
-
Hmm, so still slower receiving.
You might try disabling pf entirely (
pfctl -d
) and testing between LAN and OPT. You should see close to line rate under those conditions unless there is some thing very wrong in the setup.I will say though that your unlikely to see much over 500Mbps from WAN to LAN, even after tuning, with firewalling and NAT in place. If you need close to 1Gbps you should upgrade to the SG-3100.
Steve
-
@testgate said in SG-1100 Throughput Test:
tagging outbound packets with TOS 7
Where you seeing that - I just looked, and no that is not happening..
Please post your sniff showing that.
-
@stephenw10 Thanks again. Here goes:
Followed the below process for LAN<->WAN test:
Reset SG-1100 to factory default via console cable. Set interface(s) IP address via console WAN (mvneta0.4090) IP 10.10.10.1/24, LAN (mvnet0.4091) left default IP – 192.168.1.1 and default DHCP scope. Connected laptop1 to LAN, logged into Web GUI, completed the wizard leaving all defaults, but set new admin password. Laptop2 connected to WAN and set IP 10.10.10.150/24.
On laptop2, ran command “iperf3 -p 5001 -s”
On laptop1, ran command “iperf3 -p 5001 -c 10.10.10.150”
On laptop1, ran command “iperf3 -p 5001 -c 10.10.10.150 -R”
Result screen capture below:Followed the below process for LAN<-> OPT test:
Executed “pfctl -d” command as root via console. Configured OPT (mvneta0.4092) via Web GUI for IP 192.168.100.1/24. Moved laptop2 to OPT and set IP 192.168.100.150/24.
On laptop2, ran command “iperf3 -p 5001 -s”
On laptop1, ran command “iperf3 -p 5001 -c 192.168.100.150”
On laptop1, ran command “iperf3 -p 5001 -c 192.168.100.150 -R”
Result screen capture below:Followed the below process for laptop1<->laptop2 test:
Set laptop1 IP 192.168.100.200. Connected laptop1 directly to laptop2.
On laptop1, ran command “iperf3 -p 5001 -c 192.168.100.150”
On laptop1, ran command “iperf3 -p 5001 -c 192.168.100.150 -R”
Result screen capture below:Taking Steve’s advice, I ordered an SG-3100. We have several of these in production and have had good results. The SG-1100 was/is an experiment. Interesting results.
-
@johnpoz Apologies, I didn’t save the capture where I saw this and could not replicate it again after a factory reset.
-
The SG-3100 is definitely a better option here.
However when you disable pf using 'pfctl -d' it will be enabled again by making any changes in the gui that apply firewall or NAT changes. (unless it's disabled in the GUI). I suspect in the test where you added opt and tested from LAN to OPT it had become enabled again. Better to disable it immediately before the test.
Steve
-
@stephenw10 I think you are correct on both points. Thanks for that, I appreciate your input. The asynchronous results of LAN-WAN testing still baffles me.