SG-1100 Throughput Test



  • Hi,

    if have done a short test with iperf3 on a SG1100. With a direct connection to the testserver I get this results

    [  4]   0.00-1.00   sec   112 MBytes   940 Mbits/sec    8    539 KBytes       
    [  4]   1.00-2.00   sec   111 MBytes   930 Mbits/sec    0    543 KBytes       
    [  4]   2.00-3.00   sec   112 MBytes   936 Mbits/sec    0    680 KBytes       
    [  4]   3.00-4.00   sec   111 MBytes   928 Mbits/sec    0    792 KBytes       
    [  4]   4.00-5.00   sec   111 MBytes   930 Mbits/sec    0    894 KBytes       
    [  4]   5.00-6.00   sec   111 MBytes   929 Mbits/sec    0    984 KBytes       
    [  4]   6.00-7.00   sec   110 MBytes   926 Mbits/sec    0   1.04 MBytes       
    [  4]   7.00-8.00   sec   111 MBytes   929 Mbits/sec    0   1.12 MBytes       
    [  4]   8.00-9.00   sec   111 MBytes   931 Mbits/sec    0   1.19 MBytes       
    [  4]   9.00-10.00  sec   110 MBytes   927 Mbits/sec    0   1.26 MBytes       
    - - - - - - - - - - - - - - - - - - - - - - - - -
    [ ID] Interval           Transfer     Bandwidth       Retr
    [  4]   0.00-10.00  sec  1.08 GBytes   931 Mbits/sec    8             sender
    [  4]   0.00-10.00  sec  1.08 GBytes   927 Mbits/sec                  receiver
    
    

    When running the same test with a sg1100 between I get this.

    [  4]   1.00-2.00   sec  83.5 MBytes   700 Mbits/sec    7    403 KBytes       
    [  4]   2.00-3.00   sec  84.6 MBytes   710 Mbits/sec    0    544 KBytes       
    [  4]   3.00-4.00   sec  83.3 MBytes   699 Mbits/sec    3    474 KBytes       
    [  4]   4.00-5.00   sec  86.1 MBytes   722 Mbits/sec    0    598 KBytes       
    [  4]   5.00-6.00   sec  85.2 MBytes   715 Mbits/sec    3    520 KBytes       
    [  4]   6.00-7.00   sec  83.7 MBytes   702 Mbits/sec    0    632 KBytes       
    [  4]   7.00-8.00   sec  85.8 MBytes   720 Mbits/sec   48    567 KBytes       
    [  4]   8.00-9.00   sec  85.5 MBytes   717 Mbits/sec    3    499 KBytes       
    [  4]   9.00-10.00  sec  82.3 MBytes   690 Mbits/sec    0    614 KBytes       
    - - - - - - - - - - - - - - - - - - - - - - - - -
    [ ID] Interval           Transfer     Bandwidth       Retr
    [  4]   0.00-10.00  sec   844 MBytes   708 Mbits/sec  100             sender
    [  4]   0.00-10.00  sec   840 MBytes   705 Mbits/sec                  receiver
    

    No optimation or configuration changes. plain vanilla pfsense after initial setup.

    I dont want to complain. Is this the expected performance, is there any tuning possible or is there an error in my tests?

    Regards
    Hagen


  • LAYER 8 Global Moderator

    So these are just 2 devices connected to the sg1100 switch, or they are being routed and natted?



  • Test 1 is: client -> gigabitswitch -> server
    Test 2 is: client -> sg1100 - gigabitswitch - server


  • LAYER 8 Global Moderator

    Yeah I get you have the sg1100 in the middle... So your routing this, ie client on your lan and server on the wan.. Or server on the lan and client on the wan and you did a port forward. Or you have client and server connected to the sg1100 switch ports and they are both on lan.

    Is client on lan and server on opt and you setup vlans on opt?

    So you have added another cable in the mix as well with client to sg1100 and then sg1100 to switch, and then server on switch. Did you validate cable good, etc.

    Your questions is GREAT and yeah you would expect in the 900's I would hope even when natting and routing.. But you need to be a bit more specific on your testing connection method.

    Says right on the sg1100 page
    "For users seeking an excellent firewall with up to 1 Gbps throughput"

    Up to 1 Gbps to me would mean natting and routing at least close to 900mbps.. Not the 700 your seeing... So lets understand exactly how your doing the test.

    And also going to yell for @Derelict since he has a huge lab and would think he has a sg1100 to play with ;) to duplicate your testing of its throughput.



  • @johnpoz said in SG-1100 Throughput Test:

    So you have added another cable in the mix

    autsch. embarrassing and I will not lie to you. The new cable was a cat5.

    If I change it to a cat6 I get.

    [  4]   4.00-5.00   sec   105 MBytes   884 Mbits/sec    1    556 KBytes       
    [  4]   5.00-6.00   sec   105 MBytes   884 Mbits/sec    0    687 KBytes       
    [  4]   6.00-7.00   sec   106 MBytes   887 Mbits/sec    0    796 KBytes       
    [  4]   7.00-8.00   sec   100 MBytes   840 Mbits/sec    6    527 KBytes       
    [  4]   8.00-9.00   sec   105 MBytes   883 Mbits/sec    0    663 KBytes       
    [  4]   9.00-10.00  sec   106 MBytes   886 Mbits/sec    0    778 KBytes       
    - - - - - - - - - - - - - - - - - - - - - - - - -
    [ ID] Interval           Transfer     Bandwidth       Retr
    [  4]   0.00-10.00  sec  1.01 GBytes   868 Mbits/sec   28             sender
    [  4]   0.00-10.00  sec  1.01 GBytes   866 Mbits/sec                  receiver
    
    

    The flow is

    Client -> lan port - SG1100 - wan port -> gigabit switch -> server


  • LAYER 8 Netgate

    What are the exact iperf commands you are running?

    High 800s is pretty respectable for single-stream ARM, IMHO.

    I didn't get that notification. Wonder if it's case-sensitive @Johnpoz.


  • LAYER 8 Global Moderator

    Odd you didn't get the notification. But cat 5 is more than capable of doing gig.. Maybe just something wrong with that cable. But I would say that mid 800's is pretty respectable.

    Also you still haven't stated how you have it all connected.. I assume server or client is on wan, and other side is on the lan. But there are multiple ways this could all be connected. Which you really need to clarify.

    The actual valid test would be server on wan, and client on lan. While doing nat - which is the typical setup.


  • LAYER 8 Netgate

    We have seen crappy little "gigabit" switches that couldn't pass gig-e rates. Pull the switch and 942Mb.


  • LAYER 8 Global Moderator

    but he said when he was testing with just his switch he was seeing 940.



  • Thank you for the fast feedback.

    To further simplify the test setup I put the "Server" into the same vlan as the client. Since this was the same in both test it should not matter but anyway.

    Now I have

    • Server: is the same on both test - running iperf3 -s
    • Cable Server -> Switch - is the same on both test
    • Switch: 1GB - is the same on both test
    • Cable Switch -> [Client | SG1100] - is the same on both tests
    • SG1100 - Standard installation. no additional rules / nat /...
    • Cable SG100 -> Client: Cat 6
    • Client Command : iperf3 -c ip.address.server

    Testsetup

    Test 1 is: client -> gigabitswitch -> server
    Test 2 is: client -> LANPORT sg1100 WANPORT - gigabitswitch - server

    The only difference I can see between the two test is the fact that in test 2 the sg1100 plus one cable is added to the flow between client and gigabit switch.

    Test summary: The SG1100 "adds" 50MBits/sec to the result.

    As i said. It is not that I am unhappy. I just would like to confirm that this is "expected" or if there are some additional tuning possible

    Result test 1

    Connecting to host "server ip", port 5201
    [  4] local "client-ip" port 40144 connected to "server-ip" port 5201
    [ ID] Interval           Transfer     Bandwidth       Retr  Cwnd
    [  4]   0.00-1.00   sec   112 MBytes   935 Mbits/sec    0    409 KBytes       
    [  4]   1.00-2.00   sec   111 MBytes   934 Mbits/sec    0    430 KBytes       
    [  4]   2.00-3.00   sec   112 MBytes   939 Mbits/sec    0    604 KBytes       
    [  4]   3.00-4.00   sec   111 MBytes   933 Mbits/sec    0    604 KBytes       
    [  4]   4.00-5.00   sec   111 MBytes   932 Mbits/sec   37    428 KBytes       
    [  4]   5.00-6.00   sec   108 MBytes   909 Mbits/sec    0    501 KBytes       
    [  4]   6.00-7.00   sec   109 MBytes   915 Mbits/sec   37    387 KBytes       
    [  4]   7.00-8.00   sec   111 MBytes   933 Mbits/sec    0    406 KBytes       
    [  4]   8.00-9.00   sec   110 MBytes   927 Mbits/sec    0    426 KBytes       
    [  4]   9.00-10.00  sec   110 MBytes   921 Mbits/sec  111    334 KBytes       
    - - - - - - - - - - - - - - - - - - - - - - - - -
    [ ID] Interval           Transfer     Bandwidth       Retr
    [  4]   0.00-10.00  sec  1.08 GBytes   928 Mbits/sec  185             sender
    [  4]   0.00-10.00  sec  1.08 GBytes   925 Mbits/sec                  receiver
    

    Result test 2

    Connecting to host "server-ip", port 5201
    [  4] local "client ip" port 37190 connected to "server-ip" port 5201
    [ ID] Interval           Transfer     Bandwidth       Retr  Cwnd
    [  4]   0.00-1.00   sec   100 MBytes   840 Mbits/sec   25    549 KBytes       
    [  4]   1.00-2.00   sec   105 MBytes   879 Mbits/sec    0    682 KBytes       
    [  4]   2.00-3.00   sec   105 MBytes   878 Mbits/sec    0    789 KBytes       
    [  4]   3.00-4.00   sec   105 MBytes   881 Mbits/sec    2    667 KBytes       
    [  4]   4.00-5.00   sec   105 MBytes   882 Mbits/sec    0    776 KBytes       
    [  4]   5.00-6.00   sec   105 MBytes   883 Mbits/sec    5    655 KBytes       
    [  4]   6.00-7.00   sec   105 MBytes   883 Mbits/sec    0    766 KBytes       
    [  4]   7.00-8.00   sec   105 MBytes   884 Mbits/sec    9    639 KBytes       
    [  4]   8.00-9.00   sec   102 MBytes   860 Mbits/sec    0    752 KBytes       
    [  4]   9.00-10.00  sec   105 MBytes   880 Mbits/sec    6    625 KBytes       
    - - - - - - - - - - - - - - - - - - - - - - - - -
    [ ID] Interval           Transfer     Bandwidth       Retr
    [  4]   0.00-10.00  sec  1.02 GBytes   875 Mbits/sec   47             sender
    [  4]   0.00-10.00  sec  1.02 GBytes   872 Mbits/sec                  receiver
    

  • LAYER 8 Netgate

    Yeah.

    The SG-1100 is designed with a single gigabit link to a switch chip. That switch chip breaks out to WAN, LAN, and OPT. Everyone wanted pfSense on an espresso.bin. This is how the espresso.bin is designed.

    So it is essentially a router on a stick.

    880Mbit/sec is pretty impressive in that context.



  • @derelict thank you. It may look so but I really dont want to water down our wine. I am really impressed.

    I should have said "only adds 50MBits/sec". This device will serve every internet connection you can normally buy for decent money in Germany.

    I believe from a price value point this is excellent given the fact that the powerful pfSense capabilities are within this small little device



  • I can't get more than 250 Mbps with a Vanilla install. Which version of pfSense are you running? Mine is 2.4.4



  • @JInx-IT plain vanilla pfsense latest version that was installed out of the box. 2.4.4-p2



  • You are correct. I have, however, heard that people running 2.3.* are getting gigabit speeds. I was wondering if it was a 2.4.* issue that was throttling me to around 250 Mpbs. If other people are getting Gigabit, or close, speeds in 2.4.*, I'd like to know what they are running and how it's configured. My gut says I have something misconfigured, but I don't have a clue what it would be. I was hoping I could compare between my setup and another person's who was getting at least closer to Gigabit speeds, with the same hardware.



  • @JInx-IT and what are your results with 2.4.4-p2 and what is your test environment?


  • LAYER 8 Netgate

    Not sure what you're talking about since the SG-1100 was never supported by anything older than pfSense 2.4.4-p1.


  • LAYER 8 Global Moderator

    Maybe he is confusing the SG-1100, with the 1000?



  • I've read posts from people claiming to be running an old Dell desktop with a couple of gigabit cards, running pfSense 2.3.*, getting in the high 990 Mbps. I can plug my laptop straight into the modem and get the same. High 990 Mbps, no problem. When I put the Netgate SG-1100 between my laptop and the modem, my speeds go down to 250 Mbps or lower.



  • @JInx-IT you installed 2.3 on a sg -1100?


  • LAYER 8 Global Moderator

    @JInx-IT said in SG-1100 Throughput Test:

    getting in the high 990 Mbps.

    Nonsense - since not possible to get that on a gig interface... Do the math yourself..



  • That's with no traffic shaping or anything. It's just the basic setup.


  • LAYER 8 Global Moderator

    Dude you have been show simple out of the box the SG-1100 doing high 800's if your only seeing 250ish you got something wrong with your testing method or your hardware in the path of your test.

    You have yet to show your test method. Sorry but you aint going to see 990s on any gig interface. your going to be in the 940's as max..

    Duplicate the testing that hbauer did above.



  • I never said I am running 2.3.*, I'm running what came on it, which is 2.4.4-p2. My question was, and still is, how to achieve Gigabit speeds with an SG-1100, when it looks like the hardware itself can't support anything more than 250 Mbps. I'm hoping I'm wrong, so I'm here looking for proof that I'm configuring something incorrectly. Why can my laptop to the modem get Gigabit speeds, but a basic config in the SG-1100 maxes out at 250 Mbps? The answer, I feel, is in looking at the differences between my configuration, and somebody's who has the correct config in their SG-1100, and by correct config, I mean somebody who is getting Gigabit speeds.



  • @JInx-IT I am just a netgate customer and I can confirm the results posted above. The hardware can do it without any changes to the standard config. I suggest to open another topic with more details about your modem. I suspect that there is a need for some changes.


  • LAYER 8 Global Moderator

    Dude out of the box it will do gig speeds... You have been shown this and confirmed by multiple people..

    Benchmarks are listed here as well
    https://www.netgate.com/blog/netgates-new-sg-1100-punches-way-above-its-weight.html



  • The modem is setup the way we have had things setup for the last 4 years worth of using a pfSense router, which we traditionally ran off of a SuperMicro system. Pretty much all of our clients are running one, and they aren't going much past 200 Mbps. When we went up to over 400 Mbps, we noticed the router was bottlenecking the speeds, because bypassing the router and running a laptop straight to the modem produced the advertised speeds. The same is holding true with the SG-1100. Doing a speedtest from the router maxes out at 250 Mbps, but taking the router out, and plugging the laptop directly into the modem, the laptop gets Gigabit speeds. What we need, is a router we can suggest to our clients that will get Gigabit speeds. The issue is, it looks like we will have to use some cast off desktops to get that going, instead of using a professional hardware package. What I'm hoping for is that I'm missing something in the config settings that is impeding the throughput. I would love to offer this box as a Gigabit solution.



  • @johnpoz So, are you saying that it's a defective SG-1100? Because, out of the box, it's not getting gig speeds. Not even close.



  • @JInx-IT Do you have a change to try my setup with iperf connected to a switch? If this is better then you know its a configuration thing. if not open a ticket at netgate


  • LAYER 8 Global Moderator

    @JInx-IT said in SG-1100 Throughput Test:

    it's not getting gig speeds. Not even close.

    And you have yet to show you can actually do a valid test... Lets see your test method and showing that your client and server can actually talk doing gig, etc.

    hbauer gave exacting details of testing done showing 880's

    If after showing valid test methods and default config - then yeah open a support ticket.

    Doing a speed test to some internet site with some client behind pfsense is NOT a valid test method.



  • What I use for speedtest from pfSense and any other Linux based platform is speedtest-cli. If that's not accurate enough, I am certainly open to new methods.


  • Rebel Alliance Netgate Administrator

    @JInx-IT If you are testing it from the firewall, that is not accurate. You need to test from a device behind the firewall to a public device on the other side.



  • I get close to she same results whetherI'm running it from the firewall or the laptop behind the firewall, where the big jump comes is if I remove the firewall and configure the laptop to run straight off the modem. What do you recommend I run instead of speedtest-cli?


  • Rebel Alliance Netgate Administrator

    You could try using iperf/iperf3 to test from local to remote. As noted your speeds are less then anticipated for the device; more than likely the configuration/testing methods are not right.

    The more information use can share the better someone can assist you in resolving your problems.


  • Banned

    @JInx-IT
    Connect one device to WAN, and run iperf in server mode on it. Connect one device to LAN and let it run an iperf test to the server you just setup on the WAN device.

    Also make sure there are no limiters/traffic shaping configured, especially if you restored a config backup from a previous device. Make sure you are not testing over a VPN connection, that will reduce throughput considerably.

    Also if your Internet connection is using PPPoE that might be a limiting factor too.



  • @hbauer said in SG-1100 Throughput Test:

    iperf3

    I'll look into iperf again. I skimmed over it and moved on because the closest servers it listed were on the other side of the country from me. I didn't see, or think of, an option to create your own test server on the WAN. I'll probably set that up tomorrow. I wasn't able to restore from backup to the sg-1100, so that's not a factor and I'm not using a VPN or PPPoE for any of this. I'll post my results after I get everything set up and run it both ways.


  • LAYER 8 Global Moderator

    so did you test this? What were your results... I maintain a windows copy I compile myself for iperf3 if you want the latest and greatest version 3.6



  • I understand some time has passed on this thread, but I also see the slower speeds mentioned for this firewall.

    Using iperf3 client-to-server through a cheap 1G switch I consistently get about 930 Mbits/sec. Using the same cables, client, and server through the SG-1100 LAN-to-WAN I consistently get about 445 Mbits/sec. The results are slightly slower if I use the built-in iperf3 package within pfSense as the server with the client on the LAN link. The netgate is running 2.4.4_3.

    To specify the lab: a laptop is configured with a static IP in the WAN IP scope, is directly attached to the WAN port with a Cat6 cable, and listens with iperf3 as server. A laptop is configured via DHCP in the LAN IP scope, is attached to the LAN port with a Cat6 cable, and runs iperf3 as a client. Outside (WAN) laptop command is “iperf3 -p 5001 -s”. Inside (LAN) laptop command is “iperf3 -p 5001 -c <static IP of the WAN connected laptop>”.

    This is a fresh install of pfSense with no special sauce. The LAN→WAN firewall rule is an IPv4* any any.

    Now the really crazy part happens when I run the netgate built-in iperf3 package as a client. Running the same laptop as server on the LAN link, the netgate consistently gets about 865 Mbits/sec! Reverse this with the built-in iperf3 as server and the same laptop as client on the LAN link, back to mid 445 Mbits/sec. Huh?!?



  • @testgate said in SG-1100 Throughput Test:

    I understand some time has passed on this thread, but I also see the slower speeds mentioned for this firewall.

    Using iperf3 client-to-server through a cheap 1G switch I consistently get about 930 Mbits/sec. Using the same cables, client, and server through the SG-1100 LAN-to-WAN I consistently get about 445 Mbits/sec. The results are slightly slower if I use the built-in iperf3 package within pfSense as the server with the client on the LAN link. The netgate is running 2.4.4_3.

    To specify the lab: a laptop is configured with a static IP in the WAN IP scope, is directly attached to the WAN port with a Cat6 cable, and listens with iperf3 as server. A laptop is configured via DHCP in the LAN IP scope, is attached to the LAN port with a Cat6 cable, and runs iperf3 as a client. Outside (WAN) laptop command is “iperf3 -p 5001 -s”. Inside (LAN) laptop command is “iperf3 -p 5001 -c <static IP of the WAN connected laptop>”.

    This is a fresh install of pfSense with no special sauce. The LAN→WAN firewall rule is an IPv4* any any.

    Now the really crazy part happens when I run the netgate built-in iperf3 package as a client. Running the same laptop as server on the LAN link, the netgate consistently gets about 865 Mbits/sec! Reverse this with the built-in iperf3 as server and the same laptop as client on the LAN link, back to mid 445 Mbits/sec. Huh?!?

    netgate-iperf3-results.png


  • Netgate Administrator

    Hmm, what happens if you keep the server on the SG-1100 but run the client with the reverse option -R?

    Or run the client on the SG-1100 with reverse?

    The default in iperf is to have the client send traffic to the server (which always seemed an odd decision to me!). So in that test you are seeing nearly 900Mbps when it's sending but less than half that when it's receiving.

    Running either client with -R reverses the traffic but keeps the states opening the same way. So swapping it determines if it's the way the firewall opens states or the traffic direction.

    Steve


Log in to reply