SG-1100 Throughput Test


  • LAYER 8 Netgate

    Yeah.

    The SG-1100 is designed with a single gigabit link to a switch chip. That switch chip breaks out to WAN, LAN, and OPT. Everyone wanted pfSense on an espresso.bin. This is how the espresso.bin is designed.

    So it is essentially a router on a stick.

    880Mbit/sec is pretty impressive in that context.



  • @derelict thank you. It may look so but I really dont want to water down our wine. I am really impressed.

    I should have said "only adds 50MBits/sec". This device will serve every internet connection you can normally buy for decent money in Germany.

    I believe from a price value point this is excellent given the fact that the powerful pfSense capabilities are within this small little device



  • I can't get more than 250 Mbps with a Vanilla install. Which version of pfSense are you running? Mine is 2.4.4



  • @JInx-IT plain vanilla pfsense latest version that was installed out of the box. 2.4.4-p2



  • You are correct. I have, however, heard that people running 2.3.* are getting gigabit speeds. I was wondering if it was a 2.4.* issue that was throttling me to around 250 Mpbs. If other people are getting Gigabit, or close, speeds in 2.4.*, I'd like to know what they are running and how it's configured. My gut says I have something misconfigured, but I don't have a clue what it would be. I was hoping I could compare between my setup and another person's who was getting at least closer to Gigabit speeds, with the same hardware.



  • @JInx-IT and what are your results with 2.4.4-p2 and what is your test environment?


  • LAYER 8 Netgate

    Not sure what you're talking about since the SG-1100 was never supported by anything older than pfSense 2.4.4-p1.


  • LAYER 8 Global Moderator

    Maybe he is confusing the SG-1100, with the 1000?



  • I've read posts from people claiming to be running an old Dell desktop with a couple of gigabit cards, running pfSense 2.3.*, getting in the high 990 Mbps. I can plug my laptop straight into the modem and get the same. High 990 Mbps, no problem. When I put the Netgate SG-1100 between my laptop and the modem, my speeds go down to 250 Mbps or lower.



  • @JInx-IT you installed 2.3 on a sg -1100?


  • LAYER 8 Global Moderator

    @JInx-IT said in SG-1100 Throughput Test:

    getting in the high 990 Mbps.

    Nonsense - since not possible to get that on a gig interface... Do the math yourself..



  • That's with no traffic shaping or anything. It's just the basic setup.


  • LAYER 8 Global Moderator

    Dude you have been show simple out of the box the SG-1100 doing high 800's if your only seeing 250ish you got something wrong with your testing method or your hardware in the path of your test.

    You have yet to show your test method. Sorry but you aint going to see 990s on any gig interface. your going to be in the 940's as max..

    Duplicate the testing that hbauer did above.



  • I never said I am running 2.3.*, I'm running what came on it, which is 2.4.4-p2. My question was, and still is, how to achieve Gigabit speeds with an SG-1100, when it looks like the hardware itself can't support anything more than 250 Mbps. I'm hoping I'm wrong, so I'm here looking for proof that I'm configuring something incorrectly. Why can my laptop to the modem get Gigabit speeds, but a basic config in the SG-1100 maxes out at 250 Mbps? The answer, I feel, is in looking at the differences between my configuration, and somebody's who has the correct config in their SG-1100, and by correct config, I mean somebody who is getting Gigabit speeds.



  • @JInx-IT I am just a netgate customer and I can confirm the results posted above. The hardware can do it without any changes to the standard config. I suggest to open another topic with more details about your modem. I suspect that there is a need for some changes.


  • LAYER 8 Global Moderator

    Dude out of the box it will do gig speeds... You have been shown this and confirmed by multiple people..

    Benchmarks are listed here as well
    https://www.netgate.com/blog/netgates-new-sg-1100-punches-way-above-its-weight.html



  • The modem is setup the way we have had things setup for the last 4 years worth of using a pfSense router, which we traditionally ran off of a SuperMicro system. Pretty much all of our clients are running one, and they aren't going much past 200 Mbps. When we went up to over 400 Mbps, we noticed the router was bottlenecking the speeds, because bypassing the router and running a laptop straight to the modem produced the advertised speeds. The same is holding true with the SG-1100. Doing a speedtest from the router maxes out at 250 Mbps, but taking the router out, and plugging the laptop directly into the modem, the laptop gets Gigabit speeds. What we need, is a router we can suggest to our clients that will get Gigabit speeds. The issue is, it looks like we will have to use some cast off desktops to get that going, instead of using a professional hardware package. What I'm hoping for is that I'm missing something in the config settings that is impeding the throughput. I would love to offer this box as a Gigabit solution.



  • @johnpoz So, are you saying that it's a defective SG-1100? Because, out of the box, it's not getting gig speeds. Not even close.



  • @JInx-IT Do you have a change to try my setup with iperf connected to a switch? If this is better then you know its a configuration thing. if not open a ticket at netgate


  • LAYER 8 Global Moderator

    @JInx-IT said in SG-1100 Throughput Test:

    it's not getting gig speeds. Not even close.

    And you have yet to show you can actually do a valid test... Lets see your test method and showing that your client and server can actually talk doing gig, etc.

    hbauer gave exacting details of testing done showing 880's

    If after showing valid test methods and default config - then yeah open a support ticket.

    Doing a speed test to some internet site with some client behind pfsense is NOT a valid test method.



  • What I use for speedtest from pfSense and any other Linux based platform is speedtest-cli. If that's not accurate enough, I am certainly open to new methods.


  • Rebel Alliance Netgate Administrator

    @JInx-IT If you are testing it from the firewall, that is not accurate. You need to test from a device behind the firewall to a public device on the other side.



  • I get close to she same results whetherI'm running it from the firewall or the laptop behind the firewall, where the big jump comes is if I remove the firewall and configure the laptop to run straight off the modem. What do you recommend I run instead of speedtest-cli?


  • Rebel Alliance Netgate Administrator

    You could try using iperf/iperf3 to test from local to remote. As noted your speeds are less then anticipated for the device; more than likely the configuration/testing methods are not right.

    The more information use can share the better someone can assist you in resolving your problems.


  • Banned

    @JInx-IT
    Connect one device to WAN, and run iperf in server mode on it. Connect one device to LAN and let it run an iperf test to the server you just setup on the WAN device.

    Also make sure there are no limiters/traffic shaping configured, especially if you restored a config backup from a previous device. Make sure you are not testing over a VPN connection, that will reduce throughput considerably.

    Also if your Internet connection is using PPPoE that might be a limiting factor too.



  • @hbauer said in SG-1100 Throughput Test:

    iperf3

    I'll look into iperf again. I skimmed over it and moved on because the closest servers it listed were on the other side of the country from me. I didn't see, or think of, an option to create your own test server on the WAN. I'll probably set that up tomorrow. I wasn't able to restore from backup to the sg-1100, so that's not a factor and I'm not using a VPN or PPPoE for any of this. I'll post my results after I get everything set up and run it both ways.


  • LAYER 8 Global Moderator

    so did you test this? What were your results... I maintain a windows copy I compile myself for iperf3 if you want the latest and greatest version 3.6



  • I understand some time has passed on this thread, but I also see the slower speeds mentioned for this firewall.

    Using iperf3 client-to-server through a cheap 1G switch I consistently get about 930 Mbits/sec. Using the same cables, client, and server through the SG-1100 LAN-to-WAN I consistently get about 445 Mbits/sec. The results are slightly slower if I use the built-in iperf3 package within pfSense as the server with the client on the LAN link. The netgate is running 2.4.4_3.

    To specify the lab: a laptop is configured with a static IP in the WAN IP scope, is directly attached to the WAN port with a Cat6 cable, and listens with iperf3 as server. A laptop is configured via DHCP in the LAN IP scope, is attached to the LAN port with a Cat6 cable, and runs iperf3 as a client. Outside (WAN) laptop command is “iperf3 -p 5001 -s”. Inside (LAN) laptop command is “iperf3 -p 5001 -c <static IP of the WAN connected laptop>”.

    This is a fresh install of pfSense with no special sauce. The LAN→WAN firewall rule is an IPv4* any any.

    Now the really crazy part happens when I run the netgate built-in iperf3 package as a client. Running the same laptop as server on the LAN link, the netgate consistently gets about 865 Mbits/sec! Reverse this with the built-in iperf3 as server and the same laptop as client on the LAN link, back to mid 445 Mbits/sec. Huh?!?



  • @testgate said in SG-1100 Throughput Test:

    I understand some time has passed on this thread, but I also see the slower speeds mentioned for this firewall.

    Using iperf3 client-to-server through a cheap 1G switch I consistently get about 930 Mbits/sec. Using the same cables, client, and server through the SG-1100 LAN-to-WAN I consistently get about 445 Mbits/sec. The results are slightly slower if I use the built-in iperf3 package within pfSense as the server with the client on the LAN link. The netgate is running 2.4.4_3.

    To specify the lab: a laptop is configured with a static IP in the WAN IP scope, is directly attached to the WAN port with a Cat6 cable, and listens with iperf3 as server. A laptop is configured via DHCP in the LAN IP scope, is attached to the LAN port with a Cat6 cable, and runs iperf3 as a client. Outside (WAN) laptop command is “iperf3 -p 5001 -s”. Inside (LAN) laptop command is “iperf3 -p 5001 -c <static IP of the WAN connected laptop>”.

    This is a fresh install of pfSense with no special sauce. The LAN→WAN firewall rule is an IPv4* any any.

    Now the really crazy part happens when I run the netgate built-in iperf3 package as a client. Running the same laptop as server on the LAN link, the netgate consistently gets about 865 Mbits/sec! Reverse this with the built-in iperf3 as server and the same laptop as client on the LAN link, back to mid 445 Mbits/sec. Huh?!?

    netgate-iperf3-results.png


  • Netgate Administrator

    Hmm, what happens if you keep the server on the SG-1100 but run the client with the reverse option -R?

    Or run the client on the SG-1100 with reverse?

    The default in iperf is to have the client send traffic to the server (which always seemed an odd decision to me!). So in that test you are seeing nearly 900Mbps when it's sending but less than half that when it's receiving.

    Running either client with -R reverses the traffic but keeps the states opening the same way. So swapping it determines if it's the way the firewall opens states or the traffic direction.

    Steve



  • @stephenw10

    Thanks for your reply. Funny timing, I found that switch a few hours ago! Same results. It simply seems that the connection speed is asynchronous. The below is viewed from over the console cable to the SG-1100. The other testing device is a laptop connected to the LAN port. The text wrapped, but the second command had the -R.

    I did a full factory reset earlier and had similar results.

    Also of potential interest, I setup different scopes on LAN and OPT and tested across them as well as to/from WAN. Tests involving WAN from either LAN or OPT have the same asynchronous results. Tests between OPT and LAN were synchronous, but always below 500 Mbits/sec.

    The only other thing I noticed strange is that it seems the netgate is tagging outbound packets with TOS 7. You can even see this activity with simply ping responses from the netgate. Strange.

    I really wouldn’t ordinarily notice or care much, but this installation is going into a FIOS delivered service very close to 1Gbps. So the SG-1100 throughput will matter.

    In the below capture, the laptop connected directly to LAN was running "iperf -p 5001 -s".

    async-view.png


  • Netgate Administrator

    Hmm, so still slower receiving.

    You might try disabling pf entirely (pfctl -d) and testing between LAN and OPT. You should see close to line rate under those conditions unless there is some thing very wrong in the setup.

    I will say though that your unlikely to see much over 500Mbps from WAN to LAN, even after tuning, with firewalling and NAT in place. If you need close to 1Gbps you should upgrade to the SG-3100.

    Steve


  • LAYER 8 Global Moderator

    @testgate said in SG-1100 Throughput Test:

    tagging outbound packets with TOS 7

    Where you seeing that - I just looked, and no that is not happening..

    Please post your sniff showing that.



  • @stephenw10 Thanks again. Here goes:

    Followed the below process for LAN<->WAN test:

    Reset SG-1100 to factory default via console cable. Set interface(s) IP address via console WAN (mvneta0.4090) IP 10.10.10.1/24, LAN (mvnet0.4091) left default IP – 192.168.1.1 and default DHCP scope. Connected laptop1 to LAN, logged into Web GUI, completed the wizard leaving all defaults, but set new admin password. Laptop2 connected to WAN and set IP 10.10.10.150/24.

    On laptop2, ran command “iperf3 -p 5001 -s”
    On laptop1, ran command “iperf3 -p 5001 -c 10.10.10.150”
    On laptop1, ran command “iperf3 -p 5001 -c 10.10.10.150 -R”
    Result screen capture below:

    LAN-WAN share.png

    Followed the below process for LAN<-> OPT test:

    Executed “pfctl -d” command as root via console. Configured OPT (mvneta0.4092) via Web GUI for IP 192.168.100.1/24. Moved laptop2 to OPT and set IP 192.168.100.150/24.

    On laptop2, ran command “iperf3 -p 5001 -s”
    On laptop1, ran command “iperf3 -p 5001 -c 192.168.100.150”
    On laptop1, ran command “iperf3 -p 5001 -c 192.168.100.150 -R”
    Result screen capture below:

    LAN-OPT share.png

    Followed the below process for laptop1<->laptop2 test:

    Set laptop1 IP 192.168.100.200. Connected laptop1 directly to laptop2.

    On laptop1, ran command “iperf3 -p 5001 -c 192.168.100.150”
    On laptop1, ran command “iperf3 -p 5001 -c 192.168.100.150 -R”
    Result screen capture below:

    laptop-laptop share.png

    Taking Steve’s advice, I ordered an SG-3100. We have several of these in production and have had good results. The SG-1100 was/is an experiment. Interesting results.



  • @johnpoz Apologies, I didn’t save the capture where I saw this and could not replicate it again after a factory reset.


  • Netgate Administrator

    The SG-3100 is definitely a better option here.

    However when you disable pf using 'pfctl -d' it will be enabled again by making any changes in the gui that apply firewall or NAT changes. (unless it's disabled in the GUI). I suspect in the test where you added opt and tested from LAN to OPT it had become enabled again. Better to disable it immediately before the test.

    Steve



  • @stephenw10 I think you are correct on both points. Thanks for that, I appreciate your input. The asynchronous results of LAN-WAN testing still baffles me.


  • Netgate Administrator

    As far as I've been able to determine it's due the way NAT states are opened. It you test between interfaces that are not NATing you will see it's a lot closer or completely symmetric. So between LAN and OPT or with pf disabled or with static routes on the WAN side devices opening states directly to the LAN subnet.

    Steve



  • @stephenw10 Confirmed, my order of operation had inadvertently re-enabled pf. Rerunning the test LAN to OPT with the order you specified gave back results not very dissimilar to my laptop-to-laptop run.

    Will I not experience this same issue with the 3100? If not, is that due to the differences in HW architecture?


  • Netgate Administrator

    You may see some asymmetry in throughput but not as extreme. You should see close to Gigabit download (other variables allowing!).

    Steve


Log in to reply