SG-1000 OpenVPN performance

  • Hello !

    In my use case, all the traffic goes through my brand new SG-1000. On this SG-1000 an OpenVPN client is configured and all the traffic is routed through OpenVPN. I use firewalling, NAT and dual stack IPv4 / IPv6. Software version : 2.4.0.b.20170318.0910

    Network diagram : desktop (iperf client) ==> switch ==> SG-1000 ==>ISP box in bridge mode ==> Internet

    Performances (heavy traffic like bittorrent and TCP iperf with default options have about the same perfs) :

    • max 20 to 25 Mbps with compression and encryption disabled.

    • max 10 to 15 Mbps with encryption AES-128-CBC or BF-CFC.

    • With torrent, the top command permanently indicates 2 digits percentages for all of user, system and interrupt, and idle is stuck to 0%. Load avarages < 2. Latency through the VPN is 5 to 100 times the one when no traffic. When max bandwidth is reached. OpenVPN process is at 70% WCPU.

    • With iperf, the top command shows 2 digits interrupt percentage (20 to 30%) and idle is not stuck to 0% (60~70% idle). Load avarages < 1. Latency through the VPN is OK

    Are those perfs the one expected with SG-1000 ? Or am I missing something.

    Side notes :

    • I can't see any error on the interfaces except drops on ovpnc1 interface (netstat -idh or sysctl -a | grep cpsw).

    • here is what I tried for improvement, no success: disabling softflowd, snmpd, ladvd, remote logging via syslog, FTP client proxy, enable / disable hardware checksum offload, changing NIC on my desktop, changing all the ethernet cables, use a different switch.

    • With almost the same configuration on previous hardware I got : iperf through pfsense 200Mbps, iperf through pfsense and OpenVPN : 50Mbps. The difference in the configurations is that on SG-1000 I use a VLAN interface (previous hardware had 3 NICs). But even when unassigning the vlan interface on SG-1000 I stll have the same performance issues.

    I made other tests :

    • TCP iperf with default options to SG-1000 (iperf server on SG-1000) : 100Mbps (all hardware is gigabit capable, and all autonegs are gigabit)

    • TCP iperf with default options through SG-1000 : max 25 Mbps (even with VPN turned off). During the test the top command shows 2 digits interrupt percentage (10 to 25%) and idle is not stuck to 0% (60~70% idle). Load avarages < 2. NB : sometimes without changing anything (let's say 5% of the time or less), iperf through SG-1000 reaches 150Mbps. When this happens, "top -aSH" freezes but at then end when it's available again then I can see that interrupts are almost 100%.

    • UDP iperf to sg-1000 : "iperf -u -t 10 -c sg-1000 -b 90M" shows a result around 90Mbps, but sending more than the sg-1000 can handle makes it suffer : "iperf -u -t 10 -c sg-1000 -b 150M" shows a result around 5Mbps.

    Any suggestion on the way to improve non-OpenVPN traffic ?

  • From what I recall Jim saying, your OpenVPN numbers are about right until someone finishes the ARM crypto driver for FreeBSD and integrates it into pfSense, at which point encrypted traffic should perform almost as well as unencrypted traffic.

    However, your numbers for unencrypted traffic sound about right for a cheap ARM CPU.  (The SG-1000 isn't cheap, but its CPU is - relatively speaking, anyway.)  The SG-1000 is not the speediest thing around, nor is it intended to be.  It's intended to be small, low power, and "cheap enough".

    I would also try turning polling on and off, to see what difference that makes.

    Ultimately, based on the CPU specs for that ADI board, I doubt you'll see much past 40Mbps aggregate throughput, at least until the next-gen pfSense based on (?) netmap arrives.  And even then, 100Mbps would be about as much as the unit can handle, I think.

    The presence of gigabit interfaces isn't indicative that the unit can pass 1Gbps, unfortunately, they're there for compatibility with modern equipment.  You get what you pay for… if you need >100Mbps throughput, at least buy an SG-2240 (IMHO)!


  • Hello,

    Thanks for your reply. I am afraid that 10~20Mbps through VPN is indeed the best I can get. That would be nice if I was lucky enough so that someone from netgate / pfSense could confirm that here. Can you please provide a link containing informations regarding the ARM crypto driver and its integration into pfsense ?

    But without VPN I think there really is an issue with the maximum 25Mbps I get. Should be more than 100Mbps i think :

    I guess you refer to SG-2220. I considered it but it's for home so I try to reduce the cost. And if I'm not mistaken we still don't have precise informations about that Intel Atom C2000 broken CPUs issue.

Log in to reply