Hyper-v ipsec performance



  • Created a hyper-v setup where we have 2 fedora and a pfsense 2.2.4 connected in the same vswitch. The fedora is connecting in as mobile clients using aes-256-gcm with PSK. First we observed around 2mbit/s speeds but got an increase when the broadcom nic driver was updated. We have observed speeds of 400mbit/s+ (actual speed have to be multiplied with 2 since pfsense get the load of two clients) speeds in some cases and discovered that disabling the pfsense firewall gained a giant boost in throughput (100% boost).  The expected speed should be in the 800-900 mbit/s which we are only getting when disabling its firewall. So I guess we are looking some some missing/magical setting or bug in either hyper-v or pfsense :)

    AES-NI module is loaded
    pfsense has 2gb ram and 2 xeon cores
    fedora has 4gb ram and 2 xeon cores

    Hyper-V – pfctl enabled

    [admin@localhost ~]$ iperf -c 10.75.0.1 -P 2
    –----------------------------------------------------------
    Client connecting to 10.75.0.1, TCP port 5001
    TCP window size:  230 KByte (default)

    [  3] local 10.75.0.2 port 56482 connected with 10.75.0.1 port 5001
    [  4] local 10.75.0.2 port 56483 connected with 10.75.0.1 port 5001
    [ ID] Interval      Transfer    Bandwidth
    [  3]  0.0-10.0 sec  125 MBytes  105 Mbits/sec
    [  4]  0.0-10.0 sec  138 MBytes  116 Mbits/sec
    [SUM]  0.0-10.0 sec  263 MBytes  220 Mbits/sec

    Hyper-V – pfctl disabled

    [admin@localhost ~]$ iperf -c 10.75.0.1 -P 2
    –----------------------------------------------------------
    Client connecting to 10.75.0.1, TCP port 5001
    TCP window size:  230 KByte (default)

    [  4] local 10.75.0.2 port 56481 connected with 10.75.0.1 port 5001
    [  3] local 10.75.0.2 port 56480 connected with 10.75.0.1 port 5001
    [ ID] Interval      Transfer    Bandwidth
    [  4]  0.0-10.0 sec  353 MBytes  295 Mbits/sec
    [  3]  0.0-10.0 sec  168 MBytes  140 Mbits/sec
    [SUM]  0.0-10.0 sec  520 MBytes  435 Mbits/sec

    Tried the same test on a hardware box to see if the difference was the same.

    SG-2240 – pfctl enabled

    [test@fedoratestpc1 ~]$ iperf -c 10.75.0.2 -P 2
    –----------------------------------------------------------
    Client connecting to 10.75.0.2, TCP port 5001
    TCP window size:  264 KByte (default)

    [  4] local 10.75.0.1 port 34508 connected with 10.75.0.2 port 5001
    [  3] local 10.75.0.1 port 34507 connected with 10.75.0.2 port 5001
    [ ID] Interval      Transfer    Bandwidth
    [  3]  0.0-10.0 sec  58.6 MBytes  49.2 Mbits/sec
    [  4]  0.0-10.1 sec  53.4 MBytes  44.5 Mbits/sec
    [SUM]  0.0-10.1 sec  112 MBytes  93.4 Mbits/sec

    SG-2240 – pfctl disabled

    [test@fedoratestpc1 ~]$ iperf -c 10.75.0.2 -P 2
    –----------------------------------------------------------
    Client connecting to 10.75.0.2, TCP port 5001
    TCP window size:  264 KByte (default)

    [  4] local 10.75.0.1 port 34512 connected with 10.75.0.2 port 5001
    [  3] local 10.75.0.1 port 34511 connected with 10.75.0.2 port 5001
    [ ID] Interval      Transfer    Bandwidth
    [  4]  0.0-10.0 sec  63.5 MBytes  53.1 Mbits/sec
    [  3]  0.0-10.1 sec  77.0 MBytes  64.2 Mbits/sec
    [SUM]  0.0-10.1 sec  140 MBytes  117 Mbits/sec

    Note: Updated the pfctl (firewall control) lines to make more sense :)