Hyper-v ipsec performance

[laped]

Created a hyper-v setup where we have 2 fedora and a pfsense 2.2.4 connected in the same vswitch. The fedora is connecting in as mobile clients using aes-256-gcm with PSK. First we observed around 2mbit/s speeds but got an increase when the broadcom nic driver was updated. We have observed speeds of 400mbit/s+ (actual speed have to be multiplied with 2 since pfsense get the load of two clients) speeds in some cases and discovered that disabling the pfsense firewall gained a giant boost in throughput (100% boost).  The expected speed should be in the 800-900 mbit/s which we are only getting when disabling its firewall. So I guess we are looking some some missing/magical setting or bug in either hyper-v or pfsense :)

AES-NI module is loaded
pfsense has 2gb ram and 2 xeon cores
fedora has 4gb ram and 2 xeon cores

Hyper-V – pfctl enabled

[admin@localhost ~]$ iperf -c 10.75.0.1 -P 2
–----------------------------------------------------------
Client connecting to 10.75.0.1, TCP port 5001
TCP window size:  230 KByte (default)

[  3] local 10.75.0.2 port 56482 connected with 10.75.0.1 port 5001
[  4] local 10.75.0.2 port 56483 connected with 10.75.0.1 port 5001
[ ID] Interval      Transfer    Bandwidth
[  3]  0.0-10.0 sec  125 MBytes  105 Mbits/sec
[  4]  0.0-10.0 sec  138 MBytes  116 Mbits/sec
[SUM]  0.0-10.0 sec  263 MBytes  220 Mbits/sec

Hyper-V – pfctl disabled

[admin@localhost ~]$ iperf -c 10.75.0.1 -P 2
–----------------------------------------------------------
Client connecting to 10.75.0.1, TCP port 5001
TCP window size:  230 KByte (default)

[  4] local 10.75.0.2 port 56481 connected with 10.75.0.1 port 5001
[  3] local 10.75.0.2 port 56480 connected with 10.75.0.1 port 5001
[ ID] Interval      Transfer    Bandwidth
[  4]  0.0-10.0 sec  353 MBytes  295 Mbits/sec
[  3]  0.0-10.0 sec  168 MBytes  140 Mbits/sec
[SUM]  0.0-10.0 sec  520 MBytes  435 Mbits/sec

Tried the same test on a hardware box to see if the difference was the same.

SG-2240 – pfctl enabled

[test@fedoratestpc1 ~]$ iperf -c 10.75.0.2 -P 2
–----------------------------------------------------------
Client connecting to 10.75.0.2, TCP port 5001
TCP window size:  264 KByte (default)

[  4] local 10.75.0.1 port 34508 connected with 10.75.0.2 port 5001
[  3] local 10.75.0.1 port 34507 connected with 10.75.0.2 port 5001
[ ID] Interval      Transfer    Bandwidth
[  3]  0.0-10.0 sec  58.6 MBytes  49.2 Mbits/sec
[  4]  0.0-10.1 sec  53.4 MBytes  44.5 Mbits/sec
[SUM]  0.0-10.1 sec  112 MBytes  93.4 Mbits/sec

SG-2240 – pfctl disabled

[test@fedoratestpc1 ~]$ iperf -c 10.75.0.2 -P 2
–----------------------------------------------------------
Client connecting to 10.75.0.2, TCP port 5001
TCP window size:  264 KByte (default)

[  4] local 10.75.0.1 port 34512 connected with 10.75.0.2 port 5001
[  3] local 10.75.0.1 port 34511 connected with 10.75.0.2 port 5001
[ ID] Interval      Transfer    Bandwidth
[  4]  0.0-10.0 sec  63.5 MBytes  53.1 Mbits/sec
[  3]  0.0-10.1 sec  77.0 MBytes  64.2 Mbits/sec
[SUM]  0.0-10.1 sec  140 MBytes  117 Mbits/sec

Note: Updated the pfctl (firewall control) lines to make more sense :)