OpenVPN Site-to-Site Setup, Performance and test



  • Hello All,

    I'm setting a openvpn site-to-site between two location. Now I spent 3 days to figure out how to use openvpn and I read a lot of topic and the vpn is working, but I'm experiencing some issue.

    My Setup

    pfSense Version: 2.4.3
    Hardware: Proxmox, following the docs with best practices ( Disabled Hardware Checksum Offloading )
    Network Location A ( pfSense with OpenVPN Server ): 192.168.3.0
    Network Location B ( pfSense OpenVPN Client): 192.168.23.0

    I tried both Peer-to-Peer Shared key and SSL/TLS and I have two kind of problem:

    • VPN Speed
    • VPN setting aren't setup correctly

    On both location I have a VDSL 100mbps/30mbps, so my expectation are about 30mbps on the VPN, but with the shared key mode I got only a 3mbps, while with a SSL/TLS I got a 30mbps ( so it's ok ), but I would like to figure out why this happen. 
    Then I have some problem with the SSL/TLS configuration.

    I followed the docs on pfsense wiki and other guides on internet
    Docs Shared Key: https://doc.pfsense.org/index.php/OpenVPN_Site_To_Site
    Docs SSL/TLS: https://doc.pfsense.org/index.php/OpenVPN_Site-to-Site_PKI_(SSL)

    Here the configs:

    VPN Site-to-site TLS/SSL

    SSL/TLS server1.conf

    dev ovpns1
    verb 1
    dev-type tun
    dev-node /dev/tun1
    writepid /var/run/openvpn_server1.pid
    #user nobody
    #group nobody
    script-security 3
    daemon
    keepalive 10 60
    ping-timer-rem
    persist-tun
    persist-key
    proto udp4
    cipher AES-256-CBC
    auth SHA1
    up /usr/local/sbin/ovpn-linkup
    down /usr/local/sbin/ovpn-linkdown
    local 192.168.0.7
    ifconfig 10.0.6.1 10.0.6.2
    tls-verify "/usr/local/sbin/ovpn_auth_verify tls 'xxxx' 1"  <– Removed common name
    lport 1194
    management /var/etc/openvpn/server1.sock unix
    push "route 192.168.3.0 255.255.255.0" <–- local network on site A
    route 192.168.23.0 255.255.255.0 <–- remote network on site B
    ca /var/etc/openvpn/server1.ca
    cert /var/etc/openvpn/server1.cert
    key /var/etc/openvpn/server1.key
    dh /etc/dh-parameters.1024
    tls-auth /var/etc/openvpn/server1.tls-auth 0
    ncp-ciphers AES-256-GCM:AES-128-GCM

    SSL/TLS client1.conf

    dev ovpnc1
    verb 1
    dev-type tun
    dev-node /dev/tun1
    writepid /var/run/openvpn_client1.pid
    #user nobody
    #group nobody
    script-security 3
    daemon
    keepalive 10 60
    ping-timer-rem
    persist-tun
    persist-key
    proto udp4
    cipher AES-256-CBC
    auth SHA1
    up /usr/local/sbin/ovpn-linkup
    down /usr/local/sbin/ovpn-linkdown
    local 192.168.23.7
    tls-client
    client
    lport 0
    management /var/etc/openvpn/client1.sock unix
    remote mydomain.com 1194
    ifconfig 10.0.6.2 10.0.6.1 <– that should be blank
    ca /var/etc/openvpn/client1.ca
    cert /var/etc/openvpn/client1.cert
    key /var/etc/openvpn/client1.key
    tls-auth /var/etc/openvpn/client1.tls-auth 1
    ncp-ciphers AES-256-GCM:AES-128-GCM
    resolv-retry infinite

    Now here I got the correct speed, but I had experience some issue during the configuration

    Network B Client site-to-site ssl/tls logs

    May 21 12:08:52 openvpn 93692 Initialization Sequence Completed
    May 21 12:08:52 openvpn 93692 WARNING: this configuration may cache passwords in memory – use the auth-nocache option to prevent this
    May 21 12:08:52 openvpn 93692 /usr/local/sbin/ovpn-linkup ovpnc1 1500 1561 10.0.6.2 10.0.6.1 init
    May 21 12:08:52 openvpn 93692 /sbin/ifconfig ovpnc1 10.0.7.2 10.0.7.1 mtu 1500 netmask 255.255.255.255 up
    May 21 12:08:52 openvpn 93692 do_ifconfig, tt->did_ifconfig_ipv6_setup=0
    May 21 12:08:52 openvpn 93692 TUN/TAP device /dev/tun1 opened
    May 21 12:08:52 openvpn 93692 TUN/TAP device ovpnc1 exists previously, keep at program end
    May 21 12:08:51 openvpn 93692 [iDeneb] Peer Connection Initiated with [AF_INET]xxx:xxx:xxx:xxx:1194
    May 21 12:08:51 openvpn 93692 WARNING: 'ifconfig' is present in remote config but missing in local config, remote='ifconfig 10.0.6.2 10.0.6.1'
    May 21 12:08:51 openvpn 93692 UDPv4 link remote: [AF_INET]xxx:xxx:xxx:xxx:1194
    May 21 12:08:51 openvpn 93692 UDPv4 link local (bound): [AF_INET]192.168.23.7:0
    May 21 12:08:51 openvpn 93692 TCP/UDP: Preserving recently used remote address: [AF_INET]xxx:xxx:xxx:xxx:1194
    May 21 12:08:51 openvpn 93692 NOTE: the current –script-security setting may allow this configuration to call user-defined scripts
    May 21 12:08:51 openvpn 93692 WARNING: No server certificate verification method has been enabled. See http://openvpn.net/howto.html#mitm for more info.
    May 21 12:08:51 openvpn 93692 WARNING: using –pull/--client and --ifconfig together is probably not what you want
    May 21 12:08:51 openvpn 93582 library versions: OpenSSL 1.0.2m-freebsd 2 Nov 2017, LZO 2.10
    May 21 12:08:51 openvpn 93582 OpenVPN 2.4.4 amd64-portbld-freebsd11.1 [SSL (OpenSSL)] [LZO] [LZ4] [MH/RECVDA] [AEAD] built on Mar 16 2018
    May 21 12:08:51 openvpn 54654 SIGTERM[hard,] received, process exiting
    May 21 12:08:51 openvpn 54654 /usr/local/sbin/ovpn-linkdown ovpnc1 1500 1561 init
    May 21 12:08:51 openvpn 54654 event_wait : Interrupted system call (code=4)

    Now I now that I got the red warning because in the client I added the the tunnel ip network ( ifconfig 10.0.6.0/30 ), but without this setup the client didn't obtain the virtual ip address.
    I tried to use the client override, but this one seems doesn't work… here the override config:

    cat /var/etc/openvpn-csc/server1
    iroute 192.168.23.0 255.255.255.0
    

    I also attempted to add an ifconfig-push 10.0.6.1 10.0.6.2, but it didn't work.

    _**VPN Site-to-site Shared Key/b]

    Shared key server2.conf

    
    dev ovpns2
    verb 1
    dev-type tun
    dev-node /dev/tun2
    writepid /var/run/openvpn_server2.pid
    #user nobody
    #group nobody
    script-security 3
    daemon
    keepalive 10 60
    ping-timer-rem
    persist-tun
    persist-key
    proto udp4
    cipher AES-256-CBC
    auth SHA1
    up /usr/local/sbin/ovpn-linkup
    down /usr/local/sbin/ovpn-linkdown
    local 192.168.0.7
    ifconfig 10.0.7.1 10.0.7.2
    lport 1195
    management /var/etc/openvpn/server2.sock unix
    route 192.168.23.0 255.255.255.0
    secret /var/etc/openvpn/server2.secret
    
    

    Shared key client2.conf

    
    dev ovpnc2
    verb 11
    dev-type tun
    dev-node /dev/tun2
    writepid /var/run/openvpn_client2.pid
    #user nobody
    #group nobody
    script-security 3
    daemon
    keepalive 10 60
    ping-timer-rem
    persist-tun
    persist-key
    proto udp4
    cipher AES-256-CBC
    auth SHA1
    up /usr/local/sbin/ovpn-linkup
    down /usr/local/sbin/ovpn-linkdown
    local 192.168.23.7
    lport 0
    management /var/etc/openvpn/client2.sock unix
    remote mydomain 1195
    ifconfig 10.0.7.2 10.0.7.1
    route 192.168.3.0 255.255.255.0 
    secret /var/etc/openvpn/client2.secret
    resolv-retry infinite
    
    

    In Shared mode all settings seems works well, but I got the 10% of my speed..
    I'm not using any compression, and I tried to change encryption, etc.. but it doesn't work.

    I did the network test using iperf3 on 2 virtual machine, 1 for each site.

    Here the results:

    SSL/TLS TCP

    root@test-lxc-ubuntu-1804:~# iperf3 -c 192.168.3.20
    Connecting to host 192.168.3.20, port 5201
    [  4] local 192.168.23.20 port 47060 connected to 192.168.3.20 port 5201
    [ ID] Interval           Transfer     Bandwidth       Retr  Cwnd
    [  4]   0.00-1.00   sec  3.58 MBytes  30.0 Mbits/sec    0    340 KBytes
    [  4]   1.00-2.00   sec  3.92 MBytes  32.9 Mbits/sec   15    318 KBytes
    [  4]   2.00-3.00   sec  3.61 MBytes  30.3 Mbits/sec    0    365 KBytes
    [  4]   3.00-4.00   sec  3.55 MBytes  29.8 Mbits/sec    0    396 KBytes
    [  4]   4.00-5.00   sec  3.67 MBytes  30.8 Mbits/sec   12    303 KBytes
    [  4]   5.00-6.00   sec  3.61 MBytes  30.3 Mbits/sec    0    323 KBytes
    [  4]   6.00-7.00   sec  3.61 MBytes  30.3 Mbits/sec    1    243 KBytes
    [  4]   7.00-8.00   sec  3.55 MBytes  29.8 Mbits/sec    0    264 KBytes
    [  4]   8.00-9.00   sec  3.55 MBytes  29.8 Mbits/sec    0    276 KBytes
    [  4]   9.00-10.00  sec  3.12 MBytes  26.2 Mbits/sec    6    214 KBytes
    - - - - - - - - - - - - - - - - - - - - - - - - -
    [ ID] Interval           Transfer     Bandwidth       Retr
    [  4]   0.00-10.00  sec  35.8 MBytes  30.0 Mbits/sec   34             sender
    [  4]   0.00-10.00  sec  34.6 MBytes  29.1 Mbits/sec                  receiver
    

    Shared key TCP

    root@test-lxc-ubuntu-1804:~# iperf3 -c 192.168.3.20
    Connecting to host 192.168.3.20, port 5201
    [  4] local 192.168.23.20 port 47270 connected to 192.168.3.20 port 5201
    [ ID] Interval           Transfer     Bandwidth       Retr  Cwnd
    [  4]   0.00-1.00   sec   280 KBytes  2.29 Mbits/sec    0   22.2 KBytes
    [  4]   1.00-2.00   sec   302 KBytes  2.47 Mbits/sec    0   34.0 KBytes
    [  4]   2.00-3.00   sec   192 KBytes  1.57 Mbits/sec    0   44.4 KBytes
    [  4]   3.00-4.00   sec   320 KBytes  2.62 Mbits/sec    0   56.2 KBytes
    [  4]   4.00-5.00   sec   214 KBytes  1.76 Mbits/sec    0   70.6 KBytes
    [  4]   5.00-6.00   sec   376 KBytes  3.08 Mbits/sec    0    107 KBytes
    [  4]   6.00-7.00   sec   314 KBytes  2.57 Mbits/sec    0    157 KBytes
    [  4]   7.00-8.00   sec   439 KBytes  3.60 Mbits/sec    0    222 KBytes
    [  4]   8.00-9.00   sec  62.7 KBytes   514 Kbits/sec   39    152 KBytes
    [  4]   9.00-10.00  sec   251 KBytes  2.06 Mbits/sec    9    161 KBytes
    - - - - - - - - - - - - - - - - - - - - - - - - -
    [ ID] Interval           Transfer     Bandwidth       Retr
    [  4]   0.00-10.00  sec  2.69 MBytes  2.25 Mbits/sec   48             sender
    [  4]   0.00-10.00  sec  2.23 MBytes  1.87 Mbits/sec                  receiver
    
    iperf Done.
    
    

    UDP Session

    root@test-lxc-ubuntu-1804:~# iperf3 -c 192.168.3.20 -u -b 100m
    Connecting to host 192.168.3.20, port 5201
    [  4] local 192.168.23.20 port 56920 connected to 192.168.3.20 port 5201
    [ ID] Interval           Transfer     Bandwidth       Total Datagrams
    [  4]   0.00-1.00   sec  10.8 MBytes  90.5 Mbits/sec  1381
    [  4]   1.00-2.00   sec  11.9 MBytes   100 Mbits/sec  1526
    [  4]   2.00-3.00   sec  11.9 MBytes   100 Mbits/sec  1526
    [  4]   3.00-4.00   sec  11.9 MBytes   100 Mbits/sec  1526
    [  4]   4.00-5.00   sec  11.9 MBytes   100 Mbits/sec  1526
    [  4]   5.00-6.00   sec  11.9 MBytes   100 Mbits/sec  1526
    [  4]   6.00-7.00   sec  11.9 MBytes  99.9 Mbits/sec  1525
    [  4]   7.00-8.00   sec  11.9 MBytes   100 Mbits/sec  1527
    [  4]   8.00-9.00   sec  11.9 MBytes   100 Mbits/sec  1526
    [  4]   9.00-10.00  sec  11.9 MBytes  99.9 Mbits/sec  1525
    - - - - - - - - - - - - - - - - - - - - - - - - -
    [ ID] Interval           Transfer     Bandwidth       Jitter    Lost/Total Datagrams
    [  4]   0.00-10.00  sec   118 MBytes  99.0 Mbits/sec  3.058 ms  12846/14982 (86%)
    [  4] Sent 14982 datagrams
    
    iperf Done.
    

    I didn't understand the reason why in UDP ( both connection ) mode it's using the whole bandwitch, theorically it's up to 30mbps.

    However, also to trying to move an iso file along the network ( SMB ), it confirmed the iperf3 test:

    VPN type: ssl/tls
    File size: 293mb
    Speed: 350KB/s

    VPN type: Shared key
    File size: 293mb
    Speed: about 3,11MB/s

    I'm sorry for the long post, but I would post all the details that I could :)

    Thanks,
    Jack!**_


 

© Copyright 2002 - 2018 Rubicon Communications, LLC | Privacy Policy