IPsec site to site performance not great



  • Hey all - trying to improve the performance between two locations.  My one location is 85/85 Mbps, the other is 120 Mbps/12 Mbps however if I access a windows share and transfer a large file I get varying speeds around 350 - 750 KB/sec.  I went and set the Enable MSS clamping on VPN traffic  to on, and set it to 1300 on both sides, and got to about 1.0 MB/sec - 1.1 MB/sec.  I tried with FTP also, same results.

    How do I get this to be more like my actual upload speed?  CPU use was very low, 1-2%.  Both pfsense boxes are on ESXi Virtual Machines.

    Thanks!



  • Can you confirm that you push the traffic levels you are hoping for without the VPN involved?

    If you SSH in or look in the console, run ifconfig, what do your options look like:

    options=60009b <rxcsum,txcsum,vlan_mtu,vlan_hwtagging,vlan_hwcsum,rxcsum_ipv6,txcsum_ipv6>Have a look at:

    https://doc.pfsense.org/index.php/Tuning_and_Troubleshooting_Network_Cards

    I'm suggesting that you might be having issues with the TSO and LRO areas.

    What sort of network cards are you using under ESXi, what have you setup under FreeBSD?</rxcsum,txcsum,vlan_mtu,vlan_hwtagging,vlan_hwcsum,rxcsum_ipv6,txcsum_ipv6>



  • @Hugh:

    Can you confirm that you push the traffic levels you are hoping for without the VPN involved?

    If you SSH in or look in the console, run ifconfig, what do your options look like:

    options=60009b <rxcsum,txcsum,vlan_mtu,vlan_hwtagging,vlan_hwcsum,rxcsum_ipv6,txcsum_ipv6>Have a look at:

    https://doc.pfsense.org/index.php/Tuning_and_Troubleshooting_Network_Cards

    I'm suggesting that you might be having issues with the TSO and LRO areas.  Here is the full output of ifconfig:

    [2.1.5-RELEASE][admin@pfSense.conway.local]/root(1): ifconfig
    em0: flags=8843 <up,broadcast,running,simplex,multicast>metric 0 mtu 1500
            options=9b <rxcsum,txcsum,vlan_mtu,vlan_hwtagging,vlan_hwcsum>ether 00:50:56:88:5d:36
            inet 192.168.50.1 netmask 0xffffff00 broadcast 192.168.50.255
            inet6 fe80::250:56ff:fe88:5d36%em0 prefixlen 64 scopeid 0x1
            nd6 options=1 <performnud>media: Ethernet autoselect (1000baseT <full-duplex>)
            status: active
    em1: flags=8843 <up,broadcast,running,simplex,multicast>metric 0 mtu 1500
            options=9b <rxcsum,txcsum,vlan_mtu,vlan_hwtagging,vlan_hwcsum>ether f8:e4:fb:22:40:ee
            inet 72.92.54.39 netmask 0xffffff00 broadcast 72.92.54.255
            inet6 fe80::fae4:fbff:fe22:40ee%em1 prefixlen 64 scopeid 0x2
            nd6 options=3 <performnud,accept_rtadv>media: Ethernet autoselect (1000baseT <full-duplex>)
            status: active
    em2: flags=8843 <up,broadcast,running,simplex,multicast>metric 0 mtu 1500
            options=9b <rxcsum,txcsum,vlan_mtu,vlan_hwtagging,vlan_hwcsum>ether 00:50:56:88:08:18
            inet 192.168.200.1 netmask 0xffffff00 broadcast 192.168.200.255
            inet6 fe80::250:56ff:fe88:818%em2 prefixlen 64 scopeid 0x3
            nd6 options=1 <performnud>media: Ethernet autoselect (1000baseT <full-duplex>)
            status: active
    plip0: flags=8810 <pointopoint,simplex,multicast>metric 0 mtu 1500
    pfsync0: flags=0<> metric 0 mtu 1460
            syncpeer: 224.0.0.240 maxupd: 128 syncok: 1
    lo0: flags=8049 <up,loopback,running,multicast>metric 0 mtu 16384
            options=3 <rxcsum,txcsum>inet 127.0.0.1 netmask 0xff000000
            inet6 ::1 prefixlen 128
            inet6 fe80::1%lo0 prefixlen 64 scopeid 0x6
            nd6 options=3 <performnud,accept_rtadv>pflog0: flags=100 <promisc>metric 0 mtu 33144
    enc0: flags=41 <up,running>metric 0 mtu 1536
    ovpns1: flags=8051 <up,pointopoint,running,multicast>metric 0 mtu 1500
            options=80000 <linkstate>inet6 fe80::250:56ff:fe88:5d36%ovpns1 prefixlen 64 scopeid 0x9
            inet 10.8.0.1 –> 10.8.0.2 netmask 0xffffffff
            nd6 options=3 <performnud,accept_rtadv>Opened by PID 81705
    ovpns2: flags=8051 <up,pointopoint,running,multicast>metric 0 mtu 1500
            options=80000 <linkstate>inet6 fe80::250:56ff:fe88:5d36%ovpns2 prefixlen 64 scopeid 0xa
            inet 10.0.2.1 --> 10.0.2.2 netmask 0xffffffff
            nd6 options=3 <performnud,accept_rtadv>Opened by PID 86563
    ovpns3: flags=8051 <up,pointopoint,running,multicast>metric 0 mtu 1500
            options=80000 <linkstate>inet6 fe80::250:56ff:fe88:5d36%ovpns3 prefixlen 64 scopeid 0xb
            inet 10.8.8.1 --> 10.8.8.2 netmask 0xffffffff
            nd6 options=3 <performnud,accept_rtadv>Opened by PID 90177
    ovpns4: flags=8051 <up,pointopoint,running,multicast>metric 0 mtu 1500
            options=80000 <linkstate>inet6 fe80::250:56ff:fe88:5d36%ovpns4 prefixlen 64 scopeid 0xc
            inet 10.8.1.1 --> 10.8.1.2 netmask 0xffffffff
            nd6 options=3 <performnud,accept_rtadv>Opened by PID 94308

    Right now none of the OpenVPN servers are actively used.

    Thanks!

    What sort of network cards are you using under ESXi, what have you setup under FreeBSD?</performnud,accept_rtadv></linkstate></up,pointopoint,running,multicast></performnud,accept_rtadv></linkstate></up,pointopoint,running,multicast></performnud,accept_rtadv></linkstate></up,pointopoint,running,multicast></performnud,accept_rtadv></linkstate></up,pointopoint,running,multicast></up,running></promisc></performnud,accept_rtadv></rxcsum,txcsum></up,loopback,running,multicast></pointopoint,simplex,multicast></full-duplex></performnud></rxcsum,txcsum,vlan_mtu,vlan_hwtagging,vlan_hwcsum></up,broadcast,running,simplex,multicast></full-duplex></performnud,accept_rtadv></rxcsum,txcsum,vlan_mtu,vlan_hwtagging,vlan_hwcsum></up,broadcast,running,simplex,multicast></full-duplex></performnud></rxcsum,txcsum,vlan_mtu,vlan_hwtagging,vlan_hwcsum></up,broadcast,running,simplex,multicast></rxcsum,txcsum,vlan_mtu,vlan_hwtagging,vlan_hwcsum,rxcsum_ipv6,txcsum_ipv6>

    Thanks Hugh for the reply.  Without the VPN tunnel I can download via web server at 4.1 MB/s from one location to the other.  I am using Intel Pro/1000 VT quad port nics in each ESXi host.

    Both TSO and LRO boxes are checked on each side


Log in to reply