Slow peering = slow IPsec. Any way to route around?



  • Hey friends,
    This is more of a general networking and WAN question but since all the endpoints involved are pfSense, I thought I'd throw myself on the mercy of this group :)

    For a while I've been trying to nail down some slow IPsec performance.

    For a while, I was getting almost full line speed between my two servers outside of my tunnel. Now, it seems, I am not. And that's remained consistent.

    My two external servers are both on OVH's networks: one in canada and one in france. My 'home' location is San Francisco. San Francisco is on a symmetrical gig fiber line. Canada and France both have 250mbs connections.

    I've also spun up a digital ocean droplet in San Francisco for testing. Let's call it Digitalocean.

    Here's what I'm seeing using iperf3 (I've tested with iperf3 running on my SG-4860 directly and on a host behind the firewall. The results are the same).

    These are all outside of the tunnel

    canada <–> San Francisco: 22.9 Mbits/sec
    france <--> San Francisco: 22.7 Mbits/sec

    canada <--> digitalocean: 146 Mbits/sec
    france <--> digitalocean: 136 Mbits/sec

    digitalocean <--> san francisco: 537 Mbits/sec

    If I'm betting >100mbs between digital ocean's S.F. location and from that digital ocean droplet to my S. F. pfSense I get >500mbs...can I use that to my advantage somehow?

    I'm thinking I need to solve this before I got back to troubleshooting my IPsec issues :)

    So, here's my questions....

    1. is it reasonable to assume that I have a peering and/or external routing issue?
      1.5) is it reasonable to assume if thats true it is, generally, out of my control?
    2. if both of those are true, could I do something with a digital ocean or similar VPS to route between these remote servers and my san francisco location? I'm thinking something with less overhead than a 2nd IPsec tunnel...maybe just a simple proxy?
    3. is there a better way to skin this proverbial cat?