IPsec site-to-site 80% slower than max speed
-
Hi, I'm running PfSense 2.1 (2.1-RC0 (amd64) built on Thu Jul 11 00:16:53 EDT 2013 on one side and 2.1-RC0 (i386) built on Mon Jul 22 09:36:36 EDT 2013 on the other).
I set up an IPsec site-to-site connection and it's working fine but the speed is very low. The connection from A to B should be 50Mb/s and the other way around 6Mb/s. Site B has a 50/50Mb/s connection and A 60/6Mb/s. I'm only getting 10/2.5Mb/s where it should be 50/6Mb/s.
I tried:
-
All kinds of encryption settings, even used a matrix with all the different settings. They didn't differ that much, about 5% max.
-
Setting mss in Advanced > Miscellaneous > Maximum MSS and in the WAN MSS and MTU fields (all separately and various mixtures) to no effect - if I send a ping from site A to site B using the Do Not Fragment flag, it maxes out at 1472 bytes, as could be expected. The default vpn/ipsec value is 1400 so this should not be a problem.
-
Test the speed in different ways: ftp, http, smb.
-
Reboot both routers.
-
Test with different internet connections, different router hardware, different clients.
Internet speeds on both sides are normal (fast).
The logs mention no errors.
Everything else on PfSense works fine, including the IPsec tunnel - it's just very slow.
I don't do traffic shaping.
CPU load on one machine doesn't go above 1% on one and 10% on the other PfSense machine. (One is a dual Xeon 3.something GHz and the other a five year old laptop with an Intel chip (forgot the type). I also tested with a thin client VIA Nehemiah 1GHz. They have plenty of cpu and ram capacity.)
Network interfaces are Intel Pro/1000 and Broadcom Gb cards. They should easily be able to handle that load (and are since non-IPsec traffic is fast).I'm running out of ideas to try, any help would be very much appreciated!
(Oh, we're using IPsec because we have some software that refuses to work properly over OpenVPN. Using PfSense 2.1 and not 2.0.1 because I we need to route all traffic over HQ, which is supported for IPsec in PfSense 2.1.)
-
-
I'm actually surprised you get 10 even. I talked a little about this yesterday with someone else.
When the upload and download speeds vary greatly at both or even 1 site, I expect my connection speed in both directions to be roughly limited to the lowerer of the up/down connections at either site, especially when tunneling all traffic through the server. However, I was informed yesterday that this isn't the case, so I'm likely wrong. I guess.At that 60/6 site, I'd probably try to get a 15/15 or 20/20 or 30/30 or something like that if possible.
If you had 50/50 at both sites that would be most perfect. (I realize this isn't always possible).
-
Hi, I have the same problem, when I build a VPN site to site with IPSec is very slow, otherwise when I build other kinds of VPN are fast.
There is anyone who has found the solution?
Thanks
-
I haven't been able to solve the IPsec problem but went with OpenVPN. The problems we had with OpenVPN from a user's laptop to the corporate LAN didn't occur with a LAN-to-LAN connection.
I documented the setup here.
-
Thank you, very good documentation. I have configured the OpenVPN site to site and now I have no problems with slow networks.
Many thanks
-
Interesting post.
We have a similar setup.
HQ 100/10(d/u)mbit
RemoteSite 100/100 mbit.When i use something else then IPSEC i get (almost) 10 up and 100down (lest say 90% of max speed).
When i move some data over the IPSEC link i only get 160kbs (both ways) 0.16mbit! the pingtimes/latancy is 500ms instead of <20ms.
So I though our hardware want able do deal with this so i setup a second pfSense router.. with pfsense 2.1-RC something resent> Same problem.
SO i think, maby somebody can confirm this, that the UPC cabel (cisco EPC3925 ) modem we have at our HQ is 'broken'.
The other thing is in the past we did get at least 30mbit down (hardware wasn't fast enough to decrypt everything).
Any suggestions or thoughts on how to test this?
-
I dont no, if our problem is related to the topic's starter, but since it sounds the same I thought some more data wouldn't hurt.
when i start a transfer of a 100mb file over the ipsec the ping times spike up
the 128. is the remote site, 95 is our 'local' gateway (we have a /29)
When i transfer the same file to the same location with out using the IPsec connection (connecting direct)
the network layout looks like this:
[CLIENTS] -> [pfsense_1] -> [Cisco EPC3925 ] -> |internetz| -> [ftth (fiber to eth)] -> [pfsense_2]
pfsense box has a 95.xxx.xxx.82 ip
epc 3925 has 95.xxx.xxx.81 (so it is in bridge mode)both pfsense boxes are 2.0.1
the load on both machines is "LOW" (below 30% cpu)
test the same setup with a different pfsense_1 box, faster machine and pfsense 2.1-rc0 build 15 of august exactly the same behaviour!
We run this pfsense tunnel for more then a year now, and at the beginin i know for a fact we did 30mbit (remote site to HQ), witch we cant do at the moment.
It seems to me that there is something wrong with the EPC3925 but I havent got a clue what that is…or could be.. any suggestions?
-
This morning i did some more digging….
I first power-off the EPC3925 for 2 minutes after a restart the problem was a bit better but still not good.
After a factory rest it did work again...
So… no clue why we had this problem or what the problem actually was...
Case closed?
BTW: the ipsec uplink was rocksolid!
-
Factory reset or backup / wipe / restore is often a huge problem solver and it doesn't even take much time.
-
Factory reset or backup / wipe / restore is often a huge problem solver and it doesn't even take much time.
Unless you need to restore loads of existing settings. You could just restore an xml backup but that may reintroduce the original problem.
-
Yeah - When I say wipe, reinstall and restore, I'm talking the XML restore - Not a disk image restore. That would defeat the purpose.
-
I also have a very similar problem with slow traffic over IPsec tunnel, I am pretty newish to networking but want to know if this is normal behavior for a IPsec connection
Site A – Data center has 100/100mb in and out
Site B – Home, has virgin media fibre broadband 150mb line gives me around 10mb upload max.I have setup a PfSese server 2.2.6 at data center, my home network has a Draytek 2860.
I have a windows 2012 server in DC and when copying a file using windows explorer from home using a windows 7 machine I get speeds of around 1.5MB when copying the file to DC
I have also tried using PfSese at home to see if the draytek router was the issue, made no difference in speeds.
I have also tested IPsec using draytek router to draytek router noticed very poor speeds when copying a files across using explorer.
I have tested copying files across using FTP getting similar speed to windows explorer
I have used iperf to test speeds beteen A-site and B-site and showing up as decent bandwidth. Perhaps I am not understanding something or some kind windows SMB limit etc ?
CLIENT
Connecting to host 172.16.1.10, port 5201
[ 4] local 192.168.50.102 port 50364 connected to 172.16.1.10 port 5201
[ ID] Interval Transfer Bandwidth
[ 4] 0.00-1.00 sec 1.38 MBytes 11.5 Mbits/sec
[ 4] 1.00-2.00 sec 1.25 MBytes 10.5 Mbits/sec
[ 4] 2.00-3.00 sec 1.38 MBytes 11.5 Mbits/sec
[ 4] 3.00-4.00 sec 1.12 MBytes 9.44 Mbits/sec
[ 4] 4.00-5.00 sec 1.00 MBytes 8.38 Mbits/sec
[ 4] 5.00-6.00 sec 1.00 MBytes 8.39 Mbits/sec
[ 4] 6.00-7.00 sec 1.00 MBytes 8.39 Mbits/sec
[ 4] 7.00-8.00 sec 640 KBytes 5.24 Mbits/sec
[ 4] 8.00-9.00 sec 1.00 MBytes 8.38 Mbits/sec
[ 4] 9.00-10.00 sec 896 KBytes 7.34 Mbits/sec
[ ID] Interval Transfer Bandwidth
[ 4] 0.00-10.00 sec 10.6 MBytes 8.91 Mbits/sec sender
[ 4] 0.00-10.00 sec 10.5 MBytes 8.81 Mbits/sec receiveriperf Done.
SERVER SIDE
Server listening on 5201
–---------------------------------------------------------
Accepted connection from 192.168.50.102, port 50363
[ 5] local 172.16.1.10 port 5201 connected to 192.168.50.102 port 50364
[ ID] Interval Transfer Bandwidth
[ 5] 0.00-1.00 sec 1.16 MBytes 9.71 Mbits/sec
[ 5] 1.00-2.00 sec 1.38 MBytes 11.6 Mbits/sec
[ 5] 2.00-3.00 sec 1.33 MBytes 11.1 Mbits/sec
[ 5] 3.00-4.00 sec 1.13 MBytes 9.44 Mbits/sec
[ 5] 4.00-5.00 sec 1.09 MBytes 9.13 Mbits/sec
[ 5] 5.00-6.00 sec 954 KBytes 7.81 Mbits/sec
[ 5] 6.00-7.00 sec 986 KBytes 8.07 Mbits/sec
[ 5] 7.00-8.00 sec 653 KBytes 5.36 Mbits/sec
[ 5] 8.00-9.00 sec 1020 KBytes 8.35 Mbits/sec
[ 5] 9.00-10.00 sec 795 KBytes 6.51 Mbits/sec
[ 5] 10.00-10.10 sec 130 KBytes 10.9 Mbits/sec
[ ID] Interval Transfer Bandwidth
[ 5] 0.00-10.10 sec 0.00 Bytes 0.00 bits/sec sender
[ 5] 0.00-10.10 sec 10.5 MBytes 8.73 Mbits/sec receiver
–---------------------------------------------------------
Server listening on 5201 -
I also have a very similar problem with slow traffic over IPsec tunnel, I am pretty newish to networking but want to know if this is normal behavior for a IPsec connection
Site A – Data center has 100/100mb in and out
Site B – Home, has virgin media fibre broadband 150mb line gives me around 10mb upload max.I have setup a PfSese server 2.2.6 at data center, my home network has a Draytek 2860.
I have a windows 2012 server in DC and when copying a file using windows explorer from home using a windows 7 machine I get speeds of around 1.5MB when copying the file to DC
I have also tried using PfSese at home to see if the draytek router was the issue, made no difference in speeds.
I have also tested IPsec using draytek router to draytek router noticed very poor speeds when copying a files across using explorer.
I have tested copying files across using FTP getting similar speed to windows explorer
I have used iperf to test speeds beteen A-site and B-site and showing up as decent bandwidth. Perhaps I am not understanding something or some kind windows SMB limit etc ?
CLIENT
Connecting to host 172.16.1.10, port 5201
[ 4] local 192.168.50.102 port 50364 connected to 172.16.1.10 port 5201
[ ID] Interval Transfer Bandwidth
[ 4] 0.00-1.00 sec 1.38 MBytes 11.5 Mbits/sec
[ 4] 1.00-2.00 sec 1.25 MBytes 10.5 Mbits/sec
[ 4] 2.00-3.00 sec 1.38 MBytes 11.5 Mbits/sec
[ 4] 3.00-4.00 sec 1.12 MBytes 9.44 Mbits/sec
[ 4] 4.00-5.00 sec 1.00 MBytes 8.38 Mbits/sec
[ 4] 5.00-6.00 sec 1.00 MBytes 8.39 Mbits/sec
[ 4] 6.00-7.00 sec 1.00 MBytes 8.39 Mbits/sec
[ 4] 7.00-8.00 sec 640 KBytes 5.24 Mbits/sec
[ 4] 8.00-9.00 sec 1.00 MBytes 8.38 Mbits/sec
[ 4] 9.00-10.00 sec 896 KBytes 7.34 Mbits/sec
[ ID] Interval Transfer Bandwidth
[ 4] 0.00-10.00 sec 10.6 MBytes 8.91 Mbits/sec sender
[ 4] 0.00-10.00 sec 10.5 MBytes 8.81 Mbits/sec receiveriperf Done.
SERVER SIDE
Server listening on 5201
–---------------------------------------------------------
Accepted connection from 192.168.50.102, port 50363
[ 5] local 172.16.1.10 port 5201 connected to 192.168.50.102 port 50364
[ ID] Interval Transfer Bandwidth
[ 5] 0.00-1.00 sec 1.16 MBytes 9.71 Mbits/sec
[ 5] 1.00-2.00 sec 1.38 MBytes 11.6 Mbits/sec
[ 5] 2.00-3.00 sec 1.33 MBytes 11.1 Mbits/sec
[ 5] 3.00-4.00 sec 1.13 MBytes 9.44 Mbits/sec
[ 5] 4.00-5.00 sec 1.09 MBytes 9.13 Mbits/sec
[ 5] 5.00-6.00 sec 954 KBytes 7.81 Mbits/sec
[ 5] 6.00-7.00 sec 986 KBytes 8.07 Mbits/sec
[ 5] 7.00-8.00 sec 653 KBytes 5.36 Mbits/sec
[ 5] 8.00-9.00 sec 1020 KBytes 8.35 Mbits/sec
[ 5] 9.00-10.00 sec 795 KBytes 6.51 Mbits/sec
[ 5] 10.00-10.10 sec 130 KBytes 10.9 Mbits/sec
[ ID] Interval Transfer Bandwidth
[ 5] 0.00-10.10 sec 0.00 Bytes 0.00 bits/sec sender
[ 5] 0.00-10.10 sec 10.5 MBytes 8.73 Mbits/sec receiver
–---------------------------------------------------------
Server listening on 5201Actually i think I'm getting confused here, the file transfer i get using explorer is roughtly 1.5MB/s
1 MB/sec = 8Mbps,
so 1.5MB/s x 8 = 12Mbps, which kind of means there is no problem i just lacked basics foundations binary a network guys explained this to me which kind does add up.