Poor IPSec performance



  • Hi,

    I have the following setup:

    PC1–- Switch1---pfs1---pfs2---Switch2---PC2

    pfs1 and pfs2 are running 1.2.2

    pfs1:
    Xeon Dual Core 2.33GHz with 1GB RAM
    1 WAN and 1 LAN
    Machine is using all 10/100 Intel Pro network cards.
    WAN = Currently directly connected to pfS2 via 1000 Mbps (Full duplex)
    LAN = 100 Mbps (Full duplex)

    pfs2:
    Xeon Dual Core 2.33GHz with 1GB RAM
    1 WAN and 1 LAN
    Machine is using all 10/100 Intel Pro network cards.
    WAN = Currently directly connected to pfS2 via 1000 Mbps (Full duplex)
    LAN = 100 Mbps (Full duplex)

    IPSEC Config:
    Phase 1:
    Negotiation: Main
    Encryption: AES-256
    Hash: SHA1
    DH Key Group: 2 (1024bit)
    Authentication method: Pre-Shared Key

    Phase 2:
    Protocol: ESP
    Encryption: AES-256
    Hash: SHA1
    PFS Key: 2 (1024bit)

    All packet filtering is turned off. I am only interested in IPSec tunnel between the two points.

    Ping from PC1 to PC2 (and visa versa) results in 1ms or less.

    Attached to Switch2 is AD controller and other Windows servers (File share, print, DHCP, etc.) and the internet.

    Switch1 is L3 and acts as Default gw for PC1. Switch2 is L3 and acts as Default gw for PC2 and servers attached.

    Switch1 is bootprelay. PC1 have no problem in getting an IP.

    When I try to login on PC1 it stalles. TCPdump on the LAN interface on pfs1 and pfs2 shows that trafic is flowing, thoug it is VERY slow. The login times out.

    The same problem with performance is shown when I try to FTP from PC1 to PC2. The traffic graph is pfSense shows that 1-2 Mbps is being used. CPU load is close to zero.

    When IPSec is turned off there is no problem. Everything runs fast and the traffic graph shows near 100 Mbps use.

    Have I missed something? A configuration error? I'm lost and I have to get this running by Friday   :(
    Any help will be greatly appreciated!



  • What are your IP's for your pfSense LAN and L3 switches?  what is the Subnet Mask used?



  • PC1–- Switch1---pfs1---pfs2---Switch2---PC2

    Net between PC1 and Switch1 is 172.16.100.0/24
    Net between Switch1 and pfs1 is 192.168.42.4/30
    Net between pfs1 and pfs2 is 192.168.42.0/30
    Net between pfs2 and Switch2 is 172.16.10.0/24
    Net between PC2 and Switch2 is 172.16.65.0/26

    (All this i very simple put in terms of my real production net on the switch2 side)

    Remember that Switch2 acts as a gw for the internet also.

    My IPSec config for the networks is:

    pfs1:
    Local net = 172.16.100.0/24
    Remote net = 0.0.0.0/0

    Interface = WAN
    Remote gw = 192.168.42.1

    pfs2:
    Local net = 0.0.0.0/0
    Remote net = 172.16.100.0/24

    Interface = WAN
    Remote gw = 192.168.42.2

    Hopes this can give you an idea.

    Again with the IPSec turned off everything runs as you would expect. With IPSec turned on pings comes throug but when you starts to move larger amounts of data nothings happens.

    Cheers
    Ole



  • Couple of things….

    Why do you have 0.0.0.0/0 listed?

    Also it looks like you are using multiple subnets behind your pfsense.  Either try creating static routes or changing the LAN to /16.

    Net between pfs2 and Switch2 is 172.16.10.0/24
    Net between PC2 and Switch2 is 172.16.65.0/26



  • I have 0.0.0.0/0 liste because I want all traffic encrypted between the two pfsense machines.



  • Is this just a test or is this a prod enviroment?  Why are you directly connecting the 2 pfSense boxes to each other?  is this a point2point tunnel?  Do you want internet requests to go through the IPSEC tunnel?  Have you tried specifying the correct subnet rather than 0.0.0.0/0?  Did you modify the Outbound NAT?  You might be having a conflict with the built-in NAT trying to send anything not specified out the WAN rather than the ipsec tunnel.



  • It is suppose to go in the production environment.

    The 2 pfsense boxes are connected via a L2 connection with the soul purpose of encrypting all traffic between them.

    I have tried only having the 172.16.10.0/24 net behind pfs2 and only 172.16.100.0/26 behind pfs1. With requests from one net to the other icmp goes through. No problem. I can telnet through. But if I do a "show log" on the switch I'm telnetting to it briefly starts and then comes to a halt. This is also true if I connect to a windows share. A small amount of traffic is being sendt and the nothing.

    Again all works without IPSec.

    Modyfied::

    I should also menthion that all packet filtering and NAT is turned off.

    I was thinking about "Bypass firewall rules for traffic on the same interface", " Block RFC1918 Private Networks" and "Block bogon networks:" options. Can they play a part?



  • Another discovery I made right now is that when ever the length of an ESP packet is 1480 everything stops. The next line in a tcpdump is always a esp (in lower case). See also print out.

    Can it be the MTU in IPsec?

    13:00:47.407446 IP 192.168.42.2 > 192.168.42.1: ESP(spi=0x01e083a4,seq=0x3), length 1480
    13:00:47.407450 IP 192.168.42.2 > 192.168.42.1: esp



  • I have now performed a ping test where I tried to increase the packet size. When a packet must be fragmented the problem appears.

    Does this make sense?

    Modifyed::

    The max packet size I can ping with and get an answer is 1410. This results in a ESP packet of 1476. Anything larger than that, results in a ESP packet of 1480 and no data is coming through.

    tcpdump -vv

    Does not go through:
    13:19:19.391590 IP (tos 0x0, ttl 64, id 21771, offset 0, flags [+], proto ESP (50), length 1500) 192.168.42.1 > 192.168.42.2: ESP(spi=0x045c0bdd,seq=0xa), length 1480
    13:19:19.391593 IP (tos 0x0, ttl 64, id 21771, offset 1480, flags [none], proto ESP (50), length 32) 192.168.42.1 > 192.168.42.2: esp

    Goes through:
    13:19:23.807651 IP (tos 0x0, ttl 64, id 49629, offset 0, flags [none], proto ESP (50), length 1496) 192.168.42.1 > 192.168.42.2: ESP(spi=0x045c0bdd,seq=0xb), length 1476



  • Perhaps a "esp_frag 552;" in the racoon.conf will help on this issue but I don't know how to add it to that conf file permanent. Every time I reload racoon the line disappears from the conf file.

    Does anyone know if " esp_frag 552;" will help on this issue?



  • Does anyone have any idea on how to fix this issue?



  • Could you upgrade to 1.2.3 http://blog.pfsense.org/?p=377



  • Sure. How do I get a hold off it?





  • Upgraded to 1.2.3 with no success. Still the same problem  :-[

    Can it be a HW failure or HW incompatibility?

    I've tried a cross-over cable between the two servers with 1000baseTX. I've tried to force the interfaces to 100baseTX with no success and I've tried placing a switch in between. All with the same result.



  • did that racoon config helped you ?
    It seems like pmtu discovery problem. Can you take an tcpdump output file of 1min of traffic and attach here either on the enc0 interface and LAN and wan ones



  • I was not able to make the changes to the racoon config. It's overwriten every time racoon starts.

    I will make a tcpdump asap and attach it here.



  • Here is the dump from the wan interface on pfs1.

    I'm not getting any packages on enc0.

    I've updated both boxes to 1.2.3 20090224-0050

    em1_dump.txt



  • Has anyone any idea?



  • Commercial support might be the way to go



  • Hi Olejak,
    have you resolved the problem?
    Just interesting…
    Actually it is normal for large packets:
    13:00:47.407446 IP 192.168.42.2 > 192.168.42.1: ESP(spi=0x01e083a4,seq=0x3), length 1480
    13:00:47.407450 IP 192.168.42.2 > 192.168.42.1: esp

    I am just wondering whether you receive the same two packets on the other end? I.e. if it is a trace from FW1 do you see the same packets on FW2?



  • Try placing a switch between the 2 pfSense servers.  I ran into issues using crossover cables that were cleared up when I used a gig switch instead.

    -John



  • Set the MTU of your adapters down to 1400 and try again.  Large packets + IPSEC + no fragmentation is a common problem.

    For Windows, you can use this: http://www.dslreports.com/drtcp



  • @valnar:

    For Windows, you can use this: http://www.dslreports.com/drtcp

    Sounds very interesting. Could you explain in more details? -)



  • Can you please give me a little me details.  I have setup a similiar configuration.  I had a dual 866 with 1 gb of ram  connected over 100 mb connected to compac dl 380 with 100 nics.  no speed issues.  Teested with serveral different issues.
    RC



  • Just google MTU, fragmentation, IPSEC and VPN.



  • Hello Olejack,

    Did you finally solve your issue ?
    I'd be very interested as I have the same right now.
    I've tried to lower MTU on the WAN interface configuration but it's not taken into account even after a reboot.
    A ifconfig shows an MTU of 1500 even though I entered 1300.
    I can't find any topic where someone succeeded in modifying the IPSEC MTU.
    Im' considering to replace ipsec with openvpn maybe.

    About commercial support, I've asked once for tinydns support and never had any reply …

    Thanks for your help.


Log in to reply