One way communication problem with IPSEC - *Fixed* [Changed MTU]

jits

Hi.

I don't even know where to begin. The three weeks from hell continues.

I have set up IPSEC between three sites. Sites A, B, C.

I'm now in the process of testing communication between the Sites A and B. From Site A, I can control everything on Site B. I can ping and I can do remote desktop. However….

From Site B, while I can ping the LAN and VOIP networks, I can't load webpages of the respective services on Site A. I can't load the PFsense page for Site A, nor can I load the VOIP config page.

I have configured access both ways on Site A and B for ICMP, TCP and UDP. Am I missing anything else?

Please take a look at the follow attachments. Perhaps there is something I am not seeing that you can. Thanks in advance.

This is IPSEC log from Site A

Last 50 IPsec log entries
Oct 8 19:53:55 racoon: [ –> Stores]: INFO: IPsec-SA established: ESP X06.X14.X6.X7[500]->X04.X88.X60.X86[500] spi=200774061(0xbf791ad)
Oct 8 19:53:55 racoon: [ –> Stores]: INFO: IPsec-SA established: ESP X04.X88.X60.X86[0]->X06.X14.X6.X7[0] spi=173053058(0xa509482)
Oct 8 19:53:55 racoon: [ –> Stores]: INFO: respond new phase 2 negotiation: X06.X14.X6.X7[500]<=>X04.X88.X60.X86[500]
Oct 8 19:50:24 racoon: [ –> Stores]: ERROR: X04.X88.X60.X86 give up to get IPsec-SA due to time up to wait.
Oct 8 19:49:54 racoon: ERROR: status mismatch (db:9 msg:3)
Oct 8 19:49:54 racoon: [ –> Stores]: INFO: initiate new phase 2 negotiation: X06.X14.X6.X7[500]<=>X04.X88.X60.X86[500]
Oct 8 19:49:53 racoon: [ –> Stores]: INFO: IPsec-SA established: ESP X06.X14.X6.X7[500]->X04.X88.X60.X86[500] spi=159761783(0x985c577)
Oct 8 19:49:53 racoon: [ –> Stores]: INFO: IPsec-SA established: ESP X04.X88.X60.X86[0]->X06.X14.X6.X7[0] spi=12933555(0xc559b3)
Oct 8 19:49:53 racoon: [ –> Stores]: INFO: initiate new phase 2 negotiation: X06.X14.X6.X7[500]<=>X04.X88.X60.X86[500]
Oct 8 19:49:53 racoon: [ –> Stores]: INFO: ISAKMP-SA established X06.X14.X6.X7[500]-X04.X88.X60.X86[500] spi:92fdb0a37916e7e3:bbbeefdf392328d6
Oct 8 19:49:53 racoon: INFO: Hashing X06.X14.X6.X7[500] with algo #2
Oct 8 19:49:53 racoon: [ –> Stores]: INFO: Hashing X04.X88.X60.X86[500] with algo #2
Oct 8 19:49:53 racoon: INFO: Adding remote and local NAT-D payloads.
Oct 8 19:49:53 racoon: NOTIFY: couldn't find the proper pskey, try to get one by the peer's address.
Oct 8 19:49:53 racoon: INFO: NAT not detected
Oct 8 19:49:53 racoon: INFO: NAT-D payload #0 verified
Oct 8 19:49:53 racoon: [ –> Stores]: INFO: Hashing X04.X88.X60.X86[500] with algo #2
Oct 8 19:49:53 racoon: INFO: NAT-D payload #-1 verified
Oct 8 19:49:53 racoon: INFO: Hashing X06.X14.X6.X7[500] with algo #2
Oct 8 19:49:53 racoon: INFO: Selected NAT-T version: RFC 3947
Oct 8 19:49:53 racoon: INFO: received Vendor ID: DPD
Oct 8 19:49:53 racoon: INFO: received broken Microsoft ID: FRAGMENTATION
Oct 8 19:49:53 racoon: INFO: received Vendor ID: RFC 3947
Oct 8 19:49:36 racoon: [ –> Stores]: INFO: phase2 sa deleted X06.X14.X6.X7-X04.X88.X60.X86
Oct 8 19:49:35 racoon: INFO: request for establishing IPsec-SA was queued due to no phase1 found.
Oct 8 19:49:35 racoon: [ –> Stores]: INFO: phase2 sa expired X06.X14.X6.X7-X04.X88.X60.X86
Oct 8 19:49:13 racoon: INFO: begin Aggressive mode.
Oct 8 19:49:13 racoon: [ –> Stores]: INFO: initiate new phase 1 negotiation: X06.X14.X6.X7[500]<=>X04.X88.X60.X86[500]
Oct 8 19:49:13 racoon: [ –> Stores]: INFO: IPsec-SA request for X04.X88.X60.X86 queued due to no phase1 found.
Oct 8 19:48:39 racoon: [ –> Stores]: INFO: ISAKMP-SA deleted X06.X14.X6.X7[500]-X04.X88.X60.X86[500] spi:8ad81f02d57a0973:3bc12b6ef0da602b
Oct 8 19:48:38 racoon: INFO: DPD: remote (ISAKMP-SA spi=8ad81f02d57a0973:3bc12b6ef0da602b) seems to be dead.
Oct 8 19:48:37 racoon: [ –> Stores]: ERROR: X04.X88.X60.X86 give up to get IPsec-SA due to time up to wait.
Oct 8 19:48:07 racoon: [ –> Stores]: INFO: respond new phase 2 negotiation: X06.X14.X6.X7[500]<=>X04.X88.X60.X86[500]
Oct 8 19:48:03 racoon: [ –-> Dunba]: INFO: IPsec-SA established: ESP X06.X14.X6.X7[500]->X09.X9.X24.X70[500] spi=170103059(0xa239113)
Oct 8 19:48:03 racoon: [ –-> Dunba]: INFO: IPsec-SA established: ESP X09.X9.X24.X70[0]->X06.X14.X6.X7[0] spi=185627839(0xb1074bf)
Oct 8 19:48:03 racoon: [ –-> Dunba]: INFO: initiate new phase 2 negotiation: X06.X14.X6.X7[500]<=>X09.X9.X24.X70[500]
Oct 8 19:48:02 racoon: [ –-> Dunba]: INFO: ISAKMP-SA established X06.X14.X6.X7[500]-X09.X9.X24.X70[500] spi:9ed43c83acf6fde6:5c46544e6bc7811e
Oct 8 19:48:02 racoon: INFO: Hashing X06.X14.X6.X7[500] with algo #2
Oct 8 19:48:02 racoon: [ –-> Dunba]: INFO: Hashing X09.X9.X24.X70[500] with algo #2
Oct 8 19:48:02 racoon: INFO: Adding remote and local NAT-D payloads.
Oct 8 19:48:02 racoon: NOTIFY: couldn't find the proper pskey, try to get one by the peer's address.
Oct 8 19:48:02 racoon: INFO: NAT not detected
Oct 8 19:48:02 racoon: INFO: NAT-D payload #0 verified
Oct 8 19:48:02 racoon: [ –-> Dunba]: INFO: Hashing X09.X9.X24.X70[500] with algo #2
Oct 8 19:48:02 racoon: INFO: NAT-D payload #-1 verified
Oct 8 19:48:02 racoon: INFO: Hashing X06.X14.X6.X7[500] with algo #2
Oct 8 19:48:02 racoon: INFO: Selected NAT-T version: RFC 3947
Oct 8 19:48:02 racoon: INFO: received Vendor ID: DPD
Oct 8 19:48:02 racoon: INFO: received broken Microsoft ID: FRAGMENTATION
Oct 8 19:48:02 racoon: INFO: received Vendor ID: RFC 3947

IPSEC log from Site B

Last 50 IPsec log entries
Oct 8 19:53:55 racoon: [Stores –-> ]: INFO: IPsec-SA established: ESP X04.X88.X60.X86[500]->X06.X14.X6.X7[500] spi=173053058(0xa509482)
Oct 8 19:53:55 racoon: [Stores –-> ]: INFO: IPsec-SA established: ESP X06.X14.X6.X7[0]->X04.X88.X60.X86[0] spi=200774061(0xbf791ad)
Oct 8 19:53:55 racoon: [Stores –-> ]: INFO: initiate new phase 2 negotiation: X04.X88.X60.X86[500]<=>X06.X14.X6.X7[500]
Oct 8 19:49:53 racoon: [Stores –-> ]: INFO: IPsec-SA established: ESP X04.X88.X60.X86[500]->X06.X14.X6.X7[500] spi=12933555(0xc559b3)
Oct 8 19:49:53 racoon: [Stores –-> ]: INFO: IPsec-SA established: ESP X06.X14.X6.X7[0]->X04.X88.X60.X86[0] spi=159761783(0x985c577)
Oct 8 19:49:53 racoon: [Stores –-> ]: INFO: respond new phase 2 negotiation: X04.X88.X60.X86[500]<=>X06.X14.X6.X7[500]
Oct 8 19:49:53 racoon: [Stores –-> ]: INFO: ISAKMP-SA established X04.X88.X60.X86[500]-X06.X14.X6.X7[500] spi:92fdb0a37916e7e3:bbbeefdf392328d6
Oct 8 19:49:53 racoon: INFO: NAT not detected
Oct 8 19:49:53 racoon: INFO: NAT-D payload #1 verified
Oct 8 19:49:53 racoon: [Stores –-> ]: INFO: Hashing X06.X14.X6.X7[500] with algo #2
Oct 8 19:49:53 racoon: INFO: NAT-D payload #0 verified
Oct 8 19:49:53 racoon: INFO: Hashing X04.X88.X60.X86[500] with algo #2
Oct 8 19:49:53 racoon: INFO: Hashing X04.X88.X60.X86[500] with algo #2
Oct 8 19:49:53 racoon: [Stores –-> ]: INFO: Hashing X06.X14.X6.X7[500] with algo #2
Oct 8 19:49:53 racoon: INFO: Adding remote and local NAT-D payloads.
Oct 8 19:49:53 racoon: NOTIFY: couldn't find the proper pskey, try to get one by the peer's address.
Oct 8 19:49:53 racoon: INFO: Selected NAT-T version: RFC 3947
Oct 8 19:49:53 racoon: INFO: received Vendor ID: DPD
Oct 8 19:49:53 racoon: INFO: received Vendor ID: draft-ietf-ipsec-nat-t-ike-00
Oct 8 19:49:53 racoon: INFO: received Vendor ID: draft-ietf-ipsec-nat-t-ike-02
Oct 8 19:49:53 racoon: INFO: received Vendor ID: draft-ietf-ipsec-nat-t-ike-02
Oct 8 19:49:53 racoon: INFO: received Vendor ID: RFC 3947
Oct 8 19:49:53 racoon: INFO: received broken Microsoft ID: FRAGMENTATION
Oct 8 19:49:53 racoon: INFO: begin Aggressive mode.
Oct 8 19:49:53 racoon: [Stores –-> ]: INFO: respond new phase 1 negotiation: X04.X88.X60.X86[500]<=>X06.X14.X6.X7[500]
Oct 8 19:49:49 racoon: INFO: unsupported PF_KEY message REGISTER
Oct 8 19:49:49 racoon: INFO: X04.X88.X60.X86[500] used for NAT-T
Oct 8 19:49:49 racoon: [Self]: INFO: X04.X88.X60.X86[500] used as isakmp port (fd=17)
Oct 8 19:49:49 racoon: INFO: X04.X88.X60.X86[4500] used for NAT-T
Oct 8 19:49:49 racoon: [Self]: INFO: X04.X88.X60.X86[4500] used as isakmp port (fd=16)
Oct 8 19:49:49 racoon: INFO: Reading configuration from "/var/etc/racoon.conf"
Oct 8 19:49:49 racoon: INFO: @(#)This product linked OpenSSL 0.9.8n 24 Mar 2010 (http://www.openssl.org/)
Oct 8 19:49:49 racoon: INFO: @(#)ipsec-tools 0.7.3 (http://ipsec-tools.sourceforge.net)

Config Screenshots

ipsec_ping_trace.jpg_thumb

ipsec_webpage_noaccess.jpg_thumb

SiteA_to_SiteB_IPSEC.jpg_thumb

SiteB_to_SiteA_IPSEC.jpg_thumb

jimp

What do the firewall rules on the IPsec tab look like on each router?

jits

Hi Jimp.

Please disregard this post. I've just been handed a bigger fish to fry.

thanks anyway.

Jits

jits

Hi Jimp,

Took a look at IPsec quickly today again, and I've managed to get both ends working (good). As you know from my above posts, I was only able to access the pfsense router successfully from one end of the tunnel. I adjusted the MTU to 1492 on both ends, and I'm now able to somewhat avoid IP Fragmentation and MTU issues with IPSec.

Does anyone have the ideal MTU setting, especially if you are using SIP (VOIP) phones across subnets?

Thanks, Jits

jits

Hi. okay..I have some time now.

I am trying to join the networks of three different locations and supply one location with VOIP via IPsec.

For PFsense, I am using three Jetway atom processor fanless, LogicSupply systems with RE type NICS.

For VPN I'm using IPSec with the Following Phase 1 settings…

Negotiation Mode: Aggressive
Encryption Alg: Blowfish
Hash Alg: sha1

For Phase 2, the following is set...

Protocol: ESP
Encryption Alg: Blowfish
Hash Alg: sha1

The other settings are set, but not revealed.

For Pre-Shared keys, I followed directions from here: http://wiki.nil.com/Pre-Shared_Key_Generation

The problem experienced has been resolved, thus far, by setting the MTU to 1492 on both ends. On Site A, I could access pfsense router on Site B in quick time. However, From Site B, the only thing I could do was enter credentials for Site A Pfsense router which eventually resulted in a Connection Reset error.

Searching for "ipsec connection reset error" led me to a Cisco write up on Resolving IP Fragmentation, MTU, MSS, and PMTUD issues with GRE and IPSec.

Hope this helps someone else with similar problems.

eazydor

i have similar problems, see topic http://forum.pfsense.org/index.php/topic,29105.0.html

could you please provide the link you write about, cisco note about resolving ip fragmentation? would be awesome..
then again, you just set the mtu on both ends and that was it?

thanks.

jits

i have similar problems, see topic http://forum.pfsense.org/index.php/topic,29105.0.html

could you please provide the link you write about, cisco note about resolving ip fragmentation? would be awesome..
then again, you just set the mtu on both ends and that was it?

thanks.

Yes. Almost. I set the MTU to 1492 and today, added the third network via IPSec from Site A. An MTU of 1492 from Site C could not communicate with the PFsense Router at Site A or the VOIP Server at Site A, which is also on a VLAN.

To resolve this, From Site C, using windows 7 and the CMD screen, I did –->

Microsoft Windows [Version 6.1.7600]
Copyright 2009 Microsoft Corporation. All rights reserved.

C:\Users\supad>ping -f -l 1492 192.168.49.10

Pinging 192.168.49.10 with 1492 bytes of data:
Packet needs to be fragmented but DF set.
Packet needs to be fragmented but DF set.
Packet needs to be fragmented but DF set.
Packet needs to be fragmented but DF set.

Ping statistics for 192.168.49.10:
Packets: Sent = 4, Received = 0, Lost = 4 (100% loss),

C:\Users\supad>

Then, I went down to 1392 and got this –->

Microsoft Windows [Version 6.1.7600]
Copyright 2009 Microsoft Corporation. All rights reserved.

C:\Users\supad>ping -f -l 1392 192.168.49.10

Pinging 192.168.49.10 with 1392 bytes of data:
Reply from 192.168.49.10: bytes=1392 time<1ms TTL=63
Reply from 192.168.49.10: bytes=1392 time=3ms TTL=63
Reply from 192.168.49.10: bytes=1392 time<1ms TTL=63
Reply from 192.168.49.10: bytes=1392 time<1ms TTL=63

Ping statistics for 192.168.49.10:
Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
Minimum = 0ms, Maximum = 3ms, Average = 0ms

C:\Users\supad>

So, now, my MTU is set to 1392 on all PFsense Routers using IPSec.

I do have to say, that I removed pfSense 2.0 and now running pfSense 1.2.3 to achieve these results.

Here is the link you asked for

http://www.cisco.com/en/US/tech/tk827/tk369/technologies_white_paper09186a00800d6979.shtml

Please remember to share your fixes on the forum and label them as such so others such as myself can quickly find the help we need. Time is precious and like it or not, we do need each other.

Thanks for the interest…Jits

Here is another Link relating to MTU (Me To You)

http://help.expedient.com/broadband/mtu_ping_test.shtml

I hope this helps

eazydor

thanks, generally good paper to get a foot into what's, how, why fragmentation is happening. i posted my post in the ipsec section because i don't think its related to 2.0, but would be interesting anyway.. did you experienced problems with the same 1.2.3 setup on 2.0?

to me just setting the mtu on wan interface didn't solved my overhead problem.

you do site to site with psk, i think, there is more overhead depending on your configuration, lets say ipsec with rsa keys and x-auth has more overhead then, for site to site sufficient psk.

but as you can read on the other post, i`m by far not an network engineer and have hard times imagine myself what exactly happening in that profoundness.

if you're interested, jim advised to do mss clamping on vpn traffic, mentioned in the other post.

http://forum.pfsense.org/index.php/topic,29105.msg151281