Enable MSS clamping on VPN traffic doesn't work

ggzengel · Dec 7, 2010, 3:19 AM

I have multiple areca controllers behind ipsec tunnels and their web page doesn't load properly.
Befor with bintec router to bintec router it worked.

With pfsense if I ping -l 1391 it doesn't work. If I ping -l 1390 it works.

Enabling "Enable MSS clamping on VPN traffic" with value 1200 doesn't clear the problem.

I toggled "Clear invalid DF bits instead of dropping the packet".
Disabled "Insert a stronger id into IP header of packets passing through the filter."
No change.

cmb · Dec 7, 2010, 9:35 AM

Run:
grep scrub /tmp/rules.debug

and:
grep vpn /tmp/rules.debug

and post the output.

and get a packet capture of the traffic

ggzengel · Dec 7, 2010, 4:28 PM

scrub in from any to <vpns>max-mss 1200
scrub in on $CABLE all random-id fragment reassemble
scrub in on $LAN all random-id fragment reassemble

table <vpns>{ 10.19.8.0/22 192.168.18.0/23 192.168.165.0/24 10.19.12.0/22 192.168.192.0/24 192.168.1.0/24 172.19.16.0/22 10.19.28.0/22 192.168.33.0/24 192.168.254.0/24 192.168.251.0/24 10.19.116.0/22 10.0.0.0/28 192.168.29.0/24 10.19.120.0/22 10.19.112.0/23 }

I only can capture LAN or CABLE. Not IPSEC.
On LAN:
16:12:45.216723 IP 10.19.1.150 > 192.168.165.77: ICMP echo request, id 95, seq 53555, length 1399
16:12:47.103431 IP 10.19.1.150 > 192.168.165.77: ICMP echo request, id 95, seq 53563, length 1399
16:12:49.100064 IP 10.19.1.150 > 192.168.165.77: ICMP echo request, id 95, seq 53573, length 1399
16:12:51.112434 IP 10.19.1.150 > 192.168.165.77: ICMP echo request, id 95, seq 53581, length 1399

In firewalllog I get
Dec 7 16:14:19 enc0 192.168.165.77 10.19.1.150 ICMP
Dec 7 16:14:19 enc0 192.168.165.77 10.19.1.150 ICMP

Now I patched the Packet Capture form and added interface enc0:
16:24:12.545890 (authentic,confidential): SPI 0x73a9f4f1: IP 10.19.1.150 > 192.168.165.77: ICMP echo request, id 95, seq 57021, length 1399
16:24:12.573942 (authentic,confidential): SPI 0x0594434b: IP 192.168.165.77 > 10.19.1.150: ICMP echo reply, id 95, seq 57021, length 1392
16:24:12.576987 (authentic,confidential): SPI 0x0594434b: IP 192.168.165.77 > 10.19.1.150: icmp
16:24:14.208635 (authentic,confidential): SPI 0x73a9f4f1: IP 10.19.1.150 > 192.168.165.77: ICMP echo request, id 95, seq 57028, length 1399
16:24:14.239749 (authentic,confidential): SPI 0x0594434b: IP 192.168.165.77 > 10.19.1.150: ICMP echo reply, id 95, seq 57028, length 1392
16:24:14.242213 (authentic,confidential): SPI 0x0594434b: IP 192.168.165.77 > 10.19.1.150: icmp
16:24:16.206258 (authentic,confidential): SPI 0x73a9f4f1: IP 10.19.1.150 > 192.168.165.77: ICMP echo request, id 95, seq 57039, length 1399
16:24:16.236389 (authentic,confidential): SPI 0x0594434b: IP 192.168.165.77 > 10.19.1.150: ICMP echo reply, id 95, seq 57039, length 1392
16:24:16.239451 (authentic,confidential): SPI 0x0594434b: IP 192.168.165.77 > 10.19.1.150: icmp
16:24:18.203888 (authentic,confidential): SPI 0x73a9f4f1: IP 10.19.1.150 > 192.168.165.77: ICMP echo request, id 95, seq 57050, length 1399
16:24:18.234164 (authentic,confidential): SPI 0x0594434b: IP 192.168.165.77 > 10.19.1.150: ICMP echo reply, id 95, seq 57050, length 1392
16:24:18.237581 (authentic,confidential): SPI 0x0594434b: IP 192.168.165.77 > 10.19.1.150: icmp
16:24:20.201519 (authentic,confidential): SPI 0x73a9f4f1: IP 10.19.1.150 > 192.168.165.77: ICMP echo request, id 95, seq 57061, length 1399
16:24:20.231906 (authentic,confidential): SPI 0x0594434b: IP 192.168.165.77 > 10.19.1.150: ICMP echo reply, id 95, seq 57061, length 1392
16:24:20.236719 (authentic,confidential): SPI 0x0594434b: IP 192.168.165.77 > 10.19.1.150: icmp

Feature request: Put enc0 to the packet capture interface.</vpns></vpns>

ggzengel · Dec 7, 2010, 4:38 PM

Now again with more details:
I think the bad checksum is a interpreter failure, because the ping in the second part is working. Windows should ignore wrong packet.
Can somebody verify this?

Not working:
16:31:16.037859 (authentic,confidential): SPI 0x73a9f4f1: (tos 0x0, ttl 126, id 17458, offset 0, flags [none], proto ICMP (1), length 1419, bad cksum 80a1 (->81a1)!)
10.19.1.150 > 192.168.165.77: ICMP echo request, id 95, seq 58999, length 1399
16:31:16.067375 (authentic,confidential): SPI 0x0594434b: (tos 0x0, ttl 63, id 18113, offset 0, flags [+], proto ICMP (1), length 1412)
192.168.165.77 > 10.19.1.150: ICMP echo reply, id 95, seq 58999, length 1392
16:31:16.074431 (authentic,confidential): SPI 0x0594434b: (tos 0x0, ttl 63, id 18113, offset 1392, flags [none], proto ICMP (1), length 27)
192.168.165.77 > 10.19.1.150: icmp
16:31:17.784822 (authentic,confidential): SPI 0x73a9f4f1: (tos 0x0, ttl 126, id 17473, offset 0, flags [none], proto ICMP (1), length 1419, bad cksum 8092 (->8192)!)
10.19.1.150 > 192.168.165.77: ICMP echo request, id 95, seq 59008, length 1399
16:31:17.818824 (authentic,confidential): SPI 0x0594434b: (tos 0x0, ttl 63, id 18114, offset 0, flags [+], proto ICMP (1), length 1412)
192.168.165.77 > 10.19.1.150: ICMP echo reply, id 95, seq 59008, length 1392
16:31:17.822738 (authentic,confidential): SPI 0x0594434b: (tos 0x0, ttl 63, id 18114, offset 1392, flags [none], proto ICMP (1), length 27)
192.168.165.77 > 10.19.1.150: icmp
16:31:19.782495 (authentic,confidential): SPI 0x73a9f4f1: (tos 0x0, ttl 126, id 17488, offset 0, flags [none], proto ICMP (1), length 1419, bad cksum 8083 (->8183)!)
10.19.1.150 > 192.168.165.77: ICMP echo request, id 95, seq 59018, length 1399
16:31:19.811340 (authentic,confidential): SPI 0x0594434b: (tos 0x0, ttl 63, id 18115, offset 0, flags [+], proto ICMP (1), length 1412)
192.168.165.77 > 10.19.1.150: ICMP echo reply, id 95, seq 59018, length 1392
16:31:19.816631 (authentic,confidential): SPI 0x0594434b: (tos 0x0, ttl 63, id 18115, offset 1392, flags [none], proto ICMP (1), length 27)
192.168.165.77 > 10.19.1.150: icmp

Working:
16:31:28.509078 (authentic,confidential): SPI 0x73a9f4f1: (tos 0x0, ttl 126, id 17538, offset 0, flags [none], proto ICMP (1), length 1418, bad cksum 8052 (->8152)!)
10.19.1.150 > 192.168.165.77: ICMP echo request, id 95, seq 59061, length 1398
16:31:28.538236 (authentic,confidential): SPI 0x0594434b: (tos 0x0, ttl 63, id 18116, offset 0, flags [none], proto ICMP (1), length 1418)
192.168.165.77 > 10.19.1.150: ICMP echo reply, id 95, seq 59061, length 1398
16:31:29.521018 (authentic,confidential): SPI 0x73a9f4f1: (tos 0x0, ttl 126, id 17551, offset 0, flags [none], proto ICMP (1), length 1418, bad cksum 8045 (->8145)!)
10.19.1.150 > 192.168.165.77: ICMP echo request, id 95, seq 59068, length 1398
16:31:29.549053 (authentic,confidential): SPI 0x0594434b: (tos 0x0, ttl 63, id 18117, offset 0, flags [none], proto ICMP (1), length 1418)
192.168.165.77 > 10.19.1.150: ICMP echo reply, id 95, seq 59068, length 1398
16:31:30.535453 (authentic,confidential): SPI 0x73a9f4f1: (tos 0x0, ttl 126, id 17556, offset 0, flags [none], proto ICMP (1), length 1418, bad cksum 8040 (->8140)!)
10.19.1.150 > 192.168.165.77: ICMP echo request, id 95, seq 59073, length 1398
16:31:30.567739 (authentic,confidential): SPI 0x0594434b: (tos 0x0, ttl 63, id 18118, offset 0, flags [none], proto ICMP (1), length 1418)
192.168.165.77 > 10.19.1.150: ICMP echo reply, id 95, seq 59073, length 1398
16:31:31.549893 (authentic,confidential): SPI 0x73a9f4f1: (tos 0x0, ttl 126, id 17563, offset 0, flags [none], proto ICMP (1), length 1418, bad cksum 8039 (->8139)!)
10.19.1.150 > 192.168.165.77: ICMP echo request, id 95, seq 59080, length 1398
16:31:31.579935 (authentic,confidential): SPI 0x0594434b: (tos 0x0, ttl 63, id 18119, offset 0, flags [none], proto ICMP (1), length 1418)
192.168.165.77 > 10.19.1.150: ICMP echo reply, id 95, seq 59080, length 1398

cmb · Dec 8, 2010, 9:09 AM

MSS is TCP-only, it has no impact on ICMP, ICMP has no concept of MSS. It is setting the proper MSS clamping, just need to see some TCP traffic.

ggzengel · Dec 8, 2010, 5:08 PM

Today the max size is 1472 bytes.
ping -l 1473 doesn't reply.
ping -f -l 1473 says to clear DF bit.

I have changed ICMP rules to allow any ICMP from WAN and IPSEC.
But the webpage isn't reachable.

I can ping -l 1600 to bintec router, windows server, …

If i ping the areca controller over pfsense-pfsense it don't works to.
I never tried to ping a areca controller with ping -l 1473 befor. But the webpage worked befor.

14:39:08.407301 (authentic,confidential): SPI 0x0f45526b: (tos 0x0, ttl 63, id 19449, offset 0, flags [+], proto TCP (6), length 1412)
    192.168.165.77.80 > 10.19.1.150.49896: Flags [P.], seq 2348573:2349945, ack 707714766, win 1446, length 1372
14:39:08.413007 (authentic,confidential): SPI 0x0f45526b: (tos 0x0, ttl 63, id 19449, offset 1392, flags [none], proto TCP (6), length 94)
    192.168.165.77 > 10.19.1.150: tcp
14:39:08.414011 (authentic,confidential): SPI 0x0f45526b: (tos 0x0, ttl 63, id 19450, offset 0, flags [+], proto TCP (6), length 1412)
    192.168.165.77.80 > 10.19.1.150.49896: Flags [P.], seq 1446:2818, ack 1, win 1446, length 1372
14:39:08.414218 (authentic,confidential): SPI 0x0f45526b: (tos 0x0, ttl 63, id 19450, offset 1392, flags [none], proto TCP (6), length 94)
    192.168.165.77 > 10.19.1.150: tcp
14:39:08.414473 (authentic,confidential): SPI 0x0f45526b: (tos 0x0, ttl 63, id 19451, offset 0, flags [+], proto TCP (6), length 1412)
    192.168.165.77.80 > 10.19.1.150.49896: Flags [P.], seq 2892:4264, ack 1, win 1446, length 1372
14:39:08.414563 (authentic,confidential): SPI 0x0f45526b: (tos 0x0, ttl 63, id 19451, offset 1392, flags [none], proto TCP (6), length 94)
    192.168.165.77 > 10.19.1.150: tcp
14:39:08.415757 (authentic,confidential): SPI 0x0f45526b: (tos 0x0, ttl 63, id 19452, offset 0, flags [+], proto TCP (6), length 1412)
    192.168.165.77.80 > 10.19.1.150.49896: Flags [P.], seq 4338:5710, ack 1, win 1446, length 1372
14:39:08.418103 (authentic,confidential): SPI 0x0f45526b: (tos 0x0, ttl 63, id 19452, offset 1392, flags [none], proto TCP (6), length 94)
    192.168.165.77 > 10.19.1.150: tcp
14:39:08.418361 (authentic,confidential): SPI 0x0f45526b: (tos 0x0, ttl 63, id 19453, offset 0, flags [+], proto TCP (6), length 1412)
    192.168.165.77.80 > 10.19.1.150.49896: Flags [P.], seq 5784:7156, ack 1, win 1446, length 1372
14:39:08.418449 (authentic,confidential): SPI 0x0f45526b: (tos 0x0, ttl 63, id 19453, offset 1392, flags [none], proto TCP (6), length 94)
    192.168.165.77 > 10.19.1.150: tcp
14:39:08.418707 (authentic,confidential): SPI 0x0f45526b: (tos 0x0, ttl 63, id 19454, offset 0, flags [+], proto TCP (6), length 1412)
    192.168.165.77.80 > 10.19.1.150.49896: Flags [P.], seq 7230:8602, ack 1, win 1446, length 1372
14:39:08.418796 (authentic,confidential): SPI 0x0f45526b: (tos 0x0, ttl 63, id 19454, offset 1392, flags [none], proto TCP (6), length 94)
    192.168.165.77 > 10.19.1.150: tcp
14:39:08.419072 (authentic,confidential): SPI 0x0f45526b: (tos 0x0, ttl 63, id 19455, offset 0, flags [+], proto TCP (6), length 1412)
    192.168.165.77.80 > 10.19.1.150.49896: Flags [P.], seq 8676:10048, ack 1, win 1446, length 1372
14:39:08.419240 (authentic,confidential): SPI 0x0f45526b: (tos 0x0, ttl 63, id 19455, offset 1392, flags [none], proto TCP (6), length 94)
    192.168.165.77 > 10.19.1.150: tcp
14:39:08.420008 (authentic,confidential): SPI 0x0f45526b: (tos 0x0, ttl 63, id 19456, offset 0, flags [+], proto TCP (6), length 1412)
    192.168.165.77.80 > 10.19.1.150.49896: Flags [P.], seq 10122:11494, ack 1, win 1446, length 1372
14:39:08.420214 (authentic,confidential): SPI 0x0f45526b: (tos 0x0, ttl 63, id 19456, offset 1392, flags [none], proto TCP (6), length 94)
    192.168.165.77 > 10.19.1.150: tcp
14:39:08.420469 (authentic,confidential): SPI 0x0f45526b: (tos 0x0, ttl 63, id 19457, offset 0, flags [+], proto TCP (6), length 1412)
    192.168.165.77.80 > 10.19.1.150.49896: Flags [P.], seq 11568:12940, ack 1, win 1446, length 1372
14:39:08.425213 (authentic,confidential): SPI 0x0f45526b: (tos 0x0, ttl 63, id 19457, offset 1392, flags [none], proto TCP (6), length 94)
    192.168.165.77 > 10.19.1.150: tcp
14:39:08.425623 (authentic,confidential): SPI 0x0f45526b: (tos 0x0, ttl 63, id 19458, offset 0, flags [+], proto TCP (6), length 1412)
    192.168.165.77.80 > 10.19.1.150.49896: Flags [P.], seq 13014:14386, ack 1, win 1446, length 1372
14:39:08.426707 (authentic,confidential): SPI 0x0f45526b: (tos 0x0, ttl 63, id 19458, offset 1392, flags [none], proto TCP (6), length 94)
    192.168.165.77 > 10.19.1.150: tcp
14:39:08.431746 (authentic,confidential): SPI 0x0f45526b: (tos 0x0, ttl 63, id 19459, offset 0, flags [+], proto TCP (6), length 1412)
    192.168.165.77.80 > 10.19.1.150.49896: Flags [P.], seq 14460:15832, ack 1, win 1446, length 1372
14:39:08.432700 (authentic,confidential): SPI 0x0f45526b: (tos 0x0, ttl 63, id 19459, offset 1392, flags [none], proto TCP (6), length 94)
    192.168.165.77 > 10.19.1.150: tcp
14:39:08.432956 (authentic,confidential): SPI 0x0f45526b: (tos 0x0, ttl 63, id 19460, offset 0, flags [+], proto TCP (6), length 1412)
    192.168.165.77.80 > 10.19.1.150.49896: Flags [P.], seq 15906:17278, ack 1, win 1446, length 1372
14:39:08.442570 (authentic,confidential): SPI 0x0f45526b: (tos 0x0, ttl 63, id 19460, offset 1392, flags [none], proto TCP (6), length 94)
    192.168.165.77 > 10.19.1.150: tcp
14:39:08.445738 (authentic,confidential): SPI 0x0f45526b: (tos 0x0, ttl 63, id 19461, offset 0, flags [+], proto TCP (6), length 1412)
    192.168.165.77.80 > 10.19.1.150.49896: Flags [P.], seq 17352:18724, ack 1, win 1446, length 1372
14:39:08.452087 (authentic,confidential): SPI 0x0f45526b: (tos 0x0, ttl 63, id 19461, offset 1392, flags [none], proto TCP (6), length 94)
    192.168.165.77 > 10.19.1.150: tcp
14:39:08.452345 (authentic,confidential): SPI 0x0f45526b: (tos 0x0, ttl 63, id 19462, offset 0, flags [+], proto TCP (6), length 1412)
    192.168.165.77.80 > 10.19.1.150.49896: Flags [P.], seq 18798:20170, ack 1, win 1446, length 1372
14:39:08.452433 (authentic,confidential): SPI 0x0f45526b: (tos 0x0, ttl 63, id 19462, offset 1392, flags [none], proto TCP (6), length 94)
    192.168.165.77 > 10.19.1.150: tcp
14:39:08.452691 (authentic,confidential): SPI 0x0f45526b: (tos 0x0, ttl 63, id 19463, offset 0, flags [+], proto TCP (6), length 1412)
    192.168.165.77.80 > 10.19.1.150.49896: Flags [P.], seq 20244:21616, ack 1, win 1446, length 1372
14:39:08.452777 (authentic,confidential): SPI 0x0f45526b: (tos 0x0, ttl 63, id 19463, offset 1392, flags [none], proto TCP (6), length 94)
    192.168.165.77 > 10.19.1.150: tcp
14:39:08.454856 (authentic,confidential): SPI 0x0f45526b: (tos 0x0, ttl 63, id 19464, offset 0, flags [+], proto TCP (6), length 1412)
    192.168.165.77.80 > 10.19.1.150.49896: Flags [P.], seq 21690:23062, ack 1, win 1446, length 1372
14:39:08.460558 (authentic,confidential): SPI 0x0f45526b: (tos 0x0, ttl 63, id 19464, offset 1392, flags [none], proto TCP (6), length 94)
    192.168.165.77 > 10.19.1.150: tcp
14:39:08.460815 (authentic,confidential): SPI 0x0f45526b: (tos 0x0, ttl 63, id 19465, offset 0, flags [+], proto TCP (6), length 1412)
    192.168.165.77.80 > 10.19.1.150.49896: Flags [P.], seq 23136:24508, ack 1, win 1446, length 1372
14:39:08.460902 (authentic,confidential): SPI 0x0f45526b: (tos 0x0, ttl 63, id 19465, offset 1392, flags [none], proto TCP (6), length 94)
    192.168.165.77 > 10.19.1.150: tcp
14:39:08.461182 (authentic,confidential): SPI 0x0f45526b: (tos 0x0, ttl 63, id 19466, offset 0, flags [+], proto TCP (6), length 1412)
    192.168.165.77.80 > 10.19.1.150.49896: Flags [P.], seq 24582:25954, ack 1, win 1446, length 1372
14:39:08.461269 (authentic,confidential): SPI 0x0f45526b: (tos 0x0, ttl 63, id 19466, offset 1392, flags [none], proto TCP (6), length 94)
    192.168.165.77 > 10.19.1.150: tcp
14:39:08.465121 (authentic,confidential): SPI 0x0f45526b: (tos 0x0, ttl 63, id 19467, offset 0, flags [+], proto TCP (6), length 1412)
    192.168.165.77.80 > 10.19.1.150.49896: Flags [P.], seq 26028:27400, ack 1, win 1446, length 1372
14:39:08.471804 (authentic,confidential): SPI 0x0f45526b: (tos 0x0, ttl 63, id 19467, offset 1392, flags [none], proto TCP (6), length 94)
    192.168.165.77 > 10.19.1.150: tcp
14:39:08.472094 (authentic,confidential): SPI 0x0f45526b: (tos 0x0, ttl 63, id 19468, offset 0, flags [+], proto TCP (6), length 1412)
    192.168.165.77.80 > 10.19.1.150.49896: Flags [P.], seq 27474:28846, ack 1, win 1446, length 1372
14:39:08.472184 (authentic,confidential): SPI 0x0f45526b: (tos 0x0, ttl 63, id 19468, offset 1392, flags [none], proto TCP (6), length 94)
    192.168.165.77 > 10.19.1.150: tcp
14:39:08.475465 (authentic,confidential): SPI 0x0f45526b: (tos 0x0, ttl 63, id 19469, offset 0, flags [+], proto TCP (6), length 1412)
    192.168.165.77.80 > 10.19.1.150.49896: Flags [P.], seq 28920:30292, ack 1, win 1446, length 1372
14:39:08.481864 (authentic,confidential): SPI 0x0f45526b: (tos 0x0, ttl 63, id 19469, offset 1392, flags [none], proto TCP (6), length 94)
    192.168.165.77 > 10.19.1.150: tcp
14:39:08.485358 (authentic,confidential): SPI 0x0f45526b: (tos 0x0, ttl 63, id 19470, offset 0, flags [+], proto TCP (6), length 1412)
    192.168.165.77.80 > 10.19.1.150.49896: Flags [P.], seq 30366:31738, ack 1, win 1446, length 1372
14:39:08.490173 (authentic,confidential): SPI 0x0f45526b: (tos 0x0, ttl 63, id 19470, offset 1392, flags [none], proto TCP (6), length 94)
    192.168.165.77 > 10.19.1.150: tcp
14:39:08.492229 (authentic,confidential): SPI 0x0f45526b: (tos 0x0, ttl 63, id 19471, offset 0, flags [+], proto TCP (6), length 1412)
    192.168.165.77.80 > 10.19.1.150.49896: Flags [P.], seq 31812:33184, ack 1, win 1446, length 1372
14:39:08.502702 (authentic,confidential): SPI 0x0f45526b: (tos 0x0, ttl 63, id 19471, offset 1392, flags [none], proto TCP (6), length 94)
    192.168.165.77 > 10.19.1.150: tcp
14:39:08.502952 (authentic,confidential): SPI 0x0f45526b: (tos 0x0, ttl 63, id 19472, offset 0, flags [+], proto TCP (6), length 1412)
    192.168.165.77.80 > 10.19.1.150.49896: Flags [P.], seq 33258:34630, ack 1, win 1446, length 1372
14:39:08.503059 (authentic,confidential): SPI 0x0f45526b: (tos 0x0, ttl 63, id 19472, offset 1392, flags [none], proto TCP (6), length 94)
    192.168.165.77 > 10.19.1.150: tcp
14:39:08.503318 (authentic,confidential): SPI 0x0f45526b: (tos 0x0, ttl 63, id 19473, offset 0, flags [+], proto TCP (6), length 1412)
    192.168.165.77.80 > 10.19.1.150.49896: Flags [P.], seq 34704:36076, ack 1, win 1446, length 1372
14:39:08.503406 (authentic,confidential): SPI 0x0f45526b: (tos 0x0, ttl 63, id 19473, offset 1392, flags [none], proto TCP (6), length 94)
    192.168.165.77 > 10.19.1.150: tcp
14:39:08.503667 (authentic,confidential): SPI 0x0f45526b: (tos 0x0, ttl 63, id 19474, offset 0, flags [+], proto TCP (6), length 1412)
    192.168.165.77.80 > 10.19.1.150.49896: Flags [P.], seq 36150:37522, ack 1, win 1446, length 1372
14:39:08.503752 (authentic,confidential): SPI 0x0f45526b: (tos 0x0, ttl 63, id 19474, offset 1392, flags [none], proto TCP (6), length 94)
    192.168.165.77 > 10.19.1.150: tcp
14:39:08.507621 (authentic,confidential): SPI 0x0f45526b: (tos 0x0, ttl 63, id 19475, offset 0, flags [+], proto TCP (6), length 1412)
    192.168.165.77.80 > 10.19.1.150.49896: Flags [P.], seq 37596:38968, ack 1, win 1446, length 1372
14:39:08.515412 (authentic,confidential): SPI 0x0f45526b: (tos 0x0, ttl 63, id 19475, offset 1392, flags [none], proto TCP (6), length 94)
    192.168.165.77 > 10.19.1.150: tcp
14:39:08.515670 (authentic,confidential): SPI 0x0f45526b: (tos 0x0, ttl 63, id 19476, offset 0, flags [+], proto TCP (6), length 1412)
    192.168.165.77.80 > 10.19.1.150.49896: Flags [P.], seq 39042:40414, ack 1, win 1446, length 1372
14:39:08.515755 (authentic,confidential): SPI 0x0f45526b: (tos 0x0, ttl 63, id 19476, offset 1392, flags [none], proto TCP (6), length 94)
    192.168.165.77 > 10.19.1.150: tcp
14:39:08.517566 (authentic,confidential): SPI 0x0f45526b: (tos 0x0, ttl 63, id 19477, offset 0, flags [+], proto TCP (6), length 1412)
    192.168.165.77.80 > 10.19.1.150.49896: Flags [P.], seq 40488:41860, ack 1, win 1446, length 1372
14:39:08.522265 (authentic,confidential): SPI 0x0f45526b: (tos 0x0, ttl 63, id 19477, offset 1392, flags [none], proto TCP (6), length 94)
    192.168.165.77 > 10.19.1.150: tcp
14:39:08.524306 (authentic,confidential): SPI 0x0f45526b: (tos 0x0, ttl 63, id 19478, offset 0, flags [+], proto TCP (6), length 1412)
    192.168.165.77.80 > 10.19.1.150.49896: Flags [P.], seq 41934:43306, ack 1, win 1446, length 1372
14:39:08.531634 (authentic,confidential): SPI 0x0f45526b: (tos 0x0, ttl 63, id 19478, offset 1392, flags [none], proto TCP (6), length 94)
    192.168.165.77 > 10.19.1.150: tcp
14:39:08.537688 (authentic,confidential): SPI 0x0f45526b: (tos 0x0, ttl 63, id 19479, offset 0, flags [+], proto TCP (6), length 1412)
    192.168.165.77.80 > 10.19.1.150.49896: Flags [P.], seq 43380:44752, ack 1, win 1446, length 1372
14:39:08.547253 (authentic,confidential): SPI 0x0f45526b: (tos 0x0, ttl 63, id 19479, offset 1392, flags [none], proto TCP (6), length 94)
    192.168.165.77 > 10.19.1.150: tcp
14:39:08.547506 (authentic,confidential): SPI 0x0f45526b: (tos 0x0, ttl 63, id 19480, offset 0, flags [+], proto TCP (6), length 1412)
    192.168.165.77.80 > 10.19.1.150.49896: Flags [P.], seq 44826:46198, ack 1, win 1446, length 1372
14:39:08.547599 (authentic,confidential): SPI 0x0f45526b: (tos 0x0, ttl 63, id 19480, offset 1392, flags [none], proto TCP (6), length 94)
    192.168.165.77 > 10.19.1.150: tcp
14:39:08.547855 (authentic,confidential): SPI 0x0f45526b: (tos 0x0, ttl 63, id 19481, offset 0, flags [+], proto TCP (6), length 1412)
    192.168.165.77.80 > 10.19.1.150.49896: Flags [P.], seq 46272:47644, ack 1, win 1446, length 1372
14:39:08.547943 (authentic,confidential): SPI 0x0f45526b: (tos 0x0, ttl 63, id 19481, offset 1392, flags [none], proto TCP (6), length 94)
    192.168.165.77 > 10.19.1.150: tcp
14:39:08.548220 (authentic,confidential): SPI 0x0f45526b: (tos 0x0, ttl 63, id 19482, offset 0, flags [+], proto TCP (6), length 1412)
    192.168.165.77.80 > 10.19.1.150.49896: Flags [P.], seq 47718:49090, ack 1, win 1446, length 1372
14:39:08.548307 (authentic,confidential): SPI 0x0f45526b: (tos 0x0, ttl 63, id 19482, offset 1392, flags [none], proto TCP (6), length 94)
    192.168.165.77 > 10.19.1.150: tcp
14:39:08.549670 (authentic,confidential): SPI 0x0f45526b: (tos 0x0, ttl 63, id 19483, offset 0, flags [+], proto TCP (6), length 1412)
    192.168.165.77.80 > 10.19.1.150.49896: Flags [P.], seq 49164:50536, ack 1, win 1446, length 1372
14:39:08.556243 (authentic,confidential): SPI 0x0f45526b: (tos 0x0, ttl 63, id 19483, offset 1392, flags [none], proto TCP (6), length 94)
    192.168.165.77 > 10.19.1.150: tcp
14:39:08.556497 (authentic,confidential): SPI 0x0f45526b: (tos 0x0, ttl 63, id 19484, offset 0, flags [+], proto TCP (6), length 1412)
    192.168.165.77.80 > 10.19.1.150.49896: Flags [P.], seq 50610:51982, ack 1, win 1446, length 1372
14:39:08.556587 (authentic,confidential): SPI 0x0f45526b: (tos 0x0, ttl 63, id 19484, offset 1392, flags [none], proto TCP (6), length 94)
    192.168.165.77 > 10.19.1.150: tcp
14:39:08.556974 (authentic,confidential): SPI 0x0f45526b: (tos 0x0, ttl 63, id 19485, offset 0, flags [+], proto TCP (6), length 1412)
    192.168.165.77.80 > 10.19.1.150.49896: Flags [P.], seq 52056:53428, ack 1, win 1446, length 1372
14:39:08.557082 (authentic,confidential): SPI 0x0f45526b: (tos 0x0, ttl 63, id 19485, offset 1392, flags [none], proto TCP (6), length 94)
    192.168.165.77 > 10.19.1.150: tcp
14:39:08.562195 (authentic,confidential): SPI 0x0f45526b: (tos 0x0, ttl 63, id 19486, offset 0, flags [+], proto TCP (6), length 1412)
    192.168.165.77.80 > 10.19.1.150.49896: Flags [P.], seq 53502:54874, ack 1, win 1446, length 1372
14:39:08.566988 (authentic,confidential): SPI 0x0f45526b: (tos 0x0, ttl 63, id 19486, offset 1392, flags [none], proto TCP (6), length 94)
    192.168.165.77 > 10.19.1.150: tcp
14:39:08.575062 (authentic,confidential): SPI 0x0f45526b: (tos 0x0, ttl 63, id 19487, offset 0, flags [+], proto TCP (6), length 1412)
    192.168.165.77.80 > 10.19.1.150.49896: Flags [P.], seq 54948:56320, ack 1, win 1446, length 1372
14:39:08.575135 (authentic,confidential): SPI 0x0f45526b: (tos 0x0, ttl 63, id 19487, offset 1392, flags [none], proto TCP (6), length 94)
    192.168.165.77 > 10.19.1.150: tcp
14:39:08.576140 (authentic,confidential): SPI 0x0f45526b: (tos 0x0, ttl 63, id 19488, offset 0, flags [none], proto TCP (6), length 1179)
    192.168.165.77.80 > 10.19.1.150.49896: Flags [P.], cksum 0xe85a (correct), seq 56394:57533, ack 1, win 1446, length 1139
14:39:08.576547 (authentic,confidential): SPI 0x3bd016a4: (tos 0x0, ttl 126, id 6612, offset 0, flags [DF], proto TCP (6), length 40, bad cksum 705d (->715d)!)
    10.19.1.150.49896 > 192.168.165.77.80: Flags [.], cksum 0x9d90 (correct), seq 1, ack 0, win 65070, length 0
14:39:08.753576 (authentic,confidential): SPI 0x3bd016a4: (tos 0x0, ttl 126, id 6615, offset 0, flags [DF], proto TCP (6), length 40, bad cksum 705a (->715a)!)
    10.19.1.150.49896 > 192.168.165.77.80: Flags [R.], cksum 0x9bbb (correct), seq 1, ack 0, win 0, length 0
14:39:08.753829 (authentic,confidential): SPI 0x3bd016a4: (tos 0x0, ttl 126, id 6616, offset 0, flags [DF], proto TCP (6), length 40, bad cksum 7059 (->7159)!)
    10.19.1.150.49897 > 192.168.165.77.80: Flags [R.], cksum 0x67ce (correct), seq 3295702231, ack 1106385, win 0, length 0
14:39:08.754041 (authentic,confidential): SPI 0x3bd016a4: (tos 0x0, ttl 126, id 6617, offset 0, flags [DF], proto TCP (6), length 40, bad cksum 7058 (->7158)!)
    10.19.1.150.49898 > 192.168.165.77.80: Flags [R.], cksum 0xf8a7 (correct), seq 2938165598, ack 719301, win 0, length 0
14:39:08.757381 (authentic,confidential): SPI 0x3bd016a4: (tos 0x0, ttl 126, id 6618, offset 0, flags [DF], proto TCP (6), length 48, bad cksum 704f (->714f)!)
    10.19.1.150.49901 > 192.168.165.77.80: Flags [s], cksum 0xc5f6 (correct), seq 1740833165, win 8192, options [mss 1200,nop,nop,sackOK], length 0
14:39:08.793422 (authentic,confidential): SPI 0x0f45526b: (tos 0x0, ttl 63, id 19489, offset 0, flags [+], proto TCP (6), length 1412)
    192.168.165.77.80 > 10.19.1.150.49898: Flags [P.], seq 1:1373, ack 0, win 1446, length 1372
14:39:08.797895 (authentic,confidential): SPI 0x0f45526b: (tos 0x0, ttl 63, id 19489, offset 1392, flags [none], proto TCP (6), length 94)
    192.168.165.77 > 10.19.1.150: tcp
14:39:08.798073 (authentic,confidential): SPI 0x0f45526b: (tos 0x0, ttl 63, id 19490, offset 0, flags [none], proto TCP (6), length 44)
    192.168.165.77.80 > 10.19.1.150.49901: Flags [S.], cksum 0xd2c5 (correct), seq 2826592, ack 1740833166, win 1446, options [mss 1446], length 0
14:39:08.798411 (authentic,confidential): SPI 0x3bd016a4: (tos 0x0, ttl 126, id 6620, offset 0, flags [DF], proto TCP (6), length 40, bad cksum 7055 (->7155)!)
    10.19.1.150.49901 > 192.168.165.77.80: Flags [.], cksum 0xf1eb (correct), seq 1, ack 1, win 65070, length 0
14:39:08.798794 (authentic,confidential): SPI 0x3bd016a4: (tos 0x0, ttl 126, id 6621, offset 0, flags [DF], proto TCP (6), length 644, bad cksum 6df8 (->6ef8)!)
    10.19.1.150.49901 > 192.168.165.77.80: Flags [P.], cksum 0x5a60 (correct), seq 1:605, ack 1, win 65070, length 604
14:39:08.831836 (authentic,confidential): SPI 0x0f45526b: (tos 0x0, ttl 63, id 19491, offset 0, flags [none], proto TCP (6), length 40)
    192.168.165.77.80 > 10.19.1.150.49901: Flags [.], cksum 0xe818 (correct), seq 1, ack 605, win 1446, length 0
14:39:08.835273 (authentic,confidential): SPI 0x0f45526b: (tos 0x0, ttl 63, id 19492, offset 0, flags [none], proto TCP (6), length 770)
    192.168.165.77.80 > 10.19.1.150.49901: Flags [P.], cksum 0x072c (correct), seq 1:731, ack 605, win 1446, length 730
14:39:08.837857 (authentic,confidential): SPI 0x3bd016a4: (tos 0x0, ttl 126, id 6622, offset 0, flags [DF], proto TCP (6), length 926, bad cksum 6cdd (->6ddd)!)
    10.19.1.150.49901 > 192.168.165.77.80: Flags [P.], cksum 0x2d71 (correct), seq 605:1491, ack 731, win 64340, length 886
14:39:08.838924 (authentic,confidential): SPI 0x3bd016a4: (tos 0x0, ttl 126, id 6623, offset 0, flags [DF], proto TCP (6), length 48, bad cksum 704a (->714a)!)
    10.19.1.150.49902 > 192.168.165.77.80: Flags [s], cksum 0xc025 (correct), seq 1289560643, win 8192, options [mss 1200,nop,nop,sackOK], length 0
14:39:08.839899 (authentic,confidential): SPI 0x3bd016a4: (tos 0x0, ttl 126, id 6624, offset 0, flags [DF], proto TCP (6), length 48, bad cksum 7049 (->7149)!)
    10.19.1.150.49903 > 192.168.165.77.80: Flags [s], cksum 0x81c1 (correct), seq 3762146629, win 8192, options [mss 1200,nop,nop,sackOK], length 0
14:39:08.868181 (authentic,confidential): SPI 0x0f45526b: (tos 0x0, ttl 63, id 19493, offset 0, flags [none], proto TCP (6), length 40)
    192.168.165.77.80 > 10.19.1.150.49901: Flags [.], cksum 0xe1c8 (correct), seq 731, ack 1491, win 1446, length 0
14:39:08.872257 (authentic,confidential): SPI 0x0f45526b: (tos 0x0, ttl 63, id 19494, offset 0, flags [none], proto TCP (6), length 821)
    192.168.165.77.80 > 10.19.1.150.49901: Flags [P.], cksum 0x3e12 (correct), seq 731:1512, ack 1491, win 1446, length 781
14:39:08.877155 (authentic,confidential): SPI 0x0f45526b: (tos 0x0, ttl 63, id 19495, offset 0, flags [none], proto TCP (6), length 44)
    192.168.165.77.80 > 10.19.1.150.49902: Flags [S.], cksum 0x9167 (correct), seq 4611282, ack 1289560644, win 1446, options [mss 1446], length 0
14:39:08.877452 (authentic,confidential): SPI 0x3bd016a4: (tos 0x0, ttl 126, id 6625, offset 0, flags [DF], proto TCP (6), length 40, bad cksum 7050 (->7150)!)
    10.19.1.150.49902 > 192.168.165.77.80: Flags [.], cksum 0xb08d (correct), seq 1, ack 1, win 65070, length 0
14:39:08.877946 (authentic,confidential): SPI 0x3bd016a4: (tos 0x0, ttl 126, id 6626, offset 0, flags [DF], proto TCP (6), length 924, bad cksum 6cdb (->6ddb)!)
    10.19.1.150.49902 > 192.168.165.77.80: Flags [P.], cksum 0xf752 (correct), seq 1:885, ack 1, win 65070, length 884
14:39:08.878398 (authentic,confidential): SPI 0x0f45526b: (tos 0x0, ttl 63, id 19496, offset 0, flags [none], proto TCP (6), length 44)
    192.168.165.77.80 > 10.19.1.150.49903: Flags [S.], cksum 0x4dc8 (correct), seq 4219411, ack 3762146630, win 1446, options [mss 1446], length 0

In firewall log as blocked:
[code]  Dec 8 14:40:03 enc0   192.168.165.77    10.19.1.150  TCP: 
  Dec 8 14:40:03 enc0   192.168.165.77:80    10.19.1.150:49902  TCP:PA 
  Dec 8 14:40:03 enc0   192.168.165.77    10.19.1.150  TCP: 
  Dec 8 14:40:03 enc0   192.168.165.77:80    10.19.1.150:49902  TCP:PA 
  Dec 8 14:40:00 enc0   192.168.165.77    10.19.1.150  TCP: 
  Dec 8 14:40:00 enc0   192.168.165.77:80    10.19.1.150:49902  TCP:PA 
  Dec 8 14:40:00 enc0   192.168.165.77    10.19.1.150  TCP: 
  Dec 8 14:40:00 enc0   192.168.165.77:80    10.19.1.150:49902  TCP:PA 
  Dec 8 14:39:57 enc0   192.168.165.77    10.19.1.150  TCP: 
  Dec 8 14:39:57 enc0   192.168.165.77:80    10.19.1.150:49902  TCP:PA [/code] [/s][/s][/s]

ggzengel · Dec 8, 2010, 6:36 PM

Now i got a little bit further.
The areca controller never answer to fragmented pings.

It seems the pfsense discards fragmented packets with psh set:

1249669.492 X DATA[1414]
      0000: 00 00 45 00 05 84 0b d1  20 00 40 06 bd be 0a 13  ..E..... .@.....
      0010: 76 29 0a 13 01 96 00 50  c6 29 00 1a 50 ef 41 9e  v).....P.)..P.A.
      0020: 21 f0 50 18 05 a6                                 !.P...
             IP-Packet from 10.19.118.41 to 10.19.1.150  protocol TCP
             Fragment:  ID 3025  bytes 0 ... 1391
             TCP-Message, sourceport 80 destinationport 50729
                          sequence number 1724655
                          acknowledgement number 1100882416
                          offset 5 flags ACK PSH
                          window 1446 checksum 0x2809 urgent 0

1249669.500 X DATA[0096]
      0000: 00 00 45 00 00 5e 0b d1  00 ae 40 06 e2 36 0a 13  ..E..^....@..6..
      0010: 76 29 0a 13 01 96 69 64  74 68 3d 22 39 38 25 22  v)....idth="98%"
      0020: 3e 0d 0a 3c 74 72                                 >.. <tr<br>IP-Packet from 10.19.118.41 to 10.19.1.150  protocol TCP
             Fragment:  ID 3025  bytes 1392 ... 1465</tr<br>

firewall log:

  Dec 8 18:23:53 enc0   10.19.118.40:80    10.19.1.150:50729  TCP:PA

eri-- · Dec 8, 2010, 6:49 PM

You have to allow fragments in the ipsec rule otherwise pf will drop them.

ggzengel · Dec 8, 2010, 7:18 PM

But fragmented ICMP works?
And how should i allow fragmented packets?

1252982.843 R DATA[1630]
      0000: 01 00 45 00 06 5c 48 5e  00 00 7d 01 65 23 0a 13  ..E..\H^..}.e#..
      0010: 01 96 0a 13 74 64 08 00  08 7c 00 66 3a 6e 61 62  ....td...|.f:nab
      0020: 63 64 65 66 67 68                                 cdefgh
             IP-Packet from 10.19.1.150 to 10.19.116.100  protocol ICMP
             ICMP-Message , type echo request

1252982.851 X DATA[1414]
      0000: 00 00 45 00 05 84 26 06  20 00 40 01 a5 53 0a 13  ..E...&. .@..S..
      0010: 74 64 0a 13 01 96 00 00  10 7c 00 66 3a 6e 61 62  td.......|.f:nab
      0020: 63 64 65 66 67 68                                 cdefgh
             IP-Packet from 10.19.116.100 to 10.19.1.150  protocol ICMP
             Fragment:  ID 9734  bytes 0 ... 1391
             ICMP-Message , type echo reply

1252982.851 X DATA[0238]
      0000: 00 00 45 00 00 ec 26 06  00 ae 40 01 c9 3d 0a 13  ..E...&...@..=..
      0010: 74 64 0a 13 01 96 65 66  67 68 69 6a 6b 6c 6d 6e  td....efghijklmn
      0020: 6f 70 71 72 73 74                                 opqrst
             IP-Packet from 10.19.116.100 to 10.19.1.150  protocol ICMP
             Fragment:  ID 9734  bytes 1392 ... 1607

19:14:29.530989 (authentic,confidential): SPI 0x10907845: (tos 0x0, ttl 126, id 20210, offset 0, flags [none], proto ICMP (1), length 1628, bad cksum 5d0f (->5d8f)!)
    10.19.1.150 > 10.19.116.100: ICMP echo request, id 102, seq 16025, length 1608
19:14:29.599466 (authentic,confidential): SPI 0x07d35b41: (tos 0x0, ttl 63, id 3056, offset 0, flags [+], proto ICMP (1), length 1412)
    10.19.116.100 > 10.19.1.150: ICMP echo reply, id 102, seq 16025, length 1392
19:14:29.599611 (authentic,confidential): SPI 0x07d35b41: (tos 0x0, ttl 63, id 3056, offset 1392, flags [none], proto ICMP (1), length 236)
    10.19.116.100 > 10.19.1.150: icmp

cmb · Dec 9, 2010, 1:53 PM

The MSS clamping is doing exactly what you have it configured to do:

 10.19.1.150.49902 > 192.168.165.77.80: Flags [s], cksum 0xc025 (correct), seq 1289560643, win 8192, options [mss 1200,nop,nop,sackOK]

There isn't any ability to allow/deny fragments on a per-rule basis, not sure what Ermal is referring to. [/s]

valnar · Dec 9, 2010, 10:48 AM

I've had a similar problem in pfSense 1.23 where other VPN devices (Sonicwalls) worked fine. Through a site VPN to my work, I cannot get to certain internal web pages. I tried all manners of MSS, MTU and DF bit changes on pfSense to no avail. All other firewalls I tried worked fine. In the end, I had to just lower the MTU on my Windows machines in my home to make it work.

'Not sure where the fix could be, but like I said, all other VPN enabled firewalls I've tried (Sonicwall, Cisco, Netscreen) worked fine.

cmb · Dec 9, 2010, 1:54 PM

@valnar:

I've had a similar problem in pfSense 1.23 where other VPN devices (Sonicwalls) worked fine. Through a site VPN to my work, I cannot get to certain internal web pages. I tried all manners of MSS, MTU and DF bit changes on pfSense to no avail. All other firewalls I tried worked fine. In the end, I had to just lower the MTU on my Windows machines in my home to make it work.

That's why we added MSS clamping for VPNs (which works fine).

ggzengel · Dec 9, 2010, 6:26 PM

Why blocks the pfsense fragmented psh packets?
Or is there an other reason? Small packets with psh will pass.

1249669.492 X DATA[1414]
      0000: 00 00 45 00 05 84 0b d1  20 00 40 06 bd be 0a 13  ..E..... .@.....
      0010: 76 29 0a 13 01 96 00 50  c6 29 00 1a 50 ef 41 9e  v).....P.)..P.A.
      0020: 21 f0 50 18 05 a6                                 !.P...
             IP-Packet from 10.19.118.41 to 10.19.1.150  protocol TCP
             Fragment:  ID 3025  bytes 0 ... 1391
             TCP-Message, sourceport 80 destinationport 50729
                          sequence number 1724655
                          acknowledgement number 1100882416
                          offset 5 flags ACK PSH
                          window 1446 checksum 0x2809 urgent 0

1249669.500 X DATA[0096]
      0000: 00 00 45 00 00 5e 0b d1  00 ae 40 06 e2 36 0a 13  ..E..^....@..6..
      0010: 76 29 0a 13 01 96 69 64  74 68 3d 22 39 38 25 22  v)....idth="98%"
      0020: 3e 0d 0a 3c 74 72                                 >..<tr<br>             IP-Packet from 10.19.118.41 to 10.19.1.150  protocol TCP
             Fragment:  ID 3025  bytes 1392 ... 1465</tr<br>