Fragmentation problem when firewall is enabled

sinac · Dec 20, 2010, 4:08 PM

Hi all!

I have a site to site VPN between two pfSense with another pfsense between them on the way as a router:

SiteA-pfSense_VPN1–--INTERNET----pfSense_Router-----pfSense_VPN2-SiteB

The tunnel between pfSense_VPN1 and pfSense_VPN2 is established and traffic goes through. As long as pfSense_Router has firewall diabled everything works flawless. However, if I enable the Firewall on pfSense_Router and simply add pass_all roules in both directions, the VPN is still working, but I get packet fragmentations which breaks some stuff, especially UDP kerberos in Windows.

Can anyone tell me why and how this is, or what I can do about it? I tried setting the WAN MTUs via console etc. but nothing works. Switching Kerberos to TCP works and turning the pfSense_Router to routing only works too, but this is not what I need.

pfSense_VPN1 is an appliance, the other two are ESX VMs with e1000 NICs.

Any help would be appreciated!

Best regards!

jimp · Dec 20, 2010, 6:08 PM

Don't set the MTU on the router console, do it under Interfaces > WAN. This sets up mss clamping in pf, which will help with fragmentation over VPNs.

In 2.0 there is a checkbox in Advanced options to setup MSS clamping on VPNs without adjusting the MTU.

sinac · Dec 20, 2010, 7:29 PM

Hi, thanks for the reply!

I tried that, too. No luck though. You think the MTU is what causes the problem? I can't really wrap my mind around why disabling the firewall function on the router thats not involved in the VPN at all affects the fragmentation of vpn traffic. My idea was that the fragmentation happens due to the IPSec overhead added to the IP packages but then adjusting the MTU should have fixed it. However, it did not.

Another strange thing I noticed with the router having the firewall enabled: When I dis- and reenable the VPN on one site, the tunnel gets established again, but no traffic goes thru until I reboot the firewall on one site. When routing only is in place, this does not occur.

Any further ideas?

jimp · Dec 20, 2010, 8:07 PM

You could also try toggling the scrub option under Advanced, it could be causing some issues. pf does a lot more than just pass or block packets, it can also try to drop or clean up packets as they go through.

sinac · Dec 20, 2010, 8:59 PM

Thanks a lot, that did the trick! I disabled scrubbing on the router between the two VPN Firewalls and everything seems to be fine. From what I understand, there should be no disadvantages from disabling the packet normalization on the router as long as it stays active on the VPN endpoint Routers?

Thanks again!

jimp · Dec 20, 2010, 9:09 PM

It should be fine like that. It's just some extra added cleanup. Some use cases require it not be present, but in general it's better left on. I wouldn't worry about running with it off, though.