Problem with VPN through pfSense box
-
Hello all…
I trying to use pfSense router as a border gateway. HP Proliant DL140 with two Intel MT1000 network adapters (plus two Broadcom onboard adapters, not used). No fancy things, no NAT, just a simple router and some filtering rules. Actually, this is a third time, when I try to migrate to pfSense form old Linux box, but every time this was ended at same result. For some reason, pfSense do not pass VPN traffic containing large packets. One of my customer using some (IPSec?) tunnels, between two Juniper boxes. Packets up to 1418 bytes go fine via tunnel, but all larger packets do not reach destination. After switching back to the Linux box, everything works fine again. I even have booted up Linux (Zeroshell) at same hardware, and everything works fine. Tried to check "Disable Hardware Checksum Offloading" and "Clear DF bit instead of dropping" but no luck (does those require reboot?).
What is wrong with pfSense or most likely with BSD in general, and can it be resolved somehow?
-
What MTU are you using on the pfSense?
Have you tried to lower it? -
WAN is connected to gigabit fiber, and as I understand, if not changed then MTU is 1500 by default. I have not tried to change it. In my working Linux router it is also 1500, so pfSense should work fine with that. Or not?
This is typical fragment of tcpdump output for that VPN traffic, and biggest packet here is 1468 bytes?
13:33:40.102979 xx.xx.xx.xxx > xxx.xx.xxx.xxx: ip-proto-50 76 (ttl 60, id 60752)
13:33:40.109341 xxx.xx.xxx.xxx > xx.xx.xx.xxx: ip-proto-50 64 (frag 6840:64@0+) (ttl 57)
13:33:40.109794 xx.xx.xxx.xxx > xx.xx.xx.xxx: (frag 6840:1468@64) (ttl 57)
13:33:40.114114 xxx.xx.xxx.xxx > xx.xx.xx.xxx: ip-proto-50 372 (ttl 57, id 6851)
13:33:40.128931 xx.xx.xx.xxx > xxx.xx.xxx.xxx: ip-proto-50 84 (ttl 60, id 60755)
13:33:40.135926 xx.xx.xx.xxx > xxx.xx.xxx.xxx: ip-proto-50 76 (ttl 60, id 60756)
13:33:40.714474 xxx.xx.xxx.xxx.7800 > xx.xx.xx.xxx.1084: . ack 1232 win 32767 (DF) (ttl 56, id 51486)
13:33:40.940995 xx.xx.xx.xxx > xxx.xx.xxx.xxx: ip-proto-50 92 (ttl 60, id 60758) -
since starting this thread here, I have made many tests and got conclusion, that for some reason, pfSense is currently unable to pass large IPSec packets. Only lowering MTU on both ends of the IPSec tunnel works, but this is not solution for me. I have found, that this issue is discussed in monowall mailing list, and supposed to be fixed in their newer 1.3 betas. I have tried every pfSense release since 1.0.0 and all of them having same problem. How is this possible, that nobody has fixed it yet?
I will continue search of open source router software for 5000 computers and ~200 Mbps of traffic. Any ideas?
-
This is likely part of a well known issue, and a common problem that can occur with solutions from any vendor.
http://fengnet.com/book/CDDV/ch07lev1sec4.html
-
yes, I am aware of this, but if problem is not elsewhere, then every self respecting router or firewall should handle this. In fact, I have also tried many hardware (D-Link, Juniper, Extreme Networks, etc.) and software (OpenWRT, Coyote, Zeroshell) routers, and pfSense was only what did not pass these packets. This was very surprising to me, that such trivial problem exists in any other way - excellent router software, for a long time, and nobody care about that.