INFO: OpenVPN and MTU

phil.davis

I had some trouble with OpenVPN links when we changed to a different ISP. We found we could ping across the OpenVPN site-to-site link with packet size up to 1291 bytes:

ping other-end-pfsense -l 1291

But 1292 or bigger failed.
We could ping the public IP of other-end-pfsense with any packet size.
It seems that somehow the process of OpenVPN encrypting and adding its headers/protocol etc to the packets makes them too big for something across the new ISP link.
To fix this we tried putting the following in the Advanced box at each end of the site-to-site OpenVPN link:

fragment 1400;mssfix

This causes OpenVPN to internally fragment packets over 1400 bytes. "mssfix" causes it to notify the sender of TCP packets about the issue, so for TCP the sender can adjust their packet size.
The OpenVPN documentation says that using "fragment" adds 4 bytes to the OpenVPN header. We did indeed see that now we could ping to a maximum of 1287 bytes (compared to 1291 previously).
Up to 1287 is fine, 1288 to 1308 bytes would not ping, 1309 and above worked. Presumably a 1309 byte raw packet translates into a 1400 byte OpenVPN encrypted and encapsulated packet, and the "fragment 1400" causes that to be fragmented and happily sent across the link. Packets from 1288 to 1308 bytes were not fragmented, and did not get across the link.
We reduced the fragment parameter to see the effect:

fragment 1390;mssfix

Up to 1287 is fine, 1288 to 1298 bytes would not ping, 1299 and above worked.

fragment 1380;mssfix

Up to 1287 is fine, 1288 bytes would not ping, 1289 and above worked. With this there is just one exact packet size that does not transit the link OK!

fragment 1379;mssfix

ping works for all packet sizes.

Lessons:

Get an ISP backbone that takes the default packet sizes; failing that;
Be aware that with a largish fragment size there can be a middle range of packet sizes that still do not work. Mess with the setting and test a lot of different packet sizes, otherwise you might have really odd problems if some particular packet sizes do not work.
Reduce fragment nnnn a bit below what you exactly need - should help when your ISP gets even worse.
Test a good range of packets sizes every now and then to see if your ISP has got worse (or better) then adjust "fragment nnnn" appropriately.

Please comment if you have other useful experience of this stuff.