Posting to a forum issue



  • Hi,

    Having a strange issue posting to a forum.

    Network looks like this:
    Laptop <CAT5E > pfS <USBcable>Phone/4G
    That works fine, no problems.

    On Laptop I browse to a forum, posting there, all works fine.
    When I start OpenVPN on the Laptop to a server on the net, tunneling through pfS, I can still browse that forum but trying to post a message just hangs showing "connecting to forum.url" in the browser until it times out.

    When I take pfS out of the equation all works fine, with or without OpenVPN.

    I tried en/disabling the hardware offloading, set it to conservative instead of normal, prefer IPv4, various combinations, looking at logs finding no clue.
    There are no packages active.

    Someone have an idea of what is going on?

    Thanks.


  • Netgate Administrator

    I have seen similar odd behaviour like that when there were MTU issues on the connection. I think it was IPSec in that case but I could see it happening here.
    Try setting in the custom options field mssfix 1300 on the OpenVPN client.

    Steve



  • Hi,

    mssfix 1410 does the job, 1411 fails.

    Then my question would become, why?
    Because this is the first time I encounter this and would like to understand why when pfS is in the middle I need to mssfix it?

    Thanks.


  • Netgate Administrator

    It's failing to detect a restriction somewhere between you and that remote host for some reason.

    It was just that one host that was not working I assume?

    https://en.wikipedia.org/wiki/Maximum_segment_size

    Steve



  • Hi,

    Please define "It" in "It's failing".
    I visit about ten forums and just one showed this behavior.
    Never encountered this before, this Laptop has seen about 190 different networks, mix of wired and WiFi.

    En/disabling scrub did not help either.

    On to reading, thanks.


  • Netgate Administrator

    I would guess it's the TCP end points not correctly detecting the maximum segment size. Since you only see it with that site it could be a problem at that end. I'd have to go and refresh my memory on this stuff too though. 😉

    Steve



  • Not had the opportunity to make captures but will do.

    Although about IPv6, at the time I can only wonder if it's anyhow related to this (and other posts popping up):
    https://forum.netgate.com/topic/138406/still-weird-issues-with-fragmented-ipv6-dns-packets-2-4-4-p1
    The thing is, when pfS is removed from the path, it all works...

    Thanks.



  • Attached a capture on WAN interface.
    The gateway for pfS is 192.168.42.129, that is Android USB tether interface.
    pfS is 192.168.42.137

    Destination unreachable, fragmentation needed, MTU of next hop 1410 comes back from .129
    0_1544121910529_wanwithvpn.pcap

    Next capture is on enp1s0 (192.168.51.2) of Laptop.
    Same story, ICMP message arrives.
    0_1544121931562_enp1s0vpn.pcapng

    Next capture is on tun0 (10.8.8.80) of Laptop.
    The only thing w.r.t. destinations that show up are the DNS root servers.
    0_1544121949103_tun0withvpn.pcapng

    Laptop routes all traffic through VPN.
    Help in understanding what is going on is appreciated.

    Thanks.

    Edit:
    Uploading files did not work, disabled NoScript and now works.
    However, downloading capture from pfS results in a *.cap file which is not accepted by forum.
    Renaming wanwithvpn.cap to wanwithvpn.pcap worked.


  • Netgate Administrator

    When you remove pfSense you;re running the OpenVPN client directly on the laptop? To the same endpoint?

    We did see an issue with traffic coming across OpenVPN failing to pass packet fragments a while back. However that was only if the interface was unassigned or the rules passing the traffic were not on the assigned interface. Yours must be assigned here.

    Steve



  • The laptop is running the OpenVPN client and connecting to a remote server.
    pfS is not involved in running OpenVPN, just sits in between and when removed it works.
    Next time I will do capture on OPT1 too, Laptop is connected there.

    Thanks.


  • Netgate Administrator

    Ah, OK I misunderstood. So pfSense is not handling the VPN termination in either case?

    In that case it would appear to be limiting the route MTU in some way that OpenVPN does not detect?

    Are you able to post to the forum if you are not running through the VPN but are using pfSense?

    Steve



    1. Correct

    2. I'm not sure but I think OpenVPN does not know about outside MTU after connection is established. Please see attached diagram and --mssfix max (and --fragment) in manual 2.4:
      https://community.openvpn.net/openvpn/wiki/Openvpn24ManPage

    3. Yes

    Thanks.

    0_1544392735176_ovpn-flow09.png


  • Netgate Administrator

    It does include '--mtu-disc'. FreeBSD doesn't support it but Linux does if your laptop is running that.

    Hard to say why pfSense appears to be reducing the apparent MTU size though.

    Steve



  • Hi,

    OpenVPN's --mtu related directives are "inside tunnel" options, AFAIK.

    Also, the manual is not correct, --mtu-disc is not working IIRC.

    Thanks.



  • Made a capture on OPT1, if someone is willing to take a look at it.

    Thanks.
    0_1544569894586_opt1withvpn.pcap



  • Now reading here:
    https://github.com/OpenVPN/openvpn/blob/master/INSTALL

    If run through a firewall using OpenBSDs packet filter PF and the
    filter rules include a "scrub" directive, you may get problems talking
    to Linux hosts over the tunnel, since the scrubbing will kill packets
    sent from Linux hosts if they are fragmented. This is usually seen as
    tunnels where small packets and pings get through but large packets
    and "regular traffic" don't. To circumvent this, add "no-df" to
    the scrub directive so that the packet filter will let fragments with
    the "dont fragment"-flag set through anyway.>

    Could this somehow be tested?

    Thanks.


  • Netgate Administrator

    Hmm, Linux sends fragmented packets with df set? Seems.... interesting.

    You can disable pfscrub entirely to test that. You could probably make a custom rules file and load it temporarily with no-df set.

    I haven't see this for a long time but we have previously seen OpenVPN tunnels where fragmented packets would make it across the tunnel but never be sent out of the local interface at the other end. That only happened on OpenVPN tunnels where the interface was not assigned. Assigning it worked around it, somehow altering how pf handles the packets. I could see that being this issue.

    Steve



  • @stephenw10 said in Posting to a forum issue:

    Hmm, Linux sends fragmented packets with df set? Seems.... interesting.

    The DF flag tells routers not to fragment. The originating device is still free to fragment.


  • Netgate Administrator

    Mmm, I guess so. 🤔



  • @stephenw10 said in Posting to a forum issue:

    Mmm, I guess so. 🤔

    If the source wants to send a packtet that's too big to fit within the MTU, it might split it into more than one piece. This is typically done with UDP, but not TCP. These days, the world is moving to path MTU detection. It's mandatory on IPv6 and now used on IPv4. On Linux, with IPv4, the DF flag is set on everything, but in Windows TCP only. As for TCP, don't forget that every packet is a fragment, unless the entire data can be carried in a single packet.

    The reason for moving away from fragmentation is to reduce the performance hit at routers, along with reassembling at the destination.



  • So, ticking
    "IP Do-Not-Fragment compatibility"
    makes the issue go away.
    I'm not sure if that is the right solution or somehow tell Linux not to set DF bit on fragmented packets...

    Or is my conclusion wrong?

    Thanks.



  • @Pippin

    I don't know if that is the proper fix. My thought would be to find out what's causing this. What packets are being fragmented? If that setting only affects fragmented packets that have DF set, then I suppose it wouldn't be a problem. Still, I'd want to know why it's needed. As I mentioned, DF is used these days, for everything on Linux and TCP on Windows.



  • https://redmine.pfsense.org/issues/10189
    reminded me of this topic, wondering if it relates.


Log in to reply