Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    Upgrade to 2.4.5 broke 802.1x RADIUS WiFi over VPN

    Scheduled Pinned Locked Moved General pfSense Questions
    39 Posts 6 Posters 5.5k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • stephenw10S
      stephenw10 Netgate Administrator
      last edited by

      Really we would need to reproduce it here. Can you say how and where it was failing before assigning the intrerface?

      Like packet fragments were arriving on the OpenVPN interface pcap but never leaving on the internal pcap? That's what I expect to see if anything.

      Steve

      DAVe3283D 1 Reply Last reply Reply Quote 0
      • DAVe3283D
        DAVe3283 @stephenw10
        last edited by DAVe3283

        @stephenw10 Sure. I dug in, and found this appears to be several things coming together to cause this failure.

        In all cases, packets are fragmented at 1504 bytes inside the OpenVPN tunnel.

        The twist: On my local (main) LAN, I have jumbo frames turned on (MTU of 9000 bytes). But-- not the RADIUS server; I forgot, so it had the default MTU of 1558 bytes 😯 This... was not helping anything.

        With 2.4.5 & no OpenVPN interface, the LAN interface appears to reassemble some packets as large as 1763 bytes on the LAN side. But some remain fragmented at 1566 bytes. 🤷

        So the packets were passing through the LAN interface (yay), but were being dropped by the RADIUS server because its NIC didn't have jumbo frames enabled (boo). I bet if jumbo frames were enabled, it would have worked fine.

        I would expect that packets should either consistently be re-assembled or consistently be left fragmented as they pass from OpenVPN loopback to the LAN interface. So there probably is some undesired behavior on pfSense 2.4.5 in this config. And this is probably where the difference between 2.4.4-p3 & 2.4.5 lies.

        After assigning OpenVPN an interface on 2.4.5, fragments do not appear to be reassembled at all. This avoided the jumbo frame mis-match on the LAN, and everything works.

        Let me know if you need any more information.

        Edit: weirder and weirder. pfSense did not have anything entered for the MTU on the LAN interface (or any interface), so should have been using the default MTU of 1500. In fact, it drops anything larger than 1500 coming in. So why in the world was it assembling packets with a MTU well in excess of that? I would say that is a bug as well.

        1 Reply Last reply Reply Quote 0
        • stephenw10S
          stephenw10 Netgate Administrator
          last edited by

          Hmm, that is weird. What was it doing in 2.4.4p3? Not reassembling the packets even without the interface assigned?

          Let me see if I can find anything that might explain that in 2.4.5.

          Steve

          DAVe3283D 1 Reply Last reply Reply Quote 0
          • DAVe3283D
            DAVe3283 @stephenw10
            last edited by

            @stephenw10 Exactly. 2.4.4-p3 was not reassembling packets even without the interface assigned.

            I still find it weird that 2.4.5 reassembles some packets, seemingly at random, without the interface assigned. And assembling packets that exceed the set MTU of the adapter, at that.

            1 Reply Last reply Reply Quote 0
            • B
              BurnettC27
              last edited by BurnettC27

              Hello,

              I've been dealing with this issue for the past couple of days and stumbled upon this forum post. The two remote sites that I have updated to 2.4.5 no longer work with our RADIUS WPA2/AES SSID.

              Has this been identified as a bug? Is there a fix in the works?

              If we create an Interface for OpenVPN and assign it to the client, then our gateway group for failover will no longer be active.

              Please assist. :)

              Thanks,
              Chris

              1 Reply Last reply Reply Quote 0
              • stephenw10S
                stephenw10 Netgate Administrator
                last edited by

                The gateway group that the client is using? Or having the OpenVPN gateway as part of the group?

                1 Reply Last reply Reply Quote 0
                • B
                  BurnettC27
                  last edited by

                  Yes, the gateway group that the OpenVPN client is using. It is currently set as a TRI WAN Failover.

                  If we create a new OpenVPN interface, do we have to associate it in any way with the OpenVPN client?

                  Maybe I'm misunderstanding the solution stated above..

                  What is the purpose of creating an interface for OpenVPN? What changed in version 2.4.5?

                  Thanks for your help!

                  1 Reply Last reply Reply Quote 0
                  • stephenw10S
                    stephenw10 Netgate Administrator
                    last edited by

                    No the failover group should work exactly the same. You will have to restart the OpenVPN client once it's been assigned as an interface.

                    The purpose of assigning it as an interface if that it is then treated differently by pf. You should make sure any pass rules are on the assigned interface only rather then the general openvpn tab so states are created there.

                    Specifically pfscrub, which reassembles fragmented packets, seems to be applied differently. We have seen it fix exactly this sort situation.

                    I don't know what might have changed in 2.4.5 to change the behaviour though.

                    Steve

                    1 Reply Last reply Reply Quote 0
                    • W
                      wdup
                      last edited by

                      Hello,

                      I'm so glad to have found this thread!! I have been investigating the same issue for a few days and for some reason did not find this thread until now!!

                      I have a very similar environment:

                      • Main site connecting to multiple sites using OpenVPN site-to-site (Peer to Peer (SSL/TLS) || UDP || TUN) tunnels
                      • Main site hosting Windows NPS (RADIUS) server
                      • Branch sites with UniFi UAP access points, authenticating domain joined Windows clients against RADIUS server at main site, using WPA2-Enterprise (computer authentication)

                      As in the initial post, it is possible to connect to servers/PCs at main site and to servers/PCs between sites and firewalls are set to allow all traffic over OpenVPN tunnels.

                      First experienced a problem after upgrading sites from pfSense 2.4.4-p3 to 2.4.5 :: after the upgrades wireless clients were unable to authenticate against RADIUS server over S2S OpenVPN tunnels.

                      At the time I was able to get clients to authenticate by changing Framed-MTU on Windows NPS server as per this article:

                      https://support.microsoft.com/en-us/help/883389/how-to-reduce-the-eap-packet-size-by-using-the-framed-mtu-attribute-in

                      I used the suggested Framed-MTU = 1344

                      pfSense on the main site is hosted on Hyper-V and was not upgraded from 2.4.4-p3 due to performance problems reported for pfSense 2.4.5 running on hypervisors.

                      With performance issues resolved with 2.4.5-p1 release, I upgraded main site (and all branch sites) to 2.4.5-p1, and now wireless clients are not able to authenticate.

                      I should add, I had OpenVPN custom options in place:

                      OpenVPN custom options used on pfSense 2.4.4:

                      fragment 1400;
                      mssfix;
                      

                      OpenVPN custom options on pfSense 2.4.5:

                      tun-mtu 1500;
                      tun-mtu-extra 32;
                      mssfix 1450;
                      

                      I have tried several things, including different custom options for OpenVPN for S2S server/clients and lower values for Framed-MTU on Windows NPS server, but have not been able to get wireless clients to authenticate. Clients on the same site and the RADIUS server continue to work as expected and the only change to the environment was the pfSense upgrades to 2.4.5-p1.

                      I will assign OpenVPN interfaces and revert back.

                      DAVe3283D 1 Reply Last reply Reply Quote 0
                      • DAVe3283D
                        DAVe3283 @wdup
                        last edited by

                        @wdup said in Upgrade to 2.4.5 broke 802.1x RADIUS WiFi over VPN:

                        ...
                        I will assign OpenVPN interfaces and revert back.

                        You can either assign OpenVPN interfaces or revert back. No need to do both.

                        Actually, from what I have seen, you only need to assign OpenVPN an interface on the main site (with the NPS), not on the remote sites. I am running pfSense 2.4.5_p1 on HyperV, so it should work the same way for you.

                        IMO Netgate should mention this in the Assigning OpenVPN Interfaces documentation, as there is no indication it is necessary for proper fragmentation handling. Also IMO it should not be necessary for proper fragmentation handling; proper handling of fragmented packets should be a baseline for the VPN to be considered working. But at least a note in the docs would be nice.

                        W 1 Reply Last reply Reply Quote 0
                        • W
                          wdup @DAVe3283
                          last edited by

                          @DAVe3283 LOL ... apologies, my previous comment was ambiguous, I meant I will revert back with feedback after assigning the OpenVPN interfaces ☺

                          I have indeed assigned the OpenVPN interfaces and I'm happy to confirm the problem is resolved. ☺ 👏

                          To confirm, I have also only assigned OpenVPN interfaces on the main site where the Windows NPS server is hosted, and clients can authenticate again.

                          However, my concern is we may be in a unique situation where we experience this "problem", but I would like to understand what has in fact changed from previous pfSense versions and what the underlying cause of the "problem" is?

                          Even though the "problem" is solved by only assigning OpenVPN interfaces at the main site, I feel it might be best to assign ALL OpenVPN interfaces at ALL sites to avoid similar "problems" going forward - what do you think?

                          I agree a note in the documentation would be great!

                          1 Reply Last reply Reply Quote 0
                          • stephenw10S
                            stephenw10 Netgate Administrator
                            last edited by

                            It's probably this you're hitting: https://redmine.pfsense.org/issues/7779

                            You could confirm it by checking the packet size and if they are fragmented in a packet capture.

                            If you are using RADIUS with UDP this is more likely to be an issue. If it's using TLS, and therefore TCP, I expect it to detect the route MTU and use packets that do not fragment. If it is not doing so you should investigate that.

                            Steve

                            W 1 Reply Last reply Reply Quote 0
                            • W
                              wdup @stephenw10
                              last edited by

                              @stephenw10 Thank you for the reply.

                              If I may ask the question differently, is there any harm in assigning ALL OpenVPN interfaces?

                              1 Reply Last reply Reply Quote 0
                              • stephenw10S
                                stephenw10 Netgate Administrator
                                last edited by

                                Not really. You will need to restart any OpenVPN servers after assigning them as an interface though.

                                Also to actually make use of it make sure traffic is passed on the assigned interface firewall rules and not the 'OpenVPN' rules.

                                Steve

                                1 Reply Last reply Reply Quote 0
                                • O
                                  ogghi
                                  last edited by

                                  Hi there, I am running 2.5.1 on 2 sites, with a site2site openvpn.
                                  I would like to get radius to work in both directions in order to have fall-back NPS for Wifi.
                                  Right now there is a rule in the openvpn interface which allows all.
                                  There is also one for opt3 which is not handling any traffic though.
                                  Would it be enough to disable the rule in openvpn if to get the traffic handled by the opt3 if?
                                  I would need to do that on both sites I guess, to have it working in both directions?

                                  1 Reply Last reply Reply Quote 0
                                  • stephenw10S
                                    stephenw10 Netgate Administrator
                                    last edited by

                                    Yes, if you disable a rule on the group OpenVPN interface traffic will hit rules on the assigned interfaces and get the required reply-to tags.

                                    Steve

                                    O 1 Reply Last reply Reply Quote 1
                                    • O
                                      ogghi @stephenw10
                                      last edited by

                                      @stephenw10 Hey there, I was brave and tested to change those settings from remote :)
                                      Nothing broke. Traffic is being handled by the interface specific rule now.
                                      But still I don't get any request on the RADIUS server on the other tunnel end. Always bad UDP checksum...

                                      stephenw10S 1 Reply Last reply Reply Quote 0
                                      • stephenw10S
                                        stephenw10 Netgate Administrator @ogghi
                                        last edited by

                                        @ogghi said in Upgrade to 2.4.5 broke 802.1x RADIUS WiFi over VPN:

                                        Always bad UDP checksum...

                                        In a packet capture?
                                        That's expected if you have checksum offloading enabled on the capture interface.

                                        You're not seeing the radius traffic arrive at the server at all?

                                        Steve

                                        O 1 Reply Last reply Reply Quote 0
                                        • O
                                          ogghi @stephenw10
                                          last edited by

                                          @stephenw10 nothing arrives on the radius server from over the vpn connection.
                                          That's the weird thing. At least nothing is logged in the windows service...

                                          1 Reply Last reply Reply Quote 0
                                          • stephenw10S
                                            stephenw10 Netgate Administrator
                                            last edited by

                                            Hmm, well I'd pcap on the server to be sure. I'd also pcap at each interface in the route to see where it's failing.
                                            We have seen issues with large UDP packets not fragmenting correctly across the tunnel. You would see that in a pcap if you are hitting that or something similar.

                                            Steve

                                            O 1 Reply Last reply Reply Quote 1
                                            • First post
                                              Last post
                                            Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.