Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    22.05 - DCO and OpenVPN issue

    Scheduled Pinned Locked Moved OpenVPN
    50 Posts 7 Posters 8.9k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • S
      swixo @jimp
      last edited by

      @jimp So I tried again with MacOS - working with DCO. No erros in logs. Connected with iOS - nothing. And as soon as I connected with iOS, the MacOS connection stopped working too.

      Log errors at time of failure:

      Jun 15 10:48:35	openvpn	2705	192.168.1.212:54428 TLS Error: tls-crypt unwrapping failed from [AF_INET]192.168.1.212:54428 (via [AF_INET]192.168.1.1%)
      Jun 15 10:48:35	openvpn	2705	192.168.1.212:54428 tls-crypt unwrap error: packet replay
      Jun 15 10:48:35	openvpn	2705	192.168.1.212:54428 tls-crypt unwrap error: bad packet ID (may be a replay): [ #9 / time = (1655315314) 2022-06-15 10:48:34 ] -- see the man page entry for --no-replay and --replay-window for more info or silence this warning with --mute-replay-warnings
      Jun 15 10:48:35	openvpn	2705	192.168.1.212:54428 TLS Error: tls-crypt unwrapping failed from [AF_INET]192.168.1.212:54428 (via [AF_INET]192.168.1.1%)
      Jun 15 10:48:35	openvpn	2705	192.168.1.212:54428 tls-crypt unwrap error: packet replay
      Jun 15 10:48:35	openvpn	2705	192.168.1.212:54428 tls-crypt unwrap error: bad packet ID (may be a replay): [ #8 / time = (1655315314) 2022-06-15 10:48:34 ] -- see the man page entry for --no-replay and --replay-window for more info or silence this warning with --mute-replay-warnings
      
      jimpJ 1 Reply Last reply Reply Quote 0
      • M
        maverick_slo
        last edited by

        For me, Android now works OK.
        But I had to disable DCO, because it`s like 40% slower than without it...

        jimpJ M 2 Replies Last reply Reply Quote 0
        • jimpJ
          jimp Rebel Alliance Developer Netgate @maverick_slo
          last edited by

          @maverick_slo said in 22.05 - DCO and OpenVPN issue:

          For me, Android now works OK.
          But I had to disable DCO, because it`s like 40% slower than without it...

          If your hardware supports AES-NI, make sure the AES-NI kernel module is loaded (System > Advanced, Misc tab).

          The only time we've seen DCO be slower is when the hardware supports crypto acceleration but it's not enabled in the kernel/modules. DCO is in the kernel so it can't just latch onto AES-NI via OpenSSL like non-DCO OpenVPN can.

          Remember: Upvote with the ๐Ÿ‘ button for any user/post you find to be helpful, informative, or deserving of recognition!

          Need help fast? Netgate Global Support!

          Do not Chat/PM for help!

          1 Reply Last reply Reply Quote 1
          • jimpJ
            jimp Rebel Alliance Developer Netgate @swixo
            last edited by

            @swixo said in 22.05 - DCO and OpenVPN issue:

            @jimp So I tried again with MacOS - working with DCO. No erros in logs. Connected with iOS - nothing. And as soon as I connected with iOS, the MacOS connection stopped working too.

            Log errors at time of failure:

            Jun 15 10:48:35	openvpn	2705	192.168.1.212:54428 TLS Error: tls-crypt unwrapping failed from [AF_INET]192.168.1.212:54428 (via [AF_INET]192.168.1.1%)
            Jun 15 10:48:35	openvpn	2705	192.168.1.212:54428 tls-crypt unwrap error: packet replay
            Jun 15 10:48:35	openvpn	2705	192.168.1.212:54428 tls-crypt unwrap error: bad packet ID (may be a replay): [ #9 / time = (1655315314) 2022-06-15 10:48:34 ] -- see the man page entry for --no-replay and --replay-window for more info or silence this warning with --mute-replay-warnings
            Jun 15 10:48:35	openvpn	2705	192.168.1.212:54428 TLS Error: tls-crypt unwrapping failed from [AF_INET]192.168.1.212:54428 (via [AF_INET]192.168.1.1%)
            Jun 15 10:48:35	openvpn	2705	192.168.1.212:54428 tls-crypt unwrap error: packet replay
            Jun 15 10:48:35	openvpn	2705	192.168.1.212:54428 tls-crypt unwrap error: bad packet ID (may be a replay): [ #8 / time = (1655315314) 2022-06-15 10:48:34 ] -- see the man page entry for --no-replay and --replay-window for more info or silence this warning with --mute-replay-warnings
            

            I still can't reproduce any problems here. Replay errors suggest it's a network-level problem with packets arriving out of order, though.

            Remember: Upvote with the ๐Ÿ‘ button for any user/post you find to be helpful, informative, or deserving of recognition!

            Need help fast? Netgate Global Support!

            Do not Chat/PM for help!

            S 1 Reply Last reply Reply Quote 0
            • S
              swixo @jimp
              last edited by

              @jimp Updated to 22.05.r.20220617.0613 this AM.

              Similar situation - First client to connect - its good. Second client to connect - Both lose data flow.

              1 Reply Last reply Reply Quote 0
              • M
                maverick_slo @maverick_slo
                last edited by

                It7s loaded...
                AES-NI CPU Crypto: Yes (active)

                But still it is really slower...

                1 Reply Last reply Reply Quote 0
                • T
                  tman222
                  last edited by tman222

                  Hi @jimp - just upgraded my 22.01 system to 22.05 RC and seeing the same issues that @swixo and @maverick_slo have described when DCO is enabled on OpenVPN.

                  In my case I'm using two clients:

                  1. iPhone with iOS 15.5 and version 3.2.3 of the OpenVPN Connect client for iOS
                  2. Macbook Pro with macOS 12.4 and latest version of Tunnelblick VPN client (3.8.7a)

                  Each client will work fine when connected individually. If I connect both then both will lose data flow as soon as the second client is connected (the order in which they are connected doesn't seem to matter). I've increased the verbosity on the OpenVPN logs but can't really see anything that looks to the a problem. Are there any settings on the client or server side I should doublecheck? I basically just enabled DCO on the server and tried things out. As a sanity check, I did disable DCO and then connected both clients - everything worked fine (i.e. data flow was not interrupted).

                  EDIT: Just tried with a non Apple client as well (Debian 11.3 Linux system). As soon as second client connects both lose data flow.

                  Regarding speed:

                  1. On the iPhone (WiFi) having DCO enabled is slower for me also (especially download) compared to no DCO with Fast I/O and 512KB buffer sizes.
                  2. On the Macbook Pro (wired) it is just as fast if not a tad faster, which I thought was interesting.

                  Thanks in advance for your help.

                  1 Reply Last reply Reply Quote 1
                  • T
                    tman222
                    last edited by

                    One additional thing I thought of: I did create an OpenVPN interface ovpns1 when I originally set things up. Are there any configuration settings I should be checking with respect to that interface (e.g. MTU, MSS, etc.)? Thanks again.

                    1 Reply Last reply Reply Quote 0
                    • T
                      tman222
                      last edited by

                      I'm curious if you guys had any idea on how to troubleshoot this further? The only thing I have not tried yet is creating a brand new OpenVPN server (tunnel) from scratch to see if that resolves the data flow issues. @swixo and @maverick_slo - have you guys tried this yet? Thanks in advance.

                      S 1 Reply Last reply Reply Quote 1
                      • S
                        swixo @tman222
                        last edited by

                        @tman222 I have not tried deleting the tunnel and starting over - although I may create a new one and see if it is ok.

                        We have a bug in either case and I want to preserve the failure condition so I can verify a proper fix if offered.

                        S 1 Reply Last reply Reply Quote 0
                        • S
                          swixo @swixo
                          last edited by

                          @swixo This issue has been reliably reproduced (and still failing) on 22.05-RELEASE (amd64).

                          1 Reply Last reply Reply Quote 0
                          • jimpJ
                            jimp Rebel Alliance Developer Netgate
                            last edited by

                            There must be something different about your setup. I still can't reproduce a problem here. I have three clients connected (pfSense, Windows, and OSX) and they all have working connectivity across the VPN.

                            Remember: Upvote with the ๐Ÿ‘ button for any user/post you find to be helpful, informative, or deserving of recognition!

                            Need help fast? Netgate Global Support!

                            Do not Chat/PM for help!

                            S 1 Reply Last reply Reply Quote 0
                            • S
                              swixo @jimp
                              last edited by swixo

                              @jimp I must have a setting aggravating it - user tman seems to have same issue.
                              Just in case - here are my settings:

                              DCO=on
                              Mode: TUN
                              Proto: UDP/v4
                              interface: Any 
                              Use Tls Key: Checked
                              TLS Usage: Enc and Auth
                              TLS KeyDir: Both
                              No Cert Rev List
                              No OCSP Check
                              DH: 4096
                              ECDH: secp384r1
                              Algorithms: AES-256-GCM
                              Fallback: AES-256-GCM
                              Auth Digest: SHA256 (256)
                              HW Crypto: Intel RDRAND
                              Cert Depth (Client+Server)
                              Client Certificate Key Usage: Enforce Checked
                              Tunnel Network /24
                              no IPV6
                              Redirect: Force all IPv4 Thru tunnel: Checked
                              No Redirect IPv6 checked
                              Concurrent connections: blank
                              Compression: Refuse non stub
                              Inter Client Comm: Checked
                              Duplicate Conn: Checked
                              Dup Conn Limit: Blank
                              Client Dynamic IP: Checked
                              Topology: One IP address per client common subnet
                              Ping: Inactive: 0
                              Method: Keepalive
                              interval: 10
                              Timeout: 60
                              DNS Default Domain: Checked
                              DNS Server Enable: Checked
                              Block Outside DNS: Unchecked
                              Force DNS Update: Unchecked
                              NTP Server: Checked
                              Custom Options: push "route 1.2.3.4 255.255.255.0"
                              UDP Fast I/O: Unchecked
                              Exit Notify: Disabled
                              Send/Receive Buffer: Default
                              Gateway Creation ipv4 only
                              1 Reply Last reply Reply Quote 0
                              • jimpJ
                                jimp Rebel Alliance Developer Netgate
                                last edited by

                                I'm using RA SSL/TLS but that shouldn't matter.

                                TLS Usage: Enc and Auth

                                Mine: Auth only

                                TLS KeyDir: Both

                                Mine: Default

                                DH: 4096

                                Mine: 2048

                                ECDH: secp384r1

                                Mine: Default

                                HW Crypto: Intel RDRAND

                                Set that to "No", you aren't going to gain anything from that one, especailly with DCO.

                                Redirect: Force all IPv4 Thru tunnel: Checked

                                I don't have this set but it's probably not going to change anything.

                                Inter Client Comm: Checked

                                Mine: Unchecked

                                Duplicate Conn: Checked

                                Mine: Unchecked (each client has a unique cert)

                                Client Dynamic IP: Checked

                                Mine: Unchecked

                                DNS Default Domain: Checked

                                Mine: Unchecked

                                DNS Server Enable: Checked

                                Mine: Unchecked

                                NTP Server: Checked

                                Mine: Unchecked

                                Custom Options: push "route 1.2.3.4 255.255.255.0"

                                Mine: Blank (but that one route alone should be fine)

                                A lot of those, like the pushed DNS options, shouldn't make a difference. I'd focus on trying to change one thing at a time to see if it helps and if it doesn't, change it back and try the next one. That way you can isolate the potential problem there.

                                I'd try changing TLS auth first to auth only, I'm not sure if anyone has tried TLS enc+auth with DCO yet.

                                Also you listed several options that should be hidden when you have DCO on (like inactive and UDP fast i/o), are those really showing in your GUI with DCO checked or did you transcribe those from somewhere else?

                                Remember: Upvote with the ๐Ÿ‘ button for any user/post you find to be helpful, informative, or deserving of recognition!

                                Need help fast? Netgate Global Support!

                                Do not Chat/PM for help!

                                S JeGrJ 2 Replies Last reply Reply Quote 0
                                • S
                                  swixo @jimp
                                  last edited by

                                  @jimp said in 22.05 - DCO and OpenVPN issue:

                                  HW Crypto: Intel RDRAND

                                  Set that to "No", you aren't going to gain anything from that one, especailly with DCO.

                                  I'd try changing TLS auth first to auth only, I'm not sure if anyone has tried TLS enc+auth with DCO yet.

                                  Also you listed several options that should be hidden when you have DCO on (like inactive and UDP fast i/o), are those really showing in your GUI with DCO checked or did you transcribe those from somewhere else?

                                  My bad on that - I had flipped DCO off just to get the VPN going again and hadn't thought the UX would change the options.

                                  I will test the changes you suggest and report.

                                  S 1 Reply Last reply Reply Quote 0
                                  • S
                                    swixo @swixo
                                    last edited by

                                    @swixo Tried a bunch of these changes and made things worse.

                                    Intel Hardware accel had no effect. But the Key Dir and Auth Only/Auth+Encry setting made a lot of difference. Had to export new profiles after every change to make sure clients were in sync.

                                    Most of the time - I couldn't connect at all - or data never flowed, until turning DCO off - then it usually worked fine after that.

                                    1 Reply Last reply Reply Quote 0
                                    • JeGrJ
                                      JeGr LAYER 8 Moderator @jimp
                                      last edited by

                                      @jimp Just to chime in:

                                      we were rolling out 2 new boxes 6100/4100 to a customer and I set up OpenVPN RAS pretty default with new DCO setting.

                                      Exactly same problem: .2 client can connect and route/transfer data, all other client IPs don't get ANY data at all sent through the connection. Switching off DCO immediatly works again. No spiffy or special stuff configured, just plain dead simple RAS setup with a single LAN network that gets pushed to the clients.

                                      Clients are 2x windows boxes with win10, newest OVPN Client 2.5.9/x64 and have no problems whatsoever. Routes are just fine, traffic simply refuses to flow through the server if you're not client #.2 :)

                                      System is on 22.05 stable

                                      Cheers
                                      \jens

                                      Don't forget to upvote ๐Ÿ‘ those who kindly offered their time and brainpower to help you!

                                      If you're interested, I'm available to discuss details of German-speaking paid support (for companies) if needed.

                                      1 Reply Last reply Reply Quote 1
                                      • stephenw10S
                                        stephenw10 Netgate Administrator
                                        last edited by

                                        Hmm, were those supplied with 22.05 or upgraded from 22.01?

                                        Does that include traffic between IPs in the tunnel subnet directly?

                                        JeGrJ 1 Reply Last reply Reply Quote 0
                                        • JeGrJ
                                          JeGr LAYER 8 Moderator @stephenw10
                                          last edited by

                                          @stephenw10 Those were freshly installed with 22.01 and clean upgraded to 22.05 - at least there were no errors or other hiccups in the logs or anywhere to see.

                                          Besides that it's a simple straightforward RAS style setup:

                                          • SSL/TLS + User Auth
                                          • DCO
                                          • tun L3
                                          • UDP/1194
                                          • TLS Key with TLS Auth (not auth+enc), default direction
                                          • VPN CA, VPN Cert, VPN CRL created
                                          • ECDH only
                                          • prime256v1
                                          • SHA256
                                          • no HW crypt (but AES-NI enabled kernel module)
                                          • cert depth 1 (C+S)
                                          • Strict User-CN Matching
                                          • Enforce Key usage
                                          • IP4 tunnel network 192.168.45.0/26 (to leave space to add another VPNs server later with .45.64/26, .45.128/26, etc.)
                                          • IP4 local network 192.168.40.0/24 (LAN)
                                          • compression: refuse any non stub (most secure)
                                          • dynamic IP selected
                                          • subnet
                                          • keepalive 5 30
                                          • DNS default domain set up to the locally used domain
                                          • DNS server set up to the local MS AD server
                                          • Gateway v4 only
                                          • Verb 3

                                          nothing else set. The RAS clients aren't supposed to talk with each other so no, inter-client comm isn't a thing here :)

                                          Cheers
                                          \jens

                                          Don't forget to upvote ๐Ÿ‘ those who kindly offered their time and brainpower to help you!

                                          If you're interested, I'm available to discuss details of German-speaking paid support (for companies) if needed.

                                          1 Reply Last reply Reply Quote 0
                                          • stephenw10S
                                            stephenw10 Netgate Administrator
                                            last edited by

                                            I mean can clients other than .2 ping the server tunnel IP?

                                            1 Reply Last reply Reply Quote 0
                                            • jimpJ jimp moved this topic from Plus 22.05 Development Snapshots (Retired) on
                                            • First post
                                              Last post
                                            Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.