Openvpn keeps restarting (Authenticate/Decrypt packet error)
-
Hi,
At home my pfsense installation has 4 site to site openvpn connections. For 1 it acts as a server, the 3 other as client (to other pfsense installation in 3 datacenters). I have setup the whole VPN config a while back, and it worked without issue for 4 months. But suddenly, I have a problem with the 3 (client) VPN connections. They keep restarting after 1-3 minutes. So they connect, they work fine for a couple of seconds or minutes (I can connect to any resource on the other side), and then they restart.
Some technical info of the pfsense setup at home:
- pfsense 2.2.4
- Intel(R) Core(TM) i7-2600 CPU @ 3.40GHz - 1 GB ram / SSD (on ESX)
- HA setup with same (virtual) hardware on the other side of a dedicated CARP interface
- 2 WAN uplinks (in a gateway group, different tiers)
- Peer to peer (shared key) openvpn connections to static IP's
Things I tried:
- Disable gateway monitoring
- Change ports
- Change IP's
- Connect to IP instead of hostname
- Shutdown master node (so secondary pfsense takes over, and initiates the VPN connections)
- Route the connections over the other WAN uplink
- Bound openvpn to gw group interface
- Used ping to continious initiate traffic over the tunnel
- Added no-replay
I also tested from another pfsense installation on another location to the 3 servers, these work without issue. Also, clients connected to my home pfsense (where my home pfsense is the server), work flawless, and stay up all the time.
This is a part of the log file (of the home server):
Jul 28 20:04:03 beomrouter01 openvpn[87767]: Peer Connection Initiated with [AF_INET]109.1.1.21:11942
Jul 28 20:04:04 beomrouter01 openvpn[87767]: Initialization Sequence Completed
Jul 28 20:04:23 beomrouter01 openvpn[26987]: MANAGEMENT: Client connected from /var/etc/openvpn/server2.sock
Jul 28 20:04:23 beomrouter01 openvpn[26987]: MANAGEMENT: CMD 'status 2'
Jul 28 20:04:23 beomrouter01 openvpn[26987]: MANAGEMENT: CMD 'quit'
Jul 28 20:04:23 beomrouter01 openvpn[26987]: MANAGEMENT: Client disconnected
Jul 28 20:04:58 beomrouter01 openvpn[29981]: Inactivity timeout (–ping-restart), restarting
Jul 28 20:04:58 beomrouter01 openvpn[29981]: SIGUSR1[soft,ping-restart] received, process restarting
Jul 28 20:05:00 beomrouter01 openvpn[29981]: NOTE: the current –script-security setting may allow this configuration to call user-defined scripts
Jul 28 20:05:00 beomrouter01 openvpn[29981]: Re-using pre-shared static key
Jul 28 20:05:00 beomrouter01 openvpn[29981]: Preserving previous TUN/TAP instance: ovpnc4
Jul 28 20:05:00 beomrouter01 openvpn[29981]: UDPv4 link local (bound): [AF_INET]192.168.101.199
Jul 28 20:05:00 beomrouter01 openvpn[29981]: UDPv4 link remote: [AF_INET]31.1.1.20:11946
Jul 28 20:05:01 beomrouter01 openvpn[96721]: Inactivity timeout (–ping-restart), restarting
Jul 28 20:05:01 beomrouter01 openvpn[96721]: SIGUSR1[soft,ping-restart] received, process restarting
Jul 28 20:05:01 beomrouter01 openvpn[29981]: Peer Connection Initiated with [AF_INET]31.1.1.20:11946
Jul 28 20:05:03 beomrouter01 openvpn[29981]: Initialization Sequence Completed
Jul 28 20:05:03 beomrouter01 openvpn[96721]: NOTE: the current –script-security setting may allow this configuration to call user-defined scripts
Jul 28 20:05:03 beomrouter01 openvpn[96721]: Re-using pre-shared static key
Jul 28 20:05:03 beomrouter01 openvpn[96721]: Preserving previous TUN/TAP instance: ovpnc1
Jul 28 20:05:03 beomrouter01 openvpn[96721]: UDPv4 link local (bound): [AF_INET]192.168.101.199
Jul 28 20:05:03 beomrouter01 openvpn[96721]: UDPv4 link remote: [AF_INET]176.1.1.58:11940
Jul 28 20:05:03 beomrouter01 openvpn[96721]: Peer Connection Initiated with [AF_INET]176.1.1.58:11940
Jul 28 20:05:04 beomrouter01 openvpn[96721]: Initialization Sequence CompletedAnd this is the server log
Jul 28 23:07:13 defarouter01 openvpn[10443]: Peer Connection Initiated with [AF_INET]84.1.1.201:29431
Jul 28 23:07:22 defarouter01 openvpn[10443]: Authenticate/Decrypt packet error: bad packet ID (may be a replay): [ #9 / time = (1438117571) Tue Jul 28 23:06:11 2015 ] – see the man page entry for --no-replay and --replay-window for more info or silence this warning with --mute-replay-warnings
Jul 28 23:07:32 defarouter01 openvpn[10443]: Authenticate/Decrypt packet error: bad packet ID (may be a replay): [ #10 / time = (1438117571) Tue Jul 28 23:06:11 2015 ] – see the man page entry for --no-replay and --replay-window for more info or silence this warning with --mute-replay-warnings
Jul 28 23:07:42 defarouter01 openvpn[10443]: Authenticate/Decrypt packet error: bad packet ID (may be a replay): [ #11 / time = (1438117571) Tue Jul 28 23:06:11 2015 ] – see the man page entry for --no-replay and --replay-window for more info or silence this warning with --mute-replay-warnings
Jul 28 23:07:52 defarouter01 openvpn[10443]: Authenticate/Decrypt packet error: bad packet ID (may be a replay): [ #12 / time = (1438117571) Tue Jul 28 23:06:11 2015 ] – see the man page entry for --no-replay and --replay-window for more info or silence this warning with --mute-replay-warnings
Jul 28 23:08:02 defarouter01 openvpn[10443]: Authenticate/Decrypt packet error: bad packet ID (may be a replay): [ #13 / time = (1438117571) Tue Jul 28 23:06:11 2015 ] – see the man page entry for --no-replay and --replay-window for more info or silence this warning with --mute-replay-warnings
Jul 28 23:08:12 defarouter01 openvpn[10443]: Authenticate/Decrypt packet error: bad packet ID (may be a replay): [ #14 / time = (1438117571) Tue Jul 28 23:06:11 2015 ] – see the man page entry for --no-replay and --replay-window for more info or silence this warning with --mute-replay-warnings
Jul 28 23:08:16 defarouter01 openvpn[10443]: Peer Connection Initiated with [AF_INET]84.1.1.201:15932
Jul 28 23:08:16 defarouter01 openvpn[10443]: Authenticate/Decrypt packet error: bad packet ID (may be a replay): [ #415 / time = (1438117633) Tue Jul 28 23:07:13 2015 ] – see the man page entry for --no-replay and --replay-window for more info or silence this warning with --mute-replay-warnings
Jul 28 23:08:16 defarouter01 openvpn[10443]: Authenticate/Decrypt packet error: bad packet ID (may be a replay): [ #416 / time = (1438117633) Tue Jul 28 23:07:13 2015 ] – see the man page entry for --no-replay and --replay-window for more info or silence this warning with --mute-replay-warnings
Jul 28 23:08:16 defarouter01 openvpn[10443]: Authenticate/Decrypt packet error: bad packet ID (may be a replay): [ #417 / time = (1438117633) Tue Jul 28 23:07:13 2015 ] – see the man page entry for --no-replay and --replay-window for more info or silence this warning with --mute-replay-warnings
Jul 28 23:08:17 defarouter01 openvpn[10443]: Authenticate/Decrypt packet error: bad packet ID (may be a replay): [ #418 / time = (1438117633) Tue Jul 28 23:07:13 2015 ] – see the man page entry for --no-replay and --replay-window for more info or silence this warning with --mute-replay-warnings
Jul 28 23:08:17 defarouter01 openvpn[10443]: Authenticate/Decrypt packet error: bad packet ID (may be a replay): [ #419 / time = (1438117633) Tue Jul 28 23:07:13 2015 ] – see the man page entry for --no-replay and --replay-window for more info or silence this warning with --mute-replay-warnings
Jul 28 23:08:17 defarouter01 openvpn[10443]: Authenticate/Decrypt packet error: bad packet ID (may be a replay): [ #420 / time = (1438117633) Tue Jul 28 23:07:13 2015 ] – see the man page entry for --no-replay and --replay-window for more info or silence this warning with --mute-replay-warnings
Jul 28 23:08:17 defarouter01 openvpn[10443]: Authenticate/Decrypt packet error: bad packet ID (may be a replay): [ #421 / time = (1438117633) Tue Jul 28 23:07:13 2015 ] – see the man page entry for --no-replay and --replay-window for more info or silence this warning with --mute-replay-warnings
Jul 28 23:08:19 defarouter01 openvpn[10443]: Authenticate/Decrypt packet error: bad packet ID (may be a replay): [ #422 / time = (1438117633) Tue Jul 28 23:07:13 2015 ] – see the man page entry for --no-replay and --replay-window for more info or silence this warning with --mute-replay-warnings
Jul 28 23:08:19 defarouter01 openvpn[10443]: Authenticate/Decrypt packet error: bad packet ID (may be a replay): [ #423 / time = (1438117633) Tue Jul 28 23:07:13 2015 ] – see the man page entry for --no-replay and --replay-window for more info or silence this warning with --mute-replay-warnings
Jul 28 23:08:19 defarouter01 openvpn[10443]: Authenticate/Decrypt packet error: bad packet ID (may be a replay): [ #424 / time = (1438117633) Tue Jul 28 23:07:13 2015 ] – see the man page entry for --no-replay and --replay-window for more info or silence this warning with --mute-replay-warnings
Jul 28 23:08:20 defarouter01 openvpn[10443]: Authenticate/Decrypt packet error: bad packet ID (may be a replay): [ #425 / time = (1438117633) Tue Jul 28 23:07:13 2015 ] – see the man page entry for --no-replay and --replay-window for more info or silence this warning with --mute-replay-warnings
Jul 28 23:08:20 defarouter01 openvpn[10443]: Authenticate/Decrypt packet error: bad packet ID (may be a replay): [ #426 / time = (1438117633) Tue Jul 28 23:07:13 2015 ] – see the man page entry for --no-replay and --replay-window for more info or silence this warning with --mute-replay-warnings
Jul 28 23:08:21 defarouter01The server side logs are filled with "Authenticate/Decrypt packet error: bad packet ID (may be a replay)" errors. I have double checked the time settings (to avoid time diff) and checked the Cryptographic Settings. They are the same on both sides.
I am not sure what I can try more. Anyone has an idea what could be the cause?
Thanks for your help in advance!
-
Well, adding no-replay;mute-replay-warnings seem to solve the issue, but i'm wondering why i'm getting replay errors…
-
That's just hiding the fact it's happening. Client probably on wifi I'm guessing? Maybe the Internet connection as well? Generally it's because of packet duplication, which most often happens with wifi or cell networks.
-
@cmb:
That's just hiding the fact it's happening. Client probably on wifi I'm guessing? Maybe the Internet connection as well? Generally it's because of packet duplication, which most often happens with wifi or cell networks.
No, the client is on a 200 mbit cable modem. I ran speedtests and continious pings, which showed no issues/loss
-
It's not loss that causes it, rather duplication of packets. The outer OpenVPN with UDP won't ever be retransmitted by the client.
Something somewhere is duplicating packets. Might want to setup some packet captures and see what it looks like while it's happening.
-
Having the same issue here. Happening as of just a month or so ago. Running 2.4.3-p1 on all ends. It's also just ONE of our site-to-sites with this being a drastic issue. I did notice a few similar packets on another of the sites though but not enough to be a showstopper.
Might it be an issue with the version of openssl?
https://github.com/SoftEtherVPN/SoftEtherVPN/issues/434<quote>
"After some testing, I found out that the problem only happens with OpenSSL 1.1.0f.
Newer and older versions work fine, meaning that it's probably a regression introduced in 1.1.0f and then fixed in 1.1.0g.
Debian Stretch users should install libssl1.1 from the sid branch, as a temporary workaround."</quote> -
Not sure what you think SoftEtherVPN has to do with OpenVPN. And this thread is ancient.
-
@derelict Had nothing to do with SoftEtherVPN and moreso to do with the underlying SSL package they were using. That said, I do now see how old this is. That part of your comment was at least somewhat helpful.