OpenVPN client fatal error


  • We have an OpenVPN server + several clients that form a site-to-site VPN that have been working well for over a year. Each site is using an SG-3100 running pfSense 2.4.4-RELEASE-p3.

    Recently one of the clients will disconnect intermittently (a few days or weeks between incidents), with the following error on the client side:

    Jan  8 08:12:44 firewall openvpn[49806]: Assertion failed at crypto.c:258 (cipher_ctx_final(ctx->cipher, BEND(&work), &outlen))
    Jan  8 08:12:44 firewall openvpn[49806]: Exiting due to fatal error
    

    On the server side, the only log entry around that same time is the following:

    Jan  8 08:14:45 firewall openvpn[16421]: xxxxxxxx/x.x.x.x:1194 [xxxxxxxx] Inactivity timeout (--ping-restart), restarting
    

    The users at the client site will reboot the device to restore access.

    On the server side, we are also seeing the following types of errors, even when the client tunnel is working:

    Jan  8 09:11:20 firewall openvpn[16421]: xxxxxxxx/x.x.x.x:1194 Authenticate/Decrypt packet error: bad packet ID (may be a replay): [ #633636 ] -- see the man page entry for --no-replay and --replay-window for more info or silence this warning with --mute-replay-warnings
    

    Below is the server config:

    dev ovpns1
    verb 1
    dev-type tun
    dev-node /dev/tun1
    writepid /var/run/openvpn_server1.pid
    #user nobody
    #group nobody
    script-security 3
    daemon
    keepalive 10 60
    ping-timer-rem
    persist-tun
    persist-key
    proto udp4
    cipher AES-128-CBC
    auth SHA256
    up /usr/local/sbin/ovpn-linkup
    down /usr/local/sbin/ovpn-linkdown
    local x.x.x.x
    engine cryptodev
    tls-server
    server 172.16.0.0 255.255.255.0
    client-config-dir /var/etc/openvpn-csc/server1
    ifconfig 172.16.0.1 172.16.0.2
    tls-verify "/usr/local/sbin/ovpn_auth_verify tls 'xxxxxxxx-openvpn' 1"
    lport 1194
    management /var/etc/openvpn/server1.sock unix
    push "route 10.1.0.0 255.255.255.0"
    route 10.1.1.0 255.255.255.0
    route 10.1.2.0 255.255.255.0
    route 10.1.3.0 255.255.255.0
    route 10.1.5.0 255.255.255.0
    route 10.1.6.0 255.255.255.0
    route 10.1.7.0 255.255.255.0
    route 192.168.2.0 255.255.255.0
    ca /var/etc/openvpn/server1.ca 
    cert /var/etc/openvpn/server1.cert 
    key /var/etc/openvpn/server1.key 
    dh /etc/dh-parameters.2048
    crl-verify /var/etc/openvpn/server1.crl-verify 
    tls-auth /var/etc/openvpn/server1.tls-auth 0
    ncp-disable
    topology subnet
    

    Below is the client config:

    dev ovpnc1
    verb 1
    dev-type tun
    dev-node /dev/tun1
    writepid /var/run/openvpn_client1.pid
    #user nobody
    #group nobody
    script-security 3
    daemon
    keepalive 10 60
    ping-timer-rem
    persist-tun
    persist-key
    proto udp4
    cipher AES-128-CBC
    auth SHA256
    up /usr/local/sbin/ovpn-linkup
    down /usr/local/sbin/ovpn-linkdown
    local x.x.x.x
    engine cryptodev
    tls-client
    client
    lport 1194
    management /var/etc/openvpn/client1.sock unix
    remote x.x.x.x 1194
    ifconfig 172.16.0.2 172.16.0.1
    route 10.1.0.0 255.255.255.0
    route 10.1.1.0 255.255.255.0
    route 10.1.2.0 255.255.255.0
    route 10.1.5.0 255.255.255.0
    route 10.1.6.0 255.255.255.0
    route 10.1.7.0 255.255.255.0
    route 192.168.2.0 255.255.255.0
    route 172.16.9.0 255.255.255.0
    ca /var/etc/openvpn/client1.ca 
    cert /var/etc/openvpn/client1.cert 
    key /var/etc/openvpn/client1.key 
    crl-verify /var/etc/openvpn/client1.crl-verify 
    tls-auth /var/etc/openvpn/client1.tls-auth 1
    ncp-disable
    resolv-retry infinite
    topology subnet
    

    I haven't been able to locate any related reports of such an issue. Any ideas would be welcomed. Thanks.


  • @pfsenser_ca said in OpenVPN client fatal error:

    Any ideas would be welcomed.

    I would do this for the first time:
    https://docs.netgate.com/pfsense/en/latest/releases/2-4-5-p1.html
    😉


  • We updated both endpoints to 2.4.5_1 this weekend. Will see how the VPN performs this week. Thanks.


  • @pfsenser_ca said in OpenVPN client fatal error:

    Will see how the VPN performs this week. Thanks.

    I would even change the "cipher" to AES-256-GCM, if not too much work because of the endpoints....


  • We experienced the same issue again today, only 2 days after updating the systems. The error message on the client side is slightly different now:

    Jan 12 14:27:33 firewall openvpn[47886]: cipher_ctx_update: EVP_CipherUpdate() failed
    Jan 12 14:27:33 firewall openvpn[47886]: Exiting due to fatal error
    

    There is a single previous report of this issue noted here:

    https://forum.netgate.com/topic/133889/openvpn-sometimes-stops-working-with-cipher_ctx_update-evp_cipherupdate-failed/4

    Those users also reported the issue on an SG-3100 but no clear resolution.

    Our OpenVPN client is set to use "BSD cryptodev engine" currently, so we might try changing that to "None" to see if it helps.


  • @pfsenser_ca said in OpenVPN client fatal error:

    Our OpenVPN client is set to use "BSD cryptodev engine" currently

    This is definitely not a problem, at least I never met that (we have 100 - 120 OpenVPN clients- with BSD crypto)

    Try even these steps:

    • NCP disabling (if it is checked)

    • cipher change AES-256-GCM (GCM is faster and safer anyway, in principle)

    a2491c83-bc56-4fbe-86bc-87945afd831b-image.png