PIA VPN failing every hour



  • My PIV VPN won't stay up for very long.  Most times it comes back up on its own.  A few times I've had to restart the OpenVPN service by hand, and once I had to reboot as the OVPN service wouldn't respond at all.

    Excerpts from log:

    Mar 9 16:05:09 openvpn 58385 Initialization Sequence Completed
    Mar 9 16:05:09 openvpn 58385 /usr/local/sbin/ovpn-linkup ovpnc1 1500 1557 x.x.x.6 x.x.10.5 init
    Mar 9 16:05:09 openvpn 58385 /sbin/ifconfig ovpnc1 x.x.x.6 x.x.x.5 mtu 1500 netmask 255.255.255.255 up
    Mar 9 16:05:09 openvpn 58385 do_ifconfig, tt->did_ifconfig_ipv6_setup=0
    Mar 9 16:05:09 openvpn 58385 TUN/TAP device /dev/tun1 opened
    Mar 9 16:05:09 openvpn 58385 TUN/TAP device ovpnc1 exists previously, keep at program end
    Mar 9 16:05:07 openvpn 58385 [0b11e634ff031dfe118c0e72f207a30f] Peer Connection Initiated with [AF_INET]x.x.x.35:1198
    Mar 9 16:05:07 openvpn 58385 WARNING: 'comp-lzo' is present in remote config but missing in local config, remote='comp-lzo'
    Mar 9 16:05:07 openvpn 58385 WARNING: 'cipher' is used inconsistently, local='cipher AES-128-CBC', remote='cipher BF-CBC'
    Mar 9 16:05:07 openvpn 58385 WARNING: 'link-mtu' is used inconsistently, local='link-mtu 1557', remote='link-mtu 1542'
    Mar 9 16:05:07 openvpn 58385 WARNING: this configuration may cache passwords in memory – use the auth-nocache option to prevent this
    Mar 9 16:05:07 openvpn 58385 UDPv4 link remote: [AF_INET]x.x.x.35:1198
    Mar 9 16:05:07 openvpn 58385 UDPv4 link local (bound): [AF_INET]x.x.x.6:0
    Mar 9 16:05:07 openvpn 58385 TCP/UDP: Preserving recently used remote address: [AF_INET]x.x.x.35:1198
    Mar 9 16:05:07 openvpn 58385 Initializing OpenSSL support for engine 'rdrand'
    Mar 9 16:05:07 openvpn 58385 NOTE: the current –script-security setting may allow this configuration to call user-defined scripts
    Mar 9 16:05:07 openvpn 57856 library versions: OpenSSL 1.0.2m-freebsd 2 Nov 2017, LZO 2.10
    Mar 9 16:05:07 openvpn 57856 OpenVPN 2.4.4 amd64-portbld-freebsd11.1 [SSL (OpenSSL)] [LZO] [LZ4] [MH/RECVDA] [AEAD] built on Oct 8 2017
    Mar 9 16:05:07 openvpn 57856 WARNING: file '/var/etc/openvpn/client1.up' is group or others accessible
    Mar 9 16:05:01 openvpn 34426 Exiting due to fatal error
    Mar 9 16:05:01 openvpn 34426 TCP/UDP: Socket bind failed on local address [AF_INET]192.168.100.10:0: Can't assign requested address (errno=49)

    The last line above references an IP address that is not on my network, not sure if this is right or what is going on here.

    Mar 9 16:05:01 openvpn 34426 TCP/UDP: Preserving recently used remote address: [AF_INET]x.x.x.35:1198
    Mar 9 16:05:01 openvpn 34426 Initializing OpenSSL support for engine 'rdrand'
    Mar 9 16:05:01 openvpn 34426 NOTE: the current –script-security setting may allow this configuration to call user-defined scripts
    Mar 9 16:05:01 openvpn 34274 library versions: OpenSSL 1.0.2m-freebsd 2 Nov 2017, LZO 2.10
    Mar 9 16:05:01 openvpn 34274 OpenVPN 2.4.4 amd64-portbld-freebsd11.1 [SSL (OpenSSL)] [LZO] [LZ4] [MH/RECVDA] [AEAD] built on Oct 8 2017
    Mar 9 16:05:01 openvpn 34274 WARNING: file '/var/etc/openvpn/client1.up' is group or others accessible
    Mar 9 16:05:00 openvpn 86817 SIGTERM[hard,] received, process exiting
    Mar 9 16:05:00 openvpn 86817 event_wait : Interrupted system call (code=4)
    Mar 9 16:04:30 openvpn 86817 UDPv4 link remote: [AF_INET]x.x.x.35:1198
    Mar 9 16:04:30 openvpn 86817 UDPv4 link local (bound): [AF_INET]192.168.100.10:0

    Again, the line above… I don't know where this address is coming from.

    Mar 9 16:04:30 openvpn 86817 TCP/UDP: Preserving recently used remote address: [AF_INET]x.x.x.35:1198
    Mar 9 16:04:30 openvpn 86817 Initializing OpenSSL support for engine 'rdrand'
    Mar 9 16:04:30 openvpn 86817 NOTE: the current –script-security setting may allow this configuration to call user-defined scripts
    Mar 9 16:04:30 openvpn 86697 library versions: OpenSSL 1.0.2m-freebsd 2 Nov 2017, LZO 2.10
    Mar 9 16:04:30 openvpn 86697 OpenVPN 2.4.4 amd64-portbld-freebsd11.1 [SSL (OpenSSL)] [LZO] [LZ4] [MH/RECVDA] [AEAD] built on Oct 8 2017
    Mar 9 16:04:30 openvpn 86697 WARNING: file '/var/etc/openvpn/client1.up' is group or others accessible
    Mar 9 16:04:06 openvpn 11192 /usr/local/sbin/ovpn-linkdown ovpnc1 1500 1622 x.x.x.6 x.x.x.5 init
    Mar 9 16:04:06 openvpn 11192 ERROR: FreeBSD route delete command failed: external program exited with error status: 1
    Mar 9 16:04:06 openvpn 11192 Exiting due to fatal error
    Mar 9 16:04:06 openvpn 11192 TCP/UDP: Socket bind failed on local address [AF_INET]x.x.x.6:0: Can't assign requested address (errno=49)
    Mar 9 16:04:06 openvpn 11192 TCP/UDP: Preserving recently used remote address: [AF_INET]x.x.x.35:1198
    Mar 9 16:04:06 openvpn 11192 NOTE: the current –script-security setting may allow this configuration to call user-defined scripts
    Mar 9 16:03:56 openvpn 11192 SIGUSR1[soft,ping-restart] received, process restarting
    Mar 9 16:03:56 openvpn 11192 [0411ef342f03ddfe918c0e73f207a30f] Inactivity timeout (–ping-restart), restarting
    Mar 9 15:05:10 openvpn 11192 Initialization Sequence Completed
    Mar 9 15:05:10 openvpn 11192 /usr/local/sbin/ovpn-linkup ovpnc1 1500 1557 x.x.x.6 x.x.x.5 init
    Mar 9 15:05:10 openvpn 11192 /sbin/ifconfig ovpnc1 x.x.x.6 x.x.x.5 mtu 1500 netmask 255.255.255.255 up
    Mar 9 15:05:10 openvpn 11192 do_ifconfig, tt->did_ifconfig_ipv6_setup=0
    Mar 9 15:05:10 openvpn 11192 TUN/TAP device /dev/tun1 opened

    I'm not sure where to start troubleshooting.  The VPN was put in place less than 24 hours ago, and the firewall has been 24/7/365 reliable previous to that.



  • I just realized I had verbosity set too low, I have set 'verb 4' and restarted the service, if the above logs aren't sufficient, I will post them up when it fails again with more verbosity.

    ETA

    I've had a script running pinging 8.8.8.8 for hours now.

    Every hour, at 2 minutes after the hour, the VPN does down.  Every hour, at exactly two hours past.  A few times, leading up to the VPN going down, at 40 minutes after, ping times go from 20-25ms to 90-500ms, consistently.  As soon as the VPN comes back up, ping times go back to 20-25ms, for about 40 minutes.

    This cycle repeats over and over.

    ETA

    Noticed that when the VPN is down, the entire pfsense GUI is frozen.  ssh'ing into the box still works.  After the VPN comes back up, about 30 seconds later, the GUI is responsive and everything is normal.  Until the next time.

    System logs during this period of time show that most packages are throwing various errors and all are restarting.  Lots of "reloading filter" and "Starting all packages" messages during that time.

    I have made a few changes to the VPN config based on log entries (making things match on both ends, like compression etc), and so far nothing has worked, the connection continues to drop and reset somewhere between 2 and 3 minutes after the hours, like clockwork.



  • What server are you connecting to?

    Have you tried another server with the same results?

    also given the errors in your logs you have not followed/ matched the OVPN files.    match those as close as possible