OpenVPM woes and hard crash
-
I've posted before about severe troubles I had setting up pfsense+nordvpn but I eventually managed to get it working well for several weeks. Unfortunately, in the last few days I've faced similar issues where I periodically lose the internet connectivity.
This morning, The OpenVPN services were totally crashed (services stopped, status > OpenVPN said "service not running?", etc).
Logs:
May 29 08:41:51 openvpn 56921 /usr/local/sbin/ovpn-linkdown ovpnc2 1500 1654 10.8.2.2 255.255.255.0 init May 29 08:41:51 openvpn 56921 Closing TUN/TAP interface May 29 08:41:51 openvpn 56921 Exiting due to fatal error May 29 08:41:51 openvpn 56921 TCP/UDP: Socket bind failed on local address [AF_INET]xxx.xxx.xxx.xxx:0: Can't assign requested address (errno=49) May 29 08:41:51 openvpn 56921 Socket Buffers: R=[42080->524288] S=[57344->524288] May 29 08:41:51 openvpn 56921 TCP/UDP: Preserving recently used remote address: [AF_INET]xxx.xxx.xxx.xxx:1194 May 29 08:41:51 openvpn 40473 /usr/local/sbin/ovpn-linkdown ovpnc1 1500 1654 10.8.3.2 255.255.255.0 init May 29 08:41:51 openvpn 40473 Closing TUN/TAP interface May 29 08:41:51 openvpn 40473 Exiting due to fatal error May 29 08:41:51 openvpn 40473 TCP/UDP: Socket bind failed on local address [AF_INET]xxx.xxx.xxx.xxx:0: Can't assign requested address (errno=49) May 29 08:41:51 openvpn 40473 Socket Buffers: R=[42080->524288] S=[57344->524288] May 29 08:41:51 openvpn 40473 TCP/UDP: Preserving recently used remote address: [AF_INET]xxx.xxx.xxx.xxx:1194 May 29 08:41:51 openvpn 40473 Incoming Control Channel Authentication: Using 512 bit message hash 'SHA512' for HMAC authentication May 29 08:41:51 openvpn 40473 Outgoing Control Channel Authentication: Using 512 bit message hash 'SHA512' for HMAC authentication May 29 08:41:51 openvpn 56921 Incoming Control Channel Authentication: Using 512 bit message hash 'SHA512' for HMAC authentication May 29 08:41:51 openvpn 56921 Outgoing Control Channel Authentication: Using 512 bit message hash 'SHA512' for HMAC authentication May 29 08:41:51 openvpn 56921 NOTE: the current --script-security setting may allow this configuration to call user-defined scripts May 29 08:41:51 openvpn 40473 NOTE: the current --script-security setting may allow this configuration to call user-defined scripts May 29 08:41:50 openvpn 73684 /usr/local/sbin/ovpn-linkdown ovpnc3 1500 1654 10.8.0.8 255.255.255.0 init May 29 08:41:50 openvpn 73684 Closing TUN/TAP interface May 29 08:41:50 openvpn 73684 Exiting due to fatal error May 29 08:41:50 openvpn 73684 TCP/UDP: Socket bind failed on local address [AF_INET]xxx.xxx.xxx.xxx:0: Can't assign requested address (errno=49) May 29 08:41:50 openvpn 73684 Socket Buffers: R=[42080->524288] S=[57344->524288] May 29 08:41:50 openvpn 73684 TCP/UDP: Preserving recently used remote address: [AF_INET]xxx.xxx.xxx.xxx:1194 May 29 08:41:50 openvpn 73684 Incoming Control Channel Authentication: Using 512 bit message hash 'SHA512' for HMAC authentication May 29 08:41:50 openvpn 73684 Outgoing Control Channel Authentication: Using 512 bit message hash 'SHA512' for HMAC authentication May 29 08:41:50 openvpn 73684 NOTE: the current --script-security setting may allow this configuration to call user-defined scripts May 29 08:41:40 openvpn 73684 Restart pause, 10 second(s) May 29 08:41:40 openvpn 40473 Restart pause, 10 second(s) May 29 08:41:40 openvpn 56921 Restart pause, 10 second(s) May 29 08:41:40 openvpn 73684 SIGUSR1[soft,ping-restart] received, process restarting May 29 08:41:40 openvpn 40473 SIGUSR1[soft,ping-restart] received, process restarting May 29 08:41:40 openvpn 56921 SIGUSR1[soft,ping-restart] received, process restarting May 29 08:41:40 openvpn 56921 [gateway3.nordvpn.com] Inactivity timeout (--ping-restart), restarting May 29 08:41:40 openvpn 40473 [gateway2.nordvpn.com] Inactivity timeout (--ping-restart), restarting May 29 08:41:40 openvpn 73684 [gateway1.nordvpn.com] Inactivity timeout (--ping-restart), restarting May 29 08:38:55 openvpn 73684 write UDPv4: No route to host (code=65) May 29 08:38:55 openvpn 56921 write UDPv4: No route to host (code=65) May 29 08:38:55 openvpn 40473 write UDPv4: No route to host (code=65) May 29 08:38:55 openvpn 56921 write UDPv4: No route to host (code=65) May 29 08:38:55 openvpn 56921 write UDPv4: No route to host (code=65) May 29 08:38:55 openvpn 40473 write UDPv4: No route to host (code=65) May 29 08:38:55 openvpn 73684 write UDPv4: No route to host (code=65) May 29 08:38:55 openvpn 56921 write UDPv4: No route to host (code=65) May 29 08:38:54 openvpn 40473 write UDPv4: No route to host (code=65) May 29 08:38:54 openvpn 73684 write UDPv4: No route to host (code=65) May 29 08:38:54 openvpn 56921 write UDPv4: No route to host (code=65) May 29 08:38:54 openvpn 40473 write UDPv4: No route to host (code=65) May 29 08:38:54 openvpn 73684 write UDPv4: No route to host (code=65) May 29 08:38:54 openvpn 56921 write UDPv4: No route to host (code=65) May 29 08:38:53 openvpn 56921 write UDPv4: No route to host (code=65) May 29 08:38:53 openvpn 40473 write UDPv4: No route to host (code=65) May 29 08:38:53 openvpn 73684 write UDPv4: No route to host (code=65) May 29 08:38:53 openvpn 73684 write UDPv4: No route to host (code=65) May 29 08:38:53 openvpn 40473 write UDPv4: No route to host (code=65) May 29 08:38:53 openvpn 56921 write UDPv4: No route to host (code=65) May 29 08:38:52 openvpn 56921 write UDPv4: No route to host (code=65) May 29 08:38:52 openvpn 73684 write UDPv4: No route to host (code=65) May 29 08:38:52 openvpn 40473 write UDPv4: No route to host (code=65) May 29 08:38:52 openvpn 56921 write UDPv4: No route to host (code=65) May 29 08:38:52 openvpn 56921 write UDPv4: No route to host (code=65) May 29 08:38:52 openvpn 40473 write UDPv4: No route to host (code=65) May 29 08:38:52 openvpn 73684 write UDPv4: No route to host (code=65) May 29 08:38:52 openvpn 56921 write UDPv4: No route to host (code=65) May 29 08:38:51 openvpn 56921 write UDPv4: No route to host (code=65) May 29 08:38:51 openvpn 40473 write UDPv4: No route to host (code=65) May 29 08:38:51 openvpn 73684 write UDPv4: No route to host (code=65) May 29 08:38:51 openvpn 40473 write UDPv4: No route to host (code=65) May 29 08:38:51 openvpn 56921 write UDPv4: No route to host (code=65) May 29 08:38:51 openvpn 73684 write UDPv4: No route to host (code=65) May 29 08:38:50 openvpn 56921 write UDPv4: No route to host (code=65) May 29 08:38:50 openvpn 56921 write UDPv4: No route to host (code=65) May 29 08:38:50 openvpn 73684 write UDPv4: No route to host (code=65) May 29 08:38:50 openvpn 40473 write UDPv4: No route to host (code=65) May 29 08:38:50 openvpn 56921 write UDPv4: No route to host (code=65) May 29 08:38:50 openvpn 73684 write UDPv4: No route to host (code=65) May 29 08:38:50 openvpn 40473 write UDPv4: No route to host (code=65) May 29 08:38:50 openvpn 56921 write UDPv4: No route to host (code=65) May 29 08:38:49 openvpn 56921 write UDPv4: No route to host (code=65) May 29 08:38:49 openvpn 73684 write UDPv4: No route to host (code=65) May 29 08:38:49 openvpn 40473 write UDPv4: No route to host (code=65) May 29 08:38:49 openvpn 56921 write UDPv4: No route to host (code=65) May 29 08:38:49 openvpn 56921 write UDPv4: No route to host (code=65) May 29 08:38:49 openvpn 56921 write UDPv4: No route to host (code=65) May 29 08:38:49 openvpn 73684 write UDPv4: No route to host (code=65) May 29 08:38:49 openvpn 40473 write UDPv4: No route to host (code=65) May 29 08:38:48 openvpn 56921 write UDPv4: No route to host (code=65) May 29 08:38:48 openvpn 73684 write UDPv4: No route to host (code=65) May 29 08:38:48 openvpn 40473 write UDPv4: No route to host (code=65) May 29 08:38:47 openvpn 56921 write UDPv4: No route to host (code=65) May 29 08:38:47 openvpn 73684 write UDPv4: No route to host (code=65) May 29 08:38:47 openvpn 40473 write UDPv4: No route to host (code=65) May 29 08:38:47 openvpn 56921 write UDPv4: No route to host (code=65) May 29 08:38:47 openvpn 73684 write UDPv4: No route to host (code=65) May 29 08:38:47 openvpn 40473 write UDPv4: No route to host (code=65) May 29 08:38:46 openvpn 56921 write UDPv4: No route to host (code=65) May 29 08:38:46 openvpn 73684 write UDPv4: No route to host (code=65) May 29 08:38:46 openvpn 40473 write UDPv4: No route to host (code=65) May 29 08:38:46 openvpn 56921 write UDPv4: No route to host (code=65) May 29 08:38:46 openvpn 40473 write UDPv4: No route to host (code=65) May 29 08:38:46 openvpn 73684 write UDPv4: No route to host (code=65) May 29 08:38:45 openvpn 40473 write UDPv4: No route to host (code=65) May 29 08:38:45 openvpn 73684 write UDPv4: No route to host (code=65) May 29 08:38:45 openvpn 56921 write UDPv4: No route to host (code=65)
Another problem is, pfsense is NOT automatically switching to the standard WAN gateway when OpenVPN fails, and immediately lose all web connectivity. Default gateway is set to "Default" in System > Routing. Why is it not switching to my standard plain-vanilla ISP gateway when its having troubles connecting to nord's gateways????
-
@pftdm007
Would be interesting, what happened before these lines:openvpn - write UDPv4: No route to host (code=65)
Internet or gateway failure?
Is there a router in front of pfSense? -
Nothing unusual AFAIK... (note that I grabbed the raw log so its chronological order (oldest lines first)
May 29 07:43:34 pfsense openvpn[73684]: Validating certificate extended key usage May 29 07:43:34 pfsense openvpn[73684]: ++ Certificate has EKU (str) TLS Web Server Authentication, expects TLS Web Server Authentication May 29 07:43:34 pfsense openvpn[73684]: VERIFY EKU OK May 29 07:43:34 pfsense openvpn[73684]: VERIFY OK: depth=0, CN=gateway1.nordvpn.com May 29 07:43:34 pfsense openvpn[40473]: WARNING: 'link-mtu' is used inconsistently, local='link-mtu 1633', remote='link-mtu 1634' May 29 07:43:34 pfsense openvpn[40473]: WARNING: 'comp-lzo' is present in remote config but missing in local config, remote='comp-lzo' May 29 07:43:34 pfsense openvpn[40473]: Outgoing Data Channel: Cipher 'AES-256-GCM' initialized with 256 bit key May 29 07:43:34 pfsense openvpn[40473]: Incoming Data Channel: Cipher 'AES-256-GCM' initialized with 256 bit key May 29 07:43:34 pfsense openvpn[40473]: Control Channel: TLSv1.3, cipher TLSv1.3 TLS_AES_256_GCM_SHA384, peer certificate: 4096 bit RSA, signature: RSA-SHA512 May 29 07:43:34 pfsense openvpn[73684]: WARNING: 'link-mtu' is used inconsistently, local='link-mtu 1633', remote='link-mtu 1634' May 29 07:43:34 pfsense openvpn[73684]: WARNING: 'comp-lzo' is present in remote config but missing in local config, remote='comp-lzo' May 29 07:43:34 pfsense openvpn[73684]: Outgoing Data Channel: Cipher 'AES-256-GCM' initialized with 256 bit key May 29 07:43:34 pfsense openvpn[73684]: Incoming Data Channel: Cipher 'AES-256-GCM' initialized with 256 bit key May 29 07:43:34 pfsense openvpn[73684]: Control Channel: TLSv1.3, cipher TLSv1.3 TLS_AES_256_GCM_SHA384, peer certificate: 4096 bit RSA, signature: RSA-SHA512 May 29 07:46:45 pfsense openvpn[56921]: VERIFY WARNING: depth=0, unable to get certificate CRL: CN=gateway2.nordvpn.com May 29 07:46:45 pfsense openvpn[56921]: VERIFY WARNING: depth=1, unable to get certificate CRL: C=PA, O=NordVPN, CN=NordVPN CA7 May 29 07:46:45 pfsense openvpn[56921]: VERIFY WARNING: depth=2, unable to get certificate CRL: C=PA, O=NordVPN, CN=NordVPN Root CA May 29 07:46:45 pfsense openvpn[56921]: VERIFY OK: depth=2, C=PA, O=NordVPN, CN=NordVPN Root CA May 29 07:46:45 pfsense openvpn[56921]: VERIFY OK: depth=1, C=PA, O=NordVPN, CN=NordVPN CA7 May 29 07:46:45 pfsense openvpn[56921]: VERIFY KU OK May 29 07:46:45 pfsense openvpn[56921]: Validating certificate extended key usage May 29 07:46:45 pfsense openvpn[56921]: ++ Certificate has EKU (str) TLS Web Server Authentication, expects TLS Web Server Authentication May 29 07:46:45 pfsense openvpn[56921]: VERIFY EKU OK May 29 07:46:45 pfsense openvpn[56921]: VERIFY OK: depth=0, CN=gateway3.nordvpn.com May 29 07:46:45 pfsense openvpn[56921]: Outgoing Data Channel: Cipher 'AES-256-GCM' initialized with 256 bit key May 29 07:46:45 pfsense openvpn[56921]: Incoming Data Channel: Cipher 'AES-256-GCM' initialized with 256 bit key May 29 07:46:45 pfsense openvpn[56921]: Control Channel: TLSv1.3, cipher TLSv1.3 TLS_AES_256_GCM_SHA384, peer certificate: 4096 bit RSA, signature: RSA-SHA512 May 29 08:38:45 pfsense openvpn[56921]: write UDPv4: No route to host (code=65) May 29 08:38:45 pfsense openvpn[73684]: write UDPv4: No route to host (code=65) May 29 08:38:45 pfsense openvpn[40473]: write UDPv4: No route to host (code=65) May 29 08:38:46 pfsense openvpn[73684]: write UDPv4: No route to host (code=65) May 29 08:38:46 pfsense openvpn[40473]: write UDPv4: No route to host (code=65) May 29 08:38:46 pfsense openvpn[56921]: write UDPv4: No route to host (code=65) May 29 08:38:46 pfsense openvpn[40473]: write UDPv4: No route to host (code=65) May 29 08:38:46 pfsense openvpn[73684]: write UDPv4: No route to host (code=65) May 29 08:38:46 pfsense openvpn[56921]: write UDPv4: No route to host (code=65) May 29 08:38:47 pfsense openvpn[40473]: write UDPv4: No route to host (code=65) May 29 08:38:47 pfsense openvpn[73684]: write UDPv4: No route to host (code=65)
The internet was down during that time because the VPN ceased to function.... Other than that, I dont think I had an outage, and the WAN was still up and connecting fine....
There's an ISP cable modem upstream of pfsense but its in dumb mode (bridge mode) and has been for many years without issues....