OpenVPN seamless roaming across Multi-WAN



  • hi everybody.

    i realized the other post was badly written, i've deleted it and reposted with a much better description, kindly see below.
    my setup looks like this, both pfsense ends are the latest release (2.4.4-RELEASE-p3 thus OpenVPN 2.4.6)

    192.168.1.0/24 <-> pfsense-local <-> UDP tun via either DSL or LTE <-> pfsense-remote <-NAT-> internet
    

    The GWs for DSL and LTE have been put together in a GW group, DSL being Tier1, LTE being Tier2. The failing back and forth between DSL and LTE works nicely. however the connections routed through the vpn tunnel drop on failing over uplinks, besides the UDP tunnel being reestablished via the secondary uplink.

    what am i missing in order to have tcp connections in the vpn tunnel survive a change in the underlying transport?

    see the configs below. public IPs, common names and other sensitive data has been replaced with placeholders.

    11.11.11.11: DSL IP
    22.22.22.22: LTE IP
    99.99.99.99: Far end Public IP

    far end config:

    dev ovpns1
    verb 5
    dev-type tun
    dev-node /dev/tun1
    writepid /var/run/openvpn_server1.pid
    #user nobody
    #group nobody
    script-security 3
    daemon
    keepalive 10 60
    ping-timer-rem
    persist-tun
    persist-key
    proto udp
    cipher AES-256-OFB
    auth SHA256
    up /usr/local/sbin/ovpn-linkup
    down /usr/local/sbin/ovpn-linkdown
    multihome
    engine rdrand
    tls-server
    server 10.0.0.0 255.255.255.0
    client-config-dir /var/etc/openvpn-csc/server1
    lport 1194
    management /var/etc/openvpn/server1.sock unix
    max-clients 3
    push "redirect-gateway def1"
    duplicate-cn
    ca /var/etc/openvpn/server1.ca 
    cert /var/etc/openvpn/server1.cert 
    key /var/etc/openvpn/server1.key 
    dh /etc/dh-parameters.1024
    tls-crypt /var/etc/openvpn/server1.tls-crypt 
    ncp-disable
    comp-lzo no
    passtos
    persist-remote-ip
    float
    topology subnet
    float
    route 192.168.1.0 255.255.255.0
    client-config-dir /var/etc/openvpn/ccd
    keepalive 1 2
    status /var/log/openvpn.status 1
    status-version 2
    
    

    the ccd matching my CN has this:

    iroute 192.168.1.0 255.255.255.0
    iroute 10.0.0.0 255.255.255.0
    
    

    local end config:

    dev ovpnc1
    verb 5
    dev-type tun
    dev-node /dev/tun1
    writepid /var/run/openvpn_client1.pid
    #user nobody
    #group nobody
    script-security 3
    daemon
    keepalive 10 60
    ping-timer-rem
    persist-tun
    persist-key
    proto udp
    cipher AES-256-OFB
    auth SHA256
    up /usr/local/sbin/ovpn-linkup
    down /usr/local/sbin/ovpn-linkdown
    multihome
    engine rdrand
    tls-client
    client
    lport 0
    management /var/etc/openvpn/client1.sock unix
    remote 99.99.99.99 1194
    ifconfig 10.0.0.2 10.0.0.1
    ca /var/etc/openvpn/client1.ca 
    cert /var/etc/openvpn/client1.cert 
    key /var/etc/openvpn/client1.key 
    tls-crypt /var/etc/openvpn/client1.tls-crypt 
    ncp-disable
    comp-lzo no
    passtos
    resolv-retry infinite
    topology subnet
    redirect-gateway def1
    float
    keepalive 1 2
    

    a connection switch seen from the far-end pfsense openvpn log looks like this.

    Dec 2 16:31:59 	openvpn 	18499 	MULTI: multi_create_instance called
    Dec 2 16:31:59 	openvpn 	18499 	22.22.22.22 Re-using SSL/TLS context
    Dec 2 16:31:59 	openvpn 	18499 	22.22.22.22 Control Channel MTU parms [ L:1622 D:1156 EF:94 EB:0 ET:0 EL:3 ]
    Dec 2 16:31:59 	openvpn 	18499 	22.22.22.22 Data Channel MTU parms [ L:1622 D:1450 EF:122 EB:406 ET:0 EL:3 ]
    Dec 2 16:31:59 	openvpn 	18499 	22.22.22.22 Local Options String (VER=V4): 'V4,dev-type tun,link-mtu 1574,tun-mtu 1500,proto UDPv4,comp-lzo,cipher AES-256-OFB,auth SHA256,keysize 256,key-method 2,tls-server'
    Dec 2 16:31:59 	openvpn 	18499 	22.22.22.22 Expected Remote Options String (VER=V4): 'V4,dev-type tun,link-mtu 1574,tun-mtu 1500,proto UDPv4,comp-lzo,cipher AES-256-OFB,auth SHA256,keysize 256,key-method 2,tls-client'
    Dec 2 16:31:59 	openvpn 	18499 	22.22.22.22 TLS: Initial packet from [AF_INET6]::ffff:22.22.22.22:28833 (via ::ffff:99.99.99.99%vtnet0), sid=5a21876c 088c3d74
    Dec 2 16:32:00 	openvpn 	18499 	22.22.22.22 VERIFY OK: depth=1, CN=internal-ca
    Dec 2 16:32:00 	openvpn 	18499 	22.22.22.22 VERIFY OK: depth=0, CN=common-name
    Dec 2 16:32:00 	openvpn 	18499 	common-name/11.11.11.11 [common-name] Inactivity timeout (--ping-restart), restarting
    Dec 2 16:32:00 	openvpn 	18499 	common-name/11.11.11.11 SIGUSR1[soft,ping-restart] received, client-instance restarting
    Dec 2 16:32:00 	openvpn 	18499 	22.22.22.22 peer info: IV_VER=2.4.6
    Dec 2 16:32:00 	openvpn 	18499 	22.22.22.22 peer info: IV_PLAT=freebsd
    Dec 2 16:32:00 	openvpn 	18499 	22.22.22.22 peer info: IV_PROTO=2
    Dec 2 16:32:00 	openvpn 	18499 	22.22.22.22 peer info: IV_LZ4=1
    Dec 2 16:32:00 	openvpn 	18499 	22.22.22.22 peer info: IV_LZ4v2=1
    Dec 2 16:32:00 	openvpn 	18499 	22.22.22.22 peer info: IV_LZO=1
    Dec 2 16:32:00 	openvpn 	18499 	22.22.22.22 peer info: IV_COMP_STUB=1
    Dec 2 16:32:00 	openvpn 	18499 	22.22.22.22 peer info: IV_COMP_STUBv2=1
    Dec 2 16:32:00 	openvpn 	18499 	22.22.22.22 peer info: IV_TCPNL=1
    Dec 2 16:32:00 	openvpn 	18499 	22.22.22.22 Outgoing Data Channel: Cipher 'AES-256-OFB' initialized with 256 bit key
    Dec 2 16:32:00 	openvpn 	18499 	22.22.22.22 Outgoing Data Channel: Using 256 bit message hash 'SHA256' for HMAC authentication
    Dec 2 16:32:00 	openvpn 	18499 	22.22.22.22 Incoming Data Channel: Cipher 'AES-256-OFB' initialized with 256 bit key
    Dec 2 16:32:00 	openvpn 	18499 	22.22.22.22 Incoming Data Channel: Using 256 bit message hash 'SHA256' for HMAC authentication
    Dec 2 16:32:00 	openvpn 	18499 	22.22.22.22 Control Channel: TLSv1.2, cipher TLSv1/SSLv3 ECDHE-RSA-AES256-GCM-SHA384, 2048 bit RSA
    Dec 2 16:32:00 	openvpn 	18499 	22.22.22.22 [common-name] Peer Connection Initiated with [AF_INET6]::ffff:22.22.22.22:28833 (via ::ffff:99.99.99.99%vtnet0)
    Dec 2 16:32:00 	openvpn 	18499 	common-name/22.22.22.22 OPTIONS IMPORT: reading client specific options from: /var/etc/openvpn/ccd/common-name
    Dec 2 16:32:00 	openvpn 	18499 	common-name/22.22.22.22 MULTI_sva: pool returned IPv4=10.0.0.2, IPv6=(Not enabled)
    Dec 2 16:32:00 	openvpn 	18499 	common-name/22.22.22.22 MULTI: Learn: 10.0.0.2 -> common-name/22.22.22.22
    Dec 2 16:32:00 	openvpn 	18499 	common-name/22.22.22.22 MULTI: primary virtual IP for common-name/22.22.22.22: 10.0.0.2
    Dec 2 16:32:00 	openvpn 	18499 	common-name/22.22.22.22 MULTI: internal route 10.0.0.0/24 -> common-name/22.22.22.22
    Dec 2 16:32:00 	openvpn 	18499 	common-name/22.22.22.22 MULTI: Learn: 10.0.0.0/24 -> common-name/22.22.22.22
    Dec 2 16:32:00 	openvpn 	18499 	common-name/22.22.22.22 MULTI: internal route 192.168.1.0/24 -> common-name/22.22.22.22
    Dec 2 16:32:00 	openvpn 	18499 	common-name/22.22.22.22 MULTI: Learn: 192.168.1.0/24 -> common-name/22.22.22.22
    Dec 2 16:32:01 	openvpn 	18499 	common-name/22.22.22.22 PUSH: Received control message: 'PUSH_REQUEST'
    Dec 2 16:32:01 	openvpn 	18499 	common-name/22.22.22.22 SENT CONTROL [common-name]: 'PUSH_REPLY,redirect-gateway def1,route-gateway 10.0.0.1,topology subnet,ping 1,ping-restart 2,ifconfig 10.0.0.2 255.255.255.0,peer-id 0' (status=1)
    Dec 2 16:32:01 	openvpn 	18499 	common-name/22.22.22.22 MULTI: bad source address from client [::], packet dropped
    Dec 2 16:32:02 	openvpn 	18499 	common-name/22.22.22.22 MULTI: Learn: 192.168.1.100 -> common-name/22.22.22.22
    Dec 2 16:32:02 	openvpn 	18499 	common-name/22.22.22.22 MULTI: bad source address from client [::], packet dropped
    Dec 2 16:32:29 	openvpn 	18499 	MANAGEMENT: Client connected from /var/etc/openvpn/server1.sock
    Dec 2 16:32:29 	openvpn 	18499 	MANAGEMENT: CMD 'status 2'
    Dec 2 16:32:29 	openvpn 	18499 	MANAGEMENT: CMD 'quit'
    Dec 2 16:32:29 	openvpn 	18499 	MANAGEMENT: Client disconnected 
    

    from the local end:

    Dec 2 16:31:59 	openvpn 	77082 	[common-name] Inactivity timeout (--ping-restart), restarting
    Dec 2 16:31:59 	openvpn 	77082 	TCP/UDP: Closing socket
    Dec 2 16:31:59 	openvpn 	77082 	SIGUSR1[soft,ping-restart] received, process restarting
    Dec 2 16:31:59 	openvpn 	77082 	Restart pause, 5 second(s)
    Dec 2 16:31:59 	openvpn 	77082 	/sbin/route delete -net 99.99.99.99 172.16.0.62 255.255.255.255
    Dec 2 16:31:59 	openvpn 	77082 	/sbin/route delete -net 0.0.0.0 10.0.0.1 128.0.0.0
    Dec 2 16:31:59 	openvpn 	77082 	/sbin/route delete -net 128.0.0.0 10.0.0.1 128.0.0.0
    Dec 2 16:31:59 	openvpn 	77082 	Closing TUN/TAP interface
    Dec 2 16:31:59 	openvpn 	77082 	/usr/local/sbin/ovpn-linkdown ovpnc1 0 0 10.0.0.3 255.255.255.0 init
    Dec 2 16:31:59 	openvpn 	77082 	SIGTERM[hard,init_instance] received, process exiting
    Dec 2 16:31:59 	openvpn 	21723 	Current Parameter Settings:
    Dec 2 16:31:59 	openvpn 	21723 	config = '/var/etc/openvpn/client1.conf'
    Dec 2 16:31:59 	openvpn 	21723 	mode = 0
    Dec 2 16:31:59 	openvpn 	21723 	show_ciphers = DISABLED
    Dec 2 16:31:59 	openvpn 	21723 	show_digests = DISABLED
    Dec 2 16:31:59 	openvpn 	21723 	show_engines = DISABLED
    Dec 2 16:31:59 	openvpn 	21723 	genkey = DISABLED
    Dec 2 16:31:59 	openvpn 	21723 	key_pass_file = '[UNDEF]'
    Dec 2 16:31:59 	openvpn 	21723 	show_tls_ciphers = DISABLED
    Dec 2 16:31:59 	openvpn 	21723 	connect_retry_max = 0
    Dec 2 16:31:59 	openvpn 	21723 	Connection profiles [0]:
    Dec 2 16:31:59 	openvpn 	21723 	proto = udp
    Dec 2 16:31:59 	openvpn 	21723 	local = '[UNDEF]'
    Dec 2 16:31:59 	openvpn 	21723 	local_port = '0'
    Dec 2 16:31:59 	openvpn 	21723 	remote = '99.99.99.99'
    Dec 2 16:31:59 	openvpn 	21723 	remote_port = '1194'
    Dec 2 16:31:59 	openvpn 	21723 	remote_float = ENABLED
    Dec 2 16:31:59 	openvpn 	21723 	bind_defined = DISABLED
    Dec 2 16:31:59 	openvpn 	21723 	bind_local = ENABLED
    Dec 2 16:31:59 	openvpn 	21723 	bind_ipv6_only = DISABLED
    Dec 2 16:31:59 	openvpn 	21723 	connect_retry_seconds = 5
    Dec 2 16:31:59 	openvpn 	21723 	connect_timeout = 120
    Dec 2 16:31:59 	openvpn 	21723 	socks_proxy_server = '[UNDEF]'
    Dec 2 16:31:59 	openvpn 	21723 	socks_proxy_port = '[UNDEF]'
    Dec 2 16:31:59 	openvpn 	21723 	tun_mtu = 1500
    Dec 2 16:31:59 	openvpn 	21723 	tun_mtu_defined = ENABLED
    Dec 2 16:31:59 	openvpn 	21723 	link_mtu = 1500
    Dec 2 16:31:59 	openvpn 	21723 	link_mtu_defined = DISABLED
    Dec 2 16:31:59 	openvpn 	21723 	tun_mtu_extra = 0
    Dec 2 16:31:59 	openvpn 	21723 	tun_mtu_extra_defined = DISABLED
    Dec 2 16:31:59 	openvpn 	21723 	mtu_discover_type = -1
    Dec 2 16:31:59 	openvpn 	21723 	fragment = 0
    Dec 2 16:31:59 	openvpn 	21723 	mssfix = 1450
    Dec 2 16:31:59 	openvpn 	21723 	explicit_exit_notification = 0
    Dec 2 16:31:59 	openvpn 	21723 	Connection profiles END
    Dec 2 16:31:59 	openvpn 	21723 	remote_random = DISABLED
    Dec 2 16:31:59 	openvpn 	21723 	ipchange = '[UNDEF]'
    Dec 2 16:31:59 	openvpn 	21723 	dev = 'ovpnc1'
    Dec 2 16:31:59 	openvpn 	21723 	dev_type = 'tun'
    Dec 2 16:31:59 	openvpn 	21723 	dev_node = '/dev/tun1'
    Dec 2 16:31:59 	openvpn 	21723 	lladdr = '[UNDEF]'
    Dec 2 16:31:59 	openvpn 	21723 	topology = 3
    Dec 2 16:31:59 	openvpn 	21723 	ifconfig_local = '10.0.0.2'
    Dec 2 16:31:59 	openvpn 	21723 	ifconfig_remote_netmask = '10.0.0.1'
    Dec 2 16:31:59 	openvpn 	21723 	ifconfig_noexec = DISABLED
    Dec 2 16:31:59 	openvpn 	21723 	ifconfig_nowarn = DISABLED
    Dec 2 16:31:59 	openvpn 	21723 	ifconfig_ipv6_local = '[UNDEF]'
    Dec 2 16:31:59 	openvpn 	21723 	ifconfig_ipv6_netbits = 0
    Dec 2 16:31:59 	openvpn 	21723 	ifconfig_ipv6_remote = '[UNDEF]'
    Dec 2 16:31:59 	openvpn 	21723 	shaper = 0
    Dec 2 16:31:59 	openvpn 	21723 	mtu_test = 0
    Dec 2 16:31:59 	openvpn 	21723 	mlock = DISABLED
    Dec 2 16:31:59 	openvpn 	21723 	keepalive_ping = 1
    Dec 2 16:31:59 	openvpn 	21723 	keepalive_timeout = 2
    Dec 2 16:31:59 	openvpn 	21723 	inactivity_timeout = 0
    Dec 2 16:31:59 	openvpn 	21723 	ping_send_timeout = 1
    Dec 2 16:31:59 	openvpn 	21723 	ping_rec_timeout = 2
    Dec 2 16:31:59 	openvpn 	21723 	ping_rec_timeout_action = 2
    Dec 2 16:31:59 	openvpn 	21723 	ping_timer_remote = ENABLED
    Dec 2 16:31:59 	openvpn 	21723 	remap_sigusr1 = 0
    Dec 2 16:31:59 	openvpn 	21723 	persist_tun = ENABLED
    Dec 2 16:31:59 	openvpn 	21723 	persist_local_ip = DISABLED
    Dec 2 16:31:59 	openvpn 	21723 	persist_remote_ip = DISABLED
    Dec 2 16:31:59 	openvpn 	21723 	persist_key = ENABLED
    Dec 2 16:31:59 	openvpn 	21723 	passtos = ENABLED
    Dec 2 16:31:59 	openvpn 	21723 	resolve_retry_seconds = 1000000000
    Dec 2 16:31:59 	openvpn 	21723 	resolve_in_advance = DISABLED
    Dec 2 16:31:59 	openvpn 	21723 	username = '[UNDEF]'
    Dec 2 16:31:59 	openvpn 	21723 	groupname = '[UNDEF]'
    Dec 2 16:31:59 	openvpn 	21723 	chroot_dir = '[UNDEF]'
    Dec 2 16:31:59 	openvpn 	21723 	cd_dir = '[UNDEF]'
    Dec 2 16:31:59 	openvpn 	21723 	writepid = '/var/run/openvpn_client1.pid'
    Dec 2 16:31:59 	openvpn 	21723 	up_script = '/usr/local/sbin/ovpn-linkup'
    Dec 2 16:31:59 	openvpn 	21723 	down_script = '/usr/local/sbin/ovpn-linkdown'
    Dec 2 16:31:59 	openvpn 	21723 	down_pre = DISABLED
    Dec 2 16:31:59 	openvpn 	21723 	up_restart = DISABLED
    Dec 2 16:31:59 	openvpn 	21723 	up_delay = DISABLED
    Dec 2 16:31:59 	openvpn 	21723 	daemon = ENABLED
    Dec 2 16:31:59 	openvpn 	21723 	inetd = 0
    Dec 2 16:31:59 	openvpn 	21723 	log = DISABLED
    Dec 2 16:31:59 	openvpn 	21723 	suppress_timestamps = DISABLED
    Dec 2 16:31:59 	openvpn 	21723 	machine_readable_output = DISABLED
    Dec 2 16:31:59 	openvpn 	21723 	nice = 0
    Dec 2 16:31:59 	openvpn 	21723 	verbosity = 4
    Dec 2 16:31:59 	openvpn 	21723 	mute = 0
    Dec 2 16:31:59 	openvpn 	21723 	gremlin = 0
    Dec 2 16:31:59 	openvpn 	21723 	status_file = '[UNDEF]'
    Dec 2 16:31:59 	openvpn 	21723 	status_file_version = 1
    Dec 2 16:31:59 	openvpn 	21723 	status_file_update_freq = 60
    Dec 2 16:31:59 	openvpn 	21723 	occ = ENABLED
    Dec 2 16:31:59 	openvpn 	21723 	rcvbuf = 0
    Dec 2 16:31:59 	openvpn 	21723 	sndbuf = 0
    Dec 2 16:31:59 	openvpn 	21723 	sockflags = 1
    Dec 2 16:31:59 	openvpn 	21723 	fast_io = DISABLED
    Dec 2 16:31:59 	openvpn 	21723 	comp.alg = 1
    Dec 2 16:31:59 	openvpn 	21723 	comp.flags = 0
    Dec 2 16:31:59 	openvpn 	21723 	route_script = '[UNDEF]'
    Dec 2 16:31:59 	openvpn 	21723 	route_default_gateway = '[UNDEF]'
    Dec 2 16:31:59 	openvpn 	21723 	route_default_metric = 0
    Dec 2 16:31:59 	openvpn 	21723 	route_noexec = DISABLED
    Dec 2 16:31:59 	openvpn 	21723 	route_delay = 0
    Dec 2 16:31:59 	openvpn 	21723 	route_delay_window = 30
    Dec 2 16:31:59 	openvpn 	21723 	route_delay_defined = DISABLED
    Dec 2 16:31:59 	openvpn 	21723 	route_nopull = DISABLED
    Dec 2 16:31:59 	openvpn 	21723 	route_gateway_via_dhcp = DISABLED
    Dec 2 16:31:59 	openvpn 	21723 	allow_pull_fqdn = DISABLED
    Dec 2 16:31:59 	openvpn 	21723 	[redirect_default_gateway local=0]
    Dec 2 16:31:59 	openvpn 	21723 	management_addr = '/var/etc/openvpn/client1.sock'
    Dec 2 16:31:59 	openvpn 	21723 	management_port = 'unix'
    Dec 2 16:31:59 	openvpn 	21723 	management_user_pass = '[UNDEF]'
    Dec 2 16:31:59 	openvpn 	21723 	management_log_history_cache = 250
    Dec 2 16:31:59 	openvpn 	21723 	management_echo_buffer_size = 100
    Dec 2 16:31:59 	openvpn 	21723 	management_write_peer_info_file = '[UNDEF]'
    Dec 2 16:31:59 	openvpn 	21723 	management_client_user = '[UNDEF]'
    Dec 2 16:31:59 	openvpn 	21723 	management_client_group = '[UNDEF]'
    Dec 2 16:31:59 	openvpn 	21723 	management_flags = 256
    Dec 2 16:31:59 	openvpn 	21723 	shared_secret_file = '[UNDEF]'
    Dec 2 16:31:59 	openvpn 	21723 	key_direction = not set
    Dec 2 16:31:59 	openvpn 	21723 	ciphername = 'AES-256-OFB'
    Dec 2 16:31:59 	openvpn 	21723 	ncp_enabled = DISABLED
    Dec 2 16:31:59 	openvpn 	21723 	ncp_ciphers = 'AES-256-GCM:AES-128-GCM'
    Dec 2 16:31:59 	openvpn 	21723 	authname = 'SHA256'
    Dec 2 16:31:59 	openvpn 	21723 	prng_hash = 'SHA1'
    Dec 2 16:31:59 	openvpn 	21723 	prng_nonce_secret_len = 16
    Dec 2 16:31:59 	openvpn 	21723 	keysize = 0
    Dec 2 16:31:59 	openvpn 	21723 	engine = ENABLED
    Dec 2 16:31:59 	openvpn 	21723 	replay = ENABLED
    Dec 2 16:31:59 	openvpn 	21723 	mute_replay_warnings = DISABLED
    Dec 2 16:31:59 	openvpn 	21723 	replay_window = 64
    Dec 2 16:31:59 	openvpn 	21723 	replay_time = 15
    Dec 2 16:31:59 	openvpn 	21723 	packet_id_file = '[UNDEF]'
    Dec 2 16:31:59 	openvpn 	21723 	use_iv = ENABLED
    Dec 2 16:31:59 	openvpn 	21723 	test_crypto = DISABLED
    Dec 2 16:31:59 	openvpn 	21723 	tls_server = DISABLED
    Dec 2 16:31:59 	openvpn 	21723 	tls_client = ENABLED
    Dec 2 16:31:59 	openvpn 	21723 	key_method = 2
    Dec 2 16:31:59 	openvpn 	21723 	ca_file = '/var/etc/openvpn/client1.ca'
    Dec 2 16:31:59 	openvpn 	21723 	ca_path = '[UNDEF]'
    Dec 2 16:31:59 	openvpn 	21723 	dh_file = '[UNDEF]'
    Dec 2 16:31:59 	openvpn 	21723 	cert_file = '/var/etc/openvpn/client1.cert'
    Dec 2 16:31:59 	openvpn 	21723 	extra_certs_file = '[UNDEF]'
    Dec 2 16:31:59 	openvpn 	21723 	priv_key_file = '/var/etc/openvpn/client1.key'
    Dec 2 16:31:59 	openvpn 	21723 	pkcs12_file = '[UNDEF]'
    Dec 2 16:31:59 	openvpn 	21723 	cipher_list = '[UNDEF]'
    Dec 2 16:31:59 	openvpn 	21723 	tls_cert_profile = '[UNDEF]'
    Dec 2 16:31:59 	openvpn 	21723 	tls_verify = '[UNDEF]'
    Dec 2 16:31:59 	openvpn 	21723 	tls_export_cert = '[UNDEF]'
    Dec 2 16:31:59 	openvpn 	21723 	verify_x509_type = 0
    Dec 2 16:31:59 	openvpn 	21723 	verify_x509_name = '[UNDEF]'
    Dec 2 16:31:59 	openvpn 	21723 	crl_file = '[UNDEF]'
    Dec 2 16:31:59 	openvpn 	21723 	ns_cert_type = 0
    Dec 2 16:31:59 	openvpn 	21723 	remote_cert_ku[i] = 0
    Dec 2 16:31:59 	openvpn 	21723 	remote_cert_ku[i] = 0
    Dec 2 16:31:59 	openvpn 	21723 	remote_cert_ku[i] = 0
    Dec 2 16:31:59 	openvpn 	21723 	remote_cert_ku[i] = 0
    Dec 2 16:31:59 	openvpn 	21723 	remote_cert_ku[i] = 0
    Dec 2 16:31:59 	openvpn 	21723 	remote_cert_ku[i] = 0
    Dec 2 16:31:59 	openvpn 	21723 	remote_cert_ku[i] = 0
    Dec 2 16:31:59 	openvpn 	21723 	remote_cert_ku[i] = 0
    Dec 2 16:31:59 	openvpn 	21723 	remote_cert_ku[i] = 0
    Dec 2 16:31:59 	openvpn 	21723 	remote_cert_ku[i] = 0
    Dec 2 16:31:59 	openvpn 	21723 	remote_cert_ku[i] = 0
    Dec 2 16:31:59 	openvpn 	21723 	remote_cert_ku[i] = 0
    Dec 2 16:31:59 	openvpn 	21723 	remote_cert_ku[i] = 0
    Dec 2 16:31:59 	openvpn 	21723 	remote_cert_ku[i] = 0
    Dec 2 16:31:59 	openvpn 	21723 	remote_cert_ku[i] = 0
    Dec 2 16:31:59 	openvpn 	21723 	remote_cert_ku[i] = 0
    Dec 2 16:31:59 	openvpn 	21723 	remote_cert_eku = '[UNDEF]'
    Dec 2 16:31:59 	openvpn 	21723 	ssl_flags = 0
    Dec 2 16:31:59 	openvpn 	21723 	tls_timeout = 2
    Dec 2 16:31:59 	openvpn 	21723 	renegotiate_bytes = -1
    Dec 2 16:31:59 	openvpn 	21723 	renegotiate_packets = 0
    Dec 2 16:31:59 	openvpn 	21723 	renegotiate_seconds = 3600
    Dec 2 16:31:59 	openvpn 	21723 	handshake_window = 60
    Dec 2 16:31:59 	openvpn 	21723 	transition_window = 3600
    Dec 2 16:31:59 	openvpn 	21723 	single_session = DISABLED
    Dec 2 16:31:59 	openvpn 	21723 	push_peer_info = DISABLED
    Dec 2 16:31:59 	openvpn 	21723 	tls_exit = DISABLED
    Dec 2 16:31:59 	openvpn 	21723 	tls_auth_file = '[UNDEF]'
    Dec 2 16:31:59 	openvpn 	21723 	tls_crypt_file = '/var/etc/openvpn/client1.tls-crypt'
    Dec 2 16:31:59 	openvpn 	21723 	server_network = 0.0.0.0
    Dec 2 16:31:59 	openvpn 	21723 	server_netmask = 0.0.0.0
    Dec 2 16:31:59 	openvpn 	21723 	server_network_ipv6 = ::
    Dec 2 16:31:59 	openvpn 	21723 	server_netbits_ipv6 = 0
    Dec 2 16:31:59 	openvpn 	21723 	server_bridge_ip = 0.0.0.0
    Dec 2 16:31:59 	openvpn 	21723 	server_bridge_netmask = 0.0.0.0
    Dec 2 16:31:59 	openvpn 	21723 	server_bridge_pool_start = 0.0.0.0
    Dec 2 16:31:59 	openvpn 	21723 	server_bridge_pool_end = 0.0.0.0
    Dec 2 16:31:59 	openvpn 	21723 	ifconfig_pool_defined = DISABLED
    Dec 2 16:31:59 	openvpn 	21723 	ifconfig_pool_start = 0.0.0.0
    Dec 2 16:31:59 	openvpn 	21723 	ifconfig_pool_end = 0.0.0.0
    Dec 2 16:31:59 	openvpn 	21723 	ifconfig_pool_netmask = 0.0.0.0
    Dec 2 16:31:59 	openvpn 	21723 	ifconfig_pool_persist_filename = '[UNDEF]'
    Dec 2 16:31:59 	openvpn 	21723 	ifconfig_pool_persist_refresh_freq = 600
    Dec 2 16:31:59 	openvpn 	21723 	ifconfig_ipv6_pool_defined = DISABLED
    Dec 2 16:31:59 	openvpn 	21723 	ifconfig_ipv6_pool_base = ::
    Dec 2 16:31:59 	openvpn 	21723 	ifconfig_ipv6_pool_netbits = 0
    Dec 2 16:31:59 	openvpn 	21723 	n_bcast_buf = 256
    Dec 2 16:31:59 	openvpn 	21723 	tcp_queue_limit = 64
    Dec 2 16:31:59 	openvpn 	21723 	real_hash_size = 256
    Dec 2 16:31:59 	openvpn 	21723 	virtual_hash_size = 256
    Dec 2 16:31:59 	openvpn 	21723 	client_connect_script = '[UNDEF]'
    Dec 2 16:31:59 	openvpn 	21723 	learn_address_script = '[UNDEF]'
    Dec 2 16:31:59 	openvpn 	21723 	client_disconnect_script = '[UNDEF]'
    Dec 2 16:31:59 	openvpn 	21723 	client_config_dir = '[UNDEF]'
    Dec 2 16:31:59 	openvpn 	21723 	ccd_exclusive = DISABLED
    Dec 2 16:31:59 	openvpn 	21723 	tmp_dir = '/tmp'
    Dec 2 16:31:59 	openvpn 	21723 	push_ifconfig_defined = DISABLED
    Dec 2 16:31:59 	openvpn 	21723 	push_ifconfig_local = 0.0.0.0
    Dec 2 16:31:59 	openvpn 	21723 	push_ifconfig_remote_netmask = 0.0.0.0
    Dec 2 16:31:59 	openvpn 	21723 	push_ifconfig_ipv6_defined = DISABLED
    Dec 2 16:31:59 	openvpn 	21723 	push_ifconfig_ipv6_local = ::/0
    Dec 2 16:31:59 	openvpn 	21723 	push_ifconfig_ipv6_remote = ::
    Dec 2 16:31:59 	openvpn 	21723 	enable_c2c = DISABLED
    Dec 2 16:31:59 	openvpn 	21723 	duplicate_cn = DISABLED
    Dec 2 16:31:59 	openvpn 	21723 	cf_max = 0
    Dec 2 16:31:59 	openvpn 	21723 	cf_per = 0
    Dec 2 16:31:59 	openvpn 	21723 	max_clients = 1024
    Dec 2 16:31:59 	openvpn 	21723 	max_routes_per_client = 256
    Dec 2 16:31:59 	openvpn 	21723 	auth_user_pass_verify_script = '[UNDEF]'
    Dec 2 16:31:59 	openvpn 	21723 	auth_user_pass_verify_script_via_file = DISABLED
    Dec 2 16:31:59 	openvpn 	21723 	auth_token_generate = DISABLED
    Dec 2 16:31:59 	openvpn 	21723 	auth_token_lifetime = 0
    Dec 2 16:31:59 	openvpn 	21723 	port_share_host = '[UNDEF]'
    Dec 2 16:31:59 	openvpn 	21723 	port_share_port = '[UNDEF]'
    Dec 2 16:31:59 	openvpn 	21723 	client = ENABLED
    Dec 2 16:31:59 	openvpn 	21723 	pull = ENABLED
    Dec 2 16:31:59 	openvpn 	21723 	auth_user_pass_file = '[UNDEF]'
    Dec 2 16:31:59 	openvpn 	21723 	OpenVPN 2.4.6 amd64-portbld-freebsd11.2 [SSL (OpenSSL)] [LZO] [LZ4] [MH/RECVDA] [AEAD] built on Sep 4 2018
    Dec 2 16:31:59 	openvpn 	21723 	library versions: OpenSSL 1.0.2o-freebsd 27 Mar 2018, LZO 2.10
    Dec 2 16:31:59 	openvpn 	21726 	MANAGEMENT: unix domain socket listening on /var/etc/openvpn/client1.sock
    Dec 2 16:31:59 	openvpn 	21726 	WARNING: using --pull/--client and --ifconfig together is probably not what you want
    Dec 2 16:31:59 	openvpn 	21726 	WARNING: No server certificate verification method has been enabled. See http://openvpn.net/howto.html#mitm for more info.
    Dec 2 16:31:59 	openvpn 	21726 	NOTE: the current --script-security setting may allow this configuration to call user-defined scripts
    Dec 2 16:31:59 	openvpn 	21726 	Initializing OpenSSL support for engine 'rdrand'
    Dec 2 16:31:59 	openvpn 	21726 	Outgoing Control Channel Encryption: Cipher 'AES-256-CTR' initialized with 256 bit key
    Dec 2 16:31:59 	openvpn 	21726 	Outgoing Control Channel Encryption: Using 256 bit message hash 'SHA256' for HMAC authentication
    Dec 2 16:31:59 	openvpn 	21726 	Incoming Control Channel Encryption: Cipher 'AES-256-CTR' initialized with 256 bit key
    Dec 2 16:31:59 	openvpn 	21726 	Incoming Control Channel Encryption: Using 256 bit message hash 'SHA256' for HMAC authentication
    Dec 2 16:31:59 	openvpn 	21726 	Control Channel MTU parms [ L:1622 D:1156 EF:94 EB:0 ET:0 EL:3 ]
    Dec 2 16:31:59 	openvpn 	21726 	Data Channel MTU parms [ L:1622 D:1450 EF:122 EB:406 ET:0 EL:3 ]
    Dec 2 16:31:59 	openvpn 	21726 	Local Options String (VER=V4): 'V4,dev-type tun,link-mtu 1574,tun-mtu 1500,proto UDPv4,comp-lzo,cipher AES-256-OFB,auth SHA256,keysize 256,key-method 2,tls-client'
    Dec 2 16:31:59 	openvpn 	21726 	Expected Remote Options String (VER=V4): 'V4,dev-type tun,link-mtu 1574,tun-mtu 1500,proto UDPv4,comp-lzo,cipher AES-256-OFB,auth SHA256,keysize 256,key-method 2,tls-server'
    Dec 2 16:31:59 	openvpn 	21726 	TCP/UDP: Preserving recently used remote address: [AF_INET]99.99.99.99:1194
    Dec 2 16:31:59 	openvpn 	21726 	Socket Buffers: R=[42080->42080] S=[57344->57344]
    Dec 2 16:31:59 	openvpn 	21726 	UDP link local (bound): [AF_INET][undef]:0
    Dec 2 16:31:59 	openvpn 	21726 	UDP link remote: [AF_INET]99.99.99.99:1194
    Dec 2 16:32:00 	openvpn 	21726 	TLS: Initial packet from [AF_INET]99.99.99.99:1194 (via [AF_INET]22.22.22.22%), sid=5dc73231 7bb48237
    Dec 2 16:32:00 	openvpn 	21726 	VERIFY OK: depth=1, CN=internal-ca
    Dec 2 16:32:00 	openvpn 	21726 	VERIFY OK: depth=0, CN=common-name
    Dec 2 16:32:00 	openvpn 	21726 	Control Channel: TLSv1.2, cipher TLSv1/SSLv3 ECDHE-RSA-AES256-GCM-SHA384, 2048 bit RSA
    Dec 2 16:32:00 	openvpn 	21726 	[common-name] Peer Connection Initiated with [AF_INET]99.99.99.99:1194 (via [AF_INET]22.22.22.22%)
    Dec 2 16:32:01 	openvpn 	21726 	SENT CONTROL [common-name]: 'PUSH_REQUEST' (status=1)
    Dec 2 16:32:01 	openvpn 	21726 	Key [AF_INET]99.99.99.99:1194 (via [AF_INET]22.22.22.22%) [0] not initialized (yet), dropping packet.
    Dec 2 16:32:01 	openvpn 	21726 	PUSH: Received control message: 'PUSH_REPLY,redirect-gateway def1,route-gateway 10.0.0.1,topology subnet,ping 1,ping-restart 2,ifconfig 10.0.0.2 255.255.255.0,peer-id 0'
    Dec 2 16:32:01 	openvpn 	21726 	OPTIONS IMPORT: timers and/or timeouts modified
    Dec 2 16:32:01 	openvpn 	21726 	OPTIONS IMPORT: --ifconfig/up options modified
    Dec 2 16:32:01 	openvpn 	21726 	OPTIONS IMPORT: route options modified
    Dec 2 16:32:01 	openvpn 	21726 	OPTIONS IMPORT: route-related options modified
    Dec 2 16:32:01 	openvpn 	21726 	OPTIONS IMPORT: peer-id set
    Dec 2 16:32:01 	openvpn 	21726 	OPTIONS IMPORT: adjusting link_mtu to 1625
    Dec 2 16:32:01 	openvpn 	21726 	Data Channel MTU parms [ L:1577 D:1450 EF:77 EB:406 ET:0 EL:3 ]
    Dec 2 16:32:01 	openvpn 	21726 	Outgoing Data Channel: Cipher 'AES-256-OFB' initialized with 256 bit key
    Dec 2 16:32:01 	openvpn 	21726 	Outgoing Data Channel: Using 256 bit message hash 'SHA256' for HMAC authentication
    Dec 2 16:32:01 	openvpn 	21726 	Incoming Data Channel: Cipher 'AES-256-OFB' initialized with 256 bit key
    Dec 2 16:32:01 	openvpn 	21726 	Incoming Data Channel: Using 256 bit message hash 'SHA256' for HMAC authentication
    Dec 2 16:32:01 	openvpn 	21726 	ROUTE_GATEWAY 10.64.64.0/255.255.255.255 IFACE=ppp0 HWADDR=00:00:00:00:00:00
    Dec 2 16:32:01 	openvpn 	21726 	TUN/TAP device ovpnc1 exists previously, keep at program end
    Dec 2 16:32:01 	openvpn 	21726 	TUN/TAP device /dev/tun1 opened
    Dec 2 16:32:01 	openvpn 	21726 	do_ifconfig, tt->did_ifconfig_ipv6_setup=0
    Dec 2 16:32:01 	openvpn 	21726 	/sbin/ifconfig ovpnc1 10.0.0.2 10.0.0.1 mtu 1500 netmask 255.255.255.0 up
    Dec 2 16:32:01 	openvpn 	21726 	/sbin/route add -net 10.0.0.0 10.0.0.1 255.255.255.0
    Dec 2 16:32:01 	openvpn 	21726 	/usr/local/sbin/ovpn-linkup ovpnc1 1500 1577 10.0.0.2 255.255.255.0 init
    Dec 2 16:32:01 	openvpn 	21726 	/sbin/route add -net 99.99.99.99 10.64.64.0 255.255.255.255
    Dec 2 16:32:01 	openvpn 	21726 	/sbin/route add -net 0.0.0.0 10.0.0.1 128.0.0.0
    Dec 2 16:32:01 	openvpn 	21726 	/sbin/route add -net 128.0.0.0 10.0.0.1 128.0.0.0
    Dec 2 16:32:01 	openvpn 	21726 	WARNING: this configuration may cache passwords in memory -- use the auth-nocache option to prevent this
    Dec 2 16:32:01 	openvpn 	21726 	Initialization Sequence Completed 
    


  • As recommended in Advanced OpenVPN on pfSense 2.4 and Advanced OpenVPN Concepts on pfSense i've switched from my above setup to Quagga/OSPF. I am able to replicate the failovering results and match the failovering times i got off the above setup, but my TCP is still not surviving an underlying connection switch...

    Please help.



  • Sad to report back that a switch to OpenWRT/mwan3/WireGuard did the trick. pfSense needs WireGuard bad AF :|


Log in to reply