Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    OpenConnect + split routing

    Scheduled Pinned Locked Moved OpenVPN
    3 Posts 3 Posters 3.8k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • R
      reinderien
      last edited by

      I am attempting to achieve the following setup:

      • All traffic, by default, goes through WAN
      • OpenConnect VPN is always up
      • Traffic only routes to VPN interface on specified nets, the most important being 10.0.0.0/8
      • There are two sets of DNS servers: for WAN and VPN

      Reading a handful of guides, I've done the following:

      • Install Shellcmd and run an earlyshellcmd,```
        /sbin/ifconfig tun create; /sbin/ifconfig tun0 name vpn0
      - Modify vpnc-script to rename tun0 to vpn0 to get past the pfSense GUI hiding tun* interfaces
      - Set the WAN-specific DNS servers to use the WAN interface, but leave the interface field blank for VPN because it's "directly connected"
      - VPN has an assigned interface, and is set to static IPv4 with custom gateway, static IPv6 with no gateway
      - The VPN GW is not set as default. The WAN GW is set as default.
      - There are several dozen static routes set to be routed through the VPN GW.
      
      I haven't written a cron job to start openconnect yet; for now I'm starting it manually.
      
      An example run looks like:
      
      

      echo "$password" |
      openconnect
        --pid-file=/var/run/openconnect.pid
        --non-inter
        --user="$username"
        --passwd-on-stdin
        --cafile=/usr/share/openconnect/myserver.pem
        --interface="$tunif"
        --script=/usr/share/openconnect/vpnc-script-latest
        --timestamp
        --no-proxy
        --pfs
        --reconnect-timeout 60
        mycorpvpn.com

      [2017-08-03 15:42:40] POST https://mycorpvpn.com/
      ...

      • env
      • sort
        BLOCKSIZE=K
        CISCO_CSTP_OPTIONS=X-CSTP-Version=1
        CISCO_DEF_DOMAIN=mycorpvpn.com
        CISCO_PROXY_PAC=http://wpad/wpad.dat
        CLICOLOR=true
        FTP_PASSIVE_MODE=YES
        GROUP=wheel
        HOME=/root
        HOST=puffball.house.lan
        HOSTTYPE=FreeBSD
        INTERNAL_IP4_ADDRESS=10.65.164.129
        INTERNAL_IP4_DNS=10.209.76.198 10.209.76.197
        INTERNAL_IP4_MTU=1300
        INTERNAL_IP4_NETADDR=10.65.160.0
        INTERNAL_IP4_NETMASK=255.255.224.0
        INTERNAL_IP4_NETMASKLEN=19
        INTERNAL_IP6_ADDRESS=2606:b400:2050:24:8000::8d
        INTERNAL_IP6_NETMASK=2606:b400:2050:24:8000::8d/64
        LOGNAME=admin
        LSCOLORS=exfxcxdxbxegedabagacad
        MACHTYPE=x86_64
        MAIL=/var/mail/admin
        OSTYPE=FreeBSD
        PATH=/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/sbin:/usr/local/bin:/root/bin
        PWD=/usr/share/openconnect
        REMOTEHOST=192.168.1.100
        SHELL=/etc/rc.initial
        SHLVL=1
        SSH_CLIENT=192.168.1.100 55906 22
        SSH_CONNECTION=192.168.1.100 55906 192.168.1.1 22
        SSH_TTY=/dev/pts/0
        TERM=xterm-256color
        USER=admin
        VENDOR=amd
        VPNGATEWAY=129.xxx.xxx.247
        X-CSTP-Address-IP6=2606:b400:2050:24:8000::8d/64
        X-CSTP-Address=10.65.164.129
        X-CSTP-Client-Bypass-Protocol=false
        X-CSTP-DNS=10.209.76.197
        X-CSTP-DNS=10.209.76.198
        X-CSTP-DPD=30
        X-CSTP-Default-Domain=mycorp.com
        X-CSTP-Disable-Always-On-VPN=false
        X-CSTP-Disconnected-Timeout=21600
        X-CSTP-Hostname=vpn.mycorp.com
        X-CSTP-Idle-Timeout=21600
        X-CSTP-Keep=true
        X-CSTP-Keepalive=20
        X-CSTP-Lease-Duration=86400
        X-CSTP-MSIE-Proxy-Lockdown=false
        X-CSTP-MSIE-Proxy-PAC-URL=http://wpad/wpad.dat
        X-CSTP-MTU=1300
        X-CSTP-Netmask=255.255.224.0
        X-CSTP-Post-Auth-XML=
        X-CSTP-Protocol=Copyright (c) 2004 Cisco Systems, Inc.
        X-CSTP-Quarantine=false
        X-CSTP-Routing-Filtering-Ignore=false
        X-CSTP-Session-Timeout=86400
        X-CSTP-Smartcard-Removal-Disconnect=true
        X-CSTP-TCP-Keepalive=true
        X-CSTP-Tunnel-All-DNS=false
        reason=pre-init
      • set -x
      • PATH=/sbin:/usr/sbin:/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/sbin:/usr/local/bin:/root/bin
      • uname -s
      • OS=FreeBSD
      • HOOKS_DIR=/etc/vpnc
      • DEFAULT_ROUTE_FILE=/var/run/vpnc/defaultroute
      • RESOLV_CONF_BACKUP=/var/run/vpnc/resolv.conf-backup
      • basename /usr/share/openconnect/vpnc-script-latest
      • SCRIPTNAME=vpnc-script-latest
      • [ ! -d /var/run/vpnc ]
      • mkdir -p /var/run/vpnc
      • [ -x /sbin/restorecon ]
      • which ip
      • grep ^/
      • IPROUTE=''
      • ifconfig --help
      • grep BusyBox
      • ifconfig_syntax_inet=inet
      • [ FreeBSD = Linux ]
      • ifconfig_syntax_ptp=''
      • route_syntax_gw=''
      • route_syntax_del=delete
      • route_syntax_netmask=-netmask
      • [ FreeBSD = SunOS ]
      • route_syntax_interface=''
      • ifconfig_syntax_ptpv6=''
      • [ -r /etc/openwrt_release ]
      • [ -x /usr/bin/busctl ]
      • [ -x /sbin/resolvconf ]
      • MODIFYRESOLVCONF=modify_resolvconf_manager
      • RESTORERESOLVCONF=restore_resolvconf_manager
      • [ -z '' ]
      • TUNDEV=tun0
      • OLDTUNDEV=tun0
      • TUNDEV=vpn0
      • [ -n '' ]
      • AF_INET=2
      • [ -z pre-init ]
      • run_hooks pre-init
      • HOOK=pre-init
      • [ -d /etc/vpnc/pre-init.d ]
      • do_pre_init
      • [ FreeBSD = Linux ]
      • [ FreeBSD = FreeBSD ]
      • kldstat -q -m if_tun
      • ifconfig tun0
        ifconfig: interface tun0 does not exist
      • ifconfig tun0 create
        ifconfig: SIOCIFCREATE2: File exists
      • ifconfig tun0 name vpn0
        ifconfig: interface tun0 does not exist
      • exit 0
        [2017-08-03 15:42:55] SIOCSIFMTU: Device not configured
      • env
      • sort
        BLOCKSIZE=K
        CISCO_CSTP_OPTIONS=X-CSTP-Version=1
        CISCO_DEF_DOMAIN=mycorp.com
        CISCO_PROXY_PAC=http://wpad/wpad.dat
        CLICOLOR=true
        FTP_PASSIVE_MODE=YES
        GROUP=wheel
        HOME=/root
        HOST=puffball.house.lan
        HOSTTYPE=FreeBSD
        INTERNAL_IP4_ADDRESS=10.65.164.129
        INTERNAL_IP4_DNS=10.209.76.198 10.209.76.197
        INTERNAL_IP4_MTU=1300
        INTERNAL_IP4_NETADDR=10.65.160.0
        INTERNAL_IP4_NETMASK=255.255.224.0
        INTERNAL_IP4_NETMASKLEN=19
        INTERNAL_IP6_ADDRESS=2606:b400:2050:24:8000::8d
        INTERNAL_IP6_NETMASK=2606:b400:2050:24:8000::8d/64
        LOGNAME=admin
        LSCOLORS=exfxcxdxbxegedabagacad
        MACHTYPE=x86_64
        MAIL=/var/mail/admin
        OSTYPE=FreeBSD
        PATH=/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/sbin:/usr/local/bin:/root/bin
        PWD=/usr/share/openconnect
        REMOTEHOST=192.168.1.100
        SHELL=/etc/rc.initial
        SHLVL=1
        SSH_CLIENT=192.168.1.100 55906 22
        SSH_CONNECTION=192.168.1.100 55906 192.168.1.1 22
        SSH_TTY=/dev/pts/0
        TERM=xterm-256color
        TUNDEV=tun0
        USER=admin
        VENDOR=amd
        VPNGATEWAY=129.xx.xx.247
        X-CSTP-Address-IP6=2606:b400:2050:24:8000::8d/64
        X-CSTP-Address=10.65.164.129
        X-CSTP-Client-Bypass-Protocol=false
        X-CSTP-DNS=10.209.76.197
        X-CSTP-DNS=10.209.76.198
        X-CSTP-DPD=30
        X-CSTP-Default-Domain=mycorp.com
        X-CSTP-Disable-Always-On-VPN=false
        X-CSTP-Disconnected-Timeout=21600
        X-CSTP-Hostname=vpn.mycorp.com
        X-CSTP-Idle-Timeout=21600
        X-CSTP-Keep=true
        X-CSTP-Keepalive=20
        X-CSTP-Lease-Duration=86400
        X-CSTP-MSIE-Proxy-Lockdown=false
        X-CSTP-MSIE-Proxy-PAC-URL=http://wpad/wpad.dat
        X-CSTP-MTU=1300
        X-CSTP-Netmask=255.255.224.0
        X-CSTP-Post-Auth-XML=
        X-CSTP-Protocol=Copyright (c) 2004 Cisco Systems, Inc.
        X-CSTP-Quarantine=false
        X-CSTP-Routing-Filtering-Ignore=false
        X-CSTP-Session-Timeout=86400
        X-CSTP-Smartcard-Removal-Disconnect=true
        X-CSTP-TCP-Keepalive=true
        X-CSTP-Tunnel-All-DNS=false
        reason=connect
      • set -x
      • PATH=/sbin:/usr/sbin:/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/sbin:/usr/local/bin:/root/bin
      • uname -s
      • OS=FreeBSD
      • HOOKS_DIR=/etc/vpnc
      • DEFAULT_ROUTE_FILE=/var/run/vpnc/defaultroute
      • RESOLV_CONF_BACKUP=/var/run/vpnc/resolv.conf-backup
      • basename /usr/share/openconnect/vpnc-script-latest
      • SCRIPTNAME=vpnc-script-latest
      • [ ! -d /var/run/vpnc ]
      • which ip
      • grep ^/
      • IPROUTE=''
      • ifconfig --help
      • grep BusyBox
      • ifconfig_syntax_inet=inet
      • [ FreeBSD = Linux ]
      • ifconfig_syntax_ptp=''
      • route_syntax_gw=''
      • route_syntax_del=delete
      • route_syntax_netmask=-netmask
      • [ FreeBSD = SunOS ]
      • route_syntax_interface=''
      • ifconfig_syntax_ptpv6=''
      • [ -r /etc/openwrt_release ]
      • [ -x /usr/bin/busctl ]
      • [ -x /sbin/resolvconf ]
      • MODIFYRESOLVCONF=modify_resolvconf_manager
      • RESTORERESOLVCONF=restore_resolvconf_manager
      • [ -z tun0 ]
      • OLDTUNDEV=tun0
      • TUNDEV=vpn0
      • [ -n '' ]
      • AF_INET=2
      • [ -z connect ]
      • run_hooks connect
      • HOOK=connect
      • [ -d /etc/vpnc/connect.d ]
      • do_connect
      • [ -n '' ]
      • set_vpngateway_route
      • get_default_gw
      • netstat -r -n
      • awk '/:/ { next; } /^(default|0.0.0.0)/ { print $2; }'
      • route add -host 129.xx.xx.247 192.168.0.1
        add host 129.xx.xx.247: gateway 192.168.0.1
      • do_ifconfig
      • [ -n 1300 ]
      • MTU=1300
      • [ -z 1300 ]
      • [ -n '' ]
      • ifconfig vpn0 inet 10.65.164.129 10.65.164.129 netmask 255.255.255.255 mtu 1300 up
      • [ -n 255.255.224.0 ]
      • set_network_route 10.65.160.0 255.255.224.0 19
      • NETWORK=10.65.160.0
      • NETMASK=255.255.224.0
      • NETMASKLEN=19
      • del_network_route 10.65.160.0 255.255.224.0 19
      • NETWORK=10.65.160.0
      • NETMASK=255.255.224.0
      • NETMASKLEN=19
      • route delete -net 10.65.160.0 -netmask 255.255.224.0 10.65.164.129
        route: writing to routing socket: No such process
        delete net 10.65.160.0: gateway 10.65.164.129 fib 0: not in table
      • route add -net 10.65.160.0 -netmask 255.255.224.0 10.65.164.129
        add net 10.65.160.0: gateway 10.65.164.129
      • [ -n 2606:b400:2050:24:8000::8d ]
      • [ -z 2606:b400:2050:24:8000::8d/64 ]
      • [ -n 2606:b400:2050:24:8000::8d/64 ]
      • [ -n '' ]
      • ifconfig vpn0 inet6 2606:b400:2050:24:8000::8d/64 mtu 1300 up
      • [ -n '' ]
      • [ -n 10.65.164.129 ]
      • set_default_route
      • get_default_gw
      • netstat -r -n
      • awk '/:/ { next; } /^(default|0.0.0.0)/ { print $2; }'
      • DEFAULTGW=192.168.0.1
      • echo 192.168.0.1
      • route delete default 192.168.0.1
        delete net default: gateway 192.168.0.1
      • route add default 10.65.164.129
        add net default: gateway 10.65.164.129
      • [ -n '' ]
      • [ -n 2606:b400:2050:24:8000::8d/64 -o -n 2606:b400:2050:24:8000::8d ]
      • set_ipv6_default_route
      • route add -inet6 default 2606:b400:2050:24:8000::8d
        route: writing to routing socket: File exists
        add net default: gateway 2606:b400:2050:24:8000::8d fib 0: route already in table
      • [ -n '10.209.76.198 10.209.76.197' ]
      • modify_resolvconf_manager
      • NEW_RESOLVCONF=''
      • NEW_RESOLVCONF='
        nameserver 10.209.76.198'
      • NEW_RESOLVCONF='
        nameserver 10.209.76.198
        nameserver 10.209.76.197'
      • [ -n mycorp.com ]
      • NEW_RESOLVCONF='
        nameserver 10.209.76.198
        nameserver 10.209.76.197
        domain mycorp.com'
      • echo '
        nameserver 10.209.76.198
        nameserver 10.209.76.197
        domain mycorp.com'
      • /sbin/resolvconf -a vpn0
        cp: /dev/null.bak: Operation not supported
      • run_hooks post-connect
      • HOOK=post-connect
      • [ -d /etc/vpnc/post-connect.d ]
      • exit 0
        [2017-08-03 15:42:55] Connected tun0 as 10.65.164.129 + 2606:b400:2050:24:8000::8d/64, using SSL
        [2017-08-03 15:42:56] Established DTLS connection (using OpenSSL). Ciphersuite AES256-SHA.
        ^C[2017-08-03 15:46:27] Send BYE packet: Aborted by caller
      • env
      • sort
        BLOCKSIZE=K
        CISCO_CSTP_OPTIONS=X-CSTP-Version=1
        CISCO_DEF_DOMAIN=mycorp.com
        CISCO_PROXY_PAC=http://wpad/wpad.dat
        CLICOLOR=true
        FTP_PASSIVE_MODE=YES
        GROUP=wheel
        HOME=/root
        HOST=puffball.house.lan
        HOSTTYPE=FreeBSD
        INTERNAL_IP4_ADDRESS=10.65.164.129
        INTERNAL_IP4_DNS=10.209.76.198 10.209.76.197
        INTERNAL_IP4_MTU=1300
        INTERNAL_IP4_NETADDR=10.65.160.0
        INTERNAL_IP4_NETMASK=255.255.224.0
        INTERNAL_IP4_NETMASKLEN=19
        INTERNAL_IP6_ADDRESS=2606:b400:2050:24:8000::8d
        INTERNAL_IP6_NETMASK=2606:b400:2050:24:8000::8d/64
        LOGNAME=admin
        LSCOLORS=exfxcxdxbxegedabagacad
        MACHTYPE=x86_64
        MAIL=/var/mail/admin
        OSTYPE=FreeBSD
        PATH=/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/sbin:/usr/local/bin:/root/bin
        PWD=/usr/share/openconnect
        REMOTEHOST=192.168.1.100
        SHELL=/etc/rc.initial
        SHLVL=1
        SSH_CLIENT=192.168.1.100 55906 22
        SSH_CONNECTION=192.168.1.100 55906 192.168.1.1 22
        SSH_TTY=/dev/pts/0
        TERM=xterm-256color
        TUNDEV=tun0
        USER=admin
        VENDOR=amd
        VPNGATEWAY=129.xx.xx.247
        X-CSTP-Address-IP6=2606:b400:2050:24:8000::8d/64
        X-CSTP-Address=10.65.164.129
        X-CSTP-Client-Bypass-Protocol=false
        X-CSTP-DNS=10.209.76.197
        X-CSTP-DNS=10.209.76.198
        X-CSTP-DPD=30
        X-CSTP-Default-Domain=mycorp.com
        X-CSTP-Disable-Always-On-VPN=false
        X-CSTP-Disconnected-Timeout=21600
        X-CSTP-Hostname=vpn.mycorp.com
        X-CSTP-Idle-Timeout=21600
        X-CSTP-Keep=true
        X-CSTP-Keepalive=20
        X-CSTP-Lease-Duration=86400
        X-CSTP-MSIE-Proxy-Lockdown=false
        X-CSTP-MSIE-Proxy-PAC-URL=http://wpad/wpad.dat
        X-CSTP-MTU=1300
        X-CSTP-Netmask=255.255.224.0
        X-CSTP-Post-Auth-XML=
        X-CSTP-Protocol=Copyright (c) 2004 Cisco Systems, Inc.
        X-CSTP-Quarantine=false
        X-CSTP-Routing-Filtering-Ignore=false
        X-CSTP-Session-Timeout=86400
        X-CSTP-Smartcard-Removal-Disconnect=true
        X-CSTP-TCP-Keepalive=true
        X-CSTP-Tunnel-All-DNS=false
        reason=disconnect
      • set -x
      • PATH=/sbin:/usr/sbin:/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/sbin:/usr/local/bin:/root/bin
      • uname -s
      • OS=FreeBSD
      • HOOKS_DIR=/etc/vpnc
      • DEFAULT_ROUTE_FILE=/var/run/vpnc/defaultroute
      • RESOLV_CONF_BACKUP=/var/run/vpnc/resolv.conf-backup
      • basename /usr/share/openconnect/vpnc-script-latest
      • SCRIPTNAME=vpnc-script-latest
      • [ ! -d /var/run/vpnc ]
      • which ip
      • grep ^/
      • IPROUTE=''
      • ifconfig --help
      • grep BusyBox
      • ifconfig_syntax_inet=inet
      • [ FreeBSD = Linux ]
      • ifconfig_syntax_ptp=''
      • route_syntax_gw=''
      • route_syntax_del=delete
      • route_syntax_netmask=-netmask
      • [ FreeBSD = SunOS ]
      • route_syntax_interface=''
      • ifconfig_syntax_ptpv6=''
      • [ -r /etc/openwrt_release ]
      • [ -x /usr/bin/busctl ]
      • [ -x /sbin/resolvconf ]
      • MODIFYRESOLVCONF=modify_resolvconf_manager
      • RESTORERESOLVCONF=restore_resolvconf_manager
      • [ -z tun0 ]
      • OLDTUNDEV=tun0
      • TUNDEV=vpn0
      • [ -n '' ]
      • AF_INET=2
      • [ -z disconnect ]
      • run_hooks disconnect
      • HOOK=disconnect
      • [ -d /etc/vpnc/disconnect.d ]
      • do_disconnect
      • [ -n '' ]
      • reset_default_route
      • [ -s /var/run/vpnc/defaultroute ]
      • get_default_gw
      • netstat -r -n
      • awk '/:/ { next; } /^(default|0.0.0.0)/ { print $2; }'
      • route delete default 10.65.164.129
        delete net default: gateway 10.65.164.129
      • cat /var/run/vpnc/defaultroute
      • route add default 192.168.0.1
        add net default: gateway 192.168.0.1
      • rm -f -- /var/run/vpnc/defaultroute
      • [ -n '' ]
      • [ -n 2606:b400:2050:24:8000::8d/64 -o -n 2606:b400:2050:24:8000::8d ]
      • reset_ipv6_default_route
      • route delete -inet6 default 2606:b400:2050:24:8000::8d
        delete net default: gateway 2606:b400:2050:24:8000::8d
      • :
      • del_vpngateway_route
      • get_default_gw
      • netstat -r -n
      • awk '/:/ { next; } /^(default|0.0.0.0)/ { print $2; }'
      • route delete -host 129.xx.xx.247 192.168.0.1
        delete host 129.xx.xx.247: gateway 192.168.0.1
      • [ -n '10.209.76.198 10.209.76.197' ]
      • restore_resolvconf_manager
      • /sbin/resolvconf -d vpn0
      • [ -n '' ]
      • [ -n 10.65.164.129 ]
      • ifconfig vpn0 0.0.0.0
        ifconfig: ioctl (SIOCAIFADDR): Destination address required
      • [ -n 2606:b400:2050:24:8000::8d ]
      • [ -z 2606:b400:2050:24:8000::8d/64 ]
      • [ -n 2606:b400:2050:24:8000::8d/64 ]
      • ifconfig vpn0 inet6 del 2606:b400:2050:24:8000::8d/64
        ifconfig: del: bad value
      • destroy_tun_device
      • run_hooks post-disconnect
      • HOOK=post-disconnect
      • ifconfig vpn0 destroy
      • [ -d /etc/vpnc/post-disconnect.d ]
      • exit 0
        [2017-08-03 15:46:28] User cancelled (SIGINT); exiting.
      
      Even though the VPN looks like it's brought up correctly according to openconnect, it's still fairly broken.
      
      I cannot resolve any of the hostnames on the far end using ping. If I attempt to use dig with the server set explicitly:
      
      

      dig @10.209.76.197 somehost.lan

      
      It successfully resolves and I can then ping the IP of somehost.lan. So what should I be doing to fix DNS?
      
      Additionally, netstat -nr is missing all of the static routes that I had set to be routed through the VPN. So I wonder why the static route config is getting ignored.
      1 Reply Last reply Reply Quote 0
      • S
        SpaceBass
        last edited by

        first a clarifying question or two….

        It sounds like you want clients to connect to a OpenVPN server. Is that accurate?
        If so, are the clients things like phones, computers, etc? Or is the client also a PF box?

        It also sounds like you want the clients to use their own wan connection unless trying to reach a host on the other end of the OpenVPN. Is that accurate?

        If so - that's basically how OpenVPN works without modification.

        So it looks like a route issue. I don't see (but only skimmed) any route statements in the logs. Are you declaring remote networks?

        1 Reply Last reply Reply Quote 0
        • DerelictD
          Derelict LAYER 8 Netgate
          last edited by

          How is the client supposed to know which DNS server to use?

          Before it knows the answer to the query it has no idea if the destination is out on the internet or over the VPN.

          You are probably best off sending the queries to the DNS server over the VPN and letting it return the appropriate answer.

          Chattanooga, Tennessee, USA
          A comprehensive network diagram is worth 10,000 words and 15 conference calls.
          DO NOT set a source address/port in a port forward or firewall rule unless you KNOW you need it!
          Do Not Chat For Help! NO_WAN_EGRESS(TM)

          1 Reply Last reply Reply Quote 0
          • First post
            Last post
          Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.