[SOLVED] Unable to reach pfsense or any computer on its subnet from VPN server

OpenVPN
Log in to reply
  • F

    UPDATE:  I solved this by using tap instead of tun.
    see below

    0
  • F

    see below

    0
  • johnpoz
    LAYER 8 Global Moderator

    How about some actual details of your setup??  For starters WTF would you be doing wan rules for a vpn client to ping stuff for???

    And your wan is rfc1918.. From your thead over at openvpn this seems to be 1 side is in the google compute engine, and the other is where exactly - where is pfsense running?

    If all you want is a site to site vpn, then look at the freaking docs..
    https://doc.pfsense.org/index.php/OpenVPN_Site_To_Site

    0
  • F

    see below

    0
  • F

    Per request:

    My setup:

    
GCE CentOS OVPN Server                                                                                        pfSense Router
eth0 10.250.0.2                                            Google Gateway                                   rl0 (LAN) 10.0.0.1 
10.250.255.255                                                                                                      10.0.0.255
tun0  10.254.254.1,2           <=VPN=>               ip addr:  10.250.0.1         <=WAN/VPN=>              ovpnc1 (tun) 10.254.254.5,6
Static Public IP (no if)                                                                                              sis0  Dynamic Public IP
No NAT                                                                                                                   NAT

    Firewall Rules on GCE Network allow all ports (1-65535) for tcp, udp, and icmp on the 10.250.0.0/16 subnet for all instances (there is only the server).

    SELinux is disabled on server.

    firewalld is disabled on server.

    IP Forwarding on Server:

    
vpn-server-1 etc]$ cat /proc/sys/net/ipv4/ip_forward
1

    Server Routing Table:

    
$ route
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
default         gateway         0.0.0.0         UG    100    0        0 eth0
10.0.0.0        10.254.254.2    255.255.255.0   UG    0      0        0 tun0
gateway         0.0.0.0         255.255.255.255 UH    100    0        0 eth0
ads-vpn-server- 0.0.0.0         255.255.255.255 UH    100    0        0 eth0
10.254.254.0    10.254.254.2    255.255.255.0   UG    0      0        0 tun0
10.254.254.2    0.0.0.0         255.255.255.255 UH    0      0        0 tun0
metadata.google gateway         255.255.255.255 UGH   100    0        0 eth0

    Server's Gateway's Routes

    
Name	                                 Destination IP ranges  	Priority	Instance tags	Next hop	                        Network

ads-??-vpn-route	                 10.0.0.0/24	                 500	         None	        10.250.0.2	                        Default

ads-vpn-server-1-tun-route	         10.254.254.0/24	         500	         None	        10.250.0.2	                        Default

default-route-0dbf2173481c8cf2	 10.250.0.0/16	               1000	         None	         Virtual network	                Default

default-route-6befe203e9e08025	   0.0.0.0/0	               1000	         None	         Default internet gateway	Default

    pfSense VPN Client's Route's

    
# netstat -r
Routing tables

Internet:
Destination                            Gateway                         Flags      Netif Expire
default                                  ool-45936001.dyn.o         UGS        sis0
10.0.0.0                                link#2                             U            rl0
adsllc--pfse                            link#2                            UHS         lo0
10.250.0.0                             10.254.254.5                 UGS      ovpnc1
10.254.254.0                         10.254.254.5                 UGS      ovpnc1
10.254.254.5                         link#7                            UH       ovpnc1
10.254.254.6                         link#7                           UHS         lo0
69.115.144.0/20                      link#1                            U          sis0
ool-45936d3c.dyn.o                link#1                           UHS         lo0
localhost                                link#5                           UH          lo0
vdnssec1.srv.prnyn                 00:0f:b5:8a:b4:76         UHS        sis0
vdnssec2.srv.prnyn                 00:0f:b5:8a:b4:76         UHS        sis0

    /etc/openvpn/server.conf

    
proto udp
dev tun
ca /etc/openvpn/keys/ca.crt
cert /etc/openvpn/keys/server.crt
key /etc/openvpn/keys/server.key  # This file should be kept secret
dh /etc/openvpn/keys/dh2048.pem
server 10.254.254.0 255.255.255.0
push "route 10.250.0.0 255.255.0.0"
client-config-dir ccd
route 10.0.0.0 255.255.255.0
client-to-client
keepalive 10 120
comp-lzo
persist-key
persist-tun
status openvpn-status.log
verb 5

    /etc/openvpn/ccd/client.conf

    
iroute 10.0.0.0 255.255.255.0

    /var/etc/openvpn/client1.conf (client config, autogenerated by GUI)

    
dev ovpnc1
verb 4
dev-type tun
dev-node /dev/tun1
writepid /var/run/openvpn_client1.pid
#user nobody
#group nobody
script-security 3
daemon
keepalive 10 60
ping-timer-rem
persist-tun
persist-key
proto udp
cipher BF-CBC
auth SHA1
up /usr/local/sbin/ovpn-linkup
down /usr/local/sbin/ovpn-linkdown
local 69.115.144.60
tls-client
client
lport 0
management /var/etc/openvpn/client1.sock unix
remote 104.196.144.148 1194
ifconfig 10.254.254.2 10.254.254.1
ca /var/etc/openvpn/client1.ca
cert /var/etc/openvpn/client1.cert
key /var/etc/openvpn/client1.key
comp-lzo adaptive
resolv-retry infinite

    From server console:

    
 sudo openvpn server.conf 
Mon Mar 21 02:05:07 2016 us=690291 Current Parameter Settings:
Mon Mar 21 02:05:07 2016 us=690330   config = 'server.conf'
Mon Mar 21 02:05:07 2016 us=690337   mode = 1
Mon Mar 21 02:05:07 2016 us=690342   persist_config = DISABLED
Mon Mar 21 02:05:07 2016 us=690346   persist_mode = 1
Mon Mar 21 02:05:07 2016 us=690351   show_ciphers = DISABLED
Mon Mar 21 02:05:07 2016 us=690355   show_digests = DISABLED
Mon Mar 21 02:05:07 2016 us=690359   show_engines = DISABLED
Mon Mar 21 02:05:07 2016 us=690363   genkey = DISABLED
Mon Mar 21 02:05:07 2016 us=690367   key_pass_file = '[UNDEF]'
Mon Mar 21 02:05:07 2016 us=690374   show_tls_ciphers = DISABLED
Mon Mar 21 02:05:07 2016 us=690378 Connection profiles [default]:
Mon Mar 21 02:05:07 2016 us=690383   proto = udp
Mon Mar 21 02:05:07 2016 us=690387   local = '[UNDEF]'
Mon Mar 21 02:05:07 2016 us=690391   local_port = 1194
Mon Mar 21 02:05:07 2016 us=690395   remote = '[UNDEF]'
Mon Mar 21 02:05:07 2016 us=690400   remote_port = 1194
Mon Mar 21 02:05:07 2016 us=690404   remote_float = DISABLED
Mon Mar 21 02:05:07 2016 us=690408   bind_defined = DISABLED
Mon Mar 21 02:05:07 2016 us=690412   bind_local = ENABLED
Mon Mar 21 02:05:07 2016 us=690416   connect_retry_seconds = 5
Mon Mar 21 02:05:07 2016 us=690421   connect_timeout = 10
Mon Mar 21 02:05:07 2016 us=690425   connect_retry_max = 0
Mon Mar 21 02:05:07 2016 us=690429   socks_proxy_server = '[UNDEF]'
Mon Mar 21 02:05:07 2016 us=690434   socks_proxy_port = 0
Mon Mar 21 02:05:07 2016 us=690438   socks_proxy_retry = DISABLED
Mon Mar 21 02:05:07 2016 us=690442   tun_mtu = 1500
Mon Mar 21 02:05:07 2016 us=690446   tun_mtu_defined = ENABLED
Mon Mar 21 02:05:07 2016 us=690451   link_mtu = 1500
Mon Mar 21 02:05:07 2016 us=690455   link_mtu_defined = DISABLED
Mon Mar 21 02:05:07 2016 us=690459   tun_mtu_extra = 0
Mon Mar 21 02:05:07 2016 us=690463   tun_mtu_extra_defined = DISABLED
Mon Mar 21 02:05:07 2016 us=690468   mtu_discover_type = -1
Mon Mar 21 02:05:07 2016 us=690472   fragment = 0
Mon Mar 21 02:05:07 2016 us=690476   mssfix = 1450
Mon Mar 21 02:05:07 2016 us=690480   explicit_exit_notification = 0
Mon Mar 21 02:05:07 2016 us=690485 Connection profiles END
Mon Mar 21 02:05:07 2016 us=690490   remote_random = DISABLED
Mon Mar 21 02:05:07 2016 us=690494   ipchange = '[UNDEF]'
Mon Mar 21 02:05:07 2016 us=690498   dev = 'tun'
Mon Mar 21 02:05:07 2016 us=690502   dev_type = '[UNDEF]'
Mon Mar 21 02:05:07 2016 us=690506   dev_node = '[UNDEF]'
Mon Mar 21 02:05:07 2016 us=690510   lladdr = '[UNDEF]'
Mon Mar 21 02:05:07 2016 us=690514   topology = 1
Mon Mar 21 02:05:07 2016 us=690518   tun_ipv6 = DISABLED
Mon Mar 21 02:05:07 2016 us=690522   ifconfig_local = '10.254.254.1'
Mon Mar 21 02:05:07 2016 us=690526   ifconfig_remote_netmask = '10.254.254.2'
Mon Mar 21 02:05:07 2016 us=690530   ifconfig_noexec = DISABLED
Mon Mar 21 02:05:07 2016 us=690534   ifconfig_nowarn = DISABLED
Mon Mar 21 02:05:07 2016 us=690538   ifconfig_ipv6_local = '[UNDEF]'
Mon Mar 21 02:05:07 2016 us=690543   ifconfig_ipv6_netbits = 0
Mon Mar 21 02:05:07 2016 us=690547   ifconfig_ipv6_remote = '[UNDEF]'
Mon Mar 21 02:05:07 2016 us=690551   shaper = 0
Mon Mar 21 02:05:07 2016 us=690555   mtu_test = 0
Mon Mar 21 02:05:07 2016 us=690559   mlock = DISABLED
Mon Mar 21 02:05:07 2016 us=690563   keepalive_ping = 10
Mon Mar 21 02:05:07 2016 us=690567   keepalive_timeout = 120
Mon Mar 21 02:05:07 2016 us=690571   inactivity_timeout = 0
Mon Mar 21 02:05:07 2016 us=690575   ping_send_timeout = 10
Mon Mar 21 02:05:07 2016 us=690579   ping_rec_timeout = 240
Mon Mar 21 02:05:07 2016 us=690583   ping_rec_timeout_action = 2
Mon Mar 21 02:05:07 2016 us=690587   ping_timer_remote = DISABLED
Mon Mar 21 02:05:07 2016 us=690591   remap_sigusr1 = 0
Mon Mar 21 02:05:07 2016 us=690595   persist_tun = ENABLED
Mon Mar 21 02:05:07 2016 us=690599   persist_local_ip = DISABLED
Mon Mar 21 02:05:07 2016 us=690603   persist_remote_ip = DISABLED
Mon Mar 21 02:05:07 2016 us=690607   persist_key = ENABLED
Mon Mar 21 02:05:07 2016 us=690612   passtos = DISABLED
Mon Mar 21 02:05:07 2016 us=690616   resolve_retry_seconds = 1000000000
Mon Mar 21 02:05:07 2016 us=690620   username = '[UNDEF]'
Mon Mar 21 02:05:07 2016 us=690624   groupname = '[UNDEF]'
Mon Mar 21 02:05:07 2016 us=690628   chroot_dir = '[UNDEF]'
Mon Mar 21 02:05:07 2016 us=690632   cd_dir = '[UNDEF]'
Mon Mar 21 02:05:07 2016 us=690636   writepid = '[UNDEF]'
Mon Mar 21 02:05:07 2016 us=690639   up_script = '[UNDEF]'
Mon Mar 21 02:05:07 2016 us=690643   down_script = '[UNDEF]'
Mon Mar 21 02:05:07 2016 us=690647   down_pre = DISABLED
Mon Mar 21 02:05:07 2016 us=693362   duplicate_cn = DISABLED
Mon Mar 21 02:05:07 2016 us=693366   cf_max = 0
Mon Mar 21 02:05:07 2016 us=693370   cf_per = 0
Mon Mar 21 02:05:07 2016 us=693374   max_clients = 1024
Mon Mar 21 02:05:07 2016 us=693378   max_routes_per_client = 256
Mon Mar 21 02:05:07 2016 us=693430   auth_user_pass_verify_script = '[UNDEF]'
Mon Mar 21 02:05:07 2016 us=693436   auth_user_pass_verify_script_via_file = DISABLED
Mon Mar 21 02:05:07 2016 us=693440   port_share_host = '[UNDEF]'
Mon Mar 21 02:05:07 2016 us=693445   port_share_port = 0
Mon Mar 21 02:05:07 2016 us=693449   client = DISABLED
Mon Mar 21 02:05:07 2016 us=693453   pull = DISABLED
Mon Mar 21 02:05:07 2016 us=693457   auth_user_pass_file = '[UNDEF]'
Mon Mar 21 02:05:07 2016 us=693476 OpenVPN 2.3.10 x86_64-redhat-linux-gnu [SSL (OpenSSL)] [LZO] [EPOLL] [PKCS11] [MH] [IPv6] built on Jan  4 2016
Mon Mar 21 02:05:07 2016 us=693485 library versions: OpenSSL 1.0.1e-fips 11 Feb 2013, LZO 2.06
Mon Mar 21 02:05:07 2016 us=699218 Diffie-Hellman initialized with 2048 bit key
Mon Mar 21 02:05:07 2016 us=699565 TLS-Auth MTU parms [ L:1542 D:1212 EF:38 EB:0 ET:0 EL:3 ]
Mon Mar 21 02:05:07 2016 us=699582 Socket Buffers: R=[212992->212992] S=[212992->212992]
Mon Mar 21 02:05:07 2016 us=699650 ROUTE_GATEWAY 10.250.0.1
Mon Mar 21 02:05:07 2016 us=699852 TUN/TAP device tun0 opened
Mon Mar 21 02:05:07 2016 us=699864 TUN/TAP TX queue length set to 100
Mon Mar 21 02:05:07 2016 us=699873 do_ifconfig, tt->ipv6=0, tt->did_ifconfig_ipv6_setup=0
Mon Mar 21 02:05:07 2016 us=699890 /usr/sbin/ip link set dev tun0 up mtu 1500
Mon Mar 21 02:05:07 2016 us=703282 /usr/sbin/ip addr add dev tun0 local 10.254.254.1 peer 10.254.254.2
Mon Mar 21 02:05:07 2016 us=704819 /usr/sbin/ip route add 10.0.0.0/24 via 10.254.254.2
Mon Mar 21 02:05:07 2016 us=710749 /usr/sbin/ip route add 10.254.254.0/24 via 10.254.254.2
Mon Mar 21 02:05:07 2016 us=712357 Data Channel MTU parms [ L:1542 D:1450 EF:42 EB:143 ET:0 EL:3 AF:3/1 ]
Mon Mar 21 02:05:07 2016 us=712377 UDPv4 link local (bound): [undef]
Mon Mar 21 02:05:07 2016 us=712383 UDPv4 link remote: [undef]
Mon Mar 21 02:05:07 2016 us=712391 MULTI: multi_init called, r=256 v=256
Mon Mar 21 02:05:07 2016 us=712435 IFCONFIG POOL: base=10.254.254.4 size=62, ipv6=0
Mon Mar 21 02:05:07 2016 us=712451 Initialization Sequence Completed
Mon Mar 21 02:05:10 2016 us=123321 MULTI: multi_create_instance called
Mon Mar 21 02:05:10 2016 us=123366 69.115.144.60:65005 Re-using SSL/TLS context
Mon Mar 21 02:05:10 2016 us=123394 69.115.144.60:65005 LZO compression initialized
Mon Mar 21 02:05:10 2016 us=123491 69.115.144.60:65005 Control Channel MTU parms [ L:1542 D:1212 EF:38 EB:0 ET:0 EL:3 ]
Mon Mar 21 02:05:10 2016 us=123500 69.115.144.60:65005 Data Channel MTU parms [ L:1542 D:1450 EF:42 EB:143 ET:0 EL:3 AF:3/1 ]
Mon Mar 21 02:05:10 2016 us=123522 69.115.144.60:65005 Local Options String: 'V4,dev-type tun,link-mtu 1542,tun-mtu 1500,proto UDPv4,comp-lzo,cipher BF-CBC,auth SHA1,keysize 128,key-method 2,tls-server'
Mon Mar 21 02:05:10 2016 us=123533 69.115.144.60:65005 Expected Remote Options String: 'V4,dev-type tun,link-mtu 1542,tun-mtu 1500,proto UDPv4,comp-lzo,cipher BF-CBC,auth SHA1,keysize 128,key-method 2,tls-client'
Mon Mar 21 02:05:10 2016 us=123551 69.115.144.60:65005 Local Options hash (VER=V4): '530fdded'
Mon Mar 21 02:05:10 2016 us=123559 69.115.144.60:65005 Expected Remote Options hash (VER=V4): '41690919'
RMon Mar 21 02:05:10 2016 us=123586 69.115.144.60:65005 TLS: Initial packet from [AF_INET]69.115.144.60:65005, sid=dab3460f a9ab573f
WRRWRWRWRWWWWRRRRWRWRWRWRWRWRWRWRWRWRWRWRWRWRWRWRWRWRWRWRWRWRWRWRWRWRWRWRWRMon Mar 21 02:05:10 2016 us=842168 69.115.144.60:65005 VERIFY OK: depth=1, C=US, ST=CA, L=Newark, O=ADS, OU=MyOrganizationalUni
t, CN=ads-vpn-server-1, name=EasyRSA, emailAddress=me@myemail.com
Mon Mar 21 02:05:10 2016 us=842359 69.115.144.60:65005 VERIFY OK: depth=0, C=US, ST=CA, L=Newark, O=ADS, OU=MyOrganizationalUnit, CN=ads--pfsense, name=EasyRSA, emailAddress=me@myemail.com
WRWRWRWRWRWRWRWRWRWRMon Mar 21 02:05:10 2016 us=916578 69.115.144.60:65005 Data Channel Encrypt: Cipher 'BF-CBC' initialized with 128 bit key
Mon Mar 21 02:05:10 2016 us=916609 69.115.144.60:65005 Data Channel Encrypt: Using 160 bit message hash 'SHA1' for HMAC authentication
Mon Mar 21 02:05:10 2016 us=916655 69.115.144.60:65005 Data Channel Decrypt: Cipher 'BF-CBC' initialized with 128 bit key
Mon Mar 21 02:05:10 2016 us=916662 69.115.144.60:65005 Data Channel Decrypt: Using 160 bit message hash 'SHA1' for HMAC authentication
WRMon Mar 21 02:05:10 2016 us=949581 69.115.144.60:65005 Control Channel: TLSv1.2, cipher TLSv1/SSLv3 DHE-RSA-AES256-GCM-SHA384, 2048 bit RSA
Mon Mar 21 02:05:10 2016 us=949618 69.115.144.60:65005 [ads--pfsense] Peer Connection Initiated with [AF_INET]69.115.144.60:65005
Mon Mar 21 02:05:10 2016 us=949655 ads-??-pfsense/69.115.144.60:65005 MULTI_sva: pool returned IPv4=10.254.254.6, IPv6=(Not enabled)
Mon Mar 21 02:05:10 2016 us=949685 ads-??-pfsense/69.115.144.60:65005 MULTI: Learn: 10.254.254.6 -> ads--pfsense/69.115.144.60:65005
Mon Mar 21 02:05:10 2016 us=949692 ads-??-pfsense/69.115.144.60:65005 MULTI: primary virtual IP for ads--pfsense/69.115.144.60:65005: 10.254.254.6
RMon Mar 21 02:05:13 2016 us=117978 ads-??-pfsense/69.115.144.60:65005 PUSH: Received control message: 'PUSH_REQUEST'
Mon Mar 21 02:05:13 2016 us=118012 ads-??-pfsense/69.115.144.60:65005 send_push_reply(): safe_cap=940
Mon Mar 21 02:05:13 2016 us=118030 ads-??-pfsense/69.115.144.60:65005 SENT CONTROL [ads--pfsense]: 'PUSH_REPLY,route 10.250.0.0 255.255.0.0,route 10.254.254.0 255.255.255.0,topology net30,ping 10,ping-restart 120,ifconfig 10.254
.254.6 10.254.254.5' (status=1)
WWRRWRWWRWRWRWR

    Client Log

    
Mar 21 02:05:02	openvpn[17113]: [server] Inactivity timeout (--ping-restart), restarting
Mar 21 02:05:02	openvpn[17113]: TCP/UDP: Closing socket
Mar 21 02:05:02	openvpn[17113]: SIGUSR1[soft,ping-restart] received, process restarting
Mar 21 02:05:02	openvpn[17113]: Restart pause, 2 second(s)
Mar 21 02:05:04	openvpn[17113]: WARNING: No server certificate verification method has been enabled. See http://openvpn.net/howto.html#mitm for more info.
Mar 21 02:05:04	openvpn[17113]: NOTE: the current --script-security setting may allow this configuration to call user-defined scripts
Mar 21 02:05:04	openvpn[17113]: Re-using SSL/TLS context
Mar 21 02:05:04	openvpn[17113]: LZO compression initialized
Mar 21 02:05:04	openvpn[17113]: Control Channel MTU parms [ L:1542 D:138 EF:38 EB:0 ET:0 EL:3 ]
Mar 21 02:05:04	openvpn[17113]: Socket Buffers: R=[42080->65536] S=[57344->65536]
Mar 21 02:05:04	openvpn[17113]: Data Channel MTU parms [ L:1542 D:1450 EF:42 EB:143 ET:0 EL:3 AF:3/1 ]
Mar 21 02:05:04	openvpn[17113]: Local Options String: 'V4,dev-type tun,link-mtu 1542,tun-mtu 1500,proto UDPv4,comp-lzo,cipher BF-CBC,auth SHA1,keysize 128,key-method 2,tls-client'
Mar 21 02:05:04	openvpn[17113]: Expected Remote Options String: 'V4,dev-type tun,link-mtu 1542,tun-mtu 1500,proto UDPv4,comp-lzo,cipher BF-CBC,auth SHA1,keysize 128,key-method 2,tls-server'
Mar 21 02:05:04	openvpn[17113]: Local Options hash (VER=V4): '41690919'
Mar 21 02:05:04	openvpn[17113]: Expected Remote Options hash (VER=V4): '530fdded'
Mar 21 02:05:04	openvpn[17113]: UDPv4 link local (bound): [AF_INET]69.115.144.60
Mar 21 02:05:04	openvpn[17113]: UDPv4 link remote: [AF_INET]104.196.144.148:1194
Mar 21 02:05:10	openvpn[17113]: TLS: Initial packet from [AF_INET]104.196.144.148:1194, sid=37518aa9 5fd4ad99
Mar 21 02:05:10	openvpn[17113]: VERIFY OK: depth=1, C=US, ST=New Jersey, L=Newark, O=Atlantic Digital Solutions, LLC, OU=MyOrganizationalUnit, CN=ads-vpn-server-1, name=EasyRSA, emailAddress=brian@atlanticdigitalsolutions.com
Mar 21 02:05:10	openvpn[17113]: VERIFY OK: depth=0, C=US, ST=New Jersey, L=Newark, O=Atlantic Digital Solutions, LLC, OU=MyOrganizationalUnit, CN=server, name=EasyRSA, emailAddress=brian@atlanticdigitalsolutions.com
Mar 21 02:05:10	openvpn[17113]: Data Channel Encrypt: Cipher 'BF-CBC' initialized with 128 bit key
Mar 21 02:05:10	openvpn[17113]: Data Channel Encrypt: Using 160 bit message hash 'SHA1' for HMAC authentication
Mar 21 02:05:10	openvpn[17113]: Data Channel Decrypt: Cipher 'BF-CBC' initialized with 128 bit key
Mar 21 02:05:10	openvpn[17113]: Data Channel Decrypt: Using 160 bit message hash 'SHA1' for HMAC authentication
Mar 21 02:05:10	openvpn[17113]: Control Channel: TLSv1.2, cipher TLSv1/SSLv3 DHE-RSA-AES256-GCM-SHA384, 2048 bit RSA
Mar 21 02:05:10	openvpn[17113]: [server] Peer Connection Initiated with [AF_INET]104.196.144.148:1194
Mar 21 02:05:13	openvpn[17113]: SENT CONTROL [server]: 'PUSH_REQUEST' (status=1)
Mar 21 02:05:13	openvpn[17113]: PUSH: Received control message: 'PUSH_REPLY,route 10.250.0.0 255.255.0.0,route 10.254.254.0 255.255.255.0,topology net30,ping 10,ping-restart 120,ifconfig 10.254.254.6 10.254.254.5'
Mar 21 02:05:13	openvpn[17113]: OPTIONS IMPORT: timers and/or timeouts modified
Mar 21 02:05:13	openvpn[17113]: OPTIONS IMPORT: --ifconfig/up options modified
Mar 21 02:05:13	openvpn[17113]: OPTIONS IMPORT: route options modified
Mar 21 02:05:13	openvpn[17113]: Preserving previous TUN/TAP instance: ovpnc1
Mar 21 02:05:13	openvpn[17113]: Initialization Sequence Completed

    From client-side windows machine:

    
>ping 10.250.0.2

Pinging 10.250.0.2 with 32 bytes of data:
Reply from 10.250.0.2: bytes=32 time=33ms TTL=63
Reply from 10.250.0.2: bytes=32 time=30ms TTL=63
Reply from 10.250.0.2: bytes=32 time=37ms TTL=63
Reply from 10.250.0.2: bytes=32 time=65ms TTL=63

Ping statistics for 10.250.0.2:
    Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
    Minimum = 30ms, Maximum = 65ms, Average = 41ms

    Works!

    From server:

    
$ ping 10.0.0.1   <---  pfsense address
PING 10.0.0.1 (10.0.0.1) 56(84) bytes of data.

    Doesn't work.

    While pinging:

    
vpn-server-1 etc]$ sudo tcpdump -vv -n -i tun0|grep 10.0
tcpdump: listening on tun0, link-type RAW (Raw IP), capture size 65535 bytes
    10.254.254.1 > 10.0.0.1: ICMP echo request, id 3931, seq 52, length 64
    10.254.254.1 > 10.0.0.1: ICMP echo request, id 3931, seq 53, length 64
    10.254.254.1 > 10.0.0.1: ICMP echo request, id 3931, seq 54, length 64
    10.254.254.1 > 10.0.0.1: ICMP echo request, id 3931, seq 55, length 64
    10.254.254.1 > 10.0.0.1: ICMP echo request, id 3931, seq 56, length 64
    10.254.254.1 > 10.0.0.1: ICMP echo request, id 3931, seq 57, length 64
    10.254.254.1 > 10.0.0.1: ICMP echo request, id 3931, seq 58, length 64
    10.254.254.1 > 10.0.0.1: ICMP echo request, id 3931, seq 59, length 64
    10.254.254.1 > 10.0.0.1: ICMP echo request, id 3931, seq 60, length 64
    10.254.254.1 > 10.0.0.1: ICMP echo request, id 3931, seq 61, length 64
    10.254.254.1 > 10.0.0.1: ICMP echo request, id 3931, seq 62, length 64
    10.254.254.1 > 10.0.0.1: ICMP echo request, id 3931, seq 63, length 64
    10.254.254.1 > 10.0.0.1: ICMP echo request, id 3931, seq 64, length 64
    10.254.254.1 > 10.0.0.1: ICMP echo request, id 3931, seq 65, length 64
    10.254.254.1 > 10.0.0.1: ICMP echo request, id 3931, seq 66, length 64
    10.254.254.1 > 10.0.0.1: ICMP echo request, id 3931, seq 67, length 64
    10.254.254.1 > 10.0.0.1: ICMP echo request, id 3931, seq 68, length 64
    10.254.254.1 > 10.0.0.1: ICMP echo request, id 3931, seq 69, length 64
    10.254.254.1 > 10.0.0.1: ICMP echo request, id 3931, seq 70, length 64
    10.254.254.1 > 10.0.0.1: ICMP echo request, id 3931, seq 71, length 64
    10.254.254.1 > 10.0.0.1: ICMP echo request, id 3931, seq 72, length 64
    10.254.254.1 > 10.0.0.1: ICMP echo request, id 3931, seq 73, length 64
    10.254.254.1 > 10.0.0.1: ICMP echo request, id 3931, seq 74, length 64

    Looks good, I think:

    But on the pfSense (client) side (still pinging):

    
# tcpdump -vv -n -i sis0|grep 10.0.0
tcpdump: listening on sis0, link-type EN10MB (Ethernet), capture size 65535 bytes
capability mode sandbox enabled
^C843 packets captured
851 packets received by filter
0 packets dropped by kernel

(nothing)

# tcpdump -vv -n -i ovpnc1|grep 10.0.0
tcpdump: listening on ovpnc1, link-type NULL (BSD loopback), capture size 65535 bytes
capability mode sandbox enabled
^C0 packets captured
0 packets received by filter
0 packets dropped by kernel

(nothing)

# ping 10.250.0.2
PING 10.250.0.2 (10.250.0.2): 56 data bytes
64 bytes from 10.250.0.2: icmp_seq=0 ttl=64 time=31.611 ms
64 bytes from 10.250.0.2: icmp_seq=1 ttl=64 time=29.781 ms
^C
--- 10.250.0.2 ping statistics ---
2 packets transmitted, 2 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 29.781/30.696/31.611/0.915 ms

(other direction still works)

    pfSense firewall rules

    
pfctl -sr
scrub on sis0 all fragment reassemble
scrub on rl0 all fragment reassemble
scrub on ovpnc1 all fragment reassemble
anchor "relayd/*" all
anchor "openvpn/*" all
anchor "ipsec/*" all
block drop in log quick inet from 169.254.0.0/16 to any label "Block IPv4 link-local"
block drop in log quick inet from any to 169.254.0.0/16 label "Block IPv4 link-local"
block drop in log inet all label "Default deny rule IPv4"
block drop out log inet all label "Default deny rule IPv4"
block drop in log inet6 all label "Default deny rule IPv6"
block drop out log inet6 all label "Default deny rule IPv6"
pass quick inet6 proto ipv6-icmp all icmp6-type unreach keep state
pass quick inet6 proto ipv6-icmp all icmp6-type toobig keep state
pass quick inet6 proto ipv6-icmp all icmp6-type neighbrsol keep state
pass quick inet6 proto ipv6-icmp all icmp6-type neighbradv keep state
pass out quick inet6 proto ipv6-icmp from fe80::/10 to fe80::/10 icmp6-type echorep keep state
pass out quick inet6 proto ipv6-icmp from fe80::/10 to fe80::/10 icmp6-type routersol keep state
pass out quick inet6 proto ipv6-icmp from fe80::/10 to fe80::/10 icmp6-type routeradv keep state
pass out quick inet6 proto ipv6-icmp from fe80::/10 to fe80::/10 icmp6-type neighbrsol keep state
pass out quick inet6 proto ipv6-icmp from fe80::/10 to fe80::/10 icmp6-type neighbradv keep state
pass out quick inet6 proto ipv6-icmp from fe80::/10 to ff02::/16 icmp6-type echorep keep state
pass out quick inet6 proto ipv6-icmp from fe80::/10 to ff02::/16 icmp6-type routersol keep state
pass out quick inet6 proto ipv6-icmp from fe80::/10 to ff02::/16 icmp6-type routeradv keep state
pass out quick inet6 proto ipv6-icmp from fe80::/10 to ff02::/16 icmp6-type neighbrsol keep state
pass out quick inet6 proto ipv6-icmp from fe80::/10 to ff02::/16 icmp6-type neighbradv keep state
pass in quick inet6 proto ipv6-icmp from fe80::/10 to fe80::/10 icmp6-type echoreq keep state
pass in quick inet6 proto ipv6-icmp from fe80::/10 to fe80::/10 icmp6-type routersol keep state
pass in quick inet6 proto ipv6-icmp from fe80::/10 to fe80::/10 icmp6-type routeradv keep state
pass in quick inet6 proto ipv6-icmp from fe80::/10 to fe80::/10 icmp6-type neighbrsol keep state
pass in quick inet6 proto ipv6-icmp from fe80::/10 to fe80::/10 icmp6-type neighbradv keep state
pass in quick inet6 proto ipv6-icmp from ff02::/16 to fe80::/10 icmp6-type echoreq keep state
pass in quick inet6 proto ipv6-icmp from ff02::/16 to fe80::/10 icmp6-type routersol keep state
pass in quick inet6 proto ipv6-icmp from ff02::/16 to fe80::/10 icmp6-type routeradv keep state
pass in quick inet6 proto ipv6-icmp from ff02::/16 to fe80::/10 icmp6-type neighbrsol keep state
pass in quick inet6 proto ipv6-icmp from ff02::/16 to fe80::/10 icmp6-type neighbradv keep state
pass in quick inet6 proto ipv6-icmp from fe80::/10 to ff02::/16 icmp6-type echoreq keep state
pass in quick inet6 proto ipv6-icmp from fe80::/10 to ff02::/16 icmp6-type routersol keep state
pass in quick inet6 proto ipv6-icmp from fe80::/10 to ff02::/16 icmp6-type routeradv keep state
pass in quick inet6 proto ipv6-icmp from fe80::/10 to ff02::/16 icmp6-type neighbrsol keep state
pass in quick inet6 proto ipv6-icmp from fe80::/10 to ff02::/16 icmp6-type neighbradv keep state
block drop log quick inet proto tcp from any port = 0 to any label "Block traffic from port 0"
block drop log quick inet proto udp from any port = 0 to any label "Block traffic from port 0"
block drop log quick inet proto tcp from any to any port = 0 label "Block traffic to port 0"
block drop log quick inet proto udp from any to any port = 0 label "Block traffic to port 0"
block drop log quick inet6 proto tcp from any port = 0 to any label "Block traffic from port 0"
block drop log quick inet6 proto udp from any port = 0 to any label "Block traffic from port 0"
block drop log quick inet6 proto tcp from any to any port = 0 label "Block traffic to port 0"
block drop log quick inet6 proto udp from any to any port = 0 label "Block traffic to port 0"
block drop log quick from <snort2c>to any label "Block snort2c hosts"
block drop log quick from any to <snort2c>label "Block snort2c hosts"
block drop in log quick proto tcp from <sshlockout>to (self) port = ssh label "sshlockout"
block drop in log quick proto tcp from <webconfiguratorlockout>to (self) port = https label "webConfiguratorlockout"
block drop in log quick from <virusprot>to any label "virusprot overload table"
pass in quick on sis0 inet6 proto udp from fe80::/10 port = dhcpv6-client to fe80::/10 port = dhcpv6-client keep state label "allow dhcpv6 client in WAN"
pass in quick on sis0 proto udp from any port = dhcpv6-server to any port = dhcpv6-client keep state label "allow dhcpv6 client in WAN"
pass out quick on sis0 proto udp from any port = dhcpv6-client to any port = dhcpv6-server keep state label "allow dhcpv6 client out WAN"
block drop in log quick on sis0 from <bogons>to any label "block bogon IPv4 networks from WAN"
block drop in log quick on sis0 from <bogonsv6>to any label "block bogon IPv6 networks from WAN"
block drop in log on ! sis0 inet from 69.115.144.0/20 to any
block drop in log inet from 69.115.144.60 to any
block drop in log on sis0 inet6 from fe80::20f:b5ff:fe8a:b476 to any
pass in on sis0 proto udp from any port = bootps to any port = bootpc keep state label "allow dhcp client out WAN"
pass out on sis0 proto udp from any port = bootpc to any port = bootps keep state label "allow dhcp client out WAN"
block drop in log quick on rl0 from <bogons>to any label "block bogon IPv4 networks from LAN"
block drop in log quick on rl0 from <bogonsv6>to any label "block bogon IPv6 networks from LAN"
block drop in log on ! rl0 inet from 10.0.0.0/24 to any
block drop in log inet from 10.0.0.1 to any
block drop in log on rl0 inet6 from fe80::220:18ff:fed5:fd75 to any
pass in quick on rl0 inet proto udp from any port = bootpc to 255.255.255.255 port = bootps keep state label "allow access to DHCP server"
pass in quick on rl0 inet proto udp from any port = bootpc to 10.0.0.1 port = bootps keep state label "allow access to DHCP server"
pass out quick on rl0 inet proto udp from 10.0.0.1 port = bootps to any port = bootpc keep state label "allow access to DHCP server"
block drop in log on ! ovpnc1 inet from 10.254.254.6 to any
block drop in log inet from 10.254.254.6 to any
block drop in log on ovpnc1 inet6 from fe80::20f:b5ff:fe8a:b476 to any
pass in on lo0 inet all flags S/SA keep state label "pass IPv4 loopback"
pass out on lo0 inet all flags S/SA keep state label "pass IPv4 loopback"
pass in on lo0 inet6 all flags S/SA keep state label "pass IPv6 loopback"
pass out on lo0 inet6 all flags S/SA keep state label "pass IPv6 loopback"
pass out inet all flags S/SA keep state allow-opts label "let out anything IPv4 from firewall host itself"
pass out inet6 all flags S/SA keep state allow-opts label "let out anything IPv6 from firewall host itself"
pass out route-to (sis0 69.115.144.1) inet from 69.115.144.60 to ! 69.115.144.0/20 flags S/SA keep state allow-opts label "let out anything from firewall host itself"
pass out route-to (ovpnc1 10.254.254.5) inet from 10.254.254.6 to ! 10.254.254.6 flags S/SA keep state allow-opts label "let out anything from firewall host itself"
pass in quick on rl0 proto tcp from any to (rl0) port = https flags S/SA keep state label "anti-lockout rule"
pass in quick on rl0 proto tcp from any to (rl0) port = http flags S/SA keep state label "anti-lockout rule"
pass in quick on rl0 proto tcp from any to (rl0) port = ssh flags S/SA keep state label "anti-lockout rule"
anchor "userrules/*" all
pass in quick on openvpn inet all flags S/SA keep state label "USER_RULE"
pass in quick on sis0 reply-to (sis0 69.115.144.1) inet proto tcp from any to 10.0.0.10 port = ssh flags S/SA keep state label "USER_RULE: NAT SSH to Server"
pass in quick on sis0 reply-to (sis0 69.115.144.1) inet proto tcp from any to 10.0.0.10 port = 3389 flags S/SA keep state label "USER_RULE: NAT RDP to Server"
pass in quick on sis0 reply-to (sis0 69.115.144.1) inet from 10.250.0.0/16 to any flags S/SA keep state label "USER_RULE"
pass in quick on sis0 reply-to (sis0 69.115144.1) inet from 10.254.254.0/24 to any flags S/SA keep state label "USER_RULE"
pass in quick on sis0 reply-to (sis0 69.115.144.1) inet proto tcp from any to 10.0.0.10 port = 25565 flags S/SA keep state label "USER_RULE: NAT "
pass in quick on rl0 inet from 10.0.0.0/24 to any flags S/SA keep state label "USER_RULE: Default allow LAN to any rule"
pass in quick on rl0 inet from 10.250.0.0/16 to any flags S/SA keep state label "USER_RULE"
pass in quick on rl0 inet from 10.254.254.0/24 to any flags S/SA keep state label "USER_RULE"
pass in quick on ovpnc1 reply-to (ovpnc1 10.254.254.5) inet all flags S/SA keep state label "USER_RULE: Allow all"
anchor "tftp-proxy/*" all</bogonsv6></bogons></bogonsv6></bogons></virusprot></webconfiguratorlockout></sshlockout></snort2c></snort2c>

    I am at a total loss as to what the problem might be at this point.  Any help is much appreciated.

    0
  • johnpoz
    LAYER 8 Global Moderator

    Are you trying to do 2 vpns??

    Did you go over their doc?  They do not show 2 vpns like your showing with vpn from you device in the google cloud to the gateway, and then one from your pfsense to the gateway.. There would be only 1 vpn from gateway to pfsense

    https://cloud.google.com/compute/docs/vpn/

    Why are you showing a 10.250 address for the gateway.  Its going to have to have a public IP..

    Are you trying to do a openvpn tunnel to your vpn server through a ipsec tunnel through the gateway??

    I have a funny feeling you have not even breezed over their doc???

    0
  • F

    That's not what I'm trying to do.  I'm just running a an OpenVPN server, as per the docs, and the built in Google IPSec VPN won't work for what I need anyway.

    I showed VPN between my gateway and and the OVPN server because the gateway is not my OVPN server, but it is the next hop from it.

    0
  • johnpoz
    LAYER 8 Global Moderator

    And where do they state that is supported??

    Your going to have to put a public IP on your instance that is running, not some port forward..

    What is it that you need btw??  Can I fire up a google compute instance for low cost or free for testing?

    I see they have a $300 60 day free trial, signing up..  So what is it exactly your wanting to accomplish?

    0
  • F

    @johnpoz:

    And where do they state that is supported??

    Your going to have to put a public IP on your instance that is running, not some port forward..

    I don't want to assume it isn't.

    @johnpoz:

    What is it that you need btw??  Can I fire up a google compute instance for low cost or free for testing?

    No but I can make one for you.

    @johnpoz:

    I see they have a $300 60 day free trial, signing up..  So what is it exactly your wanting to accomplish?

    That's for support.  An instance might only cost you $5 per month if you get the teeny tiny one.

    As to second part:  I need to add more subnets as well as do site-to-client (which google's VPN server doesn't do).

    Currently I'm trying to get it working with a tap interface.

    0
  • F

    Sent you PM.

    0
  • johnpoz
    LAYER 8 Global Moderator

    Well I got in in like 5 minutes

    fired up an instance, wget the openvpn as package

    Boom connected


    0
  • F

    I fixed it on my end.

    Set up server for tap.  Set up interface accordingly (needed to reboot as ovpn client was failing to ifconfig).  Set up bridge interface with LAN and OPT1.  Was able to ping vitrual IP of pfSense client from GCE server, but not pfSense's LAN IP or anything behind it.

    did a # sudo ip route add 10.0.0.0/24 dev br0 on server and voila.

    Not sure why it is not working with tun, maybe a bug of some sort with GCE.  Not sure what you did different to get it working on your end.

    0
  • johnpoz
    LAYER 8 Global Moderator

    I didn't do anything special, installed openvpn as - connected.. using TUN.  I had to change the IP that was in the profile to the external IP..

    0
  • F

    @johnpoz:

    Well I got in in like 5 minutes

    fired up an instance, wget the openvpn as package

    Boom connected

    I had no problem connecting.  Can you ping pfSense or anything behind its nat, assuming there is NAT.

    (BTW, I erroneously said there was no NAT on my GCE slice earlier, but now I think it is 1:1 NAT.  I'm new to all this stuff.)

    0
  • johnpoz
    LAYER 8 Global Moderator

    I am routing my traffic over the connection..

    What exactly are you wanting to accomplish with the vpn connection??


    0
  • F

    @johnpoz:

    I am routing my traffic over the connection..

    What exactly are you wanting to accomplish with the vpn connection??

    I have a funny feeling you only breezed through my post :P

    For now I have accomplished what I wanted to accomplish, which is a site-to-site VPN.

    Subnets are going to be added from various physical locations with lans behind pfsense and dd-wrt (in most cases).  There will be some modestly intricate routing between them.  In this case, the default gateway is always the local one.

    On the GCE subnet side some services will service.

    There will also be client-to-server connections which will do what you are doing.

    I think I would rather try and run pfSense on GCE.  It appears to be possible and there is some documentation, but it involves making a KVM virtual disk and loading it into a new instance in GCE, and I don't have a spare PC with VT-d needed to build it.

    See here:  https://gist.github.com/mkhon/0d8867e07c6b325ae228

    Who can I bribe to make one for me?  Maybe I'll start a new thread later.

    0
  • F

    By the way:  anyone trying to do what I'm doing should know that windows firewall by default blocks pings from other subnets, android phones and linux servers do not (not sure about iOS).  That might have really screwed me up had I not read it in the tons of time I spent trying and failing to get tun to work.

    0
  • johnpoz
    LAYER 8 Global Moderator

    So your going to have multiple machines on gce?  An they are going to use this vpn machine as their gateway to your network?  Can you setup the GCE networking that way for their instances?

    0

Products

Services

Support

News

Resources

Company

Our Mission

We provide leading-edge network security at a fair price - regardless of organizational size or network sophistication. We believe that an open-source security model offers disruptive pricing along with the agility required to quickly address emerging threats.

Subscribe to our Newsletter

Product information, software announcements, and special offers. See our newsletter archive to sign up for future newsletters and to read past announcements.

© 2021 Rubicon Communications, LLC | Privacy Policy