OpenVPN clients can connect to anything except the firewall itself?



  • Hey everyone! After a lightning strike took out my old pfSense machine running 2.2.6 (along with a cable modem and a few other things, it's been a rough month), I'm finally now onboard with a new machine running 2.3.2. Everything was working perfectly on my old machine, so I dumped the config from a backup and used that as the basis for my 2.3.2 machine.

    On my old machine, I had an OpenVPN setup allowing me to connect home remotely when I travel. This mostly imported correctly and I'm more or less back to normal, except for one thing. From my client, I can connect to any LAN address except the firewall itself. This worked in 2.2.6 but for some reason is now not working in 2.3.2.

    Some details: I'm configured using 172.16.104.0/21 as an IP address range. The firewall sits at 172.16.104.1. The DHCP range for OpenVPN clients is 172.16.105.10-50. OpenVPN is configured using tap, UDP. ovpns1 is assigned to OPT1 and OPT1 and LAN are bridged (bridge0). All this works correctly for any address on my LAN network except 172.16.104.1 (the firewall).

    Interestingly, using tcpdump, I can see the packets actually hitting the firewall when I do a ping or try telnetting to port 80 to hit the web interface, but I get no responses back. I can see the firewall trying to reply, but nothing is received on the remote end.

    
    tcpdump: listening on ovpns1, link-type EN10MB (Ethernet), capture size 65535 bytes
    09:58:42.722260 IP (tos 0x0, ttl 64, id 15734, offset 0, flags [none], proto ICMP (1), length 84)
        172.16.105.10 > 172.16.104.1: ICMP echo request, id 38677, seq 0, length 64
    09:58:42.722285 IP (tos 0x0, ttl 64, id 38797, offset 0, flags [none], proto ICMP (1), length 84, bad cksum 0 (->b9ef)!)
        172.16.104.1 > 172.16.105.10: ICMP echo reply, id 38677, seq 0, length 64
    
    

    So I know the data is actually getting to the firewall but something is blocking replies back. Get the same thing when I try port 22, 80 or 443, something is blocking the replies. This feels like I'm either missing a rule to allow this, or that another rule is blocking it. I don't have any rules created that are actively blocking any outbound traffic anywhere. But I'm not sure where I should look to create a rule to allow it.


  • Rebel Alliance Global Moderator

    172.16.104.0/21

    So you have somewhere close to 2k some clients on the same L2??  What is the point of such a large mask?

    Why are you using tap and not tun?

    I ssh and web to pfsense all the time via my openvpn connection.  But I use tun, my vpn clients get a 10.0.8/24 or 10.0.200/24 depending if they come in via tcp or udp.  I then hit any of the other pfsense IPs on any of its other interfaces for the gui or ssh.  I normally just access its lan which 192.168.9.253/24



  • @johnpoz:

    172.16.104.0/21

    So you have somewhere close to 2k some clients on the same L2??  What is the point of such a large mask?

    I have that many addresses, but nowhere near that many clients. I could probably configure everything on a single /24 if I wanted to. Only using about 50 addresses.

    @johnpoz:

    Why are you using tap and not tun?

    Because inertia and broadcast/multicast packets. Things like bonjour for Apple devices, which I have a ton of, don't (or didn't years ago when I first set this up) work very well over tun networks.

    This setup has worked great for me for ~6 years now going all the way back to DD-WRT days and through a few years of pfSense. And everything is 99% there now - just missing this last piece. It's just since I upgraded to 2.3 from the backup that this hasn't worked. Not sure if I did something in older versions that I'm not doing now and forgot about, but I figured it would be restored in the backup.



  • The mystery deepens…

    If I ping the server from a client with tcpdump running in another session, it seems that I'm getting replies?

    ping -c2 172.16.104.1
    PING 172.16.104.1 (172.16.104.1) 56(84) bytes of data.
    
    --- 172.16.104.1 ping statistics ---
    2 packets transmitted, 0 received, 100% packet loss, time 1009ms
    

    But in the tcpdump session:

    IP 172.16.105.3 > 172.16.104.1: ICMP echo request, id 17069, seq 1, length 64
    IP 172.16.104.1 > 172.16.105.3: ICMP echo reply, id 17069, seq 1, length 64
    IP 172.16.105.3 > 172.16.104.1: ICMP echo request, id 17069, seq 2, length 64
    IP 172.16.104.1 > 172.16.105.3: ICMP echo reply, id 17069, seq 2, length 64
    

    And tcpdump on the server thinks it saw it and sent a reply:

    tcpdump -n -i ovpns1 -t icmp
    tcpdump: WARNING: ovpns1: no IPv4 address assigned
    tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
    listening on ovpns1, link-type EN10MB (Ethernet), capture size 65535 bytes
    IP 172.16.105.3 > 172.16.104.1: ICMP echo request, id 17084, seq 1, length 64
    IP 172.16.104.1 > 172.16.105.3: ICMP echo reply, id 17084, seq 1, length 64
    IP 172.16.105.3 > 172.16.104.1: ICMP echo request, id 17084, seq 2, length 64
    IP 172.16.104.1 > 172.16.105.3: ICMP echo reply, id 17084, seq 2, length 64
    

    This is an Ubuntu Linux machine with no firewall.

    And if I ping a different, non-server address, I get a response:

    ping -c2 172.16.104.2
    PING 172.16.104.2 (172.16.104.2) 56(84) bytes of data.
    64 bytes from 172.16.104.2: icmp_seq=1 ttl=64 time=34.6 ms
    64 bytes from 172.16.104.2: icmp_seq=2 ttl=64 time=34.1 ms
    
    --- 172.16.104.2 ping statistics ---
    2 packets transmitted, 2 received, 0% packet loss, time 1002ms
    rtt min/avg/max/mdev = 34.165/34.401/34.638/0.300 ms
    

    And in the tcpdump shell:

    IP 172.16.105.3 > 172.16.104.2: ICMP echo request, id 17068, seq 1, length 64
    IP 172.16.104.2 > 172.16.105.3: ICMP echo reply, id 17068, seq 1, length 64
    IP 172.16.105.3 > 172.16.104.2: ICMP echo request, id 17068, seq 2, length 64
    IP 172.16.104.2 > 172.16.105.3: ICMP echo reply, id 17068, seq 2, length 64
    

    What on earth is going on here?



  • I'm having what seems to be the same problem. OpenVPN bridged using a TAP interface. I can ping the LAN interface of the Pfsense, my client gets an IP address from the DHCP server running on the Pfsense, in the same subnet as the LAN, but I cannot manage the Pfsense from it's LAN interface when I'm connected using VPN and since I'm using DNS Resolver, my VPN is basically broken as I cannot resolve any hostnames. This is the first time I've tried using OpenVPN in TAP mode. TUN works just fine.



  • Got same problem. Maybe anybody found solution?



  • Now to figure out how to access services hosted on the firewall itself.

    I can ping it, but WebGUI, SSH or DNS do not work (using the LAN IP of the router)

    https://forum.pfsense.org/index.php?topic=116850.0



  • Do  you have any firewall rules on the OpenVPN interface?  Maybe some holes in the firewall are needed to pass the desired traffic.



  • I think I tried allowing any to any on the OpenVPN interface, other than that, no.

    I will do more testing later.



  • Did some more testing today.

    (just right click->view image of you need a larger version)

    I have a management network (MGMTNET) which is a VLAN on my LAN interface (em3_vlan10) and I would like to bridge it with an OpenVPN TAP interface.

    An overview of the interfaces from the pfSense console (you can ignore most of the other interfaces, the one's to really care about are MGMTNET, MGMTNETVPN, MGMTNETBRIDGED)

    Now on to the VPN settings

    Another overview of the interfaces and where they are assigned too.

    The bridge interface (MGMTNETVPN is ovpns1)

    Firewall rules for the OpenVPN "pseudo-interface" (maybe I should try only having firewall rules on the ovpns1 interface)

    Again, the same for MGMTNETBRIDGED

    Now some testing I did earlier.

    I connect to the VPN with an OS X 10.11 machine over Ethernet for testing. I have a managed switch with some ports as an isolated port group to the modem (can pull multiple public IP's that way)

    It successfully pulls an IP from the DHCP server (which is also run by this pfSense box)

    I can ping the firewall (10.3.0.1) and can get a response from it.

    Here is a tcpdump of the ovpns1 interface from that time (I think the IP changed as I briefly disconnected and reconnected the VPN to check some pfSense settings, which required moving my machine over to the management interface)

    I try to nmap and connect to the SSH interface which is running on the firewall, no dice. (ssh router is an alias for it's management IP, which would stay the same since I'm using TAP. and you see the packs come through on the interface)

    Just doing a port scan to the firewall as I know for a fact those ports are open. (This is possibly smelling like a firewall problem to me)

    I'm able to access other devices on the network just fine, for example I curl'd the admin interface login from my switch.

    I think I'm going to try the firewall rules only on MGMTNETVPN/ovpns1 soon. Other than that I have few ideas.

    Thanks

    JTL



  • The mystery deepens.

    Let's tcpdump the bridge0 interface.

    Ping the firewall, traffic doesn't go through the bridge.

    Ping/connect to a service on another device on the network, it goes through.



  • I found a solution.

    Set the IP of your LAN interface to none and set the firewalls IP on the BRIDGE interface itself.

    I did this by downloading the config, editing the XML and uploading the config again. I tried doing it from the WebUI but accidently broke my network because I couldn't remove and add the IP to to the other interface at the same time.

    Hope this helps.