TLS handshake errors



  • Hi @ll

    I am trying to configure pfsense 2.01 to allow access from windows 7 clients to my internal LAN which is 10.0.0.0/8, I was following the
    howto's in this Forum and a few other guides but whatever I do I end up in TLS Handshake errors on the client side
    I did not really understand the meaning of the tunnel network, would be great if somebody could explain this a little closer.
    Does anybody know a working guide ? I mean in the ones I have tried differ in some ways and it would be great to have a guide known
    to be working.

    Thank you in advance for your help

    cheers thafener



  • I did not really understand the meaning of the tunnel network, would be great if somebody could explain this a little closer.

    In TUN (tunnel) mode, you have:
    LAN subnet behind the server - 10.0.0.0/8 for you
    Tunnel network - a subnet between the OpenVPN server and client
    Client network - the client will be on a network somewhere else, quite often happens to be 192.168.1.0/24 or some such common private subnet in a home, small office or cafe. It might have a real public IP if it is using a mobile dongle or similar.
    Both the server routes to/from the client by using the tunnel network. The tunnel network must be a different private subnet from the LAN at the server and from the network the client happens to be on. Typically pick something unlikely to be used in someone's home or cafe - 10.111.222.0/24
    OpenVPN allocates addresses in the tunnel network to server and client. Typically it uses ".1" for the server. Then it uses a small piece of the subnet for each client (a /30 subnet, if you know what that means). You will find server/client pairs using .5 and .6, then .9 and .10 etc. Each /30 is 4 addresses (e.g. 10.111.222.4-7, 8-11, 12-15…) The first and last address of the 4 are not used.



  • Thank you very much for your helpful hint, it works now  :)



  • Ok, a further question, I have set up the connection according to your hint, I got rid of the tls handshake
    problem but I cannot reach any host in the 10.0.0.0/8 network.
    where is my problem here ?



  • LAN subnet behind the server - 10.0.0.0/8 for you
    Typically pick something unlikely to be used in someone's home or cafe - 10.111.222.0/24

    I just noticed an inconsistency in the example subnets I posted. If you really are going to use 10.0.0.0/8 as your LAN, then it uses the whole "10" address space. You have to use something else for your tunnel subnet - e.g. 192.168.157.0/24 - (pick a "random" number in the upper range of 0-255 for "157").
    Of course, there is usually no need to have that much address space on LAN - make it 10.0.0.0/16 - then you can use 10.1.0.0, 10.2.0.0 etc… for other purposes.

    Ok, a further question, I have set up the connection according to your hint, I got rid of the tls handshake
    problem but I cannot reach any host in the 10.0.0.0/8 network.
    where is my problem here ?

    You need firewall rules on the OpenVPN tab to allow traffic - e.g. allow all source IP any, destination network 10.0.0.0/8. (Edit: fixed 10.0.0.0/8 typo here)



  • Thank you Phil this works excellent according to your tips



  • Similar problem, different specifics.

    172.28.136.0/22 is the lan

    172.28.28.0/24 is the tunnel

    MacOSX, Tunnelblick is the client. Export of settings file to client appears to work.

    Set for SSL/TLS only right now. Failing on TLS handshake.

    Started with 4096 CA, down to 2048 now wondering if that was the problem. No help.

    AES-128-CBC(128 bit) - default encryption algorithm - had tried a 256 variant, again, shrinking to see if it can work. No help.

    Firewall rules include WAN pass Destination 1194 UDP (tried TCP/UDP - no help)

    and Open VPN pass any (started with nothing entered for source and destination, now with 172.28.28.0/24 source and 172.28.136.0/22 destination - and added a rule on the LAN vice versa - no help.

    Given that I have to leave campus for every variation to see if it works (not yet) it's a bit maddening. Feels like I'm hunting down bits and pieces of information scattered all over to try and resolve.

    pfsense is 2.0.2 release - haven't auto-updated to 2.0.3 because I had an auto-update fry me in recent memory. 2.1 I could not make work reliably at all here.

    Have now tried with a Windows machine - same error - TLS handshake not completing in 60 seconds.

    Hmm - at least on the windows box (have not checked a mac yet) unchecking "Enable authentication of TLS Packets" "cured" it, though I'm unclear to what extent that makes anything less secure.

    Does not "cure" either tunnelblick or viscosity on Mac OS, at first testing.

    Upgraded pfsense to 2.0.3 just in case - seemed to urvive the upgrade, anyway. Have not checked for any difference in VPN behavior yet.



  • Seeing a few references to certificate oddities being at the root of handshake problems for some, and not running a webserver here, I brought up a second instance of OpenVPN on WAN port 80 UDP, with new 4096 server certificate, new 4096 user certificates, TLS auth on, DH parameters 4096, AES-256-CBC encryption and (another thing seen in passing that did for someone, but should not affect basic function) LZO compression on.  Moved the tunnel net one digit in unused space. Let the wizard do the firewall rules (as I had before, and then I started messing with them in desperation.)

    Seemed to work peachy. Still a bit nervous about it, never have been fond of things that work or do not work for no evident reason, as they always retain the option of not working again; with no evident reason required.

    More testing ensues.



  • Something interesting is going on in DNS land, is evidently part of my problem.

    WAN is, unfortunately, dynamic. Have had a DynDNS.org domain since it was free, and pfsense is (supposedly) configured to update it, and reports it as being up to date (green.) However dyndns's own nameservers reported a different address. This may be some misguided part of their transformation to "notfree." Doesn't actually make me want to pay them, for some reason. Does not appear to be a "caching" problem. Appears to be a "reported up to date, but not up to date" problem. I just corrected it by going straight into dyn.com. That was after checking the username setting and re-pasting the password into pfsense - same username and password pasted into DYN worked, so those are right.

    Got a quick subdomain over at FreeDNS (afraid.org), had to make a guess at what the "Auth Code" was, popped that in and appear to have pfsense ACTUALLY updating it to the correct address, so a config exported with that address actually connects somewhat reliably. So far. Many twisty little passages, all alike, indeed.