Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    Working on getting OpenVPN server bridging to fly.

    Scheduled Pinned Locked Moved OpenVPN
    94 Posts 13 Posters 87.6k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • S
      sullrich
      last edited by

      I'll commit a change to force this sysctl for OpenVPN.

      Update: commited to /etc/sysctl.conf

      1 Reply Last reply Reply Quote 0
      • N
        Numbski
        last edited by

        Thanks!

        1 Reply Last reply Reply Quote 0
        • N
          Numbski
          last edited by

          Just updating the status on this.

          The watchdog daemon is still having to kill the machine if it has any active OpenVPN sessions about once every 24 hours.  If no one connects, it stays up indefinitely.

          There is definitely a difference between a pfSense box that is bridged to a carp-enabled interface vs one that is not.  I have one with an uptime of over a month with the exact same config that has traffic flowing on it pretty consistently.  The difference is that neither WAN nor LAN is running CARP, whereas on the configs where the hangups occur, both WAN and the bridged interface are part of a CARP cluster.  That fact that I'm not all that familiar with how CARP really functions underneath doesn't help matters much.  All I know is that it broadcasts (which pfSense passes all bridge traffic by default, so that means CARP broadcasts are getting onto the OpenVPN tap interface), but I don't see how that would case harm.

          1 Reply Last reply Reply Quote 0
          • N
            Numbski
            last edited by

            is a glutton for punishment, I kid you not. :P

            Doing some research on CARP and OpenVPN, I came across this document:

            http://openvpn.net/archive/openvpn-devel/2005-10/msg00017.html

            The thought occurs to me.  We synchonrize states across firewalls in a CARP cluster.  Just speculating on how this happens, but it is possible that OpenVPN on system A tries to synchonize to system B and fails somehow.

            (This is mostly a note to myself to look into after I get back into the country, feel free to ignore me!)

            1 Reply Last reply Reply Quote 0
            • N
              Numbski
              last edited by

              Since this thread is turning more into a blog and less into a support thread, I figured I should update it. :)

              I've posted a doc topic on how to get things running as I have them currently here:

              http://doc.pfsense.org/index.php/Setting_up_OpenVPN_with_pfSense#OpenVPN_Client_Bridging

              Now, what has changed for me since the last time I posted?  Well, up until sullrich beat it into me that I should not have tap0 assigned as an opt interface (heh), I had tried to bridged from the ui.  I have since scrapped that, and the bridge is brought up at boot time using shellcmd/earlyshellcmd.  Also, my uptime is at a new record since doing this….1 1/2 days. :)

              We may finally have hit stability on this.  Crossing my fingers.  I'll update if my good luck continues, and if so, I'd like someone to volunteer to do a similar config.  If we have this licked, I'll start petitioning to have the config merged into the OpenVPN webui pages.

              1 Reply Last reply Reply Quote 0
              • N
                Numbski
                last edited by

                Okie, I have 3 full days of uptime without a kernel hang condition.  I think we have this licked folks.

                Any volunteers to duplicate my config to make sure?  I'd like to get this into the webui sometime soon.

                1 Reply Last reply Reply Quote 0
                • N
                  Numbski
                  last edited by

                  I was wrong.  Problem remains.  Repeat - problem remains.

                  There is definitely a collision of some sort between bridging of tap interfaces and CARP.  I have a little script watching the connectivity of the bridge, and all of a sudden the CARP interface involved on the physical interface just stops answering.  Remove the physical interface from the bridge, wait a few secs, put it back, and all is fine again.  ???

                  Really just don't know where to go with this anymore.  When it works, it works great.  It just doesn't stay working.

                  1 Reply Last reply Reply Quote 0
                  • R
                    rajl
                    last edited by

                    Why not just turn CARP off?  Is it a service that everyone needs?

                    1 Reply Last reply Reply Quote 0
                    • I
                      Icidic
                      last edited by

                      I hate to bump an already huge topic, but, can I confirm that pfSense with OpenVPN Bridge Mode ONLY appears to kernel hang when CARP is involved?  Or does it hang regardless of whether the pfSense machine is CARP aware or not?

                      Thanks :).

                      1 Reply Last reply Reply Quote 0
                      • R
                        razor2000
                        last edited by

                        @Icidic:

                        I hate to bump an already huge topic, but, can I confirm that pfSense with OpenVPN Bridge Mode ONLY appears to kernel hang when CARP is involved?  Or does it hang regardless of whether the pfSense machine is CARP aware or not?

                        Thanks :).

                        I will add that without CARP on, I have no stability problems or kernel hangs with OpenVPN bridging enabled.  My pfsense based alix board currently has an uptime of 8 days and 2 hours.  Hope this helps…

                        1 Reply Last reply Reply Quote 0
                        • valnarV
                          valnar
                          last edited by

                          Is there a set of instructions from start to finish that will accomplish this Layer-2 bridge over the Internet on pfSense boxes?

                          Thanks.

                          1 Reply Last reply Reply Quote 0
                          • B
                            bviper47
                            last edited by

                            I would also like to know if there is a full set of instructions for this.

                            1 Reply Last reply Reply Quote 0
                            • valnarV
                              valnar
                              last edited by

                              I managed to get a L2 bridge working with DD-WRT on a pair of old Linksys WRT54G routers following this:
                              http://www.dd-wrt.com/wiki/index.php/OpenVPN_-_Site-to-Site_Bridged_VPN_Between_Two_Routers
                              I will try it on a couple pfSense boxes next, but I assume it would operate the same way.

                              I also wonder if it would have the same limitations, which I discovered is a low 'high end' frame or packet size.  If you ping across the tunnel and set the don't-fragment-bit, the largest packet allowed will be 1342 bytes.  1343 fails without fragmentation.  Of course, IP can normally handle this, but I need L2 connectivity for non-IP protocols which have no knowledge of fragments.

                              Since my underlying protocol can't fragment it's own frame, is there a way to make pfSense fragment the packet after encapsulation with this OpenVPN/Bridge method?  Because after you add all the L3 & VPN overhead, it's quite easy to exceed the MTU allowed over the Internet, resulting in dropped packets (frames) at the source.

                              1 Reply Last reply Reply Quote 0
                              • T
                                tekkon
                                last edited by

                                I am on pfSense 1.21 final. I tried the OpenVPN bridging instructions over at this link:
                                http://doc.pfsense.org/index.php/VPN_Capability_OpenVPN#OpenVPN_Client_Bridging

                                The part where entering

                                "server-bridge 172.16.11.1 255.255.255.0 172.16.11.128 172.16.11.150"

                                in the 'Custom Option' box within OpenVPN's server settings didn't work.

                                I got the following error in my OpenVPN log:

                                "openvpn[15315]: Options error: --server and --server-bridge cannot be used together"

                                Since the '–server' option cannot co-exist with the '--server-bridge' option, which part of '/etc/inc/openvpn.inc' should I manually edit out the '--server' option?

                                Another part of the instruction that didn't work in 1.21 is where it instruct to enter

                                <earlyshellcmd>ifconfig bridge0 create</earlyshellcmd>
<earlyshellcmd>ifconfig bridge0 addm em2 up</earlyshellcmd>
<shellcmd>ifconfig bridge0 addm tap0</shellcmd>

                                in '/conf/config.xml' didn't load after a reboot. I had to manually execute it in cli to get the result.

                                1 Reply Last reply Reply Quote 0
                                • T
                                  tehryan
                                  last edited by

                                  i get the same error

                                  "openvpn[15315]: Options error: –server and --server-bridge cannot be used together"

                                  however i did manage to bridge the connections together in the config.xml file - it loads fine for me and shows up in the gui that it's learning, however the above error has me stumped as well

                                  1 Reply Last reply Reply Quote 0
                                  • J
                                    jmserrano
                                    last edited by

                                    Quote the text in http://doc.pfsense.org/index.php/VPN_Capability_OpenVPN#OpenVPN_Client_Bridging

                                    "Check off "Use Static IPs". This seems like a misnomer, but we're working around the way pfSense is currently coded. Don't worry, you'll still assign IP's from the server. "

                                    It's very important. 'server' directive dissapear from config file

                                    Regards

                                    1 Reply Last reply Reply Quote 0
                                    • T
                                      tehryan
                                      last edited by

                                      i've gone back to trying to get this to work again

                                      i've got a perfectly working tunnel using the site-to-site using pki tutorial above in the stickys - i redirect gateway so all traffic goes through the tunnel, i'd like to have the client computers DHCP over the tunnel if thats even possible, right now if i traceroute from the client side i get the pfsense box on the client, so 192.168.4.1, then i get the tunnel, 192.168.5.1, i then get the server box 192.168.1.1 and then its outside network - if i can remove most of those steps although unnessary it'd be nice

                                      i've clicked the static ip box now and i no longer get that error but i get something else now- keep in mind my tunnel works perfect before i add the custom options  - dev tap0; server-bridge 192.168.1.1 255.255.255.0 192.168.1.128 192.168.1.150

                                      my server box has a lan ip of 192.168.1.1; client box is 192.168.4.1 and the tunnel is 192.168.5.0/

                                      here's the new errors i get
                                      Feb 2 22:40:33 openvpn[19541]: XX.XX.XX.XXX:1194 Re-using SSL/TLS context
                                      Feb 2 22:40:33 openvpn[19541]: XX.XX.XX.XXX:1194 WARNING: 'dev-type' is used inconsistently, local='dev-type tap', remote='dev-type tun'
                                      Feb 2 22:40:33 openvpn[19541]: XX.XX.XX.XXX:1194 WARNING: 'link-mtu' is used inconsistently, local='link-mtu 1573', remote='link-mtu 1541'
                                      Feb 2 22:40:33 openvpn[19541]: XX.XX.XX.XXX:1194 WARNING: 'tun-mtu' is used inconsistently, local='tun-mtu 1532', remote='tun-mtu 1500'
                                      Feb 2 22:40:33 openvpn[19541]: XX.XX.XX.XXX:1194 [Client2] Peer Connection Initiated with XX.XX.XX.XXX:1194

                                      here are the custom options on my server  - route 192.168.3.0 255.255.0.0;route 192.168.4.0 255.255.0.0;push "route 192.168.1.0 255.255.0.0";push "redirect-gateway dev1";dev tap0;server-bridge 192.168.1.1 255.255.255.0 192.168.1.128 192.168.1.150

                                      192.168.3.0 is the first client, my client is actually the 2nd

                                      once again, if i remove everything after push redirect-gateway dev1 and unclick the static ip box my tunnel works perfect… i've gotta be missing something or over complicating it anyway

                                      is it even possible to dhcp over a tunnel?

                                      1 Reply Last reply Reply Quote 0
                                      • T
                                        tehryan
                                        last edited by

                                        now i'm getting this nonsense

                                        openvpn[48446]: WARNING: Since you are using –dev tap, the second argument to --ifconfig must be a netmask, for example something like 255.255.255.0. (silence this warning with --ifconfig-nowarn)

                                        ... i've done everything to follow every possible instruction... i must be getting instructions messed up and have an option clicked somewhere... i've spent coutless hours on this... i dont know what i'm doing wrong

                                        1 Reply Last reply Reply Quote 0
                                        • GruensFroeschliG
                                          GruensFroeschli
                                          last edited by

                                          One one site you're using a tap interface and on the other side a tun interface.

                                          You need a tap interface on both sides.

                                          A bridge transfers ethernet-packets. Meaning you can transfer everything which is an ethernetframe. (including everything which is NOT IP based).
                                          A tun-interface is an IP routing interface. Meaning you can only transfer IP-frames.

                                          Feb 2 22:40:33  openvpn[19541]: XX.XX.XX.XXX:1194 WARNING: 'dev-type' is used inconsistently, local='dev-type tap', remote='dev-type tun'
                                          Feb 2 22:40:33  openvpn[19541]: XX.XX.XX.XXX:1194 WARNING: 'link-mtu' is used inconsistently, local='link-mtu 1573', remote='link-mtu 1541'
                                          Feb 2 22:40:33  openvpn[19541]: XX.XX.XX.XXX:1194 WARNING: 'tun-mtu' is used inconsistently, local='tun-mtu 1532', remote='tun-mtu 1500'
                                          says exactly that:
                                          You have tap on ones side and tun on the other side.
                                          Since the IP-header counts as payload in ethernetframes (with a bridge –> tap-interface)
                                          You also get MTU inconsistency messages.

                                          We do what we must, because we can.

                                          Asking questions the smart way: http://www.catb.org/esr/faqs/smart-questions.html

                                          1 Reply Last reply Reply Quote 0
                                          • T
                                            tehryan
                                            last edited by

                                            sorry yea i fixed that by putting dev tap0 on both sides… the how-to never says that but i found it in the openvpn faq on its web-site - now i'm getting the ifconfig trouble

                                            1 Reply Last reply Reply Quote 0
                                            • First post
                                              Last post
                                            Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.