OpenVPN and 1.0-BETA1



  • Hi folks,

    During the holidays I've been playing around a bit with my WRAP-Board, the newest Embedded-Image and some PHP-files on it.

    Result: I've got OpenVPN working, well - at least the server part. I didn't look into the other parts (Client, Client-Specific-Configuration and CRL), I was only eager enough to do the server part.

    Anyhow, since I'm neither into FreeBSD, nor into PHP (rather Linux & Perl, and - well, I dare not say it - Perl on Win32), I'd like to know if there would be anyone interested in looking at my changes, improve them if applicable and re-integrate them into the "source tree". (Well, and - I hope - continue where I began…)

    I'll still have to re-track my changes, my hastyness tempted me too often to edit files directly on the flash card. So, please be prepared to wait until I'm able to mount the original and the modified bsd-partitions under Linux and get a "diff -urN" running on them. Just to make sure I'll catch everything...

    <think>I hope everything "survives" till then...</think>

    Regards,
    Marc

    P.S. Well, perhaps you could also encourage/persuade me to try some FreeBSD inside a VMWare - never did it before! If it doesn't take more than three evenings until 2:00 A.M. to feel myself "home" again...well...maybe...? :-\



  • Yes, I would be very intersted in the changes that you made.  Any chance of getting a patch-set?





  • @cmb:

    on VMware, you may want to check out http://wiki.pfsense.com/wikka.php?wakka=VMWareDevelopersEdition

    …ok, I've got the hint: that version might possibly contain "diff" and wouldn't make any problems mounting the CF-partitions - I'll give it a try tomorrow.

    Thanks!!
    Marc



  • So am I still in danger of totally thrashing opt1 and beyond by attempting an OpenVPN on 1.0-BETA1?

    I'm running from CD and Floppy off of what used to be a FreeBSD 4.2 nat'ing router (firewall isn't right I don't think, it was letting everything through!) that a client had before, but I had managed to compile OpenVPN 2 and had a tunnel between our sites, and I'd like to bring that back up again.

    The other end is (laughably) a Linksys WRT54GS. :)



  • @Numbski:

    So am I still in danger of totally thrashing opt1 and beyond by attempting an OpenVPN on 1.0-BETA1?

    I'm running from CD and Floppy off of what used to be a FreeBSD 4.2 nat'ing router (firewall isn't right I don't think, it was letting everything through!) that a client had before, but I had managed to compile OpenVPN 2 and had a tunnel between our sites, and I'd like to bring that back up again.

    The other end is (laughably) a Linksys WRT54GS. :)

    Yes OpenVPN still "thrashes" the OPT code.  You have been warned.  Atleast in HEAD I have cleaned it up a bit, if you remove the OpenVPN OPT (or any OPT for that matter) the system automatically cleans up the OPT assignmets so the rules and such don't get assigned to the wrong interface.



  • Just so I'm clear….

    Right now I have three interface,  WAN, LAN, and OPT1.  All three are being used.  OPT1 is simply a second LAN belonging to another client of mine, and I use this to provide firewalling for both client on one device.

    If I attempt to add OpenVPN, I presume then that it creates what could be called "OPT2"?  From here there isn't a problem, but once you attempt to remove it, it will attempt to remove and replace mappings on OPT1?

    Just wanting to make sure I understand what I'm risking by attempting ot set this up.  I think the short answer here is "don't do this remotely" and "do it overnight, keep the lastest correct config.xml handy just in case". :D

    Just want to be sure.  If the risks are minor and wouldn't risk LAN and WAN I might be inclined to set this up remotely.  Of course, BETA. :) and the developer tag is the only reason I can see the code there anyway. ;)



  • The problem is that it will add a new OPT interface.  If for some reason you add another OPT interface after this fact then remove OpenVPN then all the mappings get royally hosed.



  • Got it.  So I'm "probably" okay to give it a shot, but of course no guarantees. :)



  • Yep.  Backup you're config.xml first.



  • Well, this sucks.  Seems that even though I've filled out all pertinent form fields, but yet openvpn doesn't actually execute.  When I run ps ax | grep openvpn, nothing is there, even though the server is checked off to run. ???

    No wonder it appears that my local firewall can't connect to it.  It isn't running.  Hmm…



  • Try running openvpn from the shell to make sure there are no obvious errors.



  • /usr/local/sbin/openvpn –help returned as exptected, so no problems there.  I haven't copied over all of my crt's, keys, and config files to run the server from command line as usual, but the config data is in config.xml.  I'm running config from a floppy, so I'm hesistant to start doing more writes than needed to that little floppy to test. :\  I set verbosity to 3, but nothing got logged when I turned on the server.



  • First of all: I'm still alive! (even if you thought me to be dead)  ;D

    I've just got some 5 minutes so I thought I'll look into the forum. Well, I didn't think this topic would get that much attention.

    Q: "Ok, where's the stuff?"
    A: The last days were a lot busier than I thought, but I'm on VMWare's trail and I hope to be able to do the diff until the end of this week.

    @sullrich:

    The problem is that it will add a new OPT interface.  If for some reason you add another OPT interface after this fact then remove OpenVPN then all the mappings get royally hosed.

    I see.
    I did have some problems at first, they were related to lowercase/uppercase label (and therefore causing "unknown alias" errors in applying pf rules), interface tun0 not visible in "Firewall->Rules" and in "Interfaces->assign", thus not being able to place any rules on that interface, but I managed to fix that.
    You might call my changes "quite ugly" since due to my lack of knowledge of PHP, FreeBSD and the pfSense internals, I was doing a little "island-hopping" from file to file, adding a little salt here and removing a little pepper there…  :)

    Probably the change with the biggest impact is in filter.inc, where I commented out the if-condition which excludes tun-interfaces. Maybe that'll break IPSEC or any other kind of VPN which uses tun-interfaces. I had to do it.

    Just to give you a picture of my changes: filter.inc, globals,inc, openvpn.inc, pfsense-utils.inc, rc.reload_interfaces.inc, util.inc, fbegin.inc and a handful of php files.

    I didn't have problems with OPT interfaces, in fact I tried it first without one, then deleted the OpenVPN config again, added an OPT interface and added an OpenVPN server again. In both cases OpenVPN created a new <opt1>"tagset" without OPT1 interface in config.xml and created a <opt2>"tagset" when <opt1>existed.

    However, I'll consider the above scenario (add VPN-OPT1, add ETH-OPT2, remove VPN-OPT1) and give it a try as soon as I've got the diff and a dd-full-backup of my CF card.

    There's only a few minor problems left. One's that the TUN0 interface shows up in the "Interfaces->assign" list, but not in the "Interfaces" menu. I still can access its configuration page using its index e.g.: http://pfsense/interfaces_opt.php?index=2 (tun0 is my second OPT interface). But maybe this "feature" won't even be needed.

    The second one is, that there are some rare configuration changes, where the whole pf needs to be restarted and the interfaces seem to get reset. OpenVPN should be restarted afterwards, but I didn't yet find the correct file where to add some lines. (Or maybe I did it already in one of my sleepless nights...I'm not sure... :))

    Maybe this little preview - eerrmm - little "chamber of horrors" will scare you off so I can take off the rest of the week and do a little vacation under sun and palms ...  ;D

    @Numbski:

    Well, this sucks.  Seems that even though I've filled out all pertinent form fields, but yet openvpn doesn't actually execute.  When I run ps ax | grep openvpn, nothing is there, even though the server is checked off to run. ???

    No wonder it appears that my local firewall can't connect to it.  It isn't running.  Hmm…

    I had the same problems with my first try. You're using vpn_openvpn.php, right? There's already a beautiful config page, you can fill it out and then – nothing. Looking at openvpn.inc I thought I'd have to change more than a few lines in it to get it working with that config page. True. And that would have been a too big problem for me to solve.

    ...until I found some OpenVPN-reladed php files "lying around" in /usr/local/www which seemed to be newer and not being linked anywhere. If you try them instead, you'll get an ugly, unformatted page, but you can fill it out and - comparing the code with the .inc file, it seemed to fit together much better.
    Just try vpn_openvpn_srv.php instead. But even if you do, you'll still be stuck that it doesn't wirte the openvpn config file. The reason is that the config pages use some "dirty-files" to track changes on them, to present you a neat little "Apply Changes"-button. The PHPs expect the path to these files as some kind of "global variables", which I - you've got it - had to define in globals.inc.

    Don't search any longer. I did the changes already and I also modified the config pages to better resemble the pfsense webgui you're used to (fbegin.inc did the trick). I didn't try it, but different themes should also work (had to change some img srcs).

    So, give me some time until the end of this week, then you can go ahead and trash your pfsense installations. ;D

    Ok now, as usually the 5 minutes have turned out to be more than 1 hour.
    Wilma's roaring and the sabretooth-tiger already fell asleep in my bed!  ;D

    regards,
    Marc</opt1></opt2></opt1>



  • Well, to keep my sanity, for now I think I'll go ahead and mount hte hard drive in the system with the old configs and re-establish the openvpn using those configs, and see if I can't set up rules to allow traffic to and from the interface that isn't listed for the moment.  Keeps me from hosing config.xml any further until you give me a more implicit way to screw up config.xml. ;)



  • Well, progress, and none.

    I copied my configs from the hard drive to the floppy, and ran my typical openvpn –config, and then tunnel came up in an hurry.  Of course, I can't send any traffic across the tunnel because there's no rule for it.  I ssh into the firewall (I'm still 3 hours away from it), and run assign interfaces.  It says tun0 is an invalid interface, despite being able to ifconfig tun0 just fine.  hmmm????

    I could manually hack in a pair of <opt2>tags and set it to tun0, but I'm quaking in my boots to do that remotely only to find that it doesn't like the interface and the firewall doesn't come the rest of the way back up.  Eep!

    So...thoughts?  What makes the firewall consider an interface to be "valid"?</opt2>



  • Okay, I just gave myself a terrible crash course in m0n0wall-ese.

    There are a bunch of *.inc php files in /etc/inc.  Amongst these is util.inc.  That file has a function that makes itself available (when included, either directly or indirectly) get_interface_list.

    As shown in the file:

    /*
    *  get_interface_list() - Return a list of all physical interfaces            *  along with MAC and status.

    *  $mode = "active" - use ifconfig -lu
    *          "media"  - use ifconfig to check physical connection
    *                      status (much slower)
    */
    function get_interface_list($mode = "active", $keyby = "physical", $vfaces = "")

    So by default, the system just uses the output of ifconfig -lu.  If I run that on the box (where tun0 is active), I get this output:

    xl0 xl1 sis0 ng1

    In plain english, WAN, LAN, OPT1, and my current PPTP session.  tun0 is nowhere to be found.

    The other option uses ifconfig -l and some filtering.  ifconfig -l gives me a huge list, including every interface, virtual or not, up or down.  Later in this same function we have:

    if(!$vfaces) {
                    $vfaces = array(
                                    'ppp',
                                    'sl',
                                    'gif',
                                    'faith',
                                    'lo',
                                    'ng',
                                    'vlan',
                                    'tun',
                                    'pflog',
                                    'pfsync',
                                    'carp'
                            );
            }

    Now,  I'm a perl coder here, so let me see if I'm reading this right - if $vfaces is not defined (not populated?), then store this list of strings (interface/driver names) as an array in $vfaces.

    I didn't really get much farther than this.  I did two really dirty hacks to test my theory on this.  I commented out 'tun' from the array, and changed the $mode to media, saved the file in /conf, and did an ln -fs /conf/util.inc /etc/inc/util.inc.

    That worked, tun0 now appears.  I tried to create rules as usual, however whatever I name the interface, OPT2, MyHome, or whatever, I get a macro error stating that the macros (whatever name I gave it) does not exist.

    I think that's enough hacking for one night.  I'm a network engineer, not a php coder. :)  The other guy working on this probably got much farther than myself.  I haven't even gotten to the point where I'd understand how openvpn settings in config.xml could be read at boot time and multiple tun interfaces brought up prior to the firewall rules getting applied. :)



  • Finally!

    Here's the long-expected patch, I hope it will work out!
    Sorry for the delay!

    regards,
    Marc

    pfSense-OpenVPN.patch.txt



  • Applied cleanly to RELENG_1 and has been commited.

    However, only 10% applied on -HEAD.    I'll work on it.



  • @Numbski:

    That worked, tun0 now appears.  I tried to create rules as usual, however whatever I name the interface, OPT2, MyHome, or whatever, I get a macro error stating that the macros (whatever name I gave it) does not exist.

    I think the problem with this error was, that pf seems to be expecting interface names in uppercase. Otherwise you'll get the macro errors - it just wants to state that it can't dereference the interface name.

    That's what my config.xml section looks like, after successful OpenVPN configuration:

    <opt2><descr>TUN0</descr>
    <if>tun0</if>
    <ipaddr>172.19.100.1</ipaddr>
    <bridge><subnet>24</subnet>
    <enable><ovpn>server_tun0</ovpn></enable></bridge></opt2>

    regards,
    Marc



  • Actually, it had nothing to do with that.

    I just had the chance to take a look at how pfctl takes things in, and I was looking at rules.debug.

    What happened is really quite simple.  When you create an opt interface, an alias is created: $WhateverNameYouGaveTheInterface. :)

    In this case $OPT2, then $MyHome.

    I had these already:

    System Aliases

    lan = "{ xl1  }"
    wan = "{ xl0  }"
    pptp = "{ ng1 ng2 ng3 ng4 ng5 ng6 ng7 ng8 ng9 ng10 ng11 ng12 ng13 ng14 }"
    pppoe = "{ ng1 ng2 ng3 ng4 ng5 ng6 ng7 ng8 ng9 ng10 ng11 ng12 ng13 ng14 }"
    OPT1 = "{ sis0 }"

    It was complaining that there was no alias for OPT2, and that was because, well, there was no alias created for OPT2 in the ruleset!  I have no idea why this is, all I did was allow get_interface_list() to see tun0, everything else should have gone by the book.  I added this line:

    OPT2 = "{ tun0 }"

    to the file and ran pfctl -f rules.debug

    Lo and behold, the tunnel now works!

    So did I find a bug in my hack, or a hack in my bug?  ;D



  • In /etc/inc/filter.inc there is a routine that creates the aliases.  Simply modify that to detect openvpn and create the alias.

    BTW: I got you're work commited to HEAD finally as well.



  • Well no wonder!  :o

    foreach ($ifdescrs as $ifdescr => $ifname) {
                    /* do not process tun interfaces */
                    if(stristr(filter_opt_interface_to_real($ifname), "tun") == true) continue;

    LOL, we're intentaionally NOT writing tun aliases? :D

    Was this due to problems with openvpn, thus a safety valve to keep people from using them, or is there some other reason we're doing this?

    That's in filter.inc, btw.

    Also, could someone tell me if I'm wasting time or if what I'm doing is at all pertinent?  I can stop anytime….I do notice that any time I reload the firewall rules, tun0 goes down, and I have to HUP the openvpn process.



  • Woops, I'll get that removed.  I honestly cannot remember why we did this other than hiding the interfaces that the user doesn't use to K.I.S.S.



  • K.  For grins I went ahead and copied filter.inc to /conf, modified it and did an ln -fs.  Nothing died.  I haven't attemped a rules change yet, will here in a few moments, see if the rules apply successfully now.  I would think they would.  I'm doing a hard drive install on the box here in about an hour or so, so we'll see how stable this is.  I just really wish I knew why openvpn has to keep being -HUP'ed every time I make a rules change. :\



  • Hupping it every time you make a change does sound rather strange to me.

    Perhaps you could detect that openvpn is running in filter_configure_sync() and HUP it at the end of the routine before it returns.



  • Huh?
    I thought I have already addressed these problems!

    • processing of tun interfaces in filter.inc
    • reloading openvpn every time pf reloads in rc.reload_interfaces

    Or - am I missing the point here?  ???

    Marc



  • @ecce:

    Huh?
    I thought I have already addressed these problems!

    • processing of tun interfaces in filter.inc
    • reloading openvpn every time pf reloads in rc.reload_interfaces

    Or - am I missing the point here?  ???

    Marc

    Yep, I looked at the code after replying and you indeed have taken care of this.



  • @sullrich:

    Applied cleanly to RELENG_1 and has been commited.

    RELENG_1 = Beta 1/Release 1?
    Is there still enough time to do some further modifications to get the client part running? (my enthusiasm has grown again)  :)
    Will the modifications I've made and openvpn be kept for 1.0?
    I've read in some other thread that you were planning to remove openvpn completely for 1.0, so I hope to have been able to avoid this!  ;)

    @sullrich:

    BTW: I got you're work commited to HEAD finally as well.

    Fine! As far as you can see - are there any problems to be expected?
    I mean - there seems to have been some reason to exclude tun interfaces from creation in util.inc.
    I've been thinking of other VPN types like IPSEC, PPTP etc which might be also using tun interfaces…



  • I mean - there seems to have been some reason to exclude tun interfaces from creation in util.inc.
    I've been thinking of other VPN types like IPSEC, PPTP etc which might be also using tun interfaces…

    …and in filter.inc ...



  • It's already in RELENG_1 which is beta 1 / release branch.  There is no plans on "removing" it, just removing the menu item that points to it which has already been done.

    However, if you can get this thing working solid enough then we will definately revisit including it.  However we'll need to backport the interface renumber code from HEAD which fixes the problem when you remove an interface.



  • Okay, I need to clarify a few things, as I've seriously muddied the waters here.  My config is as follows:

    Stock 1.0-BETA1, now installed to hard disk.

    Modified util.inc to allow tun interfaces to be detected.
    Modified filter.inc, again, to allow tun interfaces not to filtered from aliases generation.

    Created /usr/local/etc/rc.d/openvpn.sh, then when run from rc with "start" appended (as with normal freebsd rc.d stuff) starts all of my tunnels (all one of them at this point).

    Everything seems to be humming along fine.  I haven't attempted to apply the patches that were posted to this thread yet.  I was just doing an independent dig-through of what was required to get openvpn to run from the console, hoping that my conclusions would help.

    (After going back and reading the recent posting barrage….)

    Do I need to make an additional modification to prevent the tunnel from going down upon a reload?  It seems you adjusted an rc script to do this....

    It appears we all more or less wound up in the same place, albeit that he's starting openvpn the "right" way, and I'm not. ;)

    So the next step is for me to use his patch and set it up the right way.  Won't happen today, perhaps later this week though?



  • I'm debating as to whether I have time to mess with this tonight or not.  I have an extra net4501 laying here that's taunting me.

    What issues are left after all of the patches are applied in this thread?  Is it stable enough to make 1.0 final (without the developer tag)?



  • @Numbski:

    Do I need to make an additional modification to prevent the tunnel from going down upon a reload?  It seems you adjusted an rc script to do this….

    Hi,

    I had to modify the rc.reload_interfaces.inc script to restart openvpn:
    In certain cases (I don't remember now), when pf needs to restart (and reload all interfaces) the openvpn and the tun0 interface would still be running, but no connections are being accepted any more. The system log also doesn't reveal any openvpn activity at this point.
    It seems reasonable to reload the tun interface (thus restart openvpn) when pfsense needs to reload all other interfaces, so I would have come to this point either way.  ;)

    You said, that openvpn would also reload on changing of firewall rules on the tun0 interface.
    I'll check this the next few days and keep you informed!

    Marc



  • @Numbski:

    I'm debating as to whether I have time to mess with this tonight or not.  I have an extra net4501 laying here that's taunting me.

    What issues are left after all of the patches are applied in this thread?  Is it stable enough to make 1.0 final (without the developer tag)?

    About stability:
    Since I only did some modifications to some scripts in order to get openvpn up and running without using the shell, it is as stable as it would be when you configure openvpn on pfsense manually.
    I've got it running for about 1 month on a WRAP box, which has already suffered from several power losses, but always came up cleanly without any problems. I also cannot tell much about how many users it will get along with, as I am the only user.  :)

    My configuration uses certificates, the tun interface type and TCP on port 443. Never had any problems, never tried anything else.

    If you'd like to know my TODO-list, here it goes:

    Fixes:

    • check out the "restart-problem" you've told me.
    • check out the "interface renumbering" bug and maybe look at the interface renumbering code in HEAD (thx @sullrich for the hint!), then decide whether to live with it or change it.
    • find out the reason why TUN0 does not show up in the "Interfaces" menu.

    Features (seem to be just webinterface issues):

    • get OpenVPN in client mode working.
    • get the "Client-specific configuration" working.
    • get CRL lists working.

    One thing I've learnt for now: I'll do all future changes inside a VM ;)

    Anything else left?

    Marc



  • Oh, and one feature I'd like which I forgot: get the syslog messages regarding openvpn into a separate tab!

    just a side note: would you believe me that I did most of the coding on a VT420 terminal?  ;D



  • I got it to work after using ecca's patches.

    The only thing I had to fix manually is that $d_ovpnsrvdirty_path is defined nowhere. So, I just added an entry for it on guiconfig.inc (where we usually put those definitions to dirty files). The problem in doing so is that guiconfig.inc can't be included from openvpn.inc, since it's on /usr/local/www, and that makes you need authorization to include it, apparently. So, temporarily, I just defined $d_ovpnsrvdirty_path in openvpn.inc. I wonder if there's a more elegant way to do that?

    Now on to the client…



  • @ecce:

    Oh, and one feature I'd like which I forgot: get the syslog messages regarding openvpn into a separate tab!

    just a side note: would you believe me that I did most of the coding on a VT420 terminal?  ;D

    Not a problem.  I'll work on this.



  • @fernandotcl:

    The only thing I had to fix manually is that $d_ovpnsrvdirty_path is defined nowhere.

    See /etc/inc/globals.inc…it should be there. I know - maybe it's not at the right place there, but it worked.



  • I moved it to guiconfig.inc which houses all of the other dirty file locations.


Locked