Testing new packages?



  • Hey,

    How do I test the packages I created? Apparently, one solution would be creating a local mirror to serve the XML-RPC stuff and then use some ugly hacks to make pfSense point to this mirror instead of the official ones, so that I could install new packages.

    The problem is: I can't create a local mirror. To do so, I'd need to rsync to rsync://cvs.pfsense.org/mirrors, but rsync won't connect cause I'm not authorized to connect. I'd have to drop a line to the core team so that they added my IP to their firewall so that I could use rsync to create a local mirror. I'm not doing that cause this is really a local mirror, I don't have the bandwidth to host a real mirror.

    I just wanted to test some packages. I'm running out of ideas…

    Thanks in advance.



  • Setup the XMLRPC then wget http://www.pfsense.com/packages/

    We do not have mirror functionality in 1.0.  It's in 1.1.

    The other alternative is to copy your XML file to /usr/local/pkg/ and then invoke pkg_mgr.php?xml=mytestpackage.xml



  • Until the new package code (which will make repositories far easier to create) is integrated for 1.1, the easiest option would be to manually add a package entry in get_pkg_info(), located in pkg-utils.inc. This function calls the pfsense.com XMLRPC server and returns whatever information is needed by the package system.

    A simple way to accomplish this would be to make an array containing your package entries, and then array_merge() $resp and your own array together before returning it.



  • Thanks for the ideas, I'll try it out.



  • @sullrich:

    Setup the XMLRPC then wget http://www.pfsense.com/packages/

    Well, that's what I first tried, but I have no mens of setting up XML-RPC, cause that would require me to grab the contents of cvs.pfsense.com/pfSense. They are PHP scripts, so insteading of downloading them, I end up executing them.

    If someone with access to the website could send me those files, it'd help a lot, I think.

    @sullrich:

    The other alternative is to copy your XML file to /usr/local/pkg/ and then invoke pkg_mgr.php?xml=mytestpackage.xml

    Hmm that didn't go well… I wonder how that would be possible if the package information is split in two files: pkg_config.xml and <packagename>.xml.

    @colin_:

    the easiest option would be to manually add a package entry in get_pkg_info(), located in pkg-utils.inc. This function calls the pfsense.com XMLRPC server and returns whatever information is needed by the package system.

    A simple way to accomplish this would be to make an array containing your package entries, and then array_merge() $resp and your own array together before returning it.

    I did that. The problem is that although I have now my package in the packages list, installing it isn't that simple, apparently. What am I missing? I think I'll have to hack pkg-utils.inc even more, and maybe that'll take longer than if I got a XML-RPC server working.</packagename>



  • XMLRPC Code is available: www.pfsense.com/xmlrpc.tgz

    In terms of splitting the items into 2, this is already how it works.  pkg_config.xml is the manifest.  Each package then has a xml file.

    Take a look at http://cvs.pfsense.com/cgi-bin/cvsweb.cgi/tools/packages/



  • Thanks a lot, that clarified things. But now I'm puzzled.

    I've set up a local "mirror", editing globals.inc to point to this mirror, etc.. It seems to work.

    I created a few packages, but I don't know where to place them. What's puzzling me is that I have lots of XML files in $DOCROOT/config and $DOCROOT/config/packages. Those are package files, and are mostly the same, except for a few of them. The problem is: they're nothing like the packages we see at http://cvs.pfsense.com/cgi-bin/cvsweb.cgi/tools/packages/ . For example, take a look at nmap.xml. It starts with <package>in the cvs /tools dir, but it starts with <packagegui>in $DOCROOT/config. One specifies the FreeBSD package as a "dependence", whereas the other doesn't.

    In short, it seems to me that those packages at /tools are the real packages, and the ones at $DOCROOT/config were created by some sort of script, that also updated pkg_config.xml. Am I right? What script is that?

    Also, regarding the xml=<mypackage>.xml trick, it didn't work out.  I checked pkg_mgr.php and pkg_mgr_install.php, and none of them parse $_GET['xml'], apparently.

    Thanks once again.</mypackage></packagegui></package>



  • @fernandotcl:

    I created a few packages, but I don't know where to place them. What's puzzling me is that I have lots of XML files in $DOCROOT/config and $DOCROOT/config/packages. Those are package files, and are mostly the same, except for a few of them. The problem is: they're nothing like the packages we see at http://cvs.pfsense.com/cgi-bin/cvsweb.cgi/tools/packages/ . For example, take a look at nmap.xml. It starts with <package>in the cvs /tools dir, but it starts with <packagegui>in $DOCROOT/config. One specifies the FreeBSD package as a "dependence", whereas the other doesn't.</packagegui></package>

    The packages you are seeing that start with <package>are from the (old) reorganization branch that was created to test some early 1.1 code. Only packages that follow the <packagegui>format are compatible with the current installer.

    @fernandotcl:

    Also, regarding the xml=<mypackage>.xml trick, it didn't work out.  I checked pkg_mgr.php and pkg_mgr_install.php, and none of them parse $_GET['xml'], apparently.</mypackage>

    Correct - only packages defined by the repository can be installed at the moment.

    You may want to wait to begin development until some releases are built with the new package code from HEAD, which should make many of the headaches you're currently running into disappear.</packagegui></package>



  • For the archives:

    "xmlrpcbaseurl" => "www.pfsense.com",
        "xmlrpcpath" => "/pfSense/xmlrpc.php",

    Then have a look at the master package manifest located at: http://pfsense.com/cgi-bin/cvsweb.cgi/tools/pkg_config.xml

    Then have a look at the packages themselves:

    http://pfsense.com/cgi-bin/cvsweb.cgi/tools/packages/

    Finally add your package info to pkg_config.xml on the webserver and then populate the packages folder.  pfSense.com example is here: http://www.pfsense.com/packages/

    CoreGUI information is located at: (Note, we are switching over to CoreGUI2, we recommend you learn this)
    http://wiki.pfsense.com/wikka.php?wakka=CoreGUI2
    http://wiki.pfsense.com/wikka.php?wakka=CoreGUI

    Good luck!!



  • How i make this:
    Modify only
    "xmlrpcbaseurl" => "10.0.0.1", // "www.pfsense.com", // 10.0.0.1 - ip my pfsense

    extract 'xmlrpc.tgz' to '/usr/local/www' =>>  '/usr/local/www/pfSense'

    Put 'pkg_config.xml' to '/usr/local/www/packages'
    Put you files to '/usr/local/www/packages/config'

    • need use external FTP server, because on www server .inc files downloaded wrong

    NOTE:
    !!! for http access to you files - create and put to '/usr/local/www/packages' and '/usr/local/www/packages/config' index.php files with catalog list !!!



  • I just red everithing 3 times and not really sure if i really know what to do  >:(
    Why wouldnt you do the IDE for packages? Also how about ability to read packaes from manual inputed servers as core GUI (no editing)



  • @alexus:

    I just red everithing 3 times and not really sure if i really know what to do  >:(
    Why wouldnt you do the IDE for packages? Also how about ability to read packaes from manual inputed servers as core GUI (no editing)

    Huh?



  • what?



  • ???  <- thats the equivalent for Huh.

    Try using complete english sentences! For example I don't understand whether you are requesting an IDE plugin for developing pfSense packages…

    Cheers
    Daniel S. Haischt



  • i know what huh means…
    as to IDE, i just would apreciate if you have some more tools for easier package development... because I alredy have the tested script that i manually added to Advanced Config page, but now I need to put togather a package and i need to learn all your structure and related stuff which i dont really can use anywhere else... so you could of build the tool that would automaticaly generate the package ...



  • So how would such a tool look like? Are you thinking about an Eclipse plugin for example?

    Regards
    Daniel S. Haischt



  • well Eclipse is more like a compiler, you dont really need tthat advanced one, waht I woud want to see is something like gui where you can add files and directories of your app (like my has 3 files and 1 dir with a 1 file in it) and then add other text config and ability automatically either to submit to common test server or submit for appruval to add to main server. THen if you running Dev version you can view pakages on the common test server



  • From Wikipedia:

    Eclipse (software), a Java integrated development environment (IDE) and platform for rich client applications originally created by IBM

    From where did you get the information, that Eclipse is a compiler - Eclipse ships with a custom compiler or you are able to use Java, C/C++ or whatever compiler types from other vendors like Sun.

    At the moment I don't know a pfSense developer that may be interested in developing a GUI based application for pfSense package developement.

    Regards
    Daniel S. Haischt



  • @DanielSHaischt:

    From Wikipedia:

    Eclipse (software), a Java integrated development environment (IDE) and platform for rich client applications originally created by IBM

    From where did you get the information, that Eclipse is a compiler - Eclipse ships with a custom compiler or you are able to use Java, C/C++ or whatever compiler types from other vendors like Sun.

    At the moment I don't know a pfSense developer that may be interested in developing a GUI based application for pfSense package developement.

    Regards
    Daniel S. Haischt

    Nah, I see no need for one.  The only thing that we really need to do is document the process a bit better.



  • well maybe better docs could help but so far i put aside developing of a pacage and will just do a custom image for myself :-P



  • @alexus:

    well maybe better docs could help but so far i put aside developing of a pacage and will just do a custom image for myself :-P

    After all the help that folks have given so far for this?  Seems a bit unfair, no?



  • well what can i do?
    I just cant get it how to buid the package….



  • minor corrections:

    • Wikipedia is wrong about IBM - Eclipse was developed by OTI Labs, Zürich which is a subsidiary of IBM

    • I didn't want to give the impression that a pfSense GUI for package developement would be necessary

    :P

    Regards
    Daniel S. Haischt



  • @alexus:

    well what can i do?
    I just cant get it how to buid the package….

    Have you looked at the other packages in the system?  It's pretty straight forward.



  • yes that what i was doin all day toda…

    i got killed when i places xmlrpc on my host, i dono it it had to have only one file but it keeps giving me string error in line 1005 or something like that



  • @alexus:

    i got killed when i places xmlrpc on my host, i dono it it had to have only one file but it keeps giving me string error in line 1005 or something like that

    I really love such issue reports. I bet the PHP parser didn't give you the error error in line 1005 or something like that. Parsers and compilers are very specific about errors, so would you mind to provide the complete error if you are interested in getting any help?

    Cheers
    Daniel S. Haischt



  • Warning: Unexpected character in input: ' in /home/vipelite/public_html/pubnet/pfSense/xmlrpc.php on line 4205

    Warning: Unexpected character in input: ' in /home/vipelite/public_html/pubnet/pfSense/xmlrpc.php on line 4205

    Warning: Unexpected character in input: ' in /home/vipelite/public_html/pubnet/pfSense/xmlrpc.php on line 4205

    Warning: Unexpected character in input: ' in /home/vipelite/public_html/pubnet/pfSense/xmlrpc.php on line 4205

    Warning: Unexpected character in input: ' in /home/vipelite/public_html/pubnet/pfSense/xmlrpc.php on line 4205

    Warning: Unexpected character in input: ' in /home/vipelite/public_html/pubnet/pfSense/xmlrpc.php on line 4205

    Warning: Unexpected character in input: ' in /home/vipelite/public_html/pubnet/pfSense/xmlrpc.php on line 4205

    Warning: Unexpected character in input: ' in /home/vipelite/public_html/pubnet/pfSense/xmlrpc.php on line 4205



  • @alexus:

    Warning: Unexpected character in input: ' in /home/vipelite/public_html/pubnet/pfSense/xmlrpc.php on line 4205

    Warning: Unexpected character in input: ' in /home/vipelite/public_html/pubnet/pfSense/xmlrpc.php on line 4205

    Warning: Unexpected character in input: ' in /home/vipelite/public_html/pubnet/pfSense/xmlrpc.php on line 4205

    Warning: Unexpected character in input: ' in /home/vipelite/public_html/pubnet/pfSense/xmlrpc.php on line 4205

    Warning: Unexpected character in input: ' in /home/vipelite/public_html/pubnet/pfSense/xmlrpc.php on line 4205

    Warning: Unexpected character in input: ' in /home/vipelite/public_html/pubnet/pfSense/xmlrpc.php on line 4205

    Warning: Unexpected character in input: ' in /home/vipelite/public_html/pubnet/pfSense/xmlrpc.php on line 4205

    Warning: Unexpected character in input: ' in /home/vipelite/public_html/pubnet/pfSense/xmlrpc.php on line 4205

    Most likely a copy and paste error…



  • copy paste of the file?
    well should it be only one file or more then that?
    I looked inside and it referes to other files which are definetly not in the directory



  • @alexus:

    copy paste of the file?
    well should it be only one file or more then that?
    I looked inside and it referes to other files which are definetly not in the directory

    Start from square one.

    http://forum.pfsense.org/index.php/topic,312.msg16925.html#msg16925



  • so?
    I alredy memorized that page



  • @alexus:

    so?
    I alredy memorized that page

    Start over from scratch.  You obviously missed a step.



  • k



  • Having just spent ages trying to get this to work. There is one obvious (in the end) thing that was getting in the way of it working for me.
    If you run your xml-rpc server on pfsense you can't have the web gui in https mode as the xml-rpc stuff is all done in http

    Well now it is working I just need to finish the package  ;)

    Maybe it would help people make packages if the xml-rpc server and other bits in this thread could be made into a package to install.



  • @sullrich:

    Download http://www.pfsense.com/xmlrpc.tgz …  Extract to root of your webserver.  Or to pfSense's root.

    Is the xmlrpc.tgz file still needed?  It appears to be AWOL.

    Does /home/pfsense/www on the development install contain the contents of xmlrpc.tgz?



  • Check out the cvstrac page, and look in the www directory.  That should have all the files you need.



  • The XMLRPC package repo has been moved to: https://rcs.pfsense.org/projects/xmlrpc-server/


Log in to reply