How to build pfSense on FreeBSD V10



  • Hey guys…
    I just got some of the free time i wanted so bad LOL ( Got laid off )

    Which now gives me time to thinker a bit :P
    NOW...

    I searched around and only found this: https://devwiki.pfsense.org/BuildingpFSense
    But it seems a bit "ancient" LOL

    and a deeper search was fruitfull
    http://devwiki.pfsense.org/DevelopersBootStrapAndDevIso

    Any updates or hints ?



  • This is a bit of a rough "sketch" but I'm keeping track of everything as I go along…

    NOTES:

    On FreeBSD10 you have to install the following packages manually

    GIT

    #pkg install git

    FASTEST_CVSUP

    #pkg install fastest_cvsup

    Next step:

    mkdir /home

    mkdir /home/pfsense

    git clone git://github.com/pfsense/pfsense-tools.git tools

    git clone git://github.com/pfsense/freesbie2.git freesbie2



  • And a wall….

    No files present in the directories or for that matter any files in the folders



  • @_Adrian_:

    mkdir /home

    mkdir /home/pfsense

    git clone git://github.com/pfsense/pfsense-tools.git tools

    git clone git://github.com/pfsense/freesbie2.git freesbie2

    Got it !!!
    Make sure you are in # /home/pfsense and not in # /root !!

    Moving along…



  • And stuck again…
    https://devwiki.pfsense.org/DevelopersBootStrapAndDevIso#hn_4._Ensure_FreeBSD_sources_are_present

    Even after editing /usr/share/examples/cvsup/standard-supfile
    From:

    *default host=CHANGE_THIS.FreeBSD.org

    To:

    *default host=cvsup.FreeBSD.org

    I still get the following Error :

    Name Lookup failure for "fastest_cvsup -c tld -q": hostname nor servname provided, or not known

    Will retry at xx:xx:xx (time)



  • Quickly rolled up my sleeves and started into seeing if its my network or the host…
    And what do you know :P

    C:\Users\Admin>ping cvsup.freebsd.org

    Pinging bsn0-0.edmz.cwu.edu [72.233.193.64] with 32 bytes of data:
    Request timed out.
    Request timed out.
    Request timed out.
    Request timed out.

    Ping statistics for 72.233.193.64:
        Packets: Sent = 4, Received = 0, Lost = 4 (100% loss),

    I guess this ones down :/
    Next…

    C:\Users\Admin>ping cvsup3.us.freebsd.org

    Pinging xyz.csail.mit.edu [128.31.0.28] with 32 bytes of data:
    Reply from 128.31.0.28: bytes=32 time=651ms TTL=51
    Reply from 128.31.0.28: bytes=32 time=108ms TTL=51
    Reply from 128.31.0.28: bytes=32 time=109ms TTL=51
    Reply from 128.31.0.28: bytes=32 time=110ms TTL=51

    Ping statistics for 128.31.0.28:
        Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
    Approximate round trip times in milli-seconds:
        Minimum = 108ms, Maximum = 651ms, Average = 244ms

    Looks like I have some editing to do again…
    Hope this works :/



  • And still nothing >:(

    DEV / MODS…

    Any input ???


  • Netgate Administrator

    @_Adrian_:

    I just got some of the free time i wanted so bad LOL ( Got laid off )

    Ouch!  :(

    So you're trying to build 2.2 effectively?

    It's not possible to build 2.1 on top of FreeBSD 10. Most of the work that has gone into the 2.2 branch is to allow it to be built on 10.

    Not that I've tried myself of course!  ;)

    Steve



  • Yeah… I'm trying my FreeBsd skills out LOL

    I figured and try my hand at a 2.2 build with the OFED Stack



  • Well… some progress is made !
    Last night I was too tired and hopped up on Mountain Dew Voltage LOL

    I was using

    csup -h fastest_cvsup -c tld -q /usr/share/examples/cvsup/standard-supfile

    This morning i looked closer to the Wiki and saw this…

    Some ports require sources in /usr/src to build correctly.

    So instead really should have been using this

    csup -h fastest_cvsup -c tld -q /usr/src

    Now I get the following message

    Release not specified for collection "s"

    I guess im on the right track …
    Moving on and omitting the message for now LOL



  • Well.. so far so good…
    I choose the RELENG_8_3 :D

    Right now its at :

    Obtaining FreeBSD Sources RELENG_8_3-supfile. . .

    Keeping fingers crossed and hopeing it works !!


  • Netgate Administrator

    Trying and failing is better than not trying at all. You've got me beat already.  :)

    Steve



  • DONE !!
    Woohohohooo

    Not a single error too !!

    Moving onto Step 5.2A…

    Now on 5.2B



  • Ugh… errors galore !!!

    However...

    Plan for failure
    It is expected that the first try to build the ports, and the first try to build an ISO will fail, but subsequent attempts work. This is because certain parts expect files or binaries to be present that get built on the first run but not fully used until the second. This may get fixed eventually, but because the workaround is easy (just run the build again), it's not a priority.

    I ran it again and still full of errors…
    guessing thats due to V10 but i mainly think its because of this first initial error:

    Operation ./build_pfPorts.sh has started at Sun Jan 26 12:26:30 MST
      –> Preparing for pfPorts build buildports.RELENG_2_1
      --> WARNING! We are about to run make includes.
      --> Press CTRL-C to abort this operation. . . . .
    ==> Starting make includes operation. . .
    make: "/usr/pfSensesrc/src/Makefile" line 178: warning : Couldn't read shell's output for "find /usr/pfSensesrc/src/sys/sys/param.h -mtime -0s"
    make[2]: "/usr/pfSensesrc/src/Makefile" line 178: warning : Couldn't read shell's output for "find /usr/pfSensesrc/src/sys/sys/param.h -mtime -0s"
    make[2]: "/usr/pfSensesrc/src/Makefile" line 178: warning : Couldn't read shell's output for "find /usr/pfSensesrc/src/sys/sys/param.h -mtime -0s"

    ==> Compiling pfPorts. . .

    Executing Build ports.RELENG_2_1
    Setting ports List : '/home/pfSense/tools/builder_scripts/../builder_scripts/conf/pfPorts/buildports.RELENG_2_1'
    Setting MAKEJ_ports (-j) to ' '
    Setting ports make.conf: '/tmp/pfPorts_make.conf'

    After this everything is a catastrophe :/

    Build [PACKAGE] (filename). . . Failed to build. Error log in /tmp/pfPorts/buildlogs/[PACKAGE]



  • After a bit of messing around I got to menu.sh and found that there is an option that's not displayed on the wiki…
    RELENG_10_0

    Also edited and instead of merging with head I forced it to merge with 8.3
    I Will try again...

    Right now its at
    " Obtaining FreeBSD sources RELENG_10_0-supfile. . . "



  • Thanks for posting your progress on this.

    I'm interested in building for freeBSD 10, for the improved Xen support that 10 has.

    I just started downloading a v10 ISO to set up a build environment.



  • No luck yet…
    Just chasing my tail here... I'm pretty close, but something is causing the build to fail.
    So will have to keep poking at it until I either figure it out or someone drops in with a few words of wisdom...



  • After a bit of messing around I got to menu.sh and found that there is an option that's not displayed on the wiki…
    RELENG_10_0

    Since FreeBSD 10 and pfSense 2.2 is brand new, the detail of the build instructions in the WiKi is going to talk about (FreeBSD 8.1 and pfSense 2.0.n) or (FreeBSD 8.3 and pfSense 2.1.n). So you will definitely have to always change anything like that, and use an educated guess to do a similar thing for (FreeBSD 10 and pfSense 2.2). And then work out what underlying stuff has changed in some non-obvious way also (either in FreeBSD itself, or the way it is built for pfSense).
    Once the devs actually have a reasonable (FreeBSD 10 and pfSense 2.2) build themselves then I'm sure the WiKi will get updated, and there will be public snapshots.



  • Thanks for the input phil.davis !

    From what read / heard 2.2 will be built on V9 not on V10.
    Mainly I'm after a running OFED stack…



  • From what read / heard 2.2 will be built on V9 not on V10.

    pfSense 2.2 is definitely now based on FreeBSD 10 (there will not be any pfSense for FreeBSD 9).
    You have already posted to this thread: https://forum.pfsense.org/index.php/topic,69856.0.html - which has a few people interested and waiting for a useable FreeBSD 10 + pfSense 2.2 build to pop out.



  • Sorry…
    Today i just been off my rocker LOL

    I think tomorrow is a fresh start day... format and re-install everything from scratch.



  • No updates as of yet…
    I have been spending my time on the minivan that needs a motor. Will probably get back on this on the weekend



  • Maybe this will help someone :D
    But i could come to building an .iso
    These are the steps i did:

    
    DISCLAIMER these steps seemed to work for me for testing purposes.
    but i dont know if they are "correct" and wont contain serius issues.
    use at your own risk !
    
    Also i noticed after using the build image and going through the webgui installation wizard, at the last step a system panic occurs...
    After rebooting and resetting the webgui (option 11), most stuff seams to work..
    Im not sure if this is an issue caused by the early version and development progress on pfSense 2.2 or if it is caused by my steps..
    
    install VirtualBox and configure a virtual machine like this:
     - disk 20GB
     - mem  1024 MB
     - cpu  2x
     - network: 1 bridged
    
    0\. Install FreeBSD
    install:  FreeBSD-10.0-RELEASE-amd64-disc1.iso
    	default key map
    	options: lib32 ports src  NOT: doc,games
    	disk:  automatic  (19GB disk / 1GB swap)
    
    	-root pass: system
    	-configure network interface 'em0' IPv4: dhcp
    	-country: yours
    	-services: sshd,dmpdev
    
    	add users now? YES
    		username: admin
    		fullname: admin
    		uid: <enter>logingroup: <enter>invite other groups: wheel
    		class [default]: <enter>shell [sh]: <enter>home dir [/home/admin]: <enter>permissions default: <enter>password-based auth [yes]: <enter>empty/random: no
    		pass: secret
    	lockout[no]: <enter>another user: no
    
    	## Final Configuration ##
    		Exit
    		Manual Configuration?: No
    		reboot
    ## the system is now installed
    
    	## to enable SSH login on console ##
    		#become ROOT
    		su -l
    		#generate keys not needed: /usr/bin/ssh-keygen -A
    		echo sshd_enable="YES" > /etc/rc.conf
    		service sshd start
    
    	## allow root to login over ssh (NEVER DO THIS on a production machine, only for testing/easy access)
    		echo PermitRootLogin yes > /etc/ssh/sshd_config
    		/etc/rc.d/sshd reload
    
    	## SYSTEM NOW ENABLED for SSH root login ##
    		login with PuTTY / SSH as root for running commands / WinSCP for editing and transfering files (use the root account to have access to everything)
    
    # 0-a Update your FreeBSD Install
    	mkdir -p /usr/src/crypto/openssl/ssl
    	freebsd-update fetch
    	freebsd-update install	
    
    # 0-b Make sure your OS clock is accurate
    	ntpdate 0.pool.ntp.org
    
    #### checking out ports / sources. ####
    ### 1\. Begin pfSense bootstrap by running these shell commands:
    echo "WITHOUT_X11=yo" > /etc/make.conf
    echo "OPTIONS_UNSET=X11" >> /etc/make.conf
    echo "BATCH=yo" >> /etc/make.conf
    mkdir -p /home/pfsense/pfSenseGITREPO /usr/pfSensesrc
    
    ### 2\. Grab ports and build GIT
    portsnap fetch extract 
    cd /usr/ports/textproc/expat2 && make depends install
    cd /usr/ports/devel/git && make depends install
    cd /usr/ports/sysutils/fastest_cvsup/ && make depends install
    rehash   
    
    ### 3\. Continue on with the GIT repo checkouts:
    cd /home/pfsense && git clone git://github.com/pfsense/pfsense-tools.git tools
    cd /home/pfsense && git clone git://github.com/pfsense/freesbie2.git freesbie2
    cd /home/pfsense/tools/builder_scripts && chmod a+rx *.sh
    
    ### 4.0 prepare some settings
    echo ## Make sure to use the correct OS sources (otherwise applying patches will fail):
    cd /usr/home/pfsense/tools/builder_scripts/conf/svnup/
    sed s,base/releng/8.3,base/releng/10.0, < svnup.conf > svnup.conf.new
    sed s,base/stable/10,base/releng/10.0, < svnup.conf.new > svnup.conf.new2
    sed s,svn.freebsd.org,svn0.us-west.FreeBSD.org, < svnup.conf.new2 > svnup.conf
    
    cd /usr/home/pfsense/tools/builder_scripts/
    echo "export USE_SVN=yes" >> pfsense_local.sh
    sed s,stable/10,release/10.0.0, < pfsense_local.sh > pfsense_local.sh.new
    mv pfsense_local.sh.new pfsense_local.sh
    
    echo ## allow ports to be build against the newly installed openssl WITH_OPENSSL_PORT=yes
    cd /usr/home/pfsense/tools/builder_scripts/conf/pfPorts/
    sed s,WITH_OPENSSL_BASE,WITH_OPENSSL_PORT, < make.conf > make.conf.new
    mv make.conf.new make.conf
    
    ### 4\. Ensure FreeBSD sources are present
    cd /usr/ports/devel/subversion && make install clean
    rehash
    svn checkout https://svn0.us-west.FreeBSD.org/base/releng/10.0/ /usr/src
    
    cp -iprv /usr/src /usr/pfSensesrc/
    
    ### 5.1\. Set the version you would like to build, you can do this from ./menu.sh also.
    cd /home/pfsense/tools/builder_scripts
    ./set_version.sh RELENG_10_0
    
    ### 5.2\. Invoke the Apply patches option
    cd /home/pfsense/tools/builder_scripts
    ./apply_kernel_patches.sh
    
    ## 5.2a Ensure BSDInstaller is sound
    rm -rf /home/pfsense/installer
    cd /home/pfsense/tools/builder_scripts
    scripts/get_bsdinstaller.sh ; scripts/rebuild_bsdinstaller.sh
    
    ## 5.2b Build pfPorts manually (optional)
    	## when rebuilding ports, some ports want/need the base system openssl version..
    	## deinstall openssl port:
    cd /usr/ports/security/openssl
    make deinstall
    cd /home/pfsense/tools/builder_scripts
    ./build_pfPorts.sh
    	## troubleshooting building ports, check the 'failure logs' in:  /tmp/pfPort/buildlogs/  (should be empty when building is done)
    	## for an idea about progress, check current console output and log in buildlogs folder and 
    	## check progress against the file /usr/home/pfsense/tools/builder_scripts/conf/pfPorts/buildports.RELENG_2_2
    
    ## 5.3.0 to avoid error building cdrtools-3.00 "cc: error: no such file or directory: 'amd64'" while running build_iso.sh
    cd /usr/ports/sysutils/cdrtools
    make install clean	
    
    ## 5.3 Invoke the Build ISO option (no matter what kind of image you want)
    	## now start the compiling and building of memstick images and iso file
    
    cd /home/pfsense/tools/builder_scripts
    ./build_iso.sh
    
    ## p.s. this message is normal: "/usr/local/pfsense-clone/usr: write failed, filesystem is full"
    ## Which is explained a few lines above that.: "Don't worry if you see a 'filesystem full' message here" 
    
    ## now cross your fingers and wait for the message to show it has finished building which should look like this:</enter></enter></enter></enter></enter></enter></enter></enter> 
    

    p.s. if anyone knows some improvements to these steps please let me/us know!



  • These instructions seems more complicated than it really is.
    Also you would want the OPENSSL from base.



  • maybe indeed you want openssl from base, however then in my attempts then the build process isn't successful. as part of the ports compilation the newer openssl is installed again and the other ports complain they want base.
    i know building ports manually can probably be skipped, as make_iso also builds the ports… but the official wiki also does that. and it is usefull when trouble does happen.

    building cdrtools shouldnt be needed either, but solved getting the .iso besides the memstick versions which did build without that step.
    changing the svnup.conf fixed applying patches

    please tell us your "simpler" but successful build steps. as the steps in wiki didnt seem to work 100% for people (or at least for myself)



  • you can avoid installing OpenSSL ports by using portmaster. It always presents you NCurse based "Option" menu, often you can choose other options instead of using OpenSSL, for example often it allows you choosing GnuTLS instead when some package asks for SSL support.

    I did it with the Git. Did'nt install it by pkg install but went to /usr/ports/devel/git, started portmaster in it and laborously avoided choosing OpenSSL port anywhere. OpenSSL is always default choice, so you must be careful.

    If the selected options do not play out and fail, you can always go to the folder of failing port and do make config (and make) in it again, and then build the port solo. Then go back to where portmaster failed and start it again. Portmaster jumps over point of previous failure and goes on.
    So, step by step you can build necessary packages.

    Got stuck with building ipfw-classifyd module though. Whatever I do, I get error. No clue about C so I am unable fixing it myself:S

    Overall, 2.2 seems to build quite error free in it's present state

    PS. Doing it for my own pleasure and interest, no reason to be afraid of sharing out buildimages..



  • Anybody else had any luck with this ??
    I've gotten called back in and working 60hour week again :/



  • I managed to built couple of isos and tinker with them. Kept bumping into various issues but it sure was interesting. No time lately.

    Btw, at some point found that pfsense-tools had been removed from github. If you happen to have your local backups then better keep'em



  • New pfSense features are being added to the "hidden" pfSense-tools and then used in the "ordinary" pfSense code. So even if you build from an old copy of pfSense-tools, the compiled "pf" and other utilities will be missing some command line options, conf file features… and so will not work with the PHP code in the main pfSense repo that uses the new features.
    2.2 snapshots are now available at http://snapshots.pfsense.org/ - use those and join in the testing and debugging. Then we all benefit from a joint co-ordinated effort.



  • As I understand now you torpedoed the users ability to build from source on purpose? Why, so suddenly?



  • "you" should refer to Electric Sheep Fencing (ESF) - if you were referring to me in some way, then please understand that I am a private individual and am not part of ESF.
    You can read up on recent happenings in this post: https://forum.pfsense.org/index.php?topic=73101.0 and others in the Development forum section.



  • no, did not mean You personally. More general way.

    thanks for the link, it did lead me to a answer at
    http://lists.pfsense.org/pipermail/dev/2014-February/000520.html



  • trying to build 2.2 beta. get the error while building strongswan as shown in attachment

    –- kernel_pfkey_plugin.lo ---
    libtool: compile:  cc -DHAVE_CONFIG_H -I. -I../../../.. -I../../../../src/include -I../../../../src/libstrongswan -I../../../../src/libhydra -I/usr/local/include -O2 -pipe -fno-strict-aliasing -include /usr/ports/security/strongswan/work/strongswan-5.2.0/config.h -MT kernel_pfkey_plugin.lo -MD -MP -MF .deps/kernel_pfkey_plugin.Tpo -c kernel_pfkey_plugin.c  -fPIC -DPIC -o .libs/kernel_pfkey_plugin.o
    --- kernel_pfkey_ipsec.lo ---
    libtool: compile:  cc -DHAVE_CONFIG_H -I. -I../../../.. -I../../../../src/include -I../../../../src/libstrongswan -I../../../../src/libhydra -I/usr/local/include -O2 -pipe -fno-strict-aliasing -include /usr/ports/security/strongswan/work/strongswan-5.2.0/config.h -MT kernel_pfkey_ipsec.lo -MD -MP -MF .deps/kernel_pfkey_ipsec.Tpo -c kernel_pfkey_ipsec.c  -fPIC -DPIC -o .libs/kernel_pfkey_ipsec.o
    --- kernel_pfkey_plugin.lo ---
    libtool: compile:  cc -DHAVE_CONFIG_H -I. -I../../../.. -I../../../../src/include -I../../../../src/libstrongswan -I../../../../src/libhydra -I/usr/local/include -O2 -pipe -fno-strict-aliasing -include /usr/ports/security/strongswan/work/strongswan-5.2.0/config.h -MT kernel_pfkey_plugin.lo -MD -MP -MF .deps/kernel_pfkey_plugin.Tpo -c kernel_pfkey_plugin.c -o kernel_pfkey_plugin.o >/dev/null 2>&1
    --- kernel_pfkey_ipsec.lo ---
    kernel_pfkey_ipsec.c:829:23: error: use of undeclared identifier 'SADB_X_EALG_AESGCM8'
            {ENCR_AES_GCM_ICV8,            SADB_X_EALG_AESGCM8    },
                                            ^
    kernel_pfkey_ipsec.c:830:24: error: use of undeclared identifier 'SADB_X_EALG_AESGCM12'
            {ENCR_AES_GCM_ICV12,            SADB_X_EALG_AESGCM12    },
                                            ^
    kernel_pfkey_ipsec.c:831:24: error: use of undeclared identifier 'SADB_X_EALG_AESGCM16'
            {ENCR_AES_GCM_ICV16,            SADB_X_EALG_AESGCM16    },
                                            ^
    3 errors generated.
    *** [kernel_pfkey_ipsec.lo] Error code 1

    make[5]: stopped in /usr/ports/security/strongswan/work/strongswan-5.2.0/src/libhydra/plugins/kernel_pfkey
    1 error

    make[5]: stopped in /usr/ports/security/strongswan/work/strongswan-5.2.0/src/libhydra/plugins/kernel_pfkey
    *** [all-recursive] Error code 1

    make[4]: stopped in /usr/ports/security/strongswan/work/strongswan-5.2.0/src/libhydra
    1 error
    make[4]: stopped in /usr/ports/security/strongswan/work/strongswan-5.2.0/src/libhydra
    *** [all-recursive] Error code 1

    make[3]: stopped in /usr/ports/security/strongswan/work/strongswan-5.2.0/src
    1 error
    make[3]: stopped in /usr/ports/security/strongswan/work/strongswan-5.2.0/src
    *** [all-recursive] Error code 1
    make[2]: stopped in /usr/ports/security/strongswan/work/strongswan-5.2.0
    1 error

    make[2]: stopped in /usr/ports/security/strongswan/work/strongswan-5.2.0
    *** [all] Error code 2
    make[1]: stopped in /usr/ports/security/strongswan/work/strongswan-5.2.0
    1 error
    make[1]: stopped in /usr/ports/security/strongswan/work/strongswan-5.2.0
    ===> Compilation failed unexpectedly.
    Try to set MAKE_JOBS_UNSAFE=yes and rebuild before reporting the failure to
    the maintainer.
    *** Error code 1
    Stop.
    make: stopped in /usr/ports/security/strongswan

    openssl was set to base in make.conf. does it need to be changed to openssl with port?



  • I've got FreeBSD 10.1-BETA2 base system set up, (ports like git, expat2 etc. all installed). Also, following (by now erased) guidelines in old dev wiki (still visible using Internet Archive 'going back in time'). pfsense-tools repo installed (after signing necessary agreements in portal.pfsense.org)

    Questions.
    -While the April version of the 'pfsense-tools' worked in tcsh shell pretty much out-of-the-box, at some point that code seems to have received major overhaul. Trying to pick some option from (/builder_scripts/scripts/)menu.sh ends invariably with an error 'filename.sh :not found'.
    I suspect I would need to add some custom variable to the .tcshrc file.  Could someone 'in the know' help me out?

    • Is there going to be another 'dev wiki'? Oriented for development of 3rd party plugins. Present help files seem to be oriented purely for installation and usage of pfSense. Old 'dev help' links still present in 'pfsense-tools' scripts are by now inoperable.


  • I received an error stating "You must upgrade the ports-mgmt/pkg port first."

    To solve that issue:

    cd /usr/ports/ports-mgmt/pkg  
    make  
    make reinstall  
    

Log in to reply