Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    Building pfSense 2.2 / FreeBSD 10.1 - WORKING STEPS

    Scheduled Pinned Locked Moved Development
    50 Posts 14 Posters 21.4k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • rbgargaR
      rbgarga Developer Netgate Administrator
      last edited by

      @madscientist159:

      The –build-pfPorts option to ./build.sh no longer exists.  What replaced it?

      Thanks!

      Are you using branch RELENG_2_2? This is the branch to build 2.2 images, and it still contains –build-pfPorts

      Renato Botelho

      1 Reply Last reply Reply Quote 0
      • M
        madscientist159
        last edited by

        @Renato:

        Are you using branch RELENG_2_2? This is the branch to build 2.2 images, and it still contains –build-pfPorts

        That did the trick.  Thanks for the pointer in the right direction! :D

        Don't let the low karma fool you…wading into a politically charged discussion tends to be a bad idea...

        1 Reply Last reply Reply Quote 0
        • I
          irah
          last edited by

          Hi PiBa,

          I am trying to build the latest pfsense using your steps and I believe the same steps might apply now too.
          I don't find any of the aforementioned scripts (rebuild_bsdinstaller.sh and get_bsdinstaller.sh) in the repository, for rebuilding the bsdinstaller (step# 6).
          I mean for building pfsense 2.2, neither master nor RELENG_2_2 branch has those scripts. Only RELENG_2_1 has those scripts, but then it's on top of freebsd 8.3.
          Am I wrong in the steps or the repository for building it? Can you please help me out?

          P.S. These were the most recent steps that I could find for building the latest pfsense.

          Thanks.

          1 Reply Last reply Reply Quote 0
          • I
            irah
            last edited by

            Thanks PiBa, your suggestion helped! The steps mentioned earlier in this thread are applicable for the latest build too, only change being in step# 6, where bsdinstaller has been integrated into step# 7 as a static port.

            How to build: – pfSense 2.2.5 FreeBSD 10.1-PRERELEASE --

            0 - Gain access to the pfsense-tools

            Use PuTTYgen to generate a "SSH-2 RSA" key and Export the OpenSSH key to a file locally for usage later.
            also save the 'ssh-rsa ..... key-YYYYMMDD' public key to fill in the online registration form.

            Information about how the signup works also here: https://forum.pfsense.org/index.php?topic=76132.msg415051#msg415051
            -Sign the ICLA or CCLA agreement online https://www.pfsense.org/about-pfsense/#cla
            -then also sign the LA, and configure your public ssh key, that will gaining you access to the pfsense-tools repository after a few minutes.

            my VirtualBox build machine configuration

            disk : 20GB
            vcpu : 4  << more could be faster..
            mem  : 3 GB is enough to use 'memory disks' while compiling which speeds up the build process, if you have plenty more probably wont hurt ;)

            • download and install into the virtual machine : http://ftp.freebsd.org/pub/FreeBSD/snapshots/amd64/amd64/ISO-IMAGES/10.1/FreeBSD-10.1-PRERELEASE-amd64-2014????-r27????-disc1.iso
              Then also update it to the recent version:
              freebsd-update fetch
              freebsd-update install
              -reboot after that

            After this "uname -r" shows me:  10.1-RELEASE-p16

            -install: lib & src
            -enable: ssh
            -configure user: admin

            allow root to login over ssh / for easy access with winscp and putty (NEVER DO THIS on a production machine, only for testing/easy access)

            echo PermitRootLogin yes >> /etc/ssh/sshd_config
            service sshd restart

            Login with WinSCP to be able to easely transfer files from/to the FreeBSD build machine.

            • copy your earlier saved ssh private key to the system using WinSCP to: /home/admin/private_putty_key_ssh.ppk

            1. Begin pfSense bootstrap by running these shell commands:

            echo "WITHOUT_X11=yo" > /etc/make.conf
            echo "OPTIONS_UNSET=X11" >> /etc/make.conf
            echo "BATCH=yo" >> /etc/make.conf
            mkdir -p /home/pfsense/pfSenseGITREPO /usr/pfSensesrc

            portsnap fetch extract
            cd /usr/ports/devel/git && make depends install
            rehash

            2. Load SSH key and checkout sources for pfSense-tools from RELENG_2_2 branch

            eval ssh-agent -c
            chmod 600 /home/admin/private_putty_key_ssh.ppk
            ssh-add /home/admin/private_putty_key_ssh.ppk

            cd /home/pfsense && git clone -b RELENG_2_2 git@git.pfsense.org:pfsense-tools tools
            -Are you sure you want to continue connecting (yes/no)? yes  << required to type all 3 letters of y e s

            3. Set correct publicly available repositories

            'pfmechanics' is used by ESF internally for fast local mirrors of several repositories to speed up their build process it is not available from outside.

            p.s. dont try to use git@ for github, it wil fail later because it will try to use the ssh key loaded above..

            sed -i "" "s,git@git.pfmechanics.com:pfsense/pfsense-tools.git,git@git.pfsense.org:pfsense-tools," /usr/home/pfsense/tools/builder_scripts/builder_defaults.sh
            sed -i "" "s,git@git.pfmechanics.com:outsidemirrors/freebsd.git,https://github.com/freebsd/freebsd.git," /usr/home/pfsense/tools/builder_scripts/builder_defaults.sh
            sed -i "" "s,git@git.pfmechanics.com:pfsense/pfsense.git,https://github.com/pfsense/pfsense.git," /usr/home/pfsense/tools/builder_scripts/builder_defaults.sh
            sed -i "" "s,git@git.pfmechanics.com:pfsense/bsdinstaller.git,https://github.com/pfsense/bsdinstaller.git," /usr/home/pfsense/tools/builder_scripts/builder_defaults.sh

            to check if the above sed commands went ok, the command below should show nothing:

            cat /usr/home/pfsense/tools/builder_scripts/builder_defaults.sh | grep git.pfmechanics.com

            4. Checkout Freesbie2

            cd /home/pfsense && git clone git://github.com/pfsense/freesbie2.git freesbie2

            5. set version to build

            cd /home/pfsense/tools/builder_scripts
            ./set_version.sh RELENG_2_2 https://github.com/freebsd/freebsd.git

            6. Rebuild the bsdinstaller.

            This step can be skipped, creation of the bsdinstaller has been integrated into the script called at step7. It's now converted into a static port and is available under pfPorts.

            7. Build ports

            Now it will also be checking out FreeBSD sources, this will take quite some time with no visible progress, in the background the 'git' process will take a little cpu usage..

            tmp file in /usr/pfSensesrc/src/.git/objects/pack will slowly grow over 800MB..

            cd /home/pfsense/tools/builder_scripts
            ./build.sh --build-pfPorts --no-cleanrepos

            to check progress see what ports will be build in: /usr/home/pfsense/tools/builder_scripts/conf/pfPorts/buildports.RELENG_2_2

            It should show all ports build without failures..

            >>> Ports with failures:        0
              >>> Building tools/crytpo...Done!
              ==> End of pfPorts...
              >>> Operation ./build.sh has ended at Wed Nov  5 22:16:57 CET 2014

            7. Build pfSense iso

            cd /home/pfsense/tools/builder_scripts
            ./build.sh iso --no-cleanrepos

            ISO created: Wed Nov  5 22:53:02 CET 2014
            NOTE: waiting for jobs:  to finish...
            /tmp/pfSense/ now contains:
            total 239208
            drwxr-xr-x  2 root  wheel  512B Nov  5 22:53 .
            drwxrwxrwt  10 root  wheel  512B Nov  5 22:52 ..
            -rw-r--r--  1 root  wheel  234M Nov  5 22:53 pfSense-LiveCD-2.2-BETA-amd64-20141105-2218.iso
            -rw-------  1 root  wheel    0B Nov  5 22:53 pfSense-LiveCD-2.2-BETA-amd64-20141105-2218.iso.gz
            Operation ./build.sh has ended at Wed Nov  5 22:53:02 CET 2014

            8. YOUR DONE (hopefully)

            The resulting iso should show
            ls -l /tmp/pfSense/

            -rw-r--r--  1 root  wheel  90629317 Nov  5 22:53 pfSense-LiveCD-2.2-BETA-amd64-20141105-2218.iso.gz

            p.s.

            Im using --no-cleanrepos flag above to speedup second and further build attempts(the same day?) by skipping the slow checkout of FreeBSD sources.

            LOGS

            Kernel buildworld log:
            /usr/home/pfsense/tools/logs/buildworld.amd64
              ports building (failure) logs (empty is good, during building there can be temporary logfiles here):
            /tmp/pfPort/buildlogs/

            To apply a custom patch written against FreeBSD code this is one way to apply it

            • add patch file divert-reply10.1.patch to : /usr/home/pfsense/tools/patches/releng/10.1
            • add the patch to the list of patches, depending on the format of the patch git/diff you might need to specify it a little different.:
                  echo "-p1~~divert-reply10.1.patch~" >> /usr/home/pfsense/tools/builder_scripts/conf/patchlist/releng/10.1/patches
            1 Reply Last reply Reply Quote 0
            • S
              sreekupmgmail.com
              last edited by

              Hii,

              I am a new comer, Please assist me how to build own pfSense

              1 Reply Last reply Reply Quote 0
              • P
                PiBa
                last edited by

                Hi sreekupm,

                Step 1: READ the posts above yours !
                Step 2: read some more forum posts, try to do the steps, then read some more.
                Step 3: fail while trying those steps, then try to understand the error's show, gather logfiles, look in those logs for clues what might have gone wrong.
                Step 4: search the forum and use google to find possible solutions to your problems..
                Step 5: try those possible solutions.
                Step 6: repeat above steps 3 times (or more..)

                Step 7: then, and only then, clearly explain what you have done, tried and failed at, what errors you have seen, what step you could not follow..

                Now please follow these instructions and you might get a more useful reply next time..

                Some computer knowledge and is expected from a developer so we are not going to tell every mouse click you need to do..
                What would you say when i report you the problem that the program/website you created is not working properly? Wouldn't you like to know what button&page i was talking about and what behavior i expected but didn't get?

                I'm sorry if i sound a bit harsh, but please do understand that your question is useless like it is currently written..

                Regards,
                PiBa-NL

                1 Reply Last reply Reply Quote 0
                • S
                  S. Kirschner
                  last edited by

                  Hi,

                  I couldnt clone the repo's.

                  Following steps I performed
                  1. accepted Individual CLA
                  2. accepted the License Agreement (also insert my pubkey)
                  3. imported the openssh private key in my freebsd machine (eval ssh-agent -c , ssh-add /path/to/private/openssh_key
                  4. waited 1 hour
                  5. tried to clone (git clone -b RELENG_2_2 git@git.pfsense.org:pfsense-tools tools)
                  And here I´m asked for an Password (Not the password from my private key)

                  To verifiy if I made an mistake with my pub key , i uploaded the pub key to github.com and tested the verification (ssh -T git@github.com), these worked as it should and returned.
                  "Hi skirschner! You've successfully authenticated, but GitHub does not provide shell access."

                  My Pfsense profil contain my github username and the pubkey , also the subscriptions would displayed as active.

                  Have someone an idea ?

                  [root@freebsd-2836] /home/pfsense# git clone -b RELENG_2_2 git@git.pfsense.org:pfsense-tools tools
Cloning into 'tools'...
Password:
                  1 Reply Last reply Reply Quote 0
                  • P
                    PiBa
                    last edited by

                    Im not sure what is currently the best repository to clone from.. But you could try cloning from github instead. Now that you have signed the ICLA and LA you have also been added to the 'pfsense-tools team'..
                    what happens if you try? The following:

                    git clone -b RELENG_2_2 git@github.com:pfsense/pfsense-tools.git tools
                    1 Reply Last reply Reply Quote 0
                    • S
                      S. Kirschner
                      last edited by

                      Thank you , it worked for me.

                      Now i have to find out how to build a specific appilcation in a pbi.

                      1 Reply Last reply Reply Quote 0
                      • D
                        doktornotor Banned
                        last edited by

                        @S.:

                        Now i have to find out how to build a specific appilcation in a pbi.

                        Not really sure why'd anyone do that… Use pkg instead. https://doc.pfsense.org/index.php/Installing_FreeBSD_Packages#pfSense_2.2

                        1 Reply Last reply Reply Quote 0
                        • S
                          S. Kirschner
                          last edited by

                          @doktornotor:

                          @S.:

                          Now i have to find out how to build a specific appilcation in a pbi.

                          Not really sure why'd anyone do that… Use pkg instead. https://doc.pfsense.org/index.php/Installing_FreeBSD_Packages#pfSense_2.2

                          Yes allready tried it , but I would like to use some options that isnt available in "stock".

                          Thats the reason why i try to build my own pbi for the pfsense machine.

                          1 Reply Last reply Reply Quote 0
                          • P
                            PiBa
                            last edited by

                            Still using pkg instead of pbi makes sense.

                            Building a single PBI takes hours on my system.. While using a 'pkg create YourPackage' on a FreeBSD system where the package was compiled and then 'pkg add YourPackage-1.2.3.txz' on pfSense the whole process can be done in a matter of a few minutes.

                            1 Reply Last reply Reply Quote 0
                            • D
                              doktornotor Banned
                              last edited by

                              ^^^ There. Plus, worse yet, the end result with PBI is usually a completely broken thing needing tons of hacks to work.

                              1 Reply Last reply Reply Quote 0
                              • S
                                S. Kirschner
                                last edited by

                                Ok thats are some points why the pkg method gets a +  :)

                                But I try it with a pbi , for following reason:
                                1.keep the usability of the webinterface for that software
                                2.keep the possability to update the software later via the webinterface without "tons" of old useless files
                                3.publish my expierence in the forum, that other got an better idea how it works and maybe faster an update for that software could be published to all via the offical repo.

                                The software i try to build is the latest version of squid 3 (which is actual the version 3.5.8 and the latest (not fully offical) available version for pfsense is 3.5.3 at the moment).
                                And i hope if its get working that the offical version of squid 3 got updated for pfsense that all other could use the new features.

                                At the moment I stuck a bit with the default prefix path.

                                1 Reply Last reply Reply Quote 0
                                • D
                                  doktornotor Banned
                                  last edited by

                                  That's just completely wrong reason. The usability with the web GUI will be much better if you avoid PBI completely. With huge things like this, probably some 1/3 of the code could be dropped altogether if there was no PBI in the first place. Huge part of the rest could be simplified and made much less buggy.

                                  Squid 3.5.x is already compiled as PBI (https://files.pfsense.org/packages/10/All/). Upgrade never happened mostly likely because

                                  • it'd require another round of insane hacking around the PBI screwed paths in the XML/PHP package (the webGUI)
                                  • the whole binary thing doesn't work properly anyway

                                  If you want to work on Squid 3.5.x GUI integration

                                  • install Squid via pkg
                                  • take the current package code - https://github.com/pfsense/pfsense-packages/tree/master/config/squid3/34
                                  • fix the package code so that it points to the sane paths under the /usr/local prefix, ditching all the PBI and pfSense-version specific shit/generally useless garbage (if,else,blah…)
                                  1 Reply Last reply Reply Quote 0
                                  • S
                                    S. Kirschner
                                    last edited by

                                    @doktornotor:

                                    Squid 3.5.x is already compiled as PBI (https://files.pfsense.org/packages/10/All/). Upgrade never happened mostly likely because

                                    • it'd require another round of insane hacking around the PBI screwed paths in the XML/PHP package (the webGUI)
                                    • the whole binary thing doesn't work properly anyway

                                    What I dont get is , I think between 3.5.3 and 3.5.9 didnt changed so much in the file structure,
                                    why the person who build 3.5.3 couldnt fetch the new files and build 3.5.9 ?

                                    1 Reply Last reply Reply Quote 0
                                    • D
                                      doktornotor Banned
                                      last edited by

                                      Hmm? Why'd you waste hours of time building something that ain't gonna be used anywhere?

                                      1 Reply Last reply Reply Quote 0
                                      • S
                                        sreekupmgmail.com
                                        last edited by

                                        @PiBa:

                                        Hi sreekupm,

                                        Step 1: READ the posts above yours !
                                        Step 2: read some more forum posts, try to do the steps, then read some more.
                                        Step 3: fail while trying those steps, then try to understand the error's show, gather logfiles, look in those logs for clues what might have gone wrong.
                                        Step 4: search the forum and use google to find possible solutions to your problems..
                                        Step 5: try those possible solutions.
                                        Step 6: repeat above steps 3 times (or more..)

                                        Step 7: then, and only then, clearly explain what you have done, tried and failed at, what errors you have seen, what step you could not follow..

                                        Now please follow these instructions and you might get a more useful reply next time..

                                        Some computer knowledge and is expected from a developer so we are not going to tell every mouse click you need to do..
                                        What would you say when i report you the problem that the program/website you created is not working properly? Wouldn't you like to know what button&page i was talking about and what behavior i expected but didn't get?

                                        I'm sorry if i sound a bit harsh, but please do understand that your question is useless like it is currently written..

                                        Regards,
                                        PiBa-NL

                                        I had done above steps but stuck up here what I can do?

                                        [2.2.4-RELEASE][admin@pfSense.localdomain]/home/pfsense: git clone -b RELENG_2_2 git@github.com:pfsense/pfsense-tools.git tools
                                        Cloning into 'tools'…
                                        Permission denied (publickey).
                                        fatal: Could not read from remote repository.

                                        Please make sure you have the correct access rights
                                        and the repository exists.

                                        1 Reply Last reply Reply Quote 0
                                        • P
                                          PiBa
                                          last edited by

                                          Surely you did not do "step7: clearly explain what you have done, tried and failed at".
                                          Anyway..

                                          So, did you sign the ICLA or CCLA and the LA for the pfSense portal ? Did you recieve a email that you added to the pfsense-tools team? Did you configure your public ssh key to github?

                                          1 Reply Last reply Reply Quote 0
                                          • S
                                            sreekupmgmail.com
                                            last edited by

                                            Hi PiBa

                                            I already sign Individual CLA - Lifetime

                                            I am stuck up
                                            root@develop:/usr/ports/devel/git # cd /home/pfsense && git clone git@git.pfsense.org:pfsense-tools tools
                                            Cloning into 'tools'…
                                            The authenticity of host 'git.pfsense.org (208.123.73.74)' can't be established.
                                            ECDSA key fingerprint is 52:7b:22:f9:3b:7e:dd:85:85:1a:a5:98:b8:8f:7f:62.
                                            No matching host key fingerprint found in DNS.
                                            Are you sure you want to continue connecting (yes/no)? yes
                                            Warning: Permanently added 'git.pfsense.org' (ECDSA) to the list of known hosts.
                                            Password:  Which Password??
                                            Password for git@git.pfsense.org: Which Password??
                                            Password:
                                            Password for git@git.pfsense.org:
                                            Password:
                                            Password for git@git.pfsense.org:
                                            Permission denied (publickey,keyboard-interactive).
                                            fatal: Could not read from remote repository.

                                            Please make sure you have the correct access rights
                                            and the repository exists.

                                            1 Reply Last reply Reply Quote 0
                                            • First post
                                              Last post
                                            Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.