• Categories
  • Recent
  • Tags
  • Popular
  • Users
  • Search
  • Register
  • Login
Netgate Discussion Forum
  • Categories
  • Recent
  • Tags
  • Popular
  • Users
  • Search
  • Register
  • Login

pfSense compile requirements for 3rd party software

Development
8
102
26.1k
Loading More Posts
  • Oldest to Newest
  • Newest to Oldest
  • Most Votes
Reply
  • Reply as topic
Log in to reply
This topic has been deleted. Only users with topic management privileges can see it.
  • B
    bmeeks @encrypt1d
    last edited by bmeeks Feb 17, 2022, 6:00 PM Feb 17, 2022, 6:00 PM

    @encrypt1d said in pfSense compile requirements for 3rd party software:

    @bmeeks
    What do you folks use for an IDE to make browsing through C code easier than grepping? Or perhaps, do you use an IDE?

    I personally have never been too fond of IDEs. I did once work for a short time in one of the Microsoft tools when I was doing Windows-related development where I worked.

    I started programming at the literal 1s and 0s of raw machine language, then graduated to assembly. Not much in the way of IDE for that 😀. I later moved on to C, C++, C#, a touch of Java, and then some PL/SQL while working with- and administering- some Oracle databases.

    These days I do only PHP for the GUI code in the Snort and Suricata packages, and then C for making customizations in the underlying binaries.

    1 Reply Last reply Reply Quote 2
    • J
      jimp Rebel Alliance Developer Netgate @encrypt1d
      last edited by Feb 17, 2022, 6:10 PM

      @encrypt1d said in pfSense compile requirements for 3rd party software:

      What do you folks use for an IDE to make browsing through C code easier than grepping? Or perhaps, do you use an IDE?

      I use UltraEdit on Linux and Windows for most things. Not really as a full IDE, mostly as a code editor with lots of nice features. On systems where I don't have a license for that I use Kate, Notepad++, or TextMate.

      Remember: Upvote with the 👍 button for any user/post you find to be helpful, informative, or deserving of recognition!

      Need help fast? Netgate Global Support!

      Do not Chat/PM for help!

      E 1 Reply Last reply Feb 17, 2022, 6:33 PM Reply Quote 1
      • E
        encrypt1d @jimp
        last edited by Feb 17, 2022, 6:33 PM

        @jimp @bmeeks

        Thanks.
        Really just looking for something that speeds up GUI browsing on FreeBSD, not a full IDE. Tracing function calls is much easier that way.
        I was a developer for the first 10 years of my career, and miss those tools (C on VXWorks with a proprietary IDE, then moved on to C++/Visual Studio, Java/Eclipse etc). They were multimillion line code repos, so grep just didn't cut it. I googled around bit, but it gets religious fast.

        1 Reply Last reply Reply Quote 0
        • B
          bmeeks
          last edited by bmeeks Feb 19, 2022, 3:08 PM Feb 17, 2022, 6:58 PM

          One other thing I forgot to mention about the incremental building during development/troubleshooting is that I found it easier to do my initial work totally within the /usr/ports/* tree and leave the Poudriere tree alone at first. This way I can run a quick make in the /usr/ports/* directory to check for stupid C syntax errors or other coding oversights I may generate with my changes. You get immediate feedback of errors there via the console.

          When you run the ./build.sh script to fire off the build within the package builder, any compilation errors are buried in a corresponding log file down in the /usr/local/poudriere/ tree. You can find the log, but it's a bit of effort.

          So I do my initial compile in the regular /usr/ports/ tree to make sure my code compiles successfully. Then, if it does, I will copy my patch diff over to the Poudriere tree and kick off the package builder so I get a package I can copy over to pfSense and actually execute.

          1 Reply Last reply Reply Quote 1
          • E
            encrypt1d
            last edited by Feb 25, 2022, 6:26 PM

            As promised, below is a full summary of everything I needed to do to get pfSense FreeBSD port builds working in an unofficial non-Netgate environment.

            Why? My motivation was simple, and that was to develop a code fix to an issue within a package that pfSense uses, and has been broken since 2.4.5 for some configurations.

            Note - only Netgate may produce official builds with the pfSense product ID. These steps are for debugging and exploratory purposes only.

            Steps:

            1. Build a VM and install FreeBSD
            2. Install package dependencies
            3. Clone the git repos
            4. Edit the build.conf
            5. Edit the builder_common.sh
            6. Run the build setup
            7. Prepare the Poudriere environment
            8. Build ports
            9. Change code and build new package versions

            Step 1) Build a VM

            • Use your favorite virtualization software to create a FreeBSD compatible VM with as many cores and as much RAM as you can throw at it. I used 4 CPUs, and 24 GB (eventually). My initial VM had 8GB which was insufficient.

            • Download the FreeBSD ISO corresponding to your pfSense revision, in this case 2.5.2 is build on FreeBSD 12.2 STABLE.

            • Boot the VM with the FreeBSD ISO you downloaded, and be sure to use ZFS for the filesystem! I selected to include the Ports Tree and the System Source Tree in my install. You cannot create a jail on a non-ZFS filesystem.

            Step 2) Install package dependencies
            You will need these packages to get started.
            Running as root:

            pkg install git
            pkg install poudriere
            pkg install rsync
            pkg install screen
            pkg install nginx
            

            Step 3) Clone the git repos
            Make a build folder, such as /build, change to it, and then start cloning.

            mkdir /build
            cd /build
            cd /build;git clone https://github.com/pfsense/pfsense.git
            cd /build/pfsense;git checkout RELENG_2_5_2
            cd /build;git clone https://github.com/pfsense/FreeBSD-ports.git
            cd /build/FreeBSD-ports;git checkout RELENG_2_5_2
            

            You now have the following folders:

            /build/pfsense
            /build/FreeBSD-ports
            

            4) Edit the build.conf
            In the folder /build/pfsense:

            cp build.conf.sample build.conf
            

            Then edit the build.conf file and ensure you use the following options set:

            export PRODUCT_NAME="pfSense"
            export BUILD_AUTHORIZED_BY_NETGATE=yes
            export FREEBSD_REPO_BASE=https://github.com/pfsense/FreeBSD-src.git
            export FREEBSD_BRANCH="RELENG_2_5_2"
            export PKG_REPO_SERVER_DEVEL="pkg+https://beta.pfsense.org/packages"
            export PKG_REPO_SERVER_RELEASE="pkg+https://pkg.pfsense.org"
            export PKG_REPO_SERVER_STAGING="pkg+https://pkg.pfsense.org"
            export SKIP_FINAL_RSYNC=YES
            

            5) Edit the builder_common.sh script
            This file is in:

            /build/pfsense/tools/builder_common.sh
            

            You want to make 2 changes, the first is to comment out the following:

            #       if [ "${PRODUCT_NAME}" = "pfSense" -a -n "${GNID_REPO_BASE}" ]; then
            #               echo ">>> Obtaining gnid sources..."
            #               ${BUILDER_SCRIPTS}/git_checkout.sh \
            #                       -r ${GNID_REPO_BASE} \
            #                       -d ${GNID_SRC_DIR} \
            #                       -b ${GNID_BRANCH}
            #       fi
            

            Next comment out this line:

                    #pkg install ${PRODUCT_NAME}-builder
            

            6) Run the build setup

            cd /build/pfsense
            ./build.sh --setup
            

            If anything fails, you'll have to determine the reason. Assuming the previous instructions have been followed everything should be fine.

            7) Prepare the Poudriere environment
            Now the really long command. This creates the Poudriere jail environment for building the ports. This took 11 hours on a core i7-11800H with an NVME SSD. Unfortunately it does not update the screen with any progress info.

            cd /build/pfsense
            ./build.sh --setup-poudriere
            

            8) Build ports
            We're now ready to try building the port tree, which failed on the first go due to missing package dependencies for me.

            cd /build/pfsense
            ./build.sh --update-pkg-repo -a amd64.amd64
            

            I was missing these packages, you might be missing others:

            sysutils/vmdktool
            emulators/qemu-user-static
            archivers/gtar
            textproc/xmlstarlet
            

            They were not available in my repo for a simple "pkg install" command, so I compiled and installed them from the ports tree we cloned earlier:

            cd /build/FreeBSD-ports/sysutils/vmdktool/;make package
            pkg install /build/FreeBSD-ports/sysutils/vmdktool/work/pkg/vmdktool-1.4.pkg
            cd /build/FreeBSD-ports/emulators/qemu-user-static/;make package
            pkg install /build/FreeBSD-ports/emulators/qemu-user-static/work/pkg/qemu-user-static-3.1.0_12.pkg
            cd /build/FreeBSD-ports/archivers/gtar; make package
            pkg install /build/FreeBSD-ports/archivers/gtar/work/pkg/gtar-1.34.pkg
            cd /build/FreeBSD-ports/textproc/xmlstarlet; make package
            pkg install /build/FreeBSD-ports/textproc/xmlstarlet/work/pkg/xmlstarlet-1.6.1.pkg
            

            After that, the build ran but a number failed due to missing dist files it wasn't able to fetch. You can see exactly what it is trying to fetch from the repos in the build logs (one for each package) it points you to in the output. Mine was missing the following files:

            sqlite-src-3350500.zip
            zabbix-5.2.6.tar.gz
            mysql-boost-5.7.34.tar.gz
            stunnel-5.59.tar.gz
            

            I found them on the internet and placed them in:

            /usr/ports/distfiles
            

            After this, the build moved on. Then the "rust" package failed to build due to resource exhaustion (remember my original machine only had 8 GB RAM). I upped the VM RAM to 24 GB and then it passed. If you have less memory, keep trying - theoretically it should eventually finish.

            Now the build ran to completion. It does fail on the signature step, but that's expected since we don't have the environment to sign the build (nor should we!).

            9) Change code and build new package versions

            Changing the code is a bit tricky, and one needs to be cautious on how it all works.

            a) Extract the source
            b) Copy it elsewhere and make your change
            c) Copy it back
            d) Make your patch
            e) Copy your patch to the jail
            f) Update your port revision to trigger a compile
            g) Test your code on a pfSense non-production firewall

            a) Extract the Source
            We will assume our package name is "foo"

            cd /build/FreeBSD-ports/foo
            make clean
            make extract
            

            This places the source code in (the x's are a version number)

             /build/FreeBSD-ports/foo/work/foo-x.x/
            

            b) Copy your code elsewhere
            Find the source files you want to change and make 2 copies of each to somewhere outside of this folder (the "work" folder gets deleted each time you run "make clean" so you want to keep your changes safe and sound somewhere else).

            cp source1.c /tmp/source1.c
            cp source1.c /tmp/source1.c.orig
            cp source1.c /tmp/source1.h
            cp source1.c /tmp/source1.h.orig
            

            The .orig files are needed for patching later (do not change them), and the .c/.h files are where you make your changes. This step is only done once for each new file you change.

            c) After editing, copy these files back

            cp /tmp/source1.c /build/FreeBSD-ports/foo/work/foo-x.x/
            cp /tmp/source1.c.orig /build/FreeBSD-ports/foo/work/foo-x.x/
            cp /tmp/source1.h /build/FreeBSD-ports/foo/work/foo-x.x/
            cp /tmp/source1.h.orig /build/FreeBSD-ports/foo/work/foo-x.x/
            

            d) Make the patch (or just do test compiles)

            cd /build/FreeBSD-ports/foo/
            make makepatch  #Creates the diff file
            OR
            make package #test compile
            

            The makepatch command creates patch (diff) files (with names like patch-source1.c) in

            /build/FreeBSD-ports/foo/files/
            

            e) Copy the patches to the jail
            Now in order to have the port build see the patch, copy them here. Your folder may be named differently depending on your release.

            /usr/local/poudriere/ports/pfSense_v2_5_2/net/miniupnpd/files/
            

            f) Edit the Makefile
            Edit the Makefile for your port, for example:

            vi /usr/local/poudriere/ports/pfSense_v2_5_2/foo/Makefile 
            

            Change the port revision, and increase it by 1, for example:

            PORTREVISION=2
            

            g) Build your patched port
            Run your build again, it should now create a new package for your patched code

            cd /build/pfsense
            ./build.sh --update-pkg-repo -a amd64.amd64
            

            If all went well, your package is in (or a similar folder):

            /usr/local/poudriere/data/packages/pfSense_v2_5_2_amd64-pfSense_v2_5_2/All/
            

            Copy it to your test firewall (best not to test in production right?)
            From an ssh shell on the firewall, you can replace the package with:

            pkg add -f foo.x.x.txz
            

            If you need to revert to the original package from the distro:

            pkg delete -f foo
            pkg install foo
            

            A note on building iterations of your package:
            I found it was best to script the synchronization of the files I was changing, and the copying of the patch files, so I can start with a known baseline for every pass. The idea is this:

            make clean
            make extract
            copy my changed and .orig files over the freshly extracted code
            remove old patch files from poudriere jail folder and the working port folder
            make makepatch
            copy new patch files to poudriere jail folder
            Uprev the Port revision
            Build

            And that's it. Thanks again to @bmeeks and @jimp for helping me with this.

            M guiambrosG 2 Replies Last reply Feb 25, 2022, 8:53 PM Reply Quote 5
            • M
              Marc05 @encrypt1d
              last edited by Feb 25, 2022, 8:53 PM

              @encrypt1d

              This is very handy to have, thank you very much!

              1 Reply Last reply Reply Quote 0
              • B bmeeks referenced this topic on Jul 26, 2022, 9:03 PM
              • B bmeeks referenced this topic on Jul 26, 2022, 9:03 PM
              • B bmeeks referenced this topic on Jul 27, 2022, 12:59 PM
              • B bmeeks referenced this topic on Jul 27, 2022, 12:59 PM
              • B bmeeks referenced this topic on Nov 11, 2022, 2:33 PM
              • B bmeeks referenced this topic on Nov 11, 2022, 2:33 PM
              • B bmeeks referenced this topic on Nov 11, 2022, 2:37 PM
              • B bmeeks referenced this topic on Nov 11, 2022, 2:37 PM
              • B bmeeks referenced this topic on Nov 11, 2022, 2:39 PM
              • B bmeeks referenced this topic on Nov 11, 2022, 2:39 PM
              • B bmeeks referenced this topic on Nov 11, 2022, 2:40 PM
              • B bmeeks referenced this topic on Nov 11, 2022, 2:40 PM
              • B bmeeks referenced this topic on Nov 11, 2022, 2:41 PM
              • B bmeeks referenced this topic on Nov 11, 2022, 2:41 PM
              • guiambrosG
                guiambros @encrypt1d
                last edited by Jun 23, 2023, 3:49 AM

                Just to say a heartfelt thank you to @encrypt1d @bmeeks @jimp for your efforts in documenting the process here. What an incredible arduous and cumbersome process, particularly for anyone who's not a core pfSense or package maintainer.

                I too spent countless hours scratching my head trying to figure out why my manually-compiled miniupnpd wasn't working in prod (same IOCTL errors as @encrypt1d ), and eventually realized I'd have to build a pfSense dev environment in order to compile with the right libraries.

                I finally found this thread, which helped me overcome the many twists and turns (including detours due to no ZFS, lack of disk space, recreating poudriere jails 4x, etc).

                The thorniest issue for me (and in hindsight due to my own stupidity) was to ignore the "Jail is newer than host. (Jail: 1203500, Host: 1203000)". I thought it would be harmless -- c'mon, how different would 12.3.5 kernel be from 12.3.0? It turns out, they are indeed very different.

                Once I upgraded the host to 12.4 (I didn't figure out how to upgrade to an intermediate 12.3.5 release), and cleaned / recreated the packages, everything finally started working.

                I just got a fully functional miniupnpd binary running on 2.6.0. Still need to make the changes I want to make to source code, diff, etc, but much easier now that I have a working env.

                note: I also got lucky, because half of my ports packages failed due to dependencies with devel/libffi-3.3, print/texinfo-6.8 and net/zabbix54-agent-5.4.7. But thankfully miniupnpd doesn't depend on any of them, so I decided to ignore and move on. YMMV.

                THANK YOU!

                guiambrosG 1 Reply Last reply Jun 26, 2023, 5:58 AM Reply Quote 0
                • guiambrosG
                  guiambros @guiambros
                  last edited by Jun 26, 2023, 5:58 AM

                  @guiambros said in pfSense compile requirements for 3rd party software:

                  I just got a fully functional miniupnpd binary running on 2.6.0. Still need to make the changes I want to make to source code, diff, etc, but much easier now that I have a working env.

                  Ugh, I spoke too soon. Its true that with FreeBSD 12.3.5 the resulting miniupnpd binary doesn't immediately throw ioctl() errors like before, but further tests indicated that new binary still rejects UPnP requests from clients.

                  @encrypt1d , I think your initial motivation was miniupnpd as well,no? Were you able to compile successfully? If yes, would you mind sharing a tarball of your resulting source files? Or at least config.h, so I can see which options you have enabled.

                  It's incredible how painful this is. I wonder if all package developers go through the same journey to set up their initial environment, or there's a docker/vm image somewhere that simplifies this.

                  E 1 Reply Last reply Jun 26, 2023, 12:21 PM Reply Quote 0
                  • E
                    encrypt1d @guiambros
                    last edited by encrypt1d Jun 26, 2023, 12:29 PM Jun 26, 2023, 12:21 PM

                    @guiambros Really glad to hear this helped.

                    This is actually a 2 part fix, depending on your ISP setup.

                    On release 2.6.0, you need to apply a patch first:
                    "Add UPnP NAT Anchors to fix outbound NAT for multiple consoles. (Redmine #7727, Forum Thread)".
                    You can do that from the "System" Menu under patches - but you might need to add the Patching package first from the Package Manager.

                    The Miniupnpd fix specifically allows for Double NAT scenarios to work (where your ISP gives you a private address like 10.x.x.x). If you aren't double NATed, just applying the patch should do it for you.

                    The folks that maintain the miniupnpd package have included the double NAT fix in 2.3.1. PFSense 2.7.0 is dropping very soon - I am hoping it will contain the right version of minupnpd. Maybe @jimp can confirm?

                    https://github.com/miniupnp/miniupnp/issues/598#issuecomment-1462959757

                    Now we get to do this all over again with 2.7.0 ;) I built custom logging into mine so I can get alerts when things punch holes in my firewall. UPNP is scary stuff. I will pretty much always run a custom build because of that.

                    guiambrosG 1 Reply Last reply Jul 1, 2023, 8:44 AM Reply Quote 1
                    • guiambrosG
                      guiambros @encrypt1d
                      last edited by guiambros Jul 1, 2023, 8:44 AM Jul 1, 2023, 8:44 AM

                      Thanks again!

                      @encrypt1d said in pfSense compile requirements for 3rd party software:

                      Now we get to do this all over again with 2.7.0 ;)

                      Ha, indeed, 2.7.0 just arrived a couple of days ago. The upgrade went smoothly, but I still want to customize my miniupnpd. In my case I don't have an issue with double NAT, but I want to fix an issue with log spam, and add some custom logging for security.

                      I built custom logging into mine so I can get alerts when things punch holes in my firewall. UPNP is scary stuff. I will pretty much always run a custom build because of that.

                      Any idea when RELENG_2._7_0 will be tagged on pfSense repo? I see recent commits in the master branch, but no indication of which commit was used for 2.7.0.

                      I guess we'll have to wait a few more days/weeks for github to catch up with Netgate's internal repo.

                      fabricioguzzyF 1 Reply Last reply Jul 1, 2023, 9:17 PM Reply Quote 1
                      • fabricioguzzyF
                        fabricioguzzy @guiambros
                        last edited by Jul 1, 2023, 9:17 PM

                        @guiambros - I am also looking for the **"FreeBSD_ports RELENG_2_7_0 branch **" - Actually all the 2_7_0 branches are missing. (pfSense, FreeBSD and Ports)

                        guiambrosG 1 Reply Last reply Jul 3, 2023, 3:19 AM Reply Quote 0
                        • guiambrosG
                          guiambros @fabricioguzzy
                          last edited by Jul 3, 2023, 3:19 AM

                          Quick update: I was finally able to successfully compile packages - and they work properly on 2.7.0! That's a first for me; all my previous attempts with 2.6.0 resulted in a binary that had all sorts of ioctl() errors.

                          The RELENG_2_7_0 branch does not yet exist on GitHub, but the master (pfSense) and devel-main (FreeBSD-src) branches seems to be close enough to what was just released, so most packages should be the same at this point. In my case I was only interested in miniupnpd package, so YMMV.

                          My step-by-step:

                          1. VM with FreeBSD 14.0-CURRENT
                          2. Follow the excellent steps above by @encrypt1d. Only pfSense source is needed; stay on the master branch
                          3. On build.conf, use export FREEBSD_BRANCH=devel-main
                          4. Create Poudriere jail, and build the ports
                          5. In my case I had 2 packages failed (missing original sources print/psutils and sysutils/pfSense-repoc) and 10 skipped due to dependencies.
                          6. Find the package directory (e.g. /usr/local/poudriere/ports/pfSense_devel/net/miniupnpd), use make extract to get the source code, modify the source as needed, and run make package to recompile. Package will be put on .../work/stage

                          After weeks of trial-and-error, this was the first time I was able to have a functional miniupnpd running in production.

                          I still want to grab the official RELENG_2_7_0 once released and recompile everything, but for now this is already a step in the right direction.

                          E 1 Reply Last reply Feb 10, 2024, 4:00 PM Reply Quote 2
                          • guiambrosG guiambros referenced this topic on Jul 3, 2023, 3:19 AM
                          • B bmeeks referenced this topic on Oct 7, 2023, 11:52 PM
                          • E
                            encrypt1d @guiambros
                            last edited by Feb 10, 2024, 4:00 PM

                            @guiambros

                            I started down the path of trying to get a 2_7_2 build going recently. It seems that both labels RELENG2_7_1 and RELENG2_7_2 exist, but if I use RELENG2_7_2 the jail build fails with a compile failure on the AES-586 object, so I stuck with 2_7_1.

                            2_7_1 seems to create the jail ok, but then I run into build issues.

                            The variables in the build.conf file that control the repos used to be:

                            export FREEBSD_REPO_BASE=https://github.com/pfsense/FreeBSD-src.git
                            export FREEBSD_BRANCH=RELENG_2_7_1
                            export PKG_REPO_SERVER_DEVEL="pkg+https://beta.pfsense.org/packages"
                            export PKG_REPO_SERVER_RELEASE="pkg+https://pkg.pfsense.org"
                            export PKG_REPO_SERVER_STAGING="pkg+https://pkg.pfsense.org"
                            

                            These domains don't have IP address records anymore (or maybe never?).

                            What did you use for your build.conf?

                            When I ran my first build, it seems I am missing packages:

                            [00:00:11] Error: Nonexistent origin listed: net-mgmt/pfSense-pkg-zabbix-agent4
                            [00:00:11] Error: Nonexistent origin listed: net-mgmt/pfSense-pkg-zabbix-proxy4
                            

                            Trying to install these seems to use some default repos that don't exist:

                            # pkg update
                            Updating pfSense-core repository catalogue...
                            pkg: No SRV record found for the repo 'pfSense-core'
                            pkg: An error occured while fetching package
                            pkg: packagesite URL error for pkg+http://release-staging.nyi.netgate.com/ce/packages/pfSense_%%OSVERSION%%_amd64-core/meta.txz -- pkg+:// implies SRV mirror type
                            repository pfSense-core has no meta file, using default settings
                            pkg: packagesite URL error for pkg+http://release-staging.nyi.netgate.com/ce/packages/pfSense_%%OSVERSION%%_amd64-core/packagesite.pkg -- pkg+:// implies SRV mirror type
                            pkg: packagesite URL error for pkg+http://release-staging.nyi.netgate.com/ce/packages/pfSense_%%OSVERSION%%_amd64-core/packagesite.txz -- pkg+:// implies SRV mirror type
                            Unable to update repository pfSense-core
                            Updating pfSense repository catalogue...
                            pkg: No SRV record found for the repo 'pfSense'
                            pkg: An error occured while fetching package
                            pkg: packagesite URL error for pkg+http://release-staging.nyi.netgate.com/ce/packages/pfSense_%%OSVERSION%%_amd64-pfSense_%%VERSION%%/meta.txz -- pkg+:// implies SRV mirror type
                            repository pfSense has no meta file, using default settings
                            pkg: packagesite URL error for pkg+http://release-staging.nyi.netgate.com/ce/packages/pfSense_%%OSVERSION%%_amd64-pfSense_%%VERSION%%/packagesite.pkg -- pkg+:// implies SRV mirror type
                            pkg: packagesite URL error for pkg+http://release-staging.nyi.netgate.com/ce/packages/pfSense_%%OSVERSION%%_amd64-pfSense_%%VERSION%%/packagesite.txz -- pkg+:// implies SRV mirror type
                            Unable to update repository pfSense
                            Error updating repositories!
                            
                            

                            These repo domain names also don't exist. I think they are defined in builder_defaults.sh.

                            Would you mind sharing your build.conf file, or any changes to builder_defaults.sh?

                            Much appreciated.

                            E 1 Reply Last reply Feb 11, 2024, 5:58 PM Reply Quote 0
                            • E
                              encrypt1d @encrypt1d
                              last edited by Feb 11, 2024, 5:58 PM

                              I thought I was on to something when I found these files in the pfsense repo that don't seem to be included when you do a clone on the RELENG2_7_1 branch:

                              pfSense-repo-devel.abi
                              pfSense-repo-devel.altabi
                              pfSense-repo-devel.conf
                              pfSense-repo-devel.descr
                              pfSense-repo-previous.abi
                              pfSense-repo-previous.altabi
                              pfSense-repo-previous.conf
                              pfSense-repo-previous.descr
                              pfSense-repo.abi
                              pfSense-repo.altabi
                              pfSense-repo.conf
                              pfSense-repo.descr
                              

                              Putting those in ./tools/templates/pkg_repos didn't help though.

                              I watched my DNS queries on my old working 2_6_0 VM build, and they were going to pkg01-atx.netgate.com, so I tried this in my build.conf:

                              export PKG_REPO_SERVER_DEVEL="pkg+https://pkg01-atx.netgate.com/packages"
                              export PKG_REPO_SERVER_RELEASE="pkg+https://pkg01-atx.netgate.com"
                              export PKG_REPO_SERVER_STAGING="pkg+https://pkg01-atx.netgate.com"
                              
                              

                              No joy. On a fresh install, the pkg repos are ok, but after running ./build.sh --setup, the pkg repos are just broken.

                              Hoping @bmeeks or @guiambros know what works in terms of the repo list.

                              B 1 Reply Last reply Feb 11, 2024, 6:16 PM Reply Quote 0
                              • B
                                bmeeks @encrypt1d
                                last edited by bmeeks Feb 11, 2024, 6:39 PM Feb 11, 2024, 6:16 PM

                                @encrypt1d said in pfSense compile requirements for 3rd party software:

                                Hoping @bmeeks or @guiambros know what works in terms of the repo list.

                                I am reasonably sure you are just spinning wheels here on an impossible quest (that being building pfSense from the open-source repo on GitHub) -- at least since the last update to 2.7.2.

                                There is a new proprietary module called pfSense-repoc that handles all the repo-related stuff now on installation. It also handles the updates now in versions since at least 2.7.1. The source code for that package module is hosted on the private Netgate GitLab account, that's why I said it is proprietary because it is not available on the public GitHub. I strongly suspect that without that package in place, the repo stuff in your pfSense build is going to be broken.

                                There is also now another new pfSense package that needs to be built, and that package's source code is also hosted on the private Netgate GitLab and not on the public GitHub repo.

                                I don't try to build a pfSense kernel. All I build is the packages repo, but even for that now I have to go in and manually remove the runtime dependencies from the pfSense-upgrade package so that it does not try and pull in the proprietary pfSense-repoc package (which can't be built because its source code is in GitLab and not on GitHub). I also, starting with 2.7.2, have to comment out the build of the pfSense package in the poudriere.bulk file because that package's code is also on the private GitLab site.

                                E 1 Reply Last reply Feb 11, 2024, 6:49 PM Reply Quote 2
                                • E
                                  encrypt1d @bmeeks
                                  last edited by Feb 11, 2024, 6:49 PM

                                  @bmeeks said in pfSense compile requirements for 3rd party software:

                                  I don't try to build a pfSense kernel. All I build is the packages repo, but even for that now I have to go in and manually remove the runtime dependencies from the pfSense-upgrade package so that it does not try and pull in the proprietary pfSense-repoc package

                                  This is all I am really trying to do as well. I had the instructions I wrote from 2_6_0 to go on, which obviously won't work now based on everything you have pointed out that has changed.

                                  Do I simply clone FreeBSD-ports and try to build again without the jail? What might be the path forward?

                                  B 1 Reply Last reply Feb 11, 2024, 7:03 PM Reply Quote 1
                                  • B
                                    bmeeks @encrypt1d
                                    last edited by bmeeks Feb 11, 2024, 7:10 PM Feb 11, 2024, 7:03 PM

                                    @encrypt1d said in pfSense compile requirements for 3rd party software:

                                    Do I simply clone FreeBSD-ports and try to build again without the jail? What might be the path forward?

                                    You can build the jail. You just can't build a pfSense kernel itself.

                                    Clone both the FreeBSD-ports tree and the pfSense tree from the public GitHub repo.

                                    I edited the pfsense/build.conf file to point to the RELENG_2_7_2 branch.

                                    Then you need to comment out the lines pertaining to GNID in the builder_common.sh file so that the jail build does not try and pull down the proprietary Netgate ID source code files.

                                    Change into the pfsense git clone directory.

                                    Next, fetch the FreeBSD source files with: ./build.sh --update-sources.

                                    You should now be able to create the poudriere jail with ./build.sh --setup-poudriere.

                                    I then edited tools/conf/pfPorts/poudriere.bulk to comment out the pfSense package so that a build of it will not be attempted (which would fail during the fetch phase due to the GitLab host location). Here is the edit to poudriere.bulk:

                                    #security/%%PRODUCT_NAME%%
                                    

                                    Now go into your Poudriere ports tree (/usr/local/poudriere/ports/) and find the sysutils/pfSense-upgrade package. Edit its runtime depencies in the Makefile to comment out the pfSense-repoc package.

                                    Packages should build now unless I've forgotten something off the top of my head.

                                    But note the steps outlined above will NOT build a pfSense image. They only allow you to build the FreeBSD-ports packages repo tree.

                                    E 1 Reply Last reply Feb 12, 2024, 1:11 PM Reply Quote 3
                                    • E
                                      encrypt1d @bmeeks
                                      last edited by Feb 12, 2024, 1:11 PM

                                      @bmeeks

                                      Thanks, that was helpful. I assume I now need the "-c" option for build.sh so that it doesn't try to build the kernel, but yet it still seems to need it see "installing kernel" in the logs below.

                                      # ./build.sh none -a amd64.amd64 -c
                                      >>> Operation ./build.sh has started at Mon Feb 12 03:04:31 EST 2024
                                      >>> Building image type(s):
                                      >>> Cleaning up previous build environment...Please wait!
                                      >>> Cleaning build directories: Done!
                                      >>> Cleaning previously built kernel stage area...Done!
                                      >>> Cleaning previously built images...Done!
                                      >>> Cleaning previous builder logs...Done!
                                      >>> Cleaning of builder environment has finished.
                                      >>> NO_BUILDWORLD and NO_BUILDKERNEL set, skipping update of freebsd sources
                                      >>> Last known commit Luiz Otavio O Souza - cf612ab9fc5711351fef5e0678d687aa3b88355d
                                      >>> LOGFILE set to /build/pfsense/logs/buildworld.amd64.
                                      >>> NO_BUILDWORLD set, skipping build
                                      >>> Building pfSense kernel.
                                      >>> NO_BUILDKERNEL set, skipping build
                                      >>> Staging pfSense kernel...
                                      >>> Installing kernel (pfSense) for amd64 architecture...
                                      ====>> ERROR: SRCCONF is pointing to a nonexistent file /build/pfsense/tmp/FreeBSD-src/release/conf/pfSense_src.conf
                                      
                                      ####################################
                                      Something went wrong, check errors!
                                      ####################################
                                      

                                      A second issue I am having is that no matter what I try, the jail build always fails for me on 2.7.2 when it gets to this:

                                      --- secure/lib/libcrypto__L ---
                                      make[4]: make[4]: don't know how to make aes-586.S. Stop
                                      make[4]: stopped in /usr/local/poudriere/jails/pfSense_v2_7_2_amd64/usr/src/secure/lib/libcrypto
                                      make[3]: stopped in /usr/local/poudriere/jails/pfSense_v2_7_2_amd64/usr/src
                                      

                                      At a loss on how to solve that one. It works fine on 2.7.1 though.

                                      B 1 Reply Last reply Feb 12, 2024, 1:54 PM Reply Quote 1
                                      • B
                                        bmeeks @encrypt1d
                                        last edited by bmeeks Feb 12, 2024, 1:59 PM Feb 12, 2024, 1:54 PM

                                        @encrypt1d said in pfSense compile requirements for 3rd party software:

                                        @bmeeks

                                        Thanks, that was helpful. I assume I now need the "-c" option for build.sh so that it doesn't try to build the kernel, but yet it still seems to need it see "installing kernel" in the logs below.

                                        # ./build.sh none -a amd64.amd64 -c
                                        >>> Operation ./build.sh has started at Mon Feb 12 03:04:31 EST 2024
                                        >>> Building image type(s):
                                        >>> Cleaning up previous build environment...Please wait!
                                        >>> Cleaning build directories: Done!
                                        >>> Cleaning previously built kernel stage area...Done!
                                        >>> Cleaning previously built images...Done!
                                        >>> Cleaning previous builder logs...Done!
                                        >>> Cleaning of builder environment has finished.
                                        >>> NO_BUILDWORLD and NO_BUILDKERNEL set, skipping update of freebsd sources
                                        >>> Last known commit Luiz Otavio O Souza - cf612ab9fc5711351fef5e0678d687aa3b88355d
                                        >>> LOGFILE set to /build/pfsense/logs/buildworld.amd64.
                                        >>> NO_BUILDWORLD set, skipping build
                                        >>> Building pfSense kernel.
                                        >>> NO_BUILDKERNEL set, skipping build
                                        >>> Staging pfSense kernel...
                                        >>> Installing kernel (pfSense) for amd64 architecture...
                                        ====>> ERROR: SRCCONF is pointing to a nonexistent file /build/pfsense/tmp/FreeBSD-src/release/conf/pfSense_src.conf
                                        
                                        ####################################
                                        Something went wrong, check errors!
                                        ####################################
                                        

                                        A second issue I am having is that no matter what I try, the jail build always fails for me on 2.7.2 when it gets to this:

                                        --- secure/lib/libcrypto__L ---
                                        make[4]: make[4]: don't know how to make aes-586.S. Stop
                                        make[4]: stopped in /usr/local/poudriere/jails/pfSense_v2_7_2_amd64/usr/src/secure/lib/libcrypto
                                        make[3]: stopped in /usr/local/poudriere/jails/pfSense_v2_7_2_amd64/usr/src
                                        

                                        At a loss on how to solve that one. It works fine on 2.7.1 though.

                                        No, that's actually a problem with the copy of FreeBSD-src in the public repo. I had forgotten about that. There is a specific setting or optional argument to pass when building the jail. Let me consult my email history to see if I can find it.

                                        Later Update: Here is the fix for that error. Add this to the file /usr/local/etc/poudriere.d/src.conf --

                                        WITHOUT_LIB32=y
                                        
                                        1 Reply Last reply Reply Quote 3
                                        • B
                                          bmeeks
                                          last edited by Feb 12, 2024, 2:03 PM

                                          Also, if you only want to build the poudriere ports tree, then run this command:

                                          ./build.sh --update-pkg-repo -a amd64.amd64
                                          

                                          The changes I mentioned in posts above still are not likely to yield a successful kernel build. And if it builds, I'm not sure it will work correctly connecting to the official Netgate pfSense repositories for package installs and updates.

                                          E 1 Reply Last reply Feb 12, 2024, 9:50 PM Reply Quote 2
                                          • First post
                                            Last post
                                          Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.