Building pfSense 2.2 / FreeBSD 10.1 - WORKING STEPS

yakar

Hi PiBa,

How can add mysql56-server to buildports.RELENG_2_2 file or how can i build iso with mysql56-server?

Thank you for your sharing..

PiBa

We seem to be wondering off of the actual topic here (or how i intended it to be anyway.). Which would be how to build pfSense the same as how it is build by the official buildservers of the pfSense team.

Any modifications you want to do to it like adding other packages / binaries, and how to change those are beyond the scope of this topic. So perhaps you could start a new topic with the question you have and a appropriate subject.

Anyway I don't know about adding mysql and whats needed for that, ive done verry little development on the pfSense binaries/iso, a little more with the php side of things, but that doesn't help for these questions.. I can only advice you to look at existing buildscripts and try to understand what go's where from those files.. Search for an existing included items/binaries, then try to add mysql in a similar manner and run the buildscripts again, and again, try a little different, and try again..(you get the picture ;) ).

yakar

Thank you very much for your answer.. :)

sheriftk

Hello…

I was trying to build 2.2.1 on FreeBSD 10.1 x64 following the instructions above and it stops while trying to build the ISO.

Builder is running the command: env __MAKE_CONF= BOOT_COMCONSOLE_SPEED=115200 BOOT_BOOT0_COMCONSOLE_SPEED=0 WITHOUT_CROSS_COMPILER=yes script -aq /home/pfsense/tools/logs/buildworld.amd64 make -j4 __MAKE_CONF= BOOT_COMCONSOLE_SPEED=115200 BOOT_BOOT0_COMCONSOLE_SPEED=0 WITHOUT_CROSS_COMPILER=yes SRCCONF=/home/pfsense/tools/builder_scripts/conf/src/src.conf.10 TARGET=amd64 TARGET_ARCH=amd64 LOCAL_ITOOLS="uuencode uudecode ex" buildworld
make: make: don't know how to make uudecode. Stop

I have tried with various options including starting the build process in a clean state but no joy. The ports build completes successfully. Thanks in Advance.

bhawk6901

builsding on freebsd10.1-release
getting error on build of mpd5
error is as follows
******pppoe.c:334:26: warning: 'memset' call operates on objects of type 'struct ngpppoe_init_data' while the size is based on a different type 'struct ngpppoe_init_data *const' [-Wsizeof-pointer-memaccess]
memset(idata, 0, sizeof(idata));
~~~~~ ^~~~~
pppoe.c:334:26: note: did you mean to dereference the argument to 'sizeof' (and multiply it by the number of elements)?
memset(idata, 0, sizeof(idata));
^~~~~
pppoe.c:718:11: warning: cast from 'char ' to 'const struct typelist ' increases required alignment from 1 to 4 [-Wcast-align]
tlist = (const struct typelist) resp->data;
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
pppoe.c:740:6: error: use of undeclared identifier 'NGM_ETHER_ATTACH'; did you mean 'NGM_ETHER_DETACH'?
NGM_ETHER_ATTACH, &name, sizeof(name)) < 0) {
^~~~~~~~~~~~~~~~
NGM_ETHER_DETACH
/usr/include/netgraph/ng_ether.h:70:2: note: 'NGM_ETHER_DETACH' declared here
NGM_ETHER_DETACH, / our way to be shut down */
^
pppoe.c:770:10: warning: cast from 'char *' to 'const struct hooklist *' increases required alignment from 1 to 4 [-Wcast-align]
hlist = (const struct hooklist *)resp->data;
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
pppoe.c:899:8: warning: cast from 'const struct pppoe_tag *' to 'const uint32_t *' (aka 'const unsigned int *') increases required alignment from 1 to 4 [-Wcast-align]
(const uint32_t)(pt + 1) == idx)
^~~~~~~~~~~~~~~~~~~~~~~~~
4 warnings and 1 error generated.
*** [pppoe.o] Error code 1

make[2]: stopped in /usr/ports/net/mpd5/work/mpd-5.7/src
1 error

make[2]: stopped in /usr/ports/net/mpd5/work/mpd-5.7/src
*** [_sub.all] Error code 2

make[1]: stopped in /usr/ports/net/mpd5/work/mpd-5.7
1 error

make[1]: stopped in /usr/ports/net/mpd5/work/mpd-5.7
===> Compilation failed unexpectedly.
Try to set MAKE_JOBS_UNSAFE=yes and rebuild before reporting the failure to
the maintainer.
*** Error code 1******
Please help

bhawk6901

edit: built fine when i tried to build pfsense 2.2-development instead of 2.2.1 :/

dmitryluhtionov

Hi.
Can I get patched sources of mpd5 ?

I am planned to out mpd-5.8 in nearest time.

madscientist159

The –build-pfPorts option to ./build.sh no longer exists. What replaced it?

Thanks!

rbgarga

@madscientist159:

The –build-pfPorts option to ./build.sh no longer exists. What replaced it?

Thanks!

Are you using branch RELENG_2_2? This is the branch to build 2.2 images, and it still contains –build-pfPorts

madscientist159

@Renato:

Are you using branch RELENG_2_2? This is the branch to build 2.2 images, and it still contains –build-pfPorts

That did the trick. Thanks for the pointer in the right direction! :D

irah

Hi PiBa,

I am trying to build the latest pfsense using your steps and I believe the same steps might apply now too.
I don't find any of the aforementioned scripts (rebuild_bsdinstaller.sh and get_bsdinstaller.sh) in the repository, for rebuilding the bsdinstaller (step# 6).
I mean for building pfsense 2.2, neither master nor RELENG_2_2 branch has those scripts. Only RELENG_2_1 has those scripts, but then it's on top of freebsd 8.3.
Am I wrong in the steps or the repository for building it? Can you please help me out?

P.S. These were the most recent steps that I could find for building the latest pfsense.

Thanks.

irah

Thanks PiBa, your suggestion helped! The steps mentioned earlier in this thread are applicable for the latest build too, only change being in step# 6, where bsdinstaller has been integrated into step# 7 as a static port.

How to build: – pfSense 2.2.5 FreeBSD 10.1-PRERELEASE --

0 - Gain access to the pfsense-tools

Use PuTTYgen to generate a "SSH-2 RSA" key and Export the OpenSSH key to a file locally for usage later.
also save the 'ssh-rsa ..... key-YYYYMMDD' public key to fill in the online registration form.

Information about how the signup works also here: https://forum.pfsense.org/index.php?topic=76132.msg415051#msg415051
-Sign the ICLA or CCLA agreement online https://www.pfsense.org/about-pfsense/#cla
-then also sign the LA, and configure your public ssh key, that will gaining you access to the pfsense-tools repository after a few minutes.

my VirtualBox build machine configuration

disk : 20GB
vcpu : 4 << more could be faster..
mem : 3 GB is enough to use 'memory disks' while compiling which speeds up the build process, if you have plenty more probably wont hurt ;)

download and install into the virtual machine : http://ftp.freebsd.org/pub/FreeBSD/snapshots/amd64/amd64/ISO-IMAGES/10.1/FreeBSD-10.1-PRERELEASE-amd64-2014????-r27????-disc1.iso
Then also update it to the recent version:
freebsd-update fetch
freebsd-update install
-reboot after that

After this "uname -r" shows me: 10.1-RELEASE-p16

-install: lib & src
-enable: ssh
-configure user: admin

allow root to login over ssh / for easy access with winscp and putty (NEVER DO THIS on a production machine, only for testing/easy access)

echo PermitRootLogin yes >> /etc/ssh/sshd_config
service sshd restart

Login with WinSCP to be able to easely transfer files from/to the FreeBSD build machine.

copy your earlier saved ssh private key to the system using WinSCP to: /home/admin/private_putty_key_ssh.ppk

1. Begin pfSense bootstrap by running these shell commands:

echo "WITHOUT_X11=yo" > /etc/make.conf
echo "OPTIONS_UNSET=X11" >> /etc/make.conf
echo "BATCH=yo" >> /etc/make.conf
mkdir -p /home/pfsense/pfSenseGITREPO /usr/pfSensesrc

portsnap fetch extract
cd /usr/ports/devel/git && make depends install
rehash

2. Load SSH key and checkout sources for pfSense-tools from RELENG_2_2 branch

eval ssh-agent -c
chmod 600 /home/admin/private_putty_key_ssh.ppk
ssh-add /home/admin/private_putty_key_ssh.ppk

cd /home/pfsense && git clone -b RELENG_2_2 git@git.pfsense.org:pfsense-tools tools
-Are you sure you want to continue connecting (yes/no)? yes << required to type all 3 letters of y e s

3. Set correct publicly available repositories

'pfmechanics' is used by ESF internally for fast local mirrors of several repositories to speed up their build process it is not available from outside.

p.s. dont try to use git@ for github, it wil fail later because it will try to use the ssh key loaded above..

sed -i "" "s,git@git.pfmechanics.com:pfsense/pfsense-tools.git,git@git.pfsense.org:pfsense-tools," /usr/home/pfsense/tools/builder_scripts/builder_defaults.sh
sed -i "" "s,git@git.pfmechanics.com:outsidemirrors/freebsd.git,https://github.com/freebsd/freebsd.git," /usr/home/pfsense/tools/builder_scripts/builder_defaults.sh
sed -i "" "s,git@git.pfmechanics.com:pfsense/pfsense.git,https://github.com/pfsense/pfsense.git," /usr/home/pfsense/tools/builder_scripts/builder_defaults.sh
sed -i "" "s,git@git.pfmechanics.com:pfsense/bsdinstaller.git,https://github.com/pfsense/bsdinstaller.git," /usr/home/pfsense/tools/builder_scripts/builder_defaults.sh

to check if the above sed commands went ok, the command below should show nothing:

cat /usr/home/pfsense/tools/builder_scripts/builder_defaults.sh | grep git.pfmechanics.com

4. Checkout Freesbie2

cd /home/pfsense && git clone git://github.com/pfsense/freesbie2.git freesbie2

5. set version to build

cd /home/pfsense/tools/builder_scripts
./set_version.sh RELENG_2_2 https://github.com/freebsd/freebsd.git

6. Rebuild the bsdinstaller.

This step can be skipped, creation of the bsdinstaller has been integrated into the script called at step7. It's now converted into a static port and is available under pfPorts.

7. Build ports

Now it will also be checking out FreeBSD sources, this will take quite some time with no visible progress, in the background the 'git' process will take a little cpu usage..

tmp file in /usr/pfSensesrc/src/.git/objects/pack will slowly grow over 800MB..

cd /home/pfsense/tools/builder_scripts
./build.sh --build-pfPorts --no-cleanrepos

to check progress see what ports will be build in: /usr/home/pfsense/tools/builder_scripts/conf/pfPorts/buildports.RELENG_2_2

It should show all ports build without failures..

>>> Ports with failures: 0
>>> Building tools/crytpo...Done!
==> End of pfPorts...
>>> Operation ./build.sh has ended at Wed Nov 5 22:16:57 CET 2014

7. Build pfSense iso

cd /home/pfsense/tools/builder_scripts
./build.sh iso --no-cleanrepos

ISO created: Wed Nov 5 22:53:02 CET 2014
NOTE: waiting for jobs: to finish...
/tmp/pfSense/ now contains:
total 239208
drwxr-xr-x 2 root wheel 512B Nov 5 22:53 .
drwxrwxrwt 10 root wheel 512B Nov 5 22:52 ..
-rw-r--r-- 1 root wheel 234M Nov 5 22:53 pfSense-LiveCD-2.2-BETA-amd64-20141105-2218.iso
-rw------- 1 root wheel 0B Nov 5 22:53 pfSense-LiveCD-2.2-BETA-amd64-20141105-2218.iso.gz
Operation ./build.sh has ended at Wed Nov 5 22:53:02 CET 2014

8. YOUR DONE (hopefully)

The resulting iso should show
ls -l /tmp/pfSense/

-rw-r--r-- 1 root wheel 90629317 Nov 5 22:53 pfSense-LiveCD-2.2-BETA-amd64-20141105-2218.iso.gz

p.s.

Im using --no-cleanrepos flag above to speedup second and further build attempts(the same day?) by skipping the slow checkout of FreeBSD sources.

LOGS

Kernel buildworld log:
/usr/home/pfsense/tools/logs/buildworld.amd64
ports building (failure) logs (empty is good, during building there can be temporary logfiles here):
/tmp/pfPort/buildlogs/

To apply a custom patch written against FreeBSD code this is one way to apply it

add patch file divert-reply10.1.patch to : /usr/home/pfsense/tools/patches/releng/10.1
add the patch to the list of patches, depending on the format of the patch git/diff you might need to specify it a little different.:
echo "-p1~~divert-reply10.1.patch~" >> /usr/home/pfsense/tools/builder_scripts/conf/patchlist/releng/10.1/patches

sreekupmgmail.com

Hii,

I am a new comer, Please assist me how to build own pfSense

PiBa

Hi sreekupm,

Step 1: READ the posts above yours !
Step 2: read some more forum posts, try to do the steps, then read some more.
Step 3: fail while trying those steps, then try to understand the error's show, gather logfiles, look in those logs for clues what might have gone wrong.
Step 4: search the forum and use google to find possible solutions to your problems..
Step 5: try those possible solutions.
Step 6: repeat above steps 3 times (or more..)

Step 7: then, and only then, clearly explain what you have done, tried and failed at, what errors you have seen, what step you could not follow..

Now please follow these instructions and you might get a more useful reply next time..

Some computer knowledge and is expected from a developer so we are not going to tell every mouse click you need to do..
What would you say when i report you the problem that the program/website you created is not working properly? Wouldn't you like to know what button&page i was talking about and what behavior i expected but didn't get?

I'm sorry if i sound a bit harsh, but please do understand that your question is useless like it is currently written..

Regards,
PiBa-NL

S. Kirschner

Hi,

I couldnt clone the repo's.

Following steps I performed
1. accepted Individual CLA
2. accepted the License Agreement (also insert my pubkey)
3. imported the openssh private key in my freebsd machine (eval ssh-agent -c , ssh-add /path/to/private/openssh_key
4. waited 1 hour
5. tried to clone (git clone -b RELENG_2_2 git@git.pfsense.org:pfsense-tools tools)
And here I´m asked for an Password (Not the password from my private key)

To verifiy if I made an mistake with my pub key , i uploaded the pub key to github.com and tested the verification (ssh -T git@github.com), these worked as it should and returned.
"Hi skirschner! You've successfully authenticated, but GitHub does not provide shell access."

My Pfsense profil contain my github username and the pubkey , also the subscriptions would displayed as active.

Have someone an idea ?

[root@freebsd-2836] /home/pfsense# git clone -b RELENG_2_2 git@git.pfsense.org:pfsense-tools tools
Cloning into 'tools'...
Password:

PiBa

Im not sure what is currently the best repository to clone from.. But you could try cloning from github instead. Now that you have signed the ICLA and LA you have also been added to the 'pfsense-tools team'..
what happens if you try? The following:

git clone -b RELENG_2_2 git@github.com:pfsense/pfsense-tools.git tools

S. Kirschner

Thank you , it worked for me.

Now i have to find out how to build a specific appilcation in a pbi.

doktornotor

@S.:

Now i have to find out how to build a specific appilcation in a pbi.

Not really sure why'd anyone do that… Use pkg instead. https://doc.pfsense.org/index.php/Installing_FreeBSD_Packages#pfSense_2.2

S. Kirschner

@doktornotor:

@S.:

Now i have to find out how to build a specific appilcation in a pbi.

Not really sure why'd anyone do that… Use pkg instead. https://doc.pfsense.org/index.php/Installing_FreeBSD_Packages#pfSense_2.2

Yes allready tried it , but I would like to use some options that isnt available in "stock".

Thats the reason why i try to build my own pbi for the pfsense machine.

PiBa

Still using pkg instead of pbi makes sense.

Building a single PBI takes hours on my system.. While using a 'pkg create YourPackage' on a FreeBSD system where the package was compiled and then 'pkg add YourPackage-1.2.3.txz' on pfSense the whole process can be done in a matter of a few minutes.