Creating pfSense 2.0 Developers ISO on FreeBSD Beta 8.1

jimp

About point 5.), I think you need to use cvs tag "." instead of "RELENG_8_1" for fetching ports, /usr/bin/csup -h cvsup.uk.FreeBSD.org
/usr/share/examples/cvsup/ports-supfile should do the job (replace cvsup.uk with a mirror close to you).

He's using portsnap to get the ports, that supfile was just for the system sources (though since it was done earlier, it doesn't need to be done again)

jimp

I also noticed that I used this:

./set_version.sh HEAD

jimp

and for some reason, that still uses RELENG_8-supfile, instead of RELENG_8_1-supfile.

Whichever supfile shows in your pfsense-build.conf, make sure it has a line like this in it, if it uses RELENG_8_1

date=2010.06.25.00.00.00

I committed an update to that file, too, so it should work. Or it does for me again…

builder# ./set_version.sh RELENG_8_1 cvsup.livebsd.org

>>> Setting builder environment to use RELENG_8_1 ...
# Comment out the following line if you would like to automatically select an update server.

>>> Custom pfsense-build.conf contains:
---------------------------------------------------------------------------------------
# set_version.sh generated defaults
export PFSENSE_VERSION=2.0-BETA3
export FREEBSD_VERSION=8
export FREEBSD_BRANCH=RELENG_8_0
export PFSENSETAG=HEAD
export PFSPATCHFILE=/usr/home/pfsense/tools/builder_scripts/../builder_scripts/patches.RELENG_8_1
export PFSPATCHDIR=/usr/home/pfsense/tools/builder_scripts/../patches/RELENG_8_1
export SUPFILE=/usr/home/pfsense/tools/builder_scripts/../builder_scripts/RELENG_8-supfile
export CUSTOM_COPY_LIST=/usr/home/pfsense/tools/builder_scripts/../builder_scripts/copy.list.RELENG_8_0
export OVERRIDE_FREEBSD_CVSUP_HOST=cvsup.livebsd.org
export BASE_DIR=/usr/home/pfsense/tools/builder_scripts/../..
export BUILDER_TOOLS=/usr/home/pfsense/tools/builder_scripts/..
export BUILDER_SCRIPTS=/usr/home/pfsense/tools/builder_scripts
export PFSPORTSFILE=buildports.RELENG_2_0
---------------------------------------------------------------------------------------

NOTE: pfsense-build.conf values updated.  These values override pfsense_local.sh !!

NOTE2: pfPorts will be rebuilt!

builder# ./update_git_repos.sh 
>>> Updating GIT REPO freesbie2...Done!
>>> Updating GIT REPO tools...Done!
builder# ./apply_kernel_patches.sh    
>>> Setting CVSUp host to cvsup.livebsd.org
>>> Removing needed files listed in patches.RELENG_8_1 HEAD
>>> Obtaining FreeBSD sources RELENG_8-supfile...Done!
>>> Removing old patch rejects...
>>> Applying patches, please wait...Done!
>>> Finding patch rejects...
builder#

trijames

HI guys,

good news is that it now is able to get past the apply-kernel-patches.sh - thanks to the changes and info sent on how to properly setup the environment by Jimp. thanks for that.
So - that is great. I'm now trying to build the iso - and hopefully will have some good news for you in about 1 hour…

cross my fingers (or press my thumbs in german)...

regards
TJ

trijames

HI Guys,

the apply-kernel-patches.sh went through clean as I mentioned earlier…
But when I ran the build-iso.sh... I got a few errors seen below:
I'm not sure if openssl for php is really needed. But, it then looks like it is failing on dfuife_curses...

/usr/ports/security/php5-openssl/work/php-5.2.13/ext/openssl/openssl.c:153: error: 'zif_openssl_crl_new' undeclared here (not in a function)
/usr/ports/security/php5-openssl/work/php-5.2.13/ext/openssl/openssl.c:154: error: 'zif_openssl_crl_revoke_cert_by_serial' undeclared here (not in a function)
/usr/ports/security/php5-openssl/work/php-5.2.13/ext/openssl/openssl.c:155: error: 'zif_openssl_crl_revoke_cert' undeclared here (not in a function)
/usr/ports/security/php5-openssl/work/php-5.2.13/ext/openssl/openssl.c:156: error: 'zif_openssl_crl_export' undeclared here (not in a function)
/usr/ports/security/php5-openssl/work/php-5.2.13/ext/openssl/openssl.c:157: error: 'zif_openssl_crl_export_file' undeclared here (not in a function)
*** Error code 1
1 error
*** Error code 1

ERROR Building dnsmasq
ERROR Building php5-curl
ERROR Building php5-openssl
ERROR Building php5-readline
ERROR Building syslog-ng
Building athstats...main.c: In function 'getfmt':
main.c:73: warning: implicit declaration of function 'strcasecmp'

and then at the very end...

Checking BSDInstaller health... dfuife_curses missing [ERROR!]

trijames

Hi Jimp,

I'm going to and give it a try again and first run clean_build.sh before hand.
Here's the steps that I'm going to try.
btw… I've been trying the cvsup.livebsd.org mirror - but it doesn't seem to work. I assume the livebsd.org mirror is the same as the freebsd - right?

cd /home/pfsense/tools/builder_scripts

./clean_build.sh

./set_version.sh RELENG_8_1 cvsup.livebsd.org

./update_git_repos.sh

./apply_kernel_patches.sh

# ./build-iso.sh

regards
TJ

jimp

A fix for curl and readline just went into the repo from another dev.

trijames

Hi Jimp,

I went through the steps that I outlined in my earlier post and
# ./clean_build.sh (ok)
# ./set_version.sh RELENG_8_1 cvsup4.us.freebsd.org (livebsd.org doesn't seem to connect)..
# ./update_git_repos.sh (ok)
# ./apply_kernel_patches.sh (ok)
# ./build-iso.sh (errors below…)

so far... still a few errors during build-iso.sh.
It still doesn't generate an iso...
Any tips on what I can do to try and help to track these down?

.
.
...
the O_NONBLOCK symbol with a syntax error. This is apparently a
sh error. Rerunning Configure with ksh apparently fixes the
error, then please read the section "64 bit data in perl 5.6.0 and 5.6.1"
/usr/local/lib/perl5/5.10.1/perl/man/man3/CPANPLUS::Error.3

Build dependency freetype2 of rrdtool
.
.
...
Building olsrd
src/NetworkInterfaces.c:54:47: error: linux/if_ether.h: No such file or directory
src/NetworkInterfaces.c:55:92: error: linux/if_packet.h: No such file or directory
src/NetworkInterfaces.c:56:46: error: linux/if_tun.h: No such file or directory
src/Bmf.c:52:47: error: linux/if_ether.h: No such file or directory
src/Bmf.c:53:75: error: linux/if_packet.h: No such file or directory
Building pftop
.
.
...
Building php5-openssl
/usr/ports/security/php5-openssl/work/php-5.2.13/ext/openssl/openssl.c:153: error: 'zif_openssl_crl_new' undeclared here (not in a function)
/usr/ports/security/php5-openssl/work/php-5.2.13/ext/openssl/openssl.c:154: error: 'zif_openssl_crl_revoke_cert_by_serial' undeclared here (not in a function)
/usr/ports/security/php5-openssl/work/php-5.2.13/ext/openssl/openssl.c:155: error: 'zif_openssl_crl_revoke_cert' undeclared here (not in a function)
/usr/ports/security/php5-openssl/work/php-5.2.13/ext/openssl/openssl.c:156: error: 'zif_openssl_crl_export' undeclared here (not in a function)
/usr/ports/security/php5-openssl/work/php-5.2.13/ext/openssl/openssl.c:157: error: 'zif_openssl_crl_export_file' undeclared here (not in a function)
*** Error code 1
1 error
*** Error code 1
.
.
...
Building php5-readline
configure: error: Please reinstall libedit - I cannot find readline.h
.
.
...
ERROR Building dnsmasq
ERROR Building php5-openssl
ERROR Building php5-readline
ERROR Building syslog-ng
.
.
...
cp: directory /usr/home/pfsense/tools/builder_scripts/../../installer/installer/scripts/build does not exist
.: Can't open ./pfsense_local.sh: No such file or directory
.
.
...
Checking BSDInstaller health... dfuife_curses missing [ERROR!]

####################################
Something went wrong, check errors!
####################################

Press enter to continue.

Terminated…

jimp

Looks like someone is working on some PHP fixes in the tools repo, I'd hold off for a day or so and try again.

Looks like FreeBSD's switch to using php52 ports instead of php5 (which is now php 5.3) might have caused some problems.

trijames

HI Jimp,

ok thanks for the heads up. What about the other errors?
Are they all critical? Or can I manage to build an ISO with the other errors after php is fixed?

e.g.:

ERROR Building dnsmasq
ERROR Building syslog-ng
and
Building olsrd
src/NetworkInterfaces.c:54:47: error: linux/if_ether.h: No such file or directory
src/NetworkInterfaces.c:55:92: error: linux/if_packet.h: No such file or directory
src/NetworkInterfaces.c:56:46: error: linux/if_tun.h: No such file or directory
src/Bmf.c:52:47: error: linux/if_ether.h: No such file or directory
src/Bmf.c:53:75: error: linux/if_packet.h: No such file or directory
Building pftop
and
the O_NONBLOCK symbol with a syntax error. This is apparently a
sh error. Rerunning Configure with ksh apparently fixes the
error, then please read the section "64 bit data in perl 5.6.0 and 5.6.1"
/usr/local/lib/perl5/5.10.1/perl/man/man3/CPANPLUS::Error.3
Build dependency freetype2 of rrdtool

jimp

not sure about dnsmasq and syslog-ng, but the olsr and other errors are not really errors, they are normal output that just happens to contain the string "error", or harmless.

trijames

ok thanks.
I will eventually need the dnsmasq - but hoping that i can still create the iso irregardless of whether those 2 compile or not.

which forum (or url) can i see the status of the php5 changes being made to freebsd?
that way I can try to keep an eye out for it (or communicate with the developer).

thanks
TJ

jimp

Keep an eye on the tools repo commit log:

https://rcs.pfsense.org/projects/pfsense-tools

dnsmasq is being worked on right now.

The person making the changes for php52 might not finish up until Monday, they went home early to watch the world cup match. :-)

jimp

Everything may be OK now. Be sure to update your ports tree, then the git repos, then build_pfPorts.sh.

trijames

Hi Jimp,
looks like the errors are still there:
Here's what I did:
# cd /home/pfsense/tools/builder_scripts

./set_version.sh RELENG_8_1 cvsup4.us.freebsd.org (or cvsup.livebsd.org)

./clean_build.sh

./update_git_repos.sh

./apply_kernel_patches.sh

./build_pfPorts.sh

cd /home/pfsense/tools/builder_scripts

./build_iso.sh

Not sure if the build_pfPorts.sh was in the right spot…

Here are the errors I get in the build-iso.sh script.
I'm getting a lot more errors at the earlier stages of the build-iso.sh
But, the build doesn't terminate until the very end when (it looks like) fails on dfuife_curses...

.
..
...

Building port sshlockout_pf
Ports with failures: 28
ERROR Building ipsec-tools-devel
File not found ipsec-tools-devel - /usr/local/sbin/racoon
ERROR Building ipfw-classifyd
File not found ipfw-classifyd - /usr/local/sbin/ipfw-classifyd
ERROR Building check_reload_status
File not found check_reload_status - /usr/local/sbin/check_reload_status
ERROR Building dhcplease
File not found dhcplease - /usr/local/sbin/dhcpleases
ERROR Building filterdns
File not found filterdns - /usr/local/sbin/filterdns
ERROR Building dnsmasq-no-isc-dhcp-parser
ERROR Building isc-dhcp30-server
File not found isc-dhcp30-server - /usr/local/sbin/dhcpd
ERROR Building isc-dhcp30-relay
File not found isc-dhcp30-relay - /usr/local/sbin/dhcrelay
ERROR Building minicron
File not found minicron - /usr/local/bin/minicron
ERROR Building php52-pfSense-module
File not found php52-pfSense-module - /usr/local/lib/php/20060613/pfSense.so
ERROR Building 3gstat_um175
File not found 3gstat_um175 - /usr/local/bin/3gstat
ERROR Building wrapalixresetbutton
File not found wrapalixresetbutton - /usr/local/sbin/alixresetbtn
ERROR Building voucher
ERROR Building dnswatch
ERROR Building gzsig
ERROR Building ssh_tunnel_shell
ERROR Building sshlockout_pf
Building athstats...main.c: In function 'getfmt':
main.c:73: warning: implicit declaration of function 'strcasecmp'
.
..
...
Installing packages:
grub-0.97_3
Phase populate_extra...
Making devd... Done.
Using FreeBSD 7 BSDInstaller dfuibelua structure.
Merging extra items...
Running plugins: customroot customscripts pkginstall buildmodules rootmfs varmfs etcmfs Done!
==> Building syslogd...
==> Installing syslogd to /usr/local/pfsense-fs/usr/sbin/...
==> Building clog...
==> Installing clog to /usr/local/pfsense-fs/usr/sbin/...
Using /usr/home/pfsense/tools/builder_scripts/../builder_scripts/copy.list.RELENG_8_0...
Populating newer binaries found on host jail/os (usr/local)...
Installing collected library information (usr/local), please wait...
chroot'ing and running /etc/rc.php_ini_setup
Using FreeBSD 7 BSDInstaller dfuibelua structure.
Copying config.xml from conf.default/ to cf/conf/
Testing PHP installation in /usr/local/pfsense-fs: FCGI-PASSED PASSED [OK]
Checking BSDInstaller health… dfuife_curses missing [ERROR!]

jimp

Another dev and I have been making quite a lot of changes to the builder code over the weekend, polishing things up.

I don't see any reason why those other ports would have failed. They all work OK for me now, both on x86 and amd64.

# portsnap fetch && portsnap extract (Or update ports by csup, however you normally do it)
# cd /home/pfsense/tools/builder_scripts
# ./update_git_repos.sh
# ./clean_build.sh (Not really necessary most of the time, but shouldn't hurt, just adds time)
# ./set_version.sh RELENG_8_1 cvsup4.us.freebsd.org (or cvsup.livebsd.org) (Not needed except for the first run, really)
# ./apply_kernel_patches.sh
# ./build_pfPorts.sh (Actually build_iso.sh will run this, no need to do it on its own unless you are testing)
# ./build_iso.sh

trijames

Thanks Jimp,
I'll give it another try… Hopefully I can get it to work!

kpa

I'm not getting errors at the build ports phase but the build stops at the dfuife_part as well:

Checking BSDInstaller health… dfuife_curses missing [ERROR!]

Question, is it necessary to build and install any of the standard ports in /usr/ports before running build_iso.sh?

jimp

@kpa:

I'm not getting errors at the build ports phase but the build stops at the dfuife_part as well:

Checking BSDInstaller health… dfuife_curses missing [ERROR!]

Question, is it necessary to build and install any of the standard ports in /usr/ports before running build_iso.sh?

Ideally, no, but in the past it has been necessary to build at least PHP on the builder box (and more recently the pfSense PHP module).

Have you updated the git repos and done a full build today? I thought someone committed a fix for curses over the weekend.

trijames

HI Jimp,

well the good news is that it went through your listed steps as you outlined earlier cleanly this time!

(except this…
src/NetworkInterfaces.c:54:47: error: linux/if_ether.h: No such file or directory
src/NetworkInterfaces.c:55:92: error: linux/if_packet.h: No such file or directory
src/NetworkInterfaces.c:56:46: error: linux/if_tun.h: No such file or directory
)

So, it is the cleanest build so far...

But, I still got the dfuife_curses error at the very end:

Checking BSDInstaller health... dfuife_curses missing [ERROR!]

Everything else is clean!
How are you getting dfuife_curses to build?
Are you building manually?
I also set environment variable force-pkg-register
#> setenv FORCE_PKG_REGISTER

Not sure if that helped with the cleaner build…
But looks like the ISO still isn't going to be generated until I solve the dfuife_curses issue.

regards
TJ

Creating pfSense 2.0 Developers ISO on FreeBSD Beta 8.1

./set_version.sh HEAD

cd /home/pfsense/tools/builder_scripts

./clean_build.sh

./set_version.sh RELENG_8_1 cvsup.livebsd.org

./update_git_repos.sh

./apply_kernel_patches.sh

so far... still a few errors during build-iso.sh. It still doesn't generate an iso... Any tips on what I can do to try and help to track these down?

./set_version.sh RELENG_8_1 cvsup4.us.freebsd.org (or cvsup.livebsd.org)

./clean_build.sh

./update_git_repos.sh

./apply_kernel_patches.sh

./build_pfPorts.sh

cd /home/pfsense/tools/builder_scripts

./build_iso.sh

so far... still a few errors during build-iso.sh.
It still doesn't generate an iso...
Any tips on what I can do to try and help to track these down?