How to build pfSense on FreeBSD V10

PiBa

Maybe this will help someone :D
But i could come to building an .iso
These are the steps i did:


DISCLAIMER these steps seemed to work for me for testing purposes.
but i dont know if they are "correct" and wont contain serius issues.
use at your own risk !

Also i noticed after using the build image and going through the webgui installation wizard, at the last step a system panic occurs...
After rebooting and resetting the webgui (option 11), most stuff seams to work..
Im not sure if this is an issue caused by the early version and development progress on pfSense 2.2 or if it is caused by my steps..

install VirtualBox and configure a virtual machine like this:
 - disk 20GB
 - mem  1024 MB
 - cpu  2x
 - network: 1 bridged

0\. Install FreeBSD
install:  FreeBSD-10.0-RELEASE-amd64-disc1.iso
	default key map
	options: lib32 ports src  NOT: doc,games
	disk:  automatic  (19GB disk / 1GB swap)

	-root pass: system
	-configure network interface 'em0' IPv4: dhcp
	-country: yours
	-services: sshd,dmpdev

	add users now? YES
		username: admin
		fullname: admin
		uid: <enter>logingroup: <enter>invite other groups: wheel
		class [default]: <enter>shell [sh]: <enter>home dir [/home/admin]: <enter>permissions default: <enter>password-based auth [yes]: <enter>empty/random: no
		pass: secret
	lockout[no]: <enter>another user: no

	## Final Configuration ##
		Exit
		Manual Configuration?: No
		reboot
## the system is now installed

	## to enable SSH login on console ##
		#become ROOT
		su -l
		#generate keys not needed: /usr/bin/ssh-keygen -A
		echo sshd_enable="YES" > /etc/rc.conf
		service sshd start

	## allow root to login over ssh (NEVER DO THIS on a production machine, only for testing/easy access)
		echo PermitRootLogin yes > /etc/ssh/sshd_config
		/etc/rc.d/sshd reload

	## SYSTEM NOW ENABLED for SSH root login ##
		login with PuTTY / SSH as root for running commands / WinSCP for editing and transfering files (use the root account to have access to everything)

# 0-a Update your FreeBSD Install
	mkdir -p /usr/src/crypto/openssl/ssl
	freebsd-update fetch
	freebsd-update install	

# 0-b Make sure your OS clock is accurate
	ntpdate 0.pool.ntp.org

#### checking out ports / sources. ####
### 1\. Begin pfSense bootstrap by running these shell commands:
echo "WITHOUT_X11=yo" > /etc/make.conf
echo "OPTIONS_UNSET=X11" >> /etc/make.conf
echo "BATCH=yo" >> /etc/make.conf
mkdir -p /home/pfsense/pfSenseGITREPO /usr/pfSensesrc

### 2\. Grab ports and build GIT
portsnap fetch extract 
cd /usr/ports/textproc/expat2 && make depends install
cd /usr/ports/devel/git && make depends install
cd /usr/ports/sysutils/fastest_cvsup/ && make depends install
rehash   

### 3\. Continue on with the GIT repo checkouts:
cd /home/pfsense && git clone git://github.com/pfsense/pfsense-tools.git tools
cd /home/pfsense && git clone git://github.com/pfsense/freesbie2.git freesbie2
cd /home/pfsense/tools/builder_scripts && chmod a+rx *.sh

### 4.0 prepare some settings
echo ## Make sure to use the correct OS sources (otherwise applying patches will fail):
cd /usr/home/pfsense/tools/builder_scripts/conf/svnup/
sed s,base/releng/8.3,base/releng/10.0, < svnup.conf > svnup.conf.new
sed s,base/stable/10,base/releng/10.0, < svnup.conf.new > svnup.conf.new2
sed s,svn.freebsd.org,svn0.us-west.FreeBSD.org, < svnup.conf.new2 > svnup.conf

cd /usr/home/pfsense/tools/builder_scripts/
echo "export USE_SVN=yes" >> pfsense_local.sh
sed s,stable/10,release/10.0.0, < pfsense_local.sh > pfsense_local.sh.new
mv pfsense_local.sh.new pfsense_local.sh

echo ## allow ports to be build against the newly installed openssl WITH_OPENSSL_PORT=yes
cd /usr/home/pfsense/tools/builder_scripts/conf/pfPorts/
sed s,WITH_OPENSSL_BASE,WITH_OPENSSL_PORT, < make.conf > make.conf.new
mv make.conf.new make.conf

### 4\. Ensure FreeBSD sources are present
cd /usr/ports/devel/subversion && make install clean
rehash
svn checkout https://svn0.us-west.FreeBSD.org/base/releng/10.0/ /usr/src

cp -iprv /usr/src /usr/pfSensesrc/

### 5.1\. Set the version you would like to build, you can do this from ./menu.sh also.
cd /home/pfsense/tools/builder_scripts
./set_version.sh RELENG_10_0

### 5.2\. Invoke the Apply patches option
cd /home/pfsense/tools/builder_scripts
./apply_kernel_patches.sh

## 5.2a Ensure BSDInstaller is sound
rm -rf /home/pfsense/installer
cd /home/pfsense/tools/builder_scripts
scripts/get_bsdinstaller.sh ; scripts/rebuild_bsdinstaller.sh

## 5.2b Build pfPorts manually (optional)
	## when rebuilding ports, some ports want/need the base system openssl version..
	## deinstall openssl port:
cd /usr/ports/security/openssl
make deinstall
cd /home/pfsense/tools/builder_scripts
./build_pfPorts.sh
	## troubleshooting building ports, check the 'failure logs' in:  /tmp/pfPort/buildlogs/  (should be empty when building is done)
	## for an idea about progress, check current console output and log in buildlogs folder and 
	## check progress against the file /usr/home/pfsense/tools/builder_scripts/conf/pfPorts/buildports.RELENG_2_2

## 5.3.0 to avoid error building cdrtools-3.00 "cc: error: no such file or directory: 'amd64'" while running build_iso.sh
cd /usr/ports/sysutils/cdrtools
make install clean	

## 5.3 Invoke the Build ISO option (no matter what kind of image you want)
	## now start the compiling and building of memstick images and iso file

cd /home/pfsense/tools/builder_scripts
./build_iso.sh

## p.s. this message is normal: "/usr/local/pfsense-clone/usr: write failed, filesystem is full"
## Which is explained a few lines above that.: "Don't worry if you see a 'filesystem full' message here" 

## now cross your fingers and wait for the message to show it has finished building which should look like this:</enter></enter></enter></enter></enter></enter></enter></enter>

p.s. if anyone knows some improvements to these steps please let me/us know!

eri--

These instructions seems more complicated than it really is.
Also you would want the OPENSSL from base.

PiBa

maybe indeed you want openssl from base, however then in my attempts then the build process isn't successful. as part of the ports compilation the newer openssl is installed again and the other ports complain they want base.
i know building ports manually can probably be skipped, as make_iso also builds the ports… but the official wiki also does that. and it is usefull when trouble does happen.

building cdrtools shouldnt be needed either, but solved getting the .iso besides the memstick versions which did build without that step.
changing the svnup.conf fixed applying patches

please tell us your "simpler" but successful build steps. as the steps in wiki didnt seem to work 100% for people (or at least for myself)

shaqan

you can avoid installing OpenSSL ports by using portmaster. It always presents you NCurse based "Option" menu, often you can choose other options instead of using OpenSSL, for example often it allows you choosing GnuTLS instead when some package asks for SSL support.

I did it with the Git. Did'nt install it by pkg install but went to /usr/ports/devel/git, started portmaster in it and laborously avoided choosing OpenSSL port anywhere. OpenSSL is always default choice, so you must be careful.

If the selected options do not play out and fail, you can always go to the folder of failing port and do make config (and make) in it again, and then build the port solo. Then go back to where portmaster failed and start it again. Portmaster jumps over point of previous failure and goes on.
So, step by step you can build necessary packages.

Got stuck with building ipfw-classifyd module though. Whatever I do, I get error. No clue about C so I am unable fixing it myself:S

Overall, 2.2 seems to build quite error free in it's present state

PS. Doing it for my own pleasure and interest, no reason to be afraid of sharing out buildimages..

_Adrian_

Anybody else had any luck with this ??
I've gotten called back in and working 60hour week again :/

shaqan

I managed to built couple of isos and tinker with them. Kept bumping into various issues but it sure was interesting. No time lately.

Btw, at some point found that pfsense-tools had been removed from github. If you happen to have your local backups then better keep'em

phil.davis

New pfSense features are being added to the "hidden" pfSense-tools and then used in the "ordinary" pfSense code. So even if you build from an old copy of pfSense-tools, the compiled "pf" and other utilities will be missing some command line options, conf file features… and so will not work with the PHP code in the main pfSense repo that uses the new features.
2.2 snapshots are now available at http://snapshots.pfsense.org/ - use those and join in the testing and debugging. Then we all benefit from a joint co-ordinated effort.

shaqan

As I understand now you torpedoed the users ability to build from source on purpose? Why, so suddenly?

phil.davis

"you" should refer to Electric Sheep Fencing (ESF) - if you were referring to me in some way, then please understand that I am a private individual and am not part of ESF.
You can read up on recent happenings in this post: https://forum.pfsense.org/index.php?topic=73101.0 and others in the Development forum section.

shaqan

no, did not mean You personally. More general way.

thanks for the link, it did lead me to a answer at
http://lists.pfsense.org/pipermail/dev/2014-February/000520.html

bhawk6901

trying to build 2.2 beta. get the error while building strongswan as shown in attachment

–- kernel_pfkey_plugin.lo ---
libtool: compile: cc -DHAVE_CONFIG_H -I. -I../../../.. -I../../../../src/include -I../../../../src/libstrongswan -I../../../../src/libhydra -I/usr/local/include -O2 -pipe -fno-strict-aliasing -include /usr/ports/security/strongswan/work/strongswan-5.2.0/config.h -MT kernel_pfkey_plugin.lo -MD -MP -MF .deps/kernel_pfkey_plugin.Tpo -c kernel_pfkey_plugin.c -fPIC -DPIC -o .libs/kernel_pfkey_plugin.o
--- kernel_pfkey_ipsec.lo ---
libtool: compile: cc -DHAVE_CONFIG_H -I. -I../../../.. -I../../../../src/include -I../../../../src/libstrongswan -I../../../../src/libhydra -I/usr/local/include -O2 -pipe -fno-strict-aliasing -include /usr/ports/security/strongswan/work/strongswan-5.2.0/config.h -MT kernel_pfkey_ipsec.lo -MD -MP -MF .deps/kernel_pfkey_ipsec.Tpo -c kernel_pfkey_ipsec.c -fPIC -DPIC -o .libs/kernel_pfkey_ipsec.o
--- kernel_pfkey_plugin.lo ---
libtool: compile: cc -DHAVE_CONFIG_H -I. -I../../../.. -I../../../../src/include -I../../../../src/libstrongswan -I../../../../src/libhydra -I/usr/local/include -O2 -pipe -fno-strict-aliasing -include /usr/ports/security/strongswan/work/strongswan-5.2.0/config.h -MT kernel_pfkey_plugin.lo -MD -MP -MF .deps/kernel_pfkey_plugin.Tpo -c kernel_pfkey_plugin.c -o kernel_pfkey_plugin.o >/dev/null 2>&1
--- kernel_pfkey_ipsec.lo ---
kernel_pfkey_ipsec.c:829:23: error: use of undeclared identifier 'SADB_X_EALG_AESGCM8'
{ENCR_AES_GCM_ICV8, SADB_X_EALG_AESGCM8 },
^
kernel_pfkey_ipsec.c:830:24: error: use of undeclared identifier 'SADB_X_EALG_AESGCM12'
{ENCR_AES_GCM_ICV12, SADB_X_EALG_AESGCM12 },
^
kernel_pfkey_ipsec.c:831:24: error: use of undeclared identifier 'SADB_X_EALG_AESGCM16'
{ENCR_AES_GCM_ICV16, SADB_X_EALG_AESGCM16 },
^
3 errors generated.
*** [kernel_pfkey_ipsec.lo] Error code 1

make[5]: stopped in /usr/ports/security/strongswan/work/strongswan-5.2.0/src/libhydra/plugins/kernel_pfkey
1 error

make[5]: stopped in /usr/ports/security/strongswan/work/strongswan-5.2.0/src/libhydra/plugins/kernel_pfkey
*** [all-recursive] Error code 1

make[4]: stopped in /usr/ports/security/strongswan/work/strongswan-5.2.0/src/libhydra
1 error
make[4]: stopped in /usr/ports/security/strongswan/work/strongswan-5.2.0/src/libhydra
*** [all-recursive] Error code 1

make[3]: stopped in /usr/ports/security/strongswan/work/strongswan-5.2.0/src
1 error
make[3]: stopped in /usr/ports/security/strongswan/work/strongswan-5.2.0/src
*** [all-recursive] Error code 1
make[2]: stopped in /usr/ports/security/strongswan/work/strongswan-5.2.0
1 error

make[2]: stopped in /usr/ports/security/strongswan/work/strongswan-5.2.0
*** [all] Error code 2
make[1]: stopped in /usr/ports/security/strongswan/work/strongswan-5.2.0
1 error
make[1]: stopped in /usr/ports/security/strongswan/work/strongswan-5.2.0
===> Compilation failed unexpectedly.
Try to set MAKE_JOBS_UNSAFE=yes and rebuild before reporting the failure to
the maintainer.
*** Error code 1
Stop.
make: stopped in /usr/ports/security/strongswan

openssl was set to base in make.conf. does it need to be changed to openssl with port?

shaqan

I've got FreeBSD 10.1-BETA2 base system set up, (ports like git, expat2 etc. all installed). Also, following (by now erased) guidelines in old dev wiki (still visible using Internet Archive 'going back in time'). pfsense-tools repo installed (after signing necessary agreements in portal.pfsense.org)

Questions.
-While the April version of the 'pfsense-tools' worked in tcsh shell pretty much out-of-the-box, at some point that code seems to have received major overhaul. Trying to pick some option from (/builder_scripts/scripts/)menu.sh ends invariably with an error 'filename.sh :not found'.
I suspect I would need to add some custom variable to the .tcshrc file. Could someone 'in the know' help me out?

Is there going to be another 'dev wiki'? Oriented for development of 3rd party plugins. Present help files seem to be oriented purely for installation and usage of pfSense. Old 'dev help' links still present in 'pfsense-tools' scripts are by now inoperable.

Klaws

I received an error stating "You must upgrade the ports-mgmt/pkg port first."

To solve that issue:

cd /usr/ports/ports-mgmt/pkg  
make  
make reinstall