How to build pfSense on FreeBSD V10

_Adrian_

DONE !!
Woohohohooo

Not a single error too !!

Moving onto Step 5.2A…

Now on 5.2B

_Adrian_

Ugh… errors galore !!!

However...

Plan for failure
It is expected that the first try to build the ports, and the first try to build an ISO will fail, but subsequent attempts work. This is because certain parts expect files or binaries to be present that get built on the first run but not fully used until the second. This may get fixed eventually, but because the workaround is easy (just run the build again), it's not a priority.

I ran it again and still full of errors…
guessing thats due to V10 but i mainly think its because of this first initial error:

Operation ./build_pfPorts.sh has started at Sun Jan 26 12:26:30 MST
–> Preparing for pfPorts build buildports.RELENG_2_1
--> WARNING! We are about to run make includes.
--> Press CTRL-C to abort this operation. . . . .
==> Starting make includes operation. . .
make: "/usr/pfSensesrc/src/Makefile" line 178: warning : Couldn't read shell's output for "find /usr/pfSensesrc/src/sys/sys/param.h -mtime -0s"
make[2]: "/usr/pfSensesrc/src/Makefile" line 178: warning : Couldn't read shell's output for "find /usr/pfSensesrc/src/sys/sys/param.h -mtime -0s"
make[2]: "/usr/pfSensesrc/src/Makefile" line 178: warning : Couldn't read shell's output for "find /usr/pfSensesrc/src/sys/sys/param.h -mtime -0s"

==> Compiling pfPorts. . .

Executing Build ports.RELENG_2_1
Setting ports List : '/home/pfSense/tools/builder_scripts/../builder_scripts/conf/pfPorts/buildports.RELENG_2_1'
Setting MAKEJ_ports (-j) to ' '
Setting ports make.conf: '/tmp/pfPorts_make.conf'

After this everything is a catastrophe :/

Build [PACKAGE] (filename). . . Failed to build. Error log in /tmp/pfPorts/buildlogs/[PACKAGE]

_Adrian_

After a bit of messing around I got to menu.sh and found that there is an option that's not displayed on the wiki…
RELENG_10_0

Also edited and instead of merging with head I forced it to merge with 8.3
I Will try again...

Right now its at
" Obtaining FreeBSD sources RELENG_10_0-supfile. . . "

A Former User

Thanks for posting your progress on this.

I'm interested in building for freeBSD 10, for the improved Xen support that 10 has.

I just started downloading a v10 ISO to set up a build environment.

_Adrian_

No luck yet…
Just chasing my tail here... I'm pretty close, but something is causing the build to fail.
So will have to keep poking at it until I either figure it out or someone drops in with a few words of wisdom...

phil.davis

After a bit of messing around I got to menu.sh and found that there is an option that's not displayed on the wiki…
RELENG_10_0

Since FreeBSD 10 and pfSense 2.2 is brand new, the detail of the build instructions in the WiKi is going to talk about (FreeBSD 8.1 and pfSense 2.0.n) or (FreeBSD 8.3 and pfSense 2.1.n). So you will definitely have to always change anything like that, and use an educated guess to do a similar thing for (FreeBSD 10 and pfSense 2.2). And then work out what underlying stuff has changed in some non-obvious way also (either in FreeBSD itself, or the way it is built for pfSense).
Once the devs actually have a reasonable (FreeBSD 10 and pfSense 2.2) build themselves then I'm sure the WiKi will get updated, and there will be public snapshots.

_Adrian_

Thanks for the input phil.davis !

From what read / heard 2.2 will be built on V9 not on V10.
Mainly I'm after a running OFED stack…

phil.davis

From what read / heard 2.2 will be built on V9 not on V10.

pfSense 2.2 is definitely now based on FreeBSD 10 (there will not be any pfSense for FreeBSD 9).
You have already posted to this thread: https://forum.pfsense.org/index.php/topic,69856.0.html - which has a few people interested and waiting for a useable FreeBSD 10 + pfSense 2.2 build to pop out.

_Adrian_

Sorry…
Today i just been off my rocker LOL

I think tomorrow is a fresh start day... format and re-install everything from scratch.

_Adrian_

No updates as of yet…
I have been spending my time on the minivan that needs a motor. Will probably get back on this on the weekend

PiBa

Maybe this will help someone :D
But i could come to building an .iso
These are the steps i did:


DISCLAIMER these steps seemed to work for me for testing purposes.
but i dont know if they are "correct" and wont contain serius issues.
use at your own risk !

Also i noticed after using the build image and going through the webgui installation wizard, at the last step a system panic occurs...
After rebooting and resetting the webgui (option 11), most stuff seams to work..
Im not sure if this is an issue caused by the early version and development progress on pfSense 2.2 or if it is caused by my steps..

install VirtualBox and configure a virtual machine like this:
 - disk 20GB
 - mem  1024 MB
 - cpu  2x
 - network: 1 bridged

0\. Install FreeBSD
install:  FreeBSD-10.0-RELEASE-amd64-disc1.iso
	default key map
	options: lib32 ports src  NOT: doc,games
	disk:  automatic  (19GB disk / 1GB swap)

	-root pass: system
	-configure network interface 'em0' IPv4: dhcp
	-country: yours
	-services: sshd,dmpdev

	add users now? YES
		username: admin
		fullname: admin
		uid: <enter>logingroup: <enter>invite other groups: wheel
		class [default]: <enter>shell [sh]: <enter>home dir [/home/admin]: <enter>permissions default: <enter>password-based auth [yes]: <enter>empty/random: no
		pass: secret
	lockout[no]: <enter>another user: no

	## Final Configuration ##
		Exit
		Manual Configuration?: No
		reboot
## the system is now installed

	## to enable SSH login on console ##
		#become ROOT
		su -l
		#generate keys not needed: /usr/bin/ssh-keygen -A
		echo sshd_enable="YES" > /etc/rc.conf
		service sshd start

	## allow root to login over ssh (NEVER DO THIS on a production machine, only for testing/easy access)
		echo PermitRootLogin yes > /etc/ssh/sshd_config
		/etc/rc.d/sshd reload

	## SYSTEM NOW ENABLED for SSH root login ##
		login with PuTTY / SSH as root for running commands / WinSCP for editing and transfering files (use the root account to have access to everything)

# 0-a Update your FreeBSD Install
	mkdir -p /usr/src/crypto/openssl/ssl
	freebsd-update fetch
	freebsd-update install	

# 0-b Make sure your OS clock is accurate
	ntpdate 0.pool.ntp.org

#### checking out ports / sources. ####
### 1\. Begin pfSense bootstrap by running these shell commands:
echo "WITHOUT_X11=yo" > /etc/make.conf
echo "OPTIONS_UNSET=X11" >> /etc/make.conf
echo "BATCH=yo" >> /etc/make.conf
mkdir -p /home/pfsense/pfSenseGITREPO /usr/pfSensesrc

### 2\. Grab ports and build GIT
portsnap fetch extract 
cd /usr/ports/textproc/expat2 && make depends install
cd /usr/ports/devel/git && make depends install
cd /usr/ports/sysutils/fastest_cvsup/ && make depends install
rehash   

### 3\. Continue on with the GIT repo checkouts:
cd /home/pfsense && git clone git://github.com/pfsense/pfsense-tools.git tools
cd /home/pfsense && git clone git://github.com/pfsense/freesbie2.git freesbie2
cd /home/pfsense/tools/builder_scripts && chmod a+rx *.sh

### 4.0 prepare some settings
echo ## Make sure to use the correct OS sources (otherwise applying patches will fail):
cd /usr/home/pfsense/tools/builder_scripts/conf/svnup/
sed s,base/releng/8.3,base/releng/10.0, < svnup.conf > svnup.conf.new
sed s,base/stable/10,base/releng/10.0, < svnup.conf.new > svnup.conf.new2
sed s,svn.freebsd.org,svn0.us-west.FreeBSD.org, < svnup.conf.new2 > svnup.conf

cd /usr/home/pfsense/tools/builder_scripts/
echo "export USE_SVN=yes" >> pfsense_local.sh
sed s,stable/10,release/10.0.0, < pfsense_local.sh > pfsense_local.sh.new
mv pfsense_local.sh.new pfsense_local.sh

echo ## allow ports to be build against the newly installed openssl WITH_OPENSSL_PORT=yes
cd /usr/home/pfsense/tools/builder_scripts/conf/pfPorts/
sed s,WITH_OPENSSL_BASE,WITH_OPENSSL_PORT, < make.conf > make.conf.new
mv make.conf.new make.conf

### 4\. Ensure FreeBSD sources are present
cd /usr/ports/devel/subversion && make install clean
rehash
svn checkout https://svn0.us-west.FreeBSD.org/base/releng/10.0/ /usr/src

cp -iprv /usr/src /usr/pfSensesrc/

### 5.1\. Set the version you would like to build, you can do this from ./menu.sh also.
cd /home/pfsense/tools/builder_scripts
./set_version.sh RELENG_10_0

### 5.2\. Invoke the Apply patches option
cd /home/pfsense/tools/builder_scripts
./apply_kernel_patches.sh

## 5.2a Ensure BSDInstaller is sound
rm -rf /home/pfsense/installer
cd /home/pfsense/tools/builder_scripts
scripts/get_bsdinstaller.sh ; scripts/rebuild_bsdinstaller.sh

## 5.2b Build pfPorts manually (optional)
	## when rebuilding ports, some ports want/need the base system openssl version..
	## deinstall openssl port:
cd /usr/ports/security/openssl
make deinstall
cd /home/pfsense/tools/builder_scripts
./build_pfPorts.sh
	## troubleshooting building ports, check the 'failure logs' in:  /tmp/pfPort/buildlogs/  (should be empty when building is done)
	## for an idea about progress, check current console output and log in buildlogs folder and 
	## check progress against the file /usr/home/pfsense/tools/builder_scripts/conf/pfPorts/buildports.RELENG_2_2

## 5.3.0 to avoid error building cdrtools-3.00 "cc: error: no such file or directory: 'amd64'" while running build_iso.sh
cd /usr/ports/sysutils/cdrtools
make install clean	

## 5.3 Invoke the Build ISO option (no matter what kind of image you want)
	## now start the compiling and building of memstick images and iso file

cd /home/pfsense/tools/builder_scripts
./build_iso.sh

## p.s. this message is normal: "/usr/local/pfsense-clone/usr: write failed, filesystem is full"
## Which is explained a few lines above that.: "Don't worry if you see a 'filesystem full' message here" 

## now cross your fingers and wait for the message to show it has finished building which should look like this:</enter></enter></enter></enter></enter></enter></enter></enter>

p.s. if anyone knows some improvements to these steps please let me/us know!

eri--

These instructions seems more complicated than it really is.
Also you would want the OPENSSL from base.

PiBa

maybe indeed you want openssl from base, however then in my attempts then the build process isn't successful. as part of the ports compilation the newer openssl is installed again and the other ports complain they want base.
i know building ports manually can probably be skipped, as make_iso also builds the ports… but the official wiki also does that. and it is usefull when trouble does happen.

building cdrtools shouldnt be needed either, but solved getting the .iso besides the memstick versions which did build without that step.
changing the svnup.conf fixed applying patches

please tell us your "simpler" but successful build steps. as the steps in wiki didnt seem to work 100% for people (or at least for myself)

shaqan

you can avoid installing OpenSSL ports by using portmaster. It always presents you NCurse based "Option" menu, often you can choose other options instead of using OpenSSL, for example often it allows you choosing GnuTLS instead when some package asks for SSL support.

I did it with the Git. Did'nt install it by pkg install but went to /usr/ports/devel/git, started portmaster in it and laborously avoided choosing OpenSSL port anywhere. OpenSSL is always default choice, so you must be careful.

If the selected options do not play out and fail, you can always go to the folder of failing port and do make config (and make) in it again, and then build the port solo. Then go back to where portmaster failed and start it again. Portmaster jumps over point of previous failure and goes on.
So, step by step you can build necessary packages.

Got stuck with building ipfw-classifyd module though. Whatever I do, I get error. No clue about C so I am unable fixing it myself:S

Overall, 2.2 seems to build quite error free in it's present state

PS. Doing it for my own pleasure and interest, no reason to be afraid of sharing out buildimages..

_Adrian_

Anybody else had any luck with this ??
I've gotten called back in and working 60hour week again :/

shaqan

I managed to built couple of isos and tinker with them. Kept bumping into various issues but it sure was interesting. No time lately.

Btw, at some point found that pfsense-tools had been removed from github. If you happen to have your local backups then better keep'em

phil.davis

New pfSense features are being added to the "hidden" pfSense-tools and then used in the "ordinary" pfSense code. So even if you build from an old copy of pfSense-tools, the compiled "pf" and other utilities will be missing some command line options, conf file features… and so will not work with the PHP code in the main pfSense repo that uses the new features.
2.2 snapshots are now available at http://snapshots.pfsense.org/ - use those and join in the testing and debugging. Then we all benefit from a joint co-ordinated effort.

shaqan

As I understand now you torpedoed the users ability to build from source on purpose? Why, so suddenly?

phil.davis

"you" should refer to Electric Sheep Fencing (ESF) - if you were referring to me in some way, then please understand that I am a private individual and am not part of ESF.
You can read up on recent happenings in this post: https://forum.pfsense.org/index.php?topic=73101.0 and others in the Development forum section.

shaqan

no, did not mean You personally. More general way.

thanks for the link, it did lead me to a answer at
http://lists.pfsense.org/pipermail/dev/2014-February/000520.html