Re-implementing the entire backend/frontend of pfSense

MrMoo

BTW: you seem pretty interested and we all seem to be on the same page for the most part. What's it going to take to sucker you into helping us code? :P

My company (which is just me :D ) is dedicated to building two PDC and a printer/scanner/fax units before I can work on my real set of products (messaging stuff). I have a stack of various hardware and VMWare Server working busy already. My first step is trying to get a simple image with only networking and SSHD working.

If I really need to I will write a custom messaging core with glib and shared memory/sockets but i'm hoping to delay that and port it in after I have the two platforms in a working state.

The list of features I have planned is grandiose, but I have to keep things in check and actually get something working quickly. For added entertainment I need to get a English/Simplified Chinese/Traditional Chinese GUI working on it too :'(

sullrich

@MrMoo:

My company (which is just me :D ) is dedicated to building two PDC and a printer/scanner/fax units before I can work on my real set of products (messaging stuff). I have a stack of various hardware and VMWare Server working busy already. My first step is trying to get a simple image with only networking and SSHD working.

Interesting. I was just considering wrapping everything with SSH.

@MrMoo:

If I really need to I will write a custom messaging core with glib and shared memory/sockets but i'm hoping to delay that and port it in after I have the two platforms in a working state.

What would this be used for?

@MrMoo:

The list of features I have planned is grandiose, but I have to keep things in check and actually get something working quickly. For added entertainment I need to get a English/Simplified Chinese/Traditional Chinese GUI working on it too :'(

-CURRENT already has been gettext()'d. You're well on your way.

MrMoo

@sullrich:

@MrMoo:

If I really need to I will write a custom messaging core with glib and shared memory/sockets but i'm hoping to delay that and port it in after I have the two platforms in a working state.

What would this be used for?

Replacing "check_reload_status" with a more generic system so that scripts like /etc/rc.linkup end up as simple message sends and then the actual processing of the events is centralised and not subject to parallel running issues. Its more of an enterprise messaging bus way of thinking, so here's a very basic diagram with two system publishers: "rc.linkup" for an interface coming up, and "ps auxw" for a periodic poll providing the current process list.

![](http://fnjordy.umcus.org/tmp/messaging core.png)

sullrich

@MrMoo:

Replacing "check_reload_status" with a more generic system so that scripts like /etc/rc.linkup end up as simple message sends and then the actual processing of the events is centralised and not subject to parallel running issues. Its more of an enterprise messaging bus way of thinking, so here's a very basic diagram with two system publishers: "rc.linkup" for an interface coming up, and "ps auxw" for a periodic poll providing the current process list.

![](http://fnjordy.umcus.org/tmp/messaging core.png)

Beauty!!

MrMoo

Ok, things start to get messy. Here's a simple view of the most basic core:

The FreeBSD "rc" script starts up the "automator" which reads the public & private XML configuration. The private configuration specifies a "sshd" module and loads it in. The module creates a ModuleManager to manage basic state information, and a ProcessManager to look after the actual "sshd" main process.

Taking benefit of PHP5 the configuration is all SimpleXML, this allows trivial XML to Array conversion and very simple xpath load & save. It gets complicated with the concept of multiple configuraiton stores, here listed as runtime, public, and private. The Config class would first inspect runtime, then public, and finally private for any configuration data required. The question here is over the required functionality of a device such as pfSense. Currently with m0n0wall and pfSense there is no such concept of a runtime configuration, i.e. a transient configuration state that is lost when you reboot. At the development level there raises a question of the means of querying for data from the combined repository. While I would like to simply overide [] and -> on a base config object PHP does not allow this in a sufficiently powerful form as required. This means a namespacing solution is required of the form:
[code]
$config['modules.sshd.command'];

Or if a specific tree version is required:


$config->runtime['modules.sshd.command'] = '/sbin/sshd';

I have defined the ModuleManager as a fixed state engine, ideally it should be a base class for a module but I await demands of further integration. The modules themselves are required to implement a defined interface from the "Automator." The ProcessManager uses the popen suite of calls to manage all child processes similar to init on Linux. The goal is to remove the need for periodic 'ps' polling by maintaining the parent child link of each server. In this state stdout & stderr can be recorded for each process in a delayed pipe, and the actual process state can be defined as either not running, running, or trying to terminate. The additional benefit here is the ability to dissassociate the startup and kill penalty of some servers, especiallly those that have turned rogue and require a SIGKILL. Recording a state of "terminate" for a process allows us to check later for whether it has actually died and send further signals as necessary.

There's a question here that does this make syslog redundant, or should the process output be ignored and only syslog / application log file be used.

The next steps on my todo list are:

XML-RPC server in order to initiate:
Configuration or module recycling
Network configuration in order to test a non-developer image

Anything missing here?

MrMoo

Resolved the namespace issue, however a logical error with runtime/public, it needs to be an exclusive or operation in order for items removed from the public configuration to work.

Implementing load with xpath is non-trivial with SimpleXML API I need to investigate the DOM instead. Adding a SimpleXMLElement as a child throws an error when using a reference to keep the head of the XML document :-\

Oh well, you can enjoy the mistakes I made here:

http://fnjordy.umcus.org/tmp/miru-config.tar

MrMoo

In order to make xpathing in and out of the DOM easier I had to add the php5-dom module, but appears to work ok. I haven't implemented creating the parent nodes if they are missing from the destination xpath but not needed yet.

http://fnjordy.umcus.org/tmp/miru-config2.tar

The simpe test for xpathing is:

require_once 'Config.php';

$config = new Config();
$config->load_public('/conf/config.xml');
$config->load_private('/conf/private.xml');

$config->load_public('/conf/sshd.xml', '/miru/modules/sshd');

So this allows you to save and load specific nodes of the configuration, e.g. IPSEC, OpenVPN, traffic shaper and import them to a different configuration.

sullrich

I understand the topic "Re-implementing the entire backend/frontend of pfSense" but you are talking about rewriting large parts of the existing code that can claim a subnode pretty easily as an array, etc?

Just want to try and feel the entire scope of this project. Our hackathon is coming up soon and I need to finalize my list.

Example of getting a xml subnode how it is now:

require("guiconfig.inc");
$a_filter = &$config['filter']['rule'];

After looking at your code, how can it be retrofitted to work with the above example easily?

Or in your example $config is simply $config(an array)? Are there any downsides to this? Speed?

MrMoo

I'm working on process/module control and working out the requirements, firstly I think the big one is dependencies similar to Gentoo's runscript, here from svscan (supervise):


depend() {
  need net
  after net
  before ntpd ntp-client
  before spamd
  before apache apache2
}

The important area of functionality is what is required to reconfigure a process, usually one that cannot be simply SIGHUPed. The process needs to be stopped, the configuration file re-written, and the process started. If this process and dependencies that need this process they need to be stopped & started. And an awkward issue of if it takes a minute to restart an IPSEC connection should you allow other things to occur at the same time.

Here is the basic process state machine:

![](http://fnjordy.umcus.org/tmp/process manager.png)

So to reconfigure something like sshd you have to be aware of the current process state, i.e.


switch (sshd->state) {
case NONE:
case FAILED_TO_START:
case KILLED:
case DIED:
case EXITED:
case STOPPED:
  start sshd;
  break;
case STARTING:
case RUNNING:
  stop sshd;
  on( stop, { start sshd } );
  break;
case STOPPING:
  on( stop, { start sshd } );
  break;
case ZOMBIE:
  exit( "need to reboot to continue." );
}

MrMoo

@sullrich:

Example of getting a xml subnode how it is now:

require("guiconfig.inc");
$a_filter = &$config['filter']['rule'];

After looking at your code, how can it be retrofitted to work with the above example easily?

Or in your example $config is simply $config(an array)? Are there any downsides to this? Speed?

Its a non-trivial re-architecture in order to split functionality at the cost of speed. On the backend the above code would transform into this:


global $config;
$a_filter = $config->filter->rule;

I have absolutely no idea how it would appear on the frontend as a request to the configuration process would be needed to retrieve the data. The benefit is the data is live in the configuration process as opposed to re-read from file every time.

This is basically a big investigation into whether the benefits gained are sufficient to down play the side effects. When looking at a more generic appliance framework it would be nice to have the extra functionality. :)

(edit) The underlying adjustment is making the rather synchronous functionality of the current setup to be asynchronous. Is it possible to make pfSense/m0n0wall more responsive to events that occur when they occur.

MrMoo

@sullrich:

Just want to try and feel the entire scope of this project. Our hackathon is coming up soon and I need to finalize my list.

Sure, considering the duplication of work going on with CVS-head I will start working from the other end and fitting samba, ldap, cups, hylafax, and sane into the current image. By then hopefully you will have a release of your XML-RPC based code and I can help out with that instead.

Project 1: Domain controller for LTSP

Add OpenLDAP server + SASL
Add SAMBA PDC

Project 2: Printer, Scanner, Fax server

Add CUPS, hpijs, hplip, gimp-print, foomatic, foomatic-db, ghostscript
Add Hylafax
Add Sane-backends & Sane-frontends

By basing of pfSense I could remove the routing/firewall specific components and GUI and replace with parts specific to the components for each project. Extra time allows me to update the pfSense framework, and if time becomes short I will simply have to treat each project as a series of regular pfSense packages. 8)

MrMoo

Ok, bad day, neither openldap-sasl-client, openldap-sasl-server, or samba install through freesbie, however cyrus-sasl made it. Hopefully better luck tomorrow. ???

sullrich

@MrMoo:

@sullrich:

Just want to try and feel the entire scope of this project. Our hackathon is coming up soon and I need to finalize my list.

Sure, considering the duplication of work going on with CVS-head I will start working from the other end and fitting samba, ldap, cups, hylafax, and sane into the current image. By then hopefully you will have a release of your XML-RPC based code and I can help out with that instead.

Project 1: Domain controller for LTSP

Add OpenLDAP server + SASL

Add SAMBA PDC

Project 2: Printer, Scanner, Fax server

Add CUPS, hpijs, hplip, gimp-print, foomatic, foomatic-db, ghostscript

Add Hylafax

Add Sane-backends & Sane-frontends

By basing of pfSense I could remove the routing/firewall specific components and GUI and replace with parts specific to the components for each project. Extra time allows me to update the pfSense framework, and if time becomes short I will simply have to treat each project as a series of regular pfSense packages. 8)

This very much falls into my long term goals. Everything you've said so far makes a lot of sense (no pun intended, hah!)

Although all of this is great on paper, it's going to be one hell-of-a-hike to get all of this done. We need a plan of attack so that I can started retrofitting head in this direction.

Any suggestions of how I can help get this started from my end?

edit: minor spelling fixes.

MrMoo

Update: Slight change of track. In order to get something working I tried running with Voyage Linux (Debian/WRAP) and got everything up working in the morning through their Knoppix install; a copy of FreeNAS because i'm waiting back from a Taiwan developer on some NFS issues with a NAS appliance, some slight modifications in order to allow root file permissions; Ubuntu desktop install with LVM and then copy everything over to NAS for a diskless server following on with the thin client setup of LTSP MueKow implementation. Hopefully tomorrow I can test that, see that Chinese input works correctly, and think about moving from VMWare Server to the real hardware boxes, then planning moving back to BSD for the directory server.

I"m off to Korea this week, then onto Macau, so no updates for a couple of weeks :-[

To the left of the layer 4 switch is this stack of hardware, hopefully be useful:

[img]http://fnjordy.umcus.org/tmp/stack-of-boxes.jpg

I guess I should be asking you which area of pfSense would you like to see improved the most?

MrMoo

Oh no, Korea lost to Switzerland. Oh well :'(

I had a quick look at MultiSeat LTSP and with Linux VServer it looks overly complicated. I'm either going to look at web infrastructure for an embedded PDC or try moving from Voyage Linux to the FreeNAS configuration. ???

I like the admin page for WordPress however it doesn't appear easily transferable to the amount of information required with a router or PDC package. Has the core pfSense been updated completely to the new XML GUI-Builder, or is it only packages?

sullrich

Only packages for the most part.

MrMoo

Well I moved over to FreeNAS core and things are working with a web interface now. This gets me a little closer to actually helping out. I have an image with Samba 4, TFTPD, DHCPD booting LTSP server and clients. Here's the TODO list:

:D

Kernel

Remove options / modules not used: need to track down better details of all options.
Add embedded specific modules, performance options, etc.
Create an embedded (no-HDD) kernel, and a generic (IDE & SCSI HDD) kernel.

O/S

Keep tftpboot files over an upgrade, requires different method to gzip|dd for imaging.
Replace OpenSSH with DropBear to save some space (worth the effort?).
Remove PAM (possible?).

Configuration

Private / Public / Runtime configuration split to allow testing new and different options.
Management daemon to co-ordinate system tasks.

Application

Support VLAN interface configuration.
VLAN DHCPD support.
Smaller Samba 4 binaries by removing uneeded modules, there are a lot statically compiled.
Various Windows stuff as I have no experience using AD.

Frontend

i18n/l10n support like pfSenses gettext().
AJAX edits and status feedback like pfSense.
Larger, clearer interface like WordPress, mainly CSS work I would believe.
Administration interface users like pfsense (is there a use with a AD server?).

I'm using Samba 4 to see how stable and big it is, its one process compared to using both OpenLDAP & Samba 3.

MrMoo

Ok, apart from killing my "worlds worst implementation of NFS" NAS appliance I have a functional test build, unfortunately tftpd-hpa seems a little fruity on FreeBSD and requires restarting when remounting /cf. Here are some screenshots showing how little I have done:

Active Directory configuration, absolute minimum parameters required:

DHCPD configuration, adding gateway & PXE boot filename:

TFTP configuration, slightly spartan, really needs a file manager like captive portal:

User & Groups modified to take posixAccount fields:

And the ISO/IMG files:

http://fnjordy.umcus.org/tmp/test.iso [ 26MB ]
http://fnjordy.umcus.org/tmp/test-generic-pc-img [ 12MB ]

Slight mess in order to get TFTPBOOT files up you need to SSH in and run the following then SCP the files over.

mkdir /cf/tftpboot
/sbin/umount -f /cf
/sbin/mount -w -o noatime /cf

I have a kernel an initrd image from Ubuntu Dapper with two configurations. One for a LTSP Ubuntu, basically regular Ubuntu Dapper Desktop install made diskless following the wiki instructions. The other for Ubuntu MueKow which loads the ltsp-client built from the server. The server is configured with NSS_LDAP for users & groups, and PAM_HEIMDAL to allow AD based logins. I couldn't get a crypted password up for shadow, but kerberos is the more secure method, albeit Samba 4 has no security :D

I wonder if I can find anyone who would actually use it ;)

MrMoo

Network diagram to make things a bit prettier: :-*

lightning

Kewl :) 8) ;)

Keep the good work and us posted, thanks for sharing with the community 8)