PfSense: Unattended installation with Foreman

fraenki

Hi,

as a perfect addition to the new Puppet Agent for pfSense [1], it is now possible to automatically deploy pfSense with Foreman [2]. I've created a small video to showcase the deployment of pfSense using Foreman: https://www.youtube.com/watch?v=iqIR0pjkdRc. It's not very entertaining, but it should give you an impression on how it works (even if you don't know Foreman). :)

You may download the required Foreman templates from Github:
https://github.com/fraenki/foreman-pfsense
You may also want to have a look at the Foreman Documentation [3] to find out how to add these templates to your instance of Foreman.

What are the benefits?

do fully unattended installations of pfSense
in conjunction with Puppet it allows you to automate basically every task (full lifecycle management)
choose from different versions of pfSense according to your needs

How does it work?

it assumes you want to use The Foreman to provision your servers
it assumes that pfSense can be automatically provisioned similar to FreeBSD [4]
on top of that assumption it's basically a set of patches for the pfSense Installer [5]
it assumes you want to use Puppet with pfSense

[1] https://forum.pfsense.org/index.php?topic=79397.0
[2] http://theforeman.org/
[3] http://theforeman.org/manuals/1.6/index.html#4.4.3ProvisioningTemplates
[4] http://projects.theforeman.org/projects/foreman/wiki/FreeBSD
[5] https://github.com/fraenki/foreman-pfsense/blob/master/freebsd/provision_pfSense_mfsBSD.erb#L58

Feedback & contributions are very welcome!

Regards
Frank

Since the project is open, you can do as Franki details, but you can't distribute the result, and you're actively taking a different trail than where the project is headed.

We're not putting Ruby on the box. Franki's work is, while interesting, a dead end where the project is concerned.

fraenki

@gonzopancho:

We're not putting Ruby on the box.

What are the reasons for that decision?

Regards
Frank

cmlara

Thanks Frank,

The subject just came up yesterday for "how to manage the pfsense box with puppet" and part of the thought was how to automate the deployment.

Looks like I got the start to the thanks to you.

Part of the cloud mentality, everything is controlled by Foreman/Puppet and that needs to include the firewall :)

fraenki

@cmlara:

The subject just came up yesterday for "how to manage the pfsense box with puppet" and part of the thought was how to automate the deployment.

You may already be aware that I've started developing some puppet modules for pfSense:
https://forge.puppetlabs.com/modules?sort=rank&q=pfsense

Regards

Frank

As before, you can't distribute an adulterated pfSense and still call it 'pfSense'.

We're putting an API in pfSense for exactly this reason. All of this "foreman"/"puppet" work will be useless, other than the value of having done it.

fraenki

@gonzopancho:

As before, you can't distribute an adulterated pfSense and still call it 'pfSense'.

I've never indicated that I want to distribute an adulterated pfSense. Neither I nor my employer have such plans. And this is not going to change as it would be pointless.

@gonzopancho:

We're putting an API in pfSense for exactly this reason. All of this "foreman"/"puppet" work will be useless, other than the value of having done it.

That's not exactly a reason to reject a puppet agent on pfSense. It's a very different approach. You cannot compare an API to configuration management with puppet/chef.

Regards

Frank

When pfSense has an API, puppet/chef/anoble/salt stack/cfengine/…. Can all use it.

fraenki

@gonzopancho:

When pfSense has an API, puppet/chef/anoble/salt stack/cfengine/…. Can all use it.

That's not how most configuration management systems work.

Regards

Frank

Im not sure you know what you're talking about.

How do you manage IPMI ports, switches, routers, load balances, vpn gateways, etc?

fraenki

@gonzopancho:

Im not sure you know what you're talking about.

This was rude and inappropriate.

I am talking about configuration management tools, while you are talking about a pfSense API. Two completely different things in my opinion. You seem to disagree on my point of view:

@gonzopancho:

When pfSense has an API, puppet/chef/anoble/salt stack/cfengine/…. Can all use it.

The point is that these tools require client software to be installed on pfSense. And you seem to ignore this fact when pointing to the pfSense API.

This requirement applies to basically every configuration management tool you have mentioned:

Puppet
"The puppet agent […] fetches configurations from a master server."
https://docs.puppetlabs.com/learning/agent_master_basic.html

Chef
"The Chef client is installed on each node in your network."
http://www.getchef.com/chef/#how-chef-works

SaltStack
"Salt functions on a master/minion topology. […] The minions connect back to the master."
http://docs.saltstack.com/en/latest/topics/tutorials/walkthrough.html

CFEngine
"The CFEngine agent runs on each host […]"
http://cfengine.com/product/what-is-cfengine/

If you decide to boycott all client software, you isolate pfSense from all of the above configuration management tools.

@gonzopancho:

How do you manage IPMI ports, switches, routers, load balances, vpn gateways, etc?

The approach of using puppet/chef/salt stack/cfengine to manage network devices is rather new. As I said earlier [1], there is already a Puppet agent available for Juniper switches and routers. You need to install the Puppet agent on your Juniper switch or router. To get an idea how Puppet is used to configure such devices have a look at this example from Juniper:
https://github.com/Juniper/puppet-netdev-stdlib-junos#example-usage

[1] https://forum.pfsense.org/index.php?topic=79397.msg433296#msg433296

Regards

Frank

Frank, you're now making my argument for me.

The code you pointed to works via an API, and does NOT run on. Juniper device.

fraenki

@gonzopancho:

The code you pointed to works via an API, and does NOT run on. Juniper device.

You're wrong. Apparently this code connects locally to the netdev API, but it still requires a Puppet agent on the Juniper device to run.

Regards

Frank

stephenw10

Does this really have to be an either-or situation?
I'm personally unlikely to have a need for either solution (at least in the near term) but I fail to see how having the choice can be bad thing. Perhaps I'm missing something. :-\

Steve

@fraenki:

@gonzopancho:

The code you pointed to works via an API, and does NOT run on. Juniper device.

You're wrong. Apparently this code connects locally to the netdev API, but it still requires a Puppet agent on the Juniper device to run.

Regards

Frank

Frank,

In this case, I am 100% right. You're not going to run that code on a Juniper switch (or router, or wireless LAN device, etc.)

For highly similar reasons, it is inadvisable to run your Puppet stuff (including Ruby) on a pfSense installation.

@stephenw10:

Does this really have to be an either-or situation?
I'm personally unlikely to have a need for either solution (at least in the near term) but I fail to see how having the choice can be bad thing. Perhaps I'm missing something. :-\

Steve

Imagine this scenario:

Frank (or whomever) creates a "solution", and it manages to get popular with, say, 5% of the installed base of pfSense. Then pfSense changes (for whatever reason), and the solution doesn't work anymore.

The developers of pfsense suddenly have a horrible choice to make: a) support the formerly unsupported module, or b) support "the community".

This is why I've gone to lengths to make sure that Frank knows that what he's doing isn't supportable, and that my strong preference is that he work via the (still not fully-developed) API.

stephenw10

It would certainly be bad if such a package became sufficiently well used that it was assumed to be part of pfSense. I guess you could argue similar things about other packages though.
Could be another argument for dividing packages into community and official.

Steve

fraenki

@gonzopancho:

In this case, I am 100% right. You're not going to run that code on a Juniper switch (or router, or wireless LAN device, etc.)

Have a look at the Juniper documentation or just this picture:
http://www.juniper.net/techpubs/images/g041409.gif
http://www.juniper.net/techpubs/en_US/junos-puppet1.0/topics/concept/automation-junos-puppet-overview.html

It's very clear, that the code runs on the Juniper switch. Along with the Puppet agent a Ruby interpreter is installed on the Junos device to actually run code.

You may still claim that the code does not run on the device, although the official documentation clearly states that this is the case. You may need to read the puppet documentation if you insist to not accept my arguments. And yes, the same goes for every other configuration management system we've been talking about.

Regards

Frank

fraenki

@stephenw10:

Could be another argument for dividing packages into community and official.

Yes, this would be a good solution. It would shift the responsibility for unsupported packages to the community. And the developers would be able to say "look, it's unsupported, you've been warned". I'm OK with this. It gives people choice.

Regards

Frank

fraenki

@gonzopancho:

This is why I've gone to lengths to make sure that Frank knows that what he's doing isn't supportable, and that my strong preference is that he work via the (still not fully-developed) API.

It is possible to develop Puppet modules and providers that make use of the API locally (similar to the netdev approach on JunOS devices). This would make it much harder to break things. But it would still require a Puppet/Chef/… agent installed on pfSense.

I'm not against the API, but I would prefer to be able to choose how I make use of it.

Regards

Frank