PfSense: Unattended installation with Foreman
-
@gonzopancho:
Im not sure you know what you're talking about.
This was rude and inappropriate.
I am talking about configuration management tools, while you are talking about a pfSense API. Two completely different things in my opinion. You seem to disagree on my point of view:
@gonzopancho:
When pfSense has an API, puppet/chef/anoble/salt stack/cfengine/…. Can all use it.
The point is that these tools require client software to be installed on pfSense. And you seem to ignore this fact when pointing to the pfSense API.
This requirement applies to basically every configuration management tool you have mentioned:
Puppet
"The puppet agent […] fetches configurations from a master server."
https://docs.puppetlabs.com/learning/agent_master_basic.htmlChef
"The Chef client is installed on each node in your network."
http://www.getchef.com/chef/#how-chef-worksSaltStack
"Salt functions on a master/minion topology. […] The minions connect back to the master."
http://docs.saltstack.com/en/latest/topics/tutorials/walkthrough.htmlCFEngine
"The CFEngine agent runs on each host […]"
http://cfengine.com/product/what-is-cfengine/If you decide to boycott all client software, you isolate pfSense from all of the above configuration management tools.
@gonzopancho:
How do you manage IPMI ports, switches, routers, load balances, vpn gateways, etc?
The approach of using puppet/chef/salt stack/cfengine to manage network devices is rather new. As I said earlier [1], there is already a Puppet agent available for Juniper switches and routers. You need to install the Puppet agent on your Juniper switch or router. To get an idea how Puppet is used to configure such devices have a look at this example from Juniper:
https://github.com/Juniper/puppet-netdev-stdlib-junos#example-usage[1] https://forum.pfsense.org/index.php?topic=79397.msg433296#msg433296
Regards
- Frank
-
Frank, you're now making my argument for me.
The code you pointed to works via an API, and does NOT run on. Juniper device.
-
@gonzopancho:
The code you pointed to works via an API, and does NOT run on. Juniper device.
You're wrong. Apparently this code connects locally to the netdev API, but it still requires a Puppet agent on the Juniper device to run.
Regards
- Frank
-
Does this really have to be an either-or situation?
I'm personally unlikely to have a need for either solution (at least in the near term) but I fail to see how having the choice can be bad thing. Perhaps I'm missing something. :-\Steve
-
@gonzopancho:
The code you pointed to works via an API, and does NOT run on. Juniper device.
You're wrong. Apparently this code connects locally to the netdev API, but it still requires a Puppet agent on the Juniper device to run.
Regards
- Frank
Frank,
In this case, I am 100% right. You're not going to run that code on a Juniper switch (or router, or wireless LAN device, etc.)
For highly similar reasons, it is inadvisable to run your Puppet stuff (including Ruby) on a pfSense installation.
-
Does this really have to be an either-or situation?
I'm personally unlikely to have a need for either solution (at least in the near term) but I fail to see how having the choice can be bad thing. Perhaps I'm missing something. :-\Steve
Imagine this scenario:
Frank (or whomever) creates a "solution", and it manages to get popular with, say, 5% of the installed base of pfSense. Then pfSense changes (for whatever reason), and the solution doesn't work anymore.
The developers of pfsense suddenly have a horrible choice to make: a) support the formerly unsupported module, or b) support "the community".
This is why I've gone to lengths to make sure that Frank knows that what he's doing isn't supportable, and that my strong preference is that he work via the (still not fully-developed) API.
-
It would certainly be bad if such a package became sufficiently well used that it was assumed to be part of pfSense. I guess you could argue similar things about other packages though.
Could be another argument for dividing packages into community and official.Steve
-
@gonzopancho:
In this case, I am 100% right. You're not going to run that code on a Juniper switch (or router, or wireless LAN device, etc.)
Have a look at the Juniper documentation or just this picture:
http://www.juniper.net/techpubs/images/g041409.gif
http://www.juniper.net/techpubs/en_US/junos-puppet1.0/topics/concept/automation-junos-puppet-overview.htmlIt's very clear, that the code runs on the Juniper switch. Along with the Puppet agent a Ruby interpreter is installed on the Junos device to actually run code.
You may still claim that the code does not run on the device, although the official documentation clearly states that this is the case. You may need to read the puppet documentation if you insist to not accept my arguments. And yes, the same goes for every other configuration management system we've been talking about.
Regards
- Frank
-
Could be another argument for dividing packages into community and official.
Yes, this would be a good solution. It would shift the responsibility for unsupported packages to the community. And the developers would be able to say "look, it's unsupported, you've been warned". I'm OK with this. It gives people choice.
Regards
- Frank
-
@gonzopancho:
This is why I've gone to lengths to make sure that Frank knows that what he's doing isn't supportable, and that my strong preference is that he work via the (still not fully-developed) API.
It is possible to develop Puppet modules and providers that make use of the API locally (similar to the netdev approach on JunOS devices). This would make it much harder to break things. But it would still require a Puppet/Chef/… agent installed on pfSense.
I'm not against the API, but I would prefer to be able to choose how I make use of it.
Regards
- Frank
-
Could be another argument for dividing packages into community and official.
Yes, this would be a good solution. It would shift the responsibility for unsupported packages to the community. And the developers would be able to say "look, it's unsupported, you've been warned". I'm OK with this. It gives people choice.
Regards
- Frank
I think it would be a great idea to separate the packages.. Or at least mark them somehow/ maybe another column that they are community packages…
-
it goes way beyond "community" .vs "official".
-
@gonzopancho:
it goes way beyond "community" .vs "official".
Sounds like important information is still unspoken. Could you please elaborate more on this?
Regards
- Frank
-
Hello,
I'm just a recent user of pfSense, and I like it. It's currently installed as a front VM on 4 or 5 of our servers, to protect some VMs behind - using Snort and Pound reverse firewall to add some extra security.
In next months we'll probably have 10s or even 100s VMs to deploy, and many VMServers.
So of course we're looking for solutions to automate installations, system upgrades, application deployments… As pfSense is an important part of the architecture, we'd prefer not installing manually, going through our checklist.
I've seen that Puppet (seemed to) provide some solution, so it is a nice candidate for our automation tool.
-
After reading this thread, it seems that Puppet won't be an "official" package - is this right ?
-
Is there any rough estimate about the availability of an API ?
-
I guess that each module (OpenVPN, Snort...) should have to provide its API, too... Right ?
The API way seems to be a reasonable one - if I read well, it's the way used to configure and manage VMs on VMWare, isn't it ?
- If API comes in a far or unknown future, and no solution can be found for inserting Puppet on pfSense, what other solution you experienced admins would suggest ?
Thanks in advance,
Fred
-
-
The strategy to enable Puppet (Chef, Saltstack, etc) is to fit an API. The webGUI will also be re-written to access this API.
That means a lot of pfSense (a lot of the PHP) will need to be re-written.The Puppet packages published by fraenki will not become official in any sense of the word. They are too promiscuous with the system.
Jim
-
Hooray, I am responding to another dead thread….
It is not uncommon in puppet module development to have to re-work the module with each "major" (and sometimes even minor) release version of an application. That is the responsibility of the module maintainer. Anytime you are managing the configuration of something, whether it is clamav, ntpd, apache, mysql, or whatever, you always have to be careful of "updates" and how they affect the configuration files. This is not a puppet specific problem. I have had the same problem with Spacewalk and even "old fashioned" scripts that try to make changes to configuration files in the old days. Using an API, once it is available, is ideal because you can only do what the API supports, and, the API would have release notes accompanying any changes, including, most likely, a major version (/apiv1 or /apiv2) when it makes significant or non-backawards-compatible changes. With an API, testing becomes a lot more simple and finite. That is good!
However, I have run into this same problem with "services" like "Rackspace CloudDB", "Amazon RDS", etc as well. They all provide an API as well, but it is not possible to install a puppet client (for example) on those machines. I do not have a "solution" for this yet. Maybe we need to have some sort of centralized "administrative workstation" that integrates with the configuration management, such as Puppet, Chef, Ansible, etc, and then applies configuration changes via the API to the "managed device" or "service" in question. The API is the interface by which the "client" affects the operations of the appliance. The overall solution still needs to include the client side that talks to the API as well. In this case, the "client" that talks to the API would be a puppet provider.
The "normal" configuration management architecture is to run a "client" (agent) on the device being managed. The client gathers up some details about the device itself, such as memory, cpu, disk, network interfaces, etc, (facts). This "client" sends those details to a centralized configuration management system (puppet master), which is able to reference configurations of other systems (exported resources), and its own datastore (hiera) to determine what the correct configuration state of the client should be. It then returns this "state" to the client (catalog), which the client applies as necessary to make sure its configuration matches the "correct" configuration. The "applies" step should be idempotent, such that it can be applied over and over without being any different than the first time it was applied. (i.e. it doesn't reinstall a package every time, if it is already there). This "applies changes" step is where the API comes in. The API may also be involved in the "facts" gathering. In lieu of an API, there may be a "command line interface" that can apply changes through a series of administrative commands, such as https://doc.pfsense.org/index.php/Using_the_PHP_pfSense_Shell which should follow similar "release notes" type of changes to an API. However, any changes made to the system "directly" (via modifying a file, for example) would be "extra risky" and would require extensive testing by the person maintaining the configuration management module before certifying compatibility with the "next" version of pfSense. None of this would be the responsibility of the pfSense maintainers directly. This really wouldn't be that much different from any other package installed on the system. It should be tested by the maintainer of the package before it is "released" to that version of pfSense.
The long and short of it is that Puppet and the API are not competing solutions. The puppet client is an "implementation" that should (when available) use the API to make changes to the system. In the case where the "managed device" is running an OS that has a puppet client available, it makes sense to run the client locally on the hardware that is being managed. It is just less overhead. It would be great if it ran natively or using an "existing" interpreter, such as PHP, but that is another project for another day. :)
Sorry for my long winded response :D
~tommy
-
https://blog.pfsense.org/?p=1588
Now hook in some BSDploy (http://docs.bsdploy.net/en/latest/)
