PfSense in a VMware cloud environment - The power of vmtoolsd & xmllint

gi-minni

I want to share my ideas how I intend to use pfSense as a flexible dynamic router in a VMware
cloud environment. Dynamic injection and provisioning of pfSense is required.
Here my goals:

1) Filter outbound connectivity & fence internal zone from other virtual environments
Use two private VLAN's. The promiscous primary PVLAN-ID for WAN, the community
secondary PVLAND-ID for LAN. PfSense is managing both side using bi-NAT (1:1) and
virtual IP configuration. This is another story not explained here, while off-topic.

2) Rock solid & secure environment
This is accomplished by booting the Live-CD and a small persistent partition holding config.xml
and open VM tools. This sandbox environment is secured by checksums and less vulnerable
being a read-only environment in volatile memory.

3) Automated Deployment
Packed as a OVA file it can be deployed as a single file from a local source or even HTTP, FTP
repositories into an ESXi, vSphere Center, Workstation environment.
It has only few files to handle:
OVA file is a tarred file containing the OVF shipment scheme below:

pfSense.ovf - The description file holding configuration information
pfSense.iso - The pfSense Live Cd
pfSense.vmdk - The persistent disk holding config.xml & open VM tools
pfSense.mf - The manifest files for checksum & integrity validation

4) It must configure itself without human intervention
Here comes the power of vmtoolsd & xmllint.
vmtoolsd is the vmware guest daemon. it has two fantastic way how to read things.

a) - vmtoolsd –cmd machine.id.get
I defined inside a pfSense.vmx file (the VMware configuration file) a key named machine.id
machine.id = "<pfsense>…</pfsense>"

Few adjustment are needed: remove newlines and create a single config.xml line, change the
xml-tag from double to single quote (See above) and don't include pictures. I guess the
machine.id key is a string type with the limit of 65535 bytes. Include only the necessary stuff
stay below 64 KByte and everything goes smooths.

Consider also the fact to use the compressing feature of xmllint. This reduce size with the impact
to have an unreadable transportation form. Create a compressed config.xml:
xmllint –compress /conf/config.xml --output /conf/config.gz.xml or use openssl as an
encryption platform if you need to obfuscate the pfSense configuration.

This can be used on single ESXi server non-cloud environment, shipping only the pfSense.vmx,
pfSense.vmdk and pfSense.iso files and adding the pfSense.vmx file to inventory. This shipment
scheme is different from the OVA/OVF scheme above.

b) - vmtoolsd –cmd "info-get guestinfo.ovfenv"
This is the powerful way to embed the config.xml file inside an OVF descriptor and reading it
at boot time. Use xmllint to extract key=value pairs (see xmllint explanation below)

There are two OVF transport mechanism:

Read OVF descriptor with open VM tools (Option 4b)
Include an ISO file with ovf-env.xml file (the OVF descriptor) and rui.crt file (certificate file of ESXi
server or vSphere Center Server). I prefer Option 4b

5) Automatic variable extraction & validation

xmllint is shipped with pfsense and checks not only the config.xml integrity
xmllint –valid /conf/config.xml - Test the config.xml

Here comes the power of xmllint
xmllint –xpath "//interfaces//lan//ipaddr//text()" /conf/config.xml - Using the
XPATH functionality you can get everything out of the config.xml file. The above
statement read the key internal ip address (ipaddr) of the router and return the value
stored with the key e.g. "192.168.1.1"

Of course some of you will ask me is this enough to run inside a cloud environment? I say no,
this is not the whole story, whats about the glue code to handle dynamic configuration and
dynamic injection of config.xml and how can this be implemented?

I am on the road to fit every thing together one appliance & other tools are needed to build a
working environment. Stay tuned more to come …

StylusPilot

Interesting idea gi-minni

Have you also looked at VMware NSX?

what do you think your solution would bring to the table over integrating the router/firewall directly into the Hypervisor level?

maybe you could also look at using pfSense + the vmware API for layer 3+ features from pfSense and turn pfSense into a awesome competitor for the juniper and arista networks gear.

most of the NSX is based on Nicira's Open vSwitch so here's hoping for compatibility :)