PfSense on SmartOS how-to ?

leecallen

I have pfSense up & running on SmartOS and so far I am quite happy with it. I am thinking of writing a how-to for this forum. Any comments or suggestions? Is there interest?

SmartOS is a virtualization platform, built from Illumos (a descendent of OpenSolaris). It boots from a flash drive, and provides a robust ZFS filesystem, and virtualization via KVM, qemu, and Solaris zones. It is lightweight and efficient and runs on fairly inexpensive (non-RAID) hardware.

Asgaroth

I would be interested in a howto on SmartOS and pfsense:)

The only issue I have with smartOS at the moment, and also, the only reason it is not running here at home, is because the kvm implementation on SmartOS does not have pci passthrough support yet (it's been a while since I checked if they support it now, but the last time I checked there was no pci-passthrough support (vt-d) in smartOS)

leecallen

I am underwhelmed by the response to this, so I don't plan to write that how-to.

No, SmartOS still does not implement PCI pass-through, and they have no intentions of doing so.

darz82

I'd be happy to see a how-to :)

appletalk

I would love to see a this tutorial.

I'm just experimenting with SmartOS and it's awesome. I'm a little light on the KVM portion of it, so a tutorial would be a wonderful asset.

Donation link? :):)

I'm VERY interested.

leecallen

I have not written any real documentation on this… I am not seeing enough demand to warrant the effort. But I will include my own personal installation notes here. And I would be glad to answer any questions and try to help people through any problems they encounter.

Caveat: I am not yet using this (pfSense under SmartOS) for production. I have a nagging problem I have not been able to figure out. I think pfSense is not properly renewing leases with my ISP. Or something like that.

Also note this procedure assumes the SmartOS server has two physical interfaces. It can certainly be made to work with a single interface. Please let me know if that's a requirement.

Warning: There is significant potential for error in mapping the physical
ethernet interfaces through the virtual interfaces that are assigned to the
pfSense VM, and then to the actual WAN/LAN interfaces. Be very careful in
tracking these interfaces throughout this process.

download to a Linux or Mac system:

pfSense-LiveCD-2.0.3-RELEASE-amd64.iso.gz

gunzip the file

I generally create a permanent home for ISO images and JSON files:

zfs create zones/images

place the pfSense ISO there

identify the MAC addresses of the Ethernet interfaces

dladm show-phys -m

e1000g0 is probably the 'admin' interface, verify this by viewing /usbkey/config.

on my system, e1000g0 is the admin interface and is the ethernet connector

closest to the VGA connector

Then edit /usbkey/config and add a line for the 2nd MAC address as follows:

external_nic=xx:xx:xx:xx:xx:xx (substitute the 2nd MAX address)

re-boot SmartOS

Create the pfsense.json file (below) in zones/images

Modify the IP information as appropriate

The admin NIC is shared with SmartOS so it should be on the INSIDE (LAN) subnet,

the external NIC is the INTERNET/PUBLIC/WAN side

Note the vnc port number is specified - this must be unique.

The VM is set to not autoboot - change this later using 'vmadm update $UUID autoboot=true'

{
  "brand": "kvm",
  "vcpus": 1,
  "ram": 1024,
  "hostname": "pfsense",
  "alias": "pfsense",
  "resolvers": ["192.168.0.1", "8.8.8.8"],
  "vnc_port": "40000",
  "autoboot": "false",
  "disks": [
    {
      "boot": true,
      "model": "ide",
      "size": 4096
    }
  ],
  "nics": [
    {
      "nic_tag": "admin",
      "model": "e1000",
      "ip": "192.168.0.16",
      "netmask": "255.255.255.0",
      "gateway": "192.168.0.1",
      "allow_dhcp_spoofing": true,
      "allow_ip_spoofing": true,
      "allow_mac_spoofing": true,
      "allow_restricted_traffic": true,
      "primary":"1"
    },
    {
      "nic_tag": "external",
      "model": "e1000",
      "ip": "192.168.2.1",
      "netmask": "255.255.255.0",
      "gateway": "192.168.0.1",
      "allow_dhcp_spoofing": true,
      "allow_ip_spoofing": true,
      "allow_mac_spoofing": true,
      "allow_restricted_traffic": true
    }
  ]
}

vmadm create -f pfsense.json

substitute the created VM's UUID for $UUID in the following commands, or

export UUID=zoneuuid
cp /zones/images/pfSense-LiveCD-2.0.3-RELEASE-amd64.iso /zones/$UUID/root/
vmadm boot $UUID order=cd,once=d cdrom=/pfSense-LiveCD-2.0.3-RELEASE-amd64.iso,ide

This step can probably be done before booting the VM – and should be, if possible

examine the active JSON using:

vmadm get $UUID | less

and write down the last 4 digits of the MAC addresses for the admin and external nics, eg:

admin=a9:af

external=aa:ab

vnc to the IP address and port 40000

if you reach the session before the boot timeout occurs, take option "i" to install

respond to prompts as follows:

Accept these options
Quick/Easy install
Standard Kernel

after the reboot look for some lines that say:

Valid interfaces are:

em0 xx:xx:xx:xx:xx:xx

em1 xx:xx:xx:xx:xx:xx

Determine which of these matches the "admin" MAC address you noted earlier -- that is your LAN interface!

The other MAC address should match the "external" MAC address you noted - that is your WAN interface!

Do you want to setup VLANs now? N
Enter the WAN interface name...: em? (select the interface with a MAC address matching your external_nic)
Enter the LAN interface name...: em? (select the interface with a MAC address matching your admin_nic)
Enter the optional 1 interface name...: (enter)

From the menu:
2: Set interface(s) IP address
remember WAN = external_nic
enter IP, netmask as prompted
Do you want to revert to HTTP as the webConfigurator protocol? Y

Repeat menu option #2 for WAN, LAN

11: Restart webConfigurator
14: Enable Secure Shell (sshd)

koaps

I did a write on this the other day, looking at using pfSense in a lab I just built.

Can find it here:

http://nullworks.wordpress.com/2013/12/19/pfsense-on-smartos-kvm-zone/

Similar to what you did, but using pfSense 2.1 on KVM and I build a dataset image so I can put the image on multiple servers if needed.

leecallen

koaps, that is pretty cool. I may use your concise instructions & example for creating a dataset image for another project.

I see you are using virtio. Have you tested your firewall enough to know whether it's stable with virtio?

c0mputerking

I am very interested in getting this to work, and have been struggling with it for over a week now, think my main problem is that i only have one physical network interface and one outside ip address. Is it possible to use a kvm as the firewall with one ip? I have tried to many things to mention here and am thinking about getting a second ip to make this work but wondering if it is necessary?

Thrae

@c0mputerking:

I am very interested in getting this to work, and have been struggling with it for over a week now, think my main problem is that i only have one physical network interface and one outside ip address. Is it possible to use a kvm as the firewall with one ip? I have tried to many things to mention here and am thinking about getting a second ip to make this work but wondering if it is necessary?

Sure, you'd just have pfSense using pf to filter the incoming and outgoing packets for that one NIC, like a personal firewall for the machine. You couldn't do effective routing however until you add more NICs either as VLANs with a VLAN-capable switch, or another physical NIC.

Note: Since KVM on Illumos (OpenIndiana, SmartOS, OmniIT, etc.) lacks VT-d (PCI Passthrough) support, you wouldn't be able to use pfSense to setup any VLANs, and would have to do it on the host.

sbalaji_vkb

Hi leecallen, I'm also setting up pfSense in smartos KVM. I've one question from your above example as I'm new to networking.

The gateway IP "192.168.0.1" you had mentioned for both the NICs is different from LAN IP. I thought that the gateway is same as LAN IP (since LAN a is DHPC server). Is my understanding correct?

I want to use pfSense as DSHP server for LAN and use my existing wifi router as just a bridge. Thanks.

rainabba

@Asgaroth:

I would be interested in a howto on SmartOS and pfsense:)

The only issue I have with smartOS at the moment, and also, the only reason it is not running here at home, is because the kvm implementation on SmartOS does not have pci passthrough support yet (it's been a while since I checked if they support it now, but the last time I checked there was no pci-passthrough support (vt-d) in smartOS)

I'm a fan of ZFS and only just discovered Joyent SDC and SmartOS so I could see myself ending up there in the near future and will then need a pfSense instance so I'd love to see more support.