Reducing pfSense startup time and resource usage as much as possible



  • Hi everyone,

    I'm currently evaluating using pfSense as a replacement for an old floppy firewall solution that we've been using at work. While we like what we've seen so far, many questions are being asked about how to do each of the following as much as possible based on our use case:

    • Decrease the pfSense startup time

    • Reduce the memory requirement

    • Reduce the storage requirement

    I reached out to pfSense about having paid support help me figure this out, and they suggested I use these forums instead, so I'm hoping someone can assist.

    Essentially we want to use pfSense as a NAT Gateway, with DNS, DHCP, and NAT. Anything else that it offers may be useful for some scenarios, but for the specific scenario I'm looking at (replacing our floppy firewall), that's all we care about. We launch thousands of VMs a day on Hyper-V and want to use pfSense VMs with whichever of these VMs need an IP address and Internet access. Due to this scale, we really need to be critical about the resource usage and launch time of the pfSense VMs we use.

    Can anyone on this forum point me in the right direction on how I might be able to configure pfSense in a VM to be as lean as possible, focusing on DNS, DHCP, and NAT support?

    Thanks in advance. I need to make a decision on this very soon, so I appreciate anyone who has time to share their experience with pfSense and help me out with this issue.

    Kirk



  • -i don't know any way to decrease the startup time. (except tweak the bootloader delay, that might gain you a few secs)

    -i run a couple of pfSenses on 256mb of ram, this works but it is probably the low end

    hyper-V isn't freebsd's favorit hypervisor, but support is getting better i read: check the virtualization subsection and search for threads about it


  • LAYER 8 Global Moderator

    How fast does your floppy vm firewall boot?  I am running pfsense on a vm on esxi 6.5, the esxi host is using SSD.  But its an older host, and only has sata 2 for the ssd interface.  And pfsense boots really quick..  Next time I need to do a reboot I will time it..

    If your not running any heavy packages then pfsense works fine with 256 without any issues.

    You can get rid of the f1 boot thing to shave a few seconds.

    But I would be curious to how fast your current floppy vm boots.  That would be the goal to beat or meet right?



  • Thank you for the suggestions on what RAM seems to work as a minimum.

    Our virtual floppy firewall boots fully and is ready to route traffic in about 10 seconds. We don't need (or necessarily expect) pfSense to startup that quickly, but do we want to have it start as fast as possible.

    On the FreeBSD/Hyper-V comments, thank you, I will take a look in the Virtualization forums to see what was discussed there.


  • LAYER 8 Global Moderator

    hmmm, 10 seconds.. I don't think mine is that fast.. But I don't think its off by that much..  You have me curious now ;)  I will have to reboot it tonight or tmrw morning time it from off to when packets get through..



  • I have a UEFI install with pfSense 2.4 on Xen that boots in about 11 seconds, from a SSD-cached ZFS dataset. What helps is removing bootloader, menu and kernel delays and making sure your virtualisation platform doesn't mess with the bootup. Most of the time after bootup goes into expanding the XML configuration into actual configuration, so depending on how complex your configuration is you might have slower boot times. It also depends on the services you start, packages you add etc. I have most of them running on 1024MB RAM and 2 cores, but they work fine on one core and 512 and even 256. If you must go lower, then maybe pfSense isn't the right choice and a 8MB OpenWRT image might be more suitable. They are not as good als pf, but they are smaller and can be made to boot faster. They would basically be up and passing traffic within 5 seconds and you can direct-boot them from the kernel without a bootloader. Using 16MB disk space and 64MB RAM, it's about as small as you can get with a modern FOSS router. Smaller is of course possible, but you start trading off performance, capabilities and maintainablility.



  • I have a pfSense install on a production node (running XenServer 6.5) for VPN purposes. It takes about 40 seconds of boot.

    The node itself has an Intel(R) Xeon(R) CPU E5-1620 v2 @ 3.70GHz, 2x HGST HUS726020ALA610 configured in RAID1 and 128 GiB of RAM, and holds about 30 Virtual Machines.

    I am allocating 6 GiB of RAM to pfSense while it only uses about 512 MiB.
    Also the VM has a 10 GiB HDD, while only 1 GiB is occupied by the system and another 1 GiB for the manually configured SWAP partition.

    I'm happy with performance. I don't reboot the machine much.



  • @johnkeates: If I can get anywhere close to an 11 second launch, I'll be very happy. Since I'm completely new to pfSense and still learning the ropes, can you share some specifics or point me in the right direction towards being able to remove the bootloader, menu, and kernel delays?



  • Faster boot time != better overall performance. It's possible to cheat during the boot time quite a bit and that's what many Linuses do to achieve on the surface great looking performance. However, the real performance of a firewall/router has nothing to do with boot time but with the performance of the packet filter and the network stack.

    And seriously, are you going to be rebooting your router/firewall so often that the boot times actually have some significance?  :o



  • @kpa:

    Faster boot time != better overall performance. It's possible to cheat during the boot time quite a bit and that's what many Linuses do to achieve on the surface great looking performance. However, the real performance of a firewall/router has nothing to do with boot time but with the performance of the packet filter and the network stack.

    And seriously, are you going to be rebooting your router/firewall so often that the boot times actually have some significance?  :o

    I think it's more an issue of scale for his case. 100 routers using 1GB RAM and 2 CPU cores is quite expensive.


  • LAYER 8 Global Moderator

    "are you going to be rebooting your router/firewall so often that the boot times actually have some significance?"

    This is a great point - I didn't get a chance to boot mine.. But I think its going to be better than 40 seconds JorgeOliveira mentions.  And I have way older/slower hardware.. Maybe his host running so many other vms at the same time is slowing it down..  Or maybe I am too optimistic about its boot time.

    But even when I reboot the whole host, I have my pfsense vm boot first and its only a few minutes to having internet again.  And that is a full boot of the whole host.

    Went out for corned beef last night for pattys day - and this morning got side tracked before I had to leave the house..  Not planning any sort of scientific timing - just going to start a ping to outside from my workstation. Click the power button and time how long until pings start answering.



  • @johnkeates:

    @kpa:

    Faster boot time != better overall performance. It's possible to cheat during the boot time quite a bit and that's what many Linuses do to achieve on the surface great looking performance. However, the real performance of a firewall/router has nothing to do with boot time but with the performance of the packet filter and the network stack.

    And seriously, are you going to be rebooting your router/firewall so often that the boot times actually have some significance?  :o

    I think it's more an issue of scale for his case. 100 routers using 1GB RAM and 2 CPU cores is quite expensive.

    Yes, we will be booting the firewalls so often that the boot times are very significant, and John's right, it's absolutely about scale for us. We're evaluating this for use where we launch thousands of VMs daily, with individual VMs or small collections of VMs connected to a pfSense VM that is serving as their NAT gateway. Faster boot performance at that scale definitely counts, perhaps moreso than the performance of the packet filter and network stack (although we don't want to ignore the performance of those either).


Log in to reply