PfSense in a high load environement

  • Hi,

    We are currently using pfsense as primary firewall/router for our organisation
    We are also using a second pfsense for failover

    128 publics IPs
    100 mbits dedicated internet

    WAN => Public IP
    LAN => Primary natted network (We are not using the 1:1 NAT because it didn't work, we are creating Outbound and Forward rulz for each server)
    FW => Dedicated pfSense Interface
    Client001 to Client010 => Client natted network
    Private => No nat, for VPN only, our local network

    We created one CARP in each network, and created a carp for each Public IP. Our server are using the network CARP for gateway, and our NAT rulz are using the public CARP for IPs

    We are using the 1.2.3 Virtual Appliance from the pfSense website.

    RAM: Changed for 4GB
    CPU: Changed 4 vCPU

    The first errors messages was:
    maxproc limit exceeded by uid 0, please see tuning(7) and login.conf(5)
    approaching the limit on PV entries, consider increasing either the vm.pmap.shpgperproc or the vm.pmap.pv_entry_max tunable

    That made the server crashed, the firewall stopped to works, and my secondary pfsense wasn't becoming Master. I tried to Disable/Enable carp on the Secondary pfSense but, was still "Backup". All my server/NAT stopped working.

    I edited the /boot/loader.conf and added:
    kern.maxproc=20000 (was around 6000 when looking the "limit" command)
    vm.pmap.shpgperproc=500 (was 200 with sysctl vm.pmap.shpgperproc)

    Now, the error message I receive is:
    vm_thread_new: kstack allocation failed


    Can you provide me the Best Practice for large setup?

    What is the new error?

    How can I make it works?

    Is it better to use the Virtual Appliance or to re-install it with the installer in the VMs???

    Looking forward for help, my network is having a lot of issues! =-/

    Thx, pfSense is so great!!!

    Tommy Boucher
    Kenotronix Ltée

  • The maxproc limit exceeded is likely from large numbers of reflected connections, in large scale environments you don't want to use reflection. If you aren't using reflection, that would be a buggy package of some sort, it's most likely reflection though.

  • Hi, thx for reply

    We are using reflected connections because local servers wasn't able to access other server using the Public IP, only the local IP

    We need to be able to access Public IP

    How can I make it works?

    Tommy Boucher
    Kenotronix Ltée

Log in to reply