PfSense in a high load environement

Kenotronix

Hi,

We are currently using pfsense as primary firewall/router for our organisation
We are also using a second pfsense for failover

128 publics IPs
100 mbits dedicated internet

WAN => Public IP
LAN => Primary natted network (We are not using the 1:1 NAT because it didn't work, we are creating Outbound and Forward rulz for each server)
FW => Dedicated pfSense Interface
Client001 to Client010 => Client natted network
Private => No nat, for VPN only, our local network

We created one CARP in each network, and created a carp for each Public IP. Our server are using the network CARP for gateway, and our NAT rulz are using the public CARP for IPs

We are using the 1.2.3 Virtual Appliance from the pfSense website.

RAM: Changed for 4GB
CPU: Changed 4 vCPU

The first errors messages was:
maxproc limit exceeded by uid 0, please see tuning(7) and login.conf(5)
approaching the limit on PV entries, consider increasing either the vm.pmap.shpgperproc or the vm.pmap.pv_entry_max tunable

That made the server crashed, the firewall stopped to works, and my secondary pfsense wasn't becoming Master. I tried to Disable/Enable carp on the Secondary pfSense but, was still "Backup". All my server/NAT stopped working.

I edited the /boot/loader.conf and added:
kern.maxproc=20000 (was around 6000 when looking the "limit" command)
vm.pmap.shpgperproc=500 (was 200 with sysctl vm.pmap.shpgperproc)

Now, the error message I receive is:
vm_thread_new: kstack allocation failed

So,

Can you provide me the Best Practice for large setup?

What is the new error?

How can I make it works?

Is it better to use the Virtual Appliance or to re-install it with the installer in the VMs???

Looking forward for help, my network is having a lot of issues! =-/

Thx, pfSense is so great!!!

Tommy Boucher
Vice-President
Kenotronix Ltée
www.kenotronix.com

cmb

The maxproc limit exceeded is likely from large numbers of reflected connections, in large scale environments you don't want to use reflection. If you aren't using reflection, that would be a buggy package of some sort, it's most likely reflection though.

Kenotronix

Hi, thx for reply

We are using reflected connections because local servers wasn't able to access other server using the Public IP, only the local IP

We need to be able to access Public IP

How can I make it works?

Tommy Boucher
Vice-President
Kenotronix Ltée
www.kenotronix.com