PfSense - ESXi 4.1 - 4 NICs?

[Supermule]

Are you running on bare metal or in a VM on a windows/linux box?

[nojstevens]

currently bare metal with 2 dedicated intel nics. planning to go virtual with esxi. pondering if i should put the intel nics in the host or just make logical ones using the two that are there already

Jon

[Supermule]

U can emulate E1000 NIC's in a VM, so just use the ones there, and if you need more, then VLAN them. :)

[nojstevens]

Hello,

Thanks for the help so far. Attached is a drawing taken from a ss in ESXi. Is this the correct vSwitch config for pfSense? Do I need to edit any of the settings inside the vswitch?

Jon

[Supermule]

You need to attach the PFsense to the VMnic1 port group.

[nojstevens]

Supermule, thanks for bearing with me. Like this then?….

Do I need to allow 'Promiscuos' mode on either the LAN or WAN?

Jon


[Supermule]

Promiscious mode is for changing the Vswitch to a hub…..so it distributes the traffic to all ports....Not good in most setups.

Not good in this one either since you have your vmkernel network and mgt network on the same switch.

Its not good network practice and I would use VLAN tagging on the Vswitch to override the most obvoius attacks that can occur. I know its on your LAN side, but I hate when the kernel network is on the same VLAN ID as the main FW.

[cyber7]

Hi nojstevens
I run 2ESXi 4.1 with 4NIC's (2 unsupported CARDS) .  The two machines are built with exactly the same specs:
CPU - 2QUAD XEON (Giving me 8 processors)
MEM - 26GIG
HDD - 3SAS300GIG + 5SAS1TB, 2RAID 5
MB Intel Server
NiC - 2onboard + 2*D-Link(unsupported according to VMWare, but hand-built)
I use the 2 onboard NiC's for WAN traffic and the 2 PCI for LAN.
The Server has 4 VM's built:
1. pfSense
2. ZIMBRA mail server (Linux)
3. AVG server (MS2008R2)
4. Backp Server. (Linux - in-house written)

Now regarding "Promiscious mode" - In your vSphere client consol, click on CONFIGURATION/NETWORKING/PROPERTIES, choose your VMNetwork under PORTS and untick Promiscuous Mode.
Kind regards
Aubrey Kloppers

[dpg2]

Just a side-note: if you use CARP (for HA) in the future with additional pfSense VMs you will need need to re-enable promiscuous mode on the vSwitch (as well as enabling mac address changes, and forged transmits).

This helpful advice is in "the book" (pfSense:The Definitive Guide, Buechler and Pingle, 2009,  p. 405/20.10.5).

[nojstevens]

Thanks everyone for their input. I have it working, although the host crashes every now and then - it appears to be when I have a high load. Originally it was crashing every 5 mins and the CPU on the pfsense guest was showing 100%, so I reset my config to factory defaults (originally I had imported the config from my baremetal pfsense). Once i did this, CPU calmed down to 0-1%, but it still crashes the host from time to time.

Jon

[helix]

I know that his may not be directly related but I just had a high CPU usage problem (discussed on other threads as well)

Im running EXSi 3.5 U2 (due to my CPU having a bug that does not allow higher versions)
HP DLD585 , 4 x AMD Opteron Quads 2.2
2 On board NIC and a Dual Intel Pro1000  = 4 Physical Nics

So here are my 2 cents:
Dedicate a NIC for the Mgmt Network
When running pfSense as a VM, setup a separate resource pool and reserve CPU bandwidth (mem reservations do not impact much if you have sufficient RAM)
In a very aggressive setup, you might want to set CPU affinity to the pfSense VM to cores that are unused from other VMs (means change them all = VMotion problems)
And yea - disable VMotion for pfSense

I had 75-80% interrupt usage. After just setting Shares = High and reserving 1GHz, this dropped down to 20%.

I am running 18 VMs, 2 windowses and 15 openSuse + pfSense
… + a certain apps from my LAN that maintain ~3000 firewall states and it is working great now.

Cheers.
H

[nojstevens]

Thanks Helix - I will try what you suggest. I managed to stop pfsense crashing the host - i'm rock solid now - a BIOS update to my mobo made all my issues go away, but I like what you are suggesting also

Jon