UPDATE:

Still loving the install - does everything I need, and has solved several network issues I was having with my ASUS rt-ac68. 5-stars to pfSense!

Only niggly problem is that the i350-T4 under Virtualbox and kvm uses 30-40% CPU when downloading at ~20Mbit/sec. The problem with kvm is that on my hardware (p7p55d-e-pro M/B and i5-750) I can't do vt-d passthrough of the PCI-e slot, hence the i350 NIC has to run using emulation (have tried with both the e1000 and virtio driver under pfSense 2.2.4, and disabled hardware checksum offloading).

I contemplated upgrading to vt-d capable hardware (not that easy with consumer motherboards - grrrr ASUS/MSI/Gigabyte!), but an additional hassle was the odd occasion when I want to boot into windows (from Arch linux) and have to refiddle to get the virtualised pfsense router rebooted and running, which results in internet and LAN downtime (hence lower WAF). I also don't want to be bothered to move my multiple OS installs to a hypervisor environment, as they're currently multibooting from separate hard drives.

So I have now purchased an ex-lease PC for use as a standalone pfSense box. I've gone with an HP Compaq 6300 Pro (i3-3220, 4GB, 500GB) which should be relatively low power consumption and have more than enough grunt for my current and future needs which at present are: 100/20 Mbit WAN, half a dozen LAN clients, and openVPN server for me as single-client road-warrior. Most importantly, the HP box has a PCIe slot for the i350-T4 - did not want that $60 to be wasted!

Dude I have no idea what you should or shouldn't do.. I have no clue why you think you need to put public IP your proxmox interface..  Why as it not setup before?  Why would creating a vlan disturb anything?

Your statement that you "hate" vlans tells me your in the wrong field of work or play…

Just giving you your options.. But the IP on the device behind pfsense directly, use a port forward or do a 1:1 - why you think you actually need a public on your server behind pfsense I have no idea.  But if the segments are routed to you it takes 2 seconds to set that up.

If you were going to use pfsense to route your traffic to firewall your VMs then package is clickity clickity..  But to be honest if you want to really run snort, etc.  And have full control and power and feature set, etc.. Better to run it on your own VM not the package integrated into pfsense.

Its 2.2.4 with VMWaretools. VMWare - Workstation

In Pfsense 8-9%
Hosttaskmanager 30%

PFsense224highcpu.jpg
PFsense224highcpu.jpg_thumb

@VFrontDe:

To work around the issue I modified the driver to ignore invalid NVM checksums.

you make it sound so easy… also, the stuff on v-front.de is amazing. thank you for all your work.

Thanks for the input Keljian.

Another helpful user recommended I try enabling MAC address spoofing in the NICs within Hyper-V… as soon as I did that I was immediately able to connect to the admin GUI on those NICs.

@KOM:

Also, I think I remember recently people were having trouble with the VMX NICs under load, and that problem wasn't present with the E1000's.  It is also debatable as to whether or not to install VMware Tools.  FreeBSD base already has the NICs, so it's really only required if you must have the server heartbeat, and even then it is a trick to get them properly installed.  Search this forum for other posts about VMware Tools/Open-VM-Tools.

Not sure if this makes a difference, but I've noticed the latest openvm tools made a bit of a difference re latency

Thanks for your reply,
I managed to get xentools installed so thanks for that.

I however need to change a few things, if pfsense is off I can't access xenserver as it's IP is 192.168.2.2 and my computer trying to access it is 192.168.10.3.
It can't access it if pfsense is down.
If I give Xenserver a static IP in the 192.168.10.0/24 range I can then see it ok but then my Server 2012 running my exchange can't talk to the rest of the network as it's IP is 192.168.2.3

I could probably change everything to the 192.168.10.0/24 range I guess if that will work better.

Regards
Jamie

128mb is not much for an internet café….

I run 2GB for my homenetwork.

your WAN use ppoe or dhcp?

i use Proxmox v3.4 and Pfsense 2.2.3
for NIC i use Virtio  and in pfsense setting
under System –>Advanced -->Networking i tick Disable hardware checksum offload
and it work fine for me.
I don't have slow wan anymore.

Capture1.PNG
Capture1.PNG_thumb

Ok I stand corrected - and appreciate the dialogue!

Sweet and yeah I was thinking it would work like that but never tested it like that, however I will be setting up a test box today and ill see if I get the same issue.

Btw thanks for the info and a momentary of hijacking your thread

@johnpoz:

I only have 1 public IP ;)  Your issue is being able to manage it and access it via vmkern.

VPN?  8) I have some what of the same setup at home but all my servers have dul-nic's & well my vmkern only runs on my LAN side of the network.

(www)–--[ESXi-eth/nic0]–-{vm-pf}---(vswitch)---[ESXi-eth/nic1]–-[other network stuff]
                                                                  |
                                                                {VM's}

–---------------------------

OK so here is something you can do!
http://blog.romant.net/technology/configuring-nat-on-esx-and-esxi/

In a nutshell:

Create (at least) two vSwitches, one "public", connected to one of the server NICs and one "private", which is not attached to any physical NIC.

Pick an RFC1918 subnet to use on the private vSwitch, say 10.0.0.0/24.

Install pfSense in a VM, assign its WAN interface to the public vSwitch and its LAN interface to the private vSwitch. Additionally, assign the VMware vKernel management port to the private vSwitch.

Set up a VPN in pfSense along with appropriate routing to get to the private network. OpenVPN is quite easy to set up, but IPsec would be fine as well.
For any server VMs you have, assign their interface to the private network.
Create Virtual IPs in pfSense for the rest of your public IP addresses, then set up port forwards for any services you need people to be able to access from outside the host.

At this point, the pfSense VM will be the only way traffic can get from the outside to the rest of your servers and management interfaces. As such, you can specify very specific rules about which traffic is allowed and which is blocked. You will be able to use the vSphere Client after connecting to the VPN you configured in step 4.

Source: http://serverfault.com/questions/353223/recommended-way-to-setup-a-secure-esxi-environment-with-a-publicly-accessible-ra

NOTE:
I don't think you can do this in less you have more than two NIC's on the server, do to I think ESXi has to have a physical NIC for management interface. However if it does not you could make a virtual switch and add management to it and keep management on the physical NIC as well so after you install PF you will have some way to talk to the server!?!!?

well that is what power strip/surge protector is for.  I would hope you have your stuff on a ups that should have multiple plugs.

pfsense sometimes has a hard time with dual HMA vpn, IME>

I'm working on the same now.

SHOULD be doable with one pfsense… I had it running but now get problems. Can swap between.

I have tried even using a second WAN to no avail.

I have not tired with multiple IPs same provider, just multiple providers=nics.

always fully reall and passed through nics, of course

Did you ever get farther on this?

I'm going to try device polling before next reboot and see if that helps me. The tickboxes below… I can't see how those would impact no connectivity post "first" reboot. It's 100% reliable- every SECOND reboot is fine. I am sure all vmware and passed through intel nics support polling fine.

I agree 100% you need to passthrough, virt NICs are just not good enough for replacing baremetal intelligently. Even when I only had 200MB in, I could see a huge loss on the ESX nic....Even played with the 3 different driver options you can pass to hack toward pfsense, always lossy. Can't happen with voice and other stuff.

The dual reboot hing makes me wonder if its a slice thing- I know the flash installs to two slices...And seem to remember reading they alternate at every reboot. Any comments on that?

Next week I will have one wan on gigabit/300 and the other at 200/30. Of course you need a good intel card for those, and to be smart to even see the throughput behind pfsense.

I think I ordered a quad ET 82576, my dual ET plus single 82574 CT pass through fine and dandy to the two wans which still are just 300/300 and 200/30.

ESX 5.1 is on x9scm-f e3-1230 32GB running lots of PCI passthrough to other stuff too.

pfsense 2.2's limiters are bustd. so 2.1.5 is best for me for now.

I may get around to trying a 2.2 pfsense and see if reboot works. Last time I tried the upgrade it broke everything, which I later found out was just because 2.2 busted limiters.

If me with such a beast I would put esxi on it and run whatever you want as VM, be it your 2k8 server and pfsense and whatever else you might want.  I have a little n40l that runs 7 vms 24/7/365 without any issues.  1 of which is my router on pfsense.  I have more nics in there - but that is because I wanted to break vmkern on its on interface and have another interface for another physical segment.

But sure if you want to use hyper-v or player or virtualbox that works too.