Sweet and yeah I was thinking it would work like that but never tested it like that, however I will be setting up a test box today and ill see if I get the same issue. Btw thanks for the info and a momentary of hijacking your thread

@johnpoz: I only have 1 public IP ;)  Your issue is being able to manage it and access it via vmkern. VPN?  8) I have some what of the same setup at home but all my servers have dul-nic's & well my vmkern only runs on my LAN side of the network. (www)–--[ESXi-eth/nic0]–-{vm-pf}---(vswitch)---[ESXi-eth/nic1]–-[other network stuff]                                                                   |                                                                 {VM's} –--------------------------- OK so here is something you can do! http://blog.romant.net/technology/configuring-nat-on-esx-and-esxi/ In a nutshell: Create (at least) two vSwitches, one "public", connected to one of the server NICs and one "private", which is not attached to any physical NIC. Pick an RFC1918 subnet to use on the private vSwitch, say 10.0.0.0/24. Install pfSense in a VM, assign its WAN interface to the public vSwitch and its LAN interface to the private vSwitch. Additionally, assign the VMware vKernel management port to the private vSwitch. Set up a VPN in pfSense along with appropriate routing to get to the private network. OpenVPN is quite easy to set up, but IPsec would be fine as well. For any server VMs you have, assign their interface to the private network. Create Virtual IPs in pfSense for the rest of your public IP addresses, then set up port forwards for any services you need people to be able to access from outside the host. At this point, the pfSense VM will be the only way traffic can get from the outside to the rest of your servers and management interfaces. As such, you can specify very specific rules about which traffic is allowed and which is blocked. You will be able to use the vSphere Client after connecting to the VPN you configured in step 4. Source: http://serverfault.com/questions/353223/recommended-way-to-setup-a-secure-esxi-environment-with-a-publicly-accessible-ra NOTE: I don't think you can do this in less you have more than two NIC's on the server, do to I think ESXi has to have a physical NIC for management interface. However if it does not you could make a virtual switch and add management to it and keep management on the physical NIC as well so after you install PF you will have some way to talk to the server!?!!?

well that is what power strip/surge protector is for.  I would hope you have your stuff on a ups that should have multiple plugs.

pfsense sometimes has a hard time with dual HMA vpn, IME> I'm working on the same now. SHOULD be doable with one pfsense… I had it running but now get problems. Can swap between. I have tried even using a second WAN to no avail. I have not tired with multiple IPs same provider, just multiple providers=nics. always fully reall and passed through nics, of course

Did you ever get farther on this? I'm going to try device polling before next reboot and see if that helps me. The tickboxes below… I can't see how those would impact no connectivity post "first" reboot. It's 100% reliable- every SECOND reboot is fine. I am sure all vmware and passed through intel nics support polling fine. I agree 100% you need to passthrough, virt NICs are just not good enough for replacing baremetal intelligently. Even when I only had 200MB in, I could see a huge loss on the ESX nic....Even played with the 3 different driver options you can pass to hack toward pfsense, always lossy. Can't happen with voice and other stuff. The dual reboot hing makes me wonder if its a slice thing- I know the flash installs to two slices...And seem to remember reading they alternate at every reboot. Any comments on that? Next week I will have one wan on gigabit/300 and the other at 200/30. Of course you need a good intel card for those, and to be smart to even see the throughput behind pfsense. I think I ordered a quad ET 82576, my dual ET plus single 82574 CT pass through fine and dandy to the two wans which still are just 300/300 and 200/30. ESX 5.1 is on x9scm-f e3-1230 32GB running lots of PCI passthrough to other stuff too. pfsense 2.2's limiters are bustd. so 2.1.5 is best for me for now. I may get around to trying a 2.2 pfsense and see if reboot works. Last time I tried the upgrade it broke everything, which I later found out was just because 2.2 busted limiters.

If me with such a beast I would put esxi on it and run whatever you want as VM, be it your 2k8 server and pfsense and whatever else you might want.  I have a little n40l that runs 7 vms 24/7/365 without any issues.  1 of which is my router on pfsense.  I have more nics in there - but that is because I wanted to break vmkern on its on interface and have another interface for another physical segment. But sure if you want to use hyper-v or player or virtualbox that works too.

I was stuck with this issue, no replies, so I went through all the settings again.  Lo and behold!  There was no upstream gateway set on the WAN port (although I'm sure it was there at some stage before). [image: gateway%20not%20set.png] After setting it, all is well.

You do not need "vmware" tools to show correct speed and duplex.. The e1000 shows speed an duplex just fine at 1gig.  The native vmx3 does not - it just shows autoselect and duplex not reported correctly either.  This is why I went back to e1000, so that lldp and cdp via ladvd package. I just have the open vmware tools. edit: I would show a screenshot but attachments not working?

Yeah kind of hard to get to esxi via vmkern if vmkern is not connected to your network ;)

If you have a full vSphere deployment than having it virtual as part of vCenter is very convenient.  Otherwise, go with a Standalone PC.

Lower end APC's are notorious for putting out a really chunky square wave. Try stringing APC's like the SmartUPS 1000 in series, by the time you get to the third one, the output 'power' is useless, as it's been mangled so badly. Try running a small electric motor off an APC 1000, you can here it chunking away, hating the wave form. As you say, your upstream inverter is a nice true sine wave.  That's what you want your gear running on. It could be that your onboard NICs are behaving very differently to your PCI nics with respect to bad power.  Different rails on the power supply perhaps. The other distinct possibility is earth potential differences while on UPS.  Some floating earth difference is drifting across some of your ethernet cables, and smashing your packets.  Just a tiny leak or float on 230v is a big deal to 5v ethernet. Shielded ethernet can make the problem worse, better off with UTP unshielded-twisted-pair. Make sure all your gear is earthed properly. I think an oscilloscope is going to tell you a lot more than wireshark.

While yes the vmx3net drivers work.. I personally moved back to e1000 because of issues with cdp and lldp.  When using the vmx3net native that is in freebsd you only see autoselect for speed and causes switches using cdp/lldp to report duplex mismatch, etc. I like using the ladvd package on pfsense and this was causing lots of log entries in switch about duplex..  Just went back to e1000 and no issues - I really did not see any sort of performance dif between them.  But then again I not pumping any sort of serious bandwidth that you might see in a enterprise or large deployment.

@Miscue: Hello All, I'm having a bit of trouble wrapping my head around how to change my architectures (introducing pfSense) with maintaining the same functionality.  Any help would be greatly appreciated.  My current setup, illustrated with setup1 picture below. DSL Modem in Bridge Mode Asus Wifi/Router terminates that connection and acts as the NAT Asus Router is connected to VLAN1 of the SG300 which is in trunk mode (this gives all my vlans internet access) Servers and ESXi in VLAN 20 NAS in VLAN30 What I want to do is get rid of using the ASUS device for routing and replace that with pfSense.  Here's where I'm running into issues (thinking about it).  pFsense will be on ESXi01 which also houses all of the VLAN20 virtual machines.  ESXi01 has 6 physical NICS that I can leverage.  The question I have is, do I connect the modem directly the ESXi box (would be considered WAN port) and then have another NIC associated with pfSense connected back to the switch (VLAN 1) that was previously used with the ASUS router?  Picture desiredsetup is what it looks like. Cheers, Brad Depends on a couple of things.  Number of hosts you are using on the SG300 for one.  SG300 is a layer 3 switch (but limited TCAM space, so you can only do hardware routing of ~500 entries, (may have to check for the -10 model, it might be less).  I usually configure switches as layer 3 and put a separate VLAN between the firewall and switch on the inside, and add routes to the other subnets on the pfsense box.  I normally also put the Internet hand-off in another VLAN that doesn't have an IP address (strictly layer 2 VLAN, not an SVI), so I can collect statistics on the switchport.  That may not work in your instance, I believe PPPoE is layer 2 at some level, and you may need to connect your bridge-mode DSL modem directly to your ESXi hosts NIC.  Again, I do it with VLANs in my scenarios, but you could create a separate vSwitch, assign one of your physical NICs to it, connect your DSL modem to the interface, and then add a virtual NIC to your pfsense VM attached to that vSwitch.  If you use your existing proposal, and keep the SG300 in layer 2 mode, you will need to create VLAN interfaces on the "inside" interface of your pfsense VM to match your existing trunk configuration on your switch, and it needs to have the IP address that the ASUS router has in each VLAN currently to make the transition seamless (no changes needed to existing devices).  If you are within TCAM budget of your switch, I would place it in layer 3 mode, and assign each VLAN the IP address currently on the ASUS, and then the extra VLAN/Subnet between the switch and firewall, default route on the switch pointing to the firewall internal IP address.  Much cleaner design unless you need actual firewall functionality between VLANs.  Please also note, switching an SG300 between layer 2 and layer 3 mode causes an instant reboot and total reset to factory of the device (TCAM re-programming). Regards, dtb

bhunter, As johnpoz said a virtualized firewall is "NOT REALLY" more than an OS to a VM !! The four parts of Calvin document/blog are more than enough for what you need to setup a basic installation. Don't be intimidated… If you know how to install an OS in a VM, You will be able to install and setup pfSense too  ;)

well you could start by eliminating possible issue's: -esxi with only pfsense basic install –- once you get that working correctly start adding other tools 1-by-1 to complicate your setup.

My routers gateway address is 192.168.1.254 should I be entering that for the upstream gateway IP in pfsense? Yes, 192.168.1.254 should be your pfSense WAN gateway.  In my example, I used 192.168.1.1 but I see how that would be confusing considering you already told me it was at .254 in your first post. How do I obtain the correct static WAN IP address settings in order for this to work please? If you're managing this network then that's the kind of thing you should already know ;) If you aren't the official network guy then you should ask him or he may get upset if you cause disruption. That said, the first thing would be to check out your gateway's (.254) DHCP pool setting to make sure you aren't grabbing an IP address from the DHCP pool.  Then ping the address and see if you get a response – if not, use that IP address.  Not perfect but a start.  Is this a house or college dorm?  Do you know if any of the clients are static IP or not?  You can grab an address that seems to be unused but someone could turn on a device later on with the same static IP address.  Unlikely, but it happens.

Brilliant! It worked like a charm. For the record, here's what I did, based on your advice: connect a physical bridge to the physical ethernet port of the host connect the wireless AP to one of the LAN ports of the physical bridge reinstall pfSense from scratch assign the physical bridge to the LAN of pfSense (bridged mode in VirtualBox) set pfSense LAN to 10.0.0.1/24 and activate DHCP ensure Windows is using the physical port with the switch to connect to the Internet use bridged connections to the physical ports for each vm leave IP configuration of all devices to 'automatic' Thanks for help, your solution is very straightforward and saved me a lot of painful configurations :-)

I'm seeing the same type of behaviour. When the gateway is the CARP Vip my throughput out of WAN is ~3mbps max as soon as I switch to the real router LAN interface I have connection speeds of 50mbps (which is normal). No raise in CPU or memory usage either. ESXi 6.0 4gb Ram 5 CPUs

Thank you, I will try that. I set it up with time synch off after reading this post: https://forum.pfsense.org/index.php?topic=94559.0