ESXI - pfsense and FreeNAS

[messerchmidt]

supoermicro atom c2758 with 16-32-64gb ddr3L ecc (depending on the freenas pool size)

[KOM]

or did these tools scare you away ?

KOM          Posts: 2595  Karma: +277/-10
johnpoz      Posts: 5473  Karma: +232/-40
Derelict    Posts: 3523  Karma: +390/-12
attilahooper Posts: 1    Karma: +0/-0

Hmmm…..  what was that about tools?  Get back to us when you've managed to actually help someone, ok?

[doktornotor]

@KOM:

attilahooper Posts: 1    Karma: +0/-0

That karma is already outdated…  ;D ;D ;D

[johnpoz]

Pissing match?  It was a side discussion about the use of parity in home setting.  He has his views I have mine - sure and the F was not a pissing match.

[Keljian]

Getting back on track: I run mdadm raid 6 at home using Ubuntu and pfsense on the same box using ESXi 6.

I have no drops in performance, and have sorted out pass through (vt-d/iommu) of a hard drive controller (m1015 in IT mode) to Ubuntu an one of the nics from an i350-T4 to the pfsense VM for the wan.

My hardware is as follows:
Asrock B85m Pro4 motherboard
14 gig of ram (using 7; 4 for Linux and caching, 2 for pfsense, extra is for work)
i5-4570t which I picked up from eBay cheap
M1015 hard drive controller (Lsi 9211-8t)
i350-t4 nic

Load is very low (1-10% usually). Measured load at idle with 7 disks spinning, 4 fans and an average power supply is 85w (45w with the disks spun down)

So yes it can be done

Key things to observe are:
1. Use a separate interface for the management network if possible
2. Passthrough your wan port directly to the pfsense if possible to prevent the hyper visor touching it for security reasons (see later posts in this thread)
3. Use the virtio package in pfsense and the core-VM package in Linux (I believe the same exists for bsd, hence the virtio package).

[frasse]

@Keljian:

2. Passthrough your wan port directly to the pfsense if possible to prevent the hyper visor touching it for security reasons

Is this really recommended?

[Keljian]

Don't know if it's recommended to others, but it makes perfect sense to me.

Means any potential security issues with ESXi vswitches won't affect the other stuff on the box.

[johnpoz]

Security issues with vswitches?  On your wan?

No I have never seen that recommended anywhere.  There is no issue with using a vswitch to connect your wan to pfsense.  Expect for those with really really tight tinfoil hats maybe ;)

[Keljian]

straightens tinfoil hat

Still if you can do it, why not? Are you intending to use that physical port for any other purpose at the same time?

If you have pass through available, I don't see any logical reason not to, aside from a fringe case where pfsense doesn't have drivers for your nic where your hypervisor does.

[johnpoz]

Why not because it makes the setup more complicated - so why do it..  It doesn't buy you anything other than more complication if you ask me.. It sure not buying you any added "security"

KISS

[Keljian]

It is less trouble setting up direct path than it is a new vswitch, but hey, just go with what works for you

[johnpoz]

that is your opinion.  Go with what works for you - but if you ask my opinion having vswitches tied to your specific nics with them labeled is much easier to manage then worried about what what specific nic is passed through to a specific vm.

Also very difficult to passthru a multiport nic and use one port as passthru and another port tied to a vswitch for use with other vms or lan side of your router vm.

When phyical tied to a vswitch I can bring up different copies of pfsense or other router distros tied to the vswitch and switch between them pretty much just turning off one vm and turning on different vm.  Very easy to leverage port 1 for vswitch X and port 2 for vswitch Y, etc. etc.  Especially if I use the same mac on my router vm wan vnics.. My public IP doesn't even change that way.. I can bring up different version of pfsense or untangle or ipcop or any of the other router/firewall distros in a matter of couple of minutes.  Shutdown vm 1, turn on vm 2 and now my network is using different firewall/router distro for testing, etc.

When tied to a vswitch I can connect any vm I want to the "wan" for say sniffing the traffic seen on the wan.. So I can use stuff like ntop or ids to monitor that traffic without having to run it on my pfsense vm.

To me your making it more complicated and reduced functionality for some perceived added security.

physical passthru also makes it difficult to use any sort of vmotion.  Which I am currently not using in my home setup, but clearly passthru reduces the feature set of a Visualization setup.

[KOM]

physical passthru also makes it difficult to use any sort of vmotion.

I was just going to mention this part.  No real benefit for passthrough but a major drawback.

[Keljian]

Ok I stand corrected - and appreciate the dialogue!