My VMware ESXi + SuperMicro A1SAi + pfSense configuration

hedberg

Around 10 months ago I began to look for a more secure solution for my home and a couple of apartments I rent out. I wanted to seperate the networks for each apartment and also have a DMZ etc. I had previously had experiences with pfSense, so I began to look for a motherboard with low-power usage, not to expensive, but powerfull enough to run a couple of virtual machines including a virtual pfSense installation and the 250Mbit connection to the Internet.

My choice fell on the SuperMicro A1SAi motherboard. with an Intel Atom processor C2750 8-Core processor. It has 4 GbE Intel LAN ports and IPMI supper on a dedicated NIC. I installed 32GB of RAM and re-used an older SATA drive on it (the real bottleneck of the system).

VMware ESXi supported the motherboard I was installed without any problems as long as I didn't use the USB3 ports. There were some issues with pfSense 2.1 directly on the hardware, but it worked without any problems inside a virtual machine. I have since upgraded to 2.1.5 and 2.2 RC (I have also tried to boot the 2.2 RC live-installer CD directly on the motherboard with no apparant errors).

The NIC configuration:

One NIC is connected directly to the 250Mbit Internet connection.
One NIC is connected directly to my NAS using the iSCSI protocol. Here I can store multiple virtual images and ISO images.
One NIC (with all client traffic) is connected to my VLAN enabled TRENDnet TPE-1020WS switch
One NIC is unused for now
Dedicated NIC for remote control using IPMI

Internally in VMware there is multiple virtual switches and VLANs configured (8 so far), so all traffic is separated and e.g. the apartments can't see each other or my networks.

This is a wonderfull setup. I like to have the ability to test various setups or operating systems and pfSense handles all the security and traffic load wonderfully on this motherboard and it uses very little electricity while still performing quite nice (can easily handle the Internet connection) and several virtual machines at the same time. I can highly recommend it.

Cino

Nice! I've been using a SuperMicro X7SPA-HF probably for the last 4 years now… Its time for an upgrade and I want to go the ESXi route. I was able to install it on the X7SPA-HF but performance wasn't there to run ESXi with this board (4 years ago). Power consumption has always been a factor for me since I do have a few devices that are always on.

Couple of questions:
What is the wattage when idle and under load? Kill-A-Watt is good for that
What version of ESXi are you running?
Pictures?

hedberg

@Cino:

Nice! I've been using a SuperMicro X7SPA-HF probably for the last 4 years now… Its time for an upgrade and I want to go the ESXi route. I was able to install it on the X7SPA-HF but performance wasn't there to run ESXi with this board (4 years ago). Power consumption has always been a factor for me since I do have a few devices that are always on.

Couple of questions:
What is the wattage when idle and under load? Kill-A-Watt is good for that
What version of ESXi are you running?
Pictures?

I'll measure it tomorrow.

ESXi 5.5.0 2068190

What do you want pictures of?

Cino

@hedberg:

What do you want pictures of?

Of the build, the box itself

hedberg

@Cino:

@hedberg:

What do you want pictures of?

Of the build, the box itself

I'll take a picture, but it ain't pretty :-) It is placed in the attic (Thats why the IPMI remote control is importent for me).

Cino

gotcha… How is it with the heat?

hedberg

Pictures of the cabinet and the setup… I know.. It aint pretty..

rsz_img_1470.jpg_thumb

rsz_img_1468.jpg_thumb

Nice!

What can you say about noise levels?
A four bay Synology NAS is very quiet, I own a DS415Play since a month, and was wondering if it's comparable.
I read the model Supermicro you got is pretty quiet too?

hedberg

No fans on the motherboard, so it just depends on what cabinet you choose. I have this one from Supermicro called SC504-203B. It comes with a low noice power supply (it is more quiet than my NAS).

I would actually have purchased another cabinet where the NIC's were facing front, but it wasnt available here in Denmark at the time of purchase.

Tempeture on the CPU right now is 25 degrees celcius (77 fahrenheit) and the memory modules are between 19 and 24), but it is quite cold up there is the attic right now (about 5 degrees/41 fahrenheit). The tempeture up there can get higher in the summertime, but it ran all summer without problems.

Cino

awesome! The SM mb usually can handle heat but know me, i would be worried having it in an attic. Where you able to grab wattage numbers? I think this board will be my next upgrade for my box. Thanks again for sharing!

I noticed the lack of a fan near the CPU. Been looking at the SYS-5018A-FTN4 myself.
It does have a extra fan installed near the CPU. Not sure why as yours does not have it. But it's PWM so should be alright.

hedberg

@Cino:

awesome! The SM mb usually can handle heat but know me, i would be worried having it in an attic. Where you able to grab wattage numbers? I think this board will be my next upgrade for my box. Thanks again for sharing!

I will get those numbers for you.. I just forgot it was attached to a UPS the last time I was up there, so I didnt have the right cables with me.

I monitor it during the hot summer days. and it wasnt even close to max out - but notice that this is the cold north… It is a nice summer if we have 2 days of sun :-)

hedberg

@weust:

I noticed the lack of a fan near the CPU. Been looking at the SYS-5018A-FTN4 myself.
It does have a extra fan installed near the CPU. Not sure why as yours does not have it. But it's PWM so should be alright.

Nice combo of board and cabinet. It was that cabinet I could not get here locally and that I would have preferred. The board seems to be more or less the same as mine, but with a slightly slower CPU (2.4 vs 2.6)

hedberg

I just did some simple measuring on the power usage and it says that the system uses 32 watts under heavy use (in this case 8 virtual machines was being restarted in ESXi) while I was doing a network test filling the 250mbit connection.

When in idle mode with pfSense and a few virtual machines doing nothing it uses 25-26 watts.

Remember that this is a rather old harddrive I have running in the machine and a SDD would probably give much more performance and less power usage.

Br,
Thomas.

Looks nice.

Cino

Thank you for sharing the wattage info.. I'm pretty much sold on this board, just need to get funds..

vsxi-13

@Cino:

Thank you for sharing the wattage info.. I'm pretty much sold on this board, just need to get funds..

Keep an eye out on Newegg for open box. I have now ordered three A1SAi-C2758F open box from newegg. I know another forum user just picked up another one open box. They usually go for around 260-280 open box.

I haven't had any problems with my first two, which are currently acting as ESXi hosts. I haven't tried the third yet, as I need to transplant that host out of my M350 case and into it's respective Supermicro 1U chassis (it's being delivered Wednesday.) I'll post back with the results of the third (which is going to be my dedicated pfSense host.)

Edit: Clarified some poor grammar.