So I built my own Supermicro setup bare bones kit with the C2758 CPU and motherboard that supported 1600Mhz Ram. So I installed 16GB ram and a 120GB SSD.

Wow.  Holy overkill Batman!

All of my pfSense boxes have only used ~1GB of disk space.

I've also never seen it use more than a few hundred megs of RAM

@nolaquen:

Doesn't that board have a physically dedicated IPMI port? Specifically thinking the A1SRI-2758F-O. I have it, and there's not actually a BIOS option to put IPMI on the other interfaces.

There used to be an option to choose how IPMI determined what interface it would use, and the default would put IPMI on one of the other interfaces if nothing was connected to the dedicated IPMI port. Maybe they've now changed that since too many people were messing it up.

I'm glad I read this thread. I didn't see the option in the BIOS, so I thought the same thing - they just changed the defaults.

Turns out, no, they didn't. I had to configure it in the IPMI system, and it was still set to failover by default. Thanks everyone!

Doesn't that board have a physically dedicated IPMI port? Specifically thinking the A1SRI-2758F-O. I have it, and there's not actually a BIOS option to put IPMI on the other interfaces.

There used to be an option to choose how IPMI determined what interface it would use, and the default would put IPMI on one of the other interfaces if nothing was connected to the dedicated IPMI port. Maybe they've now changed that since too many people were messing it up.

I have a 1GB connection to AT&T biz fiber and am pushing out around 300Mbps - 400Mbps from about 12 noon Est. to about 10pm Est.

This depends on more things mostly. If you have 1 GBit/s at the WAN port and only 300 MBit/s - 400 MBit/s
will be there as the entire throughput this is then less then the halve. What kind of connection method do you
using? Is it PPPoE perhaps? Then only one CPU or SoC core will be in usage then.

Doesn't that board have a physically dedicated IPMI port?

Yes it has one. But this port can also be configured to be the fail back WAN port and in some
cases this could be a security issue! So configuring it to be a real dedicated IPMI "only" Port
would be the best.

Specifically thinking the A1SRI-2758F-O.

Please have a look on the left side the single LAN Port is the IPMI port.
Link

I have it, and there's not actually a BIOS option to put IPMI on the other interfaces.

The IPMI port can be a fail over WAN port or a dedicated IPMI only port so it would be the best to
set it up as IPMI only.

In the BIOS, make sure that the IPMI interface is set to "Dedicated". If it's set to "Shared", then it will become available on the first of the other ports that are detected to be active, if nothing is connected to the IPMI interface. If that happens to be your WAN port, then that's where it will reside. And since it sits at a lower level than pfSense, I don't believe pfSense will be able to block access to it.

Otherwise, everything else should pretty much be able to be left at defaults. I don't have the 2758 board, but I have the 2558. I've been very happy with the performance from my system, so yours should do just fine with a few more cores!

Doesn't that board have a physically dedicated IPMI port? Specifically thinking the A1SRI-2758F-O. I have it, and there's not actually a BIOS option to put IPMI on the other interfaces.

In the BIOS, make sure that the IPMI interface is set to "Dedicated". If it's set to "Shared", then it will become available on the first of the other ports that are detected to be active, if nothing is connected to the IPMI interface. If that happens to be your WAN port, then that's where it will reside. And since it sits at a lower level than pfSense, I don't believe pfSense will be able to block access to it.

Otherwise, everything else should pretty much be able to be left at defaults. I don't have the 2758 board, but I have the 2558. I've been very happy with the performance from my system, so yours should do just fine with a few more cores!

Interesting. I had major connection issues a while ago and had to stop using my igb0 port to get things to work. Sounds like this might have been the reason. Thanks for the tip!

Otherwise, everything else should pretty much be able to be left at defaults. I don't have the 2758 board, but I have the 2558. I've been very happy with the performance from my system, so yours should do just fine with a few more cores!