Overkill or Under Qualified?
-
For your needs, an E3-level Xeon, 4GB of RAM and 100GB of log storage is enough. So what you have selected at this point with the E5 is overkill but will definitely work. I'd suggest virtualising but passing the NIC to pfSense as a PCIe device (or use VF if it's supported).
Server CPU is better than gaming CPU. This is because of the workload differences.
-
@johnkeates:
For your needs, an E3-level Xeon, 4GB of RAM and 100GB of log storage is enough. So what you have selected at this point with the E5 is overkill but will definitely work. I'd suggest virtualising but passing the NIC to pfSense as a PCIe device (or use VF if it's supported).
Server CPU is better than gaming CPU. This is because of the workload differences.
CPU: https://www.newegg.com/Product/Product.aspx?Item=N82E16819117790
MOBO: https://www.newegg.com/Product/Product.aspx?Item=N82E16813183013
RAM: https://www.newegg.com/Product/Product.aspx?Item=9SIA98C5JA9264The core pieces I want to spend around $500 - $600. The Case / PSU are cheap enough that I can get them any time I need too. Would it be best if I did a baremetal install, and left virtualising out of the picture? I'm trying to keep the build small, but powerful. I've provided my Network Diagram, Note that ESXi-03 and 04 aren't finished yet. All of the servers have 10G Connections, I plan on pushing 10G in the cluster of the Storage Servers & ESXi Servers, it'll be limited to just those, and won't hit the network. The SG300-10 is L3 right now.
U want run pfsense on a VM? Is this box experimental/lab not production?
This is going to be a home router. I need it to be beefy, and upgrade-able. I'm trying to decide, which would be better, VM or Baremetal Install. I host quite a few ESXi Servers, and services at my place. A lot of the guys I attend college with, can't afford to purchase extra computers, or rent them for educational reasons. I built a few spare servers, and let them have access to them for the duration of their schooling. I pay a cheap price for my connection, and my power bill is less than $100 / month. I need something beefy that can take a beating on a constant basis, and continuous beating.
-
That'll work fine. I suggest you don't virtualise it since playing with virtual stuff while also running your network on top of it is going to lead to outages. Also, if you need to upgrade later on, it will probably end up being much, much different. In a few years we might get good QaT, DPDK and other fancy stuff, so instead of upgrading the hardware, a software upgrade will get you more performance.
Regarding VPN, most connections are limited to about 60% of WAN speeds, mostly due to the providers not having anything better to offer. I would not recommend running everything behind a remote VPN all the time, those services are basically one big man-in-the-middle attack. Doing it for traffic you don't care about or traffic that you know is encrypted well (not talking about the tunnel here, talking about the application protocol, i.e. HTTPS, IMAPS, S-SMTP, SSH) is fine, but you may not want to use it for normal applications.
-
@johnkeates:
That'll work fine. I suggest you don't virtualise it since playing with virtual stuff while also running your network on top of it is going to lead to outages. Also, if you need to upgrade later on, it will probably end up being much, much different. In a few years we might get good QaT, DPDK and other fancy stuff, so instead of upgrading the hardware, a software upgrade will get you more performance.
Regarding VPN, most connections are limited to about 60% of WAN speeds, mostly due to the providers not having anything better to offer. I would not recommend running everything behind a remote VPN all the time, those services are basically one big man-in-the-middle attack. Doing it for traffic you don't care about or traffic that you know is encrypted well (not talking about the tunnel here, talking about the application protocol, i.e. HTTPS, IMAPS, S-SMTP, SSH) is fine, but you may not want to use it for normal applications.
Well, the Jumpbox is there to reduce the amount of open ports on the network. Anything that has to reach out to the internet will have https, or it will only work by accessing the jumpbox. I need to build a better jumpbox, something people can remote to via VNC or something like that so they can view the web portals for ESXi. I learned my lesson to not leave those open to the world. I paid for a Dedi, and put ESXi on it. So many SSH Attempts, I was permanently locked out of my OS, lol.
As to the build, I'll start picking up gear here shortly. Rent is coming due :( Once I have all the parts, and everything is online, I'll come back and post. Thanks so much john, and Sammy!
-
Instead of using the jumpbox for everything I'd suggest using OpenVPN.
-
@johnkeates:
Instead of using the jumpbox for everything I'd suggest using OpenVPN.
Well, the way I was doing it is; VPN -> Jumpbox. I'm trying to reduce the amount of management I have to do. I'll hit you up with a pm once I've got everything, john. This has gotten off-topic a bit, and the goal of the thread was reached.
-
Based on the feedback from johnkeates. An Alternative build
CASE: https://www.newegg.com/Product/Product.aspx?Item=N82E16811139022
MOBO: Some Supermicro Motherboard
CPU: Intel E5-26xx Processor (6 core / 8 core)
RAM: 64GB DDR4 Memory
NIC: I have Dual / Quad Intel NICs
SSDs: 100GB / 200GB
HDD: 3TB for Logs, which will be uploaded to my Google Drive.If you're Running this as a VM under ESXi (or any hypervisor) a single HDD of any kind will make you sad. even for a handful of lab VMs I would recommend a RAID10. If you have => 5.5 vCenter then you can use the SSD as read cache (configured per VMDK in vCenter) otherwise you would be stuck using for swap (total waste) or as a small datastore is which case I would spend less on the HDD and more on the SSD and get the biggest one you can.
As for running running your home gateway/router as a VM, don't. Especially if your using vlans. Its just a pain in the arse. You get stuck changing you PC IP and switch port all the time to fix little things like needing to reboot your host.
For your home gateway/router, just spend the $$ and build or buy a separate router. Also if you don't mind getting your hands dirty in CentOS and need the best possible speed you could take a look at the new tnsr platform. but that's a whole nother animal. ;D
Edit: added closing quote tag.
-
"As for running running your home gateway/router as a VM, don't. Especially if your using vlans."
Huh?? Running pfsense on a vm with multiple vlan is no different then if single network.. As long as you have a switch that handles vlans, and know how to setup the switching in your VM host its really quite simple and easy to run/manage.
A simple 4095 setting on your vswitch in esxi for example allows you to tag any vlans you want to pfsense - which you can then just setup vlans in pfsense. Or you could use port groups on your vswitch with the tag of the vlan you want to pass to the vmnic you connect to pfsense, etc.
I ran like this for years on multiple vlans on esxi, even once I moved to pfsense on hardware I still run vlans into different VMs and even run a downstream pfsense VM via a transit vlan from the edge sg4860.. My sg300s are in L3 mode, but as of current only using L2.. I just put them in L3 for future lab/testing work, etc. They are more than happy to function as L2 when in L3 mode, etc. You can use both at the same time where some could be routed at your sg300, and other vlans are just L2 and routed via pfsense, etc.
-
"As for running running your home gateway/router as a VM, don't. Especially if your using vlans."
Huh?? Running pfsense on a vm with multiple vlan is no different then if single network.. As long as you have a switch that handles vlans, and know how to setup the switching in your VM host its really quite simple and easy to run/manage.
A simple 4095 setting on your vswitch in esxi for example allows you to tag any vlans you want to pfsense - which you can then just setup vlans in pfsense. Or you could use port groups on your vswitch with the tag of the vlan you want to pass to the vmnic you connect to pfsense, etc.
I ran like this for years on multiple vlans on esxi, even once I moved to pfsense on hardware I still run vlans into different VMs and even run a downstream pfsense VM via a transit vlan from the edge sg4860.. My sg300s are in L3 mode, but as of current only using L2.. I just put them in L3 for future lab/testing work, etc. They are more than happy to function as L2 when in L3 mode, etc. You can use both at the same time where some could be routed at your sg300, and other vlans are just L2 and routed via pfsense, etc.
I guess what I was getting at is the chicken egg situation. vCenter is on vlan 50 pc is on vlan 2 and you need to reboot your only host. well then you have to plug you pc into a vlan 50 port or login to the switch to reconfigure your PC port (if its on your PCs vlan) startup your pfSense VM and go back and put you PC back in its normal vlan.
Yeah You can do it and it works, but IF your using vCenter automatic startup of VMs is unsupported. If its just a stand alone host just make sure you don't have any dependence on routing for your VMs to boot up. One example would be a routed SAN. This is never a good idea but I have seen people do it in the FreeNAS forums. (lots of odd network setups over there)
-
Based on the feedback from johnkeates. An Alternative build
CASE: https://www.newegg.com/Product/Product.aspx?Item=N82E16811139022
MOBO: Some Supermicro Motherboard
CPU: Intel E5-26xx Processor (6 core / 8 core)
RAM: 64GB DDR4 Memory
NIC: I have Dual / Quad Intel NICs
SSDs: 100GB / 200GB
HDD: 3TB for Logs, which will be uploaded to my Google Drive.If you're Running this as a VM under ESXi (or any hypervisor) a single HDD of any kind will make you sad. even for a handful of lab VMs I would recommend a RAID10. If you have => 5.5 vCenter then you can use the SSD as read cache (configured per VMDK in vCenter) otherwise you would be stuck using for swap (total waste) or as a small datastore is which case I would spend less on the HDD and more on the SSD and get the biggest one you can.
As for running running your home gateway/router as a VM, don't. Especially if your using vlans. Its just a pain in the arse. You get stuck changing you PC IP and switch port all the time to fix little things like needing to reboot your host.
For your home gateway/router, just spend the $$ and build or buy a separate router. Also if you don't mind getting your hands dirty in CentOS and need the best possible speed you could take a look at the new tnsr platform. but that's a whole nother animal. ;D
Edit: added closing quote tag.
I have ESXi 6.5+, being friends with VMWare employees has its perks. I've thought about tossing it into a VM, but that's more complicated. I'm going to install it baremetal.
