Looking for hardware advice
-
Have you used a c2758 before? I just ordered one and am waiting for delivery. Curious if it runs esxi and the quickassist works?
Also why not virtualize your vm switch? I think it would be much faster that way.
-
That's pretty much what my lab looks like, except I don't have enough NICs in my FreeNAS/Hypervisor hardware so I have a lot of VLANs to simulate the different networks for storage, etc. If you have 3 NICs in the FreeNAS that looks pretty slick.
The reason for a hardware switch between ESX and pfSense is so you can connect physical devices to the same networks as your VMs.
Are you that short on ports in the DGS-1100? It should be able to handle it if it's not full. Should only take one port to pfSense and one each to the C2758s.
Just about anything will get you going. The pfSense node in front of my lab / home network is an atom D525. And APU with an SSD would do it. It's fairly simple to move to new hardware later. Worst case is editing the config and changing the interface names to new ones if the auto-assign can't figure out all your VLANs, etc (likely.) It's an easy edit.
-
Pfsense n00b here so pardon if this doesn't work: can't the vm host use trunking protocol to pfsense and hanle external members that way?
-
Have you used a c2758 before? I just ordered one and am waiting for delivery. Curious if it runs esxi and the quickassist works?
Also why not virtualize your vm switch? I think it would be much faster that way.
Both of my ESXi hosts are actually Supermicro A1SAi-2750F-0 systems. I chose this board due to its low power consumption and the fact that it has 4 Intel NICs that are supported with 5.5 U2. My NAS is a ASrockC2750D4I. You can say that I'm a bit of an Atom freak. While I could virtualize my pfSense router, I would prefer for it to be physical hardware, so the network doesn't go down when I'm rebooting as well as I do not want my virtual machines on the edge of my network. As for your other questions, the C2758's most definitely run ESXi. I cannot state if the quickassist works, as I have not setup much with encryption yet.
As for my virtual machines, I do have a vSwitch I am using for machine-to-machine communications. I was looking at the second switch to completely separate my storage traffic (iSCSI) and my regular traffic (VM, Management, vMotion, Heartbeat.) Eventually I have my eyes set on either 10gbE or running 10Gbps fiber between my hosts and my NAS, however for now it is going gigabit.
That's pretty much what my lab looks like, except I don't have enough NICs in my FreeNAS/Hypervisor hardware so I have a lot of VLANs to simulate the different networks for storage, etc. If you have 3 NICs in the FreeNAS that looks pretty slick.
The reason for a hardware switch between ESX and pfSense is so you can connect physical devices to the same networks as your VMs.
Are you that short on ports in the DGS-1100? It should be able to handle it if it's not full. Should only take one port to pfSense and one each to the C2758s.
Just about anything will get you going. The pfSense node in front of my lab / home network is an atom D525. And APU with an SSD would do it. It's fairly simple to move to new hardware later. Worst case is editing the config and changing the interface names to new ones if the auto-assign can't figure out all your VLANs, etc (likely.) It's an easy edit.
I've currently got two NICs in my FreeNAS box. I have a CIFs share for my Windows/Apple clients, and a NFS share so I can access my programs/images for my VM's. I also am running an iSCSI target for my ESXi hosts, as I have not setup local storage in my ESXi boxes.
I am close to filling up my DGS-1100. I currently have my PC(1), two laptop docks (2), NAS (2+1 IPMI), and 2X C2758's (8+2 IPMI). Assuming I setup a switch to handle my storage traffic, I would have 1 port free, as I still need to connect my DGS-1100 to my router.
My router will be turned into a WAP once I decide on the hardware to run my pfSense router. Derelict is also correct in that some of my VM's will be accessible via my main network, and others will be segmented for pure testing (most likely using pfSense in a VM to setup a separate virtual network.
I've been keeping an eye out on the older D525's. How do you find it's performance? As there a specific SOC setup you would recommend with that?
-
Pfsense n00b here so pardon if this doesn't work: can't the vm host use trunking protocol to pfsense and hanle external members that way?
Yes, but where do you plug other devices in? With a switch you can do this:
pfSense OPT10 on re0_vlan10
pfSense OPT11 on re0_vlan11
pfSense OPT12 on re0_vlan12ESX interface VMNET10 on VLAN 10
ESX interface VMNET11 on VLAN 11
ESX interface VMNET12 on VLAN 12Nowhere to plug in a laptop to get on, say, VLAN 12
If pfSense and the ESXes are plugged into switchports with tagged VLANs 10, 11, and 12, you can make a switchport untagged on VLAN 12 and jump on the VMNET12/OPT12 network with any device, for example.
-
I'm really interested in your setup. pfSense forum might not be where we should discuss it, since what you do with your VMs might be a bit off topic.
In my case I have 7 nics on the router, aside from IPMI. I would have the virtual switch and run a trunk back to the pfSense box, and then have a vlan-specific switch off on another nic.
Most of my server hardware is going to be trunking-aware.
-
Vsxi-13, I sent you a pm asking about your experiences with atom-based virtualization.
I hope you don't mind.
-
Vsxi-13, I sent you a pm asking about your experiences with atom-based virtualization.
I hope you don't mind.
No problem. I had responded on there. What are you using to run pfSense? Just looking for some more opinions for hardware given my scenario/useage.
-
I didn't get your response to the pm.
My intent is to build a painfully bare kvm host from gentoo linux, with everything unnecessary removed. If I can get PCI pass through working on this board I will donate most or all nics to the router vm(s). If that happens the host won't even have drivers for the nics.
-
I don't understand why you would hack something together when both ESXi and XenServer are free.
-
ESXi is not free and the gratis version of ESXi is only for very restricted personal non-commercial use.
-
@mir:
ESXi is not free and the gratis version of ESXi is only for very restricted personal non-commercial use.
I think you are wrong. Show me. The evaluation license for the vSphere suite is limited to non-production for 60-days but I see no such limitation on ESXi (apparently now called vSphere Hypervisor). They have even removed limitations on physical CPUs, cores, and RAM. 8 vCPU per VM limit applies and no features like live migration.
http://www.vmware.com/products/vsphere-hypervisor/gettingstarted.html
-
ESXi is not free, and my use is both commercial and personal. Xen is less active than KVM.
ESXi AFAICT has no QuickAssist support, which is the main reason for my purchase of this board. Linux and KVM support QuickAssist right now. Xen also supports QuickAssist but has less development so IMO is less viable.
Less than a week ago, Gentoo pushed a kernel into the stable branch which supports QuickAssist. Meaning that the upstream sources support it and the kernel is now mainstream on Gentoo. I know this because I've been using Gentoo for awhile and searched the source from the previous version and subsequently on the latest when it came through.
I've built KVM hosts before, using Gentoo and others. I don't really see how this is a hack? Gentoo lets you build everything from scratch, lets you omit features you don't want not only from the kernel but from all software on the system, or from specific packages as you choose. If code (meaning driver, system app, support for some protocol) does not exist, then vulnerabilities of that code can't really be exploited right?
I might choose VMware as an option if performance as a KVM guest is not good, there are some Linux drivers which are not as high performance as their VMware equivalents. But before that happens VMware needs to support QuickAssist because again that's the sole reason for me buying this system in the first place.
-
@mir:
ESXi is not free and the gratis version of ESXi is only for very restricted personal non-commercial use.
I think you are wrong. Show me. The evaluation license for the vSphere suite is limited to non-production for 60-days but I see no such limitation on ESXi (apparently now called vSphere Hypervisor). They have even removed limitations on physical CPUs, cores, and RAM. 8 vCPU per VM limit applies and no features like live migration.
http://www.vmware.com/products/vsphere-hypervisor/gettingstarted.html
If VMware is free, how do I modify the source with the Intel patches to support QuickAssist?
-
Whatever.
-
I think you are wrong. Show me. The evaluation license for the vSphere suite is limited to non-production for 60-days but I see no such limitation on ESXi (apparently now called vSphere Hypervisor). They have even removed limitations on physical CPUs, cores, and RAM. 8 vCPU per VM limit applies and no features like live migration.
When I talk about free I mean free as in free of speech and not as free as in free beer.
Returning to ESXi free license:
No vMotion, no backup, no HA, only single host, and no centralized management. With this offer you might as well choose vmware player or virtualbox.
-
I'm not trying to be an @$$ but if there's no QuickAssist support then there's really no reason for me to bother installing.
I'm inclined toward KVM anyway based on past experience.
I also didn't mean to hijack vsxi-13's thread.
-
So to get this thread back on track, does anyone else have any recommendations for my case?
Thanks!
-
Really sorry my part of this took off in a different direction.
My input to you would be to either get a netgate fw-7551 or a dual core atom board with QuickAssist, but you obviously have more experience with this than I do so I'm sure it's no help at all.
