Which Xeon 2011-3 processor would be better to choose?



  • I'm building a rack mount overpowered pfSense server, and I'm torn between two processors. I wanted a M.2 slot for the SSD, Registered ECC RAM, and 4 DIMM slots on a Micro ATX board, which limited my selection considerably. Both have AES-NI support for pfSense 2.5.

    1U Rack Mount Chassis (Micro ATX and Mini ITX)
    Flex ATX 250W Power Supply
    ASUS X99-M WS - 2x GigE NICs, onboard wireless, (Socket 2011-3)
    Intel i350-T4V2 Quad Port GigE PCIe x4 expansion card (5W)
    6x NF-A4 Noctua fans (3.6W Total)
    32GB Registered ECC DDR4-2133 RAM (4x8GB)
    Samsung SM951 128GB M.2 NVMe SSD (6.5W)

    On the processor selection, I'm torn between two Xeons (required for ECC support, and Ryzen hasn't proven itself capable of ECC support yet):

    Intel Xeon E5-1630 V3, $264
    4 cores @3.7 GHz base, @3.8 GHz boost
    140W TDP, 10MB Cache, 0 QPI Links
    https://ark.intel.com/products/82764/Intel-Xeon-Processor-E5-1630-v3-10M-Cache-3_70-GHz

    Pros: Highest clock speed of any 2011-3
    Cons: high TDP, only 4 cores, more expensive

    Intel Xeon E5-2628L V3, $196
    10 cores @2.0 GHz base, @2.5GHz boost
    75W TDP, 25MB Cache, 2 QPI Links
    https://ark.intel.com/products/81704/Intel-Xeon-Processor-E5-2628L-v3-25M-Cache-2_00-GHz

    Pros: less TDP, 10 cores, cheaper, QPI Links (?)
    Cons: lower clock speed

    Given these options, I'm tempted to take the dip in clock speed for the additional cores and lower TDP - so it will run cooler and quieter. I'm planning on running plugins and doing a decent amount of caching, but still probably not enough to take advantage of 10 cores.

    Should I go with the cheaper, lower clock, more core processor for a quieter, less power hungry server? Would the difference in clock speed be noticeable? Would the i350 NIC be able to utilize a QPI Link?



  • Everything depends on both what you need to do and what your budget is.



  • Sure; in this case the processors are nearly the same cost ($196 vs $264), and the rest of the components are the same. The $68 is fairly irrelevant in the scheme of the whole build, so my ask is which processor would be preferable (lower core count, higher clock, higher TPD, 0 QPI links vs higher core count, lower clock speed, lower TDP, 2 QPI links).

    Are QPI links relevant to the Intel i350 NICs? If so, I think that would tip the scale sufficiently in the favor of the E5-2628L V3. Would the 2.0 GHz vs 3.7 GHz base clock be noticeable between the two?



  • Again, it really depends on what you want to do. Are you going to run single-threaded tasks or multiple tasks/multithreaded tasks. How many links, want type of links, what packages, what services, any crypto etc.



  • That I don't know; this is my first pfSense build and use. I have a single WAN link, 200Mbit/s down and 10Mbit/s up, planning on using Squid for caching, Suricata, Darkstat, and wiring up a GPS receiver to the COM1 port for an accurate NTP server.

    Is there enough multithreaded workloads to even come close to saturating the 4 cores? Are QPI links somehow relevant?

    Obviously, "it depends" can be an answer to most any question, and isn't particularly helpful.



  • For pfSense it really matters. Basic routing and some OpenVPN works well on very fast cores, but multiple tasks (but not OpenVPN) run well on multiple cores. You can't really have both, so you need to make a choice. Same goes for PPPoE WAN vs. Static routed WAN.

    In your case, your CPU is best specified for the IDS/IPS/Logging setup you're building, so more cores and trading in a few 100 Mhz is the best way to go. Even if you are on PPPoE and wanted to use OpenVPN, you'll have no problems with either CPU, but the performance for the other packages will be better with the more cores model.

    There is no 'best' or 'universal' configuration since the load is so very specific. Where 'it depends' is indeed the universal question killer, in this case you couldn't make a choice without knowing the usage pattern.



  • Unless the workload is fairly unusual, I'd definitely take the E5-1630v3 over the E5-2628Lv3. In addition to significantly more (over 1.5x) clock speed, it also takes faster memory. (The E5-2626Lv3 can't even take advantage of the RAM you spec'd.) The QPI links in this context are for connecting CPUs in a multi processor configuration. Unless you're planning to use two of the E5-2626Lv3s in a 20 core configuration it's irrelevant. The TDP difference is because the L (low-power) CPU is capped, which limits its max turbo. If you're not running the CPU all-out, they'll idle at similar levels. The only time you'd be running the E5-1630v3 hotter is if you're maxing out the CPU by running it faster than the E5-2628Lv3 is capable of going.

    Given the complete lack of a traffic profile, I'm willing to guess that you're not going to be pushing the performance envelope monitoring complex network traffic. So, you'll notice single thread performance (race to finish) far more than you'll notice a lack of throughput capacity–so single core performance is more relevant than number of cores.



  • Thanks for the info! That helps on understanding QPI, I saw some literature that made it sound like the QPI would link between the processors and also to the X99 chipset, so it wasn't clear if it was something that could help or not.

    Is there somewhere that lists processor idle wattage?

    I have been deliberating this for a couple days, and I ended up going with the 10 core (purchased just before I read your reply) - it just 'seemed' a bit 'more' (additional cores, lower wattage thus lower fan noise, more cache, tolerant of higher temperature) and I choked on spending more money to get 'less'. What I've seen on the RAM speed with pfSense has claimed that is rather irrelevant.

    Either way it is fairly overkill, and with decent thermal management it should boost single cores to 2.5 GHz. Worst case, I can grab the 1630V3 and swap it out to get that little bit more CPU and RAM speed.



  • @Ehryk:

    Is there somewhere that lists processor idle wattage?

    No, the only figure listed is the maximum that the cooling system ever needs to be able to handle. Idle wattage tends to be dominated as much by the motherboard and other components as the CPU. For anyone else reading along, do not use the TDP as any kind of guide to what processor runs cooler, unless you're trying to decide if you can get by with passive cooling or somesuch. More than one person has bought a U or L series CPU because it's "lower powered", then wished they had a bit more power when it came time for a CPU-intensive job.



  • In this case, none of that matters since it's a maximum of 210Mbit bandwidth combined on WAN. You could do that with a Pentium.



  • Hmm, that sucks that the idle wattage isn't listed. I am using a fanless CPU cooler, and even though there will be 6 40x40x20 fans in the front I was somewhat concerned with 140W TDP into that copper cooler and also perhaps using too much of one of the voltage supplies out of a 250W Flex ATX power supply.

    75W TDP makes me much more comfortable about a fanless CPU cooler and staying well under the maximum current out of any of the voltage rails, and in the end "a bit cheaper with 6 more cores and more total cache" won the day. Either way I should still be plenty overpowered for a long while.

    Even though the max is only 210Mbps from the WAN side, I do want to get closer to the 1Gbps speeds transferring over the LAN with anything that has Gig NICs.

    Thanks for the info though, especially about TDP having no relation to idle power draw and QPI Links.



  • @Ehryk

    Given these options, I'm tempted to take the dip in clock speed for the additional cores and lower TDP - so it will run cooler and quieter. I'm planning on running plugins and doing a decent amount of caching, but still probably not enough to take advantage of 10 cores.

    If this will be not a really urgent build, I personally would go with the new Intel Xeon D-15xxN Chips, that are coming
    with AES-NI and QAT, 4 x 10 GbE and 2 x GB LAN Ports on top of all. Low electric power usage and more power then
    the big brothers from Intel. (My personal meaning)

    Should I go with the cheaper, lower clock, more core processor for a quieter, less power hungry server? Would the difference in clock speed be noticeable? Would the i350 NIC be able to utilize a QPI Link?

    If it is really urgent to build the unit, you may be better of to go with a Supermicro bare bone that will fits your needs.
    Supermicro SYS-E300-8D
    Board X10SDV-TP8F

    Supermicro SYS-E200-8D
    Board X10SDV-6C-TLN4F

    Board X10SDV-16C+-TLN4F

    Intel Xeon D Series with N (network) SOCs



  • Xeon D is now my first choice in router hardware recommendations. I think it even beats i3 processors when compared in terms of having multiple hosts in VM.

    Instead of going with the older Xeons, it’s best to invest in newer technology which supports pretty much all pfsense requirments and added functionalities.


Log in to reply