Intel Core processors vs. Xeon. Is it as simple as the numbers?



  • I'm looking to build what will probably be a monster pfSense box (2 actually) for 2 sides of my work to home network.  I have FIOS Gigabit on both sides and my Check Point 680 appliances are only giving me access to about half of the full speed.  Additionally the VPN speeds are pretty bad too and I notice that these boxes hit 100% when doing anything with the internet.  The new speeds have just overwhelmed these boxes.

    So I started testing with different cpu's and have determined that I'll need beefy cpu'age to do the job.

    My main question is this.  Is there any advantage to Xeon over i5 or i7 for pfsense? (understood that some i5's lack hyperthreading)

    So I was able to do a lab test between an i5 and i7 connected via gig switch.  I got pretty close to line speed over VPN with AES-NI turned on and the correct encryption settings used.  That's my main focus, VPN throughput.  Of course I also expect WAN (Internet speeds) to not be limited by this hardware either, but it would seem that the encryption overhead of ipsec/vpn becomes the biggest resource drain.

    Also understand (and this is important).  This is a few devices/users on either side and not a company connecting hundreds of users to the internet.  The vpn is basically dormant except for my cross location RDP or small file copies.  Where it is intended to perform is to move large data backups during the night.  Multi GB (100+) from work to home.  Home is the cloud for work.

    So i'm trying to use the least costly cpu's as possible and I keep going back to cpu benchmarking to help make my choices.  whenever I benchmark a xeon next to an i7 (example: E5645 vs i7-3770, neither one anywhere near latest and greatest) the benchmarks always come back and say the i7 is better in single core and multicore tests.  Is it safe for me to design based purely on cpu benchmark or does Xeon give something here that I'm not seeing in the benchmarks.

    I notice that all of the Checkpoint equipment I've looked at: with the exception of one device using an i5 everything else was using xeon's.  Understand these are 3,5,8 thousand dollar boxes.

    Here's my data:  https://lwf.fink.sh/2016/07/26/check-point-appliance-hardware-lachmann-list-update-july-26th-2016-2/

    I bought on eBay a dell optiplex 7010 i7-3770 sff computer for $178.00 to build my first box.  It has the same processor that I used in my lab testing that when paired with an i5 gave me full line speed vpn.  I can't see putting anything together much cheaper than 180 bucks?  My next level up with 700+ dollars per side.  I know it's not terribly power efficient but it's reasonably small and I'm pairing it with an intel I350T4V2BLK 4 port nic.  I'll post back with results.

    I'm all ears about the xeon vs i7 discussion and would like to hear what everyone has to say.

    Roveer

    –-update---

    I just did a little xeon vs. i7 reading and the three big factors are cores/threads, memory bandwidth and processor cache.  Of course depending on cpu model there are other differences but when comparing to closely related cpu's

    In my example above E5645 vs. i7-3770 I've come to see the following differences:

    1. Xeon 6 cores vs. 4 cores on the i7
    2. Xeon 12 threads vs. 8 threads on the i7
    3. Xeon Memory Bandwidth 32 GB/s vs. 25.6 GB/s on the i7
    4. Xeon Processor Cache 12mb vs. 8mb on the i7

    So the numbers are quite a bit different. Yet the benchmarks show the i7 outperforming the Xeon.  Of couse that's a benchmark.  As a server with those additional cores, threads, bandwidth and cache in a server workload environment the Xeon is going to outperform.

    But the big question is:  for pfSense does a xeon help in any way?  I'm thinking probably not to the point that I'd have to hunt down and specifically build a xeon box probably paying a lot more in order to do it.  I understand if I were building a EXSi box or something that I'd want all that extra power, but beyond my packet processing and ipsec/vpn needs would a xeon do me any good over i7?



  • Don't forget AES speed: http://cpuboss.com/cpus/Intel-Xeon-E5645-vs-Intel-Core-i7-3770

    Xeon E5645 ~1,340,000 MB/s
    vs
    Core i7 3770 ~2,660,000 MB/s



  • @ecfx:

    Don't forget AES speed: http://cpuboss.com/cpus/Intel-Xeon-E5645-vs-Intel-Core-i7-3770

    Xeon E5645 ~1,340,000 MB/s
    vs
    Core i7 3770 ~2,660,000 MB/s

    Well that's pretty huge for my pfSense use.  I'm guessing (please confirm or deny my guess), that the vendors using Xeon's in their larger gear are doing so for the additional cores/threads, mem bandwidth & cache because typically this gear has to move packets for a much larger number of users (hundreds) so additional "bandwidth" is useful.  In my application the users are low but the ability to move a single data stream over a ipsec/vpn would seem to be best served (considering cost) by an i7

    My initial intentions are to build 2 i7-3770 machines (used dell optiplex 7010's) and test.  My previous test with an i7-3770 and an i5 with AES-NI enabled gave me gig line speeds over vpn in a lab environment.  If it's less when connected to FIOS there's not much I can really do about that, but I know that the horsepower is there.  I'll share the results of my tests once I have everything set up and running.

    Roveer



  • OVPN AES implementation it is single core and you will benefit more from I7 or cheaper I5 than E5645 as a SOHO.
    I also plan to change my dual core E8500 firewalls that are ok now with ~100MB VPN with M92p I5-3470.



  • I'm tempted to build a new firewall with a $180 2.8ghz 6core 9MiB L3 CoffeeLake i5 with a 4Ghz turbo that pretty much runs at 4Ghz all the time. My 3.2ghz i5 Haswell can trivially(~10% cpu) handle 1Gb/s bidirectionally(2Gb/s total (4Gb/s system throughput)) with NAT and shaping, assuming normal sized packets. It's even capable of 1Gb/s line rate 64byte UDP @ ~17% CPU, still with NAT and shaping.



  • @roveer:

    Well that's pretty huge for my pfSense use.  I'm guessing (please confirm or deny my guess), that the vendors using Xeon's in their larger gear are doing so for the additional cores/threads, mem bandwidth & cache because typically this gear has to move packets for a much larger number of users (hundreds) so additional "bandwidth" is useful.

    Just saying "xeon" means pretty close to zero in terms of defining performance. (I'm old enough to remember an intel xeon that came in a cartridge that got plugged into the motherboard…it's nothing but branding that's been slapped on a lot of very different CPUs.) If you were looking at something like a v4 E3 xeon clocked at 3+ GHz instead of a westemere era xeon clocked at 2.4GHz, the performance profile would be a lot different. If you're looking at a dual-socket chip (which you are) instead of a single-socket chip, the performance is different and the cost is different (because you're paying for the parts needed for the cpus to communicate). Vendor CPU selection depends on what they're trying to do, what their environmental envelope is, what kind of contract they can get from intel for long term supply, etc. There's essentially zero relevance for what someone should buy if they're purchasing just one.

    It's also worth noting here that if you're using IPsec there's close to no information available on this forum in regard to performance and system sizing. ("VPN" is essentially synonymous with "OpenVPN".) The performance characteristics are different for IPsec, but not many people are testing & posting IPsec numbers.



  • My main question is this.  Is there any advantage to Xeon over i5 or i7 for pfsense?

    What advantage you will be expecting from a 4 or 6 Core i7 with 4,5GHZ turbo speed? All you need
    all you love or all you wish? It doesn´t matter, if you thing you will need it, just do it and buy it!

    That's my main focus, VPN throughput.

    Single threaded for OpenVPN and AES-NI integration for IPSec, and you will be also fine
    with you Intel Core i7 again, if electric power is cheap to get for you, do it!

    OVPN AES implementation it is single core and you will benefit more from I7 or cheaper I5 than E5645 as a SOHO.

    Each of us must see that he is sorted right, it must matching his budget for sure and he should count some
    space ahead on top of this for upcoming things. With an Intel Core i7 you will be long time sorted for anything!

    I got over eBay this parts, for firewall, Snort, Squid & SquidGuard, ClamAV, SARG and pfBlockerNG
    My Internet connections are 2 x 100 MBit/s and I get over one line real IPSec VPN line speed
    passing through all the installed packets it will deliver me line speed too! Nothing goes down!!

    • Intel Xeon E3-1230v2 ~99 € (4C/8T, TurboBoost, Intel Speed Step, HT)
    • 2 x 8 GB DRR3-1600 ~100 €
    • Laptop PSU 92 Watt ~79 €
    • Intel LAG 1555 cpu cooler
    • Intel DQ77KB ~60 €
    • Intel i340-T4 ~50 €
    • mSATA ~50 €
    • Case for 60 €

    So you will see I have not the same internet line speed at the WAN and also not the need of yours!
    If you have 2 x 1 GBit/s at the WAN and want to max. them out, you will need for sure other equipment
    do realize it! And maxing out the VPN connection is more the getting pure 2 x 1 GBit/s at the WAN port!

    Electric power is here in Germany higher in price as elsewhere and also used or refurbished parts are more
    seldom then in the USA, mostly double the price then here! So if not so, I am pretty sure an Intel Core i7
    embedded will be nice to have also here. But due to that circumstances here are the older and refurbished
    Intel Xeon E3 more in usage.

    One of us is loving the consumer line and the other swears on the server line, nothing wrong with it in my
    eyes. The other most thing is also the budget and the availability in my eyes.

    For sure there will be nice and newer hardware on the market available such like the Xeon D-1553N or
    Intel Atom C3858 sorted boards having all nice and new things on board, but why changing when you
    are sorted well and right? For sure if I have the money I would have one of each board.

    It's also worth noting here that if you're using IPsec there's close to no information available on this forum in regard to performance and system sizing. ("VPN" is essentially synonymous with "OpenVPN".)

    Why talking about if all is running? Let us imagine the OpenVPN users will get the highest speed
    delivered by the smallest platform, what you will think about, how many poeples will speak or ask
    then about the or any OpenVPN performance?

    The performance characteristics are different for IPsec, but not many people are testing & posting IPsec numbers.

    If all is running for you well, why posting then such numbers? With a small Intel Atom C2558 you will get
    without PPPoE something around ~470 MBit/s throughput, and if you have 3,7GHz at 4C/8T you will be
    easy outperform that if you must. And on top of this, here in Germany you will be the "king" if you get your
    hands on a FTTH 400 MBit/s line for private usage! And if this peoples are posting then their numbers
    you will be having no benefit from this  owed to the available 1 GBit/s line in many many other countries!



  • 100+ GB datasets is gigantic, especially if that's every night.  Will you be doing any differential/incremental backups or snapshots?  Significantly reduces the size and duration of subsequent backup jobs after the first one runs.



  • @Finger79:

    100+ GB datasets is gigantic, especially if that's every night.  Will you be doing any differential/incremental backups or snapshots?  Significantly reduces the size and duration of subsequent backup jobs after the first one runs.

    Won't be that much every night and yes, I'm looking at how to shape my backup methodology to minimize the amount of data I have to move.  I've also heard that eventually Verizon will look at what your doing when you exceed 10tb monthly on their GB service.  It's the limited unlimited plan just like many others out there.

    Roveer


Log in to reply