Hardware reqs for heavy Suricata.
-
You'll need a Xeon indeed.
-
@johnkeates:
You'll need a Xeon indeed.
Okay, that answers that :) and likely is better as I have a 200+ dollar ITX kaby lake board already.
So Xeon indeed has me scare though? Will a E3 not be enough, do I need a E5? Or will the E3 do?.
The best E3 I can get, seems excessively priced for what it is lol, at 4.2 and 4 cores, vs the 1240 with 4.1, granted those are boost clocks and the base clocks aren't the same, I can take care of that with microcode hacks :p.
But, Will a E3 1240 v6 be enough? So 4 cores 4 threads at 4.1, Kaby Lake? Good?
Then like 16gb of DDR4 ECC?
-
I may look at using a Ryzen Build for my next 1U Server, as currently running it on an ITX AMD APU 5000, as its has AES-NI,
I also use Surricata, but dont see a massive Performance hit, even when maxing my Bandwidth out.
Hmm what is your bandwidth?
Are you running IDS or IPS, as I plan the latter, and from reading that brings even I5us to it's knees with 100mb speeds lol.
-
@johnkeates:
You'll need a Xeon indeed.
Okay, that answers that :) and likely is better as I have a 200+ dollar ITX kaby lake board already.
So Xeon indeed has me scare though? Will a E3 not be enough, do I need a E5? Or will the E3 do?.
The best E3 I can get, seems excessively priced for what it is lol, at 4.2 and 4 cores, vs the 1240 with 4.1, granted those are boost clocks and the base clocks aren't the same, I can take care of that with microcode hacks :p.
But, Will a E3 1240 v6 be enough? So 4 cores 4 threads at 4.1, Kaby Lake? Good?
Then like 16gb of DDR4 ECC?
ECC is only a soft target. It will help with bitflips that might otherwise crash the software, but other than that it has little benefit as you won't have on-disk storage that would need bitflip protection.
Regarding the E5: that would probably get you a whole lot closer to that 1Gbit. But it doesn't have to be that way. An E3 (a fast one) will be able to push that as long as you don't add to many rules/filters/inspection engines.
-
You will need a very expensive CPU to push "heavy" suricata at gigabit throughput.
Also, I'm not sure why people are pushing you towards high clock speeds primarily? High clock will always help, but this isn't like OpenVPN. Suricata is muktithreaded.
Another thing you need to define is just exactly what heavy suricata really means.
You could easily push gigabit suricata throughput on a few simple rules.
What makes suricata usage heavy are two things:
Number of rules
Composition of rulesSome rules so very simple things, as simple as IP and Port matching like a firewall rules.
Other rules are very complex and match on multiple criteria.
The more complex the rule, the more cycles required to evaluate it.
Similarly, the more rules you are evaluating, the more cycles.You probably already know that since you knew enough to ask the question but I thought I'd throw it out there as many people do not understand suricata at all.
There is a post in an openVPN hardware thread a few months back that has a real life comparison of openvpn speeds and suricata speeds & it details the general composition of the suricata ruleset being used. If you can find that it would probably help you a lot, it includes graphs and top output with CPU time per process.
The bottom line though was that even with a moderate ruleset suricata consumes dramatically more CPU time than openvpn. Part of this is obviously because there is no AES-NI for suricata.So depending on the rules you want to run the CPU could vary from really not that powerful to multiple high end xeons. Obviously those are the two extreme ends of the spectrum.
I would strongly recommend spending a serious amount of time determining EXACTLY which rules you NEED before you attempt to purchase a CPU. It will save you time and money on the long run.
-
Please don't water cool your router…
You are obviously building a very high availability system with SLC SSDs in ZFS mirror.
Water-cooling is counterproductive as you add a possibility to instantly destroy your entire system if it ever fails, even a partial failure.
-
You will need a very expensive CPU to push "heavy" suricata at gigabit throughput.
Also, I'm not sure why people are pushing you towards high clock speeds primarily? High clock will always help, but this isn't like OpenVPN. Suricata is muktithreaded.
Another thing you need to define is just exactly what heavy suricata really means.
You could easily push gigabit suricata throughput on a few simple rules.
What makes suricata usage heavy are two things:
Number of rules
Composition of rulesSome rules so very simple things, as simple as IP and Port matching like a firewall rules.
Other rules are very complex and match on multiple criteria.
The more complex the rule, the more cycles required to evaluate it.
Similarly, the more rules you are evaluating, the more cycles.You probably already know that since you knew enough to ask the question but I thought I'd throw it out there as many people do not understand suricata at all.
There is a post in an openVPN hardware thread a few months back that has a real life comparison of openvpn speeds and suricata speeds & it details the general composition of the suricata ruleset being used. If you can find that it would probably help you a lot, it includes graphs and top output with CPU time per process.
The bottom line though was that even with a moderate ruleset suricata consumes dramatically more CPU time than openvpn. Part of this is obviously because there is no AES-NI for suricata.So depending on the rules you want to run the CPU could vary from really not that powerful to multiple high end xeons. Obviously those are the two extreme ends of the spectrum.
I would strongly recommend spending a serious amount of time determining EXACTLY which rules you NEED before you attempt to purchase a CPU. It will save you time and money on the long run.
Thanks all very good points and ideas, I will try to find that. And yes, I am well aware of Suricatas multi threading that is why I asked about Ryzen and moving to an E5, the E3 will be just as powerful clock for clock, the cores are the what is important.
As far as rules go, I will be doing my best to lighten the load with firewall rules, and only inspecting needed packets. Also a OSSIM, build is in the works as well to help lighten the load as I can pass some less important rules to it, to just want for instead of prevent. So IPS only on Pfsense, with further IDS, on OSSIMs suircata.
To the watercooling. My main concern is SOC in 1u is limited, less than leaks or failures. All the servers are going to be water-cooled.
When I say that though, what you think I mean and what I actually mean are different things :p. The watercooling will be a supernova 1260, inside of a 4u chassis, with 4 D5s in serial, and a massive resovoir (thinking about the best way to do that, might build a 1u shelf into a bunch of resovoirs).
From there, there will be back piping, with steel tube to split offs, on QDCs.
It will then travel into the server, where the waterblock will have steel piping welded to the block for tubes that then leave through holes in the back of the cases, which are then welded to 90s and very log barb fittings. Watercooling is done in massive data centers, the key is it must be done right :), there will be zero leak possibility anywhere it can damage equipment.
Anywhere there is a connection that could leak (fittings to tube) although with the fittings I am choosing the risk is low, will be far back in the rack, where it can't drip on servers below.
-
Gotya, when I saw reference to gaming boards and overclocked i7's I assumed gaming level water-cooling as well haha.
-
Gotya, when I saw reference to gaming boards and overclocked i7's I assumed gaming level water-cooling as well haha.
Oh ya I figured that was what it was :p. I don't really want to run a i7 or gaming board, just was curious if it would help. I thought cores would be better as you said though, and ya I have a server board right now rather use that.
It is techinacally gaming watercooling loop, but I am using all metal blocks and welding them together for server grade reliability :) server coolers are expensive lol, and I can weld :).
While I have you here, any suggestion for amount of ram? I was toying with the the idea of using squid, but thought likely won't do it. So pretty much just base and Suricata and maybe npotg
-
Yeah it sounds like you know what you're doing!
For RAM, I would guess 8GB would be plenty for that application. I use 8GB and lots of packages. The only time I've exceeded those needs is with pfBlockerNG when enabling TLD on a Lot of IPs.
I've seen RAM use get high with suricata only when loading up rules, after that it goes down to moderate usage. I also use a RAM Disk.
I would recommend dual channel RAM for gigabit though.
-
I'd go for multichannel RAM indeed, not only for the bare gigabit, but since it will be copied around at least 1 extra time on top of the normal process loop, (suricata needs it) saving on the round trip time for RAM helps a lot.
-
Yeah it sounds like you know what you're doing!
For RAM, I would guess 8GB would be plenty for that application. I use 8GB and lots of packages. The only time I've exceeded those needs is with pfBlockerNG when enabling TLD on a Lot of IPs.
I've seen RAM use get high with suricata only when loading up rules, after that it goes down to moderate usage. I also use a RAM Disk.
I would recommend dual channel RAM for gigabit though.
Oh… I forgot about that lol, I will definitely be running PFblocker, to block ads, and all the malicous IPs I can find. So ram, ya lol, I have been studying Suricata alot the last few days and that was a first thing to do to elimanate as much work as possible for Suricata.
Okay so multichannel for sure, now,
I think I would have to go 16 in that though, because it's tough finding 4gb sticks of Ddr4 4gbs, I found some lightly used 8gb sticks on my board QVL for 60 each on eBay.
-
used 8GB sticks would be perfect for you if you think you might be using TLD on a lot of lists.
You are building a very high end system anyways, so might as well not skimp on the RAM quantity.
-
used 8GB sticks would be perfect for you if you think you might be using TLD on a lot of lists.
You are building a very high end system anyways, so might as well not skimp on the RAM quantity.
I was thinking the same lol.