Multi-core systems



  • I am trying to find out if pfsense and specifically snort are multi-threaded. I am planning to filter up to 1gbit of traffic and am planning to build a dual six-core amd with 4gb of ram. If there are thread limitations having 12 cores won't help me and I wouldn't want to waste the money.

    Since snort uses so much memory for large amounts of rule why do I keep seeing posts that say a 64-bit version of pfsense is pointless. If I were able to add 64GB of ram I would have less problems with snort

    Thanks


  • Banned

    Whats your scenario like??? 1gbit continuous traffic requires some serious piece of hardware…....



  • it could come to that, i'm just trying to prepare. It's not an issue for me to purchase a dual six core with 64gb. I just want to make sure snort and pfsense are multi-threaded so i'm not wasting money.



  • You potentially could have snort and the kernel executing concurrently. I don't know anything about snort internals so can't comment on whether snort is multi-threaded.

    How many interfaces are you planning to use. I suspect FreeBSD won't use any more than two cores per interface (possibly one thread receiving and one thread transmitting.) I don't know if this will be any different for VLANs but suspect it might be one thread receiving per physical interface and possibly an output thread per VLAN interface. But this concurrency in the kernel tends to be fairly short lived so your limitation may well be what snort is doing.

    If you put 4GB in the systems its unlikely you will be able to use much more than 3GB (perhaps 3.5GB). That's because some of the 4GB physical address space is needed to address device and chipset registers.

    If you seriously want to get 1Gbps through a system, especially if its sustained traffic and mostly small packets for short lived connections you might need to doing a fairly detailed analysis of the code paths.



  • Routing 1Gbps doesn't require anything special.  You can do that with a low power system and a set of good NICs.  Firewalling can take a lot of RAM if you have lots of users which in turn cause lots of states to be active.

    Snort will be the linch pin in this case.  Without Snort, you should be able to do this with nothing more than a 1Ghz Celeron, 1-2GB of RAM, and a few Intel NICs.  With Snort, you will likely need a lot more CPU and potentially a lot more RAM.  I don't know the internals of Snort to know if will multithread or not.  I somehow doubt many, if any, people on this forum do.  Similarly for how much RAM you will need.  You'd need to experiment a bit to see exactly what it takes.  Bonus points if you quantify the CPU and RAM usage on the same machine over a range of sustained traffic profiles (varying traffic and # users independently).  That would give us all a clue as to how to plan a Snort deployment better.


  • Banned

    Yes but he wants to filter the traffic….So routing only is not an option here...

    Therefore he needs serious gear....

    If no one can answer, whether multicore is an advantage and thereby useful, it has to be trial and learn....

    Post results here pls.

    @kc8apf:

    Routing 1Gbps doesn't require anything special.  You can do that with a low power system and a set of good NICs.  Firewalling can take a lot of RAM if you have lots of users which in turn cause lots of states to be active.

    Snort will be the linch pin in this case.  Without Snort, you should be able to do this with nothing more than a 1Ghz Celeron, 1-2GB of RAM, and a few Intel NICs.  With Snort, you will likely need a lot more CPU and potentially a lot more RAM.  I don't know the internals of Snort to know if will multithread or not.  I somehow doubt many, if any, people on this forum do.  Similarly for how much RAM you will need.  You'd need to experiment a bit to see exactly what it takes.  Bonus points if you quantify the CPU and RAM usage on the same machine over a range of sustained traffic profiles (varying traffic and # users independently).  That would give us all a clue as to how to plan a Snort deployment better.



  • @Supermule:

    Yes but he wants to filter the traffic….So routing only is not an option here...

    Therefore he needs serious gear....

    If no one can answer, whether multicore is an advantage and thereby useful, it has to be trial and learn....

    Post results here pls.

    Most filtering is firewalling and doesn't usually require much in the way of CPU, but does potentially require some RAM.  Of course, you can have an obscene number of rules that increase the matching time, but you don't generally need that.

    Snort isn't a filter.  It's an IDS.  It inspects packets, matches them against rules, and saves an alert somewhere.  It's expensive due to the large number of complex rules it typically is setup with.  This is the only component that really has an impact on the system configuration.  The point of calling this out is that Snort can be evaluated independently for resource usage instead of trying to analyze a more complex system involving routing, firewalling, etc.  He could even setup Snort on an available machine and measure the usage against live traffic.



  • @kc8apf:

    Routing 1Gbps doesn't require anything special.  You can do that with a low power system and a set of good NICs.

    You really can't make this claim without knowing the traffic profile. I'd be surprised if your proposed box could even route/firewall 100mbps of small packets across a large number of short-lived TCP sessions. pps is far more relevant than mbps, and you'll need pretty serious hardware to get to 1Gbps of 64-byte packets in pfSense. Not that that's a typical traffic profile, but making blanket claims like this is dangerous.



  • Actually, I've run that experiment a few years ago.  An Intel Core2 Duo can easily handle a 1Gbps link with 64-byte packets and only have ~30% cpu usage.  The bigger problem with small packets isn't actually the CPU, but rather the protocol overhead on the wire.  When you get to a certain point (I believe around 16 bytes), the protocol overhead and inter-packet spacing requirements mean Ethernet can't actually do full wire rate.



  • @ktims:

    You really can't make this claim without knowing the traffic profile. I'd be surprised if your proposed box could even route/firewall 100mbps of small packets across a large number of short-lived TCP sessions. pps is far more relevant than mbps, and you'll need pretty serious hardware to get to 1Gbps of 64-byte packets in pfSense. Not that that's a typical traffic profile, but making blanket claims like this is dangerous.

    ktims and kc8apf pretty much spot on; it's PPS not Mbps. That FreeBSD has always been the PPS king doesn't hurt. It does not take a great deal of hardware to break 1M PPS - a single Xeon X5420 (2.5GHz Quad Core) is easily sufficient for >1M PPS at 64-byte size on 7.2, with mild tuning, on a broken driver.

    Snort can use a lot of memory - yep. But 64-bit is not necessary with PAE. Yes, it will require a custom pfSense build; PAE is not enabled in GENERIC. PAE also has very strict driver restrictions, not all drivers work with >4GB. But for large memory requirements, it's the least disruptive option.

    So the OP needs to answer two questions before we could make reasonable recommendations. First, we need to know the average and peak PPS rates. Second, we need to know how many connections per second snort needs to deal with. Those are the two primary consumers of CPU. Past that, it's just going to be tuning and a custom pfSense build with PAE enabled. Honestly, 64GB is likely overkill, but 4GB is also not going to be enough.


Log in to reply