New Netgate Appliance for IPS/IDS

keyser

@blaytrail said in New Netgate Appliance for IPS/IDS:

Thanks. The SG-6100 is very expensive. Do you have any other devices that are cheaper that can run Suricata at this time?

Well in my opinion (I’m not from netgate) no.. well not official appliances anyway. The SG-3100 has power to cope at a reasonable bandwidth, but the CPU is 32bit only, and already package support is mixed. It’s also end of sale for quite a while, so stay away from that one.
If you can find a SG-5100 left over somewhere, that would do the trick.

Otherwise you are looking at 3rd party hardware, and pfSense CE - and optionally a subscription for pfSense+

stephenw10

We can now (since earlier today) point you at the new 4100:
https://www.netgate.com/pfsense-plus-software/how-to-buy#4100

Steve

keyser

@stephenw10 said in New Netgate Appliance for IPS/IDS:

We can now (since earlier today) point you at the new 4100:
https://www.netgate.com/pfsense-plus-software/how-to-buy#4100

Steve

Whoaa, that’s a very nice/cool little midtier firewall. I would probably have preferred it was ARM (power consumption and performance), but this is not bad at all :-)

blaytrail

Thanks for the update. This is exactly what I needed.
I was told with the new version of pfSense 2.6 .0 you can run the software on any device. Is that true? You can run Netgate pfsense 2.6.0 on any device.

More info here: https://www.youtube.com/watch?v=jZpDRcWjIok

stephenw10

You have always been able to run CE (2.6) on any x86 device assuming supported hardware. Now you can upgrade to pfSense Plus (22.01) on that hardware.

Steve

SteveITS

@blaytrail I'm late to the thread, but my answer would be "it depends." We have several 2100s and 3100s in service at clients all running Suricata or in a couple cases Snort, with no issues. As noted the 2100 is similar to the 1100 but has dedicated interfaces.

Download bandwidth is a factor as higher speeds will use more CPU power. If you look at the product comparison page it has numbers for each for 10000 firewall rules, VPN, etc., so you can kind of guess at a high CPU load. To some degree higher CPU usage is true even without IDS, but I want to say from memory turning off Suricata drops CPU usage by about half. The 330ish Mbps I get from my home cable (300/10, with Snort) doesn't max out my 2100's CPU but it is rather high and I don't think it would get to 500.

Limiting the rulesets used has an effect of course...no need to look for web server exploits if you have no open ports or no web server. Also, put Suricata on LAN instead of WAN so it isn't scanning packets that will immediately be dropped anyway. Note most web traffic nowadays is encrypted, so nothing to scan there. So, maybe Suricata is not as useful in a home situation...at my home there are days it doesn't even log an alert.

Given the recent thread about calculating eMMC life, I checked a few and two 3100s in service since Oct/Nov 2017 were at 40% and 50% (and, the latter does not have Suricata). So I'm personally not overly worried about logging, as long as it isn't logging a lot of alerts like our data center does. https://www.netgate.com/supported-pfsense-plus-packages lists three heavy-logging packages as "requires SDD" or for others like Snort/Suricata "SSD/HDD is strongly recommended" but I think it depends on how much writing is being done. Also, one can enable a RAM disk for logs (which the two 3100s above weren't using for the last 4.3 years).

If you are thinking about upgrading anyway, you could just try it out and see, and upgrade if necessary.

keyser

@steveits said in New Netgate Appliance for IPS/IDS:

Given the recent thread about calculating eMMC life, I checked a few and two 3100s in service since Oct/Nov 2017 were at 40% and 50% (and, the latter does not have Suricata). So I'm personally not overly worried about logging, as long as it isn't logging a lot of alerts like our data center does. https://www.netgate.com/supported-pfsense-plus-packages lists three heavy-logging packages as "requires SDD" or for others like Snort/Suricata "SSD/HDD is strongly recommended" but I think it depends on how much writing is being done. Also, one can enable a RAM disk for logs (which the two 3100s above weren't using for the last 4.3 years).

That is really interesting information as Netgate has not confirmed the actual write endurance of the built-in eMMC’s.
So when I made my calculations for the write endurance thread I used a guestimated endurance of about 12Tb for the small 8Gb eMMC and 25Tb for the 16Gb ones. Those are “common” averages for small eMMCs I could find online.

Could you perhaps have a look at how much writing is being done on the 3100’s that had a 50% wear at this point? What is the average write rate on them since the last boot? Use the command “iostat -x” and paste the numbers here :-)

blaytrail

Thanks for all the information. I'm starting to lean towards getting a PC with two nic's and installing pfsense.

I agree, using a SSD/HDD with Snort/HDD will extend the life of the drive.

Can I get pfsense as an ISO to install on a server or VM?

stephenw10

@blaytrail said in New Netgate Appliance for IPS/IDS:

Can I get pfsense as an ISO to install on a server or VM?

Yes: https://nyifiles.netgate.com/mirror/downloads/pfSense-CE-2.6.0-RELEASE-amd64.iso.gz

Steve

blaytrail

Thanks Steve. I just downloaded the ISO.

If I'm not mistaken, the new version of pfsense is exactly the same as what is placed on the Netgate appliances.

SteveITS

@keyser said in New Netgate Appliance for IPS/IDS:

look at how much writing is being done on the 3100’s that had a 50% wear

extended device statistics
device       r/s     w/s     kr/s     kw/s  ms/r  ms/w  ms/o  ms/t qlen  %b
flash/sp       0       0      0.0      0.0     7     0     0     7    0   0
mmcsd0         2       1     50.6     13.0     1     3     0     1    0   0
mmcsd0bo       0       0      0.0      0.0     0     0     0     0    0   0
mmcsd0bo       0       0      0.0      0.0     0     0     0     0    0   0

That one does have bandwidthd running on it. The 50% is presumably rounded somehow but not sure which direction.

@blaytrail said in New Netgate Appliance for IPS/IDS:

the new version of pfsense is exactly the same as what is placed on the Netgate appliances

No, Netgate appliances have pfSense Plus. 2.6 is the open source/CE version. You can however upgrade to Plus if desired. Currently they are very similar.

keyser

@steveits said in New Netgate Appliance for IPS/IDS:

extended device statistics
device r/s w/s kr/s kw/s ms/r ms/w ms/o ms/t qlen %b
flash/sp 0 0 0.0 0.0 7 0 0 7 0 0
mmcsd0 2 1 50.6 13.0 1 3 0 1 0 0
mmcsd0bo 0 0 0.0 0.0 0 0 0 0 0 0
mmcsd0bo 0 0 0.0 0.0 0 0 0 0 0 0
That one does have bandwidthd running on it.  The 50% is presumably rounded somehow but not sure which direction.

Thank you Steve - That's really interesting information. This is obviously math with some fairly heavy handed assumptions, but here it goes (Based on your provided information):

Let's say the 13 KB writes/s average since last reboot is also the average across the devices lifetime. You put it into service in Oct. 2017 which is around 1600 days ago.

So 13KB/s * 60sec * 60min * 24hours * 1600days= about 1.8TB written.
Those 1.8TB caused 50% wear, which in effect means about a 4TB write endurance on the eMMC. Half of what I originally predicted and estimated my own devices lifetime from.
I know the that your devices very small sustained writes is the worst because of how SSDs works inside, so likely it has caused a much higher write amplification than a sustained 500KB/s would.

But still - it's very obvious that eMMC's are dangerous when packages does a lot of logging/writing or temporary storage. Luckily I became aware of an issue in pfBlockerNG that caused it to do sustained writes of about 350KB/s - which in effect would have killed my SG-2100 in about 6 months with the math above, or about a year with my guesstimated 11TB write endurance.

Needless to say I have upgraded my SG-2100 and 6100 with a SSD now to avoid prematurely killing them :-)

stephenw10

The 1100 is where you should really avoid heavy drive writes because it can only boot from eMMC.

On everything else you can install an SSD and recover should you have a drive failure.

The actual number of drive failures we see is far lower than you might expect from calculations like that though. Actual writes to the flash are significantly lower than the write data to the drive it would seem.

Steve

keyser

@stephenw10 said in New Netgate Appliance for IPS/IDS:

The 1100 is where you should really avoid heavy drive writes because it can only boot from eMMC.

On everything else you can install an SSD and recover should you have a drive failure.

The actual number of drive failures we see is far lower than you might expect from calculations like that though. Actual writes to the flash are significantly lower than the write data to the drive it would seem.

Steve

That's good to know - I was wondering if one could expect the box to continue to work if the eMMC was dead. That does make it less catastrophic (unless you are a SG-1100 owner)

SteveITS

@keyser said in New Netgate Appliance for IPS/IDS:

assumptions

I double checked and it turns out that router booted a couple hours ago, so it may not be great to extrapolate. Sorry about not noticing that.

The other 3100 (40%) is 3 days 7 hours uptime and:

device       r/s     w/s     kr/s     kw/s  ms/r  ms/w  ms/o  ms/t qlen  %b
flash/sp       0       0      0.0      0.0     7     0     0     7    0   0
mmcsd0         0       0      0.5     29.1     2     7     0     7    0   0
mmcsd0bo       0       0      0.0      0.0     0     0     0     0    0   0
mmcsd0bo       0       0      0.0      0.0     0     0     0     0    0   0
md0            0       0      0.0      0.0     0     0     0     0    0   0

Probably would be better to wait a few weeks and do the math. :)

keyser

@steveits said in New Netgate Appliance for IPS/IDS:

@keyser said in New Netgate Appliance for IPS/IDS:

assumptions

I double checked and it turns out that router booted a couple hours ago, so it may not be great to extrapolate. Sorry about not noticing that.

The other 3100 (40%) is 3 days 7 hours uptime and:
device       r/s     w/s     kr/s     kw/s  ms/r  ms/w  ms/o  ms/t qlen  %b
flash/sp       0       0      0.0      0.0     7     0     0     7    0   0
mmcsd0         0       0      0.5     29.1     2     7     0     7    0   0
mmcsd0bo       0       0      0.0      0.0     0     0     0     0    0   0
mmcsd0bo       0       0      0.0      0.0     0     0     0     0    0   0
md0            0       0      0.0      0.0     0     0     0     0    0   0
Probably would be better to wait a few weeks and do the math. :)

Yes, a long uptime would be much better. Those numbers posted with this box is more in line with the 11 - 12Tb Write endurance I guesstimated for the 8GB eMMC.

blaytrail

Thanks, everyone, for all the information. It appears things would be much easier to use a VM or PC to run pfsense. Is there an advantage of using an appliance besides support?

SteveITS

@blaytrail The appliances would usually use way less power than a hardware PC. For a VM it probably won't add much to power usage on the server, but VMs are more complex to set up and isolate the different interfaces.

@keyser I'm going to take our conversation to that eMMC thread.

mcury

Everything seems to be fine here with 22.01 in a SG-3100, pfblockerng running.

                       extended device statistics
device       r/s     w/s     kr/s     kw/s  ms/r  ms/w  ms/o  ms/t qlen  %b
flash/sp       0       0      0.0      0.0     8     0     0     8    0   0
mmcsd0         0       2      1.9     46.1     1     5     0     4    0   0
mmcsd0bo       0       0      0.0      0.0     0     0     0     0    0   0
mmcsd0bo       0       0      0.0      0.0     0     0     0     0    0   0

SteveITS

@mcury And what does the mmc program tell you for the life? Might be best to reply in that other thread as I feel we're digressing a bit off this one...? I don't want to hijack the original question.