Taming the beasts… aka suricata blueprint

wcrowder

Ran into a small problem with bbcan177's script. Was working wonderfully then quit updating after the last Alpha update. Running pfSense 2.2 alpha.

Here's the error:

SSL options: 81004bff
Peer verification enabled
Using CA cert file: /etc/ssl/cert.pem
Certificate verification failed for /O=www.projecthoneypot.org/OU=Domain Control Validated/CN=www.projecthoneypot.org
675141692:error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed:/usr/pfSensesrc/src/secure/lib/libssl/../../../crypto/openssl/ssl/s3_clnt.c:1169:
fetch: https://www.projecthoneypot.org/list_of_ips.php?t=d&rss=1: Authentication error

Anyone else run into this?

NEVER MIND, I'm stupid… LOL..

Mr. Jingles

Well thank you Bill and JFL for boosting my self esteem ;D ;D ;D

But now… 8)

( ;D )

I am struggling with a dnsmasq problem (blocking ad servers via dnsmasq, can't get the script to run in cron - although manually it works), and on the site that provides the lists I also noted a format for Snort:

http://pgl.yoyo.org/adservers/

(Select the second drop down - list ad servers IP adresses): there is also a Snort format of the list there.

Would it make sense to use Snort to block ad servers using that list?

I know there will be reservations; 'noob, shut up'.

Ok, I will :P

bmeeks

@Hollander:

Well thank you Bill and JFL for boosting my self esteem ;D ;D ;D

But now… 8)

( ;D )

I am struggling with a dnsmasq problem (blocking ad servers via dnsmasq, can't get the script to run in cron - although manually it works), and on the site that provides the lists I also noted a format for Snort:

http://pgl.yoyo.org/adservers/

(Select the second drop down - list ad servers IP adresses): there is also a Snort format of the list there.

Would it make sense to use Snort to block ad servers using that list?

I know there will be reservations; 'noob, shut up'.

Ok, I will :P

When you can't get a script to run via cron, the #1 cause is forgetting to provide the complete and entire path to all files used in the script. When a cron task executes, it does not inherit the user environment you have when working at a shell prompt (CLI). Some examine the script and make sure full paths are provided for all referenced files.

Bill

BBcan177

@wcrowder:

Ran into a small problem with bbcan177's script. Was working wonderfully then quit updating after the last Alpha update. Running pfSense 2.2 alpha.

Here's the error:
SSL options: 81004bff
Peer verification enabled
Using CA cert file: /etc/ssl/cert.pem
Certificate verification failed for /O=www.projecthoneypot.org/OU=Domain Control Validated/CN=www.projecthoneypot.org
675141692:error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed:/usr/pfSensesrc/src/secure/lib/libssl/../../../crypto/openssl/ssl/s3_clnt.c:1169:
fetch: https://www.projecthoneypot.org/list_of_ips.php?t=d&rss=1: Authentication error
Anyone else run into this?

NEVER MIND, I'm stupid… LOL..

SSL3_GET_SERVER_CERTIFICATE:certificate verify failed
The server certificate is invalid, either because it is signed by an invalid CA (internal CA, self signed,…), doesn't match the server's name or because it is expired.

Hi wcrowder, is this still an issue for you?

BBcan177

@bmeeks:

I do also plan to offer what is a kind of CLI interface for this. Do a Google search for enablesid.conf, modifysid.conf and disablesid.conf. You should get some hits showing how these work with packages such as PulledPork. The idea for the Suricata and Snort packages is to be able to edit these files offline and then upload them to the firewall.

This will make the Rules Configuration Process a Breeze. All we would have to do, is copy the text based conf file to any other Snort/Suricata Interface or to another Box. Rules can be disabled or enabled by SID, Category or by PCRE (ie : Autodesk or Adobe - Which will disable/enable any rules that has any of those Keywords in them. This way even as new rules are added, the PCRE will disable them at each rule-update automatically. :)

wcrowder

Sorta, I just removed the S from each HTTP and it updated, not the best practice. LOL. I've updated the cert.pem with the latest version of Mozilla's certs, no go, assuming it's something with the 2.2 Alpha. Also takes 7 minutes to get through "configuring Firewall" upon boot. Have "./pfiprep killdb" twice then pfiprep with bypass=YES after update and it works. Still now SSL. Just messing around. Really fine work on your part.

BBcan177

@wcrowder:

Sorta, I just removed the S from each HTTP and it updated, not the best practice. LOL. I've updated the cert.pem with the latest version of Mozilla's certs, assuming it's something with the 2.2 Alpha. Also takes 7 minutes to get through "configuring Firewall" upon boot. Have "./pfiprep killdb" twice. Just messing around. Really fine work on your part.

Hi wcrowder,

You need to check the pfSense Cert, as that is the machine that is fetching those files.

BBcan177

https://www.mail-archive.com/freebsd-security@freebsd.org/msg05085.html
Looks like FreeBSD 9 doesn't have a default pem file in /etc/ssl/cert.pem ?

fetch
–ca-cert=file
[SSL] Path to certificate bundle containing trusted CA cer-
tificates. If not specified, /etc/ssl/cert.pem is used. The
file may contain multiple CA certificates. The port
security/ca_root_nss is a common source of a current CA bun-
dle.

Do you have a file in that directory?
[ [b] ls -lah /etc/ssl/ ]

I'm not the best at certs. But I think you should just be able to generate a Self-signed one from the pfSense Cert Manager? I don't want to suggest you disable the cert check in the fetch cmd line.

http://www.freebsd.org/cgi/man.cgi?query=fetch(1)
My script fetchs with this line:

$pathfetch -v -o $workdir$infile -T 20 $addr$infile

Try this:

cd /tmp
fetch -v -o honeypot.txt "https://www.projecthoneypot.org/list_of_ips.php?t=d&rss=1"

Or you can switch to using WGET: (Script pfiprep has an option to switch between fetch and wget)

Script Command:
$pathwget –no-check-certificate -T 20 -O $workdir$infile $addr$infile

Try:
wget –no-check-certificate -O honeypot.txt "https://www.projecthoneypot.org/list_of_ips.php?t=d&rss=1"

I think Foetus had a similar issue with 2.2. Not sure how he made out.

Supermule

Isnt 2.2 FreeBSD 10 based??

Mr. Jingles

@bmeeks:

@Hollander:

Well thank you Bill and JFL for boosting my self esteem ;D ;D ;D

But now… 8)

( ;D )

I am struggling with a dnsmasq problem (blocking ad servers via dnsmasq, can't get the script to run in cron - although manually it works), and on the site that provides the lists I also noted a format for Snort:

http://pgl.yoyo.org/adservers/

**(Select the second drop down - list ad servers IP adresses): there is also a Snort format of the list there.

Would it make sense to use Snort to block ad servers using that list?**

I know there will be reservations; 'noob, shut up'.

Ok, I will :P

When you can't get a script to run via cron, the #1 cause is forgetting to provide the complete and entire path to all files used in the script. When a cron task executes, it does not inherit the user environment you have when working at a shell prompt (CLI). Some examine the script and make sure full paths are provided for all referenced files.

Bill

Thanks Bill ;D

I didn't mean to bother you with the problem I have to get a cron job running (I did have the full path, btw): I was only meandering away from the point in question: the text in bold in the above :P

BBcan177

@Supermule:

Isnt 2.2 FreeBSD 10 based??

Yes it is, but I couldn't find much info on this issue with my google FU…

I did find this link:
http://smyck.net/2014/01/22/freebsd-authentication-error/

I don't have a 2.2 box, so I can't test it myself. If anyone else has 2.2, can you see if these two commands work? Don't need to be using my Script to test if fetch and https work on 2.2 Alpha.

cd /tmp

fetch -v -o honeypot.txt "https://www.projecthoneypot.org/list_of_ips.php?t=d&rss=1"
wget –no-check-certificate -O honeypot.txt "https://www.projecthoneypot.org/list_of_ips.php?t=d&rss=1"

wcrowder

@BBcan177

My problem was a PEBKAC, I had a self signed cert installed in pfSense. I removed it and it works just fine. Thanks for your hard work. I can report that I'm not having any issues with the script or the widget in 2.2, all seems to be working fine except the patches for dns look up report that they can not be installed cleanly. Not important.

Thanks,
Bill

BBcan177

@wcrowder:

all seems to be working fine except the patches for dns look up report that they can not be installed cleanly. Not important.

Good to hear!

Is this the 2.2 diag_dns.php file that matches what you have on your box?
https://github.com/pfsense/pfsense/blob/master/usr/local/www/diag_dns.php

If its the same, I will post an updated version of the Patch.

wcrowder

I believe it is. On the current snapshot.

Patch can NOT be applied cleanly (detail)
Patch can NOT be reverted cleanly (detail)

Thanks,
Bill

wcrowder

Also neither widget displays list count or correctly determines failed downloads. It is correctly displaying other correct information. This is on Today's snapshot of 2.2. I didn't look into it because I can read logs… :)

PfIP Reputation2
minimize
close
(All Downloads Successful)

List Count
Alias CIDRs Packets Updated Status
IR_IB 60359 0 Jul 29 22:42
IR_PRI1 2418 0 Jul 29 20:42
IR_PRI2 24652 40 Jul 30 15:52

wcrowder

I want to think jflsakfja and BBcan177 for this thread. Fantastic work.

@wcrowder:

I want to think jflsakfja and BBcan177 for this thread. Fantastic work.

As long as you are not thinking of us while taking a bath ;)

Mr. Jingles

@jflsakfja:

@wcrowder:

I want to think jflsakfja and BBcan177 for this thread. Fantastic work.

As long as you are not thinking of us while taking a bath ;)

;D ;D ;D

Mr. Jingles

JFL, could I ask: what is your opinion on this:

Why block an attacker that's already been blocked by your firewall? Because they will continue to beat on your door till they find something they can brute force or exploit. So lets use their initial scanning to identify hostile intent and block.

http://doc.emergingthreats.net/bin/view/Main/WhatEveryIDSUserShouldDo

It would make sense to this noob ;D

@Hollander:

JFL, could I ask: what is your opinion on this:

Why block an attacker that's already been blocked by your firewall? Because they will continue to beat on your door till they find something they can brute force or exploit. So lets use their initial scanning to identify hostile intent and block.

http://doc.emergingthreats.net/bin/view/Main/WhatEveryIDSUserShouldDo

It would make sense to this noob ;D

Nice to see that most things in this topic are also mirrored in that page. Too closely mirrored, if you catch my drift ;). Now if they had the same spirit about deleting their outdated rules, then we would all be happy ;D

And yes, that's what the recommended unused ports rule (in this topic and that page) does. It identifies people that are port scanning you, but maybe managed to get a list of open ports before they were banned. Cutting off their access before them noticing that you have open ports and try to connect to those, is a sane way of preventing them from getting onto your network. If you know that they are malicious, why even let them in the network in the first place?

My recommended rule was detecting ALL traffic to unused ports. His rule detects traffic that is attemping to be established. What if the "attacker" decides to scan in an unconventional way? You should never see traffic to those ports, end of story. If even a single packet arrives at those ports, then it is malicious, since you did not ask for it.

Didn't have time to read the entire page, but from quickly going through it it looks like he is recommending a similar set as to what is described in this topic. Suricata on pfsense belongs in the network gateway category, and as such should be responsible for protecting the network proactively if it can, or reactively if it cannot (encrypted connections terminated to internal servers for example). From suricata's/pfsense's point of view, it should try to make the network conform to specific requirements, like the requirement to contact specific DNS servers.

What I will comment on though is the section "Systems That Should Never Surf the Web". This is the typical wrong approach to using an IDS/IPS system. If you don't want those systems to surf the web, use what the firewall is designed to do: Be a glorrified router that routes according to specific rules, and block those hosts using a solution as close to the network as possible. If I were responsible for systems that shouldn't surf the web, then I would just air-gap them instead. It's the closest to the network you can be, there isn't actually a network (connecting them to the Internet), a separate physical intranet is acceptable, since not everybody has ninjas sworn in blood oaths to get into their networks.

But shouldn't the firewall be used for unused ports as well? From the inside of the network, absolutely yes. From the outside of the network you need the sentry (suricata) as high on the walls as possible. Bonus points if you can give him binoculars as well. The more he can see, the faster he will alert the rest of the town (pfsense) that danger is coming. The faster the town is warned, the faster the gates will be locked (host added to banned hosts). Ask yourself, what use is the sentry on top of the wall 1km away, monitoring the market stall thieves in the center of the town? Why the medieval thinking? If you haven't yet read the Art of War, you don't belong in the network security field.