Taming the beasts… aka suricata blueprint
-
Aside from extremely enjoying your funny writing style ( ;D ) I also think you can't get too much karma for all that you are doing with regards to helping people set up Suricata (and before that: Snort).
Thank you, secret man :P
-
I've read your post 3 times and I'm having a difficult time understanding the floating rule. The default nature of the firewall is to block incoming traffic unless you add a pass rule. As I understand it, floating rules are evaluated first. So wouldn't this rule always block incoming packets on the interface regardless of the interface rules?
The giant red warning under that rule should explain it. It's a rule that will ONLY apply to traffic destined for pfsense's ports. By default pfsense could open up the webgui to an undesired interface, which will not be covered by the default rule. Depending on how far away you sit from the fan, it leads to varying amounts of brown stuff raining down when "it" hits the fan. ;)
@Hollander: Finished today analyzing logs for 3,997,696 IP addresses. Those (almost) 4mil IPs were what tripped up our security systems in the first 6 months of this year. Needless to say they gained a magical place in my "Permanently Banned" Hall of Shame.
If pfsense/suricata/other logs can help identify 4mil malicious IPs, then sure as hell they deserve all the support we can give them.
@all: List has had a couple of updates don't forget to check it regularly. I'm trying to add descriptions when I edit the list, so it's obvious what I added/removed/changed, without needing to go through the entire list.
-
@jflsakfja:
I've read your post 3 times and I'm having a difficult time understanding the floating rule. The default nature of the firewall is to block incoming traffic unless you add a pass rule. As I understand it, floating rules are evaluated first. So wouldn't this rule always block incoming packets on the interface regardless of the interface rules?
The giant red warning under that rule should explain it. It's a rule that will ONLY apply to traffic destined for pfsense's ports. By default pfsense could open up the webgui to an undesired interface, which will not be covered by the default rule. Depending on how far away you sit from the fan, it leads to varying amounts of brown stuff raining down when "it" hits the fan. ;)
@Hollander: Finished today analyzing logs for 3,997,696 IP addresses. Those (almost) 4mil IPs were what tripped up our security systems in the first 6 months of this year. Needless to say they gained a magical place in my "Permanently Banned" Hall of Shame.
If pfsense/suricata/other logs can help identify 4mil malicious IPs, then sure as hell they deserve all the support we can give them.
@all: List has had a couple of updates don't forget to check it regularly. I'm trying to add descriptions when I edit the list, so it's obvious what I added/removed/changed, without needing to go through the entire list.
jflsakfja,
Thank you for the clarification! Makes sense now.
-
An intermezzo question: did anybody try to print this thread? I wanted to start working on this, and print it to study it thoroughly first. The printing leads to iny tiny small text on the paper, not readable. I tried this from three computers, 3 browsers, all the same.
Is this a forum software thing? Would an admin perhaps mind to verify?
(text in red so admin notices it)
Thank you ;D
-
Never tried printing anything from around here, but I've been getting weird errors when posting replies. It's time to abandon the clusterf*** that is the current forum software*. I believe it will also be the solution to the black hole creation problem as well, and who knows, maybe one day we too can edit our old posts. One can only hope.
Notes:
- This is my personal opinion and I'm allowed to say it based on provisions in my country's constitution, as well as international human rights treaties.
Disclaimer:
If you are in any way related to the current clusterf*** forum software, then you should not be offended by a single person's opinion of it. If it is the majority's opinion of it though, that means that the current forum software is indeed a clusterf***, and in that case you should seriously consider abandoning the project and letting it die the slow and horrible death it deserves. -
@jflsakfja:
Never tried printing anything from around here, but I've been getting weird errors when posting replies. It's time to abandon the clusterf*** that is the current forum software*. I believe it will also be the solution to the black hole creation problem as well, and who knows, maybe one day we too can edit our old posts. One can only hope.
Notes:
- This is my personal opinion and I'm allowed to say it based on provisions in my country's constitution, as well as international human rights treaties.
Disclaimer:
If you are in any way related to the current clusterf*** forum software, then you should not be offended by a single person's opinion of it. If it is the majority's opinion of it though, that means that the current forum software is indeed a clusterf***, and in that case you should seriously consider abandoning the project and letting it die the slow and horrible death it deserves.;D ;D ;D
(Love your funny writing style :P ).
Your royalness, I am currently clusterf*cking around on my box with your tuto. For one, the floating rule blocks everything out, so I had to disable that. I've noticed in a follow up post that you wrote the floating rule was meant to prevent access to the pfSense GUI, but I think that is not what your initial instruction does (but again, keep in mind I will be the eternal noob).
A more serious question for me is, while I am now currently looking at the script: which are the lines I need to comment out to select lists? I am looking but honestly have no clue :-[ Could you give an example of a line that contains a list? Is it the ones all the way at the bottom?
Thank you ;D
-
Sorry about that floating rule, I later added the giant red warning. The floating rule should only apply to the pfsense's ports, and that shouldn't block any other traffic.
As far as the lists go, it's near the end of the script. There are instructions in the script to enable/disable the lists. Enabling a list is usually removing the # in front of the line containing the list.
-
:'( :-[ ???
( >:( )
How on earth can this be possible? I even logged out and logged in again. Where on earth does it get this directory from?
It is not difficult to remain the eternal noob when this happens ( ;D >:( )
[b]EDIT: it appears it hadn't saved the first: userfolder=/home/badips
Probably because I had the file open in both WinSCP and via the Diagnostics/edit file. I saved it in the latter, but apparently since it was also open in WinSCP it didn't tell me it couldn't write but simply said nothing.
-
@jflsakfja:
Sorry about that floating rule, I later added the giant red warning. The floating rule should only apply to the pfsense's ports, and that shouldn't block any other traffic.
As far as the lists go, it's near the end of the script. There are instructions in the script to enable/disable the lists. Enabling a list is usually removing the # in front of the line containing the list.
Thanks Jflsakfja ;D
Next up Floating tab:
Set up a rule but make these changes:| Action | Block |
| Quick | TICKED!!! |
| Interface | Hold CTRL and click on all interfaces EXCEPT LAN(admin) and SYNC |
| Direction | any |
| Source | any |
| Destination | any |DON'T CHANGE DESTINATION PORT RANGE!!!
So when I add new floating rule, the above reads to block any source, any direction, to any destination (I left the ports to 'other', which is the default when I created a new floating rule, so I didn't change it per the red text), effectively blocking all LAN out (I think, at least when I disabled the rule I had internet access again).
Remember: eternal noobs will be eternal noobs ;D
-
Re-reading it does make sense on why it blocked out traffic. I meant to say create a new floating rule, based on the previous allow rule, but this time around change the pass to a block, keeping the destination ports the same.
A)1 normal pass rule for the ports active on the interface you want to administer pfsense from.
B)1 floating rule block rule for ALL interfaces EXCEPT the one you want to administer pfsense from.
Both rules should have their destination as the alias for pfsense's ports. The allow rule well, obviously allows traffic to those ports on your admin (LAN?) interface, but the floating rule should block all traffic for those ports, on each and every >other< interface.
I don't have access to a pfsense system since I'm out of town for the weekend, one can only post so much from memory :p.
-
I have been working on a script
As I am working my way down this thread on the instructions I arrived at your script: thank you very much for creating it ;D
I only understand 10% of what it does (given my eternal noob status), but I do know that this is quite some work. My hero-list on this board keeps on getting bigger, I just added you to it as well ;D
Thank you & bye,
-
@jflsakfja:
Re-reading it does make sense on why it blocked out traffic. I meant to say create a new floating rule, based on the previous allow rule, but this time around change the pass to a block, keeping the destination ports the same.
A)1 normal pass rule for the ports active on the interface you want to administer pfsense from.
B)1 floating rule block rule for ALL interfaces EXCEPT the one you want to administer pfsense from.
Both rules should have their destination as the alias for pfsense's ports. The allow rule well, obviously allows traffic to those ports on your admin (LAN?) interface, but the floating rule should block all traffic for those ports, on each and every >other< interface.
I don't have access to a pfsense system since I'm out of town for the weekend, one can only post so much from memory :p.
Thank you Jfl :P
-
I came across this site "infragard" https://www.infragard.org/node
InfraGard is a partnership between the FBI and the private sector.
It is an association of persons who represent businesses, academic institutions,
state and local law enforcement agencies, and other participants dedicated to
sharing information and intelligence to prevent hostile acts against the U.S.Unfortunately, you need to give them your first born to gain access to their Files.
However, I have come across their most recent data, which can be viewed with these links:
https://publicintelligence.net/fbi-cyber-targeting-gov-networks/
https://publicintelligence.net/siac-cryptowall/
https://publicintelligence.net/fbi-blackshades-bulletins/
http://www.eventtracker.com/support/knowledge-update-et75asig-001/As these are static Blocklists, I have added the option in the pfIP Rep Script to download
these files once only. (Setting the schedule to "$sch0), should the download fail, you can set
the schedule to "$sch1" and run it and set it back to "$sch0" after it completes.Two of the links are for domain blocking, this info could be used for Squid or dns sinkholes.
I have also updated the pfIP_Reputation.widget.php to include a "Ack" Acknowledge button to clear any previous "FAIL" Downloads. This will just edit the Daily.log from "FAIL" to "Fail", so you can still review the Daily.log for trending issues with downloading.
Here is a screenshot of the widget
And the link to my GIST for the pfIP_Reputation2.widget.php
https://gist.github.com/BBcan17/67e8c456cb399fbe02ee#file-pfip_reputation2-widget-phpI Have updated the pf IP Reputation Manager Script to version 2.3.4
You can review the revisions in my GIST.
https://gist.github.com/BBcan17/67e8c456cb399fbe02ee
For pfiprep make the changes to your existing file or just overwrite and add your changes as required.
For pfiprepman, just backup the previous 2.3.2/3 version and replace with the latest 2.3.4 version.
Changes to pfiprep
Added the "FBI Suspicious Conus and Oconus Blocklists"
Added the "FBI Facebook FBUID Blocklist"
Added the "Suricata TOR Blocklist to the TOR Section"
INFO - OpenBL supports other Blocklist options that can be set.Changes to pfiprepman
The Script also now supports extracting IP Blocklists from .XLSX files.CountryCode Blocklists
With Cinos help, we have made some code improvements.
Added a "perl script - IPCALC" to convert the Ranges to CIDR
Found some other code changesI recommend running
[ [b]./pfiprep killdb ] with version changes or [ [b] ./pfiprep killdb dskip ]
If you find any Bugs please let me know and I will promptly fix them.
-
@Hollander:
I have been working on a script
As I am working my way down this thread on the instructions I arrived at your script: thank you very much for creating it ;D
I only understand 10% of what it does (given my eternal noob status), but I do know that this is quite some work. My hero-list on this board keeps on getting bigger, I just added you to it as well ;D
Thank you & bye,
This is what Open Source is all about. We've all caught the bug and that's why we enjoy spending time helping each other to advance of Network Security.
In regards to your comments, Thanks, its was lots of work but the best part is when people actually use it. If you have any questions, let us know or send me a PM when you need more help. :o :o
I would recommend leaving most of the settings as is, and then change things after you get it working. I would use the default Group/Tiers instead of adding all of the individual Blocklist aliases.
If anything, this is good practice to learn how to use the shell and other parts of FreeBSD that you never knew existed or maybe never wanted to know ! 8)
-
@jflsakfja:
EDIT!!!! MISSED THE QUICK CHECKBOX. TICK THAT!
Now I am confused. In your initial post on creating floating rules using the aliases you indicate NOT to tick Quick.
So just to be sure: To Tick Or Not To Tick? ;)
And does this go for both inbound and outbound?(BTW: thanks for all the time you put into this thread)
-
Tick the quick check box. This tells pfsense to process the rule right away. If its not ticked and let's say you have inbound port 80 opened,that rule would let traffic pass bypassing your floating block/reject rule
-
To explain the floating rules we need to examine the following examples:
A) 1 non-quick floating rule that blocks traffic to port 80 on interface wan
B) 1 normal (interface) rule that passes traffic to port 80 on interface wan- Floating rules get evaluated first from top to bottom
- If the floating rule is NOT quick, then proceed with further matching against other rules
- Interface rules get evaluated last from top to bottom
Going through the checklist above, tells us that a packet will first pass through the floating rule, where the rule does match, but is not the terminal match against it, so the packet will continue on until it reaches the terminal match, which is rule B. Although the floating rule says block, the packet will pass since that's the rule that matches it.
In the case that rule A is quick, the packet will immediately match that rule, which says block it, therefore it will be blocked.
-
We got it working ;D
Half of my normal sites are blocked now :P
( ;D )
(For example: www.geenstijl.nl, the links to the movies in there, the pfSense firewall says the new floating rules block them).
We = WIFE + me. WIFE, since I suffer from brain damage due to an accident, after which I can't concentrate on things suddenly. Combine that with the eternal noob status, and you will understand why WIFE, the love of my life, sometimes has to step in to help me out.
(The brain is a miraculous thing, a thing stupid economists like me will never understand. Because: not only do I have some, what some people might consider, 'advanced' post-academical degrees, but all that knowledge stays available to me. It is just that new knowledge doesn't seem to be allowed in. Perhaps I have a pfSense firewall in my brain now. It would make sense, because then nothing gets through ;D ;D ).
I forgot: Cino, thank you very much for your valuable contributions also: the list is getting longer ;D
Thank you & bye,
-
I've been setting up multiple pfsenses based on my recommendations in this topic and never had a problem with them blocking legitimate sites (aside from FPs now and then). In particular, a CARP cluster that provides connectivity to a small datacenter that houses web hosting servers, email servers and co-located servers, and internet access to remote clients, rarely gets any false positives. Those false positives are dealt with as soon as possible, which can be seen on the github list, since I took the enable all rules, then start removing rules as they are encountered approach, as recommended in this topic.
There is a reason those sites are blocked. Maybe it's a misconfiguration issue, a suricata false positive, or maybe they are not so legitimate after all.
-
@jflsakfja:
To explain the floating rules we need to examine the following examples:
A) 1 non-quick floating rule that blocks traffic to port 80 on interface wan
B) 1 normal (interface) rule that passes traffic to port 80 on interface wan- Floating rules get evaluated first from top to bottom
- If the floating rule is NOT quick, then proceed with further matching against other rules
- Interface rules get evaluated last from top to bottom
Going through the checklist above, tells us that a packet will first pass through the floating rule, where the rule does match, but is not the terminal match against it, so the packet will continue on until it reaches the terminal match, which is rule B. Although the floating rule says block, the packet will pass since that's the rule that matches it.
In the case that rule A is quick, the packet will immediately match that rule, which says block it, therefore it will be blocked.
One of my ever frustrating frustrations (yaah :P ) with the floating rules was that I never understood them. I think your explanation is very clear, so for the zillionth time: thank you, Jfl ;D
What remains is one little question: the source versus destination. You will have to set the rules twice, yes (?) because of the source versus destination difference: suppose you have multiple WAN and multiple LAN (as I do), in order to block connections from and to a specific IP you will have to set two floating rules: 1 for all WAN's where source is the aforementioned IP, and a second floating rule for all LAN's where the destination is the aforementioned IP.
I ask because in many posts (in this forum) as well as 'tutorials' on the internets it is said you only need one floating rule, and I never understood that since it depends on whether traffic is coming from the source or from the destination.
Brains: mysterious things ;D
