Suricata pass list ignored
-
@j-koopmann said in Suricata pass list ignored:
@bmeeks I guess I am just "oversimplifying" the task to match an IP address to a white list. It was my expectation that most likely a dozen of libraries exist that should be able to do this. I agree that once a crazy amount of pass-list entries come into play, this will make a difference.
Well, you could say I did use a ready-made library to do this. I am using the code already existing in the Suricata binary. I saw no need to bulk up my plugin module by either creating the Pass List logic from scratch, or importing yet another third-party library into the mix. And like I said, this code has worked. And every time I have tested it, it continues to work. For that other user I mentioned (which was early last year, if I am recalling correctly), I was never able to reproduce his issue either. I did try some things differently in examining the returned value from the built-in function call, but since I could not reproduce his issue I really couldn't test my theory. I built the altered code anyway and he tested; and if I recall, it sometimes worked for him and other times did not. English was not his primary language so there was some difficulty in communicating with each other through the translations. But it worked everytime for me. He was a user with a ton of network subnets defined on his Pass List (like maybe a hundred as I recall). He sent me his exact Pass List, and I pasted it into a list in a virtual machine, and then configured Kali Linux to spoof IP addresses on his pass list while scanning the WAN interface of my Kali machine. I would get alerts and no blocks (as desired when the IPs were on the Pass List). Removing the networks from the Pass List resulted in blocks in the virtual machine.
If the Pass List code was fundamentally flawed, I would expect to see dozens and dozens of posts here about it. After all, there are over 24,000 installations of the Snort and Suricata packages around the world. This is how many unique hits the Snort folks told me they see daily on the rules update servers where the
curluser agent identifies the client as pfSense.I am quite busy at the moment on another totally unrelated project, so it will be a while before I can look into this. But I will. In the meantime, you are welcome to dive in using the code link I provided. You should be able, if you have a FreeBSD-11.3/STABLE machine with the standard build packages in place, to create a
pkg*.tgz file and then copy it over to pfSense and install it for testing. The key is having the matching version of the pfSense underlying OS. In this case that is FreeBSD-11.3/STABLE. -
I encountered the same and just rebooted the box. Problem solved.
-
@cool_corona said in Suricata pass list ignored:
I encountered the same and just rebooted the box. Problem solved.
If a reboot solved the problem, that would point to you having a duplicate process on an interface. In that instance, the duplicate process will not see nor respond to any Pass List changes. That was a common problem in the past, but more rare now. However, if anyone uses Service Watchdog with the IDS/IPS packages, getting duplicate processes can happen. So can frequent up/down cycles of an interface (especially the WAN) which cause the pfSense subsystem to issue a series of "restart all packages" commands in quick succession.
-
@bmeeks after not running into this for months it sort of slipped my mind. Now my VPN Client connection was dropped several times and during debugging I found out I was hit once more by this. The /24 of the client network is in the PASS_LIST but was ignored.
I updated to pfsense 2.5.2 and Suricata 6.0.0_10. Have you had any chance whatsoever to look into this. Reboot does NOT solve this problem.And I just rechecked: Only one process. The entry in the pass list is there for months and several reboots had been performed.
-
@j-koopmann said in Suricata pass list ignored:
@bmeeks after not running into this for months it sort of slipped my mind. Now my VPN Client connection was dropped several times and during debugging I found out I was hit once more by this. The /24 of the client network is in the PASS_LIST but was ignored.
I updated to pfsense 2.5.2 and Suricata 6.0.0_10. Have you had any chance whatsoever to look into this. Reboot does NOT solve this problem.And I just rechecked: Only one process. The entry in the pass list is there for months and several reboots had been performed.
I have tried and tried to reproduce this issue in my test virtual machines for Suricata over the years. I have NEVER had a pass list entry not work properly in all of my testing.
Not saying you aren't seeing the problem, but if I can't reproduce it, I can't troubleshoot or fix it. I have even had other users send me their exact pass lists, which I pasted into my test VMs, and still was unable to reproduce the issue.
-
@bmeeks i appreciate this.
How can I help? I should be able to reproduce this somehow but how would that help you?
-
@j-koopmann said in Suricata pass list ignored:
@bmeeks i appreciate this.
How can I help? I should be able to reproduce this somehow but how would that help you?
Unfortunately it likely can't, because what would be needed is a full debug build of the Suricata package, a copy of the binary source code, and then loading it all up into the
gdbdebugger and stepping through the code to see where the problem is happening.You would not want all the
gdbdebugging stuff and associated libraries installed on a production firewall (or I would not want that).The Pass List feature borrows functionality from an existing shared resource in the Suricata binary -- a Radix tree. That tree is used to store, and later search for, the IP networks and/or individual host IPs pulled from the Pass List text file.
When an alert is logged, the custom blocking plugin pulls out the IP addresses and then compares them to the Pass List. It does that by searching the Radix tree to see if the IP from the alert is "covered" by an existing Radix tree entry. If not, the IP is added to the blocked table. If the Radix tree search indicates the IP is "covered" by an existing entry, the IP is not added to the blocked table. If the Radix tree code was faulty, it would be most logical for it to fail for every lookup -- not just one randomly. The only other possibility I can imagine is perhaps it is a threading problem that crops up randomly due to Suricata's multithreaded nature. That might be why I have never been able to duplicate the issue. My test VM traffic load and number of threads may not match yours.
-
@bmeeks What about putting some debug outputs around this in one of the future versions? We could then see if the radix tree match is unsuccessful or if it is working and it is a threading problem.
-
@j-koopmann said in Suricata pass list ignored:
@bmeeks What about putting some debug outputs around this in one of the future versions? We could then see if the radix tree match is unsuccessful or if it is working and it is a threading problem.
That is an idea for the future. I am hopeful the move to
iflibin FreeBSD 12, and some coming improvements to netmap support in Suricata, will lead to the custom Legacy Blocking Mode module used on pfSense being abandoned. It is not an ideal solution. It's too big of a hammer to block all traffic from a host because of a single alert. Better to use the Inline IPS Mode and just selectively drop bad packets instead of completely blocking the host IP.Before the move to the
iflibwrapper for NIC drivers in FreeBSD 12, your particular NIC hardware driver had to be patched to support netmap operation (and thus support inline IPS mode in Suricata). That limited the configurations where you could use netmap. Now,iflibwraps the netmap support up natively in FreeBSD and relieves the NIC driver from having to worry about it. There are perhaps still a few rough patches withifliband netmap, but those should get smoothed out in future FreeBSD updates. But over time I'm hopeful that netmap and Inline IPS Mode will become how you "block" in Suricata. And the Legacy Blocking Mode and its custom module will disappear. -
Here is the Redmine link:
https://redmine.pfsense.org/issues/12899 -
This post is deleted! -
@wexi Not sure it is the same problem. The pass list IS being obeyed. It is just that it seems to only handle IP addresses and not ranges (or at least in some circumstances). I fail to see how your post is related.
Are you having the range problem as well?
