Using the same whitelist in pfB and Snort

pftdm007

Could not fetch the URL 'https://127.0.0.1:468/pfblockerng/pfblockerng.php?pfb=pfB_pass_IP_v4'.

Its looking more and more like a missing feature in pfsense! (minor one that is.....) :)

Perhaps @BBcan177 would have a suggestion to link those two together?

If there's no elegant way of doing this I would suggest to be able to use a system alias in pfB since what I'm trying to do is use the same list across multiple packages so a top-down approach makes more sense to me:

System Alias
|
|---> pfBlocker ipv4 list
|---> Snort whitelist

instead of

|---> System alias------*--->
|                           |
|---< pfBlocker ipv4 list   |
      Snort whitelist<------|

*somehow managing to transform an URL alias into something Snort can digest...

smolka_J

@pftdm007 said in Using the same whitelist in pfB and Snort:

pfB_pass_IP_v4

I believe all you should need to do is take your pfB alias name "pfB_pass_IP_v4" instead of the URL for it and just place that in Snort's Passlist tab by editing the current passlist thats being used, enter only the name of the alias in the "Assigned Alias" field as noted on https://docs.netgate.com/pfsense/en/latest/packages/snort/passlist.html. The pfB alias "pfB_pass_IP_v4" you created in pfBlockerNG is already an ALIAS itself, there's no need to make an entire duplicate of it nor the need to create a whole separate one on the system ALIAS tabs, that would only double the workload to obtain the same amount of info each cron/update. I do similar for my passlist on Suricata, one for IPs and one for domains, and works well as long as RAMDISK isn't being used otherwise it will fall out of sync after reboots until pfBlockerNG is reloaded and fill logs with residual alerts until it does. I prefer Suricata's means for creating passlists, it allows me to add an entire list of multiple ALIAS's to it rather than just a single ALIAS like Snort limits you to.

pftdm007

@smolka_J

I just tried to add the pfB aliases directly in Snort's pass list and saved.

I didnt see any errors anywhere (either in GUI or in the syslog).

However when I click "View List" in a snort interface for the pass list, I only get IP's from the checkboxes (Local networks, WAN, DNS, etc etc) and those under the alias "service_suppliers" (which is a simple system alias I use in FW rules) Nothing from the other aliases starting with "pfB_"...

I tried force updating its rules, didnt help.

Unless I am wrong, there are no tables for the Snort passlists... so impossible to see their content... (at least under "Diagnostics > tables")

SteveITS

@pftdm007 Did you restart Snort?

The pass list is read in at startup. It’s not a pf table.

pftdm007

@SteveITS

Hi,

Yes I did restart Snort on all interfaces, even rebooted pfsense completely and let it do its updates...

After a few days with the pfB aliases in the pass list its still not working.

Yesterday I even tried to convert the pfB lists to "Alias Native" and created FW rules manually using those aliases. AFAIK it works perfectly at the firewall level but snort cant use those....

Some screenshots of my config:
pfB lists

Rules on interface "USR"

System Aliases (the green circled are those I want to reuse in Snort)

Snort passlist

pftdm007

Just tried creating a system alias by manually copying and importing the lists of IP's from each pfB "pass_IP" list and then adding to snort passlist.

Upon refreshing the rules in Snort the IP's got added immediately, I didnt even have to restart snort on the interfaces..... I did anyways just to make sure they are indeed taken into consideration....

So to sum up: Snort doesnt even want anything to do with pfB at any level whatsoever... At least on my setup that is.

This sounds like a basic feature that is missing..... and seems to be something fundamental that most people would want to do. Am I wrong here? What do the devs think of that? Sounds pointless to me to whitelist stuff in pfB only to have snort come back around and interfere with everything....

Is suricata any better in that regard?

smolka_J

@pftdm007 A little hard to tell for definite certainty as I don't specifically see IP's populate in my passlist that are in my pfB deny alias' URL feeds or custom list fields that I have listed in mine. But, when I view the passlist on each interface it does list out each of my alias names I entered and my suricata.log for each interface also shows that they each loaded but does not include their individual IP's in the total count that's added either. I do not though receive alerts for any that are in my pfB deny alias' that are added nor have I come across any of their listed IP's or domains' IP's being noted on my Blocks tab so my assumption is that its processing what it should from them. It would be good though to be able to see and verify what Suricata/Snort is actually seeing in the alias lists it processes maybe similar to how pfBlockerNG can in its logs to help eliminate the guessing whether it is or not.

pftdm007

@smolka_J

Thanks for your reply!

By the look of it neither suricata can handle pfB's lists? Until devs weight in or this gets any more traction, I've decided to go "old school" and create a system alias and copy - paste ALL ip's from pfB lists.

Adding this simple alias in Snort's passlist it works perfectly. Its just annoying to have to maintain two sets of identical IP's by hand. Maybe I am wrong but to me it looks like a missing feature in pfSense (unless its a bug on my install of course...).

For those using pfSense in enterprise level (actual business applications, not SOHO like me), do you really manually mantain the whitelists across the system?? More curious than anything else....

smolka_J

@pftdm007 I do still kind of feel that it is working in Suricata even though viewable list data doesn't show the same between the different ALIAS types inside of Suricata. I believe so because before I had set up my passlist like this and if/when my pfB IPv4 domain alias isn't kept up to par with my pfBlockerNG DNSBL whitelist, then I was getting erroneous alerts/blocks in Suricata for things that were whitelisted only on the DNSBL side when using various streaming apps and when my son is all over Steam/Minecraft gaming servers, as long as I keep the alias on par with my DNSBL whitelist those alerts have all vanished ever since. My box will also throw up lots of Suricata alerts any time my pfBlockerNG is out of sync needing to be reloaded to reprocess those lists also, those alerts don't go away until pfBlockerNG finishes its force reload all process. So it does appear to be working as far as I can see but difficult to try to confirm in detail not being able to view what it sees in the alias', and thats not just for pfB alias' in my passlist, thats also including what it displays for one other non-pfB system alias that is filled with domain names as well thats part of my passlist.

SteveITS

@smolka_J I pulled up a Suricata list of ours. It is an alias that contains 3 other aliases. Interestingly it seems to have resolved several aliases to IPs but ends with an alias name:

(...)
::1/128
Suricata_Trusted_Hosts

I don't recall seeing that in the past. Also I am unaware of any complaints about the IPs in it being blocked.

@bmeeks ?

bmeeks

@SteveITS said in Using the same whitelist in pfB and Snort:

@smolka_J I pulled up a Suricata list of ours. It is an alias that contains 3 other aliases. Interestingly it seems to have resolved several aliases to IPs but ends with an alias name:

(...)
::1/128
Suricata_Trusted_Hosts

I don't recall seeing that in the past. Also I am unaware of any complaints about the IPs in it being blocked.

@bmeeks ?

Not sure what your question is exactly, but I assume you are maybe asking about the Alias in a Suricata pass list?

If so, Suricata has had the ability to use Aliases in Pass Lists for quite some time. The same ability was also added to Snort. The IDS/IPS packages cannot create or edit the Aliases, but if the Alias exists on the firewall and has IP addresses in the corresponding alias table in pf, then the IDS/IPS packages can use them. They work by making a system call asking pf if a particular IP address extracted from an alert exists in the list of IP addresses pf is maintaining for the specified Alias.

SteveITS

@bmeeks OP was saying pfBlocker created aliases don't work at all in Snort. (we don't use Snort and have not tried this in Suricata)

I observed that in Suricata some aliases are resolved into IPs but not "Suricata_Trusted_Hosts". Screen cap:

and the list itself:

...where in particular "VPN_ITS" is the 192.168.x.x IPs shown above.

Suricata_Trusted_Hosts is a firewall alias containing several other aliases and also individual IPs. Those are not visible using View List.

In a quick look I do not see any errors in Suricata's log.

We've used this alias method for quite a long time but I do not recall the alias name showing under View List before.

Edit: I'm aware this is a different question than OP. :)

bmeeks

@SteveITS: pfBlockerNG aliases would not necessarily be known to Snort or Suricata.

But firewall aliases in general can be used so long as they exist by name in pf. You can check this under DIAGNOSTICS > TABLES. If the Alias is listed there and is populated with IP addresses, then the IDS/IPS packages can successfully use them.

As I mentioned earlier. all the IDS/IPS binary does is make a FreeBSD system call into the libpfctl library to query for an alias by that name and then ask if the passed IP is present in the corresponding pf table entry. The "passed IP" is simply one or both of the source and destination IP addresses from the packet being analyzed by the IDS/IPS (depending on whether the blocking is configured for SRC, DST, or BOTH).

If you carefully examine the suricata.log file you should see lines there showing that it processed the alias. If it has an issue with the alias, it will log a message and skip attempting to load that alias into the internal pass list structure in RAM.

bmeeks

Since I was tagged late in this thread, I replied directly to the user that tagged me without reading the previous posts in this thread.

To help clarify things, any host or network Alias that appears under DIAGNOSTICS > TABLES in the pfSense firewall and that is populated with IP addresses can be used in a Snort pass list. Suricata expands the list of allowable alias types to include "url table" in addition to host and network.

Aliases can only be used as part of a Pass List which is associated with Legacy Blocking Mode. You cannot use firewall aliases in any rules because the underlying binary is ignorant of pfSense aliases. The only reason they work in Pass Lists is because I wrote the custom plugin that implements the legacy blocking mode, and I wrote it to recognize and work with pfSense aliases as valid pass list IP addresses so long as the conditions outlined previously are met.

smolka_J

@bmeeks So as I was thinking this should confirm all of my pfBlockerNG "IP deny" and "alias native" alias names that I have added in my pass list are processing as desired, only ignored IPs I have are dups of those that are auto added with the check boxes:

passlist processed: Total entries parsed: 33, IP addresses/netblocks/aliases added to No Block list: 27, IP addresses/netblocks ignored because they were covered by existing entries: 6

I used to have a couple whole additional "alias native" lists setup for it at one point but it was just excess double loading since any pfBlockerNG IP/URL alias names should work the same not just ones that are "alias native" using just the IP data from them either way so I deleted them a while back to save writes

bmeeks

@smolka_J said in Using the same whitelist in pfB and Snort:

@bmeeks So as I was thinking this should confirm all of my pfBlockerNG "IP deny" and "alias native" alias names that I have added in my pass list are processing as desired, only ignored IPs I have are dups of those that are auto added with the check boxes:
passlist processed: Total entries parsed: 33, IP addresses/netblocks/aliases added to No Block list: 27, IP addresses/netblocks ignored because they were covered by existing entries: 6
I used to have a couple whole additional "alias native" lists setup for it at one point but it was just excess double loading since any pfBlockerNG IP/URL alias names should work the same not just ones that are "alias native" using just the IP data from them either way so I deleted them a while back to save writes

Yes, the summary info is telling you what it found in the supplied text-based pass list and the results of processing that list.

In Suricata you can enable a Pass List debugging feature on the INTERFACE SETTINGS tab. That will print a lot more debugging information and details about pass list processing, but it will definitely slow down packet processing, so it's not something to leave enabled.

SteveITS

@bmeeks None of the above 3 alias names in the pass list appear in the log file and I don't see any errors.

I suppose it would be "more correct" to put the latter two aliases into the Suricata_Trusted_Hosts alias since that was its purpose. I did for one but the other is a Network alias so does not work if I put it inside the Suricata_Trusted_Hosts alias (pfSense doesn't offer it in autocomplete). Which I think is why it was listed separately, IIRC now that I've written this paragraph.

If I do that, and restart Suricata, then View List still lists "Suricata_Trusted_Hosts" as a string, as above. The VPN IPs are no longer separately listed.

To be clear this Suricata_Trusted_Hosts is a pfSense firewall alias which contains other aliases and a handful of IPs.

I'm not aware of hosts in Suricata_Trusted_Hosts being blocked so I assume it's working anyway...?

bmeeks

@SteveITS said in Using the same whitelist in pfB and Snort:

I'm not aware of hosts in Suricata_Trusted_Hosts being blocked so I assume it's working anyway...?

That's the bottom line. If the hosts you do not want to get blocked are not getting blocked, then all is good.

I don't recall specifically testing with nested aliases back when I wrote the new alias functionality into the custom blocking plugin. I was mainly going after FQDNs (fully qualified domain names) at the time.

But the plugin is not digging into the alias to resolve it. It simply looks in the same pf tables that are listed under DIAGNOSTICS > TABLES. If the alias is there and is populated, then the plugin can test for IP addresses in the alias. If the alias is not listed under DIAGNOSTICS > TABLES, then Suricata is not using it even though it may show up in the View List dialog when viewing a Pass List.

I built a sort of fail-safe error handling feature into the custom code so that it will silently ignore an alias that is not found during run time. The operating assumption there is the admin might have removed the alias and I didn't want the running Suricata process to abort if that happened.