Snort broken: whitelist
-
I don't understand why it is even part of the GUI if the backend hasn't worked for numerous releases….
It amazes me that this snort package isn't classified as BETA.
-
I agree. In my view its a core functionality of a FW and it doesnt work as expected.
Simply not good enough.
-
Guys:
I've sent an e-mail to the main developer asking for some help in looking over the third-party plugin for pfSense that does the actual blocking from Snort. I'm wondering if that plugin might not have some issues, especially with virtual IPs.
I have not see this error in my personal setup, but admittedly my personal firewall is rather simple with a single WAN IP and no virtual IPs or failovers.
One thing to be aware of, when you edit the whitelist tab, be sure to manually stop and restart Snort after clicking SAVE to be sure it "sees" your edits.
Bill
-
Still seems to be broken:
2.0.3-RELEASE (amd64)
Snort 2.9.4.1 pkg v. 2.5.7It looks like the whitelist is not being populated to my
/usr/local/etc/snort/snort_3663_em0/whitelist01
file, the file only contains the checkboxed items.
-
Still seems to be broken:
2.0.3-RELEASE (amd64)
Snort 2.9.4.1 pkg v. 2.5.7It looks like the whitelist is not being populated to my
/usr/local/etc/snort/snort_3663_em0/whitelist01
file, the file only contains the checkboxed items.Just sent you a PM earlier about this, but looking at your attached screenshot triggered another thought about what may be wrong. Are any of your custom IP addresses by chance actually expressed as CIDR networks? There is a shortcoming in the 2.5.7 and earlier Snort package versions where the Whitelist building code explicitly excluded CIDR-expressed networks. This was because the old Spoink plugin could not cope with CIDR. However, Ermal fixed that last Summer so that Spoink can work with individual IPs or CIDR networks. However, the old restriction in the Snort GUI package did not get updated.
I have that fixed now in Package version 2.5.8 that I am testing now. This may be what your problem is.
Bill
-
The alias is of type Networks and its entries are 2 FQDNs which resolve to 2 single IPs.
-
The alias is of type Networks and its entries are 2 FQDNs which resolve to 2 single IPs.
OK. See the latest PM from me with my e-mail address. Send me either a screenshot of the Alias or some details about it so I can try and recreate the issue on my test machines. I want to get to the bottom of it and fix all these darn whitelist and HOME_NET issues in the Snort package.
Bill
-
Sent!
I'm looking at the whitelist file on the other pfSense machine as the whitelist is bigger and complied of nested aliases.
I have noticed that the CIDR aliases are being populated into the whitelist file, but the aliases that use FQDNs are not being populated into the whitelist file.The solution is getting complicated… Its easy to put in static ips and ranges into a file, but when you have domain names with dynamic IPs the file needs to be continually updated when the DNS changes.....
Perhaps instead of snort using its own file and populating it with IPs, the file is populated with pointers to the aliases already in use by snort.... this would help from reinventing the wheel because I assume snort automatically updates the IPs of FQDNs in their alias system...
-
The solution is getting complicated… Its easy to put in static ips and ranges into a file, but when you have domain names with dynamic IPs the file needs to be continually updated when the DNS changes.....
Perhaps instead of snort using its own file and populating it with IPs, the file is populated with pointers to the aliases already in use by snort.... this would help from reinventing the wheel because I assume snort automatically updates the IPs of FQDNs in their alias system...
The Snort package does not currently dynamically resolve IPs from hostnames. The overhead associated with that is just too much to scale with real time blocking. The issue with that is potential "thread blocking" while waiting for DNS to come back. Right now the Spoink plugin (that does the actual blocking) is a single-threaded process. It should be possible to have Snort do a DNS lookup to resolve FQDNs upon initial whitelist creation and to update it whenever Snort restarts. However, I don't see it as practical to have the Spoink plugin do DNS lookups realtime (at least not for now).
I think I can modify the whitelist logic to handle resolving FQDNs to IPs during list creation in Snort, but then the list is static until the next time Snort is restarted.
Bill
-
Would a restart include a Rules update?
At 5h55 an IP is blocked. At 6h00 the Rules and IPs are updated, and the dynamic whitelist IP changes to the newly blocked IP, does that block get removed?
-
Would a restart include a Rules update?
At 5h55 an IP is blocked. At 6h00 the Rules and IPs are updated, and the dynamic whitelist IP changes to the newly blocked IP, does that block get removed?
In looking over some pfSense documentation, I found that when you use FQDNs in an Alias, the "value" of the FQDN (it's IP address, for instance) is updated once every 5 minutes.
Snort currently operates somewhat outside that process. It works by using Spoink to directly insert a "blocked IP" into the Packet Filter table of the firewall. Before inserting the IP into the blocked table, it checks to see if it is in the current whitelist held in Spoink memory. Spoink only reads the whitelist file upon a startup of Snort, and then stores the list in memory. It does not read it again on any periodic basis. So think of the whitelist as more or less "static".
As for timing out blocks, that is done using a cron job created by the Snort packge. This job runs every 5 mins and removes IP addresses that have had no activity during the last "block time interval" that is set in the GUI. The suggested setting is 1 hour, so if an IP has been in the snort2c table for more than hour with no activity within the last hour, then it is removed by the cron job using a pfctl call.
Right now the Spoink plugin is not hooked into the firewall's 5-minute update process. I don't know enough about the internals of pfSense to make a judgement on how feasible that might be.
Bill
-
The logic for Spoink seems to be OK and not require any modification.
Looking at the rest, it seems there should be 2 independent parallel processes.
1. The timeout process which already works very well at user specified frequencies
2. The whitelist process which should
a. Update the whitelist file and memory
b. Remove whitelisted IPs from the blocklistThe frequency for 2a and 2b, I think, should be on a seperate timer than 1. Ideally it should be tied to the 5 minute pfSense timer for updating alias IPs.
I don't know how all the functions interact with each other, hopefully Spoink and the timeout processes don't need to be touched. Maybe what needs to be modified is the whitelist process to include 2a and 2b, migrating it from a static whitelist to a dynamic whitelist with the added function of removing IPs from the banned list.
Currently, with the whitelist only being read into memory once, when snort is started, I can see the following problems occuring:
User starts Snort with whitelist.
Snort whitelist has the standard checkboxes (WAN IP, WAN gateway, DNS server, Virtual IP, VPN)
1. User creates IPSEC tunnel (this new tunnel is not in the whitelist)
2. User's ISP sends a new DHCP lease, with a new IP, new Gateway, new DNS servers (none of these will be in the whitelist) -
It looks like the Snort plugin is relying on a semi broken pfSense Alias function as described here:
http://forum.pfsense.org/index.php/topic,61264.0.html -
It looks like the Snort plugin is relying on a semi broken pfSense Alias function as described here:
http://forum.pfsense.org/index.php/topic,61264.0.htmlYep, the Snort package simply uses the filter_expand_alias() function call in pfSense to turn aliases into IP addresses or networks. So if that function returns inconsistent results, then Snort will suffer the same fate. On the other hand, the Snort package is not currently equipped to handle dynamic aliases. So currently the problem of semi-broken dynamic aliases is moot for Snort.
I can see where such functionality would obviously be quite useful, but integrating it into Snort will take some work.
Bill
-
The logic for Spoink seems to be OK and not require any modification.
Looking at the rest, it seems there should be 2 independent parallel processes.
1. The timeout process which already works very well at user specified frequencies
2. The whitelist process which should
a. Update the whitelist file and memory
b. Remove whitelisted IPs from the blocklistThe frequency for 2a and 2b, I think, should be on a seperate timer than 1. Ideally it should be tied to the 5 minute pfSense timer for updating alias IPs.
I don't know how all the functions interact with each other, hopefully Spoink and the timeout processes don't need to be touched. Maybe what needs to be modified is the whitelist process to include 2a and 2b, migrating it from a static whitelist to a dynamic whitelist with the added function of removing IPs from the banned list.
Currently, with the whitelist only being read into memory once, when snort is started, I can see the following problems occuring:
User starts Snort with whitelist.
Snort whitelist has the standard checkboxes (WAN IP, WAN gateway, DNS server, Virtual IP, VPN)
1. User creates IPSEC tunnel (this new tunnel is not in the whitelist)
2. User's ISP sends a new DHCP lease, with a new IP, new Gateway, new DNS servers (none of these will be in the whitelist)Just musing off the top of my head about this, it might be possible to make the Snort 5-minute cron job a little smarter so that it could perhaps do task 2b from your list. Task 2a would more than likely have to be baked into the Spoink plugin code. I'm not 100% sure at this point how feasible that is, though.
Bill
-
The concern I have with baking in 2a with spoink is that it would only be triggered when a new blocked host is added. Putting 2a on its own independent timer I think would be more effective.
-
Currently, with the whitelist only being read into memory once, when snort is started, I can see the following problems occuring:
User starts Snort with whitelist.
Snort whitelist has the standard checkboxes (WAN IP, WAN gateway, DNS server, Virtual IP, VPN)
1. User creates IPSEC tunnel (this new tunnel is not in the whitelist)
2. User's ISP sends a new DHCP lease, with a new IP, new Gateway, new DNS servers (none of these will be in the whitelist)Looking at the above scenario, one could also argue that this may be expecting a bit much from Snort itself. In the above scenario, what would likely trigger a Snort "block"? Would it be a false positive or a real threat? If a FP (false positive), maybe tuning of Snort is the cure. If it's a real threat, then do you really want to whitelist that? Have you experienced something like this scenario in the real world, and if so, can you provide me some information to help me understand better?
Bill
-
Currently, with the whitelist only being read into memory once, when snort is started, I can see the following problems occuring:
User starts Snort with whitelist.
Snort whitelist has the standard checkboxes (WAN IP, WAN gateway, DNS server, Virtual IP, VPN)
1. User creates IPSEC tunnel (this new tunnel is not in the whitelist)
2. User's ISP sends a new DHCP lease, with a new IP, new Gateway, new DNS servers (none of these will be in the whitelist)Looking at the above scenario, one could also argue that this may be expecting a bit much from Snort itself. In the above scenario, what would likely trigger a Snort "block"? Would it be a false positive or a real threat? If a FP (false positive), maybe tuning of Snort is the cure. If it's a real threat, then do you really want to whitelist that? Have you experienced something like this scenario in the real world, and if so, can you provide me some information to help me understand better?
Bill
The only problem I've experienced with the Snort whitelist is it does not work with Aliases containing FQDNs.
After I understood how and when the whitelist is populated, those 2 additional problems immediately came to mind.
If a user selects these options in the whitelist I would think they would expect the whitelist to remain updated after they create new VPN tunnels (common task for admins) or when their ISP DHCP lease expires (common for a lot of broadband connections) -
The only problem I've experienced with the Snort whitelist is it does not work with Aliases containing FQDNs.
After I understood how and when the whitelist is populated, those 2 additional problems immediately came to mind.
If a user selects these options in the whitelist I would think they would expect the whitelist to remain updated after they create new VPN tunnels (common task for admins) or when their ISP DHCP lease expires (common for a lot of broadband connections)I agree those two examples make sense. I will research some potential solutions, but this is not something that is likely to appear in a release in the near future. There's quite a bit of work to do and some logic to rework to pull this off in the Snort module.
One long-term goal of mine was to work on resurrecting the old Snort-Dev package and letting it be the test bed for bleeding-edge updates. Something like this dynamic-aliases whitelisting would be something suited to a Snort-Dev package.
Bill
-
Well a new ip and gateway should trigger a snort reload and the new ips will be inserted in the whitelist and snort should read them up.
The difficulty is that you ask something that snort is not able to do, dynamicity.While everything can be programmed there are priorities to be set and i do not think this will be in the top list, especially since its not an easy change.
