Multiple pfsenses and Snort updates?
-
Same OINK code on all box'es. How to get them to update the rules 15mins apart?
-
For now you have to change /etc/crontab by hand (with package cron or through terminal).
-
Think we can get Bill to implement a specific time possibility in the GUI for this?
That would make enterprise scenarios a lot simpler.
-
Shouldn't be much trouble to add a mechanism in the GUI to allow selection of the update time. Can put that on the TO-DO list.
Bill
-
I owe some beers mate!! Thanks is not sufficient anymore!
-
I owe some beers mate!! Thanks is not sufficient anymore!
I thought about this some more and looked at the code changes. I think this 15-minute "cooling off period" for Snort VRT update attempts only impacts firewalls using the exact same source IP for the update. I would anticipate your firewalls would each have their own WAN IP, and thus appear to the Snort.org site to be different entities. Is this not correct? I know in VM testing during my code development work I have never seen an update fail for this reason; and I've forced Snort VRT updates many times during an hour of testing.
While some customization might be doable, the way the code currently works makes it harder than I initially thought it would be. The code now assumes everything starts at midnight (00:00) and then counts hours or days from there. It also hard-codes the update start time to be 3 minutes past the top-of-the-hour. So, for example, if you have updates set for every 12 hours the jobs run at 00:03 and 12:03 each day. It would be possible to make the minutes portion of the job schedule customizable, but I'm beginning to wonder if that would really help much since you would need to at least offset them by 15 minutes each. So that gives you maybe 3 firewalls in an hour ???
If this is a really big deal for you and others, then I can continue to work on a solution.
Bill
-
It is not a problem for me, but in another thread I reported that changing the update interval does not change anything in crontab. It is not a high priority, but can you put it on the "todo" list?
-
No worries mate.
They are running on different WAN ip segments and one runs and the other one doesnt.
Box 1 NOT running:
Apr 22 00:07:45 php: : The Rules update has finished.
Apr 22 00:07:45 php: : Snort has restarted with your new set of rules…
Apr 22 00:07:43 SnortStartup[54855]: Snort SOFT START For Internet(9626_em0)…
Apr 22 00:07:42 snort[34320]: Could not remove pid file /var/run/snort_em09626.pid: No such file or directory
Apr 22 00:07:42 snort[34320]: Could not remove pid file /var/run/snort_em09626.pid: No such file or directory
Apr 22 00:07:42 kernel: em0: promiscuous mode disabled
Apr 22 00:07:42 snort[34320]: *** Caught Term-Signal
Apr 22 00:07:42 snort[34320]: *** Caught Term-Signal
Apr 22 00:07:41 SnortStartup[51828]: Snort STOP For Internet(9626_em0)…
Apr 22 00:07:40 php: : Building new sig-msg.map file for WAN...
Apr 22 00:07:38 php: : Resolving and auto-enabling any flowbit-required rules for WAN...
Apr 22 00:07:36 php: : Updating rules configuration for: WAN ...
Apr 22 00:07:36 php: : Emerging Threat rules are up to date...
Apr 22 00:07:36 php: : Snort GPLv2 Community Rules file update downloaded succsesfully
Apr 22 00:07:35 php: : There is a new set of Snort GPLv2 Community Rules posted. Downloading...
Apr 22 00:07:34 php: : Failed Rules Filesize: 0
Apr 22 00:07:34 php: : Snort VRT rules file download failed...
Apr 22 00:07:34 php: : Snort Rules Attempts: 5
Apr 22 00:03:19 php: : There is a new set of Snort VRT rules posted. Downloading...
Apr 22 00:03:19 php: : Snort MD5 Attempts: 1Box 2 RUNNING
Apr 22 00:11:13 php: : The Rules update has finished.
Apr 22 00:11:13 php: : Snort has restarted with your new set of rules...
Apr 22 00:11:11 kernel: em0: promiscuous mode enabled
Apr 22 00:11:11 SnortStartup[41126]: Snort START For Internet(36256_em0)…
Apr 22 00:09:31 snort[48220]: Could not remove pid file /var/run/snort_em036256.pid: No such file or directory
Apr 22 00:09:31 snort[48220]: Could not remove pid file /var/run/snort_em036256.pid: No such file or directory
Apr 22 00:09:31 kernel: em0: promiscuous mode disabled
Apr 22 00:09:31 snort[48220]: *** Caught Term-Signal
Apr 22 00:09:31 snort[48220]: *** Caught Term-Signal
Apr 22 00:09:30 SnortStartup[42749]: Snort STOP For Internet(36256_em0)…
Apr 22 00:09:29 php: : Building new sig-msg.map file for WAN...
Apr 22 00:09:27 php: : Resolving and auto-enabling any flowbit-required rules for WAN...
Apr 22 00:09:25 php: : Updating rules configuration for: WAN ...
Apr 22 00:09:24 php: : EmergingThreats rules file update downloaded succsesfully
Apr 22 00:09:22 php: : There is a new set of EmergingThreats rules posted. Downloading...
Apr 22 00:09:21 php: : Snort GPLv2 Community Rules file update downloaded succsesfully
Apr 22 00:09:20 php: : There is a new set of Snort GPLv2 Community Rules posted. Downloading...
Apr 22 00:09:19 php: : Failed Rules Filesize: 0
Apr 22 00:09:19 php: : Snort VRT rules file download failed...
Apr 22 00:09:19 php: : Snort Rules Attempts: 5
Apr 22 00:06:47 php: : There is a new set of Snort VRT rules posted. Downloading...
Apr 22 00:06:47 php: : Snort MD5 Attempts: 3I owe some beers mate!! Thanks is not sufficient anymore!
I thought about this some more and looked at the code changes. I think this 15-minute "cooling off period" for Snort VRT update attempts only impacts firewalls using the exact same source IP for the update. I would anticipate your firewalls would each have their own WAN IP, and thus appear to the Snort.org site to be different entities. Is this not correct? I know in VM testing during my code development work I have never seen an update fail for this reason; and I've forced Snort VRT updates many times during an hour of testing.
While some customization might be doable, the way the code currently works makes it harder than I initially thought it would be. The code now assumes everything starts at midnight (00:00) and then counts hours or days from there. It also hard-codes the update start time to be 3 minutes past the top-of-the-hour. So, for example, if you have updates set for every 12 hours the jobs run at 00:03 and 12:03 each day. It would be possible to make the minutes portion of the job schedule customizable, but I'm beginning to wonder if that would really help much since you would need to at least offset them by 15 minutes each. So that gives you maybe 3 firewalls in an hour ???
If this is a really big deal for you and others, then I can continue to work on a solution.
Bill
-
No worries mate.
They are running on different WAN ip segments and one runs and the other one doesnt.
Box 1 NOT running:
Apr 22 00:07:45 php: : The Rules update has finished.
Apr 22 00:07:45 php: : Snort has restarted with your new set of rules…
Apr 22 00:07:43 SnortStartup[54855]: Snort SOFT START For Internet(9626_em0)…
Apr 22 00:07:42 snort[34320]: Could not remove pid file /var/run/snort_em09626.pid: No such file or directory
Apr 22 00:07:42 snort[34320]: Could not remove pid file /var/run/snort_em09626.pid: No such file or directory
Apr 22 00:07:42 kernel: em0: promiscuous mode disabled
Apr 22 00:07:42 snort[34320]: *** Caught Term-Signal
Apr 22 00:07:42 snort[34320]: *** Caught Term-Signal
Apr 22 00:07:41 SnortStartup[51828]: Snort STOP For Internet(9626_em0)…
Apr 22 00:07:40 php: : Building new sig-msg.map file for WAN...
Apr 22 00:07:38 php: : Resolving and auto-enabling any flowbit-required rules for WAN...
Apr 22 00:07:36 php: : Updating rules configuration for: WAN ...
Apr 22 00:07:36 php: : Emerging Threat rules are up to date...
Apr 22 00:07:36 php: : Snort GPLv2 Community Rules file update downloaded succsesfully
Apr 22 00:07:35 php: : There is a new set of Snort GPLv2 Community Rules posted. Downloading...
Apr 22 00:07:34 php: : Failed Rules Filesize: 0
Apr 22 00:07:34 php: : Snort VRT rules file download failed...
Apr 22 00:07:34 php: : Snort Rules Attempts: 5
Apr 22 00:03:19 php: : There is a new set of Snort VRT rules posted. Downloading...
Apr 22 00:03:19 php: : Snort MD5 Attempts: 1Box 2 RUNNING
Apr 22 00:11:13 php: : The Rules update has finished.
Apr 22 00:11:13 php: : Snort has restarted with your new set of rules...
Apr 22 00:11:11 kernel: em0: promiscuous mode enabled
Apr 22 00:11:11 SnortStartup[41126]: Snort START For Internet(36256_em0)…
Apr 22 00:09:31 snort[48220]: Could not remove pid file /var/run/snort_em036256.pid: No such file or directory
Apr 22 00:09:31 snort[48220]: Could not remove pid file /var/run/snort_em036256.pid: No such file or directory
Apr 22 00:09:31 kernel: em0: promiscuous mode disabled
Apr 22 00:09:31 snort[48220]: *** Caught Term-Signal
Apr 22 00:09:31 snort[48220]: *** Caught Term-Signal
Apr 22 00:09:30 SnortStartup[42749]: Snort STOP For Internet(36256_em0)…
Apr 22 00:09:29 php: : Building new sig-msg.map file for WAN...
Apr 22 00:09:27 php: : Resolving and auto-enabling any flowbit-required rules for WAN...
Apr 22 00:09:25 php: : Updating rules configuration for: WAN ...
Apr 22 00:09:24 php: : EmergingThreats rules file update downloaded succsesfully
Apr 22 00:09:22 php: : There is a new set of EmergingThreats rules posted. Downloading...
Apr 22 00:09:21 php: : Snort GPLv2 Community Rules file update downloaded succsesfully
Apr 22 00:09:20 php: : There is a new set of Snort GPLv2 Community Rules posted. Downloading...
Apr 22 00:09:19 php: : Failed Rules Filesize: 0
Apr 22 00:09:19 php: : Snort VRT rules file download failed...
Apr 22 00:09:19 php: : Snort Rules Attempts: 5
Apr 22 00:06:47 php: : There is a new set of Snort VRT rules posted. Downloading...
Apr 22 00:06:47 php: : Snort MD5 Attempts: 3Just out of curiosity, can you download the Snort VRT rules from Box #1 using curl from the command line? Those "Attempts" messages are intriguing. They indicate the box is having some difficulty reliably grabbing the files from the Snort.org site.
A few upgrades back I put a loop and counter in the rules download code to make multiple attempts at Snort.org. This was done because on my own box I would see the Snort VRT download frequently fail, but also frequently suceed with no apparent rhyme or reason to it. So the code now makes up to 5 attempts, waiting 30 seconds between tries) before giving up. It keeps count of how many times it had to try. Notice that Box #1 got the MD5 file on the first try, but failed after 5 times getting teh actual Rules tarball. Box #2 had to try the MD5 file 3 times before it grabbed it, and it also failed after 5 attempts at downloading the Rules tarball.
Do both of these boxes sometimes successfully download the rules, but then occasionally fail? Or does Box #1 fail all the time?
Bill
-
This is what I got today when upgrading Snort.
It seems that it only needed 1 attempt..
Apr 22 13:16:34 SnortStartup[532]: Snort START For Internet(9626_em0)…
Apr 22 13:14:46 SnortStartup[33113]: Snort STOP For Internet(9626_em0)…
Apr 22 13:14:31 php: /snort/snort_download_rules.php: The Rules update has finished.
Apr 22 13:14:31 php: /snort/snort_download_rules.php: Emerging Threat rules are up to date...
Apr 22 13:14:31 php: /snort/snort_download_rules.php: Snort GPLv2 Community Rules are up to date...
Apr 22 13:14:30 php: /snort/snort_download_rules.php: Snort VRT rules are up to date...
Apr 22 13:14:30 php: /snort/snort_download_rules.php: Snort MD5 Attempts: 1
Apr 22 12:46:22 check_reload_status: Reloading filter
Apr 22 12:46:21 check_reload_status: Syncing firewall
Apr 22 12:46:20 php: /pkg_mgr_install.php: Building new sig-msg.map file for WAN...
Apr 22 12:46:18 php: /pkg_mgr_install.php: Resolving and auto-enabling any flowbit-required rules for WAN...
Apr 22 12:46:16 php: /pkg_mgr_install.php: Updating rules configuration for: WAN ...
Apr 22 12:46:16 php: /pkg_mgr_install.php: The Rules update has finished.
Apr 22 12:46:08 php: /pkg_mgr_install.php: EmergingThreats rules file update downloaded succsesfully
Apr 22 12:46:06 php: /pkg_mgr_install.php: There is a new set of EmergingThreats rules posted. Downloading...
Apr 22 12:46:05 php: /pkg_mgr_install.php: Snort GPLv2 Community Rules file update downloaded succsesfully
Apr 22 12:46:04 php: /pkg_mgr_install.php: There is a new set of Snort GPLv2 Community Rules posted. Downloading...
Apr 22 12:46:03 php: /pkg_mgr_install.php: Snort Rules Attempts: 1 -
It is not a problem for me, but in another thread I reported that changing the update interval does not change anything in crontab. It is not a high priority, but can you put it on the "todo" list?
Yep, I can take a look at that. I did a cursory investigation this weekend and it appears the intervals should update, but I did not get a chance to acutally test changing one. Will do that this week. I have not learned too much about that part of the code. Coming along behind another package author and trying to make changes can be tricky… :D You have to tread carefully lest you break something that's connected in a weird way to code you change.
Bill
-
This is what I got today when upgrading Snort.
It seems that it only needed 1 attempt..
Apr 22 13:16:34 SnortStartup[532]: Snort START For Internet(9626_em0)…
Apr 22 13:14:46 SnortStartup[33113]: Snort STOP For Internet(9626_em0)…
Apr 22 13:14:31 php: /snort/snort_download_rules.php: The Rules update has finished.
Apr 22 13:14:31 php: /snort/snort_download_rules.php: Emerging Threat rules are up to date...
Apr 22 13:14:31 php: /snort/snort_download_rules.php: Snort GPLv2 Community Rules are up to date...
Apr 22 13:14:30 php: /snort/snort_download_rules.php: Snort VRT rules are up to date...
Apr 22 13:14:30 php: /snort/snort_download_rules.php: Snort MD5 Attempts: 1
Apr 22 12:46:22 check_reload_status: Reloading filter
Apr 22 12:46:21 check_reload_status: Syncing firewall
Apr 22 12:46:20 php: /pkg_mgr_install.php: Building new sig-msg.map file for WAN...
Apr 22 12:46:18 php: /pkg_mgr_install.php: Resolving and auto-enabling any flowbit-required rules for WAN...
Apr 22 12:46:16 php: /pkg_mgr_install.php: Updating rules configuration for: WAN ...
Apr 22 12:46:16 php: /pkg_mgr_install.php: The Rules update has finished.
Apr 22 12:46:08 php: /pkg_mgr_install.php: EmergingThreats rules file update downloaded succsesfully
Apr 22 12:46:06 php: /pkg_mgr_install.php: There is a new set of EmergingThreats rules posted. Downloading...
Apr 22 12:46:05 php: /pkg_mgr_install.php: Snort GPLv2 Community Rules file update downloaded succsesfully
Apr 22 12:46:04 php: /pkg_mgr_install.php: There is a new set of Snort GPLv2 Community Rules posted. Downloading...
Apr 22 12:46:03 php: /pkg_mgr_install.php: Snort Rules Attempts: 1That correlates with what I see on my own box sometimes. Rarely do I get a failure to download after I added the loop construct I mentioned, but I do often seen more than one attempt is required to get the file. Also, for me, the Snort VRT tarball downloads quite slowly compared to the ET tarball which is fast.
Bill
-
It could be the load on the specific server if it located in two different places :)
