[solved, bad peering from Deutsche Telekom] after suricata 5.0.4_1 Update, suricata is no longer active
-
At least for me, on 2.4.5-RELEASE-p1.
It is not showing under service status, reboot didn't helped. -
@bob-dig "ps ax" and "top" told you what ?
No traces in the system log , Another log ?Maybe suricata decided that it can't be useful any-more, as 99,9 % of traffic is invisible ;)
-
@gertjan said in after suricata 5.0.4_1 Update, suricata is no longer active:
Maybe suricata decided that it can't be useful any-more, as 99,9 % of traffic is invisible ;)
That is why I just uninstalled it right away, will reinstall in some days. Typical noob-handling of things I guess.
Forgot to mention, that is where it seems failing:
Can't finish the dl.
Edit: To be more precise, it was the download of the snort rules after this! -
@bob-dig
Ah. Things are getting more clear.
The installer blocks - or bails out. The last line can't be more clear.
All is ok then. Some remote server isn't available, or content isn't available, at that moment. -
Saw the post in this thread and just tested installing the latest Suricata package update on a pfSense-2.4.5_p1 virtual machine. It worked flawlessly. Here is the install log --
>>> Installing pfSense-pkg-suricata... Updating pfSense-core repository catalogue... pfSense-core repository is up to date. Updating pfSense repository catalogue... pfSense repository is up to date. All repositories are up to date. The following 12 package(s) will be affected (of 0 checked): New packages to be INSTALLED: hiredis: 0.13.3 [pfSense] hyperscan: 4.7.0_3 [pfSense] jansson: 2.12 [pfSense] libmaxminddb: 1.4.2 [pfSense] libnet: 1.1.6_5,1 [pfSense] libpcap: 1.9.1_1 [pfSense] libyaml: 0.2.2 [pfSense] nspr: 4.25 [pfSense] nss: 3.51 [pfSense] pfSense-pkg-suricata: 5.0.4_1 [pfSense] py37-yaml: 5.3.1 [pfSense] suricata: 5.0.4 [pfSense] Number of packages to be installed: 12 The process will require 38 MiB more space. 7 MiB to be downloaded. [1/12] Fetching pfSense-pkg-suricata-5.0.4_1.txz: .......... done [2/12] Fetching suricata-5.0.4.txz: .......... done [3/12] Fetching libyaml-0.2.2.txz: ......... done [4/12] Fetching nss-3.51.txz: .......... done [5/12] Fetching nspr-4.25.txz: .......... done [6/12] Fetching libpcap-1.9.1_1.txz: .......... done [7/12] Fetching libnet-1.1.6_5,1.txz: .......... done [8/12] Fetching libmaxminddb-1.4.2.txz: ..... done [9/12] Fetching py37-yaml-5.3.1.txz: .......... done [10/12] Fetching jansson-2.12.txz: ...... done [11/12] Fetching hyperscan-4.7.0_3.txz: .......... done [12/12] Fetching hiredis-0.13.3.txz: .......... done Checking integrity... done (0 conflicting) [1/12] Installing nspr-4.25... [1/12] Extracting nspr-4.25: .......... done [2/12] Installing libyaml-0.2.2... [2/12] Extracting libyaml-0.2.2: ......... done [3/12] Installing nss-3.51... [3/12] Extracting nss-3.51: .......... done [4/12] Installing libpcap-1.9.1_1... [4/12] Extracting libpcap-1.9.1_1: .......... done [5/12] Installing libnet-1.1.6_5,1... [5/12] Extracting libnet-1.1.6_5,1: .......... done [6/12] Installing libmaxminddb-1.4.2... [6/12] Extracting libmaxminddb-1.4.2: .......... done [7/12] Installing py37-yaml-5.3.1... [7/12] Extracting py37-yaml-5.3.1: .......... done [8/12] Installing jansson-2.12... [8/12] Extracting jansson-2.12: .......... done [9/12] Installing hyperscan-4.7.0_3... [9/12] Extracting hyperscan-4.7.0_3: .......... done [10/12] Installing hiredis-0.13.3... [10/12] Extracting hiredis-0.13.3: .......... done [11/12] Installing suricata-5.0.4... [11/12] Extracting suricata-5.0.4: .......... done [12/12] Installing pfSense-pkg-suricata-5.0.4_1... [12/12] Extracting pfSense-pkg-suricata-5.0.4_1: .......... done Saving updated package information... done. Loading package configuration... done. Configuring package components... Loading package instructions... Custom commands... Executing custom_php_install_command()...Saved settings detected... Migrating settings to new configuration... done. Downloading Emerging Threats Open rules md5 file... done. There is a new set of Emerging Threats Open rules posted. Downloading... done. Downloading Snort VRT rules md5 file... done. There is a new set of Snort rules posted. Downloading... done. Downloading Snort GPLv2 Community Rules md5 file... done. There is a new set of Snort GPLv2 Community Rules posted. Downloading... done. Installing Emerging Threats Open rules... done. Installing Snort rules... done. Installing Snort GPLv2 Community Rules... done. Updating rules configuration for: WAN ... done. Updating rules configuration for: OPT1 ... done. Updating rules configuration for: LAN ... done. Cleaning up after rules extraction... done. The Rules update has finished. Generating suricata.yaml configuration file from saved settings. Generating YAML configuration file for WAN... done. Generating YAML configuration file for OPT1... done. Generating YAML configuration file for LAN... done. Finished rebuilding Suricata configuration from saved settings. Setting package version in configuration file. done. Executing custom_php_resync_config_command()...done. Menu items... done. Services... done. Writing configuration... done. =====The most common issue folks are having with the IDS/IPS packages is other add-on packages blocking rules downloads. The latest offender seems to be various IP lists that folks are configuring in the DNSBL feature of pfBlockerNG-devel. I'm noticing a ton of users here running a number of "blocking" packages on their firewall. When you do that, you must assume that at some point those packages will step on each other and at least one of them will interfere with the operation of the other.
These blocking packages periodically download and automatically install updates to either their rules (in the case of Snort and Suricata) or their IP lists of stuff to block (pfBlockerNG-devel and the DNSBL feature). So any update to an IP list of things to block, or an update to an IDS rule might suddenly result in a block of something that was working just fine yesterday. There is a danger with installing "blocking" packages that use lists of "bad IP addresses" that you download and install from all over the web. How do you know that list is accurate? How can you tell if perfectly legitimate IP addresses are mistakenly included on that list? My favorite example of the latter is an IP list several folks were using in pfBlockerNG that suddenly had the Google DNS servers added to it as "bad". How dumb is that? So if the person maintaining that list is dumb enough to let a legitimate IP like 8.8.8.8 get on it, how can you trust that he is smart enough to find actual bad IP addresses and put all of those on it?
.Why do these IP lists chosen for use in DNSBL and pfBlockerNG impact Snort and Suricata? Because the two most popular rules vendors host their rules archive files on Amazon Web Services infrastructure (AWS). Well, Amazon hosts a crap ton of services from around the world. So it is natural that every now and then some bad actor pops up on AWS someplace. Those IP lists then quickly update themselves to "block" the bad guy. They do that by blacklisting the IP space. Well guess what? If the AWS IP space where the Snort or Emerging Threats rules are hosted is blacklisted by some IP list that you have configured in pbBlockerNG and/or DNSBL, then your rules downloads for Snort or Suricata will stop working. And to compound this error, frequently those lists of "bad IP addresses" that you configure to download every day are not purged very well. That means while they might be quick to add an IP block to their list, they are not nearly as quick to remove the IP range when the host (AWS in this example) has booted the original offender from their services and set up a new legitimate client in that address space.
So why the long rant? The very first thing you should do when running the IDS/IPS packages or pfBlockerNG and something suddenly stops working- before you post a single message on this forum- is check thoroughly that one of those packages blocking something is not the cause of your issue. 95% or more of the time you will find the solution to your problem is something you've installed on your firewall blocking something else you've installed on your firewall...
.And my last rant/request --- when you post a request for help with something not working in the IDS/IPS packages, PLEASE list every package you have running on your firewall! That prevents a lot of time wasting while troubleshooting. The more experienced folks here trying to help usually have to drag out those details bit by excruciating bit in along exchanges of back and forth messages with the OP. All is made much easier and quicker if you go ahead up front and say "I am running the following packages on my firewall:" and then list them.
-
@bmeeks said in after suricata 5.0.4_1 Update, suricata is no longer active:
problem is something you've installed on your firewall blocking something else you've installed
+10
-
@bmeeks I disabled pfBlocker, but it is still horribly slow. I do have a new ISP, so maybe there is some "bad" peering involved? I will try and let is do its thing for more time. Also, nobody else seem to have this problem.
acme security 0.6.9_2
Cron sysutils 0.3.7_4
openvpn-client-export security 1.5_4
pfBlockerNG-devel net 3.0.0_6 -
@bob-dig said in after suricata 5.0.4_1 Update, suricata is no longer active:
@bmeeks I disabled pfBlocker, but it is still horribly slow. I do have a new ISP, so maybe there is some "bad" peering involved? I will try and let is do its thing for more time. Also, nobody else seem to have this problem.
acme security 0.6.9_2
Cron sysutils 0.3.7_4
openvpn-client-export security 1.5_4
pfBlockerNG-devel net 3.0.0_6Do you use the DNSBL feature of pfBlockerNG-devel? If so, simply disabling pfBlockerNG-devel does not necessarily undo the changes to the DNS resolver done by DNSBL. At least that's the way I understand it based on some replies from other users with similar issues as you. Only after they undid the DNSBL changes did the rules download work. Simply disabling pfBlockerNG-devel was not enough.
Using the DNSBL feature causes changes to be made to probably the most crtical component of your Internet connection: your DNS where all human-readable URLs are translated into appropriate IP addresses. If that part is monkeyed with, then all sorts of stuff can break.
-
@bob-dig said in after suricata 5.0.4_1 Update, suricata is no longer active:
@bmeeks I disabled pfBlocker, but it is still horribly slow. I do have a new ISP, so maybe there is some "bad" peering involved? I will try and let is do its thing for more time. Also, nobody else seem to have this problem.
acme security 0.6.9_2
Cron sysutils 0.3.7_4
openvpn-client-export security 1.5_4
pfBlockerNG-devel net 3.0.0_6A user yesterday or the day before was experiencing a problem with the Snort GPLv2 rules downloading. His fix, turning off DNSBL, then the download worked. Apparently some IP list he had turned on in DNSBL was blocking the AWS address space where the Snort GPLv2 rules are hosted. He disabled pfBlockerNG-devel first, but no change. Then I suggested he turn off DNSBL and undo the changes it had made to the
unboundconfiguration and then everything worked.I do not mean to pick on you, @Bob-Dig, nor am I singling you out. Rather I'm saying this for the benefit of others that may read this thread.
When you configure lots of IP Block Lists from various sites, you really need to be watchful of what exactly is on those lists and also realize that they can block all kinds of legitimate stuff along with bad stuff. So when something is not working on your end, your very first troubleshooting step should be looking at all the blocks from the "blocking packages" you have installed. Next, start turning those packages of one-by-one to see if your problem corrects. As I mentioned, a large majority of the time it will be corrected by turning off the blocking things. And you also need to be really aware of what the blocking packages are doing to your firewall configuration and know exactly what it takes to truly disable the package. One example is the link between pfBlockerNG-devel, its DNSBL feature, and the critical
unbounddaemon in pfSense. Ifunboundis not working, then very little with your Internet connection will work.In the case of Snort or Suricata, simply turning off the service will not remove existing blocks. You will also need to go to the BLOCKS tab and clear all blocked IP addresses there. So all of these blocking packages can cause you trouble in several different ways. And removing them from the troubleshooting tree is usually not as simple as just disabling them. They make changes in other parts of the firewall configuration that take other manual action to revert.
-
@bob-dig said in after suricata 5.0.4_1 Update, suricata is no longer active:
Also, nobody else seem to have this problem.
YES, hassle-free for us, and I agree with the words before me...
if I know well you are in the EU, me too., so that is a good guide
-
I disabled all I could find of pfBlocker, but still... Next try will be uninstalling it.
Uninstalling also didn't helped.
So last thing I changed was my ISP, back to my old one. That snort rules where a matter of seconds...
With switching ISP I also disabled IPv6. I checked before that IPv6 worked in general, at least from the WAN Interface.
So maybe my new ISP IP or whole range is blocked (Deutsche Telekom) or it is somewhat of an IPv6 problem, can't tell for sure. -
@bob-dig said in after suricata 5.0.4_1 Update, suricata is no longer active:
(Deutsche Telekom
Hmmm T-Com
try only through IPv4 gateway
-
If it's an IPv4 <> IPv6 issue, like the IPv6 peering of your ISP isn't working as well as IPv4, you could try the new pfBlockerNG-devel-3.0.0.6 Python mode No-AAAA facility : list the URL's that should be accessed using IPv4 only.
Normally, you would know it's an IPv6 issue : the URL used by pfBlockerNG to download a feed should also work using a browser. A browser pluging will tell you if it's using IPv4, or IPv6, if IPv4 is also available, if IPv6 is also available.
No DNSSEC - using Ipv6 - the browser didn't even ask for an IPv4 :
IPv6 used, IPv4 available (and or the site uses other pages that are IPv4 only) :
Btw : shutting down pfBlockerNG, and force reloading it ( ! ) should be enough, as pfBlockerNG wouldn't place firewall rules any more, wouldn't 'instruct' unbound with an URL list, and wouldn't activate Python mode : pfBlockerNG will not interfere with any DNS transactions : at that moment, it's neutral. If only de installing pfBlockerNG would help then that's a real bug.
But normally, it's not the package. It's the settings. -
Today I took a look and saw this.
Hadn't any time to test anything before but now I did by just going to the snort site in my Browser, logging in and downloading the snort rules. With the new ISP [Deutsche Telekom AG], download was fast in the beginning and then again became very very slow and took minutes to finish, but it did finished at least.
I then switched the gateway in pfSense to old ISP [Pyur/Tele Columbus AG], cleared the Browsercache and did the same thing again, it was a matter of seconds. All tests where done with IPv4 only.So in the end I can only suggest one thing: to decouple the update of the Suricata package from the update of the rulesets. Then Suricata wouldn't be stopped and it would be more clear where the problem is coming from, although the installation log is very clear in that regard, still the consequences are harder then they should be in my opinion.
Thanks for reading.
-
@bob-dig said in [solved, ISP-peering-problem] after suricata 5.0.4_1 Update, suricata is no longer active:
With the new ISP [Deutsche Telekom AG], download was fast in the beginning and then again became very very slow and took minutes to finish, but it did finished at least.
I then switched the gateway in pfSense to old ISP [Pyur/Tele Columbus AG], cleared the Browsercache and did the same thing again, it was a matter of seconds.Hi,
You make it clear for yourself that, things will change after you change your ISP GW.
So isn't this a pfSense issue?
-
@daddygo No, I use the new ISP all the time and they are known for there bad peering (customers are hostages) but I had no real problem until with the snort rules. But I guess, it can happen to everyone (with a bad ISP). I don't even think that it is the snort guys anymore, but still they could. Will try another DSL-ISP with a friend.
-
@bob-dig said in [solved, ISP-peering-problem] after suricata 5.0.4_1 Update, suricata is no longer active:
and they are known for there bad peering (customers are hostages)
if you have a chance to escape, please
(ISP)useful readings:
https://www.theregister.com/security/
https://www.expressvpn.com/blog/5-bizarre-edward-snowden-tweets/ -
@daddygo No chance, two years bound. My cable internet ISP before was even worse, not with peering, but had regular hiccups all the time.
-
@bob-dig said in [solved, ISP-peering-problem] after suricata 5.0.4_1 Update, suricata is no longer active:
No chance, two years bound.
I love Germany / Berlin..... (Berlin Techno, Trance, etc), I have been there a lot and my relatives live there.
I’m sorry for this stupid situation, there are always such ISPsYou tried to drive the traffic directly through a VPN, (e.g. ExpressVPN, they have good Berlin servers)
-
Had another "incident", wanted to download a small app named ZenTimings, which is hosted on: amazonaws.com
It is only 516KB, but with Deutsche Telekom, you almost can't download it, switching to other ISP or VPN, no problem.Damn, I regret.
@DaddyGo I don't want to route everything through a VPN, maybe I have to.
