Why app-stats.log doesn't appear in Alterts Tab?
-
Well, whether or not they appear in the alerts depends on exactly what the rule is looking for (in other words, what triggers it). Remember the text rules that generate the alerts were created by volunteer non-paid staff at a University in Brazil and contributed to pfSense users by them. I don't think the actual text rules content has been updated for quite some time, but the Snort OpenAppID stub detectors have changed over time. So maybe the rules no longer work correctly for some applications.
Also remember that the rules only have a very few things they can actually see when it comes to an SSL session. Basically they see the initial header exchanges before the SSL key is negotiated, and they use that to decide whether or not to "trigger". Once the secured SSL session begins with encrypted packet payloads, the IDS/IPS sees nothing but random data due to the encryption. This is true for everything analyzed by Snort or Suricata when it comes to encrypted traffic. The only way the IDS/IPS can do real DPI (deep packet inspection) is when you configure SSL termination (MITM) on the firewall.
-
I understand that some rules may be outdated and layer 7 app detection is probably not 100% but i don't think that this is the problem. At least some well known applications (like facebook in your screenshot) should be alerted => but NOTHING on my box. When i filter the alerts for destination port other than 80 (!80) i have no alert at all. You screenshot shows, that your pfsense alerting on facebook (DPort 443), but it does not appear on my pfsense (I did also test with facebook).
That's why i had the assumption, that snort don't inspect other communication than http but i don't know why...
-
Snort will not limit its inspection to just HTTP unless that were the only types of rules you had enabled.
My inclination at this point is your $HOME_NET and/or $EXTERNAL_NET variables are not correct. You mentioned in a much earlier post about modifying them by assigning a custom list (you said you created a Pass List and used that). In the vast majority of installations there is never a need to modify the $HOME_NET and $EXTERNAL_NET variables from their default values. I would go back to the INTERFACE SETTINGS tab and reset both of those drop-downs to "default". Then click Save and return to the INTERFACES tab and restart Snort on that interface.
I also see from looking back at one of your screenshots that you appear to be running on a Hyper-V installation (I see the hn0 NIC) listed. Are you 100% positive that your virtual switch is correctly configured such that pfSense is actually seeing the traffic you want to get alerts from? You blacked out all the IP addresses in your screenshot, so I can't tell if there are RFC1918 addresses in use or not. If they are RFC1918 addresses, then there is really no point in blacking them out. It would be really useful if you would post up a screenshot of the contents of your $HOME_NET and $EXTERNAL_NET variables along with another screenshot of those alerts with the IP addresses visible (assuming they are all either RFC1918 or external Internet addresses).
-
Yes, pfsense is running on Hyper-V 2019. The NIC config is pretty default. Just one interface is configured as a VLAN trunk.
$HOME_NET is at follows:
10.242.5.0/24 62.2.17.60 62.2.17.61 62.2.24.158 62.2.24.162 77.59.160.1 77.59.160.79 127.0.0.1 172.16.10.0/24 172.16.10.7 172.16.10.8 172.16.22.0/24 172.16.98.0/24 ::1 fd45:c46d:6ae2:a::1 fe80::215:5dff:fe0a:f00 fe80::215:5dff:fe0a:f01 fe80::215:5dff:fe0a:f02
-
VLANs can be problematic for IDS/IPS setups. You will need to be sure all NIC offloading is disabled. Since you are running in a Hyper-V setup, I would think perhaps that is already done, but you want to make sure. There are settings within pfSense and then likely settings in Hyper-V or the Hyper-V host someplace. I don't use Hyper-V, so can't help there. I do use ESXi, but I do not run my firewall as a VM.
Are you just running Snort on the parent interface?
I guess I'm still not clear on what you are expecting to see that you are not seeing. So have you tried to duplicate my Facebook results? Enable the Social Media rules and then open Facebook on a smartphone that is running on one of the networks listed above in your HOME_NET. So for example, make sure the phone is not using Cellular Data or otherwise bypassing the networks listed in HOME_NET.
-
Regarding Hyper-V adapter settings: I have to investigate in this, will take a bit more time for testing the various settings.
"Are you just running Snort on the parent interface?" > I'm running snort neither on an interface configured as VLAN trunk nor the interface has a VLAN configured on the VM configuration.
I tested exactly the way you have described (devices are located in local WLAN/LAN of course) and additionally i have tested with Netflix. I also expect other services like FaceTime, Youtube, ect..), because i know there is traffic towards those services.
-
@Liceo said in Why app-stats.log doesn't appear in Alterts Tab?:
I'm running snort neither on an interface configured as VLAN trunk nor the interface has a VLAN configured on the VM configuration.
So you are running Snort on the WAN? Does all of the traffic you are expecting to see actually traverse that interface? I would assume so, but since it is a VM and has a virtual switch, there are ways for things to bypass. Also remember that Snort sits outside of the firewall directly in front of the NIC. So on your WAN, it will see all LAN traffic post-NAT rules applied (meaning it will not see any individual LAN hosts, but only the WAN's public IP). The recommended place to run the IDS/IPS is on the LAN so that it can see inbound traffic on the interface before NAT is applied, and outbound traffic from the interface after any NAT is undone.
I run my main Snort instances on my LAN and DMZ interfaces. I do run an instance on my WAN with just the ET-CIARMY rules, but that instance is solely to generate alerts that I use when testing new GUI package updates.
-
No i'm running Snort on LAN. I have 3 interfaces: WAN, LAN and LAN/VLAN-Trunk.
BTW, this are the system logs from Status > System Logs > System > General when restarting the Snort enabled LAN interface, maybe it helps debugging...
Oct 2 08:16:02 kernel hn1: promiscuous mode enabled Oct 2 08:16:01 snort 44143 AppInfo: AppId 2314 is UNKNOWN Oct 2 08:16:01 snort 44143 Invalid direct service AppId, 2314, for 0x80b7b4f90 0x804d9a440 Oct 2 08:16:01 snort 44143 AppInfo: AppId 2312 is UNKNOWN Oct 2 08:16:01 snort 44143 Invalid direct service AppId, 2312, for 0x80b7b4f90 0x804d9a440 Oct 2 08:16:01 snort 44143 AppId Oct 2 08:16:01 snort 44143 AppId Oct 2 08:16:01 snort 44143 AppId Oct 2 08:16:01 snort 44143 AppId Oct 2 08:16:01 snort 44143 AppId Oct 2 08:16:01 snort 44143 AppId Oct 2 08:16:01 snort 44143 AppId Oct 2 08:16:01 snort 44143 AppId Oct 2 08:16:01 php [Snort] Snort START for LAN10(hn1)... Oct 2 08:16:01 php [Snort] Building new sid-msg.map file for LAN10... Oct 2 08:16:01 php [Snort] Enabling any flowbit-required rules for: LAN10... Oct 2 08:16:01 php [Snort] Enabling any flowbit-required rules for: LAN10... Oct 2 08:16:00 php [Snort] Updating rules configuration for: LAN10 ... Oct 2 08:15:59 kernel hn1: promiscuous mode disabled Oct 2 08:15:59 snort 74522 *** Caught Term-Signal Oct 2 08:15:58 php-fpm [Snort] Snort STOP for LAN10(hn1)... Oct 2 08:15:58 php-fpm Restarting Snort on LAN10(hn1) per user request...
-
@Liceo said in Why app-stats.log doesn't appear in Alterts Tab?:
No i'm running Snort on LAN. I have 3 interfaces: WAN, LAN and LAN/VLAN-Trunk.
BTW, this are the system logs from Status > System Logs > System > General when restarting the Snort enabled LAN interface, maybe it helps debugging...
Oct 2 08:16:02 kernel hn1: promiscuous mode enabled Oct 2 08:16:01 snort 44143 AppInfo: AppId 2314 is UNKNOWN Oct 2 08:16:01 snort 44143 Invalid direct service AppId, 2314, for 0x80b7b4f90 0x804d9a440 Oct 2 08:16:01 snort 44143 AppInfo: AppId 2312 is UNKNOWN Oct 2 08:16:01 snort 44143 Invalid direct service AppId, 2312, for 0x80b7b4f90 0x804d9a440 Oct 2 08:16:01 snort 44143 AppId Oct 2 08:16:01 snort 44143 AppId Oct 2 08:16:01 snort 44143 AppId Oct 2 08:16:01 snort 44143 AppId Oct 2 08:16:01 snort 44143 AppId Oct 2 08:16:01 snort 44143 AppId Oct 2 08:16:01 snort 44143 AppId Oct 2 08:16:01 snort 44143 AppId Oct 2 08:16:01 php [Snort] Snort START for LAN10(hn1)... Oct 2 08:16:01 php [Snort] Building new sid-msg.map file for LAN10... Oct 2 08:16:01 php [Snort] Enabling any flowbit-required rules for: LAN10... Oct 2 08:16:01 php [Snort] Enabling any flowbit-required rules for: LAN10... Oct 2 08:16:00 php [Snort] Updating rules configuration for: LAN10 ... Oct 2 08:15:59 kernel hn1: promiscuous mode disabled Oct 2 08:15:59 snort 74522 *** Caught Term-Signal Oct 2 08:15:58 php-fpm [Snort] Snort STOP for LAN10(hn1)... Oct 2 08:15:58 php-fpm Restarting Snort on LAN10(hn1) per user request...
Those two "Invalid" lines are from typos in the OpenAppID text rules. The authors did not complete the spelling of "direct-tv" (or something similar to that phrase, don't remember now but I once researched it). Those are present on my system as well and have no bearing on OpenAppID working.
I'm still confused. You say you have three interfaces but run Snort on the LAN. Yet your earlier screenshots show "WAN" in the interface name. So do you mean you have Snort running on all three interfaces, two of the interfaces, or just one of them?
Also, if you have (or had in the past) multiple interfaces configured with Snort, you will have multiple sub-directories in
/var/log/snort
. Each Snort instance has its own private logging system in one of those sub-directories. So now I'm not clear from which interface theapp-stats.log
you posted a snippet of actually came from.I can tell you the OpenAppID subsystem works for me and many others. You appear to have a configuration problem on your end.
-
Sorry for the confusion, i had configured Snort on WAN first, but now it's only on the LAN Interface with same results. If have no doubts, that snort is running on many installations, i just want to make it work on my box ;-).
I think i will deploy a clean pfsense install and see if it's working. I will let you know.
But anyway, thanks for your help so far!
-
I set up a new instance of pfsense from scratch with one LAN and one WAN interface on hyper-v (gen 1). I generated some facebook and netflix traffic but Snort only shows some http alerts. Maybe a hyper-v related problem. I still think it just doesn't capture SSL traffic
-
@Liceo said in Why app-stats.log doesn't appear in Alterts Tab?:
I set up a new instance of pfsense from scratch with one LAN and one WAN interface on hyper-v (gen 1). I generated some facebook and netflix traffic but Snort only shows some http alerts. Maybe a hyper-v related problem. I still think it just doesn't capture SSL traffic
Snort captures OpenAppID alerts on SSL just fine. Look at my screenshot from my own testing. Something appears to be non-standard in your setup. Do you any kind of web proxy installed?