Suricata 1.4.6 pkg v1.0.2 – Update Release Notes

val

@val:

Hi Bill, firstly great job.

And would like to ask if the PPPoE issue still ongoing in Suricata?

The underlying binary is unchanged for this release, so yeah it probably still won't recognize the interface type. I am working now on moving to the 2.0 code branch for the Suricata binary. Maybe that will work better with a PPPoE interface.

Bill

Righto, Thanks, I will keep an eye out once you updated the underlying binary and I will test it again.

Once again, thank you very much.

A Former User

Started testing suricata, already found a bug.
Interface variables for ports, do not allow ports to be entered, with an error of "Only aliases are allowed"

HTTP_PORTS box contains (without quotes): "80,443"

When saving, the above mentioned error is shown.

Edit:

Interface variables, add FTP_SERVERS
Interface variables, add SSH_SERVERS
add option to change the syslog facility
~~blocked tabs does not update on new alerts, host shown as blocked on the alerts tab. On that note, where is the suricata blocked hosts table???~~ had to restart the box, ok now
automatically generated Pass List not recognizing IPv6 addresses
in edit rules tab, the icons do not behave as expected. disabling a user enabled rule, makes it red (normal or faded, depending on how upstream handles it), which means if upstream enables it/changes it, it could start generating false positives. Clicking the disable button on the >ALERTS< tab though, behaves correctly, the icon is faded yellow (user disabled). I noticed this a couple of months back in snort also, it makes it very hard to permanently disable rules from the rule edit tab.

Feature request: sync to CARP slave

BBcan177

I think the error means you need to create an "alias" with those ports listed, and than add the "alias" to the Suricata variables boxes. This is similar to Snort variables.

So don't try to directly enter the ports in the variable boxes.

A Former User

Assign a variable to a variable? that's counterintuitive. It needs to behave the same way as the hosts variables above it.

BBcan177

Have to keep you on your toes!! :)

I agree, but that's how it works…

A Former User

Here's a screenshot from snort, I'm already on my toes, don't need any more anxiety by switching over :P

insert Jobs speech "You can change it!"

They need to accept both aliases and values in those fields, it would make setting it up easier.

Notes on screenshot: 28days removal, almost no false positives, based on custom rules. I do believe I hold the forum record for most hosts blocked by snort, but then again I broke the snort package by enabling too many rules ;D

snort_record.jpg_thumb

bmeeks

@jflsakfja:

Assign a variable to a variable? that's counterintuitive. It needs to behave the same way as the hosts variables above it.

This is in keeping with the pfSense overall philosophy. I have mentioned this analogy before, but many of the big "commercial pay" firewalls follow the same idea. Checkpoint is one I am quite familiar with as I managed those for many years. They use "objects" instead of "aliases", but the concept and operation is exactly the same.

If you approach this with an open mind and think about the entire firewall configuration, it can actually make quite good sense. Suppose you have a web server farm and use the standard 80 and 443 ports and maybe one non-standard port for something. You would want your firewall and Snort or Suricata rules to match up with respect to inspecting those ports. You could manually enter port numbers, but then as your rules get more complex you can easily make a change in one place and forget it in two others. With an alias, you sort of can't make that error. You create the alias with the ports, then use that alias in all places where port numbers are required. If you need to change a port 2 years from now, you edit the alias and the new port is correct everywhere… :D

When I first started using Checkpoint products I felt like you did -- just let me enter a physical port number. But as I gained experience and starting creating and managing much more complicated rule sets, I began to appreciate the real power of "objects in Checkpoint" or "aliases in pfSense". Don't forget that starting in 2.0 of pfSense you can nest aliases. This can be helpful in some situations.

Bill

A Former User

@bmeeks:

This is in keeping with the pfSense overall philosophy. I have mentioned this analogy before, but many of the big "commercial pay" firewalls follow the same idea. Checkpoint is one I am quite familiar with as I managed those for many years. They use "objects" instead of "aliases", but the concept and operation is exactly the same.

If you approach this with an open mind and think about the entire firewall configuration, it can actually make quite good sense. Suppose you have a web server farm and use the standard 80 and 443 ports and maybe one non-standard port for something. You would want your firewall and Snort or Suricata rules to match up with respect to inspecting those ports. You could manually enter port numbers, but then as your rules get more complex you can easily make a change in one place and forget it in two others. With an alias, you sort of can't make that error. You create the alias with the ports, then use that alias in all places where port numbers are required. If you need to change a port 2 years from now, you edit the alias and the new port is correct everywhere… :D

When I first started using Checkpoint products I felt like you did -- just let me enter a physical port number. But as I gained experience and starting creating and managing much more complicated rule sets, I began to appreciate the real power of "objects in Checkpoint" or "aliases in pfSense". Don't forget that starting in 2.0 of pfSense you can nest aliases. This can be helpful in some situations.

Bill

If you can't beat them, join them ;D
I'll try it out and see how it works. Bill, have you seen my other post above? Those need fixing

bmeeks

@jflsakfja:

@bmeeks:

This is in keeping with the pfSense overall philosophy. I have mentioned this analogy before, but many of the big "commercial pay" firewalls follow the same idea. Checkpoint is one I am quite familiar with as I managed those for many years. They use "objects" instead of "aliases", but the concept and operation is exactly the same.

If you approach this with an open mind and think about the entire firewall configuration, it can actually make quite good sense. Suppose you have a web server farm and use the standard 80 and 443 ports and maybe one non-standard port for something. You would want your firewall and Snort or Suricata rules to match up with respect to inspecting those ports. You could manually enter port numbers, but then as your rules get more complex you can easily make a change in one place and forget it in two others. With an alias, you sort of can't make that error. You create the alias with the ports, then use that alias in all places where port numbers are required. If you need to change a port 2 years from now, you edit the alias and the new port is correct everywhere… :D

When I first started using Checkpoint products I felt like you did -- just let me enter a physical port number. But as I gained experience and starting creating and managing much more complicated rule sets, I began to appreciate the real power of "objects in Checkpoint" or "aliases in pfSense". Don't forget that starting in 2.0 of pfSense you can nest aliases. This can be helpful in some situations.

Bill

If you can't beat them, join them ;D
I'll try it out and see how it works. Bill, have you seen my other post above? Those need fixing

Yes, I saw the other reports. I will take a look. I have a really hard (well, almost impossible) time testing IPv6 stuff because my ISP does not provide or allow it. Anything I do has to all be whatever VMware can emulate/simulate.

Which syslog do you want to change the facility for? The alerts output option or the general Suricata operations logging to syslog? There is an option for changing the syslog facility if you use the Barnyard2 output plugin.

The thing with the forced enable/disable icons was done as part of fixing it so you could revert back to "default" for a particular rule. Previously you only had the option of "forcing it on" or "forcing it off". You could never click back around to just the default state. If you have a rule "forced", it will stay in whatever state you selected because it stores that SID info in the config.xml file.

Bill

A Former User

Ideally I would like to change both syslog facilities. The auth facility used by suricata now is miles from being relevant to it. It's for logins and elevating user priviledges, not an IDS/IPS. I believe that's for the alerts. I think I saw local5 mentioned somewhere, that could be for the daemon part of suricata, which should be…well... daemon.

If a rule is default enabled (red), clicking it sets it to user disabled (pale yellow). Enabling all the rules though, and then clicking a particular rule does not set it to pale yellow, but what ever the default was. If I misunderstood something, please correct me.

bmeeks

@jflsakfja:

Ideally I would like to change both syslog facilities. The auth facility used by suricata now is miles from being relevant to it. It's for logins and elevating user priviledges, not an IDS/IPS. I believe that's for the alerts. I think I saw local5 mentioned somewhere, that could be for the daemon part of suricata, which should be…well... daemon.

If a rule is default enabled (red), clicking it sets it to user disabled (pale yellow). Enabling all the rules though, and then clicking a particular rule does not set it to pale yellow, but what ever the default was. If I misunderstood something, please correct me.

I need to go back and look at how things are configured in pfSense, but off the top of my head I seem to remember that only AUTH facility messages would wind up in the system log. Other facilities are hard-coded directed to some other files if I recall. I can allow the Suricata facility to be anything (both alerts and general output are configurable), but only certain settings will actually cause the messages to show up in the system log on pfSense. I chose the defaults the way I did simply to insure the output showed up in the system log on pfSense in the expected file. If you really want custom facility outputs, I suggest using the Barnyard2 output options and feed the data to a remote syslog server.

I see what you mean about the rule icon colors. As I mentioned, the idea is to now "default" the rule back when clicked a second time. So if the rule is default disabled (pale red) and you click it, then it is forced to enabled and turns yellow. If it was forced "enabled" and you click it a second time, it reverts to its default state (that is, any "forced state" is simply removed). I have been thinking about the best way to incorporate the functionality of enablesid, disablesid and modifysid into the package (like Pulled Pork and some other third-party tools do). That would solve your problem of wanting to make sure a rule stays "always disabled".

Bill

Cino

@bmeeks:

I have a really hard (well, almost impossible) time testing IPv6 stuff because my ISP does not provide or allow it. Anything I do has to all be whatever VMware can emulate/simulate.

You could use an IPv6 Tunnel Broker like HE.NET. I've started to use them ever since IPv6 was added to pfSense testing.

A Former User

@bmeeks:

I need to go back and look at how things are configured in pfSense, but off the top of my head I seem to remember that only AUTH facility messages would wind up in the system log. Other facilities are hard-coded directed to some other files if I recall. I can allow the Suricata facility to be anything (both alerts and general output are configurable), but only certain settings will actually cause the messages to show up in the system log on pfSense. I chose the defaults the way I did simply to insure the output showed up in the system log on pfSense in the expected file. If you really want custom facility outputs, I suggest using the Barnyard2 output options and feed the data to a remote syslog server.

I've been using snort with local0 for a few years now (added via advanced options) and the logs showed up both on the pfsense log page, and the remote syslog.

As it stands now, the logs do get sent to the remote syslog, but they are tagged with the wrong facility (auth). On the remote syslog I can just direct them based on their tags, but it's not ideal. As I said, the auth facility should be used for logging user logins/logouts/users elevating priviledges through sudo (for example). Barnyard2 shouldn't be necessary, since the logs are already pushed to syslog, but just tagged wrongly.

WRT IPv6, I too highly recommend HE.net's tunnel service. It provides everything you need to experiment with IPv6.

bmeeks

@jflsakfja:

@bmeeks:

I need to go back and look at how things are configured in pfSense, but off the top of my head I seem to remember that only AUTH facility messages would wind up in the system log. Other facilities are hard-coded directed to some other files if I recall. I can allow the Suricata facility to be anything (both alerts and general output are configurable), but only certain settings will actually cause the messages to show up in the system log on pfSense. I chose the defaults the way I did simply to insure the output showed up in the system log on pfSense in the expected file. If you really want custom facility outputs, I suggest using the Barnyard2 output options and feed the data to a remote syslog server.

I've been using snort with local0 for a few years now (added via advanced options) and the logs showed up both on the pfsense log page, and the remote syslog.

As it stands now, the logs do get sent to the remote syslog, but they are tagged with the wrong facility (auth). On the remote syslog I can just direct them based on their tags, but it's not ideal. As I said, the auth facility should be used for logging user logins/logouts/users elevating priviledges through sudo (for example). Barnyard2 shouldn't be necessary, since the logs are already pushed to syslog, but just tagged wrongly.

WRT IPv6, I too highly recommend HE.net's tunnel service. It provides everything you need to experiment with IPv6.

I will put altering the syslog facility in the next Suricata release. I'm hoping that coincides with the 2.0.1 Suricata binary as well. That's my plan at this point (update the binary to 2.0.1 and add the necessary bits to the GUI to support the additional features).

The Suricata guys are also working on Netmap support. They have not published a release date or version yet, but it is showing as 50% done on their work schedule. This will allow high speed IPS operation assuming the pfSense guys will include the required kernel module in their builds.

As for the IPv6 trick, thanks for the tip and recommendation. I will check it out.

Bill

BBcan177

The Suricata guys are also working on Netmap support. They have not published a release date or version yet, but it is showing as 50% done on their work schedule. This will allow high speed IPS operation assuming the pfSense guys will include the required kernel module in their builds.

Do the pfSense Devs support moving to netmap also? I assume this will also work for the Snort package? I wonder what they expect the max throughput to be?

A Former User

Another bug: IPv6 addresses do not have an unblock button on the alerts page, although they are correctly added to the blocked table (snort2c) and the blocked page.

bmeeks

@BBcan17:

The Suricata guys are also working on Netmap support. They have not published a release date or version yet, but it is showing as 50% done on their work schedule. This will allow high speed IPS operation assuming the pfSense guys will include the required kernel module in their builds.

Do the pfSense Devs support moving to netmap also? I assume this will also work for the Snort package? I wonder what they expect the max throughput to be?

I believe they do. Haven't heard any estimates of throughput, but it should be pretty good.

Bill

bmeeks

@jflsakfja:

Another bug: IPv6 addresses do not have an unblock button on the alerts page, although they are correctly added to the blocked table (snort2c) and the blocked page.

Now that I have my own IPv6 setup working with the Hurricane Electric tunnel broker, I can do a bit more testing. The way Suricata (and Snort) get the list of currently blocked IPs is by querying pf using pfctl. Curious they showed up in the <snort2c>table and still did not have an unblock icon.

Bill</snort2c>

Cino

Great news Bill. Welcome to IPv6, maybe its me but its seems to be a lonely place right now…

1: Dk if this is a bug but I turned on pfSense notifications and I'm receiving these emails:


X-Cron-Env: <shell= bin="" sh="">X-Cron-Env: <home= root="">X-Cron-Env: <path= usr="" bin:="" bin="">X-Cron-Env: <logname=root>X-Cron-Env: <user=root>Warning: filesize(): stat failed for /var/log/suricata/suricata_em339811/stats.log in /usr/local/pkg/suricata/suricata_check_cron_misc.inc on line 129

Warning: filesize(): stat failed for /var/log/suricata/suricata_em231600/stats.log in /usr/local/pkg/suricata/suricata_check_cron_misc.inc on line 129</user=root></logname=root></path=></home=></shell=>

I didn't have statistics enabled which I believe is why this popped up. I've enabled it, and the emails seem to have gone away.

2: In Log mgt, I've changed Alerts to have NO LIMIT, upon saving; it goes back to 500KB

3: Also I noticed 75% of the time, my second sensor wont start on its on. Either a re-boot, package re-start because of dhcp or wan flap. I'll see if I can capture a log of when this happens if that will help. I can't reproduce if I manually stop/start the service

DigitalDeviant

I'm having a problem with Suricata running on my LAN interface. I have 2 instances running, WAN & LAN and WAN seems fine but anytime the rules update the LAN fails to restart. Manually starting it seems to work fine.

Suricata.log for the interface

30/5/2014 -- 02:31:30 - <info> -- Signal Received.  Stopping engine.
30/5/2014 -- 02:31:30 - <info> -- 0 new flows, 0 established flows were timed out, 0 flows in closed state
30/5/2014 -- 02:31:30 - <info> -- time elapsed 66110.758s
30/5/2014 -- 02:31:31 - <info> -- (RxPcapem01) Packets 4399339, bytes 2794681616
30/5/2014 -- 02:31:31 - <info> -- (RxPcapem01) Pcap Total:4399388 Recv:4399339 Drop:49 (0.0%).
30/5/2014 -- 02:31:31 - <info> -- AutoFP - Total flow handler queues - 6
30/5/2014 -- 02:31:31 - <info> -- AutoFP - Queue 0  - pkts: 1315978      flows: 131283      
30/5/2014 -- 02:31:31 - <info> -- AutoFP - Queue 1  - pkts: 1361933      flows: 94964       
30/5/2014 -- 02:31:31 - <info> -- AutoFP - Queue 2  - pkts: 809788       flows: 45693       
30/5/2014 -- 02:31:31 - <info> -- AutoFP - Queue 3  - pkts: 358030       flows: 15976       
30/5/2014 -- 02:31:31 - <info> -- AutoFP - Queue 4  - pkts: 297870       flows: 7314        
30/5/2014 -- 02:31:31 - <info> -- AutoFP - Queue 5  - pkts: 270079       flows: 4121        
30/5/2014 -- 02:31:31 - <info> -- Stream TCP processed 707300 TCP packets
30/5/2014 -- 02:31:31 - <info> -- alert-pf output inserted 3 IP address blocks
30/5/2014 -- 02:31:31 - <info> -- alert-pf output wrote 3 alerts
30/5/2014 -- 02:31:31 - <info> -- Fast log output wrote 7 alerts
30/5/2014 -- 02:31:31 - <info> -- HTTP logger logged 2706 requests
30/5/2014 -- 02:31:31 - <info> -- Stream TCP processed 954694 TCP packets
30/5/2014 -- 02:31:31 - <info> -- alert-pf output inserted 3 IP address blocks
30/5/2014 -- 02:31:31 - <info> -- alert-pf output wrote 3 alerts
30/5/2014 -- 02:31:31 - <info> -- Fast log output wrote 7 alerts
30/5/2014 -- 02:31:31 - <info> -- HTTP logger logged 2200 requests
30/5/2014 -- 02:31:31 - <info> -- Stream TCP processed 479062 TCP packets
30/5/2014 -- 02:31:31 - <info> -- alert-pf output inserted 3 IP address blocks
30/5/2014 -- 02:31:31 - <info> -- alert-pf output wrote 3 alerts
30/5/2014 -- 02:31:31 - <info> -- Fast log output wrote 7 alerts
30/5/2014 -- 02:31:31 - <info> -- HTTP logger logged 1843 requests
30/5/2014 -- 02:31:31 - <info> -- Stream TCP processed 95982 TCP packets
30/5/2014 -- 02:31:31 - <info> -- alert-pf output inserted 3 IP address blocks
30/5/2014 -- 02:31:31 - <info> -- alert-pf output wrote 3 alerts
30/5/2014 -- 02:31:31 - <info> -- Fast log output wrote 7 alerts
30/5/2014 -- 02:31:31 - <info> -- HTTP logger logged 1389 requests
30/5/2014 -- 02:31:31 - <info> -- Stream TCP processed 52257 TCP packets
30/5/2014 -- 02:31:31 - <info> -- alert-pf output inserted 3 IP address blocks
30/5/2014 -- 02:31:31 - <info> -- alert-pf output wrote 3 alerts
30/5/2014 -- 02:31:31 - <info> -- Fast log output wrote 7 alerts
30/5/2014 -- 02:31:31 - <info> -- HTTP logger logged 1090 requests
30/5/2014 -- 02:31:31 - <info> -- Stream TCP processed 30060 TCP packets
30/5/2014 -- 02:31:31 - <info> -- alert-pf output inserted 3 IP address blocks
30/5/2014 -- 02:31:31 - <info> -- alert-pf output wrote 3 alerts
30/5/2014 -- 02:31:31 - <info> -- Fast log output wrote 7 alerts
30/5/2014 -- 02:31:31 - <info> -- HTTP logger logged 883 requests
30/5/2014 -- 02:31:31 - <info> -- host memory usage: 194304 bytes, maximum: 16777216
30/5/2014 -- 02:31:31 - <info> -- cleaning up signature grouping structure... complete
30/5/2014 -- 02:31:32 - <error> -- [ERRCODE: UNKNOWN_ERROR(87)] - Child died unexpectedly</error></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info></info>