Taming the beasts… aka suricata blueprint

[coliflower]

Hello all,

I am not sure to be right her - so sorry in advance in case to be wrong.

For some weeks I have some troubles with my pfSense on APU1D4 :-(
The symptom is: I have no connection to internet - there is no public IP on my GW …
If I restart the pfSense, it works again for some time …
After some weeks I also tried to restart the suricata service and it helps, too.

Does it mean there is something NOK wit suricata on my pfSense ?

I did not change anything before the symptom occurred :-(
pfSense and suricata is on the last level.

if anybody know the direction to start investigation, please help :-)

Thanks a lot in advance !

[stephenw10]

Suricata/Snort is often a good place to look if you find things spontaneously get blocked.

Try running Suricata in non-blocking mode for a some time to test that.

You should also be able to clear to blocked hosts lists if that is happening and restore connectivity that way. You'll then need to find out what is being blocked and by which rules and take steps to prevent it happening again.

Steve

[coliflower]

Hello Steve,

thank you for your reply.

Suricata is running as default, pattern-match is AC, block and Barnyard2 is DISABLED.
If I look to Interfaces/WAN-Rules than I see „Rule will alert on traffic if triggered“ under Action header.

Any further ideas ?

Dariusz

[stephenw10]

If blocking is disabled then it's not Suricata or at least not in the expected way. It might be using all the available resources for example. You should see something logged if so though.

You have other packages installed?

How exactly is this presenting? You say the gateway looses its IP? The WAN gateway? The interface address itself? What type of WAN is it?

Steve

[coliflower]

OK … For a test I stopped the Suricata service to see whether I will see the same symptom „no internet“ or not …

I have other packages -> nmap and pfBlockerNG where I only put some outgoing white-list countries on the internal LAN-Interface.

The WAN is connected on a cable-modem with an public IPv4 84.112.x.x DHCP (ISP = UPC).
The Gateway widget shows me offline with red background and if I look to the dynamic DNS status than there is the old IP-address in red instead of green.
If this occurs there is no internet on my LAN-net (if I put the cable from cable-modem directly to my MBP, than it works.
Or if I restart the pfSense itself than I have internet, too

Dariusz

[pfBasic]

@stephenw10:

Try running Suricata in non-blocking mode for a some time to test that.

You should also be able to clear to blocked hosts lists if that is happening and restore connectivity that way.@coliflower:

Suricata is running as default, pattern-match is AC, block and Barnyard2 is DISABLED.

He already told you what you need to do.
Go to "Services / Suricata / Edit Interface Settings - Your_Interface"

Scroll down the page to where it says "Alert and Block Settings" Below that you'll see "Block Offenders" - UNcheck the box. Save, Apply.

Now, go to "Diagnostics / Tables", select "snort2c" from the dropdown menu - click "Empty Table".

Now I'm betting that your internet will be working again just as well as it does after you reboot it?

Over the next hours/days/weeks/months you'll need to analyze the alerts you see in Suricata and decide whether those alerts are false positives or not. Disable or suppress false positives as necessary.

DO NOT turn blocking back on until you have spent some time removing false positives (there will be a lot of them). If, after turning blocking back on your network has issues in the following days, your first suspicion should be suricata. Simply turn off blocking, clear your snort2c table and if all is well again then you know you have more false positives to weed out before you turn blocking back on.

IDS/IPS are not plug and play. Learn them before you turn on blocking or you are just going to break things.

[stephenw10]

This could also potentially be something like the WAN DHCP lease failing to renew correctly.

Instead of rebooting pfSense you might try instead:
Resaving the WAN settings.
Running ifconfig DOWN/UP on the WAN.
Disconnecting and resconnecting the WAN Ethernet cable.

See if any of those restore connectivity.

Steve

[coliflower]

@pfBasic:

@stephenw10:

Try running Suricata in non-blocking mode for a some time to test that.

You should also be able to clear to blocked hosts lists if that is happening and restore connectivity that way.@coliflower:

Suricata is running as default, pattern-match is AC, block and Barnyard2 is DISABLED.

He already told you what you need to do.
Go to "Services / Suricata / Edit Interface Settings - Your_Interface"

Scroll down the page to where it says "Alert and Block Settings" Below that you'll see "Block Offenders" - UNcheck the box. Save, Apply.

It is DISABLED (not checked) …

@pfBasic:

Now, go to "Diagnostics / Tables", select "snort2c" from the dropdown menu - click "Empty Table".

Now I'm betting that your internet will be working again just as well as it does after you reboot it?

What I already did is to delete all states in Diagnostics/States/States … Unfortunately after I restarted the Suricata service to have Internet again  :(
Diagnostic/Tables/snort2c is empty - maybe because the Suricata service is stopped ?
In the first step I will try to watch my Internet with stopped Suricata to see whether I will again lose the Internet connectivy or not - if this is not a bad idea …?  ::)

@pfBasic:

Over the next hours/days/weeks/months you'll need to analyze the alerts you see in Suricata and decide whether those alerts are false positives or not. Disable or suppress false positives as necessary.

DO NOT turn blocking back on until you have spent some time removing false positives (there will be a lot of them).

This was my original idea if I installed Suricata, unfortunately I never did  :-[
WAF = Woman Acceptance Factor was very low after birthday of our second boy …

Now it seems I need to do it but if never blocked the matched rules why I lost internet if this should be the root-cause of my problem  ;D  ?
Sorry for this question, I need to start to understand what I do  :)

[quote author=pfBasic link=topic=78062.msg737681#msg737681 date=1501541380]
If, after turning blocking back on your network has issues in the following days, your first suspicion should be suricata. Simply turn off blocking, clear your snort2c table and if all is well again then you know you have more false positives to weed out before you turn blocking back on.

OK I will do but es described, I do not block … but let us see whether I will lose internet-connectivy with stopped Suricata service or not …

@pfBasic:

IDS/IPS are not plug and play. Learn them before you turn on blocking or you are just going to break things.

I beliefe you, therefore I did not enable blocking by hard  :)
I need to start to understand the rules and the effects of them …

Thank you for your help, too

[coliflower]

@stephenw10:

This could also potentially be something like the WAN DHCP lease failing to renew correctly.

Instead of rebooting pfSense you might try instead:
Resaving the WAN settings.
Running ifconfig DOWN/UP on the WAN.
Disconnecting and resconnecting the WAN Ethernet cable.

See if any of those restore connectivity.

Steve

OK, thank you, I will also try your advices if the symptom comes again  :)

Dariusz

[pfBasic]

Are you using inline mode or legacy mode? Inline mode uses nmap which is very picky about NICs, it can bring your network down.

[coliflower]

The package is „only“ installed on pfSense under Diagnostics … How can I find out inline/legacy mode, please  :-[ ?

What could be the topic of nmap if Internet works before and causing troubles for some weeks ?

Thank you again :) !!

[Greenhill]

Hi,

I'm a bit confused with the setup, can someone explain how this can work with a floating rule - quick action ? Wont that stop processing rules on groups/interfaces as soon as it matches ? How can I make pfsense process rulesets with quick enabled after it matches on a floating rule ?

[coliflower]

Hi back after some „tests" :)

First of all, I still have some problem with having no Internet as described on July 31st …  :’(

After restarting Securita I had Internet for some days.
At the day I again lost Internet, I restarted Securita to get Internet - after that I stopped their service to see whether some actions of Security are responsible for my problem …
As the Internet went lost again, I checked the UI and saw the Service of Suricata was enabled by ? even the uptime was some days (to see whether pfSense restarts them by their self).

I have no idea what I could do or what  could cause some symptom  :-[ :’(

Any help is highly appreciated  :)

[pfBasic]

go into suricata and disable inspection on each interface and save the settings. then you will see something along the lines of "DISABLED" on each of your interfaces in suricata.

Then go to diagnostics - tables and make sure the snort2c table is empty.

After all interfaces say DISABLED or something along those lines and your snort2c table is empty, you are no longer inspecting with suricata and can rule it out of your troubleshooting.

[coliflower]

Thank you very much for your reply :-) !

I already did all your proposed topics - the snort2c table was always empty …
My next step will be to uninstall Suricata to - hopefully - get the final confirmation whether it is related to Suricata or not …

So thank you very much for your help once again - I will - of course - replay if there are still some troubles

Have a nice week-end :-) !!

[coliflower]

OK …
Suricata is uninstalled, unfortunately there are still interruption of Internet  :-[

I also figured out, if I unplug the WAN-cable an plug it again - it works again …

Even it ist not related to suricata, is there someone how can lead me to some topics I can read to continue to look for the root-cause ?

Any help is highly appreciated  :)

Thanks a lot in advance !

[stephenw10]

Did you try those tests I suggested? What was the result, did any of them restore access?

Steve

[coliflower]

Resaving the WAN settings.

Yes I did, I was able to connect to Internet.

Running ifconfig DOWN/UP on the WAN.

No, did not  :-[
To be honest, I do not know what exactly I have to do - sorry  :-[

[quote]Disconnecting and resconnecting the WAN Ethernet cable.
Yes, that was my last test (Internet OK)

Dariusz

[stephenw10]

Ah Ok. Interesting.

So to manually bring down and rebuild the interface run at the CLI or from Diag > Command Prompt:

ifconfig em0 down

Then:

ifconfig em0 up

Where em0 is your WAN interface, swap that for whatever interface your actual WAN is.

If that brings back up your connectivity then it shows that simply reloading the existing settings in pfSense can do it. Physically re-connecting it also resets whatever it's connected to and re-saving recreates the settings if they are lost.
Check the system log to see what events were triggered when you ran that. There should be something there to point to the problem.

Steve

[coliflower]

Thank you Steve,

I will check that if this failure occurs again  :)
My interface is re0 …

Dariusz