Snort & PPPoE: Snort refuses to start(?)

Boags

Probably because pppoe creates a new interface instead of current real_interface applied to WAN

Instead of bge0, it should be pppoe0

Maybe changes on get_real_interface code on snort to test interface type may fix this issue.

Not sure how I can change this?

I did generate additional online gateways (which I can't seem to remove) while creating a static connection before this update, could that be causing issues for snort? - note these additions were a few weeks ago and there were no issues prior to the v. 2.5.7 update. -

@Deadringers:

hmm well I seem to have "fixed" my issue of snort not launching…

I completely un-installed snort and then re-installed it.

working now with all the rules I had before!?

I gave un-installing and re-installing a go. That does nothing for me.

Cheers :)

bmeeks

@Boags:

@marcelloc:

Probably because pppoe creates a new interface instead of current real_interface applied to WAN

Instead of bge0, it should be pppoe0

Maybe changes on get_real_interface code on snort to test interface type may fix this issue.

Not sure how I can change this?

I did generate additional online gateways (which I can't seem to remove) while creating a static connection before this update, could that be causing issues for snort? - note these additions were a few weeks ago and there were no issues prior to the v. 2.5.7 update. -

@Deadringers:

hmm well I seem to have "fixed" my issue of snort not launching…

I completely un-installed snort and then re-installed it.

working now with all the rules I had before!?

I gave un-installing and re-installing a go. That does nothing for me.

Cheers :)

Determination of the interface on 2.0.x and higher installs with PPPoE gets to be tricky. I'm no expert on the internal workings of pfSense, but I do remember seeing in the code where one of the previous developers hard-coded a "ng0" interface when the Snort code detects PPPoE. Perhaps one of the core team developers familiar with how PPPoE maps to interfaces on FreeBSD 8.1 and higher can chime here. I will be happy to fix the Snort code, but I don't know exactly how best to do that with PPPoE. I no longer have a PPPoE setup of my own since I abandoned DSL for my residence and switched over to a cable modem connection that uses DHCP.

Bill

Boags

@bmeeks:

@Boags:

@marcelloc:

Probably because pppoe creates a new interface instead of current real_interface applied to WAN

Instead of bge0, it should be pppoe0

Maybe changes on get_real_interface code on snort to test interface type may fix this issue.

Not sure how I can change this?

I did generate additional online gateways (which I can't seem to remove) while creating a static connection before this update, could that be causing issues for snort? - note these additions were a few weeks ago and there were no issues prior to the v. 2.5.7 update. -

@Deadringers:

hmm well I seem to have "fixed" my issue of snort not launching…

I completely un-installed snort and then re-installed it.

working now with all the rules I had before!?

I gave un-installing and re-installing a go. That does nothing for me.

Cheers :)

Determination of the interface on 2.0.x and higher installs with PPPoE gets to be tricky. I'm no expert on the internal workings of pfSense, but I do remember seeing in the code where one of the previous developers hard-coded a "ng0" interface when the Snort code detects PPPoE. Perhaps one of the core team developers familiar with how PPPoE maps to interfaces on FreeBSD 8.1 and higher can chime here. I will be happy to fix the Snort code, but I don't know exactly how best to do that with PPPoE. I no longer have a PPPoE setup of my own since I abandoned DSL for my residence and switched over to a cable modem connection that uses DHCP.

Bill

Hi Bill, I also have a cable modem connection that uses DHCP. I've never connected a DSL / PPPoE line to pfSense before.

Cheers :)

bmeeks

@Boags:

Hi Bill, I also have a cable modem connection that uses DHCP. I've never connected a DSL / PPPoE line to pfSense before.

Cheers :)

OK, I think I'm confused. Did you mean in your original post that you do not think Snort is running? And according to this last post you have DHCP with a cable modem on the WAN. If I have those two assumptions, correct, then I'm back on track. Got sidetracked maybe with the PPPoE stuff .. ???

From the log snippet you posted originally, it appears Snort is up and running on two interfaces (LAN and WAN). Your logs are sorted in normal order with the oldest events listed first. The bottom two entries show Snort starting up. How are you determining Snort is not actually running? If you are depending on the icon in the Snort Interfaces tab, note that they are currently "backwards" from the other pfSense GUI. The red X means "running". That will be changing in the next package version so it more accurately matches the rest of the pfSense GUI.

Bill

Boags

@bmeeks:

@Boags:

Hi Bill, I also have a cable modem connection that uses DHCP. I've never connected a DSL / PPPoE line to pfSense before.

Cheers :)

OK, I think I'm confused. Did you mean in your original post that you do not think Snort is running? And according to this last post you have DHCP with a cable modem on the WAN. If I have those two assumptions, correct, then I'm back on track. Got sidetracked maybe with the PPPoE stuff .. ???

From the log snippet you posted originally, it appears Snort is up and running on two interfaces (LAN and WAN). Your logs are sorted in normal order with the oldest events listed first. The bottom two entries show Snort starting up. How are you determining Snort is not actually running? If you are depending on the icon in the Snort Interfaces tab, note that they are currently "backwards" from the other pfSense GUI. The red X means "running". That will be changing in the next package version so it more accurately matches the rest of the pfSense GUI.

Bill

That's correct Bill.

It would appear in the system log that Snort would actually be running, however ram usage (always below 10%), the widget icon (showing that Snort is not running) and the lack of blocking alerts indicates otherwise. Snort doesn't seem to turn on, not even initially.

Things I've tried since my last post; I was able to remove the additional gateways in routing, I turned off 'save settings' in Snort and then uninstalled, re-installed the package with fresh default settings. No luck.

Cheers :)

bmeeks

@Boags:

That's correct Bill.

It would appear in the system log that Snort would actually be running, however ram usage (always below 10%), the widget icon (showing that Snort is not running) and the lack of blocking alerts indicates otherwise. Snort doesn't seem to turn on, not even initially.

Things I've tried since my last post; I was able to remove the additional gateways in routing, I turned off 'save settings' in Snort and then uninstalled, re-installed the package with fresh default settings. No luck.

Cheers :)

OK, from a command prompt on the firewall, execute this command see what it shows:


ps -ax |grep snort

The above command should show two running Snort processes with the associated command-line arguments.

Post back with what it shows, if anything.

As for RAM usage, unless you have a lot of traffic coming through your box, RAM usage may very well stay down in that range. I have 4GB in my firewall and my RAM usage on my home network rarely exceeds 10%-14%.

Bill

Boags

@bmeeks:

@Boags:

That's correct Bill.

It would appear in the system log that Snort would actually be running, however ram usage (always below 10%), the widget icon (showing that Snort is not running) and the lack of blocking alerts indicates otherwise. Snort doesn't seem to turn on, not even initially.

Things I've tried since my last post; I was able to remove the additional gateways in routing, I turned off 'save settings' in Snort and then uninstalled, re-installed the package with fresh default settings. No luck.

Cheers :)

OK, from a command prompt on the firewall, execute this command see what it shows:
ps -ax |grep snort
The above command should show two running Snort processes with the associated command-line arguments.

Post back with what it shows, if anything.

As for RAM usage, unless you have a lot of traffic coming through your box, RAM usage may very well stay down in that range. I have 4GB in my firewall and my RAM usage on my home network rarely exceeds 10%-14%.

Bill

Alright, this is the result:

$ ps -ax |grep snort
58727 ?? S 0:00.00 sh -c ps -ax |grep snort
59007 ?? S 0:00.00 grep snort

Also to note, previously when Snort was always running, I never got below 90% memory. I also leave it on 24-7 so that it gathers a solid blocked hosts list. The performance setting I go with is AC-STD for both WAN and LAN.

Cheers :)

bmeeks

@Boags:

Alright, this is the result:

$ ps -ax |grep snort
58727 ?? S 0:00.00 sh -c ps -ax |grep snort
59007 ?? S 0:00.00 grep snort

Also to note, previously when Snort was always running, I never got below 90% memory. I also leave it on 24-7 so that it gathers a solid blocked hosts list. The performance setting I go with is AC-STD for both WAN and LAN.

Cheers :)

OK. Snort is obviously not running. I'm not familiar with the bge0 or bge1 driver, but a quick Google search suggests a relative standard Broadcom NIC, and since your basic networking is working, it has to be something else.

Let's try some really basic steps first, just in case you might have overlooked something. Follow the steps in this post. The sequence of some of the actions is very important in order for a proper snort.conf file to get created.

http://forum.pfsense.org/index.php/topic,61018.msg328717.html#msg328717

For testing purposes, select only the Emerging Threats CIARMY rule set to start with. Let's see if we can get Snort to run at all before we start throwing more complicated rules at it.

Double check your system logs for a SIGNAL from Snort such as "snort exited on SIGNAL 11" or some other similar message. It is really weird for Snort to just die with no message to the system log and not even any zombie processes left in the process list.

I think you posted earlier that you had done the complete delete and reinstall of the Snort package, but if not, try that process.

Are any other packages installed in the box? There was a library conflict with this version of the Snort binary and another package reported by a user a few weeks back. I don't remember off the top of my head the name of the other conflicting package, but it installed a library Snort did not like. Of course that user was getting an error logged on startup that helped identify the problem.

Bill

Boags

@bmeeks:

@Boags:

Alright, this is the result:

$ ps -ax |grep snort
58727 ?? S 0:00.00 sh -c ps -ax |grep snort
59007 ?? S 0:00.00 grep snort

Also to note, previously when Snort was always running, I never got below 90% memory. I also leave it on 24-7 so that it gathers a solid blocked hosts list. The performance setting I go with is AC-STD for both WAN and LAN.

Cheers :)

OK. Snort is obviously not running. I'm not familiar with the bge0 or bge1 driver, but a quick Google search suggests a relative standard Broadcom NIC, and since your basic networking is working, it has to be something else.

Let's try some really basic steps first, just in case you might have overlooked something. Follow the steps in this post. The sequence of some of the actions is very important in order for a proper snort.conf file to get created.

http://forum.pfsense.org/index.php/topic,61018.msg328717.html#msg328717

For testing purposes, select only the Emerging Threats CIARMY rule set to start with. Let's see if we can get Snort to run at all before we start throwing more complicated rules at it.

Double check your system logs for a SIGNAL from Snort such as "snort exited on SIGNAL 11" or some other similar message. It is really weird for Snort to just die with no message to the system log and not even any zombie processes left in the process list.

I think you posted earlier that you had done the complete delete and reinstall of the Snort package, but if not, try that process.

Are any other packages installed in the box? There was a library conflict with this version of the Snort binary and another package reported by a user a few weeks back. I don't remember off the top of my head the name of the other conflicting package, but it installed a library Snort did not like. Of course that user was getting an error logged on startup that helped identify the problem.

Bill

Alright, I went through the basic steps while only enabling emerging-ciarmy.rules and the system log still has no message.

The other packages I have installed are: arpwatch, pfBlocker and Dashboard Widget: Snort.

Cheers :)

bmeeks

@Boags:

Alright, I went through the basic steps while only enabling emerging-ciarmy.rules and the system log still has no message.

The other packages I have installed are: arpwatch, pfBlocker and Dashboard Widget: Snort.

Cheers :)

At this point I'm pretty much baffled. The next troubleshooting steps will all have to be command-line stuff.

Change to the directory where one of your Snort interface configurations exists, and run the series of commands shown below.

First validate the version of Snort:


/usr/local/bin/snort -V

It should print "Version 2.9.4.1" as part of the output. Next, validate the configuration file.


/usr/local/bin/snort -T -c ./snort.conf

The above will validate the local configuration. It should print several lines of output and at the end indicate the configuration file is OK. Next, try running Snort with no rules at all. In the configuration directory for one of your configured interfaces, execute the following commands to wipe out the rules files and create empty ones and then start Snort. Execute these commands in the order listed.


rm snort.rules
rm flowbit-required.rules
touch snort.rules
touch flowbit-required.rules
/usr/local/etc/rc.d/snort.sh start

The above commands should result in Snort starting on that interface with an empty rule set. To verify it is running, execute this command.


ps -ax |grep snort

You should see an active Snort process with some command-line arguments displayed in the far-right column. If you do, then Snort started successfully. Let it run a few minutes, and keep checking with the same "ps" command periodically to verify it continues to run.

If the above test still does not result in a running Snort process, then something more fundamental is wrong on your box and you might consider a total wipe-and-reload operation.

Bill

Boags

@bmeeks:

@Boags:

Alright, I went through the basic steps while only enabling emerging-ciarmy.rules and the system log still has no message.

The other packages I have installed are: arpwatch, pfBlocker and Dashboard Widget: Snort.

Cheers :)

At this point I'm pretty much baffled. The next troubleshooting steps will all have to be command-line stuff.

Change to the directory where one of your Snort interface configurations exists, and run the series of commands shown below.

First validate the version of Snort:
/usr/local/bin/snort -V
It should print "Version 2.9.4.1" as part of the output. Next, validate the configuration file.
/usr/local/bin/snort -T -c ./snort.conf
The above will validate the local configuration. It should print several lines of output and at the end indicate the configuration file is OK. Next, try running Snort with no rules at all. In the configuration directory for one of your configured interfaces, execute the following commands to wipe out the rules files and create empty ones and then start Snort. Execute these commands in the order listed.
rm snort.rules
rm flowbit-required.rules
touch snort.rules
touch flowbit-required.rules
/usr/local/etc/rc.d/snort.sh start
The above commands should result in Snort starting on that interface with an empty rule set. To verify it is running, execute this command.
ps -ax |grep snort
You should see an active Snort process with some command-line arguments displayed in the far-right column. If you do, then Snort started successfully. Let it run a few minutes, and keep checking with the same "ps" command periodically to verify it continues to run.

If the above test still does not result in a running Snort process, then something more fundamental is wrong on your box and you might consider a total wipe-and-reload operation.

Bill

I think we found the problem, while trying to execute the first few lines of command, it resulted with the following message: /libexec/ld-elf.so.1: Shared object "libpcap.so.1" not found, required by "snort"

I have no idea how a Shared object got missing. Where do we go from here?

Cheers :)

marcelloc

@Boags:

I think we found the problem, while trying to execute the first few lines of command, it resulted with the following message: /libexec/ld-elf.so.1: Shared object "libpcap.so.1" not found, required by "snort"

Try to find what version of libpcap you have on your system

find / -name "libpcap.so*"

bmeeks

Yep, this is that shared library problem I mentioned another user had. It came from a conflicting package install. This does not happen on the 2.1 version of pfSense due to the way PBI packages work. It does bite folks from time to time on 2.0.x, though.

I will be out of town the next three days and unavailable to help, but I see marcelloc has chimed in. Perhaps he can help you sort it out. If not, I will be back online in about 3 days.

Bill

Boags

@bmeeks:

Yep, this is that shared library problem I mentioned another user had. It came from a conflicting package install. This does not happen on the 2.1 version of pfSense due to the way PBI packages work. It does bite folks from time to time on 2.0.x, though.

I will be out of town the next three days and unavailable to help, but I see marcelloc has chimed in. Perhaps he can help you sort it out. If not, I will be back online in about 3 days.

Bill

Thank you for your patience and your professional and friendly support Bill.

A bit of search on the libpcap.so issue lead me to the fix with the following commands:

1.) ln -s /lib/libpcap.so.7 /lib/libpcap.so.1
2.) ln -s /usr/local/lib/snort/dynamicpreprocessor /usr/local/lib/snort_dynamicpreprocessor
3.) ln -s /usr/local/lib/snort/dynamicengine /usr/local/lib/snort_dynamicengine
4.) ln -s /usr/local/lib/snort/dynamicrules /usr/local/lib/snort_dynamicrules
5.) manually update the Snort rules.
6.) touch /usr/local/etc/snort/rules/local.rules

Snort is finally up and running. What a relief. I've missed it :)

Also thank you marcelloc.

Cheers :)

Mr. Jingles

EDIT: solved, please see below.

Then I am jumping in again ;D

For some reason, yesterday at noon snort stopped. No exit signals in the logs, nothing. I tried a 1001 things, including, finally, uninstalling and reinstalling snort, but it still wouldn't run. Searching in this fine forum I found this thread, and executed the two commands mentioned here:


[2.0.3-RELEASE][root@pfsense.localdomain]/root(1): /usr/local/bin/snort -V
/libexec/ld-elf.so.1: Shared object "libpcap.so.1" not found, required by "snort"

And:


[2.0.3-RELEASE][root@pfsense.localdomain]/root(3): find / -name "libpcap.so*"
/lib/libpcap.so.7
/usr/local/lib/libpcap.so.1.3.0
/usr/local/lib/libpcap.so.1.2.1
/usr/lib/libpcap.so

I would have expected that the complete reinstall of Snort would have taken care of this, but obviously not ( ???).

I also don't really understand why the file 'suddenly' is missing, as I didn't do anything on the box for a week or so.

Might I ask if what Boags posted right above here is really 1000% safe? Normally I wouldn't care if something goes completely wrong and I'd even have to reinstall the complete box, but I have just set up OpenVPN for my wife who left to travel to the other side of the world, and I don't want to risk her not being able to interconnect home.

Thank you in advance for an answer ;D

Bye,

EDIT: solved:
While browsing this fine forum I found this thread: http://forum.pfsense.org/index.php/topic,62928.135.html

Where Shinzo posted this:

So funny thing happend, from what i can make out from the logs. Snort rules updated last night. After that it ran the snortstart and it stopped running. Nothing in the logs showed me why it wasnt working but i typed snort into the command line and its giving me a

"/libexec/ld-elf.so.1" shared object "libpcap.so.1" not found, required by snort." So i can only assume the shared object ran off some where Tongue and no i didn't delete it

To continue my story, i found out what deleted it. bandwidthd was maxing out my cpu the other day so i figured i remove it. When i uninstalled it, it took the libpcap file with it too, i reinstalled bandwidthd but left it disabled and snort is running fine again

Exactly my problem, although I hadn't deinstalled bandwithd first. So I did that now, and reinstalled, et voila, Snort is working again ;D

bmeeks

@Hollander:

To continue my story, i found out what deleted it. bandwidthd was maxing out my cpu the other day so i figured i remove it. When i uninstalled it, it took the libpcap file with it too, i reinstalled bandwidthd but left it disabled and snort is running fine again

Exactly my problem, although I hadn't deinstalled bandwithd first. So I did that now, and reinstalled, et voila, Snort is working again ;D

I assume this is on a 2.0.x box? Shared libraries such as pcap, MySQL and others can be a real pain. That's what is so nice about the PBI setup on 2.1 boxes. In effect each app has its own sort of "application jail" where it can install and remove shared libraries as necessary without impacting other applications that may have some of the same shared libraries. In actuality, these libraries are no longer "shared" in the classic sense with the PBI setup. Each application has its own independent copy of them.

Bill

Mr. Jingles

@bmeeks:

I assume this is on a 2.0.x box? Shared libraries such as pcap, MySQL and others can be a real pain. That's what is so nice about the PBI setup on 2.1 boxes. In effect each app has its own sort of "application jail" where it can install and remove shared libraries as necessary without impacting other applications that may have some of the same shared libraries. In actuality, these libraries are no longer "shared" in the classic sense with the PBI setup. Each application has its own independent copy of them.

Bill

Thank you for your answer, Bill, and yes, you are right, this is 2.0.3. I can really appreciate what is in 2.1, and that is why I am eagerly awaiting that it is officially released :P