Snort not restart on interface
-
Okay. Give me some time to investigate in a virtual machine. I also have to look into a Suricata problem on aarch64 hardware for the pfSense team, so it may be a day or two. The only thing I can imagine at this point is maybe something is different in the
rc.start_packages
script and I need to make an adjustment to how the Snort package interracts with it. Just a guess at this point, though.However, even should I find an issue there, I still think you having the cron task will eventually be problematic should its execution happen to coincide with a time when pfSense is responding to one of those loss-of-lcp signal issues.
-
@bmeeks said in Snort not restart on interface:
Okay. Give me some time to investigate in a virtual machine. I also have to look into a Suricata problem on aarch64 hardware for the pfSense team, so it may be a day or two. The only thing I can imagine at this point is maybe something is different in the
rc.start_packages
script and I need to make an adjustment to how the Snort package interracts with it. Just a guess at this point, though.However, even should I find an issue there, I still think you having the cron task will eventually be problematic should its execution happen to coincide with a time when pfSense is responding to one of those loss-of-lcp signal issues.
The cron task is normally in the middle of the night (at 2:10) the probability to do the reconnection on a modem resync is like gaining in Lotto :-)
I hope not to put you on a wrong way. Thank you VERY MUCH for your time and pacience!Regards,
fireodo -
I took some time to have a quick look in a 2.4.5 virtual machine before starting on the Suricata issue. I can see nothing wrong with the interraction of Snort and the
rc.start_packages
script. Everything starts and stops fine in the virtual machine. I don't know what exactly it would be, but I now suspect something unique to your setup.Examine your
config.xml
file for the firewall. It should have a section for <installedpackages><service>, and the <service> section for Snort should look like this --<service> <name>snort</name> <rcfile>snort.sh</rcfile> <executable>snort</executable> <description><![CDATA[Snort IDS/IPS Daemon]]></description> </service>
This is the section that the
rc.start_packges
script scans to find the shell script to execute when starting packages. This same script and section gets called when you boot up the firewall as well. Does Snort start by itself on a firewall reboot, or do you have to intervene manually?One thing I suggest you add to your cron task is a command to shut down Snort and then later restart it again from inside the cron task. Use these commands:
/usr/local/etc/rc.d/snort.sh stop sleep 10 /usr/local/etc/rc.d/snort.sh start
The sleep() command is only required if the stop and start are immediately adjacent to each other. I suggest stopping Snort BEFORE you cycle the PPPoE interface, and then starting Snort again only AFTER the PPPoE interface is back up.
-
@bmeeks said in Snort not restart on interface:
I took some time to have a quick look in a 2.4.5 virtual machine before starting on the Suricata issue. I can see nothing wrong with the interraction of Snort and the
rc.start_packages
script. Everything starts and stops fine in the virtual machine. I don't know what exactly it would be, but I now suspect something unique to your setup.Examine your
config.xml
file for the firewall. It should have a section for <installedpackages><service>, and the <service> section for Snort should look like this --<service> <name>snort</name> <rcfile>snort.sh</rcfile> <executable>snort</executable> <description><![CDATA[Snort IDS/IPS Daemon]]></description> </service>
Is exactly so present in my config.
This is the section that the
rc.start_packges
script scans to find the shell script to execute when starting packages. This same script and section gets called when you boot up the firewall as well. Does Snort start by itself on a firewall reboot, or do you have to intervene manually?Snort starts on reboot by itself no need to intervene manually.
One thing I suggest you add to your cron task is a command to shut down Snort and then later restart it again from inside the cron task. Use these commands:
/usr/local/etc/rc.d/snort.sh stop sleep 10 /usr/local/etc/rc.d/snort.sh start
The sleep() command is only required if the stop and start are immediately adjacent to each other. I suggest stopping Snort BEFORE you cycle the PPPoE interface, and then starting Snort again only AFTER the PPPoE interface is back up.
On normal cycling of pppoe connection snort starts correctly and works correctly - only in case of modem loose connection it makes problems.
I added your command to stop/start snort at the end of the /usr/local/sbin/ppp-linkup script and now snort stays functional even if the modem resyncs! I know thats only a workaraound but I can live with it! Good Night!
-
@fireodo said in Snort not restart on interface:
@bmeeks said in Snort not restart on interface:
I took some time to have a quick look in a 2.4.5 virtual machine before starting on the Suricata issue. I can see nothing wrong with the interraction of Snort and the
rc.start_packages
script. Everything starts and stops fine in the virtual machine. I don't know what exactly it would be, but I now suspect something unique to your setup.Examine your
config.xml
file for the firewall. It should have a section for <installedpackages><service>, and the <service> section for Snort should look like this --<service> <name>snort</name> <rcfile>snort.sh</rcfile> <executable>snort</executable> <description><![CDATA[Snort IDS/IPS Daemon]]></description> </service>
Is exactly so present in my config.
This is the section that the
rc.start_packges
script scans to find the shell script to execute when starting packages. This same script and section gets called when you boot up the firewall as well. Does Snort start by itself on a firewall reboot, or do you have to intervene manually?Snort starts on reboot by itself no need to intervene manually.
One thing I suggest you add to your cron task is a command to shut down Snort and then later restart it again from inside the cron task. Use these commands:
/usr/local/etc/rc.d/snort.sh stop sleep 10 /usr/local/etc/rc.d/snort.sh start
The sleep() command is only required if the stop and start are immediately adjacent to each other. I suggest stopping Snort BEFORE you cycle the PPPoE interface, and then starting Snort again only AFTER the PPPoE interface is back up.
On normal cycling of pppoe connection snort starts correctly and works correctly - only in case of modem loose connection it makes problems.
Would it be a good idea to add this command you suggested to the /usr/local/sbin/ppp-linkup script?
I don't think it would hurt anything, but of course it would get wiped out with the next upgrade and you would have to remember to edit the script again.
I'm really not a PPPoE expert. I had DSL way, way back and dealt with PPPoE just a bit back then. But luckily with my carrier at the time, and with the DSL modem I had, I was able to switch over to PPPoA (PPP over ATM). I did that to take advantage of the full 1500-byte MTU instead of having to limit my MTU to 1492.
So you say your modem is in bridge mode, but I don't believe it actually is if you have a
pppoe0
connection in pfsense. Or at least that's not how my DSL modem worked. I put the password and login name in the modem and it handled all the PPP stuff. What I got on the LAN side of the modem was a DHCP address (the public IP assigned by my ISP). Does your modem not support a mode of operation like that? It's been a long time, so maybe that was how the PPPoA worked for me and PPPoE would have been different.Edit: the more I've tried to remember, perhaps I am remembering wrong for PPPoE. Maybe it did put a PPP interface on my pfSense and only the PPPoA mode was DHCP on the firewall side. That was 17 years ago and I can't recall the details now.
-
@bmeeks said in Snort not restart on interface:
@fireodo said in Snort not restart on interface:
@bmeeks said in Snort not restart on interface:
I took some time to have a quick look in a 2.4.5 virtual machine before starting on the Suricata issue. I can see nothing wrong with the interraction of Snort and the
rc.start_packages
script. Everything starts and stops fine in the virtual machine. I don't know what exactly it would be, but I now suspect something unique to your setup.Examine your
config.xml
file for the firewall. It should have a section for <installedpackages><service>, and the <service> section for Snort should look like this --<service> <name>snort</name> <rcfile>snort.sh</rcfile> <executable>snort</executable> <description><![CDATA[Snort IDS/IPS Daemon]]></description> </service>
Is exactly so present in my config.
This is the section that the
rc.start_packges
script scans to find the shell script to execute when starting packages. This same script and section gets called when you boot up the firewall as well. Does Snort start by itself on a firewall reboot, or do you have to intervene manually?Snort starts on reboot by itself no need to intervene manually.
One thing I suggest you add to your cron task is a command to shut down Snort and then later restart it again from inside the cron task. Use these commands:
/usr/local/etc/rc.d/snort.sh stop sleep 10 /usr/local/etc/rc.d/snort.sh start
The sleep() command is only required if the stop and start are immediately adjacent to each other. I suggest stopping Snort BEFORE you cycle the PPPoE interface, and then starting Snort again only AFTER the PPPoE interface is back up.
On normal cycling of pppoe connection snort starts correctly and works correctly - only in case of modem loose connection it makes problems.
Would it be a good idea to add this command you suggested to the /usr/local/sbin/ppp-linkup script?
I don't think it would hurt anything, but of course it would get wiped out with the next upgrade and you would have to remember to edit the script again.
That I have to do, thats no problem ... :-)
-
the more i think about it the more i find it strange
i put up a virtual machine and tested it myself
Mar 31 20:38:27 php-fpm 95633 /rc.start_packages: Restarting/Starting all packages. Mar 31 20:38:26 check_reload_status 381 Starting packages Mar 31 20:38:26 php-fpm 343 /rc.newwanip: pfSense package system has detected an IP change or dynamic WAN reconnection - 192.168.78.2 -> 192.168.78.2 - Restarting packages. Mar 31 20:38:24 php-fpm 343 /rc.newwanip: Creating rrd update script Mar 31 20:38:24 php-fpm 343 /rc.newwanip: Resyncing OpenVPN instances for interface OPT1. Mar 31 20:38:21 php-fpm 343 /rc.newwanip: rc.newwanip: on (IP address: 192.168.78.2) (interface: OPT1[opt1]) (real interface: pppoe0). Mar 31 20:38:21 php-fpm 343 /rc.newwanip: rc.newwanip: Info: starting on pppoe0. Mar 31 20:38:20 ppp 12008 [opt1] IFACE: Rename interface ng0 to pppoe0 Mar 31 20:38:20 ppp 12008 [opt1] IFACE: Up event Mar 31 20:38:20 check_reload_status 381 rc.newwanip starting pppoe0 Mar 31 20:38:19 check_reload_status 381 Rewriting resolv.conf Mar 31 20:38:19 ppp 12008 [opt1] 192.168.78.2 -> 192.168.77.1 Mar 31 20:38:19 ppp 12008 [opt1] IPCP: LayerUp Mar 31 20:38:19 ppp 12008 [opt1] IPCP: state change Ack-Sent --> Opened Mar 31 20:38:19 ppp 12008 [opt1] PRIDNS 172.17.0.100 Mar 31 20:38:19 ppp 12008 [opt1] IPADDR 192.168.78.2 Mar 31 20:38:19 ppp 12008 [opt1] IPCP: rec'd Configure Ack #8 (Ack-Sent) Mar 31 20:38:19 ppp 12008 [opt1] PRIDNS 172.17.0.100 Mar 31 20:38:19 ppp 12008 [opt1] IPADDR 192.168.78.2 Mar 31 20:38:19 ppp 12008 [opt1] IPCP: SendConfigReq #8 Mar 31 20:38:19 ppp 12008 [opt1] PRIDNS 172.17.0.100 Mar 31 20:38:19 ppp 12008 [opt1] 192.168.78.2 is OK Mar 31 20:38:19 ppp 12008 [opt1] IPADDR 192.168.78.2 Mar 31 20:38:19 ppp 12008 [opt1] IPCP: rec'd Configure Nak #7 (Ack-Sent) Mar 31 20:38:19 ppp 12008 [opt1] PRIDNS 0.0.0.0 Mar 31 20:38:19 ppp 12008 [opt1] IPADDR 0.0.0.0 Mar 31 20:38:19 ppp 12008 [opt1] IPCP: SendConfigReq #7 Mar 31 20:38:19 ppp 12008 [opt1] SECDNS 0.0.0.0 Mar 31 20:38:19 ppp 12008 [opt1] COMPPROTO VJCOMP, 16 comp. channels, no comp-cid Mar 31 20:38:19 ppp 12008 [opt1] IPCP: rec'd Configure Reject #6 (Ack-Sent) Mar 31 20:38:19 ppp 12008 [opt1_link0] rec'd unexpected protocol CCP, rejecting Mar 31 20:38:19 ppp 12008 [opt1] IPCP: state change Req-Sent --> Ack-Sent Mar 31 20:38:19 ppp 12008 [opt1] IPADDR 192.168.77.1 Mar 31 20:38:19 ppp 12008 [opt1] IPCP: SendConfigAck #1 Mar 31 20:38:19 ppp 12008 [opt1] 192.168.77.1 is OK Mar 31 20:38:19 ppp 12008 [opt1] IPADDR 192.168.77.1 Mar 31 20:38:19 ppp 12008 [opt1] IPCP: rec'd Configure Request #1 (Req-Sent) Mar 31 20:38:19 ppp 12008 [opt1] SECDNS 0.0.0.0 Mar 31 20:38:19 ppp 12008 [opt1] PRIDNS 0.0.0.0 Mar 31 20:38:19 ppp 12008 [opt1] COMPPROTO VJCOMP, 16 comp. channels, no comp-cid Mar 31 20:38:19 ppp 12008 [opt1] IPADDR 0.0.0.0 Mar 31 20:38:19 ppp 12008 [opt1] IPCP: SendConfigReq #6 Mar 31 20:38:19 ppp 12008 [opt1] IPCP: state change Starting --> Req-Sent Mar 31 20:38:19 ppp 12008 [opt1] IPCP: Up event Mar 31 20:38:19 ppp 12008 [opt1] IPCP: LayerStart Mar 31 20:38:19 ppp 12008 [opt1] IPCP: state change Initial --> Starting Mar 31 20:38:19 ppp 12008 [opt1] IPCP: Open event Mar 31 20:38:19 ppp 12008 [opt1] Bundle: Status update: up 1 link, total bandwidth 64000 bps Mar 31 20:38:19 ppp 12008 [opt1_link0] Link: Join bundle "opt1" Mar 31 20:38:19 ppp 12008 [opt1_link0] Link: Matched action 'bundle "opt1" ""' Mar 31 20:38:19 ppp 12008 [opt1_link0] LCP: authorization successful Mar 31 20:38:19 ppp 12008 [opt1_link0] MESG: Welcome Mar 31 20:38:19 ppp 12008 [opt1_link0] PAP: rec'd ACK #1 len: 12 Mar 31 20:38:19 ppp 12008 [opt1_link0] LCP: LayerUp Mar 31 20:38:19 ppp 12008 [opt1_link0] PAP: sending REQUEST #1 len: 14 Mar 31 20:38:19 ppp 12008 [opt1_link0] PAP: using authname "test" Mar 31 20:38:19 ppp 12008 [opt1_link0] LCP: auth: peer wants PAP, I want nothing Mar 31 20:38:19 ppp 12008 [opt1_link0] LCP: state change Ack-Sent --> Opened Mar 31 20:38:19 ppp 12008 [opt1_link0] MAGICNUM 0xe92dbee8 Mar 31 20:38:19 ppp 12008 [opt1_link0] MRU 1492 Mar 31 20:38:19 ppp 12008 [opt1_link0] PROTOCOMP Mar 31 20:38:19 ppp 12008 [opt1_link0] LCP: rec'd Configure Ack #3 (Ack-Sent) Mar 31 20:38:19 ppp 12008 [opt1_link0] LCP: state change Req-Sent --> Ack-Sent Mar 31 20:38:19 ppp 12008 [opt1_link0] AUTHPROTO PAP Mar 31 20:38:19 ppp 12008 [opt1_link0] MAGICNUM 0xe7f15140 Mar 31 20:38:19 ppp 12008 [opt1_link0] MRU 1492 Mar 31 20:38:19 ppp 12008 [opt1_link0] PROTOCOMP Mar 31 20:38:19 ppp 12008 [opt1_link0] LCP: SendConfigAck #1 Mar 31 20:38:19 ppp 12008 [opt1_link0] AUTHPROTO PAP Mar 31 20:38:19 ppp 12008 [opt1_link0] MAGICNUM 0xe7f15140 Mar 31 20:38:19 ppp 12008 [opt1_link0] MRU 1492 Mar 31 20:38:19 ppp 12008 [opt1_link0] PROTOCOMP Mar 31 20:38:19 ppp 12008 [opt1_link0] LCP: rec'd Configure Request #1 (Req-Sent) Mar 31 20:38:19 ppp 12008 [opt1_link0] MAGICNUM 0xe92dbee8 Mar 31 20:38:19 ppp 12008 [opt1_link0] MRU 1492 Mar 31 20:38:19 ppp 12008 [opt1_link0] PROTOCOMP Mar 31 20:38:19 ppp 12008 [opt1_link0] LCP: SendConfigReq #3 Mar 31 20:38:19 ppp 12008 [opt1_link0] LCP: state change Starting --> Req-Sent Mar 31 20:38:19 ppp 12008 [opt1_link0] LCP: Up event Mar 31 20:38:19 ppp 12008 [opt1_link0] Link: UP event Mar 31 20:38:19 ppp 12008 [opt1_link0] PPPoE: connection successful Mar 31 20:38:19 ppp 12008 PPPoE: rec'd ACNAME "pfSense.kiokoman.home" Mar 31 20:38:17 ppp 12008 [opt1_link0] PPPoE: Connecting to '' Mar 31 20:38:17 ppp 12008 [opt1_link0] Link: reconnection attempt 20 Mar 31 20:38:13 ppp 12008 [opt1_link0] Link: reconnection attempt 20 in 4 seconds ...... Mar 31 20:34:31 ppp 12008 [opt1] IPCP: LayerFinish Mar 31 20:34:31 ppp 12008 [opt1] IPCP: Down event Mar 31 20:34:31 ppp 12008 [opt1] IPCP: state change Stopping --> Closing Mar 31 20:34:31 ppp 12008 [opt1] IPCP: Close event Mar 31 20:34:31 ppp 12008 [opt1] Bundle: Status update: up 0 links, total bandwidth 9600 bps Mar 31 20:34:31 ppp 12008 [opt1_link0] Link: Leave bundle "opt1" Mar 31 20:34:31 ppp 12008 [opt1_link0] LCP: state change Opened --> Stopping Mar 31 20:34:31 ppp 12008 [opt1_link0] LCP: rec'd Terminate Request #2 (Opened) Mar 31 20:34:31 ppp 12008 [opt1] IFACE: Rename interface pppoe0 to pppoe0 Mar 31 20:34:31 ppp 12008 [opt1] IFACE: Down event Mar 31 20:34:31 check_reload_status 381 Rewriting resolv.conf Mar 31 20:34:30 ppp 12008 [opt1] IPCP: LayerDown Mar 31 20:34:30 ppp 12008 [opt1] IPCP: SendTerminateAck #5 Mar 31 20:34:30 ppp 12008 [opt1] IPCP: state change Opened --> Stopping Mar 31 20:34:30 ppp 12008 [opt1] IPCP: rec'd Terminate Request #3 (Opened) Mar 31 20:25:07 kernel pppoe0: promiscuous mode enabled Mar 31 20:25:07 SnortStartup 5730 Snort START for pppoe(pppoe0)... Mar 31 20:25:07 php-fpm 95633 /rc.start_packages: Restarting/Starting all packages. Mar 31 20:25:06 check_reload_status 381 Starting packages Mar 31 20:25:06 php-fpm 344 /rc.newwanip: pfSense package system has detected an IP change or dynamic WAN reconnection - 192.168.78.2 -> 192.168.78.2 - Restarting packages. Mar 31 20:25:04 php-fpm 344 /rc.newwanip: Creating rrd update script Mar 31 20:25:04 php-fpm 344 /rc.newwanip: Resyncing OpenVPN instances for interface OPT1. Mar 31 20:25:00 php-fpm 344 /rc.newwanip: rc.newwanip: on (IP address: 192.168.78.2) (interface: OPT1[opt1]) (real interface: pppoe0). Mar 31 20:25:00 php-fpm 344 /rc.newwanip: rc.newwanip: Info: starting on pppoe0. Mar 31 20:24:59 ppp 12008 [opt1] IFACE: Rename interface ng0 to pppoe0 Mar 31 20:24:59 ppp 12008 [opt1] IFACE: Up event Mar 31 20:24:59 check_reload_status 381 rc.newwanip starting pppoe0 Mar 31 20:24:58 check_reload_status 381 Rewriting resolv.conf Mar 31 20:24:58 ppp 12008 [opt1] 192.168.78.2 -> 192.168.77.1
snort start correctly, does not matter if i stop the pppoe server or i stop the pppoe client
but as you can see " IFACE Rename interface ng0 to pppoe" is present
this is mpd5 doing
it's not snort at fault here, you should investigate why the pppoe interface disappear leadig snort to stop working, and i don't think it's the only problem you will get from it
the test was done under pfSense 2.5.0, maybe i will try tomorrow with a new vm with 2.4.5 -
@kiokoman said in Snort not restart on interface:
the more i think about it the more i find it strange
it's not snort at fault here, you should investigate why the pppoe interface disappear leadig snort to stop working, and i don't think it's the only problem you will get from itI'll keep an eye on it!
the test was done under pfSense 2.5.0, maybe i will try tomorrow with a new vm with 2.4.5
OK, lets see what happends
Thanks for your effort!
fireodo -
@kiokoman, thank you for the testing. I too am intrigued by why the interface disappears. That will obviously confuse Snort when it can't find the interface it is configured to sniff.
Initially I thought Snort was just a victim here, but I decided to look into other possibilities. However, that search and my own testing led me to think Snort is not really at fault here.
-
ok tested it today with a vm 2.4.5, same results
no problem when i stop the pppoe server or the client, must be something on your environment,or some special event that i can't replicate. did you set any particular settings for your pppoe connection? system tunable ? -
@kiokoman said in Snort not restart on interface:
ok tested it today with a vm 2.4.5, same results
no problem when i stop the pppoe server or the client, must be something on your environment, did you set any particular settings for your pppoe connection? system tunable ?Nothing special - only the credentials necessary for provider. I dont use a pppoe server only a client. When I restart here the "wan" (pppoe) there is no problem - the problem occurs when by any circumstances the modem loose sync/or power.
DSLAM(provider)------>(my)DSL-Modem ------> pfsense (pppoe client) ------> LAN
-
yes , the pppoe server is used by me on another pfsense to simulate a provider for another vm with pfsense 2.4.5
so if i stop the pppoe server is like as you turn off your dsl modem
wan ->pfsense (pppoe server) --> another pfsense (pppoe client) -> lan -
@kiokoman said in Snort not restart on interface:
yes , the pppoe server is used by me on another pfsense to simulate a provider for another vm with pfsense 2.4.5
so if i stop the pppoe server is like as you turn off your dsl modem
wan ->pfsense (pppoe server) --> another pfsense (pppoe client) -> lanAh - OK! Thanks for testing - now I have to dig further to find what in my case went wrong.
-
@fireodo said in Snort not restart on interface:
@kiokoman said in Snort not restart on interface:
yes , the pppoe server is used by me on another pfsense to simulate a provider for another vm with pfsense 2.4.5
so if i stop the pppoe server is like as you turn off your dsl modem
wan ->pfsense (pppoe server) --> another pfsense (pppoe client) -> lanAh - OK! Thanks for testing - now I have to dig further to find what in my case went wrong.
Don't forget to either delete or go back and update your bug report on the pfSense Redmine site so that issue can maybe be closed if not an actual system bug.
-
@bmeeks said in Snort not restart on interface:
@fireodo said in Snort not restart on interface:
@kiokoman said in Snort not restart on interface:
yes , the pppoe server is used by me on another pfsense to simulate a provider for another vm with pfsense 2.4.5
so if i stop the pppoe server is like as you turn off your dsl modem
wan ->pfsense (pppoe server) --> another pfsense (pppoe client) -> lanAh - OK! Thanks for testing - now I have to dig further to find what in my case went wrong.
Don't forget to either delete or go back and update your bug report on the pfSense Redmine site so that issue can maybe be closed if not an actual system bug.
Thanks, I'll do so!
-
This is a very interesting case study, and analysis...thank you all for sharing!