Snort not restart on interface
-
Hi all,
i found an issue that occurs normally very rare. I have a VDSL internet connection that works via pppoe. VDSL Modem is in Bridge mode. Pfsense is at version 2.4.5 and Snort at version 3.2.9.10_2. Snort is activated only on the WAN Interface.
Now the problem: when the modem gets out of sync the connection gets lost and after the resync the connection gets reestablished, pppoe gets a new IP and the packages gets restarted but NOT Snort on the WAN Interface, Snort on WAN stays with the previous IP. (the rc.start_packages is restarting Snort but NOT on the WAN Interface and Snort on WAN Interface dont get the new IP). This mean in fact that Snort does not do his Job anymore. I have to restart Snort on that Interface manually, then it works again. If I restart the pppoe manually (via /var/etc/pppoe_restart_pppoe0 everything works as expected) What I also have seen is that in the dmesg, when I restart voluntary the pppoe, I get a: "ng0: changing name to 'pppoe0'" but when the modem resyncs this line is missing in dmesg.
I hope I explained the issue as good as possible. In short terms, in the regular case of disconnection and reconnection (made by cron or manually) everything went fine but in case of Modem resync (generated by Provider) the problem occurs.
Maybe someone have an Idea how to resolute this issue.Thanks in advance,
fireodo -
That missing
dmesgline is highly likely to be related to your problem. Snort can't "find" thepppoe0interface if the name of theng0interface is not changed. That's my guess based on the data you provided.You may want to open a bug report on the Redmine site here: https://redmine.pfsense.org/. But your bug really should be directed at the rc.start_packages command and not Snort itself. I think Snort is a victim in this case and not the perpetrator (the interface not being properly renamed and re-configured when the modem loses sync and re-establishes it is what I think is the root cause).
-
@bmeeks said in Snort not restart on interface:
That missing
dmesgline is highly likely to be related to your problem. Snort can't "find" thepppoe0interface if the name of theng0interface is not changed. That's my guess based on the data you provided.You may want to open a bug report on the Redmine site here: https://redmine.pfsense.org/. But your bug really should be directed at the rc.start_packages command and not Snort itself. I think Snort is a victim in this case and not the perpetrator (the interface not being properly renamed and re-configured when the modem loses sync and re-establishes it is what I think is the root cause).
OK, thank you very much for analyzing the problem. I have to create a redmine account first and then I post a bug report! Thanks also for your advice.
Here is the Bug-Report: https://redmine.pfsense.org/issues/10396 I hope thats OK so!Regards,
fireodo -
@fireodo:
One of the pfSense developer team members responded to your bug report. They need some more information from you in the form of logs with a few different test scenarios. Also they say it is not therc.start_packagesscript but might be elsewhere in other parts of FreeBSD instead.Also, after going back and reading your original post again, I realized that because of the way the sync loss is happening and being remediated by the modem, Snort is not aware of the change in IP on the
pppoe0interface. Snort has internal code that subscribes to the FreeBSD routing messages API so that it should be notified when an interface IP changes. Not really sure what happens with something like a PPPoE interface, though, since that is not exactly the same as a physical NIC interface. -
@bmeeks said in Snort not restart on interface:
@fireodo:
One of the pfSense developer team members responded to your bug report. They need some more information from you in the form of logs with a few different test scenarios. Also they say it is not therc.start_packagesscript but might be elsewhere in other parts of FreeBSD instead.Yes, I have read the message from Mr. Jim Pingle and he is right that the bug report is incomplete and as a consequence I have downloaded from my Router the ppp.log and the system.log. I intent to analyze these files and I hope to find the difference between the 2 situations: regular disconnect versus accidentally disconnect.
Also, after going back and reading your original post again, I realized that because of the way the sync loss is happening and being remediated by the modem, Snort is not aware of the change in IP on the
pppoe0interface. Snort has internal code that subscribes to the FreeBSD routing messages API so that it should be notified when an interface IP changes. Not really sure what happens with something like a PPPoE interface, though, since that is not exactly the same as a physical NIC interface.If I think correct the pppo0 interface is recreated on every successful connection, that means, following your thoughts, it has to be something in the recreation of the ng0 (pppoe0) that runs different in the 2 case scenarios.
Thanks anyway for pointing me some hints out!
Regards,
firodo -
And things get even more confusing because Snort is actually using the libpcap library to capture packets on the configured interface. So that means libpcap has to see the interface change as well.
Not saying Snort is without fault here, but there very well could be other issues as well.
-
@bmeeks said in Snort not restart on interface:
And things get even more confusing because Snort is actually using the libpcap library to capture packets on the configured interface. So that means libpcap has to see the interface change as well.
Not saying Snort is without fault here, but there very well could be other issues as well.
I'm glad that you make yourself also thoughts about this. This kind of disconnection is very rare, it occurs only when the provider makes any changes to the DSLAM (or in his Network). This was last weekend the case (after about 165 days without resync) and so I discovered the issue (thanks to Corona I am in Homeoffice so I have more time to check things in my Home-network ;-) ). One day before came the 2.4.5 update so my first thought was in context of the update but now I think that could be a more older problem ...
-
I have struggled mightily to try and keep Snort and Suricata cognizant of WAN IP changes, especially for automatic pass list protection so the WAN IP does not get blocked. That's the purpose of the FreeBSD routing message subscription code I mentioned earlier -- to let the IPS know when the WAN IP changes so it can update its internal automatic pass list.
I think your issue might be slightly different. A PPPoE interface is a virtual interface that is created on top of a physical interface (in your case that is
ng0). Did the change confuse just Snort, or did you lose Internet connectivity until you restarted the interface on pfSense? -
@bmeeks said in Snort not restart on interface:
I have struggled mightily to try and keep Snort and Suricata cognizant of WAN IP changes, especially for automatic pass list protection so the WAN IP does not get blocked. That's the purpose of the FreeBSD routing message subscription code I mentioned earlier -- to let the IPS know when the WAN IP changes so it can update its internal automatic pass list.
I think your issue might be slightly different. A PPPoE interface is a virtual interface that is created on top of a physical interface (in your case that is
ng0). Did the change confuse just Snort, or did you lose Internet connectivity until you restarted the interface on pfSense?Its only Snort. After this special disconnection occurs and it reconnects i have Internet connectivity and all other things work as expected (that why I saw the Snort-Problem only after a while) - even Snort is not dying and when I restart snort on wan it works flawless!
-
@fireodo:
So Snort is not stopping, it just stops giving alerts?If that is the case, then it is going to be related to the
pppoe0virtual interface in some manner. And further, it may belibpcapthat is "losing" the interface because it does not see the change. I am not familiar with the internals of thelibpcaplibrary. -
@bmeeks said in Snort not restart on interface:
@fireodo:
So Snort is not stopping, it just stops giving alerts?If that is the case, then it is going to be related to the
pppoe0virtual interface in some manner. And further, it may belibpcapthat is "losing" the interface because it does not see the change. I am not familiar with the internals of thelibpcaplibrary.Yes it stop giving alerts and I see in the last alert given, that the alert refers to the previous IP!
This is normal in dmesg:
pppoe0: promiscuous mode disabled # before pppoe reconnection snort shutdown
ng0: changing name to 'pppoe0' # pppoe reconnection and up
pppoe0: promiscuous mode enabled # after pppoe reconnection snort restartIn the special disconnection case the middle line is missing.
-
@fireodo said in Snort not restart on interface:
@bmeeks said in Snort not restart on interface:
@fireodo:
So Snort is not stopping, it just stops giving alerts?If that is the case, then it is going to be related to the
pppoe0virtual interface in some manner. And further, it may belibpcapthat is "losing" the interface because it does not see the change. I am not familiar with the internals of thelibpcaplibrary.Yes it stop giving alerts and I see in the last alert given, that the alert refers to the previous IP!
This is normal in dmesg:
pppoe0: promiscuous mode disabled # before pppoe reconnection snort shutdown
ng0: changing name to 'pppoe0' # pppoe reconnection and up
pppoe0: promiscuous mode enabled # after pppoe reconnection snort restartIn the special disconnection case the middle line is missing.
Yes, that could be a problem either in FreeBSD with the plumbing of PPPoE interfaces, or it could be an issue inside the
libpcaplibrary. I'm kind of leaning towardslibpcap, but that's just a hunch.Is the IP change triggering Snort to restart? It should because pfSense has (or had) an internal process that restarted packages when IP addresses changed. If Snort restarts, then it surely should see the new IP. The pfSense package restart script simply executes a shell script provided by each installed package. The Snort shell script is
/usr/local/etc/rc.d/snort.sh. That script should be called with "restart" or else "stop, start" as arguments. So you should see system log entries from that shell script showing it shutting down Snort and starting it up again.Seeing a copy of your system log while this interruption is happening would be helpful.
-
@bmeeks said in Snort not restart on interface:
@fireodo said in Snort not restart on interface:
@bmeeks said in Snort not restart on interface:
@fireodo:
So Snort is not stopping, it just stops giving alerts?If that is the case, then it is going to be related to the
pppoe0virtual interface in some manner. And further, it may belibpcapthat is "losing" the interface because it does not see the change. I am not familiar with the internals of thelibpcaplibrary.Yes it stop giving alerts and I see in the last alert given, that the alert refers to the previous IP!
This is normal in dmesg:
pppoe0: promiscuous mode disabled # before pppoe reconnection snort shutdown
ng0: changing name to 'pppoe0' # pppoe reconnection and up
pppoe0: promiscuous mode enabled # after pppoe reconnection snort restartIn the special disconnection case the middle line is missing.
Yes, that could be a problem either in FreeBSD with the plumbing of PPPoE interfaces, or it could be an issue inside the
libpcaplibrary. I'm kind of leaning towardslibpcap, but that's just a hunch.Is the IP change triggering Snort to restart? It should because pfSense has (or had) an internal process that restarted packages when IP addresses changed. If Snort restarts, then it surely should see the new IP. The pfSense package restart script simply executes a shell script provided by each installed package. The Snort shell script is
/usr/local/etc/rc.d/snort.sh. That script should be called with "restart" or else "stop, start" as arguments. So you should see system log entries from that shell script showing it shutting down Snort and starting it up again.Seeing a copy of your system log while this interruption is happening would be helpful.
I would like to send you that logfile private - how can I do that?
-
@fireodo said in Snort not restart on interface:
@bmeeks said in Snort not restart on interface:
@fireodo said in Snort not restart on interface:
@bmeeks said in Snort not restart on interface:
@fireodo:
So Snort is not stopping, it just stops giving alerts?If that is the case, then it is going to be related to the
pppoe0virtual interface in some manner. And further, it may belibpcapthat is "losing" the interface because it does not see the change. I am not familiar with the internals of thelibpcaplibrary.Yes it stop giving alerts and I see in the last alert given, that the alert refers to the previous IP!
This is normal in dmesg:
pppoe0: promiscuous mode disabled # before pppoe reconnection snort shutdown
ng0: changing name to 'pppoe0' # pppoe reconnection and up
pppoe0: promiscuous mode enabled # after pppoe reconnection snort restartIn the special disconnection case the middle line is missing.
Yes, that could be a problem either in FreeBSD with the plumbing of PPPoE interfaces, or it could be an issue inside the
libpcaplibrary. I'm kind of leaning towardslibpcap, but that's just a hunch.Is the IP change triggering Snort to restart? It should because pfSense has (or had) an internal process that restarted packages when IP addresses changed. If Snort restarts, then it surely should see the new IP. The pfSense package restart script simply executes a shell script provided by each installed package. The Snort shell script is
/usr/local/etc/rc.d/snort.sh. That script should be called with "restart" or else "stop, start" as arguments. So you should see system log entries from that shell script showing it shutting down Snort and starting it up again.Seeing a copy of your system log while this interruption is happening would be helpful.
I would like to send you that logfile private - how can I do that?
All I need to see are the handful of lines related to Snort being restarted. There is no private information in those entries. Just snip out or otherwise copy and paste the relevant lines into a post here. To keep from getting overwhelmed from users around the world, I don't do any one-on-one troubleshooting with users. I like to keep all the interractions public so the entire user community can see issues and benefit from also seeing the eventual solutions.
-
@bmeeks said in Snort not restart on interface:
@fireodo said in Snort not restart on interface:
@bmeeks said in Snort not restart on interface:
@fireodo said in Snort not restart on interface:
@bmeeks said in Snort not restart on interface:
@fireodo:
So Snort is not stopping, it just stops giving alerts?If that is the case, then it is going to be related to the
pppoe0virtual interface in some manner. And further, it may belibpcapthat is "losing" the interface because it does not see the change. I am not familiar with the internals of thelibpcaplibrary.Yes it stop giving alerts and I see in the last alert given, that the alert refers to the previous IP!
This is normal in dmesg:
pppoe0: promiscuous mode disabled # before pppoe reconnection snort shutdown
ng0: changing name to 'pppoe0' # pppoe reconnection and up
pppoe0: promiscuous mode enabled # after pppoe reconnection snort restartIn the special disconnection case the middle line is missing.
Yes, that could be a problem either in FreeBSD with the plumbing of PPPoE interfaces, or it could be an issue inside the
libpcaplibrary. I'm kind of leaning towardslibpcap, but that's just a hunch.Is the IP change triggering Snort to restart? It should because pfSense has (or had) an internal process that restarted packages when IP addresses changed. If Snort restarts, then it surely should see the new IP. The pfSense package restart script simply executes a shell script provided by each installed package. The Snort shell script is
/usr/local/etc/rc.d/snort.sh. That script should be called with "restart" or else "stop, start" as arguments. So you should see system log entries from that shell script showing it shutting down Snort and starting it up again.Seeing a copy of your system log while this interruption is happening would be helpful.
I would like to send you that logfile private - how can I do that?
All I need to see are the handful of lines related to Snort being restarted. There is no private information in those entries. Just snip out or otherwise copy and paste the relevant lines into a post here. To keep from getting overwhelmed from users around the world, I don't do any one-on-one troubleshooting with users. I like to keep all the interractions public so the entire user community can see issues and benefit from also seeing the eventual solutions.
Is this sufficient: clog /var/log/system.log | grep snort ?
-
@fireodo said in Snort not restart on interface:
@bmeeks said in Snort not restart on interface:
@fireodo said in Snort not restart on interface:
@bmeeks said in Snort not restart on interface:
@fireodo said in Snort not restart on interface:
@bmeeks said in Snort not restart on interface:
@fireodo:
So Snort is not stopping, it just stops giving alerts?If that is the case, then it is going to be related to the
pppoe0virtual interface in some manner. And further, it may belibpcapthat is "losing" the interface because it does not see the change. I am not familiar with the internals of thelibpcaplibrary.Yes it stop giving alerts and I see in the last alert given, that the alert refers to the previous IP!
This is normal in dmesg:
pppoe0: promiscuous mode disabled # before pppoe reconnection snort shutdown
ng0: changing name to 'pppoe0' # pppoe reconnection and up
pppoe0: promiscuous mode enabled # after pppoe reconnection snort restartIn the special disconnection case the middle line is missing.
Yes, that could be a problem either in FreeBSD with the plumbing of PPPoE interfaces, or it could be an issue inside the
libpcaplibrary. I'm kind of leaning towardslibpcap, but that's just a hunch.Is the IP change triggering Snort to restart? It should because pfSense has (or had) an internal process that restarted packages when IP addresses changed. If Snort restarts, then it surely should see the new IP. The pfSense package restart script simply executes a shell script provided by each installed package. The Snort shell script is
/usr/local/etc/rc.d/snort.sh. That script should be called with "restart" or else "stop, start" as arguments. So you should see system log entries from that shell script showing it shutting down Snort and starting it up again.Seeing a copy of your system log while this interruption is happening would be helpful.
I would like to send you that logfile private - how can I do that?
All I need to see are the handful of lines related to Snort being restarted. There is no private information in those entries. Just snip out or otherwise copy and paste the relevant lines into a post here. To keep from getting overwhelmed from users around the world, I don't do any one-on-one troubleshooting with users. I like to keep all the interractions public so the entire user community can see issues and benefit from also seeing the eventual solutions.
Is this sufficient: clog /var/log/system.log | grep snort ?
Yes, that should do it.
-
@bmeeks said in Snort not restart on interface:
@fireodo said in Snort not restart on interface:
@bmeeks said in Snort not restart on interface:
@fireodo said in Snort not restart on interface:
@bmeeks said in Snort not restart on interface:
@fireodo said in Snort not restart on interface:
@bmeeks said in Snort not restart on interface:
@fireodo:
So Snort is not stopping, it just stops giving alerts?If that is the case, then it is going to be related to the
pppoe0virtual interface in some manner. And further, it may belibpcapthat is "losing" the interface because it does not see the change. I am not familiar with the internals of thelibpcaplibrary.Yes it stop giving alerts and I see in the last alert given, that the alert refers to the previous IP!
This is normal in dmesg:
pppoe0: promiscuous mode disabled # before pppoe reconnection snort shutdown
ng0: changing name to 'pppoe0' # pppoe reconnection and up
pppoe0: promiscuous mode enabled # after pppoe reconnection snort restartIn the special disconnection case the middle line is missing.
Yes, that could be a problem either in FreeBSD with the plumbing of PPPoE interfaces, or it could be an issue inside the
libpcaplibrary. I'm kind of leaning towardslibpcap, but that's just a hunch.Is the IP change triggering Snort to restart? It should because pfSense has (or had) an internal process that restarted packages when IP addresses changed. If Snort restarts, then it surely should see the new IP. The pfSense package restart script simply executes a shell script provided by each installed package. The Snort shell script is
/usr/local/etc/rc.d/snort.sh. That script should be called with "restart" or else "stop, start" as arguments. So you should see system log entries from that shell script showing it shutting down Snort and starting it up again.Seeing a copy of your system log while this interruption is happening would be helpful.
I would like to send you that logfile private - how can I do that?
All I need to see are the handful of lines related to Snort being restarted. There is no private information in those entries. Just snip out or otherwise copy and paste the relevant lines into a post here. To keep from getting overwhelmed from users around the world, I don't do any one-on-one troubleshooting with users. I like to keep all the interractions public so the entire user community can see issues and benefit from also seeing the eventual solutions.
Is this sufficient: clog /var/log/system.log | grep snort ?
Yes, that should do it.
Mar 29 03:10:01 aragorn snort[78243]: spo_pf -> Received notification of IP address change on interface pppoe0. Mar 29 03:10:01 aragorn snort[78243]: spo_pf -> deleted address xxx.x30.101.206 from automatic IP Pass List. Mar 29 03:10:05 aragorn snort[78243]: spo_pf -> Received notification of IP address change on interface pppoe0. Mar 29 03:10:05 aragorn snort[78243]: spo_pf -> deleted address fe80::20d:b9ff:fe46:725c from automatic IP Pass List. Mar 29 03:10:07 aragorn snort[78243]: Can't acquire (-1) - The interface disappeared! Mar 29 03:10:08 aragorn snort[78243]: =============================================================================== Mar 29 03:10:08 aragorn snort[78243]: Packet I/O Totals: Mar 29 03:10:08 aragorn snort[78243]: Received: 931413 Mar 29 03:10:08 aragorn snort[78243]: Analyzed: 931444 (100.003%) Mar 29 03:10:08 aragorn snort[78243]: Dropped: 0 ( 0.000%) Mar 29 03:10:08 aragorn snort[78243]: Filtered: 0 ( 0.000%) Mar 29 03:10:08 aragorn snort[78243]: Outstanding: 0 ( 0.000%) Mar 29 03:10:08 aragorn snort[78243]: Injected: 0 Mar 29 03:10:08 aragorn snort[78243]: =============================================================================== Mar 29 03:10:08 aragorn snort[78243]: Breakdown by protocol (includes rebuilt packets): Mar 29 03:10:08 aragorn snort[78243]: Eth: 0 ( 0.000%) Mar 29 03:10:08 aragorn snort[78243]: VLAN: 0 ( 0.000%) Mar 29 03:10:08 aragorn snort[78243]: IP4: 932253 (100.000%) Mar 29 03:10:08 aragorn snort[78243]: Frag: 4 ( 0.000%) Mar 29 03:10:08 aragorn snort[78243]: ICMP: 129701 ( 13.913%) Mar 29 03:10:08 aragorn snort[78243]: UDP: 130579 ( 14.007%) Mar 29 03:10:08 aragorn snort[78243]: TCP: 671916 ( 72.074%) Mar 29 03:10:08 aragorn snort[78243]: IP6: 0 ( 0.000%) Mar 29 03:10:08 aragorn snort[78243]: IP6 Ext: 0 ( 0.000%) Mar 29 03:10:08 aragorn snort[78243]: IP6 Opts: 0 ( 0.000%) Mar 29 03:10:08 aragorn snort[78243]: Frag6: 0 ( 0.000%) Mar 29 03:10:08 aragorn snort[78243]: ICMP6: 0 ( 0.000%) Mar 29 03:10:08 aragorn snort[78243]: UDP6: 0 ( 0.000%) Mar 29 03:10:08 aragorn snort[78243]: TCP6: 0 ( 0.000%) Mar 29 03:10:08 aragorn snort[78243]: Teredo: 0 ( 0.000%) Mar 29 03:10:08 aragorn snort[78243]: ICMP-IP: 0 ( 0.000%) Mar 29 03:10:08 aragorn snort[78243]: IP4/IP4: 0 ( 0.000%) Mar 29 03:10:08 aragorn snort[78243]: IP4/IP6: 0 ( 0.000%) Mar 29 03:10:08 aragorn snort[78243]: IP6/IP4: 0 ( 0.000%) Mar 29 03:10:08 aragorn snort[78243]: IP6/IP6: 0 ( 0.000%) Mar 29 03:10:08 aragorn snort[78243]: GRE: 0 ( 0.000%) Mar 29 03:10:08 aragorn snort[78243]: GRE Eth: 0 ( 0.000%) Mar 29 03:10:08 aragorn snort[78243]: GRE VLAN: 0 ( 0.000%) Mar 29 03:10:08 aragorn snort[78243]: GRE IP4: 0 ( 0.000%) Mar 29 03:10:08 aragorn snort[78243]: GRE IP6: 0 ( 0.000%) Mar 29 03:10:08 aragorn snort[78243]: GRE IP6 Ext: 0 ( 0.000%) Mar 29 03:10:08 aragorn snort[78243]: GRE PPTP: 0 ( 0.000%) Mar 29 03:10:08 aragorn snort[78243]: GRE ARP: 0 ( 0.000%) Mar 29 03:10:08 aragorn snort[78243]: GRE IPX: 0 ( 0.000%) Mar 29 03:10:08 aragorn snort[78243]: GRE Loop: 0 ( 0.000%) Mar 29 03:10:08 aragorn snort[78243]: MPLS: 0 ( 0.000%) Mar 29 03:10:08 aragorn snort[78243]: ARP: 0 ( 0.000%) Mar 29 03:10:08 aragorn snort[78243]: IPX: 0 ( 0.000%) Mar 29 03:10:08 aragorn snort[78243]: Eth Loop: 0 ( 0.000%) Mar 29 03:10:08 aragorn snort[78243]: Eth Disc: 0 ( 0.000%) Mar 29 03:10:08 aragorn snort[78243]: IP4 Disc: 55 ( 0.006%) Mar 29 03:10:08 aragorn snort[78243]: IP6 Disc: 0 ( 0.000%) Mar 29 03:10:08 aragorn snort[78243]: TCP Disc: 0 ( 0.000%) Mar 29 03:10:08 aragorn snort[78243]: UDP Disc: 0 ( 0.000%) Mar 29 03:10:08 aragorn snort[78243]: ICMP Disc: 0 ( 0.000%) Mar 29 03:10:08 aragorn snort[78243]: All Discard: 55 ( 0.006%) Mar 29 03:10:08 aragorn snort[78243]: Other: 0 ( 0.000%) Mar 29 03:10:08 aragorn snort[78243]: Bad Chk Sum: 0 ( 0.000%) Mar 29 03:10:08 aragorn snort[78243]: Bad TTL: 0 ( 0.000%) Mar 29 03:10:08 aragorn snort[78243]: S5 G 1: 353 ( 0.038%) Mar 29 03:10:08 aragorn snort[78243]: S5 G 2: 454 ( 0.049%) Mar 29 03:10:08 aragorn snort[78243]: Total: 932253 Mar 29 03:10:08 aragorn snort[78243]: =============================================================================== Mar 29 03:10:08 aragorn snort[78243]: Action Stats: Mar 29 03:10:08 aragorn snort[78243]: Alerts: 396 ( 0.042%) Mar 29 03:10:08 aragorn snort[78243]: Logged: 396 ( 0.042%) Mar 29 03:10:08 aragorn snort[78243]: Passed: 0 ( 0.000%) Mar 29 03:10:08 aragorn snort[78243]: Limits: Mar 29 03:10:08 aragorn snort[78243]: Match: 0 Mar 29 03:10:08 aragorn snort[78243]: Queue: 0 Mar 29 03:10:08 aragorn snort[78243]: Log: 1 Mar 29 03:10:08 aragorn snort[78243]: Event: 216 Mar 29 03:10:08 aragorn snort[78243]: Alert: 0 Mar 29 03:10:08 aragorn snort[78243]: Verdicts: Mar 29 03:10:08 aragorn snort[78243]: Allow: 432908 ( 46.479%) Mar 29 03:10:08 aragorn snort[78243]: Block: 0 ( 0.000%) Mar 29 03:10:08 aragorn snort[78243]: Replace: 0 ( 0.000%) Mar 29 03:10:08 aragorn snort[78243]: Whitelist: 498536 ( 53.525%) Mar 29 03:10:08 aragorn snort[78243]: Blacklist: 0 ( 0.000%) Mar 29 03:10:08 aragorn snort[78243]: Ignore: 0 ( 0.000%) Mar 29 03:10:08 aragorn snort[78243]: Retry: 0 ( 0.000%) Mar 29 03:10:08 aragorn snort[78243]: =============================================================================== Mar 29 03:10:08 aragorn snort[78243]: Frag3 statistics: Mar 29 03:10:08 aragorn snort[78243]: Total Fragments: 4 Mar 29 03:10:08 aragorn snort[78243]: Frags Reassembled: 2 Mar 29 03:10:08 aragorn snort[78243]: Discards: 0 Mar 29 03:10:08 aragorn snort[78243]: Memory Faults: 0 Mar 29 03:10:08 aragorn snort[78243]: Timeouts: 0 Mar 29 03:10:08 aragorn snort[78243]: Overlaps: 0 Mar 29 03:10:08 aragorn snort[78243]: Anomalies: 0 Mar 29 03:10:08 aragorn snort[78243]: Alerts: 0 Mar 29 03:10:08 aragorn snort[78243]: Drops: 0 Mar 29 03:10:08 aragorn snort[78243]: FragTrackers Added: 2 Mar 29 03:10:08 aragorn snort[78243]: FragTrackers Dumped: 2 Mar 29 03:10:08 aragorn snort[78243]: FragTrackers Auto Freed: 0 Mar 29 03:10:08 aragorn snort[78243]: Frag Nodes Inserted: 4 Mar 29 03:10:08 aragorn snort[78243]: Frag Nodes Deleted: 4 Mar 29 03:10:08 aragorn snort[78243]: =============================================================================== Mar 29 03:10:08 aragorn snort[78243]: =============================================================================== Mar 29 03:10:08 aragorn snort[78243]: Stream statistics: Mar 29 03:10:08 aragorn snort[78243]: Total sessions: 15253 Mar 29 03:10:08 aragorn snort[78243]: TCP sessions: 12791 Mar 29 03:10:08 aragorn snort[78243]: UDP sessions: 2462 Mar 29 03:10:08 aragorn snort[78243]: ICMP sessions: 0 Mar 29 03:10:08 aragorn snort[78243]: IP sessions: 0 Mar 29 03:10:08 aragorn snort[78243]: TCP Prunes: 0 Mar 29 03:10:08 aragorn snort[78243]: UDP Prunes: 0 Mar 29 03:10:08 aragorn snort[78243]: ICMP Prunes: 0 Mar 29 03:10:08 aragorn snort[78243]: IP Prunes: 0 Mar 29 03:10:08 aragorn snort[78243]: TCP StreamTrackers Created: 12791 Mar 29 03:10:08 aragorn snort[78243]: TCP StreamTrackers Deleted: 12791 Mar 29 03:10:08 aragorn snort[78243]: TCP Timeouts: 0 Mar 29 03:10:08 aragorn snort[78243]: TCP Overlaps: 0 Mar 29 03:10:08 aragorn snort[78243]: TCP Segments Queued: 38227 Mar 29 03:10:08 aragorn snort[78243]: TCP Segments Released: 38227 Mar 29 03:10:08 aragorn snort[78243]: TCP Rebuilt Packets: 20872 Mar 29 03:10:08 aragorn snort[78243]: TCP Segments Used: 30138 Mar 29 03:10:08 aragorn snort[78243]: TCP Discards: 3388 Mar 29 03:10:08 aragorn snort[78243]: TCP Gaps: 546 Mar 29 03:10:08 aragorn snort[78243]: UDP Sessions Created: 2462 Mar 29 03:10:08 aragorn snort[78243]: UDP Sessions Deleted: 2462 Mar 29 03:10:08 aragorn snort[78243]: UDP Timeouts: 0 Mar 29 03:10:08 aragorn snort[78243]: UDP Discards: 0 Mar 29 03:10:08 aragorn snort[78243]: Events: 0 Mar 29 03:10:08 aragorn snort[78243]: Internal Events: 0 Mar 29 03:10:08 aragorn snort[78243]: TCP Port Filter Mar 29 03:10:08 aragorn snort[78243]: Filtered: 0 Mar 29 03:10:08 aragorn snort[78243]: Inspected: 0 Mar 29 03:10:08 aragorn snort[78243]: Tracked: 671109 Mar 29 03:10:08 aragorn snort[78243]: UDP Port Filter Mar 29 03:10:08 aragorn snort[78243]: Filtered: 0 Mar 29 03:10:08 aragorn snort[78243]: Inspected: 0 Mar 29 03:10:08 aragorn snort[78243]: Tracked: 2462 Mar 29 03:10:08 aragorn snort[78243]: =============================================================================== Mar 29 03:10:08 aragorn snort[78243]: HTTP Inspect - encodings (Note: stream-reassembled packets included): Mar 29 03:10:08 aragorn snort[78243]: POST methods: 67 Mar 29 03:10:08 aragorn snort[78243]: GET methods: 2919 Mar 29 03:10:08 aragorn snort[78243]: HTTP Request Headers extracted: 2989 Mar 29 03:10:08 aragorn snort[78243]: HTTP Request Cookies extracted: 20 Mar 29 03:10:08 aragorn snort[78243]: Post parameters extracted: 65 Mar 29 03:10:08 aragorn snort[78243]: HTTP response Headers extracted: 2987 Mar 29 03:10:08 aragorn snort[78243]: HTTP Response Cookies extracted: 15 Mar 29 03:10:08 aragorn snort[78243]: Unicode: 0 Mar 29 03:10:08 aragorn snort[78243]: Double unicode: 0 Mar 29 03:10:08 aragorn snort[78243]: Non-ASCII representable: 0 Mar 29 03:10:08 aragorn snort[78243]: Directory traversals: 0 Mar 29 03:10:08 aragorn snort[78243]: Extra slashes ("//"): 1954 Mar 29 03:10:08 aragorn snort[78243]: Self-referencing paths ("./"): 0 Mar 29 03:10:08 aragorn snort[78243]: HTTP Response Gzip packets extracted: 38 Mar 29 03:10:08 aragorn snort[78243]: Gzip Compressed Data Processed: 112157.00 Mar 29 03:10:08 aragorn snort[78243]: Gzip Decompressed Data Processed: 494809.00 Mar 29 03:10:08 aragorn snort[78243]: Http/2 Rebuilt Packets: 0 Mar 29 03:10:08 aragorn snort[78243]: Total packets processed: 15389 Mar 29 03:10:08 aragorn snort[78243]: =============================================================================== Mar 29 03:10:08 aragorn snort[78243]: SSL Preprocessor: Mar 29 03:10:08 aragorn snort[78243]: SSL packets decoded: 43688 Mar 29 03:10:08 aragorn snort[78243]: Client Hello: 3513 Mar 29 03:10:08 aragorn snort[78243]: Server Hello: 3468 Mar 29 03:10:08 aragorn snort[78243]: Certificate: 2276 Mar 29 03:10:08 aragorn snort[78243]: Server Done: 6820 Mar 29 03:10:08 aragorn snort[78243]: Client Key Exchange: 2166 Mar 29 03:10:08 aragorn snort[78243]: Server Key Exchange: 1942 Mar 29 03:10:08 aragorn snort[78243]: Change Cipher: 6073 Mar 29 03:10:08 aragorn snort[78243]: Finished: 0 Mar 29 03:10:08 aragorn snort[78243]: Client Application: 9752 Mar 29 03:10:08 aragorn snort[78243]: Server Application: 7820 Mar 29 03:10:08 aragorn snort[78243]: Alert: 788 Mar 29 03:10:08 aragorn snort[78243]: Unrecognized records: 15469 Mar 29 03:10:08 aragorn snort[78243]: Completed handshakes: 0 Mar 29 03:10:08 aragorn snort[78243]: Bad handshakes: 0 Mar 29 03:10:08 aragorn snort[78243]: Sessions ignored: 6374 Mar 29 03:10:08 aragorn snort[78243]: Detection disabled: 1043 Mar 29 03:10:08 aragorn snort[78243]: =============================================================================== Mar 29 03:10:08 aragorn snort[78243]: SIP Preprocessor Statistics Mar 29 03:10:08 aragorn snort[78243]: Total sessions: 37 Mar 29 03:10:08 aragorn snort[78243]: SIP anomalies : 1 Mar 29 03:10:08 aragorn snort[78243]: Total dialogs: 39 Mar 29 03:10:08 aragorn snort[78243]: Requests: 771 Mar 29 03:10:08 aragorn snort[78243]: invite: 4 Mar 29 03:10:08 aragorn snort[78243]: cancel: 1 Mar 29 03:10:08 aragorn snort[78243]: ack: 2 Mar 29 03:10:08 aragorn snort[78243]: bye: 1 Mar 29 03:10:08 aragorn snort[78243]: register: 736 Mar 29 03:10:08 aragorn snort[78243]: options: 27 Mar 29 03:10:08 aragorn snort[78243]: refer: 0 Mar 29 03:10:08 aragorn snort[78243]: subscribe: 0 Mar 29 03:10:08 aragorn snort[78243]: update: 0 Mar 29 03:10:08 aragorn snort[78243]: join: 0 Mar 29 03:10:08 aragorn snort[78243]: info: 0 Mar 29 03:10:08 aragorn snort[78243]: message: 0 Mar 29 03:10:08 aragorn snort[78243]: notify: 0 Mar 29 03:10:08 aragorn snort[78243]: prack: 0 Mar 29 03:10:08 aragorn snort[78243]: Responses: 744 Mar 29 03:10:08 aragorn snort[78243]: 1xx: 4 Mar 29 03:10:08 aragorn snort[78243]: 2xx: 605 Mar 29 03:10:08 aragorn snort[78243]: 3xx: 0 Mar 29 03:10:08 aragorn snort[78243]: 4xx: 135 Mar 29 03:10:08 aragorn snort[78243]: 5xx: 0 Mar 29 03:10:08 aragorn snort[78243]: 6xx: 0 Mar 29 03:10:08 aragorn snort[78243]: 7xx: 0 Mar 29 03:10:08 aragorn snort[78243]: 8xx: 0 Mar 29 03:10:08 aragorn snort[78243]: 9xx: 0 Mar 29 03:10:08 aragorn snort[78243]: Ignore sessions: 0 Mar 29 03:10:08 aragorn snort[78243]: Ignore channels: 0 Mar 29 03:10:08 aragorn snort[78243]: =============================================================================== Mar 29 03:10:08 aragorn snort[78243]: dcerpc2 Preprocessor Statistics Mar 29 03:10:08 aragorn snort[78243]: Total sessions: 0 Mar 29 03:10:08 aragorn snort[78243]: =============================================================================== Mar 29 03:10:08 aragorn snort[78243]: +-----------------------[filtered events]-------------------------------------- Mar 29 03:10:08 aragorn snort[78243]: | gen-id=1 sig-id=2403336 type=Limit tracking=src count=1 seconds=3600 filtered=1 Mar 29 03:10:08 aragorn snort[78243]: | gen-id=1 sig-id=2403300 type=Limit tracking=src count=1 seconds=3600 filtered=1 Mar 29 03:10:08 aragorn snort[78243]: | gen-id=1 sig-id=2403420 type=Limit tracking=src count=1 seconds=3600 filtered=1 Mar 29 03:10:08 aragorn snort[78243]: | gen-id=1 sig-id=2403306 type=Limit tracking=src count=1 seconds=3600 filtered=1 Mar 29 03:10:08 aragorn snort[78243]: | gen-id=1 sig-id=2403462 type=Limit tracking=src count=1 seconds=3600 filtered=2 Mar 29 03:10:08 aragorn snort[78243]: | gen-id=1 sig-id=2403470 type=Limit tracking=src count=1 seconds=3600 filtered=15 Mar 29 03:10:08 aragorn snort[78243]: | gen-id=1 sig-id=2403428 type=Limit tracking=src count=1 seconds=3600 filtered=4 Mar 29 03:10:08 aragorn snort[78243]: | gen-id=1 sig-id=2403478 type=Limit tracking=src count=1 seconds=3600 filtered=6 Mar 29 03:10:08 aragorn snort[78243]: | gen-id=1 sig-id=2403452 type=Limit tracking=src count=1 seconds=3600 filtered=18 Mar 29 03:10:08 aragorn snort[78243]: | gen-id=1 sig-id=2403444 type=Limit tracking=src count=1 seconds=3600 filtered=11 Mar 29 03:10:08 aragorn snort[78243]: | gen-id=1 sig-id=2403348 type=Limit tracking=src count=1 seconds=3600 filtered=1 Mar 29 03:10:08 aragorn snort[78243]: | gen-id=1 sig-id=2403372 type=Limit tracking=src count=1 seconds=3600 filtered=3 Mar 29 03:10:08 aragorn snort[78243]: | gen-id=1 sig-id=2403352 type=Limit tracking=src count=1 seconds=3600 filtered=18 Mar 29 03:10:08 aragorn snort[78243]: | gen-id=1 sig-id=2017919 type=Both tracking=dst count=2 seconds=60 filtered=1 Mar 29 03:10:08 aragorn snort[78243]: | gen-id=1 sig-id=2403434 type=Limit tracking=src count=1 seconds=3600 filtered=10 Mar 29 03:10:08 aragorn snort[78243]: | gen-id=1 sig-id=2402000 type=Limit tracking=src count=1 seconds=3600 filtered=118 Mar 29 03:10:08 aragorn snort[78243]: | gen-id=1 sig-id=2403346 type=Limit tracking=src count=1 seconds=3600 filtered=2 Mar 29 03:10:08 aragorn snort[78243]: | gen-id=1 sig-id=2403435 type=Limit tracking=src count=1 seconds=3600 filtered=2 Mar 29 03:10:08 aragorn snort[78243]: | gen-id=119 sig-id=31 type=Suppress tracking=none filtered=1 Mar 29 03:10:08 aragorn snort[78243]: spo_pf -> Received notification of IP address change on interface pppoe0. Mar 29 03:10:08 aragorn snort[78243]: spo_pf -> added address xxx.x30.108.45 to automatic interface IP Pass List. Mar 29 03:10:08 aragorn snort[78243]: spo_pf -> Received notification of IP address change on interface pppoe0. Mar 29 03:10:08 aragorn snort[78243]: spo_pf -> added address fe80::20d:b9ff:fe46:725c to automatic interface IP Pass List. Mar 29 03:10:09 aragorn snort[78243]: Snort exiting Mar 29 06:43:34 aragorn php-fpm[22416]: /snort/snort_interfaces.php: Restarting Snort on DSL(pppoe0) per user request... Mar 29 06:43:34 aragorn php-fpm[22416]: /snort/snort_interfaces.php: [Snort] Snort STOP for DSL(pppoe0)... Mar 29 06:43:35 aragorn snort[10517]: *** Caught Term-Signal Mar 29 06:43:37 aragorn php: /tmp/snort_pppoe060605_startcmd.php: [Snort] Updating rules configuration for: DSL ... Mar 29 06:43:45 aragorn php: /tmp/snort_pppoe060605_startcmd.php: [Snort] Enabling any flowbit-required rules for: DSL... Mar 29 06:43:46 aragorn php: /tmp/snort_pppoe060605_startcmd.php: [Snort] Building new sid-msg.map file for DSL... Mar 29 06:43:48 aragorn php: /tmp/snort_pppoe060605_startcmd.php: [Snort] Snort START for DSL(pppoe0)... Mar 29 06:47:50 aragorn snort[14301]: *** Caught Term-Signal Mar 29 06:48:07 aragorn php: /etc/rc.packages: Beginning package installation for snort . Mar 29 06:48:07 aragorn php: /etc/rc.packages: [Snort] There is a new set of Snort Subscriber rules posted. Downloading snortrules-snapshot-29151.tar.gz... Mar 29 06:49:39 aragorn php: /etc/rc.packages: Successfully installed package: snort. Mar 29 06:49:39 aragorn pkg-static: pfSense-pkg-snort reinstalled: 3.2.9.10_2 -> 3.2.9.10_2 Mar 29 09:58:33 aragorn php-fpm[7022]: /snort/snort_interfaces.php: Restarting Snort on DSL(pppoe0) per user request... Mar 29 09:58:33 aragorn php-fpm[7022]: /snort/snort_interfaces.php: [Snort] Snort STOP for DSL(pppoe0)... Mar 29 09:58:34 aragorn snort[4949]: *** Caught Term-Signal Mar 29 09:58:36 aragorn php: /tmp/snort_pppoe060605_startcmd.php: [Snort] Updating rules configuration for: DSL ... Mar 29 09:58:44 aragorn php: /tmp/snort_pppoe060605_startcmd.php: [Snort] Enabling any flowbit-required rules for: DSL... Mar 29 09:58:45 aragorn php: /tmp/snort_pppoe060605_startcmd.php: [Snort] Building new sid-msg.map file for DSL... Mar 29 09:58:47 aragorn php: /tmp/snort_pppoe060605_startcmd.php: [Snort] Snort START for DSL(pppoe0)... Mar 30 02:20:01 aragorn php: /usr/local/pkg/snort/snort_check_for_rule_updates.php: [Snort] Snort Subscriber rules are up to date... Mar 30 02:20:01 aragorn php: /usr/local/pkg/snort/snort_check_for_rule_updates.php: [Snort] Emerging Threats Open rules are up to date... Mar 30 02:20:01 aragorn php: /usr/local/pkg/snort/snort_check_for_rule_updates.php: [Snort] Hide Deprecated Rules is enabled. Removing obsoleted rules categories. Mar 30 02:20:01 aragorn php: /usr/local/pkg/snort/snort_check_for_rule_updates.php: [Snort] Removed 0 obsoleted rules category files. Mar 30 02:20:01 aragorn php: /usr/local/pkg/snort/snort_check_for_rule_updates.php: [Snort] The Rules update has finished. Mar 30 16:39:59 aragorn php-fpm[17783]: /snort/snort_preprocessors.php: [Snort] Updating rules configuration for: DSL ... Mar 30 16:40:07 aragorn php-fpm[17783]: /snort/snort_preprocessors.php: [Snort] Enabling any flowbit-required rules for: DSL... Mar 30 16:40:07 aragorn php-fpm[17783]: /snort/snort_preprocessors.php: [Snort] Building new sid-msg.map file for DSL... Mar 31 02:20:01 aragorn php: /usr/local/pkg/snort/snort_check_for_rule_updates.php: [Snort] Snort Subscriber rules are up to date... Mar 31 02:20:01 aragorn php: /usr/local/pkg/snort/snort_check_for_rule_updates.php: [Snort] Emerging Threats Open rules are up to date... Mar 31 02:20:01 aragorn php: /usr/local/pkg/snort/snort_check_for_rule_updates.php: [Snort] Hide Deprecated Rules is enabled. Removing obsoleted rules categories. Mar 31 02:20:01 aragorn php: /usr/local/pkg/snort/snort_check_for_rule_updates.php: [Snort] Removed 0 obsoleted rules category files. Mar 31 02:20:01 aragorn php: /usr/local/pkg/snort/snort_check_for_rule_updates.php: [Snort] The Rules update has finished. -
That log snippet was helpful. It is as I suspected. Snort and
libpcapcan't find thepppoe0interface after the PPoE connection cycles. I see the logged messages from that internal routine I mentioned that is monitoring the FreeBSD routing messages. Snort detects that the WAN IP has changed, so the automatic pass list will adjust. However, then something happens to the originalpppoe0instance such thatlibpcapcan no longer see it.Might be this is a timing issue. Maybe this is a timing issue with the way FreeBSD is adjusting to the modem losing sync and re-establishing the connection ???
These are the key lines from your logs --
Mar 29 03:10:01 aragorn snort[78243]: spo_pf -> Received notification of IP address change on interface pppoe0. Mar 29 03:10:01 aragorn snort[78243]: spo_pf -> deleted address xxx.x30.101.206 from automatic IP Pass List. Mar 29 03:10:05 aragorn snort[78243]: spo_pf -> Received notification of IP address change on interface pppoe0. Mar 29 03:10:05 aragorn snort[78243]: spo_pf -> deleted address fe80::20d:b9ff:fe46:725c from automatic IP Pass List. Mar 29 03:10:07 aragorn snort[78243]: Can't acquire (-1) - The interface disappeared!See how it loses the ability to see the
pppoe0interface.These entries a little farther down are a bit curious, though --
Mar 29 03:10:08 aragorn snort[78243]: spo_pf -> Received notification of IP address change on interface pppoe0. Mar 29 03:10:08 aragorn snort[78243]: spo_pf -> added address xxx.x30.108.45 to automatic interface IP Pass List. Mar 29 03:10:08 aragorn snort[78243]: spo_pf -> Received notification of IP address change on interface pppoe0. Mar 29 03:10:08 aragorn snort[78243]: spo_pf -> added address fe80::20d:b9ff:fe46:725c to automatic interface IP Pass List. Mar 29 03:10:09 aragorn snort[78243]: Snort exitingThe timing seems off. What made Snort exit ad 03:10:09? Was that manual intervention by you, or did some script restart Snort?
-
@bmeeks said in Snort not restart on interface:
That log snippet was helpful. It is as I suspected. Snort and
libpcapcan't find thepppoe0interface after the PPoE connection cycles. I see the logged messages from that internal routine I mentioned that is monitoring the FreeBSD routing messages. Snort detects that the WAN IP has changed, so the automatic pass list will adjust. However, then something happens to the originalpppoe0instance such thatlibpcapcan no longer see it.Might be this is a timing issue. Maybe this is a timing issue with the way FreeBSD is adjusting to the modem losing sync and re-establishing the connection ???
These are the key lines from your logs --
Mar 29 03:10:01 aragorn snort[78243]: spo_pf -> Received notification of IP address change on interface pppoe0. Mar 29 03:10:01 aragorn snort[78243]: spo_pf -> deleted address xxx.x30.101.206 from automatic IP Pass List. Mar 29 03:10:05 aragorn snort[78243]: spo_pf -> Received notification of IP address change on interface pppoe0. Mar 29 03:10:05 aragorn snort[78243]: spo_pf -> deleted address fe80::20d:b9ff:fe46:725c from automatic IP Pass List. Mar 29 03:10:07 aragorn snort[78243]: Can't acquire (-1) - The interface disappeared!See how it loses the ability to see the
pppoe0interface.These entries a little farther down are a bit curious, though --
Mar 29 03:10:08 aragorn snort[78243]: spo_pf -> Received notification of IP address change on interface pppoe0. Mar 29 03:10:08 aragorn snort[78243]: spo_pf -> added address xxx.x30.108.45 to automatic interface IP Pass List. Mar 29 03:10:08 aragorn snort[78243]: spo_pf -> Received notification of IP address change on interface pppoe0. Mar 29 03:10:08 aragorn snort[78243]: spo_pf -> added address fe80::20d:b9ff:fe46:725c to automatic interface IP Pass List. Mar 29 03:10:09 aragorn snort[78243]: Snort exitingThe timing seems off. What made Snort exit ad 03:10:09? Was that manual intervention by you, or did some script restart Snort?
At 03:10 this was a regular pppoe restart initiated by cron.
-
Initiated by cron? You have periodic restarts enabled? So if one of these periodic restarts happens to coincide with a loss of sync by your modem and subsequent reset of the
pppoe0connection, then I can see howlibpcapand Snort might get confused. For example, during the instant when the cron task is recycling PPPoE, Snort trys to connect to the interface and because the modem just lost sync and FreeBSD is trying to re-establish that interface, it is not available. Thus Snort andlibpcapcan't find it.Is the cron task something you set up? If so, I would try disabling it for a bit.
If your PPPoE connection is so unstable that you need to resort to cron task restarts on some basis, I would go fuss at my ISP and have them fix the underlying issue with your connection.
