pfSense Sporadic unable to get to internet.

panzerscope

Thanks,I have just checked and DNS registration is disabled. This is the DNS Resolver page currently.

bmeeks

I see a number of issues from the entries in the log snippet you posted.

First, you have the DNS Forwarder (dnsmasq) being monitored by the Service Watchdog package. I see multiple times in your logs where Service Watchdog thinks the DNS Forwarder is "down" and restarts it. Because the Forwarder and Resolver both want to run on the same port, you have problems. That leads to DNS failures. The DNS Resolver on pfSense is unbound. The DNS Forwarder is dnsmasq.

Second big issue is that you have Snort configured in the Service Watchdog package. I am the Snort package developer/maintainer, and I have posted here on the pfSense forums over and over that a user should NEVER configure Service Watchdog to monitor Snort. Service Watchdog does not understand how Snort works and will attempt to restart it when unnecessary. That can eventually lead to many duplicate Snort processes running on the same interface.

Finally, I see a number of netmap device errors related to running Snort in Inline IPS Mode. It might be due to duplicate processes created by the Service Watchdog package not understanding how Snort works, or it might be due to your particular NIC variant not being 100% netmap compliant.

johnpoz

@bmeeks said in pfSense Sporadic unable to get to internet.:

Snort configured in the Service Watchdog package

But he doesn't even show that as a running service, he has Suricata listed.. So trying to run both?, switched to it vs snort, but left short in the watchdog?

bmeeks

@johnpoz said in pfSense Sporadic unable to get to internet.:

@bmeeks said in pfSense Sporadic unable to get to internet.:

Snort configured in the Service Watchdog package

But he doesn't even show that as a running service, he has Suricata listed.. So trying to run both?, switched to it vs snort, but left short in the watchdog?

Yeah, I'm thinking Service Watchdog has some improper entries.

I just immediately looked into the posted log attachment and did not examine all the installed packages. But yes, you are correct. He has Suricata installed now, but Service Watchdog is trying to start Snort.

panzerscope

@bmeeks
@johnpoz

Thanks very much for your comments and help. So it is true that I switched from Snort to Suricata, thus must have ended up being a left over entry in the Service Watchdog.

I have removed Snort as well as the DNS Forwarder from the Service Watchdog. Currently my watchdog list looks like the following.

Does this list look appropriate, will retaining the unbound DNS Resolver in the list cause any issues ? Once I have this cleared up I will reboot the pfsense box and see if I get these issues again further down the line.

bmeeks

@panzerscope said in pfSense Sporadic unable to get to internet.:

@bmeeks
@johnpoz

Thanks very much for your comments and help. So it is true that I switched from Snort to Suricata, thus must have ended up being a left over entry in the Service Watchdog.

I have removed Snort as well as the DNS Forwarder from the Service Watchdog. Currently my watchdog list looks like the following.

Does this list look appropriate, will retaining the unbound DNS Resolver in the list cause any issues ? Once I have this cleared up I will reboot the pfsense box and see if I get these issues again further down the line.

Personally, there is really no need to run the Service Watchdog package. At best, if you have services randomly stopping, it is a band aid. You need to identify why the services are randomly stopping and fix that root cause. Service Watchdog is not a package I would consider installing.

unbound makes a great resolver for pfSense, but it begins to get a bit strained when you use a package like pfBlockerNG-devel to create and maintain large DNS blacklists (via the DNSBL addon, for example).

panzerscope

@bmeeks said in pfSense Sporadic unable to get to internet.:

@panzerscope said in pfSense Sporadic unable to get to internet.:

@bmeeks
@johnpoz

Thanks very much for your comments and help. So it is true that I switched from Snort to Suricata, thus must have ended up being a left over entry in the Service Watchdog.

I have removed Snort as well as the DNS Forwarder from the Service Watchdog. Currently my watchdog list looks like the following.

Does this list look appropriate, will retaining the unbound DNS Resolver in the list cause any issues ? Once I have this cleared up I will reboot the pfsense box and see if I get these issues again further down the line.

Personally, there is really no need to run the Service Watchdog package. At best, if you have services randomly stopping, it is a band aid. You need to identify why the services are randomly stopping and fix that root cause. Service Watchdog is not a package I would consider installing.

unbound makes a great resolver for pfSense, but it begins to get a bit strained when you use a package like pfBlockerNG-devel to create and maintain large DNS blacklists (via the DNSBL addon, for example).

That is a fair point to be honest, best to fix the problem over the service continuously restarting the service. With that in mind, I have gone ahead and removed Service Watchdog altogether. Stops it from being a factor after all.

panzerscope

I also meant to ask, specifically regarding this screenshot where it makes references to netmap. Is this normal ? I just cannot recall if I got this before. I know I did solve all my netmap issues when switching to the Intel I350 -T4 NIC. A discussion previously had here: https://forum.netgate.com/topic/171570/editing-loader-conf

Pfsense Netmap Message.jpg

Thanks again!

johnpoz

@panzerscope why would you have pcscd in watchdog - are you actually using it? Its got a memory leak, there are multiple threads about it. And it was set to not run on default I do believe a update or so back, etc.

I don't even have the service watchdog package installed..

bmeeks

@panzerscope said in pfSense Sporadic unable to get to internet.:

I also meant to ask, specifically regarding this screenshot where it makes references to netmap. Is this normal ? I just cannot recall if I got this before. I know I did solve all my netmap issues when switching to the Intel I350 -T4 NIC. A discussion previously had here: https://forum.netgate.com/topic/171570/editing-loader-conf

Thanks again!

Yes, those are purely informational startup messages logged by the netmap device. They show that your NIC is providing 4 TX and 4 RX netmap queues (or rings).

The messages I saw in your logs that indicated problems were the ones that said something along the lines of "... netmap_reinint ...". I don't recall that exact wording off the top of my head. But those messages in your previous system log showing netmap issues indicate that multiple threads were stepping on each other's netmap buffer (or ring) areas.

stephenw10

Yeah, you do not want pcscd running at all. It's disabled by default in 2.6.
https://redmine.pfsense.org/issues/11933

And, yeah, you probably don't need/want the services watchdog running at all. You should have a specific reason for enabling that for any service.

Steve

panzerscope

Thanks all for your help. I just wanted to come back and things seem to now be resolved due to the above steps. Fingers crossed it stays that way. Hopefully some other newb will find this useful in the future.