Net-SNMP Restarts Every Few Minutes
-
I'm having this really strange issue where Net-SNMP will restart every ~5 minutes, I do not see anything in the system logs about it exiting or the like, but if I monitor
ps -o etime -p $(pgrep snmp)
it restarts after ~5 minutes. I have 18 firewalls with SNMP and it happens consistently on like 3 of them.It doesn't happen all of the time, but usually overnight it will restart then get stuck in the restart loop for a while.
pfSense 2.7.2
pfsense-pkg-net-snmp 0.1.5_11
net-snmp 5.9.1_4,1 -
It just started happening again after it being fine for a couple of hours, here is the syslog for the last couple of restarts
Apr 9 20:28:56 snmpd 74272 nlist err: neither tcb nor _tcb found. Apr 9 20:28:56 snmpd 73937 nlist err: neither tcb nor _tcb found. Apr 9 20:28:56 snmpd 73937 nlist err: neither udb nor _udb found. Apr 9 20:28:56 snmpd 74272 nlist err: neither udb nor _udb found. Apr 9 20:28:56 snmpd 74272 Turning on AgentX master support. Apr 9 20:28:56 snmpd 73937 Turning on AgentX master support. Apr 9 20:28:56 snmpd 73937 Error opening specified endpoint "udp:redacted:161" Apr 9 20:28:56 snmpd 73937 Server Exiting with code 1 Apr 9 20:28:56 snmpd 74595 NET-SNMP version 5.9.1 Apr 9 20:35:04 snmpd 22851 nlist err: neither tcb nor _tcb found. Apr 9 20:35:04 snmpd 23114 nlist err: neither tcb nor _tcb found. Apr 9 20:35:04 snmpd 22851 nlist err: neither udb nor _udb found. Apr 9 20:35:04 snmpd 23114 nlist err: neither udb nor _udb found. Apr 9 20:35:04 snmpd 22851 Turning on AgentX master support. Apr 9 20:35:04 snmpd 23114 Turning on AgentX master support. Apr 9 20:35:04 snmpd 23114 Error opening specified endpoint "udp:redacted:161" Apr 9 20:35:04 snmpd 23114 Server Exiting with code 1 Apr 9 20:35:04 snmpd 23294 NET-SNMP version 5.9.1 Apr 9 20:41:07 snmpd 47662 nlist err: neither tcb nor _tcb found. Apr 9 20:41:07 snmpd 47762 nlist err: neither tcb nor _tcb found. Apr 9 20:41:07 snmpd 47662 nlist err: neither udb nor _udb found. Apr 9 20:41:07 snmpd 47762 nlist err: neither udb nor _udb found. Apr 9 20:41:07 snmpd 47662 Turning on AgentX master support. Apr 9 20:41:07 snmpd 47762 Turning on AgentX master support. Apr 9 20:41:07 snmpd 47762 Error opening specified endpoint "udp:redacted:161" Apr 9 20:41:07 snmpd 47762 Server Exiting with code 1 Apr 9 20:41:07 snmpd 51150 NET-SNMP version 5.9.1
-
@cct-ckatri do you have both net-snmp and bsnmp enabled?
-
@dennypage yes, bsnmp is setup to bind only to localhost, net-snmp is set to bind only on WAN, then I have
proxy -v2c -c public 127.0.0.1:161 .1.3.6.1.4.1.12325
added to the advanced options in the net-snmp config.I have this same exact setup on a bunch of pfSenses, but this issue only happens on like 3 of them.
-
Any ideas on this? It makes monitoring very annoying since it cuts out so much.
-
@cct-ckatri This was one of those "If you don't have anything good to say, then don't say anything" situations.
But since you asked, my recommendation is to disable one of the two conflicting SNMP implementations. I don't think it was ever intended that they both be active at the same time.
And FWIW, in my opinion binding an SNMP implementation to the WAN interface on a firewall is a Bad Idea.
Sorry.
-
Thanks, hopefully it's not crashing due to a conflict, cause it'll make the whole proxy thing worthless since it's not possible to get the pf MIB data from net-snmp and bsnmp doesn't support SNMPv3.
My firewall rules are setup so that only my zabbix server is able to access the SNMP server, so that's why I'm not too stressed about it being on the WAN.
Maybe I could try running a zabbix proxy on the firewall itself and just having bsnmp bound to localhost... I might have to give that a shot.
-
@cct-ckatri said in Net-SNMP Restarts Every Few Minutes:
it's not possible to get the pf MIB data from net-snmp and bsnmp doesn't support SNMPv3.
You could also extend net-snmp. It's likely that whatever data you are looking for is available via command line tools.
-
I figured it out
. My firewalls had an old unused OpenVPN client connection on it that was unstable and every time it reconnected, it got a new IP address causing pfsense to restart all packages, and since it took down SNMP, we wouldn't get alerted about the interface going down either...
So this issue is solved now