Status: Monitoring is completely broken, pfSense 2.4.5
-
@serbus No, because I want to keep my data.
-
@scurrier said in Status: Monitoring is completely broken, pfSense 2.4.5:
A long time ago I might have tried changing the RRD settings to retain more data points or something. Not sure, hard to remember.
Hello!
You could try backing up the /var/db/rrd folder and then resetting.
Maybe just manually popping the rrd file for the interface/dataset that if giving you problems.
Retaining the data may not be worth it if you cant display it the way you want to, but maybe there is an easy code workaround.John
-
@serbus I think you're right it will probably fix it. I'm not going to do it now, but may decide to try tomorrow. Regarding potentially having changed the rrd settings, is there even a place to do that in the gui? I'm not the kind to go screwing under the hood.
-
Hello!
I dont know if there are rrd tweaks in the gui.
There is a RRD Data option in Diagnostics -> Backup & Restore that could simplify saving and recovering your data if you want to fool around with it.
John
-
rrd files can be modified .
No GUI, as you're dealing with pure data chunks.
pfSense has the tool.rrdtool dump /var/db/rrd/lan-traffic.rrd /root/lan-traffic.xmlNow edit this xml file using your favorite editor.
When done :rrdtool-f /root/lan-traffic.xml /var/db/rrd/lan-traffic.rrd -
It looks like the answer has been found. It can be considered a bug, but a very obscure one that requires unusual circumstances to trigger -- namely a very, very large rrd dataset (the OP says his is 6 years old). It very well could be something rrd is doing internally once the dataset file reaches a certain size. Since that is an unusually large dataset, the other folks in the thread are unable to reproduce using their likely smaller datasets.
@scurrier: what size is your rrd file? Have you been running on the same hardware the entire 6 years? Just wondering if you do in fact have 6 years worth of data in a single contiguous file.
@scurrier: take the info you have collected, and the solution you found, and submit an official bug report on the pfSense Redmine site here: https://redmine.pfsense.org/. That will put it on the developers' plate for future work. If you have already submitted a bug report, please edit it if necessary and include all the information you collected in your posts above. That will be of great value to whomever works on the bug report.
-
6 years is a lot of data for what type of data it is.. Does that really make sense to keep the data for that long?
I just looked and mine goes back to dec 2017.. I would assume when I fired up this 4860.. But when moved to new hardware I wouldn't be bringing that data over..
While its great info for sure, but I doubt the bug report would get much looking into until someone is sitting around twiddling their thumbs - hmm, hmm what to work on ;)
A quick fix I would think would just be to truncate rrd data at X.. So it only ever goes back so far, or so many specific data points..
But yeah @scurrier great work on tracking it down..
-
There's a quick and easy version of a fix so maybe I will try to figure out which git branch to work from and submit a pull request on it that mimics the open heart surgery I did in the browser.
-
Looks like I've been running this firewall since the end of 2014.
Interestingly, the data for some metrics doesn't go back as far as others. Compare these two. Note that the first one shows nulls for part of the data even though I know the firewall was running at that time. I wonder why.
-
@scurrier said in Status: Monitoring is completely broken, pfSense 2.4.5:
I wonder why.
Have a look at the the folder where rrd files are stored. Maybe you used another type of WAN interface before, like a pppoe access - or static setup. The rrd file will have another name for that period.
Btw : didn't know that that much data is stored in a rrd file :
mine goes back to 2014 also.
The file that was just before that time was based on a pppoe access, I wiped that stale rrd file long time ago.edit :
IPv6 since 2014 also :
The big chunck was the period I tried to sync my Syno NAS with some Microsoft Office cloud over night. I'll try that again when fiber gets invented.
-
strange, but i don't have that lines on my php,
the one you mentioned here
https://forum.netgate.com/post/925667
maybe it's not up to date?
-
Hello!
Just winging it here...I dont have a rrd dataset to reproduce the problem or test any solutions...
It looks like most (all?) rrd's are setup to hold 2284 days (6.25yr) of data. Once hit, the consolidation strategy must push the resolution out to 7200sec (?).
You could always increase the max. Is 10 years enough? Maybe 12years? There would need to be special update scripts to "resize GROW" all the current data. Yuk!
Maybe just add any possible resolutions to timelookup? What if rrd starts using one that wasnt added?
The simplest solution might be to just convert the timeformat assignment to a ternary operator, like :
var timeFormat = (timeLookup.hasOwnProperty([data[0].step]) ? timeLookup[data[0].step] : "%Y-%m-%d");Maybe you could backup/zip/post your rrd data set so others could load/test possible solutions?
It is pretty cool that pfsense has the durability/longevity to run into this sort of problem :).
John
