Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    Backup of RRD fills /var on shutdown

    Scheduled Pinned Locked Moved 2.2 Snapshot Feedback and Problems - RETIRED
    3 Posts 1 Posters 1.4k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • P
      phil.davis
      last edited by

      Alix nanoBSD 256MB memory
      This is not a 2.2-specific problem, but putting here to see if anyone thinks a solution like this is worth having.
      I have a system with a bunch of VLANs (7 of them) and 2 WANs. The RRD data in /var/db/rrd is about 12MB. When the backup is happening at shutdown/reboot these files get converted into *.xml, which is much bigger - it generates 35MB of xml files. Then all the *.xml are put into a single tar in /cf/conf - that file is less than 2MB.
      During the creation of the *.xml there needs to be 35MB somewhere.
      /var memory disk runs out of space. I increased it to 80MB, but then my whole system runs out of real memory during shutdown while trying to write all the *.xml in /var/db/rrd
      It would be nice if there was a way to be generating each *.xml and piping it directly into the tar (not going through /var filesystem) - but I cannot see how to achieve this in /etc/rc.backup_rrd.sh
      Another way is to put the *.xml in a dir on a real file system that will have plenty of space. e.g. I made /backuptmp (in "/" file system - the real CF card). That means doing some extra writes to the CF card to manage the interim data, but it works with this code in /etc/rc.backup_rrd.sh

      #!/bin/sh
      
      : ${RRDDBPATH:=/var/db/rrd}
      : ${CF_CONF_PATH:=/cf/conf}
      : ${BACKUPTMP_PATH:=/backuptmp}
      
      # Save the rrd databases to the config path.
      if [ -d "${RRDDBPATH}" ]; then
      	[ -z "$NO_REMOUNT" ] && /etc/rc.conf_mount_rw
      	for rrdfile in "${RRDDBPATH}"/*.rrd ; do
      		xmlfiletmp="${rrdfile%.rrd}.xml"
      		xmlfile="${BACKUPTMP_PATH}/${xmlfiletmp##*/}"
      		/usr/bin/nice -n20 /usr/local/bin/rrdtool dump "$rrdfile" "$xmlfile"
      	done
      	cd / && tar -czf "${CF_CONF_PATH}"/rrd.tgz -C / "${BACKUPTMP_PATH#/}"/*.xml
      	rm "${BACKUPTMP_PATH}"/*.xml
      	[ -z "$NO_REMOUNT" ] && /etc/rc.conf_mount_ro
      fi
      
      

      Then I expect there would need to be similar code when unpacking the tar file at bootup to restore the rrd files via /backuptmp

      I have hard-coded BACKUPTMP_PATH in this example, but in a real solution it would need to be passed in as an option.

      Something could be added in System:Advanced Miscellaneous in the RAM Disk Settings area to give the user the option to use "/" filesystem as temporary storage for RRD backup XML.

      Is this something that people would find useful as an option on low-memory systems?

      As the Greek philosopher Isosceles used to say, "There are 3 sides to every triangle."
      If I helped you, then help someone else - buy someone a gift from the INF catalog http://secure.inf.org/gifts/usd/

      1 Reply Last reply Reply Quote 0
      • P
        phil.davis
        last edited by

        Actually it was not as difficult as I thought to code a possible solution: https://github.com/pfsense/pfsense/pull/1353
        If anyone would find this useful or has comments on my solution, go ahead…

        As the Greek philosopher Isosceles used to say, "There are 3 sides to every triangle."
        If I helped you, then help someone else - buy someone a gift from the INF catalog http://secure.inf.org/gifts/usd/

        1 Reply Last reply Reply Quote 0
        • P
          phil.davis
          last edited by

          A bit of back and forth with Ermal and I doing bits of code. One last fix to actually delete the transitory XML files on bootup/restore:
          https://github.com/pfsense/pfsense/pull/1356

          This is working very nicely for me now. I have some RRD history piling up over the last days and many boots and upgrades on my little Alix with a list of VLANs…

          Thanks Ermal for working this along.

          As the Greek philosopher Isosceles used to say, "There are 3 sides to every triangle."
          If I helped you, then help someone else - buy someone a gift from the INF catalog http://secure.inf.org/gifts/usd/

          1 Reply Last reply Reply Quote 0
          • First post
            Last post
          Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.