RAM Disk Management PR Discuss



  • RAM Disk Management  #2902
    https://github.com/pfsense/pfsense/pull/2902

    Jim,
    Have not been able to reproduce the issue you outlined in the PR.
    @jim-p:

    I applied this patch and it seemed to work at first. The first reboot after applying it picked up the previously saved RAM disk files during upgrade, but then the next reboot the RRD files were gone. Either they didn't save on shutdown or they were not restored on boot.

    Start with a fresh VM, update it, activate the RAM disk setting, reboot, apply the patch, reset the VM so it upgrades, check the graph data, then reboot and check the graph data again.

    Here are the steps I used.

    pfSense 2.4 on VirtualBox.

    
    1) Console option 13) Update from console
    (update if not already up to date)
    
    2) Console option 12) PHP shell + pfSense tools
    playback gitsync --minimal --diff --show-files git://github.com/pfSense/pfsense.git master
    Console option 5) Reboot system / y: reboot normally
    
    3) Web GUI - Status / Monitoring (graph appears fine)
    
    4) Web GUI - System / Advanced / Miscellaneous
    Enable "Use RAM Disk"
    /tmp size (40; default)
    /var size (60; default)
    RRD Backup (unset)
    DHCP Backup (unset)
    
    System reboots to enable RAM disk.
    
    4b) Web GUI - Status / Monitoring (graph appears fine)
    /cf/conf/dhcpleases.tgz appears fine.
    /cf/conf/rrd.tgz appears fine.
    /cf/conf/RAM_Disk_Store/var/db/aliastables/*.tgz appears fine.
    
    5) Console option 12) PHP shell + pfSense tools
    playback gitsync --minimal --diff --show-files git://github.com/pfSense/pfsense.git RAM_Disk_Management
    Reset VM
    
    5b) Web GUI - Status / Monitoring (graph appears fine)
    /cf/conf/rrd.tgz.old appears fine.
    /cf/conf/RAM_Disk_Store/*.tgz appears fine.  (dhcpleases.tgz, rrd.tgz, and url tables *.tgz)
    
    6) Console option 5) Reboot system / y: reboot normally
    
    6b) Web GUI - Status / Monitoring (graph appears fine)
    /cf/conf/rrd.tgz.old appears fine.
    /cf/conf/RAM_Disk_Store/*.tgz appears fine.  (dhcpleases.tgz, rrd.tgz, and url tables *.tgz)
    
    6c) System Logs Filter on "RAM"
    8 Matched General Log Entries. (Maximum 50) 
    Time Process PID Message
    Jan 6 14:14:25  root   RAM disk restore succeeded: /cf/conf/RAM_Disk_Store/Bogons_IPv4.txt.tgz  
    Jan 6 14:14:25  root   RAM disk restore succeeded: /cf/conf/RAM_Disk_Store/Bogons_IPv6.txt.tgz  
    Jan 6 14:14:25  root   RAM disk restore succeeded: /cf/conf/RAM_Disk_Store/CIPB_US_Networks.txt.tgz  
    Jan 6 14:14:25  root   RAM disk restore succeeded: /cf/conf/RAM_Disk_Store/Spamhaus_DROP.txt.tgz  
    Jan 6 14:14:25  root   RAM disk restore succeeded: /cf/conf/RAM_Disk_Store/Spamhaus_EDROP.txt.tgz  
    Jan 6 14:14:25  root   RAM disk restore succeeded: /cf/conf/RAM_Disk_Store/dhcpleases.tgz  
    Jan 6 14:14:25  root   RAM disk restore succeeded: /cf/conf/RAM_Disk_Store/rrd.tgz  
    Jan 6 14:14:25  root   RAM disk restore succeeded.  
    
    7) Console option 6) Halt system
    Start VM
    
    7b) Everything still appears fine.
    
    

  • Rebel Alliance Developer Netgate

    Try with ZFS



  • is there merits to using a ram disk when I have a ssd as the storage?  Since the ssd should easily be fast enough for any i/o needs.  The cons of needing to restore files after a reboot seen heavier than the pros.


  • Rebel Alliance Developer Netgate

    SSD should be fine, if it's decent quality or recent.

    Problem is when people want to run from media with low overall tolerance for writes (CF, SD, thumb drives…)



  • VirtualBox
    Fresh install using "pfSense-CE-2.4.0-BETA-amd64-20170106-1042.iso"
    Auto (ZFS)
    stripe  Stripe - No Redundancy

    • ada0  VBOX HARDISK

    Console 2) Assign LAN IP address
    Console 14) Enable secure shell
    WebGUI Config Wizard (defaults except primary DNS, disable dhcp override, TMZ, allow private and bogon networks)

    Let system percolate some RRD data for 5 minutes. (20:43-20:48)
    Enable ram disk (auto reboot)
    Monitoring graphs and backup files appear fine.

    Let system percolate more RRD data for 5 minutes. (20:50-20:56)
    GitSync with master:
    playback gitsync –minimal --diff --show-files git://github.com/NOYB/pfsense.git master
    Shutdown
    VM Snapshot
    Start VM

    Monitoring graphs and backup files appear fine.

    Let system percolate more RRD data for 5 minutes. (21:02-21:07)
    GitSync with the patch ("RAM_Disk_Management"):
    playback gitsync --minimal --diff --show-files git://github.com/NOYB/pfsense.git RAM_Disk_Management
    Reset VM so update is applied.

    Monitoring graphs have a gap from 20:56 to 21:10 (present boot time).
    Believe the gap is VM rest induced due to old shutdown script not being run which would create an up to date rrd backup.  Was last backed up at 20:56.

    Expected console message "Restoring contents of RAM disk store…".  Is not seen during the update boot-up.  It is seen on subsequent boot-ups though.  Nothing in the RAM disk store at this point so no console message is to be expected.
    Backup files (new ram disk store) have been updated.  Indicating the update script has run.

    Let system percolate more RRD data for 5 minutes. (21:10-21:15)
    reboot
    Expected console message "Restoring contents of RAM disk store…".  is now seen during boot-up.
    Monitoring graphs and backup files appear fine.  (VM reset induced gap but otherwise appears fine.)

    Looks like it is working far as I can tell.  Can anyone confirm or refute?


  • Rebel Alliance Developer Netgate

    OK, the problem in my case was that the new files created by the patch didn't get set executable. Once I fixed that it started to work.



  • Here's a procedure that may be better for testing this without the VM/hard reset induced gap.  Basically run the backups manually from command prompt.  Then after applying the patch, halt to stop cleanly.  Using halt should be more UFS friendly than doing a dirty/hostile hard reset.  Whereas ZFS doesn't seem to mind the hard reset.

    1. Enable RAM Disk

    2. Percolate some monitoring data for a few minutes.

    3. Run the backups manually from console command prompt.
      /etc/rc.backup_aliastables.sh
      /etc/rc.backup_dhcpleases.sh
      /etc/rc.backup_rrd.sh

    4. Apply the patch.
      playback gitsync –minimal --diff --show-files git://github.com/NOYB/pfsense.git RAM_Disk_Management

    5. Stop and restart so the system upgrades.
      halt

    6. Check upgrade result.
      a) Verify monitoring graphs and RAM disk backup files.
      b) Reboot
      c) Verify monitoring graphs and RAM disk backup files.


Log in to reply