Restore a new ZFS install from a config file made on a UFS install



  • I just got a used Dell r210 ii to load pfSense and the install and restore from backup was delightfully smooth. I was super nervous about it so I thought best not to throw too many variables in and just did the classic UFS restore, but now that I'm up and running and cause I'm a glutton for punishment, I'm thinking I should redo the whole process and turn my new install into a ZFS one.

    My question is, can I restore it with the config backup generated on my old UFS install? Is there an easier/better way than just reinstalling from scratch?


  • Rebel Alliance Netgate Administrator

    The underlying file system will have no impact on the backed up config file.

    You might want to try our new Config Backup (have a copy of it locally and use the online version to restore).

    Follow the instructions below, as this might be your first time before doing any changes I would get a local copy downloaded in case there is a setting missed (password or key).

    https://www.netgate.com/docs/pfsense/backup/autoconfigbackup.html



  • Thanks! I know this is a broad question, but would you recommend I redo my new setup now to ZFS now that I'm running on a backup (the original machine)?


  • Netgate Administrator

    UFS is still the default and recommended install FS type. However it's likely we will move to ZFS at some point.

    Right now UFS is far more widely tested, because it's the default, so you're far less likely to hit some obscure edge case issue.

    ZFS does have significant advantages; better resilience to power failure, system snapshots. It does use more memory though, in some cases a lot more. You shouldn't attempt to run it on a low RAM machine with a large drive.

    Steve



  • I would be running it on a Dell r210 ii with 8gb of ram so I think that would be plenty, however I'm far from a power user and am worried I'll experience more headache with idiosyncrasies of the new file system that benefit I'd derive from it. Thanks for your input...


  • Netgate Administrator

    In all probability you would not experience any issues with either filesystem. If you can run ZFS more people running it to expose any issues that might be there is good. ๐Ÿ˜‰

    Steve



  • I'm a glutton for punishment so I'll probably give it a try;)



  • Well, I finally got around to giving it a try, but I got stumped during the install. None of the "Pool Type/Disks:" options worked for me. They all are raids (which is kind of my understanding as being related to the benefits of ZFS) but I only have the one disk currently in my R210 ii. I thought I had read somewhere else that ZFS was still of benefit by "mirroring itself" or something. Any advice would be appreciated, but in the meantime I'll just stick with UFS.


  • Banned

    @rickybaker said in Restore a new ZFS install from a config file made on a UFS install:

    Well, I finally got around to giving it a try, but I got stumped during the install. None of the "Pool Type/Disks:" options worked for me.

    It's really easy select "Auto (ZFS)", set the pool to stripe for a single disk, mark the disk that should be used and let it install. As for redundancy on a single disk read up on the "copies" attribute, but be aware that it is far inferior to a real mirror.



  • I kept selecting the default option, selecting my disk, and it saying "failure, not enough disks 0<1" or something to that affect.


  • Netgate Administrator

    The installer doesn't make it obvious as I recall but that means that you really don't have any disks selected. You have to select at least one drive to continue at this point after just accepting all the other defaults:

     pfSense Installer
     โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€
    
    
    
    
     โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ZFS Configurationโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”
     โ”‚ โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ” โ”‚
     โ”‚ โ”‚[*] ada0         NT-32                                                  โ”‚ โ”‚
     โ”‚ โ”‚[ ] da0          IS917 innostor                                         โ”‚ โ”‚
     โ”‚ โ”‚[ ] label/swap0  GEOM LABEL device                                      โ”‚ โ”‚
     โ”‚ โ”‚[ ] mmcsd0       MMCHC M32508 5.2 SN 0B455B3B MFG 02/2018 by 112 0x0000 โ”‚ โ”‚
     โ”‚ โ”‚[ ] mmcsd0boot0  MMCHC M32508 5.2 SN 0B455B3B MFG 02/2018 by 112 0x0000 โ”‚ โ”‚
     โ”‚ โ”‚[ ] mmcsd0boot1  MMCHC M32508 5.2 SN 0B455B3B MFG 02/2018 by 112 0x0000 โ”‚ โ”‚
     โ”‚ โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜ โ”‚
     โ”œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ค
     โ”‚                       <  OK  >            < Back >                         โ”‚
     โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜
    
    

    Use space to select a drive. I've selected ada0 there.

    Steve



  • @stephenw10 OOOOOOOOOOO Thanks!l I'll give it a shot tomorrow and report back!



  • So I was able to upgrade to the ZFS install of the latest PFSense. I used the manual Restore method and everything seems alright (just wasn't that comfortable with the Autobackup and the other was so easy). I'll have to fully slot it in place of my current PFSense box to see if it's 100% working (and re-add my vlan) but seems good. One question: since I only have 1 hdd in the Del r210ii right now, is adding a mirror drive relatively easy? I assume it's from the command line and I'll need to learn a few ZFS commands, but guaging how easy to retrofit the mirror is (as opposed to building it that way from the start).


  • Netgate Administrator

    Hmm, interesting question, I've never tried.
    My instinct there is that is would require reinstalling but ZFS is far more capable so....
    More research needed.

    Steve


  • Banned

    @rickybaker said in Restore a new ZFS install from a config file made on a UFS install:

    I assume it's from the command line and I'll need to learn a few ZFS commands, but guaging how easy to retrofit the mirror is (as opposed to building it that way from the start).

    Shouln't be a problem, though I haven't tried it myself: https://www.freebsd.org/doc/handbook/zfs-zpool.html the only thing that might be a bit risky is updating the bootcode with the correct one if you use UEFI boot. On the other hand, reinstalling pfSense with config recovery is likely to be equally as fast, if not faster.



  • @grimson said in Restore a new ZFS install from a config file made on a UFS install:

    Shouln't be a problem, though I haven't tried it myself: https://www.freebsd.org/doc/handbook/zfs-zpool.html the only thing that might be a bit risky is updating the bootcode with the correct one if you use UEFI boot. On the other hand, reinstalling pfSense with config recovery is likely to be equally as fast, if not faster

    Well shoot, i was thinking it would be easier than that. Oh well, guess I'll look into installing another harddrive and do this again:)



  • So, sorry to bump my old topic, but I was experiencing some issues with this install and I was wondering if it could be related to the ZFS I installed. I had it up and running on th enew Dell r210ii in my condo and tested it for weeks as my main router without issue. I unplug it and move it to the new house and it doesn't work (WAN gives an IP of 0.0.0.0). I troubleshoot for days before giving up and going back to my condo to grab the old ufs install of pfsense I used to run and that worked without issue. It theoretically should be identical to the ufs install besides the zfs, so I'm not sure what broke, but I'm about to reinstall pfsense and start over so I'd love to know if I should abandon zfs. (the reinstall and restore should identify if it's purely hardware of that NIC I suppose)...


  • Netgate Administrator

    That seems unlikely to be filesystem related. If it was I would expect some sort of error, if it couldn't read the dhclient conf fiel for example. But even so ZFS is less likely to have that sort of issue.

    Steve



  • @stephenw10 said in Restore a new ZFS install from a config file made on a UFS install:

    if it couldn't read the dhclient conf fiel for example. But even so ZFS is less likely to have that sort of issue.

    I mean, it could be an error right? Or are you implying that reading 0.0.0.0 is indicative of normal (though unfortunate) behavior that belies there isn't an error. I haven't gotten around to reflashing pfsense, but I'm really hoping my NIC isn't shot...


  • Netgate Administrator

    I'm saying if there's a fault with the pfSense there it's probably not with reading the filesystem as much more would be broken then simply not pulling an IP address on WAN.
    Check the DHCP logs.
    Run a packet capture, make sure it's sending DHCP requests. See what, if anything, comes back.

    Steve



  • ok I'll look into how to do that immediately. Thanks for the suggestion



  • I would say zfs no question.

    Its been in FreeBSD (the parent OS of PfSense), for a long time now and is stable on a kernel level, the filesystem is designed from the ground up with data resilience in mind.

    Also since PfSense is primarily a network device in many cases there is hardly any i/o with writing of logs and graphs perhaps been the most intensive i/o expected. So even with zfs memory usage is not necessarily going to be large, on my 4 gig device I am using less then a gig of ram with zfs.

    Ideally you want to use a mirror, but if you only have one storage, then as soon as possible post install I suggest setting copies=3 (providing you have the excess space), which will store each file written 3 times, to help with bit rot protection. (note tho this wont make copies of existing files only new files written, but typically the important stuff like rrd data, config files, logs etc. will be written to so will get the copies, losing things like binaries can be recovered on a new install anyway).



  • @chrcoluk said in Restore a new ZFS install from a config file made on a UFS install:

    I would say zfs no question.

    Its been in FreeBSD (the parent OS of PfSense), for a long time now and is stable on a kernel level, the filesystem is designed from the ground up with data resilience in mind.

    Also since PfSense is primarily a network device in many cases there is hardly any i/o with writing of logs and graphs perhaps been the most intensive i/o expected. So even with zfs memory usage is not necessarily going to be large, on my 4 gig device I am using less then a gig of ram with zfs.

    Ideally you want to use a mirror, but if you only have one storage, then as soon as possible post install I suggest setting copies=3 (providing you have the excess space), which will store each file written 3 times, to help with bit rot protection. (note tho this wont make copies of existing files only new files written, but typically the important stuff like rrd data, config files, logs etc. will be written to so will get the copies, losing things like binaries can be recovered on a new install anyway).

    Thanks for that!! Where do I make the settings copies=3



  • If using default pfsense zfs installer, this should work.

    zfs set copies=3 zroot



  • @chrcoluk said in Restore a new ZFS install from a config file made on a UFS install:

    If using default pfsense zfs installer, this should work.

    zfs set copies=3 zroot

    Thanks, i wish this cleared it up more, but where exactly do I type this in? Can I do it after I've already installed pfsense or will I have to reinstall? I used the guided installer too, will I need to do something from the command line?



  • in the root shell

    Its a live command, not even a reboot needed.

    If you want to revert to default the command will be 'zfs set copies=1 zroot'.



  • @stephenw10 said in Restore a new ZFS install from a config file made on a UFS install:

    Run a packet capture, make sure it's sending DHCP requests. See what, if anything, comes back.

    So I figured out how to run a Packet Capture (still working on the DHCP logs, honestly I forgot until I came back to this post). This is what I got when I ran it on the WAN

    Screen Shot 2019-04-17 at 8.45.15 PM.png

    I apologize for needing further assistance, but I'm not really sure how to decipher this (but it doesn't look right). Anything else I should do?


  • Netgate Administrator

    The logs are available in the GUI in Status > System Logs > DHCP tab. Look for dhclient entries on WAN.

    That pcap shows all IPv6 packets so either it was filtered by IPv6 only or the WAN is not trying to pull an IP over v4. That fact there are no incoming v4 packets either makes it seem more like it was set to v6 only.

    Steve



  • @stephenw10 said in Restore a new ZFS install from a config file made on a UFS install:

    That fact there are no incoming v4 packets either makes it seem more like it was set to v6 only.

    Well that's interesting. I'll try to run the logs tonight, Where would that "v6 only" setting most likely be set? in the set up for the interface? or on the modem? It's a Motorola MB7621 btw if that's useful. Would this not be a setting carried over when I restore from the backup of the currently-working zotac/barebones Ufs install?


  • Netgate Administrator

    In the packet capture. You can set filtering options there including 'Address Family: IPv6 Only'. It isn't set by default there though it would usually capture all packets so if that was not filtered then it's very odd to see no v4 traffic.

    Try hitting DHCP lease renew or connect when the cap is running to at least send some v4 packets.

    Steve



  • @stephenw10 said in Restore a new ZFS install from a config file made on a UFS install:

    Try hitting DHCP lease renew or connect when the cap is running to at least send some v4 packets.

    OK I will. I ran it without intentionally making any modifications to the pcap. Just selected WAN and started it. Is there a length of time that's recommended to run it? Is it possible I just stopped it too soon? I will do the DHCP lease renew when it's running and the second suggestion is DHCP lease CONNECT?


  • Netgate Administrator

    That looks like you ran it for 10s only so it's possible nothing happened there. I would run for a few minutes at least. If you set the pcap to IPv4 only that will make it easier to read. You should also increasr the maximum capture size to, say, 1000 packets.

    Steve



  • @stephenw10 said in Restore a new ZFS install from a config file made on a UFS install:

    You should also increasr the maximum capture size to, say, 1000 packets

    Is the setting you're referring to the "Count" value? Currently set at the default 100?

    And where do I renew the dhcp lease? Is this under Status->DHCP leases? Or Services->DHCP Services? And I suppose in either case, what do I press after I'm in that subsection? There's a refresh button in the upper right corner that displays "Restart Service" on hover, but both of these sections seem more to do with the lan than the wan...

    Pcap is currently running filtered my ip4 traffic...



  • Here is the ip4 filtered few minutes long pcap:

    Screen Shot 2019-04-19 at 8.02.06 PM.png



  • Also, I realize this may not be neither here nor there, but the output on the attached monitor acknowledges that the ethernet is attached to both interfaces

    IMG_20190419_204909.jpg

    And the WAN gives a N/A when the modem is not attached, and 0.0.0.0 when it is. This may be something obvious to you guys but it gives me hope:)


  • Netgate Administrator

    Mmm, OK so you can see it sending DHCP broadcast requests there and nothing coming back at all.

    Did you reset the modem? Or spoof the MAC address? Many modems will lock to one MAC until reset.

    Steve



  • So I have in the past during this issue done a tiered reset of all the devices and it still had a 0.0.0.0 for the IP but I didn't reset the modem during or before this pcap. I just took the ethernet plug out of the working pfsense install and plugged it into the new non-working one. Now that you say it, besides the service (which is the same isp) the modem is the one thing that changed when I moved over. But the old pfsense install worked right when I plugged it in.

    And you can tell from the pcap what it's getting the dhcp renew you asked for? Cause I didn't do anything while the pcap was running. Just wait...


  • Netgate Administrator

    The only thing you can see in that pcap is that it's looking for a DHCP server and nothing is responding.

    Either the requests is not making it to the server or it's configured not to respond that that MAC address.

    Steve



  • So are you thinking this is a bad setting on modem or still some setting issue on the pfsense? What should I do to further test?



  • I'm not sure how to do the spoof business you mentioned but would happily try...


Log in to reply