config.xml during initial install - in vmware? [SOLVED]

avantsysadm

We're trying to make use of the pre-flight / External Configuration Locator features (as documented at https://www.netgate.com/docs/pfsense/backup/automatically-restore-during-install.html) but are running into one significant difficulty.

We're trying to do this under VMware, where it's easy to attach an emulated floppy drive, but not nearly so easy to attach an emulated USB flash drive - actually, it's not just not as easy, it's flat-out impossible.

Testing with pfSense CE 2.4.4, it doesn't seem to even bother looking at floppies during the installation and/or first-time boot. (Or, we're doing something wrong - can't tell for sure.)

Does anyone know either:
a) whether pfSense 2.4.4 will still check floppies for config.xml, and/or
b) of a way (programmatically) to attach an emulated USB mass storage device in VMware?

Thanks,
-Adam Thompson

avantsysadm

FWIW, we figured out one problem already - FreeBSD (whatever version underlies v2.4.4) doesn't like 2.88MB floppies, only 1.44MB floppies. OK, so fine, cut the dd(1) count in half when creating the image.

Still not working, though...

nodau

Hi, you can pass locally attached usb drives to the vm when using vmware remote console or vmware workstation. You also need to add an usb controller to the vm.

avantsysadm

@bahsig Yeah, that would work for one-offs in many cases, but I'm trying to automate this, so requiring a physical device to be plugged into a specific physical computer at all times? Not great. Also can't be parallelized for mass deployment.

Also, for <security reasons> I'm managing VMware from a remote Windows virtual machine that I access through RDP; USB passthrough doesn't survive the RDP->VMware Remote Console->ESXi round-trip, not too surprising really.

-Adam

avantsysadm

Well, based on 2.4.4's src/etc/ecl.php, no, floppies aren't supported any more, because FreeBSD doesn't list fd0 in the kern.disks sysctl.
I'm now trying to figure out how on earth the code in ecl.php will ever work, given that (on my system, at least) MS-DOS-formatted disks generally don't show up with slices at all, the show up with parititions instead...

stephenw10

Can you not just attach a second very small virtual drive? Anything that does appear in kern.disk?

Steve

avantsysadm

@stephenw10 said in config.xml during initial install - in vmware?:

Can you not just attach a second very small virtual drive? Anything that does appear in kern.disk?

At the moment, that appears to be the only way to make this happen more-or-less-automatically, in a virtualized environment.
The trick now is to figure out the sweet spot and what tools to use on which platform (mixture of MacOS and Linux and Windows) to build that image and pre-populate it for the installer or the ECL function.
I've opened a bugreport on this (https://redmine.pfsense.org/issues/9089), and JimP seems to agree that the simplest way to fix this is to amend the documentation rather than adjust the code to handle raw devices. Which is too bad, but given the relative importance of this feature, a reasonable approach.
-Adam

avantsysadm

@stephenw10 said in config.xml during initial install - in vmware?:

Can you not just attach a second very small virtual drive? Anything that does appear in kern.disk?

BTW, that takes care of the ECL use case, but I still haven't found the code that the installer uses to check for config.xml during, well, install.
I haven't been able to find the installer itself, actually (albeit I haven't spent a lot of time looking)!

If you could point me to the right place in the source tree, that would be awesome...
Thanks,
-Adam

avantsysadm

For anyone else reading this: as of 2.4.4, the ECL logic also can only ever work with MBR-partitioned devices, not GPT-partitioned devices.
I opened https://redmine.pfsense.org/issues/9097 about this one.

On the other hand, I still haven't found any way to make ANY of the mechanisms described in https://www.netgate.com/docs/pfsense/backup/automatically-restore-during-install.html work, at all, so this still isn't the end of the story.

stephenw10

If attach a second fat32 formatted virtual drive it doesn't pull a config file from that?

Or just that isn't a practical option?

Steve

avantsysadm

@stephenw10 said in config.xml during initial install - in vmware?:

If attach a second fat32 formatted virtual drive it doesn't pull a config file from that?

Or just that isn't a practical option?

Steve

I finally got it working this afternoon!

At present, you must attach a second virtual hard drive, MBR-formatted (not GPT), with a FAT filesystem on it.

I couldn't tell if the installer was picking up the config file or if ECL was picking it up, so I placed a copy of config.xml in both /conf and /config to cover all my bases.

On macOS, I did the following:

export config.xml (WITH RRD data) from existing firewall

run these commands locally on your macbook with both qemu and VMware Fusion installed:

dd if=/dev/zero of=ECL.img bs=1024k count=100
D=$( hdiutil attach -noMount ECL.img )
diskutil eraseDisk MS-DOS ECL MBR ${D}
mkdir -p /Volumes/ECL/conf /Volumes/ECL/config
cp ~/Downloads/config*.xml /Volumes/ECL/config.xml
cp ~/Downloads/config*.xml /Volumes/ECL/conf/config.xml
cp ~/Downloads/config*.xml /Volumes/ECL/config/config.xml
diskutil eject ${D}
hdiutil detach ${D}
qemu-img convert -f raw -O vmdk -o subformat=monolithicSparse -S 1 ECL.img ECL.vmdk
/Applications/VMware\ Fusion.app/Contents/Library/vmware-vdiskmanager -d ECL.vmdk
/Applications/VMware\ Fusion.app/Contents/Library/vmware-vdiskmanager -k ECL.vmdk

• upload the resulting ECL.vmdk file to VMware at OVH
• add it as a second hard disk to the new pfSense VM
• shut down the old pfSense VM
• boot the new pfSense VM into the installer, off the ISO image
• install pfSense
• it will automatically pick up the config file from the 2nd hard disk
• test that the firewall works
• RE-UPLOAD the config.xml file to the firewall manually (this will restore the RRD data)
• shutdown the firewall (instead of rebooting)
• remove the 2nd hard disk
• power it back on

(WARNING: those are notes-in-progress, don't just blindly paste that in anywhere!)

FWIW, the last error I made was copying the downloaded config.xml file in as-is without renaming it back to simply "config.xml".

-Adam