What is considered a clean installation???

bingo600

I would go for Opt. 3 too , and install pfSense w. ZFS (2.6.0)
Then restore the config backup.

/Bingo

thimplicity

@bingo600 Thanks! My proxmox server runs on thin-lvm on a single NVMe drive. Can I still do pfSense with ZFS or do I need to transform the complete server to ZFS first?

edit: Another question: Do you passthrough the ethernet ports to pfSense or do you have it set up with bridges as the pfSense documentation recommends? I understood the direct passthrough should deliver better performance.

Thanks!

bingo600

@thimplicity

I haven't got any experience with pfSense on a VM.
But i'd just make the pfSense VM ZFS

Re: passthrough vs bridge ....
Again i don't have any experience with pfSense & VM's

Maybe @johnpoz can give a hoint on pfSense & VM's

/Bingo

johnpoz

@thimplicity what filesystem you use on virtual disk has little to do with what the host os is using for a filesystem.

So bridge does not enough performance for your needs? You have what gig internet or you need to move from VM to physical full wire gig speeds? And your virtual interfaces connected to the bridge support only X? And you hoping if you passthru this physical interface you will get X+Y that will = full wirespeed?

Or is your internet only 100mbps, and you think physical gig vs virtual will get you more than 100mbps? Or is that bridge is only giving you 80mbps?

thimplicity

@johnpoz Thanks!

So virtual bridge it is as well as ZFS on the thin-lvm host.

Based on my googling skills this should suffice a 1gig internet connection fine. When I get more, I can still move to PCI passthrough :).

jimp

I run dozens of pfSense VMs in my lab on Proxmox VE (Currently 7.2-4) and they are all stable.

Usually those kinds of problems end up being some kind of quirk with certain versions of Proxmox VE (e.g. too old) or with VM hardware settings.

That said I've used a wide variety of different guest settings (some BIOS, some UEFI, different CPU and machine types, etc) and they're all good for me. One of the nodes in my Proxmox VE cluster has ZFS and a different node has lvm-thin and they are both fine.

One thing I do not do is PCI passthrough, so it's possible there is some hardware/compatibility or other issue there.

thimplicity

@jimp Thanks for your insights! I have not set up the VM according to the recommended settings in the pfSense documentation, so this might cause some of the issues.

Will make sure to correct those mistakes

jimp

There is a bit of leeway there, lots of other combinations do work (e.g. q35 machine w/UEFI BIOS) the one in the docs is just the basics to make sure it's a solid base. Worth trying at least.

If you can post screenshots of Proxmox VE GUI for the the Hardware and Options tabs of the problem VM (with private stuff redacted) we might be able to spot something potentially problematic there.

thimplicity

@jimp Thanks! I will go with this basic setup recommendation, as I am still facing reboots and kernel panics twice a day or so. I did some things wrong during setup (e.g. operating system as Linux and not Other), SCSI instead of VirtIO etc. and then configure it from scratch and then have it run for a few days before the next step.

I will still post the screenshots later today.

The one thing that makes me think it is not a hardware issue is that the underlying proxmox installation runs rock solid without a hitch (knockonwood)

rcoleman-netgate

@thimplicity said in What is considered a clean installation???:

Option 1: Create new VM, re-install pfSense and manually configure everything
Option 2: Create new VM, re-install pfSense and rollback a proxmox snapshot
Option 3: Create new VM, re-install pfSense and load a backup configuration from a backup config file or the cloud backup

Worth noting that if you have a TAC support subscription this will nullify the sub as you will have a new NDI.

So does adding a new interfaces (MAC addresses are used, in part, to generate your NDI).

Might be worth while putting static MACs in your system to start with especially if you think you will move your VM in the future.

jimp

@thimplicity said in What is considered a clean installation???:

I did some things wrong during setup (e.g. operating system as Linux and not Other), SCSI instead of VirtIO etc.

Choosing Linux would definitely lead to problems. I don't think non-VirtIO SCSI controller types are known to be an issue but VirtIO is better.

thimplicity

@jimp Don't get my hopes up too much :) - will do the fresh installation on Friday.

thimplicity

@thimplicity OK, I got impatient with all the reboots and started the journey yesterday. I created a new VM according to the official pfSense instructions and set it up as q35 with UEFI. I decided against just rolling back the config and will configure the stuff from scratch. So far the basics that I have configured work without a hitch, but it has only been 10h.

Thanks for the input!