pfSense on TrueNAS Scale KVM, What is the best Virtual Custom CPU to choose

shoulders

Hi

I have been running pfSense (with dedicated quad port card using PCI-E passthrough) for some weeks with no issue as a Virtual Machine on TrueNAS which uses KVM. I have been use the 'Custom' CPU option with no model selected which causes defaulting to the following CPU:

QEMU Virtual CPU version 2.5+
4 CPUs: 1 package(s) x 4 core(s)
AES-NI CPU Crypto: No
QAT Crypto: No

The Problem

This is what happens when i try the different CPU modes in KVM/QEMU on TrueNAS.

The QEMU CPU = does not support hardware AES-NI (QAT is intel only) and does not have a lot of the CPU flags a modern PC has. pfsense runs fine.
'Host Passthrough' CPU mode = allows pfSense to load but the GUI and routing does not work.
'Host Model' CPU mode = allows pfSense to load but the GUI and routing does not work.

The issues here are probably caused by my CPU being so new, AMD and pfSense running on FreeBSD.

The Question
This means I will have to select one of the premade CPUs, which one should I choose to get the best out of my CPU?

Although I have an AMD chip, if anyone has the answer for Intel chips aswell that would be good to add here for other people.

Notes

My CPU: AMD Ryzen 9 7900 12-Core Processor with 128GB
pfsense 2.7.0-RELEASE (amd64)
TrueNAS-SCALE-22.12.3.3

KVM / QEMU info

GitHub page with all of the CPU profiles and if you edit them you can see the flags.

QEMU / KVM CPU model configuration

This page has some information of the different modes, 'Host Model', 'Host passthrough', Custom
The default QEMU CPU models are designed such that they can run on all hosts. If an application does not wish to do perform any host compatibility checks before launching guests, the default is guaranteed to work.
The default CPU models will, however, leave the guest OS vulnerable to various CPU hardware flaws, so their use is strongly discouraged. Applications should follow the earlier guidance to setup a better CPU configuration, with host passthrough recommended if live migration is not needed.
The following CPU models are preferred for use on Intel hosts.
The following CPU models are preferred for use on AMD hosts.
My Guess for Intel is: Intel Xeon Processor (Cascade Lake, 2019)
My guess for AMD is: AMD EPYC Processor (2017)
newer CPUS might be availabe in newer version of KVM/QEMU in TrueNAS and elsewhere. Such as EPYC-IBPB, EPYC-Rome.

List of KVM/QEMU CPUs in TrueNAS

spoiler||
486
pentium
pentium2
pentium3
pentiumpro
coreduo
n270
core2duo
qemu32
kvm32
cpu64-rhel5
cpu64-rhel6
qemu64
kvm64
Conroe
Penryn
Nehalem
Nehalem-IBRS
Westmere
Westmere-IBRS
SandyBridge
SandyBridge-IBRS
IvyBridge
IvyBridge-IBRS
Haswell-noTSX
Haswell-noTSX-IBRS
Haswell
Haswell-IBRS
Broadwell-noTSX
Broadwell-noTSX-IBRS
Broadwell
Broadwell-IBRS
Skylake-Client
Skylake-Client-IBRS
Skylake-Client-noTSX-IBRS
Skylake-Server
Skylake-Server-IBRS
Skylake-Server-noTSX-IBRS
Cascadelake-Server
Cascadelake-Server-noTSX
Icelake-Client
Icelake-Client-noTSX
Icelake-Server
Icelake-Server-noTSX
Cooperlake
Snowridge
athlon
phenom
Opteron_G1
Opteron_G2
Opteron_G3
Opteron_G4
Opteron_G5
EPYC
EPYC-IBPB
EPYC-Rome
Dhyana
POWER6
POWER7
POWER8
POWER9
POWERPC_e5500
POWERPC_e6500
||spoiler

Thanks for any help in advance.

viragomann

@shoulders said in pfSense on TrueNAS Scale KVM, What is the best Virtual Custom CPU to choose:

'Host Model' CPU mode = allows pfSense to load but the GUI and routing does not work.

This might be the choice though. So the VM can use any feature of the physical CPU.
And I don't think that it wasn't supported by pfSense.

Did you disable "Hardware Checksum Offloading" in pfSense?

shoulders

@viragomann Yes it is.

It is down to pfSense/FreeBSD not liking something on my AMD CPU.

viragomann

@shoulders
Maybe you can get a BIOS upgrade?

shoulders

@viragomann nope, it is a CPU and Flags thing.

shoulders

Just in case anyone find this thread, i did a better article:

pfSense on TrueNAS Scale KVM, What is the best Virtual Custom CPU to choose? | QuantumWarp