Proxmox SR-IOV VF pass-through to pfSense VM
-
Hi all,
I am trying to pass-through a NIC VF to pfSense VM but failed.
For the VF creation, I followed this article https://docs.virtuozzo.com/virtuozzo_hybrid_server_7_installation_on_asrock_rack/sr-iov/assigning-sr-iov-network.html
Everything goes smooth until I boot into pfSense. It shows the error (I have 8 VFs enabled and passed-through)ixv0: <Intel(R) X550 Virtual Function> mem 0xfeb5c000-0xfeb5ffff,0xfeb60000-0xfeb63fff at device 27.0 on pci0
ixv0: ...reset_hw() failure: Reset Failed!
ixv0: IFDI_ATTACH_PRE failed 5
device_attach: ixv0 attach returned 5I am on Proxmox 7 with X550-AT2 NIC. VM is using BIOS instead of UEFI (maybe the problem? not tested and have no idea).
I wonder if anyone is having the same issue and could give some advice on configuring SR-IOV with pfSense running on Proxmox VM?
I also found this post on FreeBSD bug-track: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=211062
Is this patch applied on pfSense FreeBSD too? If anyone knows.Thank you in advance.
-
@tim4532 said in Proxmox SR-IOV VF pass-through to pfSense VM:
VM is using BIOS instead of UEFI (maybe the problem?
Could be...I always use UEFI...
-
@NollipfSense Just tried every combination and swap to OPNsense but still failed. I might have to compile my own Intel driver to support VF. Anyway thank you for the reply.
-
@tim4532 Have you looked at this: https://docs.netgate.com/pfsense/en/latest/recipes/virtualize-proxmox-ve.html
or this one: https://www.reddit.com/r/homelab/comments/cm87qr/tutorial_enabling_sriov_for_intel_nic_x550t2_on/?rdt=57094 -
@NollipfSense Thank you, and yes, the current setup is identical to those tutorials. Not really helpful for me.
-
After Netgate announced that pfSense+ is a joke and *ucks around with home users. I have carefully review my steps to enable IXV interface on OPNsense (pfSense still fails).
I found out that you must have the VF parent interface link up in order to function normally.
Also VF would not allow VLAN tagging on guest VM else it will drop those packet. Set VLAN on hypervisor E.g.ip link set [INT_NAME] vf [VF_NUM] vlan [VID]To use the VF in internal network. You must have "spoofchk off" on hypervisor. E.g.
ip link set [INT_NAME] vf [VF_NUM] spoofchk onNot tested, to allow MAC spoof on guest VM "trust on":
ip link set [INT_NAME] vf [VF_NUM] trust onHow to make these config persistent?? Use systemd to make a startup service.
REF (Proxmox persistent VF): https://forum.proxmox.com/threads/enabling-sr-iov-for-intel-nic-x550-t2-on-proxmox-6.56677/
REF (big thanks, detail VF config): https://forum.opnsense.org/index.php?topic=9576.0
REF (NVIDIA, VF config): https://enterprise-support.nvidia.com/s/article/howto-configure-mac-anti-spoofing-for-vms-over-sr-iov -
@tim4532 said in Proxmox SR-IOV VF pass-through to pfSense VM:
After Netgate announced that pfSense+ is a joke and *ucks around with home users. I have carefully review my steps to enable IXV interface on OPNsense (pfSense still fails).
This is uncalled for and should have left unsaid...glad you found a solution despite not being sure why you would need to passthrough a virtual firewall to a real firewall when there are Linux bridges...
-
@NollipfSense Kinda lost my mind there ignore it.. To reduce the overhead. Linux bridge eats a lot CPU power. Also I built the system inside an small AIO server (NAS, remote server, video rendering etc...) which has only 2x 10G port. I need the virtualization to help to split network segment and most important is MAC spoof for my ISP.
-
@tim4532 said in Proxmox SR-IOV VF pass-through to pfSense VM:
@NollipfSense Kinda lost my mind there ignore it.. To reduce the overhead. Linux bridge eats a lot CPU power. Also I built the system inside an small AIO server (NAS, remote server, video rendering etc...) which has only 2x 10G port. I need the virtualization to help to split network segment and most important is MAC spoof for my ISP.
Now I am understanding...I just got the sick joke. Onto your hardware, it's best to have a separate NIC for Proxmox management...can you hardware support another NIC?
-
@NollipfSense All PCIe slots were occupied since I have an MATX board and only have two full-size slots. My Proxmox management is on a Linux bridge (on-band), and I reserved one 1G physical port (out-band, on demand).
FYI: My board got 2x 1G and 2x 10G ports.
-
@tim4532 said in Proxmox SR-IOV VF pass-through to pfSense VM:
FYI: My board got 2x 1G and 2x 10G ports.
I would definitely use one of the 10G for pfSense WAN set for vtnet0 connected directly to your ISP modem. Once you assigned say ens2f0 to vtnet0, you don't need to passthrough the entire NIC, just plug the cable from your ISP and it will automatically passthrough...same goes for LAN if you have an external switch, do the same as you did with WAN and connect Proxmox's management port to the switch...you would have one port available for whatever you want.
