@patient0

👻 It is inexplicable to me why you must insult me while simultaneously trying to help. I just don't get it. FYI: Mr. know it all, you don't have to insult people. I may not be at your level when it comes to IT stuff, but that doesn't give you the right to bully others here in this respected forum. Insults are not necessary here, my friend. This is a friendly venue for the exchange of free ideas. Just focus on the problem if you can help. If you can't, move on... Nobody comes here to be bullied. I'm just a newbie trying to improve my little network so I can eventually install Docker/Frigate to monitor my cams in order to keep an eye on my field for potential deadly predators. I'm just a poor person from a poor country who just recently discovered this awesome tech. Why won't you share it with me? Why keep it for yourself? I may not be as rich as you, American, I presumed? But you still should respect people even if they are from a less desirable place and not from the "great USA." I have been coming to this forum for many years and for the most part, the users have been very good to me until now. Sir, you my friend is a bully! It shouldn't matter if my native tongue isn't English. By the way, I can read and write English relatively well for a poor sheep farmer. The problem is you, sir. You just don't know how to express thyself, and you made numerous assumptions about an individual you don't even know. What does that statement have to do with my issue? And was that necessary? Nobody is forcing you for assistance. You did it voluntarily. Anyway, the problem has been resolved with the help of a genteel person who tackled the problem instead of focusing on insulting me with unnecessary salty statements...Bye for the no help and must go monitor my herd. I think I see a predator coming for my flock👻

Disable the Proxmox firewall on the pfSense VM interfaces if you haven't done that already.

I'm running pfSense in production on Proxmox since a few months back and so far it works perfectly.

Quick update:

c941ae86-6ebb-40bb-a5e5-7ce0c05a60ce-image.png

Had to reboot once because auf updates, but since than rock solid no incidence. Enabled all extra IP-Lists and Suricata and so again.

@maitops Just disable all energy saving features of the nic or select high performance power profile in windows for a test.

It must be the power state switching or the system tunables I did im my last update post.

@FInnishFlash Doesn't work even with bridged interfaces, and virtio

@viragomann
I made a mistake in my previous message, sorry about that. Unfortunately, the traffic never reaches the LAN. If it did, the VM would already be accessible.

@stephenw10 Hi, we developed a Azure custom image with Opnsense including the current Azure Agent. With that Azure backup works normally. We stay with this solution, migration from pfSense was bit of work but worthwhile! You can close this task, thank you for your effort.

@adamk11 How many and what type of NICs are you using? Are we to assume that pfsense LAN, and all VLAN's share the same physical port or do you have them separated?
If they are using the same physical port, I'm thinking traffic will go.

From VM1 to switch, back into pfsense VLAN1, out again to the switch on VLAN2, and then return back on the same interface to VM2. That would create a tromboning effect that may limit your throughput, likely to something less than 5, or?

Generally speaking though, my experience from Proxmox using X520 NICs and VirtIO, I don't see any limitations in that regard. Running iperf between two FW's (over WAN interfaces) I get 8+ Gbps. I do have NIC's passed thru (IOMMU) to firewalls, but not to VM's.

Just now tested between two VM's in different VLAN's reaching 9.33GBit/s without any problems whatsoever, both VirtIO.

The problem was the bridges. When creating the bridge for lan it should be tied to the physical network port and have a static ip address. All fixed.

For your information, the two 1:1 NATs (one NAT per IP) are configured on the same interface.

Hard to see how it could be anything else. 😞

@ethanthekiwi said in SCSI error on VM:

For me this issue was caused by thin provisioning on the virtual hard drive. I followed VMware's instructions to "inflate" the disk to thick provisioning and I stopped getting these errors.

Reply

Yeap same solution (https://techdocs.broadcom.com/us/en/vmware-cis/vsphere/vsphere/8-0/vsphere-storage-8-0/working-with-datastores-in-vsphere-storage-environment/using-datastore-browser-in-vsphere-environment.html#GUID-C371B88F-C407-4A69-8F3B-FA877D6955F8-en) worked for myself as well. :)

Just an update for my posted problem.
It turns out that it caused by unstable SDN of Proxmox.

People in Proxmox forum suggest me to use the more robust "Network Bridge".
Now, the connection between the VMs seem to be more stable.
In other words, pfSense has no problem at all.

Thanks to those who've read and replied to my post.

@seyed said in Best Practice for Connecting Physical Machines to Proxmox LAN Managed by pfSense:

Network Configuration:
vmbr0 – Proxmox management bridge (Public IP)
vmbr1 – pfSense WAN interface (Public IP)
vmbr2 – pfSense LAN interface for internal VMs

Goal:
I have two physical machines, each with public IP addresses assigned to their primary NICs. I would like to route these machines through pfSense by connecting their secondary NICs to the Proxmox LAN (vmbr2), effectively placing them behind the pfSense firewall.

What do you mean with Public IPs, especially wrt vmbr0 and your 2 physical machines? Does your ISP provide multiple IP's and are these machines not behind some firewall (other than perhaps the built in one in Proxmox)?

Proposed Solution:

The Proxmox host has two unused NICs.
I am considering connecting the secondary NICs of the physical machines to the unused NICs on the Proxmox server.
These unused NICs would be bridged to vmbr2, allowing the physical machines to communicate with pfSense and other internal resources.

This sounds like you would connect one interface to the internet and the other to your LAN, and only having the "machine" in between? Do you trust that solution? What is your intent with pfsense here?
To connect anything to the LAN side of pfsense, I'd use a physical switch rather than trying to use the switching in Proxmox. It will work but may suffer performance wise and it sure makes life more complicated...

@s0ulf3re So what you are showing are the Proxmox interfaces you have set up, right?
vmbr0 with 192.168.1.234 is your Proxmox interface, isn't it??

If you only have one port on Proxmox, I'm thinking you need to use VLANs to be able to separate pfsense LAN away from your main LAN. Otherwise you will have a DHCP and subnet conflict.

Perhaps if you can show your pfsense HW setup in Proxmox?
You have to attach vmbr0 first (as this will be WAN) and then vmbr2. For vmbr2 you add a VLAN tag in Proxmox, and then all your VM's need to have the same ID on their interfaces. Also assuming you have a VLAN capable switch attached where the same VLAN tag i TAGGED on the port.

Indeed there is no aarch64 ISO installer.

You are ending up at the UEFI shell because it's failing to boot anything else.

Do you see it trying and failing to boot the ISO image? You might have to choose to boot it.

@triks for anyone in the same boat, it ended up being NTOPNG that was filling the RAMDISK. Followed many posts but couldn't get it to save to SSD so removed it and that resolved the issue.