@seyed said in Best Practice for Connecting Physical Machines to Proxmox LAN Managed by pfSense:

Network Configuration:
vmbr0 – Proxmox management bridge (Public IP)
vmbr1 – pfSense WAN interface (Public IP)
vmbr2 – pfSense LAN interface for internal VMs

Goal:
I have two physical machines, each with public IP addresses assigned to their primary NICs. I would like to route these machines through pfSense by connecting their secondary NICs to the Proxmox LAN (vmbr2), effectively placing them behind the pfSense firewall.

What do you mean with Public IPs, especially wrt vmbr0 and your 2 physical machines? Does your ISP provide multiple IP's and are these machines not behind some firewall (other than perhaps the built in one in Proxmox)?

Proposed Solution:

The Proxmox host has two unused NICs.
I am considering connecting the secondary NICs of the physical machines to the unused NICs on the Proxmox server.
These unused NICs would be bridged to vmbr2, allowing the physical machines to communicate with pfSense and other internal resources.

This sounds like you would connect one interface to the internet and the other to your LAN, and only having the "machine" in between? Do you trust that solution? What is your intent with pfsense here?
To connect anything to the LAN side of pfsense, I'd use a physical switch rather than trying to use the switching in Proxmox. It will work but may suffer performance wise and it sure makes life more complicated...

@s0ulf3re So what you are showing are the Proxmox interfaces you have set up, right?
vmbr0 with 192.168.1.234 is your Proxmox interface, isn't it??

If you only have one port on Proxmox, I'm thinking you need to use VLANs to be able to separate pfsense LAN away from your main LAN. Otherwise you will have a DHCP and subnet conflict.

Perhaps if you can show your pfsense HW setup in Proxmox?
You have to attach vmbr0 first (as this will be WAN) and then vmbr2. For vmbr2 you add a VLAN tag in Proxmox, and then all your VM's need to have the same ID on their interfaces. Also assuming you have a VLAN capable switch attached where the same VLAN tag i TAGGED on the port.

Indeed there is no aarch64 ISO installer.

You are ending up at the UEFI shell because it's failing to boot anything else.

Do you see it trying and failing to boot the ISO image? You might have to choose to boot it.

@jjuk Ok, so they can ping each other but nothing else is working is what you are saying. Anything in the firewall logs showing something is blocked? Do you run Suricata or Snort?

@triks for anyone in the same boat, it ended up being NTOPNG that was filling the RAMDISK. Followed many posts but couldn't get it to save to SSD so removed it and that resolved the issue.

Hmm, unfortunately nothing there really looks like an issue. Certainly not something that would cause it to stop responding entirely.

We could try running dtrace against it to see what's using the cycles but I think we'd need some way to trigger it before it stopped responding.

If you create a test instance in the same hypervisor does that also stop responding?

@SteveITS said in pfSense+ licensing on Proxmox HA cluster:

@Gblenn Yes it calculates the NDI based on detected hardware.

I haven’t tried but you might add a few extra NICs just in case for future use.

I guess the way @griffincash should do it is to wait with registration until decided on a good config.

Also you’ll need two Plus licenses for two routers.

Agree, since they are both active in a HA config. But I don't see that he should need more licenses when virtualizing vs the alternative of running two 6100s...?

@Gblenn I too run pfsense virtualised (proxmox) on old hardware (i7-3770S) alongside containers. I don't see any performance issues with pfsense either, in the webUI or otherwise.
I don't use pfBlocker's DNSBL just the IP blocking. Whilst I understand downloading and updating DNSBL may be CPU intensive, why would that impact performance on every visit to the dashboard? Is the pfblocker widget CPU intensive with respect to building DNSBL stats counters?

With my attempts to build small images with VB, I found the easiest way to get the image to be small is to do the install on as small of a virtual disk as possible. In your case try installing pfSense on a 1.3 or 1.4GB disk. I do not know what the minimum install disk size is, but experimentation will get you there. Once there you can always restore the config you have if needed.

There are VB options to reduce the disk, but in this case I suspect it will be easier to just re-install.

@jprez1980 Well., this hardware is 12 years old and considered obsolete.
So what do you mean by lower priced?
If you consider power consumption, this equipment is a money burner.

Gut feeling it probably tops at 3Gbit routing but you should check it with iperf and two pc's connected at 10g speeds with pf in between to be sure.
Of courseif we add packet filtering ids/ips etc this will be much less.

@JonathanLee Yeah maybe a virtual router. I'll consider that,
Thanks for the advice.

If you followed the guide (Tx checksuming of for the NICs, guest tools installed, etc), the VM is hardware virtualised (HVM), same CPU and RAM parameter as the one of ESXi and the XCP-NG host is reasonably fast then I can't see a reason why you have that limit.

I ran pfsense (plus 3 other VMs) with XCP-NG on a J3455 CPU with 16GB RAM on a 400/90 MBits PPPoE connection. That was pretty much the limit but your can be done with even quite slow hardware

What hardware to you use as a host. And what is the output of top -HaSP while you do the speedtest. I assume you run the speedtest from a client (client -> pfsense -> ISP -> speedtest server), not from pfsense itself.

And again, a network diagram would help. How are you connecting to the ISP?

@eiger3970-0 It appears I may have to move the virtual pfSense router to a hardware device.
However, I'm unsure if I could also build some 24 hour systems on the hardware device as well?

For example, a separate hardware device rather than this Desktop with 2 NICs, running a type 1 hypervisor with various 24 hour VMs.

Can a new and separate hardware device be run for a 24 hour router and 24 hour VMs?

Then this desktop can be powered on and off when needed.

@viragomann Thanks!

@viragomann The machine's Ethernet is set to a static IP 192.168.1.110/24 gateway 192.168.1.170.
The Wi-Fi is DHCP I guess?
I set up a new Network location on the machine and the Ethernet DHCP would not connect, so I switched back to the original network with the Ethernet static IP and it connected.
Guess I'll see how long it stays online until the problem returns.

@JonathanLee This is solved, somehow, I don't remember...just noting the thread can be closed.