go to the proxmox forum

@miracuru
As was mentioned by @viragomann the "Default deny rule IPv(4|6)" logs are normal. Actually they show that pfSense is doing its basic job, which is (by default) blocking all incoming connections to WAN.

You could implement a firewall rule on the WAN interface which does the same thing, but doesn't log the blocks. Enable that rule when you don't want pfSense to record all the WAN blocks in the logs. If you want to start logging the WAN blocks, just disable your rule and the defaults will kick in again.

Also, it may be possible to directly connect the enpf4s0 and enpf7s0 interfaces to pfSense via PCI-Passthrough. This will depend on hardware compatibility, but could be worth looking into; just food for thought.

Ok, have a look into the DOCSIS Telemetry.

I was hell if my ISP rollout the OFDMA to the upstream some years ago. And your problem looks similar.
Idle was nice, but if you use the bandwidth, the error rat grows and grows and with it the retransmission and the latency explode.
It takes month and 2-3 construction sites to get a nice stable connection back.
Have a look into it fist.

@viragomann said in pfSense on Proxmox via vmbr0 - got LAN access, but no WAN/internet access - why?:

@newsboost
You cannot use a passed-through NIC on Proxmox itself. The only available NIC you can use is enp1s0f3.

That makes completely sense to me and probably explains the error message, thanks! But I'm really confused now, because it seem to work, i.e. it provides VLAN 100 internet access and yet it seems that the interface is still being passed through, because enp1s0f0 = igb0 = WAN and enp1s0f1 = LAN (vlan trunk) = igb1... Are you sure this should not work, because it seem to work? And why does it work, is it kind of "undefined behaviour" perhaps? Great comment, thanks!

That's not a prlausible reason to have two subnets on Proxmox.

The explanation was not good enough... So, VLAN 1 (subnet 192.168.1.0/24) is my management VLAN and the VMs I create in Proxmox should preferably not have access to the management VLAN so I thought the safest and quickest solution would be to use another subnet for all my experimental VMs... That way, they don't have access to the more important devices/machines/printers/servers on VLAN 1... I think this is a better explanation, hopefully...

Just connect the bridge vmbr0 to a physical NIC port and assign a static (!) IP to the bridge in Proxmox. This should be a trusted subnet of course.

You're right - and I did just that and it also works:

209a52c4-6261-487e-9fff-3645ceca5665-image.png

From a logical perspective, this makes much more sense because as you wrote above and after I've been thinking about it, I think it's weird that I can bridge a NIC that has been passed through to proxmox and still get the behaviour that I wanted - but after my improved understanding and after reading your comment, now I wouldn't expect this to work any longer, but it still does... Very weird, it can bridge the NIC when passed through, apparently without internet/network problems!

So to access Proxmox in case of emergency, you have only to assign a static IP within the same subnet to a computer and connect it to the appropriate network port. Then you can access Proxmox independently from the state of pfSense.

It makes completely sense what you're writing and probably the solution could be that I should have two VMBR-interfaces:

One for emergencies, if pfSense does not respond or boot up correctly so I can plugin a network cable and ssh directly into Proxmox and One on subnet 100, such that I can isolate all the VMs from the management VLAN and do experiments without any fear...

Is it really that bad if I put vmbr0 in the VLAN 100-subnet so the proxmox interfaces can be access on two different subnets? Because I've been testing and it seems to work completely fine on two different subnets - although perhaps I would like to later block VLAN 100 from accessing the Proxmox-interface and I can do that by adding a firewall-rule using the pfSense-interface, isn't that right?

Appreciate your comments a lot, thanks!

@tibere86 Please create a new post for your question. Thanks.

@tim4532 said in Proxmox SR-IOV VF pass-through to pfSense VM:

FYI: My board got 2x 1G and 2x 10G ports.

I would definitely use one of the 10G for pfSense WAN set for vtnet0 connected directly to your ISP modem. Once you assigned say ens2f0 to vtnet0, you don't need to passthrough the entire NIC, just plug the cable from your ISP and it will automatically passthrough...same goes for LAN if you have an external switch, do the same as you did with WAN and connect Proxmox's management port to the switch...you would have one port available for whatever you want.

@root1ng said in Can someone explain to me how i can do this ?:

the network card of the motherboard is disabled in the bios

Most of us who use Proxmox reserve that port for Proxmox...makes it a lot easy, and once you passthrough the PCIe NIC in your setup, Proxmox won't have a gateway. Please visit here: https://docs.netgate.com/pfsense/en/latest/recipes/virtualize-proxmox-ve.html

Hi,
Thanks for sharing that issue. I've exactly the same. Took long time to define the issue.
So, on Proxmox 8 at least, when installing pfsense in a standard way without UEFI, we are able to start the firewall and so on. issue is that when restarting on Proxmox 8, pfsense doesn't restart (note that if trying multiple time, seems some of the versions may restart. So, consider it also as a random issue).
Point of attention may be that it is working fine under Proxmox 6. Absolutely no issues with that version.

@stephenw10

Well, I've just switched to virtio again, rebooted all of them, and it works... weird indeed.

Now, I got it right without the need to disable any firewall...my mistake was to move default gateway to pfSense. The default gateway is just for Proxmox management port to update itself and need to remain on the port originally assigned. Then, I made pfSense LAN that IP. Both accessible by my Mac Pro on the same browser tabs next to each other.

Screenshot 2023-05-06 at 3.14.07 PM.png

Guys, I just started over on this. I know I should troubleshoot these types of issues instead of starting over. But I did start over and I have a working Pfsense firewall. Something I've been working on for a long time. Now the hard part will be for me to build and configure my pentesting lab behind that firewall.

@benjaminbeckcsl said in PfSense Hetzner Dedicated Proxmox:

Nun habe ich eine dritte IP. welche die 66.77.88.3 ist. Wie bekomme ich die da auch noch drauf? Dadurch dass das GW von Hetzner bsp. 66.77.88.14 ist kann ich kein 32er Netz nehmen.

Was hat da genau das Gateway mit zu tun? So wie du das zeigst sind alle 3 IPs aus dem gleichen Netz? Dann haben die doch keine unterschiedlichen Gateways?

Ansonsten was @viragomann sagt, wenn die alle aus dem gleichen Subnetz sind, dann wird die Subnetzmaske des Netzes genommen, nicht /32. Ansonsten IP Alias und let's go. :)

pfSense will only allow access from the WAN side by default if there is only one interfaces assigned. As soon as you assign two of more interfaces all connections to WAN are blocked by default and you need to add WAN firewall rules to allow them.

@Patch @stephenw10 Thanks for your help! Replacing the switch fixed the issue! I factory reset the TP Link managed switch and it's working now too. It's sometimes the simple stuff you over look on the troubleshooting path that trip you up.

I could not determine any reason why the TP Link managed switch was preventing the Proxmox GUI from coming up. I checked everything before I reset it.

Thanks again!

Proxmox with a FreeBSD guest
Proxmox recommends using ntpd daemon in a FreeBSD client to maintain client real time synchronization. See pve.proxmox. com/wiki/FreeBSD_Guest_Notes and forum.proxmox. com/threads/time-drift-in-windows-7-guest.41268/#post-198848

Real time is set in the guest when it first starts. After which the guest clock drifts with the hosts processor crystal.

ntpd is the ntp daemon. See www.freebsd. org/cgi/man.cgi?query=ntpd&apropos=0&sektion=8&manpath=FreeBSD+12.2-stable&arch=default&format=html

Configured via /etc/ntp.conf

Command line interface ntpq See www.freebsd. org/cgi/man.cgi?query=ntpq&sektion=8&apropos=0&manpath=FreeBSD+12.2-RELEASE+and+Ports

11 minute hardware real time clock update can be enable by adding the line "SYNC_HWCLOCK=yes" to /etc/sysconfig/ntpdate

date gives time of day to 1 second resolution

Qemu guest agent
I think qemu can also sync time between the guest and Porxmox host.

Doing so is suggested github. com/aborche/qemu-guest-agent/blob/master/supported_command_reference.md

Other virtualization systems such as vmware have time synchronization between host and guest via the agent

Time synchronization should be done via only one system otherwise interaction between them decreases time system accuracy. Which mean if an agent and ntpd is used then there would need to be a way of disabling time synchronization via the agent. I'm not sure how to do this.

pfsense guest on Proxmox host
Given it is probably best to run ntp (chrony) on Proxmox then synch time in pfsense to the Proxmox host via ntpd over the LAN interface. Ideally I would like to do the following but not sure how to acheive that in pfsense

Enable ntpd on pfsense but not listening to clients on any interface. Selecting no interfaces in the GUI does the reverse.

Disable hardware real time clock update by pfsense. This is done every 11 minutes by chronyd on the Proxmox host. In pfsense the directory structure is different, so I can't goto to /etc/sysconfig/ntpdate and set SYNC_HWCLOCK=no

@jimp i just ended up reinstalling because nothing else was working. thanks for the help though