pfSense VM on Proxmox: PPPoE only works when parent NIC is PCI passthrough — virtual NIC breaks LAN→WAN traffic
-
I was able to reproduce this bug: I installed 25.07.1, restored the configuration, verified that LAN clients had Internet access, and then upgraded to the latest RC. After the upgrade, the clients no longer had Internet access.
That's fun... -
@w0w And most peobably can only ping too
-
@netblues
Yes, like only ICMP working -
@w0w So we definitely have an issue here. It can't be a configuration issue, and certainly NOT a firewall rules issue.
But I remain clueless where to look. (Besides the fact that I need to revert for practical reasons, and running another pf plus vm in parallel for testing has licensing issues too) -
@netblues
I dug a bit deeper. I compared the system that was installed from scratch with the one that was upgraded. Of course, things went a bit sideways, but overall there are noticeable differences in both libraries and some binaries, which raises some questions — although in general this could simply be a consequence of the FreeBSD version upgrade.
By the way, have you tried installing it using the Netgate installer? -
@w0w So you say that by doing a clean default install with netgate installer AND restoring the config would work in latest RC?
Can't check this right now, someone might shoot me and it would be netgates fault

-
@netblues said in pfSense VM on Proxmox: PPPoE only works when parent NIC is PCI passthrough — virtual NIC breaks LAN→WAN traffic:
So you say that by doing a clean default install with netgate installer AND restoring the config would work in latest RC?
It worked once at least, but I didn’t try it again

-
@w0w Brewing... I know in a while
Nada. Issue remains the same. I installed directly into latest rc 25.11.r.20251126.1732
and restored config.Only ping over pppoe.
-
ok… thats interesting…
ifconfig vtnet0 -rxcsum -txcsum -rxcsum6 -txcsum6 -vlanhwtag -vlanhwcsumTry this on LAN interface. Hope this works for you...
I just compared ifconfig output for both working and non working VMs and looks like working VM on every interface have options=880008<VLAN_MTU,LINKSTATE,HWSTATS>
So I applied it to the pppoe parent - still no go... and then I've tried LAN interface and it worked for me. -
@w0w said in pfSense VM on Proxmox: PPPoE only works when parent NIC is PCI passthrough — virtual NIC breaks LAN→WAN traffic:
ifconfig vtnet0 -rxcsum -txcsum -rxcsum6 -txcsum6 -vlanhwtag -vlanhwcsum
Tried that.. Still , no dice. :(
-
@netblues
Please show your ifconfig output for LAN and pppoe parent interface. -
@w0w I have now created a fresh default install
Directly install 25.11rc from netgate installer , configured everything by the gui just for a single lan, and a single pppoe connection.Automatic outbound nat etc. No changes anywhere
ping works, everything else on physical lan fails (miserably)
pfsense (and anything on virtual) can install packages, and has full Internethave tried disabling checksums too. No dice
ifconfig vtnet1 vtnet1: flags=1008843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1500 description: WAN options=8800a8<VLAN_MTU,JUMBO_MTU,VLAN_HWCSUM,LINKSTATE,HWSTATS> ether d4:5d:64:08:66:46 inet6 fe80::d65d:64ff:fe08:6646%vtnet1 prefixlen 64 scopeid 0x2 media: Ethernet autoselect (10Gbase-T <full-duplex>) status: active nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL> pppoe0: flags=10088d1<UP,POINTOPOINT,RUNNING,NOARP,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1492 description: Ftth1G options=0 inet 100.79.101.245 --> 10.106.108.100 netmask 0xffffffff inet6 fe80::d65d:64ff:fe08:6646%pppoe0 prefixlen 64 scopeid 0x7 nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>As a side note, when restoring configuration to a fresh install 25.11rc, all packages got reinstalled, however the widget says no packages.
I have tried removing it and adding again. Nada.
Tried adding a new package (from gui), package got installed the widget insists. No packages installed.Steps to reproduce. Install fresh pfplus 25.11rc, restore config that has package widget and some packages, wait for the packages reinstallation, and voila !
-
@netblues
You forgot to show your ifconfig LAN output.loader.conf.local (you need to reboot after making changes)
hw.vtnet.altq_disable=1 hw.vtnet.tso_disable=1 hw.vtnet.csum_disable=1LAN
ix0: flags=1008943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1500 options=4813828<VLAN_MTU,JUMBO_MTU,WOL_UCAST,WOL_MCAST,WOL_MAGIC,VLAN_HWFILTER,HWSTATS,MEXTPG> ether a0:3------25 inet6 fe80::aab8:e0ff:fe02:655a%ix0 prefixlen 64 scopeid 0x1 media: Ethernet autoselect (10Gbase-T <full-duplex>) status: active nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>WAN parent
vtnet0: flags=1008843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1500 options=880008<VLAN_MTU,LINKSTATE,HWSTATS> ether a-----:24 inet6 fe80::aab8:e0ff:fe02:655a%vtnet0 prefixlen 64 scopeid 0x6 media: Ethernet autoselect (10Gbase-T <full-duplex>) status: active nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>Shell Output - sysctl hw.vtnet.
hw.vtnet.altq_disable: 1 hw.vtnet.lro_mbufq_depth: 0 hw.vtnet.lro_entry_count: 128 hw.vtnet.rx_process_limit: 1024 hw.vtnet.tso_maxlen: 65535 hw.vtnet.mq_max_pairs: 32 hw.vtnet.mq_disable: 0 hw.vtnet.lro_disable: 1 hw.vtnet.tso_disable: 1 hw.vtnet.fixup_needs_csum: 0 hw.vtnet.csum_disable: 1Side note — if you have vlans on LAN you should not use -vlanhwtag posted previously, this will break vlans
-
@w0w
All vlan configuration is handled at the hypervisor level.
pf sees only virtual interfaces.Here is the output
sysctl hw.vtnet hw.vtnet.altq_disable: 1 hw.vtnet.lro_mbufq_depth: 0 hw.vtnet.lro_entry_count: 128 hw.vtnet.rx_process_limit: 1024 hw.vtnet.tso_maxlen: 65535 hw.vtnet.mq_max_pairs: 32 hw.vtnet.mq_disable: 0 hw.vtnet.lro_disable: 1 hw.vtnet.tso_disable: 1 hw.vtnet.fixup_needs_csum: 0 hw.vtnet.csum_disable: 1ifconfig vtnet0 vtnet0: flags=1008943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1500 options=880028<VLAN_MTU,JUMBO_MTU,LINKSTATE,HWSTATS> ether 52:54:00:05:01:fb inet 192.168.31.3 netmask 0xffffff00 broadcast 192.168.31.255 inet6 fe80::5054:ff:fe05:1fb%vtnet0 prefixlen 64 scopeid 0x1 media: Ethernet autoselect (10Gbase-T <full-duplex>) status: active nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL> -
@netblues
I kinda screwed up… I forgot to mention that this ix0 is part of a LAGG interface, and the LAN itself is using that LAGG. This shouldn’t really affect anything, but I’ll check if that’s the issue. Also, I don’t remember changing any settings for this NIC on the host, I think I left it as is. -
Configured LAN to use ix0 directly — nothing changed.
-
@netblues said in pfSense VM on Proxmox: PPPoE only works when parent NIC is PCI passthrough — virtual NIC breaks LAN→WAN traffic:
JUMBO_MTU
Hmm… I don't see any jumbo settings on my vtnet interfaces, did you change something? VM setiings? Nonstandard MTU?
Now I have configured it directly for both pppoe and LAN
vtnet0: flags=1008843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1500 options=880008<VLAN_MTU,LINKSTATE,HWSTATS> ether a- inet6 fe80::aab8:e0ff:fe02:655a%vtnet0 prefixlen 64 scopeid 0x5 media: Ethernet autoselect (10Gbase-T <full-duplex>) status: active nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL> vtnet1: flags=1008943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1500 description: LAN options=880008<VLAN_MTU,LINKSTATE,HWSTATS> ether a- inet 10.0.67.2 netmask 0xffffff00 broadcast 10.0.67.255 inet 10.0.67.5 netmask 0xffffff00 broadcast 10.0.67.255 vhid 5 inet 10.0.70.5 netmask 0xffffff00 broadcast 10.0.70.255 vhid 10 inet 10.0.70.11 netmask 0xffffff00 broadcast 10.0.70.255 inet6 fe80::a236:9fff:fef8:f225%vtnet1 prefixlen 64 scopeid 0x6 inet6 fd00:1234:abcd:1::2 prefixlen 64 inet6 fd00:1234:abcd:1::5 prefixlen 64 vhid 12 carp: MASTER vhid 5 advbase 5 advskew 100 peer 224.0.0.18 peer6 ff02::12 carp: MASTER vhid 10 advbase 5 advskew 100 peer 224.0.0.18 peer6 ff02::12 carp: MASTER vhid 12 advbase 5 advskew 100 peer 224.0.0.18 peer6 ff02::12 media: Ethernet autoselect (10Gbase-T <full-duplex>) status: active nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>Ok, I've changed MTU to 9000 in proxmox for the LAN card/bridge/vtnet
vtnet1: flags=1008943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1500 description: LAN options=880028<VLAN_MTU,JUMBO_MTU,LINKSTATE,HWSTATS>Still working for me… no problem.
Proxmox settings for WAN parent
:~# ethtool -k enp6s0f0 Features for enp6s0f0: rx-checksumming: on tx-checksumming: on tx-checksum-ipv4: off [fixed] tx-checksum-ip-generic: on tx-checksum-ipv6: off [fixed] tx-checksum-fcoe-crc: on [fixed] tx-checksum-sctp: on scatter-gather: on tx-scatter-gather: on tx-scatter-gather-fraglist: off [fixed] tcp-segmentation-offload: on tx-tcp-segmentation: on tx-tcp-ecn-segmentation: off [fixed] tx-tcp-mangleid-segmentation: off tx-tcp6-segmentation: on tx-tcp-accecn-segmentation: off [fixed] generic-segmentation-offload: on generic-receive-offload: on large-receive-offload: off rx-vlan-offload: on tx-vlan-offload: on ntuple-filters: off receive-hashing: on highdma: on [fixed] rx-vlan-filter: on vlan-challenged: off [fixed] tx-gso-robust: off [fixed] tx-fcoe-segmentation: on [fixed] tx-gre-segmentation: on tx-gre-csum-segmentation: on tx-ipxip4-segmentation: on tx-ipxip6-segmentation: on tx-udp_tnl-segmentation: on tx-udp_tnl-csum-segmentation: on tx-gso-partial: on tx-tunnel-remcsum-segmentation: off [fixed] tx-sctp-segmentation: off [fixed] tx-esp-segmentation: on tx-udp-segmentation: on tx-gso-list: off [fixed] tx-nocache-copy: off loopback: off [fixed] rx-fcs: off [fixed] rx-all: off tx-vlan-stag-hw-insert: off [fixed] rx-vlan-stag-hw-parse: off [fixed] rx-vlan-stag-filter: off [fixed] l2-fwd-offload: off hw-tc-offload: off esp-hw-offload: on esp-tx-csum-hw-offload: on rx-udp_tunnel-port-offload: on tls-hw-tx-offload: off [fixed] tls-hw-rx-offload: off [fixed] rx-gro-hw: off [fixed] tls-hw-record: off [fixed] rx-gro-list: off macsec-hw-offload: off [fixed] rx-udp-gro-forwarding: off hsr-tag-ins-offload: off [fixed] hsr-tag-rm-offload: off [fixed] hsr-fwd-offload: off [fixed] hsr-dup-offload: off [fixed]LAN
ethtool -k enp6s0f1 Features for enp6s0f1: rx-checksumming: on tx-checksumming: on tx-checksum-ipv4: off [fixed] tx-checksum-ip-generic: on tx-checksum-ipv6: off [fixed] tx-checksum-fcoe-crc: on [fixed] tx-checksum-sctp: on scatter-gather: on tx-scatter-gather: on tx-scatter-gather-fraglist: off [fixed] tcp-segmentation-offload: on tx-tcp-segmentation: on tx-tcp-ecn-segmentation: off [fixed] tx-tcp-mangleid-segmentation: off tx-tcp6-segmentation: on tx-tcp-accecn-segmentation: off [fixed] generic-segmentation-offload: on generic-receive-offload: on large-receive-offload: off rx-vlan-offload: on tx-vlan-offload: on ntuple-filters: off receive-hashing: on highdma: on [fixed] rx-vlan-filter: on vlan-challenged: off [fixed] tx-gso-robust: off [fixed] tx-fcoe-segmentation: on [fixed] tx-gre-segmentation: on tx-gre-csum-segmentation: on tx-ipxip4-segmentation: on tx-ipxip6-segmentation: on tx-udp_tnl-segmentation: on tx-udp_tnl-csum-segmentation: on tx-gso-partial: on tx-tunnel-remcsum-segmentation: off [fixed] tx-sctp-segmentation: off [fixed] tx-esp-segmentation: on tx-udp-segmentation: on tx-gso-list: off [fixed] tx-nocache-copy: off loopback: off [fixed] rx-fcs: off [fixed] rx-all: off tx-vlan-stag-hw-insert: off [fixed] rx-vlan-stag-hw-parse: off [fixed] rx-vlan-stag-filter: off [fixed] l2-fwd-offload: off hw-tc-offload: off esp-hw-offload: on esp-tx-csum-hw-offload: on rx-udp_tunnel-port-offload: on tls-hw-tx-offload: off [fixed] tls-hw-rx-offload: off [fixed] rx-gro-hw: off [fixed] tls-hw-record: off [fixed] rx-gro-list: off macsec-hw-offload: off [fixed] rx-udp-gro-forwarding: off hsr-tag-ins-offload: off [fixed] hsr-tag-rm-offload: off [fixed] hsr-fwd-offload: off [fixed] hsr-dup-offload: off [fixed] -
@w0w I see minor differences on the physical kvm interface, but I haven't done anything special, its at defaults.
The mtu has been adjusted to 1508, but that can't be the issue. In any case, the brigded interfaces all use 1500 as mtu.
As is, the same bridges are used at the same time by pfpls @25.07 pfplus @25.11rc and pfCE 2.8.1 with multiple pppoe connections over the same parent vlam.
Only new rc fails to work as described above.
ethtool -k enp1s0.31 Features for enp1s0.31: rx-checksumming: off [fixed] tx-checksumming: on tx-checksum-ipv4: off [fixed] tx-checksum-ip-generic: on tx-checksum-ipv6: off [fixed] tx-checksum-fcoe-crc: off [requested on] tx-checksum-sctp: off [requested on] scatter-gather: on tx-scatter-gather: on tx-scatter-gather-fraglist: off [requested on] tcp-segmentation-offload: on tx-tcp-segmentation: on tx-tcp-ecn-segmentation: on tx-tcp-mangleid-segmentation: on tx-tcp6-segmentation: on generic-segmentation-offload: on generic-receive-offload: on large-receive-offload: off [fixed] rx-vlan-offload: off [fixed] tx-vlan-offload: off [fixed] ntuple-filters: off [fixed] receive-hashing: off [fixed] highdma: on rx-vlan-filter: off [fixed] vlan-challenged: off [fixed] tx-gso-robust: off [fixed] tx-fcoe-segmentation: off [requested on] tx-gre-segmentation: off [requested on] tx-gre-csum-segmentation: off [requested on] tx-ipxip4-segmentation: off [requested on] tx-ipxip6-segmentation: off [requested on] tx-udp_tnl-segmentation: off [requested on] tx-udp_tnl-csum-segmentation: off [requested on] tx-gso-partial: off [fixed] tx-tunnel-remcsum-segmentation: off [fixed] tx-sctp-segmentation: on tx-esp-segmentation: off [fixed] tx-udp-segmentation: on tx-gso-list: on tx-nocache-copy: off loopback: off [fixed] rx-fcs: off [fixed] rx-all: off [fixed] tx-vlan-stag-hw-insert: off [fixed] rx-vlan-stag-hw-parse: off [fixed] rx-vlan-stag-filter: off [fixed] l2-fwd-offload: off [fixed] hw-tc-offload: off [fixed] esp-hw-offload: off [fixed] esp-tx-csum-hw-offload: off [fixed] rx-udp_tunnel-port-offload: off [fixed] tls-hw-tx-offload: off [fixed] tls-hw-rx-offload: off [fixed] rx-gro-hw: off [fixed] tls-hw-record: off [fixed] rx-gro-list: off macsec-hw-offload: off [fixed] rx-udp-gro-forwarding: off hsr-tag-ins-offload: off [fixed] hsr-tag-rm-offload: off [fixed] hsr-fwd-offload: off [fixed] hsr-dup-offload: off [fixed]ethtool -k enp1s0.835 Features for enp1s0.835: rx-checksumming: off [fixed] tx-checksumming: on tx-checksum-ipv4: off [fixed] tx-checksum-ip-generic: on tx-checksum-ipv6: off [fixed] tx-checksum-fcoe-crc: off [requested on] tx-checksum-sctp: off [requested on] scatter-gather: on tx-scatter-gather: on tx-scatter-gather-fraglist: off [requested on] tcp-segmentation-offload: on tx-tcp-segmentation: on tx-tcp-ecn-segmentation: on tx-tcp-mangleid-segmentation: on tx-tcp6-segmentation: on generic-segmentation-offload: on generic-receive-offload: on large-receive-offload: off [fixed] rx-vlan-offload: off [fixed] tx-vlan-offload: off [fixed] ntuple-filters: off [fixed] receive-hashing: off [fixed] highdma: on rx-vlan-filter: off [fixed] vlan-challenged: off [fixed] tx-gso-robust: off [fixed] tx-fcoe-segmentation: off [requested on] tx-gre-segmentation: off [requested on] tx-gre-csum-segmentation: off [requested on] tx-ipxip4-segmentation: off [requested on] tx-ipxip6-segmentation: off [requested on] tx-udp_tnl-segmentation: off [requested on] tx-udp_tnl-csum-segmentation: off [requested on] tx-gso-partial: off [fixed] tx-tunnel-remcsum-segmentation: off [fixed] tx-sctp-segmentation: on tx-esp-segmentation: off [fixed] tx-udp-segmentation: on tx-gso-list: on tx-nocache-copy: off loopback: off [fixed] rx-fcs: off [fixed] rx-all: off [fixed] tx-vlan-stag-hw-insert: off [fixed] rx-vlan-stag-hw-parse: off [fixed] rx-vlan-stag-filter: off [fixed] l2-fwd-offload: off [fixed] hw-tc-offload: off [fixed] esp-hw-offload: off [fixed] esp-tx-csum-hw-offload: off [fixed] rx-udp_tunnel-port-offload: off [fixed] tls-hw-tx-offload: off [fixed] tls-hw-rx-offload: off [fixed] rx-gro-hw: off [fixed] tls-hw-record: off [fixed] rx-gro-list: off macsec-hw-offload: off [fixed] rx-udp-gro-forwarding: off hsr-tag-ins-offload: off [fixed] hsr-tag-rm-offload: off [fixed] hsr-fwd-offload: off [fixed] hsr-dup-offload: off [fixed]ethtool -k enp1s0 Features for enp1s0: rx-checksumming: on tx-checksumming: on tx-checksum-ipv4: on tx-checksum-ip-generic: off [fixed] tx-checksum-ipv6: on tx-checksum-fcoe-crc: off [fixed] tx-checksum-sctp: off [fixed] scatter-gather: on tx-scatter-gather: on tx-scatter-gather-fraglist: off [fixed] tcp-segmentation-offload: on tx-tcp-segmentation: on tx-tcp-ecn-segmentation: off [fixed] tx-tcp-mangleid-segmentation: off tx-tcp6-segmentation: on generic-segmentation-offload: on generic-receive-offload: on large-receive-offload: off [fixed] rx-vlan-offload: on tx-vlan-offload: on ntuple-filters: off [fixed] receive-hashing: on highdma: on [fixed] rx-vlan-filter: on [fixed] vlan-challenged: off [fixed] tx-gso-robust: off [fixed] tx-fcoe-segmentation: off [fixed] tx-gre-segmentation: off [fixed] tx-gre-csum-segmentation: off [fixed] tx-ipxip4-segmentation: off [fixed] tx-ipxip6-segmentation: off [fixed] tx-udp_tnl-segmentation: off [fixed] tx-udp_tnl-csum-segmentation: off [fixed] tx-gso-partial: off [fixed] tx-tunnel-remcsum-segmentation: off [fixed] tx-sctp-segmentation: off [fixed] tx-esp-segmentation: off [fixed] tx-udp-segmentation: off [fixed] tx-gso-list: off [fixed] tx-nocache-copy: off loopback: off rx-fcs: off rx-all: off tx-vlan-stag-hw-insert: off rx-vlan-stag-hw-parse: on rx-vlan-stag-filter: on [fixed] l2-fwd-offload: off [fixed] hw-tc-offload: off [fixed] esp-hw-offload: off [fixed] esp-tx-csum-hw-offload: off [fixed] rx-udp_tunnel-port-offload: off [fixed] tls-hw-tx-offload: off [fixed] tls-hw-rx-offload: off [fixed] rx-gro-hw: off [fixed] tls-hw-record: off [fixed] rx-gro-list: off macsec-hw-offload: off [fixed] rx-udp-gro-forwarding: off hsr-tag-ins-offload: off [fixed] hsr-tag-rm-offload: off [fixed] hsr-fwd-offload: off [fixed] hsr-dup-offload: off [fixed] -
Can you ping across it with large packets?
When ICMP passes and nothing else does it's usually either an MTU issue or some sort of asymmetric routing problem. But neither should have changed in 25.11.
The packages widget issue is known: https://forum.netgate.com/topic/199375/zero-packages-install/
-
@stephenw10 Obviously yes
ping 8.8.4.4 -l 1472 -f
Pinging 8.8.4.4 with 1472 bytes of data:
Reply from 8.8.4.4: bytes=1472 time=14ms TTL=112
Reply from 8.8.4.4: bytes=1472 time=14ms TTL=112
Reply from 8.8.4.4: bytes=1472 time=14ms TTL=112
Reply from 8.8.4.4: bytes=1472 time=14ms TTL=112Ping statistics for 8.8.4.4:
Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
Minimum = 14ms, Maximum = 14ms, Average = 14msAnd same config couldn't cause mtu issues imho too.
And this is a plain vanila config, only one wan one lan interface, no policy routing, nothing fancy