Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    pfSense VM on Proxmox: PPPoE only works when parent NIC is PCI passthrough — virtual NIC breaks LAN→WAN traffic

    Scheduled Pinned Locked Moved Plus 25.11 Snapshots
    54 Posts 4 Posters 1.0k Views 6 Watching
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • N Offline
      netblues @w0w
      last edited by

      @w0w I see minor differences on the physical kvm interface, but I haven't done anything special, its at defaults.

      The mtu has been adjusted to 1508, but that can't be the issue. In any case, the brigded interfaces all use 1500 as mtu.

      As is, the same bridges are used at the same time by pfpls @25.07 pfplus @25.11rc and pfCE 2.8.1 with multiple pppoe connections over the same parent vlam.

      Only new rc fails to work as described above.

      ethtool -k enp1s0.31
Features for enp1s0.31:
rx-checksumming: off [fixed]
tx-checksumming: on
   tx-checksum-ipv4: off [fixed]
   tx-checksum-ip-generic: on
   tx-checksum-ipv6: off [fixed]
   tx-checksum-fcoe-crc: off [requested on]
   tx-checksum-sctp: off [requested on]
scatter-gather: on
   tx-scatter-gather: on
   tx-scatter-gather-fraglist: off [requested on]
tcp-segmentation-offload: on
   tx-tcp-segmentation: on
   tx-tcp-ecn-segmentation: on
   tx-tcp-mangleid-segmentation: on
   tx-tcp6-segmentation: on
generic-segmentation-offload: on
generic-receive-offload: on
large-receive-offload: off [fixed]
rx-vlan-offload: off [fixed]
tx-vlan-offload: off [fixed]
ntuple-filters: off [fixed]
receive-hashing: off [fixed]
highdma: on
rx-vlan-filter: off [fixed]
vlan-challenged: off [fixed]
tx-gso-robust: off [fixed]
tx-fcoe-segmentation: off [requested on]
tx-gre-segmentation: off [requested on]
tx-gre-csum-segmentation: off [requested on]
tx-ipxip4-segmentation: off [requested on]
tx-ipxip6-segmentation: off [requested on]
tx-udp_tnl-segmentation: off [requested on]
tx-udp_tnl-csum-segmentation: off [requested on]
tx-gso-partial: off [fixed]
tx-tunnel-remcsum-segmentation: off [fixed]
tx-sctp-segmentation: on
tx-esp-segmentation: off [fixed]
tx-udp-segmentation: on
tx-gso-list: on
tx-nocache-copy: off
loopback: off [fixed]
rx-fcs: off [fixed]
rx-all: off [fixed]
tx-vlan-stag-hw-insert: off [fixed]
rx-vlan-stag-hw-parse: off [fixed]
rx-vlan-stag-filter: off [fixed]
l2-fwd-offload: off [fixed]
hw-tc-offload: off [fixed]
esp-hw-offload: off [fixed]
esp-tx-csum-hw-offload: off [fixed]
rx-udp_tunnel-port-offload: off [fixed]
tls-hw-tx-offload: off [fixed]
tls-hw-rx-offload: off [fixed]
rx-gro-hw: off [fixed]
tls-hw-record: off [fixed]
rx-gro-list: off
macsec-hw-offload: off [fixed]
rx-udp-gro-forwarding: off
hsr-tag-ins-offload: off [fixed]
hsr-tag-rm-offload: off [fixed]
hsr-fwd-offload: off [fixed]
hsr-dup-offload: off [fixed]

      ethtool -k enp1s0.835
Features for enp1s0.835:
rx-checksumming: off [fixed]
tx-checksumming: on
	tx-checksum-ipv4: off [fixed]
	tx-checksum-ip-generic: on
	tx-checksum-ipv6: off [fixed]
	tx-checksum-fcoe-crc: off [requested on]
	tx-checksum-sctp: off [requested on]
scatter-gather: on
	tx-scatter-gather: on
	tx-scatter-gather-fraglist: off [requested on]
tcp-segmentation-offload: on
	tx-tcp-segmentation: on
	tx-tcp-ecn-segmentation: on
	tx-tcp-mangleid-segmentation: on
	tx-tcp6-segmentation: on
generic-segmentation-offload: on
generic-receive-offload: on
large-receive-offload: off [fixed]
rx-vlan-offload: off [fixed]
tx-vlan-offload: off [fixed]
ntuple-filters: off [fixed]
receive-hashing: off [fixed]
highdma: on
rx-vlan-filter: off [fixed]
vlan-challenged: off [fixed]
tx-gso-robust: off [fixed]
tx-fcoe-segmentation: off [requested on]
tx-gre-segmentation: off [requested on]
tx-gre-csum-segmentation: off [requested on]
tx-ipxip4-segmentation: off [requested on]
tx-ipxip6-segmentation: off [requested on]
tx-udp_tnl-segmentation: off [requested on]
tx-udp_tnl-csum-segmentation: off [requested on]
tx-gso-partial: off [fixed]
tx-tunnel-remcsum-segmentation: off [fixed]
tx-sctp-segmentation: on
tx-esp-segmentation: off [fixed]
tx-udp-segmentation: on
tx-gso-list: on
tx-nocache-copy: off
loopback: off [fixed]
rx-fcs: off [fixed]
rx-all: off [fixed]
tx-vlan-stag-hw-insert: off [fixed]
rx-vlan-stag-hw-parse: off [fixed]
rx-vlan-stag-filter: off [fixed]
l2-fwd-offload: off [fixed]
hw-tc-offload: off [fixed]
esp-hw-offload: off [fixed]
esp-tx-csum-hw-offload: off [fixed]
rx-udp_tunnel-port-offload: off [fixed]
tls-hw-tx-offload: off [fixed]
tls-hw-rx-offload: off [fixed]
rx-gro-hw: off [fixed]
tls-hw-record: off [fixed]
rx-gro-list: off
macsec-hw-offload: off [fixed]
rx-udp-gro-forwarding: off
hsr-tag-ins-offload: off [fixed]
hsr-tag-rm-offload: off [fixed]
hsr-fwd-offload: off [fixed]
hsr-dup-offload: off [fixed]

      ethtool -k enp1s0
Features for enp1s0:
rx-checksumming: on
tx-checksumming: on
	tx-checksum-ipv4: on
	tx-checksum-ip-generic: off [fixed]
	tx-checksum-ipv6: on
	tx-checksum-fcoe-crc: off [fixed]
	tx-checksum-sctp: off [fixed]
scatter-gather: on
	tx-scatter-gather: on
	tx-scatter-gather-fraglist: off [fixed]
tcp-segmentation-offload: on
	tx-tcp-segmentation: on
	tx-tcp-ecn-segmentation: off [fixed]
	tx-tcp-mangleid-segmentation: off
	tx-tcp6-segmentation: on
generic-segmentation-offload: on
generic-receive-offload: on
large-receive-offload: off [fixed]
rx-vlan-offload: on
tx-vlan-offload: on
ntuple-filters: off [fixed]
receive-hashing: on
highdma: on [fixed]
rx-vlan-filter: on [fixed]
vlan-challenged: off [fixed]
tx-gso-robust: off [fixed]
tx-fcoe-segmentation: off [fixed]
tx-gre-segmentation: off [fixed]
tx-gre-csum-segmentation: off [fixed]
tx-ipxip4-segmentation: off [fixed]
tx-ipxip6-segmentation: off [fixed]
tx-udp_tnl-segmentation: off [fixed]
tx-udp_tnl-csum-segmentation: off [fixed]
tx-gso-partial: off [fixed]
tx-tunnel-remcsum-segmentation: off [fixed]
tx-sctp-segmentation: off [fixed]
tx-esp-segmentation: off [fixed]
tx-udp-segmentation: off [fixed]
tx-gso-list: off [fixed]
tx-nocache-copy: off
loopback: off
rx-fcs: off
rx-all: off
tx-vlan-stag-hw-insert: off
rx-vlan-stag-hw-parse: on
rx-vlan-stag-filter: on [fixed]
l2-fwd-offload: off [fixed]
hw-tc-offload: off [fixed]
esp-hw-offload: off [fixed]
esp-tx-csum-hw-offload: off [fixed]
rx-udp_tunnel-port-offload: off [fixed]
tls-hw-tx-offload: off [fixed]
tls-hw-rx-offload: off [fixed]
rx-gro-hw: off [fixed]
tls-hw-record: off [fixed]
rx-gro-list: off
macsec-hw-offload: off [fixed]
rx-udp-gro-forwarding: off
hsr-tag-ins-offload: off [fixed]
hsr-tag-rm-offload: off [fixed]
hsr-fwd-offload: off [fixed]
hsr-dup-offload: off [fixed]
      1 Reply Last reply Reply Quote 0
      • stephenw10S Offline
        stephenw10 Netgate Administrator
        last edited by

        Can you ping across it with large packets?

        When ICMP passes and nothing else does it's usually either an MTU issue or some sort of asymmetric routing problem. But neither should have changed in 25.11.

        The packages widget issue is known: https://forum.netgate.com/topic/199375/zero-packages-install/

        N w0wW 2 Replies Last reply Reply Quote 0
        • N Offline
          netblues @stephenw10
          last edited by netblues

          @stephenw10 Obviously yes

          ping 8.8.4.4 -l 1472 -f

          Pinging 8.8.4.4 with 1472 bytes of data:
          Reply from 8.8.4.4: bytes=1472 time=14ms TTL=112
          Reply from 8.8.4.4: bytes=1472 time=14ms TTL=112
          Reply from 8.8.4.4: bytes=1472 time=14ms TTL=112
          Reply from 8.8.4.4: bytes=1472 time=14ms TTL=112

          Ping statistics for 8.8.4.4:
          Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
          Approximate round trip times in milli-seconds:
          Minimum = 14ms, Maximum = 14ms, Average = 14ms

          And same config couldn't cause mtu issues imho too.

          And this is a plain vanila config, only one wan one lan interface, no policy routing, nothing fancy

          w0wW 1 Reply Last reply Reply Quote 0
          • stephenw10S Offline
            stephenw10 Netgate Administrator
            last edited by

            Yup I agree it shouldn't. But PPPoE has always had MTU/MSS requirements and if_pppoe specifically had an MSS issue previously.

            w0wW 1 Reply Last reply Reply Quote 0
            • w0wW Away
              w0w @stephenw10
              last edited by

              @stephenw10 said in pfSense VM on Proxmox: PPPoE only works when parent NIC is PCI passthrough — virtual NIC breaks LAN→WAN traffic:

              some sort of asymmetric routing problem.

              I think these might be some of the commits on the FreeBSD side from the list I posted above. I don’t know whether Netgate uses the vanilla vtnet driver from FreeBSD 16 or their own custom patches. From what I can see, those checksum-related patches could potentially trigger this issue, but ifconfig vtnet0 -rxcsum -txcsum -tso -lro should disable this functionality, so I have no idea what is actually happening.

              1 Reply Last reply Reply Quote 0
              • w0wW Away
                w0w @stephenw10
                last edited by w0w

                @stephenw10 said in pfSense VM on Proxmox: PPPoE only works when parent NIC is PCI passthrough — virtual NIC breaks LAN→WAN traffic:

                and if_pppoe

                BTW, mpd have the same issue. More precisely, the bug is not related to if_pppoe or mpd5, since it appears the same way with both.

                1 Reply Last reply Reply Quote 0
                • w0wW Away
                  w0w @netblues
                  last edited by

                  @netblues

                  Do you have the same settings?

                  2f15fa41-dcea-4151-8809-7f5bb87f2d90-image.png code_text

                  N 1 Reply Last reply Reply Quote 0
                  • N Offline
                    netblues @w0w
                    last edited by

                    @w0w Sure. As I said, default installation with minimal setup, just to make the lan to nat over pppoe to the Internets.

                    2da5b0b5-043d-4462-b5dc-c2367f2fbe95-image.png

                    w0wW 1 Reply Last reply Reply Quote 1
                    • w0wW Away
                      w0w @netblues
                      last edited by w0w

                      @netblues
                      Just for reference, what hardware is used actually to run the hypervisor and version of the hypervisor?

                      N 1 Reply Last reply Reply Quote 0
                      • N Offline
                        netblues @w0w
                        last edited by

                        @w0w Red Hat Enterprise Linux 9.7 (Plow)
                        on a HP HP EliteDesk 800 G5 SFF
                        and a Mellanox Technologies MT27500 Family [ConnectX-3] (ConnectX-3 10 GbE Single Port SFP+ Adapter) mlx4_core feeding a managed l3 switch only for vlans

                        1 Reply Last reply Reply Quote 1
                        • First post
                          Last post
                        Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.