Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    Errors out going up of pfsense VM when saturating LAN interface

    General pfSense Questions
    4
    15
    1.1k
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • N
      netnerdy
      last edited by

      I'm trying to diagnose an issue that I have on my proxmox host.

      I have a pfsense 2.6 VM, running on virtio (vtnet) adapter.

      I also have ubuntu(22.04), truenas (core 13), freebsd (12.3-Release) VMs on the same host with virtio network adapters.

      Here is the issue: I see "errors out" increasing on pfsense LAN interface when I run iperf3 between pfsense and each host.

      I tried running between pfsense <-> ubuntu host (errors out increases)
      I tried running between pfsense <-> truenas host (errors out increases)
      I tried running between pfsense <-> freebsd host (errors out increases)

      I tried running between truenas <-> freebsd host (no errors out)
      I tried running between truenas <-> ubuntu host (no errors out)

      Proxmox is version 7.2-3. Kernel version is 5.15.30-2-pve.

      Any help is greatly appreciated.
      Thanks!

      Cool_CoronaC 1 Reply Last reply Reply Quote 0
      • Cool_CoronaC
        Cool_Corona @netnerdy
        last edited by

        @netnerdy Thats normal because all traffic gets bogged down when the link is saturated.

        N 1 Reply Last reply Reply Quote 0
        • N
          netnerdy @Cool_Corona
          last edited by

          @cool_corona It does not seem normal from my tests. i have used pfsense for years on different platforms. I didn’t see this behavior when I ran pfsense on bare metal or esxi or older versions of proxmox.

          Thats why I also created a vanilla freebsd vm to do the same tests. Freebsd does not show the same kind of errors when running iperf.

          V 1 Reply Last reply Reply Quote 0
          • V
            viragomann @netnerdy
            last edited by

            @netnerdy
            Did you enable hardware checksum offloading in the system advanced settings?

            N 1 Reply Last reply Reply Quote 0
            • stephenw10S
              stephenw10 Netgate Administrator
              last edited by

              Do you have flow control set the same on each?

              1 Reply Last reply Reply Quote 0
              • N
                netnerdy @viragomann
                last edited by

                @viragomann Yes I have tried enabling/disabling both tcp segmentation and hw checksum, but no avail.

                below is ifconfig output. vtnet0 is WAN, vtnet1 is LAN. My WAN and LAN both use vlans btw.

                vtnet0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
                options=900b8<VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,VLAN_HWFILTER,LINKSTATE>

                vtnet1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500 options=900b8<VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,VLAN_HWFILTER,LINKSTATE>

                1 Reply Last reply Reply Quote 0
                • stephenw10S
                  stephenw10 Netgate Administrator
                  last edited by stephenw10

                  Are you actually seeing traffic dropped?

                  Can you see what type of errors they are in the output of sysctl dev.vtnet.0?

                  N 1 Reply Last reply Reply Quote 0
                  • N
                    netnerdy @stephenw10
                    last edited by netnerdy

                    @stephenw10 Flow control is off for Hypervisor raw interface. does vtnet have its own flow control setting?

                    Right now, error count is at 967 (i had to restart my machine). here is sysctl:

                    dev.vtnet.1.txq0.rescheduled: 0
                    dev.vtnet.1.txq0.tso: 0
                    dev.vtnet.1.txq0.csum: 0
                    dev.vtnet.1.txq0.omcasts: 103
                    dev.vtnet.1.txq0.obytes: 7095877990
                    dev.vtnet.1.txq0.opackets: 14070402
                    dev.vtnet.1.rxq0.rescheduled: 143
                    dev.vtnet.1.rxq0.csum_failed: 0
                    dev.vtnet.1.rxq0.csum: 2228565
                    dev.vtnet.1.rxq0.ierrors: 0
                    dev.vtnet.1.rxq0.iqdrops: 0
                    dev.vtnet.1.rxq0.ibytes: 16575625620
                    dev.vtnet.1.rxq0.ipackets: 12832966
                    dev.vtnet.1.tx_task_rescheduled: 0
                    dev.vtnet.1.tx_tso_offloaded: 0
                    dev.vtnet.1.tx_csum_offloaded: 0
                    dev.vtnet.1.tx_defrag_failed: 0
                    dev.vtnet.1.tx_defragged: 0
                    dev.vtnet.1.tx_tso_not_tcp: 0
                    dev.vtnet.1.tx_tso_bad_ethtype: 0
                    dev.vtnet.1.tx_csum_bad_ethtype: 0
                    dev.vtnet.1.rx_task_rescheduled: 0
                    dev.vtnet.1.rx_csum_offloaded: 0
                    dev.vtnet.1.rx_csum_failed: 0
                    dev.vtnet.1.rx_csum_bad_proto: 0
                    dev.vtnet.1.rx_csum_bad_offset: 0
                    dev.vtnet.1.rx_csum_bad_ipproto: 0
                    dev.vtnet.1.rx_csum_bad_ethtype: 0
                    dev.vtnet.1.rx_mergeable_failed: 0
                    dev.vtnet.1.rx_enq_replacement_failed: 0
                    dev.vtnet.1.rx_frame_too_large: 0
                    dev.vtnet.1.mbuf_alloc_failed: 0
                    dev.vtnet.1.act_vq_pairs: 1
                    dev.vtnet.1.requested_vq_pairs: 0
                    dev.vtnet.1.max_vq_pairs: 1
                    dev.vtnet.1.%parent: virtio_pci3
                    dev.vtnet.1.%pnpinfo:
                    dev.vtnet.1.%location:
                    dev.vtnet.1.%driver: vtnet
                    dev.vtnet.1.%desc: VirtIO Networking Adapter

                    1 Reply Last reply Reply Quote 0
                    • stephenw10S
                      stephenw10 Netgate Administrator
                      last edited by

                      Hmm, nothing shown there...

                      Are you seeing that only on vtnet1?

                      Are you actually seeing connection dropped when the error rate is increasing?

                      N 1 Reply Last reply Reply Quote 0
                      • N
                        netnerdy @stephenw10
                        last edited by

                        @stephenw10 5F159528-72B7-4B21-9CCC-0410F635C5EA.jpeg

                        1 Reply Last reply Reply Quote 0
                        • stephenw10S
                          stephenw10 Netgate Administrator
                          last edited by

                          Ah, that looks like LAN is actually vtnet0. Can we see the sysctl output for that?

                          N 1 Reply Last reply Reply Quote 0
                          • N
                            netnerdy @stephenw10
                            last edited by

                            @stephenw10

                            nope LAN is on vtnet1.

                            5e7707f8-32e5-4f30-b1b2-a816ee0ad190-image.png

                            this is the sysctl output for vtnet0

                            dev.vtnet.0.txq0.rescheduled: 0
                            dev.vtnet.0.txq0.tso: 0
                            dev.vtnet.0.txq0.csum: 0
                            dev.vtnet.0.txq0.omcasts: 1139
                            dev.vtnet.0.txq0.obytes: 10781607078
                            dev.vtnet.0.txq0.opackets: 16092681
                            dev.vtnet.0.rxq0.rescheduled: 21
                            dev.vtnet.0.rxq0.csum_failed: 0
                            dev.vtnet.0.rxq0.csum: 16804156
                            dev.vtnet.0.rxq0.ierrors: 0
                            dev.vtnet.0.rxq0.iqdrops: 0
                            dev.vtnet.0.rxq0.ibytes: 29382393539
                            dev.vtnet.0.rxq0.ipackets: 26088829
                            dev.vtnet.0.tx_task_rescheduled: 0
                            dev.vtnet.0.tx_tso_offloaded: 0
                            dev.vtnet.0.tx_csum_offloaded: 0
                            dev.vtnet.0.tx_defrag_failed: 0
                            dev.vtnet.0.tx_defragged: 0
                            dev.vtnet.0.tx_tso_not_tcp: 0
                            dev.vtnet.0.tx_tso_bad_ethtype: 0
                            dev.vtnet.0.tx_csum_bad_ethtype: 0
                            dev.vtnet.0.rx_task_rescheduled: 0
                            dev.vtnet.0.rx_csum_offloaded: 0
                            dev.vtnet.0.rx_csum_failed: 0
                            dev.vtnet.0.rx_csum_bad_proto: 0
                            dev.vtnet.0.rx_csum_bad_offset: 0
                            dev.vtnet.0.rx_csum_bad_ipproto: 0
                            dev.vtnet.0.rx_csum_bad_ethtype: 0
                            dev.vtnet.0.rx_mergeable_failed: 0
                            dev.vtnet.0.rx_enq_replacement_failed: 0
                            dev.vtnet.0.rx_frame_too_large: 0
                            dev.vtnet.0.mbuf_alloc_failed: 0
                            dev.vtnet.0.act_vq_pairs: 1
                            dev.vtnet.0.requested_vq_pairs: 0
                            dev.vtnet.0.max_vq_pairs: 1
                            dev.vtnet.0.%parent: virtio_pci2
                            dev.vtnet.0.%pnpinfo:
                            dev.vtnet.0.%location:
                            dev.vtnet.0.%driver: vtnet
                            dev.vtnet.0.%desc: VirtIO Networking Adapter

                            1 Reply Last reply Reply Quote 0
                            • stephenw10S
                              stephenw10 Netgate Administrator
                              last edited by

                              Hmm, interesting the in/out bytes in your screenshot for WAN lined up exactly with the sysctl output for vtnet1. Just coincidence perhaps? Were those takes at approximately the same time?

                              You still haven't said if you're actually seeing connections drop when you see those errors?

                              Steve

                              N 1 Reply Last reply Reply Quote 0
                              • N
                                netnerdy @stephenw10
                                last edited by

                                @stephenw10 They were taken at different times. It must have been a coincidence.

                                How would I know which connections have dropped? Tcps auto retry anyway. Only weirdness i feel i have is my dns queries sometimes don’t resolve intermittently.

                                1 Reply Last reply Reply Quote 0
                                • stephenw10S
                                  stephenw10 Netgate Administrator
                                  last edited by

                                  Ah, OK do you see anything in the sysctls that looks like the same error count shown in the interface status?

                                  1 Reply Last reply Reply Quote 0
                                  • First post
                                    Last post
                                  Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.