Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    Extreme slow internet speed pfsense over proxmox

    Scheduled Pinned Locked Moved Virtualization
    20 Posts 7 Posters 10.5k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • S
      skogs
      last edited by

      lol

      1 Reply Last reply Reply Quote 0
      • A
        abhishekakt
        last edited by

        I already tried these.. no help. Something is not right. when I run iperf on vm connected to same vswitch with pfsense, it never exceeds 300 Mbps and goes does as 80 mbps. It is slower than my physical LAN

        1 Reply Last reply Reply Quote 0
        • D
          digdug3
          last edited by

          1. Use virtio
          2. Use the checkboxes as stated above and reboot VM after setting them
          3. Disabled "firewall" in the VM in Proxmox
          4. Use 2.4.4-p3 for now. 2.4.5 has issues
          O 1 Reply Last reply Reply Quote 1
          • S
            skogs
            last edited by

            So for giggles I did do some extra testing last week. Re-did the pfsense on a tiny physical machine again. Pretty much no matter what I did the physical instance drastically outperformed the virtual.

            When it was virtualized I was also mirroring off the traffic to a third virtual NIC for monitoring on the inside of the firewall anything going out. In the physical environment this is taken care of by a physical switch. Turning this off did improve my data processing somewhat, but not entirely. Lets face it; a virtual network adaptor resides in memory and shouldn't really slow things down much.

            I don't have half the network bandwidth you guys seem to have.
            Physical: averaging 65-82Mbps on what is advertised as a 100M line.
            Virtualized: depending on settings anywhere from 30Mbps to 45Mbps.

            Physical is using little Dual Core Celeron N3160 at 1.6Ghz
            Virtualized is using little Quad Core Ryzen 1200 at 3Ghz
            The ryzen would generally outperform the Celeron a bit less than 3x with a vastly better single process speed and multi process. Kind of odd. I've never seen this large of a performance hit on Proxmox before.

            1 Reply Last reply Reply Quote 0
            • D
              digdug3
              last edited by

              I'm getting 100/100 on proxmox, the same as I'm getting when connected physical.
              What NICs are you using. I only use Intel based ones for Proxmox.
              Also make sure Proxmox is updated.

              1 Reply Last reply Reply Quote 0
              • S
                skogs
                last edited by

                Confirm the proxmox is using crappy Realtek RTL8111/8168/8411 style.
                So is the bare metal.

                D 1 Reply Last reply Reply Quote 0
                • D
                  dogtreatfairy @skogs
                  last edited by

                  So I was in the same boat, and couldn't figure this out. I was about ready to buy another dedicated computer just to be my pfSense box. I'm running pfSense 2.5 inside Proxmox 6.3-3. I know it sounds stupid, but try upping your pfSense VM RAM to 6GB (6144MB). This tripled my connection speed and I'm back to full speed on my WAN. Inside pfSense, it was showing my RAM usage as ~20% at 4GB, so it makes no sense to me why this fixed my problem.

                  I also disabled the hardware checksum, tcp, and large receive offload options under System > Advanced > Networking.

                  My VM Setup

                  • RAM: 6.00 GB
                  • CPU: Host (i5-3470), 1 Socket, 4 Cores, +pcid,+aes, +ssbd
                  • BIOS: Default (SeaBIOS)
                  • Machine: Default (i440fx)
                  • SCSI Cont: VirtIO SCSI
                  • HDD: SCSI, 32GB volume.
                  • PCI Device: PCI pass through for my HP N364T 4-port gigabit NIC off ebay

                  Look through this to enable IOMMU for the PCI-E Pass Through.
                  Proxmox PCI-E Passthrough

                  S 1 Reply Last reply Reply Quote 0
                  • S
                    skogs @dogtreatfairy
                    last edited by

                    @dogtreatfairy I agree that makes no sense. :)
                    I definitely think it was disabling hardware offload options rather than the RAM.

                    DerelictD 1 Reply Last reply Reply Quote 0
                    • DerelictD
                      Derelict LAYER 8 Netgate @skogs
                      last edited by

                      @skogs Yes those must be disabled in proxmox/KVM. Quite normal to have to disable those in any virtual environment..

                      Chattanooga, Tennessee, USA
                      A comprehensive network diagram is worth 10,000 words and 15 conference calls.
                      DO NOT set a source address/port in a port forward or firewall rule unless you KNOW you need it!
                      Do Not Chat For Help! NO_WAN_EGRESS(TM)

                      T 1 Reply Last reply Reply Quote 0
                      • T
                        tibere86 @Derelict
                        last edited by

                        @derelict Is it recommended to disable hardware checksum if the WAN is on a passthrough-ed NIC and the LAN is on a virtual NIC?

                        DerelictD 1 Reply Last reply Reply Quote 0
                        • DerelictD
                          Derelict LAYER 8 Netgate @tibere86
                          last edited by Derelict

                          @tibere86 I have never tried it on a passthrough NIC. But any time there is traffic THROUGH a VM it is wise to disable those offloads. You can certain'y see what works for you in your environment but you'll at least know what to disable if throughput tanks.

                          Chattanooga, Tennessee, USA
                          A comprehensive network diagram is worth 10,000 words and 15 conference calls.
                          DO NOT set a source address/port in a port forward or firewall rule unless you KNOW you need it!
                          Do Not Chat For Help! NO_WAN_EGRESS(TM)

                          A 1 Reply Last reply Reply Quote 0
                          • A
                            abhishekakt @Derelict
                            last edited by

                            So I did few testing and found it was pfsense package causing slow down. Bandwidthd, Darkstat, pftopng.. you name it. The worst part is, even if you remove these packages, you will never get original speed back. Here's how I did the testing:

                            Fresh install pfsense. Install and run iperf3 between Proxmox host (or any vm running under same host) and pfsense. I get full gigabit speed. Which is also strange the VMBR switch is sharing same kernel so between vm's I should get 10 Gigbit (theoretical) speed which I get with other linux vm's but not pfsense. Anyway, continuing my test. '

                            I install Bandwidthd and bam, speed drops to 200 Mbps in iperf3 test between pfsense and proxmox host. even if you remove the package from pfsense, you will never get same gigabit speed again ever. So just to prove this theory, I reinstalled pfsense over 100 times each time with different combinations. At last, I figured I could use snapshot to avoid reinstallling and reconfiguring pfsense.

                            So bottomline, these bandwidth morning package just kills pfsense throughput.

                            I have 2 Internet link 300 Mbps each.

                            1 Reply Last reply Reply Quote 0
                            • D
                              dogtreatfairy
                              last edited by

                              So, I would like to admit I'm an idiot. My problem was I couldn't get full speed on Wi-Fi. Well, turns out my computer was defaulting to connecting to my 2.4GHz network. When I upped the ram and restarted the pfSense VM, for some reason my computer reconnected to the 5GHz network.

                              On the 2.4GHz network my WAN tops out at 80-90 Mbps
                              On the 5 GHz network, the WAN tops out at my rated max of 230 Mbps

                              So, I didn't actually have a problem... just thought I did and managed to convince myself of it.

                              1 Reply Last reply Reply Quote 1
                              • O
                                osidosi
                                last edited by

                                Thank you "digdug3" 👍
                                My production setup is PfSense virtualised on Proxmox, with fileover and balance gateway configuration. Max connected devices on our facility vary between 100-210.
                                HP DL380 G7
                                HP 4 port NIC 1Gbit (3 x WAN, planning to add 4th as Vodafone Radiolink backup WAN)
                                HP 2 Port 10Gbit SFP+, (LAN and Storage connections)
                                PVE 8,
                                The bandwidth of the WAN ports was sometimes slowing down to 7 mbit/s. The other bad situation was randomly packages losings and high latencies. I traing to do everiting described on forums but this article really help me. The magic word was "VirtIO"
                                What make difference for me is changing virtualizet network drivers from "Intel E1000" to "VırtlO" on PfSense virtual machine.

                                Befeore that
                                1- I was thicked these setting under System/Advanced/Networking
                                *Hardware Checksum Offloading
                                *Hardware TCP Segmentation Offloading
                                *Hardware Large Receive Offloading
                                *hn ALTQ support
                                2- Increase the Memory to 12Gb
                                3- Checked every cable and FO connections
                                4- Tested every DSL speed separately connected with pc directly to modems (speeds were as they should have been )
                                5- Get some additional support from ISP
                                6- Disable every firewall rule exept basic ones, disable, pfblockerng, disable pftopng, disable openvpn, destroy gateway group and enable only one WAN port a time.
                                7- Disable auto negotation and strict to different speeds
                                with no success.

                                Turning off the settings in step 1- after switching to VirtIO didn't make a big difference.
                                Hope this help some one.

                                1 Reply Last reply Reply Quote 0
                                • O
                                  osidosi @digdug3
                                  last edited by

                                  @digdug3 Thank you, your suggestion still works.
                                  link text

                                  1 Reply Last reply Reply Quote 0
                                  • First post
                                    Last post
                                  Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.