Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    Slow upload speeds on HP Z2 G9 PFSense Box

    Scheduled Pinned Locked Moved General pfSense Questions
    74 Posts 3 Posters 4.9k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • stephenw10S
      stephenw10 Netgate Administrator
      last edited by

      Hmm, interesting. So in both cases that's the WAN? And it's actually linked at 5G?

      Hmm, I would usually expect the X520 to behave better....

      If you check the Status > Interfaces page do you see errors or collisions?

      If so you can dig further into the mac stats in the sysctl output for each NIC.

      BearB 1 Reply Last reply Reply Quote 0
      • BearB
        Bear @stephenw10
        last edited by

        @stephenw10 said in Slow upload speeds on HP Z2 G9 PFSense Box:

        Hmm, interesting. So in both cases that's the WAN? And it's actually linked at 5G?

        Yes, the WAN is linked at 5Gbit, whether it's plugged into the 6100 or the X520 in the HP, which is in an x8 slot.

        Hmm, I would usually expect the X520 to behave better....

        Me too. I had this result with the X710-DA2, so I figured I'd go with the tried and true X520. No dice.

        If you check the Status > Interfaces page do you see errors or collisions?

        There are no errors or collisions on the WAN side. Sadly, I can't keep the new firewall on long enough to see if there are any other things going on, longer-term. I've got a lot of traffic on this business line that I'm hosting and cutting me down to 100-400Mbits has some dire consequences.

        If so you can dig further into the mac stats in the sysctl output for each NIC.

        What specifically would you like me to look at next time I connect it up?

        1 Reply Last reply Reply Quote 0
        • stephenw10S
          stephenw10 Netgate Administrator
          last edited by

          Are there errors on the LAN side? It could be a problem there.

          BearB 1 Reply Last reply Reply Quote 0
          • BearB
            Bear @stephenw10
            last edited by Bear

            @stephenw10 Not seeing any errors on the LAN side at all. Interrupts aren't too bad either.

            I'm running a filtered bridge. Same config as on the 6100.

            Getting the same results as when I used the 710 as well, completely different NIC.

            1 Reply Last reply Reply Quote 0
            • stephenw10S
              stephenw10 Netgate Administrator
              last edited by

              How are you testing? Do you see any packet loss?

              BearB 1 Reply Last reply Reply Quote 0
              • BearB
                Bear @stephenw10
                last edited by

                @stephenw10 said in Slow upload speeds on HP Z2 G9 PFSense Box:

                How are you testing? Do you see any packet loss?

                Just using speedtest and fast.com from 10GBE Macs and Linux boxes with Intel NICs - Seeing zero packet loss and 8-10ms ping times.

                1 Reply Last reply Reply Quote 0
                • stephenw10S
                  stephenw10 Netgate Administrator
                  last edited by

                  Hmm, you might try running a test from pfSense directly. It won't be accurate but if you see the same big difference between up and downloads that at least narrows it down to the WAN side.

                  Can you see anything about the link state from the connected modem.

                  BearB 1 Reply Last reply Reply Quote 0
                  • BearB
                    Bear @stephenw10
                    last edited by

                    @stephenw10

                    Both WAN and LAN are on the same X520 NIC. Is there a native speedtest on pfsense?

                    1 Reply Last reply Reply Quote 0
                    • stephenw10S
                      stephenw10 Netgate Administrator
                      last edited by

                      Yes there is a CLI client. In fact there are two:

                      [25.03-RC][admin@5100.stevew.lan]/root: pkg search speedtest
                      py311-speedtest-cli-2.1.3      Command line interface for testing internet bandwidth
                      speedtest-go-1.7.9             CLI and Go API to test internet speed using speedtest.net
                      

                      You can install and run those directly:

                      [25.03-RC][admin@5100.stevew.lan]/root: pkg install py311-speedtest-cli
                      Updating pfSense-core repository catalogue...
                      pfSense-core repository is up to date.
                      Updating pfSense repository catalogue...
                      pfSense repository is up to date.
                      All repositories are up to date.
                      The following 1 package(s) will be affected (of 0 checked):
                      
                      New packages to be INSTALLED:
                      	py311-speedtest-cli: 2.1.3 [pfSense]
                      
                      Number of packages to be installed: 1
                      
                      52 KiB to be downloaded.
                      
                      Proceed with this action? [y/N]: y
                      [1/1] Fetching py311-speedtest-cli-2.1.3.pkg: 100%   52 KiB  53.1kB/s    00:01    
                      Checking integrity... done (0 conflicting)
                      [1/1] Installing py311-speedtest-cli-2.1.3...
                      [1/1] Extracting py311-speedtest-cli-2.1.3: 100%
                      [25.03-RC][admin@5100.stevew.lan]/root: rehash
                      

                      As I said it will not give an accurate value at bandwidths that high but you can use it to compare relative rates.

                      BearB 1 Reply Last reply Reply Quote 0
                      • BearB
                        Bear
                        last edited by

                        I can try that later today. I’ll post results soon.

                        1 Reply Last reply Reply Quote 1
                        • BearB
                          Bear @stephenw10
                          last edited by

                          @stephenw10 said in Slow upload speeds on HP Z2 G9 PFSense Box:

                          Yes there is a CLI client. In fact there are two:

                          [25.03-RC][admin@5100.stevew.lan]/root: pkg search speedtest
                          py311-speedtest-cli-2.1.3      Command line interface for testing internet bandwidth
                          speedtest-go-1.7.9             CLI and Go API to test internet speed using speedtest.net
                          

                          You can install and run those directly:

                          [25.03-RC][admin@5100.stevew.lan]/root: pkg install py311-speedtest-cli
                          Updating pfSense-core repository catalogue...
                          pfSense-core repository is up to date.
                          Updating pfSense repository catalogue...
                          pfSense repository is up to date.
                          All repositories are up to date.
                          The following 1 package(s) will be affected (of 0 checked):
                          
                          New packages to be INSTALLED:
                          	py311-speedtest-cli: 2.1.3 [pfSense]
                          
                          Number of packages to be installed: 1
                          
                          52 KiB to be downloaded.
                          
                          Proceed with this action? [y/N]: y
                          [1/1] Fetching py311-speedtest-cli-2.1.3.pkg: 100%   52 KiB  53.1kB/s    00:01    
                          Checking integrity... done (0 conflicting)
                          [1/1] Installing py311-speedtest-cli-2.1.3...
                          [1/1] Extracting py311-speedtest-cli-2.1.3: 100%
                          [25.03-RC][admin@5100.stevew.lan]/root: rehash
                          

                          As I said it will not give an accurate value at bandwidths that high but you can use it to compare relative rates.

                          Sadly, it's completely unreliable running on the FW. Getting 450mbit up and 45 down on the 6100, since it's picking a completely different server than even my other FreeBSD box.

                          1 Reply Last reply Reply Quote 0
                          • BearB
                            Bear
                            last edited by Bear

                            Tried one more test - Put a QNAP Multigig switch in between the PFSense box and the RG.

                            The 6100 maintained its speeds.

                            The HP Box also maintained its lower upload speeds.

                            Went into the Z2G9's BIOS (it's running the latest) and disabled PCI Express Power Management.

                            Upload speeds remained at 450-700Mbits, their typical range. Going back to the 6100 yielded the usual 3800-4500 up and down.

                            Almost at my wit's end. This is a very standard build using HP workstation components and Intel NICs. No idea why I'm seeing this.

                            Any ideas?

                            -Bear

                            1 Reply Last reply Reply Quote 0
                            • stephenw10S
                              stephenw10 Netgate Administrator
                              last edited by

                              You can specify which server it uses:

                                --list                Display a list of speedtest.net servers sorted by distance
                                --server SERVER       Specify a server ID to test against. Can be supplied multiple times
                              
                              BearB 1 Reply Last reply Reply Quote 0
                              • BearB
                                Bear @stephenw10
                                last edited by

                                @stephenw10

                                ✓ Test Server: [68863] 0.00km San Antonio, TX by AT&T
                                ✓ Latency: 9.472954ms Jitter: 1.390244ms Min: 8.694556ms Max: 13.568589ms
                                ✓ Packet Loss Analyzer: Running in background (<= 30 Secs)
                                ✓ Download: 1350.20 Mbps (Used: 1708.20MB) (Latency: 15ms Jitter: 10ms Min: 8ms Max: 44ms)
                                ✓ Upload: 763.02 Mbps (Used: 930.43MB) (Latency: 11ms Jitter: 2ms Min: 8ms Max: 13ms)
                                ✓ Packet Loss: N/A
                                

                                I'm getting really slow speeds from the 6100 vs my Mac or Windows system with 10GBE to the point where this up speed is aligned with the ~750mbit I get on those systems with the HP/X520.

                                In other words, the slow performance test locally on the 6100 would mask the issue as I get 3500+mbits up when running speedtest from a local system vs 760 from the 6100 itself.

                                1 Reply Last reply Reply Quote 0
                                • stephenw10S
                                  stephenw10 Netgate Administrator
                                  last edited by

                                  Hmm, OK. Probably not useful data there then.

                                  Do you see the same in an iperf test?

                                  BearB 1 Reply Last reply Reply Quote 0
                                  • BearB
                                    Bear @stephenw10
                                    last edited by

                                    @stephenw10 Haven't run iPerf yet. I'll try that later, but I'm not sure what data we're after here; the performance is identical for low upload speeds using 2 different Intel NICs in the same x16 slot @ x8, and both WAN and LAN are on the same NIC.

                                    Flow control is off, PCI power management is off, and with an intermediary multigig switch, results remain identical.

                                    1 Reply Last reply Reply Quote 0
                                    • stephenw10S
                                      stephenw10 Netgate Administrator
                                      last edited by

                                      If you're able to see the same issue against a remote iperf server you can then test against a local iperf server to check if it's something specific to the WAN connection.

                                      Also you can run iperf directly from pfSense with more useful results.

                                      BearB 1 Reply Last reply Reply Quote 0
                                      • BearB
                                        Bear @stephenw10
                                        last edited by Bear

                                        @stephenw10 said in Slow upload speeds on HP Z2 G9 PFSense Box:

                                        If you're able to see the same issue against a remote iperf server you can then test against a local iperf server to check if it's something specific to the WAN connection.

                                        Also you can run iperf directly from pfSense with more useful results.

                                        IPerf isn't jibing with speedtest. Here's my recent speedtest result: https://www.speedtest.net/my-result/d/739237147

                                        Here's a few iPERF 3 runs to supposedly local addresses:

                                        Connecting to host 37.19.206.20, port 5201
                                        [  5] local 75.31.24.1 port 38531 connected to 37.19.206.20 port 5201
                                        [ ID] Interval           Transfer     Bitrate         Retr  Cwnd
                                        [  5]   0.00-1.00   sec  28.2 MBytes   237 Mbits/sec    0   6.41 MBytes
                                        [  5]   1.00-2.00   sec  46.9 MBytes   393 Mbits/sec    0   6.41 MBytes
                                        [  5]   2.00-3.01   sec  47.6 MBytes   397 Mbits/sec    0   6.41 MBytes
                                        [  5]   3.01-4.00   sec  46.9 MBytes   396 Mbits/sec    0   6.41 MBytes
                                        [  5]   4.00-5.00   sec  47.2 MBytes   396 Mbits/sec    0   6.41 MBytes
                                        [  5]   5.00-6.00   sec  47.2 MBytes   396 Mbits/sec    0   6.41 MBytes
                                        [  5]   6.00-7.03   sec  48.2 MBytes   395 Mbits/sec    0   6.41 MBytes
                                        [  5]   7.03-8.00   sec  46.0 MBytes   396 Mbits/sec    0   6.41 MBytes
                                        [  5]   8.00-9.00   sec  47.5 MBytes   398 Mbits/sec    0   6.41 MBytes
                                        [  5]   9.00-10.00  sec  47.4 MBytes   397 Mbits/sec    0   6.41 MBytes
                                        - - - - - - - - - - - - - - - - - - - - - - - - -
                                        [ ID] Interval           Transfer     Bitrate         Retr
                                        [  5]   0.00-10.00  sec   453 MBytes   380 Mbits/sec    0             sender
                                        [  5]   0.00-10.05  sec   453 MBytes   379 Mbits/sec                  receiver
                                        
                                        
                                        Connecting to host dal.speedtest.clouvider.net, port 5200
                                        [  5] local 75.31.24.1 port 48312 connected to 2.56.188.136 port 5200
                                        [ ID] Interval           Transfer     Bitrate         Retr  Cwnd
                                        [  5]   0.00-1.01   sec  5.62 MBytes  46.9 Mbits/sec    8   92.8 KBytes
                                        [  5]   1.01-2.00   sec  11.2 MBytes  94.8 Mbits/sec    0    162 KBytes
                                        [  5]   2.00-3.00   sec  17.8 MBytes   149 Mbits/sec    0    231 KBytes
                                        [  5]   3.00-4.02   sec  20.9 MBytes   172 Mbits/sec    3    220 KBytes
                                        [  5]   4.02-5.00   sec  19.1 MBytes   163 Mbits/sec    4    179 KBytes
                                        [  5]   5.00-6.01   sec  19.2 MBytes   160 Mbits/sec    0    248 KBytes
                                        [  5]   6.01-7.00   sec  24.8 MBytes   210 Mbits/sec    0    316 KBytes
                                        [  5]   7.00-8.01   sec  22.2 MBytes   186 Mbits/sec   81    114 KBytes
                                        [  5]   8.01-9.01   sec  10.9 MBytes  91.1 Mbits/sec   46   34.7 KBytes
                                        [  5]   9.01-10.00  sec  6.12 MBytes  51.5 Mbits/sec    0    104 KBytes
                                        - - - - - - - - - - - - - - - - - - - - - - - - -
                                        [ ID] Interval           Transfer     Bitrate         Retr
                                        [  5]   0.00-10.00  sec   158 MBytes   132 Mbits/sec  142             sender
                                        [  5]   0.00-10.01  sec   157 MBytes   132 Mbits/sec                  receiver
                                        

                                        I can consistently get similar speeds to other providers as well. iperf3 isn't getting me close to a gigabit. Real world transfers are in the 2.5-3.5Gbit range usually.

                                        BearB 1 Reply Last reply Reply Quote 0
                                        • BearB
                                          Bear @Bear
                                          last edited by

                                          @stephenw10

                                          Now, if I do the iperf3 from my Mac Studio with 10GBE going /through/ the 6100 Max...

                                          Connecting to host speedtest.dal13.us.leaseweb.net, port 5201
                                          [  7] local 75.31.24.8 port 61252 connected to 209.58.159.68 port 5201
                                          [ ID] Interval           Transfer     Bitrate
                                          [  7]   0.00-1.01   sec   186 MBytes  1.55 Gbits/sec                  
                                          [  7]   1.01-2.00   sec   247 MBytes  2.08 Gbits/sec                  
                                          [  7]   2.00-3.01   sec   249 MBytes  2.08 Gbits/sec                  
                                          [  7]   3.01-4.01   sec   247 MBytes  2.07 Gbits/sec                  
                                          [  7]   4.01-5.01   sec   248 MBytes  2.08 Gbits/sec                  
                                          [  7]   5.01-6.01   sec   247 MBytes  2.07 Gbits/sec                  
                                          [  7]   6.01-7.01   sec   248 MBytes  2.08 Gbits/sec                  
                                          [  7]   7.01-8.01   sec   220 MBytes  1.84 Gbits/sec                  
                                          [  7]   8.01-9.00   sec   207 MBytes  1.74 Gbits/sec                  
                                          [  7]   9.00-10.01  sec   216 MBytes  1.81 Gbits/sec                  
                                          - - - - - - - - - - - - - - - - - - - - - - - - -
                                          [ ID] Interval           Transfer     Bitrate
                                          [  7]   0.00-10.01  sec  2.26 GBytes  1.94 Gbits/sec    sender
                                          [  7]   0.00-10.02  sec  2.26 GBytes  1.94 Gbits/sec    receiver
                                          

                                          And do the same iperf3 test from the 6100 Max itself, just a moment later to the same iperf3 server...

                                          Connecting to host speedtest.dal13.us.leaseweb.net, port 5201
                                          [  5] local 75.31.24.1 port 8528 connected to 209.58.159.68 port 5201
                                          [ ID] Interval           Transfer     Bitrate         Retr  Cwnd
                                          [  5]   0.00-1.00   sec  11.4 MBytes  95.3 Mbits/sec   60    145 KBytes
                                          [  5]   1.00-2.00   sec  6.88 MBytes  57.7 Mbits/sec   37   79.5 KBytes
                                          [  5]   2.00-3.00   sec  11.9 MBytes  99.6 Mbits/sec    0    156 KBytes
                                          [  5]   3.00-4.01   sec  10.5 MBytes  87.6 Mbits/sec   68   53.7 KBytes
                                          [  5]   4.01-5.02   sec  7.50 MBytes  62.3 Mbits/sec   19    108 KBytes
                                          [  5]   5.02-6.02   sec  10.6 MBytes  88.9 Mbits/sec   19   42.7 KBytes
                                          [  5]   6.02-7.00   sec  7.88 MBytes  67.2 Mbits/sec    0    118 KBytes
                                          [  5]   7.00-8.00   sec  15.6 MBytes   131 Mbits/sec    0    194 KBytes
                                          [  5]   8.00-9.01   sec  23.8 MBytes   197 Mbits/sec    0    271 KBytes
                                          [  5]   9.01-10.02  sec  31.8 MBytes   263 Mbits/sec    0    348 KBytes
                                          - - - - - - - - - - - - - - - - - - - - - - - - -
                                          [ ID] Interval           Transfer     Bitrate         Retr
                                          [  5]   0.00-10.02  sec   138 MBytes   115 Mbits/sec  203    sender
                                          [  5]   0.00-10.03  sec   137 MBytes   115 Mbits/sec         receiver
                                          
                                          1 Reply Last reply Reply Quote 0
                                          • stephenw10S
                                            stephenw10 Netgate Administrator
                                            last edited by

                                            Hmm, interesting. Yes, as I said, running iperf from pfSense itself is always going to give a worse result than thorough it just because of the processing required to run the test itself. That's particularly true of the 6100 where each CPU core is not that powerful.

                                            Can you test using the i5?

                                            Was that with multiple parallel streams?

                                            The default direction for iperf3 is from the client to the server, so upload. Try using -R to get a download result.

                                            BearB 1 Reply Last reply Reply Quote 0
                                            • First post
                                              Last post
                                            Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.