Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    Throughput issues on pfSense 2.4.5 with APU2E4

    Scheduled Pinned Locked Moved Problems Installing or Upgrading pfSense Software
    3 Posts 2 Posters 1.2k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • S
      sToRmInG
      last edited by sToRmInG

      Hi

      Since upgrading to pfSense 2.4.5 I have a massive issue with throughput on my APU2E4. These were and are not present on 2.4.4p3 and 2.5.0.a.20200330.2017.

      I already tried the following:

      • Clean install pfSense 2.4.5 RELEASE: issues present
      • Clean install pfSense 2.4.4p3 RELEASE: no issues
      • Upgrade clean installed pfSense 2.4.4p3 RELEASE to 2.4.5 RELEASE: issues present
      • Upgrade clean installed pfSense 2.4.4p3 RELEASE to 2.4.5 RELEASE and then to 2.5.0 SNAPSHOT: issues present on 2.4.5 RELEASE are no longer present (back to normal)

      Please find below the tests I did with iperf3 (the APU is currently being set up and therefore the WAN IP is internal as well).
      pfSense 2.4.4:

      -----------------------------------------------------------
Server listening on 5201
-----------------------------------------------------------
Accepted connection from 10.77.77.183, port 39487
[  5] local 10.77.77.165 port 5201 connected to 10.77.77.183 port 23449
[  7] local 10.77.77.165 port 5201 connected to 10.77.77.183 port 6267
[  9] local 10.77.77.165 port 5201 connected to 10.77.77.183 port 49437
[ 11] local 10.77.77.165 port 5201 connected to 10.77.77.183 port 28998
[ ID] Interval           Transfer     Bandwidth
[  5]   0.00-1.00   sec  31.3 MBytes   263 Mbits/sec
[  7]   0.00-1.00   sec  14.0 MBytes   118 Mbits/sec
[  9]   0.00-1.00   sec  32.5 MBytes   272 Mbits/sec
[ 11]   0.00-1.00   sec  31.8 MBytes   267 Mbits/sec
[SUM]   0.00-1.00   sec   110 MBytes   919 Mbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[  5]   1.00-2.00   sec  28.5 MBytes   239 Mbits/sec
[  7]   1.00-2.00   sec  26.7 MBytes   224 Mbits/sec
[  9]   1.00-2.00   sec  28.4 MBytes   238 Mbits/sec
[ 11]   1.00-2.00   sec  28.2 MBytes   236 Mbits/sec
[SUM]   1.00-2.00   sec   112 MBytes   938 Mbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[  5]   2.00-3.00   sec  30.1 MBytes   253 Mbits/sec
[  7]   2.00-3.00   sec  17.3 MBytes   145 Mbits/sec
[  9]   2.00-3.00   sec  32.6 MBytes   274 Mbits/sec
[ 11]   2.00-3.00   sec  30.5 MBytes   255 Mbits/sec
[SUM]   2.00-3.00   sec   111 MBytes   927 Mbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[  5]   3.00-4.00   sec  28.3 MBytes   238 Mbits/sec
[  7]   3.00-4.00   sec  14.4 MBytes   121 Mbits/sec
[  9]   3.00-4.00   sec  36.5 MBytes   306 Mbits/sec
[ 11]   3.00-4.00   sec  32.5 MBytes   273 Mbits/sec
[SUM]   3.00-4.00   sec   112 MBytes   938 Mbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[  5]   4.00-5.00   sec  27.0 MBytes   226 Mbits/sec
[  7]   4.00-5.00   sec  24.8 MBytes   208 Mbits/sec
[  9]   4.00-5.00   sec  30.7 MBytes   258 Mbits/sec
[ 11]   4.00-5.00   sec  28.6 MBytes   240 Mbits/sec
[SUM]   4.00-5.00   sec   111 MBytes   932 Mbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[  5]   5.00-6.00   sec  31.2 MBytes   262 Mbits/sec
[  7]   5.00-6.00   sec  22.6 MBytes   190 Mbits/sec
[  9]   5.00-6.00   sec  28.4 MBytes   238 Mbits/sec
[ 11]   5.00-6.00   sec  29.2 MBytes   245 Mbits/sec
[SUM]   5.00-6.00   sec   111 MBytes   935 Mbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[  5]   6.00-7.00   sec  27.8 MBytes   233 Mbits/sec
[  7]   6.00-7.00   sec  28.5 MBytes   239 Mbits/sec
[  9]   6.00-7.00   sec  27.9 MBytes   234 Mbits/sec
[ 11]   6.00-7.00   sec  28.2 MBytes   236 Mbits/sec
[SUM]   6.00-7.00   sec   112 MBytes   942 Mbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[  5]   7.00-8.00   sec  27.9 MBytes   234 Mbits/sec
[  7]   7.00-8.00   sec  22.4 MBytes   188 Mbits/sec
[  9]   7.00-8.00   sec  29.4 MBytes   247 Mbits/sec
[ 11]   7.00-8.00   sec  29.8 MBytes   250 Mbits/sec
[SUM]   7.00-8.00   sec   110 MBytes   919 Mbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[  5]   8.00-9.00   sec  29.8 MBytes   250 Mbits/sec
[  7]   8.00-9.00   sec  12.8 MBytes   107 Mbits/sec
[  9]   8.00-9.00   sec  36.5 MBytes   306 Mbits/sec
[ 11]   8.00-9.00   sec  32.8 MBytes   275 Mbits/sec
[SUM]   8.00-9.00   sec   112 MBytes   938 Mbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[  5]   9.00-10.00  sec  28.7 MBytes   241 Mbits/sec
[  7]   9.00-10.00  sec  25.3 MBytes   212 Mbits/sec
[  9]   9.00-10.00  sec  29.2 MBytes   245 Mbits/sec
[ 11]   9.00-10.00  sec  29.1 MBytes   244 Mbits/sec
[SUM]   9.00-10.00  sec   112 MBytes   943 Mbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth
[  5]   0.00-10.00  sec   291 MBytes   244 Mbits/sec                  receiver
[  7]   0.00-10.00  sec   209 MBytes   175 Mbits/sec                  receiver
[  9]   0.00-10.00  sec   312 MBytes   262 Mbits/sec                  receiver
[ 11]   0.00-10.00  sec   301 MBytes   252 Mbits/sec                  receiver
[SUM]   0.00-10.00  sec  1.09 GBytes   933 Mbits/sec                  receiver

      pfSense 2.4.5

      -----------------------------------------------------------
Server listening on 5201
-----------------------------------------------------------
Accepted connection from 10.77.77.183, port 59825
[  5] local 10.77.77.165 port 5201 connected to 10.77.77.183 port 16642
[  7] local 10.77.77.165 port 5201 connected to 10.77.77.183 port 46320
[  9] local 10.77.77.165 port 5201 connected to 10.77.77.183 port 49564
[ 11] local 10.77.77.165 port 5201 connected to 10.77.77.183 port 12839
[ ID] Interval           Transfer     Bandwidth
[  5]   0.00-1.00   sec   292 KBytes  2.39 Mbits/sec
[  7]   0.00-1.00   sec  43.5 MBytes   365 Mbits/sec
[  9]   0.00-1.00   sec   198 KBytes  1.62 Mbits/sec
[ 11]   0.00-1.00   sec  44.2 MBytes   371 Mbits/sec
[SUM]   0.00-1.00   sec  88.2 MBytes   740 Mbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[  5]   1.00-2.00   sec  0.00 Bytes  0.00 bits/sec
[  7]   1.00-2.00   sec  45.1 MBytes   379 Mbits/sec
[  9]   1.00-2.00   sec  0.00 Bytes  0.00 bits/sec
[ 11]   1.00-2.00   sec  46.0 MBytes   386 Mbits/sec
[SUM]   1.00-2.00   sec  91.2 MBytes   765 Mbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[  5]   2.00-3.00   sec  0.00 Bytes  0.00 bits/sec
[  7]   2.00-3.00   sec  45.3 MBytes   380 Mbits/sec
[  9]   2.00-3.00   sec  37.1 KBytes   304 Kbits/sec
[ 11]   2.00-3.00   sec  45.9 MBytes   385 Mbits/sec
[SUM]   2.00-3.00   sec  91.2 MBytes   765 Mbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[  5]   3.00-4.00   sec  0.00 Bytes  0.00 bits/sec
[  7]   3.00-4.00   sec  45.4 MBytes   380 Mbits/sec
[  9]   3.00-4.00   sec  38.5 KBytes   315 Kbits/sec
[ 11]   3.00-4.00   sec  46.1 MBytes   386 Mbits/sec
[SUM]   3.00-4.00   sec  91.5 MBytes   767 Mbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[  5]   4.00-5.00   sec  39.9 KBytes   327 Kbits/sec
[  7]   4.00-5.00   sec  44.3 MBytes   372 Mbits/sec
[  9]   4.00-5.00   sec   567 KBytes  4.65 Mbits/sec
[ 11]   4.00-5.00   sec  45.6 MBytes   383 Mbits/sec
[SUM]   4.00-5.00   sec  90.6 MBytes   760 Mbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[  5]   5.00-6.00   sec  0.00 Bytes  0.00 bits/sec
[  7]   5.00-6.00   sec  45.2 MBytes   379 Mbits/sec
[  9]   5.00-6.00   sec  1.43 KBytes  11.7 Kbits/sec
[ 11]   5.00-6.00   sec  45.8 MBytes   384 Mbits/sec
[SUM]   5.00-6.00   sec  91.0 MBytes   763 Mbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[  5]   6.00-7.00   sec  0.00 Bytes  0.00 bits/sec
[  7]   6.00-7.00   sec  45.3 MBytes   380 Mbits/sec
[  9]   6.00-7.00   sec  0.00 Bytes  0.00 bits/sec
[ 11]   6.00-7.00   sec  45.9 MBytes   385 Mbits/sec
[SUM]   6.00-7.00   sec  91.1 MBytes   764 Mbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[  5]   7.00-8.00   sec  0.00 Bytes  0.00 bits/sec
[  7]   7.00-8.00   sec  45.1 MBytes   378 Mbits/sec
[  9]   7.00-8.00   sec  0.00 Bytes  0.00 bits/sec
[ 11]   7.00-8.00   sec  45.9 MBytes   385 Mbits/sec
[SUM]   7.00-8.00   sec  91.0 MBytes   763 Mbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[  5]   8.00-9.00   sec  0.00 Bytes  0.00 bits/sec
[  7]   8.00-9.00   sec  45.0 MBytes   378 Mbits/sec
[  9]   8.00-9.00   sec  0.00 Bytes  0.00 bits/sec
[ 11]   8.00-9.00   sec  45.7 MBytes   383 Mbits/sec
[SUM]   8.00-9.00   sec  90.8 MBytes   761 Mbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[  5]   9.00-10.00  sec  0.00 Bytes  0.00 bits/sec
[  7]   9.00-10.00  sec  45.2 MBytes   379 Mbits/sec
[  9]   9.00-10.00  sec  0.00 Bytes  0.00 bits/sec
[ 11]   9.00-10.00  sec  45.6 MBytes   383 Mbits/sec
[SUM]   9.00-10.00  sec  90.9 MBytes   762 Mbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth
[  5]   0.00-10.00  sec   332 KBytes   272 Kbits/sec                  receiver
[  7]   0.00-10.00  sec   449 MBytes   377 Mbits/sec                  receiver
[  9]   0.00-10.00  sec   843 KBytes   690 Kbits/sec                  receiver
[ 11]   0.00-10.00  sec   457 MBytes   383 Mbits/sec                  receiver
[SUM]   0.00-10.00  sec   907 MBytes   761 Mbits/sec                  receiver

      pfSense 2.5.0.a.20200330.2017:

      -----------------------------------------------------------
Server listening on 5201
-----------------------------------------------------------
Accepted connection from 10.77.77.183, port 36901
[  5] local 10.77.77.165 port 5201 connected to 10.77.77.183 port 64405
[  7] local 10.77.77.165 port 5201 connected to 10.77.77.183 port 9249
[  9] local 10.77.77.165 port 5201 connected to 10.77.77.183 port 24535
[ 11] local 10.77.77.165 port 5201 connected to 10.77.77.183 port 21955
[ ID] Interval           Transfer     Bandwidth
[  5]   0.00-1.00   sec  32.1 MBytes   269 Mbits/sec
[  7]   0.00-1.00   sec  27.1 MBytes   228 Mbits/sec
[  9]   0.00-1.00   sec  20.1 MBytes   169 Mbits/sec
[ 11]   0.00-1.00   sec  31.8 MBytes   267 Mbits/sec
[SUM]   0.00-1.00   sec   111 MBytes   933 Mbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[  5]   1.00-2.00   sec  32.3 MBytes   271 Mbits/sec
[  7]   1.00-2.00   sec  27.2 MBytes   228 Mbits/sec
[  9]   1.00-2.00   sec  19.9 MBytes   167 Mbits/sec
[ 11]   1.00-2.00   sec  31.4 MBytes   263 Mbits/sec
[SUM]   1.00-2.00   sec   111 MBytes   929 Mbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[  5]   2.00-3.00   sec  32.2 MBytes   270 Mbits/sec
[  7]   2.00-3.00   sec  27.2 MBytes   228 Mbits/sec
[  9]   2.00-3.00   sec  20.1 MBytes   168 Mbits/sec
[ 11]   2.00-3.00   sec  32.1 MBytes   269 Mbits/sec
[SUM]   2.00-3.00   sec   112 MBytes   936 Mbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[  5]   3.00-4.00   sec  32.1 MBytes   270 Mbits/sec
[  7]   3.00-4.00   sec  27.2 MBytes   228 Mbits/sec
[  9]   3.00-4.00   sec  19.8 MBytes   166 Mbits/sec
[ 11]   3.00-4.00   sec  31.8 MBytes   267 Mbits/sec
[SUM]   3.00-4.00   sec   111 MBytes   931 Mbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[  5]   4.00-5.00   sec  32.0 MBytes   269 Mbits/sec
[  7]   4.00-5.00   sec  27.1 MBytes   227 Mbits/sec
[  9]   4.00-5.00   sec  19.8 MBytes   166 Mbits/sec
[ 11]   4.00-5.00   sec  32.1 MBytes   269 Mbits/sec
[SUM]   4.00-5.00   sec   111 MBytes   931 Mbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[  5]   5.00-6.00   sec  32.2 MBytes   270 Mbits/sec
[  7]   5.00-6.00   sec  27.2 MBytes   228 Mbits/sec
[  9]   5.00-6.00   sec  19.9 MBytes   167 Mbits/sec
[ 11]   5.00-6.00   sec  32.1 MBytes   269 Mbits/sec
[SUM]   5.00-6.00   sec   111 MBytes   935 Mbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[  5]   6.00-7.00   sec  32.0 MBytes   268 Mbits/sec
[  7]   6.00-7.00   sec  27.2 MBytes   229 Mbits/sec
[  9]   6.00-7.00   sec  20.0 MBytes   168 Mbits/sec
[ 11]   6.00-7.00   sec  32.5 MBytes   272 Mbits/sec
[SUM]   6.00-7.00   sec   112 MBytes   937 Mbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[  5]   7.00-8.00   sec  32.0 MBytes   268 Mbits/sec
[  7]   7.00-8.00   sec  27.1 MBytes   227 Mbits/sec
[  9]   7.00-8.00   sec  20.1 MBytes   168 Mbits/sec
[ 11]   7.00-8.00   sec  32.3 MBytes   271 Mbits/sec
[SUM]   7.00-8.00   sec   111 MBytes   935 Mbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[  5]   8.00-9.00   sec  32.1 MBytes   269 Mbits/sec
[  7]   8.00-9.00   sec  27.1 MBytes   227 Mbits/sec
[  9]   8.00-9.00   sec  19.9 MBytes   167 Mbits/sec
[ 11]   8.00-9.00   sec  32.3 MBytes   271 Mbits/sec
[SUM]   8.00-9.00   sec   111 MBytes   934 Mbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[  5]   9.00-10.00  sec  32.0 MBytes   268 Mbits/sec
[  7]   9.00-10.00  sec  27.0 MBytes   226 Mbits/sec
[  9]   9.00-10.00  sec  19.9 MBytes   167 Mbits/sec
[ 11]   9.00-10.00  sec  31.9 MBytes   268 Mbits/sec
[SUM]   9.00-10.00  sec   111 MBytes   929 Mbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[  5]  10.00-10.01  sec   385 KBytes   282 Mbits/sec
[  7]  10.00-10.01  sec   311 KBytes   227 Mbits/sec
[  9]  10.00-10.01  sec   230 KBytes   168 Mbits/sec
[ 11]  10.00-10.01  sec   361 KBytes   264 Mbits/sec
[SUM]  10.00-10.01  sec  1.26 MBytes   941 Mbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth
[  5]   0.00-10.01  sec   321 MBytes   269 Mbits/sec                  receiver
[  7]   0.00-10.01  sec   272 MBytes   228 Mbits/sec                  receiver
[  9]   0.00-10.01  sec   200 MBytes   167 Mbits/sec                  receiver
[ 11]   0.00-10.01  sec   321 MBytes   269 Mbits/sec                  receiver
[SUM]   0.00-10.01  sec  1.09 GBytes   933 Mbits/sec                  receiver

      Note the 0.00 bits/sec on 2.4.5.

      Do you have any idea why this is happening?

      The following custom loader.conf.local was used and NIC hardware offloading was activated as well on all tested versions to enable gigabit throughput on PC Engines' APU2s:

      # agree with Intel license terms
legal.intel_igb.license_ack="1"
legal.intel_iwi.license_ack="1"
legal.intel_ipw.license_ack="1"
# this is the magic. If you don't set this, queues won't be utilized properly
# allow multiple processes for receive/transmit processing
hw.igb.rx_process_limit="-1"
hw.igb.tx_process_limit="-1"

      I used firmware 4.11.0.4 on the APU for all tests so it is not related to this reported issue: https://github.com/pcengines/coreboot/issues/387

      1 Reply Last reply Reply Quote 0
      • ?
        A Former User
        last edited by

        You're not alone, please see this thread.

        S 1 Reply Last reply Reply Quote 0
        • S
          sToRmInG @A Former User
          last edited by

          @muppet You're right. In my case it seems to be CPU related. The utilization is much higher than before and therefore the individual cores cannot handle the load for the queue they're assigned to.

          1 Reply Last reply Reply Quote 0
          • First post
            Last post
          Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.