Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    Limited throughput - 500Mbps VS 1Gbps ISP

    Scheduled Pinned Locked Moved Hardware
    15 Posts 4 Posters 1.7k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • D
      davecullen86 @stephenw10
      last edited by

      @stephenw10 said in Limited throughput - 500Mbps VS 1Gbps ISP:

      et.isr.numthreads

      Thank a lot!
      I made those changes, as well as some others as below from Googling:
      6f5fb1e0-4c9b-40ba-98e2-e3569a28cf28-image.png
      I see 100Mbps improvement, ~500 --to--> ~600, so definitely an improvement, but not quite there yet. Am I now at the limit of my HW and need to upgrade? Thx!!!

      stephenw10S P 2 Replies Last reply Reply Quote 0
      • stephenw10S
        stephenw10 Netgate Administrator @davecullen86
        last edited by stephenw10

        @davecullen86 said in Limited throughput - 500Mbps VS 1Gbps ISP:

        Am I now at the limit of my HW and need to upgrade?

        Probably. Check the output of top -HaSP at the CLI while you're testing. See it one CPU core is at 100%.
        But, yes, I would not expect to pass much more than that using PPPoE and that CPU.

        Steve

        D 1 Reply Last reply Reply Quote 0
        • P
          Patch @davecullen86
          last edited by

          @davecullen86 said in Limited throughput - 500Mbps VS 1Gbps ISP:

          need to upgrade?

          have a look at https://www.netgate.com/pfsense-plus-software/how-to-buy#appliances

          D 1 Reply Last reply Reply Quote 0
          • D
            davecullen86 @stephenw10
            last edited by

            @stephenw10
            Here is a video, it does not look like 100% to me. What do you think?
            https://file.io/LkYuqXCiaeHm
            (File too big to attach here)

            1 Reply Last reply Reply Quote 0
            • D
              davecullen86 @Patch
              last edited by

              @patch in an ideal world, totally I’d buy one of those appliances.
              One main point to PFsense generally for me, is the ability to use my own hardware to make the cost a lot lower.

              That’s to my detriment here of course as I am hitting limitations.

              My options I see here are:

              1. Instead of PPPoE, use the ISP router in the WAN as a L3 hop. This means double NAT as bridge mode isn’t supported.

              2. Buy new PFsense HW

              3. Use some other 3rd party HW on the wan side to support what I need. Maybe a Draytek 166, anyone had experience with this?

              Thank you guys, all responses are very much appreciated.

              Dave

              P 1 Reply Last reply Reply Quote 0
              • P
                Patch @davecullen86
                last edited by

                @davecullen86 Netgate have both specified and bench marked their hardware.

                • The best and safest solution is to buy a Netgate appliance.

                • If that is not possible you can compare hardware you have to a similar Netgate appliances to guess how it may perform. Note in practice that is not as easy as it first appears as Netgate have balanced the performance of each part of an appliance to achieve overall performance. As a result, equalling one aspect of the specification may not translate to overall performance.

                1 Reply Last reply Reply Quote 0
                • stephenw10S
                  stephenw10 Netgate Administrator
                  last edited by

                  Hmm, something odd about your ssh session there making it unclear. Looks like I missed a dash from my instructions! Should have read top -HaSP. I expect the output to look like:

                  last pid: 41269;  load averages:  0.04,  0.08,  0.08                              up 54+17:57:45  12:38:48
                  526 threads:   3 running, 510 sleeping, 13 waiting
                  CPU 0:  0.0% user,  0.0% nice,  0.2% system,  0.0% interrupt, 99.8% idle
                  CPU 1:  0.0% user,  0.0% nice,  0.2% system,  0.0% interrupt, 99.8% idle
                  Mem: 15M Active, 217M Inact, 422M Wired, 3143M Free
                  ARC: 198M Total, 30M MFU, 162M MRU, 692K Anon, 912K Header, 4358K Other
                       100M Compressed, 202M Uncompressed, 2.03:1 Ratio
                  Swap: 1024M Total, 1024M Free
                  
                    PID USERNAME    PRI NICE   SIZE    RES STATE    C   TIME    WCPU COMMAND
                     11 root        155 ki31     0B    32K CPU1     1 1298.4  99.65% [idle{idle: cpu1}]
                     11 root        155 ki31     0B    32K RUN      0 1298.6  99.46% [idle{idle: cpu0}]
                      0 root        -92    -     0B   544K -        1 187:07   0.27% [kernel{dummynet}]
                      0 root        -76    -     0B   544K -        1 127:16   0.16% [kernel{if_config_tqg_0}]
                  41269 root         20    0    14M  4772K CPU0     0   0:00   0.15% top -HaSP
                  21459 root         20    0    17M  7460K select   1  35:11   0.06% /usr/local/sbin/openvpn --config /var/e
                     12 root        -60    -     0B   208K WAIT     0  21:34   0.03% [intr{swi4: clock (0)}]
                     19 root        -16    -     0B    16K pftm     1  28:20   0.02% [pf purge]
                  89724 dhcpd        20    0    22M    13M select   0   0:40   0.02% /usr/local/sbin/dhcpd -user dhcpd -grou
                  87627 root         20    0    20M  9524K select   1   0:00   0.02% sshd: admin@pts/0 (sshd)
                      0 root        -76    -     0B   544K -        1  12:35   0.02% [kernel{if_io_tqg_1}]
                     20 root        -16    -     0B    16K -        1   9:37   0.01% [rand_harvestq]
                      0 root        -76    -     0B   544K -        0   3:28   0.01% [kernel{if_io_tqg_0}]
                  94357 root         20    0    11M  2788K nanslp   0   0:18   0.01% /usr/local/bin/dpinger -S -r 0 -i OPENV
                  94054 root         20    0    11M  2780K nanslp   1   0:15   0.01% /usr/local/bin/dpinger -S -r 0 -i 6100_
                  

                  Where you can see exactly what the loading is on each CPU core and what's causing it.

                  Steve

                  1 Reply Last reply Reply Quote 0
                  • ?
                    A Former User
                    last edited by A Former User

                    The problem here is that I can only achieve around
                    500Mbps downstream in a speedtest, both from an
                    Ethernet cable (1Gb) LAN side connected client and

                    Ok, but with a 2,0GHz CPU you might be able to archive
                    500 MBit/s according to the pfSense "Hardware recommendations" and you archive 500 - 600 MBit/s!

                    So all should be fine for you!

                    I checked the CPU load monitor on the dashboard
                    whilst performing the speedtests, and I see 69% highs
                    so it does not look like a saturated CPU to me.

                    This can be, but with PPPoE you will be "pinned" or "nailed" to only one CPU core! And 2,0GHz is for archiving
                    ~500 MBit/s the minimum on an actual CPU and yours is from 2013 and we are in 2022, please don´t forget this.

                    FYI, testing with the ISP router, I do get the advertised
                    speeds, so I know the circuit can achieve what I have
                    paid for here.

                    They (ISPs) are often soldering special ASICs on this routers and with them you will be archive full 1 GBit/s
                    pfSense comes as a Software firewall to you, and no one
                    can imagine on what hardware it will be installed by you!

                    I am using the same cabling that was WAN and LAN
                    side of the ISP router when it was used, tried alternate
                    cables also.

                    Cables makes the CPU not faster and also are not speeding up like an ASICs.

                    Can anyone advise if this is a HW limitation issue or
                    if somehow I can further troubleshoot this problem?

                    They are much tuning tips out, but this is not a single I
                    set it up and it works thing! You should be setting up or change much more points and then you combine them
                    this is often the "salt in the soup". I would considering to start at Friday over the weekend to get better in touch with it.

                    Ideally, I want full speeds of course and ideally, I want
                    to continue using PFSense!

                    We all want it, not only you!!

                    1 Reply Last reply Reply Quote 0
                    • D
                      davecullen86
                      last edited by

                      Hey guys, many thanks for your response. The more I look into this, the more I see so many others with the same issue. As you say the issue is implicit to the PPPoE single core factor and the clock speed of an individual core of my small appliance.

                      I have a solution! With another identical appliance, I have installed OpenWRT x86 and I am not getting close to 900Mbps throughput.

                      Now, THIS IS good enough for me :-). So I suggest is a good potential solution for others who are happy to offload the PPPoE function to another inline appliance.

                      Now I just need to work out if I can pass through the WAN IP somehow to my PFSense :-)
                      Thanks for your help again - I really appreciate the pointers that ultimately led me to get a working solution.
                      Cheers - Dave

                      1 Reply Last reply Reply Quote 0
                      • stephenw10S
                        stephenw10 Netgate Administrator
                        last edited by

                        Let us know if you find a way to do that. I've looked into it before and came to the conclusion that it might be possible but it involves some unconventional setup!

                        1 Reply Last reply Reply Quote 0
                        • ?
                          A Former User
                          last edited by A Former User

                          @davecullen86

                          Hey guys, many thanks for your response. The more
                          I look into this, the more I see so many others with the
                          same issue.

                          I have some, a couple of PC ENgines APU boards, and
                          I run MikroTik RouterOS, OpenWRT, pfSense on them,
                          all Linux comes more to 1 GBit/s with lower powered
                          hardware, it is a little bit more near to the hardware
                          due to better driver support and here and there not
                          so "hardware hungry", but a router and a firewall
                          that can be turned into a real UTM device is als not
                          the same! As I see it personally, you could try out as @stephenw10 was suggesting to tune your pfSense
                          a little here and there. With DanOS you might be getting
                          nearly two streams with full GBit/s on the same hardware
                          (PC Engines APUx), owed to DPDK capable LAN ports such
                          Intel i210 / i211.

                          As you say the issue is implicit to the PPPoE single core > factor and the clock speed of an individual core of my
                          small appliance.

                          Like me, but I was high up the cpu frequency to another
                          level and play now around with some other tuneable`s,
                          to get here and there more out of my hardware pointed
                          to the entire throughput. But I also know that my appliance is better cooled then other and will never goes
                          higher then 65 C° - 70 C°!!!! The CPU is normally capable
                          of 1400 MHz and runs even only at 600 MHz - 1000 MHz
                          and now it is running from 1000 MHz till 1400 MHz, but
                          if something goes wrong, I don´t complain and be angry!

                          I have a solution! With another identical appliance, I
                          have installed OpenWRT x86 and I am not getting close > to 900Mbps throughput.
                          And with DanOS you may be bidirectional getting fully
                          1 GBit/s out! But not a fully UTM in your Network!!!!!!!!

                          • Firewall
                          • Captive Portal with voucher system (voucher over sms)
                          • FreeRadius with certificates and encryption
                          • Snort or Suricata for IDS/IPS
                          • pfBlocker-NG for less spam and other unwanted things
                          • Squid & SquidGiuard as a caching proxy in fron of LAN
                          • ClamAV scanning the entire network flow for viruses
                          • (perhaps at one day WiFi a/b/g/n/ax)

                          Now, THIS IS good enough for me :-). So I suggest is
                          a good potential solution for others who are happy to
                          offload the PPPoE function to another inline appliance.

                          I run a AVM FB 7590ax in front of the pfSense and behind
                          I am running the pfSense firewall! No PPPoE anymore, but
                          double NAT situation! But all CPU cores in usage!

                          • AVM is offering some interesting APPs (VPN, telephone,..)
                          • Really nice to connect from outside (internet) and being secure on the LAN side!

                          Now I just need to work out if I can pass through
                          the WAN IP somehow to my PFSense :-)

                          • 1 LAN Port as "exposed host" to the WAN interface
                            of the pfSense firewall ("Experienced")
                          • Double NAT Situation

                          Router:
                          network (net) 192.168.178.0/24 (255.255.255.0)
                          Router IP 192.168.178.1/24 (255.255.255.0)
                          Static IP Address to the pfSense a.e. 192.168.178.10/24
                          DHCP off: all IPs will be static given to the clients

                          pfSense:
                          WAN IP 192.168.178.50/24 (255.255.255.0) static IP
                          LAN Net: 172.xx.xx.0/24 (255.255.255.0)
                          LAN IP 172.xx.xx.1/24 (255.255.255.0) static IP
                          DHCP: on/off (Like you need it and want it)

                          Thanks for your help again - I really appreciate the
                          pointers that ultimately led me to get a working
                          solution.

                          Not that problem, you are one from xyz sitting in the same
                          boat. I would also have a look on another appliance if I`ll
                          getting more then 50 MBit/s Internet speed!!!

                          P.S.
                          Please don´t forget in the WAN setup to disable the
                          following point!

                          WAN settings.jpg

                          1 Reply Last reply Reply Quote 0
                          • First post
                            Last post
                          Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.