Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    VERY slow System Update

    Scheduled Pinned Locked Moved General pfSense Questions
    46 Posts 6 Posters 6.0k Views 5 Watching
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • Sergei_ShablovskyS Offline
      Sergei_Shablovsky @stephenw10
      last edited by

      @stephenw10 said in VERY slow System Update:

      Indeed iperf would also be affected if it was going over the same route but that's unlikely. We ca say whatever part of the route that is shared does not have a problem.

      Run a pcap for traffic to/from the update server. Look for fragmentation.

      Thank You, Steve for suggestions!

      No fragmentation, checked at the first step.

      BTW, when doing “pkg upgrade” from command line on local console, after downloading first packet the output from pkg looks like stopped. No, system working and no freeze, all other LANs working...

      I even trying to “pkg unlock -a”, but this also no impact on situation...

      —
      CLOSE SKY FOR UKRAINE https://youtu.be/_tU1i8VAdCo !
      Help Ukraine to resist, save civilians people’s lives !
      (Take an active part in public protests, push on Your country’s politics, congressmans, mass media, leaders of opinion.)

      1 Reply Last reply Reply Quote 0
      • stephenw10S Offline
        stephenw10 Netgate Administrator
        last edited by

        So it sends and then never receives a reply? And doesn't just timeout?

        That sounds more like something broken in the pkg system locally.

        Steve

        Sergei_ShablovskyS 1 Reply Last reply Reply Quote 0
        • Sergei_ShablovskyS Offline
          Sergei_Shablovsky @stephenw10
          last edited by Sergei_Shablovsky

          @stephenw10 said in VERY slow System Update:

          So it sends and then never receives a reply? And doesn't just timeout?

          Thank You for patience with this case, Steve.

          Not looks like this.

          When come into local shell, trying to make manually

          pkg upgrade
          

          after depositories checks, the packet downloading process starting, even first .pkg in Queue downloaded fast (at 70-90Mb/s), but after this first .pkg (or just from start of Queue), the download speed on screen indicated as

          16,4kB/s 01:43:17ETA
          

          (this is for pfSense-kernel-pfSense-2.6.0.a.20211029.0500.pkg fetching for example)
          and on a big .pkg size connection closed, may be from the NetGate updates server side

          Let me remind, any other downloads or third party .pkg downloaded on full available bandwidth, I test this more than triple-twice.

          That sounds more like something broken in the pkg system locally.

          I also thinking about this, so try to pkg unlock -a, purging pkg cache....
          But still unsuccessfully...

          [UPDATE]
          Today I have a little bit more time to make some experiments. And found that the SLOW speed only for 2nd, 3rd, and down the list .pkg files, but FIRST FILE ALWAYS DOWNLOADING AT FULL AVAILABLE BANDWITH. (i.e. from 24 to 80MB/s).q

          —
          CLOSE SKY FOR UKRAINE https://youtu.be/_tU1i8VAdCo !
          Help Ukraine to resist, save civilians people’s lives !
          (Take an active part in public protests, push on Your country’s politics, congressmans, mass media, leaders of opinion.)

          Sergei_ShablovskyS 1 Reply Last reply Reply Quote 0
          • stephenw10S Offline
            stephenw10 Netgate Administrator
            last edited by

            Hmm, that really looks like a routing issue or an IPv6 issue. But I believe we already ruled that out?

            Do you have a VPN or some other tunnel you could try to route that traffic across as a test?

            Steve

            Sergei_ShablovskyS 2 Replies Last reply Reply Quote 1
            • Sergei_ShablovskyS Offline
              Sergei_Shablovsky @stephenw10
              last edited by Sergei_Shablovsky

              @stephenw10
              Today I have a little bit more time to make some experiments. And found that the SLOW speed only for 2nd, 3rd, and down the list .pkg files, but FIRST FILE ALWAYS DOWNLOADING AT FULL AVAILABLE BANDWITH. (i.e. from 24 to 80MB/s).

              —
              CLOSE SKY FOR UKRAINE https://youtu.be/_tU1i8VAdCo !
              Help Ukraine to resist, save civilians people’s lives !
              (Take an active part in public protests, push on Your country’s politics, congressmans, mass media, leaders of opinion.)

              1 Reply Last reply Reply Quote 0
              • Sergei_ShablovskyS Offline
                Sergei_Shablovsky @stephenw10
                last edited by

                @stephenw10 said in VERY slow System Update:

                Hmm, that really looks like a routing issue or an IPv6 issue. But I believe we already ruled that out?

                The routing - is the first that I thinking on. But if something wrong with routing **why this not impact on any other traffic from/to any other destination/source.

                Do you have a VPN or some other tunnel you could try to route that traffic across as a test?

                I'm a little bit stupid today, could You be so please to write in details what exactly you mean and how doing that? Thx a lot!!!

                —
                CLOSE SKY FOR UKRAINE https://youtu.be/_tU1i8VAdCo !
                Help Ukraine to resist, save civilians people’s lives !
                (Take an active part in public protests, push on Your country’s politics, congressmans, mass media, leaders of opinion.)

                1 Reply Last reply Reply Quote 0
                • stephenw10S Offline
                  stephenw10 Netgate Administrator
                  last edited by

                  If there is something specifically in the route that traffic is taking that throttling it then routing if over a VPN will likely change that route and might allow full speed repeatedly. It would at least prove the routing issue theory.

                  I have an OpenVPN tunnel setup to an instance in GCP for that purpose. But almost anywhere would change the route to test it.

                  Steve

                  Sergei_ShablovskyS 2 Replies Last reply Reply Quote 1
                  • Sergei_ShablovskyS Offline
                    Sergei_Shablovsky @stephenw10
                    last edited by Sergei_Shablovsky

                    @stephenw10 said in VERY slow System Update:

                    If there is something specifically in the route that traffic is taking that throttling it then routing if over a VPN will likely change that route and might allow full speed repeatedly. It would at least prove the routing issue theory.

                    I have an OpenVPN tunnel setup to an instance in GCP for that purpose. But almost anywhere would change the route to test it.

                    Thank You for all help here, Stephen!

                    Task SOLVED, due the source of this sow pfSense update was in speed/control settings for NIC's interface: need to be changed from 100TX full-duplex to 100TX (mean 100TX half-duplex).
                    After this speed of update come from fixed 16,4k to 1,5-2,5M.

                    I do not know why *the exactly pfSense updates are so slow and no other netflow affected”, but anyway, this certain issue closed and finding the source of problem come to another level, please see Errors In, but no Errors Out and No Collisions on both WAN.

                    —
                    CLOSE SKY FOR UKRAINE https://youtu.be/_tU1i8VAdCo !
                    Help Ukraine to resist, save civilians people’s lives !
                    (Take an active part in public protests, push on Your country’s politics, congressmans, mass media, leaders of opinion.)

                    1 Reply Last reply Reply Quote 0
                    • Sergei_ShablovskyS Offline
                      Sergei_Shablovsky @Sergei_Shablovsky
                      last edited by Sergei_Shablovsky

                      @sergei_shablovsky said in [SOLVED] - VERY slow System Update:

                      [UPDATE]
                      Today I have a little bit more time to make some experiments. And found that the SLOW speed only for 2nd, 3rd, and down the list .pkg files, but FIRST FILE ALWAYS DOWNLOADING AT FULL AVAILABLE BANDWITH. (i.e. from 24 to 80MB/s).q

                      Need to note about this “full speed” I make wrong conclusion: the speed on screen are just measured from already readed cache, so extremely hi-speed to view possible, like 1,200Mb on 100M physical connection & port speed. ;)
                      But the real speed are with the same limitation 16,4kBs.

                      I make some extra experiments on this server:
                      trying to download the 420Mb full pfSense install by fetch

                      fetch -v -o /dev/null https://snapshots.netgate.com/amd64/pfSense_master/installer/pfSense-CE-memstick-2.6.0-DEVELOPMENT-amd64-latest.img.gz
                      

                      The speed and ETA on screen was:

                      6770 Bps 18h08m
                      

                      I conclude this speed are max per one connect from Netgate hosting provider.

                      —
                      CLOSE SKY FOR UKRAINE https://youtu.be/_tU1i8VAdCo !
                      Help Ukraine to resist, save civilians people’s lives !
                      (Take an active part in public protests, push on Your country’s politics, congressmans, mass media, leaders of opinion.)

                      johnpozJ 1 Reply Last reply Reply Quote 1
                      • johnpozJ Offline
                        johnpoz LAYER 8 Global Moderator @Sergei_Shablovsky
                        last edited by johnpoz

                        @sergei_shablovsky said in [SOLVED] - VERY slow System Update:

                        I conclude this speed are max per one connect from Netgate hosting provider.

                        Not true at all.. Maybe from your ISP and peering issue to there.. But its clearly not a "netgate" connection or server issue.

                        download.jpg

                        An intelligent man is sometimes forced to be drunk to spend time with his fools
                        If you get confused: Listen to the Music Play
                        Please don't Chat/PM me for help, unless mod related
                        SG-4860 24.11 | Lab VMs 2.8, 24.11

                        Sergei_ShablovskyS 1 Reply Last reply Reply Quote 1
                        • Sergei_ShablovskyS Offline
                          Sergei_Shablovsky @johnpoz
                          last edited by

                          @johnpoz said in [SOLVED] - VERY slow System Update:

                          @sergei_shablovsky said in [SOLVED] - VERY slow System Update:

                          I conclude this speed are max per one connect from Netgate hosting provider.

                          Not true at all.. Maybe from your ISP and peering issue to there.. But its clearly not a "netgate" connection or server issue.

                          download.jpg

                          I also love Netgate ;)
                          And really thankful for their support the pfSense's User Community.

                          But as I wrote before in this tread (and other see Errors In, but no Errors Out and No Collisions on both WAN), the iperf3 to different geographic servers (USA, UK, France, HonKong, Latvia, ...) in different regions give me 70-88Mb/s on 100M physical connection.
                          The SAME ISP and the SAME physical connection.

                          —
                          CLOSE SKY FOR UKRAINE https://youtu.be/_tU1i8VAdCo !
                          Help Ukraine to resist, save civilians people’s lives !
                          (Take an active part in public protests, push on Your country’s politics, congressmans, mass media, leaders of opinion.)

                          johnpozJ 1 Reply Last reply Reply Quote 0
                          • johnpozJ Offline
                            johnpoz LAYER 8 Global Moderator @Sergei_Shablovsky
                            last edited by

                            @sergei_shablovsky said in [SOLVED] - VERY slow System Update:

                            the iperf3 to different geographic servers (USA, UK, France, HonKong, Latvia, ...)

                            How does that have anything to do with downloading from the netgate servers?

                            An intelligent man is sometimes forced to be drunk to spend time with his fools
                            If you get confused: Listen to the Music Play
                            Please don't Chat/PM me for help, unless mod related
                            SG-4860 24.11 | Lab VMs 2.8, 24.11

                            Sergei_ShablovskyS 1 Reply Last reply Reply Quote 0
                            • Sergei_ShablovskyS Offline
                              Sergei_Shablovsky @johnpoz
                              last edited by Sergei_Shablovsky

                              @johnpoz said in [SOLVED] - VERY slow System Update:

                              How does that have anything to do with downloading from the netgate servers?

                              Shaping on some AS.

                              Because in fact iperf3 give me ~80Mb/s worldwide, but fetch from Netgate server - only 16.4kB/s on same physical connection.

                              —
                              CLOSE SKY FOR UKRAINE https://youtu.be/_tU1i8VAdCo !
                              Help Ukraine to resist, save civilians people’s lives !
                              (Take an active part in public protests, push on Your country’s politics, congressmans, mass media, leaders of opinion.)

                              M 1 Reply Last reply Reply Quote 0
                              • M Offline
                                MoonKnight @Sergei_Shablovsky
                                last edited by

                                @sergei_shablovsky

                                Downloaded this one, and got around 2.5 MB/s
                                2ac0d823-0673-4909-916b-5578e874f24a-image.png

                                https://snapshots.pfsense.org/amd64/pfSense_master/installer/pfSense-CE-2.6.0-DEVELOPMENT-amd64-20211104-0500.iso.gz

                                --- 24.11 ---
                                Intel(R) Xeon(R) CPU D-1518 @ 2.20GHz
                                Kingston DDR4 2666MHz 16GB ECC
                                2 x HyperX Fury SSD 120GB (ZFS-mirror)
                                2 x Intel i210 (ports)
                                4 x Intel i350 (ports)

                                Sergei_ShablovskyS 1 Reply Last reply Reply Quote 1
                                • Sergei_ShablovskyS Offline
                                  Sergei_Shablovsky @MoonKnight
                                  last edited by

                                  @ciscox said in [SOLVED] - VERY slow System Update:

                                  @sergei_shablovsky

                                  Downloaded this one, and got around 2.5 MB/s
                                  2ac0d823-0673-4909-916b-5578e874f24a-image.png

                                  https://snapshots.pfsense.org/amd64/pfSense_master/installer/pfSense-CE-2.6.0-DEVELOPMENT-amd64-20211104-0500.iso.gz

                                  Slowly we collecting pfSense speed on images download worldwide ;)

                                  —
                                  CLOSE SKY FOR UKRAINE https://youtu.be/_tU1i8VAdCo !
                                  Help Ukraine to resist, save civilians people’s lives !
                                  (Take an active part in public protests, push on Your country’s politics, congressmans, mass media, leaders of opinion.)

                                  1 Reply Last reply Reply Quote 0
                                  • Sergei_ShablovskyS Offline
                                    Sergei_Shablovsky @stephenw10
                                    last edited by Sergei_Shablovsky

                                    @stephenw10 said in [SOLVED] - VERY slow System Update:

                                    there is something specifically in the route that traffic is taking that throttling

                                    That’s what about this post ;)
                                    Even on clean pfSense installation (no any rules, nothing) on that particular machine

                                    pkg upgrade
                                    

                                    or

                                    pkg update
                                    

                                    where fetch command used

                                    give me 16,4KB/s ;)

                                    —
                                    CLOSE SKY FOR UKRAINE https://youtu.be/_tU1i8VAdCo !
                                    Help Ukraine to resist, save civilians people’s lives !
                                    (Take an active part in public protests, push on Your country’s politics, congressmans, mass media, leaders of opinion.)

                                    GertjanG 1 Reply Last reply Reply Quote 0
                                    • GertjanG Offline
                                      Gertjan @Sergei_Shablovsky
                                      last edited by Gertjan

                                      @sergei_shablovsky

                                      On an old pfSense 1.x - and upgraded gradually to 2.5.2 CE - so probably not clean at all :

                                      63629071-3f6f-4c9b-906f-f08f5910ed63-image.png

                                      20Mbits is my WAN-down.

                                      No "help me" PM's please. Use the forum, the community will thank you.
                                      Edit : and where are the logs ??

                                      Sergei_ShablovskyS 1 Reply Last reply Reply Quote 1
                                      • Sergei_ShablovskyS Offline
                                        Sergei_Shablovsky @Gertjan
                                        last edited by Sergei_Shablovsky

                                        @gertjan said in VERY slow System Update:

                                        @sergei_shablovsky

                                        On an old pfSense 1.x - and upgraded gradually to 2.5.2 CE - so probably not clean at all :

                                        63629071-3f6f-4c9b-906f-f08f5910ed63-image.png

                                        20Mbits is my WAN-down.

                                        Thank You!
                                        Is this graph only for fetch image from Netgate server (I mean dev version, nor mirrors in US and other countries) ?

                                        —
                                        CLOSE SKY FOR UKRAINE https://youtu.be/_tU1i8VAdCo !
                                        Help Ukraine to resist, save civilians people’s lives !
                                        (Take an active part in public protests, push on Your country’s politics, congressmans, mass media, leaders of opinion.)

                                        GertjanG 1 Reply Last reply Reply Quote 0
                                        • GertjanG Offline
                                          Gertjan @Sergei_Shablovsky
                                          last edited by

                                          @sergei_shablovsky
                                          👍
                                          The answer is : no.
                                          My iPhone also decides to update itself to 15.1 (2 GB download) and the rest of the companies network was also actif.

                                          I also use an IPv6 connection, that tunnel IPv4 packets out over the WAN with IPv6 traffic in it. So my IPv6 goes over the IPv4 WAN (technically, I have a double WAN setup).

                                          I've shut down our main switch, so I was using the only PC on the network :
                                          Test again :

                                          [2.5.2-RELEASE][admin@pfsense.my-local-network.net]/root: fetch -v -o /dev/null https://snapshots.netgate.com/amd64/pfSense_master/installer/pfSense-CE-memstick-2.6.0-DEVELOPMENT-amd64-latest.img.gz
                                          resolving server address: snapshots.netgate.com:443
                                          SSL options: 82004854
                                          Peer verification enabled
                                          Using CA cert file: /usr/local/etc/ssl/cert.pem
                                          Verify hostname
                                          TLSv1.2 connection established using ECDHE-RSA-AES256-GCM-SHA384
                                          Certificate subject: /CN=*.netgate.com
                                          Certificate issuer: /C=GB/ST=Greater Manchester/L=Salford/O=Sectigo Limited/CN=Sectigo RSA Domain Validation Secure Server CA
                                          requesting https://snapshots.netgate.com/amd64/pfSense_master/installer/pfSense-CE-memstick-2.6.0-DEVELOPMENT-amd64-latest.img.gz
                                          remote size / mtime: 439979895 / 1636700812
                                          /dev/null                                              419 MB 1868 kBps 03m50s
                                          

                                          That is close to 19 Mbits / sec, my ADSL down bandwidth.

                                          It was using IPv4 - as IPv6 (ipv6.he.net) would be slower for me.

                                          No "help me" PM's please. Use the forum, the community will thank you.
                                          Edit : and where are the logs ??

                                          1 Reply Last reply Reply Quote 0
                                          • First post
                                            Last post
                                          Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.