Navigation

    Netgate Discussion Forum
    • Register
    • Login
    • Search
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search

    Pkg tool needs some common sense?

    2.4 Development Snapshots
    11
    28
    1734
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • C
      chrcoluk last edited by

      So today I installed 2.4.1 on my new unit.

      There was an issue with ipv6 due to a configuration issue, the unit had an active ipv6 address but traffic was dead, everything just basically timed out, ipv4 was fine.

      I noticed the pkg tool was very stupid in how it handled the problem.

      Basically it took me almost 4 hours to run a config restore because the pkg install process took ages.  it tried to connect via ipv6, waited a massive 15 mins (extremely high timeout), then fell back to ipv4, did this for every connection it had to do, before I eventually got to the end of the process.

      In the gui, if trying to do any package management, it would just spin the circle for 15 minutes before giving a vague failure message, so seems there was no fallback attempt made.

      So I suggest a few things.

      1 - more verbose output, pfsense is I would hope not a "not for dummies" design by making messages as low verbosity as possible.  Often during the cli package process it just sat there with no indications of what was going on, my patience allowed it to finish.
      2- a sane timeout such as 6 seconds, not 900 seconds.
      3 - prevent it from automatically checking for updates (extra connections needed) every time a pkg install command is run, add some intelligence to it so if e.g. it know it checked only 15 mins ago there is no need to check again right now.

      During the 4 hours, there was no firewall active, I had e.g. login attempts to sshd from internet bots, the firewall was down because the pfblockerng tables were none existant due to waiting for the pkg restore process.  So the restore process activates altered rules related to packages before the packages are restored.

      pfSense 2.6.0 - ISP AAISP UK

      1 Reply Last reply Reply Quote 0
      • D
        doktornotor Banned last edited by

        I have complained about the idiotic timeout ever since pkg has been introduced in 2.3. There's a bug about it somewhere - Bug #7604 which got reopened after Bug #6594 has been "resolved". As noted there, you can CTRL+C and exit from this idiocy assuming you have serial console access.

        I cannot even trace where do those insane cumulative timeout values come from; at one time, I managed to get some timeout value dropped from 300 to 60 secs, assuming this should help significantly, but turned out that it was only used for the GUI. And even there, it's multiplied by a factor of 15 due to some whacky retries or god knows what. The timeout values in /usr/local/sbin/pfsense-upgrade clearly are totally useless and not honored at all, or the manpage gets it wrong and the FETCH_TIMEOUT is in minutes instead of seconds.

        Otherwise, "intelligence" and pkg does not go together, and FreeBSD seems to be extremely unlucky when it comes to package managers. First the PBI junk, now this.

        @chrcoluk:

        During the 4 hours, there was no firewall active, I had e.g. login attempts to sshd from internet bots, the firewall was down because the pfblockerng tables were none existant due to waiting for the pkg restore process.

        This is Bug #6028. Should be kinda prioritized, instead of rotting there.

        1 Reply Last reply Reply Quote 0
        • johnpoz
          johnpoz LAYER 8 Global Moderator last edited by

          While I hear ya on the timeouts.. It has always been recommended to remove packages before upgrade..

          https://doc.pfsense.org/index.php/Upgrade_Guide
          "It is always safest to remove packages before upgrading to a new major release. Packages will be reinstalled afterward, but are frequently a source of problems. To ensure a smooth upgrade, note the installed packages, remove them, perform the upgrade, and then reinstall whichever packages are necessary. "

          An intelligent man is sometimes forced to be drunk to spend time with his fools
          If you get confused: Listen to the Music Play
          Please don't Chat/PM me for help, unless mod related
          2440 2.4.5p1 | 2x 3100 2.4.4p3 | 2x 3100 22.01 | 4860 22.05

          1 Reply Last reply Reply Quote 0
          • D
            doktornotor Banned last edited by

            That doesn't help much either. It still takes ~30 minutes because pkg tries to fetch the packages info no matter whether you have something installed or not. IOW, that goddamn tool is absolutely useless for any offline work.

            1 Reply Last reply Reply Quote 0
            • C
              chrcoluk last edited by

              as dok stated that wouldnt have helped.

              pfSense 2.6.0 - ISP AAISP UK

              1 Reply Last reply Reply Quote 0
              • K
                kpa last edited by

                The problem comes from libfetch (it uses an imported version of the library) as far as I know. Libfetch has a global timeout that can be configured but no provisions for configuring a specific IPv6 -> IPv4 fallback timeout in case IPv6 connections are not working.

                1 Reply Last reply Reply Quote 0
                • K
                  kpa last edited by

                  You can put this into /usr/local/etc/pkg.conf to force the use of just IPv4:

                  
                  IP_VERSION=4
                  
                  

                  You can also disable autofetching of the repo catalogs on every operation.

                  
                  REPO_AUTOUPDATE=NO
                  
                  

                  That might mess with the GUI updater though, use with care.

                  1 Reply Last reply Reply Quote 0
                  • P
                    PiBa last edited by

                    There is also a 'prefer IPv4' checkbox in the system/advanced settings.. might be safer assuming it solves this..

                    1 Reply Last reply Reply Quote 0
                    • luckman212
                      luckman212 LAYER 8 last edited by

                      @kpa:

                      You can put this into /usr/local/etc/pkg.conf to force the use of just IPv4:

                      
                      IP_VERSION=4
                      
                      

                      You can also disable autofetching of the repo catalogs on every operation.

                      
                      REPO_AUTOUPDATE=NO
                      
                      

                      That might mess with the GUI updater though, use with care.

                      Maybe this should automatically be done by /etc/rc.bootup "just in case" and then removed after platform_booting is false? Or, try to do some simplye connectivity checks to whatever URL is specified in pfSense-repo.conf … e.g.

                      1. test to see if its DNS can be resolved and then
                      2. test if it's pingable

                      before trying to do anything whatsoever with remote pkg commands?

                      1 Reply Last reply Reply Quote 0
                      • Y
                        yon last edited by

                        yes, me too, ipv6 all down in v2.4.1, i have to back to 2.4.0 ok.

                        If you are interested in free peering for clearnet and dn42,contact me !

                        1 Reply Last reply Reply Quote 0
                        • ?
                          Guest last edited by

                          @yon:

                          yes, me too, ipv6 all down in v2.4.1, i have to back to 2.4.0 ok.

                          I must have two very strange systems running 2.4.1 then, IPv6 is working fine on both of them, one using statics and one using DHCP6.

                          1 Reply Last reply Reply Quote 0
                          • D
                            doktornotor Banned last edited by

                            @yon:

                            yes, me too, ipv6 all down in v2.4.1, i have to back to 2.4.0 ok.

                            This is NOT the thread for debugging your broken 12WAN setups. Kindly leave it alone.

                            1 Reply Last reply Reply Quote 0
                            • ?
                              Guest last edited by

                              Snigger… ;D

                              1 Reply Last reply Reply Quote 0
                              • Y
                                yon last edited by

                                @doktornotor:

                                @yon:

                                yes, me too, ipv6 all down in v2.4.1, i have to back to 2.4.0 ok.

                                This is NOT the thread for debugging your broken 12WAN setups. Kindly leave it alone.

                                Are there any rules to limit the number of use? Is this a child's toy? Not a professional tool? ???
                                I would like any product to be tested in a complex and rigorous environment in order to find possible problems, rather than fear of problems and not dare to use.

                                If you are interested in free peering for clearnet and dn42,contact me !

                                1 Reply Last reply Reply Quote 0
                                • S
                                  Steve_B Developer Netgate last edited by

                                  No doubt you would like that for free, too?

                                  Als ik kan

                                  1 Reply Last reply Reply Quote 0
                                  • ?
                                    Guest last edited by

                                    @yon:

                                    @doktornotor:

                                    @yon:

                                    yes, me too, ipv6 all down in v2.4.1, i have to back to 2.4.0 ok.

                                    This is NOT the thread for debugging your broken 12WAN setups. Kindly leave it alone.

                                    Are there any rules to limit the number of use? Is this a child's toy? Not a professional tool? ???
                                    I would like any product to be tested in a complex and rigorous environment in order to find possible problems, rather than fear of problems and not dare to use.

                                    Perhaps your requirements need professional support, here's the link to enable you to purchase it. https://www.pfsense.org/get-support/

                                    1 Reply Last reply Reply Quote 0
                                    • P
                                      perlenbacher last edited by

                                      At least that support will come without all the snipey comments and abuse. Probably a good buy…

                                      1 Reply Last reply Reply Quote 0
                                      • C
                                        chrcoluk last edited by

                                        @kpa:

                                        You can put this into /usr/local/etc/pkg.conf to force the use of just IPv4:

                                        
                                        IP_VERSION=4
                                        
                                        

                                        You can also disable autofetching of the repo catalogs on every operation.

                                        
                                        REPO_AUTOUPDATE=NO
                                        
                                        

                                        That might mess with the GUI updater though, use with care.

                                        excellent thank you.

                                        I will see if I can get someone to add options for this in the GUI.

                                        pfSense 2.6.0 - ISP AAISP UK

                                        1 Reply Last reply Reply Quote 0
                                        • C
                                          chrcoluk last edited by

                                          @yon:

                                          yes, me too, ipv6 all down in v2.4.1, i have to back to 2.4.0 ok.

                                          I dont blame blame 2.4.1 for the ipv6 issue, it was down to an exotic config I had in place, and was just coincidence this is the first time I have ran a restore with that config which broke ipv6.

                                          My ipv6 has been fine since I fixed the config issue.  I waited to fix it as was advised in the pfsense UI to not make any config changes with a config restore in process.

                                          pfSense 2.6.0 - ISP AAISP UK

                                          1 Reply Last reply Reply Quote 0
                                          • ?
                                            Guest last edited by

                                            @chrcoluk:

                                            @kpa:

                                            You can put this into /usr/local/etc/pkg.conf to force the use of just IPv4:

                                            
                                            IP_VERSION=4
                                            
                                            

                                            You can also disable autofetching of the repo catalogs on every operation.

                                            
                                            REPO_AUTOUPDATE=NO
                                            
                                            

                                            That might mess with the GUI updater though, use with care.

                                            excellent thank you.

                                            I will see if I can get someone to add options for this in the GUI.

                                            OK Chris, I've added the two you mentioned in the email, threw me a bit as they are ENV vars now, no longer in a conf file!

                                            Do you or anyone else want other vars added, the two that shout at me are these:

                                            FETCH_RETRY: integer
                                              Number of times to retry a failed fetch of a file.  Default:
                                              3.

                                            FETCH_TIMEOUT: integer
                                              Maximum number of seconds to wait for any one file to down-
                                              load from the network, either by SSH or any of the protocols
                                              supported by fetch(3) functions.  Default: 30.

                                            1 Reply Last reply Reply Quote 0
                                            • C
                                              chrcoluk last edited by

                                              thanks bud, I will give it a whirl later today :)

                                              pfSense 2.6.0 - ISP AAISP UK

                                              1 Reply Last reply Reply Quote 0
                                              • ?
                                                Guest last edited by

                                                @chrcoluk:

                                                thanks bud, I will give it a whirl later today :)

                                                Forget the package timout & retry, they are already set to 5 & 2 respectively in the code anyway.

                                                1 Reply Last reply Reply Quote 0
                                                • D
                                                  doktornotor Banned last edited by

                                                  @marjohn56:

                                                  @chrcoluk:

                                                  thanks bud, I will give it a whirl later today :)

                                                  Forget the package timout & retry, they are already set to 5 & 2 respectively in the code anyway.

                                                  Yeah that has no effect at all on anything.

                                                  1 Reply Last reply Reply Quote 0
                                                  • ?
                                                    Guest last edited by

                                                    @doktornotor:

                                                    @marjohn56:

                                                    @chrcoluk:

                                                    thanks bud, I will give it a whirl later today :)

                                                    Forget the package timout & retry, they are already set to 5 & 2 respectively in the code anyway.

                                                    Yeah that has no effect at all on anything.

                                                    Perhaps the two I've added will have some effect, at least the use v4 one might.

                                                    1 Reply Last reply Reply Quote 0
                                                    • F
                                                      FranciscoFranco last edited by

                                                      @doktornotor:

                                                      Otherwise, "intelligence" and pkg does not go together, and FreeBSD seems to be extremely unlucky when it comes to package managers. First the PBI junk, now this.

                                                      As a daily user of FreeBSD I can say wholeheartedly that pkg-ng is not broke and works quite well. Don't mix ports and packages.
                                                      I have see an freebsd-update that went awry and messed with a library needed for pkg, but pkg-static handled that.

                                                      After some recent run ins with Linux installers I think FreeBSD installer is on par with the others.

                                                      These timeouts have nothing to do with FreeBSD. This is a pfSense problem.

                                                      pkg-ng does work well offline. I am using it in my NanoBSD-build chroot making packages for cross-arch usage.

                                                      Plus you can use pkg add your.txz to feed FreeBSD packages files in an offline state.
                                                      You do have to provide all the dependencies too.

                                                      It works exactly as it should from my encounters. None of the installers are perfect from my usage.
                                                      Not keeping your packages up to date can be a problem. On any platform.

                                                      I can't speak for pbi's because they were before my time. I was still over here rubbing two sticks together.
                                                      There are probably some bugs in pkg-ng but standard use cases it works like a champ.

                                                      My defense of FreeBSD pkg rests. There are plenty of broken things in FreeBSD to complain about but pkg ain't one of them.

                                                      1 Reply Last reply Reply Quote 0
                                                      • C
                                                        chrcoluk last edited by

                                                        pkg add for FreeBSD packages on pfsense used to work perfect offline, but now tries to contact the pfsense repos, so seems new behaviour in 2.4.1, I have not tested yet if this is fixed with the config option to disabled forced repo checks.

                                                        On FreeBSD I typically use ports not pkg, as I have always preferred compiling to pre compiled stuff, so cant really comment on the pkg ng implementation on FreeBSD.

                                                        pfSense 2.6.0 - ISP AAISP UK

                                                        1 Reply Last reply Reply Quote 0
                                                        • K
                                                          kpa last edited by

                                                          Pkg is fine but pfSense could use more sensible defaults like not autoupdating the repo catalogs every time, it is not required for proper operation. Also none of the options can not be changed via the webgui and the update system is now a giant black box that doesn't reveal any of its inner workings which makes debugging problems very hard.

                                                          1 Reply Last reply Reply Quote 0
                                                          • ?
                                                            Guest last edited by

                                                            @kpa:

                                                            Pkg is fine but pfSense could use more sensible defaults like not autoupdating the repo catalogs every time, it is not required for proper operation. Also none of the options can not be changed via the webgui and the update system is now a giant black box that doesn't reveal any of its inner workings which makes debugging problems very hard.

                                                            I've added a couple of options which Chris is going to test later. I can add more if needed.

                                                            1 Reply Last reply Reply Quote 0
                                                            • First post
                                                              Last post