Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    Squid3 PBI and 2.1

    Scheduled Pinned Locked Moved 2.1 Snapshot Feedback and Problems - RETIRED
    24 Posts 4 Posters 10.6k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • P
      phil.davis
      last edited by

      After trying to remember what was done for summary item (1), I think that the previous squid got built with the correct conf file location for pfSense built into the image. So there was no need to add the "-f" parameter to every reference to starting/reconfiguring/stopping squid in squid.inc (I couldn't see where any "-f" had been added in the squid(2) version of squid.inc).

      Maybe all 3 items above can be corrected in the build of the PBI used by squid3 - conf file location, support for extra options and support for transparent proxy.

      As the Greek philosopher Isosceles used to say, "There are 3 sides to every triangle."
      If I helped you, then help someone else - buy someone a gift from the INF catalog http://secure.inf.org/gifts/usd/

      1 Reply Last reply Reply Quote 0
      • C
        Cino
        last edited by

        most of the was already reported: http://forum.pfsense.org/index.php/topic,48347.msg269453.html#msg269453

        I noticed a new issue with installing… Did a complete remove this morning, reboot, then install. Install is stuck at Creating squid cache pools... One moment please... and php process has been at 100% for 5 minutes... Checked the file system, no files being built... Thinking this may because of the -f option that is needed now or squid will use its default settings.

        1 Reply Last reply Reply Quote 0
        • jimpJ
          jimp Rebel Alliance Developer Netgate
          last edited by

          The user should be fixed by (a) a new snapshot and (b) a new PBI, as the PBI tools were fixed just yesterday to supporting properly adding needed users.

          The config file was never correct in /usr/local/etc - it should be /var/etc, so long as it's fixed, and it must be fixed in squid.inc and such. A global won't work, it'll need to be a constant, for whatever reason globals don't work properly in package .inc files at bootup, there are a few other threads about it.

          The options there, delay pools and such, should be fixed also by a new PBI. Not sure why they were not proper in the current one but they were set, might just need a new build.

          As for the version, yes that does mismatch but since the .tbz version is still older it wasn't bumped.

          Remember: Upvote with the 👍 button for any user/post you find to be helpful, informative, or deserving of recognition!

          Need help fast? Netgate Global Support!

          Do not Chat/PM for help!

          1 Reply Last reply Reply Quote 0
          • C
            Cino
            last edited by

            Thanks Jim! Question, is the new snapshot for binaries? I installed last Sat but gitsync often.

            For the conf path, I've noticed a lot of packages that use /usr/local/etc… Is the plan to have all of them moved to /var/etc?

            1 Reply Last reply Reply Quote 0
            • jimpJ
              jimp Rebel Alliance Developer Netgate
              last edited by

              All of our configs should have always been in /var/etc, but historically packages haven't really cared quite so much. Many were left in /usr/local/etc simply because it was the default.

              As long as changes are being made to manually specify the config path, may as well put them where they're supposed to go.

              Remember: Upvote with the 👍 button for any user/post you find to be helpful, informative, or deserving of recognition!

              Need help fast? Netgate Global Support!

              Do not Chat/PM for help!

              1 Reply Last reply Reply Quote 0
              • P
                phil.davis
                last edited by

                I deleted all my packages first (to avoid any possibility that old binaries were left around) then upgraded to:
                2.1-BETA0 (i386)
                built on Tue Jun 19 14:25:19 EDT 2012
                FreeBSD 8.3-RELEASE-p3

                Then installed squid3. This latest version of the PBI was on http://files.pfsense.org/packages/8/All/ :
                squid-3.1.20-i386.pbi 2012-Jun-19 15:41:12 15.8M application/octet-stream

                So it should have loaded this PBI that Jim put there yesterday.

                Edited squid.inc to make all the start/stop/reconfigure commands point to the correct squid.conf (I'll submit a pull request for this in Github soon).

                [2.1-BETA0][root@test20120614.localdomain]/usr/local/etc/rc.d(28): /usr/local/sbin/squid -D -f /usr/local/etc/squid/squid.conf
2012/06/20 10:05:45| WARNING: -D command-line option is obsolete.
2012/06/20 10:05:45| cache_cf.cc(381) parseOneConfigFile: squid.conf:17 unrecognized: 'sslcrtd_children'
2012/06/20 10:05:45| cache_cf.cc(381) parseOneConfigFile: squid.conf:61 unrecognized: 'delay_pools'
2012/06/20 10:05:45| cache_cf.cc(381) parseOneConfigFile: squid.conf:62 unrecognized: 'delay_class'
2012/06/20 10:05:45| cache_cf.cc(381) parseOneConfigFile: squid.conf:63 unrecognized: 'delay_parameters'
2012/06/20 10:05:45| cache_cf.cc(381) parseOneConfigFile: squid.conf:64 unrecognized: 'delay_initial_bucket_level'
2012/06/20 10:05:45| cache_cf.cc(381) parseOneConfigFile: squid.conf:65 unrecognized: 'delay_access'

                I noticed that the "squid -D" command-line option is now obsolete - this is mentioned in a few posts on the WWW such as at
                http://squid-web-proxy-cache.1019090.n4.nabble.com/questions-with-squid-3-1-td1557011.html

                2. # sbin/squid -D
                2010/02/16 15:02:41| WARNING: -D command-line option is obsolete.

                -D is obsolete, why and what's the corresponding one to this option in
                squid-3.1?

                -D existed only to solve one problem which is now fully fixed.

                But I have trouble finding this change mentioned anywhere squid 3.1 doco!
                I'll remove "-D" in my squid.inc pull request.

                Edited squid.inc temporarily to comment out all the unrecognized options above. Then squid will start.

                /var/squid/logs/cache.log still reports:

                2012/06/20 10:12:07| Ready to serve requests.
2012/06/20 10:17:34| WARNING: transparent proxying not supported

                Issues that I still have:

                1. The various squid config options above are unrecognized.
                2. It gives the warning about transparent proxying not supported.

                I think both these issues need to be fixed inside the PBI file?

                As the Greek philosopher Isosceles used to say, "There are 3 sides to every triangle."
                If I helped you, then help someone else - buy someone a gift from the INF catalog http://secure.inf.org/gifts/usd/

                1 Reply Last reply Reply Quote 0
                • jimpJ
                  jimp Rebel Alliance Developer Netgate
                  last edited by

                  That is odd as I am specifying everything in the build that needs to be there for the options to work, and yet they seem to not be getting pulled in.

                  Others have said that squid 2.x and squidguard are working, and they both specify options the same way, so I'm not really sure why it would be failing like that. I'll have to run some tests and see for myself what it's doing.

                  Remember: Upvote with the 👍 button for any user/post you find to be helpful, informative, or deserving of recognition!

                  Need help fast? Netgate Global Support!

                  Do not Chat/PM for help!

                  1 Reply Last reply Reply Quote 0
                  • jimpJ
                    jimp Rebel Alliance Developer Netgate
                    last edited by

                    OK so I discovered that the pbi.conf variable names changed somewhere between when our scripts were written and the current code for building PBIs that we had to pull in to fix the user issue, so I made a few changes to the build script but that still didn't seem to help yet, I just tried it on a vm (feel free to try it yourself though)

                    To make sure the new binary gets pulled in, you should probably uninstall/reinstall to make sure it gets the new binary. I think it only removes the binary if there is a version difference in the binary itself, not just if the pfSense package version gets bumped, but I'd have to double check that.

                    I've got another idea cooking now, will know in a while if it's good.

                    Remember: Upvote with the 👍 button for any user/post you find to be helpful, informative, or deserving of recognition!

                    Need help fast? Netgate Global Support!

                    Do not Chat/PM for help!

                    1 Reply Last reply Reply Quote 0
                    • C
                      Cino
                      last edited by

                      Thank for all your work on getting the PBIs to work… At first I wasn't a fan of them but now I see they are good thing for pfsense...

                      1 Reply Last reply Reply Quote 0
                      • jimpJ
                        jimp Rebel Alliance Developer Netgate
                        last edited by

                        OK - looks good now - have at it!

                        Remember: Upvote with the 👍 button for any user/post you find to be helpful, informative, or deserving of recognition!

                        Need help fast? Netgate Global Support!

                        Do not Chat/PM for help!

                        1 Reply Last reply Reply Quote 0
                        • C
                          Cino
                          last edited by

                          you da man!!!

                          Squid Cache: Version 3.1.20
configure options:  '--with-default-user=squid' '--bindir=/usr/pbi/squid-i386/sbin' '--sbindir=/usr/pbi/squid-i386/sbin' '--datadir=/usr/pbi/squid-i386/etc/squid' '--libexecdir=/usr/pbi/squid-i386/libexec/squid' '--localstatedir=/var/squid' '--sysconfdir=/usr/pbi/squid-i386/etc/squid' '--with-logdir=/var/log/squid' '--with-pidfile=/var/run/squid/squid.pid' '--enable-removal-policies=lru heap' '--disable-linux-netfilter' '--disable-linux-tproxy' '--disable-epoll' '--disable-translation' '--enable-auth=basic digest negotiate ntlm' '--enable-basic-auth-helpers=DB NCSA PAM MSNT SMB squid_radius_auth LDAP SASL YP' '--enable-digest-auth-helpers=password ldap' '--enable-external-acl-helpers=ip_user session unix_group wbinfo_group ldap_group' '--enable-ntlm-auth-helpers=smb_lm' '--enable-negotiate-auth-helpers=squid_kerb_auth' '--enable-storeio=ufs diskd aufs' '--enable-disk-io=AIO Blocking DiskDaemon DiskThreads' '--enable-delay-pools' '--enable-ssl' '--with-openssl=/usr' '--enable-ssl-crtd' '--enable-icmp' '--enable-forw-via-db' '--enable-cache-digests' '--disable-wccp' '--enable-wccpv2' '--enable-referer-log' '--enable-useragent-log' '--enable-arp-acl' '--enable-ipfw-transparent' '--enable-pf-transparent' '--enable-follow-x-forwarded-for' '--disable-ecap' '--disable-loadable-modules' '--disable-kqueue' '--with-large-files' '--prefix=/usr/pbi/squid-i386' '--mandir=/usr/pbi/squid-i386/man' '--infodir=/usr/pbi/squid-i386/info/' '--build=i386-portbld-freebsd8.1' 'build_alias=i386-portbld-freebsd8.1' 'CC=cc' 'CFLAGS=-O2 -pipe -I/usr/pbi/squid-i386/include -I/usr/pbi/squid-i386/include  -I/usr/include -DLDAP_DEPRECATED -fno-strict-aliasing' 'LDFLAGS= -L/usr/pbi/squid-i386/lib -L/usr/pbi/squid-i386/lib -rpath=/usr/lib:/usr/pbi/squid-i386/lib -L/usr/lib' 'CPPFLAGS=-I/usr/pbi/squid-i386/include' 'CXX=c++' 'CXXFLAGS=-O2 -pipe -I/usr/pbi/squid-i386/include -I/usr/pbi/squid-i386/include -I/usr/include -DLDAP_DEPRECATED -fno-strict-aliasing' 'CPP=cpp' --with-squid=/usr/wrkdirprefix/usr/ports/www/squid31/work/squid-3.1.20 --enable-ltdl-convenience
                          1 Reply Last reply Reply Quote 0
                          • jimpJ
                            jimp Rebel Alliance Developer Netgate
                            last edited by

                            Great :-)

                            I tested it myself before posting this time so I was sure it was good.

                            That bug affected all PBI builds, so now I get to go back and rebuild every PBI, which will take more than a day to finish.

                            Remember: Upvote with the 👍 button for any user/post you find to be helpful, informative, or deserving of recognition!

                            Need help fast? Netgate Global Support!

                            Do not Chat/PM for help!

                            1 Reply Last reply Reply Quote 0
                            • C
                              Cino
                              last edited by

                              @jimp:

                              Great :-)

                              I tested it myself before posting this time so I was sure it was good.

                              That bug affected all PBI builds, so now I get to go back and rebuild every PBI, which will take more than a day to finish.

                              OUCH!! But glad it was caught now and not down the road.. Snort and dansguardian; i'm pretty sure have custom build options.

                              1 Reply Last reply Reply Quote 0
                              • jimpJ
                                jimp Rebel Alliance Developer Netgate
                                last edited by

                                Yes I have a list of them (easy to spot in the pkg xml), but it affected all of them not just the ones with build options.

                                Remember: Upvote with the 👍 button for any user/post you find to be helpful, informative, or deserving of recognition!

                                Need help fast? Netgate Global Support!

                                Do not Chat/PM for help!

                                1 Reply Last reply Reply Quote 0
                                • P
                                  phil.davis
                                  last edited by

                                  I deleted squid3, then upgraded to:
                                  2.1-BETA0 (i386)
                                  built on Wed Jun 20 18:13:24 EDT 2012
                                  FreeBSD 8.3-RELEASE-p3

                                  Installed squid3. It installs well and starts up without any manual intervention - thanks Jim. A basic config is running in transparent mode. During the install it tries to start squid a couple of times beofre it has actually created the squid.conf file, but gets it right in the end. Just a couple of messages appear in the system log that don't look good to the uninitiated:

                                  Jun 21 10:28:39 	php: /pkg_mgr_install.php: Beginning package installation for squid3 .
Jun 21 04:51:46 	check_reload_status: Syncing firewall
Jun 21 10:36:52 	php: /pkg_mgr_install.php: Stopping any running proxy monitors
Jun 21 10:36:53 	php: /pkg_mgr_install.php: Starting Squid
Jun 21 10:36:53 	php: /pkg_mgr_install.php: Starting a proxy monitor script
Jun 21 10:36:53 	squid: Unable to open configuration file: /usr/local/etc/squid/squid.conf: (2) No such file or directory
Jun 21 04:51:53 	check_reload_status: Reloading filter
Jun 21 10:36:58 	Squid_Alarm[58777]: Squid has exited. Reconfiguring filter.
Jun 21 10:36:58 	Squid_Alarm[59433]: Attempting restart...
Jun 21 10:36:59 	squid: Unable to open configuration file: /usr/local/etc/squid/squid.conf: (2) No such file or directory
Jun 21 10:37:02 	Squid_Alarm[62052]: Reconfiguring filter...
Jun 21 04:52:03 	check_reload_status: Syncing firewall
Jun 21 10:37:04 	php: /pkg_mgr_install.php: Creating squid log dir /var/squid/logs/
Jun 21 04:52:04 	check_reload_status: Reloading filter
Jun 21 10:37:04 	php: /pkg_mgr_install.php: Starting Squid
Jun 21 10:37:04 	squid[1545]: Squid Parent: child process 2139 started

                                  Now I will try moving the conf file into /var filesystem and see how squidguard runs on top of this.

                                  As the Greek philosopher Isosceles used to say, "There are 3 sides to every triangle."
                                  If I helped you, then help someone else - buy someone a gift from the INF catalog http://secure.inf.org/gifts/usd/

                                  1 Reply Last reply Reply Quote 0
                                  • P
                                    phil.davis
                                    last edited by

                                    squidguard-1.4_4-i386 has installed fine on top of squid3 and is happily blocking sites for me on a timed basis.
                                    I'll post an update about the Time-based Restriction stuff on the post about that at:
                                    http://forum.pfsense.org/index.php/topic,43352.15.html
                                    From the command line, pbi_info shows that squidguard-1.4_4-i386 is installed.
                                    But the Installed Packages GUI page shows 1.4_2
                                    It's a bit difficult to keep these version numbers in synch when different pfSense releases are using different versions of a package!
                                    Jim, thanks for all the work on PBIs - at least squid3 + squiguard on 2.1 is looking good.

                                    As the Greek philosopher Isosceles used to say, "There are 3 sides to every triangle."
                                    If I helped you, then help someone else - buy someone a gift from the INF catalog http://secure.inf.org/gifts/usd/

                                    1 Reply Last reply Reply Quote 0
                                    • First post
                                      Last post
                                    Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.