Minor Issues on 24.08

JonathanLee

@DefenderLLC Yeahhh I absolutely love Squid it’s working perfectly it just needs some GUI changes to status php pages is all 6.6 runs 100x faster over 5.8

DefenderLLC

@JonathanLee said in Minor Issues on 24.08:

@DefenderLLC Yeahhh I absolutely love Squid it’s working perfectly it just needs some GUI changes to status php pages is all 6.6 runs 100x faster over 5.8

Sorry, I thought you were talking about Snort. Yes, the Squid widget works fine for me in this version.

JonathanLee

@DefenderLLC YEAHHH They fixed the update dates on it also check it out it has the correct dates on it now

JonathanLee

@DefenderLLC Does it have the correct status page again on the GUI ? The new Squid 6 uses a different status page now and mgr info doesn't work anymore, other than the gui status page it should work fine

I am so happy 6.10 is part of pfSense 24.08

some small changes are made in Squid 6

cache_object://URL Scheme is not removed in Squid-6
cache_object://hostname/request@password

Is used to check status inside the pfSense GUI for this package

It has been replaced with
http://hostname/squid-internal-mgr URL-prefix.

Therefore it will now say access denied when checking status

The status page needs to be updated to reflect the new URL scheme

I spoke with Squid user support and they have confirmed the new URL scheme

Ref:
https://wiki.squid-cache.org/Features/CacheManager/CacheObjectScheme

stephenw10

It's because it sees it as a forwarding loop:

    current master transaction: master97
2024/07/04 16:30:16 kid1| WARNING: Forwarding loop detected for:
GET /squid-internal-mgr/info HTTP/1.1
Host: localhost:3128
User-Agent: squidclient/6.10
Accept: */*
Via: 1.0 localhost (squid/6.10)
X-Forwarded-For: 127.0.0.1
Cache-Control: max-age=259200
Connection: keep-alive

And the reason it does is that it doesn't recognise 127.0.0.1 as the proxy name:

[24.08-DEVELOPMENT][admin@6100.stevew.lan]/root: squidclient -l 127.0.0.1 -h 127.0.0.1 mgr:info
HTTP/1.1 403 Forbidden
Server: squid/6.10

But if you use 'localhost' it resolves it as IPv6 and Squid doesn't listen on v6:

[24.08-DEVELOPMENT][admin@6100.stevew.lan]/root: squidclient -l 127.0.0.1 -h localhost mgr:info
ERROR: Cannot connect to [::1]:3128

But if you set pfSense to prefer IPv4 in Sys > Adv > Net it then works:

[24.08-DEVELOPMENT][admin@6100.stevew.lan]/root: squidclient -l 127.0.0.1 -h localhost mgr:info
HTTP/1.1 200 OK
Server: squid/6.10
Mime-Version: 1.0

See: https://bugs.squid-cache.org/show_bug.cgi?id=5283

JonathanLee

I am using Squid 6.6 in 24, I noticed http_port tproxy directive has different requests versus 5.8, in 5.8 the tproxy option worked. Also it is missing the latest squid file system on the package GUI. Named “Rock cache_dirs”

Background on squid proprietary file systems:
Original was …

COSS “The Cyclic Object Storage Scheme (costs) is an attempt to develop a custom file system for Squid.”

Ref: squid the definite guide Duane Wessels

new version: per developer response

“Squid does not support COSS cache_dirs since v3.5. If Squid in question does disk caching, then rock cache_dirs may be the best bet.

Alex.”

Rock is not listed

====  The rock store type  ====
Usage:
        cache_dir rock Directory-Name Mbytes [options]
The Rock Store type is a database-style storage. All cached
    entries are stored in a "database" file, using fixed-size slots.
    A single entry occupies one or more slots.
If possible, Squid using Rock Store creates a dedicated kid
    process called "disker" to avoid blocking Squid worker(s) on disk
    I/O. One disker kid is created for each rock cache_dir.  Diskers
    are created only when Squid, running in daemon mode, has support
    for the IpcIo disk I/O module.
swap-timeout=msec: Squid will not start writing a miss to or
    reading a hit from disk if it estimates that the swap operation
    will take more than the specified number of milliseconds. By
    default and when set to zero, disables the disk I/O time limit
    enforcement. Ignored when using blocking I/O module because
    blocking synchronous I/O does not allow Squid to estimate the
    expected swap wait time.
max-swap-rate=swaps/sec: Artificially limits disk access using
    the specified I/O rate limit. Swap out requests that
    would cause the average I/O rate to exceed the limit are
    delayed. Individual swap in requests (i.e., hits or reads) are
    not delayed, but they do contribute to measured swap rate and
    since they are placed in the same FIFO queue as swap out
    requests, they may wait longer if max-swap-rate is smaller.
    This is necessary on file systems that buffer "too
    many" writes and then start blocking Squid and other processes
    while committing those writes to disk.  Usually used together
    with swap-timeout to avoid excessive delays and queue overflows
    when disk demand exceeds available disk "bandwidth". By default
    and when set to zero, disables the disk I/O rate limit
    enforcement. Currently supported by IpcIo module only.
slot-size=bytes: The size of a database "record" used for
    storing cached responses. A cached response occupies at least
    one slot and all database I/O is done using individual slots so
    increasing this parameter leads to more disk space waste while
    decreasing it leads to more disk I/O overheads. Should be a
    multiple of your operating system I/O page size. Defaults to
    16KBytes. A housekeeping header is stored with each slot and
    smaller slot-sizes will be rejected. The header is smaller than
    100 bytes.
==== COMMON OPTIONS ====
no-store    no new objects should be stored to this cache_dir.
min-size=n    the minimum object size in bytes this cache_dir
            will accept.  It's used to restrict a cache_dir
            to only store large objects (e.g. AUFS) while
            other stores are optimized for smaller objects
            (e.g. Rock).
            Defaults to 0.
max-size=n    the maximum object size in bytes this cache_dir
            supports.
            The value in maximum_object_size directive sets
            the default unless more specific details are
            available (ie a small store capacity).
Note: To make optimal use of the max-size limits you should order
    Note: To make optimal use of the max-size limits you should order
        the cache_dir lines with the smallest max-size value first.

Again the tproxy directive worked when you adapted the squid.conf in 5.8 it would work with squid -k parse and in 6.6 if you change intercept to tproxy it will not parse with squid -k parse command it states it must be added with the squid built now, simple one line command.

2024/07/17 21:22:41| Processing Configuration File: /usr/local/etc/squid/squid.conf (depth 0)
2024/07/17 21:22:41| Processing: http_port 192.168.1.1:3128 ssl-bump generate-host-certificates=on dynamic_cert_mem_cache_size=20MB cert=/usr/local/etc/squid/serverkey.pem cafile=/usr/local/share/certs/ca-root-nss.crt capath=/usr/local/share/certs/ cipher=EECDH+ECDSA+AESGCM:EECDH+aRSA+AESGCM:EECDH+ECDSA+SHA384:EECDH+ECDSA+SHA256:EECDH+aRSA+SHA384:EECDH+aRSA+SHA256:EECDH+aRSA+RC4:EECDH:EDH+aRSA:HIGH:!RC4:!aNULL:!eNULL:!LOW:!3DES:!MD5:!EXP:!PSK:!SRP:!DSS tls-dh=prime256v1:/etc/dh-parameters.2048 options=NO_SSLv3,SINGLE_DH_USE,SINGLE_ECDH_USE
2024/07/17 21:22:41| WARNING: UPGRADE: 'cafile=/usr/local/share/certs/ca-root-nss.crt' is deprecated in http_port. Use 'tls-cafile=' instead.
2024/07/17 21:22:41| WARNING: Failed to decode EC parameters '/etc/dh-parameters.2048'
    OpenSSL-saved error #1: 0x1e08010c
2024/07/17 21:22:41| ERROR: Unsupported TLS option SINGLE_DH_USE
2024/07/17 21:22:41| ERROR: Unsupported TLS option SINGLE_ECDH_USE
2024/07/17 21:22:41| Processing: http_port 127.0.0.1:3128 tproxy ssl-bump generate-host-certificates=on dynamic_cert_mem_cache_size=20MB cert=/usr/local/etc/squid/serverkey.pem cafile=/usr/local/share/certs/ca-root-nss.crt capath=/usr/local/share/certs/ cipher=EECDH+ECDSA+AESGCM:EECDH+aRSA+AESGCM:EECDH+ECDSA+SHA384:EECDH+ECDSA+SHA256:EECDH+aRSA+SHA384:EECDH+aRSA+SHA256:EECDH+aRSA+RC4:EECDH:EDH+aRSA:HIGH:!RC4:!aNULL:!eNULL:!LOW:!3DES:!MD5:!EXP:!PSK:!SRP:!DSS tls-dh=prime256v1:/etc/dh-parameters.2048 options=NO_SSLv3,SINGLE_DH_USE,SINGLE_ECDH_USE

This error never occurred for 5.8
**---->2024/07/17 21:22:41| ERROR: configuration failure: requires TPROXY feature to be enabled by ./configure**


    exception location: Intercept.cc(163) StartTransparency
2024/07/17 21:22:41| Not currently OK to rewrite swap log.
2024/07/17 21:22:41| storeDirWriteCleanLogs: Operation aborted.
2024/07/17 21:22:41| FATAL: Bungled /usr/local/etc/squid/squid.conf line 6: http_port 127.0.0.1:3128 tproxy ssl-bump generate-host-certificates=on dynamic_cert_mem_cache_size=20MB cert=/usr/local/etc/squid/serverkey.pem cafile=/usr/local/share/certs/ca-root-nss.crt capath=/usr/local/share/certs/ cipher=EECDH+ECDSA+AESGCM:EECDH+aRSA+AESGCM:EECDH+ECDSA+SHA384:EECDH+ECDSA+SHA256:EECDH+aRSA+SHA384:EECDH+aRSA+SHA256:EECDH+aRSA+RC4:EECDH:EDH+aRSA:HIGH:!RC4:!aNULL:!eNULL:!LOW:!3DES:!MD5:!EXP:!PSK:!SRP:!DSS tls-dh=prime256v1:/etc/dh-parameters.2048 options=NO_SSLv3,SINGLE_DH_USE,SINGLE_ECDH_USE
2024/07/17 21:22:41| Squid Cache (Version 6.6): Terminated abnormally.
CPU Usage: 0.073 seconds = 0.049 user + 0.024 sys
Maximum Resident Size: 62512 KB
Page faults with physical i/o: 0

Both minor issues to fix within the php menu squid rock is the latest best filesystem for caching I am told.

JonathanLee

@stephenw10 also you can add a password to access squid manager that is a good security feature too.

JonathanLee

@stephenw10

I use the password directive to further protect access to Squid manager I was able to adapt the mgr:info@passwordhere on the other version however on this version I can't. I also have it set to prefer ipv4

Screenshot 2024-07-17 at 20.55.19.png

JonathanLee

Screenshot 2024-07-17 at 20.57.27.png

I wonder where I put the password now

under my advanced config I have set...

cachemgr_passwd disable offline_toggle reconfigure shutdown
cachemgr_passwd REDACTED_PASSWORD_HERE all

JonathanLee

@stephenw10

I have got nothing on how to resolve the password directive that i utilize for added security

squidclient -l 127.0.0.1 -p 3128 -h localhost -U cachemgr -W REDACTED mgr:info
HTTP/1.1 403 Forbidden
Server: squid
Mime-Version: 1.0
Date: Thu, 18 Jul 2024 04:13:36 GMT
Content-Type: text/html;charset=utf-8
Content-Length: 3792
X-Squid-Error: ERR_ACCESS_DENIED 0
Vary: Accept-Language
Content-Language: en
Cache-Status: Lee_Family.home.arpa
Cache-Status: Lee_Family.home.arpa;detail=no-cache
Connection: close

Also if you use no username

squidclient -l 127.0.0.1 -p 3128 -h localhost -w redacted mgr:info
ERROR: Proxy authentication password (-w) is given, but username (-u) is missing

With password removed also I get the error

Shell Output - squidclient -l 127.0.0.1 -h localhost mgr:info
HTTP/1.1 403 Forbidden
Server: squid
Mime-Version: 1.0
Date: Thu, 18 Jul 2024 04:35:01 GMT
Content-Type: text/html;charset=utf-8
Content-Length: 3719
X-Squid-Error: ERR_ACCESS_DENIED 0
Vary: Accept-Language
Content-Language: en
Cache-Status: Lee_Family.home.arpa
Cache-Status: Lee_Family.home.arpa;detail=no-cache
Connection: close

function squid_status() {
	if (is_service_running('squid')) {
		init_config_arr(array('installedpackages', 'squidcache','config'));
		$proxy_ifaces = explode(",", config_get_path('installedpackages/squid/config/0/active_interface', ''));
		foreach ($proxy_ifaces as $iface) {
			if (get_interface_ip($iface)) {
				$ip = get_interface_ip($iface);
				$lip = '127.0.0.1';
			} else {
				$ip = get_interface_ipv6($iface);
				$lip = '::1';
			}
			exec("/usr/local/sbin/squidclient -l " . escapeshellarg($lip) .
				" -h " . escapeshellarg($ip) . " mgr:info", $result);
		}
	} else {
		return(gettext('Squid Proxy is not running.'));

Per Squid Email Support

"FWIW, I do not recommend using cachemgr.cgi and squidclient. For various reasons, both were recently removed from Squid master/v7. Squidclient can be replaced with curl or wget. The best cachemgr.cgi replacement depends on many factors; a static HTML file may be the best solution in some cases!

Without squidclient, you will need to use absolute URLs like this one:

http://correct-host-name-or-ip:port/squid-internal-mgr/info

See recent discussions on this mailing list or Bug 5283 for discussions about what correct-host-name-or-ip:port to use for these URLs. And if you got it working with squidclient, you can see what your squidclient was sending, of course.
https://bugs.squid-cache.org/show_bug.cgi?id=5283"

Per the bug

curl http://localhost:3128/squid-internal-mgr/info

is the future of squidclient ...

DefenderLLC

I guess we're getting to close to the GA release. I have not seen a single development snapshot since July 2nd.

24.08.a.20240702.0600

grandrivers

@DefenderLLC unless we are hung on our current version I have seen that happen before , but very little transparency in closed "plus" version

marcosm

There are a number of reasons why development snapshots may pause. For example, there could be some stuff coming to 24.08 that we may not want to preview just yet ;)

danjeman

@marcosm from the July newsletter maybe?

DefenderLLC

@danjeman That was my first thought too...