@viragomann said in Help with getting second server working with haproxy: @vMAC said in Help with getting second server working with haproxy: Sometimes I get a 503 error, and other times I get a Redirected Too Many times error. I'd consider these as different issues. HAproxy give 503 if the backend state is offline or the backend does not respond as expected. So first ensure, that HAproxy shows the backend as online in the stats. I'd switch over to basic health check for testing. However, "redirected to many times" might come from the browser. Best you use the browsers debugging mode to investigate, what's going on here. Got it, so here is what I found. Truenas has a Http -> Https redirect built into settings. I had it checked, unchecking it has not stopped the too many redirects, and looks to have resolved my original issue. Thank you! I am now trying to set one up for my Unifi Cloud Controller though and it is giving me a TLS mismatch error as I am trying to redirect to a 8443 port? Bad Request This combination of host and port requires TLS.

Store ID program: I am using the built in program attached here.. /usr/local/libexec/squid/storeid_file_rewrite #!/usr/local/bin/perl use strict; use warnings; use Pod::Usage; =pod =head1 NAME storeid_file_rewrite - File based Store-ID helper for Squid =head1 SYNOPSIS storeid_file_rewrite filepath =head1 DESCRIPTION This program acts as a store_id helper program, rewriting URLs passed by Squid into storage-ids that can be used to achieve better caching for websites that use different URLs for the same content. It takes a text file with two tab separated columns. Column 1: Regular expression to match against the URL Column 2: Rewrite rule to generate a Store-ID Eg: ^http:\/\/[^\.]+\.dl\.sourceforge\.net\/(.*) http://dl.sourceforge.net.squid.internal/$1 Rewrite rules are matched in the same order as they appear in the rules file. So for best performance, sort it in order of frequency of occurrence. This program will automatically detect the existence of a concurrency channel-ID and adjust appropriately. It may be used with any value 0 or above for the store_id_children concurrency= parameter. =head1 OPTIONS The only command line parameter this helper takes is the regex rules file name. =head1 AUTHOR This program and documentation was written by I<Alan Mizrahi <alan@mizrahi.com.ve>> Based on prior work by I<Eliezer Croitoru <eliezer@ngtech.co.il>> =head1 COPYRIGHT * Copyright (C) 1996-2023 The Squid Software Foundation and contributors * * Squid software is distributed under GPLv2+ license and includes * contributions from numerous individuals and organizations. * Please see the COPYING and CONTRIBUTORS files for details. Copyright (C) 2013 Alan Mizrahi <alan@mizrahi.com.ve> Based on code from Eliezer Croitoru <eliezer@ngtech.co.il> This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with this program; if not, write to the Free Software Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307, USA. =head1 QUESTIONS Questions on the usage of this program can be sent to the I<Squid Users mailing list <squid-users@lists.squid-cache.org>> =head1 REPORTING BUGS Bug reports need to be made in English. See http://wiki.squid-cache.org/SquidFaq/BugReporting for details of what you need to include with your bug report. Report bugs or bug fixes using http://bugs.squid-cache.org/ Report serious security bugs to I<Squid Bugs <squid-bugs@lists.squid-cache.org>> Report ideas for new improvements to the I<Squid Developers mailing list <squid-dev@lists.squid-cache.org>> =head1 SEE ALSO squid (8), GPL (7), The Squid wiki http://wiki.squid-cache.org/Features/StoreID The Squid Configuration Manual http://www.squid-cache.org/Doc/config/ =cut my @rules; # array of [regex, replacement string] die "Usage: $0 <rewrite-file>\n" unless $#ARGV == 0; # read config file open RULES, $ARGV[0] or die "Error opening $ARGV[0]: $!"; while (<RULES>) { chomp; next if /^\s*#?$/; if (/^\s*([^\t]+?)\s*\t+\s*([^\t]+?)\s*$/) { push(@rules, [qr/$1/, $2]); } else { print STDERR "$0: Parse error in $ARGV[0] (line $.)\n"; } } close RULES; $|=1; # read urls from squid and do the replacement URL: while (<STDIN>) { chomp; last if $_ eq 'quit'; my $channel = ""; if (s/^(\d+\s+)//o) { $channel = $1; } foreach my $rule (@rules) { if (my @match = /$rule->[0]/) { $_ = $rule->[1]; for (my $i=1; $i<=scalar(@match); $i++) { s/\$$i/$match[$i-1]/g; } print $channel, "OK store-id=$_\n"; next URL; } } print $channel, "ERR\n"; }

To Quote Squid Email Support "'On 1/01/25 21:21, Robin Wood wrote: I'm going to massively over simplify things here, but you can think of it like this. Files with html extensions are static web pages, you write them, put them on the server, and they are served as they are, no changes. Asp and the others are dynamic files, they are processed by an app on the server before they are sent to the client. This app may do nothing, so the page comes as it was, but usually it will add content. This content could be to create a CMS page by pulling the page content from a database, it could be your shopping orders pulled from your account, or it could be your current bank statement. Caching should never be done on anything that is specific to a single user, so it's fine to cache public CMS content with an asp extension, but not your bank statement. There is more to it than that, but hopefully that gives you a general idea.' That is mostly correct for simple HTTP/1.0-like behaviour. With HTTP/1.1 and later things are a little different. The biggest change is that URL no longer matters. The Content-Typereplaces "fiel extension" entirely, and Cache-Control headers take over the job of defining how and when something can be cached. For Squid, the refresh_pattern directive is what provides compatibility with HTTP 1.0 behaviour. It provides values for any Cache-Control settings the server omitted (eg for servers acting like HTTP/1.0 still). The default "refresh_pattern -i (/cgi-bin/|?) 0 0% 0" configuration line tells Squid the values which will perform HTTP/1.0 caching behaviour for any of the dynamic content coming out of broken or old cgi-bin services or anythign with query-string ('?...') URL. Jonathan: if you have not changed the refresh_pattern's you do not have to care specifically about dynamic-vs-static content caching. Whether it is plain-text HTTP(S) or SSL-Bump'ed HTTPS, it should all cache properly for its server-claimed needs. Your "cache deny" policy in squid.conf is telling Squid never to cache any URL containing the ACL-matching strings. Even if they could be cached safely. HTH Amos" Basically you do not really need to add this weird rule per the email support.. if you have not changed the refresh_pattern's you do not have to care specifically about dynamic-vs-static content caching. Whether it is plain-text HTTP(S) or SSL-Bump'ed HTTPS, it should all cache properly for its server-claimed needs. I do not know if anyone else wonders about this and stumbled on that rule on several different websites, it is not built into the Squid Package but it seems like something others have been adding.

@kennethg01 Did you notice, that the real clients IP is only sent to the backend server as value of the "X-Forwarded-For" header? You have to configure your web server to log this header, since this is not done by default.

@michmoor You ever configure Squid with http tproxy ? I tested this it was amazing again any reboot or enable reset puts it back to the old way.

@CreationGuy What did you try? How did you access the server? From inside your network or from outside? Which URL? What exactly did you get?

I managed to resolve the problem by removing the frontend name. After making this change, everything started working normally. Updated Frontend Configuration: mode tcp bind *:443 timeout client 30s use_backend k8s-ssl-pass-thru By simplifying the configuration and removing the unnecessary frontend name, the setup became functional. If anyone else is facing similar issues, I recommend checking if any redundant configuration elements can be removed.

@JeGr Many thanks. I had performed the upgrade on a SG4680 and 6100 and still got the elf error (no CE on prod). I'll try the upgrade over the weekend to 24.11 and check whether I see the same library problem on these and the fallback machines.

I use ssl intercept and it does scan https traffic. With protocols like doh, dns over https, pfblocking is just wackamole. Squid a pain to configure with ssl intercept but it works great once it is configured. ClamAV is a pain when it updates, it hogs resources. So I use cron and it updates in the early hours

@lg1980 said in New Squid 6.7 and Clamav 1.3.0: https://git.labexposed.com/lgcosta/gists/src/branch/main/squid-6x Hi I hope you are doing well. I have reinstall pfsense OS ,i need to reconfigure squid Proxy, I am unable to download pakage from above github link.Can you share the new repo link.

ahh my trouble is with one specefic server. This worked with other ones. Thanks!

I found that lancache is better at caching steam and windows updates than squid. Though you can setup squid to cache these updates. Best way to do it is the following install squid and set it up and add refresh pattens https://github.com/mmd123/squid-cache-dynamic_refresh-list Configure all clients to use the proxy manually or setup pfsense to use a WPAD to do it automatically. For software that does not support autoconfigure proxy enable transparent proxy, do not rely on only the transparent proxy as it can break things. Enable transparent SSL and under SSL/MITM Mode either select spliceall or if you want to cache some ssl select custom 4a. under Custom Options (SSL/MITM) here you can create your squid rule, for an example if you do the follow Create a txt file at /home/bumpsites.txt /home/excludeSites.txt acl bump_sites ssl::server_name "/home/bumpsites.txt" acl excludeSites ssl::server_name "/home/excludeSites.txt" acl step1 at_step SslBump1 ssl_bump peek step1 ssl_bump splice bypassusers ssl_bump bump bump_sites ssl_bump splice all the bumpsites.txt are all the sites you want do decrypt so you can cache it, an example will be like this download.nvidia.com us.download.nvidia.com international-gfe.download.nvidia.com This will bump the nvidia driver url and will allow you to cache the update While it may seem nice to bump and decrypt everything sadly that breaks a lot of things and not everything can be cached. So the best option is to see what are the biggest download urls on your network, first see if you can are able to decrypt and cache it without any issues then add it to the to the list and restart squid. Play around with it and let me know how you go.

https://redmine.pfsense.org/issues/15911

@tiago-duarte Either white list all skype domains or switch to use splice mode and manually bump sites that you want to decrypt also try manually setting devices to use the proxy then have transparent as a fallback.