Squid Tax

[JonathanLee]

Hello fellow Netgate community members,

DNS-based filtering is a house of cards compared to a properly tuned Squid deployment. While others traded their trust model for a DNS provider’s convenience, staying with the Proxy model (especially with Squid 6/7's asynchronous improvements and HTTP/3 stabs) allows for a true security perimeter. If you can handle the 'Squid tax' and master container-level tagging, you’ve built a private intelligence agency for your network rather than just checking someone else's list. Is anyone else here finding that modern high-core CPUs have finally made 10Gbps SSL Bumping viable, or is the overhead still the 'final boss' of your config?

[KOM]

@JonathanLee Nobody uses Squid properly for the same reason nobody uses HFSC properly: they're difficult to understand and implement, the docs suck, and they don't quite work properly out of the box. I've read your Squid posts over the years and I'm fairly confident that you are the premier authority on Squid here. I suspect you likely know more about it than anyone at Netgate, that's for sure.

[tinfoilmatt]

All information systems are houses of cards if not actively maintained. (The same could be said about life in general.) Defense-in-depth doesn't necessarily prioritize technical controls. DNSBL and DPI combined is better than only one or the other.

[luckman212]

@JonathanLee Got any tips for those of us not so well-versed in Squid on where to begin setting this up properly? What sort of costs or licenses are involved? The last time I installed Squid was on a Netgate 3100, which went about as well as you might expect.

[tinfoilmatt]

Crunching some resources over the past 24 hours and these stand out:

https://www.squid-cache.org/
https://github.com/squid-cache/squid
https://wiki.squid-cache.org/ (https://github.com/squid-cache/squid-cache.github.io)
https://ml-archives.squid-cache.org/squid-users/

Obviously none of this will help directly with pfSense integration specifically. But the tax!

[JonathanLee]

@luckman212 I hope this helps. This took many many years, two plus official textbooks, many emails to developers, reading everything I could on Squids webpage, plus lots of research on the Netgate forums, help from Google, and ChatGPT and Claude too towards the end. It functions so good for me again you would have to make your own custom DNS files etc for it to work. This will be 3 replies:

Before Auth

# =============================================================================
# Custom Options (Before Auth) — squid.conf
#
# This block runs before Squid's authentication stack. It configures:
#   - SSL bump / HTTPS inspection with modern cipher suites
#   - Dual-stack IPv4/IPv6 outbound routing
#   - Cache tuning, Store-ID rewriting, and QoS markings
#   - Hard blocks for DNS-over-HTTPS, private nets, and off-hours access
#   - Header sanitization to prevent proxy metadata leakage
#
# Place this content in the "Custom Options (Before Auth)" field in the
# pfSense Squid package UI (Services > Squid Proxy Server > General).
# =============================================================================


# --- SSL Bump Listener ---
# Enables HTTPS inspection on the IPv6 listener. Generates certificates on the
# fly using your local CA. Enforces strong ECDH-only cipher suites and disables
# SSLv3. The tls-dh line sets the DH group used for ephemeral key exchange.
#
#http_port [2001:db8::1]:3128 ssl-bump \
#  generate-host-certificates=on \
#  dynamic_cert_mem_cache_size=48MB \
#  cert=/usr/local/etc/squid/serverkey.pem \
#  tls-cafile=/usr/local/share/certs/ca-root-nss.crt \
#  capath=/usr/local/share/certs/ \
#  cipher=EECDH+ECDSA+AESGCM:EECDH+aRSA+AESGCM:EECDH+ECDSA+SHA384:EECDH+ECDSA+SHA256:\
#         EECDH+aRSA+SHA384:EECDH+aRSA+SHA256:EECDH+aRSA+RC4:EECDH:EDH+aRSA:\
#         HIGH:!RC4:!aNULL:!eNULL:!LOW:!3DES:!MD5:!EXP:!PSK:!SRP:!DSS \
#  tls-dh=prime256v1:/etc/dh-parameters.2048 \
#  options=NO_SSLv3


# --- Alternate NVMe Cache Disk (Worker 2 only) ---
# Pins a dedicated diskd cache dir to a second worker process. Useful if you
# have a fast NVMe/Optane device to dedicate to cache I/O on multi-worker setups.
#
#if ${process_number} = 2
#cache_dir diskd /path/to/nvme/squid_cache_b 32000 64 256
#endif


# --- Custom Log Format ---
# Extends the default 'squid' log format to include the client MAC address
# (%>eui) via Squid's EUI-48/64 lookup. Useful for per-device traffic forensics
# without needing RADIUS or DHCP correlation.
#
logformat squid %ts.%03tu %6tr %>eui %Ss/%03>Hs %<st %rm %ru %[un %Sh/%<a %mt


# --- TCP Half-Close Behavior ---
# Tells Squid not to keep half-closed TCP connections alive. Reduces stale
# connection accumulation under load.
#
half_closed_clients off


# --- Off-Hours Block ---
# Hard-denies all traffic during a maintenance window (00:30–05:00).
# TCP_RESET terminates the connection immediately rather than returning an
# HTTP error page, which is cleaner for automated clients.
#
acl block_hours time 00:30-05:00
http_access deny all block_hours
deny_info TCP_RESET block_hours


# --- Disable Unused Peer Protocols ---
# Shuts off ICP, HTCP, and SNMP listeners. None are needed in a single-proxy
# deployment and leaving them open is unnecessary attack surface.
#
icp_port 0
htcp_port 0
snmp_port 0
icp_access deny all
htcp_access deny all
snmp_access deny all


# --- Method and Address Family ACLs ---
# Used downstream to gate Store-ID rewriting (GET-only) and to control which
# outbound interface is selected based on IPv4 vs IPv6 destination.
#
acl getmethod method GET
acl to_ipv4 dst ipv4
acl to_ipv6 dst ipv6


# --- TLS Outgoing Options (Modern / Safari-Compatible) ---
# Controls how Squid opens TLS connections to origin servers.
# - Disables SSLv3, TLS 1.0, and TLS 1.1 (obsolete, vulnerable).
# - Disables TLS session tickets (NO_TICKET) to avoid session resumption
#   fingerprinting and to enforce full forward secrecy on every connection.
# - Cipher list prefers ChaCha20-Poly1305 (better on ARM/mobile) then
#   AES-GCM suites. All are ECDHE — no static RSA key exchange.
# - default-ca=on trusts the system CA bundle for outgoing connections.
#
# (Older SafeXcel AES-CBC optimized profile kept below for reference)
#tls_outgoing_options options=NO_SSLv3,NO_TLSv1,NO_TLSv1_1,NO_TLSv1_3,NO_TICKET
#tls_outgoing_options min-version=1.2
#tls_outgoing_options cipher=ECDHE-RSA-AES256-SHA384:ECDHE-RSA-AES128-SHA256:...
#tls_outgoing_options default-ca=on

tls_outgoing_options options=NO_SSLv3,NO_TLSv1,NO_TLSv1_1,NO_TICKET
tls_outgoing_options min-version=1.2
tls_outgoing_options cipher=ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384
tls_outgoing_options default-ca=on


# --- Proxy Header Sanitization ---
# Strips headers that could reveal to origin servers that a proxy is in the
# path, leak client IP via X-Forwarded-For, or expose proxy credentials.
# Prevents WPAD/PAC auto-config headers from being forwarded upstream.
#
request_header_access Proxy-Auto-Config deny all
request_header_access Proxy-Authorization deny all
request_header_access Proxy-Authenticate deny all
request_header_access Proxy-Connection deny all
request_header_access X-Forwarded-For deny all


# --- Domain List ACLs ---
# Loads domain lists from flat files managed outside squid.conf.
# HttpAccess: domains always sent direct (bypass cache).
# windowsupdate: Microsoft update CDN domains, given special cache/range treatment.
# AlwaysDirect: additional domains that must not be bumped or cached.
#
acl HttpAccess dstdomain "/usr/local/pkg/http.access"
acl windowsupdate dstdomain "/usr/local/pkg/windowsupdate"
acl AlwaysDirect dstdomain "/usr/local/pkg/always_direct"


# --- Store-ID Rewriting ---
# Normalizes cache keys for domains that serve the same content from many URLs
# (e.g. CDNs with rotating hostnames). The rewrite program maps messy URLs to
# a stable Store-ID so Squid can serve one cached copy across variants.
# - GET-only (deny CONNECT and non-GET) since only cacheable requests matter.
# - 32 workers with startup/idle tuning to handle burst rewrite requests.
#
acl rewritedoms dstdomain "/usr/local/pkg/desdom"
store_id_program /usr/local/libexec/squid/storeid_file_rewrite /var/squid/storeid/storeid_rewrite.txt
store_id_children 32 startup=8 idle=4 concurrency=0
store_id_access deny CONNECT
store_id_access deny !getmethod
store_id_access allow rewritedoms
store_id_access deny all


# --- Direct Connection Rules ---
# Bypasses the cache for domains that should never be proxied or cached.
# Local router and WPAD host are always direct to prevent routing loops.
# HttpAccess and AlwaysDirect domains skip the cache entirely.
#
always_direct allow HttpAccess
always_direct allow AlwaysDirect


# --- Cache Freshness Tuning ---
# refresh_all_ims off: don't revalidate all If-Modified-Since requests.
# reload_into_ims on: convert client Cache-Control: no-cache into IMS revalidation
#   rather than a full re-fetch — reduces origin load.
# max_stale 1 day: serve stale objects up to 1 day if origin is unreachable.
# minimum_expiry_time 0: allow objects with a 0-second expiry to still be cached.
# host_verify_strict on: reject responses where the Host header doesn't match
#   the requested destination (SSRF/rebinding mitigation).
#
refresh_all_ims off
reload_into_ims on
max_stale 1 day
minimum_expiry_time 0
host_verify_strict on


# --- QoS / DSCP Marking ---
# Marks locally-served cache hits with DSCP 0x30 (CS6 / network control class).
# Allows downstream QoS (HFSC/PRIQ on pfSense) to prioritize cache hit traffic
# differently from cache misses that require an origin fetch.
#
qos_flows tos local-hit=0x30
#qos_flows tos miss=0x20


# --- Hardware Crypto Acceleration ---
# Offloads TLS crypto to the kernel devcrypto engine (e.g. SafeXcel on Netgate
# hardware). Reduces CPU overhead for SSL bump at scale.
#
ssl_engine devcrypto


# --- Misc ---
# Suppresses client email address from error pages (privacy).
#
email_err_data off


# --- DNS Resolvers ---
# Directs Squid's internal DNS to local resolvers (Unbound/loopback + LAN
# resolver). Keeps DNS traffic on-box and benefits from local DNSSEC/caching.
#
dns_nameservers 127.0.0.1 ::1 [your-lan-resolver-ipv6]


# --- Per-Destination Outbound Interface Binding ---
# Forces specific source IPs for outbound connections based on destination.
# Netflix ACL: routes Netflix traffic out the WAN IPv4 interface to avoid
#   geo-detection issues from the proxy's IPv6 egress.
# to_local_ipv6: keeps LAN-scoped IPv6 destinations on the local segment.
# to_ipv4/to_ipv6: general interface binding for each address family.
#
acl netflix_domains dstdomain .netflix.com .nflxvideo.net .nflxso.net .nflxext.com .netflix.net
tcp_outgoing_address [wan-ipv4] netflix_domains
acl to_local_ipv6 dst [your-lan-ipv6-prefix]::/64
tcp_outgoing_address [your-lan-ipv6] to_local_ipv6
tcp_outgoing_address [wan-ipv4] to_ipv4
tcp_outgoing_address [wan-ipv6] to_ipv6


# --- Anti-Loop Guards ---
# Prevents Squid from proxying requests back to the router or WPAD host,
# which would cause a routing loop.
#
acl local_router dst [router-ipv4]
acl local_router_v6 dst [router-ipv6]
acl wpad dst [wpad-ipv4]
acl wpad dst [wpad-ipv6]
always_direct allow local_router
always_direct allow local_router_v6
always_direct allow wpad


# --- Network Blocks ---
# Denies proxy access to internal RFC-1918 subnets that clients should never
# be reaching through the proxy (e.g. isolated IoT VLAN, modem subnet).
# Also blocks link-local and ULA IPv6 ranges to prevent SSRF into private space.
#
acl block_nets dst 10.0.0.0/24
acl block_nets dst 192.168.8.0/24
acl block_nets dst [specific-host]
http_access deny block_nets
http_access deny !safeports
acl block_nets_v6 dst fc00::/7
acl block_nets_v6 dst fe80::/10
http_access deny block_nets_v6


# --- CONNECT Tunnel Restrictions ---
# Only allows CONNECT (HTTPS tunneling) to standard SSL ports.
# Exceptions carved out for Windows Update endpoints which use CONNECT
# on non-standard paths, permitted for localnet and localhost only.
#
acl CONNECT method CONNECT
http_access deny CONNECT !sslports
acl wuCONNECT dstdomain www.update.microsoft.com
acl wuCONNECT dstdomain sls.microsoft.com
acl localnet_v6 src [your-lan-ipv6-prefix]::/64
http_access allow CONNECT wuCONNECT localnet
http_access allow CONNECT wuCONNECT localhost
http_access allow CONNECT wuCONNECT localnet_v6
http_access allow CONNECT windowsupdate localnet
http_access allow CONNECT windowsupdate localhost
http_access allow CONNECT windowsupdate localnet_v6
http_access allow CONNECT HttpAccess localnet
http_access allow CONNECT HttpAccess localhost
http_access allow CONNECT HttpAccess localnet_v6


# --- Cache Manager Access ---
# Restricts cachemgr access to localhost only.
# Sensitive operations (shutdown, reconfigure, offline_toggle) are disabled.
# Replace the password below with a strong credential before deploying.
#
http_access allow manager localhost
http_access deny manager
cachemgr_passwd disable offline_toggle reconfigure shutdown
cachemgr_passwd [your-cachemgr-password] all


# --- EUI / MAC Lookup ---
# Enables MAC address resolution for clients on directly-attached segments.
# Combined with the custom logformat above, this logs MAC addresses per request.
#
eui_lookup on


# --- Facebook WebSocket Cache Bypass ---
# Prevents Squid from attempting to cache Facebook's WebSocket upgrade requests,
# which are dynamic/real-time and not cacheable. Avoids connection stalls.
#
acl no_miss url_regex -i gateway\.facebook\.com\/ws\/realtime\?
acl no_miss url_regex -i web-chat-e2ee\.facebook\.com\/ws\/chat


# --- DNS-over-HTTPS (DoH) Blocking ---
# Blocks clients from bypassing your local DNS resolver via DoH.
# Covers both the RFC 8484 URL path patterns and the response MIME types
# used by DoH providers (application/dns-message, text/dns, etc).
# Clients attempting DoH will have the request denied at the proxy layer.
#
acl deny_rep_mime_doh rep_mime_type application/dns-message
acl deny_rep_mime_doh rep_mime_type text/dns
acl deny_rep_mime_doh rep_mime_type application/dns+json
http_reply_access deny deny_rep_mime_doh
acl doh_rfc8484 urlpath_regex -i ^/dns-query
acl doh_rfc8484 urlpath_regex -i dns=
acl doh_rfc8484 urlpath_regex -i ^/resolve
acl doh_group any-of deny_rep_mime_doh doh_rfc8484
http_access deny doh_rfc8484


# --- Trusted Servers with Certificate Quirks ---
# Some internal or legacy servers have cert domain mismatches but are known-safe.
# This allows the mismatch error specifically for those domains while still
# denying all other cert errors (expired, untrusted CA, etc).
#
acl BrokenButTrustedServers dstdomain "/usr/local/pkg/dstdom.broken"
acl DomainMismatch ssl_error SQUID_X509_V_ERR_DOMAIN_MISMATCH
sslproxy_cert_error allow BrokenButTrustedServers DomainMismatch
sslproxy_cert_error deny all


# --- Login URL Cache Bypass ---
# Prevents caching of login/auth pages. Caching these can cause session fixation
# issues or serve stale login forms to users.
#
acl https_login url_regex -i login(\.php|\.aspx|\.html|\/|$)
cache deny https_login


# --- Range Request Limits ---
# Limits how far into a file a range request can seek before Squid fetches the
# whole object. Large limit for Windows Update (ISO/cab files need large seeks);
# smaller limit for everything else to avoid cache pollution from partial fetches.
#
range_offset_limit 4096 MB windowsupdate
range_offset_limit 512 MB !windowsupdate


# --- Final Allow + Abort Threshold ---
# Allows all remaining localnet_v6 traffic after the deny rules above.
# quick_abort_min -1 tells Squid to never abort an in-progress cache fill
# even if the client disconnects — ensures the object lands in cache for the
# next requester.
#
http_access allow localnet_v6
quick_abort_min -1

[JonathanLee]

@luckman212

# =============================================================================
# Custom Options (SSL Bump / Intercept Section)
#
# This block controls per-device SSL inspection policy, TLS session handling,
# connection performance tuning, and timeout behavior.
#
# The core philosophy here is a three-tier bump model:
#   1. splice_only — devices that cannot tolerate MITM (phones, tablets, etc.)
#                    TLS is passed through transparently; Squid sees SNI only.
#   2. bump_only   — devices where full SSL inspection is desired/acceptable.
#                    Squid decrypts, inspects, and re-encrypts traffic.
#   3. Everything else — terminated or spliced based on URL/domain rules.
#
# Per-device policy is enforced by matching BOTH IP and MAC address together
# (splice_only_local_group / bump_main), preventing policy bypass by IP spoofing.
# =============================================================================


# --- IPv6 Subnet ACL ---
# Matches your entire LAN IPv6 prefix. Used to apply consistent policy to
# dual-stack clients regardless of whether they connect via IPv4 or IPv6.
#
acl splice_ipv6 src [your-lan-ipv6-prefix]::/64


# ---------------------------------------------------------------------------
# Per-Device SSL Policy ACLs
#
# Each device is identified by BOTH its IP address AND its MAC address.
# The combined ACLs (splice_only_local_group / bump_main below) require both
# to match, so a client can't change their IP to escape their assigned policy.
#
# splice_only — pass-through TLS (no decryption). Appropriate for:
#   - Mobile devices with certificate pinning (iOS, Android apps)
#   - Devices where installing a custom CA cert is impractical
#
# bump_only — full SSL inspection. Appropriate for:
#   - Managed desktops/laptops with CA cert deployed
#   - Set-top boxes / Pi devices under your control
# ---------------------------------------------------------------------------

# [Device 1 — phone]
acl splice_only_ip src [device1-ipv4]
acl splice_only_ip src [device1-ipv6]
acl splice_only_mac arp [device1-mac]

# [Device 2 — desktop]
acl splice_only_ip src [device2-ipv4]
acl splice_only_ip src [device2-ipv6]
acl splice_only_mac arp [device2-mac]

# [Device 3 — phone]
acl splice_only_ip src [device3-ipv4]
acl splice_only_ip src [device3-ipv6]
acl splice_only_mac arp [device3-mac]

# [Device 4 — tablet, IPv4 only]
acl splice_only_ip src [device4-ipv4]
acl splice_only_mac arp [device4-mac]

# [Device 5 — streaming device, IPv4 only]
acl splice_only_ip src [device5-ipv4]
acl splice_only_mac arp [device5-mac]

# [Device 6 — laptop]
acl splice_only_ip src [device6-ipv4]
acl splice_only_ip src [device6-ipv6]
acl splice_only_mac arp [device6-mac]

# [Device 7 — laptop]
acl splice_only_ip src [device7-ipv4]
acl splice_only_ip src [device7-ipv6]
acl splice_only_mac arp [device7-mac]

# [Device 8 — tablet]
acl splice_only_ip src [device8-ipv4]
acl splice_only_ip src [device8-ipv6]
acl splice_only_mac arp [device8-mac]

# [Device 9 — laptop]
acl splice_only_ip src [device9-ipv4]
acl splice_only_ip src [device9-ipv6]
acl splice_only_mac arp [device9-mac]

# [bump_only — Pi / media device]
acl bump_only_ip src [bump-device1-ipv4]
acl bump_only_ip src [bump-device1-ipv6]
acl bump_only_mac arp [bump-device1-mac]

# [bump_only — media/TV device]
acl bump_only_ip src [bump-device2-ipv4]
acl bump_only_ip src [bump-device2-ipv6]
acl bump_only_mac arp [bump-device2-mac]

# [bump_only — WPAD server / Zero device]
acl bump_only_ip src [bump-device3-ipv4]
acl bump_only_ip src [bump-device3-ipv6]
acl bump_only_mac arp [bump-device3-mac]


# --- pfSense Admin Interface — Always Splice ---
# Prevents Squid from bumping connections to the router's own web UI.
# Bumping the admin interface causes certificate mismatch errors since pfSense
# presents its self-signed cert, which doesn't match what Squid would generate.
#
acl no_bump_pfsense dst [router-ipv4]
acl no_bump_pfsense dst [router-ipv6]


# --- SSL Intercept Domain Exclusions ---
# NoSSLIntercept: regex list of hostnames that must never be bumped.
#   Typically includes certificate-pinned services (e.g. Apple, banking apps).
# NoBumpDNS: domain list resolved at runtime (no-cache DNS lookup via -n flag)
#   for services that rotate IPs or use dynamic hostnames.
#
acl NoSSLIntercept ssl::server_name_regex -i "/usr/local/pkg/reg.url.nobump"
acl NoBumpDNS dstdomain -n "/usr/local/pkg/dns.nobump"


# --- Composite Group ACLs ---
# active_use: annotates a client as actively in use (used to gate miss_access below).
# terminate_group: connections that should be hard-terminated (e.g. DoH bypass attempts).
# splice_group: connections that must be spliced due to content type or domain policy.
# splice_only_local_group: IP+MAC match required — both must be true (all-of).
# splice_main: union of all splice conditions.
# bump_main: IP+MAC match required for full inspection — both must be true (all-of).
#
acl active_use annotate_client active=true
acl terminate_group any-of doh_rfc8484
acl splice_group any-of https_login NoBumpDNS NoSSLIntercept
acl splice_only_local_group all-of splice_only_mac splice_only_ip
acl splice_main any-of splice_group splice_only_local_group
acl bump_main all-of bump_only_mac bump_only_ip


# --- SSL Bump Rule Chain ---
# Rules are evaluated top-to-bottom; first match wins.
#
# terminate ... block_hours : Hard-close TLS connections during maintenance window.
#
# miss_access allow ms_delivery_domains active_use : Allows Microsoft delivery
#   CDN cache misses (origin fetches) for active clients. Prevents stale
#   Windows Update content being served from a cold cache.
#
# miss_access deny no_miss active_use : Blocks cache misses for real-time
#   WebSocket URLs (Facebook WS etc.) defined in the no_miss ACL. Forces
#   these to be splice-only — they cannot be cached or re-fetched anyway.
#
# peek step1 : Read the TLS ClientHello to extract the SNI hostname without
#   completing the handshake. Allows domain-based decisions before bumping.
#
# splice splice_main : Pass through TLS transparently for splice-policy devices
#   and excluded domains. Client and server negotiate directly; Squid observes only.
#
# splice no_bump_pfsense : Splice router admin connections to avoid cert errors.
#
# stare step2 : After peek, hold the connection open to inspect the server's
#   certificate before deciding to bump or splice. Catches cases where the
#   SNI alone isn't enough to make a policy decision.
#
# terminate terminate_group : Drop connections matching the terminate ACL
#   (e.g. DoH bypass attempts).
#
# bump bump_main : Full MITM inspection for devices in the bump_only group.
#   Squid generates a dynamic certificate signed by your local CA.
#
# terminate !activated : Catch-all — terminates any TLS connection that didn't
#   receive the active_use annotation (i.e. wasn't processed through the chain).
#   Prevents uninspected connections from falling through silently.
#
ssl_bump terminate all block_hours
acl ms_delivery_domains dstdomain .microsoft.com
miss_access allow ms_delivery_domains active_use
miss_access deny no_miss active_use
ssl_bump peek step1
ssl_bump splice splice_main active_use
ssl_bump splice no_bump_pfsense active_use
ssl_bump stare step2
ssl_bump terminate terminate_group
ssl_bump bump bump_main active_use
acl activated note active_use true
ssl_bump terminate !activated


# --- TLS Session Cache ---
# Caches negotiated TLS session parameters so returning clients can resume
# without a full handshake. 128 MB cache with 30-minute TTL balances memory
# use against the performance benefit for long-lived devices on the LAN.
#
sslproxy_session_cache_size 128 MB
sslproxy_session_ttl 1800


# --- Connection Performance ---
# Persistent connections: reuse existing TCP connections to origin servers and
#   clients rather than opening a new one per request. Major throughput win.
# pconn_timeout 120s: how long an idle persistent connection is kept open.
# collapsed_forwarding: if multiple clients request the same uncached object
#   simultaneously, only one origin fetch is made; the rest wait and share the
#   result. Prevents thundering-herd on popular uncached content.
#
server_persistent_connections on
client_persistent_connections on
pconn_timeout 120 seconds
collapsed_forwarding on


# --- Timeout Tuning ---
# Tuned to handle large downloads (Windows Updates, ISOs) without premature
# disconnection, while still reaping truly dead connections promptly.
#
# read_timeout 60m    : Maximum time to wait for data on an established connection.
#                       Large for big update packages.
# request_timeout 10m : How long Squid waits for a complete HTTP request from
#                       a client before giving up.
# connect_timeout 45s : Time allowed to establish a TCP connection to the origin.
# client_lifetime 90m : Maximum total lifespan of a client connection.
# pconn_lifetime 20m  : Maximum lifespan of a persistent connection before forced close.
# client_idle_pconn_timeout 10m  : Idle timeout for client-side persistent connections.
# server_idle_pconn_timeout 10m  : Idle timeout for server-side persistent connections.
# positive_dns_ttl 10m: How long to cache successful DNS lookups internally.
# negative_dns_ttl 1m : How long to cache failed DNS lookups (NXDOMAIN etc.).
# dns_timeout 30s     : Maximum time to wait for a DNS response.
# shutdown_lifetime 5s: How long Squid waits for in-flight requests to finish
#                       on graceful shutdown before force-killing workers.
#
read_timeout 60 minutes
request_timeout 10 minutes
connect_timeout 45 seconds
client_lifetime 90 minutes
pconn_lifetime 20 minutes
client_idle_pconn_timeout 10 minutes
server_idle_pconn_timeout 10 minutes
positive_dns_ttl 10 minutes
negative_dns_ttl 1 minute
dns_timeout 30 seconds
shutdown_lifetime 5 seconds


# --- Additional Performance Settings ---
# read_ahead_gap 2 MB : Buffer up to 2 MB ahead of what the client has consumed.
#   Smooths out bursty traffic and reduces round-trips on large transfers.
# pipeline_prefetch 15: Allows Squid to process up to 15 pipelined HTTP/1.1
#   requests ahead of responses. Improves throughput on keep-alive connections.
# happy_eyeballs_connect_timeout 300ms : RFC 6555 dual-stack racing timeout.
#   If IPv6 doesn't connect within 300ms, an IPv4 attempt is started in parallel.
# memory_pools on     : Reuses freed memory blocks internally rather than
#   returning them to the OS, reducing allocator overhead under sustained load.
# memory_pools_limit 128 MB : Cap on pooled memory Squid holds back from the OS.
# tcp_outgoing_tos 0x10 localnet : DSCP mark for LAN-bound traffic (CS0/low-latency).
#
read_ahead_gap 2 MB
pipeline_prefetch 15
happy_eyeballs_connect_timeout 300
memory_pools on
memory_pools_limit 128 MB
tcp_outgoing_tos 0x10 localnet


# --- Core Dump Directory ---
# Where Squid writes core dumps on crash. Point to fast storage.
# Replace with your actual path.
#
coredump_dir /path/to/squid/dump


# --- Hardware Crypto (SafeXcel / devcrypto) ---
# Offloads client-facing TLS operations to the kernel crypto engine.
# ssl_engine devcrypto applies to both incoming and outgoing TLS when set here.
#
ssl_engine devcrypto


# --- Client-Facing Cipher Suite ---
# Controls which ciphers Squid offers to clients during the bump handshake
# (i.e. what the browser sees). Modernized to prefer ChaCha20 (better on
# mobile/ARM) and AES-GCM, with SHA384/SHA256 RSA fallbacks for older clients.
# The older SafeXcel AES-CBC profile is kept below for reference.
#
# SafeXcel AES-CBC profile (legacy, hardware-optimized for SG-2100):
#sslproxy_cipher ECDHE-RSA-AES256-SHA384:ECDHE-RSA-AES128-SHA256:AES256-SHA256:AES128-SHA256:AES256-SHA:AES128-SHA

sslproxy_cipher ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-RSA-AES128-SHA256

[JonathanLee]

@luckman212

# =============================================================================
# refresh_pattern — Software Update Caching Rules
#
# This block controls how long Squid considers cached objects fresh, and
# whether it respects or overrides cache-control headers from origin servers.
#
# refresh_pattern syntax:
#   refresh_pattern [-i] <regex> <min> <percent> <max> [options]
#
#   min     : Minimum TTL in minutes — object is always considered fresh
#             for at least this long, regardless of headers.
#   percent : If the object has a Last-Modified header, use this percentage
#             of its age as the estimated TTL (LM-factor heuristic).
#   max     : Maximum TTL in minutes — object is never considered fresh
#             beyond this, even if headers say otherwise.
#
# Common override options used below:
#   ignore-reload          : Ignore client Cache-Control: no-cache / Pragma: no-cache.
#                            Prevents clients from forcing unnecessary re-fetches.
#   ignore-no-store        : Ignore Cache-Control: no-store from origin.
#                            Many Microsoft CDNs send this but the content is static.
#   ignore-private         : Ignore Cache-Control: private.
#                            Update payloads aren't actually user-specific.
#   ignore-must-revalidate : Don't revalidate at expiry — serve stale if needed.
#                            Keeps large updates accessible during origin outages.
#   override-expire        : Ignore Expires headers set in the past (common on CDNs).
#   override-lastmod       : Override Last-Modified-based freshness calculation.
#
# Values used throughout this block:
#   min=10080  = 7 days
#   max=129600 = 90 days
#   90%        = LM-factor heuristic
#
# This aggressive caching strategy is intentional for software updates:
# update binaries are content-addressed (URL changes when content changes),
# so serving a cached copy is always safe regardless of cache-control headers.
# =============================================================================


# --- Store-ID Internal Domains (MUST be first) ---
# When Store-ID rewriting is active (see Custom Options section), Squid rewrites
# real CDN URLs to stable internal hostnames like ms-delivery.squid.internal.
# These refresh_pattern lines MUST appear before any real-hostname patterns
# because Squid matches the rewritten Store-ID URL — not the original URL —
# when deciding freshness for objects stored under a custom Store-ID.
#
# Each internal hostname corresponds to a category of Microsoft content:
#   ms-delivery.squid.internal — Delivery Optimization / WindowsUpdate CDN
#   msstore.squid.internal     — Microsoft Store app packages
#   microsoft.squid.internal   — General Microsoft download endpoints
#   office365.squid.internal   — Office 365 / Microsoft 365 installer CDN
#   windowsupdate.squid.internal — Classic Windows Update endpoints
#   generic.squid.internal     — Catch-all for other rewritten CDN URLs
#
refresh_pattern -i ^http://ms-delivery\.squid\.internal/.* 10080 90% 129600 ignore-reload ignore-no-store ignore-private ignore-must-revalidate override-lastmod
refresh_pattern -i ^http://msstore\.squid\.internal/.* 10080 90% 129600 ignore-reload ignore-no-store ignore-private ignore-must-revalidate override-lastmod
refresh_pattern -i ^http://microsoft\.squid\.internal/.* 10080 90% 129600 ignore-reload ignore-no-store ignore-private ignore-must-revalidate override-lastmod
refresh_pattern -i ^http://office365\.squid\.internal/.* 10080 90% 129600 ignore-reload ignore-no-store ignore-private ignore-must-revalidate override-lastmod
refresh_pattern -i ^http://windowsupdate\.squid\.internal/.* 10080 90% 129600 ignore-reload ignore-no-store ignore-private ignore-must-revalidate override-lastmod
refresh_pattern -i ^http://generic\.squid\.internal/.* 10080 90% 129600 ignore-reload ignore-no-store ignore-private ignore-must-revalidate override-lastmod


# --- Microsoft Windows Update, Office 365, Defender, and Common Update Files ---
# Matches update payloads by domain + file extension across the main Microsoft
# and Akamai/Azure CDN endpoints used for:
#   - Windows Update (download.microsoft.com, windowsupdate.com, etc.)
#   - Office 365 / Microsoft 365 installers (officecdn.microsoft.com)
#   - Microsoft Edge (microsoftedge.com, edgecastcdn.net)
#   - Azure CDN-fronted delivery (azureedge.net, akamaiedge.net)
#
# Cached file types: .cab .exe .msi .msu .msf .psf .zip .dat .mum
#                    .manifest .manifest.xml .json .cat .xml
# These are all static, content-addressed update payloads — safe to cache
# aggressively. override-expire handles CDNs that set past Expires headers
# as a cache-busting trick.
#
refresh_pattern -i (download\.microsoft\.com|update\.microsoft\.com|windowsupdate\.com|windows\.com|officecdn\.microsoft\.com|officecdn\.microsoftonline\.com|officeclient\.officeapps\.live\.net|akamaiedge\.net|download\.windowsupdate\.com|download\.windowsupdate\.com\.msedge\.net|wustat\.windows\.com|microsoftedge\.com|edgecastcdn\.net|azureedge\.net)/.*\.(cab|exe|msi|msu|msf|psf|zip|dat|mum|manifest|manifest\.xml|json|cat|xml) 10080 90% 129600 ignore-reload ignore-no-store ignore-private override-expire


# --- Microsoft Delivery Optimization (Streaming / Range Request CDN) ---
# Handles filestreamingservice URLs from dl.delivery.mp.microsoft.com.
# These use range requests heavily (large updates delivered in chunks), which
# is why range_offset_limit is set generously in the main config.
# override-lastmod is added here because these URLs don't carry reliable
# Last-Modified headers — the LM-factor heuristic would produce bad results.
#
refresh_pattern -i .*\.dl\.delivery\.mp\.microsoft\.com/filestreamingservice/.* 10080 90% 129600 ignore-reload ignore-no-store ignore-private ignore-must-revalidate override-expire override-lastmod


# --- Microsoft Store App Packages ---
# Matches Store app bundles from the Store edge CDN and delivery CDN.
# File types: .appx .appxbundle .msixbundle .msix .eappx
# These are large, versioned, immutable packages — aggressive caching is safe.
# A single cached copy serves all devices on the LAN updating the same app.
#
refresh_pattern -i (storeedge\.fd\.dsp\.ms|storeedgefd\.azureedge\.net|.*\.dl\.delivery\.mp\.microsoft\.com)/.*\.(appx|appxbundle|msixbundle|msix|eappx) 10080 90% 129600 ignore-reload ignore-no-store ignore-private ignore-must-revalidate override-expire override-lastmod


# --- Microsoft Telemetry and Diagnostics — DO NOT CACHE ---
# Telemetry endpoints send per-device, session-specific data upstream.
# Caching these would cause multiple devices to share a single telemetry
# response, corrupting diagnostics data and potentially breaking Windows
# feature/settings sync (settings.win.microsoft.com, mobile.settings.windows.com).
# Setting min=0, percent=0%, max=0 with override-expire forces Squid to treat
# every response as immediately stale — effectively disabling caching for
# these domains while still allowing the connections through.
#
# Domains covered: telemetry, Watson crash reporting, SQM, Vortex, Office
# feedback, CDN telemetry endpoints, and Windows settings sync.
#
refresh_pattern -i (telemetry\.microsoft\.com|vortex\.data\.microsoft\.com|watson\.telemetry\.microsoft\.com|settings\.win\.microsoft\.com|watson\.microsoft\.com|feedback\.microsoft\.com|sqm\.users\.office\.com|compatexchange\.cloudapp\.net|diagnostics\.support\.microsoft\.com|telemetry\.office\.com|telemetry\.cdn\.microsoft\.com|.*\.cdn\.microsoft\.com\/telemetry|mobile\.settings\.windows\.com|sqm\.telemetry\.microsoft\.com|ntservicepacks\.blob\.core\.windows\.net) 0 0% 0 override-expire

[JonathanLee]

@luckman212

File Path References:

SSL bump — no-bump exclusions

reg.url.nobump regex

/usr/local/pkg/reg.url.nobump

Loaded by acl NoSSLIntercept ssl::server_name_regex. Contains regex patterns matched against the TLS SNI hostname at the start of a connection — before the full handshake. Any hostname matching a pattern here is spliced (passed through transparently) instead of bumped. Used for services with certificate pinning where injecting your CA cert would break the app — Apple push, banking apps, etc.

dns.nobump domain list

/usr/local/pkg/dns.nobump

Loaded by acl NoBumpDNS dstdomain -n. Plain domain list (one per line) of destinations that must never be SSL-inspected. The -n flag tells Squid to resolve these at connection time rather than caching the lookup — important for CDNs and services that rotate IPs. Covers domains not worth regex-matching but still needing splice-only treatment.

dstdom.broken domain list

/usr/local/pkg/dstdom.broken

Loaded by acl BrokenButTrustedServers. Lists internal or legacy servers that present a certificate where the hostname doesn't match the domain (cert domain mismatch). Without this, Squid would deny the connection. Adding a domain here tells Squid to allow that specific cert error for that specific destination — all other SSL errors are still denied.

Access control — domain lists

http.access domain list

/usr/local/pkg/http.access

Loaded by acl HttpAccess. Domains in this list are sent direct (bypass the cache) and are also allowed through CONNECT tunnels for localnet clients. Intended for sites that must not be cached or inspected — typically plain HTTP services or destinations where you explicitly want direct routing.

windowsupdate domain list

/usr/local/pkg/windowsupdate

Loaded by acl windowsupdate. All Microsoft Windows Update CDN domains. Used in three places: CONNECT tunnel allowances (lets Windows Update do HTTPS through the proxy), range_offset_limit 4096 MB (allows huge range requests for large update payloads), and miss_access rules to permit origin fetches for active clients.

always_direct domain list

/usr/local/pkg/always_direct

Loaded by acl AlwaysDirect. Domains that must always connect directly to the origin — bypassing both the cache and any upstream peer. Typically used for destinations where proxying causes authentication failures, geo-restriction issues, or where latency from caching is undesirable.

Store-ID rewriting

desdom domain list

/usr/local/pkg/desdom

Loaded by acl rewritedoms. The trigger list for Store-ID rewriting — only requests whose destination domain appears here are passed to the Store-ID rewrite program. Keeping this list tight prevents unnecessary rewrite overhead on every request. Typically contains CDN domains whose URLs vary per-client but whose content is identical (Microsoft, Akamai, etc.).

storeid_rewrite.txt rewrite map

/var/squid/storeid/storeid_rewrite.txt

The actual rewrite map consumed by storeid_file_rewrite. Each line maps a URL pattern to a stable internal Store-ID — for example, a Microsoft delivery CDN URL with a session token gets rewritten to http://ms-delivery.squid.internal/<hash>. This means all devices on the LAN requesting the same update package share one cached copy, regardless of the unique query strings in their individual URLs.

How Store-ID rewriting works end-to-end

When a client requests a CDN URL, Squid checks if the destination domain is in desdom. If yes, the URL is handed to storeid_file_rewrite, which looks it up in storeid_rewrite.txt and returns a normalised internal URL like http://ms-delivery.squid.internal/KB5034441.cab. Squid uses this as the cache key instead of the raw CDN URL. The refresh_pattern lines for *.squid.internal then control how long that cached object stays fresh. Result: one origin fetch serves the same update to every device on the LAN.

Keep in mind Squid Developers are working on getting ssl_engine to work with more modern stuff it doesnt work anymore after a certain version they have a request open. I hope that helps it's like years of trial and errors and my wife getting mad at me, it still has some small issues but that custom config I shared is mostly stable and works well, the Windows updates cache item changes every time the wind blows it used to actually save the download one time and send it out again still does for a lot of stuff but it changes all the time. Hope you enjoy

[JonathanLee]

@KOM Yeah right lol I can't even count how many times I have I have found older forum posts from you I have checked out and tested stuff with...

[aGeekhere]

@JonathanLee I have been using the following with a WPAD, i no longer use the transparent proxy because of the way encrypted Client Hello (ECH) hides the Server Name Indication (SNI) where non transparent does not have this issue.

# ---------------------------------------------------------
# General & Performance Settings
# ---------------------------------------------------------
email_err_data off
server_persistent_connections on
client_persistent_connections on

# TLS inspection stages
acl step1 at_step SslBump1
acl step2 at_step SslBump2
acl step3 at_step SslBump3

# ---------------------------------------------------------
# Custom ACL File Definitions
# ---------------------------------------------------------
# Ensure these files are readable by the 'proxy' user
acl bypassUsers src "/home/bypassUsers.txt"
acl blockSites ssl::server_name "/home/customBlockUrls.txt"
acl excludeSites ssl::server_name "/home/excludeSites.txt"
acl bumpSites ssl::server_name "/home/bumpSites.txt"

# ---------------------------------------------------------
# Access Control (http_access)
# ---------------------------------------------------------
# 1. Allow bypass users first so they aren't restricted by blocks
http_access allow bypassUsers

# 2. Deny blocked sites for everyone else (shows Squid block page)
http_access deny blockSites
# ---------------------------------------------------------
# SSL Bump Logic (The "First Match Wins" Stack)
# ---------------------------------------------------------

# Step 1: peek ClientHello to see SNI
ssl_bump peek step1

# Step 2: Decision Making
# Priority 1: Bypass users (splice immediately, no MITM)
ssl_bump splice bypassUsers

# Priority 2: Excluded domains (Banking, Privacy, etc. - splice)
ssl_bump splice excludeSites

# Priority 3: Specific Bump Sites (stare to prepare for MITM)
ssl_bump stare bumpSites

# Step 2 continued: peek ServerHello / handshake
ssl_bump peek step2

# Step 3: Final Action
# Bump only sites in bumpSites (MITM)
ssl_bump bump bumpSites

# Default safety: splice everything else
ssl_bump splice all

# ---------------------------------------------------------
# Optional: Certificate Error Handling
# ---------------------------------------------------------
# Prevents users from getting stuck on Squid error pages for upstream cert errors
acl cert_errors ssl_error SQUID_X509_V_ERR_DOMAIN_MISMATCH
ssl_bump splice step3 cert_errors

# ---------------------------------------------------------
# Notes:
# ---------------------------------------------------------
# 1. Block pages will only appear if the client trusts the Squid CA (for HTTPS sites).
# 2. Untrusted clients (without CA) will fall into default splice; MITM will not happen.
# 3. peek/stare ensures maximum TLS stability and SNI visibility.
# 4. tweak error_directory to customize Squid block pages if desired.

I found waiting after step2 to bump or splice works much better then doing that after step 1.

I am still looking for ways to improve the config but so far it has been working well.

Would be nice if we got the squid updates though.

[JonathanLee]

@aGeekhere

I think this ACL should be placed on top above bump logic so the memory has it when it hits that step?

# ---------------------------------------------------------
# Optional: Certificate Error Handling
# ---------------------------------------------------------
# Prevents users from getting stuck on Squid error pages for upstream cert errors
acl cert_errors ssl_error SQUID_X509_V_ERR_DOMAIN_MISMATCH
ssl_bump splice step3 cert_errors

[aGeekhere]

@JonathanLee
No need to move the cert_errors ACL to the top.
Squid evaluates all ssl_bump rules at each step (including step3), and the first matching rule wins. The step3 ACL already ensures the error rule only applies during SslBump3, when the server certificate is checked.
Placing it at the top doesn't preload anything in memory as ACLs are parsed at startup. Keeping it after the main bump logic (as a safety net) is cleaner and follows best practice.

https://www.squid-cache.org/Doc/config/ssl_bump/
"All ssl_bump rules are evaluated at each of the supported bumping steps. Rules with actions that are impossible at the current step are ignored. The first matching ssl_bump action wins..."

As far as my understanding goes.

[tinfoilmatt]

doh_rfc8484

How are you populating this ACL? Is it comprised of both IPs and domain names?

[JonathanLee]

@tinfoilmatt like this

acl doh_rfc8484 urlpath_regex -i ^/dns-query
acl doh_rfc8484 urlpath_regex -i dns=
acl doh_rfc8484 urlpath_regex -i ^/resolve

urlpath_regex

"urlpath_regex: URL-path regular expression pattern matching, leaves out the protocol and hostname"