Infamous /409 issue

JonathanLee

Side note:
I use Squidguard, Squid and Squidlite. My ISP does not provide me with IPv6 so I have it disabled in unbound resolver custom config. I was getting the same errors you have after I disabled IPv6 it stopped. Again I use SSL intercept for some devices and have custom devices set to splice.

(Unbound custom options for no IPv6)

server:
do-ip4: yes
prefer-ip4: yes
do-ip6: no
prefer-ip6: no
private-address: ::/0
dns64-ignore-aaaa: .
do-not-query-address: ::
do-not-query-address: ::1
do-not-query-address: ::/0

(Squid set to custom)

acl splice_only src 192.168.1.7 #Jon Android
acl splice_only src 192.168.1.8 #Tasha Apple
acl splice_only src 192.168.1.11 #Amazon Fire
acl splice_only src 192.168.1.15 #Tasha HP
acl splice_only src 192.168.1.16 #iPad
acl splice_only src 192.168.1.18 #Xbox
acl NoSSLIntercept ssl::server_name_regex -i "/usr/local/pkg/url.nobump" #custom Regex splice url list file

ssl_bump peek step1 #peek
ssl_bump splice splice_only #splice devices
ssl_bump splice NoSSLIntercept #splice only url
ssl_bump stare all # stare all defaults to bump on step 3

#Commented out lines
#acl SSLIntercept ssl::server_name_regex -i "/usr/local/pkg/url.bump" #always bump list
#ssl_bump bump SSLIntercept

vlurk

@JonathanLee
I saw your other message about force disabling IPv6.

I gave it a shot. It didn't seem to have much of an impact here. Maybe it did for some DNS records that have IPv6 addresses attached to them. But I have identified tons of DNS records randomly failing with 409 that don't resolve to any IPv6 address, whatsoever.

This is an issue with short TTLs.
Consider this DNS record: nav-edge.smartscreen.microsoft.com Edge can randomly query this for SmartScreen. The A record has a 10sec TTL.

No IPV6 address will be returned for this, ever. Yet, I am logging a few 409 for this destination, from time to time. (until I made sure all traffic toward Azure Cloud would bypass Squid). All it takes is that the web browser tries to establish a connexion to it with a given IP, and when squid attempts the same resolution, it cannot confirm that this IP match the domain in destination.

Of course, I am making sure that no DNS query can "leak" to the Internet from my clients: NAT rule for UDP port 53, block on outbound 853, and blocklist for known DOH resolvers.

This is not an IPv6 related issue.

JonathanLee

@vlurk I had some issues for Disney plus, I ended up making an alias for it so that the DNS entry was on hand already.

JonathanLee

Has anyone attempted this
http://www.squid-cache.org/Doc/config/dns_nameservers/

dns_nameservers "pi-hole dns and or dns address of whatever you require"

michmoor

@JonathanLee I haven’t. Are you going to do it?

vlurk

@JonathanLee Yes. You can configure this parameter from the GUI:

This field will set the dns_nameservers config directive in squid.conf.

I use 127.0.0.1 (Unbound), and all my clients point to the same DNS server.

JonathanLee

@vlurk o yeah that's already in the GUI

What about. ..

client_persistent_connections on
client_persistent_connections off

"Squid uses persistent connections (when allowed). You can use
this option to disable persistent connections with clients."

http://www.squid-cache.org/Doc/config/client_persistent_connections

Maybe this could help I must have read every single squid option to find something

I also have certificate adaption disabled I leave the certificates as the are and have it set to not check. The other ways had issues for me for some reason.

I am just throwing ideas out now for testing. I had no idea squid has so many other configuration options.

http://www.squid-cache.org/Doc/config/sslproxy_cert_error/

They have a certificate SSL based error conditional statement for ACL use that is domain specific I found.

http://www.squid-cache.org/Doc/config/happy_eyeballs_connect_gap/

http://www.squid-cache.org/Doc/config/happy_eyeballs_connect_timeout/

Maybe...

happy_eyeballs_connect_gab ??

Or

happy_eyeballs_connect_timeout

"Happy Eyeballs is an algorithm published by the IETF that makes dual-stack applications more responsive to users by attempting to connect using both IPv4 and IPv6 at the same time, thus minimizing common problems experienced by users with imperfect IPv6 connections or setups."

michmoor

@JonathanLee

Updating this thread for everyone..

Uninstall Squid Proxy. As i long suspected by the lack of movement in any of the redmines, Netgate has decided to deprecate Squid Proxy.
I consider this a really good thing.....

But if there is a need to MITM something i dont know of any open source alternatives other than looking at other security vendor firewalls which have custom but supported proxy configuration

https://www.netgate.com/blog/deprecation-of-squid-add-on-package-for-pfsense-software

JonathanLee

@michmoor Like Lightbeam, and many other tools that worked very well it seems soon after you can no longer use them.

I wonder what alternatives there are ?

I for one will stay with 23.09 just to use Squid. Dang :( It's sad I spent years getting this to actually work.

JonathanLee

What is the next official Netgate product that will continue to support a proxy with SSL intercept that can be purchased? Now that this is being twightlighted?

What version should I upgrade too for proxy cacheing abilities? I have a SG-2100 currently. Should users move to Palo Alto?