SQUID SSL inspection transparent problem witch chat on bing.com
-
Hello. I have recently started using pfsense. I have transparent mode and SSL inspection set in squid. With this configuration I cannot use chatgpt from microsoft (bing). I keep seeing "Trying to reconnect...". What needs to be set for it to work well?
-
S szsemla referenced this topic on
-
@szsemla please can u tell which page u access and screen of the error please to test on my side, regards!!!
-
@periko Thank you for your interest in the topic. I'm from Poland so my website looks different than yours. At the bottom of the page it says "Trying to reconnect."
https://www.bing.com/search?pglt=2081&q=podaj+dalej&cvid=88c9255e6d6e45cf8ee1108827e9a432&gs_lcrp=EgZjaHJvbWUyBggAEEUYOTIGCAEQABhAMgYIAhAAGEAyBggDEAAYQDIGCAQQABhAMgYIBRAAGEAyBggGEAAYQDIGCAcQABhAMgYICBAAGEDSAQg3ODM3ajBqMagCALACAA&FORM=ANNTA1&PC=ASTS&showconv=1
Just go to "chat" at the top of the page.
-
can you send your ssl inpection ss?
-
@greenlight ok.
-
@szsemla companies sometimes use certificate pinning. You have to set a custom splice list for some sites you require access to. It doesn't mean you can't still see the get requests, you just splice those connections. Create your splice file for your firewall.
There is no buttons to do it for you, each entry must be hand done. It's an advanced set up.
FYI Netgate also recommends to stop using Squid.
-
Splice sites you can't inspect the HTTPS tunnel like you would normally. It only sees the initial get request. It helps as the splice sites are most often trusted sites you use all the time. This also sets you up for inspection of the unknown sites still, and that's when ClamAV comes in handy still. ClamAV must have SSL interception enabled to work good or else it can't really do it's job.
It's a balance
-
@JonathanLee Can you just explain to me the preparation of the url.nobump file? What do the characters "", "^", "*" mean?
-
@JonathanLee I did exactly as you described, I saw your posts on other websites and I see that the zoom application works for you. It doesn't work for me, neither does bing. I don't know what the problem is anymore.
-
@szsemla they are regular expressions. I am confused it should work did you create your certificates? You must create your certificates in command line not in the GUI or it will never work well. Also certificates must be imported into the the devices, meaning you must own the devices also to import them. Transparent mode works however it will never break SSL chain. It's a pretty advanced set up. Also you must set it to custom.
-
@JonathanLee All websites work for me except bing.com (zoom already works - i add zoom.us to whitelist). transparent proxy works. clamav works. only bing ai doesn't work.
-
@szsemla if that is working your doing great. You need to find the URL that is showing errors thats is what you need to splice only also for Bing AI software. It's a game of security balanced with need. You need the Bing AI software but they probably do certificate pinning so you have to splice the URL for that.
Check out Squid while you access that site you will see the domain and extension it attempts to access and fails on manually add that to your splice list.
From now on you will have to tune it for new URLs and changes. Amazon for example changes every year for steaming. You never want to cache steaming video that would fill up your cache for nothing. If Squid had buttons to do that it would be used alot more. I really want to just search for errors and click and add to splice list to make it simple, however it's all done manually.
It's ok because most often you access the same sites over and over. Once you get your major sites you use tuned in it's easy to keep it going.
-
@JonathanLee Can you tell me how you added bing.com to the url.nobumb list?
-
@szsemla I normally don't use AI ever, so that URL doesn't matter for me. But if I did I would try to use it and watch the Proxy live logs for the errors and use that errored URL to create a regular expression with so that it is spliced, after add the domain to no cache also. After it will work for what you need. I normally do not use AI I never have used any of that. I fear it would create a one mind system without original ideas with all originally removed.
-
@JonathanLee Is there documentation anywhere for using/creating regular expressions?
-
@szsemla they have a regular expression tester online. Google Regex tester to find it. That's how I half way self leaned it. I am lucky I get to learn it professionally next week at the University. Our Professor is going to spend a whole week on it. That tester will give you more than what is needed for URL use. I am going to have to learn to use it within discrete mathematics. That I can't help you with you would need to take a class for the discrete math side.
Use the Regex tester with different variations of the URLs with extensions so you can get many of the URL variations into one regular expression pattern.
-
Have you all attempted to use the following custom patches
Redmine#13984
This fixed a lot for me with Squid and Squidguard
