SSLBUMP without MITM

doktornotor

Yeah, afraid that's a no go, the certs are really required - plus, the ACL (whitelist) can be anything there and be changed at any time, or have lines prepended/appended, really no way for Squid to figure out it's not going to need to fake any certs.

(But you could generate a temporary one for the CA in the background on each config resync; would need additional logic/code though in squid.inc. If you want something like this, I'd prefer having a separate checkbox for this, with some JS logic that'd gray out/disable unneeded MITM stuff in the GUI, and use a different ACL than the current whitelist for that, basically just use the built-in "all" one - like ssl_bump splice all)

shonjir

I'm running a variation of this config in transparent mode with squidguard and shallalist quite successfully.

I'm not sure why exactly, but this configuration improves over basic step 1 peek and splice all by allowing logging of the server certificate (instead of just IP address), domain blocking of SSL sites via squidguard list, and provides an alternate splice operation for certain sites that for some reason cannot connect after the certificate name is captured.

# peek at client TLS-request to get SNI
# peek at server cert (for logging)
# splice earlySplice at step 3 only
acl earlySplice ssl::server_name "*.crashplan.com"
ssl_bump peek step1
ssl_bump peek step2 earlySplice
ssl_bump splice step3 earlySplice
ssl_bump splice all
sslproxy_cert_error deny all

doktornotor

Well yes, that will no doubt improve splice since you get whole lot more information available at steps 2/3, as explained here:
https://github.com/pfsense/FreeBSD-ports/blob/devel/www/pfSense-pkg-squid/files/usr/local/pkg/squid.inc#L1539

The above makes bumping impossible, however.

It's not possible to do zillion variants in the GUI. Use Advanced Options, Custom ACLs (before auth).

shonjir

The above makes bumping impossible, however.

This is understood. For my use case, I don't need full blown MiTM - logging the site name and simple domain blocking are sufficient.

Right now I'm using custom ACLs for this. It's a hack (because it supersedes the auto-generated ssl_bump directives that follow), but it works.

It would be nice if it were available as a simple toggle in the config or if the various ssl config options were exposed for customization under 'advanced', but it works reasonably well as-is.

Thank you for your work making this tool available!

doktornotor

I'll see if something reasonable can be added for

splice everything without bump (i.e., splice all)
custom splice ACLs/settings (some advanced textarea burried somewhere, if you break it, your problem)

without overcomplicating the GUI and confusing everyone.

doktornotor

OK, how's this?

https://github.com/pfsense/FreeBSD-ports/pull/269

ssl_mitm_mode_01.png_thumb

ssl_mitm_mode_02.png_thumb

jimp

Looks great, we're getting that PR merged in right now, thanks!

doktornotor

Thanks for testing/feedback. Any volunteers to ~~fix~~rewrite SquidGuard now? (No, I cannot do it. Tried many times, unable to work with that code for more than a couple of minutes.)

fabiomello

e2guardian its more active…
if you want to replace squidGuard with e2guardian i could do this!

what you think?

https://github.com/e2guardian/e2guardian

doktornotor

@fabiomello:

e2guardian its more active…
if you want to replace squidGuard with e2guardian i could do this!

what you think?

https://github.com/e2guardian/e2guardian

Hmmm… good luck.

https://forum.pfsense.org/index.php?topic=87526.0
https://github.com/pfsense/pfsense-packages/pull/866

jimp

From the history there it looks like it had a similar problem to squidGuard. Whatever happens, it needs a ground-up rewrite with up-to-date style and practices all-around.

doktornotor

Yeah. The XMLs in the linked e2guardian PR would be a good starting point to get a grip of potential feature set. The PHP code there still needs major work, though it must have been a lot better than SquidGuard, considering I've been able to somehow finish the code style at least. :P

The PHP-generated package XML is a major can of worms there.

Plus note, that PR was for 2.2.x, lots of changes meanwhile.

susamlicubuk

Good job. Congratulations.
Tag_none / 200 for all https and ip address is not good to appear in logs
Do you have a solution for this problem?
A manual DNS problem can sometimes cause the page not found error
We continue to test this issue

Tag_None.JPG_thumb

doktornotor

And the issue is exactly what? TAG_NONE/XXX means spliced connection. It works. Really nothing to fix here. This is a TCP tunnel, Squid won't handle any HTTP there.

http://squid-web-proxy-cache.1019090.n4.nabble.com/TAG-NONE-xxxx-td4672150.html

If you have issues with some sites, you'll need to use some custom ACL and custom config for that at the moment, I'm afraid.

susamlicubuk

@doktornotor:

And the issue is exactly what? TAG_NONE/XXX means spliced connection. It works. Really nothing to fix here. This is a TCP tunnel, Squid won't handle any HTTP there.

http://squid-web-proxy-cache.1019090.n4.nabble.com/TAG-NONE-xxxx-td4672150.html

If you have issues with some sites, you'll need to use some custom ACL and custom config for that at the moment, I'm afraid.

I have tested it for a long time on the old version
I have tested the following settings
I used 2 different settings

acl https_proto proto https
always_direct allow https_proto
ssl_bump none localhost
sslproxy_options NO_SSLv2,NO_SSLv3,No_Compression
sslproxy_cipher ALL:!SSLv2:!ADH:!DSS:!MD5:!EXP:!DES:!PSK:!SRP:!RC4:!IDEA:!SEED:!aNULL:!eNULL

TLS/SSL bumping definitions

acl tls_s1_connect at_step SslBump1
acl tls_s2_client_hello at_step SslBump2
acl tls_s3_server_hello at_step SslBump3

TLS/SSL bumping steps

ssl_bump peek tls_s1_connect all
ssl_bump splice all

peek at TLS/SSL connect data

splice: no active bumping

acl bump_step1 at_step SslBump1
ssl_bump peek bump_step1 all
ssl_bump splice all
sslproxy_cert_error deny all
ssl_bump splice whitelist
ssl_bump bump all

There are too many reverse ip addresses in squid logs.
So the lightsquid report is mixed.
Maybe it's about reverse proxy.
It works fine at startup
But there are some page faults that are sometimes not found when using for a long time.
(Google search result after clicking on page unresponsive or https sites sometimes very slow)
I think squid is having trouble with https dns resolution.
I only saw problems with windows pc's
I have not received any errors on iOS and android mobile devices
This caused the reason for not continuing with wpad for PCs.
But mobile devices can use it.

We can do acl setting but we need to enter too many https domain gui den
A difficult and problematic job
But I added some sites squid- acl- white list
I'm not sure we're adding it this way

..google.com
..google.com.tr

Please manually enter external dns on windows computers and test
This problem is a bit strange
Sometimes page not found or too slow browser problem
For example 8.8.8.8 or open dns
Maybe we can solve this problem with all the DNS requests with the Pfsense redirection rule.

doktornotor

Please, start your own thread. DNS issues are totally off-topic here. Also, this thread is NOT about reverse proxy, at all. Thanks.

Finally, as noted above and linked on the Squid ML, the log is absolutely normal, expected and there is nothing that could be done here. If you don't want things spliced, then don't use splice. If you don't want logs, don't log. If you need logs filtered, you'll need to filter them as you need. And if you want something better than TAG_NONE in logs, you'll need to get that implemented upstream.

shonjir

@doktornotor:

OK, how's this?

https://github.com/pfsense/FreeBSD-ports/pull/269

This is awesome, thanks @doktornotor! I'll be installing this package and testing shortly.

Edit: Package works great! One thing I noticed is that custom MITM options only get added when 'custom' mode is set - the package works fine as is, but a note in the help text would make this clearer.

Thanks again!

shonjir

@jimp:

From the history there it looks like it had a similar problem to squidGuard. Whatever happens, it needs a ground-up rewrite with up-to-date style and practices all-around.

@doktornotor:

Yeah. The XMLs in the linked e2guardian PR would be a good starting point to get a grip of potential feature set. The PHP code there still needs major work, though it must have been a lot better than SquidGuard, considering I've been able to somehow finish the code style at least. :P

Maybe the right solution isn't to rewrite/replace squidGuard but to abandon it altogether?

There's at least one provider of blocklists that provides Squid ACL format, so from a blocklist/whitelist perspective it seems the necessary functionality is there - what's lacking is translation of the categorized blocklists into Squid ACLs.

It seems that an interface to translate existing squidGuard blocklist format and category management into the necessary ACLs might bridge the gap nicely. The e2 package might provide base for this.

Is this an approach that might be feasible?

doktornotor

@shonjir:

but a note in the help text would make this clearer.

Apparently it wouldn't, because it's already there (twice - at the checkbox info and at the textarea info), but you still didn't read it. :P (Try clicking the blue "i")

pfsensation

Thanks for the fantastic work Doktornotor. I am loving the new updates to Squid and how user friendly you are making it. Even the small things like the explanation. Helps me out a lot! Thanks again mate!

EDIT : I'm having issues with splice all messing up certain apps such as Telegram, or Instagram (sometimes). On Telegram it was getting no connection to their servers at all, I turned off the splice all setting. Everything worked perfectly, put it back on, messages went through, images didn't work. Turned splice all off, images, everything worked…It's very strange.