HTTPS Filtering + Splice All gives certificate issues
-
I thought doing a splice all would negate having certificate issues. We are looking to do domain filtering through squidguard to keep people out of places they shouldn't go. Doing just HTTP seems to work fine. Doing HTTPS has been far more problematic. I understand that if I want to scan HTTPS for infections and content then I need to Bump which requires certificates. If I just want to use SquidGuard to stop people from going to places like Netflix then I just need to splice all. I have a couple of problems and I'm not sure what they are. All tests are done on fresh installs of Windows 7 with IE 11. I've also reset pfSense to factory defaults and the only configuration is the WAN IP and adding in Squid and SquidGuard. SquidGuard is set to block gambling, movies, and porn and allow by default. Nothing else has been changed.
Problem 1:
On PC1 if I go to http://ww.google.com it automatically redirects me to https://www.google.com as I expect. On PC2 when I go to http://www.google.com it returns:
Request denied by pfSense proxy:403 Forbidden.
Client group: default
Target group: none
URL: http://www.google.com
On PC2 if I go to https://www.google.com it works just fine so it isn't being blocked.
I've tried clearing the cache of Squid and the browsers on each machine and it doesn't change.Problem 2:
Even though Squid is set to Splice all, when I go to pages such as https://www.hulu.com it returns:
This page can't be displayed
Turn on TLS 1.0, TLS 1.1, and TLS 1.2 in Advanced settings and try connecting to https://www.hulu.com again.Some more things that may help:
The block log. HTTP traffic is set to REDIRECT while HTTPS traffic is set to CONNECT REDIRECT
28.03.2017 13:39:29 192.168.1.130/ClaudiaPC.localdomain www.hulu.com:443 Request(default/blk_BL_movies/-) - CONNECT REDIRECT 28.03.2017 13:39:29 192.168.1.130/ClaudiaPC.localdomain www.hulu.com:443 Request(default/blk_BL_movies/-) - CONNECT REDIRECT 28.03.2017 13:39:22 192.168.1.130/ClaudiaPC.localdomain http://www.hulu.com/ Request(default/blk_BL_movies/-) - GET REDIRECT 28.03.2017 13:37:59 192.168.1.129/DottiePC.localdomain www.hulu.com:443 Request(default/blk_BL_movies/-) - CONNECT REDIRECT 28.03.2017 13:37:59 192.168.1.129/DottiePC.localdomain www.hulu.com:443 Request(default/blk_BL_movies/-) - CONNECT REDIRECT 28.03.2017 13:29:41 192.168.1.130/ClaudiaPC.localdomain netflix.com:443 Request(default/blk_BL_movies/-) - CONNECT REDIRECT 28.03.2017 13:29:41 192.168.1.130/ClaudiaPC.localdomain netflix.com:443 Request(default/blk_BL_movies/-) - CONNECT REDIRECT 28.03.2017 13:29:41 192.168.1.130/ClaudiaPC.localdomain netflix.com:443 Request(default/blk_BL_movies/-) - CONNECT REDIRECT 28.03.2017 13:29:41 192.168.1.130/ClaudiaPC.localdomain netflix.com:443 Request(default/blk_BL_movies/-) - CONNECT REDIRECT 28.03.2017 13:29:34 192.168.1.130/ClaudiaPC.localdomain http://www.netflix.com/ Request(default/blk_BL_movies/-) - GET REDIRECT 27.03.2017 20:16:30 192.168.1.130/claudiapc.localdomain http://www.zigiz.com/ Request(default/blk_BL_gamble/-) - GET REDIRECT 27.03.2017 20:16:27 192.168.1.130/claudiapc.localdomain http://www.hulu.com/ Request(default/blk_BL_movies/-) - GET REDIRECT 27.03.2017 20:05:40 192.168.1.130/claudiapc.localdomain http://www.zigiz.com/ Request(default/blk_BL_gamble/-) - GET REDIRECT
Proxy Config:
# This file is automatically generated by pfSense # Do not edit manually ! http_port 192.168.1.1:3128 ssl-bump generate-host-certificates=on dynamic_cert_mem_cache_size=10MB cert=/usr/local/etc/squid/serverkey.pem capath=/usr/local/share/certs/ cipher=EECDH+ECDSA+AESGCM:EECDH+aRSA+AESGCM:EECDH+ECDSA+SHA384:EECDH+ECDSA+SHA256:EECDH+aRSA+SHA384:EECDH+aRSA+SHA256:EECDH+aRSA+RC4:EECDH:EDH+aRSA:!RC4:!aNULL:!eNULL:!LOW:!3DES:!MD5:!EXP:!PSK:!SRP:!DSS tls-dh=prime256v1:/etc/dh-parameters.2048 options=NO_SSLv2,NO_SSLv3,NO_TLSv1,SINGLE_DH_USE,SINGLE_ECDH_USE http_port 127.0.0.1:3128 intercept ssl-bump generate-host-certificates=on dynamic_cert_mem_cache_size=10MB cert=/usr/local/etc/squid/serverkey.pem capath=/usr/local/share/certs/ cipher=EECDH+ECDSA+AESGCM:EECDH+aRSA+AESGCM:EECDH+ECDSA+SHA384:EECDH+ECDSA+SHA256:EECDH+aRSA+SHA384:EECDH+aRSA+SHA256:EECDH+aRSA+RC4:EECDH:EDH+aRSA:!RC4:!aNULL:!eNULL:!LOW:!3DES:!MD5:!EXP:!PSK:!SRP:!DSS tls-dh=prime256v1:/etc/dh-parameters.2048 options=NO_SSLv2,NO_SSLv3,NO_TLSv1,SINGLE_DH_USE,SINGLE_ECDH_USE https_port 127.0.0.1:3129 intercept ssl-bump generate-host-certificates=on dynamic_cert_mem_cache_size=10MB cert=/usr/local/etc/squid/serverkey.pem capath=/usr/local/share/certs/ cipher=EECDH+ECDSA+AESGCM:EECDH+aRSA+AESGCM:EECDH+ECDSA+SHA384:EECDH+ECDSA+SHA256:EECDH+aRSA+SHA384:EECDH+aRSA+SHA256:EECDH+aRSA+RC4:EECDH:EDH+aRSA:!RC4:!aNULL:!eNULL:!LOW:!3DES:!MD5:!EXP:!PSK:!SRP:!DSS tls-dh=prime256v1:/etc/dh-parameters.2048 options=NO_SSLv2,NO_SSLv3,NO_TLSv1,SINGLE_DH_USE,SINGLE_ECDH_USE icp_port 0 digest_generation off dns_v4_first off pid_filename /var/run/squid/squid.pid cache_effective_user squid cache_effective_group proxy error_default_language en icon_directory /usr/local/etc/squid/icons visible_hostname localhost cache_mgr admin@localhost access_log /dev/null cache_log /var/squid/logs/cache.log cache_store_log none netdb_filename /var/squid/logs/netdb.state pinger_enable on pinger_program /usr/local/libexec/squid/pinger sslcrtd_program /usr/local/libexec/squid/ssl_crtd -s /var/squid/lib/ssl_db -M 4MB -b 2048 sslcrtd_children 5 sslproxy_capath /usr/local/share/certs/ sslproxy_options NO_SSLv2,NO_SSLv3,NO_TLSv1,SINGLE_DH_USE,SINGLE_ECDH_USE sslproxy_cipher EECDH+ECDSA+AESGCM:EECDH+aRSA+AESGCM:EECDH+ECDSA+SHA384:EECDH+ECDSA+SHA256:EECDH+aRSA+SHA384:EECDH+aRSA+SHA256:EECDH+aRSA+RC4:EECDH:EDH+aRSA:!RC4:!aNULL:!eNULL:!LOW:!3DES:!MD5:!EXP:!PSK:!SRP:!DSS sslproxy_cert_error allow all sslproxy_flags DONT_VERIFY_PEER logfile_rotate 0 debug_options rotate=0 shutdown_lifetime 3 seconds # Allow local network(s) on interface(s) acl localnet src 192.168.1.0/24 forwarded_for on uri_whitespace strip acl dynamic urlpath_regex cgi-bin ? cache deny dynamic cache_mem 64 MB maximum_object_size_in_memory 256 KB memory_replacement_policy heap GDSF cache_replacement_policy heap LFUDA minimum_object_size 0 KB maximum_object_size 4 MB cache_dir ufs /var/squid/cache 100 16 256 offline_mode off cache_swap_low 90 cache_swap_high 95 cache allow all # Add any of your own refresh_pattern entries above these. refresh_pattern ^ftp: 1440 20% 10080 refresh_pattern ^gopher: 1440 0% 1440 refresh_pattern -i (/cgi-bin/|?) 0 0% 0 refresh_pattern . 0 20% 4320 #Remote proxies # Setup some default acls # ACLs all, manager, localhost, and to_localhost are predefined. acl allsrc src all acl safeports port 21 70 80 210 280 443 488 563 591 631 777 901 3128 3129 1025-65535 acl sslports port 443 563 acl purge method PURGE acl connect method CONNECT # Define protocols used for redirects acl HTTP proto HTTP acl HTTPS proto HTTPS # SslBump Peek and Splice # http://wiki.squid-cache.org/Features/SslPeekAndSplice # http://wiki.squid-cache.org/ConfigExamples/Intercept/SslBumpExplicit # Match against the current step during ssl_bump evaluation [fast] # Never matches and should not be used outside the ssl_bump context. # # At each SslBump step, Squid evaluates ssl_bump directives to find # the next bumping action (e.g., peek or splice). Valid SslBump step # values and the corresponding ssl_bump evaluation moments are: # SslBump1: After getting TCP-level and HTTP CONNECT info. # SslBump2: After getting TLS Client Hello info. # SslBump3: After getting TLS Server Hello info. # These ACLs exist even when 'SSL/MITM Mode' is set to 'Custom' so that # they can be used there for custom configuration. acl step1 at_step SslBump1 acl step2 at_step SslBump2 acl step3 at_step SslBump3 http_access allow manager localhost http_access deny manager http_access allow purge localhost http_access deny purge http_access deny !safeports http_access deny CONNECT !sslports # Always allow localhost connections http_access allow localhost request_body_max_size 0 KB delay_pools 1 delay_class 1 2 delay_parameters 1 -1/-1 -1/-1 delay_initial_bucket_level 100 delay_access 1 allow allsrc # Reverse Proxy settings # Package Integration url_rewrite_program /usr/local/bin/squidGuard -c /usr/local/etc/squidGuard/squidGuard.conf url_rewrite_bypass off url_rewrite_children 16 startup=8 idle=4 concurrency=0 # Custom options before auth ssl_bump peek step1 ssl_bump splice all # Setup allowed ACLs # Allow local network(s) on interface(s) http_access allow localnet # Default block all to be sure http_access deny allsrc
Filter Config:
# ============================================================ # SquidGuard configuration file # This file generated automaticly with SquidGuard configurator # (C)2006 Serg Dvoriancev # email: dv_serg@mail.ru # ============================================================ logdir /var/squidGuard/log dbhome /var/db/squidGuard # dest blk_BL_adv { domainlist blk_BL_adv/domains urllist blk_BL_adv/urls log block.log } # dest blk_BL_aggressive { domainlist blk_BL_aggressive/domains urllist blk_BL_aggressive/urls log block.log } # dest blk_BL_alcohol { domainlist blk_BL_alcohol/domains urllist blk_BL_alcohol/urls log block.log } # dest blk_BL_anonvpn { domainlist blk_BL_anonvpn/domains urllist blk_BL_anonvpn/urls log block.log } # dest blk_BL_automobile_bikes { domainlist blk_BL_automobile_bikes/domains urllist blk_BL_automobile_bikes/urls log block.log } # dest blk_BL_automobile_boats { domainlist blk_BL_automobile_boats/domains urllist blk_BL_automobile_boats/urls log block.log } # dest blk_BL_automobile_cars { domainlist blk_BL_automobile_cars/domains urllist blk_BL_automobile_cars/urls log block.log } # dest blk_BL_automobile_planes { domainlist blk_BL_automobile_planes/domains urllist blk_BL_automobile_planes/urls log block.log } # dest blk_BL_chat { domainlist blk_BL_chat/domains urllist blk_BL_chat/urls log block.log } # dest blk_BL_costtraps { domainlist blk_BL_costtraps/domains urllist blk_BL_costtraps/urls log block.log } # dest blk_BL_dating { domainlist blk_BL_dating/domains urllist blk_BL_dating/urls log block.log } # dest blk_BL_downloads { domainlist blk_BL_downloads/domains urllist blk_BL_downloads/urls log block.log } # dest blk_BL_drugs { domainlist blk_BL_drugs/domains urllist blk_BL_drugs/urls log block.log } # dest blk_BL_dynamic { domainlist blk_BL_dynamic/domains urllist blk_BL_dynamic/urls log block.log } # dest blk_BL_education_schools { domainlist blk_BL_education_schools/domains urllist blk_BL_education_schools/urls log block.log } # dest blk_BL_finance_banking { domainlist blk_BL_finance_banking/domains urllist blk_BL_finance_banking/urls log block.log } # dest blk_BL_finance_insurance { domainlist blk_BL_finance_insurance/domains urllist blk_BL_finance_insurance/urls log block.log } # dest blk_BL_finance_moneylending { domainlist blk_BL_finance_moneylending/domains urllist blk_BL_finance_moneylending/urls log block.log } # dest blk_BL_finance_other { domainlist blk_BL_finance_other/domains urllist blk_BL_finance_other/urls log block.log } # dest blk_BL_finance_realestate { domainlist blk_BL_finance_realestate/domains urllist blk_BL_finance_realestate/urls log block.log } # dest blk_BL_finance_trading { domainlist blk_BL_finance_trading/domains urllist blk_BL_finance_trading/urls log block.log } # dest blk_BL_fortunetelling { domainlist blk_BL_fortunetelling/domains urllist blk_BL_fortunetelling/urls log block.log } # dest blk_BL_forum { domainlist blk_BL_forum/domains urllist blk_BL_forum/urls log block.log } # dest blk_BL_gamble { domainlist blk_BL_gamble/domains urllist blk_BL_gamble/urls log block.log } # dest blk_BL_government { domainlist blk_BL_government/domains urllist blk_BL_government/urls log block.log } # dest blk_BL_hacking { domainlist blk_BL_hacking/domains urllist blk_BL_hacking/urls log block.log } # dest blk_BL_hobby_cooking { domainlist blk_BL_hobby_cooking/domains urllist blk_BL_hobby_cooking/urls log block.log } # dest blk_BL_hobby_games-misc { domainlist blk_BL_hobby_games-misc/domains urllist blk_BL_hobby_games-misc/urls log block.log } # dest blk_BL_hobby_games-online { domainlist blk_BL_hobby_games-online/domains urllist blk_BL_hobby_games-online/urls log block.log } # dest blk_BL_hobby_gardening { domainlist blk_BL_hobby_gardening/domains urllist blk_BL_hobby_gardening/urls log block.log } # dest blk_BL_hobby_pets { domainlist blk_BL_hobby_pets/domains urllist blk_BL_hobby_pets/urls log block.log } # dest blk_BL_homestyle { domainlist blk_BL_homestyle/domains urllist blk_BL_homestyle/urls log block.log } # dest blk_BL_hospitals { domainlist blk_BL_hospitals/domains urllist blk_BL_hospitals/urls log block.log } # dest blk_BL_imagehosting { domainlist blk_BL_imagehosting/domains urllist blk_BL_imagehosting/urls log block.log } # dest blk_BL_isp { domainlist blk_BL_isp/domains urllist blk_BL_isp/urls log block.log } # dest blk_BL_jobsearch { domainlist blk_BL_jobsearch/domains urllist blk_BL_jobsearch/urls log block.log } # dest blk_BL_library { domainlist blk_BL_library/domains urllist blk_BL_library/urls log block.log } # dest blk_BL_military { domainlist blk_BL_military/domains urllist blk_BL_military/urls log block.log } # dest blk_BL_models { domainlist blk_BL_models/domains urllist blk_BL_models/urls log block.log } # dest blk_BL_movies { domainlist blk_BL_movies/domains urllist blk_BL_movies/urls log block.log } # dest blk_BL_music { domainlist blk_BL_music/domains urllist blk_BL_music/urls log block.log } # dest blk_BL_news { domainlist blk_BL_news/domains urllist blk_BL_news/urls log block.log } # dest blk_BL_podcasts { domainlist blk_BL_podcasts/domains urllist blk_BL_podcasts/urls log block.log } # dest blk_BL_politics { domainlist blk_BL_politics/domains urllist blk_BL_politics/urls log block.log } # dest blk_BL_porn { domainlist blk_BL_porn/domains urllist blk_BL_porn/urls log block.log } # dest blk_BL_radiotv { domainlist blk_BL_radiotv/domains urllist blk_BL_radiotv/urls log block.log } # dest blk_BL_recreation_humor { domainlist blk_BL_recreation_humor/domains urllist blk_BL_recreation_humor/urls log block.log } # dest blk_BL_recreation_martialarts { domainlist blk_BL_recreation_martialarts/domains urllist blk_BL_recreation_martialarts/urls log block.log } # dest blk_BL_recreation_restaurants { domainlist blk_BL_recreation_restaurants/domains urllist blk_BL_recreation_restaurants/urls log block.log } # dest blk_BL_recreation_sports { domainlist blk_BL_recreation_sports/domains urllist blk_BL_recreation_sports/urls log block.log } # dest blk_BL_recreation_travel { domainlist blk_BL_recreation_travel/domains urllist blk_BL_recreation_travel/urls log block.log } # dest blk_BL_recreation_wellness { domainlist blk_BL_recreation_wellness/domains urllist blk_BL_recreation_wellness/urls log block.log } # dest blk_BL_redirector { domainlist blk_BL_redirector/domains urllist blk_BL_redirector/urls log block.log } # dest blk_BL_religion { domainlist blk_BL_religion/domains urllist blk_BL_religion/urls log block.log } # dest blk_BL_remotecontrol { domainlist blk_BL_remotecontrol/domains urllist blk_BL_remotecontrol/urls log block.log } # dest blk_BL_ringtones { domainlist blk_BL_ringtones/domains urllist blk_BL_ringtones/urls log block.log } # dest blk_BL_science_astronomy { domainlist blk_BL_science_astronomy/domains urllist blk_BL_science_astronomy/urls log block.log } # dest blk_BL_science_chemistry { domainlist blk_BL_science_chemistry/domains urllist blk_BL_science_chemistry/urls log block.log } # dest blk_BL_searchengines { domainlist blk_BL_searchengines/domains urllist blk_BL_searchengines/urls log block.log } # dest blk_BL_sex_education { domainlist blk_BL_sex_education/domains urllist blk_BL_sex_education/urls log block.log } # dest blk_BL_sex_lingerie { domainlist blk_BL_sex_lingerie/domains urllist blk_BL_sex_lingerie/urls log block.log } # dest blk_BL_shopping { domainlist blk_BL_shopping/domains urllist blk_BL_shopping/urls log block.log } # dest blk_BL_socialnet { domainlist blk_BL_socialnet/domains urllist blk_BL_socialnet/urls log block.log } # dest blk_BL_spyware { domainlist blk_BL_spyware/domains urllist blk_BL_spyware/urls log block.log } # dest blk_BL_tracker { domainlist blk_BL_tracker/domains urllist blk_BL_tracker/urls log block.log } # dest blk_BL_updatesites { domainlist blk_BL_updatesites/domains urllist blk_BL_updatesites/urls log block.log } # dest blk_BL_urlshortener { domainlist blk_BL_urlshortener/domains urllist blk_BL_urlshortener/urls log block.log } # dest blk_BL_violence { domainlist blk_BL_violence/domains urllist blk_BL_violence/urls log block.log } # dest blk_BL_warez { domainlist blk_BL_warez/domains urllist blk_BL_warez/urls log block.log } # dest blk_BL_weapons { domainlist blk_BL_weapons/domains urllist blk_BL_weapons/urls log block.log } # dest blk_BL_webmail { domainlist blk_BL_webmail/domains urllist blk_BL_webmail/urls log block.log } # dest blk_BL_webphone { domainlist blk_BL_webphone/domains urllist blk_BL_webphone/urls log block.log } # dest blk_BL_webradio { domainlist blk_BL_webradio/domains urllist blk_BL_webradio/urls log block.log } # dest blk_BL_webtv { domainlist blk_BL_webtv/domains urllist blk_BL_webtv/urls log block.log } # rew safesearch { s@(google..*/search?.*q=.*)@&safe=active@i s@(google..*/images.*q=.*)@&safe=active@i s@(google..*/groups.*q=.*)@&safe=active@i s@(google..*/news.*q=.*)@&safe=active@i s@(yandex..*/yandsearch?.*text=.*)@&fyandex=1@i s@(search.yahoo..*/search.*p=.*)@&vm=r&v=1@i s@(search.live..*/.*q=.*)@&adlt=strict@i s@(search.msn..*/.*q=.*)@&adlt=strict@i s@(.bing..*/.*q=.*)@&adlt=strict@i log block.log } # acl { # default { pass !blk_BL_gamble !blk_BL_movies !blk_BL_porn all redirect http://192.168.1.1:80/sgerror.php?url=403%20&a=%a&n=%n&i=%i&s=%s&t=%t&u=%u } }
I think what I'm trying to do is pretty basic. Has anyone gotten this working so that it's only Peek and Splice so no certificate is needed and SquidGuard can filter?
-
An Update: I've tried both "Splice All" and a "Custom" set that I've found.
# peek at client TLS-request to get SNI # peek at server cert (for logging) # splice earlySplice at step 3 only ssl_bump peek step1 ssl_bump peek step2 earlySplice ssl_bump splice step3 earlySplice ssl_bump splice all sslproxy_cert_error deny all
but it gives the same issue.
EDIT: SCRATCH THAT. ONCE I RESTARTED SQUID IT WOULDN'T START AT ALL. I HAD TO REMOVE THIS CODE FOR IT TO FIRE UP AGAIN. IT GAVE ME AN ERROR ON LINE 130 WHICH WAS ssl_bump peek step2 earlySplice
I've also tried every combination I can make of the options in Remote Cert Checks and Certificate Adapt.
-
Disabling squidGuard seems to work better so that's likely the reason for the SSL errors. Maybe the error page? On some PCs loading Hulu and Netflix just sit there spinning. On others they load right up. I just don't know why there is such inconsistency across the network. This is a test environment with just a few PCs and a server so nothing fancy and nothing that another wipe would hurt too much.
-
Also you can set up your own DNS servers for Squid in Squid settings - should match what's used on clients. (It doesn't need to use the ones set up in pfSense.)
-
@Stewart did you ever get a final config you were satisfied with.
I wanted the benefit of both caching and filtering and had browsing work (except for android) but my linux emails clients couldn't work. -
All squid can do when it blocks a spliced https page is break the SSL/TLS process.
The event will be logged but the user's browser will behave in different ways there.
-
@gwaitsi
No. I just gave up on it. There was another package being developed that I tinkered with for a bit but it never showed up in the repo so I never moved to it. Any SSL intercept is a MITM attack. It works in our client's Sonicwall units without issue (except for the DOJ) by just a checkbox. I'd put so much time into trying to get this working I just gave up. I didn't realize it'd been 2 years since I last tried getting this working.