Squid3 transparent proxy - commodo cert?
-
So I utilize squid3 +squidguard3 in transparent proxy mode to filter web traffic in my environment. Everything I read on including https traffic through the proxy requires a self signed certificate from a self created CA, which will throw https errors unless you add your self created CA to the browser's trusted CA list, which I don't always have control over for guest users etc.
My question is: If I purchase something like this: https://ssl.comodo.com/comodo-ssl-certificate.php?key5sk0=1907&key5sk1=f8da6c7f6057dd6850e362d53ac358e30797e07c can it be used in place of the self signed cert/self created CA and not receive the errors?
Will the above only cover my firewall or my entire internal domain? I appreciate anyone willing to provide any clarity.
-
Configure WPAD and stop wasting time with certs:
https://doc.pfsense.org/index.php/WPAD_Autoconfigure_for_Squid
-
Thanks for your quick reply, I've yet to ever look into WPAD, but I believe I understand the concept:
Put squid in non-transparent mode, and push the proxy settings via a dhcp option to all my clients?Do I need to do anything on the clients or will the use the wpad setting pushed out by default? I have guest users who connect android, iphone, ipads, windows/linux/macosx laptops (firefox, opera, IE8,9,10,11) are all known to pop up on our guest networks. Want to make sure they don't receive errors etc, but that they are using the proxy.
-
My question is: If I purchase something like this: https://ssl.comodo.com/comodo-ssl-certificate.php?key5sk0=1907&key5sk1=f8da6c7f6057dd6850e362d53ac358e30797e07c can it be used in place of the self signed cert/self created CA and not receive the errors?
No. There is no cert you can buy that won't generate errors in that scenario. It'd render all of HTTPS useless if you could.
It's a horrible idea in general to MITM SSL traffic. For machines you own, if you want to, knock yourself out I guess. Much better to just force a proxy to be configured on the client though. For guest networks, that's a huge no no.
-
Put squid in non-transparent mode, and push the proxy settings via a dhcp option to all my clients?
That's it. You need DNS and DHCP to fully cover it, but you can get away with just DNS. You also need an HTTP web server to serve the wpad.dat, wpad.da and proxy.pac files. You can use your pfSense box for this if you have WebGUI set to HTTP mode. I have seen some cases where a particular client can't find the proxy even though their system is set to auto-detect the proxy. In those few cases, you manually configure it.
-
So in my setup I am using a layer 3 device behind my pfsense box to hand routing of vlans/subnets etc. I have pfsense and an interface on the layer 3 device sharing a "WANEDGE" subnet with the pfsenses LAN interface.
I have a couple guest networks with ACLs in place preventing traffic from/to these guest networks from the other subnets/vlans.
So while I could use my intranet server to host the wpad config files for the corporate subnets, the guest networks would never b abel to receive the files from that server (by design), which laves me with either throwing up a web server in that same "WANEDGE" subnet or configuring a second lighthttpd isntance in pfsense to host the wpad config files.
My questions are:
1. If I do the 2nd instance in the pfsense box and update pfsense down the road - is that second instance wiped out with the update?
2. For devices without auto configuration enabled - is there a way to redirect to a web page so that I can display instructions on enabling auto configuration? -
1. I have no idea
2. There's a million ways to do everything. You could put up a captive portal with directions there. You could create an exemption in your rules so that everyone can reach your HTTP server and thus the WPAD files.
-
So here is my setup:
Main site: 2 pfsense units in CARP, with lan interfaces sharing a /24 with an interface on a designated "WANEDGE" vlan on 2 layer 3 switches in hot standby mode (Cisco CARP equivilent) downstream which is routing all vlans/subnets on the network. I have a guest subnet/vlan for employee devices, and a guest subnet/vlan for actual guests, as well as 9 other corprate vlans in the layer 3 switches. The guest networks are filtered from eachother and the corporate networks via ACLs in place in the layer 3 switches. All vlans utilizing DHCP are served via the same DHCP server (ip-helpers are in place to allow this), so I can easily serve whatever DHCP options to each scope. The two guest networks are served google's DNS server's for dns, not our internal dns server. A captive portal is in place, which has rules to pass the traffic from the corporate subnets through without hitting the portal, but forcing authentication on the guest networks, allowing employees to sign in with their AD credentials to access the web from their personal mobile devices, and guests/visitors of guests to access the web via voucher codes which last 2 days. All of this is in place and has been working as needed for over a year now.
Two branch locations are identical to main site, with the exception of only having one server (active directory/dns/dhcp), one pfsense unit, and one layer 3 switch at each site (no failover/standby at branch offices), with IPSEC tunnels configured to allow all subnets (guest subnets excluded) to utilize resources/talk to other machines in the subnets at the main site. Each site has its own two guest vlans etc set up identical to the main site as well. The captive portal at these sites is identical to the one at the main site (same html pages were uploaded as used at the main site, but each site does have its own instance), and voucher sync is enabled and syncing with the vouchers in the main site's primary firewall.
Due to the nature of data on the web server hosting our intranet and ticket system etc, it would be non compliant with regulations we are bound by for us to allow any guest traffic to reach it. I also do not want guest traffic from branch offices havign to take up bandwidth in my ipsec tunnels from branch offices using the proxy at the main site (bandwidth at the branch offices is limited). I COULD easily throw up some ubuntu servers running apache in the "WANEDGE" subnets mentioned earlier at each location to host the files though, which all subnets would be able to see (I have no problem doing this - I do not want to run a second instance of lighthttpd in the pfsense boxes due to fear of them getting wiped out if I do a version update (I have installed nano in pfsense before and it was nuked due to a version update - had to install it again. I'm assuming the update would have the same effect on an additional lighthttpd instance as well - dont want to have reinstall + configure this with each update)).
I have called commercial support, and been told that the only way to get squid or squid3 to filter https is to put a CA in place, issue a cert from it and install it in the pfsense unit, and use that to perform a man in the middle attack on all https traffic passing through the proxy, and deal with many certificate errors etc along the way. I was under the impression that that was the case with a proxy in transparent mode, however a configured proxy could be utilized to block https websites on the blacklist as well (in fact when speaking to commercial support when I put these in place over a year ago about web filtering capabilities, I remember being informed that the best way to utilize squid was to run it in non-transparent mode push out the proxy settings via group policy so that https was blocked too when needed - it just wasn't viable due to the guest networks in place as well).
I understand WPAD can replace the group policy piece and make this viable to corporate and guest networks, which is what I am excited about. You are saying doing it utilizing the configured proxy, I will not have to mess with certs and MATM, correct? If so - I am struggling to see what this could look like with the multiple sites each with guest networks and everything in place as described above.
The task at hand is to be able to block social networking and porn, a lot of which get through a transparent squid/squid guard setup due to using https - I would like to allow social networking on the visitor guest vlan eventually, but for now getting the filtering to work for both http and https is a huge leap forward.
Lastly but absolutely the least important: I would like to actually be able to display the "This site has been blocked" message I can set when hitting a blocked http site with squid/squidguard - is that possibel on an https page that is filtered, or just on http?
-
You must have misunderstood ESF. All of what they said is true only when using a transparent proxy. If you are using a standard proxy then you don't have to play around with certs on every client.
Everything these days supports auto-detection of proxy. WPAD is what makes it happen. You will need to spin up an HTTP server(s) to serve the wpad.dat, wpad.da and proxy.pac files. Add a DNS entry for wpad.your_domain.whatever and point it to the web server hosting the proxy files. Add a DHCP 252 entry and give it the same URL. That's pretty much it. For those few users who can't properly detect the proxy, they will have to manually configure it.
If your multiple guest networks cannot touch a common host then you will have to supply an HTTP server for each network.
-
I thought I misunderstood too, but asked in several ways to make sure that's what they were saying. I'm sure Chris had originally told me like you are saying, though.
So with the DNS entry and different sites, etc. there is no one spot I can put a web server that can be hit on all corporate networks AND all guest networks from each site - any clue on how to set that DNS piece up? I've never had to do anything liek that with DNS before.
-
I thought I misunderstood too, but asked in several ways to make sure that's what they were saying. I'm sure Chris had originally told me like you are saying, though.
So with the DNS entry and different sites, etc. there is no one spot I can put a web server that can be hit on all corporate networks AND all guest networks from each site - any clue on how to set that DNS piece up? I've never had to do anything liek that with DNS before.
A better question is - how important is the "wpad.somedoamin.com" entry in this functioning?
I could possibly set up dns forwarders on each firewall to 8.8.8.8 my ISPs dns, then set the dns in dhcp options for the guest scopes to hit he firewalls for DNS, and make a dns overide entry for wpad to point to a web server in my wanedge subnets hosting wpad files only for guest networks, and for corporate networks point them all to my internal web server, because they would see the wpad entry there. In fact this may be what I need to do, due to guest subnets not using my internal dns anyway >_<. Sorry for asking a dumb question.
-
So with the DNS entry and different sites, etc. there is no one spot I can put a web server that can be hit on all corporate networks AND all guest networks from each site - any clue on how to set that DNS piece up?
If these networks can't talk then DNS won't save you. You will need to spin up a web server reachable by everyone if you want to implement WPAD.
A better question is - how important is the "wpad.somedoamin.com" entry in this functioning?
It is critical. The way WPAD works is the client will do a DNS lookup on wpad.domain, and then go to the IP address it gets back and asks for wpad.dat|proxy.pac (depending on the browser, or OS). I don't know if you could get away with just doing it only via DHCP because I've never tried, but it might work. You will still need HTTP servers that everyone can reach. Doesn't need to be the same server. If you have 4 LANs that can't talk to each other, you will need 4 HTTP servers, one on each LAN, to handle WPAD requests.
-
@KOM:
That's it. You need DNS and DHCP to fully cover it, but you can get away with just DNS. You also need an HTTP web server to serve the wpad.dat, wpad.da and proxy.pac files. You can use your pfSense box for this if you have WebGUI set to HTTP mode.
I'm 100% in line with your comments regarding use of WPAD :)
I really don't understand why one would ever use transparent proxy, neither, this is even worst, implement MITM SSL :o ::)This said, in term of implementation, when relying on DNS, at least using "Well Known Aliases" mechanism, I though it requires A record for "wpad.domain", at least according to RFC3040.
As you're not supposed to have multiple DNS "A records" for same IP, the only way you can achieve it is to set an additional IP to your LAN interface so that your http://wpad.domain URL points to it.Am I correct? ???
(WPAD is also described here)
-
I'm not sure. I have my alias as an A record and it seems to work fine. I don't know if it makes a functional difference if it's an A or a CNAME.
-
@KOM:
I'm not sure. I have my alias as an A record and it seems to work fine. I don't know if it makes a functional difference if it's an A or a CNAME.
I don't understand why it would make any difference neither. This is something strange to me and from functional standpoint, I don't see any difference.
However, if client side, DNS requests expects RR type 1 and do not accept type 5, it may fail.As I'm curious, I decided to look further at this because if client implements what RFC describes, then only A record should be supported.
This is what I found: (here) (yes this is only the draft but may explain my previous comment, also I still don('t understand reason behind this)
4.4.3. DNS A/CNAME "Well Known Aliases"
Client implementations MUST support this mechanism. This should be
straightforward since only basic DNS lookup of A records is
required. See RFC 2219 [5] for a description of using "well known"
DNS aliases for resource discovery. We propose the "well known
alias of "wpad" for web proxy auto-discovery.
The client performs the following DNS lookup:
QNAME=wpad.TGTDOM., QCLASS=IN, QTYPE=A
Each A RR, which is returned, contains an IP address which is used
to replace the <host>default in the CURL.
Each candidate CURL so created should be pursued as specified in
section 4.5 and beyond.</host>One step further, reading RFC 2219, I'm lost :-[ because this RFC explains rational using CNAME…
So no real progress here but this may explain why it doesn't always work.Anyone having better understanding ? ???
-
So no real progress here but this may explain why it doesn't always work.
Oh? You have clients that can't find the proxy on their own? I had a few Windows boxes like that and I had to set them to manual.
-
Not currently but I remember I had few some years ago and it pushed me to implement DHCP option 252 and SRV/TXT records.
-
@KOM:
For those few users who can't properly detect the proxy, they will have to manually configure it.
KOM,
How do you get wpad working on mobile devices? Most of my ignores the dns and dhcp config and try to access internet directly. SSL cert for a guest network is also really hard to handle with. -
How do you get wpad working on mobile devices?
I'm not a mobile guy, but it seems to work for me here with Android (5.0) but it is still dumb. While it supports auto-detection, you still must manually give it the URL to the proxy.pac|wpad.dat file. So stupid. Apparently they've never heard of WPAD. I don't have any experience with Apple or Microsoft.
-
@KOM:
Apparently they've never heard of WPAD. I don't have any experience with Apple or Microsoft.
Or don't care about it, the thread is open for years…
https://code.google.com/p/android/issues/detail?id=42696
