Squid3 Configuration Trouble

jvamos

Hey there,

I am trying to run a transparent proxy but I can't visit most https websites due to certificate errors and ICAP server errors.
I have followed the guides and installed the certificate here. I think I am having an issue with clamav, there was some errant lines in my config.

Anyone know how to get this up and going? I feel like I must be missing something. I have restarted services after configuration but still no go.

Thanks all,

J

jvamos

I have made some progress but SSL is still giving me issues. Had to select the right interfaces. It seems many of the write ups I have found are incomplete.

KOM

Standard mode and WPAD are a much better way of doing it, and you don't have to screw around with installing certs on every client.

jvamos

Hey KOM,

Thanks for helping me wrap my head around this. The certs are definitely tripping me up here. Or integrating them into the test system. I have an environment where clients step in and out and jump on our wifi and definitely don't want them to have any issues. The WPAD method is a bit intimidating. It looks like it would be best to do manual configuration. The WPAD method has me intrigued.

Tell me one thing, if I enable the proxy with SSL filtering do I need to create a firewall rule and enable port 3129 for it as well?

Also, for WPAD, I have a linux box here running an FTP server that also serves HTTP, I don't want to use this for an external HTTP server for pfSense as it will get confusing, is WPAD method still a good idea?

Thanks, I wish there was better documentation for all of this.

KOM

WPAD sounds harder than it really is. If you try manual then you will end up having to help every single user configure their proxy settings, which they've likely never had to do before.

If you enable SSL filtering you don't have to do anything to the firewall. All rules are added automatically.

Your Linux box is the perfect place to host the wpad.dat file since HTTPS doesn't work with WPAD if I remember right.

Create wpad.dat, wpad.da and proxy.pac on your web server root. Stuff it with:

function FindProxyForURL(url,host)
{
return "PROXY 10.10.4.1:3128";
}

(Replace 10.10.4.1 with your pfSense LAN IP)

Add a DNS entry in your DNS server (pfSense or other) that points wpad.your_domain to your Linux box.

Add a DHCP entry for option 252 in your DHCP server (pfSense or other) and also point wpad.your_domain to your Linux box.

Done.

Don't forget to disable transparent mode on Squid.

jvamos

That sounds great, if I wanted to set this up in a production environment I should probably do all the configuration with the service stopped and bring it online once all the prep is done.

What is the linux server's purpose in this set up. I am wary because I don't want to break anything but I will anyway because trying something new is how we expand our horizons.

The web traffic will flow to the linux box as a proxy? Is this for security reasons? I know having a server running on a firewall is a risk because it's one less layer of admin level protection. Why does this help that situation?

I know I'm asking a lot of questions but I seem to have hooked a big forum fish here…..

If I got this right, the client's browser will automatically try to configure a proxy, WPAD is the mechanism and it will reach out to the DNS server for something that matches that entry on the local network. The browser then receivs that IP and hostname (from wpad.dat) for the proxy via the script? None of this occurs over http or https ports. Correct?

Once again thanks for all the help KOM!

KOM

What is the linux server's purpose in this set up.

To host the 3 autoconfig files. You could also use your pfSense box to do it if it's running HTTP and not HTTPS. Unless you're totally amazing, I can't see how creating three new files in your web server root could cause any problems.

The web traffic will flow to the linux box as a proxy? Is this for security reasons?

No. The traffic doesn't flow to the Linux server. The auto-discover of the proxy server relies on the autoconfig files which must be hosted on a web server. Once the client loads the wpad.dat file from your Linux box and parses it, it will then know to go to the address and port as listed in the wpad.dat file.

When a modern client connects to a network, it will do a lookup on wpad.default_domain, and then contact that IP address on port 80 to fetch wpad.dat, wpad.da or proxy.pac (depending on the app and OS). That's how auto-discovery of the proxy works.

jvamos

Hey KOM thanks,

That clears things up, hopefully I'm not amazing.

The domain setting can't be left blank. Is this critical?

KOM

You need the domain so that the DNS lookup works. It doesn't need to be an AD domain. Local domains will work fine. Do you not have a domain defined in pfSense System - General Setup - Domain?

jvamos

I do, it's set. I thought it needed an AD server I didn't realize the domain was defined on the firewall but I get it now.
I am testing my proxy before launch on my machine and https now works and I can see it in the realtime logs.
This is great news, but http sites are not showing up. I keep receiving ICAP server issues.
Everything on the web I have found is talking about processor issues.
Or not communicating with the server because of```
icap_service_failure_limit -1


This is a 64 bit dual core machine which really is only managing the connections for a handful of machines at a time. Once I get this working I'll move to WDAP.

KOM

I keep receiving ICAP server issues.

I can't help you with that. I turn all that crap off. AV at the firewall just slows everything down. I prefer client-based AV solutions.

jvamos

Thanks KOM,

You were right killing the antivirus (and stopping the services) did make everything work.

I now know where to look!

Thanks for all of your help!