Squid question

I have some questions about using squid 2 and squid guard with pfsense on my network. Basically, I new to using proxies and such and from what I have read, following online guides; transparent seems to be the way to go. I know the idea behind proxies are so that all traffic going out to the web is routed through a single port so that traffic can be managed and normally setting have to be applied in your browser so that it knows that it has to go a certain way. With transparent though, I am curious if this browser setup is required or not. Also, now that I have it setup transparently, how can I tell if my computers are actually using the proxy or is this something that is designed to be hidden? In a nutshell, my PC's are all setup using static IP's and I want to make sure that they are communicating properly with my router. Obviously something is right since I can browse the web but outside of this is where my questions fall. Thanks in advance.

chris4916

To me, transparent proxy is the way to avoid.
Main reasons is that transparent proxy does not handle HTTPS unless you enable "Man In The Middle" which basically relies on fake certificates and breaks SSL basic concept.

Explicit proxy is much better. The only drawback is that you have to tell your application (e.g. your browser) that it has to use proxy. This answer to this burden, when you have plenty of devices or applications, is to implement WPAD which aims at providing automatically information about proxy location and behaviour.

I'm only using this on a home network and simple feature are all I really need out of this. Plus, to make explicit work, I have to use wpad and I have read that WPAD can easily be exploited if the .dat file becomes exposed. Simply searching WPAD.dat generator renders lots of hacking related links.

KOM

Could we keep this to just one thread? I've already answered you here.

I tried to delete the old since I started over but I wasn't allowed to delete. By starting over I mean, completely wiped my pfsense installation, reinstalled and completed setup one item at a time. Your answer in the previous was above my understanding level.

KOM

Ask any questions oyu have in the other thread. I think you're making this out in your head to be much harder than it really is.

I'm not ruling that out on any level..lol Im a pfsense noob.

chris4916

@jbhowlesr:

I'm only using this on a home network and simple feature are all I really need out of this.

I doubt… if you don't mind.
Transparent proxy will not handle HTTPS. Thus this is pretty useless.

Plus, to make explicit work, I have to use wpad

no. WPAD is only used, as I wrote, when you want to minimize burden due to manual configuration in browsers.

and I have read that WPAD can easily be exploited if the .dat file becomes exposed. Simply searching WPAD.dat generator renders lots of hacking related links.

Google is not your friend here ;)
I'm not saying there is no risk but especially for purpose you target, risk is very low.
Real risk with WPAD is if someone is able to implement server matching wpad.your_local_domain and redirect HTTP(S) requests to its own "fake" proxy. quite unlikely at home…

what you say makes since but where I am stuck is how to create the file. I read the link you posted and it says it has to be crafted and I assume that means by typing out all the items in word pad and saving it the .dat files. So, not being very network savvy as I am new to this aspect of things, I am not sure what actually needs to be listed. The link shows tidbits and such which I don't know why seems so daunting to me but compound this with not being sure if my setup is working or not and I am sure you can imagine where I stand at this moment.

I configured the basic setting and added LAN and OPT1 interfaces. I enabled DHCP on both LAN and OPT1 but seems to only work on OPT1. Not sure why it is not working on the LAN interface. With the LAN interface though, the only way I could get it to communicate with the WAN interface was by setting LAN to tracked. COuld this be why? I setup SNORT and it is working as it should. I installed SQUID 2 and set it up at first with transparent without proxy putting setting in my browser and the web passes through like you might expect. When I uncheck transparent so it runs in explicit mode, leave browser proxy settings blank, my expectations is that no web traffic should pass without browser proxy settings but it does. At this point in the game is where my understanding fails. I am not sure what is going on or why. I guess the first step here is to workout why it is not working as advertised; if my understanding of proxies is correct. Additionally, the squid service seems to run without any stopping or failure. I should also point out that I run windows 8.1 64 pro on all my computers and server 2012 R2 on my home streaming server. Don't both of these OS's auto negotiate connections by default such as WPAD? Please correct me if I am wrong here on any of this?

chris4916

Please let me try to explain some stuff, not related to pfSense only (BTW my own HTTP proxy doesn't run on pfSense)

transparent proxy has a lot of drawbacks but at least one obvious advantage: as it runs on the default gateway and transparently intercept HTTP protocol (not HTTPS) redirecting it to proxy, deployment is pretty straightforward.
deploying explicit (meaning non-transparent) proxy requires a bit more of technical understanding and attention to various components.

With explicit proxy, your are not obliged to run it where your default gateway is (*). There is not flow interception but this also means, if you want that users are obliged to use your proxy, that you have to prevent direct communication between devices and internet.
As explained in a thread few days ago, it required FW rules authorizing internet access, for HTTP protocol, only from proxy.
I suspect you have not yet implemented it.

I would not reinforce enough my previous message: do not focus on WPAD. This is the very last step of you set-up, totally useless if explicit HTTP proxy doesn't work smoothly.
From scratch and step by step:
1 - configure pfSense with DHCP and DNS and ensure devices are able to access internet (from LAN and WLAN if any)
2 - configure HTTP proxy, set-up browser to explicitly use this proxy and check that you can still access internet
3 - add FW rule preventing access to internet if source is not your proxy

At this point, using proxy is mandatory

5 - if you don't want to configure each and ever device with proxy settings, deploy WPAD

(*) technically speaking, transparent proxy could run like this too but this is not that simple

Decided to give up on squid all together. The web performance is to un predictable. Frankly, the web runs better without it. Don't know why. Soon, I will simply add the proxy service to my server 2012 box and just run it there.

chris4916

@jbhowlesr:

Decided to give up on squid all together. The web performance is to un predictable. Frankly, the web runs better without it. Don't know why.

If goal is performance (I would say only), then browsing internet is quite obviously faster without any proxy.
Reason is that when done directly (i.e. w/o proxy) there is at least one stage less, thus less latency, no anti-virus, blacklist and similar stuff with slow-down impact.
This is specially true nowadays with dynamic nature of internet. Few years ago, content was more static and cache impact on speed (assuming not HTTPS and multiple users) was obvious.

Decision for proxy deployment should, IMO, rather be based on services it adds (like profiling, anti-virus, black-list) rather than speed, internet access through proxy being always slower but this is the cost for additional services.

All this I understand but considered it due to 1 main consideration; which I mentioned in the other post I made. Basically, with things being the way they are, and wanting the best possible school experience for my kids, my wife and I decided to home school our kids via K-12 online. lets' face it, with public education declining and enforcement of common core in practice, kids are being taught politically correct nonsense. I already have a sever running home to stream TV and Movies using a DLNA program and I set it up also to run a Active directory service, DNS, DHCP and WSUS to help keep our network clean and trouble free so it seemed logical to me to be able to filter the web content being viewed by my teenagers since they have their own PC's. Plus with Active Directory, I am able to set login times, block computer features and such which saves me a lot of hassle of yelling at them when they are doing things on the computer and web during school hours that they are not supposed to be doing.

I really don't mind the web being a little slow. However the issue I was having running squid was that the web would come to a stop speed quite frequently, web pages would simply not load even without squid guard loaded which is not good considering school work must be done online. I'm planning on whipping the installation again today since last time I did the easy setup which left setting intact on the drive. I will reload squid proxy when I get it up and going again.

I would like to ask if you could explain to me exactly, step by step please, how to make the wpad files. The links provided don't explain it step by step.

KOM

The pfSense WPAD page has it right there under Create wpad.dat:

The contents of the example wpad.dat file are:

function FindProxyForURL(url,host)
{
return "PROXY 192.168.1.1:3128";
}

It's just a text file with the contents above and nothing more. Replace 192.168.1.1 with your pfSense LAN IP address.

Make a copy of the wpad.dat file and name it proxy.pac

Copy both to the root of an HTTP server (not HTTPS, doesn't work with that). You can use pfSense for this if you aren't running WebGUI in HTTPS mode (copy the files to /usr/local/www).

Add an A record in your DNS that points wpad.YourDomain.whatever to the IP address of the HTTP server where the wpad.dat file is.

(Optional) Add a DHCP Option 252 record and set it to http://wpad.YourDomain.whatever

Test by trying to access http://wpad.YourDomain.whatever/wpad.dat and see if it can resolve and serve it.

That's it. You're done.

If you want, you can explore the other examples of wpad.dat that can control various aspects of proxy redirection, but this should get you going.

once I have the file created, how do I place it in the pfsense directory. I mean, is there a line command feature to enable this? Also, I'm not familiar with what you mean by A record

I think I figured out how to create the files in pfsense. I navigated to Diagnostics/Edit File, chose a random file and renamed it to wpad.dat and save it which created the file. Then while it was still open, I deleted all entries and replaced them with the specified text for the wpad file. I then saved this as wpad.da and proxy.pac. I pressed forward with the link you posted about creating this file and created the firewall rule for port 80 but it black all web traffic. I think this is what you meant by A record. Obviously, something is causing either my computer from getting the auto config or it is not working right. I should also mention my browsers are set to auto detect. You mentioned something about creating a DHCP option but I am not sure how to do this.

Would you mind clarifying the A record thing and the DHCP option. Plus let me know if I created the wpad files correctly. By the way, I gave this a shot "http://wpad.YourDomain.whatever/wpad.dat" and it worked with the setting I entered.

Plus with Active Directory, I am able to set login times, block computer features and such which saves
me a lot of hassle of

Then please it would be really easier to set up the Proxy settings by using the GPOs!
If any PC is an member of an active directory it would be really easy to do so for you.

Using a transparent proxy or a non-transparent proxy is also mostly pending on the needs
you have and the goals you want or must reach. For sure for some peoples also a philosophy.

For a transparent proxy you must bridge ports together to set them in a so called "promiscuous mode"
or at these days more common using the bypass mode function of the NICs that are in use.

But this is mostly earlier or later bringing more or less problems then benefits in my eyes.
And yes the entire speed of the Squid, SquidGuard & HAVP proxy would be significant
slow down the entire packet flow or Internet throughput, here it should be really fast
hardware or clever tuned the proxy.

chris4916

@BlueKobold:

Plus with Active Directory, I am able to set login times, block computer features and such which saves
me a lot of hassle of

Then please it would be really easier to set up the Proxy settings by using the GPOs!
If any PC is an member of an active directory it would be really easy to do so for you.

Sure. For Windows clients only (using IE ony ?) :P

Using a transparent proxy or a non-transparent proxy is also mostly pending on the needs
you have and the goals you want or must reach. For sure for some peoples also a philosophy.

For a transparent proxy you must bridge ports together to set them in a so called "promiscuous mode"
or at these days more common using the bypass mode function of the NICs that are in use.

When using iptables, transparent proxy is achieved forwarding packets to squid.
For sure it depends on needs. Some years ago, if goal was to provide HTTP cache only, I mean without content filtering neither access control, transparent proxy was a valid option. You transparently redirect all HTTP flow to Squid cache and it works. No HTTPS (that is not cached BTW). perfect 8)

But when it comes to provide ACL, profiling, anti-virus, transparent proxy doesn't work. And this is not, IMO ;) a matter of philosophy.

And yes the entire speed of the Squid, SquidGuard & HAVP proxy would be significant
slow down the entire packet flow or Internet throughput, here it should be really fast
hardware or clever tuned the proxy.

I fully share. There is an impact due to introduction of Squid and associated services in the middle. Still, if hardware is sized as expected, impact should be negligible, unless you spend (waste ???) time measuring if there is an impact or not ;D

Would you mind clarifying the A record thing and the DHCP option

chris4916

To be honest, the need for A record is something still not 100% clear in my mind.
Some RFC describe DNS content as to be an A record while some others describe either A record or CNAME.

Frankly, I don't see why it would not work with CNAME.

To rephrase it, if you create an alias in your DNS pointing to your web server as wpad.your_local_domain, it should work.

DHCP option: this is as simple as pushing option 252 with other DHCP informations.
it requires 2 lines in DHCP:

one describing the option itself as "option 252"
another describing its content

see an example here.
If you are using pfSense as your DHCP server, you can just add it to the "option" section of DHCP web interface ;)