Squid question
-
All this I understand but considered it due to 1 main consideration; which I mentioned in the other post I made. Basically, with things being the way they are, and wanting the best possible school experience for my kids, my wife and I decided to home school our kids via K-12 online. lets' face it, with public education declining and enforcement of common core in practice, kids are being taught politically correct nonsense. I already have a sever running home to stream TV and Movies using a DLNA program and I set it up also to run a Active directory service, DNS, DHCP and WSUS to help keep our network clean and trouble free so it seemed logical to me to be able to filter the web content being viewed by my teenagers since they have their own PC's. Plus with Active Directory, I am able to set login times, block computer features and such which saves me a lot of hassle of yelling at them when they are doing things on the computer and web during school hours that they are not supposed to be doing.
I really don't mind the web being a little slow. However the issue I was having running squid was that the web would come to a stop speed quite frequently, web pages would simply not load even without squid guard loaded which is not good considering school work must be done online. I'm planning on whipping the installation again today since last time I did the easy setup which left setting intact on the drive. I will reload squid proxy when I get it up and going again.
I would like to ask if you could explain to me exactly, step by step please, how to make the wpad files. The links provided don't explain it step by step.
-
The pfSense WPAD page has it right there under Create wpad.dat:
The contents of the example wpad.dat file are:
function FindProxyForURL(url,host)
{
return "PROXY 192.168.1.1:3128";
}It's just a text file with the contents above and nothing more. Replace 192.168.1.1 with your pfSense LAN IP address.
Make a copy of the wpad.dat file and name it proxy.pac
Copy both to the root of an HTTP server (not HTTPS, doesn't work with that). You can use pfSense for this if you aren't running WebGUI in HTTPS mode (copy the files to /usr/local/www).
Add an A record in your DNS that points wpad.YourDomain.whatever to the IP address of the HTTP server where the wpad.dat file is.
(Optional) Add a DHCP Option 252 record and set it to http://wpad.YourDomain.whatever
Test by trying to access http://wpad.YourDomain.whatever/wpad.dat and see if it can resolve and serve it.
That's it. You're done.
If you want, you can explore the other examples of wpad.dat that can control various aspects of proxy redirection, but this should get you going.
-
once I have the file created, how do I place it in the pfsense directory. I mean, is there a line command feature to enable this? Also, I'm not familiar with what you mean by A record
-
I think I figured out how to create the files in pfsense. I navigated to Diagnostics/Edit File, chose a random file and renamed it to wpad.dat and save it which created the file. Then while it was still open, I deleted all entries and replaced them with the specified text for the wpad file. I then saved this as wpad.da and proxy.pac. I pressed forward with the link you posted about creating this file and created the firewall rule for port 80 but it black all web traffic. I think this is what you meant by A record. Obviously, something is causing either my computer from getting the auto config or it is not working right. I should also mention my browsers are set to auto detect. You mentioned something about creating a DHCP option but I am not sure how to do this.
Would you mind clarifying the A record thing and the DHCP option. Plus let me know if I created the wpad files correctly. By the way, I gave this a shot "http://wpad.YourDomain.whatever/wpad.dat" and it worked with the setting I entered.
-
Plus with Active Directory, I am able to set login times, block computer features and such which saves
me a lot of hassle ofThen please it would be really easier to set up the Proxy settings by using the GPOs!
If any PC is an member of an active directory it would be really easy to do so for you.Using a transparent proxy or a non-transparent proxy is also mostly pending on the needs
you have and the goals you want or must reach. For sure for some peoples also a philosophy.For a transparent proxy you must bridge ports together to set them in a so called "promiscuous mode"
or at these days more common using the bypass mode function of the NICs that are in use.But this is mostly earlier or later bringing more or less problems then benefits in my eyes.
And yes the entire speed of the Squid, SquidGuard & HAVP proxy would be significant
slow down the entire packet flow or Internet throughput, here it should be really fast
hardware or clever tuned the proxy. -
@BlueKobold:
Plus with Active Directory, I am able to set login times, block computer features and such which saves
me a lot of hassle ofThen please it would be really easier to set up the Proxy settings by using the GPOs!
If any PC is an member of an active directory it would be really easy to do so for you.Sure. For Windows clients only (using IE ony ?) :P
Using a transparent proxy or a non-transparent proxy is also mostly pending on the needs
you have and the goals you want or must reach. For sure for some peoples also a philosophy.For a transparent proxy you must bridge ports together to set them in a so called "promiscuous mode"
or at these days more common using the bypass mode function of the NICs that are in use.When using iptables, transparent proxy is achieved forwarding packets to squid.
For sure it depends on needs. Some years ago, if goal was to provide HTTP cache only, I mean without content filtering neither access control, transparent proxy was a valid option. You transparently redirect all HTTP flow to Squid cache and it works. No HTTPS (that is not cached BTW). perfect 8)But when it comes to provide ACL, profiling, anti-virus, transparent proxy doesn't work. And this is not, IMO ;) a matter of philosophy.
And yes the entire speed of the Squid, SquidGuard & HAVP proxy would be significant
slow down the entire packet flow or Internet throughput, here it should be really fast
hardware or clever tuned the proxy.I fully share. There is an impact due to introduction of Squid and associated services in the middle. Still, if hardware is sized as expected, impact should be negligible, unless you spend (waste ???) time measuring if there is an impact or not ;D
-
Would you mind clarifying the A record thing and the DHCP option
-
To be honest, the need for A record is something still not 100% clear in my mind.
Some RFC describe DNS content as to be an A record while some others describe either A record or CNAME.Frankly, I don't see why it would not work with CNAME.
To rephrase it, if you create an alias in your DNS pointing to your web server as wpad.your_local_domain, it should work.
DHCP option: this is as simple as pushing option 252 with other DHCP informations.
it requires 2 lines in DHCP:- one describing the option itself as "option 252"
- another describing its content
see an example here.
If you are using pfSense as your DHCP server, you can just add it to the "option" section of DHCP web interface ;) -
Sure. For Windows clients only (using IE ony ?) :P
My answer was based on the statement shown in the next Quote 8)
I should also point out that I run windows 8.1 64 pro on all my computers and server 2012 R2
-
A CNAME is an alias for an existing A record.
http://www.zytrax.com/books/dns/
-
Got it all and I think my router is now properly configured with both squid 2 and squid guard. One question though; and I can't stress enough how much of an ear full I'm getting from my wife about this issue, is that the internet seems to now crawl to a stop since running these programs in pfsense. I don't mean it runs super slow, I mean if you reboot the router the web is fast. It will runs fast for a few minutes then it start slowing down until it comes to a stop where now web pages will load yet we are still connected to the internet. I am assuming this has something to do with web caching. I had left it set at the default 100mb and the internet had stopped like above. I changed it to 200 and end it is moving quickly again but not sure if it will keep the internet moving as it should. Any recommendations on this. Additionally, I wasn't sure what most of these setting did so I left them as default. What else should I change. See screen shots attached:
EDIT: I forgot to add here that our TV's run through the same LAN port as our pc's and my wife use the smart apps on the TV to watch Netflix and amazon prime streaming services. Is it possible that this is what is filling up the cache? If so, how do I stop the router from caching them?
EDIT 2: Under the Squid\Cache Mgmt, there is an option labeled " Hard Disk Cache System " with a drop down. I tried setting this the "diskd" to see what happens but I am curious what the affects/impacts are if I set it to null. There is a description for each and null says do not use any storage. Obviously this means it caches nothing but what will be the impact or affects of setting to null?
-
Is it possible that this is what is filling up the cache?
This could really being.
If so, how do I stop the router from caching them?
Click please on the second piture in your last posting (Untiteld1.jpg)
Please have a look at the bottom line, the last entry Do not cache
Place the right entries of the TV or streaming providers that must match
this and I think you should also reboot the pfSense then. -
I get that, I searched for Netflix IP and such and came up with a massive list:
108.175.32.0/24 Netflix Streaming Services Inc. 256
108.175.33.0/24 Netflix Streaming Services Inc. 256
108.175.34.0/24 Netflix Streaming Services Inc. 256
108.175.35.0/24 Netflix Streaming Services Inc. 256
108.175.38.0/24 Netflix Streaming Services Inc. 256
108.175.39.0/24 Netflix Streaming Services Inc. 256
108.175.40.0/24 Netflix Streaming Services Inc. 256
108.175.41.0/24 Netflix Streaming Services Inc. 256
108.175.42.0/24 Netflix Streaming Services Inc. 256
108.175.43.0/24 Netflix Streaming Services Inc. 256
108.175.44.0/24 Netflix Streaming Services Inc. 256
108.175.46.0/24 Netflix Streaming Services Inc. 256
108.175.47.0/24 Netflix Streaming Services Inc. 256
185.2.220.0/24 Netflix Streaming Services Inc. 256
185.2.221.0/24 Netflix Streaming Services Inc. 256
185.2.222.0/24 Netflix Streaming Services Inc. 256
185.2.223.0/24 Netflix Streaming Services Inc. 256
185.9.188.0/24 Netflix Streaming Services Inc. 256
185.9.189.0/24 Netflix Streaming Services Inc. 256
185.9.190.0/23 Netflix Streaming Services Inc. 512
192.173.112.0/20 Netflix Streaming Services Inc. 4,096
192.173.64.0/20 Netflix Streaming Services Inc. 4,096
192.173.64.0/24 Netflix Streaming Services Inc. 256
192.173.80.0/20 Netflix Streaming Services Inc. 4,096
192.173.96.0/20 Netflix Streaming Services Inc. 4,096
198.38.100.0/24 Netflix Streaming Services Inc. 256
198.38.101.0/24 Netflix Streaming Services Inc. 256
198.38.102.0/23 Netflix Streaming Services Inc. 512
198.38.102.0/24 Netflix Streaming Services Inc. 256
198.38.108.0/24 Netflix Streaming Services Inc. 256
198.38.109.0/24 Netflix Streaming Services Inc. 256
198.38.110.0/24 Netflix Streaming Services Inc. 256
198.38.111.0/24 Netflix Streaming Services Inc. 256
198.38.112.0/24 Netflix Streaming Services Inc. 256
198.38.113.0/24 Netflix Streaming Services Inc. 256
198.38.114.0/24 Netflix Streaming Services Inc. 256
198.38.115.0/24 Netflix Streaming Services Inc. 256
198.38.116.0/24 Netflix Streaming Services Inc. 256
198.38.117.0/24 Netflix Streaming Services Inc. 256
198.38.118.0/24 Netflix Streaming Services Inc. 256
198.38.119.0/24 Netflix Streaming Services Inc. 256
198.38.120.0/24 Netflix Streaming Services Inc. 256
198.38.121.0/24 Netflix Streaming Services Inc. 256
198.38.122.0/24 Netflix Streaming Services Inc. 256
198.38.123.0/24 Netflix Streaming Services Inc. 256
198.38.124.0/24 Netflix Streaming Services Inc. 256
198.38.125.0/24 Netflix Streaming Services Inc. 256
198.38.96.0/24 Netflix Streaming Services Inc. 256
198.38.97.0/24 Netflix Streaming Services Inc. 256
198.38.98.0/24 Netflix Streaming Services Inc. 256
198.38.99.0/24 Netflix Streaming Services Inc. 256
198.45.48.0/24 Netflix Streaming Services Inc. 256
198.45.49.0/24 Netflix Streaming Services Inc. 256
198.45.52.0/24 Netflix Streaming Services Inc. 256
198.45.53.0/24 Netflix Streaming Services Inc. 256
198.45.54.0/24 Netflix Streaming Services Inc. 256
198.45.55.0/24 Netflix Streaming Services Inc. 256
198.45.56.0/24 Netflix Streaming Services Inc. 256
198.45.57.0/24 Netflix Streaming Services Inc. 256
198.45.58.0/24 Netflix Streaming Services Inc. 256
198.45.61.0/24 Netflix Streaming Services Inc. 256
198.45.62.0/24 Netflix Streaming Services Inc. 256
198.45.63.0/24 Netflix Streaming Services Inc. 256
208.75.77.0/24 Netflix Streaming Services Inc. 256
23.246.10.0/24 Netflix Streaming Services Inc. 256
23.246.11.0/24 Netflix Streaming Services Inc. 256
23.246.12.0/24 Netflix Streaming Services Inc. 256
23.246.13.0/24 Netflix Streaming Services Inc. 256
23.246.14.0/24 Netflix Streaming Services Inc. 256
23.246.15.0/24 Netflix Streaming Services Inc. 256
23.246.16.0/24 Netflix Streaming Services Inc. 256
23.246.17.0/24 Netflix Streaming Services Inc. 256
23.246.18.0/24 Netflix Streaming Services Inc. 256
23.246.20.0/24 Netflix Streaming Services Inc. 256
23.246.2.0/24 Netflix Streaming Services Inc. 256
23.246.22.0/24 Netflix Streaming Services Inc. 256
23.246.23.0/24 Netflix Streaming Services Inc. 256
23.246.24.0/24 Netflix Streaming Services Inc. 256
23.246.25.0/24 Netflix Streaming Services Inc. 256
23.246.26.0/24 Netflix Streaming Services Inc. 256
23.246.27.0/24 Netflix Streaming Services Inc. 256
23.246.28.0/22 Netflix Streaming Services Inc. 1,024
23.246.28.0/24 Netflix Streaming Services Inc. 256
23.246.29.0/24 Netflix Streaming Services Inc. 256
23.246.30.0/24 Netflix Streaming Services Inc. 256
23.246.3.0/24 Netflix Streaming Services Inc. 256
23.246.31.0/24 Netflix Streaming Services Inc. 256
23.246.36.0/24 Netflix Streaming Services Inc. 256
23.246.37.0/24 Netflix Streaming Services Inc. 256
23.246.4.0/24 Netflix Streaming Services Inc. 256
23.246.5.0/24 Netflix Streaming Services Inc. 256
23.246.58.0/24 Netflix Streaming Services Inc. 256
23.246.59.0/24 Netflix Streaming Services Inc. 256
23.246.6.0/24 Netflix Streaming Services Inc. 256
23.246.62.0/24 Netflix Streaming Services Inc. 256
23.246.63.0/24 Netflix Streaming Services Inc. 256
23.246.7.0/24 Netflix Streaming Services Inc. 256
23.246.8.0/24 Netflix Streaming Services Inc. 256
23.246.9.0/24 Netflix Streaming Services Inc. 256
37.77.184.0/24 Netflix Streaming Services Inc. 256
37.77.185.0/24 Netflix Streaming Services Inc. 256
37.77.186.0/24 Netflix Streaming Services Inc. 256
37.77.187.0/24 Netflix Streaming Services Inc. 256
37.77.188.0/24 Netflix Streaming Services Inc. 256
37.77.189.0/24 Netflix Streaming Services Inc. 256
37.77.190.0/24 Netflix Streaming Services Inc. 256
37.77.191.0/24 Netflix Streaming Services Inc. 256
64.120.128.0/17 Netflix Streaming Services Inc. 32,768
66.197.128.0/17 Netflix Streaming Services Inc. 32,768
69.53.224.0/24 Netflix Streaming Services Inc. 256
69.53.225.0/24 Netflix Streaming Services Inc. 256
69.53.226.0/24 Netflix Streaming Services Inc. 256
69.53.229.0/24 Netflix Streaming Services Inc. 256
69.53.231.0/24 Netflix Streaming Services Inc. 256
69.53.234.0/24 Netflix Streaming Services Inc. 256
69.53.236.0/24 Netflix Streaming Services Inc. 256
69.53.237.0/24 Netflix Streaming Services Inc. 256
69.53.238.0/24 Netflix Streaming Services Inc. 256
69.53.249.0/24 Netflix Streaming Services Inc. 256
69.53.255.0/24 Netflix Streaming Services Inc. 256If delete all but the IP address and past them into that box, squid guard errors out when i try to save it. If i past the Netflix in there is accepts that but i have no idea if that works since the site shows the service have different names.
https://ipinfo.io/AS2906
I have also been looking for amazon prime streaming service IP but unable to locate those. Additionally, I need the same for PlayStation network as well.
-
You can also be inserting the domain only and then I think all IPs that will be used by this domain name
will also not to be cached! Would be much easier as millions of IP addresses to insert in. Like this;https://openconnect.netflix.com
https://www.netflix.com/
bgp.he.net/AS2906 -
Should I select null for caching? How do I make this allow PSN traffic?
-
How do I make this allow PSN traffic?
Perhaps sniffing with WireShark in your network which IP addresses will be used for this
if a game is started and then you could enter those IP addresses or you find out the domain
from the PSN. -
Been witnessing some strange behavior everyday for the last few days. Basically, squid and squid guard service stops for some unknown reason. Messing around with the setting doesn't seem to make it start and stay started. The first day it happened I was scratching my head for over an our, messing with stuff, I decided to click on update blacklist and then both services restarted on there own and will stay started till the next day. What is the cause of this? How do I fix it?
-
Another thing I am noticing. I installed lightsquid to mess around with it. I have configured wpad as mentioned earlier but when I open the proxy report for lightsquid, my computers don't show up in the real time report. If I I set the proxy setting in my browser though, my pc pops up right away in the proxy report. Why is this? Is my wpad configured correctly or is this normal behavior?
-
I found this which concerns auto updating the blacklists. Can you explain exactly how to do this?
https://forum.pfsense.org/index.php?topic=35479.0
It is from 2011 and I am not sure if it is relevant anymore.
-
I would to be truthful about it, if you starts installing a firewall such as pfSense and configuring this then
later with some clicks and it works for you is not in my meaning to be proper with or familiar with.
If things going deeper you will find out very fast that pfSense is very powerful on the one site but
also very complex and not a lightweight. So many things can really be false but the entire pfSense
is up and running proper for you. But if then things such Squid & SquidGuard or perhaps snort
coming on top of this it would be never able to find out that something is not matching in pfSense
correctly.So I really suggest you now the following,
- bring up the pfSense firewall stable, smooth and liquid running
– then save the settings (from time to time and at the end) - Set then up Squid & SquidGuard and bring them also liquid running
-- save this settings (from time to time and at the end) and so on.
Basically, squid and squid guard service stops for some unknown reason.
And this is exactly what I was talking about some lines above!
– with no saved config, you can not easily jump back to a well known working configuration
-- you will be not absolutely sure that the pfSense configuration is not the guilty oneBut on the other side you do one question after the next one, and more, and more, and more
and then at one time no one will be able to some closer to the point to help you.One tip at least from me on that, please start one thread and if this one is solved and/or clear
then please start the next one please under another topic so peoples would be easily jump in
and get a quick overview and is able to bing the solution to you. This is not the willing to bother
with you, it is more another very but truthful way to help you out of your situation and not let you
deeper and deeper running in the forest of configurations. Step by Step is the solution.I really don´t want to come to near to you and related to my poor english language skills it
could be sounding a little bit strange, trust me please it is not so.I don´t know what you want to do in the winter time, but this is the time peoples often reading
books! So would it be in your budget to get two or three book about this themes?
pfSense the definitive guide
Squid a beginners guide
snort IDS/IPS toolkit - bring up the pfSense firewall stable, smooth and liquid running
