"I am now having issues where things load randomly. 99% of stuff loads fine, some things (certain Youtube videos, sometimes pictures on shopping sites), simply don't load at all, ever."
I'm almost positive that it is Snort. The HTTP INSPECT goes wild often and for me anyways when pictures are loading on Amazon for instance this will happen:
#(http_inspect) NO CONTENT-LENGTH OR TRANSFER-ENCODING IN HTTP RESPONSE
suppress gen_id 120, sig_id 3
#(http_inspect) INVALID CONTENT-LENGTH OR CHUNK SIZE
suppress gen_id 120, sig_id 8
#(http_inspect) HTTP RESPONSE GZIP DECOMPRESSION FAILED
suppress gen_id 120, sig_id 6
AND when downloading a file this happens sometimes:
#ET POLICY PE EXE or DLL Windows file download
suppress gen_id 1, sig_id 2000419
I would just try suppressing those. Maybe even clear out your alert list afterward and try accessing the site again and then check your blocklist in Snort to make the necessary suppressions. That's the thing about Snort. It's a wonderful program but it also needs babysitting to make it work right. I have found that if my firewall rules are good then I don't even need it but then again I don't have anything facing the public.
I would also just run one package at a time to see where the problem may be coming from as well.
"After installing/giving up/uninstalling/revisiting a dozen times, I think it has left pfsense in a state where there are artifacts remaining from various packages, and the system is simply not stable or performing. "
This can also be an issue so you're on the right track. I have noticed that for instance with HAVP, if I disable the proxy but I don't clear the checkboxes strange anomalies happen where things would just be very slow etc… Especially if you checked something that uses a RAM DISK. That needs to be unchecked along with any other customizations. I know that other packages have that option so you might want to check that out.
Also, one way to fix some problems is to go to your console and run a shell and then type fsck. I think that you only have to run a shell if your console is password protected. Normally I could just press CTRL C to get the # to popup and then you can type fsck. It will check your file system for integrity problems.
"Hopefully a vanilla install will work. Although after seeing all the things squid is detecting,"
That may be your best bet. Until you get a handle on a package I just wouldn't use it and if you're really concerned about younger users and where they go, HAVP worked very well for me when I needed it for that purpose. Simply because, say they go to a site that you don't want such as something complicated where it's not just zzz.youtube.com or whatever it may be. Say it's zzz.cn.thissite.dontgothere.com Let's say the prefix changes from cn to zb. If you put an item on your blocklist like this the site and the whole domain would not be accessible.
These are the formats that are available for HAVP.
*Enter each destination URL on a new line that will be accessable to the users without scanning. Use '*' symbol for mask. Example: .github.com/, sourceforge.net/clamav-, /.xml, /.inc
So you could type in the blacklist area something like this .thissite.dontgothere.com/ so that even if the prefix changes it's blocked still. You could do it all the way up to just .dontgothere.com/ . HAVP is very powerful in that effect. As you can see, by typing something like /.xml you can block all of xml. You can do the same thing to any extension. You could block anything like .org, .mil, .cn, .php or whatever your fancy is that day. You could essentially do the same thing with the allow list but I don't recommend that. Another thing to consider is to just make your own blacklists.
I have found that downloading blacklists is not nearly accurate enough to provide a lot of use. Also, there is a great set of rules in snort that prevent going to sites that young people shouldn't be going to. Which is emerging-innapropriate.rules. Just enable them all and if there is a problem find which rule is doing it and suppress it. I had to remove that because it did not work for me. Perhaps Dans Guardian would do a better job.
Back to HAVP though. Just like any other package of this sort there will be false positives such as when Adobe flash needs to be updated it will flag it as a virus so that's when you have to do your homework and find out exactly what addresses it needs to do the updating without problems and then use the allow list. Like I said before though. If your Lan rules are golden then you really don't even need these packages. You could just make aliases and block the sites by way of ip address that you don't want people to go to. There's a lot of ways to use pfsense that are made redundant by some packages. Just something to keep in mind. Get used to using the ping tool in pfsense to help with sorting out IP addresses. Then go look it up at CIPB if you want to block an entire IP range via cidr.
Have a good day.
Cmellons
