Squid3 not setting transparent mode in 2.2



  • I just upgraded to 2.2 and cannot for the life of me get squid to work in transparent mode. I did some poking around and it looks like squid3 is NOT modifying the rules to redirect traffic on port 80. I checked pfctl and the /tmp/rules.debug file and there was nothing to redirect traffic to squid.

    On another pfsense (2.1) install I administer I have these lines in /tmp/rules.debug:

    Setup Squid proxy redirect

    no rdr on xl0 proto tcp from any to { 192.168.0.0/16, 172.16.0.0/12, 10.0.0.0/8 } port 80
    rdr on xl0 proto tcp from any to !(xl0) port 80 -> 127.0.0.1 port 3128

    And pfctl shows the following:

    pfctl -sa | grep 3128
    rdr on xl0 inet proto tcp from any to ! (xl0) port = http -> 127.0.0.1 port 3128
    pass in quick on xl0 proto tcp from any to ! (xl0) port = 3128 flags S/SA keep state

    On my new 2.2 pfsense, there is nothing about squid in rules.debug, and there is no output for pfctl -sa | grep 3128.

    Any ideas?



  • I don't know if it will fix your specific problem, but I've installed Squid 3 more than a dozen time and each time I had to reinstall it to get it to work.  Just install it, go and configure all the Squid tabs, save it and then go back to Packages and click the small PKG button to reinstall it.



  • @KOM:

    I don't know if it will fix your specific problem, but I've installed Squid 3 more than a dozen time and each time I had to reinstall it to get it to work.  Just install it, go and configure all the Squid tabs, save it and then go back to Packages and click the small PKG button to reinstall it.

    I've done that, several times.

    I'm wondering if there's some permissions issue preventing squid from modifying the pf filter set. Can someone point me in the right direction to look at that?



  • In my tests, I was always installing fresh and not upgrading.  Have you tried removing it, deleting the Squid folders and then reinstalling?



  • First point, check if you selected an interface to intercept transparent traffic.

    then check if squid config is fine looking for errors on cache.log and squid -k parse

    On i386 systems, squid package needs a recompile to include transparent mode compile options but on amd64 it's working fine.



  • @marcelloc:

    First point, check if you selected an interface to intercept transparent traffic.

    then check if squid config is fine looking for errors on cache.log and squid -k parse

    On i386 systems, squid package needs a recompile to include transparent mode compile options but on amd64 it's working fine.

    I already had interfaces selected, and I'm running amd64. I found no errors in those 2 places.

    However, I did get it solved, and I think there's a bug in the package. When I initially installed it after the upgrade, it would not stay started. The only apparent error in the system log was this:

    squid: Bungled /usr/pbi/squid-amd64/local/etc/squid/squid.conf line 27: acl localnet src 10.0.0.0/24 0.0.16.0/0.00036537033108885

    My local network is 10.0.0.0/24, but I don't know how the rest of that line is getting generated. My local network was already in the manual ACL, so I just unchecked "Allow users on interface" and then squid seemed to stay running with no errors. Well, that message must've been a red herring. Now when I check "Allow users on interface" I still get that error in the system log, but squid stays running, and more importantly, generates the appropriate pf rules and works in transparent mode! I'm sure one of my package re-installs is what actually fixed squid so it could stay running, but since checking the box kept generating those errors I had left it unchecked.



  • @megamojo:

    squid: Bungled /usr/pbi/squid-amd64/local/etc/squid/squid.conf line 27: acl localnet src 10.0.0.0/24 0.0.16.0/0.00036537033108885

    This localnet is the problem. Did it happened once? After checking all squid config tabs and saving does it wrorg on config file?



  • @marcelloc:

    @megamojo:

    squid: Bungled /usr/pbi/squid-amd64/local/etc/squid/squid.conf line 27: acl localnet src 10.0.0.0/24 0.0.16.0/0.00036537033108885

    This localnet is the problem. Did it happened once? After checking all squid config tabs and saving does it wrorg on config file?

    The config file keeps getting messed up every time the gui saves it. I even stopped the service and manually changed the line in the file to just read

    acl localnet src 10.0.0.0/24

    but as soon as you re-save from the gui it puts it right back to that strange localnet. I thought maybe that second network it adds in is some kind of binary combination using AND or XOR of my 2 interfaces, but I can't make the math work out.

    I have squid listening on 2 interfaces:
    LAN at 10.0.0.0/24
    OpenVPNserver at 10.9.86.0/24

    It's been awhile since I've done complex ACL combining for cisco certifications, but no matter what I tried I couldn't make the math work to combine those subnets to anything close to 0.0.16.0/0.00036537033108885



  • I did some more playing and have a bit more data. Turns out squid doesn't actually run well with that localnet in the acl. You can get it to start intermittently, but it's not reliable. I also determined that the 0.0.16.0/0.00036537033108885 it's defining is 100% from my extra (OPT3 tied to ovpns1) interface I had been binding to squid. If I remove the LAN interface and just have OPT3 it's still there, minus the normal 10.0.0.0/24. I thought maybe the reason it came up with that strange localnet was OPT3 has no IP address configuration, but I added a /24 address on it and squid still came up with the same strange localnet in the config.

    The whole reason that interface exists is I used to have a 20 Mbps upload speed so while mobile it was actually worthwhile to connect via VPN and browse the web through my proxy. I moved and have a paltry 896 Kbps upload, so I just took that interface out of the squid config. With just the LAN interface it works as expected.

    So it's not bothering me anymore, but I'm still curious if someone can determine why it was behaving as it was.



  • The allow subnet option and php function is the since squid2 package. You can add local subnets on acl tab and then unckeck allow local subnets option, then save.



  • @marcelloc:

    The allow subnet option and php function is the since squid2 package. You can add local subnets on acl tab and then unckeck allow local subnets option, then save.

    That's the whole problem though. If I don't check the allow users on interface box, squid doesn't modify the pf rules to make the transparent proxy work.



  • Even if you add localsubnets on acl?



  • @marcelloc:

    Even if you add localsubnets on acl?

    Right. I've been using squid transparently for years, well before that automatic checkbox was around, so I've always had the appropriate networks in the manual ACL. After the upgrade when squid wouldn't start and I saw that error in the log, I want worried at all since the manual ACL was already there. It wasn't until I had time to troubleshoot it that I realized the automatic option seemed to control whether or not squid would make the appropriate modifcations to the pf rules.



  • The code will need a fix to check resulting acl cidrs.



  • Thanks for all the replies! I've gotten it working for what I need and I'm just using it at home, so it's not too big of a deal to me.



  • can someone make a n00b guide for getting squid3 running on 2.2?

    specifically, a transparent proxy with adblocking.



  • I just messed around with ad blocking yesterday. But I didn't use squid3 for it, I used an /etc/hosts file addition.

    • make a directory somewhere convenient, I used: mkdir /usr/local/www/adblock_hosts
    • in a ssh on pfSense, install wget: pkg install wget
    • cd /usr/local/www/adblock_hosts
    • wget http://winhelp2002.mvps.org/hosts.txt
    • in pfSense webconfigurator gui -> Services -> DNS Forwarder -> Advanced, add addn-hosts=/usr/local/www/adblock_hosts/hosts.txt

    Optional:

    • set up a cron job to occasionally (e.g. once a month) update the hosts file

    However, what I found was quite a few sites stopped working. Lots of sites obviously rely on an ad being served before they progress on to the next stage (e.g. serving me the video I requested). So I scrapped the idea. I then went and created my own my_hosts.txt with just a few entries to try it out, google-analytics stuff and a couple of ad servers, a dozen lines in total for now. And then put that dnsmasq addn-hosts line with my_hosts.txt.