I am now creating new DansGuardian and Squid3 binaries.



  • Update: After checking things again, there are a few missing libraries. Look in the link above, and also grab the squid-libs.tar.gz

    Extract the contents to /usr/lib/

    If anyone is using a windows box to try to get these files onto a pfsense box, try http file server or "HFS". Its is free, and very convenient. You can then "fetch" the files from your windows machine to your pfSense system.



  • @Cino:

    @Legion:

    If you're inclined, it would be cool if you made sure squidguard was updated and worked with your squid-3.4 as well.

    a little off topic, but what do you mean? squidguard itself hasn't been updated in a couple of years, its 1.5 beta on their website…

    Yeah, I would hope that there would be no problem but I thought that maybe squidguard (and other packages) would have to be updated to their FreeBSD 10.x versions so while he was working with new Squid 3.4 he could test that the associated packages work as well. By the sounds of it dansguardian is the focus but I am using squidguard at the moment rather than DG since I found it more suitable for my needs.



  • I had previously tried SquidGuard. Although it seemed to work, I need to be able to auto-update the blacklist, among other things. Even though DansGuardian isn't quite stable, it seemed to have more potential than SquidGuard, once finished.

    I'd be happy to check out e2guardian, but one thing at a time.

    Lastly, this is mostly directed at jimp, but anyone else interested is welcome to use as desired.

    I have build instructions for building from source.

    You will need to grab the 'ports' option/project/whatever for FreeBSD 10.1. This can be done either at the point of install, or post-installation, with the commands:

    portsnap fetch
    portsnap extract
    

    Then, cd into the /usr/ports folder. cd further into the pkg-mgmt folder, and do amake install. If you already have the pkg binary installed, you will instead need to run```
    make reinstall

    
    Then, cd to /usr/ports/www/squid . Run```
    make PREFIX=/usr/pbi/squid-[b][i]archetecture[/i][/b]/ install
    ```.
    
    You will be presented with some dialog boxes the first time around, I used the following settings:
    
    > ARP_ACL=on: ARP/MAC/EUI based authentification
    >     AUTH_KERB=on: Install Kerberos authentication helpers
    >     AUTH_LDAP=on: Install LDAP authentication helpers
    >     AUTH_NIS=off: Install NIS/YP authentication helpers
    >     AUTH_SASL=off: Install SASL authentication helpers
    >     AUTH_SMB=off: Install SMB auth. helpers (req. Samba)
    >     AUTH_SQL=off: Install SQL based auth (uses MySQL)
    >     CACHE_DIGESTS=on: Use cache digests
    >     DEBUG=off: Build with extended debugging support
    >     DELAY_POOLS=on: Delay pools (bandwidth limiting)
    >     DNS_HELPER=on: Use external dnsserver processes for DNS
    >     DOCS=off: Build and/or install documentation
    >     ECAP=off: Loadable content adaptation modules
    >     ESI=on: ESI support
    >     EXAMPLES=off: Build and/or install examples
    >     FOLLOW_XFF=on: Support for the X-Following-For header
    >     FS_AUFS=on: AUFS (threaded-io) support
    >     FS_DISKD=on: DISKD storage engine controlled by separate service
    >     FS_ROCK=off: ROCK (unstable)
    >     HTCP=on: HTCP support
    >     ICAP=on: the ICAP client
    >     ICMP=on: ICMP pinging and network measurement
    >     IDENT=off: Ident lookups (RFC 931)
    >     IPV6=on: IPv6 protocol support
    >     KQUEUE=on: Kqueue(2) support
    >     LARGEFILE=on: Support large (>2GB) cache and log files
    >     LAX_HTTP=off: Do not enforce strict HTTP compliance
    >     SNMP=off: SNMP support
    >     SSL=on: SSL gatewaying support
    >     SSL_CRTD=on: Use ssl_crtd to handle SSL cert requests
    >     STACKTRACES=on: Enable automatic backtraces on fatal errors
    >     TP_IPF=off: Transparent proxying with IPFilter
    >     TP_IPFW=off: Transparent proxying with IPFW
    >     TP_PF=on: Transparent proxying with PF
    >     VIA_DB=off: Forward/Via database
    >     WCCP=on: Web Cache Coordination Protocol
    >     WCCPV2=on: Web Cache Coordination Protocol v2
    
    It will also ask you about options for the various libraries needed by squid, and will compile them along the way. I didn't make notes of my answers for these, but they shouldn't be too hard to figure out. Usually, just use the defaults. Turn off documentation and debugging options, etc., but mostly the defaults. Depending on your hardware, this may take 20 minutes or so to finish compiling.

  • Rebel Alliance Developer Netgate

    If you look in pkg_config.10.xml you'll notice for squid3 there are some build options made from variables similar to the ones you see there. If you (or someone else) can turn the list of yours into the build options style and submit a pull request, we can recompile it with the new options to see if it works.



  • JimP,

    I can try that, but as I was using the FreeBSD "ports" repo for testing, there is no ./configure file until after running make. In this scenario, "make install" is the only command needed, and that creates the "configure" program, adds the build options, configures, compiles, and links the program and all requisite libraries. This makes it easy and convenient to build, but not so easy to pass configure options to somebody else. I guess I'll try to build from official Squid source, but keep in mind, that that source doesn't have all of the FreeBSD 10 specific patches and compatibility testing which is included in the Ports repo.

    Stay tuned.


  • Rebel Alliance Developer Netgate

    The build options in the xml are the build options from freebsd ports, not configure. You can also see them in /var/db/ports/<portname>/options, where portname is xxx_yyy, xxx = the ports category, yyy being the port name, such as www_squid</portname>



  • Interesting. Good to know! I'll check that in the morning.



  • Sorry for the silence this weekend. My computer was having video issues on Friday. On Saturday, I made some new VMs, based on FreeBSD 10.2 RC1, instead of Beta 3 as before. Build process is erroring-out on glib20, while running "make" in "/usr/ports/ports-mgmt/". I am unable to compile and install the updated "pkg" binary, which is required by the rest of the ports repo. This is occurring in both amd64 and i386 versions. I'll update "ports" tomorrow and try again, to see if this is fixed.

    Thank you for your continued patience.


  • Rebel Alliance Developer Netgate

    pkg can be a little odd to upgrade sometimes, but it usually prints instructions about it when it fails.

    try this:

    pkg delete -f pkg; cd /usr/ports/ports-mgmt/pkg; env UPGRADEPKG=1 make clean install clean
    


  • @Jimp: No, I don't think that's it. I am having no end of headaches with ports on RC1. Things that worked without issues. Squid, Avahi, pkg, and more are constantly getting stuck while building dependencies. It is not happening during install phase. Also, the same errors are occurring in the same places on both i386 and amd64 builds. Everything from "aclocal 1.13 missing", to missing dependencies during configure, because they are supposed to be compiled ahead of time, but aren't. None of this was happening with beta 3. The fix has always been to look at the logs and figure out where the process is breaking down and cd to that location and "make clean && make install". Compiling squid alone took more than 3 hours, and I had to manually build about 20 different dependencies. Every time I'd fix one thing, something else further down the line would fail.

    I then spend just under 4 hours on Avahi before calling it a night. Going to try to finish up in a few minutes.

    All of this was with a fresh, clean install of freebsd 10.1 - not an upgrade or install over-the-top of the existing.


  • Rebel Alliance Developer Netgate

    Strange, I haven't used an RC yet myself, my workstation is on a late beta, but there were a ton of updates to ports over the last couple weeks.

    When all else fails, pkg delete -fa, and start over with a fresh ports tree (portsnap fetch extract)



  • I've been trying to resolve them manually. That said, I just did a portsnap fetch and portsnap update, followed closely behind by a few _make distclean_s. Portsnap fetch grabbed over 200 patches to my existing tree. Crossing my fingers…



  • Yeah, going to start with a new ports install. Running rm -rf /usr/ports now. Just ran pkg delete -fa a moment ago. Thanks for that, by the way. I didn't previously know I could force remove all pkgs! I'm assuming that's what is meant by -fa (force all).



  • I'm back on the case.

    FreeBSD was updated to RC2 on Friday. I had some things going over the weekend. It's now about midnight Monday morning. GTG.



  • Aaron - Thanks for working on this.  Any update on progress.  I am looking to install and would appreciate any guidance you can provide.
    -Chanaka



  • Actually, I've stopped working on this, as the original maintainer of squid and DansGuardian, namely user MarcelloC, managed to find the time to update them about a month ago. I assume you're having trouble? If so, you're in the right place… (Pfsense forums).



  • @aaronouthier:

    Actually, I've stopped working on this, as the original maintainer of squid and DansGuardian, namely user MarcelloC, managed to find the time to update them about a month ago. I assume you're having trouble? If so, you're in the right place… (Pfsense forums).

    That's awesome news… so now the normal Squid3 and DG packages should work under 2.2?



  • Do you know where he updated them too?

    I checked github for php/inc file changes: squid3 hasn't been updated in 2 months, Dansguardian 5 months.

    binary changes:
    Checked http://files.pfsense.org/packages/8/All/
    dansguardian-2.12.0.3_2-i386.pbi                  23-Jun-2014 13:57            19952423
    squid-3.3.10-i386.pbi                              26-Nov-2013 20:06            17598644

    Checked http://files.pfsense.org/packages/10/All/
    dansguardian-2.12.0.3_2-i386.pbi                  27-Jun-2014 03:42            16177170
    squid-3.3.11-i386.pbi                              22-Apr-2014 12:12            17568448
    squid-3.3.11_1-i386.pbi                            17-Jul-2014 22:26            17702572

    amd64 pbi have the same dates



  • Ok. I did some more checking, and, now I'm not sure who updated it, or when. I just know that around October 15 or so, I reinstalled my box, and everything worked, whereas a fresh install previously didn't work right without some modifications.

    Also, note that I am using squid3-dev, not regular squid3, and I am running it on the 2.2 beta, not the 2.1.x stable.



  • I have a fresh install of 2.2-BETA (amd64) built on Mon Oct 27 15:31:41 CDT 2014 FreeBSD 10.1-RC3
    If I install squid3-dev beta 3.3.11_1 pkg 2.2.7 platform: 2.2 - I've never managed to start it.

    On the previous install, I tried installing libraries it complained were missing ect. to see if I could get it up,
    but eventually I gave up, and reinstalled from scratch.

    Would you mind sharing which versions you're running?

    Thanks in advanced.



  • More or less the same situation here, I'm running 2.2 beta snapshot and tried to install Squid 3.3.11_1 pkg 2.2.7, it wouldn't start.
    I used the workaround described elsewhere on this forum, and now it runs.
    Downside is that the "workaround" (console commandos) have to be entered again after each update.

    So it's either "do not update" or "workaround".

    For my purposes, Squid proxy (and if possible with ad blocking) is really a must-have. 
    I'm not a programmer, but can test packages if needed.
    Please keep up the effort.

    Cheers.



  • Hmm. Not sure what happened. I just did a fresh reinstall myself. Squid now segfaults upon launch with core dump. This is with the official versions of everything. Nothing was custom-compiled or copied from another box. Amd64 build. I don't know what to say.



  • Exactly.
    It did that on my box also, but it turned out to be the cache filesystem.
    If you set it to "aufs", Squid will complain.
    Leaving it at "ufs" (default) and it runs.

    I have found a way to block ads with the help of a regex list added to Squid, and that works fine.
    So for now, all is dandy. No updates though.

    Cheers.



  • Well, I am having some frustrating issues. I have gotten squid to compile just fine, but when I do amake install, it hangs with a series of```
    lstat: file not found



  • Hmm I'm not familiar with the process, wish I could help in some way.
    Take your time.

    Cheers.



  • Today, a new Squid package was made available.
    As I was feeling adventurous, decided to hit the pkg button and….
    Installed just fine, all configs retained. No errors.

    I haven't tried to update the beta snapshot yet, better wait until it gets final.

    Cheers.



  • No luck on my end, but I am running the latest beta…

    2.2-BETA (amd64)
    built on Fri Nov 07 13:54:45 CST 2014
    FreeBSD 10.1-RC4-p1

    squid3-dev
    3.3.11_1 pkg 2.2.8

    Last 50 system log entries
    Nov 8 11:42:45 php-fpm[50841]: /rc.filter_configure_sync: SQUID is installed but not started. Not installing "filter" rules.
    Nov 8 11:42:44 php-fpm[50841]: /rc.filter_configure_sync: SQUID is installed but not started. Not installing "pfearly" rules.
    Nov 8 11:42:44 php-fpm[50841]: /rc.filter_configure_sync: SQUID is installed but not started. Not installing "nat" rules.
    Nov 8 11:42:43 check_reload_status: Reloading filter
    Nov 8 11:42:33 kernel: pid 85517 (squid), uid 0: exited on signal 11 (core dumped)
    Nov 8 11:42:33 php-fpm[92516]: /pkg_edit.php: The command '/usr/pbi/squid-amd64/sbin/squid -f /usr/pbi/squid-amd64/etc/squid/squid.conf' returned exit code '139', the output was ''
    Nov 8 11:42:32 php-fpm[92516]: /pkg_edit.php: Starting Squid
    Nov 8 11:42:32 kernel: pid 81637 (squid), uid 0: exited on signal 11 (core dumped)
    Nov 8 11:42:32 php-fpm[92516]: /pkg_edit.php: The command '/usr/pbi/squid-amd64/sbin/squid -z -f /usr/pbi/squid-amd64/etc/squid/squid.conf' returned exit code '139', the output was ''
    Nov 8 11:42:31 kernel: pid 77608 (squid), uid 0: exited on signal 11 (core dumped)
    Nov 8 11:42:31 php-fpm[92516]: /pkg_edit.php: The command '/usr/pbi/squid-amd64/sbin/squid -k kill -f /usr/pbi/squid-amd64/etc/squid/squid.conf' returned exit code '139', the output was ''
    Nov 8 11:42:26 kernel: pid 57759 (squid), uid 0: exited on signal 11 (core dumped)
    Nov 8 11:42:26 php-fpm[92516]: /pkg_edit.php: The command '/usr/pbi/squid-amd64/sbin/squid -k shutdown -f /usr/pbi/squid-amd64/etc/squid/squid.conf' returned exit code '139', the output was ''
    Nov 8 11:42:25 php-fpm[92516]: /pkg_edit.php: Creating squid cache subdirs in /var/squid/cache
    Nov 8 11:42:25 php-fpm[25084]: /rc.filter_configure_sync: SQUID is installed but not started. Not installing "filter" rules.
    Nov 8 11:42:25 php-fpm[25084]: /rc.filter_configure_sync: SQUID is installed but not started. Not installing "pfearly" rules.
    Nov 8 11:42:25 php-fpm[25084]: /rc.filter_configure_sync: SQUID is installed but not started. Not installing "nat" rules.
    Nov 8 11:42:24 php-fpm[25084]: /rc.filter_configure_sync: SQUID is installed but not started. Not installing "filter" rules.
    Nov 8 11:42:24 php-fpm[25084]: /rc.filter_configure_sync: SQUID is installed but not started. Not installing "pfearly" rules.
    Nov 8 11:42:24 php-fpm[25084]: /rc.filter_configure_sync: SQUID is installed but not started. Not installing "nat" rules.
    Nov 8 11:42:24 php-fpm[92516]: /pkg_edit.php: [Squid] - Squid_resync function call pr: bp: rpc:no
    Nov 8 11:42:24 check_reload_status: Reloading filter
    Nov 8 11:42:24 check_reload_status: Syncing firewall
    Nov 8 11:42:13 kernel: pid 39066 (squid), uid 0: exited on signal 11 (core dumped)
    Nov 8 11:42:13 php-fpm[92516]: /pkg_edit.php: The command '/usr/pbi/squid-amd64/sbin/squid -f /usr/pbi/squid-amd64/etc/squid/squid.conf' returned exit code '139', the output was ''
    Nov 8 11:42:12 php-fpm[92516]: /pkg_edit.php: Starting Squid
    Nov 8 11:42:12 kernel: pid 34816 (squid), uid 0: exited on signal 11 (core dumped)
    Nov 8 11:42:12 php-fpm[92516]: /pkg_edit.php: The command '/usr/pbi/squid-amd64/sbin/squid -z -f /usr/pbi/squid-amd64/etc/squid/squid.conf' returned exit code '139', the output was ''
    Nov 8 11:42:12 kernel: pid 31211 (squid), uid 0: exited on signal 11 (core dumped)
    Nov 8 11:42:12 php-fpm[92516]: /pkg_edit.php: The command '/usr/pbi/squid-amd64/sbin/squid -k kill -f /usr/pbi/squid-amd64/etc/squid/squid.conf' returned exit code '139', the output was ''
    Nov 8 11:42:06 kernel: pid 27894 (squid), uid 0: exited on signal 11 (core dumped)
    Nov 8 11:42:06 php-fpm[92516]: /pkg_edit.php: The command '/usr/pbi/squid-amd64/sbin/squid -k shutdown -f /usr/pbi/squid-amd64/etc/squid/squid.conf' returned exit code '139', the output was ''
    Nov 8 11:42:06 php-fpm[92516]: /pkg_edit.php: Creating squid cache subdirs in /var/squid/cache
    Nov 8 11:42:04 php-fpm[92516]: /pkg_edit.php: [Squid] - Squid_resync function call pr: bp: rpc:no
    Nov 8 11:41:20 kernel: pid 93914 (squid), uid 0: exited on signal 11 (core dumped)
    Nov 8 11:41:15 syslogd: kernel boot file is /boot/kernel/kernel



  • @Escorpiom:

    Today, a new Squid package was made available.
    As I was feeling adventurous, decided to hit the pkg button and….
    Installed just fine, all configs retained. No errors.

    I haven't tried to update the beta snapshot yet, better wait until it gets final.
     
    Cheers.

    That update for squid3 (3.1) and squid-dev (3.3) was just to add some extra checks to swapstate_check.php that were in squid (2) but had never been put into the newer squid versions. It does not effect any squid functionality, and has no change to the binaries. So it won't help any issues with running on 2.2-BETA,



  • Fix for Squid. Needs to be run on each box. Survives a reboot. Survives an update to latest beta, as far as I can tell.

    Open a command prompt:

    
    cd /usr/pbi/squid-amd64
    
    ```- for x64
    
    –- or ---
    
    

    cd /usr/pbi/squid-i386

    
    then:
    
    

    cp -R local/etc local/lib local/libexec /usr/

    
    I'm doing this from memory, so please do this on a test box, Virtual Machine, etc. and make a backup or snapshot first. I did this several days ago, and I copied only one folder at a time. I tried to condense the instructions to make it easier. Please let me know if I made a typo.
    
    You then will need to reboot your pfSense box. I was unable to use the Web interface until I did so. YMMV.


  • That's not a good fix, that'll leave behind files in places where they shouldn't be. The root cause of that issue is being looked into. If you need an immediate work around on 2.2, I guess that's OK, but you're going to want to blow away the system and reinstall from scratch once the root issue is fixed if you do that. Or know exactly what you copied into /usr/ and manually remove only those files.



  • Hi!
    I'am trying to install squid and DG on pfSense 2.2 Beta
    DG isn't working

    To run squid I do:

    
    ln -s /lib/libmd.so.6 /usr/lib/libmd5.so.1
    ln -s /usr/pbi/squid-amd64/local/etc/squid /usr/local/etc/squid
    ln -s /usr/pbi/squid-amd64/local/libexec/squid /usr/local/libexec/squid
    
    

    After that squid starting and working, but DG and squid-guard doesn't work!
    Squid transparent proxy with HTTPS doesn't work!
    Does filtering working on pfSense 2.2. Beta?



  • Squidguard + Squid dev does work, with the workaround.
    As 2.2 is still in beta, you will have to wait until issues have been resolved.

    Cheers.



  • @hmh:

    Hi!
    I'am trying to install squid and DG on pfSense 2.2 Beta
    DG isn't working

    To run squid I do:

    
    ln -s /lib/libmd.so.6 /usr/lib/libmd5.so.1
    ln -s /usr/pbi/squid-amd64/local/etc/squid /usr/local/etc/squid
    ln -s /usr/pbi/squid-amd64/local/libexec/squid /usr/local/libexec/squid
    
    

    After that squid starting and working, but DG and squid-guard doesn't work!
    Squid transparent proxy with HTTPS doesn't work!
    Does filtering working on pfSense 2.2. Beta?

    If you are going to use DansGuardian, don't use transparent mode with Squid. If you want to use SSL Filtering, set Squid to use the same port for SSL as for regular traffic.

    Squid defaults to port 3128 for regular traffic, and 3129 for SSL traffic. Change them so they both are 3128, for example. You then need to go into your computer's proxy settings, and enable the use of a proxy, and set the proxy server to be your router's IP address, and the port to the one on which DansGuardian is listening, usually port 8080.

    I highly recommend changing the Web UI port to something other than 443 or 80.



  • aaronouthier, why is it not recommended to use Squid transparent with DansGuardian enabled?
    Any reason particular? I've got it set up like that at the moment.

    Cheers.



  • @Escorpiom:

    aaronouthier, why is it not recommended to use Squid transparent with DansGuardian enabled?
    Any reason particular? I've got it set up like that at the moment.

    Cheers.

    Because the flow for DansGuardian is supposed to go Browser -> DansGuardian -> Squid -> Internet. However, with Transparent mode enabled, it forces Browser -> Squid -> Internet.

    Using transparent mode bypasses DansGuardian.

    Using SquidGuard should work with Squid in transparent mode, but not DansGuardian. Still, I could be wrong.

    To test this, download a blacklist for DansGuardian, and enable a category to block, such as "Warez". Then go to a site which should be blocked, like the pirate bay, etc. If Squid is in transparent mode, then the site won't be blocked. Disable Transparent mode, and setup a manual proxy from IE settings. Violla, Blocked!



  • Thanks for explaining. I remember having seen other posts regarding the issue.
    Squidguard is indeed working.

    Cheers.


Log in to reply