Dansguardian package for 2.0

marcelloc

start topic on documentation
http://forum.pfsense.org/index.php/topic,42664.msg220385.html#msg220385

Ok, So lets start.

Compiling in 8.1 with no args results on working instalation on pfsense.

I will try it on 64 bits

marcelloc

amd64 package to dansguardian 2.10 stable version download link

Important note about dansguardian:
DansGuardian 2 is:

licensed under the GPL version 2 with permission to link to OpenSSL
Open Source
Free Software where 'Free' means Freedom
freely (no cost) downloadable from this site for non-commercial use
freely (no cost) downloadable from this site for general purpose unix distributions like FreeBSD, Debian, Fedora, Ubuntu, etc
not freely downloadable from this site for installation by 3rd parties charging for installation or support
not freely downloadable from this site for commercial use
a registered trade mark of Daniel Barron
copyright Daniel Barron

So, if you want dansguardian for comercial use, buy a licence since you start using it.

JackL

@marcelloc:

So, if you want dansguardian for comercial use, buy a licence since you start using it.

To better understand the license and distribution of access DansGuardian: http://dansguardian.org/?page=copyright2

[]'s
Jack

ZGruk

Here are some notes about my freeBSD installation that I got dansguardian from:

I installed it in VirtualBox. Before I installed either squid or dansguardian, I installed the virtual box "helper" package that is optionally included in the freeBSD installation. (you choose to add a bunch of packages during the install process and then actually unpack the packages as needed later) During the install process for the helper package, it downloaded dozens of other dependency packages. Some of these may have been needed for the squid and dansguardian installations. Neither dansguardian or squid required that I install any other dependencies before they worked.

running```
find / dansguardian


The dansguardian version I used was 2.12.0.0 alpha 32 bit. I think it's safe to assume that the stable 2.10 version would work the same. I don't think the 64 bit version should have any problems either.

marcelloc

If you use freebsd ports, after compiling dansguardian stable version, you can do a make package to build a package instalation file.

That's what i did in x64 version.

Just install squid package and then

pkg_add -r https://github.com/downloads/marcelloc/pfsense-packages/dansguardian-2.10.1.1_1.amd64.tbz

marcelloc

64bits pfsense binary package to dansguardian 2.10 stable version download link

32bits pfsense binary package to dansguardian 2.10 stable version download link

Important note about dansguardian:
DansGuardian 2 is:

licensed under the GPL version 2 with permission to link to OpenSSL
Open Source
Free Software where 'Free' means Freedom
freely (no cost) downloadable from this site for non-commercial use
freely (no cost) downloadable from this site for general purpose unix distributions like FreeBSD, Debian, Fedora, Ubuntu, etc
not freely downloadable from this site for installation by 3rd parties charging for installation or support
not freely downloadable from this site for commercial use
a registered trade mark of Daniel Barron
copyright Daniel Barron

So, if you want dansguardian for comercial use, buy a licence since you start using it.

To understand DansGuardian license and distribution take a look on http://dansguardian.org/?page=copyright2

JackL

Show of ball marcello … Congratulations again for the work!

I made sure to add a blog post in our reporting "good news": http://www.pfsense-br.org/blog/2011/12/dansguardian-2-10-1-1-no-pfsense-2-0/

Do you have plans to develop a management interface for the GUI?

[]'s
Jack

marcelloc

@JackL:

Do you have plans to develop a management interface for the GUI?

Yes, but just after finishing mailscanner package gui.

JackL

@marcelloc:

Yes, but just after finishing mailscanner package gui.

All right marcelloc!

one thing at a time! ;-)

[]'s
Jack

marcelloc

I've started dansguardian gui for pfsense.

I'll base gui on dansguardian 2.12 version.

Here are links to 2.12 packages

i386

x64

Cino

marcelloc, as always nice work! I see you're already adding files to the github… Can't wait to test the package out when its ready for beta testing. Question, since squid is a requirement; will the package state that squid needs to be installed first? I ask because I've been using squid3 and when I added squidguard it removed the binaries for squid3 and replaced them with squid2.. No biggy since I was able to rectify from the cmdline. But other users may not be so lucky.

something to think of

marcelloc

Hi cino, good to see that this package will be usefull for pfsense.

I can publish this package without forcing a specific version of squid.

If dansguardian do not find it's dependencies, then pkg_add will look for 2.x squid version.

One think that I observed is that all good stuff from squid is in squidguard so I think that this first version will be able to replace havp portion of proxy solution as many people use time and group acls.

The result will be something like

dansguardian -> squid -> squidguard

or

squid -> squidguard -> dansguardian

I was planning to finish dansguardian.inc this week but mailscanner initialization was buggy and starting endless processes and killing machine resources. It's fixed now but I need to monitor it to be 100% sure it's fixed.

rnsc

It makes sense to me that dansguardian would be closest to the WAN so that it filters things once, then they are cached by squid. Later requests would be serviced by squid without having to be filtered by dansguardian again. This will be faster.

If squid is closest to the WAN and a browser request goes to dansguardian first, then dans will re-filter every request for the same page, even though it is simply coming from the local squid cache. This will be slower.

About squidguard, why do we need squidguard? Doesn't dansguardian support blacklists also? Or does squidguard provide functionality that dansguardian does not?

Finally, thank you VERY VERY much for doing this.

–Ray

marcelloc

@rnsc:

About squidguard, why do we need squidguard? Doesn't dansguardian support blacklists also? Or does squidguard provide functionality that dansguardian does not?

Squidguard is very well implemented in pfsense and also has timegroup lists.

Blacklists are implemented to dansguardian as well content filtering.

If I do not find a way to do time groups in dansguardian(or in squid) as squidguard does we will still need it.

Current status is still under devel(80% done).
This is the most complex packages I ever every worked on, I hope it get usefull for everybody.

Donations are always welcome ;D

marcelloc

Current status

What is already done:

Package gui can start and stop dansguardian 2.12 daemon
dansguardian.conf 100% build via gui
filtergroupsfiles (since version 2.8 dansguardian accepts up to 99 groups)
userlist and group association tab
clamav integration
Blacklists categories association in groups

What is under devel:

Blacklist downloads
test ssl filtering support available on 2.12
Integration with squid/squidguard

Dansguardian has almost 40 config files. I had no idea of that when I started writing this package :P

Cino

@marcelloc i see you published the package today. I haven't installed it yet since I have a question. I looked at the deps and I knew squid could be an issue. I'm running squid3 on my box for ipv6. Do you see an issue if I try to install the package? I have no problem manually fixing it on my box if needed… good example was when i install squid-guard, it uninstall squid3 in-place of squid2... so i manually uninstalled squid2 and put squid3 back in... after that, life was good.

marcelloc

I do not force squid install, so if you have version 3, it probably do not install version 2.x.

I'm also checking some first issues from my devel machine to a clean vm.

iamzam

Hello marcelloc i installed dansguardian on my pfsense (IPv6) 2.1-DEVELOPMENT (amd64). It seemed to install correctly, no errors in the installer that I noticed, but the service won't start.

In the system log (UI) i see this:

php: /pkg_edit.php: The command 'chown -R nobody:nobody /usr/loca/etc/dansguardian' returned exit code '1', the output was 'chown: /usr/loca/etc/dansguardian: No such file or directory'

note the path: /usr/loca/etc/dansguardian

I checked /usr/local/etc/dansguardian and it does exist and has lots of files so this may just be a typo somewhere?

I ran this command to look for "/usr/loca/":

grep -r "/usr/loca/" /usr/local/*
/usr/local/pkg/dansguardian.inc: mwexec("chown -R $daemonuser:$daemongroup /usr/loca/etc/dansguardian");

I also see this error in the system log:
php: /pkg_edit.php: The command '/usr/local/etc/rc.d/dansguardian stop' returned exit code '127', the output was '/usr/local/etc/rc.d/dansguardian: not found'

I looked in the file system for /usr/local/etc/rc.d/dansguardian and it doesn't exist there.

Below is everything in /var/log/system.log relating to dansguardian:

Jan 27 12:22:04 pfs php: /pkg_mgr_install.php: Beginning package installation for Dansguardian .
Jan 27 12:22:04 pfs check_reload_status: Syncing firewall
Jan 27 12:22:09 pfs php: /pkg_mgr_install.php: The command 'chown -R nobody:nobody /usr/loca/etc/dansguardian' returned exit code '1', the output was 'chown: /usr/loca/etc/dansguardian: No such file or directory'
Jan 27 12:22:09 pfs php: /pkg_mgr_install.php: The command 'chown -R nobody:nobody /var/log/dansguardian' returned exit code '1', the output was 'chown: /var/log/dansguardian: No such file or directory'
Jan 27 12:22:09 pfs php: /pkg_mgr_install.php: The command '/usr/local/etc/rc.d/dansguardian stop' returned exit code '127', the output was '/usr/local/etc/rc.d/dansguardian: not found'
Jan 27 12:22:10 pfs php: /pkg_mgr_install.php: The command 'chown -R nobody:nobody /usr/loca/etc/dansguardian' returned exit code '1', the output was 'chown: /usr/loca/etc/dansguardian: No such file or directory'
Jan 27 12:22:10 pfs php: /pkg_mgr_install.php: The command '/usr/local/etc/rc.d/dansguardian stop' returned exit code '127', the output was '/usr/local/etc/rc.d/dansguardian: not found'
Jan 27 12:22:10 pfs check_reload_status: Reloading filter
Jan 27 12:24:09 pfs check_reload_status: Syncing firewall
Jan 27 12:24:09 pfs check_reload_status: Syncing firewall
Jan 27 12:24:09 pfs php: /pkg_edit.php: The command 'chown -R nobody:nobody /usr/loca/etc/dansguardian' returned exit code '1', the output was 'chown: /usr/loca/etc/dansguardian: No such file or directory'
Jan 27 12:24:09 pfs php: /pkg_edit.php: The command '/usr/local/etc/rc.d/dansguardian stop' returned exit code '127', the output was '/usr/local/etc/rc.d/dansguardian: not found'
Jan 27 12:27:40 pfs check_reload_status: Syncing firewall
Jan 27 12:27:40 pfs check_reload_status: Syncing firewall
Jan 27 12:27:40 pfs php: /pkg_edit.php: The command 'chown -R nobody:nobody /usr/loca/etc/dansguardian' returned exit code '1', the output was 'chown: /usr/loca/etc/dansguardian: No such file or directory'
Jan 27 12:27:40 pfs php: /pkg_edit.php: The command '/usr/local/etc/rc.d/dansguardian stop' returned exit code '127', the output was '/usr/local/etc/rc.d/dansguardian: not found'
Jan 27 12:28:50 pfs check_reload_status: Syncing firewall
Jan 27 12:28:50 pfs check_reload_status: Syncing firewall
Jan 27 12:28:50 pfs php: /pkg_edit.php: The command 'chown -R nobody:nobody /usr/loca/etc/dansguardian' returned exit code '1', the output was 'chown: /usr/loca/etc/dansguardian: No such file or directory'
Jan 27 12:28:50 pfs php: /pkg_edit.php: The command '/usr/local/etc/rc.d/dansguardian stop' returned exit code '127', the output was '/usr/local/etc/rc.d/dansguardian: not found'

Thanks for your help!

marcelloc

Thanks for the feedback, there is a missing "l" on path.

I'm sitll looking for other issues, until commit the patches.

marcelloc

Just published some fixes and changed version to 0.1.1

features not finished yet:

blacklist update frequency
ssl man in the middle feature

Cino

figured I would give this a try this morning. first thing i noticed, there is no menu option and I dont see it under status:services. But i'm able to access it by typing https://…../pkg_edit.php?xml=dansguardian.xml. and able to see it running via cmdline

allowed 127.0.0.1 access in the squid3 config and was able to use it... enabled the porn filter for testing and worked as designed! Great job man!!! I haven't used dansguardian in years and forgot how complicate the config can be...

how does clamav update its virus def file?

marcelloc

@Cino:

figured I would give this a try this morning. first thing i noticed, there is no menu option and I dont see it under status:services. But i'm able to access it by typing https://…../pkg_edit.php?xml=dansguardian.xml. and able to see it running via cmdline

Isn't there on second menu option? ???

If tested on both amd64 and i386 but only on 2.0.1

@Cino:

I haven't used dansguardian in years and forgot how complicate the config can be…

I know that. Almost 40 config files :P
I've created almost 80% of default config files during first apply/install process.

Cino

i am using 2.1-dev so it could because of that.. i did reboot the box and it took 2-3 minutes for it to get past the starting package… it started tho.

i'll mess with it some more but if other users arn't having the same issue... then its my setup.

marcelloc

@Cino:

i am using 2.1-dev so it could because of that.. i did reboot the box and it took 2-3 minutes for it to get past the starting package… it started tho.

I think It's calling sync process for every XML file during boot.

I'm trying to workaround it.

Cino

i was able to get it to display in the services menu… i ended up re-installing just the gui items. cosmetic, but i would add it to the status: services menu down the road to know its running or not.

should i run a cron job to update the virus db for clamav (freshclam)? or is it built-in somehow?

marcelloc

The first update from clamav is automatic when package detects that there is no database.

I will be an option on gui until I finish schedulle updates( blacklist and clamav)

Cino

thanks again and great work!! You have added some nice add-ons to pfsense over the last couple of months

no issue, but here are the processes on my box.. this is how i know its running:


: ps -aux | grep dan
clamav 13281  0.0  0.4 18036 14028  ??  INs   3:00PM   0:00.08 /usr/local/sbin/dansguardian
clamav 13525  0.0  0.2  7796  4692  ??  IN    3:00PM   0:00.03 /usr/local/sbin/dansguardian
clamav 13655  0.0  0.1  7796  4672  ??  IN    3:00PM   0:00.02 /usr/local/sbin/dansguardian
clamav 13759  0.0  0.5 19060 14840  ??  IN    3:00PM   0:00.43 /usr/local/sbin/dansguardian
clamav 14008  0.0  0.5 18036 14172  ??  IN    3:00PM   0:00.35 /usr/local/sbin/dansguardian
clamav 14105  0.0  0.5 18036 14120  ??  IN    3:00PM   0:00.02 /usr/local/sbin/dansguardian
clamav 14199  0.0  0.5 18036 14120  ??  IN    3:00PM   0:00.02 /usr/local/sbin/dansguardian
clamav 14385  0.0  0.5 18036 14264  ??  IN    3:00PM   0:00.87 /usr/local/sbin/dansguardian
clamav 14451  0.0  0.4 18036 13988  ??  IN    3:00PM   0:00.01 /usr/local/sbin/dansguardian
clamav 14466  0.0  0.4 18036 13988  ??  IN    3:00PM   0:00.01 /usr/local/sbin/dansguardian
clamav 14472  0.0  0.4 18036 13988  ??  IN    3:00PM   0:00.01 /usr/local/sbin/dansguardian
clamav 16150  0.0  0.4 18036 14052  ??  IN    3:40PM   0:00.00 /usr/local/sbin/dansguardian
clamav 16221  0.0  0.4 18036 14052  ??  IN    3:40PM   0:00.00 /usr/local/sbin/dansguardian
clamav 16515  0.0  0.4 18036 14052  ??  IN    3:40PM   0:00.00 /usr/local/sbin/dansguardian
clamav 16582  0.0  0.4 18036 14052  ??  IN    3:40PM   0:00.00 /usr/local/sbin/dansguardian
clamav 16676  0.0  0.4 18036 14052  ??  IN    3:40PM   0:00.00 /usr/local/sbin/dansguardian
clamav 16741  0.0  0.4 18036 14052  ??  IN    3:40PM   0:00.00 /usr/local/sbin/dansguardian
clamav 16763  0.0  0.4 18036 14052  ??  IN    3:40PM   0:00.01 /usr/local/sbin/dansguardian
clamav 16828  0.0  0.4 18036 14052  ??  IN    3:40PM   0:00.01 /usr/local/sbin/dansguardian
clamav 16843  0.0  0.4 18036 14052  ??  IN    3:40PM   0:00.01 /usr/local/sbin/dansguardian
clamav 17095  0.0  0.4 18036 14052  ??  IN    3:40PM   0:00.00 /usr/local/sbin/dansguardian
root   13272  0.0  0.0  3524  1252   0  S+    3:47PM   0:00.00 grep dan

marcelloc

It's running very well.

I'll include service option too. Maybe tomorrow.

marcelloc

version 0.1.2 is out with some fixes and improvements

improve blacklist import and category assign
include dansguardian on service
include log, daemon and config default xml options
fix some typos
Maybe a faster boot process (01:00 on virtualbox)

still working on…

SSL men in the middle feature
Blacklist and clamav update frequency

To avoid double dansguardian link on service menu(dansguardian and Dansguardian), remove 0.1.1 package before install 0.1.2

Cino

thanks for the update.. I've give it a try in a bit

burton78

Thanks for all your hard work marcelloc! This is a great addition to pfsense! Was wondering if adding a "transparent proxy mode" could be an option as well?

Also I caught a typo for ya too under Daemon -> Parent proxy Settings -> Proxy Port: Sets port number "fro" proxy server.

marcelloc

@burton78:

Thanks for all your hard work marcelloc! This is a great addition to pfsense! Was wondering if adding a "transparent proxy mode" could be an option as well?

I'll include PAC configuration to dansguardian package first.

When you enable trasparente proxy, you are not able to authenticate or filter ssl.

I'ts not that good for security at all.

Search for WPAD, PAC proxy.js or proxy.pac to see how auto detecproxy works.

Also I caught a typo for ya too under Daemon -> Parent proxy Settings -> Proxy Port: Sets port number "fro" proxy server.

Thanks.

Cino

here is example of mine that i've been using with squid.


function FindProxyForURL(url, host) {

// If IP address is internal or hostname resolves to internal IP, send direct.

	if (
          isInNet(host, "10.0.0.0", "255.0.0.0") ||
          isInNet(host, "172.16.0.0", "255.240.0.0") ||
          isInNet(host, "192.168.0.0", "255.255.0.0") ||
		  isInNet(host, "127.0.0.0", "255.255.255.0") ||
		  isInNet(host, "192.168.0.100", "255.255.255.255") ||
		  isPlainHostName(host) ||
		  localHostOrDomainIs(host, "127.0.0.1") ||
		  dnsDomainIs(host, ".xxxx.net")
		)

		return "DIRECT";

return "PROXY 192.168.0.1:3128; DIRECT";
}

this is probably more to do with squid then danguardian. i've enabled forwardfor and/or xforwardfor but it always seem to get the IP address of the squid interface in the squid logs… Any ideas? I read there was a patch for squid but this patch is pretty old and you would think it was already built into squid2 and squid3..... i am using squid3 so it could be that...

Any ideas?

marcelloc

@Cino:

return "PROXY 192.168.0.1:3128; DIRECT";

Change this to dansguardian port and enable Auth Plugin on dansguardian general tab, this will log users.

to log x-forwarded ips you need to inlcude this x-forwarder no squid.conf log directive.

I did not found what to change but as you are using version 3.x, this doc can help
http://www.squid-cache.org/Doc/config/follow_x_forwarded_for/

EDIT
Try ip address authentication mode on dansguardian general tab.
This may log ip as a user on your squid.

You can also disable squid log and change dansguardian style to squid.

itoxygen

PfSense 2.0.1 install Dansguardian v.0.1.2
error code
php: /pkg_edit.php: The command '/usr/local/etc/rc.d/dansguardian stop' returned exit code '1', the output was 'dansguardian not running? (check /var/run/dansguardian.pid).'
Jan 30 22:13:19 dansguardian[38774]: Error reading custom image file.

where is problem?

marcelloc

@itoxygen:

PfSense 2.0.1 install Dansguardian v.0.1.2
error code
php: /pkg_edit.php: The command '/usr/local/etc/rc.d/dansguardian stop' returned exit code '1', the output was 'dansguardian not running? (check /var/run/dansguardian.pid).'
Jan 30 22:13:19 dansguardian[38774]: Error reading custom image file.

where is problem?

try to start on console:
/usr/local/etc/rc.d/dansguardian start

itoxygen

Dansguardian service status is running, but system logs error message

Jan 30 22:36:57 php: /pkg_edit.php: The command '/usr/local/etc/rc.d/dansguardian stop' returned exit code '1', the output was 'dansguardian not running? (check /var/run/dansguardian.pid).'
Jan 30 22:36:57 dansguardian[50366]: Error reading custom image file.

Content filter not working

marcelloc

Are you configuring client browser to use squid port or dansguardian port?

Can you paste return from:
ps ax | grep -i dansguardian

itoxygen

Squid transparent proxy mode port number 3128, I add the adress to the black list of squid, it works without problem

$ ps ax | grep -i dansguardian
130 ?? I 0:00.00 /usr/local/sbin/dansguardian
366 ?? I 0:00.00 /usr/local/sbin/dansguardian
527 ?? I 0:00.00 /usr/local/sbin/dansguardian
557 ?? I 0:00.00 /usr/local/sbin/dansguardian
895 ?? I 0:00.00 /usr/local/sbin/dansguardian
1176 ?? I 0:00.00 /usr/local/sbin/dansguardian
1288 ?? I 0:00.00 /usr/local/sbin/dansguardian
1464 ?? I 0:00.00 /usr/local/sbin/dansguardian
1530 ?? I 0:00.00 /usr/local/sbin/dansguardian
48666 ?? S 0:00.00 sh -c ps ax | grep -i dansguardian
48931 ?? S 0:00.00 grep -i dansguardian
63558 ?? Is 0:00.02 /usr/local/sbin/dansguardian
63861 ?? I 0:00.01 /usr/local/sbin/dansguardian

marcelloc

@itoxygen:

Squid transparent proxy mode port number 3128, I add the adress to the black list of squid, it works without problem

Your dansguardian is running.

Instead of using squid on transparent mode, config your browser to use dansguardian ip and port.

If you want o improve your transparente setup, configure WPAD/PAC on your DHCP.