Dansguardian package for 2.0

rnsc · Jan 20, 2012, 1:28 AM

It makes sense to me that dansguardian would be closest to the WAN so that it filters things once, then they are cached by squid. Later requests would be serviced by squid without having to be filtered by dansguardian again. This will be faster.

If squid is closest to the WAN and a browser request goes to dansguardian first, then dans will re-filter every request for the same page, even though it is simply coming from the local squid cache. This will be slower.

About squidguard, why do we need squidguard? Doesn't dansguardian support blacklists also? Or does squidguard provide functionality that dansguardian does not?

Finally, thank you VERY VERY much for doing this.

–Ray

marcelloc · Jan 20, 2012, 2:11 AM

@rnsc:

About squidguard, why do we need squidguard? Doesn't dansguardian support blacklists also? Or does squidguard provide functionality that dansguardian does not?

Squidguard is very well implemented in pfsense and also has timegroup lists.

Blacklists are implemented to dansguardian as well content filtering.

If I do not find a way to do time groups in dansguardian(or in squid) as squidguard does we will still need it.

Current status is still under devel(80% done).
This is the most complex packages I ever every worked on, I hope it get usefull for everybody.

Donations are always welcome ;D

marcelloc · Jan 26, 2012, 3:59 AM

Current status

What is already done:

Package gui can start and stop dansguardian 2.12 daemon
dansguardian.conf 100% build via gui
filtergroupsfiles (since version 2.8 dansguardian accepts up to 99 groups)
userlist and group association tab
clamav integration
Blacklists categories association in groups

What is under devel:

Blacklist downloads
test ssl filtering support available on 2.12
Integration with squid/squidguard

Dansguardian has almost 40 config files. I had no idea of that when I started writing this package :P

Cino · Jan 27, 2012, 5:35 PM

@marcelloc i see you published the package today. I haven't installed it yet since I have a question. I looked at the deps and I knew squid could be an issue. I'm running squid3 on my box for ipv6. Do you see an issue if I try to install the package? I have no problem manually fixing it on my box if needed… good example was when i install squid-guard, it uninstall squid3 in-place of squid2... so i manually uninstalled squid2 and put squid3 back in... after that, life was good.

marcelloc · Jan 27, 2012, 5:43 PM

I do not force squid install, so if you have version 3, it probably do not install version 2.x.

I'm also checking some first issues from my devel machine to a clean vm.

iamzam · Jan 27, 2012, 5:50 PM

Hello marcelloc i installed dansguardian on my pfsense (IPv6) 2.1-DEVELOPMENT (amd64). It seemed to install correctly, no errors in the installer that I noticed, but the service won't start.

In the system log (UI) i see this:

php: /pkg_edit.php: The command 'chown -R nobody:nobody /usr/loca/etc/dansguardian' returned exit code '1', the output was 'chown: /usr/loca/etc/dansguardian: No such file or directory'

note the path: /usr/loca/etc/dansguardian

I checked /usr/local/etc/dansguardian and it does exist and has lots of files so this may just be a typo somewhere?

I ran this command to look for "/usr/loca/":

grep -r "/usr/loca/" /usr/local/*
/usr/local/pkg/dansguardian.inc: mwexec("chown -R $daemonuser:$daemongroup /usr/loca/etc/dansguardian");

I also see this error in the system log:
php: /pkg_edit.php: The command '/usr/local/etc/rc.d/dansguardian stop' returned exit code '127', the output was '/usr/local/etc/rc.d/dansguardian: not found'

I looked in the file system for /usr/local/etc/rc.d/dansguardian and it doesn't exist there.

Below is everything in /var/log/system.log relating to dansguardian:

Jan 27 12:22:04 pfs php: /pkg_mgr_install.php: Beginning package installation for Dansguardian .
Jan 27 12:22:04 pfs check_reload_status: Syncing firewall
Jan 27 12:22:09 pfs php: /pkg_mgr_install.php: The command 'chown -R nobody:nobody /usr/loca/etc/dansguardian' returned exit code '1', the output was 'chown: /usr/loca/etc/dansguardian: No such file or directory'
Jan 27 12:22:09 pfs php: /pkg_mgr_install.php: The command 'chown -R nobody:nobody /var/log/dansguardian' returned exit code '1', the output was 'chown: /var/log/dansguardian: No such file or directory'
Jan 27 12:22:09 pfs php: /pkg_mgr_install.php: The command '/usr/local/etc/rc.d/dansguardian stop' returned exit code '127', the output was '/usr/local/etc/rc.d/dansguardian: not found'
Jan 27 12:22:10 pfs php: /pkg_mgr_install.php: The command 'chown -R nobody:nobody /usr/loca/etc/dansguardian' returned exit code '1', the output was 'chown: /usr/loca/etc/dansguardian: No such file or directory'
Jan 27 12:22:10 pfs php: /pkg_mgr_install.php: The command '/usr/local/etc/rc.d/dansguardian stop' returned exit code '127', the output was '/usr/local/etc/rc.d/dansguardian: not found'
Jan 27 12:22:10 pfs check_reload_status: Reloading filter
Jan 27 12:24:09 pfs check_reload_status: Syncing firewall
Jan 27 12:24:09 pfs check_reload_status: Syncing firewall
Jan 27 12:24:09 pfs php: /pkg_edit.php: The command 'chown -R nobody:nobody /usr/loca/etc/dansguardian' returned exit code '1', the output was 'chown: /usr/loca/etc/dansguardian: No such file or directory'
Jan 27 12:24:09 pfs php: /pkg_edit.php: The command '/usr/local/etc/rc.d/dansguardian stop' returned exit code '127', the output was '/usr/local/etc/rc.d/dansguardian: not found'
Jan 27 12:27:40 pfs check_reload_status: Syncing firewall
Jan 27 12:27:40 pfs check_reload_status: Syncing firewall
Jan 27 12:27:40 pfs php: /pkg_edit.php: The command 'chown -R nobody:nobody /usr/loca/etc/dansguardian' returned exit code '1', the output was 'chown: /usr/loca/etc/dansguardian: No such file or directory'
Jan 27 12:27:40 pfs php: /pkg_edit.php: The command '/usr/local/etc/rc.d/dansguardian stop' returned exit code '127', the output was '/usr/local/etc/rc.d/dansguardian: not found'
Jan 27 12:28:50 pfs check_reload_status: Syncing firewall
Jan 27 12:28:50 pfs check_reload_status: Syncing firewall
Jan 27 12:28:50 pfs php: /pkg_edit.php: The command 'chown -R nobody:nobody /usr/loca/etc/dansguardian' returned exit code '1', the output was 'chown: /usr/loca/etc/dansguardian: No such file or directory'
Jan 27 12:28:50 pfs php: /pkg_edit.php: The command '/usr/local/etc/rc.d/dansguardian stop' returned exit code '127', the output was '/usr/local/etc/rc.d/dansguardian: not found'

Thanks for your help!

marcelloc · Jan 27, 2012, 5:58 PM

Thanks for the feedback, there is a missing "l" on path.

I'm sitll looking for other issues, until commit the patches.

marcelloc · Jan 27, 2012, 9:00 PM

Just published some fixes and changed version to 0.1.1

features not finished yet:

blacklist update frequency
ssl man in the middle feature

Cino · Jan 28, 2012, 5:06 PM

figured I would give this a try this morning. first thing i noticed, there is no menu option and I dont see it under status:services. But i'm able to access it by typing https://…../pkg_edit.php?xml=dansguardian.xml. and able to see it running via cmdline

allowed 127.0.0.1 access in the squid3 config and was able to use it... enabled the porn filter for testing and worked as designed! Great job man!!! I haven't used dansguardian in years and forgot how complicate the config can be...

how does clamav update its virus def file?

marcelloc · Jan 28, 2012, 5:09 PM

@Cino:

figured I would give this a try this morning. first thing i noticed, there is no menu option and I dont see it under status:services. But i'm able to access it by typing https://…../pkg_edit.php?xml=dansguardian.xml. and able to see it running via cmdline

Isn't there on second menu option? ???

If tested on both amd64 and i386 but only on 2.0.1

@Cino:

I haven't used dansguardian in years and forgot how complicate the config can be…

I know that. Almost 40 config files :P
I've created almost 80% of default config files during first apply/install process.

Cino · Jan 28, 2012, 5:20 PM

i am using 2.1-dev so it could because of that.. i did reboot the box and it took 2-3 minutes for it to get past the starting package… it started tho.

i'll mess with it some more but if other users arn't having the same issue... then its my setup.

marcelloc · Jan 28, 2012, 5:28 PM

@Cino:

i am using 2.1-dev so it could because of that.. i did reboot the box and it took 2-3 minutes for it to get past the starting package… it started tho.

I think It's calling sync process for every XML file during boot.

I'm trying to workaround it.

Cino · Jan 28, 2012, 7:52 PM

i was able to get it to display in the services menu… i ended up re-installing just the gui items. cosmetic, but i would add it to the status: services menu down the road to know its running or not.

should i run a cron job to update the virus db for clamav (freshclam)? or is it built-in somehow?

marcelloc · Jan 28, 2012, 8:39 PM

The first update from clamav is automatic when package detects that there is no database.

I will be an option on gui until I finish schedulle updates( blacklist and clamav)

Cino · Jan 28, 2012, 8:49 PM

thanks again and great work!! You have added some nice add-ons to pfsense over the last couple of months

no issue, but here are the processes on my box.. this is how i know its running:


: ps -aux | grep dan
clamav 13281  0.0  0.4 18036 14028  ??  INs   3:00PM   0:00.08 /usr/local/sbin/dansguardian
clamav 13525  0.0  0.2  7796  4692  ??  IN    3:00PM   0:00.03 /usr/local/sbin/dansguardian
clamav 13655  0.0  0.1  7796  4672  ??  IN    3:00PM   0:00.02 /usr/local/sbin/dansguardian
clamav 13759  0.0  0.5 19060 14840  ??  IN    3:00PM   0:00.43 /usr/local/sbin/dansguardian
clamav 14008  0.0  0.5 18036 14172  ??  IN    3:00PM   0:00.35 /usr/local/sbin/dansguardian
clamav 14105  0.0  0.5 18036 14120  ??  IN    3:00PM   0:00.02 /usr/local/sbin/dansguardian
clamav 14199  0.0  0.5 18036 14120  ??  IN    3:00PM   0:00.02 /usr/local/sbin/dansguardian
clamav 14385  0.0  0.5 18036 14264  ??  IN    3:00PM   0:00.87 /usr/local/sbin/dansguardian
clamav 14451  0.0  0.4 18036 13988  ??  IN    3:00PM   0:00.01 /usr/local/sbin/dansguardian
clamav 14466  0.0  0.4 18036 13988  ??  IN    3:00PM   0:00.01 /usr/local/sbin/dansguardian
clamav 14472  0.0  0.4 18036 13988  ??  IN    3:00PM   0:00.01 /usr/local/sbin/dansguardian
clamav 16150  0.0  0.4 18036 14052  ??  IN    3:40PM   0:00.00 /usr/local/sbin/dansguardian
clamav 16221  0.0  0.4 18036 14052  ??  IN    3:40PM   0:00.00 /usr/local/sbin/dansguardian
clamav 16515  0.0  0.4 18036 14052  ??  IN    3:40PM   0:00.00 /usr/local/sbin/dansguardian
clamav 16582  0.0  0.4 18036 14052  ??  IN    3:40PM   0:00.00 /usr/local/sbin/dansguardian
clamav 16676  0.0  0.4 18036 14052  ??  IN    3:40PM   0:00.00 /usr/local/sbin/dansguardian
clamav 16741  0.0  0.4 18036 14052  ??  IN    3:40PM   0:00.00 /usr/local/sbin/dansguardian
clamav 16763  0.0  0.4 18036 14052  ??  IN    3:40PM   0:00.01 /usr/local/sbin/dansguardian
clamav 16828  0.0  0.4 18036 14052  ??  IN    3:40PM   0:00.01 /usr/local/sbin/dansguardian
clamav 16843  0.0  0.4 18036 14052  ??  IN    3:40PM   0:00.01 /usr/local/sbin/dansguardian
clamav 17095  0.0  0.4 18036 14052  ??  IN    3:40PM   0:00.00 /usr/local/sbin/dansguardian
root   13272  0.0  0.0  3524  1252   0  S+    3:47PM   0:00.00 grep dan

marcelloc · Jan 29, 2012, 1:42 AM

It's running very well.

I'll include service option too. Maybe tomorrow.

marcelloc · Jan 30, 2012, 3:11 AM

version 0.1.2 is out with some fixes and improvements

improve blacklist import and category assign
include dansguardian on service
include log, daemon and config default xml options
fix some typos
Maybe a faster boot process (01:00 on virtualbox)

still working on…

SSL men in the middle feature
Blacklist and clamav update frequency

To avoid double dansguardian link on service menu(dansguardian and Dansguardian), remove 0.1.1 package before install 0.1.2

Cino · Jan 30, 2012, 3:21 PM

thanks for the update.. I've give it a try in a bit

burton78 · Jan 30, 2012, 4:29 PM

Thanks for all your hard work marcelloc! This is a great addition to pfsense! Was wondering if adding a "transparent proxy mode" could be an option as well?

Also I caught a typo for ya too under Daemon -> Parent proxy Settings -> Proxy Port: Sets port number "fro" proxy server.

marcelloc · Jan 30, 2012, 4:36 PM

@burton78:

Thanks for all your hard work marcelloc! This is a great addition to pfsense! Was wondering if adding a "transparent proxy mode" could be an option as well?

I'll include PAC configuration to dansguardian package first.

When you enable trasparente proxy, you are not able to authenticate or filter ssl.

I'ts not that good for security at all.

Search for WPAD, PAC proxy.js or proxy.pac to see how auto detecproxy works.

Also I caught a typo for ya too under Daemon -> Parent proxy Settings -> Proxy Port: Sets port number "fro" proxy server.

Thanks.