Dansguardian package for 2.0

Cino

marcelloc, as always nice work! I see you're already adding files to the github… Can't wait to test the package out when its ready for beta testing. Question, since squid is a requirement; will the package state that squid needs to be installed first? I ask because I've been using squid3 and when I added squidguard it removed the binaries for squid3 and replaced them with squid2.. No biggy since I was able to rectify from the cmdline. But other users may not be so lucky.

something to think of

marcelloc

Hi cino, good to see that this package will be usefull for pfsense.

I can publish this package without forcing a specific version of squid.

If dansguardian do not find it's dependencies, then pkg_add will look for 2.x squid version.

One think that I observed is that all good stuff from squid is in squidguard so I think that this first version will be able to replace havp portion of proxy solution as many people use time and group acls.

The result will be something like

dansguardian -> squid -> squidguard

or

squid -> squidguard -> dansguardian

I was planning to finish dansguardian.inc this week but mailscanner initialization was buggy and starting endless processes and killing machine resources. It's fixed now but I need to monitor it to be 100% sure it's fixed.

rnsc

It makes sense to me that dansguardian would be closest to the WAN so that it filters things once, then they are cached by squid. Later requests would be serviced by squid without having to be filtered by dansguardian again. This will be faster.

If squid is closest to the WAN and a browser request goes to dansguardian first, then dans will re-filter every request for the same page, even though it is simply coming from the local squid cache. This will be slower.

About squidguard, why do we need squidguard? Doesn't dansguardian support blacklists also? Or does squidguard provide functionality that dansguardian does not?

Finally, thank you VERY VERY much for doing this.

–Ray

marcelloc

@rnsc:

About squidguard, why do we need squidguard? Doesn't dansguardian support blacklists also? Or does squidguard provide functionality that dansguardian does not?

Squidguard is very well implemented in pfsense and also has timegroup lists.

Blacklists are implemented to dansguardian as well content filtering.

If I do not find a way to do time groups in dansguardian(or in squid) as squidguard does we will still need it.

Current status is still under devel(80% done).
This is the most complex packages I ever every worked on, I hope it get usefull for everybody.

Donations are always welcome ;D

marcelloc

Current status

What is already done:

Package gui can start and stop dansguardian 2.12 daemon
dansguardian.conf 100% build via gui
filtergroupsfiles (since version 2.8 dansguardian accepts up to 99 groups)
userlist and group association tab
clamav integration
Blacklists categories association in groups

What is under devel:

Blacklist downloads
test ssl filtering support available on 2.12
Integration with squid/squidguard

Dansguardian has almost 40 config files. I had no idea of that when I started writing this package :P

Cino

@marcelloc i see you published the package today. I haven't installed it yet since I have a question. I looked at the deps and I knew squid could be an issue. I'm running squid3 on my box for ipv6. Do you see an issue if I try to install the package? I have no problem manually fixing it on my box if needed… good example was when i install squid-guard, it uninstall squid3 in-place of squid2... so i manually uninstalled squid2 and put squid3 back in... after that, life was good.

marcelloc

I do not force squid install, so if you have version 3, it probably do not install version 2.x.

I'm also checking some first issues from my devel machine to a clean vm.

iamzam

Hello marcelloc i installed dansguardian on my pfsense (IPv6) 2.1-DEVELOPMENT (amd64). It seemed to install correctly, no errors in the installer that I noticed, but the service won't start.

In the system log (UI) i see this:

php: /pkg_edit.php: The command 'chown -R nobody:nobody /usr/loca/etc/dansguardian' returned exit code '1', the output was 'chown: /usr/loca/etc/dansguardian: No such file or directory'

note the path: /usr/loca/etc/dansguardian

I checked /usr/local/etc/dansguardian and it does exist and has lots of files so this may just be a typo somewhere?

I ran this command to look for "/usr/loca/":

grep -r "/usr/loca/" /usr/local/*
/usr/local/pkg/dansguardian.inc: mwexec("chown -R $daemonuser:$daemongroup /usr/loca/etc/dansguardian");

I also see this error in the system log:
php: /pkg_edit.php: The command '/usr/local/etc/rc.d/dansguardian stop' returned exit code '127', the output was '/usr/local/etc/rc.d/dansguardian: not found'

I looked in the file system for /usr/local/etc/rc.d/dansguardian and it doesn't exist there.

Below is everything in /var/log/system.log relating to dansguardian:

Jan 27 12:22:04 pfs php: /pkg_mgr_install.php: Beginning package installation for Dansguardian .
Jan 27 12:22:04 pfs check_reload_status: Syncing firewall
Jan 27 12:22:09 pfs php: /pkg_mgr_install.php: The command 'chown -R nobody:nobody /usr/loca/etc/dansguardian' returned exit code '1', the output was 'chown: /usr/loca/etc/dansguardian: No such file or directory'
Jan 27 12:22:09 pfs php: /pkg_mgr_install.php: The command 'chown -R nobody:nobody /var/log/dansguardian' returned exit code '1', the output was 'chown: /var/log/dansguardian: No such file or directory'
Jan 27 12:22:09 pfs php: /pkg_mgr_install.php: The command '/usr/local/etc/rc.d/dansguardian stop' returned exit code '127', the output was '/usr/local/etc/rc.d/dansguardian: not found'
Jan 27 12:22:10 pfs php: /pkg_mgr_install.php: The command 'chown -R nobody:nobody /usr/loca/etc/dansguardian' returned exit code '1', the output was 'chown: /usr/loca/etc/dansguardian: No such file or directory'
Jan 27 12:22:10 pfs php: /pkg_mgr_install.php: The command '/usr/local/etc/rc.d/dansguardian stop' returned exit code '127', the output was '/usr/local/etc/rc.d/dansguardian: not found'
Jan 27 12:22:10 pfs check_reload_status: Reloading filter
Jan 27 12:24:09 pfs check_reload_status: Syncing firewall
Jan 27 12:24:09 pfs check_reload_status: Syncing firewall
Jan 27 12:24:09 pfs php: /pkg_edit.php: The command 'chown -R nobody:nobody /usr/loca/etc/dansguardian' returned exit code '1', the output was 'chown: /usr/loca/etc/dansguardian: No such file or directory'
Jan 27 12:24:09 pfs php: /pkg_edit.php: The command '/usr/local/etc/rc.d/dansguardian stop' returned exit code '127', the output was '/usr/local/etc/rc.d/dansguardian: not found'
Jan 27 12:27:40 pfs check_reload_status: Syncing firewall
Jan 27 12:27:40 pfs check_reload_status: Syncing firewall
Jan 27 12:27:40 pfs php: /pkg_edit.php: The command 'chown -R nobody:nobody /usr/loca/etc/dansguardian' returned exit code '1', the output was 'chown: /usr/loca/etc/dansguardian: No such file or directory'
Jan 27 12:27:40 pfs php: /pkg_edit.php: The command '/usr/local/etc/rc.d/dansguardian stop' returned exit code '127', the output was '/usr/local/etc/rc.d/dansguardian: not found'
Jan 27 12:28:50 pfs check_reload_status: Syncing firewall
Jan 27 12:28:50 pfs check_reload_status: Syncing firewall
Jan 27 12:28:50 pfs php: /pkg_edit.php: The command 'chown -R nobody:nobody /usr/loca/etc/dansguardian' returned exit code '1', the output was 'chown: /usr/loca/etc/dansguardian: No such file or directory'
Jan 27 12:28:50 pfs php: /pkg_edit.php: The command '/usr/local/etc/rc.d/dansguardian stop' returned exit code '127', the output was '/usr/local/etc/rc.d/dansguardian: not found'

Thanks for your help!

marcelloc

Thanks for the feedback, there is a missing "l" on path.

I'm sitll looking for other issues, until commit the patches.

marcelloc

Just published some fixes and changed version to 0.1.1

features not finished yet:

blacklist update frequency
ssl man in the middle feature

Cino

figured I would give this a try this morning. first thing i noticed, there is no menu option and I dont see it under status:services. But i'm able to access it by typing https://…../pkg_edit.php?xml=dansguardian.xml. and able to see it running via cmdline

allowed 127.0.0.1 access in the squid3 config and was able to use it... enabled the porn filter for testing and worked as designed! Great job man!!! I haven't used dansguardian in years and forgot how complicate the config can be...

how does clamav update its virus def file?

marcelloc

@Cino:

figured I would give this a try this morning. first thing i noticed, there is no menu option and I dont see it under status:services. But i'm able to access it by typing https://…../pkg_edit.php?xml=dansguardian.xml. and able to see it running via cmdline

Isn't there on second menu option? ???

If tested on both amd64 and i386 but only on 2.0.1

@Cino:

I haven't used dansguardian in years and forgot how complicate the config can be…

I know that. Almost 40 config files :P
I've created almost 80% of default config files during first apply/install process.

dansguardian_menu.png_thumb

Cino

i am using 2.1-dev so it could because of that.. i did reboot the box and it took 2-3 minutes for it to get past the starting package… it started tho.

i'll mess with it some more but if other users arn't having the same issue... then its my setup.

marcelloc

@Cino:

i am using 2.1-dev so it could because of that.. i did reboot the box and it took 2-3 minutes for it to get past the starting package… it started tho.

I think It's calling sync process for every XML file during boot.

I'm trying to workaround it.

Cino

i was able to get it to display in the services menu… i ended up re-installing just the gui items. cosmetic, but i would add it to the status: services menu down the road to know its running or not.

should i run a cron job to update the virus db for clamav (freshclam)? or is it built-in somehow?

marcelloc

The first update from clamav is automatic when package detects that there is no database.

I will be an option on gui until I finish schedulle updates( blacklist and clamav)

Cino

thanks again and great work!! You have added some nice add-ons to pfsense over the last couple of months

no issue, but here are the processes on my box.. this is how i know its running:


: ps -aux | grep dan
clamav 13281  0.0  0.4 18036 14028  ??  INs   3:00PM   0:00.08 /usr/local/sbin/dansguardian
clamav 13525  0.0  0.2  7796  4692  ??  IN    3:00PM   0:00.03 /usr/local/sbin/dansguardian
clamav 13655  0.0  0.1  7796  4672  ??  IN    3:00PM   0:00.02 /usr/local/sbin/dansguardian
clamav 13759  0.0  0.5 19060 14840  ??  IN    3:00PM   0:00.43 /usr/local/sbin/dansguardian
clamav 14008  0.0  0.5 18036 14172  ??  IN    3:00PM   0:00.35 /usr/local/sbin/dansguardian
clamav 14105  0.0  0.5 18036 14120  ??  IN    3:00PM   0:00.02 /usr/local/sbin/dansguardian
clamav 14199  0.0  0.5 18036 14120  ??  IN    3:00PM   0:00.02 /usr/local/sbin/dansguardian
clamav 14385  0.0  0.5 18036 14264  ??  IN    3:00PM   0:00.87 /usr/local/sbin/dansguardian
clamav 14451  0.0  0.4 18036 13988  ??  IN    3:00PM   0:00.01 /usr/local/sbin/dansguardian
clamav 14466  0.0  0.4 18036 13988  ??  IN    3:00PM   0:00.01 /usr/local/sbin/dansguardian
clamav 14472  0.0  0.4 18036 13988  ??  IN    3:00PM   0:00.01 /usr/local/sbin/dansguardian
clamav 16150  0.0  0.4 18036 14052  ??  IN    3:40PM   0:00.00 /usr/local/sbin/dansguardian
clamav 16221  0.0  0.4 18036 14052  ??  IN    3:40PM   0:00.00 /usr/local/sbin/dansguardian
clamav 16515  0.0  0.4 18036 14052  ??  IN    3:40PM   0:00.00 /usr/local/sbin/dansguardian
clamav 16582  0.0  0.4 18036 14052  ??  IN    3:40PM   0:00.00 /usr/local/sbin/dansguardian
clamav 16676  0.0  0.4 18036 14052  ??  IN    3:40PM   0:00.00 /usr/local/sbin/dansguardian
clamav 16741  0.0  0.4 18036 14052  ??  IN    3:40PM   0:00.00 /usr/local/sbin/dansguardian
clamav 16763  0.0  0.4 18036 14052  ??  IN    3:40PM   0:00.01 /usr/local/sbin/dansguardian
clamav 16828  0.0  0.4 18036 14052  ??  IN    3:40PM   0:00.01 /usr/local/sbin/dansguardian
clamav 16843  0.0  0.4 18036 14052  ??  IN    3:40PM   0:00.01 /usr/local/sbin/dansguardian
clamav 17095  0.0  0.4 18036 14052  ??  IN    3:40PM   0:00.00 /usr/local/sbin/dansguardian
root   13272  0.0  0.0  3524  1252   0  S+    3:47PM   0:00.00 grep dan

marcelloc

It's running very well.

I'll include service option too. Maybe tomorrow.

marcelloc

version 0.1.2 is out with some fixes and improvements

improve blacklist import and category assign
include dansguardian on service
include log, daemon and config default xml options
fix some typos
Maybe a faster boot process (01:00 on virtualbox)

still working on…

SSL men in the middle feature
Blacklist and clamav update frequency

To avoid double dansguardian link on service menu(dansguardian and Dansguardian), remove 0.1.1 package before install 0.1.2

Cino

thanks for the update.. I've give it a try in a bit