Dansguardian package for 2.0

rjcrowder

rjcrowder,

take a look(with a package reinstall) if fetch_blacklist and clamav call are fine again. I'll bump version after these small bugs are gone.

att,
Marcello Coutinho

Looks good from what I can tell… manually checked the code plus all the scenarios that I know of where it starts/stops clamd and dans.

There is one other behavior that is a little annoying that might be fixable... just haven't looked at the code to see how yet. If the dansguardian config is setup to use a different blacklist than the default, then a "reinstall" of the package breaks dans until you do a "fetch_blacklist". Basically, dans won't start because the blacklists in the config files don't match the blacklist directories that are created when a "reinstall" is done. One way to fix it would be to automatically do a "fetch_blacklist" at the end of a package reinstall. Not a big deal... just always have to do it manually after package reinstall...

marcelloc

@rjcrowder:

Basically, dans won't start because the blacklists in the config files don't match the blacklist directories that are created when a "reinstall" is done.

The install/reinstall code does check backlist during process install and apply it. I'll take a last look on it before version bump.

LokisMischief

@marcelloc:

@LokisMischief:

Not sure where to start looking for this one :(

Did you applied the binary patch fot fix web upload bug? you can try to downgrade binaries via pkg_add too.

Well, I have updated to your latest package, still had the same issue, child processes dropping out with sig 11's, applied your upload patch (incidentally I didn't have any web upload issues) but still the same sig 11's

Edit:

For more information on this issue, read here: http://forum.pfsense.org/index.php/topic,61811.0.html

Legion

marcelloc, I mentioned once before that DG fills up my cron table with fetch_blacklist entries:

I'm not sure why it happens. I'm guessing every time I save a config it puts another entry in the crontab? The main reason I noticed was I was doing things on my pfSense box at about midnight and noticed it suddenly spike in CPU, memory, etc so I had a look and it was trying to fetch the blacklist 100s of times at once.

maxxer

Hi. I cannot manage to have DG include blacklists in config.
It's a pfsense 2.1 beta installation, with dg 2.12.
I even tried to upgrade package gui from github, but no change.

Configuration is mostly the default. After adding blackslit url (shallalist) I go to dansguardian > ACLs > Site list, then edit the default entry. I enable the "Banned" checkbox, then select one (or all) the blacklist include, press Save. No domain in the blacklist is taken into consideration.
If I manually add a domain to the "Config" text area just below it works, correctly blocked.
If I manually add

.Include

in the same textarea sites are correcrty blocked.

what's wrong?
thanks

rjcrowder

@maxxer:

Hi. I cannot manage to have DG include blacklists in config.
It's a pfsense 2.1 beta installation, with dg 2.12.
I even tried to upgrade package gui from github, but no change.

Configuration is mostly the default. After adding blackslit url (shallalist) I go to dansguardian > ACLs > Site list, then edit the default entry. I enable the "Banned" checkbox, then select one (or all) the blacklist include, press Save. No domain in the blacklist is taken into consideration.
If I manually add a domain to the "Config" text area just below it works, correctly blocked.
If I manually add
.Include
in the same textarea sites are correcrty blocked.

what's wrong?
thanks

Did you download the new blacklists before trying to select them? The lists are not the same and the categories will change once you download the list.

maxxer

@rjcrowder:

Did you download the new blacklists before trying to select them? The lists are not the same and the categories will change once you download the list.

Yes, I first downloaded the blacklists then tried to add them to the filter. Confirmed by the populated blacklist list in the box…

rjcrowder

@maxxer:

@rjcrowder:

Did you download the new blacklists before trying to select them? The lists are not the same and the categories will change once you download the list.

Yes, I first downloaded the blacklists then tried to add them to the filter. Confirmed by the populated blacklist list in the box…

Is the selection being saved? In other words, are they highlighted in the list (as you ctrl-click them)?

If you want to check that the UI is writing the config correctly, you can do the following… First, check and see if the list is getting written to the config.xml file (in /conf). If you vi the file, search for "banned_includes" - It should occur three times in the config, once for phraselists, once for sites, and once for URL's. It will be an encoded string and you wont be able to make any sense of it, but make sure the config is getting written here first... Next, check that it is getting properly written into the dansguardian config files located in /usr/local/etc/dansguardian/lists... for example, the one for banned sites for the default group is called "bannedsitelist.Default".

marcelloc

version 2.12.0.6(comunity patches) compiled and pushed to my repo.

Thanks to Fredb. :)

amd64
http://e-sac.siteseguro.ws/packages/amd64/8/All/dansguardian-2.12.0.6.tbz

i386
http://e-sac.siteseguro.ws/packages//8/All/dansguardian-2.12.0.6.tbz

both complied with maxfiles=8192

Also, I've removed squid ports compile depend. It will not force any squid version anymore.

see changelog here

maxxer

@rjcrowder:

Is the selection being saved? In other words, are they highlighted in the list (as you ctrl-click them)?

If you want to check that the UI is writing the config correctly, you can do the following… First, check and see if the list is getting written to the config.xml file (in /conf). If you vi the file, search for "banned_includes" - It should occur three times in the config, once for phraselists, once for sites, and once for URL's. It will be an encoded string and you wont be able to make any sense of it, but make sure the config is getting written here first... Next, check that it is getting properly written into the dansguardian config files located in /usr/local/etc/dansguardian/lists... for example, the one for banned sites for the default group is called "bannedsitelist.Default".

thanks for your support!

yes, selection is saved and config.xml is correclty populated. Tough I'm missing bannedsitelist.Default. I just have bannedsitelist. So this should be the problem. I guess 2.1 is not totally supported yet?

rjcrowder

@maxxer:

thanks for your support!

yes, selection is saved and config.xml is correclty populated. Tough I'm missing bannedsitelist.Default. I just have bannedsitelist. So this should be the problem. I guess 2.1 is not totally supported yet?

Hmmm… I'm at a little bit of a loss then. I'm running 2.0.3, so I don't know if there is a problem on 2.1 or not. I would think it would still create "bannedsitelist.Default" though... Is the name "Default" filled in at the top of the page? I suppose it would have to be if it is populating into the config.xml file.

Marcello is the package owner - think he's on 2.1. Maybe he has an idea...

wheelz

Now with the new squid package I'm looking to try to get dansguardian filtering HTTPS traffic as well. I have the CA cert and I created a test server cert and enabled ssl filtering. Do I need to get the latest from your repo first? When I try to access HTTPS through I get:

Secure Connection Failed

An error occurred during a connection to www.google.com.

Improperly formatted time string.

(Error code: sec_error_invalid_time)

marcelloc

@Legion:

marcelloc, I mentioned once before that DG fills up my cron table with fetch_blacklist entries:

Try a package reinstall and then a save config.

I've fixed the cron problem but did not bumped the version.

marcelloc

@wheelz:

(Error code: sec_error_invalid_time)

Yes, it's fixed on dansguardian from my repo. But I think there are still other problems with dansguardian mitm.

try latest version and see if it's working on your setup.

Downloadski

I am new to pfsense and freebsd so i could be making user errors..

I have installed pfsense 2.1 (2.1-RC0 (amd64) built on Mon Jun 24 04:05:41 EDT 2013 FreeBSD 8.3-RELEASE-p8)

I would like to get dansguardian to work.

After reading multiple topics, i first installed suid3 (3.1.20 pkg 2.0.6) from the packages menu.
This seems to work ok, as i can see in the real time proxy monitor the sites i browse.

When i install dansguardian from the packages menu i get the following errors in the log

Jun 25 09:46:37 php: /pkg_mgr_install.php: [Dansguardian] - Save settings package call pr: bp:1 rpc:no 
Jun 25 09:46:37 php: /pkg_mgr_install.php: Starting Dansguardian 
Jun 25 09:46:37 dansguardian[15691]: Error reading PICS file: /usr/pbi/dansguardian-amd64/etc/dansguardian/lists/g_Default 
Jun 25 09:46:37 dansguardian[15691]: Error opening filter group config: /usr/pbi/dansguardian-amd64/etc/dansguardian/dansguardianf1.conf 
Jun 25 09:46:37 dansguardian[15691]: Error reading filter group conf file(s). 
Jun 25 09:46:37 dansguardian[15691]: Error parsing the dansguardian.conf file or other DansGuardian configuration files 
Jun 25 09:46:37 root: /usr/local/etc/rc.d/dansguardian.sh: WARNING: failed to start dansguardian 
Jun 25 09:46:37 php: /pkg_mgr_install.php: The command '/usr/local/etc/rc.d/dansguardian.sh start' returned exit code '1', the output was 'kern.ipc.somaxconn: 16384 -> 16384 kern.maxfiles: 131072 -> 131072 kern.maxfilesperproc: 104856 -> 104856 kern.threads.max_threads_per_proc: 4096 -> 4096 Starting dansguardian. Error reading PICS file: /usr/pbi/dansguardian-amd64/etc/dansguardian/lists/g_Default Error opening filter group config: /usr/pbi/dansguardian-amd64/etc/dansguardian/dansguardianf1.conf Error reading filter group conf file(s). Error parsing the dansguardian.conf file or other DansGuardian configuration files /usr/local/etc/rc.d/dansguardian.sh: WARNING: failed to start dansguardian'

So it seems there are items missing there.
Also when i look under services, dansguardian is stopped.

I tried to install marcelloc his latest version: pkg_add -r http://e-sac.siteseguro.ws/packages/amd64/8/All/dansguardian-2.12.0.6.tbz
But this finishes very fast and i think it is only downloaded and not installed.

[2.1-RC0][admin@pfsense.localdomain]/var/log(69): pkg_add -r http://e-sac.siteseguro.ws/packages/amd64/8/All/dansguardian-2.12.0.6.tbz
Fetching http://e-sac.siteseguro.ws/packages/amd64/8/All/dansguardian-2.12.0.6.tbz... Done.

===>   Please Note:

*******************************************************************************
       This port has created a log file named dansguardian.log that can get
       quite large.  Please read the newsyslog(8) man page for instructions
       on configuring log rotation and compression.

       WITH_CLAMAV, WITH_ICAP, WITH_KASP, WITH_NTLM are all experimental
       options that I am not currently able to test.  Let me know how these
       work (or not) for you.  (Patches always welcome.)
*******************************************************************************

[2.1-RC0][admin@pfsense.localdomain]/var/log(70):

I tried this with the 2.12.0.3 pkg v.0.1.8 from the packages menu installed, and also when i remove that. Same results.

as for my network:
I have a intel 2 port PCI-E card and a trunk to a cisco 200-8 managed switch.
There are 4 vlans and i want to use dansguardian only in 1 vlan.

So i have a native (untagged) vlan 1, and than i have vlan 2,3,4 (tagged)

I do not have a wan connection yet, but made a static GW rule via vlan 2 to the outside world on the existing network.
This is the default route.

So i have basic routing from the vlan3 to the outside world working ok.
I have squid working i think. (i have fast internet, so the caching i do not notice, but i see entrys in the proxy monitor)

MY system should have plenty resources:

running from a 64 GB SSD (not completely used)

[2.1-RC0][admin@pfsense.localdomain]/var/log(80):    df
Filesystem  1K-blocks   Used   Avail Capacity  Mounted on
/dev/ad4s1a   8121926 825832 6646340    11%    /
devfs               1      1       0   100%    /dev
/dev/md0         3694     52    3348     2%    /var/run
devfs               1      1       0   100%    /var/dhcpd/dev

memory:

2.1-RC0][admin@pfsense.localdomain]/var/log(81): dmesg | grep memory
real memory  = 17179869184 (16384 MB)
avail memory = 16442249216 (15680 MB)

cpu:

[2.1-RC0][admin@pfsense.localdomain]/var/log(82): dmesg | grep CPU
CPU: Intel(R) Celeron(R) CPU 847 @ 1.10GHz (1097.51-MHz K8-class CPU)
FreeBSD/SMP: Multiprocessor System Detected: 2 CPUs
cpu0: <acpi cpu="">on acpi0
cpu1: <acpi cpu="">on acpi0
p4tcc0: <cpu frequency="" thermal="" control="">on cpu0
p4tcc1: <cpu frequency="" thermal="" control="">on cpu1
SMP: AP CPU #1 Launched!</cpu></cpu></acpi></acpi>

Now my question:

how do i see if i do a: pkg_add -r http://e-sac.siteseguro.ws/packages/amd64/8/All/dansguardian-2.12.0.6.tbz what happens, and what do i need to do.
I can read and see:

This port has been converted to the new RC framework and should work
correctly via rcorder. Please read the comments in the startup script
for instructions on enabling the daemon.

i can look for all dansguardian filename's witha simple: find / -name dansguardian

/usr/local/sbin/dansguardian
/usr/local/etc/rc.d/dansguardian
/usr/local/etc/dansguardian
/usr/local/share/doc/dansguardian
/usr/local/share/dansguardian
/usr/local/share/dansguardian/scripts/dansguardian
/usr/pbi/dansguardian-amd64/etc/rc.d/dansguardian
/usr/pbi/dansguardian-amd64/etc/dansguardian
/usr/pbi/dansguardian-amd64/sbin/dansguardian
/usr/pbi/dansguardian-amd64/share/dansguardian
/usr/pbi/dansguardian-amd64/share/dansguardian/scripts/dansguardian
/usr/pbi/dansguardian-amd64/.sbin/dansguardian
/var/log/dansguardian

in the scripts directory i see a script :)

This fails execution:

/usr/pbi/dansguardian-amd64/share/dansguardian/scripts(108): dansguardian
Error opening/creating log file. (check ownership and access rights).
I am running as nobody and I am trying to open /var/log//access.log

here i get stuck, as i am on the console and the main/root user i think ?

marcelloc

Tbz packages are for pfsense 2.0.x. 2.1 it needs pbi packages..

Downloadski

Ok, where to find them ?

I cannot run 2.03, as my pc does noet bootup with that so i need to run 2.1

rjcrowder

@Downloadski:

This fails execution:

/usr/pbi/dansguardian-amd64/share/dansguardian/scripts(108): dansguardian
Error opening/creating log file. (check ownership and access rights).
I am running as nobody and I am trying to open /var/log//access.log

here i get stuck, as i am on the console and the main/root user i think ?

I got this error once when all of the user rights were correct, but I had dropped "execute" privileges on one of the directories in the path.

Also… the proper install order is Dansguardian - then Squid 3.

Legion

I've been getting "Error connecting via IPC socket to log: Connection refused" messages for as long as I can remember. Once I restart dg it goes away. A Google search found this:

http://www.uno-code.com/?q=node/141

I checked and your dansguardian.conf does the same thing - creates IPC files in /tmp, which I presume is cleaned out by pfsense each day. Can you expose these file paths in the GUI marcelloc? Or modify the conf file writeout and put them in a dg subdirectory like other DG files (e.g. access.log)

ZGruk

Where are the actual config files for dansguardian located? I've noticed when I change them in the GUI (ACL -> Site Lists for example) they don't change in the files on the machine (in /usr/local/dansguardian/lists/bannedsitelist for example). And conversely, editing bannedsitelist using vi doesn't actually block any sites. I assume there are actual files somewhere on the machine that are getting changed when I change it in the GUI, but I haven't been able to find them.