Dansguardian 2.12.0.3 Signal 11

Fredb

Marcelloc as you know there was a problem with pcre recently, do you compile with pcre on or off ?
Can we make a test with off ?

rjcrowder

@Fredb:

Seems close to http://forum.pfsense.org/index.php?topic=58395.0 in both case the process seem doesn't really crashed (no need to restart)

I know I've seen it since 12.0.3 Not sure prior to that…

marcelloc

@Fredb:

Marcelloc as you know there was a problem with pcre recently, do you compile with pcre on or off ?
Can we make a test with off ?

I think it's marked as a dependence on freebsd ports, I'll try to compile it without pcre.

Does dansguardian works fine without pcre????

rjcrowder

@marcelloc:

Does dansguardian works fine without pcre????

see this http://contentfilter.futuragts.com/wiki/doku.php?id=pattern_matching. However - looks like compiling without PCRE may cause a problem with some of the regex expressions in the current config…

Fredb

Yes, pcre is for advanced regex

rjcrowder

@Fredb:

Very, very strange, there is no error in debug's file between Date: Sat, 01 Jun 2013 17:02:09 GMT and Date: Sat, 01 Jun 2013 17:04:31 GMT
DG run with very low load, without problem

busychildren:1
freechildren:32
waitingfor:0

Seems close to http://forum.pfsense.org/index.php?topic=58395.0 in both case the process seem doesn't really crashed (no need to restart)

I don't know what to think, there is no problem with oldest versions ?

EDIT: For those who suffer from "dansguardian -N" issues I will release a new version with a fix next week, here http://numsys.eu/search.php?search=Dansguardian

Hey guys… just wanted to bump this thread and see if there was any update. I'm still getting the occasional signal 11. Doesn't seem to cause any perceptible problem, but I"m thinking it's something that should get fixed... Any further ideas or things I can test to help with this issue?

Fredb

@rjcrowder

We had rewritten a part of engine for Linux ( http://numsys.eu/search.php?search=Dansguardian ), but the project need help for Freebsd, testing and new codes …
I'm searching how will the project be sustained after, an sponsor ? a fork ? I do not yet know

rjcrowder

@Fredb:

@rjcrowder

We had rewritten a part of engine for Linux ( http://numsys.eu/search.php?search=Dansguardian ), but the project need help for Freebsd, testing and new codes …
I'm searching how will the project be sustained after, an sponsor ? a fork ? I do not yet know

So is there already a BSD version that uses kqueue? Does the epoll rewrite fix the sig 11 issue?

No promises, but I might pull the code down for 2.12.0.7 and take a look at it…

Fredb

Sorry about my English, I guess I'm not clear

I mean DG used not optimized codes, in Fatcontrol.cpp, Philip Pearce has rewrites large parts of codes for Linux only (epoll), it's the first step, and the result is very very impressive.

My part was to preserve the old code in compilation, so:

Linux -> Many parts with epoll (still some old parts)
BSD -> Perhaps I made some minor fixes, and perhaps I put some new bugs, but no major changes (old codes)

So for beginning this version need tests with BSD (code broken ?), it's the second step, and after will do exactly the same work for BSD ( but with Kqueue http://en.wikipedia.org/wiki/Kqueue ).

I think we also works about some new features …

rjcrowder

@Fredb:

My part was to preserve the old code in compilation, so:

Linux -> Many parts with epoll (still some old parts)
BSD -> Perhaps I made some minor fixes, and perhaps I put some new bugs, but no major changes (old codes)

Is this the latest code with your changes? http://numsys.eu/dansguardian/dansguardian-2.12.0.7.tar.gz

Fredb

@rjcrowder:

@Fredb:

My part was to preserve the old code in compilation, so:

Linux -> Many parts with epoll (still some old parts)
BSD -> Perhaps I made some minor fixes, and perhaps I put some new bugs, but no major changes (old codes)

Is this the latest code with your changes? http://numsys.eu/dansguardian/dansguardian-2.12.0.7.tar.gz

Yes right, all the code is between #ifdef linux tag in Fatcontrol.cpp (code from Philip Pearce of http://www.e2bn.org/ )
I think there are also some work to do in basesocket.cpp, we will do it later, to improve again

Fredb

Please try
http://numsys.eu/dansguardian/dansguardian-2.12.0.7.1.tar.gz

neil

This problem has grown to become a major issue for several sites I support and any help would be much appreciated. I joined this thread several months ago, at which time the problem was not so serious.

The setup is as follows:

PF Sense 2.0.3 amd64
DG 2.12.0.3 v.0.1.8
Squid 2.7.9 v.4.3.3
The server is an IBM x3100 with 8Gb RAM, 500Gb HDD.

Min/Max Children 32/1200 (anything less than 1000 for the Maximum causes swarms of Signal 11 errors with complete loss of internet access for the users)

On a test server I attempted to download the patch version 2.12.0.5 made available by Marcelloc but could not successfully install this.

I've now downloaded 2.12.0.6.sig11 made available by Marcelloc, and finally got this running on a test server.
The steps I followed were:

-Download the tbz file to a Windows machine, and copy this across to the PFsense server
(using the fetch command on the pfsense server failed with an invalid tbz file error)

-killall dansguardian

pkg_delete dansguardian- 2.12.0.3
(this created a number of errors relating to files/directories which it could not delete)
pkg_add dansguardian-2.12.0.6.sig11.tbz
This gave a warning that clamav-0.97.8 was required but clamav-0.97.6 was installed.

-attempted to start DG which failed…....

In particular, I had to edit dansguardian.conf and add entries for proxyexchange and pcontimeout, delete two files \tmp.dguardianipc and \tmp.dguardianurl.ipc - then DG would start.
Also having changed dansguardian.conf manually, any change via the WebGui wipes these out.

While the above was fine on a test server, to do the above on a live production server would be most unsatisfactory.

For the moment, upping the Max Children to 1200 seems to have reduced the frequency of the errors.

I'd be really grateful if anyone can tell me how to implement the patched version 2.12.0.6 (or later) from Marcelloc, avoiding the above issues, and at the same time preserve the groups/lists etc which have been heavily tailored.
Alternatively I see that others have gone back to 2.12.0.2 - where can I find this and would installing this destroy the lists etc .

Fredb

There is no coredump ? I don't know freebsd but with Linux you can put "ulimit -c unlimited" in the dg start script to generate
Unfortunately there is no useful information with debug, now we need a core (exited on signal 11 (core dumped)) to know exactly what happen

Soon we will open a new project, I 'm rewriting some codes now to fix another problem with 2.12.07.1 and freebsd

rjcrowder

@neil:

In particular, I had to edit dansguardian.conf and add entries for proxyexchange and pcontimeout, delete two files \tmp.dguardianipc and \tmp.dguardianurl.ipc - then DG would start.
Also having changed dansguardian.conf manually, any change via the WebGui wipes these out.

While the above was fine on a test server, to do the above on a live production server would be most unsatisfactory.

See this post earlier in the same thread http://forum.pfsense.org/index.php/topic,61811.msg338588.html#msg338588

neil

Thanks - I'll look for the template and modify it.

I'm having trouble keeping track of which version of DG is actually installed and running.
When I run pkg_delete prior to installing 2.12.0.6.sig11.tbz, the dg binary along with
a number of list files (etc) are not removed.
Then when I add 2.12.0.6.sig11, the size of the DG binary does not change.

However, if prior to installing 2.12.0.6..sig11.tbz, I mv dg to dg_bak, and then
run the install, the size of the DG binary is completely different.

So, is there a way of determining the version of the DG binary?

Fredb

@neil:

Thanks - I'll look for the template and modify it.

I'm having trouble keeping track of which version of DG is actually installed and running.
When I run pkg_delete prior to installing 2.12.0.6.sig11.tbz, the dg binary along with
a number of list files (etc) are not removed.
Then when I add 2.12.0.6.sig11, the size of the DG binary does not change.

However, if prior to installing 2.12.0.6..sig11.tbz, I mv dg to dg_bak, and then
run the install, the size of the DG binary is completely different.

So, is there a way of determining the version of the DG binary?

It's a debug version, I think you don't need this
Marcelloc and me are working about a stable release for pfsense, I hope we found something
In worst situation sig11 still present until someone rewrite some parts of code for BSD like (no bug with Linux)

neil

Thanks I did not realise that this was a debug version - I have three sites that I have set up, each with up to 300 users.
Once the number of users gets above about 200, the Signal 11 errors occur about ever 2 -3 minutes, and the users
experience sluggish internet response.
I'll wait until an update package is available.
Could you send me an email when this occurs? My email address is:
neil@darfield.school.nz. Alternatively I'll monitor this thread.

Many thanks for your efforts.

Fredb

Hi all,

We have recently fixed some bugs with BSD, history here https://github.com/e2guardian/e2guardian/issues/8
Please, someone (should be an advanced user …) can try this new build https://github.com/e2guardian/e2guardian/archive/master.zip ?

You must rename the configurations files, like this /etc/dansguardian/dansguardian.conf -> /etc/e2guardian/e2guardian.conf
And rewrite your start script (dansguardian/e2guardian) alternatively you should also just rename the new binary to dansguardian

Keep in mind it's a working version, with some news features maybe unstable ...

marcelloc

HI Fredb,

Do you plan to push it to freebsd ports? It will be much easier to compile and pack it.

Thank's for all your work on it. dansguardian is still on 2.12.0.3 with no patches applied to ip.