Dansguardian 2.12.0.3 Signal 11

LokisMischief

With the latest version you can adjust maxchildren (maximun value) with your system
For example on linux :

ulimit -n 8192 -> new ./configure option = with-filedescriptors=8192 = dansguardian.conf maxchildren=8192

Maybe this is a clue ? Perhaps this version was compiled with too much high value for the system ? Can you play with ulimit ?
How many process are running when the crash appear ? ps -edf | grep dansguard | wc -l

Well you can adjust the max/min children in the conf file, but it didn't seem to make much difference, same config file with the previous version (minus the bits added for that particular version) works. I'm afraid I cant count the processes, rolled back to 2.12.0.2 and don't currently have a dev box running only production.
If I get a chance I will run up a vm for it "later"

rjcrowder

@LokisMischief:

Well you can adjust the max/min children in the conf file, but it didn't seem to make much difference, same config file with the previous version (minus the bits added for that particular version) works. I'm afraid I cant count the processes, rolled back to 2.12.0.2 and don't currently have a dev box running only production.
If I get a chance I will run up a vm for it "later"

Yea, I'm still having he problem. About ever other day I get a half dozen or so DG processes ending with signal 11. Are you saying one of the versions doesn't do this? If so, which one?

Fredb

Signal 11 means that the program accessed a memory location that was not assigned to it, the strange thing that there is no problem in Linux (with dansguardian 2.12.0.5)

Please, Can you post your maxchildren value ? More than 1024 ?
And if someone know the value of FD_SETSIZE in types.h (or posix_types.h) and typesizes.h with FreeBSD ?
Also can you post the compilation option (dansguardian -v)

No problem at all with 2.12.0.2 ?

Thanks

rjcrowder

@Fredb:

Signal 11 means that the program accessed a memory location that was not assigned to it, the strange thing that there is no problem in Linux (with dansguardian 2.12.0.5)

Please, Can you post your maxchildren value ? More than 1024 ?
And if someone know the value of FD_SETSIZE in types.h (or posix_types.h) and typesizes.h with FreeBSD ?
Also can you post the compilation option (dansguardian -v)

No problem at all with 2.12.0.2 ?

Thanks

Based on some notes here http://contentfilter.futuragts.com/wiki/doku.php?id=faq (see FAQ 26b)
I have bumped the following sysctl values (in loader.conf.local):
kern.ipc.shmseg=512
kern.ipc.shmmni=512
kern.ipc.semmni=512
kern.ipc.msgssz=64
kern.ipc.shm_use_phys=1

at the moment, I have maxchildren set to 120 and maxsparechildren at 48

mschiek01

I am running 2.12.0.3 pkg v.0.1.7_3 and have not seen this issue at all.

All of the setting I am using are the default.

marcelloc

@mschiek01:

I am running 2.12.0.3 pkg v.0.1.7_3 and have not seen this issue at all.

mschiek01 told me some time ago an issue with a specific perl version.

Try to unistall package, remove all perl versions using pkg_delete on console and then try a dansguardian package reinstall.

L_P

I'm also suffering this issue.

Lots of Signal 11 messages show up when the system is under load - about 40 office users with normal daily activities such as web browsing, email,…

I am using 2.0.2-RELEASE (i386) with patched Dans for web uploads

rjcrowder

@rjcrowder:

Based on some notes here http://contentfilter.futuragts.com/wiki/doku.php?id=faq (see FAQ 26b)
I have bumped the following sysctl values (in loader.conf.local):
kern.ipc.shmseg=512
kern.ipc.shmmni=512
kern.ipc.semmni=512
kern.ipc.msgssz=64
kern.ipc.shm_use_phys=1

at the moment, I have maxchildren set to 120 and maxsparechildren at 48

Still getting them…

I'm not really wanting to try a "reinstall" though... This is a fresh install of pfSense 2.0.3 64 bit and the only packages that I've added are:

Cron
File Manager
vHosts
Dansguardian
Squid 3

I'm currently using the patched dansguardian 2.12.0.3 (just copied over the executable).

Fredb

Please, can you try this latest version and let me know if it works (better) for you ? http://numsys.eu/search.php?search=Squid

marcelloc

@Fredb:

Please, can you try this latest version and let me know if it works (better) for you ? http://numsys.eu/search.php?search=Squid

I'll compile it and push to my repo.

Fredb, nice to see you on pfsense forum :)

Most work I did on dansguardian 2.12 was for this package on pfsense.

Fredb

Hi,
Your work is included in "my" dansguardian version

I hope, if I can …, rewrite the engine with kqueue for *BSD and epool for Linux and remove the old select() call, maybe this point is a part of problem signal 11

marcelloc

2.12.0.6 compiled and pushed to my repo.

amd64
http://e-sac.siteseguro.ws/packages/amd64/8/All/dansguardian-2.12.0.6.tbz

i386
http://e-sac.siteseguro.ws/packages//8/All/dansguardian-2.12.0.6.tbz

both complied with maxfiles=8192

Also, I've removed squid ports compile depend. It will not force any squid version anymore.

Fredb

Configuration files http://numsys.eu/dansguardian/

Requires

Proxy timeout

Set tcp timeout between the Proxy and DansGuardian

Min 5 - Max 100

proxytimeout = 20

Proxy header exchange

Set timeout between the Proxy and DansGuardian

Min 20 - Max 300

proxyexchange = 20

Pconn timeout

how long a persistent connection will wait for other requests

squid apparently defaults to 1 minute (persistent_request_timeout),

so wait slightly less than this to avoid duff pconns.

Min 5 - Max 300

pcontimeout = 55

Now you can can disabled some (if) unused values, like maxcontentramcachescansize, I think It should be interesting about signal 11 and a potential memory leak.

rjcrowder

@Fredb:

Configuration files http://numsys.eu/dansguardian/

Requires

Proxy timeout

Set tcp timeout between the Proxy and DansGuardian

Min 5 - Max 100

proxytimeout = 20

Proxy header exchange

Set timeout between the Proxy and DansGuardian

Min 20 - Max 300

proxyexchange = 20

Pconn timeout

how long a persistent connection will wait for other requests

squid apparently defaults to 1 minute (persistent_request_timeout),

so wait slightly less than this to avoid duff pconns.

Min 5 - Max 300

pcontimeout = 55

Now you can can disabled some (if) unused values, like maxcontentramcachescansize, I think It should be interesting about signal 11 and a potential memory leak.

Marcello… are you going to add these config settings to the UI? If I manually add them to the config files, will they be dropped when I save via the UI?

marcelloc

@rjcrowder:

Marcello… are you going to add these config settings to the UI?

yes. you can force it on dansguardian.inc near

$proxytimeout=($dansguardian['proxytimeout']?$dansguardian['proxytimeout']:"30");

@rjcrowder:

If I manually add them to the config files, will they be dropped when I save via the UI?

Yes.

rjcrowder

@marcelloc:

yes. you can force it on dansguardian.inc near
$proxytimeout=($dansguardian['proxytimeout']?$dansguardian['proxytimeout']:"30");
Thanks - No problem… I'll add them on my setup.

rjcrowder

@marcelloc:

yes. you can force it on dansguardian.inc near

The easiest thing to do was to just add them into dansguardian.conf.template - so that's what I did for now.

Up and running with 2.12.0.6 - I'll keep you posted…!!!

Thanks!

Fredb

If You See Something, please try to reduce your unused values to 0

rjcrowder

@Fredb:

If You See Something, please try to reduce your unused values to 0

Besides "maxcontentramcachescansize", which ones are no longer used?

Fredb

Yes, options with maxsomething, for example those for AV scan, but not now please
EDIT: Only those you don't use of course

Max content filter size

Sometimes web servers label binary files as text which can be very

large which causes a huge drain on memory and cpu resources.

To counter this, you can limit the size of the document to be

filtered and get it to just pass it straight through.

This setting also applies to content regular expression modification.

The value must not be higher than maxcontentramcachescansize

The size is in Kibibytes - eg 2048 = 2Mb

use 0 to set it to maxcontentramcachescansize

maxcontentfiltersize = 0 -> If weightedphrasemode = 0

Max content ram cache scan size

This is only used if you use a content scanner plugin such as AV

This is the max size of file that DG will download and cache

in RAM. After this limit is reached it will cache to disk

This value must be less than or equal to maxcontentfilecachescansize.

The size is in Kibibytes - eg 10240 = 10Mb

use 0 to set it to maxcontentfilecachescansize

This option may be ignored by the configured download manager.

maxcontentramcachescansize = 0 if no AV

Max content file cache scan size

This is only used if you use a content scanner plugin such as AV

This is the max size file that DG will download

so that it can be scanned or virus checked.

This value must be greater or equal to maxcontentramcachescansize.

The size is in Kibibytes - eg 10240 = 10Mb

maxcontentfilecachescansize = 0 if no AV