PfSense 2.1-RC1 + Squid3 + DansGuardian - Not filtering correctly

timthetortoise

Hi all, if something similar has been posted, please direct me to the right thread. I could not find anything like my scenario.

Basically my problem is that DansGuardian is not filtering anything at all. I am manually setting the proxy address for DG (running on port 8080). It connects correctly to Squid (running on port 3128) and I can see access logs in both of them (client address in DG logs, 127.0.0.1 in Squid logs). I can get to anything without a problem. However, if I set the default group to "banned," it does give me the correct result (Your username is not allowed to web browse). My ACLs are all at the default settings, and I have a blacklist enabled banning porn domains. I have been scratching my head the past week on this and have no idea what else to try. I'll provide any info that's needed. Thank you in advance for any help!

PS: the same issue happens with squid2.

Edit: I just got a banned page. On a popup on sex.com. And none of the images are showing up. This is a really bizarre issue. To top it off, I can't post here with the proxy on because it blocks it as "Web upload banned."

timthetortoise

Think I have tracked down the problem. In the lists/x.g_Default files, the configurations are getting mucked up. It was adding the path and the right chevron, but no linebreak and .Include< portion. I'll have a look in the source and see if I can fix it. Not sure if this is specific to the amd64 version, but I don't believe it is.

timthetortoise

It looks like this is probably a pretty specific problem. I've made a change to the /usr/local/pkg/dansguardian.inc file, as follows.

Was:


    332         #includes preg_replace patterns
    333         $match[0]="/(.)$/";
    334         $match[1]="/\/usr.local/";
    335         $match[2]="/,/";
    336         $replace[0]="$1>\n";
    337         $replace[1]="\n.Include<".DANSGUARDIAN_DIR;
    338         $replace[2]=">";

Is now:


    332         #includes preg_replace patterns
    333         $match[0]="/(.)$/";
    334         $match[1]="/\/usr.local|".str_replace('/','\\/',DANSGUARDIAN_DIR)."/";
    335         $match[2]="/,/";
    336         $replace[0]="$1>\n";
    337         $replace[1]="\n.Include<".DANSGUARDIAN_DIR;
    338         $replace[2]=">";

I'm sure there's a more graceful way to do it, but this was a quick fix in case anyone has the same issue.

pierre_rs

Thanks for your great job!

This helped me a lot!

2.1-RC1 (i386)

rjcrowder

@timthetortoise:

To top it off, I can't post here with the proxy on because it blocks it as "Web upload banned."

See this on the web upload issue… http://forum.pfsense.org/index.php/topic,58442.0.html

timthetortoise

@pierre_rs:

Thanks for your great job!

This helped me a lot!

2.1-RC1 (i386)

Glad to see I wasn't the only person having this problem, I was getting a little nervous there. Still waiting for the pull request to get merged for this fix.

@rjcrowder:

@timthetortoise:

To top it off, I can't post here with the proxy on because it blocks it as "Web upload banned."

See this on the web upload issue… http://forum.pfsense.org/index.php/topic,58442.0.html

Looks like it's been fixed in the latest release.

timthetortoise

This issue is still not fixed on my new installs - is seriously no one else seeing this issue?

rjcrowder

@timthetortoise:

This issue is still not fixed on my new installs - is seriously no one else seeing this issue?

I'm getting a little lost on this thread. Which issue are you referrring to? The web upload issue?

timthetortoise

No, the issue of Dansguardian not writing filter lists correctly.

To illustrate what's going on, I just did a fresh install on a VM of 2.1-RELEASE.
I installed squid3, then installed Dansguardian.
Dansguardian seems to reside in /usr/pbi/dansguardian-amd64 or /usr/pbi/dansguardian-i386.
The section of code I modified only replaces for /usr/local
So when I cat /usr/pbi/dansguardian-amd64/etc/dansguardian/lists/weightedphraselist.g_Default, I get a garbled config with text like:

/usr/pbi/dansguardian-amd64/etc/dansguardian/lists/phraselists/pornography/weighted_russian_utf8>/usr/pbi/dansguardian-amd64/etc/dansguardian/lists/phraselists/pornography/weighted_spanish>/usr/pbi/dansguardian-amd64/etc/dansguardian/lists/phraselists/pornography/weighted_swedish>

and so on.

When I modify dansguardian.inc with my change and save the config, I get the correct configuration output:

.Include
.Include
.Include

and so on. I'm not sure why it still hasn't been merged on Github, as it seems like every new installation would be experiencing this problem.

rjcrowder

Nice catch… I honestly hadn't paid any attention. Probably because I had only done 2.1 installs on some test boxes. I applied your fix on both an i386 and amd64 install and it certainly does the trick. Did you submit the change on Github? I've submitted changes in the past, and it seems like it takes Marcello a little while to incorporate them... Other than that I don't know what to say.

For my own purposes I will also admit that I apply a number of custom changes every time I do a build. I actually keep a list of "diffs", custom scripts, etc. that get put into a TAR file and then I have a script that deploys them. I added your change to the diffs that are applied by my custom deployment script... If Marcello fixes it, I'll pull the change out...

timthetortoise

Yep, I submitted a pull request about a month ago, he asked if I'd tested, I said yes, nothing after that.

pfSense2User

@timthetortoise:

It looks like this is probably a pretty specific problem. I've made a change to the /usr/local/pkg/dansguardian.inc file, as follows.

Was:


    332         #includes preg_replace patterns
    333         $match[0]="/(.)$/";
    334         $match[1]="/\/usr.local/";
    335         $match[2]="/,/";
    336         $replace[0]="$1>\n";
    337         $replace[1]="\n.Include<".DANSGUARDIAN_DIR;
    338         $replace[2]=">";

Is now:


    332         #includes preg_replace patterns
    333         $match[0]="/(.)$/";
    334         $match[1]="/\/usr.local|".str_replace('/','\\/',DANSGUARDIAN_DIR)."/";
    335         $match[2]="/,/";
    336         $replace[0]="$1>\n";
    337         $replace[1]="\n.Include<".DANSGUARDIAN_DIR;
    338         $replace[2]=">";

I'm sure there's a more graceful way to do it, but this was a quick fix in case anyone has the same issue.

In 2.0.3, the package worked without a problem. In 2.1, this seems to be a bug that requires editing of the specified file. I had to do that today since my pfSense firewall wouldn't filter any longer until I edited that file with the line that was missing other codes, and it now works.

Thanks for the easy fix. I thought I had to install 2.0.3 over again. I saved myself 4 hours of re-installation, re-configuration, and so on.

Again, thanks.

timthetortoise

Got merged on Github, looks to be fixed on new installs.