Sarg package for pfsense

marcelloc · Apr 15, 2012, 1:30 PM

Hello, Just want to know, how far email option system log report work with SARG ?

I'm looking at rrd mail report package to see how it work but I have no idea on when I'll have time to implement it.

Donny · Apr 17, 2012, 5:13 AM

After install SARG I got some error : Sarg config error: dansguardian log file () does not exists. In this case I have already configured Squid3 and also Dansguardian.
At Dansquardian > Report and log tab I selected log File Format to Squid log File Format
At SARG Settings > General tab > proxy server I selected to dansguardian. reboot or not reboot system at this point I always got warning Sarg config error: dansguardian log file () does not exists If I change proxy server to Squid the warning error is disappear.

This is my system log file:

Apr 16 01:19:37 php: : Reloading Dansguardian
Apr 16 01:19:37 php: : Reloading Dansguardian
Apr 16 01:19:38 php: : Reloading Dansguardian
Apr 16 01:19:38 php: : Reloading Dansguardian
Apr 16 01:19:38 php: : Reloading Dansguardian
Apr 16 01:19:40 php: : [squid] xmlrpc sync is starting.
Apr 16 01:19:41 php: : Starting Squid
Apr 16 01:19:41 squid[19250]: Squid Parent: child process 19565 started
Apr 16 01:19:41 check_reload_status: Reloading filter
Apr 16 01:19:41 php: : [squid] xmlrpc sync is starting.
Apr 16 01:19:42 php: : Reloading Squid for configuration sync
Apr 16 01:19:42 check_reload_status: Reloading filter
Apr 16 01:19:42 php: : [squid] xmlrpc sync is starting.
Apr 16 01:19:42 php: : Reloading Squid for configuration sync
Apr 16 01:19:42 php: : [squid] xmlrpc sync is starting.
Apr 16 01:19:43 php: : Reloading Squid for configuration sync
Apr 16 01:19:43 php: : [squid] xmlrpc sync is starting.
Apr 16 01:19:43 php: : Reloading Squid for configuration sync
Apr 16 01:19:43 php: : [squid] xmlrpc sync is starting.
Apr 16 01:19:44 php: : Reloading Squid for configuration sync
Apr 16 01:19:44 php: : Not calling package sync code for dependency squidcache of squid3 because some include files are missing.
Apr 16 01:19:44 php: : Not calling package sync code for dependency squidnac of squid3 because some include files are missing.
Apr 16 01:19:44 php: : Not calling package sync code for dependency squidtraffic of squid3 because some include files are missing.
Apr 16 01:19:44 php: : Not calling package sync code for dependency squidupstream of squid3 because some include files are missing.
Apr 16 01:19:44 php: : Not calling package sync code for dependency squidreverse of squid3 because some include files are missing.
Apr 16 01:19:44 php: : Not calling package sync code for dependency squidauth of squid3 because some include files are missing.
Apr 16 01:19:44 php: : Not calling package sync code for dependency squidusers of squid3 because some include files are missing.
Apr 16 01:19:46 php: : Sarg config error: dansguardian log file () does not exists
Apr 16 01:19:46 php: : New alert found: Sarg config error: dansguardian log file () does not exists
Apr 16 01:19:46 check_reload_status: Syncing firewall
Apr 16 01:19:46 php: : [sarg] sarg_xmlrpc_sync.php is starting.
Apr 16 01:19:46 php: : Sarg config error: dansguardian log file () does not exists
Apr 16 01:19:46 php: : New alert found: Sarg config error: dansguardian log file () does not exists
Apr 16 01:19:46 check_reload_status: Syncing firewall
Apr 16 01:19:46 php: : [sarg] sarg_xmlrpc_sync.php is starting.
Apr 16 01:19:46 php: : Sarg config error: dansguardian log file () does not exists
Apr 16 01:19:46 php: : New alert found: Sarg config error: dansguardian log file () does not exists
Apr 16 01:19:47 php: : [sarg] sarg_xmlrpc_sync.php is starting.
Apr 16 01:19:47 php: : Sarg config error: dansguardian log file () does not exists
Apr 16 01:19:47 php: : New alert found: Sarg config error: dansguardian log file () does not exists
Apr 16 01:19:47 php: : [sarg] sarg_xmlrpc_sync.php is starting.
Apr 16 01:19:50 php: : IPSEC: One or more IPsec tunnel endpoints has changed its IP. Refreshing.
Apr 16 01:19:50 login: login on ttyv0 as root
Apr 16 01:19:50 sshlockout[58708]: sshlockout/webConfigurator v3.0 starting up
Apr 16 01:19:54 Squid_Alarm[11282]: Squid has exited. Reconfiguring filter.
Apr 16 01:19:54 Squid_Alarm[11604]: Attempting restart…
Apr 16 01:19:57 Squid_Alarm[13545]: Reconfiguring filter…
Apr 16 01:19:57 check_reload_status: Reloading filter
Apr 16 01:20:16 apinger: Error while feeding rrdtool: Broken pipe
Apr 16 01:21:16 apinger: /usr/local/bin/rrdtool respawning too fast, waiting 300s.

marcelloc · Apr 16, 2012, 1:06 AM

Can you check if /var/log/dansguardian folder exists?

Donny · Apr 16, 2012, 8:45 AM

@marcelloc:

Can you check if /var/log/dansguardian folder exists?

Hello Marcelloc, I already check dansguardian. It is still exists. see some screenshot

Donny · Apr 17, 2012, 5:10 PM

@Donny:

@marcelloc:

Can you check if /var/log/dansguardian folder exists?

Hello Marcelloc, I already check dansguardian. It is still exists. see some screenshot

Ok anyway I will try again.

marcelloc · Apr 17, 2012, 5:43 PM

@Donny:

Ok anyway I will try again.

Run sarg cmd on console and check if it returns errors.

Donny · Apr 17, 2012, 9:22 PM

@marcelloc:

@Donny:

Ok anyway I will try again.

Run sarg cmd on console and check if it returns errors.

I just clean install pfsense again. Installed dansguardian and squid3. I checked at system log file I saw many error there.

Apr 17 20:02:46 php: /pkg_mgr_install.php: The command '/usr/local/etc/rc.d/dansguardian stop' returned exit code '126', the output was '/usr/local/etc/rc.d/dansguardian: Permission denied'
Apr 17 20:02:46 php: : New alert found:
Apr 17 20:02:46 check_reload_status: Reloading filter
Apr 17 20:07:39 check_reload_status: Syncing firewall
Apr 17 20:07:39 php: /pkg_mgr_install.php: Beginning package installation for squid3.
Apr 17 20:07:40 check_reload_status: Syncing firewall
Apr 17 20:08:46 check_reload_status: Syncing firewall
Apr 17 20:08:46 php: /pkg_mgr_install.php: Creating squid cache subdirs in /var/squid/cache

Apr 17 20:08:46
php: /pkg_mgr_install.php: The command '/usr/local/sbin/squid -k shutdown' returned exit code '1', the output was '2012/04/17 20:08:46| WARNING: (B) '::/0' is a subnetwork of (A) '::/0' 2012/04/17 20:08:46| WARNING: because of this '::/0' is ignored to keep splay tree searching predictable 2012/04/17 20:08:46| WARNING: You should probably remove '::/0' from the ACL named 'all' 2012/04/17 20:08:46| ERROR: Directive 'upgrade_http0.9' is obsolete. 2012/04/17 20:08:46| cache_cf.cc(381) parseOneConfigFile: squid.conf:2931 unrecognized: 'broken_vary_encoding''

Apr 17 20:08:51
php: /pkg_mgr_install.php: The command '/usr/local/sbin/squid -k kill' returned exit code '1', the output was '2012/04/17 20:08:51| WARNING: (B) '::/0' is a subnetwork of (A) '::/0' 2012/04/17 20:08:51| WARNING: because of this '::/0' is ignored to keep splay tree searching predictable 2012/04/17 20:08:51| WARNING: You should probably remove '::/0' from the ACL named 'all' 2012/04/17 20:08:51| ERROR: Directive 'upgrade_http0.9' is obsolete. 2012/04/17 20:08:51| cache_cf.cc(381) parseOneConfigFile: squid.conf:2931 unrecognized: 'broken_vary_encoding''

Apr 17 20:08:51
php: /pkg_mgr_install.php: The command '/usr/local/sbin/squid -z' returned exit code '1', the output was '2012/04/17 20:08:51| WARNING: (B) '::/0' is a subnetwork of (A) '::/0' 2012/04/17 20:08:51| WARNING: because of this '::/0' is ignored to keep splay tree searching predictable 2012/04/17 20:08:51| WARNING: You should probably remove '::/0' from the ACL named 'all' 2012/04/17 20:08:51| ERROR: Directive 'upgrade_http0.9' is obsolete. 2012/04/17 20:08:51| cache_cf.cc(381) parseOneConfigFile: squid.conf:2931 unrecognized: 'broken_vary_encoding''

After I tried to reboot pfsense and squid has some files missing again.

Apr 17 20:33:28 php: : Not calling package sync code for dependency squidcache of squid3 because some include files are missing.
Apr 17 20:33:28 php: : Not calling package sync code for dependency squidnac of squid3 because some include files are missing.
Apr 17 20:33:28 php: : Not calling package sync code for dependency squidtraffic of squid3 because some include files are missing.
Apr 17 20:33:28 php: : Not calling package sync code for dependency squidupstream of squid3 because some include files are missing.
Apr 17 20:33:28 php: : Not calling package sync code for dependency squidreverse of squid3 because some include files are missing.
Apr 17 20:33:28 php: : Not calling package sync code for dependency squidauth of squid3 because some include files are missing.
Apr 17 20:33:28 php: : Not calling package sync code for dependency squidusers of squid3 because some include files are missing.

I did not install SARG yet because I want to find out some bugs between Dandsguardian and Squid.

I will running SARG cmd on console after I have installed SARG.

marcelloc · Apr 17, 2012, 6:42 PM

config squid options, save
config dansguardian save

check if errors persists.

Donny · Apr 17, 2012, 7:25 PM

another error found

Apr 17 21:13:53 php: : XML error: XML_ERR_NAME_REQUIRED at line 1 in /usr/local/pkg/dansguardian_users_footer.xml
Apr 17 21:13:53 php: : XML error: Invalid document end at line 99 in /usr/local/pkg/dansguardian_users_header.xml
Apr 17 21:13:53 php: : XML error: Invalid document end at line 100 in /usr/local/pkg/dansguardian_ips_header.xml

Both of squid and dansguardian used proxy interface: LAN (no loopback interface selected)

If I use squid as "Transparent proxy" I got this error as below but if I don't use squid as "Transparent proxy" I don't get this error (as you said before try to use squid without "Transparent proxy").

Of course, I have installed NAT rules to redirect from squid port 3128 to DG port 8080.

Apr 17 21:52:39 php: : SQUID is installed but not started. Not installing "nat" rules.
Apr 17 21:52:39 php: : SQUID is installed but not started. Not installing "pfearly" rules.
Apr 17 21:52:39 php: : SQUID is installed but not started. Not installing "filter" rules.

So it mean that using squid in transparent mode still have some bugs and problem.

config squid options, save
config dansguardian, save

No error found in the system log.

Hugovsky · Apr 18, 2012, 9:00 AM

Another problem:

"Fatal error: Allowed memory size of 134217728 bytes exhausted (tried to allocate 62998077 bytes) in /usr/local/www/sarg_frame.php on line 52"

When tryed to load Sites and users.

marcelloc · Apr 18, 2012, 12:48 PM

@Hugovsky:

Another problem:

"Fatal error: Allowed memory size of 134217728 bytes exhausted (tried to allocate 62998077 bytes) in /usr/local/www/sarg_frame.php on line 52"

When tryed to load Sites and users.

It's a pfsense php memory limitation, you can work around this by accessing HTML reports directly http://pfsense_IP/sarg-reports/

ccesario · Apr 18, 2012, 12:46 PM

Hmmm,
Maybe memory_limit change can solve this.

Something like ini_set("memory_limit","256M");

Right Marcello!? :P

marcelloc · Apr 18, 2012, 1:05 PM

@ccesario:

Hmmm,
Maybe memory_limit change can solve this.

Something like ini_set("memory_limit","256M");

Right Marcello!? :P

The memory check prevents ini settings above it's limit(128 for 32 and 256 for 64 bits)

The page is asking 62998077bytes above current limit. This value can be changed on amd64 code but not on i386.

ccesario · Apr 18, 2012, 12:59 PM

@marcelloc:

The memory check prevents ini settings above it's limit(128 for 32 and 256 for 64 bits)

The page is asking 62998077bytes. This value is higher then 512 hard limit for PHP on pfsense.

Opsss…. ignore my post :P

Thanks by info!

Hugovsky · Apr 18, 2012, 1:52 PM

@marcelloc:

@Hugovsky:

Another problem:

"Fatal error: Allowed memory size of 134217728 bytes exhausted (tried to allocate 62998077 bytes) in /usr/local/www/sarg_frame.php on line 52"

When tryed to load Sites and users.

It's a pfsense php memory limitation, you can work around this by accessing HTML reports directly http://pfsense_IP/sarg-reports/

thanks

klamath · Apr 20, 2012, 11:02 AM

Hi Marcelloc
Could you sort all reports for Bytes Reverse???

marcelloc · Apr 20, 2012, 12:45 PM

@klamath:

Hi Marcelloc
Could you sort all reports for Bytes Reverse???

You can sort sarg by clicking on column header.

klamath · Apr 20, 2012, 7:46 PM

I mean… by default! without any click!

@marcelloc:

@klamath:

Hi Marcelloc
Could you sort all reports for Bytes Reverse???

You can sort sarg by clicking on column header.

marcelloc · Apr 20, 2012, 11:21 PM

Did you tried to select bytes on user option and sort fields in reverse options on general tab?

klamath · Apr 21, 2012, 12:26 AM

@marcelloc:

Did you tried to select bytes on user option and sort fields in reverse options on general tab?

Yes… I did it for the first try.
Can I edit the file sarg.conf on the console???

now i've got this error on Redirector report page:

Fatal error: Allowed memory size of 134217728 bytes exhausted (tried to allocate 49259754 bytes) in /usr/local/www/sarg_frame.php on line 52

I'm on AMD64