[SHARE] Update Lusca Release - r14371 (November 18, 2009)

ipoelnet

Kalau keluar gini pada cache.log

2009/12/15 17:57:10| Beginning Validation Procedure
2009/12/15 17:57:10| 262144 Entries Validated so far.
2009/12/15 17:57:10| 524288 Entries Validated so far.
2009/12/15 17:57:10| Completed Validation Procedure
2009/12/15 17:57:10| Validated 559523 Entries
2009/12/15 17:57:10| store_swap_size = 6777879k
2009/12/15 17:57:10| storeLateRelease: released 0 objects

apa penyebabnya, dan apa berpengaruh Om2?

tomplee

om grage saya pake pfsense 1.2.2 lusca pake yang versi ini http://shakau.googlepages.com/vanila-arp-lusca-1.4_2.tbz
yang lusca vanila versi ini kok gak mau di gabung ma adzap versi ini adzap-20090301.tbz dia pake perl 5.8.9_3 kalau perl-nya yang tak pake yang ini sesuai permintaan adzap luscanya gak mau jalan. solusinya kira bgm ya..

grage95

coba cek dulu perlnya ang sudah terinstall

pkg_info perl*

jika masih menggunakan perl 5.8.8, update perlnya ke versi 5.8.9

coba lusca paket baru lebih responsip hasil patch utak atik gathuk :) , lumayan responsif dan stabil,
cuman cache swap nya harus di delete dulu baru di squid -z,

http://squid-proxy-pkg.googlecode.com/files/pfsense-lusca-head-r14371_3.tbz (menggunakan perl-5.8.9_3.tbz) dan bisa jalan dengan perl-5.8.9_2.tbz

atau update versi pfsensenya,
versi 1.2.3 release lebih maknyuss paralel threadnya utk proxy daripada versi 1.2.2 release dan banyak lagi perbaikannya

tomplee

@grage95:

coba cek dulu perlnya ang sudah terinstall

pkg_info perl*

jika masih menggunakan perl 5.8.8, update perlnya ke versi 5.8.9

coba lusca paket baru lebih responsip hasil patch utak atik gathuk :) , lumayan responsif dan stabil,
cuman cache swap nya harus di delete dulu baru di squid -z,

http://squid-proxy-pkg.googlecode.com/files/pfsense-lusca-head-r14371_3.tbz (menggunakan perl-5.8.9_3.tbz) dan bisa jalan dengan perl-5.8.9_2.tbz

atau update versi pfsensenya,
versi 1.2.3 release lebih maknyuss paralel threadnya utk proxy daripada versi 1.2.2 release dan banyak lagi perbaikannya

Oke tak cobanya update lusca yang baru..

zass

grage95

tukar kepada perl 5.8.9 , Lightsquid graph 500 Internal Server - Error , ???

grage95

upgrade lightsquidnya :

http://125.160.17.22/freebsd/ports/i386/packages-7-stable/www/lightsquidtbz
http://125.160.17.22/freebsd/ports/i386/packages-7.2-release/www/lightsquidtbz

yantisna

@grage95:

![](http://lethe.uwa.edu.au/munin/Servers/styx.uwa.edu.au-squid_hits-month.png[/img<br /><br /><br />[img]http://lethe.uwa.edu.au/munin/Servers/styx.uwa.edu.au-squid_requests-week.png)

download package pfSense Lusca Release - r14371 (November 18, 2009) with patch + snmp + arp

pkg_delete lusca*
or
pkg_delete squid*

and then install
pkg_add -rv http://squid-proxy-pkg.googlecode.com/files/lusca-pfsense-arp-snmp-r14371_2.tbz

enable snmp on squid.inc

acl snmp_host src 127.0.0.1
snmp_port 3401
acl snmppublic snmp_community public
snmp_access allow snmppublic snmp_host
snmp_access deny all

info :
build with
pf-bsd72# squid -v
Squid Cache: Version LUSCA_HEAD
configure options:  '--bindir=/usr/local/sbin' '--sbindir=/usr/local/sbin' '--datadir=/usr/local/etc/squid' '--libexecdir=/usr/local/libexec/squid' '--localstatedir=/usr/local/squid' '--sysconfdir=/usr/local/etc/squid' '--enable-removal-policies=heap' '--disable-linux-netfilter' '--disable-linux-tproxy' '--disable-epoll' '--with-pthreads' '--enable-storeio=aufs coss' '--enable-delay-pools' '--enable-snmp' '--disable-carp' '--disable-wccp' '--disable-ident-lookups' '--enable-arp-acl' '--enable-pf-transparent' '--with-large-files' '--enable-large-cache-files' '--enable-err-languages=English' '--enable-default-err-language=English' '--prefix=/usr/local' '--mandir=/usr/local/man' '--infodir=/usr/local/info/' '--build=i386-portbld-freebsd7.2' 'build_alias=i386-portbld-freebsd7.2' 'CC=cc' 'CFLAGS=-O2 -pipe -funroll-loops -ffast-math  -fno-strict-aliasing' 'LDFLAGS=' 'CPPFLAGS='
patch with : aggressive.patch, 2451x.patch,lusca-vary.patch,loop2.patch (fix bug looping for use store_url_rewrite)

tips install squidstats

pkg_add -rv squidstats

after finisih, lets go config

_mkdir -p /var/db/squidstats/graphs
mkdir -p /var/db/squidstats/rrd
chown www:wheel /var/db/squidstats/graphs
chown proxy:wheel /var/db/squidstats/rrd

ln -s /var/db/squidstats/graphs /usr/local/www/data/
ln -s /usr/local/www/cgi-bin/graph-summary.cgi /usr/local/www/graph-summary.cgi
su -m proxy -c "/usr/local/bin/squidstats.pl createdb"
su -m proxy -c "/usr/local/bin/squidstats.pl gather 2"_
edit /etc/crontab
add this line

*/5 * * * * /usr/local/bin/squidstats.pl gather 2 > /dev/null

restart crontab /etc/rc.d/cron restart

test open with your browser

http://your-ip/graph-summary.cgi

su -m proxy -c "/usr/local/bin/squidstats.pl createdb"

/etc/rc.initial: cannot create /tmp/last_term_seen: Permission denied

maksudnya apa ya om ? padahal saya kan udah login pake root?

grage95

@yantisna:

su -m proxy -c "/usr/local/bin/squidstats.pl createdb"

/etc/rc.initial: cannot create /tmp/last_term_seen: Permission denied

maksudnya apa ya om ? padahal saya kan udah login pake root?

apakah sudah di
chown www:wheel /var/db/squidstats/graphs
chown proxy:wheel /var/db/squidstats/rrd
kalau Permission denied dengan user proxy tidak bisa berarti data sudah tercreate sebagai root, gunakan user proxy utk keamanan, jangan root
coba delet dulu,
cek ls -ll /var/db/squidstats/
apakah masih menggunakan user root ???

disini ada yang berhasil mencontoh instalasi squidstats,
nama usernya kweteng,
step2nya sama

kweteng

Wah jadi sungkan
jadi step2nya

1. delete squid atau lusca
pkg_delete lusca*
atau
pkg_delete squid*

2. update package
http://squid-proxy-pkg.googlecode.com/files/pfsense-lusca-head-r14371_3.tbz

3. lalu masuk ke squid.inc atau tunning.conf, untuk mengaktifkan snmp tambahkan

acl snmp_host src 127.0.0.1
snmp_port 3401
acl snmppublic snmp_community public
snmp_access allow snmppublic snmp_host
snmp_access deny all

4. pastikan dengan squid -v

squid -v

Squid Cache: Version LUSCA_HEAD
configure options: '–bindir=/usr/local/sbin' '--sbindir=/usr/local/sbin' '--datadir=/usr/local/etc/squid' '--libexecdir=/usr/local/libexec/squid' '--localstatedir=/usr/local/squid' '--sysconfdir=/usr/local/etc/squid' '--enable-removal-policies=heap' '--disable-linux-netfilter' '--disable-linux-tproxy' '--disable-wccpv2' '--disable-epoll' '--with-pthreads' '--enable-storeio=aufs' '--enable-delay-pools' '–enable-snmp' '–disable-carp' '--disable-wccp' '--disable-ident-lookups' '--enable-pf-transparent' '--enable-follow-x-forwarded-for' '--with-large-files' '--enable-large-cache-files' '--enable-err-languages=English' '--enable-default-err-language=English' '--prefix=/usr/local' '--mandir=/usr/local/man' '--infodir=/usr/local/info/' '--build=i386-portbld-freebsd7.2' 'build_alias=i386-portbld-freebsd7.2' 'CC=/usr/local/libexec/ccache/world-cc' 'CFLAGS=-O2 -pipe -funroll-loops -ffast-math -fno-strict-aliasing' 'LDFLAGS=' 'CPPFLAGS='

5. lalu delete cache yang ada, dengan tujuan agar lebih stabil
stop service lusca, lewat webgui aja, tunggu bentar, br delete
rm -rf /directory-cache/*
squid -z
lalu di start lagi squidnya

6. sistem sudah siap, sekarang waktunya install squidstats
pkg_add -rv squidstats
tunggu, gak lama dikit ko

7. tunningnya

mkdir -p /var/db/squidstats/graphs
mkdir -p /var/db/squidstats/rrd
chown www:wheel /var/db/squidstats/graphs
chown proxy:wheel /var/db/squidstats/rrd

ln -s /var/db/squidstats/graphs /usr/local/www/
ln -s /usr/local/www/cgi-bin/graph-summary.cgi /usr/local/www/graph-summary.cgi
su -m proxy -c "/usr/local/bin/squidstats.pl createdb"
su -m proxy -c "/usr/local/bin/squidstats.pl gather 2"

8. edit crontab
edit /etc/crontab
tambahkan line berikut

*/5 * * * * /usr/local/bin/squidstats.pl gather 2 > /dev/null

lalu restat
restart crontab /etc/rc.d/cron restart

9. test dengan browser
http://your-ip/graph-summary.cgi

kalo ada yang kurang mohon maaf
semua berhasil atas bantuan dari bos grage, trimakasih yang sebesar2nya kepada bos grage

biatche

# /usr/local/bin/squidstats.pl createdb
Can't locate RRDs.pm in @INC (@INC contains: /usr/local/lib/perl5/5.8.8/BSDPAN /usr/local/lib/perl5/site_perl/5.8.8/mach /usr/local/lib/perl5/site_perl/5.8.8 /usr/local/lib/perl5/site_perl /usr/local/lib/perl5/5.8.8/mach /usr/local/lib/perl5/5.8.8 .) at /usr/local/bin/squidstats.pl line 4.
BEGIN failed--compilation aborted at /usr/local/bin/squidstats.pl line 4.

grage95

update perlnya
default install squidsstats jalan di pfsense veris 1.2.3.xx, jika hasil upgrade dri 1.2 upgrede juga perlnya

biatche

aku pkg_add -rfv perl 5.8.9.. sekarang ok sudah tapi … graph tak ada apa data
apa problem kalau macam ni

# su -m proxy -c "/usr/local/bin/squidstats.pl gather 2"
Got: cacheSysVMsize
Got: cacheSysStorage
Got: cacheNumObjCount
Got: cacheMemUsage
Got: cacheCpuTime
Got: cacheCurrentFileDescrCnt
Got: cacheCurrentFileDescrMax
Got: cacheProtoClientHttpRequests
Got: cacheHttpHits
Got: cacheHttpErrors
Got: cacheHttpInKb
Got: cacheHttpOutKb
Got: cacheClients
Got: cacheIcpPktsSent
Got: cacheIcpPktsRecv
Got: cacheIcpKbSent
Got: cacheIcpKbRecv
Got: cacheServerRequests
Got: cacheServerErrors
Got: cacheServerInKb
Got: cacheServerOutKb
Got: cacheHttpAllSvcTime.5
Got: cacheHttpMissSvcTime.5
Got: cacheHttpNmSvcTime.5
Got: cacheHttpNhSvcTime.5
Got: cacheHttpHitSvcTime.5
Got: cacheIcpQuerySvcTime.5
Got: cacheIcpReplySvcTime.5
Got: cacheDnsSvcTime.5
Got: cacheIpEntries
Got: cacheIpRequests
Got: cacheIpHits
Got: cacheIpMisses
Got: cacheFqdnEntries
Got: cacheFqdnRequests
Got: cacheFqdnHits
Got: cacheFqdnMisses
Got: cacheDnsRequests
Got: cacheDnsReplies
READ: 85816
READ: 311865
READ: 11059
READ: 98443
READ: 168
READ: 38
READ: 63
READ: 28847
READ: 6058
READ: 0
READ: 28496
READ: 529684
READ: 1
READ: 0
READ: 0
READ: 0
READ: 0
READ: 25908
READ: 0
READ: 387379
READ: 27479
READ: 685
READ: 851
READ: 0
READ: 649
READ: 1
READ: 0
READ: 0
READ: 0
READ: 771
READ: 13895
READ: 12469
READ: 1325
READ: 2
READ: 4894
READ: 9
READ: 4885
READ: 6
READ: 6
names: SysVMsize:SysStorage:NumObjCount:MemUsage:CpuTime:CurrentFileDescrCnt:CurrentFileDescrMax:ClientHttpRequests:HttpHits:HttpErrors:HttpInKb:HttpOutKb:Clients:IcpPktsSent:IcpPktsRecv:IcpKbSent:IcpKbRecv:ServerRequests:ServerErrors:ServerInKb:ServerOutKb:HttpAllSvcTime:HttpMissSvcTime:HttpNmSvcTime:HttpNhSvcTime:HttpHitSvcTime:IcpQuerySvcTime:IcpReplySvcTime:DnsSvcTime:IpEntries:IpRequests:IpHits:IpMisses:FqdnEntries:FqdnRequests:FqdnHits:FqdnMisses:DnsRequests:DnsReplies
values: 85816:311865:11059:98443:168:38:63:28847:6058:0:28496:529684:1:0:0:0:0:25908:0:387379:27479:685:851:0:649:1:0:0:0:771:13895:12469:1325:2:4894:9:4885:6:6
OK

UPDATE: tak apa, fix sudah

incoming.1d.png
incoming.1d.png_thumb

grage95

tunggu 15 menitan ?

apakah crond sudah aktif ??

edit crontab
edit /etc/crontab
tambahkan line berikut

*/5 * * * * /usr/local/bin/squidstats.pl gather 2 > /dev/null
dan restart crontab, tunggu 1 jam apakah sudah menampillan graph :)

agar supaya crontab tetap aktif saat di reboot,
edit /cf/conf/config.xml dan tambahkan list rules squidstats pada bagian crontab setelah kata2 <command></command>/usr/local/sbin/squid -k rotate

contoh :

```
<minute>0</minute>
<hour>0</hour>
<mday></mday>
<month></month>
<wday>*</wday>
<who>root</who>
<command></command>/usr/local/sbin/squid -k rotate

<minute>/5</minute>
<hour></hour>
<mday></mday>
<month></month>
<wday>*</wday>
<who>proxy</who>
<command></command>/usr/local/bin/squidstats.pl gather 2 > /dev/null

serangku

om grage95 …
mohon pencerahan ...

spek mesin sbb :
2 gb ram, 1 disk utk pfsense (ata), 1 disk utk coss (sata/1gb), 2 disk utk aufs (sata/2x50gb)
hasil top

top -n 3

last pid: 35439; load averages: 0.00, 0.02, 0.00 up 9+00:17:31 16:15:10
121 processes: 1 running, 117 sleeping, 3 zombie

Mem: 248M Active, 1475M Inact, 193M Wired, 59M Cache, 112M Buf, 22M Free
Swap: 4096M Total, 80K Used, 4096M Free

PID USERNAME THR PRI NICE SIZE RES STATE C TIME WCPU COMMAND
28855 proxy 65 44 0 216M 165M ucond 1 0:00 0.49% squid
958 root 1 45 0 1412K 712K select 1 3:48 0.00% supervise
943 root 1 -8 0 1404K 464K piperd 1 1:14 0.00% readproctitle

inactive memori begitu besar, free memori hanya 22M ...
normalkah ?
dan pernah swap ke disk ...
juga pernah squid page fault erorr 1121 .. ;D, setelah di restart, error kembali ke 0, menambah ram sedang direncanakan ...
hanya nilai "1475M Inact" mengganggu ... dipake kemana saja
apa karena coss kah ...

terima kasih yah om ...

tomplee

@tomplee:

@grage95:

coba cek dulu perlnya ang sudah terinstall

pkg_info perl*

jika masih menggunakan perl 5.8.8, update perlnya ke versi 5.8.9

coba lusca paket baru lebih responsip hasil patch utak atik gathuk :) , lumayan responsif dan stabil,
cuman cache swap nya harus di delete dulu baru di squid -z,

http://squid-proxy-pkg.googlecode.com/files/pfsense-lusca-head-r14371_3.tbz (menggunakan perl-5.8.9_3.tbz) dan bisa jalan dengan perl-5.8.9_2.tbz

atau update versi pfsensenya,
versi 1.2.3 release lebih maknyuss paralel threadnya utk proxy daripada versi 1.2.2 release dan banyak lagi perbaikannya

Oke tak cobanya update lusca yang baru..

om grage tanya lagi..
referensi adzap saya dari sini http://forum.pfsense.org/index.php/topic,12737.0.html
kalau nurut turorial itu redirect_program /usr/local/libexec/adzap; di taroh si custom Custom Options squid, tapi kalo tak taruh di situ squidnya gak mau jalan .. kalau di taruh di tunning.conf bisa gak? redirect_program tu seharusnya naruh urutannya gimana?

grage95

@serangku:

om grage95 …
mohon pencerahan ...

spek mesin sbb :
2 gb ram, 1 disk utk pfsense (ata), 1 disk utk coss (sata/1gb), 2 disk utk aufs (sata/2x50gb)
hasil top

top -n 3

last pid: 35439; load averages: 0.00, 0.02, 0.00 up 9+00:17:31 16:15:10
121 processes: 1 running, 117 sleeping, 3 zombie

Mem: 248M Active, 1475M Inact, 193M Wired, 59M Cache, 112M Buf, 22M Free
Swap: 4096M Total, 80K Used, 4096M Free

PID USERNAME THR PRI NICE SIZE RES STATE C TIME WCPU COMMAND
28855 proxy 65 44 0 216M 165M ucond 1 0:00 0.49% squid
958 root 1 45 0 1412K 712K select 1 3:48 0.00% supervise
943 root 1 -8 0 1404K 464K piperd 1 1:14 0.00% readproctitle

inactive memori begitu besar, free memori hanya 22M ...
normalkah ?
dan pernah swap ke disk ...
juga pernah squid page fault erorr 1121 .. ;D, setelah di restart, error kembali ke 0, menambah ram sedang direncanakan ...
hanya nilai "1475M Inact" mengganggu ... dipake kemana saja
apa karena coss kah ...

terima kasih yah om ...

kalo menggunakan coss jangan sering di restart squidnya atau servernya down,
coss smenggunakan memory banyak pada asaat rebuilding swap,

coba memory_pools nya di on kan dan batasi nilainya misal 512 MB

TAG: memory_pools_limit (bytes)

Used only with memory_pools on:

memory_pools_limit 50 MB

If set to a non-zero value, Squid will keep at most the specified

limit of allocated (but unused) memory in memory pools. All free()

requests that exceed this limit will be handled by your malloc

library. Squid does not pre-allocate any memory, just safe-keeps

objects that otherwise would be free()d. Thus, it is safe to set

memory_pools_limit to a reasonably high value even if your

configuration will use less memory.

If not set (default) or set to zero, Squid will keep all memory it

can. That is, there will be no limit on the total amount of memory

used for safe-keeping.

To disable memory allocation optimization, do not set

memory_pools_limit to 0. Set memory_pools to "off" instead.

An overhead for maintaining memory pools is not taken into account

when the limit is checked. This overhead is close to four bytes per

object kept. However, pools may actually save memory because of

reduced memory thrashing in your malloc library.

#Default:

grage95

@tomplee:

om grage tanya lagi..
referensi adzap saya dari sini http://forum.pfsense.org/index.php/topic,12737.0.html
kalau nurut turorial itu redirect_program /usr/local/libexec/adzap; di taroh si custom Custom Options squid, tapi kalo tak taruh di situ squidnya gak mau jalan .. kalau di taruh di tunning.conf bisa gak? redirect_program tu seharusnya naruh urutannya gimana?

belum pernah coba2 bermain2 dengan azap euy

rekan lainnya mungkin bisa

bfsense

omz omz semua mohon bantuan :d neh
kira kira saya settingnya salah dimana

FATAL: Bungled squid.conf line 52: snmp_access deny allcache deny dynamic
Squid Cache (Version LUSCA_HEAD): Terminated abnormally.

tq….

maklom omz semua newbie

grage95

snmp_access deny allcache deny dynamic

aclny yang salah, harus di beneren tuh

coba paste kesini lengkapnya

bfsense

@grage95:

snmp_access deny allcache deny dynamic

aclny yang salah, harus di beneren tuh

coba paste kesini lengkapnya

Setup some default acls

acl all src 0.0.0.0/0.0.0.0
acl localhost src 127.0.0.1/255.255.255.255
acl safeports port 21 70 80 210 280 443 488 563 591 631 777 901 81 3128 1025-65535
acl sslports port 443 563 81
acl manager proto cache_object
acl purge method PURGE
acl connect method CONNECT
acl dynamic urlpath_regex cgi-bin ?
acl snmp_host src 127.0.0.1
snmp_port 3401
acl snmppublic snmp_community public
snmp_access allow snmppublic snmp_host
snmp_access deny allcache deny dynamic (ga tau tiba tiba muncul ini) padahal dah ikutin step step dari omz grage…
http_access allow manager localhost

http_access deny manager
http_access allow purge localhost
http_access deny purge
http_access deny !safeports
http_access deny CONNECT !sslports

itu omz grage
sebelumnya pakek lusca biasa normal jalannya
eh pas tak test pake yang snmp kok jadi kek gini :(
tolong bantuannya brader semua :d