Squid

ipoelnet · Nov 16, 2009, 10:48 AM

3 GB OM,.. Proc DualCore 2.2

grage95 · Nov 16, 2009, 11:27 AM

3 GB OM,.. Proc DualCore 2.2

partisi fisik cache1 = 70 GB
partisi fisik cache12 = 80 GB
tambahkan option rw,noatime di fstab

cache_dir aufs /cache1 32768 64 256 min-size=65535
cache_dir aufs /cache2 40960 64 256 max-size=65535
store_dir_select_algorithm round-robin

atau bisa menggunakan coss utk file2 kecil, cuman rada ribet sedikit membuatnya :D

ipoelnet · Nov 16, 2009, 12:19 PM

Setelah Q tambah

rw,noatime – > partisinya yg di tambah param itu kok g' bisa muncul OM?
Maksudnya untuk option ini

32768 64 256 min-size/max-size=65535 –> huruf tebal apa OM

kan biasanya :

32768 16 256

Mau donk kalau bisa yang pakai Coss….biar tambah wussssssss he he

grage95 · Nov 17, 2009, 1:22 AM

@ipoelnet:

Setelah Q tambah

rw,noatime – > partisinya yg di tambah param itu kok g' bisa muncul OM?
Maksudnya untuk option ini

32768 64 256 min-size/max-size=65535 –> huruf tebal apa OM

kan biasanya :

32768 16 256

Mau donk kalau bisa yang pakai Coss….biar tambah wussssssss he he

edit /etc/fstab

contoh :


proxy# cat /etc/fstab
# Device                Mountpoint      FStype  Options         Dump    Pass#
/dev/ad0s1g             /cache            ufs     rw,noatime      2       2

supaya berefek bisa di reboot, atau di umount /cache baru di mount /cache

cara ngeceknya, ketik
mount
hasilnya salah satunya ada :
/dev/ad0s1g on /cache (ufs, local, noatime, soft-updates)

max-size=ukuran maksimum yang di izinkan di cache (dalam byte)
min-size=ukuran minum yang di izinkan di cache (dalam byte)

cache_dir aufs Directory-Name Mbytes L1 L2 [options]

L2 di rekomendaiskan 256
LI di sesuaikan dengan besaran partisi

dari om Henrik
_simplified formula:

L2 = 256
L1 = cache_dir size / 500, rounded upwards on small numbers..

If L2 is changed or you have a singnificantly different object size
distribution then use the equation above. This simplified formula is
only valid for L2 = 256 and average object size of about 13KB.

Regards
Henrik_

referensi complit filesystem coss
http://wiki.squid-cache.org/Features/CyclicObjectStorageSystem

ipoelnet · Nov 17, 2009, 1:41 AM

Wah…wah...wah...terims penalarannya OM grage95, wah butuh oprek2 pc baru untuk percobaan dulu nich... test dulu ach, ntr kalu berhasil baru langsung update ke server yg ad. ;D

ipoelnet · Nov 20, 2009, 10:40 AM

Kenapa ini ?

2009/11/20 17:12:15| WARNING: All dnsserver processes are busy.
2009/11/20 17:12:15| WARNING: up to 10 pending requests queued
2009/11/20 17:13:09| httpReadReply: Request not yet fully sent "POST http://89.248.172.86/update.php"
2009/11/20 17:13:09| httpReadReply: Request not yet fully sent "POST http://83.170.102.41/update.php"
2009/11/20 17:13:09| httpReadReply: Request not yet fully sent "POST http://83.170.102.41/update.php"
2009/11/20 17:14:48| httpReadReply: Request not yet fully sent "POST http://83.170.102.41/update.php"
2009/11/20 17:14:48| httpReadReply: Request not yet fully sent "POST http://89.248.172.86/update.php"
2009/11/20 17:14:50| httpReadReply: Request not yet fully sent "POST http://89.248.172.90/update.php"
2009/11/20 17:15:13| httpReadReply: Request not yet fully sent "POST http://83.170.102.41/update.php"
2009/11/20 17:15:20| httpReadReply: Request not yet fully sent "POST http://89.248.172.90/update.php"
2009/11/20 17:15:23| httpReadReply: Request not yet fully sent "POST http://89.248.172.86/update.php"
2009/11/20 17:15:40| httpReadReply: Request not yet fully sent "POST http://apps.facebook.com/fbml/fbjs_ajax_proxy.php"
2009/11/20 17:15:55| httpReadReply: Request not yet fully sent "POST http://89.248.172.86/update.php"
2009/11/20 17:16:17| httpReadReply: Request not yet fully sent "POST http://89.248.172.90/update.php"
2009/11/20 17:17:01| httpReadReply: Request not yet fully sent "POST http://apps.facebook.com/fbml/fbjs_ajax_proxy.php"
2009/11/20 17:17:13| parseHttpRequest: Unsupported method 'NICK'
2009/11/20 17:17:13| clientReadRequest: FD 74 (192.168.254.222:2550) Invalid Request
2009/11/20 17:17:43| parseHttpRequest: Unsupported method 'NICK'
2009/11/20 17:17:43| clientReadRequest: FD 51 (192.168.254.222:2560) Invalid Request
2009/11/20 17:17:49| httpReadReply: Request not yet fully sent "POST http://89.248.172.86/update.php"
2009/11/20 17:17:50| httpReadReply: Request not yet fully sent "POST http://89.248.172.90/update.php"
2009/11/20 17:18:16| parseHttpRequest: Unsupported method 'NICK'
2009/11/20 17:18:16| clientReadRequest: FD 57 (192.168.254.222:2568) Invalid Request
2009/11/20 17:18:49| parseHttpRequest: Unsupported method 'NICK'
2009/11/20 17:18:49| clientReadRequest: FD 66 (192.168.254.222:2577) Invalid Request
2009/11/20 17:18:53| clientProcessHit: Vary object loop!
2009/11/20 17:18:54| clientProcessHit: Vary object loop!
2009/11/20 17:18:54| clientProcessHit: Vary object loop!
2009/11/20 17:18:54| clientProcessHit: Vary object loop!
2009/11/20 17:18:55| clientProcessHit: Vary object loop!

Ada yg salah?

grage95 · Nov 20, 2009, 10:45 AM

All dnsserver processes are busy.

di pager/delete saja di squid.inc

#dns_children 32

trs edit sysctl.conf dan loader.conf

/boot/loader.conf

kern.ipc.maxsockbufs="2097152"
kern.ipc.msgmnb="8192"
kern.ipc.msgssz="64"
kern.ipc.msgtql="2048"
kern.ipc.shmseg="16"
kern.ipc.somaxconn="32768"
kern.ipc.nmbclusters="131072"
kern.ipc.maxsockets="65536"

kern.maxfiles="262144"
kern.maxfilesperproc="65536"
net.inet.tcp.tcbhashsize="4096"

/etc/sysctl.conf

net.inet.ip.fastforwarding=1
net.inet.ip.portrange.last=65535
net.inet.ip.portrange.first=1024
net.inet.icmp.icmplim=0
net.inet.icmp.icmplim_output=0
net.inet.tcp.msl=3000
net.inet.tcp.hostcache.expire=1
net.inet.tcp.inflight.enable=0
net.inet.tcp.sendspace=65535
net.inet.tcp.recvspace=65535
kern.ipc.maxsockbufs=2097152
kern.ipc.maxsockets=65536
kern.ipc.somaxconn=32768
kern.ipc.nmbclusters=131072
kern.maxfiles=262144
kern.maxfilesperproc=65536
net.inet.tcp.delayed_ack=0
net.inet.udp.recvspace=65535
net.inet.udp.maxdgram=57344
net.local.stream.recvspace=65535
net.local.stream.sendspace=65535
kern.dirdelay=6
kern.metadelay=5
kern.filedelay=7

reboot server

ipoelnet · Nov 20, 2009, 11:30 AM

Oke Om,.,., siiiip,.,. thanks. ;D

grage95 · Nov 20, 2009, 12:29 PM

httpReadReply: Request not yet fully sent "POST http://89.248.172.90/update.php"
httpReadReply: Request not yet fully sent "POST http://89.248.172.90/update.php"

salah satu client kena virus tuh hihihi

ipoelnet · Nov 20, 2009, 9:17 PM

Itu milik client hotspot OM g' tau milik sapa tuh tangtop, kalau di rule dan blacklist squid udah q block port2 yg Q anggap berbahaya dan nama2 virus yg saya ketahui:

Firewall: Rules

Proto Source Port Destination Port Gateway Schedule Description
TCP LAN net * * 65506 * Drop PhatBot, Agobot, Gaobot
TCP LAN net * * 3128 * Proxy
TCP LAN net * * 8080 * Proxy
TCP LAN net * * 8000 * Proxy
TCP LAN net * * 47624 * –---
TCP LAN net * * 8181 * -----
TCP LAN net * * 27374 * Drop SubSeven
TCP LAN net * * 17300 * Drop Kuang2
TCP LAN net * * 12345 * Drop NetBus
TCP LAN net * * 10080 * Drop MyDoom.B
TCP LAN net * * 9898 * Drop Beagle.A-B
TCP LAN net * * 8866 * Drop Beagle.B
TCP LAN net * * 5554 * Drop Sasser
TCP/UDP LAN net * * 4444 * Worm
TCP LAN net * * 3410 * Drop Backdoor OptixPro
TCP LAN net * * 3127 * Drop MyDoom
TCP LAN net * * 2745 * Drop Beagle.C-K
TCP LAN net * * 2535 * Drop Beagle
TCP LAN net * * 2283 * Drop Dumaru.Y
TCP LAN net * * 2745 * Bagle Virus
TCP LAN net * * 1377 * cichlid
TCP LAN net * * 1373 * hromgrafx
TCP LAN net * * 1368 * screen cast
TCP LAN net * * 1363 - 1364 * ndm requester & ndm Server
TCP LAN net * * 1214 * ________
TCP LAN net * * 1080 * Drop MyDoom
TCP LAN net * * 1024 - 1030 * ________
TCP LAN net * * 593 * ________
TCP/UDP LAN net * * 445 (MS DS) * Drop Blaster Worm
TCP LAN net * * 1433 - 1434 * Worm
TCP/UDP LAN net * * 135 - 139 * Drop Messenger Worm
ICMP LAN net * * * * ICMP
TCP LAN net * * 6667 - 6669 * IRC
TCP LAN net * * 5222 * GTALK
TCP LAN net * * 5050 *
TCP LAN net * * 5000 - 5010 *
TCP LAN net * * 3000 - 3129 * 3000-3129
TCP LAN net * * 3131 - 4000 * 3131-4000

mungkin dari Om ada tambahan?
Gmn cara block virusnya OM? Rule/nat?

$ pfctl -sn
nat-anchor "pftpx/" all
nat-anchor "natearly/" all
nat-anchor "natrules/" all
nat on fxp0 inet from 192.168.254.0/24 port = isakmp to any port = isakmp -> (fxp0) port 500 round-robin
nat on fxp0 inet from 192.168.254.0/24 port = 5060 to any port = 5060 -> (fxp0) port 5060 round-robin
nat on fxp0 inet from 192.168.254.0/24 to any -> (fxp0) port 1024:65535 round-robin
rdr-anchor "pftpx/" all
rdr-anchor "slb" all
no rdr on re0 inet proto tcp from any to 192.168.0.0/16 port = http
no rdr on re0 inet proto tcp from any to 172.16.0.0/12 port = http
no rdr on re0 inet proto tcp from any to 10.0.0.0/8 port = http
rdr on re0 inet proto tcp from any to ! (re0) port = http -> 127.0.0.1 port 80
rdr-anchor "imspector" all
rdr-anchor "miniupnpd" all

mgr info
Select loop called: 849129 times, 15.683 ms avg
loop called, apa berpengaruh OM?

Ini masih nongol…
2009/11/20 22:40:01| WARNING: All dnsserver processes are busy.
2009/11/20 22:40:01| WARNING: up to 10 pending requests queued
2009/11/20 22:42:35| WARNING: All dnsserver processes are busy.
2009/11/20 22:42:35| WARNING: up to 5 pending requests queued
2009/11/20 22:42:35| Consider increasing the number of dnsserver processes to at least 10 in your config file.
2009/11/20 22:42:38| dnsSubmit: queue overload, rejecting img132.imageshack.us
2009/11/20 22:43:41| WARNING: All dnsserver processes are busy.
2009/11/20 22:43:41| WARNING: up to 10 pending requests queued
2009/11/20 22:43:41| Consider increasing the number of dnsserver processes to at least 15 in your config file.

Ups,.,. :-X :-X :-X Setelah Q telity,.,. ada client yang pakai Ultrasurf http://ultrareach.net/,.,. ini program buat bypass proxy sangat mantabb tuh, g' bisa di block ta Om?

Ultrasurf pakai proxy local 127.0.0.1 port 9666, Q coba download n Q pakai,., wah ternyata bobol juga tuh proxy, Q block port 9666 eh ternyata g' mempan, dia pakai port 9666 hanya untuk local saja, terus ?

IP kadang 65.49.14.10, 65.49.2.17 dan banyak lagi…..... Q tanya mbah google, eh ternyata ultrasurf pakai port https(443), ya q Block port 443, Walkhasil email Yahoo dan Gmail dan situs yg pakai https juga g' bisa kebuka,.. Alkhamdulillah keblock kabeh, solusinya?

grage95 · Nov 21, 2009, 5:34 AM

All dnsserver processes are busy

inti masalahnya di dns, bisa karena bottleneck jaringan / karena dns server tidak cepat merespon query dns client
solusi :
1. coba sih squid -v

apakah ada option –disable-internal-dns, kalau ada upgrade squid nya, gunakan internal dns saja lebih ok

2. coba di nslookup abc.com dari client, apakah server bisa cepat merespon,
jika menggunakan dnsmasq, tambahkan cache-size=10000 (10Mb) atau naikkan pelan2, sesuikan dengan ram fisik, jika masih tetap saja bussy berarti segera buat dns-cache selain dnsmasq, dnsmasq hanya utuk net kecil, solusinya buat dedicated dns-server (bukan di box pfsense), recomend gunakan bind atau djbdns

jika menggunakan bind, tambahkan option datasize 12M; max-cache-size 10M; naikkan pelan2, dengan client +/- 2000 nilai 256M sudah sangat responsif

3. tambahkan di squid.inc half_closed_clients off

block ultra yang tunneling ke port 443 banyak cara, bisa lewat firewall/squid

1. lewat squid

tambahkan di squid.inc

acl numeric_IPs url_regex ^[0-9]+.[0-9]+.[0-9]+.[0-9]+
http_access deny CONNECT numeric_IP all

kelemahannya gak bisa buka web yang menggunakan ip, hanya bisa domain,
contohnya skype jk melakukan call menggunakan numerik ip acak, bukan domain, jadi gak bisa connect hehehe ;D

2. lewat firewall, block ip ultrasurf, lihat di attachment, banyak sekali hehehe

untuk virus, kalau client menjalankan aplikasi yang bervirus dan mengandung trojan / hijack browser, solusi satu2nya basmi virusnya di client, firewall secanggih apapun gak bisa ngapa2in, trojan itu destination ip dan portnya acak, ini yang susah

ip_ultrasurf.txt

ipoelnet · Nov 21, 2009, 5:06 PM

Oh ya om untuk Skypi Q nyontoh

http://www1.cs.columbia.edu/~salman/skype/BlockingSkype_corp.pdf
inti :

Your acl definitions

acl numeric_IPs urlpath_regex ^[0-9]+.[0-9]+.[0-9]+.[0-9]+
acl connect method CONNECT

Apply your acls

http_access deny connect numeric_IPs all
dan
http://www.riccardoriva.com/archives/275
isi :
This post will explain a quick and dirt method to block Skype for some user, but avoid to block access to https urls not defined as FQDN.

This post assume that your client have non direct Internet access and must pass trough your Squid Proxy Server to have an external connection.
This Post assume your local network is 192.168.1.0/24
This post assume you want to give SKYPE access to IPs from 192.168.1.100 to 192.168.1.200 and you want to give internet access to all your network.

Obviously you MUST change the IPs based on your REAL network configuration.

In the following configuration, I’m going to create some ACL to define my networks, the skype connection method, skype connections destinations and create a sort of WhiteList that could fill in with some exceptions to avoid https connection problems.

The WhiteList file is /etc/squid/https_url_allowed and you can fill in with a single ip address for line, example :

proxy:~ # cat /etc/squid/https_url_allowed

aaa.bbb.ccc.ddd
eee.fff.ggg.hhh
iii.jjj.kkk.lll
mmm.nnn.ooo.ppp
qqq.rrr.sss.ttt
uuu.vvv.www.xxx

proxy:~ #

All the following lines is in the main SquidProxy Configuration file, usually /etc/squid/squid.conf

# Declare an ACL to catch ALL
acl all src 0.0.0.0/0.0.0.0
# Define an ACL to define my local network
acl mynetworks src 192.168.1.0/24
# Define an ACL to have some IPs that can connect to SKYPE
acl skype_users src 192.168.1.100-192.168.1.200
# Define a CONNECT acl for the CONNECT method
acl CONNECT method CONNECT

# Define an ACL for the URLs composed only of numbers, not FQDN
acl skype_url url_regex ^[0-9]+.[0-9]+.[0-9]+.[0-9]+

# Define an ACL for use URLs composed only of numbers, not FQDN
acl https_url_allowed url_regex -i “/etc/squid/https_url_allowed”

# Allow SKYPE access for the group “skype_users”
http_access allow CONNECT skype_url skype_users

# Allow https access for IP Addresses defined in “/etc/squid/https_url_allowed”
http_access allow CONNECT https_url_allowed

# Deny Access to SKYPE and all other
http_access deny CONNECT skype_url

# Allow Internet access to all “mynetworks”
http_access allow mynetworks

# And finally deny all other access from this proxy
http_access deny all

At this point you can restart squid an check if all works with :

/etc/init.d/squid restart

Hope this help

Bye
Riccardo

Ultrasurf,.,., biarlah berlalu dulu,

111ichael · Nov 24, 2009, 3:25 AM

@grage95:

All dnsserver processes are busy

inti masalahnya di dns, bisa karena bottleneck jaringan / karena dns server tidak cepat merespon query dns client
solusi :
1. coba sih squid -v

apakah ada option –disable-internal-dns, kalau ada upgrade squid nya, gunakan internal dns saja lebih ok

2. coba di nslookup abc.com dari client, apakah server bisa cepat merespon,
jika menggunakan dnsmasq, tambahkan cache-size=10000 (10Mb) atau naikkan pelan2, sesuikan dengan ram fisik, jika masih tetap saja bussy berarti segera buat dns-cache selain dnsmasq, dnsmasq hanya utuk net kecil, solusinya buat dedicated dns-server (bukan di box pfsense), recomend gunakan bind atau djbdns

jika menggunakan bind, tambahkan option datasize 12M; max-cache-size 10M; naikkan pelan2, dengan client +/- 2000 nilai 256M sudah sangat responsif

3. tambahkan di squid.inc half_closed_clients off

block ultra yang tunneling ke port 443 banyak cara, bisa lewat firewall/squid

**1. lewat squid

tambahkan di squid.inc

acl numeric_IPs url_regex ^[0-9]+.[0-9]+.[0-9]+.[0-9]+
http_access deny CONNECT numeric_IP all

kelemahannya gak bisa buka web yang menggunakan ip, hanya bisa domain,
contohnya skype jk melakukan call menggunakan numerik ip acak, bukan domain, jadi gak bisa connect hehehe ;D**

2. lewat firewall, block ip ultrasurf, lihat di attachment, banyak sekali hehehe

untuk virus, kalau client menjalankan aplikasi yang bervirus dan mengandung trojan / hijack browser, solusi satu2nya basmi virusnya di client, firewall secanggih apapun gak bisa ngapa2in, trojan itu destination ip dan portnya acak, ini yang susah

klo lewat transparent squid bgmn yaa?? klo squid dijadikan transparent hanya port 80 yang di direct ke squid…. mohon pencerahannya....

ipoelnet · Nov 24, 2009, 6:53 AM

klo lewat transparent squid bgmn yaa?? klo squid dijadikan transparent hanya port 80 yang di direct ke squid…. mohon pencerahannya....

Untuk metode yang HTTPS atau IP Acak, ya katah Om grage itu solusinya, blm ada port 443 dijadikan transparent, kalau mau obok2 privasi misal YM, ICQ, dan email ya di squid di tambah method CONNECT seperti om grage bilang.

serangku · Nov 24, 2009, 8:55 AM

imspector package …

ipoelnet · Nov 24, 2009, 3:47 PM

Kira-kira ini karena apa yach?

2009/11/24 22:42:37| clientReadRequest: FD 74 (192.168.254.201:1441) Invalid Request
2009/11/24 22:43:07| parseHttpRequest: Unsupported method 'NICK'
2009/11/24 22:43:07| clientReadRequest: FD 117 (192.168.254.201:1442) Invalid Request
2009/11/24 22:43:37| parseHttpRequest: Unsupported method 'NICK'
2009/11/24 22:43:37| clientReadRequest: FD 74 (192.168.254.201:1443) Invalid Request

111ichael · Nov 26, 2009, 1:50 AM

Hasil dari: squidclient mgr:delay

HTTP/1.0 200 OK
Server: Lusca/LUSCA_HEAD
Date: Thu, 26 Nov 2009 01:46:15 GMT
Content-Type: text/plain
Expires: Thu, 26 Nov 2009 01:46:15 GMT
X-Cache: MISS from xx.xx.xx
Via: 1.0 proxy.pfsense:80 (Lusca/LUSCA_HEAD)
Connection: close

Delay pools configured: 2

Pool: 1
Class: 2

Aggregate:
Disabled.

Individual:
Disabled.

Pool: 2
Class: 2

Aggregate:
Disabled.

Individual:
Max: 10000
Rate: 10000
Current: 12:-57987 4:10000

Memory Used: 6792 bytes

Apa yg menyebabkan hingga delay pool trsebut mendapat nilai min(-)….???

ipoelnet · Nov 26, 2009, 5:56 AM

Hasil dari: squidclient mgr:delay

HTTP/1.0 200 OK
Server: Lusca/LUSCA_HEAD
Date: Thu, 26 Nov 2009 01:46:15 GMT
Content-Type: text/plain
Expires: Thu, 26 Nov 2009 01:46:15 GMT
X-Cache: MISS from xx.xx.xx
Via: 1.0 proxy.pfsense:80 (Lusca/LUSCA_HEAD)
Connection: close

Delay pools configured: 2

Pool: 1
Class: 2

Aggregate:
Disabled.

Individual:
Disabled.

Pool: 2
Class: 2

Aggregate:
Disabled.

Individual:
Max: 10000
Rate: 10000
Current: 12:-57987 4:10000

Memory Used: 6792 bytes

Apa yg menyebabkan hingga delay pool trsebut mendapat nilai min(-)….???

itu menngunalan berapa pools?????
kalau memang satu g' dipakai buang aja..

Squid

mungkin dari Om ada tambahan? Gmn cara block virusnya OM? Rule/nat?

Your acl definitions

Apply your acls

Bye Riccardo

Apa yg menyebabkan hingga delay pool trsebut mendapat nilai min(-)….???

mungkin dari Om ada tambahan?
Gmn cara block virusnya OM? Rule/nat?

Bye
Riccardo