Squid Custom Rule refresh_pattern help needed

khan

I have installed pfsense 2.0-BETA5 (i386) built on Fri Feb 4 14:41:42 EST 2011 with squid. now i want to cache anti virus update files. searched google, squid wiki, pfsense doc & this forum & became a little confused. According to

http://doc.pfsense.org/index.php/Squid_Package_Tuning

Rule should be like this

refresh_pattern ([^.]+.|)avg.com/.*.(bin) 2160 100% 10080 ignore-no-cache ignore-reload reload-into-ims;

Wher ([^.]+.|) is regular expression for sub-domain, "(bin)" is file type, "2160" is cache age time to mark it fresh, "100%" im factor, "10080" is cache to declare as stale & "ignore-no-cache ignore-reload reload-into-ims" is refresh pattern options.

But squid cache log returns with

squid.conf line 75: refresh_pattern ([^.]+.|)avg.com/..(bin) 2160 100% 10080 ignore-no-cache ignore-reload reload-into-ims
parse_refreshpattern: Invalid regular expression '([^.]+.|)avg.com/..(bin)': empty (sub)expression

Thats mean ([^.]+.|) regular expression is invalid.
If i put this rule like this

refresh_pattern avg.com/.*.(bin) 2160 100% 10080 ignore-no-cache ignore-reload reload-into-ims;

or

refresh_pattern ([^.]+.|) (update|dnl-02|dnl-01|dnl-03|dnl-04|dnl-05|dnl-06|dnl-07|dnl-08|dnl-09|dnl-00|dnl-01|dnl-10|dnl-11|dnl-12|dnl-13|dnl-14|dnl-15|dnl-16|dnl-17|dnl-18|dnl-19|).geo.kaspersky.com/.*.(zip|avc) 2160 100% 10080 ignore-no-cache ignore-no-store ignore-reload reload-into-ims;;

it seems ok with no error.

Now my problem is many anti virus uses several server for update such as kaspersky. They are using using more then 10 server such as

dnl-02.geo.kaspersky.com
update.kaspersky.com
dnl-03.geo.kaspersky.com
dnl-04.geo.kaspersky.com
dnl-05.geo.kaspersky.com
dnl-06.geo.kaspersky.com
dnl-07.geo.kaspersky.com
dnl-08.geo.kaspersky.com
dnl-09.geo.kaspersky.com
dnl-10.geo.kaspersky.com
dnl-11.geo.kaspersky.com
dnl-12.geo.kaspersky.com
dnl-13.geo.kaspersky.com
dnl-14.geo.kaspersky.com
dnl-15.geo.kaspersky.com
dnl-16.geo.kaspersky.com
dnl-17.geo.kaspersky.com
dnl-18.geo.kaspersky.com
dnl-19.geo.kaspersky.com
dnl-00.geo.kaspersky.com

And i want to write only 1 or 2 rule for all of them . Now which 1 is correct & Efficient
1.

refresh_pattern geo.kaspersky.com/.*.(zip|avc) 2160 100% 10080 ignore-no-cache ignore-reload reload-into-ims;

(geo.kaspersky)
2.

refresh_pattern .geo.kaspersky.com/.*.(zip|avc) 2160 100% 10080 ignore-no-cache ignore-reload reload-into-ims;

(.geo.kaspersky)
3.

refresh_pattern ([^.]+.|) (update|dnl-02|dnl-01|dnl-03|dnl-04|dnl-05|dnl-06|dnl-07|dnl-08|dnl-09|dnl-00|dnl-01|dnl-10|dnl-11|dnl-12|dnl-13|dnl-14|dnl-15|dnl-16|dnl-17|dnl-18|dnl-19|).geo.kaspersky.com/.*.(zip|avc) 2160 100% 10080 ignore-no-cache ignore-no-store ignore-reload reload-into-ims;

or
4.

kaspersky.com/.*.(zip|avc) 2160 100% 10080 ignore-no-cache ignore-reload reload-into-ims;

Or Any Other Suggestion ? Please Give me some light.
Thanks in advance.

dvserg

Try this:

.*dnl.*\.geo\.kaspersky\.com/.*

khan

Thanks dvserg
rule is ok returned with no error. need some test. i will let u know my test result as soon as possible. if any 1 can test these rules before me please post result here.

Nachtfalke

Hi,

I am using Kaspersky Updater for downloading Kaspersky Updates for several Kaspersky Versions.

I am using this:
refresh_pattern .downloads..kaspersky-labs.com/..(.) 1440 100% 1440 reload-into-ims;

the URLs for downloading this are mostly like this:
downloads1.kaspersky-labs.com
downloads2.kaspersky-labs.com
downloads3.kaspersky-labs.com
downloads4.kaspersky-labs.com
and so on.

I would say in 70% it works but there is sometimes a TCP_REFRESH_MISS

khan

@Nachtfalke:

Hi,

I am using Kaspersky Updater for downloading Kaspersky Updates for several Kaspersky Versions.

I am using this:
refresh_pattern .downloads..kaspersky-labs.com/..(.) 1440 100% 1440 reload-into-ims;

the URLs for downloading this are mostly like this:
downloads1.kaspersky-labs.com
downloads2.kaspersky-labs.com
downloads3.kaspersky-labs.com
downloads4.kaspersky-labs.com
and so on.

I would say in 70% it works but there is sometimes a TCP_REFRESH_MISS

But my squid log shows these server for download update

dnl-03.geo.kaspersky.com
…...
.......
...
dnl-19.geo.kaspersky.com

Nachtfalke

Hi,
the difference is:

I use Kaspersky Offline Updater - a Kaspersky Tool for downloading Updates for all Kaspersky products. Take a look here:
http://support.kaspersky.com/updater?level=2

The download server you metion a for starting a dowload out of kaspersky itself.

I hope this makes it clear. I missed this explanation in my last post.

khan

Sorry For late reply. These are my refresh pattern rules….

refresh_pattern -i .(gif|png|jpg|jpeg|ico) 43200 90% 129600 ignore-reload ignore-no-cache ignore-private;

refresh_pattern -i .(iso|avi|wav|mp3|mp4|mpeg|swf|flv|x-flv|mpg|wma|ogg|wmv|asx|asf) 43200 90% 432000 override-expire ignore-reload ignore-no-cache ignore-private;

refresh_pattern -i .(deb|rpm|exe|zip|tar|tgz|ram|rar|bin|ppt|doc|tiff|pdf|jar) 43200 90% 129600 override-expire ignore-reload ignore-no-cache ignore-private;

To avoid no-cache response from servers & increase hit rate.
Note: "ignore-private" command may give a warning message as "WARNING: use of 'override-expire' in 'refresh_pattern' violates HTTP". I just Ignored it.

refresh_pattern .dnl..geo.kaspersky.com/..(zip|avc|kdc) 2160 100% 10080 ignore-no-cache reload-into-ims;
refresh_pattern ..avg.com/..(bin) 2160 100% 10080 ignore-no-cache reload-into-ims;
refresh_pattern ..avast.com/..(vpu|vpaa) 2160 100% 10080 ignore-no-cache reload-into-ims;
refresh_pattern ..kaspersky-labs.com/..(cab|zip|exe|msi|msp) 4320 100% 43200 ignore-no-cache reload-into-ims;
refresh_pattern ..kaspersky.com/..(cab|zip|exe|msi|msp|avc) 2160 100% 10080 ignore-no-cache reload-into-ims;
refresh_pattern ..nai.com/.*.(gem|zip|mcs) 2160 100% 10080 ignore-no-cache reload-into-ims;

Anti virus update Cache

Not so much luck Only 5% hit increased. Its only 3 days i am running my pfsense box. HOPE BETTER PERFORMANCE AFTER SOME DAYS. any suggestion is much appreciable.
Thanks in advance.