Alternate definitions for ClamAV



  • I see that there is the option for Optional ClamAV Database Update Servers in the Squid AV page.  I've searched and found only one entry from a few years ago where someone put in the optional signatures from SaneSecurity.  Has anyone tried this?  I've found ClamAV to be somewhat ineffective and others have posted that the additional signatures from SaneSecurity has helped.  I'm not sure how to add them to the optional database field.  Would it just be "http://sanesecurity.com/usage/signatures/"?

    Alternately, has anyone used this field to add optional definitions for the ClamAV and had good success?  I've searched but can't really find anything. Thanks!



  • Update:  I've spoken with Steve over at SaneSecurity and he gave me the links to manually put into the freshclam.conf file.  After running the freshclam update ClamAV had a much higher detection rate.  This morning ClamAV's general definitions found only 12 out of my 46 infections.  The box with the new definitions found 45 of them!  It's gone from a check-box item to an actual protection mechanism.  Iis there a place that pfSense can mirror these?  I guess the current hosts don't offer a great deal of traffic and there may be some concern about sudden spikes of people using it.



  • Can also add them under "Optional ClamAV Database Update Servers"?

    db.12.clamav.net;db.34.clamav.net;db.56.clamav.net
    

    These are free definitions?



  • Yes, they are free definitions although donations wouldn't hurt and are accepted.  I don't think we can just add them under optional servers.  The way I read it is that the optional servers will just download the predefined definitions from another place.  These need to be added manually to the freshclam.conf location to see the additional files.  At least that's how I read it.  I haven't tried it so I may be wrong.



  • I'd be interested in adding the Sane Security definitions to ClamAV. Please give us the 'How To' scoop when you are able..



  • I would be very interested in adding this as well.



  • OP, any more light you can shed on this?



  • Any update on this?

    The current ClamAV definitions are pretty useless, they don't even detect Regin a year after it was disclosed.



  • I thought I would get notices about new postings in the thread.  Just came back to check and noticed there had been some activity.  I'm so sorry and don't want to leave everyone hanging.

    The updates can be changed in Services -> Squid Proxy Server -> Antivirus
    1.  Enable Manual Configuration
    2.  Click Load Advanced to load the default files for editing.
    (I think at this point you need to save)
    3.  Click Show Advanced Options
    4.  Make the changes in the DatabaseCustomURL section of freshclam.conf

    The additions I've used are:
    DatabaseCustomURL http://ftp.swin.edu.au/sanesecurity/junk.ndb
    DatabaseCustomURL http://ftp.swin.edu.au/sanesecurity/phish.ndb
    DatabaseCustomURL http://ftp.swin.edu.au/sanesecurity/rogue.hdb
    DatabaseCustomURL http://ftp.swin.edu.au/sanesecurity/foxhole_filename.cdb
    DatabaseCustomURL http://ftp.swin.edu.au/sanesecurity/foxhole_generic.cdb
    DatabaseCustomURL http://ftp.swin.edu.au/sanesecurity/foxhole_js.cdb
    DatabaseCustomURL http://ftp.swin.edu.au/sanesecurity/badmacro.ndb
    DatabaseCustomURL http://ftp.swin.edu.au/sanesecurity/scam.ndb
    DatabaseCustomURL http://ftp.swin.edu.au/sanesecurity/sanesecurity.ftm
    DatabaseCustomURL http://ftp.swin.edu.au/sanesecurity/sigwhitelist.ign2

    (Click Save)
    (At this point you may or may not need to go back to the General Tab and re-save.)

    As all this appears to be freely and openly distributed I'm confident I can share this.  However, I can't vouch for how long these files will be hosted at this mirror as there was a bandwidth concern expressed to me.  I would love it if pfSense would host it or provide some alternate mirror but at this point it seems we are on our own.  These definitions have worked well for me.  @Pippin did mention that the foxhole definitions (when exclusively check inside archives) blocked an Adobe update so YMMV.

    I'll keep checking this thread so that any further discussion doesn't get overlooked again.



  • Thanks for the how to,

    How do you know for sure if it's working in pfSense?

    My ClamAV update logs looks ok (at first they were failing)

    Message
    bytecode.cvd is up to date (version: 283, sigs: 53, f-level: 63, builder: neo)
    daily.cld is up to date (version: 22205, sigs: 609122, f-level: 63, builder: neo)
    main.cvd is up to date (version: 57, sigs: 4218790, f-level: 60, builder: amishhammer)
    sigwhitelist.ign2 is up to date (version: custom database)
    sanesecurity.ftm is up to date (version: custom database)
    scam.ndb is up to date (version: custom database)
    badmacro.ndb is up to date (version: custom database)
    foxhole_js.cdb is up to date (version: custom database)
    foxhole_generic.cdb is up to date (version: custom database)
    foxhole_filename.cdb is up to date (version: custom database)
    rogue.hdb is up to date (version: custom database)
    phish.ndb is up to date (version: custom database)
    junk.ndb is up to date (version: custom database)
    ClamAV update process started at Mon Sep 12 17:00:00 2016
    –------------------------------------
    bytecode.cvd is up to date (version: 283, sigs: 53, f-level: 63, builder: neo)
    daily.cld is up to date (version: 22205, sigs: 609122, f-level: 63, builder: neo)
    main.cvd is up to date (version: 57, sigs: 4218790, f-level: 60, builder: amishhammer)
    sigwhitelist.ign2 is up to date (version: custom database)
    sanesecurity.ftm is up to date (version: custom database)
    scam.ndb is up to date (version: custom database)
    badmacro.ndb is up to date (version: custom database)
    foxhole_js.cdb is up to date (version: custom database)
    foxhole_generic.cdb is up to date (version: custom database)
    foxhole_filename.cdb is up to date (version: custom database)
    rogue.hdb is up to date (version: custom database)
    phish.ndb is up to date (version: custom database)
    junk.ndb is up to date (version: custom database)
    ClamAV update process started at Mon Sep 12 16:00:00 2016

    bytecode.cvd is up to date (version: 283, sigs: 53, f-level: 63, builder: neo)
    daily.cld is up to date (version: 22205, sigs: 609122, f-level: 63, builder: neo)
    main.cvd is up to date (version: 57, sigs: 4218790, f-level: 60, builder: amishhammer)
    sigwhitelist.ign2 is up to date (version: custom database)
    sanesecurity.ftm is up to date (version: custom database)
    scam.ndb is up to date (version: custom database)
    badmacro.ndb is up to date (version: custom database)
    foxhole_js.cdb is up to date (version: custom database)
    foxhole_generic.cdb is up to date (version: custom database)
    foxhole_filename.cdb is up to date (version: custom database)
    rogue.hdb is up to date (version: custom database)
    phish.ndb is up to date (version: custom database)
    junk.ndb is up to date (version: custom database)
    ClamAV update process started at Mon Sep 12 15:00:00 2016

    But when I look here, I don't see them being a part of the DB? If I go to the sanesecurity website and check out the signature testing link, it gets blocked. I tried using a different connection to email myself the tests, but it didn't seem to do anything.






  • my update log is also ok but in Squid Anti-Virus on the dashboard I see this

    Statistics Unknown (no log exists)

    Have I set up ClamAV correctly?

    Anyone else get this?



  • @johnabbot:

    my update log is also ok but in Squid Anti-Virus on the dashboard I see this

    Statistics Unknown (no log exists)

    Have I set up ClamAV correctly?

    Anyone else get this?

    Maybe you haven't detected a virus yet, so nothing to show?



  • I am having a lot of legitimate updates being blocked by these additional definitions. I do till see a lot of traffic from these same apps/vendors (Apple, Sophos, Adobe, MS) making it through still though.

    Here a a few examples:

    Date-Time	Message	Virus	URL	Host	User
    16.09.2016 21:46:03	VIRUS FOUND	Sanesecurity.Foxhole.Zip_JsNum_wrd.UNOFFICIAL	http://appldnld.apple.com/ios10.0/...[i]/com_apple_MobileAsset_CoreSuggestions/6a3ed2b4a6e1de61c084583a57a0efdff20d2b48.zip	-	-
    
    16.09.2016 17:24:44	VIRUS FOUND	Sanesecurity.Foxhole.Zip_fs211.UNOFFICIAL	http://d1.sophosupd.com/update/5bd003b17a6fd2d057620295800499acx000.dat	-	-
    
    16.09.2016 17:17:06	VIRUS FOUND	Sanesecurity.Foxhole.Zip_doc_js.UNOFFICIAL	http://bg2.v4.a.dl.ws.microsoft.com/dl/content/c/updt/2016/09/19c9cb69-f6ed-49d2-81d5-24c8f4de9cb9_96035d20a923a671f386073b34d004fa5125bd7b.appxbundle?P1=1474071897
    
    5.09.2016 05:12:44	VIRUS FOUND	Sanesecurity.Foxhole.Zip_jsname.UNOFFICIAL	http://a1.mzstatic.com/us/r30/Purple71/v4/6d/30/a2/6d30a250-66eb-9f40-80d9-07fa321fe70f/icon1280x768.lsr
    
    I have removed the Foxhole definitions source from freshclam but that hasn't stopped the blocks above. Here is my additional definitions list:
    
    [code]DatabaseCustomURL http://ftp.swin.edu.au/sanesecurity/junk.ndb
    DatabaseCustomURL http://ftp.swin.edu.au/sanesecurity/phish.ndb
    DatabaseCustomURL http://ftp.swin.edu.au/sanesecurity/rogue.hdb
    DatabaseCustomURL http://ftp.swin.edu.au/sanesecurity/badmacro.ndb
    DatabaseCustomURL http://ftp.swin.edu.au/sanesecurity/scam.ndb
    DatabaseCustomURL http://ftp.swin.edu.au/sanesecurity/sanesecurity.ftm
    DatabaseCustomURL http://ftp.swin.edu.au/sanesecurity/sigwhitelist.ign2[/code]
    
    Any thoughts/ideas?
    
    Edit: Maybe the "icon1280x768.lsr is a legit virus? 
    https://www.virustotal.com/en/file/7f2c0860d6810f8557cf921f1604c01da5f4eae5155bdbf53ed7cafb2d6aeb3a/analysis/
    
    The mzstatic.com domain is an apple domain I believe.[/i]
    


  • The Foxhole definitions are the ones responsible for scanning inside of Zip files I believe.  If it's still being blocked then what is it saying is blocking it?  If it still says Foxhole then there should be some way of clearing those definitions out of ClamAV.  Maybe a Freshclam?  I'm not sure.

    Looking at the virustotal link it appears it is not a virus so maybe it is a false positive?



  • Stewart you can see in my above post, in the first 'code' section it reports that sane security.foxhole.xxx is the responsible database(s). I'm not exactly sure which that is though.

    I removed the 3 foxhole databases and it still blocked. I have just removed the sanesecurity database and I will see if it still blocks.

    In the end I might try and reporting the blocks to SaneSecurity as false positives and see what they say. Although on their site they say and database with 'unofficial' on it is not theirs.



  • Any insight on my previous post? Set it up as instructed. The DB's all update, and I checked the DB's were there. But I don't see them being applied. Also doesn't seem to be blocking anything more in my tests.

    This is what my DB says…

    Squid Version   3.5.19_1
    Antivirus Scanner   ClamAV 0.99.2    C-ICAP 0.4.3 +  SquidClamav 6.10
    Antivirus Bases
    Database Date Version Builder
    daily.cld 2016.09.19 22224 neo
    bytecode.cvd 2016.06.23 283 neo
    main.cvd 2016.03.16 57 amishhammer
    Last Update Mon Sep 19 17:03:48 2016
    Statistics Found 3 virus(es) total.



  • Hi All,

    Just popped in to say if you find any more FP's on files with foxhole_filename.cdb

    I'll need a direct file download url, so I can download and scan this end.

    I've fixed a couple of the FP's already, so thanks for pointing them out.

    Cheers,

    Steve
    Sanesecurity



  • Sanesecurity, good to see you here!

    I have several. How would you like me to get those to you?



  • @Peen:

    Any insight on my previous post? Set it up as instructed. The DB's all update, and I checked the DB's were there. But I don't see them being applied. Also doesn't seem to be blocking anything more in my tests.

    This is what my DB says…

    Squid Version   3.5.19_1
    Antivirus Scanner   ClamAV 0.99.2    C-ICAP 0.4.3 +  SquidClamav 6.10
    Antivirus Bases
    Database Date Version Builder
    daily.cld 2016.09.19 22224 neo
    bytecode.cvd 2016.06.23 283 neo
    main.cvd 2016.03.16 57 amishhammer
    Last Update Mon Sep 19 17:03:48 2016
    Statistics Found 3 virus(es) total.

    Based on the logs you posed above it looks like its working. I don't get the additional databases showing up in the dashboard widget either, but I know they are working due to the blocking happening. Also when I run freshclam I see them updating.

    I'm guessing the Squid Antivirus Widget doesn't report on custom databases, would be nice to have that fixed.



  • Good to know it's working and you have the same thing happening. I did try to open that link you posted with the icon.lsr and it did let me download it. On the WICAR malware test page, I get some blocks so I do know ClamAV is working.



  • @AR15USR:

    Sanesecurity, good to see you here!

    I have several. How would you like me to get those to you?

    You can copy/paste the links and I'll download them with wget to test.
    or perhaps you can pop them into dropbox or some other file storage and send me the link.

    Cheers,

    Steve
    Sanesecurity.com



  • @Peen:

    Any insight on my previous post? Set it up as instructed. The DB's all update, and I checked the DB's were there. But I don't see them being applied. Also doesn't seem to be blocking anything more in my tests.

    This is what my DB says…

    Squid Version   3.5.19_1
    Antivirus Scanner   ClamAV 0.99.2    C-ICAP 0.4.3 +  SquidClamav 6.10
    Antivirus Bases
    Database Date Version Builder
    daily.cld 2016.09.19 22224 neo
    bytecode.cvd 2016.06.23 283 neo
    main.cvd 2016.03.16 57 amishhammer
    Last Update Mon Sep 19 17:03:48 2016
    Statistics Found 3 virus(es) total.

    If you run a freshclam and see them updating then they are in there.  What if you download a file to the box and run a clamscan on it?  If it catches it then it may be a proxy integration thing.



  • @Stewart:

    @Peen:

    Any insight on my previous post? Set it up as instructed. The DB's all update, and I checked the DB's were there. But I don't see them being applied. Also doesn't seem to be blocking anything more in my tests.

    This is what my DB says…

    Squid Version   3.5.19_1
    Antivirus Scanner   ClamAV 0.99.2    C-ICAP 0.4.3 +  SquidClamav 6.10
    Antivirus Bases
    Database Date Version Builder
    daily.cld 2016.09.19 22224 neo
    bytecode.cvd 2016.06.23 283 neo
    main.cvd 2016.03.16 57 amishhammer
    Last Update Mon Sep 19 17:03:48 2016
    Statistics Found 3 virus(es) total.

    If you run a freshclam and see them updating then they are in there.  What if you download a file to the box and run a clamscan on it?  If it catches it then it may be a proxy integration thing.

    2 days with no more of the FP's. Looks like they are fixed, thanks sanesecurity…



  • @AR15USR:

    @Stewart:

    @Peen:

    Any insight on my previous post? Set it up as instructed. The DB's all update, and I checked the DB's were there. But I don't see them being applied. Also doesn't seem to be blocking anything more in my tests.

    This is what my DB says…

    Squid Version   3.5.19_1
    Antivirus Scanner   ClamAV 0.99.2    C-ICAP 0.4.3 +  SquidClamav 6.10
    Antivirus Bases
    Database Date Version Builder
    daily.cld 2016.09.19 22224 neo
    bytecode.cvd 2016.06.23 283 neo
    main.cvd 2016.03.16 57 amishhammer
    Last Update Mon Sep 19 17:03:48 2016
    Statistics Found 3 virus(es) total.

    If you run a freshclam and see them updating then they are in there.  What if you download a file to the box and run a clamscan on it?  If it catches it then it may be a proxy integration thing.

    2 days with no more of the FP's. Looks like they are fixed, thanks sanesecurity…

    Yup, Steve is awesome!  You guys have no idea how responsive and helpful he's been.  Thanks @sanesecurity!



  • Great thread. Thanks so much for this information and to sanesecurity for db's

    Is there a way i could whitelist a specific website in clam .conf files?



  • I also forgot to mention once you do load advanced configuration the settings on the page will be void.

    So if you want to disable clamav scanning streamed audio/video while advance mode is enabled you can add this code to the end of squidclamav.conf

    Do not scan (streamed) videos and audios

    abort ^..(flv|f4f|mp(3|4))(?.)?$
    abort ^..(m3u|pls|wmx|aac|mpeg)(?.)?$
    abortcontent ^video/x-flv$
    abortcontent ^video/mp4$
    abortcontent ^audio/mp4$
    abortcontent ^.audio/mp4.$
    abortcontent ^video/webm$
    abortcontent ^audio/webm$
    abortcontent ^video/MP2T$
    abortcontent ^audio/wmx$
    abortcontent ^audio/mpeg$
    abortcontent ^audio/aac$
    abortcontent ^.application/x-mms-framed.$

    2. In freshclam.conf don't forget to change to your nearest server. Do not touch one below described as "database.clamav.net is round-robin"

    Mine is Australia

    Uncomment the following line and replace XY with your country

    code. See http://www.iana.org/cctld/cctld-whois.htm for the full list.

    You can use db.XY.ipv6.clamav.net for IPv6 connections.

    DatabaseMirror db.au.clamav.net



  • Thanks IggyB..



  • After updating to above settings I am getting this false positive Virus detected warning in diag_edit.php of pfSense page. How can I get rid of this?

    SquidClamav 6.10: Virus detected!

    The requested URL http://192.168.1.1/diag_edit.php contains a virus
    Virus name: Sanesecurity.Malware.26368.JsHeur.UNOFFICIAL

    This file cannot be downloaded.

    Origin: - / -



  • You can try this https://forum.pfsense.org/index.php?topic=120154.msg664657#msg664657 for a temporary work around but I believe that disable's the definition.

    Otherwise you will have to contact sane security.



  • Thanks, that worked.



  • The alternative definitions have been triggering on iOS app updates of late.

    Has anyone else seen this behaviour?

    VIRUS FOUND Sanesecurity.Foxhole.JS_Zip_19.UNOFFICIAL

    http://appldnld.apple.com/ios10.0/091-00410-20170307-333298AC-FD56-11E6-A830-06ECE1925776/com_apple_MobileAsset_CoreSuggestions/5b0b88c6446d899e5bec5a5ac298ed55bbbf1cbb.zip


  • Banned

    The FPs need to be reported to the signatures maintainer. (But please understand that these things are mostly made for email AV filtering.)



  • Yes I've been getting those too. You can report it here:

    http://sanesecurity.com/support/false-positives/



  • Hi

    We are busy adding full support for pfsense to the next version of the script :  https://github.com/extremeshok/clamav-unofficial-sigs

    Please post issues here: https://github.com/extremeshok/clamav-unofficial-sigs/issues/



  • 5.6.1 released with pfsense support : https://github.com/extremeshok/clamav-unofficial-sigs

    Install guide is here : https://github.com/extremeshok/clamav-unofficial-sigs/blob/master/guides/pfsense.md

    Version 5.6.1 (updated 2017-03-18)
    
    eXtremeSHOK.com Maintenance
    Packers/Javascript_exploit_and_obfuscation.yar false positive rating increased to HIGH
    Codeclimate fixes
    Incremented the config to version 73
    Version 5.6 (updated 2017-03-17)
    
    eXtremeSHOK.com Maintenance
    PGP is now optional and no longer a requirement and pgp support is auto-detected
    Full support for MacOS / OS X and added clamav install guide
    Full support for pfSense and added clamav install guide
    Added os configs for Zimbra and Debian 8 with systemd
    Much better error messages with possible solutions given
    Better checking of possible issues
    Update all SANESECURITY signature databases
    Support for clamav-devel (clamav compiled from source)
    Added full proxy support to wget and curl
    Replace allot of "echo | cut | sed" with bash substitutions
    Added fallbacks/substitutions for various commands
    xshok_file_download and xshok_draw_time_remaining functions added to replace redundant code blocks
    Removed SANESECURITY mbl.ndb as this file is not showing up on the rsync mirrors
    Allow exit code 23 for rsync
    Major refactoring : Normalize comments, quotes, functions, conditions
    Protect various arguments and "POSIX-ize" script integrity
    Enhanced testing with travis-ci, including clamav 0.99
    Incremented the config to version 72
    


  • Thank you for this.

    All went well here except.

    WARNING: Failed connection to http://cdn.rfxn.com/downloads - SKIPPED linuxmalwaredetect rfxn.ndb update
    


  • Hi

    Where does Clam AV store the files it believes are viruses (or does it even)?

    I'd like to be able to extract them to check against Virus Total / RE them etc

    Anyone know the answer to this?



  • Hi

    I have an issue with a false positive, I've reported it ages ago but it keeps showing up.

    Anyone know how to remove Sanesecurity.Foxhole?

    The blizzard of false positives is obscuring the real viruses it catches, which is annoying.

    Cheers

    Jon


Log in to reply