Feed Update Issue -- Talos



  • Shown below is a recent update.
    UPDATE PROCESS START [ 05/02/19 09:08:59 ]

    ===[ DNSBL Process ]================================================

    Loading DNSBL Statistics... completed
    Loading DNSBL Whitelist... completed

    [ EasyList ] exists.
    [ EasyPrivacy ] exists.
    [ Adaway ] exists.
    [ D_Me_ADs ] exists.
    [ D_Me_Tracking ] exists.
    [ hpHosts_ATS ] exists.
    [ Cameleon ] exists.
    [ SBL_ADs ] exists.
    [ Yoyo ] exists.
    [ Abuse_DOMBL ] exists.
    [ Abuse_URLBL ] exists.
    [ Abuse_Zeus_BD ] exists.
    [ BBC_DC2 ] exists.
    [ SWC ] exists. [ 05/02/19 09:09:00 ]
    [ D_Me_Malv ] exists.
    [ D_Me_Malw ] exists.
    [ ISC_SDH ] exists.
    [ MDS ] exists.
    [ MDS_Immortal ] exists.
    [ MDL ] exists.
    [ MVPS ] exists.
    [ Spam404 ] exists.
    [ SFS_Toxic_BD ] exists.
    Saving DNSBL database... completed

    ===[ GeoIP Process ]============================================

    ===[ IPv4 Process ]=================================================

    [ Abuse_DYRE_v4 ] Downloading update .. 404 Not Found

    [ pfB_PRI1_v4 - Abuse_DYRE_v4 ] Download FAIL
    Firewall and/or IDS (Legacy mode only) are not blocking download.

    The Following List has been REMOVED [ Abuse_DYRE_v4 ]

    [ Abuse_Feodo_C2_v4 ] exists.
    [ Abuse_IPBL_v4 ] exists.
    [ Abuse_SSLBL_v4 ] exists.
    [ Abuse_Zeus_v4 ] exists.
    [ BBC_C2_v4 ] exists.
    [ CINS_army_v4 ] exists.
    [ ET_Block_v4 ] exists.
    [ ET_Comp_v4 ] exists.
    [ ISC_1000_30_v4 ] exists.
    [ ISC_Block_v4 ] exists.
    [ Spamhaus_Drop_v4 ] exists.
    [ Spamhaus_eDrop_v4 ] exists.
    [ Talos_BL_v4 ] Downloading update .. 403 Forbidden

    [ pfB_PRI1_v4 - Talos_BL_v4 ] Download FAIL
    Firewall and/or IDS (Legacy mode only) are not blocking download.

    The Following List has been REMOVED [ Talos_BL_v4 ]

    ===[ Aliastables / Rules ]==========================================

    No changes to Firewall rules, skipping Filter Reload
    No Changes to Aliases, Skipping pfctl Update

    UPDATE PROCESS ENDED [ 05/02/19 09:09:04 ]

    What is the solution to the Talos feed issue?



  • @ghkrauss said in Feed Update Issue -- Talos:

    What is the solution to the Talos feed issue?

    What is the issue ?

    This :

    @ghkrauss said in Feed Update Issue -- Talos:

    [ Talos_BL_v4 ] Downloading update .. 403 Forbidden
    [ pfB_PRI1_v4 - Talos_BL_v4 ] Download FAIL

    Try the 'Talos' URL yourself in a web browser.
    You should obtain some big list with IP's etc.
    Or, the server that hosts the file is in a bad shape. It throws a "404" (the file was not found) in your face if it can't give you what you're asking for.
    This happens. Servers go wako ones in a while. All depends on the admin of that site.
    Maybe the file changed it's name ?

    These lists, used by "pfBlockerNG", have to be maintained, also by you. Nothing is static, they can change.


  • Galactic Empire

    @Gertjan said in Feed Update Issue -- Talos:

    These lists, used by "pfBlockerNG", have to be maintained, also by you. Nothing is static, they can change.

    [ ISC_1000_30_v4 ]		 exists.
    [ ISC_Block_v4 ]		 exists.
    [ Spamhaus_Drop_v4 ]		 exists.
    [ Spamhaus_eDrop_v4 ]		 exists.
    [ Talos_BL_v4 ]			 Downloading update .. 403 Forbidden
    
     [ pfB_PRI1_v4 - Talos_BL_v4 ] Download FAIL
      Firewall and/or IDS (Legacy mode only) are not blocking download.
    

    I wonder if its the redirect it's not liking, I just noticed this.

    Screenshot 2019-05-02 at 16.13.43.png

    Screenshot 2019-05-02 at 16.14.31.png



  • One point for @NogBadTheBad : you just discovered that a browser is probably somewhat smarter as the 'wget' or 'curl' used by 'pfBlockerNG'.





  • @linuxmanr4 Your link doesn't seem to work anymore. I believe the extended information has caused it to expire. Same with me. Things are munged server-side.



  • That's right @provels , it worked for a while and then it did the same thing again.

    I am going to report this problem to pfBlockerNG.



  • @linuxmanr4
    There is an "Expires=3600" in the redirect URL 😒


  • Moderator



  • The user agent curlopt was resulting in a 403 from Cloudflare, seems they didn't like Google Chrome 43 circa 2015.

    I changed my user agent to plain old 'curl' and everything is working again.

    edit /usr/local/pkg/pfblockerng/pfblockerng.inc line 118:
    from:

    $pfb['curl_defaults'] = array(  CURLOPT_USERAGENT       => 'Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 Chrome/43.0.2357.65 Safari/537.36',
    

    to:

    $pfb['curl_defaults'] = array(  CURLOPT_USERAGENT       => 'curl',
    

    edit /usr/local/pkg/pfblocker/pfblockerng_install.inc line 59:
    from:

    curl_setopt($ch, CURLOPT_USERAGENT, 'Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 Chrome/43.0.2357.65 Safari/537.36');
    

    to:

    curl_setopt($ch, CURLOPT_USERAGENT, 'curl');
    

    -neo

    P.s. @BBcan177 if you're going to fix this, while you're in there can you replace 1.1.1.1 with the RFC 5737 compliant 192.0.2.0 so we can use Cloudflare DNS w/o having to edit pfblockerng.inc and pfblockerng.sh please? :) (don't forget about the regex on pfblockerng.sh line 992)

    Edit: BTW, not sure what's going on with caching, but restart php-fam didn't cause an update, I had to delete the /usr/local/pkg/pfblockerng/.pfblockerng.* files and then restart php-fam for the change to activate.



  • This worked for me, Thanks!

    [ Talos_BL_v4 ] Downloading update .. 200 OK. completed ..

    @neoaeon said in Feed Update Issue -- Talos:

    edit /usr/local/pkg/pfblockerng/pfblockerng.inc line 118:
    from:
    $pfb['curl_defaults'] = array( CURLOPT_USERAGENT => 'Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 Chrome/43.0.2357.65 Safari/537.36',

    to:
    $pfb['curl_defaults'] = array( CURLOPT_USERAGENT => 'curl',

    edit /usr/local/pkg/pfblocker/pfblockerng_install.inc line 59:
    from:
    curl_setopt($ch, CURLOPT_USERAGENT, 'Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 Chrome/43.0.2357.65 Safari/537.36');

    to:
    curl_setopt($ch, CURLOPT_USERAGENT, 'curl');



  • Thanks @neoaeon, after modifying the files the problem has been solved. ☺ 👍



  • The feed now download without any modification to User agent.



  • @RonpfS said in Feed Update Issue -- Talos:

    The feed now download without any modification to User agent.

    Thanks for the update!


Log in to reply