Download/Update Feeds Error



  • Hello everybody.

    I get cURL error 28 when download/update "Adaway" and "SBL_ADs" feeds.

    It returns unknown http status code, So I can't figure where the problem is.

    It also says firewall itself doesn't block downloads.

    Anyway to solve?
    It makes my update process more than 30 minutes.

    @BBcan177

    CRON  PROCESS  START [ 06/09/19 08:00:00 ]
    [ Adaway ]
    				( md5 feed )		 cURL Error: 28
    Operation timed out after 15017 milliseconds with 0 out of 0 bytes received Retry in 5 seconds...
    . cURL Error: 28
    Operation timed out after 300008 milliseconds with 0 bytes received Retry in 5 seconds...
    . cURL Error: 28
    Operation timed out after 300010 milliseconds with 0 bytes received Retry in 5 seconds...
    .. unknown http status code | 0
    	Failed to download Feed for md5 comparison!	Update skipped
    [ Cameleon ] [ 06/09/19 08:13:40 ]
      Remote timestamp: Sun, 18 Mar 2018 09:51:53 GMT
      Local  timestamp: Sun, 18 Mar 2018 09:51:53 GMT	Update not required
    [ D_Me_ADs ]
      Remote timestamp: Wed, 09 Mar 2016 19:46:05 GMT
      Local  timestamp: Wed, 09 Mar 2016 19:46:05 GMT	Update not required
    [ D_Me_Tracking ] [ 06/09/19 08:13:42 ]
      Remote timestamp: Fri, 31 Jul 2015 19:01:02 GMT
      Local  timestamp: Fri, 31 Jul 2015 19:01:02 GMT	Update not required
    [ hpHosts_ATS ] [ 06/09/19 08:13:43 ]
      Remote timestamp: Fri, 07 Jun 2019 12:25:44 GMT
      Local  timestamp: Fri, 07 Jun 2019 12:25:44 GMT	Update not required
    [ SBL_ADs ] [ 06/09/19 08:13:44 ]
    				( md5 feed )		 cURL Error: 28
    Operation timed out after 300012 milliseconds with 0 bytes received Retry in 5 seconds...
    . cURL Error: 28
    Operation timed out after 300000 milliseconds with 0 bytes received Retry in 5 seconds...
    .. 200 OK
    				( md5 unchanged )	Update not required
    [ Yoyo ] [ 06/09/19 08:27:05 ]
      Remote timestamp: Wed, 05 Jun 2019 09:11:26 GMT
      Local  timestamp: Wed, 05 Jun 2019 09:11:26 GMT	Update not required
    [ EasyList ] [ 06/09/19 08:27:06 ]
      Remote timestamp: Sun, 09 Jun 2019 03:43:15 GMT
      Local  timestamp: Sat, 08 Jun 2019 03:40:52 GMT	Update found
     UPDATE PROCESS START [ 06/09/19 08:27:07 ]
    
    ===[  DNSBL Process  ]================================================
    
     Loading DNSBL Statistics... completed
     Loading DNSBL Whitelist... completed
    
    [ Adaway ]			 Downloading update . cURL Error: 28
    Operation timed out after 15009 milliseconds with 0 out of 0 bytes received Retry in 5 seconds...
    . cURL Error: 28
    Operation timed out after 15014 milliseconds with 0 out of 0 bytes received Retry in 5 seconds...
    . cURL Error: 28
    Operation timed out after 300053 milliseconds with 0 bytes received Retry in 5 seconds...
    .. unknown http status code | 0
    
     [ DNSBL_ADs - Adaway ] Download FAIL [ 06/09/19 08:32:52 ]
      Firewall and/or IDS (Legacy mode only) are not blocking download.
    
      Restoring previously downloaded file
     .
      ----------------------------------------------------------------------
      Orig.    Unique     # Dups     # White    # TOP1M    Final                
      ----------------------------------------------------------------------
      409      409        0          0          0          409                  
      ----------------------------------------------------------------------
    
    [ Cameleon ]			 exists. [ 06/09/19 08:32:54 ]
    [ D_Me_ADs ]			 exists.
    [ D_Me_Tracking ]		 exists.
    [ hpHosts_ATS ]			 exists.
    [ SBL_ADs ]			 exists.
    [ Yoyo ]			 exists.
    [ EasyList ]			 Downloading update [ 06/09/19 08:32:56 ] .. 200 OK.
      ----------------------------------------------------------------------
      Orig.    Unique     # Dups     # White    # TOP1M    Final                
      ----------------------------------------------------------------------
      1062     1062       419        0          0          643                  
      ----------------------------------------------------------------------
    
    Saving DNSBL database... completed
    
    ------------------------------------------------------------------------
    Assembling DNSBL database... completed [ 06/09/19 08:33:02 ]
    Reloading Unbound Resolver..... completed [ 06/09/19 08:33:07 ]
    DNSBL update [ 64317 | PASSED  ]... completed [ 06/09/19 08:33:08 ]
    ------------------------------------------------------------------------
    
    ===[  GeoIP Process  ]============================================
    
    
    ===[  Aliastables / Rules  ]==========================================
    
    No changes to Firewall rules, skipping Filter Reload
    No Changes to Aliases, Skipping pfctl Update
    
    ===[ FINAL Processing ]=====================================
    
       [ Original IP count   ]  [ 0 ]
    
    ===[ DNSBL Domain/IP Counts ] ===================================
    
       64317 total
       39019 /var/db/pfblockerng/dnsbl/hpHosts_ATS.txt
       20522 /var/db/pfblockerng/dnsbl/Cameleon.txt
        2086 /var/db/pfblockerng/dnsbl/D_Me_ADs.txt
         805 /var/db/pfblockerng/dnsbl/Yoyo.txt
         803 /var/db/pfblockerng/dnsbl/SBL_ADs.txt
         643 /var/db/pfblockerng/dnsbl/EasyList.txt
         409 /var/db/pfblockerng/dnsbl/Adaway.txt
          30 /var/db/pfblockerng/dnsbl/D_Me_Tracking.txt
           0 /var/db/pfblockerng/dnsbl/Adaway.fail
    
    ====================[ DNSBL Last Updated List Summary ]==============
    
    Jul 31	2015	D_Me_Tracking
    Mar 9	2016	D_Me_ADs
    Jan 20	2018	Adaway
    Mar 18	2018	Cameleon
    Jun 5	13:41	Yoyo
    Jun 7	16:55	hpHosts_ATS
    Jun 7	18:45	SBL_ADs
    Jun 9	08:20	EasyList
    
    Alias table IP Counts
    -----------------------------
    
    pfSense Table Stats
    -------------------
    table-entries hard limit   400000
    Table Usage Count         38
    
     UPDATE PROCESS ENDED [ 06/09/19 08:33:09 ]
    


  • @arian_0098 They are working for me w/o issue. Perhaps it was just an Internet hiccup. I suppose you could try removing the 2 associated .txt files (Adaway.txt and SBL_ADs.txt) below /var/db/pfblockerng/dnsbl and doing a full reload.

    Can you reach the lists (or ping the sites)?
    https://adaway.org/hosts.txt
    https://www.squidblacklist.org/downloads/dg-ads.acl



  • @provels

    Today it turned ok, But can't definitely say it's hiccup as It was down for a week.

    Wait a few more days and will report again.

    Links you've mentioned can be opened without problem.



  • @arian_0098 Maybe just a bad route on the Internet. Had more than several "can't get there from here" situations in my career, usually backbone router related. Someone complains, it gets fixed.



  • @provels said in Download/Update Feeds Error:

    @arian_0098 Maybe just a bad route on the Internet. Had more than several "can't get there from here" situations in my career, usually backbone router related. Someone complains, it gets fixed.

    It made me crazy and finally put it in hold state to prevent update.

    Many bugs in first days as a pfBlockerNG newcomer such as I changed VIP addr, but it wasn't changed ...

    I plan to revert to non-devel version.


  • Moderator

    @arian_0098 said in Download/Update Feeds Error:

    From pfSense can you ping these domains:

    adaway.org
    www.squidblacklist.org
    


  • @BBcan177

    Yes.
    pfsense.myhomenet.zone - Diagnostics  Ping.png

    A strange thing I've seen is Adaway would be ok exactly after 3 timeouts and SBL_ADs after 2 timeouts.

    Doesn't matter ... Thanks for your help and this lovely package. 👏



  • @arian_0098 Just throwing this out there, but maybe try changing your update time from 00 on the hour to 15/30/45 past and see if that helps. Maybe too many users (pfSense and others) updating at the top of the hour.


  • Moderator

    @arian_0098 said in Download/Update Feeds Error:

    cURL Error: 28

    Something is causing the timeout on your box... In the pfSense Resolver increase the "Log Level" to "2", and then review the "resolver.log" for the timestamp of the next updates, and see if you see any clues... also check the pfSense system.log for the same timestamps...


Log in to reply