Download/Update Feeds Error
-
Hello everybody.
I get cURL error 28 when download/update "Adaway" and "SBL_ADs" feeds.
It returns unknown http status code, So I can't figure where the problem is.
It also says firewall itself doesn't block downloads.
Anyway to solve?
It makes my update process more than 30 minutes.CRON PROCESS START [ 06/09/19 08:00:00 ] [ Adaway ] ( md5 feed ) cURL Error: 28 Operation timed out after 15017 milliseconds with 0 out of 0 bytes received Retry in 5 seconds... . cURL Error: 28 Operation timed out after 300008 milliseconds with 0 bytes received Retry in 5 seconds... . cURL Error: 28 Operation timed out after 300010 milliseconds with 0 bytes received Retry in 5 seconds... .. unknown http status code | 0 Failed to download Feed for md5 comparison! Update skipped [ Cameleon ] [ 06/09/19 08:13:40 ] Remote timestamp: Sun, 18 Mar 2018 09:51:53 GMT Local timestamp: Sun, 18 Mar 2018 09:51:53 GMT Update not required [ D_Me_ADs ] Remote timestamp: Wed, 09 Mar 2016 19:46:05 GMT Local timestamp: Wed, 09 Mar 2016 19:46:05 GMT Update not required [ D_Me_Tracking ] [ 06/09/19 08:13:42 ] Remote timestamp: Fri, 31 Jul 2015 19:01:02 GMT Local timestamp: Fri, 31 Jul 2015 19:01:02 GMT Update not required [ hpHosts_ATS ] [ 06/09/19 08:13:43 ] Remote timestamp: Fri, 07 Jun 2019 12:25:44 GMT Local timestamp: Fri, 07 Jun 2019 12:25:44 GMT Update not required [ SBL_ADs ] [ 06/09/19 08:13:44 ] ( md5 feed ) cURL Error: 28 Operation timed out after 300012 milliseconds with 0 bytes received Retry in 5 seconds... . cURL Error: 28 Operation timed out after 300000 milliseconds with 0 bytes received Retry in 5 seconds... .. 200 OK ( md5 unchanged ) Update not required [ Yoyo ] [ 06/09/19 08:27:05 ] Remote timestamp: Wed, 05 Jun 2019 09:11:26 GMT Local timestamp: Wed, 05 Jun 2019 09:11:26 GMT Update not required [ EasyList ] [ 06/09/19 08:27:06 ] Remote timestamp: Sun, 09 Jun 2019 03:43:15 GMT Local timestamp: Sat, 08 Jun 2019 03:40:52 GMT Update found UPDATE PROCESS START [ 06/09/19 08:27:07 ] ===[ DNSBL Process ]================================================ Loading DNSBL Statistics... completed Loading DNSBL Whitelist... completed [ Adaway ] Downloading update . cURL Error: 28 Operation timed out after 15009 milliseconds with 0 out of 0 bytes received Retry in 5 seconds... . cURL Error: 28 Operation timed out after 15014 milliseconds with 0 out of 0 bytes received Retry in 5 seconds... . cURL Error: 28 Operation timed out after 300053 milliseconds with 0 bytes received Retry in 5 seconds... .. unknown http status code | 0 [ DNSBL_ADs - Adaway ] Download FAIL [ 06/09/19 08:32:52 ] Firewall and/or IDS (Legacy mode only) are not blocking download. Restoring previously downloaded file . ---------------------------------------------------------------------- Orig. Unique # Dups # White # TOP1M Final ---------------------------------------------------------------------- 409 409 0 0 0 409 ---------------------------------------------------------------------- [ Cameleon ] exists. [ 06/09/19 08:32:54 ] [ D_Me_ADs ] exists. [ D_Me_Tracking ] exists. [ hpHosts_ATS ] exists. [ SBL_ADs ] exists. [ Yoyo ] exists. [ EasyList ] Downloading update [ 06/09/19 08:32:56 ] .. 200 OK. ---------------------------------------------------------------------- Orig. Unique # Dups # White # TOP1M Final ---------------------------------------------------------------------- 1062 1062 419 0 0 643 ---------------------------------------------------------------------- Saving DNSBL database... completed ------------------------------------------------------------------------ Assembling DNSBL database... completed [ 06/09/19 08:33:02 ] Reloading Unbound Resolver..... completed [ 06/09/19 08:33:07 ] DNSBL update [ 64317 | PASSED ]... completed [ 06/09/19 08:33:08 ] ------------------------------------------------------------------------ ===[ GeoIP Process ]============================================ ===[ Aliastables / Rules ]========================================== No changes to Firewall rules, skipping Filter Reload No Changes to Aliases, Skipping pfctl Update ===[ FINAL Processing ]===================================== [ Original IP count ] [ 0 ] ===[ DNSBL Domain/IP Counts ] =================================== 64317 total 39019 /var/db/pfblockerng/dnsbl/hpHosts_ATS.txt 20522 /var/db/pfblockerng/dnsbl/Cameleon.txt 2086 /var/db/pfblockerng/dnsbl/D_Me_ADs.txt 805 /var/db/pfblockerng/dnsbl/Yoyo.txt 803 /var/db/pfblockerng/dnsbl/SBL_ADs.txt 643 /var/db/pfblockerng/dnsbl/EasyList.txt 409 /var/db/pfblockerng/dnsbl/Adaway.txt 30 /var/db/pfblockerng/dnsbl/D_Me_Tracking.txt 0 /var/db/pfblockerng/dnsbl/Adaway.fail ====================[ DNSBL Last Updated List Summary ]============== Jul 31 2015 D_Me_Tracking Mar 9 2016 D_Me_ADs Jan 20 2018 Adaway Mar 18 2018 Cameleon Jun 5 13:41 Yoyo Jun 7 16:55 hpHosts_ATS Jun 7 18:45 SBL_ADs Jun 9 08:20 EasyList Alias table IP Counts ----------------------------- pfSense Table Stats ------------------- table-entries hard limit 400000 Table Usage Count 38 UPDATE PROCESS ENDED [ 06/09/19 08:33:09 ] -
@arian_0098 They are working for me w/o issue. Perhaps it was just an Internet hiccup. I suppose you could try removing the 2 associated .txt files (Adaway.txt and SBL_ADs.txt) below /var/db/pfblockerng/dnsbl and doing a full reload.
Can you reach the lists (or ping the sites)?
https://adaway.org/hosts.txt
https://www.squidblacklist.org/downloads/dg-ads.acl -
Today it turned ok, But can't definitely say it's hiccup as It was down for a week.
Wait a few more days and will report again.
Links you've mentioned can be opened without problem.
-
@arian_0098 Maybe just a bad route on the Internet. Had more than several "can't get there from here" situations in my career, usually backbone router related. Someone complains, it gets fixed.
-
@provels said in Download/Update Feeds Error:
@arian_0098 Maybe just a bad route on the Internet. Had more than several "can't get there from here" situations in my career, usually backbone router related. Someone complains, it gets fixed.
It made me crazy and finally put it in hold state to prevent update.
Many bugs in first days as a pfBlockerNG newcomer such as I changed VIP addr, but it wasn't changed ...
I plan to revert to non-devel version.
-
@arian_0098 said in Download/Update Feeds Error:
From pfSense can you ping these domains:
adaway.org www.squidblacklist.org -
Yes.
A strange thing I've seen is Adaway would be ok exactly after 3 timeouts and SBL_ADs after 2 timeouts.
Doesn't matter ... Thanks for your help and this lovely package.
-
@arian_0098 Just throwing this out there, but maybe try changing your update time from 00 on the hour to 15/30/45 past and see if that helps. Maybe too many users (pfSense and others) updating at the top of the hour.
-
@arian_0098 said in Download/Update Feeds Error:
cURL Error: 28
Something is causing the timeout on your box... In the pfSense Resolver increase the "Log Level" to "2", and then review the "resolver.log" for the timestamp of the next updates, and see if you see any clues... also check the pfSense system.log for the same timestamps...
