Issues updating Feeds
-
@netblues said in Issues updating Feeds:
@panzerscope You definitely DONT need to update any system package by hand. And in general, don't even consider such actions.
Rest assured, feeds do work in general as we speak.
Try downloading the feed manually from pc first, and then try it from cli and see what happens
Regards@netblues said in Issues updating Feeds:
@panzerscope You definitely DONT need to update any system package by hand. And in general, don't even consider such actions.
Rest assured, feeds do work in general as we speak.
Try downloading the feed manually from pc first, and then try it from cli and see what happens
RegardsThose are options, Though unsure how to do it via CLI. However the issue is I would like to solve the actual issue so that the feeds are downloaded via the feeds section/CRON as I have a few feeds and would like to avoid having to do it manually.
-
@panzerscope You need to find you local issue first.
Getting the feeds manually is an initial step.
Noone can help you unless there is sufficient info. -
@netblues said in Issues updating Feeds:
@panzerscope You need to find you local issue first.
Getting the feeds manually is an initial step.
Noone can help you unless there is sufficient info.Other than attempting to download the feeds manually via PC to see if that works, What other info is required ? Logs etc ?
It is strange as it was working fine, no alterations and suddenly stopped working. Looking at the issue it looked to be a handshake issue between Pfsense and the servers where the feeds reside. That is what I gather from the error code 35.
Ill see later if I can download the feeds manually first and report back.
-
So as an update, I can view the feed list online and just save it out via PC, so getting to the list is not an issue. What should I be looking into next to fix the auto downloading issue as pfSense is still throwing the same error as per my first post.
Thanks
P -
@panzerscope, I don't even understand the problem, you can upload a screenshot and try to detail more!
-
@silence It looks like I have solved it.
I had a custom blacklist as per below which obviously blocks access to a resource that is required to fetch the feeds. I will have to try and locate which one causes the problem, unless anyone can see the obvious one ?
3dns.adobe.com
3dns-1.adobe.com
3dns-2.adobe.com
3dns-3.adobe.com
3dns-4.adobe.com
3dns-5.adobe.com
activate.adobe.com
activate.wip1.adobe.com
activate.wip2.adobe.com
activate.wip3.adobe.com
activate.wip4.adobe.com
activate-sea.adobe.com
activate-sjc0.adobe.com
adobe-dns.adobe.com
adobe-dns-1.adobe.com
adobe-dns-2.adobe.com
adobe-dns-3.adobe.com
adobe-dns-4.adobe.com
adobeereg.com
ereg.adobe.com
ereg.wip.adobe.com
ereg.wip1.adobe.com
ereg.wip2.adobe.com
ereg.wip3.adobe.com
ereg.wip4.adobe.com
hl2rcv.adobe.com
practivate.adobe
practivate.adobe.com
practivate.adobe.ipp
practivate.adobe.newoa
practivate.adobe.ntp
wip.adobe.com
wip1.adobe.com
wip2.adobe.com
wip3.adobe.com
wip4.adobe.com
wwis-dubc1-vip100.adobe.com
wwis-dubc1-vip101.adobe.com
wwis-dubc1-vip102.adobe.com
wwis-dubc1-vip103.adobe.com
wwis-dubc1-vip104.adobe.com
wwis-dubc1-vip105.adobe.com
wwis-dubc1-vip106.adobe.com
wwis-dubc1-vip107.adobe.com
wwis-dubc1-vip108.adobe.com
wwis-dubc1-vip109.adobe.com
wwis-dubc1-vip110.adobe.com
wwis-dubc1-vip111.adobe.com
wwis-dubc1-vip112.adobe.com
wwis-dubc1-vip113.adobe.com
wwis-dubc1-vip114.adobe.com
wwis-dubc1-vip115.adobe.com
wwis-dubc1-vip116.adobe.com
wwis-dubc1-vip117.adobe.com
wwis-dubc1-vip118.adobe.com
wwis-dubc1-vip119.adobe.com
wwis-dubc1-vip120.adobe.com
wwis-dubc1-vip121.adobe.com
wwis-dubc1-vip122.adobe.com
wwis-dubc1-vip123.adobe.com
wwis-dubc1-vip124.adobe.com
wwis-dubc1-vip125.adobe.com
wwis-dubc1-vip30.adobe.com
wwis-dubc1-vip31.adobe.com
wwis-dubc1-vip32.adobe.com
wwis-dubc1-vip33.adobe.com
wwis-dubc1-vip34.adobe.com
wwis-dubc1-vip35.adobe.com
wwis-dubc1-vip36.adobe.com
wwis-dubc1-vip37.adobe.com
wwis-dubc1-vip38.adobe.com
wwis-dubc1-vip39.adobe.com
wwis-dubc1-vip40.adobe.com
wwis-dubc1-vip41.adobe.com
wwis-dubc1-vip42.adobe.com
wwis-dubc1-vip43.adobe.com
wwis-dubc1-vip44.adobe.com
wwis-dubc1-vip45.adobe.com
wwis-dubc1-vip46.adobe.com
wwis-dubc1-vip47.adobe.com
wwis-dubc1-vip48.adobe.com
wwis-dubc1-vip49.adobe.com
wwis-dubc1-vip50.adobe.com
wwis-dubc1-vip51.adobe.com
wwis-dubc1-vip52.adobe.com
wwis-dubc1-vip53.adobe.com
wwis-dubc1-vip54.adobe.com
wwis-dubc1-vip55.adobe.com
wwis-dubc1-vip56.adobe.com
wwis-dubc1-vip57.adobe.com
wwis-dubc1-vip58.adobe.com
wwis-dubc1-vip59.adobe.com
wwis-dubc1-vip60.adobe.com
wwis-dubc1-vip60.adobe.com
wwis-dubc1-vip60.adobe.com
wwis-dubc1-vip61.adobe.com
wwis-dubc1-vip62.adobe.com
wwis-dubc1-vip63.adobe.com
wwis-dubc1-vip64.adobe.com
wwis-dubc1-vip65.adobe.com
wwis-dubc1-vip66.adobe.com
wwis-dubc1-vip67.adobe.com
wwis-dubc1-vip68.adobe.com
wwis-dubc1-vip69.adobe.com
wwis-dubc1-vip70.adobe.com
wwis-dubc1-vip71.adobe.com
wwis-dubc1-vip72.adobe.com
wwis-dubc1-vip73.adobe.com
wwis-dubc1-vip74.adobe.com
wwis-dubc1-vip75.adobe.com
wwis-dubc1-vip76.adobe.com
wwis-dubc1-vip77.adobe.com
wwis-dubc1-vip78.adobe.com
wwis-dubc1-vip79.adobe.com
wwis-dubc1-vip80.adobe.com
wwis-dubc1-vip81.adobe.com
wwis-dubc1-vip82.adobe.com
wwis-dubc1-vip83.adobe.com
wwis-dubc1-vip84.adobe.com
wwis-dubc1-vip85.adobe.com
wwis-dubc1-vip86.adobe.com
wwis-dubc1-vip87.adobe.com
wwis-dubc1-vip88.adobe.com
wwis-dubc1-vip89.adobe.com
wwis-dubc1-vip90.adobe.com
wwis-dubc1-vip91.adobe.com
wwis-dubc1-vip92.adobe.com
wwis-dubc1-vip93.adobe.com
wwis-dubc1-vip94.adobe.com
wwis-dubc1-vip95.adobe.com
wwis-dubc1-vip96.adobe.com
wwis-dubc1-vip97.adobe.com
wwis-dubc1-vip98.adobe.com
wwis-dubc1-vip99.adobe.com
crl.versign.net
ood.opsource.net
activate.adobe.com
practivate.adobe.com
ereg.adobe.com
wip3.adobe.com
activate.wip3.adobe.com
3dns-3.adobe.com
3dns-2.adobe.com
adobe-dns.adobe.com
adobe-dns-2.adobe.com
adobe-dns-3.adobe.com
ereg.wip3.adobe.com
activate-sea.adobe.com
wwis-dubc1-vip60.adobe.com
activate-sjc0.adobe.com
hl2rcv.adobe.com
lm.licenses.adobe.com
na2m-pr.licenses.adobe.com
lmlicenses.wip4.adobe.com
lm.licenses.adobe.com
na1r.services.adobe.com
hlrcv.stage.adobe.com
practivate.adobe.com
activate.adobe.com
3dns-1.adobe.com
3dns-2.adobe.com
3dns-3.adobe.com
3dns-4.adobe.com
3dns.adobe.com
activate-sea.adobe.com
activate-sjc0.adobe.com
activate.adobe.com
activate.wip.adobe.com
activate.wip1.adobe.com
activate.wip2.adobe.com
activate.wip3.adobe.com
activate.wip4.adobe.com
adobe-dns-1.adobe.com
adobe-dns-2.adobe.com
adobe-dns-3.adobe.com
adobe-dns-4.adobe.com
adobe-dns.adobe.com
adobe.activate.com
adobeereg.com
ereg.adobe.com
ereg.wip.adobe.com
ereg.wip1.adobe.com
ereg.wip2.adobe.com
ereg.wip3.adobe.com
ereg.wip4.adobe.com
hl2rcv.adobe.com
hlrcv.stage.adobe.com
lm.licenses.adobe.com
lmlicenses.wip4.adobe.com
na1r.services.adobe.com
na2m-pr.licenses.adobe.com
practivate.adobe.com
practivate.adobe.ipp
practivate.adobe.newoa
practivate.adobe.ntp
wip.adobe.com
wip1.adobe.com
wip2.adobe.com
wip3.adobe.com
wip3.adobe.com
wip4.adobe.com
wwis-dubc1-vip60.adobe.com
www.adobeereg.com
www.wip.adobe.com
www.wip1.adobe.com
www.wip2.adobe.com
www.wip3.adobe.com
www.wip4.adobe.com
hl2rcv.adobe.com
adobeereg.com
activate.adobe.com
practivate.adobe.com
ereg.adobe.com
activate.wip3.adobe.com
ereg.wip3.adobe.com
wip3.adobe.com
activate-sea.adobe.com
wwis-dubc1-vip60.adobe.com
activate-sjc0.adobe.com
3dns.adobe.com
3dns-1.adobe.com
3dns-2.adobe.com
3dns-3.adobe.com
3dns-4.adobe.com
adobe-dns.adobe.com
adobe-dns-1.adobe.com
adobe-dns-2.adobe.com
adobe-dns-3.adobe.com
adobe-dns-4.adobe.com
adobe-dns-5.adobe.com
hh-software.com
www.hh-software.com
activate.adobe.de
practivate.adobe.de
ereg.adobe.de
activate.wip3.adobe.de
wip3.adobe.de
3dns-3.adobe.de
3dns-2.adobe.de
adobe-dns.adobe.de
adobe-dns-2.adobe.de
adobe-dns-3.adobe.de
ereg.wip3.adobe.de
activate-sea.adobe.de
wwis-dubc1-vip60.adobe.de
activate-sjc0.adobe.de
wwis-dubc1-vip60.adobe.de
hl2rcv.adobe.de
nero.com
www.nero.com
activate.nero.com
www.activate.nero.com
nero.de
www.nero.de
activate.nero.de
www.activate.nero.de
validation.sls.microsoft.com
ads234.com
ads345.com
www.ads234.com
www.ads345.com
familysimulator.com
stripchat.com -
@panzerscope try again and then send a screenshot of your firewall logs
-
@silence
Looks like my changes didn't fix the issue. When I removed the list as above, I ran a manual update and all worked fine. However upon doing the same this morning, it is failing with the same message.
As soon as I started the update I captured the following from my Firewall Log.
Please note the two highlighted entries, these appeared as soon as I started the feed update.
When the process has finished I did notice these other two entries as shown below
Anything look out of place ? I suppose the first thing would be for me to add the 224.0.0.1 (IGMP) to the permitted list and possibly the other highlighted address's ?
For further reference I have attached my Feed Update process log.
Thanks very much.
-
@panzerscope
Your red squared boxes : what / who is 192.168.1.221 ? Surely not your pfSense.
pfBlockerNG runs on pfSense and doesn't go out to WAN via LAN.
So, the firewall logs are not related to your question.The question : some feed (what is the URL ? ) doesn't reply.
These are "EasyList_Turkish" "D_Me_ADs" "D_Me_Tracking" "D_Me_Malv" "D_Me_Malw" "Abuse_urlhaus" and ....Check on your side : as these curl requests (simple https:// file download requests) use TLS (aka https) : the system time on your side must be ok.
And, you'll be saying : the time on the server side ? Yep, must be ok also. But you can not control neither check that. The download will fail.
edit : what also happens a lot : server admins 'forgot' to update their certificate. The connection will fail : curl will fail. Again : open the used URL in a browser, and you will see more info.Or : the server where the feeds are hosted have 'issues'. That happens all the time, as most feeds are published by people "like me and you" : they host a small VPS, or big dedicated server, and something went wrong. These servers get hammered by all the pfBlockerNG installs, and some of theme have a limited monthly upload quantity ( bytes costs money ). This issue shouldn't really exist, as most list are not updated that often, but pfBlockerNG users could insist in downloading the same list every hour, again, and again. You understand now what might happen ....
When you can't download a feed, copy the used URL in your phone, de activate wifi and try downloading the feed in your phone.
If it still fails : the issue is on the server side : contact the feed owner.
It doesn't fail : maybe the servr owner is also using pfSense + pfBlockerNG, and your WAN IP is on a list he uses ^^ Try changing your WAN IP.Or, most easy, wait it out. If the error keeps popping up, just delete the feed.
@panzerscope said in Issues updating Feeds:
I suppose the first thing would be for me to add the 224.0.0.1 (IGMP) to the permitted list and possibly the other highlighted address's ?
Go have a talk with 192.168.1.221, and see what it's doing ;)
@panzerscope said in Issues updating Feeds:
I did some Googling and found that sometimes this can be due to an outdated Curl package. However before doing anything silly I wanted to see what the best way is to go about fixing this ? Do I need to update the Curl package, if so, how would I go about this ?
If there was a problem with the curl implementation (curl FreeBSD package) it would impact you, every time curl was used, for all downloaded, and it would happen to all of us.
In that case, no need to fire up Google.
Just look at this forum - we talk a lot about pfSense here - and the related Reddit pfSense part. Or go straight to the pfSense Bug Tracker.
You'll be seeing messages related to a curl problem right away.
A solution will be posted even before me and you had detected an issue. -
@gertjan Thanks for the feedback, good information and insights. I will see how I get on :)
