Unable to download Ubuntu Updates
-
Hello,
I'm trying to download updates using ubuntu version 22.04.1 using command apt-get update with no success.
I added a ip4 rule to whitelist AS41231 and ubuntu.com to the whitle list but still no success. When I disable pfBlockerNG-devel 3.2.0_4 I'm able to download. Below is the output when trying to update Ubuntu.Any help to isolate the issue would be much appreciated.
Thanks
Hit:1 http://repository.veeam.com/backup/linux/agent/dpkg/debian/public stable InRelease Ign:2 http://download.webmin.com/download/repository sarge InRelease Hit:3 http://download.webmin.com/download/repository sarge Release Ign:5 http://ca.archive.ubuntu.com/ubuntu jammy InRelease Ign:6 http://ca.archive.ubuntu.com/ubuntu jammy-updates InRelease Ign:7 http://ca.archive.ubuntu.com/ubuntu jammy-backports InRelease Ign:8 http://ca.archive.ubuntu.com/ubuntu jammy-security InRelease Ign:5 http://ca.archive.ubuntu.com/ubuntu jammy InRelease Ign:6 http://ca.archive.ubuntu.com/ubuntu jammy-updates InRelease Ign:7 http://ca.archive.ubuntu.com/ubuntu jammy-backports InRelease Ign:8 http://ca.archive.ubuntu.com/ubuntu jammy-security InRelease Ign:5 http://ca.archive.ubuntu.com/ubuntu jammy InRelease Ign:6 http://ca.archive.ubuntu.com/ubuntu jammy-updates InRelease Ign:7 http://ca.archive.ubuntu.com/ubuntu jammy-backports InRelease Ign:8 http://ca.archive.ubuntu.com/ubuntu jammy-security InRelease Err:5 http://ca.archive.ubuntu.com/ubuntu jammy InRelease Cannot initiate the connection to ca.archive.ubuntu.com:80 (2001:67c:1562::15). - connect (101: Network is unreachable) Cannot initiate the connection to ca.archive.ubuntu.com:80 (2001:67c:1562::18). - connect (101: Network is unreachable) Could not connect to ca.archive.ubuntu.com:80 (91.189.91.38), connection timed out Could not connect to ca.archive.ubuntu.com:80 (91.189.91.39), connection timed out Err:6 http://ca.archive.ubuntu.com/ubuntu jammy-updates InRelease Cannot initiate the connection to ca.archive.ubuntu.com:80 (2001:67c:1562::15). - connect (101: Network is unreachable) Cannot initiate the connection to ca.archive.ubuntu.com:80 (2001:67c:1562::18). - connect (101: Network is unreachable) Err:7 http://ca.archive.ubuntu.com/ubuntu jammy-backports InRelease Cannot initiate the connection to ca.archive.ubuntu.com:80 (2001:67c:1562::15). - connect (101: Network is unreachable) Cannot initiate the connection to ca.archive.ubuntu.com:80 (2001:67c:1562::18). - connect (101: Network is unreachable) Err:8 http://ca.archive.ubuntu.com/ubuntu jammy-security InRelease Cannot initiate the connection to ca.archive.ubuntu.com:80 (2001:67c:1562::15). - connect (101: Network is unreachable) Cannot initiate the connection to ca.archive.ubuntu.com:80 (2001:67c:1562::18). - connect (101: Network is unreachable) Reading package lists... Done W: http://download.webmin.com/download/repository/dists/sarge/Release.gpg: Key is stored in legacy trusted.gpg keyring (/etc/apt/trusted.gpg), see the DEPRECATION section in apt-key(8) for details. W: Failed to fetch http://ca.archive.ubuntu.com/ubuntu/dists/jammy/InRelease Cannot initiate the connection to ca.archive.ubuntu.com:80 (2001:67c:1562::15). - connect (101: Network is unreachable) Cannot initiate the connection to ca.archive.ubuntu.com:80 (2001:67c:1562::18). - connect (101: Network is unreachable) Could not connect to ca.archive.ubuntu.com:80 (91.189.91.38), connection timed out Could not connect to ca.archive.ubuntu.com:80 (91.189.91.39), connection timed out W: Failed to fetch http://ca.archive.ubuntu.com/ubuntu/dists/jammy-updates/InRelease Cannot initiate the connection to ca.archive.ubuntu.com:80 (2001:67c:1562::15). - connect (101: Network is unreachable) Cannot initiate the connection to ca.archive.ubuntu.com:80 (2001:67c:1562::18). - connect (101: Network is unreachable) W: Failed to fetch http://ca.archive.ubuntu.com/ubuntu/dists/jammy-backports/InRelease Cannot initiate the connection to ca.archive.ubuntu.com:80 (2001:67c:1562::15). - connect (101: Network is unreachable) Cannot initiate the connection to ca.archive.ubuntu.com:80 (2001:67c:1562::18). - connect (101: Network is unreachable) W: Failed to fetch http://ca.archive.ubuntu.com/ubuntu/dists/jammy-security/InRelease Cannot initiate the connection to ca.archive.ubuntu.com:80 (2001:67c:1562::15). - connect (101: Network is unreachable) Cannot initiate the connection to ca.archive.ubuntu.com:80 (2001:67c:1562::18). - connect (101: Network is unreachable) W: Some index files failed to download. They have been ignored, or old ones used instead. -
@kiekar if it’s a DNSBL block it will be logged in pfBlocker. Or else a firewall block if you have block lists/deny rules.
Did you try allowing ca.archive.ubuntu.com?
-
@SteveITS since he shows an IP there - its not a dns block..
Looks more like a problem with his IPv6 if anything..
-
@kiekar if you have this checked, see arrow in the image below, then adding Ubuntu.com to the whitelist alone won't work. You need to also add [ca.archive.ubuntu.com] as well to the whitelist, force update and force reload...sometimes a reboot is best.
-
I'm thinking you may have wildcard blocking enabled and CNAME validation also.... the canonical name for ca.archive.ubuntu.com found in nslookup is us.archive.ubuntu.com so you will want to have both of these and/or to have .archive.ubuntu.com added to your DNSBL whitelist to avoid being blocked. Whitelisting only ubuntu.com or even .ubuntu.com will only whitelist the specific domain.com or sub.domains.com but not any of the sub.sub.domains.com and I would also recommend adding those to the DNSBL Whitelist on the DNSBL tab, rather than just on an IPv4 whitelist, followed with Update>Force Reload All and once reloaded to clear your Ubuntu pc's DNS cache disconnecting/re-connecting to the network. The IPv4 whitelist will convert those domains/hostnames into their current IPs and then whitelists just those current IP addresses themselves not the domains/hostnames associated with them technically, IP whitelists for avoiding unneeded IP blacklist blocks and a DNSBL whitelist for names in DNS blacklists. Found also dealing with my own Microsh!t updates and office logins for work and many streaming services, some domains will have 5 CNAME's or even more, and if any one of those CNAME's are blocked even with its others working will lead to similar issues or lead to scratching your head with intermittent SSL errors with sites or updates not loading one second then moments/seconds later work just fine temporarily. If you can find these blocks in the Reports > Unified tab or Reports > Alerts > scroll down to DNSBL Python and utilize the + button to whitelist from these two sections, it will search for any associated CNAME's and whitelists them all as well all in one click and no reload is needed, only need to clear DNS cache/reset network connection at device endpoints otherwise.
-
@kiekar said in Unable to download Ubuntu Updates:
Cannot initiate the connection to ca.archive.ubuntu.com:80 (2001:67c:1562::15)
How is this pfblocker if he resolves the IP? And why would a IPv4 whitelist have anything to do with it trying to talk to an IPv6 address.. that is the correct address for that fqdn
;; QUESTION SECTION: ;ca.archive.ubuntu.com. IN AAAA ;; ANSWER SECTION: ca.archive.ubuntu.com. 3599 IN CNAME us.archive.ubuntu.com. us.archive.ubuntu.com. 3600 IN AAAA 2001:67c:1562::15 us.archive.ubuntu.com. 3600 IN AAAA 2001:67c:1562::18 -
Thank you all for your input
I checked my pfBlocker setup and verified that I do not have Wildcard Blocking enabled.
I added ca.archive.ubuntu.com, us.archive.ubuntu.com and canonical.com to the IPv4 white list as well as the DNSBL white list. Did a run all update and still no luck downloading.I watched the download process and noticed that the ip address is switching from (91.189.91.39) to (2001:67c:1562::15) within a second and then holds at this point for 30 seconds before continuing trying to download.
-
@kiekar did you try just with ipv4
I believe
apt-get -o Acquire::ForceIPv4=true updateCan you even ping that IPv6?
16 bytes from 2001:67c:1562::15, icmp_seq=0 hlim=57 time=54.743 ms 16 bytes from 2001:67c:1562::15, icmp_seq=1 hlim=57 time=55.054 ms 16 bytes from 2001:67c:1562::15, icmp_seq=2 hlim=57 time=56.111 ms -
@johnpoz said in Unable to download Ubuntu Updates:
2001:67c:1562::15
@johnpoz said in Unable to download Ubuntu Updates:
apt-get -o Acquire::ForceIPv4=true update
Still can't download
Could not connect to ca.archive.ubuntu.com:80 (91.189.91.38), connection timed out Could not connect to ca.archive.ubuntu.com:80 (91.189.91.39), connection timed outI am not able to ping IPv6. I don't use IPv6
Thanks
-
@kiekar said in Unable to download Ubuntu Updates:
I don't use IPv6
Well your linux box thinks it has Ipv6 or it wouldn't try to talk to it - which from your apt output it was trying to.
-
@kiekar said in Unable to download Ubuntu Updates:
I don't use IPv6
You, me, and every body else do not use IPv6.
But if this was taken from your (you ;) ) Ubuntu, then 'you' use IPv6 :
Err:5 http://ca.archive.ubuntu.com/ubuntu jammy InRelease Cannot initiate the connection to ca.archive.ubuntu.com:80 (2001:67c:1562::15). - connect (101: Network is unreachable) Cannot initiate the connection to ca.archive.ubuntu.com:80 (2001:67c:1562::18). - connect (101: Network is unreachable) Could not connect to ca.archive.ubuntu.com:80 (91.189.91.38), connection timed out Could not connect to ca.archive.ubuntu.com:80 (91.189.91.39), connection timed out Err:6 http://ca.archive.ubuntu.com/ubuntu jammy-updates InRelease Cannot initiate the connection to ca.archive.ubuntu.com:80 (2001:67c:1562::15). - connect (101: Network is unreachable) Cannot initiate the connection to ca.archive.ubuntu.com:80 (2001:67c:1562::18). - connect (101: Network is unreachable) Err:7 http://ca.archive.ubuntu.com/ubuntu jammy-backports InRelease Cannot initiate the connection to ca.archive.ubuntu.com:80 (2001:67c:1562::15). - connect (101: Network is unreachable) Cannot initiate the connection to ca.archive.ubuntu.com:80 (2001:67c:1562::18). - connect (101: Network is unreachable) Err:8 http://ca.archive.ubuntu.com/ubuntu jammy-security InRelease Cannot initiate the connection to ca.archive.ubuntu.com:80 (2001:67c:1562::15). - connect (101: Network is unreachable) Cannot initiate the connection to ca.archive.ubuntu.com:80 (2001:67c:1562::18). - connect (101: Network is unreachable) Reading package lists... Done2001:67c:1562::18is an Pv6 adres.
Ask the admin the Unbuntu system to disable IPv6 if needed.@kiekar said in Unable to download Ubuntu Updates:
Could not connect to ca.archive.ubuntu.com:80 (91.189.91.38)
So IPv4 is also blocked.
Let's start over.
pfSense, out of the box, does not block anything from it's LAN interface.
This means that any destination IPv4 'from 1.1.1.1' to '255.255.255.255' can be used by any LAN based device.
Check that this is the case.
We know it's the case, what counts that that you see for yourself this is true.Now, change settings, install pfSense packages etc.
Check again.
Now, when you see issues like "Could not connect to ca.archive.ubuntu.com:80 (91.189.91.38)" you know where things went wrong.I know, this seems a bit silly, but problem solving is based upon chopping everything in small steps, and analyse them one by one. You'll find the issue - and better : why it happens. The 'what to do' will even be easier.
-
@kiekar Since you are manually editing whitelists, you're going to want to use the "Force - Reload- All" option rather than "Force Update". Reason being is your whitelists are stored locally. Force Reload will load these local file edits to these whitelists to be able to load them. Force Update will "download" only the external files/lists/feeds that need updated from outside of your box, it will not reload the locally edited whitelist files you are altering. Without Force Re-loading after making manual whitelist edits will end up causing your "domain cache" to be offset line for line against the "IP cache" for however many whitelist lines are edited/added, and in the past on both of my boxes would then result in "google.com" for example pointed to a different websites IP address and loads the wrong data and webpage instead. Yes running a "Force Reload All" will take longer for it to run than "Update" but it is needed when manually editing local internal files/whitelists. Update is for externally downloaded feeds/blacklists
-
@smolka_J There may also be a chance these particular Ubuntu domains may not even be needed to be whitelisted once a Force Reload All is ran and completes. I am on Ubuntu Server 22.04 also, and have 11.4 million domains in my DNSBL blacklist along with 840,000 IPs blocked and whatever's leftover caught by 886 lines of REGEX blocking, and the ONLY Ubuntu domain I have in my DNSBL whitelist is "login.ubuntu.com" and updates work just fine. You may want to go to Services>DNS Resolver>Custom Options and add the following lines to Custom Options to not resolve IPv6 back to clients and then reboot your pfSense box for Unbound to load them -
server: do-ip4: yes prefer-ip4: yes do-ip6: no prefer-ip6: no -
Thank you all for your input and pushing me to find a solution.
The issue resided at the GeoIP Top Spammers rule. I changed the Action from Deny Both to Deny Inbound which resolved my issue
-
@kiekar said in Unable to download Ubuntu Updates:
I changed the Action from Deny Both to Deny Inbound which resolved my issue
Inbound : traffic coming IN = traffic that comes in your WAN.
By default : the WAN firewall rule list is empty, so : by default nothing comes in anyway.
The perfect solution would be : why do you even bother using the GEO IP list ?
Drop it, make things easier, and you'll be safe, fine, and less issue can arrive.I changed the Action from Deny Both
Because you were afraid that you could actually send mail ( ? ) or interact with these IP's ?
What you've found out : Unbuntu package servers are listed on the GEO IP you use.
Strange, as these servers never send or push any traffic to you or anybody else. They work 'on request, initiated on your side'.
Conclusion : never fully trust list you obtained from 'somewhere'. Check them. And if checking is to hard, don't use them. -
@Gertjan said in Unable to download Ubuntu Updates:
The perfect solution would be : why do you even bother using the GEO IP list ?
Because I have open ports to the public for my mail server and web server.
Thanks,
-
Right, that is a reason to use some (GEO) IP blocking for incoming connection on WAN.
