Unable to download Ubuntu Updates

NollipfSense

@kiekar if you have this checked, see arrow in the image below, then adding Ubuntu.com to the whitelist alone won't work. You need to also add [ca.archive.ubuntu.com] as well to the whitelist, force update and force reload...sometimes a reboot is best.

Screenshot 2023-05-24 at 10.20.29 PM.png

smolka_J

I'm thinking you may have wildcard blocking enabled and CNAME validation also.... the canonical name for ca.archive.ubuntu.com found in nslookup is us.archive.ubuntu.com so you will want to have both of these and/or to have .archive.ubuntu.com added to your DNSBL whitelist to avoid being blocked. Whitelisting only ubuntu.com or even .ubuntu.com will only whitelist the specific domain.com or sub.domains.com but not any of the sub.sub.domains.com and I would also recommend adding those to the DNSBL Whitelist on the DNSBL tab, rather than just on an IPv4 whitelist, followed with Update>Force Reload All and once reloaded to clear your Ubuntu pc's DNS cache disconnecting/re-connecting to the network. The IPv4 whitelist will convert those domains/hostnames into their current IPs and then whitelists just those current IP addresses themselves not the domains/hostnames associated with them technically, IP whitelists for avoiding unneeded IP blacklist blocks and a DNSBL whitelist for names in DNS blacklists. Found also dealing with my own Microsh!t updates and office logins for work and many streaming services, some domains will have 5 CNAME's or even more, and if any one of those CNAME's are blocked even with its others working will lead to similar issues or lead to scratching your head with intermittent SSL errors with sites or updates not loading one second then moments/seconds later work just fine temporarily. If you can find these blocks in the Reports > Unified tab or Reports > Alerts > scroll down to DNSBL Python and utilize the + button to whitelist from these two sections, it will search for any associated CNAME's and whitelists them all as well all in one click and no reload is needed, only need to clear DNS cache/reset network connection at device endpoints otherwise.

johnpoz

@kiekar said in Unable to download Ubuntu Updates:

Cannot initiate the connection to ca.archive.ubuntu.com:80 (2001:67c:1562::15)

How is this pfblocker if he resolves the IP? And why would a IPv4 whitelist have anything to do with it trying to talk to an IPv6 address.. that is the correct address for that fqdn

;; QUESTION SECTION:
;ca.archive.ubuntu.com.         IN      AAAA

;; ANSWER SECTION:
ca.archive.ubuntu.com.  3599    IN      CNAME   us.archive.ubuntu.com.
us.archive.ubuntu.com.  3600    IN      AAAA    2001:67c:1562::15
us.archive.ubuntu.com.  3600    IN      AAAA    2001:67c:1562::18

kiekar

Thank you all for your input

I checked my pfBlocker setup and verified that I do not have Wildcard Blocking enabled.
I added ca.archive.ubuntu.com, us.archive.ubuntu.com and canonical.com to the IPv4 white list as well as the DNSBL white list. Did a run all update and still no luck downloading.

I watched the download process and noticed that the ip address is switching from (91.189.91.39) to (2001:67c:1562::15) within a second and then holds at this point for 30 seconds before continuing trying to download.

johnpoz

@kiekar did you try just with ipv4

I believe

apt-get -o Acquire::ForceIPv4=true update

Can you even ping that IPv6?

16 bytes from 2001:67c:1562::15, icmp_seq=0 hlim=57 time=54.743 ms
16 bytes from 2001:67c:1562::15, icmp_seq=1 hlim=57 time=55.054 ms
16 bytes from 2001:67c:1562::15, icmp_seq=2 hlim=57 time=56.111 ms

kiekar

@johnpoz said in Unable to download Ubuntu Updates:

2001:67c:1562::15

@johnpoz said in Unable to download Ubuntu Updates:

apt-get -o Acquire::ForceIPv4=true update

Still can't download

 Could not connect to ca.archive.ubuntu.com:80 (91.189.91.38), connection timed out Could not connect to ca.archive.ubuntu.com:80 (91.189.91.39), connection timed out

I am not able to ping IPv6. I don't use IPv6

Thanks

johnpoz

@kiekar said in Unable to download Ubuntu Updates:

I don't use IPv6

Well your linux box thinks it has Ipv6 or it wouldn't try to talk to it - which from your apt output it was trying to.

Gertjan

@kiekar said in Unable to download Ubuntu Updates:

I don't use IPv6

You, me, and every body else do not use IPv6.

But if this was taken from your (you ;) ) Ubuntu, then 'you' use IPv6 :

Err:5 http://ca.archive.ubuntu.com/ubuntu jammy InRelease
  Cannot initiate the connection to ca.archive.ubuntu.com:80 (2001:67c:1562::15). - connect (101: Network is unreachable) Cannot initiate the connection to ca.archive.ubuntu.com:80 (2001:67c:1562::18). - connect (101: Network is unreachable) Could not connect to ca.archive.ubuntu.com:80 (91.189.91.38), connection timed out Could not connect to ca.archive.ubuntu.com:80 (91.189.91.39), connection timed out
Err:6 http://ca.archive.ubuntu.com/ubuntu jammy-updates InRelease
  Cannot initiate the connection to ca.archive.ubuntu.com:80 (2001:67c:1562::15). - connect (101: Network is unreachable) Cannot initiate the connection to ca.archive.ubuntu.com:80 (2001:67c:1562::18). - connect (101: Network is unreachable)
Err:7 http://ca.archive.ubuntu.com/ubuntu jammy-backports InRelease
  Cannot initiate the connection to ca.archive.ubuntu.com:80 (2001:67c:1562::15). - connect (101: Network is unreachable) Cannot initiate the connection to ca.archive.ubuntu.com:80 (2001:67c:1562::18). - connect (101: Network is unreachable)
Err:8 http://ca.archive.ubuntu.com/ubuntu jammy-security InRelease
  Cannot initiate the connection to ca.archive.ubuntu.com:80 (2001:67c:1562::15). - connect (101: Network is unreachable) Cannot initiate the connection to ca.archive.ubuntu.com:80 (2001:67c:1562::18). - connect (101: Network is unreachable)
Reading package lists... Done

2001:67c:1562::18

is an Pv6 adres.
Ask the admin the Unbuntu system to disable IPv6 if needed.

@kiekar said in Unable to download Ubuntu Updates:

Could not connect to ca.archive.ubuntu.com:80 (91.189.91.38)

So IPv4 is also blocked.

Let's start over.
pfSense, out of the box, does not block anything from it's LAN interface.
This means that any destination IPv4 'from 1.1.1.1' to '255.255.255.255' can be used by any LAN based device.
Check that this is the case.
We know it's the case, what counts that that you see for yourself this is true.

Now, change settings, install pfSense packages etc.

Check again.
Now, when you see issues like "Could not connect to ca.archive.ubuntu.com:80 (91.189.91.38)" you know where things went wrong.

I know, this seems a bit silly, but problem solving is based upon chopping everything in small steps, and analyse them one by one. You'll find the issue - and better : why it happens. The 'what to do' will even be easier.

smolka_J

@kiekar Since you are manually editing whitelists, you're going to want to use the "Force - Reload- All" option rather than "Force Update". Reason being is your whitelists are stored locally. Force Reload will load these local file edits to these whitelists to be able to load them. Force Update will "download" only the external files/lists/feeds that need updated from outside of your box, it will not reload the locally edited whitelist files you are altering. Without Force Re-loading after making manual whitelist edits will end up causing your "domain cache" to be offset line for line against the "IP cache" for however many whitelist lines are edited/added, and in the past on both of my boxes would then result in "google.com" for example pointed to a different websites IP address and loads the wrong data and webpage instead. Yes running a "Force Reload All" will take longer for it to run than "Update" but it is needed when manually editing local internal files/whitelists. Update is for externally downloaded feeds/blacklists

smolka_J

@smolka_J There may also be a chance these particular Ubuntu domains may not even be needed to be whitelisted once a Force Reload All is ran and completes. I am on Ubuntu Server 22.04 also, and have 11.4 million domains in my DNSBL blacklist along with 840,000 IPs blocked and whatever's leftover caught by 886 lines of REGEX blocking, and the ONLY Ubuntu domain I have in my DNSBL whitelist is "login.ubuntu.com" and updates work just fine. You may want to go to Services>DNS Resolver>Custom Options and add the following lines to Custom Options to not resolve IPv6 back to clients and then reboot your pfSense box for Unbound to load them -

server:
do-ip4: yes
prefer-ip4: yes
do-ip6: no
prefer-ip6: no

kiekar

Thank you all for your input and pushing me to find a solution.

The issue resided at the GeoIP Top Spammers rule. I changed the Action from Deny Both to Deny Inbound which resolved my issue

Gertjan

@kiekar said in Unable to download Ubuntu Updates:

I changed the Action from Deny Both to Deny Inbound which resolved my issue

Inbound : traffic coming IN = traffic that comes in your WAN.

By default : the WAN firewall rule list is empty, so : by default nothing comes in anyway.

The perfect solution would be : why do you even bother using the GEO IP list ?
Drop it, make things easier, and you'll be safe, fine, and less issue can arrive.

I changed the Action from Deny Both

Because you were afraid that you could actually send mail ( ? ) or interact with these IP's ?
What you've found out : Unbuntu package servers are listed on the GEO IP you use.
Strange, as these servers never send or push any traffic to you or anybody else. They work 'on request, initiated on your side'.
Conclusion : never fully trust list you obtained from 'somewhere'. Check them. And if checking is to hard, don't use them.

kiekar

@Gertjan said in Unable to download Ubuntu Updates:

The perfect solution would be : why do you even bother using the GEO IP list ?

Because I have open ports to the public for my mail server and web server.

Thanks,

Gertjan

@kiekar

Right, that is a reason to use some (GEO) IP blocking for incoming connection on WAN.