2.6.0 - Installed Pkgs - Unable to retrieve package information.

fabrizior · Mar 25, 2022, 3:35 PM

Something went bonkers this morning on my production router running 2.6.0... I started getting unbound restarts for no known reason via Service Watchdog. I couldn't recognize a cause in the logs. Figuring I'd try a reboot to see if it cleared, I restarted from the GUI diagnostic menu.

After the reboot, unbound has been stable.

I figured I'd check for package updates and found that the Installed Packages UI wasn't working, with the error: Unable to retrieve package information.

running pkg update from the CLI is extremely slow, lagging mostly on the Updating pfSense-core repository catalogue... operation. 7 minutes?

[2.6.0-RELEASE][root@pfSense.redacted.domain]/root: time pkg update
Updating pfSense-core repository catalogue...
pfSense-core repository is up to date.
Updating pfSense repository catalogue...
pfSense repository is up to date.
All repositories are up to date.
0.068u 0.026s 7:32.23 0.0%	3428+296k 0+6io 0pf+0w

many of the UI views, including Diagnostics>Traceroute, system logs, and others are working very slowly. I have full internet access, and good speed tests from hosts on my network.

Nothing's changed since I upgraded to 2.6.0... what would have caused this to suddenly start happening this morning?

Where should I start with triaging whatever's gone sideways on my router this morning?

mrsunfire · Mar 26, 2022, 4:58 AM

@fabrizior Having the exact same issue with 2.6.0. On Dashboard it also show „Unable to check for update“

Also GUI is kind of slow/bugged yes.

mrsunfire · Mar 26, 2022, 6:51 AM

@fabrizior Do you have multi WAN setup?

jan8p · Mar 26, 2022, 7:50 AM

Exactly the same issue here, update page and package manager page of the GUI won't load, just hangs. Used to work a day or two ago.

I guess those pages are reaching out to external URLs that are down?

bingo600 · Mar 26, 2022, 8:34 AM

Think i saw some issue related to "check for update preferring ipv6" if possible , on the forum.
Do you have a working ipv6 setup ?

Anyway i just have ipv4 here at home.
And yesterday reinstalled my homebox from scratch (usb) w. 2.6.0 , and i had no issues reinstalling the packets.
Nor did i have any issues registering my pfSense+ key , and upgrade to 22.01

I did see & report "similar" issue if i pointed my 2.6.0 to the plus upgrade repos 4 days ago.
https://forum.netgate.com/topic/171004/solved-upgrading-from-2-6-0-to-plus-unable-to-check-for-updates
But it was fixed quite fast, and i think it was a CERT error on the plus section.

I had no issues if i reverted to pointing at the 2.6.0 repos.

Not much help , but it was working for me yesterday, using ipv4.

/Bingo

jan8p · Mar 26, 2022, 8:14 AM

@bingo600 ah yes! I checked "prefer IPv4" and now it works again! Seems their repo must still be having IPv6 issues!

Thank you :)

mrsunfire · Mar 26, 2022, 8:52 AM

Thanks, that fixed it for me as well! IPv6 was the problem.

fabrizior · Mar 26, 2022, 2:31 PM

@mrsunfire
no… just one set of ipv4 and ipv6 gateways for a single wan interface.
though I have openvpn and wireguard (remote access, not site-to-site) configured

fabrizior · Mar 26, 2022, 2:33 PM

@bingo600

perhaps an enhancement request on redmine is in-order to catch the timeouts or faults against the package repo and provide a properly i formative error msg?

fabrizior · Mar 26, 2022, 2:34 PM

@mrsunfire why is ipv6 an issue? hasn’t been a problem in the past?!

mrsunfire · Mar 27, 2022, 8:50 AM

@fabrizior I think the fault is on Netgate side as they having issues with IPv6 connectivity.

Thondwe · Mar 27, 2022, 3:48 PM

@jan8p I turned on debugging and it looks like at least one repository isn't working on IPv6?

e.g. ping pkg01-atx.netgate.com works but ping6 pkg01-atx.netgate.com doesn't and the GUI/default is for scripts to use IPv6 over IPv4...

fabrizior · Mar 27, 2022, 9:45 PM

@jan8p I checked prefer IPv4 in adv. settings... and saved. System > Update and Package manager are still having connection faults. Did I miss a step?

Gertjan · Mar 28, 2022, 8:01 AM

@fabrizior said in 2.6.0 - Installed Pkgs - Unable to retrieve package information.:

are still having connection faults. Did I miss a step?

Goto console or SSH, option 8 and

ps ax | grep 'pkg'

if you see any pkk update" instances, you have to wait before they time out.
Or kill them.

See also https://forum.netgate.com/topic/171103/unable-to-check-for-updates/6?_=1648452964596

Thondwe · Mar 28, 2022, 8:56 AM

@fabrizior I just did the same and it seems to have fixed the issue, though clearly NetGate should fix their side - imagine this tick box is to help people who haven't got IPv6 working locally for whatever reason...

Gertjan · Mar 28, 2022, 12:37 PM

@thondwe said in 2.6.0 - Installed Pkgs - Unable to retrieve package information.:

though clearly NetGate should fix their side

Netgate isn't an ISP, neither peering company. Their servers have the correct IPv6 setup.
IMHO : This issue is between them and us.

And the issue isn't visible for all of us.

jan8p · Mar 28, 2022, 12:47 PM

@fabrizior All I did extra is a reboot....not sure if that is required though!

Thondwe · Mar 28, 2022, 12:59 PM

@gertjan Running a tracert on IPv4 gets me 13 steps to pkg01-atx.netgate.com and looks to get to Zcolo - so Co location Company that's part of ZAYO?

Run the tracert on IPv6 30(!) steps a lot of which timeout. Both seem to reference ZAYO - hop 9 on IP4 and hop 11 on IPv6.

So you might well be right and it's an IPV6 routing problem in the co-lo company where Netgate have their servers??

fabrizior · Mar 28, 2022, 2:56 PM

@gertjan took a few minutes but the ipv4 workaround has now been working reliably. thanks all.

fabrizior · Mar 28, 2022, 3:07 PM

@thondwe said in 2.6.0 - Installed Pkgs - Unable to retrieve package information.:

pkg01-atx.netgate.com

How many different package servers are there? Anyone have a list? will add to my telegraf monitoring. Want to see how (un)reliable these are over time.