Cannot install/update packages on fresh install

hwcltjn

Hello all,

I'm pretty new to pfSense and have it set up on a virtual machine, on a Proxmox host.

For some reason, I cannot update the package list from within the GUI or using option 13 in the console.

I keep getting the same error, even with when running pgk udpate

[2.4.4-RELEASE][root@fw-001]/root: pkg update
Updating pfSense-core repository catalogue...
pkg: Repository pfSense-core load error: access repo file(/var/db/pkg/repo-pfSense-core.sqlite) failed: No such file or directory
pkg: https://pkg.pfsense.org/pfSense_v2_4_4_amd64-core/meta.txz: No route to host
repository pfSense-core has no meta file, using default settings
pkg: https://pkg.pfsense.org/pfSense_v2_4_4_amd64-core/packagesite.txz: No route to host
Unable to update repository pfSense-core
Updating pfSense repository catalogue...
pkg: Repository pfSense load error: access repo file(/var/db/pkg/repo-pfSense.sqlite) failed: No such file or directory
pkg: https://pkg.pfsense.org/pfSense_v2_4_4_amd64-pfSense_v2_4_4/meta.txz: No route to host
repository pfSense has no meta file, using default settings
pkg: https://pkg.pfsense.org/pfSense_v2_4_4_amd64-pfSense_v2_4_4/packagesite.txz: No route to host
Unable to update repository pfSense
Error updating repositories!

Versions
pfSense - 2.4.4 p3
Proxmox 1 - 6.1-7
Proxmox 2 - 5.4-13

What I've tried/checked

pfSense can ping external hosts and is correctly resolving domain names, from both the command line and diagnostics
There is a default gateway configured
There are no gateway groups
Swapped from dev to latest and back, in "System -> Update - > System Update"
Other VM's are fully reaching the internet from behind pfSense with no issues (Ubuntu 18 test VM)
Tried a different ISO from a different mirror - NY, Frankfurt and Austin
Tried the same setup on a different Proxmox host - same result
Hardware Checksum Offloading is disabled/checked under "System -> Advanced -> Networking"
No external filtering or additional firewalls - servers are with Kimsufi
Network interfaces for the VMs on Proxmox are configured as VirtIO (paravirtualized)
DNS Servers are set as 1.1.1.1 and 8.8.8.8 under "System -> General Setup"
DNS Server Override is un-checked
Timezone and date are correct
The pfSense update SRV records are resolvable

[2.4.4-RELEASE][root@fw-001]/root: host -t srv _https._tcp.pkg.pfsense.org
_https._tcp.pkg.pfsense.org has SRV record 10 10 443 files01.netgate.com.
_https._tcp.pkg.pfsense.org has SRV record 10 10 443 files00.netgate.com.
[2.4.4-RELEASE][root@fw-001]/root: host files01.netgate.com
files01.netgate.com has address 162.208.119.40
files01.netgate.com has IPv6 address 2607:ee80:10::119:40
[2.4.4-RELEASE][root@fw-001]/root: host files00.netgate.com
files00.netgate.com has address 162.208.119.41
files00.netgate.com has IPv6 address 2607:ee80:10::119:41

The Proxmox host has 1 public IP address.
All traffic from the host is forwarded using iptables.

Contents of Proxmox /etc/network/interfaces

auto lo
iface lo inet loopback

auto eno1
iface eno1 inet manual

auto vmbr0
iface vmbr0 inet dhcp
        bridge-ports eno1
        bridge-stp off
        bridge-fd 0

auto vmbr100
iface vmbr100 inet static
        address 172.31.255.253
        netmask 24
        post-up /bin/echo 1 > /proc/sys/net/ipv4/ip_forward
        post-up /sbin/iptables -t nat -A POSTROUTING -s '172.31.255.0/24' -o vmbr0 -j MASQUERADE
        post-up /sbin/iptables -t nat -A PREROUTING -p tcp --match multiport ! --dport 8006,2221 -j DNAT --to-destination 172.31.255.254

bridge-ports none
        bridge-stp off
        bridge-fd 0

auto vmbr200
iface vmbr200 inet manual
        bridge-ports n

It's a fresh installation, nothing really configured yet and pfctl is mostly disabled (whilst trying to figure this out). I have also re-installed a few times.

On the dashboard, under "Netgate Services and Support" it's stuck on "Retrieving support information".

I asked on IRC and some awesome people were wondering why my updates are trying to be fetched from https://pkg.pfsense.org/ instead of https://files00.netgate.com/? They downloaded the same ISO's and were not able to replicate the problem in VirtualBox. Could not explain it.
I too was unable to reproduce the problem in VirtualBox on my local machine, clearly this is limited to my Proxmox setup...

Any ideas? I'm about ready to pull my hair out.

kiokoman

I repeat what I told you on freenode, in the hope that someone can add something to this.

the problem is here from my understanding,

pkg: Repository pfSense load error: access repo file(/var/db/pkg/repo-pfSense.sqlite) failed: No such file or directory

this tell me that the file system is corrupted somehow

i tried to download that iso you have and it was working on my virtualbox without problem so idk what could lead to a missing/corruption after a clean install

another possible reason maybe you are using the wrong iso like 2.4.4 instead of 2.4.4-p3

jimp

@hwcltjn said in Cannot install/update packages on fresh install:

pkg: Repository pfSense-core load error: access repo file(/var/db/pkg/repo-pfSense-core.sqlite) failed: No such file or directory

This may mean it was never able to download it correctly.

pkg: https://pkg.pfsense.org/pfSense_v2_4_4_amd64-core/meta.txz: No route to host

The real problem is here. No route to host means just that. The firewall itself has no route out. Your default route is missing or not set. Check your default gateway settings under System > Routing.

kiokoman

it was one of our idea but
no route to host come after
https://pkg.pfsense.org/pfSense_v2_4_4_amd64-core
and afaik it does not exist -> nxdomain
?

jimp

It's resolved using SRV records. DNS is fine. It's a routing problem.

EDIT: https://docs.netgate.com/pfsense/en/latest/install/upgrade-troubleshooting.html#pkg-pfsense-org-has-no-a-aaaa-record

Just step through everything on https://docs.netgate.com/pfsense/en/latest/install/upgrade-troubleshooting.html -- all the errors and fixes are covered there.

hwcltjn

@kiokoman said in Cannot install/update packages on fresh install:

this tell me that the file system is corrupted somehow

I tried a different storage controller in proxmox, didn't change anything.

I tried to download that iso you have and it was working on my virtualbox without problem so idk what could lead to a missing/corruption after a clean install

I too downloaded a fresh ISO and tried on my local VirtualBox, worked no problem. I'm pretty sure it's something with my setup.

another possible reason maybe you are using the wrong iso like 2.4.4 instead of 2.4.4-p3

100% using 2.4.4-p3 ISO

@jimp said in Cannot install/update packages on fresh install:

The real problem is here. No route to host means just that. The firewall itself has no route out. Your default route is missing or not set. Check your default gateway settings under System > Routing.

I've checked them a few times, maybe I missed something really basic ?

I also tried the steps in the links you provided, none of them worked.

Below are routing and firewall screenshots.

fw-001.test - System: Routing: Gateways 2020-03-16 18-03-10.png fw-001.test - Firewall: Rules: WAN 2020-03-16 18-05-28.png fw-001.test - Diagnostics: Routes 2020-03-16 18-04-49.png

jimp

That WAN rule is dangerous and unnecessary.

What is upstream of pfSense? Does it just go to your ISP?

Since it's vtnet, it might be something in your Hypervisor config as well.

Try doing a traceroute to files00.netgate.com and see how far it gets.

hwcltjn

@jimp said in Cannot install/update packages on fresh install:

That WAN rule is dangerous and unnecessary.

Only temporary

What is upstream of pfSense? Does it just go to your ISP?

It goes straight out. All installed on a dedicated server with Kimsufi.
pfSense --> Proxmox Host --> WAN

Since it's vtnet, it might be something in your Hypervisor config as well.

Maybe, I posted it above...

Try doing a traceroute to files00.netgate.com and see how far it gets.

[2.4.4-RELEASE][root@fw-test]/root: traceroute files00.netgate.com
traceroute to files00.netgate.com (162.208.119.41), 64 hops max, 40 byte packets
 1  172.31.255.253 (172.31.255.253)  0.200 ms  0.192 ms  0.147 ms
 2  x (91.121.x.x)  3.207 ms  1.777 ms  1.656 ms
 3  10.17.20.52 (10.17.20.52)  1.049 ms  1.063 ms  1.039 ms
 4  10.73.16.166 (10.73.16.166)  0.549 ms
    10.73.16.228 (10.73.16.228)  0.488 ms  0.582 ms
 5  10.95.64.0 (10.95.64.0)  1.817 ms  1.817 ms
    10.95.64.2 (10.95.64.2)  4.998 ms
 6  be100-1043.th2-1-a9.fr.eu (94.23.122.147)  4.686 ms  4.803 ms
    be100-1042.ldn-5-a9.uk.eu (213.251.130.103)  5.100 ms
 7  ge-2-1-0.mpr1.lhr2.uk.above.net (195.66.224.76)  6.578 ms  16.202 ms  9.992 ms
 8  ae27.cs1.cdg12.fr.eth.zayo.com (64.125.29.6)  74.554 ms
    ae11.mpr2.lhr2.uk.zip.zayo.com (64.125.30.52)  4.763 ms
    ae27.cs1.cdg12.fr.eth.zayo.com (64.125.29.6)  74.655 ms
 9  * * *
10  * * *
11  ae20.mpr2.ewr1.us.zip.zayo.com (64.125.26.143)  76.452 ms  72.746 ms  75.748 ms
12  ae3.mpr2.ewr1.us.zip.zayo.com (64.125.31.238)  77.714 ms  77.753 ms  74.664 ms
13  208.184.34.238.ipyx-076763-900-zyo.zip.zayo.com (208.184.34.238)  121.080 ms  75.352 ms  75.342 ms
14  cs90.cs99new.v.ewr.nyinternet.net (96.47.77.218)  76.641 ms  73.680 ms  76.702 ms
15  * * *
16  * * *
17  * * *
[...]
50  * * *

@hwcltjn said in Cannot install/update packages on fresh install:

What I've tried/checked

Other VM's are fully reaching the internet from behind pfSense with no issues (Ubuntu 18 test VM)

This isn't actually the case... Ubuntu VM can't go out, but it can resolve addresses and ping.

jimp

Try this:

pkg update -4 -f

Maybe your system is trying to reach out via IPv6, though from the looks of your routing table, I don't see why it would.

hwcltjn

Also fails unfortunately
I think I have a larger networking problem - going to re-examine Proxmox config

stephenw10

@hwcltjn said in Cannot install/update packages on fresh install:

traceroute files00.netgate.com

That also fails for me in exactly the same way but I am able to update packages.

It succeeds if I traceroute using ICMP though: traceroute -I files00.netgate.com

Steve