Unable to update without turning off firewall
-
Hi,
I'm having some problems updating an installation. It fails both from the web and from the cli.
However, if I disable the firewall with 'pfctl -d' the update completes just fine. I'm guessing it should not be like this.
Is there something I'm missing? Does it need some kind of rules to allow outbound traffic somehow?When I try the update from cli, it gives me this output:
pkg-static -d update
DBG(1)[31918]> pkg initialized
Updating pfSense-core repository catalogue...
DBG(1)[31918]> PkgRepo: verifying update for pfSense-core
DBG(1)[31918]> PkgRepo: need forced update of pfSense-core
DBG(1)[31918]> Pkgrepo, begin update of '/var/db/pkg/repo-pfSense-core.sqlite'
DBG(1)[31918]> Request to fetch pkg+https://pkg.pfsense.org/pfSense_v2_7_2_amd64-core/meta.conf
DBG(1)[31918]> curl_open
DBG(1)[31918]> Fetch: fetcher used: pkg+https
DBG(1)[31918]> curl> fetching https://pkg.pfsense.org/pfSense_v2_7_2_amd64-core/meta.confDBG(1)[31918]> CURL> attempting to fetch from , left retry 3
- Couldn't find host pkg01-atx.netgate.com in the .netrc file; using defaults
- Trying 208.123.73.209:443...
- Connected to pkg01-atx.netgate.com (208.123.73.209) port 443
- ALPN: curl offers http/1.1
- CAfile: none
- CApath: /etc/ssl/certs/
And then it just times out and repeats trying to connect to the update server.
Kind regards
Mario -
Do you have any outbound block rules?
Normally all traffic from the firewall itself would be allowed but you could block it with a floating outbound rule.
Steve
-
@stephenw10
I thought so to but I have no floating rules. If I add a floating rule to pass all outbound traffic it seems to work. Should this actually be needed?The system have been upgraded from 2.6, but all other things appear to be working just fine. It is just the updating that struggles.
Mario
-
No you shouldn't need to add a pass rule.
Without that do you see traffic blocked in the firewall logs?
Do you see open states to 208.123.73.X? Are they on the WAN?
-
@stephenw10
With no rules in floating I do get a few states to 208.123.73.X:
However the update just hangs on the same spot and repeats after a timeout.
Mario
-
@eangel
can you ping :
?
Btw : the update/upgrade scripts use host name.
The host name was resolved just fine, so at least the resolver can access the WAN to do the resolving. -
-
Or curl to it directly?
[24.11-RELEASE][admin@6100.stevew.lan]/root: curl -v https://pkg01-atx.netgate.com * Host pkg01-atx.netgate.com:443 was resolved. * IPv6: 2610:160:11:18::209 * IPv4: 208.123.73.209 * Trying [2610:160:11:18::209]:443... * Immediate connect fail for 2610:160:11:18::209: No route to host * Trying 208.123.73.209:443... * ALPN: curl offers h2,http/1.1 * TLSv1.3 (OUT), TLS handshake, Client hello (1): * CAfile: none * CApath: /etc/ssl/certs/ * TLSv1.3 (IN), TLS handshake, Server hello (2): * TLSv1.2 (IN), TLS handshake, Certificate (11): * TLSv1.2 (IN), TLS handshake, Server key exchange (12): * TLSv1.2 (IN), TLS handshake, Server finished (14): * TLSv1.2 (OUT), TLS handshake, Client key exchange (16): * TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1): * TLSv1.2 (OUT), TLS handshake, Finished (20): * TLSv1.2 (IN), TLS handshake, Finished (20): * SSL connection using TLSv1.2 / ECDHE-RSA-AES256-GCM-SHA384 / X25519 / RSASSA-PSS * ALPN: server accepted http/1.1 * Server certificate: * subject: CN=*.netgate.com * start date: Mar 28 00:00:00 2024 GMT * expire date: Apr 28 23:59:59 2025 GMT * subjectAltName: host "pkg01-atx.netgate.com" matched cert's "*.netgate.com" * issuer: C=GB; ST=Greater Manchester; L=Salford; O=Sectigo Limited; CN=Sectigo RSA Domain Validation Secure Server CA * SSL certificate verify ok. * Certificate level 0: Public key type RSA (2048/112 Bits/secBits), signed using sha256WithRSAEncryption * Certificate level 1: Public key type RSA (2048/112 Bits/secBits), signed using sha384WithRSAEncryption * Certificate level 2: Public key type RSA (4096/152 Bits/secBits), signed using sha384WithRSAEncryption * Connected to pkg01-atx.netgate.com (208.123.73.209) port 443 * using HTTP/1.x > GET / HTTP/1.1 > Host: pkg01-atx.netgate.com > User-Agent: curl/8.10.1 > Accept: */* > * Request completely sent off < HTTP/1.1 200 OK < Server: nginx < Date: Thu, 05 Dec 2024 14:34:18 GMT < Content-Type: text/html < Transfer-Encoding: chunked < Connection: keep-alive < Strict-Transport-Security: max-age=31536000; preload < X-Content-Type-Options: nosniff < X-XSS-Protection: 1; mode=block < X-Robots-Tag: all < X-Download-Options: noopen < X-Permitted-Cross-Domain-Policies: none < <html> <head><title>Index of /</title></head> <body> <h1>Index of /</h1><hr><pre><a href="../">../</a> <a href="bkp/">bkp/</a> 27-Feb-2022 18:01 - <a href="pfSense_v2_3_0_amd64-core/">pfSense_v2_3_0_amd64-core/</a> 27-Feb-2022 18:01 - <a href="pfSense_v2_3_0_amd64-pfSense_v2_3_0/">pfSense_v2_3_0_amd64-pfSense_v2_3_0/</a> 27-Feb-2022 18:19 - <a href="pfSense_v2_3_0_i386-core/">pfSense_v2_3_0_i386-core/</a> 27-Feb-2022 18:01 - <a href="pfSense_v2_3_0_i386-pfSense_v2_3_0/">pfSense_v2_3_0_i386-pfSense_v2_3_0/</a> 27-Feb-2022 18:19 - <a href="pfSense_v2_3_1_amd64-core/">pfSense_v2_3_1_amd64-core/</a> 27-Feb-2022 18:01 - <a href="pfSense_v2_3_1_amd64-pfSense_v2_3_1/">pfSense_v2_3_1_amd64-pfSense_v2_3_1/</a> 27-Feb-2022 18:19 - <a href="pfSense_v2_3_1_i386-core/">pfSense_v2_3_1_i386-core/</a> 27-Feb-2022 18:01 - <a href="pfSense_v2_3_1_i386-pfSense_v2_3_1/">pfSense_v2_3_1_i386-pfSense_v2_3_1/</a> 27-Feb-2022 18:19 - <a href="pfSense_v2_3_2_amd64-core/">pfSense_v2_3_2_amd64-core/</a> 27-Feb-2022 18:01 - <a href="pfSense_v2_3_2_amd64-pfSense_v2_3_2/">pfSense_v2_3_2_amd64-pfSense_v2_3_2/</a> 27-Feb-2022 18:19 - <a href="pfSense_v2_3_2_i386-core/">pfSense_v2_3_2_i386-core/</a> 27-Feb-2022 18:01 - <a href="pfSense_v2_3_2_i386-pfSense_v2_3_2/">pfSense_v2_3_2_i386-pfSense_v2_3_2/</a> 27-Feb-2022 18:19 - <a href="pfSense_v2_3_3_amd64-core/">pfSense_v2_3_3_amd64-core/</a> 27-Feb-2022 18:01 - <a href="pfSense_v2_3_3_amd64-pfSense_v2_3_3/">pfSense_v2_3_3_amd64-pfSense_v2_3_3/</a> 27-Feb-2022 18:19 - <a href="pfSense_v2_3_3_i386-core/">pfSense_v2_3_3_i386-core/</a> 27-Feb-2022 18:01 - <a href="pfSense_v2_3_3_i386-pfSense_v2_3_3/">pfSense_v2_3_3_i386-pfSense_v2_3_3/</a> 27-Feb-2022 18:19 - <a href="pfSense_v2_3_4_amd64-core/">pfSense_v2_3_4_amd64-core/</a> 27-Feb-2022 18:01 - <a href="pfSense_v2_3_4_amd64-pfSense_v2_3_4/">pfSense_v2_3_4_amd64-pfSense_v2_3_4/</a> 27-Feb-2022 18:20 - <a href="pfSense_v2_3_4_i386-core/">pfSense_v2_3_4_i386-core/</a> 27-Feb-2022 18:01 - <a href="pfSense_v2_3_4_i386-pfSense_v2_3_4/">pfSense_v2_3_4_i386-pfSense_v2_3_4/</a> 27-Feb-2022 18:20 - <a href="pfSense_v2_3_5_amd64-core/">pfSense_v2_3_5_amd64-core/</a> 27-Feb-2022 18:01 - <a href="pfSense_v2_3_5_amd64-pfSense_v2_3_5/">pfSense_v2_3_5_amd64-pfSense_v2_3_5/</a> 27-Feb-2022 20:15 - <a href="pfSense_v2_3_5_i386-core/">pfSense_v2_3_5_i386-core/</a> 27-Feb-2022 18:01 - <a href="pfSense_v2_3_5_i386-pfSense_v2_3_5/">pfSense_v2_3_5_i386-pfSense_v2_3_5/</a> 27-Feb-2022 18:20 - <a href="pfSense_v2_4_0_amd64-core/">pfSense_v2_4_0_amd64-core/</a> 27-Feb-2022 18:01 - <a href="pfSense_v2_4_0_amd64-pfSense_v2_4_0/">pfSense_v2_4_0_amd64-pfSense_v2_4_0/</a> 27-Feb-2022 18:20 - <a href="pfSense_v2_4_1_amd64-core/">pfSense_v2_4_1_amd64-core/</a> 27-Feb-2022 18:01 - <a href="pfSense_v2_4_1_amd64-pfSense_v2_4_1/">pfSense_v2_4_1_amd64-pfSense_v2_4_1/</a> 27-Feb-2022 18:20 - <a href="pfSense_v2_4_2_amd64-core/">pfSense_v2_4_2_amd64-core/</a> 27-Feb-2022 18:01 - <a href="pfSense_v2_4_2_amd64-pfSense_v2_4_2/">pfSense_v2_4_2_amd64-pfSense_v2_4_2/</a> 27-Feb-2022 18:20 - <a href="pfSense_v2_4_3_amd64-core/">pfSense_v2_4_3_amd64-core/</a> 27-Feb-2022 18:01 - <a href="pfSense_v2_4_3_amd64-pfSense_v2_4_3/">pfSense_v2_4_3_amd64-pfSense_v2_4_3/</a> 27-Feb-2022 18:20 - <a href="pfSense_v2_4_4_amd64-core/">pfSense_v2_4_4_amd64-core/</a> 27-Feb-2022 18:01 - <a href="pfSense_v2_4_4_amd64-pfSense_v2_4_4/">pfSense_v2_4_4_amd64-pfSense_v2_4_4/</a> 27-Feb-2022 18:20 - <a href="pfSense_v2_4_5_amd64-core/">pfSense_v2_4_5_amd64-core/</a> 27-Feb-2022 18:01 - <a href="pfSense_v2_4_5_amd64-pfSense_v2_4_5/">pfSense_v2_4_5_amd64-pfSense_v2_4_5/</a> 27-Feb-2022 18:20 - <a href="pfSense_v2_5_0_amd64-core/">pfSense_v2_5_0_amd64-core/</a> 27-Feb-2022 18:01 - <a href="pfSense_v2_5_0_amd64-pfSense_v2_5_0/">pfSense_v2_5_0_amd64-pfSense_v2_5_0/</a> 27-Feb-2022 18:20 - <a href="pfSense_v2_5_1_amd64-core/">pfSense_v2_5_1_amd64-core/</a> 27-Feb-2022 18:01 - <a href="pfSense_v2_5_1_amd64-pfSense_v2_5_1/">pfSense_v2_5_1_amd64-pfSense_v2_5_1/</a> 27-Feb-2022 18:20 - <a href="pfSense_v2_5_2_amd64-core/">pfSense_v2_5_2_amd64-core/</a> 27-Feb-2022 18:01 - <a href="pfSense_v2_5_2_amd64-pfSense_v2_5_2/">pfSense_v2_5_2_amd64-pfSense_v2_5_2/</a> 23-Mar-2022 17:49 - <a href="pfSense_v2_6_0_amd64-core/">pfSense_v2_6_0_amd64-core/</a> 27-Feb-2022 18:01 - <a href="pfSense_v2_6_0_amd64-pfSense_v2_6_0/">pfSense_v2_6_0_amd64-pfSense_v2_6_0/</a> 30-Oct-2023 06:12 - <a href="pfSense_v2_7_0_amd64-core/">pfSense_v2_7_0_amd64-core/</a> 29-Jun-2023 15:49 - <a href="pfSense_v2_7_0_amd64-pfSense_v2_7_0/">pfSense_v2_7_0_amd64-pfSense_v2_7_0/</a> 16-Nov-2023 20:57 - <a href="pfSense_v2_7_1_amd64-core/">pfSense_v2_7_1_amd64-core/</a> 16-Nov-2023 20:39 - <a href="pfSense_v2_7_1_amd64-pfSense_v2_7_1/">pfSense_v2_7_1_amd64-pfSense_v2_7_1/</a> 09-Feb-2024 17:49 - <a href="pfSense_v2_7_2_amd64-core/">pfSense_v2_7_2_amd64-core/</a> 07-Dec-2023 22:25 - <a href="pfSense_v2_7_2_amd64-pfSense_v2_7_2/">pfSense_v2_7_2_amd64-pfSense_v2_7_2/</a> 26-Nov-2024 00:41 - </pre><hr></body> </html> * Connection #0 to host pkg01-atx.netgate.com left intact -
And
UNTRUSTis your WAN interface there? -
@stephenw10
Yes, UNTRUST is my WAN.Curl directly actually fails at the same point:
curl -v https://pkg01-atx.netgate.com
- Host pkg01-atx.netgate.com:443 was resolved.
- IPv6: 2610:160:11:18::209
- IPv4: 208.123.73.209
- Trying 208.123.73.209:443...
- Connected to pkg01-atx.netgate.com (208.123.73.209) port 443
- ALPN: curl offers h2,http/1.1
- TLSv1.3 (OUT), TLS handshake, Client hello (1):
- CAfile: none
- CApath: /etc/ssl/certs/
-
You are upgrading from what version ?
A client device on LAN can visit ordinary sites using https = port 443 TCP ?
No other devices behind the UNTRUSTED (WAN) under your control ?
Is the date and time ok on pfSense ? It seesm like you can connect, but TLS never starts up.
( cert issues ?) -
I have upgraded from 2.6.0 via 2.7.0 to 2.7.2
Yes. Accessing from the LAN side is ok.
This is a lab environment and I have control over most of what is beyond the WAN.
Date and time is ok. However, I do have an out of date certificate on the box, but I fail to see that it could be problematic only when the firewall is active. Do you think it could be the problem?
Mario
-
Wait ... you might have to type the special @stephenw10 command ....
-
-
I would expect to see a cert error there if it was a cert problem at either end. It's just timing out as I understand it though I don't see the actual error.
The states show it is connecting but then just fails.
Try a pcap on the WAN filtered by the destination IP then run the update.
Could be an MTU issue perhaps.
-
@stephenw10
Thanks for answering. Unfortunately I don't get to work with this until tuesday next week. I'll do a capture then.I agree that it looks like a packet size issue, but I don't see how it could be, as the packets getting sent should not change size just because I turn off the firewall?
Mario
-
Well with pf-scrub it re-assembles fragments.... or tries to.
-
@stephenw10
Good morning,I got to do the capture today. I must admit that I don't know what to make of it.
I'll try to attach it here. -
Hmm, interesting. Other traffic is passi8ng the firewall OK?
The pkg server never sees the ACK packet from pfSense so it just keeps sending SYN-ACK and never moves past that.
If you look at the MAC addresses you might have some asymmetry somewhere because pfSense is sending to 60:15:2b:fe:f1:11 but the replies comes back from 00:10:db:ff:10:00.
Which of those is the expected gateway MAC/IP?You don't see anything blocked in pfSense?
-
Awesome! That was the answer.
I have 2 gateways in the same subnet connected to the router for test purposes, and somehow it was using the non-default one for outgoing traffic.
I never noticed that the traffic was asymmetric. Thanks for pointing it out for me.Mario
