Solved: Can't update bogons on a 2.4.5-p1 (cert expired)

bingo600

I jus noticed today that i can't update the bogon files via
Diagnostics -> Tables -> Bogons , and press update

The task just hangs (seems like forever)

I know the 2.4.5-p1 is "Not the latest " , but is there a quick hint please ??

/Bingo

ps aux | grep -i bogon
root    70221   0.0  0.0   6976   2728  -  I    17:49    0:00.01 /bin/sh /etc/rc.update_bogons.sh now
root    72230   0.0  0.1   9304   6012  -  I    17:49    0:00.05 /usr/bin/fetch -a -w 600 -T 30 -q -o /tmp/bogons https://files.pfsense.org/lists/fullbogons-ipv4.txt
root     8734   0.0  0.0   6560   2292  0  S+   17:58    0:00.00 grep -i bogon

/usr/bin/fetch -a -w 600 -T 30 -q -o /tmp/bogons https://files.pfsense.org/lists/fullbogons-ipv4.txt
Certificate verification failed for /O=Digital Signature Trust Co./CN=DST Root CA X3
34374270280:error:14090086:SSL routines:ssl3_get_server_certificate:certificate verify failed:/build/ce-crossbuild-245/sources/FreeBSD-src/crypto/openssl/ssl/s3_clnt.c:1269:
fetch: https://files.pfsense.org/lists/fullbogons-ipv4.txt: Authentication error

openssl s_client -showcerts -connect files.pfsense.org:443
CONNECTED(00000004)
depth=3 O = Digital Signature Trust Co., CN = DST Root CA X3
verify error:num=10:certificate has expired
notAfter=Sep 30 14:01:15 2021 GMT
---
Certificate chain
 0 s:/CN=pfsense.org
   i:/C=US/O=Let's Encrypt/CN=R3
-----BEGIN CERTIFICATE-----
MIIFRjCCBC6gAwIBAgISA0jjZjZ4Ba2iE4VHc6eh6OgZMA0GCSqGSIb3DQEBCwUA
MDIxCzAJBgNVBAYTAlVTMRYwFAYDVQQKEw1MZXQncyBFbmNyeXB0MQswCQYDVQQD
EwJSMzAeFw0yMTA5MjcwNzEwMDdaFw0yMTEyMjYwNzEwMDZaMBYxFDASBgNVBAMT
C3Bmc2Vuc2Uub3JnMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA9Acm
QTm5xxahHqvEQZafPZUjVJtDe8v1ju5UNpcNxeZdOzjMiw7I/oWD9bnSSEvyma/x
FrewsYN0conqy/LH1yqj0xTjWoCzVi796uHh00iLKsCiduOhjJp4Mp8zAPkjqdhs
DF6CvgEPE9akYOOD6sQAS4teud+xkui2PGp9y3oRPpbF/HY3lWugUAix9+jOiC4B
nYcBaHde3O5eFlHw3LPkPveSFEEch0B+QNSwMX6dO5Qhr3rozCbQYodrIGanFrJF
xcoyKX0tvNxta8fZ0CMnFJ5g6aJPnxdGr6RkHvcazsakkHdzRaQmQsqf35HxMMXn
Mb7YWQsQ1TskXUCiawIDAQABo4ICcDCCAmwwDgYDVR0PAQH/BAQDAgWgMB0GA1Ud
JQQWMBQGCCsGAQUFBwMBBggrBgEFBQcDAjAMBgNVHRMBAf8EAjAAMB0GA1UdDgQW
BBSoEt4r/VnaELgC4WoTSLIHtMfN/TAfBgNVHSMEGDAWgBQULrMXt1hWy65QCUDm
H6+dixTCxjBVBggrBgEFBQcBAQRJMEcwIQYIKwYBBQUHMAGGFWh0dHA6Ly9yMy5v
LmxlbmNyLm9yZzAiBggrBgEFBQcwAoYWaHR0cDovL3IzLmkubGVuY3Iub3JnLzBB
BgNVHREEOjA4gg0qLnBmc2Vuc2UuY29tgg0qLnBmc2Vuc2Uub3JnggtwZnNlbnNl
LmNvbYILcGZzZW5zZS5vcmcwTAYDVR0gBEUwQzAIBgZngQwBAgEwNwYLKwYBBAGC
3xMBAQEwKDAmBggrBgEFBQcCARYaaHR0cDovL2Nwcy5sZXRzZW5jcnlwdC5vcmcw
ggEDBgorBgEEAdZ5AgQCBIH0BIHxAO8AdQD2XJQv0XcwIhRUGAgwlFaO400TGTO/
3wwvIAvMTvFk4wAAAXwmTddIAAAEAwBGMEQCIDKCKkNRV5CqDQj31XSKZ5k11QE1
AcX/zGUVmHHPJqISAiBc6rAJWRD9G4k5RqVgQJlRHewTEoKdlAd8QXMmfUH1rgB2
AG9Tdqwx8DEZ2JkApFEV/3cVHBHZAsEAKQaNsgiaN9kTAAABfCZN2OQAAAQDAEcw
RQIgZuCFa1KFg6O1tzPV+jl4DNENDMJFLrbv1bPRlNOWIr4CIQDbNN1NX7ZDmfdV
JerKsj2RTS9kTI5XN/il9jNf5wEGXzANBgkqhkiG9w0BAQsFAAOCAQEAAuYnIy14
YhkMq6kDM1c60/0KULN7AkPtdIKU/msfx9kYzl2PUvZOq29o71TsCSohQyspe+Ye
oSYK+jr2ot46y9/09xg8jB37mQuKxi1E54ABVl2LwUgqKrY/CtoYTdWDX0oFDzPH
TsXwRQ2tNyz/eHgkd+mX1nh5+Vdzj7C2z8FdKsuUfNdqD98LnlH5UCDtQSS/mTrw
u1P0hm+RDCJeE+Sv1K80HFStA/Acg2JTiQ4LsPOmz2tjnqcq+6n3GnmJkyK0HoK+
nEGCanflpE/VWIfoVE9kQ/RtnTUwKAxT28ZZhcexlErvCMN9wL9l0zvRfTuJztd7
WGuG0dDSU2exrA==
-----END CERTIFICATE-----
 1 s:/C=US/O=Let's Encrypt/CN=R3
   i:/C=US/O=Internet Security Research Group/CN=ISRG Root X1
-----BEGIN CERTIFICATE-----
MIIFFjCCAv6gAwIBAgIRAJErCErPDBinU/bWLiWnX1owDQYJKoZIhvcNAQELBQAw
TzELMAkGA1UEBhMCVVMxKTAnBgNVBAoTIEludGVybmV0IFNlY3VyaXR5IFJlc2Vh
cmNoIEdyb3VwMRUwEwYDVQQDEwxJU1JHIFJvb3QgWDEwHhcNMjAwOTA0MDAwMDAw
WhcNMjUwOTE1MTYwMDAwWjAyMQswCQYDVQQGEwJVUzEWMBQGA1UEChMNTGV0J3Mg
RW5jcnlwdDELMAkGA1UEAxMCUjMwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEK
AoIBAQC7AhUozPaglNMPEuyNVZLD+ILxmaZ6QoinXSaqtSu5xUyxr45r+XXIo9cP
R5QUVTVXjJ6oojkZ9YI8QqlObvU7wy7bjcCwXPNZOOftz2nwWgsbvsCUJCWH+jdx
sxPnHKzhm+/b5DtFUkWWqcFTzjTIUu61ru2P3mBw4qVUq7ZtDpelQDRrK9O8Zutm
NHz6a4uPVymZ+DAXXbpyb/uBxa3Shlg9F8fnCbvxK/eG3MHacV3URuPMrSXBiLxg
Z3Vms/EY96Jc5lP/Ooi2R6X/ExjqmAl3P51T+c8B5fWmcBcUr2Ok/5mzk53cU6cG
/kiFHaFpriV1uxPMUgP17VGhi9sVAgMBAAGjggEIMIIBBDAOBgNVHQ8BAf8EBAMC
AYYwHQYDVR0lBBYwFAYIKwYBBQUHAwIGCCsGAQUFBwMBMBIGA1UdEwEB/wQIMAYB
Af8CAQAwHQYDVR0OBBYEFBQusxe3WFbLrlAJQOYfr52LFMLGMB8GA1UdIwQYMBaA
FHm0WeZ7tuXkAXOACIjIGlj26ZtuMDIGCCsGAQUFBwEBBCYwJDAiBggrBgEFBQcw
AoYWaHR0cDovL3gxLmkubGVuY3Iub3JnLzAnBgNVHR8EIDAeMBygGqAYhhZodHRw
Oi8veDEuYy5sZW5jci5vcmcvMCIGA1UdIAQbMBkwCAYGZ4EMAQIBMA0GCysGAQQB
gt8TAQEBMA0GCSqGSIb3DQEBCwUAA4ICAQCFyk5HPqP3hUSFvNVneLKYY611TR6W
PTNlclQtgaDqw+34IL9fzLdwALduO/ZelN7kIJ+m74uyA+eitRY8kc607TkC53wl
ikfmZW4/RvTZ8M6UK+5UzhK8jCdLuMGYL6KvzXGRSgi3yLgjewQtCPkIVz6D2QQz
CkcheAmCJ8MqyJu5zlzyZMjAvnnAT45tRAxekrsu94sQ4egdRCnbWSDtY7kh+BIm
lJNXoB1lBMEKIq4QDUOXoRgffuDghje1WrG9ML+Hbisq/yFOGwXD9RiX8F6sw6W4
avAuvDszue5L3sz85K+EC4Y/wFVDNvZo4TYXao6Z0f+lQKc0t8DQYzk1OXVu8rp2
yJMC6alLbBfODALZvYH7n7do1AZls4I9d1P4jnkDrQoxB3UqQ9hVl3LEKQ73xF1O
yK5GhDDX8oVfGKF5u+decIsH4YaTw7mP3GFxJSqv3+0lUFJoi5Lc5da149p90Ids
hCExroL1+7mryIkXPeFM5TgO9r0rvZaBFOvV2z0gp35Z0+L4WPlbuEjN/lxPFin+
HlUjr8gRsI3qfJOQFy/9rKIJR0Y/8Omwt/8oTWgy1mdeHmmjk7j1nYsvC9JSQ6Zv
MldlTTKB3zhThV1+XWYp6rjd5JW1zbVWEkLNxE7GJThEUG3szgBVGP7pSWTUTsqX
nLRbwHOoq7hHwg==
-----END CERTIFICATE-----
 2 s:/C=US/O=Internet Security Research Group/CN=ISRG Root X1
   i:/O=Digital Signature Trust Co./CN=DST Root CA X3
-----BEGIN CERTIFICATE-----
MIIFYDCCBEigAwIBAgIQQAF3ITfU6UK47naqPGQKtzANBgkqhkiG9w0BAQsFADA/
MSQwIgYDVQQKExtEaWdpdGFsIFNpZ25hdHVyZSBUcnVzdCBDby4xFzAVBgNVBAMT
DkRTVCBSb290IENBIFgzMB4XDTIxMDEyMDE5MTQwM1oXDTI0MDkzMDE4MTQwM1ow
TzELMAkGA1UEBhMCVVMxKTAnBgNVBAoTIEludGVybmV0IFNlY3VyaXR5IFJlc2Vh
cmNoIEdyb3VwMRUwEwYDVQQDEwxJU1JHIFJvb3QgWDEwggIiMA0GCSqGSIb3DQEB
AQUAA4ICDwAwggIKAoICAQCt6CRz9BQ385ueK1coHIe+3LffOJCMbjzmV6B493XC
ov71am72AE8o295ohmxEk7axY/0UEmu/H9LqMZshftEzPLpI9d1537O4/xLxIZpL
wYqGcWlKZmZsj348cL+tKSIG8+TA5oCu4kuPt5l+lAOf00eXfJlII1PoOK5PCm+D
LtFJV4yAdLbaL9A4jXsDcCEbdfIwPPqPrt3aY6vrFk/CjhFLfs8L6P+1dy70sntK
4EwSJQxwjQMpoOFTJOwT2e4ZvxCzSow/iaNhUd6shweU9GNx7C7ib1uYgeGJXDR5
bHbvO5BieebbpJovJsXQEOEO3tkQjhb7t/eo98flAgeYjzYIlefiN5YNNnWe+w5y
sR2bvAP5SQXYgd0FtCrWQemsAXaVCg/Y39W9Eh81LygXbNKYwagJZHduRze6zqxZ
Xmidf3LWicUGQSk+WT7dJvUkyRGnWqNMQB9GoZm1pzpRboY7nn1ypxIFeFntPlF4
FQsDj43QLwWyPntKHEtzBRL8xurgUBN8Q5N0s8p0544fAQjQMNRbcTa0B7rBMDBc
SLeCO5imfWCKoqMpgsy6vYMEG6KDA0Gh1gXxG8K28Kh8hjtGqEgqiNx2mna/H2ql
PRmP6zjzZN7IKw0KKP/32+IVQtQi0Cdd4Xn+GOdwiK1O5tmLOsbdJ1Fu/7xk9TND
TwIDAQABo4IBRjCCAUIwDwYDVR0TAQH/BAUwAwEB/zAOBgNVHQ8BAf8EBAMCAQYw
SwYIKwYBBQUHAQEEPzA9MDsGCCsGAQUFBzAChi9odHRwOi8vYXBwcy5pZGVudHJ1
c3QuY29tL3Jvb3RzL2RzdHJvb3RjYXgzLnA3YzAfBgNVHSMEGDAWgBTEp7Gkeyxx
+tvhS5B1/8QVYIWJEDBUBgNVHSAETTBLMAgGBmeBDAECATA/BgsrBgEEAYLfEwEB
ATAwMC4GCCsGAQUFBwIBFiJodHRwOi8vY3BzLnJvb3QteDEubGV0c2VuY3J5cHQu
b3JnMDwGA1UdHwQ1MDMwMaAvoC2GK2h0dHA6Ly9jcmwuaWRlbnRydXN0LmNvbS9E
U1RST09UQ0FYM0NSTC5jcmwwHQYDVR0OBBYEFHm0WeZ7tuXkAXOACIjIGlj26Ztu
MA0GCSqGSIb3DQEBCwUAA4IBAQAKcwBslm7/DlLQrt2M51oGrS+o44+/yQoDFVDC
5WxCu2+b9LRPwkSICHXM6webFGJueN7sJ7o5XPWioW5WlHAQU7G75K/QosMrAdSW
9MUgNTP52GE24HGNtLi1qoJFlcDyqSMo59ahy2cI2qBDLKobkx/J3vWraV0T9VuG
WCLKTVXkcGdtwlfFRjlBz4pYg1htmf5X6DYO8A4jqv2Il9DjXA6USbW1FzXSLr9O
he8Y4IWS6wY7bCkjCWDcRQJMEhg76fsO3txE+FiYruq9RUWhiF1myv4Q6W+CyBFC
Dfvp7OOGAN6dEOM4+qR9sdjoSYKEBpsr6GtPAQw4dy753ec5
-----END CERTIFICATE-----
---
Server certificate
subject=/CN=pfsense.org
issuer=/C=US/O=Let's Encrypt/CN=R3
---
No client certificate CA names sent
Peer signing digest: SHA512
Server Temp Key: ECDH, P-256, 256 bits
---
SSL handshake has read 4558 bytes and written 433 bytes
---
New, TLSv1/SSLv3, Cipher is ECDHE-RSA-AES256-GCM-SHA384
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
    Protocol  : TLSv1.2
    Cipher    : ECDHE-RSA-AES256-GCM-SHA384
    Session-ID: 5CF82F877F50686F054D0CF5816ECEB0D6C5233D93199C052799120572FC5A50
    Session-ID-ctx: 
    Master-Key: 7987BDDB5BE27B556913507E0A21C5B60FA590503536E92152321866558CBDC6134C66086078A766AF1E89CED01FEBA5
    Key-Arg   : None
    PSK identity: None
    PSK identity hint: None
    SRP username: None
    Start Time: 1634489096
    Timeout   : 300 (sec)
    Verify return code: 10 (certificate has expired)
---

bingo600

@bingo600

Success

This was a "Ride into FreeBSD" & Certificates , and multiple cert files.

1:
fetch uses the cert file in :

/usr/local/etc/ssl/cert.pem

Most other uses the cert file in :

/usr/local/share/certs/ ca-root-nss.crt

Or the cert file in:

/etc/ssl/cert.pem

That is in fact symlinked to : /usr/local/share/certs/ca-root-nss.crt

/etc/ssl/cert.pem -> /usr/local/share/certs/ca-root-nss.crt

2:
Make fetch use the same cert file as the "system" via symlink

cd /usr/local/etc/ssl/
rm cert.pem
ln -s /usr/local/share/certs/ca-root-nss.crt cert.pem

Now a

ls -l /usr/local/etc/ssl/cert.pem

Should show

cert.pem -> /usr/local/share/certs/ca-root-nss.crt

3: Let's fix the expired certificate
Idea from here :
https://www.truenas.com/community/threads/ssl-certificate-problem-certificate-has-expired-the-openssl-1-0-2-vs-letsencrypt-issue.95874/

With your favorite editor open : /usr/local/share/certs/ca-root-nss.crt

Locate the below certificate : I searched for : DST Root CA X3

Now delete all lines beginning with : Certificate

Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number:
            44:af:b0:80:d6:a3:27:ba:89:30:39:86:2e:f8:40:6b
    Signature Algorithm: sha1WithRSAEncryption
        Issuer: O=Digital Signature Trust Co., CN=DST Root CA X3
        Validity
            Not Before: Sep 30 21:12:19 2000 GMT
            Not After : Sep 30 14:01:15 2021 GMT

And further down , where you will eventually see a BEGIN CERTIFICATE and a bunch of ASCII encoded binary garbage. Delete all of this, all the way through the following , including the below line

-----END CERTIFICATE-----

The total deleted section should be 81 lines.

Save the file , and bogons update should work again:

Test with:

/usr/bin/fetch -a -w 600 -T 30 -q -o /tmp/bogons https://files.pfsense.org/lists/fullbogons-ipv4.txt

That should run without errors.

Bogons will be in : /tmp/bogons

fetch was driving me crazy ...
openssl wasn't showing errors after the removal of teh bad certificate , and curl would get the file. But fetch still showed error .....

After lot's of headscratching i finally saw someone mention that fetch used its "own" cert file

Now that "they" symlinked one file why not the other ?????

/Bingo

Gertjan

@bingo600

Tell fetch what certificate file to use :

/usr/bin/fetch -a -w 600 -T 30 -q --ca-cert=/usr/local/share/certs/ca-root-nss.crt -o /tmp/bogons https://files.pfsense.org/lists/fullbogons-ipv4.txt

All certs in this file will be trusted.

bingo600

@gertjan
That won't work with the "Auto update of bogons"
They don't specify any cert file.

You will have to edit the cert file or (imho better) , symlink to the file the other programs uses.

/Bingo

Gertjan

@bingo600 said in Solved: Can't update bogons on a 2.4.5-p1 (cert expired):

That won't work with the "Auto update of bogons"
They don't specify any cert file.

Correct - this won't help the update script'(s).
Far better is correcting the needed files.

I posted the extra part "--ca-cert=/usr/local/share/certs/ca-root-nss.crt" so files could get loaded.
The /usr/local/share/certs/ca-root-nss.crt should be corrected manually, as more root certificates will expire in the future.

Btw : even when MS updates for Windows XP stopped many years ago, there were still updates : the files with system trusted certs.

JeGr

@gertjan said in Solved: Can't update bogons on a 2.4.5-p1 (cert expired):

Btw : even when MS updates for Windows XP stopped many years ago, there were still updates : the files with system trusted certs.

Yes, but you would only need that if you would stay in 2.4.5 - which isn't the normal/desired outcome, as - especially with a security product - we should update to the next stable version. So expiration of other CAs would only hit if you'd stay with 2.4.5 which isn't recommended / supported anyways and in 2.5.2 (latest current stable) the CA file should already be correct.

johnpoz

I concur with @jegr here - the actual solution to the problem is getting the pfsense current.

While updating the CAs trusted is a temp solution to a specific problem, it is only a stopgap measure at best.

To be honest my bogon being a bit dated is least of my worries on my older pfsense installs, that yes need to be updated when can actually get into the office, etc.

bingo600

@jegr
I'm not planning on staying on 2.4.5-p1 forever.
But given the "first track record" of the 2.5.x systems , i decided to wait a bit.
Ie. the early unbound issues would have been a "killer".

I do have one test site running 2.5.2 wo. any issues, it "just upgraded" ....
AKA keeps the L2L connection up wo. any dropouts.
But it doesn't see any load or usage, it's just a "passive system" at my desk.

But on the "Job" i have 1 Central pfSense and 6 remote (openvpn) L2L coupled sites.

My sites are spread around the world, and it would be a "Major issue" if they went down.
I have a "Cold spare" on my two most urgent sites, but they "never" found the time to upgrade the pfSense OS with me, o the "secondary". That is purely "manager politics", that i try to get around, but haven't succeeded yet.
So the failover systems aren't up to it.

My central unit has 1:1 (alias) NAT to several public "outside" IP's , and i'm a bit worried about that. I read that there was some NAT issues with 2.5.x , but maybe 2.5.2 has solved it , i don't know yet.
Any hints here ?.

I have a "Central Cold spare" i could wipe & install 2.5.2 on , and then give it the 2.4.5-p1 config. But i will "Not get a prize" if it doesn't work, even if it's just for 4..6 hours.

/Bingo

johnpoz

@bingo600 said in Solved: Can't update bogons on a 2.4.5-p1 (cert expired):

But i will "Not get a prize" if it doesn't work, even if it's just for 4..6 hours.

haha - I hear ya.. But you might be finding a new job ;)

While your solution is good - I personally would of just disabled bogon if was having issues with it. While sure its the "right" thing to do blocking it, in the big picture is not high on the list of security things to make sure your blocking..

bingo600

@johnpoz
Bogon's was for my own 2 pfSenses
And a ... I'm not giving up kinda moment.
I haven't even bothered implementing that "trick" on the Job ones ....

I have 2 x 240GB Samsung EVO-870 SSD disks , just waiting for me to install in the "home/summerhouse" pfSenses, along w. ZFS & 2.5.2.
I chose 240G because i plan to use ZFS snapshots for real, on 2.5.2.

I just have to find the right time, we're streaming TV here ....
And if you thought i was nervous of the "Job Boss" ...
That's nothing compared to the "Real BOSS"

And when done "home" , the the one in the summerhouse is next.

JeGr

@bingo600 said in Solved: Can't update bogons on a 2.4.5-p1 (cert expired):

And a ... I'm not giving up kinda moment.
I haven't even bothered implementing that "trick" on the Job ones ....

I appreciate it! I have some 2.4.5 systems in the wild myself that customers aren't able to update right now and those had rising numbers of dead/zombie processes (dying bogon procs) that we were able to fix that way - so thumbs up from me for the fact finding mission

johnpoz

@jegr said in Solved: Can't update bogons on a 2.4.5-p1 (cert expired):

rising numbers of dead/zombie processes (dying bogon procs)

Hmmm - interesting.. Curious bogon believe is only suppose to update every 30 days.. How many did they have? I wonder if it fails it start hammering looking to work more often than once every 30 days.

I just looked at 1 of my older installs

52 processes: 1 running, 50 sleeping, 1 zombie

bingo600

@johnpoz
They seem to "never die/timeout"
So you'll accumulate foe each month , and if you try to update manual that'll also start an additional.

Can't remember if each try starts 3 processes.
The php master + the fetch + "i think another"

/Bingo

bingo600

@jegr said in Solved: Can't update bogons on a 2.4.5-p1 (cert expired):

@bingo600 said in Solved: Can't update bogons on a 2.4.5-p1 (cert expired):

And a ... I'm not giving up kinda moment.
I haven't even bothered implementing that "trick" on the Job ones ....

I appreciate it! I have some 2.4.5 systems in the wild myself that customers aren't able to update right now and those had rising numbers of dead/zombie processes (dying bogon procs) that we were able to fix that way - so thumbs up from me for the fact finding mission

Glad to be able to give a little back

And ...
Now i know that to tomorrow on the job for 7 firewalls
Done ....

And home fwall
Fresh install w. ZFS , and config restoren only one minor "quirk"
iftop didn't install , but the pkgmgr. was informing about that

/Bingo