Well crap! I edited every cert I found with a X3 reference and it keeps coming back. It now only exists in the backup directory where I saved the unedited certs.
The strange thing is that not only is it still appearing in the FreeRadius chain, but also in the haproxy shared-frontend.pem.
What am I missing?
Maybe it's in the config.xml?
I made a copy of my config from /conf/config.xml at /root/backup
xmllint --xpath "//*/crt" /root/backup/config.xml > cert.base64
split -p "\n" ./cert.base64
Here is where I lost the will to script stuff out. All the certs are base64 encoded and there's no base64 tool installed and I was struggling with using openssl so I went back to basics.
cat xaa | perl -MMIME::Base64=decode_base64 -e 'print decode_base64 join"",<>' > xaa.b64
cat xab | perl -MMIME::Base64=decode_base64 -e 'print decode_base64 join"",<>' > xab.b64
cat xac ...
Now to see if any contain the dreaded and undying X3 cert
openssl crl2pkcs7 -nocrl -certfile /root/backup/xaa.b64 | openssl pkcs7 -print_certs -noout | grep -i x3
openssl crl2pkcs7 -nocrl -certfile /root/backup/xab.b64 | openssl pkcs7 -print_certs -noout | grep -i x3
openssl crl2pkcs7 -nocrl -certfile /root/backup/xac.b64...
2 of these files actually contain the damn X3 cert.
xaj.b64 contains it, so I just searched the config.xml for the contents of the xaj file and found it here:
<ca>
<refid>61563c252dbc8</refid>
<descr><![CDATA[Acmecert: O=Internet Security Research Group, CN=ISRG Root X1, C=US]]></descr>
<crt>LS0tLS1CRUdJ...S0tLS0=</crt>
<serial>0</serial>
</ca>
There was also another match I traced back to:
<ca>
<refid>61578b1bd6592</refid>
<descr><![CDATA[Acmecert: O=(STAGING) Internet Security Research Group, CN=(STAGING) Pretend Pear X1, C=US]]></descr>
<crt>LS0tL...tLQ==</crt>
<serial>0</serial>
</ca>
I grabbed the serial number of the certs with X3 using this tool and then manually went through my cert manager looking for anything that matches. I found them in my cert manager and deleted them and nothing has changed. FreeRadius and HAProxy arestill somehow finding the X3 cert. I even extracted all the certs again from the updated config.xml, now with fewer certs and yet it's still appearing.
