Having problems with Firewall Aliases URLs not working since update to 2.4.5-p1

IsaacFL

Instead of updating my pfsense from 2.4.5 to 2.4.5p1, I reinstalled my pfsense router today using my old config.xml and now my URL table Aliases are not working.

Here are the Aliases from my old config:

		<alias>
			<name>GEOIP_CN_v4</name>
			<type>urltable</type>
			<url>https://www.ipdeny.com/ipblocks/data/aggregated/cn-aggregated.zone</url>
			<updatefreq>1</updatefreq>
			<address>https://www.ipdeny.com/ipblocks/data/aggregated/cn-aggregated.zone</address>
			<descr><![CDATA[China IPs]]></descr>
			<detail><![CDATA[China]]></detail>
		</alias>
		<alias>
			<name>GEOIP_CN_v6</name>
			<type>urltable</type>
			<url>https://www.ipdeny.com/ipv6/ipaddresses/aggregated/cn-aggregated.zone</url>
			<updatefreq>1</updatefreq>
			<address>https://www.ipdeny.com/ipv6/ipaddresses/aggregated/cn-aggregated.zone</address>
			<descr><![CDATA[China IPs]]></descr>
			<detail><![CDATA[China]]></detail>
		</alias>
		<alias>
			<name>GEOIP_RU_v4</name>
			<type>urltable</type>
			<url>https://www.ipdeny.com/ipblocks/data/aggregated/ru-aggregated.zone</url>
			<updatefreq>1</updatefreq>
			<address>https://www.ipdeny.com/ipblocks/data/aggregated/ru-aggregated.zone</address>
			<descr><![CDATA[Russian IPs ]]></descr>
			<detail><![CDATA[Russian IPs]]></detail>
		</alias>
		<alias>
			<name>GEOIP_RU_v6</name>
			<type>urltable</type>
			<url>https://www.ipdeny.com/ipv6/ipaddresses/aggregated/ru-aggregated.zone</url>
			<updatefreq>1</updatefreq>
			<address>https://www.ipdeny.com/ipv6/ipaddresses/aggregated/ru-aggregated.zone</address>
			<descr><![CDATA[Russian IPs]]></descr>
			<detail><![CDATA[Russian IPs]]></detail>
		</alias>

I can go my way of browser to the links and it looks like what I would expect.

Here is the error message I see in the logs.

Jun 17 12:36:13 	php-fpm 	95445 	/rc.update_urltables: : ERROR: could not update GEOIP_RU_v6 content from https://www.ipdeny.com/ipv6/ipaddresses/aggregated/ru-aggregated.zone
Jun 17 12:36:12 	php-fpm 	95445 	/rc.update_urltables: : ERROR: could not update GEOIP_RU_v4 content from https://www.ipdeny.com/ipblocks/data/aggregated/ru-aggregated.zone
Jun 17 12:36:12 	php-fpm 	95445 	/rc.update_urltables: : ERROR: could not update GEOIP_CN_v6 content from https://www.ipdeny.com/ipv6/ipaddresses/aggregated/cn-aggregated.zone
Jun 17 12:36:12 	php-fpm 	95445 	/rc.update_urltables: : ERROR: could not update GEOIP_CN_v4 content from https://www.ipdeny.com/ipblocks/data/aggregated/cn-aggregated.zone

I finally had to delete the aliases and associated rules.

Anybody seen this or could explain what I have done to break them?

Everything else works fine. I am installed on a Hyper-V VM.

Version 2.4.5-RELEASE-p1 (amd64)
built on Tue Jun 02 17:51:17 EDT 2020
FreeBSD 11.3-STABLE

viktor_g

@IsaacFL Successfully added https://www.ipdeny.com/ipblocks/data/aggregated/cn-aggregated.zone to my fw aliases on 2.4.5-p1

or is this error only occurring on updates?

IsaacFL

@viktor_g
It occurred on the update, but even after deleting them, I can't recreate them.

I always get:

The following input errors were detected:

    Unable to fetch usable data from URL https://www.ipdeny.com/ipblocks/data/aggregated/cn-aggregated.zone

At least I know it is on my end.

viktor_g

are you able to fetch it manually from pfSense?

# fetch  https://www.ipdeny.com/ipblocks/data/aggregated/cn-aggregated.zone

in command line

IsaacFL

@viktor_g said in Having problems with Firewall Aliases URLs not working since update to 2.4.5-p1:

fetch https://www.ipdeny.com/ipblocks/data/aggregated/cn-aggregated.zone

This is the error I got:

Certificate verification failed for /C=SE/O=AddTrust AB/OU=AddTrust External TTP Network/CN=AddTrust External CA Root
34374270280:error:14090086:SSL routines:ssl3_get_server_certificate:certificate verify failed:/build/ce-crossbuild-245/sources/FreeBSD-src/crypto/openssl/ssl/s3_clnt.c:1269:
fetch: https://www.ipdeny.com/ipblocks/data/aggregated/cn-aggregated.zone: Authentication error

IsaacFL

When I open the link on a browser, it says the certificate is valid. Both Edge Chromium and Firefox.

IsaacFL

Changing https to http does allow it to work.

Seems to be an issue with verifying the certificate?

IsaacFL

I do have Check certificate of aliases URLs Enabled.

Verify HTTPS certificates when downloading alias URLs Make sure the certificate is valid for all HTTPS addresses on aliases. If it's not valid or is revoked, do not download it.

Not sure when I did that.

IsaacFL

fetch https://www.spamhaus.org/drop/dropv6.txt
yields:
dropv6.txt 1068 B 14 MBps 00s

So it works for some sites.

bmeeks

I'm not a pfBlockerNG user, but I've seen this particular problem posted about previously, and the cause is the expired AddTrust certificate. There are apparently two solutions. One involves finding and manually deleting that expired certificate in the pfSense CLI. The other solution used by some was to change the URL mode in pfBlockerNG so that the cert is not validated. While not the most secure way of doing things, it was a "working" workaround for the folks posting.

IsaacFL

@bmeeks I am not using pfBlockerNG either.

Just trying to download Alias/URL Table.

Does get me something google, but for now I just changed my urls from https to http. Which to be honest it isn't needed to encrypt a list of ips.

I assume there will be a fix at some point.

viktor_g

@IsaacFL ipdeny.com https server is misconfigured and is offering an expired CA certificate (AddTrust) in the chain.

you can try this workaround: https://redmine.pfsense.org/issues/10616#note-3

more about AddTrust expriration issue: https://www.ssl.com/blogs/addtrust-external-ca-root-expired-may-30-2020/

IsaacFL

@viktor_g

Once I read about the cert issue, and that it is an external issue, I decided to just use the http (80) link to ipdeny at least for now.

I am not concerned about the country ip list being encrypted and figure they will probably fix it at some point.