Domain names in URL Table Aliases

[RobertK 1]

Hi,

I can't seem to find details on using domain names in URL Table Aliases. The documentation says that IPs & subnets can be used in such Alias lists. However domain names seems to work as well. Now my question is how often those domain names inside of a URL Table Alias are resolved? In a normal Host alias DNS names are resolved in every 5 minutes. So is it possible that DNS names are resolved in every 5 mins in a URL Table Alias as well, or is done every time when the URL Table itself is updated (once in every 1-128 days according to the URL Table setting)? Do you have any insights on this?

Thanks,
Robert

[johnpoz]

@robertk-1

The alias is built from the content returned by the specified URL but is updated by fetching the list from the URL periodically.

If you use a url table - then yes it is fetched per your setting in the alias for how often it should grab the table - if that table has updated entries would be up to who maintains the table.

[RobertK 1]

Hi @johnpoz, what I'm wondering is when I have a DNS name in the contents of the file retrieved from the URL Table URL address in every 1-128 days, so those DNS names inside the list, how often are they resolved? Are they resolved once every time when the URL Table is updated? Or are they resolved in every 5 mins? Or perhaps as a worst case scenario they are resolved only once, when they were added first time to the URL Table?

[Gertjan]

@robertk-1 said in Domain names in URL Table Aliases:

Or are they resolved in every 5 mins?

Close.
It's this one : Services > Cron > Settings, and look for the line :

[RobertK 1]

Thanks guys for the heads up, it seems that having DNS names in an URL Tables Alias is not a good idea, there is nothing that keeps the resolutions up to date between URL Tables updates, maybe modifying the crontab line like this:

*/5 * * * * root /usr/bin/nice -n20 /etc/rc.update_urltables now forceupdate

...but it's quite nasty of a patch :)

[Gertjan]

@robertk-1

/5 ?
Because a host name like whatever dot somewhere dot tld changes it's IP every 5 minutes ?
If that was the case, you shouldn't even wanted to try to keep up with it.

The DNS alias resolution, to be usd in a firewall rule, is fine for your own web domain name (normally it has a static IP anyway), or an access to your daily changing WAN IP, so you can openvpn to home using a host name, not an always changing IP. Stuff like that.

It should never be used to 'try to' list the IPv4 of
youtube/Microsoft/apple/google/facebook.twitter/etc/etc as these use thousands of IPv4 for the same host name.

The DNS used on the internet, the one that handles publicly visible web services, that changes every 5 minutes it's IP, should be banned as it breaks the entire DNS chain.
Again : this has nothing to do with the 'CDN' or one host name points to 'many IPv4' concept.
You just can't list "youtube.com" in an URL table, and then think it will get resolved to the Youtube IP, so with one easy rule you block youtube access.

[RobertK 1]

It would have been used for whitelisting some cloud services used on many sites, nothing big like some Google or Microsoft services. My idea was use the same URL Table pulled from an internal webserver for this, but I really don't like that the firewalls are not being able to pull any changes in the DNS records in a few minutes. Those records are not changing all the time, however there are changes now and then and it would be nice to follow those changes in a 5 min window. Now I'm using "Host" aliases on all firewalls for this, it lacks central mgmt of the list but works.

[Gertjan]

@robertk-1 said in Domain names in URL Table Aliases:

...but it's quite nasty of a patch :)

Instead of changing that cron tab line, what about adding your own ?

I've just added a test URL, one of my own domains :

and it was resolved right way.
I has only one IPv4 and IPv6 :

[johnpoz]

@gertjan he is talking about the table url - where a table of IPs/Names is loaded. I have never looked into if your table loads www.domain.tld how often that is then resolved.. Other than when the table is pulled.

If I get a chance today I will do a test of loading some fqdn that I can change - one of my public domains and see if once the table is loaded and I change it - when it has a short ttl if it gets updated.

edit: is table even able to do fqdn? Per the notes on table url it states IP or subnets - not FQDNs

edit2: ok - so you can put a fqdn in the url table.. And it does populate when the table is loaded.

So I created a dns record on my public dns that resolved the fqdn to 4.5.6.7 with a 1 min ttl. If I look i the table it is there

Now it been well over 5 minutes.. and table has not updated.. but if I ask pfsense to look it up - it lists the new IP I changed it too in public dns

if I then look at the table - it shows the new IP..

So it does look like a fqdn listed in table will get updated, as the ttl expires on that record and it is looked up again.

[RobertK 1]

Hi,

@johnpoz said in Domain names in URL Table Aliases:

is table even able to do fqdn? Per the notes on table url it states IP or subnets - not FQDNs

Exactly. Yet when the table is pulled the one that shows up in /var/db/aliastables contains resolved IPs.

[johnpoz]

@robertk-1 see my completed edit - on what happens when the ttl has expired on that fqdn and pfsense is asked again to look it up..

I didn't redo the table or anything - just did a dns query for the fqdn that is in the table.