Outbound connection to AWS using Alias not working
-
Hi
We are trying to deploy a new EDR and it needs to connect through our PF sense to several FQDN's for deployment and management.
We restrict outbound internet access on all our servers so we needed to create an outbound allow rule for the EDR solution.
After creating the alias i check the table and can see it's populated with IP addresses but the rule does not pass outbound traffic to the vendor. after doing a few NS lookups we can see that the IP's associated with the FQDN's are changing almost every second, think they are are AWS CDN addresses. The Alias table is not updating fast enough to keep up with these changes.
Has anyone had this sort of issue before? Has anyone figured out how to work around the issue
-
@Rosstopher a short ttl on records can be problematic - if when the client looks up the fqdn the IP is different. what is one of the fqdn your trying to use. If you don't want to post public - could you PM me one of these fqdn that is changing every few seconds.
One work around could be to set a min ttl in unbound, so that even if the ttl is like 30 seconds, you use 1 hour for example. As long as your clients are using pfsense as their dns it should be talking to one of the IPs in the alias.
They normally do a round robin with these - and just because the IP the fqdn points changes to say B, doesn't mean that the one from 30 seconds ago (A) doesn't still work.
Another work around is allowing the network or asn even to where these servers are hosted.. It is not as locked down as specific IP.. but would be limited to the network the server your wanting to talk to is hosted on.
-
Hi John
Please find a couple of The URLs below, if you do an nslookup on your local machine you'll see how frequently they are changing.
huntress-installers.s3.us-east-1.amazonaws.com
huntress-updates.s3.amazonaws.com -
@Rosstopher said in Outbound connection to AWS using Alias not working:
huntress-installers.s3.us-east-1.amazonaws.com
yeah a 5 second ttl is insane..
set your min ttl to like 3600
then you get this - see how those are round robin, see all the different IPs.. But they don't really change - but depending you would get a different on as they change order as you do another query, etc.
;; ANSWER SECTION: huntress-installers.s3.us-east-1.amazonaws.com. 3599 IN CNAME s3-r-w.us-east-1.amazonaws.com. s3-r-w.us-east-1.amazonaws.com. 3600 IN A 52.217.126.18 s3-r-w.us-east-1.amazonaws.com. 3600 IN A 52.217.206.18 s3-r-w.us-east-1.amazonaws.com. 3600 IN A 52.216.89.232 s3-r-w.us-east-1.amazonaws.com. 3600 IN A 52.216.44.162 s3-r-w.us-east-1.amazonaws.com. 3600 IN A 16.182.109.162 s3-r-w.us-east-1.amazonaws.com. 3600 IN A 3.5.0.98 s3-r-w.us-east-1.amazonaws.com. 3600 IN A 52.217.126.50 s3-r-w.us-east-1.amazonaws.com. 3600 IN A 16.182.73.18Or you could prob just manually put those IPs into your alias.
-
Hi John
Apologies if this is a silly question but are you suggesting i change the TTL on the Firewall. If so what setting is that, please?
-
-
As @johnpoz said though, that will only work as long as the servers/applicatipns are using pfSense for DNS or course.
-
@stephenw10 yup true - doesn't work if client is using some external name server.
Also while it is normally not recommended to mess with the ttls of what the record owner set.. When they set insanely low ttls - F um!!
You can still round robin if you want to spread the love of which IP some client talks too - but 5 seconds is doing nothing but forcing a whole bunch of pointless dns queries..
I am fairly sure aws has the tech to properly load balance connections - there is zero reason to set a ttl like 5 seconds other than they want some sort of tracking is my only guess.
Or they want to inflate the numbers of how many dns queries they handle?
I have been running an altered min ttl for years and years - and I have not run into any issue where like some site doesn't work. But my dns does less lookups ;) there are plenty of sites that run really low ttls.. 5 seconds is insane - but see more with like 30, or 60 or 300 seconds.. Which ok 300 would be something you might put on a ddns record.. Or ok 60 seconds even on a ddns record.. But 5 seconds into some network like AWS, with their tech they sure shouldn't have to resort to old school dns round robin to share the load..
edit:
Is this some record you have control over? This point to some service your running in aws? Can you control the dns for this service - maybe its just the default ttl for when you bring up something new, and you can alter it on aws for your service? -
Hi.
It's not a record we have control over. It's a 3rd party EDR service and they host on AWS.
i have tried altering the settings suggested but with no success, traffic still hit the default deny rule.
-
@Rosstopher well look in your table for your alias - what is in there, and where is your client going?
You know that huntress fqdn is a cname right.
;; QUESTION SECTION: ;huntress-installers.s3.us-east-1.amazonaws.com. IN A ;; ANSWER SECTION: huntress-installers.s3.us-east-1.amazonaws.com. 0 IN CNAME s3-r-w.us-east-1.amazonaws.com. s3-r-w.us-east-1.amazonaws.com. 0 IN A 16.182.69.226 s3-r-w.us-east-1.amazonaws.com. 0 IN A 52.216.54.234 s3-r-w.us-east-1.amazonaws.com. 0 IN A 3.5.19.248 s3-r-w.us-east-1.amazonaws.com. 0 IN A 3.5.8.69 s3-r-w.us-east-1.amazonaws.com. 0 IN A 52.217.34.40 s3-r-w.us-east-1.amazonaws.com. 0 IN A 3.5.31.253 s3-r-w.us-east-1.amazonaws.com. 0 IN A 52.217.161.34 s3-r-w.us-east-1.amazonaws.com. 0 IN A 52.216.221.90So you need to validate your table lists the IP of where your client is actually trying to go.
You sure your client is using unbound on pfsense as its dns.. Do a query - what ttl do you get, and what IP?
Shoot that other one is a chained cname - which is bad practice
;; ANSWER SECTION: huntress-updates.s3.amazonaws.com. 0 IN CNAME s3-1-w.amazonaws.com. s3-1-w.amazonaws.com. 0 IN CNAME s3-w.us-east-1.amazonaws.com. s3-w.us-east-1.amazonaws.com. 0 IN A 3.5.3.53 s3-w.us-east-1.amazonaws.com. 0 IN A 52.217.128.105 s3-w.us-east-1.amazonaws.com. 0 IN A 52.216.213.57 s3-w.us-east-1.amazonaws.com. 0 IN A 52.217.143.49 s3-w.us-east-1.amazonaws.com. 0 IN A 52.217.168.97 s3-w.us-east-1.amazonaws.com. 0 IN A 3.5.16.26 s3-w.us-east-1.amazonaws.com. 0 IN A 52.217.233.1 s3-w.us-east-1.amazonaws.com. 0 IN A 16.15.185.42 -
Also verify the server is actually resolving against pfSense and doesn't have some hard coded DNS built in.
