So Many IPs - How to get them all?
-
I am trying to allow access to a TV Set Top Box but only to what it needs for the IP TV to work. After wiresharking the full list of dns queries it makes to get what it needs, I added those to an Alias. The difficulty I ran into is that they list of IPs changes. Sometimes, every time you query it returns a different set of IPs.
How do I create a persistent list of all of the variations I encounter?
This is an android device so it talks to google, amazon, and a few others to maintain functionality of the installed apps.
I've been adding IPs for days. Just when I think I have it, it offers up new IPs and then it quits working.
Any advice would be greatly appreciated.
-
@Blachawk I think you're going to be chasing a moving target with this one... Why don't you simply isolate this STB on it's own VLAN, and let it do it's thing, no blocking or filtering, except from/to your other internal private LANs?
Jeff
-
Its TV Only. I have to block it from accessing the internet through browser apps and youtube. I intend to add a captive portal also with a page telling them to call to update their service for internet access.
Yes, there are still people not on the internet. Go figure!
-
@Blachawk said in So Many IPs - How to get them all?:
talks to google, amazon, and a few others to
These two (6 actually) decide what you see - and when. And how often.
I'm talking about the added 'free' content of course : mostly the adds. But non-add content can also be shown or hidden, everything depends on your profile (you know, the profile they have about you that you can't access - and they deny it's existence).These guys have entire AS "blocks", give or take a couple of million of IPv4 over hundreds of data centers, and they start them up, and shut others down as they which, when they which. For logical reasons, of course : maintenance, upgrades, usual broken devices, etc etc. The big ones do not amuse themselves to "upgrade in place" or something like that. Entire server farms are taken down, and brought up again.
Most apps (and phones, etc) download lists with actual IP addresses to use. These lists change every time.
Or the classic DNS system is used : A, AAAA and MX records change all the time.What you could do : look up what an AS is.
Are you still there ?
Ok.
Now, for the big players, look up there AS numbers, and thus you find their entire networks. These are /24 chunks or far bigger. I believe Apple already own en entire /8 .....
Make your own alias with these networks.I did all this some years ago, and found the perfect firewall rule, I was even able to remove all my aliases :
( and then I remembered that pfSense uses this rule as a default LAN rule....... months of work down the drain ... for nothing - but I learned something,)
Btw : TV's etc are normally not considered as "risky" devices. They have no open ports, processes can't (normally) be hacked, it's difficult to upload your hand made app .... TVs do not ask for 'private' data like credit card numbers, passwords etc. They do not store any data.
If you really want a TV without questions, risks and issues : ask for a TV that doesn't mention "smart" on the box. If it has no Wifi or ethetnet plug in the back, you have a 'safe' TV.
Remember : you are trying to trace what a boatload of engineers developed for years. They always will be miles ahead of you.
-
Let me clarify!
I am providing an IPTV only package to my internet clients. Analog is going away and I won't force someone to buy internet if they really don't want it.
This Android IPTV Box has my IPTV app and a few others that it comes with by default. A browser, Youtube, Play Store (We want this one to update the apps). I need to allow access to the key items that it requires to function.an example is: connectivitycheck.gstatic.com
This one doesn't change much but it does change.and then there is b562xyterwthy-ats.iot.us-east-1.amazonaws.com
This one changes almost every time its queried. Its a single client app that is required but its on amazonaws.com.Adding a FQDN alias lets pfsense query periodically. I am just looking for a way to make the list of IPs persist between queries rather than clearing and refreshing each time.
-
@Blachawk said in So Many IPs - How to get them all?:
Let me clarify!
I am providing an IPTV only package to my internet clients. Analog is going away and I won't force someone to buy internet if they really don't want it.
This Android IPTV Box has my IPTV app and a few others that it comes with by default. A browser, Youtube, Play Store (We want this one to update the apps). I need to allow access to the key items that it requires to function.an example is: connectivitycheck.gstatic.com
This one doesn't change much but it does change.and then there is b562xyterwthy-ats.iot.us-east-1.amazonaws.com
This one changes almost every time its queried. Its a single client app that is required but its on amazonaws.com.Adding a FQDN alias lets pfsense query periodically. I am just looking for a way to make the list of IPs persist between queries rather than clearing and refreshing each time.
You will likely never get this to work like you want to because of the use of CDNs (content delivery networks). Between that technology and Load Balancers, the IP addresses will be contantly changing. As you have noticed, even two consecutive DNS lookups for the same domain name yield different IP addresses. This is by design with CDN and Load Balancing systems. A FQDN alias can't keep up either. The firewall does not perform a DNS lookup on every packet. It simply runs a cron task on a 5-minute interval and updates the IP addresses then. That is probably not often enough for many CDNs.
Take the advice others have given you and allow the box to freely communicate with the Internet. I can't see how that is a problem for you unless you are doing something with the box you don't want the provider of the IPTV programming to know about. You can secure it by putting it within its own VLAN if you want to isolate it from your private LAN.
-
Thank you all for your responses. It does seem to be a daunting task. Nothing nefarious going on. We are dropping analog TV as are many providers, to free up frequency for OFDM with the new Docsis 3.1. We still have many TV only customers and telling them they have no choice but to purchase internet now is not very palatable. In some areas there are even franchise issues that dictate we provide a TV only package. Also giving them Internet for free isn't the worst thing we could do but it would be better to give what's paid for and offer an upgrade path if they want it.
I may indeed have bitten off more than I can chew but who doesn't enjoy a good project? It appears that I will have to break out wireshark again and look closer at port use rather than IP.
Again, Thanks for responding.
-
@Blachawk said in So Many IPs - How to get them all?:
Thank you all for your responses. It does seem to be a daunting task. Nothing nefarious going on. We are dropping analog TV as are many providers, to free up frequency for OFDM with the new Docsis 3.1. We still have many TV only customers and telling them they have no choice but to purchase internet now is not very palatable. In some areas there are even franchise issues that dictate we provide a TV only package. Also giving them Internet for free isn't the worst thing we could do but it would be better to give what's paid for and offer an upgrade path if they want it.
I may indeed have bitten off more than I can chew but who doesn't enjoy a good project? It appears that I will have to break out wireshark again and look closer at port use rather than IP.
Again, Thanks for responding.
Ah, so you are a Cable Television/Internet provider. That was not 100% clear from your previous posts. Sounded more like an apartment complex setup or something.
