Discussion: resolving hostnames for unpublished local IPv6 addresses.
-
Can we take another run at this? I have a pfSense system with 4 virtual windows 10 clients. They all have 4 ipv6 addresses. See below for an example. In all cases, the blue box is the prefix, even the dns.
The first address is the one allocated by pfSense using dhcp6. (It shows up in the leases.)
The second address is constructed like the link-local address, except using the prefix instead of fe80::.
The third address is the one that shows up using ipv6-test.com or other website.Every time I disable and enable the network interface, the temporary address changes. The others stay the same.
It would be nice if all of the above addresses resolved to a consistent hostname.
 -
I haven't abandoned this… It's something I'll do (one way or another) but I was hoping that people more fluent with administrating the overall system could comment on my proposed solutions.
My biggest concern would be how my proposals might break things.
-
Another possible (even if unlikely) scenario where my proposal would mess up:
MachineA gives itself a "privacy extension" ipv6 address of, for example, "prefix:1:2:3:4" My stuff detects that and causes reverse lookups for prefix:1:2:3:4 to show a host of "MachineA"
After some time, MachineA is turned off. MachineB then comes up with the exact same randomly generated privacy extension address: "prefix:1:2:3:4" (MachineB would use various IPv6 mechanisms to verify that nothing else on the local network is using prefix:1:2:3:4 at that moment.)
At this point, the reverse lookup for prefix:1:2:3:4 will still give a name for "MachineA" and will continue to do so until the pfsense box "sees" MachineB using that address (which would update it's NDP tables), AND my stuff runs to update its own tables.
The chances of this happening are.. well.. slim. Something like… 1 in (2^64 - (n + 1)) where "n" is the number of other IPv6 addresses already "claimed" in the prefix. To be clear, "2^64" is 18,446,744,073,709,551,616. That's a really, really, really big number.. and this would ONLY be a temporary error, would only impact the internal network, and only REVERSE lookups (which usually aren't used for anything other than cosmetic "human readable" things.)
Based on those odds, I'm sure it'll happen to me, personally, at least a dozen times a day. ;)
-
Still trying to understand the use case here.. Windows boxes will not register temp ipv6 address in dns. But they will register their perm global or ULA address.. This is how you would talk to them.. The whole point of temp addresses changing and not using mac in creating the temp is "privacy" hehehe
If you do not want clients doing that then turn off that feature on your clients. If the clients are not under your control then their "privacy" feature of temp addresses is working as designed if your having a hard time tracking them ;)
The whole point of temp ipv6 address is client privacy for outbound sessions. You trying to track them defeats the whole purpose of it.. So still trying to understand the use case.. If devices on the network are yours this should never be an issue just disable the feature on your clients. If the devices do not support it like some iot or phone or whatever that does not have a full OS or one that allows such administration then you for sure would know its mac and could assign it an IP to use based upon that mac that you use for admin.
So is the concern just traffic the devices network usage? The switch port or the wifi its connected to will show you its usage. Or use a proxy to log network traffic volume/dest by the user vs the IP they are using. If your trying to track down a device that logging stuff in your firewall after the fact then yeah you have a bit of a problem if logged via some temp ip address doesn't matter if ipv4 or ipv6 in this case.. With ipv4 you could use arpwatch to keep an eye on your ip to mac pairs for all kinds of reason. With ipv6 there was ndpmon or addrwatch prob what your looking for.
https://github.com/fln/addrwatch vs trying to reinvent the wheel ;)
Main features of addrwatch:
- IPv4 and IPv6 address monitoring
- Monitoring multiple network interfaces with one daemon
- Monitoring of VLAN tagged (802.1Q) packets.
- Output to stdout, plain text file, syslog, sqlite3 db, MySQL db
- IP address usage history preserving output/logging
Addrwatch is extremely useful in networks with IPv6 autoconfiguration
(RFC4862) enabled. It allows to track IPv6 addresses of hosts using IPv6
privacy extensions (RFC4941).I do not believe there is a freebsd package for addrwatch, but there for sure is for ndpmon - so are what your looking for is to get this added as base feature of pfsense or as a package?
-
Still trying to understand the use case here.. Windows boxes will not register temp ipv6 address in dns. But they will register their perm global or ULA address.. This is how you would talk to them.. The whole point of temp addresses changing and not using mac in creating the temp is "privacy" hehehe
Agreed - to some extent. It's for global privacy, not really local network privacy. In fact, using the manual process I described in the OP, it's easy to determine which specific machine a temp IPv6 belongs to (as long as your on the same local network.)
The whole point of temp ipv6 address is client privacy for outbound sessions. You trying to track them defeats the whole purpose of it.. So still trying to understand the use case.. If devices on the network are yours this should never be an issue just disable the feature on your clients. If the devices do not support it like some iot or phone or whatever that does not have a full OS or one that allows such administration then you for sure would know its mac and could assign it an IP to use based upon that mac that you use for admin.
It's not always easy (or even desirable) to turn off privacy extensions. It DOES serve the purpose of providing some (questionable) level of "privacy" on the global level. As well, unless all the machines are using some kind of central control mechanism (such as windows ADS pushing a GPO), it's difficult to sit down at each machine and manually disable privacy extensions. In addition, as you mentioned, there are devices that don't allow that type of administration.
If I can control the IPv6 of a device, and get the reverse PTR in DNS, then it's not a problem. Those devices aren't really relevant to this thread.
I'm looking to deal with "less controlled" devices. Yes, those occur in (as you mentioned) "wild west" or BYOD networks. They aren't limited to those environments, however.
As for why do I care?
I'll give an example, but please remember that this is just ONE example. There are others… Please don't pick apart the example. It's only an example. The point of the example is reverse IPv6 DNS to unregistered IP addresses. (In fact, I know there are at least 2 flaws in the example.)
<begin example="">* * *
John Doe (not related to Johnpoz ;) ) uses ntopng to display traffic data (because it's the only "official" package he has found so far that shows per client traffic AND does some kind of deep packet inspection (L7) to give an idea of what traffic is used for.) His intention is to see how and by whom the "internet" is being used from the internal network.
Mr Doe sees that a very large amount of traffic was using IPv6 addresses. A large chunk of that traffic is labeled as "youtube streaming", "microsoft.http", etc. (The point being that a very significant amount of traffic is occurring via ipv6 if it's available.)
Because ntopng uses reverse DNS lookups to resolve IP addresses, and because most of his machines use privacy extensions, ntopng can't resolve the "privacy extension" IPv6 addresses to nice hostnames/FQDN's that John Doe assigns to his machines.
John Doe is unable or unwilling to disable IPv6. John Doe is also unable to unwilling or unable to disable privacy extensions.
John Doe is left not really understanding which machines on his network are using what traffic.
<end example="">@johnpoz:
So is the concern just traffic the devices network usage?
In this example, gathering statistics of network traffic usage of the device… but not just knowing how much traffic, but also getting more detail on it.
The switch port or the wifi its connected to will show you its usage.
To some extent, yes. Perhaps more capable switches (I don't know) would be able to associate, for example, a netflow with a specific port (instead of a specific address), but if you insert a single unmanaged (or less capable) switch, then it breaks down. Less capable switches (even cheaper managed L2 switches) are limited to "port x used y packets/z bytes" type of reporting.
Or use a proxy to log network traffic volume/dest by the user vs the IP they are using.
That is something I never considered. If a proxy isn't already in use for some other reason, wouldn't inserting a proxy to the network create more overhead? (I also only ever considered a proxy for http/https traffic - not for other things.) How would the proxy identify the traffic as coming from a specific user? Would the user have to "log in" to the proxy before engaging in any other traffic? How does the proxy handle temporary IPv6 addresses? Would the user have to re-login each time their temp address changed?
With ipv6 there was ndpmon or addrwatch prob what your looking for.
Ah! That's the type of thing I was hoping to get in response to this thread! Thank you. I'll have to look into these things.
I don't care if there's already a freebsd package for the tools or not. If they resolve the problem, and they can be ported, I'll port them myself and submit github pulls as needed. If the tools don't exist, I'll create them.
FYI, my "solution" wasn't to create a monitoring tool per se. I'd prefer to let specialized monitoring tools do the monitoring, and I'd just tackle the problem of resolving "privacy extension" and other non-DNS registered IPv6 addresses.
My premise is that if I could discover the unregistered IPv6 addresses that other tools would encounter, and find hostnames for those unregistered addresses, I could simply register them locally. This would allow the monitoring and reporting tools to get hostnames for those IPv6 addresses, and allow any other tools to also resolve them using the reverse DNS lookups that they already use.
I don't know what "any other tools" might be… Personally, I don't imagine that there'd be much use for this type of thing beyond reporting and monitoring tools. What else really uses reverse DNS lookups? They only exist for us humans to see nice FQDNs instead of network format addresses.
BTW, and one thing to keep in mind when replying:
I don't have the same mindset as a network engineer or administrator. I've been doing software development for nearly 30 years now, and that certainly shapes how I think. To me, limitations aren't boundaries, but challenges to be overcome. Instead of allowing my perception of a computer/network/environment to be shaped by what is available, I just change availabilities.</end></begin>
-
I just (accidentally) stumbled on to another example where something like this might be useful: (this isn't exactly what I'm proposing above, but what I'm proposing could easily be extended to fix the below example.)
The creation of firewall aliases within pfsense….
Silly Example:
I want to block the machine named "George" (FQDN george.network.lan) from ever using TCP port 69 outbound. (Obviously, this isn't a real example.) I create a "host" alias for "george" with the FQDN of "george.network.lan", and then set up some firewall rules that block TCP, source port 80, for source host alias "george"
The machine "george" has privacy extensions enabled. George likes that. George is a DIRTY old man. ;) (That's why I have to block port 69.)
Sadly, the firewall rule isn't working, because when pfSense resolves the alias for "george.network.lan", it doesn't seem to pick up the privacy extension IPv6 (which, ironically, is the IP address that "george" uses when generating outbound traffic.) So, that nasty george is still shoving his traffic out to the internet via port 69.
....
While my proposal was intended only to address reverse lookups, it could easily be extended to facilitate this specific aspect of pfsense aliases. (Give it a FQDN, or even a single IP address, and it could ferret out all IP addresses, both IPv4 and IPv6, that the router knows about which belong to the same physical machine -- assuming they are on a local network.)
-
With ipv6 there was ndpmon or addrwatch prob what your looking for.
Yes, those do something similar. Actually, they do significantly more, but appear to be designed as stand-alone tools. They could probably be greatly simplified to just do the basic thing I'm looking for, and then adding something that would pollute reverse DNS lookups.
(I'm not trying to re-invent those wheels. I'm trying to write something simple that will allow OTHER programs to function better. In a vacuum, the tool I'm doing is useless.)
-
Your example of blocking george to 69 is only viable when you only want to block George and allow everyone else.. Or you just block 69 an be done with it.. Users that want to use 69 need auth to the proxy, etc..
"John Doe is unable or unwilling to disable IPv6."
John Doe is not controlling his network and should not be working in networking ;) If John Doe does not like or understand all the ipv6 traffic an can not control such traffic then John Doe disables IPv6 that is how it would be done in any corp setup plan and simple. While your examples are good for what your trying to do they fall apart for use case that makes any sense.
If John Doe does not want youtube streaming then he blocks youtube streaming, or forces users wanting to use youtube streaming to auth to the proxy so John Doe knows who is using youtube streaming and how much, etc.
If your seeing ipv6 traffic you do not want the sure fireup ndpmon and you will know what the machines are an all the ipv6 they use, etc. But from a normally security/network point of view.. If your seeing unwanted traffic you just block said traffic.. Users comes to you if they need said traffic ;)
Trying to watch all the traffic and understand all the traffic is going to make John Doe crazy ;) If your worried about bad traffic run ids/ips..
-
Your example of blocking george to 69 is only viable when you only want to block George and allow everyone else.. Or you just block 69 an be done with it.. Users that want to use 69 need auth to the proxy, etc..
I think I mentioned earlier that I never considered proxies. Of course, its a completely fabricated example anyway. I was just trying to give an example of a situation where a pfsense alias might not work as expected. (In particular, when an alias is used for the "source" host on traffic destined outside the local network and ipv6 privacy extensions are enabled.
"John Doe is unable or unwilling to disable IPv6."
John Doe is not controlling his network and should not be working in networking ;) If John Doe does not like or understand all the ipv6 traffic an can not control such traffic then John Doe disables IPv6 that is how it would be done in any corp setup plan and simple. While your examples are good for what your trying to do they fall apart for use case that makes any sense.
That's not a fair statement… I specifically asked not to pick apart the examples, as they are ONLY examples. My intention for giving examples is to help people to understand the problem, but you're dissecting the examples to find fault with the examples themselves.
Look back at this thread, and you'll see that you've consistently attacked the examples I've given. You've suggested that the examples are invalid because IPv6 privacy extensions should be disabled, android and iOS devices shouldn't be allowed on the network, BYOD should be banned, every single network access should have to go through a proxy, and so on. All that just so a person (or utility) can get a hostname when doing a reverse lookup. (No, sorry.. not "all that." That's an exaggeration. Some of those things can work without the others.)
You seem to be also assuming that because you, personally, have no need or issue doing reverse lookups on local ipv6 addresses, that doing so would be useless and that anyone who DOES have a need/desire and lack of ability to resolve IPv6 addresses to names must be doing something wrong. IF that's a true statement, that's great for you.
The point here is that I'm not trying to solve security issues. I'm not even trying to solve reporting issues, alias issues, or any other "end" task. There already exists quite a few options for those things already. I'm also not even asking "how do I solve this?"
It's a tool to be used by other things (directly or indirectly), nothing more. Not even a "utility" because it'd be useless by itself. Perhaps, in my efforts to try and explain the problem, I went overboard with examples that caused confusion. If so, please accept my apologies for inducing the confusion and accompanying turmoil.
I thought I had made clear that those were all just examples that can show symptoms of what I AM trying to solve:
I'm just trying to solve that "reverse DNS lookups for local IPv6 addresses might not resolve to a hostname even though the machine had enough information to derive a hostname."
Though I've tried to give examples, it doesn't really matter why a person or piece of software might want to do the reverse lookups. As well, though I've tried to give examples, it also really doesn't matter why PTR records aren't there.
Trying to watch all the traffic and understand all the traffic is going to make John Doe crazy ;)
What's wrong with a little craziness? Do you really think the people who create the software for operating systems, network utilities, firewalls, or anything else are completely sane?
heh…
-
(please feel free to ignore this post. I'm mostly just talking to myself here…)
In wondering what is the best way to "pollute" DNS reverse lookups, I've come up with four ways that appear compatible with a default pfsense configuration. (This doesn't deal with expiring entries, etc.)
1. append to /var/etc/hosts (which is linked to by /etc/hosts.) Pros: This is simple, and compatible with everything. Cons: it would only work for resolving on the local machine; it would resolve FORWARD lookups as well as reverse, and it might create problems with dhcpleases and dhcpleases6 also overwriting the file. (I think dhcpleases and dhcpleases6 are already clobbering each other... adding another process would just make it worse.)
The next 3 ideas all involve changing unbound behavior and all share the same problem that they won't work if unbound isn't being used.
2. have unbound.conf include another config file, and dedicate that file to PTR lookups from this process. It would just be a bunch of "local-data-ptr:" lines. The TTL would have to be very low in order to hint that clients shouldn't cache the results. Pros: Almost as simple as /var/etc/hosts updates, without having to compete with other processes to write to the file. I don't see any negatives other than the requirement of using unbound. Only question becomes how to force unbound to re-read it's config files. (likely something like sending a HUP signal.)
3. using unbound-control to dynamically create and update a reverse lookup zone. Like #2, the TTL would have to be very low. This is a cool thought, but when compared to #2, it doesn't make any sense.
4. use a "python module" (unbound config option "python-script") to inject reverse lookup results. This is probably one of the coolest ideas. Unlike #2 and #3, I could dynamically adjust the TTL to match the NDP record expiration at the moment the request is made. However, it requires that unbound is compiled with the script module enabled, and probably adds a considerable amount of complexity when compared with #2. (I might do this just for the experience at some point, but not actually release it.)
...
#2 seems like the best solution. It does require that unbound is being used, but several other aspects of pfsense have a similar requirement (such as the dhcp servers adding leases to DNS.)
-
(I'm really sorry that I'm (ab)using this forum thread for keeping notes, but I only have itty bitty slices of time here and there to work on this, and I'm not always someplace where I can keep good notes)
sending a HUP to the unbound process does cause it to reload it's configuration.
Using "local-data-ptr" does work for reverse lookups. (however, I should provide a switch that also creates forward lookups. I think it'd be a bad idea to use it like that, but the work is nearly 0 effort.)
if unbound.conf includes a file that doesn't exist, it will refuse to start. So, if I implement this by creating another file to be included by unbound, care must be taken that the file exists first. That means it'll be slightly more difficult to "add on" to a pfsense configuration.
The following sequence works exactly as expected:
# add include to unbound conf echo "include: /var/unbound/testfile.conf" >> /var/unbound/unbound.conf # add a test record to added config file echo local-data-ptr: \"1:2:3:4:5:6:7:8 dummyname.mydomain.hello\" > /var/unbound/testfile.conf # force unbound to reload it's config kill -HUP `cat /var/run/unbound.pid` # test it dig @localhost -x 1:2:3:4:5:6:7:8 # append another test record with a TTL of 10 included echo local-data-ptr: \"8:7:6:5:4:3:2:1 10 anotherdummy.mydomain.hello\" >> /var/unbound/testfile.conf # force unbound to reload it's config kill -HUP `cat /var/run/unbound.pid` # test it dig @localhost -x 8:7:6:5:4:3:2:1Additional thoughts:
In addition to a process that just writes reverse lookup records, it MAY be useful to provide an interactive mode that takes a hostname as a parameter, and returns a list of all IPv6 records that are NOT found in DNS that go to the same MAC/interface. This could be used by whatever process pfsense aliases use to update their alias tables.
-
Since the question of use-case was brought up, I'll add my two-bits. I'm also using pfSense for a home network. We have a variety of hosts including guests on a hyper-v server, desktop and laptop PCs, *nix, phones and various other devices. I'd like the host names to be resolved for two main purposes, monitoring traffic (bandwidth usage) and monitoring for attacks. Before switching to pfSense, we used Sophos UTM, which did a pretty good job of reporting on bandwidth usage and monitoring for attacks. In particular, the threat-management feature provided reports on which hosts were being prodded. As far as I know, it used snort, but with a reporting layer on top. I wish pfSense had something like that. It also provided good reports on traffic.
-
A quick update…
First... ever notice that you have LESS free time on a holiday weekend? pfftt...
Anyway, I have this running (somewhat) on my own pfsense box to see how well the idea works. It isn't as nice as I thought it'd be...
First, my config:
I have multiple vlans, and all use a DHCP and DNS on a windows domain server. My windows server is running "windows server essentials" which likes to figure out the IP of your router, and forcibly reconfigures it's own DNS server to use your router as a DNS forwarder. (Fun! Fun! Fun!)
Because I have multiple vlans, I'm ignoring any linklocal Ipv6 addresses. If I included them, there'd be IP address collisions. That dropped the usefulness of this a small amount.
So, I have an hourly cron job that runs this program to add a bunch of lines to an unbound configuration. Each line is just something like this:
local-data-ptr: "1:2:3:4:5:6:7:8 10 hostname.domainname"After that config is re-written, it forces unbound to re-read it's config via:
kill -HUP `cat /var/run/unbound.pid`One issue with my odd DNS configuration is that it usually takes about 2-5 minutes before the reverse lookups appear in queries. (If I use "dig @localhost -x", I can see that unbound has already updated… so I know it's just the windows crud that's taking several minutes.)
Once Windows catches up, I'm somewhat limited on what I can use to view the results. Two things in particular I'm using to test:
1. "ndp -a". This works great. No issue.
2. "ntopng" This doesn't work so great. ntopng is a bit slow on doing DNS lookups. Also, I'm finding that ntopng's ipv6 support isn't all that great. (This might be fixed in a current ntopng version - I don't know.) What happens is that it resolves the ipv4 to a hostname... and a ipv6 to the same hostname. Then, in things like "Top Hosts (local)", it might show the same hostname multiple times.
That's not so bad. What IS bad is that the "details" URL for both hosts are pointing to the same place. For example:
https://192.168.1.1:3000/lua/host_details.lua?host=hostname.domainnameIn other words, it doesn't have distinct URLs for IPv6 vs IPv4 hosts. It's confusing!
What I'm thinking about doing to try and work around that is add a switch to my program so that it mangles the hostname. For example, instead of "hostname.domainname" for ipv6 addresses, I might change it to "hostname.XXXX.domainname" (where XXXX is replaced with the last 4 nibbles of the ipv6 address.)
(Of course, ideally, ntopng would get fixed, and pfsense would be updated to use the most recent ntopng version… but that likely won't happen for a while, so I'm trying to give some option in the meantime.)
Finally, there are a couple cases where I'm not getting any names for ipv6 addresses. This is usually for hosts that are always using IPv6 for everything (meaning there's nothing in the (ipv4) arp cache to match it with.) There are ways around this (ping floods, raw ARP packets, etc), but I'm not sure I want to get into that.
... I'm still working at it... (when time permits.)
