Pi-Hole - New version leaking ??

bingo600

EDIT:
Github issue opened:
https://github.com/pi-hole/pi-hole/issues/4353

If you fee. you can contribute w. info, please do so.

****** Original post begins here

Just a FYI.

I use pihole's at home (Debian 10 VM's) , and they're setup to ONLY use my two linux bind9 servers as forwarders. This has been running excellent ... Until yesterday ...

I have made rules on the pihole vlan allowing just UDP 53 & 853 to my linux bind9's - And have never seen any "deny" log entries until i updated my piholes yesterday.

Now i see this :

    Oct 2 08:00:50 	Pihole-Vlan 	Deny Outside access DNS from local IP's (1522486856) 	Pihole-IP:51765		1.0.0.127:53		UDP
	Oct 2 08:00:45 	Pihole-Vlan 	Deny Outside access DNS from local IP's (1522486856) 	Pihole-IP:38610		1.0.0.127:53		UDP
	Oct 2 08:00:40 	Pihole-Vlan 	Deny Outside access DNS from local IP's (1522486856) 	Pihole-IP:38610		1.0.0.127:53		UDP

Now it is trying to query : 1.0.0.127 , seems to be Cloudflare.

Even though i have ONLY configured it to forward to my two bind9 servers

Wonder if this was intentional or ??

For some time you would have to allow pihole to make DNS requests to "their" dns server during update , as they use some TXT record on that DNS , to resolve some OS version stuff.

But i just open those manualy when updating , and close them again, so i WILL catch any unexpected queries.

Edit:
~~It might not be the pihole app ... Could be the Debian OS running the pihole (Debian resolver is set to "localhost" .. Pihole)~~
Will try to get a trace

Edit2: Seems like some "hourly queries" i have blocks starting every hour at 00 , lasting 50 secs , then silence until next 00.

Edit3: It has just been 00 , and i got the DNS traces.
It is a reverse dns lookup that match with my pihole Local (rfc1918) "Top Clients" - Makes sense as the hostnames of those are displayed on the pihole web page.

It can resolve the names and show on the web page, so it must query my bind9's.

But why also query Cloudflare : 1.0.0.127
pihole-1.0.0.127-dns-packetcapture.cap.zip

/Bingo

pihole-block.txt.zip

johnpoz

@bingo600 I just added rule on the vlan my pihole is on to block and log any normal dns 53 queries udp/tcp 53.. Let you know if see anything like your seeing.

Running same pihole as you
Pi-hole v5.5 FTL v5.10.2 Web Interface v5.7

Could be something else running on the pihole box?

edit: Well F me - wtf!

Pihole log showing this

It sure shouldn't be sending those to 1.0.0.127 - some sort of bug where should be going to 127.0.0.1 ???

edit2:
Those should all resolve as well - if pihole would ask who it should be asking which is pfsense locally..

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;10.9.168.192.in-addr.arpa.     IN      PTR

;; ANSWER SECTION:
10.9.168.192.in-addr.arpa. 3600 IN      PTR     nas.local.lan.

edit3:
I see you put in a issue on pihole github - nice!!! Lets see what they say.. Thanks for bringing this up here.. Really interesting, clearly some sort of bug with pihole - it shouldn't be doing this for sure..

johnpoz

@bingo600 Hey check your setting for forwarding rfc ptr

That seems to have gotten checked when updating.. I know I had it unchecked before, and unchecked it just the other day when updated, but after update to latest ftl, it was set again..

Now if I query pihole for ptr of local it resolves correctly

edit:
Lets see on the hour - if it makes more queries to 1.0.0.127

edit2: Well its still freaking doing it - even with that unchecked, but atleast it resolves them now as well

They do not show nx now

Pretty serious bug in pihole if you ask me.. I subscribed to your issue over at pihole github - really curious what is said about it there.

bingo600

@johnpoz
In the summerhouse .. SWMBOB had a "little" task for me.
Took 4hr

Mine is also set

Disabled now.

And i can reverse resolve my local ip's

Wonder if someone FSCK'ed up : 127.0.0.1 vs 1.0.0.127
Seem to remember some of the linux network calls might reverse the byte order H2N etc ...

Btw: My debian is a "Clean Deb10 VM" , think NTP is the only other stuff installed , besides pihole.

/Bingo

johnpoz

@bingo600 said in Pi-Hole - New version leaking ??:

Btw: My debian is a "Clean Deb10 VM" , think NTP is the only other stuff installed , besides pihole.

I run it on a pi, and really only thing installed on the thing is pihole.. So pretty sure its pihole wanting to resolve the IPs it has seen.. And not something else on the device..

One way to check would be to turn off pihole just before the hour, and see if queries are still made..

bingo600

@johnpoz said in Pi-Hole - New version leaking ??:

@bingo600 said in Pi-Hole - New version leaking ??:

Btw: My debian is a "Clean Deb10 VM" , think NTP is the only other stuff installed , besides pihole.

I run it on a pi, and really only thing installed on the thing is pihole.. So pretty sure its pihole wanting to resolve the IPs it has seen.. And not something else on the device..

One way to check would be to turn off pihole just before the hour, and see if queries are still made..

My arpa lookup's were consistant with the "Last TOP Clients"
And my bind servers.

My gut says Pi-Hole ....

If i stop Pi-Hole ... Then my Phone & MMedia vlan can't resolve.
My wife loves her Phone ...
That scares me more than a "Disaster at work" ...
She's the Boss i'm the most scared off ...

johnpoz

@bingo600 Well I was only thinking of killing it for couple of minutes - right at the hour mark.. ;)

Bob.Dig

@bingo600 said in Pi-Hole - New version leaking ??:

Wonder if someone FSCK'ed up : 127.0.0.1 vs 1.0.0.127

If I do a port test it seems not open, on 1.1.1.1 it is. So maybe it is that.

bingo600

@bob-dig said in Pi-Hole - New version leaking ??:

@bingo600 said in Pi-Hole - New version leaking ??:

Wonder if someone FSCK'ed up : 127.0.0.1 vs 1.0.0.127

If I do a port test it seems not open, on 1.1.1.1 it is. So maybe it is that.

Where did 1.1.1.1 come from ???
Do you mean another Cloudflare DNS or ??

Edit:
Ahh ... You mean 1.0.0.127:53 is not open , but 1.1.1.1 is

johnpoz

This has been corrected in the pihole software

https://github.com/pi-hole/FTL/pull/1196

Interesting issue - great documentation of the problem by @bingo600 and then clarification of it happening.. Then quick response by the pihole developers.. This is a perfect example of how quick and fast things can be tracked down when something goes wrong.. When sufficient information on exactly what is happening with proof of it happening that can be recreated by someone else..

Not related to pfsense in anyway ;) But perfect example of tacking down and fixing an issue.. Thanks @bingo600 for bringing it up here.. Win win for everyone, both pfsense users that use pihole (I think there are quite a few) and the overall pihole community as well all win with such detailed information being provided.

bingo600

@johnpoz

I just made it home to test at 14:00 CEST , and can verify the fixed version, doesn't send data to 1.0.0.127 anymore.

Yep ... Quick response from pihole maintainers.
Btw: I just asked how2 revert back to the master branch on our piholes.

I'm a bit ?? about the Cloudflare part confusing him.
I was just providing that info for pointing to the IP block owner, well maybe he never used whois ...

/Bingo

johnpoz

@bingo600 said in Pi-Hole - New version leaking ??:

I'm a bit ?? about the Cloudflare part confusing him.

hehehe - yeah, I see his point though - it wasn't really pihole asking cloudflare - it was just that the reverse of 127.0.0.1 (1.0.0.127) falls into the cloudflare public space..

If it would of been doing it with the local address 10.0.0.42 vs loopback, it would of looked like someone else ip space 42.0.0.10..

bingo600

@johnpoz

To revert pihole

pihole checkout master

And when mentioning Cloudflare i also gave the correct ip on the same line.
Pihole asking 1.0.0.127 - Cloudflare for RFC1918 arpa info

Well maybe i'we spend to much time on IP's & whois

johnpoz

@bingo600 Yeah I know how to revert - but thanks for posting that.. Did it actually make it to master already.. We can can go back to master?

bingo600

@johnpoz

Naah i doubt it's in master yet.

I would expect them to close the issue first ... Or is that me ???
I would think it was better they did it.

It's not in master yet

/Bingo

johnpoz

@bingo600 I hear yeah, and agree you did post the actual IP, etc. I got where you were coming from for sure. It was good info to provide - but possible the "cloudflare" mention got them thinking something else other than it was just reverse of 127.0.0.1

If clouldflare actually listened for dns and used that 1.0.0.127 it could of been more confusing ;) But you are correct that IP does fall to their space - and it could be leveraged on the public internet for sure.

edit: What is funny to think about is there has to be lots and lots of piholes out there - in the millions I would guess. And all of them started doing queries to 1.0.0.127? With this latest version - could of gotten the cloudflare guys scratching their heads. If they happen to notice the uptick in traffic to that IP.

Or has this been happening before, I was not blocking dns traffic on that vlan - because the only thing on that vlan is my pihole and my ntp server.. Which I directly control and setup and trust they wouldn't be trying to do doh or use some other dns that I didn't setup, etc. So I wouldn't have noticed that traffic unless it had been brought up.. So was this happening before? Or just something that came out of the lastest update to pihole?

bingo600

@johnpoz

I think Cloudflare might have lots of "interesting" traffic going to that 1.0.0.127 ip.
And that it was not a coincidence that they "just got" the /24 where "Reverse loopback was pointing".
Prob. better them than many others ...

/Bingo

johnpoz

@bingo600 yeah thinking the same thing - see the edit on my other post.

edit: To the ownership of that before nonused space - yeah prob lots of interesting traffic going to that space. 1.1.1.1 use to be a common IP to use in setting up stuff - cisco wireless even suggested if not defaulted to using that for what should of been just internal consumption, etc.

Bad practice to use non assigned space internally.. Just grabbing public space and trying to use it internally has always been bad idea, even if the space wasn't actually being used at the time.. Or you didn't think you would ever need to go to that space.. I can almost promise you there are many places using DOD space internally for their own use.. Since when would they ever need to actually go to those IPs ;)

bingo600

@johnpoz said in Pi-Hole - New version leaking ??:

So was this happening before? Or just something that came out of the lastest update to pihole?

I have never seen pihole not behave nicely before.
But in our trade the "tinfoil hat" can sometimes be a bit "heavy"
So since i knew pihole was only supposed to query my two binds's , i set up a "deny all others".

For me it started friday at 18:00 , and i had just updated pihole to latest at 17:32.

But at the same time Debian was also pulling in a few packets , so it was like hmmmm .....

Then i did a trace on pfSense , and saw the arpa adresses queried was consistant with my pihole "Top Clients" , and now it pointed at pihole.

I was 90% sure the Debian OS would not want to reverse lookup those ip's.
But i did have to check first, if i had installed bind on the pihole , and where resolv.conf was pointing (it was localhost).

When no bind , it could only have been pihole resolving for localhost.

On afterthought bind & dnsmasq would have been fighting for UDP 53 , and hopefully one of them would have "barfed" loudly in syslog.

Re: Cisco 1.1.1.1 captive portal .. Yupp would also be a good one to have.
And is has Cloudflare as owner too

/Bingo

bingo600

@johnpoz said in Pi-Hole - New version leaking ??:

I can almost promise you there are many places using DOD space internally for their own use.. Since when would they ever need to actually go to those IPs ;)

Murphy is going to bite your "behind" if you do that ...
Just wait ...

ISTR back in the oooleee days.
HP made a lot of teaching material , for their HP UX.
Where they used some of their "Public ip's" as setup examples.
I can't remember what Blok they has , it was an A.

But so many sysadms were using HP's ip addesses at their sites (100%) example , that HP refrained for using them (the ones used in examples) for public servers.

/Bingo