Logging DNS Requests - client IP, requested FQDN, and response addresses
-
Sorry for reopening this old thread but did not see a point in creating a new one.
Currently I am using pfblockNG and I can not see responseIPs unless on log level 5.
I added the following to the general settings -tab and custom options and applied and even rebooted pfsense but still no luck.
server:
log-queries: yeseven with this option added to the options i do not see any difference the response IPs is still missing unless i have the log level set to 5 which generates a lot of noise.
This is what is logged when on log level 5 and having the log-queries option enabled.
cat /var/log/resolver.log | grep hello.eu Nov 10 13:47:22 pfSense unbound: [8691:0] info: 192.168.0.15 hello.eu. A IN Nov 10 13:47:22 pfSense unbound: [8691:0] info: validator operate: query hello.eu. A IN Nov 10 13:47:22 pfSense unbound: [8691:0] info: resolving hello.eu. A IN Nov 10 13:47:22 pfSense unbound: [8691:0] info: processQueryTargets: hello.eu. A IN Nov 10 13:47:22 pfSense unbound: [8691:0] info: sending query: hello.eu. A IN Nov 10 13:47:22 pfSense unbound: [8691:0] info: 0RDd mod1 rep hello.eu. A IN Nov 10 13:47:22 pfSense unbound: [8691:0] info: iterator operate: query hello.eu. A IN Nov 10 13:47:22 pfSense unbound: [8691:0] info: response for hello.eu. A IN Nov 10 13:47:22 pfSense unbound: [8691:0] info: incoming scrubbed packet: ;; ->>HEADER<<- opcode: QUERY, rcode: NOERROR, id: 0 ;; flags: qr rd ra ; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 0 ;; QUESTION SECTION: hello.eu. IN A ;; ANSWER SECTION: hello.eu. 14399 IN A 165.160.15.20 hello.eu. 14399 IN A 165.160.13.20 ;; AUTHORITY SECTION: ;; ADDITIONAL SECTION: ;; MSG SIZE rcvd: 58 Nov 10 13:47:22 pfSense unbound: [8691:0] info: finishing processing for hello.eu. A IN Nov 10 13:47:22 pfSense unbound: [8691:0] info: validator operate: query hello.eu. A IN Nov 10 13:47:22 pfSense unbound: [8691:0] info: no signer, using hello.eu. TYPE0 CLASS0 Nov 10 13:47:22 pfSense unbound: [8691:0] info: validator: FindKey hello.eu. A IN Nov 10 13:47:22 pfSense unbound: [8691:0] info: target keyname hello.eu. DNSKEY IN Nov 10 13:47:22 pfSense unbound: [8691:0] info: next keyname hello.eu. DNSKEY IN Nov 10 13:47:22 pfSense unbound: [8691:0] info: generate request hello.eu. DS IN Nov 10 13:47:22 pfSense unbound: [8691:0] info: validator operate: query hello.eu. DS IN Nov 10 13:47:22 pfSense unbound: [8691:0] info: resolving hello.eu. DS IN Nov 10 13:47:22 pfSense unbound: [8691:0] info: processQueryTargets: hello.eu. DS IN Nov 10 13:47:22 pfSense unbound: [8691:0] info: sending query: hello.eu. DS IN Nov 10 13:47:22 pfSense unbound: [8691:0] info: 0vRDCD mod1 hello.eu. DS IN Nov 10 13:47:22 pfSense unbound: [8691:0] info: 1RDdc mod0 rep hello.eu. A IN Nov 10 13:47:23 pfSense unbound: [8691:0] info: 0vRDCD mod1 hello.eu. DS IN Nov 10 13:47:23 pfSense unbound: [8691:0] info: 1RDdc mod0 rep hello.eu. A IN Nov 10 13:47:23 pfSense unbound: [8691:0] debug: try edns1xx0 <hello.eu.> 8.8.8.8#53 Nov 10 13:47:23 pfSense unbound: [8691:0] info: iterator operate: query hello.eu. DS IN Nov 10 13:47:23 pfSense unbound: [8691:0] info: response for hello.eu. DS IN Nov 10 13:47:23 pfSense unbound: [8691:0] info: incoming scrubbed packet: ;; ->>HEADER<<- opcode: QUERY, rcode: NOERROR, id: 0 ;; flags: qr rd ra ; QUERY: 1, ANSWER: 0, AUTHORITY: 6, ADDITIONAL: 0 ;; QUESTION SECTION: hello.eu. IN DS ;; ANSWER SECTION: ;; AUTHORITY SECTION: QBQ65Q6097OCPPR0EUCQNSC1FHE073UA.eu. 599 IN NSEC3 1 1 1 5CA1AB1E qbq6ocgmt2jnij4jnf2ccrfi4ce4nue0 NS SOA RRSIG DNSKEY NSEC3PARAM ;{flags: optout} QBQ65Q6097OCPPR0EUCQNSC1FHE073UA.eu. 599 IN RRSIG NSEC3 8 2 600 20201115214452 20201108212822 49485 eu. I1RJ2Xid7b+8MvYFG/wlQW2ITAa3rKlieEoi327l38qQ6z5WjS1iybu5RqivBn7N3BjgYwpkZLfdh1W7LtcHRVhqJS3/dzuFoZGiyDb6ZSzqIDxb2Y6rCEIaA6MplvbbTOllVCaeiyz4QzWQD+UM/d+GmN6xmqCKhSc3YBZp+30= ;{id = 49485} eu. 599 IN SOA nl.dns.eu. tech.eurid.eu. 1066667209 3600 1800 3600000 600 eu. 599 IN RRSIG SOA 8 1 86400 20201117134714 20201110124714 49485 eu. p0OcPTTNDXO2YQOQBF0ataST4eGm5dFs9Kgwv+TWYZRn485ouk5vdTpjYsGYAFQIwIUpf10NbpNXciAoTd4pxHDZk78Nq37HzwqDFt+AL1c+1ZA9kvNnQKhlqNPyFmaEJEB6CQ6mVSPeSvr8cuj4xvlBfw4nO2y294Px4XyxJyU= ;{id = 49485} O12A97G7C5N1F6FND5CSAN376JRJ0TUB.eu. 599 IN NSEC3 1 1 1 5CA1AB1E o12fn126piloqap30lgch1fv422tjsic NS DS RRSIG ;{flags: optout} O12A97G7C5N1F6FND5CSAN376JRJ0TUB.eu. 599 IN RRSIG NSEC3 8 2 600 20201117110740 20201110105559 49485 eu. QdN9pGrYYN0mW2sqvV7CMuQLhLHku3Hd+Sh1fmv8KwyiXUQbyOFyrsSkYT4AVWJq2c98WAHMnJO9pGCaVxIuvSbXeeEoOSyCff+1gBiRaC4NszNMYULlcYgZYXvQ6f/6u6nIcz2HK1cRnh1jjI28C//jqE3GLZESohvWC4bqnF0= ;{id = 49485} ;; ADDITIONAL SECTION: ;; MSG SIZE rcvd: 733 Nov 10 13:47:23 pfSense unbound: [8691:0] info: finishing processing for hello.eu. DS IN Nov 10 13:47:23 pfSense unbound: [8691:0] info: validator operate: query hello.eu. DS IN Nov 10 13:47:23 pfSense unbound: [8691:0] info: validator: inform_super, sub is hello.eu. DS IN Nov 10 13:47:23 pfSense unbound: [8691:0] info: super is hello.eu. A IN Nov 10 13:47:23 pfSense unbound: [8691:0] info: validator operate: query hello.eu. A IN
I was hoping to get the logs in a simple format like dnsmasq
Nov 10 13:47:23 pfSense dnsmasq[1248]: query[A] mazedecrypt.top from 77.77.77.77 Nov 10 13:47:23 pfSense dnsmasq[1248]: forwarded mazedecrypt.top to 8.8.8.8 Nov 10 13:47:23 pfSense dnsmasq[1248]: reply mazedecrypt.top is 91.218.114.38
currently running version:
2.4.5-RELEASE-p1 (amd64)
built on Tue Jun 02 17:51:17 EDT 2020Any help would be appreciated.
-
Hello!
You can run a script to continually pluck out the bits you are interested in and put them into their own logfile with something like :
clog -f /var/log/resolver.log | grep --line-buffered "info: resolving" | tee -a /var/log/myqueries.log
Daemonizing this is another exercise.
This might get easier with the logging changes in 2.5John
-
log-queries ?
With this : https://nlnetlabs.nl/documentation/unbound/unbound.conf/
I set this :I saw this :
I did not change this :
edit :
The format is how the unbound guys created it - unbound is created and maintained here : https://nlnetlabs.nl/ - pfSense is 'just' importing the FreeBSD port.
Changing the log format means : changing the source and re compile.But ..... I didn't read the doc (see above) - maybe there is an command option ?
-
@serbus Thanks for the suggestion but not something we would want to go down the route of unless we had no other options.
@Gertjan
Thanks for confirming.It looks like some of the output I was hoping to be in the logs are missing such as the resolved IPs but i suppose it does not make a difference as the connections will be logged.
It will make life a little bit more awkward to investiagate incidents on our SIEM but as the SIEM also does resolution on domains and addresses its not too much trouble.
-
-
@vijay7 said in Logging DNS Requests - client IP, requested FQDN, and response addresses:
Hi i dont have dnsbl*.conf in my pfsense
Months ago, this file was put in the Resolver > Custon option box by pfBlokcerNG.
These days, its 2021 now, and several pfBlockerNG later, this file isn't needed anymore.
That is, if you're using 'Python' mode.
With the classic mode, I still have :The pfblokcerNG 'force update' shows this also :
..... Removing DNSBL Unbound python integration settings Adding DNSBL Unbound mode (Resolver adv. setting) DNS Resolver ( enabled ) unbound.conf modifications: Removed DNSBL Unbound Python mode Removed DNSBL Unbound Python mode script Added DNSBL Unbound mode ....
@vijay7 said in Logging DNS Requests - client IP, requested FQDN, and response addresses:
is there any way, to get the client IP in DNS query logs.
@BBcan177, author of pfBlokcerNG, explained in another thread that this is a shortcoming of the current unbound version that pfSense 2.4.5-p1 uses.
The new unbound version, available for the upcoming 2.5.0 uses a more recent unbound, which does show the IP's.
It's not possible to upgrade the unbound version using pfSense 2.4.5-p12@vijay7 said in Logging DNS Requests - client IP, requested FQDN, and response addresses:
@jimp Need your help here.
He will tell you that 2.5.0 is ready when .... it's ready.
-
@fabrizior That feature would be awesome
Be able to see accessed sites per interface, summarized some how
Have you found a way to do that?
Maybe I will give a go to elasticsearch
Have a good time -
@foreigner
I have not found a way yet, though I have not upgraded yet either. After upgrade, eventually, will probably ingest the dns logs to my influxdb via telegraf with a log parser input plugin since I already have telegraf gathering metrics with influxdb and a grafana dashboard. logstash and ES might be more efficient, but that seems redundant for me right now. -
I just tested pfelk: https://github.com/pfelk/pfelk
and it displays the requested domains nicelyI took me around a couple of hours to see the results
At least the outbound and firewall dashboards are usefull for me, although ELK consumes a lot of memory :S
It might help if one is in a hurry...
-
@foreigner cool. nice find. You’re right, that seems rather heavy. I might have a look at the log parsers and see if I can rev. engineer some of them into a telegraf log parser format and into my existing influxdb. And replicate the kibana dashboards in grafana… when/if I can find time.
Though… not being on 2.5 yet, no resolver request fqdn/IPs in the dns log to grab anyway if I understand all this correctly.