grep for all MACs from a certain day?
-
Is there a default limitation to the DHCP server logs on pfsense? I'm trying to use the filter to find all MAC addresses that requested an address on a certain day.
Which log file should I look in?
-rw-r--r-- 1 root wheel 43391 Jul 21 2018 bsdinstall_log -rw------- 1 root wheel 511488 Feb 21 20:19 dhcpd.log -rw-r--r-- 1 root wheel 9964 Jan 30 09:35 dmesg.boot -rw------- 1 root wheel 511488 Feb 21 20:26 filter.log -rw------- 1 root wheel 511488 Jan 30 09:35 gateways.log -rw------- 1 root wheel 511488 Jul 21 2018 ipsec.log -rw------- 1 root wheel 511488 Jul 21 2018 l2tps.log -rw-r--r-- 1 root wheel 0 Jul 21 2018 lastlog drwxr-xr-x 2 root wheel 512 Jul 21 2018 nginx -rw------- 1 root wheel 511488 Feb 21 20:25 nginx.log drwxr-xr-x 2 root wheel 512 Jul 21 2018 ntp -rw------- 1 root wheel 511488 Jan 30 09:47 ntpd.log -rw------- 1 root wheel 511488 Feb 21 16:51 openvpn.log -rw------- 1 root wheel 511488 Jul 21 2018 poes.log -rw------- 1 root wheel 511488 Jul 21 2018 portalauth.log -rw------- 1 root wheel 511488 Jul 21 2018 ppp.log drwxr-xr-x 2 redis redis 512 May 23 2018 redis -rw------- 1 root wheel 511488 Jul 21 2018 relayd.log -rw------- 1 root wheel 511488 Dec 16 14:40 resolver.log -rw------- 1 root wheel 511488 Feb 20 18:56 routing.log -rw------- 1 root wheel 511488 Feb 21 20:18 system.log -rw------- 1 root wheel 11254 Jan 30 09:35 userlog -rw-r--r-- 1 root wheel 394 Feb 21 20:18 utx.lastlogin -rw------- 1 root wheel 1584 Feb 21 20:18 utx.log -rw------- 1 root wheel 511488 Jul 21 2018 vpn.log -rw------- 1 root wheel 511488 Jul 21 2018 wireless.logI tried:
grep "Feb 12" /var/log/filter.log grep "Feb 12" /var/log/system.log grep "Feb 12" /var/log/dhcpd.logWhere'd I go wrong?
Thanks in advance!
-
@redbeardcowboy said in grep for all MACs from a certain day?:
Where'd I go wrong?
Hint 1 : check the file size of most those log files.
511488 bytes ... how can this happen ?Hint 2: most firewalls are small boxes with limit disk storage. Logs files that grow all day could be handled by something like logrottae .... pfSense has none.
Hint 3 : because it's just a half mega, did you checked what's in these file ?
Noop, you would have found that these files are binary (not much text in them) so grep can't make any soup of it.Hint 4: pfsense log
-
RTFM: https://docs.netgate.com/pfsense/en/latest/book/monitoring/system-logs.html
https://docs.netgate.com/pfsense/en/latest/monitoring/working-with-binary-circular-logs-clog.html
-
@gertjan said in grep for all MACs from a certain day?:
@redbeardcowboy said in grep for all MACs from a certain day?:
Where'd I go wrong?
Hint 1 : check the file size of most those log files.
511488 bytes ... how can this happen ?Hint 2: most firewalls are small boxes with limit disk storage. Logs files that grow all day could be handled by something like logrottae .... pfSense has none.
Hint 3 : because it's just a half mega, did you checked what's in these file ?
Noop, you would have found that these files are binary (not much text in them) so grep can't make any soup of it.Hint 4: pfsense log
@grimson said in grep for all MACs from a certain day?:
RTFM: https://docs.netgate.com/pfsense/en/latest/book/monitoring/system-logs.html
https://docs.netgate.com/pfsense/en/latest/monitoring/working-with-binary-circular-logs-clog.htmlFirst, let me say thanks for the replies and the links. clog - who'd have thunk it.
Second, I guess I'm out of luck since there's no log rotate. What a failure to not handle this differently since it's just text and you could at least compress and archive, by default. I'm used to enterprise firewalls, and I guess that's what I get for assumptions.
Lastly, what is it about IT people that make so many become jerks. I'm not sure you even know you're doing it sometimes, it's like 70% have turrets or something. Those "hint" and "RTFM" comments... seriously? Do you keyboard commandos act this condescending to people in person? Perhaps you might reflect on the unnecessary hostility.
If you've nothing good to say, say nothing.
-
@redbeardcowboy said in grep for all MACs from a certain day?:
Second, I guess I'm out of luck since there's no log rotate. What a failure to not handle this differently since it's just text and you could at least compress and archive, by default. I'm used to enterprise firewalls, and I guess that's what I get for assumptions.
RTFM some more then: https://docs.netgate.com/pfsense/en/latest/monitoring/copying-logs-to-a-remote-host-with-syslog.html
-
@grimson said in grep for all MACs from a certain day?:
@redbeardcowboy said in grep for all MACs from a certain day?:
Second, I guess I'm out of luck since there's no log rotate. What a failure to not handle this differently since it's just text and you could at least compress and archive, by default. I'm used to enterprise firewalls, and I guess that's what I get for assumptions.
RTFM some more then: https://docs.netgate.com/pfsense/en/latest/monitoring/copying-logs-to-a-remote-host-with-syslog.html
That comment was to mean that I'm out of luck, in that the topic was "finding all MACs from a certain day" which is no longer in the logs due to the lack of logrotate.
I'm aware of how syslog works.
This is just what you get with free shit.
-
Ok.
So you know how logs are stored.
pfSense gets installed on very small devices, so the (initial) limited log storage is understandable.
Up to you to size up the log file size to give space for some more history.
Then a grep => awk (today) => grep (MAC) will do.
-
@redbeardcowboy said in grep for all MACs from a certain day?:
@grimson said in grep for all MACs from a certain day?:
@redbeardcowboy said in grep for all MACs from a certain day?:
Second, I guess I'm out of luck since there's no log rotate. What a failure to not handle this differently since it's just text and you could at least compress and archive, by default. I'm used to enterprise firewalls, and I guess that's what I get for assumptions.
RTFM some more then: https://docs.netgate.com/pfsense/en/latest/monitoring/copying-logs-to-a-remote-host-with-syslog.html
That comment was to mean that I'm out of luck, in that the topic was "finding all MACs from a certain day" which is no longer in the logs due to the lack of logrotate.
I'm aware of how syslog works.
This is just what you get with free shit.
This is just what you get when you don't do your homework before deploying a solution. If logs were important, why didn't you have a strategy? Or even look at that part of the documentation?
Part of my "definition of done" is deploying with a logging solution in place; for everything. Logs are important. Setting up a linux syslog server is pretty easy, I'm sure you could find a VM template that's already baked for the task.
I too work in enterprise. I never look at device logs. I have a monitoring solution in place for that so I can do event correlation; essentially what you're trying to do.
I think SecurityOnion might be able to do this as well as a ton of other things. Set that up as part of your monitoring system, and you'll be well on your way to designing networks for the enterprise!
-
Yes. Logging to an external logging system is expected if you want more than basic debugging tools.
I have used many "enterprise" firewalls and we never depended on them to store anything but the most cursory of logs on the devices themselves. We always logged to something external if anything historical was desired.
