Need Log Analysis
-
hi
which Log Analysis is good for pfsense 2.3?
install ELK Stack but dont work -
it does work, have it running..
You just need to set it up correctly ;)
You can find a working vm here..
https://www.reddit.com/r/PFSENSE/comments/4dymci/i_made_a_simple_bare_bones_simple_elk_vm_for/This will do firewall logs, there are many guides on the net, many of them missing pieces or not very clear. But this vm works out of the box for firewall logs. If you want other logs you might need to make some adjustments
I do with we could get someone to maintain a guide or vm that logs everything with nice dashboards already made, etc. I keep meaning to make my setup available but it still needs a lot of tweaking to do everything extra pretty ;)
-
If you want to tune and tune and tune and spend time on tuning, get Elkstack, if you want to read your logs and start analyzing them immediately, use Splunk. The license is free for 500MB logs per day, for me this is two firewalls and a lot of other stuff.
Stop playing, start analyzing :)
-
Another link to a quick guide to setting up ELK with PFS.
https://elijahpaul.co.uk/monitoring-pfsense-2-1-logs-using-elk-logstash-kibana-elasticsearch/
I used this myself when setting up my own log server with my PFS setup.
An updated version is here:
https://elijahpaul.co.uk/updated-monitoring-pfsense-logs-using-elk-elasticsearch-logstash-kibana-part-1/
-
…I do with we could get someone to maintain a guide or vm that logs everything with nice dashboards already made, etc. I keep meaning to make my setup available but it still needs a lot of tweaking to do everything extra pretty ;)
Yes please! ;D
-
For those who do not use vmware
Grabbed the .ova converted it to qemu2 , flawless. Nicely done.
decompress:
tar xvf PFELK01-160408.ova
Convert:
qemu-img convert -f vmdk -O qcow2 PFELK01-160408-disk1.vmdk PFELK01.qcow2
Have a peek at the .ovf noticing the existing virtual CPU's and memory
Which happen to be 4 CPU's and 8GB ramImport = done
Change existing password from 'pf' to {your password}
passwd
Access at {IP:5601}
Easypeasy
-
U might need change timezone too that was in utc while my pfsense in central
And u prob dont need 4 cpu and 8 gig running on only 2 and 2gb
-
Another link to a quick guide to setting up ELK with PFS.
https://elijahpaul.co.uk/monitoring-pfsense-2-1-logs-using-elk-logstash-kibana-elasticsearch/
I used this myself when setting up my own log server with my PFS setup.
An updated version is here:
https://elijahpaul.co.uk/updated-monitoring-pfsense-logs-using-elk-elasticsearch-logstash-kibana-part-1/
i do it many time but dont work
-
it does work, have it running..
You just need to set it up correctly ;)
You can find a working vm here..
https://www.reddit.com/r/PFSENSE/comments/4dymci/i_made_a_simple_bare_bones_simple_elk_vm_for/This will do firewall logs, there are many guides on the net, many of them missing pieces or not very clear. But this vm works out of the box for firewall logs. If you want other logs you might need to make some adjustments
I do with we could get someone to maintain a guide or vm that logs everything with nice dashboards already made, etc. I keep meaning to make my setup available but it still needs a lot of tweaking to do everything extra pretty ;)
in pf sense only tike firewall logs or everthing?
tanks you do great work -
I got that converted to the .vmdk, but not sure how to translate that into a running VM. VirtualBox says it can't recognize that type of disk. Jonpoz or mudmanc4 can you expand a bit on your instructions for getting this working please? Thanks..
Im using VirtualBox 5.0.2 running a Ubuntu 16 VM, on MacOS 10.11.5
-
pretty sure virtualbox just lets you import a ova.
-
pretty sure virtualbox just lets you import a ova.
Failed to open the disk image file /Users/xxxxxxxxx/Downloads/PFELK01-160408.ova. Could not get the storage format of the medium '/Users/xxxxxxxxx/Downloads/PFELK01-160408.ova' (VERR_NOT_SUPPORTED). Result Code: VBOX_E_IPRT_ERROR (0x80BB0005) Component: MediumWrap Interface: IMedium {4afe423b-43e0-e9d0-82e8-ceb307940dda} Callee: IVirtualBox {0169423f-46b4-cde9-91af-1e9d5b6cd945} Callee RC: VBOX_E_OBJECT_NOT_FOUND (0x80BB0001)
-
<snip>Im using VirtualBox 5.0.2 running a Ubuntu 16 VM, on MacOS 10.11.5</snip>
Virtual disk space issue?
So your running Virtualbox on OS X, where Ubuntu is a guest, then trying to import a 1.6GB image into Ubuntu? Correct me? -
<snip>Im using VirtualBox 5.0.2 running a Ubuntu 16 VM, on MacOS 10.11.5</snip>
Virtual disk space issue?
So your running Virtualbox on OS X, where Ubuntu is a guest, then trying to import a 1.6GB image into Ubuntu? Correct me?Not sure exactly what you mean by "import a 1.6GB image into Ubuntu". I downloaded the .ova in my Ubuntu guest so I could convert to the .vmdk. From there I copied the .vmdk to OS X and tried to create a new VM guest based on Linux with 2 processors, 16gb ram, and the .vmdk as the virtual hard disk. When selecting the .vmdk as the virtual hard disk is when I get the error.
-
Not sure exactly what you mean by "import a 1.6GB image into Ubuntu". I downloaded the .ova in my Ubuntu guest so I could convert to the .vmdk. From there I copied the .vmdk to OS X and tried to create a new VM guest based on Linux with 2 processors, 16gb ram, and the .vmdk as the virtual hard disk. When selecting the .vmdk as the virtual hard disk is when I get the error.
Gotcha now.
You could not import the .ova itself? -
Not sure exactly what you mean by "import a 1.6GB image into Ubuntu". I downloaded the .ova in my Ubuntu guest so I could convert to the .vmdk. From there I copied the .vmdk to OS X and tried to create a new VM guest based on Linux with 2 processors, 16gb ram, and the .vmdk as the virtual hard disk. When selecting the .vmdk as the virtual hard disk is when I get the error.
Gotcha now.
You could not import the .ova itself?No errors out as well. What Virtual Machine software are you using?
-
No errors out as well. What Virtual Machine software are you using?
https://virt-manager.org/
Maybe it's corrupt in some way. I would just grab a new copy.
-
-
What version of Linux is the ova/vdmk based on? Maybe I was using the wrong type..
I believe it is Ubuntu 14
vmw:osType="ubuntu64Guest
-
Hmm, since Splunk has a native OS X version I think I will start exploring that.