Need Log Analysis

hamed_forum

Another link to a quick guide to setting up ELK with PFS.

https://elijahpaul.co.uk/monitoring-pfsense-2-1-logs-using-elk-logstash-kibana-elasticsearch/

I used this myself when setting up my own log server with my PFS setup.

An updated version is here:

https://elijahpaul.co.uk/updated-monitoring-pfsense-logs-using-elk-elasticsearch-logstash-kibana-part-1/

i do it many time but dont work

hamed_forum

@johnpoz:

it does work, have it running..

You just need to set it up correctly ;)

You can find a working vm here..
https://www.reddit.com/r/PFSENSE/comments/4dymci/i_made_a_simple_bare_bones_simple_elk_vm_for/

This will do firewall logs, there are many guides on the net, many of them missing pieces or not very clear. But this vm works out of the box for firewall logs. If you want other logs you might need to make some adjustments

I do with we could get someone to maintain a guide or vm that logs everything with nice dashboards already made, etc. I keep meaning to make my setup available but it still needs a lot of tweaking to do everything extra pretty ;)

in pf sense only tike firewall logs or everthing?
tanks you do great work

AR15USR

I got that converted to the .vmdk, but not sure how to translate that into a running VM. VirtualBox says it can't recognize that type of disk. Jonpoz or mudmanc4 can you expand a bit on your instructions for getting this working please? Thanks..

Im using VirtualBox 5.0.2 running a Ubuntu 16 VM, on MacOS 10.11.5

johnpoz

pretty sure virtualbox just lets you import a ova.

AR15USR

@johnpoz:

pretty sure virtualbox just lets you import a ova.

Failed to open the disk image file /Users/xxxxxxxxx/Downloads/PFELK01-160408.ova.

Could not get the storage format of the medium '/Users/xxxxxxxxx/Downloads/PFELK01-160408.ova' (VERR_NOT_SUPPORTED).

Result Code: VBOX_E_IPRT_ERROR (0x80BB0005)
Component: MediumWrap
Interface: IMedium {4afe423b-43e0-e9d0-82e8-ceb307940dda}
Callee: IVirtualBox {0169423f-46b4-cde9-91af-1e9d5b6cd945}
Callee RC: VBOX_E_OBJECT_NOT_FOUND (0x80BB0001)

mudmanc4

@AR15USR:

<snip>Im using VirtualBox 5.0.2 running a Ubuntu 16 VM, on MacOS 10.11.5</snip>

Virtual disk space issue?
So your running Virtualbox on OS X, where Ubuntu is a guest, then trying to import a 1.6GB image into Ubuntu? Correct me?

AR15USR

@mudmanc4:

@AR15USR:

<snip>Im using VirtualBox 5.0.2 running a Ubuntu 16 VM, on MacOS 10.11.5</snip>

Virtual disk space issue?
So your running Virtualbox on OS X, where Ubuntu is a guest, then trying to import a 1.6GB image into Ubuntu? Correct me?

Not sure exactly what you mean by "import a 1.6GB image into Ubuntu". I downloaded the .ova in my Ubuntu guest so I could convert to the .vmdk. From there I copied the .vmdk to OS X and tried to create a new VM guest based on Linux with 2 processors, 16gb ram, and the .vmdk as the virtual hard disk. When selecting the .vmdk as the virtual hard disk is when I get the error.

mudmanc4

@AR15USR:

Not sure exactly what you mean by "import a 1.6GB image into Ubuntu". I downloaded the .ova in my Ubuntu guest so I could convert to the .vmdk. From there I copied the .vmdk to OS X and tried to create a new VM guest based on Linux with 2 processors, 16gb ram, and the .vmdk as the virtual hard disk. When selecting the .vmdk as the virtual hard disk is when I get the error.

Gotcha now.
You could not import the .ova itself?

AR15USR

@mudmanc4:

@AR15USR:

Not sure exactly what you mean by "import a 1.6GB image into Ubuntu". I downloaded the .ova in my Ubuntu guest so I could convert to the .vmdk. From there I copied the .vmdk to OS X and tried to create a new VM guest based on Linux with 2 processors, 16gb ram, and the .vmdk as the virtual hard disk. When selecting the .vmdk as the virtual hard disk is when I get the error.

Gotcha now.
You could not import the .ova itself?

No errors out as well. What Virtual Machine software are you using?

mudmanc4

@AR15USR:

No errors out as well. What Virtual Machine software are you using?

https://virt-manager.org/

Maybe it's corrupt in some way. I would just grab a new copy.

AR15USR

@mudmanc4:

@AR15USR:

No errors out as well. What Virtual Machine software are you using?

https://virt-manager.org/

Maybe it's corrupt in some way. I would just grab a new copy.

What version of Linux is the ova/vdmk based on? Maybe I was using the wrong type..

mudmanc4

@AR15USR:

What version of Linux is the ova/vdmk based on? Maybe I was using the wrong type..

I believe it is Ubuntu 14

vmw:osType="ubuntu64Guest

AR15USR

Hmm, since Splunk has a native OS X version I think I will start exploring that.

johnpoz

why are you going through all the extraction nonsense - pretty sure virtualbox should just import that ova as an appliance. When I get home from this road trip I will give it a go.

AR15USR

@johnpoz:

why are you going through all the extraction nonsense - pretty sure virtualbox should just import that ova as an appliance. When I get home from this road trip I will give it a go.

Yes I see that now. I'll try that when I get home. I was trying to use it as a virtual hard disk, way over thinking it..

mudmanc4

And that over thinking part we are all susceptible to.

johnpoz

Ok true - but I mentioned back couple of days ago that can just import the ova into virtualbox ;)

https://forum.pfsense.org/index.php?topic=111967.msg623944#msg623944

AR15USR

…and there it goes. Up and running, thanks all. Now I just need to learn how to use it ;)