Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    Need Log Analysis

    Scheduled Pinned Locked Moved Problems Installing or Upgrading pfSense Software
    25 Posts 6 Posters 7.4k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • H
      hamed_forum
      last edited by

      hi
      which Log Analysis is good for pfsense 2.3?
      install ELK Stack but dont work

      1 Reply Last reply Reply Quote 0
      • johnpozJ
        johnpoz LAYER 8 Global Moderator
        last edited by

        it does work, have it running..

        You just need to set it up correctly ;)

        You can find a working vm here..
        https://www.reddit.com/r/PFSENSE/comments/4dymci/i_made_a_simple_bare_bones_simple_elk_vm_for/

        This will do firewall logs, there are many guides on the net, many of them missing pieces or not very clear.  But this vm works out of the box for firewall logs.  If you want other logs you might need to make some adjustments

        I do with we could get someone to maintain a guide or vm that logs everything with nice dashboards already made, etc.  I keep meaning to make my setup available but it still needs a lot of tweaking to do everything extra pretty ;)

        An intelligent man is sometimes forced to be drunk to spend time with his fools
        If you get confused: Listen to the Music Play
        Please don't Chat/PM me for help, unless mod related
        SG-4860 24.11 | Lab VMs 2.8, 24.11

        1 Reply Last reply Reply Quote 0
        • T
          tsmalmbe
          last edited by

          If you want to tune and tune and tune and spend time on tuning, get Elkstack, if you want to read your logs and start analyzing them immediately, use Splunk. The license is free for 500MB logs per day, for me this is two firewalls and a lot of other stuff.

          Stop playing, start analyzing :)

          Security Consultant at Mint Security Ltd - www.mintsecurity.fi

          1 Reply Last reply Reply Quote 0
          • M
            muswellhillbilly
            last edited by

            Another link to a quick guide to setting up ELK with PFS.

            https://elijahpaul.co.uk/monitoring-pfsense-2-1-logs-using-elk-logstash-kibana-elasticsearch/

            I used this myself when setting up my own log server with my PFS setup.

            An updated version is here:

            https://elijahpaul.co.uk/updated-monitoring-pfsense-logs-using-elk-elasticsearch-logstash-kibana-part-1/

            1 Reply Last reply Reply Quote 0
            • A
              AR15USR
              last edited by

              @johnpoz:

              …I do with we could get someone to maintain a guide or vm that logs everything with nice dashboards already made, etc.  I keep meaning to make my setup available but it still needs a lot of tweaking to do everything extra pretty ;)

              Yes please!  ;D


              2.6.0-RELEASE

              1 Reply Last reply Reply Quote 0
              • mudmanc4M
                mudmanc4
                last edited by

                For those who do not use vmware

                Grabbed the .ova converted it to qemu2 , flawless. Nicely done.

                decompress:

                tar xvf PFELK01-160408.ova
                

                Convert:

                qemu-img convert -f vmdk -O qcow2 PFELK01-160408-disk1.vmdk PFELK01.qcow2
                

                Have a peek at the .ovf noticing the existing virtual CPU's and memory
                Which happen to be 4 CPU's and 8GB ram

                Import = done

                Change existing password from 'pf' to {your password}

                passwd
                

                Access at {IP:5601}

                Easypeasy

                1 Reply Last reply Reply Quote 0
                • johnpozJ
                  johnpoz LAYER 8 Global Moderator
                  last edited by

                  U might need change timezone too that was in utc while my pfsense in central

                  And u prob dont need 4 cpu and 8 gig running on only 2 and 2gb

                  An intelligent man is sometimes forced to be drunk to spend time with his fools
                  If you get confused: Listen to the Music Play
                  Please don't Chat/PM me for help, unless mod related
                  SG-4860 24.11 | Lab VMs 2.8, 24.11

                  1 Reply Last reply Reply Quote 0
                  • H
                    hamed_forum
                    last edited by

                    @muswellhillbilly:

                    Another link to a quick guide to setting up ELK with PFS.

                    https://elijahpaul.co.uk/monitoring-pfsense-2-1-logs-using-elk-logstash-kibana-elasticsearch/

                    I used this myself when setting up my own log server with my PFS setup.

                    An updated version is here:

                    https://elijahpaul.co.uk/updated-monitoring-pfsense-logs-using-elk-elasticsearch-logstash-kibana-part-1/

                    i do it many time but dont work

                    1 Reply Last reply Reply Quote 0
                    • H
                      hamed_forum
                      last edited by

                      @johnpoz:

                      it does work, have it running..

                      You just need to set it up correctly ;)

                      You can find a working vm here..
                      https://www.reddit.com/r/PFSENSE/comments/4dymci/i_made_a_simple_bare_bones_simple_elk_vm_for/

                      This will do firewall logs, there are many guides on the net, many of them missing pieces or not very clear.  But this vm works out of the box for firewall logs.  If you want other logs you might need to make some adjustments

                      I do with we could get someone to maintain a guide or vm that logs everything with nice dashboards already made, etc.  I keep meaning to make my setup available but it still needs a lot of tweaking to do everything extra pretty ;)

                      in pf sense only tike  firewall logs or everthing?
                      tanks you do great work

                      1 Reply Last reply Reply Quote 0
                      • A
                        AR15USR
                        last edited by

                        I got that converted to the .vmdk, but not sure how to translate that into a running VM. VirtualBox says it can't recognize that type of disk. Jonpoz or mudmanc4 can you expand a bit on your instructions for getting this working please? Thanks..

                        Im using VirtualBox 5.0.2 running a Ubuntu 16 VM, on MacOS 10.11.5


                        2.6.0-RELEASE

                        1 Reply Last reply Reply Quote 0
                        • johnpozJ
                          johnpoz LAYER 8 Global Moderator
                          last edited by

                          pretty sure virtualbox just lets you import a ova.

                          An intelligent man is sometimes forced to be drunk to spend time with his fools
                          If you get confused: Listen to the Music Play
                          Please don't Chat/PM me for help, unless mod related
                          SG-4860 24.11 | Lab VMs 2.8, 24.11

                          1 Reply Last reply Reply Quote 0
                          • A
                            AR15USR
                            last edited by

                            @johnpoz:

                            pretty sure virtualbox just lets you import a ova.

                            Failed to open the disk image file /Users/xxxxxxxxx/Downloads/PFELK01-160408.ova.
                            
                            Could not get the storage format of the medium '/Users/xxxxxxxxx/Downloads/PFELK01-160408.ova' (VERR_NOT_SUPPORTED).
                            
                            Result Code: VBOX_E_IPRT_ERROR (0x80BB0005)
                            Component: MediumWrap
                            Interface: IMedium {4afe423b-43e0-e9d0-82e8-ceb307940dda}
                            Callee: IVirtualBox {0169423f-46b4-cde9-91af-1e9d5b6cd945}
                            Callee RC: VBOX_E_OBJECT_NOT_FOUND (0x80BB0001)
                            
                            

                            2.6.0-RELEASE

                            1 Reply Last reply Reply Quote 0
                            • mudmanc4M
                              mudmanc4
                              last edited by

                              @AR15USR:

                              <snip>Im using VirtualBox 5.0.2 running a Ubuntu 16 VM, on MacOS 10.11.5</snip>

                              Virtual disk space issue?
                              So your running Virtualbox on OS X, where Ubuntu is a guest, then trying to import a 1.6GB image into Ubuntu?  Correct me?

                              1 Reply Last reply Reply Quote 0
                              • A
                                AR15USR
                                last edited by

                                @mudmanc4:

                                @AR15USR:

                                <snip>Im using VirtualBox 5.0.2 running a Ubuntu 16 VM, on MacOS 10.11.5</snip>

                                Virtual disk space issue?
                                So your running Virtualbox on OS X, where Ubuntu is a guest, then trying to import a 1.6GB image into Ubuntu?  Correct me?

                                Not sure exactly what you mean by "import a 1.6GB image into Ubuntu". I downloaded the .ova in my Ubuntu guest so I could convert to the .vmdk. From there I copied the .vmdk to OS X and tried to create a new VM guest based on Linux with 2 processors, 16gb ram, and the .vmdk as the virtual hard disk. When selecting the .vmdk as the virtual hard disk is when I get the error.


                                2.6.0-RELEASE

                                1 Reply Last reply Reply Quote 0
                                • mudmanc4M
                                  mudmanc4
                                  last edited by

                                  @AR15USR:

                                  Not sure exactly what you mean by "import a 1.6GB image into Ubuntu". I downloaded the .ova in my Ubuntu guest so I could convert to the .vmdk. From there I copied the .vmdk to OS X and tried to create a new VM guest based on Linux with 2 processors, 16gb ram, and the .vmdk as the virtual hard disk. When selecting the .vmdk as the virtual hard disk is when I get the error.

                                  Gotcha now.
                                  You could not import the .ova itself?

                                  1 Reply Last reply Reply Quote 0
                                  • A
                                    AR15USR
                                    last edited by

                                    @mudmanc4:

                                    @AR15USR:

                                    Not sure exactly what you mean by "import a 1.6GB image into Ubuntu". I downloaded the .ova in my Ubuntu guest so I could convert to the .vmdk. From there I copied the .vmdk to OS X and tried to create a new VM guest based on Linux with 2 processors, 16gb ram, and the .vmdk as the virtual hard disk. When selecting the .vmdk as the virtual hard disk is when I get the error.

                                    Gotcha now.
                                    You could not import the .ova itself?

                                    No errors out as well. What Virtual Machine software are you using?


                                    2.6.0-RELEASE

                                    1 Reply Last reply Reply Quote 0
                                    • mudmanc4M
                                      mudmanc4
                                      last edited by

                                      @AR15USR:

                                      No errors out as well. What Virtual Machine software are you using?

                                      https://virt-manager.org/

                                      Maybe it's corrupt in some way. I would just grab a new copy.

                                      1 Reply Last reply Reply Quote 0
                                      • A
                                        AR15USR
                                        last edited by

                                        @mudmanc4:

                                        @AR15USR:

                                        No errors out as well. What Virtual Machine software are you using?

                                        https://virt-manager.org/

                                        Maybe it's corrupt in some way. I would just grab a new copy.

                                        What version of Linux is the ova/vdmk based on? Maybe I was using the wrong type..


                                        2.6.0-RELEASE

                                        1 Reply Last reply Reply Quote 0
                                        • mudmanc4M
                                          mudmanc4
                                          last edited by

                                          @AR15USR:

                                          What version of Linux is the ova/vdmk based on? Maybe I was using the wrong type..

                                          I believe it is Ubuntu 14

                                          vmw:osType="ubuntu64Guest
                                          
                                          1 Reply Last reply Reply Quote 0
                                          • A
                                            AR15USR
                                            last edited by

                                            Hmm, since Splunk has a native OS X version I think I will start exploring that.


                                            2.6.0-RELEASE

                                            1 Reply Last reply Reply Quote 0
                                            • First post
                                              Last post
                                            Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.