• Categories
  • Recent
  • Tags
  • Popular
  • Users
  • Search
  • Register
  • Login
Netgate Discussion Forum
  • Categories
  • Recent
  • Tags
  • Popular
  • Users
  • Search
  • Register
  • Login

Graylog server on a raspberry pi

Scheduled Pinned Locked Moved Off-Topic & Non-Support Discussion
logging
11 Posts 5 Posters 3.9k Views
Loading More Posts
  • Oldest to Newest
  • Newest to Oldest
  • Most Votes
Reply
  • Reply as topic
Log in to reply
This topic has been deleted. Only users with topic management privileges can see it.
  • M
    mcury
    last edited by mcury Oct 20, 2022, 2:11 PM Oct 20, 2022, 1:27 PM

    The settings here are for small networks only!

    How to install Graylog server in a Raspberry Pi 4b, Ubuntu Server 20.04.5 LTS:
    Use at least a 4GB model, 8GB recommended.

    I didn't test this tutorial in a VM, other device and OS.

    Update your device:

    sudo apt update
    sudo apt upgrade
    

    Install dependencies:

    sudo apt install apt-transport-https openjdk-17-jre-headless uuid-runtime pwgen
    

    mongodb 4.4 install:

    wget -qO - https://www.mongodb.org/static/pgp/server-4.4.asc | sudo apt-key add -
    echo "deb [ arch=amd64,arm64 ] https://repo.mongodb.org/apt/ubuntu focal/mongodb-org/4.4 multiverse" | sudo tee /etc/apt/sources.list.d/mongodb-org-4.4.list
    sudo apt update
    sudo apt install mongodb-org
    sudo systemctl daemon-reload
    sudo systemctl enable mongod.service
    sudo systemctl restart mongod.service
    

    ElasticSearch install:

    wget -q https://artifacts.elastic.co/GPG-KEY-elasticsearch -O myKey
    sudo apt-key add myKey
    echo "deb https://artifacts.elastic.co/packages/oss-7.x/apt stable main" | sudo tee -a /etc/apt/sources.list.d/elastic-7.x.list
    sudo apt update 
    sudo apt install elasticsearch-oss
    

    ElasticSearch configuration: Copy and paste the following:

    sudo tee -a /etc/elasticsearch/elasticsearch.yml > /dev/null <<EOT
    cluster.name: graylog
    action.auto_create_index: false
    EOT
    

    ElasticSearch enable service:

    sudo systemctl daemon-reload
    sudo systemctl enable elasticsearch.service
    sudo systemctl restart elasticsearch.service
    sudo systemctl --type=service --state=active | grep elasticsearch
    

    Graylog install: (version 5.0 is alpha, 4.3 is the latest stable):

    wget https://packages.graylog2.org/repo/packages/graylog-4.3-repository_latest.deb
    sudo dpkg -i graylog-4.3-repository_latest.deb
    sudo apt update
    sudo apt install graylog-server
    

    Check /etc/graylog/server/server.conf for password_secret and root_password_sha2 and http_bind_address, read https://docs.graylog.org/docs/ubuntu for more details.

    Enable graylog

    sudo systemctl daemon-reload
    sudo systemctl enable graylog-server.service
    sudo systemctl start graylog-server.service
    sudo systemctl --type=service --state=active | grep graylog
    

    At this time, Graylog will be running and you can access it through the browser http://ip:9000

    user: admin
    password: admin

    Configure pfsense to send logs to Graylog:

    Go to : Status>System Logs>Settings and configure similar to this:

    b2076329-9cee-45d5-bdba-48ec2251507a-image.png

    Configure Graylog to receive the logs

    Create input:

    Go to System/Inputs / Inputs:
    Drop down menu, select Syslog UDP and launch the new input.

    41efdaa8-24eb-4d3b-907a-e5217e429e4b-image.png

    Then click save.

    Create Index:
    Go to System/Inputs / Index: (This is keep your data for 3 days only and it will rotate daily).

    5fe6ce6a-97d0-4c5a-8b47-edaabfbd782b-image.png

    Create stream:

    Go to Streams and click, create stream:

    80a9ec85-5012-48c3-b269-ad0ecec2d4d2-image.png

    Now, you need to get gl2_source_input value, this will allow the data to land in the correct stream in Graylog, go to System / Inputs, click in show messages, copy the field in the search:

    c06fa4b3-5c42-4d35-8f9d-720bc66fdecf-image.png

    Go to Streams, Manage rules and paste the info gathered in the previous step, similar to this:

    3f2a7329-6bdc-454e-9d41-4940e5e80a00-image.png

    Check the Extractor link at the bottom of this tutorial to correctly parse the data inside Graylog.
    There are other extractors, feel free to change in case you want, I like this one.

    Now, you can configured personalized searches and the dashboard to your liking.

    Note: You can configure other devices to send logs to Graylog.

    Don't forget to change your admin password.
    Use SSL/TLS (optional but highly recommended)

    Example of a dashboard:
    4fe49198-5559-477b-a01e-172b1f198a48-image.png


    Credits to Lawrence System, who first showed me Graylog.

    Links:
    Graylog: https://docs.graylog.org/docs/ubuntu (Links are not up to date, so use the links above)
    Graylog SSL/HTTPS https://docs.graylog.org/docs/https
    Extractor URL: https://github.com/loganmarchione/Graylog_Extractors_pfSense
    Mongodb: https://www.mongodb.com/docs/v4.4/tutorial/install-mongodb-on-ubuntu/
    Lawrence System: https://www.youtube.com/watch?v=rtfj6W5X0YA

    dead on arrival, nowhere to be found.

    J 1 Reply Last reply Oct 20, 2022, 2:47 PM Reply Quote 1
    • M mcury referenced this topic on Oct 20, 2022, 1:28 PM
    • M mcury referenced this topic on Oct 20, 2022, 1:28 PM
    • J
      johnpoz LAYER 8 Global Moderator @mcury
      last edited by johnpoz Oct 20, 2022, 2:48 PM Oct 20, 2022, 2:47 PM

      @mcury looks pretty straight forward walkthru - but you might want to either share your dashboard, or walk thru how you created yours as an example so users could get something up and running quick that shows them interesting info.

      I have played with graylog in the past, and it has a pretty steep learning curve..

      An intelligent man is sometimes forced to be drunk to spend time with his fools
      If you get confused: Listen to the Music Play
      Please don't Chat/PM me for help, unless mod related
      SG-4860 24.11 | Lab VMs 2.7.2, 24.11

      M 1 Reply Last reply Oct 20, 2022, 3:02 PM Reply Quote 0
      • M
        mcury @johnpoz
        last edited by Oct 20, 2022, 3:02 PM

        @johnpoz said in Graylog server on a raspberry pi:

        looks pretty straight forward walkthru

        :)


        Dashboard:

        Top inbound IP:
        Note: mvneta2 and mvneta0 are my WANs interfaces; 192.168.255.245 is removed from the graphic:
        6a373854-ff3c-4df7-b09f-6e1dbdbe7149-image.png

        Top outbound IP: Excluding LAN 192.168.255.2* and 8.8.* from the graphic.
        0e7afe10-ac98-4b0c-b556-8ec867265521-image.png

        TOP inbound TCP:S WAN: I like this one to see which port the Internet is hitting mostly:
        f0f26b93-6ac5-4591-ae7b-ebc53d0915ea-image.png

        Reputable blocklists: This is the tracking ID of the firewall rule created by pfblockerNG:
        5bb1391d-679e-4688-afdc-c0dfc2da1e69-image.png

        pfsense logs: Removing firewall logs from it, to get a clean view of the logs (system, dhcp and etc):
        a56759b8-bda9-486d-8e7d-684a80126c0b-image.png

        Synology logs: So few logs that you don't need a specific filter for it:
        f51d48b0-0a92-4a5a-afe1-481fca22e1e6-image.png

        dead on arrival, nowhere to be found.

        keyserK 1 Reply Last reply Oct 20, 2022, 3:09 PM Reply Quote 0
        • keyserK
          keyser Rebel Alliance @mcury
          last edited by Oct 20, 2022, 3:09 PM

          @mcury Interesting post and thanks for sharing. Personally I have been a little weary of installing a major syslog server/log analytics tool because of the learning curve and emmense amount of “framework” configuration you need to make - in order for the product to actually be usefull.

          But perhaps i’ll give a spin one of these days….

          Love the no fuss of using the official appliances :-)

          M 1 Reply Last reply Oct 20, 2022, 3:15 PM Reply Quote 0
          • M
            mcury @keyser
            last edited by Oct 20, 2022, 3:15 PM

            @keyser said in Graylog server on a raspberry pi:

            @mcury Interesting post and thanks for sharing. Personally I have been a little weary of installing a major syslog server/log analytics tool because of the learning curve and emmense amount of “framework” configuration you need to make - in order for the product to actually be usefull.

            But perhaps i’ll give a spin one of these days….

            It is a nice tool. In case you have any doubts about this tutorial, or any other questions, just ask and I'll try to help..

            dead on arrival, nowhere to be found.

            1 Reply Last reply Reply Quote 1
            • B
              betrimble
              last edited by Dec 28, 2022, 9:57 PM

              Has the mongo install broke? Tried to run this and all I get is the following:

              betrimble@graylog:~$ wget -qO - https://www.mongodb.org/static/pgp/server-4.4.asc | sudo apt-key add -
              echo "deb [ arch=amd64,arm64 ] https://repo.mongodb.org/apt/ubuntu focal/mongodb-org/4.4 multiverse" | sudo tee /etc/apt/sources.list.d/mongodb-org-4.4.list
              sudo apt update
              sudo apt install mongodb-org
              sudo systemctl daemon-reload
              sudo systemctl enable mongod.service
              sudo systemctl restart mongod.service
              Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)).
              OK
              deb [ arch=amd64,arm64 ] https://repo.mongodb.org/apt/ubuntu focal/mongodb-org/4.4 multiverse
              Hit:1 http://us.archive.ubuntu.com/ubuntu jammy InRelease
              Hit:2 http://us.archive.ubuntu.com/ubuntu jammy-updates InRelease
              Hit:3 http://us.archive.ubuntu.com/ubuntu jammy-backports InRelease
              Ign:4 https://repo.mongodb.org/apt/ubuntu focal/mongodb-org/4.4 InRelease
              Hit:5 http://us.archive.ubuntu.com/ubuntu jammy-security InRelease
              Hit:6 https://repo.mongodb.org/apt/ubuntu focal/mongodb-org/4.4 Release
              Reading package lists... Done
              Building dependency tree... Done
              Reading state information... Done
              All packages are up to date.
              W: https://repo.mongodb.org/apt/ubuntu/dists/focal/mongodb-org/4.4/Release.gpg: Key is stored in legacy trusted.gpg keyring (/etc/apt/trusted.gpg), see the DEPRECATION section in apt-key(8) for details.
              Reading package lists... Done
              Building dependency tree... Done
              Reading state information... Done
              Some packages could not be installed. This may mean that you have
              requested an impossible situation or if you are using the unstable
              distribution that some required packages have not yet been created
              or been moved out of Incoming.
              The following information may help to resolve the situation:

              The following packages have unmet dependencies:
              mongodb-org-mongos : Depends: libssl1.1 (>= 1.1.0) but it is not installable
              mongodb-org-server : Depends: libssl1.1 (>= 1.1.0) but it is not installable
              mongodb-org-shell : Depends: libssl1.1 (>= 1.1.0) but it is not installable
              E: Unable to correct problems, you have held broken packages.
              Failed to enable unit: Unit file mongod.service does not exist.
              Failed to restart mongod.service: Unit mongod.service not found.

              M 1 Reply Last reply Dec 28, 2022, 10:56 PM Reply Quote 0
              • M
                mcury @betrimble
                last edited by Dec 28, 2022, 10:56 PM

                @betrimble said in Graylog server on a raspberry pi:

                mongodb-org-mongos : Depends: libssl1.1 (>= 1.1.0) but it is not installable
                mongodb-org-server : Depends: libssl1.1 (>= 1.1.0) but it is not installable
                mongodb-org-shell : Depends: libssl1.1 (>= 1.1.0) but it is not installable

                You are running Ubuntu 22.04.1 LTS and not 20.04.5 ?
                If that is the case, try the step in the last post in this thread:

                https://github.com/dotnet/sdk/issues/25441 or downgrade the OS to 20.05.5

                dead on arrival, nowhere to be found.

                M 1 Reply Last reply Dec 29, 2022, 8:31 PM Reply Quote 0
                • M
                  michmoor LAYER 8 Rebel Alliance @mcury
                  last edited by Dec 29, 2022, 8:31 PM

                  @mcury Great job on the write up. Ive been using graylog for a bit over a year and the hardest part was creating the dashboards. The documentation wasnt great so def a lot of trial and error. I think you did a great job showing the steps on how to do this.
                  Truthfully, the hardest part of graylog so far [community edition] is finding extracters. For example, im on the hunt for Suricata extractor. I can get the eve json sent to graylog but its still a mess. My goal is to tie that in with GeoIP to get a pretty dashboard maybe. Been to tied up with other projects to continue researching.

                  Firewall: NetGate,Palo Alto-VM,Juniper SRX
                  Routing: Juniper, Arista, Cisco
                  Switching: Juniper, Arista, Cisco
                  Wireless: Unifi, Aruba IAP
                  JNCIP,CCNP Enterprise

                  M 1 Reply Last reply Dec 29, 2022, 9:46 PM Reply Quote 0
                  • M
                    mcury @michmoor
                    last edited by Dec 29, 2022, 9:46 PM

                    @michmoor They just released Graylog 5.0. It works with Ubuntu 22.04

                    I can't test it here because it depends on mongodb 5.0, which is not compatible with raspberry.
                    MongoDB 5.0 requires ARM v8.2-A or later and the Raspberry Pi 4 uses an ARM Cortex-A72 3 which is ARM v8-A.

                    I suppose you can build mongodb by yourself using the correct flags, but I didn't try it.

                    If you are running on a x86 system you can run Graylog 5.0.
                    All you need to change is the following:

                    mongodb 5.0 install: (note here that I removed arm64, test only in x86 system).

                    wget -qO - https://www.mongodb.org/static/pgp/server-5.0.asc | sudo apt-key add -
                    echo "deb [ arch=amd64 ] https://repo.mongodb.org/apt/ubuntu focal/mongodb-org/5.0 multiverse" | sudo tee /etc/apt/sources.list.d/mongodb-org-5.0.list
                    sudo apt update
                    sudo apt install mongodb-org
                    

                    Graylog install

                    wget https://packages.graylog2.org/repo/packages/graylog-5.0-repository_latest.deb
                    sudo dpkg -i graylog-5.0-repository_latest.deb
                    sudo apt-get update && sudo apt-get install graylog-server 
                    

                    Everything else in the first post remains the same.

                    dead on arrival, nowhere to be found.

                    1 Reply Last reply Reply Quote 0
                    • M
                      michmoor LAYER 8 Rebel Alliance
                      last edited by Dec 29, 2022, 9:49 PM

                      @mcury Yep i got 5.0 installed today actually. Just reading through the changelogs.
                      Ive been curious about one thing tho...
                      The 'count' in your charts. Should we assume thats how many sessions were created on the firewall, i.e. how many times a packet hit that rule?
                      I got my kids playing Roblox and.....

                      aa9b276f-2242-4b61-b4d4-2a4e1fe202ec-image.png 8fa479d5-adfc-4eb6-b6f8-f5a364848a9d-image.png

                      Firewall: NetGate,Palo Alto-VM,Juniper SRX
                      Routing: Juniper, Arista, Cisco
                      Switching: Juniper, Arista, Cisco
                      Wireless: Unifi, Aruba IAP
                      JNCIP,CCNP Enterprise

                      M 1 Reply Last reply Dec 29, 2022, 10:01 PM Reply Quote 0
                      • M
                        mcury @michmoor
                        last edited by Dec 29, 2022, 10:01 PM

                        @michmoor said in Graylog server on a raspberry pi:

                        The 'count' in your charts. Should we assume thats how many sessions were created on the firewall, i.e. how many times a packet hit that rule?

                        Based on what I've observed so far, this would be the same thing you would see in System logs > Firewall in Pfsense logs.

                        Since its a game, it is probably using UDP, right? I never played Roblox.. So I can't tell.

                        You can click the play button inside this chart to take a look at each of those entries to check.

                        dead on arrival, nowhere to be found.

                        1 Reply Last reply Reply Quote 0
                        • First post
                          Last post
                        Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.
                          This community forum collects and processes your personal information.
                          consent.not_received