[SOLVED] How to installing bpytop (former Bashtop) and adding it to Main Menu (menu in Console)?

Sergei_Shablovsky · Nov 24, 2020, 8:46 PM

@Gertjan said in How to installing Bashtop and adding it to Main Menu (menu in in Console)?:

@Sergei_Shablovsky said in How to installing Bashtop and adding it to Main Menu (menu in in Console)?:

rater spending a hours with unknown results

IMHO : this won't take you hours ;)
The code you shows looks just fine.

Thank You! I spend a little bit more time to remind my childhood with bash, and make a little bit improved (color coding, grouping by frequency of using and grouping by functions) version, please see below.

@Sergei_Shablovsky said in How to installing Bashtop and adding it to Main Menu (menu in in Console)?:

changes intact during periodically updates

There is no 100 % guaranted solution for this.

I know very well this phrase and hate it! :) Of course, You are absolutely right.

As long as you change just a couple of lines, I advise you to install the smallest pfSense package : Notes.

Use this to document your settings, with copies of fragments of changed files.
Put in also the URLS to these posts on the forum here.
The name of the Freebsd (pfsense ported) package used.
etc.

Thank You for suggestion! I doing this already before reading :) Anyway, THANK YOU!
Why this suggestion is not in official docs?????

You could 'diff' between the original files, and your files, and have this diff merged automatically during startup or even update (I use the 'Patches' package from pfSense).

Thank You for suggestion! I using the Kaleidoscope app on macOS. VERY PRO and USEFUL app. I try on a real coding work most of apps, but Kaleidoscope in conjunction with Tower - my choice, best of the best!

But what if Netgate decided to fundamentally change the /etc/rc.initial file in the next version ? The merge won't work out. Your back to manual again.

This is a life. Life is live. :) May be some stuff from Netgate look and just implement some of this ideas. Whit no charge from our side :)

So, my advice : take notes. Do not try to 'fix' in some way ever moving code.
If an upgrade comes in, it will take you 5 minutes to re patch.
Because you know now how to do it ;)

Thank You another one time. I'l doing this way.

Sergei_Shablovsky · Nov 24, 2020, 11:02 PM

This post is deleted!

kiokoman · Nov 25, 2020, 1:47 AM

the instructions was for 2.4.5-p1 i will check later for 2.5.0

kiokoman · Nov 25, 2020, 8:32 AM

2.5.0

pkg install bash
pkg add https://pkg.FreeBSD.org/FreeBSD:12:amd64/latest/All/gsed-4.8.txz
pkg add https://pkg.FreeBSD.org/FreeBSD:12:amd64/latest/All/coreutils-8.32.txz
pkg add https://pkg.FreeBSD.org/FreeBSD:12:amd64/latest/All/py37-psutil-5.7.3.txz
pkg add https://pkg.FreeBSD.org/FreeBSD:12:amd64/latest/All/bashtop-0.9.25_1.txz
ln -s /usr/local/bin/python3.7 /bin/python3
/usr/local/bin/bashtop

Sergei_Shablovsky · Nov 26, 2020, 5:56 PM

@kiokoman said in How to installing Bashtop and adding it to Main Menu (menu in Console)?:

2.5.0

pkg install bash
pkg add https://pkg.FreeBSD.org/FreeBSD:12:amd64/latest/All/gsed-4.8.txz
pkg add https://pkg.FreeBSD.org/FreeBSD:12:amd64/latest/All/coreutils-8.32.txz
pkg add https://pkg.FreeBSD.org/FreeBSD:12:amd64/latest/All/py37-psutil-5.7.3.txz
pkg add https://pkg.FreeBSD.org/FreeBSD:12:amd64/latest/All/bashtop-0.9.25_1.txz
ln -s /usr/local/bin/python3.7 /bin/python3
/usr/local/bin/bashtop

Thank You for help!

I was on road, and not able to answering at time, so we both doing the same simultaneously;)

All 4 packages downloaded and installed correctly, ln command - success, but the problem is after Bashtop started, the initial screen halt on last command:

Creating psutil coprocess.....  ok
Checking available tools......ok
Checking cpu......ok
Calculating sizes....ok
Running cpu collection init......


39f
39f
39f
39f
39f
39f

Please, do You have any idea how to know what happened with Bashtop process at all?

P.S. Carl+z and “exit” command after return me to initial screen with “cheap menu”.

kiokoman · Nov 26, 2020, 6:12 PM

did you try to launch bash before bashtop ?

/usr/local/bin/bash
/usr/local/bin/bashtop

it works and looks good on my 2.5.0 without bash

[2.5.0-DEVELOPMENT][root@pfSense.kiokoman.home]/root: pkg add https://pkg.FreeBSD.org/FreeBSD:12:amd64/latest/All/gsed-4.8.txz
Fetching gsed-4.8.txz: 100%  315 KiB 322.5kB/s    00:01
Installing gsed-4.8...
Extracting gsed-4.8: 100%
[2.5.0-DEVELOPMENT][root@pfSense.kiokoman.home]/root: pkg add https://pkg.FreeBSD.org/FreeBSD:12:amd64/latest/All/coreutils-8.32.txz
Fetching coreutils-8.32.txz: 100%    3 MiB   2.7MB/s    00:01
Installing coreutils-8.32...
Extracting coreutils-8.32: 100%
[2.5.0-DEVELOPMENT][root@pfSense.kiokoman.home]/root: pkg add https://pkg.FreeBSD.org/FreeBSD:12:amd64/latest/All/py37-psutil-5.7.3.txz
Fetching py37-psutil-5.7.3.txz: 100%  337 KiB 345.1kB/s    00:01
Installing py37-psutil-5.7.3...
Extracting py37-psutil-5.7.3: 100%
[2.5.0-DEVELOPMENT][root@pfSense.kiokoman.home]/root: pkg add https://pkg.FreeBSD.org/FreeBSD:12:amd64/latest/All/bashtop-0.9.25_1.txz
Fetching bashtop-0.9.25_1.txz: 100%   48 KiB  49.6kB/s    00:01
Installing bashtop-0.9.25_1...
Extracting bashtop-0.9.25_1: 100%
[2.5.0-DEVELOPMENT][root@pfSense.kiokoman.home]/root: ln -s /usr/local/bin/python3.7 /bin/python3
[2.5.0-DEVELOPMENT][root@pfSense.kiokoman.home]/root: /usr/local/bin/bashtop

you can try to launch bashtop with truss to see if you can understand why it show that "39f"

truss /usr/local/bin/bashtop

Sergei_Shablovsky · Nov 28, 2020, 7:18 PM

@kiokoman said in How to installing Bashtop and adding it to Main Menu (menu in Console)?:

did you try to launch bash before bashtop ?

/usr/local/bin/bash
/usr/local/bin/bashtop

it works and looks good on my 2.5.0 without bash

Thank You for suggestions!

I try

by choosing from menu (this is actually I want);
by starting Shell (menuitem “33” from menu above, pls seek in code), then trying /usr/local/bin/bashtop w/ and w/o /usr/local/bin/bash;

The result are the same,- stuck on the same “Running cpu collection init.....”

P.S. I notice that in pfSense Dashboard “CPU Usage” stuck on “Retrieving CPU data *”. Is this linked to something like “CPU Legacy” in UEFI ?

Sergei_Shablovsky · Nov 29, 2020, 8:39 PM

@Gertjan said in How to installing Bashtop and adding it to Main Menu (menu in Console)?:

Bashtop : did not find that word on this forum ... (using Google - you might try other).
So : there a two possibilities :
No one asks that question because everybody knows the answer already.
No one ever did - or even no one is planning to do so.
As for myself : up until know, I didn't even know what bashtop was ...

I have another one option an addition to that two You mention:

ordinary SysAdmins are too lazy to improving their skills and step out the boundaries. Because of this they never improving the good tool that working good already, and very rarely reading more outside the original software documentation.
Just absence of motivation in nowadays.

stephenw10 · Nov 29, 2020, 9:31 PM

Just to be clear nobody should ever do this on a production firewall without fully understanding the consequences of doing so.

This is a fun experiment and I'm all about that but should only be done on a test/development box.

If it's something that proved useful then a package could potentially be created that would be properly vetted and built against our repo.

Steve

Gertjan · Nov 30, 2020, 6:28 AM

@Sergei_Shablovsky :

A screen that shows the status of a lot of parameters.
Good.
But what happens when the system goes down ? Where is the screen ? Are you really want to look at the screen 24/24h to detect if something is about to go go bad, or something needs attention ?
Real admins automatize.

Means years ago I installed this on pfSense : Munin.

When my system goes down, I can actually see what happened after box died - or when the symptoms started. Munin even mails me when something goes off-scale.
And I can see the info no matter where I am ^^

Simple perl/bash/sh/php/whatever scripts can be added t add more charts.

Munin is part of the base FreeBSD and present in the pfSense repo - at least, is was so in the past.
It needs to be set up manually, as no GUI part exists.

Btw : other - better ? tools (might) exist. But hey, as you already said : we stick with what we (think !) we know.

edit : the main security issue is that the centralized Munin client - the site where I host all the charts of all my devices, connects every 5 minutes to a TCP port 4949 of my pfSense.
A WAN firewall rule accepts incoming TCP 4949 connections from one IP only : the IP from my munin client.
Also : most of the scripts used don't run with admin or root rights, they can only 'read'.

stephenw10 · Nov 30, 2020, 4:38 PM

Both tools look useful. Bashtop is mostly realtime, or close to it, Munin looks to be about logging histirical changes. I can imagine using both.

Sergei_Shablovsky · Nov 30, 2020, 4:38 PM

@stephenw10 said in How to installing Bashtop and adding it to Main Menu (menu in Console)?:

Both tools look useful. Bashtop is mostly realtime, or close to it, Munin looks to be about logging histirical changes. I can imagine using both.

As I note before, I just need to extend this cheap menu because it not useful nowadays:

quick initializing action (setup WANs and LANs interfaces quickly) when installing pfSense Bare Metal, or reinstalling after hardware failure;
be able to doing some base network measurement (ping, trace, iperf) just on local console, when the technician are physically near the hardware in DC;
have on local VGA/rs232/usb console REALTIME system health state (because when hardware failure, the System Management Module with watchdogs save all sensors data records and a screenshot into separate memory for future analyses (with other health records) the source of hardware problem;

Only servers has this System Managemet Module (I am sure about IBM, Dell, HP, and even some of Supermicro servers both Enterprise and Entry Level models), ordinary “home PC” - no.

The constantly health monitoring better to doing by separate tools, we prefer Splunk (agent-based system, for overall and detailed) and LibreNMS for detailed net flow (but also able overall, SMNP-based system). Here some examples of LibreNMS

So, the conclusion are: You are right, for serious Monitoring, professional SysAdmin need both local Monitoring tool constant remote metrics collection system.

All this “story with the rewriting the cheap Main Menu” just for ability to make first part - local overall health monitoring tool and several tool for base network testing

Reasonably?

Sergei_Shablovsky · Nov 30, 2020, 4:57 PM

@Gertjan said in How to installing Bashtop and adding it to Main Menu (menu in Console)?:

@Sergei_Shablovsky :

A screen that shows the status of a lot of parameters.
Good.
But what happens when the system goes down ? Where is the screen ? Are you really want to look at the screen 24/24h to detect if something is about to go go bad, or something needs attention ?

Most of servers from reputable brands (I am sure of IBM, Dell, HP) have internal built in Server Health Monitoring PCB/chip, and common function are constantly saving logs and screenshot before hardware crash/unattended restart in separate nonvioation memory, sending alerts by sms/email.

No one sitting behind the server after hi placed in DC or separate server room.

Real admins automatize.

Means years ago I installed this on pfSense : Munin.

When my system goes down, I can actually see what happened after box died - or when the symptoms started. Munin even mails me when something goes off-scale.

Munin give You symptoms, but not picture what happened in real-time, and also have some delay between issue and you receive the alert by sms/email.

The same but **much more detailed and tunable statistic and analyse with filters and ton of plugins” give You the Splunk system. Worth to take a closer look.

And I can see the info no matter where I am ^^

And a whole word with a hackers also. What about security ? No one bank, finance structure, law structure (even a small company) agree with this data open to all worldwide.

Simple perl/bash/sh/php/whatever scripts can be added t add more charts.

Munin is part of the base FreeBSD and present in the pfSense repo - at least, is was so in the past.
It needs to be set up manually, as no GUI part exists.

Btw : other - better ? tools (might) exist. But hey, as you already said : we stick with what we (think !) we know.

This our human habits mostly play a negative role in also fast running and changing IT-world. Especially when we speak about security and a Firewalling.

edit : the main security issue is that the centralized Munin client - the site where I host all the charts of all my devices, connects every 5 minutes to a TCP port 4949 of my pfSense.
A WAN firewall rule accepts incoming TCP 4949 connections from one IP only : the IP from my munin client.
Also : most of the scripts used don't run with admin or root rights, they can only 'read'.

Thank You for a suggestion. ;) May be usable for someone who read this thread and decide to setup Munin.

Sergei_Shablovsky · Dec 3, 2020, 1:15 AM

@Gertjan said in How to installing Bashtop and adding it to Main Menu (menu in Console)?:

As for myself : up until know, I didn't even know what bashtop was ...

I told You more: zsh and fish shells able to impress You if You work in command line.

And Bashtop, Pingtop, Hegemon, Gotop, htop, Gping, Vping, also able to give a You a little help if a You have a servers without (or just not need and prefer going to each server by ssh manually for maintenance) agent-server monitoring system like Netdata, LibreNMS, or Splunk.

But another one time:
all this tools is not for ordinary home using, only if You need 24/7 running service and clients and have an obligation.

Sergei_Shablovsky · Feb 11, 2021, 3:40 AM

@kiokoman said in How to installing Bashtop and adding it to Main Menu (menu in Console)?:

2.5.0

pkg install bash
pkg add https://pkg.FreeBSD.org/FreeBSD:12:amd64/latest/All/gsed-4.8.txz
pkg add https://pkg.FreeBSD.org/FreeBSD:12:amd64/latest/All/coreutils-8.32.txz
pkg add https://pkg.FreeBSD.org/FreeBSD:12:amd64/latest/All/py37-psutil-5.7.3.txz
pkg add https://pkg.FreeBSD.org/FreeBSD:12:amd64/latest/All/bashtop-0.9.25_1.txz
ln -s /usr/local/bin/python3.7 /bin/python3
/usr/local/bin/bashtop

On a follow system:

2.5.0-RC (amd64) 
built on Wed Feb 10 09:40:34 EST 2021 
FreeBSD 12.2-STABLE

The are next error:

Updating pfSense-core repository catalogue...
pkg: https://packages-beta.netgate.com/packages/pfSense_master_amd64-core/meta.txz: No address record
repository pfSense-core has no meta file, using default settings
pkg: https://packages-beta.netgate.com/packages/pfSense_master_amd64-core/packagesite.txz: No address record
Unable to update repository pfSense-core
Updating pfSense repository catalogue...
pkg: https://packages-beta.netgate.com/packages/pfSense_master_amd64-pfSense_devel/meta.txz: No address record
repository pfSense has no meta file, using default settings
pkg: https://packages-beta.netgate.com/packages/pfSense_master_amd64-pfSense_devel/packagesite.txz: No address record
Unable to update repository pfSense
Error updating repositories!

Soooo... How to install bashop now ? :)

Thank You for Your time and patience!

Gertjan · Feb 11, 2021, 6:00 AM

@sergei_shablovsky said in How to installing Bashtop and adding it to Main Menu (menu in Console)?:

Error updating repositories!

Before calling it a day, yesterday, I saw some posts signalling that the "repository " was out of order.
Probably some product name changes or big updates or ahead.
At least, in the "2.5.0" tree.
Btw : me just thinking here. The 2.4.5-p1 tree seems fine.

kiokoman · Feb 11, 2021, 8:18 AM

@Sergei_Shablovsky
my 2 cent, they are probably moving 2.5.0 out of beta ...

Gertjan · Feb 11, 2021, 8:29 AM

Yeah, I'm also afraid that that's what's coming up.

The 'when is it ready ?' public will get transformed in the 'help a problem' group.
The calm and all goes well period comes to an end.

Sergei_Shablovsky · Feb 11, 2021, 3:13 PM

@gertjan said in How to installing Bashtop and adding it to Main Menu (menu in Console)?:

@sergei_shablovsky said in How to installing Bashtop and adding it to Main Menu (menu in Console)?:

Error updating repositories!

Before calling it a day, yesterday, I saw some posts signalling that the "repository " was out of order.
Probably some product name changes or big updates or ahead.
At least, in the "2.5.0" tree.
Btw : me just thinking here. The 2.4.5-p1 tree seems fine.

Is any other way to install bpytop manually ?

Gertjan · Feb 11, 2021, 3:13 PM

@sergei_shablovsky Is that a package in the 'github' pfSense repository ?
Here https://github.com/pfsense/FreeBSD-ports ?

Installing manually packages on a must-always-be-connected-to-the-net-router is ...... awkward.

From higher levels (Netgate), very recently - on this forum - the advise was : don't.

You are using 2.5.0. : isn't it time to go vanilla ? Or prepare yourself for a re install. (not the moment to try try things out .....)

Btw : its doesn't exist for 2.4.5-p1 :

[2.4.5-RELEASE][root@pfsense.core2-wall.localnet]/root: pkg install bpytop
Updating pfSense-core repository catalogue...
pfSense-core repository is up to date.
Updating pfSense repository catalogue...
pfSense repository is up to date.
All repositories are up to date.
Updating database digests format: 100%
pkg: No packages available to install matching 'bpytop' have been found in the repositories