[SOLVED] How to installing bpytop (former Bashtop) and adding it to Main Menu (menu in Console)?

Sergei_Shablovsky · Nov 28, 2020, 7:18 PM

@kiokoman said in How to installing Bashtop and adding it to Main Menu (menu in Console)?:

did you try to launch bash before bashtop ?

/usr/local/bin/bash
/usr/local/bin/bashtop

it works and looks good on my 2.5.0 without bash

Thank You for suggestions!

I try

by choosing from menu (this is actually I want);
by starting Shell (menuitem “33” from menu above, pls seek in code), then trying /usr/local/bin/bashtop w/ and w/o /usr/local/bin/bash;

The result are the same,- stuck on the same “Running cpu collection init.....”

P.S. I notice that in pfSense Dashboard “CPU Usage” stuck on “Retrieving CPU data *”. Is this linked to something like “CPU Legacy” in UEFI ?

Sergei_Shablovsky · Nov 29, 2020, 8:39 PM

@Gertjan said in How to installing Bashtop and adding it to Main Menu (menu in Console)?:

Bashtop : did not find that word on this forum ... (using Google - you might try other).
So : there a two possibilities :
No one asks that question because everybody knows the answer already.
No one ever did - or even no one is planning to do so.
As for myself : up until know, I didn't even know what bashtop was ...

I have another one option an addition to that two You mention:

ordinary SysAdmins are too lazy to improving their skills and step out the boundaries. Because of this they never improving the good tool that working good already, and very rarely reading more outside the original software documentation.
Just absence of motivation in nowadays.

stephenw10 · Nov 29, 2020, 9:31 PM

Just to be clear nobody should ever do this on a production firewall without fully understanding the consequences of doing so.

This is a fun experiment and I'm all about that but should only be done on a test/development box.

If it's something that proved useful then a package could potentially be created that would be properly vetted and built against our repo.

Steve

Gertjan · Nov 30, 2020, 6:28 AM

@Sergei_Shablovsky :

A screen that shows the status of a lot of parameters.
Good.
But what happens when the system goes down ? Where is the screen ? Are you really want to look at the screen 24/24h to detect if something is about to go go bad, or something needs attention ?
Real admins automatize.

Means years ago I installed this on pfSense : Munin.

When my system goes down, I can actually see what happened after box died - or when the symptoms started. Munin even mails me when something goes off-scale.
And I can see the info no matter where I am ^^

Simple perl/bash/sh/php/whatever scripts can be added t add more charts.

Munin is part of the base FreeBSD and present in the pfSense repo - at least, is was so in the past.
It needs to be set up manually, as no GUI part exists.

Btw : other - better ? tools (might) exist. But hey, as you already said : we stick with what we (think !) we know.

edit : the main security issue is that the centralized Munin client - the site where I host all the charts of all my devices, connects every 5 minutes to a TCP port 4949 of my pfSense.
A WAN firewall rule accepts incoming TCP 4949 connections from one IP only : the IP from my munin client.
Also : most of the scripts used don't run with admin or root rights, they can only 'read'.

stephenw10 · Nov 30, 2020, 4:38 PM

Both tools look useful. Bashtop is mostly realtime, or close to it, Munin looks to be about logging histirical changes. I can imagine using both.

Sergei_Shablovsky · Nov 30, 2020, 4:38 PM

@stephenw10 said in How to installing Bashtop and adding it to Main Menu (menu in Console)?:

Both tools look useful. Bashtop is mostly realtime, or close to it, Munin looks to be about logging histirical changes. I can imagine using both.

As I note before, I just need to extend this cheap menu because it not useful nowadays:

quick initializing action (setup WANs and LANs interfaces quickly) when installing pfSense Bare Metal, or reinstalling after hardware failure;
be able to doing some base network measurement (ping, trace, iperf) just on local console, when the technician are physically near the hardware in DC;
have on local VGA/rs232/usb console REALTIME system health state (because when hardware failure, the System Management Module with watchdogs save all sensors data records and a screenshot into separate memory for future analyses (with other health records) the source of hardware problem;

Only servers has this System Managemet Module (I am sure about IBM, Dell, HP, and even some of Supermicro servers both Enterprise and Entry Level models), ordinary “home PC” - no.

The constantly health monitoring better to doing by separate tools, we prefer Splunk (agent-based system, for overall and detailed) and LibreNMS for detailed net flow (but also able overall, SMNP-based system). Here some examples of LibreNMS

So, the conclusion are: You are right, for serious Monitoring, professional SysAdmin need both local Monitoring tool constant remote metrics collection system.

All this “story with the rewriting the cheap Main Menu” just for ability to make first part - local overall health monitoring tool and several tool for base network testing

Reasonably?

Sergei_Shablovsky · Nov 30, 2020, 4:57 PM

@Gertjan said in How to installing Bashtop and adding it to Main Menu (menu in Console)?:

@Sergei_Shablovsky :

A screen that shows the status of a lot of parameters.
Good.
But what happens when the system goes down ? Where is the screen ? Are you really want to look at the screen 24/24h to detect if something is about to go go bad, or something needs attention ?

Most of servers from reputable brands (I am sure of IBM, Dell, HP) have internal built in Server Health Monitoring PCB/chip, and common function are constantly saving logs and screenshot before hardware crash/unattended restart in separate nonvioation memory, sending alerts by sms/email.

No one sitting behind the server after hi placed in DC or separate server room.

Real admins automatize.

Means years ago I installed this on pfSense : Munin.

When my system goes down, I can actually see what happened after box died - or when the symptoms started. Munin even mails me when something goes off-scale.

Munin give You symptoms, but not picture what happened in real-time, and also have some delay between issue and you receive the alert by sms/email.

The same but **much more detailed and tunable statistic and analyse with filters and ton of plugins” give You the Splunk system. Worth to take a closer look.

And I can see the info no matter where I am ^^

And a whole word with a hackers also. What about security ? No one bank, finance structure, law structure (even a small company) agree with this data open to all worldwide.

Simple perl/bash/sh/php/whatever scripts can be added t add more charts.

Munin is part of the base FreeBSD and present in the pfSense repo - at least, is was so in the past.
It needs to be set up manually, as no GUI part exists.

Btw : other - better ? tools (might) exist. But hey, as you already said : we stick with what we (think !) we know.

This our human habits mostly play a negative role in also fast running and changing IT-world. Especially when we speak about security and a Firewalling.

edit : the main security issue is that the centralized Munin client - the site where I host all the charts of all my devices, connects every 5 minutes to a TCP port 4949 of my pfSense.
A WAN firewall rule accepts incoming TCP 4949 connections from one IP only : the IP from my munin client.
Also : most of the scripts used don't run with admin or root rights, they can only 'read'.

Thank You for a suggestion. ;) May be usable for someone who read this thread and decide to setup Munin.

Sergei_Shablovsky · Dec 3, 2020, 1:15 AM

@Gertjan said in How to installing Bashtop and adding it to Main Menu (menu in Console)?:

As for myself : up until know, I didn't even know what bashtop was ...

I told You more: zsh and fish shells able to impress You if You work in command line.

And Bashtop, Pingtop, Hegemon, Gotop, htop, Gping, Vping, also able to give a You a little help if a You have a servers without (or just not need and prefer going to each server by ssh manually for maintenance) agent-server monitoring system like Netdata, LibreNMS, or Splunk.

But another one time:
all this tools is not for ordinary home using, only if You need 24/7 running service and clients and have an obligation.

Sergei_Shablovsky · Feb 11, 2021, 3:40 AM

@kiokoman said in How to installing Bashtop and adding it to Main Menu (menu in Console)?:

2.5.0

pkg install bash
pkg add https://pkg.FreeBSD.org/FreeBSD:12:amd64/latest/All/gsed-4.8.txz
pkg add https://pkg.FreeBSD.org/FreeBSD:12:amd64/latest/All/coreutils-8.32.txz
pkg add https://pkg.FreeBSD.org/FreeBSD:12:amd64/latest/All/py37-psutil-5.7.3.txz
pkg add https://pkg.FreeBSD.org/FreeBSD:12:amd64/latest/All/bashtop-0.9.25_1.txz
ln -s /usr/local/bin/python3.7 /bin/python3
/usr/local/bin/bashtop

On a follow system:

2.5.0-RC (amd64) 
built on Wed Feb 10 09:40:34 EST 2021 
FreeBSD 12.2-STABLE

The are next error:

Updating pfSense-core repository catalogue...
pkg: https://packages-beta.netgate.com/packages/pfSense_master_amd64-core/meta.txz: No address record
repository pfSense-core has no meta file, using default settings
pkg: https://packages-beta.netgate.com/packages/pfSense_master_amd64-core/packagesite.txz: No address record
Unable to update repository pfSense-core
Updating pfSense repository catalogue...
pkg: https://packages-beta.netgate.com/packages/pfSense_master_amd64-pfSense_devel/meta.txz: No address record
repository pfSense has no meta file, using default settings
pkg: https://packages-beta.netgate.com/packages/pfSense_master_amd64-pfSense_devel/packagesite.txz: No address record
Unable to update repository pfSense
Error updating repositories!

Soooo... How to install bashop now ? :)

Thank You for Your time and patience!

Gertjan · Feb 11, 2021, 6:00 AM

@sergei_shablovsky said in How to installing Bashtop and adding it to Main Menu (menu in Console)?:

Error updating repositories!

Before calling it a day, yesterday, I saw some posts signalling that the "repository " was out of order.
Probably some product name changes or big updates or ahead.
At least, in the "2.5.0" tree.
Btw : me just thinking here. The 2.4.5-p1 tree seems fine.

kiokoman · Feb 11, 2021, 8:18 AM

@Sergei_Shablovsky
my 2 cent, they are probably moving 2.5.0 out of beta ...

Gertjan · Feb 11, 2021, 8:29 AM

Yeah, I'm also afraid that that's what's coming up.

The 'when is it ready ?' public will get transformed in the 'help a problem' group.
The calm and all goes well period comes to an end.

Sergei_Shablovsky · Feb 11, 2021, 3:13 PM

@gertjan said in How to installing Bashtop and adding it to Main Menu (menu in Console)?:

@sergei_shablovsky said in How to installing Bashtop and adding it to Main Menu (menu in Console)?:

Error updating repositories!

Before calling it a day, yesterday, I saw some posts signalling that the "repository " was out of order.
Probably some product name changes or big updates or ahead.
At least, in the "2.5.0" tree.
Btw : me just thinking here. The 2.4.5-p1 tree seems fine.

Is any other way to install bpytop manually ?

Gertjan · Feb 11, 2021, 3:13 PM

@sergei_shablovsky Is that a package in the 'github' pfSense repository ?
Here https://github.com/pfsense/FreeBSD-ports ?

Installing manually packages on a must-always-be-connected-to-the-net-router is ...... awkward.

From higher levels (Netgate), very recently - on this forum - the advise was : don't.

You are using 2.5.0. : isn't it time to go vanilla ? Or prepare yourself for a re install. (not the moment to try try things out .....)

Btw : its doesn't exist for 2.4.5-p1 :

[2.4.5-RELEASE][root@pfsense.core2-wall.localnet]/root: pkg install bpytop
Updating pfSense-core repository catalogue...
pfSense-core repository is up to date.
Updating pfSense repository catalogue...
pfSense repository is up to date.
All repositories are up to date.
Updating database digests format: 100%
pkg: No packages available to install matching 'bpytop' have been found in the repositories

kiokoman · Feb 11, 2021, 3:31 PM

ahh @Sergei_Shablovsky, you need to be more creative !

pkg add https://pkg.FreeBSD.org/FreeBSD:12:amd64/latest/All/py37-psutil-5.8.0.txz
pkg add https://pkg.FreeBSD.org/FreeBSD:12:amd64/latest/All/bpytop-1.0.61.txz

open a ticket and ask jimp to add it if you like bpy that much, i don't think he will bite you.. maybe ...

Gertjan · Feb 11, 2021, 3:43 PM

@kiokoman said in How to installing bpytop (former Bashtop) and adding it to Main Menu (menu in Console)?:

i don't think he will bite you.. maybe ...

You said it yourself : we entered an exceptional moment.

( !!!! Just one left !!!! (February 11, 2021 - 4:38 pm UTC )

edit : oops : this is the son of a b****** bug, with 5 years of history.

stephenw10 · Feb 11, 2021, 3:48 PM

You are set to pull from master still which is now 2.6.
Set your update repo to 2.5.0 RC and you should see something like:

[2.5.0-RC][admin@25dev.stevew.lan]/root: pkg -d update
DBG(1)[11583]> pkg initialized
Updating pfSense-core repository catalogue...
DBG(1)[11583]> PkgRepo: verifying update for pfSense-core
DBG(1)[11583]> Pkgrepo, begin update of '/root/var/db/pkg/repo-pfSense-core.sqlite'
DBG(1)[11583]> Request to fetch pkg+https://packages.netgate.com/pfSense_v2_5_0_amd64-core/meta.conf
DBG(1)[11583]> opening libfetch fetcher
DBG(1)[11583]> Fetch > libfetch: connecting
DBG(1)[11583]> Fetch: fetching from: https://files01.netgate.com/pfSense_v2_5_0_amd64-core/meta.conf with opts "i"
DBG(1)[11583]> Fetch: fetcher chosen: https
DBG(1)[11583]> Request to fetch pkg+https://packages.netgate.com/pfSense_v2_5_0_amd64-core/packagesite.txz
DBG(1)[11583]> opening libfetch fetcher
DBG(1)[11583]> Fetch > libfetch: connecting
DBG(1)[11583]> Fetch: fetching from: https://files01.netgate.com/pfSense_v2_5_0_amd64-core/packagesite.txz with opts "i"
DBG(1)[11583]> Fetch: fetcher chosen: https
pfSense-core repository is up to date.
Updating pfSense repository catalogue...
DBG(1)[11583]> PkgRepo: verifying update for pfSense
DBG(1)[11583]> Pkgrepo, begin update of '/root/var/db/pkg/repo-pfSense.sqlite'
DBG(1)[11583]> Request to fetch pkg+https://packages.netgate.com/pfSense_v2_5_0_amd64-pfSense_v2_5_0/meta.conf
DBG(1)[11583]> opening libfetch fetcher
DBG(1)[11583]> Fetch > libfetch: connecting
DBG(1)[11583]> Fetch: fetching from: https://files00.netgate.com/pfSense_v2_5_0_amd64-pfSense_v2_5_0/meta.conf with opts "i"
DBG(1)[11583]> Fetch: fetcher chosen: https
DBG(1)[11583]> Request to fetch pkg+https://packages.netgate.com/pfSense_v2_5_0_amd64-pfSense_v2_5_0/packagesite.txz
DBG(1)[11583]> opening libfetch fetcher
DBG(1)[11583]> Fetch > libfetch: connecting
DBG(1)[11583]> Fetch: fetching from: https://files00.netgate.com/pfSense_v2_5_0_amd64-pfSense_v2_5_0/packagesite.txz with opts "i"
DBG(1)[11583]> Fetch: fetcher chosen: https
pfSense repository is up to date.
All repositories are up to date.

Steve

Sergei_Shablovsky · Feb 11, 2021, 7:11 PM

@kiokoman said in How to installing bpytop (former Bashtop) and adding it to Main Menu (menu in Console)?:

ahh @Sergei_Shablovsky, you need to be more creative !
pkg add https://pkg.FreeBSD.org/FreeBSD:12:amd64/latest/All/py37-psutil-5.8.0.txz
pkg add https://pkg.FreeBSD.org/FreeBSD:12:amd64/latest/All/bpytop-1.0.61.txz
open a ticket and ask jimp to add it if you like bpy that much, i don't think he will bite you.. maybe ...

I still thinking that bpytop are so informative and flexible comparing to native pfSense top/pftop, so may be jimp would be agree to include it as standard tool for on-screen Monitoring....

P.S. The same for Gping/vping, but on 3 month ago they are not available for FreeBSD. Sad but true...

stephenw10 · Feb 11, 2021, 8:56 PM

Did you open a feature request yet? https://redmine.pfsense.org/

Sergei_Shablovsky · Feb 12, 2021, 11:49 AM

@stephenw10 said in How to installing bpytop (former Bashtop) and adding it to Main Menu (menu in Console)?:

Did you open a feature request yet? https://redmine.pfsense.org/

Right now no. But I would be happy when You doing this (for example)... :)

Due You have a much more valuable reputation in community :)