historical log for WAN latency and performance

[darkcorner]

I have been intervening for three months for random problems that are difficult to determine.
I am sure that many of these (or perhaps all of them) depend on an unstable Internet line.
When I access pfSense I find a WAN latency that goes from 0.4ms to 1200ms.
Isn't there a historical log, preferably a graphic one, that I can print and give to the technicians to testify what at the moment is only my hypothesis?

[johnpoz]

@darkcorner in the monitor under status you can view the wan quality graph, and set what time range via the little wrench in the top right corner.

Keep in mind as you select longer amounts of time, like 1 month or 1 year, etc. The resolution will get lower, vs say every 5 min or 1 min, you will be limited to 1 point per day, etc.

So as you expand the amount of time of the graph, the granularity of the fluctuations will be averaged out..

[luckman212]

@darkcorner You might want or need to look into an external tool to keep better stats and visualizations. Grafana is popular... eg https://github.com/VictorRobellini/pfSense-Dashboard

[johnpoz]

@luckman212 not saying that is not an option. But if the retention of resolution for wan quality (latency packet loss to the gateway) of the data is a problem. Might be easier to just setup say smokeping.. Which could be as easy as docker deployment..

https://hub.docker.com/r/linuxserver/smokeping

But if you just print out say the last weeks monitor graph for wan quality, it has a resolution of 1 hour.. Last 2 days 5 minutes, etc.

[darkcorner]

@luckman212 I need the current time series. If I install new software I will have the future time series

@johnpoz
I have these two graphs: one for the last week and one for the last 8 hours.
At 11:00 AM I moved the firewall to my office with a different WAN connection. The difference is noticeable. On a monthly scale I have peaks at 1800 but I see that the average is 80ms.

Is it enough to justify navigation problems and access to Web and cloud services?

For example, on the morning of day 6 they could not connect with Zoom.
Or on the LAN PCs they can surf, but a virtual machine still on the LAN did not have DNS (ping google.com did not work, ping 8.8.8.8 yes).
In the office zoom.us is not reachable, while here, in my office, it is.

Yet the firewall configuration was not changed.
Furthermore, proxy, snort and clamav have been disabled for at least a month.
I thought the problem was the unstable internet, but an average delay of 80ms doesn't seem so damaging to me.
🔒 Log in to view 🔒 Log in to view

[johnpoz]

@darkcorner what are you checking as your monitor - 80ms to your gateway seems very very high to your gateway.

While really high latency could cause connection issues for some things. I wouldn't think that would be your issue if your saying it was always that high..

What did you change that you saw such a drastic drop.. You changed to different isp?

[darkcorner]

@johnpoz
As I said, this morning I moved pfSense from the office to my laboratory. Here is another line with another ISP.
We have already changed two ISPs in the office, but evidently there is no efficient Internet in this area.
Today we have FWA. We can reactivate an old 10Mb ADSL, giving up speed and cloud backups, but the line should be more stable. I hope, at least.

[stephenw10]

Mmm, 80ms is high if you are still monitoring the gateway IP. The gateway IP doesn't give you a good idea of actual connectivity though, try setting something external like 8.8.8.8 there.

Peaks over 180ms on a 1 week average graph are also very high but wouldn't necessarily cause a connection to fail. But >20% packet loss will.

Steve

[darkcorner]

@stephenw10 said in historical log for WAN latency and performance:

Peaks over 180ms on a 1 week average graph are also very high...

There were peaks above 1800ms in the past month

@johnpoz
Today we had the total disruption of the FWA line provided by the main ISP.
The second ISP provides a line over a telephone cable; with this we had DNS, but it was still impossible to navigate in Internet.

In these conditions I can't even manage failover.

Colleagues speak of a possible "bandwidth saturation", but we are talking about two ISPs on two different "media".

[johnpoz]

@darkcorner well look to traffic under monitor.

[darkcorner]

@johnpoz
how do i know how many MB to download or upload per hour?
I see Traffic or Packets, but they are values "per second".
What do negative values correspond to?
The ISP is telling me that it blocks my line for excessive traffic in an hour.

[stephenw10]

Try using the Traffic Totals package. That will show you, among other things, hourly usage.

🔒 Log in to view

Also consider getting a new ISP.

Steve

[johnpoz]

@darkcorner said in historical log for WAN latency and performance:

how do i know how many MB to download or upload per hour?

That's not what you asked for - you asked if was saturated.. The package Steve mentioned would work for that. There is also bandwidthd, or ntop, etc.

[dennypage]

This thread reminded me that I hadn't pushed out the Influx/Grafana stuff that I use for monitoring latency with dpinger.

It's really nothing fancy, but it works. I run the monitor on a host in the lan, measuring wan latency through the firewall. The scripts can be found here if you're interested.

[Sergei_Shablovsky]

@johnpoz said in historical log for WAN latency and performance:

@luckman212 not saying that is not an option. But if the retention of resolution for wan quality (latency packet loss to the gateway) of the data is a problem. Might be easier to just setup say smokeping.. Which could be as easy as docker deployment..

https://hub.docker.com/r/linuxserver/smokeping

But if you just print out say the last weeks monitor graph for wan quality, it has a resolution of 1 hour.. Last 2 days 5 minutes, etc.

Around 1,5-2 years ago I create treads here on forum and on pfSense bug tracker about adding as third-party packages:

• Smokeping (primary for WANs monitoring);
• Telegraf + Grafana (modern system & links monitoring + modern alerting);
• zsh (as useful CLI extension);
• nano (as useful CLI file editor);

But like in case with “100-year old menu system from A to Z”, sticky to each other icons-buttons, and recommending vi as CLI editor, no button to easy locate port on back of device, ... no one care...

But users asking again and again the same questions, and have the same problems...

Looks like part of a system Devs (with all my respects to their work!) stick to the tools and UI of 70-80’ of past century ;) Sorry.

[stephenw10]

The East Editor (ee) is included by default, it does most of what you might use Nano for.

Steve

[Sergei_Shablovsky]

@stephenw10 said in historical log for WAN latency and performance:

The East Editor (ee) is included by default, it does most of what you might use Nano for.

Nano and Vim are 2 common well-known CLI editors for *nix. Vim are like improved version of vi editor. Both Nano and Vim support plugins.

But more importantly that Nano are modeless, this mean modern users receive WYSIWYG by using Nano. This is like pfSense: in ancient times monowall/pfSense born from needs to make same operations of FreeBSD/pf tuning more painless, errorfree, quickly and usable.

Similar about Zsh/Fish against ordinary bash CLI.

No one (excluding tech nerds) love to make everyday work complicated ;) Agree ?

[stephenw10]

Have you tried using Easy Editor? It's about as easy as it gets for editing a txt file.

[Sergei_Shablovsky]

@stephenw10 said in historical log for WAN latency and performance:

Have you tried using Easy Editor? It's about as easy as it gets for editing a txt file.

Ok, I try as You suggest.

But anyway, most of users prefer WYSIWYG, and plugins - are great and useful thing.

[stephenw10]

@sergei_shablovsky said in historical log for WAN latency and performance:

most of users prefer WYSIWYG

I agree, that's why ee is included.