Dashboard Traffic Graphs…SPIKES

[Valeriy]

Hi!

Is it possible to revert the traffic graphs to how it was in 2.3.2?
Now it shows unnecessary spikes even when there is none. Tried different browsers, the result is the same.
I had thought that limiters or other packages might have interfered , so I installed 2.3.3 and 2.4.b onto different VMs, ran iperf and got the same issues.

Result is the same, disregardless of:

-  different browsers, including keeping current TAB always active or inactive

• system in production or clean install (without load).
• with different time interval (1-10sec)
• different interfaces
• pfsense 2.3.3 and 2.4.0b are affected
• etc.

It shows spikes where clearly there weren't.

Verified interface load with
>systat -ifstat 1 on corresponding systems

speed remained stable ..

Nobody else experienced it?

Screenshots are attached.


[Valeriy]

Sometimes it stalls for a few seconds, and shows spike for x3 higher than average b/w load. seems to be that data is gathered by browser app in specific way directly instead of getting correct interface loads from pfSense OS.

[jimp]

Open a bug report for that on https://redmine.pfsense.org

[RonpfS]

I was able to reproduce the spikes in two ways :

1. in pfBlockerNG / Alerts, it takes about a minute to gather all the alerts and do DNS lookup
2. in pfBlockerNG / Update / View log until you click on End View

Both Status Traffic Graphs and Widget Traffic graph showed spikes.
There is probably other ways to reproduce the spikes.

[PiBa]

Tried to optimize trafficgraph data retrieval a bit, can you guys test if it mitigates the issue.?

https://github.com/pfsense/pfsense/pull/3585

The way the graphs call to ifstat.php to get the latest usage number doesn't seem to have changed much so doesnt explain why it would only start to happen with 2.3.3 / 2.4 so that by itself shouldn't be the issue.. It could be a different issue underneath somewhere. With pullrequest above large parts of the re-authentication code/initialization are skipped. Anyhow I hope it helps for this case as well.

[PiBa]

Okay new patches available :).

First one same as above but now with a 2.3 link as well:

traffic-graphs, optimize retrieval of data every x seconds

For 2.3: https://github.com/PiBa-NL/pfsense/commit/34e1ceabf20419a1e1d8436c03504a0d7706be5d
For 2.4: https://github.com/pfsense/pfsense/pull/3585

The second commit seems to have more effect, combined they are even better. From initial testing the 'big spikes' are gone, a few small ones seem to remain though. Not sure yet what can be done there without moving to a whole different method for retrieving the data needed..

flash_message, commit session when done with it

For 2.3 https://github.com/PiBa-NL/pfsense/commit/d188e833d1c7a0756517b9e92bf41535fff9973a
For 2.4 https://github.com/pfsense/pfsense/pull/3587

I'm interested in any issues that might arise.

[grandrivers]

I havent tested patches yet but on graph related problems i noticed we not apparent in widget its in the status/traffic graph page where graph does not match chart I need to make a video it then becomes obvious

thread link to problem
https://forum.pfsense.org/index.php?topic=125520.0

[Valeriy]

I am going to test the patches today.

So far I have found the following:

• seems to be phpfm process problems might have influenced the behaviour or other bottlenecks between pfsense and monitoring PC

• after Ntopng removal problem is not longer that obvious (spikes are still there but not more than 10% of current interface load average, and they do happened less frequent).

• network latency issues / monitoring PC CPU utilization 100% also trigger the issues.

Will be back after patches are tested.

[Valeriy]

Sorry for my ignorance, if somebody could quickly tell what is proper process of applying these custom patches?
…Manually editing the related files?

[heper]

install the system_patches package / then you can copy/paste the commit url

[jimp]

@Valeriy:

Sorry for my ignorance, if somebody could quickly tell what is proper process of applying these custom patches?
…Manually editing the related files?

1. Use the System Patches package which shows up under System > Packages once installed
2. Create a new patch entry, give it a name, paste in the URL to a commit -OR- if you want to test a pull request, enter the pull request URL with ".diff" added on. For example, to test https://github.com/pfsense/pfsense/pull/3585 enter https://github.com/pfsense/pfsense/pull/3585.diff
3. Save, fetch, apply

If there are changes on the PR and you need to update: Revert, fetch, apply

[Valeriy]

Thanks!