Dashboard Traffic Graphs…SPIKES

jimp

Open a bug report for that on https://redmine.pfsense.org

RonpfS

I was able to reproduce the spikes in two ways :

in pfBlockerNG / Alerts, it takes about a minute to gather all the alerts and do DNS lookup
in pfBlockerNG / Update / View log until you click on End View

Both Status Traffic Graphs and Widget Traffic graph showed spikes.
There is probably other ways to reproduce the spikes.

WidgetTGSpikes.JPG_thumb

TGSpikeCapture.JPG_thumb

PiBa

Tried to optimize trafficgraph data retrieval a bit, can you guys test if it mitigates the issue.?

https://github.com/pfsense/pfsense/pull/3585

The way the graphs call to ifstat.php to get the latest usage number doesn't seem to have changed much so doesnt explain why it would only start to happen with 2.3.3 / 2.4 so that by itself shouldn't be the issue.. It could be a different issue underneath somewhere. With pullrequest above large parts of the re-authentication code/initialization are skipped. Anyhow I hope it helps for this case as well.

PiBa

Okay new patches available :).

First one same as above but now with a 2.3 link as well:

traffic-graphs, optimize retrieval of data every x seconds

For 2.3: https://github.com/PiBa-NL/pfsense/commit/34e1ceabf20419a1e1d8436c03504a0d7706be5d
For 2.4: https://github.com/pfsense/pfsense/pull/3585

The second commit seems to have more effect, combined they are even better. From initial testing the 'big spikes' are gone, a few small ones seem to remain though. Not sure yet what can be done there without moving to a whole different method for retrieving the data needed..

flash_message, commit session when done with it

For 2.3 https://github.com/PiBa-NL/pfsense/commit/d188e833d1c7a0756517b9e92bf41535fff9973a
For 2.4 https://github.com/pfsense/pfsense/pull/3587

I'm interested in any issues that might arise.

grandrivers

I havent tested patches yet but on graph related problems i noticed we not apparent in widget its in the status/traffic graph page where graph does not match chart I need to make a video it then becomes obvious

thread link to problem
https://forum.pfsense.org/index.php?topic=125520.0

Valeriy

I am going to test the patches today.

So far I have found the following:

seems to be phpfm process problems might have influenced the behaviour or other bottlenecks between pfsense and monitoring PC
after Ntopng removal problem is not longer that obvious (spikes are still there but not more than 10% of current interface load average, and they do happened less frequent).
network latency issues / monitoring PC CPU utilization 100% also trigger the issues.

Will be back after patches are tested.

Valeriy

Sorry for my ignorance, if somebody could quickly tell what is proper process of applying these custom patches?
…Manually editing the related files?

heper

install the system_patches package / then you can copy/paste the commit url

jimp

@Valeriy:

Sorry for my ignorance, if somebody could quickly tell what is proper process of applying these custom patches?
…Manually editing the related files?

1. Use the System Patches package which shows up under System > Packages once installed
2. Create a new patch entry, give it a name, paste in the URL to a commit -OR- if you want to test a pull request, enter the pull request URL with ".diff" added on. For example, to test https://github.com/pfsense/pfsense/pull/3585 enter https://github.com/pfsense/pfsense/pull/3585.diff
3. Save, fetch, apply

If there are changes on the PR and you need to update: Revert, fetch, apply

Valeriy

Thanks!