Double counting of WAN out traffic?


  • LAYER 8 Global Moderator

    What is he saying - sure looks like mirror to me.. Other than bottom graph shows the 5M, and seems to be cut off on the top graph... But sure looks close enough to be the same.. Looks like a perfect mirror to me - which is what it should be.. Well maybe not exact to the bit, because you could have traffic hitting the lan or wan that doesn't get passed through to the other.

    But where is double?? Looks like wan out, to lan in - just the scale missing the 1M?

    This could be off a bit?

    bitskewed.png

    But its not double.. And again - there will always be traffic seen on wan that is not passed through to lan, and then traffic that hits lan that is not sent out the wan.. You also have small amounts of traffic that would be created by pfsense, dns queries.. package downloads, etc..

    Now if you were doing a speed test and getting 100Mbps, and only saw 50 on one of the graphs - But this looks fine and normal and within graphing error expectations, especially with smoothing, etc.

    How is this double - looks like a perfect mirror, which it should be - at exactly the same values..

    nodouble.png



  • I'm not sure I see it on the graph either :) but it sounds vaguely like my recent post about traffic shaper queue display:
    https://forum.netgate.com/topic/157564/bandwidth-usage-shown-on-queue-status-page-is-wrong (root queue shows double the observed speed)



  • @johnpoz I am attaching an additional screenshot.
    I have 100M/5M link so it is hard to produce a clear graph.
    But I checked numbers and traffic is doubled. Not some small difference.

    6c0c116c-878d-43f5-9089-3fe7f73de897-image.png



  • @johnpoz I think I reproduced the problem clear. Please see below
    53203033-c544-4310-87da-96a042da785c-image.png


  • LAYER 8 Global Moderator

    Where?? You looking at some instantaneous number without a timestamp, vs some other instantaneous of a graph that does smoothing.

    Sorry but these sure look pretty freaking exact to me..

    notdouble.png

    edit: here I just went to upload only internet speed test.. Sure looks pretty freaking on the ball to me..

    uploadonly test.png

    So the scale doesn't get skewed with large numbers on download, and then small numbers on upload, etc.



  • @johnpoz he's looking at WAN out/LAN in, the other side of both graphs (which don't have a scale showing due to the high download speed scale). Are you saying the traffic graph is expected to sometimes show 9.6 Mbps even though the connection speed is lower?

    @stepanov1975 to be clearer, are you saying you have a 5 Mbps upload and should not be able to get 9-10? Because that's what I saw in my thread, with the traffic shaper queue numbers.

    That said I just tried it once and though the numbers were slightly higher it wasn't double for me... the download graph showed 70-75 and per Google that's a 69, and upload showed 10-13 and per Google that is a 10.2.



  • @johnpoz Sorry that I am unable to explain the problem clearly.

    1. The problem exists only for outbound traffic of the WAN interface.
    2. The graph shows traffic that is clearly twice what shown on the LAN side
    3. It is also twice the traffic shown in the results of the speed test.
    4. It is also twice the traffic shown in the interface for the speed test connection.
    5. LAN Inbound traffic is the same as traffic shown in the speed test and also the same as shown for connection to speed test server.
    6. We have three different sources in agreement. So, we can assume the number is tight, but the graph of WAN outbound traffic shows twice this number.
    7. So my conclusion is that the graph of outbound traffic is wrong.
      I am attaching once more a screenshot showing what I said above.
      aaf36b76-3ee6-490f-8a6d-dd00f5b8c9a6-image.png


  • @teamits

    @stepanov1975 to be clearer, are you saying you have a 5 Mbps upload and should not be able to get 9-10? Because that's what I saw in my thread, with the traffic shaper queue numbers.

    This is also right. But I am actually saying something different.
    I am saying that I can check connection speed in three different places (other than the graph), and all sources agree, but the graph shows twice the speed.



  • @stepanov1975 Just brainstorming, do you have Snort or Suricata running? Or traffic shaping?



  • @teamits Triaffic shaping



  • @stepanov1975 said in Double counting of WAN out traffic?:

    Triaffic shaping

    Interesting. Can you check the bandwidth on the root queue on Status/Queues page as well, per https://forum.netgate.com/topic/157564/bandwidth-usage-shown-on-queue-status-page-is-wrong ? I can do more tests on the traffic graph later but I duplicated that doubled-bandwidth problem on four routers yesterday.


  • LAYER 8 Global Moderator

    @teamits said in Double counting of WAN out traffic?:

    Are you saying the traffic graph is expected to sometimes show 9.6 Mbps even though the connection speed is lower?

    No not with a flat line like that.. But with instantaneous numbers you could for sure see different numbers that what you get for an average..

    That instant number you see in the traffic graph can for sure be misleading to what your actually moving on an average..

    For example - my plex is serving up some data via streams

    upload.png

    you can see its bursty... And dending how you have your graphs set for when they sample, how they smooth you can see different values..

    smooth-notsmooth.png

    That instantaneous value you see in the traffic graph, wouldn't say that is all that exact to be honest if you looking to what the bandwidth move is over /time, etc..

    I am not using any queues or shaping.. And the graphs show exactly what I would expect to see.. Not double that is for sure..

    So what I am saying is I am unable to duplicate your problem no matter what graphs I look at.

    Best testing method would be to generate some very stable traffic flow - iperf for example set with specific bandwidth setting so you could move say 50mbps stable for extended period... Then what does your values show.. Keep in mind best time to test this would be with nothing else on the network that could create traffic.

    edit: More examples of variables.. And instantaneous readings..

    No smoothing, max smoothing - instant values being shown vs what the graph shows.

    variables.png

    I can not really do a stable iperf3 tests right now... because I have lots of stuff going on, and users streaming videos..



  • I started a 4 GB upload to observe. I understand traffic is a point in time measurement that will vary up and down, but for a max upload it should be reasonably consistent, and not double the connection speed.

    For me the traffic graph WAN Out hovered around 13-15 Mbps (which would be normal), and the Status/Queues/Interface WAN/root queue ranged from 26-30 Mbps as I observed in my thread (double the connection speed). That's consistent with what I saw on all four routers yesterday, I just wasn't looking at the traffic graph specifically.

    So it seems like OP and I have a similar sounding symptom but they are not the same.



  • @teamits I have PRIQ so, no root queue :(



  • @johnpoz Ok. I did a long upload test. Is good enough to see the traffic doubling?
    dcd615e3-ffcc-4918-b70b-c4bcf64522db-image.png


  • Netgate Administrator

    This has been hit a number of times in the past so there are a few redmine bugs for it but I don't think there is one open currently for 2.4.5p1.

    There is this for 2.5 where you have VLANs in use:
    https://redmine.pfsense.org/issues/10852

    I am unable to replicate this on 2.4.5p1 here though:
    Screenshot from 2020-10-16 17-04-38.png

    That's on an SG-3100 with direct WAN LAN mvneta interfaces. No VLANs, no PPP, no VPNs.

    It must be something specific to your setup. Can you give us more details about it?

    Steve


  • LAYER 8 Global Moderator

    @stepanov1975 Yeah that is great - no disputing that for sure ;)

    But I'm with @stephenw10 I can not duplicate this on 2.4.5p1.. If you are using some sort of queues or limiters or the like... Can you turn those off, and then duplicate your test above?

    Also I take that is showing Bytes, does it do the same if your set for bits?

    edit: Yeah just not seeing such an issue... Here I got someone streaming off my plex, so its very spikey traffic flow... But you can see they are mirrors, and values match both using the bit scale and the bytes scale

    bitsbytes.png

    on a side not - it would be nice if the graph listed the scale it was using ;)



  • @johnpoz said in Double counting of WAN out traffic?:

    queues or limiters or the like... Can you turn those off

    I was just going to post that. https://docs.netgate.com/pfsense/en/latest/trafficshaper/advanced.html#removing-traffic-shaper-settings. If you make a backup you can just restore the backup afterwards to get them back.

    If I remove the shaper then I can't see the queue status so my issue would "go away." :)

    His post yesterday with the hand drawn circles is in bits...status/traffic graph I don't think has a choice?


  • LAYER 8 Global Moderator

    Yeah I don't think you can set the scale in the traffic graph section under status, only in the widget..



  • @stephenw10 I actually have PPPoE.
    It is, of cause, expected that you can not reproduce this problem. Most of the bugs appear under the specific non-default conditions. This is because default and commonly used conditions tested before release.


Log in to reply