RTT & RTTsd questions



  • Hi all,

    New pfSense user here. Have a clean install on an AsRock J1900D2Y build and running just about perfectly. Newest build on pfSense, all default wan deny, lan allow rules.

    Getting some strange numbers on RTT and the deviation.  Wan is into a Motorola cable modem connected to comcast and has been getting great pings and solid 180Mbs down, 11 up for about a year on a consumer router.

    When I first boot pfSense, the RTT is about 8ms, sd .8 to 1.  After 2 to 12 hours (checking the past couple days) the RTT has been as high as 200 with sd 170, and twice (now) is RTT 90, sd, 80ish.

    I understand the standard deviation numbers, but what leads to this number? I've seen pings to the same host have that same variation. Sequence of pings are at 10ms. Run it again to the same host and 110ms. Run it again, back to 10, etc.

    It can be extremely noticeable playing an online game, painful 100+ms pings, reboot the router (rebooting the cable modem doesn't seem to matter) and get 10ms pings to the same server.  I can plug a dLink (wrt-dd) back in and this issue goes away, so feels like the new pfSense box.

    Any thoughts on what could be causing the high deviation or what I can do to figure this out?

    Thanks,

    Karim



  • I found the culprit. A crashplan node was doing a backup to their cloud service causing the high RTT. Turn it off, watch the numbers go back to RTT 8.5, sd 2. The node was set to 8Mbs cap on a solid 11Mbs up connection. Even dropping it to 4Mbs for the backup still gives 50-60ms RTT. Installing a new switch this weekend and will read up on pfsense QoS and the switches QoS and see if we can make some magic.

    K



  • Research buffer bloat.



  • Enabled Codel shaper on your LAN and WAN and set your bandwidth to 80%-95% of your stable bandwidth. Ask more questions in the Traffic Shaping forum.



  • @dennypage:

    Research buffer bloat.

    This is a vary good description of what takes place.
    https://www.dslreports.com/forum/r27252457-Internet-Frontier-FIOS-Latency-and-QoS-Where-they-fail