High Delay Average on WAN

tmatthews14

I just went through a clean install of 2.3 because of some problems during the upgrade. I am now experiencing a lot of delays from devices accessing the WAN and logged in to look at my Quality RRD graphs to find an average delay of 12.9 ms. I compare that to a couple other boxes I manage and the other two are somewhere around a 1.4ms (office of 4 people) and a 3.2ms (office of 70 people). My 12.9ms delay is at home with 2 people.

Screenshot of sudden spike in WAN delay.

Any ideas of what I can do to fix this? I'm using DNS Forwarder, IPv4 and v6, DHCP server… and that's about it so far.

tmatthews14

I thought 12ms was bad, just spiked up to 155ms.

mer

Delay from the LAN side to something on the WAN?
Client on the LAN side is that a wired connection or wireless?
What is the delay from the pfSense box itself to something on the WAN?

What's the hardware? Is the WAN connected directly to the broadband connection (ISPs router/cablemodem/whatever)?

Way too many things the we don't have any information on.

tmatthews14

Delay from wired client on LAN side:

Ping statistics for 8.8.8.8:
    Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
    Minimum = 23ms, Maximum = 26ms, Average = 25ms

Same client on LAN side but on WiFi:

Ping statistics for 8.8.8.8:
    Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
    Minimum = 24ms, Maximum = 27ms, Average = 25ms

Delay from PfSense box to WAN:

--- 8.8.8.8 ping statistics ---
4 packets transmitted, 4 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 21.695/23.496/25.590/1.441 ms

Hardware is a Netgate FW7541.

Yes, WAN is directly connected to ISP modem.

mer

Default rules, modified rules? No packages? What's CPU load and interrupt load like? pfSense is also doing NAT?
Just to make sure, the "…couple other boxes..." they are also running pfSense? They are going to the same ISP?

Looking at your times one could say that pfSense box is not adding anything from LAN side (all roughly the same). It could be something in the hops from your WAN to 8.8.8.8, basically from you to the ISP to 8.8.8.8.

tmatthews14

Basically default rules with a couple additions for some port forwarding… also I did make some changes to allow static ports on outbound NATs and enable UPnP for some xbox traffic https://forum.pfsense.org/index.php?topic=73012.0.

As for packages I have the OpenVPN export package and Snort which isn't configured or running yet.

CPU load is around 2% interrupts:

PfSense is doing NAT.

The "couple other boxes" are running PfSense, different ISP.

I would agree with you about the times… but you can see in my first post in the screenshot that my delay was minimal up until 4/22 when I did the fresh install. If it were an issue between ISP and 8.8.8.8 it would most likely have been fixed by now. The only thing that has changed is a fresh install of v2.3 and enabling IPv6 (which I just turned off to see if my delay goes down again).

cmb

Your gateway monitor delay depends on what your monitor IP is configured as. Where you're seeing 1-2 ms response times, it's either something at the same physical location as you that you're monitoring, or you have a fiber connection. Anything coax or copper, you're going to have 10+ ms to your next hop router past where it touches the coax or copper.

Nothing about that looks high at all, all very much normal.