In/out errors possible reason for latency



  • Hello All-

    New to the PFsense community and I apoligze right from the start for the details…I just want to provide as much as I can for faster troubleshooting.

    I have setup a box running PFsense 2.3.4-RELEASE-p1 (amd64) FreeBSD 10.3-RELEASE-p19 CPU Type Intel(R) Xeon(R) CPU E5-2640 v2 @ 2.00GHz 32 CPUs: 2 package(s) x 8 core(s) x 2 SMT threads
    It has been running great the majority of the time. The issue I am having is during peak  hours. It doesn't seem like we are coming anywhere close to maxing our bandwidth and we are gig switches throughout although the box itself runs on 2 10 gig broadcom nics. (1 WAN, 1 2 VLANS for LAN Traffic....Pretty basic setup)

    The only two potential issues I can see is that the NICS will not auto negotiate or allow me force 1gig full duplex. Not even an option and if I try to do it within shell on FreeBSD it breaks. I have check for MTU mismatches and have not discovered anything.

    I can only imagine it is the speed mismatch.

    So the actual issue is during peak hours (1000+ users) pings shoot up to 80-100+ms and download speeds tank, although upload seems to be semi normal. I do a traceroute to get an idea of where the latency starts and it seems like it is between the LAN gateway to our edge router. Also other LAN's our our network outside of PFsense do not have the same latency during these times. (internal traffic seems fine)

    I check for collisions and errors on interfaces and all seem fine except the WAN interface show a high count on the in side of IN/OUT errors (posted below) Which I believe would make sense with the impact on download speeds.
    MTU
    1500
    Media
    10Gbase-T <full-duplex>In/out packets
    1972132835/1021513273 (2.41 TiB/213.77 GiB)
    In/out packets (pass)
    1972132835/1021513273 (2.41 TiB/213.77 GiB)
    In/out packets (block)
    203667/2474 (137.80 MiB/149 KiB)
    In/out errors
    15908509/0
    Collisions
    0

    I guess I am stumped. I'm not sure if I am missing something obvious here or if anyone has suggestions for what else should be checked.

    I have a 10g interface I could attach to but would require a media converter or 10g copper SFP for me to use, this comes with a large expense so I'd like to be as sure as possible it is in fact a speed/duplex mismatch issue before I ask my boss to make the purchase.

    Any assistance would be greatly appreciated. Thanks in advance!</full-duplex>


  • Netgate Administrator

    Well I would not go looking for other issus until that is solved. As you say IN errors on WAN would affect download more than up.

    So that is a 10G copper NIC connecting to a 1G device? I assueme (but suggest anyway  ;)) you have tried swapping out the cable?

    Can you not use a 1G NIC directly for WAN?

    Perhaps I've misunderstood your setup.

    Steve