Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    Ping spikes on new install

    General pfSense Questions
    7
    21
    4.2k
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • N
      nazuro
      last edited by

      Hi all, this is my first pfSense build and have noticed some issues with ping spikes while gaming. The issue is not severe - it seems that I have random ping spikes (around three in my last test of 100 pings to 8.8.8.8 ). The latency will be around 24 or 25 and then will shoot up to the hundreds or even 200 ms then go straight back down again.

      My hardware is:
      Hynix 8GB DDR3 1600Mhz SODIMM
      J3455B-ITX
      Game Max GP400A 400W 80+ Bronze PSU
      Crucial MX200 250GB SATA 6Gb/s 2.5" SSD
      DELL INTEL 1000 PRO PCI-E QUAD GIGABIT NETWORK CARD - YT674 - D96950-006

      My setup is:
      TalkTalk router in bridged mode –-> <wan>pfSense <lan>-> Switch -> WAP / wired devices.

      I have narrowed down the problems with ping latency to the pfSense. I did a long ping test using just my ISP modem and it all came back fine. The problems I describe are after SSHing to my pfSense and running ping 8.8.8.8 from the shell.

      I am really lost as to how to investigate this and would really appreciate any help you the community may be able to offer. I have not really set much up with pfSense so far, just OpenVPN (not currently running).

      One thing coming to mind is maybe it's my second-hand Intel NIC, but I do not know how to test if this is the issue or not.

      Thank you

      Edit: Also noticing when I'm in SSH for pfSense that when I ping a device on my LAN I also have latency spikes (weirdly, pinging my Macbook Pro has constant latency over 10ms, between 10 and 100). Pinging another device gives me around 0-2ms and then spike to 100 or so.

      Edit2: I've tried using the Motherboard NIC as my WAN interface, and still when I ssh to pfSense and ping out I get the random ping spikes like 23 23 23 23 23 84 66 23 23 23 23 23 23 23 23 23 23 23 23 145 etc.</lan></wan>

      1 Reply Last reply Reply Quote 0
      • N
        nazuro
        last edited by

        Still facing this issue and really struggling how to diagnose :( any advice here greatly appreciated

        Thank you very much

        1 Reply Last reply Reply Quote 0
        • C
          chrcoluk
          last edited by

          disable any c-states on cpu, also if powerd is running either set it to performance mode or shut it down, this is so we can rule out power saving technologies misbehaving.
          disable hyperhtreading if enabled and supported
          consider disabling checksum offloading on nic, although as its intel I consider this unlikely to be the problem
          make sure large receive offload and send equivelent is disabled, and promiscious mode.
          google how to do it, but set the intel nic to a single queue mode only.  (loader.conf.custom).

          This is to test and resolve the lan side spikes, ignore whats going on wan side for now.

          pfSense CE 2.7.2

          1 Reply Last reply Reply Quote 0
          • N
            nazuro
            last edited by

            @chrcoluk:

            disable any c-states on cpu, also if powerd is running either set it to performance mode or shut it down, this is so we can rule out power saving technologies misbehaving.
            disable hyperhtreading if enabled and supported
            consider disabling checksum offloading on nic, although as its intel I consider this unlikely to be the problem
            make sure large receive offload and send equivelent is disabled, and promiscious mode.
            google how to do it, but set the intel nic to a single queue mode only.  (loader.conf.custom).

            This is to test and resolve the lan side spikes, ignore whats going on wan side for now.

            • powerd is disabled already

            • Diabling checksum offboarding - I haven't figured out how to do this but please note I have also tested using the motherboard NIC as my WAN interface and see the same issues.

            • Large receive offload is disabled. I can't find an option for promiscuous mode or send equivalent but I haven't changed these.

            • Disabling hyperthreading

            • Disabling c-states

            1 Reply Last reply Reply Quote 0
            • C
              chrcoluk
              last edited by

              offloading is in the pfsense gui under advanced system settings.

              you can disable hyperthreading in the loader.conf

              add this to /boot/loader.conf.local and reboot

              also Ill add lines to force your network over single threaded.  Note the igb line is only useful if your intel uses the igb driver, it may be using the em driver, which can be confirmed by running ifconfig.

              net.isr.maxthreads=1
              hw.igb.num_queues=1
              machdep.hyperthreading_allowed="0"

              I would say the LAN debugging takes priority over the WAN, the LAN spikes shouldnt be happening period, whilst WAN can be caused by all sorts of things outside your control.

              pfSense CE 2.7.2

              1 Reply Last reply Reply Quote 0
              • Z
                ZGamer
                last edited by

                The particular board in question unless your running has a pair of RTL8111GR based nics on it which "should" work. That being said the standard answer to rule it out is try switching over to using Intel nics…FreeBSD drivers for those nics are just that much better, there's also better quality control during fab.

                On the lan side when start seeing latency issues start removing hops until it's stable. What kind of switch & AP are you using? Are you running on the 2.4GHz or 5GHz band? Check for other devices broadcasting on the same channel. Period 100ms latency spikes given the original could be anything from a buggy switch firmware, buggy drivers, crap nics, failing AP, DC motor interference, microwave ovens, neighborhood AP's on the same channel, or flaky MBP wireless cards (PITA) depending on the year.

                –------------------------------------------------------------------------------------
                pfSense Documentation Wiki
                Need Commercial Support?
                Personal Blog

                1 Reply Last reply Reply Quote 0
                • C
                  chrcoluk
                  last edited by

                  Also to add if my tip doesnt fix it then go back to defaults, what I offered is troubleshooting tips and could possibly be cause of problem, but dont set the changes if the defaults are ok.

                  pfSense CE 2.7.2

                  1 Reply Last reply Reply Quote 0
                  • keyserK
                    keyser Rebel Alliance
                    last edited by

                    Definately NIC offloading that causes this - seen it so many times.

                    Try disabling all three options for hardware offloading on the NIC in advanced settings. This will fix your issue, and then selectively start enabling it and which one/combination is causing the issue.

                    Love the no fuss of using the official appliances :-)

                    1 Reply Last reply Reply Quote 0
                    • N
                      nazuro
                      last edited by

                      Thank you all for your helpful input

                      @Patrick_:

                      The particular board in question unless your running has a pair of RTL8111GR based nics on it which "should" work. That being said the standard answer to rule it out is try switching over to using Intel nics…FreeBSD drivers for those nics are just that much better, there's also better quality control during fab.

                      On the lan side when start seeing latency issues start removing hops until it's stable. What kind of switch & AP are you using? Are you running on the 2.4GHz or 5GHz band? Check for other devices broadcasting on the same channel. Period 100ms latency spikes given the original could be anything from a buggy switch firmware, buggy drivers, crap nics, failing AP, DC motor interference, microwave ovens, neighborhood AP's on the same channel, or flaky MBP wireless cards (PITA) depending on the year.

                      My motherboard only has one NIC. I did however use this and the Intel NIC as well and same problems on each. Regarding the LAN side, the latency seems to only be TO my macbook. If I ping pfSense from my macbook I see 1-2 ms latency. So I imagine it's probably something locally on my Macbook. Likewise, pinging from pfSense to another device on DHCP also sees 1,2ms. I am using a Netgear R7000 DD-WRT for WAP and a Netgear 5port unmanaged switch. I have tested the same by using a cable to my macbook, and still same very high LAN latency.

                      I sat with my Macbook plugged directly into LAN port on pfSense (no WAP or switch involved). latency not anywhere near 100 but still interesting to see it is spiking, similar to with the WAN..
                      https://pastebin.com/A32yrq5v

                      @chrcoluk:

                      add this to /boot/loader.conf.local and reboot

                      net.isr.maxthreads=1
                      hw.igb.num_queues=1
                      machdep.hyperthreading_allowed="0"

                      I have done this but still same issue. Granted, for the test I have just run I didn't see any spikes over 100ms like I did yesterday but I wouldn't say that is a result of changing this file. I did 100 pings and put it on pastebin just to show, the pattern does seem random
                      https://pastebin.com/CabxSFpy
                      Again, tested on WiFi and the response time seems very random. Could easily be problem with my macbook though
                      https://pastebin.com/ShyzBXfh

                      @chrcoluk:

                      whilst WAN can be caused by all sorts of things outside your control.

                      Strange thing is when I tested from my talktalk router I saw no issues, which leads me to assume it's something with my pfSense.

                      @keyser:

                      Try disabling all three options for hardware offloading on the NIC in advanced settings. This will fix your issue, and then selectively start enabling it and which one/combination is causing the issue.

                      Hardware Large Receive and Hardware TCP Segmentation Offloading were already selected 'Disable'. I've additionally disabled Hardware Checksum Offloading but the problems on LAN and WAN persist unfortunately.

                      1 Reply Last reply Reply Quote 0
                      • N
                        nazuro
                        last edited by

                        Still the same issues :(

                        Just did some more tests to show the issue. Really appreciate any other advice as to what the problem could be? I feel like it's not a hardware issue, I have tried WAN on the Intel card I have and also onboard motherboard NIC.

                        Pinging out from laptop plugged in to Talktalk ISP router
                        64 bytes from 8.8.8.8: icmp_seq=0 ttl=59 time=25.146 ms
                        64 bytes from 8.8.8.8: icmp_seq=1 ttl=59 time=25.048 ms
                        64 bytes from 8.8.8.8: icmp_seq=2 ttl=59 time=24.956 ms
                        64 bytes from 8.8.8.8: icmp_seq=3 ttl=59 time=24.973 ms
                        64 bytes from 8.8.8.8: icmp_seq=4 ttl=59 time=25.198 ms
                        64 bytes from 8.8.8.8: icmp_seq=5 ttl=59 time=24.880 ms
                        64 bytes from 8.8.8.8: icmp_seq=6 ttl=59 time=25.087 ms
                        64 bytes from 8.8.8.8: icmp_seq=7 ttl=59 time=25.050 ms
                        64 bytes from 8.8.8.8: icmp_seq=8 ttl=59 time=24.885 ms
                        64 bytes from 8.8.8.8: icmp_seq=9 ttl=59 time=25.227 ms
                        64 bytes from 8.8.8.8: icmp_seq=10 ttl=59 time=24.880 ms
                        64 bytes from 8.8.8.8: icmp_seq=11 ttl=59 time=24.729 ms
                        64 bytes from 8.8.8.8: icmp_seq=12 ttl=59 time=24.975 ms
                        64 bytes from 8.8.8.8: icmp_seq=13 ttl=59 time=24.889 ms
                        64 bytes from 8.8.8.8: icmp_seq=14 ttl=59 time=25.303 ms
                        64 bytes from 8.8.8.8: icmp_seq=15 ttl=59 time=24.928 ms
                        64 bytes from 8.8.8.8: icmp_seq=16 ttl=59 time=24.848 ms
                        64 bytes from 8.8.8.8: icmp_seq=17 ttl=59 time=25.005 ms

                        Pinging out from laptop on pfSense LAN
                        PING 8.8.8.8 (8.8.8.8): 56 data bytes
                        64 bytes from 8.8.8.8: icmp_seq=0 ttl=58 time=37.218 ms
                        64 bytes from 8.8.8.8: icmp_seq=1 ttl=58 time=35.790 ms
                        64 bytes from 8.8.8.8: icmp_seq=2 ttl=58 time=26.519 ms
                        64 bytes from 8.8.8.8: icmp_seq=3 ttl=58 time=31.367 ms
                        64 bytes from 8.8.8.8: icmp_seq=4 ttl=58 time=320.857 ms
                        64 bytes from 8.8.8.8: icmp_seq=5 ttl=58 time=64.461 ms
                        64 bytes from 8.8.8.8: icmp_seq=6 ttl=58 time=79.875 ms
                        64 bytes from 8.8.8.8: icmp_seq=7 ttl=58 time=27.106 ms
                        64 bytes from 8.8.8.8: icmp_seq=8 ttl=58 time=308.835 ms
                        64 bytes from 8.8.8.8: icmp_seq=9 ttl=58 time=28.096 ms
                        64 bytes from 8.8.8.8: icmp_seq=10 ttl=58 time=27.234 ms
                        64 bytes from 8.8.8.8: icmp_seq=11 ttl=58 time=28.600 ms
                        64 bytes from 8.8.8.8: icmp_seq=12 ttl=58 time=118.843 ms
                        64 bytes from 8.8.8.8: icmp_seq=13 ttl=58 time=30.498 ms

                        Pinging out from pfSense itself, through the WAN to my talktalk router
                        PING 8.8.8.8 (8.8.8.8): 56 data bytes
                        64 bytes from 8.8.8.8: icmp_seq=0 ttl=59 time=25.000 ms
                        64 bytes from 8.8.8.8: icmp_seq=1 ttl=59 time=25.163 ms
                        64 bytes from 8.8.8.8: icmp_seq=2 ttl=59 time=99.020 ms
                        64 bytes from 8.8.8.8: icmp_seq=3 ttl=59 time=25.020 ms
                        64 bytes from 8.8.8.8: icmp_seq=4 ttl=59 time=25.013 ms
                        64 bytes from 8.8.8.8: icmp_seq=5 ttl=59 time=25.466 ms
                        64 bytes from 8.8.8.8: icmp_seq=6 ttl=59 time=32.813 ms
                        64 bytes from 8.8.8.8: icmp_seq=7 ttl=59 time=53.171 ms
                        64 bytes from 8.8.8.8: icmp_seq=8 ttl=59 time=61.021 ms

                        Edit: Going to try a factory reset tomorrow and then test the WAN from there.

                        1 Reply Last reply Reply Quote 0
                        • N
                          nazuro
                          last edited by

                          I performed a factory reset, and configured ONLY the WAN as a DHCP interface.

                          I am still seeing latency issues when pinging the internet (23ms, 23ms, 23ms, 106ms, 23ms, 23ms, 23ms, 23ms, 44ms, 24ms, 23ms, 23ms, 33ms, 23ms, 101ms) etc.

                          I see the same results when using my Intel PCI card NIC as WAN and also the onboard motherboard NIC.

                          I also see the same results after I make sure all Hardware Offloading is dissabled in the Advanced menu. My changes to /boot/loader.conf.local persisted through the factory reset. Still same latency issues.

                          I do not see these issues when am pinging directly from the TalkTalk modem. The cable between the two is around 1M long. Really can't think what the issue is.. Any other ideas here would be greatly appreciated. Could it be an issue with the Apollo Lake? I read some people have issues with this with newest pfSense. Didn't read anything about latency spikes though.

                          Edit: Managed to get into the BIOS and changed C-States from C6 to Disabled. STILL getting the latency spikes. Anything else I may be able to change in the BIOS? Everything will be default apart from now c-states.
                          Thanks

                          1 Reply Last reply Reply Quote 0
                          • N
                            NaterGator
                            last edited by

                            Are you using any of the altq traffic shapers?

                            1 Reply Last reply Reply Quote 0
                            • N
                              nazuro
                              last edited by

                              @NaterGator:

                              Are you using any of the altq traffic shapers?

                              Everything is unchecked in firewall-> traffic shaper. I am using a completely out of the box install here with WAN and LAN configured.
                              Thank you

                              1 Reply Last reply Reply Quote 0
                              • N
                                nazuro
                                last edited by

                                • Fresh install 2.4.2. Only config is LAN/WAN ports and:

                                • Everything is unchecked in firewall-> traffic shaper->By interface

                                • All three Hardware Offloading settings are Disabled

                                • Added the following to /boot/loader.conf.local:
                                  net.isr.maxthreads=1
                                  hw.igb.num_queues=1
                                  machdep.hyperthreading_allowed="0"

                                • powerd is not running

                                • BIOS: CPU C States Support = Disabled

                                1 Reply Last reply Reply Quote 0
                                • ?
                                  A Former User
                                  last edited by

                                  So having read all this, the only thing I can think of is the ethernet port on your ISP router maybe?

                                  i.e. when you test from your ISP router, are you testing from a device plugged into it, or from it directly?

                                  What I mean is that running a ping on the router itself might be telling you "the router is fine" when actually if there's a problem with the router that causes latency/issues only on it's "LAN" port (the port you plug your pfSense into) you won't see it.

                                  What if you plug a laptop into the TeamTalk device and test using it?  Does that show you the latency spikes?

                                  Also, what if you remove "the internet" from the equation and configure two laptops.  One pretends to be the Internet and one is a LAN client, then you ping between the laptops via the pfsense (you'll need two NICs on your pfSense box)

                                  Does that show you latency?

                                  1 Reply Last reply Reply Quote 0
                                  • N
                                    nazuro
                                    last edited by

                                    @muppet:

                                    So having read all this, the only thing I can think of is the ethernet port on your ISP router maybe?

                                    i.e. when you test from your ISP router, are you testing from a device plugged into it, or from it directly?

                                    What I mean is that running a ping on the router itself might be telling you "the router is fine" when actually if there's a problem with the router that causes latency/issues only on it's "LAN" port (the port you plug your pfSense into) you won't see it.

                                    What if you plug a laptop into the TeamTalk device and test using it?  Does that show you the latency spikes?

                                    Also, what if you remove "the internet" from the equation and configure two laptops.  One pretends to be the Internet and one is a LAN client, then you ping between the laptops via the pfsense (you'll need two NICs on your pfSense box)

                                    Does that show you latency?

                                    Hi muppet, thanks for your time. I did come to the same thought but unfortunately disproved it by:
                                    *** Plugging Windows laptop into TalkTalk Router directly, and pings were fine

                                    • I also have latency on the LAN

                                    • Running FreeBSD on the server seems to work fine regarding latency**

                                    1 Reply Last reply Reply Quote 0
                                    • ?
                                      A Former User
                                      last edited by

                                      So even ping between two laptops, via pfSense, shows the problem?

                                      Can you post the output of dmesg? I'm not a FreeBSD guru, but maybe we can spot something in there that might be the cause of the problems.

                                      1 Reply Last reply Reply Quote 0
                                      • N
                                        nazuro
                                        last edited by

                                        @muppet:

                                        So even ping between two laptops, via pfSense, shows the problem?

                                        Can you post the output of dmesg? I'm not a FreeBSD guru, but maybe we can spot something in there that might be the cause of the problems.

                                        Unfortunately so :( and strange that FreeBSD live USB didn't have the problems! Must be some sort of setting in PfSense

                                        Me neither re FreeBSD guru but I thank you for having a skim over incase!
                                        Here's the output https://pastebin.com/3wCFpKqW

                                        1 Reply Last reply Reply Quote 0
                                        • N
                                          nazuro
                                          last edited by

                                          I'm completely lost for ideas now, may have to scrap pfSense and go back to DD-WRT :'( Or at least buy a new motherboard/processor combination…

                                          Any suggestions for ITX motherboard/CPU that has AES-NI and a proven record of being good for pfSense? Or alternatively any other troubleshooting options to test the LAN/WAN latency issues

                                          I've also tried setting the Cryptographic Hardware from BSD Crypto Device to None, to no avail :(
                                          Thank you all

                                          1 Reply Last reply Reply Quote 0
                                          • ?
                                            A Former User
                                            last edited by

                                            Maybe try disabling MSX-I and/or Flow Control?

                                            https://doc.pfsense.org/index.php/Tuning_and_Troubleshooting_Network_Cards

                                            Also try running using a RAM disk (System->Advanced->Misc), maybe your pfSense writing logs to disk causes the system to pause/freeze up for a moment?

                                            These are all just stab-in-dark guesses now :(

                                            1 Reply Last reply Reply Quote 0
                                            • First post
                                              Last post
                                            Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.