Higher than normal latency

gdeeble

I'm not sure if I'm in the right place to post this, so please excuse my ignorance if I am.

The problem I'm seeing is that going through my pfSense box, my ping times are higher than normal. If I ping google.com from WAN1 on the box, I see TTL of 45 and ping times in the 80-90ms range at the box, and see 100-120ms or even higher out on the LAN side of the network. If I connect to the modem directly from WAN1, TTL is 60 and ping is down between 10-20 ms like normal. I don't have anything more than gateway groups set up for WAN Failover, at this point. No traffic shaping currently.

The setup is as follows:

The Machine:
Pentium 4 CPU 3.0GHz w/ HT
2GB of RAM
160 GB HDD partitioned to 40GB for use
2NIC's(D-Link DGE-530T Gigabit Ethernet, Intel 82562EZ (ICH6))

The Network:
2 WAN, both DHCP
WAN1, Cable Internet
WAN2, Satellite Internet(Failover connection)

I have tried running the entire network through VLans using 1 NIC, as well as using both NIC's and having Internet via VLans on 1 and LAN on the other.

Switch: Cisco 2950

Here are a few details about the NIC's but I'm new to pfSense as well as BSD in general so I'm not 100% sure on how to read some of the information. I found reading that people were wanting to see this but I didn't quite grasp the reasoning behind it.

$ pciconf -lv | grep 20000:


fxp0@pci0:1:8:0:	class=0x020000 card=0x80f81043 chip=0x10648086 rev=0x05 hdr=0x00
skc0@pci0:1:11:0:	class=0x020000 card=0x4b011186 chip=0x4b011186 rev=0x11 hdr=0x00

$ sysctl dev.skc.0


dev.skc.0.%desc: D-Link DGE-530T Gigabit Ethernet
dev.skc.0.%driver: skc
dev.skc.0.%location: slot=11 function=0
dev.skc.0.%pnpinfo: vendor=0x1186 device=0x4b01 subvendor=0x1186 subdevice=0x4b01 class=0x020000
dev.skc.0.%parent: pci1
dev.skc.0.int_mod: 100

$ sysctl dev.fxp.0


dev.fxp.0.%desc: Intel 82562EZ (ICH6)
dev.fxp.0.%driver: fxp
dev.fxp.0.%location: slot=8 function=0
dev.fxp.0.%pnpinfo: vendor=0x8086 device=0x1064 subvendor=0x1043 subdevice=0x80f8 class=0x020000
dev.fxp.0.%parent: pci1
dev.fxp.0.int_delay: 1000
dev.fxp.0.bundle_max: 6
dev.fxp.0.rnr: 0
dev.fxp.0.stats.rx.good_frames: 434989
dev.fxp.0.stats.rx.crc_errors: 0
dev.fxp.0.stats.rx.alignment_errors: 0
dev.fxp.0.stats.rx.rnr_errors: 0
dev.fxp.0.stats.rx.overrun_errors: 0
dev.fxp.0.stats.rx.cdt_errors: 0
dev.fxp.0.stats.rx.shortframes: 0
dev.fxp.0.stats.rx.pause: 0
dev.fxp.0.stats.rx.controls: 0
dev.fxp.0.stats.rx.tco: 0
dev.fxp.0.stats.tx.good_frames: 154459
dev.fxp.0.stats.tx.maxcols: 0
dev.fxp.0.stats.tx.latecols: 0
dev.fxp.0.stats.tx.underruns: 0
dev.fxp.0.stats.tx.lostcrs: 0
dev.fxp.0.stats.tx.deffered: 0
dev.fxp.0.stats.tx.single_collisions: 0
dev.fxp.0.stats.tx.multiple_collisions: 0
dev.fxp.0.stats.tx.total_collisions: 0
dev.fxp.0.stats.tx.pause: 0
dev.fxp.0.stats.tx.tco: 0

$ sysctl hw.em


hw.em.tx_int_delay: 66
hw.em.rx_int_delay: 0
hw.em.tx_abs_int_delay: 66
hw.em.rx_abs_int_delay: 66
hw.em.rxd: 1024
hw.em.txd: 1024
hw.em.smart_pwr_down: 0
hw.em.sbp: 0
hw.em.enable_msix: 1
hw.em.rx_process_limit: 100
hw.em.eee_setting: 1

This is actually for a home setup so it's not mission critical. I just want to stop getting yelled at by the others in the home and want to resolve the high latency on the cable connection and potentially see if that helps with network drops as it seems that the network is also dropping connection here and there.

Any help is appreciated, even if it tells me to go back to searching for the answer, though with that if I can get some keyword recommendations.

Nullity

I experience the same thing occasionally but my increased lwtency is caused by intermediate internet hops, ehich a program like "mtr" will expose. I usually ping 12ms on google.com but sometimes it is 25ms.

Use mtr to check the route.

It might be better to use the RTT to your first/second hop as the benchmark.

I doubt pfSense is thr cause.

gdeeble

@Nullity:

I experience the same thing occasionally but my increased lwtency is caused by intermediate internet hops, ehich a program like "mtr" will expose. I usually ping 12ms on google.com but sometimes it is 25ms.

Use mtr to check the route.

It might be better to use the RTT to your first/second hop as the benchmark.

I doubt pfSense is thr cause.

Thanks for the fast reply. RTT on WAN1 is about 3-7ms when directly connected to the modem. Currently I decided to pull the box out and use a router for now while I'm still trying to work the bugs out(and stop getting yelled at.. lol). This is also not just hit or miss but a consistent issue.

The latency is about the same though regardless with router between or direct to the modem.

Ping:


 PING google.com (216.58.217.206) from 192.168.1.1: 56 data bytes
64 bytes from 216.58.217.206: icmp_seq=0 ttl=45 time=89.382 ms
64 bytes from 216.58.217.206: icmp_seq=1 ttl=45 time=81.441 ms
64 bytes from 216.58.217.206: icmp_seq=2 ttl=45 time=80.915 ms

--- google.com ping statistics ---
3 packets transmitted, 3 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 80.915/83.913/89.382/3.873 ms

Trace Route:


1  192.168.0.1 (192.168.0.1)  0.322 ms  0.266 ms  0.204 ms
 2  10.238.64.1 (10.238.64.1)  17.946 ms  8.940 ms  8.213 ms
 3  10.45.0.13 (10.45.0.13)  9.812 ms  8.291 ms  8.151 ms
 4  10.36.0.13 (10.36.0.13)  9.614 ms  9.774 ms  10.327 ms
 5  10.36.0.18 (10.36.0.18)  19.071 ms  17.838 ms  18.679 ms
 6  10.24.1.6 (10.24.1.6)  20.544 ms  18.442 ms  17.759 ms
 7  10.17.1.2 (10.17.1.2)  16.636 ms  15.816 ms  17.181 ms
 8  10.17.1.6 (10.17.1.6)  21.707 ms  19.734 ms  22.787 ms
 9  10.20.1.14 (10.20.1.14)  19.588 ms  22.912 ms  17.953 ms
10  10.25.0.34 (10.25.0.34)  21.485 ms  18.953 ms  19.738 ms
11  204.111.0.54 (204.111.0.54)  204.745 ms  227.148 ms  177.197 ms
12  216.239.46.248 (216.239.46.248)  35.680 ms
    216.239.46.250 (216.239.46.250)  33.103 ms
    216.239.46.248 (216.239.46.248)  39.114 ms
13  209.85.143.112 (209.85.143.112)  34.137 ms
    209.85.143.212 (209.85.143.212)  85.804 ms  85.801 ms
14  209.85.143.215 (209.85.143.215)  44.797 ms  46.584 ms
    209.85.143.115 (209.85.143.115)  53.004 ms
15  209.85.244.209 (209.85.244.209)  55.639 ms  79.083 ms  57.160 ms
16  216.239.51.169 (216.239.51.169)  60.986 ms  57.830 ms
    216.239.51.97 (216.239.51.97)  64.996 ms
17  216.239.40.8 (216.239.40.8)  93.801 ms
    216.239.46.153 (216.239.46.153)  94.068 ms  99.649 ms
18  209.85.248.124 (209.85.248.124)  83.771 ms
    64.233.174.191 (64.233.174.191)  83.781 ms
    209.85.248.124 (209.85.248.124)  83.386 ms

I'm not sure that it would be a "pfSense" software issue as much as a configuration issue/hardware issue. I can try the MTR and see if shows the same. I know when I made changes to the interfaces though, for a short period of time the ping does return to normal though, and then comes back to problematic.

cmb

The TTL is just indicative of how many hops it takes to reach the destination. Which is up to your ISP generally, though if you have really screwy internal routing with multiple routers, that could be attributable to your internal network.

Do you recognize any of those 10.x.x.x IPs in hops 2-10 there? The latency is high enough that they'd almost have to be on your ISP's network. And probably in two different locations judging by the latency difference (hops 2-4 close to you, 5-10 maybe 200-400 miles away). Something different with the WAN as it's configured on pfSense is being treated much differently by your ISP, and being routed an unusual-looking path. ISP routers shouldn't reply from private IP sources. Some will, with public IPs becoming more scarce, but only for a hop or two generally. I've never seen an ISP with private IPs across 10 consecutive hops.

That traceroute shows higher latency getting to a real Internet router than you're describing to reach Google plugged directly into the modem.

gdeeble

@cmb:

The TTL is just indicative of how many hops it takes to reach the destination. Which is up to your ISP generally, though if you have really screwy internal routing with multiple routers, that could be attributable to your internal network.

Do you recognize any of those 10.x.x.x IPs in hops 2-10 there? The latency is high enough that they'd almost have to be on your ISP's network. And probably in two different locations judging by the latency difference (hops 2-4 close to you, 5-10 maybe 200-400 miles away). Something different with the WAN as it's configured on pfSense is being treated much differently by your ISP, and being routed an unusual-looking path. ISP routers shouldn't reply from private IP sources. Some will, with public IPs becoming more scarce, but only for a hop or two generally. I've never seen an ISP with private IPs across 10 consecutive hops.

That traceroute shows higher latency getting to a real Internet router than you're describing to reach Google plugged directly into the modem.

CMB,

I now do feel rather stupid….LOL .. So looking at this, when my desktop "pings google" it's pinging a ISP server apparently, thus showing average latency being 10-15ms where real google server resides at the 216.58.217.206 IP. Ping 74.125.224.72 resulted in the same latency as the pfSense box and TTL.

I guess, at this point, I need to figure out why the network seemed to be "much slower" than it does when not going through pfSense, and why it keeps dropping out when connected as well.

Thanks again! :)