Higher than normal latency
-
I'm not sure if I'm in the right place to post this, so please excuse my ignorance if I am.
The problem I'm seeing is that going through my pfSense box, my ping times are higher than normal. If I ping google.com from WAN1 on the box, I see TTL of 45 and ping times in the 80-90ms range at the box, and see 100-120ms or even higher out on the LAN side of the network. If I connect to the modem directly from WAN1, TTL is 60 and ping is down between 10-20 ms like normal. I don't have anything more than gateway groups set up for WAN Failover, at this point. No traffic shaping currently.
The setup is as follows:
The Machine:
Pentium 4 CPU 3.0GHz w/ HT
2GB of RAM
160 GB HDD partitioned to 40GB for use
2NIC's(D-Link DGE-530T Gigabit Ethernet, Intel 82562EZ (ICH6))The Network:
2 WAN, both DHCP
WAN1, Cable Internet
WAN2, Satellite Internet(Failover connection)I have tried running the entire network through VLans using 1 NIC, as well as using both NIC's and having Internet via VLans on 1 and LAN on the other.
Switch: Cisco 2950
Here are a few details about the NIC's but I'm new to pfSense as well as BSD in general so I'm not 100% sure on how to read some of the information. I found reading that people were wanting to see this but I didn't quite grasp the reasoning behind it.
$ pciconf -lv | grep 20000:
fxp0@pci0:1:8:0: class=0x020000 card=0x80f81043 chip=0x10648086 rev=0x05 hdr=0x00 skc0@pci0:1:11:0: class=0x020000 card=0x4b011186 chip=0x4b011186 rev=0x11 hdr=0x00
$ sysctl dev.skc.0
dev.skc.0.%desc: D-Link DGE-530T Gigabit Ethernet dev.skc.0.%driver: skc dev.skc.0.%location: slot=11 function=0 dev.skc.0.%pnpinfo: vendor=0x1186 device=0x4b01 subvendor=0x1186 subdevice=0x4b01 class=0x020000 dev.skc.0.%parent: pci1 dev.skc.0.int_mod: 100
$ sysctl dev.fxp.0
dev.fxp.0.%desc: Intel 82562EZ (ICH6) dev.fxp.0.%driver: fxp dev.fxp.0.%location: slot=8 function=0 dev.fxp.0.%pnpinfo: vendor=0x8086 device=0x1064 subvendor=0x1043 subdevice=0x80f8 class=0x020000 dev.fxp.0.%parent: pci1 dev.fxp.0.int_delay: 1000 dev.fxp.0.bundle_max: 6 dev.fxp.0.rnr: 0 dev.fxp.0.stats.rx.good_frames: 434989 dev.fxp.0.stats.rx.crc_errors: 0 dev.fxp.0.stats.rx.alignment_errors: 0 dev.fxp.0.stats.rx.rnr_errors: 0 dev.fxp.0.stats.rx.overrun_errors: 0 dev.fxp.0.stats.rx.cdt_errors: 0 dev.fxp.0.stats.rx.shortframes: 0 dev.fxp.0.stats.rx.pause: 0 dev.fxp.0.stats.rx.controls: 0 dev.fxp.0.stats.rx.tco: 0 dev.fxp.0.stats.tx.good_frames: 154459 dev.fxp.0.stats.tx.maxcols: 0 dev.fxp.0.stats.tx.latecols: 0 dev.fxp.0.stats.tx.underruns: 0 dev.fxp.0.stats.tx.lostcrs: 0 dev.fxp.0.stats.tx.deffered: 0 dev.fxp.0.stats.tx.single_collisions: 0 dev.fxp.0.stats.tx.multiple_collisions: 0 dev.fxp.0.stats.tx.total_collisions: 0 dev.fxp.0.stats.tx.pause: 0 dev.fxp.0.stats.tx.tco: 0
$ sysctl hw.em
hw.em.tx_int_delay: 66 hw.em.rx_int_delay: 0 hw.em.tx_abs_int_delay: 66 hw.em.rx_abs_int_delay: 66 hw.em.rxd: 1024 hw.em.txd: 1024 hw.em.smart_pwr_down: 0 hw.em.sbp: 0 hw.em.enable_msix: 1 hw.em.rx_process_limit: 100 hw.em.eee_setting: 1
This is actually for a home setup so it's not mission critical. I just want to stop getting yelled at by the others in the home and want to resolve the high latency on the cable connection and potentially see if that helps with network drops as it seems that the network is also dropping connection here and there.
Any help is appreciated, even if it tells me to go back to searching for the answer, though with that if I can get some keyword recommendations.
-
I experience the same thing occasionally but my increased lwtency is caused by intermediate internet hops, ehich a program like "mtr" will expose. I usually ping 12ms on google.com but sometimes it is 25ms.
Use mtr to check the route.
It might be better to use the RTT to your first/second hop as the benchmark.
I doubt pfSense is thr cause.
-
I experience the same thing occasionally but my increased lwtency is caused by intermediate internet hops, ehich a program like "mtr" will expose. I usually ping 12ms on google.com but sometimes it is 25ms.
Use mtr to check the route.
It might be better to use the RTT to your first/second hop as the benchmark.
I doubt pfSense is thr cause.
Thanks for the fast reply. RTT on WAN1 is about 3-7ms when directly connected to the modem. Currently I decided to pull the box out and use a router for now while I'm still trying to work the bugs out(and stop getting yelled at.. lol). This is also not just hit or miss but a consistent issue.
The latency is about the same though regardless with router between or direct to the modem.
Ping:
PING google.com (216.58.217.206) from 192.168.1.1: 56 data bytes 64 bytes from 216.58.217.206: icmp_seq=0 ttl=45 time=89.382 ms 64 bytes from 216.58.217.206: icmp_seq=1 ttl=45 time=81.441 ms 64 bytes from 216.58.217.206: icmp_seq=2 ttl=45 time=80.915 ms --- google.com ping statistics --- 3 packets transmitted, 3 packets received, 0.0% packet loss round-trip min/avg/max/stddev = 80.915/83.913/89.382/3.873 ms
Trace Route:
1 192.168.0.1 (192.168.0.1) 0.322 ms 0.266 ms 0.204 ms 2 10.238.64.1 (10.238.64.1) 17.946 ms 8.940 ms 8.213 ms 3 10.45.0.13 (10.45.0.13) 9.812 ms 8.291 ms 8.151 ms 4 10.36.0.13 (10.36.0.13) 9.614 ms 9.774 ms 10.327 ms 5 10.36.0.18 (10.36.0.18) 19.071 ms 17.838 ms 18.679 ms 6 10.24.1.6 (10.24.1.6) 20.544 ms 18.442 ms 17.759 ms 7 10.17.1.2 (10.17.1.2) 16.636 ms 15.816 ms 17.181 ms 8 10.17.1.6 (10.17.1.6) 21.707 ms 19.734 ms 22.787 ms 9 10.20.1.14 (10.20.1.14) 19.588 ms 22.912 ms 17.953 ms 10 10.25.0.34 (10.25.0.34) 21.485 ms 18.953 ms 19.738 ms 11 204.111.0.54 (204.111.0.54) 204.745 ms 227.148 ms 177.197 ms 12 216.239.46.248 (216.239.46.248) 35.680 ms 216.239.46.250 (216.239.46.250) 33.103 ms 216.239.46.248 (216.239.46.248) 39.114 ms 13 209.85.143.112 (209.85.143.112) 34.137 ms 209.85.143.212 (209.85.143.212) 85.804 ms 85.801 ms 14 209.85.143.215 (209.85.143.215) 44.797 ms 46.584 ms 209.85.143.115 (209.85.143.115) 53.004 ms 15 209.85.244.209 (209.85.244.209) 55.639 ms 79.083 ms 57.160 ms 16 216.239.51.169 (216.239.51.169) 60.986 ms 57.830 ms 216.239.51.97 (216.239.51.97) 64.996 ms 17 216.239.40.8 (216.239.40.8) 93.801 ms 216.239.46.153 (216.239.46.153) 94.068 ms 99.649 ms 18 209.85.248.124 (209.85.248.124) 83.771 ms 64.233.174.191 (64.233.174.191) 83.781 ms 209.85.248.124 (209.85.248.124) 83.386 ms
I'm not sure that it would be a "pfSense" software issue as much as a configuration issue/hardware issue. I can try the MTR and see if shows the same. I know when I made changes to the interfaces though, for a short period of time the ping does return to normal though, and then comes back to problematic.
-
The TTL is just indicative of how many hops it takes to reach the destination. Which is up to your ISP generally, though if you have really screwy internal routing with multiple routers, that could be attributable to your internal network.
Do you recognize any of those 10.x.x.x IPs in hops 2-10 there? The latency is high enough that they'd almost have to be on your ISP's network. And probably in two different locations judging by the latency difference (hops 2-4 close to you, 5-10 maybe 200-400 miles away). Something different with the WAN as it's configured on pfSense is being treated much differently by your ISP, and being routed an unusual-looking path. ISP routers shouldn't reply from private IP sources. Some will, with public IPs becoming more scarce, but only for a hop or two generally. I've never seen an ISP with private IPs across 10 consecutive hops.
That traceroute shows higher latency getting to a real Internet router than you're describing to reach Google plugged directly into the modem.
-
@cmb:
The TTL is just indicative of how many hops it takes to reach the destination. Which is up to your ISP generally, though if you have really screwy internal routing with multiple routers, that could be attributable to your internal network.
Do you recognize any of those 10.x.x.x IPs in hops 2-10 there? The latency is high enough that they'd almost have to be on your ISP's network. And probably in two different locations judging by the latency difference (hops 2-4 close to you, 5-10 maybe 200-400 miles away). Something different with the WAN as it's configured on pfSense is being treated much differently by your ISP, and being routed an unusual-looking path. ISP routers shouldn't reply from private IP sources. Some will, with public IPs becoming more scarce, but only for a hop or two generally. I've never seen an ISP with private IPs across 10 consecutive hops.
That traceroute shows higher latency getting to a real Internet router than you're describing to reach Google plugged directly into the modem.
CMB,
I now do feel rather stupid….LOL .. So looking at this, when my desktop "pings google" it's pinging a ISP server apparently, thus showing average latency being 10-15ms where real google server resides at the 216.58.217.206 IP. Ping 74.125.224.72 resulted in the same latency as the pfSense box and TTL.
I guess, at this point, I need to figure out why the network seemed to be "much slower" than it does when not going through pfSense, and why it keeps dropping out when connected as well.
Thanks again! :)