LAN ping spikes
-
Hello Guys!
I have one little problem.
When I pinging LAN side of pFsese 2.2.4 from my workstation it sometimes got spikes - like this:Reply from 192.168.235.2: bytes=32 time<1ms TTL=64 Reply from 192.168.235.2: bytes=32 time<1ms TTL=64 Reply from 192.168.235.2: bytes=32 time=173ms TTL=64 Reply from 192.168.235.2: bytes=32 time=204ms TTL=64 Reply from 192.168.235.2: bytes=32 time=56ms TTL=64 Reply from 192.168.235.2: bytes=32 time<1ms TTL=64 Reply from 192.168.235.2: bytes=32 time<1ms TTL=64 Reply from 192.168.235.2: bytes=32 time<1ms TTL=64 Reply from 192.168.235.2: bytes=32 time<1ms TTL=64 Reply from 192.168.235.2: bytes=32 time<1ms TTL=64 Reply from 192.168.235.2: bytes=32 time<1ms TTL=64 Reply from 192.168.235.2: bytes=32 time<1ms TTL=64 Reply from 192.168.235.2: bytes=32 time<1ms TTL=64 Reply from 192.168.235.2: bytes=32 time<1ms TTL=64 Reply from 192.168.235.2: bytes=32 time<1ms TTL=64 Reply from 192.168.235.2: bytes=32 time<1ms TTL=64
The same spikes can bee seen if i ping something in the internet(pFsense box does NAT for me).
If I run the "netstat -w 1 -I em0" I see input errors exactly the same time the ping time rises:
input em0 output packets errs idrops bytes packets errs bytes colls 904 0 0 1013282 689 0 228110 0 684 0 0 631475 629 0 283227 0 922 0 0 958064 695 0 312959 0 1055 0 0 1109577 1004 0 725150 0 1411 0 0 1210390 1127 0 756909 0 1182 0 0 1077355 903 0 620903 0 1323 0 0 1470344 917 0 436095 0 6667 0 0 1197142 1405 0 1368559 0 9670 669 0 1016874 1612 0 1951349 0 2511 274 0 1157574 2159 0 2524190 0 959 0 0 974904 732 0 400994 0 1322 0 0 956421 1342 0 1389144 0 12974 710 0 1014202 1123 0 1440876 0 2658 280 0 950083 2036 0 2474490 0 1927 0 0 1430998 1753 0 1836173 0 2853 0 0 1268560 1444 0 1466548 0 5935 0 0 1516476 1557 0 1525974 0 1734 0 0 896831 619 0 371422 0 1371 0 0 1168824 1069 0 798293 0
Printout of "sysctl -a | grep em.0"
dev.em.0.%desc: Intel(R) PRO/1000 Network Connection 7.4.2 dev.em.0.%driver: em dev.em.0.%location: slot=0 function=0 dev.em.0.%pnpinfo: vendor=0x8086 device=0x10d3 subvendor=0x8086 subdevice=0xa01f class=0x020000 dev.em.0.%parent: pci1 dev.em.0.nvm: -1 dev.em.0.debug: -1 dev.em.0.fc: 3 dev.em.0.rx_int_delay: 0 dev.em.0.tx_int_delay: 66 dev.em.0.rx_abs_int_delay: 66 dev.em.0.tx_abs_int_delay: 66 dev.em.0.itr: 488 dev.em.0.rx_processing_limit: 100 dev.em.0.eee_control: 1 dev.em.0.link_irq: 758588 dev.em.0.mbuf_alloc_fail: 0 dev.em.0.cluster_alloc_fail: 0 dev.em.0.dropped: 0 dev.em.0.tx_dma_fail: 0 dev.em.0.rx_overruns: 0 dev.em.0.watchdog_timeouts: 0 dev.em.0.device_control: 1477444168 dev.em.0.rx_control: 67403778 dev.em.0.fc_high_water: 18432 dev.em.0.fc_low_water: 16932 dev.em.0.queue0.txd_head: 544 dev.em.0.queue0.txd_tail: 546 dev.em.0.queue0.tx_irq: 64261306 dev.em.0.queue0.no_desc_avail: 5 dev.em.0.queue0.rxd_head: 404 dev.em.0.queue0.rxd_tail: 402 dev.em.0.queue0.rx_irq: 65211157 dev.em.0.mac_stats.excess_coll: 0 dev.em.0.mac_stats.single_coll: 0 dev.em.0.mac_stats.multiple_coll: 0 dev.em.0.mac_stats.late_coll: 0 dev.em.0.mac_stats.collision_count: 0 dev.em.0.mac_stats.symbol_errors: 0 dev.em.0.mac_stats.sequence_errors: 0 dev.em.0.mac_stats.defer_count: 0 dev.em.0.mac_stats.missed_packets: 1414487 dev.em.0.mac_stats.recv_no_buff: 0 dev.em.0.mac_stats.recv_undersize: 0 dev.em.0.mac_stats.recv_fragmented: 0 dev.em.0.mac_stats.recv_oversize: 0 dev.em.0.mac_stats.recv_jabber: 0 dev.em.0.mac_stats.recv_errs: 0 dev.em.0.mac_stats.crc_errs: 0 dev.em.0.mac_stats.alignment_errs: 0 dev.em.0.mac_stats.coll_ext_errs: 0 dev.em.0.mac_stats.xon_recvd: 2056237 dev.em.0.mac_stats.xon_txd: 22458 dev.em.0.mac_stats.xoff_recvd: 7061457 dev.em.0.mac_stats.xoff_txd: 1436945 dev.em.0.mac_stats.total_pkts_recvd: 86865296 dev.em.0.mac_stats.good_pkts_recvd: 77619751 dev.em.0.mac_stats.bcast_pkts_recvd: 67927 dev.em.0.mac_stats.mcast_pkts_recvd: 3282 dev.em.0.mac_stats.rx_frames_64: 3139958 dev.em.0.mac_stats.rx_frames_65_127: 47622772 dev.em.0.mac_stats.rx_frames_128_255: 1542585 dev.em.0.mac_stats.rx_frames_256_511: 449763 dev.em.0.mac_stats.rx_frames_512_1023: 1423140 dev.em.0.mac_stats.rx_frames_1024_1522: 23441533 dev.em.0.mac_stats.good_octets_recvd: 39455455874 dev.em.0.mac_stats.good_octets_txd: 102685879557 dev.em.0.mac_stats.total_pkts_txd: 89834208 dev.em.0.mac_stats.good_pkts_txd: 88374805 dev.em.0.mac_stats.bcast_pkts_txd: 1508 dev.em.0.mac_stats.mcast_pkts_txd: 48 dev.em.0.mac_stats.tx_frames_64: 3296140 dev.em.0.mac_stats.tx_frames_65_127: 14504847 dev.em.0.mac_stats.tx_frames_128_255: 1033178 dev.em.0.mac_stats.tx_frames_256_511: 805231 dev.em.0.mac_stats.tx_frames_512_1023: 1253425 dev.em.0.mac_stats.tx_frames_1024_1522: 67481984 dev.em.0.mac_stats.tso_txd: 524651 dev.em.0.mac_stats.tso_ctx_fail: 0 dev.em.0.interrupts.asserts: 569449 dev.em.0.interrupts.rx_pkt_timer: 69 dev.em.0.interrupts.rx_abs_timer: 0 dev.em.0.interrupts.tx_pkt_timer: 0 dev.em.0.interrupts.tx_abs_timer: 5 dev.em.0.interrupts.tx_queue_empty: 0 dev.em.0.interrupts.tx_queue_min_thresh: 0 dev.em.0.interrupts.rx_desc_min_thresh: 0 dev.em.0.interrupts.rx_overrun: 174
Every lag I've got "mac_stats.missed_packets" increased
My box is configured
em0 LAN interface with 10 VLANS on it(users consuming 5-20Mbit), 192.168.235.2 is untaged.
em1 WAN (static IP)
re0 WAN2 (DHCP)
re1 not used.CPU Core2Quad q9650 3.0Ghz top shows CPU loads 2-4% while running a speedtest on a 100Mbit connection.
I've replaced the network adapter, switch and cable, but the behavior is the same, some time ago I've replaced the whole box and configured it from the scratch(not just restoring the config), but nothing changed.PS: I'm sorry for my newbie FreeBSD knowledge and for my terrible English.
-
A L2 cache collision in the CPU I dont think will show in the CPU utilisation, nor will any caching done by any nics with onboard processing capabilities like intel nics.
Have you tried packet capturing using a separate bridged device between pfsense and your workstation to see whats actually going on with the packets across the network, you can get a better idea of what the packets are really doing then?
Might also be useful. http://blog.serverfault.com/2011/03/23/performance-tuning-intel-nics/