Intermittent "no route to host" on my LAN-port

[tsmalmbe]

I will check this. My lan port is connected "untagged" while I am also running two other VLAN's on the same port as "tagged".

This is how the procurve log looks like. I just had to reboot pfsense a minute ago, but no relevant logs prior to that. Port 1 is my LAN port

05/14/17 13:47:17 SNTP: updated time by -4 seconds
05/15/17 08:43:51 ports: port 34 is now off-line
05/15/17 08:43:51 ports: port 1 is now off-line
05/15/17 08:43:51 ports: port 37 is now off-line
05/15/17 08:43:51 ports: port 41 is now off-line
05/15/17 08:43:54 ports: port 1 is Blocked by LACP
05/15/17 08:43:54 ports: port 34 is Blocked by LACP
05/15/17 08:43:54 ports: port 37 is Blocked by LACP
05/15/17 08:43:55 ports: port 41 is Blocked by LACP
05/15/17 08:43:57 ports: port 1 is now on-line

[tsmalmbe]

For what it is worth, I tested to ping my lan as described earlier: "no route to host". Then I pinged a VLAN which is on the same port, success and ping goes through without any problems.

[tsmalmbe]

I disabled all hardware offloading (had two out of three disabled previously). Just to see if it makes a difference. I have rebooted twice today already.

[bingo600]

@tsmalmbe:

05/14/17 13:47:17 SNTP: updated time by -4 seconds
05/15/17 08:43:51 ports: port 34 is now off-line
05/15/17 08:43:51 ports: port 1 is now off-line
05/15/17 08:43:51 ports: port 37 is now off-line
05/15/17 08:43:51 ports: port 41 is now off-line
05/15/17 08:43:54 ports: port 1 is Blocked by LACP
05/15/17 08:43:54 ports: port 34 is Blocked by LACP
05/15/17 08:43:54 ports: port 37 is Blocked by LACP
05/15/17 08:43:55 ports: port 41 is Blocked by LACP
05/15/17 08:43:57 ports: port 1 is now on-line

Next time login to the procurve and snag the log before rebooting the pfsense.

I'm a pfsense newbie (but know networking) , and would not expect it to participate in STP (spanning tree protocol) as it's a L3 firewall.
But then again … It has bridge mode ... and should be STP capable (at least i that mode)

So you have 2 tagged vlans and one untagged vlan on that interface ?

Is it only the untagged vlan that have this problem , or does the same problem occur (at the same time) on the other tagged vlans ?

/Bingo

[tsmalmbe]

Wierd as it seems, it is only the untagged VLAN that starts to behave badly, the tagged VLAN's are completely fine.

[tsmalmbe]

Surely there has to be a loglevel that can reveal what the problem is?

ping: sendto: No route to host
ping: sendto: No route to host
ping: sendto: No route to host
ping: sendto: No route to host
ping: sendto: No route to host
ping: sendto: No route to host
64 bytes from 192.168.1.9: icmp_seq=168 ttl=64 time=1.496 ms
64 bytes from 192.168.1.9: icmp_seq=169 ttl=64 time=0.721 ms
64 bytes from 192.168.1.9: icmp_seq=170 ttl=64 time=0.742 ms
64 bytes from 192.168.1.9: icmp_seq=171 ttl=64 time=0.915 ms
64 bytes from 192.168.1.9: icmp_seq=172 ttl=64 time=0.784 ms
64 bytes from 192.168.1.9: icmp_seq=173 ttl=64 time=0.510 ms
64 bytes from 192.168.1.9: icmp_seq=174 ttl=64 time=1.138 ms
64 bytes from 192.168.1.9: icmp_seq=175 ttl=64 time=0.760 ms
64 bytes from 192.168.1.9: icmp_seq=176 ttl=64 time=0.586 ms
64 bytes from 192.168.1.9: icmp_seq=177 ttl=64 time=0.664 ms
64 bytes from 192.168.1.9: icmp_seq=178 ttl=64 time=0.724 ms
ping: sendto: No route to host
ping: sendto: No route to host
ping: sendto: No route to host
ping: sendto: No route to host
ping: sendto: No route to host
ping: sendto: No route to host
64 bytes from 192.168.1.9: icmp_seq=185 ttl=64 time=1.198 ms
64 bytes from 192.168.1.9: icmp_seq=186 ttl=64 time=1.222 ms
64 bytes from 192.168.1.9: icmp_seq=187 ttl=64 time=0.845 ms
64 bytes from 192.168.1.9: icmp_seq=188 ttl=64 time=0.760 ms
64 bytes from 192.168.1.9: icmp_seq=189 ttl=64 time=0.890 ms
64 bytes from 192.168.1.9: icmp_seq=190 ttl=64 time=0.815 ms
ping: sendto: No route to host
ping: sendto: No route to host
ping: sendto: No route to host
ping: sendto: No route to host
ping: sendto: No route to host
ping: sendto: No route to host
ping: sendto: No route to host
ping: sendto: No route to host
ping: sendto: No route to host
ping: sendto: No route to host
ping: sendto: No route to host
ping: sendto: No route to host
ping: sendto: No route to host
ping: sendto: No route to host
ping: sendto: No route to host
64 bytes from 192.168.1.9: icmp_seq=206 ttl=64 time=1.395 ms
64 bytes from 192.168.1.9: icmp_seq=207 ttl=64 time=1.413 ms
64 bytes from 192.168.1.9: icmp_seq=208 ttl=64 time=0.548 ms
64 bytes from 192.168.1.9: icmp_seq=209 ttl=64 time=0.767 ms
64 bytes from 192.168.1.9: icmp_seq=210 ttl=64 time=0.872 ms
64 bytes from 192.168.1.9: icmp_seq=211 ttl=64 time=0.613 ms
64 bytes from 192.168.1.9: icmp_seq=212 ttl=64 time=0.740 ms
64 bytes from 192.168.1.9: icmp_seq=213 ttl=64 time=0.770 ms
64 bytes from 192.168.1.9: icmp_seq=214 ttl=64 time=0.585 ms
64 bytes from 192.168.1.9: icmp_seq=215 ttl=64 time=0.809 ms
64 bytes from 192.168.1.9: icmp_seq=216 ttl=64 time=0.738 ms
64 bytes from 192.168.1.9: icmp_seq=217 ttl=64 time=0.865 ms
64 bytes from 192.168.1.9: icmp_seq=218 ttl=64 time=0.780 ms
64 bytes from 192.168.1.9: icmp_seq=219 ttl=64 time=0.705 ms
ping: sendto: No route to host
ping: sendto: No route to host
ping: sendto: No route to host
ping: sendto: No route to host
ping: sendto: No route to host
ping: sendto: No route to host
ping: sendto: No route to host
ping: sendto: No route to host
64 bytes from 192.168.1.9: icmp_seq=228 ttl=64 time=1.111 ms

[bingo600]

Can you ping the pfSense ip address , on the affected untagged vlan, while the problem is there??

/Bingo

[tsmalmbe]

@bingo600:

Can you ping the pfSense ip address , on the affected untagged vlan, while the problem is there??

A lan wks -> pfSense ip FAIL
pfSense -> a lan wks FAIL
pfSense -> a tagged lan server on the same port as the untagged lan SUCCESS
A lan wks -> a tagged lan server on the same port as the untagged lan SUCCESS

Like that.

[tsmalmbe]

I now moved my LAN from untagged to tagged (I tagged it on pfSense and on the switch). It will be a bitch if my switch dies to recover, but theres always something you can do over the terminal.

Anyway, let's see if this solves the issue. Then we know that VLANs and raw LAN's and pfSense and Watchguard and Procurve donät work together well.

[Derelict]

Some people say don't mix tagged and untagged traffic on an interface for a reason.

I would suspect an ARP issue there, but those intervals are awfully short for that. Could also be a simple no carrier on the ethernet interface. Have you tried another cable? Another switchport? But if it is only the default VLAN and not the tagged interfaces, that pretty much rules out layer 1.

Dealing with a tagged port is not really a bitch to deal with if you have the right tools.


[tsmalmbe]

ARP would've been my guess also, but then I don't really understand why it only starts after a while, and then the problem doesn't go away by itself - and the cycle is quite short as you said. But let's see.

The bitchy part is mostly if I need to remove the fw and hook up a laptop directly to the LAN, then it will require a bit of fiddling to get that going. As long as you document the VLAN's it's not such a big deal. And perhaps the correct way is to use the switch to handle the mixing and matching of VLAN's on the switch ports and then just either show all tagged or all untagged to the firewall.

[tsmalmbe]

It gets worse.

My two OpenVPN tunnels are now doing the same thing.
Request timed out.
Request timed out.
Reply from 10.99.10.1: bytes=32 time=48ms TTL=63
Reply from 10.99.10.1: bytes=32 time=48ms TTL=63
Reply from 10.99.10.1: bytes=32 time=48ms TTL=63
Request timed out.
Reply from 10.99.10.1: bytes=32 time=49ms TTL=63
Reply from 10.99.10.1: bytes=32 time=52ms TTL=63
Reply from 10.99.10.1: bytes=32 time=47ms TTL=63
Request timed out.
Reply from 10.99.10.1: bytes=32 time=49ms TTL=63
Reply from 10.99.10.1: bytes=32 time=48ms TTL=63
Reply from 10.99.10.1: bytes=32 time=48ms TTL=63
Reply from 10.99.10.1: bytes=32 time=48ms TTL=63
Reply from 10.99.10.1: bytes=32 time=49ms TTL=63
Request timed out.
Request timed out.
Reply from 10.99.10.1: bytes=32 time=64ms TTL=63
Reply from 10.99.10.1: bytes=32 time=48ms TTL=63
Reply from 10.99.10.1: bytes=32 time=47ms TTL=63
Reply from 10.99.10.1: bytes=32 time=48ms TTL=63
Reply from 10.99.10.1: bytes=32 time=48ms TTL=63
Reply from 10.99.10.1: bytes=32 time=49ms TTL=63
Request timed out.
Request timed out.
Reply from 10.99.10.1: bytes=32 time=149ms TTL=63
Reply from 10.99.10.1: bytes=32 time=47ms TTL=63
Reply from 10.99.10.1: bytes=32 time=49ms TTL=63
Reply from 10.99.10.1: bytes=32 time=49ms TTL=63
Reply from 10.99.10.1: bytes=32 time=49ms TTL=63

And

Reply from 192.168.69.1: bytes=32 time=126ms TTL=63
Reply from 192.168.69.1: bytes=32 time=36ms TTL=63
Request timed out.
Request timed out.
Reply from 192.168.69.1: bytes=32 time=70ms TTL=63
Reply from 192.168.69.1: bytes=32 time=110ms TTL=63
Reply from 192.168.69.1: bytes=32 time=61ms TTL=63
Request timed out.
Request timed out.
Reply from 192.168.69.1: bytes=32 time=108ms TTL=63
Reply from 192.168.69.1: bytes=32 time=68ms TTL=63
Reply from 192.168.69.1: bytes=32 time=107ms TTL=63
Request timed out.
Request timed out.
Request timed out.
Request timed out.
Request timed out.
Request timed out.
Request timed out.
Reply from 192.168.69.1: bytes=32 time=109ms TTL=63
Request timed out.
Request timed out.
Reply from 192.168.69.1: bytes=32 time=68ms TTL=63
Reply from 192.168.69.1: bytes=32 time=100ms TTL=63
Reply from 192.168.69.1: bytes=32 time=75ms TTL=63
Reply from 192.168.69.1: bytes=32 time=103ms TTL=63
Reply from 192.168.69.1: bytes=32 time=65ms TTL=63
Request timed out.
Request timed out.
Reply from 192.168.69.1: bytes=32 time=108ms TTL=63
Reply from 192.168.69.1: bytes=32 time=68ms TTL=63
Reply from 192.168.69.1: bytes=32 time=93ms TTL=63
Reply from 192.168.69.1: bytes=32 time=67ms TTL=63
Reply from 192.168.69.1: bytes=32 time=106ms TTL=63
Request timed out.

Also pingig the firewall

Reply from 192.168.1.1: bytes=32 time=1ms TTL=64
Reply from 192.168.1.1: bytes=32 time=1ms TTL=64
Request timed out.
Reply from 192.168.1.1: bytes=32 time=1ms TTL=64
Reply from 192.168.1.1: bytes=32 time=1ms TTL=64
Reply from 192.168.1.1: bytes=32 time=1ms TTL=64
Reply from 192.168.1.1: bytes=32 time=2ms TTL=64
Reply from 192.168.1.1: bytes=32 time=1ms TTL=64
Reply from 192.168.1.1: bytes=32 time=1ms TTL=64
Request timed out.
Reply from 192.168.1.1: bytes=32 time=2ms TTL=64
Reply from 192.168.1.1: bytes=32 time=1ms TTL=64
Reply from 192.168.1.1: bytes=32 time=2ms TTL=64
Reply from 192.168.1.1: bytes=32 time=1ms TTL=64
Request timed out.
Reply from 192.168.1.1: bytes=32 time=2ms TTL=64
Request timed out.
Reply from 192.168.1.1: bytes=32 time=2ms TTL=64
Reply from 192.168.1.1: bytes=32 time=2ms TTL=64
Request timed out.
Reply from 192.168.1.1: bytes=32 time=4ms TTL=64
Reply from 192.168.1.1: bytes=32 time=1ms TTL=64
Reply from 192.168.1.1: bytes=32 time=1ms TTL=64
Reply from 192.168.1.1: bytes=32 time=1ms TTL=64
Request timed out.
Reply from 192.168.1.1: bytes=32 time=2ms TTL=64

and poinging my wan

Reply from 188.117.46.161: bytes=32 time=17ms TTL=56
Request timed out.
Reply from 188.117.46.161: bytes=32 time=19ms TTL=56
Reply from 188.117.46.161: bytes=32 time=21ms TTL=56
Reply from 188.117.46.161: bytes=32 time=45ms TTL=56
Reply from 188.117.46.161: bytes=32 time=17ms TTL=56
Reply from 188.117.46.161: bytes=32 time=19ms TTL=56
Reply from 188.117.46.161: bytes=32 time=17ms TTL=56
Reply from 188.117.46.161: bytes=32 time=17ms TTL=56
Reply from 188.117.46.161: bytes=32 time=17ms TTL=56
Request timed out.
Reply from 188.117.46.161: bytes=32 time=18ms TTL=56
Reply from 188.117.46.161: bytes=32 time=18ms TTL=56
Reply from 188.117.46.161: bytes=32 time=17ms TTL=56
Reply from 188.117.46.161: bytes=32 time=18ms TTL=56
Request timed out.
Reply from 188.117.46.161: bytes=32 time=19ms TTL=56

I really hate this shit at the moment.

[tsmalmbe]

So now everything is pretty fucked until I once again reboot the firewall.

[tsmalmbe]

When the problem (which is now different but the same) occurs, the firewall can now ping everything so no problem there. But now I lose

• VLAN (LAN) -> ovpns5,
• VLAN (LAN)->VLAN (SERVERS1) and
• VLAN (LAN) -> VLAN (SERVERS2).

VLAN (LAN) to the internet is all fine and working.

There has to be some way to debug this.

[tsmalmbe]

Dumping the routing when the issue occurs:

[2.3.4-RELEASE][root@firewall.ccccccccc.fi]/root: netstat -rn
Routing tables

Internet:
Destination        Gateway            Flags      Netif Expire
default            217.30.178.254    UGS        sk0
10.10.1.0/24      link#4            U          sk3
10.10.1.1          link#4            UHS        lo0
10.99.0.0/16      10.100.100.2      UGS      ovpns5
10.100.100.1      link#13            UHS        lo0
10.100.100.2      link#13            UH      ovpns5
10.100.101.1      link#14            UHS        lo0
10.100.101.2      link#14            UH      ovpns8
10.200.200.0/24    10.200.200.2      UGS      ovpns1
10.200.200.1      link#12            UHS        lo0
10.200.200.2      link#12            UH      ovpns1
10.200.210.0/24    10.100.100.2      UGS      ovpns5
83.145.193.133    217.30.178.254    UGHS        sk0
127.0.0.1          link#8            UH          lo0
192.168.1.0/24    link#11            U      sk1_vlan
192.168.1.1        link#11            UHS        lo0
192.168.1.3        link#11            UHS        lo0
192.168.2.21      link#11            UHS        lo0
192.168.2.21/32    link#11            U      sk1_vlan
192.168.10.0/24    link#9            U      sk1_vlan
192.168.10.1      link#9            UHS        lo0
192.168.20.0/24    link#10            U      sk1_vlan
192.168.20.1      link#10            UHS        lo0
192.168.69.0/24    10.100.101.2      UGS      ovpns8
192.168.100.0/24  link#3            U          sk2
192.168.100.1      link#3            UHS        lo0
217.30.178.0/24    link#1            U          sk0
217.30.178.237    link#1            UHS        lo0

[tsmalmbe]

After reboot

Routing tables

Internet:
Destination        Gateway            Flags      Netif Expire
default            217.30.178.254    UGS        sk0
10.10.1.0/24      link#4            U          sk3
10.10.1.1          link#4            UHS        lo0
10.99.0.0/16      10.100.100.2      UGS      ovpns5
10.100.100.1      link#13            UHS        lo0
10.100.100.2      link#13            UH      ovpns5
10.100.101.1      link#14            UHS        lo0
10.100.101.2      link#14            UH      ovpns8
10.200.200.0/24    10.200.200.2      UGS      ovpns1
10.200.200.1      link#12            UHS        lo0
10.200.200.2      link#12            UH      ovpns1
10.200.210.0/24    10.100.100.2      UGS      ovpns5
83.145.193.133    217.30.178.254    UGHS        sk0
127.0.0.1          link#8            UH          lo0
192.168.1.0/24    link#11            U      sk1_vlan
192.168.1.1        link#11            UHS        lo0
192.168.1.3        link#11            UHS        lo0
192.168.2.21      link#11            UHS        lo0
192.168.2.21/32    link#11            U      sk1_vlan
192.168.10.0/24    link#9            U      sk1_vlan
192.168.10.1      link#9            UHS        lo0
192.168.20.0/24    link#10            U      sk1_vlan
192.168.20.1      link#10            UHS        lo0
192.168.69.0/24    10.100.101.2      UGS      ovpns8
192.168.100.0/24  link#3            U          sk2
192.168.100.1      link#3            UHS        lo0
217.30.178.0/24    link#1            U          sk0
217.30.178.237    link#1            UHS        lo0

[Derelict]

As far as I know, almost nobody is using those sk NICs.

At the same time I haven't heard of anything like what you're seeing either.

[Derelict]

192.168.1.0/24    link#11            U      sk1_vlan
192.168.1.1        link#11            UHS        lo0
192.168.1.3        link#11            UHS        lo0
192.168.2.21      link#11            UHS        lo0
192.168.2.21/32    link#11            U      sk1_vlan

Why are both of those on link #11?

netstat -rnWfinet

[tsmalmbe]

1.1 is the router itself
1.3 is a virtual ip
2.21 is a virtual ip

192.168.1.3/32 LAN  IP Alias Virtual IP for wpad.malmberg.fi
192.168.2.21/32 LAN  IP Alias Virtual IP for wiki.malmberg.fi

I also realized that I have had it defined as 1.3/24 instead of 1.3/32 - this I have now changed.

I have run your suggested netstat -command BEFORE and AFTER this change.

BEFORE

netstat -rnWfinet
Routing tables

Internet:
Destination        Gateway            Flags      Use    Mtu      Netif Expire
default            217.30.178.254    UGS      307699  1500        sk0
10.10.1.0/24      link#4            U          401  1500        sk3
10.10.1.1          link#4            UHS          0  16384        lo0
10.99.0.0/16      10.100.100.2      UGS      66905  1500    ovpns5
10.100.100.1      link#13            UHS          0  16384        lo0
10.100.100.2      link#13            UH            6  1500    ovpns5
10.100.101.1      link#14            UHS          0  16384        lo0
10.100.101.2      link#14            UH            0  1500    ovpns8
10.200.200.0/24    10.200.200.2      UGS          0  1500    ovpns1
10.200.200.1      link#12            UHS          0  16384        lo0
10.200.200.2      link#12            UH            0  1500    ovpns1
10.200.210.0/24    10.100.100.2      UGS          0  1500    ovpns5
83.145.193.133    217.30.178.254    UGHS    111003  1500        sk0
127.0.0.1          link#8            UH      354961  16384        lo0
192.168.1.0/24    link#11            U      2433840  1500  sk1_vlan5
192.168.1.1        link#11            UHS          0  16384        lo0
192.168.1.3        link#11            UHS          0  16384        lo0
192.168.2.21      link#11            UHS        810  16384        lo0
192.168.2.21/32    link#11            U            0  1500  sk1_vlan5
192.168.10.0/24    link#9            U          487  1500  sk1_vlan6
192.168.10.1      link#9            UHS          0  16384        lo0
192.168.20.0/24    link#10            U        36126  1500  sk1_vlan7
192.168.20.1      link#10            UHS          0  16384        lo0
192.168.69.0/24    10.100.101.2      UGS          0  1500    ovpns8
192.168.100.0/24  link#3            U            0  1500        sk2
192.168.100.1      link#3            UHS          0  16384        lo0
217.30.178.0/24    link#1            U            0  1500        sk0
217.30.178.237    link#1            UHS          0  16384        lo0

AFTER

Routing tables

Internet:
Destination        Gateway            Flags      Use    Mtu      Netif Expire
default            217.30.178.254    UGS      309567  1500        sk0
10.10.1.0/24      link#4            U          401  1500        sk3
10.10.1.1          link#4            UHS          0  16384        lo0
10.99.0.0/16      10.100.100.2      UGS      67281  1500    ovpns5
10.100.100.1      link#13            UHS          0  16384        lo0
10.100.100.2      link#13            UH            6  1500    ovpns5
10.100.101.1      link#14            UHS          0  16384        lo0
10.100.101.2      link#14            UH            0  1500    ovpns8
10.200.200.0/24    10.200.200.2      UGS          0  1500    ovpns1
10.200.200.1      link#12            UHS          0  16384        lo0
10.200.200.2      link#12            UH            0  1500    ovpns1
10.200.210.0/24    10.100.100.2      UGS          0  1500    ovpns5
83.145.193.133    217.30.178.254    UGHS    111477  1500        sk0
127.0.0.1          link#8            UH      357517  16384        lo0
192.168.1.0/24    link#11            U      2435972  1500  sk1_vlan5
192.168.1.1        link#11            UHS          0  16384        lo0
192.168.1.3        link#11            UHS          0  16384        lo0
192.168.1.3/32    link#11            U            0  1500  sk1_vlan5
192.168.2.21      link#11            UHS          0  16384        lo0
192.168.2.21/32    link#11            U            0  1500  sk1_vlan5
192.168.10.0/24    link#9            U          490  1500  sk1_vlan6
192.168.10.1      link#9            UHS          0  16384        lo0
192.168.20.0/24    link#10            U        36269  1500  sk1_vlan7
192.168.20.1      link#10            UHS          0  16384        lo0
192.168.69.0/24    10.100.101.2      UGS          0  1500    ovpns8
192.168.100.0/24  link#3            U            0  1500        sk2
192.168.100.1      link#3            UHS          0  16384        lo0
217.30.178.0/24    link#1            U            0  1500        sk0
217.30.178.237    link#1            UHS          0  16384        lo0

[tsmalmbe]

I know this is not a perfect timeline, but two simultaneous windows though.

11:33:52.822204 ARP, Request who-has 192.168.20.7 tell 192.168.20.1, length 28
11:33:52.822442 ARP, Reply 192.168.20.7 is-at 7a:f5:5b:ff:c9:c4, length 42
11:33:52.822452 IP 192.168.1.191 > 192.168.20.7: ICMP echo request, id 6, seq 11405, length 40
11:33:52.822642 IP 192.168.20.7 > 192.168.1.191: ICMP echo reply, id 6, seq 11405, length 40
11:33:53.822241 ARP, Request who-has 192.168.20.7 tell 192.168.20.1, length 28
11:33:53.822469 ARP, Reply 192.168.20.7 is-at 7a:f5:5b:ff:c9:c4, length 42
11:33:53.822482 IP 192.168.1.191 > 192.168.20.7: ICMP echo request, id 6, seq 11407, length 40
11:33:53.822669 IP 192.168.20.7 > 192.168.1.191: ICMP echo reply, id 6, seq 11407, length 40
11:33:54.823260 ARP, Request who-has 192.168.20.7 tell 192.168.20.1, length 28
11:33:54.823490 ARP, Reply 192.168.20.7 is-at 7a:f5:5b:ff:c9:c4, length 42
11:33:54.823501 IP 192.168.1.191 > 192.168.20.7: ICMP echo request, id 6, seq 11409, length 40
11:33:54.823690 IP 192.168.20.7 > 192.168.1.191: ICMP echo reply, id 6, seq 11409, length 40
11:33:55.825579 IP 192.168.1.191 > 192.168.20.7: ICMP echo request, id 6, seq 11411, length 40
11:33:55.825812 IP 192.168.20.7 > 192.168.1.191: ICMP echo reply, id 6, seq 11411, length 40
11:33:56.826596 IP 192.168.1.191 > 192.168.20.7: ICMP echo request, id 6, seq 11414, length 40
11:33:56.826836 IP 192.168.20.7 > 192.168.1.191: ICMP echo reply, id 6, seq 11414, length 40
11:33:57.829819 IP 192.168.1.191 > 192.168.20.7: ICMP echo request, id 6, seq 11417, length 40
11:33:57.830054 IP 192.168.20.7 > 192.168.1.191: ICMP echo reply, id 6, seq 11417, length 40

Reply from 192.168.1.1: Destination host unreachable.
Reply from 192.168.1.1: Destination host unreachable.
Reply from 192.168.1.1: Destination host unreachable.
Reply from 192.168.1.1: Destination host unreachable.
Reply from 192.168.1.1: Destination host unreachable.
Reply from 192.168.1.1: Destination host unreachable.
Reply from 192.168.20.7: bytes=32 time=3ms TTL=63
Reply from 192.168.20.7: bytes=32 time=114ms TTL=63
Reply from 192.168.20.7: bytes=32 time=2ms TTL=63
Reply from 192.168.20.7: bytes=32 time=3ms TTL=63
Reply from 192.168.20.7: bytes=32 time=3ms TTL=63
Reply from 192.168.20.7: bytes=32 time=6ms TTL=63
Reply from 192.168.20.7: bytes=32 time=1ms TTL=63
Reply from 192.168.20.7: bytes=32 time=3ms TTL=63
Reply from 192.168.20.7: bytes=32 time=2ms TTL=63
Reply from 192.168.20.7: bytes=32 time=2ms TTL=63
Reply from 192.168.20.7: bytes=32 time=1ms TTL=63
Reply from 192.168.20.7: bytes=32 time=3ms TTL=63

So WTF is going on with these ARP's.