IPv6 packet loss on host machine

JKnott

No, but then I don't run pfSense in a virtual machine.

–- ipv6.google.com ping statistics ---
1000 packets transmitted, 1000 received, 0% packet loss, time 1000244ms

johnpoz

I run pfsense on VM (esxi 6.5) I run HE.. never seen this problem…

Just ran you ping -c 1000 test..

That fix was done back in 2014.. And you are NOT seeing anything like the problem they discussed which is miss a few packets.. At a repeated interval..

Your test doesn't seem valid for counting your packet loss

At 989, missed 17 packets
At 998, missed 8 packets

How do you have 17 packets missing at 989 if your only counting to 1000, and then 8 at 998... I didn't want to wait so long so set 200ms between pings..

64 bytes from ord37s09-in-x0e.1e100.net: icmp_seq=996 ttl=56 time=11.2 ms
64 bytes from ord37s09-in-x0e.1e100.net: icmp_seq=997 ttl=56 time=12.8 ms
64 bytes from ord37s09-in-x0e.1e100.net: icmp_seq=998 ttl=56 time=12.8 ms
64 bytes from ord37s09-in-x0e.1e100.net: icmp_seq=999 ttl=56 time=15.0 ms
64 bytes from ord37s09-in-x0e.1e100.net: icmp_seq=1000 ttl=56 time=11.6 ms

--- ipv6.google.com ping statistics ---
1000 packets transmitted, 1000 received, 0% packet loss, time 200513ms
rtt min/avg/max/mdev = 10.568/15.925/353.653/17.353 ms, pipe 2

pablot

Thanks for your reply, perhaps the script is not counting correctly, but the problem still exists, have a look at this ping output:

PING ipv6.google.com(atl14s80-in-x0e.1e100.net) 56 data bytes
64 bytes from atl14s80-in-x0e.1e100.net: icmp_seq=19 ttl=56 time=160 ms
64 bytes from atl14s80-in-x0e.1e100.net: icmp_seq=20 ttl=56 time=160 ms
64 bytes from atl14s80-in-x0e.1e100.net: icmp_seq=21 ttl=56 time=160 ms
64 bytes from atl14s80-in-x0e.1e100.net: icmp_seq=22 ttl=56 time=160 ms
64 bytes from atl14s80-in-x0e.1e100.net: icmp_seq=32 ttl=56 time=159 ms
64 bytes from atl14s80-in-x0e.1e100.net: icmp_seq=51 ttl=56 time=159 ms
64 bytes from atl14s80-in-x0e.1e100.net: icmp_seq=52 ttl=56 time=159 ms
64 bytes from atl14s80-in-x0e.1e100.net: icmp_seq=53 ttl=56 time=160 ms
64 bytes from atl14s80-in-x0e.1e100.net: icmp_seq=54 ttl=56 time=160 ms
64 bytes from atl14s80-in-x0e.1e100.net: icmp_seq=55 ttl=56 time=167 ms
64 bytes from atl14s80-in-x0e.1e100.net: icmp_seq=56 ttl=56 time=159 ms
64 bytes from atl14s80-in-x0e.1e100.net: icmp_seq=57 ttl=56 time=160 ms
64 bytes from atl14s80-in-x0e.1e100.net: icmp_seq=58 ttl=56 time=160 ms
64 bytes from atl14s80-in-x0e.1e100.net: icmp_seq=59 ttl=56 time=160 ms
64 bytes from atl14s80-in-x0e.1e100.net: icmp_seq=68 ttl=56 time=159 ms
64 bytes from atl14s80-in-x0e.1e100.net: icmp_seq=69 ttl=56 time=160 ms
64 bytes from atl14s80-in-x0e.1e100.net: icmp_seq=70 ttl=56 time=160 ms
64 bytes from atl14s80-in-x0e.1e100.net: icmp_seq=71 ttl=56 time=160 ms
64 bytes from atl14s80-in-x0e.1e100.net: icmp_seq=80 ttl=56 time=159 ms
64 bytes from atl14s80-in-x0e.1e100.net: icmp_seq=89 ttl=56 time=159 ms
64 bytes from atl14s80-in-x0e.1e100.net: icmp_seq=98 ttl=56 time=161 ms
64 bytes from atl14s80-in-x0e.1e100.net: icmp_seq=99 ttl=56 time=160 ms
64 bytes from atl14s80-in-x0e.1e100.net: icmp_seq=108 ttl=56 time=159 ms
64 bytes from atl14s80-in-x0e.1e100.net: icmp_seq=109 ttl=56 time=159 ms
^C
--- ipv6.google.com ping statistics ---
109 packets transmitted, 24 received, 77% packet loss, time 108327ms
rtt min/avg/max/mdev = 159.288/160.524/167.526/1.647 ms

I'm not using esxi so perhaps this is not a problem when using esxi but only when using KVM, I'm really clueless about how I can solve this….

The problem shows only in the host machine, as any other machines on the network works without missing any packets, also other VMs on the same machine and on the same NICs works ok with zero packet loss.

Any kernel parameters, KVM configurations...?

johnpoz

There is a sticky about KVM and pfsense.. Suggest you go through that.

https://forum.pfsense.org/index.php?topic=88467.0

You have no such issues with ipv4?

pablot

@johnpoz:

There is a sticky about KVM and pfsense.. Suggest you go through that.

https://forum.pfsense.org/index.php?topic=88467.0

You have no such issues with ipv4?

I've experienced that problem some time ago and done the offloading "trick", but that seems to be another different problem as that is with IPv4 and on the other guest machines.

My guests VMs are working perfectly on both IPv6 and IPv4, only host machine is having problems with IPv6.

johnpoz

How exactly are you bringing this HE tunnel into the host if the tunnel endpoints on your VM?

pablot

@johnpoz:

How exactly are you bringing this HE tunnel into the host if the tunnel endpoints on your VM?

I'm not sure what your question is, but the tunnel is from and to the pfSense VM (followed HE intructions), and the host is just another machine on the network as the other VMs or "real" machines that work ok.

johnpoz

The host is not another machine on the network.. IT has a interface - that is most likely sharing the connection?

Please explain how your physically connected and how you have your virtual network setup.. My esxi host gets IPv6 connectivity to the internet through my pfsense VM, which uses an HE tunnel. But it gets this connectivity through its vmkern interface that is connected to the vswitch that is the "lan" of pfsense via physical interface that is connected to the same network… So when my esxi host wants to talk to the internet be it ipv4 or ipv6 it goes out its vmkern interface that goes out the phy nic to the phy switch. Then back in from different port on the switch to different phy nic to the "lan" vswitch that is connected to pfsense lan then out the HE tunnel..

So vmkern or the host is connected via a phy port on a switch, that is connected to a different phy port on the host that is pfsense "lan" then in the host it routes via the pfsense vm that is also connected to a different phy nic and then connected to the modem and then the internet. See attached if that helps.. So what I am asking is how exactly does your "host" have connectivity to your pfsense vm that hosts the HE tunnel.

phyconnections.png_thumb

pablot

@johnpoz:

The host is not another machine on the network.. IT has a interface - that is most likely sharing the connection?

Please explain how your physically connected and how you have your virtual network setup…

Ok, sorry for being so "short" in my answer. I'll try to be as detailed as possible.

You're right, the host is not like other machines. My host have 3 "real" NICs, one for LAN and two for two ISPs (one cable and one aDSL). Those 3 NICs are defined as bridges on my host machine:


pablot@deathstar2:~$ cat /etc/network/interfaces
# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).

source /etc/network/interfaces.d/*

# The loopback network interface
auto lo
iface lo inet loopback

# El bridge es ahora la interface principal par ala lan
auto br0
iface br0 inet static
        address 192.168.2.1
        netmask 255.255.255.0
        network 192.168.2.0
        broadcast 192.168.2.255
        gateway 192.168.2.13
        bridge_ports enp2s0
        bridge_stp off
        bridge_fd 0
        bridge_maxwait 5
        # dns-* options are implemented by the resolvconf package, if installed
        dns-nameservers 192.168.2.13 8.8.8.8
        dns-search localdomain

auto br1
iface br1 inet manual
        bridge_ports enp4s0
        bridge_stp off
        bridge_fd 0
        bridge_maxwait 0

auto br2
iface br2 inet manual
        bridge_ports enp5s0
        bridge_stp off
        bridge_fd 0
        bridge_maxwait 0

br0 -> LAN 192.168.2.1: connected to a Netgear R7000 with Xwrt firmware. Ethernet clients are connected to its 4 port switch.
br1 -> cable ISP: connected to a cable router configured as a bridge
br2 -> aDSL ISP: connected to an adsl router configured as a bridge

This are the interfaces defined on this host:


pablot@deathstar2:~$ ifconfig
br0       Link encap:Ethernet  HWaddr e0:3f:49:0e:5e:c0
          inet addr:192.168.2.1  Bcast:192.168.2.255  Mask:255.255.255.0
          inet6 addr: 2001:---------:7346/64 Scope:Global
          inet6 addr: 2001:---------:5ec0/64 Scope:Global
          inet6 addr: fe80::e23f:49ff:fe0e:5ec0/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:92808 errors:0 dropped:146 overruns:0 frame:0
          TX packets:40968 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:13286698 (13.2 MB)  TX bytes:11077212 (11.0 MB)

br1       Link encap:Ethernet  HWaddr 18:d6:c7:05:13:33
          inet6 addr: fe80::1ad6:c7ff:fe05:1333/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:13561 errors:0 dropped:0 overruns:0 frame:0
          TX packets:102 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:1986311 (1.9 MB)  TX bytes:9164 (9.1 KB)

br2       Link encap:Ethernet  HWaddr 18:d6:c7:05:b9:99
          inet6 addr: 2001:----------:6efc/64 Scope:Global
          inet6 addr: fe80::1ad6:c7ff:fe05:b999/64 Scope:Link
          inet6 addr: 2001:----------:b999/64 Scope:Global
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:53687 errors:0 dropped:146 overruns:0 frame:0
          TX packets:1090 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:10265924 (10.2 MB)  TX bytes:93660 (93.6 KB)

enp2s0    Link encap:Ethernet  HWaddr e0:3f:49:0e:5e:c0
          inet6 addr: fe80::e23f:49ff:fe0e:5ec0/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:9920669 errors:0 dropped:0 overruns:0 frame:0
          TX packets:12343885 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:4935037983 (4.9 GB)  TX bytes:9476588488 (9.4 GB)

enp4s0    Link encap:Ethernet  HWaddr 18:d6:c7:05:13:33
          inet6 addr: fe80::1ad6:c7ff:fe05:1333/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:9472337 errors:0 dropped:0 overruns:0 frame:0
          TX packets:7692095 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:7600631722 (7.6 GB)  TX bytes:3626500190 (3.6 GB)

enp5s0    Link encap:Ethernet  HWaddr 18:d6:c7:05:b9:99
          inet6 addr: fe80::1ad6:c7ff:fe05:b999/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:3473420 errors:0 dropped:0 overruns:0 frame:0
          TX packets:2616932 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:2311752848 (2.3 GB)  TX bytes:1382280998 (1.3 GB)

lo        Link encap:Local Loopback
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:65536  Metric:1
          RX packets:541 errors:0 dropped:0 overruns:0 frame:0
          TX packets:541 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1
          RX bytes:43448 (43.4 KB)  TX bytes:43448 (43.4 KB)

virbr0    Link encap:Ethernet  HWaddr 52:54:00:4f:28:e6
          inet addr:192.168.122.1  Bcast:192.168.122.255  Mask:255.255.255.0
          UP BROADCAST MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

vnet0     Link encap:Ethernet  HWaddr fe:54:00:57:2e:07
          inet6 addr: fe80::fc54:ff:fe57:2e07/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:10118 errors:0 dropped:0 overruns:0 frame:0
          TX packets:23981 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:1859305 (1.8 MB)  TX bytes:5408313 (5.4 MB)

vnet1     Link encap:Ethernet  HWaddr fe:54:00:0a:74:e6
          inet6 addr: fe80::fc54:ff:fe0a:74e6/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:823860 errors:0 dropped:0 overruns:0 frame:0
          TX packets:592780 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:901727626 (901.7 MB)  TX bytes:89828043 (89.8 MB)

vnet2     Link encap:Ethernet  HWaddr fe:54:00:3e:1b:ef
          inet6 addr: fe80::fc54:ff:fe3e:1bef/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:553930 errors:0 dropped:0 overruns:0 frame:0
          TX packets:795473 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:82024196 (82.0 MB)  TX bytes:827958514 (827.9 MB)

vnet3     Link encap:Ethernet  HWaddr fe:54:00:15:57:0b
          inet6 addr: fe80::fc54:ff:fe15:570b/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:177061 errors:0 dropped:0 overruns:0 frame:0
          TX packets:190347 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:23176258 (23.1 MB)  TX bytes:89195814 (89.1 MB)

vnet4     Link encap:Ethernet  HWaddr fe:54:00:2f:88:6d
          inet6 addr: fe80::fc54:ff:fe2f:886d/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:224468 errors:0 dropped:0 overruns:0 frame:0
          TX packets:274035 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:25030471 (25.0 MB)  TX bytes:173332703 (173.3 MB)

The vnetX interfaces are created by KVM/QEMU to be assigned to the VMs so I have assigned interfaces as this:

vnet0 -> pfSense LAN - IP: 192.168.2.13
vnet2 -> pfSense FIBERTEL  (ISP1 cable)
vnet3 -> pfSense ARNET (ISP2 aDSL)

(See attached NICs config below - pf1.png and pf2.png)

I also have other VMs wich also use the br0 bridge through the host device vnet0 and works perfectly both on IPv4 and IPv6 (see attached NIC config below - ubuntu.png)

My HE tunnel goes through the cable ISP (will config another tunnel through the aDSL ISP in the future)

My host gets IPv6 connectivity to the internet through my pfsense VM, which uses an HE tunnel. It gets it's connectivity through the br0 interface. That same phisical connection is also the the LAN of pfSense, as the vtnet0 is assigned to the LAN interface on pfSense VM.

(I'm attaching my network diagram, there are more things, like VLANs, etc.)

pf-inter-assign.png_thumb

pf1.png_thumb

pf2.png_thumb

ubuntu.png_thumb
![Network pablot - VLANs.png](/public/imported_attachments/1/Network pablot - VLANs.png)
![Network pablot - VLANs.png_thumb](/public/imported_attachments/1/Network pablot - VLANs.png_thumb)

johnpoz

Both of bridge interfaces on your host have ipv6 on them.. So why would it be going through pfsense HE tunnel??

pablot

@johnpoz:

Both of bridge interfaces on your host have ipv6 on them.. So why would it be going through pfsense HE tunnel??

I'm sorry, but is that wrong?, do I have to remove ipv6 addresses from them?. I guess that the br2 address is not supposed to be there, but what about br0? is that wrong?

pablot

I've checked disabling completely IPv6 on the br1 and br2 interfaces and still teh same problem, so I'm not sure what else to do.