Strange IPv6 connection problem

[Alphaphi-by]

Hi all,

today I would like to seek help for a connection problem which I was not able to solve for a while now. The description below is all about IPv6. V4 works fine.

This is my setup:

                internet
                    ^
                    |
              +---------+
              |  pppoe0 |
              |         |
              |   igb0  |
              +---------+
                    |
                    v
                 LAN/Wifi

In the LAN, there are Windows, Linux and Android clients.

Android Clients don't have any problem.

Windows and Linux clients do have problems with a few websites, but not all.

• Example for a problematic site: https://www143.your-server.de
• Example for a working site: htps://ipv6.google.com or test-ipv6.com (10/10 points there)

On the problematic site, the client just gets a timeout, for example:

raspi$ time curl -6 -sS --max-time 10 https://www143.your-server.de
curl: (28) Connection timed out after 10001 milliseconds

real    0m10,066s
user    0m0,165s
sys     0m0,043s

If I do the exact same call on the pfsense itself, it works without problems:

/root: time curl -6 -sS --max-time 10 https://www143.your-server.de > /dev/null
0.068u 0.015s 0:00.13 53.8%     194+237k 0+0io 0pf+0w

In my understanding this tells me that there must be a problem on the client (not likely) or on the pfSense (most likely). From the ISP (Dt. Telekom) onwards all works fine as we see.

I took network traces on the raspi. What I see there is that the TCP handshake is OK (small packets), and I think also the get request goes out OK (packet no. 4 with length 591).

When I take a trace on pppoe0 and make a working curl-call in the pfsense, I see the same packets going in and out, but then a couple of big packets come in, which contain the payload I guess (packets 6 and 7)

Now all of this looks not very exciting. ICMP "Packet too big" is dropped by some paranoid filter, IPv6 black hole, that's it.
BUT first of all: I don't see any such ICMP packet coming in on pppoe0. There is no ICMP traffic at all on pppoe0 when I run the curl call on the raspi.
Secondly: I just don't have no firewall rule that would block ICMP.

I suspect that the pfSense is misbehaving. The reason for this is that I played around with NPt. And I read somewhere that "Packet too big" messages get lost when NPt is active. However, I disabled NPt completely and deleted any related config. And anyway there are no packet-too-big packets anyway, so none can get lost.

If someone is interested I can provide the trace files.

Does anyone here have any idea what steps I could take next?

[Alphaphi-by]

Hi all,

let me put one additional info here. I made another call to https://www143.your-server.de on the Raspi, and traced on the Raspi itself (eth0), on the LAN interface of pfSense (igb1) and on the WAN interface of pfSense (pppoe0).

Result:
eth0:

igb1:

pppoe0:

As you see, the packets are exactly the same, i.e.: nothing is dropped.
The only difference is that packets on pppoe0 are always 10 bytes larger than on igb1 and eth0. As it seems, outgoing packets grow, incoming packets shrink. No idea why that is. I have no VPN or other encapsulation techniques, other than pppoe. But does pppoe trigger this 10 bytes difference?

Again, thanks for any hint.

PS: pls note that LAN is on igb1, not igb0 (my mistake in the first posting)

[johnpoz]

@Alphaphi-by yeah pppoe adds overhead.. I thought it was 8 bytes (2+6), but been forever since have done anything with pppoe - its horrible, for starters because of the overhead.. You seeing 10 might just be how wireshark is displaying it?

But yeah if you have a pppoe connecttion, there will be additional overhead.

[Alphaphi-by]

@johnpoz Not sure where this comes from. Don't think that Wireshark is lying, and it seems to OK anyway, so no reason to dig deeper into it.

Anyway this obviously is not a hint for my original problem.

No one any ideas about where my connection problems come from, or how to narrow it down at least?

[johnpoz]

@Alphaphi-by said in Strange IPv6 connection problem:

Don't think that Wireshark is lying,

I didn't say it was lying - I said it might display the overhead differently.. For example it doesn't show you the overhead of vlan tags normally..

Be it 2 or 6 or 8 or 10.. I thought the overhead with pppoe was normally 8.. But maybe its 10.. And who knows ipv6 might be different? Again its been awhile since did anything with pppoe, let alone via a packet capture.

My point was yes there is overhead - so yes as you move from normal network with no overhead to a network with added overhead because of the pppoe.. You would see this.

As to your problem - looks like fins were sent, and then that IP sent a RST.. Other than a couple of dup mentions.. Which didn't look enough and not enough info about your network, etc. where captured, etc. .etc.. Looks like connection, opened then closed - and rst sent, which isn't uncommon to see.