Packet Loss

[jamescr]

Hi,

I've been playing with TNSR in our lab and I've come across a bizarre behaviour that I cannot explain, and hoping someone can help! This is bare metal, on a Xeon-D that is totally idle, no traffic flowing through but it does have full table BGP sessions. We have a mix of i350 for 1gb and x710 for 10gb.

The issue we're facing is one of the 1gb connections is experiencing 5% packet loss. The other end of the 1gb connection is an ADVA nte for a fibre based Internet connection.

Adva -> cat6 lead -> TNSR = 5% packet loss (verified on two different ports on TNSR)
Adva -> cat6 lead -> Brocade Switch -> cat6 lead -> TNSR = 0% packet loss (same cable, same port on tnsr, zero config changes in tnsr. Literally just physically inserting the switch in the way)
Adva -> cat6 lead -> Brocade switch -> direct attach SFP+ cable -> TNSR = 0% packet loss (into a 10gb port)
TNSR -> anything else = 0% packet loss, literally only seems to be to this adva.

So for some reason when directly connected to the adva we get packet loss, but when we put a switch in the middle we get no packet loss. The trouble shooting guide doesn't cover this scenario, nor have I been able to find much online for how to debug this. The Adva is an FSP150.

Thanks

[Derelict]

@jamescr Sounds like something in the link that your tnsr interface or tha adva doesn't like when they are directly connected.

When you have reproduced enough packet loss to measure, run

host shell sudo vppctl show hardware-interface

show packet-counters

See if anything interesting displays there.

Do the same on the adva looking at its counters and see if it has anything interesting to say.

If putting a switch in between corrects the problem you are 99.9% looking at some obscure issue at Layer 1.

What does Adva support have to say?

[jamescr]

Thanks for the reply, we don't have any support or access from Adva as it's supplied by the carrier. This circuit is also being terminated at the end of next month (the entire reason I've been using it to test, as we've already moved traffic away from it).

I've done further tests, I put a Mikrotik between the TNSR box and the Adva and I can verify via a pcap that the traffic is being transmitted out of TNSR, I can verify it's also leaving the Mikrotik to the Adva - and in Wireshark it looks in no way obviously different to the none dropped pings. Where it gets odd is the Mikrotik now gets packet loss to the other side of the Adva. It didn't used to, so I am thinking the issue is very much on the adva/carrier side and not the tnsr side. Just it only manifested itself when we started to test tnsr. So we jumped to the conclusion that it was an issue in tnsr.

I think in this situation I'm going to have to ask the carrier to investigate.

[Danil]

This post is deleted!