8200/6100 Intel x553 NIC and certain SFP+ modules

aholmes5

I'm on a 2gbps symmetric fiber plan. I get 2300mbps up and down when I use my extra switch with a dac cable in the 8200's WAN port. When I plug the PON module into the WAN port directly (doesn't matter ix0 or ix1), I get upload speeds of 2300mbps but download speeds will be 1500mbps then within a minute or two deteriorate to 30-40mbps.

stephenw10

No errors/collisions on the interface (Status > Interfaces) when that happens?

aholmes5

@stephenw10 Lots of In errors. No out errors and no collisions.

stephenw10

Interesting. Then check the stats in the sysctl to see what sort of errors they are: sysctl dev.ix.0

The C3K NICs are limited in the information they can get from modules about the link state. That can mean they negotiate incorrectly or fail to link with some media types. XPON modules that that hide the real link and present to the NIC as something else so there are a few things there that could cause problems.

aholmes5

@stephenw10 SFP in 8200.txt SFP in switch.txt

Here is what I have. I don't know how to interpret this. Lots of errors:
dev.ix.0.mac_stats.rec_len_errs: 20
dev.ix.0.mac_stats.remote_faults: 0
dev.ix.0.mac_stats.local_faults: 53
dev.ix.0.mac_stats.short_discards: 7
dev.ix.0.mac_stats.byte_errs: 1425
dev.ix.0.mac_stats.ill_errs: 873
dev.ix.0.mac_stats.crc_errs: 5468
dev.ix.0.mac_stats.rx_errs: 6374

stephenw10

Mmm, interesting no one type....

Check netstat -m make sure there's no buffer exhaustion happening.

aholmes5

@stephenw10
58799/4456/63255 mbufs in use (current/cache/total)
49273/3559/52832/1000000 mbuf clusters in use (current/cache/total/max)
3/2029 mbuf+clusters out of packet secondary zone in use (current/cache)
0/1524/1524/524288 4k (page size) jumbo clusters in use (current/cache/total/max)
0/0/0/524288 9k jumbo clusters in use (current/cache/total/max)
0/0/0/84161 16k jumbo clusters in use (current/cache/total/max)
113245K/14328K/127573K bytes allocated to network (current/cache/total)
0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters)
0/0/0 requests for mbufs delayed (mbufs/clusters/mbuf+clusters)
0/0/0 requests for jumbo clusters delayed (4k/9k/16k)
0/0/0 requests for jumbo clusters denied (4k/9k/16k)
0 sendfile syscalls
0 sendfile syscalls completed without I/O request
0 requests for I/O initiated by sendfile
0 pages read by sendfile as part of a request
0 pages were valid at time of a sendfile request
0 pages were valid and substituted to bogus page
0 pages were requested for read ahead by applications
0 pages were read ahead by sendfile
0 times sendfile encountered an already busy page
0 requests for sfbufs denied
0 requests for sfbufs delayed

lio1503

Hi @aholmes5

I have the same issue. I connect my Fiber WAN on SFP+ 10G on netgate 6100 with a 10GTEK sfp+10G and my unifi switch on the second WAN on SFP+ 10G on netgate 6100 with DAC 10G cable but my download speed is reduce of 80% but my upload is fine.
Do you found a solution ?