8200/6100 Intel x553 NIC and certain SFP+ modules
-
I have a netgate 8200 which uses the intel atom c3000 SOC with built in intel x553 Ethernet for its SFP+ ports. When I use my XGS-PON module in either of the SFP+ ports, my download speed is very slow and deteriorates over minutes to maybe 5% of expected throughput. Upload speed always remains fast. By moving the PON module to a cheap unmanaged switch with two SFP+ ports and using a passive DAC cable to connect the switch to the 8200, throughput is normal. Based on what I've read this seems to be a driver issue in BSD with the x553. I'm guessing the 6100 is also affected since it also has an Atom C3000 series with the Intel x553.
-
What speeds do you usually get? Do you see any errors on the interface?
Yes the 6100 would be identical in that respect.
-
I'm on a 2gbps symmetric fiber plan. I get 2300mbps up and down when I use my extra switch with a dac cable in the 8200's WAN port. When I plug the PON module into the WAN port directly (doesn't matter ix0 or ix1), I get upload speeds of 2300mbps but download speeds will be 1500mbps then within a minute or two deteriorate to 30-40mbps.
-
No errors/collisions on the interface (Status > Interfaces) when that happens?
-
@stephenw10 Lots of In errors. No out errors and no collisions.
-
Interesting. Then check the stats in the sysctl to see what sort of errors they are:
sysctl dev.ix.0The C3K NICs are limited in the information they can get from modules about the link state. That can mean they negotiate incorrectly or fail to link with some media types. XPON modules that that hide the real link and present to the NIC as something else so there are a few things there that could cause problems.
-
@stephenw10 SFP in 8200.txt SFP in switch.txt
Here is what I have. I don't know how to interpret this. Lots of errors:
dev.ix.0.mac_stats.rec_len_errs: 20
dev.ix.0.mac_stats.remote_faults: 0
dev.ix.0.mac_stats.local_faults: 53
dev.ix.0.mac_stats.short_discards: 7
dev.ix.0.mac_stats.byte_errs: 1425
dev.ix.0.mac_stats.ill_errs: 873
dev.ix.0.mac_stats.crc_errs: 5468
dev.ix.0.mac_stats.rx_errs: 6374 -
Mmm, interesting no one type....
Check
netstat -mmake sure there's no buffer exhaustion happening. -
@stephenw10
58799/4456/63255 mbufs in use (current/cache/total)
49273/3559/52832/1000000 mbuf clusters in use (current/cache/total/max)
3/2029 mbuf+clusters out of packet secondary zone in use (current/cache)
0/1524/1524/524288 4k (page size) jumbo clusters in use (current/cache/total/max)
0/0/0/524288 9k jumbo clusters in use (current/cache/total/max)
0/0/0/84161 16k jumbo clusters in use (current/cache/total/max)
113245K/14328K/127573K bytes allocated to network (current/cache/total)
0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters)
0/0/0 requests for mbufs delayed (mbufs/clusters/mbuf+clusters)
0/0/0 requests for jumbo clusters delayed (4k/9k/16k)
0/0/0 requests for jumbo clusters denied (4k/9k/16k)
0 sendfile syscalls
0 sendfile syscalls completed without I/O request
0 requests for I/O initiated by sendfile
0 pages read by sendfile as part of a request
0 pages were valid at time of a sendfile request
0 pages were valid and substituted to bogus page
0 pages were requested for read ahead by applications
0 pages were read ahead by sendfile
0 times sendfile encountered an already busy page
0 requests for sfbufs denied
0 requests for sfbufs delayed
