SFP with fiber hand-off, won't negotiate - XG-7100 router to Ciena 3930

stephenw10

If you run at the command line, or via Diag > Command Prompt: ifconfig -vvm ix0

That should show you the status of the SFP module. What exactly is the module you're trying to use?

Steve

shapelytraffic

This is the output, although I've redacted the public IP.
The device is a MENARA NETWORKS PN: 5LR0P20-1310.
Are the dB levels the light emission levels?

ix0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
	options=e400bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6>
	capabilities=f507bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWFILTER,VLAN_HWTSO,NETMAP,RXCSUM_IPV6,TXCSUM_IPV6>
	ether 00:08:a2:0f:0e:38
	hwaddr 00:08:a2:0f:0e:38
	inet A.B.C.18 netmask 0xfffffff8 broadcast A.B.C.23
	nd6 options=23<PERFORMNUD,ACCEPT_RTADV,AUTO_LINKLOCAL>
	media: Ethernet autoselect
	status: no carrier
	supported media:
		media autoselect
		media 10Gbase-LR
	plugged: SFP/SFP+/SFP28 10G Base-LR (LC)
	vendor: MENARA NETWORKS PN: 5LR0P20-1310 SN: NE500426 DATE: 2014-05-20
	module temperature: 30.68 C Voltage: 3.30 Volts
	RX: 0.70 mW (-1.49 dBm) TX: 0.85 mW (-0.68 dBm)

stephenw10

Yes I would expect so. What speed is it trying to connect at? It will probably need to be hard set to 1G if that's what it's trying.

Steve

shapelytraffic

This is a 2G link.
So, nothing sticks out at you as being an issue?

stephenw10

It's trying to link at 2Gbps? Like 1G/1G? Not 2.5Gbps?

That seems unlikely. The upstream connection to the ISP might be rated at 2Gbps but the local link will probably be 10G or far less commonly 2.5G.

The ix NICs in the XG-7100 will not link at 2Gbps. They are 10Gbps NICs but should link at 1Gbps also.

Steve

shapelytraffic

Sorry, 10G, don't know what I was talking about.

shapelytraffic

I am trying a work around by using a copper port on the Ciena. I can route my laptop across that link when I set the IP and connect the laptop directly to the copper hand-off. But when I connect the Netgate to the copper hand-off via Eth1, the gateway is down and wont reply to arp or ping.

Regarding the Eth1 config, I've assigned the WAN interface to 4090 on LAGG0. And I've added the VLAN as a member of port 1 on the switch. What am I doing incorrectly? Seems like the Eth1 config should be easy to set up but I'm missing something.

stephenw10

That's the default config, Eth1 as WAN using DHCP. You could just reset the 7100 to factory defaults to be sure.

Check the switch port status on the interfaces widget on the dashboard. Does Eth1 show as linked? Do you see link LEDs on it?

Steve

shapelytraffic

No worries, it turned out I had taken the VLAN off ports 9 and 10. When I put the VLAN back on 9 and 10 with tagging, it came up.

stephenw10

Ah, cool.

I would expect it to work on ix0 though. We have seen some modules that require setting 10G specifically before they link.
I don't think we have ever tested the particular module you're using.

Steve

shapelytraffic

Can you recommend a 1310, single mode SFP that will work with XG-7100? I am not getting anything to detect a carrier. Also, is there some command line I need to run in order to force the lasers on?

Derelict

Upon initial insert of an SFP you might need either of the following:

ifconfig ixX down ; ifconfig ixX up

or

ifconfig ixX media autoselect

It depends on the module.

Note that this is only for initial module insertion after boot. Once it's up and running it'll reboot fine and you can remove/reinsert the LC cable.

I have personally tested the following. They were ordered with Intel-specific code.

10Gtek "AXS13-192-10 10/1G SMF"
FiberStore "SFP-10GLR-31 10/1G SMF"

I have also tested Add On and Intel-branded MMF modules and they worked (Again, Intel-specific code ordered). I have no reason to believe their single-mode modules would not also work. SMF was too spendy for those for my personal test lab.

shapelytraffic

Hi Derelict, thanks for the advice.

I did not have success with either set of commands.

I am using FS SFP-10GLR-31 as well as 10Gtek ASX-13-192-02 and ASX-130-192-10.

I did have an Intel 850 in at first that worked with a dumb switch. That won't work with the fiber hand-off though.

Derelict

Do you have anything else to test link with besides the ISP?

You can always use the other ix port for that.

shapelytraffic

Yes, bear with me.

shapelytraffic

With a second XG-7100, I set the ix0 to the ISP address. Ping was successful. Last week, I observed while the hand-off switch port was tested. FWIW.

Derelict

I have not had to do anything to connect to a Brocade ICX-6450 10G port.

I did have to issue the following to a Cisco 3750 1G fiber port to get link to come up there:

speed nonegotiate

on the switch port connected to the XG-7100.

Derelict

Another option is going through an outside switch between the Ciena device and the XG-7100.

A benefit of that arrangement is being able to perform diagnostic captures of what's actually on the "wire" with a mirror port there.

shapelytraffic

I have a ticket open with support now, I will post any finding from the ticket transcript here. For any who are interested.

shapelytraffic

@Derelict Are you a Netgate support guru? If I give you my ticket number, do you think you could take it? I think the last post was from a person with an asynchronous work schedule to my own.