SFP transceiver to connect to ADVA box
-
Urgh. I'm sorry it's giving you such a hard time.
We have seen NICs and switches that the XG-7100 won't link to without setting the link speed or advertisement speed. But that doesn't seem to be happening here, it is linked.
If you want to try though you could set the advertisement speed using the sysctl:
dev.ix.0.advertise_speed: Control advertised link speed using these flags: 0x1 - advertise 100M 0x2 - advertise 1G 0x4 - advertise 10G 0x8 - advertise 10M 100M and 10M are only supported on certain adapters.So try:
sysctl dev.ix.0.advertise_speed=0x2Really though getting some info about the link from the other end would be very useful here. I understand getting that from the ISP (or worse Openreach) is blood-from-stone territory though.
Steve
-
@stephenw10 It's Talktalk, who are FAR more unhelpful than Openreach! I will never buy a leased line from them again as they are utterly incompetent. Bloody torture!
-
@orangehand Hi, Did you ever end up getting a resolution on this? Unfortunately I'm experiencing the exact same thing on a site.
-
@stephenw10 Hi Stephen,
Did you hear any further on this one at all? We are having an identical issue on an install to the same Network Terminating Equipment but can't get the link to come up.
We have the PFSense working using an SFP 1Gbps module to a switch but it will not work to the NTE. When connecting the NTE to a switch that works fine.
-
No sorry I'm not aware of anything specific there.
To connect the 10G NICs at 1G often requires setting the link speed fixed and that can be dependent of the module used. If you try to connect with a DAC cable for example it usually does not present the option to set 1G to the OS.
It's an ADVA FSP150CP? You know anything about how it's configured?
Steve
-
@stephenw10 Thanks for the quick reply it's much appreciated.
Unfortunately we don't have any ability to access the Adva or it's configuration. It is indeed an FSP150CP. Openreach for whatever reason will not enable the RJ-45 port which would solve this issue.
We had a lot of problems getting the XG7100 to run at 1Gbps as when using a 10Gbps optic the driver wouldn't allow us to advertise at 1Gbps as the command is not valid for this device. In the end we installed a 1Gbps LC SFP module and this seems to be doing the trick to a switch.
However to the ADVA it doesn't work. I believe the issue is on the ADVA end, we're going to try the advertisment command again on Thursday but have also sourced a media converter to bypass the IX0 & IX1 ports on the XG7100 as these are proving tricky to work with.
-
@davehayden Sorry Dave, I missed this. Short answer no. TTBiz were of no use whatsoever and although @stephenw10 tried really hard to help we were unable to sort it. I did order a media convertor but by that time we had actually given up and shelled out another 80/mo for a Cisco from TTB. Although all it does is convert SFP to GigE! Bottom line is I would NEVER EVER use TT again. They were shockingly unhelpful and frankly didn't seem to know anything. Given that the Adva is Openreach equipment, there was never any info conduit to them. It's all a bit of a racket it seems!
-
@davehayden It is utterly beyond me why for a sub-gig circuit they won't enable the RJ45. Anyone have any insight into their stupidity?!
-
@orangehand Appreciate the response thank you! It is completely baffling, I can only imagine they want to future proof it, or they have better visibility of Optics statistics than cabled, but either way the presentation out of the NTE should be down to customer desire.
-
Unfortunately it looks like those things are locked down intentionally which means testing is limited. But they are incredibly cheap on ebay... so we'll see.
BT/Openreach have very tight rules about how and where they install things. That seems legitimate in most cases but not enabling the RJ-45 port here is a bit of a mystery IMO. It could be to do with electrical isolation perhaps?
Probably it was simply never tested/certified for anything but fibre so that's all they allow.Steve
-
OK I acquired one of these to test with but unfortunately I don't think it will even try to link unless it sees a valid upstream connection and they are locked down. No chance of re-configuring.
However I'm pretty sure the one I have is configured to use the RJ-45 port. And a lot, if not most, you see for sale have the SFP port still covered with the dust plug. So I would suggest it is definitely possible to have them supplied like that.
I have so far been unable to get any sort of link at all to the SFP port. And in fact once the Adva had booted it turns off the optics completely. That's not what @orangehand was seeing so I think this is configured differently.
Steve
-
@stephenw10 That's WAY above and beyond! Because the Adva is supplied by Openreach, to whom I have no access there is nowhere to go I don't think. Deeply frustrating, and it's cost my customer a fortune for all this burgering about. But I may just take the opportuniny to say how utterly sh1te TT are! We've all learnt something I think!
-
Update: With some different SFP modules I was able to get an upstream connection on the ADVA 'A' port. With that in place I can now get a connection on the 'Access' side and it acts purely as a media converter from what I can see, a layer 2 device.
However on this particular device it's the RJ-45 port that is enabled so I'm still unable to test the SFP connection.
It does at least show it's possible to have that enabled.
Steve
-
@stephenw10 What circuit is that on and which provider? I am guessing it's on a 100mb/s bearer hence the RJ45 port being enabled? But I would love to have you tell us it's a gig bearer with TT!!
-
It's not on an Openreach circuit at all, it's just connected locally. Turns out you don't need anything from some upstream management to enable the ports as long as it sees a valid link.
As far as I know Openreach only supply that with single mode BiDi style modules. That's a, cheap as I could get, 1G MM module to local firewall.
I was only doing it to try to test linking the SFP access port to a 7100 or any other Netgate device but I still can't do that because this one is configured for RJ-45 and not SFP.
Steve
-
@stephenw10 Thanks for thinking of this!
-
Another update: I got hold of an FSP150CP that is configured to use the SFP port and prepared to do battle. However to my surprise it just worked:
That's using an ADVA module (0061003008) at the ADVA end and a similar 1000BASE-LX SM module in the 7100.
No boot loading or sysctl values set.
The interface in the 7100 appears as:[22.01-RELEASE][root@7100.stevew.lan]/root: ifconfig -vvvm ix3 ix3: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500 description: WAN options=e138bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,WOL_UCAST,WOL_MCAST,WOL_MAGIC,VLAN_HWFILTER,RXCSUM_IPV6,TXCSUM_IPV6> capabilities=f53fbb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,WOL_UCAST,WOL_MCAST,WOL_MAGIC,VLAN_HWFILTER,VLAN_HWTSO,NETMAP,RXCSUM_IPV6,TXCSUM_IPV6> ether 00:08:a2:0e:a5:92 inet6 fe80::208:a2ff:fe0e:a592%ix3 prefixlen 64 scopeid 0xe inet 172.21.16.187 netmask 0xffffff00 broadcast 172.21.16.255 media: Ethernet autoselect (Unknown <rxpause,txpause>) status: active supported media: media autoselect nd6 options=23<PERFORMNUD,ACCEPT_RTADV,AUTO_LINKLOCAL> plugged: SFP/SFP+/SFP28 1000BASE-LX (LC) vendor: Metrodata PN: S12W1310S010C15C SN: 27CI190809108 DATE: 2019-08-12 module temperature: 40.80 C Voltage: 3.34 Volts RX: 0.47 mW (-3.25 dBm) TX: 0.21 mW (-6.66 dBm) SFF8472 DUMP (0xA0 0..127 range): 03 04 07 00 00 00 02 00 00 00 00 01 0D 00 0A 64 37 37 00 00 4D 65 74 72 6F 64 61 74 61 20 20 20 20 20 20 20 00 00 17 6A 53 31 32 57 31 33 31 30 53 30 31 30 43 31 35 43 30 30 31 20 05 1E 00 72 00 1A 00 00 32 37 43 49 31 39 30 38 30 39 31 30 38 20 20 20 31 39 30 38 31 32 20 20 68 F0 03 13 00 00 06 B3 5C 9E F5 F3 09 75 2E 40 0F 11 9D AE 09 DF B1 00 00 00 00 00 00 00 00 00 18 E4 0E 4BIt's ix3 there because I have a card in the box that becomes ix0/1.
One thing I noted is that the ADVA box shows it accepts an SFP module by lighting the 'Rate' LED (green for 1G, orange for 100M). It does that without link. So if you don't see that you will never get a link whatever you do at the 7100 end.
Steve
-
@stephenw10 That's amazing! Which specific module did you use at the 7100 end?
-
In addition to the Metrodata S12W1310S010C15C shown I also tested another ADVA 0061003008:
[22.01-RELEASE][admin@7100.stevew.lan]/root: ifconfig -vvvm ix3 ix3: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500 description: ADVA options=e138bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,WOL_UCAST,WOL_MCAST,WOL_MAGIC,VLAN_HWFILTER,RXCSUM_IPV6,TXCSUM_IPV6> capabilities=f53fbb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,WOL_UCAST,WOL_MCAST,WOL_MAGIC,VLAN_HWFILTER,VLAN_HWTSO,NETMAP,RXCSUM_IPV6,TXCSUM_IPV6> ether 00:08:a2:0e:a5:92 inet6 fe80::208:a2ff:fe0e:a592%ix3 prefixlen 64 scopeid 0xe inet 172.21.16.187 netmask 0xffffff00 broadcast 172.21.16.255 media: Ethernet autoselect (Unknown <rxpause,txpause>) status: active supported media: media autoselect nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL> plugged: SFP/SFP+/SFP28 1000BASE-LX (LC) vendor: ADVA PN: 0061003008 SN: EOPW144300608 DATE: 2014-10-23 module temperature: 31.22 C Voltage: 3.30 Volts RX: 0.38 mW (-4.15 dBm) TX: 0.27 mW (-5.59 dBm) SFF8472 DUMP (0xA0 0..127 range): 03 04 07 00 00 00 02 12 00 01 01 01 0C 00 0A 64 00 00 00 00 41 44 56 41 20 20 20 20 20 20 20 20 20 20 20 20 00 00 80 EA 30 30 36 31 30 30 33 30 30 38 20 00 00 00 00 01 30 32 30 31 05 1E 00 9E 00 1A 64 14 45 4F 50 57 31 34 34 33 30 30 36 30 38 20 20 20 31 34 31 30 32 33 20 20 68 F0 01 BB 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00And an FS SFP1G-LX-31:
[22.01-RELEASE][admin@7100.stevew.lan]/root: ifconfig -vvvm ix3 ix3: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500 description: ADVA options=e138bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,WOL_UCAST,WOL_MCAST,WOL_MAGIC,VLAN_HWFILTER,RXCSUM_IPV6,TXCSUM_IPV6> capabilities=f53fbb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,WOL_UCAST,WOL_MCAST,WOL_MAGIC,VLAN_HWFILTER,VLAN_HWTSO,NETMAP,RXCSUM_IPV6,TXCSUM_IPV6> ether 00:08:a2:0e:a5:92 inet6 fe80::208:a2ff:fe0e:a592%ix3 prefixlen 64 scopeid 0xe inet 172.21.16.187 netmask 0xffffff00 broadcast 172.21.16.255 media: Ethernet autoselect (Unknown <rxpause,txpause>) status: active supported media: media autoselect nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL> plugged: SFP/SFP+/SFP28 1000BASE-LX (LC) vendor: FS PN: SFP1G-LX-31 SN: G2003027213 DATE: 2020-06-16 module temperature: 30.51 C Voltage: 3.31 Volts RX: 0.36 mW (-4.38 dBm) TX: 0.29 mW (-5.33 dBm) SFF8472 DUMP (0xA0 0..127 range): 03 04 07 00 00 00 02 22 00 01 01 01 0D 00 0A 64 00 00 00 00 46 53 20 20 20 20 20 20 20 20 20 20 20 20 20 20 00 00 00 00 53 46 50 31 47 2D 4C 58 2D 33 31 20 20 20 20 20 41 30 20 20 05 1E 00 40 00 1A 00 00 47 32 30 30 33 30 32 37 32 31 33 20 20 20 20 20 32 30 30 36 31 36 20 20 68 F0 01 BD 00 00 08 BB 69 43 0E CF C9 87 4D B5 BA 24 2D CE 49 92 7F 00 00 00 00 00 00 00 00 00 02 F6 BB 95Unfortunately I have no idea what (if any) data that FS module is programmed with but those are cheaply and easily available.
Steve
-
just to put another point of view, we have ordered many hundreds of EAD's from TalkTalk over the last ten years, we are also now using them for 10Gb bearers, clients can order a range of 1Gb, 2Gb, 3Gb, 5Gb and 10Gb bandwidth options :-)
TalkTalk like many others just provide us a L2 wholesale service, being a small ISP using our own AS and Ripe IPv4 ranges, we only supply our own routers with the correct SFP's in the first place
NB all our routers are monitored 24x7 in our North London NOC
the NETWORK SM fibre 1Gb bearer is plugged into an ADVA branded Bi-Dir SFP SC connector with DOM feature (ability to capture diags etc for BT)
the ACCESS port is already loaded with an ADVA SFP MM duplex when Openreach come to install the device, so just an MM LC pigtail is required to connect to the Netgate ( or any other router with SFP ports) , this SFP does not have DOM, so you can use a lot of different SFP's in your router provided it's also a 1Gb MM LC, we use fs.com and others without any fuss or bother. The default is MM, but SM can be requested for the ACCESS port SFP
NB do not use 10Gb SFP+ we have tried a few and also tried 10Gb DAC's, and we have had no success with them, why would you when they are more expensive that 1Gb SFP's and your chances of success are very slim !
when placing an order, the lowest speed that can be ordered on a 1Gb bearer is 100mb, and then incremental speeds up to 1Gb
by default if your SP ( Service Provider) orders an EAD on your behalf that is only 100mb or less, then an ACCESS SFP is not usually configured for the ADVA, and the RJ45 connection is used in its place.
NB orders placed after 2017 where the RJ45 ACCESS was used, can now be upgraded all the way up to 1Gb, negating the need for any SFP
when you have installed hundreds of these in some of the most difficult and hardest to reach places in the UK ( namely wind turbine, solar farms, and other miles into the countryside type of places), the onus is on us, as a small ISP to make sure the EAD works first time every time, and our relationship with TalkTalk enables this, or we would simpy not continue to use them as a supplier - hope that puts over another point of view, you can only really sensibly evaluate a supplier when you have a long term experience with them, rather than just a one off bad experience - which sadly happens in the industry from time to time !
happy to answer any further questions if it helps Netgate users :-)
