SFP+ multi-gig copper twisted pair transceiver for Supermicro X11SDV-4C-TP8
-
@patient0 impressive! I would be tempted to get the 25Gbps service, but I doubt there is a small form factor motherboard suitable for pFsense router duty that supports more than 10Gbps SFP+. So probably a huge expense for appropriate hardware, not to mention power consumption. So probably doesn't make sense, as you said.
-
The FS 1G/2.5G/5G/10Gbps NBASE-T SFP+ USXGMII Copper 30m RJ-45 Transceiver Module arrived. I powered down the Supermicro/pFsense router, plugged it in, and configured pFsense to use ixl3 (2nd SFP+ 1/10Gb SFP+ cage) for the WAN port. There was no internet access, nor did the LEDs on for the 2nd SFP+ port light up like they do for the 1st SFP+ port for the 10Gb direct-attach cable for the LAN. The new FS 1/2.5/5/10Gb transceiver was powered and warm, but apparently is not compatible with the older 1\10Gb motherboard.
So I guess I need to get a converter like the FS Mini Unmanaged 1x 100M/1G/2.5G/5G/10GBase-T RJ45 to 1x 10GBase-X SFP+ Slot 10Gigabit Ethernet Media Converter. The question is, will I find myself in the same situation where the newer multi-gig protocol is incompatible with the older 1/10Gb protocol? The labeling of the converter does imply it is converting internally to 1/10Gb SFP+. If so, I can just use a direct-attach cable, so the cost is reasonable as no SFP+ transceivers need to be purchased.
I decided against link aggregation, I don't want to introduce any more complexity into pFsense. As it is, I am barely clever enough to make it work.
-
@lifespeed said in SFP+ multi-gig copper twisted pair transceiver for Supermicro X11SDV-4C-TP8:
The FS 1G/2.5G/5G/10Gbps NBASE-T SFP+ USXGMII Copper 30m RJ-45 Transceiver Module arrived. I powered down the Supermicro/pFsense router, plugged it in, and configured pFsense to use ixl3 (2nd SFP+ 1/10Gb SFP+ cage) for the WAN port. There was no internet access, nor did the LEDs on for the 2nd SFP+ port light up like they do for the 1st SFP+ port for the 10Gb direct-attach cable for the LAN. The new FS 1/2.5.5.10Gb transceiver was powered and warm, but apparently is not compatible with the older 1\10Gb motherboard.
So I guess I need to get a converter like the FS Mini Unmanaged 1x 100M/1G/2.5G/5G/10GBase-T RJ45 to 1x 10GBase-X SFP+ Slot 10Gigabit Ethernet Media Converter. The question is, will I find myself in the same situation where the newer multi-gig protocol is incompatible with the older 1/10Gb protocol? The labeling of the converter does imply it is converting internally to 1/10Gb SFP+. If so, I can just use a direct-attach cable, so the cost is reasonable as no SFP+ transceivers need to be purchased.
I decided against link aggregation, I don't want to introduce any more complexity into pFsense. As it is, I am barely clever enough to make it work.
I think what you propose should work, but I would probably doublecheck with FS support to be certain before spending that kind of money. Apologies for not being able to provide a more direct answer - while I am currently using one of these media converters, it is on a symmetric 10gig/10gig link (10gig SFP+ to 10gig RJ-45), and for that it has worked great.
-
@lifespeed I would try a few more things with the FS transceiver module. Maybe just insert the module without changing WAN to it for the moment.
Was there anything related to this module in the logs, the reason it didn't accept it?
What is the output of
ifconfig -vvvmm ixl3and what doessysctl dev.ixl.3.supported_speedsshow? -
@patient0 good suggestion, although I suspect the final answer may remain the same.
ifconfig -vvvmm ixl3
sysctl dev.ixl.3.supported_speeds
ixl2 is the SFP+ DAC cable from pFsense to my 10Gb switch. ixl3 is the FS 1/2/5/5/10Gb transceiver. These two lines from the boot log appear to confirm it isn't going to work:
ixl3: ixl_set_link: Error getting phy capabilities -7, aq error: 5 ixl3: ixl_attach_get_link_status get link status, status: I40E_ERR_UNKNOWN_PHY aq_err=I40E_AQ_RC_EIOHere is the full boot log referencing ixl3.
ixl3: netmap queues/slots: TX 4/1024, RX 4/1024 ixl3: SR-IOV ready ixl3: ixl_set_link: Error getting phy capabilities -7, aq error: 5 ixl3: ixl_attach_get_link_status get link status, status: I40E_ERR_UNKNOWN_PHY aq_err=I40E_AQ_RC_EIO ixl3: Allocating 4 queues for PF LAN VSI; 4 queues active ixl3: Ethernet address: 00:25:90:bb:bf:0d ixl3: Using MSI-X interrupts with 5 vectors ixl3: Using 4 RX queues 4 TX queues ixl3: Using 1024 TX descriptors and 1024 RX descriptors ixl3: PF-ID[3]: VFs 32, MSI-X 129, VF MSI-X 5, QPs 384, I2C ixl3: fw 3.1.57069 api 1.5 nvm 3.33 etid 80001006 oem 1.262.0 ixl3: <Intel(R) Ethernet Connection X722 for 10GbE SFP+ - 2.3.3-k> mem 0xf7000000-0xf7ffffff,0xfb000000-0xfb007fff irq 46 at device 0.3 numa-domain 0 on pci12 ixl2: netmap queues/slots: TX 4/1024, RX 4/1024 ixl2: SR-IOV ready ixl2: Allocating 4 queues for PF LAN VSI; 4 queues active ixl2: Ethernet address: 00:25:90:bb:bf:0c ixl2: Using MSI-X interrupts with 5 vectors ixl2: Using 4 RX queues 4 TX queues ixl2: Using 1024 TX descriptors and 1024 RX descriptors ixl2: PF-ID[2]: VFs 32, MSI-X 129, VF MSI-X 5, QPs 384, I2C ixl2: fw 3.1.57069 api 1.5 nvm 3.33 etid 80001006 oem 1.262.0 ixl2: <Intel(R) Ethernet Connection X722 for 10GbE SFP+ - 2.3.3-k> mem 0xf8000000-0xf8ffffff,0xfb008000-0xfb00ffff irq 46 at device 0.2 numa-domain 0 on pci12 -
@lifespeed that "Error getting phy capabilities" indicate a driver issue or a hardware issue? Would you know that?
pfSense does recognise the transceiver correctly, the speed too (6 == 1G / 10G).
Would be interesting to boot up with a Ubuntu Live CD to see if it's a driver or hardware issue. But since it's your router that's probably not very practical.
-
@patient0 I would think I could boot using a live USB stick as this is a small form factor PC with accessible USB and display ports. However, I'm not a linux expert and don't know how to proceed as to identifying a driver vs hardware issue.
The transceiver is 1/2.5/5/10Gb, not 1/10Gb, so I'm not sure it is being recognized correctly.
Is pFsense capable of using a 1/2.5/5/10Gb interface?
-
@lifespeed said in SFP+ multi-gig copper twisted pair transceiver for Supermicro X11SDV-4C-TP8:
The transceiver is 1/2.5/5/10Gb, not 1/10Gb, so I'm not sure it is being recognized correctly.
I would have assumed that the host links at 10GB with the transceiver and the transceiver negotiates with the client between 1/2.5/5/10Gbit (but the host<->transceiver stays at 10G).
Maybe we can be so bold and involve @stephenw10?
-
@patient0 The link speed must proceed as negotiated. This means a 2.5Gb link can't transceive at the physical layer at a different speed, nor can a 10Gb link. In order for a 2.5Gb link to communicate with a 10Gb link, it must buffer enough to fill a frame, then transmit at the 10Gb physical layer speed.
I'm almost certain buffering is not part of Ethernet chip functionality, and must happen externally. I don't think this functionality is included in an SFP+ transceiver, either. Probably that is the difference between a "media converter" and an SFP+ transceiver.
-
Having confirmed the link speed translation capability with FS I have ordered the media converter and will report back with results, hopefully a successful WAN link at 2.5Gb.
-
The media converter from FS finally arrived. It does indeed allow the WAN to connect from the Netgear CM3000 multi-gig 2.5GBase-T port to the pFsense Supermicro SFP+ WAN port on ixl3.
However, the external IP4 address from my ISP changed (not surprising), and the automatic dynamic DNS update no longer appears to function. This is the pFsense functionality using the GoDaddy API key, something that previously worked. I see an error message in the logs that seem to complain about the dyndns service authentication.
Edit: Supposedly in May 2024 GoDaddy deliberately broke (disabled) the dynamic DNS API
Dec 27 15:15:50 php-fpm 400 /services_dyndns_edit.php: phpDynDNS (@): (Error) Authenticated user is not allowed access.As a point of interest and comparison with the unsuccessful SFP+ multi-gig transceiver inserted in the 1G/10G cage, below are the command line outputs:
ifconfig -vvvmm ixl3
sysctl dev.ixl.3.supported_speeds
-
That looks more like the user trying to force the update doesn't have access. Were you logged in as root/admin?
-
@stephenw10 said in SFP+ multi-gig copper twisted pair transceiver for Supermicro X11SDV-4C-TP8:
That looks more like the user trying to force the update doesn't have access. Were you logged in as root/admin?
Logged into what, pFsense? I'm just viewing the logs using the web interface logged in as "admin" a user in the administrators group in pFsense. The error message is associated with pFsense trying to update the dynamic DNS, I don't think that has anything to do with using the web interface to view or configure pFsense.
I don't understand what you mean by "logged in". pFsense is running continuously, I don't "log in" to it to cause it to perform it's functions.
-
@lifespeed said in SFP+ multi-gig copper twisted pair transceiver for Supermicro X11SDV-4C-TP8:
/services_dyndns_edit.php
Right. It's just that that script is the page the gui accesses when you edit a DynDNS entry. Not what I'd expect to see when the dyndns update is run automatically.
-
@stephenw10 said in SFP+ multi-gig copper twisted pair transceiver for Supermicro X11SDV-4C-TP8:
@lifespeed said in SFP+ multi-gig copper twisted pair transceiver for Supermicro X11SDV-4C-TP8:
/services_dyndns_edit.php
Right. It's just that that script is the page the gui accesses when you edit a DynDNS entry. Not what I'd expect to see when the dyndns update is run automatically.
There was a similar entry in the log mentioning the same lack of authorization, but it appeared to have been run automatically. Yes, I did force a manual dynamic IP check and update, but the automatic updates weren't working either.
Still haven't heard back from GoDaddy regarding them breaking this feature. Really poor behavior on their part.
-
Ah, well it could be an error coming back from the server side then. I guess they have changed the allowed methods. Assuming your login is still valid.
