Questions about 10 gbps nics
-
10G: we ship Chelsio T5 on the c2758.
We may pick up the Intel x710.Everything else is crap.
We've enabled AES-NI/AES-GCM, more work to be done. Linux does 840Gb/s IPSec on c2758 platform. We do less, investigating.
We will enable QAT on this platform and faster THIS YEAR. C2758 should be good for 8Gb/s IPSec on c2758 with QAT.
We have hw coming this year that will do 6 x 10G with IPSec @ 60Gb/s with headroom.
Yes.I.Said.This.Year.
Many in this thread have zero clue. Half-duplex 10G? WTF, over?
-
OK and I see all of those require an 8-lane pcie-v3 slot. That's really what I needed to know. It would have been nice to get at least a single-lane 10gbE port into a switch or something when the time came, but I guess it is what it is.
Thanks.
-
@gonzopancho:
10G: we ship Chelsio T5 on the c2758.
We may pick up the Intel x710Many in this thread have zero clue. Half-duplex 10G? WTF, over?
I admit I made a mistake, the figures I quoted were based on the assumption that PCI-e was serial, it isn't. Would have been nice to have it corrected rather than being told I have no clue.. But whatever..
Per lane, PCI-e, in each direction (full duplex):
v1.x: 250 MB/s (2.5 GT/s)
v2.x: 500 MB/s (5 GT/s)
v3.0: 985 MB/s (8 GT/s)
v4.0: 1969 MB/s (16 GT/s) -
PCIe is serial it's just not over single communication medium like, say, 10base2 Ethernet.
I think Jim was pointing out that 10Gbit Ethernet is not half duplex unless presumably you've wired it very very wrong. ;) A simple misunderstanding.
In fact my earlier post was incorrect. I said 4 lanes of PCIe 2.0 would give you 16Gbps total but in fact that's in both directions. So a 4X slot could saturate a 10Gb Ethernet link in theory if nothing else throttles the data.Steve
-
OK for the record I'm pretty familiar with gigabit and lower ethernet, but not at certified network admin level. I've been "the guy" for years because I've typically worked at small companies and had an interest, while nobody else does.
10 gigabit is a whole different ballgame and that's why I started the thread. Everyone makes mistakes and I'm not holding any grudges or making judgements.
I have three use cases for 10gbps and it will be awhile before I implement any of them. They are:
-
A small number of hosts (probably 3) with 2-port 10gbps nics with direct interconnect.
-
The same number of hosts (maybe +1 in the case below) connected directly to a managed or smart switch which can handle routing and some sort of security directly.
-
My new but as yet unconfigured router which is a SuperMicro c2758 board with a single-port 10gbps nic to hook into the above switch.
All of the main systems will be VM hosts. Probably the router will be too, although the plan is to install that in several different ways to evaluate what Atom c2000 systems can do for other aspects of my network. So the NICs need to aware of virtualization optimizations.
I can see that the Chelsio nics would work for any of the three main systems.
For the router, if the 10gbps switch can handle VLANs and some fairly simple firewall rules between them, all I would need is to allow near-wire-speed gigabit VLAN traffic to hit the servers without the server-side nic or my router as the bottleneck.
I can see right off the bat that the board I have can't route at high speed between two NICs at 10gbps with the 4-lane pciev2 slot it has, and having it route the high speed traffic through a single port NIC is not reasonable. So really I'm just worried about high speed VPN performance plus routing with the 7 gigabit nics and a possible 10gbps nic.
So I'm still looking for a possible single-port NIC that can work with a 4-lane pciev2 slot which is good enough to do the job.
Thanks.
-
-
Interesting topic…. But what is QAT or where does it stand for?
-
Quick Assist Technology:
http://www.intel.com/content/www/us/en/io/quickassist-technology/quickassist-technology-developer.html -
I'm just a n00b but IMO if you're doing any sort of VPN without QAT hardware you're probably doing it wrong.
The software doesn't support it yet but it will, I'm guessing soon.
-
Ha! Well it depends if you need the throughput. I have an OpenVPN server running here at home to use for remote access and my hardware is way too old to support Quickassist. It's still fast enough to stream Dr Who to America though so that's fine (if you ask my sister!). Fast enough to secure my traffic when I'm using public wifi also.
Steve
-
I'm just a n00b but IMO if you're doing any sort of VPN without QAT hardware you're probably doing it wrong.
The software doesn't support it yet but it will, I'm guessing soon.
Aes-ni is more than enough for a good proportion of vpn use..
-
When did girls start watching Dr. Who?!!? I've never heard of such a thing.
Technically I don't "need" acceleration, but if you're buying hardware in anticipation of gigabit Internet and want a VPN which can even come close to that speed, you're going to need at least AES-NI.
I'm a bit too suspicious to put all my eggs in that one basket for encryption acceleration though, which is why I'm so excited about QAT. I also have a significant need for compression acceleration.
-
When did girls start watching Dr. Who?!!? I've never heard of such a thing.
When they started giving the role to actors like David Tennant and Matt Smith. ::)
Steve
-
PCIe is serial is just not over single communication medium like, say, 10base2 Ethernet.
I think Jim was pointing out that 10Gbit Ethernet is not half duplex unless presumably you've wired it very very wrong. ;) A simple misunderstanding.
In fact my earlier post was incorrect. I said 4 lanes of PCIe 2.0 would give you 16Gbps total but in fact that's in both directions. So a 4X slot could saturate a 10Gb Ethernet link in theory if nothing else throttles the data.Steve
I could be wrong, but I thought half duplex only worked with 10BaseT and 100BaseT networks. As soon as we got to 1000BaseT, if the connection isn't running in full duplex it, it isn't functioning at all.
Regardless, I find this thread to be a very interesting read.
-
Forget half duplex, what I was getting at was that you won't see the full bandwidth if you don't have the bandwidth over a PCI-e slot
-
I'm just a n00b but IMO if you're doing any sort of VPN without QAT hardware you're probably doing it wrong.
The software doesn't support it yet but it will, I'm guessing soon.
http://www.dumpaday.com/?attachment_id=58505
-
I'm just a n00b but IMO if you're doing any sort of VPN without QAT hardware you're probably doing it wrong.
The software doesn't support it yet but it will, I'm guessing soon.
Aes-ni is more than enough for a good proportion of vpn use..
Probably, and it's the best you can get right now, so…
-
PCIe is serial is just not over single communication medium like, say, 10base2 Ethernet.
I think Jim was pointing out that 10Gbit Ethernet is not half duplex unless presumably you've wired it very very wrong. ;) A simple misunderstanding.
In fact my earlier post was incorrect. I said 4 lanes of PCIe 2.0 would give you 16Gbps total but in fact that's in both directions. So a 4X slot could saturate a 10Gb Ethernet link in theory if nothing else throttles the data.Steve
I could be wrong, but I thought half duplex only worked with 10BaseT and 100BaseT networks. As soon as we got to 1000BaseT, if the connection isn't running in full duplex it, it isn't functioning at all.
Regardless, I find this thread to be a very interesting read.
Half-duplex gigabit links connected through hubs are allowed by the specification(*), but the relevant sections of the specification is not updated anymore and full-duplex is used exclusively with switches.
(*) A single repeater per collision domain is defined in IEEE 802.3 2008/2012 Section 3:41
-
I have an OpenVPN server running here at home to use for remote access and my hardware is way too old to support Quickassist.
I suspect this will change soon enough.
-
Hello kroberts,
perhaps some informations interesting for you?
New Boards with build in Dual 10 GbE or SFP+Do you know HotLave?
They are producing 1 GB, 10 GB and 40 GB Intel based NICs!One tip of me by side to you, build with the D-1500 based boards a pfSense based
firewall and with the Xeon E3 a NAS or Server, but please don´t connect the
pfSense based firewall direct over 10 GBit/s this will be not the best effort for the
throughput, you will be better going with a Infinion SX2 card, that can be connected
directly from the pfSense based firewall to the NAS and it will be serving more speed
and throughout as the 10 GBit/s SFP+ option as I see it right. -
@BlueKobold:
Hello kroberts,
perhaps some informations interesting for you?
New Boards with build in Dual 10 GbE or SFP+Do you know HotLave?
They are producing 1 GB, 10 GB and 40 GB Intel based NICs!One tip of me by side to you, build with the D-1500 based boards a pfSense based
firewall and with the Xeon E3 a NAS or Server, but please don´t connect the
pfSense based firewall direct over 10 GBit/s this will be not the best effort for the
throughput, you will be better going with a Infinion SX2 card, that can be connected
directly from the pfSense based firewall to the NAS and it will be serving more speed
and throughout as the 10 GBit/s SFP+ option as I see it right.We'll likely be moving to Xeon-D (Supermicro at first, something better to follow).
All the HotLava 10Gbps NICs appear to be based on Intel 82599ES. These work, but don't work as well as Fortville (Intel) or T5 (Chelsio).
