2.5Gbit Ethernet on a budget - so far so good!

mark_lab_user

This is an update to another posting I made from a short time ago...

Installed the latest officially "unsupported" driver for the very high performance REALTEK 8025B based NIC cards. I say high performance because these cards are now determined to be rock solid and very cheap in my opinion. Less than $10 USD per nic from Ali-Express. Ask me how I stressed them?
Also just installed a VIMIN 2.5G Switch so that I can upgrade other home lab machines to 2.5 Gbit. It was $75 USD. Switch is 4 Ethernet ports and 2 SFP ports (not SFP+).
Lots of contention out there among the community regarding 2.5 vs. 10 Gbit. Seems 10Gbit over copper Ethernet will work for short distances but still uses insane amount of power and creates lots of heat. Heat means short life time in my books. Now 10Gbit with SFP + would be the exact opposite. Less noise resulting in very low power consumption for the transciever. With power consumption so high, Cat 5 or 6 or 6a cables are not a factor. But killowatt hours is!
My ISP's won't be delivering 10Gbits to me in the near future. They are still struggling to deliver sub 2Gbit symmetric. The spent a year ditch-witching the streets in my town.
Lesson learned here is that for every one subscriber in my area there are 99 who just want to complain about restoration of their driveway or lawn where the directional boring occurred.
Ask me why even boring seems to be disruptive to so many?
So I am ready for 1.5 Gbit symmetric WAN coming to my home lab later this week. The ISP is using passive optical. It has a theoretical limit of 2.66 due to the fibre infrastructure but maybe link aggregation will be in my future because they put enough fibre to the curb to future proof things for maybe 10 years at least. One cool thing is that the light comes from the head end and the old school fibre and/or coax needs electricity every few miles and that causes downtime.
So full steam ahead with pfSense on a 10 year old 8 core ASROCK Z87 Pro 3 Mobo. I am seeing 2.2 gbits (184k datagrams per second) on UDP with 0.002ms jitter and no packet loss

Works for me!

NetGate should build a 2.5GBE embedded router for the SME market !

johnpoz

@mark_lab_user said in 2.5Gbit Ethernet on a budget - so far so good!:

NetGate should build a 2.5GBE embedded router for the SME market !

They do.. The 4100 has

(4) 2.5 Gbps RJ-45 "direct" (unswitched) ethernet LAN ports

mark_lab_user

@johnpoz Hey thanks for that. That 4100 looks really nice. Tha C3338R Atom has some NIC's built-in to the Chip if I recall correctly, this bypassing the need for PCIe ?

johnpoz

@mark_lab_user and those for sure could be used as wan ports as well.. I think all the new models have 2.5ge on them..

mark_lab_user

@johnpoz That 4100 would cost approx $1000 CDN landed here due to exchange rate vs USD and some shipping and then 13% VAT. But the bonus would be 1 year of tech support!
Seems like the perfect anchor for anyone trying to get multi-gigabit out of their new fangled WIFI routers. With the 2.5Gbit headroom maybe some folks would actually see 1Gbit plus speeds out of their WIFI 6, 6e and 7 toys. That's if they can find a WIFI router with 2.5 gbit WAN.

mark_lab_user

@mark_lab_user Amazon sells the 4100 in Canada for 850 $CDN which equates to 960 $CDN . So I was mistaken.

Austin 0

@mark_lab_user It should be noted that netgate rates the box at only 1.4 Gb/s when doing imix tests with and with the firewall enabled. So depending on your use case it might not actually do 2.5 Gb/s.

johnpoz

@Austin-0 imix is never going to show full possible bandwidth of the interface.. By the very nature of that sort of traffic you would never see full wire speed.

Same goes for a gig interface - your not going to see full gig with imix either..

mark_lab_user

@Austin-0 Okay, that is good info. I will take that at face value. And my comment about 960 $CDN is after the VAT but I might guess that there is no 1 year support because it is Amazon!

Austin 0

@johnpoz Maybe this would be better handled on a different thread, but could you briefly explain why that it is? I thought that imix was just a with many more sessions than iperf, and thus just introduced more CPU overhead.

mark_lab_user

@Austin-0 Sure, let's switch threads. I am not familiar with imix.

Bottom line here for me is that I just wanted to post my very succesfull results with "pfSense Plus 23,05-RELEASE" in combination with some very budget-wise Realtek NICs.

I will update this thread when the WAN comes up at 1.5 Gbits over passive optical in a few days.

johnpoz

@Austin-0 said in 2.5Gbit Ethernet on a budget - so far so good!:

briefly explain why that it is?

Because imix is going to be mixed up sized frames, some would be full, other would be really small, etc. I have never heard of imix ever showing full possible bandwidth on any nic.. Do your imix test via just a switch to a server, etc. There are some tools to create imix sort of traffic.

If you want to know if you "can" see your full isp speed, or full wire speed across interfaces from vlan X to Y over your router.. You would load it up with full frames. Like you were doing a file transfer of say a 10GB file. etc..

The iperf test, which is full sized transfer would be better indication if the box is able to route at full wire speed.. I believe @stephenw10 somewhat recently in some other thread mentioned how they test the imix..

RobbieTT

@johnpoz said in 2.5Gbit Ethernet on a budget - so far so good!:

Because imix is going to be mixed up sized frames, some would be full, other would be really small, etc.

If you want to know if you "can" see your full isp speed, or full wire speed across interfaces from vlan X to Y over your router.. You would load it up with full frames. Like you were doing a file transfer of say a 10GB file. etc..

That is a bit of apples vs oranges. For clarification, when we reference 'wire-speed' it is implicit that this is at the smallest frame size for the data protocol.

The value we pay attention to for wire-speed is the packets per second (pps). For 1 Gbps this equates to a minimum 1.488 Mpps.

If a link requires larger frames than the minimum to achieve its maximum data speed then it is not capable of wire-speed.

️

johnpoz

@RobbieTT yes pps is a very important number true.. What I meant by wire speed is you can see gig (or what is expected on gig) you would never actually see gig, etc. Or 2.5 or 5 or 10, etc.

Your not going to see 2.5Mbps with imix was my point..

Thanks for the clarification..