Budget 1Gbps hardware in 1U rack format
-
Ok, so you can see how that load is not distributed across the cores. I assume re0 is your WAN and em1 the LAN you were testing from?
Just what budget are you working with?
I agree with VAMike most newer hardware should be fine here as long it's not something with loads of cores but limited single thread performance.
Steve
-
@vamike said in Budget 1Gbps hardware in 1U rack format:
You'll probably find much better options if you give up a pre-built firewall with 6 gbe ports and just pop a quad-port card into a normal 2 port server. Are you sure you need that many ports? I hope you're not trying to use a bunch of NICs as a switch.
Servers are noisy. I already have an intel and a dell. The fans run always at max speed and are not inteded to been used on home. I have my server room on top of my bedroom.... I would like to have a silent server as router.
@stephenw10 said in Budget 1Gbps hardware in 1U rack format:
Ok, so you can see how that load is not distributed across the cores. I assume re0 is your WAN and em1 the LAN you were testing from?
No idea on how to use TOP to read stuff, I always used HTOP.
I have 5 ports used right now on my P55 motherboard 4 intel and 1 realtek for the PPPoE to the ONU.
The LAGG goes to a managed switch with SERVER side vlans, VOIP with QOS priority and HOME network.
Each vlan have it's own subnet. For a total of 4 subnets.My budget goes up to 300USD shipped.
Dummy question is this PPPoE single queue problem only on freebsd? I think to have seen SMP MIPS routers with PPPoE running on multiple cores... using OpenWRT so not the offcial BSP.
-
@hitech95 said in Budget 1Gbps hardware in 1U rack format:
@vamike said in Budget 1Gbps hardware in 1U rack format:
You'll probably find much better options if you give up a pre-built firewall with 6 gbe ports and just pop a quad-port card into a normal 2 port server. Are you sure you need that many ports? I hope you're not trying to use a bunch of NICs as a switch.
Servers are noisy. I already have an intel and a dell. The fans run always at max speed and are not inteded to been used on home. I have my server room on top of my bedroom.... I would like to have a silent server as router.
Well, you decided you needed a rack. :) Racks are noisy. That said, if the fans are actually running at max speed, something is broken; servers will typically have several fan levels and won't go full blast unless one of the fans is dead. (The extra capacity is for redundancy.) There are also different levels of sound for different servers; there's quite a lot of range between "fanless" and "747", and you throw away a lot of performance and options if you insist on fanless. (And if you have other servers in the rack that have fans, that seems like a really bad deal.) In HP's lineup, for example, the bottom end is a DL20, which is fairly tiny and almost silent because its single socket E3 CPU and other components simply don't need as much cooling as, say, the dual socket DL360. In general, an integrated box is going to have cooling proportional to its requirements, and if you look for a lower power system it's going to be quieter than a higher power system. As far as the U/Y/T chips, low-TDP CPUs have basically the same performance characteristics as the higher power CPUs, but they're throttled. If you run a newish CPU at idle the fans are going to be pretty much off because it isn't drawing much power. If you have the CPU under heavy load the fans will speed up. But--if you have a low TDP CPU it just wouldn't be able to do the thing that's causing the higher TDP CPU to max out; you can get the same effect by just killing whatever is maxing out the CPU if you don't need it anyway. (Or you can limit the CPU frequency on the higher TDP CPU.)
I have 5 ports used right now on my P55 motherboard 4 intel and 1 realtek for the PPPoE to the ONU.
The LAGG goes to a managed switch with SERVER side vlans, VOIP with QOS priority and HOME network.
Each vlan have it's own subnet. For a total of 4 subnets.It's unlikely that the LAG is getting you much. You'd probably be better off putting the WAN onto one of the intel ports, then splitting up the VLANs such that the 2 highest traffic links are on their own interfaces and either trunk everything else on the last intel interface or put the lowest traffic vlan on the realtek. (Understanding that freebsd's realtek driver is really, really bad.)
My budget goes up to 300USD shipped.
I'd definitely just keep saving and waiting--nothing you get for $300 is going to be much of an upgrade from what you already have. In the meantime, see about tuning the fan speeds. More modern servers tend to handle everything automatically, older servers may need some kind of program running to set things from the OS or have settings in the bios to override that.
-
It could well be a FreeBSD only issue, or maybe a *BSD issue. I haven't ever investigated.
It's something that has relatively recently become an issue with higher speed connections becoming available using PPPoE.
Just be aware of it.Steve
-
@stephenw10 said in Budget 1Gbps hardware in 1U rack format:
Just be aware of it.
Just tried a workaround found on bsd issue tracker. But looks like it reduce the "speed" and increase ping.
-100Mbps
+5ms.
I didn't have done a proper test just a couple of speedtests.net.isr.dispatch = "deferred"
At least the queue is managed by all cores. I have installed HTOP :)
And now the load is more distributed. -
Yes, that will likely have some drawbacks. It will probably break any traffic shaping you have applied for example which might be why your ping times increase.
Steve
-
maybe you can consider atom 3000 series cpu. like netgate's sg-5100 or xg-7100 with 4-core c3558, or supermicro's c3000 series motherboard.
I am testing a supermicro C3758 barebone which has 8 cores and 4 1gbps-ports with 1u rack mount. The fan speed of this barebone can be lower than default setting(30%) if you can configure it through IPMI console tools.
-
@abcnew said in Budget 1Gbps hardware in 1U rack format:
maybe you can consider atom 3000 series cpu. like netgate's sg-5100 or xg-7100 with 4-core c3558, or supermicro's c3000 series motherboard.
Have you confirmed that hardware can do gigabit PPPoE?
-
Mmm, unfortunately I can only dream of 1G PPPoE here and I'm not sure how realistic a local test would be with PPPoE.
But I would want to run a test to be sure there. I kinda expected it to use more on the i5-750 TBH on a single core.
As another data point there is this: https://forum.netgate.com/topic/133704/poor-performance-on-igb-driver
That user seemed to be limited to 500Mbps over PPPoE with J1900.
I suspect this doesn't scale and there are a load of variables though.Steve
-
@vamike
1gbps PPPoE was an issue before because PPPoE was single threaded for a very long time in FreeBSD. (Use "PPPoE single threaded" as searching keywords.)But MikeFromOz posted 3 months ago that his i5-8400 and intel X550 can do 1gbps pppoe:
https://forum.netgate.com/topic/117313/hardware-to-achieve-gigabit-over-pppoe/5 -
@abcnew um, are you arguing that there's no need to worry about whether a low-power, low-ipc embedded cpu can do gigabit pppoe because it's possible to hit 1gbps with a 4GHz implementation of intel's latest microarchitecture?
-
Mmm, that's not a good example. The i5-8400 has a very high single thread performance. I would expect that to handle 1Gbp on a single queue easily. As the poster says there, they may have over spec'd it.
https://www.cpubenchmark.net/compare/Intel-i5-8400-vs-Intel-i5-750-vs-Intel-Celeron-J1900/3097vs772vs2131A synthetic benchmark like that can only tell you so much but it should provide some idea.
Steve
-
I think that for now I'll keep my hardware (tring to optimize it with downclock) or move to someting else different than pfsense.
Here all connections are pppoe and this is a huge drawback.Even my old mips router with 500Mhz cpu single core can push 300Mbps! And a quadcore 2Ghz+ x86 cpu can't.
-
Well you can push 300Mbps, you can't push 1Gbps over PPPoE with CPU from 2014 designed to be low power. Assuming you're referring to the J1900 there.
How much power is your current setup actually consuming?
It clearly doesn't need a quad core CPU in that application. You might save some power by just swapping out the CPU.
Steve
-
@hitech95 said in Budget 1Gbps hardware in 1U rack format:
I think that for now I'll keep my hardware (tring to optimize it with downclock) or move to someting else different than pfsense.
Here all connections are pppoe and this is a huge drawback.Yes. Ideally you could get a provider that doesn't use pppoe (which is horrible) but if you're stuck you're stuck. You'd probably see better performance with a linux based firewall, but nothing you wrote above suggested anything about a performance problem--you only mentioned a desire for better power efficiency. Also, please see what I wrote about getting rid of the LAG and changing how you use the existing interfaces. It's possible to degrade any system by configuring it sub-optimally.
Even my old mips router with 500Mhz cpu single core can push 300Mbps! And a quadcore 2Ghz+ x86 cpu can't.
Where did this 300Mbps number come from? There are plenty of quad core 2GHz+ x86 devices that can push more than that. (Although saying "2GHz x86" is mostly meaningless because it neglects to specify the implementation. A 2GHz Avoton has very different characteristics than a 2GHz Kaby Lake.) It's also worth pointing out that your mips router likely does pppoe in hardware.
-
@stephenw10 said in Budget 1Gbps hardware in 1U rack format:
You might save some power by just swapping out the CPU.
I wouldn't expect much of an ROI from doing that--the old chipsets were power hogs themselves, so the total idle consumption probably won't drop much.
-
@vamike said in Budget 1Gbps hardware in 1U rack format:
Ideally you could get a provider that doesn't use pppoe
In Italy I have seen only pppoe connections.
but nothing you wrote above suggested anything about a performance problem--you only mentioned a desire for better power efficiency
Right now I don't have problems except somes with Voip. (No idea if is due to the voip provider or pfSense)Where did this 300Mbps number come from?
It was just a reference that looks like that BSD is not using the full potential of the hardware. In comparison small, and embedded not powerfull hardware can do more.
Anyway the according to the sintetic benchmarks the i5 should be fine on handling the connection it have more points on the single thread than my i5-750.
The power cunsumption should be around 150W. The processor is quite old, the motherboard is made for gaming "extreme editions" with lot of fancy stuff. so it have lot of useless stuff for a router sucking power.
I was looking for a more efficient system.
Why should I remove the LAG? It saved my life. I bought switch capable of managing it to have that!
I move lot of stuff on my net and sometimes the router have to manage the data (different subnets) If I hadn't the LAG the internet connection would be slow.Well you can push 300Mbps, you can't push 1Gbps over PPPoE with CPU from 2014 designed to be low power. Assuming you're referring to the J1900 there.
I was talking about i3 or i5 i listed before. The j1900 is quite a low power processor and I know the limitations.
But the fact that I have to buy a Skylake+ CPU for my router is no sense. I'm on Ivy Bringe on my main desktop... and my router should be more powerfull/newer?This is no sense. Something is wrong in the "pipeline".
-
@vamike said in Budget 1Gbps hardware in 1U rack format:
I wouldn't expect much of an ROI from doing that--the old chipsets were power hogs themselves, so the total idle consumption probably won't drop much.
Yeah I agree. Even with the cost of, say, an i5-650 you probably won't see that back in power costs.
Personally I'd probably try it just to see.@hitech95 said in Budget 1Gbps hardware in 1U rack format:
The power consumption should be around 150W
It probably isn't though. I would measure it before changing anything.
Steve
-
@hitech95 said in Budget 1Gbps hardware in 1U rack format:
@vamike said in Budget 1Gbps hardware in 1U rack format:
Ideally you could get a provider that doesn't use pppoe
In Italy I have seen only pppoe connections.
no country has a monopoly on stupidity :)
Where did this 300Mbps number come from?
It was just a reference that looks like that BSD is not using the full potential of the hardware. In comparison small, and embedded not powerfull hardware can do more.
If you offload a large part of the processing then it certainly requires less CPU to do the rest. That said, I think you're engaging in a bit of hyperbole, because it's certainly possible to find a lot of consumer routers and other small embedded chips that choke on gigabit pppoe. If you get a router along with a gigabit pppoe connection, you got something that was specifically chosen for its pppoe performance.
Why should I remove the LAG? It saved my life. I bought switch capable of managing it to have that!
I move lot of stuff on my net and sometimes the router have to manage the data (different subnets) If I hadn't the LAG the internet connection would be slow.Honestly, I have no idea what you're saying here. If you want to have different subnets you can either trunk over a vlan or use different interfaces as I suggested, link aggregation has nothing to do with it. Due to the way link aggregation works, if you have a small number of devices it's extremely unlikely that you're getting balanced high utilization across the interfaces. Much more likely, you've got high utilization on one or two interfaces, and the others are idle. Especially since your wan is currently a realtek nic (which has terrible fbsd drivers) it's almost certain you'll get more bang for the buck by moving the lan to one of the intel interfaces than hoping that the traffic spreads nicely across four interfaces internally. If your high-utilization hosts happen to hash unluckily, the LAG will actually provide less bandwidth than splitting the subnets onto separate interfaces. (In the worst case, a pair of connection endpoints will be sharing a single interface rather than being forced onto two.) In general, a lot of people think that link aggregation will provide much more benefit than it actually does. There's no way link aggregation is speeding up the WAN connection, which is capped at 1gbps.
I was talking about i3 or i5 i listed before. The j1900 is quite a low power processor and I know the limitations.
But the fact that I have to buy a Skylake+ CPU for my router is no sense. I'm on Ivy Bringe on my main desktop... and my router should be more powerfull/newer?This is no sense. Something is wrong in the "pipeline".
There's certainly confusion here. You haven't reported a performance problem, so recommendations for new CPUs are based mostly on your request for something more efficient, and somewhat with an eye toward what you might want to implement on the connection you say you have. The newer CPUs are very, very power efficient at idle, and that's why I suggest looking at them rather than a six year old ivy bridge, given your stated requirement. If you want to add things with greater CPU requirements, like VPN, they have plenty of capacity for that also. The 3855U could probably handle things at your current level of activity, but you won't have a lot of excess capacity. If you try to do more with it and run into the limits, you've basically wasted your investment if you end up needing to upgrade. And (this is important) the U series part costs more than the entry level CPUs I was talking about--you pay extra for the throttling. The only reason to buy something in the U series is if you have a really compelling reason not to exceed a certain TDP at max load (for example, if you put the CPU in your lap and don't want to get too toasty). Again, you haven't said that you have a performance issue now, so I don't understand why you say you "need" to upgrade to skylake--you could just leave everything alone.
-
Hi,
I currently have lot of traffic between subnets. If I have full gbit traffic the two link get saturated. This is why I have the LAG. The lag is now running on the internet NICs. The realtek is used only for WAN to ONU.In Italy we have 5 ISPs and all uses PPPoE. (Only one uses IP but only on VDSL)
About the Skylake, as you have said only newer generations have low idle power. And the prices is too hight for my budget.
At this point I'll wait for some newer solutions later.
A mobile grade CPU would be the greatest upgrade for me. The rack mount is under the roof and ventilation is low...