Hardware requirement for Intel home router

Hikari · Sep 28, 2020, 9:43 PM

Hello everybody.

I currently have an EdgeRouter X with OpenWRT, I'm using it to manage my LAN with 4 PC, 2 NAS, 1 AP, 1 smartphone and often a laptop. I have it on Dual WAN for 2 ISPs, one offering IPv4 and IPv6 using DHCP and the other using PPPoE over GPON.

I'm willing to move to pfSense because its Dual WAN isn't working properly, when 1 ISP goes down my whole access falls and it takes many seconds to come back, and its load balancing isn't 50%-50% no matter how I configure it.

I wanna improve some monitoring tools I started developing on it but stopped due to some BusyBox limitations like its ping being limited and fails to work for IPv6 on PPPoE. I was also struggling to do it on Bash and I never take the time to do learn Python, maybe I could do some stuff with Java (yeah, I know, don't laugh).

I'm also wanting to move my AP to a dedicated port and put it on another VLAN, my router has no spare ports.

ATM, OpenWRT is using 30% of its 256MB RAM and only 4% of 16384 active connections. It's rly much over what it's required.

For pfSense, I'm considering buying some micro-ATX H410 motherboard. I guess 2x8GB RAM will be enough for it.

For CPU, is a Pentium enough, or should I go for i3? Pentium is 48W TDP so it's very interesting, but I fear it not being enough. i3 is 65W TDP and i5's price is very close, so if Pentium isn't enough I'll go for i5.

For storage I'm considering a 400GB SSD.

I have an Ubuntu server where I run stuff like private proxy, Tor middle relay, torrent, etc, so my router should only do routing stuff.

What do you think? Any suggestion will be greatly appreciated.

SteveITS · Sep 28, 2020, 10:16 PM

"It depends"...see https://www.netgate.com/blog/choosing-the-right-netgate-appliance.html and compare to https://www.netgate.com/products/appliances/. Netgate uses Atom chips for their mid-range appliances. You may wish to ensure it has AES-NI: https://www.netgate.com/blog/more-on-aes-ni.html.

Hikari · Sep 29, 2020, 2:13 PM

tnx!

I verified and Pentium G6500 does has AES-NI support.

Indeed, a XG-7100 would be way over what I need and it uses a simple Atom, 8GB RAM and 256GB SSD.

I looked on partners and found 3, gonna ask them for their prices to compare with custom hardware.

Sorry for noob question, but is there any difference on features or performance from buying a Netgate appliance and buying custom hardware and installing it? Can it be customized and software be installed as it can on custom hardware?

stephenw10 · Sep 29, 2020, 10:34 AM

There is no artificial restriction, the throughput is only limited by the hardware.

What are the bandwidth of your two WANs?

That spec system is probably massive overkill if you are going to be using it purely as a router/firewall.

Steve

SteveITS · Sep 29, 2020, 2:13 PM

@Hikari said in Hardware requirement for Intel home router:

is there any difference on features or performance from buying a Netgate appliance and buying custom hardware and installing it?

Again it kind of depends. Obviously they've tested pfSense on their own hardware. For instance I've seen several recommendations here to avoid Realtek cards due to the FreeBSD drivers. It used to be the auto/online configuration backup was limited to Netgate hardware but they opened that up.

We had looked into building an appliance and for us, where the labor cost isn't zero, it's a better deal to just go with one of theirs. It's preinstalled, tested, etc. Plus it supports the project.

I have in a pinch installed pfSense on an old PC with 2 NICs and it's fine. The CPU is not generally taxed unless you add on extra packages like Suricata or Snort, have a high bandwidth VPN, etc. And even then those run fine on their appliances.

Hikari · Sep 29, 2020, 2:40 PM

tnx again

Yeah of course I mean hardware with FreeBSD support. I mean about features, if there's any feature on Netgate's appliance that's not available when installing pfSense on our hardware, or if its appliance is limited somehow that custom hardware isn't. In example, limited to install pfSense packages or FreeBSD software or compile Unix software from source.

I ask that because all my BusyBox based devices (router and 2 NAS) don't support compiling. On my NAS I'm able to develop on Java and run my jar from their JVM, but on my router that's not possible because it doesn't have JVM.

I guess that "cheaper" Netgate appliances with 2GB RAM won't support Java either.

I'm reading XG-7100 spec, it says that its 8 GbE ports are controlled by a Marvell 88E6190, which is connected to CPU by a 2.5Gbps link. That seems to be 3GIO 1.0 x1 speed.

So, is it still using 1.0, while current Intel hardware now uses 3.0?

That's very sad. When I was looking for hardware around, I saw that most NICs with 4 GbE ports are still on 2.0 and use x4 links. That makes building it more expensive, because for 8 ports I'll need 2 x16 connectors, and for that I'll need to buy either an expensive micro-ATX board or a mid ATX board which are same price.

I had found 2 NIC models that have 4 GbE ports and use x1, but one is a generic china NIC that uses a 10 years old Realtek chip and still uses 3GIO 1.1, and the other seems to be also a generic china board without any info found by Google.

Maybe I'll just stay on my OpenWRT for a couple more years and wait for Intel to release a new NIC and hopefully Netgate also release new appliances with this hardware :/

stephenw10 · Sep 29, 2020, 5:15 PM

pfSense does not include any build tools, that's the same in Factory or CE.
It's possible to install packages from the FreeBSD repos but not recommended as they can pull in things and overwrite something that is custom in pfSense:
https://docs.netgate.com/pfsense/en/latest/recipes/freebsd-pkg-repo.html

There are a few packages that are only available in Factory: The AWS VPC VPN Connection Wizard, the IPsec Profile Generation wizard. You can do the things they automate in CE anyway the wizards just make it easier.

The internal links in the XG-7100 are 2.5Gbase-KX not PCIe.

[2.4.5-RELEASE][admin@7100.stevew.lan]/root: ifconfig ix2
ix2: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
	options=e400bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6>
	ether 00:08:a2:0e:a5:93
	hwaddr 00:08:a2:0e:a5:93
	inet 192.168.88.1 netmask 0xffffff00 broadcast 192.168.88.255
	inet6 fe80::1:1%ix2 prefixlen 64 scopeid 0x5
	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
	media: Ethernet autoselect (2500Base-KX <full-duplex,rxpause,txpause>)
	status: active

Steve

Hikari · Sep 29, 2020, 6:04 PM

@stephenw10 said in Hardware requirement for Intel home router:

The internal links in the XG-7100 are 2.5Gbase-KX not PCIe.

I don't understand it. As I could verify at http://www.smart-dv.com/vip/eth_2_5g_5g.html, 2.5Gbase-KX seems to be related with Ethernet and IP verification, it doesn't seem to be a bus between chips.

Hikari · Sep 29, 2020, 7:00 PM

@stephenw10 said in Hardware requirement for Intel home router:

pfSense does not include any build tools, that's the same in Factory or CE.
It's possible to install packages from the FreeBSD repos but not recommended as they can pull in things and overwrite something that is custom in pfSense:
https://docs.netgate.com/pfsense/en/latest/recipes/freebsd-pkg-repo.html

Interesting points on that article. It's sad that pfSense doesn't have build tools, but totally understandable.

I'm pretty sure that all features available on OpenWRT are also on pfSense. I just don't know what's on pfSense to replace Yamon, it's a third party tool that logs traffic from all LAN devices and reports how much download and upload per day and higher periods each device has been consuming.

I also guess that pfSense has better support for Multi WAN than OpenWRT and for monitoring and load balancing if any link goes down. As I said, I'm unable to monitor IPv6 on one of my ISPs because I can't manage to make BusyBox's ping work on the virtual interface that OpenWRT creates for IPv6 over PPPoE. Because of that, all IPv6 traffic is using only 1 of them.

Apart from that, I guess pfSense has JVM (not jdk) and Python support. If any script or jar I deploy breaks something, I'm to blame :) That's also why I was guessing if 16GB RAM and Pentium is enough.

Also, I have my Ubuntu server where I have some services running and I do keep the router as clean as possible. Only things I run on it are monitors that rely on choosing which network interface to use, and I struggled to continue developing because I'm not good on Bash and know nothing about Python :-x

SteveITS · Sep 29, 2020, 7:25 PM

I'm not familiar with OpenWRT or Yamon, but if you're looking for traffic monitoring we've used bandwidthd. It's listed at https://docs.netgate.com/pfsense/en/latest/monitoring/graphs/bandwidth-usage.html but not https://docs.netgate.com/pfsense/en/latest/packages/list.html. There are others as well.

Bandwidthd has a formatting (iframe size) problem with pfSense 2.4.5 but other than that it seems fine for our needs. Click the "click to remove frame" link to get around that for now.

stephenw10 · Sep 29, 2020, 7:59 PM

The C3558 CPU in the XG-7100 is a complete SoC that includes the 4 ixgbe NICs. They are not bus connected in the traditional sense.

The quoted 2.5Gbps internal link is not a PCIe bandwidth. It's the Ethernet connection between the NICs and the on-board switch chip. Base-KX there because the chips are linked directly rather then using fibre or cat6 etc. Better explanation here:
https://etherealmind.com/backplane-ethernet-gbase-kr-kx/

Steve

Hikari · Sep 29, 2020, 8:18 PM

@teamits tnx! I just created a thread asking for that lol

@stephenw10 said in Hardware requirement for Intel home router:

The C3558 CPU in the XG-7100 is a complete SoC that includes the 4 ixgbe NICs. They are not bus connected in the traditional sense.
The quoted 2.5Gbps internal link is not a PCIe bandwidth. It's the Ethernet connection between the NICs and the on-board switch chip. Base-KX there because the chips are linked directly rather then using fibre or cat6 etc. Better explanation here:
https://etherealmind.com/backplane-ethernet-gbase-kr-kx/

tnx a lot for all the help!

Indeed I was wrong. I verified Intel spec for Atom C3558 and I210-AT, and indeed 4 ports are provided directly by the CPU. I believe I210-AT uses 3GIO 2.1, as that's its supported bus and C3558 supports 3.0 and seems to be a Comet Lake.

NollipfSense · Sep 29, 2020, 8:24 PM

@Hikari All I can say is that XG-7100 makes me drool ... if you can afford it don't hesitate.

Hikari · Sep 29, 2020, 8:45 PM

@NollipfSense said in Hardware requirement for Intel home router:

@Hikari All I can say is that XG-7100 makes me drool ... if you can afford it don't hesitate.

lol why is it that much?

Isn't it as a mini-PC with a weak CPU and some very good RAM amount and reasonable storage? What's the advantage of an appliance for a built PC for a user that knows how to install pfSense?

Hikari · Sep 30, 2020, 2:27 AM

Does SG-5100 have only 8GB for storage? How much comes free for installing extra pfSense packages?

stephenw10 · Sep 30, 2020, 11:06 AM

The packages themselves don't require much drive space. It's the logging and anything that caches that does.
This is a test SG-5100 I have here running from eMMC with Suricata installed:

Disk usage:
     / 	      23% of 6.7GiB - ufs

Steve

Hikari · Oct 1, 2020, 1:58 AM

Tnx a lot! Is it possible to use flash card on it? I missed that out.

Is there an specific model or limit?

SteveITS · Oct 1, 2020, 3:55 AM

https://www.netgate.com/solutions/pfsense/sg-5100.html shows
"8GB eMMC Flash on board
Upgradable"

I think you are looking for https://docs.netgate.com/pfsense/en/latest/solutions/sg-5100/m-2-sata-installation.html.

The only time I've ever seen a pfSense router get into space issues was one where Suricata had a bug for a while where the log files were not correctly being deleted in the default configuration, and slowly grew to 6 GB or so. Otherwise for most normal use I would expect to see in the 1-3 GB usage and if it was over 2 I'd be a bit surprised unless there was really heavy package use or squid caching or something going on.

stephenw10 · Oct 1, 2020, 1:02 PM

Yup you can install an m.2 SATA drive (not NVMe) and run from that as shown there.

You might do that if you wanted to run Squid with a large local cache or use a lot of local logging for example.

Steve

Hikari · Oct 1, 2020, 5:52 PM

tnx!!

So it's not rly necessary. Anything not router related I run on my Ubuntu server.

So, SG-5100 does have enough storage, and SG-7100's extra SSD should be for running services that can/should be ran outside router. The same should be for extra RAM.

They both have same CPU, so I guess Atom is enough too.

So, basically SG-7100 is aimed at running average services not related to routing, and for LANs with many hundreds PCs?