Intel Mini-ITX Atom 8-core Hardware Build Recipe Available Here

pfcode

@dopey:

Power supply installed. Got the build installed and doing some basic tests. I just hooked it up as a DHCP client to my current internal network and did some tests with the dhcp and dns server and some throughput tests over NAT.

900+MBit/s either direction using iperf3.

So yup, definitely gigabit routing - i was expecting this.

CPU temp topped out at 39c. I currently don't have a fan on it at all. Doesn't seem like I'll need one, but I have 3 40mm and a bracket still enroute, but I think this thing would definitely function well fanless even.

I set a 100MB /tmp and 200MB /var ramdisk and I think my memory peaked out at 5% of 8gb :)

Glad you like it! I am loving mine as well. I have the 3x 40mm fans installed and it's all located here in my office and I still can't hear it.

My iperf results topped out at 930 Mbps which I'm sure could be increased with a little tweaking here and there but I am very satisfied. My Time Warner Cable speeds are now up to 350 Mbps and I cannot wait until Google Fiber is installed in my house for 1 Gbps up/down.

I am going to rocking and rolling at that point!!!

Newbie to iperf3, Do you mind letting me know how to run this iperf3 thing to test my network throughput?

codyst

@pfcode:

@codyst:

@dopey:

Power supply installed. Got the build installed and doing some basic tests. I just hooked it up as a DHCP client to my current internal network and did some tests with the dhcp and dns server and some throughput tests over NAT.

900+MBit/s either direction using iperf3.

So yup, definitely gigabit routing - i was expecting this.

CPU temp topped out at 39c. I currently don't have a fan on it at all. Doesn't seem like I'll need one, but I have 3 40mm and a bracket still enroute, but I think this thing would definitely function well fanless even.

I set a 100MB /tmp and 200MB /var ramdisk and I think my memory peaked out at 5% of 8gb :)

Glad you like it! I am loving mine as well. I have the 3x 40mm fans installed and it's all located here in my office and I still can't hear it.

My iperf results topped out at 930 Mbps which I'm sure could be increased with a little tweaking here and there but I am very satisfied. My Time Warner Cable speeds are now up to 350 Mbps and I cannot wait until Google Fiber is installed in my house for 1 Gbps up/down.

I am going to rocking and rolling at that point!!!

Newbie to iperf3, Do you mind letting me know how to run this iperf3 thing to test my network throughput?

Easiest way is to install the iPerf 2.0.5.2 from the Package Manager. Be advised, I don't believe that version is compatible with iPerf 3.0 so go to https://iperf.fr/iperf-download.php and download version 2.0.5.2 for whatever OS your computer is.

Now it doesn't matter if you run the server or the client on your pfSense box and vice versa with your computer. Just make sure and do 1 on each, obviously.

I ran the server on my computer which is running Windows 10 by opening a command prompt and navigating to the folder that the iperf files are in.
Once you've done that, type "iperf -s" without the quotes and press enter.

Next, connect to your pfSense box by SSH, Serial, or maybe you have a monitor and keyboard connected directly to your pfSense box.
Make sure you know the local IP address of the computer you ran the server on and then open shell and type "iperf -c x.x.x.x" without the quotes and fill in your other IP address for x.x.x.x

If you did everything correctly you should see the following. Mind you, I normally get about 930 Mbits/sec but am currently downloading a couple torrents! 8)

Hope this helps!

pfcode

Thank you. It worked.

I can't click the thank you button on your post buz 'I have already posted a thanks to this topic', thats odd.

dopey

@BlueKobold:

Can you have a look at the LAN Port LED´s please, are they on or off?
Thnx for that.

They are on.

@codyst:

Now it doesn't matter if you run the server or the client on your pfSense box and vice versa with your computer. Just make sure and do 1 on each, obviously.

I ran the server on my computer which is running Windows 10 by opening a command prompt and navigating to the folder that the iperf files are in.
Once you've done that, type "iperf -s" without the quotes and press enter.

Next, connect to your pfSense box by SSH, Serial, or maybe you have a monitor and keyboard connected directly to your pfSense box.
Make sure you know the local IP address of the computer you ran the server on and then open shell and type "iperf -c x.x.x.x" without the quotes and fill in your other IP address for x.x.x.x

Unless I'm misunderstanding something, you're not testing routing throughput. You're testing the speed of the NIC and the speed of your switch. To test routing throughput you really need to have a iperf system on either side of the router (not within the router. The router is "local" to both sides and thus there is no routing/NAT going on in your scenario.

They are on.

Yep, thnx.

Unless I'm misunderstanding something, you're not testing routing throughput. You're testing the speed of the NIC and the speed of your switch. To test routing throughput you really need to have a iperf system on either side of the router (not within the router. The router is "local" to both sides and thus there is no routing/NAT going on in your scenario.

But this depends more on the goal someone want to reach,
LAN - LAN throughput (two PCs with iPerf)
WAN - LAN throughput (two PCs with iPerf)

With an plain fresh new install without firewall rules, snort, squid and whatever
it would be coming near the result what the board and NICs are able to realize.
SPI/NAT are only taking something around 3% - 5% of the entire throughput.

dopey

Got my fans. Router is now in place operating as my production router.

lan->wan throughput:
[ 4] 0.00-10.00 sec 1.09 GBytes 936 Mbits/sec 213 sender

wan->lan throughput:
[ 4] 0.00-10.00 sec 1.10 GBytes 944 Mbits/sec 172 sender

Running a simultaneous client/server brings me down to around 916 Mbit/s (i.e. an attempt to do a full duplex test)

This is with zero performance tuning other than increasing mbufs :)

Oh, and CPU is running at 21c right now for temp :)

mifronte

@dopey

Are you running your LAN -> WAN and WAN->LAN by connecting the pfSense router WAN port to your internal network?

Is RAM disk really needed if you are using SATA3 SSD?

dopey

@mifronte:

Are you running your LAN -> WAN and WAN->LAN by connecting the pfSense router WAN port to your internal network?

Not any more. That's how I tested it, initially but now it's connecting to my external network. I have a /29 through my ISP, so the pfsense router is one of my public /29 IP addresses.

I have another system that's on the public /29 address that i ran iperf on as a server, and I ran iperf internally from a client wired to the switch connected to the pfsense's LAN port to test lan<->wan throughput.

@mifronte:

Is RAM disk really needed if you are using SATA3 SSD?

Probably not :) I mean I ended up with a 120GB ssd because 32 and 64 gb SSDs aren't that much cheaper (maybe save 15-20 bucks total). But I figure why waste NAND write cycles if you don't have to :)

dopey

Just for comparison, two LAN clients, wired onto the same gigabit switch, I get the exact same speeds.

I'm not using jumbo frames, so theoretical maximum throughput with 1500mtu is about 940-950Mbit/s.

So 930 or so, ain't too shabby :)

redpine

So I read all the way through the thread. Went ahead and ordered the same parts and will be putting a box together over the next couple weeks. Was wondering what people thought about the following.

1. USB wireless add on or pcie wireless add on?
2. Run in a VM under Centos(KVM)?

Thanks for all the great info.

ciccio

HI,

I would like to change my actual router (Asus RT N-16) with a PFSense DIY router.

I followed this thread (your experience have been very helpful) and I will order in next few days:

Supermicro A1SRi-C2758, 8GB RAM (2x 4GB), SSD (I have not yet decided).

My doubts are:

I would like to add a Wifi card (on PCI-E slot): what do you recommend?
What case do you recommend given the presence of the above card and since I will use the router also as a VPN gateway (I would not have a router too noisy).

Thanks in advance

@redpine

1. USB wireless add on or pcie wireless add on?

An external WiFi AP would be the best in my eyes, perhaps something cheap from Buffalo, Netgear, TP-Link
pre-installed or flashed with DD-WRT or OpenWRT would do the job nest at these days. Or a plain UBNT WLAN
AP would also be a really good choice if more then one AP is needed related to the free WLAN controller software.

I would not have a router too noisy

You can go with a C2558 Board this comes without a CPU fan and would be also sufficient enough
for all actions too.

dopey

My previous router was my Netgear R7000 running dd-wrt. I simply reflashed that back to stock Netgear firmware and use that as an AP for my wireless needs.

IMO, it's much easier to manage than having a router+wireless AP in one unit. Plus, wireless architecture is changing at a much faster rate than router technology are. You can future proof a router for a long long time. Wifi is much harder to future proof.

I'd definitely recommend getting either a consumer off the shelf AC router based on your needs (i like tp-link, netgear and asus ones for performance/value) or if you're in need of more enterprise-y gear, look at ubiquiti.

dopey

throughput. So someone else messaged me with a problem that I am able to reproduce with my C2758 rig.

If I install iperf from pfsense packages on the router itself and use it to test performance, I'm getting ~600mbit/s single threaded throughput.

Yet it clearly NATs gigabit without any problems.

If I use -P (parallel threads) I can get ~940Mbit to the router.

Anyone else seeing this behavior? It seems surprising the C2758 can't handle this level of single threaded i/o but maybe I'm overestimating it's single core performance.

In case it matters, I tried with two iperf clients - a ThinkServer running an i3-4130 and intel gigabit NIC, and a i7-2600k system also using an intel gigabit NIC and I get the same behavior from either machine.

redpine

Got all the parts, but the SSD and fans. Got IPMI up and running and PXE boot (once I have the SSD I can actually finish the install). Going to do a PXE boot install of Centos and then install Xen and the PFSense in it's own VM.

Looking at IPMI it showed the CPU running at 50C, and that's just sitting their idle. Seems a little high for a passive cooled CPU. Is this normal my other fan cooled boxes show 32 C at idle

dopey

@redpine:

Looking at IPMI it showed the CPU running at 50C, and that's just sitting their idle. Seems a little high for a passive cooled CPU. Is this normal my other fan cooled boxes show 32 C at idle

That's a little bit high. It's still in the safe range for that cpu, but mine was running in the 30s fanless and in the 20s with a few 40mm fans.

What's yoru ambient temperature? What does the system temp read?

The ambient temperature in my room is arond 65-70f

redpine

The ambient temperature in my room is arond 65-70f

Brrrrrr. Mine is 74f.

Fans aren't here yet, but that's ok. It'll take me a while to figure it all out, before I throw it in the garage and hook it up to the internet.

Seems a little high for a passive cooled CPU.

Thats pointed to the VM I think. With a native install it would be like the others here
told around and much less hot.

MikeV7896

@redpine:

The ambient temperature in my room is arond 65-70f

Brrrrrr. Mine is 74f.

Fans aren't here yet, but that's ok. It'll take me a while to figure it all out, before I throw it in the garage and hook it up to the internet.

I keep my place at 76-78 during the day, and fanless at near idle, my 2558 board would run ~35-40. Adding one of those 40x40x15 fans has brought it into the mid-20's.

These numbers are based on pfSense's reading (from the CPU itself via coretemp). The IPMI sensor always reports warmer than the CPU's own sensor on my board. Right now, pfSense (coretemp) is reporting 25 degrees, while IPMI is reporting 30 for the CPU.

is reporting 25 degrees, while IPMI is reporting 30 for the CPU.

I my eyes nothing to compare against, sorry. Yours is the C2558 and this comes from the entry
point with no active fans for the cpu, but the C2758 board does. And then on top of this, if someone
runs pfSense native on this boards it will be even cooler then running CentOS and pfSense in a VM on this.