First pfSense Build - Hardware recommendations and access points

bigjme93

Hi Everyone,

First i apologies if this is in the wrong topic or area, i am looking for some hardware recommendations for a pfSense box and access points to attach to it (it's also my first post so hi)

Below i have outlines everything i can think of that may be of use

Current Setup
First off let me give you an overhead of my current set up so give you an idea of how low end things are. My current router is a D-Link DIR-850L Model A router (here)

This router served well but now we have more and more network traffic and mobile devices we are seeing issues with the router locking up and needing reboots and being unable to load webpages via wifi even with wifi signal, we are also suffering some dead spots even with a fairly small victorian brick house. This device connects to a modem using PPPoE

Proposed pfSense Build

CPU/Mobo: ASRock C2750D4I Avoton C2750 8 Core (here)
Memory: 16GB Crucial ECC Unbuffered DDR3 8 GB 1600 MHz UDIMM (here)
Storage: 120GB SSD
Case: 1U Server Case (here)
PSU: 320W 1U Flex ATX (here)

I am fairly sure i can install pfSense on the SSD and use the SSD for cache if needed but please correct me if i am wrong

This router will need to handle the following:

ipSec Site-to-Site
OpenVPN - potentially
Squid - potentially but not needed
DarkStat
Snort
Country Block

This router will need to bridge this site to another so i can access CCTV cameras for remote recording - up to 8mbps through the vpn is all i expect to see constantly. I will also have my nas server behind this router to allow remote access to important documents and offsite backup for the other property (not huge files - 20mbps maximum throughput due to current internet limitations but wish to cater for up to 1gbps)

Power usage is important on this system and i would prefer it to be as quiet as possible - i watercooled my current 4u server so it was silent!

Access Point
I have been recommended Trendnet access points by work however work have not tested wireless AC models enough to be sure - i do need wireless ac as one machine will be losing its wired connection for a wireless ac connection

I have been reading a lot of good reviews for the Ubiquiti UAP-AC-PRO (here) which has wireless AC and i can run the management interface from my virtualization server

Any recommendations on which of these to go for would be great - the wireless is streaming multiple 1080p videos and youtube constantly

Connections
I have an 8 port POE switch already which is soon to be upgraded so everything will be connecting as follows

modem to pfsense - logged in using PPPoE
pfsense to switch
switch to everything else

Budget
My budget for router and access point is: £670 - and i am UK based

Right now my pricing is:

CPU/Mobo - £347.48
PSU - £29.96
Case - £35.99
Memory - £81.98
SSD - £39.49

Total: £534.9

Access Point - £131.48

Intended Users
5 Hardwired systems
10 Wireless devices
8 IP CCTV Cams (4mp) - 2 being accessed using the IPSec vpn

I'm going for a 1u system to go into my rack without filling up too much space (using 8 of my 15u for servers, and have switches, ups's etc. to add) - my current proposed system should fit in a 1u rack without issues but if it doesn't please let me know

Finishing Up
So i don't know what sort of benefits to expect on my current line (60 down, 20 up) but i'm hoping this will be plenty to cater for future line upgrades

IPMI on the motherboard is great so i can run it headless from my server rack
I don't want to put lower end hardware in the router just in case (this system will allow for other uses if i need it later on) and this cpu/mobo is not that expensive at all

As i have never moved away from commercial routers until now and am going to a completely custom and separated solution that i have never used i wanted to see if anyone here has anything to point out, things to consider and potential pitfalls to my solution above

Any comments are highly appreciated and i look forward to seeing what you all have to say

Jamie

whosmatt

A quick response:

You probably don't need the 8 core CPU. Four would do the job fine. Won't hurt of course.

16GB of RAM is likely overkill for a home pfSense router. ECC probably is too, but since you're buying a server board, it seems like the right choice. And if you ever want to repurpose the build then it makes sense. If I had the budget at home I'd go with a server board just for the IPMI.

Using the SSD for the OS and cache is just fine. You don't need 120GB but if the price difference is negligible then no big deal. If you want to save a few bucks go smaller.

The Ubiquiti APs are good IMO. I've just started using them. I bought the AP AC LITE for my home (and thinking of adding a AP AC PRO for outdoor use). Subsequently bought 3 of the AP AC LITE for one of the offices at work to replace an aging Cisco system with dying access points. It's great for home use, but takes some extra steps in a business environment. For example, each of the APs needs to talk directly to the RADIUS servers for 802.1x, in contrast to the controller handling that connection. Also reconfiguring the wireless networks provisions each of the APs and results in a momentary loss of wireless access. But it's a super cheap way to get enterprise level options. Plus, they look nice.

bigjme93

@whosmatt:

A quick response:

You probably don't need the 8 core CPU. Four would do the job fine. Won't hurt of course.

16GB of RAM is likely overkill for a home pfSense router. ECC probably is too, but since you're buying a server board, it seems like the right choice. And if you ever want to repurpose the build then it makes sense. If I had the budget at home I'd go with a server board just for the IPMI.

Using the SSD for the OS and cache is just fine. You don't need 120GB but if the price difference is negligible then no big deal. If you want to save a few bucks go smaller.

The Ubiquiti APs are good IMO. I've just started using them. I bought the AP AC LITE for my home (and thinking of adding a AP AC PRO for outdoor use). Subsequently bought 3 of the AP AC LITE for one of the offices at work to replace an aging Cisco system with dying access points. It's great for home use, but takes some extra steps in a business environment. For example, each of the APs needs to talk directly to the RADIUS servers for 802.1x, in contrast to the controller handling that connection. Also reconfiguring the wireless networks provisions each of the APs and results in a momentary loss of wireless access. But it's a super cheap way to get enterprise level options. Plus, they look nice.

Many thanks for the reply

I completely agree that 8 cores is overkill for this application but since the price between 4 and 8 is not high i may as well

I did see a lot of people saying that ECC was a waste - in all honesty if i drop ecc i will save around £13 and get higher speed memory, as many have pointed out - pfsense does need updates so i don't really need ecc nor will i in the future - i just wasn't sure if ecc would help with anything like doing vpn connections? of course if it doesn't i will jump on a 16gb kit of 1600MHz corsair low profile memory sticks

This server may be re-purposed to another off site backup or media player later in the future but i highly doubt it - best to plan ahead though

I did see a lot of people looking into the Xeon-D line processors but with their higher TDP and price i don't think their worth it (not to mention impossible to get hold of)

SSD price differences are so minimal in the UK that most stores won't sell anything less than 120gb anymore - in fact i can go up to 250gb for an extra £16. I honestly don't know what to expect from Squid for caching though - we have at least 2TB of browsing and video streams go through our router a month - how much of this Squid will cache i am not sure

I was very drawn to the ubiquiti access points based on design and poe - I'm not really sure what i will gain on going pro over lite or long range other than a bit of extra speed when i'm closer - the wireless machine will be about 10ft (and a wall) from the access point so it won't be far - although i do plan to leave the access point sitting in the top of my server rack for a while so i'm not sure how the metal will affect it

The pricing isn't too much of a problem on this build as it should last a long time - honestly ipmi was a huge help for me as i already have 3 video outputs from my server cab so this will save going to 4 or more when i add more hardware in the future

Again thanks for the reply

Jamie

mattyd

@bigjme93:

SSD price differences are so minimal in the UK that most stores won't sell anything less than 120gb anymore - in fact i can go up to 250gb for an extra £16. I honestly don't know what to expect from Squid for caching though - we have at least 2TB of browsing and video streams go through our router a month - how much of this Squid will cache i am not sure

If you're setting up a transparent proxy and not doing the extra hassle of https intercept, then only http traffic will get cached. You can adjust the Squid cache size. I have squid set up mostly so I can use a block list and AV scan. For my usage at home, I don't really notice it is there unless a site is unreachable. I need to put a cert on the box so the "unreachable" redirect (https) doesn't cause a cert error first before I see it. :) I hear Let's Encrypt is coming in a future pfSense update, so waiting on that.

@bigjme93:

I was very drawn to the ubiquiti access points based on design and poe - I'm not really sure what i will gain on going pro over lite or long range other than a bit of extra speed when i'm closer - the wireless machine will be about 10ft (and a wall) from the access point so it won't be far - although i do plan to leave the access point sitting in the top of my server rack for a while so i'm not sure how the metal will affect it

Two big differences, and one smaller one. The two Pro models so far have been mounting bracket compatible, which makes it easy to swap one for the other if you want to update. The other big difference is that lite and long range are non-standard PoE, so you must use their injector. The Pro is standards compliant, so your PoE switch will just work.

Pro is also 3x3 MIMO for the 5GHz range, which could potentially give you better speeds on 5GHz if your clients support it.

bigjme93

I did not spot that the non pro models did not have PoE so thank you

Honestly i have seen Squid in the recommended list of plugins but have no idea if it will have an affect - i have no websites i wish to block on this network so it may have no use at all

I do want to run virus scans on the box (as you mentioned) and any other security i can - i have rather a few servers going behind this and while they all will have anti-virus etc. the more i have the better - any recommendations on stuff like that is always welcome and wanted!

Jamie

whosmatt

@mattyd:

Two big differences, and one smaller one. The two Pro models so far have been mounting bracket compatible, which makes it easy to swap one for the other if you want to update. The other big difference is that lite and long range are non-standard PoE, so you must use their injector. The Pro is standards compliant, so your PoE switch will just work.

Pro is also 3x3 MIMO for the 5GHz range, which could potentially give you better speeds on 5GHz if your clients support it.

Another difference with the PRO is that it can be installed outdoors in a non-direct weather situation. I wasn't aware of the PoE differences either; I thought they all had either to use the provided injector or Ubiquiti's own PoE switch. That's good to know.

kapara

I did not spot that the non pro models did not have PoE so thank you

It is not that they are not POE but that they are passive POE. They do not work with 802.3af compliant devices which the Pro are and a majority of the switches only support 802.3af.

bigjme93

@kapara:

It is not that they are not POE but that they are passive POE. They do not work with 802.3af compliant devices which the Pro are and a majority of the switches only support 802.3af.

Ok that makes sense - my current switch does support 802.3af so that is not a problem

The machine connecting to the access point is just using its on-board wireless ac which is only rated at 867mb/s so i won't see much from the pro upgrade but it could mean better speed at slightly further away i guess. The PoE 802.3af is a benefit as i don't want to be using injectors at all (running out of power sockets) - the access point seems a sure win from this conversation and others i have read

I guess my final questions now are:

Is 16GB overkill for sure? I'm happy to get a single 8gb stick and upgrade later if needed - i think less than 8gb is too low but as whosmatt pointed out, 16gb may be too much (it would save me around £35 dropping the extra 8gb
I'm likely to only want squid for anti-virus purposed - is there anything i need to consider in this respect?

Following that i go for 8gb of ddr3 none ecc my new pricing would look as follows:

CPU/Mobo - £347.48
PSU - £29.96
Case - £35.99
Memory - £34.99
SSD - £39.49

Total: £487.91 (saving me £46.99 from my initial list)

This leaves me with 3 memory slots free if i need them in future

Any suggestions?

Regards,
Jamie

whosmatt

@bigjme93:

Is 16GB overkill for sure? I'm happy to get a single 8gb stick and upgrade later if needed - i think less than 8gb is too low but as whosmatt pointed out, 16gb may be too much (it would save me around £35 dropping the extra 8gb

Honestly, for most uses, I think 8GB is too much. Certainly it won't hurt, but I've got 4GB in my home build and more than 3GB of it sits unused. At work, we're running 1GB of RAM in our VMs that handle a couple hundred machines in about 15 subnets and it's never been a problem.

bigjme93

@whosmatt:

Honestly, for most uses, I think 8GB is too much. Certainly it won't hurt, but I've got 4GB in my home build and more than 3GB of it sits unused. At work, we're running 1GB of RAM in our VMs that handle a couple hundred machines in about 15 subnets and it's never been a problem.

I agree that the 8GB may be a little over kill still - a 4GB module is £19.99 compared to £34.99 for 16GB so i would pay more to upgrade later if needed (compared to 16GB being twice the 8GB price) - at that point i see no gain in going lower

Dropping to 8GB however has allowed me to budget in a better PSU than the original although a 250W PSU running this system seems like i'm going to be throwing power away due to the lower efficiency with such low power hardware involved (i'm expecting to see around 20W at the wall on average)

On the power front if anyone has similar specs and would like to share power usage that would be great

Jamie

whosmatt

@bigjme93:

I agree that the 8GB may be a little over kill still - a 4GB module is £19.99 compared to £34.99 for 16GB so i would pay more to upgrade later if needed (compared to 16GB being twice the 8GB price) - at that point i see no gain in going lower

Dropping to 8GB however has allowed me to budget in a better PSU than the original although a 250W PSU running this system seems like i'm going to be throwing power away due to the lower efficiency with such low power hardware involved (i'm expecting to see around 20W at the wall on average)

On the power front if anyone has similar specs and would like to share power usage that would be great

Jamie

Yeah, no reason not just to go for the price sweet spot as far as RAM is concerned. I just see a lot of people spec'ing 16GB of RAM for a home build.. if you look at the pfSesnse store offerings, the highest level of hardware has 16GB and is meant to serve many thousands of clients potentially. Don't mean to harp on it too much, just trying to educate people about the real RAM requirements of pfSense.

I'd get the best (highest efficiency) PSU you can fit into your budget, obviously. Sadly, I think there's a dearth of high efficiency PSUs designed to service low power systems unless you go with a PicoPSU or something like that. Maybe I missed it in a previous post, but which case did you choose (or have you chosen one yet)? With a Mini-ITX board that doesn't need PCI-e expansion cards or active cooling, you could potentially go with a DC power supply. 20 Watts should be easily handled by a 12V 4A power brick (with plenty of headroom), for example.

bigjme93

The case I was looking at is in the original post, I put hyperlinks to the majority of the parts but here is the link again: here

It's just a small 1u case with dual 40mm fans - should be enough to cool this but again I'm open to suggestions

Jamie

messerchmidt

i am an overkill kind of guy - good pick. buy ecc ram while your doing it. i would go for a 256gb ssd for not much more

for wifi AP, netgear r7000 AC1900 using ddwrt or tomato, add a cheap usb powered laptop cooler below to keep the temps down

pick a switch

whosmatt

@bigjme93:

The case I was looking at is in the original post, I put hyperlinks to the majority of the parts but here is the link again: here

It's just a small 1u case with dual 40mm fans - should be enough to cool this but again I'm open to suggestions

Jamie

Oh, right, 1U. I get these build threads confused sometimes.

bigjme93

OK so my network has crippled today and the Internet is pretty much unusable as it stands (wireless with no Internet, websites not loading) so I think it's time to up my schedule a little

I'm looking to get this system on order next week so any more recommendations would be appreciated

I am aware the server case fans will most likely be as noisy as sin so I have found some noctua quiet fans that I can order if needed

The picoPsu's look good but are almost the cost of a 1u PSU in the UK (£20 at the cheapest) so I'm thinking I may just cause more issues due to power bricks and the horrible connectors that come with them.

Granted I will lose some efficiency from using a 1u PSU but for the sake of keeping things simple I think I'm going to grab one anyway

My main concern is the PSU noise as I am used to a silent system with a PSU fan that is turned off most the time. I've heard they sound like a jet engine but never actually used a 1u case or fans for comparison

Any comments on the PSU fan noise to expect? I can potentially see the need to replace the PSU fans

Jamie

Guest

CPU/Mobo: ASRock C2750D4I Avoton C2750 8 Core (here)

Supermicro C2758 mobo
pfSense SG-4860/SG-8860

Memory: 16GB Crucial ECC Unbuffered DDR3 8 GB 1600 MHz UDIMM (here)

It might be more pointed to the circumstance how many traffic will be generated but also good for;

high up the mbuf size to 1000000
high up the Squid RAM amount
more RAM for all other usages (caches, RAM disks)
DHCP & DNS entries must be stored to be able to cache

PSU: 320W 1U Flex ATX (here)

To much in my eyes, a smaller one likes the 160 Watt PicoPSU would be enough.

I am fairly sure i can install pfSense on the SSD and use the SSD for cache if needed but please correct me if i am wrong

This router will need to handle the following:
ipSec Site-to-Site

AES-NI would here more useful to speed up the IPSec VPN part, the Supermicro C2758 comes with AES-NI!

OpenVPN - potentially

Only raw CPU power and Cores are counting

Squid - potentially but not needed
DarkStat
Snort
Country Block

firewall only & VPN 2 GB - 4 GB
firewall, Squid 4 GB - 8 GB
Squid, Snort & VPN 4 GB - 8 GB
Squid, Snort, pfBlockerNG & VPN 8 GB or more

WiFi or WLAN APs would be better in my eyes as an external device likes;
- UBNT WLAN APs
– UniFi series
- MikroTik WiFi APs
– MikroTik RB953GS-5HNT-RP &
– MikroTik R11e-5HacT (802.11 (a/n/ac))

bigjme93

To my understanding the C2750 supports AES-NI? It's on the Intel spec page for it unless there is something the motherboard needs to do also?

I know the system has no quick assist but isn't it AES-NI that does the encryption for ipsec and OpenVPN?

Guest

To my understanding the C2750 supports AES-NI? It's on the Intel spec page for it unless there is something the motherboard needs to do also?

No, the C2750 (Avoton) is not supporting AES-NI only the C2758 (Rangeley) is supporting AES-NI!

I know the system has no quick assist but isn't it AES-NI that does the encryption for ipsec and OpenVPN?

AES-NI is speeding up the VPN tasks for sure, IPSec more then OpenVPN but Intel QuickAssist is
able to compress or decompress data packets more and perhaps it will be also a gain for other things.

bigjme93

Not to argue but are you sure?

http://ark.intel.com/products/77987/Intel-Atom-Processor-C2750-4M-Cache-2_40-GHz

The last spec states AES-NI support for the chip?
And if the chip supports it then the system should surely?

Stugots

I'm very pleased with my current setup. Thought I'd share.

Firewall: PC Engines APU2C4
AP: Ubiquiti AP-AC-LR
Switch: Dlink DGS-1100

I admit the switch isn't the best, but there kind/sorta manager switches that support gigabit and are very affordable.