Need help with sizing home office network with complications.

kroberts

Hi,

I see a lot of these on the forum but I don't see anything quite like my problem. It's a bit more complex I think.

I work from home. I have to deal with data that doesn't have a lot of monetary value but it's private and I'm bound by contract to protect it. We frequently send large quantities of data across the wire. If we do that, it's both confidential and urgent, so we need a fast VPN.

The Sony Pictures hack scared me a lot. Something like that on a personal scale would sink me, just a single incident.

All my current equipment is ipv6-capable and most of it has been used on ipv6, with the possible exception of a printer. All my future equipment will be ipv6-capable. I would prefer to be entirely ipv6 but that's phase 2.

I collaborate regularly with a small team of coworkers. I need for them to get into my network with good speed. I will make recommendations to them for upgrades of their equipment as well.

My infrastructure needs are:

Outside firewall needs to be pretty bullet proof. Need rules in and out and ability to route public IP addresses in a DMZ.
6to4 and solid ipv6 capabilities. I can get 200mbps but can't get ipv6!
One NAT network (or ipv6) with typical SOHO wireless router. Has no access to higher-security networks but is protected by outside firewall. VPN gets some controlled access to this network.
One DMZ with a few machines on it, mostly https, need tight firewall rules in and out to describe exactly what can go through. VPN access gets full access to this network.
One private network which only can be reached by OpenVPN but can get to any of the above networks.
One extremely private network which can't be reached from anywhere except that network, but which can get out to any of the others.
I will need firewall rules between all these networks and the outside, but mostly the inside rules are a yes/no for all traffic.

Speed: Right now I have 60 mbps. Speed tests from speedtest.net show me exceeding that regularly, sometimes up to 70mbps for quite awhile. Speed tests transferring files to customer servers vary, but for higher performance sites I get the same speeds.

Once I get secure I will upgrade to my ISP's current max throughput of 200mbps. Within 2 years they expect gigabit access, and I have no idea if they mean 750mbps or actual gigabit. I will almost certainly upgrade to whatever I can get.

About security:

I'm a big believer in defense in depth, but I'm not sure how to implement it in my case.
I think separate hardware for the point of presence for sure.
I'm not sure how to implement the VPN. I want the best throughput I can get.
I would like multi-factor authentication with cell phones, either text or email or an actual automated call.
I will probably have a higher performance internal router/firewall, I think I would put it on the DMZ.

I'm a big fan of virtualization but not sure if it's a good idea here. I don't want a single point of failure that puts me out of business.

I've heard of accelerators such as VIA Padlock, not sure how much it can help my situation but I'm very open to suggestions about hardware acceleration. I'm assuming if I get hardware acceleration then I'm getting real hardware for the VPN.

I'm looking for the best bang for the buck here. I also need the solution to be pretty easy to work with once it's in place, not everyone I work with is a computer nerd. We have all major platforms. I'm a Linux nerd, never tried FreeBSD. I hope it's not too difficult.

My questions:

What hardware recommended for gigabit Internet, 6to4, firewall and some NAT?
What hardware for an internal high throughput router and OpenVPN with at least 400mbps?
Should I consider virtualization for some of this? What is safe?
Am I missing something critical in terms of best practices?

Keljian

No real complications I can see here.

You need either something like the PFsense 7551 (http://store.pfsense.org/FW-7551/) or a custom box with an i3 Haswell in it.

If you go custom you need::

an i3-4xxx chip or above
a motherboard to suit (H81 boards are cheap)
about 4 gig of ram, preferably 6-8.
an intel nic with at minimum 2 ports, ideally four. An i350-T4 (from ebay) would suit.
a storage device (eg hard drive/ssd)
a power supply/case

You will want an i3 haswell (or above) or an i5 previous generation processor as they include AES-NI (crypto acceleration)

I highly recommend the gold subscription to PFsense if only for the new manual as it will help you immensely for setting it all up.

I would avoid virtualisation as you're just starting out and you want to know exactly what is broken if it breaks, rather than guessing if it's your pfsense setup vs virtualisation

kroberts

Is that 7551 for the main access, the inner firewall, or both?

This is exactly the sort of info i was looking for, thanks.

Keljian

Ok I can't draw this for you well, so a description will have to do

I would have 1x 7551 or custom box that interfaces everything, and seperate subnets for each of the groups you have mentioned.

This effectively(for ease of implementation) means one "port" per group, plus one for the upstream connection.

My understanding is you want 3 networks, main, secure and dmz

For each of these I would have seperate downstream hardware (eg good consumer hardware like the Asus ac66u) or alternatively a good but basic wired switch, depending on your configuration of clients.

If you are super paranoid you can use pfsense again for these, but I do not think it is necessary.

You then want to set up NAT rules, firewall and vpn stuff to suit each port on the key pfsense box

Derelict

I don't see anything that pfSense can't do.

What inner firewall do you think you need?

Wolf666

You should match your needs with http://store.netgate.com/FW-7551.aspx, in any case I would think about atom Rangley based appliaces (C2558 or C2758).
4 GB of ECC RAM would be enough for Routing/Firewalling and VPN.
I let IT PRO to confirm but you really don't need other appliances in between to manage your scenario, except Wi-Fi AP or Switches.

Keljian

The benefit of buying directly from ESF (the guys who make pfsense) is you get one year support, warranty and knowledge that your setup will just work as it already comes pre installed, versus negate which is hardware only.

Derelict

Except that netgate also includes support, which from what I can tell is the same as you get from buying at the pfsense store. Personally, I don't know why anyone would buy from pfsense when they can get the same thing for less cash buying from netgate.

kroberts

OK so to get some things out of the way:

First, if the 7551 is actually the best fit for something I need to do then I'm certainly game for it. This is for my work, and by the time I build a box and install pfSense on it it's going to be that much anyway. The only reason I would choose to build is if building gets me something I need but that box does not have. If I build I will no doubt need to get a support service separately since I really have no idea how this all works.

pfSense does deep packet inspection and intrusion detection/prevention? Does the 7551 fit that bill? I certainly want that in the mix and forgot to put it in. I'm having a hard time believing that a single appliance can do all this at several hundred mbps. Does somebody have benchmarks for the 7551? Interested in benchmarks with and without VPN duties, and with intrusion detection/prevention.

Yes, I'm super paranoid. One decent firewall between my public-facing and normal home stuff is fine, but I need more for the data I'm bound to protect.

I guess I could start out with one device and see how it goes.

I browsed Intel chips yesterday, it seems I can get an i5 or e3 quad core for not much more than an i3. Processing power matters greatly here right? And memory, and good nics? I would much rather have extra ability and not need it than not enough ability and want it.

Speaking of nics, I have a 4-way 4-lane PCIe Realtek card still in the box. Another one just like it shows up like this in Linux:
03:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller (rev 02)
04:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller (rev 02)
05:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller (rev 02)
06:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller (rev 02)

Is there a real need for the Intel nics? Or is it preference from the poster?

I also have a 1st generation i7 sitting around with 12g RAM, but it doesn't have aes hardware acceleration. And uses a lot of power.

OK let's get back on target here. What actually needs to be at the front door? I think:

Firewall
Deep packet inspection/intrusion detection & prevention
Logging
6to4
NAT

Things I need, but could be on the second router:

OpenVPN
VLAN support
Multifactor authentication (don't really know how to do this, probably part of vpn?)

For the moment I won't be getting more than 200 mbps for awhile, so I might try just the one box until I get it set up and then split duties later.

Keljian

Ok Let's start by answering the last question first.

1. Yes. One pfSense box with adequate ports and adequate grunt will be able to handle all of this.

2. Feature needs

OK let's get back on target here. What actually needs to be at the front door? I think:
Firewall
Deep packet inspection/intrusion detection & prevention
Logging
6to4
NAT
OpenVPN
VLAN support
Multifactor authentication (don't really know how to do this, probably part of vpn?)

Firewall, logging, 6to4,Nat, vlan support, openvpn and Multifactor authentication (Captive portal) are part of the default install of pfsense

Deep packet inspection/IDS etc duties are provided by Suricata or Snort packages, or both.

Keep in mind there is a learning curve with this stuff. - This is why I recommended a subscription, as there is a manual that comes with it.

3. Hardware needs

Intel nics are significantly better supported than the competitors and generally are faster. They also allow offloading of some of the network processing required which relieves the processor, unlike the realtek series. I personally wouldn't run realtek nics on core infrastructure unless I had to.

VPN, Snort and suricata are going to be the things that hit memory and processor, the rest is relatively benign.

You need about 1 gig for Suricata (on a medium sized ruleset) and about 2 gigs of ram for snort on the same ruleset, allow 1 extra gig of ram for each interface you want these on (typically you only have it on your WAN or upstream port)

Processor wise is a bit trickier as the 7551 has quickassist which is custom hardware for running snort type packages. That said, broadly speaking every i3+ from the last 3 generations of intel processors will be able to handle snort and suricata running at the same time with plenty of throughput. More is better, naturally.

AES-NI speeds up AES-128 and AES-256 in openvpn. There is a list of processors with aes-ni support here: http://ark.intel.com/search/advanced/?s=t&AESTech=true but it is lengthy, you are better off using the wikipedia entry on it here : http://en.wikipedia.org/wiki/AES_instruction_set

Based on that your i7 (westmere) does actually have AES-NI

If you have AES-NI and choose to use it, you can expect immense throughput for VPN, in the order of 5-10gbps per core.

If you are building, priority wise, I would go nic, memory, processor.

Nic - the i350 is my choice, it's a solid performer and supports everything under the sun, while consuming little power. You need 1 port for each network, and one for uplink/wan. You can get the i350-T4 (4 ports) from ebay for $100-150 used.
Memory - you need about 4 gig, more is gravy. I run 6 and barely touch it (15%) but I don't run snort or captive portal, even if I did run snort I guess I'd only hit about 25-35%.
Processor- you need aes-ni, but you don't need an i7 to do it. if the budget allows, go with a new-ish i5, but an i3 should still be more than capable.

kroberts

Keljian,

You're being incredibly helpful. I hear you, I'm just trying to work toward some compromise between what I have in mind and what you have in mind.

Everyone insists I only need one box, but I'm trying for defense in depth. Do you think it doesn't make that much difference? One bigger box would certainly be cheaper than two smaller ones.

My free i7 box does not have AES. It's this one: http://ark.intel.com/products/37147/Intel-Core-i7-920-Processor-8M-Cache-2_66-GHz-4_80-GTs-Intel-QPI

So going with the single router idea, I'm coming up a port short of what I had in mind. If those nics can be had for $100 (I checked, there are several) then two of them is no big deal for me. A QuickAssist adapter, on the other hand, is a completely different deal. An Intel QuickAssist 8950 adapter is for sale at exactly one place I found, for $958.66. On the other hand, it will evidently do just about every cipher I've ever heard of and promises 50gbps while doing it. There's supposedly a 8920 out there somewhere but as far as I can tell nobody sells it.

Keljian

Thanks happy to help!

Ok you do not need a quick assist card.
You will not need to push 50gbps. I mean, 10gbps+ nics exist but you will not be pushing that over vpn. At most you will push 2gbps, 1 up and 1 down

Now that that's sorted, if you want to support higher levels of encryption than aes-256, then I question what you are doing and wonder whether file based encryption as well as VPN encryption might be better…. That said my answer would be to throw brute force at it, so an i7-4790 or even an i7-5xxx 6 core would not be unreasonable. The latter would require a graphics card though, as it doesn't have one integrated.

You only need one box. A second is only needed if you need redundancy.

kroberts

Sorry it's been so long for a reply. Life intrudes, need to continue even while researching.

I've been questioning a lot of what we do and how we do it. We definitely need a high security firewall and vpn, and the 7551 is definitely in the top 5 possibilities.

Unfortunately I don't think I can accelerate things in the places we most need the acceleration. A hosted db server, for example, especially a VM, is not going to have access to AES-NI or any of that. We don't generally control that provisioning in any case.

Ideally, we would have acceleration on the remote server hardware which could zip and encrypt the file before sending it. Zipping alone often takes 3 hours or more. If we had one giant VM host I could see putting a QuickAssist card in that and then making it available to all the VMs, if that were possible. It would certainly be cost effective in that scenario, especially considering the hours we've spent waiting for a file to finish zipping before pulling it off the remote server. But that's not really possible here, since we have several remote sites and no control over the hardware.

Even so, if there were built-in hardware acceleration for any of this process, even a zip that takes advantage of some hardware compression, that would help a lot.

Is it feasible with a 7551 to have a site-to-site vpn which not only has encryption but compression as well?

Just for clarity, QuickAssist hardware would allow rapid encryption even with some command-line tool like gpg, right?

In my personal case, I still see an attraction for a second firewall which could easily be cheaper hardware or even a VM. For the purposes of this discussion I don't need to talk about it. I've set up quite a few Linux firewalls before, and this one would be pretty simple.

Thanks again.

Keljian

Again I'll resort to points, due to time constraints:

1. I don't know enough about QuickAssist in linux et al to help you. I do know there are gzip cards that can compress stuff on the fly when attached to networks, I don't know if they will work for pfsense. I do know that quickassist on the 7551 and C2758 platforms will be supported (if it is not already) by pfsense

2. While you can't change the hardware of the target systems, you may be able to change the software… If zipping is taking that long to do, it might be worthwhile looking into recompiling a half decent open source zip program from source using the Intel compiler (which is about $600 from memory), You could save half of that time if it is compiled specifically for the platforms you are running on.
There is a 30 day trial you could do to experiment with this option.
$600 may save you hours if not days of transfers if this is a critical sticking point. A cheaper option if you're on linux is to use GCC with some platform specific flags, while this is not optimal it may net you a 25% speed gain (or more).

All of this would require testing.

3. Remember that openvpn includes LZO compression if enabled - so it might help with the data, depending on what it is and how it is packed. This may mean that you are better off sending uncompressed files across the vpn, to be compressed on the fly by the tunnel. You might lose out on compression ratio, but you would get time back by sending across the tunnel immediately.

There is also the next system up from the 7551 - http://store.netgate.com/Firewall/C2758.aspx - Just FYI.

Keljian

I did some googling and found this:
https://01.org/packet-processing/intel-quickassist-technology-drivers-and-patches

I suggest some more research is in order for the command line stuff you were talking about

Quickassist in the 7551 and the other pfsense c2758 is capable of 50gbps throughput, with the right hardware hooked up, this will utterly obliterate the fastest i7-4790 in crypto, once it is finished and baked into pfsense. No timeline on that though.

The thing that may stop that from happening for openvpn is that it is single threaded, which I am guessing is the reason IPSec is being given priority at the moment for high performance vpn support in pfsense.

Keljian

Gonzo posted these results for IPSec on the 7551

https://forum.pfsense.org/index.php?topic=81862.0

I'm seeing between 729mbps and 891mbps throughput in the below.

At a guess I would think openvpn would be the same ballpark…(and this is without quick assist!)

kroberts

OK thanks guys, I have some thinking to do.

I've found some motherboards based on the chips in these routers, they have quickassist and everything. Doing some checking on that end.

Thanks.

kroberts

I just bought one of these:

http://www.supermicro.com/products/motherboard/Atom/X10/A1SRM-LN7F-2758.cfm

Reviewed here:

http://www.servethehome.com/Server-detail/supermicro-a1srm-ln7f-2758-review-awesome/

I bought 16g ecc registered memory and an ocz vector 150 240gb ssd. Nobody had msata that I liked.

My intent is to put Gentoo 64-bit hardened on the bare metal as a minimal KVM host, and use PCI passthrough to one or more router VMs. One will be pfSense at least as a trial, and if I get serious about it I will probably pay support.