Google Fiber and AT&T Uverse GigaPower - what do I need?

Aluminum

Honestly your requirements are a little bit ridiculous. You want a fanless passively cooled system that will handle 300+ Mb/s throughput? You can't even do that with ASIC devices. Find a closet you can put a small server in. Even a dual core AMD with 8 GB of RAM will do the job.

I've built a i3-3220T + DQ77KB in an akasa euler thin-itx fanless case, it is rated to 35W TDP cpus. Dual intel NIC onboard, with some DIY on the backplate you could put another one or two GbE ports from a minicard.

I believe it can do more depending on environment and the very conservative TDP intel uses on recent cpus, so if you really need something faster the 45W quad core E3 1265Lv2 is quite powerful though not cheap.

So far I have not seen a haswell thin-itx as it would be nice to get AES-NI at i3 prices, and the intel DQ77KB is hard to find at a fair price. (also a caveat about certain NICs not being supported properly, 82576 = no, i350 = single channel ram issue)

There are also some passive cases coming out for NUCs, if you can figure out a way to have two physical ports* with a dongle NIC that doesn't suck those could be quite nice.
(*without doing the Very Bad Idea of using only VLANs on a switch to separate WAN from LAN)

asterix

Just go for a simple i3 with 4GB RAM config. You should be good for years. Check out the latest Haswell processor and motherboard compatibility (If I am not mistaken, its on these forums somewhere.. folks have done it). Stick a passive heat sink on it and it should be quiet enough.

@Jason:

@gonzopancho:

@Jason:

300/300 is going to require something faster than an Atom, even just for firewall+NAT.

Not really, no.

In my experience, no Atom, including the D2800 I have at home with i350 NICs, can FW+NAT more than 500-600Mbit/s. I haven't tried the current gen, but that has more to do with the fact that none of the NICs will work under FreeBSD 8.3. I suspect they'd be able to push 1Gbit/s or more through FW+NAT given the strong increase in IPC the reviews are reporting but we may not be able to actually test with pfSense until the FreeBSD 10 move.

You seem to have missed the part where we have several of these in-house now.

jasonlitka

@gonzopancho:

@Jason:

@gonzopancho:

@Jason:

300/300 is going to require something faster than an Atom, even just for firewall+NAT.

Not really, no.

In my experience, no Atom, including the D2800 I have at home with i350 NICs, can FW+NAT more than 500-600Mbit/s. I haven't tried the current gen, but that has more to do with the fact that none of the NICs will work under FreeBSD 8.3. I suspect they'd be able to push 1Gbit/s or more through FW+NAT given the strong increase in IPC the reviews are reporting but we may not be able to actually test with pfSense until the FreeBSD 10 move.

You seem to have missed the part where we have several of these in-house now.

Several of what? I don't know what you're talking about when you reference "these". I just skimmed the thread again and aside from some benchmark numbers you posted for an older Atom, I don't see you mentioning having anything specific in-house. If you've got one of the new 8-core atoms working, NICs and all, please post up some numbers; I'd love to see them.

bplein

@captainkirkw:

If you don't mind me asking, what part of Austin are you in? I live in the Milwood area in Rattan Creek in MW and have not been able to get a straight answer from AT&T about GigaPower. I got Uverse as soon as it was available in my neighborhood and have been happy with it but with the news of Google Fiber and then GigaPower right after that, I wondered how long it would take.

I am in Leander, in Crystal Falls. New development with Fiber to the Home.

bplein

@timthetortoise:

Honestly your requirements are a little bit ridiculous. You want a fanless passively cooled system that will handle 300+ Mb/s throughput? You can't even do that with ASIC devices. Find a closet you can put a small server in. Even a dual core AMD with 8 GB of RAM will do the job.

Honestly your response is a little bit ridiculous.

I asked a question because I don't know these things. Where is the secret decoder on how many MIPs it takes per Megabit of throughput? If you point it out to me, I'll do my own calculations. Besides, there are many types of passively cooled systems that are very high power. They just get cost prohibitive.

bplein

@gonzopancho:

You are aware that ESF is located in Austin, right?

Chris lives here (currently on AT&T at home). I live here (currently on Grande at home, AT&T FTTH in my neighborhood.)
The office is next door to what used to be CoreNAP (now zColo), and we run a 10Gbps fiber between our rack there and the office.

Gimme moar bandwidth. ;D ;D ;D

Wow, didn't realize that! I relocated to the Austin area in 2012 as well (just picked that up from your web site)

bplein

I may have solved my challenge for now.

I have installed pfsense in a VM on one of my ESXi hosts. I have set it up with CARP to fail over to my ALIX. Complicated due to use of VLANs to pipe the various networks between the network panel in my closet to my office where the VMware cluster lives, but it works.

If it works well and isn't flakey, I'll keep this setup and the ALIX will just be my slow speed fallback when my VM is down.

Altheran

I will just leave this here … It might a very awesome build.

GA-Q87TN : Thin Mini-ITX Motherboard
Core i5-4570 : 84W TDP (Couldnt fing the 4570T 35W ...)
4GB Ram
30 GB mSata SSD
Power Adapter
Lian-Li PC-Q05B Thin Mini-ITX Case

http://www.newegg.com/Product/Product.aspx?Item=N82E16813128670&Tpk=GA-Q87TN
http://www.newegg.com/Product/Product.aspx?Item=N82E16819116896&Tpk=Core%20i5-4570 (couldnt find the i5-4570T : 35W TDP)
http://www.newegg.com/Product/Product.aspx?Item=N82E16820231470
http://www.newegg.com/Product/Product.aspx?Item=9SIA2W014A9300
http://www.mini-box.com/19v-8-4A-160-Watt-AC-DC-Power-Adapter
http://www.newegg.com/Product/Product.aspx?Item=N82E16811112357&Tpk=PC-Q05b

Altheran

@timthetortoise:

Honestly your requirements are a little bit ridiculous. You want a fanless passively cooled system that will handle 300+ Mb/s throughput? You can't even do that with ASIC devices. Find a closet you can put a small server in. Even a dual core AMD with 8 GB of RAM will do the job.

Nope you can : Ubiquity Edge Router Lite : 99$ 1.3 Gbps Simultaneous WAN/LAN, LAN/WAN Throuput

dreamslacker

@Altheran:

I will just leave this here … It might a very awesome build.

GA-Q87TN : Thin Mini-ITX Motherboard
Core i5-4570 : 84W TDP (Couldnt fing the 4570T 35W ...)
4GB Ram
30 GB mSata SSD
Power Adapter
Lian-Li PC-Q05B Thin Mini-ITX Case

That board might not work out (at the moment). Most of the Haswell boards come with 8111G and/ or i217 NICs - neither are supported out-of-box in pfSense 2.1.

The Pentium G3220 (54W SDP) will more than suffice for >1Gbps NAT throughput and >100Mbps AES-256 VPN throughput. In fact, you should be able to squeeze out 200Mbps of VPN throughput from it, albeit at increased power consumption since it doesn't have AES-NI ASIC to offload the processor.

bplein

@Altheran:

@timthetortoise:

Honestly your requirements are a little bit ridiculous. You want a fanless passively cooled system that will handle 300+ Mb/s throughput? You can't even do that with ASIC devices. Find a closet you can put a small server in. Even a dual core AMD with 8 GB of RAM will do the job.

Nope you can : Ubiquity Edge Router Lite : 99$ 1.3 Gbps Simultaneous WAN/LAN, LAN/WAN Throuput

I love pfSense, but that seems like the 'easy button'.

I need to see what features will be coming with the new gateway device that Uverse uses for Gigapower. I'd love to ditch it and use my own (tried it with pfSense and it works) but I cannot find a way to put the RG they supply behind my router in order to have phones work (our Uverse is also for phone). If it was as simple as port forwarding and I had an exhaustive list of ports, I could do it.

@Altheran:

Nope you can : Ubiquity Edge Router Lite : 99$ 1.3 Gbps Simultaneous WAN/LAN, LAN/WAN Throuput

Has anyone verified this after Ubiquiti was forced to stop using the Vyatta 'secret' codebase?

Or is this just the "Tolly" report outcome?

bplein

For the last month or so I've run pfSense in a VM in order to take advantage of my 300Mbit symmetrical service. The added complexity of using VLANs at my closet to pipe the WAN over to my ESXi hosts was a bit excessive, so I finally bit the bullet and bought some dedicated hardware.

I went with an Intel DC2500CCE dual-core Atom motherboard with 2 onboard Intel GigE nics http://www.amazon.com/gp/product/B007MS9OI2/ref=oh_details_o00_s00_i00?ie=UTF8&psc=1, as well as this case http://www.amazon.com/gp/product/B005PI1Z7O/ref=oh_details_o00_s00_i01?ie=UTF8&psc=1

The motherboard is fanless, as is the case. I used some old laptop RAM I had lying around as well as an old laptop HDD. I will eventually put an SSD inside just to make it even more quiet.

I can run speed tests pushing the full 300Mbit in either direction, and the pfSense dashboard indicates it's pulling 42-45% CPU during that peak.

So, does this imply that I should be able to get about 600 Mbit through it when AT&T increases our speed? Surely I'll have a tough time using that kind of speed, but just trying to get a feel for what my new hardware can do.

Finally, are there any tips to getting lower CPU utilization out of it? Thanks in advance!

jasonlitka

@bplein:

300Mbit symmetrical service

I went with an Intel DC2500CCE dual-core Atom

I can run speed tests pushing the full 300Mbit in either direction, and the pfSense dashboard indicates it's pulling 42-45% CPU during that peak.

So, does this imply that I should be able to get about 600 Mbit through it when AT&T increases our speed?

That Atom will top out somewhere between 500-600Mbit/s TOTAL. That means that if you have 300/300 service you can max it out in both directions, or if you had 600/600 you could (come close to) max it out in one direction. This assumes that all you are doing is firewall & NAT. If you add in packages, traffic shaping, etc. then you will fall way short.

@bplein:

Finally, are there any tips to getting lower CPU utilization out of it? Thanks in advance!

Buy better hardware or downgrade your connection. That's about it.

razzfazz

Is simultaneous 100% utilization in and out really a realistic use case? I think as long as he can max out each direction individually, the OP can sleep soundly knowing he couldn't have gotten the same performance from a lower bandwidth tier (which seems to be the objective here).

stephenw10

That Atom doesn't do hyper-threading right? So you see 2 logical cores?
Depending on how you are measuring the CPU usage you might be closer to the limit than you think. The PF process is (currently) limited to a single thread so it can only use one core. If that core is at 80% usage then the total CPU usage, shown on the dashboard, might only show 45%. Check with 'top -SH' at the console to see how the cores are being loaded.
Have a read through this thread: https://forum.pfsense.org/index.php/topic,67411.0.html
User maverick_slo eventually managed to get 640Mbps from his D2500CC after some tuning. I can't remeber if that was between the two on board NICs or not. (probably was though since additional NICs would have to be PCI)

Steve

bplein

@Jason:

That Atom will top out somewhere between 500-600Mbit/s TOTAL. That means that if you have 300/300 service you can max it out in both directions, or if you had 600/600 you could (come close to) max it out in one direction. This assumes that all you are doing is firewall & NAT. If you add in packages, traffic shaping, etc. then you will fall way short.

Didn't you say earlier that "300/300 is going to require something faster than an Atom, even just for firewall+NAT."?

I'm really just asking if my estimations are based correctly: that CPU for throughput (assuming no other apps are running) is linear.

@bplein:

Finally, are there any tips to getting lower CPU utilization out of it? Thanks in advance!

@Jason:

Buy better hardware or downgrade your connection. That's about it.

Now that's a non-answer! I didn't say "how can I get better throughput". I asked if pfSense had any optimizations for reducing CPU utilization.

My guess is that you hate Atom. You were the first to answer incorrectly in this thread, and now you are hopping in recommending that I get "better hardware".

Please do tell me what hardware is better for my needs? Right now I can do 300/300 (unidirectional at least) with a fanless solution I can keep in my master bedroom closet. If you have another better, faster, still fanless solution that meets my requirements, please do chime in.

I really do appreciate valuable input. I just don't see how repeatedly claiming that Atom won't do it is adding value. It's already doing it…

bplein

@razzfazz:

Is simultaneous 100% utilization in and out really a realistic use case? I think as long as he can max out each direction individually, the OP can sleep soundly knowing he couldn't have gotten the same performance from a lower bandwidth tier (which seems to be the objective here).

I am not even using the full capabilities of my 300/300 today! Oh, I can speedtest it unidirectionally but realistically the line is underutilized almost all the time, but my wife and I both enjoy the rapid upload of large digital documents to our employers. It's helps us immensely. I probably wouldn't notice if it was 100/100 except for a few types of downloads and uploads that we do.

My goal is to make sure my current router is as optimized as possible, so that when I do hit a peak (especially after the free upgrade to Gigabit later this year), I am getting the most of my link. If I can get 600 one way, I'll be happy with that. If I could stretch it to 700 via optimizations, that's just smart router management!

bplein

@stephenw10:

That Atom doesn't do hyper-threading right? So you see 2 logical cores?
Depending on how you are measuring the CPU usage you might be closer to the limit than you think. The PF process is (currently) limited to a single thread so it can only use one core. If that core is at 80% usage then the total CPU usage, shown on the dashboard, might only show 45%. Check with 'top -SH' at the console to see how the cores are being loaded.
Have a read through this thread: https://forum.pfsense.org/index.php/topic,67411.0.html
User maverick_slo eventually managed to get 640Mbps from his D2500CC after some tuning. I can't remeber if that was between the two on board NICs or not. (probably was though since additional NICs would have to be PCI)

Steve

Steve,

Thanks for the thoughtful reply.

It has two cores, pfSense sees it as two cores (not hyperthreaded). It appears from the "top -SH" command that the CPU is being utilized by the interrupts, and all are showing up on CPU 0. During these test runs, I ignored the WebUI and only looked at "top", and it looks like you are correct, maybe 85% (of 200%) are eaten up by the "intr" command and all are running in CPU0. If I extrapolate that, and if it can't be distributed across cores (maybe it makes no sense to do so), then I am truly near topped out.

I'll check out that other thread. Thanks a bunch.