Digression or not, I appreciate the extra info and feedback!  Definitely relevant to my interests.

I probably won't have a server anywhere close to yours,  I can justify a lot of things (see: previous thread entries where I went somehow from APU2C4 to C2778), but that is a truly massive amount of storage.  12 spinning discs and 9 SSDs, wow!  I'll have only 5 spinning discs and a boot SSD. I'm just going to go from 5x2TB to 5x4TB and pick up transcoding ability and general purpose server functionality in the process.  I already have link aggregation, so I'll be keeping that, too.

Thanks for taking the time to help educate me.  I'll definitely be referring back to my notes from this thread when it is time to build my new NAS.

Big thanks for that information, appreciated.

Just ordered SG300-20.

@jahonix:

That ovpn.se hardware is great - the CPU doesn't even have AES-NI support which surely makes it an outstanding dedicated VPN device…
http://ark.intel.com/de/products/71995/Intel-Celeron-Processor-1037U-2M-Cache-1_80-GHz
And antenna-placement (right next to each other) will improve wireless diversity to the max.

Performance wise it does pretty ok. Check https://wiki.openwrt.org/doc/howto/benchmark.openssl

And regarding the antennas, you are absolutely correct. However I use a wireless AP so doesn't bother me.

There's an mbuf leak under some circumstance in the Chelsio driver. Almost certain that's what you're running into given the description. It's been fixed in newer, not yet released OS versions. The workaround in the mean time, put the following in /boot/loader.conf.local

hw.cxgbe.allow_mbufs_in_cluster=0

For the same ethernet chipset, onboard might have different performance than the add-on one.
There is another thread in this board (the J1900 w/4 LAN port) which shows something we might easily missed during selection.

The PCI-e bandwidth sharing can be a problem for LAN chip, some consumer grade device manufacturer might be putting quite a number of devices together on the same lane, somehow for certain low end processors they might be able to provide very limited PCI-e lanes which is not capable to have onboard LAN running at full speed all the time. But for a LAN card occuping separate PCI-e slot, most of the time this is using a dedicated PCI-e lane and so the performance can be guaranteed. Of course you don't need to worry about i3/i5/i7/xeon platform as these CPUs provide plenty of lanes for us while some low end ATOM processors might probably have this issue.

@lra:

@BlueKobold:

I would not expect any more than 40 Mbps for a single OpenVPN connection.

The APU2 comes with 4 Core CPU and only the PPPoE WAN part is single core using, the entire
OpenVPN part is fully multi CPU core usage and so you will see perhaps numbers owed to this
circumstance that you was not expecting before. But I would be counting more on the AES-NI
and IPSec (AES-GCM) that should be more pushing the entire VPN part, for sure not OpenVPN
but really fast.

I just tested my APU2, (on Linux in my test), disabled lzo-compression, "cipher AES-256-CBC" and consistently saw 58-62 Mbps using iperf.  Note iperf was not running on the APU2, and the APU2 was an OpenVPN server.

My version of iperf did not support randomized data, so I had to disable lzo-compression for a closer real-world test.

@BlueKobold, looking at "htop" on the APU2, it seemed only one core was running at 50-100% during the test.

Update,

I retested on the APU2 running iperf3 (client) on the APU2 itself, while the remote end iperf3 (server) bound to the tunnel IP of the OpenVPN client, the result was 92 Mbps.

It seems testing downstream off an external interface made the test somewhat "choppy" so a consistent, solid stream did not happen (a short pause every few seconds) and hence slower throughput.

Yes, once you stop the Host management OS from using the vSwitch, the Hyper-V host/ machine no longer has access to that pNIC. That's what you want in order to stop the host from hijacking the IP; and more importantly, to prevent it from being directly exposed to the internet.

Thank you I noticed the string was clipped when I pasted it.

@el-quique:

It works on e3131 . Thank you

The only thing to work I had to modify the file lte.cfg

from
MessageContent="55534243123456780000000000000a1106200000000000010000000000000$

to
MessageContent="55534243123456780000000000000a11062000000000000100000000000000"

:)

@willrc627:

Another update, FreeBSD 9.2 recognizes all 12 interfaces and each one is capable of IP assignment and ping/connectivity. Is there a version of Pfsense that runs FreeBSD 9.2 because that seems like the next step in this process. Either that or somehow trying 9.2 drivers on 10.3.

So, that would take you to a very old version of pfSense.. not something you'd want to do.

My guess (and just a guess) is that nobody is actually real-world testing the cas driver between releases.  If it compiles, it probably ships, because nobody has the hardware or inclination to test the driver much anymore.

As cmb said, the hardware really wasn't that good anyway.  I was never impressed with the Cassini cards - the Intel cards will run rings around them.  (Get it?  A lame attempt at an astronomy joke.  Probably sounded funnier in my head.)

Any who…  I really wouldn't waste too much time on the Sun cards, to be honest.  So in the interest of saving you a few bucks, let me ask this... why do you need 12x interfaces for pfSense?  Could you do something with a single quad-port Intel card for $20 instead of having 3 separate cards?  There's usually more than one way to skin a cat...

The router is dual band, but only the 5 GHz band works. The 2.4GHz band won't connect to the pfSense box. Did I do something wrong when I set it up?

Create two SSIDs and then please put each in a own VLAN in pfSense and create a own DHCP for each network
with his own IP range such like this;

2,4GHz - SSID - private24 - 192.168.1.0/24 5,0GHz - SSID - private50 - 192.168.2.0/24 2,4GHz - SSID - guest24 - 192.168.3.0/24 5,0GHz - SSID - guest50 - 192.168.4.0/24

Activate client isolation and set them up as running in AP mode in pfSense.

@davids355:

OK, noted…. Sort of :-/ I think I remember being told I could have all of my hosts on a VLAN. So I guess in this scenario I would only need one cable connected to the server running pfsense and the hyperv vswitch could do the rest.

And so in the same respect, if I had a hardware firewall it would be the same - If I had a physical switch in place I would only need one connection to the firewall itself, and the switch could do the rest? Even if I had VLANS set up?

But from the comments made so far I am guessing that a hardware firewall is going overboard?

For redundancy of my other VMs I am planning to impliment a live copy of VMs between multiple servers so I could impliment something similar for the pfsense VM to avoid any issues if the host running that VM went down.?

You can have all the hosts on the same vlan but they will be able to communicate with each other.

You can do the same with pfSense on physical hardware as long as you have the vlans setup on the physical switch port(s).

For redundancy, you will need to setup Hyper-V for HA clustering. At least 1 or 2 other vlans need to be setup just for the migration purpose. Also, you will need to setup a DFS fileshare for both nodes to act as a witness (for 2 node replication) and also for a 'common' repository for the snapshots.

I understand that it is possible to use just one NIC and a smart switch and I do have a TP-Link TL-SG2008, but have never set up a VLAN, although I guess that it something I should learn to do.

I have pfSense set up as a DHCP server and most clients are set up for dynamic IP. My first hurdle is accessing the switch which is on a different subnet. Should I enable DHCP on the switch?

Thanks cmb,

seems like this board is obsolete and no bios updates are available..

guesss i'll start looking for new hardware…

…..HP ProLiant SE316M1.....

Some of the HP Proliant servers, including many of the HP Proliant Micro server series are having
problems together with pfSense bare metal installations, pending on chip sets and/or the BIOS.

Thank you for your kind suggestions, I wanted to look at options and looking at a case and power supply it pushed up price.

I would like to show you my solution has intel gigabit nic with teaming and designed for 24/7 use

http://www.amazon.co.uk/gp/product/B01CS4ZX9S/ref=oh_aui_detailpage_o00_s00?ie=UTF8&psc=1

http://global.shuttle.com/news/productsDetail?productId=2007

Can't wait to receive my ordered parts

Intel Pentium Dual-Core G4520 3.6 GHz Processor CPU as it should be plenty in excess of what is needed- going to look at various options as internet appliance

what's the firmware version on the modem?

That's a bit of an odd place to crash. Usually with a somewhat random crash like that it turns out to be a more general problem like having both pfsync and limiters enabled at the same time – https://redmine.pfsense.org/issues/4310

@bennyc:

That looks tiny (size) for a firewall, I like it  8)
Any idea what NAT throughput is to be expected?

Quoting throughput needs to wait until we don't have full debugging on.

thanks seems to work fine as I’m posting this message to the forum with the PC connected to it with no issue