Thanks for all the great replies.  It seems I need to get a Xeon E3-12xx system for max performance and minimal power usage.

As an aside, thanks to "stephenw10" for pointing out an issue with my existing IPSec configuration.  After switching from Blowfish to AES128-GCM on the connection ciphers, the connection speed went from 7MB/sec to ~ 11MB/sec with 50% CPU usage (50% usage on a single core on a 4-core system).  This means my existing box might be strong enough to handle much more IPSec traffic than I initially thought.

The only side-affect I see now is high interrupts (120% and higher) on "hpet0".  Not sure if this is an IPSec issue or a hardware issue.

Use the console?

Awesome ok. We aren't all gunna be pulling in 100mbs each obviously so it should work temporarily at least until I get a little system built. Thanks!

Reset Pfsense 2.3.2 p1 to factory defaults and no change in behavoir - firewall still can't check for updates etc.

Reimaged my SSD to a copy of my install just before I upgraded from Pfsense 2.2.4 to 2.3.2 and then tried it.  The SSD worked perfectly, no problems at all !!

Conclusion:  A number of other people on the forums reported the same symptom of "can't check for updates" after upgrading to 2.3.2 and so I'm concluding two things:

1. Soekris has made some change to the hardware or bios for the 6501-50 board they sent me a year ago vs. the 6501-50 board they sent me last month. The two boards are not identical and this change is enough that the same SSD (with Pfsense 2.3.2 p1) works perfectly on the 1 yr old board but when plugged into the new board does not work properly.

2. This is likely a glitch/bug with Pfsense 2.3.2 since I can't see why it should not work with hardware that worked perfectly with version 2.2.4.

If anyone wants to try and track down the glitch I'm happy to provide whatever information I can as this might be an excellent opportunity to try and figure out what the issue in 2.3.2 is.  I'm using the exact same SSD and have two boards that are supposed to be identical, but obviously must have some minor difference, which should point to exactly where the glitch is and how to resolve it ?

Regards

Nice catch!  :)

Not at all obvious I'll have to remember that one. Did you not receive a console cable with the device?

Steve

We just did some tests with a Jetway nf9n-2930 board together with a ADE4INLANG 4x Intel 82574L plugin-card.

What tests did you? And how did you exactly this tests? Which pfSense version is in the game here? And is this a fresh and full
installation or a NanoBSD on USB pen drive installation? Are any packets such as Snort, Squid or pfBlockerNG installed too?

But we are now running into an issue with pfSense and detecting the link-state of the 4 network interfaces of the plugin-card.

The NIC is not a real NIC it is a daughter board from Jetway! And often the switch chip (PHY) behind the LAN ports is important fo
us to know. It can be that this PHY chip is not really supported by pfSense but the ordinary Intel Chip is well supported! Jetway often
uses PHY chips from Pericom as a hind for you to start searching.

But we are now running into an issue with pfSense and detecting the link-state of the 4 network interfaces of the plugin-card.

Are they well running and working?

We tried version 2.3 and 2.4-beta, but similar behavior on both.
The EM-driver is used (em0-em3).

For the Intel LAN chipset 82574L  the em driver will be the right one! It is likes all other OS and drivers you can´t choose
a driver for a device it must be written exactly for that device.

So pfs is detecting the link-status "UP" event once after plugging in a active cable, but after that the link-state always stays "UP".

Even if the cable will be plugged out?

The link led's of the network interface itself work just fine, but the link up/down events don't come trough into pfs after that initial UP-event.

This can be different here in that case because the NIC is not a plugin card, it is a Jetway daughter board that is connected in
an other way such a ordinary NIC will be.

Does anyone has seen this behavior before?

Please trust me this behaviour is based or pointed to the daughterboard and its PHY chip on it and nothing more.
It is the same as with that Jetway board here: NF9HG-2930 the board itself is running like hell, but the both
daughterboards are not supported based on the soldered PHY chip from Pericom on the both boards!

If someone will be on the safe side, to gets success and clean 10 GBit/s routed without any trouble she/he should be using a Layer3
Switch and if switching only is needed a Layer2+ switch would be the top of that roof and fairly the best bet at this time as I see it
right now. There are some nice and cheaper models from Netgear that are nice playing together in networking.
Netgear XS708T, Netgear XS712T, Netgear XS716T, Netgear XS724T, Netgear XS748T
Netgear M4300 series

I have had the same issue..
https://forum.pfsense.org/index.php?topic=84909.msg466364#msg466364
read this .. I found fix..

@Demnos:

After a lot of research and finding consensus on other forums, I decided to look into doing an Intel-based build using a core i3 CPU.  My budget is still $300, so whether or not I do this will depend on how far over budget it goes. These components look like they might be possible:

CPU~
Intel core i3-4150
core i3-4160
core i3-4170

MOTHERBOARDS~
Gigabyte GA-B85M-DS3H-A
Asus H81M-C/CSM
H97M-E/CSM

SSDs~
Intel 530 series 120GB
535 series 120GB

RAM~
Not researched it, but I definitely want 8GB; either G.Skill or Crucial.

DVD-ROM~
I require either an internal or external DVD-ROM. I guess if an M-ATX case is chosen, that requires an external (USB) optical drive, as I see no microATX cases that allow an internal drive.

CPU COOLER~
Has to be quiet, not block RAM or PCIe slots, and fit the case.

NIC~
I have an Intel PRO/1000 PT in my parts bank.

POWER SUPPLY~
Has to be quiet, and have good buyer reviews for reliability.

CASE~
I'd prefer micro-ATX but considering the router will be about six feet from my bed, soundproofing the case may be necessary…maybe the next size bigger than microATX?

So anyone have ideas for what to buy, and stay in budget?

I'd say I would follow the suggestions being given in this thread ,  well #1 you don't need a full fledged PC unless you're routing 10gb+ or a VPN or in a virtualized environment but pfsense doesn't really require a lot resources to achieve what you're asking for the most part pfsense can do it with minimal hardware. you did say your budget was around $300  so what you're doing is essentially taking  matx pc and telling it to be a router most 2nd hand PC can do this  but I would limit the size to something  smaller  MITX boards or SFF

This would be the better option
@BlueKobold:

http://jetwaycomputer.com/NF9HG.html Jetway NF9HG-2930
Pros:
fan less
max. 8 GB RAM
slim design board
4 Core CPU @2,16GHz
OnBoard 4 Intel based LAN GB Ports
PSU direct into the board from outside
2 x miniPCIe (mSATA & WIF or Modem)

Cons:
Only 2 USB Ports but one USB 3.0 Port
With PPPoE not really 1 GBit/s at the WAN
only ~650 MBit/s at the WAN

or option #2  https://www.mitxpc.com/proddetail.php?prod=JBC320U93W-2930-B

If it shows as a ugen, that means there is no driver for it. Try a 2.4 snapshot.

Hi, just wanted to confirm that the Xeon Ds are one of the best surprises for me this year. And if there is something created for pfSense, those are it. Santa (Merry Christmas!) brought a X10SDV-6C+-TLN4F system with the Xeon D 1528 (it is the one with active cooling - the CPU should be fine with only a heatsink, but it also benefits the chips surrounding it too), 2x8GB DDR4 Reg. ECC, 1U rack case 200W PSU, and a SSD SATA DOM 64GB. My power usage dropped with a whopping 140W compared to the HP server I was using! And the CPU barely hits 8% load.

I have a problem with the onboard 10Gb NICs, but I will open a separate thread. Thank you for your thoughts on the matter, this platform really worth the money (all the hardware was priced at 1150 EUROs).

15697921_1347832815247612_7694541246963020289_n.jpg
15697921_1347832815247612_7694541246963020289_n.jpg_thumb
15698179_1347832698580957_4467500990110598914_n.jpg
15698179_1347832698580957_4467500990110598914_n.jpg_thumb

Firewalling and providing reliable Wi-Fi access at scale are two completely separate things.

@xxxGODxxx:

@VAMike:

have you considered just direct-connecting the two 10G devices?

I would if I could, but my gaming pc is in my room while the nas is in another room, and there is only 1 ethernet port (cabled with cat6a cables so it can scale up to 10gbps) linking the two rooms - I am unable to add another ethernet port linking the 2 rooms cause it will require me to do some major renovation works. If I were to direct connect I would not have any internet access on my computer and would not have another ethernet port to provide the internet connection.

Rather than throwing another machine into the mix, I'd run the gaming PC to the NAS, use the NAS as the bridge. You'll get whatever bandwidth to the NAS that it's capable of supporting, and the traffic going through the NAS to the rest of the LAN/WAN is basically negligable.

I think it will be a pretty simple setup. We have two VLANs (in/out) and two physical 2 * 10GE fibre.

You will be able to use then a Chelsio card that is fully offloading the NAT part.

Authentication is not needed at the firewall as we intent to go with Option82 DHCP (DHCP Server will be apart).  There is no  need for VPN, LDAP or CaptivePortal.

Ok, that would it make more simple.

If we have say 4.000 users online - all with some sessions established the box needs to keep all the NAT states. I am just not sure if pfSense is the right product  and if its okay to go with a general purpose CPU with standard server hardware for that amount of users (throughput)

DHCP and DNS entries must also be stored for caching them too, there will be not limitations only the hardware
is setting up the highest level, from the side of pfSense you may get not be pressed down!

or if it would be better to go with a real firewall vendor using ASICs or something.

If only firewall rules SPI (netfilter) and NAT is needed pfSense would do that job with ease, only
to find the real matching hardware would be here the problem in my eyes.

The $ delta seems to be huge in favor for pfsense!

Money is not all, if the network must be running really 24/4/365 and also a HA set up might be the
best bet to give a guarantee that all is well.

The pfsense hardware requirement guide goes only up until 500Mbps (https://www.pfsense.org/hardware/).

Not really, there is written something to archive "over" 500 MBit/s that means more then 500 MBit/s or above that
you will be need - 501+ Mbps Multiple cores at > 2.0GHz are required. Server class hardware with PCI-e network adapters
2 x Xeon E5-26xxv3/v4 and 32 GB RAM or more will be your choice and way to go with as I see it right. Or a self made
Supermicro Xeon D-15x8 platform should be more then enough.

@stephenw10:

I would expect to see 1Gbps firewall and NAT throughput using any of those CPUs. Though it does depend on your traffic type. If you are passing all VoIP with tiny packets you might struggle.

It may struggle with small packets? My only experience is with my home PFSense with Haswell i5 3.2ghz + Intel i-350. A few weeks back I finally got iperf working correctly on my Windows desktop and was able to almost send 1.4Mpps of UDP. Almost 70% kernel time, it was struggling to reach line-rate, but got very close.

I found a public iperf UDP server, set PFSense to shape to 1Gb/s instead of my normal 150Mb/s, pointed at it and let it rip. PFSense was claiming about 1.4Mpps hitting the LAN interface and about 1.4Mpps leaving the WAN. This was through NAT and with HFSC still enabled, just set to 1Gb/s. To top it off, the system load graph was claiming about 15% system time and just under 20% total CPU. The graph is averaged to 1min, so I had the iperf test run for 2min to make sure I got a full minute sample.

Of course the iperf results were as expected with around 85% packetloss. That happens when you attempt to shove 1Gb of traffic down a 150Mb link.

For the benefit of the thread I worked it out, in retrospect it made perfect sense, but there ya go.

So I setup an internal hyper-v network, which only the pfsense VM can access. Then I connected the android phone and shared it with the new LAN via internet sharing, this then let Windows create a DHCP server that pfsense could grab an IP from.

Phew!

@BlueKobold:

I'll take your advice and just use the mSATA w/out the onboard eMMC. I'm not in the mood to run custom scripts for this device….

Please create an account and Register your device at the store and then you could download the ADI pfsense Image and install
that on your mSATA. It is not the community Edition (CE) it is a custom Image that matches all the given hardware from that
ADI device such as yours. So you will be sure that Tunings and all pimping will be right done for you regarding that Hardware!

Not sure which version you are talking g about. I just downloaded and installed the netgate Adi community ity version off pfsenses website.

Hello again

Thanks for the input, I have some ideas to work with now, it seems my initial idea of setting up squid probably isn't the best solution.

I asked around to see if I could figure out what was using up the montly bandwidth and it seems at least one of the family members is very fond of torrents, not only downloading but also seeding.
This is probably a pretty bad idea when you have a monthly limit on your bandwidth.
He's not particularly interested in stopping his activity and we talked about setting up a dedicated connection for torrents.

The connection will be alot slower (2-5 mbit *DSL) but it will have unlimited usage.

@VAMike:

@darkarn:

@VAMike:

@Taiidan:

Benchmark a "gigabit" realtek or broadcom you get at best 70MB/s

That's simply not true, so the rest can be safely ignored.

Hmm I don't know; I just noticed that an old integrated Atheros NIC can be easily beaten by an Intel NIC on a PCIe card in transferring stuff to and back from a NAS

I can't sustain a gigabit on my old 3c905 either, which has exactly zero relevance to whether no current realtek or broadcom chipset can achieve more than 70MB/s. That claim is easily disproven and utter nonsense. (Just as ridiculous is continuing the meme that every "realtek" is the same any more than every "intel" is the same. If someone wants to talk about NICs at the very least specify a chipset.)

That 3c905 reminds me of an old Realtek PCI NIC I saw in one of my friends' PCs!

@dreamslacker:

@darkarn:

1. I have no choice though; had to put my AC66U in center part of the house for proper coverage but not allowed to do Ethernet drop

2. That's why I looking around, especially when there are much powerful CPUs for much lower power consumption

3. Thanks for the tip, I will try this out

4. Whoa, thanks, I will keep a look out for this issue too

5. Sorry, what's an SI?

Wife/ Parents acceptance factor? If so, tough luck.

I'd just go for the Core i3 Skylake in a Mini-ITX and add on an Intel PCI-e network adapter.

Systems Integrator. Except in my case, we do practically everything with the sole exception of programming. The running joke has been that if it runs on electricity, we can do it or find someone to do it. Even had a case where we sold and installed replacement batteries for our customer's van.

1. Yep, my parents lol

2. I have actually specced up two different i3 builds but using micro-ATX instead. I may want to wait until next month due to Kaby Lake though

5. Ah I see, and whoa, I didn't know it's possible for an IT company to do auto repair work too lol