First pfSense box - Xeon build
-
I'm liking the idea of that jetway 8 port mobo for 6th gen processors
Should I go for an i3 6100T or… -
From what i have been reading, it seems that if you plan on running snort, few cores with higher Ghz is preferable to more cores chips.
Also good, if i understood it right, for traffic consisting mainly of small packets, inspected by snort.Suricata is multi-threading, Snort will it be in some time as I know it, or it is perhaps also until now happened that it is multi-threading
so many things in pfSense will be at a change at this time. OpenVPN is also multi-threaded since 2.3 and so it might be a game changer
to own a CPU with more CPU cores, but actual you will be then not getting much profit out, and so it might be the best to be also future
proof, to get a strong and powerful CPU with a higher CPU frequency (GHz) paired together with some more CPU cores too!So you will be getting out now and in the future the best results! And with an looking eyes on power saving options it might be the best
to get an Intel Xeon E3 with 4 Cores and 8 Threads (HT). Actual one of the best options as I see it right, other might see it different and
for sure a cheaper Intel Core i3 could be also an interesting platform. And if AES-NI is also on board it might be a long time running box.I'm liking the idea of that jetway 8 port mobo for 6th gen processors
Should I go for an i3 6100T or…Would be a stronger system together with AES-NI and more GHz as I see it right and so the 1 GBit/s will be even reachable for you.
I did verify its supported . via the Intel web page. For 130 dollars I can put the 1220L to work and see how it runs. Can't hurt I guess. I have never been apposed to building multiple systems so I will for sure try another with the v3 of the 1220L
Ok if the 1220L will work on that board I would suggest to give them a try, for sure this makes sense for me.
- Intel DP77KB
Now only the right matching case will be the question - mSATA 32 GB or 60 GB or 120 GB (transcend)
16GB for plain install, 32GB for Snort, 60GB for Snort & Squid and 120GB for more users, services, HotSpot, Squid, Snort,….. - 2 x 4 GB RAM (fastest as the board will be supporting please) (DDR3-1600 in your case)
Often the CPU is not saturated but the memory system is! And with 8 GB you will be able to high up the mbuf size to 1000000 - Intel PRO/1000 PT refurbished or a refurbished Intel i350-T4 would my personal choice here
Often able to get for ~$50 or ~$120 so the pfSense Shop NIC in low profile format could also be interesting
- Intel DP77KB
-
@BlueKobold:
From what i have been reading, it seems that if you plan on running snort, few cores with higher Ghz is preferable to more cores chips.
Also good, if i understood it right, for traffic consisting mainly of small packets, inspected by snort.Suricata is multi-threading, Snort will it be in some time as I know it, or it is perhaps also until now happened that it is multi-threading
so many things in pfSense will be at a change at this time. OpenVPN is also multi-threaded since 2.3 and so it might be a game changer
to own a CPU with more CPU cores, but actual you will be then not getting much profit out, and so it might be the best to be also future
proof, to get a strong and powerful CPU with a higher CPU frequency (GHz) paired together with some more CPU cores too!So you will be getting out now and in the future the best results! And with an looking eyes on power saving options it might be the best
to get an Intel Xeon E3 with 4 Cores and 8 Threads (HT). Actual one of the best options as I see it right, other might see it different and
for sure a cheaper Intel Core i3 could be also an interesting platform. And if AES-NI is also on board it might be a long time running box.I'm liking the idea of that jetway 8 port mobo for 6th gen processors
Should I go for an i3 6100T or…Would be a stronger system together with AES-NI and more GHz as I see it right and so the 1 GBit/s will be even reachable for you.
I did verify its supported . via the Intel web page. For 130 dollars I can put the 1220L to work and see how it runs. Can't hurt I guess. I have never been apposed to building multiple systems so I will for sure try another with the v3 of the 1220L
Ok if the 1220L will work on that board I would suggest to give them a try, for sure this makes sense for me.
- Intel DP77KB
Now only the right matching case will be the question - mSATA 32 GB or 60 GB or 120 GB (transcend)
16GB for plain install, 32GB for Snort, 60GB for Snort & Squid and 120GB for more users, services, HotSpot, Squid, Snort,….. - 2 x 4 GB RAM (fastest as the board will be supporting please) (DDR3-1600 in your case)
Often the CPU is not saturated but the memory system is! And with 8 GB you will be able to high up the mbuf size to 1000000 - Intel PRO/1000 PT refurbished or a refurbished Intel i350-T4 would my personal choice here
Often able to get for ~$50 or ~$120 so the pfSense Shop NIC in low profile format could also be interesting
Thank you for your post! In order to boost a the processing power of this new box,I am able to get my hands on an E3 1265Lv2 for a very very good price. I would assume it's quite power enough, right?
- Intel DP77KB
-
Thank you for your post! In order to boost a the processing power of this new box,I am able to get my hands on an E3 1265Lv2 for a very very good price. I would assume it's quite power enough, right?
I'd rather have something like a 3.5GHz kaby lake i3 or pentium for the listed requirements. The improvements in the crypto processing in the latest intel processors and the better single thread performance will count for more than the extra cache and cores. If you've got your heart set on an E3, skip the L variants and get something clocked higher (and I'd personally get something newer than an ivy bridge).
-
So I just purchased a bunch of the accessories to go with this. A ubiquiti AP pro, ubiquiti 8 port managed edgeswitch, and all the parts to move forward with using the older e3 1220L. I will be picking up a 3770T soon which should undoubtedly be more than fast enough and has aes-ni . although some of you say grab a newer top of the line Intel and everything to match it, this 4 year old technology is well within its limits of pushing 1gbe IMO. I don't forsee pfSense outgrowing this equipments capabilities within the next 5 years.
Total setup:
I7 3770T
16 gigs of corsair memory
120 GB 850 msata
Intel thin mini itx DQ77KB
Silverstone pt13D case (looks fantastic!)
Silverstone NT07-115X coolerAlso picked up an arris sb6190
-
E3 1265Lv2
Get it! It scales from 2,5GHz to 3,5GHz and has 4C/8T (HT) and AES-NI on top too.
Silverstone pt13D case (looks fantastic!)
If this is the Silverstone case where you want to put outthe extra NIC with 2 or 4 Ports?
-
Also picked up an arris sb6190
Might want to see this:
https://www.dslreports.com/forum/r31079834-ALL-SB6190-is-a-terrible-modem-Intel-Puma-6-MaxLinear-mistake
-
Also picked up an arris sb6190
Might want to see this:
https://www.dslreports.com/forum/r31079834-ALL-SB6190-is-a-terrible-modem-Intel-Puma-6-MaxLinear-mistake
Thank you ! I have canceled the order for that. I assumed that all the great reviews of the smaller model that this one would accel. Do you have a suggestion?
@BlueKobold:
E3 1265Lv2
Get it! It scales from 2,5GHz to 3,5GHz and has 4C/8T (HT) and AES-NI on top too.
Silverstone pt13D case (looks fantastic!)
If this is the Silverstone case where you want to put outthe extra NIC with 2 or 4 Ports?
I don't need them to be honest. That's why I picked of the edgeswitch. What benefit is there to having them? Is there a specific configuration that you are referring to?
Thanks everyone for the knowledge and opinions! I love it. Keep it coming
-
Can someone please comment on my above comment? About not needing the extra ports. My whole system was designed around using Tue edgeswitch in it. Are there major benefits to hook up an AP or PC directly to a pfSense box?
-
Thank you ! I have canceled the order for that. I assumed that all the great reviews of the smaller model that this one would accel. Do you have a suggestion?
I don't, off the top of my head. I just switched from cable back to vdsl (slower speeds, but unlimited data) and have sidelined my cable modem. I was running a Zoom 5370 (16x4) that worked like a champ, but you'll have to do your own research for 24 or 32 channel downstream DOCSIS 3 modems. Just wanted to make you aware of the ongoing issue. It's not just the 6190; any modem with the Intel Puma 6 chipset is affected. Look for one with a Broadcom chipset if you need that many channels.
-
So all my parts have arrived . going to be building my new box soon here.. Going to try with the e3 1220L for shits and giggles. I want to benchmark it with a VPN running, snort and squid I think. Can anyone point me in the direction of a good guide on accurately testing throughput?
-
I am rather curious because i am also planning on a build based on a xeon.
Please post your progress.Thanks -
So the box is up, configured and paired with an 8 port Edgeswitch. I have an openvpn server running already, snort installed and i have ran some tests. Everything seems to be fantastic from what i can see. throughput of 940MB/s with snort and all. Haven't tested through a VPN yet though. On a side note, i have link aggregation setup from a FreeNAS box and my Edgeswitch and i was running some tests on that as well.. here are my results.
Interface Traffic Peak Total
lagg0 in 232.836 MB/s 233.663 MB/s 80.145 GB
out 5.025 MB/s 121.171 MB/s 9.453 GBem1 in 115.722 MB/s 116.433 MB/s 50.142 GB
out 3.949 MB/s 22.756 MB/s 3.619 GBem0 in 117.335 MB/s 117.338 MB/s 30.032 GB
out 1.075 MB/s 80.398 MB/s 5.835 GBFully saturated 100% across both connections.
obviously, this isn't even doing anything to pfSense though, as it's running all through locally via the switch. Due to only having two NICS on the pfSense box, the only way i can test my throughput of pfSense is to do a makeshift configuration with a computer connected to the WAN and another connected to the LAN.. unless im missing something :)
Question/verification.. Snort only watches the WAN, correct?
-
Great results, very good indeed.
Thanks for you update Toyebox, for a Processor Base Frequency of 2.50 GHz thats very good.I am thinking of a 1151 cpu for my rig.
The edgeswitch surely plays its part, but with your setup on full power, what results do you get on terms of cpu usage?
Only one core tops out, several cores, and with snort you really get all your bandwithd ?I am asking because if i can save up money on a lower mhz Xeon, that would be super.No need to go too overkill.
If a 1220 at 3ghz does the job no need to buy a 1240.
Please keep us informed, thanks.
-
Great results, very good indeed.
Thanks for you update Toyebox, for a Processor Base Frequency of 2.50 GHz thats very good.I am thinking of a 1151 cpu for my rig.
The edgeswitch surely plays its part, but with your setup on full power, what results do you get on terms of cpu usage?
Only one core tops out, several cores, and with snort you really get all your bandwithd ?I am asking because if i can save up money on a lower mhz Xeon, that would be super.No need to go too overkill.
If a 1220 at 3ghz does the job no need to buy a 1240.
Please keep us informed, thanks.
My full setup currently is:
i7 3770S
16GB nonECC memory
120GB msata SSD (samsung)
Intel BOXDQ77KB Mobo (using the dual onboard Intel NICs)I ran some Iperf tests. The setup looks like this:
modem
|
|
|
v
random router (pfSense is DMZ)
|
|
v
WindowsPC (Iperf Server) –-------> switch ---------> pfSense ---------> FreeNAS box (Iperf client)Reasoning for the odd setup is due to the fact that i only have two NIC's in my pfSense box and i only have a 100Mbit/sec connection.. I wanted to really push it using iperf so i put a computer on the WAN side of the pfSense box.
The commands i used are:
Server: iperf -s -u -i 1 -B 192.168.1.10 -p 7001
Client: iperf -c 192.168.1.10 -B 192.168.0.5 -t 99999999 -u -i 1 -p 7001 -b 1000M -l 1250 -S 0xA0-c 192.168.1.10 = server ip
-B 192.168.0.5 = the clients IP
-t 99999999 = run infinitely
-u = use UDP
-i 1 = 1 second between bandwidth reports
-p 7001 = port used
-b 1000M = size of bandwidth to send per second
-l 1250 = length of buffer
-S 0xA0 = Type of service to report (means Critical)I tried 1000M first. Results:
[ 3] local 192.168.1.10 port 7001 connected with 192.168.1.21 port 62141
[ ID] Interval Transfer Bandwidth Jitter Lost/Total Datagrams
[ 3] 0.0- 1.0 sec 113 MBytes 944 Mbits/sec 0.017 ms 3488/97866 (3.6%)
[ 3] 1.0- 2.0 sec 112 MBytes 943 Mbits/sec 0.017 ms 5707/100023 (5.7%)
[ 3] 2.0- 3.0 sec 112 MBytes 943 Mbits/sec 0.018 ms 5638/99980 (5.6%)
[ 3] 3.0- 4.0 sec 112 MBytes 943 Mbits/sec 0.017 ms 5651/99993 (5.7%)
[ 3] 4.0- 5.0 sec 112 MBytes 943 Mbits/sec 0.018 ms 5638/99976 (5.6%)
[ 3] 5.0- 6.0 sec 112 MBytes 943 Mbits/sec 0.018 ms 5638/99976 (5.6%)About average of 5.5% loss which i believe to be pretty accurate given the max bandwidth output through a gigabit LAN is about 940Mbits/sec.
Then i tried to lower it down to 900 Mbits/sec
[ 3] local 192.168.1.10 port 7001 connected with 192.168.1.21 port 65495
[ ID] Interval Transfer Bandwidth Jitter Lost/Total Datagrams
[ 3] 0.0- 1.0 sec 107 MBytes 900 Mbits/sec 0.018 ms 0/90041 (0%)
[ 3] 1.0- 2.0 sec 107 MBytes 900 Mbits/sec 0.018 ms 0/89995 (0%)
[ 3] 2.0- 3.0 sec 107 MBytes 900 Mbits/sec 0.015 ms 0/90008 (0%)
[ 3] 3.0- 4.0 sec 107 MBytes 900 Mbits/sec 0.018 ms 0/90004 (0%)
[ 3] 4.0- 5.0 sec 107 MBytes 900 Mbits/sec 0.018 ms 0/90003 (0%)
[ 3] 5.0- 6.0 sec 107 MBytes 900 Mbits/sec 0.019 ms 0/89995 (0%)
[ 3] 6.0- 7.0 sec 107 MBytes 900 Mbits/sec 0.019 ms 0/90006 (0%)
[ 3] 7.0- 8.0 sec 107 MBytes 900 Mbits/sec 0.019 ms 0/90001 (0%)as expected, zero loss.
I am not totally sure how much stress this is putting on the system, even with snort enabled.. I am still very new at pfSense, so i am not sure if snort puts a heavy load on the system just from being enabled, or when there is a mass amount of traffic with a small window size? If anyone can clear this up for me, and give me an idea of how to correctly test the performance (if the above is incorrect) , i would appreciate it. As for now, i am very satisfied with the throughput of this system. If anyone else wants to see any other tests, feel free to ask and i will do what i can. I did try some torrenting with an ubuntu CD as the media, and easily maxed out at my max internet speed of 100Mbits/sec (hopefully 1Gbits/sec soon!!)
Thanks for the advice and help so far!
-
I thought the idea here was that you wanted something to support gigabit vpn and maybe snort. It's complete overkill for just firewalling (if you couldn't run iperf through at 1000Mbps it would be a real problem). You can test the VPN performance to get a better idea of how the CPU does. As for snort, it's mostly single thread and if it can't keep up it'll just drop traffic (won't affect the traffic going through). Testing that is a lot harder, and the performance is heavily dependent on the traffic and the signatures (you can't test just by running iperf through it).
-
I thought the idea here was that you wanted something to support gigabit vpn and maybe snort. It's complete overkill for just firewalling (if you couldn't run iperf through at 1000Mbps it would be a real problem). You can test the VPN performance to get a better idea of how the CPU does. As for snort, it's mostly single thread and if it can't keep up it'll just drop traffic (won't affect the traffic going through). Testing that is a lot harder, and the performance is heavily dependent on the traffic and the signatures (you can't test just by running iperf through it).
My apologies, i do plan on using it for VPN. I will try running a VPN through it now. Do you have suggestions on what tests to perform? Like i said, i am restricted by my actual line connection at the current time, so thats why i defaulted to iperf. I would happily take suggestions of what tests to perform to get a good idea of my performance. Like i said, i am fairly new at this. Networking isn't my main expertise.. programming is; but i would like to get better!
-
-I am having a very hard time finding a capable Mobo in mini ITX form.. Any suggestions would be greatly appreciated. This is the only one i could find. http://ark.intel.com/products/59046/Intel-Desktop-Board-DQ77KB
Would you recommend getting a i350 pciE? I would much rather just use the dual intel NIC's if i can, but only if they will be efficient enough to handle the load. I have a feeling adding the Ethernet card will also cause problems with the small form factor.
I bought a DQ77KB and a 1265L V2 over a year ago and runs it under VMware's ESXi. It runs pfsense on a dedicated msata and 3 operating systems on separate SSD. All while using a Chinese knockoff i350 quad nic and it's been running 100% solid the entire time! I can't recommend this system setup enough. It's been a marvel of reliability.
-
-I am having a very hard time finding a capable Mobo in mini ITX form.. Any suggestions would be greatly appreciated. This is the only one i could find. http://ark.intel.com/products/59046/Intel-Desktop-Board-DQ77KB
Would you recommend getting a i350 pciE? I would much rather just use the dual intel NIC's if i can, but only if they will be efficient enough to handle the load. I have a feeling adding the Ethernet card will also cause problems with the small form factor.
I bought a DQ77KB and a 1265L V2 over a year ago and runs it under VMware's ESXi. It runs pfsense on a dedicated msata and 3 operating systems on separate SSD. All while using a Chinese knockoff i350 quad nic and it's been running 100% solid the entire time! I can't recommend this system setup enough. It's been a marvel of reliability.
Very nice!! I honestly have never messed with ESXi. Maybe I will have to give it a try.
-
My apologies, i do plan on using it for VPN. I will try running a VPN through it now. Do you have suggestions on what tests to perform?
Do exactly what you did, except with a vpn server set up in the middle?