PfSense i7-4510U + 2x Intel 82574 + 2x Intel i350 (miniPCIE) Mini-ITX Build

Paint

I am a long time Unix/Linux user, who has mainly used DD-WRT for my home router setup (over 30 devices, 2 routers + 1 AP). Now that I have 100/100mbit Fiber Internet, I've decided it is time to venture into pfSense!

Any suggestions or questions are welcome!

I will update this post with the progress of my build. Here are the devices I plan on using in my setup:

pfSense Hardware:
Brand Name: HAMSING
Processor Main Frequency: 1.8GHz(Tubo 3.0GHz)
Processor Model:Intel I7 4500U
Model Number: HS-4500I
Hard Drive: Transcend 64GB SATA III 6Gb/s MSA370 mSATA Solid State Drive
RAM: 8GB 1600MHz DDR3L PC3-12800 ECC CL11 1.35V SODIMM
Video: VGA+HDMI
Audio: Realtek ALC6662
Network: Intel 82574 21000M
USB : 6usb2.0 2USB3.0
RS232: 6RS232
WIFI: 300M

Managed Switch:
Dell PowerConnect 2716

Wireless AP:
1 x Netgear R8000 with DD-WRT (used as an Access Point)
1 x Asus RT-AC66U with DD-WRT (used as an Access Point)

–------ UPDATE 7/28/2016 --------

I added a Jetway Mini-PCIe Intel i350 ADMPEIDLB 2x Gigabit adapter to this machine.
The em(4) freebsd driver used with the on-board 2x Intel 82574 adapters would cause watchdog timeouts every 2-3 days.

More information here: https://forum.pfsense.org/index.php?topic=113610.msg643350#msg643350

edwardwong

The setup looks great, but seems overkill for 100M internet…...

Paint

@edwardwong:

The setup looks great, but seems overkill for 100M internet…...

agree that it is overkill, but hey at least it is "future proof"

I was able to get the switch for 20 bucks on eBay and the whole MiniPC for less than 400 dollars. Overall I think for the price and size, you can't beat the setup above.

Guest

@Paint
Did you install pfSense and reached this throughput or is the speed test made under Linux or DD-WRT?
How many and what kind of packets are installed on your pfSense box?

Paint

@BlueKobold:

@Paint
Did you install pfSense and reached this throughput or is the speed test made under Linux or DD-WRT?
How many and what kind of packets are installed on your pfSense box?

The speed tests in my signature are using my Netgear R8000 Router running DD-WRT (Kong) on my 100/100 mbit fiber internet connection.

The pfSense box I described in my OP arrives tomorrow so it will be at least a week before I post performance etc.

Guest

The speed tests in my signature are using my Netgear R8000 Router running DD-WRT (Kong) on my 100/100 mbit fiber internet connection.

Ah ok this was not clear to me.

The pfSense box I described in my OP arrives tomorrow so it will be at least a week before I post performance etc.

I am really interested to hear about that! If you do a fresh and full install it will be really interesting
to know how good this pfSense box will be performing!

Paint

@BlueKobold:

The speed tests in my signature are using my Netgear R8000 Router running DD-WRT (Kong) on my 100/100 mbit fiber internet connection.

Ah ok this was not clear to me.

The pfSense box I described in my OP arrives tomorrow so it will be at least a week before I post performance etc.

I am really interested to hear about that! If you do a fresh and full install it will be really interesting
to know how good this pfSense box will be performing!

I am doing a clean install of pfSense 2.3.1 64-bit. I will let you know the benchmarks, etc once I have the machine built.

ddarlington36

I myself considering similar box in mind my purposes were mainly future proof and the Intel NICs plus price point is very appealing. Are you running any type of VPN ipsec etc, what packages are you running?you probably could get away with the i3/4005u i5/4200u or even a braswell n3150 if your setup is similar to mine . But myself in the same boat as you started with dd-wrt and went to pfsense my reasons were related to vlan flexibility

Paint

@ddarlington36:

I myself considering similar box in mind my purposes were mainly future proof and the Intel NICs plus price point is very appealing. Are you running any type of VPN ipsec etc, what packages are you running?you probably could get away with the i3/4005u i5/4200u or even a braswell n3150 if your setup is similar to mine . But myself in the same boat as you started with dd-wrt and went to pfsense my reasons were related to vlan flexibility

I will be using a VPN, snort, pfblocker, and possibly squid. I will post a full update once I configure the machine. I just received the MiniPC yesterday - looks like it is made pretty well actually.

Paint

Machine is built and pfSense is installed!

What performance tests would you like me to run (please provide the commands so I run the correct test)?

Thanks!

mauroman33

Hi Paint,

could you please run the simple OpenVPN benchmark referenced here:
https://forum.pfsense.org/index.php?topic=105238.msg616743#msg616743 (Reply #9 message)

Executing the command on my router with a Celeron N3150 I get
27.41 real 25.62 user 1.77 sys

(3200 / 27.41) = 117 Mbps OpenVPN performance (estimate)

This value perfectly fits to the result of a real speed test.

I recently got an upgrade to 250/100 connection and I'm considering buying a mini PC as your own if it were able to sustain this speed through the OpenVPN connection.

Thanks!

speedt1.png_thumb

Guest

I would like to know the routing power and speed between two VLANs, if you get it working.
And on top a new speed test as you where showing it in your signature.

Also a IPSec test would be fine to see but mostly it will not really running pending on the
circumstance that two VPN endpoints must be there.

If you want to do some tuning for your pfSense box you could try out this ones;
Processor Main Frequency: 1.8GHz(Tubo 3.0GHz)
Processor Model:Intel I7 4500U

Please enable PowerD (hi adaptive)
this will scale the CPU frequency from the lowest bottom to the highest top likes needed by the system and
pending of the entire network load of your network or pfSense firewall.

Hard Drive: Transcend 64GB SATA III 6Gb/s MSA370 mSATA Solid State Drive

If this drive is supporting TRIM, enable also the TRIM support on the pfSense box
If this mSATA will be supporting TRIM it should be a deal for you to activate the TRIM support
of the pfSense system too

RAM: 8GB 1600MHz DDR3L PC3-12800 ECC CL11 1.35V SODIMM

Please set the mbuf size to 1000000
You will be able to realize it and not ending up in a booting loop, if you are owing
sufficient amount of RAM and your 8 GB will be ideal for that tuning.

And at last please create a /boot/loader.conf.local file if that wasn´t done right now and enter
the line with the "mbuf size" there that this will suvive all updates/upgrades of your pfSense
system from version to version, because all files will be written totally new!

Paint

@mauroman33:

Hi Paint,

could you please run the simple OpenVPN benchmark referenced here:
https://forum.pfsense.org/index.php?topic=105238.msg616743#msg616743 (Reply #9 message)

Executing the command on my router with a Celeron N3150 I get
27.41 real 25.62 user 1.77 sys

(3200 / 27.41) = 117 Mbps OpenVPN performance (estimate)

This value perfectly fits to the result of a real speed test.

I recently got an upgrade to 250/100 connection and I'm considering buying a mini PC as your own if it were able to sustain this speed through the OpenVPN connection.

Thanks!

Here is the output:

[2.3.1-RELEASE][root@pfSense.lan]/root: openvpn --genkey --secret /tmp/secret
[2.3.1-RELEASE][root@pfSense.lan]/root: time openvpn --test-crypto --secret /tmp/secret --verb 0 --tun-mtu 20000 --cipher aes-256-cbc
10.682u 0.677s 0:11.36 99.9%    742+177k 0+0io 1pf+0w
[2.3.1-RELEASE][root@pfSense.lan]/root:

(3200 / 11.36) = 281.7 Mbps OpenVPN performance (estimate)

Paint

@BlueKobold:

I would like to know the routing power and speed between two VLANs, if you get it working.
And on top a new speed test as you where showing it in your signature.

Also a IPSec test would be fine to see but mostly it will not really running pending on the
circumstance that two VPN endpoints must be there.

If you want to do some tuning for your pfSense box you could try out this ones;
Processor Main Frequency: 1.8GHz(Tubo 3.0GHz)
Processor Model:Intel I7 4500U

Please enable PowerD (hi adaptive)
this will scale the CPU frequency from the lowest bottom to the highest top likes needed by the system and
pending of the entire network load of your network or pfSense firewall.

Hard Drive: Transcend 64GB SATA III 6Gb/s MSA370 mSATA Solid State Drive

If this drive is supporting TRIM, enable also the TRIM support on the pfSense box
If this mSATA will be supporting TRIM it should be a deal for you to activate the TRIM support
of the pfSense system too

RAM: 8GB 1600MHz DDR3L PC3-12800 ECC CL11 1.35V SODIMM

Please set the mbuf size to 1000000
You will be able to realize it and not ending up in a booting loop, if you are owing
sufficient amount of RAM and your 8 GB will be ideal for that tuning.

And at last please create a /boot/loader.conf.local file if that wasn´t done right now and enter
the line with the "mbuf size" there that this will suvive all updates/upgrades of your pfSense
system from version to version, because all files will be written totally new!

Got this mostly up and working today. I am going to do some additional tweaks before I release any speed tests, but I can report that my WAN speeds are about he same (I'm capped at 100/100mbits anyway).

I tried to enable TRIM via this post: https://forum.pfsense.org/index.php?topic=83272.msg456248#msg456248

Unfortunately, after adding ahci_load to my loader.conf.local and running touch /root/TRIM_set; /etc/rc.reboot I still do not have TRIM (I dont think its a big deal though)

[2.3.1-RELEASE][root@pfSense.lan]/root: tunefs -p /
tunefs: POSIX.1e ACLs: (-a)                                disabled
tunefs: NFSv4 ACLs: (-N)                                   disabled
tunefs: MAC multilabel: (-l)                               disabled
tunefs: soft updates: (-n)                                 enabled
tunefs: soft update journaling: (-j)                       enabled
tunefs: gjournal: (-J)                                     disabled
tunefs: trim: (-t)                                         disabled
tunefs: maximum blocks per file in a cylinder group: (-e)  4096
tunefs: average file size: (-f)                            16384
tunefs: average number of files in a directory: (-s)       64
tunefs: minimum percentage of free space: (-m)             8%
tunefs: space to hold for metadata blocks: (-k)            6408
tunefs: optimization preference: (-o)                      time
tunefs: volume label: (-L)

Here is a copy of my /boot/loader.conf.local:

ahci_load="YES"
kern.ipc.nmbclusters="1000000"
legal.intel_ipw.license_ack=1
legal.intel_iwi.license_ack=1

mauroman33

@Paint:

Here is the output:

[2.3.1-RELEASE][root@pfSense.lan]/root: openvpn --genkey --secret /tmp/secret
[2.3.1-RELEASE][root@pfSense.lan]/root: time openvpn --test-crypto --secret /tmp/secret --verb 0 --tun-mtu 20000 --cipher aes-256-cbc
10.682u 0.677s 0:11.36 99.9%    742+177k 0+0io 1pf+0w
[2.3.1-RELEASE][root@pfSense.lan]/root:

(3200 / 11.36) = 281.7 Mbps OpenVPN performance (estimate)

Thanks mate!
Now I know that I have to find my way in this cpu's class

ddarlington36

What's the CPU usage like during the tests? Is that test anything like iperf or dose it simulate the openvpn throughput/bandwidth. Pretty impressive results !! I'm sold

Guest

Got this mostly up and working today. I am going to do some additional tweaks before I release any speed tests, but I can report that my WAN speeds are about he same (I'm capped at 100/100mbits anyway).

With disabled PowerD (hi adaptive) it could be that the CPU frequency is not scaling from low to high likes it
is needed by the load, and so any kind of many tests could be not really right then! Please don´t forget this
and think about.

I tried to enable TRIM via this post: https://forum.pfsense.org/index.php?topic=83272.msg456248#msg456248

Unfortunately, after adding ahci_load to my loader.conf.local and running touch /root/TRIM_set; /etc/rc.reboot I still do not have TRIM (I dont think its a big deal though)

Please use this procedure shown in that thread/post exactly! It is right matching and well working!
Enable TRIM Support in pfSense

ahci_load="YES"
kern.ipc.nmbclusters="1000000"
legal.intel_ipw.license_ack=1
legal.intel_iwi.license_ack=1

This might be right looking to me. If you are doing tests now, you could not be running out of kernel
space or mbuf size!

Paint

@mauroman33:

@Paint:
Here is the output:
[2.3.1-RELEASE][root@pfSense.lan]/root: openvpn --genkey --secret /tmp/secret
[2.3.1-RELEASE][root@pfSense.lan]/root: time openvpn --test-crypto --secret /tmp/secret --verb 0 --tun-mtu 20000 --cipher aes-256-cbc
10.682u 0.677s 0:11.36 99.9%    742+177k 0+0io 1pf+0w
[2.3.1-RELEASE][root@pfSense.lan]/root:
(3200 / 11.36) = 281.7 Mbps OpenVPN performance (estimate)
Thanks mate!
Now I know that I have to find my way in this cpu's class

anytime! Loving this MiniPC so far!

Paint

@ddarlington36:

What's the CPU usage like during the tests? Is that test anything like iperf or dose it simulate the openvpn throughput/bandwidth. Pretty impressive results !! I'm sold

CPU is almost non existent (less than .1-.2 on the 1min top) I will provide a more detailed update once I finish my firewall/traffic shaping/snort/country blocking setup.

I still need to do an iperf test, but I believe I will get very close to 1gbps over my LAN. Therefore, CPU is your bottleneck when using VPN. The previous test shows how fast your CPU can encrypt information and backs into a mbps theoretical max.

Guest

Thanks mate!
Now I know that I have to find my way in this cpu's class

If you are unsure, money is not the real problem for you and you will be having much throughput in the WAN
and LAN area or high throughput over any VPN tunnel, go and buy a Intel Xeon E3-1240v3 and 8 GB DDR3
1600MHz RAM and you will be getting out the maximum of all! Not cheap, but very effective in any kind of.
You can also save money over a longer time or get parts refurbished!

I still need to do an iperf test, but I believe I will get very close to 1gbps over my LAN. Therefore, CPU is your bottleneck when using VPN. The previous test shows how fast your CPU can encrypt information and backs into a mbps theoretical max.

Set up at the LAN interface a subnet likes 192.168.xx and on the other LAN interface another one likes
172.xx.xx and then iPerf client to server test, you can repeat it through the WAN interface by setting up there
a small GB switch and set up outside the AN interface the iPerf server.