PfSense i7-4510U + 2x Intel 82574 + 2x Intel i350 (miniPCIE) Mini-ITX Build

Paint

Machine is built and pfSense is installed!

What performance tests would you like me to run (please provide the commands so I run the correct test)?

Thanks!

mauroman33

Hi Paint,

could you please run the simple OpenVPN benchmark referenced here:
https://forum.pfsense.org/index.php?topic=105238.msg616743#msg616743 (Reply #9 message)

Executing the command on my router with a Celeron N3150 I get
27.41 real 25.62 user 1.77 sys

(3200 / 27.41) = 117 Mbps OpenVPN performance (estimate)

This value perfectly fits to the result of a real speed test.

I recently got an upgrade to 250/100 connection and I'm considering buying a mini PC as your own if it were able to sustain this speed through the OpenVPN connection.

Thanks!

speedt1.png_thumb

Guest

I would like to know the routing power and speed between two VLANs, if you get it working.
And on top a new speed test as you where showing it in your signature.

Also a IPSec test would be fine to see but mostly it will not really running pending on the
circumstance that two VPN endpoints must be there.

If you want to do some tuning for your pfSense box you could try out this ones;
Processor Main Frequency: 1.8GHz(Tubo 3.0GHz)
Processor Model:Intel I7 4500U

Please enable PowerD (hi adaptive)
this will scale the CPU frequency from the lowest bottom to the highest top likes needed by the system and
pending of the entire network load of your network or pfSense firewall.

Hard Drive: Transcend 64GB SATA III 6Gb/s MSA370 mSATA Solid State Drive

If this drive is supporting TRIM, enable also the TRIM support on the pfSense box
If this mSATA will be supporting TRIM it should be a deal for you to activate the TRIM support
of the pfSense system too

RAM: 8GB 1600MHz DDR3L PC3-12800 ECC CL11 1.35V SODIMM

Please set the mbuf size to 1000000
You will be able to realize it and not ending up in a booting loop, if you are owing
sufficient amount of RAM and your 8 GB will be ideal for that tuning.

And at last please create a /boot/loader.conf.local file if that wasn´t done right now and enter
the line with the "mbuf size" there that this will suvive all updates/upgrades of your pfSense
system from version to version, because all files will be written totally new!

Paint

@mauroman33:

Hi Paint,

could you please run the simple OpenVPN benchmark referenced here:
https://forum.pfsense.org/index.php?topic=105238.msg616743#msg616743 (Reply #9 message)

Executing the command on my router with a Celeron N3150 I get
27.41 real 25.62 user 1.77 sys

(3200 / 27.41) = 117 Mbps OpenVPN performance (estimate)

This value perfectly fits to the result of a real speed test.

I recently got an upgrade to 250/100 connection and I'm considering buying a mini PC as your own if it were able to sustain this speed through the OpenVPN connection.

Thanks!

Here is the output:

[2.3.1-RELEASE][root@pfSense.lan]/root: openvpn --genkey --secret /tmp/secret
[2.3.1-RELEASE][root@pfSense.lan]/root: time openvpn --test-crypto --secret /tmp/secret --verb 0 --tun-mtu 20000 --cipher aes-256-cbc
10.682u 0.677s 0:11.36 99.9%    742+177k 0+0io 1pf+0w
[2.3.1-RELEASE][root@pfSense.lan]/root:

(3200 / 11.36) = 281.7 Mbps OpenVPN performance (estimate)

Paint

@BlueKobold:

I would like to know the routing power and speed between two VLANs, if you get it working.
And on top a new speed test as you where showing it in your signature.

Also a IPSec test would be fine to see but mostly it will not really running pending on the
circumstance that two VPN endpoints must be there.

If you want to do some tuning for your pfSense box you could try out this ones;
Processor Main Frequency: 1.8GHz(Tubo 3.0GHz)
Processor Model:Intel I7 4500U

Please enable PowerD (hi adaptive)
this will scale the CPU frequency from the lowest bottom to the highest top likes needed by the system and
pending of the entire network load of your network or pfSense firewall.

Hard Drive: Transcend 64GB SATA III 6Gb/s MSA370 mSATA Solid State Drive

If this drive is supporting TRIM, enable also the TRIM support on the pfSense box
If this mSATA will be supporting TRIM it should be a deal for you to activate the TRIM support
of the pfSense system too

RAM: 8GB 1600MHz DDR3L PC3-12800 ECC CL11 1.35V SODIMM

Please set the mbuf size to 1000000
You will be able to realize it and not ending up in a booting loop, if you are owing
sufficient amount of RAM and your 8 GB will be ideal for that tuning.

And at last please create a /boot/loader.conf.local file if that wasn´t done right now and enter
the line with the "mbuf size" there that this will suvive all updates/upgrades of your pfSense
system from version to version, because all files will be written totally new!

Got this mostly up and working today. I am going to do some additional tweaks before I release any speed tests, but I can report that my WAN speeds are about he same (I'm capped at 100/100mbits anyway).

I tried to enable TRIM via this post: https://forum.pfsense.org/index.php?topic=83272.msg456248#msg456248

Unfortunately, after adding ahci_load to my loader.conf.local and running touch /root/TRIM_set; /etc/rc.reboot I still do not have TRIM (I dont think its a big deal though)

[2.3.1-RELEASE][root@pfSense.lan]/root: tunefs -p /
tunefs: POSIX.1e ACLs: (-a)                                disabled
tunefs: NFSv4 ACLs: (-N)                                   disabled
tunefs: MAC multilabel: (-l)                               disabled
tunefs: soft updates: (-n)                                 enabled
tunefs: soft update journaling: (-j)                       enabled
tunefs: gjournal: (-J)                                     disabled
tunefs: trim: (-t)                                         disabled
tunefs: maximum blocks per file in a cylinder group: (-e)  4096
tunefs: average file size: (-f)                            16384
tunefs: average number of files in a directory: (-s)       64
tunefs: minimum percentage of free space: (-m)             8%
tunefs: space to hold for metadata blocks: (-k)            6408
tunefs: optimization preference: (-o)                      time
tunefs: volume label: (-L)

Here is a copy of my /boot/loader.conf.local:

ahci_load="YES"
kern.ipc.nmbclusters="1000000"
legal.intel_ipw.license_ack=1
legal.intel_iwi.license_ack=1

mauroman33

@Paint:

Here is the output:

[2.3.1-RELEASE][root@pfSense.lan]/root: openvpn --genkey --secret /tmp/secret
[2.3.1-RELEASE][root@pfSense.lan]/root: time openvpn --test-crypto --secret /tmp/secret --verb 0 --tun-mtu 20000 --cipher aes-256-cbc
10.682u 0.677s 0:11.36 99.9%    742+177k 0+0io 1pf+0w
[2.3.1-RELEASE][root@pfSense.lan]/root:

(3200 / 11.36) = 281.7 Mbps OpenVPN performance (estimate)

Thanks mate!
Now I know that I have to find my way in this cpu's class

ddarlington36

What's the CPU usage like during the tests? Is that test anything like iperf or dose it simulate the openvpn throughput/bandwidth. Pretty impressive results !! I'm sold

Guest

Got this mostly up and working today. I am going to do some additional tweaks before I release any speed tests, but I can report that my WAN speeds are about he same (I'm capped at 100/100mbits anyway).

With disabled PowerD (hi adaptive) it could be that the CPU frequency is not scaling from low to high likes it
is needed by the load, and so any kind of many tests could be not really right then! Please don´t forget this
and think about.

I tried to enable TRIM via this post: https://forum.pfsense.org/index.php?topic=83272.msg456248#msg456248

Unfortunately, after adding ahci_load to my loader.conf.local and running touch /root/TRIM_set; /etc/rc.reboot I still do not have TRIM (I dont think its a big deal though)

Please use this procedure shown in that thread/post exactly! It is right matching and well working!
Enable TRIM Support in pfSense

ahci_load="YES"
kern.ipc.nmbclusters="1000000"
legal.intel_ipw.license_ack=1
legal.intel_iwi.license_ack=1

This might be right looking to me. If you are doing tests now, you could not be running out of kernel
space or mbuf size!

Paint

@mauroman33:

@Paint:
Here is the output:
[2.3.1-RELEASE][root@pfSense.lan]/root: openvpn --genkey --secret /tmp/secret
[2.3.1-RELEASE][root@pfSense.lan]/root: time openvpn --test-crypto --secret /tmp/secret --verb 0 --tun-mtu 20000 --cipher aes-256-cbc
10.682u 0.677s 0:11.36 99.9%    742+177k 0+0io 1pf+0w
[2.3.1-RELEASE][root@pfSense.lan]/root:
(3200 / 11.36) = 281.7 Mbps OpenVPN performance (estimate)
Thanks mate!
Now I know that I have to find my way in this cpu's class

anytime! Loving this MiniPC so far!

Paint

@ddarlington36:

What's the CPU usage like during the tests? Is that test anything like iperf or dose it simulate the openvpn throughput/bandwidth. Pretty impressive results !! I'm sold

CPU is almost non existent (less than .1-.2 on the 1min top) I will provide a more detailed update once I finish my firewall/traffic shaping/snort/country blocking setup.

I still need to do an iperf test, but I believe I will get very close to 1gbps over my LAN. Therefore, CPU is your bottleneck when using VPN. The previous test shows how fast your CPU can encrypt information and backs into a mbps theoretical max.

Guest

Thanks mate!
Now I know that I have to find my way in this cpu's class

If you are unsure, money is not the real problem for you and you will be having much throughput in the WAN
and LAN area or high throughput over any VPN tunnel, go and buy a Intel Xeon E3-1240v3 and 8 GB DDR3
1600MHz RAM and you will be getting out the maximum of all! Not cheap, but very effective in any kind of.
You can also save money over a longer time or get parts refurbished!

I still need to do an iperf test, but I believe I will get very close to 1gbps over my LAN. Therefore, CPU is your bottleneck when using VPN. The previous test shows how fast your CPU can encrypt information and backs into a mbps theoretical max.

Set up at the LAN interface a subnet likes 192.168.xx and on the other LAN interface another one likes
172.xx.xx and then iPerf client to server test, you can repeat it through the WAN interface by setting up there
a small GB switch and set up outside the AN interface the iPerf server.

Paint

@BlueKobold:

Thanks mate!
Now I know that I have to find my way in this cpu's class

If you are unsure, money is not the real problem for you and you will be having much throughput in the WAN
and LAN area or high throughput over any VPN tunnel, go and buy a Intel Xeon E3-1240v3 and 8 GB DDR3
1600MHz RAM and you will be getting out the maximum of all! Not cheap, but very effective in any kind of.
You can also save money over a longer time or get parts refurbished!

I still need to do an iperf test, but I believe I will get very close to 1gbps over my LAN. Therefore, CPU is your bottleneck when using VPN. The previous test shows how fast your CPU can encrypt information and backs into a mbps theoretical max.

Set up at the LAN interface a subnet likes 192.168.xx and on the other LAN interface another one likes
172.xx.xx and then iPerf client to server test, you can repeat it through the WAN interface by setting up there
a small GB switch and set up outside the AN interface the iPerf server.

ill do a few different tests for iperf in the next few days. I already have my DHCP server cloning my G1100 MAC and DHCP request so that I can run the FIOS MoCA G1100 Quantum Router in parallel to my pfSense box - this eliminates a double NAT situation, allows me to use my own router, and keep all of the FIOS services (Remote DVR, VoD, CallerID, etc) without the need for my backend "three router" setup.

To setup a new vlan to test a fake WAN will be a piece of cake after that :P

This whole setup only cost me $400 USD + $30 USD for a Dell PowerConnect 2716 Managed Switch from eBay. For the price, I dont think it can be beat!

aGeekhere

What speed do you get from the squid cache?
Download a file
Test files here
http://mirror.internode.on.net/pub/test/
Then once it is downloaded try redownloading and check the speed from the squid cache

Paint

@aGeekHere:

What speed do you get from the squid cache?
Download a file
Test files here
http://mirror.internode.on.net/pub/test/
Then once it is downloaded try redownloading and check the speed from the squid cache

http://mirror.internode.on.net/pub/test/ this link does not work….

asterix

Use this for enabling TRIM.

https://gist.github.com/mdouchement/853fbd4185743689f58c

You don't need to do enable AHCI by adding ahci_load="YES" … it works for me without this.

aGeekhere

this link does not work….

Must be location blocked, try a Ubuntu iso or any large file that will cached.

Paint

@Asterix:

Use this for enabling TRIM.

https://gist.github.com/mdouchement/853fbd4185743689f58c

You don't need to do enable AHCI by adding ahci_load="YES" … it works for me without this.

thanks. that worked:

[2.3.1-RELEASE][root@pfSense.lan]/root: tunefs -p /
tunefs: POSIX.1e ACLs: (-a)                                disabled
tunefs: NFSv4 ACLs: (-N)                                   disabled
tunefs: MAC multilabel: (-l)                               disabled
tunefs: soft updates: (-n)                                 enabled
tunefs: soft update journaling: (-j)                       enabled
tunefs: gjournal: (-J)                                     disabled
tunefs: trim: (-t)                                         enabled
tunefs: maximum blocks per file in a cylinder group: (-e)  4096
tunefs: average file size: (-f)                            16384
tunefs: average number of files in a directory: (-s)       64
tunefs: minimum percentage of free space: (-m)             8%
tunefs: space to hold for metadata blocks: (-k)            6408
tunefs: optimization preference: (-o)                      time
tunefs: volume label: (-L)

migrated my entire network over to pfsense as the main router with two AP running DDWRT. I have done a lot of tweaking, but will finalize some stuff over the weekend. I hope to then post some performance benchmarks.

Next on to snort and traffic shaping 8) 8) 8)

Guest

You don't need to do enable AHCI by adding ahci_load="YES" … it works for me without this.

I must consider to this, I would recommend to shorten this line in the boot/loader.conf.local file, it is not
really needed for your pfSense machine.

Paint

@BlueKobold:

You don't need to do enable AHCI by adding ahci_load="YES" … it works for me without this.

I must consider to this, I would recommend to shorten this line in the boot/loader.conf.local file, it is not
really needed for your pfSense machine.

I dont use ahci_load="YES" in my /boot/loader.conf.local file.

I have made many System Tunable changes and loader.conf.local changes. Below are my /boot/loader.conf.local changes:


legal.intel_ipw.license_ack=1
legal.intel_iwi.license_ack=1
aio_load="YES"
pf_load="YES"
pflog_load="YES
if_em_load="YES"
hw.em.rxd=4096
hw.em.txd=4096
#ahci_load="YES"
cc_htcp_load="YES"
net.inet.tcp.hostcache.cachelimit="0"
hw.em.num_queues="2"
kern.ipc.nmbclusters="1000000"

asterix

Why do u need traffic shaping on a 100MBit line?