PFSense 2.1.3 on APU1C4 Slow inter LAN routing

[Rikske]

Hi,
I have build several PFSense installs on available hardware without any issues. However I want a low power 24x7 on solution. So I purchased a APU1C4 kit with optional SSD and Wifi adapter.

I want to share my experiences with you because I have some very strange results with this specific hardware.

First the good news:
Installation on this hardware is a breeze. It's stable and the high temperature issue is not a significant problem with normal use. You buy value for money with this kit. (+/- euro 200,=)
I  used the interfaces as LAN1, LAN2 and Internet. I did create the basic rules for Inter-LAN routing and Internet Access. My Internet connection is 40Mb down/ 4Mb up and the device keeps up with a sub 10% CPU load.

The bad news:
I want this device to be a core router as well for my small business network with 7 /24 IP segments. I did a performance test and througput is appr. 60Mbit between LAN1 and LAN2. This is the maximum value when I send large datapackets. Very small packages yield about 1Mbit/s. I do not expect this light hardware to blow-out a full Gb/s however these values are disappointing low. When doing maximum throughput the processor load is appr. 8%.

The strange behavior:
When I simultaneously download something from the Internet the througput skyrockets! I measured about 250Mb/s. When the download completes the low numbers return. I didn't notice this behavior on other installs. For testing purposes I installed the competing product (sorry, no offense) IPFire. Although this product lacks essential features the througput easily reaches 500+ Mb/s. So, there must be some strange software/hardware issue with PFsense on this hardware which prevents the device from reaching maximum performance. When the device operates at this high speeds for a longer time the CPU load is appr. 40% and the system seriously heats up to 60+ degrees celcius.

So, my recommendation is to add additional cooling to this box when on higher sustained load and when used 24x7x365. I ordered a CTX44 Contrinex CTX44/200 cooler (1.4K/W) and intend to mount this to the back.

Is anyone out there experiencing the same performance issues? Does anyone know what is causing this performance issue?
;)