PC Engines apu2 experiences

[maglub]

Hi,

I just set up two APU2C4 boxes with pfSense 2.3.2. After a bit of fiddling with the USB stick for TinyCore, I managed to get it installed. On a 1Gbit connection, I had >800Mbit down and up through the firewall. No tuning what so ever.

I am using pfSense-CE-2.3.2-RELEASE-4g-amd64-nanobsd.img.gz

[j4k3]

Hey Guys, I wish I was getting maglub's performance out of the box.

I just received a APU2C4 in the mail and loaded pfSense-CE-memstick-serial-2.3.2-RELEASE-amd64 on it.

It manages to boot fine, and it's fully functional, the web interface moves much quicker and more fluid than the old ALIX2D3 it had replaced.

However when I run iperf on the LAN interface from another host, I'm only getting 300Mbps or so.

I found this thread about Intel I210AT nics and am attempting their suggestions:

https://redmine.pfsense.org/issues/1221

I see they're playing with the buffers and queues.

My question is this, if I edit loader.conf.local after looking at some of the options some options have "Quotes" around them, and others don't.

If I directly edit loader.conf.local when I specify variables in there, must they use " for the value? Or will just the value suffice?

hw.igb.num_queues="4" or hw.igb.num_queues=4 ?

Thanks

[cwagz]

I always use the quotes. I have not tried it without.

[j4k3]

Thanks for the insight, I managed to finally get close to 1Gbps on the lan interface.

I had to uncheck Disable hardware large receive offload, and Disable hardware TCP segmentation offload

Under System > Advanced > Networking

Based on what I've read so far I know this unit won't route more than 500 Mbps or so but I wanted to at least understand why, the nic was so hobbled right off the bat.

[cwagz]

I am looking for some opinions on downsizing my current pfSense system with an APU2C4.

Currently I have:
Supermicro A1SRI-2558
8GB Ram
120GB SSD
Akasa Fanless Enclosure

There are 6 people in my house and 30 or so devices.  I am the only person that ever uses OpenVPN and it is usually from a mobile device on LTE so OpenVPN performance is probably not a huge deal.  I run Squid and Squidguard to proxy the internet for my kids.  Our internet connection is FiOS 150/150 Mbps.

It seems like I could build an apu2c4 and sell my current hardware.  I would probably have money left over and a smaller, slightly cooler running device for pfSense.

Do you guys see any potential performance issues or reasons why this is a bad idea?

[Stugots]

Does anyone know of a way to enable TRIM support on the SSD without having to boot of a recovery device?  Is there some sort of tunable where it can be enabled on the next reboot?

[AndrewZ]

@acascianelli:

Does anyone know of a way to enable TRIM support on the SSD without having to boot of a recovery device?

https://forum.pfsense.org/index.php?topic=66622.0
https://forum.pfsense.org/index.php?topic=113803.msg633795#msg633795

[lachendichatz]

I switched from A1SRI-2758/8GB to apu2c4 as I need the 2758 for another server.

Running iperf3 between two pcs connected by the apu (pfsense 2.3.2) I get
600Mbit/s in one direction
615Mbit/s in the other one.
CPU runs at 25% load, e.g. one core is maxed out.

I probably see different speeds because I already imported my old firewall rules and have three rules on one nic and around 20 on the other one.

I observed the same speed & load when I installed debian and configured a few iptable rules.

Adding more clients (iperf -P 8 ) gives me:
940Mbit/s direction a
690Mbit/s direction b
880MBit/s duplex

Enabling (disabling unchecked) segmentation offload gives me
940Mbit/s direction a
695Mbit/s direction b
940Mbit/s duplex
CPU runs at 85%

Speedtest (init7) shows me 930down/940up. Initially I got 720/920 & 670/920 but that was because of my slow laptop (sigh). Restarting firefox gave me consistent speeds around 940.

So I can route a single TCP connection at 600Mbit/s per core. One could probably achieve higher speeds by tuning ISR related configs but as I can saturate my gigabit line with multiple connections I won't change settings.

I set igb_numqueue to 4 and mbuf to 1mio. Unknown if it had an effect.
Somebody suggested to set a rx/tx level (or queue? dont remeber) to 8k. That did not have an effect.
powerD disabled/enabled (hiadaptive) did not make a difference

[Stugots]

@AndrewZ:

@acascianelli:

Does anyone know of a way to enable TRIM support on the SSD without having to boot of a recovery device?

https://forum.pfsense.org/index.php?topic=66622.0
https://forum.pfsense.org/index.php?topic=113803.msg633795#msg633795

Is there no way to set it so that it's enabled on the next reboot without going into single user mode?

[Guest]

Is there no way to set it so that it's enabled on the next reboot without going into single user mode?

Actually is there no way or workaround, as I am informed right.

[kevindd992002]

@j4k3:

Thanks for the insight, I managed to finally get close to 1Gbps on the lan interface.

I had to uncheck Disable hardware large receive offload, and Disable hardware TCP segmentation offload

Under System > Advanced > Networking

Based on what I've read so far I know this unit won't route more than 500 Mbps or so but I wanted to at least understand why, the nic was so hobbled right off the bat.

Does that mean these two should be unchecked to get the full potential of the NIC's of the APU2C4? Any disadvantages of keeping them unchecked (enabled)?

[hda]

@kevindd992002:

… Any disadvantages of keeping them unchecked (enabled)?

Possibly, like no or a snappy WAN-PPPoE connection.

[hda]

@acascianelli:

…Is there no way to set it so that it's enabled on the next reboot without going into single user mode?

https://forum.pfsense.org/index.php?topic=121515.msg673176#msg673176 / pfSense 2.4

[kevindd992002]

@hda:

@kevindd992002:

… Any disadvantages of keeping them unchecked (enabled)?

Possibly, like no or a snappy WAN-PPPoE connection.

But why is the NIC performance hampered with these settings disabled anyway?

@hda:

@acascianelli:

…Is there no way to set it so that it's enabled on the next reboot without going into single user mode?

https://forum.pfsense.org/index.php?topic=121515.msg673176#msg673176 / pfSense 2.4

So if I understand this correctly, a fresh install of 2.4 will already enabled TRIM automatically with no user intervention? And same goes with older versions of pfsense that upgrade 2.4, TRIM will be enabled?

[doktornotor]

@kevindd992002:

But why is the NIC performance hampered with these settings disabled anyway?

You clearly are confused. When you check them, you DISable the HW offloading features.

[kevindd992002]

I don't think I am. Clearly, unchecking the boxes = ENABLES these features. checking the boxes=DISABLES these features. It's very easy to distinguish between the two.

j4k3 said in his post: "I had to uncheck Disable hardware large receive offload, and Disable hardware TCP segmentation offload". Which means that enabling (very different from "checking") them improves performance.

So then I asked: "But why is the NIC performance hampered with these settings disabled anyway?". Or in other words: "why is the NIC performance hampered with the boxes CHECKED anyway?"

Does that make sense? Again, disable=checked and enabled=unchecked. Please check the terminologies that I used in my posts.

[Guest]

Does that mean these two should be unchecked to get the full potential of the NIC's of the APU2C4?

Here under this link you will be able to read what is really needed for getting 1 GBit/s at the
WAN interface, there is told something likes, Server grade hardware and ~2,0GHz CPU speed.
And as I see it right the APU1D4 and APU2C4 are only sorted with something around ~1,1GHz
or 1,2GHz CPU power, that's it in short. Please read under under CPU selection

Any disadvantages of keeping them unchecked (enabled)?

Tunings and pimps can be done on each machine for sure to high up the
throughput but in that case, you should be followed to that guidance
from above at first.

[cwagz]

@cwagz:

I am looking for some opinions on downsizing my current pfSense system with an APU2C4.

Currently I have:
Supermicro A1SRI-2558
8GB Ram
120GB SSD
Akasa Fanless Enclosure

There are 6 people in my house and 30 or so devices.  I am the only person that ever uses OpenVPN and it is usually from a mobile device on LTE so OpenVPN performance is probably not a huge deal.  I run Squid and Squidguard to proxy the internet for my kids.  Our internet connection is FiOS 150/150 Mbps.

It seems like I could build an apu2c4 and sell my current hardware.  I would probably have money left over and a smaller, slightly cooler running device for pfSense.

Do you guys see any potential performance issues or reasons why this is a bad idea?

I went ahead and built the apu2c4 and am very happy with the outcome.  The performance seems to be the same for our usage.  Also, the overall footprint and heat output into my small network cabinet is improved.

[HackedComputer]

Hey,

I recently took delivery of an APU2C4. It is certainly a decent performer for the size of it!

I am wondering, has anyone got the AES-NI to work with the OpenVPN? The reason I ask is that I don't appear to see any acceleration happening with AES-128-CBC / AES-256-CBC. The rough maximum I have achieved is 30Mbps.

I have tried enabling the AES-NI within Advanced Options, and then enabling the cryptodev within OpenVPN. As well as disabling AES-NI and leaving Cryptodev enabled vice-versa.

However, I see no changes whatsoever.

I am on the latest PFSense 2.3.x release

Kindest Regards
HC

[Guest]

I am wondering, has anyone got the AES-NI to work with the OpenVPN? The reason I ask is that I don't appear to see any acceleration happening with AES-128-CBC / AES-256-CBC. The rough maximum I have achieved is 30Mbps.

From what total line speed you archived the 30Mbps? And how strong was the other VPN Peer end?

I have tried enabling the AES-NI within Advanced Options, and then enabling the cryptodev within OpenVPN. As well as disabling AES-NI and leaving Cryptodev enabled vice-versa.

At the moment only IPsec is really benefitting from the AES-NI, so you might be having
perhaps more luck if the OpenVPN version 2.4 is out there.