PC Engines apu2 experiences



  • Well, I decided to have a look at these LED's. Installed the driver, LED's work.
    During boot, the LED's all dance, and if I install the gwled package, they show the gateway status. Wonderful :)

    Only, I noticed that gwled has a service, but doesn't start… it's annoying. Is this normal? See attachment. (On a sidenote, does anyone else 2x haproxy services?).

    Secondly, I noticed the following in the my /var/log/dmesg.boot:

    ada0: <samsung 850="" ssd="" evo="" msata="" 250gb="" emt41b6q="">ACS-2 ATA SATA 3.x device
    ada0: 600.000MB/s transfers (SATA 3.x, UDMA6, PIO 512bytes)
    ada0: Command Queueing enabled
    ada0: 238475MB (488397168 512 byte sectors)
    ada0: quirks=0x3<4K,NCQ_TRIM_BROKEN>

    What does this 4K,NCQ_TRIM_BROKEN mean exactly in normal English?

    I also note that they have a solution below, but again, not quite sure what the difference is between the two….

    if you're on 11.1-RELEASE or 11-RELEASE, you should add below on /boot/loader.conf.

    If no other quirks is required:
        kern.cam.ada.0.quirks="0x0"

    If you need 4k quirks but want to drop NCQ_TRIM one:
        kern.cam.ada.0.quirks="0x1"

    *4k one is bit0 (0x1), and NCQ_TRIM one is bit1 (0x2).

    The example above assumes the affected drive is recognized as ada0.
    You should change "ada.0" to whatever appropreate.


    </samsung>


  • Netgate

    @Veldkornet:

    What does this 4K,NCQ_TRIM_BROKEN mean exactly in normal English?

    4k means "TRIM only works on 4096 byte requests that are 4096 byte aligned".

    NCQ_TRIM means TRIM doesn't work if you send it as a queued request.  For some drives, the SSD stops working with first trim request is issued from the filesystem (UFS or ZFS).

    There is a bug that is fixed in 11.1-RELEASE.  These quirks keep your drive working.

    If you're running pfSense 2.4.x you can re-enable 4K sectors and TRIM by clearing the quirks with:

    kern.cam.ada.0.quirks="0x0"

    in /boot/loader.conf

    or, if you need 4k quirks but want to drop NCQ_TRIM one:
        kern.cam.ada.0.quirks="0x1"

    4k one is bit0 (0x1), and NCQ_TRIM one is bit1 (0x2).

    Or you can wait for pfSense 2.4.3, which will contain a software fix.



  • @Veldkornet:

    On a sidenote, does anyone else have 2x haproxy services?

    I havn't seen that before..you should only have the lowercase 'haproxy' service.. Probably need to edit the config.xml to remove the wrong service tag.. (backup>edit>restore)or the more tricky:(edit /conf/config.xml,delete /tmp/config.cache) just make sure to keep the xml format valid..



  • I am using an APU2C4 with BIOS version 4.0.7.

    It looks like PC Engines no longer maintains their web pages under
    https://www.pcengines.ch/apu2c4.htm

    I am unable to find any binaries for latest versions 4.0.x and 4.6.x and appreciate any hints on that.

    All I can find are related release infos/changelogs
    https://github.com/pcengines/release_manifests/blob/coreboot-4.6.x/CHANGELOG.md

    and source codes of coreboot
    https://github.com/pcengines/coreboot/releases

    Peter



  • I see some ROMs on the github:
    https://github.com/pcengines/apu2-documentation

    APU2 ROM should be there in legacy and mainline mode



  • @software:

    I see some ROMs on the github:
    https://github.com/pcengines/apu2-documentation

    APU2 ROM should be there in legacy and mainline mode

    Yeah, thanks a lot. Nevertheless, these are not the latest versions. I would have never searched for binaries in the documentation folder ;-).



  • Wow, I'm running 4.07.  Any reason to go higher for pfSense?  What's the recommended BIOS?



  • I thought 4.07 is the latest?



  • @kevindd992002:

    I thought 4.07 is the latest?

    No, development is going on. Unfortunately, version schema is confusing and PC Engines does not update their corresponding web page. Moreover, binaries are at least for me, difficult to find.

    What I have understood so far: There are two actively developed branches: 4.0.x denoted as "legacy" and a "mainline" 4.5.x/4.6.x. Latest versions are 4.0.14 and 4.6.6, respectively. Latest binary downloads for APU2 are available as 4.0.11 and 4.6.1.

    One thing that I have just become aware of: coreboot determines the version number of the APU2 BIOS ROM but the APU2 ROM consists of several other components with its own version numbers like e.g. seabios and ipxe.



  • I guess I should have read outside of the PCEngines website!  Thanks for that info.

    OK, so has everyone successfully run 4.6.6 or should I just go to 4.0.11?



  • @pvoigt:

    @kevindd992002:

    I thought 4.07 is the latest?

    No, development is going on. Unfortunately, version schema is confusing and PC Engines does not update their corresponding web page. Moreover, binaries are at least for me, difficult to find.

    What I have understood so far: There are two actively developed branches: 4.0.x denoted as "legacy" and a "mainline" 4.5.x/4.6.x. Latest versions are 4.0.14 and 4.6.6, respectively. Latest binary downloads for APU2 are available as 4.0.11 and 4.6.1.

    One thing that I have just become aware of: coreboot determines the version number of the APU2 BIOS ROM but the APU2 ROM consists of several other components with its own version numbers like e.g. seabios and ipxe.

    Oh I see. So everything is on github? I remember choosing between 4.0.x and 4.5.x before and many people we're having problems with 4.5.x for some reason. And this is why I chose to go with 4.0.7. Did this change now? Would it be recommended to go with the mainline this time?

    @valnar:

    I guess I should have read outside of the PCEngines website!  Thanks for that info.

    OK, so has everyone successfully run 4.6.6 or should I just go to 4.0.11?

    Me too! All along I thought their website was updated.



  • I just went to 4.0.11 and it's working fine.  I didn't see anything listed in the bios's after 4.0.11 that was relevant to APU2 boards.

    I also noticed that PC Engines recommends the 4.0x track here:
    http://pcengines.ch/howto.htm#bios



  • FYI, anyone updating to the 4.5.x or 4.6.x mainline firmware (https://github.com/pcengines/apu2-documentation), you need to edit the /boot/loader.conf and add the following:

    hint.ahci.0.msi="0"

    Otherwise it reboots every 4-5 hours.

    The rest of the items that they mention here were all already added on my system by default.

    I'm now running 4.6.1 without any problems.



  • @doktornotor:

    @Veldkornet:

    @Jailer:

    Mine still says PC Engines APU2 after the update to 2.3.4.

    Yeah, mine did too. I more meant after the latest firmware update.

    Can you post the output of

    
    /bin/kenv -q smbios.system.product
    /bin/kenv -q smbios.system.maker
    
    

    with the 4.0.7 FW.

    FYI, This has changed again…. below the output from firmware 4.6.1:

    /bin/kenv -q smbios.system.product -> PC Engines apu2
    /bin/kenv -q smbios.system.maker -> PC Engines



  • @PiBa:

    @Veldkornet:

    On a sidenote, does anyone else have 2x haproxy services?

    I havn't seen that before..you should only have the lowercase 'haproxy' service.. Probably need to edit the config.xml to remove the wrong service tag.. (backup>edit>restore)or the more tricky:(edit /conf/config.xml,delete /tmp/config.cache) just make sure to keep the xml format valid..

    Thanks! :)



  • Question, should I see the serial number of my PC Engines on the pfSense dashboard?

    Reason I ask is because one of the PC Engines firmwares made the serial number show random characters. However, in doing so, the serial number field all of a sudden showed up on the dashboard

    Dashboard

    However, once I got the next firmware, the field was gone again from the dashboard.

    The below command gives an actual output:

    /bin/kenv -q smbios.system.serial
    

    However, it’s not shown on the dashboard.

    Is this intentional? Or should it be shown?


  • Netgate Administrator

    It's not expected to show in 2.4.3p1.

    The input validation on that field was improved since it was first introduced. It should only appear now on devices that expose a real serial number via ACPI in the correct field.

    I think the new forum code may have cut-off your kenv output.

    Looking on an APU here it does seem to be present so perhaps the validation there could be tweaked.

    It's not an issue with your board though, that's the expected behaviour currently.

    Steve



  • I have a site-to-site VPN using an OpenVPN tunnel between two APU2C4's with pfSense on them (2.4.3-RELEASE-p1 (amd64)). I've been reading a few hours now on how to really have OpenVPN utilize hardware AES-NI as the CPU supports it. There are several threads about this but not one is clear enough to really explain how pfSense uses this.

    So, if I want to use hardware AES-NI, do I need to choose AES-NI CPU-based Acceleration under System > Advanced > Miscellaneous > Cryptographic Hardware? Or should I set it to None (no module loaded) and OpenVPN will use the AES-NI natively from the hardware without any module conflicts from the pfSense BSD OS?

    If I choose the former, under the OpenVPN Server/Client settings the only selection I have for Hardware Crypto is No Hardware Crypto Acceleration. The only time I have an extra option in the Client settings is when I choose BSD Crypto Device (cryptodev) under Miscellaneous. So which is which? I'm starting to have a headache because of the confusing pfSense GUI :)



  • I'm also curious as to what the "correct" settings are... I can say that I did a couple of tests wrt speed, and I eventually settled on (apart from a bunch of other tweaks) enabling AES-NI CPU-based Acceleration under System > Advanced > Miscellaneous > Cryptographic Hardware, and then in the OpenVPN Server/Client settings the only selection I have No Hardware Crypto Acceleration.

    I know a while back, in the OpenVPN settings you could choose between AES-NI and cryptodev, but after some update the cryptodev disappeard.

    Additionally, I think that the best speeds were achieved when both AES-NI CPU-based Acceleration and cryptodev were enabled, which is now the default if you have AES-NI CPU-based Acceleration enabled on the system.
    So, I've just assumed that it doesn't matter anymore about the setting in OpenVPN since the system is already using all it can.



  • @veldkornet said in PC Engines apu2 experiences:
    Additionally, I think that the best speeds were achieved when both AES-NI CPU-based Acceleration and cryptodev were enabled, which is now the default if you have AES-NI CPU-based Acceleration enabled on the system.
    So, I've just assumed that it doesn't matter anymore about the setting in OpenVPN since the system is already using all it can.

    Enabling both where? Let's call both places Miscellaneous settings and Client settings to avoid confusion. Like I said, I don't have any options for a cryptodev Client setting IF I keep the Miscellaneous settings to AES-NI. All I have is the No Hardware Crypto Acceleration option.



  • My mistake, I actually have the Miscellaneous set to AES-NI and BSD Crypto Device (aesni, cryptodev).
    This is the update that I referred to in my previous post where both are enabled.

    0_1530260662775_d9ee068e-c72f-4821-ae32-bbde3d0b7fad-image.png

    OpenVPN Client & Server:
    0_1530260748281_8d54c185-19b7-402f-8440-0950e3140ea4-image.png

    And these are the options available to me within OpenVPN:
    0_1530260866736_5a0be8e7-7748-46a2-8c02-11cecd47bad8-image.png



  • @veldkornet said in PC Engines apu2 experiences:

    My mistake, I actually have the Miscellaneous set to AES-NI and BSD Crypto Device (aesni, cryptodev).
    This is the update that I referred to in my previous post where both are enabled.

    0_1530260662775_d9ee068e-c72f-4821-ae32-bbde3d0b7fad-image.png

    OpenVPN Client & Server:
    0_1530260748281_8d54c185-19b7-402f-8440-0950e3140ea4-image.png

    And these are the options available to me within OpenVPN:
    0_1530260866736_5a0be8e7-7748-46a2-8c02-11cecd47bad8-image.png

    Ok, that makes more sense and we have the same set of available options in both places. Although I'm reading a lot that it generally is not a good thing to have both modules loaded (which is what you have set in Misc.) so I was wondering how you got better speeds with that?



  • @kevindd992002 said in PC Engines apu2 experiences:

    Ok, that makes more sense and we have the same set of available options in both places. Although I'm reading a lot that it generally is not a good thing to have both modules loaded (which is what you have set in Misc.) so I was wondering how you got better speeds with that?

    With regards to having both loaded, see resolved bug 7810.

    In the OpenVPN settings, selecting BSD cryptodev engine made it slower indeed and shouldn't be selected.

    Not related to the crypto, but what also made a big difference is setting the following:
    0_1530261425126_7b97f90a-589e-43be-97de-ee7c51c9e837-image.png

    Anyway, this is just my opinion of how it should be since this works the best in my situation.



  • @veldkornet said in PC Engines apu2 experiences:

    @kevindd992002 said in PC Engines apu2 experiences:

    Ok, that makes more sense and we have the same set of available options in both places. Although I'm reading a lot that it generally is not a good thing to have both modules loaded (which is what you have set in Misc.) so I was wondering how you got better speeds with that?

    With regards to having both loaded, see resolved bug 7810.

    In the OpenVPN settings, selecting BSD cryptodev engine made it slower indeed and shouldn't be selected.

    Not related to the crypto, but what also made a big difference is setting the following:
    0_1530261425126_7b97f90a-589e-43be-97de-ee7c51c9e837-image.png

    Anyway, this is just my opinion of how it should be since this works the best in my situation.

    Ahh, I see what you mean. Thanks for the heads up. I'm setting it that way then.

    Yeah, I have both UDP Fast I/O and send/receiver buffer set to those values also as I read they can speed up things.


  • Netgate Administrator

    Setting those tweaks will definitely show an improvement. Usually a significant one though I've never tested it on the APU2 myself.

    OpenSSL should use the AES-NI instructions on your CPU directly if it supports them. The danger here is that instead of using them directly it tries to use the BSD crypto framework where the AES-NI kernel module has registered itself for the algorithms it supports. That means a load of additional cycles to do the same calculation.

    As long as you don't have AES-NI+BSD crypto set in Adv. > MIsc. and BSD crypto set in openvpn you should avoid that.

    When I last tested it Fast I/O and send/receiver buffer made a greater difference to throughput.

    Steve



  • @stephenw10 said in PC Engines apu2 experiences:

    Setting those tweaks will definitely show an improvement. Usually a significant one though I've never tested it on the APU2 myself.

    OpenSSL should use the AES-NI instructions on your CPU directly if it supports them. The danger here is that instead of using them directly it tries to use the BSD crypto framework where the AES-NI kernel module has registered itself for the algorithms it supports. That means a load of additional cycles to do the same calculation.

    As long as you don't have AES-NI+BSD crypto set in Adv. > MIsc. and BSD crypto set in openvpn you should avoid that.

    When I last tested it Fast I/O and send/receiver buffer made a greater difference to throughput.

    Steve

    I see. So are you saying that setting AES-NI+BSD in Misc. and just no hardware crypto in the OpenVPN client settings would be fine?



  • @kevindd992002 said in PC Engines apu2 experiences:

    I see. So are you saying that setting AES-NI+BSD in Misc. and just no hardware crypto in the OpenVPN client settings would be fine?

    Im interested in this too! And Im a bit confused after reading 20 posts about it in this thread.
    cheers


  • Netgate Administrator

    @kevindd992002 said in PC Engines apu2 experiences:

    I see. So are you saying that setting AES-NI+BSD in Misc. and just no hardware crypto in the OpenVPN client settings would be fine?

    That's what I would expect. In 2.4 at least.

    The last time I tested this though I achieved greatest throughput with both fields set to none or BSD with AES-NI disabled. Use AES-GCM and enable fastio and larger send/rec buffers.
    That was a while back, 2.3.4 vs 2.4.0.

    More testing is always good.

    Steve



  • @stephenw10 said in PC Engines apu2 experiences:

    I achieved greatest throughput with both fields set to none or BSD with AES-NI disabled

    What is a good throughput for APU2c4? My initial tests with openvpn server running on the apu2 (128-cbc) is around 20Mbit only... (150Mbit line max).
    But I havent done your test with all OFF.



  • @daemonix said in PC Engines apu2 experiences:

    @stephenw10 said in PC Engines apu2 experiences:

    I achieved greatest throughput with both fields set to none or BSD with AES-NI disabled

    What is a good throughput for APU2c4? My initial tests with openvpn server running on the apu2 (128-cbc) is around 20Mbit only... (150Mbit line max).
    But I havent done your test with all OFF.

    I currently get around 20Mbit as well, up and down, sometimes a bit more.

    I have a 400/40 Mbit line.

    I’ll need to double check my settings, but I think I have 256bit encryption.



  • @veldkornet said in PC Engines apu2 experiences:

    @daemonix said in PC Engines apu2 experiences:

    @stephenw10 said in PC Engines apu2 experiences:

    I achieved greatest throughput with both fields set to none or BSD with AES-NI disabled

    What is a good throughput for APU2c4? My initial tests with openvpn server running on the apu2 (128-cbc) is around 20Mbit only... (150Mbit line max).
    But I havent done your test with all OFF.

    I currently get around 20Mbit as well, up and down, sometimes a bit more.

    I have a 400/40 Mbit line.

    I’ll need to double check my settings, but I think I have 256bit encryption.

    pif Im getting 40Mbit max here. 128-GCM too.

    I found this page: https://github.com/ocochard/netbenches/blob/master/AMD_GX-412TC_4Cores_Intel_i210AT/openvpn/results/fbsd11.0/README.md

    But I cant see any luck. my test config below with APU2 running the openvpn server.

    0_1530530102103_Screenshot 2018-07-02 12.12.00.png

    0_1530530135777_Screenshot 2018-07-02 12.11.38.png

    0_1530530153820_Screenshot 2018-07-02 12.11.18.png

    EDIT: this page too: https://teklager.se/en/knowledge-base/apu2-vpn-performance/



  • Just wanted to find out how everyone is getting on with 2.4.4?

    I had Firmware 4.8.0.4 on my APU2 with 2.4.3 and everything was fine, but after upgrading to 2.4.4 I had lots of “stalls”, both via the web interface and via SSH. I often had to restart the SAH session (or refresh the web page).

    I downgraded the firmware all the way back down to 4.0.7, and the stalls seem to be gone, although I can’t exactly say that everything is as snappy as I’d expect it to be, changing screens on the GUI still take quite a bit of time.

    If I restart unbound, it seems to take a good 5 minutes before it actually starts resolving.
    I seem to have to restart PHP-FPM pretty often to get the interface in a working state (never had to do this before).

    Anyone else seeing this? Or whats your experience been so far with 2.4.4?


  • Netgate Administrator

    I'd be surprised if that was anything to do with the Coreboot version really. About the only thing I could imagine doing that would be some component that is initiallised differently and only supported in FreeBSD 11.2. But I'm not aware of that.
    I would first backup the config and do a clean 2.4.4 install. If you still see the same issues you did in the upgraded 2.4.4 then did deeper. I would expect to see errors logged though.

    Steve



  • I need a recommendation on a console cable for the APU1/APU2 units. My laptop will be retired soon and so I'll no longer have a serial port to use. I'm sure there are USB cables that will connect in. Does anyone have a link to one that they use that we know will work? Thanks for the help!



  • @stewart said in PC Engines apu2 experiences:

    I need a recommendation on a console cable for the APU1/APU2 units. My laptop will be retired soon and so I'll no longer have a serial port to use. I'm sure there are USB cables that will connect in. Does anyone have a link to one that they use that we know will work? Thanks for the help!

    I have this one, works well for me: https://www.startech.com/eu/m/Cards-Adapters/Serial-Cards-Adapters/USB-to-Null-Modem-RS232-DB9-Serial-Adapter-Cable-DCE-FTDI~ICUSB232FTN



  • @ Veldkornet
    Think this is the same thing?
    https://www.amazon.com/USB-Serial-Adapter-Modem-9-pin/dp/B008634VJY/ref=sr_1_3?ie=UTF8&qid=1539289152&sr=8-3&keywords=startech+usb+null+modem

    EDIT: Found the model on the box in the image. It is indeed. Thanks for the rec!



  • @stewart said in PC Engines apu2 experiences:

    @ Veldkornet
    Think this is the same thing?
    https://www.amazon.com/USB-Serial-Adapter-Modem-9-pin/dp/B008634VJY/ref=sr_1_3?ie=UTF8&qid=1539289152&sr=8-3&keywords=startech+usb+null+modem

    EDIT: Found the model on the box in the image. It is indeed. Thanks for the rec!

    Yup, looks like the same one indeed! :)