Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    PC Engines apu2 experiences

    Scheduled Pinned Locked Moved Hardware
    711 Posts 73 Posters 777.2k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • D
      dugeem @Qinn
      last edited by

      @Qinn said in PC Engines apu2 experiences:

      2.Disable ICMP Redirects to enable tryforward routing path (under System / Advanced / System Tunables set net.inet.ip.redirect & net.inet6.ip6.redirect to 0)

      As hinted above FreeBSD has for years incorporated a fast routing path which speeds packet forwarding. Prior to pfSense 2.3 it was the ip_fastforward path - which then was changed to ip_tryforward with pfSense 2.3 (FreeBSD 10.3). Now that a bug in ICMP Redirect handling in the fast path has been fixed it is necessary to change the specified ICMP Redirects sysctls to enable the fast routing path.

      3.Add hw.igb.rx_process_limit =-1 to /boot/loader.conf.local

      Sysctl hw.igb.rx_process_limit sets the limit number of received packets which can consecutively handled by igb driver. Default is 100. On multicore systems this can usually be set to unlimited (value -1).

      Personally I've had no problems but as per recent posts from @fireodo this has caused issues with PPPoE.

      QinnQ 1 Reply Last reply Reply Quote 1
      • K
        kevindd992002 @dugeem
        last edited by

        @dugeem said in PC Engines apu2 experiences:

        @kevindd992002 said in PC Engines apu2 experiences:

        By the way, why can't ICMP redirect be enabled at the same time as tryforward path?

        This is a FreeBSD kernel restriction. I think it comes from concern that if you have to start generating ICMP packets from within the fast routing path then you can potentially overwhelm gateways with ICMP redirects.

        Also, if tryforward was used ever since does that mean ICMP redirect was disabled by default back then?

        Prior to FreeBSD 10.3 ip_tryforward() there was ip_fastforward() routing path - which had to be explicitly enabled by setting sysctl net.inet.ip.fastforwarding=1. pfSense has always had ICMP Redirects enabled - although they were not working from 2.3 to 2.4.4 due to upstream bug with FreeBSD. This is now fixed in pfSense 2.4.5

        Ok, so if I understand that right, pfsense never used ip_tryforward (pfsense 2.3 and above) or ip_fastforward (pfsense versions prior to 2.3) by default because it had ICMP Redirects enabled, correct? You need to enable them explicitly in the system tunables if you want to use them.

        D 1 Reply Last reply Reply Quote 0
        • D
          dugeem @kevindd992002
          last edited by

          @kevindd992002 said in PC Engines apu2 experiences:

          pfsense never used ip_tryforward (pfsense 2.3 and above) or ip_fastforward (pfsense versions prior to 2.3) by default because it had ICMP Redirects enabled

          The upstream bug was actually that the FreeBSD kernel didn't check the ICMP Redirect sysctls - which meant that ip_tryforward was always used and that ICMP Redirects did not work on FreeBSD 10.3 (pfsense 2.3) thru FreeBSD 11.2 (pfSense 2.4.4). So the pfSense defaults for these sysctls had no bearing on this issue until upstream fix was implemented and pfSense switched to that release (FreeBSD 11-STABLE).

          Here is the actual fix applied to FreeBSD kernel: https://svnweb.freebsd.org/base/stable/11/sys/netinet/ip_input.c?r1=332513&r2=338343&pathrev=338343

          K 1 Reply Last reply Reply Quote 0
          • QinnQ
            Qinn @dugeem
            last edited by Qinn

            @dugeem Kudos so tweak 2 only favors, if you use ipv6?

            Hardeware: Intel(R) Celeron(R) J4125 CPU @ 2.00GHz 102 GB mSATA SSD (ZFS)
            Firmware: Latest-stable-pfSense CE (amd64)
            Packages: pfBlockerNG devel-beta (beta tester) - Avahi - Notes - Ntopng - PIMD/udpbroadcastrelay - Service Watchdog - System Patches

            D 1 Reply Last reply Reply Quote 0
            • K
              kevindd992002 @dugeem
              last edited by

              @dugeem said in PC Engines apu2 experiences:

              @kevindd992002 said in PC Engines apu2 experiences:

              pfsense never used ip_tryforward (pfsense 2.3 and above) or ip_fastforward (pfsense versions prior to 2.3) by default because it had ICMP Redirects enabled

              The upstream bug was actually that the FreeBSD kernel didn't check the ICMP Redirect sysctls - which meant that ip_tryforward was always used and that ICMP Redirects did not work on FreeBSD 10.3 (pfsense 2.3) thru FreeBSD 11.2 (pfSense 2.4.4). So the pfSense defaults for these sysctls had no bearing on this issue until upstream fix was implemented and pfSense switched to that release (FreeBSD 11-STABLE).

              Here is the actual fix applied to FreeBSD kernel: https://svnweb.freebsd.org/base/stable/11/sys/netinet/ip_input.c?r1=332513&r2=338343&pathrev=338343

              Ok and for home networks where only one router/firewall (pfsense) is used anyway, ICMP Redirects aren't really being used, correct?

              D 1 Reply Last reply Reply Quote 0
              • D
                dugeem @Qinn
                last edited by

                @Qinn

                If you only use IPv4 then you just need to set net.inet.ip.redirect=0

                If you use both IPv4 & IPv6 then you just need to set net.inet.ip.redirect=0 & net.inet6.ip6.redirect=0

                QinnQ 1 Reply Last reply Reply Quote 1
                • D
                  dugeem @kevindd992002
                  last edited by

                  @kevindd992002 said in PC Engines apu2 experiences:

                  for home networks where only one router/firewall (pfsense) is used anyway, ICMP Redirects aren't really being used, correct

                  Yes. ICMP Redirects are generally only used where there are two or more routers accessible on an IP network.

                  1 Reply Last reply Reply Quote 0
                  • QinnQ
                    Qinn @dugeem
                    last edited by Qinn

                    @dugeem said in PC Engines apu2 experiences:

                    @Qinn

                    If you only use IPv4 then you just need to set net.inet.ip.redirect=0

                    If you use both IPv4 & IPv6 then you just need to set net.inet.ip.redirect=0 & net.inet6.ip6.redirect=0

                    Done it. I looked for boot.conf.local in /boot ,but only found boot.conf? Do I overlook something?

                    btw do these tweaks need a reboot?

                    Hardeware: Intel(R) Celeron(R) J4125 CPU @ 2.00GHz 102 GB mSATA SSD (ZFS)
                    Firmware: Latest-stable-pfSense CE (amd64)
                    Packages: pfBlockerNG devel-beta (beta tester) - Avahi - Notes - Ntopng - PIMD/udpbroadcastrelay - Service Watchdog - System Patches

                    K 1 Reply Last reply Reply Quote 0
                    • K
                      kevindd992002 @Qinn
                      last edited by

                      @Qinn said in PC Engines apu2 experiences:

                      @dugeem said in PC Engines apu2 experiences:

                      @Qinn

                      If you only use IPv4 then you just need to set net.inet.ip.redirect=0

                      If you use both IPv4 & IPv6 then you just need to set net.inet.ip.redirect=0 & net.inet6.ip6.redirect=0

                      Done it. I looked for boot.conf.local in /boot ,but only found boot.conf? Do I overlook something?

                      btw do these tweaks need a reboot?

                      You need to create the file if it isn't there (by default it isn't).

                      QinnQ 1 Reply Last reply Reply Quote 1
                      • QinnQ
                        Qinn @kevindd992002
                        last edited by

                        @kevindd992002 Done it!

                        Cheers Qinn

                        Hardeware: Intel(R) Celeron(R) J4125 CPU @ 2.00GHz 102 GB mSATA SSD (ZFS)
                        Firmware: Latest-stable-pfSense CE (amd64)
                        Packages: pfBlockerNG devel-beta (beta tester) - Avahi - Notes - Ntopng - PIMD/udpbroadcastrelay - Service Watchdog - System Patches

                        1 Reply Last reply Reply Quote 0
                        • S
                          stefanl
                          last edited by

                          I've changed my settings as @dugeem mentioned and I see a somewhat lower CPU load and my network stays usable even when my Tor relay is doing 300mbit in/out.

                          37cc4650-19c8-4582-a782-d6951d67a3bd-image.png

                          Next step is to see if I can hookup my fiber connection (PPPoE) directly to pfSense and maintain those speeds.

                          QinnQ 1 Reply Last reply Reply Quote 2
                          • QinnQ
                            Qinn @stefanl
                            last edited by Qinn

                            @stefanl as I see you use a UPS, may I give you some advise?

                            you are using ufs as a filesystem, I would move over to ZFS, as It will give you much more security. ZFS requires a system with ECC memory, otherwise you're still not 100% safe and the APU2C4 has ECC memory. So this, using ZFS, will guard you from filesystems damage etc. when a power surge comes along.

                            Cheers Qinn

                            Hardeware: Intel(R) Celeron(R) J4125 CPU @ 2.00GHz 102 GB mSATA SSD (ZFS)
                            Firmware: Latest-stable-pfSense CE (amd64)
                            Packages: pfBlockerNG devel-beta (beta tester) - Avahi - Notes - Ntopng - PIMD/udpbroadcastrelay - Service Watchdog - System Patches

                            QinnQ 1 Reply Last reply Reply Quote 1
                            • QinnQ
                              Qinn @Qinn
                              last edited by

                              ...and I would install Service_Watchdog, that way all you services are always up and running.

                              Hardeware: Intel(R) Celeron(R) J4125 CPU @ 2.00GHz 102 GB mSATA SSD (ZFS)
                              Firmware: Latest-stable-pfSense CE (amd64)
                              Packages: pfBlockerNG devel-beta (beta tester) - Avahi - Notes - Ntopng - PIMD/udpbroadcastrelay - Service Watchdog - System Patches

                              S 1 Reply Last reply Reply Quote 0
                              • K
                                kevindd992002
                                last edited by

                                For the apu2c4, would IPSec be faster than OpenVPN for site-to-site VPN? I have OpenVPN configured now but I'm not sure if it's wise to switch to IPSec as I read that it's almost always faster than OpenVPN.

                                C 1 Reply Last reply Reply Quote 0
                                • C
                                  cysiacom @kevindd992002
                                  last edited by

                                  @kevindd992002 On my recent experience it is 2-3 times faster, with warnings on these numbers.
                                  On lab with this settings, using just a gigabit link between both APUs simulating a WAN link:

                                  PC1iPerf3 Server---LAN --- APU3C4 --- WAN (Gigabit LAN) --- APU4D4 -- LAN---PC2iPerf3 Client

                                  iPerf3 test were made just plain, no specific parameters apart from -P10 to simulate 10 concurrent data streams.
                                  After all tuning, the faster I get is 89-92 Mb/s on OpenVPN and 240-250 Mb/s on IPsec on the same link.
                                  Be aware that my IPSec config was made to accomplish a link with a provider that request the lowest crypto-settings available, so OpenVPN and IPSec are not 1:1 comparable on crypto-computing requirements .

                                  OpenVPN: TLS Auth - 128-GCM- DH2
                                  Ipsec; Phase1: PSK-3DES-SHA1-DH2 and Phase2: 3DES-SHA1

                                  Maybe if requirements are higher on IPSec the numbers will get closer. I don't know.
                                  I had been deploying OpenVPN Site2Site links without giving IPSec a chance. Don't ask me why. Stubborn on OpenVPN maybe.
                                  Now we're retesting our configurations and moving them to IPSec if all test are successful and no other requirements appears.

                                  K 1 Reply Last reply Reply Quote 0
                                  • K
                                    kevindd992002 @cysiacom
                                    last edited by

                                    @cysiacom said in PC Engines apu2 experiences:

                                    @kevindd992002 On my recent experience it is 2-3 times faster, with warnings on these numbers.
                                    On lab with this settings, using just a gigabit link between both APUs simulating a WAN link:

                                    PC1iPerf3 Server---LAN --- APU3C4 --- WAN (Gigabit LAN) --- APU4D4 -- LAN---PC2iPerf3 Client

                                    iPerf3 test were made just plain, no specific parameters apart from -P10 to simulate 10 concurrent data streams.
                                    After all tuning, the faster I get is 89-92 Mb/s on OpenVPN and 240-250 Mb/s on IPsec on the same link.
                                    Be aware that my IPSec config was made to accomplish a link with a provider that request the lowest crypto-settings available, so OpenVPN and IPSec are not 1:1 comparable on crypto-computing requirements .

                                    OpenVPN: TLS Auth - 128-GCM- DH2
                                    Ipsec; Phase1: PSK-3DES-SHA1-DH2 and Phase2: 3DES-SHA1

                                    Maybe if requirements are higher on IPSec the numbers will get closer. I don't know.
                                    I had been deploying OpenVPN Site2Site links without giving IPSec a chance. Don't ask me why. Stubborn on OpenVPN maybe.
                                    Now we're retesting our configurations and moving them to IPSec if all test are successful and no other requirements appears.

                                    I don't understand though, why is there a "provider" involved between the IPSec tunnel that poses these crypto settings requirement? I thought it's just like OpenVPN where one end would be the VPN server and the other end the client?

                                    C 1 Reply Last reply Reply Quote 0
                                    • S
                                      stefanl @Qinn
                                      last edited by

                                      @Qinn Thanks for the tips. I don't think I'll reinstall pfSense to a single disk ZFS. My NAS haves it, but it haves 12x 4TB disks.

                                      Regarding watchdog, it's enabled and working, but somehow it isn't showing on the dashboard nor in status --> services.

                                      QinnQ 1 Reply Last reply Reply Quote 0
                                      • C
                                        cysiacom @kevindd992002
                                        last edited by

                                        @kevindd992002 I'm afraid I didn't explain myself clearly.
                                        Sorry for that.

                                        One customer needed to access to their provider servers.
                                        The access is granted via Site2Site IPsec VPN with very low crypto requirements and also quite unusual IPs (they use "fake" public IPs inside the tunnel).
                                        We had some trouble getting Phase2 working for that customer and provider so we planned some lab tests.
                                        There's no need for provider at all in general but just for this case, our customer, in particular.

                                        When doing local tests we did realize the speed change on IPSec tunnels compared to OpenVPN so we did some other test for our own purposes.
                                        Again.

                                        There's no need for external or any provider. It was just only on our specific customer needs.

                                        K 1 Reply Last reply Reply Quote 0
                                        • K
                                          kevindd992002 @cysiacom
                                          last edited by kevindd992002

                                          @cysiacom said in PC Engines apu2 experiences:

                                          @kevindd992002 I'm afraid I didn't explain myself clearly.
                                          Sorry for that.

                                          One customer needed to access to their provider servers.
                                          The access is granted via Site2Site IPsec VPN with very low crypto requirements and also quite unusual IPs (they use "fake" public IPs inside the tunnel).
                                          We had some trouble getting Phase2 working for that customer and provider so we planned some lab tests.
                                          There's no need for provider at all in general but just for this case, our customer, in particular.

                                          When doing local tests we did realize the speed change on IPSec tunnels compared to OpenVPN so we did some other test for our own purposes.
                                          Again.

                                          There's no need for external or any provider. It was just only on our specific customer needs.

                                          No worries, I see what you mean now. Would you happen to have any best practice guide in setting up an IPSec tunnel in pfsense, at least for a home setup with "relaxed but secure" crypto requirements?

                                          As for Remote Access VPN, is OpenVPN still the way to go? There's nothing stopping me of creating an IPSec s2s link and an OpenVPN remote access VPN gateway, right?

                                          1 Reply Last reply Reply Quote 0
                                          • QinnQ
                                            Qinn @stefanl
                                            last edited by Qinn

                                            @stefanl said in PC Engines apu2 experiences:

                                            @Qinn Thanks for the tips. I don't think I'll reinstall pfSense to a single disk ZFS. My NAS haves it, but it haves 12x 4TB disks.

                                            Regarding watchdog, it's enabled and working, but somehow it isn't showing on the dashboard nor in status --> services.

                                            I thought and have the same, this service isn't present in the services status, but if you stop 1 service, you will see it will come up again (if you have an mail address setup for notices, you should also receive a mail. Maybe read https://forum.netgate.com/topic/59761/new-package-service-watchdog/9

                                            Hardeware: Intel(R) Celeron(R) J4125 CPU @ 2.00GHz 102 GB mSATA SSD (ZFS)
                                            Firmware: Latest-stable-pfSense CE (amd64)
                                            Packages: pfBlockerNG devel-beta (beta tester) - Avahi - Notes - Ntopng - PIMD/udpbroadcastrelay - Service Watchdog - System Patches

                                            1 Reply Last reply Reply Quote 0
                                            • First post
                                              Last post
                                            Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.