Netgate Discussion Forum
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Search
    • Register
    • Login

    Hardware support for Intel QuickAssist?

    Scheduled Pinned Locked Moved Hardware
    43 Posts 15 Posters 30.0k Views
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • ?
      Guest
      last edited by

      Pardon my lack of being more descriptive in my assumption. What I am trying to say is that if you have a more powerful CPU, such as an i5, i7 or Xeon then having AES-NI and quick assist may not be necessary since these CPU's can crunch AES far more capably. Again, AES-NI and QuickAssist are designed to aid a CPU in performing this task and this is why I believe it comes only on low power CPU's. If I'm wrong, please explain. I'm trying to learn something here.

      1 Reply Last reply Reply Quote 0
      • ?
        Guest
        last edited by

        It all depends on who is doing what for how many and where, as I see it right.

        And last buit not least it is more then a feature when the software you are using it is taking any kind
        of advantage of it. With AES-NI you will today get something around of the x4 or x5 throughput of your
        IPSec VPN and that is much in my eyes. And with OpenVPN 2.4 also OpenVPN will be getting more out
        by using it depending on the new (HMAC) inside. Link to that information

        Intel QuickAssist is coming in 2016 and then all peoples will be really able to use it or not likes he can do
        it by the presence inside of the hardware he is using. It is a hardware related function, as the hardware
        must be comes with Intel QuickAssist support or together with a add on card likes ADI or Netgate are
        offering them in the shops to go for because the Intel Xeon D-15x8, E3 and E5 CPUs are only supporting
        AES-NI and comes without Intel QuickAssist support.  Link to the Intel QuickAssist status

        This all can even be differ each from another, but are being also on the other side two different points
        AES-NI is in usage and runs good and so I will assume that it will also run very good for OpenVPN too.

        Gaming hardware comes often with AES-NI support based of its CPU that comes with it inside, but
        Intel QuickAssist is something that is more for servers or server grade hardware mostly used more
        in the professional area. And I am glad about the situation that Intel is willing and doing it right as
        today now, because they had one of this cards in earlier days, fu***ng hard to pay and it was then
        a lame duck that will never fly! With capabilities of 20 GBit/s to 50 GBit/s of encrypted or compressed
        packet flow we should all be sorted right and be lucky over that on top. For sure this is not for the cost
        that any home user will be able to go with, but there fore the Intel Atom C2x58 (Rangeley) will be strong
        enough. Please don´t forget that in many countries the hardware encryption or encryption in general will
        be prohibited by law! And so this peoples will be able over the Intel Atom C2x58 SoC to get also their nice
        VPN throughput accelerated fine as we all others.

        I don´t get it. The 8955 adapter costs $899 while the Atom C2000 processors have QuickAssist built-in for a fraction of the cost.

        Who goes with the Intel Atom SoC is not needing this adapter but all others who are using Intels Xeon
        D-15x8, E3 or E5 CPUs will be able to benefiting too from Intel QuickAssist too over that adapters.

        My assumption. It's very hard to get a new perspective unless you engage conversation. So, instead of giving me a link, why not explain why you think I'm wrong.

        AES-NI is not in really inside of all CPUs and Intel QuickAssist is also not available in gaming hardware
        and on top not done in Software likes DPDK (enabled software)!

        Again, AES-NI and QuickAssist are designed to aid a CPU in performing this task and this is why I believe it comes only on low power CPU's. If I'm wrong, please explain. I'm trying to learn something here.

        The Intel Atom C2x58 series is the only one I really know that comes beside with QuickAssist all others are
        only coming with the AES-NI inside. And please see the adapters that are not really in a home, SOHO or Pro
        range or area, it is more based on the enterprise or big data segment, base don the throughput numbers
        this will be not really matching smaller SoCs but more bigger CPU to handle that amount of stuff likes
        D-1500, E3- or E5 CPUs. See all the prices and then you will know that this will be not the same what
        is inside of the lower end Intel Atom CPUs or SoCs.
        ADI
        Intel
        Netgate
        On Amazon.com

        If I'm wrong, please explain.

        I really don´t think that this Intel Atom SoC will be able to handle the same load of this adapters above.
        But I am really lucky about that they are able to buy for anybody who want it. So if this might be only
        inserted inside of lower Atom SoCs why then this adapters are needed? It is more a server side think
        and not foe the end users with their lower end Atoms. You will need much more horse power to route and
        perform 20 GBit/s - 50 GBit/s of encrypted or compressed traffic then an Intel Atom will be able to realize.

        1 Reply Last reply Reply Quote 0
        • O
          oletuv
          last edited by

          @Blade:

          BlueKobold is correct although info is not readily apparent. The Xeon D 15x8 series does support AES-NI and QuickAssist Technology.

          Link 1

          http://www.servethehome.com/intel-xeon-d-15x8-networking-accelerated-skus/

          Link 2

          http://www.cpu-world.com/CPUs/Xeon_D/Intel-Xeon%20D-1518.html

          The above link shows AES instructions however it's identical to AES-NI.

          The Xeon D-15x8 SKUs do not have onboard QuickAssist acceleration according to Patrick Kennedy @ STH. Here´s what he replied to my question regarding QA:

          **Hi,

          The Xeon D does not have an onboard QAT accelerator so you need a Coleto Creek QAT PCIe card for Quick Assist with this generation.

          Regards,
          Patrick**

          1 Reply Last reply Reply Quote 0
          • ?
            Guest
            last edited by

            The Xeon D does not have an onboard QAT accelerator so you need a Coleto Creek QAT PCIe card for Quick Assist with this generation.

            +1 from me for that information! This would clarifying it and bringing it to the point.

            1 Reply Last reply Reply Quote 0
            • O
              oletuv
              last edited by

              @BlueKobold:

              The Xeon D does not have an onboard QAT accelerator so you need a Coleto Creek QAT PCIe card for Quick Assist with this generation.

              +1 from me for that information! This would clarifying it and bringing it to the point.

              Thanks. Since support for QuickAssist probably will be added to pfSense during 2016, I think a Atom C2758 SKU would be a better option for a dedicated pfSense box. Personally I´m going to order a prebuilt Mini-ITX with A1SRi-2758F motherboard from Supermicro. QuickAssist onboard and a lot cheaper than a Xeon D-15x8 based config too.

              1 Reply Last reply Reply Quote 0
              • ?
                Guest
                last edited by

                Thanks. Since support for QuickAssist probably will be added to pfSense during 2016,

                As I was getting it out of another thread here it will be 2016 but not really when and in which version!
                If in version 2.3 or 2.4 that was not clearly or directly told about.

                I think a Atom C2758 SKU would be a better option for a dedicated pfSense box.

                Yes this might be right but there will be a lack of DPDK (enabled software) and as I was thinking
                before the newer D-15x8 platforms were coming with all three things together likes AES-NI, QAT
                and DPDK, it owuld be for me and my self a more interesting solution as the Intel Atom C2x58 series.
                And that not only for private usage!!!! Also for many productive networks. But ok I can live with that
                status quo for now. Then I am going with the C2758 variant or the SG-8860 variant and the D-15x8
                would be better to add a QuickAssist adapter then if needed.

                Personally I´m going to order a prebuilt Mini-ITX with A1SRi-2758F motherboard from Supermicro. A lot cheaper than a Xeon D-15x8 based config too.

                Cheap was not my really concern, so it was nice to think on to build a very heavy and strong sorted firewall
                together with the M.2 MNVe SSD (Samsung950 Pro) and very fast RAM (DDR4-2133) and a 8C/16T SoC that
                is supporting all three things. (AES-NI, QAT & DPDK) related to be more future proof and the QAT adapter was
                more something what I was thinking for the higher level CPUs then likes E3 and E5 as an add on card. But again
                I am pretty sure the QAT support will be a real bomb for pfSense as the OpenVPN AES-GCM support too.

                So its really nice to know it now better, but I am also a little bit sad about that information.

                1 Reply Last reply Reply Quote 0
                • O
                  oletuv
                  last edited by

                  @BlueKobold:

                  –---
                  So its really nice to know it now better, but I am also a little bit sad about that information.

                  Yes, I was also very eager to buy a Xeon D-15x8 Mini-ITX pc and is very disappointed about the confirmation that the otherwise attractive Xeon D-15x8 processors don´t have QuickAssist onboard. Well, life will go on, sort of  :D

                  1 Reply Last reply Reply Quote 0
                  • ?
                    Guest
                    last edited by

                    this may be redundant by this point but I want to say that I'm personally a little bit disenfranchised with the AES-NI naming convention. It suggests that it is something new entirely when in this is not the case when in all actuality, it is more of an addition. I wish Intel would have provided a better name for this new tech but for me it helps to think of it this way; AES + NI = AES-NI because the chip is using the added instructions to assist AES.

                    1 Reply Last reply Reply Quote 0
                    • O
                      oletuv
                      last edited by

                      @oletuv:

                      Since support for QuickAssist probably will be added to pfSense during 2016, I think a Atom C2758 SKU would be a better option for a dedicated pfSense box. Personally I´m going to order a prebuilt Mini-ITX with A1SRi-2758F motherboard from Supermicro. QuickAssist onboard and a lot cheaper than a Xeon D-15x8 based config too.

                      Hm.. Common sense tells me that Atom C2758 with onboard QuickAssist acceleration is a better option than the more costly Xeon D-15x8 with no onboard QuickAssist acceleration for a pfSense build. However, Atom Rangeley is an older processor released Q3/13 while the Xeon D-15x8 processors are newly released.

                      I suppose the new Xeon D processors will outperform Atom Rangeley except for crypto-heavy stuff like VPN. I will configure my pfSense firewall with IPSEC or OpenVPN, but will probably use it infrequently, typically to access my home network in Norway from my Spanish home.

                      The Supermicro X10SDV-6C+-TLN4F board (Xeon D-1528) with active CPU-cooling looks extremely tempting. ;)

                      Frank, what do you think?  ;D

                      1 Reply Last reply Reply Quote 0
                      • ?
                        Guest
                        last edited by

                        Hm.. Common sense tells me that Atom C2758 with onboard QuickAssist acceleration is a better option than the more costly Xeon D-15x8 with no onboard QuickAssist acceleration for a pfSense build.

                        For SMB or home usage it will be easy to answer, for sure it is likes you were telling!

                        However, Atom Rangeley is an older processor released Q3/13 while the Xeon D-15x8 processors are newly released.

                        It is also the CPU design and the circumstance that not each cpu core is comparable to any other cpu core.
                        If the netmap-fwd, QAT and perhaps DPDK over AVX/AVX2 registers will be available it could really be that
                        the Intel Atom C2000 (Rangely) platform will be attractive as on its first release day or much more then this.

                        I suppose the new Xeon D processors will outperform Atom Rangeley except for crypto-heavy stuff like VPN.

                        QAT, netmap-fwd and OpenVPN 2.4 in pfSense 2.3 will be able to change many things.

                        I will configure my pfSense firewall with IPSEC or OpenVPN, but will probably use it infrequently, typically to access my home network in Norway from my Spanish home.

                        As for now I am still using IPSec VPN (AES-GCM) it is the best supported VPN form the AES-NI CPU instructions.
                        If you have many side-to-side VPNs it will be really useful to get the best performance as it is able to realize
                        and with AES-NI it is able to get something around the x4 or x5 of the normal throughput. Again, this can be
                        turned around if the intel QAT is in usage inside of pfSense and then the cards will mixed up new again.

                        Also the iOS devices from apple are coming together with IPSec APPs and it is a fine thing if on both
                        side a pfSense firewall is able to use the AES-NI to speed up the entire IPSec tunnel.

                        The Supermicro X10SDV-6C+-TLN4F board (Xeon D-1528) with active CPU-cooling looks extremely tempting. ;)

                        M.2 SSD slot, 2 x 10 GbE ports, many CPU core, HT and TurboBoost, DDR4-2133MHz UDIMM support
                        PCIe 3.0 x16 what should I say a really nice platform to go with and the ability to add if needed

                        • Chelsio 2 Port 10 GbE NIC
                        • Netgate Intel QAT Adapter
                        • Intel QAT Adapter with 4 GBit/s LAN Ports

                        Frank, what do you think?  ;D

                        For home and for SMB usage go with the Intel Atom C2x58 SoC or similar from Negate store or the
                        pfSense store. If you will needing more horse power and/or throughput the Xeon D-15x8 will be a
                        really nice option to know too.

                        1 Reply Last reply Reply Quote 0
                        • R
                          reggie14
                          last edited by

                          @oletuv:

                          @oletuv:

                          Since support for QuickAssist probably will be added to pfSense during 2016, I think a Atom C2758 SKU would be a better option for a dedicated pfSense box. Personally I´m going to order a prebuilt Mini-ITX with A1SRi-2758F motherboard from Supermicro. QuickAssist onboard and a lot cheaper than a Xeon D-15x8 based config too.

                          Hm.. Common sense tells me that Atom C2758 with onboard QuickAssist acceleration is a better option than the more costly Xeon D-15x8 with no onboard QuickAssist acceleration for a pfSense build. However, Atom Rangeley is an older processor released Q3/13 while the Xeon D-15x8 processors are newly released.

                          Just be aware that QAT support might not be coming to the C2758.

                          When asked about QAT support earlier today, gonzopancho wrote:

                          "When it's done." Maybe 2.4, and then maybe only for 895x and newer.
                          I'm still not decided if it will go in the community edition.

                          1 Reply Last reply Reply Quote 0
                          • MikeV7896M
                            MikeV7896
                            last edited by

                            All I'll say is I hope that isn't the case, both for the devices supported (C2x58 processors should be supported!) and for the lack of presence in the community edition. I have a lot more that I want to say, but I'll refrain for the time being, since there's still lots of time before it sees any sort of daylight in a released fashion.

                            The S in IOT stands for Security

                            1 Reply Last reply Reply Quote 0
                            • K
                              Keljian
                              last edited by

                              So much FUD…

                              Quickassist is a Very Good Thing TM IF you are pushing >20GB across VPNs or are doing extensive UTM type tasks. Or if you are looking to get line rate over 10-40Gbit ..

                              This is unnecessary for home use at this time.

                              It is safe to say that with AES-NI most desktop grade processors can push sufficient packets for up to about 5 Gigabit/s worth of speed  (source: https://calomel.org/aesni_ssl_performance.html - Note these are based on a single core) - For example an i3-6100 should be able to stretch to about 20Gbps 256bit AES-CBC in a pinch.

                              For a home connection, this is more than sufficient. You need 15-20Mbit for 4k Streaming, so at 100mbps you can handle 4-5 streams happily - there is plenty of hardware out there that can push this, encrypted if you want.

                              While I appreciate having "all the cool stuff" is nice, it is totally unnecessary for home users of pfSense and holding off purchasing hardware based on whether QAT is supported is silly if your use case is "in the home".

                              If on the other hand it is "in the office" then you should be buying supported hardware (you are aren't you?) from the pfSense store or Netgate, preferably the former, to support the project, and in turn get support for things like QAT.

                              If you are NOT buying supported hardware then if you are using it in an office environment you should be paying for support, and realistically using server grade xeon hardware which will have AES-NI and Lotsa CoresTM so speed will become largely irrelevant - AND you should be able to afford a coleto creek card if you need huge bandwidth..

                              1 Reply Last reply Reply Quote 0
                              • R
                                reggie14
                                last edited by

                                @Keljian:

                                So much FUD…

                                Quickassist is a Very Good Thing TM IF you are pushing >20GB across VPNs or are doing extensive UTM type tasks. Or if you are looking to get line rate over 10-40Gbit ..

                                This is unnecessary for home use at this time.

                                It is safe to say that with AES-NI most desktop grade processors can push sufficient packets for up to about 5 Gigabit/s worth of speed  (source: https://calomel.org/aesni_ssl_performance.html - Note these are based on a single core) - For example an i3-6100 should be able to stretch to about 20Gbps 256bit AES-CBC in a pinch.

                                For a home connection, this is more than sufficient. You need 15-20Mbit for 4k Streaming, so at 100mbps you can handle 4-5 streams happily - there is plenty of hardware out there that can push this, encrypted if you want.

                                While I appreciate having "all the cool stuff" is nice, it is totally unnecessary for home users of pfSense and holding off purchasing hardware based on whether QAT is supported is silly if your use case is "in the home".

                                I largely agree with this conclusion.  Except some of your figures are quite a bit off.  As one example, AES-CBC performance isn't what you should be looking at.  AES-GCM is.  And that's assuming you're using AES-GCM for message authentication.  If you're using a legacy suite based on AES-CBC and HMAC-SHA1, then HMAC-SHA1 will become a performance issue.  This is an issue for OpenVPN, which still doesn't have GCM in their stable release.  A home user with a high-speed connection running over OpenVPN on a C2758 might get a benefit from QAT, although there are other performance bottlenecks with OpenVPN that probably become an issue first.

                                So yes, I certainly agree QAT is unnecessary for all but the most extraordinary home use cases. If you're really in that boat, then it probably makes sense to get something faster than a Rangeley.

                                1 Reply Last reply Reply Quote 0
                                • M
                                  manaox2
                                  last edited by

                                  QAT was originally in the road map for 3.0 back in February of 2015, so I can be patient. I hope it does not require a hardware purchase to receive support, as many people are looking forward to this for home usage after being recommended so much on the forums.

                                  I wish I that knew then what I know now, I agree that I'll probably need something more powerful than my c2758 system which is a shame.

                                  Still looking forward to seeing the results of the DevSummit 2016 in June where they are working on the QAT FreeBSD driver port. I would like to squeeze as much as possible running both traffic through OpenVPN while using Suricata at home.

                                  https://twitter.com/gonzopancho/status/715262054832033792

                                  #Intel #QuickAssist driver update at #BSDCan (#FreeBSD dev summit) https://wiki.freebsd.org/DevSummit/201606#Schedule-1 … Likely to be ready for #pfSense 2.4

                                  1 Reply Last reply Reply Quote 0
                                  • ?
                                    Guest
                                    last edited by

                                    This is an issue for OpenVPN, which still doesn't have GCM in their stable release.

                                    This will be there in the OpenVPN version 2.4 and this could be going into the pfSense version of
                                    2.3 or 2.4 stable. So the OpenVPN users might be getting the same benefit from that AES-NI
                                    CPU registers likes the IPSec users that get something about 400% speed improvement.

                                    But the Intel QAT will be not only doing crypto work it is also for compression and decompression
                                    and this might be perhaps also a real gain and benefit that could be really interesting, either for
                                    home or professional usage. If it is in action on both sides I pretty sure it will be speed up more
                                    then only the encryption part.

                                    I hope it does not require a hardware purchase to receive support,

                                    Could be perhaps for the first one year that pfSense shop or Netgate customers are getting some more
                                    benefit from their units, but all after all I would surely also prefer and love it to see that it finds its way into
                                    the community version of pfSense.

                                    as many people are looking forward to this for home usage after being recommended so much on the forums.

                                    Me too.

                                    I wish I that knew then what I know now, I agree that I'll probably need something more powerful than my c2758 system which is a shame.

                                    For a home usage that board will be really well as I see it right and I also think about it, that we all could
                                    not imagine the real or full potential that can be unleashed. And mostly if many things comes together, the
                                    impact is more then only one long awaited thing. With netmap-fwd, QAT, OpenVPN 2.4, perhaps later DPDK
                                    over AVX/AVX2 registers of that SoC and multi core usage in PPPoE at the WAN port it must be really rocking.

                                    while using Suricata at home.

                                    In former times Intel was announcing that IDS/IPS software will also benefit from that QAT,
                                    but this statement was then taken back from the public and so only the encryption and
                                    compression part are only profiting from that QAT only.

                                    1 Reply Last reply Reply Quote 0
                                    • Z
                                      zanthos
                                      last edited by

                                      Anyone have seen this:
                                      Lanner AV-ICE01 - VPN Acceleration Card with Intel® Cave Creek DH8910CC
                                      Lanner AV-ICE02 - VPN Acceleration Card with Intel® Coleto Creek 8925/8950
                                      Lanner AV-ICE04 - The Gen. 3 PCIe x8 Network Processing/Acceleration Card with Intel Coleto Creek 8955 PCH

                                      So far I've been offered:
                                      AV-ICE01 ~250€
                                      AV-ICE02 ~440€

                                      I think the AV-ICE01 would be a real deal breaker. Up to 10Gbps hardware offload assistance should be enough for most of us…
                                      Therefore I hope the upcoming implementation of Intel QAT in FreeBSD will support Intel Communication Chipset 8910 Series.

                                      1 Reply Last reply Reply Quote 0
                                      • ?
                                        Guest
                                        last edited by

                                        @Zanthos

                                        I first only was finding the ADI and Netgate boards at a higher price point.
                                        Cryptographic Accelerator CPIC Adapter 8955 with QuickAssist
                                        CPIC: Intel 8920/8955

                                        But now I found also a plugin module that will be fitting right, but only for some
                                        appliances from the same vendor! And yes they are not really low in price too. :-[
                                        No price labeling was there to get a good overview, but nice and interesting looking.
                                        [url=http://www.axiomtek.de/Default.aspx?MenuId=Products&FunctionId=ProductView&ItemId=15145&upcat=233]Axiomtek NA361R
                                        Axiomtek NA570
                                        Axiomtek NA552
                                        Axiomtek VPN Module

                                        1 Reply Last reply Reply Quote 0
                                        • S
                                          shelbystripes
                                          last edited by

                                          @Blade:

                                          BlueKobold is correct although info is not readily apparent. The Xeon D 15x8 series does support AES-NI and QuickAssist Technology.

                                          Link 1

                                          http://www.servethehome.com/intel-xeon-d-15x8-networking-accelerated-skus/

                                          Link 2

                                          http://www.cpu-world.com/CPUs/Xeon_D/Intel-Xeon%20D-1518.html

                                          The above link shows AES instructions however it's identical to AES-NI.

                                          I want to point out that the STH page has a slide showing "Intel QuickAssist Technology Crypto Accelerator (Coleto Creek) Support". It doesn't say QuickAssist is integrated into any Xeon D models. "Coleto Creek" is Intel's code name for its 8950-series PCIe QuickAssist accelerator, which is a standalone chip typically sold on a PCIe add-in card. Intel also usually doesn't use the word "support" to describe an integrated feature, they use it to mean compatibility with external hardware or software.

                                          It sounds like Intel is just saying the Xeon-D works with an 8950 card if you want QuickAssist. That implies that QuickAssist is NOT in the CPU/SoC itself. And the cpuworld link doesn't show QuickAssist built-in either… I wish it did, but nothing I can find says it's actually there.

                                          1 Reply Last reply Reply Quote 0
                                          • J
                                            jwt Netgate
                                            last edited by

                                            @shelbystripes:

                                            @Blade:

                                            BlueKobold is correct although info is not readily apparent. The Xeon D 15x8 series does support AES-NI and QuickAssist Technology.

                                            Link 1

                                            http://www.servethehome.com/intel-xeon-d-15x8-networking-accelerated-skus/

                                            Link 2

                                            http://www.cpu-world.com/CPUs/Xeon_D/Intel-Xeon%20D-1518.html

                                            The above link shows AES instructions however it's identical to AES-NI.

                                            I want to point out that the STH page has a slide showing "Intel QuickAssist Technology Crypto Accelerator (Coleto Creek) Support". It doesn't say QuickAssist is integrated into any Xeon D models.

                                            True, it doesn't say that….. I can't say more.

                                            @shelbystripes:

                                            "Coleto Creek" is Intel's code name for its 8950-series PCIe QuickAssist accelerator,

                                            Technically, Coleto Creek also includes some DH892x models. 
                                            http://ark.intel.com/products/codename/60172/Coleto-Creek#@Embedded

                                            Confusingly, the previous generation (Cave Creek) includes the DH8920
                                            http://ark.intel.com/products/codename/44946/Cave-Creek#@Embedded

                                            @shelbystripes:

                                            which is a standalone chip typically sold on a PCIe add-in card.

                                            Like this: http://store.netgate.com/ADI/QuickAssist8955.aspx

                                            @shelbystripes:

                                            Intel also usually doesn't use the word "support" to describe an integrated feature, they use it to mean compatibility with external hardware or software.

                                            It sounds like Intel is just saying the Xeon-D works with an 8950 card if you want QuickAssist. That implies that QuickAssist is NOT in the CPU/SoC itself. And the cpuworld link doesn't show QuickAssist built-in either… I wish it did, but nothing I can find says it's actually there.

                                            yeah, well…note that every QuickAssist part is a Platform Controller Hub, and that when you see these in a PCIe card form factor, they're being used in "end-point" mode.  Xeon-D (Broadwell-DE) has an integrated PCH.

                                            See if you can't piece it together from there.

                                            1 Reply Last reply Reply Quote 0
                                            • First post
                                              Last post
                                            Copyright 2025 Rubicon Communications LLC (Netgate). All rights reserved.