Netgate refuses to stand behind performance claims with *full* money-back guarantee
-
I am very interested in upgrading from my SG-3100 to a 6100 to take advantage of the 1200+ Mbps speeds now offered by my ISP. Based on conversations with Netgate sales, it appears the router should be able to easily handle such bandwidth. However, before I drop $699 + tax and shipping on a new router (and go through the hassle of selling my old one), I asked Netgate for assurance that I would get 1.2+ Gbps bandwidth through the 2.5 Gbps NICs on the 6100, using an OOTB configuration of 21.x pfSense. (Currently, I get 940 Mbps (*) bandwidth from my cable modem through my SG-3100 to a laptop connected to it, and that same laptop gets 1200+ Mbps when directly connected to my cable modem). Again, the specs for the 6100 indicate that it should trivially be able to offer this bandwidth (to attached devices with appropriate NICs, which both my cable modem and laptop have) -- and in all likelihood this would be an utter non-issue.
I am willing to buy the 6100, but I asked Netgate to offer me a full refund, including shipping, should the router fail to meet the advertised bandwidth specs. I am not asking them to make this guarantee based on a random configuration -- but on a stock vanilla install of 21.x with only my cable modem and laptop attached (a scenario which gets line speed easily with 2.4.5p1 on the SG-3100). They refused -- not only would I not be refunded shipping, but I'd have to pay return shipping plus an egregious 25% restocking fee. I had conversations all the way up to their VP level, and they refused to budge. Since the increased performance is the ONLY reason I want the 6100, I cannot take a $200+ risk that it does not work as promised. I offered to test out an evaluation model or open box 6100, and they refused that too. I just can't see dealing with a company that refuses to stand behind its advertising. So now I am compelled to consider an entirely alternate vendor.
I am extremely disappointed and frankly stunned at Netgate's attitude.
(*) I get full line speed on my SG-3100 when running 2.4.5p1, but 21.05 has an acknowledged performance bug on the SG-3100 that lowers the performance to 825 Mbps or so. Netgate says this will be fixed in 21.09.
-
I won't make excuses for the NetGate team, nor their VP...
But I would like to add that even when dealing with $250k nextgen firewalls, performance is rarely what is advertised. The key metric is "Threat-prevention throughput", and that metric is always orders of magnitude slower than raw speed (i.e., single any-any rule, routed traffic). Quality firewall vendors use a large PCAP file of "enterprise mix" traffic to get realistic performance values, but of course, your particular network/environment may vary in traffic composition, affecting performance.
The LT;DR of it is that real-world performance will always be lower than lab-tested performance, because there are a multitude of different variables the vendor cannot account for. Always buy a bit more performance than you really need.
-
@msf2000 The specs for the 6100 far exceed 1.2 Gbps bandwidth, which make it all the more perplexing that Netgate refuses to back them up with a full money-back guarantee.
-
I’m not sure what you all are really asking for. Netgte provides specs that simulate lab (iperf) and real world (imix) experiences. What Netgate cannot control are other factors that would influence network performance such as additional packages, issues with switches, cables, other devices, etc. The number of time I’ve read posts of people who have misconfigured things like Suricata/Snort, PfBlocker, or other packages and then had performance issues, devices rebooting, etc are endless. Not to mention all sorts of threads where people have bad cables or straight up misconfigured their device. What you are demanding is for Netgate to guarantee that you’re not one of those that screws things up and then complains about the device. Are you expecting them to spend hours verifying that your configuration correct? I’m sure someone will respond with a “I literally have one subnet that goes out over ISP WAN and is only used for viewing videos of kittens on YouTube and I have no packages installed” but you get where I’m going I hope.
I would propose this - Netgate could offer a money back guarantee however you will have to purchase the device AND a support plan. This is after all what all the big players do right? CISCO will send you a new device overnight but guess what, you pay for that with insane subscription fees. So in this case, you buy the device, you get a support plan that pays Netage to have one its employees dedicated to support and troubleshoot your router and, maybe after all that, Netgate offers to refund you the cost of the router if truly it cannot work as expected/advertised. You’ll still eat the service plan cost because you received services in exchange for that cost, but you’ll get your money back for the device. I see no other plausible way for Netgate to protect its reputation and fiscal health while also guaranteeing customers good performance etc.
I’m nobody. I don’t work for Netgate. I’m simply blown away how anyone can demand a money back guarantee for something that has so many other variables as a whole. The only way to mitigate that is for the customer to pay for dedicated support. Even then, heaven forbid you have bad cables or other networking hardware or don’t have a solid foundation in networking. I’m struggling to figure out who else you’re going to go to for a money back guarantee that doesn’t make hardware sold at Best Buy.
-
@gabacho4 Please read my post more carefully: "I am not asking them to make this guarantee based on a random configuration -- but on a stock vanilla install of 21.x with only my cable modem and laptop attached (a scenario which gets line speed easily with 2.4.5p1 on the SG-3100)." This speaks directly to the point you're making, with which I mostly agree: Netgate cannot be expected to promise particular performance metrics in any possible configuration a customer might try. I have not asked them to do that -- I have asked them to guarantee that an OOTB vanilla configuration will give at least 1.2+ Gbps bandwidth in the same simple setup that the SG-3100 with 2.4.5p1 consistently gives 940Mbps. I explicitly agreed that I would work with support to do whatever they wanted to try if there was a problem before giving up -- because I certainly want it to work. Given the specs of the 6100 of 18Gbps for L3 routing with its 10/2.5Gbps NICs, this seems like a no-brainer when connected only to a cable modem and laptop, both with 2.5Gbps NICs. Frankly, if Netgate isn't testing with such a scenario as a baseline metric, that's absurd.
I might have been willing to budge on the shipping fees with Netgate, but a 25% restocking fee if they can't get the performance promised in a stock configuration is just unreasonable. That is asking me to take a $200+ gamble that their router works as advertised. No other manufacturer of any piece of computer or electronic equipment I have EVER purchased has ever taken this position for a return within a short window as long as the equipment was still in new condition. Many vendors don't even require a reason and for those that do, restocking and shipping fees are certainly waived for returns where the equipment failed to work as advertised.
-
@bldnightowl I feel you. However, issue is that if Netgate does this for you, they have to do it for everyone. I don’t believe this is a business model they are looking for as it would include those that I allude to in my previous post. Yes, your proposal is very straight forward. It would still require Negate to give you personalized attention, which takes me back to a support plan. For every one like you (straight forward use case/strong experience) there are 10 knuckle draggers.
That said, I cannot believe for a minute that the 6100 is incapable of doing what Netgate says it can. I’ve never heard any accusations that its hardware can’t push packets at the rate it claims. My experience with 2440s and 5100s says the same. The cases where things don’t perform as listed at best are when people have installed packages and services that reduce throughput by their very nature, or tax the routing system. At worst they are the result of misconfigured devices.
-
@gabacho4 Frankly, I think Netgate should offer a full refund less only shipping charges for any reason as long as the equipment is returned within a short window (no less than 7 days) and in original condition (and even shipping charges should be covered both ways if there is a legitimate defect causing the return). A 25% restocking fee is just obscene and unjustifiable imo. I asked them about buying an open-box return that some other customer had returned or testing an evaluation model before purchase, and they said they did neither of those things. (If they don't sell open-box returns, what do they do with them then?) Bottom line -- there must be an avenue for a customer to recoup all of their money if the product is not satisfactory for any reason, especially the stock configurations that Netgate will easily be able to resell. A support contract should not be necessary for this type of short term testing that the product is as advertised -- I reject that notion completely. I have never seen a company be so recalcitrant about standing behind their spec promises -- it makes me not trust the company. (After the agony I've had with 21.x upgrades on the SG-3100 this year, including having a hard time convincing them that there was a performance bug in 21.05 on the SG-3100 -- which they finally acknowledged should be fixed in 21.09, this just adds to my general lack of respect and mistrust for the company).
I am now faced with switching to some other router or waiting for some third party that will offer a reasonable money-back guarantee to sell me a 6100 in the future.
-
@bldnightowl I definitely hear you on the 3100. I haven’t been particularly impressed with any of the arm devices and have regularly been thankful I stuck with the atom-powered devices. The number of regressions and bugs in the 3100 and 1100 has been unsettling; especially as there have been software point releases just to address the problems on those devices.
As far as 3rd party - I believe some or all of the Netgate devices have shown up on Amazon. As far as ease of return, you don’t get much better than Amazon…unless Costco starts to sell them. I’ve noticed the prices are a bit more though and would be curious if that difference ultimately covers the cost of returns you’d pay if you had bought directly from Netgate.
-
@bldnightowl said in Netgate refuses to stand behind performance claims with *full* money-back guarantee:
performance bug in 21.05 on the SG-3100 -- which they finally acknowledged should be fixed in 21.09
This one? (fixed in 21.05.01)
-
@steveits I was told originally that it was an internal bug they wouldn't be showing tracking for publicly, but that sure looks like it. I'm a little hesitant to try upgrading from 2.4.5p1 to 21.05.1 -- since for the moment everything is stable for me. Support told me to track this page to know when 21.09 was ready (but as they indicated, no performance bug is mentioned there). So maybe they changed their minds. I will ask on my support case.
Update: Yes, Netgate has confirmed the performance regression was fixed in 21.05.1.
Update #2: And I can confirm directly that I get similar line speed performance from 21.05.1 that I do with 2.4.5p1.
-
@gabacho4 said in Netgate refuses to stand behind performance claims with *full* money-back guarantee:
As far as 3rd party - I believe some or all of the Netgate devices have shown up on Amazon.
I bought my SG-3100 through Amazon, which shipped directly from Netgate. But that's free delivery at least along with other Amazon discounts. And none of their Amazon seller profile pages or links mention a restocking fee, which I presume is an error -- or maybe Amazon doesn't allow them to do that. It's moot currently; they aren't offering the 6100 on Amazon yet.
-
@bldnightowl since your performance testing involves your ISP, you're asking Netgate to guarantee your ISP's performance in addition to theirs.
On a more basic level, your money-back guarantee expectation would be like expecting to return a car if its 0-60 times or fuel efficiency claims couldn't be replicated on a variety of road surfaces, atmospheric conditions, and driver weights.
In the electronics space, it would be like expecting the Xbox Series X and PS5 claims of supporting 4K@120fps to justify a return if those results weren't achieved in various games.
If you're in the queue for the August batch of 6100s, just buy and and sell it to me if you're not happy (I'd rather not wait for the September batch). :)
-
@flyzipper Uh, no, please read my post more carefully -- I am not expecting them to support any such thing. I am asking them to guarantee in a controlled setup where the 6100 is the only variable that OOTB with a vanilla/unmodified configuration get at least 1.2Gbps+ throughput in the same environment where my laptop attached directly to the modem gets 1.3Gbps+. My current SG-3100 taps out at 940Mbps on its 1Gbps NICs. Based on their specs, this should be an utter no-brainer, so it is utterly mystifying to me that they refuse to stand behind it.
And I wish I were and could have been in the August batch, but I was and am not willing to order from them without this assurance.
-
@bldnightowl if it does not meet their published performance data under ideal conditions it will be due to user testing errors. I would not guarantee user testing capability either.
Do you really think Netgate would claim performance they could not verify?
-
@patch Of course I told Netgate I would work with support to verify and troubleshoot before attempting any return -- I want this to work. They have the ability to remote login to the router (if I provide credentials) and do their own testing. I can also provide any performance data they want (screenshots, etc.) -- which is how I helped them identify a performance regression in 21.05 for the SG-3100 (and other ARM routers) that they fixed in 21.05.1.
You and other readers seem to think I am asking Netgate to make open-ended promises based on vapor -- so I guess my original post did not communicate well. Of course I would expect them to independently verify any claims on my part and point out any problems not due to the 6100 itself -- the last thing I want is to have to return the router! In my "testbed," my laptop connected through my SG-3100 with a vanilla configuration to my ISP gets 940Mbps consistently. My laptop directly connected to my ISP bypassing the SG-3100 gets 1.2Gbps or better. I want assurance that the 6100 will get at least that performance as well -- that's why I said the only variable would be the 6100 itself. I think Netgate support should be willing to sign up for this without blinking an eye.
But forget performance guarantees -- I also think any hardware vendor should offer 100% money-back no questions asked (less shipping of course) within a reasonably short time window as long as everything is returned in mint condition subject to inspection. A 25% restocking fee is absurd!
-
@bldnightowl said in Netgate refuses to stand behind performance claims with *full* money-back guarantee:
A 25% restocking fee is absurd!
You understand even if pristine they can not actually sell it as new, only as refurbished or open box, etc. So they would end up taking a loss, unless they can make up such a difference with a restock fee, etc..
These are not iphones, they are not being sold by the millions where loss of a some sales is no big deal, etc.
Even Cisco won't take back opened products, unless they are defective.. And even if they do take it back.. There is possibility of of restocking fee 20% etc..
https://www.cisco.com/c/en/us/buy/customer-support-information/manage-order/return-a-product.html
-
@bldnightowl think what you want about Netgate's sales practices, however, "the only variable would be the 6100 itself", is untrue...
Original post...
"I get 940 Mbps (*) bandwidth from my cable modem through my SG-3100 to a laptop connected to it, and that same laptop gets 1200+ Mbps when directly connected to my cable modem)."This post...
"In my "testbed," my laptop connected through my SG-3100 with a vanilla configuration to my ISP gets 940Mbps consistently. My laptop directly connected to my ISP bypassing the SG-3100 gets 1.2Gbps or better. "How are you testing performance through your cable modem if not using your (unguaranteed) internet connection? What end-point are you exchanging data with? What other load is hitting that end-point? How many hops are there between your laptop and that end-point?
There's a reason people do controlled lab testing with their own IPERF and IMIX endpoints -- to eliminate the variables your testbed doesn't.
-
@johnpoz I offered to test an evaluation model ahead of time -- they wouldn't do that either. And somewhat tongue in cheek, if a company is charging a fee more than Cisco, that's a sure sign it's really exorbitant.
Bottom line -- it is not reasonable of a company to expect a consumer to take a $200+ risk that their performance metric claims are legitimate. It is reasonable to expect proof / substantiation that the equipment and not the environment is the problem, and I am willing to do whatever it takes to satisfy them in that regard.@flyzipper Your point is well taken, but in my case, I do consistently get speeds I cited directly through the modem. But if push came to shove, I would certainly be willing to put a another device on another port of the 6100 and do strictly local iperf3 tests through the 6100 to prove the point. Netgate wasn't interested in that either. I really think their position is untenable.
-
D delitrium locked this topic on
-
B bldnightowl referenced this topic on
