Netgate 6100 SSDs

cdturri

Hi,

This reddit thread covers most of the questions I had regarding SSD upgrades for the Netgate 6100 apart from the following:

SSDs need to be B+M keyed, M.2 SSD NVMe but is there a max supported capacity?
I see the 6100 has a second M.2 slot, can this slot be used as well? (see this video of the 6100 Max)
Of both slots can be used is it possible to install pfSense in two mirrored custom SSDs for high availability?

Thanks!

rcoleman-netgate

@cdturri Upgrading, replacing, or adding, an SSD to the 6100 is not supported and may void your warranty. We do not provide an SSD for sale that can be added nor do we provide instructions on how to make the replacement as the entire system is very tightly fit into the case and you can break connections opening and re-securing the enclosure. It happened to a number of our staff during early testing and I have personally confirmed the tight fit on production models of the 4100 (which is basically the same form factor).

At present time there are no uses for the additional ports/slots on the 6100.

cdturri

Ummm didn’t that was the oficial stand in terms of user upgrades. Honestly I wouldn’t have bought the unit if I knew that was the case. The whole point in buying a Netgate appliance is to have official hardware support and get all in one box that i don’t need to design nor make sure everything it’s compatible or works well. Adding an SSD will not change that, at least not in my mind.

What’s the point in selling a product with an M.2 slot if users can’t upgrade it? If your main concern is that customers shouldn’t open the unit or that it’s a “tight fit” you can easily solve this by adding access for the M.2 slot. Or sell a unit that is non-upgradable. I really think that you guys are missing something here by being user hostile with upgrades. Having said that what’s done it’s done and I don’t really expect a reply from what I said above. Might be news to me but I suspect it had been argued before.

But I do want to know what drives I can install on my unit. While you can argue I may have broken the warranty I can still claim I own the unit. So what SSDs can I install in them? Will the second slot work as well? I’m not asking in terms of support here, I am asking what will work.

Thanks
Christian

cdturri

@cdturri …I really think that you guys are missing something here by being user hostile with upgrades. Having said that what’s done it’s done and I don’t really expect a reply from what I said above. Might be news to me but I su expect it had been argued before.

But I do want to know what drives I can install on my unit. While you can argue I may have broken the warranty I can still claim I own the unit. So what SSDs can I install in them? Will the second slot work as well? I’m not asking in terms of support here, I am asking what will work.

Thanks
Christian

rcoleman-netgate

@cdturri I don't think I'm being hostile towards the users at all - the advisory is 100% legitimate. We do not offer the purchase of the drives, we do not recommend users attempt to install the a drive because of the tight constraints of the case to the board as well as the very large heatsink needed for this system.

With the other systems we openly publish the procedures as well as the type of drive you would want in addition to also selling the very drive that is optionally added to our systems.

And regarding other ports to date we do not have support added to the software to allow use of the other ports, but in the future it is possible these things may change.

If you want to see what others have installed there has been discussion here and on our Subreddit as to what others have purchased.

cdturri

@rcoleman-netgate It’s user hostile because it’s misleading. Can you please tell me where in the 6100 product page says that the M.2 slots are not user upgradable?

https://shop.netgate.com/products/6100-base-pfsense

I am really thinking in returning the unit now. I have installed a 256GB SSD disk already but if you guys won’t support my hardware setup then what’s the point in me buying a Netgate appliance? I can build my own PfSense appliance and have to support it myself as well.

Where can I see which appliances are user upgradable and which ones are not?

rcoleman-netgate

@cdturri said in Netgate 6100 SSDs:

if you guys won’t support my hardware setup

I don't recall saying we won't support your hardware setup. What I wrote was adding a drive "may void your warranty".

In the demo/announce videos it was stated that there are expansion ports that are not supported at this time but may be made functional in future releases. This has not changed.

In regards to systems with upgradable parts our current lineup includes the 2100 (m.2 mSATA drive, mPCIe for LTE or WiFi), 5100 (same + RAM), 7100 (same as 5100 + SATA SSD/HDD/PCIe support with an optional adapter), as well as the 1537 and 1541 systems.

Why do we discourage opening the 4100 and 6100? Because of the tight proximity of the board to the case, the lack of actual user serviceable parts and the systems reliance on the bottom heatsink to dissipate heat any of these things can go wrong if the unit is opened and damaged in the process.

If we had a recommendation drive to put in the system we'd have it for sale in our store like we do for the m.2 mSATA drives that can go in our older models.

stephenw10

From a purely technical point of view:

I'm not aware of any capacity limit on the NVMe drives though I don't think we've tested anything larger than 256GB. There really isn't a good reason to use anything larger in pfSense.
Yes both slots support NVMe drives.
Yes it's possible to create a mirrored drive pair using them both:

[22.09-DEVELOPMENT][admin@6100.stevew.lan]/root: zpool list -v zroot
NAME         SIZE  ALLOC   FREE  CKPOINT  EXPANDSZ   FRAG    CAP  DEDUP  HEALTH  ALTROOT
zroot         13G   824M  12.2G        -         -     5%     6%  1.00x  ONLINE  -
  mirror      13G   824M  12.2G        -         -     5%  6.18%
    nvd0p3      -      -      -        -         -      -      -
    nvd1p3      -      -      -        -         -      -      -

But as stated we don't recommend it because if the board is damaged by fitting 3rd party hardware we cannot warranty that.

Steve

cybrnook

This post is deleted!

cdturri

@stephenw10 Thanks Steve, that's helpful. "There really isn't a good reason to use anything larger in pfSense." => Where have I seen this before? Ah yes, Gates' "640K ought to be enough for anybody" famous quote.
When I invest in hardware I want to deploy something and have it running for years to come. Some people claim to use 100GB of logs in pfSense. But even at 25 GB/year 256 GB will barely last for 10 years.

cdturri

@rcoleman-netgate said in Netgate 6100 SSDs:

@cdturri said in Netgate 6100 SSDs:

if you guys won’t support my hardware setup

I don't recall saying we won't support your hardware setup. What I wrote was adding a drive "may void your warranty".

My bad, apologies for jumping the gun. Still I think that adding SSDs to the 6100 should not void the warranty. I take you said "may" not "will" so I guess it's subject to Netgate deciding if any damage was done when opening the unit, which I think it's fair. Personally I didn't think opening and closing the 6100 was of much difficulty, I have seen much worse! (Apple!). I think this should be noted in the product page although I would much prefer a redesign of the 6100's top cover to allow access to the 2x M.2 ports without opening the unit. That will make the unit much more customer friendly.

Final question, why did Netgate decided to go with B+M keyed M.2 NVMe SSDs? B+M SSDs are a lot harder to find (and therefore a lot more expensive) than just M keyed M.2 NVMe SSDs which are standard in consumer PCs. Even if you didn't want customers to install SSDs why make this harder on yourself?

rcoleman-netgate

@cdturri said in Netgate 6100 SSDs:

Still I think that adding SSDs to the 6100 should not void the warranty

I stand by my statement of the boards are so tightly placed that if you are not careful you can break the plastic and the PCB. Colleagues broke the plastic covers on test units and I think we've seen at least one repair claim on a broken PCB from someone that tried to upgrade their storage.

Proceed at your own risk, of course, but we aren't going to endorse the practice for this specific reason and we do not publish a list of "compatible drives".

rcoleman-netgate

@cdturri said in Netgate 6100 SSDs:

Final question, why did Netgate decided to go with B+M keyed M.2 NVMe SSDs?

https://www.atpinc.com/blog/what-is-m.2-M-B-BM-key-socket-3

The keys have specific PCIe definitions.

stephenw10

@cdturri said in Netgate 6100 SSDs:

"There really isn't a good reason to use anything larger in pfSense." => Where have I seen this before? Ah yes, Gates' "640K ought to be enough for anybody" famous quote.

Ha, well that's always possible. However there isn't a limit I'm aware of so should you discover something that needs TBs of storage on your firewall you could, in theory, add it.
Logging is not one of those things though. 100GB of logs stored on pfSense is next to useless. If you need that sort of long term log storage it should be exported to a syslog server where it can be usefully viewed and analyzed.
You can see in my test mirror setup I have two 16GB drives and I'm using 6% of that.

And, yes, the slots are also B keyed because they are wired for USB with the SIM slots.

Steve

cdturri

@stephenw10 said in Netgate 6100 SSDs:

Logging is not one of those things though. 100GB of logs stored on pfSense is next to useless. If you need that sort of long term log storage it should be exported to a syslog server where it can be usefully viewed and analyzed.

I certainly agreed on that. I don't intend to use the Netgate as reporting solution but I do prefer to keep the original logs in situ so that I can choose whichever analysis solution I may want to use. Export to a syslog server it's possible but not all packages use syslog and this adds a potential failure point. Often I find that I need to go back and analise what happened in a period of time on my network. Without the logs I can't do that hence why I want to be sure I can keep the original logs. Thanks for all the replies, very helpful!

hayescompatible

@rcoleman-netgate said in Netgate 6100 SSDs:

@cdturri said in Netgate 6100 SSDs:

Still I think that adding SSDs to the 6100 should not void the warranty

I stand by my statement of the boards are so tightly placed that if you are not careful you can break the plastic and the PCB. Colleagues broke the plastic covers on test units and I think we've seen at least one repair claim on a broken PCB from someone that tried to upgrade their storage.

Proceed at your own risk, of course, but we aren't going to endorse the practice for this specific reason and we do not publish a list of "compatible drives".

Not to belabour the point but Tom Lawrence opens a 6100 up in his review and suggests it’s quite easy to do. He does say that there’s only one serviceable part (the battery) while also saying that we gotta wait for official Netgate support for internal peripherals. Given the limited lifespan of the internal eMMC on the base units I’m surprised there isn’t official support for adding an after-market SSD even though it is probably technically possible.

I think if care is taken when removing the case, it should not be a problem but point taken about voiding warranties, etc.

keyser

@stephenw10 True that, but for me it’s not about the space. It’s about the endurance of the storage.
I found it very easy to install my own SSD in my SG-6100, and I choose a 512Gb size based on endurance only.
This allows me to log everything I want and have Suricata, pfBlocker and NtopNG do their thing with proper logging without worrying that it will kill my storage.
My calculations said the eMMC would be dead in about 8-10 months with my current load - With the SSD, it says about 10 years.

muenchris

@keyser I opened my 6100 after the warranty was over and my 6100 failed to recognize the internal drive (probably due to over exiting logs). Since the cooling block is glued to some of the components, I rather do not take that off, is there any of the M2 slots usable for an SSD? I see three slots, two of them NVM and one that looks like a PC-Express Slot. The two NVMs slots are just "B-Keyed" not "B+M" as suggested here. How did you exchange your SSD?

keyser

@muenchris I just saw this Video on Youtube about how the MAX was equiped on the inside (and a guide in disassembling it).

https://www.youtube.com/watch?v=Twp07m-SvV4

muenchris

@keyser said in Netgate 6100 SSDs:

https://www.youtube.com/watch?v=Twp07m-SvV4

Thanks, in my 6100 all slots are empty. But I got confirmation I can use any of the two NVME slots to add a new drive. Opening the 6100 is actually very easy if you have the right torx screwdriver.