Netgate 6100 SSDs

stephenw10

From a purely technical point of view:

I'm not aware of any capacity limit on the NVMe drives though I don't think we've tested anything larger than 256GB. There really isn't a good reason to use anything larger in pfSense.
Yes both slots support NVMe drives.
Yes it's possible to create a mirrored drive pair using them both:

[22.09-DEVELOPMENT][admin@6100.stevew.lan]/root: zpool list -v zroot
NAME         SIZE  ALLOC   FREE  CKPOINT  EXPANDSZ   FRAG    CAP  DEDUP  HEALTH  ALTROOT
zroot         13G   824M  12.2G        -         -     5%     6%  1.00x  ONLINE  -
  mirror      13G   824M  12.2G        -         -     5%  6.18%
    nvd0p3      -      -      -        -         -      -      -
    nvd1p3      -      -      -        -         -      -      -

But as stated we don't recommend it because if the board is damaged by fitting 3rd party hardware we cannot warranty that.

Steve

cybrnook

This post is deleted!

cdturri

@stephenw10 Thanks Steve, that's helpful. "There really isn't a good reason to use anything larger in pfSense." => Where have I seen this before? Ah yes, Gates' "640K ought to be enough for anybody" famous quote.
When I invest in hardware I want to deploy something and have it running for years to come. Some people claim to use 100GB of logs in pfSense. But even at 25 GB/year 256 GB will barely last for 10 years.

cdturri

@rcoleman-netgate said in Netgate 6100 SSDs:

@cdturri said in Netgate 6100 SSDs:

if you guys won’t support my hardware setup

I don't recall saying we won't support your hardware setup. What I wrote was adding a drive "may void your warranty".

My bad, apologies for jumping the gun. Still I think that adding SSDs to the 6100 should not void the warranty. I take you said "may" not "will" so I guess it's subject to Netgate deciding if any damage was done when opening the unit, which I think it's fair. Personally I didn't think opening and closing the 6100 was of much difficulty, I have seen much worse! (Apple!). I think this should be noted in the product page although I would much prefer a redesign of the 6100's top cover to allow access to the 2x M.2 ports without opening the unit. That will make the unit much more customer friendly.

Final question, why did Netgate decided to go with B+M keyed M.2 NVMe SSDs? B+M SSDs are a lot harder to find (and therefore a lot more expensive) than just M keyed M.2 NVMe SSDs which are standard in consumer PCs. Even if you didn't want customers to install SSDs why make this harder on yourself?

rcoleman-netgate

@cdturri said in Netgate 6100 SSDs:

Still I think that adding SSDs to the 6100 should not void the warranty

I stand by my statement of the boards are so tightly placed that if you are not careful you can break the plastic and the PCB. Colleagues broke the plastic covers on test units and I think we've seen at least one repair claim on a broken PCB from someone that tried to upgrade their storage.

Proceed at your own risk, of course, but we aren't going to endorse the practice for this specific reason and we do not publish a list of "compatible drives".

rcoleman-netgate

@cdturri said in Netgate 6100 SSDs:

Final question, why did Netgate decided to go with B+M keyed M.2 NVMe SSDs?

https://www.atpinc.com/blog/what-is-m.2-M-B-BM-key-socket-3

The keys have specific PCIe definitions.

stephenw10

@cdturri said in Netgate 6100 SSDs:

"There really isn't a good reason to use anything larger in pfSense." => Where have I seen this before? Ah yes, Gates' "640K ought to be enough for anybody" famous quote.

Ha, well that's always possible. However there isn't a limit I'm aware of so should you discover something that needs TBs of storage on your firewall you could, in theory, add it.
Logging is not one of those things though. 100GB of logs stored on pfSense is next to useless. If you need that sort of long term log storage it should be exported to a syslog server where it can be usefully viewed and analyzed.
You can see in my test mirror setup I have two 16GB drives and I'm using 6% of that.

And, yes, the slots are also B keyed because they are wired for USB with the SIM slots.

Steve

cdturri

@stephenw10 said in Netgate 6100 SSDs:

Logging is not one of those things though. 100GB of logs stored on pfSense is next to useless. If you need that sort of long term log storage it should be exported to a syslog server where it can be usefully viewed and analyzed.

I certainly agreed on that. I don't intend to use the Netgate as reporting solution but I do prefer to keep the original logs in situ so that I can choose whichever analysis solution I may want to use. Export to a syslog server it's possible but not all packages use syslog and this adds a potential failure point. Often I find that I need to go back and analise what happened in a period of time on my network. Without the logs I can't do that hence why I want to be sure I can keep the original logs. Thanks for all the replies, very helpful!

hayescompatible

@rcoleman-netgate said in Netgate 6100 SSDs:

@cdturri said in Netgate 6100 SSDs:

Still I think that adding SSDs to the 6100 should not void the warranty

I stand by my statement of the boards are so tightly placed that if you are not careful you can break the plastic and the PCB. Colleagues broke the plastic covers on test units and I think we've seen at least one repair claim on a broken PCB from someone that tried to upgrade their storage.

Proceed at your own risk, of course, but we aren't going to endorse the practice for this specific reason and we do not publish a list of "compatible drives".

Not to belabour the point but Tom Lawrence opens a 6100 up in his review and suggests it’s quite easy to do. He does say that there’s only one serviceable part (the battery) while also saying that we gotta wait for official Netgate support for internal peripherals. Given the limited lifespan of the internal eMMC on the base units I’m surprised there isn’t official support for adding an after-market SSD even though it is probably technically possible.

I think if care is taken when removing the case, it should not be a problem but point taken about voiding warranties, etc.

keyser

@stephenw10 True that, but for me it’s not about the space. It’s about the endurance of the storage.
I found it very easy to install my own SSD in my SG-6100, and I choose a 512Gb size based on endurance only.
This allows me to log everything I want and have Suricata, pfBlocker and NtopNG do their thing with proper logging without worrying that it will kill my storage.
My calculations said the eMMC would be dead in about 8-10 months with my current load - With the SSD, it says about 10 years.

muenchris

@keyser I opened my 6100 after the warranty was over and my 6100 failed to recognize the internal drive (probably due to over exiting logs). Since the cooling block is glued to some of the components, I rather do not take that off, is there any of the M2 slots usable for an SSD? I see three slots, two of them NVM and one that looks like a PC-Express Slot. The two NVMs slots are just "B-Keyed" not "B+M" as suggested here. How did you exchange your SSD?

keyser

@muenchris I just saw this Video on Youtube about how the MAX was equiped on the inside (and a guide in disassembling it).

https://www.youtube.com/watch?v=Twp07m-SvV4

muenchris

@keyser said in Netgate 6100 SSDs:

https://www.youtube.com/watch?v=Twp07m-SvV4

Thanks, in my 6100 all slots are empty. But I got confirmation I can use any of the two NVME slots to add a new drive. Opening the 6100 is actually very easy if you have the right torx screwdriver.

stephenw10

You can use either m.2 slot. Or both. The slots are B-keyed, yes, but since all NVMe drives are M-keyed you need one that's B and M keyed to fit. Or at least I've never seen a B only keyed NVMe drive. If one exists that would probably also work.

Steve

keyser

Thanks, in my 6100 all slots are empty. But I got confirmation I can use any of the two NVME slots to add a new drive. Opening the 6100 is actually very easy if you have the right torx screwdriver.

If all your slots are empty it’s because you did not buy the MAX model (just like me). So what you have worn out is the built in 16Gb eMMC. Good thing you can install a SSD, otherwise the box would be useless.

muenchris

@keyser Yes that is what Netgate support told me, too.

muenchris

@muenchris Well, I got my drive and put it in one of the eNVM slots, but during the install the drive is not recognized.
Is there any way to "mount" or format the drive for use by the pfsense installer? (Sorry I am a windows guy and linux is not my expertise)

stephenw10

The installer will partition and format the drive you don't normally need to do anything to prepare it.

Do you see it detected in the boot log?

Like:

nvme0: Allocated 16MB host memory buffer
nvd0: <M.2 (P80) 3TE6> NVMe namespace
nvd0: 114473MB (234441648 512 byte sectors)

muenchris

@stephenw10 Not that text. I see "NVME Lens loading/installing" but not any other entries

keyser

@muenchris Weird. I would check the install again, or perhaps try the other keyed slot for Storage - Just to to rule out a misalignment/contact issue. If that doesn’t work either, it’s probably the SSD….