List of USB Flash with hardware ECC support?

WithWings

Since most (all?) SSDs have a hardware based ECC implementation, they should correct bit rot, or 'silent corruption' on the fly without the OS (BSD) ever knowing about it, potentially even recovering corrupt data (not just marking it 'bad'). So it should be entirely OS agnostic, and not specifically need support from the kernel?

No I don't trust fsck. It has failed me before (faulty storage)

No it won't protect from logical corruption (faulty RAM, processor, overheating chipset, bug in the code, etc.)

I managed to find an Apacer model that claims hardware ECC support:

https://media.digikey.com/pdf/Data%20Sheets/Apacer/APHAzzzzax0xG-xT(M).pdf
http://www.mouser.com/ds/2/24/Industrial-USB_EH353-786251.pdf

SiliconMotion claims their controllers have hardware ECC:
http://www.siliconmotion.com/A3.2_Features_Detail.php?sn=35

As does a company called 'cypres':
http://www.cypress.com/file/139266/download

I chatted with a SanDisk support person, and he stated they don't have ECC support in USB devices, but he came back pretty quick, so I'm not sure he checked, as they might not have documentation for a question like that. Typically I like the SanDisk Ultra Fits for personal stuff, and trust their SSDs as much as Crucial MXxxx, and Samsung 8x0 SSDs. Even thought TBW endurance aren't published / as high.

I think it would be tremendously beneficial if we can compile a list of hardware ECC flash drives, so that people building this have more options to choose from for storage, and logging, especially for small appliances.

I'm especially curious if you can just get a 32 GB device, and format an 8 GB partition, would the flash controller be smart enough to use the other ~24 GB as spare flash for re-locating sectors? My gut tells me no, at least for most of them.

There is a utility that I won't link to called 'chip genius' (due to Virus Total flags for various versions) that supposedly can tell you what controller you're running. I don't know of any other vendors claiming hardware ECC for flash memory controllers (USB / MMC etc.).

Found here: (DON'T DOWNLOAD CHIP GENIUS!)
http://inigosolvedit.blogspot.com/2013/06/how-to-recoverfix-usb-flash-drives.html

VirusTotal link (Chip Genius)
https://www.virustotal.com/en/file/5b5bdae7a63b6b7816f25cdac671588862224452217d195bb87c303bef0f8d52/analysis/1486678914/

Links pulled from Google:
https://www.google.com/search?sourceid=chrome-psyapi2&ion=1&espv=2&ie=UTF-8&q=hardware%20ecc%20usb%20flash

If anyone knows of USB / MMC storage controllers that have ECC (and who uses them), or Flash Drives that claim to support it, please post them.

pfBasic

EDIT: sorry, I didn't pay attention to the links you posted to the same stuff I just did haha ::).
So obviously you already know the cost associated with it and are still interested.

Out of curiosity, other than sparing a SATA port, why pay all that money for a flash drive when you can get an EVO 950 with SLC cache and all s/hhd have ECC? Or an SLC SATA-DOM? Or be really cheap and mirror or raidz2 normal flash drives.

Check out digikey, they have datasheets you can Ctrl+F "ECC".

Just looking really quick Apacer sells a bunch.

ECC flash drives are serious $$$ though, The 32GB Apacer is SLC and has hardware ECC, but it will run you $275/pop.

With 2.4 coming up in the near future with ZFS installs fully supported, you might be better off picking up some SanDisk Cruzer Fits, at ~$8.25/ea for the 16GB version, you can run a pair in a mirror for 16GB or 4 in raidz2 for 32GB and double disk failure tolerance.
You can go crazy and run ECC RAM on your router, and still be cheaper than buying super high end usb drives, and your data will be safer.

https://www.amazon.com/SanDisk-SDCZ33-016G-B35-2PK-Everything-Stromboli-Lanyard/dp/B00DTZA5S0/ref=sr_1_3?s=electronics&ie=UTF8&qid=1486699701&sr=1-3&keywords=cruzer+fit

DigiKey flashdrives:
http://www.digikey.com/products/en/memory-cards-modules/usb-flash-drives/504?FV=ffe001f8&mnonly=0&newproducts=0&ColumnSort=-142&page=1&stock=0&pbfree=0&rohs=0&quantity=0&ptm=0&fid=0&pageSize=25

Apacer Data Sheet:
http://media.digikey.com/pdf/Data%20Sheets/Apacer/APHAzzzzax0xG-xT(M).pdf

WithWings

I didn't realize those drives were that expensive. I've never seen it listed as an option. I didn't realize they were going to be supporting ZFS. With that in a mirror it'd be pretty unlikely that the data would go corrupt in time to not be caught, and pulled from the backup device.

The biggest reason is just size constraints, and some older (perfect for certain uses) devices don't have M.2 ports, or mini pcie, but pretty much everything has usb, well P4+ anyway.

I figured that some OEMs might be including ECC just because the chips are pretty cheap once they're in quantity. I guess I'll just wait for 2.4

In one of the hangout videos I thought they mentioned not supporting x86 anymore? I can't find any documentation to that effect.

pfBasic

Yeah the ZFS install support will be very useful to those who need to set a system up and not physically access it again for a long time or ever, also for those that need 100% uptime.

For most users though, one of the great things about pfsense is that so long as you back up your config file, it's really just an inconvenience if your boot drive fails. Just add another drive, reinstall and import your config. Done.
That being said, with USB's as cheap as they are you can have a 32GB raidz2 boot for <$25 that can sustain any two drive failures. So why not?

As far as dropping x86 in 2.4, they are dropping 32 bit in 2.4 but will continue to maintain the 2.3.x line for sometime in order to provide legacy support while those that need it migrate to new hardware. Here's a link to that. https://forum.pfsense.org/index.php?topic=121255.0

whosmatt

@pfBasic:

Yeah the ZFS install support will be very useful to those who need to set a system up and not physically access it again for a long time or ever, also for those that need 100% uptime.

Yeah, doing that here on freenas, where the installer routine actually recommends flash drives.

@pfBasic:

For most users though, one of the great things about pfsense is that so long as you back up your config file, it's really just an inconvenience if your boot drive fails. Just add another drive, reinstall and import your config. Done.

Precisely. And if you're using a flash drive, it's easy enough to keep one ready to go (even up to date) with pfSense for a plug and play replacement. But I'm all for using the full install and SSD unless it can't be helped.

WithWings

Thanks for the link.

A larger issue for me is that you can't be 100% certain that those rules will reman as you set them up, so given time, what if a piece of the config is forgotten due to bitrot, that opens up the network?

whosmatt

@WithWings:

A larger issue for me is that you can't be 100% certain that those rules will reman as you set them up, so given time, what if a piece of the config is forgotten due to bitrot, that opens up the network?

I can't even imagine how that would happen. I'd worry more about a flash drive dying due to excessive writing than I would about the chance that data would corrupt in such a way that it would write a valid rule to open access to your network. That would be akin to giving a monkey a typewriter and having it randomly reproduce one of Shakespeare's works.

Remember, by default, no traffic is allowed in on any interface on pfSense. You have to explicitly write rules to allow traffic. There are some default ones created for you by the setup wizard, but in the absence of valid rules, all traffic is dropped.