#2- "MLC NAnd" corrupts-on-read how oft? Would your's have "ECCs", and use them to boot correctly?
The questions in THIS "e"mail, are regarding the "MLC NAnd" mentioned (as planned in the computer-cards via "Crowd Supply") in the "Q/A"-section on the campaign-page- https://www.crowdsupply.com/eoma68/micro-desktop (My earlier off-list question on this, and Luke's reply, are quoted at the bottom.)
I have been reading on "MLC NAnd", and it seems that now I better understand the problem of corruption-on-read. (I have included links at the bottom, to some literature which I have looked at. This might give you a sense of my current state of understanding on this. And others of you might even find that some of the linked information is news to you.)
But I have a few more questions.
(a) How common is it for corruption-on-read to occur with "MLC NAnd" (like by something as basic as reads done by the built-in "ROM" "boot"-loader)? (Perhaps the answer is like "N % probability that one or more of the [1 to 4] bits in a cell, shall wrongly change it's logical value, after X reads of one or more pages in the same block, Y writes to one or more pages in the same block, and Z erases of the block".) At first I was thinking that the FIRST time data (mini "boot"-loader or otherwise) is read from the "NAnd", corruption might likely occur. But perhaps this corruption-on-read naturally happens ONLY after many reads or writes in a block and many erases of a block. So how common is it?
(b) Would the "MLC NAnd" planned in the computer-cards via "Crowd Supply", have Error-Correction Codes?
(c) If so, then does whatever reads the "NAnd" on "booting" (I guess it is called the "eGON boot-ROM"), know that it should (and know how to) use those "ECCs", to correct errors (if any) which it encounters when trying to start the "booting" process, so that it loads the correct original bits of the "boot"-loader ("minimalist" or otherwise)?
I am looking forward to receiving answers to my questions, and soon sending my money to back many of the current offerings via "Crowd Supply". Sincerely, Chad. (:^)
Links-
a little info' on corruption, and on patented ]:^[ work to solve it- https://storagegaga.wordpress.com/category/data-corruption/
a little on how "NAnd-flash" works, and file-systems- http://www.linux-mtd.infradead.org/doc/nand.html
more on how it works and how it can get corrupted- http://www.eetimes.com/document.asp?doc_id=1279762 followed by pages 2 to http://www.eetimes.com/document.asp?doc_id=1279762&page_number=4
some research on read-disturbs and mitigation- https://users.ece.cmu.edu/~saugatag/papers/15dsn.pdf
detailed comparison of "MLC" and "SLC", with advice on how to use both- https://cdn.selinc.com/assets/Literature/Publications/White Papers/0015_NANDflash_IO_20141211.pdf?v=20151004-191546 (There should be a single space between "White" and "Papers".)
booting "Allwinner" chips- http://linux-sunxi.org/BROM
(Quotes below, might have minor changes, and might have additions enclosed by {}, and ~ for omissions.)
On 16.8.26 12:11, lkcl . lkcl@rhombus-tech.net wrote:
On ~, {8th month}~, 2016 at 6:33 PM, Crowd Supply orders@crowdsupply.com wrote:
chadvellacott@sasktel.net submitted a question about your project, "Earth-friendly EOMA68 Computing Devices":
~~~~~
~ it seems that your latest plan is or was, to use MLC-NAnd-flash for theinternal "hard disk drive". And I noticed the mention that that can corrupt itself on read, not just on write. ~~~
primarily you should be looking to store a "mostly-read-only" OS on the internal NAND, which then mounts external USB drives and Micro-SD cards as well as mounts tmpfs's for the stuff you really don't need to be persistent. i referenced somebody's script called "readonlyrootfs.sh" which i've successfully used with CompactFlash rootfs booting in the past.
also you should really be looking to use jffs2 or better ubifs - some work's going to be needed here - so that you don't end up losing data. you can see online that people are talking about just dropping ext4 directly onto raw NAND just because that's what allwinner does with android... this is incredibly stupid and you clearly shouldn't be doing it... but ignorance like this tends to get copied wholesale on BBS forums... *sigh*.
in the meantime you can always boot from external micro-sd card, that's what i do all the time, i have a minimalist bootloader at the front of the NAND and leave the rest of the NAND entirely alone.
i *may* have to ship people with configs like that (a small 4gb or 8gb microsd included)... just have to see how it goes.
And I noticed the mention that you are considering using SLC instead.
~~~~~~
l.
On 08/09/16 03:18, chadvellacott@sasktel.net wrote:
The questions in THIS "e"mail, are regarding the "MLC NAnd" mentioned (as planned in the computer-cards via "Crowd Supply") in the "Q/A"-section on the campaign-page- https://www.crowdsupply.com/eoma68/micro-desktop
[...]
Links-
a little info' on corruption, and on patented ]:^[ work to solve it- https://storagegaga.wordpress.com/category/data-corruption/
a little on how "NAnd-flash" works, and file-systems- http://www.linux-mtd.infradead.org/doc/nand.html
more on how it works and how it can get corrupted- http://www.eetimes.com/document.asp?doc_id=1279762 followed by pages 2 to http://www.eetimes.com/document.asp?doc_id=1279762&page_number=4
some research on read-disturbs and mitigation- https://users.ece.cmu.edu/~saugatag/papers/15dsn.pdf
detailed comparison of "MLC" and "SLC", with advice on how to use both- https://cdn.selinc.com/assets/Literature/Publications/White Papers/0015_NANDflash_IO_20141211.pdf?v=20151004-191546 (There should be a single space between "White" and "Papers".)
booting "Allwinner" chips- http://linux-sunxi.org/BROM
Thanks for collating these links.
Bunnie Huang and Xobs have also been doing some work on flash memory.
Their key quote, as mentioned in the write-up below: "with flash memory you are not really storing your data; what you are storing is a probabilistic approximation of your data."
(Plus, they demonstrated that your SD card can MITM you.)
Modern computers may be astoundingly portable and inexpensive, but they are, as yet, far from being perfectly reliable or secure.
https://threatpost.com/flash-memory-cards-contain-powerful-unsecured-microco...
On 2016-09-07 20:18 -0600, chadvellacott@sasktel.net wrote:
I have been reading on "MLC NAnd", and it seems that now I betterunderstand the problem of corruption-on-read.
I have some experience of NAND due to working on YAFFS a few years ago. My info may be slightly out of date as NAND has got even fatter in the last few years.
(a) How common is it for corruption-on-read to occur with "MLC NAnd" (like by something as basic as reads done by the built-in "ROM" "boot"-loader)? (Perhaps the answer is like "N % probability that one or more of the [1 to 4] bits in a cell, shall wrongly change it's logical value, after X reads of one or more pages in the same block, Y writes to one or more pages in the same block, and Z erases of the block".) At first I was thinking that the FIRST time data (mini "boot"-loader or otherwise) is read from the "NAnd", corruption might likely occur. But perhaps this corruption-on-read naturally happens ONLY after many reads or writes in a block and many erases of a block. So how common is it?
'Rare'. Corruption-on-read (called 'read disturb' in the literature) will only happen after there have been quite a few reads aligned on just the 'wrong' page. 20,000 or so might do it in modern MLC. write-disturb is much more likely than read-disturb. So each read that energises the same 'row' in the flash layout increases the error potential a tiny amount. Each write increases it quite a lot more (but in modern flash with a flash-aware filesystem you only ever write a page once before erasing it so this is not an issue).
Bits are 'refreshed' (and the probabilities of error reset) when the bits in question are rewritten. So a really smart flash filesystem will ensure that 'old' data that is near pages that have been read a lot, gets moved.
(b) Would the "MLC NAnd" planned in the computer-cards via "Crowd Supply", have Error-Correction Codes?
yes. All NAND-flash has this otherwise it would be uselessly unreliable.
(c) If so, then does whatever reads the "NAnd" on "booting" (I guess it is called the "eGON boot-ROM"), know that it should (and know how to) use those "ECCs", to correct errors (if any) which it encounters when trying to start the "booting" process, so that it loads the correct original bits of the "boot"-loader ("minimalist" or otherwise)?
yes. all NAND reads check the ECC.
The YAFFS site has a load of info on the issue of NAND (un)reliability, and what it does to manage/mitigate it: http://yaffs.net/documents/yaffs-nand-flash-failure-mitigation Specifically: http://yaffs.net/documents/yaffs-nand-flash-failure-mitigation#Read_disturb
HTH
Wookey
wookey just wanted to say thank you for responding here. l.
On Sat, Sep 10, 2016 at 2:09 AM, Wookey wookey@wookware.org wrote:
On 2016-09-07 20:18 -0600, chadvellacott@sasktel.net wrote:
I have been reading on "MLC NAnd", and it seems that now I betterunderstand the problem of corruption-on-read.
I have some experience of NAND due to working on YAFFS a few years ago. My info may be slightly out of date as NAND has got even fatter in the last few years.
arm-netbook@lists.phcomp.co.uk
-
chadvellacott＠sasktel.net -
Luke Kenneth Casson Leighton -
Sam Pablo Kuper -
Wookey