mtdchar: add MEMREAD ioctl
Subsystems (5)
v4 cover mtdchar: add MEMREAD ioctl
This patch series adds a new mtdchar ioctl, MEMREAD. Its purpose is to serve as a read counterpart of the MEMWRITE ioctl, exposing a broader set of capabilities for read operations (e.g. use of MTD_OPS_AUTO_OOB, access to ECC statistics) to user-space applications making use of MTD devices via /dev/mtd* character devices.
Changes from v3:
- Fixed two issues found by Richard Weinberger [1]:
- prevent integer overflow,
- use kvmalloc() for potentially large allocations.
(Identical fixes have already been applied to mtdchar_write_ioctl(), i.e. the write counterpart of the ioctl proposed in this patch series, in mtd/next. [2] [3])
Changes from v2:
- Squashed patch 1/5 into patch 5/5 to prevent breaking bisectability due to an incompatible ABI change between those two patches. Revised commit messages accordingly.
Changes from v1:
- Added patches 2-5 which enable the new MEMREAD ioctl to report ECC statistics for the read operation back to user space. (There are obviously different ways these changes can be split up into separate commits; I was aiming for maximum ease of review.)
- The 'retlen' and 'oobretlen' fields were not set in the struct mtd_read_req returned to userspace. This was done properly in Boris' original draft patch [1], but I missed it in my v1.
- Invalid IS_ERR() checks were replaced with NULL checks. This was an artifact of copy-pasting mtdchar_write_ioctl() in v1: unlike memdup_user() used therein, kmalloc() always returns NULL on error.
- Minor subject prefix adjustment for patch 1/5 ("mtd" -> "mtdchar").
[1] http://lists.infradead.org/pipermail/linux-mtd/2022-February/091276.html
[2] http://lists.infradead.org/pipermail/linux-mtd/2022-June/093829.html
[3] http://lists.infradead.org/pipermail/linux-mtd/2022-June/093828.html
Michał Kępień (4):
mtd: track maximum number of bitflips for each read request
mtd: always initialize 'stats' in struct mtd_oob_ops
mtd: add ECC error accounting for each read request
mtdchar: add MEMREAD ioctl
drivers/mtd/devices/docg3.c | 8 ++ drivers/mtd/inftlcore.c | 6 +- drivers/mtd/mtdchar.c | 139 ++++++++++++++++++++++++ drivers/mtd/mtdcore.c | 5 + drivers/mtd/mtdswap.c | 6 +- drivers/mtd/nand/onenand/onenand_base.c | 16 ++- drivers/mtd/nand/onenand/onenand_bbt.c | 2 +- drivers/mtd/nand/raw/nand_base.c | 10 ++ drivers/mtd/nand/raw/nand_bbt.c | 8 +- drivers/mtd/nand/raw/sm_common.c | 2 +- drivers/mtd/nand/spi/core.c | 10 ++ drivers/mtd/nftlcore.c | 6 +- drivers/mtd/sm_ftl.c | 4 +- drivers/mtd/ssfdc.c | 2 +- drivers/mtd/tests/nandbiterrs.c | 2 +- drivers/mtd/tests/oobtest.c | 8 +- drivers/mtd/tests/readtest.c | 2 +- fs/jffs2/wbuf.c | 6 +- include/linux/mtd/mtd.h | 7 ++ include/uapi/mtd/mtd-abi.h | 64 ++++++++++- 20 files changed, 279 insertions(+), 34 deletions(-)
Signature · 7 lines
2.37.0 ______________________________________________________ Linux MTD discussion mailing list http://lists.infradead.org/mailman/listinfo/linux-mtd/
PATCH v4 1/4 mtd: track maximum number of bitflips for each read request
mtd_read_oob() callers are currently oblivious to the details of ECC errors detected during the read operation - they only learn (through the return value) whether any corrected bitflips or uncorrectable errors occurred. More detailed ECC information can be useful to user-space applications for making better-informed choices about moving data around.
Extend struct mtd_oob_ops with a pointer to a newly-introduced struct mtd_req_stats and set its 'max_bitflips' field to the maximum number of bitflips found in a single ECC step during the read operation performed by mtd_read_oob(). This is a prerequisite for ultimately passing that value back to user space.
--- drivers/mtd/mtdcore.c | 5 +++++ include/linux/mtd/mtd.h | 5 +++++ 2 files changed, 10 insertions(+)
drivers/mtd/mtdcore.c +5 -0
@@ -1622,6 +1622,9 @@ int mtd_read_oob(struct mtd_info *mtd, loff_t from, struct mtd_oob_ops *ops)if (!master->_read_oob && (!master->_read || ops->oobbuf))return -EOPNOTSUPP ; +if (ops->stats) +memset (ops->stats,0 ,sizeof (*ops->stats)); +if (mtd->flags &MTD_SLC_ON_MLC_EMULATION ) ret_code =mtd_io_emulated_slc (mtd, from, true, ops);else
@@ -1639,6 +1642,8 @@ int mtd_read_oob(struct mtd_info *mtd, loff_t from, struct mtd_oob_ops *ops)return ret_code;if (mtd->ecc_strength ==0 )return 0 ;/* device lacks ecc */ +if (ops->stats) + ops->stats->max_bitflips = ret_code;return ret_code >= mtd->bitflip_threshold ? -EUCLEAN :0 ; }EXPORT_SYMBOL_GPL (mtd_read_oob);
include/linux/mtd/mtd.h +5 -0
@@ -40,6 +40,10 @@ struct mtd_erase_region_info {unsigned long *lockmap;/* If keeping bitmap of locks */ }; +struct mtd_req_stats { +unsigned int max_bitflips; +}; +/** * struct mtd_oob_ops - oob operation operands * @mode: operation mode
@@ -70,6 +74,7 @@ struct mtd_oob_ops {uint32_t ooboffs;uint8_t *datbuf;uint8_t *oobbuf; +struct mtd_req_stats *stats; };/**
Signature · 7 lines
2.37.0 ______________________________________________________ Linux MTD discussion mailing list http://lists.infradead.org/mailman/listinfo/linux-mtd/
Re: [PATCH v4 1/4] mtd: track maximum number of bitflips for each read request
Quoted 7 lines · On Wed, 2022-06-29 at 12:57:34 UTC, =?utf-8?b?TWljaGHFgiBLxJlwaWXFhA==?= wrote:
mtd_read_oob()callers are currently oblivious to the details of ECC errors detected during the read operation - they only learn (through the return value) whether any corrected bitflips or uncorrectable errors occurred. More detailed ECC information can be useful to user-space applications for making better-informed choices about moving data around.
Quoted 8 lines
Extend struct
mtd_oob_opswith a pointer to a newly-introduced structmtd_req_statsand set its 'max_bitflips' field to the maximum number of bitflips found in a single ECC step during the read operation performed bymtd_read_oob(). This is a prerequisite for ultimately passing that value back to user space.Suggested-by Boris Brezillon <[email protected]>Signed-off-by Michał Kępień <[email protected]>
Applied to https://git.kernel.org/pub/scm/linux/kernel/git/mtd/linux.git mtd/next, thanks.
Miquel
PATCH v4 2/4 mtd: always initialize 'stats' in struct mtd_oob_ops
In reply to Michał KępieńAs the 'stats' field in struct mtd_oob_ops is used in conditional expressions, ensure it is always zero-initialized in all such structures to prevent random stack garbage from being interpreted as a pointer.
Strictly speaking, this problem currently only needs to be fixed for struct mtd_oob_ops structures subsequently passed to mtd_read_oob(). However, this commit goes a step further and makes all instances of struct mtd_oob_ops in the tree zero-initialized, in hope of preventing future problems, e.g. if struct mtd_req_stats gets extended with write statistics at some point.
---
Obviously this objective can be achieved in various ways. I was aiming for a minimal diff which does the job.
drivers/mtd/inftlcore.c | 6 +++--- drivers/mtd/mtdswap.c | 6 +++--- drivers/mtd/nand/onenand/onenand_base.c | 4 ++-- drivers/mtd/nand/onenand/onenand_bbt.c | 2 +- drivers/mtd/nand/raw/nand_bbt.c | 8 ++++---- drivers/mtd/nand/raw/sm_common.c | 2 +- drivers/mtd/nftlcore.c | 6 +++--- drivers/mtd/sm_ftl.c | 4 ++-- drivers/mtd/ssfdc.c | 2 +- drivers/mtd/tests/nandbiterrs.c | 2 +- drivers/mtd/tests/oobtest.c | 8 ++++---- drivers/mtd/tests/readtest.c | 2 +- fs/jffs2/wbuf.c | 6 +++--- 13 files changed, 29 insertions(+), 29 deletions(-)
drivers/mtd/inftlcore.c +3 -3
@@ -136,7 +136,7 @@ static void inftl_remove_dev(struct mtd_blktrans_dev *dev)int inftl_read_oob (struct mtd_info *mtd,loff_t offs,size_t len,size_t *retlen,uint8_t *buf) { -struct mtd_oob_ops ops; +struct mtd_oob_ops ops = { };int res; ops.mode =MTD_OPS_PLACE_OOB ;
@@ -156,7 +156,7 @@ int inftl_read_oob(struct mtd_info *mtd, loff_t offs, size_t len,int inftl_write_oob (struct mtd_info *mtd,loff_t offs,size_t len,size_t *retlen,uint8_t *buf) { -struct mtd_oob_ops ops; +struct mtd_oob_ops ops = { };int res; ops.mode =MTD_OPS_PLACE_OOB ;
@@ -176,7 +176,7 @@ int inftl_write_oob(struct mtd_info *mtd, loff_t offs, size_t len,static int inftl_write (struct mtd_info *mtd,loff_t offs,size_t len,size_t *retlen,uint8_t *buf,uint8_t *oob) { -struct mtd_oob_ops ops; +struct mtd_oob_ops ops = { };int res; ops.mode =MTD_OPS_PLACE_OOB ;
drivers/mtd/mtdswap.c +3 -3
@@ -323,7 +323,7 @@ static int mtdswap_read_markers(struct mtdswap_dev *d, struct swap_eb *eb)struct mtdswap_oobdata *data, *data2;int ret;loff_t offset; -struct mtd_oob_ops ops; +struct mtd_oob_ops ops = { }; offset =mtdswap_eb_offset (d, eb);
@@ -370,7 +370,7 @@ static int mtdswap_write_marker(struct mtdswap_dev *d, struct swap_eb *eb,struct mtdswap_oobdata n;int ret;loff_t offset; -struct mtd_oob_ops ops; +struct mtd_oob_ops ops = { }; ops.ooboffs =0 ; ops.oobbuf = (uint8_t *)&n;
@@ -878,7 +878,7 @@ static unsigned int mtdswap_eblk_passes(struct mtdswap_dev *d,loff_t base, pos;unsigned int *p1 = (unsigned int *)d->page_buf;unsigned char *p2 = (unsigned char *)d->oob_buf; -struct mtd_oob_ops ops; +struct mtd_oob_ops ops = { };int ret; ops.mode =MTD_OPS_AUTO_OOB ;
drivers/mtd/nand/onenand/onenand_base.c +2 -2
@@ -2935,7 +2935,7 @@ static int do_otp_write(struct mtd_info *mtd, loff_t to, size_t len,struct onenand_chip *this = mtd->priv;unsigned char *pbuf = buf;int ret; -struct mtd_oob_ops ops; +struct mtd_oob_ops ops = { };/* Force buffer page aligned */ if (len < mtd->writesize) {
@@ -2977,7 +2977,7 @@ static int do_otp_lock(struct mtd_info *mtd, loff_t from, size_t len, size_t *retlen, u_char *buf) { struct onenand_chip *this = mtd->priv; -struct mtd_oob_ops ops; +struct mtd_oob_ops ops = { };int ret;if (FLEXONENAND (this)) {
drivers/mtd/nand/onenand/onenand_bbt.c +1 -1
@@ -61,7 +61,7 @@ static int create_bbt(struct mtd_info *mtd, uint8_t *buf, struct nand_bbt_descrint startblock;loff_t from;size_t readlen; -struct mtd_oob_ops ops; +struct mtd_oob_ops ops = { };int rgn;printk (KERN_INFO "Scanning device for bad blocks\n" );
drivers/mtd/nand/raw/nand_bbt.c +4 -4
@@ -313,7 +313,7 @@ static int scan_read_oob(struct nand_chip *this, uint8_t *buf, loff_t offs, size_t len) {struct mtd_info *mtd =nand_to_mtd (this); -struct mtd_oob_ops ops; +struct mtd_oob_ops ops = { };int res, ret =0 ; ops.mode =MTD_OPS_PLACE_OOB ;
@@ -354,7 +354,7 @@ static int scan_write_bbt(struct nand_chip *this, loff_t offs, size_t len, uint8_t *buf, uint8_t *oob) { struct mtd_info *mtd = nand_to_mtd(this); -struct mtd_oob_ops ops; +struct mtd_oob_ops ops = { }; ops.mode =MTD_OPS_PLACE_OOB ; ops.ooboffs =0 ;
@@ -416,7 +416,7 @@ static int scan_block_fast(struct nand_chip *this, struct nand_bbt_descr *bd, {struct mtd_info *mtd =nand_to_mtd (this); -struct mtd_oob_ops ops; +struct mtd_oob_ops ops = { };int ret, page_offset; ops.ooblen = mtd->oobsize;
@@ -756,7 +756,7 @@ static int write_bbt(struct nand_chip *this, uint8_t *buf,uint8_t rcode = td->reserved_block_code;size_t retlen, len =0 ;loff_t to; -struct mtd_oob_ops ops; +struct mtd_oob_ops ops = { }; ops.ooblen = mtd->oobsize; ops.ooboffs =0 ;
drivers/mtd/nand/raw/sm_common.c +1 -1
@@ -99,7 +99,7 @@ static const struct mtd_ooblayout_ops oob_sm_small_ops = {static int sm_block_markbad (struct nand_chip *chip,loff_t ofs) {struct mtd_info *mtd =nand_to_mtd (chip); -struct mtd_oob_ops ops; +struct mtd_oob_ops ops = { };struct sm_oob oob;int ret;
drivers/mtd/nftlcore.c +3 -3
@@ -124,7 +124,7 @@ int nftl_read_oob(struct mtd_info *mtd, loff_t offs, size_t len, size_t *retlen, uint8_t *buf) { loff_t mask = mtd->writesize - 1; -struct mtd_oob_ops ops; +struct mtd_oob_ops ops = { };int res; ops.mode =MTD_OPS_PLACE_OOB ;
@@ -145,7 +145,7 @@ int nftl_write_oob(struct mtd_info *mtd, loff_t offs, size_t len, size_t *retlen, uint8_t *buf) { loff_t mask = mtd->writesize - 1; -struct mtd_oob_ops ops; +struct mtd_oob_ops ops = { };int res; ops.mode =MTD_OPS_PLACE_OOB ;
@@ -168,7 +168,7 @@ static int nftl_write(struct mtd_info *mtd, loff_t offs, size_t len, size_t *retlen, uint8_t *buf, uint8_t *oob) { loff_t mask = mtd->writesize - 1; -struct mtd_oob_ops ops; +struct mtd_oob_ops ops = { };int res; ops.mode =MTD_OPS_PLACE_OOB ;
drivers/mtd/sm_ftl.c +2 -2
@@ -239,7 +239,7 @@ static int sm_read_sector(struct sm_ftl *ftl, uint8_t *buffer, struct sm_oob *oob) { struct mtd_info *mtd = ftl->trans->mtd; -struct mtd_oob_ops ops; +struct mtd_oob_ops ops = { };struct sm_oob tmp_oob;int ret = -EIO ;int try =0 ;
@@ -323,7 +323,7 @@ static int sm_write_sector(struct sm_ftl *ftl, int zone, int block, int boffset, uint8_t *buffer, struct sm_oob *oob) { - struct mtd_oob_ops ops; + struct mtd_oob_ops ops = { };struct mtd_info *mtd = ftl->trans->mtd;int ret;
drivers/mtd/ssfdc.c +1 -1
@@ -163,7 +163,7 @@ static int read_physical_sector(struct mtd_info *mtd, uint8_t *sect_buf,/* Read redundancy area (wrapper to MTD_READ_OOB */ static int read_raw_oob (struct mtd_info *mtd,loff_t offs,uint8_t *buf) { -struct mtd_oob_ops ops; +struct mtd_oob_ops ops = { };int ret; ops.mode =MTD_OPS_RAW ;
drivers/mtd/tests/nandbiterrs.c +1 -1
@@ -99,7 +99,7 @@ static int write_page(int log)static int rewrite_page (int log) {int err =0 ; -struct mtd_oob_ops ops; +struct mtd_oob_ops ops = { };if (log)pr_info ("rewrite page\n" );
drivers/mtd/tests/oobtest.c +4 -4
@@ -56,7 +56,7 @@ static void do_vary_offset(void)static int write_eraseblock (int ebnum) {int i; -struct mtd_oob_ops ops; +struct mtd_oob_ops ops = { };int err =0 ;loff_t addr = (loff_t )ebnum * mtd->erasesize;
@@ -165,7 +165,7 @@ static size_t memffshow(loff_t addr, loff_t offset, const void *cs,static int verify_eraseblock (int ebnum) {int i; -struct mtd_oob_ops ops; +struct mtd_oob_ops ops = { };int err =0 ;loff_t addr = (loff_t )ebnum * mtd->erasesize;size_t bitflips;
@@ -260,7 +260,7 @@ static int verify_eraseblock(int ebnum)static int verify_eraseblock_in_one_go (int ebnum) { -struct mtd_oob_ops ops; +struct mtd_oob_ops ops = { };int err =0 ;loff_t addr = (loff_t )ebnum * mtd->erasesize;size_t len = mtd->oobavail * pgcnt;
@@ -338,7 +338,7 @@ static int __init mtd_oobtest_init(void)int err =0 ;unsigned int i;uint64_t tmp; -struct mtd_oob_ops ops; +struct mtd_oob_ops ops = { };loff_t addr =0 , addr0;printk (KERN_INFO "\n" );
drivers/mtd/tests/readtest.c +1 -1
@@ -47,7 +47,7 @@ static int read_eraseblock_by_page(int ebnum) err = ret; }if (mtd->oobsize) { -struct mtd_oob_ops ops; +struct mtd_oob_ops ops = { }; ops.mode =MTD_OPS_PLACE_OOB ; ops.len =0 ;
fs/jffs2/wbuf.c +3 -3
@@ -1035,7 +1035,7 @@ int jffs2_check_oob_empty(struct jffs2_sb_info *c, {int i, ret;int cmlen =min_t (int, c->oobavail,OOB_CM_SIZE ); -struct mtd_oob_ops ops; +struct mtd_oob_ops ops = { }; ops.mode =MTD_OPS_AUTO_OOB ; ops.ooblen =NR_OOB_SCAN_PAGES * c->oobavail;
@@ -1076,7 +1076,7 @@ int jffs2_check_oob_empty(struct jffs2_sb_info *c,int jffs2_check_nand_cleanmarker (struct jffs2_sb_info *c,struct jffs2_eraseblock *jeb) { -struct mtd_oob_ops ops; +struct mtd_oob_ops ops = { };int ret, cmlen =min_t (int, c->oobavail,OOB_CM_SIZE ); ops.mode =MTD_OPS_AUTO_OOB ;
@@ -1101,7 +1101,7 @@ int jffs2_write_nand_cleanmarker(struct jffs2_sb_info *c, struct jffs2_eraseblock *jeb) {int ret; -struct mtd_oob_ops ops; +struct mtd_oob_ops ops = { };int cmlen =min_t (int, c->oobavail,OOB_CM_SIZE ); ops.mode =MTD_OPS_AUTO_OOB ;
Signature · 7 lines
2.37.0 ______________________________________________________ Linux MTD discussion mailing list http://lists.infradead.org/mailman/listinfo/linux-mtd/
Re: [PATCH v4 2/4] mtd: always initialize 'stats' in struct mtd_oob_ops
Quoted 4 lines · On Wed, 2022-06-29 at 12:57:35 UTC, =?utf-8?b?TWljaGHFgiBLxJlwaWXFhA==?= wrote:
As the 'stats' field in struct
mtd_oob_opsis used in conditional expressions, ensure it is always zero-initialized in all such structures to prevent random stack garbage from being interpreted as a pointer.
Quoted 8 lines
Strictly speaking, this problem currently only needs to be fixed for struct
mtd_oob_opsstructures subsequently passed tomtd_read_oob(). However, this commit goes a step further and makes all instances of structmtd_oob_opsin the tree zero-initialized, in hope of preventing future problems, e.g. if structmtd_req_statsgets extended with write statistics at some point.Signed-off-by Michał Kępień <[email protected]>
Applied to https://git.kernel.org/pub/scm/linux/kernel/git/mtd/linux.git mtd/next, thanks.
Miquel
PATCH v4 3/4 mtd: add ECC error accounting for each read request
In reply to Michał KępieńExtend struct mtd_req_stats with two new fields holding the number of corrected bitflips and uncorrectable errors detected during a read operation. This is a prerequisite for ultimately passing those counters to user space, where they can be useful to applications for making better-informed choices about moving data around.
Unlike 'max_bitflips' (which is set - in a common code path - to the return value of a function called while the MTD device's mutex is held), these counters have to be maintained in each MTD driver which defines the '_read_oob' callback because the statistics need to be calculated while the MTD device's mutex is held.
--- drivers/mtd/devices/docg3.c | 8 ++++++++ drivers/mtd/nand/onenand/onenand_base.c | 12 ++++++++++++ drivers/mtd/nand/raw/nand_base.c | 10 ++++++++++ drivers/mtd/nand/spi/core.c | 10 ++++++++++ include/linux/mtd/mtd.h | 2 ++ 5 files changed, 42 insertions(+)
drivers/mtd/devices/docg3.c +8 -0
@@ -871,6 +871,7 @@ static int doc_read_oob(struct mtd_info *mtd, loff_t from,u8 *buf = ops->datbuf;size_t len, ooblen, nbdata, nboob;u8 hwecc[DOC_ECC_BCH_SIZE ], eccconf1; +struct mtd_ecc_stats old_stats;int max_bitflips =0 ;if (buf)
@@ -895,6 +896,7 @@ static int doc_read_oob(struct mtd_info *mtd, loff_t from, ret =0 ; skip = from %DOC_LAYOUT_PAGE_SIZE ;mutex_lock (&docg3->cascade->lock); + old_stats = mtd->ecc_stats;while (ret >=0 && (len >0 || ooblen >0 )) {calc_block_sector (from - skip, &block0, &block1, &page, &ofs, docg3->reliable);
@@ -966,6 +968,12 @@ static int doc_read_oob(struct mtd_info *mtd, loff_t from, } out: +if (ops->stats) { + ops->stats->uncorrectable_errors += + mtd->ecc_stats.failed - old_stats.failed; + ops->stats->corrected_bitflips += + mtd->ecc_stats.corrected - old_stats.corrected; + }mutex_unlock (&docg3->cascade->lock);return ret; err_in_read:
drivers/mtd/nand/onenand/onenand_base.c +12 -0
@@ -1440,6 +1440,7 @@ static int onenand_read_oob(struct mtd_info *mtd, loff_t from, struct mtd_oob_ops *ops) {struct onenand_chip *this = mtd->priv; +struct mtd_ecc_stats old_stats;int ret;switch (ops->mode) {
@@ -1453,12 +1454,23 @@ static int onenand_read_oob(struct mtd_info *mtd, loff_t from, }onenand_get_device (mtd,FL_READING ); + + old_stats = mtd->ecc_stats; +if (ops->datbuf) ret =ONENAND_IS_4KB_PAGE (this) ?onenand_mlc_read_ops_nolock (mtd, from, ops) :onenand_read_ops_nolock (mtd, from, ops);else ret =onenand_read_oob_nolock (mtd, from, ops); + +if (ops->stats) { + ops->stats->uncorrectable_errors += + mtd->ecc_stats.failed - old_stats.failed; + ops->stats->corrected_bitflips += + mtd->ecc_stats.corrected - old_stats.corrected; + } +onenand_release_device (mtd);return ret;
drivers/mtd/nand/raw/nand_base.c +10 -0
@@ -3818,6 +3818,7 @@ static int nand_read_oob(struct mtd_info *mtd, loff_t from, struct mtd_oob_ops *ops) {struct nand_chip *chip =mtd_to_nand (mtd); +struct mtd_ecc_stats old_stats;int ret; ops->retlen =0 ;
@@ -3829,11 +3830,20 @@ static int nand_read_oob(struct mtd_info *mtd, loff_t from,nand_get_device (chip); + old_stats = mtd->ecc_stats; +if (!ops->datbuf) ret =nand_do_read_oob (chip, from, ops);else ret =nand_do_read_ops (chip, from, ops); +if (ops->stats) { + ops->stats->uncorrectable_errors += + mtd->ecc_stats.failed - old_stats.failed; + ops->stats->corrected_bitflips += + mtd->ecc_stats.corrected - old_stats.corrected; + } +nand_release_device (chip);return ret; }
drivers/mtd/nand/spi/core.c +10 -0
@@ -635,6 +635,7 @@ static int spinand_mtd_read(struct mtd_info *mtd, loff_t from, {struct spinand_device *spinand =mtd_to_spinand (mtd);struct nand_device *nand =mtd_to_nanddev (mtd); +struct mtd_ecc_stats old_stats;unsigned int max_bitflips =0 ;struct nand_io_iter iter;bool disable_ecc = false;
@@ -646,6 +647,8 @@ static int spinand_mtd_read(struct mtd_info *mtd, loff_t from,mutex_lock (&spinand->lock); + old_stats = mtd->ecc_stats; +nanddev_io_for_each_page (nand,NAND_PAGE_READ , from, ops, &iter) {if (disable_ecc) iter.req.mode =MTD_OPS_RAW ;
@@ -668,6 +671,13 @@ static int spinand_mtd_read(struct mtd_info *mtd, loff_t from, ops->oobretlen += iter.req.ooblen; } +if (ops->stats) { + ops->stats->uncorrectable_errors += + mtd->ecc_stats.failed - old_stats.failed; + ops->stats->corrected_bitflips += + mtd->ecc_stats.corrected - old_stats.corrected; + } +mutex_unlock (&spinand->lock);if (ecc_failed && !ret)
include/linux/mtd/mtd.h +2 -0
@@ -41,6 +41,8 @@ struct mtd_erase_region_info { };struct mtd_req_stats { +unsigned int uncorrectable_errors; +unsigned int corrected_bitflips;unsigned int max_bitflips; };
Signature · 7 lines
2.37.0 ______________________________________________________ Linux MTD discussion mailing list http://lists.infradead.org/mailman/listinfo/linux-mtd/
Re: [PATCH v4 3/4] mtd: add ECC error accounting for each read request
Quoted 6 lines · On Wed, 2022-06-29 at 12:57:36 UTC, =?utf-8?b?TWljaGHFgiBLxJlwaWXFhA==?= wrote:
Extend struct
mtd_req_statswith two new fields holding the number of corrected bitflips and uncorrectable errors detected during a read operation. This is a prerequisite for ultimately passing those counters to user space, where they can be useful to applications for making better-informed choices about moving data around.
Quoted 8 lines
Unlike '
max_bitflips' (which is set - in a common code path - to the return value of a function called while the MTD device's mutex is held), these counters have to be maintained in each MTD driver which defines the '_read_oob' callback because the statistics need to be calculated while the MTD device's mutex is held.Suggested-by Boris Brezillon <[email protected]>Signed-off-by Michał Kępień <[email protected]>
Applied to https://git.kernel.org/pub/scm/linux/kernel/git/mtd/linux.git mtd/next, thanks.
Miquel
PATCH v4 4/4 mtdchar: add MEMREAD ioctl
In reply to Michał KępieńUser-space applications making use of MTD devices via /dev/mtd* character devices currently have limited capabilities for reading data:
- only deprecated methods of accessing OOB layout information exist,
- there is no way to explicitly specify MTD operation mode to use; it is auto-selected based on the MTD file mode (MTD_FILE_MODE_*) set for the character device; in particular, this prevents using MTD_OPS_AUTO_OOB for reads,
- all existing user-space interfaces which cause mtd_read() or mtd_read_oob() to be called (via mtdchar_read() and mtdchar_read_oob(), respectively) return success even when those functions return -EUCLEAN or -EBADMSG; this renders user-space applications using these interfaces unaware of any corrected bitflips or uncorrectable ECC errors detected during reads.
Note that the existing MEMWRITE ioctl allows the MTD operation mode to be explicitly set, allowing user-space applications to write page data and OOB data without requiring them to know anything about the OOB layout of the MTD device they are writing to (MTD_OPS_AUTO_OOB). Also, the MEMWRITE ioctl does not mangle the return value of mtd_write_oob().
Add a new ioctl, MEMREAD, which addresses the above issues. It is intended to be a read-side counterpart of the existing MEMWRITE ioctl. Similarly to the latter, the read operation is performed in a loop which processes at most mtd->erasesize bytes in each iteration. This is done to prevent unbounded memory allocations caused by calling kmalloc() with the 'size' argument taken directly from the struct mtd_read_req provided by user space. However, the new ioctl is implemented so that the values it returns match those that would have been returned if just a single mtd_read_oob() call was issued to handle the entire read operation in one go.
Note that while just returning -EUCLEAN or -EBADMSG to user space would already be a valid and useful indication of the ECC algorithm detecting errors during a read operation, that signal would not be granular enough to cover all use cases. For example, knowing the maximum number of bitflips detected in a single ECC step during a read operation performed on a given page may be useful when dealing with an MTD partition whose ECC layout varies across pages (e.g. a partition consisting of a bootloader area using a "custom" ECC layout followed by data pages using a "standard" ECC layout). To address that, include ECC statistics in the structure returned to user space by the new MEMREAD ioctl.
--- drivers/mtd/mtdchar.c | 139 +++++++++++++++++++++++++++++++++++++ include/uapi/mtd/mtd-abi.h | 64 +++++++++++++++-- 2 files changed, 198 insertions(+), 5 deletions(-)
drivers/mtd/mtdchar.c +139 -0
@@ -688,6 +688,137 @@ static int mtdchar_write_ioctl(struct mtd_info *mtd,return ret; } +static int mtdchar_read_ioctl (struct mtd_info *mtd, +struct mtd_read_req __user *argp) +{ +struct mtd_info *master =mtd_get_master (mtd); +struct mtd_read_req req; +void __user *usr_data, *usr_oob; +uint8_t *datbuf =NULL , *oobbuf =NULL ; +size_t datbuf_len, oobbuf_len; +size_t orig_len, orig_ooblen; +int ret =0 ; + +if (copy_from_user (&req, argp,sizeof (req))) +return -EFAULT ; + + orig_len = req.len; + orig_ooblen = req.ooblen; + + usr_data = (void __user *)(uintptr_t )req.usr_data; + usr_oob = (void __user *)(uintptr_t )req.usr_oob; + +if (!master->_read_oob) +return -EOPNOTSUPP ; + +if (!usr_data) + req.len =0 ; + +if (!usr_oob) + req.ooblen =0 ; + + req.ecc_stats.uncorrectable_errors =0 ; + req.ecc_stats.corrected_bitflips =0 ; + req.ecc_stats.max_bitflips =0 ; + + req.len &=0xffffffff ; + req.ooblen &=0xffffffff ; + +if (req.start + req.len > mtd->size) { + ret = -EINVAL ; + goto out; + } + + datbuf_len =min_t (size_t, req.len, mtd->erasesize); +if (datbuf_len >0 ) { + datbuf =kvmalloc (datbuf_len,GFP_KERNEL ); +if (!datbuf) { + ret = -ENOMEM ; + goto out; + } + } + + oobbuf_len =min_t (size_t, req.ooblen, mtd->erasesize); +if (oobbuf_len >0 ) { + oobbuf =kvmalloc (oobbuf_len,GFP_KERNEL ); +if (!oobbuf) { + ret = -ENOMEM ; + goto out; + } + } + +while (req.len >0 || (!usr_data && req.ooblen >0 )) { +struct mtd_req_stats stats; +struct mtd_oob_ops ops = { + .mode = req.mode, + .len =min_t (size_t, req.len, datbuf_len), + .ooblen =min_t (size_t, req.ooblen, oobbuf_len), + .datbuf = datbuf, + .oobbuf = oobbuf, + .stats = &stats, + }; + +/* +* Shorten non-page-aligned, eraseblock-sized reads so that the +* read ends on an eraseblock boundary. This is necessary in +* order to prevent OOB data for some pages from being +* duplicated in the output of non-page-aligned reads requiring +* multiple mtd_read_oob() calls to be completed. +*/ +if (ops.len == mtd->erasesize) + ops.len -=mtd_mod_by_ws (req.start + ops.len, mtd); + + ret =mtd_read_oob (mtd, (loff_t )req.start, &ops); + + req.ecc_stats.uncorrectable_errors += + stats.uncorrectable_errors; + req.ecc_stats.corrected_bitflips += stats.corrected_bitflips; + req.ecc_stats.max_bitflips = +max (req.ecc_stats.max_bitflips, stats.max_bitflips); + +if (ret && !mtd_is_bitflip_or_eccerr (ret)) +break ; + +if (copy_to_user (usr_data, ops.datbuf, ops.retlen) || +copy_to_user (usr_oob, ops.oobbuf, ops.oobretlen)) { + ret = -EFAULT ; +break ; + } + + req.start += ops.retlen; + req.len -= ops.retlen; + usr_data += ops.retlen; + + req.ooblen -= ops.oobretlen; + usr_oob += ops.oobretlen; + } + +/* +* As multiple iterations of the above loop (and therefore multiple +* mtd_read_oob() calls) may be necessary to complete the read request, +* adjust the final return code to ensure it accounts for all detected +* ECC errors. +*/ +if (!ret ||mtd_is_bitflip (ret)) { +if (req.ecc_stats.uncorrectable_errors >0 ) + ret = -EBADMSG ; +else if (req.ecc_stats.corrected_bitflips >0 ) + ret = -EUCLEAN ; + } + +out: + req.len = orig_len - req.len; + req.ooblen = orig_ooblen - req.ooblen; + +if (copy_to_user (argp, &req,sizeof (req))) + ret = -EFAULT ; + +kvfree (datbuf); +kvfree (oobbuf); + +return ret; +} +static int mtdchar_ioctl (struct file *file,u_int cmd,u_long arg) {struct mtd_file_info *mfi = file->private_data;
@@ -710,6 +841,7 @@ static int mtdchar_ioctl(struct file *file, u_int cmd, u_long arg)case MEMGETINFO :case MEMREADOOB :case MEMREADOOB64 : +case MEMREAD :case MEMISLOCKED :case MEMGETOOBSEL :case MEMGETBADBLOCK :
@@ -884,6 +1016,13 @@ static int mtdchar_ioctl(struct file *file, u_int cmd, u_long arg)break ; } +case MEMREAD : + { + ret =mtdchar_read_ioctl (mtd, + (struct mtd_read_req __user *)arg); +break ; + } +case MEMLOCK : {struct erase_info_user einfo;
include/uapi/mtd/mtd-abi.h +59 -5
88 lines not on this page · raw message
Signature · 7 lines
2.37.0 ______________________________________________________ Linux MTD discussion mailing list http://lists.infradead.org/mailman/listinfo/linux-mtd/
----- Ursprüngliche Mail -----
Quoted 52 lines
Von: "Michał Kępień" <[email protected]>
An: "Miquel Raynal" <[email protected]>, "richard" <[email protected]>, "Vignesh Raghavendra" <[email protected]>
CC: "Boris Brezillon" <[email protected]>, "linux-mtd" <[email protected]>, "linux-kernel" <[email protected]>
Gesendet: Mittwoch, 29. Juni 2022 14:57:37
Betreff: [PATCH v4 4/4] mtdchar: add MEMREAD ioctlUser-space applications making use of MTD devices via
/dev/mtd*character devices currently have limited capabilities for reading data:- only deprecated methods of accessing OOB layout information exist,
- there is no way to explicitly specify MTD operation mode to use; it is auto-selected based on the MTD file mode (MTD_FILE_MODE_*) set for the character device; in particular, this prevents using
MTD_OPS_AUTO_OOBfor reads,- all existing user-space interfaces which cause
mtd_read()ormtd_read_oob()to be called (viamtdchar_read()andmtdchar_read_oob(), respectively) return success even when those functions return-EUCLEANor-EBADMSG; this renders user-space applications using these interfaces unaware of any corrected bitflips or uncorrectable ECC errors detected during reads.Note that the existing MEMWRITE ioctl allows the MTD operation mode to be explicitly set, allowing user-space applications to write page data and OOB data without requiring them to know anything about the OOB layout of the MTD device they are writing to (
MTD_OPS_AUTO_OOB). Also, the MEMWRITE ioctl does not mangle the return value ofmtd_write_oob().Add a new ioctl, MEMREAD, which addresses the above issues. It is intended to be a read-side counterpart of the existing MEMWRITE ioctl. Similarly to the latter, the read operation is performed in a loop which processes at most
mtd->erasesizebytes in each iteration. This is done to prevent unbounded memory allocations caused by callingkmalloc()with the 'size' argument taken directly from the structmtd_read_reqprovided by user space. However, the new ioctl is implemented so that the values it returns match those that would have been returned if just a singlemtd_read_oob()call was issued to handle the entire read operation in one go.Note that while just returning
-EUCLEANor-EBADMSGto user space would already be a valid and useful indication of the ECC algorithm detecting errors during a read operation, that signal would not be granular enough to cover all use cases. For example, knowing the maximum number of bitflips detected in a single ECC step during a read operation performed on a given page may be useful when dealing with an MTD partition whose ECC layout varies across pages (e.g. a partition consisting of a bootloader area using a "custom" ECC layout followed by data pages using a "standard" ECC layout). To address that, include ECC statistics in the structure returned to user space by the new MEMREAD ioctl.
Quoted 8 lines
Suggested-by Boris Brezillon <[email protected]>Signed-off-by Michał Kępień <[email protected]>---
drivers/mtd/mtdchar.c| 139 +++++++++++++++++++++++++++++++++++++include/uapi/mtd/mtd-abi.h| 64 +++++++++++++++--2 files changed, 198 insertions(+), 5 deletions(-)
Thanks,
//richard
Quoted 45 lines · On Wed, 2022-06-29 at 12:57:37 UTC, =?utf-8?b?TWljaGHFgiBLxJlwaWXFhA==?= wrote:
User-space applications making use of MTD devices via
/dev/mtd*character devices currently have limited capabilities for reading data:- only deprecated methods of accessing OOB layout information exist,
- there is no way to explicitly specify MTD operation mode to use; it is auto-selected based on the MTD file mode (MTD_FILE_MODE_*) set for the character device; in particular, this prevents using
MTD_OPS_AUTO_OOBfor reads,- all existing user-space interfaces which cause
mtd_read()ormtd_read_oob()to be called (viamtdchar_read()andmtdchar_read_oob(), respectively) return success even when those functions return-EUCLEANor-EBADMSG; this renders user-space applications using these interfaces unaware of any corrected bitflips or uncorrectable ECC errors detected during reads.Note that the existing MEMWRITE ioctl allows the MTD operation mode to be explicitly set, allowing user-space applications to write page data and OOB data without requiring them to know anything about the OOB layout of the MTD device they are writing to (
MTD_OPS_AUTO_OOB). Also, the MEMWRITE ioctl does not mangle the return value ofmtd_write_oob().Add a new ioctl, MEMREAD, which addresses the above issues. It is intended to be a read-side counterpart of the existing MEMWRITE ioctl. Similarly to the latter, the read operation is performed in a loop which processes at most
mtd->erasesizebytes in each iteration. This is done to prevent unbounded memory allocations caused by callingkmalloc()with the 'size' argument taken directly from the structmtd_read_reqprovided by user space. However, the new ioctl is implemented so that the values it returns match those that would have been returned if just a singlemtd_read_oob()call was issued to handle the entire read operation in one go.Note that while just returning
-EUCLEANor-EBADMSGto user space would already be a valid and useful indication of the ECC algorithm detecting errors during a read operation, that signal would not be granular enough to cover all use cases. For example, knowing the maximum number of bitflips detected in a single ECC step during a read operation performed on a given page may be useful when dealing with an MTD partition whose ECC layout varies across pages (e.g. a partition consisting of a bootloader area using a "custom" ECC layout followed by data pages using a "standard" ECC layout). To address that, include ECC statistics in the structure returned to user space by the new MEMREAD ioctl.
Quoted 5 lines
Suggested-by Boris Brezillon <[email protected]>Signed-off-by Michał Kępień <[email protected]>
Applied to https://git.kernel.org/pub/scm/linux/kernel/git/mtd/linux.git mtd/next, thanks.
Miquel