Skip to contentSkip to conversation

nandwrite: warn about writing 0xff blocks

PATCH Applied Rafał Miłecki 4 participants 7 messages Mar 25, 2022 – Mar 28, 2022 Memory technology devices (MTD)

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 nandwrite: warn about writing 0xff blocks

From: Rafał Miłecki <[email protected]>

Such blocks may be incorrectly treated as empty (even though they may have non-erase OOB). Warn about it so people may start suing --skip-all-ffs .

---
 nand-utils/nandwrite.c | 13 ++++++++++++-
 1 file changed, 12 insertions(+), 1 deletion(-)
nand-utils/nandwrite.c +12 -1
index e8a210c..cd53a17 100644
@@ -280,6 +280,7 @@ int main(int argc, char * const argv[])
 	libmtd_t mtd_desc;
 	int ebsize_aligned;
 	uint8_t write_mode;
+	size_t all_ffs_cnt = 0;
 
 	process_options(argc, argv);
 
@@ -417,6 +418,8 @@ int main(int argc, char * const argv[])
 	 */
 	while ((imglen > 0 || writebuf < filebuf + filebuf_len)
 		&& mtdoffset < mtd.size) {
+		bool allffs;
+
 		/*
 		 * New eraseblock, check for bad block(s)
 		 * Stay in the loop to be sure that, if mtdoffset changes because
@@ -555,7 +558,8 @@ int main(int argc, char * const argv[])
 		}
 
 		ret = 0;
-		if (!skipallffs || !buffer_check_pattern(writebuf, mtd.min_io_size, 0xff)) {
+		allffs = buffer_check_pattern(writebuf, mtd.min_io_size, 0xff);
+		if (!allffs || !skipallffs) {
 			/* Write out data */
 			ret = mtd_write(mtd_desc, &mtd, fd, mtdoffset / mtd.eb_size,
 					mtdoffset % mtd.eb_size,
@@ -564,6 +568,8 @@ int main(int argc, char * const argv[])
 					writeoob ? oobbuf : NULL,
 					writeoob ? mtd.oob_size : 0,
 					write_mode);
+			if (!ret && allffs)
+				all_ffs_cnt++;
 		}
 
 		if (ret) {
@@ -615,6 +621,11 @@ closeall:
 		   || (writebuf < filebuf + filebuf_len))
 		sys_errmsg_die("Data was only partially written due to error");
 
+	if (all_ffs_cnt) {
+		fprintf(stderr, "Written %zu blocks containing only 0xff bytes\n", all_ffs_cnt);
+		fprintf(stderr, "Those block may be incorrectly treated as empty!\n");
+	}
+
 	/* Return happy */
 	return EXIT_SUCCESS;
 }
Signature · 6 lines
2.34.1


______________________________________________________
Linux MTD discussion mailing list
http://lists.infradead.org/mailman/listinfo/linux-mtd/
#2 lore raw

----- Ursprüngliche Mail -----

Quoted 7 lines

Von: "david oberhollenzer" <[email protected]>
An: "Rafał Miłecki" <[email protected]>, "Miquel Raynal" <[email protected]>, "richard" <[email protected]>
CC: "linux-mtd" <[email protected]>, "Rafał Miłecki" <[email protected]>
Gesendet: Montag, 28. März 2022 08:51:38
Betreff: Re: [PATCH mtd-utils] nandwrite: warn about writing 0xff blocks

Applied to mtd-utils.git master.

Please slow down a little. :)

Thanks,
//richard

______________________________________________________
Linux MTD discussion mailing list
http://lists.infradead.org/mailman/listinfo/linux-mtd/

#3 lore raw
In reply to Rafał Miłecki

----- Ursprüngliche Mail -----

Quoted 39 lines

Von: "Rafał Miłecki" <[email protected]>
An: "Miquel Raynal" <[email protected]>, "richard" <[email protected]>
CC: "linux-mtd" <[email protected]>, "Rafał Miłecki" <[email protected]>
Gesendet: Freitag, 25. März 2022 13:00:25
Betreff: [PATCH mtd-utils] nandwrite: warn about writing 0xff blocks

From: Rafał Miłecki <[email protected]>

Such blocks may be incorrectly treated as empty (even though they may have non-erase OOB). Warn about it so people may start suing --skip-all-ffs .

---
nand-utils/nandwrite.c | 13 ++++++++++++-

1 file changed, 12 insertions(+), 1 deletion(-)
nand-utils/nandwrite.c +3 -0
index e8a210c..cd53a17 100644
@@ -280,6 +280,7 @@ int main(int argc, char * const argv[])
 	libmtd_t mtd_desc;
 	int ebsize_aligned;
 	uint8_t write_mode;
+	size_t all_ffs_cnt = 0;
 
 	process_options(argc, argv);
 
@@ -417,6 +418,8 @@ int main(int argc, char * const argv[])
 	 */
 	while ((imglen > 0 || writebuf < filebuf + filebuf_len)
 		&& mtdoffset < mtd.size) {
+		bool allffs;
+
 		/*
 		 * New eraseblock, check for bad block(s)
 		 * Stay in the loop to be sure that, if mtdoffset changes because

nand-utils/nandwrite.c int main(int argc, char * const argv[]) · line 562

Quoted 6 lines
nand-utils/nandwrite.c +2 -1
@@ -555,7 +558,8 @@ int main(int argc, char * const argv[])
 		}
 
 		ret = 0;
-		if (!skipallffs || !buffer_check_pattern(writebuf, mtd.min_io_size, 0xff)) {
+		allffs = buffer_check_pattern(writebuf, mtd.min_io_size, 0xff);
+		if (!allffs || !skipallffs) {

Why is checking for allffs needed here?

nand-utils/nandwrite.c int main(int argc, char * const argv[]) · line 571

Quoted 8 lines
nand-utils/nandwrite.c +1 -0
 			/* Write out data */
 			ret = mtd_write(mtd_desc, &mtd, fd, mtdoffset / mtd.eb_size,
 					mtdoffset % mtd.eb_size,
@@ -564,6 +568,8 @@ int main(int argc, char * const argv[])
 					writeoob ? oobbuf : NULL,
 					writeoob ? mtd.oob_size : 0,
 					write_mode);
+			if (!ret && allffs)

Why checking for !ret?

Quoted 5 lines
nand-utils/nandwrite.c +1 -0
+				all_ffs_cnt++;
 		}
 
 		if (ret) {

nand-utils/nandwrite.c closeall: · line 629

Quoted 9 lines
nand-utils/nandwrite.c +5 -0
@@ -615,6 +621,11 @@ closeall:
 		   || (writebuf < filebuf + filebuf_len))
 		sys_errmsg_die("Data was only partially written due to error");
 
+	if (all_ffs_cnt) {
+		fprintf(stderr, "Written %zu blocks containing only 0xff bytes\n",
 all_ffs_cnt);
+		fprintf(stderr, "Those block may be incorrectly treated as empty!\n");
+	}
+

While I like the patch I'm still not so convinced why we can't make skipallffs=true by default.

Thanks,
//richard

______________________________________________________
Linux MTD discussion mailing list
http://lists.infradead.org/mailman/listinfo/linux-mtd/

#4 lore raw
Quoted 42 lines · Richard Weinberger

----- Ursprüngliche Mail -----

Quoted 41 lines

Von: "Rafał Miłecki" <[email protected]>
An: "Miquel Raynal" <[email protected]>, "richard" <[email protected]>
CC: "linux-mtd" <[email protected]>, "Rafał Miłecki" <[email protected]>
Gesendet: Freitag, 25. März 2022 13:00:25
Betreff: [PATCH mtd-utils] nandwrite: warn about writing 0xff blocks

From: Rafał Miłecki <[email protected]>

Such blocks may be incorrectly treated as empty (even though they may have non-erase OOB). Warn about it so people may start suing --skip-all-ffs .

---
nand-utils/nandwrite.c | 13 ++++++++++++-

1 file changed, 12 insertions(+), 1 deletion(-)
nand-utils/nandwrite.c +3 -0
index e8a210c..cd53a17 100644
@@ -280,6 +280,7 @@ int main(int argc, char * const argv[])
 	libmtd_t mtd_desc;
 	int ebsize_aligned;
 	uint8_t write_mode;
+	size_t all_ffs_cnt = 0;
 
 	process_options(argc, argv);
 
@@ -417,6 +418,8 @@ int main(int argc, char * const argv[])
 	 */
 	while ((imglen > 0 || writebuf < filebuf + filebuf_len)
 		&& mtdoffset < mtd.size) {
+		bool allffs;
+
 		/*
 		 * New eraseblock, check for bad block(s)
 		 * Stay in the loop to be sure that, if mtdoffset changes because
@@ -555,7 +558,8 @@ int main(int argc, char * const argv[])
 		}
 
Quoted 6 lines

nand-utils/nandwrite.c int main(int argc, char * const argv[]) · line 562

Quoted 4 lines
nand-utils/nandwrite.c +2 -1
 		ret = 0;
-		if (!skipallffs || !buffer_check_pattern(writebuf, mtd.min_io_size, 0xff)) {
+		allffs = buffer_check_pattern(writebuf, mtd.min_io_size, 0xff);
+		if (!allffs || !skipallffs) {

Why is checking for allffs needed here?

With --skip-all-ffs we want to write block if it contains data.

In other words this check is equal to:
if (contains_data || write_all_block)

Quoted 10 lines

nand-utils/nandwrite.c int main(int argc, char * const argv[]) · line 571

Quoted 8 lines
nand-utils/nandwrite.c +1 -0
 			/* Write out data */
 			ret = mtd_write(mtd_desc, &mtd, fd, mtdoffset / mtd.eb_size,
 					mtdoffset % mtd.eb_size,
@@ -564,6 +568,8 @@ int main(int argc, char * const argv[])
 					writeoob ? oobbuf : NULL,
 					writeoob ? mtd.oob_size : 0,
 					write_mode);
+			if (!ret && allffs)

Why checking for !ret?

If mtd_write() returns error we didn't actualy write anything.

Quoted 8 lines
Quoted 8 lines
nand-utils/nandwrite.c +1 -0
+				all_ffs_cnt++;
 		}
 
 		if (ret) {
@@ -615,6 +621,11 @@ closeall:
 		   || (writebuf < filebuf + filebuf_len))
 		sys_errmsg_die("Data was only partially written due to error");
 
Quoted 8 lines

nand-utils/nandwrite.c closeall: · line 628

Quoted 6 lines
nand-utils/nandwrite.c +5 -0
+	if (all_ffs_cnt) {
+		fprintf(stderr, "Written %zu blocks containing only 0xff bytes\n",
 all_ffs_cnt);
+		fprintf(stderr, "Those block may be incorrectly treated as empty!\n");
+	}
+

While I like the patch I'm still not so convinced why we can't make skipallffs=true by default.

I thought it's about changing / breaking user interface:

[2022-03-25] [11:40:53 CET] <derRichard> mraynal: i think we should make --skip-all-ffs default in nandwrite [2022-03-25] [11:40:56 CET] <derRichard> what do you think?
[2022-03-25] [11:42:00 CET] <rmilecki> i was preparing a patch with warning if any 0xff block has been written [2022-03-25] [11:42:33 CET] <rmilecki> i didn't know defaulting to --skip-all-ffs can be done [2022-03-25] [11:47:05 CET] <mraynal> well, that would break the user interface

______________________________________________________
Linux MTD discussion mailing list
http://lists.infradead.org/mailman/listinfo/linux-mtd/

maintainer
#5 lore raw

Hi Rafał,

[email protected] wrote on Mon, 28 Mar 2022 10:29:15 +0200:

Quoted 87 lines · Richard Weinberger
Quoted 42 lines

----- Ursprüngliche Mail -----

Quoted 41 lines

Von: "Rafał Miłecki" <[email protected]>
An: "Miquel Raynal" <[email protected]>, "richard" <[email protected]>
CC: "linux-mtd" <[email protected]>, "Rafał Miłecki" <[email protected]>
Gesendet: Freitag, 25. März 2022 13:00:25
Betreff: [PATCH mtd-utils] nandwrite: warn about writing 0xff blocks

From: Rafał Miłecki <[email protected]>

Such blocks may be incorrectly treated as empty (even though they may have non-erase OOB). Warn about it so people may start suing --skip-all-ffs .

---
nand-utils/nandwrite.c | 13 ++++++++++++-

1 file changed, 12 insertions(+), 1 deletion(-)
nand-utils/nandwrite.c +3 -0
index e8a210c..cd53a17 100644
@@ -280,6 +280,7 @@ int main(int argc, char * const argv[])
 	libmtd_t mtd_desc;
 	int ebsize_aligned;
 	uint8_t write_mode;
+	size_t all_ffs_cnt = 0;
 
 	process_options(argc, argv);
 
@@ -417,6 +418,8 @@ int main(int argc, char * const argv[])
 	 */
 	while ((imglen > 0 || writebuf < filebuf + filebuf_len)
 		&& mtdoffset < mtd.size) {
+		bool allffs;
+
 		/*
 		 * New eraseblock, check for bad block(s)
 		 * Stay in the loop to be sure that, if mtdoffset changes because
@@ -555,7 +558,8 @@ int main(int argc, char * const argv[])
 		}
 
Quoted 6 lines

nand-utils/nandwrite.c int main(int argc, char * const argv[]) · line 562

Quoted 4 lines
nand-utils/nandwrite.c +2 -1
 		ret = 0;
-		if (!skipallffs || !buffer_check_pattern(writebuf, mtd.min_io_size, 0xff)) {
+		allffs = buffer_check_pattern(writebuf, mtd.min_io_size, 0xff);
+		if (!allffs || !skipallffs) {  

Why is checking for allffs needed here?

With --skip-all-ffs we want to write block if it contains data.

In other words this check is equal to:
if (contains_data || write_all_block)

Quoted 10 lines

nand-utils/nandwrite.c int main(int argc, char * const argv[]) · line 571

Quoted 8 lines
nand-utils/nandwrite.c +1 -0
 			/* Write out data */
 			ret = mtd_write(mtd_desc, &mtd, fd, mtdoffset / mtd.eb_size,
 					mtdoffset % mtd.eb_size,
@@ -564,6 +568,8 @@ int main(int argc, char * const argv[])
 					writeoob ? oobbuf : NULL,
 					writeoob ? mtd.oob_size : 0,
 					write_mode);
+			if (!ret && allffs)  

Why checking for !ret?

If mtd_write() returns error we didn't actualy write anything.

Quoted 8 lines
Quoted 8 lines
nand-utils/nandwrite.c +1 -0
+				all_ffs_cnt++;
 		}
 
 		if (ret) {
@@ -615,6 +621,11 @@ closeall:
 		   || (writebuf < filebuf + filebuf_len))
 		sys_errmsg_die("Data was only partially written due to error");
 
Quoted 8 lines

nand-utils/nandwrite.c closeall: · line 628

Quoted 6 lines
nand-utils/nandwrite.c +5 -0
+	if (all_ffs_cnt) {
+		fprintf(stderr, "Written %zu blocks containing only 0xff bytes\n",
 all_ffs_cnt);
+		fprintf(stderr, "Those block may be incorrectly treated as empty!\n");
+	}
+  

While I like the patch I'm still not so convinced why we can't make skipallffs=true by default.

Quoted 7 lines

I thought it's about changing / breaking user interface:

[2022-03-25] [11:40:53 CET] <derRichard> mraynal: i think we should make --skip-all-ffs default in nandwrite [2022-03-25] [11:40:56 CET] <derRichard> what do you think?
[2022-03-25] [11:42:00 CET] <rmilecki> i was preparing a patch with warning if any 0xff block has been written [2022-03-25] [11:42:33 CET] <rmilecki> i didn't know defaulting to --skip-all-ffs can be done [2022-03-25] [11:47:05 CET] <mraynal> well, that would break the user interface

Indeed I raised this issue but I am not opposed to this change if everybody agrees that it's a good move. In particular, the NAND sublayer will give the same "empty" data to userspace whether it programmed empty pages or skipped them. So I guess we are fine against direct regressions (but we might break clever scripts doing strange things).

I would go for this solution:
- Bring support for the double flag -[no-]skip-ffs
- Make the use of -skip-ffs the default
So that it is easy for scripting people to ensure their way is fine?

Thanks,
Miquèl

______________________________________________________
Linux MTD discussion mailing list
http://lists.infradead.org/mailman/listinfo/linux-mtd/

#6 lore raw

----- Ursprüngliche Mail -----

Quoted 22 lines

Von: "Miquel Raynal" <[email protected]>

Quoted 5 lines
Quoted 2 lines

While I like the patch I'm still not so convinced why we can't make skipallffs=true by default.

I thought it's about changing / breaking user interface:

Quoted 8 lines

[2022-03-25] [11:40:53 CET] <derRichard> mraynal: i think we should make --skip-all-ffs default in nandwrite
[2022-03-25] [11:40:56 CET] <derRichard> what do you think? [2022-03-25] [11:42:00 CET] <rmilecki> i was preparing a patch with warning if any 0xff block has been written
[2022-03-25] [11:42:33 CET] <rmilecki> i didn't know defaulting to --skip-all-ffs can be done
[2022-03-25] [11:47:05 CET] <mraynal> well, that would break the user interface

Indeed I raised this issue but I am not opposed to this change if everybody agrees that it's a good move. In particular, the NAND sublayer will give the same "empty" data to userspace whether it programmed empty pages or skipped them. So I guess we are fine against direct regressions (but we might break clever scripts doing strange things).

Quoted 4 lines

I would go for this solution:
- Bring support for the double flag -[no-]skip-ffs
- Make the use of -skip-ffs the default
So that it is easy for scripting people to ensure their way is fine?

Also emit a warning that a 0xff page is being skipped if the user no not specify anything at the command line.
Then users will notice that the default has changed.

Thanks,
//richard

______________________________________________________
Linux MTD discussion mailing list
http://lists.infradead.org/mailman/listinfo/linux-mtd/