ubiattach fails with "bad image sequence number"

If I ubiformat my NAND partition with an UBIFS image I keep on getting "bad image sequence number" when I try to do the ubiattach. If I just do a plain ubiformat with no image then everything works fine when I ubiattach.

I was wondering if somebody ran into something similar. The steps are so trivial it seems like something fundamentally wrong with either the mkfs.ubifs or the ubiformat.

I did not see anything related to this on the UBFI FAQ or documentation.

Here is how the image is being created on the x86 host:
======================================================

$ mkfs.ubifs -r build/targetfs_tiny -m 4096 -e 262144 -c 2047 -o tiny.ubifs

$ ubinize -s 1024 -o ubifs1b.img -m 4096 -p 256KiB ubifs1.cfg

$ more ubifs1.cfg
[ubifs]
mode=ubi
image=tiny.ubifs
vol_id=0
vol_size=500MiB
vol_type=dynamic
vol_name=fs1
vol_alignment=1
vol_flags=autoresize

And here is the sequence on the PowerPC target where the problem has manifested itself:
==================================================================

# mtdinfo -u /dev/mtd8
mtd8
Name:                           fs1
Type:                           nand
Eraseblock size:                262144 bytes, 256.0 KiB
Amount of eraseblocks:          2048 (536870912 bytes, 512.0 MiB)
Minimum input/output unit size: 4096 bytes
Sub-page size:                  1024 bytes
OOB size:                       128 bytes
Character device major/minor:   90:16
Bad blocks are allowed:         true
Device is writable:             true
Default UBI VID header offset:  1024
Default UBI data offset:        4096
Default UBI LEB size:           258048 bytes, 252.0 KiB
Maximum UBI volumes count:      128

# ubiformat -s 1024 -f ubifs1b.img /dev/mtd8
ubiformat: mtd8 (nand), size 536870912 bytes (512.0 MiB), 2048 eraseblocks of 262144 bytes (256.0 KiB), min. I/O size 4096 bytes
libscan: scanning eraseblock 2047 -- 100 % complete
ubiformat: 2045 eraseblocks have valid erase counter, mean value is 4
ubiformat: 3 bad eraseblocks found, numbers: 72, 930, 1846
ubiformat: flashing eraseblock 30 -- 100 % complete
ubiformat: formatting eraseblock 2047 -- 100 % complete

# ubiattach /dev/ubi_ctrl -m 8
[ 2614.265521] UBI: attaching mtd8 to ubi0
[ 2614.269732] UBI: physical eraseblock size:   262144 bytes (256 KiB)
[ 2614.276645] UBI: logical eraseblock size:    258048 bytes
[ 2614.282420] UBI: smallest flash I/O unit:    4096
[ 2614.287251] UBI: sub-page size:              1024
[ 2614.292063] UBI: VID header offset:          1024 (aligned 1024)
[ 2614.298178] UBI: data offset:                4096
[ 2614.321544] UBI error: process_eb: bad image sequence number 1294632561 in PEB 31, expected 1571790334
[ 2614.331112] UBI error: ubi_attach_mtd_dev: failed to attach by scanning, error -22
ubiattach: error!: cannot attach mtd8
           error 22 (Invalid argument)

Jeff Angielski
The PTR Group
www.theptrgroup.com

Hi,

I believe this is an ubiformat bug. Thanks for reporting and sorry for inconvenience.

I attach and inline 2 patches which should fix this. I cannot have a possibility to test them now, so they are untested. They are against the latest mtd-utils.git (commit a4e502d99129da8ebba6d40b373a4422a175e9af).

Here we go.

From: Artem Bityutskiy <[email protected]>
Date: Sun, 10 Jan 2010 01:33:31 +0200
Subject: [PATCH 1/2] ubiformat: always initialize seq number

For some reasons sequence number was set to 0 in some case, which is wrong.

---
 ubi-utils/src/ubiformat.c |    3 ++-
 1 files changed, 2 insertions(+), 1 deletions(-)

index 3b0f49b..80f3a6d 100644
@@ -912,7 +912,8 @@ int main(int argc, char * const argv[])
 				printf("yes\n");
 		} else
 			ubigen_info_init(&ui, mtd.eb_size, mtd.min_io_size, 0,
-					 si->vid_hdr_offs, args.ubi_ver, 0);
+					 si->vid_hdr_offs, args.ubi_ver,
+					 args.image_seq);
 		normsg("use offsets %d and %d",  ui.vid_hdr_offs, ui.data_offs);
 	}
 

1.6.2.5

From: Artem Bityutskiy <[email protected]>
Date: Sun, 10 Jan 2010 01:39:24 +0200
Subject: [PATCH 2/2] ubiformat: be consistent with sequence numbers

This commit fixes a stupid an nasty bug. When we flash an UBI image, we do not change its sequence numbers. But when we format the rest of the PEBs (beyond the flashed image), we use a random (or specified via cmdline) sequence number. As a result, we have a broken flash format and UBI refuses it, because half of it has one sequence number, another half has a different one.

What we have to do instead, we have to substitute image's sequence number with ours.

---
 ubi-utils/src/ubiformat.c |   11 +++++++----
 1 files changed, 7 insertions(+), 4 deletions(-)

index 80f3a6d..2316d67 100644
@@ -295,7 +295,8 @@ static void print_bad_eraseblocks(const struct mtd_dev_info *mtd,
 	printf("\n");
 }
 
-static int change_ec(struct ubi_ec_hdr *hdr, long long ec)
+static int change_ech(struct ubi_ec_hdr *hdr, uint32_t image_seq,
+		      long long ec)
 {
 	uint32_t crc;
 

@@ -309,6 +310,7 @@ static int change_ec(struct ubi_ec_hdr *hdr, long long ec)
 		return errmsg("bad CRC %#08x, should be %#08x\n",
 			      crc, be32_to_cpu(hdr->hdr_crc));
 
+	hdr->image_seq = cpu_to_be32(image_seq);
 	hdr->ec = cpu_to_be64(ec);
 	crc = crc32(UBI_CRC32_INIT, hdr, UBI_EC_HDR_SIZE_CRC);
 	hdr->hdr_crc = cpu_to_be32(crc);

@@ -438,7 +440,8 @@ static int mark_bad(const struct mtd_dev_info *mtd, struct ubi_scan_info *si, in
 	return consecutive_bad_check(eb);
 }
 
-static int flash_image(const struct mtd_dev_info *mtd, struct ubi_scan_info *si)
+static int flash_image(const struct mtd_dev_info *mtd, const struct ubigen_info *ui,
+		       struct ubi_scan_info *si)
 {
 	int fd, img_ebs, eb, written_ebs = 0, divisor;
 	off_t st_size;

@@ -518,7 +521,7 @@ static int flash_image(const struct mtd_dev_info *mtd, struct ubi_scan_info *si)
 			fflush(stdout);
 		}
 
-		err = change_ec((struct ubi_ec_hdr *)buf, ec);
+		err = change_ech((struct ubi_ec_hdr *)buf, ui->image_seq, ec);
 		if (err) {
 			errmsg("bad EC header at eraseblock %d of \"%s\"",
 			       written_ebs, args.image);

@@ -918,7 +921,7 @@ int main(int argc, char * const argv[])
 	}
 
 	if (args.image) {
-		err = flash_image(&mtd, si);
+		err = flash_image(&mtd, &ui, si);
 		if (err < 0)
 			goto out_free;
 

1.6.2.5

-- 
Best Regards,
Artem Bityutskiy (Артём Битюцкий)

From: Artem Bityutskiy <[email protected]>
Date: Sun, 10 Jan 2010 01:33:31 +0200
Subject: [PATCH 1/2] ubiformat: always initialize seq number

For some reasons sequence number was set to 0 in some case, which is wrong.

---
 ubi-utils/src/ubiformat.c |    3 ++-
 1 files changed, 2 insertions(+), 1 deletions(-)

index 3b0f49b..80f3a6d 100644
@@ -912,7 +912,8 @@ int main(int argc, char * const argv[])
 				printf("yes\n");
 		} else
 			ubigen_info_init(&ui, mtd.eb_size, mtd.min_io_size, 0,
-					 si->vid_hdr_offs, args.ubi_ver, 0);
+					 si->vid_hdr_offs, args.ubi_ver,
+					 args.image_seq);
 		normsg("use offsets %d and %d",  ui.vid_hdr_offs, ui.data_offs);
 	}
 

1.6.2.5

From: Artem Bityutskiy <[email protected]>
Date: Sun, 10 Jan 2010 01:39:24 +0200
Subject: [PATCH 2/2] ubiformat: be consistent with sequence numbers

This commit fixes a stupid an nasty bug. When we flash an UBI image, we do not change its sequence numbers. But when we format the rest of the PEBs (beyond the flashed image), we use a random (or specified via cmdline) sequence number. As a result, we have a broken flash format and UBI refuses it, because half of it has one sequence number, another half has a different one.

What we have to do instead, we have to substitute image's sequence number with ours.

---
 ubi-utils/src/ubiformat.c |   11 +++++++----
 1 files changed, 7 insertions(+), 4 deletions(-)

index 80f3a6d..2316d67 100644
@@ -295,7 +295,8 @@ static void print_bad_eraseblocks(const struct mtd_dev_info *mtd,
 	printf("\n");
 }
 
-static int change_ec(struct ubi_ec_hdr *hdr, long long ec)
+static int change_ech(struct ubi_ec_hdr *hdr, uint32_t image_seq,
+		      long long ec)
 {
 	uint32_t crc;
 

@@ -309,6 +310,7 @@ static int change_ec(struct ubi_ec_hdr *hdr, long long ec)
 		return errmsg("bad CRC %#08x, should be %#08x\n",
 			      crc, be32_to_cpu(hdr->hdr_crc));
 
+	hdr->image_seq = cpu_to_be32(image_seq);
 	hdr->ec = cpu_to_be64(ec);
 	crc = crc32(UBI_CRC32_INIT, hdr, UBI_EC_HDR_SIZE_CRC);
 	hdr->hdr_crc = cpu_to_be32(crc);

@@ -438,7 +440,8 @@ static int mark_bad(const struct mtd_dev_info *mtd, struct ubi_scan_info *si, in
 	return consecutive_bad_check(eb);
 }
 
-static int flash_image(const struct mtd_dev_info *mtd, struct ubi_scan_info *si)
+static int flash_image(const struct mtd_dev_info *mtd, const struct ubigen_info *ui,
+		       struct ubi_scan_info *si)
 {
 	int fd, img_ebs, eb, written_ebs = 0, divisor;
 	off_t st_size;

@@ -518,7 +521,7 @@ static int flash_image(const struct mtd_dev_info *mtd, struct ubi_scan_info *si)
 			fflush(stdout);
 		}
 
-		err = change_ec((struct ubi_ec_hdr *)buf, ec);
+		err = change_ech((struct ubi_ec_hdr *)buf, ui->image_seq, ec);
 		if (err) {
 			errmsg("bad EC header at eraseblock %d of \"%s\"",
 			       written_ebs, args.image);

@@ -918,7 +921,7 @@ int main(int argc, char * const argv[])
 	}
 
 	if (args.image) {
-		err = flash_image(&mtd, si);
+		err = flash_image(&mtd, &ui, si);
 		if (err < 0)
 			goto out_free;
 

1.6.2.5

Could you please try them and report if they fix your problem or not?

P.S. when we fix this, we'll have to tag a new mtd-utils version, since this is a very bad bug.

Best Regards,
Artem Bityutskiy (Артём Битюцкий)

This patch worked for me.

FWIW, I am using 2.6.31 kernel and the mtd-utils last commit was: a4e502d99129da8ebba6d40b373a4422a175e9af.

I did receive a warning "ubi_eba_init_scan: cannot reserve enough PEBs for bad PEB handling, reserved 9, need 20" but I believe that is operator error related to the large volume size in the mtd partition.

I have one slight change from the original command line commands I posted. For mkfs.ubifs my original -e was incorrectly the PEB instead of the LEB. So for those kids following along, don't use the original command sequence.

So the new command line sequence on the x86 host looks like:

$ mkfs.ubifs -r build/targetfs_tiny -m 4096 -e 258048 -c 2047 -o tiny.ubifs

$ ubinize -s 1024 -o ubifs1b.img -m 4096 -p 256KiB ubifs1.cfg

$ cat ubifs1.cfg
[ubifs]
mode=ubi
image=tiny.ubifs
vol_id=0
vol_size=500MiB
vol_type=dynamic
vol_name=fs1
vol_alignment=1
vol_flags=autoresize

And on the PowerPC target:

# ubiformat -s 1024 -f ubifs1b.img /dev/mtd8
ubiformat: mtd8 (nand), size 536870912 bytes (512.0 MiB), 2048 eraseblocks of 262144 bytes (256.0 KiB), min. I/O size 4096 bytes
libscan: scanning eraseblock 2047 -- 100 % complete 

ubiformat: 2045 eraseblocks have valid erase counter, mean value is 6

ubiformat: 3 bad eraseblocks found, numbers: 72, 930, 1846

ubiformat: flashing eraseblock 30 -- 100 % complete

ubiformat: formatting eraseblock 2047 -- 100 % complete

# ubiattach /dev/ubi_ctrl -m 8
[ 3149.923007] UBI: attaching mtd8 to ubi0
[ 3149.927224] UBI: physical eraseblock size:   262144 bytes (256 KiB)
[ 3149.934138] UBI: logical eraseblock size:    258048 bytes
[ 3149.939914] UBI: smallest flash I/O unit:    4096
[ 3149.944762] UBI: sub-page size:              1024
[ 3149.949573] UBI: VID header offset:          1024 (aligned 1024)
[ 3149.955688] UBI: data offset:                4096
[ 3151.183443] UBI warning: ubi_eba_init_scan: cannot reserve enough PEBs for bad PEB handling, reserved 9, need 20
[ 3151.209875] UBI: volume 0 ("fs1") re-sized from 2032 to 2032 LEBs
[ 3151.224125] UBI: attached mtd8 to ubi0
[ 3151.228345] UBI: MTD device name:            "fs1"
[ 3151.234848] UBI: MTD device size:            512 MiB
[ 3151.239993] UBI: number of good PEBs:        2045
[ 3151.244830] UBI: number of bad PEBs:         3
[ 3151.249382] UBI: max. allowed volumes:       128
[ 3151.254106] UBI: wear-leveling threshold:    4096
[ 3151.258916] UBI: number of internal volumes: 1
[ 3151.263465] UBI: number of user volumes:     1
[ 3151.268013] UBI: available PEBs:             0
[ 3151.272562] UBI: total number of reserved PEBs: 2045
[ 3151.277633] UBI: number of PEBs reserved for bad PEB handling: 9
[ 3151.283748] UBI: max/mean erase counter: 8/7
[ 3151.288124] UBI: image sequence number: 882190247
[ 3151.292941] UBI: background thread "ubi_bgt0d" started, PID 1055
UBI device number 0, total 2045 LEBs (527708160 bytes, 503.3 MiB), available 0 LEBs (0 bytes), LEB size 258048 bytes (252.0 KiB)

# mount -t ubifs ubi0:fs1 /mnt/ubifs
[ 3165.535851] UBIFS: mounted UBI device 0, volume 0, name "fs1"
[ 3165.541636] UBIFS: file system size:   521773056 bytes (509544 KiB, 497 MiB, 2022 LEBs)
[ 3165.549640] UBIFS: journal size:       9420800 bytes (9200 KiB, 8 MiB, 37 LEBs)
[ 3165.556948] UBIFS: media format:       w4/r0 (latest is w4/r0)
[ 3165.562779] UBIFS: default compressor: lzo
[ 3165.566873] UBIFS: reserved for root:  0 bytes (0 KiB)

Jeff Angielski
The PTR Group
www.theptrgroup.com

Ok, pushed them.

Yes, create smaller volumes.

You need to make vol_size=500MiB smaller to get rid of the warning.

Best Regards,
Artem Bityutskiy (Артём Битюцкий)

Hi,

Artem> Ok, pushed them.

David, could we have a 1.3.1 tarball on
ftp://ftp.infradead.org/pub/mtd-utils/ as well?

Thanks.

Bye, Peter Korsgaard

Hi,

Artem> Ok, pushed them.

Peter> David, could we have a 1.3.1 tarball on Peter> ftp://ftp.infradead.org/pub/mtd-utils/ as well?

Peter> Thanks.

Ping?

Bye, Peter Korsgaard

Done.

dwmw2