Skip to contentSkip to conversation

ubi-utils: Introduce a utility ubidump

RFC PATCH Superseded hujianyang 4 participants 21 messages Jul 29, 2014 – Aug 5, 2014

Subsystems (3)

v2 of 3 Superseded Latest is v3 →

lore
Share & tools
lore

hujianyang

author
#0 lore raw

RFC PATCH v2 1/1 ubi-utils: Introduce a utility ubidump

Hi all,

After discussing with Artem, I've applied some of his ideas and rewrite this utility.

Bill also gives me help. I reference his code about getting data from MTD device.

Thank you all~!

v1:
http://lists.infradead.org/pipermail/linux-mtd/2014-July/054541.html

Changes in v2:
- modified ioctl() in ubi_vol which just return pnum now.
- change libubifs.c/libubifs.h into libdump.c/libdump.h and remove ubi-level header dump function into libdump.c.
- read data in eraseblock from MTD functionality.
- add a new function "ubidump" to handle peb dumping.
- modified options -H/-N into --ubi/--ubifs.

I found this utility can't dump lpt area correctly because we use different structure to format this area. I will enable it soon. I think we should confirm the mechanism of this utility first.

author
#1 lore raw

RFC PATCH v2 1/1 UBI: New ioctl() to support ubidump

An ioctl() return pnum of a specified leb.

---
 drivers/mtd/ubi/cdev.c      | 26 ++++++++++++++++++++++++++
 include/uapi/mtd/ubi-user.h | 12 ++++++++++++
 2 files changed, 38 insertions(+)
drivers/mtd/ubi/cdev.c +26 -0
index 7646220..6fa7346 100644
@@ -581,6 +581,32 @@ static long vol_cdev_ioctl(struct file *file, unsigned int cmd,
 		break;
 	}
 
+	/* Get pnum of a specified leb command */
+	case UBI_IOCEBGETPNUM:
+	{
+		struct ubi_lnum2pnum_req req;
+		int pnum;
+
+		err = copy_from_user(&req, argp,
+				sizeof(struct ubi_lnum2pnum_req));
+		if (err) {
+			err = -EFAULT;
+			break;
+		}
+
+		err = ubi_is_mapped(desc, req.lnum);
+		if (err <= 0)
+			break;
+		pnum = vol->eba_tbl[req.lnum];
+		req.pnum = pnum;
+
+		err = copy_to_user(argp, &req,
+				sizeof(struct ubi_lnum2pnum_req));
+		if (err)
+			err = -EFAULT;
+		break;
+	}
+
 	default:
 		err = -ENOTTY;
 		break;
include/uapi/mtd/ubi-user.h +12 -0
index 1927b0d..fc41ddb 100644
@@ -205,6 +205,8 @@
 #define UBI_IOCVOLCRBLK _IOW(UBI_VOL_IOC_MAGIC, 7, struct ubi_blkcreate_req)
 /* Remove the R/O block device */
 #define UBI_IOCVOLRMBLK _IO(UBI_VOL_IOC_MAGIC, 8)
+/* Get pnum of a specified leb */
+#define UBI_IOCEBGETPNUM _IOW(UBI_VOL_IOC_MAGIC, 9, struct ubi_lnum2pnum_req)
 
 /* Maximum MTD device name length supported by UBI */
 #define MAX_UBI_MTD_NAME_LEN 127
@@ -442,4 +444,14 @@ struct ubi_blkcreate_req {
 	__s8  padding[128];
 }  __packed;
 
+/**
+ * struct ubi_lnum2pnum_req - a data structure used in lnum translate requests.
+ * @lnum: logical eraseblock num to translate
+ * @pnum: physical eraseblock num @lnum mapped
+ */
+struct ubi_lnum2pnum_req {
+	__s32 lnum;
+	__s32 pnum;
+}  __packed;
+
 #endif /* __UBI_USER_H__ */
Signature · 1 line
1.8.1.4
#2 lore raw

Re: [PATCH RFC v2] UBI: New ioctl() to support ubidump

Quoted 7 lines · On 29 Jul 2014, [email protected] wrote:

An ioctl() return pnum of a specified leb.

---
drivers/mtd/ubi/cdev.c | 26 ++++++++++++++++++++++++++
include/uapi/mtd/ubi-user.h | 12 ++++++++++++

2 files changed, 38 insertions(+)

Please look at 'ubi_scan()',

http://sourcecodebrowser.com/mtd-utils/20110107/libscan_8c.html
http://sourcecodebrowser.com/mtd-utils/20110107/libscan_8h.html

After this call info->ec[pnum] is,

  EB_EMPTY      (ffffffff) - erased
  EB_CORRUPTED  (fffffffe) - inconsistent UBI data.
  EB_ALIEN      (fffffffd) - non-ubi erase sector
  EB_BAD        (fffffffc) - bad block

or the LNUM.

To write your code, you can either scan the array (~32-2048 entries) each time you want an 'lnum' or you could run through the array and construct the opposite table; a 'logical index' gives a 'physical eb'; this would require keeping track of the 'generation' count.

I think a patch to 'ubi_scan()' to create an 'pmap' array might be better or more accepted than a Linux/MTD/UBI patch? Then only the 'ubidump' code is needed and not a properly configured/versioned kernel; or at least only the nandsim module which is similar to some other utilities. If you had a 'ubi_scan()' which has an 'info->pmap[leb]' which had,

EB_EMPTY (ffffffff) - not mapped
or the PNUM.

Would you need to patch the kernel?

Fwiw,
Bill Pringlemeir.

author
#3 lore raw

Re: [PATCH RFC v2] UBI: New ioctl() to support ubidump

Quoted 8 lines

I think a patch to 'ubi_scan()' to create an 'pmap' array might be better or more accepted than a Linux/MTD/UBI patch? Then only the 'ubidump' code is needed and not a properly configured/versioned kernel; or at least only the nandsim module which is similar to some other utilities. If you had a 'ubi_scan()' which has an 'info->pmap[leb]' which had,

Yes, I think you are right~! I don't want this utility can't be run without a properly configured kernel and this full-scan method will be used again when we dumping with an image file.

But I'm worry about the performance. As we dump one specified peb now, we can't record the mapping table and need to do full scan each time running this utility.

Anyway, I will try what you said first~!

Thanks~!

Hu

#4 lore raw

Re: [PATCH RFC v2] UBI: New ioctl() to support ubidump

Quoted 3 lines · On 29 Jul 2014, [email protected] wrote:

But I'm worry about the performance. As we dump one specified peb now, we can't record the mapping table and need to do full scan each time running this utility.

You can record the table if you know that the MTD is not changing. If you introduce the 'ioctl()' to the kernel, it will most likely be on an active file system. I guess this is a big decision in the tools design.

Supporting an active file system is much more difficult. For the case of an real MTD backing store, I was thinking this would run from another partition or an initramfs; UBI/UbiFS would not be active. It could possibly attempt to repair the UBI/UbiFs, while nothing else is using it. The other use case is to read out the flash via some tools and loopback mount it with nandsim on a PC for diagnosis; a post-mortem analysis.

If you wish to run the utility on an active MTD, I would suggest some 'freeze/thaw' type function to stop UBI/UbiFs and then just use the tool on the 'frozen' MTD. Maybe this exists already for suspend/resume or something else. It might only be a debug type interface; so conditionally compiled for developers. It needs a hook so you can trigger the freeze on some condition via code.

I think others don't understand why you would want the 'ubidump' to run on an active file system? At least, I don't understand this? Is there some use case we are missing? I am quite sure that you can perform better diagnostics on a 'frozen' image where you can look at the whole image state.

Fwiw,
Bill Pringlemeir.

author
#5 lore raw

Re: [PATCH RFC v2] UBI: New ioctl() to support ubidump

Quoted 4 lines · Bill Pringlemeir

You can record the table if you know that the MTD is not changing. If you introduce the 'ioctl()' to the kernel, it will most likely be on an active file system. I guess this is a big decision in the tools design.

This ioctl() doesn't need UBIFS filesystem to be mounted, just needs UBI driver. We can keep this partition not active during this tool running.

But a properly configured kernel requirement you said in last mail worries me most. I think you are right and want to have a try. But I've discussed these stuff with Artem and he said, just quote:

""
So I envision that the tool would work like this.

$ ubidump my.img --lnum 5
  - dump LEB 5, will need to scan the entire image.
$ ubidump /dev/ubiX_Y --lnum 5
  - will just ask the UBI driver to give the PEB number for LEB 5, then
find out the MTD device for this volume (should be possible by checking
the sysfs files), and then reads the MTD device, and gets the UBI-level
information from there.

Something like this, just quick thoughts.
""
http://lists.infradead.org/pipermail/linux-mtd/2014-July/054674.html

We can keep a temporary file to record the mapping table but that will make the initial submission much more complicated. But this way you mentioned to record the table should be considered to add into this tool in the future.

Quoted 7 lines

I think others don't understand why you would want the 'ubidump' to run on an active file system? At least, I don't understand this? Is there some use case we are missing? I am quite sure that you can perform better diagnostics on a 'frozen' image where you can look at the whole image state.

Yes, I want this tool to be flexible before. But dumping with an active partition really useless as you said now. Because in most cases, the volumes we want to dump must have something wrong.

I think we need to make a lighter version and submit it to ubi-utils at beginning, then make it strongly step by step. Despite all that, we should have a clear design of what we need now. So I'm happy with your sharing of your thoughts.

The "freeze/thaw" method will result a complex tool. We can use some functionality to make sure UBIFS partition is not active. Should I need to add them in this version or enable them in the future? Do you think it is a key problem of this tool? This utility may tolerance some data mistake caused by peb update race in my considering.

#6 lore raw

Re: [PATCH RFC v2] UBI: New ioctl() to support ubidump

Quoted 20 lines · hujianyang
Quoted 4 lines · Bill Pringlemeir

You can record the table if you know that the MTD is not changing. If you introduce the 'ioctl()' to the kernel, it will most likely be on an active file system. I guess this is a big decision in the tools design.

This ioctl() doesn't need UBIFS filesystem to be mounted, just needs UBI driver. We can keep this partition not active during this tool running.

But a properly configured kernel requirement you said in last mail worries me most. I think you are right and want to have a try. But I've discussed these stuff with Artem and he said, just quote:

""
So I envision that the tool would work like this.

$ ubidump my.img --lnum 5
  - dump LEB 5, will need to scan the entire image.
Quoted 8 lines
$ ubidump /dev/ubiX_Y --lnum 5
  - will just ask the UBI driver to give the PEB number for LEB 5, then
find out the MTD device for this volume (should be possible by checking
the sysfs files), and then reads the MTD device, and gets the UBI-level
information from there.

Something like this, just quick thoughts.
""

Having a tool which does not need any kernel help is a lot more preferable. I though we may have an ioctl like this if no one has time to write full scanning support in user-space, and then maintain it.

Quoted 4 lines

We can keep a temporary file to record the mapping table but that will make the initial submission much more complicated. But this way you mentioned to record the table should be considered to add into this tool in the future.

Hujianyang, AFAIU, right now you need an ability to dump UBIFS stuff. You do not really need to know PEB number. So ubidump may analyze /dev/ubiX_Y without knowing the PEB number, right? It can just read from /dev/ubiX_Y directly. And print you all the UBIFS nodes.

I mean, if what user gives you is an UBI volume, you do not try going to the MTD level.

Signature · 2 lines
Best Regards,
Artem Bityutskiy
author
#7 lore raw

Re: [PATCH RFC v2] UBI: New ioctl() to support ubidump

Quoted 2 lines · Artem Bityutskiy

Hujianyang, AFAIU, right now you need an ability to dump UBIFS stuff.

Yes, but not all of what I want. I need this utility dumping UBI-level stuff too. The ability, not only to dump UBIFS stuff but also to dump UBI stuff will make it a useful tool.

Quoted 4 lines

You do not really need to know PEB number. So ubidump may analyze /dev/ubiX_Y without knowing the PEB number, right? It can just read from /dev/ubiX_Y directly. And print you all the UBIFS nodes.

Last version I sent to this maillist was reading data from UBI driver directly. But our .read() function in UBI driver jump over UBI stuff, so I read UBI stuff from an ioctl(). This isn't a common way as you said.

I have to say I am about to leave Beijing for at least 3 months and have no time for the development of this tool. I aspire if we can push some of my work to ubi-utils before my leaving. So we should hurry up~!

First, we can leave UBI/UBIFS format dumping behind, just dump binary data, that will make initial submission smaller.

Then, think about what we want at this time.

1) No UBI stuff and read UBIFS stuff from UBI-driver
This is a easiest way and we don't need any ioctl either. We can't get UBI-level stuff right now but we can put this into our design of dumping an image file.

2) Read data from MTD-driver and find a way to get pnum This way is basing on my v2 work. We can put an ioctl() to kernel and if kernel don't support this ioctl(), this tool will do a full-scan on MTD device. It's transparent to user. But we should take some time to consider how to do this full-scan.

3) Re-design this tool about dropping UBI driver based dumping. This way is like the exist ubiformat utility. We don't use UBI driver and just do read/write(maybe we can do some repairing later) with MTD device or image files. It's not small.

I think you will prefer to solution 1. Although it's not fit all my desires, I will agree with it if you insist. Or do you have any other suggestions?

Thanks~!

Hu

#8 lore raw

Re: [PATCH RFC v2] UBI: New ioctl() to support ubidump

Quoted 7 lines · hujianyang
Quoted 2 lines · Artem Bityutskiy

Hujianyang, AFAIU, right now you need an ability to dump UBIFS stuff.

Yes, but not all of what I want. I need this utility dumping UBI-level stuff too. The ability, not only to dump UBIFS stuff but also to dump UBI stuff will make it a useful tool.

OK.

Quoted 8 lines
Quoted 4 lines

You do not really need to know PEB number. So ubidump may analyze /dev/ubiX_Y without knowing the PEB number, right? It can just read from /dev/ubiX_Y directly. And print you all the UBIFS nodes.

Last version I sent to this maillist was reading data from UBI driver directly. But our .read() function in UBI driver jump over UBI stuff, so I read UBI stuff from an ioctl(). This isn't a common way as you said.

There are 2 layers involved: MTD and UBI.

On the MTD level, the entire flash contents is available (/dev/mtdX) On the UBI level, only UBI volumes are available (/dev/ubiX_Y)

If I give you an UBI volume dump (cp /dev/ubiX_Y ubi.img), you will only see the contents of the UBI volume. If there was UBIFS file-system, you will see UBIFS nodes in the "ubi.img" file. You will not see UBI headers there.

The ideal design would be treating /dev/ubiX_Y the same way as 'ubi.img'. IOW:

ubidump ubi.img
and
ubidump /dev/ubiX_Y

should give the same results.

If I give you an MTD image (nanddump -o mtd.img /dev/mtdX), you should be able to see both UBIFS and UBI stuff in 'mtd.img'. You may see the same information in /dev/mtdX. To get the LEB<->PEB mapping, you need to do scanning, etc. So

ubidump mtd.img
and
ubidump /dev/mtdX

should be the same.

This is the basic picture.

Now what you want is to add an exception to this scheme. Namely, for the 'ubidump /dev/ubiX_Y' case: you want to get help from the driver, using the ioctl you introduced.

Frankly, I am not 100% sure this is a good idea, would be nice to hear from other people. I'd need to think a bit on this.

So what I suggested is the you do not submit this exception so far. Just teach ubidump to handle UBI volumes. I.e., make it work with

ubidump ubi.img
ubidump /dev/ubiX_Y

yes, these 2 will dump only ubifs stuff. Limited functionality.

Quoted 4 lines

1) No UBI stuff and read UBIFS stuff from UBI-driver
This is a easiest way and we don't need any ioctl either. We can't get UBI-level stuff right now but we can put this into our design of dumping an image file.

Yes. That's a good first step.

Quoted 5 lines

2) Read data from MTD-driver and find a way to get pnum This way is basing on my v2 work. We can put an ioctl() to kernel and if kernel don't support this ioctl(), this tool will do a full-scan on MTD device. It's transparent to user. But we should take some time to consider how to do this full-scan.

I guess, I'd need to think some more about the ioctl(), and may be someone else suggests something.

Quoted 4 lines

3) Re-design this tool about dropping UBI driver based dumping. This way is like the exist ubiformat utility. We don't use UBI driver and just do read/write(maybe we can do some repairing later) with MTD device or image files. It's not small.

Probably, we'll think about this when we are done with 1) and may be 2).

Signature · 2 lines
Best Regards,
Artem Bityutskiy
author
#9 lore raw

Re: [PATCH RFC v2] UBI: New ioctl() to support ubidump

Quoted 37 lines · Artem Bityutskiy

There are 2 layers involved: MTD and UBI.

On the MTD level, the entire flash contents is available (/dev/mtdX) On the UBI level, only UBI volumes are available (/dev/ubiX_Y)

If I give you an UBI volume dump (cp /dev/ubiX_Y ubi.img), you will only see the contents of the UBI volume. If there was UBIFS file-system, you will see UBIFS nodes in the "ubi.img" file. You will not see UBI headers there.

The ideal design would be treating /dev/ubiX_Y the same way as 'ubi.img'. IOW:

ubidump ubi.img
and
ubidump /dev/ubiX_Y

should give the same results.

If I give you an MTD image (nanddump -o mtd.img /dev/mtdX), you should be able to see both UBIFS and UBI stuff in 'mtd.img'. You may see the same information in /dev/mtdX. To get the LEB<->PEB mapping, you need to do scanning, etc. So

ubidump mtd.img
and
ubidump /dev/mtdX

should be the same.

This is the basic picture.

Now what you want is to add an exception to this scheme. Namely, for the 'ubidump /dev/ubiX_Y' case: you want to get help from the driver, using the ioctl you introduced.

Quoted 9 lines

Frankly, I am not 100% sure this is a good idea, would be nice to hear from other people. I'd need to think a bit on this.

So what I suggested is the you do not submit this exception so far. Just teach ubidump to handle UBI volumes. I.e., make it work with

ubidump ubi.img
ubidump /dev/ubiX_Y

Oh, I've confused these things before. I want this utility work with mtd.img and /dev/ubiX_Y. It seems they are not same.

Quoted 9 lines

yes, these 2 will dump only ubifs stuff. Limited functionality.

Quoted 4 lines

1) No UBI stuff and read UBIFS stuff from UBI-driver
This is a easiest way and we don't need any ioctl either. We can't get UBI-level stuff right now but we can put this into our design of dumping an image file.

Yes. That's a good first step.

How about writing a new version basing on this?

Quoted 9 lines
Quoted 5 lines

2) Read data from MTD-driver and find a way to get pnum This way is basing on my v2 work. We can put an ioctl() to kernel and if kernel don't support this ioctl(), this tool will do a full-scan on MTD device. It's transparent to user. But we should take some time to consider how to do this full-scan.

I guess, I'd need to think some more about the ioctl(), and may be someone else suggests something.

OK.

Have a good weekend~!

Hu

#10 lore raw

Re: [PATCH RFC v2] UBI: New ioctl() to support ubidump

In reply to Artem Bityutskiy
Quoted 8 lines · On 1 Aug 2014, [email protected] wrote:
Quoted 4 lines · hujianyang

1) No UBI stuff and read UBIFS stuff from UBI-driver This is a easiest way and we don't need any ioctl either. We can't get UBI-level stuff right now but we can put this into our design of dumping an image file.

Yes. That's a good first step.

I agree that this is a good way to partition the functionality.

Quoted 8 lines
Quoted 5 lines

2) Read data from MTD-driver and find a way to get pnum This way is basing on my v2 work. We can put an ioctl() to kernel and if kernel don't support this ioctl(), this tool will do a full-scan on MTD device. It's transparent to user. But we should take some time to consider how to do this full-scan.

I guess, I'd need to think some more about the ioctl(), and may be someone else suggests something.

Isn't this just more work? The tool will rely on two things?

Quoted 7 lines
Quoted 4 lines

3) Re-design this tool about dropping UBI driver based dumping. This way is like the exist ubiformat utility. We don't use UBI driver and just do read/write(maybe we can do some repairing later) with MTD device or image files. It's not small.

Probably, we'll think about this when we are done with 1) and may be 2).

Some of these points I don't understand. The 'ubiformat' already uses 'libscan.c', which have the basics for creating a static 'PEB->LEB' mapping. So while I agree that maintaining two version of UBI is not good, I think we already have this?

UBI has to deal with writing, especially atomic writes, and concurrency. I don't think the tool needs either, so the 2nd version of UBI is significantly simpler. Just look at 'libscan.c' versus 'driver/mtd/ubi/*.c'.

Also, I think a lot of people will have issues where they can not mount the device and the only thing they can get is the whole 'nanddump'; which won't be done through Linux, but some other means. Ie, some ROM boot code (which doesn't understand either UBI or UbiFS) to read out NAND, a NAND programmer, etc.

Finally, the separate implementation can be good. Having two versions of code use the same data structures will often show issues. If we want to debug UBI, then using it is not so helpful. Having the separate (simpler) implementation with worse performance is exactly the kind of thing you want for debugging and repair, isn't it? Features like 'fastmap' are optional and we should fall back to a linear scan. I think that future UBI changes will strive to be backwards compatible unless there is some really good reason not to be?

The big advantage of doing everything outside the kernel is we can eventually add lots of extra consistency checks that you would never want to do in a running file system. Ie, verify a linear scan and the fastmap are consistent, etc.

My intent is never to make work for anyone. Adding a patch to libscan.c seems easy to me.

1. Allocate a 'pmap' array and a temporary 'sequence' array.
2. Initialize 'pmap' to -1.
3a. For each valid 'eb', get 'lnum' and 'sqnum' from vid header.
b. if (pmap[lnum] == -1 || sequence[lnum] < sqnum)

      pmap[lnum] = eb, sequence[lnum] = sqnum;

but maybe I don't understand all the complexities. Isn't it that simple?

Now if we want to use the tool on live volumes, 'libscan.c' will not work for that. I thought that the concepts of a debugger may apply. It can look at a core file, or a paused processes memory. But maybe it is crazy to 'pause' Ubi/UbiFs. Someone might want such 'snapshots' to examine the evolution of storage or some race issues... but I guess that is not the main reason of using the ioctl().

Fwiw,
Bill Pringlemeir.

author
#11 lore raw

Re: [PATCH RFC v2] UBI: New ioctl() to support ubidump

Quoted 7 lines · Bill Pringlemeir

1. Allocate a 'pmap' array and a temporary 'sequence' array.
2. Initialize 'pmap' to -1.
3a. For each valid 'eb', get 'lnum' and 'sqnum' from vid header.
b. if (pmap[lnum] == -1 || sequence[lnum] < sqnum)

      pmap[lnum] = eb, sequence[lnum] = sqnum;

I've researched the function ubi_scan() in libscan.c today and found it's not easy to just add small changes to enable LEB->PEB mapping table as we want.

There are two methods on enabling this functionality:
1) Add a mapping table in struct ubi_scan_info
2) Add some new parameters for function ubi_scan()

You know each MTD device may contain more than one UBI volume and each volume should has a private mapping table. So it's not easy to enable it in struct ubi_scan_info like method 1) because we don't know the actual counts of volumes.

Further more, @sqnum is not enough for peb determining, we should consider @on_copy flag for wear-leveling and atomic write. If @on_copy flag is set, we need to read the whole leb and check CRC to determine which peb is right. As this is a debugging tool, I think printing all pebs have same lnum(of course in same volume) is better. Now, we have to start thinking how to record this stuff, that's a big problem.

In another way like method 2), we can add a new structure named ubi_translate, put lnum and vol_id in it and returns a list or an array contains the set of pebs which has a lnum equal to what we set in translate struct. Original call like ubiformat set this structure pointer to NULL and only ubidump use it. But if it's better than writing a new function to do this stuff separately? The code in ubi_scan is complicated now. Lnum translation is not needed for ubiformat and the calculate of ec is not needed for ubidump.

How about adding a new function to scan the whole MTD device and just return a list contain all the pebs(mostly 1 or 2) in same vol_id and same lnum?

in libscan.c

struct ubi_translate_info {
int lnum,
int vol_id,
int find, // counts of peb we find
int *array // dynamic or static(5? in size) array for
// recording peb num
};

int ubi_translate(struct ubi_translate_info *info /* or **info */)

{
	// full scan the MTD device
	while (...) {
		read(ec);
		
		// determine if it is a valid peb
		...

		read(vid);

		if (vid->vol_id == info->vol_id &&
		    vid->lnum == info->lnum) {
			// add to list @array
			...

			find++;			
		}
	}

	return err; // MTD is bad or something else
}

Just quick thought.

What's your opinion?

By the way, I have to say this separating of UBI and UBIFS makes it really hard to read data from both sides.

#12 lore raw

Re: [PATCH RFC v2] UBI: New ioctl() to support ubidump

Quoted 12 lines · On 3 Aug 2014, [email protected] wrote:
Quoted 5 lines · Bill Pringlemeir

1. Allocate a 'pmap' array and a temporary 'sequence' array.
2. Initialize 'pmap' to -1.
3a. For each valid 'eb', get 'lnum' and 'sqnum' from vid header. b. if (pmap[lnum] == -1 || sequence[lnum] < sqnum) pmap[lnum] = eb, sequence[lnum] = sqnum;

I've researched the function ubi_scan() in libscan.c today and found it's not easy to just add small changes to enable LEB->PEB mapping table as we want.

Quoted 8 lines

There are two methods on enabling this functionality:
1) Add a mapping table in struct ubi_scan_info
2) Add some new parameters for function ubi_scan()

You know each MTD device may contain more than one UBI volume and each volume should has a private mapping table. So it's not easy to enable it in struct ubi_scan_info like method 1) because we don't know the actual counts of volumes.

Yes, I think you are completely correct. The array only keeps track of the erase count not the LEB. Sorry, I was confused about that.

Quoted 7 lines

Further more, @sqnum is not enough for peb determining, we should consider @on_copy flag for wear-leveling and atomic write. If @on_copy flag is set, we need to read the whole leb and check CRC to determine which peb is right. As this is a debugging tool, I think printing all pebs have same lnum(of course in same volume) is better. Now, we have to start thinking how to record this stuff, that's a big problem.

Validating the header CRC is always part of seeing if the block is valid. For a real physical device, the unstable bits maybe an issue. So, it is probably better to examine 'copy_flag'. Many of these are abnormal cases, which need to be handled, but the tool would not normally run through these paths for every block. Running the CRC on the data maybe particularly slow.

Quoted 9 lines

In another way like method 2), we can add a new structure named ubi_translate, put lnum and vol_id in it and returns a list or an array contains the set of pebs which has a lnum equal to what we set in translate struct. Original call like ubiformat set this structure pointer to NULL and only ubidump use it. But if it's better than writing a new function to do this stuff separately? The code in ubi_scan is complicated now. Lnum translation is not needed for ubiformat and the calculate of ec is not needed for ubidump.

I agree it is complex. It could be decomposed into functions.

Quoted 28 lines

How about adding a new function to scan the whole MTD device and just return a list contain all the pebs(mostly 1 or 2) in same vol_id and same lnum?

in libscan.c

struct ubi_translate_info {
int lnum,
int vol_id,
int find, // counts of peb we find
int *array // dynamic or static(5? in size) array for
// recording peb num
};

int ubi_translate(struct ubi_translate_info *info /* or **info */)

{
	// full scan the MTD device
	while (...) {
		read(ec);
		
		// determine if it is a valid peb
		...

		read(vid);

		if (vid->vol_id == info->vol_id &&
		    vid->lnum == info->lnum) {
			// add to list @array
Quoted 8 lines
			...

			find++;			
		}
	}

	return err; // MTD is bad or something else
}

ubi_translate() is more complex. You need to read the VID header and possibly data as well. The ubi_scan() never reads these.

However, a lot of the erase header checking will be the same. So besides all 'si->...=' lines most of ubi_scan() loop body is needed for the 'ubi_translate()' to validate the erase header. Then you need the extra step to read the 'vid'. I think you could have a function like,

/* ech invalid unless return < MAX_EC */
static uint32_t read_ehdr(struct mtd_dev_info *mtd, int fd,
int eb, struct ubi_ec_hdr *ech)
{
...

That returns the 'enum' or erase count. The 'enum' could be amended with a 'fatal', which is the current 'goto out_ec' path (or you need another parameter). Then, the current ubi_scan() could be,

	for (eb = 0; eb < mtd->eb_cnt; eb++) {
		uint32_t ec;
		ec = read_ehdr(mtd, fd, eb, &ech);
                switch(ec) {
                   ...
                /* Do stuff to 'si->... here. */
			break;
		case EB_FATAL:
			goto out_ec;

And then ubi_translate() could reuse the read_ehdr() to do the 'determine if a valid peb' part. Ie, whether you should even read the 'vid'.

Quoted 2 lines

By the way, I have to say this separating of UBI and UBIFS makes it really hard to read data from both sides.

You mean both layers at once? Ie, layers == sides? Or do you mean to support reading from an 'mtd' or a 'ubi' device?

Regards,
Bill Pringlemeir.

#13 lore raw

Re: [PATCH RFC v2] UBI: New ioctl() to support ubidump

In reply to Bill Pringlemeir
Quoted 5 lines · Bill Pringlemeir

I think others don't understand why you would want the 'ubidump' to run on an active file system? At least, I don't understand this? Is there some use case we are missing? I am quite sure that you can perform better diagnostics on a 'frozen' image where you can look at the whole image state.

I think you are right that in ideal world ubidump should be able to figure out the LEB->PEB mapping without asking the driver. It should be able to scan the image and figure everything out itself.

However, writing such a toll is _hard_. Maintaining it is _hard too_. You basically need to copy part of the driver to user-space, and maintain it there. Whenever someone adds a new UBI feature like "fastmap", you need to port that to the tool too.

This is doable, e.g., e2fsprogs project basically does this. But so far, no one did this for UBI/UBIFS. And the UBIFS user-base is relatively small. Many people expressed a desire to have a 'chkfs.ubifs' and 'ubidump', but no one went ahead ad did this - this is not easy and this costs.

So I initially thought may be a little help from the kernel would not hurt. But I am not 100% sure this is a good idea.

Signature · 2 lines
Best Regards,
Artem Bityutskiy
#14 lore raw

Re: [PATCH RFC v2] UBI: New ioctl() to support ubidump

In reply to hujianyang
Quoted 25 lines · hujianyang

An ioctl() return pnum of a specified leb.

---
 drivers/mtd/ubi/cdev.c      | 26 ++++++++++++++++++++++++++
 include/uapi/mtd/ubi-user.h | 12 ++++++++++++
 2 files changed, 38 insertions(+)
drivers/mtd/ubi/cdev.c +8 -0
index 7646220..6fa7346 100644
@@ -581,6 +581,32 @@ static long vol_cdev_ioctl(struct file *file, unsigned int cmd,
                break;
        }
 
+       /* Get pnum of a specified leb command */
+       case UBI_IOCEBGETPNUM:
+       {
+               struct ubi_lnum2pnum_req req;
+               int pnum;
+
+               err = copy_from_user(&req, argp,
+                               sizeof(struct ubi_lnum2pnum_req));

drivers/mtd/ubi/cdev.c static long vol_cdev_ioctl(struct file *file, unsigned int cmd, · line 601

Quoted 10 lines
drivers/mtd/ubi/cdev.c +10 -0
+               if (err) {
+                       err = -EFAULT;
+                       break;
+               }
+
+               err = ubi_is_mapped(desc, req.lnum);
+               if (err <= 0)
+                       break;
+               pnum = vol->eba_tbl[req.lnum];
+               req.pnum = pnum;

Isn't this racy?
i.e. If a LEB change happens between ubi_is_mapped() and "pnum = vol->eba_tbl[req.lnum]".

Quoted 45 lines
drivers/mtd/ubi/cdev.c +7 -0
+               err = copy_to_user(argp, &req,
+                               sizeof(struct ubi_lnum2pnum_req));
+               if (err)
+                       err = -EFAULT;
+               break;
+       }
+
        default:
                err = -ENOTTY;
                break;
include/uapi/mtd/ubi-user.h +12 -1
index 1927b0d..fc41ddb 100644
@@ -205,6 +205,8 @@
 #define UBI_IOCVOLCRBLK _IOW(UBI_VOL_IOC_MAGIC, 7, struct ubi_blkcreate_req)
 /* Remove the R/O block device */
 #define UBI_IOCVOLRMBLK _IO(UBI_VOL_IOC_MAGIC, 8)
+/* Get pnum of a specified leb */
+#define UBI_IOCEBGETPNUM _IOW(UBI_VOL_IOC_MAGIC, 9, struct ubi_lnum2pnum_req)
 
 /* Maximum MTD device name length supported by UBI */
 #define MAX_UBI_MTD_NAME_LEN 127
@@ -442,4 +444,14 @@ struct ubi_blkcreate_req {
        __s8  padding[128];
 }  __packed;
 
+/**
+ * struct ubi_lnum2pnum_req - a data structure used in lnum translate requests.
+ * @lnum: logical eraseblock num to translate
+ * @pnum: physical eraseblock num @lnum mapped
+ */
+struct ubi_lnum2pnum_req {
+       __s32 lnum;
+       __s32 pnum;
+}  __packed;
+
 #endif /* __UBI_USER_H__ */
--

1.8.1.4

______________________________________________________
Linux MTD discussion mailing list
http://lists.infradead.org/mailman/listinfo/linux-mtd/

Signature · 2 lines
Thanks,
//richard
author
#15 lore raw

Re: [PATCH RFC v2] UBI: New ioctl() to support ubidump

Quoted 5 lines

Isn't this racy?
i.e. If a LEB change happens between ubi_is_mapped() and "pnum = vol->eba_tbl[req.lnum]".

As I was writing this code, I used this ubi_is_mapped() to make sure lnum is less than @vol->reserved_pebs and !@vol->upd_marker in order to avoid kernel panic when we get pnum from eba_tbl.

Even we locked this ioctl, peb may also be changed before we read it from MTD. That's why I didn't focus on this race. We can use some condition check to make sure this vol is not mounted but user may use this utility on rootfs.

Bill showed me a new method to get pnum without UBI driver. I will try that first but this new method can't avoid peb changes before we read it either. We should consider more on it.

Thanks!

Hu

author
#16 lore raw

PATCH 1/5 ubi-utils: Add libdump files

In reply to hujianyang

I've changed the name of libubifs.c/libubifs.h into libdump.c/h and move ubi-level header dump functions into libdump.c.

---
 ubi-utils/include/libdump.h |  72 +++++++
 ubi-utils/libdump.c         | 473 ++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 545 insertions(+)
 create mode 100644 ubi-utils/include/libdump.h
 create mode 100644 ubi-utils/libdump.c
ubi-utils/include/libdump.h new file +72 -0
new file mode 100644
index 0000000..89a50d7
@@ -0,0 +1,72 @@
+/*
+ * Copyright (c) International Business Machines Corp., 2006
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See
+ * the GNU General Public License for more details.
+ *
+ * Author: Hu Jianyang <[email protected]>
+ *
+ * UBIFS library.
+ */
+
+#ifndef __LIBDUMP_H__
+#define __LIBDUMP_H__
+
+#include <mtd/ubifs-media.h>
+#include <mtd/ubi-media.h>
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+#define ALIGN(x,a) __ALIGN_MASK(x,(typeof(x))(a)-1)
+#define __ALIGN_MASK(x,mask) (((x)+(mask))&~(mask))
+
+#define min_t(t,x,y) ({ \
+	typeof((x)) _x = (x); \
+	typeof((y)) _y = (y); \
+	(_x < _y) ? _x : _y; \
+})
+
+/**
+ * ubidump - dump ubi/ubifs information
+ * @buf: buffer to dump
+ * @peb_size: size of buffer/MTD eraseblock size
+ * @ubi: dump ubi-level stuff
+ * @ubifs: dump ubifs-level stuff
+ *
+ * This function dump ubi/ubifs information on the buffer.
+ */
+void ubidump(void *buf, int peb_size, int ubi, int ubifs, int detailed);
+
+/*
+ * 'ubifs_scan_a_node()' return values.
+ *
+ * SCANNED_GARBAGE:  scanned garbage
+ * SCANNED_EMPTY_SPACE: scanned empty space
+ * SCANNED_A_NODE: scanned a valid node
+ * SCANNED_A_CORRUPT_NODE: scanned a corrupted node
+ * SCANNED_A_BAD_PAD_NODE: scanned a padding node with invalid pad length
+ *
+ * Greater than zero means: 'scanned that number of padding bytes'
+ */
+enum {
+	SCANNED_GARBAGE        = 0,
+	SCANNED_EMPTY_SPACE    = -1,
+	SCANNED_A_NODE         = -2,
+	SCANNED_A_CORRUPT_NODE = -3,
+	SCANNED_A_BAD_PAD_NODE = -4,
+};
+
+#ifdef __cplusplus
+}
+#endif
+
+#endif /*!__LIBDUMP_H__ */
ubi-utils/libdump.c new file +473 -0

473 lines not on this page · raw message

Signature · 1 line
1.8.1.4
author
#17 lore raw

PATCH 2/5 ubi-utils: Enable lnum2pnum ioctl in userspace

In reply to hujianyang

Introduce ubi_lnum_to_pnum() to translate lnum into pnum by the a new ioctl.

---
 include/mtd/ubi-user.h     | 12 ++++++++++++
 ubi-utils/include/libubi.h | 11 +++++++++++
 ubi-utils/libubi.c         | 12 ++++++++++++
 3 files changed, 35 insertions(+)
include/mtd/ubi-user.h +12 -0
index 2b50dad..e4f57c6 100644
@@ -200,6 +200,8 @@
 #define UBI_IOCVOLCRBLK _IOW(UBI_VOL_IOC_MAGIC, 7, struct ubi_blkcreate_req)
 /* Remove the R/O block device */
 #define UBI_IOCVOLRMBLK _IO(UBI_VOL_IOC_MAGIC, 8)
+/* Get pnum of a specified leb */
+#define UBI_IOCEBGETPNUM _IOW(UBI_VOL_IOC_MAGIC, 9, struct ubi_lnum2pnum_req)
 
 /* Maximum MTD device name length supported by UBI */
 #define MAX_UBI_MTD_NAME_LEN 127
@@ -437,4 +439,14 @@ struct ubi_blkcreate_req {
 	int8_t  padding[128];
 }  __attribute__((packed));
 
+/**
+ * struct ubi_lnum2pnum_req - a data structure used in lnum translate requests.
+ * @lnum: logical eraseblock num to translate
+ * @pnum: physical eraseblock num @lnum mapped
+ */
+struct ubi_lnum2pnum_req {
+	int32_t lnum;
+	int32_t pnum;
+}  __attribute__((packed));
+
 #endif /* __UBI_USER_H__ */
ubi-utils/include/libubi.h +11 -0
index 4d6a7ee..618a6ba 100644
@@ -478,6 +478,17 @@ int ubi_leb_unmap(int fd, int lnum);
  */
 int ubi_is_mapped(int fd, int lnum);
 
+/**
+ * ubi_lnum_to_pnum - get the pnum of a specified leb.
+ * @fd: volume character device file descriptor
+ * @lnum: logical eraseblock number to translate
+ * @pnum: physical eraseblock number to return
+ *
+ * This function return %0 in case of success and %-1 in case of failure. The
+ * pnum of the specified leb is returned by @pnum.
+ */
+int ubi_lnum_to_pnum(int fd, int lnum, int *pnum);
+
 #ifdef __cplusplus
 }
 #endif
ubi-utils/libubi.c +12 -0
index 1e08b7d..9650b6a 100644
@@ -1123,6 +1123,18 @@ int ubi_vol_block_remove(int fd)
 	return ioctl(fd, UBI_IOCVOLRMBLK);
 }
 
+int ubi_lnum_to_pnum(int fd, int lnum, int *pnum)
+{
+	struct ubi_lnum2pnum_req request;
+
+	request.pnum = -1;
+	request.lnum = lnum;
+	if (ioctl(fd, UBI_IOCEBGETPNUM, &request))
+		return -1;
+	*pnum = request.pnum;
+	return 0;
+}
+
 int ubi_update_start(libubi_t desc, int fd, long long bytes)
 {
 	desc = desc;
Signature · 1 line
1.8.1.4
author
#18 lore raw

PATCH 3/5 ubi-utils: Introduce ubidump

In reply to hujianyang

Read data by MTD functionality now. Also move some dump code to libdump.

rename options:

    -n   --->  -l
    -N   --->  --ubifs
    -H   --->  --ubi
---
 ubi-utils/ubidump.c | 293 ++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 293 insertions(+)
 create mode 100644 ubi-utils/ubidump.c
ubi-utils/ubidump.c new file +293 -0
new file mode 100644
index 0000000..95d055b
@@ -0,0 +1,293 @@
+/*
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See
+ * the GNU General Public License for more details.
+ */
+
+/*
+ * An utility to dump UBI/UBIFS format data in eraseblock
+ *
+ * Author: Hu Jianyang <[email protected]>
+ */
+
+#define PROGRAM_NAME	"ubidump"
+
+#include <stdlib.h>
+#include <getopt.h>
+#include <unistd.h>
+
+#include <libubi.h>
+#include <libdump.h>
+#include <libmtd.h>
+#include "common.h"
+#include "ubiutils-common.h"
+
+#define MTD_DEV_PATT "/dev/mtd%d"
+
+/* The variables below are set by command line arguments */
+struct args {
+	const char *vol;
+	int lnum;
+	int info:1;
+	int ubi:1;
+	int ubifs:1;
+};
+
+static struct args args =
+{
+	.vol	= NULL,
+	.lnum	= -1,
+	.info	= 0,
+	.ubi	= 0,
+	.ubifs	= 0,
+};
+
+static const char doc[] = PROGRAM_NAME " version " VERSION
+		" - an utility to dump UBI/UBIFS format data in eraseblock";
+
+static const char optionsstr[] =
+"-h, --help		print help message\n"
+"-l, --lnum		logic eraseblock num to dump\n"
+"-i, --info		show explicit information about ubifs-level\n"
+"    --ubi		dump ubi-level stuff only\n"
+"    --ubifs		dump ubifs-level stuff only\n"
+"-V, --version		print program version";
+
+static const char usage[] =
+"Usage: " PROGRAM_NAME " <UBI volume node file name> [-l <lnum>] [-i]\n"
+"\t\t\t[--help] [--version]\n\n"
+"Example 1: " PROGRAM_NAME " /dev/ubi0_1 --lnum 2 - dump leb 2 in volume 1\n"
+"Example 2: " PROGRAM_NAME " /dev/ubi0_0 -l 1234 -i - dump leb 1234 with explicit info\n";
+
+static const struct option long_options[] = {
+	{ .name = "ubi",     .has_arg = 0, .flag = NULL, .val = 0   },
+	{ .name = "ubifs",   .has_arg = 0, .flag = NULL, .val = 0   },
+	{ .name = "help",    .has_arg = 0, .flag = NULL, .val = 'h' },
+	{ .name = "lnum",    .has_arg = 1, .flag = NULL, .val = 'l' },
+	{ .name = "info",    .has_arg = 0, .flag = NULL, .val = 'i' },
+	{ .name = "version", .has_arg = 0, .flag = NULL, .val = 'V' },
+	{ NULL, 0, NULL, 0}
+};
+
+static int parse_opt(int argc, char * const argv[])
+{
+	while (1) {
+		int option_index, key, error = 0;
+
+		key = getopt_long(argc, argv, "h?il:", long_options, &option_index);
+		if (key == -1)
+			break;
+
+		switch (key) {
+		case 0:
+			switch (option_index) {
+			case 0:
+				args.ubi = 1;
+				break;
+			case 1:
+				args.ubifs = 1;
+				break;
+			}
+			break;
+		case 'l':
+			args.lnum = simple_strtoul(optarg, &error);
+			if (error || args.lnum < 0)
+				return errmsg("bad lnum: \"%s\"", optarg);
+			break;
+
+		case 'i':
+			args.info = 1;
+			break;
+
+		case 'h':
+		case '?':
+			printf("%s\n\n", doc);
+			printf("%s\n\n", usage);
+			printf("%s\n", optionsstr);
+			exit(EXIT_SUCCESS);
+
+		case 'V':
+			common_print_version();
+			exit(EXIT_SUCCESS);
+
+		case ':':
+			return errmsg("parameter is missing");
+
+		default:
+			fprintf(stderr, "Use -h for help\n");
+			return -1;
+		}
+	}
+	if (optind == argc)
+		return errmsg("UBI device name was not specified (use -h for help)");
+	else if (optind != argc - 1)
+		return errmsg("more then one UBI device specified (use -h for help)");
+
+	args.vol = argv[optind];
+
+	if (args.lnum < 0)
+		return errmsg("lnum was not specified (use -h for help)");
+	if (args.ubi && (args.info || args.ubifs))
+		return errmsg("cannot specify --ubi with -i or --ubifs");
+
+	return 0;
+}
+
+static int leb_get_pnum(int *pnum)
+{
+	int fd, err;
+
+	fd = open(args.vol, O_RDONLY);
+	if (fd == -1) {
+		sys_errmsg("cannot open UBI volume \"%s\"", args.vol);
+		return -1;
+	}
+
+	*pnum = -1;
+	err = ubi_lnum_to_pnum(fd, args.lnum, pnum);
+	if (err)
+		sys_errmsg("ubi_lnum_to_pnum() failed, err = %d", err);
+
+	close(fd);
+	return err;
+}
+
+static int dump_eraseblock(int mtd_num, int pnum)
+{
+	int err, fd, peb_size, ret;
+	int ubi = !args.ubifs;
+	int ubifs = !args.ubi;
+	off_t offs;
+	libmtd_t libmtd;
+	struct mtd_info mtd_info;
+	struct mtd_dev_info mtd;
+	char file[strlen(MTD_DEV_PATT) + 100];
+	char *buf;
+
+	libmtd = libmtd_open();
+	if (!libmtd)
+		return errmsg("MTD subsystem is not present");
+
+	err = mtd_get_info(libmtd, &mtd_info);
+	if (err) {
+		if (errno == ENODEV)
+			errmsg("MTD is not present");
+		sys_errmsg("cannot get MTD information");
+		goto out;
+	}
+
+	err = mtd_get_dev_info1(libmtd, mtd_num, &mtd);
+	if (err) {
+		sys_errmsg("cannot get information about mtd%d\n", mtd_num);
+		goto out;
+	}
+
+	peb_size = mtd.eb_size;
+	buf = malloc(peb_size);
+	if (!buf) {
+		sys_errmsg("cannot alloc %d bytes memoroy", peb_size);
+		err = -ENOMEM;
+		goto out;
+	}
+
+	sprintf(file, MTD_DEV_PATT, mtd_num);
+	fd = open(file, O_RDONLY);
+	if (fd == -1) {
+		sys_errmsg("cannot open MTD device %s, mtd_num = %d", file, mtd_num);
+		goto out_free;
+	}
+
+	offs = pnum * peb_size;
+	ret = pread(fd, buf, peb_size, offs);
+	if (ret != peb_size) {
+		errmsg("cannot read %d bytes at pnum %d, read %d", peb_size,
+		       pnum, ret);
+		err = -EIO;
+		goto out_fd;
+	}
+
+	printf("leb %d is mapped to peb %d, start to dump:\n", args.lnum, pnum);
+	ubidump(buf, peb_size, ubi, ubifs, args.info);
+
+out_fd:
+	close(fd);
+out_free:
+	free(buf);
+out:
+	libmtd_close(libmtd);
+	return err;
+}
+
+int main(int argc, char * const argv[])
+{
+	int err, pnum, mtd_num;
+	libubi_t libubi;
+	struct ubi_vol_info vol_info;
+	struct ubi_dev_info dev_info;
+
+	err = parse_opt(argc, argv);
+	if (err)
+		return -1;
+
+	libubi = libubi_open();
+	if (!libubi) {
+		if (errno == 0)
+			errmsg("UBI is not present in the system");
+		else
+			sys_errmsg("cannot open libubi");
+		goto out_libubi;
+	}
+
+	err = ubi_probe_node(libubi, args.vol);
+	if (err == 1) {
+		errmsg("\"%s\" is an UBI device node, not an UBI volume node",
+		       args.vol);
+		goto out_libubi;
+	} else if (err < 0) {
+		if (errno == ENODEV)
+			errmsg("\"%s\" is not an UBI volume node", args.vol);
+		else
+			sys_errmsg("error while probing \"%s\"", args.vol);
+		goto out_libubi;
+	}
+
+	err = ubi_get_vol_info(libubi, args.vol, &vol_info);
+	if (err) {
+		sys_errmsg("cannot get information about UBI volume \"%s\"",
+			   args.vol);
+		goto out_libubi;
+	}
+
+	err = leb_get_pnum(&pnum);
+	if (err) {
+		sys_errmsg("cannot get pnum of leb %d", args.lnum);
+		goto out_libubi;
+	} else if (pnum == -1) {
+		errmsg("leb %d is not mapped", args.lnum);
+		goto out_libubi;
+	}
+
+	err = ubi_get_dev_info1(libubi, vol_info.dev_num, &dev_info);
+	if (err) {
+		sys_errmsg("cannot get information about UBI%d", vol_info.dev_num);
+		goto out_libubi;
+	}
+	mtd_num = dev_info.mtd_num;
+
+	err = dump_eraseblock(mtd_num, pnum);
+	if (err)
+		goto out_libubi;
+
+	libubi_close(libubi);
+	return 0;
+
+out_libubi:
+	libubi_close(libubi);
+	return -1;
+}
Signature · 1 line
1.8.1.4
author
#19 lore raw

PATCH 4/5 ubi-utils: Add ubifs-media.h

In reply to hujianyang

Add ubifs-media.h to dump ubifs-level stuff.

---
 include/mtd/ubifs-media.h | 295 ++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 295 insertions(+)
 create mode 100644 include/mtd/ubifs-media.h
include/mtd/ubifs-media.h new file +295 -0
new file mode 100644
index 0000000..e1767fb
@@ -0,0 +1,295 @@
+/*
+ * Copyright (c) International Business Machines Corp., 2006
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See
+ * the GNU General Public License for more details.
+ *
+ * Author: Hu Jianyang <[email protected]>
+ *
+ * UBIFS library.
+ */
+
+/*
+ * This file defines the layout of UBI headers and all the other UBIiFS
+ * on-flash data structures.
+ */
+
+#ifndef __UBIFS_MEDIA_H__
+#define __UBIFS_MEDIA_H__
+
+#include <asm/byteorder.h>
+
+/* UBIFS node magic number (must not have the padding byte first or last) */
+#define UBIFS_NODE_MAGIC  0x06101831
+
+/* UBIFS padding byte pattern (must not be first or last byte of node magic) */
+#define UBIFS_PADDING_BYTE 0xCE
+
+/* Node sizes (N.B. these are guaranteed to be multiples of 8) */
+#define UBIFS_CH_SZ        sizeof(struct ubifs_ch)
+#define UBIFS_PAD_NODE_SZ  sizeof(struct ubifs_pad_node)
+
+/*
+ * UBIFS node types.
+ *
+ * UBIFS_INO_NODE: inode node
+ * UBIFS_DATA_NODE: data node
+ * UBIFS_DENT_NODE: directory entry node
+ * UBIFS_XENT_NODE: extended attribute node
+ * UBIFS_TRUN_NODE: truncation node
+ * UBIFS_PAD_NODE: padding node
+ * UBIFS_SB_NODE: superblock node
+ * UBIFS_MST_NODE: master node
+ * UBIFS_REF_NODE: LEB reference node
+ * UBIFS_IDX_NODE: index node
+ * UBIFS_CS_NODE: commit start node
+ * UBIFS_ORPH_NODE: orphan node
+ * UBIFS_NODE_TYPES_CNT: count of supported node types
+ *
+ * Note, we index arrays by these numbers, so keep them low and contiguous.
+ * Node type constants for inodes, direntries and so on have to be the same as
+ * corresponding key type constants.
+ */
+enum {
+	UBIFS_INO_NODE,
+	UBIFS_DATA_NODE,
+	UBIFS_DENT_NODE,
+	UBIFS_XENT_NODE,
+	UBIFS_TRUN_NODE,
+	UBIFS_PAD_NODE,
+	UBIFS_SB_NODE,
+	UBIFS_MST_NODE,
+	UBIFS_REF_NODE,
+	UBIFS_IDX_NODE,
+	UBIFS_CS_NODE,
+	UBIFS_ORPH_NODE,
+	UBIFS_NODE_TYPES_CNT,
+};
+
+/*
+ * Supported key hash functions.
+ *
+ * UBIFS_KEY_HASH_R5: R5 hash
+ * UBIFS_KEY_HASH_TEST: test hash which just returns first 4 bytes of the name
+ */
+enum {
+	UBIFS_KEY_HASH_R5,
+	UBIFS_KEY_HASH_TEST,
+};
+
+/*
+ * Supported key formats.
+ *
+ * UBIFS_SIMPLE_KEY_FMT: simple key format
+ */
+enum {
+	UBIFS_SIMPLE_KEY_FMT,
+};
+
+/*
+ * Node group type (used by recovery to recover whole group or none).
+ *
+ * UBIFS_NO_NODE_GROUP: this node is not part of a group
+ * UBIFS_IN_NODE_GROUP: this node is a part of a group
+ * UBIFS_LAST_OF_NODE_GROUP: this node is the last in a group
+ */
+enum {
+	UBIFS_NO_NODE_GROUP = 0,
+	UBIFS_IN_NODE_GROUP,
+	UBIFS_LAST_OF_NODE_GROUP,
+};
+
+/*
+ * Superblock flags.
+ *
+ * UBIFS_FLG_BIGLPT: if "big" LPT model is used if set
+ * UBIFS_FLG_SPACE_FIXUP: first-mount "fixup" of free space within LEBs needed
+ */
+enum {
+	UBIFS_FLG_BIGLPT = 0x02,
+	UBIFS_FLG_SPACE_FIXUP = 0x04,
+};
+
+/**
+ * struct ubifs_ch - common header node.
+ * @magic: UBIFS node magic number (%UBIFS_NODE_MAGIC)
+ * @crc: CRC-32 checksum of the node header
+ * @sqnum: sequence number
+ * @len: full node length
+ * @node_type: node type
+ * @group_type: node group type
+ * @padding: reserved for future, zeroes
+ *
+ * Every UBIFS node starts with this common part. If the node has a key, the
+ * key always goes next.
+ */
+struct ubifs_ch {
+	__le32 magic;
+	__le32 crc;
+	__le64 sqnum;
+	__le32 len;
+	__u8 node_type;
+	__u8 group_type;
+	__u8 padding[2];
+} __attribute__ ((packed));
+
+/**
+ * struct ubifs_pad_node - padding node.
+ * @ch: common header
+ * @pad_len: how many bytes after this node are unused (because padded)
+ * @padding: reserved for future, zeroes
+ */
+struct ubifs_pad_node {
+	struct ubifs_ch ch;
+	__le32 pad_len;
+} __attribute__ ((packed));
+
+/**
+ * struct ubifs_sb_node - superblock node.
+ * @ch: common header
+ * @padding: reserved for future, zeroes
+ * @key_hash: type of hash function used in keys
+ * @key_fmt: format of the key
+ * @flags: file-system flags (%UBIFS_FLG_BIGLPT, etc)
+ * @min_io_size: minimal input/output unit size
+ * @leb_size: logical eraseblock size in bytes
+ * @leb_cnt: count of LEBs used by file-system
+ * @max_leb_cnt: maximum count of LEBs used by file-system
+ * @max_bud_bytes: maximum amount of data stored in buds
+ * @log_lebs: log size in logical eraseblocks
+ * @lpt_lebs: number of LEBs used for lprops table
+ * @orph_lebs: number of LEBs used for recording orphans
+ * @jhead_cnt: count of journal heads
+ * @fanout: tree fanout (max. number of links per indexing node)
+ * @lsave_cnt: number of LEB numbers in LPT's save table
+ * @fmt_version: UBIFS on-flash format version
+ * @default_compr: default compression algorithm (%UBIFS_COMPR_LZO, etc)
+ * @padding1: reserved for future, zeroes
+ * @rp_uid: reserve pool UID
+ * @rp_gid: reserve pool GID
+ * @rp_size: size of the reserved pool in bytes
+ * @padding2: reserved for future, zeroes
+ * @time_gran: time granularity in nanoseconds
+ * @uuid: UUID generated when the file system image was created
+ * @ro_compat_version: UBIFS R/O compatibility version
+ */
+struct ubifs_sb_node {
+	struct ubifs_ch ch;
+	__u8 padding[2];
+	__u8 key_hash;
+	__u8 key_fmt;
+	__le32 flags;
+	__le32 min_io_size;
+	__le32 leb_size;
+	__le32 leb_cnt;
+	__le32 max_leb_cnt;
+	__le64 max_bud_bytes;
+	__le32 log_lebs;
+	__le32 lpt_lebs;
+	__le32 orph_lebs;
+	__le32 jhead_cnt;
+	__le32 fanout;
+	__le32 lsave_cnt;
+	__le32 fmt_version;
+	__le16 default_compr;
+	__u8 padding1[2];
+	__le32 rp_uid;
+	__le32 rp_gid;
+	__le64 rp_size;
+	__le32 time_gran;
+	__u8 uuid[16];
+	__le32 ro_compat_version;
+	__u8 padding2[3968];
+} __attribute__ ((packed));
+
+/**
+ * struct ubifs_mst_node - master node.
+ * @ch: common header
+ * @highest_inum: highest inode number in the committed index
+ * @cmt_no: commit number
+ * @flags: various flags (%UBIFS_MST_DIRTY, etc)
+ * @log_lnum: start of the log
+ * @root_lnum: LEB number of the root indexing node
+ * @root_offs: offset within @root_lnum
+ * @root_len: root indexing node length
+ * @gc_lnum: LEB reserved for garbage collection (%-1 value means the LEB was
+ * not reserved and should be reserved on mount)
+ * @ihead_lnum: LEB number of index head
+ * @ihead_offs: offset of index head
+ * @index_size: size of index on flash
+ * @total_free: total free space in bytes
+ * @total_dirty: total dirty space in bytes
+ * @total_used: total used space in bytes (includes only data LEBs)
+ * @total_dead: total dead space in bytes (includes only data LEBs)
+ * @total_dark: total dark space in bytes (includes only data LEBs)
+ * @lpt_lnum: LEB number of LPT root nnode
+ * @lpt_offs: offset of LPT root nnode
+ * @nhead_lnum: LEB number of LPT head
+ * @nhead_offs: offset of LPT head
+ * @ltab_lnum: LEB number of LPT's own lprops table
+ * @ltab_offs: offset of LPT's own lprops table
+ * @lsave_lnum: LEB number of LPT's save table (big model only)
+ * @lsave_offs: offset of LPT's save table (big model only)
+ * @lscan_lnum: LEB number of last LPT scan
+ * @empty_lebs: number of empty logical eraseblocks
+ * @idx_lebs: number of indexing logical eraseblocks
+ * @leb_cnt: count of LEBs used by file-system
+ * @padding: reserved for future, zeroes
+ */
+struct ubifs_mst_node {
+	struct ubifs_ch ch;
+	__le64 highest_inum;
+	__le64 cmt_no;
+	__le32 flags;
+	__le32 log_lnum;
+	__le32 root_lnum;
+	__le32 root_offs;
+	__le32 root_len;
+	__le32 gc_lnum;
+	__le32 ihead_lnum;
+	__le32 ihead_offs;
+	__le64 index_size;
+	__le64 total_free;
+	__le64 total_dirty;
+	__le64 total_used;
+	__le64 total_dead;
+	__le64 total_dark;
+	__le32 lpt_lnum;
+	__le32 lpt_offs;
+	__le32 nhead_lnum;
+	__le32 nhead_offs;
+	__le32 ltab_lnum;
+	__le32 ltab_offs;
+	__le32 lsave_lnum;
+	__le32 lsave_offs;
+	__le32 lscan_lnum;
+	__le32 empty_lebs;
+	__le32 idx_lebs;
+	__le32 leb_cnt;
+	__u8 padding[344];
+} __attribute__ ((packed));
+
+/**
+ * struct ubifs_ref_node - logical eraseblock reference node.
+ * @ch: common header
+ * @lnum: the referred logical eraseblock number
+ * @offs: start offset in the referred LEB
+ * @jhead: journal head number
+ * @padding: reserved for future, zeroes
+ */
+struct ubifs_ref_node {
+	struct ubifs_ch ch;
+	__le32 lnum;
+	__le32 offs;
+	__le32 jhead;
+	__u8 padding[28];
+} __attribute__ ((packed));
+
+#endif /*!__UBIFS_MEDIA_H__ */
Signature · 1 line
1.8.1.4
author
#20 lore raw

PATCH 5/5 ubi-utils: Modified Makefile

In reply to hujianyang

Compile support.

---
 Makefile | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)
Makefile +5 -2
index eade234..9dc9645 100644
@@ -28,7 +28,8 @@ MTD_BINS = \
 	sumtool jffs2reader
 UBI_BINS = \
 	ubiupdatevol ubimkvol ubirmvol ubicrc32 ubinfo ubiattach \
-	ubidetach ubinize ubiformat ubirename mtdinfo ubirsvol ubiblock
+	ubidetach ubinize ubiformat ubirename mtdinfo ubirsvol ubiblock \
+	ubidump
 
 BINS = $(MTD_BINS)
 BINS += mkfs.ubifs/mkfs.ubifs
@@ -115,10 +116,12 @@ obj-libiniparser.a = libiniparser.o dictionary.o
 obj-libscan.a      = libscan.o
 obj-libubi.a       = libubi.o
 obj-libubigen.a    = libubigen.o
+obj-libdump.a      = libdump.o
 
 obj-mtdinfo   = libubigen.a
 obj-ubinize   = libubigen.a libiniparser.a
 obj-ubiformat = libubigen.a libscan.a
+obj-ubidump   = libdump.a
 
-$(foreach v,libubi.a libubigen.a libiniparser.a libscan.a,$(eval $(call _mkdep,ubi-utils/,$(v))))
+$(foreach v,libubi.a libubigen.a libiniparser.a libscan.a libdump.a,$(eval $(call _mkdep,ubi-utils/,$(v))))
 $(foreach v,$(UBI_BINS),$(eval $(call mkdep,ubi-utils/,$(v),libubi.a ubiutils-common.o)))
Signature · 1 line
1.8.1.4