ubi-utils: Add a new utility ubidump
Subsystems (3)
RFC PATCH 1/1 ubi-utils: Add a new utility ubidump
Hi all,
I would like to show my work about a new utility named ubidump. This utility is different from jffs2dump. It can dump a specific leb to user space(maybe a arrange of lebs in the future).
We have two logic layers in UBIFS and data on the eraseblocks is separated into two parts: UBI header which include EC header and VID header, UBIFS filesystem data which is formatted as different kinds of NODEs.
We don't have a direct way to get both UBI info and UBIFS info in the past(really?) because these two layers are transparent to each other. Furthermore, when debugging with UBIFS, I found some key information is invisible and I need to modify the kernel to get them. For example, when I was dealing with commit 642f63e (UBIFS: fix fatal race condition), I can't find a way to dump the whole LOG area.
I think dynamic debug is a helpful mechanism but in some embedded environment, kernel messages are stored in a ring buffer and are wrapped commonly.
So I want to make a new utility to get data on eraseblocks in user space and analysis filesystem behaviors in an easy way. This utility can dump both UBI info and UBIFS info and all the prints are formatted as UBI or UBIFS style.
I just want to make it a lighter utility but I have to copy lots of ubifs-media codes and debug codes to mtd-utils. UBI headers are got from a new ioctl(is there other way?) and UBIFS NODEs are got from a simply read(), then print them by functions copied from UBI/UBIFS debug.c in kernel.
Usage: ubidump <UBI volume node file name> [-l <lnum>] [-i]
[--help] [--version]
Example 1: ubidump /dev/ubi0_1 -n 2 - dump leb 2 in volume 1 Example 2: ubidump /dev/ubi0_0 -n 1234 -i - dump leb 1234 with explicit info
-h, --help print help message
-n, --lnum logic eraseblock num to dump
-i, --info show explicit information about NODEs
-H, --header show only header information
-N, --node show only NODEs information
-V, --version print program version
# ./ubidump /dev/ubi0_0 -n 0 -i lnum 0 is mapped to pnum 2054 Erase counter header dump: magic 0x55424923 version 1 ec 1 vid_hdr_offset 256 data_offset 512 image_seq 1256176231 hdr_crc 0x7758fc8b Volume identifier header dump: magic 55424921 version 1 vol_type 1 copy_flag 0 compat 0 vol_id 0 lnum 0 data_size 0 used_ebs 0 data_pad 0 sqnum 8 hdr_crc 29b2783a scan LEB 0:0 look at LEB 0:0 (15872 bytes left) scanning superblock node at LEB 0:0 magic 0x6101831 crc 0x88b345d6 node_type 6 (superblock node) group_type 0 (no node group) sqnum 1 len 4096 key_hash 0 (R5) key_fmt 0 (simple) flags 0x2 big_lpt 1 space_fixup 0 min_io_size 512 leb_size 15872 leb_cnt 8028 max_leb_cnt 8028 max_bud_bytes 5920256 log_lebs 28 lpt_lebs 11 orph_lebs 2 jhead_cnt 1 fanout 8 lsave_cnt 256 default_compr 1 rp_size 5242880 rp_uid 0 rp_gid 0 fmt_version 4 time_gran 1000000000 UUID 0x105248cUB look at LEB 0:4096 (11776 bytes left) hit empty space at LEB 0:4096 stop scanning LEB 0 at offset 15872
PATCH 1/7 UBI: Add a new ioctl to support ubidump
Add a new ioctl to dump EC header and VID header to user space. This ioctl is called without lock so I think users *should not* run ubidump when volume is mounted.
Struct ubi_ebdump_req is used to transfer data between user space and kernel space. Can I find a way to set buffer length from '64' to UBI_VID_HDR_SIZE and UBI_EC_HDR_SIZE?
--- drivers/mtd/ubi/cdev.c | 50 +++++++++++++++++++++++++++++++++++++++++++++ include/uapi/mtd/ubi-user.h | 15 ++++++++++++++ 2 files changed, 65 insertions(+)
drivers/mtd/ubi/cdev.c +50 -0
@@ -581,6 +581,56 @@ static long vol_cdev_ioctl(struct file *file, unsigned int cmd,break ; } +case UBI_IOCEBDUMP : + { +int pnum; +struct ubi_ebdump_req req; +struct ubi_ec_hdr *ec_hdr; +struct ubi_vid_hdr *vid_hdr; + + err =copy_from_user (&req, argp, +sizeof (struct ubi_ebdump_req )); +if (err) { + err = -EFAULT ; +break ; + } + + err =ubi_is_mapped (desc, req.lnum); +if (err <=0 ) +break ; + pnum = vol->eba_tbl[req.lnum]; + req.pnum = pnum; + +/* +* No need to check return value of ubi_io_read(), +* we will confirm the date in user space. +*/ + ec_hdr =kzalloc (ubi->ec_hdr_alsize,GFP_NOFS ); +if (!ec_hdr) { + err = -ENOMEM ; +break ; + } +ubi_io_read_ec_hdr (ubi, pnum, ec_hdr,0 ); +memcpy (req.ec_hdr, ec_hdr,UBI_EC_HDR_SIZE ); +kfree (ec_hdr); + + vid_hdr =ubi_zalloc_vid_hdr (ubi,GFP_NOFS ); +if (!vid_hdr) { + err = -ENOMEM ; +break ; + } +ubi_io_read_vid_hdr (ubi, pnum, vid_hdr,1 ); +memcpy (req.vid_hdr, vid_hdr,UBI_VID_HDR_SIZE ); +ubi_free_vid_hdr (ubi, vid_hdr); + + err =copy_to_user (argp, &req, +sizeof (struct ubi_ebdump_req )); +if (err) + err = -EFAULT ; + +break ; + } +default : err = -ENOTTY ;break ;
include/uapi/mtd/ubi-user.h +15 -0
@@ -205,6 +205,8 @@#define UBI_IOCVOLCRBLK _IOW(UBI_VOL_IOC_MAGIC, 7, struct ubi_blkcreate_req)/* Remove the R/O block device */ #define UBI_IOCVOLRMBLK _IO(UBI_VOL_IOC_MAGIC, 8) +/* Dump LEB header */ +#define UBI_IOCEBDUMP _IOW(UBI_VOL_IOC_MAGIC, 9, struct ubi_ebdump_req)/* Maximum MTD device name length supported by UBI */ #define MAX_UBI_MTD_NAME_LEN 127
@@ -442,4 +444,17 @@ struct ubi_blkcreate_req { __s8 padding[128]; } __packed; +/** +* struct ubi_ebdump_req - a data structure used in dump eraseblock header. +* @lnum: logical eraseblock num to dump +* @ec_hdr: ec_hdr to set +* @vid_hdr: vid_hdr to set +*/ +struct ubi_ebdump_req { +__s32 lnum; +__s32 pnum; +char ec_hdr[64 ]; +char vid_hdr[64 ]; +} __packed; +#endif /* __UBI_USER_H__ */
Signature · 1 line
1.8.1.4
Re: [PATCH 1/7] UBI: Add a new ioctl to support ubidump
Quoted 10 lines · hujianyang
Add a new ioctl to dump EC header and VID header to user space. This ioctl is called without lock so I think users *should not* run ubidump when volume is mounted.
Struct
ubi_ebdump_reqis used to transfer data between user space and kernel space. Can I find a way to set buffer length from '64' toUBI_VID_HDR_SIZEandUBI_EC_HDR_SIZE?Signed-off-by hujianyang <[email protected]>
I've never seen an ioctl for retrieving an on-the-media data structure, do you have any examples? I think the traditional way is that the user-space reads all the information from the disk itself.
Do you really need to dump UBI headers?
Could you please drop this functionality for now, and provide an ubidump which only dumps UBIFS data structures for now.
Then we can talk and think about the UBI-level headers dumping.
Signature · 2 lines
Best Regards, Artem Bityutskiy
Re: [PATCH 1/7] UBI: Add a new ioctl to support ubidump
Hi Artem,
Thank you for reviewing these patches.
Quoted 3 lines
Do you really need to dump UBI headers?
Yes, because I think ec_counter in ec_hdr and copy_flag, sqnum in vid_hdr are useful.
Current ->read() can just read data from data_offset where the leb located. And reading UBI headers by mtd functionality are very complicated, we don't know pnum either.
Maybe we can try other way instead of this ioctl, by adding an new function which can read the hole peb specified by lnum to user space.
Re: [PATCH 1/7] UBI: Add a new ioctl to support ubidump
Quoted 11 lines · hujianyang
Hi Artem,
Thank you for reviewing these patches.
Quoted 3 lines
Do you really need to dump UBI headers?
Yes, because I think
ec_counterinec_hdrandcopy_flag, sqnum invid_hdrare useful.
Quoted 7 lines
Current ->read() can just read data from
data_offsetwhere the leb located. And reading UBI headers by mtd functionality are very complicated, we don't know pnum either.Maybe we can try other way instead of this ioctl, by adding an new function which can read the hole peb specified by lnum to user space.
Yes, this is a lot better approach. But I'd still suggest to start with a smaller step - ubidump without the UBI headers dumping support. Then add the UBI headers dumping support as a separate step.
Signature · 2 lines
Best Regards, Artem Bityutskiy
PATCH 2/7 Add new ioctl in userspace
In reply to hujianyangAdd same ioctl in user space.
--- include/mtd/ubi-user.h | 15 +++++++++++++++ ubi-utils/libubi.c | 8 ++++++++ 2 files changed, 23 insertions(+)
include/mtd/ubi-user.h +15 -0
@@ -200,6 +200,8 @@#define UBI_IOCVOLCRBLK _IOW(UBI_VOL_IOC_MAGIC, 7, struct ubi_blkcreate_req)/* Remove the R/O block device */ #define UBI_IOCVOLRMBLK _IO(UBI_VOL_IOC_MAGIC, 8) +/* Dump LEB header */ +#define UBI_IOCEBDUMP _IOW(UBI_VOL_IOC_MAGIC, 9, struct ubi_ebdump_req)/* Maximum MTD device name length supported by UBI */ #define MAX_UBI_MTD_NAME_LEN 127
@@ -437,4 +439,17 @@ struct ubi_blkcreate_req {int8_t padding[128 ]; } __attribute__((packed)); +/** +* struct ubi_ebdump_req - a data structure used in dump eraseblock header. +* @lnum: logical eraseblock num to dump +* @ec_hdr: ec_hdr to set +* @vid_hdr: vid_hdr to set +*/ +struct ubi_ebdump_req { +int32_t lnum; +int32_t pnum; +char ec_hdr[64 ]; +char vid_hdr[64 ]; +} __attribute__((packed)); +#endif /* __UBI_USER_H__ */
ubi-utils/libubi.c +8 -0
@@ -1123,6 +1123,14 @@ int ubi_vol_block_remove(int fd)return ioctl (fd,UBI_IOCVOLRMBLK ); } +int ubi_leb_dump_header (libubi_t desc,int fd,struct ubi_ebdump_req *request) +{ + desc = desc; +if (ioctl (fd,UBI_IOCEBDUMP , request)) +return -1 ; +return 0 ; +} +int ubi_update_start (libubi_t desc,int fd,long long bytes) { desc = desc;
Signature · 1 line
1.8.1.4
PATCH 3/7 Add ubifs-media.h
In reply to hujianyangCopy from fs/ubifs/ubifs-media.h but not the whole file, just what I need(SB_NODE, MST_NODE, LOG_NODE, CH_NODE, PAD_NODE, REF_NODE).
Could I add "Auther: Artem Bityutskiy" because these files are based on his work.
--- include/mtd/ubifs-media.h | 295 ++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 295 insertions(+) create mode 100644 include/mtd/ubifs-media.h
include/mtd/ubifs-media.h new file +295 -0
@@ -0,0 +1,295 @@ +/* +* Copyright (c) International Business Machines Corp., 2006 +* +* This program is free software; you can redistribute it and/or modify +* it under the terms of the GNU General Public License as published by +* the Free Software Foundation; either version 2 of the License, or +* (at your option) any later version. +* +* This program is distributed in the hope that it will be useful, +* but WITHOUT ANY WARRANTY; without even the implied warranty of +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See +* the GNU General Public License for more details. +* +* Author: Hu Jianyang <[email protected]> +* +* UBIFS library. +*/ + +/* +* This file defines the layout of UBI headers and all the other UBIiFS +* on-flash data structures. +*/ + +#ifndef __UBIFS_MEDIA_H__ +#define __UBIFS_MEDIA_H__ + +#include <asm/byteorder.h> + +/* UBIFS node magic number (must not have the padding byte first or last) */ +#define UBIFS_NODE_MAGIC 0x06101831 + +/* UBIFS padding byte pattern (must not be first or last byte of node magic) */ +#define UBIFS_PADDING_BYTE 0xCE + +/* Node sizes (N.B. these are guaranteed to be multiples of 8) */ +#define UBIFS_CH_SZ sizeof(struct ubifs_ch) +#define UBIFS_PAD_NODE_SZ sizeof(struct ubifs_pad_node) + +/* +* UBIFS node types. +* +* UBIFS_INO_NODE: inode node +* UBIFS_DATA_NODE: data node +* UBIFS_DENT_NODE: directory entry node +* UBIFS_XENT_NODE: extended attribute node +* UBIFS_TRUN_NODE: truncation node +* UBIFS_PAD_NODE: padding node +* UBIFS_SB_NODE: superblock node +* UBIFS_MST_NODE: master node +* UBIFS_REF_NODE: LEB reference node +* UBIFS_IDX_NODE: index node +* UBIFS_CS_NODE: commit start node +* UBIFS_ORPH_NODE: orphan node +* UBIFS_NODE_TYPES_CNT: count of supported node types +* +* Note, we index arrays by these numbers, so keep them low and contiguous. +* Node type constants for inodes, direntries and so on have to be the same as +* corresponding key type constants. +*/ +enum { +UBIFS_INO_NODE , +UBIFS_DATA_NODE , +UBIFS_DENT_NODE , +UBIFS_XENT_NODE , +UBIFS_TRUN_NODE , +UBIFS_PAD_NODE , +UBIFS_SB_NODE , +UBIFS_MST_NODE , +UBIFS_REF_NODE , +UBIFS_IDX_NODE , +UBIFS_CS_NODE , +UBIFS_ORPH_NODE , +UBIFS_NODE_TYPES_CNT , +}; + +/* +* Supported key hash functions. +* +* UBIFS_KEY_HASH_R5: R5 hash +* UBIFS_KEY_HASH_TEST: test hash which just returns first 4 bytes of the name +*/ +enum { +UBIFS_KEY_HASH_R5 , +UBIFS_KEY_HASH_TEST , +}; + +/* +* Supported key formats. +* +* UBIFS_SIMPLE_KEY_FMT: simple key format +*/ +enum { +UBIFS_SIMPLE_KEY_FMT , +}; + +/* +* Node group type (used by recovery to recover whole group or none). +* +* UBIFS_NO_NODE_GROUP: this node is not part of a group +* UBIFS_IN_NODE_GROUP: this node is a part of a group +* UBIFS_LAST_OF_NODE_GROUP: this node is the last in a group +*/ +enum { +UBIFS_NO_NODE_GROUP =0 , +UBIFS_IN_NODE_GROUP , +UBIFS_LAST_OF_NODE_GROUP , +}; + +/* +* Superblock flags. +* +* UBIFS_FLG_BIGLPT: if "big" LPT model is used if set +* UBIFS_FLG_SPACE_FIXUP: first-mount "fixup" of free space within LEBs needed +*/ +enum { +UBIFS_FLG_BIGLPT =0x02 , +UBIFS_FLG_SPACE_FIXUP =0x04 , +}; + +/** +* struct ubifs_ch - common header node. +* @magic: UBIFS node magic number (%UBIFS_NODE_MAGIC) +* @crc: CRC-32 checksum of the node header +* @sqnum: sequence number +* @len: full node length +* @node_type: node type +* @group_type: node group type +* @padding: reserved for future, zeroes +* +* Every UBIFS node starts with this common part. If the node has a key, the +* key always goes next. +*/ +struct ubifs_ch { +__le32 magic; +__le32 crc; +__le64 sqnum; +__le32 len; +__u8 node_type; +__u8 group_type; +__u8 padding[2 ]; +} __attribute__ ((packed)); + +/** +* struct ubifs_pad_node - padding node. +* @ch: common header +* @pad_len: how many bytes after this node are unused (because padded) +* @padding: reserved for future, zeroes +*/ +struct ubifs_pad_node { +struct ubifs_ch ch; +__le32 pad_len; +} __attribute__ ((packed)); + +/** +* struct ubifs_sb_node - superblock node. +* @ch: common header +* @padding: reserved for future, zeroes +* @key_hash: type of hash function used in keys +* @key_fmt: format of the key +* @flags: file-system flags (%UBIFS_FLG_BIGLPT, etc) +* @min_io_size: minimal input/output unit size +* @leb_size: logical eraseblock size in bytes +* @leb_cnt: count of LEBs used by file-system +* @max_leb_cnt: maximum count of LEBs used by file-system +* @max_bud_bytes: maximum amount of data stored in buds +* @log_lebs: log size in logical eraseblocks +* @lpt_lebs: number of LEBs used for lprops table +* @orph_lebs: number of LEBs used for recording orphans +* @jhead_cnt: count of journal heads +* @fanout: tree fanout (max. number of links per indexing node) +* @lsave_cnt: number of LEB numbers in LPT's save table +* @fmt_version: UBIFS on-flash format version +* @default_compr: default compression algorithm (%UBIFS_COMPR_LZO, etc) +* @padding1: reserved for future, zeroes +* @rp_uid: reserve pool UID +* @rp_gid: reserve pool GID +* @rp_size: size of the reserved pool in bytes +* @padding2: reserved for future, zeroes +* @time_gran: time granularity in nanoseconds +* @uuid: UUID generated when the file system image was created +* @ro_compat_version: UBIFS R/O compatibility version +*/ +struct ubifs_sb_node { +struct ubifs_ch ch; +__u8 padding[2 ]; +__u8 key_hash; +__u8 key_fmt; +__le32 flags; +__le32 min_io_size; +__le32 leb_size; +__le32 leb_cnt; +__le32 max_leb_cnt; +__le64 max_bud_bytes; +__le32 log_lebs; +__le32 lpt_lebs; +__le32 orph_lebs; +__le32 jhead_cnt; +__le32 fanout; +__le32 lsave_cnt; +__le32 fmt_version; +__le16 default_compr; +__u8 padding1[2 ]; +__le32 rp_uid; +__le32 rp_gid; +__le64 rp_size; +__le32 time_gran; +__u8 uuid[16 ]; +__le32 ro_compat_version; +__u8 padding2[3968 ]; +} __attribute__ ((packed)); + +/** +* struct ubifs_mst_node - master node. +* @ch: common header +* @highest_inum: highest inode number in the committed index +* @cmt_no: commit number +* @flags: various flags (%UBIFS_MST_DIRTY, etc) +* @log_lnum: start of the log +* @root_lnum: LEB number of the root indexing node +* @root_offs: offset within @root_lnum +* @root_len: root indexing node length +* @gc_lnum: LEB reserved for garbage collection (%-1 value means the LEB was +* not reserved and should be reserved on mount) +* @ihead_lnum: LEB number of index head +* @ihead_offs: offset of index head +* @index_size: size of index on flash +* @total_free: total free space in bytes +* @total_dirty: total dirty space in bytes +* @total_used: total used space in bytes (includes only data LEBs) +* @total_dead: total dead space in bytes (includes only data LEBs) +* @total_dark: total dark space in bytes (includes only data LEBs) +* @lpt_lnum: LEB number of LPT root nnode +* @lpt_offs: offset of LPT root nnode +* @nhead_lnum: LEB number of LPT head +* @nhead_offs: offset of LPT head +* @ltab_lnum: LEB number of LPT's own lprops table +* @ltab_offs: offset of LPT's own lprops table +* @lsave_lnum: LEB number of LPT's save table (big model only) +* @lsave_offs: offset of LPT's save table (big model only) +* @lscan_lnum: LEB number of last LPT scan +* @empty_lebs: number of empty logical eraseblocks +* @idx_lebs: number of indexing logical eraseblocks +* @leb_cnt: count of LEBs used by file-system +* @padding: reserved for future, zeroes +*/ +struct ubifs_mst_node { +struct ubifs_ch ch; +__le64 highest_inum; +__le64 cmt_no; +__le32 flags; +__le32 log_lnum; +__le32 root_lnum; +__le32 root_offs; +__le32 root_len; +__le32 gc_lnum; +__le32 ihead_lnum; +__le32 ihead_offs; +__le64 index_size; +__le64 total_free; +__le64 total_dirty; +__le64 total_used; +__le64 total_dead; +__le64 total_dark; +__le32 lpt_lnum; +__le32 lpt_offs; +__le32 nhead_lnum; +__le32 nhead_offs; +__le32 ltab_lnum; +__le32 ltab_offs; +__le32 lsave_lnum; +__le32 lsave_offs; +__le32 lscan_lnum; +__le32 empty_lebs; +__le32 idx_lebs; +__le32 leb_cnt; +__u8 padding[344 ]; +} __attribute__ ((packed)); + +/** +* struct ubifs_ref_node - logical eraseblock reference node. +* @ch: common header +* @lnum: the referred logical eraseblock number +* @offs: start offset in the referred LEB +* @jhead: journal head number +* @padding: reserved for future, zeroes +*/ +struct ubifs_ref_node { +struct ubifs_ch ch; +__le32 lnum; +__le32 offs; +__le32 jhead; +__u8 padding[28 ]; +} __attribute__ ((packed)); + +#endif /*!__UBIFS_MEDIA_H__ */
Signature · 1 line
1.8.1.4
PATCH 4/7 Add libubifs.h
In reply to hujianyangHeader file of libubifs.c.
Could I have other ways to get 'ALIGN' and 'min_t' instead of define them in this file?
--- ubi-utils/include/libubifs.h | 87 ++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 87 insertions(+) create mode 100644 ubi-utils/include/libubifs.h
ubi-utils/include/libubifs.h new file +87 -0
@@ -0,0 +1,87 @@ +/* +* Copyright (c) International Business Machines Corp., 2006 +* +* This program is free software; you can redistribute it and/or modify +* it under the terms of the GNU General Public License as published by +* the Free Software Foundation; either version 2 of the License, or +* (at your option) any later version. +* +* This program is distributed in the hope that it will be useful, +* but WITHOUT ANY WARRANTY; without even the implied warranty of +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See +* the GNU General Public License for more details. +* +* Author: Hu Jianyang <[email protected]> +* +* UBIFS library. +*/ + +#ifndef __LIBUBIFS_H__ +#define __LIBUBIFS_H__ + +#include <mtd/ubifs-media.h> + +#ifdef __cplusplus +extern "C" { +#endif + +#define ALIGN (x,a) __ALIGN_MASK(x,(typeof(x))(a)-1) +#define __ALIGN_MASK (x,mask) (((x)+(mask))&~(mask)) + +#define min_t (t,x,y) ({ \ + typeof((x)) _x = (x); \ + typeof((y)) _y = (y); \ + (_x < _y) ? _x : _y; \ +}) + +/* +* 'ubifs_scan_a_node()' return values. +* +* SCANNED_GARBAGE: scanned garbage +* SCANNED_EMPTY_SPACE: scanned empty space +* SCANNED_A_NODE: scanned a valid node +* SCANNED_A_CORRUPT_NODE: scanned a corrupted node +* SCANNED_A_BAD_PAD_NODE: scanned a padding node with invalid pad length +* +* Greater than zero means: 'scanned that number of padding bytes' +*/ +enum { +SCANNED_GARBAGE =0 , +SCANNED_EMPTY_SPACE =-1 , +SCANNED_A_NODE =-2 , +SCANNED_A_CORRUPT_NODE =-3 , +SCANNED_A_BAD_PAD_NODE =-4 , +}; + +void ubifs_dump_node (const void *node); + +/** +* ubifs_scan_a_node - scan for a node or padding. +* @buf: buffer to scan +* @size: logical eraseblock size +* @len: length of buffer +* @lnum: logical eraseblock number +* @offs: offset within the logical eraseblock +* +* This function returns a scanning code to indicate what was scanned. +*/ +int ubifs_scan_a_node (void *buf,int leb_size,int len,int lnum,int offs); + +/** +* ubifs_scan - scan a logical eraseblock. +* @lnum: logical eraseblock number +* @sbuf: scan buffer +* @size: size of @buf in bytes +* @offs: offset to start at (usually zero) +* @detailed: print NODEs detailed info +* +* This function scans LEB and prints complete information about +* its contents. +*/ +void ubifs_scan (int lnum,void *buf,int leb_size,int offs,int detailed); + +#ifdef __cplusplus +} +#endif + +#endif /*!__LIBUBIFS_H__ */
Signature · 1 line
1.8.1.4
PATCH 5/7 Add libubifs.c
In reply to hujianyangThese functions are copied from fs/ubifs/scan.c and fs/ubifs/debug.c with some small changes. We don't have 'ubifs_info *c' in user space and we don't need to check NODEs, just print what they are.
I think if data is corrupted, kernel will warn it and we can't check it here because the checking codes are to heavy.
case SCANNED_A_NODE:
We will get the whole detailed info about this node if we set '-i'.
I just dump detailed info with some kinds of NODEs because I think others are useless for debugging and you can simply add what you need if it is necessary.
--- ubi-utils/libubifs.c | 399 +++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 399 insertions(+) create mode 100644 ubi-utils/libubifs.c
ubi-utils/libubifs.c new file +399 -0
@@ -0,0 +1,399 @@ +/* +* Copyright (c) International Business Machines Corp., 2006 +* +* This program is free software; you can redistribute it and/or modify +* it under the terms of the GNU General Public License as published by +* the Free Software Foundation; either version 2 of the License, or +* (at your option) any later version. +* +* This program is distributed in the hope that it will be useful, +* but WITHOUT ANY WARRANTY; without even the implied warranty of +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See +* the GNU General Public License for more details. +* +* Author: Hu Jianyang <[email protected]> +* +* UBIFS library. +*/ + +#define PROGRAM_NAME "libubifs" + +#include <stdlib.h> +#include <stdio.h> +#include <libubifs.h> +#include <mtd_swab.h> +#include "common.h" + +static const char *get_key_fmt (int fmt) +{ +switch (fmt) { +case UBIFS_SIMPLE_KEY_FMT : +return "simple" ; +default : +return "unknown/invalid format" ; + } +} + +static const char *get_key_hash (int hash) +{ +switch (hash) { +case UBIFS_KEY_HASH_R5 : +return "R5" ; +case UBIFS_KEY_HASH_TEST : +return "test" ; +default : +return "unknown/invalid name hash" ; + } +} + +static const char *node_ntype (int type) +{ +switch (type) { +case UBIFS_PAD_NODE : +return "padding node" ; +case UBIFS_SB_NODE : +return "superblock node" ; +case UBIFS_MST_NODE : +return "master node" ; +case UBIFS_REF_NODE : +return "reference node" ; +case UBIFS_INO_NODE : +return "inode node" ; +case UBIFS_DENT_NODE : +return "direntry node" ; +case UBIFS_XENT_NODE : +return "xentry node" ; +case UBIFS_DATA_NODE : +return "data node" ; +case UBIFS_TRUN_NODE : +return "truncate node" ; +case UBIFS_IDX_NODE : +return "indexing node" ; +case UBIFS_CS_NODE : +return "commit start node" ; +case UBIFS_ORPH_NODE : +return "orphan node" ; +default : +return "unknown node" ; + } +} + +static const char *node_gtype (int type) +{ +switch (type) { +case UBIFS_NO_NODE_GROUP : +return "no node group" ; +case UBIFS_IN_NODE_GROUP : +return "in node group" ; +case UBIFS_LAST_OF_NODE_GROUP : +return "last of node group" ; +default : +return "unknown" ; + } +} + +static void dump_ch (const struct ubifs_ch *ch) +{ +printf ("\tmagic %#x\n" ,le32_to_cpu (ch->magic)); +printf ("\tcrc %#x\n" ,le32_to_cpu (ch->crc)); +printf ("\tnode_type %d (%s)\n" , ch->node_type, +node_ntype (ch->node_type)); +printf ("\tgroup_type %d (%s)\n" , ch->group_type, +node_gtype (ch->group_type)); +printf ("\tsqnum %llu\n" , + (unsigned long long )le64_to_cpu (ch->sqnum)); +printf ("\tlen %u\n" ,le32_to_cpu (ch->len)); +} + +void ubifs_dump_node (const void *node) +{ +const struct ubifs_ch *ch = node; + +if (le32_to_cpu (ch->magic) !=UBIFS_NODE_MAGIC ) { +printf ("Not a node, first %zu bytes:\n" ,UBIFS_CH_SZ ); +return ; + } + +dump_ch (node); + +switch (ch->node_type) { +case UBIFS_PAD_NODE : + { +const struct ubifs_pad_node *pad = node; + +printf ("\tpad_len %u\n" ,le32_to_cpu (pad->pad_len)); +break ; + } +case UBIFS_SB_NODE : + { +const struct ubifs_sb_node *sup = node; +unsigned int sup_flags =le32_to_cpu (sup->flags); + +printf ("\tkey_hash %d (%s)\n" , + (int )sup->key_hash,get_key_hash (sup->key_hash)); +printf ("\tkey_fmt %d (%s)\n" , + (int )sup->key_fmt,get_key_fmt (sup->key_fmt)); +printf ("\tflags %#x\n" , sup_flags); +printf ("\tbig_lpt %u\n" , + !!(sup_flags &UBIFS_FLG_BIGLPT )); +printf ("\tspace_fixup %u\n" , + !!(sup_flags &UBIFS_FLG_SPACE_FIXUP )); +printf ("\tmin_io_size %u\n" ,le32_to_cpu (sup->min_io_size)); +printf ("\tleb_size %u\n" ,le32_to_cpu (sup->leb_size)); +printf ("\tleb_cnt %u\n" ,le32_to_cpu (sup->leb_cnt)); +printf ("\tmax_leb_cnt %u\n" ,le32_to_cpu (sup->max_leb_cnt)); +printf ("\tmax_bud_bytes %llu\n" , + (unsigned long long )le64_to_cpu (sup->max_bud_bytes)); +printf ("\tlog_lebs %u\n" ,le32_to_cpu (sup->log_lebs)); +printf ("\tlpt_lebs %u\n" ,le32_to_cpu (sup->lpt_lebs)); +printf ("\torph_lebs %u\n" ,le32_to_cpu (sup->orph_lebs)); +printf ("\tjhead_cnt %u\n" ,le32_to_cpu (sup->jhead_cnt)); +printf ("\tfanout %u\n" ,le32_to_cpu (sup->fanout)); +printf ("\tlsave_cnt %u\n" ,le32_to_cpu (sup->lsave_cnt)); +printf ("\tdefault_compr %u\n" , + (int )le16_to_cpu (sup->default_compr)); +printf ("\trp_size %llu\n" , + (unsigned long long )le64_to_cpu (sup->rp_size)); +printf ("\trp_uid %u\n" ,le32_to_cpu (sup->rp_uid)); +printf ("\trp_gid %u\n" ,le32_to_cpu (sup->rp_gid)); +printf ("\tfmt_version %u\n" ,le32_to_cpu (sup->fmt_version)); +printf ("\ttime_gran %u\n" ,le32_to_cpu (sup->time_gran)); +printf ("\tUUID %pUB\n" , sup->uuid); +break ; + } +case UBIFS_MST_NODE : + { +const struct ubifs_mst_node *mst = node; + +printf ("\thighest_inum %llu\n" , + (unsigned long long )le64_to_cpu (mst->highest_inum)); +printf ("\tcommit number %llu\n" , + (unsigned long long )le64_to_cpu (mst->cmt_no)); +printf ("\tflags %#x\n" ,le32_to_cpu (mst->flags)); +printf ("\tlog_lnum %u\n" ,le32_to_cpu (mst->log_lnum)); +printf ("\troot_lnum %u\n" ,le32_to_cpu (mst->root_lnum)); +printf ("\troot_offs %u\n" ,le32_to_cpu (mst->root_offs)); +printf ("\troot_len %u\n" ,le32_to_cpu (mst->root_len)); +printf ("\tgc_lnum %u\n" ,le32_to_cpu (mst->gc_lnum)); +printf ("\tihead_lnum %u\n" ,le32_to_cpu (mst->ihead_lnum)); +printf ("\tihead_offs %u\n" ,le32_to_cpu (mst->ihead_offs)); +printf ("\tindex_size %llu\n" , + (unsigned long long )le64_to_cpu (mst->index_size)); +printf ("\tlpt_lnum %u\n" ,le32_to_cpu (mst->lpt_lnum)); +printf ("\tlpt_offs %u\n" ,le32_to_cpu (mst->lpt_offs)); +printf ("\tnhead_lnum %u\n" ,le32_to_cpu (mst->nhead_lnum)); +printf ("\tnhead_offs %u\n" ,le32_to_cpu (mst->nhead_offs)); +printf ("\tltab_lnum %u\n" ,le32_to_cpu (mst->ltab_lnum)); +printf ("\tltab_offs %u\n" ,le32_to_cpu (mst->ltab_offs)); +printf ("\tlsave_lnum %u\n" ,le32_to_cpu (mst->lsave_lnum)); +printf ("\tlsave_offs %u\n" ,le32_to_cpu (mst->lsave_offs)); +printf ("\tlscan_lnum %u\n" ,le32_to_cpu (mst->lscan_lnum)); +printf ("\tleb_cnt %u\n" ,le32_to_cpu (mst->leb_cnt)); +printf ("\tempty_lebs %u\n" ,le32_to_cpu (mst->empty_lebs)); +printf ("\tidx_lebs %u\n" ,le32_to_cpu (mst->idx_lebs)); +printf ("\ttotal_free %llu\n" , + (unsigned long long )le64_to_cpu (mst->total_free)); +printf ("\ttotal_dirty %llu\n" , + (unsigned long long )le64_to_cpu (mst->total_dirty)); +printf ("\ttotal_used %llu\n" , + (unsigned long long )le64_to_cpu (mst->total_used)); +printf ("\ttotal_dead %llu\n" , + (unsigned long long )le64_to_cpu (mst->total_dead)); +printf ("\ttotal_dark %llu\n" , + (unsigned long long )le64_to_cpu (mst->total_dark)); +break ; + } +case UBIFS_REF_NODE : + { +const struct ubifs_ref_node *ref = node; + +printf ("\tlnum %u\n" ,le32_to_cpu (ref->lnum)); +printf ("\toffs %u\n" ,le32_to_cpu (ref->offs)); +printf ("\tjhead %u\n" ,le32_to_cpu (ref->jhead)); +break ; + } +case UBIFS_CS_NODE : +break ; +case UBIFS_INO_NODE : +case UBIFS_DENT_NODE : +case UBIFS_XENT_NODE : +case UBIFS_DATA_NODE : +case UBIFS_TRUN_NODE : +case UBIFS_IDX_NODE : +case UBIFS_ORPH_NODE : +printf ("cannot dump node, type not support\n" ); +default : +printf ("node type %d was not recognized\n" , + (int )ch->node_type); + } +} + +/** +* scan_padding_bytes - scan for padding bytes. +* @buf: buffer to scan +* @len: length of buffer +* +* This function returns the number of padding bytes on success and +* %SCANNED_GARBAGE on failure. +*/ +static int scan_padding_bytes (void *buf,int len) +{ +int pad_len =0 , max_pad_len =min_t (int,UBIFS_PAD_NODE_SZ , len); +uint8_t *p = buf; + +printf ("not a node\n" ); + +while (pad_len < max_pad_len && *p++ ==UBIFS_PADDING_BYTE ) + pad_len +=1 ; + +if (!pad_len || (pad_len &7 )) +return SCANNED_GARBAGE ; + +printf ("%d padding bytes\n" , pad_len); + +return pad_len; +} + +/** +* ubifs_scan_a_node - scan for a node or padding. +* @buf: buffer to scan +* @size: logical eraseblock size +* @len: length of buffer +* @lnum: logical eraseblock number +* @offs: offset within the logical eraseblock +* +* This function returns a scanning code to indicate what was scanned. +*/ +int ubifs_scan_a_node (void *buf,int leb_size,int len,int lnum,int offs) +{ +struct ubifs_ch *ch = buf; +uint32_t magic; + + magic =le32_to_cpu (ch->magic); + +if (magic ==0xFFFFFFFF ) { +printf ("hit empty space at LEB %d:%d\n" , lnum, offs); +return SCANNED_EMPTY_SPACE ; + } + +if (magic !=UBIFS_NODE_MAGIC ) +return scan_padding_bytes (buf, len); + +if (len <UBIFS_CH_SZ ) +return SCANNED_GARBAGE ; + +printf ("scanning %s at LEB %d:%d\n" , +node_ntype (ch->node_type), lnum, offs); + +/* No ubifs_check_nodei() perform here */ + +if (ch->node_type ==UBIFS_PAD_NODE ) { +struct ubifs_pad_node *pad = buf; +int pad_len =le32_to_cpu (pad->pad_len); +int node_len =le32_to_cpu (ch->len); + +/* Validate the padding node */ +if (pad_len <0 || + offs + node_len + pad_len > leb_size) { +printf ("bad pad node at LEB %d:%d\n" , lnum, offs); +ubifs_dump_node (pad); +return SCANNED_A_BAD_PAD_NODE ; + } + +/* Make the node pads to 8-byte boundary */ +if ((node_len + pad_len) &7 ) { +printf ("bad padding length %d - %d\n" , + offs, offs + node_len + pad_len); +return SCANNED_A_BAD_PAD_NODE ; + } + +printf ("%d bytes padded at LEB %d:%d, offset now %d\n" , pad_len, + lnum, offs,ALIGN (offs + node_len + pad_len,8 )); + +return node_len + pad_len; + } + +return SCANNED_A_NODE ; +} + +/** +* ubifs_scan - scan a logical eraseblock. +* @lnum: logical eraseblock number +* @sbuf: scan buffer +* @size: size of @buf in bytes +* @offs: offset to start at (usually zero) +* @detailed: print NODEs detailed info +* +* This function scans LEB and prints complete information about +* its contents. +*/ +void ubifs_scan (int lnum,void *sbuf,int leb_size,int offs,int detailed) +{ +void *buf = sbuf + offs; +int err, len = leb_size - offs; + +printf ("scan LEB %d:%d\n" , lnum, offs); + +while (len >=8 ) { +struct ubifs_ch *ch = buf; +int node_len, ret; + +printf ("look at LEB %d:%d (%d bytes left)\n" , + lnum, offs, len); + + ret =ubifs_scan_a_node (buf, leb_size, len, lnum, offs); +if (ret >0 ) { +/* Padding bytes or a valid padding node */ + offs += ret; + buf += ret; + len -= ret; +continue ; + } + +if (ret ==SCANNED_EMPTY_SPACE ) +/* Empty space is checked later */ +break ; + +switch (ret) { +case SCANNED_GARBAGE : +printf ("garbage\n" ); + goto corrupted; +case SCANNED_A_NODE : +if (detailed) +ubifs_dump_node (buf); +break ; +case SCANNED_A_CORRUPT_NODE : +case SCANNED_A_BAD_PAD_NODE : +printf ("bad node\n" ); + goto corrupted; +default : +printf ("unknown\n" ); + err = -EINVAL ; + goto error; + } + + node_len =ALIGN (le32_to_cpu (ch->len),8 ); + offs += node_len; + buf += node_len; + len -= node_len; + } + +for (; len >4 ; offs +=4 , buf = buf +4 , len -=4 ) +if (*(uint32_t *)buf !=0xffffffff ) +break ; +for (; len; offs++, buf++, len--) +if (*(uint8_t *)buf !=0xff ) { +printf ("corrupt empty space at LEB %d:%d\n" , + lnum, offs); + goto corrupted; + } + +printf ("stop scanning LEB %d at offset %d\n" , lnum, offs); +return ; + +corrupted: +printf ("corruption at LEB %d:%d\n" , lnum, offs); + err = -EUCLEAN ; +error: +printf ("LEB %d scanning failed, error %d\n" , lnum, err); +}
Signature · 1 line
1.8.1.4
PATCH 6/7 New utility ubidump
In reply to hujianyangThis utility is referred to ubiupdatevol.
I think I should move 'dump_header()' in this file to other place but I don't know where.
This utility don't support dump binary date because I didn't find a useful helper function like 'print_dump_hex()' in kernel and kernel may dump the data which is corrupted so I think no need to dump them here.
--- ubi-utils/ubidump.c | 287 ++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 287 insertions(+) create mode 100644 ubi-utils/ubidump.c
ubi-utils/ubidump.c new file +287 -0
@@ -0,0 +1,287 @@ +/* +* This program is free software; you can redistribute it and/or modify +* it under the terms of the GNU General Public License as published by +* the Free Software Foundation; either version 2 of the License, or +* (at your option) any later version. +* +* This program is distributed in the hope that it will be useful, +* but WITHOUT ANY WARRANTY; without even the implied warranty of +* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See +* the GNU General Public License for more details. +*/ + +/* +* An utility to dump UBI/UBIFS format data in eraseblock +* +* Author: Hu Jianyang <[email protected]> +*/ + +#define PROGRAM_NAME "ubidump" + +#include <stdlib.h> +#include <getopt.h> + +#include <libubi.h> +#include <libubifs.h> +#include <libmtd.h> +#include <mtd_swab.h> +#include "common.h" +#include "ubiutils-common.h" + +/* The variables below are set by command line arguments */ +struct args { +const char *vol; +int lnum; +int info:1 ; +int header:1 ; +int node:1 ; +}; + +static struct args args = +{ + .vol =NULL , + .lnum =-1 , + .info =0 , + .header =0 , + .node =0 , +}; + +static const char doc[] =PROGRAM_NAME " version " VERSION +" - an utility to dump UBI/UBIFS format data in eraseblock" ; + +static const char optionsstr[] = +"-h, --help print help message\n" +"-n, --lnum logic eraseblock num to dump\n" +"-i, --info show explicit information about NODEs\n" +"-H, --header show only header information\n" +"-N, --node show only NODEs information\n" +"-V, --version print program version" ; + +static const char usage[] = +"Usage: " PROGRAM_NAME " <UBI volume node file name> [-l <lnum>] [-i]\n" +"\t\t\t[--help] [--version]\n\n" +"Example 1: " PROGRAM_NAME " /dev/ubi0_1 -n 2 - dump leb 2 in volume 1\n" +"Example 2: " PROGRAM_NAME " /dev/ubi0_0 -n 1234 -i - dump leb 1234 with explicit info\n" ; + +static const struct option long_options[] = { + { .name ="help" , .has_arg =0 , .flag =NULL , .val ='h' }, + { .name ="lnum" , .has_arg =1 , .flag =NULL , .val ='n' }, + { .name ="info" , .has_arg =0 , .flag =NULL , .val ='i' }, + { .name ="header" , .has_arg =0 , .flag =NULL , .val ='H' }, + { .name ="node" , .has_arg =0 , .flag =NULL , .val ='N' }, + { .name ="version" , .has_arg =0 , .flag =NULL , .val ='s' }, + {NULL ,0 ,NULL ,0 } +}; + +static int parse_opt (int argc,char *const argv[]) +{ +while (1 ) { +int key, error =0 ; + + key =getopt_long (argc, argv,"h?in:HN" , long_options,NULL ); +if (key ==-1 ) +break ; + +switch (key) { +case 'n' : + args.lnum =simple_strtoul (optarg, &error); +if (error || args.lnum <0 ) +return errmsg ("bad lnum: \"%s\"" , optarg); +break ; + +case 'i' : + args.info =1 ; +break ; + +case 'H' : + args.header =1 ; +break ; + +case 'N' : + args.node =1 ; +break ; + +case 'h' : +case '?' : +printf ("%s\n\n" , doc); +printf ("%s\n\n" , usage); +printf ("%s\n" , optionsstr); +exit (EXIT_SUCCESS ); + +case 'V' : +common_print_version (); +exit (EXIT_SUCCESS ); + +case ':' : +return errmsg ("parameter is missing" ); + +default : +fprintf (stderr,"Use -h for help\n" ); +return -1 ; + } + } +if (optind == argc) +return errmsg ("UBI device name was not specified (use -h for help)" ); +else if (optind != argc -1 ) +return errmsg ("more then one UBI device specified (use -h for help)" ); + + args.vol = argv[optind]; + +if (args.lnum <0 ) +return errmsg ("lnum was not specified (use -h for help)" ); +if (args.header && (args.info || args.node)) +return errmsg ("cannot specify -H with -i or -N" ); + +return 0 ; +} + +static void dump_header (struct ubi_ebdump_req *header) +{ +struct ubi_ec_hdr *ec_hdr = (struct ubi_ec_hdr *)header->ec_hdr; +struct ubi_vid_hdr *vid_hdr = (struct ubi_vid_hdr *)header->vid_hdr; + +printf ("lnum %d is mapped to pnum %d\n" , header->lnum, header->pnum); + +printf ("Erase counter header dump:\n" ); +printf ("\tmagic %#08x\n" ,be32_to_cpu (ec_hdr->magic)); +printf ("\tversion %d\n" , (int )ec_hdr->version); +printf ("\tec %llu\n" , (long long )be64_to_cpu (ec_hdr->ec)); +printf ("\tvid_hdr_offset %d\n" ,be32_to_cpu (ec_hdr->vid_hdr_offset)); +printf ("\tdata_offset %d\n" ,be32_to_cpu (ec_hdr->data_offset)); +printf ("\timage_seq %d\n" ,be32_to_cpu (ec_hdr->image_seq)); +printf ("\thdr_crc %#08x\n" ,be32_to_cpu (ec_hdr->hdr_crc)); + +printf ("Volume identifier header dump:\n" ); +printf ("\tmagic %08x\n" ,be32_to_cpu (vid_hdr->magic)); +printf ("\tversion %d\n" , (int )vid_hdr->version); +printf ("\tvol_type %d\n" , (int )vid_hdr->vol_type); +printf ("\tcopy_flag %d\n" , (int )vid_hdr->copy_flag); +printf ("\tcompat %d\n" , (int )vid_hdr->compat); +printf ("\tvol_id %d\n" ,be32_to_cpu (vid_hdr->vol_id)); +printf ("\tlnum %d\n" ,be32_to_cpu (vid_hdr->lnum)); +printf ("\tdata_size %d\n" ,be32_to_cpu (vid_hdr->data_size)); +printf ("\tused_ebs %d\n" ,be32_to_cpu (vid_hdr->used_ebs)); +printf ("\tdata_pad %d\n" ,be32_to_cpu (vid_hdr->data_pad)); +printf ("\tsqnum %llu\n" , + (unsigned long long )be64_to_cpu (vid_hdr->sqnum)); +printf ("\thdr_crc %08x\n" ,be32_to_cpu (vid_hdr->hdr_crc)); +} + +static int dump_leb (libubi_t libubi,int fd,struct ubi_vol_info *vol_info) +{ +int ret, leb_size, lnum; +char *buf; + + leb_size = vol_info->leb_size; + lnum = args.lnum; + + ret =ubi_is_mapped (fd, lnum); +if (ret ==0 ) { +errmsg ("lnum %d is not mapped" , lnum); + goto out; + }else if (ret <0 ) { +sys_errmsg ("ubi_is_mapped() failed" ); + goto out; + } + +/* dump peb header */ +if (!args.node) { +struct ubi_ebdump_req request; + +memset (&request,0 ,sizeof (struct ubi_ebdump_req )); + request.lnum = lnum; + request.pnum =-1 ; + ret =ubi_leb_dump_header (libubi, fd, &request); +if (ret <0 ) { +sys_errmsg ("cannot dump header of lnum %d" , lnum); + goto out; + }else if (request.pnum ==-1 ) { +errmsg ("lnum %d is not mapped" , lnum); + goto out; + } + +dump_header (&request); + } + +/* dump NODEs */ +if (!args.header) { +off_t offs = lnum * leb_size; + + buf =malloc (leb_size); +if (!buf) +return errmsg ("cannot allocate %d bytes of memory" , + leb_size); + + ret =pread (fd, buf, leb_size, offs); +if (ret != leb_size) { +errmsg ("cannot read %d bytes at lnum %d, read %d" , + leb_size, lnum, ret); + goto out_free; + } +ubifs_scan (lnum, buf, leb_size,0 , args.info); + } + +out_free: +free (buf); +out: +return ret; +} + +int main (int argc,char *const argv[]) +{ +int err, fd; +libubi_t libubi; +struct ubi_vol_info vol_info; + + err =parse_opt (argc, argv); +if (err) +return -1 ; + + libubi =libubi_open (); +if (!libubi) { +if (errno ==0 ) +errmsg ("UBI is not present in the system" ); +else +sys_errmsg ("cannot open libubi" ); + goto out_libubi; + } + + err =ubi_probe_node (libubi, args.vol); +if (err ==1 ) { +errmsg ("\"%s\" is an UBI device node, not an UBI volume node" , + args.vol); + goto out_libubi; + }else if (err <0 ) { +if (errno ==ENODEV ) +errmsg ("\"%s\" is not an UBI volume node" , args.vol); +else +sys_errmsg ("error while probing \"%s\"" , args.vol); + goto out_libubi; + } + + err =ubi_get_vol_info (libubi, args.vol, &vol_info); +if (err) { +sys_errmsg ("cannot get information about UBI volume \"%s\"" , + args.vol); + goto out_libubi; + } + + fd =open (args.vol,O_RDONLY ); +if (fd ==-1 ) { +sys_errmsg ("cannot open UBI volume \"%s\"" , args.vol); + goto out_libubi; + } + err =dump_leb (libubi, fd, &vol_info); +if (err) + goto out_fd; + +close (fd); +libubi_close (libubi); +return 0 ; + +out_fd: +close (fd); +out_libubi: +libubi_close (libubi); +return -1 ; +}
Signature · 1 line
1.8.1.4
Re: [PATCH 6/7] New utility ubidump
Quoted 2 lines · hujianyang
+"-h, --help print help message\n"
+"-n, --lnum logic eraseblock num to dump\n"
I envision that in the future people may want to specify PEB number instead of LEB number, and '-n' becomes a confusing name for the option. Could you please only support '--lnum', and drop '-n' altogether. Then in the future someone may add '--pnum' support.
Quoted 1 line
+"-i, --info show explicit information about NODEs\n"
I do not understand what this means from the description. Would you please try to describe it better in the help text?
Quoted 2 lines
+"-H, --header show only header information\n"
+"-N, --node show only NODEs information\n"
Same for these.
Thanks!
Signature · 2 lines
Best Regards, Artem Bityutskiy
Re: [PATCH 6/7] New utility ubidump
Quoted 4 lines
I envision that in the future people may want to specify PEB number instead of LEB number, and '-n' becomes a confusing name for the option. Could you please only support '--lnum', and drop '-n' altogether. Then in the future someone may add '--pnum' support.
Yes, you are right. I'll change it in next version.
Quoted 6 lines
Quoted 2 lines
+"-i, --info show explicit information about NODEs\n"
I do not understand what this means from the description. Would you please try to describe it better in the help text?
Quoted 7 lines
Quoted 2 lines
+"-H, --header show only header information\n"
+"-N, --node show only NODEs information\n"Same for these.
Thanks!
-H for UBI-level info and -N for UBIFS-level info.
-i means dump UBIFS Nodes by function ubifs_dump_node, not just scan them.
I will re-describe them.
Thanks.
Re: [PATCH 6/7] New utility ubidump
Quoted 25 lines · hujianyang
Quoted 4 lines
I envision that in the future people may want to specify PEB number instead of LEB number, and '-n' becomes a confusing name for the option. Could you please only support '--lnum', and drop '-n' altogether. Then in the future someone may add '--pnum' support.
Yes, you are right. I'll change it in next version.
Quoted 13 lines
Quoted 2 lines
+"-i, --info show explicit information about NODEs\n"
I do not understand what this means from the description. Would you please try to describe it better in the help text?
Quoted 2 lines
+"-H, --header show only header information\n"
+"-N, --node show only NODEs information\n"Same for these.
Thanks!
(quoted diff) +0
-2-H for UBI-level info and -N for UBIFS-level info. -i means dump UBIFS Nodes by function ubifs_dump_node, not justscan them.
Quoted 1 line
I will re-describe them.
Things like UBI volume table, UBI fast-map stuff are not "headers", so I am not sure if using word "headers" a good idea. Probably we want options like --ubifs and --ubi to denote ubi and ubifs-level stuff. The default would be "everything".
But again, if you start smaller, and upstream a good tool for UBIFS-level stuff, it will be easier to add UBI stuff separately.
Besides, I have some additional vision, which you do not have to implement, but which should be taken into account. E.g., ubidump which does not need UBI/UBIFS drivers, ubidump which can deal with an image generated with nanddump without "mounting" it, etc. So I was thinking doing small steps at a time would make it easier for me and for you to make a tool which has limited functionality today, but which can be later extended to support more functionality.
Signature · 2 lines
Best Regards, Artem Bityutskiy
Re: [PATCH 6/7] New utility ubidump
Quoted 9 lines
Things like UBI volume table, UBI fast-map stuff are not "headers", so I am not sure if using word "headers" a good idea. Probably we want options like --ubifs and --ubi to denote ubi and ubifs-level stuff. The default would be "everything".
But again, if you start smaller, and upstream a good tool for UBIFS-level stuff, it will be easier to add UBI stuff separately.
Quoted 8 lines
Besides, I have some additional vision, which you do not have to implement, but which should be taken into account. E.g., ubidump which does not need UBI/UBIFS drivers, ubidump which can deal with an image generated with nanddump without "mounting" it, etc. So I was thinking doing small steps at a time would make it easier for me and for you to make a tool which has limited functionality today, but which can be later extended to support more functionality.
It seems you what more than me. I think you are right. I need to reconsider how to realize these above, not hole of them, but a good architecture that can be extended easily.
So I think getting data from mtd driver is a good choice and then run it with an image file. Current UBI functionality has lots of limit and it is basing on UBI driver. But we need to do more work in user space (rebuilding volume table and so on) in this way. I think it's worth.
Give me some time to re-create this utility. I will send it to you if I finished it. If you get some new ideas, please tell me as soon as possible.
Thanks~!
Hu
Re: [PATCH 6/7] New utility ubidump
Quoted 28 lines · hujianyang
Quoted 9 lines
Things like UBI volume table, UBI fast-map stuff are not "headers", so I am not sure if using word "headers" a good idea. Probably we want options like --ubifs and --ubi to denote ubi and ubifs-level stuff. The default would be "everything".
But again, if you start smaller, and upstream a good tool for UBIFS-level stuff, it will be easier to add UBI stuff separately.
Quoted 8 lines
Besides, I have some additional vision, which you do not have to implement, but which should be taken into account. E.g., ubidump which does not need UBI/UBIFS drivers, ubidump which can deal with an image generated with nanddump without "mounting" it, etc. So I was thinking doing small steps at a time would make it easier for me and for you to make a tool which has limited functionality today, but which can be later extended to support more functionality.
It seems you what more than me. I think you are right. I need to reconsider how to realize these above, not hole of them, but a good architecture that can be extended easily.
So I think getting data from mtd driver is a good choice and then run it with an image file. Current UBI functionality has lots of limit and it is basing on UBI driver. But we need to do more work in user space (rebuilding volume table and so on) in this way. I think it's worth.
Quoted 3 lines
Give me some time to re-create this utility. I will send it to you if I finished it. If you get some new ideas, please tell me as soon as possible.
I am not trying to make you do a lot more than you need.
If you are analyzing an MTD image without "mounting" it, the only way to find out the LEB->PEB mapping is to do full scan, and basically copy the UBI driver code. This is a lot of work.
In your case you have the driver, and it is fine to add an ioctl which provides LEB->PEB mapping. This is a lot simpler.
So I envision that the tool would work like this.
$ ubidump my.img --lnum 5 - dump LEB 5, will need to scan the entire image.
$ ubidump /dev/ubiX_Y --lnum 5 - will just ask the UBI driver to give the PEB number for LEB 5, then find out the MTD device for this volume (should be possible by checking the sysfs files), and then reads the MTD device, and gets the UBI-level information from there.
Something like this, just quick thoughts.
Signature · 2 lines
Best Regards, Artem Bityutskiy
Re: [PATCH 6/7] New utility ubidump
In reply to hujianyangQuoted 2 lines · hujianyang
If you get some new ideas, please tell me as soon
as possible.
The problem is that I do not know if I have more ideas. You may spend a lot of time on this stuff and then I send one of my ideas may render your work useless.
This is why I suggest to start small, do just the UBIFS nodes dump, nothing else. Just forget about UBI. Then let's review that and make part of mtd-utils. This will clarify the vision, and we may add UBI support then separately.
Signature · 2 lines
Best Regards, Artem Bityutskiy
Re: [PATCH 6/7] New utility ubidump
Quoted 6 lines
This is why I suggest to start small, do just the UBIFS nodes dump, nothing else. Just forget about UBI. Then let's review that and make part of mtd-utils. This will clarify the vision, and we may add UBI support then separately.
So just use simple way to do UBIFS dump and then step by step make it a strong utility.
It's not too hard. I will first realize an UBIFS dump as you showed by mtd driver.
Re: [PATCH 6/7] New utility ubidump
In reply to hujianyangQuoted 21 lines · hujianyang at huawei.com
Quoted 10 lines
But again, if you start smaller, and upstream a good tool for UBIFS-level stuff, it will be easier to add UBI stuff separately.
Besides, I have some additional vision, which you do not have to implement, but which should be taken into account. E.g., ubidump which does not need UBI/UBIFS drivers, ubidump which can deal with an image generated with nanddump without "mounting" it, etc. So I was thinking doing small steps at a time would make it easier for me and for you to make a tool which has limited functionality today, but which can be later extended to support more functionality.
It seems you what more than me. I think you are right. I need to reconsider how to realize these above, not hole of them, but a good architecture that can be extended easily.
So I think getting data from mtd driver is a good choice and then run it with an image file. Current UBI functionality has lots of limit and it is basing on UBI driver. But we need to do more work in user space (rebuilding volume table and so on) in this way. I think it's worth.
Quoted 3 lines
Give me some time to re-create this utility. I will send it to you if I finished it. If you get some new ideas, please tell me as soon as possible.
I started some thing like this. The source is attached. As really you only want 'read only' access, there is a minimal portion of the UBI/UbiFs code that is needed. I think that only the 'ubi-media.h' is needed. The Linux is high performance and handles read/write with different fault conditions.
If you write code for read-only/single thread I think it will be much more simple than the active UBI/UbiFS code in the Linux kernel. I did not look at the UbiFS layer as much. It is far more complex that UBI imho; of course, I just saw a little of the internal volumes and the fast map features. However, fast map itself should not be needed for this utility.
For your use case of analysis in a running system, I think that reads of '/dev/mtd0ro', etc can be used. This way recovery features can also be developed if we identified some inconsistency; Ie, it is not required to 'mount' the volume before analysis.
The attached source just scans an file image and create an leb/peb mapping. I only used the 'ubi-media.h' as documentation.
Fwiw,
Bill Pringlemeir.
/* -*- mode: c; compile-command: "gcc -Wall -g -o parse_ubi parse_ubi.c"; -*- */
#include <stdio.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <unistd.h>
#include <endian.h>
#include <string.h>
#include <stdlib.h>
#include <ctype.h>
#define __packed __attribute__((packed))
#include "ubi-media.h"
#define bswap16 be16toh
#define bswap32 be32toh
#define bswap64 be64toh
static int dump_vid = 0;
#define CRCPOLY_LE 0xedb88320
static unsigned int crc32(unsigned int crc, void const *_p, size_t len)
{
unsigned char const *p = _p;
int i;
while (len--) {
crc ^= *p++;
for (i = 0; i < 8; i++)
crc = (crc >> 1) ^ ((crc & 1) ? CRCPOLY_LE : 0);
}
return crc;
}#define ALEN(a) (sizeof(a)/sizeof(a[0]))
static void print_ec(struct ubi_ec_hdr *ec)
{
if(ec->version != UBI_VERSION || ec->magic != UBI_EC_HDR_MAGIC) {
printf(" Magic: %x\n", ec->magic);
printf(" Version: %d\n", (int)ec->version);
printf(" EC: %llx\n", ec->ec);
printf(" VID offset: %x\n", ec->vid_hdr_offset);
printf(" Data offset: %x\n", ec->data_offset);
printf(" Image seq: %x\n", ec->image_seq);
exit(-1);
}
}static void read_ec(int fd, struct ubi_ec_hdr *ec)
{
int rval = read(fd, ec,sizeof(*ec));
if(rval == sizeof(*ec)) {
unsigned int crc;
crc = crc32(UBI_CRC32_INIT, ec, UBI_EC_HDR_SIZE_CRC);
ec->magic = bswap32(ec->magic);
ec->vid_hdr_offset = bswap32(ec->vid_hdr_offset);
ec->data_offset = bswap32(ec->data_offset);
ec->image_seq = bswap32(ec->image_seq);
ec->hdr_crc = bswap32(ec->hdr_crc);
ec->ec = bswap64(ec->ec);
if(crc != ec->hdr_crc)
printf("EC CRC: %x/%x\n", crc, ec->hdr_crc);
} else
memset(ec, 0, sizeof(*ec));
}static void print_vid(int vid_num, struct ubi_vid_hdr *vid)
{
if(vid->magic != UBI_VID_HDR_MAGIC)
printf(" Magic: %x\n", vid->magic);
if(vid->version != UBI_VERSION)
printf(" Version: %d\n", (int)vid->version);
if(!dump_vid) return;
printf("VID %d\n", vid_num);
/* This is usually the same. */
if(vid->vol_id >= UBI_INTERNAL_VOL_START)
printf("Internal vol_id: %d\n", vid->vol_id - UBI_INTERNAL_VOL_START);
if(vid->vol_type != UBI_VID_DYNAMIC)
printf(" vol_type: %s\n",
vid->vol_type == UBI_VID_DYNAMIC ? "dynamic" : "static");
if(vid->used_ebs)
printf(" used_ebs: %d\n", vid->used_ebs);
if(vid->data_pad)
printf(" data_pad: %d\n", vid->data_pad);
if((vid->copy_flag != 1 && vid->data_size) ||
(vid->copy_flag == 0 && vid->data_size))
printf(" copy_flag: %d\n", (int)vid->copy_flag);
printf(" lnum: %d\n", vid->lnum);
if(vid->compat) {
const char *compat[] = {
[UBI_COMPAT_DELETE] = "delete",
[UBI_COMPAT_RO] = "ro",
[UBI_COMPAT_PRESERVE] = "preserve",
[UBI_COMPAT_REJECT] = "reject"
};
printf(" compat: %s\n", compat[vid->compat]);
}
printf(" data_size: %d\n", vid->data_size);
/* printf(" data_crc: %x\n", vid->data_crc); */
printf(" hdr_crc: %x\n", vid->hdr_crc);
printf(" sqnum: %lld\n", vid->sqnum);
}static int read_vid(int fd, struct ubi_vid_hdr *vid)
{
int rval = read(fd, vid,sizeof(*vid));
if(rval == sizeof(*vid)) {
unsigned int crc;
crc = crc32(UBI_CRC32_INIT, vid, UBI_EC_HDR_SIZE_CRC);
vid->magic = bswap32(vid->magic);
vid->vol_id = bswap32(vid->vol_id);
vid->lnum = bswap32(vid->lnum);
vid->data_size = bswap32(vid->data_size);
vid->used_ebs = bswap32(vid->used_ebs);
vid->data_pad = bswap32(vid->data_pad);
vid->data_crc = bswap32(vid->data_crc);
vid->hdr_crc = bswap32(vid->hdr_crc);
vid->sqnum = bswap64(vid->sqnum);
if(crc != vid->hdr_crc && vid->magic == UBI_VID_HDR_MAGIC)
printf("VID CRC: %x/%x\n", crc, vid->hdr_crc);
#if 0
/* Apparently data after VID is semi-random. */
/* Read extra 32 bits and check for all 0xff. */
{
off_t cur = lseek(fd, 0, SEEK_CUR);
if(read(fd, &crc, sizeof(crc)) == sizeof(crc)) {
if(crc != 0xffffffffUL)
printf("Corrupt VID extra @ %d with %x\n", vid->lnum, crc);
} else {
printf("Can read VID extra @ %d.\n", vid->lnum);
}
lseek(fd, cur, SEEK_SET);
}
#endif
} else
memset(vid, 0, sizeof(*vid));
return rval;
}static void print_vtbl(struct ubi_vtbl_record *vtbl)
{
printf(" Found vtbl [%d] %s\n", vtbl->name_len, vtbl->name);
printf(" Reserved PEBs: %d\n", vtbl->reserved_pebs);
printf(" Align: %d\n", vtbl->alignment);
printf(" Pad: %d\n", vtbl->data_pad);
if(vtbl->vol_type != UBI_VID_DYNAMIC)
printf(" vol_type: %s\n",
vtbl->vol_type == UBI_VID_DYNAMIC ? "dynamic" : "static");
printf(" Update: %d\n", vtbl->upd_marker);
printf(" Flags: %d\n", (int)vtbl->flags);
}static void read_vtbl(int fd, struct ubi_vtbl_record *vtbl)
{
int rval = read(fd, vtbl, sizeof(*vtbl));
if(rval == sizeof(*vtbl)) {
vtbl->reserved_pebs = bswap32(vtbl->reserved_pebs);
vtbl->alignment = bswap32(vtbl->alignment);
vtbl->data_pad = bswap32(vtbl->data_pad);
vtbl->crc = bswap32(vtbl->crc);
vtbl->name_len = bswap16(vtbl->name_len);
} else
memset(vtbl, 0, sizeof(*vtbl));
}static void print_fm_sb(struct ubi_fm_sb *fm_sb)
{
int i;
if(fm_sb->magic != UBI_FM_SB_MAGIC)
printf(" Magic: %x\n", fm_sb->magic);
if(fm_sb->version != UBI_VERSION)
printf(" Version: %d\n", (int)fm_sb->version);
printf(" data_crc: %x\n", fm_sb->data_crc);
printf(" used_blocks: %x\n", fm_sb->used_blocks);
for(i = 0; i < fm_sb->used_blocks; i++)
printf(" block_loc[%d]: %d\n", i, fm_sb->block_loc[i]);
for(i=0; i < fm_sb->used_blocks; i++)
printf(" block_ec[%d]: %d\n", i, fm_sb->block_ec[i]);
printf(" sqnum: %lld\n", fm_sb->sqnum);
}static void read_fm_sb(int fd, struct ubi_fm_sb *fm_sb)
{
int rval = read(fd, fm_sb, sizeof(*fm_sb));
if(rval == sizeof(*fm_sb)) {
int i;
fm_sb->magic = bswap32(fm_sb->magic);
fm_sb->data_crc = bswap32(fm_sb->data_crc);
fm_sb->used_blocks = bswap32(fm_sb->used_blocks);
for(i=0; i < UBI_FM_MAX_BLOCKS; i++)
fm_sb->block_loc[i] = bswap32(fm_sb->block_loc[i]);
for(i=0; i < UBI_FM_MAX_BLOCKS; i++)
fm_sb->block_ec[i] = bswap32(fm_sb->block_ec[i]);
fm_sb->sqnum = bswap64(fm_sb->sqnum);
} else
memset(fm_sb, 0, sizeof(*fm_sb));
}/* Set logical block at physical. */
static int eba_map[1920];
static int pba_map[1920];
static void usage(char *name)
{
printf("Usage: %s -b [erase block size] -e -v <ubi file> \n", name);
printf("Where,\n -e is dump the logic to physical block map.\n");
printf(" -v is dump the VID headers.\n");
printf(" -b [size] sets the erase block size (flash dependent).\n");}
typedef struct fastmap {
struct ubi_fm_sb fm_sb;
struct ubi_fm_hdr hdr;
struct ubi_fm_scan_pool pool1;
struct ubi_fm_scan_pool pool2;
/* Free, Used, Scrub and Erase */
struct ubi_fm_ec ec[0];
/* ... */
/* struct ubi_fm_volhdr vol; */
/* struct ubi_fm_eba eba[0]; */} fastmap;
int main (int argc, char *argv[])
{
int fd, i, erase_block = 0, eba_flag = 0;
int c;
struct ubi_ec_hdr ec;
struct ubi_vid_hdr vid;
int erase_size = 0x20000;
int leb_size;
off_t cur_ec = 0;
int vidless_blocks = 0;
while ((c = getopt (argc, argv, "hveb:")) != -1)
switch (c)
{
case 'h': /* Help */
usage(argv[0]);
goto out;
case 'b':
erase_size = atoi(optarg);
break;
case 'e':
eba_flag = 1;
break;
case 'v':
dump_vid = 1;
break;
case '?':
if (optopt == 'b')
fprintf (stderr, "Option -%c requires an argument.\n", optopt);
else if (isprint (optopt))
fprintf (stderr, "Unknown option `-%c'.\n", optopt);
else
fprintf (stderr,
"Unknown option character `\\x%x'.\n",
optopt);
return 1;
default:
goto out;
}
if(optind >= argc) {
usage(argv[0]);
goto out;
}
fd = open(argv[optind], O_RDONLY);
if(fd < 0) {
printf("Bad file: %s\n", argv[1]);
goto out;
}
memset(eba_map, -1, sizeof(eba_map));
memset(pba_map, -1, sizeof(pba_map));
/* Process each 'erase block'. */
read_ec(fd,&ec);
while(ec.magic == UBI_EC_HDR_MAGIC) {
leb_size = erase_size - ec.data_offset;
print_ec(&ec);
/* VID present? */
if(lseek(fd, ec.vid_hdr_offset-sizeof(ec), SEEK_CUR) == -1) {
printf("Seek error: %s\n", argv[1]);
goto out;
}
if(read_vid(fd,&vid) != sizeof(vid)) {
printf("File too small: %s\n", argv[1]);
goto out;
}
if(vid.magic == UBI_VID_HDR_MAGIC) {
print_vid(erase_block, &vid);
if(vid.vol_id == 3) {
if(eba_map[vid.lnum] != -1)
printf("EBA dup: %d %d\n", eba_map[vid.lnum], erase_block);
eba_map[vid.lnum] = erase_block;
}
pba_map[erase_block] = vid.lnum;
/* Read volume table. */
if(vid.vol_id == UBI_INTERNAL_VOL_START) {
/* Seek to PEB data offset. */
if(lseek(fd,
ec.data_offset - ec.vid_hdr_offset - sizeof(vid),
SEEK_CUR) == -1)
printf("Seek error: %s\n", argv[1]);
else {
int i;
struct ubi_vtbl_record vtbl;
for(i = 0; i < UBI_MAX_VOLUMES; i++) {
read_vtbl(fd, &vtbl);
if(vtbl.reserved_pebs ||
vtbl.name_len ||
strcmp((char*)vtbl.name, "") != 0) {
printf("VTBL %d\n", i);
print_vtbl(&vtbl);
}
}
}
} else if(vid.vol_id == UBI_FM_SB_VOLUME_ID) {
printf("Found Fastmap super block @PEB %d.\n", erase_block);
if(lseek(fd,
ec.data_offset - ec.vid_hdr_offset - sizeof(vid),
SEEK_CUR) == -1)
printf("Seek error: %s\n", argv[1]);
else {
void *data = alloca(leb_size);
struct ubi_fm_sb *fm_sb = data;
read_fm_sb(fd, data);
print_fm_sb(fm_sb);
}
} else if(vid.vol_id == UBI_FM_DATA_VOLUME_ID) {
printf("Found Fastmap data block @PEB %d.\n", erase_block);
printf("UNSUPPORTED!!!\n");
}
} else if(vid.magic != 0xffffffff){
printf("VID %d corrupt! %x\n", erase_block, vid.magic);
} else {
vidless_blocks++;
}
erase_block++;
cur_ec += erase_size;
cur_ec = lseek(fd, cur_ec, SEEK_SET);
/* Process Erase counter. */
read_ec(fd,&ec);
}
printf("Found %d vidless (free) blocks.\n", vidless_blocks);
if(eba_flag) {
printf("Logical to physical.\n");
for(i = 0; i < ALEN(eba_map); i+=8)
printf("%4d: %4d %4d %4d %4d %4d %4d %4d %4d"
" %4d %4d %4d %4d %4d %4d %4d %4d\n", i,
eba_map[i], eba_map[i+1],
eba_map[i+2], eba_map[i+3],
eba_map[i+4], eba_map[i+5],
eba_map[i+6], eba_map[i+7],
eba_map[i+8], eba_map[i+9],
eba_map[i+10], eba_map[i+11],
eba_map[i+12], eba_map[i+13],
eba_map[i+14], eba_map[i+15]);
printf("Physical to logical.\n");
for(i = 0; i < ALEN(pba_map); i+=8)
printf("%4d: %4d %4d %4d %4d %4d %4d %4d %4d"
" %4d %4d %4d %4d %4d %4d %4d %4d\n", i,
pba_map[i], pba_map[i+1],
pba_map[i+2], pba_map[i+3],
pba_map[i+4], pba_map[i+5],
pba_map[i+6], pba_map[i+7],
pba_map[i+8], pba_map[i+9],
pba_map[i+10], pba_map[i+11],
pba_map[i+12], pba_map[i+13],
pba_map[i+14], pba_map[i+15]);
}
out:
return 0;
}Re: [PATCH 6/7] New utility ubidump
Quoted 19 lines
I started some thing like this. The source is attached. As really you only want 'read only' access, there is a minimal portion of the UBI/UbiFs code that is needed. I think that only the '
ubi-media.h' is needed. The Linux is high performance and handles read/write with different fault conditions.If you write code for read-only/single thread I think it will be much more simple than the active UBI/UbiFS code in the Linux kernel. I did not look at the UbiFS layer as much. It is far more complex that UBI imho; of course, I just saw a little of the internal volumes and the fast map features. However, fast map itself should not be needed for this utility.
For your use case of analysis in a running system, I think that reads of '
/dev/mtd0ro', etc can be used. This way recovery features can also be developed if we identified some inconsistency; Ie, it is not required to 'mount' the volume before analysis.
Quoted 6 lines
The attached source just scans an file image and create an leb/peb mapping. I only used the '
ubi-media.h' as documentation.Fwiw,
Bill Pringlemeir.
Hi Bill,
I've read your code. This code shows me a better way to get EC header and VID header than my 'ioctl' design. Thanks~!
I think my former work started in a wrong way. Using MTD functionality as yours seems better to develop a new utility we want.
So in the next step, I would like to resend a new patch set which just read fs data by mtd_read(). I think a new ioctl is needed to translate specified eraseblock num from lnum to pnum. I will not care about volume table or fastmap this time. This patch set will not support dumping image file(something like -f) either. As Artem said, we should do this step by step.
Thanks for your advice. I will resend this utility soon and Cc to you.
Hu
Re: [PATCH 6/7] New utility ubidump
Quoted 9 lines · On 22 Jul 2014, [email protected] wrote:
I've read your code. This code shows me a better way to get EC header and VID header than my 'ioctl' design. Thanks~!
I think my former work started in a wrong way. Using MTD functionality as yours seems better to develop a new utility we want.
So in the next step, I would like to resend a new patch set which just read fs data by
mtd_read(). I think a new ioctl is needed to translate specified eraseblock num from lnum to pnum.
The code I sent is not very clear on this. However, the variables 'eba_map' and 'pba_map' are static arrays that do this mapping. Basically, the algorithm is,
for each erase block,
read EC header.
if(EC good)
read VID header
if(VID good)
place_in_map().
The structure of the base UBI is very simple. An issue the code I sent doesn't handle is when two headers map to the same 'logical block'. The 'sqnum' should resolve this.
place_in_map()
if(LEB empty)
map PEB.
else if(LEB sqnum < new sqnum) /* care of wrapping or large change? */
map new PEB to LEB.
For this utility, we might wish to keep the older PEB around in case the current version is corrupt at a higher (UbiFS) layer or maybe not. However, writing our own layer gives this flexibility and we don't have to add code to the Linux MTD layers. More importantly, the utility will not need (a compatible) Linux MTD layer on the host machine to check the UBI/UbiFs dump, if we can do this without an 'ioctl()'.
Thanks for attempting this. I think many people realize the value of such a tool.
Regards,
Bill Pringlemeir.
Re: [PATCH 6/7] New utility ubidump
In reply to hujianyangQuoted 6 lines · hujianyang
I've read your code. This code shows me a better way to get EC header and VID header than my 'ioctl' design. Thanks~!
I think my former work started in a wrong way. Using MTD functionality as yours seems better to develop a new utility we want.
Quoted 8 lines
So in the next step, I would like to resend a new patch set which just read fs data by
mtd_read(). I think a new ioctl is needed to translate specified eraseblock num from lnum to pnum. I will not care about volume table or fastmap this time. This patch set will not support dumping image file(something like -f) either. As Artem said, we should do this step by step.Thanks for your advice. I will resend this utility soon and Cc to you.
If possible, please, try to implement few things at a time. Try to make your initial submission smaller, and then add more stuff on top. This way it is going to be easier to merge the work. Thanks!
Signature · 2 lines
Best Regards, Artem Bityutskiy
Re: [PATCH 6/7] New utility ubidump
Quoted 5 lines
If possible, please, try to implement few things at a time. Try to make your initial submission smaller, and then add more stuff on top. This way it is going to be easier to merge the work. Thanks!
I've resend the v2 patches. I spend a short vacation last week and was busy with other stuff when I came back to work.
I think we can just apply binary data dump, ignore ubif/ubifs format data at first to make initial submission smaller. How do you feel about this?
just like:
ubidump /dev/ubiX_Y --lnum n
output binary/hex data.
PATCH 7/7 Compile support
In reply to hujianyangCompile enable.
--- Makefile | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-)
Makefile +5 -3
@@ -28,7 +28,8 @@ MTD_BINS = \ sumtool jffs2reader UBI_BINS = \ ubiupdatevol ubimkvol ubirmvol ubicrc32 ubinfo ubiattach \ - ubidetach ubinize ubiformat ubirename mtdinfo ubirsvol ubiblock + ubidetach ubinize ubiformat ubirename mtdinfo ubirsvol ubiblock \ + ubidump BINS = $(MTD_BINS) BINS += mkfs.ubifs/mkfs.ubifs
@@ -114,11 +115,12 @@ $(call mkdep,mkfs.ubifs/,mkfs.ubifs,,ubi-utils/libubi.a) obj-libiniparser.a = libiniparser.o dictionary.o obj-libscan.a = libscan.o obj-libubi.a = libubi.o +obj-libubifs.a = libubifs.o obj-libubigen.a = libubigen.o obj-mtdinfo = libubigen.a obj-ubinize = libubigen.a libiniparser.a obj-ubiformat = libubigen.a libscan.a -$(foreach v,libubi.a libubigen.a libiniparser.a libscan.a,$(eval $(call _mkdep,ubi-utils/,$(v)))) -$(foreach v,$(UBI_BINS),$(eval $(call mkdep,ubi-utils/,$(v),libubi.a ubiutils-common.o))) +$(foreach v,libubi.a libubigen.a libiniparser.a libscan.a libubifs,$(eval $(call _mkdep,ubi-utils/,$(v)))) +$(foreach v,$(UBI_BINS),$(eval $(call mkdep,ubi-utils/,$(v),libubi.a libubifs.a ubiutils-common.o)))
Signature · 1 line
1.8.1.4