xfs_db: add -R option

From: Hal Pomeranz <[email protected]>

-R is similar to -r, but allows for blockget on a mounted file system or "dirty" file system image with pending log entries. This makes xfs_db more useful for forensics where we are often dealing with these types of images. Flushing log entries to disk by mounting/unmounting the file system would allow us to use blockget, but would make changes to the file system state which are not desirable in forensics contexts.

Signed-off-by Hal Pomeranz <[email protected]>

---
 db/init.c         | 7 +++++--
 db/init.h         | 1 +
 db/sb.c           | 2 +-
 man/man8/xfs_db.8 | 8 ++++++++
 4 files changed, 15 insertions(+), 3 deletions(-)

db/init.c +0 -0

index 29fc344..96a0e78 100644

db/init.c +2 -2

 struct xfs_mount	xmount;
 struct xfs_mount	*mp;
 struct xlog		xlog;

@@ -46,7 +47,7 @@ static void
 usage(void)
 {
 	fprintf(stderr, _(
-		"Usage: %s [-ifFrxV] [-p prog] [-l logdev] [-c cmd]... device\n"
+		"Usage: %s [-ifFrRxV] [-p prog] [-l logdev] [-c cmd]... device\n"
 		), progname);
 	exit(1);
 }

@@ -66,7 +67,7 @@ init(
 	textdomain(PACKAGE);
 
 	progname = basename(argv[0]);
-	while ((c = getopt(argc, argv, "c:fFip:rxVl:")) != EOF) {
+	while ((c = getopt(argc, argv, "c:fFip:rRxVl:")) != EOF) {
 		switch (c) {
 		case 'c':
 			cmdline = xrealloc(cmdline, (ncmdline+1)*sizeof(char*));

db/init.c +0 -0

 		case 'r':
 			x.isreadonly = LIBXFS_ISREADONLY;
 			break;

db/init.h +1 -0

index b09389e..f6bfda9 100644
@@ -20,6 +20,7 @@ extern char		*fsdevice;
 extern int		blkbb;
 extern int		exitcode;
 extern int		expert_mode;
+extern int              ignore_dirty;
 extern xfs_mount_t	*mp;
 extern libxfs_init_t	x;
 extern xfs_agnumber_t	cur_agno;

db/sb.c +0 -0

index c7fbfd6..4c04d79 100644

man/man8/xfs_db.8 +8 -0

index 524b1ef..89be1aa 100644
@@ -90,6 +90,14 @@ It is only necessary to omit this flag if a command that changes data
 .RB ( write ", " blocktrash ", " crc )
 is to be used.
 .TP
+.B -R
+Like
+.B -r
+but allows the 
+.B blockget
+command even if the file system is "dirty" (unreconciled transactions
+in the log).
+.TP
 .B \-x
 Specifies expert mode.
 This enables the

Signature · 6 lines

1.8.3.1

--
To unsubscribe from this list: send the line "unsubscribe linux-xfs" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Why skip the warning? I think xfs_db should always warn on a dirty log, but perhaps we could relax the 'return 0' at the end of this hunk if the fs was opened readonly?

i.e.

dbprintf(_("ERROR: The filesystem has...")); return (x.isreadonly & LIBXFS_ISREADONLY) ? 1 : 0;

which would also make adding the -R option unnecessary.

Quoted 10 lines

Why skip the warning? I think xfs_db should always warn on a dirty log, but perhaps we could relax the 'return 0' at the end of this hunk if the fs was opened readonly?

i.e.

dbprintf(_("ERROR: The filesystem has...")); return (x.isreadonly & LIBXFS_ISREADONLY) ? 1 : 0;

which would also make adding the -R option unnecessary.

I like this solution very much. I'll send in this patch in a separate email.

Hal Pomeranz

Darrick's suggested patch to sb_logcheck() is the way to accomplish what I want with the minimum amount of code changes. But as I look at the code, I'm not sure doing it the way Darrick suggests actually expresses what we're trying to accomplish.

Let's state the problem as "Recognize that the log is dirty but allow blockget to proceed if '-r' is used". If you accept that problem statement, then sb_logcheck() should just tell the caller whether the log is dirty or not. It's up to the caller to decide what to do with that information-- in this case the init() function in db/check.c.

However, right now sb_logcheck() returns 0 for the case where there's a fatal error (like not being able to find the log) and for the log being dirty. init() would need to distinguish between those two cases and act appropriately. That means changing the return semantics of sb_logcheck() to something like -1 on error, 0 on dirty, and 1 on clean.

Note that sb_logcheck() is also called by sb_logzero() and that code would have to be tweaked to handle the new return values.

So the end result is more code changes to do it this way, but I feel like it's a more "correct" fix. I have patches created for both scenarios-- does anybody have a strong opinion about which way to proceed?

--Hal

Quoted 9 lines · On Mon May 07 06:35, [email protected] wrote:

Quoted 9 lines

Why skip the warning? I think xfs_db should always warn on a dirty log, but perhaps we could relax the 'return 0' at the end of this hunk if the fs was opened readonly?

i.e.

dbprintf(_("ERROR: The filesystem has...")); return (x.isreadonly & LIBXFS_ISREADONLY) ? 1 : 0;

Quoted 6 lines

Quoted 1 line

which would also make adding the -R option unnecessary.

I like this solution very much. I'll send in this patch in a separate email.

Hal Pomeranz

--
To unsubscribe from this list: send the line "unsubscribe linux-xfs" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

From: Hal Pomeranz <[email protected]>

-R is similar to -r, but allows for blockget on a mounted file system or "dirty" file system image with pending log entries.

This makes
xfs_db more useful for forensics where we are often dealing with these types of images. Flushing log entries to disk by mounting/unmounting the file system would allow us to use blockget, but would make changes to the file system state which are not desirable in forensics contexts.

Signed-off-by Hal Pomeranz <[email protected]>

---
 db/init.c         | 7 +++++--
 db/init.h         | 1 +
 db/sb.c           | 2 +-
 man/man8/xfs_db.8 | 8 ++++++++
 4 files changed, 15 insertions(+), 3 deletions(-)

db/init.c +5 -2

index 29fc344..96a0e78 100644
@@ -36,6 +36,7 @@ int			blkbb;
 int			exitcode;
 int			expert_mode;
 int			force;
+int                     ignore_dirty;
 struct xfs_mount	xmount;
 struct xfs_mount	*mp;
 struct xlog		xlog;

@@ -46,7 +47,7 @@ static void
 usage(void)
 {
 	fprintf(stderr, _(
-		"Usage: %s [-ifFrxV] [-p prog] [-l logdev] [-c cmd]... device\n"
+		"Usage: %s [-ifFrRxV] [-p prog] [-l logdev] [-c cmd]... device\n"
 		), progname);
 	exit(1);
 }

@@ -66,7 +67,7 @@ init(
 	textdomain(PACKAGE);
 
 	progname = basename(argv[0]);
-	while ((c = getopt(argc, argv, "c:fFip:rxVl:")) != EOF) {
+	while ((c = getopt(argc, argv, "c:fFip:rRxVl:")) != EOF) {
 		switch (c) {
 		case 'c':
 			cmdline = xrealloc(cmdline, (ncmdline+1)*sizeof(char*));

@@ -84,6 +85,8 @@ init(
 		case 'p':
 			progname = optarg;
 			break;
+		case 'R':
+		        ignore_dirty = 1;
 		case 'r':
 			x.isreadonly = LIBXFS_ISREADONLY;
 			break;

db/init.h +1 -0

index b09389e..f6bfda9 100644
@@ -20,6 +20,7 @@ extern char		*fsdevice;
 extern int		blkbb;
 extern int		exitcode;
 extern int		expert_mode;
+extern int              ignore_dirty;
 extern xfs_mount_t	*mp;
 extern libxfs_init_t	x;
 extern xfs_agnumber_t	cur_agno;

db/sb.c +1 -1

index c7fbfd6..4c04d79 100644
@@ -251,7 +251,7 @@ sb_logcheck(void)
 	if (dirty == -1) {
 		dbprintf(_("ERROR: cannot find log head/tail, run xfs_repair\n"));
 		return 0;
-	} else if (dirty == 1) {
+	} else if (dirty == 1 && ignore_dirty == 0) {
 		dbprintf(_(
 "ERROR: The filesystem has valuable metadata changes in a log which needs to\n"
 "be replayed.  Mount the filesystem to replay the log, and unmount it before\n"

man/man8/xfs_db.8 +8 -0

index 524b1ef..89be1aa 100644
@@ -90,6 +90,14 @@ It is only necessary to omit this flag if a command that changes data
 .RB ( write ", " blocktrash ", " crc )
 is to be used.
 .TP
+.B -R
+Like
+.B -r
+but allows the 
+.B blockget
+command even if the file system is "dirty" (unreconciled transactions
+in the log).
+.TP
 .B \-x
 Specifies expert mode.
 This enables the

Hm, so this probably makes it possible that blockget will wander off into inconsistencies and fail; there's a /reason/ for the check. So if anything this is probably best-effort. I doubt you can rely on blockget even completing without error, let alone being correct. In that case, is it still useful to you?

What sort of information do you hope to gather after running blockget without a replay?

Given that xfs_db is a command for experts, and forensics analysis on an unreplayed filesystem even more so, what about modifying the logformat command to allow zeroing of a dirty log with some type of force option, which would then allow blkget to proceed? (assuming you were working on a copy, of course.) That would make things pretty explicit.

Is that too convoluted?

Otherwise I might suggest adding a switch specific to the blockget_f function, which would just skip the sb_logcheck() altogether. That would be more targeted, and wouldn't be some global switch which affects every current and future caller of sb_logcheck. A warning about how the log is being ignored and results may be inconsistent could then be added specifically to blockget_f.

Quoted 6 lines

Otherwise I might suggest adding a switch specific to the blockget_f function, which would just skip the sb_logcheck() altogether. That would be more targeted, and wouldn't be some global switch which affects every current and future caller of sb_logcheck. A warning about how the log is being ignored and results may be inconsistent could then be added specifically to blockget_f.

Which sort of brings us back to the conversation that Darrick and I have been having. Adding a command-line switch seems wrong-- I'm fully convinced of that. But I'd still like to be able to blockget to try to work if the file system is dirty but "-r" is used.

Darrick came up with one fix, which is the first patch option below. After staring at Darrick's suggestion a while, I came up with a different fix (second patch) that requires more code changes but I think more accurately addresses the problem statement. See previous messages in this thread for more detail.

Let me know which you prefer and I'll submit the patch in the proper format to the list.

Cheers!

--Hal

Well, I don't really like either patch for the reasons stated above. xfs_db /can/ write to the filesystem, and -r disables that... but -r as a modifier to affect the behavior of a read-only command is unintuitive.

"It is only necessary to omit this flag if a command that changes data (write, blocktrash, crc) is to be used."

What are the objections to a blockget modifier option? That seemed like the most direct & obvious solution to me.

"blockget/check -L : attempt to perform the blockget and check functions even if the log contains unreplayed metadata," or something like that?

Quoted 5 lines

Well, I don't really like either patch for the reasons stated above. xfs_db /can/ write to the filesystem, and -r disables that... but -r as a modifier to affect the behavior of a read-only command is unintuitive.

Quoted 8 lines

"It is only necessary to omit this flag if a command that changes data (write, blocktrash, crc) is to be used."

What are the objections to a blockget modifier option? That seemed like the most direct & obvious solution to me.

"blockget/check -L : attempt to perform the blockget and check functions even if the log contains unreplayed metadata," or something like that?

Ah, OK. I'm getting what you're saying now. Let me poke around with the code some and produce a patch that does what you're suggesting.

--Hal

--
To unsubscribe from this list: send the line "unsubscribe linux-xfs" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

Quoted 5 lines

Hm, so this probably makes it possible that blockget will wander off into inconsistencies and fail; there's a /reason/ for the check. So if anything this is probably best-effort. I doubt you can rely on blockget even completing without error, let alone being correct. In that case, is it still useful to you?

I've been testing a modified version of xfs_db on live file systems and dirty image files, and I have yet to have blockget fail. I agree that it's certainly possible for blockget to blow up under these circumstances, but it's working often enough to be useful.

Quoted 2 lines

What sort of information do you hope to gather after running blockget without a replay?

Suppose I can match a string or magic number for a file of interest at a specific byte offset in the file system image. A little arithmetic and I have a daddr value. xfs_db let's me convert that to an fsblock and then call blockuse -n to get the inode and file path. I've tried it, and it works with my modifed xfs_db.

Until we get XFS support into libsleuthkit (which I'm working towards, but it's going to be a while), having this functionality lets me use xfs_db as a forensic tool. And it also lets me use xfs_db to validate other forensic tools.