Skip to contentSkip to conversation

xfsprogs/io: add getdents command

PATCH Superseded Brian Foster 2 participants 4 messages Jul 11, 2013 – Jul 23, 2013

v1 of 3 Superseded Latest is v3 →

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 xfsprogs/io: add getdents command

getdents reads the directory entries from an open directory from the provided offset (or 0 if not specified). On completion, getdents prints summary information regarding the number of operations and bytes transferred. Options are available to specify the starting offset, buffer size and verbose mode to dump directory entry information from the buffer.

---

Hi all,

This is something I hacked together for testing something for gluster. Thoughts appreciated!

Brian

 io/Makefile   |   4 ++
 io/getdents.c | 175 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 io/init.c     |   1 +
 io/io.h       |   1 +
 4 files changed, 181 insertions(+)
 create mode 100644 io/getdents.c
io/Makefile +4 -0
index 50edf91..cfd1b11 100644
@@ -80,6 +80,10 @@ ifeq ($(HAVE_PREADV),yes)
 LCFLAGS += -DHAVE_PREADV -DHAVE_PWRITEV
 endif
 
+ifeq ($(PKG_PLATFORM),linux)
+CFILES += getdents.c
+endif
+
 default: depend $(LTCOMMAND)
 
 include $(BUILDRULES)
io/getdents.c new file +175 -0

175 lines not on this page · raw message

io/init.c +1 -0
index ca3055a..44cce44 100644
@@ -60,6 +60,7 @@ init_commands(void)
 	file_init();
 	freeze_init();
 	fsync_init();
+	getdents_init();
 	getrusage_init();
 	help_init();
 	imap_init();
io/io.h +1 -0
index 91f0e3e..5658b13 100644
@@ -94,6 +94,7 @@ extern void		bmap_init(void);
 extern void		file_init(void);
 extern void		freeze_init(void);
 extern void		fsync_init(void);
+extern void		getdents_init(void);
 extern void		getrusage_init(void);
 extern void		help_init(void);
 extern void		imap_init(void);
Signature · 6 lines
1.8.1.4

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
#1 lore raw

io/Makefile ifeq ($(HAVE_PREADV),yes) · line 85

Quoted 3 lines
io/Makefile +3 -0
+ifeq ($(PKG_PLATFORM),linux)
+CFILES += getdents.c
+endif

I'd make a real test for getdents() rather than tying it to Linux. Just copy what's done for sync_file_range :).

io/getdents.c line 37

Quoted 6 lines
io/getdents.c new file +6 -0
+struct linux_dirent {
+	unsigned long	d_ino;     /* Inode number */
+	unsigned long	d_off;     /* Offset to next linux_dirent */
+	unsigned short	d_reclen;  /* Length of this linux_dirent */
+	char		d_name[];  /* Filename (null-terminated) */
+};

If we're only going to use one syscall interface, I'd use getdents64().

struct linux_dirent64 {
        u64             d_ino;
        s64             d_off;
        unsigned short  d_reclen;
        unsigned char   d_type;
        char            d_name[0];
};      

But it could also be useful to see the native 'long' interface for 32bit apps, though, say if you're trying to debug ext4 htree d_off values which are magically truncated for 32bit compat tasks.

Quoted 6 lines
io/getdents.c new file +6 -0
+static void
+dump_dir_buffer(
+	struct linux_dirent *dirp,
+	long long offset,
+	long long length)
+{

io/getdents.c line 54

Quoted 10 lines
io/getdents.c new file +10 -0
+	while (length > 0) {
+		printf("%08llx: 0x%lx (%s)\n",
+			offset, dirp->d_ino, dirp->d_name);
+
+		offset = dirp->d_off;
+		length -= dirp->d_reclen;
+
+		dirp = (struct linux_dirent *) ((char *) dirp + dirp->d_reclen);
+	}
+}

You could also print out d_type. In getdents64() it's a proper struct member, in the bad old 'long' interface it's hidden in padding after the null in d_name.

It'd be nice to see d_off for the last entry. btrfs, for example, sets it to CRAZY and that can be good to know. In the past it has caused 32bit glibc to return -EOVERFLOW when reading entries from getdents64() whose d_off overflowed 32bit off_t.

Interestingly, notice that with getdents() you don't know what the offset of the first entry is. You just know the offset that you started reading from and the offset of the next entry.

- z
(*throws another bag of nickels in the readdir-design-disaster jar*)

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs

#2 lore raw
Quoted 7 lines · Zach Brown

io/Makefile ifeq ($(HAVE_PREADV),yes) · line 85

Quoted 3 lines
io/Makefile +3 -0
+ifeq ($(PKG_PLATFORM),linux)
+CFILES += getdents.c
+endif

I'd make a real test for getdents() rather than tying it to Linux. Just copy what's done for sync_file_range :).

Having a look back, I think a reason why I avoided this was I didn't have a libc interface for getdents. Perhaps there's still a way to test that, but I'm also wondering if this would be more broadly useful if I just converted it over to generic readdir(). Thoughts?

Brian

Quoted 47 lines

io/getdents.c line 37

Quoted 6 lines
io/getdents.c new file +6 -0
+struct linux_dirent {
+	unsigned long	d_ino;     /* Inode number */
+	unsigned long	d_off;     /* Offset to next linux_dirent */
+	unsigned short	d_reclen;  /* Length of this linux_dirent */
+	char		d_name[];  /* Filename (null-terminated) */
+};

If we're only going to use one syscall interface, I'd use getdents64().

struct linux_dirent64 {
        u64             d_ino;
        s64             d_off;
        unsigned short  d_reclen;
        unsigned char   d_type;
        char            d_name[0];
};      

But it could also be useful to see the native 'long' interface for 32bit apps, though, say if you're trying to debug ext4 htree d_off values which are magically truncated for 32bit compat tasks.

Quoted 6 lines
io/getdents.c new file +6 -0
+static void
+dump_dir_buffer(
+	struct linux_dirent *dirp,
+	long long offset,
+	long long length)
+{

io/getdents.c line 54

Quoted 10 lines
io/getdents.c new file +10 -0
+	while (length > 0) {
+		printf("%08llx: 0x%lx (%s)\n",
+			offset, dirp->d_ino, dirp->d_name);
+
+		offset = dirp->d_off;
+		length -= dirp->d_reclen;
+
+		dirp = (struct linux_dirent *) ((char *) dirp + dirp->d_reclen);
+	}
+}

You could also print out d_type. In getdents64() it's a proper struct member, in the bad old 'long' interface it's hidden in padding after the null in d_name.

It'd be nice to see d_off for the last entry. btrfs, for example, sets it to CRAZY and that can be good to know. In the past it has caused 32bit glibc to return -EOVERFLOW when reading entries from getdents64() whose d_off overflowed 32bit off_t.

Quoted 7 lines

Interestingly, notice that with getdents() you don't know what the offset of the first entry is. You just know the offset that you started reading from and the offset of the next entry.

- z
(*throws another bag of nickels in the readdir-design-disaster jar*)

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs

#3 lore raw
Quoted 11 lines · Brian Foster
Quoted 7 lines · Zach Brown

io/Makefile ifeq ($(HAVE_PREADV),yes) · line 85

Quoted 3 lines
io/Makefile +3 -0
+ifeq ($(PKG_PLATFORM),linux)
+CFILES += getdents.c
+endif

I'd make a real test for getdents() rather than tying it to Linux. Just copy what's done for sync_file_range :).

Having a look back, I think a reason why I avoided this was I didn't have a libc interface for getdents.

Bleh, yeah. I guess you could test for the syscall(__NR,) signature.

Quoted 3 lines

Perhaps there's still a way to test that, but I'm also wondering if this would be more broadly useful if I just converted it over to generic readdir(). Thoughts?

That seems reasonable, given that's what apps actually use. If tests really wanted to inspect getdents specifically they can worry about it.

- z

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs