Skip to contentSkip to conversation

btrfs-progs: fix kernel version parsing on some versions past 3.0

PATCH Superseded Adam Borowski 2 participants 9 messages Sep 3, 2018 Btrfs file system

lore
Share & tools
lore
#0 lore raw

PATCH 1/2 btrfs-progs: fix kernel version parsing on some versions past 3.0

The code fails if the third section is missing (like "4.18") or is followed by anything but "." or "-". This happens for example if we're not exactly at a tag and CONFIG_LOCALVERSION_AUTO=n (which results in "4.18.5+").

---
 fsfeatures.c | 5 +----
 1 file changed, 1 insertion(+), 4 deletions(-)
fsfeatures.c +1 -4
index 7d85d60f..66111bf4 100644
@@ -216,11 +216,8 @@ u32 get_running_kernel_version(void)
 		return (u32)-1;
 	version |= atoi(tmp) << 8;
 	tmp = strtok_r(NULL, ".", &saveptr);
-	if (tmp) {
-		if (!string_is_numerical(tmp))
-			return (u32)-1;
+	if (tmp && string_is_numerical(tmp))
 		version |= atoi(tmp);
-	}
 
 	return version;
 }
Signature · 1 line
2.19.0.rc1
#1 lore raw

PATCH 2/2 btrfs-progs: defrag: open files RO on new enough kernels or if root

Fixes EXTXBSY races.

---
 cmds-filesystem.c | 10 ++++++++--
 1 file changed, 8 insertions(+), 2 deletions(-)
cmds-filesystem.c +8 -2
index 06c8311b..4c9df69f 100644
@@ -26,6 +26,7 @@
 #include <ftw.h>
 #include <mntent.h>
 #include <linux/limits.h>
+#include <linux/version.h>
 #include <getopt.h>
 
 #include <btrfsutil.h>
@@ -39,12 +40,14 @@
 #include "list_sort.h"
 #include "disk-io.h"
 #include "help.h"
+#include "fsfeatures.h"
 
 /*
  * for btrfs fi show, we maintain a hash of fsids we've already printed.
  * This way we don't print dups if a given FS is mounted more than once.
  */
 static struct seen_fsid *seen_fsid_hash[SEEN_FSID_HASH_SIZE] = {NULL,};
+static mode_t defrag_ro = O_RDONLY;
 
 static const char * const filesystem_cmd_group_usage[] = {
 	"btrfs filesystem [<group>] <command> [<args>]",
@@ -877,7 +880,7 @@ static int defrag_callback(const char *fpath, const struct stat *sb,
 	if ((typeflag == FTW_F) && S_ISREG(sb->st_mode)) {
 		if (defrag_global_verbose)
 			printf("%s\n", fpath);
-		fd = open(fpath, O_RDWR);
+		fd = open(fpath, defrag_ro);
 		if (fd < 0) {
 			goto error;
 		}
@@ -914,6 +917,9 @@ static int cmd_filesystem_defrag(int argc, char **argv)
 	int compress_type = BTRFS_COMPRESS_NONE;
 	DIR *dirstream;
 
+	if (get_running_kernel_version() < KERNEL_VERSION(4,19,0) && getuid())
+		defrag_ro = O_RDWR;
+
 	/*
 	 * Kernel has a different default (256K) that is supposed to be safe,
 	 * but it does not defragment very well. The 32M will likely lead to
@@ -1014,7 +1020,7 @@ static int cmd_filesystem_defrag(int argc, char **argv)
 		int defrag_err = 0;
 
 		dirstream = NULL;
-		fd = open_file_or_dir(argv[i], &dirstream);
+		fd = open_file_or_dir3(argv[i], &dirstream, defrag_ro);
 		if (fd < 0) {
 			error("cannot open %s: %m", argv[i]);
 			ret = -errno;
Signature · 1 line
2.19.0.rc1
#2 lore raw

Re: [PATCH 2/2] btrfs-progs: defrag: open files RO on new enough kernels or if root

Quoted 1 line · Adam Borowski

Fixes EXTXBSY races.

You have to be more eloquent than that and explain at least one race condition.

Quoted 20 lines
---
 cmds-filesystem.c | 10 ++++++++--
 1 file changed, 8 insertions(+), 2 deletions(-)
cmds-filesystem.c +1 -0
index 06c8311b..4c9df69f 100644
@@ -26,6 +26,7 @@
 #include <ftw.h>
 #include <mntent.h>
 #include <linux/limits.h>
+#include <linux/version.h>
 #include <getopt.h>
 
 #include <btrfsutil.h>
@@ -39,12 +40,14 @@
 #include "list_sort.h"

cmds-filesystem.c line 50

Quoted 10 lines
cmds-filesystem.c +2 -0
 #include "disk-io.h"
 #include "help.h"
+#include "fsfeatures.h"
 
 /*
  * for btrfs fi show, we maintain a hash of fsids we've already printed.
  * This way we don't print dups if a given FS is mounted more than once.
  */
 static struct seen_fsid *seen_fsid_hash[SEEN_FSID_HASH_SIZE] = {NULL,};
+static mode_t defrag_ro = O_RDONLY;

This brings no value whatsoever, just use O_RDONLY directly

Quoted 32 lines
cmds-filesystem.c +5 -2
 static const char * const filesystem_cmd_group_usage[] = {
 	"btrfs filesystem [<group>] <command> [<args>]",
@@ -877,7 +880,7 @@ static int defrag_callback(const char *fpath, const struct stat *sb,
 	if ((typeflag == FTW_F) && S_ISREG(sb->st_mode)) {
 		if (defrag_global_verbose)
 			printf("%s\n", fpath);
-		fd = open(fpath, O_RDWR);
+		fd = open(fpath, defrag_ro);
 		if (fd < 0) {
 			goto error;
 		}
@@ -914,6 +917,9 @@ static int cmd_filesystem_defrag(int argc, char **argv)
 	int compress_type = BTRFS_COMPRESS_NONE;
 	DIR *dirstream;
 
+	if (get_running_kernel_version() < KERNEL_VERSION(4,19,0) && getuid())
+		defrag_ro = O_RDWR;
+
 	/*
 	 * Kernel has a different default (256K) that is supposed to be safe,
 	 * but it does not defragment very well. The 32M will likely lead to
@@ -1014,7 +1020,7 @@ static int cmd_filesystem_defrag(int argc, char **argv)
 		int defrag_err = 0;
 
 		dirstream = NULL;
-		fd = open_file_or_dir(argv[i], &dirstream);
+		fd = open_file_or_dir3(argv[i], &dirstream, defrag_ro);
 		if (fd < 0) {
 			error("cannot open %s: %m", argv[i]);
 			ret = -errno;
#3 lore raw

Re: [PATCH 2/2] btrfs-progs: defrag: open files RO on new enough kernels or if root

Quoted 5 lines · Nikolay Borisov
Quoted 1 line · Adam Borowski

Fixes EXTXBSY races.

You have to be more eloquent than that and explain at least one race condition.

If you try to defrag an executable that's currently running:

ERROR: cannot open XXX: Text file busy
total 1 failures

If you try to run an executable that's being defragged:

-bash: XXX: Text file busy

The former tends to be a long-lasting condition but has only benign fallout (executables almost never get fragmented, not recompressing a single file is not the end of the world), the latter is only a brief window of time but has potential for data loss.

Quoted 3 lines
Quoted 1 line

+static mode_t defrag_ro = O_RDONLY;

This brings no value whatsoever, just use O_RDONLY directly

On old kernels it gets overwritten with:

Quoted 2 lines

cmds-filesystem.c static int cmd_filesystem_defrag(int argc, char **argv) · line 921

Quoted 2 lines
cmds-filesystem.c +2 -0
+	if (get_running_kernel_version() < KERNEL_VERSION(4,19,0) && getuid())
+		defrag_ro = O_RDWR;

Meow!

Signature · 4 lines
⢀⣴⠾⠻⢶⣦⠀ What Would Jesus Do, MUD/MMORPG edition:
⣾⠁⢰⠒⠀⣿⡁ • multiplay with an admin char to benefit your mortal [Mt3:16-17]
⢿⡄⠘⠷⠚⠋⠀ • abuse item cloning bugs [Mt14:17-20, Mt15:34-37]
⠈⠳⣄⠀⠀⠀⠀ • use glitches to walk on water [Mt14:25-26]
#4 lore raw

PATCH v2 1/1 btrfs-progs: defrag: open files RO on new enough kernels

In reply to Nikolay Borisov

Defragging an executable conflicts both way with it being run, resulting in ETXTBSY. This either makes defrag fail or prevents the program from being executed.

Kernels 4.19-rc1 and later allow defragging files you could have possibly opened rw, even if the passed descriptor is ro.

---
v2: more eloquent description; root can't defrag RO on old kernels (unlike dedupe)

 cmds-filesystem.c | 10 ++++++++--
 1 file changed, 8 insertions(+), 2 deletions(-)
cmds-filesystem.c +8 -2
index 06c8311b..17e992a3 100644
@@ -26,6 +26,7 @@
 #include <ftw.h>
 #include <mntent.h>
 #include <linux/limits.h>
+#include <linux/version.h>
 #include <getopt.h>
 
 #include <btrfsutil.h>
@@ -39,12 +40,14 @@
 #include "list_sort.h"
 #include "disk-io.h"
 #include "help.h"
+#include "fsfeatures.h"
 
 /*
  * for btrfs fi show, we maintain a hash of fsids we've already printed.
  * This way we don't print dups if a given FS is mounted more than once.
  */
 static struct seen_fsid *seen_fsid_hash[SEEN_FSID_HASH_SIZE] = {NULL,};
+static mode_t defrag_ro = O_RDONLY;
 
 static const char * const filesystem_cmd_group_usage[] = {
 	"btrfs filesystem [<group>] <command> [<args>]",
@@ -877,7 +880,7 @@ static int defrag_callback(const char *fpath, const struct stat *sb,
 	if ((typeflag == FTW_F) && S_ISREG(sb->st_mode)) {
 		if (defrag_global_verbose)
 			printf("%s\n", fpath);
-		fd = open(fpath, O_RDWR);
+		fd = open(fpath, defrag_ro);
 		if (fd < 0) {
 			goto error;
 		}
@@ -914,6 +917,9 @@ static int cmd_filesystem_defrag(int argc, char **argv)
 	int compress_type = BTRFS_COMPRESS_NONE;
 	DIR *dirstream;
 
+	if (get_running_kernel_version() < KERNEL_VERSION(4,19,0))
+		defrag_ro = O_RDWR;
+
 	/*
 	 * Kernel has a different default (256K) that is supposed to be safe,
 	 * but it does not defragment very well. The 32M will likely lead to
@@ -1014,7 +1020,7 @@ static int cmd_filesystem_defrag(int argc, char **argv)
 		int defrag_err = 0;
 
 		dirstream = NULL;
-		fd = open_file_or_dir(argv[i], &dirstream);
+		fd = open_file_or_dir3(argv[i], &dirstream, defrag_ro);
 		if (fd < 0) {
 			error("cannot open %s: %m", argv[i]);
 			ret = -errno;
Signature · 1 line
2.19.0.rc1
#5 lore raw

Re: [PATCH v2] btrfs-progs: defrag: open files RO on new enough kernels

Quoted 8 lines · Adam Borowski

Defragging an executable conflicts both way with it being run, resulting in ETXTBSY. This either makes defrag fail or prevents the program from being executed.

Kernels 4.19-rc1 and later allow defragging files you could have possibly opened rw, even if the passed descriptor is ro.

So this commit really seems to be the userspace counterpart of 616d374efa23 ("btrfs: allow defrag on a file opened read-only that has rw permissions") as such IMO it will be good if you reference it.

Quoted 45 lines

---
v2: more eloquent description; root can't defrag RO on old kernels (unlike dedupe)

 cmds-filesystem.c | 10 ++++++++--
 1 file changed, 8 insertions(+), 2 deletions(-)
cmds-filesystem.c +4 -1
index 06c8311b..17e992a3 100644
@@ -26,6 +26,7 @@
 #include <ftw.h>
 #include <mntent.h>
 #include <linux/limits.h>
+#include <linux/version.h>
 #include <getopt.h>
 
 #include <btrfsutil.h>
@@ -39,12 +40,14 @@
 #include "list_sort.h"
 #include "disk-io.h"
 #include "help.h"
+#include "fsfeatures.h"
 
 /*
  * for btrfs fi show, we maintain a hash of fsids we've already printed.
  * This way we don't print dups if a given FS is mounted more than once.
  */
 static struct seen_fsid *seen_fsid_hash[SEEN_FSID_HASH_SIZE] = {NULL,};
+static mode_t defrag_ro = O_RDONLY;
 
 static const char * const filesystem_cmd_group_usage[] = {
 	"btrfs filesystem [<group>] <command> [<args>]",
@@ -877,7 +880,7 @@ static int defrag_callback(const char *fpath, const struct stat *sb,
 	if ((typeflag == FTW_F) && S_ISREG(sb->st_mode)) {
 		if (defrag_global_verbose)
 			printf("%s\n", fpath);
-		fd = open(fpath, O_RDWR);
+		fd = open(fpath, defrag_ro);
 		if (fd < 0) {
 			goto error;
 		}

cmds-filesystem.c static int cmd_filesystem_defrag(int argc, char **argv) · line 921

Quoted 5 lines
cmds-filesystem.c +2 -0
@@ -914,6 +917,9 @@ static int cmd_filesystem_defrag(int argc, char **argv)
 	int compress_type = BTRFS_COMPRESS_NONE;
 	DIR *dirstream;
 
+	if (get_running_kernel_version() < KERNEL_VERSION(4,19,0))
+		defrag_ro = O_RDWR;

I completely missed those lines in the previous posting, so alongside the context information of the kernel commit this makes sense. However, defrag_ro is a really bad name because there are case where, well, it's not going to be ro, is it ? How about something like "defrag_open_mode" or "defrag_open_flags" or something neutral which won't be implying the mode of operation.

Quoted 14 lines
cmds-filesystem.c +2 -1
+
 	/*
 	 * Kernel has a different default (256K) that is supposed to be safe,
 	 * but it does not defragment very well. The 32M will likely lead to
@@ -1014,7 +1020,7 @@ static int cmd_filesystem_defrag(int argc, char **argv)
 		int defrag_err = 0;
 
 		dirstream = NULL;
-		fd = open_file_or_dir(argv[i], &dirstream);
+		fd = open_file_or_dir3(argv[i], &dirstream, defrag_ro);
 		if (fd < 0) {
 			error("cannot open %s: %m", argv[i]);
 			ret = -errno;
#6 lore raw

PATCH v3 1/1 btrfs-progs: defrag: open files RO on new enough kernels

Defragging an executable conflicts both way with it being run, resulting in ETXTBSY. This either makes defrag fail or prevents the program from being executed.

Kernels 4.19-rc1 and later allow defragging files you could have possibly opened rw, even if the passed descriptor is ro (commit 616d374efa23).

---
v2: more eloquent description; root can't defrag RO on old kernels (unlike dedupe)
v3: more eloquentier description; s/defrag_ro/defrag_open_mode/

 cmds-filesystem.c | 10 ++++++++--
 1 file changed, 8 insertions(+), 2 deletions(-)
cmds-filesystem.c +8 -2
index 06c8311b..99e2aec0 100644
@@ -26,6 +26,7 @@
 #include <ftw.h>
 #include <mntent.h>
 #include <linux/limits.h>
+#include <linux/version.h>
 #include <getopt.h>
 
 #include <btrfsutil.h>
@@ -39,12 +40,14 @@
 #include "list_sort.h"
 #include "disk-io.h"
 #include "help.h"
+#include "fsfeatures.h"
 
 /*
  * for btrfs fi show, we maintain a hash of fsids we've already printed.
  * This way we don't print dups if a given FS is mounted more than once.
  */
 static struct seen_fsid *seen_fsid_hash[SEEN_FSID_HASH_SIZE] = {NULL,};
+static mode_t defrag_open_mode = O_RDONLY;
 
 static const char * const filesystem_cmd_group_usage[] = {
 	"btrfs filesystem [<group>] <command> [<args>]",
@@ -877,7 +880,7 @@ static int defrag_callback(const char *fpath, const struct stat *sb,
 	if ((typeflag == FTW_F) && S_ISREG(sb->st_mode)) {
 		if (defrag_global_verbose)
 			printf("%s\n", fpath);
-		fd = open(fpath, O_RDWR);
+		fd = open(fpath, defrag_open_mode);
 		if (fd < 0) {
 			goto error;
 		}
@@ -914,6 +917,9 @@ static int cmd_filesystem_defrag(int argc, char **argv)
 	int compress_type = BTRFS_COMPRESS_NONE;
 	DIR *dirstream;
 
+	if (get_running_kernel_version() < KERNEL_VERSION(4,19,0))
+		defrag_open_mode = O_RDWR;
+
 	/*
 	 * Kernel has a different default (256K) that is supposed to be safe,
 	 * but it does not defragment very well. The 32M will likely lead to
@@ -1014,7 +1020,7 @@ static int cmd_filesystem_defrag(int argc, char **argv)
 		int defrag_err = 0;
 
 		dirstream = NULL;
-		fd = open_file_or_dir(argv[i], &dirstream);
+		fd = open_file_or_dir3(argv[i], &dirstream, defrag_open_mode);
 		if (fd < 0) {
 			error("cannot open %s: %m", argv[i]);
 			ret = -errno;
Signature · 1 line
2.19.0.rc1
#7 lore raw

Re: [PATCH 2/2] btrfs-progs: defrag: open files RO on new enough kernels or if root

In reply to Adam Borowski
Quoted 35 lines · Adam Borowski

Fixes EXTXBSY races.

---
 cmds-filesystem.c | 10 ++++++++--
 1 file changed, 8 insertions(+), 2 deletions(-)
cmds-filesystem.c +3 -0
index 06c8311b..4c9df69f 100644
@@ -26,6 +26,7 @@
 #include <ftw.h>
 #include <mntent.h>
 #include <linux/limits.h>
+#include <linux/version.h>
 #include <getopt.h>
 
 #include <btrfsutil.h>
@@ -39,12 +40,14 @@
 #include "list_sort.h"
 #include "disk-io.h"
 #include "help.h"
+#include "fsfeatures.h"
 
 /*
  * for btrfs fi show, we maintain a hash of fsids we've already printed.
  * This way we don't print dups if a given FS is mounted more than once.
  */
 static struct seen_fsid *seen_fsid_hash[SEEN_FSID_HASH_SIZE] = {NULL,};
+static mode_t defrag_ro = O_RDONLY;
 
 static const char * const filesystem_cmd_group_usage[] = {
 	"btrfs filesystem [<group>] <command> [<args>]",

cmds-filesystem.c static int defrag_callback(const char *fpath, const struct stat *sb, · line 883

Quoted 5 lines
cmds-filesystem.c +1 -1
@@ -877,7 +880,7 @@ static int defrag_callback(const char *fpath, const struct stat *sb,
 	if ((typeflag == FTW_F) && S_ISREG(sb->st_mode)) {
 		if (defrag_global_verbose)
 			printf("%s\n", fpath);
-		fd = open(fpath, O_RDWR);
+		fd = open(fpath, defrag_ro);

Looking at the kernel code I think this is in fact incorrect, because in ioctl.c we have:

                if (!(file->f_mode & FMODE_WRITE)) {

                        ret = -EINVAL;

                        goto out;

                }

So it seems a hard requirement to have opened a file for RW when you want to defragment it.

Quoted 23 lines
cmds-filesystem.c +4 -1
 		if (fd < 0) {
 			goto error;
 		}
@@ -914,6 +917,9 @@ static int cmd_filesystem_defrag(int argc, char **argv)
 	int compress_type = BTRFS_COMPRESS_NONE;
 	DIR *dirstream;
 
+	if (get_running_kernel_version() < KERNEL_VERSION(4,19,0) && getuid())
+		defrag_ro = O_RDWR;
+
 	/*
 	 * Kernel has a different default (256K) that is supposed to be safe,
 	 * but it does not defragment very well. The 32M will likely lead to
@@ -1014,7 +1020,7 @@ static int cmd_filesystem_defrag(int argc, char **argv)
 		int defrag_err = 0;
 
 		dirstream = NULL;
-		fd = open_file_or_dir(argv[i], &dirstream);
+		fd = open_file_or_dir3(argv[i], &dirstream, defrag_ro);
 		if (fd < 0) {
 			error("cannot open %s: %m", argv[i]);
 			ret = -errno;
#8 lore raw

Re: [PATCH 2/2] btrfs-progs: defrag: open files RO on new enough kernels or if root

Quoted 15 lines · Nikolay Borisov

cmds-filesystem.c static int defrag_callback(const char *fpath, const struct stat *sb, · line 883

Quoted 2 lines · Adam Borowski
cmds-filesystem.c +1 -1
-		fd = open(fpath, O_RDWR);
+		fd = open(fpath, defrag_ro);

Looking at the kernel code I think this is in fact incorrect, because in ioctl.c we have:

                if (!(file->f_mode & FMODE_WRITE)) {

                        ret = -EINVAL;

                        goto out;

                }
Quoted 2 lines

So it seems a hard requirement to have opened a file for RW when you want to defragment it.

Oif! I confused this with dedup, which does allow root to dedup RO even on old kernels. Good catch.

Signature · 4 lines
⢀⣴⠾⠻⢶⣦⠀ What Would Jesus Do, MUD/MMORPG edition:
⣾⠁⢰⠒⠀⣿⡁ • multiplay with an admin char to benefit your mortal [Mt3:16-17]
⢿⡄⠘⠷⠚⠋⠀ • abuse item cloning bugs [Mt14:17-20, Mt15:34-37]
⠈⠳⣄⠀⠀⠀⠀ • use glitches to walk on water [Mt14:25-26]