Skip to contentSkip to conversation

btrfs: add read mirror policy

PATCH In review Anand Jain 5 participants 15 messages May 16, 2018 – Jan 22, 2019 Btrfs file system

lore
Share & tools
lore

Anand Jain

author
#0 lore raw

v2 cover btrfs: add read mirror policy

Not yet ready for the integration. As I need to introduce -o no_read_mirror_policy instead of -o read_mirror_policy=-<devid> to reset the policy as in 3/3. But I am sending this early so that we can use it for btrfs/161 in the ML, and this patch-set is stable enough for the testing.

Anand Jain (3):
btrfs: add mount option read_mirror_policy
btrfs: add read_mirror_policy parameter devid
btrfs: read_mirror_policy ability to reset

 fs/btrfs/ctree.h   |  2 ++
 fs/btrfs/super.c   | 44 ++++++++++++++++++++++++++++++++++++++++++++
 fs/btrfs/volumes.c | 18 +++++++++++++++++-
 fs/btrfs/volumes.h |  7 +++++++
 4 files changed, 70 insertions(+), 1 deletion(-)
Signature · 1 line
2.7.0
author
#1 lore raw

PATCH v2 1/3 btrfs: add mount option read_mirror_policy

In case of RAID1 and RAID10 devices are mirror-ed, a read IO can pick any device for reading. This choice of picking a device for reading should be configurable. In short not one policy would satisfy all types of workload and configs.

So before we add more policies, this patch-set makes existing $pid policy configurable from the mount option.

For example..
mount -o read_mirror_policy=pid (which is also default)

---
 fs/btrfs/ctree.h   |  2 ++
 fs/btrfs/super.c   | 10 ++++++++++
 fs/btrfs/volumes.c |  8 +++++++-
 fs/btrfs/volumes.h |  5 +++++
 4 files changed, 24 insertions(+), 1 deletion(-)
fs/btrfs/ctree.h +2 -0
index bfa96697209a..0d997939cdae 100644
@@ -1121,6 +1121,8 @@ struct btrfs_fs_info {
 	spinlock_t ref_verify_lock;
 	struct rb_root block_tree;
 #endif
+	/* Policy to balance read across mirrored devices */
+	int read_mirror_policy;
 };
 
 static inline struct btrfs_fs_info *btrfs_sb(struct super_block *sb)
fs/btrfs/super.c +10 -0
index c67fafaa2fe7..21eff0ac1e4f 100644
@@ -345,6 +345,7 @@ enum {
 #ifdef CONFIG_BTRFS_FS_REF_VERIFY
 	Opt_ref_verify,
 #endif
+	Opt_read_mirror_policy,
 	Opt_err,
 };
 
@@ -414,6 +415,7 @@ static const match_table_t tokens = {
 #ifdef CONFIG_BTRFS_FS_REF_VERIFY
 	{Opt_ref_verify, "ref_verify"},
 #endif
+	{Opt_read_mirror_policy, "read_mirror_policy=%s"},
 	{Opt_err, NULL},
 };
 
@@ -844,6 +846,14 @@ int btrfs_parse_options(struct btrfs_fs_info *info, char *options,
 			btrfs_set_opt(info->mount_opt, REF_VERIFY);
 			break;
 #endif
+		case Opt_read_mirror_policy:
+			if (strcmp(args[0].from, "pid") == 0) {
+				info->read_mirror_policy =
+					BTRFS_READ_MIRROR_BY_PID;
+				break;
+			}
+			ret = -EINVAL;
+			goto out;
 		case Opt_err:
 			btrfs_info(info, "unrecognized mount option '%s'", p);
 			ret = -EINVAL;
fs/btrfs/volumes.c +7 -1
index 81fb38884cac..64dba5c4cf33 100644
@@ -5219,7 +5219,13 @@ static int find_live_mirror(struct btrfs_fs_info *fs_info,
 	else
 		num_stripes = map->num_stripes;
 
-	preferred_mirror = first + current->pid % num_stripes;
+	switch(fs_info->read_mirror_policy) {
+	case BTRFS_READ_MIRROR_DEFAULT:
+	case BTRFS_READ_MIRROR_BY_PID:
+	default:
+		preferred_mirror = first + current->pid % num_stripes;
+		break;
+	}
 
 	if (dev_replace_is_ongoing &&
 	    fs_info->dev_replace.cont_reading_from_srcdev_mode ==
fs/btrfs/volumes.h +5 -0
index 5139ec8daf4c..953df9925832 100644
@@ -34,6 +34,11 @@ struct btrfs_pending_bios {
 #define btrfs_device_data_ordered_init(device) do { } while (0)
 #endif
 
+enum btrfs_read_mirror_type {
+	BTRFS_READ_MIRROR_DEFAULT,
+	BTRFS_READ_MIRROR_BY_PID,
+};
+
 #define BTRFS_DEV_STATE_WRITEABLE	(0)
 #define BTRFS_DEV_STATE_IN_FS_METADATA	(1)
 #define BTRFS_DEV_STATE_MISSING		(2)
Signature · 1 line
2.7.0
author
#2 lore raw

PATCH v2 2/3 btrfs: add read_mirror_policy parameter devid

In reply to Anand Jain

Adds the mount option:
mount -o read_mirror_policy=<devid>

To set the devid of the device which should be used for read. That means all the normal reads will go to that particular device only.

This also helps testing and gives a better control for the test scripts including mount context reads.

---
 fs/btrfs/super.c   | 21 +++++++++++++++++++++
 fs/btrfs/volumes.c | 10 ++++++++++
 fs/btrfs/volumes.h |  2 ++
 3 files changed, 33 insertions(+)
fs/btrfs/super.c +21 -0
index 21eff0ac1e4f..7ddecf4178a6 100644
@@ -852,6 +852,27 @@ int btrfs_parse_options(struct btrfs_fs_info *info, char *options,
 					BTRFS_READ_MIRROR_BY_PID;
 				break;
 			}
+
+			intarg = 0;
+			if (match_int(&args[0], &intarg) == 0) {
+				struct btrfs_device *device;
+
+				device = btrfs_find_device(info, intarg,
+							   NULL, NULL);
+				if (!device) {
+					btrfs_err(info,
+					  "read_mirror_policy: invalid devid %d",
+					  intarg);
+					ret = -EINVAL;
+					goto out;
+				}
+				info->read_mirror_policy =
+						BTRFS_READ_MIRROR_BY_DEV;
+				set_bit(BTRFS_DEV_STATE_READ_MIRROR,
+					&device->dev_state);
+				break;
+			}
+
 			ret = -EINVAL;
 			goto out;
 		case Opt_err:
fs/btrfs/volumes.c +10 -0
index 64dba5c4cf33..3a9c3804ff17 100644
@@ -5220,6 +5220,16 @@ static int find_live_mirror(struct btrfs_fs_info *fs_info,
 		num_stripes = map->num_stripes;
 
 	switch(fs_info->read_mirror_policy) {
+	case BTRFS_READ_MIRROR_BY_DEV:
+		preferred_mirror = first;
+		if (test_bit(BTRFS_DEV_STATE_READ_MIRROR,
+			     &map->stripes[preferred_mirror].dev->dev_state))
+			break;
+		if (test_bit(BTRFS_DEV_STATE_READ_MIRROR,
+			     &map->stripes[++preferred_mirror].dev->dev_state))
+			break;
+		preferred_mirror = first;
+		break;
 	case BTRFS_READ_MIRROR_DEFAULT:
 	case BTRFS_READ_MIRROR_BY_PID:
 	default:
fs/btrfs/volumes.h +2 -0
index 953df9925832..55b2eebbf183 100644
@@ -37,6 +37,7 @@ struct btrfs_pending_bios {
 enum btrfs_read_mirror_type {
 	BTRFS_READ_MIRROR_DEFAULT,
 	BTRFS_READ_MIRROR_BY_PID,
+	BTRFS_READ_MIRROR_BY_DEV,
 };
 
 #define BTRFS_DEV_STATE_WRITEABLE	(0)
@@ -44,6 +45,7 @@ enum btrfs_read_mirror_type {
 #define BTRFS_DEV_STATE_MISSING		(2)
 #define BTRFS_DEV_STATE_REPLACE_TGT	(3)
 #define BTRFS_DEV_STATE_FLUSH_SENT	(4)
+#define BTRFS_DEV_STATE_READ_MIRROR	(5)
 
 struct btrfs_device {
 	struct list_head dev_list;
Signature · 1 line
2.7.0
Tested-by on v2 2/3
#3 lore raw

Re: [PATCH v2 2/3] btrfs: add read_mirror_policy parameter devid

On 2018-05-16 11:03, Anand Jain wrote:

Going back to an old patchset I was testing this weekend:

Quoted 10 lines

Adds the mount option:
mount -o read_mirror_policy=<devid>

To set the devid of the device which should be used for read. That means all the normal reads will go to that particular device only.

This also helps testing and gives a better control for the test scripts including mount context reads.

Quoted 23 lines
---
 fs/btrfs/super.c   | 21 +++++++++++++++++++++
 fs/btrfs/volumes.c | 10 ++++++++++
 fs/btrfs/volumes.h |  2 ++
 3 files changed, 33 insertions(+)
fs/btrfs/super.c +8 -0
index 21eff0ac1e4f..7ddecf4178a6 100644
@@ -852,6 +852,27 @@ int btrfs_parse_options(struct btrfs_fs_info
 *info, char *options,
 					BTRFS_READ_MIRROR_BY_PID;
 				break;
 			}
+
+			intarg = 0;
+			if (match_int(&args[0], &intarg) == 0) {
+				struct btrfs_device *device;
+
+				device = btrfs_find_device(info, intarg,
+							   NULL, NULL);
+				if (!device) {

fs/btrfs/super.c int btrfs_parse_options(struct btrfs_fs_info · line 873

Quoted 10 lines
fs/btrfs/super.c +10 -0
+					btrfs_err(info,
+					  "read_mirror_policy: invalid devid %d",
+					  intarg);
+					ret = -EINVAL;
+					goto out;
+				}
+				info->read_mirror_policy =
+						BTRFS_READ_MIRROR_BY_DEV;
+				set_bit(BTRFS_DEV_STATE_READ_MIRROR,
+					&device->dev_state);

Not an expert, but might this be overwritten with another state? e.g. BTRFS_DEV_STATE_REPLACE_TGT. The device would then lose its READ_MIRROR flag and the fs would always use the first device for reading.

fs/btrfs/super.c int btrfs_parse_options(struct btrfs_fs_info *info, char *options, · line 874

Quoted 2 lines
fs/btrfs/super.c +2 -0
+				break;
+			}

I noticed that it's possible to pass this option multiple times at mount, which sets multiple devices as read mirrors. While that doesn't do anything harmful, the code below will only use the first device. It may be worth at least mentioning this in documentation.

Quoted 12 lines
fs/btrfs/super.c +1 -0
+
 			ret = -EINVAL;
 			goto out;
fs/btrfs/super.c +0 -0
 		case Opt_err:
fs/btrfs/volumes.c +0 -0
index 64dba5c4cf33..3a9c3804ff17 100644
@@ -5220,6 +5220,16 @@ static int find_live_mirror(struct
 btrfs_fs_info *fs_info,
 		num_stripes = map->num_stripes;
 

fs/btrfs/volumes.c static int find_live_mirror(struct · line 5232

Quoted 10 lines
fs/btrfs/volumes.c +9 -0
 	switch(fs_info->read_mirror_policy) {
+	case BTRFS_READ_MIRROR_BY_DEV:
+		preferred_mirror = first;
+		if (test_bit(BTRFS_DEV_STATE_READ_MIRROR,
+			     &map->stripes[preferred_mirror].dev->dev_state))
+			break;
+		if (test_bit(BTRFS_DEV_STATE_READ_MIRROR,
+			     &map->stripes[++preferred_mirror].dev->dev_state))
+			break;
+		preferred_mirror = first;

Why set preferred_mirror again? The only effect of re-setting it will be to use the lowest devid (1) rather than the highest (2). Is there any difference? Either way it should never happen, because the above code traps it (except when the BTRFS_DEV_STATE_READ_MIRROR flag has been changed as mentioned above).

From Goffredo's comment[1] on Timofey's similar effect patch, if it becomes possible to have more mirrors in a RAID1/RAID10 fs then this code will need to be updated to test dev_state for more than two devices. Would it be sensible to implement this as a for loop straight away?

Quoted 17 lines
fs/btrfs/volumes.c +1 -0
+		break;
 	case BTRFS_READ_MIRROR_DEFAULT:
 	case BTRFS_READ_MIRROR_BY_PID:
fs/btrfs/volumes.c +0 -0
 	default:
fs/btrfs/volumes.h +1 -0
index 953df9925832..55b2eebbf183 100644
@@ -37,6 +37,7 @@ struct btrfs_pending_bios {
 enum btrfs_read_mirror_type {
 	BTRFS_READ_MIRROR_DEFAULT,
 	BTRFS_READ_MIRROR_BY_PID,
+	BTRFS_READ_MIRROR_BY_DEV,
 };
 
 #define BTRFS_DEV_STATE_WRITEABLE	(0)

fs/btrfs/volumes.h enum btrfs_read_mirror_type { · line 51

Quoted 7 lines
fs/btrfs/volumes.h +1 -0
@@ -44,6 +45,7 @@ enum btrfs_read_mirror_type {
 #define BTRFS_DEV_STATE_MISSING		(2)
 #define BTRFS_DEV_STATE_REPLACE_TGT	(3)
 #define BTRFS_DEV_STATE_FLUSH_SENT	(4)
+#define BTRFS_DEV_STATE_READ_MIRROR	(5)
 
 struct btrfs_device {
 	struct list_head dev_list;

Happy to add Tested-By on patch 1/3 too, but I haven't looked at 3/3 yet.

[1]: https://patchwork.kernel.org/patch/10678531/#22315779

Signature · 1 line
Steven Davies
author
#4 lore raw

Re: [PATCH v2 2/3] btrfs: add read_mirror_policy parameter devid

Quoted 7 lines · Steven Davies

On 2018-05-16 11:03, Anand Jain wrote:

Going back to an old patchset I was testing this weekend:

Quoted 3 lines

Adds the mount option:
  mount -o read_mirror_policy=<devid>

Quoted 8 lines
Quoted 7 lines

To set the devid of the device which should be used for read. That means all the normal reads will go to that particular device only.

This also helps testing and gives a better control for the test scripts including mount context reads.

Thanks for testing.

Quoted 28 lines
Quoted 28 lines

---
 fs/btrfs/super.c   | 21 +++++++++++++++++++++
 fs/btrfs/volumes.c | 10 ++++++++++
 fs/btrfs/volumes.h |  2 ++

 3 files changed, 33 insertions(+)
fs/btrfs/super.c +12 -0
index 21eff0ac1e4f..7ddecf4178a6 100644
@@ -852,6 +852,27 @@ int btrfs_parse_options(struct btrfs_fs_info
 *info, char *options,
                      BTRFS_READ_MIRROR_BY_PID;
                  break;
              }
+
+            intarg = 0;
+            if (match_int(&args[0], &intarg) == 0) {
+                struct btrfs_device *device;
+
+                device = btrfs_find_device(info, intarg,
+                               NULL, NULL);
+                if (!device) {
+                    btrfs_err(info,
+                      "read_mirror_policy: invalid devid %d",
+                      intarg);
+                    ret = -EINVAL;
Quoted 8 lines

fs/btrfs/super.c int btrfs_parse_options(struct btrfs_fs_info *info, char *options, · line 872

Quoted 6 lines
fs/btrfs/super.c +6 -0
+                    goto out;
+                }
+                info->read_mirror_policy =
+                        BTRFS_READ_MIRROR_BY_DEV;
+                set_bit(BTRFS_DEV_STATE_READ_MIRROR,
+                    &device->dev_state);

Not an expert, but might this be overwritten with another state? e.g. BTRFS_DEV_STATE_REPLACE_TGT. The device would then lose its READ_MIRROR flag and the fs would always use the first device for reading.

It won't it is defined as bitmap.

Quoted 4 lines

fs/btrfs/super.c int btrfs_parse_options(struct btrfs_fs_info *info, char *options, · line 874

Quoted 2 lines
fs/btrfs/super.c +2 -0
+                break;
+            }

I noticed that it's possible to pass this option multiple times at mount, which sets multiple devices as read mirrors. While that doesn't do anything harmful, the code below will only use the first device. It may be worth at least mentioning this in documentation.

There were few feedback if read_mirror_policy should be a mount option or a sysfs or a property. IMO property is better as it would be persistent. In sysfs and mount-option, the user or a config file has to remember. Will fix.

Quoted 12 lines
Quoted 12 lines
fs/btrfs/super.c +1 -0
+
              ret = -EINVAL;
              goto out;

         case Opt_err:

fs/btrfs/volumes.c +0 -0
index 64dba5c4cf33..3a9c3804ff17 100644
@@ -5220,6 +5220,16 @@ static int find_live_mirror(struct
 btrfs_fs_info *fs_info,
          num_stripes = map->num_stripes;
 
Quoted 12 lines

fs/btrfs/volumes.c static int find_live_mirror(struct btrfs_fs_info *fs_info, · line 5231

Quoted 10 lines
fs/btrfs/volumes.c +9 -0
      switch(fs_info->read_mirror_policy) {
+    case BTRFS_READ_MIRROR_BY_DEV:
+        preferred_mirror = first;
+        if (test_bit(BTRFS_DEV_STATE_READ_MIRROR,
+                 &map->stripes[preferred_mirror].dev->dev_state))
+            break;
+        if (test_bit(BTRFS_DEV_STATE_READ_MIRROR,
+                 &map->stripes[++preferred_mirror].dev->dev_state)) <--[*]
+            break;
+        preferred_mirror = first;

Why set preferred_mirror again? The only effect of re-setting it will be to use the lowest devid (1) rather than the highest (2). Is there any difference? Either way it should never happen, because the above code traps it (except when the BTRFS_DEV_STATE_READ_MIRROR flag has been changed as mentioned above).

Code at [*] above does ++preferred_mirror. So the following
preferred_mirror = first;
is not redundant.

Quoted 2 lines

From Goffredo's comment[1] on Timofey's similar effect patch, if it
becomes possible to have more mirrors in a RAID1/RAID10 fs then this code will need to be updated to test dev_state for more than two devices. Would it be sensible to implement this as a for loop straight away?

Right. A loop is better, will add.

Quoted 22 lines
Quoted 22 lines
fs/btrfs/volumes.c +1 -0
+        break;

     case BTRFS_READ_MIRROR_DEFAULT:
     case BTRFS_READ_MIRROR_BY_PID:
     default:

fs/btrfs/volumes.h +2 -0
index 953df9925832..55b2eebbf183 100644
@@ -37,6 +37,7 @@ struct btrfs_pending_bios {
  enum btrfs_read_mirror_type {
      BTRFS_READ_MIRROR_DEFAULT,
      BTRFS_READ_MIRROR_BY_PID,
+    BTRFS_READ_MIRROR_BY_DEV,
  };
 
  #define BTRFS_DEV_STATE_WRITEABLE    (0)
@@ -44,6 +45,7 @@ enum btrfs_read_mirror_type {
  #define BTRFS_DEV_STATE_MISSING        (2)
  #define BTRFS_DEV_STATE_REPLACE_TGT    (3)
  #define BTRFS_DEV_STATE_FLUSH_SENT    (4)
+#define BTRFS_DEV_STATE_READ_MIRROR    (5)
 
Quoted 6 lines

fs/btrfs/volumes.h enum btrfs_read_mirror_type { · line 51

Quoted 2 lines
fs/btrfs/volumes.h +0 -0
  struct btrfs_device {
      struct list_head dev_list;

Happy to add Tested-By on patch 1/3 too, but I haven't looked at 3/3 yet.

[1]: https://patchwork.kernel.org/patch/10678531/#22315779

Thanks,
Anand.

#5 lore raw

Re: [PATCH v2 2/3] btrfs: add read_mirror_policy parameter devid

Quoted 6 lines · Anand Jain
Quoted 5 lines · Steven Davies
Quoted 2 lines · Anand Jain

+                break;
+            }

Quoted 6 lines
Quoted 1 line

I noticed that it's possible to pass this option multiple times at mount, which sets multiple devices as read mirrors. While that doesn't do anything harmful, the code below will only use the first device. It may be worth at least mentioning this in documentation.

There were few feedback if read_mirror_policy should be a mount option or a sysfs or a property. IMO property is better as it would be persistent. In sysfs and mount-option, the user or a config file has to remember. Will fix.

I agree. Would/could this be set at mkfs time? It would then be similar to how mdadm's --write-mostly is set up.

Quoted 26 lines
Quoted 12 lines
Quoted 12 lines

+
             ret = -EINVAL;
             goto out;
         case Opt_err:

fs/btrfs/volumes.c +0 -0
index 64dba5c4cf33..3a9c3804ff17 100644
@@ -5220,6 +5220,16 @@ static int find_live_mirror(struct
 btrfs_fs_info *fs_info,
          num_stripes = map->num_stripes;
 
Quoted 13 lines
Quoted 11 lines
fs/btrfs/volumes.c +7 -0
      switch(fs_info->read_mirror_policy) {
+    case BTRFS_READ_MIRROR_BY_DEV:
+        preferred_mirror = first;
+        if (test_bit(BTRFS_DEV_STATE_READ_MIRROR,
+                 &map->stripes[preferred_mirror].dev->dev_state))
+            break;
+        if (test_bit(BTRFS_DEV_STATE_READ_MIRROR,
+                 &map->stripes[++preferred_mirror].dev->dev_state)) 

<--[*]
+            break;
+        preferred_mirror = first;

Why set preferred_mirror again? The only effect of re-setting it will be to use the lowest devid (1) rather than the highest (2). Is there any difference? Either way it should never happen, because the above code traps it (except when the BTRFS_DEV_STATE_READ_MIRROR flag has been changed as mentioned above).

Quoted 3 lines

Code at [*] above does ++preferred_mirror. So the following
preferred_mirror = first;
is not redundant.

Yes, but preferred_mirror will be first + num_stripes; meaning that without that line the last stripe is preferred and with it the first stripe is preferred. It only changes the fallback case from reading from devid 2 to devid 1, which barely matters. Perhaps it would be even better to fall back to pid % num_stripes in this case anyway?

Quoted 4 lines
Quoted 2 lines

From Goffredo's comment[1] on Timofey's similar effect patch, if it
becomes possible to have more mirrors in a RAID1/RAID10 fs then this code will need to be updated to test dev_state for more than two devices. Would it be sensible to implement this as a for loop straight away?

Right. A loop is better, will add.

Thinking more about this, if it becomes possible to have more than two devices in a RAID1 or part-RAID10 then do we also need to consider that there may be more than one READ_MIRROR drive? e.g. slow+fast+fast drives in a RAID1 with this approach would result in all chunks being read from the first drive with READ_MIRROR set if both chunks are on the fast drives. This is where Timofey's queue length patch in [1] would become useful - in any case, that is an existing problem and probably deserving of an entirely new patch.

Quoted 5 lines
Quoted 1 line

[1]: https://patchwork.kernel.org/patch/10678531/#22315779

Thanks,
Anand.

Steve

author
#6 lore raw

PATCH v2 3/3 btrfs: read_mirror_policy ability to reset

In reply to Anand Jain

Adds an ability to change the read_mirror_policy at remount.

---
 fs/btrfs/super.c | 19 ++++++++++++++++---
 1 file changed, 16 insertions(+), 3 deletions(-)
fs/btrfs/super.c +16 -3
index 7ddecf4178a6..e70592036b95 100644
@@ -856,6 +856,12 @@ int btrfs_parse_options(struct btrfs_fs_info *info, char *options,
 			intarg = 0;
 			if (match_int(&args[0], &intarg) == 0) {
 				struct btrfs_device *device;
+				int clear = 0;
+
+				if (intarg < 0) {
+					clear = 1;
+					intarg = -intarg;
+				}
 
 				device = btrfs_find_device(info, intarg,
 							   NULL, NULL);
@@ -866,10 +872,17 @@ int btrfs_parse_options(struct btrfs_fs_info *info, char *options,
 					ret = -EINVAL;
 					goto out;
 				}
-				info->read_mirror_policy =
+
+				if (clear) {
+					info->read_mirror_policy = 0;
+					clear_bit(BTRFS_DEV_STATE_READ_MIRROR,
+						&device->dev_state);
+				} else {
+					info->read_mirror_policy =
 						BTRFS_READ_MIRROR_BY_DEV;
-				set_bit(BTRFS_DEV_STATE_READ_MIRROR,
-					&device->dev_state);
+					set_bit(BTRFS_DEV_STATE_READ_MIRROR,
+						  &device->dev_state);
+				}
 				break;
 			}
 
Signature · 1 line
2.7.0
#7 lore raw
In reply to Anand Jain
Quoted 2 lines · Anand Jain

Not yet ready for the integration. As I need to introduce -o no_read_mirror_policy instead of -o read_mirror_policy=-<devid>

Mount option is mostly likely not the right interface for setting such options, as usual.

author
#8 lore raw
Quoted 6 lines · David Sterba
Quoted 2 lines · Anand Jain

Not yet ready for the integration. As I need to introduce -o no_read_mirror_policy instead of -o read_mirror_policy=-<devid>

Mount option is mostly likely not the right interface for setting such options, as usual.

I am ok to make it ioctl for the final. What do you think?

But to reproduce the bug posted in
Btrfs: fix the corruption by reading stale btree blocks
It needs to be a mount option, as randomly the pid can still pick the disk specified in the mount option.

-Anand

Quoted 5 lines

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

#9 lore raw
Quoted 10 lines · Anand Jain
Quoted 6 lines · David Sterba
Quoted 2 lines · Anand Jain

Not yet ready for the integration. As I need to introduce -o no_read_mirror_policy instead of -o read_mirror_policy=-<devid>

Mount option is mostly likely not the right interface for setting such options, as usual.

Quoted 8 lines

 I am ok to make it ioctl for the final. What do you think?

 But to reproduce the bug posted in
   Btrfs: fix the corruption by reading stale btree blocks It needs to be a mount option, as randomly the pid can still pick the disk specified in the mount option.

Personally, I'd vote for filesystem property (thus handled through the standard btrfs property command) that can be overridden by a mount option. With that approach, no new tool (or change to an existing tool) would be needed, existing volumes could be converted to use it in a backwards compatible manner (old kernels would just ignore the property), and you could still have the behavior you want in tests (and in theory it could easily be adapted to be a per-subvolume setting if we ever get per-subvolume chunk profile support).

Of course, I'd actually like to see most of the mount options available as filesystem level properties with the option to override through mount options, but that's a lot more ambitious of an undertaking.

#10 lore raw
Quoted 19 lines · Austin S. Hemmelgarn
Quoted 10 lines · Anand Jain
Quoted 6 lines · David Sterba
Quoted 2 lines · Anand Jain

Not yet ready for the integration. As I need to introduce -o no_read_mirror_policy instead of -o read_mirror_policy=-<devid>

Mount option is mostly likely not the right interface for setting such options, as usual.

Quoted 8 lines

  I am ok to make it ioctl for the final. What do you think?

  But to reproduce the bug posted in
    Btrfs: fix the corruption by reading stale btree blocks It needs to be a mount option, as randomly the pid can still pick the disk specified in the mount option.

Quoted 8 lines

Personally, I'd vote for filesystem property (thus handled through the standard btrfs property command) that can be overridden by a mount option.  With that approach, no new tool (or change to an existing tool) would be needed, existing volumes could be converted to use it in a backwards compatible manner (old kernels would just ignore the property), and you could still have the behavior you want in tests (and in theory it could easily be adapted to be a per-subvolume setting if we ever get per-subvolume chunk profile support).

Properties are a combination of interfaces presented through a single command. Although the kernel API would allow a direct-to-property interface via the btrfs.* extended attributes, those are currently limited to a single inode. The label property is set via ioctl and stored in the superblock. The read-only subvolume property is also set by ioctl but stored in the root flags.

As it stands, every property is explicitly defined in the tools, so any addition would require tools changes. This is a bigger discussion, though. We *could* use the xattr interface to access per-root or fs-global properties, but we'd need to define that interface. btrfs_listxattr could get interesting, though I suppose we could simplify it by only allowing the per-subvolume and fs-global operations on root inodes.

-Jeff

Signature · 2 lines
Jeff Mahoney
SUSE Labs
author
#11 lore raw

Thanks Austin and Jeff for the suggestion.

I am not particularly a fan of mount option either mainly because those options aren't persistent and host independent luns will have tough time to have them synchronize manually.

Properties are better as it is persistent. And we can apply this read_mirror_policy property on the fsid object.

But if we are talking about the properties then it can be stored as extended attributes or ondisk key value pair, and I am doubt if ondisk key value pair will get a nod.
I can explore the extended attribute approach but appreciate more comments.

-Anand

Quoted 46 lines · Jeff Mahoney
Quoted 19 lines · Austin S. Hemmelgarn
Quoted 10 lines · Anand Jain
Quoted 6 lines · David Sterba
Quoted 2 lines · Anand Jain

Not yet ready for the integration. As I need to introduce -o no_read_mirror_policy instead of -o read_mirror_policy=-<devid>

Mount option is mostly likely not the right interface for setting such options, as usual.

Quoted 8 lines

  I am ok to make it ioctl for the final. What do you think?

  But to reproduce the bug posted in
    Btrfs: fix the corruption by reading stale btree blocks It needs to be a mount option, as randomly the pid can still pick the disk specified in the mount option.

Quoted 8 lines

Personally, I'd vote for filesystem property (thus handled through the standard btrfs property command) that can be overridden by a mount option.  With that approach, no new tool (or change to an existing tool) would be needed, existing volumes could be converted to use it in a backwards compatible manner (old kernels would just ignore the property), and you could still have the behavior you want in tests (and in theory it could easily be adapted to be a per-subvolume setting if we ever get per-subvolume chunk profile support).

Properties are a combination of interfaces presented through a single command. Although the kernel API would allow a direct-to-property interface via the btrfs.* extended attributes, those are currently limited to a single inode. The label property is set via ioctl and stored in the superblock. The read-only subvolume property is also set by ioctl but stored in the root flags.

As it stands, every property is explicitly defined in the tools, so any addition would require tools changes. This is a bigger discussion, though. We *could* use the xattr interface to access per-root or fs-global properties, but we'd need to define that interface. btrfs_listxattr could get interesting, though I suppose we could simplify it by only allowing the per-subvolume and fs-global operations on root inodes.

-Jeff

#12 lore raw
Quoted 8 lines · Anand Jain

Thanks Austin and Jeff for the suggestion.

I am not particularly a fan of mount option either mainly because those options aren't persistent and host independent luns will have tough time to have them synchronize manually.

Quoted 8 lines

Properties are better as it is persistent. And we can apply this read_mirror_policy property on the fsid object.

But if we are talking about the properties then it can be stored as extended attributes or ondisk key value pair, and I am doubt if ondisk key value pair will get a nod.
I can explore the extended attribute approach but appreciate more comments.

Hmm, thinking a bit further, might it be easier to just keep this as a mount option, and add something that lets you embed default mount options in the volume in a free-form manner? Then, you could set this persistently there, and could specify any others you want too. Doing that would also give very well defined behavior for exactly when changes would apply (the next time you mount or remount the volume), though handling of whether or not an option came from there or was specified on the command-line might be a bit complicated.

Quoted 49 lines · Jeff Mahoney
Quoted 46 lines · Austin S. Hemmelgarn
Quoted 19 lines · Anand Jain
Quoted 10 lines · David Sterba
Quoted 6 lines · Anand Jain
Quoted 2 lines

Not yet ready for the integration. As I need to introduce -o no_read_mirror_policy instead of -o read_mirror_policy=-<devid>

Mount option is mostly likely not the right interface for setting such options, as usual.

Quoted 8 lines

   I am ok to make it ioctl for the final. What do you think?

   But to reproduce the bug posted in
     Btrfs: fix the corruption by reading stale btree blocks It needs to be a mount option, as randomly the pid can still pick the disk specified in the mount option.

Quoted 8 lines

Personally, I'd vote for filesystem property (thus handled through the standard btrfs property command) that can be overridden by a mount option.  With that approach, no new tool (or change to an existing tool) would be needed, existing volumes could be converted to use it in a backwards compatible manner (old kernels would just ignore the property), and you could still have the behavior you want in tests (and in theory it could easily be adapted to be a per-subvolume setting if we ever get per-subvolume chunk profile support).

Properties are a combination of interfaces presented through a single command.  Although the kernel API would allow a direct-to-property interface via the btrfs.* extended attributes, those are currently limited to a single inode.  The label property is set via ioctl and stored in the superblock.  The read-only subvolume property is also set by ioctl but stored in the root flags.

As it stands, every property is explicitly defined in the tools, so any addition would require tools changes.  This is a bigger discussion, though.  We *could* use the xattr interface to access per-root or fs-global properties, but we'd need to define that interface. btrfs_listxattr could get interesting, though I suppose we could simplify it by only allowing the per-subvolume and fs-global operations on root inodes.

-Jeff

#13 lore raw
In reply to David Sterba
Quoted 6 lines · David Sterba
Quoted 2 lines · Anand Jain

Not yet ready for the integration. As I need to introduce -o no_read_mirror_policy instead of -o read_mirror_policy=-<devid>

Mount option is mostly likely not the right interface for setting such options, as usual.

I've seen a few alternate suggestions in the thread. I suppose the real question is: what and where is the intended persistence for this choice?

A mount option gets it via fstab. How would a user be expected to set it consistently via ioctl on each mount? Properties could work, but there's more discussion needed there. Personally, I like the property idea since it could conceivably be used on a per-file basis.

-Jeff

Signature · 2 lines
Jeff Mahoney
SUSE Labs
#14 lore raw
Quoted 9 lines · Jeff Mahoney
Quoted 6 lines · David Sterba
Quoted 2 lines · Anand Jain

Not yet ready for the integration. As I need to introduce -o no_read_mirror_policy instead of -o read_mirror_policy=-<devid>

Mount option is mostly likely not the right interface for setting such options, as usual.

Quoted 7 lines

I've seen a few alternate suggestions in the thread. I suppose the real question is: what and where is the intended persistence for this choice?

A mount option gets it via fstab. How would a user be expected to set it consistently via ioctl on each mount? Properties could work, but there's more discussion needed there. Personally, I like the property idea since it could conceivably be used on a per-file basis.

For the specific proposed use case (the tests), it probably doesn't need to be persistent beyond mount options.

However, this also allows for a trivial configuration using a slow storage device to provide redundancy for a fast storage device of the same size, which is potentially very useful for some people. In that case, I can see most people who would be using it wanting it to follow the filesystem regardless of what context it's being mounted in (for example, it shouldn't need an extra option if mounted from a recovery environment or if it's moved to another system).

Most of my reason for recommending properties is that filesystem level properties appear to be the best thing BTRFS has to store per-volume configuration that's supposed to stay with the volume, despite not really being used for that even though there are quite a few mount options that are logical candidates for this type of thing (for example, the ssd options, metadata_ratio, and max_inline all make more logical sense as a property of the volume, not the mount).