Skip to contentSkip to conversation

Add suspended state check for target messages

PATCH In review Mike Anderson 5 participants 12 messages Nov 13, 2009 – Nov 16, 2009 Device mapper

lore
Share & tools
lore
#0 lore raw

cover Add suspended state check for target messages

This patch adds a accessor function that allows determining if a mapped_device is in the suspended state and then adds this check to the multipath targets multipath_message function.

As previously described in the email at the archive url provided below the target_message ioctl can call into the target after suspend completes. This can cause new work to be queued and actions to be taken by the target that should be prevented in the suspended state.

http://permalink.gmane.org/gmane.linux.kernel.device-mapper.devel/10486

---

Mike Anderson (2):
dm: Add accessor dm_table_md_suspended
dm: Add suspended check to multipath_message

 drivers/md/dm-mpath.c         |    3 +++
 drivers/md/dm-table.c         |    6 ++++++
 include/linux/device-mapper.h |    1 +
 3 files changed, 10 insertions(+), 0 deletions(-)

--
-andmike

Michael Anderson <[email protected]>

#1 lore raw

PATCH 1/2 dm: Add accessor dm_table_md_suspended

Add a dm_table accessor function to check if md is suspended.

---
 drivers/md/dm-table.c         |    6 ++++++
 include/linux/device-mapper.h |    1 +
 2 files changed, 7 insertions(+), 0 deletions(-)
drivers/md/dm-table.c +6 -0
index 1a6cb3c..6616598 100644
@@ -1245,6 +1245,11 @@ struct mapped_device *dm_table_get_md(struct dm_table *t)
 	return t->md;
 }
 
+int dm_table_md_suspended(struct dm_table *t)
+{
+	return dm_suspended(t->md);
+}
+
 EXPORT_SYMBOL(dm_vcalloc);
 EXPORT_SYMBOL(dm_get_device);
 EXPORT_SYMBOL(dm_put_device);
@@ -1255,3 +1260,4 @@ EXPORT_SYMBOL(dm_table_get_md);
 EXPORT_SYMBOL(dm_table_put);
 EXPORT_SYMBOL(dm_table_get);
 EXPORT_SYMBOL(dm_table_unplug_all);
+EXPORT_SYMBOL(dm_table_md_suspended);
include/linux/device-mapper.h +1 -0
index df7607e..3440c54 100644
@@ -287,6 +287,7 @@ sector_t dm_table_get_size(struct dm_table *t);
 unsigned int dm_table_get_num_targets(struct dm_table *t);
 fmode_t dm_table_get_mode(struct dm_table *t);
 struct mapped_device *dm_table_get_md(struct dm_table *t);
+int dm_table_md_suspended(struct dm_table *t);
 
 /*
  * Trigger an event.
Acked-by on v1 1/2
#2 lore raw

Re: [PATCH 1/2] dm: Add accessor dm_table_md_suspended

Quoted 3 lines · Mike Anderson

Add a dm_table accessor function to check if md is suspended.

I was thinking of doing then same because I have a need to test if a table is suspended in the snapshot target. I stopped short of doing so but it is clear that it is better to put this generic DM export in place rather than have each target add their own 'suspended' flag in presuspend (or some equivalent).

#3 lore raw

PATCH 2/2 dm: Add suspended check to multipath_message

In reply to Mike Anderson

Add suspended check to multipath_message

---
 drivers/md/dm-mpath.c |    3 +++
 1 files changed, 3 insertions(+), 0 deletions(-)
drivers/md/dm-mpath.c +3 -0
index 32d0b87..ee56ef7 100644
@@ -1402,6 +1402,9 @@ static int multipath_message(struct dm_target *ti, unsigned argc, char **argv)
 	struct multipath *m = (struct multipath *) ti->private;
 	action_fn action;
 
+	if (dm_table_md_suspended(ti->table))
+		return -EBUSY;
+
 	if (argc == 1) {
 		if (!strnicmp(argv[0], MESG_STR("queue_if_no_path")))
 			return queue_if_no_path(m, 1, 0);
#4 lore raw

Re: [PATCH 2/2] dm: Add suspended check to multipath_message

Quoted 12 lines · Mike Anderson [[email protected]] wrote:

Add suspended check to multipath_message

---
 drivers/md/dm-mpath.c |    3 +++
 1 files changed, 3 insertions(+), 0 deletions(-)
drivers/md/dm-mpath.c +0 -0
index 32d0b87..ee56ef7 100644

drivers/md/dm-mpath.c static int multipath_message(struct dm_target *ti, unsigned argc, char **argv) · line 1406

Quoted 5 lines
drivers/md/dm-mpath.c +2 -0
@@ -1402,6 +1402,9 @@ static int multipath_message(struct dm_target *ti, unsigned argc, char **argv)
 	struct multipath *m = (struct multipath *) ti->private;
 	action_fn action;
 
+	if (dm_table_md_suspended(ti->table))
+		return -EBUSY;

Same question as DELETING. SUSPEND bit could be set just after the above check. Is it OK if the following code executes after the SUSPEND bit is set???

Quoted 9 lines
drivers/md/dm-mpath.c +1 -1
+
 	if (argc == 1) {
 		if (!strnicmp(argv[0], MESG_STR("queue_if_no_path")))
 			return queue_if_no_path(m, 1, 0);
 
--

dm-devel mailing list
[email protected]
https://www.redhat.com/mailman/listinfo/dm-devel

#5 lore raw
In reply to Mike Anderson

Hi Mike,

Quoted 9 lines · Mike Anderson

This patch adds a accessor function that allows determining if a mapped_device is in the suspended state and then adds this check to the multipath targets multipath_message function.

As previously described in the email at the archive url provided below the target_message ioctl can call into the target after suspend completes. This can cause new work to be queued and actions to be taken by the target that should be prevented in the suspended state.

Quoted 7 lines

http://permalink.gmane.org/gmane.linux.kernel.device-mapper.devel/10486

---

Mike Anderson (2):
dm: Add accessor dm_table_md_suspended
dm: Add suspended check to multipath_message

Thank you for the patch.
But this patch-set doesn't solve the race problem.

Multipath must start rejecting message ioctl *before* flushing workqueues in postsuspend.
(*) With my patch, all multipath works are flushed in postsuspend.

      http://patchwork.kernel.org/patch/59556/

However, your patch starts rejecting message ioctl *after* postsuspend. So there is a small window below:
dm_suspend() multipath_message()
------------------------------------------------------------------
dm_table_post_suspend()

    -> flush_multipath_works()
                                      if (!test_bit(DMF_SUSPENDED))
                                          queue_work()
  set_bit(DMF_SUSPENDED)
  ...
  dm_swap_table()
                                      [de]activate_path()

Thanks,
Kiyoshi Ueda

#7 lore raw
Quoted 1 line · Alasdair G Kergon [[email protected]] wrote:

Would DMF_BLOCK_IO_FOR_SUSPEND work instead?

Should, provided that multipath_message() and multipath_postsuspend() use some lock to synchronize. Without the lock, multipath_postsuspend() could be executed between multipath_message's bit flag check and its action().

multipath_message()

{
	lock();
	if (DMF_BLOCK_IO_FOR_SUSPEND is set) {
		unlock and fail;
	} else { /* postsuspend has not been executed yet and won't be
	completed until we are done here */
		action();
	}
	unlock();
}

multipath_postsuspend()

{
	lock();
	flush work();
	unlock();
}
#8 lore raw
In reply to Alasdair G Kergon
Quoted 2 lines · Alasdair G Kergon

Would DMF_BLOCK_IO_FOR_SUSPEND work instead?

DMF_BLOCK_IO_FOR_SUSPEND appears to be a better choice, but since other targets like dm-raid already have a internal suspend bit being used unclear on the value of creating a generic accessor vs. just adding a suspend atomic to the multipath target.

To close another window I believe we need a form of synchronization to ensure a current multipath_message has completed prior to multipath_postsuspend calling flush_multipath_work. Later multipath_message calls would return without adding work if a bit check indicated that we where suspended.

I will redo the patch(s) and resubmit.

-andmike
--
Michael Anderson
[email protected]

#9 lore raw
In reply to Alasdair G Kergon

Hi Alasdair,

Quoted 1 line · Alasdair G Kergon

Would DMF_BLOCK_IO_FOR_SUSPEND work instead?

Maybe.
But we must remember that using the flag means that multipath can't use message during flushing I/Os, although I'm not sure how it hurts us.

If we'd like to keep the ability to use message during flushing I/Os, we should use another flag for this purpose and set it after dm_wait_for_completion().

Thanks,
Kiyoshi Ueda

#10 lore raw
Quoted 5 lines · Kiyoshi Ueda

Hi Alasdair,

Quoted 1 line · Alasdair G Kergon

Would DMF_BLOCK_IO_FOR_SUSPEND work instead?

Quoted 8 lines

Maybe.
But we must remember that using the flag means that multipath can't use message during flushing I/Os, although I'm not sure how it hurts us.

If we'd like to keep the ability to use message during flushing I/Os, we should use another flag for this purpose and set it after dm_wait_for_completion().

I just re-rolled / re-submitted a update series that use a multipath internal flag for the suspended state plus a mutex. This would avoid a extended meaning for DMF_BLOCK_IO_FOR_SUSPEND if we needed to avoid that case.

-andmike
--
Michael Anderson
[email protected]

#11 lore raw
In reply to Kiyoshi Ueda
Quoted 5 lines · Kiyoshi Ueda
Quoted 1 line · Alasdair G Kergon

Would DMF_BLOCK_IO_FOR_SUSPEND work instead?

If we'd like to keep the ability to use message during flushing I/Os, we should use another flag for this purpose and set it after dm_wait_for_completion().

We should rename the flag as it's original meaning has changed:-) FOR_SUSPEND_OR_FLUSH

Alasdair