Skip to contentSkip to conversation

block, dm: restack queue_limits

LSF/MM TOPIC Mike Snitzer 4 participants 4 messages Jan 29, 2018 – Feb 2, 2018

Subsystems (3)
SCSI Block layer Device mapper
lore
Share & tools
lore

Mike Snitzer

author
#0 lore raw

We currently don't restack the queue_limits if the lowest, or intermediate, layer of an IO stack changes.

This is particularly unfortunate in the case of FLUSH/FUA which may change if/when a HW controller's BBU fails; whereby requiring the device advertise that it has a volatile write cache (WCE=1).

But in the context of DM, really it'd be best if the entire stack of devices had their limits restacked if any underlying layer's limits change.

In the past, Martin and I discussed that we should "just do it" but never did. Not sure we need a lengthy discussion but figured I'd put it out there.

Maybe I'll find time, between now and April, to try implementing it.

Thanks,
Mike

#1 lore raw
Quoted 7 lines · Mike Snitzer

We currently don't restack the queue_limits if the lowest, or intermediate, layer of an IO stack changes.

This is particularly unfortunate in the case of FLUSH/FUA which may change if/when a HW controller's BBU fails; whereby requiring the device advertise that it has a volatile write cache (WCE=1).

Uh-oh. Device rescan.
Would be a valid topic on its own...

Quoted 10 lines

But in the context of DM, really it'd be best if the entire stack of devices had their limits restacked if any underlying layer's limits change.

In the past, Martin and I discussed that we should "just do it" but never did. Not sure we need a lengthy discussion but figured I'd put it out there.

Maybe I'll find time, between now and April, to try implementing it.

For SCSI the device capabilities are pretty much set in stone after the initial scan; there are hooks for rescanning, but they will only work half of the time.
Plus we can't really change the device type on the fly (eg if the SCSI device type changes; if it moves away from '0' we would need to unbind the sd driver, and if it moves to '0' we'll need to rescan the sd device. None of this is happening right now.)

So I'd be glad to have a discussion around this.

Cheers,

Hannes

Signature · 5 lines
Dr. Hannes Reinecke		   Teamlead Storage & Networking
[email protected]			               +49 911 74053 688
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg
GF: F. Imendörffer, J. Smithard, J. Guild, D. Upmanyu, G. Norton
HRB 21284 (AG Nürnberg)
#2 lore raw
Quoted 29 lines · Hannes Reinecke
Quoted 7 lines · Mike Snitzer

We currently don't restack the queue_limits if the lowest, or intermediate, layer of an IO stack changes.

This is particularly unfortunate in the case of FLUSH/FUA which may change if/when a HW controller's BBU fails; whereby requiring the device advertise that it has a volatile write cache (WCE=1).

Uh-oh. Device rescan.
Would be a valid topic on its own...

Quoted 10 lines

But in the context of DM, really it'd be best if the entire stack of devices had their limits restacked if any underlying layer's limits change.

In the past, Martin and I discussed that we should "just do it" but never did. Not sure we need a lengthy discussion but figured I'd put it out there.

Maybe I'll find time, between now and April, to try implementing it.

For SCSI the device capabilities are pretty much set in stone after the initial scan; there are hooks for rescanning, but they will only work half of the time.
Plus we can't really change the device type on the fly (eg if the SCSI device type changes; if it moves away from '0' we would need to unbind the sd driver, and if it moves to '0' we'll need to rescan the sd device. None of this is happening right now.)

Quoted 1 line

So I'd be glad to have a discussion around this.

At least array vendor has also desired the ability to change various attributes of the device after the initial scan, such as the model name. Not sure what would break if we did this, since who knows what userspace software might be caching this info, but...

-Ewan

#3 lore raw
In reply to Mike Snitzer
Quoted 7 lines · Mike Snitzer

We currently don't restack the queue_limits if the lowest, or intermediate, layer of an IO stack changes.

This is particularly unfortunate in the case of FLUSH/FUA which may change if/when a HW controller's BBU fails; whereby requiring the device advertise that it has a volatile write cache (WCE=1).

Quoted 7 lines

But in the context of DM, really it'd be best if the entire stack of devices had their limits restacked if any underlying layer's limits change.

In the past, Martin and I discussed that we should "just do it" but never did. Not sure we need a lengthy discussion but figured I'd put it out there.

So much "yes"!!
Just a notifier chain would probably do.

I would like the notification to support changing the size of the device too.
I see this as being two-stage.
1/ I'm going to change the device size to X - are you all OK with that? 2/ Device size is now X.

That allows md and dm to check that filesystems aren't going to get mad when devices are made smaller, and can adjust (if they want to) when devices get bigger.

Thanks,
NeilBrown

Quoted 10 lines

Maybe I'll find time, between now and April, to try implementing it.

Thanks,
Mike

--
dm-devel mailing list
[email protected]
https://www.redhat.com/mailman/listinfo/dm-devel