Skip to contentSkip to conversation

enforce read-only state at the block layer

PATCH In review Christoph Hellwig 3 participants 10 messages Jun 1, 2023 – Jun 7, 2023

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

Hi all,

I've recently got a report where a file system can write to a read-only block device, and while I've not found the root cause yet, it is very clear that we should not prevents writes to read-only at all.

This did in fact get fixed 5 years ago, but Linus reverted it as older lvm2 tools relying on this broken behavior. This series tries to restore it, although I'm still worried about thee older lvm2 tools to be honest. Question to the device mapper maintainers: is the any good way to work around that behavior in device mapper if needed instead of leaving the core block layer and drivers exposed?

Diffstat:

 block/blk-core.c       |   20 ++++++++------------
 include/linux/blkdev.h |    1 -
 2 files changed, 8 insertions(+), 13 deletions(-)
#1 lore raw

PATCH 1/3 block: remove a duplicate bdev_read_only declaration

bdev_read_only is already declared as in inline in blkdev.h, don't add another declaration after it.

---
 include/linux/blkdev.h | 1 -
 1 file changed, 1 deletion(-)
include/linux/blkdev.h +0 -1
index d89c2da1469872..20bbce92a91c11 100644
@@ -1455,7 +1455,6 @@ static inline void bio_end_io_acct(struct bio *bio, unsigned long start_time)
 	return bio_end_io_acct_remapped(bio, start_time, bio->bi_bdev);
 }
 
-int bdev_read_only(struct block_device *bdev);
 int set_blocksize(struct block_device *bdev, int size);
 
 int lookup_bdev(const char *pathname, dev_t *dev);
Signature · 1 line
2.39.2
#2 lore raw

PATCH 2/3 block: simplify the check for flushes in bio_check_ro

In reply to Christoph Hellwig

The only writes without no sectors are pure flush requests, so drop the extra op_is_flush check.

---
 block/blk-core.c | 5 ++---
 1 file changed, 2 insertions(+), 3 deletions(-)
block/blk-core.c +2 -3
index 2ae22bebeb3ee1..4ba243968e41eb 100644
@@ -494,9 +494,8 @@ late_initcall(fail_make_request_debugfs);
 
 static inline void bio_check_ro(struct bio *bio)
 {
-	if (op_is_write(bio_op(bio)) && bdev_read_only(bio->bi_bdev)) {
-		if (op_is_flush(bio->bi_opf) && !bio_sectors(bio))
-			return;
+	if (op_is_write(bio_op(bio)) && bio_sectors(bio) &&
+	    bdev_read_only(bio->bi_bdev)) {
 		pr_warn("Trying to write to read-only block-device %pg\n",
 			bio->bi_bdev);
 		/* Older lvm-tools actually trigger this */
Signature · 1 line
2.39.2
#3 lore raw

PATCH 3/3 block: fail writes to read-only devices

In reply to Christoph Hellwig

Currently callers can happily submit writes to block devices that are marked read-only, including to drivers that don't even support writes and will crash when fed such bios.

While bio submitter should check for read-only devices, that's not a very robust way of dealing with this.

Note that the last attempt to do this got reverted by Linus in commit a32e236eb93e ("Partially revert "block: fail op_is_write() requests to read-only partitions") because device mapper relyied on not enforcing the read-only state when used together with older lvm-tools.

The lvm side got fixed in:

    https://sourceware.org/git/?p=lvm2.git;a=commit;h=a6fdb9d9d70f51c49ad11a87ab4243344e6701a3

but if people still have older lvm2 tools in use we probably need to find a workaround for this in device mapper rather than lacking the core block layer checks.

---
 block/blk-core.c | 19 ++++++++-----------
 1 file changed, 8 insertions(+), 11 deletions(-)
block/blk-core.c +8 -11
index 4ba243968e41eb..ef41816bd0eade 100644
@@ -492,16 +492,6 @@ static int __init fail_make_request_debugfs(void)
 late_initcall(fail_make_request_debugfs);
 #endif /* CONFIG_FAIL_MAKE_REQUEST */
 
-static inline void bio_check_ro(struct bio *bio)
-{
-	if (op_is_write(bio_op(bio)) && bio_sectors(bio) &&
-	    bdev_read_only(bio->bi_bdev)) {
-		pr_warn("Trying to write to read-only block-device %pg\n",
-			bio->bi_bdev);
-		/* Older lvm-tools actually trigger this */
-	}
-}
-
 static noinline int should_fail_bio(struct bio *bio)
 {
 	if (should_fail_request(bdev_whole(bio->bi_bdev), bio->bi_iter.bi_size))
@@ -735,7 +725,14 @@ void submit_bio_noacct(struct bio *bio)
 
 	if (should_fail_bio(bio))
 		goto end_io;
-	bio_check_ro(bio);
+
+	if (op_is_write(bio_op(bio)) && bio_sectors(bio) &&
+	    bdev_read_only(bdev)) {
+		pr_warn("Trying to write to read-only block-device %pg\n",
+			bdev);
+		goto end_io;
+	}
+
 	if (!bio_flagged(bio, BIO_REMAPPED)) {
 		if (unlikely(bio_check_eod(bio)))
 			goto end_io;
Signature · 1 line
2.39.2
#4 lore raw

Re: [PATCH 3/3] block: fail writes to read-only devices

Quoted 5 lines · Christoph Hellwig

Note that the last attempt to do this got reverted by Linus in commit a32e236eb93e ("Partially revert "block: fail op_is_write() requests to read-only partitions") because device mapper relyied on not enforcing the read-only state when used together with older lvm-tools.

I'm certainly happy to try again, but I have my doubts whether this will work.

Note that the problem case is for a device that *used* to be writable, but became read-only later, and there's an existing writer that needs that writability.

The lvm fix was not to stop writing, but to simply not mark it read-only too early.

So I do think that the problem here is purely in the block layer.

The logic wrt "bdev_read_only()" is not necessarily a "right now it's read-only", but more of a thing that should be checked purely when the device is opened. Which is pretty much exactly what we do.

So honestly, that whole test for

+ if (op_is_write(bio_op(bio)) && bio_sectors(bio) &&
+ bdev_read_only(bdev)) {

may look "obviously correct", but it's also equally valid to view it as "obviously garbage", simply because the test is being done at the wrong point.

The same way you can write to a file that was opened for writing, but has then become read-only afterwards, writing to a device with a bdev that was writable when you *started* writing is not at all necessarily wrong.

So please at least consider the possibility that that test - while it looks obvious - is simply buggy.

Linus

#5 lore raw

Re: [PATCH 3/3] block: fail writes to read-only devices

[quoting your reply out of order, because I think it makes sense that
way]

Quoted 5 lines · Linus Torvalds

So honestly, that whole test for

block/blk-core.c +2 -0
+       if (op_is_write(bio_op(bio)) && bio_sectors(bio) &&
+           bdev_read_only(bdev)) {
Quoted 8 lines

may look "obviously correct", but it's also equally valid to view it as "obviously garbage", simply because the test is being done at the wrong point.

The same way you can write to a file that was opened for writing, but has then become read-only afterwards, writing to a device with a bdev that was writable when you *started* writing is not at all necessarily wrong.

files, or more specifically file descriptors really are the wrong analogy here. A file descriptor allows you to keep writing to a file that you were allowed to write to at open time. And that's fine (at least most of the time, people keep wanting a revoke and keep implementing broken special cases of it, but I disgress).

The struct block_device is not such a handle, it's the underlying object. And the equivalent here is that we allow writes to inodes that don't even implement a write method, or have the immutable bit set.

Quoted 3 lines

The logic wrt "bdev_read_only()" is not necessarily a "right now it's read-only", but more of a thing that should be checked purely when the device is opened. Which is pretty much exactly what we do.

Except the whole make a thing readonly just for fun is the corner case. DM does it, and we have a sysfs file to allow it. But the usual case is that a block device has been read-only all the time, or has been force to be read-only by the actual storage device, which doesn't know anything about the file descriptor model, and will not be happy.

So maybe a lazy read-only after the last writer goes away would be nice (not that we actully track writers right now, but that whole area is comletely fucked up and I'm looking into fixing it at the moment).

And for extra fun blkdev_get_by_dev doesn't check for read-only because we've historically allowed to open writable file descriptors on read-only block devices for ioctls (in addition to the magic (flags & O_ACCMODE) == 3 mode just ioctl).

#6 lore raw

Re: [PATCH 3/3] block: fail writes to read-only devices

Quoted 7 lines · Christoph Hellwig

Except the whole make a thing readonly just for fun is the corner case. DM does it, and we have a sysfs file to allow it. But the usual case is that a block device has been read-only all the time, or has been force to be read-only by the actual storage device, which doesn't know anything about the file descriptor model, and will not be happy.

The "it's always been read-only" case is unambiguous.

So I do wonder if we should have two read-only bits: a "hard" and "soft" bit, and make the open-time one the hard bit.

Anyway, I'm ok trying this simple thing once more, but if we end up getting reports of breakage again, I think you may just need to accept the fact that "somebody turned the device read-only after the fact" may just be something the kernel will have to continue to deal with.

We might be able to squirrell the "read-only at time of open" bit away in the file descriptor in the FMODE_CAN_WRITE bit or something like that (although that would gives the wrong error for write(): -EINVAL instead of -EROFS or whatever)

Linus

maintainer Reviewed-by on v1 3/3
#7 lore raw

Re: [PATCH 3/3] block: fail writes to read-only devices

In reply to Christoph Hellwig
Quoted 14 lines · Christoph Hellwig

Currently callers can happily submit writes to block devices that are marked read-only, including to drivers that don't even support writes and will crash when fed such bios.

While bio submitter should check for read-only devices, that's not a very robust way of dealing with this.

Note that the last attempt to do this got reverted by Linus in commit a32e236eb93e ("Partially revert "block: fail op_is_write() requests to read-only partitions") because device mapper relyied on not enforcing the read-only state when used together with older lvm-tools.

The lvm side got fixed in:

Quoted 7 lines
    https://sourceware.org/git/?p=lvm2.git;a=commit;h=a6fdb9d9d70f51c49ad11a87ab4243344e6701a3

but if people still have older lvm2 tools in use we probably need to find a workaround for this in device mapper rather than lacking the core block layer checks.

--
dm-devel mailing list
[email protected]
https://listman.redhat.com/mailman/listinfo/dm-devel

maintainer
#8 lore raw
In reply to Christoph Hellwig
Quoted 6 lines · Christoph Hellwig

Hi all,

I've recently got a report where a file system can write to a read-only block device, and while I've not found the root cause yet, it is very clear that we should not prevents writes to read-only at all.

Quoted 6 lines

This did in fact get fixed 5 years ago, but Linus reverted it as older lvm2 tools relying on this broken behavior. This series tries to restore it, although I'm still worried about thee older lvm2 tools to be honest. Question to the device mapper maintainers: is the any good way to work around that behavior in device mapper if needed instead of leaving the core block layer and drivers exposed?

Given the block core change (in patch 3) _and_ old lvm2 code: it'll obviously fail.

Not sure of a crafty hack to workaround. Hopefully 5 year old lvm2 remains tightly coupled to kernels of the same vintage and we get lucky moving forward.

So I agree with Linus, worth trying this simple change again and seeing if there is fallout. Revert/worry about it again as needed.

Mike

--
dm-devel mailing list
[email protected]
https://listman.redhat.com/mailman/listinfo/dm-devel

#9 lore raw
Quoted 6 lines

Not sure of a crafty hack to workaround. Hopefully 5 year old lvm2 remains tightly coupled to kernels of the same vintage and we get lucky moving forward.

So I agree with Linus, worth trying this simple change again and seeing if there is fallout. Revert/worry about it again as needed.

I'm actually looking into implementing Linus' suggestion now and track in the block_device if it has ever been opened for writing.