Skip to contentSkip to conversation

iomap: Fix WARN_ON_ONCE() from unprivileged users

PATCH Superseded Qian Cai 3 participants 5 messages Aug 31, 2020

Subsystems (2)

v2 of 4 Superseded Latest is v4 →

lore
Share & tools
lore

Qian Cai

author
#0 lore raw

PATCH v2 1/1 iomap: Fix WARN_ON_ONCE() from unprivileged users

It is trivial to trigger a WARN_ON_ONCE(1) in iomap_dio_actor() by unprivileged users which would taint the kernel, or worse - panic if panic_on_warn or panic_on_taint is set. Hence, just convert it to pr_warn_ratelimited() to let users know their workloads are racing. Thanks Dave Chinner for the initial analysis of the racing reproducers.

---

v2: Record the path, pid and command as well.

 fs/iomap/direct-io.c | 17 ++++++++++++++++-
 1 file changed, 16 insertions(+), 1 deletion(-)
fs/iomap/direct-io.c +16 -1
index c1aafb2ab990..66a4502ef675 100644
@@ -374,6 +374,7 @@ iomap_dio_actor(struct inode *inode, loff_t pos, loff_t length,
 		void *data, struct iomap *iomap, struct iomap *srcmap)
 {
 	struct iomap_dio *dio = data;
+	char pathname[128], *path;
 
 	switch (iomap->type) {
 	case IOMAP_HOLE:
@@ -389,7 +390,21 @@ iomap_dio_actor(struct inode *inode, loff_t pos, loff_t length,
 	case IOMAP_INLINE:
 		return iomap_dio_inline_actor(inode, pos, length, dio, iomap);
 	default:
-		WARN_ON_ONCE(1);
+		/*
+		 * DIO is not serialised against mmap() access at all, and so
+		 * if the page_mkwrite occurs between the writeback and the
+		 * iomap_apply() call in the DIO path, then it will see the
+		 * DELALLOC block that the page-mkwrite allocated.
+		 */
+		path = file_path(dio->iocb->ki_filp, pathname,
+				 sizeof(pathname));
+		if (IS_ERR(path))
+			path = "(unknown)";
+
+		pr_warn_ratelimited("page_mkwrite() is racing with DIO read (iomap->type = %u).\n"
+				    "File: %s PID: %d Comm: %.20s\n",
+				    iomap->type, path, current->pid,
+				    current->comm);
 		return -EIO;
 	}
 }
Signature · 1 line
2.18.4
#1 lore raw
Quoted 27 lines · Qian Cai

It is trivial to trigger a WARN_ON_ONCE(1) in iomap_dio_actor() by unprivileged users which would taint the kernel, or worse - panic if panic_on_warn or panic_on_taint is set. Hence, just convert it to pr_warn_ratelimited() to let users know their workloads are racing. Thanks Dave Chinner for the initial analysis of the racing reproducers.

---

v2: Record the path, pid and command as well.

 fs/iomap/direct-io.c | 17 ++++++++++++++++-
 1 file changed, 16 insertions(+), 1 deletion(-)
fs/iomap/direct-io.c +1 -0
index c1aafb2ab990..66a4502ef675 100644
@@ -374,6 +374,7 @@ iomap_dio_actor(struct inode *inode, loff_t pos, loff_t length,
 		void *data, struct iomap *iomap, struct iomap *srcmap)
 {
 	struct iomap_dio *dio = data;
+	char pathname[128], *path;
 
 	switch (iomap->type) {
 	case IOMAP_HOLE:

fs/iomap/direct-io.c iomap_dio_actor(struct inode *inode, loff_t pos, loff_t length, · line 392

Quoted 4 lines
fs/iomap/direct-io.c +0 -1
@@ -389,7 +390,21 @@ iomap_dio_actor(struct inode *inode, loff_t pos, loff_t length,
 	case IOMAP_INLINE:
 		return iomap_dio_inline_actor(inode, pos, length, dio, iomap);
 	default:
-		WARN_ON_ONCE(1);

It seems like we should explicitly catch IOMAP_DELALLOC for this case, and leave the
default: as a WARN_ON that is not user-triggerable? i.e.

case IOMAP_DELALLOC:

	<all the fancy warnings>
	return -EIO;
default:
	WARN_ON_ONCE(1);
	return -EIO;
Quoted 5 lines
fs/iomap/direct-io.c +5 -0
+		/*
+		 * DIO is not serialised against mmap() access at all, and so
+		 * if the page_mkwrite occurs between the writeback and the
+		 * iomap_apply() call in the DIO path, then it will see the
+		 * DELALLOC block that the page-mkwrite allocated.

fs/iomap/direct-io.c iomap_dio_actor(struct inode *inode, loff_t pos, loff_t length, · line 407

Quoted 10 lines
fs/iomap/direct-io.c +10 -0
+		 */
+		path = file_path(dio->iocb->ki_filp, pathname,
+				 sizeof(pathname));
+		if (IS_ERR(path))
+			path = "(unknown)";
+
+		pr_warn_ratelimited("page_mkwrite() is racing with DIO read (iomap->type = %u).\n"
+				    "File: %s PID: %d Comm: %.20s\n",
+				    iomap->type, path, current->pid,
+				    current->comm);

This is very specific ...

Do we know that mmap/page_mkwrite is (and will always be) the only way to reach this point?

It seems to me that this message won't be very useful for the admin; "pg_mkwrite" may mean something to us, but doubtful for the general public. And "type = 1" won't mean much to the reader, either.

Maybe something like:

"DIO encountered delayed allocation block, racing buffered+direct? File: %s Comm: %.20s\n"

It just seems that a user-facing warning should be something the admin has a chance of acting on without needing to file a bug for analysis by the developers.

(though TBH "delayed allocation" probably doesn't mean much to the admin, either)

-Eric

Quoted 4 lines
fs/iomap/direct-io.c +0 -0
 		return -EIO;
 	}
 }
#2 lore raw
Quoted 68 lines · Eric Sandeen
Quoted 27 lines · Qian Cai

It is trivial to trigger a WARN_ON_ONCE(1) in iomap_dio_actor() by unprivileged users which would taint the kernel, or worse - panic if panic_on_warn or panic_on_taint is set. Hence, just convert it to pr_warn_ratelimited() to let users know their workloads are racing. Thanks Dave Chinner for the initial analysis of the racing reproducers.

---

v2: Record the path, pid and command as well.

 fs/iomap/direct-io.c | 17 ++++++++++++++++-
 1 file changed, 16 insertions(+), 1 deletion(-)
fs/iomap/direct-io.c +1 -0
index c1aafb2ab990..66a4502ef675 100644
@@ -374,6 +374,7 @@ iomap_dio_actor(struct inode *inode, loff_t pos, loff_t length,
 		void *data, struct iomap *iomap, struct iomap *srcmap)
 {
 	struct iomap_dio *dio = data;
+	char pathname[128], *path;
 
 	switch (iomap->type) {
 	case IOMAP_HOLE:

fs/iomap/direct-io.c iomap_dio_actor(struct inode *inode, loff_t pos, loff_t length, · line 392

Quoted 4 lines
fs/iomap/direct-io.c +0 -1
@@ -389,7 +390,21 @@ iomap_dio_actor(struct inode *inode, loff_t pos, loff_t length,
 	case IOMAP_INLINE:
 		return iomap_dio_inline_actor(inode, pos, length, dio, iomap);
 	default:
-		WARN_ON_ONCE(1);

It seems like we should explicitly catch IOMAP_DELALLOC for this case, and leave the
default: as a WARN_ON that is not user-triggerable? i.e.

case IOMAP_DELALLOC:

	<all the fancy warnings>
	return -EIO;
default:
	WARN_ON_ONCE(1);
	return -EIO;
Quoted 5 lines
fs/iomap/direct-io.c +5 -0
+		/*
+		 * DIO is not serialised against mmap() access at all, and so
+		 * if the page_mkwrite occurs between the writeback and the
+		 * iomap_apply() call in the DIO path, then it will see the
+		 * DELALLOC block that the page-mkwrite allocated.

fs/iomap/direct-io.c iomap_dio_actor(struct inode *inode, loff_t pos, loff_t length, · line 407

Quoted 10 lines
fs/iomap/direct-io.c +10 -0
+		 */
+		path = file_path(dio->iocb->ki_filp, pathname,
+				 sizeof(pathname));
+		if (IS_ERR(path))
+			path = "(unknown)";
+
+		pr_warn_ratelimited("page_mkwrite() is racing with DIO read (iomap->type = %u).\n"
+				    "File: %s PID: %d Comm: %.20s\n",
+				    iomap->type, path, current->pid,
+				    current->comm);

This is very specific ...

Do we know that mmap/page_mkwrite is (and will always be) the only way to reach this point?

It seems to me that this message won't be very useful for the admin; "pg_mkwrite" may mean something to us, but doubtful for the general public. And "type = 1" won't mean much to the reader, either.

Quoted 8 lines

Maybe something like:

"DIO encountered delayed allocation block, racing buffered+direct? File: %s Comm: %.20s\n"

It just seems that a user-facing warning should be something the admin has a chance of acting on without needing to file a bug for analysis by the developers.

(though TBH "delayed allocation" probably doesn't mean much to the admin, either)

/me suggests

"Direct I/O collision with buffered write! File: %s..."?

I concede that we ought to leave the nastier WARN for the default case since there are no other IOMAP_ types and so any other code is a sign of a serious screwup.

--D

Quoted 7 lines

-Eric

Quoted 4 lines
fs/iomap/direct-io.c +0 -0
 		return -EIO;
 	}
 }
#3 lore raw
Quoted 73 lines · Darrick J. Wong
Quoted 72 lines · Eric Sandeen
Quoted 27 lines · Qian Cai

It is trivial to trigger a WARN_ON_ONCE(1) in iomap_dio_actor() by unprivileged users which would taint the kernel, or worse - panic if panic_on_warn or panic_on_taint is set. Hence, just convert it to pr_warn_ratelimited() to let users know their workloads are racing. Thanks Dave Chinner for the initial analysis of the racing reproducers.

---

v2: Record the path, pid and command as well.

 fs/iomap/direct-io.c | 17 ++++++++++++++++-
 1 file changed, 16 insertions(+), 1 deletion(-)
fs/iomap/direct-io.c +1 -0
index c1aafb2ab990..66a4502ef675 100644
@@ -374,6 +374,7 @@ iomap_dio_actor(struct inode *inode, loff_t pos, loff_t length,
 		void *data, struct iomap *iomap, struct iomap *srcmap)
 {
 	struct iomap_dio *dio = data;
+	char pathname[128], *path;
 
 	switch (iomap->type) {
 	case IOMAP_HOLE:

fs/iomap/direct-io.c iomap_dio_actor(struct inode *inode, loff_t pos, loff_t length, · line 392

Quoted 4 lines
fs/iomap/direct-io.c +0 -1
@@ -389,7 +390,21 @@ iomap_dio_actor(struct inode *inode, loff_t pos, loff_t length,
 	case IOMAP_INLINE:
 		return iomap_dio_inline_actor(inode, pos, length, dio, iomap);
 	default:
-		WARN_ON_ONCE(1);

It seems like we should explicitly catch IOMAP_DELALLOC for this case, and leave the
default: as a WARN_ON that is not user-triggerable? i.e.

case IOMAP_DELALLOC:

	<all the fancy warnings>
	return -EIO;
default:
	WARN_ON_ONCE(1);
	return -EIO;
Quoted 5 lines
fs/iomap/direct-io.c +5 -0
+		/*
+		 * DIO is not serialised against mmap() access at all, and so
+		 * if the page_mkwrite occurs between the writeback and the
+		 * iomap_apply() call in the DIO path, then it will see the
+		 * DELALLOC block that the page-mkwrite allocated.

fs/iomap/direct-io.c iomap_dio_actor(struct inode *inode, loff_t pos, loff_t length, · line 407

Quoted 10 lines
fs/iomap/direct-io.c +10 -0
+		 */
+		path = file_path(dio->iocb->ki_filp, pathname,
+				 sizeof(pathname));
+		if (IS_ERR(path))
+			path = "(unknown)";
+
+		pr_warn_ratelimited("page_mkwrite() is racing with DIO read (iomap->type = %u).\n"
+				    "File: %s PID: %d Comm: %.20s\n",
+				    iomap->type, path, current->pid,
+				    current->comm);

This is very specific ...

Do we know that mmap/page_mkwrite is (and will always be) the only way to reach this point?

It seems to me that this message won't be very useful for the admin; "pg_mkwrite" may mean something to us, but doubtful for the general public. And "type = 1" won't mean much to the reader, either.

Maybe something like:

"DIO encountered delayed allocation block, racing buffered+direct? File: %s Comm: %.20s\n"

Quoted 8 lines
Quoted 4 lines

It just seems that a user-facing warning should be something the admin has a chance of acting on without needing to file a bug for analysis by the developers.

(though TBH "delayed allocation" probably doesn't mean much to the admin, either)

/me suggests

"Direct I/O collision with buffered write! File: %s..."?

Sure, that sounds good to me. Terser is better.

Quoted 3 lines

I concede that we ought to leave the nastier WARN for the default case since there are no other IOMAP_ types and so any other code is a sign of a serious screwup.

*nod* thanks.

-Eric

author
#4 lore raw
In reply to Eric Sandeen
Quoted 61 lines · Eric Sandeen
Quoted 27 lines · Qian Cai

It is trivial to trigger a WARN_ON_ONCE(1) in iomap_dio_actor() by unprivileged users which would taint the kernel, or worse - panic if panic_on_warn or panic_on_taint is set. Hence, just convert it to pr_warn_ratelimited() to let users know their workloads are racing. Thanks Dave Chinner for the initial analysis of the racing reproducers.

---

v2: Record the path, pid and command as well.

 fs/iomap/direct-io.c | 17 ++++++++++++++++-
 1 file changed, 16 insertions(+), 1 deletion(-)
fs/iomap/direct-io.c +1 -0
index c1aafb2ab990..66a4502ef675 100644
@@ -374,6 +374,7 @@ iomap_dio_actor(struct inode *inode, loff_t pos, loff_t length,
 		void *data, struct iomap *iomap, struct iomap *srcmap)
 {
 	struct iomap_dio *dio = data;
+	char pathname[128], *path;
 
 	switch (iomap->type) {
 	case IOMAP_HOLE:

fs/iomap/direct-io.c iomap_dio_actor(struct inode *inode, loff_t pos, loff_t length, · line 392

Quoted 4 lines
fs/iomap/direct-io.c +0 -1
@@ -389,7 +390,21 @@ iomap_dio_actor(struct inode *inode, loff_t pos, loff_t length,
 	case IOMAP_INLINE:
 		return iomap_dio_inline_actor(inode, pos, length, dio, iomap);
 	default:
-		WARN_ON_ONCE(1);

It seems like we should explicitly catch IOMAP_DELALLOC for this case, and leave the
default: as a WARN_ON that is not user-triggerable? i.e.

case IOMAP_DELALLOC:

	<all the fancy warnings>
	return -EIO;
default:
	WARN_ON_ONCE(1);
	return -EIO;
Quoted 5 lines
fs/iomap/direct-io.c +5 -0
+		/*
+		 * DIO is not serialised against mmap() access at all, and so
+		 * if the page_mkwrite occurs between the writeback and the
+		 * iomap_apply() call in the DIO path, then it will see the
+		 * DELALLOC block that the page-mkwrite allocated.

fs/iomap/direct-io.c iomap_dio_actor(struct inode *inode, loff_t pos, loff_t length, · line 407

Quoted 10 lines
fs/iomap/direct-io.c +10 -0
+		 */
+		path = file_path(dio->iocb->ki_filp, pathname,
+				 sizeof(pathname));
+		if (IS_ERR(path))
+			path = "(unknown)";
+
+		pr_warn_ratelimited("page_mkwrite() is racing with DIO read (iomap->type = %u).\n"
+				    "File: %s PID: %d Comm: %.20s\n",
+				    iomap->type, path, current->pid,
+				    current->comm);

This is very specific ...

Quoted 2 lines

Do we know that mmap/page_mkwrite is (and will always be) the only way to reach this point?

I don't know, so this could indeed be a bit misleading.

Quoted 5 lines

It seems to me that this message won't be very useful for the admin; "pg_mkwrite" may mean something to us, but doubtful for the general public. And "type = 1" won't mean much to the reader, either.

Quoted 8 lines

Maybe something like:

"DIO encountered delayed allocation block, racing buffered+direct? File: %s Comm: %.20s\n"

It just seems that a user-facing warning should be something the admin has a chance of acting on without needing to file a bug for analysis by the developers.

(though TBH "delayed allocation" probably doesn't mean much to the admin, either)

I agree with your suggestions. I'll submit a new version.