Skip to contentSkip to conversation

io_uring: fix sq array offset calculation

PATCH In review Dmitry Vyukov 3 participants 14 messages Jul 11, 2020 – Jul 17, 2020

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 io_uring: fix sq array offset calculation

rings_size() sets sq_offset to the total size of the rings (the returned value which is used for memory allocation). This is wrong: sq array should be located within the rings, not after them. Set sq_offset to where it should be.

Cc Hristo Venev <[email protected]>
Fixes 75b28affdd6a ("io_uring: allocate the two rings together")

---
This looks so wrong and yet io_uring works.
So I am either missing something very obvious here, or io_uring worked only due to lucky side-effects of rounding size to power-of-2 number of pages (which gave it enough slack at the end),
maybe reading/writing some unrelated memory
with some sizes.
If I am wrong, please poke my nose into what I am not seeing. Otherwise, we probably need to CC stable as well.

---
 fs/io_uring.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
fs/io_uring.c +3 -3
index ca8abde48b6c7..c4c3731ed41e9 100644
@@ -7063,6 +7063,9 @@ static unsigned long rings_size(unsigned sq_entries, unsigned cq_entries,
 		return SIZE_MAX;
 #endif
 
+	if (sq_offset)
+		*sq_offset = off;
+
 	sq_array_size = array_size(sizeof(u32), sq_entries);
 	if (sq_array_size == SIZE_MAX)
 		return SIZE_MAX;
@@ -7070,9 +7073,6 @@ static unsigned long rings_size(unsigned sq_entries, unsigned cq_entries,
 	if (check_add_overflow(off, sq_array_size, &off))
 		return SIZE_MAX;
 
-	if (sq_offset)
-		*sq_offset = off;
-
 	return off;
 }
 
Signature · 1 line
2.27.0.383.g050319c2ae-goog
Acked-by on v1 1/1
#1 lore raw
Quoted 9 lines · Dmitry Vyukov

rings_size() sets sq_offset to the total size of the rings (the returned value which is used for memory allocation). This is wrong: sq array should be located within the rings, not after them. Set sq_offset to where it should be.

Cc Hristo Venev <[email protected]>
Fixes 75b28affdd6a ("io_uring: allocate the two rings together")

Oops.

Quoted 41 lines

---
This looks so wrong and yet io_uring works.
So I am either missing something very obvious here, or io_uring worked only due to lucky side-effects of rounding size to power-of-2 number of pages (which gave it enough slack at the end),
maybe reading/writing some unrelated memory
with some sizes.
If I am wrong, please poke my nose into what I am not seeing. Otherwise, we probably need to CC stable as well.

---
 fs/io_uring.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
fs/io_uring.c +3 -0
index ca8abde48b6c7..c4c3731ed41e9 100644
@@ -7063,6 +7063,9 @@ static unsigned long rings_size(unsigned
 sq_entries, unsigned cq_entries,
 		return SIZE_MAX;
 #endif
 
+	if (sq_offset)
+		*sq_offset = off;
+
 	sq_array_size = array_size(sizeof(u32), sq_entries);
 	if (sq_array_size == SIZE_MAX)
fs/io_uring.c +0 -3
 		return SIZE_MAX;
@@ -7070,9 +7073,6 @@ static unsigned long rings_size(unsigned
 sq_entries, unsigned cq_entries,
 	if (check_add_overflow(off, sq_array_size, &off))
 		return SIZE_MAX;
 
-	if (sq_offset)
-		*sq_offset = off;
-
 	return off;
 }
maintainer
#2 lore raw
In reply to Dmitry Vyukov
Quoted 10 lines · Dmitry Vyukov

rings_size() sets sq_offset to the total size of the rings (the returned value which is used for memory allocation). This is wrong: sq array should be located within the rings, not after them. Set sq_offset to where it should be.

Cc Hristo Venev <[email protected]>
Fixes 75b28affdd6a ("io_uring: allocate the two rings together")
Quoted 10 lines

---
This looks so wrong and yet io_uring works.
So I am either missing something very obvious here, or io_uring worked only due to lucky side-effects of rounding size to power-of-2 number of pages (which gave it enough slack at the end),
maybe reading/writing some unrelated memory
with some sizes.
If I am wrong, please poke my nose into what I am not seeing. Otherwise, we probably need to CC stable as well.

Well that's a noodle scratcher, it's definitely been working fine, and I've never seen any out-of-bounds on any of the testing I do. I regularly run anything with KASAN enabled too.

In any case, the patch is obviously correct, I'll queue it up.

Signature · 1 line
Jens Axboe
#3 lore raw
Quoted 23 lines · Jens Axboe
Quoted 10 lines · Dmitry Vyukov

rings_size() sets sq_offset to the total size of the rings (the returned value which is used for memory allocation). This is wrong: sq array should be located within the rings, not after them. Set sq_offset to where it should be.

Cc Hristo Venev <[email protected]>
Fixes 75b28affdd6a ("io_uring: allocate the two rings together")
Quoted 10 lines

---
This looks so wrong and yet io_uring works.
So I am either missing something very obvious here, or io_uring worked only due to lucky side-effects of rounding size to power-of-2 number of pages (which gave it enough slack at the end),
maybe reading/writing some unrelated memory
with some sizes.
If I am wrong, please poke my nose into what I am not seeing. Otherwise, we probably need to CC stable as well.

Quoted 3 lines

Well that's a noodle scratcher, it's definitely been working fine, and I've never seen any out-of-bounds on any of the testing I do. I regularly run anything with KASAN enabled too.

Looking at the code more, I am not sure how it may not corrupt memory. There definitely should be some combinations where accessing sq_entries*sizeof(u32) more memory won't be OK.
May be worth adding a test that allocates all possible sizes for sq/cq and fills both rings.

maintainer
#4 lore raw
Quoted 28 lines · Dmitry Vyukov
Quoted 23 lines · Jens Axboe
Quoted 10 lines · Dmitry Vyukov

rings_size() sets sq_offset to the total size of the rings (the returned value which is used for memory allocation). This is wrong: sq array should be located within the rings, not after them. Set sq_offset to where it should be.

Cc Hristo Venev <[email protected]>
Fixes 75b28affdd6a ("io_uring: allocate the two rings together")
Quoted 10 lines

---
This looks so wrong and yet io_uring works.
So I am either missing something very obvious here, or io_uring worked only due to lucky side-effects of rounding size to power-of-2 number of pages (which gave it enough slack at the end),
maybe reading/writing some unrelated memory
with some sizes.
If I am wrong, please poke my nose into what I am not seeing. Otherwise, we probably need to CC stable as well.

Quoted 3 lines

Well that's a noodle scratcher, it's definitely been working fine, and I've never seen any out-of-bounds on any of the testing I do. I regularly run anything with KASAN enabled too.

Quoted 5 lines

Looking at the code more, I am not sure how it may not corrupt memory. There definitely should be some combinations where accessing sq_entries*sizeof(u32) more memory won't be OK.
May be worth adding a test that allocates all possible sizes for sq/cq and fills both rings.

Yeah, actually doing that right now just to verify it.

Signature · 1 line
Jens Axboe
maintainer
#5 lore raw
Quoted 29 lines · Jens Axboe
Quoted 28 lines · Dmitry Vyukov
Quoted 26 lines · Jens Axboe
Quoted 10 lines · Dmitry Vyukov

rings_size() sets sq_offset to the total size of the rings (the returned value which is used for memory allocation). This is wrong: sq array should be located within the rings, not after them. Set sq_offset to where it should be.

Cc Hristo Venev <[email protected]>
Fixes 75b28affdd6a ("io_uring: allocate the two rings together")
Quoted 10 lines

---
This looks so wrong and yet io_uring works.
So I am either missing something very obvious here, or io_uring worked only due to lucky side-effects of rounding size to power-of-2 number of pages (which gave it enough slack at the end),
maybe reading/writing some unrelated memory
with some sizes.
If I am wrong, please poke my nose into what I am not seeing. Otherwise, we probably need to CC stable as well.

Well that's a noodle scratcher, it's definitely been working fine, and I've never seen any out-of-bounds on any of the testing I do. I regularly run anything with KASAN enabled too.

Quoted 7 lines
Quoted 5 lines

Looking at the code more, I am not sure how it may not corrupt memory. There definitely should be some combinations where accessing sq_entries*sizeof(u32) more memory won't be OK.
May be worth adding a test that allocates all possible sizes for sq/cq and fills both rings.

Yeah, actually doing that right now just to verify it.

Did that, full utilization of the sq ring and the cq ring, and not seeing anything trigger or wrong. I'd need to look closer, but it just might be that the power-of-2 sizes end up saving us from doom and gloom.

Signature · 1 line
Jens Axboe
#6 lore raw
In reply to Dmitry Vyukov
Quoted 7 lines · Dmitry Vyukov

Looking at the code more, I am not sure how it may not corrupt memory.
There definitely should be some combinations where accessing sq_entries*sizeof(u32) more memory won't be OK.
May be worth adding a test that allocates all possible sizes for sq/cq
and fills both rings.

The layout (after the fix) is roughly as follows:

1. struct io_rings - ~192 bytes, maybe 256
2. cqes - (32 << n) bytes
3. sq_array - (4 << n) bytes

The bug was that the sq_array was offset by (4 << n) bytes. I think issues can only occur when

    PAGE_ALIGN(192 + (32 << n) + (4 << n) + (4 << n))
    !=
    PAGE_ALIGN(192 + (32 << n) + (4 << n))

It looks like this never happens. We got lucky.

maintainer
#7 lore raw
Quoted 15 lines · Hristo Venev
Quoted 7 lines · Dmitry Vyukov

Looking at the code more, I am not sure how it may not corrupt memory.
There definitely should be some combinations where accessing sq_entries*sizeof(u32) more memory won't be OK.
May be worth adding a test that allocates all possible sizes for sq/cq
and fills both rings.

The layout (after the fix) is roughly as follows:

1. struct io_rings - ~192 bytes, maybe 256
2. cqes - (32 << n) bytes
3. sq_array - (4 << n) bytes

Quoted 8 lines

The bug was that the sq_array was offset by (4 << n) bytes. I think issues can only occur when

    PAGE_ALIGN(192 + (32 << n) + (4 << n) + (4 << n))
    !=
    PAGE_ALIGN(192 + (32 << n) + (4 << n))

It looks like this never happens. We got lucky.

A bit of luck, but if that wasn't the case, then I'm sure we would have found it when the original patch was tested. But thanks for double checking!

Signature · 1 line
Jens Axboe
#8 lore raw
In reply to Hristo Venev
Quoted 4 lines · Hristo Venev

On Sat, 2020-07-11 at 17:31 +0200, Dmitry
PAGE_ALIGN(192 + (32 << n) + (4 << n) + (4 << n)) !=
PAGE_ALIGN(192 + (32 << n) + (4 << n))

* also power-of-2 aligned.

#9 lore raw
In reply to Hristo Venev
Quoted 16 lines · Hristo Venev
Quoted 7 lines · Dmitry Vyukov

Looking at the code more, I am not sure how it may not corrupt memory.
There definitely should be some combinations where accessing sq_entries*sizeof(u32) more memory won't be OK.
May be worth adding a test that allocates all possible sizes for sq/cq
and fills both rings.

The layout (after the fix) is roughly as follows:

1. struct io_rings - ~192 bytes, maybe 256
2. cqes - (32 << n) bytes
3. sq_array - (4 << n) bytes

Quoted 8 lines

The bug was that the sq_array was offset by (4 << n) bytes. I think issues can only occur when

    PAGE_ALIGN(192 + (32 << n) + (4 << n) + (4 << n))
    !=
    PAGE_ALIGN(192 + (32 << n) + (4 << n))

It looks like this never happens. We got lucky.

Interesting. CQ entries are larger and we have at least that many of them as SQ entries. I guess this + power-of-2-pages can make it never overflow.

#10 lore raw
Quoted 25 lines · Dmitry Vyukov
Quoted 23 lines · Hristo Venev
Quoted 7 lines · Dmitry Vyukov

Looking at the code more, I am not sure how it may not corrupt memory.
There definitely should be some combinations where accessing sq_entries*sizeof(u32) more memory won't be OK.
May be worth adding a test that allocates all possible sizes for sq/cq
and fills both rings.

The layout (after the fix) is roughly as follows:

1. struct io_rings - ~192 bytes, maybe 256
2. cqes - (32 << n) bytes
3. sq_array - (4 << n) bytes

The bug was that the sq_array was offset by (4 << n) bytes. I think issues can only occur when

    PAGE_ALIGN(192 + (32 << n) + (4 << n) + (4 << n))
    !=
    PAGE_ALIGN(192 + (32 << n) + (4 << n))
Quoted 5 lines
Quoted 1 line

It looks like this never happens. We got lucky.

Interesting. CQ entries are larger and we have at least that many of them as SQ entries. I guess this + power-of-2-pages can make it never overflow.

Hi Jens,

I see this patch is in block/for-5.9/io_uring
Is this tree merged into linux-next? I don't see it in linux-next yet. Or is it possible to get it into 5.8?

The reason I am asking is that we have an intern (Necip in CC) working on significantly extending io_uring coverage in syzkaller:
https://github.com/google/syzkaller/pull/1926
Unfortunately we had to hardcode offset computation logic b/c the intended way of using io_uring for normal programs represents an additional obstacle for the fuzzer and the relations between syscalls and writes to shared memory are even hard to express for the fuzzer. We want to hardcode this new updated way of computing offsets, but this means we probably won't get good coverage until the intern term ends (+ may be good to discover some io_uring bugs before the release).

If it won't get into linux-next/mainline until 5.9, it's not a big deal, but I wanted to ask.

maintainer
#11 lore raw
Quoted 32 lines · Dmitry Vyukov
Quoted 25 lines · Dmitry Vyukov
Quoted 23 lines · Hristo Venev
Quoted 7 lines · Dmitry Vyukov

Looking at the code more, I am not sure how it may not corrupt memory.
There definitely should be some combinations where accessing sq_entries*sizeof(u32) more memory won't be OK.
May be worth adding a test that allocates all possible sizes for sq/cq
and fills both rings.

The layout (after the fix) is roughly as follows:

1. struct io_rings - ~192 bytes, maybe 256
2. cqes - (32 << n) bytes
3. sq_array - (4 << n) bytes

The bug was that the sq_array was offset by (4 << n) bytes. I think issues can only occur when

    PAGE_ALIGN(192 + (32 << n) + (4 << n) + (4 << n))
    !=
    PAGE_ALIGN(192 + (32 << n) + (4 << n))
Quoted 5 lines
Quoted 1 line

It looks like this never happens. We got lucky.

Interesting. CQ entries are larger and we have at least that many of them as SQ entries. I guess this + power-of-2-pages can make it never overflow.

Quoted 5 lines

Hi Jens,

I see this patch is in block/for-5.9/io_uring
Is this tree merged into linux-next? I don't see it in linux-next yet. Or is it possible to get it into 5.8?

Yes, that tree is in linux-next, and I'm surprised you don't see it there as it's been queued up for almost a week. Are you sure?

I'm not going to apply it to both 5.9 and 5.8 trees. The bug has been there for a while, but doesn't really impact functionality. Hence I just queued it up for 5.9. If this had been a 5.8 commit that introduced it, I would have queued it up for 5.8.

Quoted 11 lines

The reason I am asking is that we have an intern (Necip in CC) working on significantly extending io_uring coverage in syzkaller:
https://github.com/google/syzkaller/pull/1926
Unfortunately we had to hardcode offset computation logic b/c the intended way of using io_uring for normal programs represents an additional obstacle for the fuzzer and the relations between syscalls and writes to shared memory are even hard to express for the fuzzer. We want to hardcode this new updated way of computing offsets, but this means we probably won't get good coverage until the intern term ends (+ may be good to discover some io_uring bugs before the release).

Sounds good

Quoted 2 lines

If it won't get into linux-next/mainline until 5.9, it's not a big deal, but I wanted to ask.

That's the plan, it'll go in as part of the 5.9 merge window.

Signature · 1 line
Jens Axboe
maintainer
#12 lore raw
Quoted 33 lines · Jens Axboe
Quoted 32 lines · Dmitry Vyukov
Quoted 30 lines · Dmitry Vyukov
Quoted 16 lines · Hristo Venev
Quoted 7 lines · Dmitry Vyukov

Looking at the code more, I am not sure how it may not corrupt memory.
There definitely should be some combinations where accessing sq_entries*sizeof(u32) more memory won't be OK.
May be worth adding a test that allocates all possible sizes for sq/cq
and fills both rings.

The layout (after the fix) is roughly as follows:

1. struct io_rings - ~192 bytes, maybe 256
2. cqes - (32 << n) bytes
3. sq_array - (4 << n) bytes

Quoted 8 lines

The bug was that the sq_array was offset by (4 << n) bytes. I think issues can only occur when

    PAGE_ALIGN(192 + (32 << n) + (4 << n) + (4 << n))
    !=
    PAGE_ALIGN(192 + (32 << n) + (4 << n))

It looks like this never happens. We got lucky.

Interesting. CQ entries are larger and we have at least that many of them as SQ entries. I guess this + power-of-2-pages can make it never overflow.

Quoted 8 lines
Quoted 5 lines

Hi Jens,

I see this patch is in block/for-5.9/io_uring
Is this tree merged into linux-next? I don't see it in linux-next yet. Or is it possible to get it into 5.8?

Yes, that tree is in linux-next, and I'm surprised you don't see it there as it's been queued up for almost a week. Are you sure?

I see it in there:

https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git/commit/fs/io_uring.c?id=a4968ff8b6314631a73fdc945a66fd8645dfe8cc

Signature · 1 line
Jens Axboe
#13 lore raw
In reply to Jens Axboe
Quoted 60 lines · Jens Axboe
Quoted 32 lines · Dmitry Vyukov
Quoted 25 lines · Dmitry Vyukov
Quoted 23 lines · Hristo Venev
Quoted 7 lines · Dmitry Vyukov

Looking at the code more, I am not sure how it may not corrupt memory.
There definitely should be some combinations where accessing sq_entries*sizeof(u32) more memory won't be OK.
May be worth adding a test that allocates all possible sizes for sq/cq
and fills both rings.

The layout (after the fix) is roughly as follows:

1. struct io_rings - ~192 bytes, maybe 256
2. cqes - (32 << n) bytes
3. sq_array - (4 << n) bytes

The bug was that the sq_array was offset by (4 << n) bytes. I think issues can only occur when

    PAGE_ALIGN(192 + (32 << n) + (4 << n) + (4 << n))
    !=
    PAGE_ALIGN(192 + (32 << n) + (4 << n))
Quoted 5 lines
Quoted 1 line

It looks like this never happens. We got lucky.

Interesting. CQ entries are larger and we have at least that many of them as SQ entries. I guess this + power-of-2-pages can make it never overflow.

Quoted 5 lines

Hi Jens,

I see this patch is in block/for-5.9/io_uring
Is this tree merged into linux-next? I don't see it in linux-next yet. Or is it possible to get it into 5.8?

Yes, that tree is in linux-next, and I'm surprised you don't see it there as it's been queued up for almost a week. Are you sure?

I'm not going to apply it to both 5.9 and 5.8 trees. The bug has been there for a while, but doesn't really impact functionality. Hence I just queued it up for 5.9. If this had been a 5.8 commit that introduced it, I would have queued it up for 5.8.

Quoted 11 lines

The reason I am asking is that we have an intern (Necip in CC) working on significantly extending io_uring coverage in syzkaller:
https://github.com/google/syzkaller/pull/1926
Unfortunately we had to hardcode offset computation logic b/c the intended way of using io_uring for normal programs represents an additional obstacle for the fuzzer and the relations between syscalls and writes to shared memory are even hard to express for the fuzzer. We want to hardcode this new updated way of computing offsets, but this means we probably won't get good coverage until the intern term ends (+ may be good to discover some io_uring bugs before the release).

Quoted 6 lines

Sounds good

Quoted 2 lines

If it won't get into linux-next/mainline until 5.9, it's not a big deal, but I wanted to ask.

That's the plan, it'll go in as part of the 5.9 merge window.

Thanks.
linux-next is good enough, we test it. And the commit is actually already there, now that I looked closer.