io_uring: fix sq array offset calculation
PATCH 1/1 io_uring: fix sq array offset calculation
rings_size() sets sq_offset to the total size of the rings (the returned value which is used for memory allocation). This is wrong: sq array should be located within the rings, not after them. Set sq_offset to where it should be.
---
This looks so wrong and yet io_uring works.
So I am either missing something very obvious here, or io_uring worked only due to lucky side-effects of rounding size to power-of-2 number of pages (which gave it enough slack at the end),
maybe reading/writing some unrelated memory
with some sizes.
If I am wrong, please poke my nose into what I am not seeing. Otherwise, we probably need to CC stable as well.
--- fs/io_uring.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)
fs/io_uring.c +3 -3
@@ -7063,6 +7063,9 @@ static unsigned long rings_size(unsigned sq_entries, unsigned cq_entries,return SIZE_MAX ;#endif +if (sq_offset) + *sq_offset = off; + sq_array_size =array_size (sizeof (u32), sq_entries);if (sq_array_size ==SIZE_MAX )return SIZE_MAX ;
@@ -7070,9 +7073,6 @@ static unsigned long rings_size(unsigned sq_entries, unsigned cq_entries,if (check_add_overflow (off, sq_array_size, &off))return SIZE_MAX ; -if (sq_offset) - *sq_offset = off; -return off; }
Signature · 1 line
2.27.0.383.g050319c2ae-goog
Quoted 9 lines · Dmitry Vyukov
rings_size()setssq_offsetto the total size of the rings (the returned value which is used for memory allocation). This is wrong: sq array should be located within the rings, not after them. Setsq_offsetto where it should be.Signed-off-by Dmitry Vyukov <[email protected]>Cc Hristo Venev <[email protected]>Fixes 75b28affdd6a ("io_uring: allocate the two rings together")
Oops.
Quoted 41 lines
---
This looks so wrong and yetio_uringworks.
So I am either missing something very obvious here, orio_uringworked only due to lucky side-effects of rounding size to power-of-2 number of pages (which gave it enough slack at the end),
maybe reading/writing some unrelated memory
with some sizes.
If I am wrong, please poke my nose into what I am not seeing. Otherwise, we probably need to CC stable as well.--- fs/io_uring.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)fs/io_uring.c +3
-0@@ -7063,6 +7063,9 @@ static unsigned long rings_size(unsigned sq_entries, unsigned cq_entries, return SIZE_MAX;#endif +if (sq_offset) + *sq_offset = off; + sq_array_size =array_size (sizeof (u32), sq_entries);if (sq_array_size ==SIZE_MAX )fs/io_uring.c +0
-3return SIZE_MAX ;@@ -7070,9 +7073,6 @@ static unsigned long rings_size(unsigned sq_entries, unsigned cq_entries, if (check_add_overflow(off, sq_array_size, &off))return SIZE_MAX ; -if (sq_offset) - *sq_offset = off; -return off; }
Jens Axboe
maintainerQuoted 10 lines · Dmitry Vyukov
rings_size()setssq_offsetto the total size of the rings (the returned value which is used for memory allocation). This is wrong: sq array should be located within the rings, not after them. Setsq_offsetto where it should be.Signed-off-by Dmitry Vyukov <[email protected]>Cc Hristo Venev <[email protected]>Fixes 75b28affdd6a ("io_uring: allocate the two rings together")
Quoted 10 lines
---
This looks so wrong and yetio_uringworks.
So I am either missing something very obvious here, orio_uringworked only due to lucky side-effects of rounding size to power-of-2 number of pages (which gave it enough slack at the end),
maybe reading/writing some unrelated memory
with some sizes.
If I am wrong, please poke my nose into what I am not seeing. Otherwise, we probably need to CC stable as well.
Well that's a noodle scratcher, it's definitely been working fine, and I've never seen any out-of-bounds on any of the testing I do. I regularly run anything with KASAN enabled too.
In any case, the patch is obviously correct, I'll queue it up.
Signature · 1 line
Jens Axboe
Quoted 23 lines · Jens Axboe
Quoted 10 lines · Dmitry Vyukov
rings_size()setssq_offsetto the total size of the rings (the returned value which is used for memory allocation). This is wrong: sq array should be located within the rings, not after them. Setsq_offsetto where it should be.Signed-off-by Dmitry Vyukov <[email protected]>Cc Hristo Venev <[email protected]>Fixes 75b28affdd6a ("io_uring: allocate the two rings together")Quoted 10 lines
---
This looks so wrong and yetio_uringworks.
So I am either missing something very obvious here, orio_uringworked only due to lucky side-effects of rounding size to power-of-2 number of pages (which gave it enough slack at the end),
maybe reading/writing some unrelated memory
with some sizes.
If I am wrong, please poke my nose into what I am not seeing. Otherwise, we probably need to CC stable as well.
Quoted 3 lines
Well that's a noodle scratcher, it's definitely been working fine, and I've never seen any out-of-bounds on any of the testing I do. I regularly run anything with KASAN enabled too.
Looking at the code more, I am not sure how it may not corrupt memory. There definitely should be some combinations where accessing sq_entries*sizeof(u32) more memory won't be OK.
May be worth adding a test that allocates all possible sizes for sq/cq and fills both rings.
Jens Axboe
maintainerQuoted 28 lines · Dmitry Vyukov
Quoted 23 lines · Jens Axboe
Quoted 10 lines · Dmitry Vyukov
rings_size()setssq_offsetto the total size of the rings (the returned value which is used for memory allocation). This is wrong: sq array should be located within the rings, not after them. Setsq_offsetto where it should be.Signed-off-by Dmitry Vyukov <[email protected]>Cc Hristo Venev <[email protected]>Fixes 75b28affdd6a ("io_uring: allocate the two rings together")Quoted 10 lines
---
This looks so wrong and yetio_uringworks.
So I am either missing something very obvious here, orio_uringworked only due to lucky side-effects of rounding size to power-of-2 number of pages (which gave it enough slack at the end),
maybe reading/writing some unrelated memory
with some sizes.
If I am wrong, please poke my nose into what I am not seeing. Otherwise, we probably need to CC stable as well.Quoted 3 lines
Well that's a noodle scratcher, it's definitely been working fine, and I've never seen any out-of-bounds on any of the testing I do. I regularly run anything with KASAN enabled too.
Quoted 5 lines
Looking at the code more, I am not sure how it may not corrupt memory. There definitely should be some combinations where accessing sq_entries*sizeof(u32) more memory won't be OK.
May be worth adding a test that allocates all possible sizes for sq/cq and fills both rings.
Yeah, actually doing that right now just to verify it.
Signature · 1 line
Jens Axboe
Jens Axboe
maintainerQuoted 29 lines · Jens Axboe
Quoted 28 lines · Dmitry Vyukov
Quoted 26 lines · Jens Axboe
Quoted 10 lines · Dmitry Vyukov
rings_size()setssq_offsetto the total size of the rings (the returned value which is used for memory allocation). This is wrong: sq array should be located within the rings, not after them. Setsq_offsetto where it should be.Signed-off-by Dmitry Vyukov <[email protected]>Cc Hristo Venev <[email protected]>Fixes 75b28affdd6a ("io_uring: allocate the two rings together")Quoted 10 lines
---
This looks so wrong and yetio_uringworks.
So I am either missing something very obvious here, orio_uringworked only due to lucky side-effects of rounding size to power-of-2 number of pages (which gave it enough slack at the end),
maybe reading/writing some unrelated memory
with some sizes.
If I am wrong, please poke my nose into what I am not seeing. Otherwise, we probably need to CC stable as well.Well that's a noodle scratcher, it's definitely been working fine, and I've never seen any out-of-bounds on any of the testing I do. I regularly run anything with KASAN enabled too.
Quoted 7 lines
Quoted 5 lines
Looking at the code more, I am not sure how it may not corrupt memory. There definitely should be some combinations where accessing sq_entries*sizeof(u32) more memory won't be OK.
May be worth adding a test that allocates all possible sizes for sq/cq and fills both rings.Yeah, actually doing that right now just to verify it.
Did that, full utilization of the sq ring and the cq ring, and not seeing anything trigger or wrong. I'd need to look closer, but it just might be that the power-of-2 sizes end up saving us from doom and gloom.
Signature · 1 line
Jens Axboe
Quoted 7 lines · Dmitry Vyukov
Looking at the code more, I am not sure how it may not corrupt memory.
There definitely should be some combinations where accessing sq_entries*sizeof(u32) more memory won't be OK.
May be worth adding a test that allocates all possible sizes for sq/cq
and fills both rings.
The layout (after the fix) is roughly as follows:
1. struct io_rings - ~192 bytes, maybe 256
2. cqes - (32 << n) bytes
3. sq_array - (4 << n) bytes
The bug was that the sq_array was offset by (4 << n) bytes. I think issues can only occur when
PAGE_ALIGN(192 + (32 << n) + (4 << n) + (4 << n))
!=
PAGE_ALIGN(192 + (32 << n) + (4 << n))It looks like this never happens. We got lucky.
Jens Axboe
maintainerQuoted 15 lines · Hristo Venev
Quoted 7 lines · Dmitry Vyukov
Looking at the code more, I am not sure how it may not corrupt memory.
There definitely should be some combinations where accessing sq_entries*sizeof(u32) more memory won't be OK.
May be worth adding a test that allocates all possible sizes for sq/cq
and fills both rings.The layout (after the fix) is roughly as follows:
1. struct
io_rings- ~192 bytes, maybe 256
2. cqes - (32 << n) bytes
3.sq_array- (4 << n) bytes
Quoted 8 lines
The bug was that the
sq_arraywas offset by (4 << n) bytes. I think issues can only occur whenPAGE_ALIGN(192 + (32 << n) + (4 << n) + (4 << n)) != PAGE_ALIGN(192 + (32 << n) + (4 << n))It looks like this never happens. We got lucky.
A bit of luck, but if that wasn't the case, then I'm sure we would have found it when the original patch was tested. But thanks for double checking!
Signature · 1 line
Jens Axboe
Quoted 4 lines · Hristo Venev
On Sat, 2020-07-11 at 17:31 +0200, Dmitry
PAGE_ALIGN(192 + (32 << n) + (4 << n) + (4 << n))!=
PAGE_ALIGN(192 + (32 << n) + (4 << n))
* also power-of-2 aligned.
Quoted 16 lines · Hristo Venev
Quoted 7 lines · Dmitry Vyukov
Looking at the code more, I am not sure how it may not corrupt memory.
There definitely should be some combinations where accessing sq_entries*sizeof(u32) more memory won't be OK.
May be worth adding a test that allocates all possible sizes for sq/cq
and fills both rings.The layout (after the fix) is roughly as follows:
1. struct
io_rings- ~192 bytes, maybe 256
2. cqes - (32 << n) bytes
3.sq_array- (4 << n) bytes
Quoted 8 lines
The bug was that the
sq_arraywas offset by (4 << n) bytes. I think issues can only occur whenPAGE_ALIGN(192 + (32 << n) + (4 << n) + (4 << n)) != PAGE_ALIGN(192 + (32 << n) + (4 << n))It looks like this never happens. We got lucky.
Interesting. CQ entries are larger and we have at least that many of them as SQ entries. I guess this + power-of-2-pages can make it never overflow.
Quoted 25 lines · Dmitry Vyukov
Quoted 23 lines · Hristo Venev
Quoted 7 lines · Dmitry Vyukov
Looking at the code more, I am not sure how it may not corrupt memory.
There definitely should be some combinations where accessing sq_entries*sizeof(u32) more memory won't be OK.
May be worth adding a test that allocates all possible sizes for sq/cq
and fills both rings.The layout (after the fix) is roughly as follows:
1. struct
io_rings- ~192 bytes, maybe 256
2. cqes - (32 << n) bytes
3.sq_array- (4 << n) bytesThe bug was that the
sq_arraywas offset by (4 << n) bytes. I think issues can only occur whenPAGE_ALIGN(192 + (32 << n) + (4 << n) + (4 << n)) != PAGE_ALIGN(192 + (32 << n) + (4 << n))
Quoted 5 lines
Quoted 1 line
It looks like this never happens. We got lucky.
Interesting. CQ entries are larger and we have at least that many of them as SQ entries. I guess this + power-of-2-pages can make it never overflow.
Hi Jens,
I see this patch is in block/for-5.9/io_uring
Is this tree merged into linux-next? I don't see it in linux-next yet. Or is it possible to get it into 5.8?
The reason I am asking is that we have an intern (Necip in CC) working on significantly extending io_uring coverage in syzkaller:
https://github.com/google/syzkaller/pull/1926
Unfortunately we had to hardcode offset computation logic b/c the intended way of using io_uring for normal programs represents an additional obstacle for the fuzzer and the relations between syscalls and writes to shared memory are even hard to express for the fuzzer. We want to hardcode this new updated way of computing offsets, but this means we probably won't get good coverage until the intern term ends (+ may be good to discover some io_uring bugs before the release).
If it won't get into linux-next/mainline until 5.9, it's not a big deal, but I wanted to ask.
Jens Axboe
maintainerQuoted 32 lines · Dmitry Vyukov
Quoted 25 lines · Dmitry Vyukov
Quoted 23 lines · Hristo Venev
Quoted 7 lines · Dmitry Vyukov
Looking at the code more, I am not sure how it may not corrupt memory.
There definitely should be some combinations where accessing sq_entries*sizeof(u32) more memory won't be OK.
May be worth adding a test that allocates all possible sizes for sq/cq
and fills both rings.The layout (after the fix) is roughly as follows:
1. struct
io_rings- ~192 bytes, maybe 256
2. cqes - (32 << n) bytes
3.sq_array- (4 << n) bytesThe bug was that the
sq_arraywas offset by (4 << n) bytes. I think issues can only occur whenPAGE_ALIGN(192 + (32 << n) + (4 << n) + (4 << n)) != PAGE_ALIGN(192 + (32 << n) + (4 << n))Quoted 5 lines
Quoted 1 line
It looks like this never happens. We got lucky.
Interesting. CQ entries are larger and we have at least that many of them as SQ entries. I guess this + power-of-2-pages can make it never overflow.
Quoted 5 lines
Hi Jens,
I see this patch is in
block/for-5.9/io_uring
Is this tree merged into linux-next? I don't see it in linux-next yet. Or is it possible to get it into 5.8?
Yes, that tree is in linux-next, and I'm surprised you don't see it there as it's been queued up for almost a week. Are you sure?
I'm not going to apply it to both 5.9 and 5.8 trees. The bug has been there for a while, but doesn't really impact functionality. Hence I just queued it up for 5.9. If this had been a 5.8 commit that introduced it, I would have queued it up for 5.8.
Quoted 11 lines
The reason I am asking is that we have an intern (Necip in CC) working on significantly extending
io_uringcoverage in syzkaller:
https://github.com/google/syzkaller/pull/1926
Unfortunately we had to hardcode offset computation logic b/c the intended way of usingio_uringfor normal programs represents an additional obstacle for the fuzzer and the relations between syscalls and writes to shared memory are even hard to express for the fuzzer. We want to hardcode this new updated way of computing offsets, but this means we probably won't get good coverage until the intern term ends (+ may be good to discover someio_uringbugs before the release).
Sounds good
Quoted 2 lines
If it won't get into linux-next/mainline until 5.9, it's not a big deal, but I wanted to ask.
That's the plan, it'll go in as part of the 5.9 merge window.
Signature · 1 line
Jens Axboe
Jens Axboe
maintainerQuoted 33 lines · Jens Axboe
Quoted 32 lines · Dmitry Vyukov
Quoted 30 lines · Dmitry Vyukov
Quoted 16 lines · Hristo Venev
Quoted 7 lines · Dmitry Vyukov
Looking at the code more, I am not sure how it may not corrupt memory.
There definitely should be some combinations where accessing sq_entries*sizeof(u32) more memory won't be OK.
May be worth adding a test that allocates all possible sizes for sq/cq
and fills both rings.The layout (after the fix) is roughly as follows:
1. struct
io_rings- ~192 bytes, maybe 256
2. cqes - (32 << n) bytes
3.sq_array- (4 << n) bytesQuoted 8 lines
The bug was that the
sq_arraywas offset by (4 << n) bytes. I think issues can only occur whenPAGE_ALIGN(192 + (32 << n) + (4 << n) + (4 << n)) != PAGE_ALIGN(192 + (32 << n) + (4 << n))It looks like this never happens. We got lucky.
Interesting. CQ entries are larger and we have at least that many of them as SQ entries. I guess this + power-of-2-pages can make it never overflow.
Quoted 8 lines
Quoted 5 lines
Hi Jens,
I see this patch is in
block/for-5.9/io_uring
Is this tree merged into linux-next? I don't see it in linux-next yet. Or is it possible to get it into 5.8?Yes, that tree is in linux-next, and I'm surprised you don't see it there as it's been queued up for almost a week. Are you sure?
I see it in there:
Signature · 1 line
Jens Axboe
Quoted 60 lines · Jens Axboe
Quoted 32 lines · Dmitry Vyukov
Quoted 25 lines · Dmitry Vyukov
Quoted 23 lines · Hristo Venev
Quoted 7 lines · Dmitry Vyukov
Looking at the code more, I am not sure how it may not corrupt memory.
There definitely should be some combinations where accessing sq_entries*sizeof(u32) more memory won't be OK.
May be worth adding a test that allocates all possible sizes for sq/cq
and fills both rings.The layout (after the fix) is roughly as follows:
1. struct
io_rings- ~192 bytes, maybe 256
2. cqes - (32 << n) bytes
3.sq_array- (4 << n) bytesThe bug was that the
sq_arraywas offset by (4 << n) bytes. I think issues can only occur whenPAGE_ALIGN(192 + (32 << n) + (4 << n) + (4 << n)) != PAGE_ALIGN(192 + (32 << n) + (4 << n))Quoted 5 lines
Quoted 1 line
It looks like this never happens. We got lucky.
Interesting. CQ entries are larger and we have at least that many of them as SQ entries. I guess this + power-of-2-pages can make it never overflow.
Quoted 5 lines
Hi Jens,
I see this patch is in
block/for-5.9/io_uring
Is this tree merged into linux-next? I don't see it in linux-next yet. Or is it possible to get it into 5.8?Yes, that tree is in linux-next, and I'm surprised you don't see it there as it's been queued up for almost a week. Are you sure?
I'm not going to apply it to both 5.9 and 5.8 trees. The bug has been there for a while, but doesn't really impact functionality. Hence I just queued it up for 5.9. If this had been a 5.8 commit that introduced it, I would have queued it up for 5.8.
Quoted 11 lines
The reason I am asking is that we have an intern (Necip in CC) working on significantly extending
io_uringcoverage in syzkaller:
https://github.com/google/syzkaller/pull/1926
Unfortunately we had to hardcode offset computation logic b/c the intended way of usingio_uringfor normal programs represents an additional obstacle for the fuzzer and the relations between syscalls and writes to shared memory are even hard to express for the fuzzer. We want to hardcode this new updated way of computing offsets, but this means we probably won't get good coverage until the intern term ends (+ may be good to discover someio_uringbugs before the release).
Quoted 6 lines
Sounds good
Quoted 2 lines
If it won't get into linux-next/mainline until 5.9, it's not a big deal, but I wanted to ask.
That's the plan, it'll go in as part of the 5.9 merge window.
Thanks.
linux-next is good enough, we test it. And the commit is actually already there, now that I looked closer.