Skip to contentSkip to conversation

io_uring: add support for fchmod

PATCH lizetao 3 participants 5 messages Nov 26, 2024 – Dec 14, 2024 io_uring

lore
Share & tools
lore

lizetao

author
#0 lore raw

Hi,

Quoted 16 lines · lizetao
Quoted 15 lines

Hi

Quoted 5 lines
Quoted 3 lines · lizetao

Adds support for doing chmod through io_uring. IORING_OP_FCHMOD behaves like fchmod(2) and takes the same arguments.

Looks pretty straight forward. The only downside is the forced use of REQ_F_FORCE_ASYNC - did you look into how feasible it would be to allow non-blocking issue of this? Would imagine the majority of fchmod calls end up not blocking in the first place.

Yes, I considered fchmod to allow asynchronous execution and wrote a test case to test it, the results are as follows:

fchmod:
real 0m1.413s
user 0m0.253s
sys 0m1.079s

Quoted 8 lines
Quoted 6 lines

io_uring + fchmod:
real 0m1.268s
user 0m0.015s
sys 0m5.739s

There is about a 10% improvement.

And that makes sense if you're keeping some fchmod inflight, as you'd generally just have one io-wq processing them and running things in parallel with submission. But what you you keep an indepth count of 1, eg do sync fchmod? Then it'd be considerably slower than the syscall.

Indeed, When performing REQ_F_FORCE_ASYNC operations at depth 1, performance is degraded. The results are as follows:

fchmod:
real 0m2.285s
user 0m0.050s
sys 0m1.996s

io_uring + fchmod:
real 0m2.541s
user 0m0.013s
sys 0m2.379s

Quoted 1 line

This isn't necessarily something to worry about, but fact is that if you can do a nonblock issue and have it succeed most of the time, that'll be more efficient (and faster for low/sync fchmod) than something that just offloads to io-wq. You can see that from your results too, comparing the sys number netween the two.

However, when I remove REQ_F_FORCE_ASYNC and use IO_URING_F_NONBLOCK, the performance is not improved. The measured results are as follows:
fchmod:
real 0m2.132s
user 0m0.048s
sys 0m1.845s

io_uring + fchmod:
real 0m2.196s
user 0m0.005s
sys 0m2.097s

Quoted 1 line

Hence why I'm asking if you looked into doing a nonblocking issue at all. This won't necessarily gate the inclusion of the patch, and it is something that can be changed down the line, I'm mostly just curious.

Does this result meet expectations? Or maybe I missed something, please let me know

Quoted 2 lines

--
Jens Axboe

---
Li Zetao

maintainer
#1 lore raw
Quoted 44 lines · lizetao

Hi,

Quoted 26 lines · lizetao
Quoted 25 lines

Hi

Quoted 8 lines
Quoted 3 lines · lizetao

Adds support for doing chmod through io_uring. IORING_OP_FCHMOD behaves like fchmod(2) and takes the same arguments.

Looks pretty straight forward. The only downside is the forced use of REQ_F_FORCE_ASYNC - did you look into how feasible it would be to allow non-blocking issue of this? Would imagine the majority of fchmod calls end up not blocking in the first place.

Yes, I considered fchmod to allow asynchronous execution and wrote a test case to test it, the results are as follows:

fchmod:
real 0m1.413s
user 0m0.253s
sys 0m1.079s

io_uring + fchmod:
real 0m1.268s
user 0m0.015s
sys 0m5.739s

Quoted 7 lines
Quoted 1 line

There is about a 10% improvement.

And that makes sense if you're keeping some fchmod inflight, as you'd generally just have one io-wq processing them and running things in parallel with submission. But what you you keep an indepth count of 1, eg do sync fchmod? Then it'd be considerably slower than the syscall.

Indeed, When performing REQ_F_FORCE_ASYNC operations at depth 1, performance is degraded. The results are as follows:

fchmod:
real 0m2.285s
user 0m0.050s
sys 0m1.996s

Quoted 4 lines

io_uring + fchmod:
real 0m2.541s
user 0m0.013s
sys 0m2.379s

That's what I expected. But actually looks like io-wq does a good job in this case, that's pretty close.

Quoted 13 lines
Quoted 5 lines

This isn't necessarily something to worry about, but fact is that if you can do a nonblock issue and have it succeed most of the time, that'll be more efficient (and faster for low/sync fchmod) than something that just offloads to io-wq. You can see that from your results too, comparing the sys number netween the two.

However, when I remove REQ_F_FORCE_ASYNC and use IO_URING_F_NONBLOCK, the performance is not improved. The measured results are as follows:
fchmod:
real 0m2.132s
user 0m0.048s
sys 0m1.845s

Quoted 4 lines

io_uring + fchmod:
real 0m2.196s
user 0m0.005s
sys 0m2.097s

You would not expect it to be faster, as it's really just doing the same work through a different mechanism. I'd expect that to roughly be within normal variance, and if you're not doing a submit_and_wait mechanism (eg you're doing submit and wait separately, hence doing 2 syscalls for each fchmod), then that likely explains the discrepancy, if there is any.

And you'd also need to actually be able to remove REQ_F_FORCE_ASYNC to have this as something that could be included. Otherwise if vfs_fchmod() blocks, then you're now stalling the whole pipeline. Removing it just as a test is fine, as you did.

Quoted 7 lines
Quoted 4 lines

Hence why I'm asking if you looked into doing a nonblocking issue at all. This won't necessarily gate the inclusion of the patch, and it is something that can be changed down the line, I'm mostly just curious.

Does this result meet expectations? Or maybe I missed something, please let me know

Yep that looks like I expected. io-wq offload will be fine if you're doing a bunch of fchmod, in fact it'll probably end up being faster as you reported. But if you're doing single (or few) fchmod at the time, then io-wq offload will be a bit slower.

Signature · 1 line
Jens Axboe
#2 lore raw
In reply to lizetao
Quoted 8 lines · lizetao
Quoted 8 lines
Quoted 8 lines
Quoted 5 lines
Quoted 3 lines · lizetao

Adds support for doing chmod through io_uring. IORING_OP_FCHMOD behaves like fchmod(2) and takes the same arguments.

Looks pretty straight forward. The only downside is the forced use of REQ_F_FORCE_ASYNC - did you look into how feasible it would be to allow non-blocking issue of this? Would imagine the majority of fchmod calls end up not blocking in the first place.

Yes, I considered fchmod to allow asynchronous execution and wrote a test case to test it, the results are as follows:

FYI, this email got into spam.

Quoted 39 lines
Quoted 5 lines
Quoted 5 lines

fchmod:
real 0m1.413s
user 0m0.253s
sys 0m1.079s

Quoted 8 lines
Quoted 6 lines

io_uring + fchmod:
real 0m1.268s
user 0m0.015s
sys 0m5.739s

There is about a 10% improvement.

And that makes sense if you're keeping some fchmod inflight, as you'd generally just have one io-wq processing them and running things in parallel with submission. But what you you keep an indepth count of 1, eg do sync fchmod? Then it'd be considerably slower than the syscall.

Indeed, When performing REQ_F_FORCE_ASYNC operations at depth 1, performance is degraded. The results are as follows:

fchmod:
real 0m2.285s
user 0m0.050s
sys 0m1.996s

io_uring + fchmod:
real 0m2.541s
user 0m0.013s
sys 0m2.379s

Quoted 1 line

This isn't necessarily something to worry about, but fact is that if you can do a nonblock issue and have it succeed most of the time, that'll be more efficient (and faster for low/sync fchmod) than something that just offloads to io-wq. You can see that from your results too, comparing the sys number netween the two.

However, when I remove REQ_F_FORCE_ASYNC and use IO_URING_F_NONBLOCK, the performance is not improved. The measured results are as follows:
fchmod:
real 0m2.132s
user 0m0.048s
sys 0m1.845s

io_uring + fchmod:
real 0m2.196s
user 0m0.005s
sys 0m2.097s

Quoted 1 line

Hence why I'm asking if you looked into doing a nonblocking issue at all. This won't necessarily gate the inclusion of the patch, and it is something that can be changed down the line, I'm mostly just curious.

Does this result meet expectations? Or maybe I missed something, please let me know

Signature · 1 line
Pavel Begunkov
author
#3 lore raw

Hi,

Quoted 18 lines

-----Original Message-----
From: Pavel Begunkov <[email protected]>
Sent: Tuesday, December 3, 2024 10:44 PM
To: lizetao <[email protected]>; Jens Axboe <[email protected]>
Cc: [email protected]
Subject: Re: [PATCH -next] io_uring: add support for fchmod

Quoted 2 lines · lizetao
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines · lizetao

Adds support for doing chmod through io_uring.

IORING_OP_FCHMOD

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 1 line

behaves like fchmod(2) and takes the same arguments.

Looks pretty straight forward. The only downside is the forced use of

REQ_F_FORCE_ASYNC - did you look into how feasible it would be to allow non-blocking issue of this? Would imagine the majority of fchmod calls end up not blocking in the first place.

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 5 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line

Yes, I considered fchmod to allow asynchronous execution and wrote a

test case to test it, the results are as follows:

Quoted 1 line
Quoted 1 line
Quoted 1 line

FYI, this email got into spam.

Sorry to bother everyone, but I would like to know if there are any plans to implement asynchronous system calls through io_uring, and which system calls are in the planning.

--
Li Zetao

maintainer
#4 lore raw
Quoted 20 lines · lizetao

Hi,

Quoted 18 lines

-----Original Message-----
From: Pavel Begunkov <[email protected]>
Sent: Tuesday, December 3, 2024 10:44 PM
To: lizetao <[email protected]>; Jens Axboe <[email protected]>
Cc: [email protected]
Subject: Re: [PATCH -next] io_uring: add support for fchmod

Quoted 2 lines · lizetao
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines · lizetao

Adds support for doing chmod through io_uring.

IORING_OP_FCHMOD

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 1 line

behaves like fchmod(2) and takes the same arguments.

Looks pretty straight forward. The only downside is the forced use of

REQ_F_FORCE_ASYNC - did you look into how feasible it would be to allow non-blocking issue of this? Would imagine the majority of fchmod calls end up not blocking in the first place.

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 8 lines
Quoted 5 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line

Yes, I considered fchmod to allow asynchronous execution and wrote a

test case to test it, the results are as follows:

Quoted 1 line
Quoted 1 line
Quoted 1 line

FYI, this email got into spam.

Sorry to bother everyone, but I would like to know if there are any plans to implement asynchronous system calls through io_uring, and which system calls are in the planning.

No specific plans, existing syscalls are mostly sync by nature of the API for them. Supporting syscalls through io_uring in an efficient manner generally necessitates changing something ala:

syscall_foo(..)

{
	return do_foo_syscall(...);
}

into

syscall_foo(...)

{
	start_foo_syscall(...);
	return wait_on_foo_syscall(...);
}

where the act of issuing and waiting for the completion of it are two separate entities, generally where the waiting is a waitqueue and the wait_on_foo_syscall() simply waits on it to be completed, and io_uring can use this waitqueue to get a callback when it has finished.

That allows efficient processing of syscalls through io_uring. If you don't do that, then you're stuck with do_foo_syscall(), and then io_uring can only support it by punting to the io-wq worker threads which will do the sync do_foo_sycall() part.

Signature · 1 line
Jens Axboe