Optimize get_current_ioprio() a bit
Jens Axboe
maintainercover Optimize get_current_ioprio() a bit
Hi,
Came across this one in some recent profiling, and it's actually quite grim:
+ 2.71% io_uring [kernel.vmlinux] [k] __get_task_ioprio ▒
Just do the easy thing and move it into the header so we avoid a function call per IO for this. Patch 2 is just a general cleanup.
Signature · 1 line
Jens Axboe
Jens Axboe
maintainerPATCH 1/2 block: move __get_task_ioprio() into header file
We call this once per IO, which can be millions of times per second. Since nobody really uses io priorities, or at least it isn't very common, this is all wasted time and can amount to as much as 3% of the total kernel time.
--- block/ioprio.c | 26 -------------------------- include/linux/ioprio.h | 25 ++++++++++++++++++++++++- 2 files changed, 24 insertions(+), 27 deletions(-)
block/ioprio.c +0 -26
@@ -139,32 +139,6 @@ SYSCALL_DEFINE3(ioprio_set, int, which, int, who, int, ioprio)return ret; } -/* -* If the task has set an I/O priority, use that. Otherwise, return -* the default I/O priority. -* -* Expected to be called for current task or with task_lock() held to keep -* io_context stable. -*/ -int __get_task_ioprio (struct task_struct *p) -{ -struct io_context *ioc = p->io_context; -int prio; - -if (p != current) -lockdep_assert_held (&p->alloc_lock); -if (ioc) - prio = ioc->ioprio; -else - prio =IOPRIO_DEFAULT ; - -if (IOPRIO_PRIO_CLASS (prio) ==IOPRIO_CLASS_NONE ) - prio =IOPRIO_PRIO_VALUE (task_nice_ioclass (p), -task_nice_ioprio (p)); -return prio; -} -EXPORT_SYMBOL_GPL (__get_task_ioprio); -static int get_task_ioprio (struct task_struct *p) {int ret;
include/linux/ioprio.h +24 -1
@@ -47,7 +47,30 @@ static inline int task_nice_ioclass(struct task_struct *task) }#ifdef CONFIG_BLOCK -int __get_task_ioprio (struct task_struct *p); +/* +* If the task has set an I/O priority, use that. Otherwise, return +* the default I/O priority. +* +* Expected to be called for current task or with task_lock() held to keep +* io_context stable. +*/ +static inline int __get_task_ioprio (struct task_struct *p) +{ +struct io_context *ioc = p->io_context; +int prio; + +if (p != current) +lockdep_assert_held (&p->alloc_lock); +if (ioc) + prio = ioc->ioprio; +else + prio =IOPRIO_DEFAULT ; + +if (IOPRIO_PRIO_CLASS (prio) ==IOPRIO_CLASS_NONE ) + prio =IOPRIO_PRIO_VALUE (task_nice_ioclass (p), +task_nice_ioprio (p)); +return prio; +}#else static inline int __get_task_ioprio (struct task_struct *p) {
Signature · 1 line
2.43.0
Re: [PATCH 1/2] block: move __get_task_ioprio() into header file
Quoted 2 lines · Jens Axboe
We call this once per IO, which can be millions of times per second. Since nobody really uses io priorities, or at least it isn't very
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
We have plans to set an I/O priority for most Android processes in the near future. According to what Damien wrote on linux-block, there is probably a significant number of hard disks for which I/O priority is configured.
Quoted 2 lines
common, this is all wasted time and can amount to as much as 3% of the total kernel time.
Anyway:
Jens Axboe
maintainerRe: [PATCH 1/2] block: move __get_task_ioprio() into header file
Quoted 9 lines · Bart Van Assche
Quoted 2 lines · Jens Axboe
We call this once per IO, which can be millions of times per second. Since nobody really uses io priorities, or at least it isn't very
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^We have plans to set an I/O priority for most Android processes in the near future. According to what Damien wrote on linux-block, there is probably a significant number of hard disks for which I/O priority is configured.
The rates at which Android does IO means this won't really make any difference, should probably have qualified the statement with "nobody uses io priorities and does fast IO".
Quoted 6 lines
Quoted 2 lines
common, this is all wasted time and can amount to as much as 3% of the total kernel time.
Anyway:
Reviewed-by Bart Van Assche <[email protected]>
Thanks!
Signature · 1 line
Jens Axboe
Re: [PATCH 1/2] block: move __get_task_ioprio() into header file
In reply to Jens AxboeQuoted 8 lines · Jens Axboe
We call this once per IO, which can be millions of times per second. Since nobody really uses io priorities, or at least it isn't very common, this is all wasted time and can amount to as much as 3% of the total kernel time.
Signed-off-by Jens Axboe <[email protected]>---
Looks good.
-ck
Jens Axboe
maintainerPATCH 2/2 block: make __get_task_ioprio() easier to read
In reply to Jens AxboeWe don't need to do any gymnastics if we don't have an io_context assigned at all, so just return early with our default priority.
--- include/linux/ioprio.h | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-)
include/linux/ioprio.h +4 -4
@@ -59,13 +59,13 @@ static inline int __get_task_ioprio(struct task_struct *p)struct io_context *ioc = p->io_context;int prio; +if (!ioc) +return IOPRIO_DEFAULT ; +if (p != current)lockdep_assert_held (&p->alloc_lock); -if (ioc) - prio = ioc->ioprio; -else - prio =IOPRIO_DEFAULT ; + prio = ioc->ioprio;if (IOPRIO_PRIO_CLASS (prio) ==IOPRIO_CLASS_NONE ) prio =IOPRIO_PRIO_VALUE (task_nice_ioclass (p),task_nice_ioprio (p));
Signature · 1 line
2.43.0
Re: [PATCH 2/2] block: make __get_task_ioprio() easier to read
Quoted 20 lines · Jens Axboe
We don't need to do any gymnastics if we don't have an
io_contextassigned at all, so just return early with our default priority.Signed-off-by Jens Axboe <[email protected]>--- include/linux/ioprio.h | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-)include/linux/ioprio.h +3
-0@@ -59,13 +59,13 @@ static inline int __get_task_ioprio(struct task_struct *p)struct io_context *ioc = p->io_context;int prio; +if (!ioc) +return IOPRIO_DEFAULT ; +if (p != current)
include/linux/ioprio.h static inline int __get_task_ioprio(struct task_struct *p) · line 71
Quoted 10 lines
include/linux/ioprio.h +1
-4lockdep_assert_held (&p->alloc_lock); -if (ioc) - prio = ioc->ioprio; -else - prio =IOPRIO_DEFAULT ; + prio = ioc->ioprio;if (IOPRIO_PRIO_CLASS (prio) ==IOPRIO_CLASS_NONE ) prio =IOPRIO_PRIO_VALUE (task_nice_ioclass (p),task_nice_ioprio (p));
Shouldn't it be mentioned in the subject that this patch is a performance optimization? Anyway:
Jens Axboe
maintainerRe: [PATCH 2/2] block: make __get_task_ioprio() easier to read
Quoted 24 lines · Bart Van Assche
Quoted 23 lines · Jens Axboe
We don't need to do any gymnastics if we don't have an
io_contextassigned at all, so just return early with our default priority.Signed-off-by Jens Axboe <[email protected]>--- include/linux/ioprio.h | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-)include/linux/ioprio.h +2
-3@@ -59,13 +59,13 @@ static inline int __get_task_ioprio(struct task_struct *p)struct io_context *ioc = p->io_context;int prio; + if (!ioc) +return IOPRIO_DEFAULT ; +if (p != current)lockdep_assert_held (&p->alloc_lock); -if (ioc) - prio = ioc->ioprio; -else
Quoted 8 lines
include/linux/ioprio.h static inline int __get_task_ioprio(struct task_struct *p) · line 71
Quoted 5 lines
include/linux/ioprio.h +0
-1- prio =IOPRIO_DEFAULT ; + prio = ioc->ioprio;if (IOPRIO_PRIO_CLASS (prio) ==IOPRIO_CLASS_NONE ) prio =IOPRIO_PRIO_VALUE (task_nice_ioclass (p),task_nice_ioprio (p));Shouldn't it be mentioned in the subject that this patch is a performance optimization? Anyway:
I doubt this matters for performance really, it's more of a readability thing for me.
Quoted 1 line
Reviewed-by Bart Van Assche <[email protected]>
Thanks!
Signature · 1 line
Jens Axboe
Re: [PATCH 2/2] block: make __get_task_ioprio() easier to read
In reply to Jens AxboeQuoted 6 lines · Jens Axboe
We don't need to do any gymnastics if we don't have an
io_contextassigned at all, so just return early with our default priority.Signed-off-by Jens Axboe <[email protected]>---
Looks good.
-ck