mm/userfaultfd: fix likely/unlikely annotation in move_pages()
Subsystems (2)
PATCH 1/1 mm/userfaultfd: fix likely/unlikely annotation in move_pages()
All other userfaultfd paths use unlikely() for the mmap_changing check, since normally mmap is not changing. The move_pages() path incorrectly uses likely(), which is the opposite of the intended branch prediction hint.
This is a performance-only fix - the logic is correct but the branch prediction annotation is wrong, potentially causing a minor performance penalty on the fast path.
--- mm/userfaultfd.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
mm/userfaultfd.c +1 -1
@@ -1793,7 +1793,7 @@ ssize_t move_pages(struct userfaultfd_ctx *ctx, unsigned long dst_start,/* Re-check after taking map_changing_lock */ err = -EAGAIN ;down_read (&ctx->map_changing_lock); -if (likely (atomic_read (&ctx->mmap_changing))) +if (unlikely (atomic_read (&ctx->mmap_changing))) goto out_unlock;/* * Make sure the vma is not shared, that the src and dst remap
Signature · 1 line
2.43.0
Quoted 1 line · Aswin Kumar
All other userfaultfd paths use
unlikely()for themmap_changingcheck,
No they don't?
$ git grep mmap_changing mm mm/userfaultfd.c: if (atomic_read(&ctx->mmap_changing)) mm/userfaultfd.c: if (atomic_read(&ctx->mmap_changing)) mm/userfaultfd.c: if (atomic_read(&ctx->mmap_changing)) mm/userfaultfd.c: if (likely(atomic_read(&ctx->mmap_changing)))
Quoted 3 lines
This is a performance-only fix - the logic is correct but the branch prediction annotation is wrong, potentially causing a minor performance penalty on the fast path.
Can you measure it? Bet you can't.
Honestly, I'd just remove the likely() annotation, not change it to unlikely().
Quoted 11 lines · Matthew Wilcox
Quoted 1 line · Aswin Kumar
All other userfaultfd paths use
unlikely()for themmap_changingcheck,No they don't?
$ git grep mmap_changing mm mm/userfaultfd.c: if (atomic_read(&ctx->mmap_changing)) mm/userfaultfd.c: if (atomic_read(&ctx->mmap_changing)) mm/userfaultfd.c: if (atomic_read(&ctx->mmap_changing)) mm/userfaultfd.c: if (likely(atomic_read(&ctx->mmap_changing)))
Quoted 8 lines
Quoted 3 lines
This is a performance-only fix - the logic is correct but the branch prediction annotation is wrong, potentially causing a minor performance penalty on the fast path.
Can you measure it? Bet you can't.
Honestly, I'd just remove the
likely()annotation, not change it tounlikely().
Agreed.
Signature · 3 lines
Cheers, David