Skip to contentSkip to conversation

mm/userfaultfd: fix likely/unlikely annotation in move_pages()

PATCH In review Aswin Kumar 3 participants 3 messages Feb 4 – Feb 12

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 mm/userfaultfd: fix likely/unlikely annotation in move_pages()

All other userfaultfd paths use unlikely() for the mmap_changing check, since normally mmap is not changing. The move_pages() path incorrectly uses likely(), which is the opposite of the intended branch prediction hint.

This is a performance-only fix - the logic is correct but the branch prediction annotation is wrong, potentially causing a minor performance penalty on the fast path.

Fixes e0a58ef0faa7 ("userfaultfd: UFFDIO_MOVE uABI")
---
 mm/userfaultfd.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
mm/userfaultfd.c +1 -1
index e6dfd5f28..d27080348 100644
@@ -1793,7 +1793,7 @@ ssize_t move_pages(struct userfaultfd_ctx *ctx, unsigned long dst_start,
 	/* Re-check after taking map_changing_lock */
 	err = -EAGAIN;
 	down_read(&ctx->map_changing_lock);
-	if (likely(atomic_read(&ctx->mmap_changing)))
+	if (unlikely(atomic_read(&ctx->mmap_changing)))
 		goto out_unlock;
 	/*
 	 * Make sure the vma is not shared, that the src and dst remap
Signature · 1 line
2.43.0
#1 lore raw
Quoted 1 line · Aswin Kumar

All other userfaultfd paths use unlikely() for the mmap_changing check,

No they don't?

$ git grep mmap_changing mm
mm/userfaultfd.c:               if (atomic_read(&ctx->mmap_changing))
mm/userfaultfd.c:       if (atomic_read(&ctx->mmap_changing))
mm/userfaultfd.c:       if (atomic_read(&ctx->mmap_changing))
mm/userfaultfd.c:       if (likely(atomic_read(&ctx->mmap_changing)))
Quoted 3 lines

This is a performance-only fix - the logic is correct but the branch prediction annotation is wrong, potentially causing a minor performance penalty on the fast path.

Can you measure it? Bet you can't.

Honestly, I'd just remove the likely() annotation, not change it to unlikely().

#2 lore raw
Quoted 11 lines · Matthew Wilcox
Quoted 1 line · Aswin Kumar

All other userfaultfd paths use unlikely() for the mmap_changing check,

No they don't?

$ git grep mmap_changing mm
mm/userfaultfd.c:               if (atomic_read(&ctx->mmap_changing))
mm/userfaultfd.c:       if (atomic_read(&ctx->mmap_changing))
mm/userfaultfd.c:       if (atomic_read(&ctx->mmap_changing))
mm/userfaultfd.c:       if (likely(atomic_read(&ctx->mmap_changing)))
Quoted 8 lines
Quoted 3 lines

This is a performance-only fix - the logic is correct but the branch prediction annotation is wrong, potentially causing a minor performance penalty on the fast path.

Can you measure it? Bet you can't.

Honestly, I'd just remove the likely() annotation, not change it to unlikely().

Agreed.

Signature · 3 lines
Cheers,

David