mm/memory.c: do_numa_page(): remove a redundant page table read
Subsystems (2)
PATCH 1/1 mm/memory.c: do_numa_page(): remove a redundant page table read
do_numa_page() is reading from the same page table entry, twice, while holding the page table lock: once while checking that the pte hasn't changed, and again in order to modify the pte.
Instead, just read the pte once, and save it in the same old_pte variable that already exists. This has no effect on behavior, other than to provide a tiny potential improvement to performance, by avoiding the redundant memory read (which the compiler cannot elide, due to READ_ONCE()).
Also improve the associated comments nearby.
--- mm/memory.c | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-)
mm/memory.c +6 -6
@@ -4928,18 +4928,18 @@ static vm_fault_t do_numa_page(struct vm_fault *vmf)int flags =0 ;/* -* The "pte" at this point cannot be used safely without -* validation through pte_unmap_same(). It's of NUMA type but -* the pfn may be screwed if the read is non atomic. +* The pte cannot be used safely until we verify, while holding the page +* table lock, that its contents have not changed during fault handling. */ spin_lock (vmf->ptl); -if (unlikely (!pte_same (ptep_get (vmf->pte), vmf->orig_pte))) { +/* Read the live PTE from the page tables: */ + old_pte =ptep_get (vmf->pte); + +if (unlikely (!pte_same (old_pte, vmf->orig_pte))) {pte_unmap_unlock (vmf->pte, vmf->ptl); goto out; } -/* Get the normal PTE */ - old_pte =ptep_get (vmf->pte); pte =pte_modify (old_pte, vma->vm_page_prot);/*
Signature · 1 line
2.44.0
Quoted 38 lines · John Hubbard
do_numa_page()is reading from the same page table entry, twice, while holding the page table lock: once while checking that the pte hasn't changed, and again in order to modify the pte.Instead, just read the pte once, and save it in the same
old_ptevariable that already exists. This has no effect on behavior, other than to provide a tiny potential improvement to performance, by avoiding the redundant memory read (which the compiler cannot elide, due toREAD_ONCE()).Also improve the associated comments nearby.
Cc Ryan Roberts <[email protected]>Cc David Hildenbrand <[email protected]>Signed-off-by John Hubbard <[email protected]>--- mm/memory.c | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-)mm/memory.c +5
-4@@ -4928,18 +4928,18 @@ static vm_fault_t do_numa_page(struct vm_fault *vmf)int flags =0 ;/* -* The "pte" at this point cannot be used safely without -* validation through pte_unmap_same(). It's of NUMA type but -* the pfn may be screwed if the read is non atomic. +* The pte cannot be used safely until we verify, while holding the page +* table lock, that its contents have not changed during fault handling. */ spin_lock (vmf->ptl); -if (unlikely (!pte_same (ptep_get (vmf->pte), vmf->orig_pte))) { +/* Read the live PTE from the page tables: */ + old_pte =ptep_get (vmf->pte); +
mm/memory.c static vm_fault_t do_numa_page(struct vm_fault *vmf) · line 4945
Quoted 10 lines
mm/memory.c +1
-2+if (unlikely (!pte_same (old_pte, vmf->orig_pte))) {pte_unmap_unlock (vmf->pte, vmf->ptl); goto out; } -/* Get the normal PTE */ - old_pte =ptep_get (vmf->pte); pte =pte_modify (old_pte, vma->vm_page_prot);/*
Signature · 3 lines
Cheers, David / dhildenb
Quoted 10 lines · John Hubbard
do_numa_page()is reading from the same page table entry, twice, while holding the page table lock: once while checking that the pte hasn't changed, and again in order to modify the pte.Instead, just read the pte once, and save it in the same
old_ptevariable that already exists. This has no effect on behavior, other than to provide a tiny potential improvement to performance, by avoiding the redundant memory read (which the compiler cannot elide, due toREAD_ONCE()).
Quoted 5 lines
Also improve the associated comments nearby.
Cc Ryan Roberts <[email protected]>Cc David Hildenbrand <[email protected]>Signed-off-by John Hubbard <[email protected]>
Quoted 33 lines
--- mm/memory.c | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-)mm/memory.c +6
-6@@ -4928,18 +4928,18 @@ static vm_fault_t do_numa_page(struct vm_fault *vmf)int flags =0 ;/* -* The "pte" at this point cannot be used safely without -* validation through pte_unmap_same(). It's of NUMA type but -* the pfn may be screwed if the read is non atomic. +* The pte cannot be used safely until we verify, while holding the page +* table lock, that its contents have not changed during fault handling. */ spin_lock (vmf->ptl); -if (unlikely (!pte_same (ptep_get (vmf->pte), vmf->orig_pte))) { +/* Read the live PTE from the page tables: */ + old_pte =ptep_get (vmf->pte); + +if (unlikely (!pte_same (old_pte, vmf->orig_pte))) {pte_unmap_unlock (vmf->pte, vmf->ptl); goto out; } -/* Get the normal PTE */ - old_pte =ptep_get (vmf->pte); pte =pte_modify (old_pte, vma->vm_page_prot);/*