mm: pagemap: restrict pagewalk to the requested range
Subsystems (4)
PATCH 1/1 mm: pagemap: restrict pagewalk to the requested range
The pagewalk in pagemap_read reads one PTE past the end of the requested range, and stops when the buffer runs out of space. While it produces the right result, the extra read is unnecessary and less performant.
I timed the following command before and after this patch:
dd count=100000 if=/proc/self/pagemap of=/dev/null
The results are consistently within 0.001s across 5 runs.
Before:
100000+0 records in
100000+0 records out
51200000 bytes (51 MB) copied, 0.0763159 s, 671 MB/s
real 0m0.078s user 0m0.012s sys 0m0.065s
After:
100000+0 records in
100000+0 records out
51200000 bytes (51 MB) copied, 0.0487928 s, 1.0 GB/s
real 0m0.050s user 0m0.011s sys 0m0.039s
--- fs/proc/task_mmu.c | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-)
fs/proc/task_mmu.c +6 -6
@@ -1689,23 +1689,23 @@ static ssize_t pagemap_read(struct file *file, char __user *buf,/* watch out for wraparound */ start_vaddr = end_vaddr;if (svpfn <= (ULONG_MAX >>PAGE_SHIFT )) { +unsigned long end; + ret =mmap_read_lock_killable (mm);if (ret) goto out_free; start_vaddr =untagged_addr_remote (mm, svpfn <<PAGE_SHIFT );mmap_read_unlock (mm); + + end = start_vaddr + ((count /PM_ENTRY_BYTES ) <<PAGE_SHIFT ); +if (end >= start_vaddr && end < mm->task_size) + end_vaddr = end; }/* Ensure the address is inside the task */ if (start_vaddr > mm->task_size) start_vaddr = end_vaddr; -/* -* The odds are that this will stop walking way -* before end_vaddr, because the length of the -* user buffer is tracked in "pm", and the walk -* will stop when we hit the end of the buffer. -*/ ret =0 ;while (count && (start_vaddr < end_vaddr)) {int len;
Signature · 1 line
2.40.1.606.ga4b1b128d6-goog
Quoted 22 lines · Yuanchu Xie
The pagewalk in
pagemap_readreads one PTE past the end of the requested range, and stops when the buffer runs out of space. While it produces the right result, the extra read is unnecessary and less performant.I timed the following command before and after this patch:
dd count=100000 if=/proc/self/pagemap of=/dev/nullThe results are consistently within 0.001s across 5 runs.
Before:
100000+0 records in
100000+0 records out
51200000 bytes (51 MB) copied, 0.0763159 s, 671 MB/sreal 0m0.078s user 0m0.012s sys 0m0.065sAfter:
100000+0 records in
100000+0 records out
51200000 bytes (51 MB) copied, 0.0487928 s, 1.0 GB/s
Quoted 5 lines
real 0m0.050s user 0m0.011s sys 0m0.039sSigned-off-by Yuanchu Xie <[email protected]>
Signature · 1 line
Peter Xu
Quoted 22 lines · Yuanchu Xie
The pagewalk in
pagemap_readreads one PTE past the end of the requested range, and stops when the buffer runs out of space. While it produces the right result, the extra read is unnecessary and less performant.I timed the following command before and after this patch:
dd count=100000 if=/proc/self/pagemap of=/dev/nullThe results are consistently within 0.001s across 5 runs.
Before:
100000+0 records in
100000+0 records out
51200000 bytes (51 MB) copied, 0.0763159 s, 671 MB/sreal 0m0.078s user 0m0.012s sys 0m0.065sAfter:
100000+0 records in
100000+0 records out
51200000 bytes (51 MB) copied, 0.0487928 s, 1.0 GB/s
Quoted 5 lines
real 0m0.050s user 0m0.011s sys 0m0.039sSigned-off-by Yuanchu Xie <[email protected]>
Quoted 42 lines
--- fs/proc/task_mmu.c | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-)fs/proc/task_mmu.c +6
-6@@ -1689,23 +1689,23 @@ static ssize_t pagemap_read(struct file *file, char __user *buf,/* watch out for wraparound */ start_vaddr = end_vaddr;if (svpfn <= (ULONG_MAX >>PAGE_SHIFT )) { +unsigned long end; + ret =mmap_read_lock_killable (mm);if (ret) goto out_free; start_vaddr =untagged_addr_remote (mm, svpfn <<PAGE_SHIFT );mmap_read_unlock (mm); + + end = start_vaddr + ((count /PM_ENTRY_BYTES ) <<PAGE_SHIFT ); +if (end >= start_vaddr && end < mm->task_size) + end_vaddr = end; }/* Ensure the address is inside the task */ if (start_vaddr > mm->task_size) start_vaddr = end_vaddr; -/* -* The odds are that this will stop walking way -* before end_vaddr, because the length of the -* user buffer is tracked in "pm", and the walk -* will stop when we hit the end of the buffer. -*/ ret =0 ;while (count && (start_vaddr < end_vaddr)) {int len;Signature · 1 line
2.40.1.606.ga4b1b128d6-goog
Signature · 2 lines
Sincerely yours, Mike.
Quoted 23 lines · Yuanchu Xie
The pagewalk in
pagemap_readreads one PTE past the end of the requested range, and stops when the buffer runs out of space. While it produces the right result, the extra read is unnecessary and less performant.I timed the following command before and after this patch:
dd count=100000 if=/proc/self/pagemap of=/dev/nullThe results are consistently within 0.001s across 5 runs.
Before:
100000+0 records in
100000+0 records out
51200000 bytes (51 MB) copied, 0.0763159 s, 671 MB/sreal 0m0.078s user 0m0.012s sys 0m0.065sAfter:
100000+0 records in
100000+0 records out
51200000 bytes (51 MB) copied, 0.0487928 s, 1.0 GB/s
Quoted 5 lines
real 0m0.050s user 0m0.011s sys 0m0.039sSigned-off-by Yuanchu Xie <[email protected]>
Quoted 41 lines
--- fs/proc/task_mmu.c | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-)fs/proc/task_mmu.c +6
-7@@ -1689,23 +1689,23 @@ static ssize_t pagemap_read(struct file *file, char __user *buf,/* watch out for wraparound */ start_vaddr = end_vaddr;if (svpfn <= (ULONG_MAX >>PAGE_SHIFT )) { +unsigned long end; + ret =mmap_read_lock_killable (mm);if (ret) goto out_free; start_vaddr =untagged_addr_remote (mm, svpfn <<PAGE_SHIFT );mmap_read_unlock (mm); + + end = start_vaddr + ((count /PM_ENTRY_BYTES ) <<PAGE_SHIFT ); +if (end >= start_vaddr && end < mm->task_size) + end_vaddr = end; }/* Ensure the address is inside the task */ if (start_vaddr > mm->task_size) start_vaddr = end_vaddr; -/* -* The odds are that this will stop walking way -* before end_vaddr, because the length of the -* user buffer is tracked in "pm", and the walk -* will stop when we hit the end of the buffer. -*/ ret =0 ;while (count && (start_vaddr < end_vaddr)) {int len; --2.40.1.606.ga4b1b128d6-goog
Quoted 22 lines · Yuanchu Xie
The pagewalk in
pagemap_readreads one PTE past the end of the requested range, and stops when the buffer runs out of space. While it produces the right result, the extra read is unnecessary and less performant.I timed the following command before and after this patch:
dd count=100000 if=/proc/self/pagemap of=/dev/nullThe results are consistently within 0.001s across 5 runs.
Before:
100000+0 records in
100000+0 records out
51200000 bytes (51 MB) copied, 0.0763159 s, 671 MB/sreal 0m0.078s user 0m0.012s sys 0m0.065sAfter:
100000+0 records in
100000+0 records out
51200000 bytes (51 MB) copied, 0.0487928 s, 1.0 GB/s
Quoted 5 lines
real 0m0.050s user 0m0.011s sys 0m0.039sSigned-off-by Yuanchu Xie <[email protected]>