ARM: copy_{from,to}_user() for vmsplit 4g/4g
Subsystems (3)
cover ARM: copy_{from,to}_user() for vmsplit 4g/4g
Hi,
copy_{from,to}_user() uaccess helpers are implemented by user page pinning, followed by temporary kernel mapping & then memcpy(). This helps to achieve user page copy when current virtual address mapping of the CPU excludes user pages.
Other uaccess routines are also planned to be modified to make use of pinning plus kmap_atomic() based on the feedback here.
This is done as one of the initial steps to achieve 4G virtual address mapping for user as well as Kernel on ARMv7 w/ LPAE.
Motive behind this is to enable Kernel access till 4GiB (almost) as lowmem, thus helping in removing highmem support for platforms having upto 4GiB RAM. In the case of platforms having >4GiB, highmem is still required for the Kernel to be able to access whole RAM.
Performance wise, results are not encouraging, 'dd' on tmpfs results,
ARM Cortex-A8, BeagleBone White (256MiB RAM):
w/o series - ~29.5 MB/s
w/ series - ~20.5 MB/s
w/ series & highmem disabled - ~21.2 MB/s
On Cortex-A15(2GiB RAM) in QEMU:
w/o series - ~4 MB/s
w/ series - ~2.6 MB/s
Roughly a one-third drop in performance. Disabling highmem improves performance only slightly.
'hackbench' also showed a similar pattern.
Ways to improve the performance has to be explored, if any one has thoughts on it, please share.
uaccess routines using page pinning & temporary kernel mapping is not something new, it has been done by Ingo long long ago [1] as part of 4G/4G user/kernel mapping implementation on x86, though not merged in mainline.
Arnd has outlined basic design for vmsplit 4g/4g, uaccess routines using user page pinning plus kmap_atomic() is one part of that.
[1] https://lore.kernel.org/lkml/[email protected]/
Last 2 patches are only meant for testing first patch.
Regards
afzal
afzal mohammed (3):
lib: copy_{from,to}_user using gup & kmap_atomic()
ARM: uaccess: let UACCESS_GUP_KMAP_MEMCPY enabling
ARM: provide CONFIG_VMSPLIT_4G_DEV for development
arch/arm/Kconfig | 9 ++ arch/arm/include/asm/uaccess.h | 20 ++++ arch/arm/kernel/armksyms.c | 2 + arch/arm/lib/Makefile | 7 +- lib/Kconfig | 4 + lib/Makefile | 3 + lib/uaccess_gup_kmap_memcpy.c | 162 +++++++++++++++++++++++++++++++++ 7 files changed, 205 insertions(+), 2 deletions(-) create mode 100644 lib/uaccess_gup_kmap_memcpy.c
Signature · 1 line
2.26.2
RFC 1/3 lib: copy_{from,to}_user using gup & kmap_atomic()
copy_{from,to}_user() uaccess helpers are implemented by user page pinning, followed by temporary kernel mapping & then memcpy(). This helps to achieve user page copy when current virtual address mapping of the CPU excludes user pages.
Performance wise, results are not encouraging, 'dd' on tmpfs results,
ARM Cortex-A8, BeagleBone White (256MiB RAM):
w/o series - ~29.5 MB/s
w/ series - ~20.5 MB/s
w/ series & highmem disabled - ~21.2 MB/s
On Cortex-A15(2GiB RAM) in QEMU:
w/o series - ~4 MB/s
w/ series - ~2.6 MB/s
Roughly a one-third drop in performance. Disabling highmem improves performance only slightly.
'hackbench' also showed a similar pattern.
uaccess routines using page pinning & temporary kernel mapping is not something new, it has been done long long ago by Ingo [1] as part of 4G/4G user/kernel mapping implementation on x86, though not merged in mainline.
[1] https://lore.kernel.org/lkml/[email protected]/
--- lib/Kconfig | 4 + lib/Makefile | 3 + lib/uaccess_gup_kmap_memcpy.c | 162 ++++++++++++++++++++++++++++++++++ 3 files changed, 169 insertions(+) create mode 100644 lib/uaccess_gup_kmap_memcpy.c
lib/Kconfig +4 -0
@@ -622,6 +622,10 @@ config ARCH_HAS_MEMREMAP_COMPAT_ALIGN config UACCESS_MEMCPY bool +# pin page + kmap_atomic + memcpy for user copies, intended for vmsplit 4g/4g +config UACCESS_GUP_KMAP_MEMCPY + bool + config ARCH_HAS_UACCESS_FLUSHCACHE bool
lib/Makefile +3 -0
@@ -309,3 +309,6 @@ obj-$(CONFIG_OBJAGG) += objagg.o # KUnit tests obj-$(CONFIG_LIST_KUNIT_TEST) += list-test.o + +# uaccess +obj-$(CONFIG_UACCESS_GUP_KMAP_MEMCPY) += uaccess_gup_kmap_memcpy.o
lib/uaccess_gup_kmap_memcpy.c new file +162 -0
162 lines not on this page · raw message
Signature · 1 line
2.26.2
Re: [RFC 1/3] lib: copy_{from,to}_user using gup & kmap_atomic()
Quoted 20 lines · afzal mohammed
copy_{from,to}_user() uaccess helpers are implemented by user page pinning, followed by temporary kernel mapping & then
memcpy(). This helps to achieve user page copy when current virtual address mapping of the CPU excludes user pages.Performance wise, results are not encouraging, 'dd' on tmpfs results,
ARM Cortex-A8, BeagleBone White (256MiB RAM):
w/o series - ~29.5 MB/s
w/ series - ~20.5 MB/s
w/ series & highmem disabled - ~21.2 MB/sOn Cortex-A15(2GiB RAM) in QEMU:
w/o series - ~4 MB/s
w/ series - ~2.6 MB/sRoughly a one-third drop in performance. Disabling highmem improves performance only slightly.
Quoted 8 lines
'hackbench' also showed a similar pattern.
uaccess routines using page pinning & temporary kernel mapping is not something new, it has been done long long ago by Ingo [1] as part of 4G/4G user/kernel mapping implementation on x86, though not merged in mainline.
Nice changelog text! I agree the performance drop is not great. There are probably some things that can be done to optimize it, but I guess most of the overhead is from the page table operations and cannot be avoided.
What was the exact 'dd' command you used, in particular the block size? Note that by default, 'dd' will request 512 bytes at a time, so you usually only access a single page. It would be interesting to see the overhead with other typical or extreme block sizes, e.g. '1', '64', '4K', '64K' or '1M'.
If you want to drill down into where exactly the overhead is (i.e. get_user_pages or kmap_atomic, or something different), using 'perf record dd ..', and 'perf report' may be helpful.
Quoted 7 lines
lib/uaccess_gup_kmap_memcpy.c new file +7
-0+static int copy_chunk_from_user (unsigned long from,int len,void *arg) +{ +unsigned long *to_ptr = arg, to = *to_ptr; + +memcpy ((void *) to, (void *) from, len); + *to_ptr += len; +return 0 ;
lib/uaccess_gup_kmap_memcpy.c line 90
Quoted 10 lines
lib/uaccess_gup_kmap_memcpy.c new file +10
-0+} + +static int copy_chunk_to_user (unsigned long to,int len,void *arg) +{ +unsigned long *from_ptr = arg, from = *from_ptr; + +memcpy ((void *) to, (void *) from, len); + *from_ptr += len; +return 0 ; +}
Will gcc optimize away the indirect function call and inline everything? If not, that would be a small part of the overhead.
lib/uaccess_gup_kmap_memcpy.c line 103
Quoted 12 lines
lib/uaccess_gup_kmap_memcpy.c new file +12
-0+unsigned long gup_kmap_copy_from_user (void *to,const void __user *from,unsigned long n) +{ +struct page **pages; +int num_pages, ret, i; + +if (uaccess_kernel ()) { +memcpy (to, (__force void *)from, n); +return 0 ; + } + + num_pages =DIV_ROUND_UP ((unsigned long )from + n,PAGE_SIZE ) - + (unsigned long )from /PAGE_SIZE ;
Make sure this doesn't turn into actual division operations but uses shifts. It might even be clearer here to open-code the shift operation so readers can see what this is meant to compile into.
lib/uaccess_gup_kmap_memcpy.c line 106
Quoted 3 lines
lib/uaccess_gup_kmap_memcpy.c new file +3
-0+ pages =kmalloc_array (num_pages,sizeof (*pages),GFP_KERNEL | __GFP_ZERO); +if (!pages) + goto end;
Another micro-optimization may be to avoid the kmalloc for the common case, e.g. anything with "num_pages <= 64", using an array on the stack.
lib/uaccess_gup_kmap_memcpy.c line 115
Quoted 8 lines
lib/uaccess_gup_kmap_memcpy.c new file +8
-0+ ret =get_user_pages_fast ((unsigned long )from, num_pages,0 , pages); +if (ret <0 ) + goto free_pages; + +if (ret != num_pages) { + num_pages = ret; + goto put_pages; + }
I think this is technically incorrect: if get_user_pages_fast() only gets some of the
pages, you should continue with the short buffer and return the number of remaining
bytes rather than not copying anything. I think you did that correctly for a failedkmap_atomic(), but this has to use the same logic.
Arnd
Re: [RFC 1/3] lib: copy_{from,to}_user using gup & kmap_atomic()
Hi,
Quoted 8 lines · Arnd Bergmann
Quoted 2 lines · afzal mohammed
Roughly a one-third drop in performance. Disabling highmem improves performance only slightly.
There are probably some things that can be done to optimize it, but I guess most of the overhead is from the page table operations and cannot be avoided.
Ingo's series did a follow_page() first, then as a fallback did it invoke get_user_pages(), i will try that way as well.
Yes, i too feel get_user_pages_fast() path is the most time consuming, will instrument & check.
Quoted 4 lines
What was the exact 'dd' command you used, in particular the block size? Note that by default, 'dd' will request 512 bytes at a time, so you usually only access a single page. It would be interesting to see the overhead with other typical or extreme block sizes, e.g. '1', '64', '4K', '64K' or '1M'.
It was the default(512), more test results follows (in MB/s),
512 1K 4K 16K 32K 64K 1M
w/o series 30 46 89 95 90 85 65
w/ series 22 36 72 79 78 75 61
perf drop 26% 21% 19% 16% 13% 12% 6%
Hmm, results ain't that bad :)
Quoted 3 lines
If you want to drill down into where exactly the overhead is (i.e.
get_user_pagesorkmap_atomic, or something different), using 'perf record dd ..', and 'perf report' may be helpful.
Let me dig deeper & try to find out where the major overhead and try to figure out ways to reduce it.
One reason to disable highmem & test (results mentioned earlier) was to make kmap_atomic() very lightweight, that was not making much difference, around 3% only.
Quoted 12 lines
Quoted 12 lines
lib/uaccess_gup_kmap_memcpy.c new file +12
-0+static int copy_chunk_from_user (unsigned long from,int len,void *arg) +{ +unsigned long *to_ptr = arg, to = *to_ptr; + +memcpy ((void *) to, (void *) from, len); + *to_ptr += len; +return 0 ; +} + +static int copy_chunk_to_user (unsigned long to,int len,void *arg) +{ +unsigned long *from_ptr = arg, from = *from_ptr;
Quoted 8 lines
lib/uaccess_gup_kmap_memcpy.c line 90
Quoted 5 lines
lib/uaccess_gup_kmap_memcpy.c new file +5
-0+ +memcpy ((void *) to, (void *) from, len); + *from_ptr += len; +return 0 ; +}Will gcc optimize away the indirect function call and inline everything? If not, that would be a small part of the overhead.
i think not, based on objdump, i will make these & wherever other places possible inline & see the difference.
Quoted 6 lines
lib/uaccess_gup_kmap_memcpy.c line 103
Quoted 2 lines
lib/uaccess_gup_kmap_memcpy.c new file +2
-0+ num_pages =DIV_ROUND_UP ((unsigned long )from + n,PAGE_SIZE ) - + (unsigned long )from /PAGE_SIZE ;Make sure this doesn't turn into actual division operations but uses shifts. It might even be clearer here to open-code the shift operation so readers can see what this is meant to compile into.
Okay
Quoted 7 lines
lib/uaccess_gup_kmap_memcpy.c line 106
Quoted 4 lines
lib/uaccess_gup_kmap_memcpy.c new file +3
-0+ pages =kmalloc_array (num_pages,sizeof (*pages),GFP_KERNEL | __GFP_ZERO); +if (!pages) + goto end;Another micro-optimization may be to avoid the kmalloc for the common case, e.g. anything with "
num_pages<= 64", using an array on the stack.
Okay
Quoted 9 lines
lib/uaccess_gup_kmap_memcpy.c line 115
Quoted 8 lines
lib/uaccess_gup_kmap_memcpy.c new file +8
-0+ ret =get_user_pages_fast ((unsigned long )from, num_pages,0 , pages); +if (ret <0 ) + goto free_pages; + +if (ret != num_pages) { + num_pages = ret; + goto put_pages; + }
Quoted 7 lines
I think this is technically incorrect: if
get_user_pages_fast()only gets some of the
pages, you should continue with the short buffer and return the number of remaining
bytes rather than not copying anything. I think you did that correctly for a failedkmap_atomic(), but this has to use the same logic.
yes, will fix that.
Regards
afzal
Re: [RFC 1/3] lib: copy_{from,to}_user using gup & kmap_atomic()
Quoted 6 lines · afzal mohammed
Quoted 4 lines · Arnd Bergmann
Quoted 2 lines · afzal mohammed
Roughly a one-third drop in performance. Disabling highmem improves performance only slightly.
Quoted 6 lines
Quoted 3 lines
There are probably some things that can be done to optimize it, but I guess most of the overhead is from the page table operations and cannot be avoided.
Ingo's series did a
follow_page()first, then as a fallback did it invokeget_user_pages(), i will try that way as well.
Right, that could help, in particular for the small copies. I think a lot of usercopy calls are only for a few bytes, though this is of course highly workload dependent and you might only care about the large ones.
Quoted 12 lines
Yes, i too feel
get_user_pages_fast()path is the most time consuming, will instrument & check.Quoted 4 lines
What was the exact 'dd' command you used, in particular the block size? Note that by default, 'dd' will request 512 bytes at a time, so you usually only access a single page. It would be interesting to see the overhead with other typical or extreme block sizes, e.g. '1', '64', '4K', '64K' or '1M'.
It was the default(512), more test results follows (in MB/s),
512 1K 4K 16K 32K 64K 1M
Quoted 7 lines
w/o series 30 46 89 95 90 85 65
w/ series 22 36 72 79 78 75 61
perf drop 26% 21% 19% 16% 13% 12% 6%
Hmm, results ain't that bad :)
There is also still hope of optimizing small aligned copies like
set_ttbr0(user_ttbr);ldm();set_ttbr0(kernel_ttbr);stm();
which could do e.g. 32 bytes at a time, but with more overhead if you have to loop around it.
Arnd
Re: [RFC 1/3] lib: copy_{from,to}_user using gup & kmap_atomic()
Hi,
Quoted 4 lines · Arnd Bergmann
I think a lot
of usercopy calls are only for a few bytes, though this is of course highly workload dependent and you might only care about the large ones.
Observation is that max. pages reaching copy_{from,to}_user() is 2, observed maximum of n (number of bytes) being 1 page size. i think C library cuts any size read, write to page size (if it exceeds) & invokes the system call. Max. pages reaching 2, happens when 'n' crosses page boundary, this has been observed w/ small size request as well w/ ones of exact page size (but not page aligned).
Even w/ dd of various size >4K, never is the number of pages required to be mapped going greater than 2 (even w/ 'dd' 'bs=1M')
i have a worry (don't know whether it is an unnecessary one): even if we improve performance w/ large copy sizes, it might end up in a sluggishness w.r.t user experience due to most (hence a high amount) of user copy calls being few bytes & there the penalty being higher. And benchmark would not be able to detect anything abnormal since usercopy are being tested on large sizes.
Quickly comparing boot-time on Beagle Bone White, boot time increases by only 4%, perhaps this worry is irrelevant, but just thought will put it across.
Quoted 6 lines
There is also still hope of optimizing small aligned copies like
set_ttbr0(user_ttbr);ldm();set_ttbr0(kernel_ttbr);stm();
Hmm, more needs to be done to be in a position to test it.
Regards
afzal
Re: [RFC 1/3] lib: copy_{from,to}_user using gup & kmap_atomic()
Quoted 17 lines · afzal mohammed
Observation is that max. pages reaching copy_{from,to}_user() is 2, observed maximum of n (number of bytes) being 1 page size. i think C library cuts any size read, write to page size (if it exceeds) & invokes the system call. Max. pages reaching 2, happens when 'n' crosses page boundary, this has been observed w/ small size request as well w/ ones of exact page size (but not page aligned).
Even w/ dd of various size >4K, never is the number of pages required to be mapped going greater than 2 (even w/ 'dd' 'bs=1M')
i have a worry (don't know whether it is an unnecessary one): even if we improve performance w/ large copy sizes, it might end up in a sluggishness w.r.t user experience due to most (hence a high amount) of user copy calls being few bytes & there the penalty being higher. And benchmark would not be able to detect anything abnormal since usercopy are being tested on large sizes.
Quoted 3 lines
Quickly comparing boot-time on Beagle Bone White, boot time increases by only 4%, perhaps this worry is irrelevant, but just thought will put it across.
Do stat(2) of the same tmpfs file in a loop (on tmpfs, to eliminate the filesystem playing silly buggers). And I wouldn't expect anything good there...
Re: [RFC 1/3] lib: copy_{from,to}_user using gup & kmap_atomic()
Quoted 19 lines · Al Viro
Quoted 17 lines · afzal mohammed
Observation is that max. pages reaching copy_{from,to}_user() is 2, observed maximum of n (number of bytes) being 1 page size. i think C library cuts any size read, write to page size (if it exceeds) & invokes the system call. Max. pages reaching 2, happens when 'n' crosses page boundary, this has been observed w/ small size request as well w/ ones of exact page size (but not page aligned).
Even w/ dd of various size >4K, never is the number of pages required to be mapped going greater than 2 (even w/ 'dd' 'bs=1M')
i have a worry (don't know whether it is an unnecessary one): even if we improve performance w/ large copy sizes, it might end up in a sluggishness w.r.t user experience due to most (hence a high amount) of user copy calls being few bytes & there the penalty being higher. And benchmark would not be able to detect anything abnormal since usercopy are being tested on large sizes.
Quoted 7 lines
Quoted 3 lines
Quickly comparing boot-time on Beagle Bone White, boot time increases by only 4%, perhaps this worry is irrelevant, but just thought will put it across.
Do stat(2) of the same tmpfs file in a loop (on tmpfs, to eliminate the filesystem playing silly buggers). And I wouldn't expect anything good there...
Incidentally, what about get_user()/put_user()? _That_ is where it's going to really hurt...
Re: [RFC 1/3] lib: copy_{from,to}_user using gup & kmap_atomic()
Hi,
Quoted 2 lines · Al Viro
Incidentally, what about get_user()/put_user()? _That_ is where it's going to really hurt...
All other uaccess routines are also planned to be added, posting only copy_{from,to}_user() was to get early feedback (mentioned in the cover letter)
Regards
afzal
Re: [RFC 1/3] lib: copy_{from,to}_user using gup & kmap_atomic()
Quoted 10 lines · afzal mohammed
Hi,
Quoted 2 lines · Al Viro
Incidentally, what about get_user()/put_user()? _That_ is where it's going to really hurt...
All other uaccess routines are also planned to be added, posting only copy_{from,to}_user() was to get early feedback (mentioned in the cover letter)
Sure, but what I mean is that I'd expect the performance loss to be dominated by that, not by copy_from_user/copy_to_user on large amounts of data. Especially on the loads like kernel builds - a lot of stat() and getdents() calls there.
Re: [RFC 1/3] lib: copy_{from,to}_user using gup & kmap_atomic()
Quoted 8 lines · Al Viro
Quoted 7 lines · afzal mohammed
Hi,
Quoted 2 lines · Al Viro
Incidentally, what about get_user()/put_user()? _That_ is where it's going to really hurt...
Quoted 8 lines
Quoted 3 lines
All other uaccess routines are also planned to be added, posting only copy_{from,to}_user() was to get early feedback (mentioned in the cover letter)
Sure, but what I mean is that I'd expect the performance loss to be dominated by that, not by copy_from_user/copy_to_user on large amounts of data. Especially on the loads like kernel builds - a lot of
stat()andgetdents()calls there.
To clarify: stat() means small copy_to_user(), getdents() - a mix of put_user() and small copy_to_user(). I would be very surprised if it does not hurt a lot.
Re: [RFC 1/3] lib: copy_{from,to}_user using gup & kmap_atomic()
Quoted 13 lines · Al Viro
Quoted 12 lines · Al Viro
Quoted 10 lines · afzal mohammed
Hi,
Quoted 2 lines · Al Viro
Incidentally, what about get_user()/put_user()? _That_ is where it's going to really hurt...
All other uaccess routines are also planned to be added, posting only copy_{from,to}_user() was to get early feedback (mentioned in the cover letter)
Quoted 8 lines
Quoted 4 lines
Sure, but what I mean is that I'd expect the performance loss to be dominated by that, not by copy_from_user/copy_to_user on large amounts of data. Especially on the loads like kernel builds - a lot of
stat()andgetdents()calls there.To clarify:
stat()means smallcopy_to_user(),getdents()- a mix ofput_user()and smallcopy_to_user(). I would be very surprised if it does not hurt a lot.
PS: there's another fun issue here:
fill a file with zeroes
mmap that file in two areas, MAP_SHARED
thread 1:munmap() the first area
fill the second one with 'X'
thread 2:write() from the first area into pipe
One could expect that nothing by zeroes gets written into pipe - it might be a short write() (or -EFAULT), but finding any 'X' there would be a bug.
Your patches allow for a possibility of write() doing get_user_pages_fast(), getting the first page just as munmap() is about to remove it from page tables and bugger off. Then thread 1 proceeds with the store (via the second area). And then thread 2 does memcpy() from that thing via a kmap_atomic()-created alias, observing the effect of the store.
That might or might not be a POSIX violation, but it does look like a QoI issue...
Re: [RFC 1/3] lib: copy_{from,to}_user using gup & kmap_atomic()
Quoted 45 lines · Al Viro
Quoted 13 lines · Al Viro
Quoted 12 lines · Al Viro
Quoted 10 lines · afzal mohammed
Hi,
Quoted 2 lines · Al Viro
Incidentally, what about get_user()/put_user()? _That_ is where it's going to really hurt...
All other uaccess routines are also planned to be added, posting only copy_{from,to}_user() was to get early feedback (mentioned in the cover letter)
Quoted 8 lines
Quoted 4 lines
Sure, but what I mean is that I'd expect the performance loss to be dominated by that, not by copy_from_user/copy_to_user on large amounts of data. Especially on the loads like kernel builds - a lot of
stat()andgetdents()calls there.To clarify:
stat()means smallcopy_to_user(),getdents()- a mix ofput_user()and smallcopy_to_user(). I would be very surprised if it does not hurt a lot.PS: there's another fun issue here:
fill a file with zeroes
mmap that file in two areas,MAP_SHARED
thread 1:munmap()the first area
fill the second one with 'X'
thread 2:write()from the first area into pipeOne could expect that nothing by zeroes gets written into pipe - it might be a short
write()(or-EFAULT), but finding any 'X' there would be a bug.Your patches allow for a possibility of
write()doingget_user_pages_fast(), getting the first page just asmunmap()is about to remove it from page tables and bugger off. Then thread 1 proceeds with the store (via the second area). And then thread 2 doesmemcpy()from that thing via a kmap_atomic()-created alias, observing the effect of the store.
Quoted 2 lines
That might or might not be a POSIX violation, but it does look like a QoI issue...
I assume this problem exists in arch/um/kernel/skas/uaccess.c and in Ingo's old x86 VMSPLIT_4G_4G patch as well, right?
I guess holding mmap_read_lock() would prevent it but make it even more expensive.
Arnd
RE: [RFC 1/3] lib: copy_{from,to}_user using gup & kmap_atomic()
In reply to Al ViroFrom: Al Viro
Quoted 9 lines
Sent: 13 June 2020 16:31
Quoted 7 lines · afzal mohammed
Hi,
Quoted 2 lines · Al Viro
Incidentally, what about get_user()/put_user()? _That_ is where it's going to really hurt...
Quoted 8 lines
Quoted 3 lines
All other uaccess routines are also planned to be added, posting only copy_{from,to}_user() was to get early feedback (mentioned in the cover letter)
Sure, but what I mean is that I'd expect the performance loss to be dominated by that, not by copy_from_user/copy_to_user on large amounts of data. Especially on the loads like kernel builds - a lot of
stat()andgetdents()calls there.
Or any network traffic where the number of usercopies involved in, for example, sendmsg() gives a measurable performance decrease when HARDENED_USERCOPY is enabled.
Do you have issues with cache aliasing?
(Is aliasing the right term?)
Where potentially the temporary kernel map doesn't use the same cache lines as the user processes map.
I'm not sure what problem you are trying to solve, but for 64bit systems it may be possible to map all of physical memory into the kernel address map, them you (loosely) only have to find the KVA that matches the user-VA to do the copy.
IIRC our SYSV kernels used to do that - until we had 384MB of physical memory and ran out of kernel address space.
David
-
Registered Address Lakeside, Bramley Road, Mount Farm, Milton Keynes, MK1 1PT, UK Registration No: 1397386 (Wales)
Russell King - ARM Linux admin
maintainerRe: [RFC 1/3] lib: copy_{from,to}_user using gup & kmap_atomic()
In reply to afzal mohammedQuoted 9 lines · afzal mohammed
Hi,
Quoted 4 lines · Arnd Bergmann
I think a lot
of usercopy calls are only for a few bytes, though this is of course highly workload dependent and you might only care about the large ones.
Quoted 6 lines
Observation is that max. pages reaching copy_{from,to}_user() is 2, observed maximum of n (number of bytes) being 1 page size. i think C library cuts any size read, write to page size (if it exceeds) & invokes the system call. Max. pages reaching 2, happens when 'n' crosses page boundary, this has been observed w/ small size request as well w/ ones of exact page size (but not page aligned).
You can't make that assumption about read(2). stdio in the C library may read a page size of data at a time, but programs are allowed to call read(2) directly, and the C library will pass such a call straight through to the kernel. So, if userspace requests a 16k read via read(2), then read(2) will be invoked covering 16k.
As an extreme case, for example:
$ strace -e read dd if=/dev/zero of=/dev/null bs=1048576 count=1 read(0, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 1048576) = 1048576
Signature · 2 lines
RMK's Patch system: https://www.armlinux.org.uk/developer/patches/ FTTP is here! 40Mbps down 10Mbps up. Decent connectivity at last!
Re: [RFC 1/3] lib: copy_{from,to}_user using gup & kmap_atomic()
Hi,
Quoted 12 lines · Russell King - ARM Linux admin
Quoted 3 lines · afzal mohammed
i think C
library cuts any size read, write to page size (if it exceeds) & invokes the system call.You can't make that assumption about read(2). stdio in the C library may read a page size of data at a time, but programs are allowed to call read(2) directly, and the C library will pass such a call straight through to the kernel. So, if userspace requests a 16k read via read(2), then read(2) will be invoked covering 16k.
Quoted 4 lines
As an extreme case, for example:
$ strace -e read dd if=/dev/zero of=/dev/null bs=1048576 count=1 read(0, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 1048576) = 1048576
Okay. Yes, observed that dd is passing whatever is the 'bs' to Kernel and from the 'dd' sources (of busybox), it is invoking read system call directly passing 'bs', so it is the tmpfs read that is splitting it to page size as mentioned by Arnd.
Regards
afzal
Re: [RFC 1/3] lib: copy_{from,to}_user using gup & kmap_atomic()
In reply to afzal mohammedQuoted 7 lines · afzal mohammed
Quoted 4 lines · Arnd Bergmann
I think a lot
of usercopy calls are only for a few bytes, though this is of course highly workload dependent and you might only care about the large ones.
Quoted 6 lines
Observation is that max. pages reaching copy_{from,to}_user() is 2, observed maximum of n (number of bytes) being 1 page size. i think C library cuts any size read, write to page size (if it exceeds) & invokes the system call. Max. pages reaching 2, happens when 'n' crosses page boundary, this has been observed w/ small size request as well w/ ones of exact page size (but not page aligned).
Right, this is apparently because tmpfs uses shmem_file_read_iter() to copy the file pages one at a time. generic_file_buffered_read() seems similar, to copying between an aligned kernel page and address in user space that is not page aligned would be an important case to optimize for.
Quoted 3 lines
Quickly comparing boot-time on Beagle Bone White, boot time increases by only 4%, perhaps this worry is irrelevant, but just thought will put it across.
4% boot time increase sounds like a lot, especially if that is only for copy_from_user/copy_to_user. In the end it really depends on how well get_user()/put_user() and small copies can be optimized in the end.
Quoted 8 lines
Quoted 6 lines
There is also still hope of optimizing small aligned copies like
set_ttbr0(user_ttbr);ldm();set_ttbr0(kernel_ttbr);stm();Hmm, more needs to be done to be in a position to test it.
This is going to be highly microarchitecture specific, so anything you test on the Beaglebone's Cortex-A8 might not apply to A7/A15/A17 systems, but if you want to test what the overhead is, you could try changing /dev/zero (or a different chardev like it) to use a series of put_user(0, u32uptr++) in place of whatever it has, and then replace the 'str' instruction with dummy writes to ttbr0 using the value it already has, like:
mcr p15, 0, %0, c2, c0, 0 /* set_ttbr0() */
isb /* prevent speculative access to kernel table */
str %1, [%2],0 /* write 32 bit to user space */
mcr p15, 0, %0, c2, c0, 0 /* set_ttbr0() */
isb /* prevent speculative access to user table */This is obviously going to be very slow compared to the simple store there is today but maybe cheaper than the CONFIG_ARM64_SW_TTBR0_PAN uaccess_en/disable()
on arm64 on a single get_user()/put_user().
It would be interesting to compare it to the overhead of a get_user_page_fast() based implementation. From the numbers you measured, it seems the beaglebone currently needs an extra ~6µs or 3µs per copy_to/from_user() call with your patch, depending on what your benchmark was (MB/s for just reading or writing vs MB/s for copying from one file to another through a user space buffer).
Arnd
Re: [RFC 1/3] lib: copy_{from,to}_user using gup & kmap_atomic()
Quoted 8 lines · Arnd Bergmann
Quoted 6 lines · afzal mohammed
Observation is that max. pages reaching copy_{from,to}_user() is 2, observed maximum of n (number of bytes) being 1 page size. i think C library cuts any size read, write to page size (if it exceeds) & invokes the system call. Max. pages reaching 2, happens when 'n' crosses page boundary, this has been observed w/ small size request as well w/ ones of exact page size (but not page aligned).
Quoted 5 lines
Right, this is apparently because tmpfs uses
shmem_file_read_iter()to copy the file pages one at a time.generic_file_buffered_read()seems similar, to copying between an aligned kernel page and address in user space that is not page aligned would be an important case to optimize for.
This is kind of the nature of the page cache. The kernel doesn't necessarily have contiguous memory in the page cache, so it's going to be split on page boundaries. This is going to change with my THP series (I haven't actually changed generic_file_buffered_read(), but it'll come later).
Quoted 7 lines
Quoted 3 lines
Quickly comparing boot-time on Beagle Bone White, boot time increases by only 4%, perhaps this worry is irrelevant, but just thought will put it across.
4% boot time increase sounds like a lot, especially if that is only for copy_from_user/copy_to_user. In the end it really depends on how well get_user()/put_user() and small copies can be optimized in the end.
The write path should also be paid attention to. Look at generic_perform_write() which first calls iov_iter_fault_in_readable() (for the entire length of the write) and then actually does the copy later with iov_iter_copy_from_user_atomic(). So you're going to want to optimise the case where you access the same pages multiple times.
Re: [RFC 1/3] lib: copy_{from,to}_user using gup & kmap_atomic()
In reply to Arnd BergmannHi,
Quoted 3 lines · Arnd Bergmann
4% boot time increase sounds like a lot, especially if that is only for copy_from_user/copy_to_user. In the end it really depends on how well get_user()/put_user() and small copies can be optimized in the end.
i mentioned the worst case(happened only once), normally it was in the range 2-3%
Quoted 5 lines
From the numbers you
measured, it seems the beaglebone currently needs an extra ~6µs or 3µs per copy_to/from_user() call with your patch, depending on what your benchmark was (MB/s for just reading or writing vs MB/s for copying from one file to another through a user space buffer).
It is MB/s for copying one file to another via user space buffer, i.e. the value coreutils 'dd' shows w/ status=progress (here it was busybox 'dd', so instead it was enabling a compile time option)
Quoted 6 lines
but if you want to test what the overhead is, you could try changing
/dev/zero(or a different chardev like it) to use a series ofput_user(0, u32uptr++)in place of whatever it has, and then replace the 'str' instruction with dummy writes to ttbr0 using the value it already has, like:
Quoted 8 lines
mcr p15, 0, %0, c2, c0, 0 /* set_ttbr0() */ isb /* prevent speculative access to kernel table */ str %1, [%2],0 /* write 32 bit to user space */ mcr p15, 0, %0, c2, c0, 0 /* set_ttbr0() */ isb /* prevent speculative access to user table */It would be interesting to compare it to the overhead of a
get_user_page_fast()based implementation.
i have to relocate & be on quarantine couple of weeks, so i will temporarily stop here, otherwise might end up in roadside.
Reading feedbacks from everyone, some of it i could grasp only bits & pieces, familiarizing more w/ mm & vfs would help me add value better to the goal/discussion. Linus Walleij, if you wish to explore things, feel free, right now don't know how my connectivity would be for next 3 weeks.
Regards
afzal
Re: [RFC 1/3] lib: copy_{from,to}_user using gup & kmap_atomic()
Hi,
Quoted 3 lines · afzal mohammed
It is MB/s for copying one file to another via user space buffer, i.e. the value coreutils 'dd' shows w/ status=progress (here it was busybox 'dd', so instead it was enabling a compile time option)
Just for correctness, status=progress is not required, it's there in the default 3rd line of coreutils 'dd' o/p
Regards
afzal
Re: [RFC 1/3] lib: copy_{from,to}_user using gup & kmap_atomic()
In reply to afzal mohammedQuoted 20 lines · afzal mohammed
copy_{from,to}_user() uaccess helpers are implemented by user page pinning, followed by temporary kernel mapping & then
memcpy(). This helps to achieve user page copy when current virtual address mapping of the CPU excludes user pages.Performance wise, results are not encouraging, 'dd' on tmpfs results,
ARM Cortex-A8, BeagleBone White (256MiB RAM):
w/o series - ~29.5 MB/s
w/ series - ~20.5 MB/s
w/ series & highmem disabled - ~21.2 MB/sOn Cortex-A15(2GiB RAM) in QEMU:
w/o series - ~4 MB/s
w/ series - ~2.6 MB/sRoughly a one-third drop in performance. Disabling highmem improves performance only slightly.
Quoted 8 lines
'hackbench' also showed a similar pattern.
uaccess routines using page pinning & temporary kernel mapping is not something new, it has been done long long ago by Ingo [1] as part of 4G/4G user/kernel mapping implementation on x86, though not merged in mainline.
Some comments (more related to generic things).
...
Quoted 1 line
+// Started from
arch/um/kernel/skas/uaccess.c
Does it mean you will deduplicate it there?
...
lib/uaccess_gup_kmap_memcpy.c line 7
Quoted 4 lines
lib/uaccess_gup_kmap_memcpy.c new file +4
-0+#include <linux/err.h> +#include <linux/slab.h> +#include <linux/highmem.h> +#include <linux/mm.h>
Perhaps ordered?
lib/uaccess_gup_kmap_memcpy.c line 14
Quoted 3 lines
lib/uaccess_gup_kmap_memcpy.c new file +3
-0+static int do_op_one_page (unsigned long addr,int len, +int (*op)(unsigned long addr,int len,void *arg),void *arg, +struct page *page)
Maybe typedef for the func() ?
lib/uaccess_gup_kmap_memcpy.c line 18
Quoted 4 lines
lib/uaccess_gup_kmap_memcpy.c new file +4
-0+{ +int n; + + addr = (unsigned long )kmap_atomic (page) + (addr & ~PAGE_MASK );
I don't remember about this one...
Quoted 4 lines
lib/uaccess_gup_kmap_memcpy.c new file +4
-0+ n = (*op)(addr, len, arg); +kunmap_atomic ((void *)addr); + +return n;
lib/uaccess_gup_kmap_memcpy.c line 31
Quoted 10 lines
lib/uaccess_gup_kmap_memcpy.c new file +9
-0+} + +static long buffer_op (unsigned long addr,int len, +int (*op)(unsigned long ,int ,void *),void *arg, +struct page **pages) +{ +long size, remain, n; + + size =min (PAGE_ALIGN (addr) - addr, (unsigned long ) len);
...but here seems to me you can use helpers (offset_in_page() or how it's called).
Also consider to use macros like PFN_DOWN(), PFN_UP(), etc in your code.
lib/uaccess_gup_kmap_memcpy.c line 38
Quoted 8 lines
lib/uaccess_gup_kmap_memcpy.c new file +7
-0+ remain = len; +if (size ==0 ) + goto page_boundary; + + n =do_op_one_page (addr, size, op, arg, *pages); +if (n !=0 ) { + remain = (n <0 ? remain :0 );
Why duplicate three times (!) this line, if you can move it to under 'out'?
Quoted 11 lines
lib/uaccess_gup_kmap_memcpy.c new file +11
-0+ goto out; + } + + pages++; + addr += size; + remain -= size; + +page_boundary: +if (remain ==0 ) + goto out; +while (addr < ((addr + remain) &PAGE_MASK )) {
lib/uaccess_gup_kmap_memcpy.c line 59
Quoted 10 lines
lib/uaccess_gup_kmap_memcpy.c new file +10
-0+ n =do_op_one_page (addr,PAGE_SIZE , op, arg, *pages); +if (n !=0 ) { + remain = (n <0 ? remain :0 ); + goto out; + } + + pages++; + addr +=PAGE_SIZE ; + remain -=PAGE_SIZE ; + }
Sounds like this can be refactored to iterate over pages rather than addresses.
Quoted 14 lines
lib/uaccess_gup_kmap_memcpy.c new file +13
-0+if (remain ==0 ) + goto out; + + n =do_op_one_page (addr, remain, op, arg, *pages); +if (n !=0 ) { + remain = (n <0 ? remain :0 ); + goto out; + } + +return 0 ; +out: +return remain; +}
...
lib/uaccess_gup_kmap_memcpy.c line 78
Quoted 5 lines
lib/uaccess_gup_kmap_memcpy.c new file +5
-0+static int copy_chunk_from_user (unsigned long from,int len,void *arg) +{ +unsigned long *to_ptr = arg, to = *to_ptr; + +memcpy ((void *) to, (void *) from, len);
What is the point in the casting to void *?
lib/uaccess_gup_kmap_memcpy.c line 88
Quoted 10 lines
lib/uaccess_gup_kmap_memcpy.c new file +10
-0+ *to_ptr += len; +return 0 ; +} + +static int copy_chunk_to_user (unsigned long to,int len,void *arg) +{ +unsigned long *from_ptr = arg, from = *from_ptr; + +memcpy ((void *) to, (void *) from, len); + *from_ptr += len;
Ditto.
Quoted 6 lines
lib/uaccess_gup_kmap_memcpy.c new file +6
-0+return 0 ; +} + +unsigned long gup_kmap_copy_from_user (void *to,const void __user *from,unsigned long n) +{ +struct page **pages;
lib/uaccess_gup_kmap_memcpy.c line 103
Quoted 10 lines
lib/uaccess_gup_kmap_memcpy.c new file +9
-0+int num_pages, ret, i; + +if (uaccess_kernel ()) { +memcpy (to, (__force void *)from, n); +return 0 ; + } + + num_pages =DIV_ROUND_UP ((unsigned long )from + n,PAGE_SIZE ) - + (unsigned long )from /PAGE_SIZE ;
PFN_UP() ?
Quoted 13 lines
lib/uaccess_gup_kmap_memcpy.c new file +13
-0+ pages =kmalloc_array (num_pages,sizeof (*pages),GFP_KERNEL | __GFP_ZERO); +if (!pages) + goto end; + + ret =get_user_pages_fast ((unsigned long )from, num_pages,0 , pages); +if (ret <0 ) + goto free_pages; + +if (ret != num_pages) { + num_pages = ret; + goto put_pages; + } +
lib/uaccess_gup_kmap_memcpy.c line 126
Quoted 10 lines
lib/uaccess_gup_kmap_memcpy.c new file +10
-0+ n =buffer_op ((unsigned long ) from, n, copy_chunk_from_user, &to, pages); + +put_pages: +for (i =0 ; i < num_pages; i++) +put_page (pages[i]); +free_pages: +kfree (pages); +end: +return n; +}
...
I think you can clean up the code a bit after you will get the main functionality working.
Signature · 2 lines
With Best Regards, Andy Shevchenko
Re: [RFC 1/3] lib: copy_{from,to}_user using gup & kmap_atomic()
Hi,
Quoted 5 lines · Andy Shevchenko
Quoted 1 line · afzal mohammed
+// Started from
arch/um/kernel/skas/uaccess.cDoes it mean you will deduplicate it there?
What i meant was, that file was taken as a template & nothing more, at same time i wanted to give credit to that file, i will explicitly mention it next time.
It is not meant to deduplicate it. Functionality here is completely different.
In the case here, there would be different virtual address mapping that CPU will be see once in Kernel as compared to user mode.
Here a facility is provided to access the user page, when the current virtual address mapping of the CPU excludes it. This is for providing full 4G virtual address to both user & kernel on 32bit ARM to avoid using highmem or reduce the impact of highmem, i.e. so that Kernel can address till 4GiB (almost) as lowmem.
Here assumption is that user mapping is not a subset of virtual address mapped by CPU, but a separate one. Upon Kernel entry ttbr0 is changed to Kernel lowmem, while upon Kernel exit is changed back to user pages (ttbrx in ARM, iiuc, equivalent to cr3 in x86)
Now realize that i am unable to put coherently the problem being attempted to solve here to a person not familiar w/ the issue w/o taking considerable time. If above explanation is not enough, i will try to explain later in a better way.
Quoted 6 lines
lib/uaccess_gup_kmap_memcpy.c line 7
Quoted 4 lines
lib/uaccess_gup_kmap_memcpy.c new file +4
-0+#include <linux/err.h> +#include <linux/slab.h> +#include <linux/highmem.h> +#include <linux/mm.h> Perhaps ordered?
will take care
Quoted 5 lines
lib/uaccess_gup_kmap_memcpy.c line 14
Quoted 3 lines
lib/uaccess_gup_kmap_memcpy.c new file +3
-0+static int do_op_one_page (unsigned long addr,int len, +int (*op)(unsigned long addr,int len,void *arg),void *arg, +struct page *page)Maybe typedef for the
func()?
will take care
Quoted 6 lines
lib/uaccess_gup_kmap_memcpy.c line 18
Quoted 4 lines
lib/uaccess_gup_kmap_memcpy.c new file +4
-0+{ +int n; + + addr = (unsigned long )kmap_atomic (page) + (addr & ~PAGE_MASK );I don't remember about this one...
i am not following you here, for my case !CONFIG_64BIT case in that file was required, hence only it was picked (or rather not deleted)
Quoted 4 lines
Quoted 1 line
+ size =
min(PAGE_ALIGN(addr) - addr, (unsigned long) len);...but here seems to me you can use helpers (
offset_in_page()or how it's called).
i was not aware of it, will use it as required.
Quoted 2 lines
Also consider to use macros like
PFN_DOWN(),PFN_UP(), etc in your code.
Okay
Quoted 11 lines
lib/uaccess_gup_kmap_memcpy.c line 38
Quoted 9 lines
lib/uaccess_gup_kmap_memcpy.c new file +7
-0+ remain = len; +if (size ==0 ) + goto page_boundary; + + n =do_op_one_page (addr, size, op, arg, *pages); +if (n !=0 ) { + remain = (n <0 ? remain :0 );Why duplicate three times (!) this line, if you can move it to under 'out'?
yes better to move there
Quoted 16 lines
Quoted 16 lines
lib/uaccess_gup_kmap_memcpy.c new file +15
-0+ goto out; + } + + pages++; + addr += size; + remain -= size; + +page_boundary: +if (remain ==0 ) + goto out; +while (addr < ((addr + remain) &PAGE_MASK )) { + n =do_op_one_page (addr,PAGE_SIZE , op, arg, *pages); +if (n !=0 ) { + remain = (n <0 ? remain :0 ); + goto out;
Quoted 8 lines
lib/uaccess_gup_kmap_memcpy.c line 59
Quoted 6 lines
lib/uaccess_gup_kmap_memcpy.c new file +6
-0+ } + + pages++; + addr +=PAGE_SIZE ; + remain -=PAGE_SIZE ; + }Sounds like this can be refactored to iterate over pages rather than addresses.
Okay, i will check
Quoted 7 lines
lib/uaccess_gup_kmap_memcpy.c line 78
Quoted 5 lines
lib/uaccess_gup_kmap_memcpy.c new file +5
-0+static int copy_chunk_from_user (unsigned long from,int len,void *arg) +{ +unsigned long *to_ptr = arg, to = *to_ptr; + +memcpy ((void *) to, (void *) from, len);What is the point in the casting to void *?
The reason it was there was because of copy-paste :), passing unsigned long as 'void *' or 'const void *' requires casting right ?, or you meant something else ?
now i checked removing the cast, compiler is abusing me :), says 'makes pointer from integer without a cast'
Quoted 4 lines
lib/uaccess_gup_kmap_memcpy.c line 103
Quoted 2 lines
lib/uaccess_gup_kmap_memcpy.c new file +2
-0+ num_pages =DIV_ROUND_UP ((unsigned long )from + n,PAGE_SIZE ) - + (unsigned long )from /PAGE_SIZE ;
PFN_UP()?
Okay
Quoted 2 lines
I think you can clean up the code a bit after you will get the main functionality working.
Yes, surely, intention was to post proof-of-concept ASAP, perhaps contents will change drastically in next version so that any resemblence of arch/um/kernel/skas/uaccess.c might not be there.
Regards
afzal
RFC 2/3 ARM: uaccess: let UACCESS_GUP_KMAP_MEMCPY enabling
In reply to afzal mohammedTurn off existing raw_copy_{from,to}_user() using arm_copy_{from,to}_user() when CONFIG_UACCESS_GUP_KMAP_MEMCPY is enabled.
--- arch/arm/include/asm/uaccess.h | 20 ++++++++++++++++++++ arch/arm/kernel/armksyms.c | 2 ++ arch/arm/lib/Makefile | 7 +++++-- 3 files changed, 27 insertions(+), 2 deletions(-)
arch/arm/include/asm/uaccess.h +20 -0
@@ -512,6 +512,15 @@ do { \extern unsigned long __must_check arm_copy_from_user (void *to,const void __user *from,unsigned long n); +#ifdef CONFIG_UACCESS_GUP_KMAP_MEMCPY +extern unsigned long __must_check +gup_kmap_copy_from_user (void *to,const void __user *from,unsigned long n); +static inline __must_check unsigned long +raw_copy_from_user (void *to,const void __user *from,unsigned long n) +{ +return gup_kmap_copy_from_user (to, from, n); +} +#else static inline unsigned long __must_check raw_copy_from_user (void *to,const void __user *from,unsigned long n) {
@@ -522,12 +531,22 @@ raw_copy_from_user(void *to, const void __user *from, unsigned long n)uaccess_restore (__ua_flags);return n; } +#endif extern unsigned long __must_check arm_copy_to_user (void __user *to,const void *from,unsigned long n);extern unsigned long __must_check __copy_to_user_std (void __user *to,const void *from,unsigned long n); +#ifdef CONFIG_UACCESS_GUP_KMAP_MEMCPY +extern unsigned long __must_check +gup_kmap_copy_to_user (void __user *to,const void *from,unsigned long n); +static inline __must_check unsigned long +raw_copy_to_user (void __user *to,const void *from,unsigned long n) +{ +return gup_kmap_copy_to_user (to, from, n); +} +#else static inline unsigned long __must_check raw_copy_to_user (void __user *to,const void *from,unsigned long n) {
@@ -541,6 +560,7 @@ raw_copy_to_user(void __user *to, const void *from, unsigned long n)return arm_copy_to_user (to, from, n); #endif } +#endif extern unsigned long __must_check arm_clear_user (void __user *addr,unsigned long n);
arch/arm/kernel/armksyms.c +2 -0
@@ -96,8 +96,10 @@ EXPORT_SYMBOL(mmiocpy);#ifdef CONFIG_MMU EXPORT_SYMBOL (copy_page); +#ifndef CONFIG_UACCESS_GUP_KMAP_MEMCPY EXPORT_SYMBOL (arm_copy_from_user);EXPORT_SYMBOL (arm_copy_to_user); +#endif EXPORT_SYMBOL (arm_clear_user);EXPORT_SYMBOL (__get_user_1);
arch/arm/lib/Makefile +5 -2
@@ -16,8 +16,11 @@ lib-y := changebit.o csumipv6.o csumpartial.o \ io-readsb.o io-writesb.o io-readsl.o io-writesl.o \ call_with_stack.o bswapsdi2.o -mmu-y := clear_user.o copy_page.o getuser.o putuser.o \ - copy_from_user.o copy_to_user.o +mmu-y := clear_user.o copy_page.o getuser.o putuser.o + +ifndef CONFIG_UACCESS_GUP_KMAP_MEMCPY + mmu-y += copy_from_user.o copy_to_user.o +endif ifdef CONFIG_CC_IS_CLANG lib-y += backtrace-clang.o
Signature · 1 line
2.26.2
RFC 3/3 ARM: provide CONFIG_VMSPLIT_4G_DEV for development
In reply to afzal mohammedSelect UACCESS_GUP_KMAP_MEMCPY initially.
--- arch/arm/Kconfig | 9 +++++++++ 1 file changed, 9 insertions(+)
arch/arm/Kconfig +9 -0
@@ -1326,6 +1326,15 @@ config PAGE_OFFSET default 0xB0000000 if VMSPLIT_3G_OPT default 0xC0000000 +config VMSPLIT_4G_DEV + bool "Experimental changes for 4G/4G user/kernel split" + depends on ARM_LPAE + select UACCESS_GUP_KMAP_MEMCPY + help + Experimental changes during 4G/4G user/kernel split development. + Existing vmsplit config option is used, once development is done, + this would be put as a new choice & _DEV suffix removed. + config NR_CPUS int "Maximum number of CPUs (2-32)" range 2 32
Signature · 1 line
2.26.2
Quoted 7 lines · afzal mohammed
Performance wise, results are not encouraging, 'dd' on tmpfs results,
ARM Cortex-A8, BeagleBone White (256MiB RAM):
w/o series - ~29.5 MB/s
w/ series - ~20.5 MB/s
w/ series & highmem disabled - ~21.2 MB/s
Quoted 6 lines
On Cortex-A15(2GiB RAM) in QEMU:
w/o series - ~4 MB/s
w/ series - ~2.6 MB/sRoughly a one-third drop in performance. Disabling highmem improves performance only slightly.
Could you compare with CONFIG_UACCESS_WITH_MEMCPY as well?
Nicolas
Hi,
Quoted 5 lines · Nicolas Pitre
Quoted 1 line · afzal mohammed
Performance wise, results are not encouraging, 'dd' on tmpfs results,
Could you compare with
CONFIG_UACCESS_WITH_MEMCPYas well?
512 1K 4K 16K 32K 64K 1M
normal 30 46 89 95 90 85 65
uaccess_w_memcpy 28.5 45 85 92 91 85 65
w/ series 22 36 72 79 78 75 61
There are variations in the range +/-2 in some readings when repeated, not put above, to keep comparison simple.
Regards
afzal
Hi,
Quoted 7 lines · afzal mohammed
512 1K 4K 16K 32K 64K 1M
normal 30 46 89 95 90 85 65
uaccess_w_memcpy28.5 45 85 92 91 85 65w/ series 22 36 72 79 78 75 61
For the sake of completeness all in MB/s, w/ various 'dd' 'bs' sizes.
Regards
afzal