ioremap vs remap_pfn_range, VMSPLIT, etc
Hello everyone,
Yesterday, I used /dev/mem to mmap 2 GB and (to my surprise) it worked. Specifically, I opened /dev/mem O_RDWR | O_SYNC
then called
mmap(NULL, 1U<<31, PROT_WRITE, MAP_SHARED, fd, 0x80000000);
And mmap returned a valid pointer.
I was expecting it to fail.
- the kernel is configured with VMSPLIT_3G (3G/1G user/kernel)
- the kernel manages 256 MB RAM
- there is roughly 750 MB of VMALLOC space, no highmem
I know /dev/mem's mmap calls remap_pfn_range, but I was expecting the VMALLOC space to impose a limit on the size of the mapping. Obviously this was incorrect?
If I requested the same mapping *within the kernel* using ioremap, would that fail because of limited VMALLOC space?
Moving to arch-specific questions (namely ARM Cortex-A9). If I understand correctly (which is very possibly NOT the case) the CPU has two registers pointing to page tables, one for the current process, one for the kernel. And the CPU automatically picks the correct one, based on the active context?
It would seem possible to have a full 4G for process, and a full 4G for the kernel, using that method, no? (Like Ingo's old 4G/4G split). Without the performance overhead of fiddling with the page tables. What am I missing?
Regards.
Quoted 6 lines · Mason
Hello everyone,
Yesterday, I used
/dev/memto mmap 2 GB and (to my surprise) it worked. Specifically, I opened/dev/memO_RDWR|O_SYNC
then called
mmap(NULL, 1U<<31, PROT_WRITE, MAP_SHARED, fd, 0x80000000);
So you asked to map 2GB starting@2GB physical.
Quoted 1 line
And mmap returned a valid pointer.
And that mapping would have been created to map physical addresses 0x80000000-0xffffffff inclusive.
Quoted 3 lines
I was expecting it to fail.
- the kernel is configured with
VMSPLIT_3G(3G/1G user/kernel)
This has no bearing on the above.
Quoted 2 lines
- the kernel manages 256 MB RAM
- there is roughly 750 MB of VMALLOC space, no highmem
vmalloc has no bearing on the above, mmap() doesn't allocate anything into vmalloc space.
Quoted 2 lines
If I requested the same mapping *within the kernel* using ioremap, would that fail because of limited VMALLOC space?
Correct.
Quoted 9 lines
Moving to arch-specific questions (namely ARM Cortex-A9). If I understand correctly (which is very possibly NOT the case) the CPU has two registers pointing to page tables, one for the current process, one for the kernel. And the CPU automatically picks the correct one, based on the active context?
It would seem possible to have a full 4G for process, and a full 4G for the kernel, using that method, no? (Like Ingo's old 4G/4G split). Without the performance overhead of fiddling with the page tables. What am I missing?
It's possible to use both, but the CPU selects the page table register according to the virtual address. So it's not possible to have 4G for both. There's only a restricted set of options: 2G / 2G, where the bottom 2G of virtual space uses TTBR0 and the upper 2G uses TTBR1. 1G / 3G (1G for TTBR0, 3G for TTBR1).
We don't use it because most people run with 3G for userspace, which isn't supported in hardware.
Signature · 2 lines
FTTC broadband for 0.8mile line: currently at 10.5Mbps down 400kbps up according to speedtest.net.
Quoted 11 lines · Russell King - ARM Linux
Quoted 4 lines · Mason
Yesterday, I used
/dev/memto mmap 2 GB and (to my surprise) it worked. Specifically, I opened/dev/memO_RDWR|O_SYNC
then called
mmap(NULL, 1U<<31, PROT_WRITE, MAP_SHARED, fd, 0x80000000);So you asked to map 2GB starting at 2GB physical.
Quoted 1 line
And mmap returned a valid pointer.
Quoted 8 lines
And that mapping would have been created to map physical addresses 0x80000000-0xffffffff inclusive.
Quoted 3 lines
I was expecting it to fail.
- the kernel is configured with
VMSPLIT_3G(3G/1G user/kernel)This has no bearing on the above.
I don't understand why.
mmap allocates virtual addresses in the user-space process, yes? So if I had VMSPLIT_2G, user-space processes would be limited to 2G virtual addresses, and could not create a single 2G map on top of its stack and text space. Or am I missing something?
Quoted 5 lines
Quoted 2 lines
- the kernel manages 256 MB RAM
- there is roughly 750 MB of VMALLOC space, no highmemvmalloc has no bearing on the above,
mmap()doesn't allocate anything into vmalloc space.
This means remap_pfn_range doesn't "put" anything in the kernel's virtual address space.
Quoted 4 lines
Quoted 2 lines
If I requested the same mapping *within the kernel* using ioremap, would that fail because of limited VMALLOC space?
Correct.
OK.
Quoted 10 lines
Quoted 9 lines
Moving to arch-specific questions (namely ARM Cortex-A9). If I understand correctly (which is very possibly NOT the case) the CPU has two registers pointing to page tables, one for the current process, one for the kernel. And the CPU automatically picks the correct one, based on the active context?
It would seem possible to have a full 4G for process, and a full 4G for the kernel, using that method, no? (Like Ingo's old 4G/4G split). Without the performance overhead of fiddling with the page tables. What am I missing?
Quoted 8 lines
It's possible to use both, but the CPU selects the page table register according to the virtual address. So it's not possible to have 4G for both. There's only a restricted set of options: 2G / 2G, where the bottom 2G of virtual space uses TTBR0 and the upper 2G uses TTBR1. 1G / 3G (1G for TTBR0, 3G for TTBR1).
We don't use it because most people run with 3G for userspace, which isn't supported in hardware.
I see. Thanks for spelling it out.
Regards.
Quoted 20 lines · Mason
Quoted 18 lines · Russell King - ARM Linux
Quoted 4 lines · Mason
Yesterday, I used
/dev/memto mmap 2 GB and (to my surprise) it worked. Specifically, I opened/dev/memO_RDWR|O_SYNC
then called
mmap(NULL, 1U<<31, PROT_WRITE, MAP_SHARED, fd, 0x80000000);So you asked to map 2GB starting at 2GB physical.
Quoted 1 line
And mmap returned a valid pointer.
And that mapping would have been created to map physical addresses 0x80000000-0xffffffff inclusive.
Quoted 3 lines
I was expecting it to fail.
- the kernel is configured with
VMSPLIT_3G(3G/1G user/kernel)
Quoted 9 lines
Quoted 1 line
This has no bearing on the above.
I don't understand why.
mmap allocates virtual addresses in the user-space process, yes? So if I had
VMSPLIT_2G, user-space processes would be limited to 2G virtual addresses, and could not create a single 2G map on top of its stack and text space. Or am I missing something?
Because you are mmaping special file (dev/mem) mmap call is routed to the dedicated hook, responsible for all "magic" you see. Please, take a look at drivers/char/mem.c for details.
Vladimir
Quoted 46 lines
Quoted 5 lines
Quoted 2 lines
- the kernel manages 256 MB RAM
- there is roughly 750 MB of VMALLOC space, no highmemvmalloc has no bearing on the above,
mmap()doesn't allocate anything into vmalloc space.This means
remap_pfn_rangedoesn't "put" anything in the kernel's virtual address space.Quoted 4 lines
Quoted 2 lines
If I requested the same mapping *within the kernel* using ioremap, would that fail because of limited VMALLOC space?
Correct.
OK.
Quoted 10 lines
Quoted 9 lines
Moving to arch-specific questions (namely ARM Cortex-A9). If I understand correctly (which is very possibly NOT the case) the CPU has two registers pointing to page tables, one for the current process, one for the kernel. And the CPU automatically picks the correct one, based on the active context?
It would seem possible to have a full 4G for process, and a full 4G for the kernel, using that method, no? (Like Ingo's old 4G/4G split). Without the performance overhead of fiddling with the page tables. What am I missing?Quoted 8 lines
It's possible to use both, but the CPU selects the page table register according to the virtual address. So it's not possible to have 4G for both. There's only a restricted set of options: 2G / 2G, where the bottom 2G of virtual space uses TTBR0 and the upper 2G uses TTBR1. 1G / 3G (1G for TTBR0, 3G for TTBR1).
We don't use it because most people run with 3G for userspace, which isn't supported in hardware.
I see. Thanks for spelling it out.
Regards.
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel at lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
Hello Vladimir,
Quoted 25 lines · Vladimir Murzin
Quoted 24 lines · Mason
Quoted 11 lines · Russell King - ARM Linux
Quoted 4 lines · Mason
Yesterday, I used
/dev/memto mmap 2 GB and (to my surprise) it worked. Specifically, I opened/dev/memO_RDWR|O_SYNC
then called
mmap(NULL, 1U<<31, PROT_WRITE, MAP_SHARED, fd, 0x80000000);So you asked to map 2GB starting at 2GB physical.
Quoted 1 line
And mmap returned a valid pointer.
Quoted 8 lines
And that mapping would have been created to map physical addresses 0x80000000-0xffffffff inclusive.
Quoted 3 lines
I was expecting it to fail.
- the kernel is configured with
VMSPLIT_3G(3G/1G user/kernel)This has no bearing on the above.
I don't understand why.
Quoted 8 lines
Quoted 4 lines
mmap allocates virtual addresses in the user-space process, yes? So if I had
VMSPLIT_2G, user-space processes would be limited to 2G virtual addresses, and could not create a single 2G map on top of its stack and text space. Or am I missing something?Because you are mmaping special file (dev/mem) mmap call is routed to the dedicated hook, responsible for all "magic" you see. Please, take a look at
drivers/char/mem.cfor details.
Errr, I thought it was clear I had read the source ;-) ("I know /dev/mem's mmap calls remap_pfn_range [...]")
http://lxr.free-electrons.com/source/drivers/char/mem.c#L307
Hence my ioremap vs remap_pfn_range subject ;-)
So are you saying I could use remap_pfn_range to map the full 4G of PA space into a process's VA space?
Or, if I picked the VMSPLIT_2G option, are you saying mmap'ing 2G would succeed? (I will test these two scenarios ASAP.)
Regards.
Quoted 7 lines · Mason
Quoted 5 lines · Russell King - ARM Linux
Quoted 3 lines · Mason
I was expecting it to fail.
- the kernel is configured with
VMSPLIT_3G(3G/1G user/kernel)
Quoted 8 lines
Quoted 1 line
This has no bearing on the above.
I don't understand why.
mmap allocates virtual addresses in the user-space process, yes? So if I had
VMSPLIT_2G, user-space processes would be limited to 2G virtual addresses, and could not create a single 2G map on top of its stack and text space. Or am I missing something?
I explained in the paragraph you quoted from me below:
Quoted 8 lines
Quoted 5 lines
Quoted 2 lines
- the kernel manages 256 MB RAM
- there is roughly 750 MB of VMALLOC space, no highmemvmalloc has no bearing on the above,
mmap()doesn't allocate anything into vmalloc space.This means
remap_pfn_rangedoesn't "put" anything in the kernel's virtual address space.
Correct.
To answer a query you had elsewhere, if you have the 2G/2G VM split, then no, you can't map 2G, because you will already have userspace mappings (eg, for the stack, and program text) which will consume some of the 2G.
So, asking mmap() to make 2G will fail, because it won't fit in the space available to the user program.
Signature · 2 lines
FTTC broadband for 0.8mile line: currently at 10.5Mbps down 400kbps up according to speedtest.net.