kernel page table mapping for >1GB <3 GB for x86 arch without PAE
Assuming I have 3GB RAM, why does not kernel create page tables in a way that all of this 3GB physical memory can be addressed . (for x86 arch 32 bit with PAE disabled )
I understand the 3GB/1GB split for user/kernel address space.Isn't this just a matter of setting up the page table entries in such a way that so all the available RAM upto 3GB can mapped in the kernel page tables. In this way CPU's MMU can take care of mapping any virtual address within first 4GB to physical address using the page table entries regardless of a process is in kernel/user mode.
Why can't some of first 768 page directory entries ( + 1024 page table
entries for each PDE) be used to map btw 1GB and 3GB. What is stopping the kernel from doing this?
Instead why does kernel uses high memory mapping with zone model for addressing physical memory above 1GB?
Can anyone please shed some light into this,
Thanks,
Quoted 4 lines
Assuming I have 3GB RAM, why does not kernel create page tables in a way that all of this 3GB physical memory can be addressed . (for x86 arch 32 bit with PAE disabled )
I understand the 3GB/1GB split for user/kernel address space.Isn't this just a matter of setting up the page table entries in such a way that so all the available RAM upto 3GB can mapped in the kernel page tables. In this way CPU's MMU can take care of mapping any virtual address within first 4GB to physical address using the page table entries regardless of a process is in kernel/user mode.
On a 32-bit architecture, there is only 4GB of address space available. If the kernel uses 3GB of address space for a direct mapping of physical memory, then only 1GB is available for everything else (userspace, vmalloc, etc). BTW, this is available by enabling CONFIG_VMSPLIT_1G.
- R.
Thanks for the reply,
But my question is with regard to setting up the page tables during the kernel initialization in such a way so that entire RAM upto 4GB can be mapped. i.e virtual to physical address mapping. My question is strictly with reg. to physical address mapping only.
Why does kernel stop at 1GB (896 MB to be exact) while mapping from linear to physical address.
Since a given process runs in both user/kernel mode how does it matter whether page table mapping is for entire 4GB or 1 GB. Just because kernel setup the page tables does not mean that allocated dynamic memory cannot be accessed by given process while running in user mode.
For the sake of discussion, let us say I have 3.5 GB of RAM of x86 with PAE disabled and I have only one process running other than kernel.
Why can't kernel simply setup the page table mapping in a way that entire 3.5GB is mapped from linear to physical during the initialization. Assuming this single process is given access upto 3 GB, while running in user mode this single process can access any memory location in the allocated 2 or 3 GB virtual space.
Quoted 29 lines · Roland Dreier
From: Roland Dreier <[email protected]>
Subject: Re: kernel page table mapping for >1GB <3 GB for x86 arch without PAE
To: "joe Shmoe" <[email protected]>
Cc: [email protected]
Date: Friday, August 28, 2009, 2:12 PMQuoted 1 line
Assuming I have 3GB RAM, why does not kernel create
page tables in a way that all of this 3GB physical memory can be addressed . (for x86 arch 32 bit with PAE disabled )
Quoted 2 lines
I understand the 3GB/1GB split for user/kernel
address space.Isn't this just a matter of setting up the page table entries in such a way that so all the available RAM upto 3GB can mapped in the kernel page tables. In this way CPU's MMU can take care of mapping any virtual address within first 4GB to physical address using the page table entries regardless of a process is in kernel/user mode.
On a 32-bit architecture, there is only 4GB of address space available.
If the kernel uses 3GB of address space for a direct mapping of physical
memory, then only 1GB is available for everything else (userspace,
vmalloc, etc). BTW, this is available by enablingCONFIG_VMSPLIT_1G.- R.
Quoted 2 lines · joe Shmoe
For the sake of discussion, let us say I have 3.5 GB of RAM of x86 with PAE disabled and I have only one process running other than kernel.
Oh, so an embedded environment. Sure, you could play games with that.
But most Linux systems have more processes running than just 'init'. And in that case, what do you do with your memory map? It gets a bit more complicated then....
No I am not talking about embedded systems. It is just an example.
My question is what is preventing the kernel from setting up the page tables so that entire RAM can be mapped upto 4 GB.
page tables have to be setup for every process anyway with first 768 PDE entries being unique and remaining 256 entries being same for all the processes.
Quoted 23 lines · --- On Fri, 8/28/09, [email protected] <[email protected]> wrote:
From: [email protected] <[email protected]>
Subject: Re: kernel page table mapping for >1GB <3 GB for x86 arch without PAE
To: "joe Shmoe" <[email protected]>
Cc: "Roland Dreier" <[email protected]>, [email protected]
Date: Friday, August 28, 2009, 3:13 PMQuoted 1 line · joe Shmoe
For the sake of discussion, let us say I have 3.5 GB
of RAM of x86 with PAE
Quoted 1 line
disabled and I have only one process running other
than kernel.
Oh, so an embedded environment. Sure, you could play games with that.
But most Linux systems have more processes running than just 'init'. And
in that case, what do you do with your memory map? It gets a bit more
complicated then....
Quoted 4 lines
But my question is with regard to setting up the page tables during the kernel initialization in such a way so that entire RAM upto 4GB can be mapped. i.e virtual to physical address mapping. My question is strictly with reg. to physical address mapping only.
Why does kernel stop at 1GB (896 MB to be exact) while mapping from linear to physical address.
Because it wants to be able to give userspace 3GB of address space. And userspace mapping cannot be a linear mapping to physical memory -- it must be real virtual memory that has proper access permissions and can handle mmap'ed files, shared libraries, etc etc. So userspace cannot share the kernel's linear mapping.
Quoted 2 lines · On Fri, 28 Aug 2009 12:44:25 -0700 (PDT) joe Shmoe <[email protected]> wrote:
No I am not talking about embedded systems. It is just an example.
My question is what is preventing the kernel from setting up the page tables so that entire RAM can be mapped upto 4 GB.
The design of the x86 processor
You only have 4GB of virtual address space at any one time so you have to fit the following within your 4GB virtual space
- Virtual addresses of the user space
- Mappings for physical memory
- Kernel virtual spaces (vmalloc etc)
- Various miscellaneous kernel mappings
- I/O MMIO space
Now you could in theory switch the page tables on kernel entry to get a very big user space mapping and then instead of relying on the user space being mapped you could map specific user pages or access them via their physical address lookup.
That however means reloading %cr3 (page table base) which is very expensive, and some very hard to implement locking on user space access.
Some other processor families have the notion of "spaces" or MMU bypassing. The Sparc64 for example can access MMIO space without an MMU mapping or virtual address being needed for it. x86 does not - it has segments but the segments map into the 4GB space so they don't help here hence the current implementation and 3G/1G split.
Quoted 2 lines
That however means reloading %cr3 (page table base) which is very expensive, and some very hard to implement locking on user space access
I understand the implications of reloading CR3. But once the page tables are setup to map all the available physical RAM to virtual (linear) address it could be for eg. 1, 2, 3 or 4 GB how does it matter.
Now If a process wants to access a memory location which is above 1GB , since kernel already setup the page tables CPU MMU will do the translation since it consults this page table setup by kernel. Of course it also makes sure the page table has appropriate privilege level for access.
Am I missing something here?
Quoted 51 lines · Alan Cox
From: Alan Cox <[email protected]>
Subject: Re: kernel page table mapping for >1GB <3 GB for x86 arch without PAE
To: "joe Shmoe" <[email protected]>
Cc: [email protected], [email protected]
Date: Friday, August 28, 2009, 3:53 PM
On Fri, 28 Aug 2009 12:44:25 -0700
(PDT)
joe Shmoe <[email protected]>Quoted 1 line · wrote:
No I am not talking about embedded systems. It is just
an example.
Quoted 1 line
My question is what is preventing the kernel from
setting up the page tables so that entire RAM can be mapped upto 4 GB.
The design of the x86 processor
You only have 4GB of virtual address space at any one time so you have to
fit the following within your 4GB virtual space- Virtual addresses of the user space
- Mappings for physical memory
- Kernel virtual spaces (vmalloc etc)
- Various miscellaneous kernel mappings
- I/O MMIO spaceNow you could in theory switch the page tables on kernel entry to get a
very big user space mapping and then instead of relying on the user space
being mapped you could map specific user pages or access them via their
physical address lookup.That however means reloading %cr3 (page table base) which is very
expensive, and some very hard to implement locking on user space access.Some other processor families have the notion of "spaces" or MMU
bypassing. The Sparc64 for example can access MMIO space without an MMU
mapping or virtual address being needed for it. x86 does not - it has
segments but the segments map into the 4GB space so they don't help here
hence the current implementation and 3G/1G split.
doesn't CR3 gets reloaded for process switch? If so , how does it matter if you setup page table mapping for all the available RAM instead of limiting to just 1GB
Quoted 51 lines · Alan Cox
From: Alan Cox <[email protected]>
Subject: Re: kernel page table mapping for >1GB <3 GB for x86 arch without PAE
To: "joe Shmoe" <[email protected]>
Cc: [email protected], [email protected]
Date: Friday, August 28, 2009, 3:53 PM
On Fri, 28 Aug 2009 12:44:25 -0700
(PDT)
joe Shmoe <[email protected]>Quoted 1 line · wrote:
No I am not talking about embedded systems. It is just
an example.
Quoted 1 line
My question is what is preventing the kernel from
setting up the page tables so that entire RAM can be mapped upto 4 GB.
The design of the x86 processor
You only have 4GB of virtual address space at any one time so you have to
fit the following within your 4GB virtual space- Virtual addresses of the user space
- Mappings for physical memory
- Kernel virtual spaces (vmalloc etc)
- Various miscellaneous kernel mappings
- I/O MMIO spaceNow you could in theory switch the page tables on kernel entry to get a
very big user space mapping and then instead of relying on the user space
being mapped you could map specific user pages or access them via their
physical address lookup.That however means reloading %cr3 (page table base) which is very
expensive, and some very hard to implement locking on user space access.Some other processor families have the notion of "spaces" or MMU
bypassing. The Sparc64 for example can access MMIO space without an MMU
mapping or virtual address being needed for it. x86 does not - it has
segments but the segments map into the 4GB space so they don't help here
hence the current implementation and 3G/1G split.
Quoted 1 line · On Fri, 28 Aug 2009 14:07:01 -0700 (PDT) joe Shmoe <[email protected]> wrote:
doesn't CR3 gets reloaded for process switch? If so , how does it matter if you setup page table mapping for all the available RAM instead of limiting to just 1GB
Because
- You don't switch process every syscall
- You don't switch process every interrupt
The latter is the real killer.
O> I understand the implications of reloading CR3. But once the page tables are setup to map all the available physical RAM to virtual (linear) address it could be for eg. 1, 2, 3 or 4 GB how does it matter.
Where are you putting the user virtual addresses. User addresses don't map direct to physical addresses so you need both sets of translations at once
Right now you have
0-3GB MMU translations to arbitary basically random
physical addresses (with many pages shared and some absent)
3GB+ Physical mapping only accessible in kernel mode
If user applications ran with a 1:1 mapping of application space to physical addresses you would be fine - but they don't and it would be rather hard to run like that because you want page sharing, lazy unshare, vfork etc to all work.
Thanks for the reply Alan,
What I am looking for is this: Let us keep aside the virtual addresses for a process aside for a second. My question does not relate to that at all.
why does kernel map only 896MB of RAM to linear addresses in kernel page tables at the startup even if RAM size is more than that say 3.5GB of RAM Why does it use dynamic memory mapping technique using zones for mapping the remaining 2.5 GB of RAM( 3.5GB - 1GB = 2.5GB)
Why not it does the following?
kernel sets up the page tables during initialization phase such that it maps all the available physical RAM i.e 3.5GB to linear addresses. what is wrong with this approach.
Quoted 35 lines · Alan Cox
From: Alan Cox <[email protected]>
Subject: Re: kernel page table mapping for >1GB <3 GB for x86 arch without PAE
To: "joe Shmoe" <[email protected]>
Cc: [email protected]
Date: Friday, August 28, 2009, 6:16 PM
O> I understand the implications
of reloading CR3. But once the page tables are setup
to map all the available physical RAM to virtual (linear)
address it could be for eg. 1, 2, 3 or 4 GB how does it
matter.Where are you putting the user virtual addresses. User addresses don't
map direct to physical addresses so you need both sets of translations at
onceRight now you have
0-3GB MMU translations
to arbitary basically random
physical addresses (with many pages shared and some absent)3GB+ Physical mapping
only accessible in kernel modeIf user applications ran with a 1:1 mapping of application space to
physical addresses you would be fine - but they don't and it would be
rather hard to run like that because you want page sharing, lazy unshare,
vfork etc to all work.
Quoted 2 lines · On Fri, 28 Aug 2009 17:20:31 -0700 (PDT) joe Shmoe <[email protected]> wrote:
Thanks for the reply Alan,
What I am looking for is this: Let us keep aside the virtual addresses for a process aside for a second. My question does not relate to that at all.
When you understand why that first statement is wrong you will understand the answer.
Alan
Hey Alan,
May be I confused you. Sorry
Quoted 4 lines
0-3GB MMU translations
to arbitary basically random
physical addresses (with many pages shared and some absent)
Can those "arbitary basically random physical addresses" fall outside
1GB-3GB is my question. Why does kernel stop at 896MB during page table setup. Why not go ahead and map the remaining available RAM (where RAM size is > 1GB < 4GB) But kernel uses different technique for addressing the high memory > 896MB
Not sure this is being done so that it will be easier for kernel to maintain page table assignments/swaping to various processes.
Quoted 2 lines
3GB+ Physical mapping
only accessible in kernel mode
Yes. This part I understand obviously these addresses are issued only when a process is running in kernel mode.
Quoted 35 lines · Alan Cox
From: Alan Cox <[email protected]>
Subject: Re: kernel page table mapping for >1GB <3 GB for x86 arch without PAE
To: "joe Shmoe" <[email protected]>
Cc: [email protected]
Date: Friday, August 28, 2009, 6:16 PM
O> I understand the implications
of reloading CR3. But once the page tables are setup
to map all the available physical RAM to virtual (linear)
address it could be for eg. 1, 2, 3 or 4 GB how does it
matter.Where are you putting the user virtual addresses. User addresses don't
map direct to physical addresses so you need both sets of translations at
onceRight now you have
0-3GB MMU translations
to arbitary basically random
physical addresses (with many pages shared and some absent)3GB+ Physical mapping
only accessible in kernel modeIf user applications ran with a 1:1 mapping of application space to
physical addresses you would be fine - but they don't and it would be
rather hard to run like that because you want page sharing, lazy unshare,
vfork etc to all work.