Skip to contentSkip to conversation

question about memory-hotplug

DISCUSSION tgh 2 participants 9 messages Apr 3, 2007 – Apr 5, 2007

lore
Share & tools
lore

tgh

author
#0 lore raw

hi
I try to understand the memory_hotplug.c ,but I am confused about
the mechanism in it
Does the mem_map is allocated when setup for all the memory page both
the existing and the hotplug , or does the mem_map is allocated dynamically when a chip of memory is hotpluged ?
if mem_map is allocated dynamically,does the mem_map is maintained in
the list or how to deal with it ?
and if mem_map is allocated dynamically,does system need new node and
new zone structure to manage the dynamically-allocated memmap ,or not ?
if system need new node and new zone structure to manage the
dynamically-allocated memmap,does system have to be a NUMA-surpport one or not?

I am confused about it

could you help me
Thanks in advance

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CIDÞVDEV
_______________________________________________
Linux-hotplug-devel mailing list http://linux-hotplug.sourceforge.net
[email protected]
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel

#1 lore raw

Hello.

Quoted 10 lines

hi
I try to understand the memory_hotplug.c ,but I am confused about
the mechanism in it
Does the mem_map is allocated when setup for all the memory page both
the existing and the hotplug , or does the mem_map is allocated dynamically when a chip of memory is hotpluged ?
if mem_map is allocated dynamically,does the mem_map is maintained in
the list or how to deal with it ?
and if mem_map is allocated dynamically,does system need new node and
new zone structure to manage the dynamically-allocated memmap ,or not ?

Basically, memory hotplug code depends on sparsemem. So, I'll recommend that you should see mm/sparse.c too.
In sparsemem code, mem_map is allocated dynamically, and its pointed by mem_section[] array.

(However, x86-64 code has original code. IIRC, its memmap was allcated
statically....)

Quoted 3 lines

if system need new node and new zone structure to manage the
dynamically-allocated memmap,does system have to be a NUMA-surpport one or not?

Pgdat and zone can be allocated dynamically for NUMA system, but memmap is not pointed by them if sparsemem is on.

Thanks for your concern. :-)

---
Yasunori Goto

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CIDÞVDEV
_______________________________________________
Linux-hotplug-devel mailing list http://linux-hotplug.sourceforge.net
[email protected]
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel

author
#2 lore raw
In reply to tgh

Thank you for your reply

Yasunori Goto 写�:

Quoted 15 lines

Hello.

Quoted 11 lines

hi
I try to understand the memory_hotplug.c ,but I am confused about
the mechanism in it
Does the mem_map is allocated when setup for all the memory page both
the existing and the hotplug , or does the mem_map is allocated dynamically when a chip of memory is hotpluged ?
if mem_map is allocated dynamically,does the mem_map is maintained in
the list or how to deal with it ?
and if mem_map is allocated dynamically,does system need new node and
new zone structure to manage the dynamically-allocated memmap ,or not ?

Quoted 5 lines

Basically, memory hotplug code depends on sparsemem. So, I'll recommend that you should see mm/sparse.c too.
In sparsemem code, mem_map is allocated dynamically, and its pointed by mem_section[] array.

In the traditional linux without sparsemem, there are node-zone-memmap mechanism for managing the physical memory,is it right? while in the sparsemem mechanism, what is the architure of node-zone-memmap or node-zone-mem_section[]-mem_map like?

and if the sparsemem is on , buddy still does its work in a mem_map as before or does it work in a new way?
and how does virt-to-phy macro (or something like it) work,when there are a list of mem_map in the sparsemem system?

Quoted 3 lines

(However, x86-64 code has original code. IIRC, its memmap was allcated
statically....)

does x86-64 not surpport the memory-hotplug,or how does it achieve the memory-hotplug ?

Quoted 9 lines
Quoted 5 lines

if system need new node and new zone structure to manage the
dynamically-allocated memmap,does system have to be a NUMA-surpport one or not?

Pgdat and zone can be allocated dynamically for NUMA system, but memmap is not pointed by them if sparsemem is on.

which structure points it ,mem_section[] does it? then what is the architure of the node-zone-mem_section[]-mem_map like?

Quoted 9 lines

Thanks for your concern. :-)

---
Yasunori Goto

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CIDÞVDEV
_______________________________________________
Linux-hotplug-devel mailing list http://linux-hotplug.sourceforge.net
[email protected]
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel

#3 lore raw
In reply to tgh
Quoted 2 lines

In the traditional linux without sparsemem, there are node-zone-memmap mechanism for managing the physical memory,is it right?

Yes.

Quoted 2 lines

while in the sparsemem mechanism, what is the architure of node-zone-memmap or node-zone-mem_section[]-mem_map like?

mem_section[] is "static array" which is defined in sparse.c. So, no pointer is necessary to point mem_section array. Its index can be found by only pfn like ((pfn >> PFN_SECTION_SHIFT)). Each mem_section has pointer of mem_map.

In addition, each page->flags has area for saving mem_section's index. page_to_section() is (page->flags >> SECTIONS_PGSHIFT) & SECTIONS_MASK).

Tracing __pfn_to_page() and __page_to_pfn in include/asm-generic/memory_model.h will be good answer for your understanding.

Quoted 2 lines

and if the sparsemem is on , buddy still does its work in a mem_map as before or does it work in a new way?

2.6.10 or older kernel has bitmaps to manage buddy page. Current one doesn't have it. Its information is saved in page->private. Others are same.
1 section size is bigger than MAX_ORDER. So, current buddy system doensn't concern about section size.
This means memory hotplug unit size is section size. (At least now)

Quoted 8 lines

and how does virt-to-phy macro (or something like it) work,when there are a list of mem_map in the sparsemem system?

Quoted 3 lines

(However, x86-64 code has original code. IIRC, its memmap was allcated
statically....)

does x86-64 not surpport the memory-hotplug,or how does it achieve the memory-hotplug ?

x86-64 supports both of sparse and non sparse. It is seleced by kernel config option.

Bye.

Signature · 13 lines
Yasunori Goto 



-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CIDÞVDEV
_______________________________________________
Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
[email protected]
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel
author
#4 lore raw
In reply to tgh

Thank you for your reply

If sparsememory is on ,are there node and zone structures or not in the systme?

Thanks in advance

Yasunori Goto 写�:

Quoted 44 lines
Quoted 3 lines

In the traditional linux without sparsemem, there are node-zone-memmap mechanism for managing the physical memory,is it right?

Yes.

Quoted 3 lines

while in the sparsemem mechanism, what is the architure of node-zone-memmap or node-zone-mem_section[]-mem_map like?

mem_section[] is "static array" which is defined in sparse.c. So, no pointer is necessary to point mem_section array. Its index can be found by only pfn like ((pfn >> PFN_SECTION_SHIFT)). Each mem_section has pointer of mem_map.

In addition, each page->flags has area for saving mem_section's index. page_to_section() is (page->flags >> SECTIONS_PGSHIFT) & SECTIONS_MASK).

Tracing __pfn_to_page() and __page_to_pfn in include/asm-generic/memory_model.h will be good answer for your understanding.

Quoted 3 lines

and if the sparsemem is on , buddy still does its work in a mem_map as before or does it work in a new way?

2.6.10 or older kernel has bitmaps to manage buddy page. Current one doesn't have it. Its information is saved in page->private. Others are same.
1 section size is bigger than MAX_ORDER. So, current buddy system doensn't concern about section size.
This means memory hotplug unit size is section size. (At least now)

Quoted 8 lines

and how does virt-to-phy macro (or something like it) work,when there are a list of mem_map in the sparsemem system?

Quoted 4 lines

(However, x86-64 code has original code. IIRC, its memmap was allcated
statically....)

Quoted 10 lines
Quoted 3 lines

does x86-64 not surpport the memory-hotplug,or how does it achieve the memory-hotplug ?

x86-64 supports both of sparse and non sparse. It is seleced by kernel config option.

Bye.

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CIDÞVDEV
_______________________________________________
Linux-hotplug-devel mailing list http://linux-hotplug.sourceforge.net
[email protected]
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel

#5 lore raw
In reply to tgh
Quoted 1 line

Thank you for your reply

You are welcome.

Quoted 2 lines

If sparsememory is on ,are there node and zone structures or not in the systme?

Yes there are. Zone is still necessary for page reclaim. Node and zonelist are used for NUMA system.

Thanks.

Signature · 13 lines
Yasunori Goto 



-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CIDÞVDEV
_______________________________________________
Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
[email protected]
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel
author
#6 lore raw
In reply to tgh

Thank you for your reply

Quoted 3 lines

Yes there are. Zone is still necessary for page reclaim. Node and zonelist are used for NUMA system.

Buddy allocate memory with the mem_section[] and without zone-aware ,while it reclaim memory with zone,is it right?
I am confused about the functions of the zone and node in the system ,when the sparsemem is on

could you help me
Thanks in advance

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CIDÞVDEV
_______________________________________________
Linux-hotplug-devel mailing list http://linux-hotplug.sourceforge.net
[email protected]
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel

#7 lore raw
In reply to tgh
Quoted 7 lines
Quoted 3 lines

Yes there are. Zone is still necessary for page reclaim. Node and zonelist are used for NUMA system.

Buddy allocate memory with the mem_section[] and without zone-aware ,while it reclaim memory with zone,is it right?
I am confused about the functions of the zone and node in the system ,when the sparsemem is on

Basically, mem_section[] is for pointer to mem_map.
It doesn't count how many pages are allocated/freed in its section. This just divides mem_map[] as discontiguous.

Buddy allocater must know 2 things.

First is the address of buddy page against target page.
It can be calculated as --relative offset-- from target page. (See: __page_find_buddy()). So, buddy allocater doesn't know mem_map is discontigous or not.
From this view point, Buddy allocater doen't concern about any zone, node, nor mem_section.

Second is free_list for managing free pages. Its list is connected to zone. So, buddy allocator must know about only zone's address and target page's address. It doesn't know mem_map is contiguous or not too.

Current zone is used for manage freed/active/inactive pages. Zone has link lists and counters of their pages.
There is no assumption that zone is contiguous from this view point. Even if there is a memory hole in zone, zone can work.

Just memmap array has assumption that physicall memory is contigous. Index of memmap is decided by Page Frame Number. If there is big memory hole in memory, many space of it will be wasted. This is why mem_section is used for sparse mem.

Node is for NUMA node now. Each node have each pgdat. This is unit for managing memory areas which has different access speed. Kernel can use nearest memory from cpu by finding appropriate pgdat.

Old DISCONTIG_MEM defines one pgdat for one contigous memory area. But many people dislike now. If system has too much memory area which area is small size, too much pgdats and big zonelists are big cause of system load.

Bye.

Signature · 13 lines
Yasunori Goto 



-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CIDÞVDEV
_______________________________________________
Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
[email protected]
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel
author
#8 lore raw
In reply to tgh

Thank you for your reply

Thank you

Yasunori Goto 写�:

Quoted 43 lines
Quoted 9 lines
Quoted 4 lines

Yes there are. Zone is still necessary for page reclaim. Node and zonelist are used for NUMA system.

Buddy allocate memory with the mem_section[] and without zone-aware ,while it reclaim memory with zone,is it right?
I am confused about the functions of the zone and node in the system ,when the sparsemem is on

Basically, mem_section[] is for pointer to mem_map.
It doesn't count how many pages are allocated/freed in its section. This just divides mem_map[] as discontiguous.

Buddy allocater must know 2 things.

First is the address of buddy page against target page.
It can be calculated as --relative offset-- from target page. (See: __page_find_buddy()). So, buddy allocater doesn't know mem_map is discontigous or not.
From this view point, Buddy allocater doen't concern about any zone, node, nor mem_section.

Second is free_list for managing free pages. Its list is connected to zone. So, buddy allocator must know about only zone's address and target page's address. It doesn't know mem_map is contiguous or not too.

Current zone is used for manage freed/active/inactive pages. Zone has link lists and counters of their pages.
There is no assumption that zone is contiguous from this view point. Even if there is a memory hole in zone, zone can work.

Just memmap array has assumption that physicall memory is contigous. Index of memmap is decided by Page Frame Number. If there is big memory hole in memory, many space of it will be wasted. This is why mem_section is used for sparse mem.

Node is for NUMA node now. Each node have each pgdat. This is unit for managing memory areas which has different access speed. Kernel can use nearest memory from cpu by finding appropriate pgdat.

Quoted 9 lines

Old DISCONTIG_MEM defines one pgdat for one contigous memory area. But many people dislike now. If system has too much memory area which area is small size, too much pgdats and big zonelists are big cause of system load.

Bye.

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CIDÞVDEV
_______________________________________________
Linux-hotplug-devel mailing list http://linux-hotplug.sourceforge.net
[email protected]
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel