questions on discontgmem.
Hi,
I'd like to use discontigmem to access several RAM memories on an _no_ NUMA embedded system. In that case, does a node mean a single RAM whose start address is very different from the others ?
When CONFIG_NUMA is _not_ set, how is a node choosen during allocation ? any pointers are welcome !
With such mechanism, is it possible to create a block device (let say /dev/my_ram) whose (meta) data belongs to a _single_ node ?
Thanks.
--
Franck
A message that is not in the archive (sent off-list or not yet received): [email protected]
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 19 lines · Franck
2005/9/26, Marc Singer <[email protected]>:
Quoted 13 lines · Franck
Quoted 5 lines
Hi,
I'd like to use discontigmem to access several RAM memories on an _no_ NUMA embedded system. In that case, does a node mean a single RAM whose start address is very different from the others ?
I don't know what you mean by this. The difference between discontiguous memory and non-discontiguous [sic] memory is that, well, the latter is contiguous. If you have 64MiB of RAM, it is addressed starting at a base address and all 64 MiB's of addresses following it are valid.
Quoted 4 lines
Discontiguous memory means that that isn't true, there is more than one start address with gaps of addresses between them which are either invalid, or which are aliases of other addresses.
Quoted 5 lines
yeah, I know what is discontiguous memory. What I'm trying to ask is:
in this case (discontiguous memory), the kernel seems to describe each memory with a single node (cfalloc_page_nodefor instance). Unlike for NUMA system where several memories can belong to the same node. Is that correct ?
I don't know what you mean by a 'memory'. Are you talking about an IC? If so, Linux doesn't care about the physical expression of memory hardware. It only cares about the addresses of the RAM and, possibly, performance characteristics of that access. I write possibly because I've not worked with that attribute of discontigmem.
There is a potential misunderstanding in how discontigmem really works. There may be a memory system which is truly discontiguous, but that uses a single node. It is wasteful because the node tables are accessed as arrays, so non-existent pages require data structure space just as existent ones do.
With discontigmem, we can use multiple nodes in order to cluster pages and eliminate unused array entries. Assuming that this is *not* done for the sake of differing memory access times, the advantage is in saving node table space. There are macros in the architecture/machine specific include files that create the mapping between physical addresses and nodes/pages. At least, that's how ARM does it.
Quoted 1 line
If so, how does the kernel select a node when allocating a page for a UMA ?
Perhaps someone else can fill in that part. I've been assuming that pages with equivalent access times are put in zones, on free-page lists and allocated as needed. Nothing special is done with respect to the different discontigmem nodes.
Cheers.
Hi Marc,
2005/9/27, Marc Singer <[email protected]>:
Quoted 20 lines · Franck
Quoted 19 lines
2005/9/26, Marc Singer <[email protected]>:
Quoted 17 lines · Franck
Quoted 5 lines
Hi,
I'd like to use discontigmem to access several RAM memories on an _no_ NUMA embedded system. In that case, does a node mean a single RAM whose start address is very different from the others ?
I don't know what you mean by this. The difference between discontiguous memory and non-discontiguous [sic] memory is that, well, the latter is contiguous. If you have 64MiB of RAM, it is addressed starting at a base address and all 64 MiB's of addresses following it are valid.
Discontiguous memory means that that isn't true, there is more than one start address with gaps of addresses between them which are either invalid, or which are aliases of other addresses.
Quoted 8 lines
Quoted 5 lines
yeah, I know what is discontiguous memory. What I'm trying to ask is:
in this case (discontiguous memory), the kernel seems to describe each memory with a single node (cfalloc_page_nodefor instance). Unlike for NUMA system where several memories can belong to the same node. Is that correct ?I don't know what you mean by a 'memory'. Are you talking about an IC?
yes I'm taking about IC. But each IC has a different start base address with huge hole between each of them. For instance I have 2 RAM mapped as shown below:
RAM1 -> physical address at 0x2000 0000, size = 32 Mo RAM2 -> physical address at 0x3000 0000, size = 2 Mo
Quoted 3 lines
performance characteristics of that access. I write possibly because I've not worked with that attribute of discontigmem.
did you work on ARM discontig memory ?
Quoted 6 lines
There is a potential misunderstanding in how discontigmem really works. There may be a memory system which is truly discontiguous, but that uses a single node. It is wasteful because the node tables are accessed as arrays, so non-existent pages require data structure space just as existent ones do.
Quoted 7 lines
With discontigmem, we can use multiple nodes in order to cluster pages and eliminate unused array entries. Assuming that this is *not* done for the sake of differing memory access times, the advantage is in saving node table space. There are macros in the architecture/machine specific include files that create the mapping between physical addresses and nodes/pages. At least, that's how ARM does it.
yes that what I understand. CONFIG_DISCONTIGMEM is used on UMA system to avoid mem_map to blow up when using RAMs whose start base addresses are separated with huge hole.
Quoted 7 lines
Quoted 1 line
If so, how does the kernel select a node when allocating a page for a UMA ?
Perhaps someone else can fill in that part. I've been assuming that pages with equivalent access times are put in zones, on free-page lists and allocated as needed. Nothing special is done with respect to the different discontigmem nodes.
hmm, not sure it works the way you described. I thought there were nodes gathering several RAM whose start address are _contiguous_. Each node has several zones (dma, normal and highmem). But I don't see how allocation can be done using *all* nodes...
I found in "include/linux/gfp.h", for a UMA system:
#define alloc_pages(gfp_mask, order) \
alloc_pages_node(numa_node_id(), gfp_mask, order)
and numa_node_id expands to 0 for a UP system.
Any Idea ?
Thanks
--
Franck