Skip to contentSkip to conversation

Interleaved allocation

DISCUSSION Martin Ichilevici de Oliveira 2 participants 3 messages Dec 8, 2014 – Dec 12, 2014

lore
Share & tools
lore
#0 lore raw

Hello,

I'm trying to understand what's the exact behavior of the interleaved allocation in libnuma. I have two questions, the first regarding numa_alloc_interleaved() and the second regarding numactl. Suppose I'm running in a 4-node machine.

1. What happens when I try to allocate 100 pages with a call like this:

numa_alloc_interleaved(100 * numa_pagesize())

I can think of two possible allocation schemes:
a. Pages 0-24 are allocated to node 0, pages 25-49 are allocated to node 1, pages 50-74 are allocated on node 2 and pages 75-99 are allocated on node 3.

b. Pages 0,4,8... are allocated on node 0, pages 1,5,9... are allocated on node 1 and so on

2. What's the effect of using numactl --interleaved=0,1,2,3 -- ./myprog? I would assume it "substitutes" any malloc's to numa_alloc_interleaved(), but I'm not sure if it's just that.

Thank you,
Martin

#1 lore raw
Quoted 11 lines · Martin Ichilevici de Oliveira

1. What happens when I try to allocate 100 pages with a call like this:

numa_alloc_interleaved(100 * numa_pagesize())

I can think of two possible allocation schemes:
a. Pages 0-24 are allocated to node 0, pages 25-49 are allocated to node 1, pages 50-74 are allocated on node 2 and pages 75-99 are allocated on node 3.

b. Pages 0,4,8... are allocated on node 0, pages 1,5,9... are allocated on node 1 and so on

It's approximately (b)

Quoted 3 lines

2. What's the effect of using numactl --interleaved=0,1,2,3 -- ./myprog? I would assume it "substitutes" any malloc's to numa_alloc_interleaved(), but I'm not sure if it's just that.

Any page allocated by the process is interleaved in access order (which is different from what numa_alloc_interleaved does)

-Andi

Signature · 1 line
[email protected] -- Speaking for myself only
#2 lore raw
Quoted 14 lines · Andi Kleen
Quoted 11 lines · Martin Ichilevici de Oliveira

1. What happens when I try to allocate 100 pages with a call like this:

numa_alloc_interleaved(100 * numa_pagesize())

I can think of two possible allocation schemes:
a. Pages 0-24 are allocated to node 0, pages 25-49 are allocated to node 1, pages 50-74 are allocated on node 2 and pages 75-99 are allocated on node 3.

b. Pages 0,4,8... are allocated on node 0, pages 1,5,9... are allocated on node 1 and so on

Quoted 8 lines

It's approximately (b)

Quoted 3 lines

2. What's the effect of using numactl --interleaved=0,1,2,3 -- ./myprog? I would assume it "substitutes" any malloc's to numa_alloc_interleaved(), but I'm not sure if it's just that.

Any page allocated by the process is interleaved in access order (which is different from what numa_alloc_interleaved does)

Andi,

Thank you for clarifying these up.

Martin