Skip to contentSkip to conversation

Ideas for SLUB allocator

LSF/MM TOPIC Jesper Dangaard Brouer 5 participants 6 messages Apr 12, 2016 Memory management

lore
Share & tools
lore
#0 lore raw

Hi Rik,

I have another topic, which is very MM-specific.

I have some ideas for improving SLUB allocator further, after my work on implementing the slab bulk APIs. Maybe you can give me a small slot, I only have 7 guidance slides. Or else I hope we/I can talk about these ideas in a hallway track with Christoph and others involved in slab development...

I've already published the preliminary slides here:
http://people.netfilter.org/hawk/presentations/MM-summit2016/slab_mm_summit2016.odp

Signature · 11 lines
Best regards,
  Jesper Dangaard Brouer
  MSc.CS, Principal Kernel Engineer at Red Hat
  Author of http://www.iptv-analyzer.org
  LinkedIn: http://www.linkedin.com/in/brouer

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected].  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
#1 lore raw
Quoted 4 lines · Jesper Dangaard Brouer

Hi Rik,

I have another topic, which is very MM-specific.

Quoted 8 lines

I have some ideas for improving SLUB allocator further, after my work on implementing the slab bulk APIs. Maybe you can give me a small slot, I only have 7 guidance slides. Or else I hope we/I can talk about these ideas in a hallway track with Christoph and others involved in slab development...

I've already published the preliminary slides here:
http://people.netfilter.org/hawk/presentations/MM-summit2016/slab_mm_summit2016.odp

The current bulk API returns the pointers in an array. What the radix tree would like is the ability to bulk allocate from a slab and chain the allocations through an offset. See __radix_tree_preload() in lib/radix-tree.c. I don't know if this is a common thing to do elsewhere in the kernel. Obviously, radix-tree could allocate the array on the stack and set up the chain itself, but I would think it would be just as easy for slab to do it itself and save the stack space.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

#2 lore raw
Quoted 14 lines · On Tue, 12 Apr 2016 09:37:28 -0400 Matthew Wilcox <[email protected]> wrote:
Quoted 4 lines · Jesper Dangaard Brouer

Hi Rik,

I have another topic, which is very MM-specific.

Quoted 8 lines

I have some ideas for improving SLUB allocator further, after my work on implementing the slab bulk APIs. Maybe you can give me a small slot, I only have 7 guidance slides. Or else I hope we/I can talk about these ideas in a hallway track with Christoph and others involved in slab development...

I've already published the preliminary slides here:
http://people.netfilter.org/hawk/presentations/MM-summit2016/slab_mm_summit2016.odp

Quoted 7 lines

The current bulk API returns the pointers in an array. What the radix tree would like is the ability to bulk allocate from a slab and chain the allocations through an offset. See __radix_tree_preload() in lib/radix-tree.c. I don't know if this is a common thing to do elsewhere in the kernel. Obviously, radix-tree could allocate the array on the stack and set up the chain itself, but I would think it would be just as easy for slab to do it itself and save the stack space.

It does look like a good candidate for bulk alloc in __radix_tree_preload(). Especially because you have an annoying preempt_disable() and preempt_enable() interaction loop, and reloading of this_cpu_ptr.

And RADIX_TREE_PRELOAD_SIZE==21 is not excessive alloc bulking. Considering local_irq's are disabled during the bulk alloc.

The allocator is delivering "raw" memory, thus it does not know anything about the callers data structure, and shouldn't. (I have considered delivering bulk objects single linked via offset 0, as this already happens internally in SLUB, but I decided against it)

Looking closer at your specific data structures, they are also a bit complicated to deliver "linked" easily...

struct radix_tree_preload {
	int nr;
	/* nodes->private_data points to next preallocated node */
	struct radix_tree_node *nodes;
};
struct radix_tree_node {
	unsigned int	path;	/* Offset in parent & height from the bottom */
	unsigned int	count;
	union {
		struct {
			/* Used when ascending tree */
			struct radix_tree_node *parent;
			/* For tree user */
			void *private_data;
		};
		/* Used when freeing node */
		struct rcu_head	rcu_head;
	};
	/* For tree user */
	struct list_head private_list;
	void __rcu	*slots[RADIX_TREE_MAP_SIZE];
	unsigned long	tags[RADIX_TREE_MAX_TAGS][RADIX_TREE_TAG_LONGS];
};
Signature · 11 lines
Best regards,
  Jesper Dangaard Brouer
  MSc.CS, Principal Kernel Engineer at Red Hat
  Author of http://www.iptv-analyzer.org
  LinkedIn: http://www.linkedin.com/in/brouer

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected].  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
#3 lore raw
In reply to Jesper Dangaard Brouer
Quoted 5 lines · Jesper Dangaard Brouer

I have some ideas for improving SLUB allocator further, after my work on implementing the slab bulk APIs. Maybe you can give me a small slot, I only have 7 guidance slides. Or else I hope we/I can talk about these ideas in a hallway track with Christoph and others involved in slab development...

I will be there.

Quoted 2 lines

I've already published the preliminary slides here:
http://people.netfilter.org/hawk/presentations/MM-summit2016/slab_mm_summit2016.odp

Re Autotuning: SLUB obj per page:
SLUB can combine pages of different orders in a slab cache so this would be possible.

per CPU freelist per page:
Could we drop the per cpu partial lists if this works?

Clearing memory:
Could exploit the fact that the page is zero on alloc and also zap when no object in the page is in use?

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

#4 lore raw
Quoted 21 lines · Christoph Lameter
Quoted 7 lines · Jesper Dangaard Brouer

I have some ideas for improving SLUB allocator further, after my work
on implementing the slab bulk APIs.  Maybe you can give me a small slot, I only have 7 guidance slides.  Or else I hope we/I can talk about these ideas in a hallway track with Christoph and others involved
in slab development...

I will be there.

Quoted 3 lines

I've already published the preliminary slides here:
 http://people.netfilter.org/hawk/presentations/MM-summit2016/slab_
mm_summit2016.odp

Re Autotuning: SLUB obj per page:
SLUB can combine pages of different orders in a slab cache so
this would
be possible.

Quoted 7 lines

per CPU freelist per page:
Could we drop the per cpu partial lists if this works?

Clearing memory:
Could exploit the fact that the page is zero on alloc and also
zap
when no object in the page is in use?

Between the SLUB things both of you want to discuss, do you think one 30 minute slot will be enough to start with, or should we schedule a whole hour?

We have some free slots left on the second day, where discussions can overflow if necessary.

Signature · 1 line
All rights reversed
#5 lore raw
Quoted 30 lines · On Tue, 12 Apr 2016 14:13:48 -0400 Rik van Riel <[email protected]> wrote:
Quoted 21 lines · Christoph Lameter
Quoted 7 lines · Jesper Dangaard Brouer

I have some ideas for improving SLUB allocator further, after my work
on implementing the slab bulk APIs.  Maybe you can give me a small slot, I only have 7 guidance slides.  Or else I hope we/I can talk about these ideas in a hallway track with Christoph and others involved
in slab development...

I will be there.

Quoted 3 lines

I've already published the preliminary slides here:
 http://people.netfilter.org/hawk/presentations/MM-summit2016/slab_
mm_summit2016.odp

Re Autotuning: SLUB obj per page:
SLUB can combine pages of different orders in a slab cache so
this would
be possible.

Quoted 7 lines

per CPU freelist per page:
Could we drop the per cpu partial lists if this works?

Clearing memory:
Could exploit the fact that the page is zero on alloc and also
zap
when no object in the page is in use?

Quoted 7 lines

Between the SLUB things both of you want to discuss, do you think one 30 minute slot will be enough to start with, or should we schedule a whole hour?

We have some free slots left on the second day, where discussions can overflow if necessary.

30 min slot is fine by me :-)

Signature · 11 lines
Best regards,
  Jesper Dangaard Brouer
  MSc.CS, Principal Kernel Engineer at Red Hat
  Author of http://www.iptv-analyzer.org
  LinkedIn: http://www.linkedin.com/in/brouer

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected].  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>