move the mempool list_head out of the managed elements

The mempool code embeds a list_head inside the managed objects.

This is a problem when the mempool client wishes that memory to be in a known state. For example, the radix-tree pagecache code backs the mempool by a slab cache allocator, with a constructor which clears the memory.

But when the page allocator starts failing, and radix tree node allocations are satisfied from the mempool, they come back with a defunct list_head at the first eight bytes, and the ratcache dies in subtle ways.

A nasty part about this is that the problem only exhibits under very high load.

I considered extending mempool so the client could pass in a special constructor which just puts those eight bytes into a known state, but this is not really adequate because that memory may simply not be reconstructable in that context.

In highmem.c, the object being managed is actually a struct page. So the mempool code is overwriting the page frame structure with a list_head. This just happens to work, because that part of the page struct is the page's own list_head, which is not used at that time. But ugh.

Also, RAID1 is assuming that the mempooled memory is all zeroed, but that will not the case when page allocations start to fail.

The patch converts the mempool's intrusive list into a non-intrusive one. So mempool never alters the objects which it is managing.

@@ -7,13 +7,27 @@
 #include <linux/list.h>
 #include <linux/wait.h>
 
-struct mempool_s;
-typedef struct mempool_s mempool_t;
-
 typedef void * (mempool_alloc_t)(int gfp_mask, void *pool_data);
 typedef void (mempool_free_t)(void *element, void *pool_data);
 
-struct mempool_s {
+/*
+ * A structure for linking multiple client objects into
+ * a mempool_t
+ */
+typedef struct mempool_node_s {
+	struct list_head list;
+	void *element;
+} mempool_node_t;
+
+/*
+ * The elements list has full mempool_node_t's at ->next, and empty ones
+ * at ->prev.  Emptiness is signified by mempool_node_t.element == NULL.
+ *
+ * curr_nr refers to how many full mempool_node_t's are at ->elements.
+ * We don't track the total number of mempool_node_t's at ->elements;
+ * it is always equal to min_nr.
+ */
+typedef struct mempool_s {
 	spinlock_t lock;
 	int min_nr, curr_nr;
 	struct list_head elements;

@@ -22,7 +36,7 @@ struct mempool_s {
 	mempool_alloc_t *alloc;
 	mempool_free_t *free;
 	wait_queue_head_t wait;
-};
+} mempool_t;
 extern mempool_t * mempool_create(int min_nr, mempool_alloc_t *alloc_fn,
 				 mempool_free_t *free_fn, void *pool_data);
 extern void mempool_resize(mempool_t *pool, int new_min_nr, int gfp_mask);

-

good catch - your patch looks good to me.

Ingo