Skip to contentSkip to conversation

compressed in-memory swapping take2

PATCH In review Nitin Gupta 5 participants 18 messages Mar 20, 2009 – Mar 21, 2009

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

cover compressed in-memory swapping take2

Hi,

It allows creating a RAM based block device which acts as swap disk. Pages swapped to this device are compressed and stored in memory itself. This is a big win over swapping to slow hard-disk which are typically used as swap disk. For flash, these suffer from wear-leveling issues when used as swap disk - so again its helpful. For swapless systems, it allows more apps to run.

* Changelog: take2 vs initial revision:
xvmalloc changes:
- Use Linux kernel coding sytle for xvmalloc
- Collapse all individual flag test/set/get to generic {test_set_get}_flag
- Added BLOCK_NEXT() macro to reach next contiguous block
- Other minor cleanups - no functional changes
compcache block device code:
- compcache core changes due to change in xvmalloc interface names

Links to performance numbers, use cases can be found in original thread:
http://lkml.org/lkml/2009/3/17/116

Thanks to Christoph and Pekka for feedback on initial revision.

Thanks,
Nitin

#1 lore raw

PATCH 1/3 compressed RAM block device

 drivers/block/Kconfig     |   25 ++
 drivers/block/Makefile    |    1 +
 drivers/block/compcache.c |  993 +++++++++++++++++++++++++++++++++++++++++++++
 drivers/block/compcache.h |  160 ++++++++
 4 files changed, 1179 insertions(+), 0 deletions(-)

Creates RAM based block device (ramzswap0) which can be used as swap device. Pages swapped to this are compressed and stored in memory itself.

The module is called compcache.ko. It depends on:
- xvmalloc.ko: memory allocator
- lzo_compress.ko
- lzo_decompress.ko

See Documentation/blockdev/compcache.txt for usage details.

Project home: http://code.google.com/p/compcache/

---

drivers/block/Kconfig +25 -0
index 0344a8a..39da94f 100644
@@ -348,6 +348,31 @@ config BLK_DEV_RAM_SIZE
 	  The default value is 4096 kilobytes. Only change this if you know
 	  what you are doing.
 
+config BLK_DEV_COMPCACHE
+	tristate "Compressed RAM swap device"
+	depends on XVMALLOC
+	depends on LZO_COMPRESS
+	depends on LZO_DECOMPRESS
+	help
+	  Saying Y here will allow you to use in-memory compressed swapping.
+	  It creates a pseudo block device (named ramzswap) which acts as
+	  swap device. Pages swapped to this device are compressed and stored
+	  in memory itself.
+
+	  Project home: http://code.google.com/p/compcache/
+	  For details, read <file:Documentation/blockdev/compcache.txt>
+
+	  To compile this driver as a module, choose M here: the
+	  module will be called compcache.
+
+config BLK_DEV_COMPCACHE_STATS
+	bool "Collect statistics"
+	depends on BLK_DEV_COMPCACHE
+	default y
+	help
+	  If enabled, compcache statistics are available via /proc/compcache.
+	  If unsure, say Y.
+
 config BLK_DEV_XIP
 	bool "Support XIP filesystems on RAM block device"
 	depends on BLK_DEV_RAM
drivers/block/Makefile +1 -0
index 87e120e..06ec9dd 100644
@@ -13,6 +13,7 @@ obj-$(CONFIG_PS3_VRAM)		+= ps3vram.o
 obj-$(CONFIG_ATARI_FLOPPY)	+= ataflop.o
 obj-$(CONFIG_AMIGA_Z2RAM)	+= z2ram.o
 obj-$(CONFIG_BLK_DEV_RAM)	+= brd.o
+obj-$(CONFIG_BLK_DEV_COMPCACHE)	+= compcache.o
 obj-$(CONFIG_BLK_DEV_LOOP)	+= loop.o
 obj-$(CONFIG_BLK_DEV_XD)	+= xd.o
 obj-$(CONFIG_BLK_CPQ_DA)	+= cpqarray.o
drivers/block/compcache.c new file +993 -0

993 lines not on this page · raw message

drivers/block/compcache.h new file +160 -0

160 lines not on this page · raw message

#2 lore raw

PATCH 2/3 xvmalloc memory allocator

In reply to Nitin Gupta
 init/Kconfig      |    6 +
 mm/Makefile       |    1 +
 mm/xvmalloc.c     |  572 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 mm/xvmalloc_int.h |   95 +++++++++
 4 files changed, 674 insertions(+), 0 deletions(-)

xvmalloc is a memory allocator designed specifically for compcache project.

* Features:
- Low metadata overhead (just 4 bytes per object)
- O(1) Alloc/Free - except when we have to call system page allocator to get additional memory.
- Very low fragmentation: In all tests, xvMalloc memory usage is within 12%

    of "Ideal".

One of the main highlights is that it maps pages only when required. So, it does not hog vmalloc area which is very small on 32-bit systems.

Slub allocator could not be used due to fragmentation issues:
http://code.google.com/p/compcache/wiki/AllocatorsComparison
Data here shows kmalloc using ~43% more memory than TLSF and xvMalloc is showed ~2% more space efficiency than TLSF (due to smaller metadata).

* Implementation:
It uses two-level bitmap search to find free list containing block of correct size. This idea is taken from TLSF (Two-Level Segregate Fit) allocator and is well explained in its paper (see [Links] below). Highlights:
- Pool based allocator: each pool can grow/shrink.
- Immediate coalescing of free blocks.
- Maps/Unmaps memory pages only when required.

* Limitations:
- Poor scalability: No per-cpu data structures (work in progress).

[Links]
1. Details and Performance data:
http://code.google.com/p/compcache/wiki/xvMalloc
http://code.google.com/p/compcache/wiki/xvMallocPerformance

2. TLSF memory allocator:
home: http://rtportal.upv.es/rtmalloc/
paper: http://rtportal.upv.es/rtmalloc/files/MRBC_2008.pdf

---

init/Kconfig +6 -0
index 6a5c5fe..fa41598 100644
@@ -930,6 +930,12 @@ config SLOB
 
 endchoice
 
+config XVMALLOC
+	tristate "xvMalloc memory allocator"
+	help
+	   This is a simple, low fragmentation, O(1) allocator.
+	   Details: http://code.google.com/p/compcache/wiki/xvMalloc
+
 config PROFILING
 	bool "Profiling support (EXPERIMENTAL)"
 	help
mm/Makefile +1 -0
index 72255be..b6b705f 100644
@@ -26,6 +26,7 @@ obj-$(CONFIG_SLOB) += slob.o
 obj-$(CONFIG_MMU_NOTIFIER) += mmu_notifier.o
 obj-$(CONFIG_SLAB) += slab.o
 obj-$(CONFIG_SLUB) += slub.o
+obj-$(CONFIG_XVMALLOC) += xvmalloc.o
 obj-$(CONFIG_FAILSLAB) += failslab.o
 obj-$(CONFIG_MEMORY_HOTPLUG) += memory_hotplug.o
 obj-$(CONFIG_FS_XIP) += filemap_xip.o
mm/xvmalloc.c new file +572 -0

572 lines not on this page · raw message

mm/xvmalloc_int.h new file +95 -0

95 lines not on this page · raw message

#3 lore raw

Re: [PATCH 2/3] xvmalloc memory allocator

Quoted 1 line · Nitin Gupta

xvmalloc is a memory allocator designed specifically for compcache project.

Its an allocator that is highmem capable? Looks like an entirely new animal to me.

Quoted 2 lines

* Features:
- Low metadata overhead (just 4 bytes per object)

SLUB has 0 byte overhead. SLOB has 2 bytes.

Quoted 2 lines

- O(1) Alloc/Free - except when we have to call system page allocator to get additional memory.

SLOB is not O(1) okay but the others are.

Quoted 2 lines

- Very low fragmentation: In all tests, xvMalloc memory usage is within 12%

    of "Ideal".

Maybe try a fair test instead of relying on kmalloc rounding up to the next power of 2 size?

Quoted 2 lines

One of the main highlights is that it maps pages only when required. So, it does not hog vmalloc area which is very small on 32-bit systems.

Got some difficulty understanding what is going on here. So this allocator is highmem capable? Doesnt that mean that you must make function calls to ensure that an object is mapped before accessing it.

mm/xvmalloc.c line 28

Quoted 6 lines
mm/xvmalloc.c new file +6 -0
+#include "xvmalloc_int.h"
+
+static void stat_inc(u64 *value)
+{
+	*value = *value + 1;
+}

(*value) += 1?

atomic_inc?

local_inc?

mm/xvmalloc.c line 38

Quoted 4 lines
mm/xvmalloc.c new file +4 -0
+static void bitmap_set(u32 *map, u32 idx)
+{
+	*map |= (u32)(1 << idx);
+}

We have bitops for that purpose. Please use those.

mm/xvmalloc.c line 92

Quoted 7 lines
mm/xvmalloc.c new file +7 -0
+/*
+ * Get index of free list having blocks of size greater than
+ * or equal to requested size.
+ */
+static u32 get_index(u32 size)
+{
+	size = (size + FL_DELTA_MASK) & ~FL_DELTA_MASK;

See the ALIGN macro.

mm/xvmalloc.c line 120

Quoted 8 lines
mm/xvmalloc.c new file +8 -0
+/*
+ * Allocate a memory page. Called when a pool needs to grow.
+ */
+static u32 xv_alloc_page(void)
+{
+	struct page *page;
+
+	page = alloc_page(GFP_NOIO | __GFP_HIGHMEM);

Yes a highmem based allocator!!!!

mm/xvmalloc.c line 123

Quoted 3 lines
mm/xvmalloc.c new file +3 -0
+
+	if (unlikely(!page))
+		return INVALID_PGNUM;

Return NULL?

Quoted 1 line

+#define INVALID_PGNUM ((u32)(-1))

NULL

Quoted 1 line

+#define ROUNDUP(x, y) (((x) + (y) - 1) / (y) * (y))

There is a global macro available for that purpose

mm/xvmalloc_int.h line 22

Quoted 2 lines
mm/xvmalloc_int.h new file +2 -0
+/* Each individual bitmap is 32-bit */
+#define BITMAP_BITS	32

Use kernel constants please BITS_PER_LONG here.

Quoted 1 line

+#define ROUNDUP_ALIGN(x) (((x) + XV_ALIGN_MASK) & ~XV_ALIGN_MASK)

== ALIGN?

Well I think this allocator is pretty useful for systems that depend to a large degree on highmem. This is basically x86 32 bit configuration swith more than 1G memmory.

#4 lore raw

Re: [PATCH 2/3] xvmalloc memory allocator

Quoted 11 lines · Christoph Lameter
Quoted 1 line · Nitin Gupta

xvmalloc is a memory allocator designed specifically for compcache project.

Its an allocator that is highmem capable? Looks like an entirely new animal to me.

Quoted 2 lines

* Features:
- Low metadata overhead (just 4 bytes per object)

SLUB has 0 byte overhead. SLOB has 2 bytes.

SLUB: 0 metadata but lot of wastage due to rounding-off.
SLOB: 2 bytes header but I think we should really return object aligned to at least 4 bytes on most archs (including x86 and x64). Apart from smaller header, it has too many other problems as I detailed in previous mail.

Quoted 5 lines
Quoted 3 lines

- O(1) Alloc/Free - except when we have to call system page allocator to get additional memory.

SLOB is not O(1) okay but the others are.

Only SLOB is currently there for good packing, so it is the only one worth comparing against. We loose this mem-compression game if we waste too much :)

Quoted 7 lines
Quoted 3 lines

- Very low fragmentation: In all tests, xvMalloc memory usage is within 12%

    of "Ideal".

Maybe try a fair test instead of relying on kmalloc rounding up to the next power of 2 size?

Okay, for testing, I will make some wrappers around SLOB that directly use slob_alloc() to avoid any of this rounding-off. I hope to show some data on this soon. But considering other SLOB issues, this should not, hopefully, be a blocker for compcache.

Quoted 7 lines
Quoted 2 lines

One of the main highlights is that it maps pages only when required. So, it does not hog vmalloc area which is very small on 32-bit systems.

Got some difficulty understanding what is going on here. So this allocator is highmem capable? Doesnt that mean that you must make function calls to ensure that an object is mapped before accessing it.

Yes. xvmalloc caller gets <pagenum, offset> pair. Caller has to separately map it to get dereferenceable pointer.

Quoted 9 lines

mm/xvmalloc.c line 28

Quoted 6 lines
mm/xvmalloc.c new file +6 -0
+#include "xvmalloc_int.h"
+
+static void stat_inc(u64 *value)
+{
+	*value = *value + 1;
+}

(*value) += 1?

Looks better.

Quoted 2 lines

atomic_inc?

There is really no need to make these stat variables atomic.

Quoted 2 lines

local_inc?

This one looks useful when we work on making compcache code scalable.

Quoted 7 lines

mm/xvmalloc.c line 38

Quoted 4 lines
mm/xvmalloc.c new file +4 -0
+static void bitmap_set(u32 *map, u32 idx)
+{
+	*map |= (u32)(1 << idx);
+}

We have bitops for that purpose. Please use those.

Ok.

Quoted 10 lines

mm/xvmalloc.c line 92

Quoted 7 lines
mm/xvmalloc.c new file +7 -0
+/*
+ * Get index of free list having blocks of size greater than
+ * or equal to requested size.
+ */
+static u32 get_index(u32 size)
+{
+	size = (size + FL_DELTA_MASK) & ~FL_DELTA_MASK;

See the ALIGN macro.

ALIGN is doing same thing - will use it instead.

Quoted 17 lines

mm/xvmalloc.c line 120

Quoted 8 lines
mm/xvmalloc.c new file +8 -0
+/*
+ * Allocate a memory page. Called when a pool needs to grow.
+ */
+static u32 xv_alloc_page(void)
+{
+	struct page *page;
+
+	page = alloc_page(GFP_NOIO | __GFP_HIGHMEM);

Yes a highmem based allocator!!!!

mm/xvmalloc.c line 123

Quoted 3 lines
mm/xvmalloc.c new file +3 -0
+
+	if (unlikely(!page))
+		return INVALID_PGNUM;

Return NULL?

Quoted 4 lines
Quoted 1 line

+#define INVALID_PGNUM ((u32)(-1))

NULL

okay, INVALID_PGNUM -> NULL

Quoted 4 lines
Quoted 1 line

+#define ROUNDUP(x, y) (((x) + (y) - 1) / (y) * (y))

There is a global macro available for that purpose

Ok, will use that.

Quoted 5 lines

mm/xvmalloc_int.h line 22

Quoted 2 lines
mm/xvmalloc_int.h new file +2 -0
+/* Each individual bitmap is 32-bit */
+#define BITMAP_BITS	32

Use kernel constants please BITS_PER_LONG here.

Ok.

Quoted 3 lines
Quoted 1 line

+#define ROUNDUP_ALIGN(x) (((x) + XV_ALIGN_MASK) & ~XV_ALIGN_MASK)

== ALIGN?

Yup.

Quoted 5 lines

Well I think this allocator is pretty useful for systems that depend to a large degree on highmem. This is basically x86 32 bit configuration swith more than 1G memmory.

Not just highmem.
1) Its O(1) and as a side effect causes less cache pollution than SLOB which does this absolutely crazy freelist scanning.
2) Better packing (will post comparison data soon).

I think, with a bit playing around with interfaces, it can be turned into general purpose allocator (this will most probably lack highmem support).

Thanks for your feedback.
Nitin

#5 lore raw

Re: [PATCH 2/3] xvmalloc memory allocator

Quoted 8 lines · Nitin Gupta
Quoted 3 lines

Maybe try a fair test instead of relying on kmalloc rounding up to the next power of 2 size?

Okay, for testing, I will make some wrappers around SLOB that directly use slob_alloc() to avoid any of this rounding-off. I hope to show some data on this soon. But considering other SLOB issues, this should not, hopefully, be a blocker for compcache.

SLOB is not rounding off. SLAB/SLUB are. You need to create custom kmem_caches for this.

Quoted 2 lines

I think, with a bit playing around with interfaces, it can be turned into general purpose allocator (this will most probably lack highmem support).

Then it would need to implement the SLAB api (see include/linux/slab.h). Thus we are getting slab allocator #5.

#6 lore raw

Re: [PATCH 2/3] xvmalloc memory allocator

Quoted 5 lines · Christoph Lameter
Quoted 2 lines

I think, with a bit playing around with interfaces, it can be turned into general purpose allocator (this will most probably lack highmem support).

Then it would need to implement the SLAB api (see include/linux/slab.h). Thus we are getting slab allocator #5.

I do not see the point in that. As I suggested earlier, you should probably just move this into drivers/block/ and make it a private compcache allocator.

#7 lore raw

Re: [PATCH 2/3] xvmalloc memory allocator

Quoted 10 lines · Pekka Enberg
Quoted 4 lines · Christoph Lameter
Quoted 2 lines

I think, with a bit playing around with interfaces, it can be turned into general purpose allocator (this will most probably lack highmem support).

Then it would need to implement the SLAB api (see include/linux/slab.h). Thus we are getting slab allocator #5.

I do not see the point in that. As I suggested earlier, you should probably just move this into drivers/block/ and make it a private compcache allocator.

Your wish. But, really, we should not dismiss an O(1) allocator with great space-efficiency so easily. I think it will be great at least for embedded devices (its counterpart, SLOB is simply funny).

Just to add to this, Xen recently included a variant of TLSF allocator which was used in earlier versions of compcache. TLSF is allocator on which xvmalloc is based.
http://xenbits.xensource.com/xen-unstable.hg?file/0477f9061c8a/xen/common/xmalloc_tlsf.c
(Though I do not know which parts of xen depend on this allocator).

and xvmalloc vs tlsf arguments are here:
http://code.google.com/p/compcache/wiki/xvMalloc

tlsf:
http://rtportal.upv.es/rtmalloc/

Anyways, I will move it to drivers/block.

Thanks,
Nitin

maintainer
#8 lore raw

Re: [PATCH 2/3] xvmalloc memory allocator

Quoted 22 lines · On Sat, 21 Mar 2009 01:13:42 +0530 Nitin Gupta <[email protected]> wrote:
Quoted 10 lines · Pekka Enberg
Quoted 4 lines · Christoph Lameter
Quoted 2 lines

I think, with a bit playing around with interfaces, it can be turned into general purpose allocator (this will most probably lack highmem support).

Then it would need to implement the SLAB api (see include/linux/slab.h). Thus we are getting slab allocator #5.

I do not see the point in that. As I suggested earlier, you should probably just move this into drivers/block/ and make it a private compcache allocator.

Your wish. But, really, we should not dismiss an O(1) allocator with great space-efficiency so easily. I think it will be great at least for embedded devices (its counterpart, SLOB is simply funny).

Just to add to this, Xen recently included a variant of TLSF allocator which was used in earlier versions of compcache. TLSF is allocator on which xvmalloc is based.
http://xenbits.xensource.com/xen-unstable.hg?file/0477f9061c8a/xen/common/xmalloc_tlsf.c
(Though I do not know which parts of xen depend on this allocator).

Quoted 5 lines

and xvmalloc vs tlsf arguments are here:
http://code.google.com/p/compcache/wiki/xvMalloc

tlsf:
http://rtportal.upv.es/rtmalloc/

Well, xvmalloc may or may not be a good thing and we can discuss that separately.

But what is regrettable is that xvmalloc appears to be tied to compressed-swap in some manner. Is it not possible to split these two initiatives apart so that neither is dependent upon the other? Or is compressed-swap hopelessly crippled without xvmalloc?

(compcache is a terrible name, btw - it isn't a "compressed cache" at all!)

Quoted 1 line

Anyways, I will move it to drivers/block.

This sounds like it might be a backward step.

#9 lore raw

Re: [PATCH 2/3] xvmalloc memory allocator

Quoted 6 lines · Andrew Morton

On Sat, 21 Mar 2009 01:13:42 +0530 Nitin Gupta <[email protected]> wrote:

But what is regrettable is that xvmalloc appears to be tied to compressed-swap in some manner. Is it not possible to split these two initiatives apart so that neither is dependent upon the other? Or is compressed-swap hopelessly crippled without xvmalloc?

xvmalloc itself is completely independent of compressed-swap. Infact, its loaded as separate kernel module (xvmalloc.ko)
However, this compression project is almost useless without this specialized allocator.

Quoted 3 lines

(compcache is a terrible name, btw - it isn't a "compressed cache" at all!)

I have now heard this many times and my conscious is beginning to hurt now :)
I will change it to match name of its block device: ramzswap sounds better?

Quoted 3 lines
Quoted 1 line

Anyways, I will move it to drivers/block.

This sounds like it might be a backward step.

I'm bit confused here. Last thing I want to do is block mainline merge because of such issues. Its real pain to maintain these things separately.

Thanks,
Nitin

maintainer
#10 lore raw

Re: [PATCH 2/3] xvmalloc memory allocator

Quoted 10 lines · On Sat, 21 Mar 2009 17:42:52 +0530 Nitin Gupta <[email protected]> wrote:
Quoted 6 lines · Andrew Morton

On Sat, 21 Mar 2009 01:13:42 +0530 Nitin Gupta <[email protected]> wrote:

But what is regrettable is that xvmalloc appears to be tied to compressed-swap in some manner. Is it not possible to split these two initiatives apart so that neither is dependent upon the other? Or is compressed-swap hopelessly crippled without xvmalloc?

xvmalloc itself is completely independent of compressed-swap. Infact, its loaded as separate kernel module (xvmalloc.ko)

That sounds good.

Quoted 2 lines

However, this compression project is almost useless without this specialized allocator.

Why? Important information!!

See, being told all this helps us understand why xvmalloc exists. Plus once we have a good description of _why_ xvmalloc is needed, perhaps we can come up with alternatives which are more palatable than merging a whole new allocator. Such as enhancing an existing one.

Quoted 6 lines
Quoted 3 lines

(compcache is a terrible name, btw - it isn't a "compressed cache" at all!)

I have now heard this many times and my conscious is beginning to hurt now :)
I will change it to match name of its block device: ramzswap sounds better?

Is there anything swap-specific about it? It's a block device, yes? I should be able to run mkfs.ext2 on it and mount the thing?

Quoted 7 lines
Quoted 3 lines
Quoted 1 line

Anyways, I will move it to drivers/block.

This sounds like it might be a backward step.

I'm bit confused here. Last thing I want to do is block mainline merge because of such issues. Its real pain to maintain these things separately.

This is why I tell myself to never use the word "it" in an email message.

I assumed that you were referring to moving xvmalloc() down into drivers/block. That would be bad, because then xvmalloc() will _never_ be usable by anything other than ramzblock <new name!>?

#11 lore raw

Re: [PATCH 2/3] xvmalloc memory allocator

Quoted 16 lines · Andrew Morton
Quoted 8 lines · On Sat, 21 Mar 2009 17:42:52 +0530 Nitin Gupta <[email protected]> wrote:
Quoted 5 lines · Andrew Morton

On Sat, 21 Mar 2009 01:13:42 +0530 Nitin Gupta <[email protected]> wrote:
But what is regrettable is that xvmalloc appears to be tied to compressed-swap in some manner. Is it not possible to split these two initiatives apart so that neither is dependent upon the other? Or is compressed-swap hopelessly crippled without xvmalloc?

xvmalloc itself is completely independent of compressed-swap. Infact, its loaded as separate kernel module (xvmalloc.ko)

That sounds good.

Quoted 2 lines

However, this compression project is almost useless without this specialized allocator.

Quoted 7 lines

Why? Important information!!

See, being told all this helps us understand why xvmalloc exists. Plus once we have a good description of _why_ xvmalloc is needed, perhaps we can come up with alternatives which are more palatable than merging a whole new allocator. Such as enhancing an existing one.

xvmalloc is needed by compressed swap since:
- Its O(1)
- It is very memory efficient
- It can use "high memory" for allocation

* space efficiency:

- comparison with SLUB:
http://code.google.com/p/compcache/wiki/AllocatorsComparison
shows that tlsf (allocator on which xvmalloc is based) uses ~40% less memory than kmalloc() backed by SLUB. Christoph suggested creating multiple slabs of different sizes for this test -- which will be a more fair comparison as kmalloc just uses some predefined slabs. I hope to present this data soon. Also, SLUB is limited to using "low memory" - this is blocker issue for compress swap project (on 32-bit system with >1G RAM). xvmalloc can use high memory.

- comparison with SLOB:
In some previous mail in this thread, I explained all the issues that exist with SLOB that make it unacceptable for use in this project.

Quoted 8 lines
Quoted 4 lines
Quoted 2 lines

(compcache is a terrible name, btw - it isn't a "compressed cache" at all!)

I have now heard this many times and my conscious is beginning to hurt now :)
I will change it to match name of its block device: ramzswap sounds better?

Is there anything swap-specific about it? It's a block device, yes? I should be able to run mkfs.ext2 on it and mount the thing?

No. It can handle page-aligned I/O only. Maybe its not too difficult to extend it to handle arbitrary I/O. But as a swap device, handling just page-aligned I/O is good enough.

Quoted 12 lines
Quoted 5 lines
Quoted 2 lines
Quoted 1 line

Anyways, I will move it to drivers/block.

This sounds like it might be a backward step.

I'm bit confused here. Last thing I want to do is block mainline merge because of such issues. Its real pain to maintain these things separately.

This is why I tell myself to never use the word "it" in an email message.

I assumed that you were referring to moving xvmalloc() down into drivers/block. That would be bad, because then xvmalloc() will _never_ be usable by anything other than ramzblock <new name!>?

I was also referring to moving xvmalloc to drivers/block. I meant that for now maybe move it to drivers/block it that can help speed up the merge. Maybe later if someone else find it useful too then we can work to move it back to the real place: mm/ :)

Thanks,
Nitin

#12 lore raw

Re: [PATCH 2/3] xvmalloc memory allocator

In reply to Andrew Morton
Quoted 3 lines · Andrew Morton

I assumed that you were referring to moving xvmalloc() down into drivers/block.  That would be bad, because then xvmalloc() will _never_ be usable by anything other than ramzblock <new name!>?

Who is going to use it? The only reason compcache needs something special is because it wants to take advantage of GFP_HIGHMEM pages. Are there other subsystems that need this capability as well?

#13 lore raw

Re: [PATCH 2/3] xvmalloc memory allocator

Quoted 10 lines · Pekka Enberg
Quoted 3 lines · Andrew Morton

I assumed that you were referring to moving xvmalloc() down into drivers/block. That would be bad, because then xvmalloc() will _never_ be usable by anything other than ramzblock <new name!>?

Who is going to use it? The only reason compcache needs something special is because it wants to take advantage of GFP_HIGHMEM pages. Are there other subsystems that need this capability as well?

As I mentioned earlier, highmem is not the only advantage. Don't forget O(1) alloc/free and low fragmentation. Sometime in next week, I will post additional numbers comparing SLUB and xvmalloc.

One point I noted in SLUB is that, it needs to allocate higher order pages to minimize space wastage at end of every page. For in-memory swap compression, we simply cannot allocate higher order pages since its going to used under memory crunch (its a swap device!) and we cannot hope to find lot of higher order pages under such conditions. If we enforce it to use 0-order pages then we cannot allocate > 2048b since all such allocations will end-up using entire page!
Also, if we decide to use SLUB for objects of size < 2048b only then how will we store bigger objects provided we can only use 0-order pages? (we need storage for range, say, [32, 3/4*PAGE_SIZE]).

Nitin

#14 lore raw

PATCH 3/3 compcache documentation

In reply to Nitin Gupta
 Documentation/blockdev/compcache.txt |   67 ++++++++++++++++++++++++++++++++++
 Documentation/kernel-parameters.txt  |    9 +++++
 init/Kconfig                         |    6 +++
 3 files changed, 82 insertions(+), 0 deletions(-)

Documentation on how to use compcache module.

---

Documentation/blockdev/compcache.txt new file +67 -0
new file mode 100644
index 0000000..46c33a5
@@ -0,0 +1,67 @@
+compcache: Compressed RAM swap device
+-------------------------------------
+
+Project home: http://code.google.com/p/compcache/
+
+This module creates RAM based block device (named ramzswap0) which acts
+as swap disk. Pages swapped to this disk are compressed and stored in
+memory itself.
+
+It uses these components:
+ - xvMalloc: memory allocator (xvmalloc.ko)
+ - LZO1X: de/compressor: (lzo_compress.ko, lzo_decompress.ko)
+
+Usage:
+ - modprobe compcache [memlimit_kb=<val>|disksize_kb=<val>] [backing_dev=<dev>]
+
+   memlimit_kb: This param is applicable only when backing_dev is given.
+	It is limit on amount compressed data stored in memory. Any
+	additional data is forwarded to backing_dev. It cannot be greater
+	than backing device size. If missing or 0, default value is used:
+	15% of RAM or backing device size, whichever is smaller.
+
+   disksize_kb: This param is applicable only when backing_dev is not given.
+	It is limit on amount of *uncompressed* worth of data stored in
+	memory. For e.g. disksize_kb=1024 means it can hold 1024kb worth of
+	uncompressed data even if this data compresses to just, say, 100kb.
+	If missing or 0, default value is used: 25% of RAM.
+
+   backing_dev: This is block device to be used as backing store for compcache.
+	It must be a valid swap partition. We move data to this device when we
+	encounter incompressible page or memlimit is reached. TODO: we may also
+	move some pages from compcache to this device in case system is really
+	low on memory.
+	This device is not directly visible to kernel as a swap device
+	(/proc/swaps will only show /dev/ramzswap0 and not this device).
+	Managing this backing device is the job of compcache module.
+
+Examples:
+	1) modprobe compcache memlimit_kb=10240 backing_dev=/dev/sda2
+	sets compcache limit as 10MB and /dev/sda2 as backing swap device.
+	NOTE: here /dev/sda2 is a valid swap partition.
+
+	2) modprobe compcache backing_dev=/dev/sda2
+	same as (1) but memlimit is set to default: 15% of RAM or size of
+	backing swap device, whichever is smaller.
+
+	3) modprobe compcache disksize_kb=10240
+	sets compcache disk size as 10MB.
+
+	4) modprobe compcache.ko
+	same as (3) but compcache disk size will be set to default:
+	25% of RAM size.
+
+	Once module is loaded, activate this swap with highest priority:
+	swapon /dev/ramzswap0 -p 100
+	(-p param set swap priority)
+
+Notes:
+ - compcache stats are exported via /proc/compcache
+ - If you give non-swap partition as backing_dev, nothing bad will happen -
+   swapon will simply fail to recognize /dev/ramzswap0 as swap partition.
+   So, in this case, unload the module and reload with correct backing_dev.
+
+Please report any problems to:
+
+Nitin Gupta
[email protected]
Documentation/kernel-parameters.txt +9 -0
index 54f21a5..a23f0ae 100644
@@ -414,6 +414,15 @@ and is between 256 and 4096 characters. It is defined in the file
 			possible to determine what the correct size should be.
 			This option provides an override for these situations.
 
+	compcache.memlimit_kb=
+			See Documentation/blockdev/compcache.txt.
+
+	compcache.disksize_kb=
+			See Documentation/blockdev/compcache.txt.
+
+	compcache.backing_dev=
+			See Documentation/blockdev/compcache.txt.
+
 	security=	[SECURITY] Choose a security module to enable at boot.
 			If this boot parameter is not specified, only the first
 			security module asking for security registration will be
init/Kconfig +6 -0
index 6a5c5fe..fa41598 100644
@@ -930,6 +930,12 @@ config SLOB
 
 endchoice
 
+config XVMALLOC
+	tristate "xvMalloc memory allocator"
+	help
+	   This is a simple, low fragmentation, O(1) allocator.
+	   Details: http://code.google.com/p/compcache/wiki/xvMalloc
+
 config PROFILING
 	bool "Profiling support (EXPERIMENTAL)"
 	help
maintainer
#15 lore raw

Re: [PATCH 3/3] compcache documentation

Documentation/blockdev/compcache.txt line 36

Quoted 10 lines · On Fri, 20 Mar 2009 19:44:08 +0530 Nitin Gupta <[email protected]> wrote:

...

Documentation/blockdev/compcache.txt new file +8 -0
+   backing_dev: This is block device to be used as backing store for compcache.
+	It must be a valid swap partition. We move data to this device when we
+	encounter incompressible page or memlimit is reached. TODO: we may also
+	move some pages from compcache to this device in case system is really
+	low on memory.
+	This device is not directly visible to kernel as a swap device
+	(/proc/swaps will only show /dev/ramzswap0 and not this device).
+	Managing this backing device is the job of compcache module.

hrm, OK, it kinda/sorta/maybe might be a little bit like a "compressed cache". But if someone comes up with a real compressed cache (ie:
pagecache) then they'll want their name back.

Also, the kernel already has a widespread and well-established "backing device" concept, in include/linux/backing-dev.h. This new feature's poaching of that identifier is a bit confusing. It would be nice to do a wholesale rename if possible. secondary_dev? nv_dev?

#16 lore raw

Re: [PATCH 3/3] compcache documentation

Quoted 4 lines · Andrew Morton
Quoted 2 lines · On Fri, 20 Mar 2009 19:44:08 +0530 Nitin Gupta <[email protected]> wrote:

...

Quoted 13 lines

Documentation/blockdev/compcache.txt line 36

Quoted 8 lines
Documentation/blockdev/compcache.txt new file +8 -0
+   backing_dev: This is block device to be used as backing store for compcache.
+	It must be a valid swap partition. We move data to this device when we
+	encounter incompressible page or memlimit is reached. TODO: we may also
+	move some pages from compcache to this device in case system is really
+	low on memory.
+	This device is not directly visible to kernel as a swap device
+	(/proc/swaps will only show /dev/ramzswap0 and not this device).
+	Managing this backing device is the job of compcache module.

hrm, OK, it kinda/sorta/maybe might be a little bit like a "compressed cache". But if someone comes up with a real compressed cache (ie:
pagecache) then they'll want their name back.

Rename is in progress: compcache->ramzswap :)

Quoted 5 lines

Also, the kernel already has a widespread and well-established "backing device" concept, in include/linux/backing-dev.h. This new feature's poaching of that identifier is a bit confusing. It would be nice to do a wholesale rename if possible. secondary_dev? nv_dev?

backing_swap?

Thanks,
Nitin

maintainer
#17 lore raw

Re: [PATCH 3/3] compcache documentation

Quoted 20 lines · On Sat, 21 Mar 2009 18:01:40 +0530 Nitin Gupta <[email protected]> wrote:
Quoted 4 lines · Andrew Morton
Quoted 2 lines · On Fri, 20 Mar 2009 19:44:08 +0530 Nitin Gupta <[email protected]> wrote:

...

Quoted 13 lines

Documentation/blockdev/compcache.txt line 36

Quoted 8 lines
Documentation/blockdev/compcache.txt new file +8 -0
+   backing_dev: This is block device to be used as backing store for compcache.
+	It must be a valid swap partition. We move data to this device when we
+	encounter incompressible page or memlimit is reached. TODO: we may also
+	move some pages from compcache to this device in case system is really
+	low on memory.
+	This device is not directly visible to kernel as a swap device
+	(/proc/swaps will only show /dev/ramzswap0 and not this device).
+	Managing this backing device is the job of compcache module.

hrm, OK, it kinda/sorta/maybe might be a little bit like a "compressed cache". But if someone comes up with a real compressed cache (ie:
pagecache) then they'll want their name back.

Rename is in progress: compcache->ramzswap :)

Quoted 7 lines
Quoted 5 lines

Also, the kernel already has a widespread and well-established "backing device" concept, in include/linux/backing-dev.h. This new feature's poaching of that identifier is a bit confusing. It would be nice to do a wholesale rename if possible. secondary_dev? nv_dev?

backing_swap?

Well that gets us back to my other question. It's a block device driver, yes? It is in fact not tied to swap in any way, apart from swap being the expected application?