Tmem [PATCH 0/5] (Take 3): Transcendent memory
Hi Dan,
(I'm not sure if gmane.org interface sends mail to everyone in CC list, so sending again. Sorry if you are getting duplicate mail).
Quoted 3 lines · Dan Magenheimer <dan.magenheimer <at> oracle.com> writes:
Tmem [PATCH 0/5] (Take 3): Transcendent memory
Transcendent memory
<snip>
Quoted 11 lines
Normal memory is directly addressable by the kernel, of a known normally-fixed size, synchronously accessible, and persistent (though not across a reboot).
What if there was a class of memory that is of unknown and dynamically variable size, is addressable only indirectly by the kernel, can be configured either as persistent or as "ephemeral" (meaning it will be around for awhile, but might disappear without warning), and is still fast enough to be synchronously accessible?
I really like the idea of allocating cache memory from hypervisor directly. This is much more flexible than assigning fixed size memory to guests.
Quoted 15 lines
"Frontswap" is so named because it can be thought of as the opposite of a "backing store". Frontswap IS persistent, but for various reasons may not always be available for use, again due to factors that may not be visible to the kernel. (But, briefly, if the kernel is being "good" and has shared its resources nicely, then it will be able to use frontswap, else it will not.) Once a page is put, a get on the page will always succeed. So when the kernel finds itself in a situation where it needs to swap out a page, it first attempts to use frontswap. If the put works, a disk write and (usually) a disk read are avoided. If it doesn't, the page is written to swap as usual. Unlike cleancache, whether a page is stored in frontswap vs swap is recorded in kernel data structures, so when a page needs to be fetched, the kernel does a get if it is in frontswap and reads from swap if it is not in frontswap.
I think 'frontswap' part seriously overlaps the functionality provided by 'ramzswap' which is a virtual block device driver recently added to drivers/staging/ramzswap/. This device acts as a swap disk which compresses and stores pages in memory itself.
To provide frontswap functionality, ramzswap needs few changes only:
instead of:
compress --> alloc and store within guest.
do:
compress --> send out to hypervisor (tmem_put_page).
Also, ramzswap driver supports multiple /dev/ramzswap{0,1,2...} devices. Each of these devices can have separate backing partition/file which is used to flush out incompressible pages or when (per-device) memory limit is exceeded. When used on native systems, it uses custom xvmalloc allocator which is specially designed to handle these compressed pages.
We can use all this by just a minor change in ramzswap as mentioned above.
Quoted 10 lines
"Cleancache" can be thought of as a page-granularity victim cache for clean pages that the kernel's pageframe replacement algorithm (PFRA) would like
to keep around, but can't since there isn't enough memory. So when the
PFRA "evicts" a page, it first puts it into the cleancache via a call to tmem. And any time a filesystem reads a page from disk, it first attempts to get the page from cleancache. If it's there, a disk access is eliminated. If not, the filesystem just goes to the disk like normal. Cleancache is "ephemeral" so whether a page is kept in cleancache (between the "put" and the "get") is dependent on a number of factors that are invisible to the kernel.
Just an idea: as an alternate approach, we can create an 'in-memory compressed storage' backend for FS-Cache. This way, all filesystems modified to use fs-cache can benefit from this backend. To make it virtualization friendly like tmem, we can again provide (per-cache?) option to allocate from hypervisor i.e. tmem_{put,get}_page() or use [compress]+alloc natively.
For guest<-->hypervisor interface, maybe we can use virtio so that all hypervisors can benefit? Not quite sure about this one.
Thanks,
Nitin
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Quoted 3 lines
From: Nitin Gupta [mailto:[email protected]]
Hi Dan,
Hi Nitin --
Thanks for your review!
Quoted 3 lines
(I'm not sure if gmane.org interface sends mail to everyone in CC list, so
sending again. Sorry if you are getting duplicate mail).
FWIW, I only got this one copy (at least so far)!
Quoted 3 lines
I really like the idea of allocating cache memory from hypervisor directly. This
is much more flexible than assigning fixed size memory to guests.
Thanks!
Quoted 2 lines
I think 'frontswap' part seriously overlaps the functionality provided by 'ramzswap'
Could be, but I suspect there's a subtle difference. A key part of the tmem frontswap api is that any "put" at any time can be rejected. There's no way for the kernel to know a priori whether the put will be rejected or not, and the kernel must be able to react by writing the page to a "true" swap device and must keep track of which pages were put to tmem frontswap and which were written to disk. As a result, tmem frontswap cannot be configured or used as a true swap "device".
This is critical to acheive the flexibility you commented above that you like. Only the hypervisor knows if a free page is available "now" because it is flexibly managing tmem requests from multiple guest kernels.
If my understanding of ramzswap is incorrect or you have some clever solution that I misunderstood, please let me know.
Quoted 6 lines
Quoted 2 lines
Cleancache is
"ephemeral" so whether a page is kept in cleancache(between the "put" and
Quoted 2 lines
the "get") is dependent on a number of factors that are invisible to the kernel.
Quoted 9 lines
Just an idea: as an alternate approach, we can create an 'in-memory compressed
storage' backend for FS-Cache. This way, all filesystems modified to use
fs-cache can benefit from this backend. To make it virtualization friendly like
tmem, we can again provide (per-cache?) option to allocate from hypervisor i.e.
tmem_{put,get}_page() or use [compress]+alloc natively.
I looked at FS-Cache and cachefiles and thought I understood that it is not restricted to clean pages only, thus not a good match for tmem cleancache.
Again, if I'm wrong (or if it is easy to tell FS-Cache that pages may "disappear" underneath it), let me know.
BTW, pages put to tmem (both frontswap and cleancache) can be optionally compressed.
Quoted 2 lines
For guest<-->hypervisor interface, maybe we can use virtio so that all hypervisors can benefit? Not quite sure about this one.
I'm not very familiar with virtio, but the existence of "I/O" in the name concerns me because tmem is entirely synchronous.
Also, tmem is well-layered so very little work needs to be done on the Linux side for other hypervisors to benefit. Of course these other hypervisors would need to implement the hypervisor-side of tmem as well, but there is a well-defined API to guide other hypervisor-side implementations... and the opensource tmem code in Xen has a clear split between the hypervisor-dependent and hypervisor-independent code, which should simplify implementation for other opensource hypervisors.
I realize in "Take 3" I didn't provide the URL for more information:
http://oss.oracle.com/projects/tmem
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Hi Dan,
(mail to Rusty [at] rcsinet15.oracle.com was failing, so I removed this address from CC list).
Quoted 17 lines · Dan Magenheimer
Quoted 5 lines
From: Nitin Gupta [mailto:[email protected]]
I think 'frontswap' part seriously overlaps the functionality provided by 'ramzswap'
Could be, but I suspect there's a subtle difference. A key part of the tmem frontswap api is that any "put" at any time can be rejected. There's no way for the kernel to know a priori whether the put will be rejected or not, and the kernel must be able to react by writing the page to a "true" swap device and must keep track of which pages were put to tmem frontswap and which were written to disk. As a result, tmem frontswap cannot be configured or used as a true swap "device".
Quoted 6 lines
This is critical to acheive the flexibility you commented above that you like. Only the hypervisor knows if a free page is available "now" because it is flexibly managing tmem requests from multiple guest kernels.
ramzswap devices can easily track which pages it sent to hypervisor, which pages are in backing swap (physical) disk and which are in (compressed) memory. Its simply a matter of adding some more flags. Latter two are already done in this driver.
So, to gain flexibility of frontswap, we can have hypervisor send the driver a callback whenever it wants to discard swap pages under its domain. If you want to avoid even this callback, then kernel will have to keep a copy within guest, which I think defeats the whole purpose of swapping to hypervisor. Such "ephemeral" pools should be used only for clean fs cache and not for swap.
Swapping to hypervisor is mainly useful to overcome 'static partitioning' problem you mentioned in article:
http://oss.oracle.com/projects/tmem/
...such 'para-swap' can shrink/expand outside of VM constraints.
Quoted 17 lines
Quoted 7 lines
Quoted 3 lines
Cleancache is
"ephemeral" so whether a page is kept in cleancache(between the "put" and
Quoted 2 lines
the "get") is dependent on a number of factors that are invisible to the kernel.
Quoted 9 lines
Just an idea: as an alternate approach, we can create an 'in-memory compressed
storage' backend for FS-Cache. This way, all filesystems modified to use
fs-cache can benefit from this backend. To make it virtualization friendly like
tmem, we can again provide (per-cache?) option to allocate from hypervisor i.e.
tmem_{put,get}_page() or use [compress]+alloc natively.
Quoted 7 lines
I looked at FS-Cache and cachefiles and thought I understood that it is not restricted to clean pages only, thus not a good match for tmem cleancache.
Again, if I'm wrong (or if it is easy to tell FS-Cache that pages may "disappear" underneath it), let me know.
fs-cache backend can keep 'dirty' pages within guest and forward clean pages to hypervisor. These clean pages can be added to ephemeral pools which can be reclaimed at any time by hypervisor. BTW, I have not yet started work on any such fs-cache backend, so we might later encounter some hidder/dangerous problems :)
Quoted 3 lines
BTW, pages put to tmem (both frontswap and cleancache) can be optionally compressed.
If ramzswap is extended for this virtualization case, then enforcing compression might not be good. We can then throw out pages to hvisor
even before compression stage. All such changes to ramzswap are IMHO
pretty straight forward to do.
Quoted 6 lines
Quoted 2 lines
For guest<-->hypervisor interface, maybe we can use virtio so that all hypervisors can benefit? Not quite sure about this one.
I'm not very familiar with virtio, but the existence of "I/O" in the name concerns me because tmem is entirely synchronous.
Is synchronous working a *requirement* for tmem to work correctly?
Quoted 9 lines
Also, tmem is well-layered so very little work needs to be done on the Linux side for other hypervisors to benefit. Of course these other hypervisors would need to implement the hypervisor-side of tmem as well, but there is a well-defined API to guide other hypervisor-side implementations... and the opensource tmem code in Xen has a clear split between the hypervisor-dependent and hypervisor-independent code, which should simplify implementation for other opensource hypervisors.
As I mentioned, I really like the idea behind tmem. All I am proposing is that we should probably explore some alternatives to achive this using some existing infrastructure in kernel. I also don't have experience working on virtio[1] or virtual-bus[2] but I have the feeling that once guest to hvisor channels are created, both ramzswap extension and fs-cache backend can share the same code.
[1] virtio: http://portal.acm.org/citation.cfm?id=1400097.1400108
[2] virtual-bus: http://developer.novell.com/wiki/index.php/Virtual-bus
Thanks,
Nitin
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Quoted 3 lines
As I mentioned, I really like the idea behind tmem. All I am proposing is that we should probably explore some alternatives to achive this using some existing infrastructure in kernel.
Hi Nitin --
Sorry if I sounded overly negative... too busy around the holidays.
I'm definitely OK with exploring alternatives. I just think that existing kernel mechanisms are very firmly rooted in the notion that either the kernel owns the memory/cache or an asynchronous device owns it. Tmem falls somewhere in between and is very carefully designed to maximize memory flexibility *outside* of the kernel -- across all guests in a virtualized environment -- with minimal impact to the kernel, while still providing the kernel with the ability to use -- but not own, directly address, or control -- additional memory when conditions allow. And these conditions are not only completely invisible to the kernel, but change frequently and asynchronously from the kernel, unlike most external devices for which the kernel can "reserve" space and use it asynchronously later.
Maybe ramzswap and FS-cache could be augmented to have similar advantages in a virtualized environment, but I suspect they'd end up with something very similar to tmem. Since the objective of both is to optimize memory that IS owned (used, directly addressable, and controlled) by the kernel, they are entirely complementary with tmem.
Quoted 1 line
Is synchronous working a *requirement* for tmem to work correctly?
Yes. Asynchronous behavior would introduce lots of race conditions between the hypervisor and kernel which would greatly increase complexity and reduce performance. And tmem then essentially becomes an I/O device, which defeats its purpose, especially compared to a fast SSD.
Quoted 4 lines
Swapping to hypervisor is mainly useful to overcome 'static partitioning' problem you mentioned in article:
http://oss.oracle.com/projects/tmem/
...such 'para-swap' can shrink/expand outside of VM constraints.
Frontswap is very different than "hypervisor swapping" as what's done by VMware as a side-effect of transparent page-sharing. With frontswap, the kernel still decides which pages are swapped out. If frontswap says there is space, the swap goes "fast" to tmem; if not, the kernel writes it to its own swapdisk. So there's no "double paging" or random page selection/swapping. On the downside, kernels must have real swap configured and, to avoid DoS issues, frontswap is limited by the same constraint as ballooning (ie. can NOT expand outside of VM constraints).
Thanks,
Dan
P.S. If you want to look at implementing FS-cache or ramzswap on top of tmem, I'd be happy to help, but I'll bet your concern:
Quoted 1 line
we might later encounter some hidder/dangerous problems :)
will prove to be correct.
Quoted 138 lines
-----Original Message-----
From: Nitin Gupta [mailto:[email protected]]
Sent: Tuesday, December 22, 2009 11:28 PM
To: Dan Magenheimer
Cc: Nick Piggin; Andrew Morton; [email protected]; [email protected]; [email protected]; Rusty Russell;
Rik van Riel; Dave Mccracken; Sunil Mushran; Avi Kivity; Schwidefsky; Balbir Singh; Marcelo Tosatti; Alan Cox; Chris Mason; Pavel Machek; linux-mm; linux-kernel
Subject: Re: Tmem [PATCH 0/5] (Take 3): Transcendent memoryHi Dan,
(mail to Rusty [at] rcsinet15.oracle.com was failing, so I removed this address from CC list).
Quoted 17 lines · Dan Magenheimer
Quoted 5 lines
From: Nitin Gupta [mailto:[email protected]]
I think 'frontswap' part seriously overlaps the functionality provided by 'ramzswap'
Could be, but I suspect there's a subtle difference. A key part of the tmem frontswap api is that any "put" at any time can be rejected. There's no way for the kernel to know a priori whether the put will be rejected or not, and the kernel must be able to react by writing the page to a "true" swap device and must keep track of which pages were put to tmem frontswap and which were written to disk. As a result, tmem frontswap cannot be configured or used as a true swap "device".
Quoted 6 lines
This is critical to acheive the flexibility you commented above that you like. Only the hypervisor knows if a free page is available "now" because it is flexibly managing tmem requests from multiple guest kernels.
ramzswap devices can easily track which pages it sent to hypervisor, which pages are in backing swap (physical) disk and which are in (compressed) memory. Its simply a matter of adding some more flags. Latter two are already done in this driver.
So, to gain flexibility of frontswap, we can have hypervisor send the driver a callback whenever it wants to discard swap pages under its domain. If you want to avoid even this callback, then kernel will have to keep a copy within guest, which I think defeats the whole purpose of swapping to hypervisor. Such "ephemeral" pools should be used only for clean fs cache and not for swap.
Swapping to hypervisor is mainly useful to overcome 'static partitioning' problem you mentioned in article:
http://oss.oracle.com/projects/tmem/
...such 'para-swap' can shrink/expand outside of VM constraints.Quoted 5 lines
Quoted 5 lines
Quoted 3 lines
Cleancache is
"ephemeral" so whether a page is kept in cleancache(between the "put" and
Quoted 1 line
the "get") is dependent on a number of factors that are
invisible to
Quoted 12 lines
Quoted 11 lines
Quoted 1 line
the kernel.
Just an idea: as an alternate approach, we can create an 'in-memory compressed
storage' backend for FS-Cache. This way, all filesystems modified to use
fs-cache can benefit from this backend. To make it virtualization friendly like
tmem, we can again provide (per-cache?) option to allocate from hypervisor i.e.
tmem_{put,get}_page() or use [compress]+alloc natively.Quoted 7 lines
I looked at FS-Cache and cachefiles and thought I understood that it is not restricted to clean pages only, thus not a good match for tmem cleancache.
Again, if I'm wrong (or if it is easy to tell FS-Cache that pages may "disappear" underneath it), let me know.
fs-cache backend can keep 'dirty' pages within guest and forward clean pages to hypervisor. These clean pages can be added to ephemeral pools which can be reclaimed at any time by hypervisor. BTW, I have not yet started work on any such fs-cache backend, so we might later encounter some hidder/dangerous problems :)
Quoted 3 lines
BTW, pages put to tmem (both frontswap and cleancache) can be optionally compressed.
If ramzswap is extended for this virtualization case, then enforcing compression might not be good. We can then throw out pages to hvisor
even before compression stage. All such changes to ramzswap are IMHO
pretty straight forward to do.Quoted 1 line
Quoted 1 line
For guest<-->hypervisor interface, maybe we can use virtio
so that all
Quoted 5 lines
Quoted 1 line
hypervisors can benefit? Not quite sure about this one.
I'm not very familiar with virtio, but the existence of "I/O" in the name concerns me because tmem is entirely synchronous.
Is synchronous working a *requirement* for tmem to work correctly?
Quoted 9 lines
Also, tmem is well-layered so very little work needs to be done on the Linux side for other hypervisors to benefit. Of course these other hypervisors would need to implement the hypervisor-side of tmem as well, but there is a well-defined API to guide other hypervisor-side implementations... and the opensource tmem code in Xen has a clear split between the hypervisor-dependent and hypervisor-independent code, which should simplify implementation for other opensource hypervisors.
As I mentioned, I really like the idea behind tmem. All I am proposing is that we should probably explore some alternatives to achive this using
some existing infrastructure in kernel. I also don't have experience working
on virtio[1] or virtual-bus[2] but I have the feeling that once guest to hvisor channels are created, both ramzswap extension and fs-cache backend
can share the same code.
Quoted 2 lines
[1] virtio: http://portal.acm.org/citation.cfm?id=1400097.1400108
[2] virtual-bus:
http://developer.novell.com/wiki/index.php/Virtual-bus
Thanks,
Nitin
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Hi Dan,
Quoted 14 lines · Dan Magenheimer
I'm definitely OK with exploring alternatives. I just think that existing kernel mechanisms are very firmly rooted in the notion that either the kernel owns the memory/cache or an asynchronous device owns it. Tmem falls somewhere in between and is very carefully designed to maximize memory flexibility *outside* of the kernel -- across all guests in a virtualized environment -- with minimal impact to the kernel, while still providing the kernel with the ability to use -- but not own, directly address, or control -- additional memory when conditions allow. And these conditions are not only completely invisible to the kernel, but change frequently and asynchronously from the kernel, unlike most external devices for which the kernel can "reserve" space and use it asynchronously later.
Quoted 7 lines
Maybe ramzswap and FS-cache could be augmented to have similar advantages in a virtualized environment, but I suspect they'd end up with something very similar to tmem. Since the objective of both is to optimize memory that IS owned (used, directly addressable, and controlled) by the kernel, they are entirely complementary with tmem.
What we want is surely tmem but attempt is to better integrate with existing infrastructure. Please give me few days as I try to develop a prototype.
Quoted 5 lines
Quoted 4 lines
Swapping to hypervisor is mainly useful to overcome 'static partitioning' problem you mentioned in article:
http://oss.oracle.com/projects/tmem/
...such 'para-swap' can shrink/expand outside of VM constraints.
Quoted 10 lines
Frontswap is very different than "hypervisor swapping" as what's done by VMware as a side-effect of transparent page-sharing. With frontswap, the kernel still decides which pages are swapped out. If frontswap says there is space, the swap goes "fast" to tmem; if not, the kernel writes it to its own swapdisk. So there's no "double paging" or random page selection/swapping. On the downside, kernels must have real swap configured and, to avoid DoS issues, frontswap is limited by the same constraint as ballooning (ie. can NOT expand outside of VM constraints).
I think I did not explain my point regarding "para-swap" correctly. What I meant was a virtual swap device which appears as swap disk to kernel. Kernel swaps to this disk as usual (hence the kernel decides what pages to swap out). This device tries to send these pages to hvisor but if that fails, it will fall back to swapping inside guest only. There is no double swapping. I think this correctly explains purpose of Frontswap.
Quoted 1 line
Quoted 1 line
...such 'para-swap' can shrink/expand outside of VM constraints.
<snip>
Quoted 3 lines
frontswap is limited by the same constraint
as ballooning (ie. can NOT expand outside of VM constraints).
What I meant was: A VM can have 512M of memory while this "para swap" disk can have any size, say 1G. Now kernel can swapout 1G worth of data to this swap which can (potentially) send all these pages to hvisor. In future, if we want even more RAM for this VM, we can add another such swap device to guest. Thus, in a way, we are able to overcome rigid static partitioning of VMs w.r.t. memory resource.
Quoted 8 lines
P.S. If you want to look at implementing FS-cache or ramzswap on top of tmem, I'd be happy to help, but I'll bet your concern:Quoted 1 line
we might later encounter some hidder/dangerous problems :)
will prove to be correct.
Please allow me few days to experiment with 'virtswap' which will be virtualization aware ramzswap driver. This will help us understand problems we might face with such an approach. I am new to this virtio thing, so it might take some time.
If we find virtswap to be feasible with virtio, we can go for fs-cache backend we talked about.
Thanks,
Nitin
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Quoted 3 lines
What we want is surely tmem but attempt is to better integrate with existing infrastructure. Please give me few days as I try to develop a prototype.
Sounds good. I have a few more comments but will switch to cc'ing just the tmem-devel* list as most people and lists are probably not interested in this level of detail, but it will be archived on tmem-devel in case anyone else does want to follow it.
Thanks,
Dan
* http://oss.oracle.com/pipermail/tmem-devel/
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Quoted 4 lines · Dan Magenheimer
Quoted 3 lines
As I mentioned, I really like the idea behind tmem. All I am proposing is that we should probably explore some alternatives to achive this using some existing infrastructure in kernel.
Quoted 8 lines
Hi Nitin --
Sorry if I sounded overly negative... too busy around the holidays.
I'm definitely OK with exploring alternatives. I just think that existing kernel mechanisms are very firmly rooted in the notion that either the kernel owns the memory/cache or an asynchronous device owns it. Tmem falls somewhere in between and is very
Well... compcache seems to be very similar to preswap: in preswap case you don't know if hypervisor will have space, in ramzswap you don't know if data are compressible.
Pavel
Signature · 8 lines
(english) http://www.livejournal.com/~pavelmachek (cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Quoted 12 lines
From: Pavel Machek [mailto:[email protected]]
Quoted 1 line
Quoted 1 line
As I mentioned, I really like the idea behind tmem. All I
am proposing
Quoted 1 line
Quoted 1 line
is that we should probably explore some alternatives to
achive this using
Quoted 6 lines
Quoted 1 line
some existing infrastructure in kernel.
Hi Nitin --
Sorry if I sounded overly negative... too busy around the holidays.
Quoted 8 lines
Quoted 4 lines
I'm definitely OK with exploring alternatives. I just think that existing kernel mechanisms are very firmly rooted in the notion that either the kernel owns the memory/cache or an asynchronous device owns it. Tmem falls somewhere in between and is very
Well... compcache seems to be very similar to preswap: in preswap case you don't know if hypervisor will have space, in ramzswap you don't know if data are compressible.
Hi Pavel --
Yes there are definitely similarities too. In fact, I started prototyping preswap (now called frontswap) with Nitin's compcache code. IIRC I ran into some problems with compcache's difficulties in dealing with failed "puts" due to dynamic changes in size of hypervisor-available-memory.
Nitin may have addressed this in later versions of ramzswap.
One feature of frontswap which is different than ramzswap is that frontswap acts as a "fronting store" for all configured swap devices, including SAN/NAS swap devices. It doesn't need to be separately configured as a "highest priority" swap device. In many installations and depending on how ramzswap is configured, this difference probably doesn't make much difference though.
Thanks,
Dan
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Hi!
Quoted 18 lines
Quoted 7 lines
achive this using
Quoted 6 lines
Quoted 1 line
some existing infrastructure in kernel.
Hi Nitin --
Sorry if I sounded overly negative... too busy around the holidays.
Quoted 8 lines
Quoted 4 lines
I'm definitely OK with exploring alternatives. I just think that existing kernel mechanisms are very firmly rooted in the notion that either the kernel owns the memory/cache or an asynchronous device owns it. Tmem falls somewhere in between and is very
Well... compcache seems to be very similar to preswap: in preswap case you don't know if hypervisor will have space, in ramzswap you don't know if data are compressible.
Hi Pavel --
Quoted 7 lines
Yes there are definitely similarities too. In fact, I started prototyping preswap (now called frontswap) with Nitin's compcache code. IIRC I ran into some problems with compcache's difficulties in dealing with failed "puts" due to dynamic changes in size of hypervisor-available-memory.
Nitin may have addressed this in later versions of ramzswap.
That would be cool to find out.
Quoted 5 lines
One feature of frontswap which is different than ramzswap is that frontswap acts as a "fronting store" for all configured swap devices, including SAN/NAS swap devices. It doesn't need to be separately configured as a "highest priority" swap device. In many installations and depending on how ramzswap
Ok, I'd call it a bug, not a feature :-).
Pavel
Signature · 8 lines
(english) http://www.livejournal.com/~pavelmachek (cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Quoted 8 lines
Quoted 5 lines
One feature of frontswap which is different than ramzswap is that frontswap acts as a "fronting store" for all configured swap devices, including SAN/NAS swap devices. It doesn't need to be separately configured as a "highest priority" swap device. In many installations and depending on how ramzswap
Ok, I'd call it a bug, not a feature :-).
Pavel
I agree it has little value (or might be considered a bug) when managing Linux on a physical machine. But when Linux is running in a virtual machine, it's one less thing that a sysadmin needs to understand and configure.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Quoted 14 lines · Dan Magenheimer
Quoted 11 lines
From: Pavel Machek [mailto:[email protected]]
Quoted 4 lines
I'm definitely OK with exploring alternatives. I just think that existing kernel mechanisms are very firmly rooted in the notion that either the kernel owns the memory/cache or an asynchronous device owns it. Tmem falls somewhere in between and is very
Well... compcache seems to be very similar to preswap: in preswap case you don't know if hypervisor will have space, in ramzswap you don't know if data are compressible.
Hi Pavel --
Quoted 8 lines
Yes there are definitely similarities too. In fact, I started prototyping preswap (now called frontswap) with Nitin's compcache code. IIRC I ran into some problems with compcache's difficulties in dealing with failed "puts" due to dynamic changes in size of hypervisor-available-memory.
Nitin may have addressed this in later versions of ramzswap.
Any kind of swap device that works entirely within guest
(or in native case), will always have problems with any write(put) failure -- we want to reclaim a page but due to write failure, we can't. Problem! So, ramzswap also cannot afford to have lot of write failures.
However, the story is different when ramzswap is "virtualization aware". In this case, we can surely afford to have any numnber of "put" failures to hypervisor. When this put fails, we will either compress the page and keep it in guest memory itself or forward it to ramzswap backing swap device (if present).
Another side point is that we can achieve all this with ramzswap approach of virtual block devices without any kernel changes as everything is a module.
Quoted 8 lines
One feature of frontswap which is different than ramzswap is that frontswap acts as a "fronting store" for all configured swap devices, including SAN/NAS swap devices. It doesn't need to be separately configured as a "highest priority" swap device. In many installations and depending on how ramzswap is configured, this difference probably doesn't make much difference though.
Having a frontswap layer over *every* swap might not be desirable. I think such things should be completely out of way when not desired. This was one the primary reasons to have virtual block device approach for ramzswap. You can create any number of such devices (/dev/ramzswap{0,1,2...}) with each having separate backing device (optional), memory pools, buffers etc. which adds additional flexibility and helps with scalability.
On a downside however, as you pointed out, managing all this can be a problem for sysadmins. To ease this, some userspace magic can help which will dynamically manage these virtual disks, though I have not yet thought much in this direction.
Thanks,
Nitin
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>