Structure in Compressed Cache
Hello,
In my implementation of compressed cache (kernel 2.2.16), I
started the project having my cache as a slab cache, structure provided by kernel. I have all step 1 (a cache with no compression) done, but I had a problem with marking pages in my cache. After an email sent to the list about this subject, I started looking at shared memory mechanism (mainly ipc/shm.c), and I saw that there's another way of making it: with a page table allocation and memory mapping. I could go on with my initial idea (with slab cache) but I think that doing the latter way (with page table and memory mapping) would be more complete (and, of course, harder). I will have a pool of (compressed) pages that gotta be always in memory and will be "between" physical memory and swap. As the project is growing I would like to define now which path to follow, taking in account completeness and upgradeability (to future versions of kernel). Which way do you think that is better? Please, I also ask you to tell me in case you know if there's another way, maybe better, of doing it.
Thank you in advance,
Signature · 7 lines
Rodrigo S. de Castro <[email protected]> University of Sao Paulo - Brazil Compressed caching - http://tutu.ime.usp.br -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM, see: http://www.linux.eu.org/Linux-MM/
Quoted 18 lines · Rodrigo S. de Castro
Hello,
In my implementation of compressed cache (kernel 2.2.16), I
started the project having my cache as a slab cache, structure provided by kernel. I have all step 1 (a cache with no compression) done, but I had a problem with marking pages in my cache. After an email sent to the list about this subject, I started looking at shared memory mechanism (mainlyipc/shm.c), and I saw that there's another way of making it: with a page table allocation and memory mapping. I could go on with my initial idea (with slab cache) but I think that doing the latter way (with page table and memory mapping) would be more complete (and, of course, harder). I will have a pool of (compressed) pages that gotta be always in memory and will be "between" physical memory and swap. As the project is growing I would like to define now which path to follow, taking in account completeness and upgradeability (to future versions of kernel). Which way do you think that is better? Please, I also ask you to tell me in case you know if there's another way, maybe better, of doing it.
Slab cache memory is physically contiguous and non swappable, so it may be a waste to use it to cache userspace data.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux.eu.org/Linux-MM/
Quoted 18 lines · Marcelo Tosatti
Quoted 8 lines · Rodrigo S. de Castro
In my implementation of compressed cache (kernel 2.2.16), I
started the project having my cache as a slab cache, structure provided by kernel. I have all step 1 (a cache with no compression) done, but I had a problem with marking pages in my cache. After an email sent to the list about this subject, I started looking at shared memory mechanism (mainlyipc/shm.c), and I saw that there's another way of making it: with a page table allocation and memory mapping. I could go on with my initial idea (with slab cache) but I think thatQuoted 8 lines
doing the latter way (with page table and memory mapping) would be more complete (and, of course, harder). I will have a pool of (compressed) pages that gotta be always in memory and will be "between" physical memory and swap. As the project is growing I would like to define now which path to follow, taking in account completeness and upgradeability (to future versions of kernel). Which way do you think that is better? Please, I also ask you to tell me in case you know if there's another way, maybe better, of doing it.
Quoted 2 lines
Slab cache memory is physically contiguous and non swappable, so it may be a waste to use it to cache userspace data.
Data in slab cache is not supposed to be swapped. Currently,
the slab cache stores only a structure with some information (such as task, original pte) and a pointer to a physical page that is allocated at the beginning of my compressed_init() function. I manage, when cache fills up, to swap data present in slab cache. It will be user data (every data that may be swapped), but it will managed by our functions. It seemed to me that a page table (with memory mapping) would be a better choice to put all compressed levels in the same "level" as shared pages, for example, and it seemed to be more complete. What do you think? I hope I haven't said something too wrong. :-)
Would I have problem with slab cache when it grows and
shrinks, in one advanced phase of our project, due to have contiguous data? I couldn't notice this problem when I read some papers about slab cache, but I am not sure as well.
Slab cache was our first choice because it allowed this
dynamism (growing and shrinking) with no big effort. In the case of having a page table, we must have all engine built to support this feature.
Thank you very much for your help,
Signature · 9 lines
Rodrigo S. de Castro <[email protected]> University of Sao Paulo - Brazil Compressed Caching - http://tutu.ime.usp.br -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM, see: http://www.linux.eu.org/Linux-MM/
Quoted 18 lines · Marcelo Tosatti
Quoted 8 lines · Rodrigo S. de Castro
In my implementation of compressed cache (kernel 2.2.16), I
started the project having my cache as a slab cache, structure provided by kernel. I have all step 1 (a cache with no compression) done, but I had a problem with marking pages in my cache. After an email sent to the list about this subject, I started looking at shared memory mechanism (mainlyipc/shm.c), and I saw that there's another way of making it: with a page table allocation and memory mapping. I could go on with my initial idea (with slab cache) but I think thatQuoted 8 lines
doing the latter way (with page table and memory mapping) would be more complete (and, of course, harder). I will have a pool of (compressed) pages that gotta be always in memory and will be "between" physical memory and swap. As the project is growing I would like to define now which path to follow, taking in account completeness and upgradeability (to future versions of kernel). Which way do you think that is better? Please, I also ask you to tell me in case you know if there's another way, maybe better, of doing it.
Quoted 2 lines
Slab cache memory is physically contiguous and non swappable, so it may be a waste to use it to cache userspace data.
Today I reread Bonwick's paper (The Slab Allocator: An
Object-Caching Kernel Memory Allocator) and I saw that a slab of objects 'consists of one or more pages of virtually contiguous memory' and the whole cache (that has many slabs) doesn't, necessarily. I am storing on a slab cache only a small structure that holds some information about and a pointer to a physical page that is allocated in its constructor. In that way, a slab will certainly have a size of a page (since its structure is smaller than 1/8 of page), and thus there's no problem for us at all, because there won't have actually any continguous data. I hope you could get it. I am wondering if I missing something, because I can't see a problem that this approach would waste. :-) I did tried to understand some code of slab.c and even then I couldn't see contiguous memory problem in our whole cache. Well, I ask anyone who knows something about slab cache to join this discussion. :-)
[]'s
Signature · 10 lines
Rodrigo S. de Castro <[email protected]> University of Sao Paulo - Brazil Compressed Caching - http://tutu.ime.usp.br -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM, see: http://www.linux.eu.org/Linux-MM/