Wrapper fs Code (wrapfs)
Hello All,
I have some questions regarding the wrapper fs code downloaded from filesystems.org
The questions are below:
(1) Finding out the meanings of FIST_FILTER_SCA and FIST_FILTER_DATA
(2) How wrapfs handles the dentry and inode cache of wrapfs and not the actual fs. While reading the code of struct dentry_operations of wrapfs, I found out that the operations involved don't do anything to the wrapfs dentry cache, these operations only deal with the actual fs dcache using actual fs dentry operations.
(3) When I update some file in the actual fs e.g. a file in the folder on ext3 file system, the changes done there are not reflected on the same file on wrapper fs but the vice versa is working correctly.
I want some details regarding these questions. Hope I am able to convey my point.
Thanks,
Uzair Lakhani,
Karachi, Pakistan.
__________________________________________________
Do You Yahoo!?
Tired of spam? Yahoo! Mail has the best spam protection around
http://mail.yahoo.com
On Thu, Aug 03, 2006 at 11:09:10PM -0700, UZAIR LAKHANI wrote:
...
Quoted 1 line
(1) Finding out the meanings of
FIST_FILTER_SCAand
The stackable filesystem changes the size of the data: for example, gzipfs takes n bytes and compresses them to m bytes.
Quoted 1 line
FIST_FILTER_DATA
Data gets transformed, but the size is the same: for example, rot13fs takes n bytes and runs them though rot13, the result is still n bytes.
Quoted 4 lines
(3) When I update some file in the actual fs e.g. a file in the folder on ext3 file system, the changes done there are not reflected on the same file on wrapper fs but the vice versa is working correctly.
That is an artifact of how the Linux VFS works. When you modify a file though wrapfs (or any other stackable file system) the wrapfs code gets called. It then calls the lower file system code on your behalf. When you modify the lower file directly, ext3 gets the called, but since Linux has no way for this change to be announced up the stack, wrapfs happily uses the old, cached data. If you have ideas about how to solve this, please share them - right now, there is a lot of work being done to get Unionfs (ours) and eCryptfs into the vanilla linux kernel.
Jeff.
Signature · 3 lines
Linux, n.: Generous programmers from around the world all join forces to help you shoot yourself in the foot for free.
Quoted 25 lines · --- Josef Sipek
On Thu, Aug 03, 2006 at 11:09:10PM -0700, UZAIR
LAKHANI wrote:
...Quoted 1 line
(1) Finding out the meanings of
FIST_FILTER_SCAand
The stackable filesystem changes the size of the
data: for example, gzipfs
takes n bytes and compresses them to m bytes.Quoted 1 line
FIST_FILTER_DATAData gets transformed, but the size is the same: for example, rot13fs takes
n bytes and runs them though rot13, the result is still n bytes.Quoted 1 line
(3) When I update some file in the actual fs e.g.
a
Quoted 1 line
file in the folder on ext3 file system, the
changes
Quoted 2 lines
done there are not reflected on the same file on
wrapper fs but the vice versa is workingcorrectly.
Quoted 16 lines
That is an artifact of how the Linux VFS works. When you modify a file
though wrapfs (or any other stackable file system) the wrapfs code gets
called. It then calls the lower file system code on your behalf. When you
modify the lower file directly, ext3 gets the called, but since Linux has no
way for this change to be announced up the stack, wrapfs happily uses the
old, cached data. If you have ideas about how to solve this, please share
them - right now, there is a lot of work being done to get Unionfs (ours)
and eCryptfs into the vanilla linux kernel.
Hello All,
Thanks for the reply. Can you please tell me where VFS caches the data, I have an idea about the dentry and inode cache but how VFS caches the actual data.
Can you please give some details.
Thanks,
Uzair Lakhani,
Karachi, Pakistan.
Quoted 14 lines
Jeff.
Signature · 10 lines
Linux, n.: Generous programmers from around the world all join forces to help you shoot yourself in the foot for free. - To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
__________________________________________________
Do You Yahoo!?
Tired of spam? Yahoo! Mail has the best spam protection around
http://mail.yahoo.com
Quoted 4 lines · UZAIR LAKHANI
Hello All,
Thanks for the reply. Can you please tell me where VFS caches the data, I have an idea about the dentry and inode cache but how VFS caches the actual data.
The data is cached in the page cache.
Josef "Jeff" Sipek.
Signature · 1 line
Bad pun of the week: The formula 1 control computer suffered from a race condition
Quoted 10 lines · --- Josef Sipek
Quoted 2 lines · UZAIR LAKHANI
Hello All,
Thanks for the reply. Can you please tell me whereVFS
Quoted 1 line
caches the data, I have an idea about the dentry
and
Quoted 1 line
inode cache but how VFS caches the actual data.
The data is cached in the page cache.
Hello All,
Can we not by some way invalidate the old page and force the kernel to read the actual page i.e. can we not by any way by-pass the page cache for wrapfs read.
How ext3 e.g. always get the updated data on read. Can anyone give some detail explanations.
Thanks,
Uzair Lakhani,
Karachi, Pakistan.
Quoted 13 lines
Josef "Jeff" Sipek.
Signature · 8 lines
Bad pun of the week: The formula 1 control computer suffered from a race condition - To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
__________________________________________________
Do You Yahoo!?
Tired of spam? Yahoo! Mail has the best spam protection around
http://mail.yahoo.com
Quoted 6 lines · UZAIR LAKHANI
Quoted 1 line · --- Josef Sipek
The data is cached in the page cache.
Can we not by some way invalidate the old page and force the kernel to read the actual page i.e. can we not by any way by-pass the page cache for wrapfs read.
Currently, no. We (the stackable fs) would need to invalidate the page associated with ext3. ext3 won't invalidate our pages, so we would have to do it ourselves. To do that, we'd need some kind of callback so that when an ext3 page changes, we can propagate the change.
Quoted 1 line
How ext3 e.g. always get the updated data on read.
I am not sure what you mean.
Josef "Jeff" Sipek.
Signature · 4 lines
The reasonable man adapts himself to the world; the unreasonable one persists in trying to adapt the world to himself. Therefore all progress depends on the unreasonable man. - George Bernard Shaw
Quoted 11 lines · --- Josef Sipek
Quoted 5 lines · UZAIR LAKHANI
Quoted 1 line · --- Josef Sipek
The data is cached in the page cache.
Can we not by some way invalidate the old page and force the kernel to read the actual page i.e. can
we
Quoted 1 line
not by any way by-pass the page cache for wrapfs
read.
Quoted 7 lines
Currently, no. We (the stackable fs) would need to invalidate the page
associated with ext3. ext3 won't invalidate our pages, so we would have to
do it ourselves. To do that, we'd need some kind of callback so that when an
ext3 page changes, we can propagate the change.
Thanks for the consistent and regular replies. Can you please give some more details about the callback idea. Do we will need to change the kernel code to implement this callback idea but this will require kernel re-compilation.
Can we solve the problem by implementing our own address space operations.
One more question, as far as I know, distributed file systems (e.g NFS) are also like a wrapper fs on server side. How then these distributed file systems handle cache coherency problem.
Thanks,
Uzair Lakhani
Quoted 21 lines
Quoted 2 lines
How ext3 e.g. always get the updated data on read.
I am not sure what you mean.
Josef "Jeff" Sipek.
Signature · 12 lines
The reasonable man adapts himself to the world; the unreasonable one persists in trying to adapt the world to himself. Therefore all progress depends on the unreasonable man. - George Bernard Shaw - To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
__________________________________________________
Do You Yahoo!?
Tired of spam? Yahoo! Mail has the best spam protection around
http://mail.yahoo.com
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
"Josef" == Josef Sipek <[email protected]> writes:
Josef> That is an artifact of how the Linux VFS works. When you modify a Josef> file though wrapfs (or any other stackable file system) the Josef> wrapfs code gets called. It then calls the lower file system code Josef> on your behalf. When you modify the lower file directly, ext3 Josef> gets the called, but since Linux has no way for this change to be Josef> announced up the stack, wrapfs happily uses the old, cached Josef> data. If you have ideas about how to solve this, please share Josef> them - right now, there is a lot of work being done to get Josef> Unionfs (ours) and eCryptfs into the vanilla linux kernel.
While constructed for a vnode system, the ideas in a set of Ficus-related papers and theses could be helpful here. This is John Heidemann's PhD thesis on the topic (covers necessary VM designs, callbacks, etc.):
http://www.isi.edu/people/johnh/PAPERS/Heidemann95e.html
Signature · 5 lines
John Kohl Senior Software Engineer - Rational Software - IBM Software Group Lexington, Massachusetts, USA [email protected] <http://www.ibm.com/software/rational/>
Quoted 12 lines · John T. Kohl
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
"Josef" == Josef Sipek <[email protected]> writes:
Josef> That is an artifact of how the Linux VFS works. When you modify a Josef> file though wrapfs (or any other stackable file system) the Josef> wrapfs code gets called. It then calls the lower file system code Josef> on your behalf. When you modify the lower file directly, ext3 Josef> gets the called, but since Linux has no way for this change to be Josef> announced up the stack, wrapfs happily uses the old, cached Josef> data. If you have ideas about how to solve this, please share Josef> them - right now, there is a lot of work being done to get Josef> Unionfs (ours) and eCryptfs into the vanilla linux kernel.
Quoted 5 lines
While constructed for a vnode system, the ideas in a set of Ficus-related papers and theses could be helpful here. This is John Heidemann's PhD thesis on the topic (covers necessary VM designs, callbacks, etc.):
http://www.isi.edu/people/johnh/PAPERS/Heidemann95e.html
I have read the SOSP '95 paper by the same author a while back (I skimmed thought the paper you linked to, and it looks like chapters 6-8 cover the SOSP paper material) and I'd like to point out that such architecture is not trivial to implement - at least as described by the paper. There are, however, few interesting ideas there that might be worth looking into.
I think that the current thinking is, let's get ecyptfs/unionfs out there, and then let's fix what _needs_ to be fixed, instead of redoing the page cache from the ground up. Sure, we might end up with something similar to what the paper describes, but "Linux is evolution, not intelligent design."
Thanks for the link,
Josef "Jeff" Sipek.
Signature · 1 line
Bus Error: passangers dumped.