Trying to cause a writepage from commit_write?
Hi there,
I'm writing a custom filesystem that does not use a block device as it's backing store. I think it should be possible to arrange for a writepage to be called at some point after a commit_write - for instance by just dirtying the page and waiting for it to be flushed.
I've tried various methods for dirtying the page, but my writepage() method never seems to get called. I can't see anything obvious that I'm missing, but most other filesystems end up calling into the block layer at some point, so they don't correspond very well to what I'm trying to do.
Any suggestions or pointers would be greatly appreciated.
Cheers,
Mark
Signature · 4 lines
Dave: Just a question. What use is a unicyle with no seat? And no pedals! Mark: To answer a question with a question: What use is a skateboard? Dave: Skateboards have wheels. Mark: My wheel has a wheel!
Hi,
Quoted 7 lines · Mark Williamson
Hi there,
I'm writing a custom filesystem that does not use a block device as it's backing store. I think it should be possible to arrange for a writepage to be called at some point after a
commit_write- for instance by just dirtying the page and waiting for it to be flushed.
Quoted 6 lines
I've tried various methods for dirtying the page, but my
writepage()method never seems to get called. I can't see anything obvious that I'm missing, but most other filesystems end up calling into the block layer at some point, so they don't correspond very well to what I'm trying to do.Any suggestions or pointers would be greatly appreciated.
Well, given the information you provided it is anyone's guess, but four things came into my mind at the moment:
1) Check that you have a handler for the commit_write() address space operation.
2) Check that the handler marks the page as dirty by calling set_page_dirty(). Its variants like __set_page_dirty_buffers() and __set_page_dirty_nobuffers() may also be ok but since you are not block device based I don't think you should be using buffer_heads. Specifically SetPageDirty() is NOT enough because it does not mark
the page dirty in the mapping tree (See
documentation/filesystems/Locking for other details).
3) If you have your own set_page_dirty() address space operation, make
sure it does the right thing by eventually calling
__set_page_dirty_nobuffers() (assuming you don't use buffer heads).
4) Check that writepages() does not kick in earlier and does not do the job instead.
HTH
Martin Jambor
Quoted 3 lines
Quoted 1 line
Any suggestions or pointers would be greatly appreciated.
Well, given the information you provided it is anyone's guess,
Yes, sorry about that :-) It's just nice to have a sanity check on the basic principle - to check that I hadn't missed anything glaringly obvious... I do have a strange feeling this isn't as hard as I'm finding it.
Quoted 2 lines
1) Check that you have a handler for the
commit_write()address space operation.
Got that, and it's definitely being called.
Quoted 7 lines
2) Check that the handler marks the page as dirty by calling
set_page_dirty(). Its variants like__set_page_dirty_buffers()and__set_page_dirty_nobuffers()may also be ok but since you are not block device based I don't think you should be usingbuffer_heads. SpecificallySetPageDirty()is NOT enough because it does not mark
the page dirty in the mapping tree (See
documentation/filesystems/Locking for other details).
I've experimented with various versions of set_page_dirty(), and now have settled back on the vanilla version.
I'd figured out that SetPageDirty() wasn't enough - eventually :-) Thanks for noting it though.
Quoted 3 lines
3) If you have your own
set_page_dirty()address space operation, make
sure it does the right thing by eventually calling
__set_page_dirty_nobuffers()(assuming you don't use buffer heads).
I'm not overriding set_page_dirty() anywhere (at the moment)
Quoted 2 lines
4) Check that
writepages()does not kick in earlier and does not do the job instead.
I'm not implementing a writepages() method either, at the moment.
For more concrete discussion, here's a snippet of the code. This is my current implementation of commit_write:
static int xenfs_commit_write(struct file *file, struct page *page, unsigned from, unsigned to)
{
/* Some code taken from buffer.c::generic_commit_write */
struct inode *inode = page->mapping->host;
loff_t pos = ((loff_t)page->index << PAGE_CACHE_SHIFT) + to;
/*
* No need to use i_size_read() here, the i_size
* cannot change under us because we hold i_mutex.
*/
if (pos > inode->i_size) {
i_size_write(inode, pos);
mark_inode_dirty(inode);
}
printk("xenfs_commit_write\n");/* dumps out some data from the buffer so I can check it's correct. */
dumpbuf(page_address(page) + from, to - from);
printk("Setting page dirty!\n");
set_page_dirty(page);
printk("PageLocked = %d, PageUpToDate = %d, PageDirty = %d\n",
PageLocked(page), PageUptodate(page), PageDirty(page));
printk("Inode size = %lu", file->f_mapping->host->i_size);
/* Queue a writepage? */
return 0;
}The first section of the code is just copied from the generic_commit_write() function, comment included.
My printks here tell me:
PageLocked = 1, PageUpToDate = 1, PageDirty = 1
By the end of the function, as expected.
However, as far as I can tell, I don't get a call to my writepage() method at any point after this, even if I run sync (although that does cause the modified inode to get written out).
Am I doing something wrong in my commit_write() - where I've been assuming the problem is - or should I be looking elsewhere?
Any help would be greatly appreciated!
Thanks very much,
Mark
Signature · 4 lines
Dave: Just a question. What use is a unicyle with no seat? And no pedals! Mark: To answer a question with a question: What use is a skateboard? Dave: Skateboards have wheels. Mark: My wheel has a wheel!
Quoted 4 lines · Mark Williamson
Quoted 2 lines
4) Check that
writepages()does not kick in earlier and does not do the job instead.I'm not implementing a
writepages()method either, at the moment.
In that case have you remembered to set 'generic_writepages' as your default writepages callback?
Cheers,
Trond
Hi Trond,
Thanks for your reply!
Quoted 7 lines
Quoted 4 lines
Quoted 2 lines
4) Check that
writepages()does not kick in earlier and does not do the job instead.I'm not implementing a
writepages()method either, at the moment.In that case have you remembered to set '
generic_writepages' as your default writepages callback?
I tried that, and thought it fixed the problem but have been unable to reproduce the correct behaviour.
I've now hooked in a custom writepages() implementation that just printks and returns - it doesn't appear to get called. I know I'm dirtying the page on commit_write, and sync is writing out the inode. But it doesn't look like I'm receiving any callbacks to actually write out the data.
By the looks of page-writeback.c::do_writepages() if there isn't a specified writepages method, it'll fall back to the generic implementation anyhow, so I don't think this can have been my problem.
It looks like do_writepages isn't ever being reached, for some reason.
Maybe I'm missing something obvious...
Cheers,
Mark
Signature · 4 lines
Dave: Just a question. What use is a unicyle with no seat? And no pedals! Mark: To answer a question with a question: What use is a skateboard? Dave: Skateboards have wheels. Mark: My wheel has a wheel!
Quoted 12 lines · Mark Williamson
Hi Trond,
Thanks for your reply!
Quoted 7 lines
Quoted 4 lines
Quoted 2 lines
4) Check that
writepages()does not kick in earlier and does not do the job instead.I'm not implementing a
writepages()method either, at the moment.In that case have you remembered to set '
generic_writepages' as your default writepages callback?
Quoted 6 lines
I tried that, and thought it fixed the problem but have been unable to reproduce the correct behaviour.
I've now hooked in a custom
writepages()implementation that just printks and returns - it doesn't appear to get called. I know I'm dirtying the page oncommit_write, and sync is writing out the inode.
What do you mean by "writing out the inode" ? Does your writepage() gets called or not ?
And also, you mentioned that you don't have a backing block-device. What is the mapping->backing_dev_info set to ?
For example, sync_sb_inodes() skips the inodes and superblock
if (!bdi_cap_writeback_dirty(bdi)) {
...
/*
* Dirty memory-backed inode against a filesystem other
* than the kernel-internal bdev filesystem. Skip the
* entire superblock.
*/
break;
}Thanks,
Badari