Introduction: VFS documentation and tidy up
Subsystems (5)
PATCH 1/1 Introduction: VFS documentation and tidy up
The following 4 patches improve the documentation for vfs address_space operations, and clean up some code oddities I found while writing it.
All have been sent the these lists before for review and only one comment was received: a positive note about the documentation.
Please target them for 2.6.17.
Thanks,
NeilBrown
[PATCH 001 of 4] Update some VFS documentation.
[PATCH 002 of 4] Honour AOP_TRUNCATE_PAGE returns in page_symlink [PATCH 003 of 4] Make address_space_operations->sync_page return void. [PATCH 004 of 4] Make address_space_operations->invalidatepage return void
PATCH 1/1 Update some VFS documentation.
Flesh out the description of the address_space operations.
### Diffstat output
./Documentation/filesystems/vfs.txt | 216 ++++++++++++++++++++++++++++++++---- 1 file changed, 194 insertions(+), 22 deletions(-)
Documentation/filesystems/vfs.txt renamed from Documentation/filesystems/vfs.txt~current~ +194 -22
@@ -230,10 +230,14 @@ only called from a process context (i.e. or bottom half). alloc_inode: this method is called by inode_alloc() to allocate memory - for struct inode and initialize it. + for struct inode and initialize it. If this function is not + defined, a simple 'struct inode' is allocated. Normally + alloc_inode will be used to allocate a larger structure which + contains a 'struct inode' embedded within it. destroy_inode: this method is called by destroy_inode() to release - resources allocated for struct inode. + resources allocated for struct inode. It is only required of + ->alloc_inode was defined and simply does a deallocate. read_inode: this method is called to read a specific inode from the mounted filesystem. The i_ino member in the struct inode is
@@ -443,14 +447,81 @@ otherwise noted. The Address Space Object ======================== -The address space object is used to identify pages in the page cache. +The address space object is used to group and manage pages in the page +cache. It can be used to keep track of the pages in a file (or +anything else) and also track the mapping of sections of the file into +process address spaces. + +There are a number of distinct yet related services that an +address-space can provide. These include communicating memory +pressure, page lookup by address, and keeping track of pages tagged as +Dirty or Writeback. + +The first can be used independantly to the others. The vm can try to +either write dirty pages in order to clean them, or release clean +pages in order to reuse them. To do this it can call the ->writepage +method on dirty pages, and ->releasepage on clean pages with +PagePrivate set. Clean pages without PagePrivate and with no external +references will be released without notice being given to the +address_space. + +To achieve this functionality, pages need to be placed on an lru with +lru_cache_add and mark_page_active needs to be called whenever the +page is used. + +Pages are normally kept in a radix tree index by ->index. This tree +maintains information about the PG_Dirty and PG_Writeback status of +each page, so that pages with either of these flags can be found +quickly. + +The Dirty tag is primarily used by mpage_writepages - the default +->writepages method. It uses the tag to find dirty pages to call +->writepage on. If mpage_writepages is not used (i.e. the address +provides it's own ->writepages) , the PAGECACHE_TAG_DIRTY tag is +almost unused. write_inode_now and sync_inode do use it (through +__sync_single_inode) to check if ->writepages has been successful in +writing out the whole address_space. + +The Writeback tag is used by filemap*wait* and sync_page* functions, +though wait_on_page_writeback_range, to wait for all writeback to +complete. While waiting ->sync_page (if defined) will be called on +each page that is found to require writeback + +An address_space handler may attach extra information to a page, +typically using the 'private' field in the 'struct page'. If such +information is attached, the PG_Private flag should be set. This will +cause various mm routines to make extra calls into the address_space +handler to deal with that data. + +An address space acts as an intermediate between storage and +application. Data is read into the address space a whole page at a +time, and provided to the application either by copying of the page, +or by memory-mapping the page. +Data is written into the address space by the application, and then +written-back to storage typically in whole pages, however the +address_space has finner control of write sizes. + +The read process essentially only requires 'readpage'. The write +process is more complicated and uses prepare_write/commit_write or +set_page_dirty to write data into the address_space, and writepage, +sync_page, and writepages to writeback data to storage. + +Adding and removing pages to/from an address_space is protected by the +inode's i_mutex. + +When data is written to a page, the PG_Dirty flag should be set. It +typically remains set until writepage asks for it to be written. This +should clear PG_Dirty and set PG_Writeback. It can be actually +written at any point after PG_Dirty is clear. Once it is known to be +safe, PG_Writeback is cleared. +Writeback makes use of a writeback_control structure... struct address_space_operations ------------------------------- This describes how the VFS can manipulate mapping of a file to page cache in -your filesystem. As of kernel 2.6.13, the following members are defined: +your filesystem. As of kernel 2.6.16, the following members are defined: struct address_space_operations { int (*writepage)(struct page *page, struct writeback_control *wbc);
@@ -469,47 +540,148 @@ struct address_space_operations { loff_t offset, unsigned long nr_segs); struct page* (*get_xip_page)(struct address_space *, sector_t, int); + /* migrate the contents of a page to the specified target */ + int (*migratepage) (struct page *, struct page *); }; - writepage: called by the VM write a dirty page to backing store. + writepage: called by the VM to write a dirty page to backing store. + This may happen for data integrity reason (i.e. 'sync'), or + to free up memory (flush). The difference can be seen in + wbc->sync_mode. + The PG_Dirty flag has been cleared and PageLocked is true. + writepage should start writeout, should set PG_Writeback, + and should make sure the page is Unlocked, either synchronously + or asynchronously when the write operation completes. + + If wbc->sync_mode is WB_SYNC_NONE, ->writepage doesn't have to + try too hard if there are problems, and may choose to write out a + different page from the mapping if that would be more + appropriate. If it chooses not to start writeout, it should + return AOP_WRITEPAGE_ACTIVATE so that the VM will not keep + calling ->writepage on that page. + + See the file "Locking" for more details. readpage: called by the VM to read a page from backing store. + The page will be Locked when readpage is called, and should be + unlocked and marked uptodate once the read completes. + If ->readpage discovers that it needs to unlock the page for + some reason, it can do so, and then return AOP_TRUNCATED_PAGE. + In this case, the page will be re-located, re-locked and if + that all succeeds, ->readpage will be called again. sync_page: called by the VM to notify the backing store to perform all queued I/O operations for a page. I/O operations for other pages - associated with this address_space object may also be performed. + associated with this address_space object may also be + performed. + This function is optional and is called only for pages with + PG_Writeback set while waiting for the writeback to complete. writepages: called by the VM to write out pages associated with the - address_space object. + address_space object. If WBC_SYNC_ALL, then the + writeback_control will specify a range of pages that must be + written out. If WBC_SYNC_NONE, then a nr_to_write is given + and that many pages should be written if possible. + If no ->writepages is given, then mpage_writepages is used + instead. This will choose pages from the addresspace that are + tagged as DIRTY and will pass them to ->writepage. set_page_dirty: called by the VM to set a page dirty. + This is particularly needed if an address space attaches + private data to a page, and that data needs to be updated when + a page is dirtied. This is called, for example, when a memory + mapped page gets modified. + If defined, it should set the PageDirty flag, and the + PAGECACHE_TAG_DIRTY tag in the radix tree. readpages: called by the VM to read pages associated with the address_space - object. + object. This is essentailly just a vector version of + readpage. Instead of just one page, several pages are + requested. + readpages is only used for readahead, so read errors are + ignored. If anything goes wrong, feel free to give up. prepare_write: called by the generic write path in VM to set up a write - request for a page. - - commit_write: called by the generic write path in VM to write page to - its backing store. + request for a page. This indicates to the address space that + the given range of bytes are about to be written. The + address_space should check that the write will be able to + complete, by allocating space if necessary and doing any other + internal house keeping. If the write will update parts some + some basic-blocks on storage, then those blocks should be + pre-read (if they haven't been read already) so that the + update will not leave half-blocks that need to be written out. + The page will be locked. If prepare_write wants to unlock the + page it, like readpage, may do so and return + AOP_TRUNCATED_PAGE. + In this case the prepare_write will be retried one the lock is + regained. + + commit_write: If prepare_write succeeds, new data will be copied + into the page and then commit_write will be called. It will + typically update the size of the file (if appropriate) and + mark the inode as dirty, and do any other related housekeeping + operations. It should avoid returning an error if possible - + errors should have been handled by prepare_write. bmap: called by the VFS to map a logical block offset within object to - physical block number. This method is use by for the legacy FIBMAP - ioctl. Other uses are discouraged. - - invalidatepage: called by the VM on truncate to disassociate a page from its - address_space mapping. - - releasepage: called by the VFS to release filesystem specific metadata from - a page. - - direct_IO: called by the VM for direct I/O writes and reads. + physical block number. This method is used by for the FIBMAP + ioctl and for working with swap-files. To be able to swap to + a file, the file must have as stable mapping to a block + device. The swap system does not go through the filesystem + but instead uses bmap to find out where the blocks in the file + are and uses those addresses directly. + + + invalidatepage: If a page has PagePrivate set, then invalidatepage + will be called when part or all of the page is to be removed + from the address space. This generally corresponds either a + truncation or a complete invalidation of the address space + (in the latter case 'offset' will always be 0). + Any private data associated with the page should be updated + to reflect this truncation. If offset is 0, then + the private data should be released, because the page + must be able to be completely discarded. This may be done by + calling the ->releasepage function, but in this case the + release MUST succeed. + + releasepage: releasepage is called on PagePrivate pages to indicate + that the page should be freed if possible. ->releasepage + should remove any private data from the page and clear the + PagePrivate flag. It may also remove the page from the + address_space. If this fails for some reason, it may indicate + failure with a 0 return value. + This is used in two distinct though related cases. The first + is when the VM finds a clean page with no active users and + wants to make it a free page. If ->releasepage succeeds, the + page will be removed from the address_space and become free. + + The second case if when a request has been made to invalidate + some or all pages in an address_space. This can happen + through the fadvice(POSIX_FADV_DONTNEED) system call or by the + filesystem explicitly requesting it as nfs and 9fs do (when + they believe the cache may be out of date with storage) by + calling invalidate_inode_pages2(). + If the filesystem makes such a call, and needs to be certain + that all pages are invalidated, then it's releasepage will + need to ensure this. Possibly it can clear the PageUptodate + bit if it cannot free private data yet. + + direct_IO: called by the generic read/write routines to perform + direct_IO - that is IO requests which bypass the page cache + and tranfer data directly between the storage and the + application's address space. get_xip_page: called by the VM to translate a block number to a page. The page is valid until the corresponding filesystem is unmounted. Filesystems that want to use execute-in-place (XIP) need to implement it. An example implementation can be found in fs/ext2/xip.c. + migrate_page: This is used to compact the physical memory usage. + If the VM wants to relocate a page (maybe off a memory card + that is signalling imminent failure) it will pass a new page + and an old page to this function. migrate_page should + transfer any private data across and update any references + that it has to the page. The File Object ===============
Re: [PATCH 001 of 4] Update some VFS documentation.
Did a quick scan - some minor corrections in text.
On Mon, 2006-03-13 at 10:53 +1100, NeilBrown wrote:
<snip>
Documentation/filesystems/vfs.txt only called from a process context (i.e. · line 239
Quoted 3 lines
destroy_inode: this method is called bydestroy_inode()to releaseDocumentation/filesystems/vfs.txt renamed from Documentation/filesystems/vfs.txt~current~ +1
-1- resources allocated for struct inode. + resources allocated for struct inode. It is only required of
"of" -> "if"
Quoted 1 line
+ ->alloc_inode was defined and simply does a deallocate.
"simply does a deallocate" -> "undoes anything done by ->alloc_inode"
<snip>
Quoted 1 line
+The first can be used independantly to the others. The vm can try to
"independantly" -> "independently"
<snip>
Documentation/filesystems/vfs.txt struct address_space_operations { · line 553
Quoted 8 lines
Documentation/filesystems/vfs.txt renamed from Documentation/filesystems/vfs.txt~current~ +7
-1- writepage: called by the VM write a dirty page to backing store. + writepage: called by the VM to write a dirty page to backing store. + This may happen for data integrity reason (i.e. 'sync'), or + to free up memory (flush). The difference can be seen in + wbc->sync_mode. + The PG_Dirty flag has been cleared and PageLocked is true. + writepage should start writeout, should set PG_Writeback, + and should make sure the page is Unlocked, either synchronously
"Unlocked" -> "unlocked"
<snip>
Documentation/filesystems/vfs.txt struct address_space_operations { · line 576
Quoted 6 lines
sync_page: called by the VM to notify the backing store to perform all
queued I/O operations for a page. I/O operations for other pagesDocumentation/filesystems/vfs.txt renamed from Documentation/filesystems/vfs.txt~current~ +2
-1- associated with this address_space object may also be performed. + associated with this address_space object may also be + performed.
Why did this line get split?
<snip>
Documentation/filesystems/vfs.txt struct address_space_operations { · line 595
Quoted 7 lines
set_page_dirty: called by the VM to set a page dirty.Documentation/filesystems/vfs.txt renamed from Documentation/filesystems/vfs.txt~current~ +6
-0+ This is particularly needed if an address space attaches + private data to a page, and that data needs to be updated when + a page is dirtied. This is called, for example, when a memory + mapped page gets modified. + If defined, it should set the PageDirty flag, and the + PAGECACHE_TAG_DIRTY tag in the radix tree.
Indentation is off by one.
Documentation/filesystems/vfs.txt struct address_space_operations { · line 598
Quoted 3 lines
readpages: called by the VM to read pages associated with the
address_spaceDocumentation/filesystems/vfs.txt renamed from Documentation/filesystems/vfs.txt~current~ +1
-1- object. + object. This is essentailly just a vector version of
"essentailly" -> "essentially"
Quoted 8 lines
Documentation/filesystems/vfs.txt renamed from Documentation/filesystems/vfs.txt~current~ +4
-2+ readpage. Instead of just one page, several pages are + requested. + readpages is only used for readahead, so read errors are + ignored. If anything goes wrong, feel free to give up. prepare_write: called by the generic write path in VM to set up a write - request for a page. -
Documentation/filesystems/vfs.txt struct address_space_operations { · line 612
Quoted 10 lines
Documentation/filesystems/vfs.txt renamed from Documentation/filesystems/vfs.txt~current~ +8
-2- commit_write: called by the generic write path in VM to write page to - its backing store. + request for a page. This indicates to the address space that + the given range of bytes are about to be written. The + address_space should check that the write will be able to + complete, by allocating space if necessary and doing any other + internal house keeping. If the write will update parts some + some basic-blocks on storage, then those blocks should be + pre-read (if they haven't been read already) so that the + update will not leave half-blocks that need to be written out.
"some" appears twice in a row in this last sentence. Anyway, it is confusing. I would re-word it to something like:
"If the write will update partial storage blocks, those blocks should be read at this point (if they haven't been already) so that the updated blocks can be properly written out."
<snip>
Avishay Traeger
http://www.fsl.cs.sunysb.edu/~avishay/
Re: [PATCH 001 of 4] Update some VFS documentation fix
Quoted 1 line · On Sunday March 12, [email protected] wrote:
Did a quick scan - some minor corrections in text.
Thanks. The following patch incorporated most of them.
I didn't fix the indenting oddity that you noticed. The file is wildly inconsistent in its uses of tabs or spaces for indenting. That should be fixed up in a separate patch.
Thanks.
-----------------
Corrections to recent updates for vfs.txt
Thanks Avishay Traeger <[email protected]>
### Diffstat output
./Documentation/filesystems/vfs.txt | 19 ++++++++++--------- 1 file changed, 10 insertions(+), 9 deletions(-)
Documentation/filesystems/vfs.txt renamed from Documentation/filesystems/vfs.txt~current~ +10 -9
@@ -236,8 +236,9 @@ or bottom half). contains a 'struct inode' embedded within it. destroy_inode: this method is called by destroy_inode() to release - resources allocated for struct inode. It is only required of - ->alloc_inode was defined and simply does a deallocate. + resources allocated for struct inode. It is only required if + ->alloc_inode was defined and simply undoes anything done by + ->alloc_inode. read_inode: this method is called to read a specific inode from the mounted filesystem. The i_ino member in the struct inode is
@@ -550,7 +551,7 @@ struct address_space_operations { wbc->sync_mode. The PG_Dirty flag has been cleared and PageLocked is true. writepage should start writeout, should set PG_Writeback, - and should make sure the page is Unlocked, either synchronously + and should make sure the page is unlocked, either synchronously or asynchronously when the write operation completes. If wbc->sync_mode is WB_SYNC_NONE, ->writepage doesn't have to
@@ -572,8 +573,8 @@ struct address_space_operations { sync_page: called by the VM to notify the backing store to perform all queued I/O operations for a page. I/O operations for other pages - associated with this address_space object may also be - performed. + associated with this address_space object may also be performed. + This function is optional and is called only for pages with PG_Writeback set while waiting for the writeback to complete.
@@ -595,7 +596,7 @@ struct address_space_operations { PAGECACHE_TAG_DIRTY tag in the radix tree. readpages: called by the VM to read pages associated with the address_space - object. This is essentailly just a vector version of + object. This is essentially just a vector version of readpage. Instead of just one page, several pages are requested. readpages is only used for readahead, so read errors are
@@ -606,10 +607,10 @@ struct address_space_operations { the given range of bytes are about to be written. The address_space should check that the write will be able to complete, by allocating space if necessary and doing any other - internal house keeping. If the write will update parts some - some basic-blocks on storage, then those blocks should be + internal house keeping. If the write will update parts of + any basic-blocks on storage, then those blocks should be pre-read (if they haven't been read already) so that the - update will not leave half-blocks that need to be written out. + updated blocks can be written out properly. The page will be locked. If prepare_write wants to unlock the page it, like readpage, may do so and return AOP_TRUNCATED_PAGE.
Re: [PATCH 001 of 4] Update some VFS documentation.
In reply to NeilBrowna few more; you are too fast.
and thanks for doing this.
Documentation/filesystems/vfs.txt otherwise noted. · line 455
Quoted 10 lines · On Mon, 13 Mar 2006 10:53:16 +1100 NeilBrown wrote:
Documentation/filesystems/vfs.txt renamed from Documentation/filesystems/vfs.txt~current~ +6
-0@@ -443,14 +447,81 @@ otherwise noted. The Address Space Object ======================== +There are a number of distinct yet related services that an +address-space can provide. These include communicating memory +pressure, page lookup by address, and keeping track of pages tagged as +Dirty or Writeback. + +The first can be used independantly to the others. The vm can try to
s/vm/VM/
Documentation/filesystems/vfs.txt otherwise noted. · line 468
Quoted 8 lines
Documentation/filesystems/vfs.txt renamed from Documentation/filesystems/vfs.txt~current~ +8
-0+either write dirty pages in order to clean them, or release clean +pages in order to reuse them. To do this it can call the ->writepage +method on dirty pages, and ->releasepage on clean pages with +PagePrivate set. Clean pages without PagePrivate and with no external +references will be released without notice being given to the +address_space. + +To achieve this functionality, pages need to be placed on an lru with
prefer s/lru/LRU/
Documentation/filesystems/vfs.txt otherwise noted. · line 480
Quoted 12 lines
Documentation/filesystems/vfs.txt renamed from Documentation/filesystems/vfs.txt~current~ +12
-0+lru_cache_add and mark_page_active needs to be called whenever the +page is used. + +Pages are normally kept in a radix tree index by ->index. This tree +maintains information about the PG_Dirty and PG_Writeback status of +each page, so that pages with either of these flags can be found +quickly. + +The Dirty tag is primarily used by mpage_writepages - the default +->writepages method. It uses the tag to find dirty pages to call +->writepage on. If mpage_writepages is not used (i.e. the address +provides it's own ->writepages) , the PAGECACHE_TAG_DIRTY tag is
its
Documentation/filesystems/vfs.txt otherwise noted. · line 486
Quoted 6 lines
Documentation/filesystems/vfs.txt renamed from Documentation/filesystems/vfs.txt~current~ +6
-0+almost unused. write_inode_now and sync_inode do use it (through +__sync_single_inode) to check if ->writepages has been successful in +writing out the whole address_space. + +The Writeback tag is used by filemap*wait* and sync_page* functions, +though wait_on_page_writeback_range, to wait for all writeback to
s/though/through/ ??
Documentation/filesystems/vfs.txt otherwise noted. · line 488
Quoted 2 lines
Documentation/filesystems/vfs.txt renamed from Documentation/filesystems/vfs.txt~current~ +2
-0+complete. While waiting ->sync_page (if defined) will be called on +each page that is found to require writeback
full stop.
Documentation/filesystems/vfs.txt otherwise noted. · line 493
Quoted 4 lines
Documentation/filesystems/vfs.txt renamed from Documentation/filesystems/vfs.txt~current~ +4
-0+An address_space handler may attach extra information to a page, +typically using the 'private' field in the 'struct page'. If such +information is attached, the PG_Private flag should be set. This will +cause various mm routines to make extra calls into the address_space
prefer s/mm/MM/ (or use VM as above)
Documentation/filesystems/vfs.txt otherwise noted. · line 502
Quoted 9 lines
Documentation/filesystems/vfs.txt renamed from Documentation/filesystems/vfs.txt~current~ +9
-0+handler to deal with that data. + +An address space acts as an intermediate between storage and +application. Data is read into the address space a whole page at a +time, and provided to the application either by copying of the page, +or by memory-mapping the page. +Data is written into the address space by the application, and then +written-back to storage typically in whole pages, however the +address_space has finner control of write sizes.
finer
Documentation/filesystems/vfs.txt otherwise noted.
Quoted 9 lines
Documentation/filesystems/vfs.txt renamed from Documentation/filesystems/vfs.txt~current~ +7
-1+ +The read process essentially only requires 'readpage'. The write +process is more complicated and uses prepare_write/commit_write or +set_page_dirty to write data into the address_space, and writepage, +sync_page, and writepages to writeback data to storage. - writepage: called by the VM write a dirty page to backing store. + writepage: called by the VM to write a dirty page to backing store. + This may happen for data integrity reason (i.e. 'sync'), or
reasons
Documentation/filesystems/vfs.txt struct address_space_operations { · line 559
Quoted 11 lines
Documentation/filesystems/vfs.txt renamed from Documentation/filesystems/vfs.txt~current~ +11
-0+ to free up memory (flush). The difference can be seen in + wbc->sync_mode. + The PG_Dirty flag has been cleared and PageLocked is true. + writepage should start writeout, should set PG_Writeback, + and should make sure the page is Unlocked, either synchronously + or asynchronously when the write operation completes. + + If wbc->sync_mode is WB_SYNC_NONE, ->writepage doesn't have to + try too hard if there are problems, and may choose to write out a + different page from the mapping if that would be more + appropriate. If it chooses not to start writeout, it should
any definition of appropriate?
Documentation/filesystems/vfs.txt struct address_space_operations { · line 570
Quoted 11 lines
Documentation/filesystems/vfs.txt renamed from Documentation/filesystems/vfs.txt~current~ +9
-0+ return AOP_WRITEPAGE_ACTIVATE so that the VM will not keep + calling ->writepage on that page. + + See the file "Locking" for more details. readpage: called by the VM to read a page from backing store. + The page will be Locked when readpage is called, and should be + unlocked and marked uptodate once the read completes. + If ->readpage discovers that it needs to unlock the page for + some reason, it can do so, and then return AOP_TRUNCATED_PAGE. + In this case, the page will be re-located, re-locked and if
drop the hyphens
Documentation/filesystems/vfs.txt struct address_space_operations { · line 581
Quoted 13 lines
Documentation/filesystems/vfs.txt renamed from Documentation/filesystems/vfs.txt~current~ +6
-2+ that all succeeds, ->readpage will be called again. sync_page: called by the VM to notify the backing store to perform all queued I/O operations for a page. I/O operations for other pages - associated with this address_space object may also be performed. + associated with this address_space object may also be + performed. + This function is optional and is called only for pages with + PG_Writeback set while waiting for the writeback to complete. writepages: called by the VM to write out pages associated with the - address_space object. + address_space object. If WBC_SYNC_ALL, then the
If WBC_SYNC_ALL <what> ?
Documentation/filesystems/vfs.txt struct address_space_operations { · line 583
Quoted 2 lines
Documentation/filesystems/vfs.txt renamed from Documentation/filesystems/vfs.txt~current~ +2
-0+ writeback_control will specify a range of pages that must be + written out. If WBC_SYNC_NONE, then a nr_to_write is given
If WBC_SYNC_NONE <what> ?
Documentation/filesystems/vfs.txt struct address_space_operations { · line 606
Quoted 9 lines
Documentation/filesystems/vfs.txt renamed from Documentation/filesystems/vfs.txt~current~ +6
-2+ and that many pages should be written if possible. + If no ->writepages is given, then mpage_writepages is used + instead. This will choose pages from the addresspace that are + tagged as DIRTY and will pass them to ->writepage. - commit_write: called by the generic write path in VM to write page to - its backing store. + request for a page. This indicates to the address space that + the given range of bytes are about to be written. The
is about to be written.
Documentation/filesystems/vfs.txt struct address_space_operations { · line 609
Quoted 3 lines
Documentation/filesystems/vfs.txt renamed from Documentation/filesystems/vfs.txt~current~ +3
-0+ address_space should check that the write will be able to + complete, by allocating space if necessary and doing any other + internal house keeping. If the write will update parts some
housekeeping.
Quoted 9 lines
Documentation/filesystems/vfs.txt renamed from Documentation/filesystems/vfs.txt~current~ +9
-0+ some basic-blocks on storage, then those blocks should be + pre-read (if they haven't been read already) so that the + update will not leave half-blocks that need to be written out. + The page will be locked. If prepare_write wants to unlock the + page it, like readpage, may do so and return + AOP_TRUNCATED_PAGE. + In this case the prepare_write will be retried one the lock is + regained. +
Documentation/filesystems/vfs.txt struct address_space_operations {
Quoted 10 lines
Documentation/filesystems/vfs.txt renamed from Documentation/filesystems/vfs.txt~current~ +7
-1+ commit_write: If prepare_write succeeds, new data will be copied + into the page and then commit_write will be called. It will + typically update the size of the file (if appropriate) and + mark the inode as dirty, and do any other related housekeeping + operations. It should avoid returning an error if possible - + errors should have been handled by prepare_write. - direct_IO: called by the VM for direct I/O writes and reads. + physical block number. This method is used by for the FIBMAP
by for ??
Documentation/filesystems/vfs.txt struct address_space_operations { · line 629
Quoted 2 lines
Documentation/filesystems/vfs.txt renamed from Documentation/filesystems/vfs.txt~current~ +2
-0+ ioctl and for working with swap-files. To be able to swap to + a file, the file must have as stable mapping to a block
must have a
Documentation/filesystems/vfs.txt struct address_space_operations { · line 637
Quoted 8 lines
Documentation/filesystems/vfs.txt renamed from Documentation/filesystems/vfs.txt~current~ +8
-0+ device. The swap system does not go through the filesystem + but instead uses bmap to find out where the blocks in the file + are and uses those addresses directly. + + + invalidatepage: If a page has PagePrivate set, then invalidatepage + will be called when part or all of the page is to be removed + from the address space. This generally corresponds either a
corresponds to
Quoted 18 lines
Documentation/filesystems/vfs.txt renamed from Documentation/filesystems/vfs.txt~current~ +18
-0+ truncation or a complete invalidation of the address space + (in the latter case 'offset' will always be 0). + Any private data associated with the page should be updated + to reflect this truncation. If offset is 0, then + the private data should be released, because the page + must be able to be completely discarded. This may be done by + calling the ->releasepage function, but in this case the + release MUST succeed. + + releasepage: releasepage is called on PagePrivate pages to indicate + that the page should be freed if possible. ->releasepage + should remove any private data from the page and clear the + PagePrivate flag. It may also remove the page from the + address_space. If this fails for some reason, it may indicate + failure with a 0 return value. + This is used in two distinct though related cases. The first + is when the VM finds a clean page with no active users and + wants to make it a free page. If ->releasepage succeeds, the
Documentation/filesystems/vfs.txt struct address_space_operations { · line 665
Quoted 10 lines
Documentation/filesystems/vfs.txt renamed from Documentation/filesystems/vfs.txt~current~ +10
-0+ page will be removed from the address_space and become free. + + The second case if when a request has been made to invalidate + some or all pages in an address_space. This can happen + through the fadvice(POSIX_FADV_DONTNEED) system call or by the + filesystem explicitly requesting it as nfs and 9fs do (when + they believe the cache may be out of date with storage) by + calling invalidate_inode_pages2(). + If the filesystem makes such a call, and needs to be certain + that all pages are invalidated, then it's releasepage will
its
Documentation/filesystems/vfs.txt struct address_space_operations { · line 671
Quoted 6 lines
Documentation/filesystems/vfs.txt renamed from Documentation/filesystems/vfs.txt~current~ +6
-0+ need to ensure this. Possibly it can clear the PageUptodate + bit if it cannot free private data yet. + + direct_IO: called by the generic read/write routines to perform + direct_IO - that is IO requests which bypass the page cache + and tranfer data directly between the storage and the
transfer
Quoted 2 lines
+ application's address space.
HTH.
---
~Randy
You can't do anything without having to do something else first. -- Belefant's Law
PATCH 1/1 Honour AOP_TRUNCATE_PAGE returns in page_symlink
In reply to NeilBrownAs prepare_write, commit_write and readpage are allowed to return AOP_TRUNCATE_PAGE, page_symlink should respond to them.
### Diffstat output
./fs/namei.c | 19 ++++++++++++++++--- 1 file changed, 16 insertions(+), 3 deletions(-)
fs/namei.c renamed from fs/namei.c~current~ +16 -3
@@ -2628,19 +2628,31 @@ void page_put_link(struct dentry *dentryint page_symlink (struct inode *inode,const char *symname,int len) {struct address_space *mapping = inode->i_mapping; -struct page *page =grab_cache_page (mapping,0 ); +struct page *page;int err = -ENOMEM ;char *kaddr; + retry: + page =grab_cache_page (mapping,0 );if (!page) goto fail; err = mapping->a_ops->prepare_write (NULL , page,0 , len-1 ); +if (err ==AOP_TRUNCATED_PAGE ) { +page_cache_release (page); + goto retry; + }if (err) goto fail_map; kaddr =kmap_atomic (page,KM_USER0 );memcpy (kaddr, symname, len-1 );kunmap_atomic (kaddr,KM_USER0 ); - mapping->a_ops->commit_write (NULL , page,0 , len-1 ); + err = mapping->a_ops->commit_write (NULL , page,0 , len-1 ); +if (err ==AOP_TRUNCATED_PAGE ) { +page_cache_release (page); + goto retry; + } +if (err) + goto fail_map;/* * Notice that we are _not_ going to block here - end of page is * unmapped, so this will only try to map the rest of page, see
@@ -2650,7 +2662,8 @@ int page_symlink(struct inode *inode, co*/ if (!PageUptodate (page)) { err = mapping->a_ops->readpage (NULL , page); -wait_on_page_locked (page); +if (err !=AOP_TRUNCATED_PAGE ) +wait_on_page_locked (page); }else {unlock_page (page); }
PATCH 1/1 Make address_space_operations->sync_page return void.
In reply to NeilBrownThe only user ignores the return value, and the only instanace (block_sync_page) always returns 0...
### Diffstat output
./fs/buffer.c | 3 +-- ./fs/cifs/file.c | 6 ++++-- ./include/linux/buffer_head.h | 2 +- ./include/linux/fs.h | 2 +- 4 files changed, 7 insertions(+), 6 deletions(-)
fs/buffer.c renamed from fs/buffer.c~current~ +1 -2
@@ -3024,7 +3024,7 @@ out: }EXPORT_SYMBOL (try_to_free_buffers); -int block_sync_page (struct page *page) +void block_sync_page (struct page *page) {struct address_space *mapping;
@@ -3032,7 +3032,6 @@ int block_sync_page(struct page *page) mapping =page_mapping (page);if (mapping)blk_run_backing_dev (mapping->backing_dev_info, page); -return 0 ; }/*
fs/cifs/file.c renamed from fs/cifs/file.c~current~ +4 -2
@@ -1377,7 +1377,7 @@ int cifs_fsync(struct file *file, structreturn rc; } -/* static int cifs_sync_page(struct page *page) +/* static void cifs_sync_page(struct page *page) { struct address_space *mapping; struct inode *inode;
@@ -1391,16 +1391,18 @@ int cifs_fsync(struct file *file, structreturn 0; inode = mapping->host; if (!inode) -return 0; */ +return; */ /* fill in rpages then result = cifs_pagein_inode(inode, index, rpages); */ /* BB finish */ /* cFYI(1, ("rpages is %d for sync page of Index %ld ", rpages, index)); +#if 0 if (rc < 0) return rc; return 0; +#endif } */ /*
include/linux/buffer_head.h renamed from include/linux/buffer_head.h~current~ +1 -1
@@ -203,7 +203,7 @@ int cont_prepare_write(struct page*, unsint generic_cont_expand (struct inode *inode,loff_t size);int generic_cont_expand_simple (struct inode *inode,loff_t size);int block_commit_write (struct page *page,unsigned from,unsigned to); -int block_sync_page (struct page *); +void block_sync_page (struct page *);sector_t generic_block_bmap (struct address_space *,sector_t ,get_block_t *);int generic_commit_write (struct file *,struct page *,unsigned ,unsigned );int block_truncate_page (struct address_space *,loff_t ,get_block_t *);
include/linux/fs.h renamed from include/linux/fs.h~current~ +1 -1
@@ -345,7 +345,7 @@ struct writeback_control;struct address_space_operations {int (*writepage)(struct page *page,struct writeback_control *wbc);int (*readpage)(struct file *,struct page *); -int (*sync_page)(struct page *); +void (*sync_page)(struct page *);/* Write back some dirty pages from this mapping. */ int (*writepages)(struct address_space *,struct writeback_control *);
PATCH 1/1 Make address_space_operations->invalidatepage return void
In reply to NeilBrownThe return value of this function is never used, so let's be honest and declare it as void.
Some places where invalidatepage returned 0, I have inserted comments suggesting a BUG_ON.
### Diffstat output
./fs/afs/file.c | 6 +++--- ./fs/buffer.c | 22 +++++++++++----------- ./fs/ext3/inode.c | 4 ++-- ./fs/jbd/transaction.c | 12 +++++------- ./fs/jfs/jfs_metapage.c | 7 +++---- ./fs/reiser4/as_ops.c | 13 ++++++------- ./fs/reiser4/vfs_ops.h | 2 +- ./fs/reiserfs/inode.c | 8 +++++--- ./fs/xfs/linux-2.6/xfs_aops.c | 4 ++-- ./include/linux/buffer_head.h | 4 ++-- ./include/linux/fs.h | 2 +- ./include/linux/jbd.h | 2 +- 12 files changed, 42 insertions(+), 44 deletions(-)
fs/afs/file.c renamed from fs/afs/file.c~current~ +3 -3
@@ -28,7 +28,7 @@ static int afs_file_release(struct inode#endif static int afs_file_readpage (struct file *file,struct page *page); -static int afs_file_invalidatepage (struct page *page,unsigned long offset); +static void afs_file_invalidatepage (struct page *page,unsigned long offset);static int afs_file_releasepage (struct page *page,gfp_t gfp_flags);struct inode_operations afs_file_inode_operations = {
@@ -212,7 +212,7 @@ int afs_cache_get_page_cookie(struct pag/* * invalidate part or all of a page */ -static int afs_file_invalidatepage (struct page *page,unsigned long offset) +static void afs_file_invalidatepage (struct page *page,unsigned long offset) {int ret =1 ;
@@ -238,11 +238,11 @@ static int afs_file_invalidatepage(strucif (!PageWriteback (page)) ret = page->mapping->a_ops->releasepage (page,0 ); +/* possibly should BUG_ON(!ret); - neilb */ } }_leave (" = %d" , ret); -return ret; }/* end afs_file_invalidatepage() */ /*****************************************************************************/
fs/buffer.c renamed from fs/buffer.c~current~ +11 -11
@@ -1603,11 +1603,10 @@ EXPORT_SYMBOL(try_to_release_page);* point. Because the caller is about to free (and possibly reuse) those * blocks on-disk. */ -int block_invalidatepage (struct page *page,unsigned long offset) +void block_invalidatepage (struct page *page,unsigned long offset) {struct buffer_head *head, *bh, *next;unsigned int curr_off =0 ; -int ret =1 ;BUG_ON (!PageLocked (page));if (!page_has_buffers (page))
@@ -1633,20 +1632,21 @@ int block_invalidatepage(struct page *pa* The get_block cached value has been unconditionally invalidated, * so real IO is not possible anymore. */ -if (offset ==0 ) - ret =try_to_release_page (page,0 ); +if (offset ==0 ) { +int ret =try_to_release_page (page,0 ); +BUG_ON (!ret); + } out: -return ret; +return ; }EXPORT_SYMBOL (block_invalidatepage); -int do_invalidatepage (struct page *page,unsigned long offset) +void do_invalidatepage (struct page *page,unsigned long offset) { -int (*invalidatepage)(struct page *,unsigned long ); - invalidatepage = page->mapping->a_ops->invalidatepage; -if (invalidatepage ==NULL ) - invalidatepage = block_invalidatepage; -return (*invalidatepage)(page, offset); +void (*invalidatepage)(struct page *,unsigned long ); + invalidatepage = page->mapping->a_ops->invalidatepage ? : + block_invalidatepage; + (*invalidatepage)(page, offset); }/*
fs/ext3/inode.c renamed from fs/ext3/inode.c~current~ +2 -2
@@ -1583,7 +1583,7 @@ ext3_readpages(struct file *file, structreturn mpage_readpages (mapping, pages, nr_pages, ext3_get_block); } -static int ext3_invalidatepage (struct page *page,unsigned long offset) +static void ext3_invalidatepage (struct page *page,unsigned long offset) {journal_t *journal =EXT3_JOURNAL (page->mapping->host);
@@ -1593,7 +1593,7 @@ static int ext3_invalidatepage(struct paif (offset ==0 )ClearPageChecked (page); -return journal_invalidatepage (journal, page, offset); +journal_invalidatepage (journal, page, offset); }static int ext3_releasepage (struct page *page,gfp_t wait)
fs/jbd/transaction.c renamed from fs/jbd/transaction.c~current~ +5 -7
@@ -1873,16 +1873,15 @@ zap_buffer_unlocked: }/** -* int journal_invalidatepage() +* void journal_invalidatepage() * @journal: journal to use for flush... * @page: page to flush * @offset: length of page to invalidate. * * Reap page buffers containing data after offset in page. * -* Return non-zero if the page's buffers were successfully reaped. */ -int journal_invalidatepage (journal_t *journal, +void journal_invalidatepage (journal_t *journal,struct page *page,unsigned long offset) {
@@ -1893,7 +1892,7 @@ int journal_invalidatepage(journal_t *joif (!PageLocked (page))BUG ();if (!page_has_buffers (page)) -return 1 ; +return ;/* We will potentially be playing with lists other than just the * data lists (especially for journaled data mode), so be
@@ -1916,11 +1915,10 @@ int journal_invalidatepage(journal_t *jo }while (bh != head);if (!offset) { -if (!may_free || !try_to_free_buffers (page)) -return 0 ; +/* Maybe should BUG_ON !may_free - neilb */ +try_to_free_buffers (page);J_ASSERT (!page_has_buffers (page)); } -return 1 ; }/*
fs/jfs/jfs_metapage.c renamed from fs/jfs/jfs_metapage.c~current~ +3 -4
@@ -578,14 +578,13 @@ static int metapage_releasepage(struct preturn 0 ; } -static int metapage_invalidatepage (struct page *page,unsigned long offset) +static void metapage_invalidatepage (struct page *page,unsigned long offset) {BUG_ON (offset); -if (PageWriteback (page)) -return 0 ; +BUG_ON (PageWriteback (page)); -return metapage_releasepage (page,0 ); +metapage_releasepage (page,0 ); }struct address_space_operations jfs_metapage_aops = {
fs/reiser4/as_ops.c renamed from fs/reiser4/as_ops.c~current~ +6 -7
@@ -170,7 +170,7 @@ reiser4_readpages(struct file *file, str* @offset: starting offset for partial invalidation * */ -int reiser4_invalidatepage (struct page *page,unsigned long offset) +void reiser4_invalidatepage (struct page *page,unsigned long offset) {int ret =0 ;reiser4_context *ctx;
@@ -208,11 +208,11 @@ int reiser4_invalidatepage(struct page ** them. Check for this, and do nothing. */ if (get_super_fake (inode->i_sb) == inode) -return 0 ; +return ;if (get_cc_fake (inode->i_sb) == inode) -return 0 ; +return ;if (get_bitmap_fake (inode->i_sb) == inode) -return 0 ; +return ;assert ("vs-1426" ,PagePrivate (page));assert ("vs-1427" , page->mapping ==jnode_get_mapping (jnode_by_page (page)));
@@ -222,7 +222,7 @@ int reiser4_invalidatepage(struct page * ctx =init_context (inode->i_sb);if (IS_ERR (ctx)) -return PTR_ERR (ctx); +return ; node =jprivate (page);spin_lock_jnode (node);
@@ -236,7 +236,7 @@ int reiser4_invalidatepage(struct page *unhash_unformatted_jnode (node);jput (node);reiser4_exit_context (ctx); -return 0 ; +return ; }spin_unlock_jnode (node);
@@ -265,7 +265,6 @@ int reiser4_invalidatepage(struct page * }reiser4_exit_context (ctx); -return ret; }/* help function called from reiser4_releasepage(). It returns true if jnode
fs/reiser4/vfs_ops.h renamed from fs/reiser4/vfs_ops.h~current~ +1 -1
@@ -24,7 +24,7 @@ int reiser4_writepage(struct page *, strint reiser4_set_page_dirty (struct page *);int reiser4_readpages (struct file *,struct address_space *,struct list_head *pages,unsigned nr_pages); -int reiser4_invalidatepage (struct page *,unsigned long offset); +void reiser4_invalidatepage (struct page *,unsigned long offset);int reiser4_releasepage (struct page *,gfp_t );extern int reiser4_update_sd (struct inode *);
fs/reiserfs/inode.c renamed from fs/reiserfs/inode.c~current~ +5 -3
@@ -2792,7 +2792,7 @@ static int invalidatepage_can_drop(struc }/* clm -- taken from fs/buffer.c:block_invalidate_page */ -static int reiserfs_invalidatepage (struct page *page,unsigned long offset) +static void reiserfs_invalidatepage (struct page *page,unsigned long offset) {struct buffer_head *head, *bh, *next;struct inode *inode = page->mapping->host;
@@ -2831,10 +2831,12 @@ static int reiserfs_invalidatepage(struc* The get_block cached value has been unconditionally invalidated, * so real IO is not possible anymore. */ -if (!offset && ret) +if (!offset && ret) { ret =try_to_release_page (page,0 ); +/* maybe should BUG_ON(!ret); - neilb */ + } out: -return ret; +return ; }static int reiserfs_set_page_dirty (struct page *page)
fs/xfs/linux-2.6/xfs_aops.c renamed from fs/xfs/linux-2.6/xfs_aops.c~current~ +2 -2
@@ -1370,14 +1370,14 @@ out_unlock:return error; } -STATIC int +STATIC voidlinvfs_invalidate_page (struct page *page,unsigned long offset) {xfs_page_trace (XFS_INVALIDPAGE_ENTER , page->mapping->host, page, offset); -return block_invalidatepage (page, offset); +block_invalidatepage (page, offset); }/*
include/linux/buffer_head.h renamed from include/linux/buffer_head.h~current~ +2 -2
@@ -192,8 +192,8 @@ extern int buffer_heads_over_limit;* address_spaces. */ int try_to_release_page (struct page * page,gfp_t gfp_mask); -int block_invalidatepage (struct page *page,unsigned long offset); -int do_invalidatepage (struct page *page,unsigned long offset); +void block_invalidatepage (struct page *page,unsigned long offset); +void do_invalidatepage (struct page *page,unsigned long offset);int block_write_full_page (struct page *page,get_block_t *get_block,struct writeback_control *wbc);int block_read_full_page (struct page *,get_block_t *);
include/linux/fs.h renamed from include/linux/fs.h~current~ +1 -1
@@ -364,7 +364,7 @@ struct address_space_operations {int (*commit_write)(struct file *,struct page *,unsigned ,unsigned );/* Unfortunately this kludge is needed for FIBMAP. Don't use it */ sector_t (*bmap)(struct address_space *,sector_t ); -int (*invalidatepage) (struct page *,unsigned long ); +void (*invalidatepage) (struct page *,unsigned long );int (*releasepage) (struct page *,gfp_t );ssize_t (*direct_IO)(int ,struct kiocb *,const struct iovec *iov,loff_t offset,unsigned long nr_segs);
include/linux/jbd.h renamed from include/linux/jbd.h~current~ +1 -1
@@ -895,7 +895,7 @@ extern int journal_dirty_metadata (handextern void journal_release_buffer (handle_t *,struct buffer_head *);extern int journal_forget (handle_t *,struct buffer_head *);extern void journal_sync_buffer (struct buffer_head *); -extern int journal_invalidatepage (journal_t *, +extern void journal_invalidatepage (journal_t *,struct page *,unsigned long );extern int journal_try_to_free_buffers (journal_t *,struct page *,gfp_t );extern int journal_stop (handle_t *);
Re: [PATCH 004 of 4] Make address_space_operations->invalidatepage return void
Quoted 8 lines · NeilBrown
diff
./fs/jfs/jfs_metapage.c~current~./fs/jfs/jfs_metapage.c
---./fs/jfs/jfs_metapage.c~current~2006-03-09 17:29:35.000000000
+1100
+++./fs/jfs/jfs_metapage.c2006-03-13 10:46:55.000000000 +1100fs/jfs/jfs_metapage.c renamed from fs/jfs/jfs_metapage.c~current~ +0
-0@@ -578,14 +578,13 @@ static int metapage_releasepage(struct preturn 0 ; }
fs/jfs/jfs_metapage.c static int metapage_releasepage(struct p · line 587
Quoted 10 lines
fs/jfs/jfs_metapage.c renamed from fs/jfs/jfs_metapage.c~current~ +2
-3-static int metapage_invalidatepage (struct page *page,unsigned long offset) +static void metapage_invalidatepage(struct page *page, unsigned long offset) {BUG_ON (offset); -if (PageWriteback (page)) -return 0 ; +BUG_ON (PageWriteback (page));
I'm a little concerned about adding a BUG_ON for something this function used to allow, but it looks like the BUG_ON is valid. I'm asking myself why did I add the test for PageWriteback in the first place.
Quoted 6 lines
fs/jfs/jfs_metapage.c renamed from fs/jfs/jfs_metapage.c~current~ +1
-1-return metapage_releasepage (page,0 ); +metapage_releasepage (page,0 ); }struct
address_space_operationsjfs_metapage_aops= {
I'll try to stress test jfs with these patches to see if I can trigger the an oops here.
Signature · 2 lines
David Kleikamp IBM Linux Technology Center
Re: [PATCH 004 of 4] Make address_space_operations->invalidatepage return void
Quoted 2 lines · Dave Kleikamp
I'll try to stress test jfs with these patches to see if I can trigger the an oops here.
While stress testing on a jfs volume (dbench), I hit an assert in jbd:
Assertion failure in journal_invalidatepage() at fs/jbd/transaction.c:1920: "!page_has_buffers(page)"
------------[ cut here ]------------ kernel BUG at fs/jbd/transaction.c:1920! invalid opcode: 0000 [#1] PREEMPT last sysfs file: /block/hda/hda11/stat Modules linked in: radeon irda crc_ccitt airo e1000 pcmcia yenta_socket rsrc_nonstatic pcmcia_core ntfs jfs rtc CPU: 0 EIP: 0060:[<c01c2782>] Not tainted VLI EFLAGS: 00010282 (2.6.16-rc5-mm3-nb #1)
EIP is at journal_invalidatepage+0xba/0xcb
eax: 00000069 ebx: db6e67f0 ecx: d5806dc8 edx: c0452aa3
esi: c1106620 edi: db6e67f0 ebp: db6e67f0 esp: d5806dc4
ds: 007b es: 007b ss: 0068
Process evolution (pid: 11320, threadinfo=d5806000 task=d5731570)
Stack: <0>c0452aa3 c041bc55 c0454d69 00000780 c04551ac 00001000 c1106620 00000000
c1106620 00000000 c01b4d10 df30b800 c1106620 00000000 c0141369 c1106620
00000000 00000001 c01414fb dbdca3e8 c1106620 00000000 ffffffff 00000000
Call Trace:
<c01b4d10> ext3_invalidatepage+0x2f/0x33 <c0141369> truncate_complete_page+0x1d/0x42
<c01414fb> truncate_inode_pages_range+0xbe/0x27a <c017977c> inotify_inode_is_dead+0x14/0x6e
<c0413061> __mutex_lock_slowpath+0x25e/0x375 <c04130cd> __mutex_lock_slowpath+0x2ca/0x375
<c017977c> inotify_inode_is_dead+0x14/0x6e <c01b34d2> ext3_delete_inode+0x0/0xc9
<c01416cc> truncate_inode_pages+0x15/0x19 <c01b34e8> ext3_delete_inode+0x16/0xc9
<c01b34d2> ext3_delete_inode+0x0/0xc9 <c016cd2d> generic_delete_inode+0x89/0x10e
<c016a193> dput+0x19e/0x1b7 <c0164a28> do_rename+0x135/0x16c
<c0144d2a> unmap_vmas+0xc7/0x19e <c014452b> free_pgtables+0x5b/0x65
<c024c19b> strncpy_from_user+0x35/0x55 <c016147a> do_getname+0x3e/0x5c
<c0164a9a> sys_renameat+0x3b/0x60 <c0164ad0> sys_rename+0x11/0x15
<c0102c3b> syscall_call+0x7/0xb
Code: e8 d7 6c f9 ff 58 8b 06 f6 c4 08 74 29 68 ac 51 45 c0 68 80 07 00 00 68 69 4d 45 c0 68 55 bc 41 c0 68 a3 2a 45 c0 e8 6a 83 f5 ff <0f> 0b 80 07 69 4d 45 c0 83 c4 14 58 5b 5e 5f 5d c3 55 31 ed 57
BUG: evolution/11320, lock held at task exit time!
[dae25194] {inode_init_once}.. held by: evolution:11320 [d5731570, 115]
... acquired at: lock_rename+0x8e/0x94
Signature · 2 lines
David Kleikamp IBM Linux Technology Center
Re: [PATCH 004 of 4] Make address_space_operations->invalidatepage return void
Quoted 8 lines · Dave Kleikamp
Quoted 2 lines · Dave Kleikamp
I'll try to stress test jfs with these patches to see if I can trigger the an oops here.
While stress testing on a jfs volume (dbench), I hit an assert in jbd:
Assertion failure in
journal_invalidatepage()atfs/jbd/transaction.c:1920: "!page_has_buffers(page)"
Yes, thanks, that assertion has become wrong.
fs/jbd/transaction.c renamed from fs/jbd/transaction.c~make-address_space_operations-invalidatepage-return-void-jbd-fix +2 -3
@@ -1915,9 +1915,8 @@ void journal_invalidatepage(journal_t *j }while (bh != head);if (!offset) { -/* Maybe should BUG_ON !may_free - neilb */ -try_to_free_buffers (page); -J_ASSERT (!page_has_buffers (page)); +if (may_free &&try_to_free_buffers (page)) +J_ASSERT (!page_has_buffers (page)); } }
However I'm more inclined to drop the whole patch, really - having ->invalidatepage() return a success indication makes sense. The fact that we're currently not using that return value doesn't mean that we shouldn't, didn't and won't.
Re: [PATCH 004 of 4] Make address_space_operations->invalidatepage return void
Quoted 27 lines · On Monday March 13, [email protected] wrote:
Quoted 8 lines · Dave Kleikamp
Quoted 2 lines · Dave Kleikamp
I'll try to stress test jfs with these patches to see if I can trigger the an oops here.
While stress testing on a jfs volume (dbench), I hit an assert in jbd:
Assertion failure in
journal_invalidatepage()atfs/jbd/transaction.c:1920: "!page_has_buffers(page)"Yes, thanks, that assertion has become wrong.
fs/jbd/transaction.c renamed from fs/jbd/transaction.c~make-address_space_operations-invalidatepage-return-void-jbd-fix +2
-3@@ -1915,9 +1915,8 @@ void journal_invalidatepage(journal_t *j }while (bh != head);if (!offset) { -/* Maybe should BUG_ON !may_free - neilb */ -try_to_free_buffers (page); -J_ASSERT (!page_has_buffers (page)); +if (may_free &&try_to_free_buffers (page)) +J_ASSERT (!page_has_buffers (page)); } }
Quoted 4 lines
However I'm more inclined to drop the whole patch, really - having ->invalidatepage() return a success indication makes sense. The fact that we're currently not using that return value doesn't mean that we shouldn't, didn't and won't.
->invalidatepage is called either when truncating a file or when purging the mapping of a file from the page cache.
The VM has waited for read or write back to complete and has ensured that no new io will happen on the page. The page, at least from 'offset' onwards, is idle.
Immediately after ->invalidatepage with offset==0 completes, the page is removed from the pagecache. It's a goner!
So I really don't think there is any sense for invalidation to be able to fail. The VM is saying "this page (or part of it) is going way. Now." The filesystem really has to let go.
I'm a little bit worried by this behaviour of journal_invalidatepage in that it doesn't always free the buffers...
I guess it hasn't finished committing a write when a truncate that discards that data comes along. So it needs to hold on to the page to write out those data blocks before forgetting about them.
But these machinations are completely "under-the-hood". The VM really doesn't need to know that ext3 is holding on to the page a bit longer. A return value to tell the VM still doesn't make sense.
Note that ->releasepage does a similar thing but does have a return value and here it is very meaningful. Here the VM is say "I'd like this page if you've finished with it". It isn't that the file is being truncated. It is just a memory-reclamation action. So a return value makes sense.
Finally, having a return value leads developers to think they need to return a value, which gives a wrong impression of what ->invalidatepage is for.
However, if you still don't like it.....
NeilBrown
Re: [PATCH 004 of 4] Make address_space_operations->invalidatepage return void
In reply to Dave KleikampQuoted 16 lines · On Monday March 13, [email protected] wrote:
Quoted 15 lines · NeilBrown
diff
./fs/jfs/jfs_metapage.c~current~./fs/jfs/jfs_metapage.c
---./fs/jfs/jfs_metapage.c~current~2006-03-09 17:29:35.000000000
+1100
+++./fs/jfs/jfs_metapage.c2006-03-13 10:46:55.000000000 +1100fs/jfs/jfs_metapage.c renamed from fs/jfs/jfs_metapage.c~current~ +1
-1@@ -578,14 +578,13 @@ static int metapage_releasepage(struct preturn 0 ; } -static int metapage_invalidatepage (struct page *page,unsigned long offset) +static void metapage_invalidatepage(struct page *page, unsigned long offset) {BUG_ON (offset);
Quoted 7 lines
fs/jfs/jfs_metapage.c static int metapage_releasepage(struct p · line 585
Quoted 3 lines
fs/jfs/jfs_metapage.c renamed from fs/jfs/jfs_metapage.c~current~ +1
-2-if (PageWriteback (page)) -return 0 ; +BUG_ON (PageWriteback (page));I'm a little concerned about adding a
BUG_ONfor something this function used to allow, but it looks like theBUG_ONis valid. I'm asking myself why did I add the test for PageWriteback in the first place.
Yes.... as far as I can tell, ->invalidatepage is only called with the page locked, and with Writeback clear, so PageWriteback can never be true. So the BUG_ON should be a no-op.
Quoted 10 lines
Quoted 7 lines
fs/jfs/jfs_metapage.c renamed from fs/jfs/jfs_metapage.c~current~ +1
-1-return metapage_releasepage (page,0 ); +metapage_releasepage (page,0 ); }struct
address_space_operationsjfs_metapage_aops= {I'll try to stress test jfs with these patches to see if I can trigger the an oops here.
Thanks. I'd be very interested if you do.
I got on oops with a similar bug_on in the new nfs_invalidatepage and it turned out to be a bug in radixtree (which I had already found and fixed, but not in that source tree).
NeilBrown
Re: [PATCH 004 of 4] Make address_space_operations->invalidatepage return void
Quoted 9 lines · Neil Brown
Quoted 3 lines · On Monday March 13, [email protected] wrote:
I'm a little concerned about adding a
BUG_ONfor something this function used to allow, but it looks like theBUG_ONis valid. I'm asking myself why did I add the test for PageWriteback in the first place.Yes.... as far as I can tell, ->invalidatepage is only called with the page locked, and with Writeback clear, so PageWriteback can never be true. So the
BUG_ONshould be a no-op.
Either I was being overly paranoid when I put in that test, or some code previously called invalidatepage with Writeback set and it's since been fixed.
Quoted 7 lines
Quoted 2 lines
I'll try to stress test jfs with these patches to see if I can trigger the an oops here.
Thanks. I'd be very interested if you do.
I got on oops with a similarbug_onin the newnfs_invalidatepageand it turned out to be a bug in radixtree (which I had already found and fixed, but not in that source tree).
Aside from the jbd assert that Andrew fixed, my stress testing of jfs was successful. I ACK the jfs-specific part of the patch.
Signature · 2 lines
David Kleikamp IBM Linux Technology Center