reiser4: fix readv
PATCH 1/1 reiser4: fix readv
Hello, Andrew
reiser4 in 2.6.18-rc6-mm2 has a bug. It can not do readv.
The attached patch fixes it by implementing reiser4' aio_read file operation. Unfortunately, it appeared to get a loop which is very similar to the one of fs/read_write.c:do_loop_readv_writev().
Alternatively, if do_loop_readv_writev were EXPORT_SYMBOL-ed
reiser4' aio_read could use it instead. But, there is a problem with do_loop_readv_writev EXPORT_SYMBOL-ing:
one if its arguments is io_fn_t, which is declared in fs/read_write.h.
If it is ok to move io_fn_t and do_loop_readv_writev declarations to include/linux/fs.h and to EXPORT_SYMBOL do_loop_readv_writev the fix will be smaller. Please, let me know what would you prefer.
From: Vladimir Saveliev <[email protected]>
This patch adds implementation of aio_read file operation for reiser4.
It is needed because in reiser4 there are files which can not be dealt with via generic page cache routines.
In case of readv, reiser4 has no meaning to find out file type and to choose proper way to read it. As result generic page cache read gets called for files which can not be read that way. Reiser4' aio_read method is to fix that problem.
fs/reiser4/plugin/object.c renamed from fs/reiser4/plugin/object.c~reiser4-add-aio_read +1 -1
@@ -101,7 +101,7 @@ file_plugin file_plugins[LAST_FILE_PLUGI .llseek = generic_file_llseek, .read = read_unix_file, .write = do_sync_write, - .aio_read = generic_file_aio_read, + .aio_read = aio_read_unix_file, .aio_write = generic_file_aio_write, .ioctl = ioctl_unix_file, .mmap = mmap_unix_file,
fs/reiser4/plugin/file/file.c renamed from fs/reiser4/plugin/file/file.c~reiser4-add-aio_read +48 -0
54 lines not on this page · raw message
fs/reiser4/plugin/file/file.h renamed from fs/reiser4/plugin/file/file.h~reiser4-add-aio_read +2 -0
@@ -15,6 +15,8 @@ int setattr_unix_file(struct dentry *, s/* file operations */ ssize_t read_unix_file (struct file *,char __user *buf,size_t read_amount,loff_t *off); +ssize_t aio_read_unix_file (struct kiocb *,const struct iovec *, +unsigned long nr_segs,loff_t pos);ssize_t write_unix_file (struct file *,const char __user *buf,size_t write_amount,loff_t * off); int ioctl_unix_file(struct inode *, struct file *, unsigned int cmd,
_
Hi Vladimir,
this fixes a bunch of random user space oddities for me. Just patched 2.6.18-rc6-mm2 with this, and some annoyances I could not trace back to a change in user space just went bye-bye.
Thanks a lot!
Chris
Quoted 8 lines · Christian Trefzer
Hi Vladimir,
this fixes a bunch of random user space oddities for me. Just patched 2.6.18-rc6-mm2 with this, and some annoyances I could not trace back to a change in user space just went bye-bye.
Thanks a lot!
Chris
This does not patch against 2.6.17-3 patchset however. Any possibility this may be related to some startup issues as noted on other threads here?
Thx
Signature · 6 lines
Peter +++++ Do not reply to this email, it is a spam trap and not monitored. I can be reached via this list, or via jabber: pete4abw at jabber.org ICQ: 73676357
Quoted 2 lines · Peter
This does not patch against 2.6.17-3 patchset however. Any possibility this may be related to some startup issues as noted on other threads here?
I have seen the issues at bootup you noticed some time ago, but only once after the root fs had been mounted in a different way, i.e. laptop hdd stuffed into a USB case and connected to my desktop, mounted to /mnt/something.
The most annoying problems that have been fixed for me were related to gentoo's postfix and spamd startup scripts. For some strange reason, they did not detect properly that postfix had been started / spamd had been stopped. Now everything seems kosher - I survived a few suspend/resume cycles without the hickups that have become common during the last few months. Sadly, I had more serious trouble to worry about, otherwise I had bugged the list myself : P
Kind regards,
Chris
Hello
Quoted 4 lines · Peter
Quoted 2 lines · Christian Trefzer
Hi Vladimir,
Quoted 8 lines
Quoted 6 lines
this fixes a bunch of random user space oddities for me. Just patched 2.6.18-rc6-mm2 with this, and some annoyances I could not trace back to a change in user space just went bye-bye.
Thanks a lot!
ChrisThis does not patch against 2.6.17-3 patchset however.
2.6.17-3 does not have the problem which that patch fixes. So you do not need it.
Quoted 3 lines
Any possibility
this may be related to some startup issues as noted on other threads here?
no.
Quoted 2 lines
Thx
Quoted 4 lines · On Wed, 13 Sep 2006 22:12:54 +0400 "Vladimir V. Saveliev" <[email protected]>…
Hello, Andrew
reiser4 in 2.6.18-rc6-mm2 has a bug. It can not do readv.
Quoted 9 lines
The attached patch fixes it by implementing reiser4'
aio_readfile operation. Unfortunately, it appeared to get a loop which is very similar to the one of fs/read_write.c:do_loop_readv_writev().
Alternatively, ifdo_loop_readv_writevwere EXPORT_SYMBOL-ed
reiser4'aio_readcould use it instead. But, there is a problem withdo_loop_readv_writevEXPORT_SYMBOL-ing:
one if its arguments isio_fn_t, which is declared infs/read_write.h.
If it is ok to moveio_fn_tanddo_loop_readv_writevdeclarations toinclude/linux/fs.hand toEXPORT_SYMBOLdo_loop_readv_writevthe fix will be smaller. Please, let me know what would you prefer.
Yes, I'd say that do_loop_readv_writev() is suitable for exporting to filesystems, and that doing so is preferable to duplicating it.
That'd be two patches, please: one to do the export and one to use it in reiser4.
I assume there's a good reason why reiser4 cannot use generic_file_aio_read() or vfs_readv(). Please capture that discussion in the changelog for the first patch, thanks.
Quoted 98 lines
From: Vladimir Saveliev <[email protected]>
This patch adds implementation of
aio_readfile operation for reiser4.
It is needed because in reiser4 there are files which can not be dealt with via generic page cache routines.
In case of readv, reiser4 has no meaning to find out file type and to choose proper way to read it. As result generic page cache read gets called for files which can not be read that way. Reiser4'aio_readmethod is to fix that problem.Signed-off-by Vladimir Saveliev <[email protected]>diff -puN
fs/reiser4/plugin/object.c~reiser4-add-aio_readfs/reiser4/plugin/object.cfs/reiser4/plugin/object.c renamed from fs/reiser4/plugin/object.c~reiser4-add-aio_read +1
-1@@ -101,7 +101,7 @@ file_plugin file_plugins[LAST_FILE_PLUGI .llseek = generic_file_llseek, .read = read_unix_file, .write = do_sync_write, - .aio_read = generic_file_aio_read, + .aio_read = aio_read_unix_file, .aio_write = generic_file_aio_write, .ioctl = ioctl_unix_file, .mmap = mmap_unix_file,diff -puN
fs/reiser4/plugin/file/file.c~reiser4-add-aio_readfs/reiser4/plugin/file/file.cfs/reiser4/plugin/file/file.c renamed from fs/reiser4/plugin/file/file.c~reiser4-add-aio_read +48
-0@@ -2011,6 +2011,54 @@ out:return result; } +/** +* aio_read_unix_file - aio_read of struct file_operations +* @iocb: i/o control block +* @iov: i/o vector +* @nr_segs: number of segments in the i/o vector +* @pos: file position to read from +* +* When it is called within reiser4 context (this happens when sys_read is +* reading a file built of extents) - just call generic_file_aio_read to +* perform read into page cache. When it is called without reiser4 context +* (sys_readv) - call read_unix_file for each segments of i/o vector, so that +* read_unix_file will be able to choose whether the file is to be read into +* page cache or the file is built of tail items and page cache read is not +* suitable for it. +*/ +ssize_t aio_read_unix_file (struct kiocb *iocb,const struct iovec *iov, +unsigned long nr_segs,loff_t pos) +{ +ssize_t ret =0 ; + +if (is_in_reiser4_context ()) +return generic_file_aio_read (iocb, iov, nr_segs, pos); + +while (nr_segs >0 ) { +void __user *base; +size_t len; +ssize_t nr; + + base = iov->iov_base; + len = iov->iov_len; + iov++; + nr_segs--; + + nr =read_unix_file (iocb->ki_filp, base, len, &iocb->ki_pos); +if (nr <0 ) { +if (!ret) + ret = nr; +break ; + } + ret += nr; +if (nr != len) +break ; + } + +return ret; + +} +static ssize_t read_unix_file_container_tails (struct file *file,char __user *buf,size_t read_amount,loff_t *off) {diff -puN
fs/reiser4/plugin/file/file.h~reiser4-add-aio_readfs/reiser4/plugin/file/file.hfs/reiser4/plugin/file/file.h renamed from fs/reiser4/plugin/file/file.h~reiser4-add-aio_read +2
-0@@ -15,6 +15,8 @@ int setattr_unix_file(struct dentry *, s/* file operations */ ssize_t read_unix_file (struct file *,char __user *buf,size_t read_amount,loff_t *off); +ssize_t aio_read_unix_file (struct kiocb *,const struct iovec *, +unsigned long nr_segs,loff_t pos);ssize_t write_unix_file (struct file *,const char __user *buf,size_t write_amount,loff_t * off); int ioctl_unix_file(struct inode *, struct file *, unsigned int cmd,_
Hello
Quoted 20 lines · Andrew Morton
Quoted 4 lines · On Wed, 13 Sep 2006 22:12:54 +0400 "Vladimir V. Saveliev" <[email protected]>…
Hello, Andrew
reiser4 in 2.6.18-rc6-mm2 has a bug. It can not do readv.
Quoted 9 lines
The attached patch fixes it by implementing reiser4'
aio_readfile operation. Unfortunately, it appeared to get a loop which is very similar to the one of fs/read_write.c:do_loop_readv_writev().
Alternatively, ifdo_loop_readv_writevwere EXPORT_SYMBOL-ed
reiser4'aio_readcould use it instead. But, there is a problem withdo_loop_readv_writevEXPORT_SYMBOL-ing:
one if its arguments isio_fn_t, which is declared infs/read_write.h.
If it is ok to moveio_fn_tanddo_loop_readv_writevdeclarations toinclude/linux/fs.hand toEXPORT_SYMBOLdo_loop_readv_writevthe fix will be smaller. Please, let me know what would you prefer.Yes, I'd say that
do_loop_readv_writev()is suitable for exporting to filesystems, and that doing so is preferable to duplicating it.
Quoted 7 lines
That'd be two patches, please: one to do the export and one to use it in reiser4.
I assume there's a good reason why reiser4 cannot use
generic_file_aio_read()orvfs_readv(). Please capture that discussion in the changelog for the first patch, thanks.
It seems the problem can be fixed a bit simpler. Currently, there is a difference between read and readv: read calls f_op->read if it is defined, but readv calls f_op->read if f_op->aio_read is not defined. The latest is a bit unlogical imho:
wouldn't it be more consistent if readv worked via f_op->read if it is defined?
If we fixed readv (do_readv_writev) that way - reiser4 would not need anything from its aio_read but generic_file_aio_read.
From: Vladimir Saveliev <[email protected]>
There is some asymmetry between read and readv:
read (vfs_read, namely) calls f_op->read when it is defined,
while readv (do_readv_writev) calls f_op->read when f_op->aio_read is not defined. This patch makes do_readv_writev to call do_loop_readv_writev
(which calls f_op->read for each segment of i/o vector) if f_op->read is defined.
fs/read_write.c renamed from fs/read_write.c~fix-do_readv_writev +3 -4
@@ -608,7 +608,6 @@ static ssize_t do_readv_writev(int type,if (ret) goto out; - fnv =NULL ;if (type ==READ ) { fn = file->f_op->read; fnv = file->f_op->aio_read;
@@ -617,11 +616,11 @@ static ssize_t do_readv_writev(int type, fnv = file->f_op->aio_write; } -if (fnv) +if (fn) + ret =do_loop_readv_writev (file, iov, nr_segs, pos, fn); +else ret =do_sync_readv_writev (file, iov, nr_segs, tot_len, pos, fnv); -else - ret =do_loop_readv_writev (file, iov, nr_segs, pos, fn); out:if (iov != iovstack)
_
Quoted 77 lines · On Sun, 17 Sep 2006 22:39:07 +0400 "Vladimir V. Saveliev" <[email protected]>…
Hello
Quoted 20 lines · Andrew Morton
Quoted 4 lines · On Wed, 13 Sep 2006 22:12:54 +0400 "Vladimir V. Saveliev" <[email protected]>…
Hello, Andrew
reiser4 in 2.6.18-rc6-mm2 has a bug. It can not do readv.
Quoted 9 lines
The attached patch fixes it by implementing reiser4'
aio_readfile operation. Unfortunately, it appeared to get a loop which is very similar to the one of fs/read_write.c:do_loop_readv_writev().
Alternatively, ifdo_loop_readv_writevwere EXPORT_SYMBOL-ed
reiser4'aio_readcould use it instead. But, there is a problem withdo_loop_readv_writevEXPORT_SYMBOL-ing:
one if its arguments isio_fn_t, which is declared infs/read_write.h.
If it is ok to moveio_fn_tanddo_loop_readv_writevdeclarations toinclude/linux/fs.hand toEXPORT_SYMBOLdo_loop_readv_writevthe fix will be smaller. Please, let me know what would you prefer.Yes, I'd say that
do_loop_readv_writev()is suitable for exporting to filesystems, and that doing so is preferable to duplicating it.Quoted 7 lines
That'd be two patches, please: one to do the export and one to use it in reiser4.
I assume there's a good reason why reiser4 cannot use
generic_file_aio_read()orvfs_readv(). Please capture that discussion in the changelog for the first patch, thanks.It seems the problem can be fixed a bit simpler. Currently, there is a difference between read and readv: read calls
f_op->readif it is defined, but readv callsf_op->readiff_op->aio_readis not defined. The latest is a bit unlogical imho:
wouldn't it be more consistent if readv worked viaf_op->readif it is defined?
If we fixed readv (do_readv_writev) that way - reiser4 would not need anything from itsaio_readbutgeneric_file_aio_read.From: Vladimir Saveliev <[email protected]>
There is some asymmetry between read and readv:
read (vfs_read, namely) callsf_op->readwhen it is defined,
while readv (do_readv_writev) callsf_op->readwhenf_op->aio_readis not defined. This patch makesdo_readv_writevto calldo_loop_readv_writev
(which callsf_op->readfor each segment of i/o vector) iff_op->readis defined.Signed-off-by Vladimir Saveliev <[email protected]>diff -puN
fs/read_write.c~fix-do_readv_writevfs/read_write.cfs/read_write.c renamed from fs/read_write.c~fix-do_readv_writev +3
-4@@ -608,7 +608,6 @@ static ssize_t do_readv_writev(int type,if (ret) goto out; - fnv =NULL ;if (type ==READ ) { fn = file->f_op->read; fnv = file->f_op->aio_read;@@ -617,11 +616,11 @@ static ssize_t do_readv_writev(int type, fnv = file->f_op->aio_write; } -if (fnv) +if (fn) + ret =do_loop_readv_writev (file, iov, nr_segs, pos, fn); +else ret =do_sync_readv_writev (file, iov, nr_segs, tot_len, pos, fnv); -else - ret =do_loop_readv_writev (file, iov, nr_segs, pos, fn); out:if (iov != iovstack)
Quoted 1 line
_
OK, thanks. I'll wait to hear from Badari and Christoph on that.
btw, please be aware that the post-2.6.17 deadlock fixes which went into generic_file_buffered_write() caused a few problems:
a) Significant NFS overwrite performance regression (due to running prepare_write/commit_write) against each writev segment.
b) Smaller but significant performance regression for all writev() usages (I expect - haven't measured this yet).
c) It's still theoretically deadlockable.
I am (slowly) working on addressing these issues - more info later.
Basically, we will again need to be able to traverse multiple iovec segments within a single prepare_write/commit_write instance. This will impact your batch_write patch, because that patch somewhat codifies in the function layout the concept that we do one-segment-pre-prepare_write().
So that patch will need significant rework, I suspect.
What I'm hoping to be able to do is to simply revert the two deadlock-fix patches (so we go back to the 2.6.17 code) and to remove fault_in_pages_readable() and to simply unlock the page after ->prepare_write() and lock it again prior to ->commit_write(). But I haven't tried that yet - there's quite a bit of preparatory work needed.
Quoted 4 lines · Vladimir V. Saveliev
It seems the problem can be fixed a bit simpler. Currently, there is a difference between read and readv: read calls
f_op->readif it is defined, but readv callsf_op->readiff_op->aio_readis not defined. The latest is a bit unlogical imho:
wouldn't it be more consistent if readv worked viaf_op->readif it is defined?
If we fixed readv (do_readv_writev) that way - reiser4 would not need anything from itsaio_readbutgeneric_file_aio_read.
This behaviour is historic baggae. readv used to call ->readv and fall back to ->read when it wasn't available. We now merged ->readv into ->aio_read and kept that behaviour. I think it makes sense, though - if the filesystem supports vator operations via ->aio_read we should use them and not fall back to the manual looping around ->read.
I'd rather change read to do the same thing as readv so we have consistent behaviour.
why does this matter for reiser4?
Hello
Quoted 6 lines · Christoph Hellwig
Quoted 4 lines · Vladimir V. Saveliev
It seems the problem can be fixed a bit simpler. Currently, there is a difference between read and readv: read calls
f_op->readif it is defined, but readv callsf_op->readiff_op->aio_readis not defined. The latest is a bit unlogical imho:
wouldn't it be more consistent if readv worked viaf_op->readif it is defined?
If we fixed readv (do_readv_writev) that way - reiser4 would not need anything from itsaio_readbutgeneric_file_aio_read.
Quoted 9 lines
This behaviour is historic baggae. readv used to call ->readv and fall back to ->read when it wasn't available. We now merged ->readv into ->aio_read and kept that behaviour. I think it makes sense, though - if the filesystem supports vator operations via ->aio_read we should use them and not fall back to the manual looping around ->read.
I'd rather change read to do the same thing as readv so we have consistent behaviour.
Can we put it that way: if a filesystem can use page_cache directly - it has to set f_op->aio_read to generic_file_aio_read and set f_op->read to NULL. If the filesystem wants to try to screw things up a bit -
it implements f_op->read and its f_op->read is called by sys_read and sys_readv
regardless to whether it has aio_read or not?
Quoted 2 lines
why does this matter for reiser4?
reiser4 reads some files via generic page cache routines. In that case reiser4' read calls do_sync_read. Therefore, it has to define f_op->aio_read.
OTOH, there are files for which reiser4' read does not call do_sync_read.
In case of readv, f_op->aio_read is called directly (if it is defined), which may result in that reiser4' aio_read is called for files for which reiser4' read would never call do_sync_read.
To avoid the problem we have to implement reiser4_aio_read which either calls generic_file_aio_read or does something very similar to do_loop_readv_writev.
Quoted 4 lines · Vladimir V. Saveliev
Can we put it that way: if a filesystem can use
page_cachedirectly - it has to setf_op->aio_readtogeneric_file_aio_readand setf_op->readto NULL. If the filesystem wants to try to screw things up a bit -
it implementsf_op->readand itsf_op->readis called bysys_readandsys_readv
regardless to whether it hasaio_reador not?
Not entirely. The idea I had in my mind was the following:
- real filesystems should always implement the aio_ methods and must support async and vectored I/O
- drivers or simple synthetic filesystems can implement just ->read and ->write and stay out of all the complexity.
In the end I would like to enforce and invariant where a fileesystem or driver implements either the aio_ or normal methods, but we can't do that yet as there are far too many places calling thos directly. Thus we have do_sync_read and do_sync_write that are used as read and write methods for those more complex filesystems. (Anyone's got an idea how to enforce that people never set ->read and ->write to anything else when they implement the aio methods?)
Quoted 10 lines
Quoted 2 lines
why does this matter for reiser4?
reiser4 reads some files via generic page cache routines. In that case reiser4' read calls
do_sync_read. Therefore, it has to definef_op->aio_read.
OTOH, there are files for which reiser4' read does not calldo_sync_read.In case of readv,
f_op->aio_readis called directly (if it is defined), which may result in that reiser4'aio_readis called for files for which reiser4' read would never calldo_sync_read.
To avoid the problem we have to implementreiser4_aio_readwhich either callsgeneric_file_aio_reador does something very similar todo_loop_readv_writev.
In that case reiserfs should only implement aio_read and aio_write methods and use do_loop_readv_writev which we should export for a beginning. Longer term you should try to implement full blown aio and vector support even for those odd files (or find a way to migrate the pagecache). Do files change from odd to normal while they're instanciated? Otherwise you could just declare to sets of file_operations, once that uses the pagecache and one that doesn't and decide at inode instanciation time which one to use.