CIFS: Make read call work with strict cache mode (try #4)
Subsystems (2)
PATCH 2/6 CIFS: Make read call work with strict cache mode (try #4)
Add cifs_file_aio_read where we read from the cache if we have at least Level II oplock - otherwise read from the server.
--- fs/cifs/cifsfs.c | 44 ++++++++++++++++++++++++++++++++++++++++++-- 1 files changed, 42 insertions(+), 2 deletions(-)
fs/cifs/cifsfs.c +42 -2
@@ -568,6 +568,46 @@ cifs_do_mount(struct file_system_type *fs_type,return dget (sb->s_root); } +static ssize_t cifs_file_aio_read (struct kiocb *iocb,const struct iovec *iov, +unsigned long nr_segs,loff_t pos) +{ +struct inode *inode; +struct cifs_sb_info *cifs_sb; +ssize_t read =0 , retval; +unsigned long i; + + inode = iocb->ki_filp->f_path.dentry->d_inode; + cifs_sb =CIFS_SB (iocb->ki_filp->f_path.dentry->d_sb); + +if ((cifs_sb->mnt_cifs_flags &CIFS_MOUNT_STRICT_IO ) ==0 || +CIFS_I (inode)->clientCanCacheRead) +return generic_file_aio_read (iocb, iov, nr_segs, pos); + +/* +* In strict cache mode we need to read from the server all the time +* if we don't have level II oplock because the server can delay mtime +* change - so we can't make a decision about inode invalidating. +* And we can also fail with pagereading if there are mandatory locks +* on pages affected by this read but not on the region from pos to +* pos+len-1. +*/ + +for (i =0 ; i < nr_segs; i++) { + retval =cifs_user_read (iocb->ki_filp, iov[i].iov_base, + iov[i].iov_len, &pos); +if (retval <0 ) { + read = read ? read : retval; +break ; + } + + read += retval; + } + + iocb->ki_pos = pos; + +return read; +} +static ssize_t cifs_file_aio_write (struct kiocb *iocb,const struct iovec *iov,unsigned long nr_segs,loff_t pos) {
@@ -690,7 +730,7 @@ const struct inode_operations cifs_symlink_inode_ops = {const struct file_operations cifs_file_ops = { .read = do_sync_read, .write = do_sync_write, - .aio_read = generic_file_aio_read, + .aio_read = cifs_file_aio_read, .aio_write = cifs_file_aio_write, .open = cifs_open, .release = cifs_close,
@@ -727,7 +767,7 @@ const struct file_operations cifs_file_direct_ops = {const struct file_operations cifs_file_nobrl_ops = { .read = do_sync_read, .write = do_sync_write, - .aio_read = generic_file_aio_read, + .aio_read = cifs_file_aio_read, .aio_write = cifs_file_aio_write, .open = cifs_open, .release = cifs_close,
Signature · 1 line
1.7.3.2
A message that is not in the archive (sent off-list or not yet received): 1291103671-2983-1-git-send-email-piastryyy-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org
PATCH 3/6 CIFS: Make write call work with strict cache mode (try #4)
On strict cache mode if we don't have Exclusive oplock we write a data to the server through cifs_user_write. Then if we Level II oplock store it in the cache, otherwise - invalidate inode pages affected by this writing.
--- fs/cifs/cifsfs.c | 51 ++++++++++++++++++++++++++++++++++++++++++++++----- 1 files changed, 46 insertions(+), 5 deletions(-)
fs/cifs/cifsfs.c +46 -5
@@ -611,12 +611,53 @@ static ssize_t cifs_file_aio_read(struct kiocb *iocb, const struct iovec *iov,static ssize_t cifs_file_aio_write (struct kiocb *iocb,const struct iovec *iov,unsigned long nr_segs,loff_t pos) { -struct inode *inode = iocb->ki_filp->f_path.dentry->d_inode; -ssize_t written; +struct inode *inode; +struct cifs_sb_info *cifs_sb; +ssize_t written =0 , retval; +unsigned long i; + + inode = iocb->ki_filp->f_path.dentry->d_inode; + +if (CIFS_I (inode)->clientCanCacheAll) +return generic_file_aio_write (iocb, iov, nr_segs, pos); + + cifs_sb =CIFS_SB (iocb->ki_filp->f_path.dentry->d_sb); + +if ((cifs_sb->mnt_cifs_flags &CIFS_MOUNT_STRICT_IO ) ==0 ) { +int rc; + + written =generic_file_aio_write (iocb, iov, nr_segs, pos); + + rc =filemap_fdatawrite (inode->i_mapping); +if (rc) +cFYI (1 ,"cifs_file_aio_write: %d rc on %p inode" , + rc, inode); +return written; + } + +/* +* In strict cache mode we need to write the data to the server exactly +* from the pos to pos+len-1 rather than flush all affected pages +* because it may cause a error with mandatory locks on these pages but +* not on the region from pos to ppos+len-1. +*/ + +for (i =0 ; i < nr_segs; i++) { + retval =cifs_user_write (iocb->ki_filp, iov[i].iov_base, + iov[i].iov_len, &pos); +if (retval <0 ) { + written = written ? written : retval; +break ; + } + + written += retval; + } + + iocb->ki_pos = pos; + +if (written >0 ) +CIFS_I (inode)->invalid_mapping = true; - written =generic_file_aio_write (iocb, iov, nr_segs, pos); -if (!CIFS_I (inode)->clientCanCacheAll) -filemap_fdatawrite (inode->i_mapping);return written; }
Signature · 1 line
1.7.3.2
A message that is not in the archive (sent off-list or not yet received): 1291103671-2983-2-git-send-email-piastryyy-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org
Re: [PATCH 3/6] CIFS: Make write call work with strict cache mode (try #4)
2010/11/30 Pavel Shilovsky <[email protected]>:
Quoted 73 lines
On strict cache mode if we don't have Exclusive oplock we write a data to the server through
cifs_user_write. Then if we Level II oplock store it in the cache, otherwise - invalidate inode pages affected by this writing.Signed-off-by Pavel Shilovsky <[email protected]>---
fs/cifs/cifsfs.c | 51 ++++++++++++++++++++++++++++++++++++++++++++++-----1 files changed, 46 insertions(+), 5 deletions(-)fs/cifs/cifsfs.c +46
-6@@ -611,12 +611,53 @@ static ssize_t cifs_file_aio_read(struct kiocb *iocb, const struct iovec *iov, static ssize_t cifs_file_aio_write(struct kiocb *iocb, const struct iovec *iov, unsigned long nr_segs, loff_t pos) { - struct inode *inode = iocb->ki_filp->f_path.dentry->d_inode; - ssize_t written; + struct inode *inode; + struct cifs_sb_info *cifs_sb; + ssize_t written = 0, retval; + unsigned long i; + + inode = iocb->ki_filp->f_path.dentry->d_inode; + + if (CIFS_I(inode)->clientCanCacheAll) + return generic_file_aio_write(iocb, iov, nr_segs, pos); + + cifs_sb = CIFS_SB(iocb->ki_filp->f_path.dentry->d_sb); + + if ((cifs_sb->mnt_cifs_flags & CIFS_MOUNT_STRICT_IO) == 0) { + int rc; + + written = generic_file_aio_write(iocb, iov, nr_segs, pos); + + rc = filemap_fdatawrite(inode->i_mapping); + if (rc) + cFYI(1, "cifs_file_aio_write: %d rc on %p inode", + rc, inode); + return written; + } + + /* + * In strict cache mode we need to write the data to the server exactly + * from the pos to pos+len-1 rather than flush all affected pages + * because it may cause a error with mandatory locks on these pages but + * not on the region from pos to ppos+len-1. + */ + + for (i = 0; i < nr_segs; i++) { + retval = cifs_user_write(iocb->ki_filp, iov[i].iov_base, + iov[i].iov_len, &pos); + if (retval < 0) { + written = written ? written : retval; + break; + } + + written += retval; + } + + iocb->ki_pos = pos; + + if (written > 0) + CIFS_I(inode)->invalid_mapping = true; - written = generic_file_aio_write(iocb, iov, nr_segs, pos); - if (!CIFS_I(inode)->clientCanCacheAll) - filemap_fdatawrite(inode->i_mapping); return written; } --
Quoted 3 lines
1.7.3.2
What's about this one and read patch #4? fsync and mmap patches #3?
Signature · 2 lines
Best regards, Pavel Shilovsky.
Re: [PATCH 3/6] CIFS: Make write call work with strict cache mode (try #4)
In reply to a message not in the archiveQuoted 51 lines · On Tue, 30 Nov 2010 10:54:31 +0300 Pavel Shilovsky…
On strict cache mode if we don't have Exclusive oplock we write a data to the server through
cifs_user_write. Then if we Level II oplock store it in the cache, otherwise - invalidate inode pages affected by this writing.Signed-off-by Pavel Shilovsky <[email protected]>--- fs/cifs/cifsfs.c | 51 ++++++++++++++++++++++++++++++++++++++++++++++----- 1 files changed, 46 insertions(+), 5 deletions(-)fs/cifs/cifsfs.c +32
-2@@ -611,12 +611,53 @@ static ssize_t cifs_file_aio_read(struct kiocb *iocb, const struct iovec *iov,static ssize_t cifs_file_aio_write (struct kiocb *iocb,const struct iovec *iov,unsigned long nr_segs,loff_t pos) { -struct inode *inode = iocb->ki_filp->f_path.dentry->d_inode; -ssize_t written; +struct inode *inode; +struct cifs_sb_info *cifs_sb; +ssize_t written =0 , retval; +unsigned long i; + + inode = iocb->ki_filp->f_path.dentry->d_inode; + +if (CIFS_I (inode)->clientCanCacheAll) +return generic_file_aio_write (iocb, iov, nr_segs, pos); + + cifs_sb =CIFS_SB (iocb->ki_filp->f_path.dentry->d_sb); + +if ((cifs_sb->mnt_cifs_flags &CIFS_MOUNT_STRICT_IO ) ==0 ) { +int rc; + + written =generic_file_aio_write (iocb, iov, nr_segs, pos); + + rc =filemap_fdatawrite (inode->i_mapping); +if (rc) +cFYI (1 ,"cifs_file_aio_write: %d rc on %p inode" , + rc, inode); +return written; + } + +/* +* In strict cache mode we need to write the data to the server exactly +* from the pos to pos+len-1 rather than flush all affected pages +* because it may cause a error with mandatory locks on these pages but +* not on the region from pos to ppos+len-1. +*/ + +for (i =0 ; i < nr_segs; i++) {
fs/cifs/cifsfs.c static ssize_t cifs_file_aio_read(struct kiocb *iocb, const struct iovec *iov, · line 655
Quoted 10 lines
fs/cifs/cifsfs.c +10
-0+ retval =cifs_user_write (iocb->ki_filp, iov[i].iov_base, + iov[i].iov_len, &pos); +if (retval <0 ) { + written = written ? written : retval; +break ; + } + + written += retval; + } +
Same problem as with the read patch. This is going to mean a bunch of small writes on the wire just because the application sent you a set of small iovecs. I think you want to batch those up into smaller set of writes.
Quoted 11 lines
fs/cifs/cifsfs.c +4
-3+ iocb->ki_pos = pos; + +if (written >0 ) +CIFS_I (inode)->invalid_mapping = true; - written =generic_file_aio_write (iocb, iov, nr_segs, pos); -if (!CIFS_I (inode)->clientCanCacheAll) -filemap_fdatawrite (inode->i_mapping);return written; }
Signature · 1 line
Jeff Layton <[email protected]>
2010/11/30 Pavel Shilovsky <[email protected]>:
Quoted 4 lines
Add
cifs_file_aio_readwhere we read from the cache if we have at least Level II oplock - otherwise read from the server.Signed-off-by Pavel Shilovsky <[email protected]>
What's about this?
Signature · 2 lines
Best regards, Pavel Shilovsky.
Quoted 41 lines · On Tue, 30 Nov 2010 10:54:30 +0300 Pavel Shilovsky…
Add
cifs_file_aio_readwhere we read from the cache if we have at least Level II oplock - otherwise read from the server.Signed-off-by Pavel Shilovsky <[email protected]>--- fs/cifs/cifsfs.c | 44 ++++++++++++++++++++++++++++++++++++++++++-- 1 files changed, 42 insertions(+), 2 deletions(-)fs/cifs/cifsfs.c +25
-0@@ -568,6 +568,46 @@ cifs_do_mount(struct file_system_type *fs_type,return dget (sb->s_root); } +static ssize_t cifs_file_aio_read (struct kiocb *iocb,const struct iovec *iov, +unsigned long nr_segs,loff_t pos) +{ +struct inode *inode; +struct cifs_sb_info *cifs_sb; +ssize_t read =0 , retval; +unsigned long i; + + inode = iocb->ki_filp->f_path.dentry->d_inode; + cifs_sb =CIFS_SB (iocb->ki_filp->f_path.dentry->d_sb); + +if ((cifs_sb->mnt_cifs_flags &CIFS_MOUNT_STRICT_IO ) ==0 || +CIFS_I (inode)->clientCanCacheRead) +return generic_file_aio_read (iocb, iov, nr_segs, pos); + +/* +* In strict cache mode we need to read from the server all the time +* if we don't have level II oplock because the server can delay mtime +* change - so we can't make a decision about inode invalidating. +* And we can also fail with pagereading if there are mandatory locks +* on pages affected by this read but not on the region from pos to +* pos+len-1. +*/ + +for (i =0 ; i < nr_segs; i++) {
fs/cifs/cifsfs.c cifs_do_mount(struct file_system_type *fs_type, · line 605
Quoted 10 lines
fs/cifs/cifsfs.c +10
-0+ retval =cifs_user_read (iocb->ki_filp, iov[i].iov_base, + iov[i].iov_len, &pos); +if (retval <0 ) { + read = read ? read : retval; +break ; + } + + read += retval; + } +
Hmmm...that looks a little ugly. There is no batching here, so if the application sends you an array of very small iovecs, then that becomes a set of very small reads on the wire. I don't think that's what you want.
Quoted 26 lines
fs/cifs/cifsfs.c +7
-2+ iocb->ki_pos = pos; + +return read; +} +static ssize_t cifs_file_aio_write (struct kiocb *iocb,const struct iovec *iov,unsigned long nr_segs,loff_t pos) {@@ -690,7 +730,7 @@ const struct inode_operations cifs_symlink_inode_ops = {const struct file_operations cifs_file_ops = { .read = do_sync_read, .write = do_sync_write, - .aio_read = generic_file_aio_read, + .aio_read = cifs_file_aio_read, .aio_write = cifs_file_aio_write, .open = cifs_open, .release = cifs_close,@@ -727,7 +767,7 @@ const struct file_operations cifs_file_direct_ops = {const struct file_operations cifs_file_nobrl_ops = { .read = do_sync_read, .write = do_sync_write, - .aio_read = generic_file_aio_read, + .aio_read = cifs_file_aio_read, .aio_write = cifs_file_aio_write, .open = cifs_open, .release = cifs_close,
Signature · 1 line
Jeff Layton <[email protected]>
A message that is not in the archive (sent off-list or not yet received): 20101207072610.50c5810a-9yPaYZwiELC+kQycOl6kW4xkIHaj4LzF@public.gmane.org
2010/12/7 Jeff Layton <[email protected]>:
Quoted 4 lines
Hmmm...that looks a little ugly. There is no batching here, so if the application sends you an array of very small iovecs, then that becomes a set of very small reads on the wire. I don't think that's what you want.
Jeff, I investigated this a little and found out a problem with mandatory byte-range locks:
For example, the first user locks file from 11 to 12, the second user specifies two buffers with 10 and 20 lengths and wants to fill them with readv call. In your case we merge these buffers into one request, sends it and got error. As the result we don't have any data returned from the server. So the second user thinks that the file is locked on both regions (from 0 to 10 and from 10 to 30) but it is not true! Almost the same things we can see in write case. So, what's why I think we shouldn't do it in strict cache case.
Thoughts?
Signature · 2 lines
Best regards, Pavel Shilovsky.
A message that is not in the archive (sent off-list or not yet received): AANLkTikrDA_NnK_unas1FABStcre2MPCbV0iJoV2HcYG-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org
Quoted 18 lines · On Sat, 11 Dec 2010 22:47:42 +0300 Pavel Shilovsky…
2010/12/7 Jeff Layton <[email protected]>:
Quoted 4 lines
Hmmm...that looks a little ugly. There is no batching here, so if the application sends you an array of very small iovecs, then that becomes a set of very small reads on the wire. I don't think that's what you want.
Jeff, I investigated this a little and found out a problem with mandatory byte-range locks:
For example, the first user locks file from 11 to 12, the second user specifies two buffers with 10 and 20 lengths and wants to fill them with readv call. In your case we merge these buffers into one request, sends it and got error. As the result we don't have any data returned from the server. So the second user thinks that the file is locked on both regions (from 0 to 10 and from 10 to 30) but it is not true! Almost the same things we can see in write case. So, what's why I think we shouldn't do it in strict cache case.
Quoted 2 lines
Thoughts?
I think that the number of buffers to be filled shouldn't affect the behavior of the client on the wire. If someone happens to trip over a BRL here, then tough luck for them.
The multiple buffer case is really no different than the case of a single buffer, other than the fact that you're stuffing the result into multiple buffers rather than a single one. We're certainly under no obligation to partially satisfy a read or write in such a case.
Signature · 1 line
Jeff Layton <[email protected]>