Skip to contentSkip to conversation

AIO and vectored I/O support for cifs

DISCUSSION Steve French 2 participants 3 messages Mar 25, 2005 – Apr 4, 2005 VFS

lore
Share & tools
lore
#0 lore raw

Christoph,
I had time to add the generic vectored i/o and async i/o calls to cifs that you had suggested last month. They are within the ifdef for the CIFS_EXPERIMENTAL config option for the time being. I would like to do more testing of these though - are there any tests (even primitive ones) for readv/writev and async i/o?

Is there an easy way measuring the performance benefit of these (vs. the fallback routines in fs/read_write.c) - since presumably async and vectored i/o never kick in on a standard copy command such as cp or dd and require a modified application that is vector i/o aware or async i/o aware.

You had mentioned do_sync_read - is there a reason to change the current call to generic_file_read in the cifs read entry point to do_sync_read. Some filesystems which export aio routines still call generic_file_read and others call do_sync_read and it was not obvious to me what that would change.

This is partly to better limit reading from the pagecache when the read oplock is lost (ie when we do not have the network caching token allowing readahead from the server) but primarily because I would like to see if this could help with getting more parallelism in the single client to single server large file sequential copy case. Currently CIFS can do large operations (as large as 128K for read or write in some cases) - and this is much more efficient for network transfer - but without mounting with forcedirectio I had limited my cifs_readpages to 16K (4 pages typically) - and because I do the SMBread synchronously I am severely limiting parallelism in the case of a single threaded app. So where I would like to get to is during readahead having multiple SMB reads for the same inode on the wire at one time - with the SMB reads each larger than a page (between 4 and 30 pages) - and was hoping that the aio and readv/writev support would make that easier.

I probably need to look more at the NFS direct i/o example to see if there are easy changes I can make to enable it on a per inode basis (rather than as only a mount option), and to double check what other filesystem do for returning errors on mmap and sendfile on inodes that are marked direct i/o.

#1 lore raw

cc'ing linux-aio, for the AIO part of the discussion. You might be able to find some of your answers in the archives.

Quoted 7 lines · Steve French

Christoph,
I had time to add the generic vectored i/o and async i/o calls to cifs that you had suggested last month. They are within the ifdef for the
CIFS_EXPERIMENTAL config option for the time being. I would like to do
more testing of these though - are there any tests (even primitive ones) for readv/writev and async i/o?

Quoted 5 lines

Is there an easy way measuring the performance benefit of these (vs. the fallback routines in fs/read_write.c) - since presumably async and vectored i/o never kick in on a standard copy command such as cp or dd and require a modified application that is vector i/o aware or async i/o aware.

there are several tests for AIO - I tend to use Chris Mason's aio-stress which can be used to compare performance in terms of throughput for streaming reads/writes for different variations of options.

(the following page isn't exactly up-to-date, but should still give you some pointers: lse.sf.net/io/aio.html)

Quoted 6 lines

You had mentioned do_sync_read - is there a reason to change the current call to generic_file_read in the cifs read entry point to do_sync_read. Some filesystems which export aio routines still call generic_file_read and others call do_sync_read and it was not obvious to me what that would change.

I think you could keep it the way it is - generic_file_read will take care of things. But maybe I should comment only after I see your patch. Are you planning to post it some time ?

Regards
Suparna

Quoted 25 lines

This is partly to better limit reading from the pagecache when the read oplock is lost (ie when we do not have the network caching token allowing readahead from the server) but primarily because I would like to see if this could help with getting more parallelism in the single
client to single server large file sequential copy case. Currently
CIFS can do large operations (as large as 128K for read or write in some cases) - and this is much more efficient for network transfer - but without mounting with forcedirectio I had limited my cifs_readpages to 16K (4 pages typically) - and because I do the SMBread synchronously I am severely limiting parallelism in the case of a single threaded app. So where I would like to get to is during readahead having multiple SMB reads for the same inode on the wire at one time - with the SMB reads each larger than a page (between 4 and 30 pages) - and was hoping that the aio and readv/writev support would make that easier.

I probably need to look more at the NFS direct i/o example to see if there are easy changes I can make to enable it on a per inode basis (rather than as only a mount option), and to double check what other filesystem do for returning errors on mmap and sendfile on inodes that are marked direct i/o.
-
To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

Signature · 3 lines
Suparna Bhattacharya ([email protected])
Linux Technology Center
IBM Software Lab, India
#2 lore raw
Quoted 11 lines · Suparna Bhattacharya

cc'ing linux-aio, for the AIO part of the discussion. You might be able to find some of your answers in the archives.

there are several tests for AIO - I tend to use Chris Mason's aio-stress which can be used to compare performance in terms of throughput for streaming reads/writes for different variations of options.

(the following page isn't exactly up-to-date, but should still give you some pointers: lse.sf.net/io/aio.html)

Thanks - those were lists that I was not aware of.

Quoted 4 lines
Quoted 3 lines

You had mentioned do_sync_read - is there a reason to change the current call to generic_file_read in the cifs read entry point to do_sync_read. Some filesystems which export aio routines still call generic_file_read and others call do_sync_read and it was not obvious to me what that would change.

Quoted 8 lines

I think you could keep it the way it is - generic_file_read will take care of things. But maybe I should comment only after I see your patch. Are you planning to post it some time ?

Regards
Suparna

This went in with the patch http://cifs.bkbits.net:8080/linux-2.5cifs/gnupatch@424470a3SsdVpix9tJE4NDebxqyRSg
and merged into mainline about six days ago (note that it is disabled by default - unless CONFIG_CIFS_EXPERIMENTAL is selected).