Copy offload
Quite a lot of discussion has come up recently on supporting Copy Offload (aka SCSI XCOPY) with linux.
During the course of this several topics were found which need some discussion:
- Interface: do we need a new syscall? Should we try to resurrect the original sys_copyfile() approach from Joel Becker? What are the areas and use-cases this syscall should cover?
- Scope: SCSI XCOPY is not the only possible user here, CIFS and NFSv4 have similar needs. We should aim for integrating all of these use-cases. However, we need to revisit them to figure out if and to what extend they are really compatible.
- Implementation: The new interface is likely to reside on the filesystem level. To quote Dave Chinner:
Quoted 6 lines
e.g. for an array offload, we have to flush the source file page cache first so that the data being copied is known to be on disk, then invalidate the destination page cache if overwriting or extend and pre-allocate blocks if not. Then we have to map both files and hand that off to the array.
Quoted 6 lines
Then there's a whole bunch of tricky questions about what the state of the destination file should look like while the copy is in progress, whether the source file should be allowed to change (e.g. it can't be truncated and have blocks freed and then reused by other files half way through the copy offload operation), and so on.
- Backends: Should we concentrate on the new 'XCOPY LITE' proposal or should we try to implement the original XCOPY command, too?
I guess this'll warrant a joint session, as at least filesystem and storage people will be involved here.
Cheers,
Hannes
Signature · 8 lines
Dr. Hannes Reinecke zSeries & Storage [email protected] +49 911 74053 688 SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg GF: J. Hawn, J. Guild, F. Imendörffer, HRB 16746 (AG Nürnberg) -- To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
Quoted 30 lines · Hannes Reinecke
Quite a lot of discussion has come up recently on supporting Copy Offload (aka SCSI XCOPY) with linux.
During the course of this several topics were found which need some discussion:
- Interface: do we need a new syscall? Should we try to resurrect the original
sys_copyfile()approach from Joel Becker? What are the areas and use-cases this syscall should cover?- Scope: SCSI XCOPY is not the only possible user here, CIFS and NFSv4 have similar needs. We should aim for integrating all of these use-cases. However, we need to revisit them to figure out if and to what extend they are really compatible.
- Implementation: The new interface is likely to reside on the filesystem level. To quote Dave Chinner:
Quoted 6 lines
e.g. for an array offload, we have to flush the source file page cache first so that the data being copied is known to be on disk, then invalidate the destination page cache if overwriting or extend and pre-allocate blocks if not. Then we have to map both files and hand that off to the array.
Quoted 6 lines
Then there's a whole bunch of tricky questions about what the state of the destination file should look like while the copy is in progress, whether the source file should be allowed to change (e.g. it can't be truncated and have blocks freed and then reused by other files half way through the copy offload operation), and so on.
Quoted 5 lines
- Backends: Should we concentrate on the new 'XCOPY LITE' proposal or should we try to implement the original XCOPY command, too?
I guess this'll warrant a joint session, as at least filesystem and storage people will be involved here.
I'll join in for sure. I've owed code for over a year now
(hanging head in shame).
Joel
Signature · 7 lines
"Lately I've been talking in my sleep. Can't imagine what I'd have to say. Except my world will be right When love comes back my way." http://www.jlbec.org/ [email protected]
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
"Hannes" == Hannes Reinecke <[email protected]> writes:
Hannes> - Backends: Should we concentrate on the new 'XCOPY LITE'
Hannes> proposal or should we try to implement the original XCOPY
Hannes> command, too?
I know of very few vendors interested in implementing the original XCOPY stuff. Only the standards people seem to be excited about it. It's way too complex, IMHO.
XCOPY lite is a much better fit for how we actually do I/O. So that's what I'm tracking closely. As we talked about in Prague, I've dabbled a bit with the design from an I/O stack perspective and it fits nicely.
My gut feeling is that XCOPYv2 might get ratified but companies will actually end up implementing XCOPY lite. Regardless of whether the lite proposal gets into the spec or not.
T10 appears to be designing a lot of stuff right now that's an extremely poor fit for how modern operating systems work. I think we may see a bigger divergence between what's in the spec and what companies choose to implement.
Signature · 1 line
Martin K. Petersen Oracle Linux Engineering
In general, with my target vendor hat on, I'd be very interested in what special SCSI commands the Linux initiator stack might want to use. Extended copy is one (very complex) case -- and BTW Microsoft seems to plan on using XCOPY LITE with NTFS.
Another possibly interesting example would be COMPARE AND WRITE for clustered filesystems.
Any other storage operations that FS developers might want from the block device? (Either standard SCSI commands or more speculative stuff that T10 hasn't thought of yet)
- R.
Quoted 5 lines · Roland Dreier
In general, with my target vendor hat on, I'd be very interested in what special SCSI commands the Linux initiator stack might want to use. Extended copy is one (very complex) case -- and BTW Microsoft seems to plan on using XCOPY LITE with NTFS.
Quoted 7 lines
Another possibly interesting example would be COMPARE AND WRITE for clustered filesystems.
Any other storage operations that FS developers might want from the block device? (Either standard SCSI commands or more speculative stuff that T10 hasn't thought of yet)
copy-and-modify
the command looks like a a write with destination [write_start, num_sectors] The command also receives a read destination [read_start, num_sectors]
where read_start <= write_start && write_start+num_sectors <= read_start+num_sectors
That's for the like of pNFS blocks layout where the FS maintains a copy-on-write with FS blocks larger than a sector. Today it needs to do the copy then the modification. (2 + 1) operations instead of 1
Na, too marginal. Is it worth it? It might depend on the FS block size.
Just some crazy idea
Boaz
Quoted 5 lines
- R.
--
To unsubscribe from this list: send the line "unsubscribe linux-scsi" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html