Implementing NFS v4.2
Subsystems (2)
Hello all,
I'm interested in implementing the draft specification for NFS v4.2 as a Google Summer of Code project. That includes server-side copying, sparse files, and carrying fadvise() calls through to the server, among other things.
The current document can be found here:
http://tools.ietf.org/html/draft-ietf-nfsv4-minorversion2-07
Is this something that you need to be done? If so, I'd very much like to be involved. :)
Thanks for your time,
Calvin Owens
T24gVGh1LCAyMDEyLTA0LTA1IGF0IDE5OjAzIC0wNTAwLCBDYWx2aW4gT3dlbnMgd3JvdGU6DQo+ IEhlbGxvIGFsbCwNCj4gDQo+IEknbSBpbnRlcmVzdGVkIGluIGltcGxlbWVudGluZyB0aGUgZHJh ZnQgc3BlY2lmaWNhdGlvbiBmb3IgTkZTIHY0LjIgYXMgYSANCj4gR29vZ2xlIFN1bW1lciBvZiBD b2RlIHByb2plY3QuIFRoYXQgaW5jbHVkZXMgc2VydmVyLXNpZGUgY29weWluZywgc3BhcnNlIA0K PiBmaWxlcywgYW5kIGNhcnJ5aW5nIGZhZHZpc2UoKSBjYWxscyB0aHJvdWdoIHRvIHRoZSBzZXJ2 ZXIsIGFtb25nIG90aGVyIA0KPiB0aGluZ3MuDQo+IA0KPiBUaGUgY3VycmVudCBkb2N1bWVudCBj YW4gYmUgZm91bmQgaGVyZToNCj4gaHR0cDovL3Rvb2xzLmlldGYub3JnL2h0bWwvZHJhZnQtaWV0 Zi1uZnN2NC1taW5vcnZlcnNpb24yLTA3DQo+IA0KPiBJcyB0aGlzIHNvbWV0aGluZyB0aGF0IHlv dSBuZWVkIHRvIGJlIGRvbmU/IElmIHNvLCBJJ2QgdmVyeSBtdWNoIGxpa2UgdG8gDQo+IGJlIGlu dm9sdmVkLiA6KQ0KDQpIaSBDYWx2aW4sDQoNCkxhYmVsbGVkIE5GUyBpcyBsaWtlbHkgdG8gYmUg bWVyZ2VkIGludG8gMy41IChpZiBEYXZlIFEgZmluZHMgdGhlIHRpbWUNCnRvIHBvcnQgaGlzIGV4 aXN0aW5nIGNvZGUpLg0KDQpDb3B5IG9mZmxvYWQgYWxyZWFkeSBleGlzdHMgaW4gcHJvdG90eXBl IGZvcm0uIFRoZSBtYWluIHJlbWFpbmluZyBpc3N1ZQ0KaXMgd29ya2luZyBvdXQgdGhlIHVzZXIg c3lzY2FsbCBpbnRlcmZhY2UsIHdoaWNoIHJlYWxseSByZXF1aXJlcyBnZXR0aW5nDQphbGwgdGhl IGludGVyZXN0ZWQgZmlsZXN5c3RlbSBtYWludGFpbmVycyB0byBhZ3JlZSAod2UndmUgc3RhcnRl ZCBvbg0KZG9pbmcgdGhhdCkuDQoNCklmIHlvdSdkIGxpa2UgdG8gY29udHJpYnV0ZSwgdGhlbiBJ J2Qgc3VnZ2VzdCBsb29raW5nIGludG8gU0VFSyAoZm9yDQpwcm92aWRpbmcgbHNlZWsoU0VFS19I T0xFL1NFRUtfREFUQSkgc3VwcG9ydC4gVGhlcmUgaXMgYWxzbyB0aGUgaG9sZQ0KcHVuY2hpbmcv c3BhY2UgcmVzZXJ2YXRpb24sIHRoYXQgc2hvdWxkIGZpdCBuaWNlbHkgaW50byB0aGUgZmFsbG9j YXRlKCkNCnN5c3RlbSBjYWxsLg0KVGhlIGVmZmljaWVudCBzcGFyc2UgZmlsZSByZWFkIGFuZCBm YWR2aXNlIHN1cHBvcnQgbWlnaHQgYmUgbmljZSB0b28sDQpidXQgSSdkIGxpa2UgdG8gc2VlIG51 bWJlcnMgZm9yIGhvdyB0aGV5IGltcHJvdmUgbWF0dGVycyBiZWZvcmUgSSBmZWVsDQpjb21mb3J0 YWJsZSBzYXlpbmcgeWVhIG9yIG5heSB0byBhZGRpbmcgdGhvc2Ugc3BlY2lmaWMgZmVhdHVyZXMu DQoNCk5vdGUgdGhhdCB0aGVyZSBhcmUgYWxzbyBhIGJ1bmNoIG9mIE5GU3Y0LjEgZmVhdHVyZXMg dGhhdCBoYXZlIHlldCB0byBiZQ0KaW1wbGVtZW50ZWQsIHNvIHRoZSBhYm92ZSBsaXN0IG9mIHRh c2tzIGlzIG5vdCBleGhhdXN0aXZlLiBJJ2QgYmUgaGFwcHkNCnRvIHdvcmsgd2l0aCB5b3UgdG8g ZmluZCBzb21ldGhpbmcuLi4NCg0KQ2hlZXJzDQogIFRyb25kDQotLSANClRyb25kIE15a2xlYnVz dA0KTGludXggTkZTIGNsaWVudCBtYWludGFpbmVyDQoNCk5ldEFwcA0KVHJvbmQuTXlrbGVidXN0 QG5ldGFwcC5jb20NCnd3dy5uZXRhcHAuY29tDQoNCg==
Quoted 24 lines · Trond
Quoted 7 lines · Calvin Owens
Hello all,
I'm interested in implementing the draft specification for NFS v4.2 as a Google Summer of Code project. That includes server-side copying, sparse files, and carrying
fadvise()calls through to the server, among other things.Quoted 5 lines
The current document can be found here:
http://tools.ietf.org/html/draft-ietf-nfsv4-minorversion2-07Is this something that you need to be done? If so, I'd very much like to be involved. :)
Hi Calvin,
Labelled NFS is likely to be merged into 3.5 (if Dave Q finds the time to port his existing code).
Copy offload already exists in prototype form. The main remaining issue is working out the user syscall interface, which really requires getting all the interested filesystem maintainers to agree (we've started on doing that).
Quoted 4 lines
If you'd like to contribute, then I'd suggest looking into SEEK (for providing
lseek(SEEK_HOLE/SEEK_DATA)support. There is also the hole punching/space reservation, that should fit nicely into thefallocate()system call.
Yes, that sounds good. I'll start looking into that.
Quoted 3 lines
The efficient sparse file read and fadvise support might be nice too, but I'd like to see numbers for how they improve matters before I feel comfortable saying yea or nay to adding those specific features.
I definitely see your point. It does seem to be a lot of added complexity for a negligible benefit.
Quoted 3 lines
Note that there are also a bunch of NFSv4.1 features that have yet to be implemented, so the above list of tasks is not exhaustive. I'd be happy to work with you to find something...
What else, precisely, would you like to get done? I need to get some more detailed objectives to put into my application. (Forgive me for so blatantly not looking myself, but it seems more efficient than me digging through the code finding missing features and lobbing them at you for approval, when I'm sure you know exactly where your priorities are...)
Thanks very much,
Calvin Owens
Quoted 50 lines · Calvin Owens
Quoted 24 lines · Trond
Quoted 7 lines · Calvin Owens
Hello all,
I'm interested in implementing the draft specification for NFS v4.2 as a Google Summer of Code project. That includes server-side copying, sparse files, and carrying
fadvise()calls through to the server, among other things.Quoted 5 lines
The current document can be found here:
http://tools.ietf.org/html/draft-ietf-nfsv4-minorversion2-07Is this something that you need to be done? If so, I'd very much like to be involved. :)
Hi Calvin,
Labelled NFS is likely to be merged into 3.5 (if Dave Q finds the time to port his existing code).
Copy offload already exists in prototype form. The main remaining issue is working out the user syscall interface, which really requires getting all the interested filesystem maintainers to agree (we've started on doing that).
Quoted 4 lines
If you'd like to contribute, then I'd suggest looking into SEEK (for providing
lseek(SEEK_HOLE/SEEK_DATA)support. There is also the hole punching/space reservation, that should fit nicely into thefallocate()system call.Yes, that sounds good. I'll start looking into that.
Quoted 3 lines
The efficient sparse file read and fadvise support might be nice too, but I'd like to see numbers for how they improve matters before I feel comfortable saying yea or nay to adding those specific features.
I definitely see your point. It does seem to be a lot of added complexity for a negligible benefit.
Quoted 3 lines
Note that there are also a bunch of NFSv4.1 features that have yet to be implemented, so the above list of tasks is not exhaustive. I'd be happy to work with you to find something...
What else, precisely, would you like to get done? I need to get some more detailed objectives to put into my application. (Forgive me for so blatantly not looking myself, but it seems more efficient than me digging through the code finding missing features and lobbing them at you for approval, when I'm sure you know exactly where your priorities are...)
Quoted 6 lines
Thanks very much,
Calvin Owens
--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Finishing up the port of Labeled NFS would be some low hanging fruit if someone wants to help tackle it. I'll be trying to find time to work on it but its unclear how soon I'll be able to get around to it.
Dave
Quoted 61 lines · David Quigley
Quoted 53 lines · Calvin Owens
Quoted 27 lines · Trond
Quoted 9 lines · Calvin Owens
Hello all,
I'm interested in implementing the draft specification for NFS v4.2 as a
Google Summer of Code project. That includes server-side copying, sparse
files, and carryingfadvise()calls through to the server, among other things.Quoted 6 lines
The current document can be found here:
http://tools.ietf.org/html/draft-ietf-nfsv4-minorversion2-07Is this something that you need to be done? If so, I'd very much like to
be involved. :)Hi Calvin,
Labelled NFS is likely to be merged into 3.5 (if Dave Q finds the time to port his existing code).
Copy offload already exists in prototype form. The main remaining issue is working out the user syscall interface, which really requires getting all the interested filesystem maintainers to agree (we've started on doing that).
Quoted 4 lines
If you'd like to contribute, then I'd suggest looking into SEEK (for providing
lseek(SEEK_HOLE/SEEK_DATA)support. There is also the hole punching/space reservation, that should fit nicely into thefallocate()system call.Yes, that sounds good. I'll start looking into that.
Quoted 3 lines
The efficient sparse file read and fadvise support might be nice too, but I'd like to see numbers for how they improve matters before I feel comfortable saying yea or nay to adding those specific features.
I definitely see your point. It does seem to be a lot of added complexity for a negligible benefit.
Quoted 3 lines
Note that there are also a bunch of NFSv4.1 features that have yet to be implemented, so the above list of tasks is not exhaustive. I'd be happy to work with you to find something...
What else, precisely, would you like to get done? I need to get some more detailed objectives to put into my application. (Forgive me for so blatantly not looking myself, but it seems more efficient than me digging through the code finding missing features and lobbing them at you for approval, when I'm sure you know exactly where your priorities are...)
Quoted 6 lines
Thanks very much,
Calvin Owens
--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Quoted 5 lines
Finishing up the port of Labeled NFS would be some low hanging fruit if someone wants to help tackle it. I'll be trying to find time to work on it but its unclear how soon I'll be able to get around to it.
Dave
Okay. I took a look at your lnfs git tree - just to make sure I understand: the patchset was originally against 2.6.33, and you're working to make it apply to the current kernel, with the same functionality? And stuff has changed enough since then that it's not trivial? Or am I completely missing the point?
Thanks,
Calvin Owens
Quoted 76 lines · Calvin Owens
Quoted 63 lines · David Quigley
Quoted 55 lines · Calvin Owens
Quoted 28 lines · Trond
Quoted 9 lines · Calvin Owens
Hello all,
I'm interested in implementing the draft specification for NFS v4.2 as a
Google Summer of Code project. That includes server-side copying, sparse
files, and carryingfadvise()calls through to the server, among other things.Quoted 6 lines
The current document can be found here:
http://tools.ietf.org/html/draft-ietf-nfsv4-minorversion2-07Is this something that you need to be done? If so, I'd very much like to
be involved. :)Hi Calvin,
Labelled NFS is likely to be merged into 3.5 (if Dave Q finds the time to port his existing code).
Copy offload already exists in prototype form. The main remaining issue is working out the user syscall interface, which really requires getting
all the interested filesystem maintainers to agree (we've started on doing that).Quoted 4 lines
If you'd like to contribute, then I'd suggest looking into SEEK (for providing
lseek(SEEK_HOLE/SEEK_DATA)support. There is also the hole punching/space reservation, that should fit nicely into thefallocate()system call.Yes, that sounds good. I'll start looking into that.
Quoted 3 lines
The efficient sparse file read and fadvise support might be nice too, but I'd like to see numbers for how they improve matters before I feel comfortable saying yea or nay to adding those specific features.
I definitely see your point. It does seem to be a lot of added complexity for a negligible benefit.
Quoted 4 lines
Note that there are also a bunch of NFSv4.1 features that have yet to be
implemented, so the above list of tasks is not exhaustive. I'd be happy to work with you to find something...What else, precisely, would you like to get done? I need to get some more detailed objectives to put into my application. (Forgive me for so blatantly not looking myself, but it seems more efficient than me digging through the code finding missing features and lobbing them at you for approval, when I'm sure you know exactly where your priorities are...)
Quoted 6 lines
Thanks very much,
Calvin Owens
--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.htmlQuoted 5 lines
Finishing up the port of Labeled NFS would be some low hanging fruit if someone wants to help tackle it. I'll be trying to find time to work on it but its unclear how soon I'll be able to get around to it.
Dave
Okay. I took a look at your lnfs git tree - just to make sure I
understand: the patchset was originally against 2.6.33, and you're working to make it apply to the current kernel, with the same functionality? And stuff has changed enough since then that it's not trivial? Or am I completely missing the point?
Quoted 7 lines
Thanks,
Calvin Owens
--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Sorry I somehow completely missed this email. The latest tree should be against 3.2 or something like that. Currently there is an issue where I couldn't port one of the parts over in lookup (see a previous email I sent to trond on list for the details). Once that issue is fixed I would assume the forward port to 3.5 would be mostly trivial.
The Labeled NFS tree is managed as two git trees for people's convenience. The tree listed first below [1] is the guilt patch set which I keep track of. The second tree is a git tree which has had the patches applied to it[2]. The tree currently has a v3.2-lnfs branch which needs the one fix figured out. The way I do development is to check out the kernel tag and the patch-set and just work from there.
Another issue is that currently, SELinux is really the only thing you can use to test Labeled NFS. I have a setup script which should give a small test environment to mess around with as well. If you're not familiar with SELinux I can help you with that part as well.
If you find this interesting and want to try to fix the issue and do the forward port I'd be glad to help you. It would be nice to get these changes upstream.
[1] http://git.selinuxproject.org/git/?p=users/dpquigl/lnfs-patchset/.git;a=summary
[2] http://git.selinuxproject.org/git/?p=users/dpquigl/lnfs/.git;a=summary
Dave
Quoted 6 lines
The efficient sparse file read and fadvise support might be nice too, but I'd like to see numbers for how they improve matters before I feel comfortable saying yea or nay to adding those specific features.
Instead of '...but I'd like to see...' I assume you meant, '...since I'd like to see...', as it would be hard to see how they improve matters without actually implementing them.
Although for sparse file reads, I think the low overhead design to avoid bloating every file upon read in NFSv4.2 makes it easy to see the benefit without any implementation. For ioadvise, I think the benefit will be dependent upon support from the exported file system (or possibly nfs server). So any prototype would have to be combined with appropriate support in the server file system or nfs server.
Dean
Quoted 7 lines · Dean
Quoted 6 lines
The efficient sparse file read and fadvise support might be nice too, but I'd like to see numbers for how they improve matters before I feel comfortable saying yea or nay to adding those specific features.
Quoted 5 lines
Instead of '...but I'd like to see...' I assume you meant, '...since I'd like to see...', as it would be hard to see how they improve matters without actually implementing them.
Although for sparse file reads, I think the low overhead design to avoid bloating every file upon read in NFSv4.2 makes it easy to see the benefit without any implementation. For ioadvise, I think the benefit will be dependent upon support from the exported file system (or possibly nfs server). So any prototype would have to be combined with appropriate support in the server file system or nfs server.
Dean
Well, that would make my project a lot more straightforward: if you guys want the ADB sparse file reads, I could just call it "Implementing Sparse File Support for NFS", and implement sections 3, 4, and 6 of the v4.2 internet-draft. That seems like enough.
fadvise() has the same behaviour for all filesystems, doesn't it? If wanting more efficient VM virtual disks over NFS is driving the sparse file stuff, I can see how POSIX_FADV_RANDOM on the server end would help with that.
Calvin
Quoted 91 lines · Dave Quigley
Quoted 70 lines · Calvin Owens
Quoted 63 lines · David Quigley
Quoted 55 lines · Calvin Owens
Quoted 28 lines · Trond
Quoted 9 lines · Calvin Owens
Hello all,
I'm interested in implementing the draft specification for NFS v4.2 as a
Google Summer of Code project. That includes server-side copying, sparse
files, and carryingfadvise()calls through to the server, among other things.Quoted 6 lines
The current document can be found here:
http://tools.ietf.org/html/draft-ietf-nfsv4-minorversion2-07Is this something that you need to be done? If so, I'd very much like to
be involved. :)Hi Calvin,
Labelled NFS is likely to be merged into 3.5 (if Dave Q finds the time to port his existing code).
Copy offload already exists in prototype form. The main remaining issue is working out the user syscall interface, which really requires getting
all the interested filesystem maintainers to agree (we've started on doing that).Quoted 4 lines
If you'd like to contribute, then I'd suggest looking into SEEK (for providing
lseek(SEEK_HOLE/SEEK_DATA)support. There is also the hole punching/space reservation, that should fit nicely into thefallocate()system call.Yes, that sounds good. I'll start looking into that.
Quoted 3 lines
The efficient sparse file read and fadvise support might be nice too, but I'd like to see numbers for how they improve matters before I feel comfortable saying yea or nay to adding those specific features.
I definitely see your point. It does seem to be a lot of added complexity for a negligible benefit.
Quoted 4 lines
Note that there are also a bunch of NFSv4.1 features that have yet to be
implemented, so the above list of tasks is not exhaustive. I'd be happy to work with you to find something...What else, precisely, would you like to get done? I need to get some more detailed objectives to put into my application. (Forgive me for so blatantly not looking myself, but it seems more efficient than me digging through the code finding missing features and lobbing them at you for approval, when I'm sure you know exactly where your priorities are...)
Quoted 6 lines
Thanks very much,
Calvin OwensSignature · 3 lines
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.htmlQuoted 5 lines
Finishing up the port of Labeled NFS would be some low hanging fruit if someone wants to help tackle it. I'll be trying to find time to work on it but its unclear how soon I'll be able to get around to it.
Dave
Quoted 13 lines
Okay. I took a look at your lnfs git tree - just to make sure I
understand: the patchset was originally against 2.6.33, and you're working to make it apply to the current kernel, with the same functionality? And stuff has changed enough since then that it's not trivial? Or am I completely missing the point?Thanks,
Calvin OwensSignature · 3 lines
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.htmlSorry I somehow completely missed this email. The latest tree should be against 3.2 or something like that. Currently there is an issue where I couldn't port one of the parts over in lookup (see a previous email I sent to trond on list for the details). Once that issue is fixed I would assume the forward port to 3.5 would be mostly trivial.
The Labeled NFS tree is managed as two git trees for people's convenience. The tree listed first below [1] is the guilt patch set which I keep track of. The second tree is a git tree which has had the patches applied to it[2]. The tree currently has a v3.2-lnfs branch which needs the one fix figured out. The way I do development is to check out the kernel tag and the patch-set and just work from there.
Another issue is that currently, SELinux is really the only thing you can use to test Labeled NFS. I have a setup script which should give a small test environment to mess around with as well. If you're not familiar with SELinux I can help you with that part as well.
Quoted 7 lines
If you find this interesting and want to try to fix the issue and do the forward port I'd be glad to help you. It would be nice to get these changes upstream.
[1] http://git.selinuxproject.org/git/?p=users/dpquigl/lnfs-patchset/.git;a=summary
[2] http://git.selinuxproject.org/git/?p=users/dpquigl/lnfs/.git;a=summaryDave
Well, after spending some time looking at it, I think it's probably better left to somebody more familiar with SELinux... I have absolutely no experience with it whatsoever.
Thanks very much for the info though.
Calvin