back channel flags, CREATE_SESSION, BIND_CONN_TO_SESSION
Subsystems (2)
A message that is not in the archive (sent off-list or not yet received): 1988930626.161.1317955756425.JavaMail.root@thunderbeast.private.linuxbox.com
Hi Trond,
I appreciate your point of view. I appreciate Rick's comments. I hope some other comments will trickle in. Do I think you and Rick are saying the same thing? Not really. Do I think you're making sense when you imply I want to change the standard? No, I don't follow that. BIND_CONN_TO_SESSION is in the published standard. Can I show a dedicated back channel improves performance? Not at present. That's a sensible question. I think if the protocol in general is doing what it is intended to do, it should be possible for some workloads, at some point. (If callbacks are, somehow generally detrimental to performance, as you state, I think maybe we have some more work to do.) (I note only one more thing--I did raise this topic on list 12 months ago. Only Bruce commented, at the time.)
Thanks again,
Matt
Signature · 11 lines
Matt Benjamin The Linux Box 206 South Fifth Ave. Suite 150 Ann Arbor, MI 48104 http://linuxbox.com tel. 734-761-4689 fax. 734-769-8938 cel. 734-216-5309
PiAtLS0tLU9yaWdpbmFsIE1lc3NhZ2UtLS0tLQ0KPiBGcm9tOiBNYXR0IFcuIEJlbmphbWluIFtt YWlsdG86bWF0dEBsaW51eGJveC5jb21dDQo+IFNlbnQ6IFRodXJzZGF5LCBPY3RvYmVyIDA2LCAy MDExIDEwOjU1IFBNDQo+IFRvOiBNeWtsZWJ1c3QsIFRyb25kDQo+IENjOiBsaW51eC1uZnM7IG5m cy1nYW5lc2hhLWRldmVsOyBuZnN2NA0KPiBTdWJqZWN0OiBSZTogW25mc3Y0XSBiYWNrIGNoYW5u ZWwgZmxhZ3MsIENSRUFURV9TRVNTSU9OLA0KPiBCSU5EX0NPTk5fVE9fU0VTU0lPTg0KPiANCj4g SGkgVHJvbmQsDQo+IA0KPiBJIGFwcHJlY2lhdGUgeW91ciBwb2ludCBvZiB2aWV3LiAgSSBhcHBy ZWNpYXRlIFJpY2sncyBjb21tZW50cy4gIEkgaG9wZSBzb21lDQo+IG90aGVyIGNvbW1lbnRzIHdp bGwgdHJpY2tsZSBpbi4gIERvIEkgdGhpbmsgeW91IGFuZCBSaWNrIGFyZSBzYXlpbmcgdGhlIHNh bWUNCj4gdGhpbmc/ICBOb3QgcmVhbGx5LiAgRG8gSSB0aGluayB5b3UncmUgbWFraW5nIHNlbnNl IHdoZW4geW91IGltcGx5IEkgd2FudCB0bw0KPiBjaGFuZ2UgdGhlIHN0YW5kYXJkPyAgTm8sIEkg ZG9uJ3QgZm9sbG93IHRoYXQuICBCSU5EX0NPTk5fVE9fU0VTU0lPTiBpcyBpbg0KPiB0aGUgcHVi bGlzaGVkIHN0YW5kYXJkLiAgQ2FuIEkgc2hvdyBhIGRlZGljYXRlZCBiYWNrIGNoYW5uZWwgaW1w cm92ZXMNCg0KSSBuZXZlciBhY2N1c2VkIHlvdSBvZiB0cnlpbmcgdG8gY2hhbmdlIHRoZSBzdGFu ZGFyZC4gV2hhdCBJIHNhaWQgd2FzIHRoYXQgeW91IG5lZWQgdG8gY29udmluY2UgbWUgdGhhdCB0 aGVyZSBpcyBhIGdvb2QgcmVhc29uIHRvIGFjY2VwdCB0d28gaW1wbGVtZW50YXRpb25zIG9mIHRo ZSBzYW1lIGZlYXR1cmUgaW4gdGhlIGNsaWVudC4NCg0KPiBwZXJmb3JtYW5jZT8gIE5vdCBhdCBw cmVzZW50LiAgVGhhdCdzIGEgc2Vuc2libGUgcXVlc3Rpb24uICBJIHRoaW5rIGlmIHRoZQ0KPiBw cm90b2NvbCBpbiBnZW5lcmFsIGlzIGRvaW5nIHdoYXQgaXQgaXMgaW50ZW5kZWQgdG8gZG8sIGl0 IHNob3VsZCBiZSBwb3NzaWJsZSBmb3INCj4gc29tZSB3b3JrbG9hZHMsIGF0IHNvbWUgcG9pbnQu ICAoSWYgY2FsbGJhY2tzIGFyZSwgc29tZWhvdyBnZW5lcmFsbHkNCj4gZGV0cmltZW50YWwgdG8g cGVyZm9ybWFuY2UsIGFzIHlvdSBzdGF0ZSwgSSB0aGluayBtYXliZSB3ZSBoYXZlIHNvbWUgbW9y ZQ0KPiB3b3JrIHRvIGRvLikgIChJIG5vdGUgb25seSBvbmUgbW9yZSB0aGluZy0tSSBkaWQgcmFp c2UgdGhpcyB0b3BpYyBvbiBsaXN0IDEyDQo+IG1vbnRocyBhZ28uICBPbmx5IEJydWNlIGNvbW1l bnRlZCwgYXQgdGhlIHRpbWUuKQ0KDQpGYWlyIGVub3VnaCwgYnV0IDEyIG1vbnRocyBhZ28sIHJm YzU2NjEgd2FzIGluIGxhc3QgY2FsbCBzdGF0dXMsIGFuZCB3ZSB3ZXJlIGFsbCB0cnlpbmcgdG8g bW92ZSBvbiB0byBmaW5hbCBpbXBsZW1lbnRhdGlvbnMuDQoNCkkgaGF2ZSBfYWx3YXlzXyBhcmd1 ZWQgdGhhdCBjYWxsYmFja3Mgc2hvdWxkIGJlIHRoZSBleGNlcHRpb24gYW5kIG5vdCB0aGUgcnVs ZSwgZHVlIHRvIHRoZSBmYWN0IHRoYXQgaW4gcHJldHR5IG11Y2ggYWxsIGNhc2VzIHRoZSBzZXJ2 ZXIgd2lsbCBoYXZlIHRvIHNlbmQgb3V0IGEgTkZTNEVSUl9ERUxBWSByZXN1bHQgdG8gd2hhdGV2 ZXIgUlBDIGNhbGwgaXQgaXMgdGhhdCB0cmlnZ2VyZWQgYSBjYWxsYmFjay4gTm90aGluZyBuZXcg dGhlcmUgaW4gTkZTdjQuMS4gSSdsbCBhZGQgdGhhdCBub3IgaGFzIGFueWJvZHkgc3VjY2Vzc2Z1 bGx5IGFyZ3VlZCB0aGF0IHRoZXJlIGlzIGFueSBjaGFuZ2UgaW4gTkZTdjQuMSBjb21wYXJlZCB0 byBvdXIgZXhpc3RpbmcgTkZTdjQgaW1wbGVtZW50YXRpb25zLiBORlN2NCBkZWxlZ2F0aW9uIGNh bGxiYWNrcyBhcmUgKG1lYXN1cmFibHkpIGV4dHJlbWVseSBzbG93LCBhbmQgc28gSSdkIGV4cGVj dCB0aGUgZXhhY3Qgc2FtZSBiZWhhdmlvciBmb3IgTkZTdjQuMSBsYXlvdXQgcmVjYWxscyBldGMu IE5vdGhpbmcgaGFzIGNoYW5nZWQgaW4gTkZTdjQgY2FsbGJhY2sgdGVjaG5vbG9neSB0byB3YXJy YW50IGFuIGV4cGVjdGF0aW9uIG9mIHBlcmZvcm1hbmNlIGltcHJvdmVtZW50cy4NCg0KSU9XOiBO b2JvZHkgKGluY2x1ZGluZyB5b3UpIGhhcyBldmVyIGNvbnZpbmNlZCBtZSB0aGF0IHdlIG5lZWQg dG8gY29uc2lkZXIgY2FsbGJhY2tzIGFzIGEgc2NhbGFiaWxpdHkgaXNzdWUuIEknZCByZXF1aXJl IHNvbWUgcHJldHR5IGZpcm0gZXZpZGVuY2UgYmVmb3JlIEkgY291bGQgYWNjZXB0IHRoYXQgYXMg YSBmYWN0LiBTbyBmYXIsIGFsbCBJIGhhdmUgc2VlbiBpcyBtb3JlIG9yIGxlc3MgZWxvcXVlbnQg cHJvc2UuIE5vIGhhcmQgbnVtYmVycy4NCg0KVHJvbmQNCg==
Quoted 1 line
I have _always_ argued that callbacks should be the exception and not the rule,
Sounds right.
Quoted 2 lines
due to the fact that in pretty much all cases the server will have to send out a
NFS4ERR_DELAYresult to whatever RPC call it is that triggered a callback.
I may disagree depending on what you mean by "petty much" always.
V4.1 does have callbacks that are specified so as to useful asynchronously, and if people take advantage of them, they won't have this NFS4ERR_DELAY problem.
Examples are CB_NOTIFY, CB_RECALL_ANY, CB_PUSH_DELEG. Over time, future versions may expand this list and make callbacks less exceptional, although not the rule.
-----Original Message-----
From: [email protected] [mailto:[email protected]] On Behalf Of Myklebust, Trond
Sent: Thursday, October 06, 2011 11:39 PM
To: Matt W. Benjamin
Cc: linux-nfs; nfs-ganesha-devel; nfsv4
Subject: Re: [nfsv4] back channel flags, CREATE_SESSION, BIND_CONN_TO_SESSION
Quoted 8 lines
-----Original Message-----
From: Matt W. Benjamin [mailto:[email protected]]
Sent: Thursday, October 06, 2011 10:55 PM
To: Myklebust, Trond
Cc: linux-nfs; nfs-ganesha-devel; nfsv4
Subject: Re: [nfsv4] back channel flags,CREATE_SESSION,BIND_CONN_TO_SESSION
Quoted 7 lines
Hi Trond,
I appreciate your point of view. I appreciate Rick's comments. I hope some other comments will trickle in. Do I think you and Rick are saying the same thing? Not really. Do I think you're making sense when you imply I want to change the standard? No, I don't follow that.
BIND_CONN_TO_SESSIONis in the published standard. Can I show a dedicated back channel improves
I never accused you of trying to change the standard. What I said was that you need to convince me that there is a good reason to accept two implementations of the same feature in the client.
Quoted 6 lines
performance? Not at present. That's a sensible question. I think if the protocol in general is doing what it is intended to do, it should be possible for some workloads, at some point. (If callbacks are, somehow generally detrimental to performance, as you state, I think maybe we have some more work to do.) (I note only one more thing--I did raise this topic on list 12 months ago. Only Bruce commented, at the time.)
Fair enough, but 12 months ago, rfc5661 was in last call status, and we were all trying to move on to final implementations.
I have _always_ argued that callbacks should be the exception and not the rule, due to the fact that in pretty much all cases the server will have to send out a NFS4ERR_DELAY result to whatever RPC call it is that triggered a callback. Nothing new there in NFSv4.1. I'll add that nor has anybody successfully argued that there is any change in NFSv4.1 compared to our existing NFSv4 implementations. NFSv4 delegation callbacks are (measurably) extremely slow, and so I'd expect the exact same behavior for NFSv4.1 layout recalls etc. Nothing has changed in NFSv4 callback technology to warrant an expectation of performance improvements.
IOW: Nobody (including you) has ever convinced me that we need to consider callbacks as a scalability issue. I'd require some pretty firm evidence before I could accept that as a fact. So far, all I have seen is more or less eloquent prose. No hard numbers.
Trond
_______________________________________________
nfsv4 mailing list
[email protected]
https://www.ietf.org/mailman/listinfo/nfsv4
Quoted 7 lines · On Tue, 2011-10-18 at 17:28 -0400, [email protected] wrote:
Quoted 1 line
I have _always_ argued that callbacks should be the exception and not the rule,
Sounds right.
Quoted 2 lines
due to the fact that in pretty much all cases the server will have to send out a
NFS4ERR_DELAYresult to whatever RPC call it is that triggered a callback.
Quoted 7 lines
I may disagree depending on what you mean by "petty much" always.
V4.1 does have callbacks that are specified so as to useful asynchronously, and if people take advantage of them, they won't have this
NFS4ERR_DELAYproblem.Examples are
CB_NOTIFY,CB_RECALL_ANY,CB_PUSH_DELEG. Over time, future versions may expand this list and make callbacks less exceptional, although not the rule.
I'm not sure why you included CB_RECALL_ANY in the above list. Did you mean CB_RECALLABLE_OBJ_AVAIL?
The main consumer of callbacks in NFSv4.1 is expected by some to be layouts, which do not really have an asynchronous mode to notify you when a particular layout has been made available.
Ditto for the NFSv4.0 number 1 consumer: delegations. The new CB_PUSH_DELEG operation notifies you that you may now request a delegation, but it doesn't help when you are just trying to OPEN a file, and the server has to recall a delegation from someone else.
So I still think that NFS4ERR_DELAY will still be needed in order to slow things down, unless we're careful about getting into these situations where recalls are needed.
Quoted 48 lines
-----Original Message-----
From: [email protected] [mailto:[email protected]] On Behalf Of Myklebust, Trond
Sent: Thursday, October 06, 2011 11:39 PM
To: Matt W. Benjamin
Cc: linux-nfs; nfs-ganesha-devel; nfsv4
Subject: Re: [nfsv4] back channel flags,CREATE_SESSION,BIND_CONN_TO_SESSIONQuoted 8 lines
-----Original Message-----
From: Matt W. Benjamin [mailto:[email protected]]
Sent: Thursday, October 06, 2011 10:55 PM
To: Myklebust, Trond
Cc: linux-nfs; nfs-ganesha-devel; nfsv4
Subject: Re: [nfsv4] back channel flags,CREATE_SESSION,BIND_CONN_TO_SESSIONQuoted 7 lines
Hi Trond,
I appreciate your point of view. I appreciate Rick's comments. I hope some other comments will trickle in. Do I think you and Rick are saying the same thing? Not really. Do I think you're making sense when you imply I want to change the standard? No, I don't follow that.
BIND_CONN_TO_SESSIONis in the published standard. Can I show a dedicated back channel improvesI never accused you of trying to change the standard. What I said was that you need to convince me that there is a good reason to accept two implementations of the same feature in the client.
Quoted 6 lines
performance? Not at present. That's a sensible question. I think if the protocol in general is doing what it is intended to do, it should be possible for some workloads, at some point. (If callbacks are, somehow generally detrimental to performance, as you state, I think maybe we have some more work to do.) (I note only one more thing--I did raise this topic on list 12 months ago. Only Bruce commented, at the time.)
Fair enough, but 12 months ago, rfc5661 was in last call status, and we were all trying to move on to final implementations.
I have _always_ argued that callbacks should be the exception and not the rule, due to the fact that in pretty much all cases the server will have to send out a
NFS4ERR_DELAYresult to whatever RPC call it is that triggered a callback. Nothing new there in NFSv4.1. I'll add that nor has anybody successfully argued that there is any change in NFSv4.1 compared to our existing NFSv4 implementations. NFSv4 delegation callbacks are (measurably) extremely slow, and so I'd expect the exact same behavior for NFSv4.1 layout recalls etc. Nothing has changed in NFSv4 callback technology to warrant an expectation of performance improvements.IOW: Nobody (including you) has ever convinced me that we need to consider callbacks as a scalability issue. I'd require some pretty firm evidence before I could accept that as a fact. So far, all I have seen is more or less eloquent prose. No hard numbers.
Trond
_______________________________________________
nfsv4 mailing list
[email protected]
https://www.ietf.org/mailman/listinfo/nfsv4_______________________________________________
nfsv4 mailing list
[email protected]
https://www.ietf.org/mailman/listinfo/nfsv4
Signature · 6 lines
Trond Myklebust Linux NFS client maintainer NetApp [email protected] www.netapp.com
I meant CB_RECALL_ANY. The server can use this to make sure that it is Likely to have an extra recallable object available in hand. If he indicates the desire early, than the interaction is asynchronous, and so the first request requesting the layout/delegation get that one and you start another asynchronous operation to have another given back.
In some cases, not having space does not man DELAY but rather not getting the delegation you want, with that being provided to the client by CB_RECALL_OBJ_AVAIL.
This sort of interaction suggests to me that requests and callbacks may wind up being more connected than we have thought before and that we might address this in some v4.x for x >= 3.
-----Original Message-----
From: Trond Myklebust [mailto:[email protected]]
Sent: Tuesday, October 18, 2011 6:39 PM
To: Noveck, David
Cc: [email protected]; [email protected]; [email protected]; [email protected]
Subject: Re: [nfsv4] back channel flags, CREATE_SESSION, BIND_CONN_TO_SESSION
Quoted 7 lines · On Tue, 2011-10-18 at 17:28 -0400, [email protected] wrote:
Quoted 1 line
I have _always_ argued that callbacks should be the exception and not the rule,
Sounds right.
Quoted 2 lines
due to the fact that in pretty much all cases the server will have to send out a
NFS4ERR_DELAYresult to whatever RPC call it is that triggered a callback.
Quoted 7 lines
I may disagree depending on what you mean by "petty much" always.
V4.1 does have callbacks that are specified so as to useful asynchronously, and if people take advantage of them, they won't have this
NFS4ERR_DELAYproblem.Examples are
CB_NOTIFY,CB_RECALL_ANY,CB_PUSH_DELEG. Over time, future versions may expand this list and make callbacks less exceptional, although not the rule.
I'm not sure why you included CB_RECALL_ANY in the above list. Did you mean CB_RECALLABLE_OBJ_AVAIL?
The main consumer of callbacks in NFSv4.1 is expected by some to be layouts, which do not really have an asynchronous mode to notify you when a particular layout has been made available.
Ditto for the NFSv4.0 number 1 consumer: delegations. The new CB_PUSH_DELEG operation notifies you that you may now request a delegation, but it doesn't help when you are just trying to OPEN a file, and the server has to recall a delegation from someone else.
So I still think that NFS4ERR_DELAY will still be needed in order to slow things down, unless we're careful about getting into these situations where recalls are needed.
Quoted 48 lines
-----Original Message-----
From: [email protected] [mailto:[email protected]] On Behalf Of Myklebust, Trond
Sent: Thursday, October 06, 2011 11:39 PM
To: Matt W. Benjamin
Cc: linux-nfs; nfs-ganesha-devel; nfsv4
Subject: Re: [nfsv4] back channel flags,CREATE_SESSION,BIND_CONN_TO_SESSIONQuoted 8 lines
-----Original Message-----
From: Matt W. Benjamin [mailto:[email protected]]
Sent: Thursday, October 06, 2011 10:55 PM
To: Myklebust, Trond
Cc: linux-nfs; nfs-ganesha-devel; nfsv4
Subject: Re: [nfsv4] back channel flags,CREATE_SESSION,BIND_CONN_TO_SESSIONQuoted 7 lines
Hi Trond,
I appreciate your point of view. I appreciate Rick's comments. I hope some other comments will trickle in. Do I think you and Rick are saying the same thing? Not really. Do I think you're making sense when you imply I want to change the standard? No, I don't follow that.
BIND_CONN_TO_SESSIONis in the published standard. Can I show a dedicated back channel improvesI never accused you of trying to change the standard. What I said was that you need to convince me that there is a good reason to accept two implementations of the same feature in the client.
Quoted 6 lines
performance? Not at present. That's a sensible question. I think if the protocol in general is doing what it is intended to do, it should be possible for some workloads, at some point. (If callbacks are, somehow generally detrimental to performance, as you state, I think maybe we have some more work to do.) (I note only one more thing--I did raise this topic on list 12 months ago. Only Bruce commented, at the time.)
Fair enough, but 12 months ago, rfc5661 was in last call status, and we were all trying to move on to final implementations.
I have _always_ argued that callbacks should be the exception and not the rule, due to the fact that in pretty much all cases the server will have to send out a
NFS4ERR_DELAYresult to whatever RPC call it is that triggered a callback. Nothing new there in NFSv4.1. I'll add that nor has anybody successfully argued that there is any change in NFSv4.1 compared to our existing NFSv4 implementations. NFSv4 delegation callbacks are (measurably) extremely slow, and so I'd expect the exact same behavior for NFSv4.1 layout recalls etc. Nothing has changed in NFSv4 callback technology to warrant an expectation of performance improvements.IOW: Nobody (including you) has ever convinced me that we need to consider callbacks as a scalability issue. I'd require some pretty firm evidence before I could accept that as a fact. So far, all I have seen is more or less eloquent prose. No hard numbers.
Trond
_______________________________________________
nfsv4 mailing list
[email protected]
https://www.ietf.org/mailman/listinfo/nfsv4_______________________________________________
nfsv4 mailing list
[email protected]
https://www.ietf.org/mailman/listinfo/nfsv4
Signature · 6 lines
Trond Myklebust Linux NFS client maintainer NetApp [email protected] www.netapp.com