Skip to contentSkip to conversation

pNFS layout, ROC and multiple opens

DISCUSSION Mkrtchyan, Tigran 2 participants 2 messages Mar 27, 2015

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

Hi folks,

we run into following issue:

on a client host we have two processes - A and B. Process A opens a file and reads some data:

OPEN -> LAYOUTGET -> READ.
Notice, that layout has a roc set to true.

Process B opens the same file:

OPEN -> READ.

Client obviously re-used layout information. Now, process A closes the file and as a result, layout becomes invalid. Nevertheless, process B still assumes that layout still valid:

I thinks this is the source of the problem:

fs/nfs/pnfs.c:

1074 list_for_each_entry(ctx, &nfsi->open_files, list) {

1075                 state = ctx->state;
1076                 /* Don't return layout if there is open file state */
1077                 if (state != NULL && state->state != 0)

1078 goto out_noroc;
1079 }
1080

this is with 3.19. The RHEL7 kernel send yet another LAYOUTGET when the process A closes the file.

Tigran.

#1 lore raw
Quoted 12 lines · Tigran

Hi folks,

we run into following issue:

on a client host we have two processes - A and B. Process A opens a file and reads some data:

OPEN -> LAYOUTGET -> READ.
Notice, that layout has a roc set to true.

Quoted 9 lines

Process B opens the same file:

OPEN -> READ.

Client obviously re-used layout information. Now, process A closes the file and as a result, layout becomes invalid. Nevertheless, process B still assumes that layout still valid:

So what happened at the CLOSE here? Was there an OPEN_DOWNGRADE or something, or do these processes represent different openowners?

My understanding is that layouts with ROC flag set are valid until the _last_ close and until any delegations are returned. The spec says this, which is somewhat vague, but seems to imply that that interpretation is correct:

"A client may also hold a layout across multiple OPENs."

With the pnfs server we were playing with internally, we had an object that tracked layouts on a per-file+per-client basis. Each open and delegation held a reference to this object and when the last reference was put, we returned the layout to the fs.

Quoted 8 lines

I thinks this is the source of the problem:

fs/nfs/pnfs.c:

1074 list_for_each_entry(ctx, &nfsi->open_files, list) {

1075                 state = ctx->state;
1076                 /* Don't return layout if there is open file state */
1077                 if (state != NULL && state->state != 0)
Quoted 8 lines

1078 goto out_noroc;
1079 }
1080

this is with 3.19. The RHEL7 kernel send yet another LAYOUTGET when the process A closes the file.

That seems like correct behavior to me. I don't think the server should invalidate the layout in this situation.

That said, we probably need to have some clarification on the ROC flag in the spec, or we end up with clients and servers potentially disagreeing about when a layout is returned to the fs.

Signature · 1 line
Jeff Layton <[email protected]>