Skip to contentSkip to conversation

Should lockd get into grace_period when statd start but not stop?

RFC Mi Jinlong 2 participants 8 messages Mar 10, 2010 – Mar 16, 2010

Subsystems (2)
lore
Share & tools
lore

Mi Jinlong

author
#0 lore raw

Hi,

When using command "service nfslock stop" and "service nfsklock start"
to restart the nfslock service at RHEL with kernel 2.6.31, if start the service after stop it more than the grace_period time, lock which lockd get before cann't be reclaimed for the grace_period is timeout.

So, IMO, the lockd should get into grace_period when statd start not stop?

Some code at kernel: fs/lockd/svclock.c
....
411 if (locks_in_grace() && !reclaim) {
412 ret = nlm_lck_denied_grace_period;
413 goto out;
414 }
415 if (reclaim && !locks_in_grace()) {
416 ret = nlm_lck_denied_grace_period;
417 goto out;
418 }
....

I think it can implement it like that:
1) When statd stop, it send a KILL signal to lockd,
and lockd only release the lock, but don't get into grace_period.
2) When statd start, it send one some other signal to lockd, and lockd only get into grace_period. The lock will be reclaimed.

thanks,
Mi Jinlong

#1 lore raw
Quoted 8 lines · Mi Jinlong

Hi,

When using command "service nfslock stop" and "service nfsklock start"
to restart the nfslock service at RHEL with kernel 2.6.31, if start the service after stop it more than the grace_period time, lock which lockd get before cann't be reclaimed for the grace_period is timeout.

So, IMO, the lockd should get into grace_period when statd start not stop?

Sorry, I'm not sure I understand.

Are you saying that lockd's grace period starts when the last lock is shut down, instead of when the new lockd is started? That would be a bug, I agree.

But are you really shutting down lockd completely?

--b.

Quoted 26 lines

Some code at kernel: fs/lockd/svclock.c
....
411 if (locks_in_grace() && !reclaim) {
412 ret = nlm_lck_denied_grace_period;
413 goto out;
414 }
415 if (reclaim && !locks_in_grace()) {
416 ret = nlm_lck_denied_grace_period;
417 goto out;
418 }
....

I think it can implement it like that:
1) When statd stop, it send a KILL signal to lockd,
and lockd only release the lock, but don't get into grace_period.
2) When statd start, it send one some other signal to lockd, and lockd only get into grace_period. The lock will be reclaimed.

thanks,
Mi Jinlong

--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

author
#2 lore raw

J. Bruce Fields :

Quoted 8 lines · Mi Jinlong
Quoted 7 lines

Hi,

When using command "service nfslock stop" and "service nfsklock start"
to restart the nfslock service at RHEL with kernel 2.6.31, if start the service after stop it more than the grace_period time, lock which lockd get before cann't be reclaimed for the grace_period is timeout.

Quoted 7 lines
Quoted 1 line

So, IMO, the lockd should get into grace_period when statd start not stop?

Sorry, I'm not sure I understand.

Are you saying that lockd's grace period starts when the last lock is shut down, instead of when the new lockd is started? That would be a bug, I agree.

I means that using command "service nfslock stop" to stop the nfslock service at RHEL, it only case statd stop, but lockd is still running.

When statd stop, it send a KILL signal to lockd, lockd will release all locks and get into grace_period state to wait reclaimed lock request. It means lockd get into grace_period state when statd stop.

But the reclaimed lock request only be caused when client receive a SM_NOTIFY which is send by server's statd at start. If statd start when lockd isn't at grace_period state, the reclaimed lock cann't be reclaimed.

So, I think, lockd should get into grace_period state when statd start, not stop.

Quoted 2 lines

But are you really shutting down lockd completely?

Lockd don't shut down when nfslock service stop, the service stop only case lockd stop at RHEL.

Quoted 31 lines

--b.

Quoted 25 lines

Some code at kernel: fs/lockd/svclock.c
....
411 if (locks_in_grace() && !reclaim) {
412 ret = nlm_lck_denied_grace_period;
413 goto out;
414 }
415 if (reclaim && !locks_in_grace()) {
416 ret = nlm_lck_denied_grace_period;
417 goto out;
418 }
....

I think it can implement it like that:
1) When statd stop, it send a KILL signal to lockd,
and lockd only release the lock, but don't get into grace_period.
2) When statd start, it send one some other signal to lockd, and lockd only get into grace_period. The lock will be reclaimed.

thanks,
Mi Jinlong

--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

Signature · 2 lines
Regards
Mi Jinlong
#3 lore raw
Quoted 29 lines · Mi Jinlong

J. Bruce Fields :

Quoted 8 lines · Mi Jinlong
Quoted 7 lines

Hi,

When using command "service nfslock stop" and "service nfsklock start"
to restart the nfslock service at RHEL with kernel 2.6.31, if start the service after stop it more than the grace_period time, lock which lockd get before cann't be reclaimed for the grace_period is timeout.

Quoted 7 lines
Quoted 1 line

So, IMO, the lockd should get into grace_period when statd start not stop?

Sorry, I'm not sure I understand.

Are you saying that lockd's grace period starts when the last lock is shut down, instead of when the new lockd is started? That would be a bug, I agree.

I means that using command "service nfslock stop" to stop the nfslock service at RHEL, it only case statd stop, but lockd is still running.

When statd stop, it send a KILL signal to lockd, lockd will release all locks and get into grace_period state to wait reclaimed lock request. It means lockd get into grace_period state when statd stop.

But the reclaimed lock request only be caused when client receive a SM_NOTIFY which is send by server's statd at start. If statd start when lockd isn't at grace_period state, the reclaimed lock cann't be reclaimed.

Quoted 7 lines

So, I think, lockd should get into grace_period state when statd start, not stop.

Quoted 2 lines

But are you really shutting down lockd completely?

Lockd don't shut down when nfslock service stop, the service stop only case lockd stop at RHEL.

OK, so that's the problem. Lockd doesn't shut down completely, if I remember correctly, until nfsd server and all clients do.

Our current NFS implementation just isn't designed to be able to shut down some components while leaving others running.

Is there some reason you *need* to do what you're doing?

--b.

author
#4 lore raw

J. Bruce Fields:

Quoted 30 lines · Mi Jinlong
Quoted 29 lines

J. Bruce Fields :

Quoted 8 lines · Mi Jinlong
Quoted 7 lines

Hi,

When using command "service nfslock stop" and "service nfsklock start"
to restart the nfslock service at RHEL with kernel 2.6.31, if start the service after stop it more than the grace_period time, lock which lockd get before cann't be reclaimed for the grace_period is timeout.

Quoted 6 lines
Quoted 1 line

So, IMO, the lockd should get into grace_period when statd start not stop?

Sorry, I'm not sure I understand.

Are you saying that lockd's grace period starts when the last lock is shut down, instead of when the new lockd is started? That would be a bug, I agree.

I means that using command "service nfslock stop" to stop the nfslock service at RHEL, it only case statd stop, but lockd is still running.

When statd stop, it send a KILL signal to lockd, lockd will release all locks and get into grace_period state to wait reclaimed lock request. It means lockd get into grace_period state when statd stop.

But the reclaimed lock request only be caused when client receive a SM_NOTIFY which is send by server's statd at start. If statd start when lockd isn't at grace_period state, the reclaimed lock cann't be reclaimed.

So, I think, lockd should get into grace_period state when statd start, not stop.

Quoted 1 line

But are you really shutting down lockd completely?

Quoted 8 lines
Quoted 2 lines

Lockd don't shut down when nfslock service stop, the service stop only case lockd stop at RHEL.

OK, so that's the problem. Lockd doesn't shut down completely, if I remember correctly, until nfsd server and all clients do.

Our current NFS implementation just isn't designed to be able to shut down some components while leaving others running.

Really? But the lockd started with nfs service start, but not nfslock service. And, lockd can't stop with statd at the same time.
Sometimes, the lockd will not synchronous with statd. Maybe this problem is a good example.

Quoted 2 lines

Is there some reason you *need* to do what you're doing?

When using the NFSv3 at RHEL, I restart the nfslock service with intermediate wait 90s, those lock that lockd get before can't be reclaimed.
I think it's a bug of the kernel, so I want geting some opinion.

If you think it's a bug too, I will try to make a patch as below to fix it.

1) When statd stop, it send a KILL signal to lockd,
and lockd only release the lock, but don't get into grace_period.
2) When statd start, it send one some other signal to lockd, and lockd only get into grace_period. The lock will be reclaimed.

thanks,
Mi Jinlong

#5 lore raw
Quoted 9 lines · Mi Jinlong

J. Bruce Fields:

Quoted 2 lines

Our current NFS implementation just isn't designed to be able to shut down some components while leaving others running.

Really? But the lockd started with nfs service start, but not nfslock service. And, lockd can't stop with statd at the same time.
Sometimes, the lockd will not synchronous with statd. Maybe this problem is a good example.

I'm sorry, I still don't understand.

Please take a look at section 3.1, 3.2, and 3.3 of the nfs-utils README file. That describes the order in which servers should be started and stopped.

--b.

author
#6 lore raw

J. Bruce Fields :

Quoted 9 lines · Mi Jinlong
Quoted 7 lines

J. Bruce Fields:

Quoted 2 lines

Our current NFS implementation just isn't designed to be able to shut down some components while leaving others running.

Really? But the lockd started with nfs service start, but not nfslock service. And, lockd can't stop with statd at the same time.
Sometimes, the lockd will not synchronous with statd. Maybe this problem is a good example.

Quoted 5 lines

I'm sorry, I still don't understand.

Please take a look at section 3.1, 3.2, and 3.3 of the nfs-utils README file. That describes the order in which servers should be started and stopped.

Maybe that's my problem.
The status of lockd and statd, when testing.

lockd statd
| | <== service nfslock stop
get KILL signal stopd ^

   and get into grace_period    |     |
           |                    |     | more than grace_period time
           |                    |     v
           |                    |   <== service nfslock start
      normal state              |
           |                  start   Client receive SM_NOTIFY and reclaime lock,
           |                    |     but out of grace_period time.

v v

As above, after nfslock service start, client cannot reclaime lock success.

thanks,
Mi Jinlong

#7 lore raw
Quoted 18 lines · Mi Jinlong

J. Bruce Fields :

Quoted 9 lines · Mi Jinlong
Quoted 7 lines

J. Bruce Fields:

Quoted 2 lines

Our current NFS implementation just isn't designed to be able to shut down some components while leaving others running.

Really? But the lockd started with nfs service start, but not nfslock service. And, lockd can't stop with statd at the same time.
Sometimes, the lockd will not synchronous with statd. Maybe this problem is a good example.

Quoted 5 lines

I'm sorry, I still don't understand.

Please take a look at section 3.1, 3.2, and 3.3 of the nfs-utils README file. That describes the order in which servers should be started and stopped.

Quoted 7 lines

Maybe that's my problem.
The status of lockd and statd, when testing.

lockd statd
| | <== service nfslock stop
get KILL signal stopd ^
and get into grace_period | |

I believe you also need to shut down nfsd here.

--b.

Quoted 9 lines
           |                    |     | more than grace_period time
           |                    |     v
           |                    |   <== service nfslock start
      normal state              |
           |                  start   Client receive SM_NOTIFY and reclaime lock,
           |                    |     but out of grace_period time.
           v                    v

As above, after nfslock service start, client cannot reclaime lock success.