Make sm-notify faster if there are no servers to notify
Subsystems (2)
Dear Experts,
sm-notify was taking a long time while my laptop booted. This was odd because I use NFS only rarely - via autofs - on that machine, and sm-notify actually has no-one to notify most of the time. So I have patched it as follows. Is this a legitimate thing to do?
utils/statd/sm-notify.c +4 -0
@@ -169,6 +169,10 @@backup_hosts (_SM_DIR_PATH, _SM_BAK_PATH);get_hosts (_SM_BAK_PATH); +if (!hosts) { +return 0 ; + } +/* Get and update the NSM state. This will call sync() */ nsm_state =nsm_get_state (opt_update_state);set_kernel_nsm_state (nsm_state);
Regards, Phil.
(Please Cc: me in any replies.)
A message that is not in the archive (sent off-list or not yet received): 1225239200402-YnoLgZYwwYuCbKHnblo0pmrPP3OPMK55cpQHUIT47Ck@public.gmane.org
Quoted 6 lines · Phil Endecott
Dear Experts,
sm-notify was taking a long time while my laptop booted. This was odd because I use NFS only rarely - via autofs - on that machine, and sm-notify actually has no-one to notify most of the time. So I have patched it as follows. Is this a legitimate thing to do?
It looks like your patch was committed to nfs-utils a couple weeks ago:
see c8d18e26d2a53d9036a32c2dafebccaf4ce1634d from
git://linux-nfs.org/nfs-utils
--b.
Quoted 26 lines
diff -ur
nfs-utils-1.1.3.orig/utils/statd/sm-notify.cnfs-utils-1.1.3/utils/statd/sm-notify.cutils/statd/sm-notify.c +4
-0@@ -169,6 +169,10 @@backup_hosts (_SM_DIR_PATH, _SM_BAK_PATH);get_hosts (_SM_BAK_PATH); +if (!hosts) { +return 0 ; + } +/* Get and update the NSM state. This will call sync() */ nsm_state =nsm_get_state (opt_update_state);set_kernel_nsm_state (nsm_state);Regards, Phil.
(Please Cc: me in any replies.)--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Quoted 11 lines · J. Bruce Fields
Quoted 3 lines · Phil Endecott
Dear Experts,
sm-notify was taking a long time while my laptop booted. This was odd because I use NFS only rarely - via autofs - on that machine, and sm-notify actually has no-one to notify most of the time. So I have patched it as follows. Is this a legitimate thing to do?
It looks like your patch was committed to nfs-utils a couple weeks ago:
seec8d18e26d2a53d9036a32c2dafebccaf4ce1634dfrom
git://linux-nfs.org/nfs-utils--b.
How curious. I guess someone saw my Debian bug report. No mention of it on this list as far as I can see though.
I presume from this that it is considered a safe thing to do.
Phil.
A message that is not in the archive (sent off-list or not yet received): 1225301403729-YnoLgZYwwYuCbKHnblo0pmrPP3OPMK55cpQHUIT47Ck@public.gmane.org
Quoted 12 lines · Phil Endecott
Quoted 11 lines · J. Bruce Fields
Quoted 6 lines · Phil Endecott
Dear Experts,
sm-notify was taking a long time while my laptop booted. This was odd because I use NFS only rarely - via autofs - on that machine, and sm-notify actually has no-one to notify most of the time. So I have patched it as follows. Is this a legitimate thing to do?
It looks like your patch was committed to nfs-utils a couple weeks ago:
seec8d18e26d2a53d9036a32c2dafebccaf4ce1634dfrom
Quoted 8 lines
Quoted 3 lines
git://linux-nfs.org/nfs-utils--b.
How curious. I guess someone saw my Debian bug report. No mention of it on this list as far as I can see though.
I presume from this that it is considered a safe thing to do.
It looks right to me. Hopefully somebody actually has tested this on a client that holds locks when it reboots?
I remember this was one of the things Arjan mentioned having to disable in his "5-second boot" talk at the Linux Plumbers Conference, so you're not the only one to have noticed the problem....
--b.
Quoted 12 lines · J. Bruce Fields
Quoted 11 lines · Phil Endecott
Quoted 10 lines · J. Bruce Fields
Quoted 3 lines · Phil Endecott
Dear Experts,
sm-notify was taking a long time while my laptop booted. This was odd because I use NFS only rarely - via autofs - on that machine, and sm-notify actually has no-one to notify most of the time. So I have patched it as follows. Is this a legitimate thing to do?
It looks like your patch was committed to nfs-utils a couple weeks ago:
seec8d18e26d2a53d9036a32c2dafebccaf4ce1634dfrom
git://linux-nfs.org/nfs-utils
Quoted 8 lines
Quoted 5 lines
Quoted 1 line
--b.
How curious. I guess someone saw my Debian bug report. No mention of it on this list as far as I can see though.
I presume from this that it is considered a safe thing to do.
It looks right to me. Hopefully somebody actually has tested this on a client that holds locks when it reboots?
Not me.
Quoted 3 lines
I remember this was one of the things Arjan mentioned having to disable in his "5-second boot" talk at the Linux Plumbers Conference, so you're not the only one to have noticed the problem....
Yes; I noticed the huge pause due to the sync() and applied this fix. It was only later that I looked at Arjan's slides (I had to wait until someone had converted then from PowerPoint to PDF...) and saw that he had the same thing in his bootchart.
Phil.
A message that is not in the archive (sent off-list or not yet received): 1225302305994-YnoLgZYwwYuCbKHnblo0pmrPP3OPMK55cpQHUIT47Ck@public.gmane.org
Quoted 22 lines · Phil Endecott
Quoted 21 lines · J. Bruce Fields
Quoted 20 lines · Phil Endecott
Quoted 9 lines · J. Bruce Fields
Quoted 7 lines · Phil Endecott
Dear Experts,
sm-notify was taking a long time while my laptop booted. This was odd because I use NFS only rarely - via autofs - on that machine, and sm-notify actually has no-one to notify most of the time. So I have patched it as follows. Is this a legitimate thing to do?
Quoted 6 lines
It looks like your patch was committed to nfs-utils a couple weeks ago:
seec8d18e26d2a53d9036a32c2dafebccaf4ce1634dfrom
git://linux-nfs.org/nfs-utils--b.
How curious. I guess someone saw my Debian bug report. No mention of it on this list as far as I can see though.
Quoted 6 lines
Quoted 4 lines
Quoted 1 line
I presume from this that it is considered a safe thing to do.
It looks right to me. Hopefully somebody actually has tested this on a client that holds locks when it reboots?
Not me.
Ugh.
We really need someone doing regular lock recovery tests with both the latest kernel and latest nfs-utils.
Ideally we'd also have tests that could be easily run by anyone. Though there may be too many site-specific details involved in writing scripts that interact with (and reboot) multiple machines.
Quoted 8 lines
Quoted 3 lines
I remember this was one of the things Arjan mentioned having to disable in his "5-second boot" talk at the Linux Plumbers Conference, so you're not the only one to have noticed the problem....
Yes; I noticed the huge pause due to the
sync()and applied this fix. It was only later that I looked at Arjan's slides (I had to wait until someone had converted then from PowerPoint to PDF...) and saw that he had the same thing in his bootchart.
Oh, so the time was all spent in the sync() in nsm_get_state()?
Anyway, I think the nsm state updating shouldn't matter if you don't even have any peers to notify.
--b.
Quoted 43 lines · J. Bruce Fields
Quoted 22 lines · Phil Endecott
Quoted 21 lines · J. Bruce Fields
Quoted 20 lines · Phil Endecott
Quoted 9 lines · J. Bruce Fields
Quoted 7 lines · Phil Endecott
Dear Experts,
sm-notify was taking a long time while my laptop booted. This was odd because I use NFS only rarely - via autofs - on that machine, and sm-notify actually has no-one to notify most of the time. So I have patched it as follows. Is this a legitimate thing to do?
Quoted 6 lines
It looks like your patch was committed to nfs-utils a couple weeks ago:
seec8d18e26d2a53d9036a32c2dafebccaf4ce1634dfrom
git://linux-nfs.org/nfs-utils--b.
How curious. I guess someone saw my Debian bug report. No mention of it on this list as far as I can see though.
Quoted 6 lines
Quoted 4 lines
Quoted 1 line
I presume from this that it is considered a safe thing to do.
It looks right to me. Hopefully somebody actually has tested this on a client that holds locks when it reboots?
Not me.
Ugh.
We really need someone doing regular lock recovery tests with both the latest kernel and latest nfs-utils.
Ideally we'd also have tests that could be easily run by anyone. Though there may be too many site-specific details involved in writing scripts that interact with (and reboot) multiple machines.
Quoted 4 lines
Quoted 3 lines
I remember this was one of the things Arjan mentioned having to disable in his "5-second boot" talk at the Linux Plumbers Conference, so you're not the only one to have noticed the problem....
Quoted 6 lines
Quoted 4 lines
Yes; I noticed the huge pause due to the
sync()and applied this fix. It was only later that I looked at Arjan's slides (I had to wait until someone had converted then from PowerPoint to PDF...) and saw that he had the same thing in his bootchart.Oh, so the time was all spent in the
sync()innsm_get_state()?
I assume sync() is required because this logic performs a rename as well as a simple write?
Quoted 2 lines
Anyway, I think the nsm state updating shouldn't matter if you don't even have any peers to notify.
It probably does matter.
When a system is initially installed, it likely does not have a state file in /var/lib/nfs. This may be harmless if it's not present; rpc.statd probably does the right thing in this case.
However, the rest of the logic in nsm_get_state() is needed to bump the system's state value properly after every reboot. It may be inconsequential if there were no mounts or no NFS clients during the last reboot, but this is subtle. I wouldn't bet on it.
Quoted 7 lines
--b.
--
To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
--
Chuck Lever
chuck[dot]lever[at]oracle[dot]com
Quoted 2 lines · Chuck Lever
I assume
sync()is required because this logic performs a rename as well as a simple write?
I think an fsync() on the containing directory (together with an fsync() of the file itself) would do the job if you wanted to avoid the globaly sync(). I don't think ext3 is capable of doing anything finer-grained than a whole-filesystem sync, though, so this doesn't help many people in practice right now.
In any case, the rename adds an extra level of safety by ensuring the nsm state is updated atomically, so we shouldn't get rid of it.
Quoted 8 lines
Quoted 2 lines
Anyway, I think the nsm state updating shouldn't matter if you don't even have any peers to notify.
It probably does matter.
When a system is initially installed, it likely does not have a state file in
/var/lib/nfs. This may be harmless if it's not present; rpc.statd probably does the right thing in this case.
The "right thing" in that case would be, I guess, to create a state file with "0" in it. It doesn't do that. So this patch *does* break stuff. Oops!
So should we revert it and do something else, or patch statd to create a new state file if necessary?
Quoted 4 lines
However, the rest of the logic in
nsm_get_state()is needed to bump the system's state value properly after every reboot. It may be inconsequential if there were no mounts or no NFS clients during the last reboot, but this is subtle. I wouldn't bet on it.
If the state is only every communicated to hosts by notifications, then if we're not notifying, the update of the state can't matter.
(So is that premise correct?)
--b.
Quoted 22 lines · bfields
Quoted 2 lines · Chuck Lever
I assume
sync()is required because this logic performs a rename as well as a simple write?I think an
fsync()on the containing directory (together with anfsync()of the file itself) would do the job if you wanted to avoid the globalysync(). I don't think ext3 is capable of doing anything finer-grained than a whole-filesystem sync, though, so this doesn't help many people in practice right now.In any case, the rename adds an extra level of safety by ensuring the nsm state is updated atomically, so we shouldn't get rid of it.
Quoted 8 lines
Quoted 2 lines
Anyway, I think the nsm state updating shouldn't matter if you don't even have any peers to notify.
It probably does matter.
When a system is initially installed, it likely does not have a state file in
/var/lib/nfs. This may be harmless if it's not present; rpc.statd probably does the right thing in this case.
Quoted 6 lines
The "right thing" in that case would be, I guess, to create a state file with "0" in it. It doesn't do that. So this patch *does* break stuff. Oops!
So should we revert it and do something else, or patch statd to create a new state file if necessary?
It looks like this still needs to be fixed? I think it would be good enough just to teach rpc.nfsd to create the file if it doesn't exist.
--b.
Quoted 12 lines
Quoted 5 lines
However, the rest of the logic in
nsm_get_state()is needed to bump the system's state value properly after every reboot. It may be inconsequential if there were no mounts or no NFS clients during the last reboot, but this is subtle. I wouldn't bet on it.If the state is only every communicated to hosts by notifications, then if we're not notifying, the update of the state can't matter.
(So is that premise correct?)
--b.
Quoted 27 lines · J. Bruce Fields
Quoted 26 lines · bfields
Quoted 2 lines · Chuck Lever
I assume
sync()is required because this logic performs a rename as well as a simple write?I think an
fsync()on the containing directory (together with anfsync()of the file itself) would do the job if you wanted to avoid the globalysync(). I don't think ext3 is capable of doing anything finer-grained than a whole-filesystem sync, though, so this doesn't help many people in practice right now.In any case, the rename adds an extra level of safety by ensuring the nsm state is updated atomically, so we shouldn't get rid of it.
Quoted 8 lines
Quoted 2 lines
Anyway, I think the nsm state updating shouldn't matter if you don't even have any peers to notify.
It probably does matter.
When a system is initially installed, it likely does not have a state file in
/var/lib/nfs. This may be harmless if it's not present; rpc.statd probably does the right thing in this case.The "right thing" in that case would be, I guess, to create a state file with "0" in it. It doesn't do that. So this patch *does* break stuff. Oops!
Quoted 5 lines
Quoted 2 lines
So should we revert it and do something else, or patch statd to create a new state file if necessary?
It looks like this still needs to be fixed? I think it would be good enough just to teach rpc.nfsd to create the file if it doesn't exist.
Meh. I'd rather manage the state file in one place, rather than have multiple user space entities fiddle with it.
I think we should find out exactly what breaks when sm-notify quits early. Steve hasn't found a problem with the patch already in nfs-utils, but the corner cases here are really narrow.
Without a lot of testing (which we currently don't have the resources for), I don't feel 100% positive about sm-notify quitting early. My preferred solution would involve working around the sync(2) call instead (ie fixing sm-notify so we don't need it, or somehow doing it in the background so it doesn't hold up the boot-up process). I think we will end up waiting until this actually bites someone, but chances are it will be a long wait.
Quoted 14 lines
--b.
Quoted 12 lines
Quoted 5 lines
However, the rest of the logic in
nsm_get_state()is needed to bump the system's state value properly after every reboot. It may be inconsequential if there were no mounts or no NFS clients during the last reboot, but this is subtle. I wouldn't bet on it.If the state is only every communicated to hosts by notifications, then if we're not notifying, the update of the state can't matter.
(So is that premise correct?)
--b.
Signature · 2 lines
Chuck Lever chuck[dot]lever[at]oracle[dot]com
Quoted 37 lines · Chuck Lever
Quoted 36 lines · J. Bruce Fields
Quoted 35 lines · bfields
Quoted 3 lines · Chuck Lever
I assume
sync()is required because this logic performs a rename as well
as a simple write?I think an
fsync()on the containing directory (together with anfsync()
of the file itself) would do the job if you wanted to avoid the globalysync(). I don't think ext3 is capable of doing anything finer- grained
than a whole-filesystem sync, though, so this doesn't help many people
in practice right now.In any case, the rename adds an extra level of safety by ensuring the nsm state is updated atomically, so we shouldn't get rid of it.
Quoted 10 lines
Quoted 3 lines
Anyway, I think the nsm state updating shouldn't matter if you don't
even have any peers to notify.It probably does matter.
When a system is initially installed, it likely does not have a state
file in/var/lib/nfs. This may be harmless if it's not present; rpc.statd probably does the right thing in this case.The "right thing" in that case would be, I guess, to create a state file
with "0" in it. It doesn't do that. So this patch *does* break stuff.
Oops!
Quoted 6 lines
Quoted 6 lines
Quoted 3 lines
So should we revert it and do something else, or patch statd to create
a new state file if necessary?It looks like this still needs to be fixed? I think it would be good enough just to teach rpc.nfsd to create the file if it doesn't exist.
^^^^^^^^
(Err, sorry, note I meant rpc.statd, not rpc.nfsd.)
--b.
Quoted 14 lines
Meh. I'd rather manage the state file in one place, rather than have multiple user space entities fiddle with it.
I think we should find out exactly what breaks when sm-notify quits early. Steve hasn't found a problem with the patch already in nfs- utils, but the corner cases here are really narrow.
Without a lot of testing (which we currently don't have the resources for), I don't feel 100% positive about sm-notify quitting early. My preferred solution would involve working around the sync(2) call instead (ie fixing sm-notify so we don't need it, or somehow doing it in the background so it doesn't hold up the boot-up process). I think we will end up waiting until this actually bites someone, but chances are it will be a long wait.
Quoted 42 lines · J. Bruce Fields
Quoted 41 lines · Chuck Lever
Quoted 40 lines · J. Bruce Fields
Quoted 38 lines · bfields
Quoted 3 lines · Chuck Lever
I assume
sync()is required because this logic performs a rename as well
as a simple write?I think an
fsync()on the containing directory (together with anfsync()
of the file itself) would do the job if you wanted to avoid the globalysync(). I don't think ext3 is capable of doing anything finer- grained
than a whole-filesystem sync, though, so this doesn't help many people
in practice right now.In any case, the rename adds an extra level of safety by ensuring the nsm state is updated atomically, so we shouldn't get rid of it.
Quoted 10 lines
Quoted 3 lines
Anyway, I think the nsm state updating shouldn't matter if you don't
even have any peers to notify.It probably does matter.
When a system is initially installed, it likely does not have a state
file in/var/lib/nfs. This may be harmless if it's not present; rpc.statd probably does the right thing in this case.The "right thing" in that case would be, I guess, to create a state file
with "0" in it. It doesn't do that. So this patch *does* break stuff.
Oops!So should we revert it and do something else, or patch statd to create
a new state file if necessary?
Quoted 10 lines
Quoted 2 lines
Quoted 2 lines
It looks like this still needs to be fixed? I think it would be good enough just to teach rpc.nfsd to create the file if it doesn't exist.
^^^^^^^^(Err, sorry, note I meant rpc.statd, not rpc.nfsd.)
--b.
Quoted 2 lines
Meh. I'd rather manage the state file in one place, rather than have multiple user space entities fiddle with it.
Acknowledged. My comments below still stand, I think. More testing! Yay!
Quoted 11 lines
Quoted 11 lines
I think we should find out exactly what breaks when sm-notify quits early. Steve hasn't found a problem with the patch already in nfs- utils, but the corner cases here are really narrow.
Without a lot of testing (which we currently don't have the resources for), I don't feel 100% positive about sm-notify quitting early. My preferred solution would involve working around the sync(2) call instead (ie fixing sm-notify so we don't need it, or somehow doing it in the background so it doesn't hold up the boot-up process). I think we will end up waiting until this actually bites someone, but chances are it will be a long wait.
In any event, I will keep this in mind (along with the idea of allowing only client or server peers to be notified of a reboot) as I audit this code for IPv6 support.
Signature · 2 lines
Chuck Lever chuck[dot]lever[at]oracle[dot]com
Quoted 1 line · Chuck Lever
My preferred solution would involve working around the sync(2) call instead (ie fixing sm-notify so we don't need it, or somehow doing it in the background so it doesn't hold up the boot-up process).
Doing it in the background doesn't really help, as it's the huge blob of disk writes that it generates that's the problem as much as the elapsed time. Boot time is more likely to be I/O bound than CPU bound.
BTW, I have written an article about my efforts to speed up booting which will should appear at debian-administration.org later today.
Cheers, Phil.
Quoted 21 lines · J. Bruce Fields
Quoted 2 lines · Chuck Lever
I assume
sync()is required because this logic performs a rename as well as a simple write?I think an
fsync()on the containing directory (together with anfsync()of the file itself) would do the job if you wanted to avoid the globalysync(). I don't think ext3 is capable of doing anything finer-grained than a whole-filesystem sync, though, so this doesn't help many people in practice right now.In any case, the rename adds an extra level of safety by ensuring the nsm state is updated atomically, so we shouldn't get rid of it.
Quoted 7 lines
Quoted 2 lines
Anyway, I think the nsm state updating shouldn't matter if you don't even have any peers to notify.
It probably does matter.
When a system is initially installed, it likely does not have a state file in
/var/lib/nfs. This may be harmless if it's not present; rpc.statd probably does the right thing in this case.
Quoted 6 lines
The "right thing" in that case would be, I guess, to create a state file with "0" in it. It doesn't do that. So this patch *does* break stuff. Oops!
So should we revert it and do something else, or patch statd to create a new state file if necessary?
I have to agree with Chuck, that managing the state file from one place is desirable... Although does it make sense to move that management into a init script? Maybe insuring the state is different before any daemons are started? Would we still need the sync()?
Quoted 8 lines
Quoted 5 lines
However, the rest of the logic in
nsm_get_state()is needed to bump the system's state value properly after every reboot. It may be inconsequential if there were no mounts or no NFS clients during the last reboot, but this is subtle. I wouldn't bet on it.If the state is only every communicated to hosts by notifications, then if we're not notifying, the update of the state can't matter.
I had the same notion... If there are no clients to notify, why would any clients care if our state changed?? With that said.... I do have a sinking feeling that this patch will come back to bite us... It was just too easy of a fix... :-)
steved.
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 39 lines · Steve Dickson
Quoted 21 lines · J. Bruce Fields
Quoted 2 lines · Chuck Lever
I assume
sync()is required because this logic performs a rename as well as a simple write?I think an
fsync()on the containing directory (together with anfsync()of the file itself) would do the job if you wanted to avoid the globalysync(). I don't think ext3 is capable of doing anything finer-grained than a whole-filesystem sync, though, so this doesn't help many people in practice right now.In any case, the rename adds an extra level of safety by ensuring the nsm state is updated atomically, so we shouldn't get rid of it.
Quoted 7 lines
Quoted 2 lines
Anyway, I think the nsm state updating shouldn't matter if you don't even have any peers to notify.
It probably does matter.
When a system is initially installed, it likely does not have a state file in
/var/lib/nfs. This may be harmless if it's not present; rpc.statd probably does the right thing in this case.Quoted 6 lines
The "right thing" in that case would be, I guess, to create a state file with "0" in it. It doesn't do that. So this patch *does* break stuff. Oops!
So should we revert it and do something else, or patch statd to create a new state file if necessary?
I have to agree with Chuck, that managing the state file from one place is desirable... Although does it make sense to move that management into a init script? Maybe insuring the state is different before any daemons are started? Would we still need the
sync()?Quoted 6 lines
Quoted 5 lines
However, the rest of the logic in
nsm_get_state()is needed to bump the system's state value properly after every reboot. It may be inconsequential if there were no mounts or no NFS clients during the last reboot, but this is subtle. I wouldn't bet on it.
Quoted 6 lines
Quoted 2 lines
If the state is only every communicated to hosts by notifications, then if we're not notifying, the update of the state can't matter.
I had the same notion... If there are no clients to notify, why would any clients care if our state changed?? With that said.... I do have a sinking feeling that this patch will come back to bite us... It was just too easy of a fix... :-)
Any progress on this? I don't think we can release in the current state, since as far as I can tell that means on a new system, unless the install scripts create /var/lib/nfs/state, neither sm-notify nor statd ever writes to /proc/sys/fs/nfs/nsm_local_state, and (without testing) it looks to me like that means lockd defaults to a state of 0, which is nonsense?
One possible compromise that I think would be correct might be to skip the sync in the case where there are no peers to notify, instead of exiting early in that case. As long as a sync happens at some point before we grant any locks (which statd can guarantee), I think that will work....
--b.
Quoted 7 lines · On Thursday December 4, [email protected] wrote:
Any progress on this? I don't think we can release in the current state, since as far as I can tell that means on a new system, unless the install scripts create
/var/lib/nfs/state, neither sm-notify nor statd ever writes to/proc/sys/fs/nfs/nsm_local_state, and (without testing) it looks to me like that means lockd defaults to a state of 0, which is nonsense?
Why is '0' nonsense?
The only real requirement on 'state' is that it changes when the host reboots while some peer is monitoring it. Even if it got reset to zero every time the sm and sm.bak became empty it would still work just fine. If we reboot and find that both sm and sm.bak are empty there is really no point in changing 'state'.
I think it would still be valuable to replace the 'sync' with two 'fsync's, one of the file, one on the directory.
This may not be a win on ext3 today (I'm not 100% certain about that) but there are other filesystems and more seem to be coming.
NeilBrown
A message that is not in the archive (sent off-list or not yet received): 18744.41310.635618.148281-wvvUuzkyo1EYVZTmpyfIwg@public.gmane.org
Quoted 10 lines · Neil Brown
Quoted 7 lines · On Thursday December 4, [email protected] wrote:
Any progress on this? I don't think we can release in the current state, since as far as I can tell that means on a new system, unless the install scripts create
/var/lib/nfs/state, neither sm-notify nor statd ever writes to/proc/sys/fs/nfs/nsm_local_state, and (without testing) it looks to me like that means lockd defaults to a state of 0, which is nonsense?Why is '0' nonsense?
Even numbers are supposed to mean that a host is down, odd that it's up. In practice noone ever uses that fact--they only ever advertise odd numbers. So we might get away with using an even number. But then again a peer would be in its rights to check the parity and complain, and some may well do that.
Quoted 4 lines
The only real requirement on 'state' is that it changes when the host reboots while some peer is monitoring it. Even if it got reset to zero every time the sm and sm.bak became empty it would still work just fine.
Hm, I don't know; it might be that an inconveniently timed network partition combined with nsm states that repeat themselves could prevent a client from knowing about some reboot that it should have known about.
It's safer to keep a counter and ensure that the state never repeats (well, anyway, not until it overflows after a few billion reboots).
Quoted 2 lines
If we reboot and find that both sm and sm.bak are empty there is really no point in changing 'state'.
I think I've convinced myself of that, yes. For the purposes of locking, a reboot which didn't require notifying any client may as well have not been a reboot, since no state was lost.
But we should at least make sure that the state is properly initialized and nondecreasing.
Quoted 2 lines
I think it would still be valuable to replace the 'sync' with two 'fsync's, one of the file, one on the directory.
Sure, may as well.--b.
Quoted 2 lines
This may not be a win on ext3 today (I'm not 100% certain about that) but there are other filesystems and more seem to be coming.
Quoted 6 lines · J. Bruce Fields
Quoted 3 lines
I think it would still be valuable to replace the 'sync' with two 'fsync's, one of the file, one on the directory.
Sure, may as well.--b.
Something similar to this:
utils/statd/sm-notify.c renamed from utils/statd/sm-notify.c.orig +3 -7
@@ -211,12 +211,6 @@ usage: fprintf(stderr,backup_hosts (_SM_DIR_PATH, _SM_BAK_PATH);get_hosts (_SM_BAK_PATH); -/* If there are not hosts to notify, just exit */ -if (!hosts) { -nsm_log (LOG_DEBUG ,"No hosts to notify; exiting" ); -return 0 ; - } -/* Get and update the NSM state. This will call sync() */ nsm_state =nsm_get_state (opt_update_state);set_kernel_nsm_state (nsm_state);
@@ -694,6 +688,7 @@ nsm_get_state(int update) {char newfile[PATH_MAX ];int fd, state; +DIR *fp;if ((fd =open (_SM_STATE_PATH,O_RDONLY )) <0 ) {if (!opt_quiet) {
@@ -730,13 +725,16 @@ nsm_get_state(int update) "Failed to write state to %s", newfile);exit (1 ); } +fsync (fd);close (fd); + fp =opendir (_SM_STATE_PATH);if (rename (newfile, _SM_STATE_PATH) <0 ) {nsm_log (LOG_ERR ,"Cannot create %s: %m" , _SM_STATE_PATH);exit (1 ); } -sync ();
If so, I'll make it happen...
steved.
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 13 lines · Steve Dickson
Quoted 6 lines · J. Bruce Fields
Quoted 3 lines
I think it would still be valuable to replace the 'sync' with two 'fsync's, one of the file, one on the directory.
Sure, may as well.--b.
Something similar to this:
diff -up nfs-utils/utils/statd/sm-notify.c.orig
nfs-utils/utils/statd/sm-notify.c
utils/statd/sm-notify.c usage: fprintf(stderr, · line 218
Quoted 8 lines
utils/statd/sm-notify.c renamed from utils/statd/sm-notify.c.orig +0
-5@@ -211,12 +211,6 @@ usage: fprintf(stderr,backup_hosts (_SM_DIR_PATH, _SM_BAK_PATH);get_hosts (_SM_BAK_PATH); -/* If there are not hosts to notify, just exit */ -if (!hosts) { -nsm_log (LOG_DEBUG ,"No hosts to notify; exiting" ); -return 0 ; - }
This was still a huge boot-time win in the common case, so now that we've committed to it I'd rather not regress. Let's just skip the sync()s/fsncy()s in the !hosts case--that looks to me like the simplest correct solution for now.
Quoted 13 lines
utils/statd/sm-notify.c renamed from utils/statd/sm-notify.c.orig +1
-1-/* Get and update the NSM state. This will call sync() */ nsm_state =nsm_get_state (opt_update_state);set_kernel_nsm_state (nsm_state);@@ -694,6 +688,7 @@ nsm_get_state(int update) {char newfile[PATH_MAX ];int fd, state; +DIR *fp;if ((fd =open (_SM_STATE_PATH,O_RDONLY )) <0 ) {if (!opt_quiet) {
utils/statd/sm-notify.c nsm_get_state(int update) · line 730
Quoted 6 lines
utils/statd/sm-notify.c renamed from utils/statd/sm-notify.c.orig +2
-0@@ -730,13 +725,16 @@ nsm_get_state(int update) "Failed to write state to %s", newfile);exit (1 ); } +fsync (fd);close (fd); + fp =opendir (_SM_STATE_PATH);
Also, I think you meant to:
fsync(fp); close(fp);
?
--b.
Quoted 10 lines
utils/statd/sm-notify.c renamed from utils/statd/sm-notify.c.orig +0
-1if (rename (newfile, _SM_STATE_PATH) <0 ) {nsm_log (LOG_ERR ,"Cannot create %s: %m" , _SM_STATE_PATH);exit (1 ); } -sync ();If so, I'll make it happen...
steved.
Quoted 17 lines · bfields
Quoted 16 lines · Steve Dickson
Quoted 6 lines · J. Bruce Fields
Quoted 3 lines
I think it would still be valuable to replace the 'sync' with two 'fsync's, one of the file, one on the directory.
Sure, may as well.--b.
Something similar to this:
diff -up nfs-utils/utils/statd/sm-notify.c.orig
nfs-utils/utils/statd/sm-notify.cutils/statd/sm-notify.c renamed from utils/statd/sm-notify.c.orig +0
-0@@ -211,12 +211,6 @@ usage: fprintf(stderr,backup_hosts (_SM_DIR_PATH, _SM_BAK_PATH);get_hosts (_SM_BAK_PATH);
Quoted 10 lines
utils/statd/sm-notify.c usage: fprintf(stderr, · line 218
Quoted 5 lines
utils/statd/sm-notify.c renamed from utils/statd/sm-notify.c.orig +0
-5-/* If there are not hosts to notify, just exit */ -if (!hosts) { -nsm_log (LOG_DEBUG ,"No hosts to notify; exiting" ); -return 0 ; - }This was still a huge boot-time win in the common case, so now that we've committed to it I'd rather not regress. Let's just skip the sync()s/fsncy()s in the !hosts case--that looks to me like the simplest correct solution for now.
My argument for correctness: if we don't sync in that case, then on reboot the rename that updates the state will either have happened or (if a crash comes too soon) not.
It is OK for that update to not happen as long as we're assured it happens before the first lock request is made or replied to, or the first monitor request completes, as, in the absence of any notifies, those are the only points at which the new state will be exposed to the outside world.
The first lock request will also require an upcall to statd. So we're OK as long as any monitor requests (from either the local kernel or remote peers) do a sync.
And statd should be doing a sync before responding to any monitor request. As long as the SM_DIR is on the same filesystem as the state file, that would do the job.... But now that I look, I see statd is using an open with O_SYNC to ensure the new statd record hits stable storage. Which we can't count on being enough.
How about adding an explicit fsync() of the state file (and parent directory) to statd's first succesful creation of a statd record, together with a comment explaining this? So around about line 194 in utils/statd/monitor.c:sm_mon_1_svc()?
In fact, we could delete the sync entirely and do the same before the first notification, and then we wouldn't have to wait for the sync in the case host records are present either.... (statd would, but perhaps we could still get other work done in the mean time).
(Am I missing something?
--b.
Quoted 56 lines · J. Bruce Fields
Quoted 19 lines · bfields
Quoted 18 lines · Steve Dickson
Quoted 6 lines · J. Bruce Fields
Quoted 3 lines
I think it would still be valuable to replace the 'sync' with two 'fsync's, one of the file, one on the directory.
Sure, may as well.--b.
Something similar to this:
diff -up nfs-utils/utils/statd/sm-notify.c.orig
nfs-utils/utils/statd/sm-notify.c
--- nfs-utils/utils/statd/sm-notify.c.orig 2008-11-17
15:06:13.000000000 -0500
+++nfs-utils/utils/statd/sm-notify.c2008-12-05
08:21:52.000000000 -0500utils/statd/sm-notify.c renamed from utils/statd/sm-notify.c.orig +0
-0@@ -211,12 +211,6 @@ usage: fprintf(stderr,backup_hosts (_SM_DIR_PATH, _SM_BAK_PATH);get_hosts (_SM_BAK_PATH);Quoted 10 lines
utils/statd/sm-notify.c usage: fprintf(stderr, · line 218
Quoted 5 lines
utils/statd/sm-notify.c renamed from utils/statd/sm-notify.c.orig +0
-5-/* If there are not hosts to notify, just exit */ -if (!hosts) { -nsm_log (LOG_DEBUG ,"No hosts to notify; exiting" ); -return 0 ; - }This was still a huge boot-time win in the common case, so now that we've committed to it I'd rather not regress. Let's just skip the sync()s/fsncy()s in the !hosts case--that looks to me like the simplest correct solution for now.
My argument for correctness: if we don't sync in that case, then on reboot the rename that updates the state will either have happened or (if a crash comes too soon) not.
It is OK for that update to not happen as long as we're assured it happens before the first lock request is made or replied to, or the first monitor request completes, as, in the absence of any notifies, those are the only points at which the new state will be exposed to the outside world.
The first lock request will also require an upcall to statd. So we're OK as long as any monitor requests (from either the local kernel or remote peers) do a sync.
And statd should be doing a sync before responding to any monitor request. As long as the
SM_DIRis on the same filesystem as the state file, that would do the job.... But now that I look, I see statd is using an open withO_SYNCto ensure the new statd record hits stable storage. Which we can't count on being enough.How about adding an explicit
fsync()of the state file (and parent directory) to statd's first succesful creation of a statd record, together with a comment explaining this? So around about line 194 in utils/statd/monitor.c:sm_mon_1_svc()?
Quoted 6 lines
In fact, we could delete the sync entirely and do the same before the first notification, and then we wouldn't have to wait for the sync in the case host records are present either.... (statd would, but perhaps we could still get other work done in the mean time).
(Am I missing something?)
This all might work, but I think we're adding a lot of complexity as a workaround. Someone should fix the real problem, which is the implementation of sync().
That would probably have other benefits too.
--
Chuck Lever
chuck[dot]lever[at]oracle[dot]com
Quoted 12 lines · Chuck Lever
Quoted 11 lines · J. Bruce Fields
How about adding an explicit
fsync()of the state file (and parent directory) to statd's first succesful creation of a statd record, together with a comment explaining this? So around about line 194 in utils/statd/monitor.c:sm_mon_1_svc()?In fact, we could delete the sync entirely and do the same before the first notification, and then we wouldn't have to wait for the sync in the case host records are present either.... (statd would, but perhaps
we could still get other work done in the mean time).
Quoted 4 lines
Quoted 1 line
(Am I missing something?)
This all might work, but I think we're adding a lot of complexity as a workaround.
I think you mean the justification is too subtle--the code itself (just a couple syncs or fsyncs) is pretty simple.
Quoted 2 lines
Someone should fix the real problem, which is the
implementation ofsync().
I think you mean of fsync(). My understanding of past discussions on the issue is that it's not really fixable on ext3, at least. So default setups will have this problem for a while.
--b.
Quoted 1 line
That would probably have other benefits too.
Hi Bruce-
Quoted 12 lines · J. Bruce Fields
Quoted 11 lines · Chuck Lever
Quoted 10 lines · J. Bruce Fields
How about adding an explicit
fsync()of the state file (and parent directory) to statd's first succesful creation of a statd record, together with a comment explaining this? So around about line 194 in utils/statd/monitor.c:sm_mon_1_svc()?In fact, we could delete the sync entirely and do the same before the first notification, and then we wouldn't have to wait for the sync in the case host records are present either.... (statd would, but perhaps we could still get other work done in the mean time).
Quoted 7 lines
Quoted 4 lines
Quoted 1 line
(Am I missing something?)
This all might work, but I think we're adding a lot of complexity as a workaround.
I think you mean the justification is too subtle--the code itself (just a couple syncs or fsyncs) is pretty simple.
Let me state it another way. Your suggested redesign makes assumptions about the order in which these operations are performed. Can the code provide any guarantee that this ordering is always true? This may have been feasible when rpc.statd did notification too, but these are now two separate programs. Can you think of a way of implementing this where the ordering dependencies are coded instead of commented?
It would be significantly less fragile (ie immune to ordering problems and long-term changes to system start-up and nfs-utils code) and simpler to understand if sm-notify does whatever syncing it needs, and rpc.statd does whatever syncing it needs.
All I'm saying is this scheme may be easy to break by accident, and any future problem here is probably not going to show up until someone really needs this to work right.
Quoted 6 lines
Quoted 2 lines
Someone should fix the real problem, which is the
implementation ofsync().I think you mean of
fsync(). My understanding of past discussions on the issue is that it's not really fixable on ext3, at least. So default setups will have this problem for a while.
--
Chuck Lever
chuck[dot]lever[at]oracle[dot]com
Quoted 28 lines · Chuck Lever
Hi Bruce-
Quoted 17 lines · J. Bruce Fields
Quoted 16 lines · Chuck Lever
Quoted 14 lines · J. Bruce Fields
How about adding an explicit
fsync()of the state file (and parent directory) to statd's first succesful creation of a statd record, together with a comment explaining this? So around about line 194 in
utils/statd/monitor.c:sm_mon_1_svc()?In fact, we could delete the sync entirely and do the same before the
first notification, and then we wouldn't have to wait for the sync in
the case host records are present either.... (statd would, but perhaps we could still get other work done in the mean time).(Am I missing something?)
Quoted 7 lines
Quoted 3 lines
This all might work, but I think we're adding a lot of complexity as a
workaround.I think you mean the justification is too subtle--the code itself (just
a couple syncs or fsyncs) is pretty simple.
Quoted 6 lines
Let me state it another way. Your suggested redesign makes assumptions about the order in which these operations are performed. Can the code provide any guarantee that this ordering is always true? This may have been feasible when rpc.statd did notification too, but these are now two separate programs. Can you think of a way of implementing this where the ordering dependencies are coded instead of commented?
The existing code requires that we update the on-disk nsm state number before doing notification or accepting statd calls. That's the only ordering I'd require.
Maybe I should try to write a patch.
--b.
Quoted 20 lines
It would be significantly less fragile (ie immune to ordering problems and long-term changes to system start-up and nfs-utils code) and simpler to understand if sm-notify does whatever syncing it needs, and rpc.statd does whatever syncing it needs.
All I'm saying is this scheme may be easy to break by accident, and any future problem here is probably not going to show up until someone really needs this to work right.
Quoted 6 lines
Quoted 2 lines
Someone should fix the real problem, which is the
implementation ofsync().I think you mean of
fsync(). My understanding of past discussions on the issue is that it's not really fixable on ext3, at least. So default setups will have this problem for a while.--
Chuck Lever
chuck[dot]lever[at]oracle[dot]com
Quoted 30 lines · J. Bruce Fields
Quoted 15 lines · bfields
Quoted 14 lines · Steve Dickson
Quoted 4 lines · J. Bruce Fields
Quoted 2 lines
I think it would still be valuable to replace the 'sync' with two 'fsync's, one of the file, one on the directory.
Sure, may as well.--b.
Something similar to this:
diff -up nfs-utils/utils/statd/sm-notify.c.orig
nfs-utils/utils/statd/sm-notify.cutils/statd/sm-notify.c renamed from utils/statd/sm-notify.c.orig +0
-0@@ -211,12 +211,6 @@ usage: fprintf(stderr,backup_hosts (_SM_DIR_PATH, _SM_BAK_PATH);get_hosts (_SM_BAK_PATH);Quoted 9 lines
utils/statd/sm-notify.c usage: fprintf(stderr, · line 218
Quoted 5 lines
utils/statd/sm-notify.c renamed from utils/statd/sm-notify.c.orig +0
-5-/* If there are not hosts to notify, just exit */ -if (!hosts) { -nsm_log (LOG_DEBUG ,"No hosts to notify; exiting" ); -return 0 ; - }This was still a huge boot-time win in the common case, so now that we've committed to it I'd rather not regress. Let's just skip the sync()s/fsncy()s in the !hosts case--that looks to me like the simplest correct solution for now.
My argument for correctness: if we don't sync in that case, then on reboot the rename that updates the state will either have happened or (if a crash comes too soon) not.
Quoted 5 lines
It is OK for that update to not happen as long as we're assured it happens before the first lock request is made or replied to, or the first monitor request completes, as, in the absence of any notifies, those are the only points at which the new state will be exposed to the outside world.
Doesn't the sync() have to happen before the file is first read by stated. Meaning before statd:main() calls load_state_number()?
Quoted 11 lines
The first lock request will also require an upcall to statd. So we're OK as long as any monitor requests (from either the local kernel or remote peers) do a sync.
And statd should be doing a sync before responding to any monitor request. As long as the
SM_DIRis on the same filesystem as the state file, that would do the job.... But now that I look, I see statd is using an open withO_SYNCto ensure the new statd record hits stable storage. Which we can't count on being enough.
Quoted 4 lines
How about adding an explicit
fsync()of the state file (and parent directory) to statd's first succesful creation of a statd record, together with a comment explaining this? So around about line 194 in utils/statd/monitor.c:sm_mon_1_svc()?
If we do the sync()/fsync() here we will also have to update MY_STATE since that's what is the number used in the RPCs. But also I think doing the sync this late be a bit waste since there is real good chance the rename has already been sync-ed out by previous sync() during boot up... or am I missing something...
steved.
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 31 lines · Steve Dickson
Quoted 30 lines · J. Bruce Fields
Quoted 15 lines · bfields
Quoted 14 lines · Steve Dickson
Quoted 4 lines · J. Bruce Fields
Quoted 2 lines
I think it would still be valuable to replace the 'sync' with two 'fsync's, one of the file, one on the directory.
Sure, may as well.--b.
Something similar to this:
diff -up nfs-utils/utils/statd/sm-notify.c.orig
nfs-utils/utils/statd/sm-notify.cutils/statd/sm-notify.c renamed from utils/statd/sm-notify.c.orig +0
-0@@ -211,12 +211,6 @@ usage: fprintf(stderr,backup_hosts (_SM_DIR_PATH, _SM_BAK_PATH);get_hosts (_SM_BAK_PATH);Quoted 9 lines
utils/statd/sm-notify.c usage: fprintf(stderr, · line 218
Quoted 5 lines
utils/statd/sm-notify.c renamed from utils/statd/sm-notify.c.orig +0
-5-/* If there are not hosts to notify, just exit */ -if (!hosts) { -nsm_log (LOG_DEBUG ,"No hosts to notify; exiting" ); -return 0 ; - }This was still a huge boot-time win in the common case, so now that we've committed to it I'd rather not regress. Let's just skip the sync()s/fsncy()s in the !hosts case--that looks to me like the simplest correct solution for now.
My argument for correctness: if we don't sync in that case, then on reboot the rename that updates the state will either have happened or (if a crash comes too soon) not.
Quoted 7 lines
Quoted 5 lines
It is OK for that update to not happen as long as we're assured it happens before the first lock request is made or replied to, or the first monitor request completes, as, in the absence of any notifies, those are the only points at which the new state will be exposed to the outside world.
Doesn't the
sync()have to happen before the file is first read by stated. Meaning before statd:main() callsload_state_number()?
I can't see why. sync() of course can't have any effect on a subsequent read of the file.
Quoted 10 lines
Quoted 10 lines
The first lock request will also require an upcall to statd. So we're OK as long as any monitor requests (from either the local kernel or remote peers) do a sync.
And statd should be doing a sync before responding to any monitor request. As long as the
SM_DIRis on the same filesystem as the state file, that would do the job.... But now that I look, I see statd is using an open withO_SYNCto ensure the new statd record hits stable storage. Which we can't count on being enough.
Quoted 6 lines
Quoted 4 lines
How about adding an explicit
fsync()of the state file (and parent directory) to statd's first succesful creation of a statd record, together with a comment explaining this? So around about line 194 in utils/statd/monitor.c:sm_mon_1_svc()?If we do the sync()/fsync() here we will also have to update
MY_STATEsince that's what is the number used in the RPCs.
I don't believe that's true.
Quoted 4 lines
But also I think
doing the sync this late be a bit waste since there is real good chance the rename has already been sync-ed out by previoussync()during boot up... or am I missing something...
The whole boot gets held up waiting for this one sync to complete; those later sync's mainly only delay bringing up nfs.
--b.
Quoted 15 lines · J. Bruce Fields
Quoted 14 lines · Steve Dickson
Quoted 4 lines · J. Bruce Fields
Quoted 2 lines
I think it would still be valuable to replace the 'sync' with two 'fsync's, one of the file, one on the directory.
Sure, may as well.--b.
Something similar to this:
diff -up nfs-utils/utils/statd/sm-notify.c.orig
nfs-utils/utils/statd/sm-notify.cutils/statd/sm-notify.c renamed from utils/statd/sm-notify.c.orig +0
-0@@ -211,12 +211,6 @@ usage: fprintf(stderr,backup_hosts (_SM_DIR_PATH, _SM_BAK_PATH);get_hosts (_SM_BAK_PATH);
Quoted 8 lines
utils/statd/sm-notify.c usage: fprintf(stderr, · line 218
Quoted 5 lines
utils/statd/sm-notify.c renamed from utils/statd/sm-notify.c.orig +0
-5-/* If there are not hosts to notify, just exit */ -if (!hosts) { -nsm_log (LOG_DEBUG ,"No hosts to notify; exiting" ); -return 0 ; - }This was still a huge boot-time win in the common case, so now that we've committed to it I'd rather not regress.
I thought the idea was the state had to be updated even when there are no hosts...
Quoted 10 lines
utils/statd/sm-notify.c nsm_get_state(int update) · line 730
Quoted 7 lines
utils/statd/sm-notify.c renamed from utils/statd/sm-notify.c.orig +2
-0@@ -730,13 +725,16 @@ nsm_get_state(int update) "Failed to write state to %s", newfile);exit (1 ); } +fsync (fd);close (fd); + fp =opendir (_SM_STATE_PATH);Also, I think you meant to:
Quoted 2 lines
fsync(fp); close(fp);
No... I don't think so... the fd is an open() of the new file and the fp is a DIR stream... But there was a cut/paste error... The fsync() of the DIR stream was missing... here is is again...
@@ -694,6 +688,7 @@ nsm_get_state(int update)
{
char newfile[PATH_MAX];
int fd, state;
+ DIR *fp;
if ((fd = open(_SM_STATE_PATH, O_RDONLY)) < 0) {
if (!opt_quiet) {
@@ -730,13 +725,18 @@ nsm_get_state(int update)
"Failed to write state to %s", newfile);
exit(1);
}
+ fsync(fd);
close(fd);
+ fp = opendir(_SM_STATE_PATH);
if (rename(newfile, _SM_STATE_PATH) < 0) {
nsm_log(LOG_ERR,
"Cannot create %s: %m", _SM_STATE_PATH);
exit(1);
}
- sync();
+ if (fp != NULL) {
+ fsync(dirfd(fp));
+ closedir(fp);
+ }
}
return state;
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 24 lines · Steve Dickson
Quoted 15 lines · J. Bruce Fields
Quoted 14 lines · Steve Dickson
Quoted 4 lines · J. Bruce Fields
Quoted 2 lines
I think it would still be valuable to replace the 'sync' with two 'fsync's, one of the file, one on the directory.
Sure, may as well.--b.
Something similar to this:
diff -up nfs-utils/utils/statd/sm-notify.c.orig
nfs-utils/utils/statd/sm-notify.cutils/statd/sm-notify.c renamed from utils/statd/sm-notify.c.orig +0
-0@@ -211,12 +211,6 @@ usage: fprintf(stderr,backup_hosts (_SM_DIR_PATH, _SM_BAK_PATH);get_hosts (_SM_BAK_PATH);Quoted 8 lines
utils/statd/sm-notify.c usage: fprintf(stderr, · line 218
Quoted 5 lines
utils/statd/sm-notify.c renamed from utils/statd/sm-notify.c.orig +0
-5-/* If there are not hosts to notify, just exit */ -if (!hosts) { -nsm_log (LOG_DEBUG ,"No hosts to notify; exiting" ); -return 0 ; - }This was still a huge boot-time win in the common case, so now that we've committed to it I'd rather not regress.
Quoted 2 lines
I thought the idea was the state had to be updated even when there are no hosts...
I would prefer to increment the state file even in the absence of hosts (it might be useful e.g. to a client that missed a previous grace period due to a network problem to know that the current boot instance is later than the one it needed to reclaim in). But I don't believe it's necessary.
The more immediate problem is that the state file no longer gets created
at all on a new system. (Unless the package install scripts do it. I
haven't checked.)
So, we could either make sure the state file gets created some other way, or delay the syncs as described before.
Quoted 10 lines
Quoted 10 lines
utils/statd/sm-notify.c nsm_get_state(int update) · line 730
Quoted 7 lines
utils/statd/sm-notify.c renamed from utils/statd/sm-notify.c.orig +2
-0@@ -730,13 +725,16 @@ nsm_get_state(int update) "Failed to write state to %s", newfile);exit (1 ); } +fsync (fd);close (fd); + fp =opendir (_SM_STATE_PATH);Also, I think you meant to:
Quoted 5 lines
Quoted 2 lines
fsync(fp); close(fp);No... I don't think so... the fd is an
open()of the new file and the fp is a DIR stream... But there was a cut/paste error... Thefsync()of the DIR stream was missing... here is is again...
OK.
Quoted 11 lines
utils/statd/sm-notify.c renamed from utils/statd/sm-notify.c.orig +1
-0@@ -694,6 +688,7 @@ nsm_get_state(int update) {char newfile[PATH_MAX ];int fd, state; +DIR *fp;if ((fd =open (_SM_STATE_PATH,O_RDONLY )) <0 ) {if (!opt_quiet) {@@ -730,13 +725,18 @@ nsm_get_state(int update) "Failed to write state to %s", newfile);exit (1 );
utils/statd/sm-notify.c nsm_get_state(int update) · line 739
Quoted 10 lines
utils/statd/sm-notify.c renamed from utils/statd/sm-notify.c.orig +2
-1} +fsync (fd);close (fd); + fp =opendir (_SM_STATE_PATH);if (rename (newfile, _SM_STATE_PATH) <0 ) {nsm_log (LOG_ERR ,"Cannot create %s: %m" , _SM_STATE_PATH);exit (1 ); } -sync ();
May as well move that opendir down here where it's used, though.
Review nsm_get_state(int update) · line 742
Quoted 7 lines
(quoted diff) +4
-0+ if (fp != NULL) { + fsync(dirfd(fp)); + closedir(fp); + } } return state;
--b.
Quoted 14 lines · J. Bruce Fields
Quoted 13 lines · Phil Endecott
Quoted 12 lines · J. Bruce Fields
Quoted 6 lines · Phil Endecott
Dear Experts,
sm-notify was taking a long time while my laptop booted. This was odd because I use NFS only rarely - via autofs - on that machine, and sm-notify actually has no-one to notify most of the time. So I have patched it as follows. Is this a legitimate thing to do?
It looks like your patch was committed to nfs-utils a couple weeks ago:
seec8d18e26d2a53d9036a32c2dafebccaf4ce1634dfrom
git://linux-nfs.org/nfs-utils
Quoted 8 lines
Quoted 5 lines
Quoted 1 line
--b.
How curious. I guess someone saw my Debian bug report. No mention of it on this list as far as I can see though.
I presume from this that it is considered a safe thing to do.
It looks right to me. Hopefully somebody actually has tested this on a client that holds locks when it reboots?
I did... things worked as expected....
steved.