fat: allocate persistent inode numbers
PATCH v2 1/5 fat: allocate persistent inode numbers
From: Namjae Jeon <[email protected]>
All the files on a FAT partition have an on-disk directory entry. The location of these entries, i_pos, is unique and is constructed by the fat_make_i_pos() function.We can use this as the inode number making it peristent across remounts.
--- fs/fat/inode.c | 5 ++++- fs/fat/namei_vfat.c | 2 ++ 2 files changed, 6 insertions(+), 1 deletion(-)
fs/fat/inode.c +4 -1
@@ -458,7 +458,10 @@ struct inode *fat_build_inode(struct super_block *sb, inode =ERR_PTR (-ENOMEM ); goto out; } - inode->i_ino =iunique (sb,MSDOS_ROOT_INO ); +if (MSDOS_SB (sb)->options.nfs) + inode->i_ino = i_pos; +else + inode->i_ino =iunique (sb,MSDOS_ROOT_INO ); inode->i_version =1 ; err =fat_fill_inode (inode, de);if (err) {
fs/fat/namei_vfat.c +2 -0
@@ -954,6 +954,8 @@ static int vfat_rename(struct inode *old_dir, struct dentry *old_dentry, new_dir->i_version++;fat_detach (old_inode); +if (MSDOS_SB (sb)->options.nfs) + old_inode->i_ino = new_i_pos;fat_attach (old_inode, new_i_pos);if (IS_DIRSYNC (new_dir)) { err =fat_sync_inode (old_inode);
Signature · 1 line
1.7.9.5
Quoted 10 lines · Namjae Jeon
From: Namjae Jeon <[email protected]>
All the files on a FAT partition have an on-disk directory entry. The location of these entries,
i_pos, is unique and is constructed by thefat_make_i_pos()function.We can use this as the inode number making it peristent across remounts.
fs/fat/namei_vfat.c static int vfat_rename(struct inode *old_dir, struct dentry *old_dentry, · line 958
Quoted 5 lines
fs/fat/namei_vfat.c +2
-0@@ -954,6 +954,8 @@ static int vfat_rename(struct inode *old_dir, struct dentry *old_dentry, new_dir->i_version++;fat_detach (old_inode); +if (MSDOS_SB (sb)->options.nfs) + old_inode->i_ino = new_i_pos;
Sigh... Inode numbers are reported by fstat() in stat.st_ino. They must
* remain constant from open() to close(), even if file gets
unlinked or renamed.
* be equal for two simultaneously opened descriptors with the samest_dev *ONLY* if those descriptors refer to the same file (i.e. if writing through one of those would change the data read through another, etc.)
And yes, the userland code does depend on those properties. There's a damn good reason why we had gone for all those convolutions with separate hash, etc.
inode->i_ino on a live struct inode is _never_ changed. Period.
2012/9/5, Al Viro <[email protected]>:
Quoted 14 lines · Namjae Jeon
Quoted 13 lines
From: Namjae Jeon <[email protected]>
All the files on a FAT partition have an on-disk directory entry. The location of these entries,
i_pos, is unique and is constructed by thefat_make_i_pos()function.We can use this as the inode number making it peristent across remounts.fs/fat/namei_vfat.c +0
-0@@ -954,6 +954,8 @@ static int vfat_rename(struct inode *old_dir, struct dentry *old_dentry, new_dir->i_version++;
Quoted 4 lines
fs/fat/namei_vfat.c static int vfat_rename(struct inode *old_dir, struct dentry *old_dentry, · line 958
Quoted 3 lines
fs/fat/namei_vfat.c +2
-0fat_detach (old_inode); +if (MSDOS_SB (sb)->options.nfs) + old_inode->i_ino = new_i_pos;
A brief background to this patch-set : We are using FAT over NFS in our environment. While App just is browsing files(ls -lR, etc..) on the client we observed a numerous times when the server cache was evicted.
Our purpose was same as mentioned by Neil Brown also -> "that it is important to maintain support for NFS export of VFAT on a best-effort basis.
With the patch series we are able to provide a 100% safe solution -> there is no ESTALE issues at least for the normal user scenarios (i.e, rename, unlink do not happen while the file is open). After the review comments from you, we proposed a new solution which takes care of 'unlink' also.
Quoted 7 lines
Sigh... Inode numbers are reported by
fstat()in stat.st_ino. They must
* remain constant fromopen()toclose(), even if file gets
unlinked or renamed.
* be equal for two simultaneously opened descriptors with the samest_dev*ONLY* if those descriptors refer to the same file (i.e. if writing through one of those would change the data read through another, etc.)
Quoted 5 lines
And yes, the userland code does depend on those properties. There's a damn good reason why we had gone for all those convolutions with separate hash, etc.
inode->i_inoon a live struct inode is _never_ changed. Period.
Hi Al.
Even without these patches, when a file is opened at the client and is still 'live',
it is possible for the inode number to change due to cache eviction at server. (because each read/write NFS transaction from client is translated into a open-> read/write->close operation and in between such transactions, the server cache may be evicted). This is why we updated the i_ino in vfat_rename() immediately, as it helps rebuild the inode in such cases.
But to comply with your explanation(constant i_ino from open() till close() ), I can remove this change.
And
Hi. OGAWA.
In this long discusstion about the FAT acceptance over NFS, our belief is still that
the objective should be to reduce errors as much as possible and then if there are
certain scenarios - at least that could be highlighted as a limitation in Documentation
instead of completely discarding the usage of FAT over NFS.
So how about puttting rename scenario as a limitation ? In ideal scenario how many
times this is going to happen ?
Thanks.
Quoted 1 line
OGAWA Hirofumi
maintainerQuoted 7 lines · Namjae Jeon
In this long discusstion about the FAT acceptance over NFS, our belief is still that the objective should be to reduce errors as much as possible and then if there are certain scenarios - at least that could be highlighted as a limitation in Documentation instead of completely discarding the usage of FAT over NFS. So how about puttting rename scenario as a limitation ? In ideal scenario how many times this is going to happen ?
My understanding of your patches is to introduce the silent corruption bug by getting wrong location via ino on some cases, instead of ESTALE. And make surprise to userland by change ino.
Why is it safe to change ino? If you are saying to remove the changing ino on rename, how handle the case of collision?
Signature · 1 line
OGAWA Hirofumi <[email protected]>
2012/9/5 OGAWA Hirofumi <[email protected]>:
Quoted 10 lines
Quoted 7 lines · Namjae Jeon
In this long discusstion about the FAT acceptance over NFS, our belief is still that the objective should be to reduce errors as much as possible and then if there are certain scenarios - at least that could be highlighted as a limitation in Documentation instead of completely discarding the usage of FAT over NFS. So how about puttting rename scenario as a limitation ? In ideal scenario how many times this is going to happen ?
Quoted 6 lines
My understanding of your patches is to introduce the silent corruption bug by getting wrong location via ino on some cases, instead of ESTALE. And make surprise to userland by change ino.
Why is it safe to change ino? If you are saying to remove the changing ino on rename, how handle the case of collision?
Yes, agreed this would lead to collision. So, If we are choosing 'i_pos' as inode
number, We need to have a mechanism to avoid this 'i_pos' being reused.
We can have one thing over here. As a part of avoidance for such scenarios - We can return EBUSY for this rename operation. i.e., If the inode is being referenced then in such cases it makes sense to return EBUSY over NFS and
forcus on the large part of the solution which is making FAT stable.
Let me know your opinion.
Thanks OGAWA.
Quoted 2 lines
--
OGAWA Hirofumi <[email protected]>
OGAWA Hirofumi
maintainerQuoted 18 lines · Namjae Jeon
Quoted 8 lines
Quoted 7 lines
In this long discusstion about the FAT acceptance over NFS, our belief is still that the objective should be to reduce errors as much as possible and then if there are certain scenarios - at least that could be highlighted as a limitation in Documentation instead of completely discarding the usage of FAT over NFS. So how about puttting rename scenario as a limitation ? In ideal scenario how many times this is going to happen ?
Quoted 6 lines
My understanding of your patches is to introduce the silent corruption bug by getting wrong location via ino on some cases, instead of ESTALE. And make surprise to userland by change ino.
Why is it safe to change ino? If you are saying to remove the changing ino on rename, how handle the case of collision?
Yes, agreed this would lead to collision. So, If we are choosing '
i_pos' as inode
number, We need to have a mechanism to avoid this 'i_pos' being reused.
Quoted 6 lines
We can have one thing over here. As a part of avoidance for such scenarios - We can return EBUSY for this rename operation. i.e., If the inode is being referenced then in such cases it makes sense to return EBUSY over NFS and
forcus on the large part of the solution which is making FAT stable.Let me know your opinion.
It sounds like sane to provide the limited but stable behavior, with the option. But at first, I'd like to see the read-only export but clean solution, and make it stable. (I'm not thinking about the implementation detail. It may be the stable ino solution, or may not be.)
After that, we can make it writable incrementally.
Signature · 1 line
OGAWA Hirofumi <[email protected]>
2012/9/6 OGAWA Hirofumi <[email protected]>:
Quoted 25 lines
Quoted 23 lines · Namjae Jeon
Quoted 8 lines
Quoted 7 lines
In this long discusstion about the FAT acceptance over NFS, our belief is still that the objective should be to reduce errors as much as possible and then if there are certain scenarios - at least that could be highlighted as a limitation in Documentation instead of completely discarding the usage of FAT over NFS. So how about puttting rename scenario as a limitation ? In ideal scenario how many times this is going to happen ?
Quoted 6 lines
My understanding of your patches is to introduce the silent corruption bug by getting wrong location via ino on some cases, instead of ESTALE. And make surprise to userland by change ino.
Why is it safe to change ino? If you are saying to remove the changing ino on rename, how handle the case of collision?
Yes, agreed this would lead to collision. So, If we are choosing '
i_pos' as inode
number, We need to have a mechanism to avoid this 'i_pos' being reused.We can have one thing over here. As a part of avoidance for such scenarios - We can return EBUSY for this rename operation. i.e., If the inode is being referenced then in such cases it makes sense to return EBUSY over NFS and
forcus on the large part of the solution which is making FAT stable.
Quoted 6 lines
Quoted 1 line
Let me know your opinion.
It sounds like sane to provide the limited but stable behavior, with the option. But at first, I'd like to see the read-only export but clean solution, and make it stable. (I'm not thinking about the implementation detail. It may be the stable ino solution, or may not be.)
Hi. OGAWA.
I Agree. Once, I will check whether it is possible to implement read-only export or not.
And If I face some problem or have some concern, I will discuss about it with you.
Quoted 2 lines
After that, we can make it writable incrementally.
Okay.
Thanks a lot.
Quoted 2 lines
--
OGAWA Hirofumi <[email protected]>
Hi. OGAWA.
I checked read-only option for export on FAT. I think that there are 3 approaches as mentioned below.
1. As per the current scenario – user already has the option of setting ‘ro’ in /etc/exports – so that can also be used to make it read-only.
2. Forcefully set to “read-only” while executing FAT export operation. -> As you know, we can set read-only(ro) export in /etc/exports. If we set read-only export regardless of /etc/exports, This is "HACK" and it will work regardless of user setting.
3. When FAT is mounted with -onfs option,-> Make it ‘ro’ at the mount time itself.
-> It is simple to implement, but VFAT of NFS Server will be set to read-only as well as NFS client.
4. As already suggested – only problem left now is of ‘RENAME’ all other points can be taken care in the current solution. So, how about putting RENAME as a limitation? or we can return EBUSY about rename case to avoid collision.
Let me know which approach you prefer. Especially through more insights on ‘rename’ part as all through the discussion this has been the bottleneck part left now. And sincerely fruitful discussion on this ‘rename’ might result in stable FAT over NFS.
Please let me know your opinion.
Thanks.
2012/9/6, Namjae Jeon <[email protected]>:
Quoted 48 lines
2012/9/6 OGAWA Hirofumi <[email protected]>:
Quoted 28 lines
Quoted 26 lines · Namjae Jeon
Quoted 8 lines
Quoted 7 lines
In this long discusstion about the FAT acceptance over NFS, our belief is still that the objective should be to reduce errors as much as possible and then if there are certain scenarios - at least that could be highlighted as a limitation in Documentation instead of completely discarding the usage of FAT over NFS. So how about puttting rename scenario as a limitation ? In ideal scenario how many times this is going to happen ?
Quoted 6 lines
My understanding of your patches is to introduce the silent corruption bug by getting wrong location via ino on some cases, instead of ESTALE. And make surprise to userland by change ino.
Why is it safe to change ino? If you are saying to remove the changing ino on rename, how handle the case of collision?
Yes, agreed this would lead to collision. So, If we are choosing '
i_pos' as inode
number, We need to have a mechanism to avoid this 'i_pos' being reused.We can have one thing over here. As a part of avoidance for such scenarios -
We can return EBUSY for this rename operation. i.e., If the inode is being
referenced then in such cases it makes sense to return EBUSY over NFS and
forcus on the large part of the solution which is making FAT stable.Quoted 6 lines
Quoted 1 line
Let me know your opinion.
It sounds like sane to provide the limited but stable behavior, with the option. But at first, I'd like to see the read-only export but clean solution, and make it stable. (I'm not thinking about the implementation detail. It may be the stable ino solution, or may not be.)
Hi. OGAWA.
I Agree. Once, I will check whether it is possible to implement read-only export or not.
And If I face some problem or have some concern, I will discuss about it with you.Quoted 2 lines
After that, we can make it writable incrementally.
Okay.
Thanks a lot.Quoted 2 lines
--
OGAWA Hirofumi <[email protected]>
Quoted 14 lines · Namjae Jeon
Hi. OGAWA.
I checked read-only option for export on FAT. I think that there are 3 approaches as mentioned below.
1. As per the current scenario – user already has the option of setting ‘ro’ in
/etc/exports– so that can also be used to make it read-only.2. Forcefully set to “read-only” while executing FAT export operation. -> As you know, we can set read-only(ro) export in
/etc/exports. If we set read-only export regardless of/etc/exports, This is "HACK" and it will work regardless of user setting.
Quoted 4 lines
3. When FAT is mounted with -onfs option,-> Make it ‘ro’ at the mount time itself.
-> It is simple to implement, but VFAT of NFS Server will be set to read-only as well as NFS client.
I argue against (2) and (3). A change that drops any possibility of NFS-mounting VFAT filesystems read-write will break my use case. Where ESTALE is an issue, there are client-side solutions, either mounting with lookupcache=none (which admittedly has a severe performance impact) or the VFS patches to handle ESTALE that are working their way towards mainline. I recognize that not everyone can take advantage of client-side features, but options (2) and (3) make life worse for those who can.
------------------------------------------------------------------------
Steven J. Magnani "I claim this network for MARS!
www.digidescorp.com Earthling, return my space modulator!"
#include <standard.disclaimer>
OGAWA Hirofumi
maintainerQuoted 20 lines · Steven J. Magnani
Quoted 14 lines · Namjae Jeon
Hi. OGAWA.
I checked read-only option for export on FAT. I think that there are 3 approaches as mentioned below.
1. As per the current scenario – user already has the option of setting ‘ro’ in
/etc/exports– so that can also be used to make it read-only.2. Forcefully set to “read-only” while executing FAT export operation. -> As you know, we can set read-only(ro) export in
/etc/exports. If we set read-only export regardless of/etc/exports, This is "HACK" and it will work regardless of user setting.Quoted 4 lines
3. When FAT is mounted with -onfs option,-> Make it ‘ro’ at the mount time itself.
-> It is simple to implement, but VFAT of NFS Server will be set to read-only as well as NFS client.
Quoted 8 lines
I argue against (2) and (3). A change that drops any possibility of NFS-mounting VFAT filesystems read-write will break my use case. Where ESTALE is an issue, there are client-side solutions, either mounting with lookupcache=none (which admittedly has a severe performance impact) or the VFS patches to handle ESTALE that are working their way towards mainline. I recognize that not everyone can take advantage of client-side features, but options (2) and (3) make life worse for those who can.
What is your use case? I'm assuming current NFS support of FAT is not unstable behavior even with your patches. Is this true?
Well, this plan is to provide the stable/clean read-only behavior at first. After that, make it writable with some limitations (e.g. rename may be unsupported).
If your patches in -mm is enough for now, we will not need to do those. Namjae, were you tested it? or what are you thinking?
Signature · 1 line
OGAWA Hirofumi <[email protected]>
OGAWA Hirofumi
maintainerQuoted 2 lines · OGAWA Hirofumi
What is your use case? I'm assuming current NFS support of FAT is not unstable behavior even with your patches. Is this true?
s/is not unstable/is still unstable/
Signature · 1 line
OGAWA Hirofumi <[email protected]>
2012/9/9, OGAWA Hirofumi <[email protected]>:
Quoted 4 lines
Quoted 2 lines · OGAWA Hirofumi
What is your use case? I'm assuming current NFS support of FAT is still unstable behavior even with your patches. Is this true?
Hi. OGAWA.
Yes, It is true(current VFAT of -mm tree is not stable). Although we set lookupcache=none while mounting, ESTALE error can still occur in rename case.
So there still remain ESTALE error issue from rename case on current -mm tree. plz See the step as the following
1. on client write to file.
2. on client, move/rename file.
3. on server, do drop_caches. etc to somehow evict indoe number so that it gets new inode number
4. on client, resume the program to write to file. write will fail (write: Stale NFS file handle)
-----
As I know, I think that Steve patch tried to fix ESTALE error on best-effort. And Steve said also his patch can perfectly not avoid ESTLE error. And The aim of Steve's patch can be just improved under memory pressure(dentry eviction) on best-effort.
------------------------------------------------------------------------------
Under memory pressure, the system may evict dentries from cache. When the FAT driver receives a NFS request involving an evicted dentry, it is unable to reconnect it to the filesystem root.
This causes the request to fail, often with ENOENT.
.....
Note that while this patch set improves FAT's NFS support, it does not eliminate ESTALE errors completely.
The following should be considered for NFS clients who are sensitive to ESTALE:
* Mounting with lookupcache=none
Unfortunately this can degrade performance severely, particularly for deep filesystems.
* Incorporating VFS patches to retry ESTALE failures on the client-side, such as https://lkml.org/lkml/2012/6/29/381
* Handling ESTALE errors in client application code
-------------------------------------------------------------------------------------
And ......
If we mount NFS with lookupcache=none, FAT file lookup performance is severely dropped.
LOOKUP performance is very poor on slow network and slow device. I do not recommend to disable lookup cache on NFS.
And that is why reconstructing inode is already implemented in other filesystem (e.g. EXT4, XFS etc..)
Currently lookupcache is enabled by default in NFS, it means users already have disclosed and experienced ESTALE issues on NFS over VFAT.
I agree wth you to make NFS over VFAT read-only filesystem to avoid all issues. Eventually we can make it writable with rename limitation when we decide that it is pretty stable in mainline.
So, I suggest to add 'nfs_ro' mount option instead of 'nfs' option.
Let me know your opinion.
Thanks.
Quoted 5 lines
s/is not unstable/is still unstable/
--
OGAWA Hirofumi <[email protected]>
OGAWA Hirofumi
maintainerQuoted 11 lines · Namjae Jeon
Yes, It is true(current VFAT of -mm tree is not stable). Although we set lookupcache=none while mounting, ESTALE error can still occur in rename case.
So there still remain ESTALE error issue from rename case on current -mm tree. plz See the step as the following
1. on client write to file.
2. on client, move/rename file.
3. on server, dodrop_caches. etc to somehow evict indoe number so that it gets new inode number
4. on client, resume the program to write to file. write will fail (write: Stale NFS file handle)
Since rename() will be disabled on stable ino patches, this will be unfixable, so rather maybe it is worse.
Did you checked why it returns -ESTALE? Or rename() issue also is unfixable on -mm?
Quoted 10 lines
And ......
If we mount NFS with lookupcache=none, FAT file lookup performance is severely dropped.
LOOKUP performance is very poor on slow network and slow device. I do not recommend to disable lookup cache on NFS.
And that is why reconstructing inode is already implemented in other filesystem (e.g. EXT4, XFS etc..)
Currently lookupcache is enabled by default in NFS, it means users already have disclosed and experienced ESTALE issues on NFS over VFAT.
Quoted 4 lines
I agree wth you to make NFS over VFAT read-only filesystem to avoid all issues. Eventually we can make it writable with rename limitation when we decide that it is pretty stable in mainline.
So, I suggest to add 'nfs_ro' mount option instead of 'nfs' option.
-mm seems to be more stable than I thought. As he said, sounds like rename() is an only known issue on -mm, true?
And are you tried https://lkml.org/lkml/2012/6/29/381 patches? It sounds like to improve performance by enabling lookupcache. I'd like to be knowing the critical reason we have to replace it.
Thanks.
Signature · 1 line
OGAWA Hirofumi <[email protected]>
2012/9/10, OGAWA Hirofumi <[email protected]>:
Quoted 15 lines
Quoted 4 lines · Namjae Jeon
Yes, It is true(current VFAT of -mm tree is not stable). Although we set lookupcache=none while mounting, ESTALE error can still occur in rename case.
So there still remain ESTALE error issue from rename case on current -mmQuoted 8 lines
tree.
plz See the step as the following
1. on client write to file.
2. on client, move/rename file.
3. on server, dodrop_caches. etc to somehow evict indoe number so that it gets new inode number
4. on client, resume the program to write to file. write will fail (write: Stale NFS file handle)
Hi OGAWA.
Quoted 2 lines
Since
rename()will be disabled on stable ino patches, this will be unfixable, so rather maybe it is worse.
Currently with our patchset : only rename issue (could not find any correct approach to ignore this. If we do not update this immediately at i_pos change – it is just delaying the problem). And we can return EBUSY when rename is called while process is opening file with rename limitation. Without our patchset also - the rename issue can occur over NFS file access - when the inode is evicted from the SERVER cache.
Quoted 3 lines
Did you checked why it returns
-ESTALE? Orrename()issue also is unfixable on -mm?
It is reproducible regardless of lookupcache is enable or disable.
The inode is not found in server inode cache. So whend_obtain_alias(inode) is called, it returns ESTALE.
Call path like this. fh_verify()-->nfsd_set_fh_dentry()-->exportfs_decode_fh()-->nop->fh_to_dentry()-->fat_fh_to_dentry()-->generic_fh_to_dentry()-->get_inode()-->fat_nfs_get_inode()
static struct inode *fat_nfs_get_inode(struct super_block *sb,
u64 ino, u32 generation)
{
......
inode = ilookup(sb, ino); ->This looks up in inode cache and
returns null
if (inode && generation && (inode->i_generation != generation)) {
iput(inode);
inode = NULL;
}
return inode;
}I think that it is unfixable because we can not know i_pos of inode changed by rename.
And even though we know it, there is no rebuild inode routine in -mm. And It even can not fix in our patches.
Quoted 11 lines
Quoted 11 lines
And ......
If we mount NFS with lookupcache=none, FAT file lookup performance is severely dropped.
LOOKUP performance is very poor on slow network and slow device. I do not recommend to disable lookup cache on NFS.
And that is why reconstructing inode is already implemented in other filesystem (e.g. EXT4, XFS etc..)
Currently lookupcache is enabled by default in NFS, it means users already have disclosed and experienced ESTALE issues on NFS over VFAT.
Quoted 8 lines
Quoted 5 lines
I agree wth you to make NFS over VFAT read-only filesystem to avoid all issues.
Eventually we can make it writable with rename limitation when we decide that it is pretty stable in mainline.
So, I suggest to add 'nfs_ro' mount option instead of 'nfs' option.-mm seems to be more stable than I thought. As he said, sounds like
rename()is an only known issue on -mm, true?
Yes, There is only rename issue in stability if we use lookcache is disable. But performance will severely be dropped
But If lookup cache is enable, there are estale and rename issue in -mm.
Quoted 3 lines
And are you tried https://lkml.org/lkml/2012/6/29/381 patches? It sounds like to improve performance by enabling lookupcache.
We checked this patches when facing estale issue in -mm. But It is no use, these patches just retry system call one more when estale error.
Quoted 1 line
I'd like to be knowing the critical reason we have to replace it.
I arrange to help your decision as the following.
1. lookup cache is enable at default in NFS. So estale error can be easily occurred in -mm.
2. If lookup cache is disable, there is rename issue and file lookup performance is dropped in -mm.
4. If we use our patches, there is rename issue. but we can use VFAT over NFS with lookup cache enable.
5. If we use read-only with our patches, there is no issue.
Thanks.
Quoted 5 lines
Thanks.
--
OGAWA Hirofumi <[email protected]>
OGAWA Hirofumi
maintainerQuoted 9 lines · Namjae Jeon
Quoted 2 lines
Since
rename()will be disabled on stable ino patches, this will be unfixable, so rather maybe it is worse.Currently with our patchset : only rename issue (could not find any correct approach to ignore this. If we do not update this immediately at
i_poschange – it is just delaying the problem). And we can return EBUSY when rename is called while process is opening file with rename limitation. Without our patchset also - the rename issue can occur over NFS file access - when the inode is evicted from the SERVER cache.
Important difference is whether rename issue is unfixable or not.
Quoted 10 lines
I think that it is unfixable because we can not know
i_posof inode changed by rename.
And even though we know it, there is no rebuild inode routine in -mm. And It even can not fix in our patches.Quoted 2 lines
And are you tried https://lkml.org/lkml/2012/6/29/381 patches? It sounds like to improve performance by enabling lookupcache.
We checked this patches when facing estale issue in -mm. But It is no use, these patches just retry system call one more when estale error.
What happens if client retried from lookup() after -ESTALE? (client NFS doesn't have the name of entry anymore?)
I'm assuming the retry means - it restarts from building the NFS file handle. I might be just wrong here though.
Quoted 10 lines
Quoted 1 line
I'd like to be knowing the critical reason we have to replace it.
I arrange to help your decision as the following.
1. lookup cache is enable at default in NFS. So estale error can be easily occurred in -mm.
2. If lookup cache is disable, there is rename issue and file lookup performance is dropped in -mm.
4. If we use our patches, there is rename issue. but we can use VFAT over NFS with lookup cache enable.
5. If we use read-only with our patches, there is no issue.
Again, I'm care about whether rename issue is unfixable or not. In stable ino patches, it will never be fixable.
What do you think about this rename issue, Steven?
Signature · 1 line
OGAWA Hirofumi <[email protected]>
2012/9/11 OGAWA Hirofumi <[email protected]>:
Quoted 12 lines
Quoted 9 lines · Namjae Jeon
Quoted 2 lines
Since
rename()will be disabled on stable ino patches, this will be unfixable, so rather maybe it is worse.Currently with our patchset : only rename issue (could not find any correct approach to ignore this. If we do not update this immediately at
i_poschange – it is just delaying the problem). And we can return EBUSY when rename is called while process is opening file with rename limitation. Without our patchset also - the rename issue can occur over NFS file access - when the inode is evicted from the SERVER cache.
Quoted 1 line
Important difference is whether rename issue is unfixable or not.
Currently I don't have any idea to fix rename issue in -mm.
Quoted 6 lines
Quoted 5 lines
I think that it is unfixable because we can not know
i_posof inode changed by rename.
And even though we know it, there is no rebuild inode routine in -mm. And It even can not fix in our patches.
Quoted 8 lines
Quoted 5 lines
Quoted 2 lines
And are you tried https://lkml.org/lkml/2012/6/29/381 patches? It sounds like to improve performance by enabling lookupcache.
We checked this patches when facing estale issue in -mm. But It is no use, these patches just retry system call one more when estale error.
What happens if client retried from
lookup()after-ESTALE? (client NFS doesn't have the name of entry anymore?)
Need to rebuild inode routine because inode cache is already evicted on Server.
Quoted 3 lines
I'm assuming the retry means - it restarts from building the NFS file handle. I might be just wrong here though.
As I remember, just retry in VFS of NFS client..I heard this patch is needed for
a very specific set of circumstances where an entry goes stale once between the lookup and the actual operation(s).
It is not related with current issues(inode cache eviction on server).
Quoted 12 lines
Quoted 11 lines
Quoted 2 lines
I'd like to be knowing the critical reason we have to replace it.
I arrange to help your decision as the following.
1. lookup cache is enable at default in NFS. So estale error can be easily occurred in -mm.
2. If lookup cache is disable, there is rename issue and file lookup performance is dropped in -mm.
4. If we use our patches, there is rename issue. but we can use VFAT over NFS with lookup cache enable.
5. If we use read-only with our patches, there is no issue.
Quoted 1 line
Again, I'm care about whether rename issue is unfixable or not.
I think that It is unfixable in -mm.
Quoted 6 lines
In stable ino patches, it will never be fixable.
What do you think about this rename issue, Steven? --
OGAWA Hirofumi <[email protected]>
OGAWA Hirofumi
maintainerQuoted 15 lines · Namjae Jeon
Quoted 5 lines
Quoted 4 lines
I think that it is unfixable because we can not know
i_posof inode changed by rename.
And even though we know it, there is no rebuild inode routine in -mm. And It even can not fix in our patches.Quoted 8 lines
Quoted 5 lines
Quoted 2 lines
And are you tried https://lkml.org/lkml/2012/6/29/381 patches? It sounds like to improve performance by enabling lookupcache.
We checked this patches when facing estale issue in -mm. But It is no use, these patches just retry system call one more when estale error.
What happens if client retried from
lookup()after-ESTALE? (client NFS doesn't have the name of entry anymore?)Need to rebuild inode routine because inode cache is already evicted on Server.
Quoted 1 line
Quoted 7 lines
Quoted 2 lines
I'm assuming the retry means - it restarts from building the NFS file handle. I might be just wrong here though.
As I remember, just retry in VFS of NFS client..I heard this patch is needed for
a very specific set of circumstances where an entry goes stale once between the lookup and the actual operation(s).
It is not related with current issues(inode cache eviction on server).
Supposing, the server/client state is after cold boot, and client try to rename at first without any cache on client/server.
Even if this state, does the server return ESTALE? If it doesn't return ESTALE, I can't understand why it is really unfixable.
If it returns ESTALE, why does it return? I'm assuming the previous code path is the cached FH path.
Signature · 1 line
OGAWA Hirofumi <[email protected]>
2012/9/12 OGAWA Hirofumi <[email protected]>:
Quoted 25 lines
Quoted 15 lines · Namjae Jeon
Quoted 5 lines
Quoted 4 lines
I think that it is unfixable because we can not know
i_posof inode changed by rename.
And even though we know it, there is no rebuild inode routine in -mm. And It even can not fix in our patches.Quoted 8 lines
Quoted 5 lines
Quoted 2 lines
And are you tried https://lkml.org/lkml/2012/6/29/381 patches? It sounds like to improve performance by enabling lookupcache.
We checked this patches when facing estale issue in -mm. But It is no use, these patches just retry system call one more when estale error.
What happens if client retried from
lookup()after-ESTALE? (client NFS doesn't have the name of entry anymore?)Need to rebuild inode routine because inode cache is already evicted on Server.
Quoted 1 line
Quoted 7 lines
Quoted 2 lines
I'm assuming the retry means - it restarts from building the NFS file handle. I might be just wrong here though.
As I remember, just retry in VFS of NFS client..I heard this patch is needed for
a very specific set of circumstances where an entry goes stale once between the lookup and the actual operation(s).
It is not related with current issues(inode cache eviction on server).
Quoted 5 lines
Supposing, the server/client state is after cold boot, and client try to rename at first without any cache on client/server.
Even if this state, does the server return ESTALE? If it doesn't return ESTALE, I can't understand why it is really unfixable.
Hi OGAWA.
Server will not return ESTALE in this case. because the client does not have any information for files yet.
I mean NFS client does not have any old NFS FH(containing old inode number) for this.
Quoted 3 lines
If it returns ESTALE, why does it return? I'm assuming the previous code path is the cached FH path.
The main point for observation is the file handle-which is used for all the NFS operation.
So for all the NFS operation(read/write....) which makes use of the NFS file handle in between if there is a change in inode number It will result in ESTALE.
Changing inode number on rename happened at NFS server by inode cache eviction with memory pressure.
lookupcache is used at NFS client to reduce number of LOOKUP operations. But , we can still get ESTALE if inode number at NFS Server change after LOOKUP, although lookupcache is disable.
LOOKUP return NFS FH->[inode number changed at NFS Server] -> But we still use old NFS FH returned from LOOKUP for any file operation(write,read,etc..)
-> ESTALE will be returned.
Thanks!
Quoted 2 lines
--
OGAWA Hirofumi <[email protected]>
Quoted 26 lines · Namjae Jeon
2012/9/12 OGAWA Hirofumi <[email protected]>:
Quoted 25 lines
Quoted 22 lines · Namjae Jeon
Quoted 5 lines
Quoted 4 lines
I think that it is unfixable because we can not know
i_posof inode changed by rename.
And even though we know it, there is no rebuild inode routine in -mm. And It even can not fix in our patches.Quoted 8 lines
Quoted 5 lines
Quoted 2 lines
And are you tried https://lkml.org/lkml/2012/6/29/381 patches? It sounds like to improve performance by enabling lookupcache.
We checked this patches when facing estale issue in -mm. But It is no use, these patches just retry system call one more when estale error.
What happens if client retried from
lookup()after-ESTALE? (client NFS doesn't have the name of entry anymore?)Need to rebuild inode routine because inode cache is already evicted on Server.
Quoted 3 lines
I'm assuming the retry means - it restarts from building the NFS file handle. I might be just wrong here though.
As I remember, just retry in VFS of NFS client..I heard this patch is needed for
a very specific set of circumstances where an entry goes stale once between the lookup and the actual operation(s).
It is not related with current issues(inode cache eviction on server).
Quoted 8 lines
Quoted 5 lines
Supposing, the server/client state is after cold boot, and client try to rename at first without any cache on client/server.
Even if this state, does the server return ESTALE? If it doesn't return ESTALE, I can't understand why it is really unfixable.
Hi OGAWA.
Server will not return ESTALE in this case. because the client does not have any information for files yet.
It does if the client mounted before the server rebooted. NFS is designed so that servers can reboot without causing clients to fail. (Applications will just see a delay during the reboot.)
It probably isn't possible to this work in the case of fat.
But from fat's point of view there probably isn't much difference between a filehandle lookup after a reboot and a filehandle lookup after the inode's gone from cache.
I really don't see what you can do to help here. Won't anything that allows looking up an uncached inode by filehandle also risk finding the wrong file?
(If looking up the same filehandle ever results in finding a *different* file from before, that's a bug. Probably a more dangerous bug than an ESTALE--in the ESTALE case the failure is obvious whereas in the case where you get the wrong file, you may silently corrupt data.)
--b.
Quoted 26 lines
I mean NFS client does not have any old NFS FH(containing old inode number) for this.
Quoted 3 lines
If it returns ESTALE, why does it return? I'm assuming the previous code path is the cached FH path.
The main point for observation is the file handle-which is used for all the NFS operation.
So for all the NFS operation(read/write....) which makes use of the NFS file handle in between if there is a change in inode number It will result in ESTALE.
Changing inode number on rename happened at NFS server by inode cache eviction with memory pressure.lookupcache is used at NFS client to reduce number of LOOKUP operations. But , we can still get ESTALE if inode number at NFS Server change after LOOKUP, although lookupcache is disable.
LOOKUP return NFS FH->[inode number changed at NFS Server] -> But we still use old NFS FH returned from LOOKUP for any file
operation(write,read,etc..)
-> ESTALE will be returned.Thanks!
Quoted 2 lines
--
OGAWA Hirofumi <[email protected]>
OGAWA Hirofumi
maintainerQuoted 13 lines · J. Bruce Fields
Quoted 8 lines
Quoted 5 lines
Supposing, the server/client state is after cold boot, and client try to rename at first without any cache on client/server.
Even if this state, does the server return ESTALE? If it doesn't return ESTALE, I can't understand why it is really unfixable.
Hi OGAWA.
Server will not return ESTALE in this case. because the client does not have any information for files yet.It does if the client mounted before the server rebooted. NFS is designed so that servers can reboot without causing clients to fail. (Applications will just see a delay during the reboot.)
Quoted 5 lines
It probably isn't possible to this work in the case of fat.
But from fat's point of view there probably isn't much difference between a filehandle lookup after a reboot and a filehandle lookup after the inode's gone from cache.
This is talking about to retry by client side, not server side solution. What happens if client retry after got ESTALE? (Yes, this would not be the solution for all NFS clients. But, I guess this solution can be for linux NFS client.)
Quoted 3 lines
I really don't see what you can do to help here. Won't anything that allows looking up an uncached inode by filehandle also risk finding the wrong file?
On other view (as server side solution), we are thinking there is possible to make the stable filehandle on FAT if we disabled some operations (e.g. rename(), unlink()) which change inode location in FAT.
Yes, it would be stable, but supporting limited operations.
This is server side solution, and we comparing it with client solution.
Quoted 10 lines
Quoted 10 lines
Quoted 2 lines
If it returns ESTALE, why does it return? I'm assuming the previous code path is the cached FH path.
The main point for observation is the file handle-which is used for all the NFS operation.
So for all the NFS operation(read/write....) which makes use of the NFS file handle in between if there is a change in inode number It will result in ESTALE.
Changing inode number on rename happened at NFS server by inode cache eviction with memory pressure.
Quoted 8 lines
Quoted 8 lines
lookupcache is used at NFS client to reduce number of LOOKUP operations. But , we can still get ESTALE if inode number at NFS Server change after LOOKUP, although lookupcache is disable.
LOOKUP return NFS FH->[inode number changed at NFS Server] -> But we still use old NFS FH returned from LOOKUP for any file
operation(write,read,etc..)
-> ESTALE will be returned.
Yes. And I'm expecting as client side solution,
-> ESTALE will be returned -> discard old FH -> restart from LOOKUP -> make cached inode -> use returned new FH.
Yeah, I know this is unstable (there is no perfect solution for now), but if this works, this doesn't limit any operations.
We would want to compare client solution (-mm) and server solution (stable ino). Or I'd like to know which my knowledges/understanding are wrong here.
Signature · 1 line
OGAWA Hirofumi <[email protected]>
Quoted 30 lines · OGAWA Hirofumi
Quoted 13 lines · J. Bruce Fields
Quoted 8 lines
Quoted 5 lines
Supposing, the server/client state is after cold boot, and client try to rename at first without any cache on client/server.
Even if this state, does the server return ESTALE? If it doesn't return ESTALE, I can't understand why it is really unfixable.
Hi OGAWA.
Server will not return ESTALE in this case. because the client does not have any information for files yet.It does if the client mounted before the server rebooted. NFS is designed so that servers can reboot without causing clients to fail. (Applications will just see a delay during the reboot.)
Quoted 5 lines
It probably isn't possible to this work in the case of fat.
But from fat's point of view there probably isn't much difference between a filehandle lookup after a reboot and a filehandle lookup after the inode's gone from cache.
This is talking about to retry by client side, not server side solution. What happens if client retry after got ESTALE? (Yes, this would not be the solution for all NFS clients. But, I guess this solution can be for linux NFS client.)
Quoted 3 lines
I really don't see what you can do to help here. Won't anything that allows looking up an uncached inode by filehandle also risk finding the wrong file?
Quoted 7 lines
On other view (as server side solution), we are thinking there is possible to make the stable filehandle on FAT if we disabled some operations (e.g.
rename(),unlink()) which change inode location in FAT.Yes, it would be stable, but supporting limited operations.
This is server side solution, and we comparing it with client solution.
Is that useful to anyone?
Quoted 19 lines
Quoted 10 lines
Quoted 10 lines
Quoted 2 lines
If it returns ESTALE, why does it return? I'm assuming the previous code path is the cached FH path.
The main point for observation is the file handle-which is used for all the NFS operation.
So for all the NFS operation(read/write....) which makes use of the NFS file handle in between if there is a change in inode number It will result in ESTALE.
Changing inode number on rename happened at NFS server by inode cache eviction with memory pressure.Quoted 8 lines
Quoted 8 lines
lookupcache is used at NFS client to reduce number of LOOKUP operations. But , we can still get ESTALE if inode number at NFS Server change after LOOKUP, although lookupcache is disable.
LOOKUP return NFS FH->[inode number changed at NFS Server] -> But we still use old NFS FH returned from LOOKUP for any file
operation(write,read,etc..)
-> ESTALE will be returned.
Quoted 6 lines
Yes. And I'm expecting as client side solution,
-> ESTALE will be returned -> discard old FH -> restart from LOOKUP -> make cached inode -> use returned new FH.
Yeah, I know this is unstable (there is no perfect solution for now),
You may end up with a totally different file, of course:
client: server: open "/foo/bar" rename "/foo/baz"->"/foo/bar" write to file
And now we're writing to the file that was originally named /foo/baz when we should have gotten ESTALE.
--b.
Quoted 7 lines
but if this works, this doesn't limit any operations.
We would want to compare client solution (-mm) and server solution (stable ino). Or I'd like to know which my knowledges/understanding are wrong here.
Signature · 1 line
OGAWA Hirofumi <[email protected]>
OGAWA Hirofumi
maintainerQuoted 9 lines · J. Bruce Fields
Quoted 7 lines
On other view (as server side solution), we are thinking there is possible to make the stable filehandle on FAT if we disabled some operations (e.g.
rename(),unlink()) which change inode location in FAT.Yes, it would be stable, but supporting limited operations.
This is server side solution, and we comparing it with client solution.
Is that useful to anyone?
Good question. I'm not sure though, Namjae is asking. And I was asked about stable read-only export in past.
Quoted 14 lines
Quoted 11 lines
Quoted 4 lines
Quoted 4 lines
LOOKUP return NFS FH->[inode number changed at NFS Server] -> But we still use old NFS FH returned from LOOKUP for any file
operation(write,read,etc..)
-> ESTALE will be returned.Yes. And I'm expecting as client side solution,
-> ESTALE will be returned -> discard old FH -> restart from LOOKUP -> make cached inode -> use returned new FH.
Yeah, I know this is unstable (there is no perfect solution for now),
You may end up with a totally different file, of course:
Quoted 8 lines
client: server: open "/foo/bar" rename "/foo/baz"->"/foo/bar" write to fileAnd now we're writing to the file that was originally named /foo/baz when we should have gotten ESTALE.
I see. So, client can't solve the ESTALE if inode cache was evicted, right? (without application changes)
Signature · 1 line
OGAWA Hirofumi <[email protected]>
Quoted 35 lines · OGAWA Hirofumi
Quoted 9 lines · J. Bruce Fields
Quoted 7 lines
On other view (as server side solution), we are thinking there is possible to make the stable filehandle on FAT if we disabled some operations (e.g.
rename(),unlink()) which change inode location in FAT.Yes, it would be stable, but supporting limited operations.
This is server side solution, and we comparing it with client solution.
Is that useful to anyone?
Good question. I'm not sure though, Namjae is asking. And I was asked about stable read-only export in past.
Quoted 20 lines
Quoted 11 lines
Quoted 4 lines
Quoted 4 lines
LOOKUP return NFS FH->[inode number changed at NFS Server] -> But we still use old NFS FH returned from LOOKUP for any file
operation(write,read,etc..)
-> ESTALE will be returned.Yes. And I'm expecting as client side solution,
-> ESTALE will be returned -> discard old FH -> restart from LOOKUP -> make cached inode -> use returned new FH.
Yeah, I know this is unstable (there is no perfect solution for now),
You may end up with a totally different file, of course:
client: server: open "/foo/bar" rename "/foo/baz"->"/foo/bar" write to file
Quoted 5 lines
Quoted 2 lines
And now we're writing to the file that was originally named /foo/baz when we should have gotten ESTALE.
I see. So, client can't solve the ESTALE if inode cache was evicted, right? (without application changes)
I don't see how.
As another server-side workaround: maybe they could try tuning the inode caching to make eviction less likely?
Grepping around... Documentation/sysctl/vm.txt mentions a vfs_cache_pressure parameter.
--b.
OGAWA Hirofumi
maintainerQuoted 5 lines · J. Bruce Fields
As another server-side workaround: maybe they could try tuning the inode caching to make eviction less likely?
Grepping around...
Documentation/sysctl/vm.txtmentions avfs_cache_pressureparameter.
Yeah. And dirty hack will be possible to adjust sb->s_shrink.batch.
Signature · 1 line
OGAWA Hirofumi <[email protected]>
Quoted 13 lines
Quoted 13 lines
Quoted 4 lines
-> ESTALE will be returned -> discard old FH -> restart from LOOKUP -> make cached inode -> use returned new FH.
Yeah, I know this is unstable (there is no perfect solution for now),
You may end up with a totally different file, of course:
client: server: open "/foo/bar" rename "/foo/baz"->"/foo/bar" write to file
Quoted 5 lines
Quoted 2 lines
And now we're writing to the file that was originally named /foo/baz when we should have gotten ESTALE.
I see. So, client can't solve the ESTALE if inode cache was evicted, right? (without application changes)
There can be situation where we may get not only ESTALE but EIO also.
For example,
-------------------------------
fd = open(“foo.txt”);
while (1) {
sleep(1);
write(fd..);
}--------------------------------
Here “write” may fail when inode number of “foo.txt” is changed at server due to cache eviction under memory pressure.
When we tried a similar test, we found that “write” is retuning “EIO” instead of “ESTALE”
---------------------------------------------------------------------------------------------------------
#> ./write_test_dbg bbb 1000 0
FILE : bbb, SIZE : 1048576000 , FSYNC : OFF , RECORD_SIZE = 4096 106264 -rwxr-xr-x 1 root 0 0 Jan 1 00:14 bbb
write failed after 60080128 bytes:, errno = 5: Input/output error
---------------------------------------------------------------------------------------------------------
As we get EIO instead of ESTALE, it may be difficult to decide when
"restart from LOOKUP” in such situation.
Also, as per Bruce opinion, we can not avoid ESTALE from inode number change in rebooted server case.
In reboot case, it is worst as it may attempt to write in a different file if NFS handle at NFS client match with inode number of some other file at NFS server.
Quoted 6 lines
We would want to compare client solution (-mm) and server solution (stable ino). Or I'd like to know which my knowledges/understanding are wrong here.
I see. So, client can't solve the ESTALE if inode cache was evicted, right? (without application changes)
I don't see how.
Yes, I think we can not fix inode number changing issue on two situation (reboot, inode cache eviction).
And Inode number can change because currently FAT is not allocating stable inode number in this situation.
Quoted 3 lines
Grepping around...
Documentation/sysctl/vm.txtmentions avfs_cache_pressureparameter.
Yeah. And dirty hack will be possible to adjustsb->s_shrink.batch.
I am worrying if it could lead to OOM condition on embedded system(short memory(DRAM) and support 3TB HDD disk of big size.)
Please let me know if any issues or queries.
Thanks.
OGAWA Hirofumi
maintainerQuoted 26 lines · Namjae Jeon
Quoted 2 lines
I see. So, client can't solve the ESTALE if inode cache was evicted, right? (without application changes)
There can be situation where we may get not only ESTALE but EIO also.
For example,
-------------------------------
fd = open(“foo.txt”);while (1) { sleep(1); write(fd..); }--------------------------------
Here “write” may fail when inode number of “foo.txt” is changed at server due to cache eviction under memory pressure.
When we tried a similar test, we found that “write” is retuning “EIO” instead of “ESTALE”---------------------------------------------------------------------------------------------------------
#> ./write_test_dbg bbb 1000 0
FILE : bbb, SIZE : 1048576000 , FSYNC : OFF ,RECORD_SIZE= 4096 106264 -rwxr-xr-x 1 root 0 0 Jan 1 00:14 bbb
write failed after 60080128 bytes:, errno = 5: Input/output error
---------------------------------------------------------------------------------------------------------
Quoted 7 lines
As we get EIO instead of ESTALE, it may be difficult to decide when
"restart from LOOKUP” in such situation.
Also, as per Bruce opinion, we can not avoid ESTALE from inode number change in rebooted server case.
In reboot case, it is worst as it may attempt to write in a different file if NFS handle at NFS client match with inode number of some other file at NFS server.
I see.
Quoted 7 lines
Quoted 3 lines
Grepping around...
Documentation/sysctl/vm.txtmentions avfs_cache_pressureparameter.
Yeah. And dirty hack will be possible to adjustsb->s_shrink.batch.I am worrying if it could lead to OOM condition on embedded system(short memory(DRAM) and support 3TB HDD disk of big size.)
Please let me know if any issues or queries.
So, now I think stable inode number may be useful if there are users of it. And I guess those functionality is no collisions with -mm. And I suppose we can add two modes for "nfs" option (e.g. nfs=1 and nfs=2).
If nfs=1, works like current -mm without no limited operations. If nfs=2, try to make stable FH and limit some operations
(option name doesn't matter here.)
Does this work fine?
Signature · 1 line
OGAWA Hirofumi <[email protected]>
Quoted 44 lines · OGAWA Hirofumi
Quoted 26 lines · Namjae Jeon
Quoted 2 lines
I see. So, client can't solve the ESTALE if inode cache was evicted, right? (without application changes)
There can be situation where we may get not only ESTALE but EIO also.
For example,
-------------------------------
fd = open(“foo.txt”);while (1) { sleep(1); write(fd..); }--------------------------------
Here “write” may fail when inode number of “foo.txt” is changed at server due to cache eviction under memory pressure.
When we tried a similar test, we found that “write” is retuning “EIO” instead of “ESTALE”---------------------------------------------------------------------------------------------------------
#> ./write_test_dbg bbb 1000 0
FILE : bbb, SIZE : 1048576000 , FSYNC : OFF ,RECORD_SIZE= 4096 106264 -rwxr-xr-x 1 root 0 0 Jan 1 00:14 bbb
write failed after 60080128 bytes:, errno = 5: Input/output error
---------------------------------------------------------------------------------------------------------Quoted 7 lines
As we get EIO instead of ESTALE, it may be difficult to decide when
"restart from LOOKUP” in such situation.
Also, as per Bruce opinion, we can not avoid ESTALE from inode number change in rebooted server case.
In reboot case, it is worst as it may attempt to write in a different file if NFS handle at NFS client match with inode number of some other file at NFS server.I see.
Quoted 6 lines
Quoted 3 lines
Grepping around...
Documentation/sysctl/vm.txtmentions avfs_cache_pressureparameter.
Yeah. And dirty hack will be possible to adjustsb->s_shrink.batch.I am worrying if it could lead to OOM condition on embedded system(short memory(DRAM) and support 3TB HDD disk of big size.)
Quoted 7 lines
Quoted 1 line
Please let me know if any issues or queries.
So, now I think stable inode number may be useful if there are users of it. And I guess those functionality is no collisions with -mm. And I suppose we can add two modes for "nfs" option (e.g. nfs=1 and nfs=2).
If nfs=1, works like current -mm without no limited operations.
Apologies, I haven't been following the conversation carefully: remind me what "works like current -mm" means?
--b.
Quoted 7 lines
If nfs=2, try to make stable FH and limit some operations
(option name doesn't matter here.)
Does this work fine?
Signature · 1 line
OGAWA Hirofumi <[email protected]>
OGAWA Hirofumi
maintainerQuoted 8 lines · J. Bruce Fields
Quoted 8 lines
Quoted 7 lines
Quoted 3 lines
Grepping around...
Documentation/sysctl/vm.txtmentions avfs_cache_pressureparameter.
Yeah. And dirty hack will be possible to adjustsb->s_shrink.batch.I am worrying if it could lead to OOM condition on embedded system(short memory(DRAM) and support 3TB HDD disk of big size.)
Please let me know if any issues or queries.
Quoted 8 lines
Quoted 5 lines
So, now I think stable inode number may be useful if there are users of it. And I guess those functionality is no collisions with -mm. And I suppose we can add two modes for "nfs" option (e.g. nfs=1 and nfs=2).
If nfs=1, works like current -mm without no limited operations.
Apologies, I haven't been following the conversation carefully: remind me what "works like current -mm" means?
Current -mm means the best-effort work only if inode cache is not evicted. I.e. if there is no inode cache anymore on server, server would return ESTALE. So I guess the behavior would not be stable relatively.
Thanks.
Quoted 5 lines
Quoted 5 lines
If nfs=2, try to make stable FH and limit some operations
(option name doesn't matter here.)
Does this work fine?
Signature · 1 line
OGAWA Hirofumi <[email protected]>
2012/9/13, OGAWA Hirofumi <[email protected]>:
Quoted 16 lines
Quoted 14 lines · J. Bruce Fields
Quoted 13 lines
Quoted 7 lines
Quoted 3 lines
Grepping around...
Documentation/sysctl/vm.txtmentions avfs_cache_pressureparameter.
Yeah. And dirty hack will be possible to adjustsb->s_shrink.batch.I am worrying if it could lead to OOM condition on embedded system(short memory(DRAM) and support 3TB HDD disk of big size.)
Please let me know if any issues or queries.
So, now I think stable inode number may be useful if there are users of it. And I guess those functionality is no collisions with -mm. And I suppose we can add two modes for "nfs" option (e.g. nfs=1 and nfs=2).
If nfs=1, works like current -mm without no limited operations.
Quoted 7 lines
Quoted 2 lines
Apologies, I haven't been following the conversation carefully: remind me what "works like current -mm" means?
Current -mm means the best-effort work only if inode cache is not evicted. I.e. if there is no inode cache anymore on server, server would return ESTALE. So I guess the behavior would not be stable relatively.
Hi OGAWA.
Sorry for late response.
Okay, I will resend patchset include your suggeston.(-o nfs=2) Do you mind adding busy list patch to avoid unlink issue ? And in case of rename, FAT retrun EBUSY while opening file. We can limit only rename.
Let me know your opinion.
Thanks OGAWA!
Quoted 11 lines
Thanks.
Quoted 5 lines
Quoted 5 lines
If nfs=2, try to make stable FH and limit some operations
(option name doesn't matter here.)
Does this work fine?
--
OGAWA Hirofumi <[email protected]>
Quoted 24 lines · Namjae Jeon
2012/9/13, OGAWA Hirofumi <[email protected]>:
Quoted 16 lines
Quoted 14 lines · J. Bruce Fields
Quoted 13 lines
Quoted 7 lines
Quoted 3 lines
Grepping around...
Documentation/sysctl/vm.txtmentions avfs_cache_pressureparameter.
Yeah. And dirty hack will be possible to adjustsb->s_shrink.batch.I am worrying if it could lead to OOM condition on embedded system(short memory(DRAM) and support 3TB HDD disk of big size.)
Please let me know if any issues or queries.
So, now I think stable inode number may be useful if there are users of it. And I guess those functionality is no collisions with -mm. And I suppose we can add two modes for "nfs" option (e.g. nfs=1 and nfs=2).
If nfs=1, works like current -mm without no limited operations.
Quoted 7 lines
Quoted 2 lines
Apologies, I haven't been following the conversation carefully: remind me what "works like current -mm" means?
Current -mm means the best-effort work only if inode cache is not evicted. I.e. if there is no inode cache anymore on server, server would return ESTALE. So I guess the behavior would not be stable relatively.
Quoted 6 lines
Hi OGAWA.
Sorry for late response.
Okay, I will resend patchset include your suggeston.(-o nfs=2) Do you mind adding busy list patch to avoid unlink issue ? And in case of rename, FAT retrun EBUSY while opening file. We can limit only rename.
The server doesn't necessarily know whether a client has the file open, so does that really help?
--b.
Quoted 15 lines
Let me know your opinion.
Thanks OGAWA!
Quoted 11 lines
Thanks.
Quoted 5 lines
Quoted 5 lines
If nfs=2, try to make stable FH and limit some operations
(option name doesn't matter here.)
Does this work fine?
--
OGAWA Hirofumi <[email protected]>
OGAWA Hirofumi
maintainerQuoted 11 lines · J. Bruce Fields
Quoted 10 lines
Quoted 4 lines
Current -mm means the best-effort work only if inode cache is not evicted. I.e. if there is no inode cache anymore on server, server would return ESTALE. So I guess the behavior would not be stable relatively.
Hi OGAWA.
Sorry for late response.
Okay, I will resend patchset include your suggeston.(-o nfs=2) Do you mind adding busy list patch to avoid unlink issue ? And in case of rename, FAT retrun EBUSY while opening file. We can limit only rename.
Quoted 2 lines
The server doesn't necessarily know whether a client has the file open, so does that really help?
If you are assuming to do rename() somehow, it would not be true. And if so, you might want to rethink about the patch. (btw, I'm not thinking deeply yet though, I guess we have to limit unlink() too)
Well, anyway, I'd like to see stable/clean read-only support at first. (with new nfs option, and with MS_RDONLY). After that, we can enable write support.
Signature · 1 line
OGAWA Hirofumi <[email protected]>
2012/9/14, OGAWA Hirofumi <[email protected]>:
Quoted 13 lines
Quoted 11 lines · J. Bruce Fields
Quoted 10 lines
Quoted 4 lines
Current -mm means the best-effort work only if inode cache is not evicted. I.e. if there is no inode cache anymore on server, server would return ESTALE. So I guess the behavior would not be stable relatively.
Hi OGAWA.
Sorry for late response.
Okay, I will resend patchset include your suggeston.(-o nfs=2) Do you mind adding busy list patch to avoid unlink issue ? And in case of rename, FAT retrun EBUSY while opening file. We can limit only rename.
Quoted 6 lines
Quoted 2 lines
The server doesn't necessarily know whether a client has the file open, so does that really help?
If you are assuming to do
rename()somehow, it would not be true. And if so, you might want to rethink about the patch. (btw, I'm not thinking deeply yet though, I guess we have to limitunlink()too)
Hi OGAWA.
Sorry, There are some unclear things in unlink solution. When we tested before, That did really help.
So, we will check unlink solution more and after fully convinced will send in separate patch later with updates in changelog of patch when enabling write support..
Quoted 4 lines
Well, anyway, I'd like to see stable/clean read-only support at first. (with new nfs option, and with
MS_RDONLY). After that, we can enable write support.
Okay, I will send stable ino patch(read-only) soon.
Thanks a lot!
Quoted 3 lines
--
OGAWA Hirofumi <[email protected]>
Quoted 1 line · OGAWA Hirofumi
What is your use case?
What Neil Brown refers to as a "general file access protocol" - basically making a flash disk available to a small embedded network for random-access file I/O.
The flash disk is required to interoperate with Windoze, which forces us into VFAT-backed NFS.
Quoted 2 lines
I'm assuming current NFS support of FAT is still unstable behavior even with your patches. Is this true?
We use lookupcache=none, which I thought had stabilized things. Based on what Namjae has found with ESTALE on rename/drop_caches I guess there is a hole.
------------------------------------------------------------------------
Steven J. Magnani "I claim this network for MARS!
www.digidescorp.com Earthling, return my space modulator!"
#include <standard.disclaimer>