... is going on with current->fs->{root,mnt} accesses in pohmelfs
Al Viro
maintainera) pohmelfs_construct_path_string() will do interesting things if you call it while chrooted into jail and pohmelfs mounted deeper in that jail. Try it.
b) just why do we care about root of chroot jail in pohmelfs_path_length()? Not to mention anything else, current->fs->root/mnt may be changed under you if you share current->fs with another thread, but even aside of that, why does filesystem care about chroot of caller at all?
What's going on there?
Quoted 3 lines · Al Viro
a)
pohmelfs_construct_path_string()will do interesting things if you call it while chrooted into jail and pohmelfs mounted deeper in that jail. Try it.
Should it walk upto mountpoint?
Quoted 6 lines
b) just why do we care about root of chroot jail in
pohmelfs_path_length()? Not to mention anything else, current->fs->root/mnt may be changed under you if you sharecurrent->fswith another thread, but even aside of that, why does filesystem care about chroot of caller at all?What's going on there?
It tries to construct a full path upto mountpoint. Effectively it should do similar to non-exported dentry_path() things. There is a race between getting buffer size and filling with the actual path, but we take care about that by restarting if needed.
Signature · 1 line
Evgeniy Polyakov
Al Viro
maintainerQuoted 6 lines · Evgeniy Polyakov
Quoted 3 lines · Al Viro
a)
pohmelfs_construct_path_string()will do interesting things if you call it while chrooted into jail and pohmelfs mounted deeper in that jail. Try it.Should it walk upto mountpoint?
It will happily give you path from absolute root to root of chroot jail + path from fs root to your dentry. Which is probably not what you want.
Quoted 11 lines
Quoted 6 lines
b) just why do we care about root of chroot jail in
pohmelfs_path_length()? Not to mention anything else, current->fs->root/mnt may be changed under you if you sharecurrent->fswith another thread, but even aside of that, why does filesystem care about chroot of caller at all?What's going on there?
It tries to construct a full path upto mountpoint. Effectively it should do similar to non-exported
dentry_path()things. There is a race between getting buffer size and filling with the actual path, but we take care about that by restarting if needed.
To mountpoint or to fs root? And what's going on with d_find_alias()? AFAICS, you are doing that for regular files as well as directories, and you do support link(2) in there, so dentry (and path) obtained from that will be random.
Quoted 10 lines · Al Viro
Quoted 6 lines · Evgeniy Polyakov
Quoted 3 lines · Al Viro
a)
pohmelfs_construct_path_string()will do interesting things if you call it while chrooted into jail and pohmelfs mounted deeper in that jail. Try it.Should it walk upto mountpoint?
It will happily give you path from absolute root to root of chroot jail + path from fs root to your dentry. Which is probably not what you want.
I thouht of always providing path to mountpoint instead of just root. Full path names are rather messy to work with, so I consider per-directory objects only in the next version, but drawback is lookup time.
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
b) just why do we care about root of chroot jail in
pohmelfs_path_length()? Not to mention anything else, current->fs->root/mnt may be changed under you if you sharecurrent->fswith another thread, but even aside of that, why does filesystem care about chroot of caller at all?
Quoted 8 lines
Quoted 6 lines
Quoted 1 line
What's going on there?
It tries to construct a full path upto mountpoint. Effectively it should do similar to non-exported
dentry_path()things. There is a race between getting buffer size and filling with the actual path, but we take care about that by restarting if needed.To mountpoint or to fs root? And what's going on with
d_find_alias()?
To root if it happend to be under mountpoint.
Quoted 3 lines
AFAICS, you are doing that for regular files as well as directories, and you do support link(2) in there, so dentry (and path) obtained from that will be random.
Not exactly random, but can change.
Links support is rather subtle because of that, yes.
Plan was to add external attribute or increase inode size to include parent name, but when I coded that it was so messy in respect of renames, that was dropped.
Signature · 1 line
Evgeniy Polyakov
Al Viro
maintainerQuoted 3 lines · Evgeniy Polyakov
Quoted 1 line
To mountpoint or to fs root? And what's going on with
d_find_alias()?To root if it happend to be under mountpoint.
HUH? How the hell can root of filesystem be under the mountpoint of that filesystem? What are you talking about?
Quoted 10 lines
Quoted 3 lines
AFAICS, you are doing that for regular files as well as directories, and you do support link(2) in there, so dentry (and path) obtained from that will be random.
Not exactly random, but can change.
Links support is rather subtle because of that, yes.Plan was to add external attribute or increase inode size to include parent name, but when I coded that it was so messy in respect of renames, that was dropped.
Why not use the dentries you've been given by VFS?
Quoted 6 lines · Al Viro
Quoted 3 lines
Quoted 1 line
To mountpoint or to fs root? And what's going on with
d_find_alias()?To root if it happend to be under mountpoint.
HUH? How the hell can root of filesystem be under the mountpoint of that filesystem? What are you talking about?
Let me guess... Mmmm, it was in the yesterday newspaper, I remember. Maybe when we chroot somewhere. I meant not mounted fs root, but thread's root.
Quoted 4 lines
Quoted 4 lines
Quoted 3 lines
AFAICS, you are doing that for regular files as well as directories, and you do support link(2) in there, so dentry (and path) obtained from that will be random.
Quoted 8 lines
Quoted 6 lines
Not exactly random, but can change.
Links support is rather subtle because of that, yes.Plan was to add external attribute or increase inode size to include parent name, but when I coded that it was so messy in respect of renames, that was dropped.
Why not use the dentries you've been given by VFS?
At writeback we do not have parents, so must find a path somehow.
Signature · 1 line
Evgeniy Polyakov
Al Viro
maintainerQuoted 11 lines · Evgeniy Polyakov
Quoted 6 lines · Al Viro
Quoted 3 lines
Quoted 1 line
To mountpoint or to fs root? And what's going on with
d_find_alias()?To root if it happend to be under mountpoint.
HUH? How the hell can root of filesystem be under the mountpoint of that filesystem? What are you talking about?
Let me guess... Mmmm, it was in the yesterday newspaper, I remember. Maybe when we chroot somewhere. I meant not mounted fs root, but thread's root.
Why would a filesystem give a damn about the chroot of syscall originator in the first place?
Quoted 3 lines
Quoted 1 line
Why not use the dentries you've been given by VFS?
At writeback we do not have parents, so must find a path somehow.
Most of the places do have those just fine and unlike the writeback, rename et.al. really care which pathname is being dealt with...
BTW, what prevents writeback vs. rename races?
Quoted 6 lines · Al Viro
Quoted 3 lines
Let me guess... Mmmm, it was in the yesterday newspaper, I remember. Maybe when we chroot somewhere. I meant not mounted fs root, but thread's root.
Why would a filesystem give a damn about the chroot of syscall originator in the first place?
That's the point - it is not needed.
Quoted 6 lines
Quoted 3 lines
Quoted 1 line
Why not use the dentries you've been given by VFS?
At writeback we do not have parents, so must find a path somehow.
Most of the places do have those just fine and unlike the writeback, rename et.al. really care which pathname is being dealt with...
POHMELFS uses writeback cache also for metadata, so effectively most of such operations are also postponed. Later I turned that off though.
Quoted 1 line
BTW, what prevents writeback vs. rename races?
There are proper locks for such operations.
Signature · 1 line
Evgeniy Polyakov
Al Viro
maintainerQuoted 7 lines · Evgeniy Polyakov
Quoted 6 lines
Quoted 3 lines
Quoted 1 line
Why not use the dentries you've been given by VFS?
At writeback we do not have parents, so must find a path somehow.
Most of the places do have those just fine and unlike the writeback, rename et.al. really care which pathname is being dealt with...
Quoted 6 lines
POHMELFS uses writeback cache also for metadata, so effectively most of such operations are also postponed. Later I turned that off though.
Quoted 1 line
BTW, what prevents writeback vs. rename races?
There are proper locks for such operations.
Which would be... ? E.g. between writepages() and rename(). What serializes your write_inode_create() wrt renames? IOW, how can the server decide that data from writepages() should go to the same object regardless of the rename?
Quoted 4 lines · Al Viro
Which would be... ? E.g. between
writepages()andrename(). What serializes yourwrite_inode_create()wrt renames? IOW, how can the server decide that data fromwritepages()should go to the same object regardless of the rename?
rename and some other metadata operations as well as write itself
request remote lock (if not grabbed already), acknowledge forces writeback to old path.
Signature · 1 line
Evgeniy Polyakov
Al Viro
maintainerQuoted 8 lines · Evgeniy Polyakov
Quoted 4 lines · Al Viro
Which would be... ? E.g. between
writepages()andrename(). What serializes yourwrite_inode_create()wrt renames? IOW, how can the server decide that data fromwritepages()should go to the same object regardless of the rename?rename and some other metadata operations as well as write itself
request remote lock (if not grabbed already), acknowledge forces writeback to old path.
Um. You do realize that d_move() happens with none of your locks held, right? It's done in vfs_rename_{other,dir}() and the only thing held is s_vfs_rename_sem and i_mutex on parents. How could your code in writeback be able to distinguish
rename() is doned_move() has happened, we see new pathname in dcache
from
rename() is doned_move() has not yet happened, we see old pathname in dcache
and generate the right on-the-wire traffic in both cases? Note that here server has already seen rename request; as far as server and client are concerned the rename() is over.
Quoted 4 lines · Al Viro
Um. You do realize that
d_move()happens with none of your locks held, right? It's done in vfs_rename_{other,dir}() and the only thing held iss_vfs_rename_semandi_mutexon parents. How could your code in writeback be able to distinguish
No, it happens with my lock held. It is not a lock, but kind of IO delegation, i.e. it is not dropped when rename or other protected operation completed. Instead another client sends request to grab it and server asks current holder to drop cache, perform writeback or whatever else is needed.
It can be a problem though if d_move() is called outside of path protected by the VFS dir operations like rename/created/unlink and so on, i.e. on behalf of some entity in the kernel which decides to move dentries on itself. In this case POHMELFS is not protected.
Signature · 1 line
Evgeniy Polyakov
Al Viro
maintainerQuoted 11 lines · Evgeniy Polyakov
Quoted 4 lines · Al Viro
Um. You do realize that
d_move()happens with none of your locks held, right? It's done in vfs_rename_{other,dir}() and the only thing held iss_vfs_rename_semandi_mutexon parents. How could your code in writeback be able to distinguishNo, it happens with my lock held. It is not a lock, but kind of IO delegation, i.e. it is not dropped when rename or other protected operation completed. Instead another client sends request to grab it and server asks current holder to drop cache, perform writeback or whatever else is needed.
And should such a request come between return from ->rename() and call of d_move() that follows it?
Quoted 4 lines
It can be a problem though if
d_move()is called outside of path protected by the VFS dir operations like rename/created/unlink and so on, i.e. on behalf of some entity in the kernel which decides to move dentries on itself. In this case POHMELFS is not protected.
Not an issue; there's no such fs-independent callers.
Fundamentally, how do you deal with MOESI when the mapping from strings you are using as object IDs to actual objects can change as the result of operations? What's more, operation on one object can change that mapping for a huge number of other objects (rename() close to fs root changing pathnames of all files anywhere in the subtree being moved).
Quoted 8 lines · Al Viro
Quoted 5 lines
No, it happens with my lock held. It is not a lock, but kind of IO delegation, i.e. it is not dropped when rename or other protected operation completed. Instead another client sends request to grab it and server asks current holder to drop cache, perform writeback or whatever else is needed.
And should such a request come between return from ->rename() and call of
d_move()that follows it?
We mark inode dirty (including parent) and it has to be written back thus there will be no ack until writeback completed, which in turn messes with i_mutex, so will be postponed until rename and d_move() completed.
Quoted 5 lines
Quoted 4 lines
It can be a problem though if
d_move()is called outside of path protected by the VFS dir operations like rename/created/unlink and so on, i.e. on behalf of some entity in the kernel which decides to move dentries on itself. In this case POHMELFS is not protected.
Quoted 7 lines
Not an issue; there's no such fs-independent callers.
Fundamentally, how do you deal with MOESI when the mapping from strings you are using as object IDs to actual objects can change as the result of operations? What's more, operation on one object can change that mapping for a huge number of other objects (
rename()close to fs root changing pathnames of all files anywhere in the subtree being moved).
MOESI is actually a crap in a clustered filesystem. It does not scale well since number of messages grows exponentially with nodes and interconnects. Plus without PAXOS it does not provide needed redundancy level.
Basic idea currently is to delegate every operation and ask server if it is allowed or not. Thus server-side locking must be serialized to that level. I believe that i_mutex is enough there - its the only lock used actually except in some helpers like d_path().
Signature · 1 line
Evgeniy Polyakov