Skip to contentSkip to conversation

... is going on with current->fs->{root,mnt} accesses in pohmelfs

WTF Al Viro 2 participants 14 messages Feb 10, 2010 – Feb 11, 2010 VFS

lore
Share & tools
lore

Al Viro

author maintainer
#0 lore raw

a) pohmelfs_construct_path_string() will do interesting things if you call it while chrooted into jail and pohmelfs mounted deeper in that jail. Try it.

b) just why do we care about root of chroot jail in pohmelfs_path_length()? Not to mention anything else, current->fs->root/mnt may be changed under you if you share current->fs with another thread, but even aside of that, why does filesystem care about chroot of caller at all?

What's going on there?

#1 lore raw
Quoted 3 lines · Al Viro

a) pohmelfs_construct_path_string() will do interesting things if you call it while chrooted into jail and pohmelfs mounted deeper in that jail. Try it.

Should it walk upto mountpoint?

Quoted 6 lines

b) just why do we care about root of chroot jail in pohmelfs_path_length()? Not to mention anything else, current->fs->root/mnt may be changed under you if you share current->fs with another thread, but even aside of that, why does filesystem care about chroot of caller at all?

What's going on there?

It tries to construct a full path upto mountpoint. Effectively it should do similar to non-exported dentry_path() things. There is a race between getting buffer size and filling with the actual path, but we take care about that by restarting if needed.

Signature · 1 line
	Evgeniy Polyakov
author maintainer
#2 lore raw
Quoted 6 lines · Evgeniy Polyakov
Quoted 3 lines · Al Viro

a) pohmelfs_construct_path_string() will do interesting things if you call it while chrooted into jail and pohmelfs mounted deeper in that jail. Try it.

Should it walk upto mountpoint?

It will happily give you path from absolute root to root of chroot jail + path from fs root to your dentry. Which is probably not what you want.

Quoted 11 lines
Quoted 6 lines

b) just why do we care about root of chroot jail in pohmelfs_path_length()? Not to mention anything else, current->fs->root/mnt may be changed under you if you share current->fs with another thread, but even aside of that, why does filesystem care about chroot of caller at all?

What's going on there?

It tries to construct a full path upto mountpoint. Effectively it should do similar to non-exported dentry_path() things. There is a race between getting buffer size and filling with the actual path, but we take care about that by restarting if needed.

To mountpoint or to fs root? And what's going on with d_find_alias()? AFAICS, you are doing that for regular files as well as directories, and you do support link(2) in there, so dentry (and path) obtained from that will be random.

#3 lore raw
Quoted 10 lines · Al Viro
Quoted 6 lines · Evgeniy Polyakov
Quoted 3 lines · Al Viro

a) pohmelfs_construct_path_string() will do interesting things if you call it while chrooted into jail and pohmelfs mounted deeper in that jail. Try it.

Should it walk upto mountpoint?

It will happily give you path from absolute root to root of chroot jail + path from fs root to your dentry. Which is probably not what you want.

I thouht of always providing path to mountpoint instead of just root. Full path names are rather messy to work with, so I consider per-directory objects only in the next version, but drawback is lookup time.

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

b) just why do we care about root of chroot jail in pohmelfs_path_length()? Not to mention anything else, current->fs->root/mnt may be changed under you if you share current->fs with another thread, but even aside of that, why does filesystem care about chroot of caller at all?

Quoted 8 lines
Quoted 6 lines
Quoted 1 line

What's going on there?

It tries to construct a full path upto mountpoint. Effectively it should do similar to non-exported dentry_path() things. There is a race between getting buffer size and filling with the actual path, but we take care about that by restarting if needed.

To mountpoint or to fs root? And what's going on with d_find_alias()?

To root if it happend to be under mountpoint.

Quoted 3 lines

AFAICS, you are doing that for regular files as well as directories, and you do support link(2) in there, so dentry (and path) obtained from that will be random.

Not exactly random, but can change.
Links support is rather subtle because of that, yes.

Plan was to add external attribute or increase inode size to include parent name, but when I coded that it was so messy in respect of renames, that was dropped.

Signature · 1 line
	Evgeniy Polyakov
author maintainer
#4 lore raw
Quoted 3 lines · Evgeniy Polyakov
Quoted 1 line

To mountpoint or to fs root? And what's going on with d_find_alias()?

To root if it happend to be under mountpoint.

HUH? How the hell can root of filesystem be under the mountpoint of that filesystem? What are you talking about?

Quoted 10 lines
Quoted 3 lines

AFAICS, you are doing that for regular files as well as directories, and you do support link(2) in there, so dentry (and path) obtained from that will be random.

Not exactly random, but can change.
Links support is rather subtle because of that, yes.

Plan was to add external attribute or increase inode size to include parent name, but when I coded that it was so messy in respect of renames, that was dropped.

Why not use the dentries you've been given by VFS?

#5 lore raw
Quoted 6 lines · Al Viro
Quoted 3 lines
Quoted 1 line

To mountpoint or to fs root? And what's going on with d_find_alias()?

To root if it happend to be under mountpoint.

HUH? How the hell can root of filesystem be under the mountpoint of that filesystem? What are you talking about?

Let me guess... Mmmm, it was in the yesterday newspaper, I remember. Maybe when we chroot somewhere. I meant not mounted fs root, but thread's root.

Quoted 4 lines
Quoted 4 lines
Quoted 3 lines

AFAICS, you are doing that for regular files as well as directories, and you do support link(2) in there, so dentry (and path) obtained from that will be random.

Quoted 8 lines
Quoted 6 lines

Not exactly random, but can change.
Links support is rather subtle because of that, yes.

Plan was to add external attribute or increase inode size to include parent name, but when I coded that it was so messy in respect of renames, that was dropped.

Why not use the dentries you've been given by VFS?

At writeback we do not have parents, so must find a path somehow.

Signature · 1 line
	Evgeniy Polyakov
author maintainer
#6 lore raw
Quoted 11 lines · Evgeniy Polyakov
Quoted 6 lines · Al Viro
Quoted 3 lines
Quoted 1 line

To mountpoint or to fs root? And what's going on with d_find_alias()?

To root if it happend to be under mountpoint.

HUH? How the hell can root of filesystem be under the mountpoint of that filesystem? What are you talking about?

Let me guess... Mmmm, it was in the yesterday newspaper, I remember. Maybe when we chroot somewhere. I meant not mounted fs root, but thread's root.

Why would a filesystem give a damn about the chroot of syscall originator in the first place?

Quoted 3 lines
Quoted 1 line

Why not use the dentries you've been given by VFS?

At writeback we do not have parents, so must find a path somehow.

Most of the places do have those just fine and unlike the writeback, rename et.al. really care which pathname is being dealt with...

BTW, what prevents writeback vs. rename races?

#7 lore raw
Quoted 6 lines · Al Viro
Quoted 3 lines

Let me guess... Mmmm, it was in the yesterday newspaper, I remember. Maybe when we chroot somewhere. I meant not mounted fs root, but thread's root.

Why would a filesystem give a damn about the chroot of syscall originator in the first place?

That's the point - it is not needed.

Quoted 6 lines
Quoted 3 lines
Quoted 1 line

Why not use the dentries you've been given by VFS?

At writeback we do not have parents, so must find a path somehow.

Most of the places do have those just fine and unlike the writeback, rename et.al. really care which pathname is being dealt with...

POHMELFS uses writeback cache also for metadata, so effectively most of such operations are also postponed. Later I turned that off though.

Quoted 1 line

BTW, what prevents writeback vs. rename races?

There are proper locks for such operations.

Signature · 1 line
	Evgeniy Polyakov
author maintainer
#8 lore raw
Quoted 7 lines · Evgeniy Polyakov
Quoted 6 lines
Quoted 3 lines
Quoted 1 line

Why not use the dentries you've been given by VFS?

At writeback we do not have parents, so must find a path somehow.

Most of the places do have those just fine and unlike the writeback, rename et.al. really care which pathname is being dealt with...

Quoted 6 lines

POHMELFS uses writeback cache also for metadata, so effectively most of such operations are also postponed. Later I turned that off though.

Quoted 1 line

BTW, what prevents writeback vs. rename races?

There are proper locks for such operations.

Which would be... ? E.g. between writepages() and rename(). What serializes your write_inode_create() wrt renames? IOW, how can the server decide that data from writepages() should go to the same object regardless of the rename?

#9 lore raw
Quoted 4 lines · Al Viro

Which would be... ? E.g. between writepages() and rename(). What serializes your write_inode_create() wrt renames? IOW, how can the server decide that data from writepages() should go to the same object regardless of the rename?

rename and some other metadata operations as well as write itself
request remote lock (if not grabbed already), acknowledge forces writeback to old path.

Signature · 1 line
	Evgeniy Polyakov
author maintainer
#10 lore raw
Quoted 8 lines · Evgeniy Polyakov
Quoted 4 lines · Al Viro

Which would be... ? E.g. between writepages() and rename(). What serializes your write_inode_create() wrt renames? IOW, how can the server decide that data from writepages() should go to the same object regardless of the rename?

rename and some other metadata operations as well as write itself
request remote lock (if not grabbed already), acknowledge forces writeback to old path.

Um. You do realize that d_move() happens with none of your locks held, right? It's done in vfs_rename_{other,dir}() and the only thing held is s_vfs_rename_sem and i_mutex on parents. How could your code in writeback be able to distinguish

rename() is done
d_move() has happened, we see new pathname in dcache

from

rename() is done
d_move() has not yet happened, we see old pathname in dcache

and generate the right on-the-wire traffic in both cases? Note that here server has already seen rename request; as far as server and client are concerned the rename() is over.

#11 lore raw
Quoted 4 lines · Al Viro

Um. You do realize that d_move() happens with none of your locks held, right? It's done in vfs_rename_{other,dir}() and the only thing held is s_vfs_rename_sem and i_mutex on parents. How could your code in writeback be able to distinguish

No, it happens with my lock held. It is not a lock, but kind of IO delegation, i.e. it is not dropped when rename or other protected operation completed. Instead another client sends request to grab it and server asks current holder to drop cache, perform writeback or whatever else is needed.

It can be a problem though if d_move() is called outside of path protected by the VFS dir operations like rename/created/unlink and so on, i.e. on behalf of some entity in the kernel which decides to move dentries on itself. In this case POHMELFS is not protected.

Signature · 1 line
	Evgeniy Polyakov
author maintainer
#12 lore raw
Quoted 11 lines · Evgeniy Polyakov
Quoted 4 lines · Al Viro

Um. You do realize that d_move() happens with none of your locks held, right? It's done in vfs_rename_{other,dir}() and the only thing held is s_vfs_rename_sem and i_mutex on parents. How could your code in writeback be able to distinguish

No, it happens with my lock held. It is not a lock, but kind of IO delegation, i.e. it is not dropped when rename or other protected operation completed. Instead another client sends request to grab it and server asks current holder to drop cache, perform writeback or whatever else is needed.

And should such a request come between return from ->rename() and call of d_move() that follows it?

Quoted 4 lines

It can be a problem though if d_move() is called outside of path protected by the VFS dir operations like rename/created/unlink and so on, i.e. on behalf of some entity in the kernel which decides to move dentries on itself. In this case POHMELFS is not protected.

Not an issue; there's no such fs-independent callers.

Fundamentally, how do you deal with MOESI when the mapping from strings you are using as object IDs to actual objects can change as the result of operations? What's more, operation on one object can change that mapping for a huge number of other objects (rename() close to fs root changing pathnames of all files anywhere in the subtree being moved).

#13 lore raw
Quoted 8 lines · Al Viro
Quoted 5 lines

No, it happens with my lock held. It is not a lock, but kind of IO delegation, i.e. it is not dropped when rename or other protected operation completed. Instead another client sends request to grab it and server asks current holder to drop cache, perform writeback or whatever else is needed.

And should such a request come between return from ->rename() and call of d_move() that follows it?

We mark inode dirty (including parent) and it has to be written back thus there will be no ack until writeback completed, which in turn messes with i_mutex, so will be postponed until rename and d_move() completed.

Quoted 5 lines
Quoted 4 lines

It can be a problem though if d_move() is called outside of path protected by the VFS dir operations like rename/created/unlink and so on, i.e. on behalf of some entity in the kernel which decides to move dentries on itself. In this case POHMELFS is not protected.

Quoted 7 lines

Not an issue; there's no such fs-independent callers.

Fundamentally, how do you deal with MOESI when the mapping from strings you are using as object IDs to actual objects can change as the result of operations? What's more, operation on one object can change that mapping for a huge number of other objects (rename() close to fs root changing pathnames of all files anywhere in the subtree being moved).

MOESI is actually a crap in a clustered filesystem. It does not scale well since number of messages grows exponentially with nodes and interconnects. Plus without PAXOS it does not provide needed redundancy level.

Basic idea currently is to delegate every operation and ask server if it is allowed or not. Thus server-side locking must be serialized to that level. I believe that i_mutex is enough there - its the only lock used actually except in some helpers like d_path().

Signature · 1 line
	Evgeniy Polyakov