e4defrag and immutable files
Hi all,
I have looked a bit at e4defrag, and it does not appear to prevent the relocation of files marked immutable. This is a problem for boot loaders, which may need to have a block pointer to a specific file in order to bootstrap themselves.
Would it be possible to either:
a) prevent e4defrag and/or EXT4_IOC_MOVE_EXT from relocating an immutable file, or
b) add a new attribute with the above property (in case there is legitimate need to move around immutable files)?
-hpa
Quoted 8 lines · H. Peter Anvin
Hi all,
I have looked a bit at e4defrag, and it does not appear to prevent the relocation of files marked immutable. This is a problem for boot loaders, which may need to have a block pointer to a specific file in order to bootstrap themselves.
Quoted 8 lines
Would it be possible to either:
a) prevent e4defrag and/or
EXT4_IOC_MOVE_EXTfrom relocating an immutable file, or
b) add a new attribute with the above property (in case there is legitimate need to move around immutable files)?-hpa
Other projects / tools than e4defrag use EXT4_IOC_MOVE_EXT, so it would be best to do this in the kernel in my opinion.
Greg
--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Quoted 4 lines · Greg Freemyer
Other projects / tools than e4defrag use
EXT4_IOC_MOVE_EXT, so it would be best to do this in the kernel in my opinion.
I fully agree. If nothing else, it will provide protection against broken userspace programs.
-hpa
Quoted 7 lines · H. Peter Anvin
Hi all,
I have looked a bit at e4defrag, and it does not appear to prevent the relocation of files marked immutable. This is a problem for boot loaders, which may need to have a block pointer to a specific file in order to bootstrap themselves.
Quoted 6 lines
Would it be possible to either:
a) prevent e4defrag and/or
EXT4_IOC_MOVE_EXTfrom relocating an immutable file, or
b) add a new attribute with the above property (in case there is legitimate need to move around immutable files)?
seems like the immutable flag isn't necessarily the right thing to key on; at least it's an addition to what the manpage says immutable means...
Quoted 5 lines
A file with the ‘i’ attribute cannot be modified: it cannot be deleted or renamed, no link can be created to this file and no data can be written to the file. Only the superuser or a process possessing the
CAP_LINUX_IMMUTABLEcapability can set or clear this attribute.
Changing the physical mapping doesn't do any of those things.
I think a new flag, no-move or something, may be better, if it's needed. But how many files would ever get this; maybe defragging boot files falls into the category of "don't do that?" :)
-Eric
Quoted 1 line
-hpa
--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Quoted 7 lines · H. Peter Anvin
Hi all,
I have looked a bit at e4defrag, and it does not appear to prevent the relocation of files marked immutable. This is a problem for boot loaders, which may need to have a block pointer to a specific file in order to bootstrap themselves.
Quoted 7 lines
Would it be possible to either:
a) prevent e4defrag and/or
EXT4_IOC_MOVE_EXTfrom relocating an immutable file, or
b) add a new attribute with the above property (in case there is legitimate need to move around immutable files)?
We (ocfs2) are looking to add a new attribute to denote files that have a fixed allocation on disk. But at the same time, allow writes that do not change the allocation on disk. No truncating, extending, filling holes, etc. We were thinking of calling it "Static" files.
No meddling with the allocation will allow ocfs2 to not have any cluster locks associated with that inode. And no cluster locks means no need to pause ios during node recovery. This could be of interest to vm users who are willing have fully allocated images for no stoppage during cluster reconfiguration.
Thanks
Sunil
Quoted 5 lines · Sunil Mushran
We (ocfs2) are looking to add a new attribute to denote files that have a fixed allocation on disk. But at the same time, allow writes that do not change the allocation on disk. No truncating, extending, filling holes, etc. We were thinking of calling it "Static" files.
That's an interesting set of semantics, and it might make sense to conflate that with a local disk "don't move or defrag" the file option. I'm not crazy with the name "static", since it could mean a number of other things in other contexts, but I admit I can't think of a better name.
- Ted
Quoted 7 lines · On Tue, Jun 1, 2010 at 3:49 PM, <[email protected]> wrote:
Quoted 5 lines · Sunil Mushran
We (ocfs2) are looking to add a new attribute to denote files that have a fixed allocation on disk. But at the same time, allow writes that do not change the allocation on disk. No truncating, extending, filling holes, etc. We were thinking of calling it "Static" files.
Quoted 5 lines
That's an interesting set of semantics, and it might make sense to conflate that with a local disk "don't move or defrag" the file option. I'm not crazy with the name "static", since it could mean a number of other things in other contexts, but I admit I can't think of a better name.
FROZEN?
PRE-ALLOCATED?
FALLOCATE_ONLY?
FROZEN_ALLOCATION?
STATIC_ALLOCATION?
I like the last 2 choices myself.
Greg
--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Quoted 6 lines · [email protected] wrote:
Quoted 4 lines · Sunil Mushran
We (ocfs2) are looking to add a new attribute to denote files that have a fixed allocation on disk. But at the same time, allow writes that do not change the allocation on disk. No truncating, extending, filling holes, etc. We were thinking of calling it "Static" files.
Quoted 7 lines
That's an interesting set of semantics, and it might make sense to conflate that with a local disk "don't move or defrag" the file option. I'm not crazy with the name "static", since it could mean a number of other things in other contexts, but I admit I can't think of a better name.
- Ted
fixed-mapping files maybe?
-Eric
Quoted 15 lines · Eric Sandeen
Quoted 13 lines · [email protected] wrote:
Quoted 5 lines · Sunil Mushran
We (ocfs2) are looking to add a new attribute to denote files that have a fixed allocation on disk. But at the same time, allow writes that do not change the allocation on disk. No truncating, extending, filling holes, etc. We were thinking of calling it "Static" files.
That's an interesting set of semantics, and it might make sense to conflate that with a local disk "don't move or defrag" the file option. I'm not crazy with the name "static", since it could mean a number of other things in other contexts, but I admit I can't think of a better name.
Quoted 4 lines
Quoted 2 lines
- Ted
fixed-mapping files maybe?
We had thought of the term static because we wanted the inode to be frozen. Entire metadata including the mapping. Our aim is to do away with cluster locks for such inodes. For ios, we were planning to limit it to only odirect and not change the mtime. That also means no links, touch. Yes, "static" does not fit it like a glove but that's the best we could come up with.
Sunil
Quoted 7 lines · Sunil Mushran
We had thought of the term static because we wanted the inode to be frozen. Entire metadata including the mapping. Our aim is to do away with cluster locks for such inodes. For ios, we were planning to limit it to only odirect and not change the mtime. That also means no links, touch. Yes, "static" does not fit it like a glove but that's the best we could come up with.
Hmm, maybe "fixed_metadata"? The one thing that worries a bit is not touching mtime is definitely icky. I suppose it would improve performance even on local-disk file systems if we don't update mtime, ctime, or atime. So maybe that would be useful for folks who are trying to extract the last bit of performance out of their enterprise database.
One thing I do like about fixed_metadata is that as far as chattr and lsattr is concerned 's' and 'S' are already taken. But 'f' and 'F' aren't allocated yet. :-)
- Ted
Quoted 16 lines · On 06/01/2010 03:26 PM, [email protected] wrote:
Quoted 7 lines · Sunil Mushran
We had thought of the term static because we wanted the inode to be frozen. Entire metadata including the mapping. Our aim is to do away with cluster locks for such inodes. For ios, we were planning to limit it to only odirect and not change the mtime. That also means no links, touch. Yes, "static" does not fit it like a glove but that's the best we could come up with.
Hmm, maybe "
fixed_metadata"? The one thing that worries a bit is not touching mtime is definitely icky. I suppose it would improve performance even on local-disk file systems if we don't update mtime, ctime, or atime. So maybe that would be useful for folks who are trying to extract the last bit of performance out of their enterprise database.
Quoted 5 lines
One thing I do like about
fixed_metadatais that as far as chattr and lsattr is concerned 's' and 'S' are already taken. But 'f' and 'F' aren't allocated yet. :-)
Fixed Metadata sounds good to me.
The mtime restriction is icky. But for us it is a requirement considering we cannot safely update it without cluster locks. I am ok if we can make that file system specific.
Quoted 7 lines · On 06/01/2010 12:49 PM, [email protected] wrote:
Quoted 5 lines · Sunil Mushran
We (ocfs2) are looking to add a new attribute to denote files that have a fixed allocation on disk. But at the same time, allow writes that do not change the allocation on disk. No truncating, extending, filling holes, etc. We were thinking of calling it "Static" files.
Quoted 6 lines
That's an interesting set of semantics, and it might make sense to conflate that with a local disk "don't move or defrag" the file option. I'm not crazy with the name "static", since it could mean a number of other things in other contexts, but I admit I can't think of a better name.
For what it's worth, this sort of seems to be what one would expect if a file is *both* "fixed" and "immutable".
-hpa
Quoted 8 lines · H. Peter Anvin
Quoted 7 lines · On 06/01/2010 12:49 PM, [email protected] wrote:
Quoted 5 lines · Sunil Mushran
We (ocfs2) are looking to add a new attribute to denote files that have a fixed allocation on disk. But at the same time, allow writes that do not change the allocation on disk. No truncating, extending, filling holes, etc. We were thinking of calling it "Static" files.
Quoted 8 lines
Quoted 6 lines
That's an interesting set of semantics, and it might make sense to conflate that with a local disk "don't move or defrag" the file option. I'm not crazy with the name "static", since it could mean a number of other things in other contexts, but I admit I can't think of a better name.
For what it's worth, this sort of seems to be what one would expect if a file is *both* "fixed" and "immutable".
"Immutable" means the contents do not change. But the file mappings could change.
"Fixed mapping" means the mappings do not change but contents could (as long as the ondisk mappings don't).
"Fixed metadata" means the entire inode (mappings included) cannot change but the contents could (as long as the ondisk mappings don't). (This does have the side effect of allowing writes without touching the mtime. Like XFS' invisible i/o.)
What the boot loader needs is "Fixed mapping". I am sure there could be other use cases for it too.
My suggestion would be to implement "Fixed mapping" as is and not have it tied to "Fixed metadata". While we would like to use a chattr flag for this feature, it is not a requirement.
Sunil
Quoted 22 lines · Sunil Mushran
Quoted 15 lines · H. Peter Anvin
Quoted 7 lines · On 06/01/2010 12:49 PM, [email protected] wrote:
Quoted 5 lines · Sunil Mushran
We (ocfs2) are looking to add a new attribute to denote files that have a fixed allocation on disk. But at the same time, allow writes that do not change the allocation on disk. No truncating, extending, filling holes, etc. We were thinking of calling it "Static" files.
Quoted 6 lines
That's an interesting set of semantics, and it might make sense to conflate that with a local disk "don't move or defrag" the file option. I'm not crazy with the name "static", since it could mean a number of other things in other contexts, but I admit I can't think of a better name.
Quoted 2 lines
For what it's worth, this sort of seems to be what one would expect if a file is *both* "fixed" and "immutable".
"Immutable" means the contents do not change. But the file mappings could change.
Quoted 8 lines
"Fixed mapping" means the mappings do not change but contents could (as long as the ondisk mappings don't).
"Fixed metadata" means the entire inode (mappings included) cannot change but the contents could (as long as the ondisk mappings don't). (This does have the side effect of allowing writes without touching the mtime. Like XFS' invisible i/o.)
Actually, if you're going to have three flags you might as well make them orthogonal. That is, separate "fixed contents", "fixed mappings", "fixed metadata" -- and don't consider the mapping as metadata for this purpose.
-hpa
Quoted 13 lines · H. Peter Anvin
Quoted 11 lines · Sunil Mushran
"Immutable" means the contents do not change. But the file mappings could change.
"Fixed mapping" means the mappings do not change but contents could (as long as the ondisk mappings don't).
"Fixed metadata" means the entire inode (mappings included) cannot change but the contents could (as long as the ondisk mappings don't). (This does have the side effect of allowing writes without touching the mtime. Like XFS' invisible i/o.)
Quoted 4 lines
Actually, if you're going to have three flags you might as well make them orthogonal. That is, separate "fixed contents", "fixed mappings", "fixed metadata" -- and don't consider the mapping as metadata for this purpose.
I think Sunil was defining terms, not suggesting three flags ;-)
That said, it does allow all the possible characteristics in the discussion. The only ugly think I can see is that we'll have three new flags, plus the old immutable flag that means the same as setting all three new flags.
Joel
Signature · 10 lines
"But all my words come back to me In shades of mediocrity. Like emptiness in harmony I need someone to comfort me." Joel Becker Principal Software Developer Oracle E-mail: [email protected] Phone: (650) 506-8127
Quoted 18 lines · Joel Becker
Quoted 12 lines · H. Peter Anvin
Quoted 10 lines · Sunil Mushran
"Immutable" means the contents do not change. But the file mappings could change.
"Fixed mapping" means the mappings do not change but contents could (as long as the ondisk mappings don't).
"Fixed metadata" means the entire inode (mappings included) cannot change but the contents could (as long as the ondisk mappings don't). (This does have the side effect of allowing writes without touching the mtime. Like XFS' invisible i/o.)
Quoted 4 lines
Actually, if you're going to have three flags you might as well make them orthogonal. That is, separate "fixed contents", "fixed mappings", "fixed metadata" -- and don't consider the mapping as metadata for this purpose.
Quoted 5 lines
I think Sunil was defining terms, not suggesting three flags ;-)
That said, it does allow all the possible characteristics in the discussion. The only ugly think I can see is that we'll have three new flags, plus the old immutable flag that means the same as setting all three new flags.
Umm, what I think is being proposed --- and certainly what I think makes sense --- is that immutable just means the data contents (just as it has always meant), and we add two new flags, one which means "fixed mappings", or that the location on disk must not change, and "fixed metadata" which means all of the file metadata (not including the mappings) must not change.
- Ted