Skip to contentSkip to conversation

erofs-utils: mkfs: Use extended inodes when ctime is set

PATCH Superseded David Anderson via Linux-erofs 2 participants 7 messages Mar 1, 2022 – Mar 18, 2022 Library code

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 erofs-utils: mkfs: Use extended inodes when ctime is set

Currently ctime is effectively ignored because most inodes are compact. If ctime was set, and it's different from the build time, then extended inodes should be used instead.

To guarantee that timestamps do not cause extended inodes, a fixed timestamp should be used (-T).

Change-Id I3d2e5c5ffe2bdcabea6791b5def5973b507aa316
---
 lib/inode.c | 4 ++++
 1 file changed, 4 insertions(+)
lib/inode.c +4 -0
index f0a71a8..6f36c09 100644
@@ -730,6 +730,10 @@ static bool erofs_should_use_inode_extended(struct erofs_inode *inode)
 		return true;
 	if (inode->i_nlink > USHRT_MAX)
 		return true;
+	if (inode->i_ctime != sbi.build_time ||
+	    inode->i_ctime_nsec != sbi.build_time_nsec) {
+		return true;
+	}
 	return false;
 }
 
Signature · 1 line
2.35.1.574.g5d30c73bfb-goog
#1 lore raw

Hi David,

Quoted 6 lines · David Anderson via Linux-erofs

Currently ctime is effectively ignored because most inodes are compact. If ctime was set, and it's different from the build time, then extended inodes should be used instead.

To guarantee that timestamps do not cause extended inodes, a fixed timestamp should be used (-T).

Thanks for the patch.

How about introducing some option like '-E preserve-time' since compact inodes are mainly used for reproducible builds, which I think per-file timestamp is useless...

Also after I checked ociv2-brainstorm, they'd like to avoid timestamp as well, which can be effective used by EROFS compact inodes...

https://hackmd.io/@cyphar/ociv2-brainstorm

Thanks,
Gao Xiang

Quoted 24 lines
Change-Id I3d2e5c5ffe2bdcabea6791b5def5973b507aa316
---
 lib/inode.c | 4 ++++
 1 file changed, 4 insertions(+)
lib/inode.c +4 -0
index f0a71a8..6f36c09 100644
@@ -730,6 +730,10 @@ static bool erofs_should_use_inode_extended(struct erofs_inode *inode)
 		return true;
 	if (inode->i_nlink > USHRT_MAX)
 		return true;
+	if (inode->i_ctime != sbi.build_time ||
+	    inode->i_ctime_nsec != sbi.build_time_nsec) {
+		return true;
+	}
 	return false;
 }
 
Signature · 1 line
2.35.1.574.g5d30c73bfb-goog
#2 lore raw
Quoted 6 lines · Gao Xiang

How about introducing some option like '-E preserve-time' since compact inodes are mainly used for reproducible builds, which I think per-file timestamp is useless...

Also after I checked ociv2-brainstorm, they'd like to avoid timestamp as well, which can be effective used by EROFS compact inodes...

Sounds good, I'll post a new patch with this option.

Best,

-David

#3 lore raw
Quoted 9 lines · David Anderson
Quoted 6 lines · Gao Xiang

How about introducing some option like '-E preserve-time' since compact inodes are mainly used for reproducible builds, which I think per-file timestamp is useless...

Also after I checked ociv2-brainstorm, they'd like to avoid timestamp as well, which can be effective used by EROFS compact inodes...

Sounds good, I'll post a new patch with this option.

(Or maybe just "--preserve-mtime" if it's supposed to be used frequently in
specific use cases...)

Thanks,
Gao Xiang

Quoted 4 lines

Best,

-David

#4 lore raw
Quoted 4 lines · Gao Xiang

(Or maybe just "--preserve-mtime" if it's supposed to be used frequently in
specific use cases...)

Looking at this again - is a new flag needed?

If -T is specified, the build will be reproducible. All timestamps will be equal, and we'll never force an extended inode due to mismatching timestamps. The patch has no effect in this case.

If -T is *not* specified, inode mtime *could* be reproducible (depending on how the user creates the files), but sbi.build_time isn't. Any of these inodes that gets the compact layout will use the unreproducible build time instead. So, switching to extended inodes here strictly improves reproducibility and determinism.

That said, it is a change in behavior, so maybe that warrants a flag anyway.

For what it's worth, in AOSP, "-T" is specified for production builds. Non-production builds have "mtime" preserved for pushing incremental changes.

Best,

-David

#5 lore raw

Hi David,

Quoted 12 lines · David Anderson via Linux-erofs
Quoted 4 lines · Gao Xiang

(Or maybe just "--preserve-mtime" if it's supposed to be used frequently in
specific use cases...)

Looking at this again - is a new flag needed?

If -T is specified, the build will be reproducible. All timestamps will be equal, and we'll never force an extended inode due to mismatching timestamps. The patch has no effect in this case.

Quoted 5 lines

If -T is *not* specified, inode mtime *could* be reproducible (depending on how the user creates the files), but sbi.build_time isn't. Any of these inodes that gets the compact layout will use the unreproducible build time instead. So, switching to extended inodes here strictly improves reproducibility and determinism.

Yeah, I know what's your main point here. Ok, that's fine we could change the current behavior. Yet I still think the current behavior may benefit some use cases (e.g. metadata size is sensible, we need to avoid extended inodes no matter per-filetimestamp is)..

So instead, how about introducing --ignore-mtime instead for the current behavior, and change the default behavior into using extended inodes?

Quoted 6 lines

That said, it is a change in behavior, so maybe that warrants a flag anyway.

For what it's worth, in AOSP, "-T" is specified for production builds. Non-production builds have "mtime" preserved for pushing incremental changes.

Thanks for the information!

Thanks,
Gao Xiang

Quoted 4 lines

Best,

-David

#6 lore raw
Quoted 10 lines · Gao Xiang

Yeah, I know what's your main point here. Ok, that's fine we could change the current behavior. Yet I still think the current behavior may benefit some use cases (e.g. metadata size is sensible, we need to avoid extended inodes no matter per-filetimestamp is)..

So instead, how about introducing --ignore-mtime instead for the current behavior, and change the default behavior into using extended inodes?

Sounds good to me, I'll make this change.