Skip to contentSkip to conversation

AOSP: erofs-utils: fix sub directory prefix path in canned fs_config

PATCH In review Yue Hu 2 participants 20 messages Dec 22, 2020 – Dec 24, 2020 Library code

lore
Share & tools
lore

Yue Hu

author
#0 lore raw

PATCH 1/1 AOSP: erofs-utils: fix sub directory prefix path in canned fs_config

From: Yue Hu <[email protected]>

We observe that creating image failed to find [%s] in canned fs_config using --fs-config-file option under Android 10.

Notice that canned fs_config has a prefix to sub directory if mount_point presents. However, erofs_fspath() does not contain the prefix.

Moreover, we should not add the mount point to fspath on root inode for fs_config() branch.

---
 include/erofs/config.h |  4 ++++
 lib/inode.c            | 29 +++++++++++++++++++----------
 mkfs/main.c            | 14 ++++++++++----
 3 files changed, 33 insertions(+), 14 deletions(-)
include/erofs/config.h +4 -0
index 02ddf59..1277eda 100644
@@ -58,6 +58,10 @@ struct erofs_configure {
 	char *mount_point;
 	char *target_out_path;
 	char *fs_config_file;
+	void (*fs_config_func)(const char *path, int dir,
+			       const char *target_out_path,
+			       unsigned *uid, unsigned *gid,
+			       unsigned *mode, uint64_t *capabilities);
 #endif
 };
 
lib/inode.c +19 -10
index eb2e0f2..d0805cd 100644
@@ -684,20 +684,29 @@ int erofs_droid_inode_fsconfig(struct erofs_inode *inode,
 	char *fspath;
 
 	inode->capabilities = 0;
-	if (cfg.fs_config_file)
-		canned_fs_config(erofs_fspath(path),
-				 S_ISDIR(st->st_mode),
-				 cfg.target_out_path,
-				 &uid, &gid, &mode, &inode->capabilities);
-	else if (cfg.mount_point) {
+
+	if (erofs_fspath(path)[0] == '\0')
+		goto e_fspath;
+
+	if (cfg.mount_point) {
 		if (asprintf(&fspath, "%s/%s", cfg.mount_point,
 			     erofs_fspath(path)) <= 0)
 			return -ENOMEM;
-
-		fs_config(fspath, S_ISDIR(st->st_mode),
-			  cfg.target_out_path,
-			  &uid, &gid, &mode, &inode->capabilities);
+		if (cfg.fs_config_func)
+			cfg.fs_config_func(fspath,
+					   S_ISDIR(st->st_mode),
+					   cfg.target_out_path,
+					   &uid, &gid, &mode,
+					   &inode->capabilities);
 		free(fspath);
+	} else {
+e_fspath:
+		if (cfg.fs_config_func)
+			cfg.fs_config_func(erofs_fspath(path),
+					   S_ISDIR(st->st_mode),
+					   cfg.target_out_path,
+					   &uid, &gid, &mode,
+					   &inode->capabilities);
 	}
 	st->st_uid = uid;
 	st->st_gid = gid;
mkfs/main.c +10 -4
index c63b274..684767c 100644
@@ -474,10 +474,16 @@ int main(int argc, char **argv)
 	}
 
 #ifdef WITH_ANDROID
-	if (cfg.fs_config_file &&
-	    load_canned_fs_config(cfg.fs_config_file) < 0) {
-		erofs_err("failed to load fs config %s", cfg.fs_config_file);
-		return 1;
+	cfg.fs_config_func = NULL;
+	if (cfg.fs_config_file) {
+		if (load_canned_fs_config(cfg.fs_config_file) < 0) {
+			erofs_err("failed to load fs config %s",
+					cfg.fs_config_file);
+			return 1;
+		}
+		cfg.fs_config_func = canned_fs_config;
+	} else if (cfg.mount_point) {
+		cfg.fs_config_func = fs_config;
 	}
 #endif
 
Signature · 1 line
1.9.1
#1 lore raw

Hi Yue,

Quoted 8 lines · Yue Hu

From: Yue Hu <[email protected]>

We observe that creating image failed to find [%s] in canned fs_config using --fs-config-file option under Android 10.

Notice that canned fs_config has a prefix to sub directory if mount_point presents. However, erofs_fspath() does not contain the prefix.

Thanks for your patch. Let me play with it a bit this weekend (I'm not quite familiar with canned fs_config, it would be of great help to add some hints/steps for me to confirm this issue.... since some other vendors already use it without report (maybe they don't use canned fs_config.)

Quoted 2 lines

Moreover, we should not add the mount point to fspath on root inode for fs_config() branch.

Is there some descriptive words or reference for this? To be honest, I'm quite unsure about this kind of Android-specific things... :(

Thanks,
Gao Xiang

Quoted 95 lines
---
 include/erofs/config.h |  4 ++++
 lib/inode.c            | 29 +++++++++++++++++++----------
 mkfs/main.c            | 14 ++++++++++----
 3 files changed, 33 insertions(+), 14 deletions(-)
include/erofs/config.h +4 -0
index 02ddf59..1277eda 100644
@@ -58,6 +58,10 @@ struct erofs_configure {
 	char *mount_point;
 	char *target_out_path;
 	char *fs_config_file;
+	void (*fs_config_func)(const char *path, int dir,
+			       const char *target_out_path,
+			       unsigned *uid, unsigned *gid,
+			       unsigned *mode, uint64_t *capabilities);
 #endif
 };
 
lib/inode.c +19 -10
index eb2e0f2..d0805cd 100644
@@ -684,20 +684,29 @@ int erofs_droid_inode_fsconfig(struct erofs_inode *inode,
 	char *fspath;
 
 	inode->capabilities = 0;
-	if (cfg.fs_config_file)
-		canned_fs_config(erofs_fspath(path),
-				 S_ISDIR(st->st_mode),
-				 cfg.target_out_path,
-				 &uid, &gid, &mode, &inode->capabilities);
-	else if (cfg.mount_point) {
+
+	if (erofs_fspath(path)[0] == '\0')
+		goto e_fspath;
+
+	if (cfg.mount_point) {
 		if (asprintf(&fspath, "%s/%s", cfg.mount_point,
 			     erofs_fspath(path)) <= 0)
 			return -ENOMEM;
-
-		fs_config(fspath, S_ISDIR(st->st_mode),
-			  cfg.target_out_path,
-			  &uid, &gid, &mode, &inode->capabilities);
+		if (cfg.fs_config_func)
+			cfg.fs_config_func(fspath,
+					   S_ISDIR(st->st_mode),
+					   cfg.target_out_path,
+					   &uid, &gid, &mode,
+					   &inode->capabilities);
 		free(fspath);
+	} else {
+e_fspath:
+		if (cfg.fs_config_func)
+			cfg.fs_config_func(erofs_fspath(path),
+					   S_ISDIR(st->st_mode),
+					   cfg.target_out_path,
+					   &uid, &gid, &mode,
+					   &inode->capabilities);
 	}
 	st->st_uid = uid;
 	st->st_gid = gid;
mkfs/main.c +10 -4
index c63b274..684767c 100644
@@ -474,10 +474,16 @@ int main(int argc, char **argv)
 	}
 
 #ifdef WITH_ANDROID
-	if (cfg.fs_config_file &&
-	    load_canned_fs_config(cfg.fs_config_file) < 0) {
-		erofs_err("failed to load fs config %s", cfg.fs_config_file);
-		return 1;
+	cfg.fs_config_func = NULL;
+	if (cfg.fs_config_file) {
+		if (load_canned_fs_config(cfg.fs_config_file) < 0) {
+			erofs_err("failed to load fs config %s",
+					cfg.fs_config_file);
+			return 1;
+		}
+		cfg.fs_config_func = canned_fs_config;
+	} else if (cfg.mount_point) {
+		cfg.fs_config_func = fs_config;
 	}
 #endif
 
Signature · 1 line
1.9.1
author
#2 lore raw
Quoted 8 lines · On Tue, 22 Dec 2020 11:44:55 +0800 Gao Xiang <[email protected]> wrote:

Hi Yue,

Quoted 5 lines · Yue Hu

From: Yue Hu <[email protected]>

We observe that creating image failed to find [%s] in canned fs_config using --fs-config-file option under Android 10.

Quoted 8 lines
Quoted 3 lines

Notice that canned fs_config has a prefix to sub directory if mount_point presents. However, erofs_fspath() does not contain the prefix.

Thanks for your patch. Let me play with it a bit this weekend (I'm not quite familiar with canned fs_config, it would be of great help to add some hints/steps for me to confirm this issue.... since some other vendors already use it without report (maybe they don't use canned fs_config.)

Hi Xiang,

It's been observed under QC/QSSI platform with dynamic partition.

such as product.img/product_filesystem_config.txt:

```txt
 0 0 755 selabel=u:object_r:rootfs:s0 capabilities=0x0
product/app 0 0 755 selabel=u:object_r:system_file:s0 capabilities=0x0
product/app/CalculatorGoogle 0 0 755 selabel=u:object_r:system_file:s0 capabilities=0x0
```

product_filesystem_config.txt should be from below build:

$(call fs_config,$(zip_root)/PRODUCT,product/) > $(zip_root)/META/product_filesystem_config.txt

Do not observe this issue in squashfs, so i check related code of squashfs, squashfs have considered the mount point, also ext4 does. So, erofs should be same as one long used before.

After this patch, build & boot are working fine by test.

Here's a change from mksqushfs: Allow passing fs_config file for generating mksquashfs

Quoted 6 lines
Quoted 3 lines

Moreover, we should not add the mount point to fspath on root inode for fs_config() branch.

Is there some descriptive words or reference for this? To be honest, I'm quite unsure about this kind of Android-specific things... :(

Refer to change: mksquashfs: Run android_fs_config() on the root inode

I think erofs of AOSP has this issue also. Am i right?

Thx.

Quoted 100 lines

Thanks,
Gao Xiang

Quoted 95 lines
---
 include/erofs/config.h |  4 ++++
 lib/inode.c            | 29 +++++++++++++++++++----------
 mkfs/main.c            | 14 ++++++++++----
 3 files changed, 33 insertions(+), 14 deletions(-)
include/erofs/config.h +4 -0
index 02ddf59..1277eda 100644
@@ -58,6 +58,10 @@ struct erofs_configure {
 	char *mount_point;
 	char *target_out_path;
 	char *fs_config_file;
+	void (*fs_config_func)(const char *path, int dir,
+			       const char *target_out_path,
+			       unsigned *uid, unsigned *gid,
+			       unsigned *mode, uint64_t *capabilities);
 #endif
 };
 
lib/inode.c +19 -10
index eb2e0f2..d0805cd 100644
@@ -684,20 +684,29 @@ int erofs_droid_inode_fsconfig(struct erofs_inode *inode,
 	char *fspath;
 
 	inode->capabilities = 0;
-	if (cfg.fs_config_file)
-		canned_fs_config(erofs_fspath(path),
-				 S_ISDIR(st->st_mode),
-				 cfg.target_out_path,
-				 &uid, &gid, &mode, &inode->capabilities);
-	else if (cfg.mount_point) {
+
+	if (erofs_fspath(path)[0] == '\0')
+		goto e_fspath;
+
+	if (cfg.mount_point) {
 		if (asprintf(&fspath, "%s/%s", cfg.mount_point,
 			     erofs_fspath(path)) <= 0)
 			return -ENOMEM;
-
-		fs_config(fspath, S_ISDIR(st->st_mode),
-			  cfg.target_out_path,
-			  &uid, &gid, &mode, &inode->capabilities);
+		if (cfg.fs_config_func)
+			cfg.fs_config_func(fspath,
+					   S_ISDIR(st->st_mode),
+					   cfg.target_out_path,
+					   &uid, &gid, &mode,
+					   &inode->capabilities);
 		free(fspath);
+	} else {
+e_fspath:
+		if (cfg.fs_config_func)
+			cfg.fs_config_func(erofs_fspath(path),
+					   S_ISDIR(st->st_mode),
+					   cfg.target_out_path,
+					   &uid, &gid, &mode,
+					   &inode->capabilities);
 	}
 	st->st_uid = uid;
 	st->st_gid = gid;
mkfs/main.c +10 -4
index c63b274..684767c 100644
@@ -474,10 +474,16 @@ int main(int argc, char **argv)
 	}
 
 #ifdef WITH_ANDROID
-	if (cfg.fs_config_file &&
-	    load_canned_fs_config(cfg.fs_config_file) < 0) {
-		erofs_err("failed to load fs config %s", cfg.fs_config_file);
-		return 1;
+	cfg.fs_config_func = NULL;
+	if (cfg.fs_config_file) {
+		if (load_canned_fs_config(cfg.fs_config_file) < 0) {
+			erofs_err("failed to load fs config %s",
+					cfg.fs_config_file);
+			return 1;
+		}
+		cfg.fs_config_func = canned_fs_config;
+	} else if (cfg.mount_point) {
+		cfg.fs_config_func = fs_config;
 	}
 #endif
 
Signature · 1 line
1.9.1
#3 lore raw
Quoted 34 lines · Yue Hu
Quoted 8 lines · On Tue, 22 Dec 2020 11:44:55 +0800 Gao Xiang <[email protected]> wrote:

Hi Yue,

Quoted 5 lines · Yue Hu

From: Yue Hu <[email protected]>

We observe that creating image failed to find [%s] in canned fs_config using --fs-config-file option under Android 10.

Quoted 8 lines
Quoted 3 lines

Notice that canned fs_config has a prefix to sub directory if mount_point presents. However, erofs_fspath() does not contain the prefix.

Thanks for your patch. Let me play with it a bit this weekend (I'm not quite familiar with canned fs_config, it would be of great help to add some hints/steps for me to confirm this issue.... since some other vendors already use it without report (maybe they don't use canned fs_config.)

Hi Xiang,

It's been observed under QC/QSSI platform with dynamic partition.

such as product.img/product_filesystem_config.txt:

```txt
 0 0 755 selabel=u:object_r:rootfs:s0 capabilities=0x0
product/app 0 0 755 selabel=u:object_r:system_file:s0 capabilities=0x0
product/app/CalculatorGoogle 0 0 755 selabel=u:object_r:system_file:s0 capabilities=0x0
```

product_filesystem_config.txt should be from below build:

Quoted 8 lines

$(call fs_config,$(zip_root)/PRODUCT,product/) > $(zip_root)/META/product_filesystem_config.txt

Do not observe this issue in squashfs, so i check related code of squashfs, squashfs have considered the mount point, also ext4 does. So, erofs should be same as one long used before.

After this patch, build & boot are working fine by test.

Here's a change from mksqushfs: Allow passing fs_config file for generating mksquashfs

Ok, I will try to find some clue to verify later.

Quoted 11 lines
Quoted 7 lines
Quoted 4 lines

Moreover, we should not add the mount point to fspath on root inode for fs_config() branch.

Is there some descriptive words or reference for this? To be honest, I'm quite unsure about this kind of Android-specific things... :(

Refer to change: mksquashfs: Run android_fs_config() on the root inode

I think erofs of AOSP has this issue also. Am i right?

Not quite sure if it effects non-canned fs_config after reading the commit message...
https://android.googlesource.com/platform/external/squashfs-tools/+/85a6bc1e52bb911f195c5dc0890717913938c2d1%5E%21/#F0

And no permission to access Bug 72745016, so... maybe we need to limit this to non-canned fs_config only? (at least confirming the original case would be better)

BTW, Also, from its testcase command line in the commit message:
"mksquashfs path system.raw.img -fs-config-file fs_config -android-fs-config"

I'm not sure if "--mount-point" is passed in so I think for such case no need to use such "goto" as well?

Thanks,
Gao Xiang

#4 lore raw

Hi Yue,

On Tue, Dec 22, 2020 at 02:31:12PM +0800, Gao Xiang wrote:

...

Quoted 23 lines

Ok, I will try to find some clue to verify later.

Quoted 11 lines
Quoted 7 lines
Quoted 4 lines

Moreover, we should not add the mount point to fspath on root inode for fs_config() branch.

Is there some descriptive words or reference for this? To be honest, I'm quite unsure about this kind of Android-specific things... :(

Refer to change: mksquashfs: Run android_fs_config() on the root inode

I think erofs of AOSP has this issue also. Am i right?

Not quite sure if it effects non-canned fs_config after reading the commit message...
https://android.googlesource.com/platform/external/squashfs-tools/+/85a6bc1e52bb911f195c5dc0890717913938c2d1%5E%21/#F0

And no permission to access Bug 72745016, so... maybe we need to limit this to non-canned fs_config only? (at least confirming the original case would be better)

Quoted 8 lines

BTW, Also, from its testcase command line in the commit message:
"mksquashfs path system.raw.img -fs-config-file fs_config -android-fs-config"

I'm not sure if "--mount-point" is passed in so I think for such case no need to use such "goto" as well?

Thanks,
Gao Xiang

Could you verify the following patch if possible? (without compilation, I don't have test environment now since AOSP code is on my PC)

From: Yue Hu <[email protected]>
Date: Tue, 22 Dec 2020 14:52:22 +0800
Subject: [PATCH] AOSP: erofs-utils: fix sub-directory prefix for canned
fs_config

"failed to find [%s] in canned fs_config" is observed by using "--fs-config-file" option under Android 10.

Notice that canned fs_config has a prefix to sub-directory if "--mount-point" presents. However, such prefix cannot be added by just using erofs_fspath().

Fixes 8a9e8046f170 ("AOSP: erofs-utils: add fs_config support")
---
 lib/inode.c | 18 +++++++++++-------
 1 file changed, 11 insertions(+), 7 deletions(-)
lib/inode.c +11 -7
index 3d634fc..9469074 100644
@@ -701,21 +701,25 @@ int erofs_droid_inode_fsconfig(struct erofs_inode *inode,
 	char *fspath;
 
 	inode->capabilities = 0;
+	if (!cfg.mount_point)
+		fspath = erofs_fspath(path);
+	else if (asprintf(&fspath, "%s/%s", cfg.mount_point,
+			  erofs_fspath(path)) <= 0)
+		return -ENOMEM;
+
+
 	if (cfg.fs_config_file)
-		canned_fs_config(erofs_fspath(path),
+		canned_fs_config(fspath,
 				 S_ISDIR(st->st_mode),
 				 cfg.target_out_path,
 				 &uid, &gid, &mode, &inode->capabilities);
-	else if (cfg.mount_point) {
-		if (asprintf(&fspath, "%s/%s", cfg.mount_point,
-			     erofs_fspath(path)) <= 0)
-			return -ENOMEM;
-
+	else
 		fs_config(fspath, S_ISDIR(st->st_mode),
 			  cfg.target_out_path,
 			  &uid, &gid, &mode, &inode->capabilities);
+
+	if (cfg.mount_point)
 		free(fspath);
-	}
 	st->st_uid = uid;
 	st->st_gid = gid;
 	st->st_mode = mode | stat_file_type_mask;
Signature · 1 line
2.27.0
author
#5 lore raw
Quoted 32 lines · On Tue, 22 Dec 2020 15:04:58 +0800 Gao Xiang <[email protected]> wrote:

Hi Yue,

On Tue, Dec 22, 2020 at 02:31:12PM +0800, Gao Xiang wrote:

...

Quoted 26 lines

Ok, I will try to find some clue to verify later.

Quoted 11 lines
Quoted 7 lines
Quoted 4 lines

Moreover, we should not add the mount point to fspath on root inode for fs_config() branch.

Is there some descriptive words or reference for this? To be honest, I'm quite unsure about this kind of Android-specific things... :(

Refer to change: mksquashfs: Run android_fs_config() on the root inode

I think erofs of AOSP has this issue also. Am i right?

Not quite sure if it effects non-canned fs_config after reading the commit message...
https://android.googlesource.com/platform/external/squashfs-tools/+/85a6bc1e52bb911f195c5dc0890717913938c2d1%5E%21/#F0

And no permission to access Bug 72745016, so... maybe we need to limit this to non-canned fs_config only? (at least confirming the original case would be better)

BTW, Also, from its testcase command line in the commit message:
"mksquashfs path system.raw.img -fs-config-file fs_config -android-fs-config"

Quoted 8 lines
Quoted 5 lines

I'm not sure if "--mount-point" is passed in so I think for such case no need to use such "goto" as well?

Thanks,
Gao Xiang

Could you verify the following patch if possible? (without compilation, I don't have test environment now since AOSP code is on my PC)

Ok, i will verify today.

Thx.

Quoted 57 lines

From: Yue Hu <[email protected]>
Date: Tue, 22 Dec 2020 14:52:22 +0800
Subject: [PATCH] AOSP: erofs-utils: fix sub-directory prefix for canned
fs_config

"failed to find [%s] in canned fs_config" is observed by using "--fs-config-file" option under Android 10.

Notice that canned fs_config has a prefix to sub-directory if "--mount-point" presents. However, such prefix cannot be added by just using erofs_fspath().

Fixes 8a9e8046f170 ("AOSP: erofs-utils: add fs_config support")
---
 lib/inode.c | 18 +++++++++++-------
 1 file changed, 11 insertions(+), 7 deletions(-)
lib/inode.c +11 -7
index 3d634fc..9469074 100644
@@ -701,21 +701,25 @@ int erofs_droid_inode_fsconfig(struct erofs_inode *inode,
 	char *fspath;
 
 	inode->capabilities = 0;
+	if (!cfg.mount_point)
+		fspath = erofs_fspath(path);
+	else if (asprintf(&fspath, "%s/%s", cfg.mount_point,
+			  erofs_fspath(path)) <= 0)
+		return -ENOMEM;
+
+
 	if (cfg.fs_config_file)
-		canned_fs_config(erofs_fspath(path),
+		canned_fs_config(fspath,
 				 S_ISDIR(st->st_mode),
 				 cfg.target_out_path,
 				 &uid, &gid, &mode, &inode->capabilities);
-	else if (cfg.mount_point) {
-		if (asprintf(&fspath, "%s/%s", cfg.mount_point,
-			     erofs_fspath(path)) <= 0)
-			return -ENOMEM;
-
+	else
 		fs_config(fspath, S_ISDIR(st->st_mode),
 			  cfg.target_out_path,
 			  &uid, &gid, &mode, &inode->capabilities);
+
+	if (cfg.mount_point)
 		free(fspath);
-	}
 	st->st_uid = uid;
 	st->st_gid = gid;
 	st->st_mode = mode | stat_file_type_mask;
author
#6 lore raw
In reply to Gao Xiang
Quoted 65 lines · On Tue, 22 Dec 2020 15:04:58 +0800 Gao Xiang <[email protected]> wrote:

Hi Yue,

On Tue, Dec 22, 2020 at 02:31:12PM +0800, Gao Xiang wrote:

...

Quoted 23 lines

Ok, I will try to find some clue to verify later.

Quoted 11 lines
Quoted 7 lines
Quoted 4 lines

Moreover, we should not add the mount point to fspath on root inode for fs_config() branch.

Is there some descriptive words or reference for this? To be honest, I'm quite unsure about this kind of Android-specific things... :(

Refer to change: mksquashfs: Run android_fs_config() on the root inode

I think erofs of AOSP has this issue also. Am i right?

Not quite sure if it effects non-canned fs_config after reading the commit message...
https://android.googlesource.com/platform/external/squashfs-tools/+/85a6bc1e52bb911f195c5dc0890717913938c2d1%5E%21/#F0

And no permission to access Bug 72745016, so... maybe we need to limit this to non-canned fs_config only? (at least confirming the original case would be better)

Quoted 8 lines

BTW, Also, from its testcase command line in the commit message:
"mksquashfs path system.raw.img -fs-config-file fs_config -android-fs-config"

I'm not sure if "--mount-point" is passed in so I think for such case no need to use such "goto" as well?

Thanks,
Gao Xiang

Could you verify the following patch if possible? (without compilation, I don't have test environment now since AOSP code is on my PC)

From: Yue Hu <[email protected]>
Date: Tue, 22 Dec 2020 14:52:22 +0800
Subject: [PATCH] AOSP: erofs-utils: fix sub-directory prefix for canned
fs_config

"failed to find [%s] in canned fs_config" is observed by using "--fs-config-file" option under Android 10.

Notice that canned fs_config has a prefix to sub-directory if "--mount-point" presents. However, such prefix cannot be added by just using erofs_fspath().

Fixes 8a9e8046f170 ("AOSP: erofs-utils: add fs_config support")
---
 lib/inode.c | 18 +++++++++++-------
 1 file changed, 11 insertions(+), 7 deletions(-)
lib/inode.c +0 -0
index 3d634fc..9469074 100644

lib/inode.c int erofs_droid_inode_fsconfig(struct erofs_inode *inode, · line 705

Quoted 5 lines
lib/inode.c +2 -0
@@ -701,21 +701,25 @@ int erofs_droid_inode_fsconfig(struct erofs_inode *inode,
 	char *fspath;
 
 	inode->capabilities = 0;
+	if (!cfg.mount_point)
+		fspath = erofs_fspath(path);

lib/inode.c:688:10: error: assigning to 'char *' from 'const char *' discards...

                fspath = erofs_fspath(path);
                       ^ ~~~~~~~~~~~~~~~~~~

- fspath = erofs_fspath(path);
+ fspath = (char *)erofs_fspath(path);

lib/inode.c int erofs_droid_inode_fsconfig(struct erofs_inode *inode, · line 707

Quoted 2 lines
lib/inode.c +2 -0
+	else if (asprintf(&fspath, "%s/%s", cfg.mount_point,
+			  erofs_fspath(path)) <= 0)

The argument of path will be root directory. And canned fs_config for root directory as below:

0 0 755 selabel=u:object_r:rootfs:s0 capabilities=0x0

So, cannot add mount point to root directory for canned fs_config. And what about non-canned fs_config?

Thx.

Quoted 25 lines
lib/inode.c +7 -7
+		return -ENOMEM;
+
+
 	if (cfg.fs_config_file)
-		canned_fs_config(erofs_fspath(path),
+		canned_fs_config(fspath,
 				 S_ISDIR(st->st_mode),
 				 cfg.target_out_path,
 				 &uid, &gid, &mode, &inode->capabilities);
-	else if (cfg.mount_point) {
-		if (asprintf(&fspath, "%s/%s", cfg.mount_point,
-			     erofs_fspath(path)) <= 0)
-			return -ENOMEM;
-
+	else
 		fs_config(fspath, S_ISDIR(st->st_mode),
 			  cfg.target_out_path,
 			  &uid, &gid, &mode, &inode->capabilities);
+
+	if (cfg.mount_point)
 		free(fspath);
-	}
 	st->st_uid = uid;
 	st->st_gid = gid;
 	st->st_mode = mode | stat_file_type_mask;
#7 lore raw
Quoted 78 lines · Yue Hu
Quoted 65 lines · On Tue, 22 Dec 2020 15:04:58 +0800 Gao Xiang <[email protected]> wrote:

Hi Yue,

On Tue, Dec 22, 2020 at 02:31:12PM +0800, Gao Xiang wrote:

...

Quoted 23 lines

Ok, I will try to find some clue to verify later.

Quoted 11 lines
Quoted 7 lines
Quoted 4 lines

Moreover, we should not add the mount point to fspath on root inode for fs_config() branch.

Is there some descriptive words or reference for this? To be honest, I'm quite unsure about this kind of Android-specific things... :(

Refer to change: mksquashfs: Run android_fs_config() on the root inode

I think erofs of AOSP has this issue also. Am i right?

Not quite sure if it effects non-canned fs_config after reading the commit message...
https://android.googlesource.com/platform/external/squashfs-tools/+/85a6bc1e52bb911f195c5dc0890717913938c2d1%5E%21/#F0

And no permission to access Bug 72745016, so... maybe we need to limit this to non-canned fs_config only? (at least confirming the original case would be better)

Quoted 8 lines

BTW, Also, from its testcase command line in the commit message:
"mksquashfs path system.raw.img -fs-config-file fs_config -android-fs-config"

I'm not sure if "--mount-point" is passed in so I think for such case no need to use such "goto" as well?

Thanks,
Gao Xiang

Could you verify the following patch if possible? (without compilation, I don't have test environment now since AOSP code is on my PC)

From: Yue Hu <[email protected]>
Date: Tue, 22 Dec 2020 14:52:22 +0800
Subject: [PATCH] AOSP: erofs-utils: fix sub-directory prefix for canned
fs_config

"failed to find [%s] in canned fs_config" is observed by using "--fs-config-file" option under Android 10.

Notice that canned fs_config has a prefix to sub-directory if "--mount-point" presents. However, such prefix cannot be added by just using erofs_fspath().

Fixes 8a9e8046f170 ("AOSP: erofs-utils: add fs_config support")
---
 lib/inode.c | 18 +++++++++++-------
 1 file changed, 11 insertions(+), 7 deletions(-)
lib/inode.c +0 -0
index 3d634fc..9469074 100644

lib/inode.c int erofs_droid_inode_fsconfig(struct erofs_inode *inode, · line 705

Quoted 5 lines
lib/inode.c +2 -0
@@ -701,21 +701,25 @@ int erofs_droid_inode_fsconfig(struct erofs_inode *inode,
 	char *fspath;
 
 	inode->capabilities = 0;
+	if (!cfg.mount_point)
+		fspath = erofs_fspath(path);

lib/inode.c:688:10: error: assigning to 'char *' from 'const char *' discards...

                fspath = erofs_fspath(path);
                       ^ ~~~~~~~~~~~~~~~~~~

Review

Quoted 2 lines
(quoted diff) +1 -1
-           fspath = erofs_fspath(path);
+         fspath = (char *)erofs_fspath(path);

oops, I think it can be modified as a temporary workaround, will submit a formal patch after verification.

Quoted 8 lines

lib/inode.c int erofs_droid_inode_fsconfig(struct erofs_inode *inode, · line 707

Quoted 4 lines
lib/inode.c +2 -0
+	else if (asprintf(&fspath, "%s/%s", cfg.mount_point,
+			  erofs_fspath(path)) <= 0)

The argument of path will be root directory. And canned fs_config for root directory as below:

Quoted 4 lines

0 0 755 selabel=u:object_r:rootfs:s0 capabilities=0x0

So, cannot add mount point to root directory for canned fs_config. And what about non-canned fs_config?

Not quite sure what you mean. For non-canned fs_config, we didn't observed any strange before (I ported to cuttlefish and hikey960 with boot success, also as I mentioned before some other vendors already use it.)

I think the following commit is only useful for squashfs since its (non)root inode workflows are different, so need to add in two difference place. But mkfs.erofs is not.
https://android.googlesource.com/platform/external/squashfs-tools/+/85a6bc1e52bb911f195c5dc0890717913938c2d1%5E%21/#F0

For root inode is erofs, I think erofs_fspath(path) would return "", so that case is included as well.... Am I missing something?

Thanks,
Gao Xiang

Quoted 30 lines

Thx.

Quoted 25 lines
lib/inode.c +7 -7
+		return -ENOMEM;
+
+
 	if (cfg.fs_config_file)
-		canned_fs_config(erofs_fspath(path),
+		canned_fs_config(fspath,
 				 S_ISDIR(st->st_mode),
 				 cfg.target_out_path,
 				 &uid, &gid, &mode, &inode->capabilities);
-	else if (cfg.mount_point) {
-		if (asprintf(&fspath, "%s/%s", cfg.mount_point,
-			     erofs_fspath(path)) <= 0)
-			return -ENOMEM;
-
+	else
 		fs_config(fspath, S_ISDIR(st->st_mode),
 			  cfg.target_out_path,
 			  &uid, &gid, &mode, &inode->capabilities);
+
+	if (cfg.mount_point)
 		free(fspath);
-	}
 	st->st_uid = uid;
 	st->st_gid = gid;
 	st->st_mode = mode | stat_file_type_mask;
#8 lore raw

On Tue, Dec 22, 2020 at 05:13:40PM +0800, Gao Xiang wrote:

...

Quoted 29 lines

Review

Quoted 8 lines

lib/inode.c int erofs_droid_inode_fsconfig(struct erofs_inode *inode, · line 705

Quoted 1 line
lib/inode.c +1 -0
+		fspath = erofs_fspath(path);

lib/inode.c:688:10: error: assigning to 'char *' from 'const char *' discards...

                fspath = erofs_fspath(path);
                       ^ ~~~~~~~~~~~~~~~~~~
(quoted diff) +1 -1
-           fspath = erofs_fspath(path);
+         fspath = (char *)erofs_fspath(path);

oops, I think it can be modified as a temporary workaround, will submit a formal patch after verification.

Quoted 8 lines

lib/inode.c int erofs_droid_inode_fsconfig(struct erofs_inode *inode, · line 707

Quoted 4 lines
lib/inode.c +2 -0
+	else if (asprintf(&fspath, "%s/%s", cfg.mount_point,
+			  erofs_fspath(path)) <= 0)

The argument of path will be root directory. And canned fs_config for root directory as below:

Quoted 4 lines

0 0 755 selabel=u:object_r:rootfs:s0 capabilities=0x0

So, cannot add mount point to root directory for canned fs_config. And what about non-canned fs_config?

Not quite sure what you mean. For non-canned fs_config, we didn't observed any strange before (I ported to cuttlefish and hikey960 with boot success, also as I mentioned before some other vendors already use it.)

Quoted 6 lines

I think the following commit is only useful for squashfs since its (non)root inode workflows are different, so need to add in two difference place. But mkfs.erofs is not.
https://android.googlesource.com/platform/external/squashfs-tools/+/85a6bc1e52bb911f195c5dc0890717913938c2d1%5E%21/#F0

For root inode is erofs, I think erofs_fspath(path) would return "", so that case is included as well.... Am I missing something?

Also I don't find any special handling logic in make_ext4fs for root inode
https://android.googlesource.com/platform/system/extras/+/refs/heads/oreo-release/ext4_utils/make_ext4fs.c#229

Actually I think the squashfs commit above might be wrong if "--mount-point" is specified if my understanding is correct. Anyway, could you help verify it in advance?

Thanks,
Gao Xiang

Quoted 6 lines

Thanks,
Gao Xiang

Quoted 2 lines

Thx.

author
#9 lore raw
In reply to Gao Xiang
Quoted 102 lines · On Tue, 22 Dec 2020 17:13:40 +0800 Gao Xiang <[email protected]> wrote:
Quoted 78 lines · Yue Hu
Quoted 65 lines · On Tue, 22 Dec 2020 15:04:58 +0800 Gao Xiang <[email protected]> wrote:

Hi Yue,

On Tue, Dec 22, 2020 at 02:31:12PM +0800, Gao Xiang wrote:

...

Quoted 23 lines

Ok, I will try to find some clue to verify later.

Quoted 11 lines
Quoted 7 lines
Quoted 4 lines

Moreover, we should not add the mount point to fspath on root inode for fs_config() branch.

Is there some descriptive words or reference for this? To be honest, I'm quite unsure about this kind of Android-specific things... :(

Refer to change: mksquashfs: Run android_fs_config() on the root inode

I think erofs of AOSP has this issue also. Am i right?

Not quite sure if it effects non-canned fs_config after reading the commit message...
https://android.googlesource.com/platform/external/squashfs-tools/+/85a6bc1e52bb911f195c5dc0890717913938c2d1%5E%21/#F0

And no permission to access Bug 72745016, so... maybe we need to limit this to non-canned fs_config only? (at least confirming the original case would be better)

Quoted 8 lines

BTW, Also, from its testcase command line in the commit message:
"mksquashfs path system.raw.img -fs-config-file fs_config -android-fs-config"

I'm not sure if "--mount-point" is passed in so I think for such case no need to use such "goto" as well?

Thanks,
Gao Xiang

Could you verify the following patch if possible? (without compilation, I don't have test environment now since AOSP code is on my PC)

From: Yue Hu <[email protected]>
Date: Tue, 22 Dec 2020 14:52:22 +0800
Subject: [PATCH] AOSP: erofs-utils: fix sub-directory prefix for canned
fs_config

"failed to find [%s] in canned fs_config" is observed by using "--fs-config-file" option under Android 10.

Notice that canned fs_config has a prefix to sub-directory if "--mount-point" presents. However, such prefix cannot be added by just using erofs_fspath().

Fixes 8a9e8046f170 ("AOSP: erofs-utils: add fs_config support")
---
 lib/inode.c | 18 +++++++++++-------
 1 file changed, 11 insertions(+), 7 deletions(-)
lib/inode.c +0 -0
index 3d634fc..9469074 100644

lib/inode.c int erofs_droid_inode_fsconfig(struct erofs_inode *inode, · line 705

Quoted 5 lines
lib/inode.c +2 -0
@@ -701,21 +701,25 @@ int erofs_droid_inode_fsconfig(struct erofs_inode *inode,
 	char *fspath;
 
 	inode->capabilities = 0;
+	if (!cfg.mount_point)
+		fspath = erofs_fspath(path);  

lib/inode.c:688:10: error: assigning to 'char *' from 'const char *' discards...

                fspath = erofs_fspath(path);
                       ^ ~~~~~~~~~~~~~~~~~~

Review

Quoted 2 lines
(quoted diff) +1 -1
-           fspath = erofs_fspath(path);
+         fspath = (char *)erofs_fspath(path);  

oops, I think it can be modified as a temporary workaround, will submit a formal patch after verification.

Quoted 8 lines

lib/inode.c int erofs_droid_inode_fsconfig(struct erofs_inode *inode, · line 707

Quoted 4 lines
lib/inode.c +2 -0
+	else if (asprintf(&fspath, "%s/%s", cfg.mount_point,
+			  erofs_fspath(path)) <= 0)  

The argument of path will be root directory. And canned fs_config for root directory as below:

Quoted 4 lines

0 0 755 selabel=u:object_r:rootfs:s0 capabilities=0x0

So, cannot add mount point to root directory for canned fs_config. And what about non-canned fs_config?

Not quite sure what you mean. For non-canned fs_config, we didn't observed any strange before (I ported to cuttlefish and hikey960 with boot success, also as I mentioned before some other vendors already use it.)

Quoted 6 lines

I think the following commit is only useful for squashfs since its (non)root inode workflows are different, so need to add in two difference place. But mkfs.erofs is not.
https://android.googlesource.com/platform/external/squashfs-tools/+/85a6bc1e52bb911f195c5dc0890717913938c2d1%5E%21/#F0

For root inode is erofs, I think erofs_fspath(path) would return "", so that case is included as well.... Am I missing something?

Yes, erofs_fspath(path) returns "" for root inode. However, the above patch add the mount point to fspath when specify it, so the real path is "vendor/" which does not exist in canned fs_config file. build will report below error:

failed to find [/vendor/] in canned fs_config

Quoted 35 lines

Thanks,
Gao Xiang

Quoted 30 lines

Thx.

Quoted 25 lines
lib/inode.c +7 -7
+		return -ENOMEM;
+
+
 	if (cfg.fs_config_file)
-		canned_fs_config(erofs_fspath(path),
+		canned_fs_config(fspath,
 				 S_ISDIR(st->st_mode),
 				 cfg.target_out_path,
 				 &uid, &gid, &mode, &inode->capabilities);
-	else if (cfg.mount_point) {
-		if (asprintf(&fspath, "%s/%s", cfg.mount_point,
-			     erofs_fspath(path)) <= 0)
-			return -ENOMEM;
-
+	else
 		fs_config(fspath, S_ISDIR(st->st_mode),
 			  cfg.target_out_path,
 			  &uid, &gid, &mode, &inode->capabilities);
+
+	if (cfg.mount_point)
 		free(fspath);
-	}
 	st->st_uid = uid;
 	st->st_gid = gid;
 	st->st_mode = mode | stat_file_type_mask;  
#10 lore raw

On Tue, Dec 22, 2020 at 05:30:14PM +0800, Yue Hu wrote:

...

Quoted 21 lines
Quoted 21 lines
Quoted 8 lines

Review

Quoted 4 lines
(quoted diff) +2 -0
+	else if (asprintf(&fspath, "%s/%s", cfg.mount_point,
+			  erofs_fspath(path)) <= 0)  

The argument of path will be root directory. And canned fs_config for root directory as below:

Quoted 4 lines

0 0 755 selabel=u:object_r:rootfs:s0 capabilities=0x0

So, cannot add mount point to root directory for canned fs_config. And what about non-canned fs_config?

Not quite sure what you mean. For non-canned fs_config, we didn't observed any strange before (I ported to cuttlefish and hikey960 with boot success, also as I mentioned before some other vendors already use it.)

I think the following commit is only useful for squashfs since its (non)root inode workflows are different, so need to add in two difference place. But mkfs.erofs is not.
https://android.googlesource.com/platform/external/squashfs-tools/+/85a6bc1e52bb911f195c5dc0890717913938c2d1%5E%21/#F0

Quoted 8 lines
Quoted 2 lines

For root inode is erofs, I think erofs_fspath(path) would return "", so that case is included as well.... Am I missing something?

Yes, erofs_fspath(path) returns "" for root inode. However, the above patch add the mount point to fspath when specify it, so the real path is "vendor/" which does not exist in canned fs_config file. build will report below error:

failed to find [/vendor/] in canned fs_config

Hmmm... such design is quite strange for me....
Could you try the following diff?

lib/inode.c +6 -3
index 9469074..9af6179 100644
@@ -698,11 +698,14 @@ int erofs_droid_inode_fsconfig(struct erofs_inode *inode,
 	/* filesystem_config does not preserve file type bits */
 	mode_t stat_file_type_mask = st->st_mode & S_IFMT;
 	unsigned int uid = 0, gid = 0, mode = 0;
+	bool alloced;
 	char *fspath;
 
 	inode->capabilities = 0;
-	if (!cfg.mount_point)
-		fspath = erofs_fspath(path);
+
+	alloced = (cfg.mount_point && erofs_fspath(path)[0] != '\0');
+	if (!alloced)
+		fspath = (char *)erofs_fspath(path);
 	else if (asprintf(&fspath, "%s/%s", cfg.mount_point,
 			  erofs_fspath(path)) <= 0)
 		return -ENOMEM;
@@ -718,7 +721,7 @@ int erofs_droid_inode_fsconfig(struct erofs_inode *inode,
 			  cfg.target_out_path,
 			  &uid, &gid, &mode, &inode->capabilities);
 
-	if (cfg.mount_point)
+	if (alloced)
 		free(fspath);
 	st->st_uid = uid;
 	st->st_gid = gid;

if it works, will redo a formal patch then....

Thanks,
Gao Xiang

Quoted 37 lines
Quoted 36 lines

Thanks,
Gao Xiang

Quoted 30 lines

Thx.

Quoted 25 lines
(quoted diff) +7 -7
+		return -ENOMEM;
+
+
 	if (cfg.fs_config_file)
-		canned_fs_config(erofs_fspath(path),
+		canned_fs_config(fspath,
 				 S_ISDIR(st->st_mode),
 				 cfg.target_out_path,
 				 &uid, &gid, &mode, &inode->capabilities);
-	else if (cfg.mount_point) {
-		if (asprintf(&fspath, "%s/%s", cfg.mount_point,
-			     erofs_fspath(path)) <= 0)
-			return -ENOMEM;
-
+	else
 		fs_config(fspath, S_ISDIR(st->st_mode),
 			  cfg.target_out_path,
 			  &uid, &gid, &mode, &inode->capabilities);
+
+	if (cfg.mount_point)
 		free(fspath);
-	}
 	st->st_uid = uid;
 	st->st_gid = gid;
 	st->st_mode = mode | stat_file_type_mask;  
author
#11 lore raw
Quoted 28 lines · On Tue, 22 Dec 2020 17:39:52 +0800 Gao Xiang <[email protected]> wrote:

On Tue, Dec 22, 2020 at 05:30:14PM +0800, Yue Hu wrote:

...

Quoted 24 lines
Quoted 23 lines
Quoted 8 lines

Review

Quoted 4 lines
(quoted diff) +2 -0
+	else if (asprintf(&fspath, "%s/%s", cfg.mount_point,
+			  erofs_fspath(path)) <= 0)    

The argument of path will be root directory. And canned fs_config for root directory as below:

Quoted 4 lines

0 0 755 selabel=u:object_r:rootfs:s0 capabilities=0x0

So, cannot add mount point to root directory for canned fs_config. And what about non-canned fs_config?

Not quite sure what you mean. For non-canned fs_config, we didn't observed any strange before (I ported to cuttlefish and hikey960 with boot success, also as I mentioned before some other vendors already use it.)

I think the following commit is only useful for squashfs since its (non)root inode workflows are different, so need to add in two difference place. But mkfs.erofs is not.
https://android.googlesource.com/platform/external/squashfs-tools/+/85a6bc1e52bb911f195c5dc0890717913938c2d1%5E%21/#F0

For root inode is erofs, I think erofs_fspath(path) would return "", so that case is included as well.... Am I missing something?

Quoted 7 lines
Quoted 5 lines

Yes, erofs_fspath(path) returns "" for root inode. However, the above patch add the mount point to fspath when specify it, so the real path is "vendor/" which does not exist in canned fs_config file. build will report below error:

failed to find [/vendor/] in canned fs_config

Hmmm... such design is quite strange for me....

:) i checked the squashfs before, seems root directory is handled in some position. Separated with sub directory fs_config. so i add the goto code in the 1st patch.

Quoted 1 line

Could you try the following diff?

Let's me verify.

Quoted 75 lines
lib/inode.c +6 -3
index 9469074..9af6179 100644
@@ -698,11 +698,14 @@ int erofs_droid_inode_fsconfig(struct erofs_inode *inode,
 	/* filesystem_config does not preserve file type bits */
 	mode_t stat_file_type_mask = st->st_mode & S_IFMT;
 	unsigned int uid = 0, gid = 0, mode = 0;
+	bool alloced;
 	char *fspath;
 
 	inode->capabilities = 0;
-	if (!cfg.mount_point)
-		fspath = erofs_fspath(path);
+
+	alloced = (cfg.mount_point && erofs_fspath(path)[0] != '\0');
+	if (!alloced)
+		fspath = (char *)erofs_fspath(path);
 	else if (asprintf(&fspath, "%s/%s", cfg.mount_point,
 			  erofs_fspath(path)) <= 0)
 		return -ENOMEM;
@@ -718,7 +721,7 @@ int erofs_droid_inode_fsconfig(struct erofs_inode *inode,
 			  cfg.target_out_path,
 			  &uid, &gid, &mode, &inode->capabilities);
 
-	if (cfg.mount_point)
+	if (alloced)
 		free(fspath);
 	st->st_uid = uid;
 	st->st_gid = gid;

if it works, will redo a formal patch then....

Thanks,
Gao Xiang

Quoted 37 lines
Quoted 36 lines

Thanks,
Gao Xiang

Quoted 30 lines

Thx.

Quoted 25 lines
(quoted diff) +7 -7
+		return -ENOMEM;
+
+
 	if (cfg.fs_config_file)
-		canned_fs_config(erofs_fspath(path),
+		canned_fs_config(fspath,
 				 S_ISDIR(st->st_mode),
 				 cfg.target_out_path,
 				 &uid, &gid, &mode, &inode->capabilities);
-	else if (cfg.mount_point) {
-		if (asprintf(&fspath, "%s/%s", cfg.mount_point,
-			     erofs_fspath(path)) <= 0)
-			return -ENOMEM;
-
+	else
 		fs_config(fspath, S_ISDIR(st->st_mode),
 			  cfg.target_out_path,
 			  &uid, &gid, &mode, &inode->capabilities);
+
+	if (cfg.mount_point)
 		free(fspath);
-	}
 	st->st_uid = uid;
 	st->st_gid = gid;
 	st->st_mode = mode | stat_file_type_mask;    
#12 lore raw
Quoted 35 lines · Yue Hu
Quoted 32 lines · On Tue, 22 Dec 2020 17:39:52 +0800 Gao Xiang <[email protected]> wrote:

On Tue, Dec 22, 2020 at 05:30:14PM +0800, Yue Hu wrote:

...

Quoted 28 lines
Quoted 17 lines
Quoted 8 lines

Review

Quoted 4 lines
(quoted diff) +2 -0
+	else if (asprintf(&fspath, "%s/%s", cfg.mount_point,
+			  erofs_fspath(path)) <= 0)    

The argument of path will be root directory. And canned fs_config for root directory as below:

Quoted 4 lines

0 0 755 selabel=u:object_r:rootfs:s0 capabilities=0x0

So, cannot add mount point to root directory for canned fs_config. And what about non-canned fs_config?

Not quite sure what you mean. For non-canned fs_config, we didn't observed any strange before (I ported to cuttlefish and hikey960 with boot success, also as I mentioned before some other vendors already use it.)

Quoted 6 lines

I think the following commit is only useful for squashfs since its (non)root inode workflows are different, so need to add in two difference place. But mkfs.erofs is not.
https://android.googlesource.com/platform/external/squashfs-tools/+/85a6bc1e52bb911f195c5dc0890717913938c2d1%5E%21/#F0

For root inode is erofs, I think erofs_fspath(path) would return "", so that case is included as well.... Am I missing something?

Yes, erofs_fspath(path) returns "" for root inode. However, the above patch add the mount point to fspath when specify it, so the real path is "vendor/" which does not exist in canned fs_config file. build will report below error:

Quoted 6 lines
Quoted 3 lines
Quoted 1 line

failed to find [/vendor/] in canned fs_config

Hmmm... such design is quite strange for me....

:) i checked the squashfs before, seems root directory is handled in some position. Separated with sub directory fs_config. so i add the goto code in the 1st patch.

What confuses me a lot is that we didn't get any strange without canned fs_config if mount point prefix is added. I will look into other implementation about this later (Another guess is that relates to Android 10 only?).

Yeah, the opensource fs_config implementation might be different from HUAWEI internal fs_config version since such part was not originally written by me and I didn't pay more attention about this part when I was in my previous company. But anyway, this cleanup opensource version is already used for some vendors as well and I don't get such report... And any formal description about this would be helpful for me to understand how fs_config really works..

Thanks,
Gao Xiang

Quoted 5 lines
Quoted 2 lines

Could you try the following diff?

Let's me verify.

author
#13 lore raw
Quoted 38 lines · On Tue, 22 Dec 2020 17:59:06 +0800 Gao Xiang <[email protected]> wrote:
Quoted 37 lines · Yue Hu
Quoted 34 lines · On Tue, 22 Dec 2020 17:39:52 +0800 Gao Xiang <[email protected]> wrote:

On Tue, Dec 22, 2020 at 05:30:14PM +0800, Yue Hu wrote:

...

Quoted 29 lines
Quoted 17 lines
Quoted 8 lines

Review

Quoted 4 lines
(quoted diff) +2 -0
+	else if (asprintf(&fspath, "%s/%s", cfg.mount_point,
+			  erofs_fspath(path)) <= 0)      

The argument of path will be root directory. And canned fs_config for root directory as below:

Quoted 4 lines

0 0 755 selabel=u:object_r:rootfs:s0 capabilities=0x0

So, cannot add mount point to root directory for canned fs_config. And what about non-canned fs_config?

Not quite sure what you mean. For non-canned fs_config, we didn't observed any strange before (I ported to cuttlefish and hikey960 with boot success, also as I mentioned before some other vendors already use it.)

Quoted 6 lines

I think the following commit is only useful for squashfs since its (non)root inode workflows are different, so need to add in two difference place. But mkfs.erofs is not.
https://android.googlesource.com/platform/external/squashfs-tools/+/85a6bc1e52bb911f195c5dc0890717913938c2d1%5E%21/#F0

For root inode is erofs, I think erofs_fspath(path) would return "", so that case is included as well.... Am I missing something?

Yes, erofs_fspath(path) returns "" for root inode. However, the above patch add the mount point to fspath when specify it, so the real path is "vendor/" which does not exist in canned fs_config file. build will report below error:

failed to find [/vendor/] in canned fs_config

Quoted 8 lines
Quoted 4 lines
Quoted 1 line

Hmmm... such design is quite strange for me....

:) i checked the squashfs before, seems root directory is handled in some position. Separated with sub directory fs_config. so i add the goto code in the 1st patch.

What confuses me a lot is that we didn't get any strange without canned fs_config if mount point prefix is added. I will look into other implementation about this later (Another guess is that relates to Android 10 only?).

maybe relates to dynamic partition(intro from Android 10) which not be enabled by some vendors.

Quoted 7 lines

Yeah, the opensource fs_config implementation might be different from HUAWEI internal fs_config version since such part was not originally written by me and I didn't pay more attention about this part when I was in my previous company. But anyway, this cleanup opensource version is already used for some vendors as well and I don't get such report... And any formal description about this would be helpful for me to understand how fs_config really works..

Now i'm not familar with fs_config also :) I will continue to check when i have enough time.

Anyway, i observed the issue in canned fs_config since i'm using it. so i decide to report it and patch it to upstream to verify if it's a real one.

Thx.

Quoted 10 lines

Thanks,
Gao Xiang

Quoted 5 lines
Quoted 2 lines

Could you try the following diff?

Let's me verify.

#14 lore raw
Quoted 43 lines · Yue Hu
Quoted 40 lines · On Tue, 22 Dec 2020 17:59:06 +0800 Gao Xiang <[email protected]> wrote:
Quoted 39 lines · Yue Hu
Quoted 35 lines · On Tue, 22 Dec 2020 17:39:52 +0800 Gao Xiang <[email protected]> wrote:

On Tue, Dec 22, 2020 at 05:30:14PM +0800, Yue Hu wrote:

...

Quoted 21 lines
Quoted 21 lines
Quoted 8 lines

Review

Quoted 4 lines
(quoted diff) +2 -0
+	else if (asprintf(&fspath, "%s/%s", cfg.mount_point,
+			  erofs_fspath(path)) <= 0)      

The argument of path will be root directory. And canned fs_config for root directory as below:

Quoted 4 lines

0 0 755 selabel=u:object_r:rootfs:s0 capabilities=0x0

So, cannot add mount point to root directory for canned fs_config. And what about non-canned fs_config?

Not quite sure what you mean. For non-canned fs_config, we didn't observed any strange before (I ported to cuttlefish and hikey960 with boot success, also as I mentioned before some other vendors already use it.)

I think the following commit is only useful for squashfs since its (non)root inode workflows are different, so need to add in two difference place. But mkfs.erofs is not.
https://android.googlesource.com/platform/external/squashfs-tools/+/85a6bc1e52bb911f195c5dc0890717913938c2d1%5E%21/#F0

Quoted 8 lines
Quoted 2 lines

For root inode is erofs, I think erofs_fspath(path) would return "", so that case is included as well.... Am I missing something?

Yes, erofs_fspath(path) returns "" for root inode. However, the above patch add the mount point to fspath when specify it, so the real path is "vendor/" which does not exist in canned fs_config file. build will report below error:

failed to find [/vendor/] in canned fs_config

Hmmm... such design is quite strange for me....

Quoted 8 lines
Quoted 6 lines
Quoted 2 lines

:) i checked the squashfs before, seems root directory is handled in some position. Separated with sub directory fs_config. so i add the goto code in the 1st patch.

What confuses me a lot is that we didn't get any strange without canned fs_config if mount point prefix is added. I will look into other implementation about this later (Another guess is that relates to Android 10 only?).

maybe relates to dynamic partition(intro from Android 10) which not be enabled by some vendors.

I think some of them use dynamic partition AFAIK, but might not be with QSSI enabled (I'm not sure, anyway, that is minor...)

Quoted 9 lines
Quoted 8 lines

Yeah, the opensource fs_config implementation might be different from HUAWEI internal fs_config version since such part was not originally written by me and I didn't pay more attention about this part when I was in my previous company. But anyway, this cleanup opensource version is already used for some vendors as well and I don't get such report... And any formal description about this would be helpful for me to understand how fs_config really works..

Quoted 5 lines

Now i'm not familar with fs_config also :) I will continue to check when i have enough time.

Anyway, i observed the issue in canned fs_config since i'm using it. so i decide to report it and patch it to upstream to verify if it's a real one.

Yeah, that is somewhat bad and needs fixing if canned fs_config doesn't work as expected...
My confusion for now is that how to deal with root dir properly (it seems make_ext4fs doesn't even care about rootdir fs_config at all if my understanding is correct.)

Also,
https://android.googlesource.com/platform/system/core/+/master/libcutils/fs_config.cpp
https://android.googlesource.com/platform/system/core/+/master/libcutils/canned_fs_config.cpp

are implemented quite different. So look forward to your test result (I tend to add prefix for fs_config, but drop prefix for canned_fs_config instead.)

Thanks,
Gao Xiang

Quoted 14 lines

Thx.

Quoted 10 lines

Thanks,
Gao Xiang

Quoted 5 lines
Quoted 2 lines

Could you try the following diff?

Let's me verify.

author
#15 lore raw
Quoted 77 lines · On Tue, 22 Dec 2020 18:33:20 +0800 Gao Xiang <[email protected]> wrote:
Quoted 43 lines · Yue Hu
Quoted 40 lines · On Tue, 22 Dec 2020 17:59:06 +0800 Gao Xiang <[email protected]> wrote:
Quoted 39 lines · Yue Hu
Quoted 35 lines · On Tue, 22 Dec 2020 17:39:52 +0800 Gao Xiang <[email protected]> wrote:

On Tue, Dec 22, 2020 at 05:30:14PM +0800, Yue Hu wrote:

...

Quoted 21 lines
Quoted 21 lines
Quoted 8 lines

Review

Quoted 4 lines
(quoted diff) +2 -0
+	else if (asprintf(&fspath, "%s/%s", cfg.mount_point,
+			  erofs_fspath(path)) <= 0)        

The argument of path will be root directory. And canned fs_config for root directory as below:

Quoted 4 lines

0 0 755 selabel=u:object_r:rootfs:s0 capabilities=0x0

So, cannot add mount point to root directory for canned fs_config. And what about non-canned fs_config?

Not quite sure what you mean. For non-canned fs_config, we didn't observed any strange before (I ported to cuttlefish and hikey960 with boot success, also as I mentioned before some other vendors already use it.)

I think the following commit is only useful for squashfs since its (non)root inode workflows are different, so need to add in two difference place. But mkfs.erofs is not.
https://android.googlesource.com/platform/external/squashfs-tools/+/85a6bc1e52bb911f195c5dc0890717913938c2d1%5E%21/#F0

Quoted 8 lines
Quoted 2 lines

For root inode is erofs, I think erofs_fspath(path) would return "", so that case is included as well.... Am I missing something?

Yes, erofs_fspath(path) returns "" for root inode. However, the above patch add the mount point to fspath when specify it, so the real path is "vendor/" which does not exist in canned fs_config file. build will report below error:

failed to find [/vendor/] in canned fs_config

Hmmm... such design is quite strange for me....

Quoted 8 lines
Quoted 6 lines
Quoted 2 lines

:) i checked the squashfs before, seems root directory is handled in some position. Separated with sub directory fs_config. so i add the goto code in the 1st patch.

What confuses me a lot is that we didn't get any strange without canned fs_config if mount point prefix is added. I will look into other implementation about this later (Another guess is that relates to Android 10 only?).

maybe relates to dynamic partition(intro from Android 10) which not be enabled by some vendors.

I think some of them use dynamic partition AFAIK, but might not be with QSSI enabled (I'm not sure, anyway, that is minor...)

Quoted 9 lines
Quoted 8 lines

Yeah, the opensource fs_config implementation might be different from HUAWEI internal fs_config version since such part was not originally written by me and I didn't pay more attention about this part when I was in my previous company. But anyway, this cleanup opensource version is already used for some vendors as well and I don't get such report... And any formal description about this would be helpful for me to understand how fs_config really works..

Quoted 5 lines

Now i'm not familar with fs_config also :) I will continue to check when i have enough time.

Anyway, i observed the issue in canned fs_config since i'm using it. so i decide to report it and patch it to upstream to verify if it's a real one.

Yeah, that is somewhat bad and needs fixing if canned fs_config doesn't work as expected...
My confusion for now is that how to deal with root dir properly (it seems make_ext4fs doesn't even care about rootdir fs_config at all if my understanding is correct.)

Quoted 6 lines

Also,
https://android.googlesource.com/platform/system/core/+/master/libcutils/fs_config.cpp
https://android.googlesource.com/platform/system/core/+/master/libcutils/canned_fs_config.cpp

are implemented quite different. So look forward to your test result (I tend to add prefix for fs_config, but drop prefix for canned_fs_config instead.)

It works for canned fs_config i'm using. We can simplify the test enviroment. canned fs_config file content/format (e.g. mount point is vendor) as below:

0 2000 755 selabel=u:object_r:rootfs:s0 capabilities=0x0
vendor/app 0 2000 755 selabel=u:object_r:vendor_app_file:s0 capabilities=0x0 vendor/app/CACertService 0 2000 755 selabel=u:object_r:vendor_app_file:s0 capabilities=0x0 vendor/app/CACertService/CACertService.apk 0 0 644 selabel=u:object_r:vendor_app_file:s0 capabilities=0x0 vendor/app/CACertService/oat 0 2000 755 selabel=u:object_r:vendor_app_file:s0 capabilities=0x0 vendor/app/CACertService/oat/arm64 0 2000 755 selabel=u:object_r:vendor_app_file:s0 capabilities=0x0

The 1st line is for root inode search and the others are for sub inode like search.

Quoted 19 lines

Thanks,
Gao Xiang

Quoted 14 lines

Thx.

Quoted 10 lines

Thanks,
Gao Xiang

Quoted 5 lines
Quoted 2 lines

Could you try the following diff?

Let's me verify.

author
#16 lore raw
In reply to Gao Xiang
Quoted 68 lines · On Tue, 22 Dec 2020 17:39:52 +0800 Gao Xiang <[email protected]> wrote:

On Tue, Dec 22, 2020 at 05:30:14PM +0800, Yue Hu wrote:

...

Quoted 21 lines
Quoted 21 lines
Quoted 8 lines

Review

Quoted 4 lines
(quoted diff) +2 -0
+	else if (asprintf(&fspath, "%s/%s", cfg.mount_point,
+			  erofs_fspath(path)) <= 0)    

The argument of path will be root directory. And canned fs_config for root directory as below:

Quoted 4 lines

0 0 755 selabel=u:object_r:rootfs:s0 capabilities=0x0

So, cannot add mount point to root directory for canned fs_config. And what about non-canned fs_config?

Not quite sure what you mean. For non-canned fs_config, we didn't observed any strange before (I ported to cuttlefish and hikey960 with boot success, also as I mentioned before some other vendors already use it.)

I think the following commit is only useful for squashfs since its (non)root inode workflows are different, so need to add in two difference place. But mkfs.erofs is not.
https://android.googlesource.com/platform/external/squashfs-tools/+/85a6bc1e52bb911f195c5dc0890717913938c2d1%5E%21/#F0

Quoted 8 lines
Quoted 2 lines

For root inode is erofs, I think erofs_fspath(path) would return "", so that case is included as well.... Am I missing something?

Yes, erofs_fspath(path) returns "" for root inode. However, the above patch add the mount point to fspath when specify it, so the real path is "vendor/" which does not exist in canned fs_config file. build will report below error:

failed to find [/vendor/] in canned fs_config

Hmmm... such design is quite strange for me....
Could you try the following diff?

lib/inode.c +6 -3
index 9469074..9af6179 100644
@@ -698,11 +698,14 @@ int erofs_droid_inode_fsconfig(struct erofs_inode *inode,
 	/* filesystem_config does not preserve file type bits */
 	mode_t stat_file_type_mask = st->st_mode & S_IFMT;
 	unsigned int uid = 0, gid = 0, mode = 0;
+	bool alloced;
 	char *fspath;
 
 	inode->capabilities = 0;
-	if (!cfg.mount_point)
-		fspath = erofs_fspath(path);
+
+	alloced = (cfg.mount_point && erofs_fspath(path)[0] != '\0');
+	if (!alloced)
+		fspath = (char *)erofs_fspath(path);
 	else if (asprintf(&fspath, "%s/%s", cfg.mount_point,
 			  erofs_fspath(path)) <= 0)
 		return -ENOMEM;
@@ -718,7 +721,7 @@ int erofs_droid_inode_fsconfig(struct erofs_inode *inode,
 			  cfg.target_out_path,
 			  &uid, &gid, &mode, &inode->capabilities);
 
-	if (cfg.mount_point)
+	if (alloced)
 		free(fspath);
 	st->st_uid = uid;
 	st->st_gid = gid;
Quoted 1 line

if it works, will redo a formal patch then....

Works for me for canned fs_config.

Thx.

Quoted 42 lines

Thanks,
Gao Xiang

Quoted 37 lines
Quoted 36 lines

Thanks,
Gao Xiang

Quoted 30 lines

Thx.

Quoted 25 lines
(quoted diff) +7 -7
+		return -ENOMEM;
+
+
 	if (cfg.fs_config_file)
-		canned_fs_config(erofs_fspath(path),
+		canned_fs_config(fspath,
 				 S_ISDIR(st->st_mode),
 				 cfg.target_out_path,
 				 &uid, &gid, &mode, &inode->capabilities);
-	else if (cfg.mount_point) {
-		if (asprintf(&fspath, "%s/%s", cfg.mount_point,
-			     erofs_fspath(path)) <= 0)
-			return -ENOMEM;
-
+	else
 		fs_config(fspath, S_ISDIR(st->st_mode),
 			  cfg.target_out_path,
 			  &uid, &gid, &mode, &inode->capabilities);
+
+	if (cfg.mount_point)
 		free(fspath);
-	}
 	st->st_uid = uid;
 	st->st_gid = gid;
 	st->st_mode = mode | stat_file_type_mask;    
#17 lore raw

On Tue, Dec 22, 2020 at 06:34:11PM +0800, Yue Hu wrote:

...

Quoted 35 lines
Quoted 35 lines

Hmmm... such design is quite strange for me....
Could you try the following diff?

lib/inode.c +6 -3
index 9469074..9af6179 100644
@@ -698,11 +698,14 @@ int erofs_droid_inode_fsconfig(struct erofs_inode *inode,
 	/* filesystem_config does not preserve file type bits */
 	mode_t stat_file_type_mask = st->st_mode & S_IFMT;
 	unsigned int uid = 0, gid = 0, mode = 0;
+	bool alloced;
 	char *fspath;
 
 	inode->capabilities = 0;
-	if (!cfg.mount_point)
-		fspath = erofs_fspath(path);
+
+	alloced = (cfg.mount_point && erofs_fspath(path)[0] != '\0');
+	if (!alloced)
+		fspath = (char *)erofs_fspath(path);
 	else if (asprintf(&fspath, "%s/%s", cfg.mount_point,
 			  erofs_fspath(path)) <= 0)
 		return -ENOMEM;
@@ -718,7 +721,7 @@ int erofs_droid_inode_fsconfig(struct erofs_inode *inode,
 			  cfg.target_out_path,
 			  &uid, &gid, &mode, &inode->capabilities);
 
-	if (cfg.mount_point)
+	if (alloced)
 		free(fspath);
 	st->st_uid = uid;
 	st->st_gid = gid;
Quoted 3 lines
Quoted 1 line

if it works, will redo a formal patch then....

Works for me for canned fs_config.

Ok, if it also works fine for non-canned fs_config on your side, I will redo a formal patch later... sigh :(

Thanks,
Gao Xiang

Quoted 2 lines

Thx.

author
#18 lore raw
Quoted 39 lines · On Tue, 22 Dec 2020 18:47:00 +0800 Gao Xiang <[email protected]> wrote:

On Tue, Dec 22, 2020 at 06:34:11PM +0800, Yue Hu wrote:

...

Quoted 35 lines
Quoted 35 lines

Hmmm... such design is quite strange for me....
Could you try the following diff?

lib/inode.c +6 -3
index 9469074..9af6179 100644
@@ -698,11 +698,14 @@ int erofs_droid_inode_fsconfig(struct erofs_inode *inode,
 	/* filesystem_config does not preserve file type bits */
 	mode_t stat_file_type_mask = st->st_mode & S_IFMT;
 	unsigned int uid = 0, gid = 0, mode = 0;
+	bool alloced;
 	char *fspath;
 
 	inode->capabilities = 0;
-	if (!cfg.mount_point)
-		fspath = erofs_fspath(path);
+
+	alloced = (cfg.mount_point && erofs_fspath(path)[0] != '\0');
+	if (!alloced)
+		fspath = (char *)erofs_fspath(path);
 	else if (asprintf(&fspath, "%s/%s", cfg.mount_point,
 			  erofs_fspath(path)) <= 0)
 		return -ENOMEM;
@@ -718,7 +721,7 @@ int erofs_droid_inode_fsconfig(struct erofs_inode *inode,
 			  cfg.target_out_path,
 			  &uid, &gid, &mode, &inode->capabilities);
 
-	if (cfg.mount_point)
+	if (alloced)
 		free(fspath);
 	st->st_uid = uid;
 	st->st_gid = gid;
Quoted 6 lines
Quoted 3 lines
Quoted 1 line

if it works, will redo a formal patch then....

Works for me for canned fs_config.

Ok, if it also works fine for non-canned fs_config on your side, I will redo a formal patch later... sigh :(

Hi Xiang,

I just remove the "--fs-config-file" in my test enviroment, after that, build and boot are all working fine.

If canned fs_config never used by others before, i think it's a bug.

Thx.

Quoted 7 lines

Thanks,
Gao Xiang

Quoted 2 lines

Thx.

#19 lore raw

Hi Yue,

Quoted 3 lines · Yue Hu

On Tue, 22 Dec 2020 18:47:00 +0800
Gao Xiang <[email protected]> wrote:

...

Quoted 11 lines
Quoted 4 lines
Quoted 1 line

Works for me for canned fs_config.

Ok, if it also works fine for non-canned fs_config on your side, I will redo a formal patch later... sigh :(

Hi Xiang,

I just remove the "--fs-config-file" in my test enviroment, after that, build and boot are all working fine.

If canned fs_config never used by others before, i think it's a bug.

Thanks for your report. I agree with your conclusion. Let me seek some extra free time later to form a formal patch and test on my PC then.

I'll finish this this week, don't worry :)

Thanks,
Gao Xiang

Quoted 11 lines

Thx.

Quoted 7 lines

Thanks,
Gao Xiang

Quoted 2 lines

Thx.