Skip to contentSkip to conversation

env: ext4: corrections and add test for env in ext4

PATCH Superseded Patrick Delaunay 4 participants 27 messages Jun 16, 2020 – Jun 24, 2020

v2 of 4 Superseded Latest is v4 →

lore
Share & tools
lore
#0 lore raw

v2 cover env: ext4: corrections and add test for env in ext4

Hi,

V2 is only a rebase and adaptation of the serie [1].

In this serie, I add sandbox test with CONFIG_ENV_IS_NOWHERE activated with other location: at least one CONFIG_ENV_IS_IN_... is defined and ENV_IS_IN_DEVICE is automatically defined.

To test this feature, I activate and test ENV_IS_IN_EXT4 in sandbox; I add a new command "env_loc" to change this ENV location.

This serie depends on previous env test introduced in [2] "cmd: env: add option for quiet output on env info"

To be able to test invalid file (bad CRC), I also add the support of the command "env erase" for EXT4 env location.

[1] http://patchwork.ozlabs.org/project/uboot/list/?series=158160
[2] http://patchwork.ozlabs.org/project/uboot/list/?series=183438

Regards

Patrick

Changes in v2:
- change cmd_tbl_t to struct cmd_tbl
- use CONFIG_IS_ENABLED to set .erase (same as .save)

Patrick Delaunay (9):
env: add absolute path at CONFIG_ENV_EXT4_FILE
env: ext4: set gd->env_valid
env: correctly handle result in env_init
sandbox: activate env in ext4 support
sandbox: support the change of env location
test: environment in ext4
env: ext4: introduce new function env_ext4_save_buffer
env: ext4: add support of command env erase
test: sandbox: add test for erase command

 board/sandbox/sandbox.c            |  52 +++++++++++++++
 configs/sandbox64_defconfig        |   5 ++
 configs/sandbox_defconfig          |   5 ++
 configs/sandbox_flattree_defconfig |   5 ++
 configs/sandbox_spl_defconfig      |   5 ++
 env/Kconfig                        |   2 +-
 env/env.c                          |   5 +-
 env/ext4.c                         |  54 ++++++++++++---
 test/py/tests/test_env.py          | 103 +++++++++++++++++++++++++++++
 9 files changed, 226 insertions(+), 10 deletions(-)
Signature · 1 line
2.17.1
#1 lore raw

PATCH v2 1/9 env: add absolute path at CONFIG_ENV_EXT4_FILE

Add the absolute path to the default value of
CONFIG_ENV_EXT4_FILE = "/uboot.env".

This patch avoid the error :
Saving Environment to EXT4... File System is consistent Please supply Absolute path

---

For information, it is the value used today by all the boards:

dragonboard820c_defconfig:29:CONFIG_ENV_EXT4_FILE="/uboot.env" hikey960_defconfig:25:CONFIG_ENV_EXT4_FILE="/uboot.env" stm32mp15_basic_defconfig:64:CONFIG_ENV_EXT4_FILE="/uboot.env" stm32mp15_trusted_defconfig:50:CONFIG_ENV_EXT4_FILE="/uboot.env"

(no changes since v1)

 env/Kconfig | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
env/Kconfig +1 -1
index ca7fef682b..9dad1cf8c1 100644
@@ -469,7 +469,7 @@ config ENV_EXT4_DEVICE_AND_PART
 config ENV_EXT4_FILE
 	string "Name of the EXT4 file to use for the environment"
 	depends on ENV_IS_IN_EXT4
-	default "uboot.env"
+	default "/uboot.env"
 	help
 	  It's a string of the EXT4 file name. This file use to store the
 	  environment (explicit path to the file)
Signature · 1 line
2.17.1
#2 lore raw

PATCH v2 1/9 env: add absolute path at CONFIG_ENV_EXT4_FILE

Quoted 8 lines · Patrick Delaunay

Add the absolute path to the default value of
CONFIG_ENV_EXT4_FILE = "/uboot.env".

This patch avoid the error :
Saving Environment to EXT4... File System is consistent Please supply Absolute path

Signature · 8 lines
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 659 bytes
Desc: not available
URL: <https://lists.denx.de/pipermail/u-boot/attachments/20200618/49f28b4f/attachment.sig>
#3 lore raw

PATCH v2 2/9 env: ext4: set gd->env_valid

In reply to Patrick Delaunay

Add a missing initialization of gd->env_valid in env_ext4_load as it is already done in some other env device.

Set gd->env_valid = ENV_VALID in env_ext4_save() and env_ext4_load().

This patch allows to have a correct information in 'env info' command.

---

(no changes since v1)

 env/ext4.c | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)
env/ext4.c +8 -1
index 8e90bb71b7..ac9f126bec 100644
@@ -32,6 +32,8 @@
 #include <ext4fs.h>
 #include <mmc.h>
 
+DECLARE_GLOBAL_DATA_PTR;
+
 __weak const char *env_ext4_get_intf(void)
 {
 	return (const char *)CONFIG_ENV_EXT4_INTERFACE;
@@ -79,6 +81,7 @@ static int env_ext4_save(void)
 			CONFIG_ENV_EXT4_FILE, ifname, dev, part);
 		return 1;
 	}
+	gd->env_valid = ENV_VALID;
 
 	puts("done\n");
 	return 0;
@@ -124,7 +127,11 @@ static int env_ext4_load(void)
 		goto err_env_relocate;
 	}
 
-	return env_import(buf, 1);
+	err = env_import(buf, 1);
+	if (!err)
+		gd->env_valid = ENV_VALID;
+
+	return err;
 
 err_env_relocate:
 	env_set_default(NULL, 0);
Signature · 1 line
2.17.1
#4 lore raw

PATCH v2 2/9 env: ext4: set gd->env_valid

Quoted 8 lines · Patrick Delaunay

Add a missing initialization of gd->env_valid in env_ext4_load as it is already done in some other env device.

Set gd->env_valid = ENV_VALID in env_ext4_save() and env_ext4_load().

This patch allows to have a correct information in 'env info' command.

Signature · 8 lines
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 659 bytes
Desc: not available
URL: <https://lists.denx.de/pipermail/u-boot/attachments/20200618/827490d9/attachment.sig>
#5 lore raw

PATCH v2 3/9 env: correctly handle result in env_init

In reply to Patrick Delaunay

Don't return error with ret=-ENOENT when the optional ops drv->init is absent but only if env_driver_lookup doesn't found driver.

This patch correct an issue for the code
if (!env_init())

     env_load()

When only ext4 is supported (CONFIG_ENV_IS_IN_EXT4), as the backend env/ext4.c doesn't define an ops .init

---

(no changes since v1)

 env/env.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)
env/env.c +4 -1
index dcc25c030b..819c88f729 100644
@@ -295,7 +295,10 @@ int env_init(void)
 	int prio;
 
 	for (prio = 0; (drv = env_driver_lookup(ENVOP_INIT, prio)); prio++) {
-		if (!drv->init || !(ret = drv->init()))
+		ret = 0;
+		if (drv->init)
+			ret = drv->init();
+		if (!ret)
 			env_set_inited(drv->location);
 
 		debug("%s: Environment %s init done (ret=%d)\n", __func__,
Signature · 1 line
2.17.1
#6 lore raw

PATCH v2 3/9 env: correctly handle result in env_init

Quoted 23 lines · Patrick Delaunay

Don't return error with ret=-ENOENT when the optional ops drv->init is absent but only if env_driver_lookup doesn't found driver.

This patch correct an issue for the code
if (!env_init())

     env_load()

When only ext4 is supported (CONFIG_ENV_IS_IN_EXT4), as the backend env/ext4.c doesn't define an ops .init

---

(no changes since v1)

 env/env.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)
env/env.c +0 -0
index dcc25c030b..819c88f729 100644
@@ -295,7 +295,10 @@ int env_init(void)
 	int prio;

env/env.c int env_init(void) · line 304

Quoted 10 lines
env/env.c +4 -1
 
 	for (prio = 0; (drv = env_driver_lookup(ENVOP_INIT, prio)); prio++) {
-		if (!drv->init || !(ret = drv->init()))
+		ret = 0;
+		if (drv->init)
+			ret = drv->init();
+		if (!ret)
 			env_set_inited(drv->location);
 
 		debug("%s: Environment %s init done (ret=%d)\n", __func__,

I'm adding in Marek here because this reminds me of similar questions / concerns I had looking in to his series. At root, I think we're not being consistent in each of our env backing implementations about where flags such as ENV_VALID are set, and return values / checks of functions.

Just outside of the start of the patch context here, we set ret to -ENOENT and just past this, if still -ENOENT we say ENV_VALID and point at the default environment.

But, I don't follow the patch commit message here. If we don't have drv->init we call env_set_inited(drv->location) but we won't have change ret to 0, which means that later on down the function we go back to default environment.

So isn't this a problem in most environment cases then? Thanks!

Signature · 8 lines
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 659 bytes
Desc: not available
URL: <https://lists.denx.de/pipermail/u-boot/attachments/20200618/e4e8eac0/attachment.sig>
#7 lore raw

PATCH v2 3/9 env: correctly handle result in env_init

Hi Tom and Marek,

Quoted 44 lines

From: Tom Rini <[email protected]>
Sent: jeudi 18 juin 2020 21:16

Quoted 23 lines · Patrick Delaunay

Don't return error with ret=-ENOENT when the optional ops drv->init is absent but only if env_driver_lookup doesn't found driver.

This patch correct an issue for the code
if (!env_init())

     env_load()

When only ext4 is supported (CONFIG_ENV_IS_IN_EXT4), as the backend env/ext4.c doesn't define an ops .init

---

(no changes since v1)

 env/env.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)
env/env.c +0 -0
index dcc25c030b..819c88f729 100644
@@ -295,7 +295,10 @@ int env_init(void)
 	int prio;

env/env.c int env_init(void) · line 304

Quoted 10 lines
env/env.c +4 -1
 
 	for (prio = 0; (drv = env_driver_lookup(ENVOP_INIT, prio)); prio++) {
-		if (!drv->init || !(ret = drv->init()))
+		ret = 0;
+		if (drv->init)
+			ret = drv->init();
+		if (!ret)
 			env_set_inited(drv->location);
 
 		debug("%s: Environment %s init done (ret=%d)\n", __func__,

I'm adding in Marek here because this reminds me of similar questions / concerns I had looking in to his series. At root, I think we're not being consistent in each of our env backing implementations about where flags such as ENV_VALID are set, and return values / checks of functions.

Quoted 8 lines

Just outside of the start of the patch context here, we set ret to -ENOENT and just past this, if still -ENOENT we say ENV_VALID and point at the default environment.

But, I don't follow the patch commit message here. If we don't have drv->init we call env_set_inited(drv->location) but we won't have change ret to 0, which means that later on down the function we go back to default environment.

The cause of the issue is because the init() ops is optional in "struct env_driver".

When this opt is absent, I assume that the initialization is not mandatory but this inititialization need to be tagged in gd->env_has_init with the call of env_set_inited() function

And the ENV backend is FOUND (don't return -ENOENT)

else the next call of env_has_inited(drv->location) always failed : in env_load()

I see the error in EXT4 env backend,.all the other backend as a env_init() function

But some othe backend don't define the .init operation and have the issue

eeprom.c:235:U_BOOT_ENV_LOCATION(eeprom) = {
ext4.c:135:U_BOOT_ENV_LOCATION(ext4) = {
fat.c:128:U_BOOT_ENV_LOCATION(fat) = {
mmc.c:393:U_BOOT_ENV_LOCATION(mmc) = {
onenand.c:108:U_BOOT_ENV_LOCATION(onenand) = {
sata.c:117:U_BOOT_ENV_LOCATION(sata) = {
ubi.c:179:U_BOOT_ENV_LOCATION(ubi) = {

The other don't have issue:

flash.c:358:U_BOOT_ENV_LOCATION(flash) = {
flash.c:368: .init = env_flash_init,
nand.c:382:U_BOOT_ENV_LOCATION(nand) = {
nand.c:389: .init = env_nand_init,
nowhere.c:30:U_BOOT_ENV_LOCATION(nowhere) = {
nowhere.c:32: .init = env_nowhere_init,
nvram.c:117:U_BOOT_ENV_LOCATION(nvram) = {
nvram.c:122: .init = env_nvram_init,
remote.c:54:U_BOOT_ENV_LOCATION(remote) = {
remote.c:59: .init = env_remote_init,
sf.c:306:U_BOOT_ENV_LOCATION(sf) = {
sf.c:312: .init = env_sf_init,

Quoted 1 line

So isn't this a problem in most environment cases then? Thanks!

I don't sure which environment configuration can case issue (only one ENV without drc->init() function) But no issue if at least one CONFIG_ENV_IS_ is activated with driver implementing init ops

But I see the issue in SANDBOX when I activate EXT4 only target. (CONFIG_ENV_IS_IN_EXT4), And no more issue when I add CONFIG_ENV_IS_NOWHERE.

PS: no direct issue if env_init result is not checked

       but I check this result in the sandbox tests in next patches:
	if (!env_init())
	     env_load()
 
       but anyway inconsistent value of gd->env_has_init 
       which can be a problem for any env_has_inited() calls

Regards

Patrick

#8 lore raw

PATCH v2 3/9 env: correctly handle result in env_init

Quoted 50 lines · Patrick DELAUNAY

Hi Tom and Marek,

Quoted 48 lines

From: Tom Rini <[email protected]>
Sent: jeudi 18 juin 2020 21:16

Quoted 23 lines · Patrick Delaunay

Don't return error with ret=-ENOENT when the optional ops drv->init is absent but only if env_driver_lookup doesn't found driver.

This patch correct an issue for the code
if (!env_init())

     env_load()

When only ext4 is supported (CONFIG_ENV_IS_IN_EXT4), as the backend env/ext4.c doesn't define an ops .init

---

(no changes since v1)

 env/env.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)
env/env.c +0 -0
index dcc25c030b..819c88f729 100644
@@ -295,7 +295,10 @@ int env_init(void)
 	int prio;

env/env.c int env_init(void) · line 304

Quoted 10 lines
env/env.c +4 -1
 
 	for (prio = 0; (drv = env_driver_lookup(ENVOP_INIT, prio)); prio++) {
-		if (!drv->init || !(ret = drv->init()))
+		ret = 0;
+		if (drv->init)
+			ret = drv->init();
+		if (!ret)
 			env_set_inited(drv->location);
 
 		debug("%s: Environment %s init done (ret=%d)\n", __func__,

I'm adding in Marek here because this reminds me of similar questions / concerns I had looking in to his series. At root, I think we're not being consistent in each of our env backing implementations about where flags such as ENV_VALID are set, and return values / checks of functions.

Just outside of the start of the patch context here, we set ret to -ENOENT and just past this, if still -ENOENT we say ENV_VALID and point at the default environment.

Quoted 6 lines
Quoted 4 lines

But, I don't follow the patch commit message here. If we don't have drv->init we call env_set_inited(drv->location) but we won't have change ret to 0, which means that later on down the function we go back to default environment.

The cause of the issue is because the init() ops is optional in "struct env_driver".

Right.

Quoted 3 lines

When this opt is absent, I assume that the initialization is not mandatory but this inititialization need to be tagged in gd->env_has_init with the call of env_set_inited() function

So when drv->init isn't set we are already calling env_set_inited(...). If that's not the case, what's going on?

Quoted 18 lines

And the ENV backend is FOUND (don't return -ENOENT)

else the next call of env_has_inited(drv->location) always failed : in env_load()

I see the error in EXT4 env backend,.all the other backend as a env_init() function

But some othe backend don't define the .init operation and have the issue

eeprom.c:235:U_BOOT_ENV_LOCATION(eeprom) = {
ext4.c:135:U_BOOT_ENV_LOCATION(ext4) = {
fat.c:128:U_BOOT_ENV_LOCATION(fat) = {
mmc.c:393:U_BOOT_ENV_LOCATION(mmc) = {
onenand.c:108:U_BOOT_ENV_LOCATION(onenand) = {
sata.c:117:U_BOOT_ENV_LOCATION(sata) = {
ubi.c:179:U_BOOT_ENV_LOCATION(ubi) = {

The other don't have issue:

Quoted 12 lines

flash.c:358:U_BOOT_ENV_LOCATION(flash) = {
flash.c:368: .init = env_flash_init,
nand.c:382:U_BOOT_ENV_LOCATION(nand) = {
nand.c:389: .init = env_nand_init,
nowhere.c:30:U_BOOT_ENV_LOCATION(nowhere) = {
nowhere.c:32: .init = env_nowhere_init,
nvram.c:117:U_BOOT_ENV_LOCATION(nvram) = {
nvram.c:122: .init = env_nvram_init,
remote.c:54:U_BOOT_ENV_LOCATION(remote) = {
remote.c:59: .init = env_remote_init,
sf.c:306:U_BOOT_ENV_LOCATION(sf) = {
sf.c:312: .init = env_sf_init,

Right, there should be a problem showing up on a ton of locations, not just ext4 which is why I'm concerned / confused here. While ext4 isn't as widely used yet as I would expect, FAT/MMC are.

Quoted 8 lines
Quoted 1 line

So isn't this a problem in most environment cases then? Thanks!

I don't sure which environment configuration can case issue (only one ENV without drc->init() function) But no issue if at least one CONFIG_ENV_IS_ is activated with driver implementing init ops

But I see the issue in SANDBOX when I activate EXT4 only target. (CONFIG_ENV_IS_IN_EXT4), And no more issue when I add CONFIG_ENV_IS_NOWHERE.

Quoted 7 lines

PS: no direct issue if env_init result is not checked

       but I check this result in the sandbox tests in next patches:
	if (!env_init())
	     env_load()
 
       but anyway inconsistent value of gd->env_has_init 
       which can be a problem for any env_has_inited() calls

Right. I think there's some bigger inconsistency going on here that needs to be fixed. I'm also confused / concerned how you're not seeing env_set_inite(..) being called. if (!NULL) is true. Thanks!

Signature · 8 lines
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 659 bytes
Desc: not available
URL: <https://lists.denx.de/pipermail/u-boot/attachments/20200619/3bde9267/attachment.sig>
#9 lore raw

PATCH v2 3/9 env: correctly handle result in env_init

Hi Tom,

Quoted 127 lines

From: Tom Rini <[email protected]>
Sent: vendredi 19 juin 2020 20:05

Quoted 45 lines · Patrick DELAUNAY

Hi Tom and Marek,

Quoted 43 lines

From: Tom Rini <[email protected]>
Sent: jeudi 18 juin 2020 21:16

Quoted 23 lines · Patrick Delaunay

Don't return error with ret=-ENOENT when the optional ops drv->init is absent but only if env_driver_lookup doesn't found driver.

This patch correct an issue for the code
if (!env_init())

     env_load()

When only ext4 is supported (CONFIG_ENV_IS_IN_EXT4), as the backend env/ext4.c doesn't define an ops .init

---

(no changes since v1)

 env/env.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)
env/env.c index dcc25c030b..819c88f729 renamed from env/env.c +0 -0

100644

env/env.c +0 -0
@@ -295,7 +295,10 @@ int env_init(void)
 	int prio;

env/env.c int env_init(void) · line 304

Quoted 10 lines
env/env.c +4 -1
 
 	for (prio = 0; (drv = env_driver_lookup(ENVOP_INIT, prio)); prio++) {
-		if (!drv->init || !(ret = drv->init()))
+		ret = 0;
+		if (drv->init)
+			ret = drv->init();
+		if (!ret)
 			env_set_inited(drv->location);
 
 		debug("%s: Environment %s init done (ret=%d)\n", __func__,

I'm adding in Marek here because this reminds me of similar questions / concerns I had looking in to his series. At root, I think we're not being consistent in each of our env backing implementations about where flags such as ENV_VALID are set, and return

values / checks of functions.

Quoted 5 lines
Quoted 5 lines

Just outside of the start of the patch context here, we set ret to -ENOENT and just past this, if still -ENOENT we say ENV_VALID and point at the default environment.

Quoted 7 lines
Quoted 5 lines

But, I don't follow the patch commit message here. If we don't have drv->init we call env_set_inited(drv->location) but we won't have drv->change
ret to 0, which means that later on down the function we go back to default environment.

The cause of the issue is because the init() ops is optional in "struct

env_driver".

Right.

Quoted 4 lines

When this opt is absent, I assume that the initialization is not mandatory but this inititialization need to be tagged in gd->env_has_init with the call of
env_set_inited() function

So when drv->init isn't set we are already calling env_set_inited(...). If that's not the case, what's going on?

Quoted 21 lines

And the ENV backend is FOUND (don't return -ENOENT)

else the next call of env_has_inited(drv->location) always failed : in env_load()

I see the error in EXT4 env backend,.all the other backend as a env_init() function

But some othe backend don't define the .init operation and have the issue

eeprom.c:235:U_BOOT_ENV_LOCATION(eeprom) = {
ext4.c:135:U_BOOT_ENV_LOCATION(ext4) = {
fat.c:128:U_BOOT_ENV_LOCATION(fat) = {
mmc.c:393:U_BOOT_ENV_LOCATION(mmc) = {
onenand.c:108:U_BOOT_ENV_LOCATION(onenand) = {
sata.c:117:U_BOOT_ENV_LOCATION(sata) = {
ubi.c:179:U_BOOT_ENV_LOCATION(ubi) = {

The other don't have issue:

Quoted 12 lines

flash.c:358:U_BOOT_ENV_LOCATION(flash) = {
flash.c:368: .init = env_flash_init,
nand.c:382:U_BOOT_ENV_LOCATION(nand) = {
nand.c:389: .init = env_nand_init,
nowhere.c:30:U_BOOT_ENV_LOCATION(nowhere) = {
nowhere.c:32: .init = env_nowhere_init,
nvram.c:117:U_BOOT_ENV_LOCATION(nvram) = {
nvram.c:122: .init = env_nvram_init,
remote.c:54:U_BOOT_ENV_LOCATION(remote) = {
remote.c:59: .init = env_remote_init,
sf.c:306:U_BOOT_ENV_LOCATION(sf) = {
sf.c:312: .init = env_sf_init,

Right, there should be a problem showing up on a ton of locations, not just ext4 which is why I'm concerned / confused here. While ext4 isn't as widely used yet as I would expect, FAT/MMC are.

Quoted 8 lines
Quoted 1 line

So isn't this a problem in most environment cases then? Thanks!

I don't sure which environment configuration can case issue (only one ENV without drc->init() function) But no issue if at least one CONFIG_ENV_IS_ is activated with driver implementing init ops

But I see the issue in SANDBOX when I activate EXT4 only target. (CONFIG_ENV_IS_IN_EXT4), And no more issue when I add

CONFIG_ENV_IS_NOWHERE.

Quoted 6 lines

PS: no direct issue if env_init result is not checked

       but I check this result in the sandbox tests in next patches:
	if (!env_init())
	     env_load()
Quoted 6 lines
Quoted 2 lines

but anyway inconsistent value of gd->env_has_init which can be a problem for any env_has_inited() calls

Right. I think there's some bigger inconsistency going on here that needs to be fixed. I'm also confused / concerned how you're not seeing env_set_inite(..) being called. if (!NULL) is true. Thanks!

I was confused also...
and I check again the code

And I make a error in my first analysis.

For the case where init ops is not defined in only one backend.

ret = -ENOENT

And the last test is true

	if (ret == -ENOENT) {
		gd->env_addr = (ulong)&default_environment[0];
		gd->env_valid = ENV_VALID;

		return 0;
	}

In fact this function return the LAST result for 'drv->init()' call and that it is strange when several backend are activated.

So this function have no issue when only one ENV backend is activated, and it is the configuration today for most of boards...

I will change my patch in the V3 serie:
env_init() return 0 if, at least, one backend is correctly initialized (when no ops init or the ops init result is OK)

But I don't understood 2 thinks in the last test

1/ Why set ENV Is set to VALID:

	gd->env_valid = ENV_VALID;

	in nowhere backend, for same case of default env, it is set ENV_INVALID....

2/ Why flags is not update

	gd->flags |= GD_FLG_ENV_DEFAULT;

But as it s error case, in should never occurs And I will sent a separate patch for this part to review.

Patrick

#10 lore raw

PATCH v2 3/9 env: correctly handle result in env_init

Quoted 132 lines · Patrick DELAUNAY

Hi Tom,

Quoted 130 lines

From: Tom Rini <[email protected]>
Sent: vendredi 19 juin 2020 20:05

Quoted 45 lines · Patrick DELAUNAY

Hi Tom and Marek,

Quoted 43 lines

From: Tom Rini <[email protected]>
Sent: jeudi 18 juin 2020 21:16

Quoted 23 lines · Patrick Delaunay

Don't return error with ret=-ENOENT when the optional ops drv->init is absent but only if env_driver_lookup doesn't found driver.

This patch correct an issue for the code
if (!env_init())

     env_load()

When only ext4 is supported (CONFIG_ENV_IS_IN_EXT4), as the backend env/ext4.c doesn't define an ops .init

---

(no changes since v1)

 env/env.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)
env/env.c index dcc25c030b..819c88f729 renamed from env/env.c +0 -0

100644

env/env.c +0 -0
@@ -295,7 +295,10 @@ int env_init(void)
 	int prio;

env/env.c int env_init(void) · line 304

Quoted 10 lines
env/env.c +4 -1
 
 	for (prio = 0; (drv = env_driver_lookup(ENVOP_INIT, prio)); prio++) {
-		if (!drv->init || !(ret = drv->init()))
+		ret = 0;
+		if (drv->init)
+			ret = drv->init();
+		if (!ret)
 			env_set_inited(drv->location);
 
 		debug("%s: Environment %s init done (ret=%d)\n", __func__,

I'm adding in Marek here because this reminds me of similar questions / concerns I had looking in to his series. At root, I think we're not being consistent in each of our env backing implementations about where flags such as ENV_VALID are set, and return

values / checks of functions.

Quoted 5 lines
Quoted 5 lines

Just outside of the start of the patch context here, we set ret to -ENOENT and just past this, if still -ENOENT we say ENV_VALID and point at the default environment.

Quoted 7 lines
Quoted 5 lines

But, I don't follow the patch commit message here. If we don't have drv->init we call env_set_inited(drv->location) but we won't have drv->change
ret to 0, which means that later on down the function we go back to default environment.

The cause of the issue is because the init() ops is optional in "struct

env_driver".

Right.

Quoted 4 lines

When this opt is absent, I assume that the initialization is not mandatory but this inititialization need to be tagged in gd->env_has_init with the call of
env_set_inited() function

So when drv->init isn't set we are already calling env_set_inited(...). If that's not the case, what's going on?

Quoted 21 lines

And the ENV backend is FOUND (don't return -ENOENT)

else the next call of env_has_inited(drv->location) always failed : in env_load()

I see the error in EXT4 env backend,.all the other backend as a env_init() function

But some othe backend don't define the .init operation and have the issue

eeprom.c:235:U_BOOT_ENV_LOCATION(eeprom) = {
ext4.c:135:U_BOOT_ENV_LOCATION(ext4) = {
fat.c:128:U_BOOT_ENV_LOCATION(fat) = {
mmc.c:393:U_BOOT_ENV_LOCATION(mmc) = {
onenand.c:108:U_BOOT_ENV_LOCATION(onenand) = {
sata.c:117:U_BOOT_ENV_LOCATION(sata) = {
ubi.c:179:U_BOOT_ENV_LOCATION(ubi) = {

The other don't have issue:

Quoted 12 lines

flash.c:358:U_BOOT_ENV_LOCATION(flash) = {
flash.c:368: .init = env_flash_init,
nand.c:382:U_BOOT_ENV_LOCATION(nand) = {
nand.c:389: .init = env_nand_init,
nowhere.c:30:U_BOOT_ENV_LOCATION(nowhere) = {
nowhere.c:32: .init = env_nowhere_init,
nvram.c:117:U_BOOT_ENV_LOCATION(nvram) = {
nvram.c:122: .init = env_nvram_init,
remote.c:54:U_BOOT_ENV_LOCATION(remote) = {
remote.c:59: .init = env_remote_init,
sf.c:306:U_BOOT_ENV_LOCATION(sf) = {
sf.c:312: .init = env_sf_init,

Right, there should be a problem showing up on a ton of locations, not just ext4 which is why I'm concerned / confused here. While ext4 isn't as widely used yet as I would expect, FAT/MMC are.

Quoted 8 lines
Quoted 1 line

So isn't this a problem in most environment cases then? Thanks!

I don't sure which environment configuration can case issue (only one ENV without drc->init() function) But no issue if at least one CONFIG_ENV_IS_ is activated with driver implementing init ops

But I see the issue in SANDBOX when I activate EXT4 only target. (CONFIG_ENV_IS_IN_EXT4), And no more issue when I add

CONFIG_ENV_IS_NOWHERE.

Quoted 8 lines

PS: no direct issue if env_init result is not checked

       but I check this result in the sandbox tests in next patches:
	if (!env_init())
	     env_load()

       but anyway inconsistent value of gd->env_has_init
       which can be a problem for any env_has_inited() calls
Quoted 6 lines
Quoted 3 lines

Right. I think there's some bigger inconsistency going on here that needs to be fixed. I'm also confused / concerned how you're not seeing env_set_inite(..) being called. if (!NULL) is true. Thanks!

I was confused also...
and I check again the code

Thanks. It's a very tricky section of the code and I think I got some part of it wrong too. What got me off-track was unrolling the test isn't required in what you tried, just adding:
ret = 0; // We found at least one env exits.
before if (!drv->init || !(ret = drv->init()))
would do the same. That said...

Quoted 15 lines

And I make a error in my first analysis.

For the case where init ops is not defined in only one backend.

ret = -ENOENT

And the last test is true

	if (ret == -ENOENT) {
		gd->env_addr = (ulong)&default_environment[0];
		gd->env_valid = ENV_VALID;

		return 0;
	}
Quoted 2 lines

In fact this function return the LAST result for 'drv->init()' call and that it is strange when several backend are activated.

Yes. Things are very fragile in the multi-env case. There being underlying bugs would not surprise me.

Quoted 2 lines

So this function have no issue when only one ENV backend is activated, and it is the configuration today for most of boards...

Right, the common case is the way things have worked historically, env exists in one defined location and when that fails (device not present), we get the built-in environment and saveenv needs to fail rather than crash the board (due to writing to non-existent HW, etc).

Quoted 11 lines

I will change my patch in the V3 serie:
env_init() return 0 if, at least, one backend is correctly initialized (when no ops init or the ops init result is OK)

But I don't understood 2 thinks in the last test

1/ Why set ENV Is set to VALID:

	gd->env_valid = ENV_VALID;

	in nowhere backend, for same case of default env, it is set ENV_INVALID....

A good question. I see:
commit eeba55cb4a8a29a47d0d26692c188b47ba6bf396

Author: Tom Rini <[email protected]>
Date:   Sat Aug 19 22:27:57 2017 -0400

env: Correct case of no sub-init function

changed that from setting it to 0 to setting it to ENV_VALID, where 0 meant ENV_INVALID at the time, but the comments in that commit say it used to be setting it to valid. Which brings us back to:
commit 203e94f6c9ca03e260175ce240f5856507395585

Author: Simon Glass <[email protected]>
Date:   Thu Aug 3 12:21:56 2017 -0600

env: Add an enum for environment state

as adding a enum for 0 = ENV_INVALID, 1 = ENV_VALID, 2 = redundant.

And there's basically part of a longer series to clean up the environment code. In particular:
commit 7938822a6b75b69fff9793b6b1769dddf1249525

Author: Simon Glass <[email protected]>
Date:   Thu Aug 3 12:22:02 2017 -0600

env: Drop common init() function

is the root of this particular question. Before this change, the logic was "For the init function in of an env location, point at the in-memory default environment, declare it valid". With this change the new common env init function (now env_init() then env_init_new() as functionality migrated to it) when we did not have an explicit drv->init() call we need to point at the in-memory default environment and (was wrong at the time, later fixed to...) mark env as valid.

So today, ret = -ENOENT is for the case of no explicit drv->init() function so do what those env drivers were doing before of just pointing gd->env_addr to the default environment and mark it valid.

What we're doing today works but feels too clever, given the amount of time it takes to dig in to see what is supposed to be going on.

In the case of one environment location, this logic works as expected. In the case of multiple environments, I would need to write down all of the possible combinations and walk the code to be sure what happens, especially since it's link-order dependent on the order we try drivers in.

Quoted 7 lines

2/ Why flags is not update

	gd->flags |= GD_FLG_ENV_DEFAULT;

But as it s error case, in should never occurs And I will sent a separate patch for this part to review.

With all of the above in mind, it's not an error case, but being clever with variable names and our errno meanings. So we don't set this flag here because the "default" drv->init() that we're making this snippet of code act like never did that. We only set this flag later on when we're at the point where we know that we cannot get at a stored environment. the "default_environment" variable plays the role of both being the literal default environment as well as being the allocated space we use and move around as gd->env_addr in many cases. The env_init() code block for -ENOENT is to set things up for that case.

Please let me know if this makes sense, or if you think I misread something (which is quite possible). Thanks!

Signature · 8 lines
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 659 bytes
Desc: not available
URL: <https://lists.denx.de/pipermail/u-boot/attachments/20200623/af89f17c/attachment.sig>
#11 lore raw

PATCH v2 3/9 env: correctly handle result in env_init

Hi Tom,

Quoted 169 lines

From: Tom Rini <[email protected]>
Sent: mardi 23 juin 2020 17:17

Quoted 38 lines · Patrick DELAUNAY

Hi Tom,

Quoted 36 lines

From: Tom Rini <[email protected]>
Sent: vendredi 19 juin 2020 20:05

Quoted 32 lines · Patrick DELAUNAY

Hi Tom and Marek,

Quoted 30 lines

From: Tom Rini <[email protected]>
Sent: jeudi 18 juin 2020 21:16

Quoted 25 lines · Patrick Delaunay

Don't return error with ret=-ENOENT when the optional ops drv->init is absent but only if env_driver_lookup doesn't found driver.

This patch correct an issue for the code
if (!env_init())

     env_load()

When only ext4 is supported (CONFIG_ENV_IS_IN_EXT4), as the backend env/ext4.c doesn't define an ops .init

---

(no changes since v1)

 env/env.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)
env/env.c index renamed from env/env.c +0 -0

dcc25c030b..819c88f729
100644

env/env.c +0 -0
@@ -295,7 +295,10 @@ int env_init(void)
 	int prio;
 
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

env/env.c int env_init(void) · line 297

Quoted 1 line
env/env.c +0 -0
 	for (prio = 0; (drv = env_driver_lookup(ENVOP_INIT, prio));

prio++) {

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines

env/env.c int env_init(void)

Quoted 8 lines
env/env.c +4 -1
-		if (!drv->init || !(ret = drv->init()))
+		ret = 0;
+		if (drv->init)
+			ret = drv->init();
+		if (!ret)
 			env_set_inited(drv->location);
 
 		debug("%s: Environment %s init done (ret=%d)\n",

__func__,

Quoted 90 lines
Quoted 90 lines
Quoted 6 lines
Quoted 6 lines

I'm adding in Marek here because this reminds me of similar questions / concerns I had looking in to his series. At root, I think we're not being consistent in each of our env backing implementations about where flags such as ENV_VALID are set, and return

values / checks of functions.

Quoted 12 lines
Quoted 11 lines

Just outside of the start of the patch context here, we set ret to -ENOENT and just past this, if still -ENOENT we say ENV_VALID and point at the default environment.

But, I don't follow the patch commit message here. If we don't have
drv->init we call env_set_inited(drv->location) but we won't drv->have change
ret to 0, which means that later on down the function we go back to default environment.

Quoted 2 lines

The cause of the issue is because the init() ops is optional in "struct

env_driver".

Right.

Quoted 4 lines

When this opt is absent, I assume that the initialization is not mandatory but this inititialization need to be tagged in gd->env_has_init with the call of
env_set_inited() function

So when drv->init isn't set we are already calling env_set_inited(...). If that's not the case, what's going on?

Quoted 22 lines

And the ENV backend is FOUND (don't return -ENOENT)

else the next call of env_has_inited(drv->location) always failed : in
env_load()

I see the error in EXT4 env backend,.all the other backend as a env_init() function

But some othe backend don't define the .init operation and have the issue

eeprom.c:235:U_BOOT_ENV_LOCATION(eeprom) = {
ext4.c:135:U_BOOT_ENV_LOCATION(ext4) = {
fat.c:128:U_BOOT_ENV_LOCATION(fat) = {
mmc.c:393:U_BOOT_ENV_LOCATION(mmc) = {
onenand.c:108:U_BOOT_ENV_LOCATION(onenand) = {
sata.c:117:U_BOOT_ENV_LOCATION(sata) = {
ubi.c:179:U_BOOT_ENV_LOCATION(ubi) = {

The other don't have issue:

Quoted 12 lines

flash.c:358:U_BOOT_ENV_LOCATION(flash) = {
flash.c:368: .init = env_flash_init,
nand.c:382:U_BOOT_ENV_LOCATION(nand) = {
nand.c:389: .init = env_nand_init,
nowhere.c:30:U_BOOT_ENV_LOCATION(nowhere) = {
nowhere.c:32: .init = env_nowhere_init,
nvram.c:117:U_BOOT_ENV_LOCATION(nvram) = {
nvram.c:122: .init = env_nvram_init,
remote.c:54:U_BOOT_ENV_LOCATION(remote) = {
remote.c:59: .init = env_remote_init,
sf.c:306:U_BOOT_ENV_LOCATION(sf) = {
sf.c:312: .init = env_sf_init,

Right, there should be a problem showing up on a ton of locations, not just ext4 which is why I'm concerned / confused here. While ext4 isn't as widely used yet as I would expect, FAT/MMC are.

Quoted 8 lines
Quoted 1 line

So isn't this a problem in most environment cases then? Thanks!

I don't sure which environment configuration can case issue (only one ENV without drc->init() function) But no issue if at least one CONFIG_ENV_IS_ is activated with driver implementing init ops

But I see the issue in SANDBOX when I activate EXT4 only target. (CONFIG_ENV_IS_IN_EXT4), And no more issue when I add

CONFIG_ENV_IS_NOWHERE.

Quoted 8 lines

PS: no direct issue if env_init result is not checked

       but I check this result in the sandbox tests in next patches:
	if (!env_init())
	     env_load()

       but anyway inconsistent value of gd->env_has_init
       which can be a problem for any env_has_inited() calls
Quoted 7 lines
Quoted 4 lines

Right. I think there's some bigger inconsistency going on here that needs to be fixed. I'm also confused / concerned how you're not seeing
env_set_inite(..) being called. if (!NULL) is true. Thanks!

I was confused also...
and I check again the code

Thanks. It's a very tricky section of the code and I think I got some part of it wrong too. What got me off-track was unrolling the test isn't required in what you tried, just adding:
ret = 0; // We found at least one env exits.
before if (!drv->init || !(ret = drv->init())) would do the same. That said...

Quoted 12 lines

And I make a error in my first analysis.

For the case where init ops is not defined in only one backend.

ret = -ENOENT

And the last test is true

	if (ret == -ENOENT) {
		gd->env_addr = (ulong)&default_environment[0];
		gd->env_valid = ENV_VALID;
Quoted 8 lines
Quoted 5 lines
		return 0;
	}

In fact this function return the LAST result for 'drv->init()' call and that it is strange when several backend are activated.

Yes. Things are very fragile in the multi-env case. There being underlying bugs would not surprise me.

I dig deeper in the code and I agree: I will just sent a v3 with minimal update.

The multi-env cases it could be long story.

Quoted 7 lines
Quoted 2 lines

So this function have no issue when only one ENV backend is activated, and it is the configuration today for most of boards...

Right, the common case is the way things have worked historically, env exists in one defined location and when that fails (device not present), we get the built-in environment and saveenv needs to fail rather than crash the board (due to writing to non-existent HW, etc).

Yes tested today on sandbox
env save failed to avoid crash but without any trace.

=> I push a patch to add trace is that make sense. [PATCH] env: add failing trace in env_save
http://patchwork.ozlabs.org/project/uboot/list/?series=185459

Quoted 50 lines
Quoted 11 lines

I will change my patch in the V3 serie:
env_init() return 0 if, at least, one backend is correctly initialized (when no ops init or the ops init result is OK)

But I don't understood 2 thinks in the last test

1/ Why set ENV Is set to VALID:

	gd->env_valid = ENV_VALID;

	in nowhere backend, for same case of default env, it is set

ENV_INVALID....

A good question. I see:
commit eeba55cb4a8a29a47d0d26692c188b47ba6bf396

Author: Tom Rini <[email protected]>
Date:   Sat Aug 19 22:27:57 2017 -0400

env: Correct case of no sub-init function

changed that from setting it to 0 to setting it to ENV_VALID, where 0 meant ENV_INVALID at the time, but the comments in that commit say it used to be setting it to valid. Which brings us back to:
commit 203e94f6c9ca03e260175ce240f5856507395585

Author: Simon Glass <[email protected]>
Date:   Thu Aug 3 12:21:56 2017 -0600

env: Add an enum for environment state

as adding a enum for 0 = ENV_INVALID, 1 = ENV_VALID, 2 = redundant.

And there's basically part of a longer series to clean up the environment code. In particular:
commit 7938822a6b75b69fff9793b6b1769dddf1249525

Author: Simon Glass <[email protected]>
Date:   Thu Aug 3 12:22:02 2017 -0600

env: Drop common init() function

is the root of this particular question. Before this change, the logic was "For the init function in of an env location, point at the in-memory default environment, declare it valid". With this change the new common env init function (now env_init() then env_init_new() as functionality migrated to it) when we did not have an explicit drv->init() call we need to point at the in-memory default environment and (was wrong at the time, later fixed to...) mark env as valid.

So today, ret = -ENOENT is for the case of no explicit drv->init() function so do what those env drivers were doing before of just pointing gd->env_addr to the default environment and mark it valid.

Quoted 7 lines

What we're doing today works but feels too clever, given the amount of time it takes to dig in to see what is supposed to be going on.

In the case of one environment location, this logic works as expected. In the case of multiple environments, I would need to write down all of the possible combinations and walk the code to be sure what happens, especially since it's link-order dependent on the order we try drivers in.

I check the code deeper and
I see many incoherency in multi-env support

For example sometime the global ENV configuration

	gd->env_valid = ENV_VALID;

is initialiazed in init function of each backend...
for me it should be initialzed only on the load ops when the ENV is really valid.

and for default ENV configuration

we have

	gd->env_addr	= (ulong)&default_environment[0];
	gd->env_valid	= ENV_INVALID;

	in this case set gd->env_addr with ENV_INVALID is not necessary 

or

	gd->env_addr	= (ulong)&default_environment[0];
	gd->env_valid	= ENV_VALID;

it seems incoherent or uneeded with

int env_get_char(int index)

{
	if (gd->env_valid == ENV_INVALID)
		return default_environment[index];
	else
		return env_get_char_spec(index);
}

I will continue to check the code deeper but it seems out of the scope for this serie.

I will sent a v3 soon.

Quoted 15 lines
Quoted 7 lines

2/ Why flags is not update

	gd->flags |= GD_FLG_ENV_DEFAULT;

But as it s error case, in should never occurs And I will sent a separate patch for this part to review.

With all of the above in mind, it's not an error case, but being clever with variable names and our errno meanings. So we don't set this flag here because the "default" drv->init() that we're making this snippet of code act like never did that. We only set this flag later on when we're at the point where we know that we cannot get at a stored environment.
the "default_environment" variable plays the role of both being the literal default environment as well as being the allocated space we use and move around as gd-

Quoted 5 lines
Quoted 1 line

env_addr in many cases. The env_init() code block for -ENOENT is to set

things up for that case.

Please let me know if this makes sense, or if you think I misread something (which is quite possible). Thanks!

Yes that make sense: it is fallback to default ENV when all .init failed, I will update my patch in V3 with minimal update of this part and with a new comment.

The env code use many gloabal tag, sometime managed by common code, sometime managed by backend... it is difficult to know the responsibility of each part in multi-env context.

I will continue check of ENV code in background (and perhaps activate and test a other ENV backend in sandbox)

Thanks for your time.

Patrick

#12 lore raw

PATCH v2 3/9 env: correctly handle result in env_init

Quoted 318 lines · Patrick DELAUNAY

Hi Tom,

Quoted 169 lines

From: Tom Rini <[email protected]>
Sent: mardi 23 juin 2020 17:17

Quoted 38 lines · Patrick DELAUNAY

Hi Tom,

Quoted 36 lines

From: Tom Rini <[email protected]>
Sent: vendredi 19 juin 2020 20:05

Quoted 32 lines · Patrick DELAUNAY

Hi Tom and Marek,

Quoted 30 lines

From: Tom Rini <[email protected]>
Sent: jeudi 18 juin 2020 21:16

Quoted 25 lines · Patrick Delaunay

Don't return error with ret=-ENOENT when the optional ops drv->init is absent but only if env_driver_lookup doesn't found driver.

This patch correct an issue for the code
if (!env_init())

     env_load()

When only ext4 is supported (CONFIG_ENV_IS_IN_EXT4), as the backend env/ext4.c doesn't define an ops .init

---

(no changes since v1)

 env/env.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)
env/env.c index renamed from env/env.c +0 -0

dcc25c030b..819c88f729
100644

env/env.c +0 -0
@@ -295,7 +295,10 @@ int env_init(void)
 	int prio;
 
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

env/env.c int env_init(void) · line 297

Quoted 1 line
env/env.c +0 -0
 	for (prio = 0; (drv = env_driver_lookup(ENVOP_INIT, prio));

prio++) {

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines

env/env.c int env_init(void) · line 304

Quoted 8 lines
env/env.c +4 -1
-		if (!drv->init || !(ret = drv->init()))
+		ret = 0;
+		if (drv->init)
+			ret = drv->init();
+		if (!ret)
 			env_set_inited(drv->location);
 
 		debug("%s: Environment %s init done (ret=%d)\n",

__func__,

Quoted 90 lines
Quoted 90 lines
Quoted 6 lines
Quoted 6 lines

I'm adding in Marek here because this reminds me of similar questions / concerns I had looking in to his series. At root, I think we're not being consistent in each of our env backing implementations about where flags such as ENV_VALID are set, and return

values / checks of functions.

Quoted 12 lines
Quoted 11 lines

Just outside of the start of the patch context here, we set ret to -ENOENT and just past this, if still -ENOENT we say ENV_VALID and point at the default environment.

But, I don't follow the patch commit message here. If we don't have
drv->init we call env_set_inited(drv->location) but we won't drv->have change
ret to 0, which means that later on down the function we go back to default environment.

Quoted 2 lines

The cause of the issue is because the init() ops is optional in "struct

env_driver".

Right.

Quoted 4 lines

When this opt is absent, I assume that the initialization is not mandatory but this inititialization need to be tagged in gd->env_has_init with the call of
env_set_inited() function

So when drv->init isn't set we are already calling env_set_inited(...). If that's not the case, what's going on?

Quoted 22 lines

And the ENV backend is FOUND (don't return -ENOENT)

else the next call of env_has_inited(drv->location) always failed : in
env_load()

I see the error in EXT4 env backend,.all the other backend as a env_init() function

But some othe backend don't define the .init operation and have the issue

eeprom.c:235:U_BOOT_ENV_LOCATION(eeprom) = {
ext4.c:135:U_BOOT_ENV_LOCATION(ext4) = {
fat.c:128:U_BOOT_ENV_LOCATION(fat) = {
mmc.c:393:U_BOOT_ENV_LOCATION(mmc) = {
onenand.c:108:U_BOOT_ENV_LOCATION(onenand) = {
sata.c:117:U_BOOT_ENV_LOCATION(sata) = {
ubi.c:179:U_BOOT_ENV_LOCATION(ubi) = {

The other don't have issue:

Quoted 12 lines

flash.c:358:U_BOOT_ENV_LOCATION(flash) = {
flash.c:368: .init = env_flash_init,
nand.c:382:U_BOOT_ENV_LOCATION(nand) = {
nand.c:389: .init = env_nand_init,
nowhere.c:30:U_BOOT_ENV_LOCATION(nowhere) = {
nowhere.c:32: .init = env_nowhere_init,
nvram.c:117:U_BOOT_ENV_LOCATION(nvram) = {
nvram.c:122: .init = env_nvram_init,
remote.c:54:U_BOOT_ENV_LOCATION(remote) = {
remote.c:59: .init = env_remote_init,
sf.c:306:U_BOOT_ENV_LOCATION(sf) = {
sf.c:312: .init = env_sf_init,

Right, there should be a problem showing up on a ton of locations, not just ext4 which is why I'm concerned / confused here. While ext4 isn't as widely used yet as I would expect, FAT/MMC are.

Quoted 8 lines
Quoted 1 line

So isn't this a problem in most environment cases then? Thanks!

I don't sure which environment configuration can case issue (only one ENV without drc->init() function) But no issue if at least one CONFIG_ENV_IS_ is activated with driver implementing init ops

But I see the issue in SANDBOX when I activate EXT4 only target. (CONFIG_ENV_IS_IN_EXT4), And no more issue when I add

CONFIG_ENV_IS_NOWHERE.

Quoted 8 lines

PS: no direct issue if env_init result is not checked

       but I check this result in the sandbox tests in next patches:
	if (!env_init())
	     env_load()

       but anyway inconsistent value of gd->env_has_init
       which can be a problem for any env_has_inited() calls
Quoted 7 lines
Quoted 4 lines

Right. I think there's some bigger inconsistency going on here that needs to be fixed. I'm also confused / concerned how you're not seeing
env_set_inite(..) being called. if (!NULL) is true. Thanks!

I was confused also...
and I check again the code

Thanks. It's a very tricky section of the code and I think I got some part of it wrong too. What got me off-track was unrolling the test isn't required in what you tried, just adding:
ret = 0; // We found at least one env exits.
before if (!drv->init || !(ret = drv->init())) would do the same. That said...

Quoted 12 lines

And I make a error in my first analysis.

For the case where init ops is not defined in only one backend.

ret = -ENOENT

And the last test is true

	if (ret == -ENOENT) {
		gd->env_addr = (ulong)&default_environment[0];
		gd->env_valid = ENV_VALID;
Quoted 8 lines
Quoted 5 lines
		return 0;
	}

In fact this function return the LAST result for 'drv->init()' call and that it is strange when several backend are activated.

Yes. Things are very fragile in the multi-env case. There being underlying bugs would not surprise me.

I dig deeper in the code and I agree: I will just sent a v3 with minimal update.

The multi-env cases it could be long story.

Quoted 7 lines
Quoted 2 lines

So this function have no issue when only one ENV backend is activated, and it is the configuration today for most of boards...

Right, the common case is the way things have worked historically, env exists in one defined location and when that fails (device not present), we get the built-in environment and saveenv needs to fail rather than crash the board (due to writing to non-existent HW, etc).

Yes tested today on sandbox
env save failed to avoid crash but without any trace.

=> I push a patch to add trace is that make sense. [PATCH] env: add failing trace in env_save
http://patchwork.ozlabs.org/project/uboot/list/?series=185459

Quoted 50 lines
Quoted 11 lines

I will change my patch in the V3 serie:
env_init() return 0 if, at least, one backend is correctly initialized (when no ops init or the ops init result is OK)

But I don't understood 2 thinks in the last test

1/ Why set ENV Is set to VALID:

	gd->env_valid = ENV_VALID;

	in nowhere backend, for same case of default env, it is set

ENV_INVALID....

A good question. I see:
commit eeba55cb4a8a29a47d0d26692c188b47ba6bf396

Author: Tom Rini <[email protected]>
Date:   Sat Aug 19 22:27:57 2017 -0400

env: Correct case of no sub-init function

changed that from setting it to 0 to setting it to ENV_VALID, where 0 meant ENV_INVALID at the time, but the comments in that commit say it used to be setting it to valid. Which brings us back to:
commit 203e94f6c9ca03e260175ce240f5856507395585

Author: Simon Glass <[email protected]>
Date:   Thu Aug 3 12:21:56 2017 -0600

env: Add an enum for environment state

as adding a enum for 0 = ENV_INVALID, 1 = ENV_VALID, 2 = redundant.

And there's basically part of a longer series to clean up the environment code. In particular:
commit 7938822a6b75b69fff9793b6b1769dddf1249525

Author: Simon Glass <[email protected]>
Date:   Thu Aug 3 12:22:02 2017 -0600

env: Drop common init() function

is the root of this particular question. Before this change, the logic was "For the init function in of an env location, point at the in-memory default environment, declare it valid". With this change the new common env init function (now env_init() then env_init_new() as functionality migrated to it) when we did not have an explicit drv->init() call we need to point at the in-memory default environment and (was wrong at the time, later fixed to...) mark env as valid.

So today, ret = -ENOENT is for the case of no explicit drv->init() function so do what those env drivers were doing before of just pointing gd->env_addr to the default environment and mark it valid.

Quoted 7 lines

What we're doing today works but feels too clever, given the amount of time it takes to dig in to see what is supposed to be going on.

In the case of one environment location, this logic works as expected. In the case of multiple environments, I would need to write down all of the possible combinations and walk the code to be sure what happens, especially since it's link-order dependent on the order we try drivers in.

I check the code deeper and
I see many incoherency in multi-env support

For example sometime the global ENV configuration

	gd->env_valid = ENV_VALID;

is initialiazed in init function of each backend...
for me it should be initialzed only on the load ops when the ENV is really valid.

and for default ENV configuration

we have

	gd->env_addr	= (ulong)&default_environment[0];
	gd->env_valid	= ENV_INVALID;

	in this case set gd->env_addr with ENV_INVALID is not necessary 

or

	gd->env_addr	= (ulong)&default_environment[0];
	gd->env_valid	= ENV_VALID;

it seems incoherent or uneeded with

int env_get_char(int index)

{
	if (gd->env_valid == ENV_INVALID)
		return default_environment[index];
	else
		return env_get_char_spec(index);
}

I will continue to check the code deeper but it seems out of the scope for this serie.

I will sent a v3 soon.

Quoted 15 lines
Quoted 7 lines

2/ Why flags is not update

	gd->flags |= GD_FLG_ENV_DEFAULT;

But as it s error case, in should never occurs And I will sent a separate patch for this part to review.

With all of the above in mind, it's not an error case, but being clever with variable names and our errno meanings. So we don't set this flag here because the "default" drv->init() that we're making this snippet of code act like never did that. We only set this flag later on when we're at the point where we know that we cannot get at a stored environment.
the "default_environment" variable plays the role of both being the literal default environment as well as being the allocated space we use and move around as gd-

Quoted 5 lines
Quoted 1 line

env_addr in many cases. The env_init() code block for -ENOENT is to set

things up for that case.

Please let me know if this makes sense, or if you think I misread something (which is quite possible). Thanks!

Yes that make sense: it is fallback to default ENV when all .init failed, I will update my patch in V3 with minimal update of this part and with a new comment.

Quoted 8 lines

The env code use many gloabal tag, sometime managed by common code, sometime managed by backend... it is difficult to know the responsibility of each part in multi-env context.

I will continue check of ENV code in background (and perhaps activate and test a other ENV backend in sandbox)

Thanks for your time.

Thanks for looking here as well. Any patches you can add to clarify things or make them more consistent are greatly appreciated.

Signature · 8 lines
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 659 bytes
Desc: not available
URL: <https://lists.denx.de/pipermail/u-boot/attachments/20200624/85e7226b/attachment.sig>
#13 lore raw

PATCH v2 4/9 sandbox: activate env in ext4 support

In reply to Patrick Delaunay

The default environment is still used with "ENVL_NOWHERE" indicated by the weak function env_get_location() and activated by CONFIG_ENV_IS_NOWHERE.

---

(no changes since v1)

 board/sandbox/sandbox.c            | 12 ++++++++++++
 configs/sandbox64_defconfig        |  4 ++++
 configs/sandbox_defconfig          |  4 ++++
 configs/sandbox_flattree_defconfig |  4 ++++
 configs/sandbox_spl_defconfig      |  4 ++++
 5 files changed, 28 insertions(+)
board/sandbox/sandbox.c +12 -0
index 1372003018..9cb5fe5c75 100644
@@ -7,6 +7,7 @@
 #include <cpu_func.h>
 #include <cros_ec.h>
 #include <dm.h>
+#include <env_internal.h>
 #include <init.h>
 #include <led.h>
 #include <os.h>
@@ -44,6 +45,17 @@ unsigned long timer_read_counter(void)
 }
 #endif
 
+enum env_location env_get_location(enum env_operation op, int prio)
+{
+	/* only one location supported */
+	if (prio != 0)
+		return ENVL_UNKNOWN;
+
+	gd->env_load_prio = 0;
+
+	return ENVL_NOWHERE;
+}
+
 int dram_init(void)
 {
 	gd->ram_size = CONFIG_SYS_SDRAM_SIZE;
configs/sandbox64_defconfig +4 -0
index a2410b3368..b70272c0b0 100644
@@ -81,6 +81,10 @@ CONFIG_OF_CONTROL=y
 CONFIG_OF_LIVE=y
 CONFIG_OF_HOSTFILE=y
 CONFIG_DEFAULT_DEVICE_TREE="sandbox64"
+CONFIG_ENV_IS_NOWHERE=y
+CONFIG_ENV_IS_IN_EXT4=y
+CONFIG_ENV_EXT4_INTERFACE="host"
+CONFIG_ENV_EXT4_DEVICE_AND_PART="0:0"
 CONFIG_NETCONSOLE=y
 CONFIG_IP_DEFRAG=y
 CONFIG_REGMAP=y
configs/sandbox_defconfig +4 -0
index 0c9e0b9f1f..715f5dc39d 100644
@@ -90,6 +90,10 @@ CONFIG_OF_CONTROL=y
 CONFIG_OF_LIVE=y
 CONFIG_OF_HOSTFILE=y
 CONFIG_DEFAULT_DEVICE_TREE="sandbox"
+CONFIG_ENV_IS_NOWHERE=y
+CONFIG_ENV_IS_IN_EXT4=y
+CONFIG_ENV_EXT4_INTERFACE="host"
+CONFIG_ENV_EXT4_DEVICE_AND_PART="0:0"
 CONFIG_NETCONSOLE=y
 CONFIG_IP_DEFRAG=y
 CONFIG_REGMAP=y
configs/sandbox_flattree_defconfig +4 -0
index bc4819f998..ce806270bd 100644
@@ -64,6 +64,10 @@ CONFIG_AMIGA_PARTITION=y
 CONFIG_OF_CONTROL=y
 CONFIG_OF_HOSTFILE=y
 CONFIG_DEFAULT_DEVICE_TREE="sandbox"
+CONFIG_ENV_IS_NOWHERE=y
+CONFIG_ENV_IS_IN_EXT4=y
+CONFIG_ENV_EXT4_INTERFACE="host"
+CONFIG_ENV_EXT4_DEVICE_AND_PART="0:0"
 CONFIG_NETCONSOLE=y
 CONFIG_IP_DEFRAG=y
 CONFIG_REGMAP=y
configs/sandbox_spl_defconfig +4 -0
index 64f2ed5102..ea11c9bded 100644
@@ -80,6 +80,10 @@ CONFIG_SPL_OF_CONTROL=y
 CONFIG_OF_HOSTFILE=y
 CONFIG_DEFAULT_DEVICE_TREE="sandbox"
 CONFIG_SPL_OF_PLATDATA=y
+CONFIG_ENV_IS_NOWHERE=y
+CONFIG_ENV_IS_IN_EXT4=y
+CONFIG_ENV_EXT4_INTERFACE="host"
+CONFIG_ENV_EXT4_DEVICE_AND_PART="0:0"
 CONFIG_NETCONSOLE=y
 CONFIG_IP_DEFRAG=y
 CONFIG_SPL_DM=y
Signature · 1 line
2.17.1
#14 lore raw

PATCH v2 5/9 sandbox: support the change of env location

In reply to Patrick Delaunay

Add support of environment location with a new sandbox command 'env_loc'.

When the user change the environment location with the command 'env_loc <location>' the env is reinitialized and saved; the GD_FLG_ENV_DEFAULT flag is also updated.

When the user set the same env location, the environment is re-loaded.

---

Changes in v2:
- change cmd_tbl_t to struct cmd_tbl

 board/sandbox/sandbox.c | 42 ++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 41 insertions(+), 1 deletion(-)
board/sandbox/sandbox.c +41 -1
index 9cb5fe5c75..bd99141fa8 100644
@@ -4,6 +4,7 @@
  */
 
 #include <common.h>
+#include <command.h>
 #include <cpu_func.h>
 #include <cros_ec.h>
 #include <dm.h>
@@ -21,6 +22,9 @@
  */
 gd_t *gd;
 
+/* env location: current location used during test */
+static enum env_location sandbox_env_location = ENVL_NOWHERE;
+
 /* Add a simple GPIO device */
 U_BOOT_DEVICE(gpio_sandbox) = {
 	.name = "gpio_sandbox",
@@ -53,9 +57,45 @@ enum env_location env_get_location(enum env_operation op, int prio)
 
 	gd->env_load_prio = 0;
 
-	return ENVL_NOWHERE;
+	return sandbox_env_location;
 }
 
+static int do_env_loc(struct cmd_tbl *cmdtp, int flag, int argc,
+		      char * const argv[])
+{
+	enum env_location loc;
+
+	if (argc < 2)
+		return CMD_RET_USAGE;
+
+	loc = (enum env_location)simple_strtoul(argv[1], NULL, 10);
+	if (loc >= ENVL_COUNT)
+		return CMD_RET_FAILURE;
+
+	if (sandbox_env_location != loc) {
+		sandbox_env_location = loc;
+		if (loc == ENVL_NOWHERE) {
+			gd->flags |= GD_FLG_ENV_DEFAULT;
+			gd->env_valid = ENV_VALID;
+		} else {
+			if (gd->flags & GD_FLG_ENV_DEFAULT) {
+				gd->flags &= ~GD_FLG_ENV_DEFAULT;
+				if (!env_init())
+					env_save();
+			}
+		}
+	} else {
+		if (!env_init())
+			env_load();
+	}
+
+	return CMD_RET_SUCCESS;
+}
+
+U_BOOT_CMD(env_loc, 2, 1, do_env_loc,
+	   "set the environment location", "[loc]"
+);
+
 int dram_init(void)
 {
 	gd->ram_size = CONFIG_SYS_SDRAM_SIZE;
Signature · 1 line
2.17.1
#15 lore raw

PATCH v2 5/9 sandbox: support the change of env location

Quoted 11 lines · Patrick Delaunay

Add support of environment location with a new sandbox command 'env_loc'.

When the user change the environment location with the command 'env_loc <location>' the env is reinitialized and saved; the GD_FLG_ENV_DEFAULT flag is also updated.

When the user set the same env location, the environment is re-loaded.

Quoted 8 lines

---

Changes in v2:
- change cmd_tbl_t to struct cmd_tbl

 board/sandbox/sandbox.c | 42 ++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 41 insertions(+), 1 deletion(-)
#16 lore raw

PATCH v2 5/9 sandbox: support the change of env location

In reply to Patrick Delaunay
Quoted 10 lines · Patrick Delaunay

Add support of environment location with a new sandbox command 'env_loc'.

When the user change the environment location with the command 'env_loc <location>' the env is reinitialized and saved; the GD_FLG_ENV_DEFAULT flag is also updated.

When the user set the same env location, the environment is re-loaded.

Quoted 8 lines

---

Changes in v2:
- change cmd_tbl_t to struct cmd_tbl

 board/sandbox/sandbox.c | 42 ++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 41 insertions(+), 1 deletion(-)

This is for testing, which is why it's on sandbox? But I think we should have this be a generic opt-in feature as changing where environment is saved at run time has use cases when we have multiple available. Thanks!

Signature · 8 lines
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 659 bytes
Desc: not available
URL: <https://lists.denx.de/pipermail/u-boot/attachments/20200618/a7c0e7df/attachment.sig>
#17 lore raw

PATCH v2 5/9 sandbox: support the change of env location

Hi,

Quoted 20 lines

From: Tom Rini <[email protected]>
Sent: jeudi 18 juin 2020 21:17

Quoted 15 lines · Patrick Delaunay

Add support of environment location with a new sandbox command 'env_loc'.

When the user change the environment location with the command 'env_loc <location>' the env is reinitialized and saved; the GD_FLG_ENV_DEFAULT flag is also updated.

When the user set the same env location, the environment is re-loaded.

---

Changes in v2:
- change cmd_tbl_t to struct cmd_tbl

Quoted 7 lines
Quoted 3 lines
 board/sandbox/sandbox.c | 42
++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 41 insertions(+), 1 deletion(-)

This is for testing, which is why it's on sandbox? But I think we should have this be a generic opt-in feature as changing where environment is saved at run time has use cases when we have multiple available. Thanks!

Yes in my mind it was only for testing on sandbox....

But I agree, I can a add a opt-in generic command to select and load ENV on one target.

Someting as "env load [<target>] " which loads with the request backend and update gd->env_load_prio

With <target> = name of the name define in backend with ENV_NAME macro And using the default location gd->env_load_prio when absent.

Or split in 2 new commands

- env select <target>
- env load

Perhaps this last proposal with 2 command is more flexible.... to be combined with other command (env save / env erase)

if this proposal is OK, I will work on it.....

Patrick

#18 lore raw

PATCH v2 5/9 sandbox: support the change of env location

Quoted 41 lines · Patrick DELAUNAY

Hi,

Quoted 20 lines

From: Tom Rini <[email protected]>
Sent: jeudi 18 juin 2020 21:17

Quoted 15 lines · Patrick Delaunay

Add support of environment location with a new sandbox command 'env_loc'.

When the user change the environment location with the command 'env_loc <location>' the env is reinitialized and saved; the GD_FLG_ENV_DEFAULT flag is also updated.

When the user set the same env location, the environment is re-loaded.

---

Changes in v2:
- change cmd_tbl_t to struct cmd_tbl

Quoted 7 lines
Quoted 3 lines
 board/sandbox/sandbox.c | 42
++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 41 insertions(+), 1 deletion(-)

This is for testing, which is why it's on sandbox? But I think we should have this be a generic opt-in feature as changing where environment is saved at run time has use cases when we have multiple available. Thanks!

Yes in my mind it was only for testing on sandbox....

But I agree, I can a add a opt-in generic command to select and load ENV on one target.

Someting as "env load [<target>] " which loads with the request backend and update gd->env_load_prio

With <target> = name of the name define in backend with ENV_NAME macro And using the default location gd->env_load_prio when absent.

Or split in 2 new commands

Quoted 7 lines

- env select <target>
- env load

Perhaps this last proposal with 2 command is more flexible.... to be combined with other command (env save / env erase)

if this proposal is OK, I will work on it.....

Sounds good to me, thanks!

Signature · 8 lines
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 659 bytes
Desc: not available
URL: <https://lists.denx.de/pipermail/u-boot/attachments/20200619/b992f1a8/attachment.sig>
#19 lore raw

PATCH v2 6/9 test: environment in ext4

In reply to Patrick Delaunay

Add basic test to persistent environment in ext4:
save and load in host ext4 file 'uboot.env'.

On first execution an empty EXT4 file system is created in persistent data dir: env.ext4.img.

---

(no changes since v1)

 test/py/tests/test_env.py | 87 +++++++++++++++++++++++++++++++++++++++
 1 file changed, 87 insertions(+)
test/py/tests/test_env.py +87 -0
index cbdb41031c..6f1d94b953 100644
@@ -4,6 +4,10 @@
 
 # Test operation of shell commands relating to environment variables.
 
+import os
+import os.path
+from subprocess import call, check_call
+
 import pytest
 import u_boot_utils
 
@@ -380,3 +384,86 @@ def test_env_info_quiet(state_test_env):
     assert response == ""
     response = c.run_command('echo $?')
     assert response == "1"
+
+def mk_env_ext4(state_test_env):
+    c = state_test_env.u_boot_console
+
+    """Create a empty ext4 file system volume.
+    """
+    filename = 'env.ext4.img'
+    persistent = c.config.persistent_data_dir + '/' + filename
+    fs_img = c.config.result_dir  + '/' + filename
+
+    if os.path.exists(persistent):
+        c.log.action('Disk image file ' + persistent + ' already exists')
+    else:
+        try:
+            check_call('rm -f %s' % persistent, shell=True)
+            check_call('dd if=/dev/zero of=%s bs=1M count=16'
+                % persistent, shell=True)
+            check_call('mkfs.ext4 -O ^metadata_csum %s' % persistent, shell=True)
+        except CalledProcessError:
+            call('rm -f %s' % persistent, shell=True)
+            raise
+
+    call('cp -f %s %s' % (persistent, fs_img), shell=True)
+    return fs_img
+
+ at pytest.mark.boardspec('sandbox')
+ at pytest.mark.buildconfigspec('cmd_nvedit_info')
+ at pytest.mark.buildconfigspec('cmd_echo')
+ at pytest.mark.buildconfigspec('env_is_in_ext4')
+def test_env_ext4(state_test_env):
+
+    c = state_test_env.u_boot_console
+
+    fs_img = mk_env_ext4(state_test_env)
+    c.run_command('host bind 0  %s' % fs_img)
+
+    response = c.run_command('ext4ls host 0:0')
+    assert 'uboot.env' not in response
+
+    """ env location: ENVL_EXT4 (2)
+    """
+    response = c.run_command('env_loc 2')
+    assert 'Saving Environment to EXT4' in response
+
+    response = c.run_command('env_loc 2')
+    assert 'Loading Environment from EXT4... OK' in response
+
+    response = c.run_command('ext4ls host 0:0')
+    assert '8192 uboot.env' in response
+
+    response = c.run_command('env info')
+    assert 'env_valid = valid' in response
+    assert 'env_ready = true' in response
+    assert 'env_use_default = false' in response
+
+    response = c.run_command('env info -p -d')
+    assert 'Environment was loaded from persistent storage' in response
+    assert 'Environment can be persisted' in response
+
+    response = c.run_command('env info -d -q')
+    assert response == ""
+    response = c.run_command('echo $?')
+    assert response == "1"
+
+    response = c.run_command('env info -p -q')
+    assert response == ""
+    response = c.run_command('echo $?')
+    assert response == "0"
+
+    """ restore env location: ENVL_NOWHERE (12)
+    """
+    c.run_command('env_loc 12')
+
+    response = c.run_command('env info')
+    assert 'env_valid = valid' in response
+    assert 'env_ready = true' in response
+    assert 'env_use_default = true' in response
+
+    response = c.run_command('env info -p -d')
+    assert 'Default environment is used' in response
+    assert 'Environment cannot be persisted' in response
+
+    call('rm -f %s' % fs_img, shell=True)
Signature · 1 line
2.17.1
#20 lore raw

PATCH v2 6/9 test: environment in ext4

Quoted 7 lines · Patrick Delaunay

Add basic test to persistent environment in ext4:
save and load in host ext4 file 'uboot.env'.

On first execution an empty EXT4 file system is created in persistent data dir: env.ext4.img.

test/py/tests/test_env.py def test_env_info_quiet(state_test_env): · line 401

Quoted 8 lines
test/py/tests/test_env.py +6 -0
 
+def mk_env_ext4(state_test_env):
 
+    if os.path.exists(persistent):
+        c.log.action('Disk image file ' + persistent + ' already exists')
+    else:
+        try:
+            check_call('rm -f %s' % persistent, shell=True)

This should be run with the results logged to the overall test log file so that if there are failures, it's possible to see what they were. Use util.run_and_log() for this.

Also, this particular command doesn't seem useful, since 4 lines above the code verified that the file doesn't exist.

test/py/tests/test_env.py def test_env_info_quiet(state_test_env): · line 418

Quoted 7 lines

+ at pytest.mark.boardspec('sandbox')
+ at pytest.mark.buildconfigspec('cmd_nvedit_info')
+ at pytest.mark.buildconfigspec('cmd_echo')
+ at pytest.mark.buildconfigspec('env_is_in_ext4')

test/py/tests/test_env.py +3 -0
+def test_env_ext4(state_test_env):
+
+    c = state_test_env.u_boot_console

Nit: That blank line is a bit odd.

test/py/tests/test_env.py def test_env_info_quiet(state_test_env): · line 427

Quoted 8 lines
test/py/tests/test_env.py +8 -0
+    fs_img = mk_env_ext4(state_test_env)
+    c.run_command('host bind 0  %s' % fs_img)
+
+    response = c.run_command('ext4ls host 0:0')
+    assert 'uboot.env' not in response
+
+    """ env location: ENVL_EXT4 (2)
+    """

Nit: Wrap the trailing """ onto the same line; no need to force it to be a multi-line string. Also a comman may be better rather than a docstring. Same for the other docstring later.

Quoted 1 line

+ call('rm -f %s' % fs_img, shell=True)

This won't happen if the test fails earlier. Should there be a try/except block or wrapper function with exception handling to resolve this?

#21 lore raw

PATCH v2 6/9 test: environment in ext4

Hi,

Quoted 20 lines

From: Stephen Warren <[email protected]>
Sent: lundi 22 juin 2020 20:57

Quoted 7 lines · Patrick Delaunay

Add basic test to persistent environment in ext4:
save and load in host ext4 file 'uboot.env'.

On first execution an empty EXT4 file system is created in persistent data dir: env.ext4.img.

test/py/tests/test_env.py def test_env_info_quiet(state_test_env): · line 401

Quoted 8 lines
test/py/tests/test_env.py +6 -0
 
+def mk_env_ext4(state_test_env):
 
+    if os.path.exists(persistent):
+        c.log.action('Disk image file ' + persistent + ' already exists')
+    else:
+        try:
+            check_call('rm -f %s' % persistent, shell=True)
Quoted 3 lines

This should be run with the results logged to the overall test log file so that if there are failures, it's possible to see what they were. Use
util.run_and_log() for this.

Ok, I will modifiy this part (I copy ext4 file create is copied form py/tests/test_fs/conftest.py:166 mk_fs() )

Quoted 2 lines

Also, this particular command doesn't seem useful, since 4 lines above the code verified that the file doesn't exist.

Yes not needed, I remove this line.

Quoted 9 lines

test/py/tests/test_env.py def test_env_info_quiet(state_test_env): · line 418

Quoted 7 lines

+ at pytest.mark.boardspec('sandbox')
+ at pytest.mark.buildconfigspec('cmd_nvedit_info')
+ at pytest.mark.buildconfigspec('cmd_echo')
+ at pytest.mark.buildconfigspec('env_is_in_ext4')

test/py/tests/test_env.py +3 -0
+def test_env_ext4(state_test_env):
+
+    c = state_test_env.u_boot_console

Nit: That blank line is a bit odd.

OK

Quoted 4 lines
Quoted 4 lines
test/py/tests/test_env.py +4 -0
+    fs_img = mk_env_ext4(state_test_env)
+    c.run_command('host bind 0  %s' % fs_img)
+
+    response = c.run_command('ext4ls host 0:0')
Quoted 8 lines

test/py/tests/test_env.py def test_env_info_quiet(state_test_env): · line 427

Quoted 4 lines
test/py/tests/test_env.py +4 -0
+    assert 'uboot.env' not in response
+
+    """ env location: ENVL_EXT4 (2)
+    """

Nit: Wrap the trailing """ onto the same line; no need to force it to be a multi-line string. Also a comman may be better rather than a docstring. Same for the other docstring later.

OK, I don't realized that it was docstring.

Quoted 4 lines
Quoted 1 line

+ call('rm -f %s' % fs_img, shell=True)

This won't happen if the test fails earlier. Should there be a try/except block or wrapper function with exception handling to resolve this?

Yes, I add it for V3

Patrick

#22 lore raw

PATCH v2 7/9 env: ext4: introduce new function env_ext4_save_buffer

In reply to Patrick Delaunay

Split the function env_ext4_save to prepare the erase support.

---

(no changes since v1)

 env/ext4.c | 28 ++++++++++++++++++++--------
 1 file changed, 20 insertions(+), 8 deletions(-)
env/ext4.c +20 -8
index ac9f126bec..027a721b69 100644
@@ -44,9 +44,8 @@ __weak const char *env_ext4_get_dev_part(void)
 	return (const char *)CONFIG_ENV_EXT4_DEVICE_AND_PART;
 }
 
-static int env_ext4_save(void)
+static int env_ext4_save_buffer(env_t *env_new)
 {
-	env_t	env_new;
 	struct blk_desc *dev_desc = NULL;
 	struct disk_partition info;
 	int dev, part;
@@ -54,10 +53,6 @@ static int env_ext4_save(void)
 	const char *ifname = env_ext4_get_intf();
 	const char *dev_and_part = env_ext4_get_dev_part();
 
-	err = env_export(&env_new);
-	if (err)
-		return err;
-
 	part = blk_get_device_part_str(ifname, dev_and_part,
 				       &dev_desc, &info, 1);
 	if (part < 0)
@@ -72,7 +67,7 @@ static int env_ext4_save(void)
 		return 1;
 	}
 
-	err = ext4fs_write(CONFIG_ENV_EXT4_FILE, (void *)&env_new,
+	err = ext4fs_write(CONFIG_ENV_EXT4_FILE, (void *)env_new,
 			   sizeof(env_t), FILETYPE_REG);
 	ext4fs_close();
 
@@ -81,9 +76,26 @@ static int env_ext4_save(void)
 			CONFIG_ENV_EXT4_FILE, ifname, dev, part);
 		return 1;
 	}
-	gd->env_valid = ENV_VALID;
 
+	return 0;
+}
+
+static int env_ext4_save(void)
+{
+	env_t	env_new;
+	int err;
+
+	err = env_export(&env_new);
+	if (err)
+		return err;
+
+	err = env_ext4_save_buffer(&env_new);
+	if (err)
+		return err;
+
+	gd->env_valid = ENV_VALID;
 	puts("done\n");
+
 	return 0;
 }
 
Signature · 1 line
2.17.1
#23 lore raw

PATCH v2 8/9 env: ext4: add support of command env erase

In reply to Patrick Delaunay

Add support of opts erase for env in ext4,
this opts is used by command 'env erase'.

This command only fill the env file (CONFIG_ENV_EXT4_FILE) with 0, the CRC and the saved environment becomes invalid.

---

Changes in v2:
- use CONFIG_IS_ENABLED to set .erase (same as .save)

 env/ext4.c | 19 +++++++++++++++++++
 1 file changed, 19 insertions(+)
env/ext4.c +19 -0
index 027a721b69..7a104c1615 100644
@@ -99,6 +99,23 @@ static int env_ext4_save(void)
 	return 0;
 }
 
+static int env_ext4_erase(void)
+{
+	env_t	env_new;
+	int err;
+
+	memset(&env_new, 0, sizeof(env_t));
+
+	err = env_ext4_save_buffer(&env_new);
+	if (err)
+		return err;
+
+	gd->env_valid = ENV_INVALID;
+	puts("done\n");
+
+	return 0;
+}
+
 static int env_ext4_load(void)
 {
 	ALLOC_CACHE_ALIGN_BUFFER(char, buf, CONFIG_ENV_SIZE);
@@ -156,4 +173,6 @@ U_BOOT_ENV_LOCATION(ext4) = {
 	ENV_NAME("EXT4")
 	.load		= env_ext4_load,
 	.save		= ENV_SAVE_PTR(env_ext4_save),
+	.erase		= CONFIG_IS_ENABLED(CMD_ERASEENV) ? env_ext4_erase :
+							    NULL,
 };
Signature · 1 line
2.17.1
#24 lore raw

PATCH v2 9/9 test: sandbox: add test for erase command

In reply to Patrick Delaunay

Add test for the erase command tested on ENV in EXT4.

---

(no changes since v1)

 configs/sandbox64_defconfig        |  1 +
 configs/sandbox_defconfig          |  1 +
 configs/sandbox_flattree_defconfig |  1 +
 configs/sandbox_spl_defconfig      |  1 +
 test/py/tests/test_env.py          | 16 ++++++++++++++++
 5 files changed, 20 insertions(+)
configs/sandbox64_defconfig +1 -0
index b70272c0b0..3913b6392e 100644
@@ -27,6 +27,7 @@ CONFIG_CMD_BOOTEFI_HELLO=y
 # CONFIG_CMD_ELF is not set
 CONFIG_CMD_ASKENV=y
 CONFIG_CMD_GREPENV=y
+CONFIG_CMD_ERASEENV=y
 CONFIG_CMD_ENV_CALLBACK=y
 CONFIG_CMD_ENV_FLAGS=y
 CONFIG_CMD_NVEDIT_EFI=y
configs/sandbox_defconfig +1 -0
index 715f5dc39d..c59e20579d 100644
@@ -31,6 +31,7 @@ CONFIG_CMD_ABOOTIMG=y
 # CONFIG_CMD_ELF is not set
 CONFIG_CMD_ASKENV=y
 CONFIG_CMD_GREPENV=y
+CONFIG_CMD_ERASEENV=y
 CONFIG_CMD_ENV_CALLBACK=y
 CONFIG_CMD_ENV_FLAGS=y
 CONFIG_CMD_NVEDIT_EFI=y
configs/sandbox_flattree_defconfig +1 -0
index ce806270bd..99b750b4f8 100644
@@ -24,6 +24,7 @@ CONFIG_CMD_BOOTEFI_HELLO=y
 # CONFIG_CMD_ELF is not set
 CONFIG_CMD_ASKENV=y
 CONFIG_CMD_GREPENV=y
+CONFIG_CMD_ERASEENV=y
 CONFIG_CMD_NVEDIT_INFO=y
 CONFIG_LOOPW=y
 CONFIG_CMD_MD5SUM=y
configs/sandbox_spl_defconfig +1 -0
index ea11c9bded..52e7625bd7 100644
@@ -32,6 +32,7 @@ CONFIG_CMD_BOOTEFI_HELLO=y
 # CONFIG_CMD_ELF is not set
 CONFIG_CMD_ASKENV=y
 CONFIG_CMD_GREPENV=y
+CONFIG_CMD_ERASEENV=y
 CONFIG_CMD_ENV_CALLBACK=y
 CONFIG_CMD_ENV_FLAGS=y
 CONFIG_CMD_NVEDIT_INFO=y
test/py/tests/test_env.py +16 -0
index 6f1d94b953..a71b4c2571 100644
@@ -453,6 +453,22 @@ def test_env_ext4(state_test_env):
     response = c.run_command('echo $?')
     assert response == "0"
 
+    response = c.run_command('env erase')
+    assert 'OK' in response
+
+    response = c.run_command('env_loc 2')
+    assert 'Loading Environment from EXT4... ' in response
+    assert 'bad CRC, using default environment' in response
+
+    response = c.run_command('env info')
+    assert 'env_valid = invalid' in response
+    assert 'env_ready = true' in response
+    assert 'env_use_default = true' in response
+
+    response = c.run_command('env info -p -d')
+    assert 'Default environment is used' in response
+    assert 'Environment can be persisted' in response
+
     """ restore env location: ENVL_NOWHERE (12)
     """
     c.run_command('env_loc 12')
Signature · 1 line
2.17.1
#25 lore raw

PATCH v2 9/9 test: sandbox: add test for erase command

Quoted 6 lines · Patrick Delaunay

Add test for the erase command tested on ENV in EXT4.

---

Quoted 9 lines

(no changes since v1)

 configs/sandbox64_defconfig        |  1 +
 configs/sandbox_defconfig          |  1 +
 configs/sandbox_flattree_defconfig |  1 +
 configs/sandbox_spl_defconfig      |  1 +
 test/py/tests/test_env.py          | 16 ++++++++++++++++
 5 files changed, 20 insertions(+)