Skip to contentSkip to conversation

env: Add support for environments on arbitrary filesystems

PATCH New Fiach Antaw 2 participants 9 messages Jan 25, 2017 – Jul 25, 2017

lore
Share & tools
lore
#0 lore raw

cover env: Add support for environments on arbitrary filesystems

This patch series consolidates the redundancy implementations used by env_nand, env_mmc and env_ubi, and adds support for a generic env_fs (also using the same redundancy implementation) that supports all filesystems supported by the filesystem code.

env_fs should be able to replace env_fat and env_ext4 in all cases, since it implements the same functionality.

Unfortunately, I am unable to verify the changes to env_nand, env_mmc and env_ubi on real hardware, though I have verified that they do compile.

Fiach Antaw (3):
env: Add generic redundant environment implementation
env: Switch env_nand, env_mmc and env_ubi to env_import_redund
env: Add support for FS environment

 README                       |  22 ++++++++
 cmd/nvedit.c                 |   1 +
 common/Makefile              |   1 +
 common/env_common.c          |  51 ++++++++++++++++++
 common/env_fs.c              | 126 +++++++++++++++++++++++++++++++++++++++++++
 common/env_mmc.c             |  42 +++------------
 common/env_nand.c            |  45 +++-------------
 common/env_ubi.c             |  39 +-------------
 include/environment.h        |  21 ++++++++
 scripts/config_whitelist.txt |   5 ++
 10 files changed, 243 insertions(+), 110 deletions(-)
 create mode 100644 common/env_fs.c
Signature · 1 line
2.7.4
#1 lore raw

PATCH 1/3 env: Add generic redundant environment implementation

All current environments that implement redundancy use almost identical implementations. This patch implements the env_nand implementation as a function in env_common, and updates the env_export function to export an env_nand-style 'flags' field by default.

---

 common/env_common.c   | 51 +++++++++++++++++++++++++++++++++++++++++++++++++++
 include/environment.h |  5 +++++
 2 files changed, 56 insertions(+)
common/env_common.c +51 -0
index 7fb62e8..ea2f11f 100644
@@ -226,6 +226,53 @@ int env_import(const char *buf, int check)
 	return 0;
 }
 
+#ifdef CONFIG_SYS_REDUNDAND_ENVIRONMENT
+static unsigned char env_flags;
+
+int env_import_redund(const char *buf1, const char *buf2)
+{
+	int crc1_ok, crc2_ok;
+	env_t *ep, *tmp_env1, *tmp_env2;
+
+	tmp_env1 = (env_t *)buf1;
+	tmp_env2 = (env_t *)buf2;
+
+	crc1_ok = crc32(0, tmp_env1->data, ENV_SIZE) ==
+			tmp_env1->crc;
+	crc2_ok = crc32(0, tmp_env2->data, ENV_SIZE) ==
+			tmp_env2->crc;
+
+	if (!crc1_ok && !crc2_ok) {
+		set_default_env("!bad CRC");
+		return 0;
+	} else if (crc1_ok && !crc2_ok) {
+		gd->env_valid = 1;
+	} else if (!crc1_ok && crc2_ok) {
+		gd->env_valid = 2;
+	} else {
+		/* both ok - check serial */
+		if (tmp_env1->flags == 255 && tmp_env2->flags == 0)
+			gd->env_valid = 2;
+		else if (tmp_env2->flags == 255 && tmp_env1->flags == 0)
+			gd->env_valid = 1;
+		else if (tmp_env1->flags > tmp_env2->flags)
+			gd->env_valid = 1;
+		else if (tmp_env2->flags > tmp_env1->flags)
+			gd->env_valid = 2;
+		else /* flags are equal - almost impossible */
+			gd->env_valid = 1;
+	}
+
+	if (gd->env_valid == 1)
+		ep = tmp_env1;
+	else
+		ep = tmp_env2;
+
+	env_flags = ep->flags;
+	return env_import((char *)ep, 0);
+}
+#endif /* CONFIG_SYS_REDUNDAND_ENVIRONMENT */
+
 /* Export the environment and generate CRC for it. */
 int env_export(env_t *env_out)
 {
@@ -247,6 +294,10 @@ int env_export(env_t *env_out)
 
 	env_out->crc = crc32(0, env_out->data, ENV_SIZE);
 
+#ifdef CONFIG_SYS_REDUNDAND_ENVIRONMENT
+	env_out->flags = ++env_flags; /* increase the serial */
+#endif
+
 	return 0;
 }
 
include/environment.h +5 -0
index b602e8a..0c718eb 100644
@@ -222,6 +222,11 @@ int env_import(const char *buf, int check);
 /* Export from hash table into binary representation */
 int env_export(env_t *env_out);
 
+#ifdef CONFIG_SYS_REDUNDAND_ENVIRONMENT
+/* Select and import one of two redundant environments */
+int env_import_redund(const char *buf1, const char *buf2);
+#endif
+
 #endif /* DO_DEPS_ONLY */
 
 #endif /* _ENVIRONMENT_H_ */
Signature · 1 line
2.7.4
#2 lore raw

PATCH 1/3 env: Add generic redundant environment implementation

Quoted 7 lines · Fiach Antaw

All current environments that implement redundancy use almost identical implementations. This patch implements the env_nand implementation as a function in env_common, and updates the env_export function to export an env_nand-style 'flags' field by default.

Signature · 8 lines
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: Digital signature
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20170128/5cee082d/attachment.sig>
#3 lore raw

PATCH 1/3 env: Add generic redundant environment implementation

In reply to Fiach Antaw
Quoted 8 lines · Fiach Antaw

All current environments that implement redundancy use almost identical implementations. This patch implements the env_nand implementation as a function in env_common, and updates the env_export function to export an env_nand-style 'flags' field by default.

Applied to u-boot/master, thanks!

Signature · 8 lines
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: Digital signature
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20170724/d1642df4/attachment.sig>
#4 lore raw

PATCH 2/3 env: Switch env_nand, env_mmc and env_ubi to env_import_redund

In reply to Fiach Antaw

The env_nand, env_mmc and env_ubi implementations all implement redundancy using an identical serial-number scheme. This commit migrates them to use the implementation in env_common, which is functionally identical.

---

 common/env_mmc.c  | 42 ++++++------------------------------------
 common/env_nand.c | 45 ++++++++-------------------------------------
 common/env_ubi.c  | 39 ++-------------------------------------
 3 files changed, 16 insertions(+), 110 deletions(-)
common/env_mmc.c +6 -36
index 16f6a17..bb27f57 100644
@@ -138,10 +138,6 @@ static inline int write_env(struct mmc *mmc, unsigned long size,
 	return (n == blk_cnt) ? 0 : -1;
 }
 
-#ifdef CONFIG_ENV_OFFSET_REDUND
-static unsigned char env_flags;
-#endif
-
 int saveenv(void)
 {
 	ALLOC_CACHE_ALIGN_BUFFER(env_t, env_new, 1);
@@ -162,8 +158,6 @@ int saveenv(void)
 		goto fini;
 
 #ifdef CONFIG_ENV_OFFSET_REDUND
-	env_new->flags	= ++env_flags; /* increase the serial */
-
 	if (gd->env_valid == 1)
 		copy = 1;
 #endif
@@ -214,8 +208,6 @@ void env_relocate_spec(void)
 	struct mmc *mmc;
 	u32 offset1, offset2;
 	int read1_fail = 0, read2_fail = 0;
-	int crc1_ok = 0, crc2_ok = 0;
-	env_t *ep;
 	int ret;
 	int dev = mmc_get_env_dev();
 	const char *errmsg = NULL;
@@ -250,42 +242,20 @@ void env_relocate_spec(void)
 		puts("*** Warning - some problems detected "
 		     "reading environment; recovered successfully\n");
 
-	crc1_ok = !read1_fail &&
-		(crc32(0, tmp_env1->data, ENV_SIZE) == tmp_env1->crc);
-	crc2_ok = !read2_fail &&
-		(crc32(0, tmp_env2->data, ENV_SIZE) == tmp_env2->crc);
-
-	if (!crc1_ok && !crc2_ok) {
+	if (read1_fail && read2_fail) {
 		errmsg = "!bad CRC";
 		ret = 1;
 		goto fini;
-	} else if (crc1_ok && !crc2_ok) {
+	} else if (!read1_fail && read2_fail) {
 		gd->env_valid = 1;
-	} else if (!crc1_ok && crc2_ok) {
+		env_import((char *)tmp_env1, 1);
+	} else if (read1_fail && !read2_fail) {
 		gd->env_valid = 2;
+		env_import((char *)tmp_env2, 1);
 	} else {
-		/* both ok - check serial */
-		if (tmp_env1->flags == 255 && tmp_env2->flags == 0)
-			gd->env_valid = 2;
-		else if (tmp_env2->flags == 255 && tmp_env1->flags == 0)
-			gd->env_valid = 1;
-		else if (tmp_env1->flags > tmp_env2->flags)
-			gd->env_valid = 1;
-		else if (tmp_env2->flags > tmp_env1->flags)
-			gd->env_valid = 2;
-		else /* flags are equal - almost impossible */
-			gd->env_valid = 1;
+		env_import_redund((char *)tmp_env1, (char *)tmp_env2);
 	}
 
-	free(env_ptr);
-
-	if (gd->env_valid == 1)
-		ep = tmp_env1;
-	else
-		ep = tmp_env2;
-
-	env_flags = ep->flags;
-	env_import((char *)ep, 0);
 	ret = 0;
 
 fini:
common/env_nand.c +8 -37

75 lines not on this page · raw message

common/env_ubi.c +2 -37
index 0ac2f65..95b527d 100644
@@ -33,8 +33,6 @@ int env_init(void)
 
 #ifdef CONFIG_CMD_SAVEENV
 #ifdef CONFIG_SYS_REDUNDAND_ENVIRONMENT
-static unsigned char env_flags;
-
 int saveenv(void)
 {
 	ALLOC_CACHE_ALIGN_BUFFER(env_t, env_new, 1);
@@ -50,8 +48,6 @@ int saveenv(void)
 		return 1;
 	}
 
-	env_new->flags = ++env_flags; /* increase the serial */
-
 	if (gd->env_valid == 1) {
 		puts("Writing to redundant UBI... ");
 		if (ubi_volume_write(CONFIG_ENV_UBI_VOLUME_REDUND,
@@ -112,8 +108,7 @@ void env_relocate_spec(void)
 {
 	ALLOC_CACHE_ALIGN_BUFFER(char, env1_buf, CONFIG_ENV_SIZE);
 	ALLOC_CACHE_ALIGN_BUFFER(char, env2_buf, CONFIG_ENV_SIZE);
-	int crc1_ok = 0, crc2_ok = 0;
-	env_t *ep, *tmp_env1, *tmp_env2;
+	env_t *tmp_env1, *tmp_env2;
 
 	/*
 	 * In case we have restarted u-boot there is a chance that buffer
@@ -148,37 +143,7 @@ void env_relocate_spec(void)
 		       CONFIG_ENV_UBI_PART, CONFIG_ENV_UBI_VOLUME_REDUND);
 	}
 
-	crc1_ok = crc32(0, tmp_env1->data, ENV_SIZE) == tmp_env1->crc;
-	crc2_ok = crc32(0, tmp_env2->data, ENV_SIZE) == tmp_env2->crc;
-
-	if (!crc1_ok && !crc2_ok) {
-		set_default_env("!bad CRC");
-		return;
-	} else if (crc1_ok && !crc2_ok) {
-		gd->env_valid = 1;
-	} else if (!crc1_ok && crc2_ok) {
-		gd->env_valid = 2;
-	} else {
-		/* both ok - check serial */
-		if (tmp_env1->flags == 255 && tmp_env2->flags == 0)
-			gd->env_valid = 2;
-		else if (tmp_env2->flags == 255 && tmp_env1->flags == 0)
-			gd->env_valid = 1;
-		else if (tmp_env1->flags > tmp_env2->flags)
-			gd->env_valid = 1;
-		else if (tmp_env2->flags > tmp_env1->flags)
-			gd->env_valid = 2;
-		else /* flags are equal - almost impossible */
-			gd->env_valid = 1;
-	}
-
-	if (gd->env_valid == 1)
-		ep = tmp_env1;
-	else
-		ep = tmp_env2;
-
-	env_flags = ep->flags;
-	env_import((char *)ep, 0);
+	env_import_redund((char *)tmp_env1, (char *)tmp_env2);
 }
 #else /* ! CONFIG_SYS_REDUNDAND_ENVIRONMENT */
 void env_relocate_spec(void)
Signature · 1 line
2.7.4
#5 lore raw

PATCH 2/3 env: Switch env_nand, env_mmc and env_ubi to env_import_redund

Quoted 6 lines · Fiach Antaw

The env_nand, env_mmc and env_ubi implementations all implement redundancy using an identical serial-number scheme. This commit migrates them to use the implementation in env_common, which is functionally identical.

Signature · 8 lines
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: Digital signature
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20170128/9ddb0d6b/attachment.sig>
#6 lore raw

PATCH 2/3 env: Switch env_nand, env_mmc and env_ubi to env_import_redund

In reply to Fiach Antaw
Quoted 7 lines · Fiach Antaw

The env_nand, env_mmc and env_ubi implementations all implement redundancy using an identical serial-number scheme. This commit migrates them to use the implementation in env_common, which is functionally identical.

Applied to u-boot/master, thanks!

Signature · 8 lines
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: Digital signature
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20170724/a324b541/attachment.sig>
#7 lore raw

PATCH 3/3 env: Add support for FS environment

In reply to Fiach Antaw

This patch adds support for fat/ext4-style environments on top of the FS api, allowing any filesystem to be used to store the u-boot environment. This implementation also support redundancy in the form of a secondary environment file on the same filesystem.

---

 README                       |  22 ++++++++
 cmd/nvedit.c                 |   1 +
 common/Makefile              |   1 +
 common/env_fs.c              | 126 +++++++++++++++++++++++++++++++++++++++++++
 include/environment.h        |  16 ++++++
 scripts/config_whitelist.txt |   5 ++
 6 files changed, 171 insertions(+)
 create mode 100644 common/env_fs.c
README +22 -0
index a95348a..ac0dfc2 100644
@@ -4094,6 +4094,28 @@ but it can not erase, write this NOR flash by SRIO or PCIE interface.
 	  You will probably want to define these to avoid a really noisy system
 	  when storing the env in UBI.
 
+- CONFIG_ENV_IS_IN_FS:
+
+	Define this if you want to store the environment inside a filesystem.
+
+	- CONFIG_ENV_FS_INTERFACE:
+
+	  Define this to the interface to the target environment filesystem.
+
+	- CONFIG_ENV_FS_DEVICE_AND_PART:
+
+	  Define this to the device and partition of the target environment,
+	  using standard syntax (see FAT_ENV_DEVICE_AND_PART).
+
+	- CONFIG_ENV_FS_FILE:
+
+	  Define this to the filename of the environment file.
+
+	- CONFIG_ENV_FS_FILE_REDUND:
+
+	  Define this to the filename of the alternate environment file,
+	  if redundancy is desired.
+
 - CONFIG_ENV_IS_IN_FAT:
        Define this if you want to use the FAT file system for the environment.
 
cmd/nvedit.c +1 -0
index 9ca5cb5..cd407b9 100644
@@ -53,6 +53,7 @@ DECLARE_GLOBAL_DATA_PTR;
 	!defined(CONFIG_ENV_IS_IN_SPI_FLASH)	&& \
 	!defined(CONFIG_ENV_IS_IN_REMOTE)	&& \
 	!defined(CONFIG_ENV_IS_IN_UBI)		&& \
+	!defined(CONFIG_ENV_IS_IN_FS)		&& \
 	!defined(CONFIG_ENV_IS_NOWHERE)
 # error Define one of CONFIG_ENV_IS_IN_{EEPROM|FLASH|DATAFLASH|ONENAND|\
 SATA|SPI_FLASH|NVRAM|MMC|FAT|EXT4|REMOTE|UBI} or CONFIG_ENV_IS_NOWHERE
common/Makefile +1 -0
index ecc23e6..1705774 100644
@@ -42,6 +42,7 @@ obj-$(CONFIG_ENV_IS_IN_NVRAM) += env_embedded.o
 obj-$(CONFIG_ENV_IS_IN_FLASH) += env_flash.o
 obj-$(CONFIG_ENV_IS_IN_MMC) += env_mmc.o
 obj-$(CONFIG_ENV_IS_IN_FAT) += env_fat.o
+obj-$(CONFIG_ENV_IS_IN_FS) += env_fs.o
 obj-$(CONFIG_ENV_IS_IN_EXT4) += env_ext4.o
 obj-$(CONFIG_ENV_IS_IN_NAND) += env_nand.o
 obj-$(CONFIG_ENV_IS_IN_NVRAM) += env_nvram.o
common/env_fs.c new file +126 -0

126 lines not on this page · raw message

include/environment.h +16 -0
index 0c718eb..d4765d9 100644
@@ -103,6 +103,22 @@ extern unsigned long nand_env_oob_offset;
 # endif
 #endif /* CONFIG_ENV_IS_IN_UBI */
 
+#if defined(CONFIG_ENV_IS_IN_FS)
+# ifndef CONFIG_ENV_FS_INTERFACE
+#  error "Need to define CONFIG_ENV_FS_INTERFACE when using CONFIG_ENV_IS_IN_FS"
+# endif
+# ifndef CONFIG_ENV_FS_DEVICE_AND_PART
+#  error "Need to define CONFIG_ENV_FS_DEVICE_AND_PART when using "
+#  error "CONFIG_ENV_IS_IN_FS"
+# endif
+# ifndef CONFIG_ENV_FS_FILE
+#  error "Need to define CONFIG_ENV_FS_FILE when using CONFIG_ENV_IS_IN_FS"
+# endif
+# ifdef CONFIG_ENV_FS_FILE_REDUND
+#  define CONFIG_SYS_REDUNDAND_ENVIRONMENT
+# endif
+#endif /* CONFIG_ENV_IS_IN_FS */
+
 /* Embedded env is only supported for some flash types */
 #ifdef CONFIG_ENV_IS_EMBEDDED
 # if	!defined(CONFIG_ENV_IS_IN_FLASH)	&& \
scripts/config_whitelist.txt +5 -0
index fb9fb34..4c52ee0 100644
@@ -976,6 +976,10 @@ CONFIG_ENV_FIT_UCBOOT
 CONFIG_ENV_FLAGS_LIST_DEFAULT
 CONFIG_ENV_FLAGS_LIST_STATIC
 CONFIG_ENV_FLASHBOOT
+CONFIG_ENV_FS_DEVICE_AND_PART
+CONFIG_ENV_FS_FILE
+CONFIG_ENV_FS_FILE_REDUND
+CONFIG_ENV_FS_INTERFACE
 CONFIG_ENV_IS_EMBEDDED
 CONFIG_ENV_IS_EMBEDDED_IN_LDR
 CONFIG_ENV_IS_IN_
@@ -983,6 +987,7 @@ CONFIG_ENV_IS_IN_DATAFLASH
 CONFIG_ENV_IS_IN_EEPROM
 CONFIG_ENV_IS_IN_FAT
 CONFIG_ENV_IS_IN_FLASH
+CONFIG_ENV_IS_IN_FS
 CONFIG_ENV_IS_IN_MMC
 CONFIG_ENV_IS_IN_MRAM
 CONFIG_ENV_IS_IN_NAND
Signature · 1 line
2.7.4
#8 lore raw

PATCH 3/3 env: Add support for FS environment

Quoted 9 lines · Fiach Antaw

This patch adds support for fat/ext4-style environments on top of the FS api, allowing any filesystem to be used to store the u-boot environment. This implementation also support redundancy in the form of a secondary environment file on the same filesystem.

---

Quoted 8 lines
 README                       |  22 ++++++++
 cmd/nvedit.c                 |   1 +
 common/Makefile              |   1 +
 common/env_fs.c              | 126 +++++++++++++++++++++++++++++++++++++++++++
 include/environment.h        |  16 ++++++
 scripts/config_whitelist.txt |   5 ++
 6 files changed, 171 insertions(+)
 create mode 100644 common/env_fs.c

Functionally, I like this idea, it's a step in the right direction. My first question is, did you test this on some real hardware that's using environment stored in a filesystem, on another media not listed here (say SATA?) ?

Second, it's really, really, not a good idea to add more Kconfig options to the whitelist. Can you please take a look at starting to move the env choices to Kconfig (I would assume ENV_IS_NOWHERE would be the easiest to move) ? Then we can add this there, thanks!

Signature · 8 lines
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: Digital signature
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20170128/43fde886/attachment.sig>