Skip to contentSkip to conversation

Add support for the SquashFS filesystem

PATCH New Joao Marcos Costa 2 participants 16 messages Jul 30, 2020 – Aug 10, 2020

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

v4 cover Add support for the SquashFS filesystem

Hello!

This series adds support for the SquashFS filesystem. For now, zlib is the only supported compression type. This is my first contribution to U-Boot as well as to a major Open Source project.

Changes in v4:
- Add tests for the SquashFS commands
- Add missing endianness convertions
- Improve pointers management
Changes in v3:
- Replace CONFIG_IS_ENABLED by IS_ENABLED in fs/fs.c
Changes in v2:
- Replace sqfs_ls() by U-Boot's fs_ls_generic()
- Add info. to MAINTAINERS
- Fix build failures
- Fix style problems

Joao Marcos Costa (6):
fs/squashfs: new filesystem
fs/squashfs: add filesystem commands include/u-boot, lib/zlib: add sources for zlib decompression fs/squashfs: add support for zlib decompression fs/fs.c: add symbolic link case to fs_ls_generic() test/py: Add tests for the SquashFS commands

 MAINTAINERS                                   |    8 +
 cmd/Kconfig                                   |    6 +
 cmd/Makefile                                  |    1 +
 cmd/sqfs.c                                    |   42 +
 common/spl/Kconfig                            |    9 +
 configs/sandbox_defconfig                     |    1 +
 fs/Kconfig                                    |    2 +
 fs/Makefile                                   |    2 +
 fs/fs.c                                       |   18 +
 fs/squashfs/Kconfig                           |   11 +
 fs/squashfs/Makefile                          |    7 +
 fs/squashfs/sqfs.c                            | 1538 +++++++++++++++++
 fs/squashfs/sqfs_decompressor.c               |   59 +
 fs/squashfs/sqfs_decompressor.h               |   58 +
 fs/squashfs/sqfs_dir.c                        |   91 +
 fs/squashfs/sqfs_filesystem.h                 |  300 ++++
 fs/squashfs/sqfs_inode.c                      |  155 ++
 fs/squashfs/sqfs_utils.h                      |   49 +
 include/fs.h                                  |    1 +
 include/squashfs.h                            |   25 +
 include/u-boot/zlib.h                         |   32 +
 lib/Kconfig                                   |    7 +
 lib/zlib/uncompr.c                            |   97 ++
 lib/zlib/zlib.c                               |    3 +
 .../test_fs/test_squashfs/sqfs_common.py      |   42 +
 .../test_fs/test_squashfs/test_sqfs_load.py   |   33 +
 .../test_fs/test_squashfs/test_sqfs_ls.py     |   26 +
 27 files changed, 2623 insertions(+)
 create mode 100644 cmd/sqfs.c
 create mode 100644 fs/squashfs/Kconfig
 create mode 100644 fs/squashfs/Makefile
 create mode 100644 fs/squashfs/sqfs.c
 create mode 100644 fs/squashfs/sqfs_decompressor.c
 create mode 100644 fs/squashfs/sqfs_decompressor.h
 create mode 100644 fs/squashfs/sqfs_dir.c
 create mode 100644 fs/squashfs/sqfs_filesystem.h
 create mode 100644 fs/squashfs/sqfs_inode.c
 create mode 100644 fs/squashfs/sqfs_utils.h
 create mode 100644 include/squashfs.h
 create mode 100644 lib/zlib/uncompr.c
 create mode 100644 test/py/tests/test_fs/test_squashfs/sqfs_common.py
 create mode 100644 test/py/tests/test_fs/test_squashfs/test_sqfs_load.py
 create mode 100644 test/py/tests/test_fs/test_squashfs/test_sqfs_ls.py
Signature · 1 line
2.17.1
#1 lore raw

PATCH v4 1/6 fs/squashfs: new filesystem

Add support for SquashFS filesystem. Right now, it does not support compression but support for zlib will be added in a follow-up commit.

---
Changes in v4:
- Fix issues that happenned while resolving symlinks
- Add missing endianness convertions
- Improve pointers management
Changes in v3:
- Replace CONFIG_IS_ENABLED by IS_ENABLED in fs/fs.c
Changes in v2:
- Add info. to MAINTAINERS
- Replace sqfs_ls() by U-Boot's fs_ls_generic()
- Improve string parsing when resolving symlink targets
- Replace cmd_tbl_t type by the cmd_tbl struct
- Add missing <command.h> in cmd/sqfs.c
- Fix build failures
- Replace "struct disk_partition_t" by "disk_partition"
- Remove CMD_RET* from the filesystem code
- Simplify superblock handling by adding "struct squashfs_ctxt", which encloses the complete global context (superblock, cur_dev and cur_part_info). The struct members are set up during probe() and cleaned up during close().

 MAINTAINERS                     |    6 +
 common/spl/Kconfig              |    9 +
 fs/Kconfig                      |    2 +
 fs/Makefile                     |    2 +
 fs/fs.c                         |   15 +
 fs/squashfs/Kconfig             |   10 +
 fs/squashfs/Makefile            |    7 +
 fs/squashfs/sqfs.c              | 1538 +++++++++++++++++++++++++++++++
 fs/squashfs/sqfs_decompressor.c |   29 +
 fs/squashfs/sqfs_decompressor.h |   58 ++
 fs/squashfs/sqfs_dir.c          |   91 ++
 fs/squashfs/sqfs_filesystem.h   |  300 ++++++
 fs/squashfs/sqfs_inode.c        |  155 ++++
 fs/squashfs/sqfs_utils.h        |   49 +
 include/fs.h                    |    1 +
 include/squashfs.h              |   25 +
 16 files changed, 2297 insertions(+)
 create mode 100644 fs/squashfs/Kconfig
 create mode 100644 fs/squashfs/Makefile
 create mode 100644 fs/squashfs/sqfs.c
 create mode 100644 fs/squashfs/sqfs_decompressor.c
 create mode 100644 fs/squashfs/sqfs_decompressor.h
 create mode 100644 fs/squashfs/sqfs_dir.c
 create mode 100644 fs/squashfs/sqfs_filesystem.h
 create mode 100644 fs/squashfs/sqfs_inode.c
 create mode 100644 fs/squashfs/sqfs_utils.h
 create mode 100644 include/squashfs.h
MAINTAINERS +6 -0
index 889a73f15f..c2501e82f6 100644
@@ -969,6 +969,12 @@ S:	Maintained
 F:	drivers/spmi/
 F:	include/spmi/
 
+SQUASHFS
+M:	Joao Marcos Costa <[email protected]>
+S:	Maintained
+F:	fs/squashfs/
+F:	include/sqfs.h
+
 TARGET_BCMNS3
 M:	Bharat Gooty <[email protected]>
 M:	Rayagonda Kokatanur <[email protected]>
common/spl/Kconfig +9 -0
index 72c7165dc2..b93944daa1 100644
@@ -577,6 +577,15 @@ config SPL_FS_EXT4
 	  filesystem from within SPL. Support for the underlying block
 	  device (e.g. MMC or USB) must be enabled separately.
 
+config SPL_FS_SQUASHFS
+	bool "Support SquashFS filesystems"
+	select FS_SQUASHFS
+	help
+	  Enable support for SquashFS filesystems with SPL. This permits
+	  U-Boot (or Linux in Falcon mode) to be loaded from a SquashFS
+	  filesystem from within SPL. Support for the underlying block
+	  device (e.g. MMC or USB) must be enabled separately.
+
 config SPL_FS_FAT
 	bool "Support FAT filesystems"
 	select FS_FAT
fs/Kconfig +2 -0
index 1cb9831be8..620af7f044 100644
@@ -22,4 +22,6 @@ source "fs/cramfs/Kconfig"
 
 source "fs/yaffs2/Kconfig"
 
+source "fs/squashfs/Kconfig"
+
 endmenu
fs/Makefile +2 -0
index 42e669c40c..937cbcf6e8 100644
@@ -9,6 +9,7 @@ obj-$(CONFIG_FS_LOADER) += fs.o
 obj-$(CONFIG_SPL_FS_FAT) += fat/
 obj-$(CONFIG_SPL_FS_EXT4) += ext4/
 obj-$(CONFIG_SPL_FS_CBFS) += cbfs/
+obj-$(CONFIG_SPL_FS_SQUASHFS) += squashfs/
 else
 obj-y				+= fs.o
 
@@ -23,5 +24,6 @@ obj-$(CONFIG_SANDBOX) += sandbox/
 obj-$(CONFIG_CMD_UBIFS) += ubifs/
 obj-$(CONFIG_YAFFS2) += yaffs2/
 obj-$(CONFIG_CMD_ZFS) += zfs/
+obj-$(CONFIG_FS_SQUASHFS) += squashfs/
 endif
 obj-y += fs_internal.o
fs/fs.c +15 -0
index edd8adc21b..5b31a369f7 100644
@@ -22,6 +22,7 @@
 #include <div64.h>
 #include <linux/math64.h>
 #include <efi_loader.h>
+#include <squashfs.h>
 
 DECLARE_GLOBAL_DATA_PTR;
 
@@ -276,6 +277,20 @@ static struct fstype_info fstypes[] = {
 		.mkdir = fs_mkdir_unsupported,
 		.ln = fs_ln_unsupported,
 	},
+#endif
+#if IS_ENABLED(CONFIG_FS_SQUASHFS)
+	{
+		.fstype = FS_TYPE_SQUASHFS,
+		.name = "squashfs",
+		.probe = sqfs_probe,
+		.opendir = sqfs_opendir,
+		.readdir = sqfs_readdir,
+		.ls = fs_ls_generic,
+		.read = sqfs_read,
+		.size = sqfs_size,
+		.close = sqfs_close,
+		.closedir = sqfs_closedir,
+	},
 #endif
 	{
 		.fstype = FS_TYPE_ANY,
fs/squashfs/Kconfig new file +10 -0
new file mode 100644
index 0000000000..b9772e5619
@@ -0,0 +1,10 @@
+config FS_SQUASHFS
+	bool "Enable SquashFS filesystem support"
+	help
+	  This provides support for reading images from SquashFS filesystem.
+	  Squashfs is a compressed read-only filesystem for Linux.
+	  It uses zlib, lz4, lzo, or xz compression to compress files, inodes
+	  and directories. Squashfs is intended for general read-only
+	  filesystem use, for archival use (i.e. in cases where a .tar.gz file
+	  may be used), and in constrained block device/memory systems (e.g.
+	  embedded systems) where low overhead is needed.
fs/squashfs/Makefile new file +7 -0
new file mode 100644
index 0000000000..ba66ee821c
@@ -0,0 +1,7 @@
+# SPDX-License-Identifier: GPL-2.0+
+#
+
+obj-$(CONFIG_$(SPL_)FS_SQUASHFS) = sqfs.o \
+				sqfs_inode.o \
+				sqfs_dir.o \
+				sqfs_decompressor.o
fs/squashfs/sqfs.c new file +1538 -0

1,538 lines not on this page · raw message

fs/squashfs/sqfs_decompressor.c new file +29 -0
new file mode 100644
index 0000000000..a899a5704b
@@ -0,0 +1,29 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (C) 2020 Bootlin
+ *
+ * Author: Joao Marcos Costa <[email protected]>
+ */
+
+#include <errno.h>
+#include <stdint.h>
+#include <stdio.h>
+#include <stdlib.h>
+
+#include "sqfs_decompressor.h"
+#include "sqfs_filesystem.h"
+#include "sqfs_utils.h"
+
+int sqfs_decompress(u16 comp_type, void *dest, unsigned long *dest_len,
+		    void *source, u32 lenp)
+{
+	int ret = 0;
+
+	switch (comp_type) {
+	default:
+		printf("Error: unknown compression type.\n");
+		return -EINVAL;
+	}
+
+	return ret;
+}
fs/squashfs/sqfs_decompressor.h new file +58 -0

58 lines not on this page · raw message

fs/squashfs/sqfs_dir.c new file +91 -0

91 lines not on this page · raw message

fs/squashfs/sqfs_filesystem.h new file +300 -0

300 lines not on this page · raw message

fs/squashfs/sqfs_inode.c new file +155 -0

155 lines not on this page · raw message

fs/squashfs/sqfs_utils.h new file +49 -0

49 lines not on this page · raw message

include/fs.h +1 -0
index b08b1f40c5..0794b50d10 100644
@@ -15,6 +15,7 @@ struct cmd_tbl;
 #define FS_TYPE_SANDBOX	3
 #define FS_TYPE_UBIFS	4
 #define FS_TYPE_BTRFS	5
+#define FS_TYPE_SQUASHFS 6
 
 struct blk_desc;
 
include/squashfs.h new file +25 -0
new file mode 100644
index 0000000000..819cf8c2da
@@ -0,0 +1,25 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (C) 2020 Bootlin
+ *
+ * Author: Joao Marcos Costa <[email protected]>
+ *
+ * squashfs.h: SquashFS filesystem implementation.
+ */
+
+#ifndef _SQFS_H_
+#define _SQFS_H_
+
+struct disk_partition;
+
+int sqfs_opendir(const char *filename, struct fs_dir_stream **dirsp);
+int sqfs_readdir(struct fs_dir_stream *dirs, struct fs_dirent **dentp);
+int sqfs_probe(struct blk_desc *fs_dev_desc,
+	       struct disk_partition *fs_partition);
+int sqfs_read(const char *filename, void *buf, loff_t offset,
+	      loff_t len, loff_t *actread);
+int sqfs_size(const char *filename, loff_t *size);
+void sqfs_close(void);
+void sqfs_closedir(struct fs_dir_stream *dirs);
+
+#endif /* SQFS_H  */
Signature · 1 line
2.17.1
#2 lore raw

PATCH v4 1/6 fs/squashfs: new filesystem

Quoted 4 lines · Joao Marcos Costa

Add support for SquashFS filesystem. Right now, it does not support compression but support for zlib will be added in a follow-up commit.

Applied to u-boot/master, thanks!

Signature · 8 lines
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 659 bytes
Desc: not available
URL: <https://lists.denx.de/pipermail/u-boot/attachments/20200808/7e440598/attachment.sig>
#3 lore raw

PATCH v4 1/6 fs/squashfs: new filesystem

In reply to Joao Marcos Costa
Quoted 4 lines · Joao Marcos Costa

Add support for SquashFS filesystem. Right now, it does not support compression but support for zlib will be added in a follow-up commit.

Applied to u-boot/master, thanks!

Signature · 8 lines
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 659 bytes
Desc: not available
URL: <https://lists.denx.de/pipermail/u-boot/attachments/20200808/e3834c15/attachment.sig>
#4 lore raw

PATCH v4 2/6 fs/squashfs: add filesystem commands

In reply to Joao Marcos Costa

Add 'ls' and 'load' commands.

---
Changes in v4:
- No changes since v3.
Changes in v3:
- No changes since v2.
Changes in v2:
- Add info. to MAINTAINERS
- Add 'static' keyword to do_sqfs_*(). These functions' prototypes are no longer needed in include/fs.h.
- Change commit message.

 MAINTAINERS  |  1 +
 cmd/Kconfig  |  6 ++++++
 cmd/Makefile |  1 +
 cmd/sqfs.c   | 42 ++++++++++++++++++++++++++++++++++++++++++
 4 files changed, 50 insertions(+)
 create mode 100644 cmd/sqfs.c
MAINTAINERS +1 -0
index c2501e82f6..8ec6c5db81 100644
@@ -974,6 +974,7 @@ M:	Joao Marcos Costa <[email protected]>
 S:	Maintained
 F:	fs/squashfs/
 F:	include/sqfs.h
+F:	cmd/sqfs.c
 
 TARGET_BCMNS3
 M:	Bharat Gooty <[email protected]>
cmd/Kconfig +6 -0
index e2b0a4fbc0..697eba3dd7 100644
@@ -2060,6 +2060,12 @@ config CMD_FAT
 	help
 	  Support for the FAT fs
 
+config CMD_SQUASHFS
+	bool "SquashFS command support"
+	select FS_SQUASHFS
+	help
+	  Enables SquashFS filesystem commands (e.g. load, ls).
+
 config CMD_FS_GENERIC
 	bool "filesystem commands"
 	help
cmd/Makefile +1 -0
index 7952138dc2..ef2da1c928 100644
@@ -62,6 +62,7 @@ obj-$(CONFIG_CMD_EXT4) += ext4.o
 obj-$(CONFIG_CMD_EXT2) += ext2.o
 obj-$(CONFIG_CMD_FAT) += fat.o
 obj-$(CONFIG_CMD_FDT) += fdt.o
+obj-$(CONFIG_CMD_SQUASHFS) += sqfs.o
 obj-$(CONFIG_CMD_FLASH) += flash.o
 obj-$(CONFIG_CMD_FPGA) += fpga.o
 obj-$(CONFIG_CMD_FPGAD) += fpgad.o
cmd/sqfs.c new file +42 -0

42 lines not on this page · raw message

Signature · 1 line
2.17.1
#5 lore raw

PATCH v4 2/6 fs/squashfs: add filesystem commands

Quoted 3 lines · Joao Marcos Costa

Add 'ls' and 'load' commands.

Applied to u-boot/master, thanks!

Signature · 8 lines
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 659 bytes
Desc: not available
URL: <https://lists.denx.de/pipermail/u-boot/attachments/20200808/54a0e97e/attachment.sig>
#6 lore raw

PATCH v4 3/6 include/u-boot, lib/zlib: add sources for zlib decompression

In reply to Joao Marcos Costa

Add zlib (v1.2.11) uncompr() function to U-Boot. SquashFS depends on this function to decompress data from a raw disk image. The actual support for zlib into SquashFS sources will be added in a follow-up commit.

---
Changes in v4:
- Add new config ZLIB_UNCOMPRESS to link zlib's new functionality only if SquashFS support is enabled.
Changes in v3:
- No changes since v2.
Changes in v2:

	- Changed commit title and message, which were wrong and/or misleading
	in v1.
 fs/squashfs/Kconfig   |  1 +
 include/u-boot/zlib.h | 32 ++++++++++++++
 lib/Kconfig           |  7 ++++
 lib/zlib/uncompr.c    | 97 +++++++++++++++++++++++++++++++++++++++++++
 lib/zlib/zlib.c       |  3 ++
 5 files changed, 140 insertions(+)
 create mode 100644 lib/zlib/uncompr.c
fs/squashfs/Kconfig +1 -0
index b9772e5619..54ab1618f1 100644
@@ -1,5 +1,6 @@
 config FS_SQUASHFS
 	bool "Enable SquashFS filesystem support"
+	select ZLIB_UNCOMPRESS
 	help
 	  This provides support for reading images from SquashFS filesystem.
 	  Squashfs is a compressed read-only filesystem for Linux.
include/u-boot/zlib.h +32 -0
index e23ceb50ca..a33cc8780d 100644
@@ -110,6 +110,12 @@ extern "C" {
 #  define voidp                 z_voidp
 #endif
 
+#if defined(ZLIB_CONST) && !defined(z_const)
+#  define z_const const
+#else
+#  define z_const
+#endif
+
 #if defined(__MSDOS__) && !defined(MSDOS)
 #  define MSDOS
 #endif
@@ -710,6 +716,32 @@ ZEXTERN  uInt ZEXPORT crc32  OF((uInt crc, const Bytef *buf, uInt len));
      if (crc != original_crc) error();
 */
 
+ZEXTERN int ZEXPORT uncompress OF((Bytef *dest, uLongf *destLen,
+				   const Bytef *source, uLong sourceLen));
+/*
+     Decompresses the source buffer into the destination buffer.  sourceLen is
+   the byte length of the source buffer.  Upon entry, destLen is the total size
+   of the destination buffer, which must be large enough to hold the entire
+   uncompressed data.  (The size of the uncompressed data must have been saved
+   previously by the compressor and transmitted to the decompressor by some
+   mechanism outside the scope of this compression library.) Upon exit, destLen
+   is the actual size of the uncompressed data.
+
+     uncompress returns Z_OK if success, Z_MEM_ERROR if there was not
+   enough memory, Z_BUF_ERROR if there was not enough room in the output
+   buffer, or Z_DATA_ERROR if the input data was corrupted or incomplete.  In
+   the case where there is not enough room, uncompress() will fill the output
+   buffer with the uncompressed data up to that point.
+*/
+
+ZEXTERN int ZEXPORT uncompress2 OF((Bytef *dest, uLongf *destLen,
+				    const Bytef *source, uLong *sourceLen));
+/*
+     Same as uncompress, except that sourceLen is a pointer, where the
+   length of the source is *sourceLen.  On return, *sourceLen is the number of
+   source bytes consumed.
+*/
+
 ZEXTERN int ZEXPORT inflateInit2_ OF((z_streamp strm, int  windowBits,
                                       const char *version, int stream_size));
 #define inflateInit(strm) \
lib/Kconfig +7 -0
index 2142bd06e6..089348af73 100644
@@ -460,6 +460,13 @@ config GZIP
 	help
 	  This enables support for GZIP compression algorithm.
 
+config ZLIB_UNCOMPRESS
+	bool "Enables zlib's uncompress() functionality"
+	help
+	  This enables an extra zlib functionality: the uncompress() function,
+	  which decompresses data from a buffer into another, knowing their
+	  sizes. Unlike gunzip(), there is no header parsing.
+
 config GZIP_COMPRESSED
 	bool
 	select ZLIB
lib/zlib/uncompr.c new file +97 -0

97 lines not on this page · raw message

lib/zlib/zlib.c +3 -0
index 7e1570292c..90e05e7d4d 100644
@@ -30,3 +30,6 @@
 #include "inflate.c"
 #include "zutil.c"
 #include "adler32.c"
+#if IS_ENABLED(CONFIG_ZLIB_UNCOMPRESS)
+#include "uncompr.c"
+#endif
Signature · 1 line
2.17.1
#7 lore raw

PATCH v4 3/6 include/u-boot, lib/zlib: add sources for zlib decompression

Quoted 6 lines · Joao Marcos Costa

Add zlib (v1.2.11) uncompr() function to U-Boot. SquashFS depends on this function to decompress data from a raw disk image. The actual support for zlib into SquashFS sources will be added in a follow-up commit.

Applied to u-boot/master, thanks!

Signature · 8 lines
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 659 bytes
Desc: not available
URL: <https://lists.denx.de/pipermail/u-boot/attachments/20200808/088eda44/attachment.sig>
#8 lore raw

PATCH v4 4/6 fs/squashfs: add support for zlib decompression

In reply to Joao Marcos Costa

Add call to zlib's 'uncompress' function. Add function to display the right error message depending on the decompression's return value.

---
Changes in v4:
- Add preprocessor directives to check if a given compression option is enabled, starting by ZLIB. As long as other options get supported, their respective directives will be added as well.
Changes in v3:
- No changes since v2.
Changes in v2:
- No changes since v1.

 fs/squashfs/sqfs_decompressor.c | 30 ++++++++++++++++++++++++++++++
 1 file changed, 30 insertions(+)
fs/squashfs/sqfs_decompressor.c +30 -0
index a899a5704b..09ca6cf6d0 100644
@@ -9,17 +9,47 @@
 #include <stdint.h>
 #include <stdio.h>
 #include <stdlib.h>
+#if IS_ENABLED(CONFIG_ZLIB)
+#include <u-boot/zlib.h>
+#endif
 
 #include "sqfs_decompressor.h"
 #include "sqfs_filesystem.h"
 #include "sqfs_utils.h"
 
+#if IS_ENABLED(CONFIG_ZLIB)
+static void zlib_decompression_status(int ret)
+{
+	switch (ret) {
+	case Z_BUF_ERROR:
+		printf("Error: 'dest' buffer is not large enough.\n");
+		break;
+	case Z_DATA_ERROR:
+		printf("Error: corrupted compressed data.\n");
+		break;
+	case Z_MEM_ERROR:
+		printf("Error: insufficient memory.\n");
+		break;
+	}
+}
+#endif
+
 int sqfs_decompress(u16 comp_type, void *dest, unsigned long *dest_len,
 		    void *source, u32 lenp)
 {
 	int ret = 0;
 
 	switch (comp_type) {
+#if IS_ENABLED(CONFIG_ZLIB)
+	case SQFS_COMP_ZLIB:
+		ret = uncompress(dest, dest_len, source, lenp);
+		if (ret) {
+			zlib_decompression_status(ret);
+			return -EINVAL;
+		}
+
+		break;
+#endif
 	default:
 		printf("Error: unknown compression type.\n");
 		return -EINVAL;
Signature · 1 line
2.17.1
#9 lore raw

PATCH v4 4/6 fs/squashfs: add support for zlib decompression

Quoted 4 lines · Joao Marcos Costa

Add call to zlib's 'uncompress' function. Add function to display the right error message depending on the decompression's return value.

Applied to u-boot/master, thanks!

Signature · 8 lines
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 659 bytes
Desc: not available
URL: <https://lists.denx.de/pipermail/u-boot/attachments/20200808/e9d3e22c/attachment.sig>
#10 lore raw

PATCH v4 5/6 fs/fs.c: add symbolic link case to fs_ls_generic()

In reply to Joao Marcos Costa

Adds an 'else if' statement inside the loop to check for symbolic links.

---
Changes in v4:
- No changes since v3.
Changes in v3:
- No changes since v2.
Changes in v2:
- This patch was added in v2 and did not exist in the last version.

 fs/fs.c | 3 +++
 1 file changed, 3 insertions(+)
fs/fs.c +3 -0
index 5b31a369f7..17e4bc33f7 100644
@@ -59,6 +59,9 @@ static int fs_ls_generic(const char *dirname)
 		if (dent->type == FS_DT_DIR) {
 			printf("            %s/\n", dent->name);
 			ndirs++;
+		} else if (dent->type == FS_DT_LNK) {
+			printf("    <SYM>   %s\n", dent->name);
+			nfiles++;
 		} else {
 			printf(" %8lld   %s\n", dent->size, dent->name);
 			nfiles++;
Signature · 1 line
2.17.1
#11 lore raw

PATCH v4 5/6 fs/fs.c: add symbolic link case to fs_ls_generic()

Quoted 3 lines · Joao Marcos Costa

Adds an 'else if' statement inside the loop to check for symbolic links.

Applied to u-boot/master, thanks!

Signature · 8 lines
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 659 bytes
Desc: not available
URL: <https://lists.denx.de/pipermail/u-boot/attachments/20200808/9a608aaa/attachment.sig>
#12 lore raw

PATCH v4 6/6 test/py: Add tests for the SquashFS commands

In reply to Joao Marcos Costa

Add Python scripts to test 'ls' and 'load' commands. The scripts generate a SquashFS image and clean the directory after the assertions, or if an exception is raised.

---
Changes in v4:
- This patch was added in v4 and did not exist in the last version.

 MAINTAINERS                                   |  1 +
 configs/sandbox_defconfig                     |  1 +
 .../test_fs/test_squashfs/sqfs_common.py      | 42 +++++++++++++++++++
 .../test_fs/test_squashfs/test_sqfs_load.py   | 33 +++++++++++++++
 .../test_fs/test_squashfs/test_sqfs_ls.py     | 26 ++++++++++++
 5 files changed, 103 insertions(+)
 create mode 100644 test/py/tests/test_fs/test_squashfs/sqfs_common.py
 create mode 100644 test/py/tests/test_fs/test_squashfs/test_sqfs_load.py
 create mode 100644 test/py/tests/test_fs/test_squashfs/test_sqfs_ls.py
MAINTAINERS +1 -0
index 8ec6c5db81..b12074ff3f 100644
@@ -975,6 +975,7 @@ S:	Maintained
 F:	fs/squashfs/
 F:	include/sqfs.h
 F:	cmd/sqfs.c
+F:	test/py/tests/test_fs/test_squashfs/
 
 TARGET_BCMNS3
 M:	Bharat Gooty <[email protected]>
configs/sandbox_defconfig +1 -0
index 829056e9ce..4dd5f87c68 100644
@@ -84,6 +84,7 @@ CONFIG_CMD_TPM=y
 CONFIG_CMD_TPM_TEST=y
 CONFIG_CMD_BTRFS=y
 CONFIG_CMD_CBFS=y
+CONFIG_CMD_SQUASHFS=y
 CONFIG_CMD_CRAMFS=y
 CONFIG_CMD_EXT4_WRITE=y
 CONFIG_CMD_MTDPARTS=y
test/py/tests/test_fs/test_squashfs/sqfs_common.py new file +42 -0

42 lines not on this page · raw message

test/py/tests/test_fs/test_squashfs/test_sqfs_load.py new file +33 -0
new file mode 100644
index 0000000000..9b828fdf04
@@ -0,0 +1,33 @@
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (C) 2020 Bootlin
+# Author: Joao Marcos Costa <[email protected]>
+
+import os
+import pytest
+from sqfs_common import *
+
+ at pytest.mark.boardspec('sandbox')
+ at pytest.mark.buildconfigspec('cmd_fs_generic')
+ at pytest.mark.buildconfigspec('cmd_squashfs')
+ at pytest.mark.buildconfigspec('fs_squashfs')
+ at pytest.mark.requiredtool('mksquashfs')
+def test_sqfs_load(u_boot_console):
+    sqfs_generate_image()
+    command = "sqfsload host 0 $kernel_addr_r "
+    path = "test/py/tests/test_fs/test_squashfs/sqfs"
+
+    try:
+        output = u_boot_console.run_command("host bind 0 " + path)
+        output = u_boot_console.run_command(command + "xxx")
+        assert "File not found." in output
+        output = u_boot_console.run_command(command + "frag_only")
+        assert "100 bytes read in" in output
+        output = u_boot_console.run_command(command + "blks_frag")
+        assert "5100 bytes read in" in output
+        output = u_boot_console.run_command(command + "blks_only")
+        assert "4096 bytes read in" in output
+        output = u_boot_console.run_command(command + "sym")
+        assert "100 bytes read in" in output
+    except:
+        sqfs_clean()
+    sqfs_clean()
test/py/tests/test_fs/test_squashfs/test_sqfs_ls.py new file +26 -0
new file mode 100644
index 0000000000..dc31f1a50e
@@ -0,0 +1,26 @@
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (C) 2020 Bootlin
+# Author: Joao Marcos Costa <[email protected]>
+
+import os
+import pytest
+from sqfs_common import *
+
+ at pytest.mark.boardspec('sandbox')
+ at pytest.mark.buildconfigspec('cmd_fs_generic')
+ at pytest.mark.buildconfigspec('cmd_squashfs')
+ at pytest.mark.buildconfigspec('fs_squashfs')
+ at pytest.mark.requiredtool('mksquashfs')
+def test_sqfs_ls(u_boot_console):
+    sqfs_generate_image()
+    path = "test/py/tests/test_fs/test_squashfs/sqfs"
+    try:
+        output = u_boot_console.run_command("host bind 0 " + path)
+        output = u_boot_console.run_command("sqfsls host 0")
+        assert "4 file(s), 0 dir(s)" in output
+        assert "<SYM>   sym" in output
+        output = u_boot_console.run_command("sqfsls host 0 xxx")
+        assert "** Cannot find directory. **" in output
+    except:
+        sqfs_clean()
+    sqfs_clean()
Signature · 1 line
2.17.1
#13 lore raw

PATCH v4 6/6 test/py: Add tests for the SquashFS commands

Quoted 5 lines · Joao Marcos Costa

Add Python scripts to test 'ls' and 'load' commands. The scripts generate a SquashFS image and clean the directory after the assertions, or if an exception is raised.

Applied to u-boot/master, thanks!

Signature · 8 lines
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 659 bytes
Desc: not available
URL: <https://lists.denx.de/pipermail/u-boot/attachments/20200808/0f906ecc/attachment.sig>
#14 lore raw

PATCH v4 6/6 test/py: Add tests for the SquashFS commands

Quoted 9 lines · Tom Rini
Quoted 5 lines · Joao Marcos Costa

Add Python scripts to test 'ls' and 'load' commands. The scripts generate a SquashFS image and clean the directory after the assertions, or if an exception is raised.

Applied to u-boot/master, thanks!

But... CI got backed up and I missed a real failure. Over at
https://dev.azure.com/u-boot/u-boot/_build/results?buildId=1053&view=logs&j=50449d1b-398e-53ae-48fa-6bf338edeb51&t=97605dd2-f5a5-5dd7-2118-315ffdc8bcd6&l=558
the problem is:

Quoted 1 line

os.mkdir(src)

E PermissionError: [Errno 13] Permission denied: 'test/py/tests/test_fs/test_squashfs/sqfs_src/'

because our source directory is read-only. I believe you need to use cons.config.build_dir instead to get to our build directory. Both tests fail in this manner. Can you please confirm my suggestion fixes this? If you don't want to configure Azure support directly you can submit a PR against the github mirror and that in turn will trigger an Azure run you can follow. Thanks!

Quoted 3 lines
Signature · 2 lines
-- 
Tom
Signature · 8 lines
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 659 bytes
Desc: not available
URL: <https://lists.denx.de/pipermail/u-boot/attachments/20200808/7248bbec/attachment.sig>
#15 lore raw

PATCH v4 6/6 test/py: Add tests for the SquashFS commands

Hello!

Quoted 18 lines · On Sat, 8 Aug 2020 10:41:38 -0400 Tom Rini <[email protected]> wrote:
Quoted 9 lines · Tom Rini
Quoted 5 lines · Joao Marcos Costa

Add Python scripts to test 'ls' and 'load' commands. The scripts generate a SquashFS image and clean the directory after the assertions, or if an exception is raised.

Applied to u-boot/master, thanks!

But... CI got backed up and I missed a real failure. Over at
https://dev.azure.com/u-boot/u-boot/_build/results?buildId=1053&view=logs&j=50449d1b-398e-53ae-48fa-6bf338edeb51&t=97605dd2-f5a5-5dd7-2118-315ffdc8bcd6&l=558
the problem is:

Quoted 1 line

os.mkdir(src)

E PermissionError: [Errno 13] Permission denied:
'test/py/tests/test_fs/test_squashfs/sqfs_src/'

Quoted 6 lines

because our source directory is read-only. I believe you need to use cons.config.build_dir instead to get to our build directory. Both tests fail in this manner. Can you please confirm my suggestion fixes this? If you don't want to configure Azure support directly you can submit a PR against the github mirror and that in turn will trigger an Azure run you can follow. Thanks!

I submitted the PR that applies your suggestion and it really solves the problem. Here is the link to the log:
https://travis-ci.org/github/u-boot/u-boot/jobs/716488309#L1307

I will send the patch in a few minutes. Thanks!

Quoted 7 lines
Quoted 4 lines
Signature · 2 lines
-- 
Tom  

Joao Marcos