Skip to contentSkip to conversation

exfatprogs: remove sscanf(), add "make check" support

PATCH In review David Timber 2 participants 8 messages Aug 21

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

v2 cover exfatprogs: remove sscanf(), add "make check" support

continued from Github PR #386

Changes from v0:

- confirmed no behaviour change in exfat_parse_ulong() according to the x86 disassembly (other than saved_errno)
- boolean environment variables now accept non-decimal integer values (updated commit message clear up the confusion for reviewers)
- trailing garbage characters after an integer value in boolean environment variables are now rejected(this is the intended behaviour Suggested-by: Hyunchul Lee <[email protected]>)
- make check now runs test programs in lib/tests/

    (./configure --enable-tests not required)

- Run make check in Github Actions
- Add missing comma in configure.ac

The previous implementation of exfat_cmp_kernel_ver() misbehaves in some cases and this is why we should move away from *scanf(). The change in behaviour is intentional and correct.

There's no regression in exfat_cmp_kernel_ver() for Linux kernel version strings because the Linux kernel always returns the version in 3 numbers (make kernelversion). The new implementation is the one I originally intended. The test case should cover all the possible sane values for kernel version strings.

David Timber (2):
libexfat: remove use of sscanf()
treewide: run internal library tests in "make check"

 .github/workflows/c-cpp.yml |   2 +-
 configure.ac                |   2 +-
 include/libexfat.h          |  41 ++++++++++++-
 lib/Makefile.am             |  10 ++-
 lib/libexfat.c              | 117 ++++++++++++++++++++++++------------
 lib/tests/run-all.sh        |   3 +
 lib/tests/suite0001-parse.c | 101 +++++++++++++++++++++++++++++++
 7 files changed, 230 insertions(+), 46 deletions(-)
 create mode 100644 lib/tests/suite0001-parse.c
Signature · 1 line
2.55.0
#1 lore raw

PATCH v2 1/2 libexfat: remove use of sscanf()

Due to a bug inherent in the C programming language itself, the *scanf() functions are unsafe because they exhibit undefined behaviour in error cases. The recent patches introduced functions that use sscanf(), leading to some input sanitisation issues.

Remove the use of sscanf() by replacing it with strto*() functions. Add additional exfat_parse_*() functions using macros. exfat_parse_*() functions including exfat_parse_ulong() do not modify errno on success.

Add tests to refactor exfat_cmp_kernel_ver().

The environment variables that take boolean values are now properly validated before use. The boolean environment variables now accept non-decimal(%d) integer values.

---
 include/libexfat.h          |  41 ++++++++++++-
 lib/Makefile.am             |   5 +-
 lib/libexfat.c              | 117 ++++++++++++++++++++++++------------
 lib/tests/suite0001-parse.c | 101 +++++++++++++++++++++++++++++++
 4 files changed, 222 insertions(+), 42 deletions(-)
 create mode 100644 lib/tests/suite0001-parse.c
include/libexfat.h +40 -1
index b343621..effa13b 100644
@@ -317,7 +317,46 @@ int exfat_o2c(struct exfat *exfat, off_t device_offset,
 bool exfat_heap_clus(struct exfat *exfat, clus_t clus);
 int exfat_root_clus_count(struct exfat *exfat);
 int read_boot_sect(struct exfat_blk_dev *bdev, struct pbr **bs);
-int exfat_parse_ulong(const char *s, unsigned long *out);
+
+#define DECLARE_PARSE_NUM(OUT_T, NAME) int NAME(const char *s, OUT_T *out)
+DECLARE_PARSE_NUM(unsigned long, exfat_parse_ulong);
+DECLARE_PARSE_NUM(unsigned long long, exfat_parse_ulonglong);
+DECLARE_PARSE_NUM(long, exfat_parse_long);
+DECLARE_PARSE_NUM(long long, exfat_parse_longlong);
+#undef DECLARE_PARSE_NUM
+
+/*
+ * Parse a software version string
+ *
+ * Process a string until it encounters an invalid character. If at least one
+ * valid version number is parsed, output with the numbers parsed. Otherwise,
+ * return -1 and errno is set to EINVAL.
+ *
+ * Note that the function was originally implemented to parse Linux kernel
+ * version strings, which can be followed by an arbitrary user-supplied string
+ * (aka. CONFIG_LOCALVERSION). Therefore, the function stops processing when it
+ * encounters the trailing garbage(local version) in the string.
+ *
+ * For example, following strings are all valid:
+ *
+ *   "6.1.157-android14-11-gbd23337e42e7-ab14791245":	6.1.157
+ *   "6.6.77-8.el10.altarch.aarch64+64k":		6.6.77
+ *   "7.0.8-200.fc44.x86_64":				7.0.8
+ *   "6.17.0-1019-aws":					6.17.0
+ *
+ * And including(although confusingly):
+ *
+ *   "2foo.6":		2.0.0
+ *   "2.6foo.3":	2.6.0
+ *   "2.":		2.0.0
+ *   "2.6.":		2.6.0
+ *
+ * The Linux kernel versions are always in 3 numbers. However, the function also
+ * accepts version strings with 1 and 2 numbers for future use and correctness.
+ */
+int exfat_parse_swver(const char *in, unsigned short *out);
+int exfat_cmp_swver(const unsigned short *a, const unsigned short *b);
+
 int exfat_check_name(__le16 *utf16_name, int len);
 /*
  * Read back from the target device to confirm the successful write.
lib/Makefile.am +4 -1
index cdef6ef..90c09c1 100644
@@ -5,9 +5,12 @@ libexfat_a_SOURCES = libexfat.c exfat_fs.c exfat_dir.c utils.c
 
 if ENABLE_TESTS
 
-noinst_PROGRAMS = tests/suite0000-bitmap
+noinst_PROGRAMS = tests/suite0000-bitmap tests/suite0001-parse
 
 tests_suite0000_bitmap_SOURCES = tests/suite0000-bitmap.c
 tests_suite0000_bitmap_LDADD = libexfat.a
 
+tests_suite0001_parse_SOURCES = tests/suite0001-parse.c
+tests_suite0001_parse_LDADD = libexfat.a
+
 endif
lib/libexfat.c +77 -40

145 lines not on this page · raw message

lib/tests/suite0001-parse.c new file +101 -0

101 lines not on this page · raw message

Signature · 1 line
2.55.0
maintainer
#2 lore raw

Re: [PATCH v2 1/2] libexfat: remove use of sscanf()

lib/libexfat.c static int get_intbool_envvar(const char *name) · line 172

Quoted 13 lines
lib/libexfat.c +2 -2
index ff0b25b..9cbb17f 100644
@@ -167,9 +167,9 @@ static int get_intbool_envvar(const char *name)
        const char *env = getenv(name);
 
        if (env != NULL) {
-               int v = -1;
+               long v = -1;
 
-               if (sscanf(env, "%d", &v) == 1)
+               if (exfat_parse_long(env, &v) == 0)

I have updated this line like the following one.

+ if (exfat_parse_long(env, &v) == 0 && v >= 0)

And I will apply this patch if you don't object to it. Thanks!

#3 lore raw

Re: [PATCH v2 1/2] libexfat: remove use of sscanf()

Quoted 11 lines · Namjae Jeon
Quoted 11 lines
lib/libexfat.c +1 -1
index ff0b25b..9cbb17f 100644
@@ -167,9 +167,9 @@ static int get_intbool_envvar(const char *name)
        const char *env = getenv(name);
 
        if (env != NULL) {
-               int v = -1;
+               long v = -1;
 
Quoted 9 lines

lib/libexfat.c static int get_intbool_envvar(const char *name) · line 172

Quoted 2 lines
lib/libexfat.c +1 -1
-               if (sscanf(env, "%d", &v) == 1)
+               if (exfat_parse_long(env, &v) == 0)

I have updated this line like the following one.

+ if (exfat_parse_long(env, &v) == 0 && v >= 0)

And I will apply this patch if you don't object to it. Thanks!

My intention was to accept negative values as well, but I guess it doesn't really matter.

All good!

#4 lore raw

PATCH v2 2/2 treewide: run internal library tests in "make check"

In reply to David Timber

The library tests were intended to be used internally for dev purposes, but users have suggested that they should be run when "make check" is run.

Hook up the test programs to the TESTS target. "./configure --enable-tests" now only hooks up the test programs to the noinst_PROGRAMS target. "./configure --enable-tests" is not required to run the test programs using "make check".

Fix stray "enable_tests=yes" output in ./configure script due to a missing comma.

Run make check in the "Test building exfatprogs on arbitrary directory" step in the Github Actions CI pipelines.

Please do not close #391 as the fsck test cases should be run as well. This is not an easy fix and further work is being done to achieve this.

---
 .github/workflows/c-cpp.yml | 2 +-
 configure.ac                | 2 +-
 lib/Makefile.am             | 7 ++++---
 lib/tests/run-all.sh        | 3 +++
 4 files changed, 9 insertions(+), 5 deletions(-)
.github/workflows/c-cpp.yml +1 -1
index d332f04..a95f420 100644
@@ -220,7 +220,7 @@ jobs:
         BUILD_DIR="$(pwd)"
         "${CONFIG_SCRIPT}"
         make -j$((`nproc`+1))
-        sudo make install
+        sudo make check install
         popd
 
         rm -rf "$BUILD_DIR"
configure.ac +1 -1
index ab6fb5d..99bad24 100644
@@ -52,7 +52,7 @@ PKG_CHECK_MODULES([BLKID], [blkid >= 2.20], [AC_DEFINE([HAVE_BLKID], [1], [Defin
 	[PKG_CHECK_MODULES([EXT2_BLKID], [blkid], [AC_DEFINE([HAVE_EXT2_BLKID], [1], [Define if libblkid is from libext2])])])
 
 AC_ARG_ENABLE([tests],
-	AS_HELP_STRING([--enable-tests], [build internal test programs])
+	AS_HELP_STRING([--enable-tests], [always build internal test programs (not just for "make check")]),
 	[enable_tests=yes]
 )
 AM_CONDITIONAL([ENABLE_TESTS], [test "x$enable_tests" = xyes])
lib/Makefile.am +4 -3
index 90c09c1..4e9a183 100644
@@ -3,9 +3,8 @@ noinst_LIBRARIES = libexfat.a
 
 libexfat_a_SOURCES = libexfat.c exfat_fs.c exfat_dir.c utils.c
 
-if ENABLE_TESTS
-
-noinst_PROGRAMS = tests/suite0000-bitmap tests/suite0001-parse
+check_PROGRAMS = tests/suite0000-bitmap tests/suite0001-parse
+TESTS = tests/suite0000-bitmap tests/suite0001-parse
 
 tests_suite0000_bitmap_SOURCES = tests/suite0000-bitmap.c
 tests_suite0000_bitmap_LDADD = libexfat.a
@@ -13,4 +12,6 @@ tests_suite0000_bitmap_LDADD = libexfat.a
 tests_suite0001_parse_SOURCES = tests/suite0001-parse.c
 tests_suite0001_parse_LDADD = libexfat.a
 
+if ENABLE_TESTS
+noinst_PROGRAMS = $(check_PROGRAMS)
 endif
lib/tests/run-all.sh +3 -0
index 86e08da..332199a 100755
@@ -1,4 +1,7 @@
 #!/bin/sh
+# Note: this is only a convenience script for running test suites with Valgrind.
+# This is used in the CI pipelines(Github Actions). For normal cases, the tests
+# may be run with `make check` as well.
 
 main()
 {
Signature · 1 line
2.55.0
maintainer
#5 lore raw

Re: [PATCH v2 2/2] treewide: run internal library tests in "make check"

configure.ac PKG_CHECK_MODULES([BLKID], [blkid >= 2.20], [AC_DEFINE([HAVE_BLKID], [1], [Defin · line 56

Quoted 11 lines
configure.ac +1 -1
index ab6fb5d..99bad24 100644
@@ -52,7 +52,7 @@ PKG_CHECK_MODULES([BLKID], [blkid >= 2.20], [AC_DEFINE([HAVE_BLKID], [1], [Defin
        [PKG_CHECK_MODULES([EXT2_BLKID], [blkid], [AC_DEFINE([HAVE_EXT2_BLKID], [1], [Define if libblkid is from libext2])])])
 
 AC_ARG_ENABLE([tests],
-       AS_HELP_STRING([--enable-tests], [build internal test programs])
+       AS_HELP_STRING([--enable-tests], [always build internal test programs (not just for "make check")]),
        [enable_tests=yes]

I have updated it like this and will apply it to #exfat-next.

- AS_HELP_STRING([--enable-tests], [build internal test programs])
+ AS_HELP_STRING([--enable-tests], [always build internal test
programs (not just for "make check")]),

-       [enable_tests=yes]
+       [enable_tests=$enableval],
+       [enable_tests=no]

Thanks!

#6 lore raw

Re: [PATCH v2 2/2] treewide: run internal library tests in "make check"

Quoted 20 lines · Namjae Jeon

configure.ac PKG_CHECK_MODULES([BLKID], [blkid >= 2.20], [AC_DEFINE([HAVE_BLKID], [1], [Defin · line 56

Quoted 11 lines
configure.ac +1 -1
index ab6fb5d..99bad24 100644
@@ -52,7 +52,7 @@ PKG_CHECK_MODULES([BLKID], [blkid >= 2.20], [AC_DEFINE([HAVE_BLKID], [1], [Defin
        [PKG_CHECK_MODULES([EXT2_BLKID], [blkid], [AC_DEFINE([HAVE_EXT2_BLKID], [1], [Define if libblkid is from libext2])])])
 
 AC_ARG_ENABLE([tests],
-       AS_HELP_STRING([--enable-tests], [build internal test programs])
+       AS_HELP_STRING([--enable-tests], [always build internal test programs (not just for "make check")]),
        [enable_tests=yes]

I have updated it like this and will apply it to #exfat-next.

(quoted diff) +1 -1
-       AS_HELP_STRING([--enable-tests], [build internal test programs])
+       AS_HELP_STRING([--enable-tests], [always build internal test

programs (not just for "make check")]),

(quoted diff) +2 -1
-       [enable_tests=yes]
+       [enable_tests=$enableval],
+       [enable_tests=no]
Quoted 1 line

Thanks!

Just a quick question: is this a real problem where "enable_tests" is set to "yes" before running ./configure? I don't object to explicitly specifying [enable_tests=no] as a common pattern going forward. I'm just curious if there's any reasoning behind this.

Davo

maintainer
#7 lore raw

Re: [PATCH v2 2/2] treewide: run internal library tests in "make check"

Quoted 22 lines · David Timber
Quoted 20 lines · Namjae Jeon

configure.ac PKG_CHECK_MODULES([BLKID], [blkid >= 2.20], [AC_DEFINE([HAVE_BLKID], [1], [Defin · line 56

Quoted 11 lines
configure.ac +1 -1
index ab6fb5d..99bad24 100644
@@ -52,7 +52,7 @@ PKG_CHECK_MODULES([BLKID], [blkid >= 2.20], [AC_DEFINE([HAVE_BLKID], [1], [Defin
        [PKG_CHECK_MODULES([EXT2_BLKID], [blkid], [AC_DEFINE([HAVE_EXT2_BLKID], [1], [Define if libblkid is from libext2])])])
 
 AC_ARG_ENABLE([tests],
-       AS_HELP_STRING([--enable-tests], [build internal test programs])
+       AS_HELP_STRING([--enable-tests], [always build internal test programs (not just for "make check")]),
        [enable_tests=yes]

I have updated it like this and will apply it to #exfat-next.

(quoted diff) +1 -1
-       AS_HELP_STRING([--enable-tests], [build internal test programs])
+       AS_HELP_STRING([--enable-tests], [always build internal test

programs (not just for "make check")]),

(quoted diff) +2 -1
-       [enable_tests=yes]
+       [enable_tests=$enableval],
+       [enable_tests=no]
Quoted 5 lines
Quoted 1 line

Thanks!

Just a quick question: is this a real problem where "enable_tests" is set to "yes" before running ./configure? I don't object to explicitly specifying [enable_tests=no] as a common pattern going forward. I'm just curious if there's any reasoning behind this.

[enable_tests=no] was not intended to fix an actual bug. I added it to make the default explicit when no option is specified and to follow the same pattern used by the iostats option below. The functionally important change is [enable_tests=$enableval].

Thanks.