exfatprogs: remove sscanf(), add "make check" support
Subsystems (2)
v2 cover exfatprogs: remove sscanf(), add "make check" support
continued from Github PR #386
Changes from v0:
- confirmed no behaviour change in exfat_parse_ulong() according to the x86 disassembly (other than saved_errno)
- boolean environment variables now accept non-decimal integer values (updated commit message clear up the confusion for reviewers)
- trailing garbage characters after an integer value in boolean environment variables are now rejected(this is the intended behaviour Suggested-by: Hyunchul Lee <[email protected]>)
- make check now runs test programs in lib/tests/
(./configure --enable-tests not required)
- Run make check in Github Actions
- Add missing comma in configure.ac
The previous implementation of exfat_cmp_kernel_ver() misbehaves in some cases and this is why we should move away from *scanf(). The change in behaviour is intentional and correct.
There's no regression in exfat_cmp_kernel_ver() for Linux kernel version strings because the Linux kernel always returns the version in 3 numbers (make kernelversion). The new implementation is the one I originally intended. The test case should cover all the possible sane values for kernel version strings.
David Timber (2):
libexfat: remove use of sscanf()
treewide: run internal library tests in "make check"
.github/workflows/c-cpp.yml | 2 +- configure.ac | 2 +- include/libexfat.h | 41 ++++++++++++- lib/Makefile.am | 10 ++- lib/libexfat.c | 117 ++++++++++++++++++++++++------------ lib/tests/run-all.sh | 3 + lib/tests/suite0001-parse.c | 101 +++++++++++++++++++++++++++++++ 7 files changed, 230 insertions(+), 46 deletions(-) create mode 100644 lib/tests/suite0001-parse.c
Signature · 1 line
2.55.0
PATCH v2 1/2 libexfat: remove use of sscanf()
Due to a bug inherent in the C programming language itself, the *scanf() functions are unsafe because they exhibit undefined behaviour in error cases. The recent patches introduced functions that use sscanf(), leading to some input sanitisation issues.
Remove the use of sscanf() by replacing it with strto*() functions. Add additional exfat_parse_*() functions using macros. exfat_parse_*() functions including exfat_parse_ulong() do not modify errno on success.
Add tests to refactor exfat_cmp_kernel_ver().
The environment variables that take boolean values are now properly validated before use. The boolean environment variables now accept non-decimal(%d) integer values.
--- include/libexfat.h | 41 ++++++++++++- lib/Makefile.am | 5 +- lib/libexfat.c | 117 ++++++++++++++++++++++++------------ lib/tests/suite0001-parse.c | 101 +++++++++++++++++++++++++++++++ 4 files changed, 222 insertions(+), 42 deletions(-) create mode 100644 lib/tests/suite0001-parse.c
include/libexfat.h +40 -1
@@ -317,7 +317,46 @@ int exfat_o2c(struct exfat *exfat, off_t device_offset,bool exfat_heap_clus (struct exfat *exfat,clus_t clus);int exfat_root_clus_count (struct exfat *exfat);int read_boot_sect (struct exfat_blk_dev *bdev,struct pbr **bs); -int exfat_parse_ulong (const char *s,unsigned long *out); + +#define DECLARE_PARSE_NUM (OUT_T ,NAME ) int NAME(const char *s, OUT_T *out) +DECLARE_PARSE_NUM(unsigned long, exfat_parse_ulong); +DECLARE_PARSE_NUM(unsigned long long, exfat_parse_ulonglong); +DECLARE_PARSE_NUM (long, exfat_parse_long); +DECLARE_PARSE_NUM(long long, exfat_parse_longlong); +#undef DECLARE_PARSE_NUM + +/* +* Parse a software version string +* +* Process a string until it encounters an invalid character. If at least one +* valid version number is parsed, output with the numbers parsed. Otherwise, +* return -1 and errno is set to EINVAL. +* +* Note that the function was originally implemented to parse Linux kernel +* version strings, which can be followed by an arbitrary user-supplied string +* (aka. CONFIG_LOCALVERSION). Therefore, the function stops processing when it +* encounters the trailing garbage(local version) in the string. +* +* For example, following strings are all valid: +* +* "6.1.157-android14-11-gbd23337e42e7-ab14791245": 6.1.157 +* "6.6.77-8.el10.altarch.aarch64+64k": 6.6.77 +* "7.0.8-200.fc44.x86_64": 7.0.8 +* "6.17.0-1019-aws": 6.17.0 +* +* And including(although confusingly): +* +* "2foo.6": 2.0.0 +* "2.6foo.3": 2.6.0 +* "2.": 2.0.0 +* "2.6.": 2.6.0 +* +* The Linux kernel versions are always in 3 numbers. However, the function also +* accepts version strings with 1 and 2 numbers for future use and correctness. +*/ +int exfat_parse_swver (const char *in,unsigned short *out); +int exfat_cmp_swver (const unsigned short *a,const unsigned short *b); +int exfat_check_name (__le16 *utf16_name,int len);/* * Read back from the target device to confirm the successful write.
lib/Makefile.am +4 -1
@@ -5,9 +5,12 @@ libexfat_a_SOURCES = libexfat.c exfat_fs.c exfat_dir.c utils.c if ENABLE_TESTS -noinst_PROGRAMS = tests/suite0000-bitmap +noinst_PROGRAMS = tests/suite0000-bitmap tests/suite0001-parse tests_suite0000_bitmap_SOURCES = tests/suite0000-bitmap.c tests_suite0000_bitmap_LDADD = libexfat.a +tests_suite0001_parse_SOURCES = tests/suite0001-parse.c +tests_suite0001_parse_LDADD = libexfat.a + endif
lib/libexfat.c +77 -40
145 lines not on this page · raw message
lib/tests/suite0001-parse.c new file +101 -0
101 lines not on this page · raw message
Signature · 1 line
2.55.0
Namjae Jeon
maintainerRe: [PATCH v2 1/2] libexfat: remove use of sscanf()
lib/libexfat.c static int get_intbool_envvar(const char *name) · line 172
Quoted 13 lines
lib/libexfat.c +2
-2@@ -167,9 +167,9 @@ static int get_intbool_envvar(const char *name)const char *env =getenv (name);if (env !=NULL ) { -int v =-1 ; +long v =-1 ; -if (sscanf (env,"%d" , &v) ==1 ) +if (exfat_parse_long (env, &v) ==0 )
I have updated this line like the following one.
+ if (exfat_parse_long(env, &v) == 0 && v >= 0)
And I will apply this patch if you don't object to it. Thanks!
Re: [PATCH v2 1/2] libexfat: remove use of sscanf()
Quoted 11 lines · Namjae Jeon
Quoted 11 lines
lib/libexfat.c +1
-1@@ -167,9 +167,9 @@ static int get_intbool_envvar(const char *name)const char *env =getenv (name);if (env !=NULL ) { -int v =-1 ; +long v =-1 ;
Quoted 9 lines
lib/libexfat.c static int get_intbool_envvar(const char *name) · line 172
Quoted 2 lines
lib/libexfat.c +1
-1-if (sscanf (env,"%d" , &v) ==1 ) +if (exfat_parse_long (env, &v) ==0 )I have updated this line like the following one.
+ if (
exfat_parse_long(env, &v)== 0 && v >= 0)And I will apply this patch if you don't object to it. Thanks!
My intention was to accept negative values as well, but I guess it doesn't really matter.
All good!
PATCH v2 2/2 treewide: run internal library tests in "make check"
In reply to David TimberThe library tests were intended to be used internally for dev purposes, but users have suggested that they should be run when "make check" is run.
Hook up the test programs to the TESTS target. "./configure --enable-tests" now only hooks up the test programs to the noinst_PROGRAMS target. "./configure --enable-tests" is not required to run the test programs using "make check".
Fix stray "enable_tests=yes" output in ./configure script due to a missing comma.
Run make check in the "Test building exfatprogs on arbitrary directory" step in the Github Actions CI pipelines.
Please do not close #391 as the fsck test cases should be run as well. This is not an easy fix and further work is being done to achieve this.
--- .github/workflows/c-cpp.yml | 2 +- configure.ac | 2 +- lib/Makefile.am | 7 ++++--- lib/tests/run-all.sh | 3 +++ 4 files changed, 9 insertions(+), 5 deletions(-)
.github/workflows/c-cpp.yml +1 -1
@@ -220,7 +220,7 @@ jobs: BUILD_DIR="$(pwd)" "${CONFIG_SCRIPT}" make -j$((`nproc`+1)) - sudo make install + sudo make check install popd rm -rf "$BUILD_DIR"
configure.ac +1 -1
@@ -52,7 +52,7 @@ PKG_CHECK_MODULES([BLKID], [blkid >= 2.20], [AC_DEFINE([HAVE_BLKID], [1], [Defin [PKG_CHECK_MODULES([EXT2_BLKID], [blkid], [AC_DEFINE([HAVE_EXT2_BLKID], [1], [Define if libblkid is from libext2])])]) AC_ARG_ENABLE([tests], - AS_HELP_STRING([--enable-tests], [build internal test programs]) + AS_HELP_STRING([--enable-tests], [always build internal test programs (not just for "make check")]), [enable_tests=yes] ) AM_CONDITIONAL([ENABLE_TESTS], [test "x$enable_tests" = xyes])
lib/Makefile.am +4 -3
@@ -3,9 +3,8 @@ noinst_LIBRARIES = libexfat.a libexfat_a_SOURCES = libexfat.c exfat_fs.c exfat_dir.c utils.c -if ENABLE_TESTS - -noinst_PROGRAMS = tests/suite0000-bitmap tests/suite0001-parse +check_PROGRAMS = tests/suite0000-bitmap tests/suite0001-parse +TESTS = tests/suite0000-bitmap tests/suite0001-parse tests_suite0000_bitmap_SOURCES = tests/suite0000-bitmap.c tests_suite0000_bitmap_LDADD = libexfat.a
@@ -13,4 +12,6 @@ tests_suite0000_bitmap_LDADD = libexfat.a tests_suite0001_parse_SOURCES = tests/suite0001-parse.c tests_suite0001_parse_LDADD = libexfat.a +if ENABLE_TESTS +noinst_PROGRAMS = $(check_PROGRAMS) endif
lib/tests/run-all.sh +3 -0
@@ -1,4 +1,7 @@ #!/bin/sh +# Note: this is only a convenience script for running test suites with Valgrind. +# This is used in the CI pipelines(Github Actions). For normal cases, the tests +# may be run with `make check` as well. main() {
Signature · 1 line
2.55.0
Namjae Jeon
maintainerRe: [PATCH v2 2/2] treewide: run internal library tests in "make check"
configure.ac PKG_CHECK_MODULES([BLKID], [blkid >= 2.20], [AC_DEFINE([HAVE_BLKID], [1], [Defin · line 56
Quoted 11 lines
configure.ac +1
-1@@ -52,7 +52,7 @@ PKG_CHECK_MODULES([BLKID], [blkid >= 2.20], [AC_DEFINE([HAVE_BLKID], [1], [Defin [PKG_CHECK_MODULES([EXT2_BLKID], [blkid], [AC_DEFINE([HAVE_EXT2_BLKID], [1], [Define if libblkid is from libext2])])]) AC_ARG_ENABLE([tests], - AS_HELP_STRING([--enable-tests], [build internal test programs]) + AS_HELP_STRING([--enable-tests], [always build internal test programs (not just for "make check")]), [enable_tests=yes]
I have updated it like this and will apply it to #exfat-next.
- AS_HELP_STRING([--enable-tests], [build internal test programs])
+ AS_HELP_STRING([--enable-tests], [always build internal test
programs (not just for "make check")]),
- [enable_tests=yes] + [enable_tests=$enableval], + [enable_tests=no]
Thanks!
Re: [PATCH v2 2/2] treewide: run internal library tests in "make check"
Quoted 20 lines · Namjae Jeon
configure.ac PKG_CHECK_MODULES([BLKID], [blkid >= 2.20], [AC_DEFINE([HAVE_BLKID], [1], [Defin · line 56
Quoted 11 lines
configure.ac +1
-1@@ -52,7 +52,7 @@ PKG_CHECK_MODULES([BLKID], [blkid >= 2.20], [AC_DEFINE([HAVE_BLKID], [1], [Defin [PKG_CHECK_MODULES([EXT2_BLKID], [blkid], [AC_DEFINE([HAVE_EXT2_BLKID], [1], [Define if libblkid is from libext2])])]) AC_ARG_ENABLE([tests], - AS_HELP_STRING([--enable-tests], [build internal test programs]) + AS_HELP_STRING([--enable-tests], [always build internal test programs (not just for "make check")]), [enable_tests=yes]I have updated it like this and will apply it to #exfat-next.
(quoted diff) +1
-1- AS_HELP_STRING([--enable-tests], [build internal test programs]) + AS_HELP_STRING([--enable-tests], [always build internal testprograms (not just for "make check")]),
(quoted diff) +2
-1- [enable_tests=yes] + [enable_tests=$enableval], + [enable_tests=no]
Quoted 1 line
Thanks!
Just a quick question: is this a real problem where "enable_tests" is set to "yes" before running ./configure? I don't object to explicitly specifying [enable_tests=no] as a common pattern going forward. I'm just curious if there's any reasoning behind this.
Davo
Namjae Jeon
maintainerRe: [PATCH v2 2/2] treewide: run internal library tests in "make check"
Quoted 22 lines · David Timber
Quoted 20 lines · Namjae Jeon
configure.ac PKG_CHECK_MODULES([BLKID], [blkid >= 2.20], [AC_DEFINE([HAVE_BLKID], [1], [Defin · line 56
Quoted 11 lines
configure.ac +1
-1@@ -52,7 +52,7 @@ PKG_CHECK_MODULES([BLKID], [blkid >= 2.20], [AC_DEFINE([HAVE_BLKID], [1], [Defin [PKG_CHECK_MODULES([EXT2_BLKID], [blkid], [AC_DEFINE([HAVE_EXT2_BLKID], [1], [Define if libblkid is from libext2])])]) AC_ARG_ENABLE([tests], - AS_HELP_STRING([--enable-tests], [build internal test programs]) + AS_HELP_STRING([--enable-tests], [always build internal test programs (not just for "make check")]), [enable_tests=yes]I have updated it like this and will apply it to #exfat-next.
(quoted diff) +1
-1- AS_HELP_STRING([--enable-tests], [build internal test programs]) + AS_HELP_STRING([--enable-tests], [always build internal testprograms (not just for "make check")]),
(quoted diff) +2
-1- [enable_tests=yes] + [enable_tests=$enableval], + [enable_tests=no]
Quoted 5 lines
Quoted 1 line
Thanks!
Just a quick question: is this a real problem where "
enable_tests" is set to "yes" before running ./configure? I don't object to explicitly specifying [enable_tests=no] as a common pattern going forward. I'm just curious if there's any reasoning behind this.
[enable_tests=no] was not intended to fix an actual bug. I added it to make the default explicit when no option is specified and to follow the same pattern used by the iostats option below. The functionally important change is [enable_tests=$enableval].
Thanks.