Skip to contentSkip to conversation

kunit: Support for Parameterized Testing

PATCH Superseded Arpitha Raghunandan 4 participants 6 messages Nov 15, 2020 – Nov 30, 2020

Subsystems (3)

v8 of 9 Superseded Latest is v9 →

lore
Share & tools
lore
#0 lore raw

PATCH v8 1/2 kunit: Support for Parameterized Testing

Implementation of support for parameterized testing in KUnit. This approach requires the creation of a test case using the KUNIT_CASE_PARAM() macro that accepts a generator function as input.

This generator function should return the next parameter given the previous parameter in parameterized tests. It also provides a macro to generate common-case generators based on arrays. Generators may also optionally provide a human-readable description of parameters, which is displayed where available.

Note, currently the result of each parameter run is displayed in diagnostic lines, and only the overall test case output summarizes TAP-compliant success or failure of all parameter runs. In future, when supported by kunit-tool, these can be turned into subsubtest outputs.

---
Changes v7->v8:
- Increase KUNIT_PARAM_DESC_SIZE to 128
- Format pointer style appropriately

Changes v6->v7:
- Clarify commit message.
- Introduce ability to optionally generate descriptions for parameters; if no description is provided, we'll still print 'param-N'.
- Change diagnostic line format to:
# <test-case-name>: <ok|not ok> N - [<param description>]

Changes v5->v6:
- Fix alignment to maintain consistency

Changes v4->v5:
- Update kernel-doc comments.
- Use const void* for generator return and prev value types.
- Add kernel-doc comment for KUNIT_ARRAY_PARAM.
- Rework parameterized test case execution strategy: each parameter is executed as if it was its own test case, with its own test initialization and cleanup (init and exit are called, etc.). However, we cannot add new test cases per TAP protocol once we have already started execution. Instead, log the result of each parameter run as a diagnostic comment.

Changes v3->v4:
- Rename kunit variables
- Rename generator function helper macro
- Add documentation for generator approach
- Display test case name in case of failure along with param index

Changes v2->v3:
- Modifictaion of generator macro and method

Changes v1->v2:
- Use of a generator method to access test case parameters Changes v6->v7:
- Clarify commit message.
- Introduce ability to optionally generate descriptions for parameters; if no description is provided, we'll still print 'param-N'.
- Change diagnostic line format to:
# <test-case-name>: <ok|not ok> N - [<param description>]
- Before execution of parameterized test case, count number of parameters and display number of parameters. Currently also as a diagnostic line, but this may be used in future to generate a subsubtest plan. A requirement of this change is that generators must generate a deterministic number of parameters.

Changes v5->v6:
- Fix alignment to maintain consistency

Changes v4->v5:
- Update kernel-doc comments.
- Use const void* for generator return and prev value types.
- Add kernel-doc comment for KUNIT_ARRAY_PARAM.
- Rework parameterized test case execution strategy: each parameter is executed as if it was its own test case, with its own test initialization and cleanup (init and exit are called, etc.). However, we cannot add new test cases per TAP protocol once we have already started execution. Instead, log the result of each parameter run as a diagnostic comment.

Changes v3->v4:
- Rename kunit variables
- Rename generator function helper macro
- Add documentation for generator approach
- Display test case name in case of failure along with param index

Changes v2->v3:
- Modifictaion of generator macro and method

Changes v1->v2:
- Use of a generator method to access test case parameters

 include/kunit/test.h | 51 ++++++++++++++++++++++++++++++++++++++
 lib/kunit/test.c     | 59 ++++++++++++++++++++++++++++++++++----------
 2 files changed, 97 insertions(+), 13 deletions(-)
include/kunit/test.h +51 -0
index db1b0ae666c4..27b42a008c7a 100644
@@ -94,6 +94,9 @@ struct kunit;
 /* Size of log associated with test. */
 #define KUNIT_LOG_SIZE	512
 
+/* Maximum size of parameter description string. */
+#define KUNIT_PARAM_DESC_SIZE 128
+
 /*
  * TAP specifies subtest stream indentation of 4 spaces, 8 spaces for a
  * sub-subtest.  See the "Subtests" section in
@@ -107,6 +110,7 @@ struct kunit;
  *
  * @run_case: the function representing the actual test case.
  * @name:     the name of the test case.
+ * @generate_params: the generator function for parameterized tests.
  *
  * A test case is a function with the signature,
  * ``void (*)(struct kunit *)``
@@ -141,6 +145,7 @@ struct kunit;
 struct kunit_case {
 	void (*run_case)(struct kunit *test);
 	const char *name;
+	const void* (*generate_params)(const void *prev, char *desc);
 
 	/* private: internal use only. */
 	bool success;
@@ -163,6 +168,27 @@ static inline char *kunit_status_to_string(bool status)
  */
 #define KUNIT_CASE(test_name) { .run_case = test_name, .name = #test_name }
 
+/**
+ * KUNIT_CASE_PARAM - A helper for creation a parameterized &struct kunit_case
+ *
+ * @test_name: a reference to a test case function.
+ * @gen_params: a reference to a parameter generator function.
+ *
+ * The generator function::
+ *
+ *	const void* gen_params(const void *prev, char *desc)
+ *
+ * is used to lazily generate a series of arbitrarily typed values that fit into
+ * a void*. The argument @prev is the previously returned value, which should be
+ * used to derive the next value; @prev is set to NULL on the initial generator
+ * call. When no more values are available, the generator must return NULL.
+ * Optionally write a string into @desc (size of KUNIT_PARAM_DESC_SIZE)
+ * describing the parameter.
+ */
+#define KUNIT_CASE_PARAM(test_name, gen_params)			\
+		{ .run_case = test_name, .name = #test_name,	\
+		  .generate_params = gen_params }
+
 /**
  * struct kunit_suite - describes a related collection of &struct kunit_case
  *
@@ -208,6 +234,10 @@ struct kunit {
 	const char *name; /* Read only after initialization! */
 	char *log; /* Points at case log after initialization */
 	struct kunit_try_catch try_catch;
+	/* param_value is the current parameter value for a test case. */
+	const void *param_value;
+	/* param_index stores the index of the parameter in parameterized tests. */
+	int param_index;
 	/*
 	 * success starts as true, and may only be set to false during a
 	 * test case; thus, it is safe to update this across multiple
@@ -1742,4 +1772,25 @@ do {									       \
 						fmt,			       \
 						##__VA_ARGS__)
 
+/**
+ * KUNIT_ARRAY_PARAM() - Define test parameter generator from an array.
+ * @name:  prefix for the test parameter generator function.
+ * @array: array of test parameters.
+ * @get_desc: function to convert param to description; NULL to use default
+ *
+ * Define function @name_gen_params which uses @array to generate parameters.
+ */
+#define KUNIT_ARRAY_PARAM(name, array, get_desc)						\
+	static const void *name##_gen_params(const void *prev, char *desc)			\
+	{											\
+		typeof((array)[0]) *__next = prev ? ((typeof(__next)) prev) + 1 : (array);	\
+		if (__next - (array) < ARRAY_SIZE((array))) {					\
+			void (*__get_desc)(typeof(__next), char *) = get_desc;			\
+			if (__get_desc)								\
+				__get_desc(__next, desc);					\
+			return __next;								\
+		}										\
+		return NULL;									\
+	}
+
 #endif /* _KUNIT_TEST_H */
lib/kunit/test.c +46 -13

85 lines not on this page · raw message

Signature · 1 line
2.25.1
#1 lore raw

PATCH v8 2/2 fs: ext4: Modify inode-test.c to use KUnit parameterized testing feature

Modify fs/ext4/inode-test.c to use the parameterized testing feature of KUnit.

---
Changes v7->v8:
- Replace strcpy() with strncpy() in timestamp_expectation_to_desc() Changes v6->v7:
- Introduce timestamp_expectation_to_desc() to convert param to description.
Changes v5->v6:
- No change to this patch of the patch series
Changes v4->v5:
- No change to this patch of the patch series
Changes v3->v4:
- Modification based on latest implementation of KUnit parameterized testing Changes v2->v3:
- Marked hardcoded test data const
- Modification based on latest implementation of KUnit parameterized testing Changes v1->v2:
- Modification based on latest implementation of KUnit parameterized testing

 fs/ext4/inode-test.c | 323 ++++++++++++++++++++++---------------------
 1 file changed, 167 insertions(+), 156 deletions(-)
fs/ext4/inode-test.c +167 -156
index d62d802c9c12..2c0c00c45c6b 100644
@@ -80,6 +80,148 @@ struct timestamp_expectation {
 	bool lower_bound;
 };
 
+static const struct timestamp_expectation test_data[] = {
+	{
+		.test_case_name = LOWER_BOUND_NEG_NO_EXTRA_BITS_CASE,
+		.msb_set = true,
+		.lower_bound = true,
+		.extra_bits = 0,
+		.expected = {.tv_sec = -0x80000000LL, .tv_nsec = 0L},
+	},
+
+	{
+		.test_case_name = UPPER_BOUND_NEG_NO_EXTRA_BITS_CASE,
+		.msb_set = true,
+		.lower_bound = false,
+		.extra_bits = 0,
+		.expected = {.tv_sec = -1LL, .tv_nsec = 0L},
+	},
+
+	{
+		.test_case_name = LOWER_BOUND_NONNEG_NO_EXTRA_BITS_CASE,
+		.msb_set = false,
+		.lower_bound = true,
+		.extra_bits = 0,
+		.expected = {0LL, 0L},
+	},
+
+	{
+		.test_case_name = UPPER_BOUND_NONNEG_NO_EXTRA_BITS_CASE,
+		.msb_set = false,
+		.lower_bound = false,
+		.extra_bits = 0,
+		.expected = {.tv_sec = 0x7fffffffLL, .tv_nsec = 0L},
+	},
+
+	{
+		.test_case_name = LOWER_BOUND_NEG_LO_1_CASE,
+		.msb_set = true,
+		.lower_bound = true,
+		.extra_bits = 1,
+		.expected = {.tv_sec = 0x80000000LL, .tv_nsec = 0L},
+	},
+
+	{
+		.test_case_name = UPPER_BOUND_NEG_LO_1_CASE,
+		.msb_set = true,
+		.lower_bound = false,
+		.extra_bits = 1,
+		.expected = {.tv_sec = 0xffffffffLL, .tv_nsec = 0L},
+	},
+
+	{
+		.test_case_name = LOWER_BOUND_NONNEG_LO_1_CASE,
+		.msb_set = false,
+		.lower_bound = true,
+		.extra_bits = 1,
+		.expected = {.tv_sec = 0x100000000LL, .tv_nsec = 0L},
+	},
+
+	{
+		.test_case_name = UPPER_BOUND_NONNEG_LO_1_CASE,
+		.msb_set = false,
+		.lower_bound = false,
+		.extra_bits = 1,
+		.expected = {.tv_sec = 0x17fffffffLL, .tv_nsec = 0L},
+	},
+
+	{
+		.test_case_name = LOWER_BOUND_NEG_HI_1_CASE,
+		.msb_set = true,
+		.lower_bound = true,
+		.extra_bits =  2,
+		.expected = {.tv_sec = 0x180000000LL, .tv_nsec = 0L},
+	},
+
+	{
+		.test_case_name = UPPER_BOUND_NEG_HI_1_CASE,
+		.msb_set = true,
+		.lower_bound = false,
+		.extra_bits = 2,
+		.expected = {.tv_sec = 0x1ffffffffLL, .tv_nsec = 0L},
+	},
+
+	{
+		.test_case_name = LOWER_BOUND_NONNEG_HI_1_CASE,
+		.msb_set = false,
+		.lower_bound = true,
+		.extra_bits = 2,
+		.expected = {.tv_sec = 0x200000000LL, .tv_nsec = 0L},
+	},
+
+	{
+		.test_case_name = UPPER_BOUND_NONNEG_HI_1_CASE,
+		.msb_set = false,
+		.lower_bound = false,
+		.extra_bits = 2,
+		.expected = {.tv_sec = 0x27fffffffLL, .tv_nsec = 0L},
+	},
+
+	{
+		.test_case_name = UPPER_BOUND_NONNEG_HI_1_NS_1_CASE,
+		.msb_set = false,
+		.lower_bound = false,
+		.extra_bits = 6,
+		.expected = {.tv_sec = 0x27fffffffLL, .tv_nsec = 1L},
+	},
+
+	{
+		.test_case_name = LOWER_BOUND_NONNEG_HI_1_NS_MAX_CASE,
+		.msb_set = false,
+		.lower_bound = true,
+		.extra_bits = 0xFFFFFFFF,
+		.expected = {.tv_sec = 0x300000000LL,
+			     .tv_nsec = MAX_NANOSECONDS},
+	},
+
+	{
+		.test_case_name = LOWER_BOUND_NONNEG_EXTRA_BITS_1_CASE,
+		.msb_set = false,
+		.lower_bound = true,
+		.extra_bits = 3,
+		.expected = {.tv_sec = 0x300000000LL, .tv_nsec = 0L},
+	},
+
+	{
+		.test_case_name = UPPER_BOUND_NONNEG_EXTRA_BITS_1_CASE,
+		.msb_set = false,
+		.lower_bound = false,
+		.extra_bits = 3,
+		.expected = {.tv_sec = 0x37fffffffLL, .tv_nsec = 0L},
+	}
+};
+
+static void timestamp_expectation_to_desc(const struct timestamp_expectation *t,
+					  char *desc)
+{
+	int desc_length = strlen(t->test_case_name);
+
+	strncpy(desc, t->test_case_name, desc_length);
+	desc[desc_length] = '\0';
+}
+
+KUNIT_ARRAY_PARAM(ext4_inode, test_data, timestamp_expectation_to_desc);
+
 static time64_t get_32bit_time(const struct timestamp_expectation * const test)
 {
 	if (test->msb_set) {
@@ -101,166 +243,35 @@ static time64_t get_32bit_time(const struct timestamp_expectation * const test)
  */
 static void inode_test_xtimestamp_decoding(struct kunit *test)
 {
-	const struct timestamp_expectation test_data[] = {
-		{
-			.test_case_name = LOWER_BOUND_NEG_NO_EXTRA_BITS_CASE,
-			.msb_set = true,
-			.lower_bound = true,
-			.extra_bits = 0,
-			.expected = {.tv_sec = -0x80000000LL, .tv_nsec = 0L},
-		},
-
-		{
-			.test_case_name = UPPER_BOUND_NEG_NO_EXTRA_BITS_CASE,
-			.msb_set = true,
-			.lower_bound = false,
-			.extra_bits = 0,
-			.expected = {.tv_sec = -1LL, .tv_nsec = 0L},
-		},
-
-		{
-			.test_case_name = LOWER_BOUND_NONNEG_NO_EXTRA_BITS_CASE,
-			.msb_set = false,
-			.lower_bound = true,
-			.extra_bits = 0,
-			.expected = {0LL, 0L},
-		},
-
-		{
-			.test_case_name = UPPER_BOUND_NONNEG_NO_EXTRA_BITS_CASE,
-			.msb_set = false,
-			.lower_bound = false,
-			.extra_bits = 0,
-			.expected = {.tv_sec = 0x7fffffffLL, .tv_nsec = 0L},
-		},
-
-		{
-			.test_case_name = LOWER_BOUND_NEG_LO_1_CASE,
-			.msb_set = true,
-			.lower_bound = true,
-			.extra_bits = 1,
-			.expected = {.tv_sec = 0x80000000LL, .tv_nsec = 0L},
-		},
-
-		{
-			.test_case_name = UPPER_BOUND_NEG_LO_1_CASE,
-			.msb_set = true,
-			.lower_bound = false,
-			.extra_bits = 1,
-			.expected = {.tv_sec = 0xffffffffLL, .tv_nsec = 0L},
-		},
-
-		{
-			.test_case_name = LOWER_BOUND_NONNEG_LO_1_CASE,
-			.msb_set = false,
-			.lower_bound = true,
-			.extra_bits = 1,
-			.expected = {.tv_sec = 0x100000000LL, .tv_nsec = 0L},
-		},
-
-		{
-			.test_case_name = UPPER_BOUND_NONNEG_LO_1_CASE,
-			.msb_set = false,
-			.lower_bound = false,
-			.extra_bits = 1,
-			.expected = {.tv_sec = 0x17fffffffLL, .tv_nsec = 0L},
-		},
-
-		{
-			.test_case_name = LOWER_BOUND_NEG_HI_1_CASE,
-			.msb_set = true,
-			.lower_bound = true,
-			.extra_bits =  2,
-			.expected = {.tv_sec = 0x180000000LL, .tv_nsec = 0L},
-		},
-
-		{
-			.test_case_name = UPPER_BOUND_NEG_HI_1_CASE,
-			.msb_set = true,
-			.lower_bound = false,
-			.extra_bits = 2,
-			.expected = {.tv_sec = 0x1ffffffffLL, .tv_nsec = 0L},
-		},
-
-		{
-			.test_case_name = LOWER_BOUND_NONNEG_HI_1_CASE,
-			.msb_set = false,
-			.lower_bound = true,
-			.extra_bits = 2,
-			.expected = {.tv_sec = 0x200000000LL, .tv_nsec = 0L},
-		},
-
-		{
-			.test_case_name = UPPER_BOUND_NONNEG_HI_1_CASE,
-			.msb_set = false,
-			.lower_bound = false,
-			.extra_bits = 2,
-			.expected = {.tv_sec = 0x27fffffffLL, .tv_nsec = 0L},
-		},
-
-		{
-			.test_case_name = UPPER_BOUND_NONNEG_HI_1_NS_1_CASE,
-			.msb_set = false,
-			.lower_bound = false,
-			.extra_bits = 6,
-			.expected = {.tv_sec = 0x27fffffffLL, .tv_nsec = 1L},
-		},
-
-		{
-			.test_case_name = LOWER_BOUND_NONNEG_HI_1_NS_MAX_CASE,
-			.msb_set = false,
-			.lower_bound = true,
-			.extra_bits = 0xFFFFFFFF,
-			.expected = {.tv_sec = 0x300000000LL,
-				     .tv_nsec = MAX_NANOSECONDS},
-		},
-
-		{
-			.test_case_name = LOWER_BOUND_NONNEG_EXTRA_BITS_1_CASE,
-			.msb_set = false,
-			.lower_bound = true,
-			.extra_bits = 3,
-			.expected = {.tv_sec = 0x300000000LL, .tv_nsec = 0L},
-		},
-
-		{
-			.test_case_name = UPPER_BOUND_NONNEG_EXTRA_BITS_1_CASE,
-			.msb_set = false,
-			.lower_bound = false,
-			.extra_bits = 3,
-			.expected = {.tv_sec = 0x37fffffffLL, .tv_nsec = 0L},
-		}
-	};
-
 	struct timespec64 timestamp;
-	int i;
-
-	for (i = 0; i < ARRAY_SIZE(test_data); ++i) {
-		timestamp.tv_sec = get_32bit_time(&test_data[i]);
-		ext4_decode_extra_time(&timestamp,
-				       cpu_to_le32(test_data[i].extra_bits));
-
-		KUNIT_EXPECT_EQ_MSG(test,
-				    test_data[i].expected.tv_sec,
-				    timestamp.tv_sec,
-				    CASE_NAME_FORMAT,
-				    test_data[i].test_case_name,
-				    test_data[i].msb_set,
-				    test_data[i].lower_bound,
-				    test_data[i].extra_bits);
-		KUNIT_EXPECT_EQ_MSG(test,
-				    test_data[i].expected.tv_nsec,
-				    timestamp.tv_nsec,
-				    CASE_NAME_FORMAT,
-				    test_data[i].test_case_name,
-				    test_data[i].msb_set,
-				    test_data[i].lower_bound,
-				    test_data[i].extra_bits);
-	}
+
+	struct timestamp_expectation *test_param =
+			(struct timestamp_expectation *)(test->param_value);
+
+	timestamp.tv_sec = get_32bit_time(test_param);
+	ext4_decode_extra_time(&timestamp,
+			       cpu_to_le32(test_param->extra_bits));
+
+	KUNIT_EXPECT_EQ_MSG(test,
+			    test_param->expected.tv_sec,
+			    timestamp.tv_sec,
+			    CASE_NAME_FORMAT,
+			    test_param->test_case_name,
+			    test_param->msb_set,
+			    test_param->lower_bound,
+			    test_param->extra_bits);
+	KUNIT_EXPECT_EQ_MSG(test,
+			    test_param->expected.tv_nsec,
+			    timestamp.tv_nsec,
+			    CASE_NAME_FORMAT,
+			    test_param->test_case_name,
+			    test_param->msb_set,
+			    test_param->lower_bound,
+			    test_param->extra_bits);
 }
 
 static struct kunit_case ext4_inode_test_cases[] = {
-	KUNIT_CASE(inode_test_xtimestamp_decoding),
+	KUNIT_CASE_PARAM(inode_test_xtimestamp_decoding, ext4_inode_gen_params),
 	{}
 };
 
Signature · 6 lines
2.25.1

_______________________________________________
Linux-kernel-mentees mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/linux-kernel-mentees
#2 lore raw

Re: [PATCH v8 2/2] fs: ext4: Modify inode-test.c to use KUnit parameterized testing feature

Quoted 164 lines · Arpitha Raghunandan

Modify fs/ext4/inode-test.c to use the parameterized testing feature of KUnit.

---
Changes v7->v8:
- Replace strcpy() with strncpy() in timestamp_expectation_to_desc() Changes v6->v7:
- Introduce timestamp_expectation_to_desc() to convert param to description.
Changes v5->v6:
- No change to this patch of the patch series
Changes v4->v5:
- No change to this patch of the patch series
Changes v3->v4:
- Modification based on latest implementation of KUnit parameterized testing Changes v2->v3:
- Marked hardcoded test data const
- Modification based on latest implementation of KUnit parameterized testing Changes v1->v2:
- Modification based on latest implementation of KUnit parameterized testing

 fs/ext4/inode-test.c | 323 ++++++++++++++++++++++---------------------
 1 file changed, 167 insertions(+), 156 deletions(-)
fs/ext4/inode-test.c +129 -0
index d62d802c9c12..2c0c00c45c6b 100644
@@ -80,6 +80,148 @@ struct timestamp_expectation {
        bool lower_bound;
 };
 
+static const struct timestamp_expectation test_data[] = {
+       {
+               .test_case_name = LOWER_BOUND_NEG_NO_EXTRA_BITS_CASE,
+               .msb_set = true,
+               .lower_bound = true,
+               .extra_bits = 0,
+               .expected = {.tv_sec = -0x80000000LL, .tv_nsec = 0L},
+       },
+
+       {
+               .test_case_name = UPPER_BOUND_NEG_NO_EXTRA_BITS_CASE,
+               .msb_set = true,
+               .lower_bound = false,
+               .extra_bits = 0,
+               .expected = {.tv_sec = -1LL, .tv_nsec = 0L},
+       },
+
+       {
+               .test_case_name = LOWER_BOUND_NONNEG_NO_EXTRA_BITS_CASE,
+               .msb_set = false,
+               .lower_bound = true,
+               .extra_bits = 0,
+               .expected = {0LL, 0L},
+       },
+
+       {
+               .test_case_name = UPPER_BOUND_NONNEG_NO_EXTRA_BITS_CASE,
+               .msb_set = false,
+               .lower_bound = false,
+               .extra_bits = 0,
+               .expected = {.tv_sec = 0x7fffffffLL, .tv_nsec = 0L},
+       },
+
+       {
+               .test_case_name = LOWER_BOUND_NEG_LO_1_CASE,
+               .msb_set = true,
+               .lower_bound = true,
+               .extra_bits = 1,
+               .expected = {.tv_sec = 0x80000000LL, .tv_nsec = 0L},
+       },
+
+       {
+               .test_case_name = UPPER_BOUND_NEG_LO_1_CASE,
+               .msb_set = true,
+               .lower_bound = false,
+               .extra_bits = 1,
+               .expected = {.tv_sec = 0xffffffffLL, .tv_nsec = 0L},
+       },
+
+       {
+               .test_case_name = LOWER_BOUND_NONNEG_LO_1_CASE,
+               .msb_set = false,
+               .lower_bound = true,
+               .extra_bits = 1,
+               .expected = {.tv_sec = 0x100000000LL, .tv_nsec = 0L},
+       },
+
+       {
+               .test_case_name = UPPER_BOUND_NONNEG_LO_1_CASE,
+               .msb_set = false,
+               .lower_bound = false,
+               .extra_bits = 1,
+               .expected = {.tv_sec = 0x17fffffffLL, .tv_nsec = 0L},
+       },
+
+       {
+               .test_case_name = LOWER_BOUND_NEG_HI_1_CASE,
+               .msb_set = true,
+               .lower_bound = true,
+               .extra_bits =  2,
+               .expected = {.tv_sec = 0x180000000LL, .tv_nsec = 0L},
+       },
+
+       {
+               .test_case_name = UPPER_BOUND_NEG_HI_1_CASE,
+               .msb_set = true,
+               .lower_bound = false,
+               .extra_bits = 2,
+               .expected = {.tv_sec = 0x1ffffffffLL, .tv_nsec = 0L},
+       },
+
+       {
+               .test_case_name = LOWER_BOUND_NONNEG_HI_1_CASE,
+               .msb_set = false,
+               .lower_bound = true,
+               .extra_bits = 2,
+               .expected = {.tv_sec = 0x200000000LL, .tv_nsec = 0L},
+       },
+
+       {
+               .test_case_name = UPPER_BOUND_NONNEG_HI_1_CASE,
+               .msb_set = false,
+               .lower_bound = false,
+               .extra_bits = 2,
+               .expected = {.tv_sec = 0x27fffffffLL, .tv_nsec = 0L},
+       },
+
+       {
+               .test_case_name = UPPER_BOUND_NONNEG_HI_1_NS_1_CASE,
+               .msb_set = false,
+               .lower_bound = false,
+               .extra_bits = 6,
+               .expected = {.tv_sec = 0x27fffffffLL, .tv_nsec = 1L},
+       },
+
+       {
+               .test_case_name = LOWER_BOUND_NONNEG_HI_1_NS_MAX_CASE,
+               .msb_set = false,
+               .lower_bound = true,
+               .extra_bits = 0xFFFFFFFF,
+               .expected = {.tv_sec = 0x300000000LL,
+                            .tv_nsec = MAX_NANOSECONDS},
+       },
+
+       {
+               .test_case_name = LOWER_BOUND_NONNEG_EXTRA_BITS_1_CASE,
+               .msb_set = false,
+               .lower_bound = true,
+               .extra_bits = 3,
+               .expected = {.tv_sec = 0x300000000LL, .tv_nsec = 0L},
+       },
+
+       {
+               .test_case_name = UPPER_BOUND_NONNEG_EXTRA_BITS_1_CASE,
+               .msb_set = false,
+               .lower_bound = false,
+               .extra_bits = 3,
+               .expected = {.tv_sec = 0x37fffffffLL, .tv_nsec = 0L},
+       }

fs/ext4/inode-test.c struct timestamp_expectation { · line 221

Quoted 10 lines
fs/ext4/inode-test.c +10 -0
+};
+
+static void timestamp_expectation_to_desc(const struct timestamp_expectation *t,
+                                         char *desc)
+{
+       int desc_length = strlen(t->test_case_name);
+
+       strncpy(desc, t->test_case_name, desc_length);
+       desc[desc_length] = '\0';
+}

This unfortunately won't prevent out-of-bounds accesses if the description is longer than KUNIT_PARAM_DESC_SIZE.

With strncpy() we want to avoid copying more bytes than the destination buffer can hold. This can be done by simply a strncpy(desc, t->test_case_name, KUNIT_PARAM_DESC_SIZE). But, strncpy() is unsafe in certain cases, too, so the kernel introduced strscpy() -- see the note about strncpy() in Documentation/process/deprecated.rst. Also have a look at the documentation about str{n,l,s}cpy() in lib/string.c.

So, finally, what we want here is just 1 line:

    strscpy(desc, t->test_case_name, KUNIT_PARAM_DESC_SIZE);

Patch 1/2 looks fine though, so hopefully v9 will be final. :-)

Thanks,
-- Marco
_______________________________________________
Linux-kernel-mentees mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/linux-kernel-mentees

#4 lore raw

Re: [PATCH v8 2/2] fs: ext4: Modify inode-test.c to use KUnit parameterized testing feature

In reply to Arpitha Raghunandan
Thank you for the patch! Perhaps something to improve:

Hi Arpitha,

Thank you for the patch! Perhaps something to improve:

[auto build test WARNING on ext4/dev]
[also build test WARNING on linus/master v5.10-rc3 next-20201113] [cannot apply to tytso-fscrypt/master]
[If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch]

url:    https://github.com/0day-ci/linux/commits/Arpitha-Raghunandan/kunit-Support-for-Parameterized-Testing/20201116-030137
base:   https://git.kernel.org/pub/scm/linux/kernel/git/tytso/ext4.git dev
config: microblaze-randconfig-r031-20201115 (attached as .config)
compiler: microblaze-linux-gcc (GCC) 9.3.0

reproduce (this is a W=1 build):
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross chmod +x ~/bin/make.cross
# https://github.com/0day-ci/linux/commit/002a7d4700bd033cceff2f2ab10bcc4a440f1377
git remote add linux-review https://github.com/0day-ci/linux
git fetch --no-tags linux-review Arpitha-Raghunandan/kunit-Support-for-Parameterized-Testing/20201116-030137 git checkout 002a7d4700bd033cceff2f2ab10bcc4a440f1377
# save the attached .config to linux build tree
COMPILER_INSTALL_PATH=$HOME/0day COMPILER=gcc-9.3.0 make.cross ARCH=microblaze

If you fix the issue, kindly add following tag as appropriate

All warnings (new ones prefixed by >>):

In function 'timestamp_expectation_to_desc',

       inlined from 'ext4_inode_gen_params' at fs/ext4/inode-test.c:223:1:
>> fs/ext4/inode-test.c:219:2: warning: 'strncpy' output truncated before terminating nul copying as many bytes from a string as its length [-Wstringop-truncation]
     219 |  strncpy(desc, t->test_case_name, desc_length);
         |  ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
   fs/ext4/inode-test.c: In function 'ext4_inode_gen_params':
   fs/ext4/inode-test.c:217:20: note: length computed here
     217 |  int desc_length = strlen(t->test_case_name);
         |                    ^~~~~~~~~~~~~~~~~~~~~~~~~

vim +/strncpy +219 fs/ext4/inode-test.c

213
214 static void timestamp_expectation_to_desc(const struct timestamp_expectation *t,
215 char *desc)
216 {
217 int desc_length = strlen(t->test_case_name);
218

 > 219		strncpy(desc, t->test_case_name, desc_length);

220 desc[desc_length] = '\0';
221 }
222

---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all(a)lists.01.org

#3 lore raw

Re: [PATCH v8 2/2] fs: ext4: Modify inode-test.c to use KUnit parameterized testing feature

In reply to Marco Elver
Quoted 187 lines · Marco Elver
Quoted 164 lines · Arpitha Raghunandan

Modify fs/ext4/inode-test.c to use the parameterized testing feature of KUnit.

---
Changes v7->v8:
- Replace strcpy() with strncpy() in timestamp_expectation_to_desc() Changes v6->v7:
- Introduce timestamp_expectation_to_desc() to convert param to description.
Changes v5->v6:
- No change to this patch of the patch series
Changes v4->v5:
- No change to this patch of the patch series
Changes v3->v4:
- Modification based on latest implementation of KUnit parameterized testing Changes v2->v3:
- Marked hardcoded test data const
- Modification based on latest implementation of KUnit parameterized testing Changes v1->v2:
- Modification based on latest implementation of KUnit parameterized testing

 fs/ext4/inode-test.c | 323 ++++++++++++++++++++++---------------------
 1 file changed, 167 insertions(+), 156 deletions(-)
fs/ext4/inode-test.c +129 -0
index d62d802c9c12..2c0c00c45c6b 100644
@@ -80,6 +80,148 @@ struct timestamp_expectation {
        bool lower_bound;
 };
 
+static const struct timestamp_expectation test_data[] = {
+       {
+               .test_case_name = LOWER_BOUND_NEG_NO_EXTRA_BITS_CASE,
+               .msb_set = true,
+               .lower_bound = true,
+               .extra_bits = 0,
+               .expected = {.tv_sec = -0x80000000LL, .tv_nsec = 0L},
+       },
+
+       {
+               .test_case_name = UPPER_BOUND_NEG_NO_EXTRA_BITS_CASE,
+               .msb_set = true,
+               .lower_bound = false,
+               .extra_bits = 0,
+               .expected = {.tv_sec = -1LL, .tv_nsec = 0L},
+       },
+
+       {
+               .test_case_name = LOWER_BOUND_NONNEG_NO_EXTRA_BITS_CASE,
+               .msb_set = false,
+               .lower_bound = true,
+               .extra_bits = 0,
+               .expected = {0LL, 0L},
+       },
+
+       {
+               .test_case_name = UPPER_BOUND_NONNEG_NO_EXTRA_BITS_CASE,
+               .msb_set = false,
+               .lower_bound = false,
+               .extra_bits = 0,
+               .expected = {.tv_sec = 0x7fffffffLL, .tv_nsec = 0L},
+       },
+
+       {
+               .test_case_name = LOWER_BOUND_NEG_LO_1_CASE,
+               .msb_set = true,
+               .lower_bound = true,
+               .extra_bits = 1,
+               .expected = {.tv_sec = 0x80000000LL, .tv_nsec = 0L},
+       },
+
+       {
+               .test_case_name = UPPER_BOUND_NEG_LO_1_CASE,
+               .msb_set = true,
+               .lower_bound = false,
+               .extra_bits = 1,
+               .expected = {.tv_sec = 0xffffffffLL, .tv_nsec = 0L},
+       },
+
+       {
+               .test_case_name = LOWER_BOUND_NONNEG_LO_1_CASE,
+               .msb_set = false,
+               .lower_bound = true,
+               .extra_bits = 1,
+               .expected = {.tv_sec = 0x100000000LL, .tv_nsec = 0L},
+       },
+
+       {
+               .test_case_name = UPPER_BOUND_NONNEG_LO_1_CASE,
+               .msb_set = false,
+               .lower_bound = false,
+               .extra_bits = 1,
+               .expected = {.tv_sec = 0x17fffffffLL, .tv_nsec = 0L},
+       },
+
+       {
+               .test_case_name = LOWER_BOUND_NEG_HI_1_CASE,
+               .msb_set = true,
+               .lower_bound = true,
+               .extra_bits =  2,
+               .expected = {.tv_sec = 0x180000000LL, .tv_nsec = 0L},
+       },
+
+       {
+               .test_case_name = UPPER_BOUND_NEG_HI_1_CASE,
+               .msb_set = true,
+               .lower_bound = false,
+               .extra_bits = 2,
+               .expected = {.tv_sec = 0x1ffffffffLL, .tv_nsec = 0L},
+       },
+
+       {
+               .test_case_name = LOWER_BOUND_NONNEG_HI_1_CASE,
+               .msb_set = false,
+               .lower_bound = true,
+               .extra_bits = 2,
+               .expected = {.tv_sec = 0x200000000LL, .tv_nsec = 0L},
+       },
+
+       {
+               .test_case_name = UPPER_BOUND_NONNEG_HI_1_CASE,
+               .msb_set = false,
+               .lower_bound = false,
+               .extra_bits = 2,
+               .expected = {.tv_sec = 0x27fffffffLL, .tv_nsec = 0L},
+       },
+
+       {
+               .test_case_name = UPPER_BOUND_NONNEG_HI_1_NS_1_CASE,
+               .msb_set = false,
+               .lower_bound = false,
+               .extra_bits = 6,
+               .expected = {.tv_sec = 0x27fffffffLL, .tv_nsec = 1L},
+       },
+
+       {
+               .test_case_name = LOWER_BOUND_NONNEG_HI_1_NS_MAX_CASE,
+               .msb_set = false,
+               .lower_bound = true,
+               .extra_bits = 0xFFFFFFFF,
+               .expected = {.tv_sec = 0x300000000LL,
+                            .tv_nsec = MAX_NANOSECONDS},
+       },
+
+       {
+               .test_case_name = LOWER_BOUND_NONNEG_EXTRA_BITS_1_CASE,
+               .msb_set = false,
+               .lower_bound = true,
+               .extra_bits = 3,
+               .expected = {.tv_sec = 0x300000000LL, .tv_nsec = 0L},
+       },
+
+       {
+               .test_case_name = UPPER_BOUND_NONNEG_EXTRA_BITS_1_CASE,
+               .msb_set = false,
+               .lower_bound = false,
+               .extra_bits = 3,
+               .expected = {.tv_sec = 0x37fffffffLL, .tv_nsec = 0L},
+       }

fs/ext4/inode-test.c struct timestamp_expectation { · line 221

Quoted 10 lines
fs/ext4/inode-test.c +10 -0
+};
+
+static void timestamp_expectation_to_desc(const struct timestamp_expectation *t,
+                                         char *desc)
+{
+       int desc_length = strlen(t->test_case_name);
+
+       strncpy(desc, t->test_case_name, desc_length);
+       desc[desc_length] = '\0';
+}

This unfortunately won't prevent out-of-bounds accesses if the description is longer than KUNIT_PARAM_DESC_SIZE.

With strncpy() we want to avoid copying more bytes than the destination buffer can hold. This can be done by simply a strncpy(desc, t->test_case_name, KUNIT_PARAM_DESC_SIZE). But, strncpy() is unsafe in certain cases, too, so the kernel introduced strscpy() -- see the note about strncpy() in Documentation/process/deprecated.rst. Also have a look at the documentation about str{n,l,s}cpy() in lib/string.c.

Quoted 4 lines

So, finally, what we want here is just 1 line:

    strscpy(desc, t->test_case_name, KUNIT_PARAM_DESC_SIZE);

Okay, I'll look this up and make this change for v9.

Quoted 5 lines

Patch 1/2 looks fine though, so hopefully v9 will be final. :-)

Thanks,
-- Marco

Thanks!

_______________________________________________
Linux-kernel-mentees mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/linux-kernel-mentees

Acked-by on v8 1/2
#5 lore raw
In reply to Arpitha Raghunandan
Quoted 11 lines · Arpitha Raghunandan

Implementation of support for parameterized testing in KUnit. This approach requires the creation of a test case using the KUNIT_CASE_PARAM() macro that accepts a generator function as input.

This generator function should return the next parameter given the previous parameter in parameterized tests. It also provides a macro to generate common-case generators based on arrays. Generators may also optionally provide a human-readable description of parameters, which is displayed where available.

Quoted 8 lines

Note, currently the result of each parameter run is displayed in diagnostic lines, and only the overall test case output summarizes TAP-compliant success or failure of all parameter runs. In future, when supported by kunit-tool, these can be turned into subsubtest outputs.

_______________________________________________
Linux-kernel-mentees mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/linux-kernel-mentees