Skip to contentSkip to conversation

perf test: Check test suite description only

PATCH Superseded Namhyung Kim 2 participants 9 messages Jun 30, 2025 – Jul 1, 2025 perf events

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore

Namhyung Kim

author maintainer
#0 lore raw

PATCH 1/3 perf test: Check test suite description only

Currently perf test checks the given string with descriptions for both test suites and cases (subtests). But sometimes it's confusing since the subtests don't contain the important keyword.

I think it's better to check the suite level and run the whole suite together.

Before:

  $ perf test hwmon
  (no output)

After:

  $ perf test hwmon
   10: Hwmon PMU                                                       :
   10.1: Basic parsing test                                            : Ok
   10.2: Parsing without PMU name                                      : Ok
   10.3: Parsing with PMU name                                         : Ok
---
 tools/perf/tests/builtin-test.c | 4 ----
 1 file changed, 4 deletions(-)
tools/perf/tests/builtin-test.c +0 -4
index 80375ca39a37a256..dfaff4185eb05a1a 100644
@@ -567,10 +567,6 @@ static int __cmd_test(struct test_suite **suites, int argc, const char *argv[],
 
 			for (unsigned int run = 0; run < runs_per_test; run++) {
 				test_suite__for_each_test_case(*t, curr_test_case) {
-					if (!perf_test__matches(test_description(*t, curr_test_case),
-								curr_suite, argc, argv))
-						continue;
-
 					err = start_test(*t, curr_suite, curr_test_case,
 							 &child_tests[child_test_num++],
 							 width, pass);
Signature · 1 line
2.50.0.727.gbf7dc18ff4-goog
author maintainer
#1 lore raw

PATCH 2/3 perf test: Add libsubcmd help tests

Add a set of tests for subcmd routines. Currently it fails the last one since there's a bug. It'll be fixed by the next commit.

  $ perf test subcmd
   69: libsubcmd help tests                                            :
   69.1: Load subcmd names                                             : Ok
   69.2: Uniquify subcmd names                                         : Ok
   69.3: Exclude duplicate subcmd names                                : FAILED!
---
 tools/perf/tests/Build          |   1 +
 tools/perf/tests/builtin-test.c |   1 +
 tools/perf/tests/subcmd-help.c  | 109 ++++++++++++++++++++++++++++++++
 tools/perf/tests/tests.h        |   1 +
 4 files changed, 112 insertions(+)
 create mode 100644 tools/perf/tests/subcmd-help.c
tools/perf/tests/Build +1 -0
index 2181f5a92148b0b9..13a81154ec1e4cd2 100644
@@ -69,6 +69,7 @@ perf-test-y += symbols.o
 perf-test-y += util.o
 perf-test-y += hwmon_pmu.o
 perf-test-y += tool_pmu.o
+perf-test-y += subcmd-help.o
 
 ifeq ($(SRCARCH),$(filter $(SRCARCH),x86 arm arm64 powerpc))
 perf-test-$(CONFIG_DWARF_UNWIND) += dwarf-unwind.o
tools/perf/tests/builtin-test.c +1 -0
index dfaff4185eb05a1a..2da9b69864da53c2 100644
@@ -139,6 +139,7 @@ static struct test_suite *generic_tests[] = {
 	&suite__event_groups,
 	&suite__symbols,
 	&suite__util,
+	&suite__subcmd_help,
 	NULL,
 };
 
tools/perf/tests/subcmd-help.c new file +109 -0

109 lines not on this page · raw message

tools/perf/tests/tests.h +1 -0
index 4c128a9594413b32..13cabf85185ed2d3 100644
@@ -177,6 +177,7 @@ DECLARE_SUITE(sigtrap);
 DECLARE_SUITE(event_groups);
 DECLARE_SUITE(symbols);
 DECLARE_SUITE(util);
+DECLARE_SUITE(subcmd_help);
 
 /*
  * PowerPC and S390 do not support creation of instruction breakpoints using the
Signature · 1 line
2.50.0.727.gbf7dc18ff4-goog
reviewer Reviewed-by on v1 2/3
#2 lore raw

Re: [PATCH 2/3] perf test: Add libsubcmd help tests

Quoted 11 lines · Namhyung Kim

Add a set of tests for subcmd routines. Currently it fails the last one since there's a bug. It'll be fixed by the next commit.

  $ perf test subcmd
   69: libsubcmd help tests                                            :
   69.1: Load subcmd names                                             : Ok
   69.2: Uniquify subcmd names                                         : Ok
   69.3: Exclude duplicate subcmd names                                : FAILED!
Quoted 38 lines
---
 tools/perf/tests/Build          |   1 +
 tools/perf/tests/builtin-test.c |   1 +
 tools/perf/tests/subcmd-help.c  | 109 ++++++++++++++++++++++++++++++++
 tools/perf/tests/tests.h        |   1 +
 4 files changed, 112 insertions(+)
 create mode 100644 tools/perf/tests/subcmd-help.c
tools/perf/tests/Build +1 -0
index 2181f5a92148b0b9..13a81154ec1e4cd2 100644
@@ -69,6 +69,7 @@ perf-test-y += symbols.o
 perf-test-y += util.o
 perf-test-y += hwmon_pmu.o
 perf-test-y += tool_pmu.o
+perf-test-y += subcmd-help.o
 
 ifeq ($(SRCARCH),$(filter $(SRCARCH),x86 arm arm64 powerpc))
 perf-test-$(CONFIG_DWARF_UNWIND) += dwarf-unwind.o
tools/perf/tests/builtin-test.c +1 -0
index dfaff4185eb05a1a..2da9b69864da53c2 100644
@@ -139,6 +139,7 @@ static struct test_suite *generic_tests[] = {
        &suite__event_groups,
        &suite__symbols,
        &suite__util,
+       &suite__subcmd_help,
        NULL,
 };
 
tools/perf/tests/subcmd-help.c new file +0 -0
new file mode 100644
index 0000000000000000..d31259340ae302af

tools/perf/tests/subcmd-help.c line 5

Quoted 5 lines
tools/perf/tests/subcmd-help.c new file +5 -0
@@ -0,0 +1,109 @@
+// SPDX-License-Identifier: GPL-2.0
+#include "tests.h"
+#include <stdio.h>
+#include <subcmd/help.h>
+#include "debug.h"

nit: I don't think stdio.h and debug.h are used here.

Quoted 119 lines
tools/perf/tests/subcmd-help.c new file +104 -0
+
+static int test__load_cmdnames(struct test_suite *test __maybe_unused,
+                              int subtest __maybe_unused)
+{
+       struct cmdnames cmds = {};
+
+       add_cmdname(&cmds, "aaa", 3);
+       add_cmdname(&cmds, "foo", 3);
+       add_cmdname(&cmds, "xyz", 3);
+
+       TEST_ASSERT_VAL("cannot find cmd", is_in_cmdlist(&cmds, "aaa") == 1);
+       TEST_ASSERT_VAL("wrong cmd", is_in_cmdlist(&cmds, "bar") == 0);
+       TEST_ASSERT_VAL("case sensitive", is_in_cmdlist(&cmds, "XYZ") == 0);
+
+       clean_cmdnames(&cmds);
+       return TEST_OK;
+}
+
+static int test__uniq_cmdnames(struct test_suite *test __maybe_unused,
+                              int subtest __maybe_unused)
+{
+       struct cmdnames cmds = {};
+
+       /* uniq() assumes it's sorted */
+       add_cmdname(&cmds, "aaa", 3);
+       add_cmdname(&cmds, "aaa", 3);
+       add_cmdname(&cmds, "bbb", 3);
+
+       TEST_ASSERT_VAL("invalid original size", cmds.cnt == 3);
+       /* uniquify command names (to remove second 'aaa') */
+       uniq(&cmds);
+       TEST_ASSERT_VAL("invalid final size", cmds.cnt == 2);
+
+       TEST_ASSERT_VAL("cannot find cmd", is_in_cmdlist(&cmds, "aaa") == 1);
+       TEST_ASSERT_VAL("cannot find cmd", is_in_cmdlist(&cmds, "bbb") == 1);
+       TEST_ASSERT_VAL("wrong cmd", is_in_cmdlist(&cmds, "ccc") == 0);
+
+       clean_cmdnames(&cmds);
+       return TEST_OK;
+}
+
+static int test__exclude_cmdnames(struct test_suite *test __maybe_unused,
+                                 int subtest __maybe_unused)
+{
+       struct cmdnames cmds1 = {};
+       struct cmdnames cmds2 = {};
+
+       add_cmdname(&cmds1, "aaa", 3);
+       add_cmdname(&cmds1, "bbb", 3);
+       add_cmdname(&cmds1, "ccc", 3);
+       add_cmdname(&cmds1, "ddd", 3);
+       add_cmdname(&cmds1, "eee", 3);
+       add_cmdname(&cmds1, "fff", 3);
+       add_cmdname(&cmds1, "ggg", 3);
+       add_cmdname(&cmds1, "hhh", 3);
+       add_cmdname(&cmds1, "iii", 3);
+       add_cmdname(&cmds1, "jjj", 3);
+
+       add_cmdname(&cmds2, "bbb", 3);
+       add_cmdname(&cmds2, "eee", 3);
+       add_cmdname(&cmds2, "jjj", 3);
+
+       TEST_ASSERT_VAL("invalid original size", cmds1.cnt == 10);
+       TEST_ASSERT_VAL("invalid original size", cmds2.cnt == 3);
+
+       /* remove duplicate command names in cmds1 */
+       exclude_cmds(&cmds1, &cmds2);
+
+       TEST_ASSERT_VAL("invalid excluded size", cmds1.cnt == 7);
+       TEST_ASSERT_VAL("invalid excluded size", cmds2.cnt == 3);
+
+       /* excluded commands should not belong to cmds1 */
+       TEST_ASSERT_VAL("cannot find cmd", is_in_cmdlist(&cmds1, "aaa") == 1);
+       TEST_ASSERT_VAL("wrong cmd", is_in_cmdlist(&cmds1, "bbb") == 0);
+       TEST_ASSERT_VAL("cannot find cmd", is_in_cmdlist(&cmds1, "ccc") == 1);
+       TEST_ASSERT_VAL("cannot find cmd", is_in_cmdlist(&cmds1, "ddd") == 1);
+       TEST_ASSERT_VAL("wrong cmd", is_in_cmdlist(&cmds1, "eee") == 0);
+       TEST_ASSERT_VAL("cannot find cmd", is_in_cmdlist(&cmds1, "fff") == 1);
+       TEST_ASSERT_VAL("cannot find cmd", is_in_cmdlist(&cmds1, "ggg") == 1);
+       TEST_ASSERT_VAL("cannot find cmd", is_in_cmdlist(&cmds1, "hhh") == 1);
+       TEST_ASSERT_VAL("cannot find cmd", is_in_cmdlist(&cmds1, "iii") == 1);
+       TEST_ASSERT_VAL("wrong cmd", is_in_cmdlist(&cmds1, "jjj") == 0);
+
+       /* they should be only in cmds2 */
+       TEST_ASSERT_VAL("cannot find cmd", is_in_cmdlist(&cmds2, "bbb") == 1);
+       TEST_ASSERT_VAL("cannot find cmd", is_in_cmdlist(&cmds2, "eee") == 1);
+       TEST_ASSERT_VAL("cannot find cmd", is_in_cmdlist(&cmds2, "jjj") == 1);
+
+       clean_cmdnames(&cmds1);
+       clean_cmdnames(&cmds2);
+       return TEST_OK;
+}
+
+static struct test_case tests__subcmd_help[] = {
+       TEST_CASE("Load subcmd names", load_cmdnames),
+       TEST_CASE("Uniquify subcmd names", uniq_cmdnames),
+       TEST_CASE("Exclude duplicate subcmd names", exclude_cmdnames),
+       {       .name = NULL, }
+};
+
+struct test_suite suite__subcmd_help = {
+       .desc = "libsubcmd help tests",
+       .test_cases = tests__subcmd_help,
+};
tools/perf/tests/tests.h +1 -1
index 4c128a9594413b32..13cabf85185ed2d3 100644
@@ -177,6 +177,7 @@ DECLARE_SUITE(sigtrap);
 DECLARE_SUITE(event_groups);
 DECLARE_SUITE(symbols);
 DECLARE_SUITE(util);
+DECLARE_SUITE(subcmd_help);
 
 /*
  * PowerPC and S390 do not support creation of instruction breakpoints using the
--

2.50.0.727.gbf7dc18ff4-goog

author maintainer
#3 lore raw

Re: [PATCH 2/3] perf test: Add libsubcmd help tests

Quoted 11 lines · Ian Rogers
Quoted 10 lines · Namhyung Kim

Add a set of tests for subcmd routines. Currently it fails the last one since there's a bug. It'll be fixed by the next commit.

  $ perf test subcmd
   69: libsubcmd help tests                                            :
   69.1: Load subcmd names                                             : Ok
   69.2: Uniquify subcmd names                                         : Ok
   69.3: Exclude duplicate subcmd names                                : FAILED!
Quoted 3 lines
Quoted 1 line

Thanks for your review!

Quoted 33 lines
Quoted 33 lines
---
 tools/perf/tests/Build          |   1 +
 tools/perf/tests/builtin-test.c |   1 +
 tools/perf/tests/subcmd-help.c  | 109 ++++++++++++++++++++++++++++++++
 tools/perf/tests/tests.h        |   1 +
 4 files changed, 112 insertions(+)
 create mode 100644 tools/perf/tests/subcmd-help.c
tools/perf/tests/Build +1 -0
index 2181f5a92148b0b9..13a81154ec1e4cd2 100644
@@ -69,6 +69,7 @@ perf-test-y += symbols.o
 perf-test-y += util.o
 perf-test-y += hwmon_pmu.o
 perf-test-y += tool_pmu.o
+perf-test-y += subcmd-help.o
 
 ifeq ($(SRCARCH),$(filter $(SRCARCH),x86 arm arm64 powerpc))
 perf-test-$(CONFIG_DWARF_UNWIND) += dwarf-unwind.o
tools/perf/tests/builtin-test.c +1 -0
index dfaff4185eb05a1a..2da9b69864da53c2 100644
@@ -139,6 +139,7 @@ static struct test_suite *generic_tests[] = {
        &suite__event_groups,
        &suite__symbols,
        &suite__util,
+       &suite__subcmd_help,
        NULL,
 };
 
Quoted 13 lines

tools/perf/tests/subcmd-help.c line 5

Quoted 11 lines
tools/perf/tests/subcmd-help.c new file +5 -0
new file mode 100644
index 0000000000000000..d31259340ae302af
@@ -0,0 +1,109 @@
+// SPDX-License-Identifier: GPL-2.0
+#include "tests.h"
+#include <stdio.h>
+#include <subcmd/help.h>
+#include "debug.h"

nit: I don't think stdio.h and debug.h are used here.

Yeah, will remove. I think I added it for debugging and removed the debug code later.

Thanks,
Namhyung

Quoted 120 lines
Quoted 120 lines
tools/perf/tests/subcmd-help.c new file +104 -0
+
+static int test__load_cmdnames(struct test_suite *test __maybe_unused,
+                              int subtest __maybe_unused)
+{
+       struct cmdnames cmds = {};
+
+       add_cmdname(&cmds, "aaa", 3);
+       add_cmdname(&cmds, "foo", 3);
+       add_cmdname(&cmds, "xyz", 3);
+
+       TEST_ASSERT_VAL("cannot find cmd", is_in_cmdlist(&cmds, "aaa") == 1);
+       TEST_ASSERT_VAL("wrong cmd", is_in_cmdlist(&cmds, "bar") == 0);
+       TEST_ASSERT_VAL("case sensitive", is_in_cmdlist(&cmds, "XYZ") == 0);
+
+       clean_cmdnames(&cmds);
+       return TEST_OK;
+}
+
+static int test__uniq_cmdnames(struct test_suite *test __maybe_unused,
+                              int subtest __maybe_unused)
+{
+       struct cmdnames cmds = {};
+
+       /* uniq() assumes it's sorted */
+       add_cmdname(&cmds, "aaa", 3);
+       add_cmdname(&cmds, "aaa", 3);
+       add_cmdname(&cmds, "bbb", 3);
+
+       TEST_ASSERT_VAL("invalid original size", cmds.cnt == 3);
+       /* uniquify command names (to remove second 'aaa') */
+       uniq(&cmds);
+       TEST_ASSERT_VAL("invalid final size", cmds.cnt == 2);
+
+       TEST_ASSERT_VAL("cannot find cmd", is_in_cmdlist(&cmds, "aaa") == 1);
+       TEST_ASSERT_VAL("cannot find cmd", is_in_cmdlist(&cmds, "bbb") == 1);
+       TEST_ASSERT_VAL("wrong cmd", is_in_cmdlist(&cmds, "ccc") == 0);
+
+       clean_cmdnames(&cmds);
+       return TEST_OK;
+}
+
+static int test__exclude_cmdnames(struct test_suite *test __maybe_unused,
+                                 int subtest __maybe_unused)
+{
+       struct cmdnames cmds1 = {};
+       struct cmdnames cmds2 = {};
+
+       add_cmdname(&cmds1, "aaa", 3);
+       add_cmdname(&cmds1, "bbb", 3);
+       add_cmdname(&cmds1, "ccc", 3);
+       add_cmdname(&cmds1, "ddd", 3);
+       add_cmdname(&cmds1, "eee", 3);
+       add_cmdname(&cmds1, "fff", 3);
+       add_cmdname(&cmds1, "ggg", 3);
+       add_cmdname(&cmds1, "hhh", 3);
+       add_cmdname(&cmds1, "iii", 3);
+       add_cmdname(&cmds1, "jjj", 3);
+
+       add_cmdname(&cmds2, "bbb", 3);
+       add_cmdname(&cmds2, "eee", 3);
+       add_cmdname(&cmds2, "jjj", 3);
+
+       TEST_ASSERT_VAL("invalid original size", cmds1.cnt == 10);
+       TEST_ASSERT_VAL("invalid original size", cmds2.cnt == 3);
+
+       /* remove duplicate command names in cmds1 */
+       exclude_cmds(&cmds1, &cmds2);
+
+       TEST_ASSERT_VAL("invalid excluded size", cmds1.cnt == 7);
+       TEST_ASSERT_VAL("invalid excluded size", cmds2.cnt == 3);
+
+       /* excluded commands should not belong to cmds1 */
+       TEST_ASSERT_VAL("cannot find cmd", is_in_cmdlist(&cmds1, "aaa") == 1);
+       TEST_ASSERT_VAL("wrong cmd", is_in_cmdlist(&cmds1, "bbb") == 0);
+       TEST_ASSERT_VAL("cannot find cmd", is_in_cmdlist(&cmds1, "ccc") == 1);
+       TEST_ASSERT_VAL("cannot find cmd", is_in_cmdlist(&cmds1, "ddd") == 1);
+       TEST_ASSERT_VAL("wrong cmd", is_in_cmdlist(&cmds1, "eee") == 0);
+       TEST_ASSERT_VAL("cannot find cmd", is_in_cmdlist(&cmds1, "fff") == 1);
+       TEST_ASSERT_VAL("cannot find cmd", is_in_cmdlist(&cmds1, "ggg") == 1);
+       TEST_ASSERT_VAL("cannot find cmd", is_in_cmdlist(&cmds1, "hhh") == 1);
+       TEST_ASSERT_VAL("cannot find cmd", is_in_cmdlist(&cmds1, "iii") == 1);
+       TEST_ASSERT_VAL("wrong cmd", is_in_cmdlist(&cmds1, "jjj") == 0);
+
+       /* they should be only in cmds2 */
+       TEST_ASSERT_VAL("cannot find cmd", is_in_cmdlist(&cmds2, "bbb") == 1);
+       TEST_ASSERT_VAL("cannot find cmd", is_in_cmdlist(&cmds2, "eee") == 1);
+       TEST_ASSERT_VAL("cannot find cmd", is_in_cmdlist(&cmds2, "jjj") == 1);
+
+       clean_cmdnames(&cmds1);
+       clean_cmdnames(&cmds2);
+       return TEST_OK;
+}
+
+static struct test_case tests__subcmd_help[] = {
+       TEST_CASE("Load subcmd names", load_cmdnames),
+       TEST_CASE("Uniquify subcmd names", uniq_cmdnames),
+       TEST_CASE("Exclude duplicate subcmd names", exclude_cmdnames),
+       {       .name = NULL, }
+};
+
+struct test_suite suite__subcmd_help = {
+       .desc = "libsubcmd help tests",
+       .test_cases = tests__subcmd_help,
+};
tools/perf/tests/tests.h +1 -1
index 4c128a9594413b32..13cabf85185ed2d3 100644
@@ -177,6 +177,7 @@ DECLARE_SUITE(sigtrap);
 DECLARE_SUITE(event_groups);
 DECLARE_SUITE(symbols);
 DECLARE_SUITE(util);
+DECLARE_SUITE(subcmd_help);
 
 /*
  * PowerPC and S390 do not support creation of instruction breakpoints using the
--

2.50.0.727.gbf7dc18ff4-goog

author maintainer
#4 lore raw

PATCH 3/3 perf tools: Fix use-after-free in help_unknown_cmd()

In reply to Namhyung Kim

Currently perf aborts when it finds an invalid command. I guess it depends on the environment as I have some custom commands in the path.

  $ perf bad-command
  perf: 'bad-command' is not a perf-command. See 'perf --help'.
  Aborted (core dumped)

It's because the exclude_cmds() in libsubcmd has a use-after-free when it removes some entries. After copying one to another entry, it keeps the pointer in the both position. And the next copy operation will free the later one but it's the same entry in the previous one.

For example, let's say cmds = { A, B, C, D, E } and excludes = { B, E }.

ci cj ei cmds-name excludes
-----------+--------------------

   0   0   0 |     A         B       :    cmp < 0, ci == cj
   1   1   0 |     B         B       :    cmp == 0
   2   1   1 |     C         E       :    cmp < 0, ci != cj

At this point, it frees cmds->names[1] and cmds->names[1] is assigned to cmds->names[2].

3 2 1 | D E : cmp < 0, ci != cj

Now it frees cmds->names[2] but it's the same as cmds->names[1]. So accessing cmds->names[1] will be invalid.

This makes the subcmd tests succeed.

  $ perf test subcmd
   69: libsubcmd help tests                                            :
   69.1: Load subcmd names                                             : Ok
   69.2: Uniquify subcmd names                                         : Ok
   69.3: Exclude duplicate subcmd names                                : Ok
Fixes 657a3efee43a ("libsubcmd: Avoid SEGV/use-after-free when commands aren't excluded")
---
 tools/lib/subcmd/help.c | 12 +++++++-----
 1 file changed, 7 insertions(+), 5 deletions(-)
tools/lib/subcmd/help.c +7 -5
index 8561b0f01a247690..9ef569492560efd7 100644
@@ -9,6 +9,7 @@
 #include <sys/stat.h>
 #include <unistd.h>
 #include <dirent.h>
+#include <assert.h>
 #include "subcmd-util.h"
 #include "help.h"
 #include "exec-cmd.h"
@@ -82,10 +83,11 @@ void exclude_cmds(struct cmdnames *cmds, struct cmdnames *excludes)
 				ci++;
 				cj++;
 			} else {
-				zfree(&cmds->names[cj]);
-				cmds->names[cj++] = cmds->names[ci++];
+				cmds->names[cj++] = cmds->names[ci];
+				cmds->names[ci++] = NULL;
 			}
 		} else if (cmp == 0) {
+			zfree(&cmds->names[ci]);
 			ci++;
 			ei++;
 		} else if (cmp > 0) {
@@ -94,12 +96,12 @@ void exclude_cmds(struct cmdnames *cmds, struct cmdnames *excludes)
 	}
 	if (ci != cj) {
 		while (ci < cmds->cnt) {
-			zfree(&cmds->names[cj]);
-			cmds->names[cj++] = cmds->names[ci++];
+			cmds->names[cj++] = cmds->names[ci];
+			cmds->names[ci++] = NULL;
 		}
 	}
 	for (ci = cj; ci < cmds->cnt; ci++)
-		zfree(&cmds->names[ci]);
+		assert(cmds->names[ci] == NULL);
 	cmds->cnt = cj;
 }
 
Signature · 1 line
2.50.0.727.gbf7dc18ff4-goog
reviewer Reviewed-by on v1 3/3
#5 lore raw

Re: [PATCH 3/3] perf tools: Fix use-after-free in help_unknown_cmd()

Quoted 31 lines · Namhyung Kim

Currently perf aborts when it finds an invalid command. I guess it depends on the environment as I have some custom commands in the path.

  $ perf bad-command
  perf: 'bad-command' is not a perf-command. See 'perf --help'.
  Aborted (core dumped)

It's because the exclude_cmds() in libsubcmd has a use-after-free when it removes some entries. After copying one to another entry, it keeps the pointer in the both position. And the next copy operation will free the later one but it's the same entry in the previous one.

For example, let's say cmds = { A, B, C, D, E } and excludes = { B, E }.

ci cj ei cmds-name excludes
-----------+--------------------

   0   0   0 |     A         B       :    cmp < 0, ci == cj
   1   1   0 |     B         B       :    cmp == 0
   2   1   1 |     C         E       :    cmp < 0, ci != cj

At this point, it frees cmds->names[1] and cmds->names[1] is assigned to cmds->names[2].

   3   2   1 |     D         E       :    cmp < 0, ci != cj

Now it frees cmds->names[2] but it's the same as cmds->names[1]. So accessing cmds->names[1] will be invalid.

This makes the subcmd tests succeed.

Quoted 8 lines
  $ perf test subcmd
   69: libsubcmd help tests                                            :
   69.1: Load subcmd names                                             : Ok
   69.2: Uniquify subcmd names                                         : Ok
   69.3: Exclude duplicate subcmd names                                : Ok
Fixes 657a3efee43a ("libsubcmd: Avoid SEGV/use-after-free when commands aren't excluded")

Fwiw, it seems a shame we're doing this and the code in git is drifting apart:

https://github.com/git/git/blob/83014dc05f6fc9275c0a02886cb428805abaf9e5/help.c#L204

Thanks,
Ian

Quoted 49 lines
---
 tools/lib/subcmd/help.c | 12 +++++++-----
 1 file changed, 7 insertions(+), 5 deletions(-)
tools/lib/subcmd/help.c +7 -6
index 8561b0f01a247690..9ef569492560efd7 100644
@@ -9,6 +9,7 @@
 #include <sys/stat.h>
 #include <unistd.h>
 #include <dirent.h>
+#include <assert.h>
 #include "subcmd-util.h"
 #include "help.h"
 #include "exec-cmd.h"
@@ -82,10 +83,11 @@ void exclude_cmds(struct cmdnames *cmds, struct cmdnames *excludes)
                                ci++;
                                cj++;
                        } else {
-                               zfree(&cmds->names[cj]);
-                               cmds->names[cj++] = cmds->names[ci++];
+                               cmds->names[cj++] = cmds->names[ci];
+                               cmds->names[ci++] = NULL;
                        }
                } else if (cmp == 0) {
+                       zfree(&cmds->names[ci]);
                        ci++;
                        ei++;
                } else if (cmp > 0) {
@@ -94,12 +96,12 @@ void exclude_cmds(struct cmdnames *cmds, struct cmdnames *excludes)
        }
        if (ci != cj) {
                while (ci < cmds->cnt) {
-                       zfree(&cmds->names[cj]);
-                       cmds->names[cj++] = cmds->names[ci++];
+                       cmds->names[cj++] = cmds->names[ci];
+                       cmds->names[ci++] = NULL;
                }
        }
        for (ci = cj; ci < cmds->cnt; ci++)
-               zfree(&cmds->names[ci]);
+               assert(cmds->names[ci] == NULL);
        cmds->cnt = cj;
 }
 
--

2.50.0.727.gbf7dc18ff4-goog

author maintainer
#6 lore raw

Re: [PATCH 3/3] perf tools: Fix use-after-free in help_unknown_cmd()

Quoted 38 lines · Ian Rogers
Quoted 37 lines · Namhyung Kim

Currently perf aborts when it finds an invalid command. I guess it depends on the environment as I have some custom commands in the path.

  $ perf bad-command
  perf: 'bad-command' is not a perf-command. See 'perf --help'.
  Aborted (core dumped)

It's because the exclude_cmds() in libsubcmd has a use-after-free when it removes some entries. After copying one to another entry, it keeps the pointer in the both position. And the next copy operation will free the later one but it's the same entry in the previous one.

For example, let's say cmds = { A, B, C, D, E } and excludes = { B, E }.

ci cj ei cmds-name excludes
-----------+--------------------

   0   0   0 |     A         B       :    cmp < 0, ci == cj
   1   1   0 |     B         B       :    cmp == 0
   2   1   1 |     C         E       :    cmp < 0, ci != cj

At this point, it frees cmds->names[1] and cmds->names[1] is assigned to cmds->names[2].

   3   2   1 |     D         E       :    cmp < 0, ci != cj

Now it frees cmds->names[2] but it's the same as cmds->names[1]. So accessing cmds->names[1] will be invalid.

This makes the subcmd tests succeed.

  $ perf test subcmd
   69: libsubcmd help tests                                            :
   69.1: Load subcmd names                                             : Ok
   69.2: Uniquify subcmd names                                         : Ok
   69.3: Exclude duplicate subcmd names                                : Ok
Quoted 8 lines
Quoted 2 lines
Fixes 657a3efee43a ("libsubcmd: Avoid SEGV/use-after-free when commands aren't excluded")

Fwiw, it seems a shame we're doing this and the code in git is drifting apart:

https://github.com/git/git/blob/83014dc05f6fc9275c0a02886cb428805abaf9e5/help.c#L204

Maybe we can send a fix for them too.

Thanks,
Namhyung

reviewer
#7 lore raw
In reply to Namhyung Kim
Quoted 12 lines · Namhyung Kim

Currently perf test checks the given string with descriptions for both test suites and cases (subtests). But sometimes it's confusing since the subtests don't contain the important keyword.

I think it's better to check the suite level and run the whole suite together.

Before:

  $ perf test hwmon
  (no output)
Quoted 6 lines

After:

  $ perf test hwmon
   10: Hwmon PMU                                                       :
   10.1: Basic parsing test                                            : Ok
   10.2: Parsing without PMU name                                      : Ok
   10.3: Parsing with PMU name                                         : Ok

This is better, thanks!

Quoted 10 lines
---
 tools/perf/tests/builtin-test.c | 4 ----
 1 file changed, 4 deletions(-)
tools/perf/tests/builtin-test.c +0 -0
index 80375ca39a37a256..dfaff4185eb05a1a 100644

tools/perf/tests/builtin-test.c static int __cmd_test(struct test_suite **suites, int argc, const char *argv[], · line 573

Quoted 7 lines
tools/perf/tests/builtin-test.c +0 -4
@@ -567,10 +567,6 @@ static int __cmd_test(struct test_suite **suites, int argc, const char *argv[],
 
                        for (unsigned int run = 0; run < runs_per_test; run++) {
                                test_suite__for_each_test_case(*t, curr_test_case) {
-                                       if (!perf_test__matches(test_description(*t, curr_test_case),
-                                                               curr_suite, argc, argv))
-                                               continue;
-

This will change the behavior so that if a sub-test matches but the test suite as a whole doesn't the whole test suite will now be run. For example:

```
$ perf test list
...
 39: CPU map
39.1: Synthesize cpu map
39.2: Print cpu map
39.3: Merge cpu map
39.4: Intersect cpu map
39.5: Equal cpu map
...
$ perf test -v "Equal cpu map"
39.5: Equal cpu map                                                 : Ok
```

whereas with this change the whole of the "CPU map" test suite will be run. I think the condition:

```
if (!perf_test__matches(test_description(*t, curr_test_case),
curr_suite, argc, argv))
```

should be:

```
if (!perf_test__matches(test_description(*t, curr_test_case),
curr_suite, argc, argv) &&
    !perf_test__matches(test_description(*t, -1), curr_suite, argc, argv))
```

But you could avoid computing the extra perf_test__matches with a boolean as that test is done immediately before.

Thanks,
Ian

Quoted 6 lines
tools/perf/tests/builtin-test.c +0 -1
                                        err = start_test(*t, curr_suite, curr_test_case,
                                                         &child_tests[child_test_num++],
                                                         width, pass);
--

2.50.0.727.gbf7dc18ff4-goog

author maintainer
#8 lore raw
Quoted 68 lines · Ian Rogers
Quoted 12 lines · Namhyung Kim

Currently perf test checks the given string with descriptions for both test suites and cases (subtests). But sometimes it's confusing since the subtests don't contain the important keyword.

I think it's better to check the suite level and run the whole suite together.

Before:

  $ perf test hwmon
  (no output)
Quoted 6 lines

After:

  $ perf test hwmon
   10: Hwmon PMU                                                       :
   10.1: Basic parsing test                                            : Ok
   10.2: Parsing without PMU name                                      : Ok
   10.3: Parsing with PMU name                                         : Ok

This is better, thanks!

Quoted 10 lines
---
 tools/perf/tests/builtin-test.c | 4 ----
 1 file changed, 4 deletions(-)
tools/perf/tests/builtin-test.c +0 -0
index 80375ca39a37a256..dfaff4185eb05a1a 100644

tools/perf/tests/builtin-test.c static int __cmd_test(struct test_suite **suites, int argc, const char *argv[], · line 573

Quoted 7 lines
tools/perf/tests/builtin-test.c +0 -4
@@ -567,10 +567,6 @@ static int __cmd_test(struct test_suite **suites, int argc, const char *argv[],
 
                        for (unsigned int run = 0; run < runs_per_test; run++) {
                                test_suite__for_each_test_case(*t, curr_test_case) {
-                                       if (!perf_test__matches(test_description(*t, curr_test_case),
-                                                               curr_suite, argc, argv))
-                                               continue;
-

This will change the behavior so that if a sub-test matches but the test suite as a whole doesn't the whole test suite will now be run. For example:

```
$ perf test list
...
 39: CPU map
39.1: Synthesize cpu map
39.2: Print cpu map
39.3: Merge cpu map
39.4: Intersect cpu map
39.5: Equal cpu map
...
$ perf test -v "Equal cpu map"
39.5: Equal cpu map                                                 : Ok
```

whereas with this change the whole of the "CPU map" test suite will be run. I think the condition:

```
if (!perf_test__matches(test_description(*t, curr_test_case),
curr_suite, argc, argv))
```

should be:

Quoted 8 lines
```
if (!perf_test__matches(test_description(*t, curr_test_case),
curr_suite, argc, argv) &&
    !perf_test__matches(test_description(*t, -1), curr_suite, argc, argv))
```

But you could avoid computing the extra perf_test__matches with a boolean as that test is done immediately before.

Oh, I overlooked the condition in the loop. Will check the both can skip if it only matched to some subtests.

Thanks,
Namhyung