Skip to contentSkip to conversation

perf test: Add build test for JEVENTS_ARCH=all

PATCH In review Ian Rogers 3 participants 13 messages Aug 5, 2024 – Aug 6, 2024

Subsystems (4)
lore
Share & tools
lore

Ian Rogers

author reviewer
#0 lore raw

PATCH 1/5 perf test: Add build test for JEVENTS_ARCH=all

Building with JEVENTS_ARCH=all builds all CPU types and allows things like assertions to check the validity of the input json.

---
 tools/perf/tests/make | 2 ++
 1 file changed, 2 insertions(+)
tools/perf/tests/make +2 -0
index a1f8adf85367..8a9edf758f10 100644
@@ -70,6 +70,7 @@ make_python_perf_so := $(python_perf_so)
 make_debug          := DEBUG=1
 make_nondistro      := BUILD_NONDISTRO=1
 make_extra_tests    := EXTRA_TESTS=1
+make_jevents_all    := JEVENTS_ARCH=all
 make_no_bpf_skel    := BUILD_BPF_SKEL=0
 make_gen_vmlinux_h  := GEN_VMLINUX_H=1
 make_no_libperl     := NO_LIBPERL=1
@@ -140,6 +141,7 @@ run += make_python_perf_so
 run += make_debug
 run += make_nondistro
 run += make_extra_tests
+run += make_jevents_all
 run += make_no_bpf_skel
 run += make_gen_vmlinux_h
 run += make_no_libperl
Signature · 1 line
2.46.0.rc2.264.g509ed76dc8-goog
author reviewer
#1 lore raw

PATCH 2/5 perf pmu-events: Change dependencies for empty-pmu-events.c test

Switch from $? (all the prerequisites that are newer than the target) to $^ (all the prerequisites) as touching jevents.py will mean that empty-pmu-events.c won't be passed to the diff command breaking the build.

---
 tools/perf/pmu-events/Build | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
tools/perf/pmu-events/Build +1 -1
index c3fa43c49706..d941bc9d16e9 100644
@@ -39,7 +39,7 @@ $(TEST_EMPTY_PMU_EVENTS_C): $(JSON) $(JSON_TEST) $(JEVENTS_PY) $(METRIC_PY) $(ME
 
 $(EMPTY_PMU_EVENTS_TEST_LOG): $(EMPTY_PMU_EVENTS_C) $(TEST_EMPTY_PMU_EVENTS_C)
 	$(call rule_mkdir)
-	$(Q)$(call echo-cmd,test)diff -u $? 2> $@ || (cat $@ && false)
+	$(Q)$(call echo-cmd,test)diff -u $^ 2> $@ || (cat $@ && false)
 
 $(PMU_EVENTS_C): $(JSON) $(JSON_TEST) $(JEVENTS_PY) $(METRIC_PY) $(METRIC_TEST_LOG) $(EMPTY_PMU_EVENTS_TEST_LOG)
 	$(call rule_mkdir)
Signature · 1 line
2.46.0.rc2.264.g509ed76dc8-goog
maintainer Tested-by on v1 2/5
#2 lore raw

Re: [PATCH v1 2/5] perf pmu-events: Change dependencies for empty-pmu-events.c test

Quoted 4 lines · Ian Rogers

Switch from $? (all the prerequisites that are newer than the target) to $^ (all the prerequisites) as touching jevents.py will mean that empty-pmu-events.c won't be passed to the diff command breaking the build.

Thanks, added:

Quoted 21 lines
---
 tools/perf/pmu-events/Build | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
tools/perf/pmu-events/Build +1 -1
index c3fa43c49706..d941bc9d16e9 100644
@@ -39,7 +39,7 @@ $(TEST_EMPTY_PMU_EVENTS_C): $(JSON) $(JSON_TEST) $(JEVENTS_PY) $(METRIC_PY) $(ME
 
 $(EMPTY_PMU_EVENTS_TEST_LOG): $(EMPTY_PMU_EVENTS_C) $(TEST_EMPTY_PMU_EVENTS_C)
 	$(call rule_mkdir)
-	$(Q)$(call echo-cmd,test)diff -u $? 2> $@ || (cat $@ && false)
+	$(Q)$(call echo-cmd,test)diff -u $^ 2> $@ || (cat $@ && false)
 
 $(PMU_EVENTS_C): $(JSON) $(JSON_TEST) $(JEVENTS_PY) $(METRIC_PY) $(METRIC_TEST_LOG) $(EMPTY_PMU_EVENTS_TEST_LOG)
 	$(call rule_mkdir)
Signature · 1 line
2.46.0.rc2.264.g509ed76dc8-goog
author reviewer
#3 lore raw

PATCH 3/5 perf pmu-events: Remove duplicated ampereone event

In reply to Ian Rogers

OP_SPEC is repeated twice in the file which will break invariants in perf list as discussed in this thread:
https://lore.kernel.org/linux-perf-users/[email protected]/

---
 .../pmu-events/arch/arm64/ampere/ampereone/instruction.json    | 3 ---
 1 file changed, 3 deletions(-)
tools/perf/pmu-events/arch/arm64/ampere/ampereone/instruction.json +0 -3
index 18d1f2f76a23..9fe697d12fe0 100644
@@ -77,9 +77,6 @@
     {
         "ArchStdEvent": "OP_RETIRED"
     },
-    {
-        "ArchStdEvent": "OP_SPEC"
-    },
     {
         "PublicDescription": "Operation speculatively executed, NOP",
         "EventCode": "0x100",
Signature · 1 line
2.46.0.rc2.264.g509ed76dc8-goog
author reviewer
#4 lore raw

PATCH 4/5 perf pmu-events: Remove duplicated riscv firmware event

In reply to Ian Rogers

FW_SFENCE_VMA_RECEIVED is repeated twice in the file which will break invariants in perf list as discussed in this thread:
https://lore.kernel.org/linux-perf-users/[email protected]/

---
 tools/perf/pmu-events/arch/riscv/andes/ax45/firmware.json      | 3 ---
 tools/perf/pmu-events/arch/riscv/sifive/u74/firmware.json      | 3 ---
 .../perf/pmu-events/arch/riscv/starfive/dubhe-80/firmware.json | 3 ---
 .../perf/pmu-events/arch/riscv/thead/c900-legacy/firmware.json | 3 ---
 4 files changed, 12 deletions(-)
tools/perf/pmu-events/arch/riscv/andes/ax45/firmware.json +0 -3
index 9b4a032186a7..5a996fa4b837 100644
@@ -35,9 +35,6 @@
   {
     "ArchStdEvent": "FW_SFENCE_VMA_RECEIVED"
   },
-  {
-    "ArchStdEvent": "FW_SFENCE_VMA_RECEIVED"
-  },
   {
     "ArchStdEvent": "FW_SFENCE_VMA_ASID_RECEIVED"
   },
tools/perf/pmu-events/arch/riscv/sifive/u74/firmware.json +0 -3
index 9b4a032186a7..5a996fa4b837 100644
@@ -35,9 +35,6 @@
   {
     "ArchStdEvent": "FW_SFENCE_VMA_RECEIVED"
   },
-  {
-    "ArchStdEvent": "FW_SFENCE_VMA_RECEIVED"
-  },
   {
     "ArchStdEvent": "FW_SFENCE_VMA_ASID_RECEIVED"
   },
tools/perf/pmu-events/arch/riscv/starfive/dubhe-80/firmware.json +0 -3
index 9b4a032186a7..5a996fa4b837 100644
@@ -35,9 +35,6 @@
   {
     "ArchStdEvent": "FW_SFENCE_VMA_RECEIVED"
   },
-  {
-    "ArchStdEvent": "FW_SFENCE_VMA_RECEIVED"
-  },
   {
     "ArchStdEvent": "FW_SFENCE_VMA_ASID_RECEIVED"
   },
tools/perf/pmu-events/arch/riscv/thead/c900-legacy/firmware.json +0 -3
index 9b4a032186a7..5a996fa4b837 100644
@@ -35,9 +35,6 @@
   {
     "ArchStdEvent": "FW_SFENCE_VMA_RECEIVED"
   },
-  {
-    "ArchStdEvent": "FW_SFENCE_VMA_RECEIVED"
-  },
   {
     "ArchStdEvent": "FW_SFENCE_VMA_ASID_RECEIVED"
   },
Signature · 1 line
2.46.0.rc2.264.g509ed76dc8-goog
#5 lore raw

Re: [PATCH v1 4/5] perf pmu-events: Remove duplicated riscv firmware event

Hi Ian,

I've sent a patch to fix it and the patch already merged. Thanks.
https://lore.kernel.org/all/[email protected]/

Best regards,
Eric Lin

Quoted 72 lines · Ian Rogers

FW_SFENCE_VMA_RECEIVED is repeated twice in the file which will break invariants in perf list as discussed in this thread:
https://lore.kernel.org/linux-perf-users/[email protected]/

---
 tools/perf/pmu-events/arch/riscv/andes/ax45/firmware.json      | 3 ---
 tools/perf/pmu-events/arch/riscv/sifive/u74/firmware.json      | 3 ---
 .../perf/pmu-events/arch/riscv/starfive/dubhe-80/firmware.json | 3 ---
 .../perf/pmu-events/arch/riscv/thead/c900-legacy/firmware.json | 3 ---
 4 files changed, 12 deletions(-)
tools/perf/pmu-events/arch/riscv/andes/ax45/firmware.json +0 -3
index 9b4a032186a7..5a996fa4b837 100644
@@ -35,9 +35,6 @@
   {
     "ArchStdEvent": "FW_SFENCE_VMA_RECEIVED"
   },
-  {
-    "ArchStdEvent": "FW_SFENCE_VMA_RECEIVED"
-  },
   {
     "ArchStdEvent": "FW_SFENCE_VMA_ASID_RECEIVED"
   },
tools/perf/pmu-events/arch/riscv/sifive/u74/firmware.json +0 -3
index 9b4a032186a7..5a996fa4b837 100644
@@ -35,9 +35,6 @@
   {
     "ArchStdEvent": "FW_SFENCE_VMA_RECEIVED"
   },
-  {
-    "ArchStdEvent": "FW_SFENCE_VMA_RECEIVED"
-  },
   {
     "ArchStdEvent": "FW_SFENCE_VMA_ASID_RECEIVED"
   },
tools/perf/pmu-events/arch/riscv/starfive/dubhe-80/firmware.json +0 -3
index 9b4a032186a7..5a996fa4b837 100644
@@ -35,9 +35,6 @@
   {
     "ArchStdEvent": "FW_SFENCE_VMA_RECEIVED"
   },
-  {
-    "ArchStdEvent": "FW_SFENCE_VMA_RECEIVED"
-  },
   {
     "ArchStdEvent": "FW_SFENCE_VMA_ASID_RECEIVED"
   },
tools/perf/pmu-events/arch/riscv/thead/c900-legacy/firmware.json +0 -4
index 9b4a032186a7..5a996fa4b837 100644
@@ -35,9 +35,6 @@
   {
     "ArchStdEvent": "FW_SFENCE_VMA_RECEIVED"
   },
-  {
-    "ArchStdEvent": "FW_SFENCE_VMA_RECEIVED"
-  },
   {
     "ArchStdEvent": "FW_SFENCE_VMA_ASID_RECEIVED"
   },
--

2.46.0.rc2.264.g509ed76dc8-goog

author reviewer
#6 lore raw

Re: [PATCH v1 4/5] perf pmu-events: Remove duplicated riscv firmware event

Quoted 5 lines · Eric Lin

Hi Ian,

I've sent a patch to fix it and the patch already merged. Thanks.
https://lore.kernel.org/all/[email protected]/

Hi Eric/Arnaldo,

Right, I already commented this should have gone through the perf-tools-next tree:
https://lore.kernel.org/all/CAP-5=fV3NXkKsCP1WH0_qLRNpL+WuP8S3h1=cHaUMH5MFkVHQg@mail.gmail.com/
Arnaldo, please take Eric's patch in preference to this one.

Thanks,
Ian

#7 lore raw

Re: [PATCH v1 4/5] perf pmu-events: Remove duplicated riscv firmware event

Quoted 7 lines · Ian Rogers
Quoted 5 lines · Eric Lin

Hi Ian,

I've sent a patch to fix it and the patch already merged. Thanks.
https://lore.kernel.org/all/[email protected]/

Quoted 6 lines

Hi Eric/Arnaldo,

Right, I already commented this should have gone through the perf-tools-next tree:
https://lore.kernel.org/all/CAP-5=fV3NXkKsCP1WH0_qLRNpL+WuP8S3h1=cHaUMH5MFkVHQg@mail.gmail.com/
Arnaldo, please take Eric's patch in preference to this one.

So I removed your version from perf-tools-next, eventually we'll get this merged upstream, I'm just checking this isn't going to get in our way of testing what we have in perf-tools-next...

- Arnaldo

#8 lore raw

Re: [PATCH v1 4/5] perf pmu-events: Remove duplicated riscv firmware event

Quoted 5 lines · Arnaldo Carvalho de Melo
Quoted 3 lines · Ian Rogers
Quoted 2 lines · Eric Lin

I've sent a patch to fix it and the patch already merged. Thanks.
https://lore.kernel.org/all/[email protected]/

Quoted 8 lines
Quoted 4 lines

Right, I already commented this should have gone through the perf-tools-next tree:
https://lore.kernel.org/all/CAP-5=fV3NXkKsCP1WH0_qLRNpL+WuP8S3h1=cHaUMH5MFkVHQg@mail.gmail.com/
Arnaldo, please take Eric's patch in preference to this one.

So I removed your version from perf-tools-next, eventually we'll get this merged upstream, I'm just checking this isn't going to get in our way of testing what we have in perf-tools-next...

As expected:

⬢[acme@toolbox perf-tools-next]$ git log --oneline -1 ; time make -C tools/perf build-test
<SNIP>
make_no_libdw_dwarf_unwind_O: cd . && make NO_LIBDW_DWARF_UNWIND=1 FEATURES_DUMP=/home/acme/git/perf-tools-next/tools/perf/BUILD_TEST_FEATURE_DUMP -j28 O=/tmp/tmp.1xBTPCPJh1 DESTDIR=/tmp/tmp.6upZRBhUpN

          make_jevents_all_O: cd . && make JEVENTS_ARCH=all FEATURES_DUMP=/home/acme/git/perf-tools-next/tools/perf/BUILD_TEST_FEATURE_DUMP -j28 O=/tmp/tmp.ncjKsS9Ddm DESTDIR=/tmp/tmp.z2lxV44jzL

cd . && make JEVENTS_ARCH=all FEATURES_DUMP=/home/acme/git/perf-tools-next/tools/perf/BUILD_TEST_FEATURE_DUMP -j28 O=/tmp/tmp.ncjKsS9Ddm DESTDIR=/tmp/tmp.z2lxV44jzL
BUILD: Doing 'make -j28' parallel build
Warning: Kernel ABI header differences:
diff -u tools/include/uapi/drm/i915_drm.h include/uapi/drm/i915_drm.h
<SNIP>

  LD      /tmp/tmp.ncjKsS9Ddm/util/perf-util-in.o
  LD      /tmp/tmp.ncjKsS9Ddm/perf-util-in.o
  AR      /tmp/tmp.ncjKsS9Ddm/libperf-util.a
Traceback (most recent call last):
  File "/home/acme/git/perf-tools-next/tools/perf/pmu-events/jevents.py", line 1317, in <module>
    main()
  File "/home/acme/git/perf-tools-next/tools/perf/pmu-events/jevents.py", line 1308, in main
    ftw(arch_path, [], process_one_file)
  File "/home/acme/git/perf-tools-next/tools/perf/pmu-events/jevents.py", line 1245, in ftw
    ftw(item.path, parents + [item.name], action)
  File "/home/acme/git/perf-tools-next/tools/perf/pmu-events/jevents.py", line 1243, in ftw
    action(parents, item)
  File "/home/acme/git/perf-tools-next/tools/perf/pmu-events/jevents.py", line 646, in process_one_file
    print_pending_events()
  File "/home/acme/git/perf-tools-next/tools/perf/pmu-events/jevents.py", line 510, in print_pending_events
    assert event.name != last_name, f"Duplicate event: {last_pmu}/{last_name}/ in {_pending_events_tblname}"
           ^^^^^^^^^^^^^^^^^^^^^^^

AssertionError: Duplicate event: default_core/fw_sfence_vma_received/ in pmu_events__andes_ax45 make[5]: *** [pmu-events/Build:46: /tmp/tmp.ncjKsS9Ddm/pmu-events/pmu-events.c] Error 1 make[5]: *** Deleting file '/tmp/tmp.ncjKsS9Ddm/pmu-events/pmu-events.c' make[4]: *** [Makefile.perf:763: /tmp/tmp.ncjKsS9Ddm/pmu-events/pmu-events-in.o] Error 2

So I'll have to cherry pick that patch and have it in perf-tools-next, we'll get notified at some point that linux-next has the same patch, will ack that, etc.

Please send patches for this area in the future to the perf-tools-next tree.

- Arnaldo

#9 lore raw

Re: [PATCH v1 4/5] perf pmu-events: Remove duplicated riscv firmware event

In reply to Arnaldo Carvalho de Melo
Quoted 6 lines · Arnaldo Carvalho de Melo
Quoted 5 lines · Ian Rogers
Quoted 4 lines · Eric Lin

Hi Ian,

I've sent a patch to fix it and the patch already merged. Thanks.

Also, already merged where? Upstream? In the riscv kernel tree? where?

- Arnaldo

Quoted 15 lines
Quoted 8 lines
Quoted 1 line

https://lore.kernel.org/all/[email protected]/

Hi Eric/Arnaldo,

Right, I already commented this should have gone through the perf-tools-next tree:
https://lore.kernel.org/all/CAP-5=fV3NXkKsCP1WH0_qLRNpL+WuP8S3h1=cHaUMH5MFkVHQg@mail.gmail.com/
Arnaldo, please take Eric's patch in preference to this one.

So I removed your version from perf-tools-next, eventually we'll get this merged upstream, I'm just checking this isn't going to get in our way of testing what we have in perf-tools-next...

- Arnaldo

#10 lore raw

Re: [PATCH v1 4/5] perf pmu-events: Remove duplicated riscv firmware event

Hi Arnaldo,

Quoted 11 lines · Arnaldo Carvalho de Melo
Quoted 6 lines · Arnaldo Carvalho de Melo
Quoted 5 lines · Ian Rogers
Quoted 4 lines · Eric Lin

Hi Ian,

I've sent a patch to fix it and the patch already merged. Thanks.

Also, already merged where? Upstream? In the riscv kernel tree? where?

The patch already merged to Linux upstream:
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=63ba5b0fb4f54db256ec43b3062b2606b383055d

Thanks.

Best regards,
Eric Lin

Quoted 17 lines

- Arnaldo

Quoted 15 lines
Quoted 8 lines
Quoted 1 line

https://lore.kernel.org/all/[email protected]/

Hi Eric/Arnaldo,

Right, I already commented this should have gone through the perf-tools-next tree:
https://lore.kernel.org/all/CAP-5=fV3NXkKsCP1WH0_qLRNpL+WuP8S3h1=cHaUMH5MFkVHQg@mail.gmail.com/
Arnaldo, please take Eric's patch in preference to this one.

So I removed your version from perf-tools-next, eventually we'll get this merged upstream, I'm just checking this isn't going to get in our way of testing what we have in perf-tools-next...

- Arnaldo

#11 lore raw

Re: [PATCH v1 4/5] perf pmu-events: Remove duplicated riscv firmware event

Quoted 4 lines · Eric Lin
Quoted 2 lines · Arnaldo Carvalho de Melo
Quoted 1 line · Arnaldo Carvalho de Melo

On Mon, Aug 05, 2024 at 09:05:26PM -0700, Ian Rogers wrote:

Quoted 8 lines
Quoted 5 lines
Quoted 3 lines
Quoted 3 lines · Eric Lin
Quoted 2 lines

I've sent a patch to fix it and the patch already merged. Thanks.

Also, already merged where? Upstream? In the riscv kernel tree? where?

The patch already merged to Linux upstream:
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=63ba5b0fb4f54db256ec43b3062b2606b383055d

Ok, I'm resolving this with removing Ian's patch and doing a merge with upstream to pick this patch and keep perf-tools-next building.

- Arnaldo

author reviewer
#12 lore raw

PATCH 5/5 perf jevents.py: Ensure event names aren't duplicated

In reply to Ian Rogers

Duplicate event names break invariants in perf list. Assert that an event name isn't duplicated so that broken json won't build.

---
 tools/perf/pmu-events/jevents.py | 4 ++++
 1 file changed, 4 insertions(+)
tools/perf/pmu-events/jevents.py +4 -0
index fcf0158438b5..1d96b2204e52 100755
@@ -503,8 +503,11 @@ def print_pending_events() -> None:
 
   first = True
   last_pmu = None
+  last_name = None
   pmus = set()
   for event in sorted(_pending_events, key=event_cmp_key):
+    if last_pmu and last_pmu == event.pmu:
+      assert event.name != last_name, f"Duplicate event: {last_pmu}/{last_name}/ in {_pending_events_tblname}"
     if event.pmu != last_pmu:
       if not first:
         _args.output_file.write('};\n')
@@ -516,6 +519,7 @@ def print_pending_events() -> None:
       pmus.add((event.pmu, pmu_name))
 
     _args.output_file.write(event.to_c_string(metric=False))
+    last_name = event.name
   _pending_events = []
 
   _args.output_file.write(f"""
Signature · 1 line
2.46.0.rc2.264.g509ed76dc8-goog