Skip to contentSkip to conversation

Revert "perf hist: Fix bogus profiles when filters are enabled"

PATCH In review Arnaldo Carvalho de Melo 3 participants 7 messages Aug 20, 2025 – Sep 1, 2025 perf events

lore
Share & tools
lore

Arnaldo Carvalho de Melo

author maintainer
#0 lore raw

PATCH 1/1 Revert "perf hist: Fix bogus profiles when filters are enabled"

This reverts commit 8b4799e4f0f40a4ec737bf870aa38d06288bf0fb.

Not combining entries in 'perf top', so we're getting multiple lines for the same symbol, with the same address.

To test it, simply run 'perf top', then do /acpi to see just symbols starting with acpi_ and notice that there are various lines with the same symbol, press V to see the address and its the same.

---
 tools/perf/util/hist.c | 11 +----------
 1 file changed, 1 insertion(+), 10 deletions(-)
tools/perf/util/hist.c +1 -10
index 64ff427040c34112..f3f64aff9b882303 100644
@@ -1391,16 +1391,7 @@ hist_entry__cmp_impl(struct perf_hpp_list *hpp_list, struct hist_entry *left,
 	struct hists *hists = left->hists;
 	struct perf_hpp_fmt *fmt;
 	perf_hpp_fmt_cmp_t *fn;
-	int64_t cmp;
-
-	/*
-	 * Never collapse filtered and non-filtered entries.
-	 * Note this is not the same as having an extra (invisible) fmt
-	 * that corresponds to the filtered status.
-	 */
-	cmp = (int64_t)!!left->filtered - (int64_t)!!right->filtered;
-	if (cmp)
-		return cmp;
+	int64_t cmp = 0;
 
 	perf_hpp_list__for_each_sort_list(hpp_list, fmt) {
 		if (ignore_dynamic && perf_hpp__is_dynamic_entry(fmt) &&
Signature · 1 line
2.50.1
#1 lore raw
Quoted 9 lines · Arnaldo Carvalho de Melo

This reverts commit 8b4799e4f0f40a4ec737bf870aa38d06288bf0fb.

Not combining entries in 'perf top', so we're getting multiple lines for the same symbol, with the same address.

To test it, simply run 'perf top', then do /acpi to see just symbols starting with acpi_ and notice that there are various lines with the same symbol, press V to see the address and its the same.

With this revert, does it show 1 entry but with a wrong percent? I am not sure why there are 2 entries for the same symbol, but if we merge them, we can sum of percents. Is it the right thing to do?

Quoted 30 lines
---
 tools/perf/util/hist.c | 11 +----------
 1 file changed, 1 insertion(+), 10 deletions(-)
tools/perf/util/hist.c +1 -11
index 64ff427040c34112..f3f64aff9b882303 100644
@@ -1391,16 +1391,7 @@ hist_entry__cmp_impl(struct perf_hpp_list *hpp_list, struct hist_entry *left,
        struct hists *hists = left->hists;
        struct perf_hpp_fmt *fmt;
        perf_hpp_fmt_cmp_t *fn;
-       int64_t cmp;
-
-       /*
-        * Never collapse filtered and non-filtered entries.
-        * Note this is not the same as having an extra (invisible) fmt
-        * that corresponds to the filtered status.
-        */
-       cmp = (int64_t)!!left->filtered - (int64_t)!!right->filtered;
-       if (cmp)
-               return cmp;
+       int64_t cmp = 0;
 
        perf_hpp_list__for_each_sort_list(hpp_list, fmt) {
                if (ignore_dynamic && perf_hpp__is_dynamic_entry(fmt) &&
--

2.50.1

maintainer
#2 lore raw

Hello,

Quoted 7 lines · Dmitry Vyukov
Quoted 6 lines · Arnaldo Carvalho de Melo

This reverts commit 8b4799e4f0f40a4ec737bf870aa38d06288bf0fb.

Not combining entries in 'perf top', so we're getting multiple lines for the same symbol, with the same address.

Quoted 7 lines
Quoted 3 lines

To test it, simply run 'perf top', then do /acpi to see just symbols starting with acpi_ and notice that there are various lines with the same symbol, press V to see the address and its the same.

With this revert, does it show 1 entry but with a wrong percent? I am not sure why there are 2 entries for the same symbol, but if we merge them, we can sum of percents. Is it the right thing to do?

I don't think it'd have a wrong percent. The hists maintain stats for filtered entries separately.

Based on the position of filtered entries in the RB tree, I think it might not merge correct samples together and create multiple entries with the same info.

Filtering by unused sort keys would be undefined. We probably want to warn users instead.

Thanks,
Namhyung

#3 lore raw
Quoted 15 lines · Namhyung Kim

Hello,

Quoted 11 lines · Dmitry Vyukov
Quoted 9 lines · Arnaldo Carvalho de Melo

This reverts commit 8b4799e4f0f40a4ec737bf870aa38d06288bf0fb.

Not combining entries in 'perf top', so we're getting multiple lines for the same symbol, with the same address.

To test it, simply run 'perf top', then do /acpi to see just symbols starting with acpi_ and notice that there are various lines with the same symbol, press V to see the address and its the same.

Quoted 6 lines
Quoted 3 lines

With this revert, does it show 1 entry but with a wrong percent? I am not sure why there are 2 entries for the same symbol, but if we merge them, we can sum of percents. Is it the right thing to do?

I don't think it'd have a wrong percent. The hists maintain stats for filtered entries separately.

I still don't fully follow.

If we merge a filtered entry into non-filtered entry (with the revert), now we attribute what was filtered out to the non-filtered entry, and the non-filtered entry has wrong overhead, no?

If we merge the other way around: non-filtered entry into filtered entry, then we won't show it at all.

Quoted 3 lines

Based on the position of filtered entries in the RB tree, I think it might not merge correct samples together and create multiple entries with the same info.

The second thing I don't understand: without this revert we don't merge filtered and non-filtered entries, top shows duplicate entries, does it mean it shows filtered out entries? This also looks wrong...

Quoted 2 lines

Filtering by unused sort keys would be undefined. We probably want to warn users instead.

Do you mean that the filtered=1 is set incorrectly in this case? Do you mean that with this revert 2 bugs just compensate each other by
luck: we wrongly set filtered=1, and then wrongly merge them together, so it all works out in the end?

maintainer
#4 lore raw
Quoted 20 lines · Dmitry Vyukov
Quoted 19 lines · Namhyung Kim

Hello,

Quoted 14 lines · Dmitry Vyukov
Quoted 9 lines · Arnaldo Carvalho de Melo

This reverts commit 8b4799e4f0f40a4ec737bf870aa38d06288bf0fb.

Not combining entries in 'perf top', so we're getting multiple lines for the same symbol, with the same address.

To test it, simply run 'perf top', then do /acpi to see just symbols starting with acpi_ and notice that there are various lines with the same symbol, press V to see the address and its the same.

With this revert, does it show 1 entry but with a wrong percent? I am not sure why there are 2 entries for the same symbol, but if we merge them, we can sum of percents. Is it the right thing to do?

Quoted 8 lines
Quoted 2 lines

I don't think it'd have a wrong percent. The hists maintain stats for filtered entries separately.

I still don't fully follow.

If we merge a filtered entry into non-filtered entry (with the revert), now we attribute what was filtered out to the non-filtered entry, and the non-filtered entry has wrong overhead, no?

Oh, my bad. I thought we track both periods in the hist_entry, but it seems it's only in the hists for total.

Quoted 11 lines

If we merge the other way around: non-filtered entry into filtered entry, then we won't show it at all.

Quoted 3 lines

Based on the position of filtered entries in the RB tree, I think it might not merge correct samples together and create multiple entries with the same info.

The second thing I don't understand: without this revert we don't merge filtered and non-filtered entries, top shows duplicate entries, does it mean it shows filtered out entries? This also looks wrong...

I checked those duplicated entries all non-filtered (filtered = 0). I suspect an entry can miss existing one (to be merged) when it sees unrelated filtered entries in the middle of RB tree traversal.

Quoted 8 lines
Quoted 3 lines

Filtering by unused sort keys would be undefined. We probably want to warn users instead.

Do you mean that the filtered=1 is set incorrectly in this case? Do you mean that with this revert 2 bugs just compensate each other by
luck: we wrongly set filtered=1, and then wrongly merge them together, so it all works out in the end?

I mean you should not allow users to use filters not listed in the sort keys. For example, perf report -s comm --tid=123 would return error. They can use perf report -s tid or perf report -s comm,tid -H.

Thanks,
Namhyung

#5 lore raw
Quoted 42 lines · Namhyung Kim
Quoted 20 lines · Dmitry Vyukov
Quoted 19 lines · Namhyung Kim

Hello,

Quoted 14 lines · Dmitry Vyukov
Quoted 9 lines · Arnaldo Carvalho de Melo

This reverts commit 8b4799e4f0f40a4ec737bf870aa38d06288bf0fb.

Not combining entries in 'perf top', so we're getting multiple lines for the same symbol, with the same address.

To test it, simply run 'perf top', then do /acpi to see just symbols starting with acpi_ and notice that there are various lines with the same symbol, press V to see the address and its the same.

With this revert, does it show 1 entry but with a wrong percent? I am not sure why there are 2 entries for the same symbol, but if we merge them, we can sum of percents. Is it the right thing to do?

Quoted 8 lines
Quoted 2 lines

I don't think it'd have a wrong percent. The hists maintain stats for filtered entries separately.

I still don't fully follow.

If we merge a filtered entry into non-filtered entry (with the revert), now we attribute what was filtered out to the non-filtered entry, and the non-filtered entry has wrong overhead, no?

Oh, my bad. I thought we track both periods in the hist_entry, but it seems it's only in the hists for total.

Quoted 8 lines

If we merge the other way around: non-filtered entry into filtered entry, then we won't show it at all.

Quoted 3 lines

Based on the position of filtered entries in the RB tree, I think it might not merge correct samples together and create multiple entries with the same info.

Quoted 7 lines
Quoted 3 lines

The second thing I don't understand: without this revert we don't merge filtered and non-filtered entries, top shows duplicate entries, does it mean it shows filtered out entries? This also looks wrong...

I checked those duplicated entries all non-filtered (filtered = 0). I suspect an entry can miss existing one (to be merged) when it sees unrelated filtered entries in the middle of RB tree traversal.

Does it mean that the sorting and merging predicates somehow end up being different, while the assumption is that they should be the same?

Will it help if filtered field is checked as the last condition, rather than the first? Then I assume unrelated entries should still be compared as they were before.

Quoted 11 lines
Quoted 7 lines
Quoted 2 lines

Filtering by unused sort keys would be undefined. We probably want to warn users instead.

Do you mean that the filtered=1 is set incorrectly in this case? Do you mean that with this revert 2 bugs just compensate each other by
luck: we wrongly set filtered=1, and then wrongly merge them together, so it all works out in the end?

I mean you should not allow users to use filters not listed in the sort keys. For example, perf report -s comm --tid=123 would return error. They can use perf report -s tid or perf report -s comm,tid -H.

Is it the root cause of the reported duplicate entries, or is it a separate issue?
Arnaldo said to use just 'perf top' w/o any flags, so I am not sure if this is related or not.

author maintainer
#6 lore raw
In reply to Dmitry Vyukov
Quoted 4 lines · Dmitry Vyukov
Quoted 1 line · Arnaldo Carvalho de Melo

This reverts commit 8b4799e4f0f40a4ec737bf870aa38d06288bf0fb.

Quoted 8 lines
Quoted 6 lines

Not combining entries in 'perf top', so we're getting multiple lines for the same symbol, with the same address.

To test it, simply run 'perf top', then do /acpi to see just symbols starting with acpi_ and notice that there are various lines with the same symbol, press V to see the address and its the same.

With this revert, does it show 1 entry but with a wrong percent?

Unsure, I have not investigated it further, just noticed the strange output with multiple entries for the same symbol, bisected and doing the revert found what looks like a familiar output.

It is possible that the familiar output is problematic, but I haven't dig deeper so far.

Since it is a change of behaviour, can we apply the revert and continue the discussion from that point?

'perf top' decays histogram entries, which is in addition to the "static" processing done in 'perf report', maybe this is a problem that is there since forever but statistically, by means of decaying, goes away and only now we're noticing? Merits investigation.

- Arnaldo

Quoted 33 lines

I am not sure why there are 2 entries for the same symbol, but if we merge them, we can sum of percents. Is it the right thing to do?

Quoted 30 lines
---
 tools/perf/util/hist.c | 11 +----------
 1 file changed, 1 insertion(+), 10 deletions(-)
tools/perf/util/hist.c +1 -11
index 64ff427040c34112..f3f64aff9b882303 100644
@@ -1391,16 +1391,7 @@ hist_entry__cmp_impl(struct perf_hpp_list *hpp_list, struct hist_entry *left,
        struct hists *hists = left->hists;
        struct perf_hpp_fmt *fmt;
        perf_hpp_fmt_cmp_t *fn;
-       int64_t cmp;
-
-       /*
-        * Never collapse filtered and non-filtered entries.
-        * Note this is not the same as having an extra (invisible) fmt
-        * that corresponds to the filtered status.
-        */
-       cmp = (int64_t)!!left->filtered - (int64_t)!!right->filtered;
-       if (cmp)
-               return cmp;
+       int64_t cmp = 0;
 
        perf_hpp_list__for_each_sort_list(hpp_list, fmt) {
                if (ignore_dynamic && perf_hpp__is_dynamic_entry(fmt) &&
--

2.50.1