perf/cxlpmu: Misc sashiko raised issues fixes
Dave Jiang
maintainerv4 cover perf/cxlpmu: Misc sashiko raised issues fixes
[resend, mangled linux-cxl mailing addr]
Teed off of Davidlohr's misc cxlpmu series [1]. I had Claude pick up all the sashiko raised issues and then continued multiple internal review iterations to pick up a number of fixes. I'm no CXL PMU or perf expert, but the fixes look reasonable to me AFAICT.
Since v3:
- 4/11 and 5/11 swapped, so the one-line MSI vector 0 fix comes before the info->msi_vec rename and backports on its own (Robin).
- 6/11: new patch, add the PMUs after configuring events.
- 7/11: no code change; the commit message no longer overstates what dropping IRQF_SHARED costs a device that shares a vector.
- 8/11: quote the spec sentence that makes the freeze sticky, and correct the claim about what the migration window leaves enabled (Richard, sashiko-bot).
- 10/11: split the probe-time overflow clear out, so this is back to the single hunk Jonathan acked.
- 11/11: new patch. Clear a counter's stale overflow status when starting an event, which is where the reuse case bites; the probe clear moved here from 10/11 (Richard).
Since v2:
- 1/11: check event_idx alongside counter_idx (Jonathan).
- 2/11: use FIELD_MODIFY() rather than mask-then-OR (Jonathan).
- 3/11: rewrite the rationale, no code change (Jonathan).
- 5/11: new patch, split the MSI vector out of info->irq (Robin, Jonathan).
- 7/11: drop IRQF_SHARED as well as adding IRQF_NOBALANCING, and retitle (Jonathan, Robin).
- 8/11: don't unfreeze if the PMU has been disabled meanwhile (Robin, Jonathan).
- 10/11: retitle, and drop the cxl_pmu_offline_cpu() hunk since that dev_err() cannot be reached (Robin).
- Dropped the old 9/9 that guarded cpumask_show() against cpumask_of(-1). The init case was a false positive and neither reviewer liked the shape of it (Jonathan, Robin).
Since v1:
- Updated 3/9 to address sashiko issue.
- No other changes as sashiko only raised issue with pre-existing.
Three follow-ups this series does not attempt:
cxl_pmu_read() relies on local64_t being same-CPU, which is the only reason the interrupt has to be pinned at all. Moving prev_count and event->count off local64_t would make the affinity question moot, and Robin may have a view on whether that is worth doing.
cxl_pmu_offline_cpu() still leaves on_cpu at -1 across a sleeping perf_pmu_migrate_context(), and still has an unreachable no-target branch. Both want removing, possibly on top of the generic PMU hotplug rework Jonathan pointed at [2].
Firmware can leave a counter enabled with Global Freeze on Overflow but not Interrupt on Overflow, which stalls the whole CPMU with no interrupt to say so. Quiescing the block at probe needs two writes per counter to stay within the spec's rule about changing an enabled counter, so it belongs in its own patch.
[1]: https://lore.kernel.org/linux-cxl/[email protected]/
[2]: https://lore.kernel.org/linux-arm-kernel/[email protected]/
Dave Jiang (11):
perf/cxl: Program the requested event group on configurable counters perf/cxl: Clear stale event fields before reprogramming a counter perf/cxl: Fix the counter overflow delta fixup
perf/cxl: Accept an overflow interrupt on MSI message number 0 perf/cxl: Split the MSI vector out of info->irq
cxl/pci: Add the PMUs after configuring events
perf/cxl: Don't share the overflow interrupt, and keep it pinned perf/cxl: Unfreeze counters after handling an overflow interrupt perf/cxl: Validate the hardware-reported counter width perf/cxl: Don't log through pmu.dev in the overflow interrupt handler perf/cxl: Clear stale overflow status before using a counter
drivers/cxl/pci.c | 11 +++-- drivers/perf/cxl_pmu.c | 107 ++++++++++++++++++++++++++++++++--------- 2 files changed, 90 insertions(+), 28 deletions(-)
base-commit: f5098b6bae761e346ebcd9da7f95622c04733cff
Signature · 1 line
2.54.0
Dave Jiang
maintainerPATCH RESEND v4 1/11 perf/cxl: Program the requested event group on configurable counters
cxl_pmu_get_event_idx() sets *counter_idx for a configurable counter but never *event_idx. hwc->event_base stays 0, so cxl_pmu_event_start() programs event group 0 no matter which group the user asked for.
Return the matched capability's event_idx.
--- drivers/perf/cxl_pmu.c | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-)
drivers/perf/cxl_pmu.c +6 -2
@@ -513,7 +513,10 @@ static const struct attribute_group *cxl_pmu_attr_groups[] = { NULL }; -/* If counter_idx == NULL, don't try to allocate a counter. */ +/* +* If counter_idx == NULL, don't try to allocate a counter. Callers that +* allocate pass both counter_idx and event_idx. +*/ static int cxl_pmu_get_event_idx (struct perf_event *event,int *counter_idx,int *event_idx) {
@@ -541,7 +544,7 @@ static int cxl_pmu_get_event_idx(struct perf_event *event, int *counter_idx, pmu_ev =cxl_pmu_find_config_counter_ev_cap (info, vid, gid, mask);if (!IS_ERR (pmu_ev)) { -if (!counter_idx) +if (!counter_idx || !event_idx)return 0 ; bitmap_andnot(configurable_and_free, info->conf_counter_bm,
@@ -552,6 +555,7 @@ static int cxl_pmu_get_event_idx(struct perf_event *event, int *counter_idx,return -EINVAL ; *counter_idx = i; + *event_idx = pmu_ev->event_idx;return 0 ; }
Signature · 1 line
2.54.0
[email protected]
AI reviewRe: [RESEND PATCH v4 01/11] perf/cxl: Program the requested event group on configurable counters
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
Pre-existing issues:
- [High] Missing unfreeze of CPMU counters in the IRQ handler causes the PMU to permanently stop counting after the first overflow.
- [High] Incorrect wrap-around arithmetic in __cxl_pmu_read double-counts events on overflow and triggers undefined behavior. --
--- Patch [1]: [RESEND PATCH v4 01/11] perf/cxl: Program the requested event group on configurable counters --- commit eb034a4b2ed07ff56c863ab7711fcb1eb6012a8b
Author: Dave Jiang <[email protected]>
perf/cxl: Program the requested event group on configurable counters
cxl_pmu_get_event_idx() sets *counter_idx for a configurable counter but never *event_idx. hwc->event_base stays 0, so cxl_pmu_event_start() programs event group 0 no matter which group the user asked for. This commit fixes the issue by returning the matched capability's event_idx.
Quoted 4 lines
drivers/perf/cxl_pmu.c +0
-0
[ ... ]
drivers/perf/cxl_pmu.c static int cxl_pmu_get_event_idx(struct perf_event *event, int *counter_idx, · line 550
Quoted 9 lines
drivers/perf/cxl_pmu.c +1
-1@@ -541,7 +544,7 @@ static int cxl_pmu_get_event_idx(struct perf_event *event, int *counter_idx, pmu_ev =cxl_pmu_find_config_counter_ev_cap (info, vid, gid, mask);if (!IS_ERR (pmu_ev)) { -if (!counter_idx) +if (!counter_idx || !event_idx)return 0 ; bitmap_andnot(configurable_and_free, info->conf_counter_bm,
While reviewing this patch, I noticed a couple of pre-existing issues in this file:
drivers/perf/cxl_pmu.c:__cxl_pmu_read() {
...
delta = (new_cnt - prev_cnt) & GENMASK_ULL(info->counter_width - 1, 0);
if (overflow && delta < GENMASK_ULL(info->counter_width - 1, 0))
delta += (1UL << info->counter_width);
...
}[Severity: High]
This isn't a bug introduced by this patch, but does this wrap-around arithmetic double-count events on overflow and trigger undefined behavior?
Because the wrap-around is already accounted for in delta during the unsigned subtraction, adding 2^width erroneously double-counts the overflow, injecting an enormous spurious value into event->count every time an interrupt fires.
Furthermore, if the hardware specifies a 64-bit counter (info->counter_width == 64), evaluating 1UL << 64 invokes undefined behavior in C.
drivers/perf/cxl_pmu.c:cxl_pmu_irq() {
...
__cxl_pmu_read(event, true);
}
writeq(overflowed, base + CXL_PMU_OVERFLOW_REG);
return IRQ_HANDLED;
}[Severity: High]
This is a pre-existing issue, but does this interrupt handler need to write 0 to CXL_PMU_FREEZE_REG to unfreeze the counters?
In cxl_pmu_event_start(), the configuration sets CXL_PMU_COUNTER_CFG_FREEZE_ON_OVRFLW. According to the CXL 3.0 specification, when this is set, an overflow causes the hardware to freeze counters by setting bits in the Freeze Register.
The IRQ handler clears the overflow register (CXL_PMU_OVERFLOW_REG) but fails to explicitly clear the freeze register (CXL_PMU_FREEZE_REG). Consequently, the counters remain permanently frozen and stop generating events after processing the first overflow.
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1
Dave Jiang
maintainerPATCH RESEND v4 2/11 perf/cxl: Clear stale event fields before reprogramming a counter
In reply to Dave Jiangcxl_pmu_event_start() ORs the event group id, event mask, edge and invert selections into a configurable counter's config register without clearing them first, and cxl_pmu_event_stop() leaves them set. Reuse the counter for another event and the new selection lands on top of the old one, so the counter counts the wrong events with stale edge and invert behaviour.
Use FIELD_MODIFY() so each field is replaced rather than accumulated.
--- drivers/perf/cxl_pmu.c | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-)
drivers/perf/cxl_pmu.c +8 -8
@@ -646,17 +646,17 @@ static void cxl_pmu_event_start(struct perf_event *event, int flags) cfg |=FIELD_PREP (CXL_PMU_COUNTER_CFG_INT_ON_OVRFLW ,1 ); cfg |=FIELD_PREP (CXL_PMU_COUNTER_CFG_FREEZE_ON_OVRFLW ,1 ); cfg |=FIELD_PREP (CXL_PMU_COUNTER_CFG_ENABLE ,1 ); - cfg |=FIELD_PREP (CXL_PMU_COUNTER_CFG_EDGE , -cxl_pmu_config1_get_edge (event) ?1 :0 ); - cfg |=FIELD_PREP (CXL_PMU_COUNTER_CFG_INVERT , -cxl_pmu_config1_get_invert (event) ?1 :0 ); +FIELD_MODIFY (CXL_PMU_COUNTER_CFG_EDGE , &cfg, +cxl_pmu_config1_get_edge (event) ?1 :0 ); +FIELD_MODIFY (CXL_PMU_COUNTER_CFG_INVERT , &cfg, +cxl_pmu_config1_get_invert (event) ?1 :0 );/* Fixed purpose counters have next two fields RO */ if (test_bit (hwc->idx, info->conf_counter_bm)) { - cfg |=FIELD_PREP (CXL_PMU_COUNTER_CFG_EVENT_GRP_ID_IDX_MSK , - hwc->event_base); - cfg |=FIELD_PREP (CXL_PMU_COUNTER_CFG_EVENTS_MSK , -cxl_pmu_config_get_mask (event)); +FIELD_MODIFY (CXL_PMU_COUNTER_CFG_EVENT_GRP_ID_IDX_MSK , &cfg, + hwc->event_base); +FIELD_MODIFY (CXL_PMU_COUNTER_CFG_EVENTS_MSK , &cfg, +cxl_pmu_config_get_mask (event)); } cfg &= ~CXL_PMU_COUNTER_CFG_THRESHOLD_MSK ;/*
Signature · 1 line
2.54.0
[email protected]
AI reviewRe: [RESEND PATCH v4 02/11] perf/cxl: Clear stale event fields before reprogramming a counter
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
Pre-existing issues:
- [High] Incorrect 64-bit write (writeq) to a 32-bit hardware filter configuration register.
- [High] Incorrect PMU counter overflow handling logic leads to massive overcounting and Undefined Behavior (UB) via out-of-bounds bitwise shift. --
--- Patch [2]: [RESEND PATCH v4 02/11] perf/cxl: Clear stale event fields before reprogramming a counter --- commit 215e20dd2e223d030442e8b7a911140cd3ebb0bf
Author: Dave Jiang <[email protected]>
perf/cxl: Clear stale event fields before reprogramming a counter
This patch fixes an issue in cxl_pmu_event_start() where previous event settings were accumulated instead of cleared. It uses FIELD_MODIFY() to ensure event group ID, mask, edge, and invert selections replace the old values, preventing counters from using stale configuration.
Quoted 20 lines
drivers/perf/cxl_pmu.c +4
-5@@ -646,17 +646,17 @@ static void cxl_pmu_event_start(struct perf_event *event, int flags) cfg |=FIELD_PREP (CXL_PMU_COUNTER_CFG_INT_ON_OVRFLW ,1 ); cfg |=FIELD_PREP (CXL_PMU_COUNTER_CFG_FREEZE_ON_OVRFLW ,1 ); cfg |=FIELD_PREP (CXL_PMU_COUNTER_CFG_ENABLE ,1 ); - cfg |=FIELD_PREP (CXL_PMU_COUNTER_CFG_EDGE , -cxl_pmu_config1_get_edge (event) ?1 :0 ); - cfg |=FIELD_PREP (CXL_PMU_COUNTER_CFG_INVERT , -cxl_pmu_config1_get_invert (event) ?1 :0 ); +FIELD_MODIFY (CXL_PMU_COUNTER_CFG_EDGE , &cfg, +cxl_pmu_config1_get_edge (event) ?1 :0 ); +FIELD_MODIFY (CXL_PMU_COUNTER_CFG_INVERT , &cfg, +cxl_pmu_config1_get_invert (event) ?1 :0 );/* Fixed purpose counters have next two fields RO */ if (test_bit (hwc->idx, info->conf_counter_bm)) { - cfg |= FIELD_PREP(CXL_PMU_COUNTER_CFG_EVENT_GRP_ID_IDX_MSK,
drivers/perf/cxl_pmu.c static void cxl_pmu_event_start(struct perf_event *event, int flags) · line 662
Quoted 10 lines
drivers/perf/cxl_pmu.c +4
-3- hwc->event_base); - cfg |=FIELD_PREP (CXL_PMU_COUNTER_CFG_EVENTS_MSK , -cxl_pmu_config_get_mask (event)); +FIELD_MODIFY (CXL_PMU_COUNTER_CFG_EVENT_GRP_ID_IDX_MSK , &cfg, + hwc->event_base); +FIELD_MODIFY (CXL_PMU_COUNTER_CFG_EVENTS_MSK , &cfg, +cxl_pmu_config_get_mask (event)); } cfg &= ~CXL_PMU_COUNTER_CFG_THRESHOLD_MSK ;/*
[Severity: High]
This isn't a bug introduced by this patch, but while looking at cxl_pmu_event_start(), does this writeq() overwrite the configuration for the adjacent filter?
drivers/perf/cxl_pmu.c:cxl_pmu_event_start() {
...
if (info->filter_hdm) {
if (cxl_pmu_config1_hdm_filter_en(event))
cfg = cxl_pmu_config2_get_hdm_decoder(event);
else
cfg = GENMASK(31, 0); /* No filtering if 0xFFFF_FFFF */
writeq(cfg, base + CXL_PMU_FILTER_CFG_REG(hwc->idx, 0));
}
...
}CXL_PMU_FILTER_CFG_REG has a 4-byte stride, making it a 32-bit hardware register. Using the 64-bit writeq() here spills into filter 1. If filter 0 is disabled (cfg = 0xFFFFFFFF), writeq() writes 0x00000000 to filter 1, which would inadvertently enable filter 1 with a 0-match condition.
[Severity: High]
This is a pre-existing issue, but I also noticed a potential undefined behavior in the overflow handling logic nearby:
drivers/perf/cxl_pmu.c:__cxl_pmu_read() {
...
delta = (new_cnt - prev_cnt) & GENMASK_ULL(info->counter_width - 1, 0);
if (overflow && delta < GENMASK_ULL(info->counter_width - 1, 0))
delta += (1UL << info->counter_width);
local64_add(delta, &event->count);
}Since delta already accounts for one wraparound via the bitwise AND when new_cnt is less than prev_cnt, does unconditionally adding 2^width on overflow cause massive overcounting?
Also, can (1UL << info->counter_width) cause an out-of-bounds bitwise shift resulting in undefined behavior? The CXL specification allows counter_width to be 64, which would shift beyond the width of an unsigned long, and on 32-bit architectures this could happen for a width of 32 or greater.
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=2
Re: [RESEND PATCH v4 02/11] perf/cxl: Clear stale event fields before reprogramming a counter
In reply to Dave JiangQuoted 6 lines · On Wed, 5 Aug 2026 08:59:02 -0700 Dave Jiang <[email protected]> wrote:
cxl_pmu_event_start()ORs the event group id, event mask, edge and invert selections into a configurable counter's config register without clearing them first, andcxl_pmu_event_stop()leaves them set. Reuse the counter for another event and the new selection lands on top of the old one, so the counter counts the wrong events with stale edge and invert behaviour.
Quoted 7 lines
Use
FIELD_MODIFY()so each field is replaced rather than accumulated.Fixes 5d7107c72796 ("perf: CXL Performance Monitoring Unit driver")Reported-by [email protected]Assisted-by Claude:claude-opus-4-8Signed-off-by Dave Jiang <[email protected]>
Dave Jiang
maintainerPATCH RESEND v4 3/11 perf/cxl: Fix the counter overflow delta fixup
In reply to Dave JiangThe counter is masked to counter_width, so the subtraction in __cxl_pmu_read() throws away the bit that records the wrap. A delta of one whole period reads back as 0, the same as no events at all, and only the overflow status tells the two apart - which is why __cxl_pmu_read() takes an overflow argument.
The fixup keys off the delta rather than the operands:
delta = (new_cnt - prev_cnt) & GENMASK_ULL(counter_width - 1, 0); if (overflow && delta < GENMASK_ULL(counter_width - 1, 0)) delta += (1UL << counter_width);
so it cannot tell which of these it is looking at:
event_start polled read wrap ctr = 0 .......... ctr = P/2 ......... mask -> 0 (+r) prev = 0 prev = P/2 IRQ reads new = r
prev = 0 (no read yet): new >= prev, fell short -> add period prev = P/2 (polled): new < prev, spans wrap -> add nothing
Both rows are the same interrupt and the old guard adds a period in both, so a mid-period read makes the event over-count. 'perf stat -I' hits that.
Condition the fixup on new_cnt >= prev_cnt, the one case the subtraction cannot express. Dropping it outright would break the first row. It stays exact whatever the residual r is, which matters because some events increment by more than 1 per cycle (CXL r4.0 8.2.7.2.1, Threshold) and can step past 0 as they wrap.
Use mask + 1 for the period rather than a shift. It is 0 for a 64-bit counter, and avoids the old 1UL << counter_width - undefined for width 64, and for >= 32 on 32-bit kernels.
--- drivers/perf/cxl_pmu.c | 16 ++++++++++------ 1 file changed, 10 insertions(+), 6 deletions(-)
drivers/perf/cxl_pmu.c +10 -6
@@ -689,7 +689,7 @@ static void __cxl_pmu_read(struct perf_event *event, bool overflow) {struct cxl_pmu_info *info =pmu_to_cxl_pmu_info (event->pmu);struct hw_perf_event *hwc = &event->hw; -u64 new_cnt, prev_cnt, delta; +u64 new_cnt, prev_cnt, delta, mask; do { prev_cnt = local64_read(&hwc->prev_count);
@@ -697,12 +697,16 @@ static void __cxl_pmu_read(struct perf_event *event, bool overflow) }while (local64_cmpxchg (&hwc->prev_count, prev_cnt, new_cnt) != prev_cnt);/* -* If we know an overflow occur then take that into account. -* Note counter is not reset as that would lose events +* The mask discards the bit that says the counter wrapped, so a delta of +* one whole period reads back as 0 - the same as no events at all. Only +* the overflow status separates them, and new_cnt >= prev_cnt is that +* case, so add the period back. mask + 1 is 2^counter_width, which comes +* out as 0 for a 64-bit counter and avoids an undefined 1 << 64. */ - delta = (new_cnt - prev_cnt) &GENMASK_ULL (info->counter_width -1 ,0 ); -if (overflow && delta <GENMASK_ULL (info->counter_width -1 ,0 )) - delta += (1UL << info->counter_width); + mask =GENMASK_ULL (info->counter_width -1 ,0 ); + delta = (new_cnt - prev_cnt) & mask; +if (overflow && new_cnt >= prev_cnt) + delta += mask +1 ;local64_add (delta, &event->count); }
Signature · 1 line
2.54.0
Re: [RESEND PATCH v4 03/11] perf/cxl: Fix the counter overflow delta fixup
Quoted 34 lines · On Wed, 5 Aug 2026 08:59:03 -0700 Dave Jiang <[email protected]> wrote:
The counter is masked to
counter_width, so the subtraction in__cxl_pmu_read()throws away the bit that records the wrap. A delta of one whole period reads back as 0, the same as no events at all, and only the overflow status tells the two apart - which is why__cxl_pmu_read()takes an overflow argument.The fixup keys off the delta rather than the operands:
delta = (new_cnt - prev_cnt) & GENMASK_ULL(counter_width - 1, 0); if (overflow && delta < GENMASK_ULL(counter_width - 1, 0)) delta += (1UL << counter_width);so it cannot tell which of these it is looking at:
event_start polled read wrap ctr = 0 .......... ctr = P/2 ......... mask -> 0 (+r) prev = 0 prev = P/2 IRQ reads new = rprev = 0 (no read yet): new >= prev, fell short -> add period prev = P/2 (polled): new < prev, spans wrap -> add nothingBoth rows are the same interrupt and the old guard adds a period in both, so a mid-period read makes the event over-count. 'perf stat -I' hits that.
Condition the fixup on
new_cnt>=prev_cnt, the one case the subtraction cannot express. Dropping it outright would break the first row. It stays exact whatever the residual r is, which matters because some events increment by more than 1 per cycle (CXL r4.0 8.2.7.2.1, Threshold) and can step past 0 as they wrap.Use mask + 1 for the period rather than a shift. It is 0 for a 64-bit counter, and avoids the old 1UL <<
counter_width- undefined for width 64, and for >= 32 on 32-bit kernels.
Quoted 5 lines
Fixes 5d7107c72796 ("perf: CXL Performance Monitoring Unit driver")Reported-by [email protected]Assisted-by Claude:claude-opus-4-8Signed-off-by Dave Jiang <[email protected]>
Dave Jiang
maintainerPATCH RESEND v4 4/11 perf/cxl: Accept an overflow interrupt on MSI message number 0
In reply to Dave Jiangcxl_pmu_probe() rejects the PMU when info->irq <= 0, but at that point the field still holds the MSI/MSI-X message number the device signals overflow on, not a Linux virq. That number is 0-based, and -1 means no interrupt support. Message number 0 is valid and pci_irq_vector() takes a 0-based index, so a compliant device signalling on the first vector fails to probe.
Reject only the no-interrupt case, matching how the CXL mailbox and event interrupts handle it.
---
v4:
- Move ahead of the info->msi_vec rename so this one-liner stands alone for backporting (Robin).
--- drivers/perf/cxl_pmu.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/perf/cxl_pmu.c +1 -1
@@ -873,7 +873,7 @@ static int cxl_pmu_probe(struct device *dev) .capabilities = PERF_PMU_CAP_NO_EXCLUDE, }; -if (info->irq <=0 ) +if (info->irq <0 )return -EINVAL ; rc =pci_irq_vector (pdev, info->irq);
Signature · 1 line
2.54.0
Dave Jiang
maintainerPATCH RESEND v4 5/11 perf/cxl: Split the MSI vector out of info->irq
In reply to Dave Jianginfo->irq means two different things depending on how far probe has got. cxl_pmu_parse_caps() sets it to the MSI/MSI-X message number from the CPMU Capability register. cxl_pmu_probe() passes that to pci_irq_vector() and then overwrites it with the Linux virq, which is what the hotplug callbacks hand to irq_set_affinity().
Both are ints called irq, so nothing catches a mix-up, and the valid ranges differ. 0 is a fine message number but never a valid virq.
Give the message number its own field. No functional change.
---
v4:
- Move after the one-line fix so that fix has no dependency on this rename (Robin).
--- drivers/perf/cxl_pmu.c | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-)
drivers/perf/cxl_pmu.c +5 -4
@@ -106,6 +106,7 @@ struct cxl_pmu_info {int on_cpu;struct hlist_node node;bool filter_hdm; +int msi_vec;int irq; };
@@ -143,9 +144,9 @@ static int cxl_pmu_parse_caps(struct device *dev, struct cxl_pmu_info *info) info->filter_hdm =FIELD_GET (CXL_PMU_CAP_FILTERS_SUP_MSK , val) &CXL_PMU_FILTER_HDM ;if (FIELD_GET (CXL_PMU_CAP_INT , val)) - info->irq =FIELD_GET (CXL_PMU_CAP_MSI_N_MSK , val); + info->msi_vec =FIELD_GET (CXL_PMU_CAP_MSI_N_MSK , val);else - info->irq =-1 ; + info->msi_vec =-1 ;/* First handle fixed function counters; note if configurable counters found */ for (i =0 ; i < info->num_counters; i++) {
@@ -873,10 +874,10 @@ static int cxl_pmu_probe(struct device *dev) .capabilities = PERF_PMU_CAP_NO_EXCLUDE, }; -if (info->irq <0 ) +if (info->msi_vec <0 )return -EINVAL ; - rc =pci_irq_vector (pdev, info->irq); + rc =pci_irq_vector (pdev, info->msi_vec);if (rc <0 )return rc; irq = rc;
Signature · 1 line
2.54.0
Re: [RESEND PATCH v4 05/11] perf/cxl: Split the MSI vector out of info->irq
Quoted 6 lines · On Wed, 5 Aug 2026 08:59:05 -0700 Dave Jiang <[email protected]> wrote:
info->irqmeans two different things depending on how far probe has got.cxl_pmu_parse_caps()sets it to the MSI/MSI-X message number from the CPMU Capability register.cxl_pmu_probe()passes that topci_irq_vector()and then overwrites it with the Linux virq, which is what the hotplug callbacks hand toirq_set_affinity().
Quoted 8 lines
Both are ints called irq, so nothing catches a mix-up, and the valid ranges differ. 0 is a fine message number but never a valid virq.
Give the message number its own field. No functional change.
Assisted-by Claude:claude-opus-4-8Reviewed-by Robin Murphy <[email protected]>Signed-off-by Dave Jiang <[email protected]>
Dave Jiang
maintainerPATCH RESEND v4 6/11 cxl/pci: Add the PMUs after configuring events
In reply to Dave Jiangcxl_pci_probe() adds the CPMU instances before configuring the event interrupts. Both take an MSI/MSI-X vector the device names in its own capability registers, and the spec lets a device point several capabilities at one message number, so the two can collide.
They do not fail the same way. A failed devm_cxl_pmu_add() only breaks out of the loop, while a failed cxl_event_config() aborts probe and the memory device never appears. Going PMUs-first therefore lets a peripheral facility take the vector and deny it to one that matters.
Configure events first, so a collision costs the PMU rather than the memdev. Return 0 explicitly afterwards, since rc still holds any PMU error that cxl_event_config() used to overwrite on the way past.
---
v4:
- New patch, ahead of the PMU dropping IRQF_SHARED. Without this that change can turn a shared vector into a probe failure for the whole memdev rather than a missing PMU (sashiko-bot).
--- drivers/cxl/pci.c | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-)
drivers/cxl/pci.c +6 -5
@@ -894,6 +894,10 @@ static int cxl_pci_probe(struct pci_dev *pdev, const struct pci_device_id *id)if (rc)dev_dbg (&pdev->dev,"No CXL FWCTL setup\n" ); + rc =cxl_event_config (host_bridge, mds, irq_avail); +if (rc) +return rc; + pmu_count =cxl_count_regblock (pdev,CXL_REGLOC_RBI_PMU );if (pmu_count <0 )return pmu_count;
@@ -920,13 +924,10 @@ static int cxl_pci_probe(struct pci_dev *pdev, const struct pci_device_id *id) } } - rc =cxl_event_config (host_bridge, mds, irq_avail); -if (rc) -return rc; -pci_save_state (pdev); -return rc; +/* A missing PMU is not fatal, the memdev is still usable */ +return 0 ; }static const struct pci_device_id cxl_mem_pci_tbl[] = {
Signature · 1 line
2.54.0
Re: [RESEND PATCH v4 06/11] cxl/pci: Add the PMUs after configuring events
Quoted 5 lines · Dave Jiang
cxl_pci_probe()adds the CPMU instances before configuring the event interrupts. Both take an MSI/MSI-X vector the device names in its own capability registers, and the spec lets a device point several capabilities at one message number, so the two can collide.
Quoted 8 lines
They do not fail the same way. A failed
devm_cxl_pmu_add()only breaks out of the loop, while a failedcxl_event_config()aborts probe and the memory device never appears. Going PMUs-first therefore lets a peripheral facility take the vector and deny it to one that matters.Configure events first, so a collision costs the PMU rather than the memdev. Return 0 explicitly afterwards, since rc still holds any PMU error that
cxl_event_config()used to overwrite on the way past.
Re: [RESEND PATCH v4 06/11] cxl/pci: Add the PMUs after configuring events
In reply to Dave JiangQuoted 14 lines · On Wed, 5 Aug 2026 08:59:06 -0700 Dave Jiang <[email protected]> wrote:
cxl_pci_probe()adds the CPMU instances before configuring the event interrupts. Both take an MSI/MSI-X vector the device names in its own capability registers, and the spec lets a device point several capabilities at one message number, so the two can collide.They do not fail the same way. A failed
devm_cxl_pmu_add()only breaks out of the loop, while a failedcxl_event_config()aborts probe and the memory device never appears. Going PMUs-first therefore lets a peripheral facility take the vector and deny it to one that matters.Configure events first, so a collision costs the PMU rather than the memdev. Return 0 explicitly afterwards, since rc still holds any PMU error that
cxl_event_config()used to overwrite on the way past.
Quoted 7 lines
Assisted-by Claude:claude-opus-4-8Signed-off-by Dave Jiang <[email protected]>---
v4:
- New patch, ahead of the PMU droppingIRQF_SHARED. Without this that change can turn a shared vector into a probe failure for the whole memdev rather than a missing PMU (sashiko-bot).
Make sense. Anyone crazy enough to share this interrupt gets to keep the pieces.
Dave Jiang
maintainerPATCH RESEND v4 7/11 perf/cxl: Don't share the overflow interrupt, and keep it pinned
In reply to Dave JiangThe PMU pins its overflow interrupt to info->on_cpu in the hotplug callbacks, but requests it with only IRQF_SHARED | IRQF_NO_THREAD. Without IRQF_NOBALANCING, irqbalance or a userspace smp_affinity write can move it elsewhere, and cxl_pmu_irq() then runs local64_cmpxchg() and local64_add() on hwc->prev_count and event->count at the same time as the managing CPU. local64_t is only atomic against same-CPU access, so the counts corrupt.
The flag alone does not fix that while the line is shared. __setup_irq() only acts on it for the first action on a line, and a co-owner has no reason to want our affinity - it would keep taking the interrupt wherever its own points, running our handler on the wrong CPU.
Drop IRQF_SHARED and add IRQF_NOBALANCING. The spec only recommends a distinct Interrupt Message Number per CPMU instance (CXL r4.0 8.2.7.1.1), so a device may put several on one vector, or share with the mailbox or event log. Such a device now loses a PMU rather than silently miscounting - and only the PMU, since the preceding patch adds the CPMUs after the event configuration.
--- drivers/perf/cxl_pmu.c | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-)
drivers/perf/cxl_pmu.c +9 -2
@@ -783,7 +783,7 @@ static irqreturn_t cxl_pmu_irq(int irq, void *data) overflowed =readq (base +CXL_PMU_OVERFLOW_REG ); -/* Interrupt may be shared, so maybe it isn't ours */ +/* Nothing overflowed, so the device did not raise this */ if (!overflowed)return IRQ_NONE ;
@@ -886,7 +886,14 @@ static int cxl_pmu_probe(struct device *dev)if (!irq_name)return -ENOMEM ; - rc =devm_request_irq (dev, irq, cxl_pmu_irq,IRQF_SHARED |IRQF_NO_THREAD , +/* +* The handler must run on info->on_cpu, so the interrupt cannot be +* shared - IRQF_NOBALANCING is only honoured for the first action on a +* line, and a co-owner would keep taking the interrupt wherever its own +* affinity points. +*/ + rc =devm_request_irq (dev, irq, cxl_pmu_irq, +IRQF_NO_THREAD |IRQF_NOBALANCING , irq_name, info);if (rc)return rc;
Signature · 1 line
2.54.0
Dave Jiang
maintainerPATCH RESEND v4 8/11 perf/cxl: Unfreeze counters after handling an overflow interrupt
In reply to Dave JiangThe counters run with Freeze on Overflow set, so one overflow freezes every counter in the block (CXL r4.0 8.2.7.2.1), and a frozen counter "remains frozen until explicitly unfrozen by software" (8.2.7.1.3, Table 8-183). cxl_pmu_irq() reads the overflowed counters and clears the overflow status but never unfreezes, so everything stays frozen until the next pmu_enable() and events in that window are lost.
Unfreeze after clearing the status, unless the PMU has been disabled in the meantime - cxl_pmu_disable() freezes the block deliberately. Track the enabled state and leave it frozen in that case.
--- drivers/perf/cxl_pmu.c | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+)
drivers/perf/cxl_pmu.c +19 -0
@@ -108,6 +108,8 @@ struct cxl_pmu_info {bool filter_hdm;int msi_vec;int irq; +/* Set between pmu_enable() and pmu_disable(), read by the IRQ handler */ +bool enabled; };#define pmu_to_cxl_pmu_info (_pmu) container_of(_pmu, struct cxl_pmu_info, pmu)
@@ -596,6 +598,7 @@ static void cxl_pmu_enable(struct pmu *pmu)void __iomem *base = info->base;/* Can assume frozen at this stage */ +WRITE_ONCE (info->enabled, true);writeq (0 , base +CXL_PMU_FREEZE_REG ); }
@@ -604,6 +607,7 @@ static void cxl_pmu_disable(struct pmu *pmu)struct cxl_pmu_info *info =pmu_to_cxl_pmu_info (pmu);void __iomem *base = info->base; +WRITE_ONCE (info->enabled, false);/* * Whilst bits above number of counters are RsvdZ * they are unlikely to be repurposed given
@@ -802,6 +806,21 @@ static irqreturn_t cxl_pmu_irq(int irq, void *data)writeq (overflowed, base +CXL_PMU_OVERFLOW_REG ); +/* +* An overflow freezes every counter in the CPMU, so unfreeze once the +* overflowed ones have been read and their status cleared. Otherwise +* they stay frozen until the next pmu_enable() and events are lost. +* +* Not while the PMU is disabled, so as not to undo an intentional freeze. +* The check is advisory, not exclusive: pmu_disable() normally runs on +* info->on_cpu with interrupts off, where the pinned handler cannot +* preempt it. In the one window where it does not - the migration in +* cxl_pmu_offline_cpu() - the counters are legitimately running again, +* so unfreezing is correct there anyway. +*/ +if (READ_ONCE (info->enabled)) +writeq (0 , base +CXL_PMU_FREEZE_REG ); +return IRQ_HANDLED ; }
Signature · 1 line
2.54.0
[email protected]
AI reviewRe: [RESEND PATCH v4 08/11] perf/cxl: Unfreeze counters after handling an overflow interrupt
Thank you for your contribution! Sashiko AI review found 5 potential issue(s) to consider:
Thank you for your contribution! Sashiko AI review found 5 potential issue(s) to consider:
New issues:
- [High] A cross-CPU TOCTOU race condition during CPU migration can leave the PMU permanently unfrozen when it is supposed to be disabled, leading to a specification violation where a running counter is written to.
Pre-existing issues:
- [High] __cxl_pmu_read is vulnerable to a cross-CPU data race on hwc->prev_count during CPU migration because it relies on local64_cmpxchg.
- [High] Pending overflow interrupts can be incorrectly attributed to newly scheduled events on the same counter, corrupting their counts by adding a spurious full period.
- [High] Writing 1s to RsvdZ (Reserved, must be zero) bits in the Freeze register violates the CXL specification and risks fatal hardware errors.
- [High] Concurrent PMU reads and overflows cause massive double-counting or loss of events due to stateless overflow handling.
--
--- Patch [8]: [RESEND PATCH v4 08/11] perf/cxl: Unfreeze counters after handling an overflow interrupt --- commit 77054eb72889cfc4fca68e72d632bb6f5cfbe02d
Author: Dave Jiang <[email protected]>
perf/cxl: Unfreeze counters after handling an overflow interrupt
The counters run with Freeze on Overflow set, so one overflow freezes every counter in the block. cxl_pmu_irq() reads the overflowed counters and clears the overflow status but never unfreezes, so everything stays frozen until the next pmu_enable(). This patch unfreezes after clearing the status, unless the PMU has been disabled in the meantime.
Quoted 4 lines
drivers/perf/cxl_pmu.c +0
-0
[ ... ]
drivers/perf/cxl_pmu.c
Quoted 13 lines
drivers/perf/cxl_pmu.c +1
-0@@ -604,6 +607,7 @@ static void cxl_pmu_disable(struct pmu *pmu)struct cxl_pmu_info *info =pmu_to_cxl_pmu_info (pmu);void __iomem *base = info->base; +WRITE_ONCE (info->enabled, false);/* * Whilst bits above number of counters are RsvdZ * they are unlikely to be repurposed given drivers/perf/cxl_pmu.c +0
-0* number of counters is allowed to be 64 leaving * no reserved bits. Hence this is only slightly * naughty. */ writeq (GENMASK_ULL (63 ,0 ), base +CXL_PMU_FREEZE_REG );
[Severity: High]
This isn't a bug introduced by this patch, but does writing a full 64-bit mask to the freeze register risk fatal hardware errors on strict CXL implementations?
In cxl_pmu_disable(), the driver writes GENMASK_ULL(63, 0) to CXL_PMU_FREEZE_REG to freeze all counters.
According to the CXL 3.0 specification, bits above the implemented number of counters (info->num_counters) are RsvdZ, and software is strictly required to write 0 to these fields.
Writing 1s to reserved fields could cause AER faults or Poisoned transactions.
[ ... ]
Quoted 7 lines
drivers/perf/cxl_pmu.c +3
-0@@ -802,6 +806,21 @@ static irqreturn_t cxl_pmu_irq(int irq, void *data)writeq (overflowed, base +CXL_PMU_OVERFLOW_REG ); +/* +* An overflow freezes every counter in the CPMU, so unfreeze once the +* overflowed ones have been read and their status cleared. Otherwise
drivers/perf/cxl_pmu.c static irqreturn_t cxl_pmu_irq(int irq, void *data) · line 821
Quoted 10 lines
drivers/perf/cxl_pmu.c +10
-0+* they stay frozen until the next pmu_enable() and events are lost. +* +* Not while the PMU is disabled, so as not to undo an intentional freeze. +* The check is advisory, not exclusive: pmu_disable() normally runs on +* info->on_cpu with interrupts off, where the pinned handler cannot +* preempt it. In the one window where it does not - the migration in +* cxl_pmu_offline_cpu() - the counters are legitimately running again, +* so unfreezing is correct there anyway. +*/ +if (READ_ONCE (info->enabled))
[Severity: High]
Could this advisory check on info->enabled cause a cross-CPU race condition during CPU migration?
During CPU hotplug migration, perf_pmu_migrate_context() in cxl_pmu_offline_cpu() moves events to the target CPU, but the IRQ affinity is not updated until afterwards.
If the IRQ fires on the old CPU, it can read a stale true for info->enabled concurrently with the target CPU executing cxl_pmu_disable(), which sets it to false and freezes the PMU.
The old CPU would then write 0 to unfreeze the PMU here. The target CPU, expecting the PMU to be frozen, proceeds to cxl_pmu_event_start(), enabling the counter via CXL_PMU_COUNTER_CFG_ENABLE and writing 0 to CXL_PMU_COUNTER_REG.
Because the PMU was incorrectly unfrozen, the counter starts immediately, leading to a write to a running counter, which violates the CXL 3.0 spec.
drivers/perf/cxl_pmu.c static irqreturn_t cxl_pmu_irq(int irq, void *data) · line 825
Quoted 4 lines
drivers/perf/cxl_pmu.c +2
-0+writeq (0 , base +CXL_PMU_FREEZE_REG ); +return IRQ_HANDLED ; }
I also noticed a few other pre-existing issues in this file while reviewing.
[Severity: High]
This isn't a bug introduced by this patch, but is the use of local64_cmpxchg in __cxl_pmu_read() safe across multiple CPUs?
drivers/perf/cxl_pmu.c:__cxl_pmu_read() {
...
do {
prev_cnt = local64_read(&hwc->prev_count);
new_cnt = cxl_pmu_read_counter(event);
} while (local64_cmpxchg(&hwc->prev_count, prev_cnt, new_cnt) != prev_cnt);
...
}During the CPU hotplug migration window in cxl_pmu_offline_cpu(), events are migrated to the target CPU before IRQ affinity is updated via irq_set_affinity().
If an overflow IRQ fires in this window, it runs on the old CPU while the target CPU can concurrently execute cxl_pmu_read().
Since local64_t only guarantees atomicity for local CPU accesses, both CPUs iterating over the same event and concurrently executing local64_cmpxchg() could cause a data race and silent counter corruption.
[Severity: High]
This isn't a bug introduced by this patch, but can pending overflow interrupts be incorrectly attributed to newly scheduled events?
When a counter overflows, the hardware asserts an interrupt and sets the corresponding bit in CXL_PMU_OVERFLOW_REG.
If the current event is deleted before the IRQ is serviced, neither cxl_pmu_event_del() nor cxl_pmu_event_start() clears the overflow bit:
drivers/perf/cxl_pmu.c:cxl_pmu_event_start() {
...
local64_set(&hwc->prev_count, 0);
writeq(0, base + CXL_PMU_COUNTER_REG(hwc->idx));
...
}When cxl_pmu_event_add() maps a new event to the same counter index, and the pending IRQ is serviced, cxl_pmu_irq() calls __cxl_pmu_read(event, true).
For the new event, both new_cnt and prev_cnt are near 0, meaning the following check is satisfied:
drivers/perf/cxl_pmu.c:__cxl_pmu_read() {
...
if (overflow && new_cnt >= prev_cnt)
delta += mask + 1;
...
}Could this cause the driver to falsely believe a wrap occurred and corrupt the count by adding a spurious full period?
[Severity: High]
This isn't a bug introduced by this patch, but does the stateless overflow handling in __cxl_pmu_read() cause double-counting or lost events?
drivers/perf/cxl_pmu.c:__cxl_pmu_read() {
...
delta = (new_cnt - prev_cnt) & mask;
if (overflow && new_cnt >= prev_cnt)
delta += mask + 1;
...
}If a polled read (overflow = false) occurs after the counter wraps and freezes, but before the IRQ handler runs, it adds the correct delta and updates prev_cnt.
When the IRQ handler subsequently runs, new_cnt and prev_cnt will be equal, satisfying new_cnt >= prev_cnt.
Wouldn't the IRQ handler then erroneously add a full period on top of the already correct count? Conversely, if the event is deleted before the IRQ handler runs, could the full period be permanently lost?
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=8
Dave Jiang
maintainerRe: [RESEND PATCH v4 08/11] perf/cxl: Unfreeze counters after handling an overflow interrupt
Quoted 4 lines · On 8/5/26 9:15 AM, [email protected] wrote:
Thank you for your contribution! Sashiko AI review found 5 potential issue(s) to consider:
New issues:
- [High] A cross-CPU TOCTOU race condition during CPU migration can leave the PMU permanently unfrozen when it is supposed to be disabled, leading to a specification violation where a running counter is written to.
The window is real but the spec conclusion is not. CXL r4.0 8.2.7.1.1, Counters Writable while Frozen, ties writability to enabled or frozen, not to running:
1 = ... software is permitted to write and modify any Fixed-function
Counter Data register or any Configurable Counter Data register while it is frozen.
cxl_pmu_parse_caps() refuses to probe unless that bit is set, so the write event_start() does is explicitly permitted. The stray unfreeze takes away the frozen state, which the grant does not cover. We get a garbage initial count on one event, on a CPU that is going away, and the destination's pmu_enable() unfreezes a moment later anyway.
A lock would close the gap between the flag read and the freeze write, but the handler still races event_start() on the other CPU, so it narrows the race rather than removing it.
The real fix is not depending on same-CPU semantics at all. Moving prev_count and event->count off local64_t makes this class of question go away. Needs follow changes if that is the direction we want to go.
DJ
Quoted 183 lines
Pre-existing issues:
- [High]__cxl_pmu_readis vulnerable to a cross-CPU data race onhwc->prev_countduring CPU migration because it relies onlocal64_cmpxchg.
- [High] Pending overflow interrupts can be incorrectly attributed to newly scheduled events on the same counter, corrupting their counts by adding a spurious full period.
- [High] Writing 1s to RsvdZ (Reserved, must be zero) bits in the Freeze register violates the CXL specification and risks fatal hardware errors.
- [High] Concurrent PMU reads and overflows cause massive double-counting or loss of events due to stateless overflow handling.
----- Patch [8]: [RESEND PATCH v4 08/11] perf/cxl: Unfreeze counters after handling an overflow interrupt --- commit
77054eb72889cfc4fca68e72d632bb6f5cfbe02d
Author: Dave Jiang <[email protected]>perf/cxl: Unfreeze counters after handling an overflow interrupt
The counters run with Freeze on Overflow set, so one overflow freezes every counter in the block.
cxl_pmu_irq()reads the overflowed counters and clears the overflow status but never unfreezes, so everything stays frozen until the nextpmu_enable(). This patch unfreezes after clearing the status, unless the PMU has been disabled in the meantime.Quoted 4 lines
drivers/perf/cxl_pmu.c +0
-0[ ... ]
drivers/perf/cxl_pmu.c
Quoted 13 lines
drivers/perf/cxl_pmu.c +1
-0@@ -604,6 +607,7 @@ static void cxl_pmu_disable(struct pmu *pmu)struct cxl_pmu_info *info =pmu_to_cxl_pmu_info (pmu);void __iomem *base = info->base; +WRITE_ONCE (info->enabled, false);/* * Whilst bits above number of counters are RsvdZ * they are unlikely to be repurposed given drivers/perf/cxl_pmu.c +0
-0* number of counters is allowed to be 64 leaving * no reserved bits. Hence this is only slightly * naughty. */ writeq (GENMASK_ULL (63 ,0 ), base +CXL_PMU_FREEZE_REG );[Severity: High]
This isn't a bug introduced by this patch, but does writing a full 64-bit mask to the freeze register risk fatal hardware errors on strict CXL implementations?In
cxl_pmu_disable(), the driver writesGENMASK_ULL(63, 0)toCXL_PMU_FREEZE_REGto freeze all counters.According to the CXL 3.0 specification, bits above the implemented number of counters (
info->num_counters) are RsvdZ, and software is strictly required to write 0 to these fields.Writing 1s to reserved fields could cause AER faults or Poisoned transactions.
[ ... ]
Quoted 7 lines
drivers/perf/cxl_pmu.c +3
-0@@ -802,6 +806,21 @@ static irqreturn_t cxl_pmu_irq(int irq, void *data)writeq (overflowed, base +CXL_PMU_OVERFLOW_REG ); +/* +* An overflow freezes every counter in the CPMU, so unfreeze once the +* overflowed ones have been read and their status cleared. Otherwise drivers/perf/cxl_pmu.c static irqreturn_t cxl_pmu_irq(int irq, void *data) · line 821
Quoted 10 lines
drivers/perf/cxl_pmu.c +10
-0+* they stay frozen until the next pmu_enable() and events are lost. +* +* Not while the PMU is disabled, so as not to undo an intentional freeze. +* The check is advisory, not exclusive: pmu_disable() normally runs on +* info->on_cpu with interrupts off, where the pinned handler cannot +* preempt it. In the one window where it does not - the migration in +* cxl_pmu_offline_cpu() - the counters are legitimately running again, +* so unfreezing is correct there anyway. +*/ +if (READ_ONCE (info->enabled))[Severity: High]
Could this advisory check oninfo->enabledcause a cross-CPU race condition during CPU migration?During CPU hotplug migration,
perf_pmu_migrate_context()incxl_pmu_offline_cpu()moves events to the target CPU, but the IRQ affinity is not updated until afterwards.If the IRQ fires on the old CPU, it can read a stale true for
info->enabledconcurrently with the target CPU executingcxl_pmu_disable(), which sets it to false and freezes the PMU.The old CPU would then write 0 to unfreeze the PMU here. The target CPU, expecting the PMU to be frozen, proceeds to
cxl_pmu_event_start(), enabling the counter viaCXL_PMU_COUNTER_CFG_ENABLEand writing 0 toCXL_PMU_COUNTER_REG.Because the PMU was incorrectly unfrozen, the counter starts immediately, leading to a write to a running counter, which violates the CXL 3.0 spec.
drivers/perf/cxl_pmu.c static irqreturn_t cxl_pmu_irq(int irq, void *data) · line 825
Quoted 4 lines
drivers/perf/cxl_pmu.c +2
-0+writeq (0 , base +CXL_PMU_FREEZE_REG ); +return IRQ_HANDLED ; }I also noticed a few other pre-existing issues in this file while reviewing.
[Severity: High]
This isn't a bug introduced by this patch, but is the use oflocal64_cmpxchgin__cxl_pmu_read()safe across multiple CPUs?drivers/perf/cxl_pmu.c:__cxl_pmu_read() { ... do { prev_cnt = local64_read(&hwc->prev_count); new_cnt = cxl_pmu_read_counter(event); } while (local64_cmpxchg(&hwc->prev_count, prev_cnt, new_cnt) != prev_cnt); ... }During the CPU hotplug migration window in
cxl_pmu_offline_cpu(), events are migrated to the target CPU before IRQ affinity is updated viairq_set_affinity().If an overflow IRQ fires in this window, it runs on the old CPU while the target CPU can concurrently execute
cxl_pmu_read().Since
local64_tonly guarantees atomicity for local CPU accesses, both CPUs iterating over the same event and concurrently executinglocal64_cmpxchg()could cause a data race and silent counter corruption.[Severity: High]
This isn't a bug introduced by this patch, but can pending overflow interrupts be incorrectly attributed to newly scheduled events?When a counter overflows, the hardware asserts an interrupt and sets the corresponding bit in
CXL_PMU_OVERFLOW_REG.If the current event is deleted before the IRQ is serviced, neither
cxl_pmu_event_del()norcxl_pmu_event_start()clears the overflow bit:drivers/perf/cxl_pmu.c:cxl_pmu_event_start() { ... local64_set(&hwc->prev_count, 0); writeq(0, base + CXL_PMU_COUNTER_REG(hwc->idx)); ... }When
cxl_pmu_event_add()maps a new event to the same counter index, and the pending IRQ is serviced,cxl_pmu_irq()calls__cxl_pmu_read(event, true).For the new event, both
new_cntandprev_cntare near 0, meaning the following check is satisfied:drivers/perf/cxl_pmu.c:__cxl_pmu_read() { ... if (overflow && new_cnt >= prev_cnt) delta += mask + 1; ... }Could this cause the driver to falsely believe a wrap occurred and corrupt the count by adding a spurious full period?
[Severity: High]
This isn't a bug introduced by this patch, but does the stateless overflow handling in__cxl_pmu_read()cause double-counting or lost events?drivers/perf/cxl_pmu.c:__cxl_pmu_read() { ... delta = (new_cnt - prev_cnt) & mask; if (overflow && new_cnt >= prev_cnt) delta += mask + 1; ... }If a polled read (overflow = false) occurs after the counter wraps and freezes, but before the IRQ handler runs, it adds the correct delta and updates
prev_cnt.When the IRQ handler subsequently runs,
new_cntandprev_cntwill be equal, satisfyingnew_cnt>=prev_cnt.Wouldn't the IRQ handler then erroneously add a full period on top of the already correct count? Conversely, if the event is deleted before the IRQ handler runs, could the full period be permanently lost?
Dave Jiang
maintainerPATCH RESEND v4 9/11 perf/cxl: Validate the hardware-reported counter width
In reply to Dave Jiangcxl_pmu_parse_caps() takes the Counter Width straight from the CPMU Capability register without a bounds check. The Counter Data register is 64 bits wide, so a device reporting 0 or more than 64 is reporting nonsense, and GENMASK_ULL(width - 1, 0) in the read path then shifts out of range. That is undefined behaviour, and a UBSAN splat where it is enabled.
Reject a counter width outside 1..64 at probe.
--- drivers/perf/cxl_pmu.c | 9 +++++++++ 1 file changed, 9 insertions(+)
drivers/perf/cxl_pmu.c +9 -0
@@ -142,6 +142,15 @@ static int cxl_pmu_parse_caps(struct device *dev, struct cxl_pmu_info *info) info->num_counters =FIELD_GET (CXL_PMU_CAP_NUM_COUNTERS_MSK , val) +1 ; info->counter_width =FIELD_GET (CXL_PMU_CAP_COUNTER_WIDTH_MSK , val); +/* +* The Counter Data register is 64 bits wide, so a Counter Width of 0 or +* >64 is invalid. Reject it rather than let GENMASK_ULL(width - 1, 0) in +* the read path shift out of range. +*/ +if (info->counter_width ==0 || info->counter_width >64 ) { +dev_err (dev,"Invalid counter width %d\n" , info->counter_width); +return -ENODEV ; + } info->num_event_capabilities =FIELD_GET (CXL_PMU_CAP_NUM_EVN_CAP_REG_SUP_MSK , val) +1 ; info->filter_hdm =FIELD_GET (CXL_PMU_CAP_FILTERS_SUP_MSK , val) &CXL_PMU_FILTER_HDM ;
Signature · 1 line
2.54.0
[email protected]
AI reviewRe: [RESEND PATCH v4 09/11] perf/cxl: Validate the hardware-reported counter width
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
Pre-existing issues:
- [High] Race condition between manual perf event reads and the PMU overflow interrupt handler causes spurious full-period additions.
- [High] Missing cleanup of the hardware overflow status bit when stopping or starting a counter leads to massive overcounting for subsequently scheduled events. --
--- Patch [9]: [RESEND PATCH v4 09/11] perf/cxl: Validate the hardware-reported counter width --- commit 194e381f35d23ddbc6cc781235e1bc6b38fb673a
Author: Dave Jiang <[email protected]>
perf/cxl: Validate the hardware-reported counter width
cxl_pmu_parse_caps() takes the Counter Width straight from the CPMU Capability register without a bounds check. Reject a counter width outside 1..64 at probe to prevent shifting out of range.
Quoted 10 lines
drivers/perf/cxl_pmu.c +2
-0@@ -142,6 +142,15 @@ static int cxl_pmu_parse_caps(struct device *dev, struct cxl_pmu_info *info) info->num_counters =FIELD_GET (CXL_PMU_CAP_NUM_COUNTERS_MSK , val) +1 ; info->counter_width =FIELD_GET (CXL_PMU_CAP_COUNTER_WIDTH_MSK , val); +/* +* The Counter Data register is 64 bits wide, so a Counter Width of 0 or
drivers/perf/cxl_pmu.c static int cxl_pmu_parse_caps(struct device *dev, struct cxl_pmu_info *info) · line 156
Quoted 10 lines
drivers/perf/cxl_pmu.c +7
-0+* >64 is invalid. Reject it rather than let GENMASK_ULL(width - 1, 0) in +* the read path shift out of range. +*/ +if (info->counter_width ==0 || info->counter_width >64 ) { +dev_err (dev,"Invalid counter width %d\n" , info->counter_width); +return -ENODEV ; + } info->num_event_capabilities =FIELD_GET (CXL_PMU_CAP_NUM_EVN_CAP_REG_SUP_MSK , val) +1 ; info->filter_hdm =FIELD_GET (CXL_PMU_CAP_FILTERS_SUP_MSK , val) &CXL_PMU_FILTER_HDM ;
[Severity: High]
This is a pre-existing issue, but can a race condition between manual perf event reads and the PMU overflow interrupt handler cause spurious additions?
drivers/perf/cxl_pmu.c:__cxl_pmu_read() {
...
mask = GENMASK_ULL(info->counter_width - 1, 0);
delta = (new_cnt - prev_cnt) & mask;
if (overflow && new_cnt >= prev_cnt)
delta += mask + 1;
...
}If a hardware counter wraps and asserts the PMU interrupt, and a manual read occurs before the interrupt handler executes, the manual read handles the wrap natively via 2's complement subtraction and updates prev_count.
When the IRQ handler subsequently runs, it sees the stale hardware overflow bit still set and evaluates new_cnt >= prev_cnt. Will it then spuriously add a full period (mask + 1) to the event count?
[Severity: High]
This is a pre-existing issue, but does missing cleanup of the hardware overflow status bit when stopping or starting a counter lead to overcounting for subsequently scheduled events?
drivers/perf/cxl_pmu.c:cxl_pmu_event_stop() {
...
cfg &= ~(FIELD_PREP(CXL_PMU_COUNTER_CFG_INT_ON_OVRFLW, 1) |
FIELD_PREP(CXL_PMU_COUNTER_CFG_ENABLE, 1));
writeq(cfg, base + CXL_PMU_COUNTER_CFG_REG(hwc->idx));
...
}If cxl_pmu_event_stop disables a counter's interrupt but does not clear the hardware overflow status bit in CXL_PMU_OVERFLOW_REG, the bit remains set.
drivers/perf/cxl_pmu.c:cxl_pmu_event_start() {
...
cfg |= FIELD_PREP(CXL_PMU_COUNTER_CFG_INT_ON_OVRFLW, 1);
cfg |= FIELD_PREP(CXL_PMU_COUNTER_CFG_FREEZE_ON_OVRFLW, 1);
cfg |= FIELD_PREP(CXL_PMU_COUNTER_CFG_ENABLE, 1);
...
}When cxl_pmu_event_start later reuses the counter index for a new event, it initializes the counter to 0 and re-enables the interrupt without clearing the pending overflow bit.
Could the IRQ handler process the stale overflow bit and incorrectly add mask + 1 to the new event's count?
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=9
Dave Jiang
maintainerPATCH RESEND v4 10/11 perf/cxl: Don't log through pmu.dev in the overflow interrupt handler
In reply to Dave Jiangperf_pmu_unregister() frees pmu->dev without clearing the pointer, and cxl_pmu_probe() orders its devm actions so that teardown runs it before free_irq(). Nothing in between masks the interrupt, and cxl_pmu_event_stop() leaves the overflow status bit set, so an overflow taken just before teardown is still delivered and still reaches the dev_dbg() in the !event branch - by which point info->pmu.dev is freed.
Log through info->pmu.parent instead, which is devm-managed and outlives every teardown action.
---
v4:
- Split the probe-time overflow clear out, so this is back to the single hunk Jonathan acked in v2.
--- drivers/perf/cxl_pmu.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/perf/cxl_pmu.c +1 -1
@@ -805,7 +805,7 @@ static irqreturn_t cxl_pmu_irq(int irq, void *data)struct perf_event *event = info->hw_events[i];if (!event) { -dev_dbg (info->pmu.dev, +dev_dbg (info->pmu.parent,"overflow but on non enabled counter %d\n" , i);continue ; }
Signature · 1 line
2.54.0
Dave Jiang
maintainerPATCH RESEND v4 11/11 perf/cxl: Clear stale overflow status before using a counter
In reply to Dave Jiangcxl_pmu_event_stop() leaves the counter's bit set in the overflow status register and only cxl_pmu_irq() ever clears it, so a counter handed from one event to the next starts with the previous owner's overflow pending. The perf core stops and starts back to back with interrupts off, so an interrupt latched over the handover lands on the new event. prev_count is 0 by then, so the read path adds a full period: 2^counter_width phantom counts.
Clear the bit in cxl_pmu_event_start(), before the config write arms Interrupt on Overflow. Clear the whole register at probe too, for counters no event ever owns - the driver never touched it, so firmware or a previous kernel can leave a bit set there.
---
v4:
- New patch. The event_start() clear is Richard's; the probe clear was a second hunk on the pmu.dev patch in v3.
- Clear before the config write rather than after it, so the question of whether arming the interrupt over a set status bit can raise an MSI does not arise (sashiko-bot).
--- drivers/perf/cxl_pmu.c | 17 +++++++++++++++++ 1 file changed, 17 insertions(+)
drivers/perf/cxl_pmu.c +17 -0
@@ -683,6 +683,15 @@ static void cxl_pmu_event_start(struct perf_event *event, int flags)*/ cfg |=FIELD_PREP (CXL_PMU_COUNTER_CFG_THRESHOLD_MSK ,cxl_pmu_config1_get_threshold (event)); + +/* +* Drop any overflow the previous owner of this counter left pending; +* cxl_pmu_event_stop() does not. Otherwise an interrupt latched over +* the handover gets charged to this event. Do it before arming Interrupt +* on Overflow below, and note RW1C leaves the other counters alone. +*/ +writeq (BIT_ULL (hwc->idx), base +CXL_PMU_OVERFLOW_REG ); +writeq (cfg, base +CXL_PMU_COUNTER_CFG_REG (hwc->idx));local64_set (&hwc->prev_count,0 );
@@ -914,6 +923,14 @@ static int cxl_pmu_probe(struct device *dev)if (!irq_name)return -ENOMEM ; +/* +* Same for counters no event owns yet: clear whatever firmware or a +* previous kernel left set before the handler goes live. Bits above the +* implemented counters are reserved, so only write those. +*/ +writeq (GENMASK_ULL (info->num_counters -1 ,0 ), + info->base +CXL_PMU_OVERFLOW_REG ); +/* * The handler must run on info->on_cpu, so the interrupt cannot be * shared - IRQF_NOBALANCING is only honoured for the first action on a
Signature · 1 line
2.54.0
[email protected]
AI reviewRe: [RESEND PATCH v4 11/11] perf/cxl: Clear stale overflow status before using a counter
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
Pre-existing issues:
- [High] Flawed overflow handling adds phantom counts to events when cxl_pmu_read() races with the overflow interrupt. --
--- Patch [11]: [RESEND PATCH v4 11/11] perf/cxl: Clear stale overflow status before using a counter --- commit 7eae9f25fd541ecbfdfed643b25d0495c2ec7444
Author: Dave Jiang <[email protected]>
perf/cxl: Clear stale overflow status before using a counter
This commit clears the overflow status register in cxl_pmu_event_start() and cxl_pmu_probe(). This ensures that a counter handed from one event to the next does not start with a previous owner's pending overflow, preventing phantom counts from being added on handover.
Quoted 5 lines
drivers/perf/cxl_pmu.c +0
-0@@ -683,6 +683,15 @@ static void cxl_pmu_event_start(struct perf_event *event, int flags)*/
drivers/perf/cxl_pmu.c static void cxl_pmu_event_start(struct perf_event *event, int flags) · line 693
Quoted 10 lines
drivers/perf/cxl_pmu.c +8
-0cfg |=FIELD_PREP (CXL_PMU_COUNTER_CFG_THRESHOLD_MSK ,cxl_pmu_config1_get_threshold (event)); + +/* +* Drop any overflow the previous owner of this counter left pending; +* cxl_pmu_event_stop() does not. Otherwise an interrupt latched over +* the handover gets charged to this event. Do it before arming Interrupt +* on Overflow below, and note RW1C leaves the other counters alone. +*/ +writeq (BIT_ULL (hwc->idx), base +CXL_PMU_OVERFLOW_REG );
[Severity: High]
This is a pre-existing issue, but while we are addressing phantom counts from stale overflows here, is there still a race condition between cxl_pmu_read() and the overflow interrupt that can also result in phantom periods?
If the hardware counter wraps, it sets the overflow bit and raises an MSI. Before the ISR can run, cxl_pmu_read() might be called (for instance, via a system call or a periodic timer) while local IRQs are disabled:
cxl_pmu_read()
__cxl_pmu_read(event, false)
This calculates the correct delta, adds it to the event count, and updates prev_count to match new_cnt.
Once IRQs are re-enabled, the pending MSI fires. cxl_pmu_irq() runs and calls:
cxl_pmu_irq()
__cxl_pmu_read(event, true)
Because prev_count was just updated, prev_count and new_cnt are now equal or nearly equal. In __cxl_pmu_read(), we have:
if (overflow && new_cnt >= prev_cnt) delta += mask + 1;
Since new_cnt is greater than or equal to prev_cnt and the overflow flag is passed as true by the ISR, does this logic erroneously add a full mask + 1 phantom period to the performance counter?
[ ... ]
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=11
Re: [RESEND PATCH v4 11/11] perf/cxl: Clear stale overflow status before using a counter
In reply to Dave JiangQuoted 13 lines · On Wed, 5 Aug 2026 08:59:11 -0700 Dave Jiang <[email protected]> wrote:
cxl_pmu_event_stop()leaves the counter's bit set in the overflow status register and onlycxl_pmu_irq()ever clears it, so a counter handed from one event to the next starts with the previous owner's overflow pending. The perf core stops and starts back to back with interrupts off, so an interrupt latched over the handover lands on the new event.prev_countis 0 by then, so the read path adds a full period: 2^counter_width phantom counts.Clear the bit in
cxl_pmu_event_start(), before the config write arms Interrupt on Overflow. Clear the whole register at probe too, for counters no event ever owns - the driver never touched it, so firmware or a previous kernel can leave a bit set there.
Quoted 5 lines
Fixes 5d7107c72796 ("perf: CXL Performance Monitoring Unit driver")Reported-by Richard Cheng <[email protected]>Assisted-by Claude:claude-opus-4-8Signed-off-by Dave Jiang <[email protected]>
Looks good to me as does the whole series. Thanks for hard work cleaning up my mess! Now there is actual hardware out there, I'm kind of expecting a lot of extensions to the features of this driver as we deliberately kept it minimal until more was known about what people were actually building. I wonder if the limited support did guide folk away from the sillier things the spec allowed. Will be fun to see.
Quoted 54 lines · Dave Jiang
[resend, mangled linux-cxl mailing addr]
Teed off of Davidlohr's misc cxlpmu series [1]. I had Claude pick up all the sashiko raised issues and then continued multiple internal review iterations to pick up a number of fixes. I'm no CXL PMU or perf expert, but the fixes look reasonable to me AFAICT.
Since v3:
- 4/11 and 5/11 swapped, so the one-line MSI vector 0 fix comes before theinfo->msi_vecrename and backports on its own (Robin).
- 6/11: new patch, add the PMUs after configuring events.
- 7/11: no code change; the commit message no longer overstates what droppingIRQF_SHAREDcosts a device that shares a vector.
- 8/11: quote the spec sentence that makes the freeze sticky, and correct the claim about what the migration window leaves enabled (Richard, sashiko-bot).
- 10/11: split the probe-time overflow clear out, so this is back to the single hunk Jonathan acked.
- 11/11: new patch. Clear a counter's stale overflow status when starting an event, which is where the reuse case bites; the probe clear moved here from 10/11 (Richard).Since v2:
- 1/11: checkevent_idxalongsidecounter_idx(Jonathan).
- 2/11: useFIELD_MODIFY()rather than mask-then-OR (Jonathan).
- 3/11: rewrite the rationale, no code change (Jonathan).
- 5/11: new patch, split the MSI vector out ofinfo->irq(Robin, Jonathan).
- 7/11: dropIRQF_SHAREDas well as addingIRQF_NOBALANCING, and retitle (Jonathan, Robin).
- 8/11: don't unfreeze if the PMU has been disabled meanwhile (Robin, Jonathan).
- 10/11: retitle, and drop thecxl_pmu_offline_cpu()hunk since thatdev_err()cannot be reached (Robin).
- Dropped the old 9/9 that guardedcpumask_show()againstcpumask_of(-1). The init case was a false positive and neither reviewer liked the shape of it (Jonathan, Robin).Since v1:
- Updated 3/9 to address sashiko issue.
- No other changes as sashiko only raised issue with pre-existing.Three follow-ups this series does not attempt:
cxl_pmu_read()relies onlocal64_tbeing same-CPU, which is the only reason the interrupt has to be pinned at all. Movingprev_countandevent->countofflocal64_twould make the affinity question moot, and Robin may have a view on whether that is worth doing.
cxl_pmu_offline_cpu()still leaveson_cpuat -1 across a sleepingperf_pmu_migrate_context(), and still has an unreachable no-target branch. Both want removing, possibly on top of the generic PMU hotplug rework Jonathan pointed at [2].
Quoted 8 lines
Firmware can leave a counter enabled with Global Freeze on Overflow but not Interrupt on Overflow, which stalls the whole CPMU with no interrupt to say so. Quiescing the block at probe needs two writes per counter to stay within the spec's rule about changing an enabled counter, so it belongs in its own patch.
[1]: https://lore.kernel.org/linux-cxl/[email protected]/
[2]: https://lore.kernel.org/linux-arm-kernel/[email protected]/
Thanks Dave, I notice that everything seems ready to land for this series. Is there anything pending? Sashiko mentioned pre-existing issues.
Thanks,
Ian
Quoted 23 lines
Dave Jiang (11):
perf/cxl: Program the requested event group on configurable counters perf/cxl: Clear stale event fields before reprogramming a counter perf/cxl: Fix the counter overflow delta fixup
perf/cxl: Accept an overflow interrupt on MSI message number 0 perf/cxl: Split the MSI vector out ofinfo->irq
cxl/pci: Add the PMUs after configuring events
perf/cxl: Don't share the overflow interrupt, and keep it pinned perf/cxl: Unfreeze counters after handling an overflow interrupt perf/cxl: Validate the hardware-reported counter width perf/cxl: Don't log through pmu.dev in the overflow interrupt handler perf/cxl: Clear stale overflow status before using a counterdrivers/cxl/pci.c | 11 +++-- drivers/perf/cxl_pmu.c | 107 ++++++++++++++++++++++++++++++++--------- 2 files changed, 90 insertions(+), 28 deletions(-)base-commit:
f5098b6bae761e346ebcd9da7f95622c04733cff--
2.54.0
Dave Jiang
maintainerQuoted 61 lines · Ian Rogers
Quoted 60 lines · Dave Jiang
[resend, mangled linux-cxl mailing addr]
Teed off of Davidlohr's misc cxlpmu series [1]. I had Claude pick up all the sashiko raised issues and then continued multiple internal review iterations to pick up a number of fixes. I'm no CXL PMU or perf expert, but the fixes look reasonable to me AFAICT.
Since v3:
- 4/11 and 5/11 swapped, so the one-line MSI vector 0 fix comes before theinfo->msi_vecrename and backports on its own (Robin).
- 6/11: new patch, add the PMUs after configuring events.
- 7/11: no code change; the commit message no longer overstates what droppingIRQF_SHAREDcosts a device that shares a vector.
- 8/11: quote the spec sentence that makes the freeze sticky, and correct the claim about what the migration window leaves enabled (Richard, sashiko-bot).
- 10/11: split the probe-time overflow clear out, so this is back to the single hunk Jonathan acked.
- 11/11: new patch. Clear a counter's stale overflow status when starting an event, which is where the reuse case bites; the probe clear moved here from 10/11 (Richard).Since v2:
- 1/11: checkevent_idxalongsidecounter_idx(Jonathan).
- 2/11: useFIELD_MODIFY()rather than mask-then-OR (Jonathan).
- 3/11: rewrite the rationale, no code change (Jonathan).
- 5/11: new patch, split the MSI vector out ofinfo->irq(Robin, Jonathan).
- 7/11: dropIRQF_SHAREDas well as addingIRQF_NOBALANCING, and retitle (Jonathan, Robin).
- 8/11: don't unfreeze if the PMU has been disabled meanwhile (Robin, Jonathan).
- 10/11: retitle, and drop thecxl_pmu_offline_cpu()hunk since thatdev_err()cannot be reached (Robin).
- Dropped the old 9/9 that guardedcpumask_show()againstcpumask_of(-1). The init case was a false positive and neither reviewer liked the shape of it (Jonathan, Robin).Since v1:
- Updated 3/9 to address sashiko issue.
- No other changes as sashiko only raised issue with pre-existing.Three follow-ups this series does not attempt:
cxl_pmu_read()relies onlocal64_tbeing same-CPU, which is the only reason the interrupt has to be pinned at all. Movingprev_countandevent->countofflocal64_twould make the affinity question moot, and Robin may have a view on whether that is worth doing.
cxl_pmu_offline_cpu()still leaveson_cpuat -1 across a sleepingperf_pmu_migrate_context(), and still has an unreachable no-target branch. Both want removing, possibly on top of the generic PMU hotplug rework Jonathan pointed at [2].Firmware can leave a counter enabled with Global Freeze on Overflow but not Interrupt on Overflow, which stalls the whole CPMU with no interrupt to say so. Quiescing the block at probe needs two writes per counter to stay within the spec's rule about changing an enabled counter, so it belongs in its own patch.
Quoted 7 lines
Quoted 2 lines
[1]: https://lore.kernel.org/linux-cxl/[email protected]/
[2]: https://lore.kernel.org/linux-arm-kernel/[email protected]/Thanks Dave, I notice that everything seems ready to land for this series. Is there anything pending? Sashiko mentioned pre-existing issues.
Hi Ian. Nothing is pending. Don't plan to address the pre-existing issues in the current series. Will look at follow-ups once this series is picked up. Thanks!
Quoted 27 lines
Thanks,
IanQuoted 23 lines
Dave Jiang (11):
perf/cxl: Program the requested event group on configurable counters perf/cxl: Clear stale event fields before reprogramming a counter perf/cxl: Fix the counter overflow delta fixup
perf/cxl: Accept an overflow interrupt on MSI message number 0 perf/cxl: Split the MSI vector out ofinfo->irq
cxl/pci: Add the PMUs after configuring events
perf/cxl: Don't share the overflow interrupt, and keep it pinned perf/cxl: Unfreeze counters after handling an overflow interrupt perf/cxl: Validate the hardware-reported counter width perf/cxl: Don't log through pmu.dev in the overflow interrupt handler perf/cxl: Clear stale overflow status before using a counterdrivers/cxl/pci.c | 11 +++-- drivers/perf/cxl_pmu.c | 107 ++++++++++++++++++++++++++++++++--------- 2 files changed, 90 insertions(+), 28 deletions(-)base-commit:
f5098b6bae761e346ebcd9da7f95622c04733cff--
2.54.0
Jonathan Cameron
maintainerQuoted 6 lines · On Wed, 5 Aug 2026 08:59:00 -0700 Dave Jiang <[email protected]> wrote:
[resend, mangled linux-cxl mailing addr]
Teed off of Davidlohr's misc cxlpmu series [1]. I had Claude pick up all the sashiko raised issues and then continued multiple internal review iterations to pick up a number of fixes. I'm no CXL PMU or perf expert, but the fixes look reasonable to me AFAICT.
+CC Robert given Robert is talking about what needs adding to this driver at LPC and so presumably has hardware (and will send lots of nice patches for this driver shortly ;)
https://lpc.events/event/20/contributions/2521/
Quoted 75 lines
Since v3:
- 4/11 and 5/11 swapped, so the one-line MSI vector 0 fix comes before theinfo->msi_vecrename and backports on its own (Robin).
- 6/11: new patch, add the PMUs after configuring events.
- 7/11: no code change; the commit message no longer overstates what droppingIRQF_SHAREDcosts a device that shares a vector.
- 8/11: quote the spec sentence that makes the freeze sticky, and correct the claim about what the migration window leaves enabled (Richard, sashiko-bot).
- 10/11: split the probe-time overflow clear out, so this is back to the single hunk Jonathan acked.
- 11/11: new patch. Clear a counter's stale overflow status when starting an event, which is where the reuse case bites; the probe clear moved here from 10/11 (Richard).Since v2:
- 1/11: checkevent_idxalongsidecounter_idx(Jonathan).
- 2/11: useFIELD_MODIFY()rather than mask-then-OR (Jonathan).
- 3/11: rewrite the rationale, no code change (Jonathan).
- 5/11: new patch, split the MSI vector out ofinfo->irq(Robin, Jonathan).
- 7/11: dropIRQF_SHAREDas well as addingIRQF_NOBALANCING, and retitle (Jonathan, Robin).
- 8/11: don't unfreeze if the PMU has been disabled meanwhile (Robin, Jonathan).
- 10/11: retitle, and drop thecxl_pmu_offline_cpu()hunk since thatdev_err()cannot be reached (Robin).
- Dropped the old 9/9 that guardedcpumask_show()againstcpumask_of(-1). The init case was a false positive and neither reviewer liked the shape of it (Jonathan, Robin).Since v1:
- Updated 3/9 to address sashiko issue.
- No other changes as sashiko only raised issue with pre-existing.Three follow-ups this series does not attempt:
cxl_pmu_read()relies onlocal64_tbeing same-CPU, which is the only reason the interrupt has to be pinned at all. Movingprev_countandevent->countofflocal64_twould make the affinity question moot, and Robin may have a view on whether that is worth doing.
cxl_pmu_offline_cpu()still leaveson_cpuat -1 across a sleepingperf_pmu_migrate_context(), and still has an unreachable no-target branch. Both want removing, possibly on top of the generic PMU hotplug rework Jonathan pointed at [2].Firmware can leave a counter enabled with Global Freeze on Overflow but not Interrupt on Overflow, which stalls the whole CPMU with no interrupt to say so. Quiescing the block at probe needs two writes per counter to stay within the spec's rule about changing an enabled counter, so it belongs in its own patch.
[1]: https://lore.kernel.org/linux-cxl/[email protected]/
[2]: https://lore.kernel.org/linux-arm-kernel/[email protected]/Dave Jiang (11):
perf/cxl: Program the requested event group on configurable counters perf/cxl: Clear stale event fields before reprogramming a counter perf/cxl: Fix the counter overflow delta fixup
perf/cxl: Accept an overflow interrupt on MSI message number 0 perf/cxl: Split the MSI vector out ofinfo->irq
cxl/pci: Add the PMUs after configuring events
perf/cxl: Don't share the overflow interrupt, and keep it pinned perf/cxl: Unfreeze counters after handling an overflow interrupt perf/cxl: Validate the hardware-reported counter width perf/cxl: Don't log through pmu.dev in the overflow interrupt handler perf/cxl: Clear stale overflow status before using a counterdrivers/cxl/pci.c | 11 +++-- drivers/perf/cxl_pmu.c | 107 ++++++++++++++++++++++++++++++++--------- 2 files changed, 90 insertions(+), 28 deletions(-)base-commit:
f5098b6bae761e346ebcd9da7f95622c04733cff
Quoted 8 lines · Dave Jiang
[resend, mangled linux-cxl mailing addr]
Teed off of Davidlohr's misc cxlpmu series [1]. I had Claude pick up all the sashiko raised issues and then continued multiple internal review iterations to pick up a number of fixes. I'm no CXL PMU or perf expert, but the fixes look reasonable to me AFAICT.
[...]
Applied to arm64 (for-next/perf), thanks!
[01/11] perf/cxl: Program the requested event group on configurable counters
https://git.kernel.org/arm64/c/b22dcc114298 [02/11] perf/cxl: Clear stale event fields before reprogramming a counter https://git.kernel.org/arm64/c/7406d6cf76b1 [03/11] perf/cxl: Fix the counter overflow delta fixup https://git.kernel.org/arm64/c/0326269ed735 [04/11] perf/cxl: Accept an overflow interrupt on MSI message number 0 https://git.kernel.org/arm64/c/af06fba45746 [05/11] perf/cxl: Split the MSI vector out of info->irq https://git.kernel.org/arm64/c/97fe5d94b7ce [06/11] cxl/pci: Add the PMUs after configuring events https://git.kernel.org/arm64/c/28b2b667f48f [07/11] perf/cxl: Don't share the overflow interrupt, and keep it pinned https://git.kernel.org/arm64/c/b899fd3487b0 [08/11] perf/cxl: Unfreeze counters after handling an overflow interrupt https://git.kernel.org/arm64/c/2e4fe234c72d [09/11] perf/cxl: Validate the hardware-reported counter width https://git.kernel.org/arm64/c/1c5499e67303 [10/11] perf/cxl: Don't log through pmu.dev in the overflow interrupt handler https://git.kernel.org/arm64/c/984920c893f0 [11/11] perf/cxl: Clear stale overflow status before using a counter https://git.kernel.org/arm64/c/05d82cf377f7
Cheers,
Signature · 5 lines
Will https://fixes.arm64.dev https://next.arm64.dev https://will.arm64.dev