CXL: Process event logs
v2 cover CXL: Process event logs
From: Ira Weiny <[email protected]>
Changes from V1
Address comments, from Jonathan, Dave, and Alison
Main comment was to allow for a full payload size number of event records to be processed on each Get event cyle.
Pick up tags
This code has been tested with a newer qemu which allows for more events to be returned at a time as well ad additional QMP event and interrupt injection. Thos patches will follow once they have been cleaned up.
The series is in 5 parts:
0) Davidlohrs irq patch modified for 16 vectors
1) Base functionality
2) Parsing specific events (Dynamic Capacity Event Record is defered)
3) Event interrupt support
4) cxl-test infrastructure for basic tests
While I believe this entire series is ready to be merged I realize that the interrupt support may still have some discussion around it. Therefor parts 1, 2, and 4 could be merged without irq support as cxl-test provides testing for that. Interrupt testing requires Qemu but it too is fully tested and ready to go.
Changes from RFC v2
Integrated Davidlohr's irq patch, allocate up to 16 vectors, and base
my irq support on modifications to that patch.
Smita
Check event status before reading each log.
Jonathan
Process more than 1 record at a time
Remove reserved fields
Steven
Prefix trace points with 'cxl_'
Davidlohr
PUll in his patch
Changes from RFC v1
Add event irqs
General simplification of the code. Resolve field alignment questions Update to rev 3.0 for comments and structures Add reserved fields and output them
Event records inform the OS of various device events. Events are not needed for any kernel operation but various user level software will want to track events.
Add event reporting through the trace event mechanism. On driver load read and clear all device events.
Enable all event logs for interrupts and process each log on interrupt.
TESTING:
Testing of this was performed with additions to QEMU in the following repo:
https://github.com/weiny2/qemu/tree/ira-cxl-events-latest
Changes to this repo are not finalized yet so I'm not posting those patches right away. But there is enough functionality added to further test this.
1) event status register
2) additional event injection capabilities
3) Process more than 1 record at a time in Get/Clear mailbox commands
Davidlohr Bueso (1):
cxl/pci: Add generic MSI-X/MSI irq support
Ira Weiny (10):
cxl/mem: Implement Get Event Records command
cxl/mem: Implement Clear Event Records command
cxl/mem: Clear events on driver load
cxl/mem: Trace General Media Event Record
cxl/mem: Trace DRAM Event Record
cxl/mem: Trace Memory Module Event Record
cxl/mem: Wire up event interrupts
cxl/test: Add generic mock events
cxl/test: Add specific events
cxl/test: Simulate event log overflow
MAINTAINERS | 1 + drivers/cxl/core/mbox.c | 260 +++++++++++++++++ drivers/cxl/cxl.h | 7 + drivers/cxl/cxlmem.h | 188 ++++++++++++ drivers/cxl/cxlpci.h | 6 + drivers/cxl/pci.c | 155 ++++++++++ include/trace/events/cxl.h | 486 ++++++++++++++++++++++++++++++++ include/uapi/linux/cxl_mem.h | 4 + tools/testing/cxl/test/Kbuild | 2 +- tools/testing/cxl/test/events.c | 362 ++++++++++++++++++++++++ tools/testing/cxl/test/events.h | 9 + tools/testing/cxl/test/mem.c | 35 +++ 12 files changed, 1514 insertions(+), 1 deletion(-) create mode 100644 include/trace/events/cxl.h create mode 100644 tools/testing/cxl/test/events.c create mode 100644 tools/testing/cxl/test/events.h
base-commit: aae703b02f92bde9264366c545e87cec451de471
Signature · 1 line
2.37.2
PATCH v2 1/11 cxl/pci: Add generic MSI-X/MSI irq support
From: Davidlohr Bueso <[email protected]>
Currently the only CXL features targeted for irq support require their message numbers to be within the first 16 entries. The device may however support less than 16 entries depending on the support it provides.
Attempt to allocate these 16 irq vectors. If the device supports less then the PCI infrastructure will allocate that number. Store the number of vectors actually allocated in the device state for later use by individual functions.
Upon successful allocation, users can plug in their respective isr at any point thereafter, for example, if the irq setup is not done in the PCI driver, such as the case of the CXL-PMU.
---
Changes from V1:
Jonathan
pci_alloc_irq_vectors() cleans up the vectors automatically use msi_enabled rather than nr_irq_vecs
Changes from Ira
Remove reviews
Allocate up to a static 16 vectors.
Change cover letter
--- drivers/cxl/cxlmem.h | 3 +++ drivers/cxl/cxlpci.h | 6 ++++++ drivers/cxl/pci.c | 23 +++++++++++++++++++++++ 3 files changed, 32 insertions(+)
drivers/cxl/cxlmem.h +3 -0
@@ -211,6 +211,7 @@ struct cxl_endpoint_dvsec_info {* @info: Cached DVSEC information about the device. * @serial: PCIe Device Serial Number * @doe_mbs: PCI DOE mailbox array +* @msi_enabled: MSI-X/MSI has been enabled * @mbox_send: @dev specific transport for transmitting mailbox commands * * See section 8.2.9.5.2 Capacity Configuration and Label Storage for
@@ -247,6 +248,8 @@ struct cxl_dev_state {struct xarray doe_mbs; +bool msi_enabled; +int (*mbox_send)(struct cxl_dev_state *cxlds,struct cxl_mbox_cmd *cmd); };
drivers/cxl/cxlpci.h +6 -0
@@ -53,6 +53,12 @@#define CXL_DVSEC_REG_LOCATOR_BLOCK_ID_MASK GENMASK(15, 8)#define CXL_DVSEC_REG_LOCATOR_BLOCK_OFF_LOW_MASK GENMASK(31, 16) +/* +* NOTE: Currently all the functions which are enabled for CXL require their +* vectors to be in the first 16. Use this as the max. +*/ +#define CXL_PCI_REQUIRED_VECTORS 16 +/* Register Block Identifier (RBI) */ enum cxl_regloc_type {CXL_REGLOC_RBI_EMPTY =0 ,
drivers/cxl/pci.c +23 -0
@@ -428,6 +428,27 @@ static void devm_cxl_pci_create_doe(struct cxl_dev_state *cxlds) } } +static void cxl_pci_alloc_irq_vectors (struct cxl_dev_state *cxlds) +{ +struct device *dev = cxlds->dev; +struct pci_dev *pdev =to_pci_dev (dev); +int nvecs; + +/* +* NOTE: pci_alloc_irq_vectors() handles calling pci_free_irq_vectors() +* automatically despite not being called pcim_*. See +* pci_setup_msi_context(). +*/ + nvecs =pci_alloc_irq_vectors (pdev,1 ,CXL_PCI_REQUIRED_VECTORS , +PCI_IRQ_MSIX |PCI_IRQ_MSI ); +if (nvecs <0 ) { +dev_dbg (dev,"Failed to alloc irq vectors; use polling instead.\n" ); +return ; + } + + cxlds->msi_enabled = true; +} +static int cxl_pci_probe (struct pci_dev *pdev,const struct pci_device_id *id) {struct cxl_register_map map;
@@ -494,6 +515,8 @@ static int cxl_pci_probe(struct pci_dev *pdev, const struct pci_device_id *id)if (rc)return rc; +cxl_pci_alloc_irq_vectors (cxlds); + cxlmd =devm_cxl_add_memdev (cxlds);if (IS_ERR (cxlmd))return PTR_ERR (cxlmd);
Signature · 1 line
2.37.2
Re: [PATCH V2 01/11] cxl/pci: Add generic MSI-X/MSI irq support
Quoted 16 lines · On Wed, 30 Nov 2022 16:27:09 -0800 [email protected] wrote:
From: Davidlohr Bueso <[email protected]>
Currently the only CXL features targeted for irq support require their message numbers to be within the first 16 entries. The device may however support less than 16 entries depending on the support it provides.
Attempt to allocate these 16 irq vectors. If the device supports less then the PCI infrastructure will allocate that number. Store the number of vectors actually allocated in the device state for later use by individual functions.
Upon successful allocation, users can plug in their respective isr at any point thereafter, for example, if the irq setup is not done in the PCI driver, such as the case of the CXL-PMU.
Quoted 5 lines
Cc Bjorn Helgaas <[email protected]>Cc Jonathan Cameron <[email protected]>Co-developed-by Ira Weiny <[email protected]>Signed-off-by Ira Weiny <[email protected]>Signed-off-by Davidlohr Bueso <[email protected]>
Quoted 96 lines
---
Changes from V1:
Jonathan
pci_alloc_irq_vectors()cleans up the vectors automatically usemsi_enabledrather thannr_irq_vecsChanges from Ira
Remove reviews
Allocate up to a static 16 vectors.
Change cover letter--- drivers/cxl/cxlmem.h | 3 +++ drivers/cxl/cxlpci.h | 6 ++++++ drivers/cxl/pci.c | 23 +++++++++++++++++++++++ 3 files changed, 32 insertions(+)drivers/cxl/cxlmem.h +3
-0@@ -211,6 +211,7 @@ struct cxl_endpoint_dvsec_info {* @info: Cached DVSEC information about the device. * @serial: PCIe Device Serial Number * @doe_mbs: PCI DOE mailbox array +* @msi_enabled: MSI-X/MSI has been enabled * @mbox_send: @dev specific transport for transmitting mailbox commands * * See section 8.2.9.5.2 Capacity Configuration and Label Storage for @@ -247,6 +248,8 @@ struct cxl_dev_state {struct xarray doe_mbs; +bool msi_enabled; +int (*mbox_send)(struct cxl_dev_state *cxlds,struct cxl_mbox_cmd *cmd); };drivers/cxl/cxlpci.h +6
-0@@ -53,6 +53,12 @@#define CXL_DVSEC_REG_LOCATOR_BLOCK_ID_MASK GENMASK(15, 8)#define CXL_DVSEC_REG_LOCATOR_BLOCK_OFF_LOW_MASK GENMASK(31, 16) +/* +* NOTE: Currently all the functions which are enabled for CXL require their +* vectors to be in the first 16. Use this as the max. +*/ +#define CXL_PCI_REQUIRED_VECTORS 16 +/* Register Block Identifier (RBI) */ enum cxl_regloc_type {CXL_REGLOC_RBI_EMPTY =0 ,drivers/cxl/pci.c +23
-0@@ -428,6 +428,27 @@ static void devm_cxl_pci_create_doe(struct cxl_dev_state *cxlds) } } +static void cxl_pci_alloc_irq_vectors (struct cxl_dev_state *cxlds) +{ +struct device *dev = cxlds->dev; +struct pci_dev *pdev =to_pci_dev (dev); +int nvecs; + +/* +* NOTE: pci_alloc_irq_vectors() handles calling pci_free_irq_vectors() +* automatically despite not being called pcim_*. See +* pci_setup_msi_context(). +*/ + nvecs =pci_alloc_irq_vectors (pdev,1 ,CXL_PCI_REQUIRED_VECTORS , +PCI_IRQ_MSIX |PCI_IRQ_MSI ); +if (nvecs <0 ) { +dev_dbg (dev,"Failed to alloc irq vectors; use polling instead.\n" ); +return ; + } + + cxlds->msi_enabled = true; +} +static int cxl_pci_probe (struct pci_dev *pdev,const struct pci_device_id *id) {struct cxl_register_map map;@@ -494,6 +515,8 @@ static int cxl_pci_probe(struct pci_dev *pdev, const struct pci_device_id *id)if (rc)return rc; +cxl_pci_alloc_irq_vectors (cxlds); + cxlmd =devm_cxl_add_memdev (cxlds);if (IS_ERR (cxlmd))return PTR_ERR (cxlmd);
Re: [PATCH V2 01/11] cxl/pci: Add generic MSI-X/MSI irq support
In reply to [email protected]Quoted 16 lines · On 11/30/2022 5:27 PM, [email protected] wrote:
From: Davidlohr Bueso <[email protected]>
Currently the only CXL features targeted for irq support require their message numbers to be within the first 16 entries. The device may however support less than 16 entries depending on the support it provides.
Attempt to allocate these 16 irq vectors. If the device supports less then the PCI infrastructure will allocate that number. Store the number of vectors actually allocated in the device state for later use by individual functions.
Upon successful allocation, users can plug in their respective isr at any point thereafter, for example, if the irq setup is not done in the PCI driver, such as the case of the CXL-PMU.
Quoted 5 lines
Cc Bjorn Helgaas <[email protected]>Cc Jonathan Cameron <[email protected]>Co-developed-by Ira Weiny <[email protected]>Signed-off-by Ira Weiny <[email protected]>Signed-off-by Davidlohr Bueso <[email protected]>
Quoted 96 lines
---
Changes from V1:
Jonathan
pci_alloc_irq_vectors()cleans up the vectors automatically usemsi_enabledrather thannr_irq_vecsChanges from Ira
Remove reviews
Allocate up to a static 16 vectors.
Change cover letter--- drivers/cxl/cxlmem.h | 3 +++ drivers/cxl/cxlpci.h | 6 ++++++ drivers/cxl/pci.c | 23 +++++++++++++++++++++++ 3 files changed, 32 insertions(+)drivers/cxl/cxlmem.h +3
-0@@ -211,6 +211,7 @@ struct cxl_endpoint_dvsec_info {* @info: Cached DVSEC information about the device. * @serial: PCIe Device Serial Number * @doe_mbs: PCI DOE mailbox array +* @msi_enabled: MSI-X/MSI has been enabled * @mbox_send: @dev specific transport for transmitting mailbox commands * * See section 8.2.9.5.2 Capacity Configuration and Label Storage for @@ -247,6 +248,8 @@ struct cxl_dev_state {struct xarray doe_mbs; +bool msi_enabled; +int (*mbox_send)(struct cxl_dev_state *cxlds,struct cxl_mbox_cmd *cmd); };drivers/cxl/cxlpci.h +6
-0@@ -53,6 +53,12 @@#define CXL_DVSEC_REG_LOCATOR_BLOCK_ID_MASK GENMASK(15, 8)#define CXL_DVSEC_REG_LOCATOR_BLOCK_OFF_LOW_MASK GENMASK(31, 16) +/* +* NOTE: Currently all the functions which are enabled for CXL require their +* vectors to be in the first 16. Use this as the max. +*/ +#define CXL_PCI_REQUIRED_VECTORS 16 +/* Register Block Identifier (RBI) */ enum cxl_regloc_type {CXL_REGLOC_RBI_EMPTY =0 ,drivers/cxl/pci.c +23
-0@@ -428,6 +428,27 @@ static void devm_cxl_pci_create_doe(struct cxl_dev_state *cxlds) } } +static void cxl_pci_alloc_irq_vectors (struct cxl_dev_state *cxlds) +{ +struct device *dev = cxlds->dev; +struct pci_dev *pdev =to_pci_dev (dev); +int nvecs; + +/* +* NOTE: pci_alloc_irq_vectors() handles calling pci_free_irq_vectors() +* automatically despite not being called pcim_*. See +* pci_setup_msi_context(). +*/ + nvecs =pci_alloc_irq_vectors (pdev,1 ,CXL_PCI_REQUIRED_VECTORS , +PCI_IRQ_MSIX |PCI_IRQ_MSI ); +if (nvecs <0 ) { +dev_dbg (dev,"Failed to alloc irq vectors; use polling instead.\n" ); +return ; + } + + cxlds->msi_enabled = true; +} +static int cxl_pci_probe (struct pci_dev *pdev,const struct pci_device_id *id) {struct cxl_register_map map;@@ -494,6 +515,8 @@ static int cxl_pci_probe(struct pci_dev *pdev, const struct pci_device_id *id)if (rc)return rc; +cxl_pci_alloc_irq_vectors (cxlds); + cxlmd =devm_cxl_add_memdev (cxlds);if (IS_ERR (cxlmd))return PTR_ERR (cxlmd);
RE: [PATCH V2 01/11] cxl/pci: Add generic MSI-X/MSI irq support
In reply to [email protected]Quoted 9 lines · ira.weiny@ wrote:
From: Davidlohr Bueso <[email protected]>
Currently the only CXL features targeted for irq support require their message numbers to be within the first 16 entries. The device may however support less than 16 entries depending on the support it provides.
Attempt to allocate these 16 irq vectors. If the device supports less then the PCI infrastructure will allocate that number.
What happens if the device supports 16, but irq-core allocates less? I believe the answer is with the first user, but this patch does not include a user.
Quoted 2 lines
Store the number of vectors actually allocated in the device state for later use by individual functions.
The patch does not do that.
I know this patch has gone through a lot of discussion, but this mismatch shows it should really be squashed with the first user because it does not stand on its own anymore.
Quoted 39 lines
Upon successful allocation, users can plug in their respective isr at any point thereafter, for example, if the irq setup is not done in the PCI driver, such as the case of the CXL-PMU.
Cc Bjorn Helgaas <[email protected]>Cc Jonathan Cameron <[email protected]>Co-developed-by Ira Weiny <[email protected]>Signed-off-by Ira Weiny <[email protected]>Signed-off-by Davidlohr Bueso <[email protected]>---
Changes from V1:
Jonathan
pci_alloc_irq_vectors()cleans up the vectors automatically usemsi_enabledrather thannr_irq_vecsChanges from Ira
Remove reviews
Allocate up to a static 16 vectors.
Change cover letter--- drivers/cxl/cxlmem.h | 3 +++ drivers/cxl/cxlpci.h | 6 ++++++ drivers/cxl/pci.c | 23 +++++++++++++++++++++++ 3 files changed, 32 insertions(+)drivers/cxl/cxlmem.h +1
-0@@ -211,6 +211,7 @@ struct cxl_endpoint_dvsec_info {* @info: Cached DVSEC information about the device. * @serial: PCIe Device Serial Number * @doe_mbs: PCI DOE mailbox array +* @msi_enabled: MSI-X/MSI has been enabled * @mbox_send: @dev specific transport for transmitting mailbox commands * * See section 8.2.9.5.2 Capacity Configuration and Label Storage for
drivers/cxl/cxlmem.h struct cxl_dev_state { · line 252
Quoted 5 lines
drivers/cxl/cxlmem.h +2
-0@@ -247,6 +248,8 @@ struct cxl_dev_state {struct xarray doe_mbs; +bool msi_enabled; +
This goes unused in this patch and it also duplicates what the core offers with pdev->{msi,msix}_enabled.
Quoted 31 lines
drivers/cxl/cxlmem.h +0
-0int (*mbox_send)(struct cxl_dev_state *cxlds,struct cxl_mbox_cmd *cmd); };drivers/cxl/cxlpci.h +6
-0@@ -53,6 +53,12 @@#define CXL_DVSEC_REG_LOCATOR_BLOCK_ID_MASK GENMASK(15, 8)#define CXL_DVSEC_REG_LOCATOR_BLOCK_OFF_LOW_MASK GENMASK(31, 16) +/* +* NOTE: Currently all the functions which are enabled for CXL require their +* vectors to be in the first 16. Use this as the max. +*/ +#define CXL_PCI_REQUIRED_VECTORS 16 +/* Register Block Identifier (RBI) */ enum cxl_regloc_type {CXL_REGLOC_RBI_EMPTY =0 ,drivers/cxl/pci.c +3
-0@@ -428,6 +428,27 @@ static void devm_cxl_pci_create_doe(struct cxl_dev_state *cxlds) } } +static void cxl_pci_alloc_irq_vectors (struct cxl_dev_state *cxlds) +{ +struct device *dev = cxlds->dev;
drivers/cxl/pci.c static void devm_cxl_pci_create_doe(struct cxl_dev_state *cxlds) · line 443
Quoted 10 lines
drivers/cxl/pci.c +10
-0+struct pci_dev *pdev =to_pci_dev (dev); +int nvecs; + +/* +* NOTE: pci_alloc_irq_vectors() handles calling pci_free_irq_vectors() +* automatically despite not being called pcim_*. See +* pci_setup_msi_context(). +*/ + nvecs =pci_alloc_irq_vectors (pdev,1 ,CXL_PCI_REQUIRED_VECTORS , +PCI_IRQ_MSIX |PCI_IRQ_MSI );
clang-format would scooch that second line in for you.
Might also be worth a comment for the next person that goes looking for why this isn't PCI_IRQ_ALL_TYPES.
From CXL 3.0 3.1.1 CXL.io Endpoint:
A Function on a CXL device must not generate INTx messages if that Function participates in CXL.cache protocol or CXL.mem protocols.
Quoted 23 lines
drivers/cxl/pci.c +10
-0+if (nvecs <0 ) { +dev_dbg (dev,"Failed to alloc irq vectors; use polling instead.\n" ); +return ; + } + + cxlds->msi_enabled = true; +} +static int cxl_pci_probe (struct pci_dev *pdev,const struct pci_device_id *id) {struct cxl_register_map map;@@ -494,6 +515,8 @@ static int cxl_pci_probe(struct pci_dev *pdev, const struct pci_device_id *id)if (rc)return rc; +cxl_pci_alloc_irq_vectors (cxlds); + cxlmd =devm_cxl_add_memdev (cxlds);if (IS_ERR (cxlmd))return PTR_ERR (cxlmd);Signature · 1 line
2.37.2
Re: [PATCH V2 01/11] cxl/pci: Add generic MSI-X/MSI irq support
Quoted 11 lines · Dan Williams
Quoted 9 lines · ira.weiny@ wrote:
From: Davidlohr Bueso <[email protected]>
Currently the only CXL features targeted for irq support require their message numbers to be within the first 16 entries. The device may however support less than 16 entries depending on the support it provides.
Attempt to allocate these 16 irq vectors. If the device supports less then the PCI infrastructure will allocate that number.
Quoted 8 lines
What happens if the device supports 16, but irq-core allocates less? I believe the answer is with the first user, but this patch does not include a user.
Quoted 2 lines
Store the number of vectors actually allocated in the device state for later use by individual functions.
The patch does not do that.
Sorry missed updating this message.
Quoted 4 lines
I know this patch has gone through a lot of discussion, but this mismatch shows it should really be squashed with the first user because it does not stand on its own anymore.
It is separate because it was Davidlohr's to begin with.
I'll squash it back.
Quoted 41 lines
Quoted 41 lines
Upon successful allocation, users can plug in their respective isr at any point thereafter, for example, if the irq setup is not done in the PCI driver, such as the case of the CXL-PMU.
Cc Bjorn Helgaas <[email protected]>Cc Jonathan Cameron <[email protected]>Co-developed-by Ira Weiny <[email protected]>Signed-off-by Ira Weiny <[email protected]>Signed-off-by Davidlohr Bueso <[email protected]>---
Changes from V1:
Jonathan
pci_alloc_irq_vectors()cleans up the vectors automatically usemsi_enabledrather thannr_irq_vecsChanges from Ira
Remove reviews
Allocate up to a static 16 vectors.
Change cover letter--- drivers/cxl/cxlmem.h | 3 +++ drivers/cxl/cxlpci.h | 6 ++++++ drivers/cxl/pci.c | 23 +++++++++++++++++++++++ 3 files changed, 32 insertions(+)drivers/cxl/cxlmem.h +1
-0@@ -211,6 +211,7 @@ struct cxl_endpoint_dvsec_info {* @info: Cached DVSEC information about the device. * @serial: PCIe Device Serial Number * @doe_mbs: PCI DOE mailbox array +* @msi_enabled: MSI-X/MSI has been enabled * @mbox_send: @dev specific transport for transmitting mailbox commands * * See section 8.2.9.5.2 Capacity Configuration and Label Storage for @@ -247,6 +248,8 @@ struct cxl_dev_state {
Quoted 7 lines
drivers/cxl/cxlmem.h struct cxl_dev_state { · line 252
Quoted 4 lines
drivers/cxl/cxlmem.h +2
-0struct xarray doe_mbs; +bool msi_enabled; +This goes unused in this patch and it also duplicates what the core offers with pdev->{msi,msix}_enabled.
I tried to argue that with Jonathan and lost. What I had in V1 was to store the number actually allocated. Then if a function reports something higher later it can't be used.
I admit that at this point I really don't understand PCI interrupts at all. Every time this patch is discussed I get (what is to me) confusing information. And I've been unable to discern from the spec how exactly this is all supposed to work.
Quoted 45 lines
Quoted 32 lines
drivers/cxl/cxlmem.h +0
-0int (*mbox_send)(struct cxl_dev_state *cxlds,struct cxl_mbox_cmd *cmd); };drivers/cxl/cxlpci.h +6
-0@@ -53,6 +53,12 @@#define CXL_DVSEC_REG_LOCATOR_BLOCK_ID_MASK GENMASK(15, 8)#define CXL_DVSEC_REG_LOCATOR_BLOCK_OFF_LOW_MASK GENMASK(31, 16) +/* +* NOTE: Currently all the functions which are enabled for CXL require their +* vectors to be in the first 16. Use this as the max. +*/ +#define CXL_PCI_REQUIRED_VECTORS 16 +/* Register Block Identifier (RBI) */ enum cxl_regloc_type {CXL_REGLOC_RBI_EMPTY =0 ,drivers/cxl/pci.c +3
-0@@ -428,6 +428,27 @@ static void devm_cxl_pci_create_doe(struct cxl_dev_state *cxlds) } } +static void cxl_pci_alloc_irq_vectors (struct cxl_dev_state *cxlds) +{ +struct device *dev = cxlds->dev;drivers/cxl/pci.c static void devm_cxl_pci_create_doe(struct cxl_dev_state *cxlds) · line 443
Quoted 10 lines
drivers/cxl/pci.c +10
-0+struct pci_dev *pdev =to_pci_dev (dev); +int nvecs; + +/* +* NOTE: pci_alloc_irq_vectors() handles calling pci_free_irq_vectors() +* automatically despite not being called pcim_*. See +* pci_setup_msi_context(). +*/ + nvecs =pci_alloc_irq_vectors (pdev,1 ,CXL_PCI_REQUIRED_VECTORS , +PCI_IRQ_MSIX |PCI_IRQ_MSI );clang-format would scooch that second line in for you.
Quoted 6 lines
Might also be worth a comment for the next person that goes looking for why this isn't
PCI_IRQ_ALL_TYPES.From CXL 3.0 3.1.1 CXL.io Endpoint:
A Function on a CXL device must not generate INTx messages if that Function participates in CXL.cache protocol or CXL.mem protocols.
Seems reasonable.
Ira
Quoted 27 lines
Quoted 25 lines
drivers/cxl/pci.c +10
-0+if (nvecs <0 ) { +dev_dbg (dev,"Failed to alloc irq vectors; use polling instead.\n" ); +return ; + } + + cxlds->msi_enabled = true; +} +static int cxl_pci_probe (struct pci_dev *pdev,const struct pci_device_id *id) {struct cxl_register_map map;@@ -494,6 +515,8 @@ static int cxl_pci_probe(struct pci_dev *pdev, const struct pci_device_id *id)if (rc)return rc; +cxl_pci_alloc_irq_vectors (cxlds); + cxlmd =devm_cxl_add_memdev (cxlds);if (IS_ERR (cxlmd))return PTR_ERR (cxlmd);Signature · 1 line
2.37.2
Re: [PATCH V2 01/11] cxl/pci: Add generic MSI-X/MSI irq support
Quoted 81 lines · Ira Weiny
Quoted 11 lines · Dan Williams
Quoted 9 lines · ira.weiny@ wrote:
From: Davidlohr Bueso <[email protected]>
Currently the only CXL features targeted for irq support require their message numbers to be within the first 16 entries. The device may however support less than 16 entries depending on the support it provides.
Attempt to allocate these 16 irq vectors. If the device supports less then the PCI infrastructure will allocate that number.
Quoted 8 lines
What happens if the device supports 16, but irq-core allocates less? I believe the answer is with the first user, but this patch does not include a user.
Quoted 2 lines
Store the number of vectors actually allocated in the device state for later use by individual functions.
The patch does not do that.
Sorry missed updating this message.
Quoted 4 lines
I know this patch has gone through a lot of discussion, but this mismatch shows it should really be squashed with the first user because it does not stand on its own anymore.
It is separate because it was Davidlohr's to begin with.
I'll squash it back.
Quoted 41 lines
Quoted 41 lines
Upon successful allocation, users can plug in their respective isr at any point thereafter, for example, if the irq setup is not done in the PCI driver, such as the case of the CXL-PMU.
Cc Bjorn Helgaas <[email protected]>Cc Jonathan Cameron <[email protected]>Co-developed-by Ira Weiny <[email protected]>Signed-off-by Ira Weiny <[email protected]>Signed-off-by Davidlohr Bueso <[email protected]>---
Changes from V1:
Jonathan
pci_alloc_irq_vectors()cleans up the vectors automatically usemsi_enabledrather thannr_irq_vecsChanges from Ira
Remove reviews
Allocate up to a static 16 vectors.
Change cover letter--- drivers/cxl/cxlmem.h | 3 +++ drivers/cxl/cxlpci.h | 6 ++++++ drivers/cxl/pci.c | 23 +++++++++++++++++++++++ 3 files changed, 32 insertions(+)drivers/cxl/cxlmem.h +1
-0@@ -211,6 +211,7 @@ struct cxl_endpoint_dvsec_info {* @info: Cached DVSEC information about the device. * @serial: PCIe Device Serial Number * @doe_mbs: PCI DOE mailbox array +* @msi_enabled: MSI-X/MSI has been enabled * @mbox_send: @dev specific transport for transmitting mailbox commands * * See section 8.2.9.5.2 Capacity Configuration and Label Storage for @@ -247,6 +248,8 @@ struct cxl_dev_state {Quoted 7 lines
drivers/cxl/cxlmem.h struct cxl_dev_state { · line 252
Quoted 4 lines
drivers/cxl/cxlmem.h +2
-0struct xarray doe_mbs; +bool msi_enabled; +This goes unused in this patch and it also duplicates what the core offers with pdev->{msi,msix}_enabled.
Quoted 3 lines
I tried to argue that with Jonathan and lost. What I had in V1 was to store the number actually allocated. Then if a function reports something higher later it can't be used.
A successful pci_alloc_irq_vectors() call assigns a vector number to all interrupt sources on the device regardless of how many interrupt sources there are. If the device has 32 interrupt sources and 16 irqs are returned from pci_alloc_irq_vectors() then each interrupt source will be sharing a vector with one or more other vectors. All PCI IRQ vectors are shared.
So I do not see the point of this msi_enabled flag cxl_dev_state. If pci_alloc_irq_vectors() returns at least 1 then you are good to go.
Re: [PATCH V2 01/11] cxl/pci: Add generic MSI-X/MSI irq support
Quoted 86 lines · On Thu, 1 Dec 2022 18:00:59 -0800 Dan Williams <[email protected]> wrote:
Quoted 81 lines · Ira Weiny
Quoted 11 lines · Dan Williams
Quoted 9 lines · ira.weiny@ wrote:
From: Davidlohr Bueso <[email protected]>
Currently the only CXL features targeted for irq support require their message numbers to be within the first 16 entries. The device may however support less than 16 entries depending on the support it provides.
Attempt to allocate these 16 irq vectors. If the device supports less then the PCI infrastructure will allocate that number.
Quoted 8 lines
What happens if the device supports 16, but irq-core allocates less? I believe the answer is with the first user, but this patch does not include a user.
Quoted 2 lines
Store the number of vectors actually allocated in the device state for later use by individual functions.
The patch does not do that.
Sorry missed updating this message.
Quoted 4 lines
I know this patch has gone through a lot of discussion, but this mismatch shows it should really be squashed with the first user because it does not stand on its own anymore.
It is separate because it was Davidlohr's to begin with.
I'll squash it back.
Quoted 41 lines
Quoted 41 lines
Upon successful allocation, users can plug in their respective isr at any point thereafter, for example, if the irq setup is not done in the PCI driver, such as the case of the CXL-PMU.
Cc Bjorn Helgaas <[email protected]>Cc Jonathan Cameron <[email protected]>Co-developed-by Ira Weiny <[email protected]>Signed-off-by Ira Weiny <[email protected]>Signed-off-by Davidlohr Bueso <[email protected]>---
Changes from V1:
Jonathan
pci_alloc_irq_vectors()cleans up the vectors automatically usemsi_enabledrather thannr_irq_vecsChanges from Ira
Remove reviews
Allocate up to a static 16 vectors.
Change cover letter--- drivers/cxl/cxlmem.h | 3 +++ drivers/cxl/cxlpci.h | 6 ++++++ drivers/cxl/pci.c | 23 +++++++++++++++++++++++ 3 files changed, 32 insertions(+)drivers/cxl/cxlmem.h +1
-0@@ -211,6 +211,7 @@ struct cxl_endpoint_dvsec_info {* @info: Cached DVSEC information about the device. * @serial: PCIe Device Serial Number * @doe_mbs: PCI DOE mailbox array +* @msi_enabled: MSI-X/MSI has been enabled * @mbox_send: @dev specific transport for transmitting mailbox commands * * See section 8.2.9.5.2 Capacity Configuration and Label Storage for @@ -247,6 +248,8 @@ struct cxl_dev_state {Quoted 7 lines
drivers/cxl/cxlmem.h struct cxl_dev_state { · line 252
Quoted 4 lines
drivers/cxl/cxlmem.h +2
-0struct xarray doe_mbs; +bool msi_enabled; +This goes unused in this patch and it also duplicates what the core offers with pdev->{msi,msix}_enabled.
Quoted 3 lines
I tried to argue that with Jonathan and lost. What I had in V1 was to store the number actually allocated. Then if a function reports something higher later it can't be used.
Quoted 5 lines
A successful
pci_alloc_irq_vectors()call assigns a vector number to all interrupt sources on the device regardless of how many interrupt sources there are. If the device has 32 interrupt sources and 16 irqs are returned frompci_alloc_irq_vectors()then each interrupt source will be sharing a vector with one or more other vectors. All PCI IRQ vectors are shared.
Assuming my understanding is correct...
Subtle tweak to that description (not that it matters in practice). Some of the vectors will be shared. For MSI at least it is up to the device to assign msgnums in whatever way it likes such that they are fit in the number that were enabled. So it 'could' put them all on the first msgnum if it wants to, or put any that would otherwise have been greater than 16 on msgnum 15. Impdef how it decides that spread.
MSIX is has a layer of indirection in control of software, so it gets more complex...
Quoted 3 lines
So I do not see the point of this
msi_enabledflagcxl_dev_state. Ifpci_alloc_irq_vectors()returns at least 1 then you are good to go.
PATCH v2 2/11 cxl/mem: Implement Get Event Records command
In reply to [email protected]From: Ira Weiny <[email protected]>
CXL devices have multiple event logs which can be queried for CXL event records. Devices are required to support the storage of at least one event record in each event log type.
Devices track event log overflow by incrementing a counter and tracking the time of the first and last overflow event seen.
Software queries events via the Get Event Record mailbox command; CXL rev 3.0 section 8.2.9.2.2.
Issue the Get Event Record mailbox command on driver load. Trace each record found with a generic record trace. Trace any overflow conditions.
The device can return up to 1MB worth of event records per query. Allocate a shared large buffer to handle the max number of records based on the mailbox payload size.
This patch traces a raw event record only and leaves the specific event record types to subsequent patches.
Macros are created to use for tracing the common CXL Event header fields.
---
Change from V1:
Ignore useless More Event Flag
defer DCD support
Jonathan
delete extra blank line
Use all caps for flags
Jonathan/Dan/Ira
Allocate event MB buffer on start up.
Alison
s/pl_nr/nr_pl
Change from RFC v2:
Support reading 3 events at once.
Reverse Jonathan's suggestion and check for positive number of
records. Because the record count may have been returned as something > 3 based on what the device thinks it can send back even though the core Linux mbox processing truncates the data.
Alison and Dave Jiang
Change header uuid type to uuid_t for better user space processing
Smita
Check status reg before reading log.
Steven
Prefix all trace points with 'cxl_'
Use static branch <trace>_enabled() calls
Jonathan
s/CXL_EVENT_TYPE_INFO/0 s/{first,last}/{first,last}_ts
Remove Reserved field from header
Fix header issue for cxl_event_log_type_str()
Change from RFC:
Remove redundant error message in get event records loop s/EVENT_RECORD_DATA_LENGTH/CXL_EVENT_RECORD_DATA_LENGTH Use hdr_uuid for the header UUID field
Use cxl_event_log_type_str() for the trace events
Create macros for the header fields and common entries of each event Add reserved buffer output dump
Report error if event query fails
Remove unused record_cnt variable
Steven - reorder overflow record
Remove NOTE about checkpatch
Jonathan
check for exactly 1 record
s/v3.0/rev 3.0
Use 3 byte fields for 24bit fields
Add 3.0 Maintenance Operation Class
Add Dynamic Capacity log type
Fix spelling
Dave Jiang/Dan/Alison
s/cxl-event/cxl
trace/events/cxl-events => trace/events/cxl.h s/cxl_event_overflow/overflow s/cxl_event/generic_event
--- MAINTAINERS | 1 + drivers/cxl/core/mbox.c | 105 +++++++++++++++++++++++++++++ drivers/cxl/cxl.h | 7 ++ drivers/cxl/cxlmem.h | 72 ++++++++++++++++++++ include/trace/events/cxl.h | 126 +++++++++++++++++++++++++++++++++++ include/uapi/linux/cxl_mem.h | 1 + 6 files changed, 312 insertions(+) create mode 100644 include/trace/events/cxl.h
MAINTAINERS +1 -0
@@ -5223,6 +5223,7 @@ M: Dan Williams <[email protected]> L: [email protected] S: Maintained F: drivers/cxl/ +F: include/trace/events/cxl.h F: include/uapi/linux/cxl_mem.h CONEXANT ACCESSRUNNER USB DRIVER
drivers/cxl/core/mbox.c +105 -0
129 lines not on this page · raw message
drivers/cxl/cxl.h +7 -0
@@ -132,6 +132,13 @@ static inline int ways_to_cxl(unsigned int ways, u8 *iw)#define CXLDEV_CAP_CAP_ID_SECONDARY_MAILBOX 0x3#define CXLDEV_CAP_CAP_ID_MEMDEV 0x4000 +/* CXL 3.0 8.2.8.3.1 Event Status Register */ +#define CXLDEV_DEV_EVENT_STATUS_OFFSET 0x00 +#define CXLDEV_EVENT_STATUS_INFO BIT(0) +#define CXLDEV_EVENT_STATUS_WARN BIT(1) +#define CXLDEV_EVENT_STATUS_FAIL BIT(2) +#define CXLDEV_EVENT_STATUS_FATAL BIT(3) +/* CXL 2.0 8.2.8.4 Mailbox Registers */ #define CXLDEV_MBOX_CAPS_OFFSET 0x00#define CXLDEV_MBOX_CAP_PAYLOAD_SIZE_MASK GENMASK(4, 0)
drivers/cxl/cxlmem.h +72 -0
102 lines not on this page · raw message
include/trace/events/cxl.h new file +126 -0
126 lines not on this page · raw message
include/uapi/linux/cxl_mem.h +1 -0
@@ -24,6 +24,7 @@___C (IDENTIFY ,"Identify Command" ), \___C (RAW ,"Raw device command" ), \___C (GET_SUPPORTED_LOGS ,"Get Supported Logs" ), \ +___C (GET_EVENT_RECORD ,"Get Event Record" ), \___C (GET_FW_INFO ,"Get FW Info" ), \___C (GET_PARTITION_INFO ,"Get Partition Information" ), \ ___C(GET_LSA, "Get Label Storage Area"), \
Signature · 1 line
2.37.2
Re: [PATCH V2 02/11] cxl/mem: Implement Get Event Records command
Quoted 20 lines · On Wed, 30 Nov 2022 16:27:10 -0800 [email protected] wrote:
From: Ira Weiny <[email protected]>
CXL devices have multiple event logs which can be queried for CXL event records. Devices are required to support the storage of at least one event record in each event log type.
Devices track event log overflow by incrementing a counter and tracking the time of the first and last overflow event seen.
Software queries events via the Get Event Record mailbox command; CXL rev 3.0 section 8.2.9.2.2.
Issue the Get Event Record mailbox command on driver load. Trace each record found with a generic record trace. Trace any overflow conditions.
The device can return up to 1MB worth of event records per query. Allocate a shared large buffer to handle the max number of records based on the mailbox payload size.
Quoted 8 lines
This patch traces a raw event record only and leaves the specific event record types to subsequent patches.
Macros are created to use for tracing the common CXL Event header fields.
Cc Steven Rostedt <[email protected]>Signed-off-by Ira Weiny <[email protected]>
Hi Ira,
Looks good to me. A few trivial suggestions inline. Either way,
Quoted 5 lines
drivers/cxl/core/mbox.c +0
-0@@ -7,6 +7,9 @@
...
drivers/cxl/core/mbox.c int cxl_enumerate_cmds(struct cxl_dev_state *cxlds) · line 758
Quoted 6 lines
drivers/cxl/core/mbox.c +6
-0+ +static void cxl_mem_free_event_buffer (void *data) +{ +struct cxl_dev_state *cxlds = data; + +kvfree (cxlds->event_buf);
Trivial, but why not just pass in the event_buf?
Quoted 4 lines
drivers/cxl/core/mbox.c +4
-0+} + +/* +* There is a single buffer for reading event logs from the mailbox. All logs
drivers/cxl/core/mbox.c int cxl_enumerate_cmds(struct cxl_dev_state *cxlds) · line 772
Quoted 10 lines
drivers/cxl/core/mbox.c +10
-0+* share this buffer protected by the cxlds->event_buf_lock. +*/ +static struct cxl_get_event_payload *alloc_event_buf (struct cxl_dev_state *cxlds) +{ +struct cxl_get_event_payload *buf; + +dev_dbg (cxlds->dev,"Allocating event buffer size %zu\n" , + cxlds->payload_size); + + buf =kvmalloc (cxlds->payload_size,GFP_KERNEL );
huh. I assumed there would be a devm_kvmalloc() but apparently not.. Ah well
- whilst it might makes sense to add one, let's not tie that up with this series.
drivers/cxl/core/mbox.c int cxl_enumerate_cmds(struct cxl_dev_state *cxlds) · line 775
Quoted 3 lines
drivers/cxl/core/mbox.c +3
-0+if (buf &&devm_add_action_or_reset (cxlds->dev, + cxl_mem_free_event_buffer, cxlds)) +return NULL ;
Trivial, but I'd go for a more wordy but more conventional pattern of
if (!buf) return NULL; if (devm_add_action_or_reset()) return NULL return buff;
Quoted 3 lines
drivers/cxl/core/mbox.c +3
-0+return buf; +} +
...
Quoted 13 lines
drivers/cxl/cxlmem.h +1
-0@@ -4,6 +4,7 @@#define __CXL_MEM_H__#include <uapi/linux/cxl_mem.h> #include <linux/cdev.h> +#include <linux/uuid.h> #include "cxl.h" /* CXL 2.0 8.2.8.5.1.1 Memory Device Status Register */
drivers/cxl/cxlmem.h struct cxl_dev_state { · line 254
Quoted 4 lines
drivers/cxl/cxlmem.h +1
-0@@ -250,12 +251,16 @@ struct cxl_dev_state {bool msi_enabled; +struct cxl_get_event_payload *event_buf;
Whilst it is obvious (and document at point of allocation), I think one of the static checkers still warns that all locks must have comments. Probably easier to add one now than wait for the inevitable warning report.
Quoted 5 lines
drivers/cxl/cxlmem.h +2
-0+struct mutex event_buf_lock; +int (*mbox_send)(struct cxl_dev_state *cxlds,struct cxl_mbox_cmd *cmd); };
Re: [PATCH V2 02/11] cxl/mem: Implement Get Event Records command
Quoted 47 lines · Jonathan Cameron
Quoted 20 lines · On Wed, 30 Nov 2022 16:27:10 -0800 [email protected] wrote:
From: Ira Weiny <[email protected]>
CXL devices have multiple event logs which can be queried for CXL event records. Devices are required to support the storage of at least one event record in each event log type.
Devices track event log overflow by incrementing a counter and tracking the time of the first and last overflow event seen.
Software queries events via the Get Event Record mailbox command; CXL rev 3.0 section 8.2.9.2.2.
Issue the Get Event Record mailbox command on driver load. Trace each record found with a generic record trace. Trace any overflow conditions.
The device can return up to 1MB worth of event records per query. Allocate a shared large buffer to handle the max number of records based on the mailbox payload size.
Quoted 8 lines
This patch traces a raw event record only and leaves the specific event record types to subsequent patches.
Macros are created to use for tracing the common CXL Event header fields.
Cc Steven Rostedt <[email protected]>Signed-off-by Ira Weiny <[email protected]>Hi Ira,
Looks good to me. A few trivial suggestions inline. Either way,
Reviewed-by Jonathan Cameron <[email protected]>Quoted 5 lines
drivers/cxl/core/mbox.c +0
-0@@ -7,6 +7,9 @@...
Quoted 8 lines
drivers/cxl/core/mbox.c int cxl_enumerate_cmds(struct cxl_dev_state *cxlds) · line 758
Quoted 6 lines
drivers/cxl/core/mbox.c +6
-0+ +static void cxl_mem_free_event_buffer (void *data) +{ +struct cxl_dev_state *cxlds = data; + +kvfree (cxlds->event_buf);Trivial, but why not just pass in the
event_buf?
Just following the pattern that 'cxl_mem_*' functions take a cxlds parameter. <shrug>
I'm going to leave this because it is tested.
Quoted 17 lines
Quoted 5 lines
drivers/cxl/core/mbox.c +4
-0+} + +/* +* There is a single buffer for reading event logs from the mailbox. All logs drivers/cxl/core/mbox.c int cxl_enumerate_cmds(struct cxl_dev_state *cxlds) · line 772
Quoted 10 lines
drivers/cxl/core/mbox.c +10
-0+* share this buffer protected by the cxlds->event_buf_lock. +*/ +static struct cxl_get_event_payload *alloc_event_buf (struct cxl_dev_state *cxlds) +{ +struct cxl_get_event_payload *buf; + +dev_dbg (cxlds->dev,"Allocating event buffer size %zu\n" , + cxlds->payload_size); + + buf =kvmalloc (cxlds->payload_size,GFP_KERNEL );huh. I assumed there would be a
devm_kvmalloc()but apparently not.. Ah well
Nope I've learned my lesson and checked first!
Quoted 1 line
- whilst it might makes sense to add one, let's not tie that up with this series.
Yep I did not want to hold this up for something like that.
Quoted 11 lines
drivers/cxl/core/mbox.c int cxl_enumerate_cmds(struct cxl_dev_state *cxlds) · line 775
Quoted 4 lines
drivers/cxl/core/mbox.c +3
-0+if (buf &&devm_add_action_or_reset (cxlds->dev, + cxl_mem_free_event_buffer, cxlds)) +return NULL ;Trivial, but I'd go for a more wordy but more conventional pattern of
if (!buf) return NULL; if (devm_add_action_or_reset()) return NULL
I've been beat up in the past for not combining statements before. So I've a bad habit sometimes.
This pattern is a bit more clear. Since I'm adding the comment below I'll change it.
Quoted 23 lines
return buff;Quoted 3 lines
drivers/cxl/core/mbox.c +3
-0+return buf; +} +...
Quoted 14 lines
drivers/cxl/cxlmem.h +1
-0@@ -4,6 +4,7 @@#define __CXL_MEM_H__#include <uapi/linux/cxl_mem.h> #include <linux/cdev.h> +#include <linux/uuid.h> #include "cxl.h" /* CXL 2.0 8.2.8.5.1.1 Memory Device Status Register */ @@ -250,12 +251,16 @@ struct cxl_dev_state {
Quoted 7 lines
drivers/cxl/cxlmem.h struct cxl_dev_state { · line 254
Quoted 3 lines
drivers/cxl/cxlmem.h +1
-0bool msi_enabled; +struct cxl_get_event_payload *event_buf;Whilst it is obvious (and document at point of allocation), I think one of the static checkers still warns that all locks must have comments. Probably easier to add one now than wait for the inevitable warning report.
Well 0-day did not complain. :-/ But I know there are other checkers out there; better to add now, thanks.
Thanks for the review,
Ira
Quoted 8 lines
Quoted 6 lines
drivers/cxl/cxlmem.h +2
-0+struct mutex event_buf_lock; +int (*mbox_send)(struct cxl_dev_state *cxlds,struct cxl_mbox_cmd *cmd); };
Steven Rostedt
maintainerRe: [PATCH V2 02/11] cxl/mem: Implement Get Event Records command
In reply to [email protected]Quoted 244 lines · On Wed, 30 Nov 2022 16:27:10 -0800 [email protected] wrote:
CONEXANT ACCESSRUNNER USB DRIVER
drivers/cxl/core/mbox.c +105
-0@@ -7,6 +7,9 @@#include <cxlmem.h> #include <cxl.h> +#define CREATE_TRACE_POINTS +#include <trace/events/cxl.h> +#include "core.h" static bool cxl_raw_allow_all;@@ -48,6 +51,7 @@ static struct cxl_mem_command cxl_mem_commands[CXL_MEM_COMMAND_ID_MAX] = {CXL_CMD (RAW ,CXL_VARIABLE_PAYLOAD ,CXL_VARIABLE_PAYLOAD ,0 ), #endifCXL_CMD (GET_SUPPORTED_LOGS ,0 ,CXL_VARIABLE_PAYLOAD ,CXL_CMD_FLAG_FORCE_ENABLE ), +CXL_CMD (GET_EVENT_RECORD ,1 ,CXL_VARIABLE_PAYLOAD ,0 ),CXL_CMD (GET_FW_INFO ,0 ,0x50 ,0 ),CXL_CMD (GET_PARTITION_INFO ,0 ,0x20 ,0 ), CXL_CMD(GET_LSA, 0x8, CXL_VARIABLE_PAYLOAD, 0),@@ -704,6 +708,106 @@ int cxl_enumerate_cmds(struct cxl_dev_state *cxlds) }EXPORT_SYMBOL_NS_GPL (cxl_enumerate_cmds,CXL ); +static void cxl_mem_get_records_log (struct cxl_dev_state *cxlds, +enum cxl_event_log_type type) +{ +struct cxl_get_event_payload *payload; +u16 nr_rec; + +mutex_lock (&cxlds->event_buf_lock); + + payload = cxlds->event_buf; + +do { +u8 log_type = type; +int rc; + + rc =cxl_mbox_send_cmd (cxlds,CXL_MBOX_OP_GET_EVENT_RECORD , + &log_type,sizeof (log_type), + payload, cxlds->payload_size); +if (rc) { +dev_err (cxlds->dev,"Event log '%s': Failed to query event records : %d" , +cxl_event_log_type_str (type), rc); + goto unlock_buffer; + } + + nr_rec =le16_to_cpu (payload->record_count); +if (trace_cxl_generic_event_enabled ()) { +int i; + +for (i =0 ; i < nr_rec; i++) +trace_cxl_generic_event (dev_name (cxlds->dev), + type, + &payload->records[i]); + } + +if (trace_cxl_overflow_enabled () && + (payload->flags &CXL_GET_EVENT_FLAG_OVERFLOW )) +trace_cxl_overflow (dev_name (cxlds->dev), type, payload); + + }while (nr_rec); + +unlock_buffer: +mutex_unlock (&cxlds->event_buf_lock); +} + +static void cxl_mem_free_event_buffer (void *data) +{ +struct cxl_dev_state *cxlds = data; + +kvfree (cxlds->event_buf); +} + +/* +* There is a single buffer for reading event logs from the mailbox. All logs +* share this buffer protected by the cxlds->event_buf_lock. +*/ +static struct cxl_get_event_payload *alloc_event_buf (struct cxl_dev_state *cxlds) +{ +struct cxl_get_event_payload *buf; + +dev_dbg (cxlds->dev,"Allocating event buffer size %zu\n" , + cxlds->payload_size); + + buf =kvmalloc (cxlds->payload_size,GFP_KERNEL ); +if (buf &&devm_add_action_or_reset (cxlds->dev, + cxl_mem_free_event_buffer, cxlds)) +return NULL ; +return buf; +} + +/** +* cxl_mem_get_event_records - Get Event Records from the device +* @cxlds: The device data for the operation +* +* Retrieve all event records available on the device and report them as trace +* events. +* +* See CXL rev 3.0 @8.2.9.2.2 Get Event Records +*/ +void cxl_mem_get_event_records (struct cxl_dev_state *cxlds) +{ +u32 status =readl (cxlds->regs.status +CXLDEV_DEV_EVENT_STATUS_OFFSET ); + +dev_dbg (cxlds->dev,"Reading event logs: %x\n" , status); + +if (!cxlds->event_buf) { + cxlds->event_buf =alloc_event_buf (cxlds); +if (WARN_ON_ONCE (!cxlds->event_buf)) +return ; + } + +if (status &CXLDEV_EVENT_STATUS_INFO ) +cxl_mem_get_records_log (cxlds,CXL_EVENT_TYPE_INFO ); +if (status &CXLDEV_EVENT_STATUS_WARN ) +cxl_mem_get_records_log (cxlds,CXL_EVENT_TYPE_WARN ); +if (status &CXLDEV_EVENT_STATUS_FAIL ) +cxl_mem_get_records_log (cxlds,CXL_EVENT_TYPE_FAIL ); +if (status &CXLDEV_EVENT_STATUS_FATAL ) +cxl_mem_get_records_log (cxlds,CXL_EVENT_TYPE_FATAL ); +} +EXPORT_SYMBOL_NS_GPL (cxl_mem_get_event_records,CXL ); +/** * cxl_mem_get_partition_info - Get partition info * @cxlds: The device data for the operation @@ -846,6 +950,7 @@ struct cxl_dev_state *cxl_dev_state_create(struct device *dev) }mutex_init (&cxlds->mbox_mutex); +mutex_init (&cxlds->event_buf_lock); cxlds->dev = dev;return cxlds;drivers/cxl/cxl.h +7
-0@@ -132,6 +132,13 @@ static inline int ways_to_cxl(unsigned int ways, u8 *iw)#define CXLDEV_CAP_CAP_ID_SECONDARY_MAILBOX 0x3#define CXLDEV_CAP_CAP_ID_MEMDEV 0x4000 +/* CXL 3.0 8.2.8.3.1 Event Status Register */ +#define CXLDEV_DEV_EVENT_STATUS_OFFSET 0x00 +#define CXLDEV_EVENT_STATUS_INFO BIT(0) +#define CXLDEV_EVENT_STATUS_WARN BIT(1) +#define CXLDEV_EVENT_STATUS_FAIL BIT(2) +#define CXLDEV_EVENT_STATUS_FATAL BIT(3) +/* CXL 2.0 8.2.8.4 Mailbox Registers */ #define CXLDEV_MBOX_CAPS_OFFSET 0x00#define CXLDEV_MBOX_CAP_PAYLOAD_SIZE_MASK GENMASK(4, 0)drivers/cxl/cxlmem.h +60
-0@@ -4,6 +4,7 @@#define __CXL_MEM_H__#include <uapi/linux/cxl_mem.h> #include <linux/cdev.h> +#include <linux/uuid.h> #include "cxl.h" /* CXL 2.0 8.2.8.5.1.1 Memory Device Status Register */ @@ -250,12 +251,16 @@ struct cxl_dev_state {bool msi_enabled; +struct cxl_get_event_payload *event_buf; +struct mutex event_buf_lock; +int (*mbox_send)(struct cxl_dev_state *cxlds,struct cxl_mbox_cmd *cmd); };enum cxl_opcode {CXL_MBOX_OP_INVALID =0x0000 ,CXL_MBOX_OP_RAW =CXL_MBOX_OP_INVALID , +CXL_MBOX_OP_GET_EVENT_RECORD =0x0100 ,CXL_MBOX_OP_GET_FW_INFO =0x0200 ,CXL_MBOX_OP_ACTIVATE_FW =0x0202 ,CXL_MBOX_OP_GET_SUPPORTED_LOGS =0x0400 ,@@ -325,6 +330,72 @@ struct cxl_mbox_identify {u8 qos_telemetry_caps; } __packed; +/* +* Common Event Record Format +* CXL rev 3.0 section 8.2.9.2.1; Table 8-42 +*/ +struct cxl_event_record_hdr { +uuid_t id; +u8 length; +u8 flags[3 ]; +__le16 handle; +__le16 related_handle; +__le64 timestamp; +u8 maint_op_class; +u8 reserved[0xf ]; +} __packed; + +#define CXL_EVENT_RECORD_DATA_LENGTH 0x50 +struct cxl_event_record_raw { +struct cxl_event_record_hdr hdr; +u8 data[CXL_EVENT_RECORD_DATA_LENGTH ]; +} __packed; + +/* +* Get Event Records output payload +* CXL rev 3.0 section 8.2.9.2.2; Table 8-50 +*/ +#define CXL_GET_EVENT_FLAG_OVERFLOW BIT(0) +#define CXL_GET_EVENT_FLAG_MORE_RECORDS BIT(1) +struct cxl_get_event_payload { +u8 flags; +u8 reserved1; +__le16 overflow_err_count; +__le64 first_overflow_timestamp; +__le64 last_overflow_timestamp; +__le16 record_count; +u8 reserved2[0xa ]; +struct cxl_event_record_raw records[]; +} __packed; + +/* +* CXL rev 3.0 section 8.2.9.2.2; Table 8-49 +*/ +enum cxl_event_log_type { +CXL_EVENT_TYPE_INFO =0x00 , +CXL_EVENT_TYPE_WARN , +CXL_EVENT_TYPE_FAIL , +CXL_EVENT_TYPE_FATAL , +CXL_EVENT_TYPE_MAX +}; + +static inline const char *cxl_event_log_type_str (enum cxl_event_log_type type) +{ +switch (type) { +case CXL_EVENT_TYPE_INFO : +return "Informational" ; +case CXL_EVENT_TYPE_WARN :
drivers/cxl/cxlmem.h struct cxl_mbox_identify { · line 397
Quoted 10 lines
drivers/cxl/cxlmem.h +10
-0+return "Warning" ; +case CXL_EVENT_TYPE_FAIL : +return "Failure" ; +case CXL_EVENT_TYPE_FATAL : +return "Fatal" ; +default : +break ; + } +return "<unknown>" ; +}
So you are using this in a TP_printk() section, which means perf and trace-cmd have no idea how to parse it. Can I recommend instead having:
#define cxl_event_log_type_str(type) \
__print_symbolic(type, \
{ CXL_EVENT_TYPE_INFO, "Informational" }, \
{ CXL_EVENT_TYPE_WARN, "Warning" }, \
{ CXL_EVENT_TYPE_FAIL, "Failure" }, \
{ CXL_EVENT_TYPE_FATAL, "Fatal" })#ifndef CREATE_TRACE_POINTS
static inline const char *__cxl_event_log_type_str(enum cxl_event_log_type type,
struct trace_print_flags *symbols)
{
for (; symbols->mask >= 0; symbols++) {
if (type == symbols->mask)
return symbols->name;
}
return "<unknown>";
}
#define __print_symbolic(value, symbol_array...) \
({ \
static const struct trace_print_flags symbols[] = \
{ symbol_array, { -1, NULL }}; \
__cxl_event_log_type_str(value, symbols); \
})#endif
Note, I did not even try to compile the above. But it should be close to working.
This way, the cxl_event_log_type_str() for trace events will be converted into the __print_symbolic() which can be parsed by perf and trace-cmd. For all other use cases, it is converted into the function above to return the string.
-- Steve
Quoted 156 lines
drivers/cxl/cxlmem.h +2
-0+struct cxl_mbox_get_partition_info {__le64 active_volatile_cap;__le64 active_persistent_cap;@@ -384,6 +455,7 @@ int cxl_mem_create_range_info(struct cxl_dev_state *cxlds);struct cxl_dev_state *cxl_dev_state_create (struct device *dev);void set_exclusive_cxl_commands (struct cxl_dev_state *cxlds,unsigned long *cmds);void clear_exclusive_cxl_commands (struct cxl_dev_state *cxlds,unsigned long *cmds); +void cxl_mem_get_event_records (struct cxl_dev_state *cxlds);#ifdef CONFIG_CXL_SUSPEND void cxl_mem_active_inc (void );void cxl_mem_active_dec (void );include/trace/events/cxl.h new file +126
-0@@ -0,0 +1,126 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#undef TRACE_SYSTEM +#define TRACE_SYSTEM cxl + +#if !defined(_CXL_TRACE_EVENTS_H) || defined(TRACE_HEADER_MULTI_READ ) +#define _CXL_TRACE_EVENTS_H + +#include <asm-generic/unaligned.h> +#include <linux/tracepoint.h> +#include <cxlmem.h> + +TRACE_EVENT (cxl_overflow, + + TP_PROTO(const char *dev_name, enum cxl_event_log_type log, + struct cxl_get_event_payload *payload), + +TP_ARGS (dev_name, log, payload), + +TP_STRUCT__entry ( + __string(dev_name, dev_name) + __field(int, log) + __field(u64, first_ts) + __field(u64, last_ts) +__field (u16, count) + ), + +TP_fast_assign ( + __assign_str(dev_name, dev_name); + __entry->log = log; + __entry->count = le16_to_cpu(payload->overflow_err_count); + __entry->first_ts = le64_to_cpu(payload->first_overflow_timestamp); + __entry->last_ts = le64_to_cpu(payload->last_overflow_timestamp); + ), + +TP_printk ("%s: EVENT LOG OVERFLOW log=%s : %u records from %llu to %llu" , +__get_str (dev_name),cxl_event_log_type_str (__entry->log), + __entry->count, __entry->first_ts, __entry->last_ts) + +); + +/* +* Common Event Record Format +* CXL 3.0 section 8.2.9.2.1; Table 8-42 +*/ +#define CXL_EVENT_RECORD_FLAG_PERMANENT BIT(2) +#define CXL_EVENT_RECORD_FLAG_MAINT_NEEDED BIT(3) +#define CXL_EVENT_RECORD_FLAG_PERF_DEGRADED BIT(4) +#define CXL_EVENT_RECORD_FLAG_HW_REPLACE BIT(5) +#define show_hdr_flags (flags) __print_flags(flags, " | ", \ + { CXL_EVENT_RECORD_FLAG_PERMANENT, "PERMANENT_CONDITION" }, \ + { CXL_EVENT_RECORD_FLAG_MAINT_NEEDED, "MAINTENANCE_NEEDED" }, \ + { CXL_EVENT_RECORD_FLAG_PERF_DEGRADED, "PERFORMANCE_DEGRADED" }, \ + { CXL_EVENT_RECORD_FLAG_HW_REPLACE, "HARDWARE_REPLACEMENT_NEEDED" } \ +) + +/* +* Define macros for the common header of each CXL event. +* +* Tracepoints using these macros must do 3 things: +* +* 1) Add CXL_EVT_TP_entry to TP_STRUCT__entry +* 2) Use CXL_EVT_TP_fast_assign within TP_fast_assign; +* pass the dev_name, log, and CXL event header +* 3) Use CXL_EVT_TP_printk() instead of TP_printk() +* +* See the generic_event tracepoint as an example. +*/ +#define CXL_EVT_TP_entry \ + __string(dev_name, dev_name) \ + __field(int, log) \ + __field_struct(uuid_t, hdr_uuid) \ + __field(u32, hdr_flags) \ + __field(u16, hdr_handle) \ + __field(u16, hdr_related_handle) \ + __field(u64, hdr_timestamp) \ + __field(u8, hdr_length) \ + __field(u8, hdr_maint_op_class) + +#define CXL_EVT_TP_fast_assign (dname, l, hdr) \ + __assign_str(dev_name, (dname)); \ + __entry->log = (l); \ + memcpy(&__entry->hdr_uuid, &(hdr).id, sizeof(uuid_t)); \ + __entry->hdr_length = (hdr).length; \ + __entry->hdr_flags = get_unaligned_le24((hdr).flags); \ + __entry->hdr_handle = le16_to_cpu((hdr).handle); \ + __entry->hdr_related_handle = le16_to_cpu((hdr).related_handle); \ + __entry->hdr_timestamp = le64_to_cpu((hdr).timestamp); \ + __entry->hdr_maint_op_class = (hdr).maint_op_class + +#define CXL_EVT_TP_printk (fmt, ...) \ + TP_printk("%s log=%s : time=%llu uuid=%pUb len=%d flags='%s' " \ + "handle=%x related_handle=%x maint_op_class=%u" \ + " : " fmt, \ + __get_str(dev_name), cxl_event_log_type_str(__entry->log), \ + __entry->hdr_timestamp, &__entry->hdr_uuid, __entry->hdr_length,\ + show_hdr_flags(__entry->hdr_flags), __entry->hdr_handle, \ + __entry->hdr_related_handle, __entry->hdr_maint_op_class, \ + ##__VA_ARGS__) + +TRACE_EVENT (cxl_generic_event, + + TP_PROTO(const char *dev_name, enum cxl_event_log_type log, + struct cxl_event_record_raw *rec), + +TP_ARGS (dev_name, log, rec), + +TP_STRUCT__entry ( + CXL_EVT_TP_entry +__array (u8, data,CXL_EVENT_RECORD_DATA_LENGTH ) + ), + +TP_fast_assign ( + CXL_EVT_TP_fast_assign(dev_name, log, rec->hdr); + memcpy(__entry->data, &rec->data, CXL_EVENT_RECORD_DATA_LENGTH); + ), + +CXL_EVT_TP_printk ("%s" , +__print_hex (__entry->data,CXL_EVENT_RECORD_DATA_LENGTH )) +); + +#endif /* _CXL_TRACE_EVENTS_H */ + +/* This part must be outside protection */ +#undef TRACE_INCLUDE_FILE +#define TRACE_INCLUDE_FILE cxl +#include <trace/define_trace.h> include/uapi/linux/cxl_mem.h +1
-0@@ -24,6 +24,7 @@___C (IDENTIFY ,"Identify Command" ), \___C (RAW ,"Raw device command" ), \___C (GET_SUPPORTED_LOGS ,"Get Supported Logs" ), \ +___C (GET_EVENT_RECORD ,"Get Event Record" ), \___C (GET_FW_INFO ,"Get FW Info" ), \___C (GET_PARTITION_INFO ,"Get Partition Information" ), \ ___C(GET_LSA, "Get Label Storage Area"), \
Re: [PATCH V2 02/11] cxl/mem: Implement Get Event Records command
Quoted 3 lines · Steven Rostedt
On Wed, 30 Nov 2022 16:27:10 -0800
[email protected] wrote:
[snip]
Quoted 55 lines
Quoted 18 lines
(quoted diff) +18
-0+ +/* + * CXL rev 3.0 section 8.2.9.2.2; Table 8-49 + */ +enum cxl_event_log_type { + CXL_EVENT_TYPE_INFO = 0x00, + CXL_EVENT_TYPE_WARN, + CXL_EVENT_TYPE_FAIL, + CXL_EVENT_TYPE_FATAL, + CXL_EVENT_TYPE_MAX +}; + +static inline const char *cxl_event_log_type_str(enum cxl_event_log_type type) +{ + switch (type) { + case CXL_EVENT_TYPE_INFO: + return "Informational"; + case CXL_EVENT_TYPE_WARN:Review
Quoted 10 lines
(quoted diff) +10
-0+ return "Warning"; + case CXL_EVENT_TYPE_FAIL: + return "Failure"; + case CXL_EVENT_TYPE_FATAL: + return "Fatal"; + default: + break; + } + return "<unknown>"; +}So you are using this in a
TP_printk()section, which means perf and trace-cmd have no idea how to parse it. Can I recommend instead having:#define cxl_event_log_type_str(type) \ __print_symbolic(type, \ { CXL_EVENT_TYPE_INFO, "Informational" }, \ { CXL_EVENT_TYPE_WARN, "Warning" }, \ { CXL_EVENT_TYPE_FAIL, "Failure" }, \ { CXL_EVENT_TYPE_FATAL, "Fatal" })#ifndef
CREATE_TRACE_POINTS
static inline const char *__cxl_event_log_type_str(enumcxl_event_log_typetype,
structtrace_print_flags*symbols){ for (; symbols->mask >= 0; symbols++) { if (type == symbols->mask) return symbols->name; } return "<unknown>"; } #define __print_symbolic(value, symbol_array...) \ ({ \ static const struct trace_print_flags symbols[] = \ { symbol_array, { -1, NULL }}; \ __cxl_event_log_type_str(value, symbols); \ })
Quoted 4 lines
#endif
Note, I did not even try to compile the above. But it should be close to working.
Dropping that into cxlmem.h does not compile. I've given it another go but because I use cxl_event_log_type_str() in a file where trace points are used CREATE_TRACE_POINTS is defined and I get the following error.
|| drivers/cxl/core/mbox.c: In function ‘cxl_mem_get_records_log’: drivers/cxl/cxlmem.h|386 col 7| error: implicit declaration of function ‘__print_symbolic’; did you mean ‘sprint_symbol’? [-Werror=implicit-function-declaration] || 386 | __print_symbolic(type, \ || | ^~~~~~~~~~~~~~~~
I got it to work with the patch below on top of this one.[3] But it is kind of ugly. The only way I could get __print_symbolic() to be defined was to redefine it in mbox.c.[1] Then throw it in it's own header as in [3]
NOTE that patch [2] which I think _should_ work on top of patch [1] does not. I can't understand why.
Quoted 5 lines
This way, the
cxl_event_log_type_str()for trace events will be converted into the__print_symbolic()which can be parsed by perf and trace-cmd. For all other use cases, it is converted into the function above to return the string.
I would like to have this support. I really tried to share this code a while back. What you have is seems nicer than what I remember coming up with but it is still a bit hacky IMO. And I'm afraid of how fragile this seems right now.
At this point I'm just going to define cxl_event_log_type_str() separate from the __print_symbolic() in the trace code. Comment that they should both be updated if changed and move forward.
Thanks for the suggestion but I think it is going to be more complicated than it is worth. At least for mere mortals such as myself.
Ira
[1]
For mbox.c I have to have the special redefinition of __print_symbolic() in the c file itself. Code including cxlmem.h without CREATE_TRACE_POINTS defined (like pci.c) works just fine with what you had.
commit 43a30047962312be2e532dff542d47a132949c08
Author: Ira Weiny <[email protected]> Date: Thu Dec 1 13:50:14 2022 -0800
squash: Redefine __print_symbolic to have a central define of the log string print function.
drivers/cxl/core/mbox.c +19 -1
@@ -4,12 +4,30 @@#include <linux/security.h> #include <linux/debugfs.h> #include <linux/mutex.h> -#include <cxlmem.h> #include <cxl.h> #define CREATE_TRACE_POINTS +/* Must be after CREATE_TRACE_POINTS */ +#include <cxlmem.h> #include <trace/events/cxl.h> +static inline const char *__cxl_event_log_type_str (enum cxl_event_log_type type, +const struct trace_print_flags *symbols) +{ +for (; symbols->mask >=0 ; symbols++) { +if (type == symbols->mask) +return symbols->name; + } +return "<unknown>" ; +} + +#define __print_symbolic(value, symbol_array...) \ + ({ \ + static const struct trace_print_flags symbols[] = \ + { symbol_array, { -1, NULL }}; \ + __cxl_event_log_type_str(value, symbols); \ + }) +#include "core.h" static bool cxl_raw_allow_all;
drivers/cxl/cxlmem.h +23 -14
@@ -381,23 +381,32 @@ enum cxl_event_log_type {CXL_EVENT_TYPE_MAX }; -static inline const char *cxl_event_log_type_str (enum cxl_event_log_type type) +#define cxl_event_log_type_str (type) \ + __print_symbolic(type, \ + { CXL_EVENT_TYPE_INFO, "Informational" }, \ + { CXL_EVENT_TYPE_WARN, "Warning" }, \ + { CXL_EVENT_TYPE_FAIL, "Failure" }, \ + { CXL_EVENT_TYPE_FATAL, "Fatal" }) + +#ifndef CREATE_TRACE_POINTS +static inline const char *__cxl_event_log_type_str (enum cxl_event_log_type type, +const struct trace_print_flags *symbols) { -switch (type) { -case CXL_EVENT_TYPE_INFO : -return "Informational" ; -case CXL_EVENT_TYPE_WARN : -return "Warning" ; -case CXL_EVENT_TYPE_FAIL : -return "Failure" ; -case CXL_EVENT_TYPE_FATAL : -return "Fatal" ; -default : -break ; - } -return "<unknown>" ; +for (; symbols->mask >=0 ; symbols++) { +if (type == symbols->mask) +return symbols->name; + } +return "<unknown>" ; } +#define __print_symbolic(value, symbol_array...) \ + ({ \ + static const struct trace_print_flags symbols[] = \ + { symbol_array, { -1, NULL }}; \ + __cxl_event_log_type_str(value, symbols); \ + }) +#endif +struct cxl_mbox_get_partition_info {__le64 active_volatile_cap;__le64 active_persistent_cap;
[2]
Why can't this work? Why does the undef of CREATE_TRACE_POINTS not work?
I've also tried to include __cxl_event_log_type_str() and this redefintion of __print_symbolic() in trace/events/cxl.h and that does not work.
I'm also worried this somehow breaks the support you want. But I'm still not sure how the trace headers and multiple passes work.
commit ad08110d2432fb24d4513fe9e75bb9be94870e6f
Author: Ira Weiny <[email protected]> Date: Thu Dec 1 15:01:05 2022 -0800
squash: broken!
drivers/cxl/core/mbox.c +2 -18
@@ -7,26 +7,10 @@#include <cxl.h> #define CREATE_TRACE_POINTS -/* Must be after CREATE_TRACE_POINTS */ -#include <cxlmem.h> #include <trace/events/cxl.h> -static inline const char *__cxl_event_log_type_str (enum cxl_event_log_type type, -const struct trace_print_flags *symbols) -{ -for (; symbols->mask >=0 ; symbols++) { -if (type == symbols->mask) -return symbols->name; - } -return "<unknown>" ; -} - -#define __print_symbolic(value, symbol_array...) \ - ({ \ - static const struct trace_print_flags symbols[] = \ - { symbol_array, { -1, NULL }}; \ - __cxl_event_log_type_str(value, symbols); \ - }) +#undef CREATE_TRACE_POINTS +#include <cxlmem.h> #include "core.h"
[3]
This seems to be the cleanest thing I have gotten to work. Work == compiles and trace points still print the strings.
commit c04f87639164737605c9ff503f8060b901c1b83a
Author: Ira Weiny <[email protected]> Date: Thu Dec 1 13:50:14 2022 -0800
squash: Redefine __print_symbolic to have a central define of the log string print function.
drivers/cxl/core/mbox.c +8 -1
@@ -4,12 +4,19 @@#include <linux/security.h> #include <linux/debugfs.h> #include <linux/mutex.h> -#include <cxlmem.h> #include <cxl.h> #define CREATE_TRACE_POINTS +/* Must be after CREATE_TRACE_POINTS */ +#include <cxlmem.h> #include <trace/events/cxl.h> +/* +* Must be included explicitly after trace header +* because CREATE_TRACE_POINTS can't be undefined for cxlmem.h??? +*/ +#include <cxl_event_log.h> +#include "core.h" static bool cxl_raw_allow_all;
drivers/cxl/cxl_event_log.h new file +23 -0
@@ -0,0 +1,23 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* Copyright(c) 2020 Intel Corporation. All rights reserved. */ + +/* +* Redefine __print_symbolic() from the trace code to be used in regular C code +* This compliments the define of cxl_event_log_type_str() in cxlmem.h +*/ +static inline const char *__cxl_event_log_type_str (enum cxl_event_log_type type, +const struct trace_print_flags *symbols) +{ +for (; symbols->mask >=0 ; symbols++) { +if (type == symbols->mask) +return symbols->name; + } +return "<unknown>" ; +} + +#define __print_symbolic(value, symbol_array...) \ + ({ \ + static const struct trace_print_flags symbols[] = \ + { symbol_array, { -1, NULL }}; \ + __cxl_event_log_type_str(value, symbols); \ + })
drivers/cxl/cxlmem.h +10 -16
@@ -381,22 +381,16 @@ enum cxl_event_log_type {CXL_EVENT_TYPE_MAX }; -static inline const char *cxl_event_log_type_str (enum cxl_event_log_type type) -{ -switch (type) { -case CXL_EVENT_TYPE_INFO : -return "Informational" ; -case CXL_EVENT_TYPE_WARN : -return "Warning" ; -case CXL_EVENT_TYPE_FAIL : -return "Failure" ; -case CXL_EVENT_TYPE_FATAL : -return "Fatal" ; -default : -break ; - } -return "<unknown>" ; -} +#define cxl_event_log_type_str (type) \ + __print_symbolic(type, \ + { CXL_EVENT_TYPE_INFO, "Informational" }, \ + { CXL_EVENT_TYPE_WARN, "Warning" }, \ + { CXL_EVENT_TYPE_FAIL, "Failure" }, \ + { CXL_EVENT_TYPE_FATAL, "Fatal" }) + +#ifndef CREATE_TRACE_POINTS +#include "cxl_event_log.h" +#endif struct cxl_mbox_get_partition_info {__le64 active_volatile_cap;
Steven Rostedt
maintainerRe: [PATCH V2 02/11] cxl/mem: Implement Get Event Records command
Quoted 4 lines · On Thu, 1 Dec 2022 16:09:17 -0800 Ira Weiny <[email protected]> wrote:
Dropping that into
cxlmem.hdoes not compile. I've given it another go but because I usecxl_event_log_type_str()in a file where trace points are usedCREATE_TRACE_POINTSis defined and I get the following error.
Quoted 8 lines
|| drivers/cxl/core/mbox.c: In function ‘cxl_mem_get_records_log’: drivers/cxl/cxlmem.h|386 col 7| error: implicit declaration of function ‘__print_symbolic’; did you mean ‘sprint_symbol’? [-Werror=implicit-function-declaration] || 386 | __print_symbolic(type, \ || | ^~~~~~~~~~~~~~~~I got it to work with the patch below on top of this one.[3] But it is kind of ugly. The only way I could get
__print_symbolic()to be defined was to redefine it in mbox.c.[1] Then throw it in it's own header as in [3]
I played around a bit, and with the below patch, you can just have:
#define cxl_event_log_type_str(type) \
__print_symbolic(type, \
{ CXL_EVENT_TYPE_INFO, "Informational" }, \
{ CXL_EVENT_TYPE_WARN, "Warning" }, \
{ CXL_EVENT_TYPE_FAIL, "Failure" }, \
{ CXL_EVENT_TYPE_FATAL, "Fatal" })And everything else should "just work" :-)
I can work on a more formal patch if this works for you. And thinking about this, perhaps we could add this throughout the kernel!
-- Steve
include/trace/define_trace.h +21 -0
@@ -132,4 +132,25 @@/* We may be processing more files */ #define CREATE_TRACE_POINTS +#ifndef __DEFINE_PRINT_SYMBOLIC_STR +#define __DEFINE_PRINT_SYMBOLIC_STR +static inline const char * +__print_symbolic_str (int type,struct trace_print_flags *symbols) +{ +for (; symbols->name !=NULL ; symbols++) { +if (type == symbols->mask) +return symbols->name; + } +return "<invalid>" ; +} +#endif + +#undef __print_symbolic +#define __print_symbolic(value, symbol_array...) \ + ({ \ + static const struct trace_print_flags symbols[] = \ + { symbol_array, { -1, NULL }}; \ + __print_symbolic_str(value, symbols); \ + }) +#endif /* CREATE_TRACE_POINTS */
include/trace/stages/stage7_class_define.h +0 -1
@@ -6,7 +6,6 @@#define __entry REC#undef __print_flags -#undef __print_symbolic#undef __print_hex#undef __print_hex_str#undef __get_dynamic_array
Steven Rostedt
maintainerRe: [PATCH V2 02/11] cxl/mem: Implement Get Event Records command
Review
Quoted 9 lines · On Thu, 1 Dec 2022 23:40:52 -0500 Steven Rostedt <[email protected]> wrote:
(quoted diff) +8
-0+#undef __print_symbolic +#define __print_symbolic(value, symbol_array...) \ + ({ \ + static const struct trace_print_flags symbols[] = \ + { symbol_array, { -1, NULL }}; \ + __print_symbolic_str(value, symbols); \ + }) + #endif /* CREATE_TRACE_POINTS */
Bah, I want this outside that #ifdef
include/trace/stages/stage7_class_define.h
Quoted 5 lines
include/trace/stages/stage7_class_define.h +0
-0@@ -6,7 +6,6 @@
I also don't think I need to touch stage7.
New patch:
include/trace/define_trace.h +21 -0
@@ -133,3 +133,24 @@#define CREATE_TRACE_POINTS #endif /* CREATE_TRACE_POINTS */ + +#ifndef __DEFINE_PRINT_SYMBOLIC_STR +#define __DEFINE_PRINT_SYMBOLIC_STR +static inline const char * +__print_symbolic_str (int type,struct trace_print_flags *symbols) +{ +for (; symbols->name !=NULL ; symbols++) { +if (type == symbols->mask) +return symbols->name; + } +return "<invalid>" ; +} +#endif + +#undef __print_symbolic +#define __print_symbolic(value, symbol_array...) \ + ({ \ + static const struct trace_print_flags symbols[] = \ + { symbol_array, { -1, NULL }}; \ + __print_symbolic_str(value, symbols); \ + })
Re: [PATCH V2 02/11] cxl/mem: Implement Get Event Records command
Quoted 23 lines · Steven Rostedt
include/trace/define_trace.h
Quoted 9 lines · On Thu, 1 Dec 2022 23:40:52 -0500 Steven Rostedt <[email protected]> wrote:
include/trace/define_trace.h +8
-0+#undef __print_symbolic +#define __print_symbolic(value, symbol_array...) \ + ({ \ + static const struct trace_print_flags symbols[] = \ + { symbol_array, { -1, NULL }}; \ + __print_symbolic_str(value, symbols); \ + }) +#endif /* CREATE_TRACE_POINTS */ Bah, I want this outside that #ifdef
include/trace/stages/stage7_class_define.h
Quoted 5 lines
include/trace/stages/stage7_class_define.h +0
-0@@ -6,7 +6,6 @@I also don't think I need to touch stage7.
Quoted 1 line
New patch:
I'm still going to defer this but will include a follow up patch to add this back in after the bulk of the series gets in.
Thanks for helping here. I still want to understand this all better but I have to focus on the main features ATM.
Thanks again!
Ira
Quoted 30 lines
include/trace/define_trace.h +21
-0@@ -133,3 +133,24 @@#define CREATE_TRACE_POINTS #endif /* CREATE_TRACE_POINTS */ + +#ifndef __DEFINE_PRINT_SYMBOLIC_STR +#define __DEFINE_PRINT_SYMBOLIC_STR +static inline const char * +__print_symbolic_str (int type,struct trace_print_flags *symbols) +{ +for (; symbols->name !=NULL ; symbols++) { +if (type == symbols->mask) +return symbols->name; + } +return "<invalid>" ; +} +#endif + +#undef __print_symbolic +#define __print_symbolic(value, symbol_array...) \ + ({ \ + static const struct trace_print_flags symbols[] = \ + { symbol_array, { -1, NULL }}; \ + __print_symbolic_str(value, symbols); \ + })
RE: [PATCH V2 02/11] cxl/mem: Implement Get Event Records command
In reply to [email protected]Quoted 122 lines · ira.weiny@ wrote:
From: Ira Weiny <[email protected]>
CXL devices have multiple event logs which can be queried for CXL event records. Devices are required to support the storage of at least one event record in each event log type.
Devices track event log overflow by incrementing a counter and tracking the time of the first and last overflow event seen.
Software queries events via the Get Event Record mailbox command; CXL rev 3.0 section 8.2.9.2.2.
Issue the Get Event Record mailbox command on driver load. Trace each record found with a generic record trace. Trace any overflow conditions.
The device can return up to 1MB worth of event records per query. Allocate a shared large buffer to handle the max number of records based on the mailbox payload size.
This patch traces a raw event record only and leaves the specific event record types to subsequent patches.
Macros are created to use for tracing the common CXL Event header fields.
Cc Steven Rostedt <[email protected]>Signed-off-by Ira Weiny <[email protected]>---
Change from V1:
Ignore useless More Event Flag
defer DCD support
Jonathan
delete extra blank line
Use all caps for flags
Jonathan/Dan/Ira
Allocate event MB buffer on start up.
Alison
s/pl_nr/nr_plChange from RFC v2:
Support reading 3 events at once.
Reverse Jonathan's suggestion and check for positive number of
records. Because the record count may have been returned as something > 3 based on what the device thinks it can send back even though the core Linux mbox processing truncates the data.
Alison and Dave Jiang
Change header uuid type touuid_tfor better user space processing
Smita
Check status reg before reading log.
Steven
Prefix all trace points with 'cxl_'
Use static branch <trace>_enabled() calls
Jonathan
s/CXL_EVENT_TYPE_INFO/0 s/{first,last}/{first,last}_ts
Remove Reserved field from header
Fix header issue forcxl_event_log_type_str()Change from RFC:
Remove redundant error message in get event records loop s/EVENT_RECORD_DATA_LENGTH/CXL_EVENT_RECORD_DATA_LENGTH Usehdr_uuidfor the header UUID field
Usecxl_event_log_type_str()for the trace events
Create macros for the header fields and common entries of each event Add reserved buffer output dump
Report error if event query fails
Remove unusedrecord_cntvariable
Steven - reorder overflow record
Remove NOTE about checkpatch
Jonathan
check for exactly 1 record
s/v3.0/rev 3.0
Use 3 byte fields for 24bit fields
Add 3.0 Maintenance Operation Class
Add Dynamic Capacity log type
Fix spelling
Dave Jiang/Dan/Alison
s/cxl-event/cxl
trace/events/cxl-events =>trace/events/cxl.hs/cxl_event_overflow/overflow s/cxl_event/generic_event--- MAINTAINERS | 1 + drivers/cxl/core/mbox.c | 105 +++++++++++++++++++++++++++++ drivers/cxl/cxl.h | 7 ++ drivers/cxl/cxlmem.h | 72 ++++++++++++++++++++ include/trace/events/cxl.h | 126 +++++++++++++++++++++++++++++++++++ include/uapi/linux/cxl_mem.h | 1 + 6 files changed, 312 insertions(+) create mode 100644 include/trace/events/cxl.hMAINTAINERS +1
-0@@ -5223,6 +5223,7 @@ M: Dan Williams <[email protected]> L: [email protected] S: Maintained F: drivers/cxl/ +F: include/trace/events/cxl.h F: include/uapi/linux/cxl_mem.h CONEXANT ACCESSRUNNER USB DRIVERdrivers/cxl/core/mbox.c +3
-0@@ -7,6 +7,9 @@#include <cxlmem.h> #include <cxl.h> +#define CREATE_TRACE_POINTS +#include <trace/events/cxl.h> +#include "core.h" static bool cxl_raw_allow_all;
drivers/cxl/core/mbox.c static struct cxl_mem_command cxl_mem_commands[CXL_MEM_COMMAND_ID_MAX] = { · line 57
Quoted 7 lines
drivers/cxl/core/mbox.c +1
-0@@ -48,6 +51,7 @@ static struct cxl_mem_command cxl_mem_commands[CXL_MEM_COMMAND_ID_MAX] = {CXL_CMD (RAW ,CXL_VARIABLE_PAYLOAD ,CXL_VARIABLE_PAYLOAD ,0 ), #endifCXL_CMD (GET_SUPPORTED_LOGS ,0 ,CXL_VARIABLE_PAYLOAD ,CXL_CMD_FLAG_FORCE_ENABLE ), +CXL_CMD (GET_EVENT_RECORD ,1 ,CXL_VARIABLE_PAYLOAD ,0 ),CXL_CMD (GET_FW_INFO ,0 ,0x50 ,0 ),CXL_CMD (GET_PARTITION_INFO ,0 ,0x20 ,0 ), CXL_CMD(GET_LSA, 0x8, CXL_VARIABLE_PAYLOAD, 0),
Similar to this patch:
CXL_MEM_COMMAND_ID_GET_EVENT_RECORD, should be added to the "always kernel" / cxlds->exclusive_cmds mask.
Quoted 19 lines
drivers/cxl/core/mbox.c +15
-0@@ -704,6 +708,106 @@ int cxl_enumerate_cmds(struct cxl_dev_state *cxlds) }EXPORT_SYMBOL_NS_GPL (cxl_enumerate_cmds,CXL ); +static void cxl_mem_get_records_log(struct cxl_dev_state *cxlds, + enum cxl_event_log_type type) +{ + struct cxl_get_event_payload *payload; + u16 nr_rec; + + mutex_lock(&cxlds->event_buf_lock); + + payload = cxlds->event_buf; + + do { + u8 log_type = type; + int rc; + + rc = cxl_mbox_send_cmd(cxlds, CXL_MBOX_OP_GET_EVENT_RECORD,
drivers/cxl/core/mbox.c int cxl_enumerate_cmds(struct cxl_dev_state *cxlds) · line 735
Quoted 10 lines
drivers/cxl/core/mbox.c +10
-0+ &log_type,sizeof (log_type), + payload, cxlds->payload_size); +if (rc) { +dev_err (cxlds->dev,"Event log '%s': Failed to query event records : %d" , +cxl_event_log_type_str (type), rc); + goto unlock_buffer; + } + + nr_rec =le16_to_cpu (payload->record_count); +if (trace_cxl_generic_event_enabled ()) {
This feels like a premature micro-optimization as none of this code is fast path and it is dwarfed by the cost of executing the mailbox command. I started with trying to reduce the 80 column collision pressure, but then stepped back even further and asked, why?
drivers/cxl/core/mbox.c int cxl_enumerate_cmds(struct cxl_dev_state *cxlds) · line 741
Quoted 6 lines
drivers/cxl/core/mbox.c +6
-0+int i; + +for (i =0 ; i < nr_rec; i++) +trace_cxl_generic_event (dev_name (cxlds->dev), + type, + &payload->records[i]);
As far as I can tell trace_cxl_generic_event() always expects a device-name as its first argument. So why not enforce that with type-safety? I.e. I think trace_cxl_generic_event() should take a "struct device *", not a string unless it is really the case that any old string will do as the first argument to the trace event. Otherwise the trace point can do "__string(dev_name, dev_name(dev))", and mandate that callers pass devices.
Quoted 47 lines
drivers/cxl/core/mbox.c +47
-0+ } + +if (trace_cxl_overflow_enabled () && + (payload->flags &CXL_GET_EVENT_FLAG_OVERFLOW )) +trace_cxl_overflow (dev_name (cxlds->dev), type, payload); + + }while (nr_rec); + +unlock_buffer: +mutex_unlock (&cxlds->event_buf_lock); +} + +static void cxl_mem_free_event_buffer (void *data) +{ +struct cxl_dev_state *cxlds = data; + +kvfree (cxlds->event_buf); +} + +/* +* There is a single buffer for reading event logs from the mailbox. All logs +* share this buffer protected by the cxlds->event_buf_lock. +*/ +static struct cxl_get_event_payload *alloc_event_buf (struct cxl_dev_state *cxlds) +{ +struct cxl_get_event_payload *buf; + +dev_dbg (cxlds->dev,"Allocating event buffer size %zu\n" , + cxlds->payload_size); + + buf =kvmalloc (cxlds->payload_size,GFP_KERNEL ); +if (buf &&devm_add_action_or_reset (cxlds->dev, + cxl_mem_free_event_buffer, cxlds)) +return NULL ; +return buf; +} + +/** +* cxl_mem_get_event_records - Get Event Records from the device +* @cxlds: The device data for the operation +* +* Retrieve all event records available on the device and report them as trace +* events. +* +* See CXL rev 3.0 @8.2.9.2.2 Get Event Records +*/ +void cxl_mem_get_event_records (struct cxl_dev_state *cxlds)
drivers/cxl/core/mbox.c int cxl_enumerate_cmds(struct cxl_dev_state *cxlds) · line 798
Quoted 10 lines
drivers/cxl/core/mbox.c +10
-0+{ +u32 status =readl (cxlds->regs.status +CXLDEV_DEV_EVENT_STATUS_OFFSET ); + +dev_dbg (cxlds->dev,"Reading event logs: %x\n" , status); + +if (!cxlds->event_buf) { + cxlds->event_buf =alloc_event_buf (cxlds); +if (WARN_ON_ONCE (!cxlds->event_buf)) +return ; + }
What's the point of having an event_buf_lock if event_buf is reallocated every call?
Just allocate event_buf once at the beginning of time during init if the device supports event log retrieval, and fail the driver load if that allocation fails. No runtime WARN() for memory allocation.
I notice this patch does not clear events, I trust that comes later in the series, but I think it belongs here to make this patch a complete standalone thought.
drivers/cxl/core/mbox.c int cxl_enumerate_cmds(struct cxl_dev_state *cxlds) · line 807
Quoted 8 lines
drivers/cxl/core/mbox.c +8
-0+if (status &CXLDEV_EVENT_STATUS_INFO ) +cxl_mem_get_records_log (cxlds,CXL_EVENT_TYPE_INFO ); +if (status &CXLDEV_EVENT_STATUS_WARN ) +cxl_mem_get_records_log (cxlds,CXL_EVENT_TYPE_WARN ); +if (status &CXLDEV_EVENT_STATUS_FAIL ) +cxl_mem_get_records_log (cxlds,CXL_EVENT_TYPE_FAIL ); +if (status &CXLDEV_EVENT_STATUS_FATAL ) +cxl_mem_get_records_log (cxlds,CXL_EVENT_TYPE_FATAL );
This retrieval order should be flipped. If there is a FATAL pending I expect a monitor wants that first and would be happy to parse the INFO later. I would go so far as to say that if the INFO logger is looping and a FATAL comes in the driver should get that out first before going back for more INFO logs. That would mean executing Clear Events and looping through the logs by priority until all the status bits fall silent inside cxl_mem_get_records_log().
Quoted 45 lines
drivers/cxl/core/mbox.c +4
-0+} +EXPORT_SYMBOL_NS_GPL (cxl_mem_get_event_records,CXL ); +/** * cxl_mem_get_partition_info - Get partition info * @cxlds: The device data for the operation @@ -846,6 +950,7 @@ struct cxl_dev_state *cxl_dev_state_create(struct device *dev) }mutex_init (&cxlds->mbox_mutex); +mutex_init (&cxlds->event_buf_lock); cxlds->dev = dev;return cxlds;drivers/cxl/cxl.h +7
-0@@ -132,6 +132,13 @@ static inline int ways_to_cxl(unsigned int ways, u8 *iw)#define CXLDEV_CAP_CAP_ID_SECONDARY_MAILBOX 0x3#define CXLDEV_CAP_CAP_ID_MEMDEV 0x4000 +/* CXL 3.0 8.2.8.3.1 Event Status Register */ +#define CXLDEV_DEV_EVENT_STATUS_OFFSET 0x00 +#define CXLDEV_EVENT_STATUS_INFO BIT(0) +#define CXLDEV_EVENT_STATUS_WARN BIT(1) +#define CXLDEV_EVENT_STATUS_FAIL BIT(2) +#define CXLDEV_EVENT_STATUS_FATAL BIT(3) +/* CXL 2.0 8.2.8.4 Mailbox Registers */ #define CXLDEV_MBOX_CAPS_OFFSET 0x00#define CXLDEV_MBOX_CAP_PAYLOAD_SIZE_MASK GENMASK(4, 0)drivers/cxl/cxlmem.h +1
-0@@ -4,6 +4,7 @@#define __CXL_MEM_H__#include <uapi/linux/cxl_mem.h> #include <linux/cdev.h> +#include <linux/uuid.h> #include "cxl.h" /* CXL 2.0 8.2.8.5.1.1 Memory Device Status Register */
drivers/cxl/cxlmem.h struct cxl_dev_state { · line 256
Quoted 6 lines
drivers/cxl/cxlmem.h +3
-0@@ -250,12 +251,16 @@ struct cxl_dev_state {bool msi_enabled; +struct cxl_get_event_payload *event_buf; +struct mutex event_buf_lock; +
Missing kdoc.
Quoted 17 lines
drivers/cxl/cxlmem.h +4
-0int (*mbox_send)(struct cxl_dev_state *cxlds,struct cxl_mbox_cmd *cmd); };enum cxl_opcode {CXL_MBOX_OP_INVALID =0x0000 ,CXL_MBOX_OP_RAW =CXL_MBOX_OP_INVALID , +CXL_MBOX_OP_GET_EVENT_RECORD =0x0100 ,CXL_MBOX_OP_GET_FW_INFO =0x0200 ,CXL_MBOX_OP_ACTIVATE_FW =0x0202 ,CXL_MBOX_OP_GET_SUPPORTED_LOGS =0x0400 ,@@ -325,6 +330,72 @@ struct cxl_mbox_identify {u8 qos_telemetry_caps; } __packed; +/* +* Common Event Record Format +* CXL rev 3.0 section 8.2.9.2.1; Table 8-42
drivers/cxl/cxlmem.h struct cxl_mbox_identify { · line 345
Quoted 10 lines
drivers/cxl/cxlmem.h +10
-0+*/ +struct cxl_event_record_hdr { +uuid_t id; +u8 length; +u8 flags[3 ]; +__le16 handle; +__le16 related_handle; +__le64 timestamp; +u8 maint_op_class; +u8 reserved[0xf ];
Nit, but lets not copy the spec here and just make all the field sizes decimal. It even saves a character to write 15 instead of 0xf, and @flags is also decimal.
Quoted 12 lines
drivers/cxl/cxlmem.h +12
-0+} __packed; + +#define CXL_EVENT_RECORD_DATA_LENGTH 0x50 +struct cxl_event_record_raw { +struct cxl_event_record_hdr hdr; +u8 data[CXL_EVENT_RECORD_DATA_LENGTH ]; +} __packed; + +/* +* Get Event Records output payload +* CXL rev 3.0 section 8.2.9.2.2; Table 8-50 +*/
drivers/cxl/cxlmem.h struct cxl_mbox_identify { · line 367
Quoted 10 lines
drivers/cxl/cxlmem.h +10
-0+#define CXL_GET_EVENT_FLAG_OVERFLOW BIT(0) +#define CXL_GET_EVENT_FLAG_MORE_RECORDS BIT(1) +struct cxl_get_event_payload { +u8 flags; +u8 reserved1; +__le16 overflow_err_count; +__le64 first_overflow_timestamp; +__le64 last_overflow_timestamp; +__le16 record_count; +u8 reserved2[0xa ];
Same nit.
Quoted 179 lines
drivers/cxl/cxlmem.h +32
-0+struct cxl_event_record_raw records[]; +} __packed; + +/* +* CXL rev 3.0 section 8.2.9.2.2; Table 8-49 +*/ +enum cxl_event_log_type { +CXL_EVENT_TYPE_INFO =0x00 , +CXL_EVENT_TYPE_WARN , +CXL_EVENT_TYPE_FAIL , +CXL_EVENT_TYPE_FATAL , +CXL_EVENT_TYPE_MAX +}; + +static inline const char *cxl_event_log_type_str (enum cxl_event_log_type type) +{ +switch (type) { +case CXL_EVENT_TYPE_INFO : +return "Informational" ; +case CXL_EVENT_TYPE_WARN : +return "Warning" ; +case CXL_EVENT_TYPE_FAIL : +return "Failure" ; +case CXL_EVENT_TYPE_FATAL : +return "Fatal" ; +default : +break ; + } +return "<unknown>" ; +} +struct cxl_mbox_get_partition_info {__le64 active_volatile_cap;__le64 active_persistent_cap;@@ -384,6 +455,7 @@ int cxl_mem_create_range_info(struct cxl_dev_state *cxlds);struct cxl_dev_state *cxl_dev_state_create (struct device *dev);void set_exclusive_cxl_commands (struct cxl_dev_state *cxlds,unsigned long *cmds);void clear_exclusive_cxl_commands (struct cxl_dev_state *cxlds,unsigned long *cmds); +void cxl_mem_get_event_records (struct cxl_dev_state *cxlds);#ifdef CONFIG_CXL_SUSPEND void cxl_mem_active_inc (void );void cxl_mem_active_dec (void );include/trace/events/cxl.h new file +126
-0@@ -0,0 +1,126 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#undef TRACE_SYSTEM +#define TRACE_SYSTEM cxl + +#if !defined(_CXL_TRACE_EVENTS_H) || defined(TRACE_HEADER_MULTI_READ ) +#define _CXL_TRACE_EVENTS_H + +#include <asm-generic/unaligned.h> +#include <linux/tracepoint.h> +#include <cxlmem.h> + +TRACE_EVENT (cxl_overflow, + + TP_PROTO(const char *dev_name, enum cxl_event_log_type log, + struct cxl_get_event_payload *payload), + +TP_ARGS (dev_name, log, payload), + +TP_STRUCT__entry ( + __string(dev_name, dev_name) + __field(int, log) + __field(u64, first_ts) + __field(u64, last_ts) +__field (u16, count) + ), + +TP_fast_assign ( + __assign_str(dev_name, dev_name); + __entry->log = log; + __entry->count = le16_to_cpu(payload->overflow_err_count); + __entry->first_ts = le64_to_cpu(payload->first_overflow_timestamp); + __entry->last_ts = le64_to_cpu(payload->last_overflow_timestamp); + ), + +TP_printk ("%s: EVENT LOG OVERFLOW log=%s : %u records from %llu to %llu" , +__get_str (dev_name),cxl_event_log_type_str (__entry->log), + __entry->count, __entry->first_ts, __entry->last_ts) + +); + +/* +* Common Event Record Format +* CXL 3.0 section 8.2.9.2.1; Table 8-42 +*/ +#define CXL_EVENT_RECORD_FLAG_PERMANENT BIT(2) +#define CXL_EVENT_RECORD_FLAG_MAINT_NEEDED BIT(3) +#define CXL_EVENT_RECORD_FLAG_PERF_DEGRADED BIT(4) +#define CXL_EVENT_RECORD_FLAG_HW_REPLACE BIT(5) +#define show_hdr_flags (flags) __print_flags(flags, " | ", \ + { CXL_EVENT_RECORD_FLAG_PERMANENT, "PERMANENT_CONDITION" }, \ + { CXL_EVENT_RECORD_FLAG_MAINT_NEEDED, "MAINTENANCE_NEEDED" }, \ + { CXL_EVENT_RECORD_FLAG_PERF_DEGRADED, "PERFORMANCE_DEGRADED" }, \ + { CXL_EVENT_RECORD_FLAG_HW_REPLACE, "HARDWARE_REPLACEMENT_NEEDED" } \ +) + +/* +* Define macros for the common header of each CXL event. +* +* Tracepoints using these macros must do 3 things: +* +* 1) Add CXL_EVT_TP_entry to TP_STRUCT__entry +* 2) Use CXL_EVT_TP_fast_assign within TP_fast_assign; +* pass the dev_name, log, and CXL event header +* 3) Use CXL_EVT_TP_printk() instead of TP_printk() +* +* See the generic_event tracepoint as an example. +*/ +#define CXL_EVT_TP_entry \ + __string(dev_name, dev_name) \ + __field(int, log) \ + __field_struct(uuid_t, hdr_uuid) \ + __field(u32, hdr_flags) \ + __field(u16, hdr_handle) \ + __field(u16, hdr_related_handle) \ + __field(u64, hdr_timestamp) \ + __field(u8, hdr_length) \ + __field(u8, hdr_maint_op_class) + +#define CXL_EVT_TP_fast_assign (dname, l, hdr) \ + __assign_str(dev_name, (dname)); \ + __entry->log = (l); \ + memcpy(&__entry->hdr_uuid, &(hdr).id, sizeof(uuid_t)); \ + __entry->hdr_length = (hdr).length; \ + __entry->hdr_flags = get_unaligned_le24((hdr).flags); \ + __entry->hdr_handle = le16_to_cpu((hdr).handle); \ + __entry->hdr_related_handle = le16_to_cpu((hdr).related_handle); \ + __entry->hdr_timestamp = le64_to_cpu((hdr).timestamp); \ + __entry->hdr_maint_op_class = (hdr).maint_op_class + +#define CXL_EVT_TP_printk (fmt, ...) \ + TP_printk("%s log=%s : time=%llu uuid=%pUb len=%d flags='%s' " \ + "handle=%x related_handle=%x maint_op_class=%u" \ + " : " fmt, \ + __get_str(dev_name), cxl_event_log_type_str(__entry->log), \ + __entry->hdr_timestamp, &__entry->hdr_uuid, __entry->hdr_length,\ + show_hdr_flags(__entry->hdr_flags), __entry->hdr_handle, \ + __entry->hdr_related_handle, __entry->hdr_maint_op_class, \ + ##__VA_ARGS__) + +TRACE_EVENT (cxl_generic_event, + + TP_PROTO(const char *dev_name, enum cxl_event_log_type log, + struct cxl_event_record_raw *rec), + +TP_ARGS (dev_name, log, rec), + +TP_STRUCT__entry ( + CXL_EVT_TP_entry +__array (u8, data,CXL_EVENT_RECORD_DATA_LENGTH ) + ), + +TP_fast_assign ( + CXL_EVT_TP_fast_assign(dev_name, log, rec->hdr); + memcpy(__entry->data, &rec->data, CXL_EVENT_RECORD_DATA_LENGTH); + ), + +CXL_EVT_TP_printk ("%s" , +__print_hex (__entry->data,CXL_EVENT_RECORD_DATA_LENGTH )) +); + +#endif /* _CXL_TRACE_EVENTS_H */ + +/* This part must be outside protection */ +#undef TRACE_INCLUDE_FILE +#define TRACE_INCLUDE_FILE cxl +#include <trace/define_trace.h> include/uapi/linux/cxl_mem.h +0
-0
include/uapi/linux/cxl_mem.h line 30
Quoted 7 lines
include/uapi/linux/cxl_mem.h +1
-0@@ -24,6 +24,7 @@___C (IDENTIFY ,"Identify Command" ), \___C (RAW ,"Raw device command" ), \___C (GET_SUPPORTED_LOGS ,"Get Supported Logs" ), \ +___C (GET_EVENT_RECORD ,"Get Event Record" ), \___C (GET_FW_INFO ,"Get FW Info" ), \___C (GET_PARTITION_INFO ,"Get Partition Information" ), \ ___C(GET_LSA, "Get Label Storage Area"), \
Yikes, no, this is an enum. New commands need to come at the end otherwise different kernels will disagree about the command numbering. Likely needs a comment here alerting future devs about the ABI breakage danger here.
Re: [PATCH V2 02/11] cxl/mem: Implement Get Event Records command
Quoted 3 lines · Dan Williams
Quoted 2 lines · ira.weiny@ wrote:
From: Ira Weiny <[email protected]>
[snip]
Quoted 18 lines
Quoted 8 lines
drivers/cxl/core/mbox.c +3
-0+#define CREATE_TRACE_POINTS +#include <trace/events/cxl.h> +#include "core.h" static bool cxl_raw_allow_all;drivers/cxl/core/mbox.c static struct cxl_mem_command cxl_mem_commands[CXL_MEM_COMMAND_ID_MAX] = { · line 57
Quoted 7 lines
drivers/cxl/core/mbox.c +1
-0@@ -48,6 +51,7 @@ static struct cxl_mem_command cxl_mem_commands[CXL_MEM_COMMAND_ID_MAX] = {CXL_CMD (RAW ,CXL_VARIABLE_PAYLOAD ,CXL_VARIABLE_PAYLOAD ,0 ), #endifCXL_CMD (GET_SUPPORTED_LOGS ,0 ,CXL_VARIABLE_PAYLOAD ,CXL_CMD_FLAG_FORCE_ENABLE ), +CXL_CMD (GET_EVENT_RECORD ,1 ,CXL_VARIABLE_PAYLOAD ,0 ),CXL_CMD (GET_FW_INFO ,0 ,0x50 ,0 ),CXL_CMD (GET_PARTITION_INFO ,0 ,0x20 ,0 ), CXL_CMD(GET_LSA, 0x8, CXL_VARIABLE_PAYLOAD, 0),Similar to this patch:
Quoted 4 lines
CXL_MEM_COMMAND_ID_GET_EVENT_RECORD, should be added to the "always kernel" /cxlds->exclusive_cmdsmask.
Done for all the commands. I'll rebase as well before sending this out.
Quoted 27 lines
Quoted 27 lines
drivers/cxl/core/mbox.c +22
-0@@ -704,6 +708,106 @@ int cxl_enumerate_cmds(struct cxl_dev_state *cxlds) }EXPORT_SYMBOL_NS_GPL (cxl_enumerate_cmds,CXL ); +static void cxl_mem_get_records_log (struct cxl_dev_state *cxlds, +enum cxl_event_log_type type) +{ +struct cxl_get_event_payload *payload; +u16 nr_rec; + +mutex_lock (&cxlds->event_buf_lock); + + payload = cxlds->event_buf; + + do { + u8 log_type = type; + int rc; + + rc = cxl_mbox_send_cmd(cxlds, CXL_MBOX_OP_GET_EVENT_RECORD, + &log_type, sizeof(log_type), + payload, cxlds->payload_size); + if (rc) { + dev_err(cxlds->dev, "Event log '%s': Failed to query event records : %d", + cxl_event_log_type_str(type), rc); + goto unlock_buffer; + }
Quoted 8 lines
drivers/cxl/core/mbox.c int cxl_enumerate_cmds(struct cxl_dev_state *cxlds) · line 735
Quoted 3 lines
drivers/cxl/core/mbox.c +3
-0+ + nr_rec =le16_to_cpu (payload->record_count); +if (trace_cxl_generic_event_enabled ()) {This feels like a premature micro-optimization as none of this code is fast path and it is dwarfed by the cost of executing the mailbox command. I started with trying to reduce the 80 column collision pressure, but then stepped back even further and asked, why?
Because Steven told me to. :-( I should have been smarter than that.
Quoted 15 lines
drivers/cxl/core/mbox.c int cxl_enumerate_cmds(struct cxl_dev_state *cxlds) · line 741
Quoted 7 lines
drivers/cxl/core/mbox.c +6
-0+int i; + +for (i =0 ; i < nr_rec; i++) +trace_cxl_generic_event (dev_name (cxlds->dev), + type, + &payload->records[i]);As far as I can tell
trace_cxl_generic_event()always expects a device-name as its first argument. So why not enforce that with type-safety? I.e. I thinktrace_cxl_generic_event()should take a "struct device *", not a string unless it is really the case that any old string will do as the first argument to the trace event. Otherwise the trace point can do "__string(dev_name, dev_name(dev))", and mandate that callers pass devices.
From a trace point view 'any old string' will do. There was nothing else the trace needed from struct device so I skipped it.
[snip]
Quoted 16 lines
Quoted 16 lines
drivers/cxl/core/mbox.c +16
-0+ +/** +* cxl_mem_get_event_records - Get Event Records from the device +* @cxlds: The device data for the operation +* +* Retrieve all event records available on the device and report them as trace +* events. +* +* See CXL rev 3.0 @8.2.9.2.2 Get Event Records +*/ +void cxl_mem_get_event_records (struct cxl_dev_state *cxlds) +{ +u32 status =readl (cxlds->regs.status +CXLDEV_DEV_EVENT_STATUS_OFFSET ); + +dev_dbg (cxlds->dev,"Reading event logs: %x\n" , status); +
Quoted 8 lines
drivers/cxl/core/mbox.c int cxl_enumerate_cmds(struct cxl_dev_state *cxlds) · line 798
Quoted 5 lines
drivers/cxl/core/mbox.c +5
-0+if (!cxlds->event_buf) { + cxlds->event_buf =alloc_event_buf (cxlds); +if (WARN_ON_ONCE (!cxlds->event_buf)) +return ; + }What's the point of having an
event_buf_lockifevent_bufis reallocated every call?
This is only called on start up.
Quoted 4 lines
Just allocate
event_bufonce at the beginning of time during init if the device supports event log retrieval, and fail the driver load if that allocation fails. No runtimeWARN()for memory allocation.
It was. I'll make that more clear in the next series.
Quoted 4 lines
I notice this patch does not clear events, I trust that comes later in the series, but I think it belongs here to make this patch a complete standalone thought.
Squashed. But it does make for a large patch. Which I'm not a fan of for review. Lucky that now we have a lot of review on the parts.
Quoted 17 lines
drivers/cxl/core/mbox.c int cxl_enumerate_cmds(struct cxl_dev_state *cxlds) · line 807
Quoted 9 lines
drivers/cxl/core/mbox.c +8
-0+if (status &CXLDEV_EVENT_STATUS_INFO ) +cxl_mem_get_records_log (cxlds,CXL_EVENT_TYPE_INFO ); +if (status &CXLDEV_EVENT_STATUS_WARN ) +cxl_mem_get_records_log (cxlds,CXL_EVENT_TYPE_WARN ); +if (status &CXLDEV_EVENT_STATUS_FAIL ) +cxl_mem_get_records_log (cxlds,CXL_EVENT_TYPE_FAIL ); +if (status &CXLDEV_EVENT_STATUS_FATAL ) +cxl_mem_get_records_log (cxlds,CXL_EVENT_TYPE_FATAL );This retrieval order should be flipped. If there is a FATAL pending I expect a monitor wants that first and would be happy to parse the INFO later. I would go so far as to say that if the INFO logger is looping and a FATAL comes in the driver should get that out first before going back for more INFO logs. That would mean executing Clear Events and looping through the logs by priority until all the status bits fall silent inside
cxl_mem_get_records_log().
I'll flip them. And determine if this is really what we want to do for the irq.
The issue with the irq handling calling a single function which checks all status is that we may end up with some odd interrupts doing nothing depending on racing etc.
A buffer per log would eliminate this to some extent if the message numbers are not shared. I don't doubt that vendors are unlikely to burn more than 1 message number so the irq may indeed be shared and serialized anyway.
For simplicity I'll throw them all together.
Quoted 47 lines
Quoted 47 lines
drivers/cxl/core/mbox.c +4
-0+} +EXPORT_SYMBOL_NS_GPL (cxl_mem_get_event_records,CXL ); +/** * cxl_mem_get_partition_info - Get partition info * @cxlds: The device data for the operation @@ -846,6 +950,7 @@ struct cxl_dev_state *cxl_dev_state_create(struct device *dev) }mutex_init (&cxlds->mbox_mutex); +mutex_init (&cxlds->event_buf_lock); cxlds->dev = dev;return cxlds;drivers/cxl/cxl.h +7
-0@@ -132,6 +132,13 @@ static inline int ways_to_cxl(unsigned int ways, u8 *iw)#define CXLDEV_CAP_CAP_ID_SECONDARY_MAILBOX 0x3#define CXLDEV_CAP_CAP_ID_MEMDEV 0x4000 +/* CXL 3.0 8.2.8.3.1 Event Status Register */ +#define CXLDEV_DEV_EVENT_STATUS_OFFSET 0x00 +#define CXLDEV_EVENT_STATUS_INFO BIT(0) +#define CXLDEV_EVENT_STATUS_WARN BIT(1) +#define CXLDEV_EVENT_STATUS_FAIL BIT(2) +#define CXLDEV_EVENT_STATUS_FATAL BIT(3) +/* CXL 2.0 8.2.8.4 Mailbox Registers */ #define CXLDEV_MBOX_CAPS_OFFSET 0x00#define CXLDEV_MBOX_CAP_PAYLOAD_SIZE_MASK GENMASK(4, 0)drivers/cxl/cxlmem.h +1
-0@@ -4,6 +4,7 @@#define __CXL_MEM_H__#include <uapi/linux/cxl_mem.h> #include <linux/cdev.h> +#include <linux/uuid.h> #include "cxl.h" /* CXL 2.0 8.2.8.5.1.1 Memory Device Status Register */ @@ -250,12 +251,16 @@ struct cxl_dev_state {
Quoted 7 lines
drivers/cxl/cxlmem.h struct cxl_dev_state { · line 256
Quoted 5 lines
drivers/cxl/cxlmem.h +3
-0bool msi_enabled; +struct cxl_get_event_payload *event_buf; +struct mutex event_buf_lock; +Missing kdoc.
Got it from Jonathan.
Quoted 24 lines
Quoted 24 lines
drivers/cxl/cxlmem.h +10
-0int (*mbox_send)(struct cxl_dev_state *cxlds,struct cxl_mbox_cmd *cmd); };enum cxl_opcode {CXL_MBOX_OP_INVALID =0x0000 ,CXL_MBOX_OP_RAW =CXL_MBOX_OP_INVALID , +CXL_MBOX_OP_GET_EVENT_RECORD =0x0100 ,CXL_MBOX_OP_GET_FW_INFO =0x0200 ,CXL_MBOX_OP_ACTIVATE_FW =0x0202 ,CXL_MBOX_OP_GET_SUPPORTED_LOGS =0x0400 ,@@ -325,6 +330,72 @@ struct cxl_mbox_identify {u8 qos_telemetry_caps; } __packed; +/* +* Common Event Record Format +* CXL rev 3.0 section 8.2.9.2.1; Table 8-42 +*/ +struct cxl_event_record_hdr { +uuid_t id; +u8 length; +u8 flags[3 ]; +__le16 handle;
Quoted 8 lines
drivers/cxl/cxlmem.h struct cxl_mbox_identify { · line 345
Quoted 4 lines
drivers/cxl/cxlmem.h +4
-0+__le16 related_handle; +__le64 timestamp; +u8 maint_op_class; +u8 reserved[0xf ];Nit, but lets not copy the spec here and just make all the field sizes decimal. It even saves a character to write 15 instead of
0xf, and @flags is also decimal.
Ok.
Quoted 17 lines
Quoted 17 lines
drivers/cxl/cxlmem.h +16
-0+} __packed; + +#define CXL_EVENT_RECORD_DATA_LENGTH 0x50 +struct cxl_event_record_raw { +struct cxl_event_record_hdr hdr; +u8 data[CXL_EVENT_RECORD_DATA_LENGTH ]; +} __packed; + +/* +* Get Event Records output payload +* CXL rev 3.0 section 8.2.9.2.2; Table 8-50 +*/ +#define CXL_GET_EVENT_FLAG_OVERFLOW BIT(0) +#define CXL_GET_EVENT_FLAG_MORE_RECORDS BIT(1) +struct cxl_get_event_payload { +u8 flags;
Quoted 8 lines
drivers/cxl/cxlmem.h struct cxl_mbox_identify { · line 367
Quoted 6 lines
drivers/cxl/cxlmem.h +6
-0+u8 reserved1; +__le16 overflow_err_count; +__le64 first_overflow_timestamp; +__le64 last_overflow_timestamp; +__le16 record_count; +u8 reserved2[0xa ];Same nit.
Done.
[snip]
Quoted 12 lines
Quoted 12 lines
drivers/cxl/cxlmem.h +5
-0+ +/* This part must be outside protection */ +#undef TRACE_INCLUDE_FILE +#define TRACE_INCLUDE_FILE cxl +#include <trace/define_trace.h> include/uapi/linux/cxl_mem.h +0
-0@@ -24,6 +24,7 @@___C (IDENTIFY ,"Identify Command" ), \ ___C(RAW, "Raw device command"), \
Quoted 8 lines
include/uapi/linux/cxl_mem.h line 30
Quoted 5 lines
include/uapi/linux/cxl_mem.h +1
-0___C (GET_SUPPORTED_LOGS ,"Get Supported Logs" ), \ +___C (GET_EVENT_RECORD ,"Get Event Record" ), \___C (GET_FW_INFO ,"Get FW Info" ), \___C (GET_PARTITION_INFO ,"Get Partition Information" ), \ ___C(GET_LSA, "Get Label Storage Area"), \Yikes, no, this is an enum. New commands need to come at the end otherwise different kernels will disagree about the command numbering.
Ooops sorry about that. I somehow thought these were tied to the command values.
Thanks, Changed for all the commands,
Quoted 2 lines
Likely needs a comment here alerting future devs about the ABI breakage danger here.
Additional patch added.
Ira
Re: [PATCH V2 02/11] cxl/mem: Implement Get Event Records command
Quoted 68 lines · Ira Weiny
Quoted 3 lines · Dan Williams
Quoted 2 lines · ira.weiny@ wrote:
From: Ira Weiny <[email protected]>
[snip]
Quoted 18 lines
Quoted 8 lines
drivers/cxl/core/mbox.c +3
-0+#define CREATE_TRACE_POINTS +#include <trace/events/cxl.h> +#include "core.h" static bool cxl_raw_allow_all;drivers/cxl/core/mbox.c static struct cxl_mem_command cxl_mem_commands[CXL_MEM_COMMAND_ID_MAX] = { · line 57
Quoted 7 lines
drivers/cxl/core/mbox.c +1
-0@@ -48,6 +51,7 @@ static struct cxl_mem_command cxl_mem_commands[CXL_MEM_COMMAND_ID_MAX] = {CXL_CMD (RAW ,CXL_VARIABLE_PAYLOAD ,CXL_VARIABLE_PAYLOAD ,0 ), #endifCXL_CMD (GET_SUPPORTED_LOGS ,0 ,CXL_VARIABLE_PAYLOAD ,CXL_CMD_FLAG_FORCE_ENABLE ), +CXL_CMD (GET_EVENT_RECORD ,1 ,CXL_VARIABLE_PAYLOAD ,0 ),CXL_CMD (GET_FW_INFO ,0 ,0x50 ,0 ),CXL_CMD (GET_PARTITION_INFO ,0 ,0x20 ,0 ), CXL_CMD(GET_LSA, 0x8, CXL_VARIABLE_PAYLOAD, 0),Similar to this patch:
Quoted 4 lines
CXL_MEM_COMMAND_ID_GET_EVENT_RECORD, should be added to the "always kernel" /cxlds->exclusive_cmdsmask.Done for all the commands. I'll rebase as well before sending this out.
Quoted 27 lines
Quoted 27 lines
drivers/cxl/core/mbox.c +22
-0@@ -704,6 +708,106 @@ int cxl_enumerate_cmds(struct cxl_dev_state *cxlds) }EXPORT_SYMBOL_NS_GPL (cxl_enumerate_cmds,CXL ); +static void cxl_mem_get_records_log (struct cxl_dev_state *cxlds, +enum cxl_event_log_type type) +{ +struct cxl_get_event_payload *payload; +u16 nr_rec; + +mutex_lock (&cxlds->event_buf_lock); + + payload = cxlds->event_buf; + + do { + u8 log_type = type; + int rc; + + rc = cxl_mbox_send_cmd(cxlds, CXL_MBOX_OP_GET_EVENT_RECORD, + &log_type, sizeof(log_type), + payload, cxlds->payload_size); + if (rc) { + dev_err(cxlds->dev, "Event log '%s': Failed to query event records : %d", + cxl_event_log_type_str(type), rc); + goto unlock_buffer; + }Quoted 8 lines
drivers/cxl/core/mbox.c int cxl_enumerate_cmds(struct cxl_dev_state *cxlds) · line 735
Quoted 3 lines
drivers/cxl/core/mbox.c +3
-0+ + nr_rec =le16_to_cpu (payload->record_count); +if (trace_cxl_generic_event_enabled ()) {This feels like a premature micro-optimization as none of this code is fast path and it is dwarfed by the cost of executing the mailbox command. I started with trying to reduce the 80 column collision pressure, but then stepped back even further and asked, why?
Quoted 1 line
Because Steven told me to. :-( I should have been smarter than that.
You did the right thing, I failed to jump in sooner on this set.
Quoted 17 lines
Quoted 16 lines
drivers/cxl/core/mbox.c int cxl_enumerate_cmds(struct cxl_dev_state *cxlds) · line 741
Quoted 8 lines
drivers/cxl/core/mbox.c +6
-0+int i; + +for (i =0 ; i < nr_rec; i++) +trace_cxl_generic_event (dev_name (cxlds->dev), + type, + &payload->records[i]);As far as I can tell
trace_cxl_generic_event()always expects a device-name as its first argument. So why not enforce that with type-safety? I.e. I thinktrace_cxl_generic_event()should take a "struct device *", not a string unless it is really the case that any old string will do as the first argument to the trace event. Otherwise the trace point can do "__string(dev_name, dev_name(dev))", and mandate that callers pass devices.
Quoted 2 lines
From a trace point view 'any old string' will do. There was nothing else the trace needed from struct device so I skipped it.
I'd prefer more fine-grained type safety wherever possible.
Quoted 28 lines
[snip]
Quoted 16 lines
Quoted 16 lines
drivers/cxl/core/mbox.c +16
-0+ +/** +* cxl_mem_get_event_records - Get Event Records from the device +* @cxlds: The device data for the operation +* +* Retrieve all event records available on the device and report them as trace +* events. +* +* See CXL rev 3.0 @8.2.9.2.2 Get Event Records +*/ +void cxl_mem_get_event_records (struct cxl_dev_state *cxlds) +{ +u32 status =readl (cxlds->regs.status +CXLDEV_DEV_EVENT_STATUS_OFFSET ); + +dev_dbg (cxlds->dev,"Reading event logs: %x\n" , status); +Quoted 8 lines
drivers/cxl/core/mbox.c int cxl_enumerate_cmds(struct cxl_dev_state *cxlds) · line 798
Quoted 5 lines
drivers/cxl/core/mbox.c +5
-0+if (!cxlds->event_buf) { + cxlds->event_buf =alloc_event_buf (cxlds); +if (WARN_ON_ONCE (!cxlds->event_buf)) +return ; + }What's the point of having an
event_buf_lockifevent_bufis reallocated every call?
Quoted 1 line
This is only called on start up.
cxl_mem_get_event_records() is called all the time. The place to allocate buffers attached to 'struct cxl_dev_state' at start up is cxl_dev_state_create(), or sometime after cxl_enumerate_cmds() if you want to wait and see if the device supports events and CXL _OSC says the driver can drive events.
Quoted 32 lines
Quoted 3 lines
Just allocate
event_bufonce at the beginning of time during init if the device supports event log retrieval, and fail the driver load if that allocation fails. No runtimeWARN()for memory allocation.It was. I'll make that more clear in the next series.
Quoted 4 lines
I notice this patch does not clear events, I trust that comes later in the series, but I think it belongs here to make this patch a complete standalone thought.
Squashed. But it does make for a large patch. Which I'm not a fan of for review. Lucky that now we have a lot of review on the parts.
Quoted 17 lines
drivers/cxl/core/mbox.c int cxl_enumerate_cmds(struct cxl_dev_state *cxlds) · line 807
Quoted 9 lines
drivers/cxl/core/mbox.c +8
-0+if (status &CXLDEV_EVENT_STATUS_INFO ) +cxl_mem_get_records_log (cxlds,CXL_EVENT_TYPE_INFO ); +if (status &CXLDEV_EVENT_STATUS_WARN ) +cxl_mem_get_records_log (cxlds,CXL_EVENT_TYPE_WARN ); +if (status &CXLDEV_EVENT_STATUS_FAIL ) +cxl_mem_get_records_log (cxlds,CXL_EVENT_TYPE_FAIL ); +if (status &CXLDEV_EVENT_STATUS_FATAL ) +cxl_mem_get_records_log (cxlds,CXL_EVENT_TYPE_FATAL );This retrieval order should be flipped. If there is a FATAL pending I expect a monitor wants that first and would be happy to parse the INFO later. I would go so far as to say that if the INFO logger is looping and a FATAL comes in the driver should get that out first before going back for more INFO logs. That would mean executing Clear Events and looping through the logs by priority until all the status bits fall silent inside
cxl_mem_get_records_log().
Quoted 6 lines
I'll flip them. And determine if this is really what we want to do for the irq.
The issue with the irq handling calling a single function which checks all status is that we may end up with some odd interrupts doing nothing depending on racing etc.
If an event handler wakes and reads 0-status bits because another handler did it then return IRQ_HANDLED. You'll have this problem whether you have a central function or not, because there's only one status register for multiple sources.
PATCH v2 3/11 cxl/mem: Implement Clear Event Records command
In reply to [email protected]From: Ira Weiny <[email protected]>
CXL rev 3.0 section 8.2.9.2.3 defines the Clear Event Records mailbox command. After an event record is read it needs to be cleared from the event log.
Implement cxl_clear_event_record() to clear all record retrieved from the device.
Each record is cleared explicitly. A clear all bit is specified but events could arrive between a get and any final clear all operation. This means events would be missed.
Therefore each event is cleared specifically.
---
Changes from V1:
Clear Event Record allows for u8 handles while Get Event Record allows for u16 records to be returned. Based on Jonathan's feedback; allow for all event records to be handled in this clear. Which means a double loop with potentially multiple Clear Event payloads being sent to clear all events sent.
Changes from RFC:
Jonathan
Clean up init of payload and use return code. Also report any error to clear the event. s/v3.0/rev 3.0
--- drivers/cxl/core/mbox.c | 61 +++++++++++++++++++++++++++++++----- drivers/cxl/cxlmem.h | 14 +++++++++ include/uapi/linux/cxl_mem.h | 1 + 3 files changed, 69 insertions(+), 7 deletions(-)
drivers/cxl/core/mbox.c +54 -7
@@ -52,6 +52,7 @@ static struct cxl_mem_command cxl_mem_commands[CXL_MEM_COMMAND_ID_MAX] = {#endif CXL_CMD (GET_SUPPORTED_LOGS ,0 ,CXL_VARIABLE_PAYLOAD ,CXL_CMD_FLAG_FORCE_ENABLE ),CXL_CMD (GET_EVENT_RECORD ,1 ,CXL_VARIABLE_PAYLOAD ,0 ), +CXL_CMD (CLEAR_EVENT_RECORD ,CXL_VARIABLE_PAYLOAD ,0 ,0 ),CXL_CMD (GET_FW_INFO ,0 ,0x50 ,0 ),CXL_CMD (GET_PARTITION_INFO ,0 ,0x20 ,0 ), CXL_CMD(GET_LSA, 0x8, CXL_VARIABLE_PAYLOAD, 0),
@@ -708,6 +709,42 @@ int cxl_enumerate_cmds(struct cxl_dev_state *cxlds) }EXPORT_SYMBOL_NS_GPL (cxl_enumerate_cmds,CXL ); +static int cxl_clear_event_record (struct cxl_dev_state *cxlds, +enum cxl_event_log_type log, +struct cxl_get_event_payload *get_pl, +u16 total) +{ +struct cxl_mbox_clear_event_payload payload = { + .event_log = log, + }; +int cnt; + +/* +* Clear Event Records uses u8 for the handle cnt while Get Event +* Record can return up to 0xffff records. +*/ +for (cnt =0 ; cnt < total;/* cnt incremented internally */ ) { +u8 nr_recs =min_t (u8, (total - cnt), +CXL_CLEAR_EVENT_MAX_HANDLES ); +int i, rc; + +for (i =0 ; i < nr_recs; i++, cnt++) { + payload.handle[i] = get_pl->records[cnt].hdr.handle; +dev_dbg (cxlds->dev,"Event log '%s': Clearning %u\n" , +cxl_event_log_type_str (log), +le16_to_cpu (payload.handle[i])); + } + payload.nr_recs = nr_recs; + + rc =cxl_mbox_send_cmd (cxlds,CXL_MBOX_OP_CLEAR_EVENT_RECORD , + &payload,sizeof (payload),NULL ,0 ); +if (rc) +return rc; + } + +return 0 ; +} +static void cxl_mem_get_records_log (struct cxl_dev_state *cxlds,enum cxl_event_log_type type) {
@@ -732,13 +769,22 @@ static void cxl_mem_get_records_log(struct cxl_dev_state *cxlds, } nr_rec =le16_to_cpu (payload->record_count); -if (trace_cxl_generic_event_enabled ()) { +if (nr_rec >0 ) {int i; -for (i =0 ; i < nr_rec; i++) -trace_cxl_generic_event (dev_name (cxlds->dev), - type, - &payload->records[i]); +if (trace_cxl_generic_event_enabled ()) { +for (i =0 ; i < nr_rec; i++) +trace_cxl_generic_event (dev_name (cxlds->dev), + type, + &payload->records[i]); + } + + rc =cxl_clear_event_record (cxlds, type, payload, nr_rec); +if (rc) { +dev_err (cxlds->dev,"Event log '%s': Failed to clear events : %d" , +cxl_event_log_type_str (type), rc); +return ; + } } if (trace_cxl_overflow_enabled() &&
@@ -780,10 +826,11 @@ static struct cxl_get_event_payload *alloc_event_buf(struct cxl_dev_state *cxlds* cxl_mem_get_event_records - Get Event Records from the device * @cxlds: The device data for the operation * -* Retrieve all event records available on the device and report them as trace -* events. +* Retrieve all event records available on the device, report them as trace +* events, and clear them. * * See CXL rev 3.0 @8.2.9.2.2 Get Event Records +* See CXL rev 3.0 @8.2.9.2.3 Clear Event Records */ void cxl_mem_get_event_records (struct cxl_dev_state *cxlds) {
drivers/cxl/cxlmem.h +14 -0
@@ -261,6 +261,7 @@ enum cxl_opcode {CXL_MBOX_OP_INVALID =0x0000 ,CXL_MBOX_OP_RAW =CXL_MBOX_OP_INVALID ,CXL_MBOX_OP_GET_EVENT_RECORD =0x0100 , +CXL_MBOX_OP_CLEAR_EVENT_RECORD =0x0101 ,CXL_MBOX_OP_GET_FW_INFO =0x0200 ,CXL_MBOX_OP_ACTIVATE_FW =0x0202 , CXL_MBOX_OP_GET_SUPPORTED_LOGS = 0x0400,
@@ -396,6 +397,19 @@ static inline const char *cxl_event_log_type_str(enum cxl_event_log_type type)return "<unknown>" ; } +/* +* Clear Event Records input payload +* CXL rev 3.0 section 8.2.9.2.3; Table 8-51 +*/ +#define CXL_CLEAR_EVENT_MAX_HANDLES (0xff) +struct cxl_mbox_clear_event_payload { +u8 event_log;/* enum cxl_event_log_type */ +u8 clear_flags; +u8 nr_recs; +u8 reserved[3 ]; +__le16 handle[CXL_CLEAR_EVENT_MAX_HANDLES ]; +}; +struct cxl_mbox_get_partition_info {__le64 active_volatile_cap;__le64 active_persistent_cap;
include/uapi/linux/cxl_mem.h +1 -0
@@ -25,6 +25,7 @@___C (RAW ,"Raw device command" ), \___C (GET_SUPPORTED_LOGS ,"Get Supported Logs" ), \___C (GET_EVENT_RECORD ,"Get Event Record" ), \ +___C (CLEAR_EVENT_RECORD ,"Clear Event Record" ), \___C (GET_FW_INFO ,"Get FW Info" ), \___C (GET_PARTITION_INFO ,"Get Partition Information" ), \ ___C(GET_LSA, "Get Label Storage Area"), \
Signature · 1 line
2.37.2
Re: [PATCH V2 03/11] cxl/mem: Implement Clear Event Records command
Quoted 9 lines · On Wed, 30 Nov 2022 16:27:11 -0800 [email protected] wrote:
From: Ira Weiny <[email protected]>
CXL rev 3.0 section 8.2.9.2.3 defines the Clear Event Records mailbox command. After an event record is read it needs to be cleared from the event log.
Implement
cxl_clear_event_record()to clear all record retrieved from the device.
Quoted 6 lines
Each record is cleared explicitly. A clear all bit is specified but events could arrive between a get and any final clear all operation. This means events would be missed.
Therefore each event is cleared specifically.Signed-off-by Ira Weiny <[email protected]>
I think there is a type issue on the min_t() calculation with that addressed this looks good to me.
Quoted 43 lines
---
Changes from V1:
Clear Event Record allows for u8 handles while Get Event Record allows for u16 records to be returned. Based on Jonathan's feedback; allow for all event records to be handled in this clear. Which means a double loop with potentially multiple Clear Event payloads being sent to clear all events sent.Changes from RFC:
Jonathan
Clean up init of payload and use return code. Also report any error to clear the event. s/v3.0/rev 3.0--- drivers/cxl/core/mbox.c | 61 +++++++++++++++++++++++++++++++----- drivers/cxl/cxlmem.h | 14 +++++++++ include/uapi/linux/cxl_mem.h | 1 + 3 files changed, 69 insertions(+), 7 deletions(-)drivers/cxl/core/mbox.c +8
-0@@ -52,6 +52,7 @@ static struct cxl_mem_command cxl_mem_commands[CXL_MEM_COMMAND_ID_MAX] = {#endif CXL_CMD (GET_SUPPORTED_LOGS ,0 ,CXL_VARIABLE_PAYLOAD ,CXL_CMD_FLAG_FORCE_ENABLE ),CXL_CMD (GET_EVENT_RECORD ,1 ,CXL_VARIABLE_PAYLOAD ,0 ), +CXL_CMD (CLEAR_EVENT_RECORD ,CXL_VARIABLE_PAYLOAD ,0 ,0 ),CXL_CMD (GET_FW_INFO ,0 ,0x50 ,0 ),CXL_CMD (GET_PARTITION_INFO ,0 ,0x20 ,0 ), CXL_CMD(GET_LSA, 0x8, CXL_VARIABLE_PAYLOAD, 0),@@ -708,6 +709,42 @@ int cxl_enumerate_cmds(struct cxl_dev_state *cxlds) }EXPORT_SYMBOL_NS_GPL (cxl_enumerate_cmds,CXL ); +static int cxl_clear_event_record (struct cxl_dev_state *cxlds, +enum cxl_event_log_type log, +struct cxl_get_event_payload *get_pl, +u16 total) +{ +struct cxl_mbox_clear_event_payload payload = { + .event_log = log,
drivers/cxl/core/mbox.c int cxl_enumerate_cmds(struct cxl_dev_state *cxlds) · line 728
Quoted 10 lines
drivers/cxl/core/mbox.c +10
-0+ }; +int cnt; + +/* +* Clear Event Records uses u8 for the handle cnt while Get Event +* Record can return up to 0xffff records. +*/ +for (cnt =0 ; cnt < total;/* cnt incremented internally */ ) { +u8 nr_recs =min_t (u8, (total - cnt), +CXL_CLEAR_EVENT_MAX_HANDLES );
I might be half asleep but isn't this assuming that (total - cnt) fits in an u8? Shouldn't this be min_t(u16, ..)
Also, maybe u16 cnt would be simpler.
Hmm. This is safe but only because of how you call it alongside handling of a particular Get event records response (which must have fitted in the mailbox and has a longer header).
Looking at this function in isolation, I think the mailbox could be small enough that we might not fit 255 records + the header. Perhaps we need a comment to say that, or at minimum a check and error return if it won't fit?
drivers/cxl/core/mbox.c static void cxl_mem_get_records_log(struct cxl_dev_state *cxlds,
Quoted 23 lines
drivers/cxl/core/mbox.c +19
-0+int i, rc; + +for (i =0 ; i < nr_recs; i++, cnt++) { + payload.handle[i] = get_pl->records[cnt].hdr.handle; +dev_dbg (cxlds->dev,"Event log '%s': Clearning %u\n" , +cxl_event_log_type_str (log), +le16_to_cpu (payload.handle[i])); + } + payload.nr_recs = nr_recs; + + rc =cxl_mbox_send_cmd (cxlds,CXL_MBOX_OP_CLEAR_EVENT_RECORD , + &payload,sizeof (payload),NULL ,0 ); +if (rc) +return rc; + } + +return 0 ; +} +static void cxl_mem_get_records_log (struct cxl_dev_state *cxlds,enum cxl_event_log_type type) {@@ -732,13 +769,22 @@ static void cxl_mem_get_records_log(struct cxl_dev_state *cxlds,
This feels miss named now but I can't immediately think of better naming so on that basis fine to leave it as is if you don't have a better idea!.
Quoted 25 lines
drivers/cxl/core/mbox.c +14
-5} nr_rec =le16_to_cpu (payload->record_count); -if (trace_cxl_generic_event_enabled ()) { +if (nr_rec >0 ) {int i; -for (i =0 ; i < nr_rec; i++) -trace_cxl_generic_event (dev_name (cxlds->dev), - type, - &payload->records[i]); +if (trace_cxl_generic_event_enabled ()) { +for (i =0 ; i < nr_rec; i++) +trace_cxl_generic_event (dev_name (cxlds->dev), + type, + &payload->records[i]); + } + + rc =cxl_clear_event_record (cxlds, type, payload, nr_rec); +if (rc) { +dev_err (cxlds->dev,"Event log '%s': Failed to clear events : %d" , +cxl_event_log_type_str (type), rc); +return ; + } }
Re: [PATCH V2 03/11] cxl/mem: Implement Clear Event Records command
Quoted 71 lines · Jonathan Cameron
Quoted 9 lines · On Wed, 30 Nov 2022 16:27:11 -0800 [email protected] wrote:
From: Ira Weiny <[email protected]>
CXL rev 3.0 section 8.2.9.2.3 defines the Clear Event Records mailbox command. After an event record is read it needs to be cleared from the event log.
Implement
cxl_clear_event_record()to clear all record retrieved from the device.Quoted 6 lines
Each record is cleared explicitly. A clear all bit is specified but events could arrive between a get and any final clear all operation. This means events would be missed.
Therefore each event is cleared specifically.Signed-off-by Ira Weiny <[email protected]>I think there is a type issue on the
min_t()calculation with that addressed this looks good to me.Reviewed-by Jonathan Cameron <[email protected]>Quoted 48 lines
---
Changes from V1:
Clear Event Record allows for u8 handles while Get Event Record allows for u16 records to be returned. Based on Jonathan's feedback; allow for all event records to be handled in this clear. Which means a double loop with potentially multiple Clear Event payloads being sent to clear all events sent.Changes from RFC:
Jonathan
Clean up init of payload and use return code. Also report any error to clear the event. s/v3.0/rev 3.0--- drivers/cxl/core/mbox.c | 61 +++++++++++++++++++++++++++++++----- drivers/cxl/cxlmem.h | 14 +++++++++ include/uapi/linux/cxl_mem.h | 1 + 3 files changed, 69 insertions(+), 7 deletions(-)drivers/cxl/core/mbox.c +13
-0@@ -52,6 +52,7 @@ static struct cxl_mem_command cxl_mem_commands[CXL_MEM_COMMAND_ID_MAX] = {#endif CXL_CMD (GET_SUPPORTED_LOGS ,0 ,CXL_VARIABLE_PAYLOAD ,CXL_CMD_FLAG_FORCE_ENABLE ),CXL_CMD (GET_EVENT_RECORD ,1 ,CXL_VARIABLE_PAYLOAD ,0 ), +CXL_CMD (CLEAR_EVENT_RECORD ,CXL_VARIABLE_PAYLOAD ,0 ,0 ),CXL_CMD (GET_FW_INFO ,0 ,0x50 ,0 ),CXL_CMD (GET_PARTITION_INFO ,0 ,0x20 ,0 ), CXL_CMD(GET_LSA, 0x8, CXL_VARIABLE_PAYLOAD, 0),@@ -708,6 +709,42 @@ int cxl_enumerate_cmds(struct cxl_dev_state *cxlds) }EXPORT_SYMBOL_NS_GPL (cxl_enumerate_cmds,CXL ); +static int cxl_clear_event_record (struct cxl_dev_state *cxlds, +enum cxl_event_log_type log, +struct cxl_get_event_payload *get_pl, +u16 total) +{ +struct cxl_mbox_clear_event_payload payload = { + .event_log = log, + }; +int cnt; + +/* +* Clear Event Records uses u8 for the handle cnt while Get Event
Quoted 8 lines
drivers/cxl/core/mbox.c int cxl_enumerate_cmds(struct cxl_dev_state *cxlds) · line 728
Quoted 5 lines
drivers/cxl/core/mbox.c +5
-0+* Record can return up to 0xffff records. +*/ +for (cnt =0 ; cnt < total;/* cnt incremented internally */ ) { +u8 nr_recs =min_t (u8, (total - cnt), +CXL_CLEAR_EVENT_MAX_HANDLES );I might be half asleep but isn't this assuming that (total - cnt) fits in an u8? Shouldn't this be
min_t(u16, ..)
This cast will ensure the value is never out of range for nr_recs which needs to be u8 and (total - cnt) will never be negative.
But now you have me double thinking myself.
Quoted 10 lines
Also, maybe u16 cnt would be simpler.
Hmm. This is safe but only because of how you call it alongside handling of a particular Get event records response (which must have fitted in the mailbox and has a longer header).
Looking at this function in isolation, I think the mailbox could be small enough that we might not fit 255 records + the header. Perhaps we need a comment to say that, or at minimum a check and error return if it won't fit?
I did not realize that Payload Size applied to input payloads as well. :-/ There is no check in the send command for that ATM. Looking at the spec I think you are right.
I'll further limit the payload size here too.
And with this I might get rid of the min_t() and just cap based on that value.
Quoted 18 lines
Quoted 18 lines
drivers/cxl/core/mbox.c +17
-0+int i, rc; + +for (i =0 ; i < nr_recs; i++, cnt++) { + payload.handle[i] = get_pl->records[cnt].hdr.handle; +dev_dbg (cxlds->dev,"Event log '%s': Clearning %u\n" , +cxl_event_log_type_str (log), +le16_to_cpu (payload.handle[i])); + } + payload.nr_recs = nr_recs; + + rc =cxl_mbox_send_cmd (cxlds,CXL_MBOX_OP_CLEAR_EVENT_RECORD , + &payload,sizeof (payload),NULL ,0 ); +if (rc) +return rc; + } + +return 0 ;
Quoted 8 lines
drivers/cxl/core/mbox.c static void cxl_mem_get_records_log(struct cxl_dev_state *cxlds,
Quoted 6 lines
drivers/cxl/core/mbox.c +2
-0+} +static void cxl_mem_get_records_log (struct cxl_dev_state *cxlds,enum cxl_event_log_type type) {@@ -732,13 +769,22 @@ static void cxl_mem_get_records_log(struct cxl_dev_state *cxlds,This feels miss named now but I can't immediately think of better naming so on that basis fine to leave it as is if you don't have a better idea!.
So we leave it. Naming is hard! :-D
Thanks for the quick review, V3 coming ASAP.
Ira
RE: [PATCH V2 03/11] cxl/mem: Implement Clear Event Records command
In reply to [email protected]Quoted 6 lines · ira.weiny@ wrote:
From: Ira Weiny <[email protected]>
CXL rev 3.0 section 8.2.9.2.3 defines the Clear Event Records mailbox command. After an event record is read it needs to be cleared from the event log.
Quoted 7 lines
Implement
cxl_clear_event_record()to clear all record retrieved from the device.Each record is cleared explicitly. A clear all bit is specified but events could arrive between a get and any final clear all operation. This means events would be missed.
Therefore each event is cleared specifically.
Note that the spec has a better reason for why Clear All has limited usage:
"Clear All Events is only allowed when the Event Log has overflowed; otherwise, the device shall return Invalid Input."
Will need to wait and see if we need that to keep pace with a device with a high event frequency.
Quoted 45 lines
Signed-off-by Ira Weiny <[email protected]>---
Changes from V1:
Clear Event Record allows for u8 handles while Get Event Record allows for u16 records to be returned. Based on Jonathan's feedback; allow for all event records to be handled in this clear. Which means a double loop with potentially multiple Clear Event payloads being sent to clear all events sent.Changes from RFC:
Jonathan
Clean up init of payload and use return code. Also report any error to clear the event. s/v3.0/rev 3.0--- drivers/cxl/core/mbox.c | 61 +++++++++++++++++++++++++++++++----- drivers/cxl/cxlmem.h | 14 +++++++++ include/uapi/linux/cxl_mem.h | 1 + 3 files changed, 69 insertions(+), 7 deletions(-)drivers/cxl/core/mbox.c +8
-0@@ -52,6 +52,7 @@ static struct cxl_mem_command cxl_mem_commands[CXL_MEM_COMMAND_ID_MAX] = {#endif CXL_CMD (GET_SUPPORTED_LOGS ,0 ,CXL_VARIABLE_PAYLOAD ,CXL_CMD_FLAG_FORCE_ENABLE ),CXL_CMD (GET_EVENT_RECORD ,1 ,CXL_VARIABLE_PAYLOAD ,0 ), +CXL_CMD (CLEAR_EVENT_RECORD ,CXL_VARIABLE_PAYLOAD ,0 ,0 ),CXL_CMD (GET_FW_INFO ,0 ,0x50 ,0 ),CXL_CMD (GET_PARTITION_INFO ,0 ,0x20 ,0 ), CXL_CMD(GET_LSA, 0x8, CXL_VARIABLE_PAYLOAD, 0),@@ -708,6 +709,42 @@ int cxl_enumerate_cmds(struct cxl_dev_state *cxlds) }EXPORT_SYMBOL_NS_GPL (cxl_enumerate_cmds,CXL ); +static int cxl_clear_event_record (struct cxl_dev_state *cxlds, +enum cxl_event_log_type log, +struct cxl_get_event_payload *get_pl, +u16 total) +{ +struct cxl_mbox_clear_event_payload payload = { + .event_log = log,
drivers/cxl/core/mbox.c int cxl_enumerate_cmds(struct cxl_dev_state *cxlds) · line 728
Quoted 10 lines
drivers/cxl/core/mbox.c +10
-0+ }; +int cnt; + +/* +* Clear Event Records uses u8 for the handle cnt while Get Event +* Record can return up to 0xffff records. +*/ +for (cnt =0 ; cnt < total;/* cnt incremented internally */ ) { +u8 nr_recs =min_t (u8, (total - cnt), +CXL_CLEAR_EVENT_MAX_HANDLES );
This seems overly complicated. @total is a duplicate of @get_pl->record_count, and the 2 loops feel like it could be cut down to one.
drivers/cxl/core/mbox.c int cxl_enumerate_cmds(struct cxl_dev_state *cxlds) · line 733
Quoted 5 lines
drivers/cxl/core/mbox.c +5
-0+int i, rc; + +for (i =0 ; i < nr_recs; i++, cnt++) { + payload.handle[i] = get_pl->records[cnt].hdr.handle; + dev_dbg(cxlds->dev, "Event log '%s': Clearning %u\n",
While I do think this operation is a mix of clearing and cleaning event records, I don't think "Clearning" is a word.
Quoted 20 lines
drivers/cxl/core/mbox.c +14
-0+cxl_event_log_type_str (log), + le16_to_cpu(payload.handle[i])); + } + payload.nr_recs = nr_recs; + + rc =cxl_mbox_send_cmd (cxlds,CXL_MBOX_OP_CLEAR_EVENT_RECORD , + &payload,sizeof (payload),NULL ,0 ); +if (rc) +return rc; + } + +return 0 ; +} +static void cxl_mem_get_records_log (struct cxl_dev_state *cxlds,enum cxl_event_log_type type) {@@ -732,13 +769,22 @@ static void cxl_mem_get_records_log(struct cxl_dev_state *cxlds, }
drivers/cxl/core/mbox.c static void cxl_mem_get_records_log(struct cxl_dev_state *cxlds, · line 775
Quoted 10 lines
drivers/cxl/core/mbox.c +2
-5nr_rec =le16_to_cpu (payload->record_count); -if (trace_cxl_generic_event_enabled ()) { +if (nr_rec >0 ) {int i; -for (i =0 ; i < nr_rec; i++) -trace_cxl_generic_event (dev_name (cxlds->dev), - type, - &payload->records[i]); +if (trace_cxl_generic_event_enabled ()) {
Again, trace_cxl_generic_event_enabled() injects some awkward formatting here to micro-optimize looping. Any performance benefit this code might offer is likely offset by the extra human effort to read it.
Quoted 66 lines
drivers/cxl/core/mbox.c +15
-2+for (i =0 ; i < nr_rec; i++) +trace_cxl_generic_event (dev_name (cxlds->dev), + type, + &payload->records[i]); + } + + rc =cxl_clear_event_record (cxlds, type, payload, nr_rec); +if (rc) { +dev_err (cxlds->dev,"Event log '%s': Failed to clear events : %d" , +cxl_event_log_type_str (type), rc); +return ; + } } if (trace_cxl_overflow_enabled() &&@@ -780,10 +826,11 @@ static struct cxl_get_event_payload *alloc_event_buf(struct cxl_dev_state *cxlds* cxl_mem_get_event_records - Get Event Records from the device * @cxlds: The device data for the operation * -* Retrieve all event records available on the device and report them as trace -* events. +* Retrieve all event records available on the device, report them as trace +* events, and clear them. * * See CXL rev 3.0 @8.2.9.2.2 Get Event Records +* See CXL rev 3.0 @8.2.9.2.3 Clear Event Records */ void cxl_mem_get_event_records (struct cxl_dev_state *cxlds) {drivers/cxl/cxlmem.h +14
-0@@ -261,6 +261,7 @@ enum cxl_opcode {CXL_MBOX_OP_INVALID =0x0000 ,CXL_MBOX_OP_RAW =CXL_MBOX_OP_INVALID ,CXL_MBOX_OP_GET_EVENT_RECORD =0x0100 , +CXL_MBOX_OP_CLEAR_EVENT_RECORD =0x0101 ,CXL_MBOX_OP_GET_FW_INFO =0x0200 ,CXL_MBOX_OP_ACTIVATE_FW =0x0202 , CXL_MBOX_OP_GET_SUPPORTED_LOGS = 0x0400,@@ -396,6 +397,19 @@ static inline const char *cxl_event_log_type_str(enum cxl_event_log_type type)return "<unknown>" ; } +/* +* Clear Event Records input payload +* CXL rev 3.0 section 8.2.9.2.3; Table 8-51 +*/ +#define CXL_CLEAR_EVENT_MAX_HANDLES (0xff) +struct cxl_mbox_clear_event_payload { +u8 event_log;/* enum cxl_event_log_type */ +u8 clear_flags; +u8 nr_recs; +u8 reserved[3 ]; +__le16 handle[CXL_CLEAR_EVENT_MAX_HANDLES ]; +}; +struct cxl_mbox_get_partition_info {__le64 active_volatile_cap;__le64 active_persistent_cap;include/uapi/linux/cxl_mem.h +0
-0
include/uapi/linux/cxl_mem.h line 31
Quoted 7 lines
include/uapi/linux/cxl_mem.h +1
-0@@ -25,6 +25,7 @@___C (RAW ,"Raw device command" ), \___C (GET_SUPPORTED_LOGS ,"Get Supported Logs" ), \___C (GET_EVENT_RECORD ,"Get Event Record" ), \ +___C (CLEAR_EVENT_RECORD ,"Clear Event Record" ), \___C (GET_FW_INFO ,"Get FW Info" ), \___C (GET_PARTITION_INFO ,"Get Partition Information" ), \ ___C(GET_LSA, "Get Label Storage Area"), \
Same, "yikes" / "must be at the end of the enum" feedback.
Re: [PATCH V2 03/11] cxl/mem: Implement Clear Event Records command
Quoted 14 lines
Quoted 14 lines
drivers/cxl/core/mbox.c +13
-0+static int cxl_clear_event_record (struct cxl_dev_state *cxlds, +enum cxl_event_log_type log, +struct cxl_get_event_payload *get_pl, +u16 total) +{ +struct cxl_mbox_clear_event_payload payload = { + .event_log = log, + }; +int cnt; + +/* +* Clear Event Records uses u8 for the handle cnt while Get Event +* Record can return up to 0xffff records.
Quoted 8 lines
drivers/cxl/core/mbox.c int cxl_enumerate_cmds(struct cxl_dev_state *cxlds) · line 728
Quoted 4 lines
drivers/cxl/core/mbox.c +4
-0+*/ +for (cnt =0 ; cnt < total;/* cnt incremented internally */ ) { +u8 nr_recs =min_t (u8, (total - cnt), +CXL_CLEAR_EVENT_MAX_HANDLES );This seems overly complicated. @total is a duplicate of @get_pl->record_count, and the 2 loops feel like it could be cut down to one.
You could do something nasty like
for (i = 0; i < total; i++) {
...
payload.handle[i % CLEAR_EVENT_MAX_HANDLES] = ...
if (i % CXL_CLEAR_EVENT_MAX_HANDLES == CXL_CLEAR_EVENT_MAX_HANDLE - 1) {
send command.
}
}but that looks worse to me than the double loop.
Making outer loop
for (j = 0; j <= total / CXL_CLEAR_EVENT_MAX_HANDLES; j++)
might bet clearer but then you'd have to do records[j * CXL_CLEAR_EVENT_MAX_HANDLES + i] which isn't nice.
Ah well, Ira gets to try and find a happy compromise.
...
Quoted 6 lines
Quoted 6 lines
include/uapi/linux/cxl_mem.h +0
-0@@ -25,6 +25,7 @@ ___C(RAW, "Raw device command"), \
Quoted 8 lines
include/uapi/linux/cxl_mem.h line 31
Quoted 6 lines
include/uapi/linux/cxl_mem.h +1
-0___C (GET_SUPPORTED_LOGS ,"Get Supported Logs" ), \___C (GET_EVENT_RECORD ,"Get Event Record" ), \ +___C (CLEAR_EVENT_RECORD ,"Clear Event Record" ), \___C (GET_FW_INFO ,"Get FW Info" ), \___C (GET_PARTITION_INFO ,"Get Partition Information" ), \ ___C(GET_LSA, "Get Label Storage Area"), \Same, "yikes" / "must be at the end of the enum" feedback.
Macro magic makes that non obvious.. Not that I'd ever said I thought this trick was a bad idea ;)
Steven Rostedt
maintainerRe: [PATCH V2 03/11] cxl/mem: Implement Clear Event Records command
In reply to Dan WilliamsQuoted 11 lines · On Thu, 1 Dec 2022 18:29:20 -0800 Dan Williams <[email protected]> wrote:
Quoted 11 lines
static void cxl_mem_get_records_log(struct
cxl_dev_state*cxlds,drivers/cxl/core/mbox.c +1
-1enum cxl_event_log_type type) {@@ -732,13 +769,22 @@ static void cxl_mem_get_records_log(struct cxl_dev_state *cxlds, } nr_rec =le16_to_cpu (payload->record_count); -if (trace_cxl_generic_event_enabled ()) { +if (nr_rec >0 ) {int i;
Quoted 9 lines
drivers/cxl/core/mbox.c static void cxl_mem_get_records_log(struct cxl_dev_state *cxlds, · line 775
Quoted 5 lines
drivers/cxl/core/mbox.c +1
-4-for (i =0 ; i < nr_rec; i++) -trace_cxl_generic_event (dev_name (cxlds->dev), - type, - &payload->records[i]); +if (trace_cxl_generic_event_enabled ()) {Again,
trace_cxl_generic_event_enabled()injects some awkward formatting here to micro-optimize looping. Any performance benefit this code might offer is likely offset by the extra human effort to read it.
This is commonly used throughout the kernel, and highly suggested for use to encapsulate any work being done only for tracing, when tracing is disabled. It uses static_braches/jump_labels which makes the loop into a 'nop' when tracing is off. That is, there is zero overhead for the for loop below (and there's not even a branch to skip it!)
But sure, if you really don't care as it's not a fast path, then keep it out. I like people to keep the habit of doing this, because otherwise it tends to creep into the fast paths.
-- Steve
Quoted 15 lines
Quoted 15 lines
drivers/cxl/core/mbox.c +12
-0+for (i =0 ; i < nr_rec; i++) +trace_cxl_generic_event (dev_name (cxlds->dev), + type, + &payload->records[i]); + } + + rc =cxl_clear_event_record (cxlds, type, payload, nr_rec); +if (rc) { +dev_err (cxlds->dev,"Event log '%s': Failed to clear events : %d" , +cxl_event_log_type_str (type), rc); +return ; + } }
Re: [PATCH V2 03/11] cxl/mem: Implement Clear Event Records command
Quoted 30 lines · Steven Rostedt
Quoted 11 lines · On Thu, 1 Dec 2022 18:29:20 -0800 Dan Williams <[email protected]> wrote:
Quoted 11 lines
static void cxl_mem_get_records_log(struct
cxl_dev_state*cxlds,drivers/cxl/core/mbox.c +1
-1enum cxl_event_log_type type) {@@ -732,13 +769,22 @@ static void cxl_mem_get_records_log(struct cxl_dev_state *cxlds, } nr_rec =le16_to_cpu (payload->record_count); -if (trace_cxl_generic_event_enabled ()) { +if (nr_rec >0 ) {int i;Quoted 9 lines
drivers/cxl/core/mbox.c static void cxl_mem_get_records_log(struct cxl_dev_state *cxlds, · line 775
Quoted 5 lines
drivers/cxl/core/mbox.c +1
-4-for (i =0 ; i < nr_rec; i++) -trace_cxl_generic_event (dev_name (cxlds->dev), - type, - &payload->records[i]); +if (trace_cxl_generic_event_enabled ()) {Again,
trace_cxl_generic_event_enabled()injects some awkward formatting here to micro-optimize looping. Any performance benefit this code might offer is likely offset by the extra human effort to read it.This is commonly used throughout the kernel, and highly suggested for use to encapsulate any work being done only for tracing, when tracing is disabled. It uses static_braches/jump_labels which makes the loop into a 'nop' when tracing is off. That is, there is zero overhead for the for loop below (and there's not even a branch to skip it!)
Quoted 3 lines
But sure, if you really don't care as it's not a fast path, then keep it out. I like people to keep the habit of doing this, because otherwise it tends to creep into the fast paths.
Duly noted. It makes a lot of sense when you are tracing in a fast path to skip any and all preamble code. In this case we are doing it after doing a whole series of uncached PCI mmio reads with all the stalling and serialization that implies.
Speaking of which, this probably wants a cond_resched() after each loop iteration.
I'll note it is also a tracepoint that is likely to be enabled most of the time in production.
Re: [PATCH V2 03/11] cxl/mem: Implement Clear Event Records command
Quoted 31 lines · Dan Williams
Quoted 30 lines · Steven Rostedt
Quoted 11 lines · On Thu, 1 Dec 2022 18:29:20 -0800 Dan Williams <[email protected]> wrote:
Quoted 11 lines
static void cxl_mem_get_records_log(struct
cxl_dev_state*cxlds,drivers/cxl/core/mbox.c +1
-1enum cxl_event_log_type type) {@@ -732,13 +769,22 @@ static void cxl_mem_get_records_log(struct cxl_dev_state *cxlds, } nr_rec =le16_to_cpu (payload->record_count); -if (trace_cxl_generic_event_enabled ()) { +if (nr_rec >0 ) {int i;Quoted 9 lines
drivers/cxl/core/mbox.c static void cxl_mem_get_records_log(struct cxl_dev_state *cxlds, · line 775
Quoted 5 lines
drivers/cxl/core/mbox.c +1
-4-for (i =0 ; i < nr_rec; i++) -trace_cxl_generic_event (dev_name (cxlds->dev), - type, - &payload->records[i]); +if (trace_cxl_generic_event_enabled ()) {Again,
trace_cxl_generic_event_enabled()injects some awkward formatting here to micro-optimize looping. Any performance benefit this code might offer is likely offset by the extra human effort to read it.This is commonly used throughout the kernel, and highly suggested for use to encapsulate any work being done only for tracing, when tracing is disabled. It uses static_braches/jump_labels which makes the loop into a 'nop' when tracing is off. That is, there is zero overhead for the for loop below (and there's not even a branch to skip it!)
Quoted 3 lines
Quoted 3 lines
But sure, if you really don't care as it's not a fast path, then keep it out. I like people to keep the habit of doing this, because otherwise it tends to creep into the fast paths.
Thanks for chiming in here Steven. I should have pushed back on this.
Quoted 11 lines
Duly noted. It makes a lot of sense when you are tracing in a fast path to skip any and all preamble code. In this case we are doing it after doing a whole series of uncached PCI mmio reads with all the stalling and serialization that implies.
Speaking of which, this probably wants a
cond_resched()after each loop iteration.I'll note it is also a tracepoint that is likely to be enabled most of the time in production.
Ok I did not have any of these in there originally and I will remove them now.
Thanks!
Ira
Re: [PATCH V2 03/11] cxl/mem: Implement Clear Event Records command
In reply to Dan WilliamsQuoted 15 lines · Dan Williams
Quoted 6 lines · ira.weiny@ wrote:
From: Ira Weiny <[email protected]>
CXL rev 3.0 section 8.2.9.2.3 defines the Clear Event Records mailbox command. After an event record is read it needs to be cleared from the event log.
Quoted 7 lines
Implement
cxl_clear_event_record()to clear all record retrieved from the device.Each record is cleared explicitly. A clear all bit is specified but events could arrive between a get and any final clear all operation. This means events would be missed.
Therefore each event is cleared specifically.
Quoted 8 lines
Note that the spec has a better reason for why Clear All has limited usage:
"Clear All Events is only allowed when the Event Log has overflowed; otherwise, the device shall return Invalid Input."
Will need to wait and see if we need that to keep pace with a device with a high event frequency.
Perhaps. But yea I would wait and see.
[snip]
Quoted 13 lines
Quoted 13 lines
drivers/cxl/core/mbox.c +13
-0+static int cxl_clear_event_record (struct cxl_dev_state *cxlds, +enum cxl_event_log_type log, +struct cxl_get_event_payload *get_pl, +u16 total) +{ +struct cxl_mbox_clear_event_payload payload = { + .event_log = log, + }; +int cnt; + +/* +* Clear Event Records uses u8 for the handle cnt while Get Event +* Record can return up to 0xffff records.
Quoted 8 lines
drivers/cxl/core/mbox.c int cxl_enumerate_cmds(struct cxl_dev_state *cxlds) · line 728
Quoted 4 lines
drivers/cxl/core/mbox.c +4
-0+*/ +for (cnt =0 ; cnt < total;/* cnt incremented internally */ ) { +u8 nr_recs =min_t (u8, (total - cnt), +CXL_CLEAR_EVENT_MAX_HANDLES );This seems overly complicated. @total is a duplicate of @get_pl->record_count, and the 2 loops feel like it could be cut down to one.
Sure, total is redundant to pass to the function.
However, 2 loops is IMO not at all overly complicated. Note that the 2 loops do not do the same thing. The inner loop is filling in the payload for the Clear command. There is really no way around doing this.
Now that I've had time to think about it:
Are you suggesting we issue a single mailbox command for every handle?
That would be a single loop. But a lot more mailbox commands.
Quoted 9 lines
drivers/cxl/core/mbox.c int cxl_enumerate_cmds(struct cxl_dev_state *cxlds) · line 733
Quoted 6 lines
drivers/cxl/core/mbox.c +5
-0+int i, rc; + +for (i =0 ; i < nr_recs; i++, cnt++) { + payload.handle[i] = get_pl->records[cnt].hdr.handle; + dev_dbg(cxlds->dev, "Event log '%s': Clearning %u\n",While I do think this operation is a mix of clearing and cleaning event records, I don't think "Clearning" is a word.
LOL... I'll fix it. :-D
Quoted 26 lines
Quoted 26 lines
drivers/cxl/core/mbox.c +15
-1+cxl_event_log_type_str (log), + le16_to_cpu(payload.handle[i])); + } + payload.nr_recs = nr_recs; + + rc =cxl_mbox_send_cmd (cxlds,CXL_MBOX_OP_CLEAR_EVENT_RECORD , + &payload,sizeof (payload),NULL ,0 ); +if (rc) +return rc; + } + +return 0 ; +} +static void cxl_mem_get_records_log (struct cxl_dev_state *cxlds,enum cxl_event_log_type type) {@@ -732,13 +769,22 @@ static void cxl_mem_get_records_log(struct cxl_dev_state *cxlds, } nr_rec =le16_to_cpu (payload->record_count); -if (trace_cxl_generic_event_enabled ()) { +if (nr_rec >0 ) {int i;
Quoted 9 lines
drivers/cxl/core/mbox.c static void cxl_mem_get_records_log(struct cxl_dev_state *cxlds, · line 775
Quoted 5 lines
drivers/cxl/core/mbox.c +1
-4-for (i =0 ; i < nr_rec; i++) -trace_cxl_generic_event (dev_name (cxlds->dev), - type, - &payload->records[i]); +if (trace_cxl_generic_event_enabled ()) {Again,
trace_cxl_generic_event_enabled()injects some awkward formatting here to micro-optimize looping. Any performance benefit this code might offer is likely offset by the extra human effort to read it.
Agreed. Gone.
Quoted 68 lines
Quoted 68 lines
drivers/cxl/core/mbox.c +15
-2+for (i =0 ; i < nr_rec; i++) +trace_cxl_generic_event (dev_name (cxlds->dev), + type, + &payload->records[i]); + } + + rc =cxl_clear_event_record (cxlds, type, payload, nr_rec); +if (rc) { +dev_err (cxlds->dev,"Event log '%s': Failed to clear events : %d" , +cxl_event_log_type_str (type), rc); +return ; + } } if (trace_cxl_overflow_enabled() &&@@ -780,10 +826,11 @@ static struct cxl_get_event_payload *alloc_event_buf(struct cxl_dev_state *cxlds* cxl_mem_get_event_records - Get Event Records from the device * @cxlds: The device data for the operation * -* Retrieve all event records available on the device and report them as trace -* events. +* Retrieve all event records available on the device, report them as trace +* events, and clear them. * * See CXL rev 3.0 @8.2.9.2.2 Get Event Records +* See CXL rev 3.0 @8.2.9.2.3 Clear Event Records */ void cxl_mem_get_event_records (struct cxl_dev_state *cxlds) {drivers/cxl/cxlmem.h +14
-0@@ -261,6 +261,7 @@ enum cxl_opcode {CXL_MBOX_OP_INVALID =0x0000 ,CXL_MBOX_OP_RAW =CXL_MBOX_OP_INVALID ,CXL_MBOX_OP_GET_EVENT_RECORD =0x0100 , +CXL_MBOX_OP_CLEAR_EVENT_RECORD =0x0101 ,CXL_MBOX_OP_GET_FW_INFO =0x0200 ,CXL_MBOX_OP_ACTIVATE_FW =0x0202 , CXL_MBOX_OP_GET_SUPPORTED_LOGS = 0x0400,@@ -396,6 +397,19 @@ static inline const char *cxl_event_log_type_str(enum cxl_event_log_type type)return "<unknown>" ; } +/* +* Clear Event Records input payload +* CXL rev 3.0 section 8.2.9.2.3; Table 8-51 +*/ +#define CXL_CLEAR_EVENT_MAX_HANDLES (0xff) +struct cxl_mbox_clear_event_payload { +u8 event_log;/* enum cxl_event_log_type */ +u8 clear_flags; +u8 nr_recs; +u8 reserved[3 ]; +__le16 handle[CXL_CLEAR_EVENT_MAX_HANDLES ]; +}; +struct cxl_mbox_get_partition_info {__le64 active_volatile_cap;__le64 active_persistent_cap;include/uapi/linux/cxl_mem.h +0
-0@@ -25,6 +25,7 @@ ___C(RAW, "Raw device command"), \
Quoted 8 lines
include/uapi/linux/cxl_mem.h line 31
Quoted 6 lines
include/uapi/linux/cxl_mem.h +1
-0___C (GET_SUPPORTED_LOGS ,"Get Supported Logs" ), \___C (GET_EVENT_RECORD ,"Get Event Record" ), \ +___C (CLEAR_EVENT_RECORD ,"Clear Event Record" ), \___C (GET_FW_INFO ,"Get FW Info" ), \___C (GET_PARTITION_INFO ,"Get Partition Information" ), \ ___C(GET_LSA, "Get Label Storage Area"), \Same, "yikes" / "must be at the end of the enum" feedback.
Yep,
Ira
Re: [PATCH V2 03/11] cxl/mem: Implement Clear Event Records command
Quoted 57 lines · Ira Weiny
Quoted 15 lines · Dan Williams
Quoted 6 lines · ira.weiny@ wrote:
From: Ira Weiny <[email protected]>
CXL rev 3.0 section 8.2.9.2.3 defines the Clear Event Records mailbox command. After an event record is read it needs to be cleared from the event log.
Quoted 7 lines
Implement
cxl_clear_event_record()to clear all record retrieved from the device.Each record is cleared explicitly. A clear all bit is specified but events could arrive between a get and any final clear all operation. This means events would be missed.
Therefore each event is cleared specifically.Quoted 8 lines
Note that the spec has a better reason for why Clear All has limited usage:
"Clear All Events is only allowed when the Event Log has overflowed; otherwise, the device shall return Invalid Input."
Will need to wait and see if we need that to keep pace with a device with a high event frequency.
Perhaps. But yea I would wait and see.
[snip]
Quoted 13 lines
Quoted 13 lines
drivers/cxl/core/mbox.c +13
-0+static int cxl_clear_event_record (struct cxl_dev_state *cxlds, +enum cxl_event_log_type log, +struct cxl_get_event_payload *get_pl, +u16 total) +{ +struct cxl_mbox_clear_event_payload payload = { + .event_log = log, + }; +int cnt; + +/* +* Clear Event Records uses u8 for the handle cnt while Get Event +* Record can return up to 0xffff records. Quoted 8 lines
drivers/cxl/core/mbox.c int cxl_enumerate_cmds(struct cxl_dev_state *cxlds) · line 728
Quoted 4 lines
drivers/cxl/core/mbox.c +4
-0+*/ +for (cnt =0 ; cnt < total;/* cnt incremented internally */ ) { +u8 nr_recs =min_t (u8, (total - cnt), +CXL_CLEAR_EVENT_MAX_HANDLES );This seems overly complicated. @total is a duplicate of @get_pl->record_count, and the 2 loops feel like it could be cut down to one.
Sure, total is redundant to pass to the function.
However, 2 loops is IMO not at all overly complicated. Note that the 2 loops do not do the same thing. The inner loop is filling in the payload for the Clear command. There is really no way around doing this.
Quoted 5 lines
Now that I've had time to think about it:
Are you suggesting we issue a single mailbox command for every handle?
That would be a single loop. But a lot more mailbox commands.
I was thinking something like this pseudo code
int tosend = le16_to_cpu(get_pl->record_count);
int added = 0;
for (i = 0; i < tosend; i++) {
add_to_clear(added++);
if (added == MAX)
send_mailbox();
added = 0;
}
if (added)
send_mailbox();...where it batches and sends every 256 and one more send afterwards for any stragglers.
Re: [PATCH V2 03/11] cxl/mem: Implement Clear Event Records command
Quoted 79 lines · Dan Williams
Quoted 57 lines · Ira Weiny
Quoted 15 lines · Dan Williams
Quoted 6 lines · ira.weiny@ wrote:
From: Ira Weiny <[email protected]>
CXL rev 3.0 section 8.2.9.2.3 defines the Clear Event Records mailbox command. After an event record is read it needs to be cleared from the event log.
Quoted 7 lines
Implement
cxl_clear_event_record()to clear all record retrieved from the device.Each record is cleared explicitly. A clear all bit is specified but events could arrive between a get and any final clear all operation. This means events would be missed.
Therefore each event is cleared specifically.Quoted 8 lines
Note that the spec has a better reason for why Clear All has limited usage:
"Clear All Events is only allowed when the Event Log has overflowed; otherwise, the device shall return Invalid Input."
Will need to wait and see if we need that to keep pace with a device with a high event frequency.
Perhaps. But yea I would wait and see.
[snip]
Quoted 13 lines
Quoted 13 lines
drivers/cxl/core/mbox.c +13
-0+static int cxl_clear_event_record (struct cxl_dev_state *cxlds, +enum cxl_event_log_type log, +struct cxl_get_event_payload *get_pl, +u16 total) +{ +struct cxl_mbox_clear_event_payload payload = { + .event_log = log, + }; +int cnt; + +/* +* Clear Event Records uses u8 for the handle cnt while Get Event +* Record can return up to 0xffff records. Quoted 8 lines
drivers/cxl/core/mbox.c int cxl_enumerate_cmds(struct cxl_dev_state *cxlds) · line 728
Quoted 4 lines
drivers/cxl/core/mbox.c +4
-0+*/ +for (cnt =0 ; cnt < total;/* cnt incremented internally */ ) { +u8 nr_recs =min_t (u8, (total - cnt), +CXL_CLEAR_EVENT_MAX_HANDLES );This seems overly complicated. @total is a duplicate of @get_pl->record_count, and the 2 loops feel like it could be cut down to one.
Sure, total is redundant to pass to the function.
However, 2 loops is IMO not at all overly complicated. Note that the 2 loops do not do the same thing. The inner loop is filling in the payload for the Clear command. There is really no way around doing this.
Quoted 5 lines
Now that I've had time to think about it:
Are you suggesting we issue a single mailbox command for every handle?
That would be a single loop. But a lot more mailbox commands.
I was thinking something like this pseudo code
int tosend =
le16_to_cpu(get_pl->record_count);
int added = 0;for (i = 0; i < tosend; i++) { add_to_clear(added++); if (added == MAX) send_mailbox(); added = 0; } if (added) send_mailbox();
Quoted 2 lines
...where it batches and sends every 256 and one more send afterwards for any stragglers.
Ok I'm not convinced it makes that much difference but I don't have the fortitude to try and look at the assembly to argue... ;-)
Done.
Ira
PATCH v2 4/11 cxl/mem: Clear events on driver load
In reply to [email protected]From: Ira Weiny <[email protected]>
The information contained in the events prior to the driver loading can be queried at any time through other mailbox commands.
Ensure a clean slate of events by reading and clearing the events. The events are sent to the trace buffer but it is not anticipated to have anyone listening to it at driver load time.
--- drivers/cxl/pci.c | 2 ++ tools/testing/cxl/test/mem.c | 2 ++ 2 files changed, 4 insertions(+)
drivers/cxl/pci.c +2 -0
@@ -521,6 +521,8 @@ static int cxl_pci_probe(struct pci_dev *pdev, const struct pci_device_id *id)if (IS_ERR (cxlmd))return PTR_ERR (cxlmd); +cxl_mem_get_event_records (cxlds); +if (resource_size (&cxlds->pmem_res) &&IS_ENABLED (CONFIG_CXL_PMEM )) rc =devm_cxl_add_nvdimm (&pdev->dev, cxlmd);
tools/testing/cxl/test/mem.c +2 -0
@@ -285,6 +285,8 @@ static int cxl_mock_mem_probe(struct platform_device *pdev)if (IS_ERR (cxlmd))return PTR_ERR (cxlmd); +cxl_mem_get_event_records (cxlds); +if (resource_size (&cxlds->pmem_res) &&IS_ENABLED (CONFIG_CXL_PMEM )) rc =devm_cxl_add_nvdimm (dev, cxlmd);
Signature · 1 line
2.37.2
Re: [PATCH V2 04/11] cxl/mem: Clear events on driver load
Quoted 5 lines · On Wed, 30 Nov 2022 16:27:12 -0800 [email protected] wrote:
From: Ira Weiny <[email protected]>
The information contained in the events prior to the driver loading can be queried at any time through other mailbox commands.
Quoted 7 lines
Ensure a clean slate of events by reading and clearing the events. The events are sent to the trace buffer but it is not anticipated to have anyone listening to it at driver load time.
Reviewed-by Jonathan Cameron <[email protected]>Reviewed-by Dave Jiang <[email protected]>Signed-off-by Ira Weiny <[email protected]>
Probably not worth addressing but there is a corner case where this might fail if some broken software already messed with reading out the events.
Imagine it read the first mailbox sized chunk, but didn't clear them...
If that happens, then we'd end up seeing the whole list, but in non temporal order and hence trying to clear them out of order with predictable fails.
Maybe this is the category of things we 'fix' if we ever hear of it actually happening.
So with that caveat called out so I can say 'I told you so' :), fine to keep my tag on this.
Thanks,
Jonathan
Quoted 31 lines
--- drivers/cxl/pci.c | 2 ++ tools/testing/cxl/test/mem.c | 2 ++ 2 files changed, 4 insertions(+)drivers/cxl/pci.c +2
-0@@ -521,6 +521,8 @@ static int cxl_pci_probe(struct pci_dev *pdev, const struct pci_device_id *id)if (IS_ERR (cxlmd))return PTR_ERR (cxlmd); +cxl_mem_get_event_records (cxlds); +if (resource_size (&cxlds->pmem_res) &&IS_ENABLED (CONFIG_CXL_PMEM )) rc =devm_cxl_add_nvdimm (&pdev->dev, cxlmd);tools/testing/cxl/test/mem.c +2
-0@@ -285,6 +285,8 @@ static int cxl_mock_mem_probe(struct platform_device *pdev)if (IS_ERR (cxlmd))return PTR_ERR (cxlmd); +cxl_mem_get_event_records (cxlds); +if (resource_size (&cxlds->pmem_res) &&IS_ENABLED (CONFIG_CXL_PMEM )) rc =devm_cxl_add_nvdimm (dev, cxlmd);
Re: [PATCH V2 04/11] cxl/mem: Clear events on driver load
Quoted 12 lines · Jonathan Cameron
Quoted 9 lines · On Wed, 30 Nov 2022 16:27:12 -0800 [email protected] wrote:
From: Ira Weiny <[email protected]>
The information contained in the events prior to the driver loading can be queried at any time through other mailbox commands.
Ensure a clean slate of events by reading and clearing the events. The events are sent to the trace buffer but it is not anticipated to have anyone listening to it at driver load time.
Quoted 6 lines
Quoted 3 lines
Reviewed-by Jonathan Cameron <[email protected]>Reviewed-by Dave Jiang <[email protected]>Signed-off-by Ira Weiny <[email protected]>Probably not worth addressing but there is a corner case where this might fail if some broken software already messed with reading out the events.
Yea they can keep the pieces if they have done that.
Quoted 11 lines
Imagine it read the first mailbox sized chunk, but didn't clear them...
If that happens, then we'd end up seeing the whole list, but in non temporal order and hence trying to clear them out of order with predictable fails.
Maybe this is the category of things we 'fix' if we ever hear of it actually happening.
So with that caveat called out so I can say 'I told you so' :), fine to keep my tag on this.
Sure! We probably owe you this T-Shirt already!
https://www.amazon.com/Big-Bang-Theory-Informed-Thusly/dp/B06XYCSQRF
:-D
Ira
Quoted 38 lines
Thanks,
Jonathan
Quoted 31 lines
--- drivers/cxl/pci.c | 2 ++ tools/testing/cxl/test/mem.c | 2 ++ 2 files changed, 4 insertions(+)drivers/cxl/pci.c +2
-0@@ -521,6 +521,8 @@ static int cxl_pci_probe(struct pci_dev *pdev, const struct pci_device_id *id)if (IS_ERR (cxlmd))return PTR_ERR (cxlmd); +cxl_mem_get_event_records (cxlds); +if (resource_size (&cxlds->pmem_res) &&IS_ENABLED (CONFIG_CXL_PMEM )) rc =devm_cxl_add_nvdimm (&pdev->dev, cxlmd);tools/testing/cxl/test/mem.c +2
-0@@ -285,6 +285,8 @@ static int cxl_mock_mem_probe(struct platform_device *pdev)if (IS_ERR (cxlmd))return PTR_ERR (cxlmd); +cxl_mem_get_event_records (cxlds); +if (resource_size (&cxlds->pmem_res) &&IS_ENABLED (CONFIG_CXL_PMEM )) rc =devm_cxl_add_nvdimm (dev, cxlmd);
RE: [PATCH V2 04/11] cxl/mem: Clear events on driver load
In reply to [email protected]cxl/mem is cxl_mem.ko, This is cxl/pci.
Quoted 8 lines · ira.weiny@ wrote:
From: Ira Weiny <[email protected]>
The information contained in the events prior to the driver loading can be queried at any time through other mailbox commands.
Ensure a clean slate of events by reading and clearing the events. The events are sent to the trace buffer but it is not anticipated to have anyone listening to it at driver load time.
This is easy to guarantee with modprobe policy, so I am not sure it is worth stating.
This breakdown feels odd. I would split the trace event definitions into its own lead in patch since that is a pile of definitions that can be merged on their own. Then squash get, clear, and this patch into one patch as they don't have much reason to go in separately.
Quoted 26 lines
Reviewed-by Jonathan Cameron <[email protected]>Reviewed-by Dave Jiang <[email protected]>Signed-off-by Ira Weiny <[email protected]>--- drivers/cxl/pci.c | 2 ++ tools/testing/cxl/test/mem.c | 2 ++ 2 files changed, 4 insertions(+)drivers/cxl/pci.c +2
-0@@ -521,6 +521,8 @@ static int cxl_pci_probe(struct pci_dev *pdev, const struct pci_device_id *id)if (IS_ERR (cxlmd))return PTR_ERR (cxlmd); +cxl_mem_get_event_records (cxlds); +if (resource_size (&cxlds->pmem_res) &&IS_ENABLED (CONFIG_CXL_PMEM )) rc =devm_cxl_add_nvdimm (&pdev->dev, cxlmd);tools/testing/cxl/test/mem.c +0
-0
tools/testing/cxl/test/mem.c static int cxl_mock_mem_probe(struct platform_device *pdev) · line 289
Quoted 5 lines
tools/testing/cxl/test/mem.c +2
-0@@ -285,6 +285,8 @@ static int cxl_mock_mem_probe(struct platform_device *pdev)if (IS_ERR (cxlmd))return PTR_ERR (cxlmd); +cxl_mem_get_event_records (cxlds); +
This hunk likely goes with the first patch that actually implements some mocked events.
Quoted 6 lines
tools/testing/cxl/test/mem.c +0
-0if (resource_size (&cxlds->pmem_res) &&IS_ENABLED (CONFIG_CXL_PMEM )) rc =devm_cxl_add_nvdimm (dev, cxlmd);Signature · 1 line
2.37.2
Re: [PATCH V2 04/11] cxl/mem: Clear events on driver load
Quoted 8 lines · Dan Williams
cxl/mem is cxl_mem.ko, This is cxl/pci.
Quoted 5 lines · ira.weiny@ wrote:
From: Ira Weiny <[email protected]>
The information contained in the events prior to the driver loading can be queried at any time through other mailbox commands.
Quoted 6 lines
Quoted 3 lines
Ensure a clean slate of events by reading and clearing the events. The events are sent to the trace buffer but it is not anticipated to have anyone listening to it at driver load time.
This is easy to guarantee with modprobe policy, so I am not sure it is worth stating.
Fair enough. But there was some discussion early on regarding why reading and clearing on startup was a good thing. This showed that we chose to do that and why we don't care. I'll remove it.
Quoted 5 lines
This breakdown feels odd. I would split the trace event definitions into its own lead in patch since that is a pile of definitions that can be merged on their own. Then squash get, clear, and this patch into one patch as they don't have much reason to go in separately.
I agree that splitting the Get/Clear/and this patch was odd. However, splitting Get/Clear made the discussion on those operations easier IMO.
As a result this did not really belong in either of those patches on their own.
It is also very clearly a do one thing per patch situation.
Quoted 30 lines
Quoted 30 lines
Reviewed-by Jonathan Cameron <[email protected]>Reviewed-by Dave Jiang <[email protected]>Signed-off-by Ira Weiny <[email protected]>--- drivers/cxl/pci.c | 2 ++ tools/testing/cxl/test/mem.c | 2 ++ 2 files changed, 4 insertions(+)drivers/cxl/pci.c +2
-0@@ -521,6 +521,8 @@ static int cxl_pci_probe(struct pci_dev *pdev, const struct pci_device_id *id)if (IS_ERR (cxlmd))return PTR_ERR (cxlmd); +cxl_mem_get_event_records (cxlds); +if (resource_size (&cxlds->pmem_res) &&IS_ENABLED (CONFIG_CXL_PMEM )) rc =devm_cxl_add_nvdimm (&pdev->dev, cxlmd);tools/testing/cxl/test/mem.c +0
-0@@ -285,6 +285,8 @@ static int cxl_mock_mem_probe(struct platform_device *pdev)if (IS_ERR (cxlmd))return PTR_ERR (cxlmd);
Quoted 5 lines
tools/testing/cxl/test/mem.c static int cxl_mock_mem_probe(struct platform_device *pdev) · line 289
Quoted 2 lines
tools/testing/cxl/test/mem.c +2
-0+cxl_mem_get_event_records (cxlds); +This hunk likely goes with the first patch that actually implements some mocked events.
If this patch was squashed into the other patches yes. But as a patch which does exactly 1 thing "Clear events on driver load" it works IMO. I could just have well put this patch at the very end.
Now that the Get/Clear operations are more settled I'll split this out and squash it as you suggest. Jonathan suggested squashing Get/Clear too but again I really prefer the 1 thing/patch and each of those operations seemed like a good breakdown.
Ira
Quoted 9 lines
Quoted 7 lines
tools/testing/cxl/test/mem.c +0
-0if (resource_size (&cxlds->pmem_res) &&IS_ENABLED (CONFIG_CXL_PMEM )) rc =devm_cxl_add_nvdimm (dev, cxlmd);Signature · 1 line
2.37.2
Re: [PATCH V2 04/11] cxl/mem: Clear events on driver load
Quoted 69 lines · Ira Weiny
Quoted 8 lines · Dan Williams
cxl/mem is cxl_mem.ko, This is cxl/pci.
Quoted 5 lines · ira.weiny@ wrote:
From: Ira Weiny <[email protected]>
The information contained in the events prior to the driver loading can be queried at any time through other mailbox commands.
Quoted 6 lines
Quoted 3 lines
Ensure a clean slate of events by reading and clearing the events. The events are sent to the trace buffer but it is not anticipated to have anyone listening to it at driver load time.
This is easy to guarantee with modprobe policy, so I am not sure it is worth stating.
Fair enough. But there was some discussion early on regarding why reading and clearing on startup was a good thing. This showed that we chose to do that and why we don't care. I'll remove it.
Quoted 5 lines
This breakdown feels odd. I would split the trace event definitions into its own lead in patch since that is a pile of definitions that can be merged on their own. Then squash get, clear, and this patch into one patch as they don't have much reason to go in separately.
I agree that splitting the Get/Clear/and this patch was odd. However, splitting Get/Clear made the discussion on those operations easier IMO.
As a result this did not really belong in either of those patches on their own.
It is also very clearly a do one thing per patch situation.
Quoted 30 lines
Quoted 30 lines
Reviewed-by Jonathan Cameron <[email protected]>Reviewed-by Dave Jiang <[email protected]>Signed-off-by Ira Weiny <[email protected]>--- drivers/cxl/pci.c | 2 ++ tools/testing/cxl/test/mem.c | 2 ++ 2 files changed, 4 insertions(+)drivers/cxl/pci.c +2
-0@@ -521,6 +521,8 @@ static int cxl_pci_probe(struct pci_dev *pdev, const struct pci_device_id *id)if (IS_ERR (cxlmd))return PTR_ERR (cxlmd); +cxl_mem_get_event_records (cxlds); +if (resource_size (&cxlds->pmem_res) &&IS_ENABLED (CONFIG_CXL_PMEM )) rc =devm_cxl_add_nvdimm (&pdev->dev, cxlmd);tools/testing/cxl/test/mem.c +0
-0@@ -285,6 +285,8 @@ static int cxl_mock_mem_probe(struct platform_device *pdev)if (IS_ERR (cxlmd))return PTR_ERR (cxlmd);Quoted 5 lines
tools/testing/cxl/test/mem.c static int cxl_mock_mem_probe(struct platform_device *pdev) · line 289
Quoted 2 lines
tools/testing/cxl/test/mem.c +2
-0+cxl_mem_get_event_records (cxlds); +This hunk likely goes with the first patch that actually implements some mocked events.
Quoted 9 lines
If this patch was squashed into the other patches yes. But as a patch which does exactly 1 thing "Clear events on driver load" it works IMO. I could just have well put this patch at the very end.
Now that the Get/Clear operations are more settled I'll split this out and squash it as you suggest. Jonathan suggested squashing Get/Clear too but again I really prefer the 1 thing/patch and each of those operations seemed like a good breakdown.
I'll preface this by saying if you ask 3 kernel developers how to split a patch series you'll get 5 answers. For me though, a patch should be a bisectable full-thought. That at each step of a series the kernel is incrementally better in a way that makes sense. The kernel that gets Get Events likely needs to clear them too to complete 1 full thought about enbling Event handling. Otherwise a kernel that just retrieves some events until they overflow feels like a POC.
Re: [PATCH V2 04/11] cxl/mem: Clear events on driver load
Quoted 3 lines · Dan Williams
Quoted 2 lines · Ira Weiny
Quoted 1 line · Dan Williams
cxl/mem is cxl_mem.ko, This is cxl/pci.
[snip]
Quoted 10 lines
Quoted 10 lines
Quoted 5 lines
Review
Quoted 2 lines
(quoted diff) +2
-0+ cxl_mem_get_event_records(cxlds); +This hunk likely goes with the first patch that actually implements some mocked events.
If this patch was squashed into the other patches yes. But as a patch which does exactly 1 thing "Clear events on driver load" it works IMO. I could just have well put this patch at the very end.
Quoted 8 lines
Quoted 5 lines
Now that the Get/Clear operations are more settled I'll split this out and squash it as you suggest. Jonathan suggested squashing Get/Clear too but again I really prefer the 1 thing/patch and each of those operations seemed like a good breakdown.
I'll preface this by saying if you ask 3 kernel developers how to split a patch series you'll get 5 answers.
Indeed.
Quoted 6 lines
For me though, a patch should be a
bisectable full-thought. That at each step of a series the kernel is incrementally better in a way that makes sense. The kernel that gets Get Events likely needs to clear them too to complete 1 full thought about enbling Event handling. Otherwise a kernel that just retrieves some events until they overflow feels like a POC.
I've squashed it.
Ira
PATCH v2 5/11 cxl/mem: Trace General Media Event Record
In reply to [email protected]From: Ira Weiny <[email protected]>
CXL rev 3.0 section 8.2.9.2.1.1 defines the General Media Event Record.
Determine if the event read is a general media record and if so trace the record as a General Media Event Record.
---
Changes from V1:
Jonathan
fix spec references for CXL rev 3.0 Make flags all caps
Changes from RFC v2:
Output DPA flags as a single field
Ensure names of fields match what TP_print outputs Steven
prefix TRACE_EVENT with 'cxl_'
Jonathan
Remove Reserved field
Changes from RFC:
Add reserved byte array
Use common CXL event header record macros Jonathan
Use unaligned_le{24,16} for unaligned fields Don't use the inverse of phy addr mask
Dave Jiang
s/cxl_gen_media_event/general_media s/cxl_evt_gen_media/cxl_event_gen_media
--- drivers/cxl/core/mbox.c | 40 ++++++++++-- drivers/cxl/cxlmem.h | 19 ++++++ include/trace/events/cxl.h | 124 +++++++++++++++++++++++++++++++++++++ 3 files changed, 179 insertions(+), 4 deletions(-)
drivers/cxl/core/mbox.c +36 -4
@@ -709,6 +709,38 @@ int cxl_enumerate_cmds(struct cxl_dev_state *cxlds) }EXPORT_SYMBOL_NS_GPL (cxl_enumerate_cmds,CXL ); +/* +* General Media Event Record +* CXL rev 3.0 Section 8.2.9.2.1.1; Table 8-43 +*/ +static const uuid_t gen_media_event_uuid = +UUID_INIT (0xfbcd0a77 ,0xc260 ,0x417f , +0x85 ,0xa9 ,0x08 ,0x8b ,0x16 ,0x21 ,0xeb ,0xa6 ); + +static bool cxl_event_tracing_enabled (void ) +{ +return trace_cxl_generic_event_enabled () || +trace_cxl_general_media_enabled (); +} + +static void cxl_trace_event_record (const char *dev_name, +enum cxl_event_log_type type, +struct cxl_event_record_raw *record) +{ +uuid_t *id = &record->hdr.id; + +if (uuid_equal (id, &gen_media_event_uuid)) { +struct cxl_event_gen_media *rec = + (struct cxl_event_gen_media *)record; + +trace_cxl_general_media (dev_name, type, rec); +return ; + } + +/* For unknown record types print just the header */ +trace_cxl_generic_event (dev_name, type, record); +} + static int cxl_clear_event_record(struct cxl_dev_state *cxlds, enum cxl_event_log_type log, struct cxl_get_event_payload *get_pl,
@@ -772,11 +804,11 @@ static void cxl_mem_get_records_log(struct cxl_dev_state *cxlds,if (nr_rec >0 ) {int i; -if (trace_cxl_generic_event_enabled ()) { +if (cxl_event_tracing_enabled ()) {for (i =0 ; i < nr_rec; i++) -trace_cxl_generic_event (dev_name (cxlds->dev), - type, - &payload->records[i]); +cxl_trace_event_record (dev_name (cxlds->dev), + type, + &payload->records[i]); } rc =cxl_clear_event_record (cxlds, type, payload, nr_rec);
drivers/cxl/cxlmem.h +19 -0
@@ -410,6 +410,25 @@ struct cxl_mbox_clear_event_payload {__le16 handle[CXL_CLEAR_EVENT_MAX_HANDLES ]; }; +/* +* General Media Event Record +* CXL rev 3.0 Section 8.2.9.2.1.1; Table 8-43 +*/ +#define CXL_EVENT_GEN_MED_COMP_ID_SIZE 0x10 +struct cxl_event_gen_media { +struct cxl_event_record_hdr hdr; +__le64 phys_addr; +u8 descriptor; +u8 type; +u8 transaction_type; +u8 validity_flags[2 ]; +u8 channel; +u8 rank; +u8 device[3 ]; +u8 component_id[CXL_EVENT_GEN_MED_COMP_ID_SIZE ]; +u8 reserved[0x2e ]; +} __packed; +struct cxl_mbox_get_partition_info {__le64 active_volatile_cap;__le64 active_persistent_cap;
include/trace/events/cxl.h +124 -0
130 lines not on this page · raw message
Signature · 1 line
2.37.2
Re: [PATCH V2 05/11] cxl/mem: Trace General Media Event Record
Quoted 10 lines · On 11/30/2022 5:27 PM, [email protected] wrote:
From: Ira Weiny <[email protected]>
CXL rev 3.0 section 8.2.9.2.1.1 defines the General Media Event Record.
Determine if the event read is a general media record and if so trace the record as a General Media Event Record.
Reviewed-by Steven Rostedt (Google) <[email protected]>Reviewed-by Jonathan Cameron <[email protected]>Signed-off-by Ira Weiny <[email protected]>
Quoted 254 lines
---
Changes from V1:
Jonathan
fix spec references for CXL rev 3.0 Make flags all capsChanges from RFC v2:
Output DPA flags as a single field
Ensure names of fields match whatTP_printoutputs Steven
prefixTRACE_EVENTwith 'cxl_'
Jonathan
Remove Reserved fieldChanges from RFC:
Add reserved byte array
Use common CXL event header record macros Jonathan
Use unaligned_le{24,16} for unaligned fields Don't use the inverse of phy addr mask
Dave Jiang
s/cxl_gen_media_event/general_media s/cxl_evt_gen_media/cxl_event_gen_media--- drivers/cxl/core/mbox.c | 40 ++++++++++-- drivers/cxl/cxlmem.h | 19 ++++++ include/trace/events/cxl.h | 124 +++++++++++++++++++++++++++++++++++++ 3 files changed, 179 insertions(+), 4 deletions(-)drivers/cxl/core/mbox.c +36
-4@@ -709,6 +709,38 @@ int cxl_enumerate_cmds(struct cxl_dev_state *cxlds) }EXPORT_SYMBOL_NS_GPL (cxl_enumerate_cmds,CXL ); +/* +* General Media Event Record +* CXL rev 3.0 Section 8.2.9.2.1.1; Table 8-43 +*/ +static const uuid_t gen_media_event_uuid = +UUID_INIT (0xfbcd0a77 ,0xc260 ,0x417f , +0x85 ,0xa9 ,0x08 ,0x8b ,0x16 ,0x21 ,0xeb ,0xa6 ); + +static bool cxl_event_tracing_enabled (void ) +{ +return trace_cxl_generic_event_enabled () || +trace_cxl_general_media_enabled (); +} + +static void cxl_trace_event_record (const char *dev_name, +enum cxl_event_log_type type, +struct cxl_event_record_raw *record) +{ +uuid_t *id = &record->hdr.id; + +if (uuid_equal (id, &gen_media_event_uuid)) { +struct cxl_event_gen_media *rec = + (struct cxl_event_gen_media *)record; + +trace_cxl_general_media (dev_name, type, rec); +return ; + } + +/* For unknown record types print just the header */ +trace_cxl_generic_event (dev_name, type, record); +} + static int cxl_clear_event_record(struct cxl_dev_state *cxlds, enum cxl_event_log_type log, struct cxl_get_event_payload *get_pl,@@ -772,11 +804,11 @@ static void cxl_mem_get_records_log(struct cxl_dev_state *cxlds,if (nr_rec >0 ) {int i; -if (trace_cxl_generic_event_enabled ()) { +if (cxl_event_tracing_enabled ()) {for (i =0 ; i < nr_rec; i++) -trace_cxl_generic_event (dev_name (cxlds->dev), - type, - &payload->records[i]); +cxl_trace_event_record (dev_name (cxlds->dev), + type, + &payload->records[i]); } rc =cxl_clear_event_record (cxlds, type, payload, nr_rec);drivers/cxl/cxlmem.h +19
-0@@ -410,6 +410,25 @@ struct cxl_mbox_clear_event_payload {__le16 handle[CXL_CLEAR_EVENT_MAX_HANDLES ]; }; +/* +* General Media Event Record +* CXL rev 3.0 Section 8.2.9.2.1.1; Table 8-43 +*/ +#define CXL_EVENT_GEN_MED_COMP_ID_SIZE 0x10 +struct cxl_event_gen_media { +struct cxl_event_record_hdr hdr; +__le64 phys_addr; +u8 descriptor; +u8 type; +u8 transaction_type; +u8 validity_flags[2 ]; +u8 channel; +u8 rank; +u8 device[3 ]; +u8 component_id[CXL_EVENT_GEN_MED_COMP_ID_SIZE ]; +u8 reserved[0x2e ]; +} __packed; +struct cxl_mbox_get_partition_info {__le64 active_volatile_cap;__le64 active_persistent_cap;include/trace/events/cxl.h +124
-0@@ -118,6 +118,130 @@ TRACE_EVENT(cxl_generic_event, __print_hex(__entry->data, CXL_EVENT_RECORD_DATA_LENGTH)) ); +/* +* Physical Address field masks +* +* General Media Event Record +* CXL rev 3.0 Section 8.2.9.2.1.1; Table 8-43 +* +* DRAM Event Record +* CXL rev 3.0 section 8.2.9.2.1.2; Table 8-44 +*/ +#define CXL_DPA_FLAGS_MASK 0x3F +#define CXL_DPA_MASK (~CXL_DPA_FLAGS_MASK) + +#define CXL_DPA_VOLATILE BIT(0) +#define CXL_DPA_NOT_REPAIRABLE BIT(1) +#define show_dpa_flags (flags) __print_flags(flags, "|", \ + { CXL_DPA_VOLATILE, "VOLATILE" }, \ + { CXL_DPA_NOT_REPAIRABLE, "NOT_REPAIRABLE" } \ +) + +/* +* General Media Event Record - GMER +* CXL rev 3.0 Section 8.2.9.2.1.1; Table 8-43 +*/ +#define CXL_GMER_EVT_DESC_UNCORECTABLE_EVENT BIT(0) +#define CXL_GMER_EVT_DESC_THRESHOLD_EVENT BIT(1) +#define CXL_GMER_EVT_DESC_POISON_LIST_OVERFLOW BIT(2) +#define show_event_desc_flags (flags) __print_flags(flags, "|", \ + { CXL_GMER_EVT_DESC_UNCORECTABLE_EVENT, "UNCORRECTABLE_EVENT" }, \ + { CXL_GMER_EVT_DESC_THRESHOLD_EVENT, "THRESHOLD_EVENT" }, \ + { CXL_GMER_EVT_DESC_POISON_LIST_OVERFLOW, "POISON_LIST_OVERFLOW" } \ +) + +#define CXL_GMER_MEM_EVT_TYPE_ECC_ERROR 0x00 +#define CXL_GMER_MEM_EVT_TYPE_INV_ADDR 0x01 +#define CXL_GMER_MEM_EVT_TYPE_DATA_PATH_ERROR 0x02 +#define show_mem_event_type (type) __print_symbolic(type, \ + { CXL_GMER_MEM_EVT_TYPE_ECC_ERROR, "ECC Error" }, \ + { CXL_GMER_MEM_EVT_TYPE_INV_ADDR, "Invalid Address" }, \ + { CXL_GMER_MEM_EVT_TYPE_DATA_PATH_ERROR, "Data Path Error" } \ +) + +#define CXL_GMER_TRANS_UNKNOWN 0x00 +#define CXL_GMER_TRANS_HOST_READ 0x01 +#define CXL_GMER_TRANS_HOST_WRITE 0x02 +#define CXL_GMER_TRANS_HOST_SCAN_MEDIA 0x03 +#define CXL_GMER_TRANS_HOST_INJECT_POISON 0x04 +#define CXL_GMER_TRANS_INTERNAL_MEDIA_SCRUB 0x05 +#define CXL_GMER_TRANS_INTERNAL_MEDIA_MANAGEMENT 0x06 +#define show_trans_type (type) __print_symbolic(type, \ + { CXL_GMER_TRANS_UNKNOWN, "Unknown" }, \ + { CXL_GMER_TRANS_HOST_READ, "Host Read" }, \ + { CXL_GMER_TRANS_HOST_WRITE, "Host Write" }, \ + { CXL_GMER_TRANS_HOST_SCAN_MEDIA, "Host Scan Media" }, \ + { CXL_GMER_TRANS_HOST_INJECT_POISON, "Host Inject Poison" }, \ + { CXL_GMER_TRANS_INTERNAL_MEDIA_SCRUB, "Internal Media Scrub" }, \ + { CXL_GMER_TRANS_INTERNAL_MEDIA_MANAGEMENT, "Internal Media Management" } \ +) + +#define CXL_GMER_VALID_CHANNEL BIT(0) +#define CXL_GMER_VALID_RANK BIT(1) +#define CXL_GMER_VALID_DEVICE BIT(2) +#define CXL_GMER_VALID_COMPONENT BIT(3) +#define show_valid_flags (flags) __print_flags(flags, "|", \ + { CXL_GMER_VALID_CHANNEL, "CHANNEL" }, \ + { CXL_GMER_VALID_RANK, "RANK" }, \ + { CXL_GMER_VALID_DEVICE, "DEVICE" }, \ + { CXL_GMER_VALID_COMPONENT, "COMPONENT" } \ +) + +TRACE_EVENT (cxl_general_media, + + TP_PROTO(const char *dev_name, enum cxl_event_log_type log, + struct cxl_event_gen_media *rec), + +TP_ARGS (dev_name, log, rec), + +TP_STRUCT__entry ( + CXL_EVT_TP_entry +/* General Media */ + __field(u64, dpa) + __field(u8, descriptor) + __field(u8, type) + __field(u8, transaction_type) + __field(u8, channel) + __field(u32, device) + __array(u8, comp_id, CXL_EVENT_GEN_MED_COMP_ID_SIZE) + __field(u16, validity_flags) +/* Following are out of order to pack trace record */ + __field(u8, rank) +__field (u8, dpa_flags) + ), + +TP_fast_assign ( + CXL_EVT_TP_fast_assign(dev_name, log, rec->hdr); + +/* General Media */ + __entry->dpa = le64_to_cpu(rec->phys_addr); + __entry->dpa_flags = __entry->dpa & CXL_DPA_FLAGS_MASK; + /* Mask after flags have been parsed */ + __entry->dpa &= CXL_DPA_MASK; + __entry->descriptor = rec->descriptor; + __entry->type = rec->type; + __entry->transaction_type = rec->transaction_type; + __entry->channel = rec->channel; + __entry->rank = rec->rank; + __entry->device = get_unaligned_le24(rec->device); + memcpy(__entry->comp_id, &rec->component_id, + CXL_EVENT_GEN_MED_COMP_ID_SIZE); + __entry->validity_flags =get_unaligned_le16 (&rec->validity_flags); + ), + +CXL_EVT_TP_printk ("dpa=%llx dpa_flags='%s' " \ +"descriptor='%s' type='%s' transaction_type='%s' channel=%u rank=%u " \ +"device=%x comp_id=%s validity_flags='%s'" , + __entry->dpa,show_dpa_flags (__entry->dpa_flags), +show_event_desc_flags (__entry->descriptor), +show_mem_event_type (__entry->type), +show_trans_type (__entry->transaction_type), + __entry->channel, __entry->rank, __entry->device, +__print_hex (__entry->comp_id,CXL_EVENT_GEN_MED_COMP_ID_SIZE ), +show_valid_flags (__entry->validity_flags) + ) +); +#endif /* _CXL_TRACE_EVENTS_H */ /* This part must be outside protection */
RE: [PATCH V2 05/11] cxl/mem: Trace General Media Event Record
In reply to [email protected]Quoted 46 lines · ira.weiny@ wrote:
From: Ira Weiny <[email protected]>
CXL rev 3.0 section 8.2.9.2.1.1 defines the General Media Event Record.
Determine if the event read is a general media record and if so trace the record as a General Media Event Record.
Reviewed-by Steven Rostedt (Google) <[email protected]>Reviewed-by Jonathan Cameron <[email protected]>Signed-off-by Ira Weiny <[email protected]>---
Changes from V1:
Jonathan
fix spec references for CXL rev 3.0 Make flags all capsChanges from RFC v2:
Output DPA flags as a single field
Ensure names of fields match whatTP_printoutputs Steven
prefixTRACE_EVENTwith 'cxl_'
Jonathan
Remove Reserved fieldChanges from RFC:
Add reserved byte array
Use common CXL event header record macros Jonathan
Use unaligned_le{24,16} for unaligned fields Don't use the inverse of phy addr mask
Dave Jiang
s/cxl_gen_media_event/general_media s/cxl_evt_gen_media/cxl_event_gen_media--- drivers/cxl/core/mbox.c | 40 ++++++++++-- drivers/cxl/cxlmem.h | 19 ++++++ include/trace/events/cxl.h | 124 +++++++++++++++++++++++++++++++++++++ 3 files changed, 179 insertions(+), 4 deletions(-)drivers/cxl/core/mbox.c +0
-0@@ -709,6 +709,38 @@ int cxl_enumerate_cmds(struct cxl_dev_state *cxlds) }
drivers/cxl/core/mbox.c int cxl_enumerate_cmds(struct cxl_dev_state *cxlds) · line 719
Quoted 10 lines
drivers/cxl/core/mbox.c +8
-0EXPORT_SYMBOL_NS_GPL (cxl_enumerate_cmds,CXL ); +/* +* General Media Event Record +* CXL rev 3.0 Section 8.2.9.2.1.1; Table 8-43 +*/ +static const uuid_t gen_media_event_uuid = +UUID_INIT (0xfbcd0a77 ,0xc260 ,0x417f , +0x85 ,0xa9 ,0x08 ,0x8b ,0x16 ,0x21 ,0xeb ,0xa6 ); +
Move this next to the other UUID_INITs in cxlmem.h.
drivers/cxl/core/mbox.c int cxl_enumerate_cmds(struct cxl_dev_state *cxlds) · line 724
Quoted 5 lines
drivers/cxl/core/mbox.c +5
-0+static bool cxl_event_tracing_enabled (void ) +{ +return trace_cxl_generic_event_enabled () || +trace_cxl_general_media_enabled (); +}
...and now the micro-optimization gets more complicated. The mailbox command is an uncached PCI mmio memcpy(), the incremental cycles this enabled check is saving would be difficult to spot in a profile. So unless it has a worthwhile perf-profile impact I prefer the simplicity of the straight through code.
Quoted 25 lines
drivers/cxl/core/mbox.c +19
-0+ +static void cxl_trace_event_record (const char *dev_name, +enum cxl_event_log_type type, +struct cxl_event_record_raw *record) +{ +uuid_t *id = &record->hdr.id; + +if (uuid_equal (id, &gen_media_event_uuid)) { +struct cxl_event_gen_media *rec = + (struct cxl_event_gen_media *)record; + +trace_cxl_general_media (dev_name, type, rec); +return ; + } + +/* For unknown record types print just the header */ +trace_cxl_generic_event (dev_name, type, record); +} + static int cxl_clear_event_record(struct cxl_dev_state *cxlds, enum cxl_event_log_type log, struct cxl_get_event_payload *get_pl,@@ -772,11 +804,11 @@ static void cxl_mem_get_records_log(struct cxl_dev_state *cxlds,if (nr_rec >0 ) {int i;
drivers/cxl/core/mbox.c static void cxl_mem_get_records_log(struct cxl_dev_state *cxlds, · line 811
Quoted 10 lines
drivers/cxl/core/mbox.c +4
-4-if (trace_cxl_generic_event_enabled ()) { +if (cxl_event_tracing_enabled ()) {for (i =0 ; i < nr_rec; i++) -trace_cxl_generic_event (dev_name (cxlds->dev), - type, - &payload->records[i]); +cxl_trace_event_record (dev_name (cxlds->dev), + type, + &payload->records[i]);
Changing the same lines multiple times in the same series is sometihng that sets off my complexity alarm bells.
Quoted 18 lines
drivers/cxl/core/mbox.c +0
-0} rc =cxl_clear_event_record (cxlds, type, payload, nr_rec);drivers/cxl/cxlmem.h +7
-0@@ -410,6 +410,25 @@ struct cxl_mbox_clear_event_payload {__le16 handle[CXL_CLEAR_EVENT_MAX_HANDLES ]; }; +/* +* General Media Event Record +* CXL rev 3.0 Section 8.2.9.2.1.1; Table 8-43 +*/ +#define CXL_EVENT_GEN_MED_COMP_ID_SIZE 0x10 +struct cxl_event_gen_media { +struct cxl_event_record_hdr hdr;
drivers/cxl/cxlmem.h struct cxl_mbox_clear_event_payload { · line 429
Quoted 10 lines
drivers/cxl/cxlmem.h +10
-0+__le64 phys_addr; +u8 descriptor; +u8 type; +u8 transaction_type; +u8 validity_flags[2 ]; +u8 channel; +u8 rank; +u8 device[3 ]; +u8 component_id[CXL_EVENT_GEN_MED_COMP_ID_SIZE ]; +u8 reserved[0x2e ];
s/0x2e/46/
Quoted 143 lines
drivers/cxl/cxlmem.h +2
-0+} __packed; +struct cxl_mbox_get_partition_info {__le64 active_volatile_cap;__le64 active_persistent_cap;include/trace/events/cxl.h +124
-0@@ -118,6 +118,130 @@ TRACE_EVENT(cxl_generic_event, __print_hex(__entry->data, CXL_EVENT_RECORD_DATA_LENGTH)) ); +/* +* Physical Address field masks +* +* General Media Event Record +* CXL rev 3.0 Section 8.2.9.2.1.1; Table 8-43 +* +* DRAM Event Record +* CXL rev 3.0 section 8.2.9.2.1.2; Table 8-44 +*/ +#define CXL_DPA_FLAGS_MASK 0x3F +#define CXL_DPA_MASK (~CXL_DPA_FLAGS_MASK) + +#define CXL_DPA_VOLATILE BIT(0) +#define CXL_DPA_NOT_REPAIRABLE BIT(1) +#define show_dpa_flags (flags) __print_flags(flags, "|", \ + { CXL_DPA_VOLATILE, "VOLATILE" }, \ + { CXL_DPA_NOT_REPAIRABLE, "NOT_REPAIRABLE" } \ +) + +/* +* General Media Event Record - GMER +* CXL rev 3.0 Section 8.2.9.2.1.1; Table 8-43 +*/ +#define CXL_GMER_EVT_DESC_UNCORECTABLE_EVENT BIT(0) +#define CXL_GMER_EVT_DESC_THRESHOLD_EVENT BIT(1) +#define CXL_GMER_EVT_DESC_POISON_LIST_OVERFLOW BIT(2) +#define show_event_desc_flags (flags) __print_flags(flags, "|", \ + { CXL_GMER_EVT_DESC_UNCORECTABLE_EVENT, "UNCORRECTABLE_EVENT" }, \ + { CXL_GMER_EVT_DESC_THRESHOLD_EVENT, "THRESHOLD_EVENT" }, \ + { CXL_GMER_EVT_DESC_POISON_LIST_OVERFLOW, "POISON_LIST_OVERFLOW" } \ +) + +#define CXL_GMER_MEM_EVT_TYPE_ECC_ERROR 0x00 +#define CXL_GMER_MEM_EVT_TYPE_INV_ADDR 0x01 +#define CXL_GMER_MEM_EVT_TYPE_DATA_PATH_ERROR 0x02 +#define show_mem_event_type (type) __print_symbolic(type, \ + { CXL_GMER_MEM_EVT_TYPE_ECC_ERROR, "ECC Error" }, \ + { CXL_GMER_MEM_EVT_TYPE_INV_ADDR, "Invalid Address" }, \ + { CXL_GMER_MEM_EVT_TYPE_DATA_PATH_ERROR, "Data Path Error" } \ +) + +#define CXL_GMER_TRANS_UNKNOWN 0x00 +#define CXL_GMER_TRANS_HOST_READ 0x01 +#define CXL_GMER_TRANS_HOST_WRITE 0x02 +#define CXL_GMER_TRANS_HOST_SCAN_MEDIA 0x03 +#define CXL_GMER_TRANS_HOST_INJECT_POISON 0x04 +#define CXL_GMER_TRANS_INTERNAL_MEDIA_SCRUB 0x05 +#define CXL_GMER_TRANS_INTERNAL_MEDIA_MANAGEMENT 0x06 +#define show_trans_type (type) __print_symbolic(type, \ + { CXL_GMER_TRANS_UNKNOWN, "Unknown" }, \ + { CXL_GMER_TRANS_HOST_READ, "Host Read" }, \ + { CXL_GMER_TRANS_HOST_WRITE, "Host Write" }, \ + { CXL_GMER_TRANS_HOST_SCAN_MEDIA, "Host Scan Media" }, \ + { CXL_GMER_TRANS_HOST_INJECT_POISON, "Host Inject Poison" }, \ + { CXL_GMER_TRANS_INTERNAL_MEDIA_SCRUB, "Internal Media Scrub" }, \ + { CXL_GMER_TRANS_INTERNAL_MEDIA_MANAGEMENT, "Internal Media Management" } \ +) + +#define CXL_GMER_VALID_CHANNEL BIT(0) +#define CXL_GMER_VALID_RANK BIT(1) +#define CXL_GMER_VALID_DEVICE BIT(2) +#define CXL_GMER_VALID_COMPONENT BIT(3) +#define show_valid_flags (flags) __print_flags(flags, "|", \ + { CXL_GMER_VALID_CHANNEL, "CHANNEL" }, \ + { CXL_GMER_VALID_RANK, "RANK" }, \ + { CXL_GMER_VALID_DEVICE, "DEVICE" }, \ + { CXL_GMER_VALID_COMPONENT, "COMPONENT" } \ +) + +TRACE_EVENT (cxl_general_media, + + TP_PROTO(const char *dev_name, enum cxl_event_log_type log, + struct cxl_event_gen_media *rec), + +TP_ARGS (dev_name, log, rec), + +TP_STRUCT__entry ( + CXL_EVT_TP_entry +/* General Media */ + __field(u64, dpa) + __field(u8, descriptor) + __field(u8, type) + __field(u8, transaction_type) + __field(u8, channel) + __field(u32, device) + __array(u8, comp_id, CXL_EVENT_GEN_MED_COMP_ID_SIZE) + __field(u16, validity_flags) +/* Following are out of order to pack trace record */ + __field(u8, rank) +__field (u8, dpa_flags) + ), + +TP_fast_assign ( + CXL_EVT_TP_fast_assign(dev_name, log, rec->hdr); + +/* General Media */ + __entry->dpa = le64_to_cpu(rec->phys_addr); + __entry->dpa_flags = __entry->dpa & CXL_DPA_FLAGS_MASK; + /* Mask after flags have been parsed */ + __entry->dpa &= CXL_DPA_MASK; + __entry->descriptor = rec->descriptor; + __entry->type = rec->type; + __entry->transaction_type = rec->transaction_type; + __entry->channel = rec->channel; + __entry->rank = rec->rank; + __entry->device = get_unaligned_le24(rec->device); + memcpy(__entry->comp_id, &rec->component_id, + CXL_EVENT_GEN_MED_COMP_ID_SIZE); + __entry->validity_flags =get_unaligned_le16 (&rec->validity_flags); + ), + +CXL_EVT_TP_printk ("dpa=%llx dpa_flags='%s' " \ +"descriptor='%s' type='%s' transaction_type='%s' channel=%u rank=%u " \ +"device=%x comp_id=%s validity_flags='%s'" , + __entry->dpa,show_dpa_flags (__entry->dpa_flags), +show_event_desc_flags (__entry->descriptor), +show_mem_event_type (__entry->type), +show_trans_type (__entry->transaction_type), + __entry->channel, __entry->rank, __entry->device, +__print_hex (__entry->comp_id,CXL_EVENT_GEN_MED_COMP_ID_SIZE ), +show_valid_flags (__entry->validity_flags) + ) +); +#endif /* _CXL_TRACE_EVENTS_H */ /* This part must be outside protection */ Signature · 1 line
2.37.2
PATCH v2 6/11 cxl/mem: Trace DRAM Event Record
In reply to [email protected]From: Ira Weiny <[email protected]>
CXL rev 3.0 section 8.2.9.2.1.2 defines the DRAM Event Record.
Determine if the event read is a DRAM event record and if so trace the record.
---
Changes from RFC v2:
Output DPA flags as a separate field. Ensure field names match TP_print output Steven
prefix TRACE_EVENT with 'cxl_'
Jonathan
Formatting fix
Remove reserved field
Changes from RFC:
Add reserved byte data
Use new CXL header macros
Jonathan
Use get_unaligned_le{24,16}() for unaligned fields Use 'else if'
Dave Jiang
s/cxl_dram_event/dram s/cxl_evt_dram_rec/cxl_event_dram
Adjust for new phys addr mask
--- drivers/cxl/core/mbox.c | 16 ++++++- drivers/cxl/cxlmem.h | 23 ++++++++++ include/trace/events/cxl.h | 92 ++++++++++++++++++++++++++++++++++++++ 3 files changed, 130 insertions(+), 1 deletion(-)
drivers/cxl/core/mbox.c +15 -1
@@ -717,10 +717,19 @@ static const uuid_t gen_media_event_uuid =UUID_INIT (0xfbcd0a77 ,0xc260 ,0x417f ,0x85 ,0xa9 ,0x08 ,0x8b ,0x16 ,0x21 ,0xeb ,0xa6 ); +/* +* DRAM Event Record +* CXL rev 3.0 section 8.2.9.2.1.2; Table 8-44 +*/ +static const uuid_t dram_event_uuid = +UUID_INIT (0x601dcbb3 ,0x9c06 ,0x4eab , +0xb8 ,0xaf ,0x4e ,0x9b ,0xfb ,0x5c ,0x96 ,0x24 ); +static bool cxl_event_tracing_enabled (void ) {return trace_cxl_generic_event_enabled () || -trace_cxl_general_media_enabled (); +trace_cxl_general_media_enabled () || +trace_cxl_dram_enabled (); } static void cxl_trace_event_record(const char *dev_name,
@@ -735,6 +744,11 @@ static void cxl_trace_event_record(const char *dev_name,trace_cxl_general_media (dev_name, type, rec);return ; + }else if (uuid_equal (id, &dram_event_uuid)) { +struct cxl_event_dram *rec = (struct cxl_event_dram *)record; + +trace_cxl_dram (dev_name, type, rec); +return ; }/* For unknown record types print just the header */
drivers/cxl/cxlmem.h +23 -0
@@ -429,6 +429,29 @@ struct cxl_event_gen_media {u8 reserved[0x2e ]; } __packed; +/* +* DRAM Event Record - DER +* CXL rev 3.0 section 8.2.9.2.1.2; Table 3-44 +*/ +#define CXL_EVENT_DER_CORRECTION_MASK_SIZE 0x20 +struct cxl_event_dram { +struct cxl_event_record_hdr hdr; +__le64 phys_addr; +u8 descriptor; +u8 type; +u8 transaction_type; +u8 validity_flags[2 ]; +u8 channel; +u8 rank; +u8 nibble_mask[3 ]; +u8 bank_group; +u8 bank; +u8 row[3 ]; +u8 column[2 ]; +u8 correction_mask[CXL_EVENT_DER_CORRECTION_MASK_SIZE ]; +u8 reserved[0x17 ]; +} __packed; +struct cxl_mbox_get_partition_info {__le64 active_volatile_cap;__le64 active_persistent_cap;
include/trace/events/cxl.h +92 -0
98 lines not on this page · raw message
Signature · 1 line
2.37.2
Re: [PATCH V2 06/11] cxl/mem: Trace DRAM Event Record
Quoted 11 lines · On 11/30/2022 5:27 PM, [email protected] wrote:
From: Ira Weiny <[email protected]>
CXL rev 3.0 section 8.2.9.2.1.2 defines the DRAM Event Record.
Determine if the event read is a DRAM event record and if so trace the record.
Reviewed-by Steven Rostedt (Google) <[email protected]>Reviewed-by Jonathan Cameron <[email protected]>Reviewed-by Dave Jiang <[email protected]>Signed-off-by Ira Weiny <[email protected]>
Quoted 201 lines
---
Changes from RFC v2:
Output DPA flags as a separate field. Ensure field names matchTP_printoutput Steven
prefixTRACE_EVENTwith 'cxl_'
Jonathan
Formatting fix
Remove reserved fieldChanges from RFC:
Add reserved byte data
Use new CXL header macros
Jonathan
Use get_unaligned_le{24,16}() for unaligned fields Use 'else if'
Dave Jiang
s/cxl_dram_event/dram s/cxl_evt_dram_rec/cxl_event_dram
Adjust for new phys addr mask--- drivers/cxl/core/mbox.c | 16 ++++++- drivers/cxl/cxlmem.h | 23 ++++++++++ include/trace/events/cxl.h | 92 ++++++++++++++++++++++++++++++++++++++ 3 files changed, 130 insertions(+), 1 deletion(-)drivers/cxl/core/mbox.c +15
-1@@ -717,10 +717,19 @@ static const uuid_t gen_media_event_uuid =UUID_INIT (0xfbcd0a77 ,0xc260 ,0x417f ,0x85 ,0xa9 ,0x08 ,0x8b ,0x16 ,0x21 ,0xeb ,0xa6 ); +/* +* DRAM Event Record +* CXL rev 3.0 section 8.2.9.2.1.2; Table 8-44 +*/ +static const uuid_t dram_event_uuid = +UUID_INIT (0x601dcbb3 ,0x9c06 ,0x4eab , +0xb8 ,0xaf ,0x4e ,0x9b ,0xfb ,0x5c ,0x96 ,0x24 ); +static bool cxl_event_tracing_enabled (void ) {return trace_cxl_generic_event_enabled () || -trace_cxl_general_media_enabled (); +trace_cxl_general_media_enabled () || +trace_cxl_dram_enabled (); } static void cxl_trace_event_record(const char *dev_name,@@ -735,6 +744,11 @@ static void cxl_trace_event_record(const char *dev_name,trace_cxl_general_media (dev_name, type, rec);return ; + }else if (uuid_equal (id, &dram_event_uuid)) { +struct cxl_event_dram *rec = (struct cxl_event_dram *)record; + +trace_cxl_dram (dev_name, type, rec); +return ; }/* For unknown record types print just the header */ drivers/cxl/cxlmem.h +23
-0@@ -429,6 +429,29 @@ struct cxl_event_gen_media {u8 reserved[0x2e ]; } __packed; +/* +* DRAM Event Record - DER +* CXL rev 3.0 section 8.2.9.2.1.2; Table 3-44 +*/ +#define CXL_EVENT_DER_CORRECTION_MASK_SIZE 0x20 +struct cxl_event_dram { +struct cxl_event_record_hdr hdr; +__le64 phys_addr; +u8 descriptor; +u8 type; +u8 transaction_type; +u8 validity_flags[2 ]; +u8 channel; +u8 rank; +u8 nibble_mask[3 ]; +u8 bank_group; +u8 bank; +u8 row[3 ]; +u8 column[2 ]; +u8 correction_mask[CXL_EVENT_DER_CORRECTION_MASK_SIZE ]; +u8 reserved[0x17 ]; +} __packed; +struct cxl_mbox_get_partition_info {__le64 active_volatile_cap;__le64 active_persistent_cap;include/trace/events/cxl.h +92
-0@@ -242,6 +242,98 @@ TRACE_EVENT(cxl_general_media, ) ); +/* +* DRAM Event Record - DER +* +* CXL rev 3.0 section 8.2.9.2.1.2; Table 8-44 +*/ +/* +* DRAM Event Record defines many fields the same as the General Media Event +* Record. Reuse those definitions as appropriate. +*/ +#define CXL_DER_VALID_CHANNEL BIT(0) +#define CXL_DER_VALID_RANK BIT(1) +#define CXL_DER_VALID_NIBBLE BIT(2) +#define CXL_DER_VALID_BANK_GROUP BIT(3) +#define CXL_DER_VALID_BANK BIT(4) +#define CXL_DER_VALID_ROW BIT(5) +#define CXL_DER_VALID_COLUMN BIT(6) +#define CXL_DER_VALID_CORRECTION_MASK BIT(7) +#define show_dram_valid_flags (flags) __print_flags(flags, "|", \ + { CXL_DER_VALID_CHANNEL, "CHANNEL" }, \ + { CXL_DER_VALID_RANK, "RANK" }, \ + { CXL_DER_VALID_NIBBLE, "NIBBLE" }, \ + { CXL_DER_VALID_BANK_GROUP, "BANK GROUP" }, \ + { CXL_DER_VALID_BANK, "BANK" }, \ + { CXL_DER_VALID_ROW, "ROW" }, \ + { CXL_DER_VALID_COLUMN, "COLUMN" }, \ + { CXL_DER_VALID_CORRECTION_MASK, "CORRECTION MASK" } \ +) + +TRACE_EVENT (cxl_dram, + + TP_PROTO(const char *dev_name, enum cxl_event_log_type log, + struct cxl_event_dram *rec), + +TP_ARGS (dev_name, log, rec), + +TP_STRUCT__entry ( + CXL_EVT_TP_entry +/* DRAM */ + __field(u64, dpa) + __field(u8, descriptor) + __field(u8, type) + __field(u8, transaction_type) + __field(u8, channel) + __field(u16, validity_flags) + __field(u16, column) /* Out of order to pack trace record */ + __field(u32, nibble_mask) + __field(u32, row) + __array(u8, cor_mask, CXL_EVENT_DER_CORRECTION_MASK_SIZE) + __field(u8, rank) /* Out of order to pack trace record */ + __field(u8, bank_group) /* Out of order to pack trace record */ + __field(u8, bank) /* Out of order to pack trace record */ +__field (u8, dpa_flags)/* Out of order to pack trace record */ + ), + +TP_fast_assign ( + CXL_EVT_TP_fast_assign(dev_name, log, rec->hdr); + +/* DRAM */ + __entry->dpa = le64_to_cpu(rec->phys_addr); + __entry->dpa_flags = __entry->dpa & CXL_DPA_FLAGS_MASK; + __entry->dpa &= CXL_DPA_MASK; + __entry->descriptor = rec->descriptor; + __entry->type = rec->type; + __entry->transaction_type = rec->transaction_type; + __entry->validity_flags = get_unaligned_le16(rec->validity_flags); + __entry->channel = rec->channel; + __entry->rank = rec->rank; + __entry->nibble_mask = get_unaligned_le24(rec->nibble_mask); + __entry->bank_group = rec->bank_group; + __entry->bank = rec->bank; + __entry->row = get_unaligned_le24(rec->row); + __entry->column = get_unaligned_le16(rec->column); + memcpy(__entry->cor_mask, &rec->correction_mask, + CXL_EVENT_DER_CORRECTION_MASK_SIZE); + ), + +CXL_EVT_TP_printk ("dpa=%llx dpa_flags='%s' descriptor='%s' type='%s' " \ +"transaction_type='%s' channel=%u rank=%u nibble_mask=%x " \ +"bank_group=%u bank=%u row=%u column=%u cor_mask=%s " \ +"validity_flags='%s'" , + __entry->dpa,show_dpa_flags (__entry->dpa_flags), +show_event_desc_flags (__entry->descriptor), +show_mem_event_type (__entry->type), +show_trans_type (__entry->transaction_type), + __entry->channel, __entry->rank, __entry->nibble_mask, + __entry->bank_group, __entry->bank, + __entry->row, __entry->column, +__print_hex (__entry->cor_mask,CXL_EVENT_DER_CORRECTION_MASK_SIZE ), +show_dram_valid_flags (__entry->validity_flags) + ) +); +#endif /* _CXL_TRACE_EVENTS_H */ /* This part must be outside protection */
PATCH v2 7/11 cxl/mem: Trace Memory Module Event Record
In reply to [email protected]From: Ira Weiny <[email protected]>
CXL rev 3.0 section 8.2.9.2.1.3 defines the Memory Module Event Record.
Determine if the event read is memory module record and if so trace the record.
---
Changes from V1:
Use all caps for flag fields
Changes from RFC v2:
Ensure field names match TP_print output Steven
prefix TRACE_EVENT with 'cxl_'
Jonathan
Remove reserved field
Define a 1bit and 2 bit status decoder Fix paren alignment
Changes from RFC:
Clean up spec reference
Add reserved data
Use new CXL header macros
Jonathan
Use else if
Use get_unaligned_le*() for unaligned fields
Dave Jiang
s/cxl_mem_mod_event/memory_module s/cxl_evt_mem_mod_rec/cxl_event_mem_module
--- drivers/cxl/core/mbox.c | 17 ++++- drivers/cxl/cxlmem.h | 26 +++++++ include/trace/events/cxl.h | 144 +++++++++++++++++++++++++++++++++++++ 3 files changed, 186 insertions(+), 1 deletion(-)
drivers/cxl/core/mbox.c +16 -1
@@ -725,11 +725,20 @@ static const uuid_t dram_event_uuid =UUID_INIT (0x601dcbb3 ,0x9c06 ,0x4eab ,0xb8 ,0xaf ,0x4e ,0x9b ,0xfb ,0x5c ,0x96 ,0x24 ); +/* +* Memory Module Event Record +* CXL rev 3.0 section 8.2.9.2.1.3; Table 8-45 +*/ +static const uuid_t mem_mod_event_uuid = +UUID_INIT (0xfe927475 ,0xdd59 ,0x4339 , +0xa5 ,0x86 ,0x79 ,0xba ,0xb1 ,0x13 ,0xb7 ,0x74 ); +static bool cxl_event_tracing_enabled (void ) {return trace_cxl_generic_event_enabled () ||trace_cxl_general_media_enabled () || -trace_cxl_dram_enabled (); +trace_cxl_dram_enabled () || +trace_cxl_memory_module_enabled (); } static void cxl_trace_event_record(const char *dev_name,
@@ -749,6 +758,12 @@ static void cxl_trace_event_record(const char *dev_name,trace_cxl_dram (dev_name, type, rec);return ; + }else if (uuid_equal (id, &mem_mod_event_uuid)) { +struct cxl_event_mem_module *rec = + (struct cxl_event_mem_module *)record; + +trace_cxl_memory_module (dev_name, type, rec); +return ; }/* For unknown record types print just the header */
drivers/cxl/cxlmem.h +26 -0
@@ -452,6 +452,32 @@ struct cxl_event_dram {u8 reserved[0x17 ]; } __packed; +/* +* Get Health Info Record +* CXL rev 3.0 section 8.2.9.8.3.1; Table 8-100 +*/ +struct cxl_get_health_info { +u8 health_status; +u8 media_status; +u8 add_status; +u8 life_used; +u8 device_temp[2 ]; +u8 dirty_shutdown_cnt[4 ]; +u8 cor_vol_err_cnt[4 ]; +u8 cor_per_err_cnt[4 ]; +} __packed; + +/* +* Memory Module Event Record +* CXL rev 3.0 section 8.2.9.2.1.3; Table 8-45 +*/ +struct cxl_event_mem_module { +struct cxl_event_record_hdr hdr; +u8 event_type; +struct cxl_get_health_info info; +u8 reserved[0x3d ]; +} __packed; +struct cxl_mbox_get_partition_info {__le64 active_volatile_cap;__le64 active_persistent_cap;
include/trace/events/cxl.h +144 -0
150 lines not on this page · raw message
Signature · 1 line
2.37.2
Re: [PATCH V2 07/11] cxl/mem: Trace Memory Module Event Record
Quoted 9 lines · On 11/30/2022 5:27 PM, [email protected] wrote:
From: Ira Weiny <[email protected]>
CXL rev 3.0 section 8.2.9.2.1.3 defines the Memory Module Event Record.
Determine if the event read is memory module record and if so trace the record.
Reviewed-by Steven Rostedt (Google) <[email protected]>Signed-off-by Ira Weiny <[email protected]>
Quoted 261 lines
---
Changes from V1:
Use all caps for flag fieldsChanges from RFC v2:
Ensure field names matchTP_printoutput Steven
prefixTRACE_EVENTwith 'cxl_'
Jonathan
Remove reserved field
Define a 1bit and 2 bit status decoder Fix paren alignmentChanges from RFC:
Clean up spec reference
Add reserved data
Use new CXL header macros
Jonathan
Use else if
Use get_unaligned_le*() for unaligned fields
Dave Jiang
s/cxl_mem_mod_event/memory_module s/cxl_evt_mem_mod_rec/cxl_event_mem_module--- drivers/cxl/core/mbox.c | 17 ++++- drivers/cxl/cxlmem.h | 26 +++++++ include/trace/events/cxl.h | 144 +++++++++++++++++++++++++++++++++++++ 3 files changed, 186 insertions(+), 1 deletion(-)drivers/cxl/core/mbox.c +16
-1@@ -725,11 +725,20 @@ static const uuid_t dram_event_uuid =UUID_INIT (0x601dcbb3 ,0x9c06 ,0x4eab ,0xb8 ,0xaf ,0x4e ,0x9b ,0xfb ,0x5c ,0x96 ,0x24 ); +/* +* Memory Module Event Record +* CXL rev 3.0 section 8.2.9.2.1.3; Table 8-45 +*/ +static const uuid_t mem_mod_event_uuid = +UUID_INIT (0xfe927475 ,0xdd59 ,0x4339 , +0xa5 ,0x86 ,0x79 ,0xba ,0xb1 ,0x13 ,0xb7 ,0x74 ); +static bool cxl_event_tracing_enabled (void ) {return trace_cxl_generic_event_enabled () ||trace_cxl_general_media_enabled () || -trace_cxl_dram_enabled (); +trace_cxl_dram_enabled () || +trace_cxl_memory_module_enabled (); } static void cxl_trace_event_record(const char *dev_name,@@ -749,6 +758,12 @@ static void cxl_trace_event_record(const char *dev_name,trace_cxl_dram (dev_name, type, rec);return ; + }else if (uuid_equal (id, &mem_mod_event_uuid)) { +struct cxl_event_mem_module *rec = + (struct cxl_event_mem_module *)record; + +trace_cxl_memory_module (dev_name, type, rec); +return ; }/* For unknown record types print just the header */ drivers/cxl/cxlmem.h +26
-0@@ -452,6 +452,32 @@ struct cxl_event_dram {u8 reserved[0x17 ]; } __packed; +/* +* Get Health Info Record +* CXL rev 3.0 section 8.2.9.8.3.1; Table 8-100 +*/ +struct cxl_get_health_info { +u8 health_status; +u8 media_status; +u8 add_status; +u8 life_used; +u8 device_temp[2 ]; +u8 dirty_shutdown_cnt[4 ]; +u8 cor_vol_err_cnt[4 ]; +u8 cor_per_err_cnt[4 ]; +} __packed; + +/* +* Memory Module Event Record +* CXL rev 3.0 section 8.2.9.2.1.3; Table 8-45 +*/ +struct cxl_event_mem_module { +struct cxl_event_record_hdr hdr; +u8 event_type; +struct cxl_get_health_info info; +u8 reserved[0x3d ]; +} __packed; +struct cxl_mbox_get_partition_info {__le64 active_volatile_cap;__le64 active_persistent_cap;include/trace/events/cxl.h +144
-0@@ -334,6 +334,150 @@ TRACE_EVENT(cxl_dram, ) ); +/* +* Memory Module Event Record - MMER +* +* CXL res 3.0 section 8.2.9.2.1.3; Table 8-45 +*/ +#define CXL_MMER_HEALTH_STATUS_CHANGE 0x00 +#define CXL_MMER_MEDIA_STATUS_CHANGE 0x01 +#define CXL_MMER_LIFE_USED_CHANGE 0x02 +#define CXL_MMER_TEMP_CHANGE 0x03 +#define CXL_MMER_DATA_PATH_ERROR 0x04 +#define CXL_MMER_LAS_ERROR 0x05 +#define show_dev_evt_type (type) __print_symbolic(type, \ + { CXL_MMER_HEALTH_STATUS_CHANGE, "Health Status Change" }, \ + { CXL_MMER_MEDIA_STATUS_CHANGE, "Media Status Change" }, \ + { CXL_MMER_LIFE_USED_CHANGE, "Life Used Change" }, \ + { CXL_MMER_TEMP_CHANGE, "Temperature Change" }, \ + { CXL_MMER_DATA_PATH_ERROR, "Data Path Error" }, \ + { CXL_MMER_LAS_ERROR, "LSA Error" } \ +) + +/* +* Device Health Information - DHI +* +* CXL res 3.0 section 8.2.9.8.3.1; Table 8-100 +*/ +#define CXL_DHI_HS_MAINTENANCE_NEEDED BIT(0) +#define CXL_DHI_HS_PERFORMANCE_DEGRADED BIT(1) +#define CXL_DHI_HS_HW_REPLACEMENT_NEEDED BIT(2) +#define show_health_status_flags (flags) __print_flags(flags, "|", \ + { CXL_DHI_HS_MAINTENANCE_NEEDED, "MAINTENANCE_NEEDED" }, \ + { CXL_DHI_HS_PERFORMANCE_DEGRADED, "PERFORMANCE_DEGRADED" }, \ + { CXL_DHI_HS_HW_REPLACEMENT_NEEDED, "REPLACEMENT_NEEDED" } \ +) + +#define CXL_DHI_MS_NORMAL 0x00 +#define CXL_DHI_MS_NOT_READY 0x01 +#define CXL_DHI_MS_WRITE_PERSISTENCY_LOST 0x02 +#define CXL_DHI_MS_ALL_DATA_LOST 0x03 +#define CXL_DHI_MS_WRITE_PERSISTENCY_LOSS_EVENT_POWER_LOSS 0x04 +#define CXL_DHI_MS_WRITE_PERSISTENCY_LOSS_EVENT_SHUTDOWN 0x05 +#define CXL_DHI_MS_WRITE_PERSISTENCY_LOSS_IMMINENT 0x06 +#define CXL_DHI_MS_WRITE_ALL_DATA_LOSS_EVENT_POWER_LOSS 0x07 +#define CXL_DHI_MS_WRITE_ALL_DATA_LOSS_EVENT_SHUTDOWN 0x08 +#define CXL_DHI_MS_WRITE_ALL_DATA_LOSS_IMMINENT 0x09 +#define show_media_status (ms) __print_symbolic(ms, \ + { CXL_DHI_MS_NORMAL, \ + "Normal" }, \ + { CXL_DHI_MS_NOT_READY, \ + "Not Ready" }, \ + { CXL_DHI_MS_WRITE_PERSISTENCY_LOST, \ + "Write Persistency Lost" }, \ + { CXL_DHI_MS_ALL_DATA_LOST, \ + "All Data Lost" }, \ + { CXL_DHI_MS_WRITE_PERSISTENCY_LOSS_EVENT_POWER_LOSS, \ + "Write Persistency Loss in the Event of Power Loss" }, \ + { CXL_DHI_MS_WRITE_PERSISTENCY_LOSS_EVENT_SHUTDOWN, \ + "Write Persistency Loss in Event of Shutdown" }, \ + { CXL_DHI_MS_WRITE_PERSISTENCY_LOSS_IMMINENT, \ + "Write Persistency Loss Imminent" }, \ + { CXL_DHI_MS_WRITE_ALL_DATA_LOSS_EVENT_POWER_LOSS, \ + "All Data Loss in Event of Power Loss" }, \ + { CXL_DHI_MS_WRITE_ALL_DATA_LOSS_EVENT_SHUTDOWN, \ + "All Data loss in the Event of Shutdown" }, \ + { CXL_DHI_MS_WRITE_ALL_DATA_LOSS_IMMINENT, \ + "All Data Loss Imminent" } \ +) + +#define CXL_DHI_AS_NORMAL 0x0 +#define CXL_DHI_AS_WARNING 0x1 +#define CXL_DHI_AS_CRITICAL 0x2 +#define show_two_bit_status (as) __print_symbolic(as, \ + { CXL_DHI_AS_NORMAL, "Normal" }, \ + { CXL_DHI_AS_WARNING, "Warning" }, \ + { CXL_DHI_AS_CRITICAL, "Critical" } \ +) +#define show_one_bit_status (as) __print_symbolic(as, \ + { CXL_DHI_AS_NORMAL, "Normal" }, \ + { CXL_DHI_AS_WARNING, "Warning" } \ +) + +#define CXL_DHI_AS_LIFE_USED (as) (as & 0x3) +#define CXL_DHI_AS_DEV_TEMP (as) ((as & 0xC) >> 2) +#define CXL_DHI_AS_COR_VOL_ERR_CNT (as) ((as & 0x10) >> 4) +#define CXL_DHI_AS_COR_PER_ERR_CNT (as) ((as & 0x20) >> 5) + +TRACE_EVENT (cxl_memory_module, + + TP_PROTO(const char *dev_name, enum cxl_event_log_type log, + struct cxl_event_mem_module *rec), + +TP_ARGS (dev_name, log, rec), + +TP_STRUCT__entry ( + CXL_EVT_TP_entry + +/* Memory Module Event */ + __field(u8, event_type) + +/* Device Health Info */ + __field(u8, health_status) + __field(u8, media_status) + __field(u8, life_used) + __field(u32, dirty_shutdown_cnt) + __field(u32, cor_vol_err_cnt) + __field(u32, cor_per_err_cnt) + __field(s16, device_temp) +__field (u8, add_status) + ), + +TP_fast_assign ( + CXL_EVT_TP_fast_assign(dev_name, log, rec->hdr); + +/* Memory Module Event */ + __entry->event_type = rec->event_type; + +/* Device Health Info */ + __entry->health_status = rec->info.health_status; + __entry->media_status = rec->info.media_status; + __entry->life_used = rec->info.life_used; + __entry->dirty_shutdown_cnt = get_unaligned_le32(rec->info.dirty_shutdown_cnt); + __entry->cor_vol_err_cnt = get_unaligned_le32(rec->info.cor_vol_err_cnt); + __entry->cor_per_err_cnt = get_unaligned_le32(rec->info.cor_per_err_cnt); + __entry->device_temp = get_unaligned_le16(rec->info.device_temp); + __entry->add_status = rec->info.add_status; + ), + +CXL_EVT_TP_printk ("event_type='%s' health_status='%s' media_status='%s' " \ +"as_life_used=%s as_dev_temp=%s as_cor_vol_err_cnt=%s " \ +"as_cor_per_err_cnt=%s life_used=%u device_temp=%d " \ +"dirty_shutdown_cnt=%u cor_vol_err_cnt=%u cor_per_err_cnt=%u" , +show_dev_evt_type (__entry->event_type), +show_health_status_flags (__entry->health_status), +show_media_status (__entry->media_status), +show_two_bit_status (CXL_DHI_AS_LIFE_USED (__entry->add_status)), +show_two_bit_status (CXL_DHI_AS_DEV_TEMP (__entry->add_status)), +show_one_bit_status (CXL_DHI_AS_COR_VOL_ERR_CNT (__entry->add_status)), +show_one_bit_status (CXL_DHI_AS_COR_PER_ERR_CNT (__entry->add_status)), + __entry->life_used, __entry->device_temp, + __entry->dirty_shutdown_cnt, __entry->cor_vol_err_cnt, + __entry->cor_per_err_cnt + ) +); + +#endif /* _CXL_TRACE_EVENTS_H */ /* This part must be outside protection */
RE: [PATCH V2 07/11] cxl/mem: Trace Memory Module Event Record
In reply to [email protected]Quoted 66 lines · ira.weiny@ wrote:
From: Ira Weiny <[email protected]>
CXL rev 3.0 section 8.2.9.2.1.3 defines the Memory Module Event Record.
Determine if the event read is memory module record and if so trace the record.
Reviewed-by Steven Rostedt (Google) <[email protected]>Signed-off-by Ira Weiny <[email protected]>---
Changes from V1:
Use all caps for flag fieldsChanges from RFC v2:
Ensure field names matchTP_printoutput Steven
prefixTRACE_EVENTwith 'cxl_'
Jonathan
Remove reserved field
Define a 1bit and 2 bit status decoder Fix paren alignmentChanges from RFC:
Clean up spec reference
Add reserved data
Use new CXL header macros
Jonathan
Use else if
Use get_unaligned_le*() for unaligned fields
Dave Jiang
s/cxl_mem_mod_event/memory_module s/cxl_evt_mem_mod_rec/cxl_event_mem_module--- drivers/cxl/core/mbox.c | 17 ++++- drivers/cxl/cxlmem.h | 26 +++++++ include/trace/events/cxl.h | 144 +++++++++++++++++++++++++++++++++++++ 3 files changed, 186 insertions(+), 1 deletion(-)drivers/cxl/core/mbox.c +10
-1@@ -725,11 +725,20 @@ static const uuid_t dram_event_uuid =UUID_INIT (0x601dcbb3 ,0x9c06 ,0x4eab ,0xb8 ,0xaf ,0x4e ,0x9b ,0xfb ,0x5c ,0x96 ,0x24 ); +/* +* Memory Module Event Record +* CXL rev 3.0 section 8.2.9.2.1.3; Table 8-45 +*/ +static const uuid_t mem_mod_event_uuid = +UUID_INIT (0xfe927475 ,0xdd59 ,0x4339 , +0xa5 ,0x86 ,0x79 ,0xba ,0xb1 ,0x13 ,0xb7 ,0x74 ); +static bool cxl_event_tracing_enabled (void ) {return trace_cxl_generic_event_enabled () ||trace_cxl_general_media_enabled () || -trace_cxl_dram_enabled (); +trace_cxl_dram_enabled () || +trace_cxl_memory_module_enabled (); } static void cxl_trace_event_record(const char *dev_name,
drivers/cxl/core/mbox.c static void cxl_trace_event_record(const char *dev_name, · line 766
Quoted 9 lines
drivers/cxl/core/mbox.c +6
-0@@ -749,6 +758,12 @@ static void cxl_trace_event_record(const char *dev_name,trace_cxl_dram (dev_name, type, rec);return ; + }else if (uuid_equal (id, &mem_mod_event_uuid)) { +struct cxl_event_mem_module *rec = + (struct cxl_event_mem_module *)record; + +trace_cxl_memory_module (dev_name, type, rec); +return ;
Replace these early returns with a final else that calls trace_cxl_generic_event()
Quoted 25 lines
drivers/cxl/core/mbox.c +0
-0}/* For unknown record types print just the header */ drivers/cxl/cxlmem.h +14
-0@@ -452,6 +452,32 @@ struct cxl_event_dram {u8 reserved[0x17 ]; } __packed; +/* +* Get Health Info Record +* CXL rev 3.0 section 8.2.9.8.3.1; Table 8-100 +*/ +struct cxl_get_health_info { +u8 health_status; +u8 media_status; +u8 add_status; +u8 life_used; +u8 device_temp[2 ]; +u8 dirty_shutdown_cnt[4 ]; +u8 cor_vol_err_cnt[4 ]; +u8 cor_per_err_cnt[4 ]; +} __packed;
drivers/cxl/cxlmem.h struct cxl_event_dram { · line 478
Quoted 10 lines
drivers/cxl/cxlmem.h +10
-0+ +/* +* Memory Module Event Record +* CXL rev 3.0 section 8.2.9.2.1.3; Table 8-45 +*/ +struct cxl_event_mem_module { +struct cxl_event_record_hdr hdr; +u8 event_type; +struct cxl_get_health_info info; +u8 reserved[0x3d ];
Decimal size please, otherwise the rest looks good to me.
Quoted 163 lines
drivers/cxl/cxlmem.h +2
-0+} __packed; +struct cxl_mbox_get_partition_info {__le64 active_volatile_cap;__le64 active_persistent_cap;include/trace/events/cxl.h +144
-0@@ -334,6 +334,150 @@ TRACE_EVENT(cxl_dram, ) ); +/* +* Memory Module Event Record - MMER +* +* CXL res 3.0 section 8.2.9.2.1.3; Table 8-45 +*/ +#define CXL_MMER_HEALTH_STATUS_CHANGE 0x00 +#define CXL_MMER_MEDIA_STATUS_CHANGE 0x01 +#define CXL_MMER_LIFE_USED_CHANGE 0x02 +#define CXL_MMER_TEMP_CHANGE 0x03 +#define CXL_MMER_DATA_PATH_ERROR 0x04 +#define CXL_MMER_LAS_ERROR 0x05 +#define show_dev_evt_type (type) __print_symbolic(type, \ + { CXL_MMER_HEALTH_STATUS_CHANGE, "Health Status Change" }, \ + { CXL_MMER_MEDIA_STATUS_CHANGE, "Media Status Change" }, \ + { CXL_MMER_LIFE_USED_CHANGE, "Life Used Change" }, \ + { CXL_MMER_TEMP_CHANGE, "Temperature Change" }, \ + { CXL_MMER_DATA_PATH_ERROR, "Data Path Error" }, \ + { CXL_MMER_LAS_ERROR, "LSA Error" } \ +) + +/* +* Device Health Information - DHI +* +* CXL res 3.0 section 8.2.9.8.3.1; Table 8-100 +*/ +#define CXL_DHI_HS_MAINTENANCE_NEEDED BIT(0) +#define CXL_DHI_HS_PERFORMANCE_DEGRADED BIT(1) +#define CXL_DHI_HS_HW_REPLACEMENT_NEEDED BIT(2) +#define show_health_status_flags (flags) __print_flags(flags, "|", \ + { CXL_DHI_HS_MAINTENANCE_NEEDED, "MAINTENANCE_NEEDED" }, \ + { CXL_DHI_HS_PERFORMANCE_DEGRADED, "PERFORMANCE_DEGRADED" }, \ + { CXL_DHI_HS_HW_REPLACEMENT_NEEDED, "REPLACEMENT_NEEDED" } \ +) + +#define CXL_DHI_MS_NORMAL 0x00 +#define CXL_DHI_MS_NOT_READY 0x01 +#define CXL_DHI_MS_WRITE_PERSISTENCY_LOST 0x02 +#define CXL_DHI_MS_ALL_DATA_LOST 0x03 +#define CXL_DHI_MS_WRITE_PERSISTENCY_LOSS_EVENT_POWER_LOSS 0x04 +#define CXL_DHI_MS_WRITE_PERSISTENCY_LOSS_EVENT_SHUTDOWN 0x05 +#define CXL_DHI_MS_WRITE_PERSISTENCY_LOSS_IMMINENT 0x06 +#define CXL_DHI_MS_WRITE_ALL_DATA_LOSS_EVENT_POWER_LOSS 0x07 +#define CXL_DHI_MS_WRITE_ALL_DATA_LOSS_EVENT_SHUTDOWN 0x08 +#define CXL_DHI_MS_WRITE_ALL_DATA_LOSS_IMMINENT 0x09 +#define show_media_status (ms) __print_symbolic(ms, \ + { CXL_DHI_MS_NORMAL, \ + "Normal" }, \ + { CXL_DHI_MS_NOT_READY, \ + "Not Ready" }, \ + { CXL_DHI_MS_WRITE_PERSISTENCY_LOST, \ + "Write Persistency Lost" }, \ + { CXL_DHI_MS_ALL_DATA_LOST, \ + "All Data Lost" }, \ + { CXL_DHI_MS_WRITE_PERSISTENCY_LOSS_EVENT_POWER_LOSS, \ + "Write Persistency Loss in the Event of Power Loss" }, \ + { CXL_DHI_MS_WRITE_PERSISTENCY_LOSS_EVENT_SHUTDOWN, \ + "Write Persistency Loss in Event of Shutdown" }, \ + { CXL_DHI_MS_WRITE_PERSISTENCY_LOSS_IMMINENT, \ + "Write Persistency Loss Imminent" }, \ + { CXL_DHI_MS_WRITE_ALL_DATA_LOSS_EVENT_POWER_LOSS, \ + "All Data Loss in Event of Power Loss" }, \ + { CXL_DHI_MS_WRITE_ALL_DATA_LOSS_EVENT_SHUTDOWN, \ + "All Data loss in the Event of Shutdown" }, \ + { CXL_DHI_MS_WRITE_ALL_DATA_LOSS_IMMINENT, \ + "All Data Loss Imminent" } \ +) + +#define CXL_DHI_AS_NORMAL 0x0 +#define CXL_DHI_AS_WARNING 0x1 +#define CXL_DHI_AS_CRITICAL 0x2 +#define show_two_bit_status (as) __print_symbolic(as, \ + { CXL_DHI_AS_NORMAL, "Normal" }, \ + { CXL_DHI_AS_WARNING, "Warning" }, \ + { CXL_DHI_AS_CRITICAL, "Critical" } \ +) +#define show_one_bit_status (as) __print_symbolic(as, \ + { CXL_DHI_AS_NORMAL, "Normal" }, \ + { CXL_DHI_AS_WARNING, "Warning" } \ +) + +#define CXL_DHI_AS_LIFE_USED (as) (as & 0x3) +#define CXL_DHI_AS_DEV_TEMP (as) ((as & 0xC) >> 2) +#define CXL_DHI_AS_COR_VOL_ERR_CNT (as) ((as & 0x10) >> 4) +#define CXL_DHI_AS_COR_PER_ERR_CNT (as) ((as & 0x20) >> 5) + +TRACE_EVENT (cxl_memory_module, + + TP_PROTO(const char *dev_name, enum cxl_event_log_type log, + struct cxl_event_mem_module *rec), + +TP_ARGS (dev_name, log, rec), + +TP_STRUCT__entry ( + CXL_EVT_TP_entry + +/* Memory Module Event */ + __field(u8, event_type) + +/* Device Health Info */ + __field(u8, health_status) + __field(u8, media_status) + __field(u8, life_used) + __field(u32, dirty_shutdown_cnt) + __field(u32, cor_vol_err_cnt) + __field(u32, cor_per_err_cnt) + __field(s16, device_temp) +__field (u8, add_status) + ), + +TP_fast_assign ( + CXL_EVT_TP_fast_assign(dev_name, log, rec->hdr); + +/* Memory Module Event */ + __entry->event_type = rec->event_type; + +/* Device Health Info */ + __entry->health_status = rec->info.health_status; + __entry->media_status = rec->info.media_status; + __entry->life_used = rec->info.life_used; + __entry->dirty_shutdown_cnt = get_unaligned_le32(rec->info.dirty_shutdown_cnt); + __entry->cor_vol_err_cnt = get_unaligned_le32(rec->info.cor_vol_err_cnt); + __entry->cor_per_err_cnt = get_unaligned_le32(rec->info.cor_per_err_cnt); + __entry->device_temp = get_unaligned_le16(rec->info.device_temp); + __entry->add_status = rec->info.add_status; + ), + +CXL_EVT_TP_printk ("event_type='%s' health_status='%s' media_status='%s' " \ +"as_life_used=%s as_dev_temp=%s as_cor_vol_err_cnt=%s " \ +"as_cor_per_err_cnt=%s life_used=%u device_temp=%d " \ +"dirty_shutdown_cnt=%u cor_vol_err_cnt=%u cor_per_err_cnt=%u" , +show_dev_evt_type (__entry->event_type), +show_health_status_flags (__entry->health_status), +show_media_status (__entry->media_status), +show_two_bit_status (CXL_DHI_AS_LIFE_USED (__entry->add_status)), +show_two_bit_status (CXL_DHI_AS_DEV_TEMP (__entry->add_status)), +show_one_bit_status (CXL_DHI_AS_COR_VOL_ERR_CNT (__entry->add_status)), +show_one_bit_status (CXL_DHI_AS_COR_PER_ERR_CNT (__entry->add_status)), + __entry->life_used, __entry->device_temp, + __entry->dirty_shutdown_cnt, __entry->cor_vol_err_cnt, + __entry->cor_per_err_cnt + ) +); + +#endif /* _CXL_TRACE_EVENTS_H */ /* This part must be outside protection */ Signature · 1 line
2.37.2
Re: [PATCH V2 07/11] cxl/mem: Trace Memory Module Event Record
In reply to [email protected]Quoted 10 lines · On Wed, 30 Nov 2022 16:27:15 -0800 [email protected] wrote:
From: Ira Weiny <[email protected]>
CXL rev 3.0 section 8.2.9.2.1.3 defines the Memory Module Event Record.
Determine if the event read is memory module record and if so trace the record.
Reviewed-by Steven Rostedt (Google) <[email protected]>Signed-off-by Ira Weiny <[email protected]>
LGTM
PATCH v2 8/11 cxl/mem: Wire up event interrupts
In reply to [email protected]From: Ira Weiny <[email protected]>
CXL device events are signaled via interrupts. Each event log may have a different interrupt message number. These message numbers are reported in the Get Event Interrupt Policy mailbox command.
Add interrupt support for event logs. Interrupts are allocated as shared interrupts. Therefore, all or some event logs can share the same message number.
---
Changes from V1:
Remove unneeded evt_int_policy from struct cxl_dev_state defer Dynamic Capacity support
Dave Jiang
s/irq/rc
use IRQ_NONE to signal the irq was not for us.
Jonathan
use msi_enabled rather than nr_irq_vec
On failure explicitly set CXL_INT_NONE
Add comment for Get Event Interrupt Policy use devm_request_threaded_irq()
Use individual handler/thread functions for each of the logs rather than struct cxl_event_irq_id.
Changes from RFC v2
Adjust to new irq 16 vector allocation
Jonathan
Remove CXL_INT_RES
Use irq threads to ensure mailbox commands are executed outside irq context Adjust for optional Dynamic Capacity log
--- drivers/cxl/core/mbox.c | 44 +++++++++++- drivers/cxl/cxlmem.h | 30 ++++++++ drivers/cxl/pci.c | 130 +++++++++++++++++++++++++++++++++++ include/uapi/linux/cxl_mem.h | 2 + 4 files changed, 204 insertions(+), 2 deletions(-)
drivers/cxl/core/mbox.c +42 -2
@@ -53,6 +53,8 @@ static struct cxl_mem_command cxl_mem_commands[CXL_MEM_COMMAND_ID_MAX] = {CXL_CMD (GET_SUPPORTED_LOGS ,0 ,CXL_VARIABLE_PAYLOAD ,CXL_CMD_FLAG_FORCE_ENABLE ),CXL_CMD (GET_EVENT_RECORD ,1 ,CXL_VARIABLE_PAYLOAD ,0 ),CXL_CMD (CLEAR_EVENT_RECORD ,CXL_VARIABLE_PAYLOAD ,0 ,0 ), +CXL_CMD (GET_EVT_INT_POLICY ,0 ,0x5 ,0 ), +CXL_CMD (SET_EVT_INT_POLICY ,0x5 ,0 ,0 ),CXL_CMD (GET_FW_INFO ,0 ,0x50 ,0 ),CXL_CMD (GET_PARTITION_INFO ,0 ,0x20 ,0 ), CXL_CMD(GET_LSA, 0x8, CXL_VARIABLE_PAYLOAD, 0),
@@ -806,8 +808,8 @@ static int cxl_clear_event_record(struct cxl_dev_state *cxlds,return 0 ; } -static void cxl_mem_get_records_log (struct cxl_dev_state *cxlds, -enum cxl_event_log_type type) +void cxl_mem_get_records_log (struct cxl_dev_state *cxlds, +enum cxl_event_log_type type) {struct cxl_get_event_payload *payload;u16 nr_rec;
@@ -857,6 +859,7 @@ static void cxl_mem_get_records_log(struct cxl_dev_state *cxlds, unlock_buffer:mutex_unlock (&cxlds->event_buf_lock); } +EXPORT_SYMBOL_NS_GPL (cxl_mem_get_records_log,CXL );static void cxl_mem_free_event_buffer (void *data) {
@@ -916,6 +919,43 @@ void cxl_mem_get_event_records(struct cxl_dev_state *cxlds) }EXPORT_SYMBOL_NS_GPL (cxl_mem_get_event_records,CXL ); +int cxl_event_config_msgnums (struct cxl_dev_state *cxlds, +struct cxl_event_interrupt_policy *policy) +{ +int rc; + + policy->info_settings =CXL_INT_MSI_MSIX ; + policy->warn_settings =CXL_INT_MSI_MSIX ; + policy->failure_settings =CXL_INT_MSI_MSIX ; + policy->fatal_settings =CXL_INT_MSI_MSIX ; + + rc =cxl_mbox_send_cmd (cxlds,CXL_MBOX_OP_SET_EVT_INT_POLICY , + policy,sizeof (*policy),NULL ,0 ); +if (rc <0 ) { +dev_err (cxlds->dev,"Failed to set event interrupt policy : %d" , + rc); + + policy->info_settings =CXL_INT_NONE ; + policy->warn_settings =CXL_INT_NONE ; + policy->failure_settings =CXL_INT_NONE ; + policy->fatal_settings =CXL_INT_NONE ; + +return rc; + } + +/* Retrieve interrupt message numbers */ + rc =cxl_mbox_send_cmd (cxlds,CXL_MBOX_OP_GET_EVT_INT_POLICY ,NULL ,0 , + policy,sizeof (*policy)); +if (rc <0 ) { +dev_err (cxlds->dev,"Failed to get event interrupt policy : %d" , + rc); +return rc; + } + +return 0 ; +} +EXPORT_SYMBOL_NS_GPL (cxl_event_config_msgnums,CXL ); +/** * cxl_mem_get_partition_info - Get partition info * @cxlds: The device data for the operation
drivers/cxl/cxlmem.h +30 -0
@@ -179,6 +179,30 @@ struct cxl_endpoint_dvsec_info {struct range dvsec_range[2 ]; }; +/** +* Event Interrupt Policy +* +* CXL rev 3.0 section 8.2.9.2.4; Table 8-52 +*/ +enum cxl_event_int_mode { +CXL_INT_NONE =0x00 , +CXL_INT_MSI_MSIX =0x01 , +CXL_INT_FW =0x02 +}; +#define CXL_EVENT_INT_MODE_MASK 0x3 +#define CXL_EVENT_INT_MSGNUM (setting) (((setting) & 0xf0) >> 4) +struct cxl_event_interrupt_policy { +u8 info_settings; +u8 warn_settings; +u8 failure_settings; +u8 fatal_settings; +} __packed; + +static inline bool cxl_evt_int_is_msi (u8 setting) +{ +return CXL_INT_MSI_MSIX == (setting &CXL_EVENT_INT_MODE_MASK ); +} +/** * struct cxl_dev_state - The driver device state *
@@ -262,6 +286,8 @@ enum cxl_opcode {CXL_MBOX_OP_RAW =CXL_MBOX_OP_INVALID ,CXL_MBOX_OP_GET_EVENT_RECORD =0x0100 ,CXL_MBOX_OP_CLEAR_EVENT_RECORD =0x0101 , +CXL_MBOX_OP_GET_EVT_INT_POLICY =0x0102 , +CXL_MBOX_OP_SET_EVT_INT_POLICY =0x0103 ,CXL_MBOX_OP_GET_FW_INFO =0x0200 ,CXL_MBOX_OP_ACTIVATE_FW =0x0202 , CXL_MBOX_OP_GET_SUPPORTED_LOGS = 0x0400,
@@ -537,7 +563,11 @@ int cxl_mem_create_range_info(struct cxl_dev_state *cxlds);struct cxl_dev_state *cxl_dev_state_create (struct device *dev);void set_exclusive_cxl_commands (struct cxl_dev_state *cxlds,unsigned long *cmds);void clear_exclusive_cxl_commands (struct cxl_dev_state *cxlds,unsigned long *cmds); +void cxl_mem_get_records_log (struct cxl_dev_state *cxlds, +enum cxl_event_log_type type);void cxl_mem_get_event_records (struct cxl_dev_state *cxlds); +int cxl_event_config_msgnums (struct cxl_dev_state *cxlds, +struct cxl_event_interrupt_policy *policy);#ifdef CONFIG_CXL_SUSPEND void cxl_mem_active_inc (void );void cxl_mem_active_dec (void );
drivers/cxl/pci.c +130 -0
142 lines not on this page · raw message
include/uapi/linux/cxl_mem.h +2 -0
@@ -26,6 +26,8 @@___C (GET_SUPPORTED_LOGS ,"Get Supported Logs" ), \___C (GET_EVENT_RECORD ,"Get Event Record" ), \___C (CLEAR_EVENT_RECORD ,"Clear Event Record" ), \ +___C (GET_EVT_INT_POLICY ,"Get Event Interrupt Policy" ), \ +___C (SET_EVT_INT_POLICY ,"Set Event Interrupt Policy" ), \___C (GET_FW_INFO ,"Get FW Info" ), \___C (GET_PARTITION_INFO ,"Get Partition Information" ), \ ___C(GET_LSA, "Get Label Storage Area"), \
Signature · 1 line
2.37.2
Re: [PATCH V2 08/11] cxl/mem: Wire up event interrupts
Quoted 11 lines · On Wed, 30 Nov 2022 16:27:16 -0800 [email protected] wrote:
From: Ira Weiny <[email protected]>
CXL device events are signaled via interrupts. Each event log may have a different interrupt message number. These message numbers are reported in the Get Event Interrupt Policy mailbox command.
Add interrupt support for event logs. Interrupts are allocated as shared interrupts. Therefore, all or some event logs can share the same message number.
Signed-off-by Ira Weiny <[email protected]>
A few trivial comments, but only superficially code style stuff which you can ignore if you feel strongly about current style or it matches existing file style etc...
Quoted 49 lines
---
Changes from V1:
Remove unneededevt_int_policyfrom structcxl_dev_statedefer Dynamic Capacity support
Dave Jiang
s/irq/rc
useIRQ_NONEto signal the irq was not for us.
Jonathan
usemsi_enabledrather thannr_irq_vec
On failure explicitly setCXL_INT_NONE
Add comment for Get Event Interrupt Policy usedevm_request_threaded_irq()
Use individual handler/thread functions for each of the logs rather than structcxl_event_irq_id.Changes from RFC v2
Adjust to new irq 16 vector allocation
Jonathan
RemoveCXL_INT_RES
Use irq threads to ensure mailbox commands are executed outside irq context Adjust for optional Dynamic Capacity log--- drivers/cxl/core/mbox.c | 44 +++++++++++- drivers/cxl/cxlmem.h | 30 ++++++++ drivers/cxl/pci.c | 130 +++++++++++++++++++++++++++++++++++ include/uapi/linux/cxl_mem.h | 2 + 4 files changed, 204 insertions(+), 2 deletions(-)drivers/cxl/cxlmem.h +12
-0@@ -179,6 +179,30 @@ struct cxl_endpoint_dvsec_info {struct range dvsec_range[2 ]; }; +/** +* Event Interrupt Policy +* +* CXL rev 3.0 section 8.2.9.2.4; Table 8-52 +*/ +enum cxl_event_int_mode { +CXL_INT_NONE =0x00 , +CXL_INT_MSI_MSIX =0x01 , +CXL_INT_FW =0x02 +}; +#define CXL_EVENT_INT_MODE_MASK 0x3 +#define CXL_EVENT_INT_MSGNUM (setting) (((setting) & 0xf0) >> 4)
drivers/cxl/cxlmem.h struct cxl_endpoint_dvsec_info { · line 203
Quoted 10 lines
drivers/cxl/cxlmem.h +10
-0+struct cxl_event_interrupt_policy { +u8 info_settings; +u8 warn_settings; +u8 failure_settings; +u8 fatal_settings; +} __packed; + +static inline bool cxl_evt_int_is_msi (u8 setting) +{ +return CXL_INT_MSI_MSIX == (setting &CXL_EVENT_INT_MODE_MASK );
Maybe a case for FIELD_GET() though given the defines are all local it is already obvious what this is doing so fine if you prefer to keep it as is.
Quoted 1 line
drivers/cxl/cxlmem.h +1
-0+}
...
Quoted 5 lines
drivers/cxl/pci.c +0
-0
drivers/cxl/pci.c static void cxl_pci_alloc_irq_vectors(struct cxl_dev_state *cxlds) · line 458
Quoted 10 lines
drivers/cxl/pci.c +7
-0@@ -449,6 +449,134 @@ static void cxl_pci_alloc_irq_vectors(struct cxl_dev_state *cxlds) cxlds->msi_enabled = true; } +static irqreturn_t cxl_event_info_thread (int irq,void *id) +{ +struct cxl_dev_state *cxlds = id; + +cxl_mem_get_records_log (cxlds,CXL_EVENT_TYPE_INFO ); +return IRQ_HANDLED ; +}
I'm not a great fan of macros, but maybe this is a case for them.
drivers/cxl/pci.c static void cxl_pci_alloc_irq_vectors(struct cxl_dev_state *cxlds) · line 463
Quoted 5 lines
drivers/cxl/pci.c +5
-0+ +static irqreturn_t cxl_event_info_handler (int irq,void *id) +{ +struct cxl_dev_state *cxlds = id; +u32 status =readl (cxlds->regs.status +CXLDEV_DEV_EVENT_STATUS_OFFSET );
Superficial and this is guaranteed to work (8.2.8 allow all sizes of read up to 64 bytes), but maybe should treat this as a 64 bit register as that aligns better with spec?
drivers/cxl/pci.c static void cxl_pci_alloc_irq_vectors(struct cxl_dev_state *cxlds) · line 465
Quoted 2 lines
drivers/cxl/pci.c +2
-0+ +if (CXLDEV_EVENT_STATUS_INFO & status)
Another maybe FIELD_GET() case?
drivers/cxl/pci.c static void cxl_pci_alloc_irq_vectors(struct cxl_dev_state *cxlds) · line 472
Quoted 7 lines
drivers/cxl/pci.c +7
-0+return IRQ_WAKE_THREAD ; +return IRQ_NONE ; +} + +static irqreturn_t cxl_event_warn_thread (int irq,void *id) +{ +struct cxl_dev_state *cxlds = id;
Why id? I'd call it what it is (maybe _cxlsd) and not bother with the local variable in this case as it is only used once and doesn't need the type.
static irqreturn_t cxl_event_warn_thread(int irq, void *cxlds)
{
cxl_mem_get_records_log(cxlds, CXL_EVENT_TYPE_WARN);
return IRQ_HANDLED;
}Quoted 5 lines
drivers/cxl/pci.c +5
-0+ +cxl_mem_get_records_log (cxlds,CXL_EVENT_TYPE_WARN ); +return IRQ_HANDLED ; +} +
...
Re: [PATCH V2 08/11] cxl/mem: Wire up event interrupts
Quoted 13 lines · Jonathan Cameron
Quoted 10 lines · On Wed, 30 Nov 2022 16:27:16 -0800 [email protected] wrote:
From: Ira Weiny <[email protected]>
CXL device events are signaled via interrupts. Each event log may have a different interrupt message number. These message numbers are reported in the Get Event Interrupt Policy mailbox command.
Add interrupt support for event logs. Interrupts are allocated as shared interrupts. Therefore, all or some event logs can share the same message number.
Quoted 5 lines
Quoted 1 line
Signed-off-by Ira Weiny <[email protected]>A few trivial comments, but only superficially code style stuff which you can ignore if you feel strongly about current style or it matches existing file style etc...
Thanks I'm leaving this one I think.
Quoted 78 lines
Reviewed-by Jonathan Cameron <[email protected]>Quoted 49 lines
---
Changes from V1:
Remove unneededevt_int_policyfrom structcxl_dev_statedefer Dynamic Capacity support
Dave Jiang
s/irq/rc
useIRQ_NONEto signal the irq was not for us.
Jonathan
usemsi_enabledrather thannr_irq_vec
On failure explicitly setCXL_INT_NONE
Add comment for Get Event Interrupt Policy usedevm_request_threaded_irq()
Use individual handler/thread functions for each of the logs rather than structcxl_event_irq_id.Changes from RFC v2
Adjust to new irq 16 vector allocation
Jonathan
RemoveCXL_INT_RES
Use irq threads to ensure mailbox commands are executed outside irq context Adjust for optional Dynamic Capacity log--- drivers/cxl/core/mbox.c | 44 +++++++++++- drivers/cxl/cxlmem.h | 30 ++++++++ drivers/cxl/pci.c | 130 +++++++++++++++++++++++++++++++++++ include/uapi/linux/cxl_mem.h | 2 + 4 files changed, 204 insertions(+), 2 deletions(-)drivers/cxl/cxlmem.h +12
-0@@ -179,6 +179,30 @@ struct cxl_endpoint_dvsec_info {struct range dvsec_range[2 ]; }; +/** +* Event Interrupt Policy +* +* CXL rev 3.0 section 8.2.9.2.4; Table 8-52 +*/ +enum cxl_event_int_mode { +CXL_INT_NONE =0x00 , +CXL_INT_MSI_MSIX =0x01 , +CXL_INT_FW =0x02 +}; +#define CXL_EVENT_INT_MODE_MASK 0x3 +#define CXL_EVENT_INT_MSGNUM (setting) (((setting) & 0xf0) >> 4)drivers/cxl/cxlmem.h struct cxl_endpoint_dvsec_info { · line 203
Quoted 10 lines
drivers/cxl/cxlmem.h +10
-0+struct cxl_event_interrupt_policy { +u8 info_settings; +u8 warn_settings; +u8 failure_settings; +u8 fatal_settings; +} __packed; + +static inline bool cxl_evt_int_is_msi (u8 setting) +{ +return CXL_INT_MSI_MSIX == (setting &CXL_EVENT_INT_MODE_MASK );Maybe a case for
FIELD_GET()though given the defines are all local it is already obvious what this is doing so fine if you prefer to keep it as is.Quoted 1 line
drivers/cxl/cxlmem.h +1
-0+}...
Quoted 8 lines
drivers/cxl/pci.c +0
-0@@ -449,6 +449,134 @@ static void cxl_pci_alloc_irq_vectors(struct cxl_dev_state *cxlds) cxlds->msi_enabled = true; }
Quoted 9 lines
drivers/cxl/pci.c static void cxl_pci_alloc_irq_vectors(struct cxl_dev_state *cxlds) · line 458
Quoted 7 lines
drivers/cxl/pci.c +7
-0+static irqreturn_t cxl_event_info_thread (int irq,void *id) +{ +struct cxl_dev_state *cxlds = id; + +cxl_mem_get_records_log (cxlds,CXL_EVENT_TYPE_INFO ); +return IRQ_HANDLED ; +}I'm not a great fan of macros, but maybe this is a case for them.
Nor am I. I particularly don't like when functions are defined with macros as it makes git grep and cscope harder to use on the code.
Quoted 10 lines
drivers/cxl/pci.c static void cxl_pci_alloc_irq_vectors(struct cxl_dev_state *cxlds) · line 463
Quoted 6 lines
drivers/cxl/pci.c +5
-0+ +static irqreturn_t cxl_event_info_handler (int irq,void *id) +{ +struct cxl_dev_state *cxlds = id; +u32 status =readl (cxlds->regs.status +CXLDEV_DEV_EVENT_STATUS_OFFSET );Superficial and this is guaranteed to work (8.2.8 allow all sizes of read up to 64 bytes), but maybe should treat this as a 64 bit register as that aligns better with spec?
I suppose it does. When I looked at that I only noticed the 32bit field and thought the register was 32bits. But most of our reads are to u32 fields.
I think it is fine for now.
Quoted 5 lines
drivers/cxl/pci.c static void cxl_pci_alloc_irq_vectors(struct cxl_dev_state *cxlds) · line 465
Quoted 3 lines
drivers/cxl/pci.c +2
-0+ +if (CXLDEV_EVENT_STATUS_INFO & status)Another maybe
FIELD_GET()case?
Are we using field get for individual bits? I see we are for some things. :-(
I was thinking the 'field' is 'Event Status' and the 32 bit read of the register already got that.
Quoted 10 lines
drivers/cxl/pci.c static void cxl_pci_alloc_irq_vectors(struct cxl_dev_state *cxlds) · line 472
Quoted 8 lines
drivers/cxl/pci.c +7
-0+return IRQ_WAKE_THREAD ; +return IRQ_NONE ; +} + +static irqreturn_t cxl_event_warn_thread (int irq,void *id) +{ +struct cxl_dev_state *cxlds = id;Why id?
Shortened from 'dev_id' from the devm_request_threaded_irq() function signature.
Quoted 7 lines
I'd call it what it is (maybe
_cxlsd) and not bother with the local variable in this case as it is only used once and doesn't need the type.static
irqreturn_tcxl_event_warn_thread(int irq, void *cxlds){ cxl_mem_get_records_log(cxlds, CXL_EVENT_TYPE_WARN);
Yea in this case it is pretty clear but I hate having to look at function signatures to figure out what a (void *) callback type is supposed to be.
This ...
Quoted 3 lines
drivers/cxl/pci.c static void cxl_pci_alloc_irq_vectors(struct cxl_dev_state *cxlds) · line 472
Quoted 3 lines
drivers/cxl/pci.c +3
-0+static irqreturn_t cxl_event_warn_thread (int irq,void *id) +{ +struct cxl_dev_state *cxlds = id;
... helps to self document cxl_event_warn_thread() IMO, and the compiler will optimize.
Thanks for looking!
Ira
Quoted 15 lines
return IRQ_HANDLED; }Quoted 5 lines
drivers/cxl/pci.c +5
-0+ +cxl_mem_get_records_log (cxlds,CXL_EVENT_TYPE_WARN ); +return IRQ_HANDLED ; +} +...
Re: [PATCH V2 08/11] cxl/mem: Wire up event interrupts
In reply to [email protected]Quoted 11 lines · On Wed, 30 Nov 2022, [email protected] wrote:
From: Ira Weiny <[email protected]>
CXL device events are signaled via interrupts. Each event log may have a different interrupt message number. These message numbers are reported in the Get Event Interrupt Policy mailbox command.
Add interrupt support for event logs. Interrupts are allocated as shared interrupts. Therefore, all or some event logs can share the same message number.
Signed-off-by Ira Weiny <[email protected]>
RE: [PATCH V2 08/11] cxl/mem: Wire up event interrupts
In reply to [email protected]Quoted 9 lines · ira.weiny@ wrote:
From: Ira Weiny <[email protected]>
CXL device events are signaled via interrupts. Each event log may have a different interrupt message number. These message numbers are reported in the Get Event Interrupt Policy mailbox command.
Add interrupt support for event logs. Interrupts are allocated as shared interrupts. Therefore, all or some event logs can share the same message number.
Definitely squash patch1 with this one, especially because this shows that the ->msi_enabled portion of patch1 was unnecessary, see below.
Quoted 66 lines
Signed-off-by Ira Weiny <[email protected]>---
Changes from V1:
Remove unneededevt_int_policyfrom structcxl_dev_statedefer Dynamic Capacity support
Dave Jiang
s/irq/rc
useIRQ_NONEto signal the irq was not for us.
Jonathan
usemsi_enabledrather thannr_irq_vec
On failure explicitly setCXL_INT_NONE
Add comment for Get Event Interrupt Policy usedevm_request_threaded_irq()
Use individual handler/thread functions for each of the logs rather than structcxl_event_irq_id.Changes from RFC v2
Adjust to new irq 16 vector allocation
Jonathan
RemoveCXL_INT_RES
Use irq threads to ensure mailbox commands are executed outside irq context Adjust for optional Dynamic Capacity log--- drivers/cxl/core/mbox.c | 44 +++++++++++- drivers/cxl/cxlmem.h | 30 ++++++++ drivers/cxl/pci.c | 130 +++++++++++++++++++++++++++++++++++ include/uapi/linux/cxl_mem.h | 2 + 4 files changed, 204 insertions(+), 2 deletions(-)drivers/cxl/core/mbox.c +5
-2@@ -53,6 +53,8 @@ static struct cxl_mem_command cxl_mem_commands[CXL_MEM_COMMAND_ID_MAX] = {CXL_CMD (GET_SUPPORTED_LOGS ,0 ,CXL_VARIABLE_PAYLOAD ,CXL_CMD_FLAG_FORCE_ENABLE ),CXL_CMD (GET_EVENT_RECORD ,1 ,CXL_VARIABLE_PAYLOAD ,0 ),CXL_CMD (CLEAR_EVENT_RECORD ,CXL_VARIABLE_PAYLOAD ,0 ,0 ), +CXL_CMD (GET_EVT_INT_POLICY ,0 ,0x5 ,0 ), +CXL_CMD (SET_EVT_INT_POLICY ,0x5 ,0 ,0 ),CXL_CMD (GET_FW_INFO ,0 ,0x50 ,0 ),CXL_CMD (GET_PARTITION_INFO ,0 ,0x20 ,0 ), CXL_CMD(GET_LSA, 0x8, CXL_VARIABLE_PAYLOAD, 0),@@ -806,8 +808,8 @@ static int cxl_clear_event_record(struct cxl_dev_state *cxlds,return 0 ; } -static void cxl_mem_get_records_log (struct cxl_dev_state *cxlds, -enum cxl_event_log_type type) +void cxl_mem_get_records_log (struct cxl_dev_state *cxlds, +enum cxl_event_log_type type) {struct cxl_get_event_payload *payload;u16 nr_rec;@@ -857,6 +859,7 @@ static void cxl_mem_get_records_log(struct cxl_dev_state *cxlds, unlock_buffer:mutex_unlock (&cxlds->event_buf_lock); } +EXPORT_SYMBOL_NS_GPL (cxl_mem_get_records_log,CXL );static void cxl_mem_free_event_buffer (void *data) {@@ -916,6 +919,43 @@ void cxl_mem_get_event_records(struct cxl_dev_state *cxlds) }EXPORT_SYMBOL_NS_GPL (cxl_mem_get_event_records,CXL );
drivers/cxl/core/mbox.c void cxl_mem_get_event_records(struct cxl_dev_state *cxlds) · line 930
Quoted 10 lines
drivers/cxl/core/mbox.c +9
-0+int cxl_event_config_msgnums (struct cxl_dev_state *cxlds, +struct cxl_event_interrupt_policy *policy) +{ +int rc; + + policy->info_settings =CXL_INT_MSI_MSIX ; + policy->warn_settings =CXL_INT_MSI_MSIX ; + policy->failure_settings =CXL_INT_MSI_MSIX ; + policy->fatal_settings =CXL_INT_MSI_MSIX ;
I think this needs to be careful not to undo events that the BIOS steered to itself in firmware-first mode, which raises another question, does firmware-first mean more the OS needs to backoff on some event-log handling as well?
Quoted 128 lines
drivers/cxl/core/mbox.c +28
-0+ + rc =cxl_mbox_send_cmd (cxlds,CXL_MBOX_OP_SET_EVT_INT_POLICY , + policy,sizeof (*policy),NULL ,0 ); +if (rc <0 ) { +dev_err (cxlds->dev,"Failed to set event interrupt policy : %d" , + rc); + + policy->info_settings =CXL_INT_NONE ; + policy->warn_settings =CXL_INT_NONE ; + policy->failure_settings =CXL_INT_NONE ; + policy->fatal_settings =CXL_INT_NONE ; + +return rc; + } + +/* Retrieve interrupt message numbers */ + rc =cxl_mbox_send_cmd (cxlds,CXL_MBOX_OP_GET_EVT_INT_POLICY ,NULL ,0 , + policy,sizeof (*policy)); +if (rc <0 ) { +dev_err (cxlds->dev,"Failed to get event interrupt policy : %d" , + rc); +return rc; + } + +return 0 ; +} +EXPORT_SYMBOL_NS_GPL (cxl_event_config_msgnums,CXL ); +/** * cxl_mem_get_partition_info - Get partition info * @cxlds: The device data for the operation drivers/cxl/cxlmem.h +30
-0@@ -179,6 +179,30 @@ struct cxl_endpoint_dvsec_info {struct range dvsec_range[2 ]; }; +/** +* Event Interrupt Policy +* +* CXL rev 3.0 section 8.2.9.2.4; Table 8-52 +*/ +enum cxl_event_int_mode { +CXL_INT_NONE =0x00 , +CXL_INT_MSI_MSIX =0x01 , +CXL_INT_FW =0x02 +}; +#define CXL_EVENT_INT_MODE_MASK 0x3 +#define CXL_EVENT_INT_MSGNUM (setting) (((setting) & 0xf0) >> 4) +struct cxl_event_interrupt_policy { +u8 info_settings; +u8 warn_settings; +u8 failure_settings; +u8 fatal_settings; +} __packed; + +static inline bool cxl_evt_int_is_msi (u8 setting) +{ +return CXL_INT_MSI_MSIX == (setting &CXL_EVENT_INT_MODE_MASK ); +} +/** * struct cxl_dev_state - The driver device state * @@ -262,6 +286,8 @@ enum cxl_opcode {CXL_MBOX_OP_RAW =CXL_MBOX_OP_INVALID ,CXL_MBOX_OP_GET_EVENT_RECORD =0x0100 ,CXL_MBOX_OP_CLEAR_EVENT_RECORD =0x0101 , +CXL_MBOX_OP_GET_EVT_INT_POLICY =0x0102 , +CXL_MBOX_OP_SET_EVT_INT_POLICY =0x0103 ,CXL_MBOX_OP_GET_FW_INFO =0x0200 ,CXL_MBOX_OP_ACTIVATE_FW =0x0202 , CXL_MBOX_OP_GET_SUPPORTED_LOGS = 0x0400,@@ -537,7 +563,11 @@ int cxl_mem_create_range_info(struct cxl_dev_state *cxlds);struct cxl_dev_state *cxl_dev_state_create (struct device *dev);void set_exclusive_cxl_commands (struct cxl_dev_state *cxlds,unsigned long *cmds);void clear_exclusive_cxl_commands (struct cxl_dev_state *cxlds,unsigned long *cmds); +void cxl_mem_get_records_log (struct cxl_dev_state *cxlds, +enum cxl_event_log_type type);void cxl_mem_get_event_records (struct cxl_dev_state *cxlds); +int cxl_event_config_msgnums (struct cxl_dev_state *cxlds, +struct cxl_event_interrupt_policy *policy);#ifdef CONFIG_CXL_SUSPEND void cxl_mem_active_inc (void );void cxl_mem_active_dec (void );drivers/cxl/pci.c +33
-0@@ -449,6 +449,134 @@ static void cxl_pci_alloc_irq_vectors(struct cxl_dev_state *cxlds) cxlds->msi_enabled = true; } +static irqreturn_t cxl_event_info_thread (int irq,void *id) +{ +struct cxl_dev_state *cxlds = id; + +cxl_mem_get_records_log (cxlds,CXL_EVENT_TYPE_INFO ); +return IRQ_HANDLED ; +} + +static irqreturn_t cxl_event_info_handler (int irq,void *id) +{ +struct cxl_dev_state *cxlds = id; +u32 status =readl (cxlds->regs.status +CXLDEV_DEV_EVENT_STATUS_OFFSET ); + +if (CXLDEV_EVENT_STATUS_INFO & status) +return IRQ_WAKE_THREAD ; +return IRQ_NONE ; +} + +static irqreturn_t cxl_event_warn_thread (int irq,void *id) +{ +struct cxl_dev_state *cxlds = id; + +cxl_mem_get_records_log (cxlds,CXL_EVENT_TYPE_WARN ); +return IRQ_HANDLED ; +} + +static irqreturn_t cxl_event_warn_handler (int irq,void *id) +{ +struct cxl_dev_state *cxlds = id; +u32 status =readl (cxlds->regs.status +CXLDEV_DEV_EVENT_STATUS_OFFSET ); + +if (CXLDEV_EVENT_STATUS_WARN & status) +return IRQ_WAKE_THREAD ;
drivers/cxl/pci.c static void cxl_pci_alloc_irq_vectors(struct cxl_dev_state *cxlds) · line 494
Quoted 10 lines
drivers/cxl/pci.c +10
-0+return IRQ_NONE ; +} + +static irqreturn_t cxl_event_failure_thread (int irq,void *id) +{ +struct cxl_dev_state *cxlds = id; + +cxl_mem_get_records_log (cxlds,CXL_EVENT_TYPE_FAIL ); +return IRQ_HANDLED ; +}
So I think one of the nice side effects of moving log priorty handling inside of cxl_mem_get_records_log() and looping through all log types in priority order until all status is clear is that an INFO interrupt also triggers a check of the FATAL status for free.
You likely do not even need to do the status read in hardirq part of the handler, just unconditionally wake the event handler thread. I.e. just pass NULL for @handler to devm_request_threaded_irq() and let the thread_fn figure it all out in priority order.
Quoted 32 lines
drivers/cxl/pci.c +32
-0+ +static irqreturn_t cxl_event_failure_handler (int irq,void *id) +{ +struct cxl_dev_state *cxlds = id; +u32 status =readl (cxlds->regs.status +CXLDEV_DEV_EVENT_STATUS_OFFSET ); + +if (CXLDEV_EVENT_STATUS_FAIL & status) +return IRQ_WAKE_THREAD ; +return IRQ_NONE ; +} + +static irqreturn_t cxl_event_fatal_thread (int irq,void *id) +{ +struct cxl_dev_state *cxlds = id; + +cxl_mem_get_records_log (cxlds,CXL_EVENT_TYPE_FATAL ); +return IRQ_HANDLED ; +} + +static irqreturn_t cxl_event_fatal_handler (int irq,void *id) +{ +struct cxl_dev_state *cxlds = id; +u32 status =readl (cxlds->regs.status +CXLDEV_DEV_EVENT_STATUS_OFFSET ); + +if (CXLDEV_EVENT_STATUS_FATAL & status) +return IRQ_WAKE_THREAD ; +return IRQ_NONE ; +} + +static void cxl_event_irqsetup (struct cxl_dev_state *cxlds) +{ +struct cxl_event_interrupt_policy policy;
drivers/cxl/pci.c static void cxl_pci_alloc_irq_vectors(struct cxl_dev_state *cxlds) · line 536
Quoted 10 lines
drivers/cxl/pci.c +10
-0+struct device *dev = cxlds->dev; +struct pci_dev *pdev =to_pci_dev (dev); +u8 setting; +int rc; + +if (cxl_event_config_msgnums (cxlds, &policy)) +return ; + + setting = policy.info_settings; +if (cxl_evt_int_is_msi (setting)) {
So pci_irq_vector() automatically handles checking if msi is enabled and will return a failure if either MSI is not enabled, or the message number did not get a vector.
With that insight I would do something like this (untested):
@@ -521,7 +521,14 @@ static irqreturn_t cxl_event_fatal_handler(int irq, void *id)
return IRQ_NONE;
}
-static void cxl_event_irqsetup(struct cxl_dev_state *cxlds) +static int cxl_evt_irq(struct pci_dev *pdev, u8 setting) +{
+ if ((setting & CXL_EVENT_INT_MODE_MASK) != CXL_INT_MSI_MSIX)
+ return -ENXIO;
+ return pci_irq_vector(pdev, CXL_EVENT_INT_MSGNUM(setting));
+}
+
+static int cxl_event_irqsetup(struct cxl_dev_state *cxlds)
{
struct cxl_event_interrupt_policy policy;
struct device *dev = cxlds->dev;
@@ -529,18 +536,17 @@ static void cxl_event_irqsetup(struct cxl_dev_state *cxlds)
u8 setting;
int rc;- if (cxl_event_config_msgnums(cxlds, &policy))
- return;
+ rc = cxl_event_config_msgnums(cxlds, &policy);
+ if (rc)
+ return rc;
- setting = policy.info_settings;
- if (cxl_evt_int_is_msi(setting)) {
- rc = devm_request_threaded_irq(dev,
- pci_irq_vector(pdev, CXL_EVENT_INT_MSGNUM(setting)), - cxl_event_info_handler, cxl_event_info_thread, - IRQF_SHARED, NULL, cxlds);
- if (rc)
- dev_err(dev, "Failed to get interrupt for %s event log\n",
- cxl_event_log_type_str(CXL_EVENT_TYPE_INFO));
+ rc = devm_request_threaded_irq(dev,
+ cxl_evt_irq(pdev, policy.info_settings),
+ NULL, cxl_event_info_thread, IRQF_SHARED,
+ NULL, cxlds);
+ if (rc) {
+ dev_err(dev, "Failed to get interrupt for INFO event log\n");
+ return rc;
}
setting = policy.warn_settings;drivers/cxl/pci.c static void cxl_pci_alloc_irq_vectors(struct cxl_dev_state *cxlds) · line 543
Quoted 7 lines
drivers/cxl/pci.c +7
-0+ rc =devm_request_threaded_irq (dev, +pci_irq_vector (pdev,CXL_EVENT_INT_MSGNUM (setting)), + cxl_event_info_handler, cxl_event_info_thread, +IRQF_SHARED ,NULL , cxlds); +if (rc) +dev_err (dev,"Failed to get interrupt for %s event log\n" , +cxl_event_log_type_str (CXL_EVENT_TYPE_INFO ));
Per above, no need to use cxl_event_log_type_str() in these.
Quoted 40 lines
drivers/cxl/pci.c +36
-0+ } + + setting = policy.warn_settings; +if (cxl_evt_int_is_msi (setting)) { + rc =devm_request_threaded_irq (dev, +pci_irq_vector (pdev,CXL_EVENT_INT_MSGNUM (setting)), + cxl_event_warn_handler, cxl_event_warn_thread, +IRQF_SHARED ,NULL , cxlds); +if (rc) +dev_err (dev,"Failed to get interrupt for %s event log\n" , +cxl_event_log_type_str (CXL_EVENT_TYPE_WARN )); + } + + setting = policy.failure_settings; +if (cxl_evt_int_is_msi (setting)) { + rc =devm_request_threaded_irq (dev, +pci_irq_vector (pdev,CXL_EVENT_INT_MSGNUM (setting)), + cxl_event_failure_handler, cxl_event_failure_thread, +IRQF_SHARED ,NULL , cxlds); +if (rc) +dev_err (dev,"Failed to get interrupt for %s event log\n" , +cxl_event_log_type_str (CXL_EVENT_TYPE_FAIL )); + } + + setting = policy.fatal_settings; +if (cxl_evt_int_is_msi (setting)) { + rc =devm_request_threaded_irq (dev, +pci_irq_vector (pdev,CXL_EVENT_INT_MSGNUM (setting)), + cxl_event_fatal_handler, cxl_event_fatal_thread, +IRQF_SHARED ,NULL , cxlds); +if (rc) +dev_err (dev,"Failed to get interrupt for %s event log\n" , +cxl_event_log_type_str (CXL_EVENT_TYPE_FATAL )); + } +} +static int cxl_pci_probe (struct pci_dev *pdev,const struct pci_device_id *id) {struct cxl_register_map map;
drivers/cxl/pci.c static int cxl_pci_probe(struct pci_dev *pdev, const struct pci_device_id *id) · line 646
Quoted 3 lines
drivers/cxl/pci.c +0
-0@@ -516,6 +644,8 @@ static int cxl_pci_probe(struct pci_dev *pdev, const struct pci_device_id *id)return rc;cxl_pci_alloc_irq_vectors (cxlds);
There should be fail return here, or a comment why this can be skipped, especially if the device claims to support events.
drivers/cxl/pci.c static int cxl_pci_probe(struct pci_dev *pdev, const struct pci_device_id *id) · line 648
Quoted 2 lines
drivers/cxl/pci.c +2
-0+if (cxlds->msi_enabled) +cxl_event_irqsetup (cxlds);
Per above, do this unconditionally.
Quoted 8 lines
drivers/cxl/pci.c +0
-0cxlmd =devm_cxl_add_memdev (cxlds);if (IS_ERR (cxlmd))include/uapi/linux/cxl_mem.h +0
-0
include/uapi/linux/cxl_mem.h line 33
Quoted 8 lines
include/uapi/linux/cxl_mem.h +2
-0@@ -26,6 +26,8 @@___C (GET_SUPPORTED_LOGS ,"Get Supported Logs" ), \___C (GET_EVENT_RECORD ,"Get Event Record" ), \___C (CLEAR_EVENT_RECORD ,"Clear Event Record" ), \ +___C (GET_EVT_INT_POLICY ,"Get Event Interrupt Policy" ), \ +___C (SET_EVT_INT_POLICY ,"Set Event Interrupt Policy" ), \___C (GET_FW_INFO ,"Get FW Info" ), \___C (GET_PARTITION_INFO ,"Get Partition Information" ), \ ___C(GET_LSA, "Get Label Storage Area"), \
Same, "at the end" comment.
Re: [PATCH V2 08/11] cxl/mem: Wire up event interrupts
Quoted 15 lines
drivers/cxl/core/mbox.c void cxl_mem_get_event_records(struct cxl_dev_state *cxlds) · line 930
Quoted 10 lines
drivers/cxl/core/mbox.c +9
-0+int cxl_event_config_msgnums (struct cxl_dev_state *cxlds, +struct cxl_event_interrupt_policy *policy) +{ +int rc; + + policy->info_settings =CXL_INT_MSI_MSIX ; + policy->warn_settings =CXL_INT_MSI_MSIX ; + policy->failure_settings =CXL_INT_MSI_MSIX ; + policy->fatal_settings =CXL_INT_MSI_MSIX ;I think this needs to be careful not to undo events that the BIOS steered to itself in firmware-first mode, which raises another question, does firmware-first mean more the OS needs to backoff on some event-log handling as well?
Hmm. Does the _OSC cover these. There is one for Memory error reporting that I think covers it (refers to 12.2.3.2)
Note that should cover any means of obtaining these, not just interrupt driven - so including the initial record clear.
..
Quoted 5 lines
Quoted 5 lines
drivers/cxl/pci.c +5
-0+ +static irqreturn_t cxl_event_failure_thread (int irq,void *id) +{ +struct cxl_dev_state *cxlds = id; +
Quoted 9 lines
drivers/cxl/pci.c static void cxl_pci_alloc_irq_vectors(struct cxl_dev_state *cxlds) · line 494
Quoted 3 lines
drivers/cxl/pci.c +3
-0+cxl_mem_get_records_log (cxlds,CXL_EVENT_TYPE_FAIL ); +return IRQ_HANDLED ; +}So I think one of the nice side effects of moving log priorty handling inside of
cxl_mem_get_records_log()and looping through all log types in priority order until all status is clear is that an INFO interrupt also triggers a check of the FATAL status for free.
I go the opposite way on this in thinking that an interrupt should only ever be used to handle the things it was registered for - so we should not be clearing fatal records in the handler triggered for info events.
Doing other actions like this relies on subtlies of the generic interrupt handling code which happens to force interrupt threads on a shared interrupt line to be serialized. I'm not sure we are safe at all the interrupt isn't shared unless we put a lock around the whole thing (we have one because of the buffer mutex though).
If going this way I think the lock needs a rename.
It's not just protecting the buffer used, but also serialize multiple interrupt threads.
Jonathan
Quoted 4 lines
You likely do not even need to do the status read in hardirq part of the handler, just unconditionally wake the event handler thread. I.e. just pass NULL for @handler to
devm_request_threaded_irq()and let thethread_fnfigure it all out in priority order.
Re: [PATCH V2 08/11] cxl/mem: Wire up event interrupts
Quoted 39 lines · Jonathan Cameron
Quoted 15 lines
drivers/cxl/core/mbox.c void cxl_mem_get_event_records(struct cxl_dev_state *cxlds) · line 930
Quoted 10 lines
drivers/cxl/core/mbox.c +9
-0+int cxl_event_config_msgnums (struct cxl_dev_state *cxlds, +struct cxl_event_interrupt_policy *policy) +{ +int rc; + + policy->info_settings =CXL_INT_MSI_MSIX ; + policy->warn_settings =CXL_INT_MSI_MSIX ; + policy->failure_settings =CXL_INT_MSI_MSIX ; + policy->fatal_settings =CXL_INT_MSI_MSIX ;I think this needs to be careful not to undo events that the BIOS steered to itself in firmware-first mode, which raises another question, does firmware-first mean more the OS needs to backoff on some event-log handling as well?
Hmm. Does the
_OSCcover these. There is one for Memory error reporting that I think covers it (refers to 12.2.3.2)Note that should cover any means of obtaining these, not just interrupt driven - so including the initial record clear.
..
Quoted 5 lines
Quoted 5 lines
drivers/cxl/pci.c +5
-0+ +static irqreturn_t cxl_event_failure_thread (int irq,void *id) +{ +struct cxl_dev_state *cxlds = id; +Quoted 9 lines
drivers/cxl/pci.c static void cxl_pci_alloc_irq_vectors(struct cxl_dev_state *cxlds) · line 494
Quoted 3 lines
drivers/cxl/pci.c +3
-0+cxl_mem_get_records_log (cxlds,CXL_EVENT_TYPE_FAIL ); +return IRQ_HANDLED ; +}So I think one of the nice side effects of moving log priorty handling inside of
cxl_mem_get_records_log()and looping through all log types in priority order until all status is clear is that an INFO interrupt also triggers a check of the FATAL status for free.
Quoted 3 lines
I go the opposite way on this in thinking that an interrupt should only ever be used to handle the things it was registered for - so we should not be clearing fatal records in the handler triggered for info events.
I would agree with you if this was a fast path and if the hardware mechanism did not involve shared status register that tells you that both FATAL and INFO are pending retrieval through a mechanism. Compare that to the separation between admin and IO queues in NVME.
If the handler is going to loop on the status register then it must be careful not to starve out FATAL while processing INFO.
Quoted 5 lines
Doing other actions like this relies on subtlies of the generic interrupt handling code which happens to force interrupt threads on a shared interrupt line to be serialized. I'm not sure we are safe at all the interrupt isn't shared unless we put a lock around the whole thing (we have one because of the buffer mutex though).
The interrupt is likely shared since there is no performance benefit to entice hardware vendors spend transistor budget on more vector space for events. The events architecture does not merit that spend.
Quoted 3 lines
If going this way I think the lock needs a rename.
It's not just protecting the buffer used, but also serialize multiple interrupt threads.
I will let Ira decide if he wants to rename, but in my mind the shared event buffer *is* the data being locked, the fact that multiple threads might be contending for it is immaterial.
Re: [PATCH V2 08/11] cxl/mem: Wire up event interrupts
Quoted 62 lines · On Fri, 2 Dec 2022 11:43:29 -0800 Dan Williams <[email protected]> wrote:
Quoted 39 lines · Jonathan Cameron
Quoted 15 lines
drivers/cxl/core/mbox.c void cxl_mem_get_event_records(struct cxl_dev_state *cxlds) · line 930
Quoted 10 lines
drivers/cxl/core/mbox.c +9
-0+int cxl_event_config_msgnums (struct cxl_dev_state *cxlds, +struct cxl_event_interrupt_policy *policy) +{ +int rc; + + policy->info_settings =CXL_INT_MSI_MSIX ; + policy->warn_settings =CXL_INT_MSI_MSIX ; + policy->failure_settings =CXL_INT_MSI_MSIX ; + policy->fatal_settings =CXL_INT_MSI_MSIX ;I think this needs to be careful not to undo events that the BIOS steered to itself in firmware-first mode, which raises another question, does firmware-first mean more the OS needs to backoff on some event-log handling as well?
Hmm. Does the
_OSCcover these. There is one for Memory error reporting that I think covers it (refers to 12.2.3.2)Note that should cover any means of obtaining these, not just interrupt driven - so including the initial record clear.
..
Quoted 5 lines
Quoted 5 lines
drivers/cxl/pci.c +5
-0+ +static irqreturn_t cxl_event_failure_thread (int irq,void *id) +{ +struct cxl_dev_state *cxlds = id; +Quoted 9 lines
drivers/cxl/pci.c static void cxl_pci_alloc_irq_vectors(struct cxl_dev_state *cxlds) · line 494
Quoted 3 lines
drivers/cxl/pci.c +3
-0+cxl_mem_get_records_log (cxlds,CXL_EVENT_TYPE_FAIL ); +return IRQ_HANDLED ; +}So I think one of the nice side effects of moving log priorty handling inside of
cxl_mem_get_records_log()and looping through all log types in priority order until all status is clear is that an INFO interrupt also triggers a check of the FATAL status for free.Quoted 3 lines
I go the opposite way on this in thinking that an interrupt should only ever be used to handle the things it was registered for - so we should not be clearing fatal records in the handler triggered for info events.
I would agree with you if this was a fast path and if the hardware mechanism did not involve shared status register that tells you that both FATAL and INFO are pending retrieval through a mechanism. Compare that to the separation between admin and IO queues in NVME.
If the handler is going to loop on the status register then it must be careful not to starve out FATAL while processing INFO.
Quoted 5 lines
Doing other actions like this relies on subtlies of the generic interrupt handling code which happens to force interrupt threads on a shared interrupt line to be serialized. I'm not sure we are safe at all the interrupt isn't shared unless we put a lock around the whole thing (we have one because of the buffer mutex though).
The interrupt is likely shared since there is no performance benefit to entice hardware vendors spend transistor budget on more vector space for events. The events architecture does not merit that spend.
Quoted 7 lines
Quoted 3 lines
If going this way I think the lock needs a rename.
It's not just protecting the buffer used, but also serialize multiple interrupt threads.I will let Ira decide if he wants to rename, but in my mind the shared event buffer *is* the data being locked, the fact that multiple threads might be contending for it is immaterial.
It isn't he only thing being protected. Access to the device is also being serialized including the data in it's registers.
If someone comes along later and decides to implement multiple buffers and there for gets rid of the lock. boom.
Jonathan
Re: [PATCH V2 08/11] cxl/mem: Wire up event interrupts
Quoted 73 lines · Jonathan Cameron
Quoted 62 lines · On Fri, 2 Dec 2022 11:43:29 -0800 Dan Williams <[email protected]> wrote:
Quoted 39 lines · Jonathan Cameron
Quoted 15 lines
drivers/cxl/core/mbox.c void cxl_mem_get_event_records(struct cxl_dev_state *cxlds) · line 930
Quoted 10 lines
drivers/cxl/core/mbox.c +9
-0+int cxl_event_config_msgnums (struct cxl_dev_state *cxlds, +struct cxl_event_interrupt_policy *policy) +{ +int rc; + + policy->info_settings =CXL_INT_MSI_MSIX ; + policy->warn_settings =CXL_INT_MSI_MSIX ; + policy->failure_settings =CXL_INT_MSI_MSIX ; + policy->fatal_settings =CXL_INT_MSI_MSIX ;I think this needs to be careful not to undo events that the BIOS steered to itself in firmware-first mode, which raises another question, does firmware-first mean more the OS needs to backoff on some event-log handling as well?
Hmm. Does the
_OSCcover these. There is one for Memory error reporting that I think covers it (refers to 12.2.3.2)Note that should cover any means of obtaining these, not just interrupt driven - so including the initial record clear.
..
Quoted 5 lines
Quoted 5 lines
drivers/cxl/pci.c +5
-0+ +static irqreturn_t cxl_event_failure_thread (int irq,void *id) +{ +struct cxl_dev_state *cxlds = id; +Quoted 9 lines
drivers/cxl/pci.c static void cxl_pci_alloc_irq_vectors(struct cxl_dev_state *cxlds) · line 494
Quoted 3 lines
drivers/cxl/pci.c +3
-0+cxl_mem_get_records_log (cxlds,CXL_EVENT_TYPE_FAIL ); +return IRQ_HANDLED ; +}So I think one of the nice side effects of moving log priorty handling inside of
cxl_mem_get_records_log()and looping through all log types in priority order until all status is clear is that an INFO interrupt also triggers a check of the FATAL status for free.Quoted 3 lines
I go the opposite way on this in thinking that an interrupt should only ever be used to handle the things it was registered for - so we should not be clearing fatal records in the handler triggered for info events.
I would agree with you if this was a fast path and if the hardware mechanism did not involve shared status register that tells you that both FATAL and INFO are pending retrieval through a mechanism. Compare that to the separation between admin and IO queues in NVME.
If the handler is going to loop on the status register then it must be careful not to starve out FATAL while processing INFO.
Quoted 5 lines
Doing other actions like this relies on subtlies of the generic interrupt handling code which happens to force interrupt threads on a shared interrupt line to be serialized. I'm not sure we are safe at all the interrupt isn't shared unless we put a lock around the whole thing (we have one because of the buffer mutex though).
The interrupt is likely shared since there is no performance benefit to entice hardware vendors spend transistor budget on more vector space for events. The events architecture does not merit that spend.
Quoted 7 lines
Quoted 3 lines
If going this way I think the lock needs a rename.
It's not just protecting the buffer used, but also serialize multiple interrupt threads.I will let Ira decide if he wants to rename, but in my mind the shared event buffer *is* the data being locked, the fact that multiple threads might be contending for it is immaterial.
Quoted 5 lines
It isn't he only thing being protected. Access to the device is also being serialized including the data in it's registers.
If someone comes along later and decides to implement multiple buffers and there for gets rid of the lock. boom.
That's what the mailbox mutex is protecting against. If there is an aspect of the hardware state that is not protected by that then that's a bug.
Re: [PATCH V2 08/11] cxl/mem: Wire up event interrupts
Quoted 77 lines · On Mon, 5 Dec 2022 08:35:34 -0800 Dan Williams <[email protected]> wrote:
Quoted 76 lines · Jonathan Cameron
Quoted 62 lines · On Fri, 2 Dec 2022 11:43:29 -0800 Dan Williams <[email protected]> wrote:
Quoted 39 lines · Jonathan Cameron
Quoted 15 lines
drivers/cxl/core/mbox.c void cxl_mem_get_event_records(struct cxl_dev_state *cxlds) · line 930
Quoted 10 lines
drivers/cxl/core/mbox.c +9
-0+int cxl_event_config_msgnums (struct cxl_dev_state *cxlds, +struct cxl_event_interrupt_policy *policy) +{ +int rc; + + policy->info_settings =CXL_INT_MSI_MSIX ; + policy->warn_settings =CXL_INT_MSI_MSIX ; + policy->failure_settings =CXL_INT_MSI_MSIX ; + policy->fatal_settings =CXL_INT_MSI_MSIX ;I think this needs to be careful not to undo events that the BIOS steered to itself in firmware-first mode, which raises another question, does firmware-first mean more the OS needs to backoff on some event-log handling as well?
Hmm. Does the
_OSCcover these. There is one for Memory error reporting that I think covers it (refers to 12.2.3.2)Note that should cover any means of obtaining these, not just interrupt driven - so including the initial record clear.
..
Quoted 5 lines
Quoted 5 lines
drivers/cxl/pci.c +5
-0+ +static irqreturn_t cxl_event_failure_thread (int irq,void *id) +{ +struct cxl_dev_state *cxlds = id; +Quoted 9 lines
drivers/cxl/pci.c static void cxl_pci_alloc_irq_vectors(struct cxl_dev_state *cxlds) · line 494
Quoted 3 lines
drivers/cxl/pci.c +3
-0+cxl_mem_get_records_log (cxlds,CXL_EVENT_TYPE_FAIL ); +return IRQ_HANDLED ; +}So I think one of the nice side effects of moving log priorty handling inside of
cxl_mem_get_records_log()and looping through all log types in priority order until all status is clear is that an INFO interrupt also triggers a check of the FATAL status for free.Quoted 3 lines
I go the opposite way on this in thinking that an interrupt should only ever be used to handle the things it was registered for - so we should not be clearing fatal records in the handler triggered for info events.
I would agree with you if this was a fast path and if the hardware mechanism did not involve shared status register that tells you that both FATAL and INFO are pending retrieval through a mechanism. Compare that to the separation between admin and IO queues in NVME.
If the handler is going to loop on the status register then it must be careful not to starve out FATAL while processing INFO.
Quoted 5 lines
Doing other actions like this relies on subtlies of the generic interrupt handling code which happens to force interrupt threads on a shared interrupt line to be serialized. I'm not sure we are safe at all the interrupt isn't shared unless we put a lock around the whole thing (we have one because of the buffer mutex though).
The interrupt is likely shared since there is no performance benefit to entice hardware vendors spend transistor budget on more vector space for events. The events architecture does not merit that spend.
Quoted 7 lines
Quoted 3 lines
If going this way I think the lock needs a rename.
It's not just protecting the buffer used, but also serialize multiple interrupt threads.I will let Ira decide if he wants to rename, but in my mind the shared event buffer *is* the data being locked, the fact that multiple threads might be contending for it is immaterial.
It isn't he only thing being protected. Access to the device is also being serialized including the data in it's registers.
Quoted 7 lines
Quoted 2 lines
If someone comes along later and decides to implement multiple buffers and there for gets rid of the lock. boom.
That's what the mailbox mutex is protecting against. If there is an aspect of the hardware state that is not protected by that then that's a bug.
Wrong level of locking. This is about a race on multiple commands
1) Read data from interrupt thread 1.
2) Read same data from interrupt thread 2.
3) Clear data from interrupt thread 1.
4) Clear data from interrupt thread 2. Boom (well minor error we probably eat but not good practice).
PATCH v2 9/11 cxl/test: Add generic mock events
In reply to [email protected]From: Ira Weiny <[email protected]>
Facilitate testing basic Get/Clear Event functionality by creating multiple logs and generic events with made up UUID's.
Data is completely made up with data patterns which should be easy to spot in trace output.
A single sysfs entry resets the event data and triggers collecting the events for testing.
Events are returned one at a time which is within the specification even though it does not exercise the full capabilities of what a device may do.
Test traces are easy to obtain with a small script such as this:
#!/bin/bash -x
devices=find /sys/devices/platform -name cxl_mem*
# Turn on tracing
echo "" > /sys/kernel/tracing/trace echo 1 > /sys/kernel/tracing/events/cxl/enable echo 1 > /sys/kernel/tracing/tracing_on
# Generate fake interrupt
for device in $devices; do
echo 1 > $device/event_trigger
done
# Turn off tracing and report events
echo 0 > /sys/kernel/tracing/tracing_on
cat /sys/kernel/tracing/trace
---
Changes from v1:
Fix up for new structures
Jonathan
Update based on specification discussion
Changes from RFC v2:
Adjust to simulate the event status register
Changes from RFC:
Separate out the event code
Adjust for struct changes.
Clean up devm_cxl_mock_event_logs()
Clean up naming and comments
Jonathan
Remove dynamic allocation of event logs Clean up comment
Remove unneeded xarray
Ensure event_trigger sysfs is valid prior to the driver going active.
Dan
Remove the fill/reset event sysfs as these operations can be done together
--- drivers/cxl/core/mbox.c | 33 +++-- drivers/cxl/cxlmem.h | 1 + tools/testing/cxl/test/Kbuild | 2 +- tools/testing/cxl/test/events.c | 242 ++++++++++++++++++++++++++++++++ tools/testing/cxl/test/events.h | 9 ++ tools/testing/cxl/test/mem.c | 35 ++++- 6 files changed, 307 insertions(+), 15 deletions(-) create mode 100644 tools/testing/cxl/test/events.c create mode 100644 tools/testing/cxl/test/events.h
drivers/cxl/core/mbox.c +20 -13
@@ -886,20 +886,9 @@ static struct cxl_get_event_payload *alloc_event_buf(struct cxl_dev_state *cxldsreturn buf; } -/** -* cxl_mem_get_event_records - Get Event Records from the device -* @cxlds: The device data for the operation -* -* Retrieve all event records available on the device, report them as trace -* events, and clear them. -* -* See CXL rev 3.0 @8.2.9.2.2 Get Event Records -* See CXL rev 3.0 @8.2.9.2.3 Clear Event Records -*/ -void cxl_mem_get_event_records (struct cxl_dev_state *cxlds) +/* Direct call for mock testing */ +void __cxl_mem_get_event_records (struct cxl_dev_state *cxlds,u32 status) { -u32 status =readl (cxlds->regs.status +CXLDEV_DEV_EVENT_STATUS_OFFSET ); -dev_dbg (cxlds->dev,"Reading event logs: %x\n" , status);if (!cxlds->event_buf) {
@@ -917,6 +906,24 @@ void cxl_mem_get_event_records(struct cxl_dev_state *cxlds)if (status &CXLDEV_EVENT_STATUS_FATAL )cxl_mem_get_records_log (cxlds,CXL_EVENT_TYPE_FATAL ); } +EXPORT_SYMBOL_NS_GPL (__cxl_mem_get_event_records,CXL ); + +/** +* cxl_mem_get_event_records - Get Event Records from the device +* @cxlds: The device data for the operation +* +* Retrieve all event records available on the device, report them as trace +* events, and clear them. +* +* See CXL rev 3.0 @8.2.9.2.2 Get Event Records +* See CXL rev 3.0 @8.2.9.2.3 Clear Event Records +*/ +void cxl_mem_get_event_records (struct cxl_dev_state *cxlds) +{ +u32 status =readl (cxlds->regs.status +CXLDEV_DEV_EVENT_STATUS_OFFSET ); + +__cxl_mem_get_event_records (cxlds, status); +}EXPORT_SYMBOL_NS_GPL (cxl_mem_get_event_records,CXL ); int cxl_event_config_msgnums(struct cxl_dev_state *cxlds,
drivers/cxl/cxlmem.h +1 -0
@@ -565,6 +565,7 @@ void set_exclusive_cxl_commands(struct cxl_dev_state *cxlds, unsigned long *cmdsvoid clear_exclusive_cxl_commands (struct cxl_dev_state *cxlds,unsigned long *cmds);void cxl_mem_get_records_log (struct cxl_dev_state *cxlds,enum cxl_event_log_type type); +void __cxl_mem_get_event_records (struct cxl_dev_state *cxlds,u32 status);void cxl_mem_get_event_records (struct cxl_dev_state *cxlds);int cxl_event_config_msgnums (struct cxl_dev_state *cxlds,struct cxl_event_interrupt_policy *policy);
tools/testing/cxl/test/Kbuild +1 -1
@@ -7,4 +7,4 @@ obj-m += cxl_mock_mem.o cxl_test-y := cxl.o cxl_mock-y := mock.o -cxl_mock_mem-y := mem.o +cxl_mock_mem-y := mem.o events.o
tools/testing/cxl/test/events.c new file +242 -0
242 lines not on this page · raw message
tools/testing/cxl/test/events.h new file +9 -0
@@ -0,0 +1,9 @@ +/* SPDX-License-Identifier: GPL-2.0 */ + +#include <cxlmem.h> + +int mock_get_event (struct cxl_dev_state *cxlds,struct cxl_mbox_cmd *cmd); +int mock_clear_event (struct cxl_dev_state *cxlds,struct cxl_mbox_cmd *cmd); +u32 cxl_mock_add_event_logs (struct cxl_dev_state *cxlds); +void cxl_mock_remove_event_logs (struct device *dev); +void cxl_mock_event_trigger (struct device *dev);
tools/testing/cxl/test/mem.c +34 -1
@@ -8,6 +8,7 @@#include <linux/sizes.h> #include <linux/bits.h> #include <cxlmem.h> +#include "events.h" #define LSA_SIZE SZ_128K#define DEV_SIZE SZ_2G
@@ -224,6 +225,12 @@ static int cxl_mock_mbox_send(struct cxl_dev_state *cxlds, struct cxl_mbox_cmd *case CXL_MBOX_OP_GET_PARTITION_INFO : rc =mock_partition_info (cxlds, cmd);break ; +case CXL_MBOX_OP_GET_EVENT_RECORD : + rc =mock_get_event (cxlds, cmd); +break ; +case CXL_MBOX_OP_CLEAR_EVENT_RECORD : + rc =mock_clear_event (cxlds, cmd); +break ;case CXL_MBOX_OP_SET_LSA : rc =mock_set_lsa (cxlds, cmd);break ;
@@ -245,11 +252,27 @@ static void label_area_release(void *lsa)vfree (lsa); } +static ssize_t event_trigger_store (struct device *dev, +struct device_attribute *attr, +const char *buf,size_t count) +{ +cxl_mock_event_trigger (dev); +return count; +} +static DEVICE_ATTR_WO (event_trigger); + +static struct attribute *cxl_mock_event_attrs[] = { + &dev_attr_event_trigger.attr, +NULL +}; +ATTRIBUTE_GROUPS (cxl_mock_event); +static int cxl_mock_mem_probe (struct platform_device *pdev) {struct device *dev = &pdev->dev;struct cxl_memdev *cxlmd;struct cxl_dev_state *cxlds; +u32 ev_status;void *lsa;int rc;
@@ -281,11 +304,13 @@ static int cxl_mock_mem_probe(struct platform_device *pdev)if (rc)return rc; + ev_status =cxl_mock_add_event_logs (cxlds); + cxlmd =devm_cxl_add_memdev (cxlds);if (IS_ERR (cxlmd))return PTR_ERR (cxlmd); -cxl_mem_get_event_records (cxlds); +__cxl_mem_get_event_records (cxlds, ev_status);if (resource_size (&cxlds->pmem_res) &&IS_ENABLED (CONFIG_CXL_PMEM )) rc =devm_cxl_add_nvdimm (dev, cxlmd);
@@ -293,6 +318,12 @@ static int cxl_mock_mem_probe(struct platform_device *pdev)return 0 ; } +static int cxl_mock_mem_remove (struct platform_device *pdev) +{ +cxl_mock_remove_event_logs (&pdev->dev); +return 0 ; +} +static const struct platform_device_id cxl_mock_mem_ids[] = { { .name ="cxl_mem" , }, { },
@@ -301,9 +332,11 @@ MODULE_DEVICE_TABLE(platform, cxl_mock_mem_ids);static struct platform_driver cxl_mock_mem_driver = { .probe = cxl_mock_mem_probe, + .remove = cxl_mock_mem_remove, .id_table = cxl_mock_mem_ids, .driver = { .name =KBUILD_MODNAME , + .dev_groups = cxl_mock_event_groups, }, };
Signature · 1 line
2.37.2
Re: [PATCH V2 09/11] cxl/test: Add generic mock events
Quoted 31 lines · On Wed, 30 Nov 2022 16:27:17 -0800 [email protected] wrote:
From: Ira Weiny <[email protected]>
Facilitate testing basic Get/Clear Event functionality by creating multiple logs and generic events with made up UUID's.
Data is completely made up with data patterns which should be easy to spot in trace output.
A single sysfs entry resets the event data and triggers collecting the events for testing.
Events are returned one at a time which is within the specification even though it does not exercise the full capabilities of what a device may do.
Test traces are easy to obtain with a small script such as this:
#!/bin/bash -x
devices=
find /sys/devices/platform -name cxl_mem*# Turn on tracing
echo "" >/sys/kernel/tracing/traceecho 1 >/sys/kernel/tracing/events/cxl/enableecho 1 >/sys/kernel/tracing/tracing_on# Generate fake interrupt
for device in $devices; do
echo 1 > $device/event_trigger
done
Quoted 6 lines
# Turn off tracing and report events
echo 0 >/sys/kernel/tracing/tracing_on
cat/sys/kernel/tracing/traceSigned-off-by Ira Weiny <[email protected]>
A minor comment on xarray cleanup inline
Jonathan
tools/testing/cxl/test/events.c line 240
Quoted 5 lines
tools/testing/cxl/test/events.c new file +5
-0+void cxl_mock_remove_event_logs (struct device *dev) +{ +struct mock_event_store *mes; + + mes =xa_erase (&mock_dev_event_store, (unsigned long )dev);
As below, I'd move this into a devm_add_action_or_reset() so that we don't need to deal with doing it manually.
Quoted 2 lines
(quoted diff) +2
-0+} +EXPORT_SYMBOL_GPL(cxl_mock_remove_event_logs);
...
Quoted 10 lines
static int
cxl_mock_mem_probe(struct platform_device *pdev){tools/testing/cxl/test/mem.c +1
-0struct device *dev = &pdev->dev;struct cxl_memdev *cxlmd;struct cxl_dev_state *cxlds; +u32 ev_status;void *lsa;int rc;
tools/testing/cxl/test/mem.c static int cxl_mock_mem_probe(struct platform_device *pdev) · line 307
Quoted 4 lines
tools/testing/cxl/test/mem.c +1
-0@@ -281,11 +304,13 @@ static int cxl_mock_mem_probe(struct platform_device *pdev)if (rc)return rc; + ev_status =cxl_mock_add_event_logs (cxlds);
On error later in this function these leak. Just use devm_add_action_or_reset() inside cxl_mock_add_event_logs() so we don't have to care about that.
Quoted 11 lines
tools/testing/cxl/test/mem.c +2
-1+ cxlmd =devm_cxl_add_memdev (cxlds);if (IS_ERR (cxlmd))return PTR_ERR (cxlmd); -cxl_mem_get_event_records (cxlds); +__cxl_mem_get_event_records (cxlds, ev_status);if (resource_size (&cxlds->pmem_res) &&IS_ENABLED (CONFIG_CXL_PMEM )) rc =devm_cxl_add_nvdimm (dev, cxlmd);
tools/testing/cxl/test/mem.c static int cxl_mock_mem_probe(struct platform_device *pdev) · line 323
Quoted 6 lines
tools/testing/cxl/test/mem.c +3
-0@@ -293,6 +318,12 @@ static int cxl_mock_mem_probe(struct platform_device *pdev)return 0 ; } +static int cxl_mock_mem_remove (struct platform_device *pdev) +{ +cxl_mock_remove_event_logs (&pdev->dev);
Why not use devm_add_action_or_reset()?
Quoted 5 lines
tools/testing/cxl/test/events.c new file +3
-0+return 0 ; +} +
Re: [PATCH V2 09/11] cxl/test: Add generic mock events
Quoted 46 lines · Jonathan Cameron
Quoted 31 lines · On Wed, 30 Nov 2022 16:27:17 -0800 [email protected] wrote:
From: Ira Weiny <[email protected]>
Facilitate testing basic Get/Clear Event functionality by creating multiple logs and generic events with made up UUID's.
Data is completely made up with data patterns which should be easy to spot in trace output.
A single sysfs entry resets the event data and triggers collecting the events for testing.
Events are returned one at a time which is within the specification even though it does not exercise the full capabilities of what a device may do.
Test traces are easy to obtain with a small script such as this:
#!/bin/bash -x
devices=
find /sys/devices/platform -name cxl_mem*# Turn on tracing
echo "" >/sys/kernel/tracing/traceecho 1 >/sys/kernel/tracing/events/cxl/enableecho 1 >/sys/kernel/tracing/tracing_on# Generate fake interrupt
for device in $devices; do
echo 1 > $device/event_trigger
doneQuoted 6 lines
# Turn off tracing and report events
echo 0 >/sys/kernel/tracing/tracing_on
cat/sys/kernel/tracing/traceSigned-off-by Ira Weiny <[email protected]>A minor comment on xarray cleanup inline
Jonathan
Quoted 8 lines
tools/testing/cxl/test/events.c line 240
Quoted 5 lines
tools/testing/cxl/test/events.c new file +5
-0+void cxl_mock_remove_event_logs (struct device *dev) +{ +struct mock_event_store *mes; + + mes =xa_erase (&mock_dev_event_store, (unsigned long )dev);As below, I'd move this into a
devm_add_action_or_reset()so that we don't need to deal with doing it manually.
yea
Quoted 15 lines
Quoted 3 lines
(quoted diff) +2
-0+} +EXPORT_SYMBOL_GPL(cxl_mock_remove_event_logs);...
Quoted 9 lines
static int
cxl_mock_mem_probe(struct platform_device *pdev){tools/testing/cxl/test/mem.c +1
-0struct device *dev = &pdev->dev;struct cxl_memdev *cxlmd;struct cxl_dev_state *cxlds; +u32 ev_status;void *lsa;int rc;
Quoted 8 lines
tools/testing/cxl/test/mem.c static int cxl_mock_mem_probe(struct platform_device *pdev) · line 307
Quoted 5 lines
tools/testing/cxl/test/mem.c +1
-0@@ -281,11 +304,13 @@ static int cxl_mock_mem_probe(struct platform_device *pdev)if (rc)return rc; + ev_status =cxl_mock_add_event_logs (cxlds);On error later in this function these leak. Just use
devm_add_action_or_reset()insidecxl_mock_add_event_logs()so we don't have to care about that.
I see it now. There is an entry in the xarray which is leaked.
Quoted 15 lines
Quoted 15 lines
tools/testing/cxl/test/mem.c +2
-1+ cxlmd =devm_cxl_add_memdev (cxlds);if (IS_ERR (cxlmd))return PTR_ERR (cxlmd); -cxl_mem_get_event_records (cxlds); +__cxl_mem_get_event_records (cxlds, ev_status);if (resource_size (&cxlds->pmem_res) &&IS_ENABLED (CONFIG_CXL_PMEM )) rc =devm_cxl_add_nvdimm (dev, cxlmd);@@ -293,6 +318,12 @@ static int cxl_mock_mem_probe(struct platform_device *pdev)return 0 ; }
Quoted 4 lines
tools/testing/cxl/test/mem.c static int cxl_mock_mem_probe(struct platform_device *pdev) · line 323
Quoted 3 lines
tools/testing/cxl/test/mem.c +3
-0+static int cxl_mock_mem_remove (struct platform_device *pdev) +{ +cxl_mock_remove_event_logs (&pdev->dev);Why not use
devm_add_action_or_reset()?
Yea that is a better pattern.
Ira
RE: [PATCH V2 09/11] cxl/test: Add generic mock events
In reply to [email protected]Quoted 26 lines · ira.weiny@ wrote:
From: Ira Weiny <[email protected]>
Facilitate testing basic Get/Clear Event functionality by creating multiple logs and generic events with made up UUID's.
Data is completely made up with data patterns which should be easy to spot in trace output.
A single sysfs entry resets the event data and triggers collecting the events for testing.
Events are returned one at a time which is within the specification even though it does not exercise the full capabilities of what a device may do.
Test traces are easy to obtain with a small script such as this:
#!/bin/bash -x
devices=
find /sys/devices/platform -name cxl_mem*# Turn on tracing
echo "" >/sys/kernel/tracing/traceecho 1 >/sys/kernel/tracing/events/cxl/enableecho 1 >/sys/kernel/tracing/tracing_on
Quoted 9 lines
# Generate fake interrupt
for device in $devices; do
echo 1 > $device/event_trigger
done# Turn off tracing and report events
echo 0 >/sys/kernel/tracing/tracing_on
cat/sys/kernel/tracing/trace
Nice, should be straightforward to copy-pasta that into a unit test script.
Quoted 76 lines
Signed-off-by Ira Weiny <[email protected]>---
Changes from v1:
Fix up for new structures
Jonathan
Update based on specification discussionChanges from RFC v2:
Adjust to simulate the event status registerChanges from RFC:
Separate out the event code
Adjust for struct changes.
Clean updevm_cxl_mock_event_logs()
Clean up naming and comments
Jonathan
Remove dynamic allocation of event logs Clean up comment
Remove unneeded xarray
Ensureevent_triggersysfs is valid prior to the driver going active.
Dan
Remove the fill/reset event sysfs as these operations can be done together--- drivers/cxl/core/mbox.c | 33 +++-- drivers/cxl/cxlmem.h | 1 + tools/testing/cxl/test/Kbuild | 2 +- tools/testing/cxl/test/events.c | 242 ++++++++++++++++++++++++++++++++ tools/testing/cxl/test/events.h | 9 ++ tools/testing/cxl/test/mem.c | 35 ++++- 6 files changed, 307 insertions(+), 15 deletions(-) create mode 100644 tools/testing/cxl/test/events.c create mode 100644 tools/testing/cxl/test/events.hdrivers/cxl/core/mbox.c +11
-13@@ -886,20 +886,9 @@ static struct cxl_get_event_payload *alloc_event_buf(struct cxl_dev_state *cxldsreturn buf; } -/** -* cxl_mem_get_event_records - Get Event Records from the device -* @cxlds: The device data for the operation -* -* Retrieve all event records available on the device, report them as trace -* events, and clear them. -* -* See CXL rev 3.0 @8.2.9.2.2 Get Event Records -* See CXL rev 3.0 @8.2.9.2.3 Clear Event Records -*/ -void cxl_mem_get_event_records (struct cxl_dev_state *cxlds) +/* Direct call for mock testing */ +void __cxl_mem_get_event_records (struct cxl_dev_state *cxlds,u32 status) { -u32 status =readl (cxlds->regs.status +CXLDEV_DEV_EVENT_STATUS_OFFSET ); -dev_dbg (cxlds->dev,"Reading event logs: %x\n" , status);if (!cxlds->event_buf) {@@ -917,6 +906,24 @@ void cxl_mem_get_event_records(struct cxl_dev_state *cxlds)if (status &CXLDEV_EVENT_STATUS_FATAL )cxl_mem_get_records_log (cxlds,CXL_EVENT_TYPE_FATAL ); } +EXPORT_SYMBOL_NS_GPL (__cxl_mem_get_event_records,CXL ); + +/** +* cxl_mem_get_event_records - Get Event Records from the device +* @cxlds: The device data for the operation +* +* Retrieve all event records available on the device, report them as trace +* events, and clear them. +*
drivers/cxl/core/mbox.c void cxl_mem_get_event_records(struct cxl_dev_state *cxlds) · line 927
Quoted 10 lines
drivers/cxl/core/mbox.c +9
-0+* See CXL rev 3.0 @8.2.9.2.2 Get Event Records +* See CXL rev 3.0 @8.2.9.2.3 Clear Event Records +*/ +void cxl_mem_get_event_records (struct cxl_dev_state *cxlds) +{ +u32 status =readl (cxlds->regs.status +CXLDEV_DEV_EVENT_STATUS_OFFSET ); + +__cxl_mem_get_event_records (cxlds, status); +}EXPORT_SYMBOL_NS_GPL (cxl_mem_get_event_records,CXL );
The unit tests should be minimally disruptive to the mainlie code, and giving them their own private export is outside that spirit. Can you rearrange so that the unit tests don't need their own export? Like all callers pass the status?
Quoted 185 lines
drivers/cxl/core/mbox.c +0
-0int cxl_event_config_msgnums(struct cxl_dev_state *cxlds,drivers/cxl/cxlmem.h +1
-0@@ -565,6 +565,7 @@ void set_exclusive_cxl_commands(struct cxl_dev_state *cxlds, unsigned long *cmdsvoid clear_exclusive_cxl_commands (struct cxl_dev_state *cxlds,unsigned long *cmds);void cxl_mem_get_records_log (struct cxl_dev_state *cxlds,enum cxl_event_log_type type); +void __cxl_mem_get_event_records (struct cxl_dev_state *cxlds,u32 status);void cxl_mem_get_event_records (struct cxl_dev_state *cxlds);int cxl_event_config_msgnums (struct cxl_dev_state *cxlds,struct cxl_event_interrupt_policy *policy);tools/testing/cxl/test/Kbuild +1
-1@@ -7,4 +7,4 @@ obj-m += cxl_mock_mem.o cxl_test-y := cxl.o cxl_mock-y := mock.o -cxl_mock_mem-y := mem.o +cxl_mock_mem-y := mem.o events.otools/testing/cxl/test/events.c new file +155
-0@@ -0,0 +1,242 @@ +// SPDX-License-Identifier: GPL-2.0-only +// Copyright(c) 2022 Intel Corporation. All rights reserved. + +#include <cxlmem.h> +#include <trace/events/cxl.h> + +#include "events.h" + +#define CXL_TEST_EVENT_CNT_MAX 15 + +/* Set a number of events to return at a time for simulation. */ +#define CXL_TEST_EVENT_CNT 3 + +struct mock_event_log { +u16 clear_idx; +u16 cur_idx; +u16 nr_events; +struct cxl_event_record_raw *events[CXL_TEST_EVENT_CNT_MAX ]; +}; + +struct mock_event_store { +struct cxl_dev_state *cxlds; +struct mock_event_log mock_logs[CXL_EVENT_TYPE_MAX ]; +u32 ev_status; +}; + +DEFINE_XARRAY (mock_dev_event_store); + +struct mock_event_log *find_event_log (struct device *dev,int log_type) +{ +struct mock_event_store *mes =xa_load (&mock_dev_event_store, + (unsigned long )dev); + +if (!mes || log_type >=CXL_EVENT_TYPE_MAX ) +return NULL ; +return &mes->mock_logs[log_type]; +} + +struct cxl_event_record_raw *get_cur_event (struct mock_event_log *log) +{ +return log->events[log->cur_idx]; +} + +void reset_event_log (struct mock_event_log *log) +{ + log->cur_idx =0 ; + log->clear_idx =0 ; +} + +/* Handle can never be 0 use 1 based indexing for handle */ +u16 get_clear_handle (struct mock_event_log *log) +{ +return log->clear_idx +1 ; +} + +/* Handle can never be 0 use 1 based indexing for handle */ +__le16 get_cur_event_handle (struct mock_event_log *log) +{ +u16 cur_handle = log->cur_idx +1 ; + +return cpu_to_le16 (cur_handle); +} + +static bool log_empty (struct mock_event_log *log) +{ +return log->cur_idx == log->nr_events; +} + +static void event_store_add_event (struct mock_event_store *mes, +enum cxl_event_log_type log_type, +struct cxl_event_record_raw *event) +{ +struct mock_event_log *log; + +if (WARN_ON (log_type >=CXL_EVENT_TYPE_MAX )) +return ; + + log = &mes->mock_logs[log_type]; +if (WARN_ON (log->nr_events >=CXL_TEST_EVENT_CNT_MAX )) +return ; + + log->events[log->nr_events] = event; + log->nr_events++; +} + +int mock_get_event (struct cxl_dev_state *cxlds,struct cxl_mbox_cmd *cmd) +{ +struct cxl_get_event_payload *pl; +struct mock_event_log *log; +u8 log_type; +int i; + +if (cmd->size_in !=sizeof (log_type)) +return -EINVAL ; + +if (cmd->size_out <struct_size (pl, records,CXL_TEST_EVENT_CNT )) +return -EINVAL ; + + log_type = *((u8 *)cmd->payload_in); +if (log_type >=CXL_EVENT_TYPE_MAX ) +return -EINVAL ; + +memset (cmd->payload_out,0 , cmd->size_out); + + log =find_event_log (cxlds->dev, log_type); +if (!log ||log_empty (log)) +return 0 ; + + pl = cmd->payload_out; + +for (i =0 ; i <CXL_TEST_EVENT_CNT && !log_empty (log); i++) { +memcpy (&pl->records[i],get_cur_event (log),sizeof (pl->records[i])); + pl->records[i].hdr.handle =get_cur_event_handle (log); + log->cur_idx++; + } + + pl->record_count =cpu_to_le16 (i); +if (!log_empty (log)) + pl->flags |=CXL_GET_EVENT_FLAG_MORE_RECORDS ; + +return 0 ; +} +EXPORT_SYMBOL_GPL (mock_get_event); + +int mock_clear_event (struct cxl_dev_state *cxlds,struct cxl_mbox_cmd *cmd) +{ +struct cxl_mbox_clear_event_payload *pl = cmd->payload_in; +struct mock_event_log *log; +u8 log_type = pl->event_log; +u16 handle; +int nr; + +if (log_type >=CXL_EVENT_TYPE_MAX ) +return -EINVAL ; + + log =find_event_log (cxlds->dev, log_type); +if (!log) +return 0 ;/* No mock data in this log */ + +/* +* This check is technically not invalid per the specification AFAICS. +* (The host could 'guess' handles and clear them in order). +* However, this is not good behavior for the host so test it. +*/ +if (log->clear_idx + pl->nr_recs > log->cur_idx) { +dev_err (cxlds->dev, +"Attempting to clear more events than returned!\n" ); +return -EINVAL ; + } + +/* Check handle order prior to clearing events */ +for (nr =0 , handle =get_clear_handle (log); + nr < pl->nr_recs; + nr++, handle++) { +if (handle !=le16_to_cpu (pl->handle[nr])) {
tools/testing/cxl/test/events.c line 165
Quoted 10 lines
tools/testing/cxl/test/events.c new file +10
-0+dev_err (cxlds->dev,"Clearing events out of order\n" ); +return -EINVAL ; + } + } + +/* Clear events */ + log->clear_idx += pl->nr_recs; +return 0 ; +} +EXPORT_SYMBOL_GPL (mock_clear_event);
Why is this exported? The caller is in the same compilation unit.
Quoted 8 lines
tools/testing/cxl/test/events.c new file +8
-0+ +void cxl_mock_event_trigger (struct device *dev) +{ +struct mock_event_store *mes =xa_load (&mock_dev_event_store, + (unsigned long )dev); +int i; + +for (i =CXL_EVENT_TYPE_INFO ; i <CXL_EVENT_TYPE_MAX ; i++) {
tools/testing/cxl/test/events.c line 183
Quoted 10 lines
tools/testing/cxl/test/events.c new file +10
-0+struct mock_event_log *log; + + log =find_event_log (dev, i); +if (log) +reset_event_log (log); + } + +__cxl_mem_get_event_records (mes->cxlds, mes->ev_status); +} +EXPORT_SYMBOL_GPL (cxl_mock_event_trigger);
Ditto on the export question.
Quoted 23 lines
tools/testing/cxl/test/events.c new file +23
-0+ +struct cxl_event_record_raw maint_needed = { + .hdr = { + .id =UUID_INIT (0xDEADBEEF ,0xCAFE ,0xBABE , +0xa5 ,0x5a ,0xa5 ,0x5a ,0xa5 ,0xa5 ,0x5a ,0xa5 ), + .length =sizeof (struct cxl_event_record_raw ), + .flags[0 ] =CXL_EVENT_RECORD_FLAG_MAINT_NEEDED , +/* .handle = Set dynamically */ + .related_handle =cpu_to_le16 (0xa5b6 ), + }, + .data = {0xDE ,0xAD ,0xBE ,0xEF }, +}; + +struct cxl_event_record_raw hardware_replace = { + .hdr = { + .id =UUID_INIT (0xBABECAFE ,0xBEEF ,0xDEAD , +0xa5 ,0x5a ,0xa5 ,0x5a ,0xa5 ,0xa5 ,0x5a ,0xa5 ), + .length =sizeof (struct cxl_event_record_raw ), + .flags[0 ] =CXL_EVENT_RECORD_FLAG_HW_REPLACE , +/* .handle = Set dynamically */ + .related_handle =cpu_to_le16 (0xb6a5 ), + }, + .data = {0xDE ,0xAD ,0xBE ,0xEF },
tools/testing/cxl/test/events.c line 216
Quoted 10 lines
tools/testing/cxl/test/events.c new file +10
-0+}; + +u32 cxl_mock_add_event_logs (struct cxl_dev_state *cxlds) +{ +struct device *dev = cxlds->dev; +struct mock_event_store *mes; + + mes =devm_kzalloc (dev,sizeof (*mes),GFP_KERNEL ); +if (WARN_ON (!mes)) +return 0 ;
Why a WARN? Perhaps make this return 'struct mock_event_store *' or an ERR_PTR.
tools/testing/cxl/test/events.c line 222
Quoted 6 lines
tools/testing/cxl/test/events.c new file +6
-0+ mes->cxlds = cxlds; + +if (xa_insert (&mock_dev_event_store, (unsigned long )dev, mes, +GFP_KERNEL )) { +dev_err (dev,"Event store not available for %s\n" , +dev_name (dev));
Downgraded to a dev_err() rather than a WARN is nice, but I think this just wants to be something that hangs off of dev_get_drvdata(). See what Dave and I came up with for his extra variables in the security unit test. You might want to base on the for-6.2/cxl-security branch since there are some other conflicts with dev_groups too.
tools/testing/cxl/test/events.c line 234
Quoted 12 lines
tools/testing/cxl/test/events.c new file +12
-0+return 0 ; + } + +event_store_add_event (mes,CXL_EVENT_TYPE_INFO , &maint_needed); + mes->ev_status |=CXLDEV_EVENT_STATUS_INFO ; + +event_store_add_event (mes,CXL_EVENT_TYPE_FATAL , &hardware_replace); + mes->ev_status |=CXLDEV_EVENT_STATUS_FATAL ; + +return mes->ev_status; +} +EXPORT_SYMBOL_GPL (cxl_mock_add_event_logs);
Another export to remove.
tools/testing/cxl/test/events.c line 236
Quoted 2 lines
tools/testing/cxl/test/events.c new file +2
-0+ +void cxl_mock_remove_event_logs (struct device *dev)
Unnecessary if cxl_mock_event_store move to dev_get_drvdata().
Quoted 90 lines
tools/testing/cxl/test/events.c new file +6
-0+{ +struct mock_event_store *mes; + + mes =xa_erase (&mock_dev_event_store, (unsigned long )dev); +} +EXPORT_SYMBOL_GPL (cxl_mock_remove_event_logs);tools/testing/cxl/test/events.h new file +9
-0@@ -0,0 +1,9 @@ +/* SPDX-License-Identifier: GPL-2.0 */ + +#include <cxlmem.h> + +int mock_get_event (struct cxl_dev_state *cxlds,struct cxl_mbox_cmd *cmd); +int mock_clear_event (struct cxl_dev_state *cxlds,struct cxl_mbox_cmd *cmd); +u32 cxl_mock_add_event_logs (struct cxl_dev_state *cxlds); +void cxl_mock_remove_event_logs (struct device *dev); +void cxl_mock_event_trigger (struct device *dev);tools/testing/cxl/test/mem.c +26
-1@@ -8,6 +8,7 @@#include <linux/sizes.h> #include <linux/bits.h> #include <cxlmem.h> +#include "events.h" #define LSA_SIZE SZ_128K#define DEV_SIZE SZ_2G@@ -224,6 +225,12 @@ static int cxl_mock_mbox_send(struct cxl_dev_state *cxlds, struct cxl_mbox_cmd *case CXL_MBOX_OP_GET_PARTITION_INFO : rc =mock_partition_info (cxlds, cmd);break ; +case CXL_MBOX_OP_GET_EVENT_RECORD : + rc =mock_get_event (cxlds, cmd); +break ; +case CXL_MBOX_OP_CLEAR_EVENT_RECORD : + rc =mock_clear_event (cxlds, cmd); +break ;case CXL_MBOX_OP_SET_LSA : rc =mock_set_lsa (cxlds, cmd);break ;@@ -245,11 +252,27 @@ static void label_area_release(void *lsa)vfree (lsa); } +static ssize_t event_trigger_store (struct device *dev, +struct device_attribute *attr, +const char *buf,size_t count) +{ +cxl_mock_event_trigger (dev); +return count; +} +static DEVICE_ATTR_WO (event_trigger); + +static struct attribute *cxl_mock_event_attrs[] = { + &dev_attr_event_trigger.attr, +NULL +}; +ATTRIBUTE_GROUPS (cxl_mock_event); +static int cxl_mock_mem_probe (struct platform_device *pdev) {struct device *dev = &pdev->dev;struct cxl_memdev *cxlmd;struct cxl_dev_state *cxlds; +u32 ev_status;void *lsa;int rc;@@ -281,11 +304,13 @@ static int cxl_mock_mem_probe(struct platform_device *pdev)if (rc)return rc; + ev_status =cxl_mock_add_event_logs (cxlds); + cxlmd =devm_cxl_add_memdev (cxlds);if (IS_ERR (cxlmd))return PTR_ERR (cxlmd); -cxl_mem_get_event_records (cxlds); +__cxl_mem_get_event_records (cxlds, ev_status);if (resource_size (&cxlds->pmem_res) &&IS_ENABLED (CONFIG_CXL_PMEM )) rc =devm_cxl_add_nvdimm (dev, cxlmd);
tools/testing/cxl/test/mem.c static int cxl_mock_mem_probe(struct platform_device *pdev) · line 321
Quoted 4 lines
tools/testing/cxl/test/mem.c +1
-0@@ -293,6 +318,12 @@ static int cxl_mock_mem_probe(struct platform_device *pdev)return 0 ; } +static int cxl_mock_mem_remove (struct platform_device *pdev)
It's moot with the comment above about moving to drvdata, but I otherwise would have expected:
devm_add_action_or_reset(dev, cxl_mock_remove_event_logs, mes);
...in the probe path. Otherwse, I just got rid of the only .revove() handler in the CXL subsystem the for-6.2/cxl-rch branch.
Quoted 9 lines
tools/testing/cxl/test/mem.c +5
-0+{ +cxl_mock_remove_event_logs (&pdev->dev); +return 0 ; +} +static const struct platform_device_id cxl_mock_mem_ids[] = { { .name ="cxl_mem" , }, { },
tools/testing/cxl/test/mem.c MODULE_DEVICE_TABLE(platform, cxl_mock_mem_ids); · line 339
Quoted 8 lines
tools/testing/cxl/test/mem.c +2
-0@@ -301,9 +332,11 @@ MODULE_DEVICE_TABLE(platform, cxl_mock_mem_ids);static struct platform_driver cxl_mock_mem_driver = { .probe = cxl_mock_mem_probe, + .remove = cxl_mock_mem_remove, .id_table = cxl_mock_mem_ids, .driver = { .name =KBUILD_MODNAME , + .dev_groups = cxl_mock_event_groups,
Looks good, but you'll need to append yours to the ones that Dave added for the security state machine.
Quoted 6 lines
tools/testing/cxl/test/mem.c +0
-0}, };Signature · 1 line
2.37.2
PATCH v2 10/11 cxl/test: Add specific events
In reply to [email protected]From: Ira Weiny <[email protected]>
Each type of event has different trace point outputs.
Add mock General Media Event, DRAM event, and Memory Module Event records to the mock list of events returned.
---
Changes from V1:
Jonathan
use put_unaligned_le16()
fix spacing
Changes from RFC:
Adjust for struct changes
adjust for unaligned fields
--- tools/testing/cxl/test/events.c | 73 +++++++++++++++++++++++++++++++++ 1 file changed, 73 insertions(+)
tools/testing/cxl/test/events.c +73 -0
@@ -206,6 +206,66 @@ struct cxl_event_record_raw hardware_replace = { .data = { 0xDE, 0xAD, 0xBE, 0xEF }, }; +struct cxl_event_gen_media gen_media = { + .hdr = { + .id =UUID_INIT (0xfbcd0a77 ,0xc260 ,0x417f , +0x85 ,0xa9 ,0x08 ,0x8b ,0x16 ,0x21 ,0xeb ,0xa6 ), + .length =sizeof (struct cxl_event_gen_media ), + .flags[0 ] =CXL_EVENT_RECORD_FLAG_PERMANENT , +/* .handle = Set dynamically */ + .related_handle =cpu_to_le16 (0 ), + }, + .phys_addr =cpu_to_le64 (0x2000 ), + .descriptor =CXL_GMER_EVT_DESC_UNCORECTABLE_EVENT , + .type =CXL_GMER_MEM_EVT_TYPE_DATA_PATH_ERROR , + .transaction_type =CXL_GMER_TRANS_HOST_WRITE , +/* .validity_flags = <set below> */ + .channel =1 , + .rank =30 +}; + +struct cxl_event_dram dram = { + .hdr = { + .id =UUID_INIT (0x601dcbb3 ,0x9c06 ,0x4eab , +0xb8 ,0xaf ,0x4e ,0x9b ,0xfb ,0x5c ,0x96 ,0x24 ), + .length =sizeof (struct cxl_event_dram ), + .flags[0 ] =CXL_EVENT_RECORD_FLAG_PERF_DEGRADED , +/* .handle = Set dynamically */ + .related_handle =cpu_to_le16 (0 ), + }, + .phys_addr =cpu_to_le64 (0x8000 ), + .descriptor =CXL_GMER_EVT_DESC_THRESHOLD_EVENT , + .type =CXL_GMER_MEM_EVT_TYPE_INV_ADDR , + .transaction_type =CXL_GMER_TRANS_INTERNAL_MEDIA_SCRUB , +/* .validity_flags = <set below> */ + .channel =1 , + .bank_group =5 , + .bank =2 , + .column = {0xDE ,0xAD }, +}; + +struct cxl_event_mem_module mem_module = { + .hdr = { + .id =UUID_INIT (0xfe927475 ,0xdd59 ,0x4339 , +0xa5 ,0x86 ,0x79 ,0xba ,0xb1 ,0x13 ,0xb7 ,0x74 ), + .length =sizeof (struct cxl_event_mem_module ), +/* .handle = Set dynamically */ + .related_handle =cpu_to_le16 (0 ), + }, + .event_type =CXL_MMER_TEMP_CHANGE , + .info = { + .health_status =CXL_DHI_HS_PERFORMANCE_DEGRADED , + .media_status =CXL_DHI_MS_ALL_DATA_LOST , + .add_status = (CXL_DHI_AS_CRITICAL <<2 ) | + (CXL_DHI_AS_WARNING <<4 ) | + (CXL_DHI_AS_WARNING <<5 ), + .device_temp = {0xDE ,0xAD }, + .dirty_shutdown_cnt = {0xde ,0xad ,0xbe ,0xef }, + .cor_vol_err_cnt = {0xde ,0xad ,0xbe ,0xef }, + .cor_per_err_cnt = {0xde ,0xad ,0xbe ,0xef }, + } +}; +u32 cxl_mock_add_event_logs (struct cxl_dev_state *cxlds) {struct device *dev = cxlds->dev;
@@ -223,10 +283,23 @@ u32 cxl_mock_add_event_logs(struct cxl_dev_state *cxlds)return 0 ; } +put_unaligned_le16 (CXL_GMER_VALID_CHANNEL |CXL_GMER_VALID_RANK , + &gen_media.validity_flags); + +put_unaligned_le16 (CXL_DER_VALID_CHANNEL |CXL_DER_VALID_BANK_GROUP | +CXL_DER_VALID_BANK |CXL_DER_VALID_COLUMN , + &dram.validity_flags); +event_store_add_event (mes,CXL_EVENT_TYPE_INFO , &maint_needed); +event_store_add_event (mes,CXL_EVENT_TYPE_INFO , + (struct cxl_event_record_raw *)&gen_media); +event_store_add_event (mes,CXL_EVENT_TYPE_INFO , + (struct cxl_event_record_raw *)&mem_module); mes->ev_status |=CXLDEV_EVENT_STATUS_INFO ;event_store_add_event (mes,CXL_EVENT_TYPE_FATAL , &hardware_replace); +event_store_add_event (mes,CXL_EVENT_TYPE_FATAL , + (struct cxl_event_record_raw *)&dram); mes->ev_status |=CXLDEV_EVENT_STATUS_FATAL ;return mes->ev_status;
Signature · 1 line
2.37.2
Re: [PATCH V2 10/11] cxl/test: Add specific events
Quoted 9 lines · On 11/30/2022 5:27 PM, [email protected] wrote:
From: Ira Weiny <[email protected]>
Each type of event has different trace point outputs.
Add mock General Media Event, DRAM event, and Memory Module Event records to the mock list of events returned.
Reviewed-by Jonathan Cameron <[email protected]>Signed-off-by Ira Weiny <[email protected]>
Quoted 109 lines
---
Changes from V1:
Jonathan
useput_unaligned_le16()
fix spacingChanges from RFC:
Adjust for struct changes
adjust for unaligned fields--- tools/testing/cxl/test/events.c | 73 +++++++++++++++++++++++++++++++++ 1 file changed, 73 insertions(+)tools/testing/cxl/test/events.c +73
-0@@ -206,6 +206,66 @@ struct cxl_event_record_raw hardware_replace = { .data = { 0xDE, 0xAD, 0xBE, 0xEF }, }; +struct cxl_event_gen_media gen_media = { + .hdr = { + .id =UUID_INIT (0xfbcd0a77 ,0xc260 ,0x417f , +0x85 ,0xa9 ,0x08 ,0x8b ,0x16 ,0x21 ,0xeb ,0xa6 ), + .length =sizeof (struct cxl_event_gen_media ), + .flags[0 ] =CXL_EVENT_RECORD_FLAG_PERMANENT , +/* .handle = Set dynamically */ + .related_handle =cpu_to_le16 (0 ), + }, + .phys_addr =cpu_to_le64 (0x2000 ), + .descriptor =CXL_GMER_EVT_DESC_UNCORECTABLE_EVENT , + .type =CXL_GMER_MEM_EVT_TYPE_DATA_PATH_ERROR , + .transaction_type =CXL_GMER_TRANS_HOST_WRITE , +/* .validity_flags = <set below> */ + .channel =1 , + .rank =30 +}; + +struct cxl_event_dram dram = { + .hdr = { + .id =UUID_INIT (0x601dcbb3 ,0x9c06 ,0x4eab , +0xb8 ,0xaf ,0x4e ,0x9b ,0xfb ,0x5c ,0x96 ,0x24 ), + .length =sizeof (struct cxl_event_dram ), + .flags[0 ] =CXL_EVENT_RECORD_FLAG_PERF_DEGRADED , +/* .handle = Set dynamically */ + .related_handle =cpu_to_le16 (0 ), + }, + .phys_addr =cpu_to_le64 (0x8000 ), + .descriptor =CXL_GMER_EVT_DESC_THRESHOLD_EVENT , + .type =CXL_GMER_MEM_EVT_TYPE_INV_ADDR , + .transaction_type =CXL_GMER_TRANS_INTERNAL_MEDIA_SCRUB , +/* .validity_flags = <set below> */ + .channel =1 , + .bank_group =5 , + .bank =2 , + .column = {0xDE ,0xAD }, +}; + +struct cxl_event_mem_module mem_module = { + .hdr = { + .id =UUID_INIT (0xfe927475 ,0xdd59 ,0x4339 , +0xa5 ,0x86 ,0x79 ,0xba ,0xb1 ,0x13 ,0xb7 ,0x74 ), + .length =sizeof (struct cxl_event_mem_module ), +/* .handle = Set dynamically */ + .related_handle =cpu_to_le16 (0 ), + }, + .event_type =CXL_MMER_TEMP_CHANGE , + .info = { + .health_status =CXL_DHI_HS_PERFORMANCE_DEGRADED , + .media_status =CXL_DHI_MS_ALL_DATA_LOST , + .add_status = (CXL_DHI_AS_CRITICAL <<2 ) | + (CXL_DHI_AS_WARNING <<4 ) | + (CXL_DHI_AS_WARNING <<5 ), + .device_temp = {0xDE ,0xAD }, + .dirty_shutdown_cnt = {0xde ,0xad ,0xbe ,0xef }, + .cor_vol_err_cnt = {0xde ,0xad ,0xbe ,0xef }, + .cor_per_err_cnt = {0xde ,0xad ,0xbe ,0xef }, + } +}; +u32 cxl_mock_add_event_logs (struct cxl_dev_state *cxlds) {struct device *dev = cxlds->dev;@@ -223,10 +283,23 @@ u32 cxl_mock_add_event_logs(struct cxl_dev_state *cxlds)return 0 ; } +put_unaligned_le16 (CXL_GMER_VALID_CHANNEL |CXL_GMER_VALID_RANK , + &gen_media.validity_flags); + +put_unaligned_le16 (CXL_DER_VALID_CHANNEL |CXL_DER_VALID_BANK_GROUP | +CXL_DER_VALID_BANK |CXL_DER_VALID_COLUMN , + &dram.validity_flags); +event_store_add_event (mes,CXL_EVENT_TYPE_INFO , &maint_needed); +event_store_add_event (mes,CXL_EVENT_TYPE_INFO , + (struct cxl_event_record_raw *)&gen_media); +event_store_add_event (mes,CXL_EVENT_TYPE_INFO , + (struct cxl_event_record_raw *)&mem_module); mes->ev_status |=CXLDEV_EVENT_STATUS_INFO ;event_store_add_event (mes,CXL_EVENT_TYPE_FATAL , &hardware_replace); +event_store_add_event (mes,CXL_EVENT_TYPE_FATAL , + (struct cxl_event_record_raw *)&dram); mes->ev_status |=CXLDEV_EVENT_STATUS_FATAL ;return mes->ev_status;
PATCH v2 11/11 cxl/test: Simulate event log overflow
In reply to [email protected]From: Ira Weiny <[email protected]>
Log overflow is marked by a separate trace message.
Simulate a log with lots of messages and flag overflow until it is drained a bit.
---
Changes from RFC
Adjust for new struct changes
--- tools/testing/cxl/test/events.c | 49 ++++++++++++++++++++++++++++++++- 1 file changed, 48 insertions(+), 1 deletion(-)
tools/testing/cxl/test/events.c +48 -1
@@ -15,6 +15,8 @@ struct mock_event_log {u16 clear_idx;u16 cur_idx;u16 nr_events; +u16 nr_overflow; +u16 overflow_reset;struct cxl_event_record_raw *events[CXL_TEST_EVENT_CNT_MAX ]; };
@@ -45,6 +47,7 @@ void reset_event_log(struct mock_event_log *log) { log->cur_idx =0 ; log->clear_idx =0 ; + log->nr_overflow = log->overflow_reset; }/* Handle can never be 0 use 1 based indexing for handle */
@@ -76,8 +79,12 @@ static void event_store_add_event(struct mock_event_store *mes,return ; log = &mes->mock_logs[log_type]; -if (WARN_ON (log->nr_events >=CXL_TEST_EVENT_CNT_MAX )) + +if ((log->nr_events +1 ) >CXL_TEST_EVENT_CNT_MAX ) { + log->nr_overflow++; + log->overflow_reset = log->nr_overflow;return ; + } log->events[log->nr_events] = event; log->nr_events++;
@@ -87,6 +94,7 @@ int mock_get_event(struct cxl_dev_state *cxlds, struct cxl_mbox_cmd *cmd) {struct cxl_get_event_payload *pl;struct mock_event_log *log; +u16 nr_overflow;u8 log_type;int i;
@@ -118,6 +126,21 @@ int mock_get_event(struct cxl_dev_state *cxlds, struct cxl_mbox_cmd *cmd)if (!log_empty (log)) pl->flags |=CXL_GET_EVENT_FLAG_MORE_RECORDS ; +if (log->nr_overflow) { +u64 ns; + + pl->flags |=CXL_GET_EVENT_FLAG_OVERFLOW ; + pl->overflow_err_count =cpu_to_le16 (nr_overflow); + ns =ktime_get_real_ns (); + ns -=5000000000 ;/* 5s ago */ + pl->first_overflow_timestamp =cpu_to_le64 (ns); + ns =ktime_get_real_ns (); + ns -=1000000000 ;/* 1s ago */ + pl->last_overflow_timestamp =cpu_to_le64 (ns); + + log->nr_overflow =0 ; + } +return 0 ; }EXPORT_SYMBOL_GPL (mock_get_event);
@@ -297,6 +320,30 @@ u32 cxl_mock_add_event_logs(struct cxl_dev_state *cxlds) (struct cxl_event_record_raw *)&mem_module); mes->ev_status |=CXLDEV_EVENT_STATUS_INFO ; +event_store_add_event (mes,CXL_EVENT_TYPE_FAIL , &maint_needed); +event_store_add_event (mes,CXL_EVENT_TYPE_FAIL , &hardware_replace); +event_store_add_event (mes,CXL_EVENT_TYPE_FAIL , + (struct cxl_event_record_raw *)&dram); +event_store_add_event (mes,CXL_EVENT_TYPE_FAIL , + (struct cxl_event_record_raw *)&gen_media); +event_store_add_event (mes,CXL_EVENT_TYPE_FAIL , + (struct cxl_event_record_raw *)&mem_module); +event_store_add_event (mes,CXL_EVENT_TYPE_FAIL , &hardware_replace); +event_store_add_event (mes,CXL_EVENT_TYPE_FAIL , + (struct cxl_event_record_raw *)&dram); +/* Overflow this log */ +event_store_add_event (mes,CXL_EVENT_TYPE_FAIL , &hardware_replace); +event_store_add_event (mes,CXL_EVENT_TYPE_FAIL , &hardware_replace); +event_store_add_event (mes,CXL_EVENT_TYPE_FAIL , &hardware_replace); +event_store_add_event (mes,CXL_EVENT_TYPE_FAIL , &hardware_replace); +event_store_add_event (mes,CXL_EVENT_TYPE_FAIL , &hardware_replace); +event_store_add_event (mes,CXL_EVENT_TYPE_FAIL , &hardware_replace); +event_store_add_event (mes,CXL_EVENT_TYPE_FAIL , &hardware_replace); +event_store_add_event (mes,CXL_EVENT_TYPE_FAIL , &hardware_replace); +event_store_add_event (mes,CXL_EVENT_TYPE_FAIL , &hardware_replace); +event_store_add_event (mes,CXL_EVENT_TYPE_FAIL , &hardware_replace); + mes->ev_status |=CXLDEV_EVENT_STATUS_FAIL ; +event_store_add_event (mes,CXL_EVENT_TYPE_FATAL , &hardware_replace);event_store_add_event (mes,CXL_EVENT_TYPE_FATAL , (struct cxl_event_record_raw *)&dram);
Signature · 1 line
2.37.2
Re: [PATCH V2 11/11] cxl/test: Simulate event log overflow
Quoted 9 lines · On 11/30/2022 5:27 PM, [email protected] wrote:
From: Ira Weiny <[email protected]>
Log overflow is marked by a separate trace message.
Simulate a log with lots of messages and flag overflow until it is drained a bit.
Reviewed-by Jonathan Cameron <[email protected]>Signed-off-by Ira Weiny <[email protected]>
Quoted 104 lines
---
Changes from RFC
Adjust for new struct changes--- tools/testing/cxl/test/events.c | 49 ++++++++++++++++++++++++++++++++- 1 file changed, 48 insertions(+), 1 deletion(-)tools/testing/cxl/test/events.c +48
-1@@ -15,6 +15,8 @@ struct mock_event_log {u16 clear_idx;u16 cur_idx;u16 nr_events; +u16 nr_overflow; +u16 overflow_reset;struct cxl_event_record_raw *events[CXL_TEST_EVENT_CNT_MAX ]; };@@ -45,6 +47,7 @@ void reset_event_log(struct mock_event_log *log) { log->cur_idx =0 ; log->clear_idx =0 ; + log->nr_overflow = log->overflow_reset; }/* Handle can never be 0 use 1 based indexing for handle */ @@ -76,8 +79,12 @@ static void event_store_add_event(struct mock_event_store *mes,return ; log = &mes->mock_logs[log_type]; -if (WARN_ON (log->nr_events >=CXL_TEST_EVENT_CNT_MAX )) + +if ((log->nr_events +1 ) >CXL_TEST_EVENT_CNT_MAX ) { + log->nr_overflow++; + log->overflow_reset = log->nr_overflow;return ; + } log->events[log->nr_events] = event; log->nr_events++;@@ -87,6 +94,7 @@ int mock_get_event(struct cxl_dev_state *cxlds, struct cxl_mbox_cmd *cmd) {struct cxl_get_event_payload *pl;struct mock_event_log *log; +u16 nr_overflow;u8 log_type;int i;@@ -118,6 +126,21 @@ int mock_get_event(struct cxl_dev_state *cxlds, struct cxl_mbox_cmd *cmd)if (!log_empty (log)) pl->flags |=CXL_GET_EVENT_FLAG_MORE_RECORDS ; +if (log->nr_overflow) { +u64 ns; + + pl->flags |=CXL_GET_EVENT_FLAG_OVERFLOW ; + pl->overflow_err_count =cpu_to_le16 (nr_overflow); + ns =ktime_get_real_ns (); + ns -=5000000000 ;/* 5s ago */ + pl->first_overflow_timestamp =cpu_to_le64 (ns); + ns =ktime_get_real_ns (); + ns -=1000000000 ;/* 1s ago */ + pl->last_overflow_timestamp =cpu_to_le64 (ns); + + log->nr_overflow =0 ; + } +return 0 ; }EXPORT_SYMBOL_GPL (mock_get_event);@@ -297,6 +320,30 @@ u32 cxl_mock_add_event_logs(struct cxl_dev_state *cxlds) (struct cxl_event_record_raw *)&mem_module); mes->ev_status |=CXLDEV_EVENT_STATUS_INFO ; +event_store_add_event (mes,CXL_EVENT_TYPE_FAIL , &maint_needed); +event_store_add_event (mes,CXL_EVENT_TYPE_FAIL , &hardware_replace); +event_store_add_event (mes,CXL_EVENT_TYPE_FAIL , + (struct cxl_event_record_raw *)&dram); +event_store_add_event (mes,CXL_EVENT_TYPE_FAIL , + (struct cxl_event_record_raw *)&gen_media); +event_store_add_event (mes,CXL_EVENT_TYPE_FAIL , + (struct cxl_event_record_raw *)&mem_module); +event_store_add_event (mes,CXL_EVENT_TYPE_FAIL , &hardware_replace); +event_store_add_event (mes,CXL_EVENT_TYPE_FAIL , + (struct cxl_event_record_raw *)&dram); +/* Overflow this log */ +event_store_add_event (mes,CXL_EVENT_TYPE_FAIL , &hardware_replace); +event_store_add_event (mes,CXL_EVENT_TYPE_FAIL , &hardware_replace); +event_store_add_event (mes,CXL_EVENT_TYPE_FAIL , &hardware_replace); +event_store_add_event (mes,CXL_EVENT_TYPE_FAIL , &hardware_replace); +event_store_add_event (mes,CXL_EVENT_TYPE_FAIL , &hardware_replace); +event_store_add_event (mes,CXL_EVENT_TYPE_FAIL , &hardware_replace); +event_store_add_event (mes,CXL_EVENT_TYPE_FAIL , &hardware_replace); +event_store_add_event (mes,CXL_EVENT_TYPE_FAIL , &hardware_replace); +event_store_add_event (mes,CXL_EVENT_TYPE_FAIL , &hardware_replace); +event_store_add_event (mes,CXL_EVENT_TYPE_FAIL , &hardware_replace); + mes->ev_status |=CXLDEV_EVENT_STATUS_FAIL ; +event_store_add_event (mes,CXL_EVENT_TYPE_FATAL , &hardware_replace);event_store_add_event (mes,CXL_EVENT_TYPE_FATAL , (struct cxl_event_record_raw *)&dram);