Skip to contentSkip to conversation

Add support for Amazon's Annapurna Labs EDAC for L1/L2

PATCH Superseded Hanna Hawa 2 participants 8 messages Oct 7, 2019 – Oct 10, 2019

Subsystems (2)

v6 of 9 Superseded Latest is v9 →

lore
Share & tools
lore

Hanna Hawa

author
#0 lore raw

v6 cover Add support for Amazon's Annapurna Labs EDAC for L1/L2

This series adds L1 cache and L2 cache error detection and correction support for Amazon's Annapurna Labs SoCs.

Alpine SoCs supports L1 and L2 single bit correction and two bits detection capability based on ARM implementation.

The CPU cores in the SoC are the same and all of them support ECC.

Changes since v5:
-----------------
- Use top-level machine compatible to bind the EDAC device
- Remove DT bindings
- Add initcall to create platform device and register the edac driver
- follow 'next-level-cache' phandle to create CPU topology for L2 driver
- Change the driver to be tristate
- Move register read to function flow
- EXPORT_SYMBOL_GPL of_find_next_cache_node

Changes since v4:
-----------------
- Added include for cpumask.h in al_l2_edac.c
- Fix RAMID error print according to ARM TRM
- Use for_each_possible_cpu() to parse information for DT.
- Add missing of_node_put() call.

Changes since v3:
-----------------
- Added include for smp.h sysreg.h
- Use scnprintf instead of snprintf
- Move write_sysreg_s after valid check to minimize the window between read/write.
- Use IS_ERR_OR_NULL instead of IS_ERR, because edac_device_alloc_ctl_info may return NULL.

Changes since v2:
-----------------
- Use BIT for single bit instead of GENMASK
- Use BIT_ULL and GENMASK_ULL for 64bit vector
- Fix the mod_name/ctrl_name.

Changes since v1:
-----------------
- Split into two drivers
- Get cpu-mask according to l2-cache handler from devicetree
- Remove parameter casting
- Use GENMASK() in bit mask
- Use FIELD_GET()
- Update define description PLRU_RAM -> PF_RAM
- Use sys_reg() and read_sysreg_s()
- Remove all write/read wrappers
- Check fatal field to set if the error correctable or not
- Remove un-relevant information from error prints.
- Update smp_call_function_single() call function to wait
- remove usage of get_online_cpus/put_online_cpus
- Use on_each_cpu() and smp_call_function_any() instead of loop with for_each_cpu.
- use buffer for error prints and pass to edac API
- Remove edac_op_state set
- Add for loop to report on repeated errors of the same type
- Fix error name of the TLB to be L2_TLB as written in ARM TRM
- Minor change in Kconfig
- Minor changes in commit message

Hanna Hawa (3):
edac: Add support for Amazon's Annapurna Labs L1 EDAC
of: EXPORT_SYMBOL_GPL of_find_next_cache_node
edac: Add support for Amazon's Annapurna Labs L2 EDAC

 MAINTAINERS               |  10 ++
 drivers/edac/Kconfig      |  16 +++
 drivers/edac/Makefile     |   2 +
 drivers/edac/al_l1_edac.c | 190 +++++++++++++++++++++++++++++
 drivers/edac/al_l2_edac.c | 251 ++++++++++++++++++++++++++++++++++++++
 drivers/of/base.c         |   1 +
 6 files changed, 470 insertions(+)
 create mode 100644 drivers/edac/al_l1_edac.c
 create mode 100644 drivers/edac/al_l2_edac.c
Signature · 1 line
2.17.1
author
#1 lore raw

PATCH v6 1/3 edac: Add support for Amazon's Annapurna Labs L1 EDAC

Adds support for Amazon's Annapurna Labs L1 EDAC driver to detect and report L1 errors.

---
 MAINTAINERS               |   5 +
 drivers/edac/Kconfig      |   8 ++
 drivers/edac/Makefile     |   1 +
 drivers/edac/al_l1_edac.c | 190 ++++++++++++++++++++++++++++++++++++++
 4 files changed, 204 insertions(+)
 create mode 100644 drivers/edac/al_l1_edac.c
MAINTAINERS +5 -0
index 77eae44bf5de..7887a62dc843 100644
@@ -743,6 +743,11 @@ F:	drivers/tty/serial/altera_jtaguart.c
 F:	include/linux/altera_uart.h
 F:	include/linux/altera_jtaguart.h
 
+AMAZON ANNAPURNA LABS L1 EDAC
+M:	Hanna Hawa <[email protected]>
+S:	Maintained
+F:	drivers/edac/al_l1_edac.c
+
 AMAZON ANNAPURNA LABS THERMAL MMIO DRIVER
 M:	Talel Shenhar <[email protected]>
 S:	Maintained
drivers/edac/Kconfig +8 -0
index 200c04ce5b0e..e8161d7c7469 100644
@@ -74,6 +74,14 @@ config EDAC_GHES
 
 	  In doubt, say 'Y'.
 
+config EDAC_AL_L1
+	tristate "Amazon's Annapurna Labs L1 EDAC"
+	depends on ARCH_ALPINE
+	help
+	  Support for L1 error detection and correction
+	  for Amazon's Annapurna Labs SoCs.
+	  This driver detects errors of L1 caches.
+
 config EDAC_AMD64
 	tristate "AMD64 (Opteron, Athlon64)"
 	depends on AMD_NB && EDAC_DECODE_MCE
drivers/edac/Makefile +1 -0
index 165ca65e1a3a..caa2dc91e8a0 100644
@@ -22,6 +22,7 @@ obj-$(CONFIG_EDAC_GHES)			+= ghes_edac.o
 edac_mce_amd-y				:= mce_amd.o
 obj-$(CONFIG_EDAC_DECODE_MCE)		+= edac_mce_amd.o
 
+obj-$(CONFIG_EDAC_AL_L1)		+= al_l1_edac.o
 obj-$(CONFIG_EDAC_AMD76X)		+= amd76x_edac.o
 obj-$(CONFIG_EDAC_CPC925)		+= cpc925_edac.o
 obj-$(CONFIG_EDAC_I5000)		+= i5000_edac.o
drivers/edac/al_l1_edac.c new file +190 -0

190 lines not on this page · raw message

Signature · 1 line
2.17.1
author
#2 lore raw

PATCH v6 2/3 of: EXPORT_SYMBOL_GPL of_find_next_cache_node

In reply to Hanna Hawa

Make of_find_next_cache_node() available for modules.

---
 drivers/of/base.c | 1 +
 1 file changed, 1 insertion(+)
drivers/of/base.c +1 -0
index 20e0e7ee4edf..fe22c7428958 100644
@@ -2172,6 +2172,7 @@ struct device_node *of_find_next_cache_node(const struct device_node *np)
 
 	return NULL;
 }
+EXPORT_SYMBOL_GPL(of_find_next_cache_node);
 
 /**
  * of_find_last_cache_level - Find the level at which the last cache is
Signature · 1 line
2.17.1
author
#3 lore raw

PATCH v6 3/3 edac: Add support for Amazon's Annapurna Labs L2 EDAC

In reply to Hanna Hawa

Adds support for Amazon's Annapurna Labs L2 EDAC driver to detect and report L2 errors.

---
 MAINTAINERS               |   5 +
 drivers/edac/Kconfig      |   8 ++
 drivers/edac/Makefile     |   1 +
 drivers/edac/al_l2_edac.c | 251 ++++++++++++++++++++++++++++++++++++++
 4 files changed, 265 insertions(+)
 create mode 100644 drivers/edac/al_l2_edac.c
MAINTAINERS +5 -0
index 7887a62dc843..0eabcfcf91a9 100644
@@ -748,6 +748,11 @@ M:	Hanna Hawa <[email protected]>
 S:	Maintained
 F:	drivers/edac/al_l1_edac.c
 
+AMAZON ANNAPURNA LABS L2 EDAC
+M:	Hanna Hawa <[email protected]>
+S:	Maintained
+F:	drivers/edac/al_l2_edac.c
+
 AMAZON ANNAPURNA LABS THERMAL MMIO DRIVER
 M:	Talel Shenhar <[email protected]>
 S:	Maintained
drivers/edac/Kconfig +8 -0
index e8161d7c7469..cb394aff1cab 100644
@@ -82,6 +82,14 @@ config EDAC_AL_L1
 	  for Amazon's Annapurna Labs SoCs.
 	  This driver detects errors of L1 caches.
 
+config EDAC_AL_L2
+	tristate "Amazon's Annapurna Labs L2 EDAC"
+	depends on ARCH_ALPINE
+	help
+	  Support for L2 error detection and correction
+	  for Amazon's Annapurna Labs SoCs.
+	  This driver detects errors of L2 caches.
+
 config EDAC_AMD64
 	tristate "AMD64 (Opteron, Athlon64)"
 	depends on AMD_NB && EDAC_DECODE_MCE
drivers/edac/Makefile +1 -0
index caa2dc91e8a0..60a6b8bbe2f8 100644
@@ -23,6 +23,7 @@ edac_mce_amd-y				:= mce_amd.o
 obj-$(CONFIG_EDAC_DECODE_MCE)		+= edac_mce_amd.o
 
 obj-$(CONFIG_EDAC_AL_L1)		+= al_l1_edac.o
+obj-$(CONFIG_EDAC_AL_L2)		+= al_l2_edac.o
 obj-$(CONFIG_EDAC_AMD76X)		+= amd76x_edac.o
 obj-$(CONFIG_EDAC_CPC925)		+= cpc925_edac.o
 obj-$(CONFIG_EDAC_I5000)		+= i5000_edac.o
drivers/edac/al_l2_edac.c new file +251 -0

251 lines not on this page · raw message

Signature · 1 line
2.17.1
#4 lore raw

Re: [PATCH v6 3/3] edac: Add support for Amazon's Annapurna Labs L2 EDAC

+Sudeep

Quoted 3 lines · Hanna Hawa

Adds support for Amazon's Annapurna Labs L2 EDAC driver to detect and report L2 errors.

I was curious why you needed a DT cache parsing function...

[...]

Quoted 43 lines
drivers/edac/al_l2_edac.c new file +43 -0
+static int al_l2_edac_probe(struct platform_device *pdev)
+{
+       struct edac_device_ctl_info *edac_dev;
+       struct al_l2_edac *al_l2;
+       struct device *dev = &pdev->dev;
+       int ret, i;
+
+       edac_dev = edac_device_alloc_ctl_info(sizeof(*al_l2), DRV_NAME, 1, "L",
+                                             1, 2, NULL, 0,
+                                             edac_device_alloc_index());
+       if (!edac_dev)
+               return -ENOMEM;
+
+       al_l2 = edac_dev->pvt_info;
+       edac_dev->edac_check = al_l2_edac_check;
+       edac_dev->dev = dev;
+       edac_dev->mod_name = DRV_NAME;
+       edac_dev->dev_name = dev_name(dev);
+       edac_dev->ctl_name = "L2_cache";
+       platform_set_drvdata(pdev, edac_dev);
+
+       INIT_LIST_HEAD(&al_l2->l2_caches);
+
+       for_each_possible_cpu(i) {
+               struct device_node *cpu;
+               struct device_node *cpu_cache;
+               struct al_l2_cache *l2_cache;
+               bool found = false;
+
+               cpu = of_get_cpu_node(i, NULL);
+               if (!cpu)
+                       continue;
+
+               cpu_cache = of_find_next_cache_node(cpu);
+               list_for_each_entry(l2_cache, &al_l2->l2_caches, list_node) {
+                       if (l2_cache->of_node == cpu_cache) {
+                               found = true;
+                               break;
+                       }
+               }
+
+               if (found) {
+                       cpumask_set_cpu(i, &l2_cache->cluster_cpus);

drivers/edac/al_l2_edac.c line 169

Quoted 10 lines
drivers/edac/al_l2_edac.c new file +10 -0
+               } else {
+                       l2_cache = devm_kzalloc(dev, sizeof(*l2_cache),
+                                               GFP_KERNEL);
+                       l2_cache->of_node = cpu_cache;
+                       list_add(&l2_cache->list_node, &al_l2->l2_caches);
+                       cpumask_set_cpu(i, &l2_cache->cluster_cpus);
+               }
+
+               of_node_put(cpu);
+       }

We already have what's probably similar code to parse DT and populate cacheinfo data. Does that not work for you? If not, why not and can we extend it?

Then your driver might work if the data comes from ACPI instead (or maybe that's all different, I don't know).

Rob

#5 lore raw

Re: [PATCH v6 3/3] edac: Add support for Amazon's Annapurna Labs L2 EDAC

Quoted 60 lines · Rob Herring

+Sudeep

Quoted 3 lines · Hanna Hawa

Adds support for Amazon's Annapurna Labs L2 EDAC driver to detect and report L2 errors.

I was curious why you needed a DT cache parsing function...

[...]

Quoted 49 lines
drivers/edac/al_l2_edac.c new file +49 -0
+static int al_l2_edac_probe(struct platform_device *pdev)
+{
+       struct edac_device_ctl_info *edac_dev;
+       struct al_l2_edac *al_l2;
+       struct device *dev = &pdev->dev;
+       int ret, i;
+
+       edac_dev = edac_device_alloc_ctl_info(sizeof(*al_l2), DRV_NAME, 1, "L",
+                                             1, 2, NULL, 0,
+                                             edac_device_alloc_index());
+       if (!edac_dev)
+               return -ENOMEM;
+
+       al_l2 = edac_dev->pvt_info;
+       edac_dev->edac_check = al_l2_edac_check;
+       edac_dev->dev = dev;
+       edac_dev->mod_name = DRV_NAME;
+       edac_dev->dev_name = dev_name(dev);
+       edac_dev->ctl_name = "L2_cache";
+       platform_set_drvdata(pdev, edac_dev);
+
+       INIT_LIST_HEAD(&al_l2->l2_caches);
+
+       for_each_possible_cpu(i) {
+               struct device_node *cpu;
+               struct device_node *cpu_cache;
+               struct al_l2_cache *l2_cache;
+               bool found = false;
+
+               cpu = of_get_cpu_node(i, NULL);
+               if (!cpu)
+                       continue;
+
+               cpu_cache = of_find_next_cache_node(cpu);
+               list_for_each_entry(l2_cache, &al_l2->l2_caches, list_node) {
+                       if (l2_cache->of_node == cpu_cache) {
+                               found = true;
+                               break;
+                       }
+               }
+
+               if (found) {
+                       cpumask_set_cpu(i, &l2_cache->cluster_cpus);
+               } else {
+                       l2_cache = devm_kzalloc(dev, sizeof(*l2_cache),
+                                               GFP_KERNEL);
+                       l2_cache->of_node = cpu_cache;
+                       list_add(&l2_cache->list_node, &al_l2->l2_caches);
+                       cpumask_set_cpu(i, &l2_cache->cluster_cpus);
Quoted 8 lines

drivers/edac/al_l2_edac.c line 169

Quoted 4 lines
drivers/edac/al_l2_edac.c new file +4 -0
+               }
+
+               of_node_put(cpu);
+       }

We already have what's probably similar code to parse DT and populate cacheinfo data. Does that not work for you? If not, why not and can we extend it?

As I saw in cacheinfo it will return the cacheinfo for the online CPUs only, correct me if I'm wrong..

Here I'm parsing all the L2 info for all CPUs depends on DT to get "cluster_cpus", and using smp_call_function_any() will call the online cpu to read the L2MERRSR status register.

Quoted 3 lines

Then your driver might work if the data comes from ACPI instead (or maybe that's all different, I don't know).

No plan to get it work on ACPI, at least in the near future.

Quoted 3 lines

Rob

#6 lore raw

Re: [PATCH v6 3/3] edac: Add support for Amazon's Annapurna Labs L2 EDAC

In reply to Hanna Hawa
Quoted 34 lines · Hanna Hawa

Adds support for Amazon's Annapurna Labs L2 EDAC driver to detect and report L2 errors.

---
 MAINTAINERS               |   5 +
 drivers/edac/Kconfig      |   8 ++
 drivers/edac/Makefile     |   1 +
 drivers/edac/al_l2_edac.c | 251 ++++++++++++++++++++++++++++++++++++++
 4 files changed, 265 insertions(+)
 create mode 100644 drivers/edac/al_l2_edac.c
MAINTAINERS +5 -0
index 7887a62dc843..0eabcfcf91a9 100644
@@ -748,6 +748,11 @@ M: Hanna Hawa <[email protected]>
 S:     Maintained
 F:     drivers/edac/al_l1_edac.c
 
+AMAZON ANNAPURNA LABS L2 EDAC
+M:     Hanna Hawa <[email protected]>
+S:     Maintained
+F:     drivers/edac/al_l2_edac.c
+
 AMAZON ANNAPURNA LABS THERMAL MMIO DRIVER
 M:     Talel Shenhar <[email protected]>
 S:     Maintained
drivers/edac/Kconfig +0 -0
index e8161d7c7469..cb394aff1cab 100644

drivers/edac/Kconfig config EDAC_AL_L1 · line 86

Quoted 5 lines
drivers/edac/Kconfig +2 -0
@@ -82,6 +82,14 @@ config EDAC_AL_L1
          for Amazon's Annapurna Labs SoCs.
          This driver detects errors of L1 caches.
 
+config EDAC_AL_L2
+       tristate "Amazon's Annapurna Labs L2 EDAC"

I still think this should be an "A57 L2 ECC" driver, but if no one cares I'll shut up and the 2nd person can rename everything.

drivers/edac/Kconfig config EDAC_AL_L1 · line 87

Quoted 1 line
drivers/edac/Kconfig +1 -0
+       depends on ARCH_ALPINE

|| COMPILE_TEST

Maybe it needs an ARM64 dependency too in this case?

Quoted 37 lines
drivers/edac/Kconfig +33 -1
+       help
+         Support for L2 error detection and correction
+         for Amazon's Annapurna Labs SoCs.
+         This driver detects errors of L2 caches.
+
 
+
+       ret = platform_driver_register(&al_l2_edac_driver);
+       if (ret) {
+               pr_err("Failed to register %s (%d)\n", DRV_NAME, ret);
+               return ret;
+       }
+
+       edac_l2_device = platform_device_register_simple(DRV_NAME, -1, NULL, 0);
+       if (IS_ERR(edac_l2_device)) {
+               pr_err("Failed to register EDAC AL L2 platform device\n");
+               return PTR_ERR(edac_l2_device);
+       }
+
+       return 0;
+}
+
+static void __exit al_l2_exit(void)
+{
+       platform_device_unregister(edac_l2_device);
+       platform_driver_unregister(&al_l2_edac_driver);
+}
+
+late_initcall(al_l2_init);
+module_exit(al_l2_exit);
+
+MODULE_LICENSE("GPL v2");
+MODULE_AUTHOR("Hanna Hawa <[email protected]>");
+MODULE_DESCRIPTION("Amazon's Annapurna Lab's L2 EDAC Driver");
--

2.17.1

#7 lore raw

Re: [PATCH v6 3/3] edac: Add support for Amazon's Annapurna Labs L2 EDAC

Hi,

Quoted 45 lines · Rob Herring
Quoted 34 lines · Hanna Hawa

Adds support for Amazon's Annapurna Labs L2 EDAC driver to detect and report L2 errors.

---
  MAINTAINERS               |   5 +
  drivers/edac/Kconfig      |   8 ++
  drivers/edac/Makefile     |   1 +
  drivers/edac/al_l2_edac.c | 251 ++++++++++++++++++++++++++++++++++++++
  4 files changed, 265 insertions(+)

create mode 100644 drivers/edac/al_l2_edac.c

MAINTAINERS +5 -0
index 7887a62dc843..0eabcfcf91a9 100644
@@ -748,6 +748,11 @@ M: Hanna Hawa <[email protected]>
  S:     Maintained
  F:     drivers/edac/al_l1_edac.c
 
+AMAZON ANNAPURNA LABS L2 EDAC
+M:     Hanna Hawa <[email protected]>
+S:     Maintained
+F:     drivers/edac/al_l2_edac.c
+
  AMAZON ANNAPURNA LABS THERMAL MMIO DRIVER
  M:     Talel Shenhar <[email protected]>
  S:     Maintained
drivers/edac/Kconfig +0 -0
index e8161d7c7469..cb394aff1cab 100644

drivers/edac/Kconfig config EDAC_AL_L1 · line 86

Quoted 5 lines
drivers/edac/Kconfig +2 -0
@@ -82,6 +82,14 @@ config EDAC_AL_L1
           for Amazon's Annapurna Labs SoCs.
           This driver detects errors of L1 caches.
 
+config EDAC_AL_L2
+       tristate "Amazon's Annapurna Labs L2 EDAC"

I still think this should be an "A57 L2 ECC" driver, but if no one cares I'll shut up and the 2nd person can rename everything.

Quoted 3 lines

drivers/edac/Kconfig config EDAC_AL_L1 · line 87

Quoted 1 line
drivers/edac/Kconfig +1 -0
+       depends on ARCH_ALPINE

|| COMPILE_TEST

Will be add in next patchset.

Quoted 2 lines

Maybe it needs an ARM64 dependency too in this case?

Yes, it need ARM64 dependency, I'll add.

Thanks,
Hanna

Quoted 38 lines
Quoted 38 lines
drivers/edac/Kconfig +33 -1
+       help
+         Support for L2 error detection and correction
+         for Amazon's Annapurna Labs SoCs.
+         This driver detects errors of L2 caches.
+
 
+
+       ret = platform_driver_register(&al_l2_edac_driver);
+       if (ret) {
+               pr_err("Failed to register %s (%d)\n", DRV_NAME, ret);
+               return ret;
+       }
+
+       edac_l2_device = platform_device_register_simple(DRV_NAME, -1, NULL, 0);
+       if (IS_ERR(edac_l2_device)) {
+               pr_err("Failed to register EDAC AL L2 platform device\n");
+               return PTR_ERR(edac_l2_device);
+       }
+
+       return 0;
+}
+
+static void __exit al_l2_exit(void)
+{
+       platform_device_unregister(edac_l2_device);
+       platform_driver_unregister(&al_l2_edac_driver);
+}
+
+late_initcall(al_l2_init);
+module_exit(al_l2_exit);
+
+MODULE_LICENSE("GPL v2");
+MODULE_AUTHOR("Hanna Hawa <[email protected]>");
+MODULE_DESCRIPTION("Amazon's Annapurna Lab's L2 EDAC Driver");
--

2.17.1