EDAC - fix for i10nm driver + upgrade to support Sapphire Rapids
Subsystems (3)
Tony Luck
maintainercover EDAC - fix for i10nm driver + upgrade to support Sapphire Rapids
First patch is a bug fix (sort of) we haven't directly seen problems in the driver, but when adding other bits to it there have been cases where the compiler re-ordered the access to the MMIO registers. Which can be a problem.
Second patch adds new type for DDR5 memory
Final patch updates the i10nm EDAC driver to support Sapphire Rapids.
This series on top of:git://git.kernel.org/pub/scm/linux/kernel/git/ras/ras.git edac-igen6 (which also adds some new memory types)
Qiuxu Zhuo (3):
EDAC/i10nm: Use readl() to access MMIO registers
EDAC: Add DDR5 new memory type
EDAC/i10nm: Add Intel Sapphire Rapids server support
drivers/edac/edac_mc.c | 1 + drivers/edac/i10nm_base.c | 39 +++++++++++++++++++++++++++++---------- drivers/edac/skx_base.c | 6 +++--- drivers/edac/skx_common.c | 23 ++++++++++++++++++----- drivers/edac/skx_common.h | 16 ++++++++++++---- include/linux/edac.h | 3 +++ 6 files changed, 66 insertions(+), 22 deletions(-)
Signature · 1 line
2.21.1
Tony Luck
maintainerPATCH 1/3 EDAC/i10nm: Use readl() to access MMIO registers
From: Qiuxu Zhuo <[email protected]>
Instead of raw access, use readl() to access MMIO registers of memory controller to avoid possible compiler re-ordering.
--- drivers/edac/i10nm_base.c | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-)
drivers/edac/i10nm_base.c +7 -4
@@ -6,6 +6,7 @@*/ #include <linux/kernel.h> +#include <linux/io.h> #include <asm/cpu_device_id.h> #include <asm/intel-family.h> #include <asm/mce.h>
@@ -19,14 +20,16 @@ #define i10nm_printk(level, fmt, arg...) \ edac_printk(level, "i10nm", fmt, ##arg) -#define I10NM_GET_SCK_BAR (d, reg) \ +#define I10NM_GET_SCK_BAR (d, reg) \ pci_read_config_dword((d)->uracu, 0xd0, &(reg))#define I10NM_GET_IMC_BAR (d, i, reg) \ pci_read_config_dword((d)->uracu, 0xd8 + (i) * 4, &(reg))#define I10NM_GET_DIMMMTR (m, i, j) \ - (*(u32 *)((m)->mbase + 0x2080c + (i) * 0x4000 + (j) * 4)) + readl((m)->mbase + 0x2080c + (i) * 0x4000 + (j) * 4)#define I10NM_GET_MCDDRTCFG (m, i, j) \ - (*(u32 *)((m)->mbase + 0x20970 + (i) * 0x4000 + (j) * 4)) + readl((m)->mbase + 0x20970 + (i) * 0x4000 + (j) * 4) +#define I10NM_GET_MCMTR (m, i) \ + readl((m)->mbase + 0x20ef8 + (i) * 0x4000)#define I10NM_GET_SCK_MMIO_BASE (reg) (GET_BITFIELD(reg, 0, 28) << 23)#define I10NM_GET_IMC_MMIO_OFFSET (reg) (GET_BITFIELD(reg, 0, 10) << 12)
@@ -148,7 +151,7 @@ static bool i10nm_check_ecc(struct skx_imc *imc, int chan) {u32 mcmtr; - mcmtr = *(u32 *)(imc->mbase +0x20ef8 + chan *0x4000 ); + mcmtr =I10NM_GET_MCMTR (imc, chan);edac_dbg (1 ,"ch%d mcmtr reg %x\n" , chan, mcmtr);return !!GET_BITFIELD (mcmtr,2 ,2 );
Signature · 1 line
2.21.1
Tony Luck
maintainerPATCH 2/3 EDAC: Add DDR5 new memory type
In reply to Tony LuckFrom: Qiuxu Zhuo <[email protected]>
Add a new entry to 'enum mem_type' and a new string to 'edac_mem_types[]' for DDR5 new memory type.
--- drivers/edac/edac_mc.c | 1 + include/linux/edac.h | 3 +++ 2 files changed, 4 insertions(+)
drivers/edac/edac_mc.c +1 -0
@@ -163,6 +163,7 @@ const char * const edac_mem_types[] = { [MEM_RDDR4] = "Registered-DDR4", [MEM_LPDDR4] = "Low-Power-DDR4-RAM", [MEM_LRDDR4] = "Load-Reduced-DDR4-RAM", + [MEM_DDR5] = "Unbuffered-DDR5", [MEM_NVDIMM] = "Non-volatile-RAM", [MEM_WIO2] = "Wide-IO-2", };
include/linux/edac.h +3 -0
@@ -181,6 +181,7 @@ static inline char *mc_event_error_type(const unsigned int err_type)* This is a variant of the DDR4 memories. * @MEM_LRDDR4: Load-Reduced DDR4 memory. * @MEM_LPDDR4: Low-Power DDR4 memory. +* @MEM_DDR5: Unbuffered DDR5 RAM * @MEM_NVDIMM: Non-volatile RAM * @MEM_WIO2: Wide I/O 2. */
@@ -208,6 +209,7 @@ enum mem_type {MEM_RDDR4 ,MEM_LRDDR4 ,MEM_LPDDR4 , +MEM_DDR5 ,MEM_NVDIMM , MEM_WIO2, };
@@ -234,6 +236,7 @@ enum mem_type {#define MEM_FLAG_RDDR4 BIT(MEM_RDDR4)#define MEM_FLAG_LRDDR4 BIT(MEM_LRDDR4)#define MEM_FLAG_LPDDR4 BIT(MEM_LPDDR4) +#define MEM_FLAG_DDR5 BIT(MEM_DDR5)#define MEM_FLAG_NVDIMM BIT(MEM_NVDIMM)#define MEM_FLAG_WIO2 BIT(MEM_WIO2)
Signature · 1 line
2.21.1
Tony Luck
maintainerPATCH 3/3 EDAC/i10nm: Add Intel Sapphire Rapids server support
In reply to Tony LuckFrom: Qiuxu Zhuo <[email protected]>
The Sapphire Rapids CPU model shares the same memory controller architecture with Ice Lake server. There are some configurations different from Ice Lake server as below:
- The device ID for configuration agent.
- The size for per channel memory-mapped I/O.
- The DDR5 memory support.
So add the above configurations and the Sapphire Rapids CPU model ID for EDAC support.
--- drivers/edac/i10nm_base.c | 34 +++++++++++++++++++++++++--------- drivers/edac/skx_base.c | 6 +++--- drivers/edac/skx_common.c | 23 ++++++++++++++++++----- drivers/edac/skx_common.h | 16 ++++++++++++---- 4 files changed, 58 insertions(+), 21 deletions(-)
drivers/edac/i10nm_base.c +25 -9
@@ -13,7 +13,7 @@#include "edac_module.h" #include "skx_common.h" -#define I10NM_REVISION "v0.0.3" +#define I10NM_REVISION "v0.0.4"#define EDAC_MOD_STR "i10nm_edac"/* Debug macros */
@@ -25,11 +25,13 @@#define I10NM_GET_IMC_BAR (d, i, reg) \ pci_read_config_dword((d)->uracu, 0xd8 + (i) * 4, &(reg))#define I10NM_GET_DIMMMTR (m, i, j) \ - readl((m)->mbase + 0x2080c + (i) * 0x4000 + (j) * 4) + readl((m)->mbase + 0x2080c + (i) * (m)->chan_mmio_sz + (j) * 4)#define I10NM_GET_MCDDRTCFG (m, i, j) \ - readl((m)->mbase + 0x20970 + (i) * 0x4000 + (j) * 4) + readl((m)->mbase + 0x20970 + (i) * (m)->chan_mmio_sz + (j) * 4)#define I10NM_GET_MCMTR (m, i) \ - readl((m)->mbase + 0x20ef8 + (i) * 0x4000) + readl((m)->mbase + 0x20ef8 + (i) * (m)->chan_mmio_sz) +#define I10NM_GET_AMAP (m, i) \ + readl((m)->mbase + 0x20814 + (i) * (m)->chan_mmio_sz)#define I10NM_GET_SCK_MMIO_BASE (reg) (GET_BITFIELD(reg, 0, 28) << 23)#define I10NM_GET_IMC_MMIO_OFFSET (reg) (GET_BITFIELD(reg, 0, 10) << 12)
@@ -129,12 +131,22 @@ static struct res_config i10nm_cfg0 = { .type = I10NM, .decs_did = 0x3452, .busno_cfg_offset = 0xcc, + .ddr_chan_mmio_sz = 0x4000, };static struct res_config i10nm_cfg1 = { .type =I10NM , .decs_did =0x3452 , .busno_cfg_offset =0xd0 , + .ddr_chan_mmio_sz =0x4000 , +}; + +static struct res_config spr_cfg = { + .type =SPR , + .decs_did =0x3252 , + .busno_cfg_offset =0xd0 , + .ddr_chan_mmio_sz =0x8000 , + .support_ddr5 = true, };static const struct x86_cpu_id i10nm_cpuids[] = {
@@ -143,6 +155,7 @@ static const struct x86_cpu_id i10nm_cpuids[] = {X86_MATCH_INTEL_FAM6_MODEL_STEPPINGS (ICELAKE_X ,X86_STEPPINGS (0x0 ,0x3 ), &i10nm_cfg0),X86_MATCH_INTEL_FAM6_MODEL_STEPPINGS (ICELAKE_X ,X86_STEPPINGS (0x4 ,0xf ), &i10nm_cfg1),X86_MATCH_INTEL_FAM6_MODEL_STEPPINGS (ICELAKE_D ,X86_STEPPINGS (0x0 ,0xf ), &i10nm_cfg1), +X86_MATCH_INTEL_FAM6_MODEL_STEPPINGS (SAPPHIRERAPIDS_X ,X86_STEPPINGS (0x0 ,0xf ), &spr_cfg), {} };MODULE_DEVICE_TABLE (x86cpu, i10nm_cpuids);
@@ -157,12 +170,13 @@ static bool i10nm_check_ecc(struct skx_imc *imc, int chan)return !!GET_BITFIELD (mcmtr,2 ,2 ); } -static int i10nm_get_dimm_config (struct mem_ctl_info *mci) +static int i10nm_get_dimm_config (struct mem_ctl_info *mci, +struct res_config *cfg) {struct skx_pvt *pvt = mci->pvt_info;struct skx_imc *imc = pvt->imc; +u32 mtr, amap, mcddrtcfg;struct dimm_info *dimm; -u32 mtr, mcddrtcfg;int i, j, ndimms;for (i =0 ; i <I10NM_NUM_CHANNELS ; i++) {
@@ -170,6 +184,7 @@ static int i10nm_get_dimm_config(struct mem_ctl_info *mci)continue ; ndimms =0 ; + amap =I10NM_GET_AMAP (imc, i);for (j =0 ; j <I10NM_NUM_DIMMS ; j++) { dimm =edac_get_dimm (mci, i, j,0 ); mtr =I10NM_GET_DIMMMTR (imc, i, j);
@@ -178,8 +193,8 @@ static int i10nm_get_dimm_config(struct mem_ctl_info *mci) mtr, mcddrtcfg, imc->mc, i, j);if (IS_DIMM_PRESENT (mtr)) - ndimms +=skx_get_dimm_info (mtr,0 ,0 , dimm, - imc, i, j); + ndimms +=skx_get_dimm_info (mtr,0 , amap, dimm, + imc, i, j, cfg);else if (IS_NVDIMM_PRESENT (mcddrtcfg, j)) ndimms +=skx_get_nvdimm_info (dimm, imc, i, j,EDAC_MOD_STR );
@@ -303,10 +318,11 @@ static int __init i10nm_init(void) d->imc[i].lmc = i; d->imc[i].src_id = src_id; d->imc[i].node_id = node_id; + d->imc[i].chan_mmio_sz = cfg->ddr_chan_mmio_sz; rc =skx_register_mci (&d->imc[i], d->imc[i].mdev,"Intel_10nm Socket" ,EDAC_MOD_STR , - i10nm_get_dimm_config); + i10nm_get_dimm_config, cfg);if (rc <0 ) goto fail; }
drivers/edac/skx_base.c +3 -3
@@ -174,7 +174,7 @@ static bool skx_check_ecc(u32 mcmtr)return !!GET_BITFIELD (mcmtr,2 ,2 ); } -static int skx_get_dimm_config (struct mem_ctl_info *mci) +static int skx_get_dimm_config (struct mem_ctl_info *mci,struct res_config *cfg) {struct skx_pvt *pvt = mci->pvt_info;u32 mtr, mcmtr, amap, mcddrtcfg;
@@ -195,7 +195,7 @@ static int skx_get_dimm_config(struct mem_ctl_info *mci)pci_read_config_dword (imc->chan[i].cdev,0x80 +4 * j, &mtr);if (IS_DIMM_PRESENT (mtr)) { - ndimms +=skx_get_dimm_info (mtr, mcmtr, amap, dimm, imc, i, j); + ndimms +=skx_get_dimm_info (mtr, mcmtr, amap, dimm, imc, i, j, cfg); }else if (IS_NVDIMM_PRESENT (mcddrtcfg, j)) { ndimms +=skx_get_nvdimm_info (dimm, imc, i, j,EDAC_MOD_STR );
@@ -702,7 +702,7 @@ static int __init skx_init(void) d->imc[i].node_id = node_id; rc =skx_register_mci (&d->imc[i], d->imc[i].chan[0 ].cdev,"Skylake Socket" ,EDAC_MOD_STR , - skx_get_dimm_config); + skx_get_dimm_config, cfg);if (rc <0 ) goto fail; }
drivers/edac/skx_common.c +18 -5
@@ -304,15 +304,25 @@ static int skx_get_dimm_attr(u32 reg, int lobit, int hibit, int add,#define numcol (reg) skx_get_dimm_attr(reg, 0, 1, 10, 0, 2, "cols")int skx_get_dimm_info (u32 mtr,u32 mcmtr,u32 amap,struct dimm_info *dimm, -struct skx_imc *imc,int chan,int dimmno) +struct skx_imc *imc,int chan,int dimmno, +struct res_config *cfg) { -int banks =16 , ranks, rows, cols, npages; +int banks, ranks, rows, cols, npages; +enum mem_type mtype;u64 size; ranks =numrank (mtr); rows =numrow (mtr); cols =numcol (mtr); +if (cfg->support_ddr5 && (amap &0x8 )) { + banks =32 ; + mtype =MEM_DDR5 ; + }else { + banks =16 ; + mtype =MEM_DDR4 ; + } +/* * Compute size in 8-byte (2^3) words, then shift to MiB (2^20) */
@@ -332,7 +342,7 @@ int skx_get_dimm_info(u32 mtr, u32 mcmtr, u32 amap, struct dimm_info *dimm, dimm->nr_pages = npages; dimm->grain =32 ; dimm->dtype =get_width (mtr); - dimm->mtype =MEM_DDR4 ; + dimm->mtype = mtype; dimm->edac_mode =EDAC_SECDED ;/* likely better than this */ snprintf (dimm->label,sizeof (dimm->label),"CPU_SrcID#%u_MC#%u_Chan#%u_DIMM#%u" , imc->src_id, imc->lmc, chan, dimmno);
@@ -390,7 +400,8 @@ int skx_get_nvdimm_info(struct dimm_info *dimm, struct skx_imc *imc,int skx_register_mci (struct skx_imc *imc,struct pci_dev *pdev,const char *ctl_name,const char *mod_str, -get_dimm_config_f get_dimm_config) +get_dimm_config_f get_dimm_config, +struct res_config *cfg) {struct mem_ctl_info *mci;struct edac_mc_layer layers[2 ];
@@ -425,13 +436,15 @@ int skx_register_mci(struct skx_imc *imc, struct pci_dev *pdev, } mci->mtype_cap =MEM_FLAG_DDR4 |MEM_FLAG_NVDIMM ; +if (cfg->support_ddr5) + mci->mtype_cap |=MEM_FLAG_DDR5 ; mci->edac_ctl_cap =EDAC_FLAG_NONE ; mci->edac_cap =EDAC_FLAG_NONE ; mci->mod_name = mod_str; mci->dev_name =pci_name (pdev); mci->ctl_page_to_phys =NULL ; - rc =get_dimm_config (mci); + rc =get_dimm_config (mci, cfg);if (rc <0 ) goto fail;
drivers/edac/skx_common.h +12 -4
@@ -59,6 +59,7 @@ struct skx_dev {struct mem_ctl_info *mci;struct pci_dev *mdev;/* for i10nm CPU */ void __iomem *mbase;/* for i10nm CPU */ +int chan_mmio_sz;/* for i10nm CPU */ u8 mc;/* system wide mc# */ u8 lmc;/* socket relative mc# */ u8 src_id, node_id;
@@ -82,7 +83,8 @@ struct skx_pvt {enum type {SKX , -I10NM +I10NM , +SPR };enum {
@@ -118,9 +120,13 @@ struct res_config {unsigned int decs_did;/* Default bus number configuration register offset */ int busno_cfg_offset; +/* Per DDR channel memory-mapped I/O size */ +int ddr_chan_mmio_sz; +bool support_ddr5; }; -typedef int (*get_dimm_config_f )(struct mem_ctl_info *mci); +typedef int (*get_dimm_config_f )(struct mem_ctl_info *mci, +struct res_config *cfg);typedef bool (*skx_decode_f )(struct decoded_addr *res);typedef void (*skx_show_retry_log_f )(struct decoded_addr *res,char *msg,int len);
@@ -136,14 +142,16 @@ int skx_get_all_bus_mappings(struct res_config *cfg, struct list_head **list);int skx_get_hi_lo (unsigned int did,int off[],u64 *tolm,u64 *tohm);int skx_get_dimm_info (u32 mtr,u32 mcmtr,u32 amap,struct dimm_info *dimm, -struct skx_imc *imc,int chan,int dimmno); +struct skx_imc *imc,int chan,int dimmno, +struct res_config *cfg);int skx_get_nvdimm_info (struct dimm_info *dimm,struct skx_imc *imc,int chan,int dimmno,const char *mod_str);int skx_register_mci (struct skx_imc *imc,struct pci_dev *pdev,const char *ctl_name,const char *mod_str, -get_dimm_config_f get_dimm_config); +get_dimm_config_f get_dimm_config, +struct res_config *cfg);int skx_mce_check_error (struct notifier_block *nb,unsigned long val,void *data);
Signature · 1 line
2.21.1