Skip to contentSkip to conversation

AMD64_EDAC: Fix amd64_map_to_dcs_mask

PATCH In review Keith Mannthey 2 participants 7 messages Sep 18, 2009 – Sep 22, 2009

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 AMD64_EDAC: Fix amd64_map_to_dcs_mask

I tested 2.6.31 and the mainline amd64_edac driver. Once errors were getting reported I noticed that a good amount of valid looking system addressed were not being correctly decoded to the csrow level. I was only able to correctly decode errors on channel 0 of a given csrow. Errors on channel 1 were unable to be mapped.

After some digging I realized that the there was an issue with handling of Dram Chip Select Masks. Specifically that amd64_map_to_dcs_mask was returning incorrect values on my Rev F based system. See AMD #32559, 4.5.4, starting on pg 90 for a Rev F explanation of the correct mapping of DRAM CS Base and DRAM CS Mask regs. This lead to the code below. I can provide further explanation if needed.

I have tested this code on Rev F based system with ecc debug dimms and fully expect it to work on earlier and later cpus. I am now able to correctly decode and map errors to csrows rows on this system.

---

drivers/edac/amd64_edac.c +4 -2
diff -urN linux-2.6.31/drivers/edac/amd64_edac.c linux-2.6.31-fixed/drivers/edac/amd64_edac.c
@@ -1,6 +1,8 @@
-#include "amd64_edac.h"
+#include <linux/log2.h>
 #include <asm/k8.h>
 
+#include "amd64_edac.h"
+
 static struct edac_pci_ctl_info *amd64_ctl_pci;
 
 static int report_gart_errors;
@@ -132,7 +134,7 @@
 /* Map from a CSROW entry to the mask entry that operates on it */
 static inline u32 amd64_map_to_dcs_mask(struct amd64_pvt *pvt, int csrow)
 {
-	return csrow >> (pvt->num_dcsm >> 3);
+	return csrow >> (8 >> (ilog2(pvt->num_dcsm)+1));
 }
 
 /* return the 'base' address the i'th CS entry of the 'dct' DRAM controller */
drivers/edac/amd64_edac.h +1 -1
diff -urN linux-2.6.31/drivers/edac/amd64_edac.h linux-2.6.31-fixed/drivers/edac/amd64_edac.h
@@ -526,7 +526,7 @@
 	/*
 	 * The following fields are set at (load) run time, after CPU revision
 	 * has been determined, since the dct_base and dct_mask registers vary
-	 * based on revision
+	 * based on revision.  num_dcsm is assumed to be a power of 2. 
 	 */
 	u32 dcsb_base;		/* DCSB base bits */
 	u32 dcsm_mask;		/* DCSM mask bits */
#1 lore raw
Quoted 14 lines · Keith Mannthey

I tested 2.6.31 and the mainline amd64_edac driver. Once errors were getting reported I noticed that a good amount of valid looking system addressed were not being correctly decoded to the csrow level. I was only able to correctly decode errors on channel 0 of a given csrow. Errors on channel 1 were unable to be mapped.

After some digging I realized that the there was an issue with handling of Dram Chip Select Masks. Specifically that amd64_map_to_dcs_mask was returning incorrect values on my Rev F based system. See AMD #32559, 4.5.4, starting on pg 90 for a Rev F explanation of the correct mapping of DRAM CS Base and DRAM CS Mask regs. This lead to the code below. I can provide further explanation if needed.

Quoted 7 lines

I have tested this code on Rev F based system with ecc debug dimms and fully expect it to work on earlier and later cpus. I am now able to correctly decode and map errors to csrows rows on this system.

---

This whole DSC[BM] handling is rather long-winded and overengineered. It is on my to-be-rewritten-and-simplified list.

Quoted 15 lines

diff -urN linux-2.6.31/drivers/edac/amd64_edac.c linux-2.6.31-fixed/drivers/edac/amd64_edac.c

drivers/edac/amd64_edac.c +3 -1
@@ -1,6 +1,8 @@
-#include "amd64_edac.h"
+#include <linux/log2.h>
 #include <asm/k8.h>
 
+#include "amd64_edac.h"
+
 static struct edac_pci_ctl_info *amd64_ctl_pci;
 
 static int report_gart_errors;

drivers/edac/amd64_edac.c line 137

Quoted 5 lines
drivers/edac/amd64_edac.c +1 -1
@@ -132,7 +134,7 @@
 /* Map from a CSROW entry to the mask entry that operates on it */
 static inline u32 amd64_map_to_dcs_mask(struct amd64_pvt *pvt, int csrow)
 {
-	return csrow >> (pvt->num_dcsm >> 3);
+	return csrow >> (8 >> (ilog2(pvt->num_dcsm)+1));

Almost. You have 8 DCSMs on RevE, 4 on RevF and F10h and 2 on F11h and this way you get wrong DCSM offsets for F11h. A dirty fix would be:

if (boot_cpu_data.x86 == 0xf && pvt->ext_model < OPTERON_CPU_REV_E) {
	return csrow;
else
	return csrow >> 1;

The problem is, the csrow thing still goes over 0..7 which is obviously wrong on F11h but I'll fix that later. Care to redo your patch according to these and the comments from my previous mail and resend?

By the way, your patches made me look harder at that code region and I've found some more problems with it which I've fixed. Would you like to test the whole bunch of fixes on your setup?

Thanks.

Signature · 8 lines
Regards/Gruss,
Boris.

Operating | Advanced Micro Devices GmbH
  System  | Karl-Hammerschmidt-Str. 34, 85609 Dornach b. München, Germany
 Research | Geschäftsführer: Andrew Bowd, Thomas M. McCoy, Giuliano Meroni
  Center  | Sitz: Dornach, Gemeinde Aschheim, Landkreis München
  (OSRC)  | Registergericht München, HRB Nr. 43632
#2 lore raw
Quoted 1 line · Borislav Petkov

On Thu, Sep 17, 2009 at 07:09:41PM -0700, Keith Mannthey wrote:

<snip>

Quoted 8 lines

drivers/edac/amd64_edac.c line 137

Quoted 5 lines

/* Map from a CSROW entry to the mask entry that operates on it */ static inline u32 amd64_map_to_dcs_mask(struct amd64_pvt *pvt, int csrow) {

drivers/edac/amd64_edac.c +1 -1
-	return csrow >> (pvt->num_dcsm >> 3);
+	return csrow >> (8 >> (ilog2(pvt->num_dcsm)+1));

Almost. You have 8 DCSMs on RevE, 4 on RevF and F10h and 2 on F11h and this way you get wrong DCSM offsets for F11h. A dirty fix would be:

I think this will still be ok for F11.

ilog2(2) = 1

1 + 1 == 2

8 >> 2 == 2

csrow >> 2

This would be ok rev F11 assuming 8 total.

Am I missing something else?

Quoted 4 lines
if (boot_cpu_data.x86 == 0xf && pvt->ext_model < OPTERON_CPU_REV_E) {
	return csrow;
else
	return csrow >> 1;

Still busted for F11.

Quoted 3 lines

The problem is, the csrow thing still goes over 0..7 which is obviously wrong on F11h but I'll fix that later. Care to redo your patch according to these and the comments from my previous mail and resend?

Are there more than 8 csrows any any version (I don't currently have F11 specs). Maybe should just move to a map rather than a math trick to get to the right index?

Quoted 3 lines

By the way, your patches made me look harder at that code region and I've found some more problems with it which I've fixed. Would you like to test the whole bunch of fixes on your setup?

Yes please send any changes you have. I have a decent test setup for live errors.

Thanks,
Keith

#3 lore raw
Quoted 10 lines · Keith Mannthey
Quoted 2 lines

Almost. You have 8 DCSMs on RevE, 4 on RevF and F10h and 2 on F11h and this way you get wrong DCSM offsets for F11h. A dirty fix would be:

I think this will still be ok for F11.

ilog2(2) = 1

1 + 1 == 2

Quoted 7 lines

8 >> 2 == 2

csrow >> 2

This would be ok rev F11 assuming 8 total.

Am I missing something else?

Yes, F11h has only 4 DCSB and 2 DCSM registers. So the first two DCSB registers F2x[1,0]40 and F2x[1,0]44 use F2x[1,0]60 as a mask register and F2x[1,0]48 and F2x[1,0]4C use F2x[1,0]64. You can look at F11h as a F10h but with only the half of the DSC[BM] registers present. You can find the F11h BKDG at
http://support.amd.com/us/Processor_TechDocs/41256.pdf and especially Table 24 on page 115.

So, in that case csrow >> 1 is still valid but csrow going beyond 3 is out of range that's why it needs to be fixed differently.

Quoted 3 lines

Are there more than 8 csrows any any version (I don't currently have F11 specs). Maybe should just move to a map rather than a math trick to get to the right index?

I'll think up something on Monday.

Quoted 6 lines
Quoted 3 lines

By the way, your patches made me look harder at that code region and I've found some more problems with it which I've fixed. Would you like to test the whole bunch of fixes on your setup?

Yes please send any changes you have. I have a decent test setup for live errors.

Cool, I'll get back to you when I have them ready, thanks.

Signature · 8 lines
Regards/Gruss,
Boris.

Operating | Advanced Micro Devices GmbH
  System  | Karl-Hammerschmidt-Str. 34, 85609 Dornach b. München, Germany
 Research | Geschäftsführer: Andrew Bowd, Thomas M. McCoy, Giuliano Meroni
  Center  | Sitz: Dornach, Gemeinde Aschheim, Landkreis München
  (OSRC)  | Registergericht München, HRB Nr. 43632
#4 lore raw
In reply to Keith Mannthey

Hi Keith,

Quoted 6 lines · Keith Mannthey
Quoted 3 lines

By the way, your patches made me look harder at that code region and I've found some more problems with it which I've fixed. Would you like to test the whole bunch of fixes on your setup?

Yes please send any changes you have. I have a decent test setup for live errors.

The patch queue is at

git://git.kernel.org/pub/scm/linux/kernel/git/bp/bp.git

the 'edac-fixes' branch.

Please do test and let me know of any problems you might have.

Thanks a lot.

Signature · 8 lines
Regards/Gruss,
Boris.

Operating | Advanced Micro Devices GmbH
  System  | Karl-Hammerschmidt-Str. 34, 85609 Dornach b. München, Germany
 Research | Geschäftsführer: Andrew Bowd, Thomas M. McCoy, Giuliano Meroni
  Center  | Sitz: Dornach, Gemeinde Aschheim, Landkreis München
  (OSRC)  | Registergericht München, HRB Nr. 43632
#5 lore raw
Quoted 7 lines · Borislav Petkov

Hi Keith,

Quoted 4 lines · Keith Mannthey
Quoted 3 lines

By the way, your patches made me look harder at that code region and I've found some more problems with it which I've fixed. Would you like to test the whole bunch of fixes on your setup?

Quoted 8 lines
Quoted 2 lines

Yes please send any changes you have. I have a decent test setup for live errors.

The patch queue is at

git://git.kernel.org/pub/scm/linux/kernel/git/bp/bp.git

the 'edac-fixes' branch.

Please check you have synced with that git tree. When I pull the tree it is 2.6.30-rc5 without the amd64_edac driver. It has no branch outside of "master".

Thanks,
Keith

#6 lore raw
Quoted 13 lines · Keith Mannthey
Quoted 12 lines · Borislav Petkov

Hi Keith,

Quoted 6 lines · Keith Mannthey
Quoted 3 lines

By the way, your patches made me look harder at that code region and I've found some more problems with it which I've fixed. Would you like to test the whole bunch of fixes on your setup?

Yes please send any changes you have. I have a decent test setup for live errors.

The patch queue is at

Quoted 7 lines
Quoted 3 lines

git://git.kernel.org/pub/scm/linux/kernel/git/bp/bp.git

the 'edac-fixes' branch.

Please check you have synced with that git tree. When I pull the tree it is 2.6.30-rc5 without the amd64_edac driver. It has no branch outside of "master".

This happens because it is a bare repository without a working tree. You'll have to add it as a remote repository to an already existing (with a working tree) one you have locally like so:

(in your repository do:)

git remote add edac git://git.kernel.org/pub/scm/linux/kernel/git/bp/bp.git

git fetch edac

git checkout -b edac-test edac/edac-fixes

Now you should be on the branch with the patchset.

Let me know how it goes.

Thanks.

Signature · 8 lines
Regards/Gruss,
Boris.

Operating | Advanced Micro Devices GmbH
  System  | Karl-Hammerschmidt-Str. 34, 85609 Dornach b. München, Germany
 Research | Geschäftsführer: Andrew Bowd, Thomas M. McCoy, Giuliano Meroni
  Center  | Sitz: Dornach, Gemeinde Aschheim, Landkreis München
  (OSRC)  | Registergericht München, HRB Nr. 43632