amd64_edac: add module registration routines
Quoted 35 lines · Ingo Molnar
From: Ingo Molnar <[email protected]>
Subject: Re: [PATCH 21/21]amd64_edac: add module registration routines
To: "Borislav Petkov" <[email protected]>
Cc: [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], "Doug Thompson" <[email protected]>
Date: Thursday, May 7, 2009, 3:58 PMQuoted 1 line · Borislav Petkov
+ Recent Opterons (Family 10h
and later) provide for Memory Error
Quoted 1 line
+ Injection into the ECC
detection circuits. The
amd64_edacmoduleQuoted 1 line
+ allows the operator/user to
inject Uncorrectable and Correctable
Quoted 3 lines
+ errors into DRAM.
+
+ When enabled, in each ofthe respective memory controller directories
Quoted 1 line
+
(
/sys/devices/system/edac/mc/mcX), there are 3 input files:Quoted 2 lines
+
+ -z_inject_section(0..3,16-byte section of 64-byte cacheline),
Quoted 1 line
+ -
z_inject_word(0..8,16-bit word of 16-byte section),
Quoted 1 line
+ -
z_inject_bit_map(hexbitmap vector: mask bits of 16 bit word to
Quoted 3 lines
+ error-out)
+
+ In addition, there are twocontrol files,
z_inject_readandQuoted 1 line
+
z_inject_write, whichtrigger the Read and Write errors respectively.
Quoted 9 lines
I think the file names should follow existing EDAC driver practices,
and be named according to the existing inject_data_* pattern?There's nothing more annoying to users (and tools) than inconsistent
driver namespaces.
originally the injection was enabled only for development debug, but
was changed to a .config switch. The 'z-*' use was to sort the directory list with those debug files to the bottom of the list.
So now, yes they should have the 'z-*' prefix removed.
Quoted 13 lines
Quoted 1 line
+ /* Exam the capabilities of the
northbridge in order to reflect them
Quoted 1 line
+ * in the presentation
via sysfs attributes, etc
Quoted 1 line
+ */
please use the customary comment style:
/*
* Comment .....
* ...... goes here:
*/
Quoted 5 lines
also described in
Documentation/CodingStyle.This is a continuing observation for a number of other cases where
you introduce half-winged comments.
yeah, we missed some of those
Quoted 33 lines
Quoted 2 lines
+
+ /* IMPORTANT: Set the polling'check' function in this module */
Quoted 4 lines
+
mci->edac_check=amd64_check;
+
+ /* memory scrubber interface */
+mci->set_sdram_scrub_rate=
amd64_set_scrub_rate;Quoted 1 line
+
mci->get_sdram_scrub_rate=
amd64_get_scrub_rate;Review
Quoted 6 lines
(quoted diff) +6
-0+} + +/* + * amd64_probe_one_instance + * + * probe function to determine if thereis a DRAM Controller device is
Quoted 1 line
+ * present and to construct data tables
for it.
Quoted 2 lines
+ *
+ * Due to a hardware feature on Family10H cpus, the Enable Extended
Quoted 1 line
+ * Configuration Space feature MUST be
enabled on ALL Processors prior to
Quoted 1 line
+ * actually reading from the ECS
registers. Since the loading of the module
Quoted 1 line
+ * can occur on any 'core', and cores
don't 'see' all the other processors
Quoted 1 line
+ * ECS data when the others are NOT
enabled. Our solution is to first
Quoted 1 line
+ * enable ECS access in this routine on
all processors, gather some data in
Quoted 1 line
+ * a
amd64_pvtstructure and later come
Quoted 9 lines
back in a '
finishup_setup' functionQuoted 1 line
+ * to perform that final
initialization.
Quoted 3 lines
+ *
+ * See alsoamd64_init_2nd_stage().
+ */Nice comment!
IBM guys created that one with their patch after they found the 'issue'
Quoted 25 lines
Quoted 1 line
+static int amd64_probe_one_instance(struct
pci_dev
*dram_f2_ctl,Quoted 1 line
+
int
mc_type_index)Review
Quoted 6 lines
(quoted diff) +6
-0+{ + struct amd64_pvt *pvt; + int err; + int rc; + + pvt = kzalloc(sizeof(struct
amd64_pvt),GFP_KERNEL);Quoted 4 lines
+ if (pvt == NULL) {
+ rc =-ENOMEM;
+ gotoexit_now;
+ }The customary pattern is:
ret =
-ENOMEM;
pvt = kzalloc(sizeof(structamd64_pvt),GFP_KERNEL);
if (!pvt)
goto err;
Quoted 7 lines
Note the four differences:
- use 'ret' instead of 'rc' for integer returns
- pre-initialize 'ret', not in the exception branch
- !pvt instead of 'pvt == NULL'
- use 'err*' naming for error labels
ok, thanks
Quoted 6 lines
Quoted 2 lines
+
pvt->mc_node_id=
get_mc_node_id_from_pdev(dram_f2_ctl);Quoted 2 lines
+
+ debugf0("=========== %s(Instance=%d) ===========\n",
Quoted 8 lines
Quoted 1 line
+
__func__,
pvt->mc_node_id);please use something like
trace_printk()instead. That will allow
high-performance tracing of this code, should the need arise. Also
'debugf0' says nothing - what does it mean?
debugfX() is a set of macros in the EDAC CORE (already in the kernel) that has 4 levels of verbosity, X=(0..3). Increase the number increases the output. This output is only enabled via the DEBUG switch. In normal operation, DEBUG should be off. It accesses a debug variable in sysfs which allows the developer to increase error reporting granularity.
Quoted 9 lines
Quoted 2 lines
+
+ rc =
amd64_check_ecc_enabled(pvt);Quoted 2 lines
+ if (rc)
+ goto
exit_release_devices;Bug: in the error case there's now a memory leak of 'pvt'.
ow, yeah. Thanks for that catch
Quoted 24 lines
Quoted 3 lines
+
+ /*
+ * Save the pointer tothe private data for use in 2nd initialization
Quoted 3 lines
+ * stage
+ */
+ pvt_lookup[pvt->mc_node_id] =pvt;
Quoted 2 lines
+
+ debugf0("%s(): init 1st stage donepvt-%d\n",
__func__,Quoted 1 line
+
pvt->mc_node_id);Review
Quoted 6 lines
+ return 0;
(quoted diff) +5
-0+ + +exit_release_devices: + pci_dev_put(pvt->addr_f1_ctl); + pci_dev_put(pvt->misc_f3_ctl);this teardown is correct but a bit unclean: it open-codes the
reverse direction of:
Quoted 7 lines
err =
amd64_reserve_mc_sibling_devices(pvt, mc_type_index);It would be better to add a
amd64_free_mc_sibling_devices(pvt)
method.
ah, good point
Quoted 10 lines
Quoted 3 lines
+
+ /*
+ * transfer the infofrom the interium pvt area to the private area of
Quoted 1 line
+ * the MC instance
structure
Quoted 3 lines
+ */
+ pvt =mci->pvt_info;
+ *pvt =*pvt_temp;
Quoted 5 lines
hm, such copying can be dangerous. If
pvt_tempever grows something
like alist_head, then the copy can corrupt the list. Could this be
avoided?
Yeah, it was a misgiving to me as well. It came from the 1st/2nd stage patch set. Needed to 'save' various harvested information during the 1st stage (after all the CPU's had been setup).
The constructor of the MCI object (in EDAC CORE) is a single memory allocated area to hold all the info, but it was dependent on the how many CSROWS and CHANNELS there were to save memory. So the actual 'private' area is within the bounds of the MCI allocation. The temporary allocation was a temp storage area until the MCI structure was finally allocated, which occurs in the 2nd stage pass.
The 'private' area is defined differently in each of the various EDAC modules. The amd64 module definitely has an involved twist on initialization.
Quoted 46 lines
Quoted 2 lines
+
+
kfree(pvt_lookup[pvt->mc_node_id]);Review
Quoted 13 lines
+ pvt_lookup[node_id] = NULL;
(quoted diff) +12
-0+ + return rc; +} + + +/* + * amd64_init_one_instance + * + * initialize just one device + * + * returns: + *count (>= 0), or
Quoted 1 line
+ * negative on
error
Quoted 2 lines
+ */
+static int__devinitamd64_init_one_instance(struct
pci_dev*pdev,Quoted 1 line
+
const
structpci_device_id*mc_type)Review
Quoted 4 lines
(quoted diff) +4
-0+{ + int rc; + + debugf0("%s(MCnode=%d,mc_type='%s')\n",
Quoted 2 lines
+
__func__,
+
get_mc_node_id_from_pdev(pdev),Quoted 1 line
+
get_amd_family_name(mc_type->driver_data));Quoted 7 lines
+
+ /* wake up and enable device */
+ rc =pci_enable_device(pdev);
+ if (rc < 0)
+ rc =-EIO;
+ else
+ rc =
amd64_probe_one_instance(pdev, mc_type->driver_data);Quoted 3 lines
+
+ if (rc < 0)
+ debugf0("%s()
Quoted 7 lines
rc=%d\n",
__func__, rc);there's way too many debugf*() calls in the whole code, often
obscuring the real structure. I think we'll be far better off with
having just a few key ones.
Probably, but the reverse mapping of the amd64 uses many data fields. I added so many debug output to harvest that data so as to debug the code. I added some lines to be able to partition the output to isolate various data output in order to "see" it faster in the log.
But there ARE some lines that are remnants that could be removed.
Quoted 139 lines
Review
Quoted 7 lines
(quoted diff) +7
-0+}; + + +/* + * amd64_setup_pci_device + * + * setup the PCI Device Driver formonitoring PCI errors
Review
Quoted 7 lines
+ */
(quoted diff) +6
-0+static void amd64_setup_pci_device(void) +{ + struct mem_ctl_info *mci; + struct amd64_pvt *pvt; + + if (!amd64_ctl_pci) {this branch goes on:
Quoted 2 lines
+
+ mci =mci_lookup[0];
Quoted 2 lines
+ if (mci) {
+debugf1("%s(): Registering ONE PCI
control\n",Quoted 1 line
+
__func__);and on:
Quoted 2 lines
+
+pvt =
mci->pvt_info;Quoted 1 line
+
amd64_ctl_pci=
edac_pci_create_generic_ctl(Quoted 1 line
+
&pvt->dram_f2_ctl->dev,Quoted 1 line
+
EDAC_MOD_STR);Quoted 1 line
+
if (!amd64_ctl_pci) {
and on:
Quoted 1 line
+
printk(KERN_WARNING
Quoted 1 line
+
"%s(): Unable to create PCI control\n",Quoted 1 line
+
__func__);And here you have too many tabs (five of them), that break up that
printk in an ugly way.The better solution is to realize that the branch could be inverted
and a full level of indentation won by doing:if (
amd64_ctl_pci)
return;And winning another level of indentation later on by doing:
if (mci) {
debugf1("%s(): ONE
PCI control already registered\n",
__func__);
return;
}Btw., please change all instances of:
printk(KERN_WARNING,
=> pr_warning(
printk(KERN_INFO,
=> pr_info(etc.
Quoted 1 line
+
printk(KERN_WARNING
Quoted 1 line
+
"%s(): PCI error report via EDAC "Quoted 1 line
+
"not setup\n",Quoted 1 line
+
__func__);Quoted 1 line
+
}
Quoted 2 lines
+ } else {
+debugf1("%s(): ONE PCI control already
registered\n",Quoted 1 line
+
__func__);Review
Quoted 10 lines
+ }
+ }(quoted diff) +8
-0+} + +static int __init amd64_edac_init(void) +{ + int err; + int node; + + edac_printk(KERN_INFO,
EDAC_MOD_STR,EDAC_AMD64_VERSION"\n");Review
Quoted 4 lines
(quoted diff) +4
-0+ + opstate_init(); + + debugf0("%s()****************** ENTRY
**********************\n",Quoted 1 line
+
__func__);if you want a debug facility, please shorten it to something like:
dbg(" **** ENTRY ****\n")and make the printing of
__func__implicit in thedbg()method.
This will shorten these things quite a bit. But please get rid of
most of them - there's no excuse for having such things in the
kernel beyond the first 1-2 days of the development of this
Quoted 2 lines
function.
Good idea to fold the __func__ into the macro.
For each CPU socket this gets called and I used these to isolate each instance in a 2, 4 or 8 socket based system True, used for ease of debugging
Quoted 20 lines
Quoted 1 line
+ /* Attempt to register drivers for
instances
Quoted 1 line
+ * DUE to the failure of
some 'cores' to access Extended Config Space
Quoted 1 line
+ * prior to all memory
controllers having their ECS register enabled,
Quoted 1 line
+ * the initialization
has been created into 2 stages. Here we
Quoted 1 line
+ * call for the 1st
stage. After all have been enabled, then we
Quoted 1 line
+ * do the 2nd stage to
finishup setup.
Quoted 2 lines
+ */
+ err =
pci_register_driver(&amd64_pci_driver);Quoted 2 lines
+
+ /* At this point, the array'pvt_lookup[]' contains pointers to
Quoted 1 line
+ * allocated struct
amd64_pvtcontrol structures. These will be used
Quoted 8 lines
Quoted 1 line
+ * in the 2nd stage init
function to finish initialization of
Quoted 2 lines
+ * the MC instances.
+ */after using good comment style for a while, bad comment style shows
up again.
yeah, again, didn't get all of them yet
Quoted 6 lines
Quoted 3 lines
+
+ /* if no error occurred on firstpass init, then do 2nd pass init */
Quoted 1 line
+ if (!err) {
Quoted 9 lines
Again, by doing:
if (err)
goto err;You can save an identation level below, and avoid line-wrap
artifacts.
ok
Quoted 127 lines
Quoted 1 line
+
for_each_online_node(node){Quoted 1 line
+
if (!pvt_lookup[node])
Quoted 1 line
+
continue;
Quoted 2 lines
+
+/* If any failure then need to clean up
*/Quoted 1 line
+
err =
amd64_init_2nd_stage(pvt_lookup[node]);Quoted 1 line
+
if (err) {
Quoted 1 line
+
debugf0("%s()
'finish_setup' stage failed\n",Quoted 1 line
+
__func__);Quoted 2 lines
+
+/* undo prior
instances' registrationsQuoted 1 line
+
* and leave
as failedQuoted 1 line
+
*/
Quoted 1 line
+
pci_unregister_driver(&amd64_pci_driver);Quoted 1 line
+
goto
error_exit;Quoted 1 line
+
}
Quoted 2 lines
+ }
+
amd64_setup_pci_device();Review
Quoted 10 lines
+ }
(quoted diff) +9
-0+ +error_exit: + return err; +} + +static void __exit amd64_edac_exit(void) +{ + if (amd64_ctl_pci) +
edac_pci_release_generic_ctl(amd64_ctl_pci);Quoted 2 lines
+
+
pci_unregister_driver(&amd64_pci_driver);Review
Quoted 8 lines
(quoted diff) +8
-0+} + +module_init(amd64_edac_init); +module_exit(amd64_edac_exit); + +MODULE_LICENSE("GPL"); +MODULE_AUTHOR("SoftwareBitMaker: Doug Thompson, " + "Dave Peterson,Thayne Harbaugh");
Quoted 1 line
+MODULE_DESCRIPTION("MC support for AMD64 memory
controllers - "
Quoted 1 line
+
EDAC_AMD64_VERSION);Review
Quoted 3 lines
(quoted diff) +3
-0+ +module_param(edac_op_state, int, 0444); +MODULE_PARM_DESC(edac_op_state, "EDAC Error Reportingstate: 0=Poll,1=NMI");
Review
Quoted 1 line
(quoted diff) +0
-0
b/drivers/edac/amd64_edac.hdrivers/edac/amd64_edac.h
Quoted 10 lines
index d435ace..a3603f7 100644
drivers/edac/amd64_edac.h +0
-0@@ -891,6 +891,11 @@ extern const char *ii_msgs[4]; extern const char *ext_msgs[32]; extern const char *htlink_msgs[8];drivers/edac/amd64_edac.h +3
-0+#define NUM_DBG_ATTRS 9 +#define NUM_INJ_ATTRS 5 +extern struct mcidev_sysfs_attributeamd64_dbg_attrs[NUM_DBG_ATTRS],
Quoted 1 line
+
amd64_inj_attrs[NUM_INJ_ATTRS];Quoted 3 lines
+
/*
* Each of the PCI Device IDs typeshave their own set of hardware
Quoted 1 line
* accessor function and per device
encoding/decoding logic.
Review
Quoted 1 line
(quoted diff) +0
-0
b/drivers/edac/amd64_edac_dbg.cdrivers/edac/amd64_edac_dbg.c static ssize_t
Quoted 4 lines
index 3741af9..f538129 100644
drivers/edac/amd64_edac_dbg.c +0
-0@@ -211,6 +211,9 @@ static ssize_t
amd64_hole_show(struct mem_ctl_info *mci, char *data)Quoted 1 line
hole_size);Quoted 6 lines
}
+/*
+ * updateNUM_DBG_ATTRSin case you add new members
+ */
structmcidev_sysfs_attributeamd64_dbg_attrs[]= {
Review
Quoted 3 lines
{(quoted diff) +0
-0
b/drivers/edac/amd64_edac_inj.cdrivers/edac/amd64_edac_inj.c static ssize_t
Quoted 4 lines
index 7f978cb..8706954 100644
drivers/edac/amd64_edac_inj.c +0
-0@@ -155,6 +155,9 @@ static ssize_tamd64_inject_write_store(struct
mem_ctl_info*mci,Quoted 7 lines
drivers/edac/amd64_edac_inj.c +0
-0return 0; }
+/*
+ * updateNUM_INJ_ATTRSin case you add new members
+ */
structmcidev_sysfs_attributeamd64_inj_attrs[]= {
Quoted 9 lines
Quoted 2 lines
{Looks like these fixlets dont really belong in this patch and should
either be backmerged or put into a separate patch?Ingo
thanks for the diligence Ingo