Skip to contentSkip to conversation

powerpc/fsl: Add support for pci(e) machine check exception on E500MC / E5500

PATCH In review Guenter Roeck 4 participants 14 messages Sep 29, 2014 – Oct 9, 2014 PowerPC

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 powerpc/fsl: Add support for pci(e) machine check exception on E500MC / E5500

From: Jojy G Varghese <[email protected]>

For E500MC and E5500, a machine check exception in pci(e) memory space crashes the kernel.

Testing shows that the MCAR(U) register is zero on a MC exception for the E5500 core. At the same time, DEAR register has been found to have the address of the faulty load address during an MC exception for this core.

This fix changes the current behavior to fixup the result register and instruction pointers in the case of a load operation on a faulty PCI address.

The changes are:
- Added the hook to pci machine check handing to the e500mc machine check exception handler.
- For the E5500 core, load faulting address from SPRN_DEAR register. As mentioned above, this is necessary because the E5500 core does not report the fault address in the MCAR register.

Cc Scott Wood <[email protected]>
[Guenter Roeck: updated description]
---
 arch/powerpc/kernel/traps.c   | 3 ++-
 arch/powerpc/sysdev/fsl_pci.c | 5 +++++
 2 files changed, 7 insertions(+), 1 deletion(-)
arch/powerpc/kernel/traps.c +2 -1
index 0dc43f9..ecb709b 100644
@@ -494,7 +494,8 @@ int machine_check_e500mc(struct pt_regs *regs)
 	int recoverable = 1;
 
 	if (reason & MCSR_LD) {
-		recoverable = fsl_rio_mcheck_exception(regs);
+		recoverable = fsl_rio_mcheck_exception(regs) ||
+			fsl_pci_mcheck_exception(regs);
 		if (recoverable == 1)
 			goto silent_out;
 	}
arch/powerpc/sysdev/fsl_pci.c +5 -0
index c507767..bdb956b 100644
@@ -1021,6 +1021,11 @@ int fsl_pci_mcheck_exception(struct pt_regs *regs)
 #endif
 	addr += mfspr(SPRN_MCAR);
 
+#ifdef CONFIG_E5500_CPU
+	if (mfspr(SPRN_EPCR) & SPRN_EPCR_ICM)
+		addr = PFN_PHYS(vmalloc_to_pfn((void *)mfspr(SPRN_DEAR)));
+#endif
+
 	if (is_in_pci_mem_space(addr)) {
 		if (user_mode(regs)) {
 			pagefault_disable();
Signature · 1 line
1.9.1
#1 lore raw
Quoted 49 lines · Guenter Roeck

From: Jojy G Varghese <[email protected]>

For E500MC and E5500, a machine check exception in pci(e) memory space crashes the kernel.

Testing shows that the MCAR(U) register is zero on a MC exception for the E5500 core. At the same time, DEAR register has been found to have the address of the faulty load address during an MC exception for this core.

This fix changes the current behavior to fixup the result register and instruction pointers in the case of a load operation on a faulty PCI address.

The changes are:
- Added the hook to pci machine check handing to the e500mc machine check exception handler.
- For the E5500 core, load faulting address from SPRN_DEAR register. As mentioned above, this is necessary because the E5500 core does not report the fault address in the MCAR register.

Cc Scott Wood <[email protected]>
[Guenter Roeck: updated description]
---
 arch/powerpc/kernel/traps.c   | 3 ++-
 arch/powerpc/sysdev/fsl_pci.c | 5 +++++
 2 files changed, 7 insertions(+), 1 deletion(-)
arch/powerpc/kernel/traps.c +2 -1
index 0dc43f9..ecb709b 100644
@@ -494,7 +494,8 @@ int machine_check_e500mc(struct pt_regs *regs)
 	int recoverable = 1;
 
 	if (reason & MCSR_LD) {
-		recoverable = fsl_rio_mcheck_exception(regs);
+		recoverable = fsl_rio_mcheck_exception(regs) ||
+			fsl_pci_mcheck_exception(regs);
 		if (recoverable == 1)
 			goto silent_out;
 	}
arch/powerpc/sysdev/fsl_pci.c +0 -0
index c507767..bdb956b 100644

arch/powerpc/sysdev/fsl_pci.c int fsl_pci_mcheck_exception(struct pt_regs *regs) · line 1027

Quoted 7 lines
arch/powerpc/sysdev/fsl_pci.c +4 -0
@@ -1021,6 +1021,11 @@ int fsl_pci_mcheck_exception(struct pt_regs *regs)
 #endif
 	addr += mfspr(SPRN_MCAR);
 
+#ifdef CONFIG_E5500_CPU
+	if (mfspr(SPRN_EPCR) & SPRN_EPCR_ICM)
+		addr = PFN_PHYS(vmalloc_to_pfn((void *)mfspr(SPRN_DEAR)));
+#endif

Kconfig tells you what hardware is supported, not what hardware you're actually running on.

Jia Hongtao, do you know anything about this issue? Is there an erratum? What chips are affected by the the erratum covered by
<http://patchwork.ozlabs.org/patch/240239/>?

Can we rely on DEAR or is this just a side effect of likely having taken a TLB miss for the address recently? Perhaps we should use the instruction emulation to determine the effective address instead.

Guenter, is this patch intended to deal with an erratum or are you covering up legitimate errors?

-Scott

#2 lore raw
Quoted 53 lines · Scott Wood
Quoted 52 lines · Guenter Roeck

From: Jojy G Varghese <[email protected]>

For E500MC and E5500, a machine check exception in pci(e) memory space crashes the kernel.

Testing shows that the MCAR(U) register is zero on a MC exception for the E5500 core. At the same time, DEAR register has been found to have the address of the faulty load address during an MC exception for this core.

This fix changes the current behavior to fixup the result register and instruction pointers in the case of a load operation on a faulty PCI address.

The changes are:
- Added the hook to pci machine check handing to the e500mc machine check exception handler.
- For the E5500 core, load faulting address from SPRN_DEAR register. As mentioned above, this is necessary because the E5500 core does not report the fault address in the MCAR register.

Cc Scott Wood <[email protected]>
[Guenter Roeck: updated description]
---
 arch/powerpc/kernel/traps.c   | 3 ++-
 arch/powerpc/sysdev/fsl_pci.c | 5 +++++
 2 files changed, 7 insertions(+), 1 deletion(-)
arch/powerpc/kernel/traps.c +2 -1
index 0dc43f9..ecb709b 100644
@@ -494,7 +494,8 @@ int machine_check_e500mc(struct pt_regs *regs)
 	int recoverable = 1;
 
 	if (reason & MCSR_LD) {
-		recoverable = fsl_rio_mcheck_exception(regs);
+		recoverable = fsl_rio_mcheck_exception(regs) ||
+			fsl_pci_mcheck_exception(regs);
 		if (recoverable == 1)
 			goto silent_out;
 	}
arch/powerpc/sysdev/fsl_pci.c +0 -0
index c507767..bdb956b 100644
@@ -1021,6 +1021,11 @@ int fsl_pci_mcheck_exception(struct pt_regs *regs)
 #endif
 	addr += mfspr(SPRN_MCAR);
 
Quoted 8 lines

arch/powerpc/sysdev/fsl_pci.c int fsl_pci_mcheck_exception(struct pt_regs *regs) · line 1027

Quoted 4 lines
arch/powerpc/sysdev/fsl_pci.c +4 -0
+#ifdef CONFIG_E5500_CPU
+	if (mfspr(SPRN_EPCR) & SPRN_EPCR_ICM)
+		addr = PFN_PHYS(vmalloc_to_pfn((void *)mfspr(SPRN_DEAR)));
+#endif

Kconfig tells you what hardware is supported, not what hardware you're actually running on.

Hi Scott,

Good point. Jojy, guess we'll have to check if the CPU is actually an E5500. Can you look into that ?

Quoted 4 lines

Jia Hongtao, do you know anything about this issue? Is there an erratum? What chips are affected by the the erratum covered by
<http://patchwork.ozlabs.org/patch/240239/>?

We already have and use the above patch(es) in our kernel. It works fine for E500 (P2020), but does not address E5500 (P5020/P5040).

Quoted 7 lines

Can we rely on DEAR or is this just a side effect of likely having taken a TLB miss for the address recently? Perhaps we should use the instruction emulation to determine the effective address instead.

Guenter, is this patch intended to deal with an erratum or are you covering up legitimate errors?

Those are errors related to PCIe hotplug, and are seen with unexpected PCIe device removals (triggered, for example, by removing power from a PCIe adapter). The behavior we see on E5500 is quite similar to the same behavior on E500:
If unhandled, the CPU keeps executing the same instruction over and over again if there is an error on a PCIe access and thus stalls. I don't know if this is considered an erratum or expected behavior, but it is one we have to address since we have to be able to handle that condition. Ultimately, we'll want to implement PCIe error handlers for the affected drivers, but that will be a next step.

Please let me know if you have a better solution to address this problem.

Thanks,
Guenter

#3 lore raw

DQoNCk9uIDkvMjkvMTQgMTI6MDYgUE0sICJHdWVudGVyIFJvZWNrIiA8bGludXhAcm9lY2stdXMu bmV0PiB3cm90ZToNCg0KPk9uIE1vbiwgU2VwIDI5LCAyMDE0IGF0IDAxOjM2OjA2UE0gLTA1MDAs IFNjb3R0IFdvb2Qgd3JvdGU6DQo+PiBPbiBNb24sIDIwMTQtMDktMjkgYXQgMDk6NDggLTA3MDAs IEd1ZW50ZXIgUm9lY2sgd3JvdGU6DQo+PiA+IEZyb206IEpvankgRyBWYXJnaGVzZSA8am9qeXZA anVuaXBlci5uZXQ+DQo+PiA+IA0KPj4gPiBGb3IgRTUwME1DIGFuZCBFNTUwMCwgYSBtYWNoaW5l IGNoZWNrIGV4Y2VwdGlvbiBpbiBwY2koZSkgbWVtb3J5IHNwYWNlDQo+PiA+IGNyYXNoZXMgdGhl IGtlcm5lbC4NCj4+ID4gDQo+PiA+IFRlc3Rpbmcgc2hvd3MgdGhhdCB0aGUgTUNBUihVKSByZWdp c3RlciBpcyB6ZXJvIG9uIGEgTUMgZXhjZXB0aW9uIGZvcg0KPj50aGUNCj4+ID4gRTU1MDAgY29y ZS4gQXQgdGhlIHNhbWUgdGltZSwgREVBUiByZWdpc3RlciBoYXMgYmVlbiBmb3VuZCB0byBoYXZl IHRoZQ0KPj4gPiBhZGRyZXNzIG9mIHRoZSBmYXVsdHkgbG9hZCBhZGRyZXNzIGR1cmluZyBhbiBN QyBleGNlcHRpb24gZm9yIHRoaXMNCj4+Y29yZS4NCj4+ID4gDQo+PiA+IFRoaXMgZml4IGNoYW5n ZXMgdGhlIGN1cnJlbnQgYmVoYXZpb3IgdG8gZml4dXAgdGhlIHJlc3VsdCByZWdpc3Rlcg0KPj4g PiBhbmQgaW5zdHJ1Y3Rpb24gcG9pbnRlcnMgaW4gdGhlIGNhc2Ugb2YgYSBsb2FkIG9wZXJhdGlv biBvbiBhIGZhdWx0eQ0KPj4gPiBQQ0kgYWRkcmVzcy4NCj4+ID4gDQo+PiA+IFRoZSBjaGFuZ2Vz IGFyZToNCj4+ID4gLSBBZGRlZCB0aGUgaG9vayB0byBwY2kgbWFjaGluZSBjaGVjayBoYW5kaW5n IHRvIHRoZSBlNTAwbWMgbWFjaGluZQ0KPj5jaGVjaw0KPj4gPiAgIGV4Y2VwdGlvbiBoYW5kbGVy Lg0KPj4gPiAtIEZvciB0aGUgRTU1MDAgY29yZSwgbG9hZCBmYXVsdGluZyBhZGRyZXNzIGZyb20g U1BSTl9ERUFSIHJlZ2lzdGVyLg0KPj4gPiAgIEFzIG1lbnRpb25lZCBhYm92ZSwgdGhpcyBpcyBu ZWNlc3NhcnkgYmVjYXVzZSB0aGUgRTU1MDAgY29yZSBkb2VzDQo+Pm5vdA0KPj4gPiAgIHJlcG9y dCB0aGUgZmF1bHQgYWRkcmVzcyBpbiB0aGUgTUNBUiByZWdpc3Rlci4NCj4+ID4gDQo+PiA+IENj OiBTY290dCBXb29kIDxzY290dHdvb2RAZnJlZXNjYWxlLmNvbT4NCj4+ID4gU2lnbmVkLW9mZi1i eTogSm9qeSBHIFZhcmdoZXNlIDxqb2p5dkBqdW5pcGVyLm5ldD4NCj4+ID4gW0d1ZW50ZXIgUm9l Y2s6IHVwZGF0ZWQgZGVzY3JpcHRpb25dDQo+PiA+IFNpZ25lZC1vZmYtYnk6IEd1ZW50ZXIgUm9l Y2sgPGdyb2Vja0BqdW5pcGVyLm5ldD4NCj4+ID4gU2lnbmVkLW9mZi1ieTogR3VlbnRlciBSb2Vj ayA8bGludXhAcm9lY2stdXMubmV0Pg0KPj4gPiAtLS0NCj4+ID4gIGFyY2gvcG93ZXJwYy9rZXJu ZWwvdHJhcHMuYyAgIHwgMyArKy0NCj4+ID4gIGFyY2gvcG93ZXJwYy9zeXNkZXYvZnNsX3BjaS5j IHwgNSArKysrKw0KPj4gPiAgMiBmaWxlcyBjaGFuZ2VkLCA3IGluc2VydGlvbnMoKyksIDEgZGVs ZXRpb24oLSkNCj4+ID4gDQo+PiA+IGRpZmYgLS1naXQgYS9hcmNoL3Bvd2VycGMva2VybmVsL3Ry YXBzLmMgYi9hcmNoL3Bvd2VycGMva2VybmVsL3RyYXBzLmMNCj4+ID4gaW5kZXggMGRjNDNmOS4u ZWNiNzA5YiAxMDA2NDQNCj4+ID4gLS0tIGEvYXJjaC9wb3dlcnBjL2tlcm5lbC90cmFwcy5jDQo+ PiA+ICsrKyBiL2FyY2gvcG93ZXJwYy9rZXJuZWwvdHJhcHMuYw0KPj4gPiBAQCAtNDk0LDcgKzQ5 NCw4IEBAIGludCBtYWNoaW5lX2NoZWNrX2U1MDBtYyhzdHJ1Y3QgcHRfcmVncyAqcmVncykNCj4+ ID4gIAlpbnQgcmVjb3ZlcmFibGUgPSAxOw0KPj4gPiAgDQo+PiA+ICAJaWYgKHJlYXNvbiAmIE1D U1JfTEQpIHsNCj4+ID4gLQkJcmVjb3ZlcmFibGUgPSBmc2xfcmlvX21jaGVja19leGNlcHRpb24o cmVncyk7DQo+PiA+ICsJCXJlY292ZXJhYmxlID0gZnNsX3Jpb19tY2hlY2tfZXhjZXB0aW9uKHJl Z3MpIHx8DQo+PiA+ICsJCQlmc2xfcGNpX21jaGVja19leGNlcHRpb24ocmVncyk7DQo+PiA+ICAJ CWlmIChyZWNvdmVyYWJsZSA9PSAxKQ0KPj4gPiAgCQkJZ290byBzaWxlbnRfb3V0Ow0KPj4gPiAg CX0NCj4+ID4gZGlmZiAtLWdpdCBhL2FyY2gvcG93ZXJwYy9zeXNkZXYvZnNsX3BjaS5jDQo+PmIv YXJjaC9wb3dlcnBjL3N5c2Rldi9mc2xfcGNpLmMNCj4+ID4gaW5kZXggYzUwNzc2Ny4uYmRiOTU2 YiAxMDA2NDQNCj4+ID4gLS0tIGEvYXJjaC9wb3dlcnBjL3N5c2Rldi9mc2xfcGNpLmMNCj4+ID4g KysrIGIvYXJjaC9wb3dlcnBjL3N5c2Rldi9mc2xfcGNpLmMNCj4+ID4gQEAgLTEwMjEsNiArMTAy MSwxMSBAQCBpbnQgZnNsX3BjaV9tY2hlY2tfZXhjZXB0aW9uKHN0cnVjdCBwdF9yZWdzDQo+Pipy ZWdzKQ0KPj4gPiAgI2VuZGlmDQo+PiA+ICAJYWRkciArPSBtZnNwcihTUFJOX01DQVIpOw0KPj4g PiAgDQo+PiA+ICsjaWZkZWYgQ09ORklHX0U1NTAwX0NQVQ0KPj4gPiArCWlmIChtZnNwcihTUFJO X0VQQ1IpICYgU1BSTl9FUENSX0lDTSkNCj4+ID4gKwkJYWRkciA9IFBGTl9QSFlTKHZtYWxsb2Nf dG9fcGZuKCh2b2lkICopbWZzcHIoU1BSTl9ERUFSKSkpOw0KPj4gPiArI2VuZGlmDQo+PiANCj4+ IEtjb25maWcgdGVsbHMgeW91IHdoYXQgaGFyZHdhcmUgaXMgc3VwcG9ydGVkLCBub3Qgd2hhdCBo YXJkd2FyZSB5b3UncmUNCj4+IGFjdHVhbGx5IHJ1bm5pbmcgb24uDQo+PiANCj5IaSBTY290dCwN Cj4NCj5Hb29kIHBvaW50LiBKb2p5LCBndWVzcyB3ZSdsbCBoYXZlIHRvIGNoZWNrIGlmIHRoZSBD UFUgaXMgYWN0dWFsbHkgYW4NCj5FNTUwMC4NCj5DYW4geW91IGxvb2sgaW50byB0aGF0ID8NCg0K DQoiL3Byb2MvY3B1aW5mbyIgc2hvd3MgdGhlIGNwdSBhcyAiZTU1MDAiLiBTY290dCwgYXJlIHlv dSBzdWdnZXN0aW5nIHRoYXQNCndlIHVzZSBhIHJ1bnRpbWUgbWV0aG9kIG9mIGRldGVybWluaW5n IHRoZSBjcHUgdHlwZSAoY3B1X3NwZWMncyBjcHVfbmFtZQ0KZm9yDQpleGFtcGxlKS4gIA0KDQoN Cj4NCj4+IEppYSBIb25ndGFvLCBkbyB5b3Uga25vdyBhbnl0aGluZyBhYm91dCB0aGlzIGlzc3Vl PyAgSXMgdGhlcmUgYW4NCj4+IGVycmF0dW0/ICBXaGF0IGNoaXBzIGFyZSBhZmZlY3RlZCBieSB0 aGUgdGhlIGVycmF0dW0gY292ZXJlZCBieQ0KPj4gPGh0dHA6Ly9wYXRjaHdvcmsub3psYWJzLm9y Zy9wYXRjaC8yNDAyMzkvPj8NCj4+IA0KPldlIGFscmVhZHkgaGF2ZSBhbmQgdXNlIHRoZSBhYm92 ZSBwYXRjaChlcykgaW4gb3VyIGtlcm5lbC4gSXQgd29ya3MgZmluZQ0KPmZvciBFNTAwIChQMjAy MCksIGJ1dCBkb2VzIG5vdCBhZGRyZXNzIEU1NTAwIChQNTAyMC9QNTA0MCkuDQo+DQo+PiBDYW4g d2UgcmVseSBvbiBERUFSIG9yIGlzIHRoaXMganVzdCBhIHNpZGUgZWZmZWN0IG9mIGxpa2VseSBo YXZpbmcgdGFrZW4NCj4+IGEgVExCIG1pc3MgZm9yIHRoZSBhZGRyZXNzIHJlY2VudGx5PyAgUGVy aGFwcyB3ZSBzaG91bGQgdXNlIHRoZQ0KPj4gaW5zdHJ1Y3Rpb24gZW11bGF0aW9uIHRvIGRldGVy bWluZSB0aGUgZWZmZWN0aXZlIGFkZHJlc3MgaW5zdGVhZC4NCj4+IA0KPj4gR3VlbnRlciwgaXMg dGhpcyBwYXRjaCBpbnRlbmRlZCB0byBkZWFsIHdpdGggYW4gZXJyYXR1bSBvciBhcmUgeW91DQo+ PiBjb3ZlcmluZyB1cCBsZWdpdGltYXRlIGVycm9ycz8NCj4+IA0KPlRob3NlIGFyZSBlcnJvcnMg cmVsYXRlZCB0byBQQ0llIGhvdHBsdWcsIGFuZCBhcmUgc2VlbiB3aXRoIHVuZXhwZWN0ZWQNCj5Q Q0llDQo+ZGV2aWNlIHJlbW92YWxzICh0cmlnZ2VyZWQsIGZvciBleGFtcGxlLCBieSByZW1vdmlu ZyBwb3dlciBmcm9tIGEgUENJZQ0KPmFkYXB0ZXIpLg0KPlRoZSBiZWhhdmlvciB3ZSBzZWUgb24g RTU1MDAgaXMgcXVpdGUgc2ltaWxhciB0byB0aGUgc2FtZSBiZWhhdmlvciBvbg0KPkU1MDA6DQo+ SWYgdW5oYW5kbGVkLCB0aGUgQ1BVIGtlZXBzIGV4ZWN1dGluZyB0aGUgc2FtZSBpbnN0cnVjdGlv biBvdmVyIGFuZCBvdmVyDQo+YWdhaW4NCj5pZiB0aGVyZSBpcyBhbiBlcnJvciBvbiBhIFBDSWUg YWNjZXNzIGFuZCB0aHVzIHN0YWxscy4gSSBkb24ndCBrbm93IGlmDQo+dGhpcw0KPmlzIGNvbnNp ZGVyZWQgYW4gZXJyYXR1bSBvciBleHBlY3RlZCBiZWhhdmlvciwgYnV0IGl0IGlzIG9uZSB3ZSBo YXZlIHRvDQo+YWRkcmVzcw0KPnNpbmNlIHdlIGhhdmUgdG8gYmUgYWJsZSB0byBoYW5kbGUgdGhh dCBjb25kaXRpb24uIFVsdGltYXRlbHksIHdlJ2xsIHdhbnQNCj50bw0KPmltcGxlbWVudCBQQ0ll IGVycm9yIGhhbmRsZXJzIGZvciB0aGUgYWZmZWN0ZWQgZHJpdmVycywgYnV0IHRoYXQgd2lsbCBi ZQ0KPmEgbmV4dA0KPnN0ZXAuDQoNCkFjY29yZGluZyB0byB0aGUgc3BlYywgd2UgTUNBUiBpcyBz dXBwb3NlZCB0byBob2xkIHRoZSBmYXVsdHkgZGF0YSBhZGRyZXNzDQpidXQgZm9yIDU1MDAgY29y ZSwgd2UgZm91bmQgdGhhdCBNQ0FSIGlzIHplcm8uIFlvdSBhcmUgcmlnaHQgdGhhdCBERUFSDQpl bnRyeSBjb3VsZA0KYmUgYSByZXN1bHRPZiBhIFRMQiBtaXNzIGJ1dCB0aGF0qfZzIHRoZSByZWdp c3RlciB3ZSBjb3VsZCByZWx5IG9uLg0KDQpXaGF0IGRvIHlvdSBtZWFuIGJ5ICJpbnN0cnVjdGlv biBlbXVsYXRpb24iPyBBcmUgeW91IHN1Z2dlc3RpbmcgdGhhdCB3ZQ0KZXhhbWluZSB0aGUgUkQs IFJTIA0KcmVnaXN0ZXJzIGZvciB0aGUgaW5zdHJ1Y3Rpb24/DQoNCg0KDQo+DQo+UGxlYXNlIGxl dCBtZSBrbm93IGlmIHlvdSBoYXZlIGEgYmV0dGVyIHNvbHV0aW9uIHRvIGFkZHJlc3MgdGhpcyBw cm9ibGVtLg0KPg0KPlRoYW5rcywNCj5HdWVudGVyDQoNCg0KVGhhbmtzDQpKb2p5DQoNCg==

#4 lore raw
Quoted 63 lines · Jojy Varghese
Quoted 60 lines · Guenter Roeck
Quoted 59 lines · Scott Wood
Quoted 6 lines · Guenter Roeck

From: Jojy G Varghese <[email protected]>

For E500MC and E5500, a machine check exception in pci(e) memory space crashes the kernel.

Testing shows that the MCAR(U) register is zero on a MC exception for

the

Quoted 2 lines

E5500 core. At the same time, DEAR register has been found to have the address of the faulty load address during an MC exception for this

core.

Quoted 7 lines

This fix changes the current behavior to fixup the result register and instruction pointers in the case of a load operation on a faulty PCI address.

The changes are:
- Added the hook to pci machine check handing to the e500mc machine

check

Quoted 3 lines

exception handler.
- For the E5500 core, load faulting address from SPRN_DEAR register. As mentioned above, this is necessary because the E5500 core does

not

Review

Quoted 27 lines

report the fault address in the MCAR register.

Cc Scott Wood <[email protected]>
[Guenter Roeck: updated description]
---
 arch/powerpc/kernel/traps.c   | 3 ++-
 arch/powerpc/sysdev/fsl_pci.c | 5 +++++
 2 files changed, 7 insertions(+), 1 deletion(-)
arch/powerpc/kernel/traps.c +2 -1
index 0dc43f9..ecb709b 100644
@@ -494,7 +494,8 @@ int machine_check_e500mc(struct pt_regs *regs)
 	int recoverable = 1;
 
 	if (reason & MCSR_LD) {
-		recoverable = fsl_rio_mcheck_exception(regs);
+		recoverable = fsl_rio_mcheck_exception(regs) ||
+			fsl_pci_mcheck_exception(regs);
 		if (recoverable == 1)
 			goto silent_out;
 	}
(quoted diff) +0 -0

b/arch/powerpc/sysdev/fsl_pci.c

arch/powerpc/sysdev/fsl_pci.c int fsl_pci_mcheck_exception(struct pt_regs

Quoted 4 lines

index c507767..bdb956b 100644

arch/powerpc/sysdev/fsl_pci.c +0 -0
@@ -1021,6 +1021,11 @@ int fsl_pci_mcheck_exception(struct pt_regs

*regs)

Quoted 3 lines
arch/powerpc/sysdev/fsl_pci.c +0 -0
 #endif
 	addr += mfspr(SPRN_MCAR);
 
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines

arch/powerpc/sysdev/fsl_pci.c int fsl_pci_mcheck_exception(struct pt_regs *regs) · line 1027

Quoted 4 lines
arch/powerpc/sysdev/fsl_pci.c +4 -0
+#ifdef CONFIG_E5500_CPU
+	if (mfspr(SPRN_EPCR) & SPRN_EPCR_ICM)
+		addr = PFN_PHYS(vmalloc_to_pfn((void *)mfspr(SPRN_DEAR)));
+#endif

Kconfig tells you what hardware is supported, not what hardware you're actually running on.

Plus, CONFIG_E5500_CPU may not even be set when running on an e5500, as it is used for selecting GCC optimization settings. You could have CONFIG_GENERIC_CPU instead.

And the subject says "E500MC / E5500", not just "E5500". :-)

Quoted 11 lines
Quoted 5 lines

Hi Scott,

Good point. Jojy, guess we'll have to check if the CPU is actually an E5500.
Can you look into that ?

"/proc/cpuinfo" shows the cpu as "e5500". Scott, are you suggesting that we use a runtime method of determining the cpu type (cpu_spec's cpu_name for
example).

Yes, if there's a bug to be worked around, and we don't want to apply the workaround unconditionally, you should use PVR to determine whether you're running on an affected core.

Quoted 8 lines
Quoted 7 lines
Quoted 7 lines

Can we rely on DEAR or is this just a side effect of likely having taken a TLB miss for the address recently? Perhaps we should use the instruction emulation to determine the effective address instead.

Guenter, is this patch intended to deal with an erratum or are you covering up legitimate errors?

Quoted 13 lines
Quoted 5 lines

Those are errors related to PCIe hotplug, and are seen with unexpected PCIe
device removals (triggered, for example, by removing power from a PCIe adapter).
The behavior we see on E5500 is quite similar to the same behavior on

Quoted 8 lines

E500:
If unhandled, the CPU keeps executing the same instruction over and over again
if there is an error on a PCIe access and thus stalls. I don't know if this
is considered an erratum or expected behavior, but it is one we have to address
since we have to be able to handle that condition.

The reason I ask is that the handling for e500 was described as an erratum workaround. If it is an erratum it would be nice to know the erratum number and the full list of affected chips.

Quoted 5 lines
Quoted 5 lines

Ultimately, we'll want
to
implement PCIe error handlers for the affected drivers, but that will be a next
step.

For now can we at least print a ratelimited error message? I don't like the idea of silently ignoring these errors. I suppose it's a separate issue from extending the workaround to cover e500mc, though.

Quoted 2 lines

According to the spec, we MCAR is supposed to hold the faulty data address but for 5500 core, we found that MCAR is zero.

Which specific chip and revision did you see this on? What is the value in MCSR?

Quoted 2 lines

You are right that DEAR entry could
be a resultOf a TLB miss but that¹s the register we could rely on.

If it's the result of a previous TLB miss then we can't rely on it. The translation might have been loaded into the TLB before the hotplug event, or there might have been an interrupt between loading the translation into the TLB and using the translation.

Quoted 1 line

What do you mean by "instruction emulation"?

mcheck_handle_load()

Quoted 3 lines

Are you suggesting that we
examine the RD, RS
registers for the instruction?

Yes, if we don't have a simpler reliable source of the address.

-Scott

#5 lore raw
Quoted 116 lines · Scott Wood
Quoted 63 lines · Jojy Varghese
Quoted 60 lines · Guenter Roeck
Quoted 59 lines · Scott Wood
Quoted 6 lines · Guenter Roeck

From: Jojy G Varghese <[email protected]>

For E500MC and E5500, a machine check exception in pci(e) memory space crashes the kernel.

Testing shows that the MCAR(U) register is zero on a MC exception for

the

Quoted 2 lines

E5500 core. At the same time, DEAR register has been found to have the address of the faulty load address during an MC exception for this

core.

Quoted 7 lines

This fix changes the current behavior to fixup the result register and instruction pointers in the case of a load operation on a faulty PCI address.

The changes are:
- Added the hook to pci machine check handing to the e500mc machine

check

Quoted 3 lines

exception handler.
- For the E5500 core, load faulting address from SPRN_DEAR register. As mentioned above, this is necessary because the E5500 core does

not

Review

Quoted 27 lines

report the fault address in the MCAR register.

Cc Scott Wood <[email protected]>
[Guenter Roeck: updated description]
---
 arch/powerpc/kernel/traps.c   | 3 ++-
 arch/powerpc/sysdev/fsl_pci.c | 5 +++++
 2 files changed, 7 insertions(+), 1 deletion(-)
arch/powerpc/kernel/traps.c +2 -1
index 0dc43f9..ecb709b 100644
@@ -494,7 +494,8 @@ int machine_check_e500mc(struct pt_regs *regs)
 	int recoverable = 1;
 
 	if (reason & MCSR_LD) {
-		recoverable = fsl_rio_mcheck_exception(regs);
+		recoverable = fsl_rio_mcheck_exception(regs) ||
+			fsl_pci_mcheck_exception(regs);
 		if (recoverable == 1)
 			goto silent_out;
 	}
(quoted diff) +0 -0

b/arch/powerpc/sysdev/fsl_pci.c

arch/powerpc/sysdev/fsl_pci.c int fsl_pci_mcheck_exception(struct pt_regs

Quoted 4 lines

index c507767..bdb956b 100644

arch/powerpc/sysdev/fsl_pci.c +0 -0
@@ -1021,6 +1021,11 @@ int fsl_pci_mcheck_exception(struct pt_regs

*regs)

Quoted 3 lines
arch/powerpc/sysdev/fsl_pci.c +0 -0
 #endif
 	addr += mfspr(SPRN_MCAR);
 
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines

arch/powerpc/sysdev/fsl_pci.c int fsl_pci_mcheck_exception(struct pt_regs *regs) · line 1027

Quoted 4 lines
arch/powerpc/sysdev/fsl_pci.c +4 -0
+#ifdef CONFIG_E5500_CPU
+	if (mfspr(SPRN_EPCR) & SPRN_EPCR_ICM)
+		addr = PFN_PHYS(vmalloc_to_pfn((void *)mfspr(SPRN_DEAR)));
+#endif

Kconfig tells you what hardware is supported, not what hardware you're actually running on.

Plus, CONFIG_E5500_CPU may not even be set when running on an e5500, as it is used for selecting GCC optimization settings. You could have CONFIG_GENERIC_CPU instead.

And the subject says "E500MC / E5500", not just "E5500". :-)

Quoted 11 lines
Quoted 5 lines

Hi Scott,

Good point. Jojy, guess we'll have to check if the CPU is actually an E5500.
Can you look into that ?

"/proc/cpuinfo" shows the cpu as "e5500". Scott, are you suggesting that we use a runtime method of determining the cpu type (cpu_spec's cpu_name for
example).

Yes, if there's a bug to be worked around, and we don't want to apply the workaround unconditionally, you should use PVR to determine whether you're running on an affected core.

Quoted 8 lines
Quoted 7 lines
Quoted 7 lines

Can we rely on DEAR or is this just a side effect of likely having taken a TLB miss for the address recently? Perhaps we should use the instruction emulation to determine the effective address instead.

Guenter, is this patch intended to deal with an erratum or are you covering up legitimate errors?

Quoted 13 lines
Quoted 5 lines

Those are errors related to PCIe hotplug, and are seen with unexpected PCIe
device removals (triggered, for example, by removing power from a PCIe adapter).
The behavior we see on E5500 is quite similar to the same behavior on

Quoted 8 lines

E500:
If unhandled, the CPU keeps executing the same instruction over and over again
if there is an error on a PCIe access and thus stalls. I don't know if this
is considered an erratum or expected behavior, but it is one we have to address
since we have to be able to handle that condition.

Quoted 4 lines

The reason I ask is that the handling for e500 was described as an erratum workaround. If it is an erratum it would be nice to know the erratum number and the full list of affected chips.

My understanding, which may be wrong, was that this is expected behavior, at least for E5500. I actually thought I had seen it somewhere in the specification (response to PCIe errors), but I don't recall where exactly.

At least for my part I am not aware of an erratum.

Quoted 10 lines
Quoted 5 lines
Quoted 5 lines

Ultimately, we'll want
to
implement PCIe error handlers for the affected drivers, but that will be a next
step.

For now can we at least print a ratelimited error message? I don't like the idea of silently ignoring these errors. I suppose it's a separate issue from extending the workaround to cover e500mc, though.

I don't really like the idea of printing an error message pretty much each time when an unexpected hotplug event occurs.

Quoted 6 lines
Quoted 2 lines

According to the spec, we MCAR is supposed to hold the faulty data address but for 5500 core, we found that MCAR is zero.

Which specific chip and revision did you see this on? What is the value in MCSR?

Jojy can answer that, at least for P5020. We have seen it on P5040 as well, though, so it is not just limited to one chip/revision.

Guenter

Quoted 21 lines
Quoted 2 lines

You are right that DEAR entry could
be a resultOf a TLB miss but that¹s the register we could rely on.

If it's the result of a previous TLB miss then we can't rely on it. The translation might have been loaded into the TLB before the hotplug event, or there might have been an interrupt between loading the translation into the TLB and using the translation.

Quoted 1 line

What do you mean by "instruction emulation"?

mcheck_handle_load()

Quoted 3 lines

Are you suggesting that we
examine the RD, RS
registers for the instruction?

Yes, if we don't have a simpler reliable source of the address.

-Scott

#6 lore raw
Quoted 165 lines · Guenter Roeck
Quoted 133 lines · Scott Wood
Quoted 8 lines · Jojy Varghese
Quoted 5 lines · Guenter Roeck
Quoted 4 lines · Scott Wood
Quoted 3 lines · Guenter Roeck

From: Jojy G Varghese <[email protected]>

For E500MC and E5500, a machine check exception in pci(e) memory

space

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

crashes the kernel.

Testing shows that the MCAR(U) register is zero on a MC

exception for

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

the

Quoted 1 line

E5500 core. At the same time, DEAR register has been found to

have the

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

address of the faulty load address during an MC exception for

this

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

core.

Quoted 2 lines

This fix changes the current behavior to fixup the result

register

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

and instruction pointers in the case of a load operation on a

faulty

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

PCI address.

The changes are:
- Added the hook to pci machine check handing to the e500mc

machine

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

check

Quoted 2 lines

exception handler.
- For the E5500 core, load faulting address from SPRN_DEAR

register.

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

As mentioned above, this is necessary because the E5500 core

does

Quoted 13 lines
Quoted 13 lines
Quoted 13 lines

not

Quoted 12 lines

report the fault address in the MCAR register.

Cc Scott Wood <[email protected]>
[Guenter Roeck: updated description]
---
 arch/powerpc/kernel/traps.c   | 3 ++-
 arch/powerpc/sysdev/fsl_pci.c | 5 +++++
 2 files changed, 7 insertions(+), 1 deletion(-)
Quoted 1 line
Quoted 1 line
Quoted 1 line

Review

Quoted 1 line
(quoted diff) +0 -0

b/arch/powerpc/kernel/traps.c

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

arch/powerpc/kernel/traps.c int machine_check_e500mc(struct pt_regs

Quoted 4 lines

index 0dc43f9..ecb709b 100644

arch/powerpc/kernel/traps.c +0 -0
@@ -494,7 +494,8 @@ int machine_check_e500mc(struct pt_regs

*regs)

Quoted 11 lines
Quoted 11 lines
Quoted 11 lines

Review

Quoted 10 lines
arch/powerpc/kernel/traps.c +2 -1
 	int recoverable = 1;
 
 	if (reason & MCSR_LD) {
-		recoverable = fsl_rio_mcheck_exception(regs);
+		recoverable = fsl_rio_mcheck_exception(regs) ||
+			fsl_pci_mcheck_exception(regs);
 		if (recoverable == 1)
 			goto silent_out;
 	}
(quoted diff) +0 -0

b/arch/powerpc/sysdev/fsl_pci.c

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

arch/powerpc/sysdev/fsl_pci.c int fsl_pci_mcheck_exception(struct

Quoted 4 lines

index c507767..bdb956b 100644

arch/powerpc/sysdev/fsl_pci.c +0 -0
@@ -1021,6 +1021,11 @@ int fsl_pci_mcheck_exception(struct

pt_regs

Quoted 10 lines
Quoted 10 lines
Quoted 10 lines

*regs)

arch/powerpc/sysdev/fsl_pci.c int fsl_pci_mcheck_exception(struct pt_regs *regs) · line 1027

Quoted 7 lines
arch/powerpc/sysdev/fsl_pci.c +4 -0
 #endif
 	addr += mfspr(SPRN_MCAR);
 
+#ifdef CONFIG_E5500_CPU
+	if (mfspr(SPRN_EPCR) & SPRN_EPCR_ICM)
+		addr = PFN_PHYS(vmalloc_to_pfn((void *)mfspr(SPRN_DEAR)));
+#endif

Kconfig tells you what hardware is supported, not what hardware

you're

Quoted 1 line
Quoted 1 line
Quoted 1 line

actually running on.

Plus, CONFIG_E5500_CPU may not even be set when running on an e5500, as it is used for selecting GCC optimization settings. You could have CONFIG_GENERIC_CPU instead.

And the subject says "E500MC / E5500", not just "E5500". :-)

Quoted 8 lines
Quoted 5 lines

Hi Scott,

Good point. Jojy, guess we'll have to check if the CPU is actually an E5500.
Can you look into that ?

"/proc/cpuinfo" shows the cpu as "e5500". Scott, are you suggesting

that

Quoted 1 line

we use a runtime method of determining the cpu type (cpu_spec's

cpu_name

Quoted 2 lines

for
example).

Yes, if there's a bug to be worked around, and we don't want to apply the workaround unconditionally, you should use PVR to determine whether you're running on an affected core.

Quoted 1 line
Quoted 1 line
Quoted 1 line

Can we rely on DEAR or is this just a side effect of likely having

taken

Quoted 8 lines
Quoted 8 lines
Quoted 6 lines

a TLB miss for the address recently? Perhaps we should use the instruction emulation to determine the effective address instead.

Guenter, is this patch intended to deal with an erratum or are you covering up legitimate errors?

Those are errors related to PCIe hotplug, and are seen with

unexpected

Quoted 2 lines
Quoted 2 lines

PCIe
device removals (triggered, for example, by removing power from a

PCIe

Quoted 4 lines
Quoted 4 lines

adapter).
The behavior we see on E5500 is quite similar to the same behavior on E500:
If unhandled, the CPU keeps executing the same instruction over and

over

Quoted 2 lines
Quoted 2 lines

again
if there is an error on a PCIe access and thus stalls. I don't know

if

Quoted 2 lines
Quoted 2 lines

this
is considered an erratum or expected behavior, but it is one we have

Quoted 8 lines

to

Quoted 2 lines
Quoted 2 lines

address
since we have to be able to handle that condition.

The reason I ask is that the handling for e500 was described as an erratum workaround. If it is an erratum it would be nice to know the erratum number and the full list of affected chips.

My understanding, which may be wrong, was that this is expected behavior, at least for E5500. I actually thought I had seen it somewhere in the specification (response to PCIe errors), but I don't recall where exactly.

At least for my part I am not aware of an erratum.

Quoted 11 lines
Quoted 3 lines
Quoted 3 lines

Ultimately, we'll want
to
implement PCIe error handlers for the affected drivers, but that

will be

Quoted 2 lines
Quoted 2 lines

a next
step.

For now can we at least print a ratelimited error message? I don't like the idea of silently ignoring these errors. I suppose it's a separate issue from extending the workaround to cover e500mc, though.

I don't really like the idea of printing an error message pretty much each time
when an unexpected hotplug event occurs.

Quoted 2 lines
Quoted 1 line

According to the spec, we MCAR is supposed to hold the faulty data

address

Quoted 8 lines
Quoted 5 lines
Quoted 1 line

but for 5500 core, we found that MCAR is zero.

Which specific chip and revision did you see this on? What is the value in MCSR?

Jojy can answer that, at least for P5020. We have seen it on P5040 as well,
though, so it is not just limited to one chip/revision.

The specifics are:
PVR: 0x80240012
Instruction that causes the MC exception: lwbrx
The faulty load address is also present in RB. So we could change the
logic to use that
instead of DEAR. What I don’t know is of there are other cases also which escapes the current logic.

Quoted 24 lines

Guenter

Quoted 21 lines
Quoted 2 lines

You are right that DEAR entry could
be a resultOf a TLB miss but that¹s the register we could rely on.

If it's the result of a previous TLB miss then we can't rely on it. The translation might have been loaded into the TLB before the hotplug event, or there might have been an interrupt between loading the translation into the TLB and using the translation.

Quoted 1 line

What do you mean by "instruction emulation"?

mcheck_handle_load()

Quoted 3 lines

Are you suggesting that we examine the RD, RS
registers for the instruction?

Yes, if we don't have a simpler reliable source of the address.

-Scott

�淸�{.n�+돴윯돪�+%듚�깁負쪐w��{.n�+돴쪐{깸G쳺��{ay����,j�f"톒쉹��묎�z_溫(�쉸듶줷"얎�m����G쳺�◀?솳鈺�&x�~뤳턨O뺞촾퇉v�^�m��� �띠�◀?뻀�

#7 lore raw
Quoted 11 lines · Jojy Varghese
Quoted 7 lines · Guenter Roeck
Quoted 3 lines · Scott Wood

Which specific chip and revision did you see this on? What is the value in MCSR?

Jojy can answer that, at least for P5020. We have seen it on P5040 as well,
though, so it is not just limited to one chip/revision.

Quoted 7 lines

The specifics are:
PVR: 0x80240012
Instruction that causes the MC exception: lwbrx
The faulty load address is also present in RB. So we could change the
logic to use that
instead of DEAR. What I don’t know is of there are other cases also which escapes the current logic.

Could you find out what MCSR was when that happened? I'm most interested in whether MAV was set, but the other bits would be interesting as well.

-Scott

#8 lore raw
Quoted 22 lines · Scott Wood
Quoted 12 lines · Jojy Varghese
Quoted 2 lines · Guenter Roeck
Quoted 1 line · Scott Wood

Which specific chip and revision did you see this on? What is the

value

Quoted 5 lines
Quoted 2 lines

in MCSR?

Jojy can answer that, at least for P5020. We have seen it on P5040 as well,
though, so it is not just limited to one chip/revision.

Quoted 8 lines

The specifics are:
PVR: 0x80240012
Instruction that causes the MC exception: lwbrx
The faulty load address is also present in RB. So we could change the
logic to use that
instead of DEAR. What I don¹t know is of there are other cases also which
escapes the current logic.

Quoted 3 lines

Could you find out what MCSR was when that happened? I'm most interested in whether MAV was set, but the other bits would be interesting as well.

MCSR=a000 ( Load Error Report)

Quoted 4 lines

-Scott

Thanks
Jojy

#9 lore raw
In reply to Guenter Roeck
Quoted 23 lines · Guenter Roeck
Quoted 18 lines · Scott Wood
Quoted 16 lines · Jojy Varghese
Quoted 5 lines · Guenter Roeck

Those are errors related to PCIe hotplug, and are seen with unexpected PCIe
device removals (triggered, for example, by removing power from a PCIe adapter).
The behavior we see on E5500 is quite similar to the same behavior on

Quoted 8 lines

E500:
If unhandled, the CPU keeps executing the same instruction over and over again
if there is an error on a PCIe access and thus stalls. I don't know if this
is considered an erratum or expected behavior, but it is one we have to address
since we have to be able to handle that condition.

Quoted 4 lines

The reason I ask is that the handling for e500 was described as an erratum workaround. If it is an erratum it would be nice to know the erratum number and the full list of affected chips.

Quoted 5 lines

My understanding, which may be wrong, was that this is expected behavior, at least for E5500. I actually thought I had seen it somewhere in the specification (response to PCIe errors), but I don't recall where exactly.

At least for my part I am not aware of an erratum.

Jia Hongtao, can you comment here?

Quoted 6 lines
Quoted 6 lines
Quoted 5 lines
Quoted 5 lines

Ultimately, we'll want
to
implement PCIe error handlers for the affected drivers, but that will be a next
step.

Quoted 6 lines
Quoted 4 lines

For now can we at least print a ratelimited error message? I don't like the idea of silently ignoring these errors. I suppose it's a separate issue from extending the workaround to cover e500mc, though.

I don't really like the idea of printing an error message pretty much each time when an unexpected hotplug event occurs.

Unexpected events seem like the sort of thing you'd want to log, but my concern is that this might not be the only cause of PCI errors.

-Scott

#10 lore raw

DQoNCj4gLS0tLS1PcmlnaW5hbCBNZXNzYWdlLS0tLS0NCj4gRnJvbTogV29vZCBTY290dC1CMDc0 MjENCj4gU2VudDogV2VkbmVzZGF5LCBPY3RvYmVyIDAxLCAyMDE0IDg6NDQgQU0NCj4gVG86IEd1 ZW50ZXIgUm9lY2sNCj4gQ2M6IEpvankgVmFyZ2hlc2U7IEJlbmphbWluIEhlcnJlbnNjaG1pZHQ7 IFBhdWwgTWFja2VycmFzOyBNaWNoYWVsDQo+IEVsbGVybWFuOyBsaW51eHBwYy1kZXZAbGlzdHMu b3psYWJzLm9yZzsgbGludXgta2VybmVsQHZnZXIua2VybmVsLm9yZzsNCj4gR3VlbnRlciBSb2Vj azsgSmlhIEhvbmd0YW8tQjM4OTUxDQo+IFN1YmplY3Q6IFJlOiBbUEFUQ0hdIHBvd2VycGMvZnNs OiBBZGQgc3VwcG9ydCBmb3IgcGNpKGUpIG1hY2hpbmUgY2hlY2sNCj4gZXhjZXB0aW9uIG9uIEU1 MDBNQyAvIEU1NTAwDQo+IA0KPiBPbiBUdWUsIDIwMTQtMDktMzAgYXQgMDg6NTAgLTA3MDAsIEd1 ZW50ZXIgUm9lY2sgd3JvdGU6DQo+ID4gT24gTW9uLCBTZXAgMjksIDIwMTQgYXQgMDY6MzE6MDZQ TSAtMDUwMCwgU2NvdHQgV29vZCB3cm90ZToNCj4gPiA+IE9uIE1vbiwgMjAxNC0wOS0yOSBhdCAy MzowMyArMDAwMCwgSm9qeSBWYXJnaGVzZSB3cm90ZToNCj4gPiA+ID4NCj4gPiA+ID4gT24gOS8y OS8xNCAxMjowNiBQTSwgIkd1ZW50ZXIgUm9lY2siIDxsaW51eEByb2Vjay11cy5uZXQ+IHdyb3Rl Og0KPiA+ID4gPg0KPiA+ID4gPiA+VGhvc2UgYXJlIGVycm9ycyByZWxhdGVkIHRvIFBDSWUgaG90 cGx1ZywgYW5kIGFyZSBzZWVuIHdpdGgNCj4gPiA+ID4gPnVuZXhwZWN0ZWQgUENJZSBkZXZpY2Ug cmVtb3ZhbHMgKHRyaWdnZXJlZCwgZm9yIGV4YW1wbGUsIGJ5DQo+ID4gPiA+ID5yZW1vdmluZyBw b3dlciBmcm9tIGEgUENJZSBhZGFwdGVyKS4NCj4gPiA+ID4gPlRoZSBiZWhhdmlvciB3ZSBzZWUg b24gRTU1MDAgaXMgcXVpdGUgc2ltaWxhciB0byB0aGUgc2FtZQ0KPiA+ID4gPiA+YmVoYXZpb3Ig b24NCj4gPiA+ID4gPkU1MDA6DQo+ID4gPiA+ID5JZiB1bmhhbmRsZWQsIHRoZSBDUFUga2VlcHMg ZXhlY3V0aW5nIHRoZSBzYW1lIGluc3RydWN0aW9uIG92ZXINCj4gPiA+ID4gPmFuZCBvdmVyIGFn YWluIGlmIHRoZXJlIGlzIGFuIGVycm9yIG9uIGEgUENJZSBhY2Nlc3MgYW5kIHRodXMNCj4gPiA+ ID4gPnN0YWxscy4gSSBkb24ndCBrbm93IGlmIHRoaXMgaXMgY29uc2lkZXJlZCBhbiBlcnJhdHVt IG9yIGV4cGVjdGVkDQo+ID4gPiA+ID5iZWhhdmlvciwgYnV0IGl0IGlzIG9uZSB3ZSBoYXZlIHRv IGFkZHJlc3Mgc2luY2Ugd2UgaGF2ZSB0byBiZQ0KPiA+ID4gPiA+YWJsZSB0byBoYW5kbGUgdGhh dCBjb25kaXRpb24uDQo+ID4gPg0KPiA+ID4gVGhlIHJlYXNvbiBJIGFzayBpcyB0aGF0IHRoZSBo YW5kbGluZyBmb3IgZTUwMCB3YXMgZGVzY3JpYmVkIGFzIGFuDQo+ID4gPiBlcnJhdHVtIHdvcmth cm91bmQuICBJZiBpdCBpcyBhbiBlcnJhdHVtIGl0IHdvdWxkIGJlIG5pY2UgdG8ga25vdw0KPiA+ ID4gdGhlIGVycmF0dW0gbnVtYmVyIGFuZCB0aGUgZnVsbCBsaXN0IG9mIGFmZmVjdGVkIGNoaXBz Lg0KPiA+ID4NCj4gPiBNeSB1bmRlcnN0YW5kaW5nLCB3aGljaCBtYXkgYmUgd3JvbmcsIHdhcyB0 aGF0IHRoaXMgaXMgZXhwZWN0ZWQNCj4gPiBiZWhhdmlvciwgYXQgbGVhc3QgZm9yIEU1NTAwLiBJ IGFjdHVhbGx5IHRob3VnaHQgSSBoYWQgc2VlbiBpdA0KPiA+IHNvbWV3aGVyZSBpbiB0aGUgc3Bl Y2lmaWNhdGlvbiAocmVzcG9uc2UgdG8gUENJZSBlcnJvcnMpLCBidXQgSSBkb24ndA0KPiByZWNh bGwgd2hlcmUgZXhhY3RseS4NCj4gPg0KPiA+IEF0IGxlYXN0IGZvciBteSBwYXJ0IEkgYW0gbm90 IGF3YXJlIG9mIGFuIGVycmF0dW0uDQo+IA0KPiBKaWEgSG9uZ3RhbywgY2FuIHlvdSBjb21tZW50 IGhlcmU/DQoNCkkgZGlkIG5vdCBmaW5kIGFueSByZWxhdGVkIGVycmF0dW0gZWl0aGVyLg0KDQo+ IA0KPiA+ID4gPiA+VWx0aW1hdGVseSwgd2UnbGwgd2FudA0KPiA+ID4gPiA+dG8NCj4gPiA+ID4g PmltcGxlbWVudCBQQ0llIGVycm9yIGhhbmRsZXJzIGZvciB0aGUgYWZmZWN0ZWQgZHJpdmVycywg YnV0IHRoYXQNCj4gPiA+ID4gPndpbGwgYmUgYSBuZXh0IHN0ZXAuDQo+ID4gPg0KPiA+ID4gRm9y IG5vdyBjYW4gd2UgYXQgbGVhc3QgcHJpbnQgYSByYXRlbGltaXRlZCBlcnJvciBtZXNzYWdlPyAg SSBkb24ndA0KPiA+ID4gbGlrZSB0aGUgaWRlYSBvZiBzaWxlbnRseSBpZ25vcmluZyB0aGVzZSBl cnJvcnMuICBJIHN1cHBvc2UgaXQncyBhDQo+ID4gPiBzZXBhcmF0ZSBpc3N1ZSBmcm9tIGV4dGVu ZGluZyB0aGUgd29ya2Fyb3VuZCB0byBjb3ZlciBlNTAwbWMsIHRob3VnaC4NCj4gPiA+DQo+ID4g SSBkb24ndCByZWFsbHkgbGlrZSB0aGUgaWRlYSBvZiBwcmludGluZyBhbiBlcnJvciBtZXNzYWdl IHByZXR0eSBtdWNoDQo+ID4gZWFjaCB0aW1lIHdoZW4gYW4gdW5leHBlY3RlZCBob3RwbHVnIGV2 ZW50IG9jY3Vycy4NCj4gDQo+IFVuZXhwZWN0ZWQgZXZlbnRzIHNlZW0gbGlrZSB0aGUgc29ydCBv ZiB0aGluZyB5b3UnZCB3YW50IHRvIGxvZywgYnV0IG15DQo+IGNvbmNlcm4gaXMgdGhhdCB0aGlz IG1pZ2h0IG5vdCBiZSB0aGUgb25seSBjYXVzZSBvZiBQQ0kgZXJyb3JzLg0KPiANCj4gLVNjb3R0 DQo+IA0KDQo=

#11 lore raw
In reply to Scott Wood
Quoted 72 lines

-----Original Message-----
From: Wood Scott-B07421
Sent: Tuesday, September 30, 2014 2:36 AM
To: Guenter Roeck
Cc: Benjamin Herrenschmidt; Paul Mackerras; Michael Ellerman; linuxppc- [email protected]; [email protected]; Jojy G Varghese; Guenter Roeck; Jia Hongtao-B38951
Subject: Re: [PATCH] powerpc/fsl: Add support for pci(e) machine check exception on E500MC / E5500

Quoted 10 lines · Guenter Roeck

From: Jojy G Varghese <[email protected]>

For E500MC and E5500, a machine check exception in pci(e) memory space crashes the kernel.

Testing shows that the MCAR(U) register is zero on a MC exception for the
E5500 core. At the same time, DEAR register has been found to have the address of the faulty load address during an MC exception for this core.

Quoted 6 lines

This fix changes the current behavior to fixup the result register and instruction pointers in the case of a load operation on a faulty PCI address.

The changes are:
- Added the hook to pci machine check handing to the e500mc machine

check

Quoted 34 lines

exception handler.
- For the E5500 core, load faulting address from SPRN_DEAR register. As mentioned above, this is necessary because the E5500 core does not report the fault address in the MCAR register.

updated description]

---
 arch/powerpc/kernel/traps.c   | 3 ++-
 arch/powerpc/sysdev/fsl_pci.c | 5 +++++
 2 files changed, 7 insertions(+), 1 deletion(-)
arch/powerpc/kernel/traps.c +2 -1
index 0dc43f9..ecb709b 100644
@@ -494,7 +494,8 @@ int machine_check_e500mc(struct pt_regs *regs)
 	int recoverable = 1;
 
 	if (reason & MCSR_LD) {
-		recoverable = fsl_rio_mcheck_exception(regs);
+		recoverable = fsl_rio_mcheck_exception(regs) ||
+			fsl_pci_mcheck_exception(regs);
 		if (recoverable == 1)
 			goto silent_out;
 	}
(quoted diff) +0 -0

b/arch/powerpc/sysdev/fsl_pci.c index c507767..bdb956b 100644

arch/powerpc/sysdev/fsl_pci.c int fsl_pci_mcheck_exception(struct pt_regs · line 1026

Quoted 6 lines
arch/powerpc/sysdev/fsl_pci.c +3 -0
@@ -1021,6 +1021,11 @@ int fsl_pci_mcheck_exception(struct pt_regs
 *regs)  #endif
 	addr += mfspr(SPRN_MCAR);
 
+#ifdef CONFIG_E5500_CPU
+	if (mfspr(SPRN_EPCR) & SPRN_EPCR_ICM)
+		addr = PFN_PHYS(vmalloc_to_pfn((void *)mfspr(SPRN_DEAR)));

#endif

Quoted 4 lines

Kconfig tells you what hardware is supported, not what hardware you're actually running on.

Jia Hongtao, do you know anything about this issue? Is there an erratum?

Sorry for the late response, I just return from my vacation. I don't know this issue.

Quoted 2 lines

What chips are affected by the the erratum covered by
<http://patchwork.ozlabs.org/patch/240239/>?

MPC8544, MPC8548, MPC8572 are affected by this erratum. I checked P4080 which using e500mc and no such erratum is found.

Quoted 10 lines

Can we rely on DEAR or is this just a side effect of likely having taken a TLB miss for the address recently? Perhaps we should use the instruction emulation to determine the effective address instead.

Guenter, is this patch intended to deal with an erratum or are you covering up legitimate errors?

-Scott

���{.n�+�������+%�����ݶ��w��{.n�+����{��G�����{ay�ʇڙ�,j��f���h��������z_�(�階�ݢj"���m������G������?����&���~��iO��z��v�^�m���� ������?�I�

#12 lore raw
Quoted 76 lines · On Tue, 2014-10-07 at 22:08 -0500, Jia Hongtao-B38951 wrote:
Quoted 71 lines

-----Original Message-----
From: Wood Scott-B07421
Sent: Tuesday, September 30, 2014 2:36 AM
To: Guenter Roeck
Cc: Benjamin Herrenschmidt; Paul Mackerras; Michael Ellerman; linuxppc- [email protected]; [email protected]; Jojy G Varghese; Guenter Roeck; Jia Hongtao-B38951
Subject: Re: [PATCH] powerpc/fsl: Add support for pci(e) machine check exception on E500MC / E5500

Quoted 10 lines · Guenter Roeck

From: Jojy G Varghese <[email protected]>

For E500MC and E5500, a machine check exception in pci(e) memory space crashes the kernel.

Testing shows that the MCAR(U) register is zero on a MC exception for the
E5500 core. At the same time, DEAR register has been found to have the address of the faulty load address during an MC exception for this core.

Quoted 6 lines

This fix changes the current behavior to fixup the result register and instruction pointers in the case of a load operation on a faulty PCI address.

The changes are:
- Added the hook to pci machine check handing to the e500mc machine

check

Quoted 34 lines

exception handler.
- For the E5500 core, load faulting address from SPRN_DEAR register. As mentioned above, this is necessary because the E5500 core does not report the fault address in the MCAR register.

updated description]

---
 arch/powerpc/kernel/traps.c   | 3 ++-
 arch/powerpc/sysdev/fsl_pci.c | 5 +++++
 2 files changed, 7 insertions(+), 1 deletion(-)
arch/powerpc/kernel/traps.c +2 -1
index 0dc43f9..ecb709b 100644
@@ -494,7 +494,8 @@ int machine_check_e500mc(struct pt_regs *regs)
 	int recoverable = 1;
 
 	if (reason & MCSR_LD) {
-		recoverable = fsl_rio_mcheck_exception(regs);
+		recoverable = fsl_rio_mcheck_exception(regs) ||
+			fsl_pci_mcheck_exception(regs);
 		if (recoverable == 1)
 			goto silent_out;
 	}
(quoted diff) +0 -0

b/arch/powerpc/sysdev/fsl_pci.c index c507767..bdb956b 100644

arch/powerpc/sysdev/fsl_pci.c int fsl_pci_mcheck_exception(struct pt_regs · line 1026

Quoted 6 lines
arch/powerpc/sysdev/fsl_pci.c +3 -0
@@ -1021,6 +1021,11 @@ int fsl_pci_mcheck_exception(struct pt_regs
 *regs)  #endif
 	addr += mfspr(SPRN_MCAR);
 
+#ifdef CONFIG_E5500_CPU
+	if (mfspr(SPRN_EPCR) & SPRN_EPCR_ICM)
+		addr = PFN_PHYS(vmalloc_to_pfn((void *)mfspr(SPRN_DEAR)));

#endif

Quoted 4 lines

Kconfig tells you what hardware is supported, not what hardware you're actually running on.

Jia Hongtao, do you know anything about this issue? Is there an erratum?

Quoted 7 lines

Sorry for the late response, I just return from my vacation. I don't know this issue.

Quoted 2 lines

What chips are affected by the the erratum covered by
<http://patchwork.ozlabs.org/patch/240239/>?

MPC8544, MPC8548, MPC8572 are affected by this erratum.

What is the erratum number?

Quoted 1 line

I checked P4080 which using e500mc and no such erratum is found.

What is the erratum behavior, and how does it differ from the problem that Jojy and Guenter are trying to solve?

-Scott

#13 lore raw

DQoNCj4gLS0tLS1PcmlnaW5hbCBNZXNzYWdlLS0tLS0NCj4gRnJvbTogV29vZCBTY290dC1CMDc0 MjENCj4gU2VudDogVGh1cnNkYXksIE9jdG9iZXIgMDksIDIwMTQgNzo0OCBBTQ0KPiBUbzogSmlh IEhvbmd0YW8tQjM4OTUxDQo+IENjOiBHdWVudGVyIFJvZWNrOyBCZW5qYW1pbiBIZXJyZW5zY2ht aWR0OyBQYXVsIE1hY2tlcnJhczsgTWljaGFlbA0KPiBFbGxlcm1hbjsgbGludXhwcGMtZGV2QGxp c3RzLm96bGFicy5vcmc7IGxpbnV4LWtlcm5lbEB2Z2VyLmtlcm5lbC5vcmc7DQo+IEpvankgRyBW YXJnaGVzZTsgR3VlbnRlciBSb2Vjaw0KPiBTdWJqZWN0OiBSZTogW1BBVENIXSBwb3dlcnBjL2Zz bDogQWRkIHN1cHBvcnQgZm9yIHBjaShlKSBtYWNoaW5lIGNoZWNrDQo+IGV4Y2VwdGlvbiBvbiBF NTAwTUMgLyBFNTUwMA0KPiANCj4gT24gVHVlLCAyMDE0LTEwLTA3IGF0IDIyOjA4IC0wNTAwLCBK aWEgSG9uZ3Rhby1CMzg5NTEgd3JvdGU6DQo+ID4NCj4gPiA+IC0tLS0tT3JpZ2luYWwgTWVzc2Fn ZS0tLS0tDQo+ID4gPiBGcm9tOiBXb29kIFNjb3R0LUIwNzQyMQ0KPiA+ID4gU2VudDogVHVlc2Rh eSwgU2VwdGVtYmVyIDMwLCAyMDE0IDI6MzYgQU0NCj4gPiA+IFRvOiBHdWVudGVyIFJvZWNrDQo+ ID4gPiBDYzogQmVuamFtaW4gSGVycmVuc2NobWlkdDsgUGF1bCBNYWNrZXJyYXM7IE1pY2hhZWwg RWxsZXJtYW47DQo+ID4gPiBsaW51eHBwYy0gZGV2QGxpc3RzLm96bGFicy5vcmc7IGxpbnV4LWtl cm5lbEB2Z2VyLmtlcm5lbC5vcmc7IEpvankgRw0KPiA+ID4gVmFyZ2hlc2U7IEd1ZW50ZXIgUm9l Y2s7IEppYSBIb25ndGFvLUIzODk1MQ0KPiA+ID4gU3ViamVjdDogUmU6IFtQQVRDSF0gcG93ZXJw Yy9mc2w6IEFkZCBzdXBwb3J0IGZvciBwY2koZSkgbWFjaGluZQ0KPiA+ID4gY2hlY2sgZXhjZXB0 aW9uIG9uIEU1MDBNQyAvIEU1NTAwDQo+ID4gPg0KPiA+ID4gT24gTW9uLCAyMDE0LTA5LTI5IGF0 IDA5OjQ4IC0wNzAwLCBHdWVudGVyIFJvZWNrIHdyb3RlOg0KPiA+ID4gPiBGcm9tOiBKb2p5IEcg VmFyZ2hlc2UgPGpvanl2QGp1bmlwZXIubmV0Pg0KPiA+ID4gPg0KPiA+ID4gPiBGb3IgRTUwME1D IGFuZCBFNTUwMCwgYSBtYWNoaW5lIGNoZWNrIGV4Y2VwdGlvbiBpbiBwY2koZSkgbWVtb3J5DQo+ ID4gPiA+IHNwYWNlIGNyYXNoZXMgdGhlIGtlcm5lbC4NCj4gPiA+ID4NCj4gPiA+ID4gVGVzdGlu ZyBzaG93cyB0aGF0IHRoZSBNQ0FSKFUpIHJlZ2lzdGVyIGlzIHplcm8gb24gYSBNQyBleGNlcHRp b24NCj4gPiA+ID4gZm9yIHRoZQ0KPiA+ID4gPiBFNTUwMCBjb3JlLiBBdCB0aGUgc2FtZSB0aW1l LCBERUFSIHJlZ2lzdGVyIGhhcyBiZWVuIGZvdW5kIHRvIGhhdmUNCj4gPiA+ID4gdGhlIGFkZHJl c3Mgb2YgdGhlIGZhdWx0eSBsb2FkIGFkZHJlc3MgZHVyaW5nIGFuIE1DIGV4Y2VwdGlvbiBmb3IN Cj4gdGhpcyBjb3JlLg0KPiA+ID4gPg0KPiA+ID4gPiBUaGlzIGZpeCBjaGFuZ2VzIHRoZSBjdXJy ZW50IGJlaGF2aW9yIHRvIGZpeHVwIHRoZSByZXN1bHQgcmVnaXN0ZXINCj4gPiA+ID4gYW5kIGlu c3RydWN0aW9uIHBvaW50ZXJzIGluIHRoZSBjYXNlIG9mIGEgbG9hZCBvcGVyYXRpb24gb24gYQ0K PiA+ID4gPiBmYXVsdHkgUENJIGFkZHJlc3MuDQo+ID4gPiA+DQo+ID4gPiA+IFRoZSBjaGFuZ2Vz IGFyZToNCj4gPiA+ID4gLSBBZGRlZCB0aGUgaG9vayB0byBwY2kgbWFjaGluZSBjaGVjayBoYW5k aW5nIHRvIHRoZSBlNTAwbWMNCj4gPiA+ID4gbWFjaGluZQ0KPiA+ID4gY2hlY2sNCj4gPiA+ID4g ICBleGNlcHRpb24gaGFuZGxlci4NCj4gPiA+ID4gLSBGb3IgdGhlIEU1NTAwIGNvcmUsIGxvYWQg ZmF1bHRpbmcgYWRkcmVzcyBmcm9tIFNQUk5fREVBUiByZWdpc3Rlci4NCj4gPiA+ID4gICBBcyBt ZW50aW9uZWQgYWJvdmUsIHRoaXMgaXMgbmVjZXNzYXJ5IGJlY2F1c2UgdGhlIEU1NTAwIGNvcmUg ZG9lcw0KPiBub3QNCj4gPiA+ID4gICByZXBvcnQgdGhlIGZhdWx0IGFkZHJlc3MgaW4gdGhlIE1D QVIgcmVnaXN0ZXIuDQo+ID4gPiA+DQo+ID4gPiA+IENjOiBTY290dCBXb29kIDxzY290dHdvb2RA ZnJlZXNjYWxlLmNvbT4NCj4gPiA+ID4gU2lnbmVkLW9mZi1ieTogSm9qeSBHIFZhcmdoZXNlIDxq b2p5dkBqdW5pcGVyLm5ldD4gW0d1ZW50ZXIgUm9lY2s6DQo+ID4gPiA+IHVwZGF0ZWQgZGVzY3Jp cHRpb25dDQo+ID4gPiA+IFNpZ25lZC1vZmYtYnk6IEd1ZW50ZXIgUm9lY2sgPGdyb2Vja0BqdW5p cGVyLm5ldD4NCj4gPiA+ID4gU2lnbmVkLW9mZi1ieTogR3VlbnRlciBSb2VjayA8bGludXhAcm9l Y2stdXMubmV0Pg0KPiA+ID4gPiAtLS0NCj4gPiA+ID4gIGFyY2gvcG93ZXJwYy9rZXJuZWwvdHJh cHMuYyAgIHwgMyArKy0NCj4gPiA+ID4gIGFyY2gvcG93ZXJwYy9zeXNkZXYvZnNsX3BjaS5jIHwg NSArKysrKw0KPiA+ID4gPiAgMiBmaWxlcyBjaGFuZ2VkLCA3IGluc2VydGlvbnMoKyksIDEgZGVs ZXRpb24oLSkNCj4gPiA+ID4NCj4gPiA+ID4gZGlmZiAtLWdpdCBhL2FyY2gvcG93ZXJwYy9rZXJu ZWwvdHJhcHMuYw0KPiA+ID4gPiBiL2FyY2gvcG93ZXJwYy9rZXJuZWwvdHJhcHMuYyBpbmRleCAw ZGM0M2Y5Li5lY2I3MDliIDEwMDY0NA0KPiA+ID4gPiAtLS0gYS9hcmNoL3Bvd2VycGMva2VybmVs L3RyYXBzLmMNCj4gPiA+ID4gKysrIGIvYXJjaC9wb3dlcnBjL2tlcm5lbC90cmFwcy5jDQo+ID4g PiA+IEBAIC00OTQsNyArNDk0LDggQEAgaW50IG1hY2hpbmVfY2hlY2tfZTUwMG1jKHN0cnVjdCBw dF9yZWdzICpyZWdzKQ0KPiA+ID4gPiAgCWludCByZWNvdmVyYWJsZSA9IDE7DQo+ID4gPiA+DQo+ ID4gPiA+ICAJaWYgKHJlYXNvbiAmIE1DU1JfTEQpIHsNCj4gPiA+ID4gLQkJcmVjb3ZlcmFibGUg PSBmc2xfcmlvX21jaGVja19leGNlcHRpb24ocmVncyk7DQo+ID4gPiA+ICsJCXJlY292ZXJhYmxl ID0gZnNsX3Jpb19tY2hlY2tfZXhjZXB0aW9uKHJlZ3MpIHx8DQo+ID4gPiA+ICsJCQlmc2xfcGNp X21jaGVja19leGNlcHRpb24ocmVncyk7DQo+ID4gPiA+ICAJCWlmIChyZWNvdmVyYWJsZSA9PSAx KQ0KPiA+ID4gPiAgCQkJZ290byBzaWxlbnRfb3V0Ow0KPiA+ID4gPiAgCX0NCj4gPiA+ID4gZGlm ZiAtLWdpdCBhL2FyY2gvcG93ZXJwYy9zeXNkZXYvZnNsX3BjaS5jDQo+ID4gPiA+IGIvYXJjaC9w b3dlcnBjL3N5c2Rldi9mc2xfcGNpLmMgaW5kZXggYzUwNzc2Ny4uYmRiOTU2YiAxMDA2NDQNCj4g PiA+ID4gLS0tIGEvYXJjaC9wb3dlcnBjL3N5c2Rldi9mc2xfcGNpLmMNCj4gPiA+ID4gKysrIGIv YXJjaC9wb3dlcnBjL3N5c2Rldi9mc2xfcGNpLmMNCj4gPiA+ID4gQEAgLTEwMjEsNiArMTAyMSwx MSBAQCBpbnQgZnNsX3BjaV9tY2hlY2tfZXhjZXB0aW9uKHN0cnVjdCBwdF9yZWdzDQo+ID4gPiA+ ICpyZWdzKSAgI2VuZGlmDQo+ID4gPiA+ICAJYWRkciArPSBtZnNwcihTUFJOX01DQVIpOw0KPiA+ ID4gPg0KPiA+ID4gPiArI2lmZGVmIENPTkZJR19FNTUwMF9DUFUNCj4gPiA+ID4gKwlpZiAobWZz cHIoU1BSTl9FUENSKSAmIFNQUk5fRVBDUl9JQ00pDQo+ID4gPiA+ICsJCWFkZHIgPSBQRk5fUEhZ Uyh2bWFsbG9jX3RvX3Bmbigodm9pZA0KPiAqKW1mc3ByKFNQUk5fREVBUikpKTsNCj4gPiA+ICNl bmRpZg0KPiA+ID4NCj4gPiA+IEtjb25maWcgdGVsbHMgeW91IHdoYXQgaGFyZHdhcmUgaXMgc3Vw cG9ydGVkLCBub3Qgd2hhdCBoYXJkd2FyZQ0KPiA+ID4geW91J3JlIGFjdHVhbGx5IHJ1bm5pbmcg b24uDQo+ID4gPg0KPiA+ID4gSmlhIEhvbmd0YW8sIGRvIHlvdSBrbm93IGFueXRoaW5nIGFib3V0 IHRoaXMgaXNzdWU/ICBJcyB0aGVyZSBhbg0KPiBlcnJhdHVtPw0KPiA+DQo+ID4gU29ycnkgZm9y IHRoZSBsYXRlIHJlc3BvbnNlLCBJIGp1c3QgcmV0dXJuIGZyb20gbXkgdmFjYXRpb24uDQo+ID4g SSBkb24ndCBrbm93IHRoaXMgaXNzdWUuDQo+ID4NCj4gPiA+IFdoYXQgY2hpcHMgYXJlIGFmZmVj dGVkIGJ5IHRoZSB0aGUgZXJyYXR1bSBjb3ZlcmVkIGJ5DQo+ID4gPiA8aHR0cDovL3BhdGNod29y ay5vemxhYnMub3JnL3BhdGNoLzI0MDIzOS8+Pw0KPiA+DQo+ID4gTVBDODU0NCwgTVBDODU0OCwg TVBDODU3MiBhcmUgYWZmZWN0ZWQgYnkgdGhpcyBlcnJhdHVtLg0KPiANCj4gV2hhdCBpcyB0aGUg ZXJyYXR1bSBudW1iZXI/DQoNClRoZSBudW1iZXIgb2YgdGhpcyBlcnJhdHVtIGZvciBlYWNoIGNo aXAgaXMgbm90IGNvbnNpc3RlbnQuDQpNUEM4NTQ0OiBQQ0llIDQNCk1QQzg1NDg6IFBDSS1FeCAz OQ0KTVBDODU3MjogUENJLUV4IDMNCg0KPiANCj4gPiBJIGNoZWNrZWQgUDQwODAgd2hpY2ggdXNp bmcgZTUwMG1jIGFuZCBubyBzdWNoIGVycmF0dW0gaXMgZm91bmQuDQo+IA0KPiBXaGF0IGlzIHRo ZSBlcnJhdHVtIGJlaGF2aW9yLCBhbmQgaG93IGRvZXMgaXQgZGlmZmVyIGZyb20gdGhlIHByb2Js ZW0NCj4gdGhhdCBKb2p5IGFuZCBHdWVudGVyIGFyZSB0cnlpbmcgdG8gc29sdmU/DQoNCkhlcmUg aXMgdGhlIGRlc2NyaXB0aW9uIG9mIHRoZSBlcnJhdHVtOg0KDQoiV2hlbiBpdHMgbGluayBnb2Vz IGRvd24sIHRoZSBQQ0kgRXhwcmVzcyBjb250cm9sbGVyIGNsZWFycyBhbGwgb3V0c3RhbmRpbmcg dHJhbnNhY3Rpb25zIHdpdGggYW4NCmVycm9yIGluZGljYXRvciBhbmQgc2VuZHMgYSBsaW5rIGRv d24gZXhjZXB0aW9uIHRvIHRoZSBpbnRlcnJ1cHQgY29udHJvbGxlciBpZg0KUEVYX1BNRV9NRVNf RElTUltMREREXSA9IDAuIElmLCBob3dldmVyLCBhbnkgdHJhbnNhY3Rpb25zIGFyZSBzZW50IHRv IHRoZSBjb250cm9sbGVyDQphZnRlciB0aGUgbGluayBkb3duIGV2ZW50LCB0aGV5IHdpbGwgYmUg YWNjZXB0ZWQgYnkgdGhlIGNvbnRyb2xsZXIgYW5kIHdhaXQgZm9yIHRoZSBsaW5rIHRvIGNvbWUN CmJhY2sgdXAgYmVmb3JlIHN0YXJ0aW5nIGFueSB0aW1lb3V0IGNvdW50ZXJzIChlLmcuIGNvbXBs ZXRpb24gdGltZW91dCkuIFRoZXJlIGlzIG5vIG1lY2hhbmlzbQ0KdG8gY2FuY2VsIHRoZSBuZXcg dHJhbnNhY3Rpb25zIHNob3J0IG9mIGEgZGV2aWNlIEhSRVNFVC4iDQoNCkZvciBlNTAwbWMgYXMg Sm9qeSBhbmQgR3VlbnRlciBkZXNjcmliZWQgaXQncyBsaWtlIHRoZSBzYW1lIGVycmF0dW0gb24g ZTUwMCwgbm90IDEwMCUgc3VyZS4NCg0KRm9yIGU1NTAwIEkgZG9uJ3QgcXVpdGUgdW5kZXJzdGFu ZCB5ZXQuDQoNCj4gDQo+IC1TY290dA0KPiANCg0K