Skip to contentSkip to conversation

some new unaligned access while booting ia64 (HP rx2620)

PATCH In review Moore, Robert 4 participants 7 messages Feb 10, 2006 – Feb 14, 2006 ACPI

lore
Share & tools
lore
#0 lore raw

This only contains the output of stores to the ACPI "debug" object, not the full trace output. However, "_CRS 0" may help

i.e., like this:

Quoted 30 lines

exstoren-0075 [18] ex_resolve_object : ----Entry
exstoren-0155 [18] ex_resolve_object : ----Exit- AE_OK

-----Original Message-----
From: Luck, Tony
Sent: Friday, February 10, 2006 3:25 PM
To: Moore, Robert; 'Thomas Renninger'; Brown, Len
Cc: '[email protected]'; '[email protected]'
Subject: RE: some new unaligned access while booting ia64 (HP rx2620)

A 16MB log_buf was enough.

Here is the initial trace up until the unaligned access messages:

Linux version 2.6.16-rc2-zx1-smp (aegl@linux-t10) (gcc version 3.4.3

[ACPI Debug]  Integer: 0x00000000
[ACPI Debug]  String: [0x1F] "PARS.GUID: evaluating for COMP2"
[ACPI Debug]  Integer: 0x00000001
[ACPI Debug]  String: [0x12] "SERIAL DEV Ptr is "
[ACPI Debug]  Integer: 0x3FEF99A0

kernel unaligned access to 0xe000004040e7a434, ip=0xa0000001003c81a0 kernel unaligned access to 0xe000004040e7a46c, ip=0xa0000001003c81a0 kernel unaligned access to 0xe000004040e7a4bc, ip=0xa0000001003c81a0 kernel unaligned access to 0xe000004040e7a42c, ip=0xa0000001003c6ca1 kernel unaligned access to 0xe000004040e7a434, ip=0xa0000001003c6cb1

[ACPI Debug]  String: [0x06] "_CRS 0"
[ACPI Debug]  String: [0x0A] "CLIB.LBA 1"
[ACPI Debug]  Integer: 0x00000000
[ACPI Debug]  String: [0x0B] "PARS.GPTR 2"
#1 lore raw
Quoted 4 lines · Robert

This only contains the output of stores to the ACPI "debug" object, not the full trace output. However, "_CRS 0" may help

Something is strange here.
With rc2-gitXY I got this:

-----------------------------------------------------------------------
slab error in cache_free_debugcheck(): cache `size-256': double free, or memory outside object was overwritten

Call Trace:
[<a0000001000145c0>] show_stack+0x40/0xa0
                               sp=e00000011b7f7ae0 bsp=e00000011b7f1400
[<a000000100014650>] dump_stack+0x30/0x60
                               sp=e00000011b7f7cb0 bsp=e00000011b7f13e0
[<a000000100134a50>] __slab_error+0x50/0x80
                               sp=e00000011b7f7cb0 bsp=e00000011b7f13b0
[<a000000100135ee0>] cache_free_debugcheck+0x220/0x600
                               sp=e00000011b7f7cb0 bsp=e00000011b7f1368
[<a000000100139b70>] kfree+0xd0/0x480
                               sp=e00000011b7f7cb0 bsp=e00000011b7f1328
[<a0000001002ea460>] acpi_os_free+0x20/0x40
                               sp=e00000011b7f7cc0 bsp=e00000011b7f1308
[<a00000010031b980>] acpi_walk_resources+0x240/0x280
                               sp=e00000011b7f7cc0 bsp=e00000011b7f12c8
[<a0000001003e4a10>] pci_acpi_scan_root+0x130/0x3e0
                               sp=e00000011b7f7cd0 bsp=e00000011b7f1278
[<a000000100330380>] acpi_pci_root_add+0x480/0x620
                               sp=e00000011b7f7cf0 bsp=e00000011b7f1230
[<a0000001003354c0>] acpi_bus_driver_init+0x80/0xe0
                               sp=e00000011b7f7d10 bsp=e00000011b7f1208
[<a000000100337c80>] acpi_add_single_object+0x1480/0x1680
                               sp=e00000011b7f7d10 bsp=e00000011b7f11a0
[<a000000100338110>] acpi_bus_scan+0x290/0x400
                               sp=e00000011b7f7d30 bsp=e00000011b7f1160
[<a000000100645840>] acpi_scan_init+0x220/0x2c0
                               sp=e00000011b7f7d50 bsp=e00000011b7f1130
[<a000000100009ee0>] init+0x540/0x880
                               sp=e00000011b7f7d60 bsp=e00000011b7f1108
[<a0000001000128b0>] kernel_thread_helper+0xd0/0x100
                               sp=e00000011b7f7e30 bsp=e00000011b7f10e0
[<a0000001000094a0>] start_kernel_thread+0x20/0x40
                               sp=e00000011b7f7e30 bsp=e00000011b7f10e0
e0000040f8de54c8: redzone 1: 0x170fc2a5, redzone 2: 0xf901001a00000028.

slab error in cache_free_debugcheck(): cache `size-256': double free, or memory outside object was overwritten

Call Trace:
[<a0000001000145c0>] show_stack+0x40/0xa0
                               sp=e00000011b7f7ae0 bsp=e00000011b7f1400
[<a000000100014650>] dump_stack+0x30/0x60
                               sp=e00000011b7f7cb0 bsp=e00000011b7f13e0
[<a000000100134a50>] __slab_error+0x50/0x80
                               sp=e00000011b7f7cb0 bsp=e00000011b7f13b0
[<a000000100135ee0>] cache_free_debugcheck+0x220/0x600
                               sp=e00000011b7f7cb0 bsp=e00000011b7f1368
[<a000000100139b70>] kfree+0xd0/0x480
                               sp=e00000011b7f7cb0 bsp=e00000011b7f1328
[<a0000001002ea460>] acpi_os_free+0x20/0x40
                               sp=e00000011b7f7cc0 bsp=e00000011b7f1308
[<a00000010031b980>] acpi_walk_resources+0x240/0x280
                               sp=e00000011b7f7cc0 bsp=e00000011b7f12c8
[<a0000001003e4af0>] pci_acpi_scan_root+0x210/0x3e0
                               sp=e00000011b7f7cd0 bsp=e00000011b7f1278
[<a000000100330380>] acpi_pci_root_add+0x480/0x620
                               sp=e00000011b7f7cf0 bsp=e00000011b7f1230
[<a0000001003354c0>] acpi_bus_driver_init+0x80/0xe0
                               sp=e00000011b7f7d10 bsp=e00000011b7f1208
[<a000000100337c80>] acpi_add_single_object+0x1480/0x1680
                               sp=e00000011b7f7d10 bsp=e00000011b7f11a0
[<a000000100338110>] acpi_bus_scan+0x290/0x400
                               sp=e00000011b7f7d30 bsp=e00000011b7f1160
[<a000000100645840>] acpi_scan_init+0x220/0x2c0
                               sp=e00000011b7f7d50 bsp=e00000011b7f1130
[<a000000100009ee0>] init+0x540/0x880
                               sp=e00000011b7f7d60 bsp=e00000011b7f1108
[<a0000001000128b0>] kernel_thread_helper+0xd0/0x100
                               sp=e00000011b7f7e30 bsp=e00000011b7f10e0
[<a0000001000094a0>] start_kernel_thread+0x20/0x40
                               sp=e00000011b7f7e30 bsp=e00000011b7f10e0
e0000040f89694c8: redzone 1: 0x170fc2a5, redzone 2: 0xf901001a00000028.

slab error in cache_free_debugcheck(): cache `size-256': double free, or memory outside object was overwritten

Call Trace:
[<a0000001000145c0>] show_stack+0x40/0xa0
                               sp=e00000011b7f7a30 bsp=e00000011b7f16d8
[<a000000100014650>] dump_stack+0x30/0x60
                               sp=e00000011b7f7c00 bsp=e00000011b7f16c0
[<a000000100134a50>] __slab_error+0x50/0x80
                               sp=e00000011b7f7c00 bsp=e00000011b7f1690
[<a000000100135ee0>] cache_free_debugcheck+0x220/0x600
                               sp=e00000011b7f7c00 bsp=e00000011b7f1648
[<a000000100139b70>] kfree+0xd0/0x480
                               sp=e00000011b7f7c00 bsp=e00000011b7f1608
[<a0000001002ea460>] acpi_os_free+0x20/0x40
                               sp=e00000011b7f7c10 bsp=e00000011b7f15e0
[<a00000010031b980>] acpi_walk_resources+0x240/0x280
                               sp=e00000011b7f7c10 bsp=e00000011b7f15a0
[<a00000010032c990>] find_pci_rootbridge+0x1d0/0x3a0
                               sp=e00000011b7f7c20 bsp=e00000011b7f1560
[<a0000001003102d0>] acpi_ns_get_device_callback+0x270/0x2e0
                               sp=e00000011b7f7c60 bsp=e00000011b7f1508
[<a000000100314880>] acpi_ns_walk_namespace+0x280/0x2e0
                               sp=e00000011b7f7c80 bsp=e00000011b7f1498
[<a00000010030ff70>] acpi_get_devices+0xb0/0xe0
                               sp=e00000011b7f7c80 bsp=e00000011b7f1458
[<a00000010032c790>] acpi_get_pci_rootbridge_handle+0x70/0xa0
                               sp=e00000011b7f7ca0 bsp=e00000011b7f1428
[<a0000001002b8720>] pci_acpi_find_root_bridge+0x80/0xe0
                               sp=e00000011b7f7cb0 bsp=e00000011b7f13f0
[<a00000010032c0a0>] acpi_platform_notify+0x120/0x4c0
                               sp=e00000011b7f7cc0 bsp=e00000011b7f13b8
[<a000000100391050>] device_add+0x230/0x300
                               sp=e00000011b7f7cd0 bsp=e00000011b7f1378
[<a000000100391150>] device_register+0x30/0x60
                               sp=e00000011b7f7cd0 bsp=e00000011b7f1358
[<a0000001002a7a90>] pci_create_bus+0x1b0/0x600
                               sp=e00000011b7f7cd0 bsp=e00000011b7f1300
[<a0000001002aa130>] pci_scan_bus_parented+0x30/0xa0
                               sp=e00000011b7f7cd0 bsp=e00000011b7f12c8
[<a0000001003e4b20>] pci_acpi_scan_root+0x240/0x3e0
                               sp=e00000011b7f7cd0 bsp=e00000011b7f1278
[<a000000100330380>] acpi_pci_root_add+0x480/0x620
                               sp=e00000011b7f7cf0 bsp=e00000011b7f1230
[<a0000001003354c0>] acpi_bus_driver_init+0x80/0xe0
                               sp=e00000011b7f7d10 bsp=e00000011b7f1208
[<a000000100337c80>] acpi_add_single_object+0x1480/0x1680
                               sp=e00000011b7f7d10 bsp=e00000011b7f11a0
[<a000000100338110>] acpi_bus_scan+0x290/0x400
                               sp=e00000011b7f7d30 bsp=e00000011b7f1160
[<a000000100645840>] acpi_scan_init+0x220/0x2c0
                               sp=e00000011b7f7d50 bsp=e00000011b7f1130
[<a000000100009ee0>] init+0x540/0x880
                               sp=e00000011b7f7d60 bsp=e00000011b7f1108
[<a0000001000128b0>] kernel_thread_helper+0xd0/0x100
                               sp=e00000011b7f7e30 bsp=e00000011b7f10e0
[<a0000001000094a0>] start_kernel_thread+0x20/0x40
                               sp=e00000011b7f7e30 bsp=e00000011b7f10e0
e00000011b7e76b0: redzone 1: 0x170fc2a5, redzone 2: 0xf901001a00000028.

slab error in cache_free_debugcheck(): cache `size-256': double free, or memory outside object was overwritten

Call Trace:
[<a0000001000145c0>] show_stack+0x40/0xa0
                               sp=e00000011b7f7a30 bsp=e00000011b7f16d8
[<a000000100014650>] dump_stack+0x30/0x60
                               sp=e00000011b7f7c00 bsp=e00000011b7f16c0
[<a000000100134a50>] __slab_error+0x50/0x80
                               sp=e00000011b7f7c00 bsp=e00000011b7f1690
[<a000000100135ee0>] cache_free_debugcheck+0x220/0x600
                               sp=e00000011b7f7c00 bsp=e00000011b7f1648
[<a000000100139b70>] kfree+0xd0/0x480
                               sp=e00000011b7f7c00 bsp=e00000011b7f1608
[<a0000001002ea460>] acpi_os_free+0x20/0x40
                               sp=e00000011b7f7c10 bsp=e00000011b7f15e0
[<a00000010031b980>] acpi_walk_resources+0x240/0x280
                               sp=e00000011b7f7c10 bsp=e00000011b7f15a0
[<a00000010032c990>] find_pci_rootbridge+0x1d0/0x3a0
                               sp=e00000011b7f7c20 bsp=e00000011b7f1560
[<a0000001003102d0>] acpi_ns_get_device_callback+0x270/0x2e0
                               sp=e00000011b7f7c60 bsp=e00000011b7f1508
[<a000000100314880>] acpi_ns_walk_namespace+0x280/0x2e0
                               sp=e00000011b7f7c80 bsp=e00000011b7f1498
[<a00000010030ff70>] acpi_get_devices+0xb0/0xe0
                               sp=e00000011b7f7c80 bsp=e00000011b7f1458
[<a00000010032c790>] acpi_get_pci_rootbridge_handle+0x70/0xa0
                               sp=e00000011b7f7ca0 bsp=e00000011b7f1428
[<a0000001002b8720>] pci_acpi_find_root_bridge+0x80/0xe0
                               sp=e00000011b7f7cb0 bsp=e00000011b7f13f0
[<a00000010032c0a0>] acpi_platform_notify+0x120/0x4c0
                               sp=e00000011b7f7cc0 bsp=e00000011b7f13b8
[<a000000100391050>] device_add+0x230/0x300
                               sp=e00000011b7f7cd0 bsp=e00000011b7f1378
[<a000000100391150>] device_register+0x30/0x60
                               sp=e00000011b7f7cd0 bsp=e00000011b7f1358
[<a0000001002a7a90>] pci_create_bus+0x1b0/0x600
                               sp=e00000011b7f7cd0 bsp=e00000011b7f1300
[<a0000001002aa130>] pci_scan_bus_parented+0x30/0xa0
                               sp=e00000011b7f7cd0 bsp=e00000011b7f12c8
[<a0000001003e4b20>] pci_acpi_scan_root+0x240/0x3e0
                               sp=e00000011b7f7cd0 bsp=e00000011b7f1278
[<a000000100330380>] acpi_pci_root_add+0x480/0x620
                               sp=e00000011b7f7cf0 bsp=e00000011b7f1230
[<a0000001003354c0>] acpi_bus_driver_init+0x80/0xe0
                               sp=e00000011b7f7d10 bsp=e00000011b7f1208
[<a000000100337c80>] acpi_add_single_object+0x1480/0x1680
                               sp=e00000011b7f7d10 bsp=e00000011b7f11a0
[<a000000100338110>] acpi_bus_scan+0x290/0x400
                               sp=e00000011b7f7d30 bsp=e00000011b7f1160
[<a000000100645840>] acpi_scan_init+0x220/0x2c0
                               sp=e00000011b7f7d50 bsp=e00000011b7f1130
[<a000000100009ee0>] init+0x540/0x880
                               sp=e00000011b7f7d60 bsp=e00000011b7f1108
[<a0000001000128b0>] kernel_thread_helper+0xd0/0x100
                               sp=e00000011b7f7e30 bsp=e00000011b7f10e0
[<a0000001000094a0>] start_kernel_thread+0x20/0x40
                               sp=e00000011b7f7e30 bsp=e00000011b7f10e0
e0000040fab29d88: redzone 1: 0x170fc2a5, redzone 2: 0xf901001a00000028.

Slab corruption: start=e0000040fab29ea8, len=256
Redzone: 0xad0e3701d2244a/0x5a2cf071.
Last user: [<a00000010028dbc0>](kobject_uevent+0x8a0/0x940) 000: 00 04 01 02 04 02 04 04 6b 6b 6b 6b 07 00 00 00 010: 0c 00 00 00 00 00 00 00 6b 6b 6b 6b 6b 6b 6b 6b Prev obj: start=e0000040fab29d90, len=256
Redzone: 0x5a2cf071/0x5a2cf071.
Last user: [<a0000001002ea460>](acpi_os_free+0x20/0x40) 000: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 010: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b Next obj: start=e0000040fab29fc0, len=256
Redzone: 0x5a2cf071/0x5a2cf071.
Last user: [<a000000100415b20>](skb_release_data+0x1a0/0x1c0) 000: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 010: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b
-----------------------------------------------------------------------

I startet to pack the acpi functions in arch/ia64/pci/pci.c (pci_acpi_scan_root()) into:
acpi_dbg_level=0x21FFFF; /* Enable ACPI DEBUG */
...
acpi_dbg_level=0xF; /* Disable ACPI_DEBUG */

However I now used a rc3 kernel and the pci_acpi_scan_root() function with it's ACPI invokations succeeded. Has there any work been made?

The slab error now happens a bit later in (acpi_sba_ioc_add()). So I set the acpi_debug trace there.

It is probably the:
ACPI_MEM_FREE(dev_info);
in arch/ia64/hp/common/sba_iommu.c:

Output and changes made are attached.
Any ideas?

Thanks,

Thomas

#2 lore raw
Quoted 5 lines · Thomas Renninger

Something is strange here.
With rc2-gitXY I got this:

-----------------------------------------------------------------------
slab error in cache_free_debugcheck(): cache `size-256': double free, or memory outside object was overwritten

This is unrelated to Tony's original "unaligned access" problem, right?

If I understand correctly, you have CONFIG_DEBUG_SLAB turned on, and it reported a double free in the pci_acpi_scan_root() path in the rc2 kernel. Then you tried rc3 and saw a similar slab problem in the acpi_sba_ioc_add() path.

I looked at both of those paths, and I don't see anything wrong. I tried unsuccessfully to reproduce it with an rc1-mm4 kernel on an rx2600. I also tried an rc3 kernel (hg pull from 10 minutes ago) and couldn't reproduce it.

You have ACPI debug stuff turned on, don't you? I didn't enable that. Can you reproduce the slab problem with that turned off?

#3 lore raw
Quoted 3 lines · Bjorn Helgaas

I looked at both of those paths, and I don't see anything wrong. I tried unsuccessfully to reproduce it with an rc1-mm4 kernel on an rx2600.

Please try rx7620 or rx8640. Apparently this only happens on the NUMA systems.

Andreas.

Signature · 8 lines
Andreas Schwab, SuSE Labs, [email protected]
SuSE Linux Products GmbH, Maxfeldstraße 5, 90409 Nürnberg, Germany
PGP key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."
-
To unsubscribe from this list: send the line "unsubscribe linux-ia64" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
#4 lore raw
Quoted 7 lines · Andreas Schwab
Quoted 3 lines · Bjorn Helgaas

I looked at both of those paths, and I don't see anything wrong. I tried unsuccessfully to reproduce it with an rc1-mm4 kernel on an rx2600.

Please try rx7620 or rx8640. Apparently this only happens on the NUMA systems.

While I was reading your mail, I was booting an rx7620 for something else. When I looked at it again, it was spewing the slab errors :-)

I don't have time to debug it completely today, but it looks like the conversion of an AML vendor resource is overrunning the buffer. p1 and p2 are pointers to the redzones before and after the object. 0x170fc2a5 is the correct value:

acpi_rs_create_resource_list: p1 0xe0000007fc1518f0 p2 0xe0000007fc1519f8 acpi_rs_create_resource_list: *p1 0x170fc2a5 *p2 0x170fc2a5 acpi_rs_convert_aml_to_resources: p1 0xe0000007fc1518f0 p2 0xe0000007fc1519f8 acpi_rs_convert_aml_to_resources: *p1 0x170fc2a5 *p2 0x170fc2a5 acpi_rs_convert_aml_to_resources: before converting AML resource 0x88: *p1 0x170fc2a5 *p2 0x170fc2a5 acpi_rs_convert_aml_to_resources: before converting AML resource 0x8a: *p1 0x170fc2a5 *p2 0x170fc2a5 acpi_rs_convert_aml_to_resources: before converting AML resource 0x8a: *p1 0x170fc2a5 *p2 0x170fc2a5 acpi_rs_convert_aml_to_resources: before converting AML resource 0x84: *p1 0x170fc2a5 *p2 0x170fc2a5 acpi_rs_convert_aml_to_resources: before converting AML resource 0x84: *p1 0x170fc2a5 *p2 0x170fc2a5 acpi_rs_convert_aml_to_resources: before converting AML resource 0x79: *p1 0x170fc2a5 *p2 0xf901001a00000028

AML resource type 0x84 is a vendor-defined descriptor. This makes sense because rx7620 and rx8620 have large vendor-defined descriptors, while the smaller boxes like rx2600 do not.

#5 lore raw

PATCH 1/1 ACPI: fix vendor resource length computation

acpi_rs_get_list_length() needs to account for all the vendor-defined data bytes. Failing to include these causes buffers to be sized too small, which causes slab corruption when we later convert AML to resources and run off the end of the buffer.

I'm no expert on this code, so please scrutinize this carefully.

This causes slab corruption on machines that use ACPI vendor-defined resources. All HP ia64 machines do, and I'm told that some NEC machines may as well. So if the fix is correct, it would be good to have it in 2.6.16.

drivers/acpi/resources/rscalc.c +2 -4
Index: work-mm4/drivers/acpi/resources/rscalc.c
===================================================================
@@ -391,8 +391,7 @@
 			 * Ensure a 32-bit boundary for the structure
 			 */
 			extra_struct_bytes =
-			    ACPI_ROUND_UP_to_32_bITS(resource_length) -
-			    resource_length;
+			    ACPI_ROUND_UP_to_32_bITS(resource_length);
 			break;
 
 		case ACPI_RESOURCE_NAME_END_TAG:
@@ -408,8 +407,7 @@
 			 * Add vendor data and ensure a 32-bit boundary for the structure
 			 */
 			extra_struct_bytes =
-			    ACPI_ROUND_UP_to_32_bITS(resource_length) -
-			    resource_length;
+			    ACPI_ROUND_UP_to_32_bITS(resource_length);
 			break;
 
 		case ACPI_RESOURCE_NAME_ADDRESS32:
#6 lore raw

Re: [PATCH] ACPI: fix vendor resource length computation

Quoted 5 lines · Bjorn Helgaas

acpi_rs_get_list_length() needs to account for all the vendor-defined data bytes. Failing to include these causes buffers to be sized too small, which causes slab corruption when we later convert AML to resources and run off the end of the buffer.

Quoted 8 lines

I'm no expert on this code, so please scrutinize this carefully.

This causes slab corruption on machines that use ACPI vendor-defined resources. All HP ia64 machines do, and I'm told that some NEC machines may as well. So if the fix is correct, it would be good to have it in 2.6.16.

I forgot to mention that this patch may be used under either the GPL or the BSD license used for the ACPI CA.

Quoted 29 lines

Index: work-mm4/drivers/acpi/resources/rscalc.c
===================================================================

drivers/acpi/resources/rscalc.c +2 -5
@@ -391,8 +391,7 @@
 			 * Ensure a 32-bit boundary for the structure
 			 */
 			extra_struct_bytes =
-			    ACPI_ROUND_UP_to_32_bITS(resource_length) -
-			    resource_length;
+			    ACPI_ROUND_UP_to_32_bITS(resource_length);
 			break;
 
 		case ACPI_RESOURCE_NAME_END_TAG:
@@ -408,8 +407,7 @@
 			 * Add vendor data and ensure a 32-bit boundary for the structure
 			 */
 			extra_struct_bytes =
-			    ACPI_ROUND_UP_to_32_bITS(resource_length) -
-			    resource_length;
+			    ACPI_ROUND_UP_to_32_bITS(resource_length);
 			break;
 
 		case ACPI_RESOURCE_NAME_ADDRESS32:
-

To unsubscribe from this list: send the line "unsubscribe linux-acpi" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html