ACPICA: avoid double free when object already has a zero reference count
Subsystems (2)
PATCH 1/1 ACPICA: avoid double free when object already has a zero reference count
The first trip into acpi_ut_update_ref_count() for an object where 'object->common.reference_count' is 1 and we are performing a REF_DECREMENT will result in 'new_count' being 0 and thus the object is deleted via acpi_ut_delete_internal_obj().
If for some reason we make a subsequent trip into acpi_ut_update_ref_count() with the same object, object->common.reference_count' will be 0 and performing a REF_DECREMENT will produce a warning msg "Reference Count is already zero, cannot decrement", 'new_count' will again be 0 and the already deleted object will be attempted to be deleted again via acpi_ut_delete_internal_obj().
Since the object deletion doesn't NULL the object the calls to acpi_ut_delete_internal_obj(), acpi_ut_delete_object_desc(), acpi_os_release_object(), kmem_cache_free() will operate on the object as if it hasn't been deleted. In many cases this can result in no issues, but if you are using the slab and a new object has been created with the same address this can be the cause slab corruption.
Adding a check if we are decrementing to 0 for the first time and only calling acpi_ut_delete_internal_obj() in this case will prevent another attempt at deleting the object.
--- drivers/acpi/acpica/utdelete.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
drivers/acpi/acpica/utdelete.c +2 -2
@@ -421,9 +421,9 @@ acpi_ut_update_ref_count(union acpi_operand_object *object, u32 action)ACPI_GET_FUNCTION_NAME , object, object->common.type, new_count)); -/* Actually delete the object on a reference count of zero */ +/* If we haven't already, actually delete the object on a reference count of zero */ -if (new_count ==0 ) { +if (new_count ==0 && original_count !=0 ) {acpi_ut_delete_internal_obj (object); } message ="Decrement" ;
Signature · 1 line
2.17.1
Rafael J. Wysocki
maintainer+Erik and Bob
Quoted 49 lines · Mark Asselstine
The first trip into
acpi_ut_update_ref_count()for an object where 'object->common.reference_count' is 1 and we are performing aREF_DECREMENTwill result in 'new_count' being 0 and thus the object is deleted viaacpi_ut_delete_internal_obj().If for some reason we make a subsequent trip into
acpi_ut_update_ref_count()with the same object,object->common.reference_count' will be 0 and performing aREF_DECREMENTwill produce a warning msg "Reference Count is already zero, cannot decrement", 'new_count' will again be 0 and the already deleted object will be attempted to be deleted again viaacpi_ut_delete_internal_obj().Since the object deletion doesn't NULL the object the calls to
acpi_ut_delete_internal_obj(),acpi_ut_delete_object_desc(),acpi_os_release_object(),kmem_cache_free()will operate on the object as if it hasn't been deleted. In many cases this can result in no issues, but if you are using the slab and a new object has been created with the same address this can be the cause slab corruption.Adding a check if we are decrementing to 0 for the first time and only calling
acpi_ut_delete_internal_obj()in this case will prevent another attempt at deleting the object.Signed-off-by Mark Asselstine <[email protected]>--- drivers/acpi/acpica/utdelete.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)drivers/acpi/acpica/utdelete.c +2
-3@@ -421,9 +421,9 @@ acpi_ut_update_ref_count(union acpi_operand_object *object, u32 action)ACPI_GET_FUNCTION_NAME , object, object->common.type, new_count)); -/* Actually delete the object on a reference count of zero */ +/* If we haven't already, actually delete the object on a reference count of zero */ -if (new_count ==0 ) { +if (new_count ==0 && original_count !=0 ) {acpi_ut_delete_internal_obj (object); } message ="Decrement" ; --2.17.1
Quoted 11 lines
-----Original Message-----
From: Rafael J. Wysocki <[email protected]>
Sent: Thursday, October 29, 2020 7:06 AM
To: Mark Asselstine <[email protected]>
Cc: ACPI Devel Maling List <[email protected]>; Kaneda, Erik <[email protected]>; Moore, Robert <[email protected]>
Subject: Re: [PATCH] ACPICA: avoid double free when object already has a zero reference count
Quoted 17 lines
+Erik and Bob
Quoted 13 lines · Mark Asselstine
The first trip into
acpi_ut_update_ref_count()for an object where 'object->common.reference_count' is 1 and we are performing aREF_DECREMENTwill result in 'new_count' being 0 and thus the object is deleted viaacpi_ut_delete_internal_obj().If for some reason we make a subsequent trip into
acpi_ut_update_ref_count()with the same object,object->common.reference_count' will be 0 and performing aREF_DECREMENTwill produce a warning msg "Reference Count is already zero, cannot decrement", 'new_count' will again be 0 and the already deleted object will be attempted to be deleted again viaacpi_ut_delete_internal_obj().
Mark, Do you have an example of AML/ASL that you used to determine this double free?
Thanks,
Erik
Quoted 38 lines
drivers/acpi/acpica/utdelete.c acpi_ut_update_ref_count(union
Quoted 22 lines
Since the object deletion doesn't NULL the object the calls to
acpi_ut_delete_internal_obj(),acpi_ut_delete_object_desc(),acpi_os_release_object(),kmem_cache_free()will operate on the object as if it hasn't been deleted. In many cases this can result in no issues, but if you are using the slab and a new object has been created with the same address this can be the cause slab corruption.Adding a check if we are decrementing to 0 for the first time and only calling
acpi_ut_delete_internal_obj()in this case will prevent another attempt at deleting the object.Signed-off-by Mark Asselstine <[email protected]>--- drivers/acpi/acpica/utdelete.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)drivers/acpi/acpica/utdelete.c +0
-0@@ -421,9 +421,9 @@ acpi_ut_update_ref_count(union
acpi_operand_object*object, u32 action)drivers/acpi/acpica/utdelete.c acpi_ut_update_ref_count(union acpi_operand_object *object, u32 action)
Quoted 5 lines
drivers/acpi/acpica/utdelete.c +1
-1ACPI_GET_FUNCTION_NAME , object, object->common.type, new_count)); -/* Actually delete the object on a reference count of zero */ +/* If we haven't already, actually delete the object on a reference count of zero */
Quoted 9 lines
drivers/acpi/acpica/utdelete.c +1
-2-if (new_count ==0 ) { +if (new_count ==0 && original_count !=0 ) {acpi_ut_delete_internal_obj (object); } message ="Decrement" ; --2.17.1
Quoted 38 lines · Erik
Quoted 13 lines
-----Original Message-----
From: Rafael J. Wysocki <[email protected]>
Sent: Thursday, October 29, 2020 7:06 AM
To: Mark Asselstine <[email protected]>
Cc: ACPI Devel Maling List <[email protected]>; Kaneda, Erik <[email protected]>; Moore, Robert <[email protected]>
Subject: Re: [PATCH] ACPICA: avoid double free when object already has a zero reference countQuoted 23 lines
+Erik and Bob
Quoted 16 lines · Mark Asselstine
The first trip into
acpi_ut_update_ref_count()for an object where 'object->common.reference_count' is 1 and we are performing aREF_DECREMENTwill result in 'new_count' being 0 and thus the object is deleted viaacpi_ut_delete_internal_obj().If for some reason we make a subsequent trip into
acpi_ut_update_ref_count()with the same object,object->common.reference_count' will be 0 and performing aREF_DECREMENTwill produce a warning msg "Reference Count is already zero, cannot decrement", 'new_count' will again be 0 and the already deleted object will be attempted to be deleted again viaacpi_ut_delete_internal_obj().
Quoted 2 lines
Mark, Do you have an example of AML/ASL that you used to determine this double free?
Unfortunately no. It is a rare occurance and a consequence of several actions taking place at the same time during boot, including a PCI rescan. Unfortunately due to circumstances I am sure you would rather not have to be concerned about, we have so far had to focus our efforts on an older kernel which also has the preempt-rt patchset applied. It is for this reason I also didn't include the dmesg and eventual kernel BUG_ON in my submission.
It is unclear at this time if the additional locking or other changes that have been merged since the kernel version we are on would prevent this from occuring on an up to date kernel.
I have reviewed the code in the latest linux kernel and as far as I can tell the deficiency is still present. If you do go into acpi_ut_update_ref_count() with an object with a reference count of 0 and an action of REF_DECREMENT the following may be called:
acpi_ut_update_ref_count
-> acpi_ut_delete_internal_obj
-> acpi_ut_delete_object_desc
-> acpi_os_release_object
-> kmem_cache_freeI completely understand if you have concerns about the change since I can't hand you a reproducer. I was hoping the merrits of the change would stand on their own as there is no reason to call acpi_ut_delete_internal_obj() if we have already done so, even if the rest of the call chain was well behaved. In our case the eventual slab corruption ended up affecting the 'Acpi-Operand' dedicated cache. We see the corruption happen shortly after we see the msg warning that the reference count is already at zero.
Please let me know if there is anything else I can provide to help out and thanks for your time reviewing this change.
Thanks,
Mark
Quoted 24 lines
Thanks,
ErikQuoted 20 lines
Quoted 20 lines
Since the object deletion doesn't NULL the object the calls to
acpi_ut_delete_internal_obj(),acpi_ut_delete_object_desc(),acpi_os_release_object(),kmem_cache_free()will operate on the object as if it hasn't been deleted. In many cases this can result in no issues, but if you are using the slab and a new object has been created with the same address this can be the cause slab corruption.Adding a check if we are decrementing to 0 for the first time and only calling
acpi_ut_delete_internal_obj()in this case will prevent another attempt at deleting the object.Signed-off-by Mark Asselstine <[email protected]>---
Quoted 7 lines
Quoted 7 lines
drivers/acpi/acpica/utdelete.c
Quoted 7 lines
drivers/acpi/acpica/utdelete.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)drivers/acpi/acpica/utdelete.c +0
-0
index 4c0d4e434196..c6b860fd9eb5
Quoted 7 lines
Quoted 7 lines
drivers/acpi/acpica/utdelete.c acpi_ut_update_ref_count(union
Quoted 4 lines
100644
drivers/acpi/acpica/utdelete.c +0
-0@@ -421,9 +421,9 @@ acpi_ut_update_ref_count(union
acpi_operand_object*object, u32 action)
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
drivers/acpi/acpica/utdelete.c +0
-1ACPI_GET_FUNCTION_NAME , object, object->common.type, new_count)); -/* Actually delete the object on a reference count of zero */
+ /* If we haven't already, actually delete the
Quoted 17 lines
Quoted 17 lines
Quoted 1 line
object on a reference>
count of zero */
Quoted 14 lines
drivers/acpi/acpica/utdelete.c +1
-2-if (new_count ==0 ) { +if (new_count ==0 && original_count !=0 ) {acpi_ut_delete_internal_obj (object); } message ="Decrement" ; --2.17.1
Quoted 91 lines
-----Original Message-----
From: Mark Asselstine <[email protected]>
Sent: Monday, November 2, 2020 10:51 AM
To: Rafael J. Wysocki <[email protected]>; Kaneda, Erik <[email protected]>
Cc: ACPI Devel Maling List <[email protected]>; Moore, Robert <[email protected]>
Subject: Re: [PATCH] ACPICA: avoid double free when object already has a zero reference countQuoted 9 lines · Erik
Quoted 9 lines
-----Original Message-----
From: Rafael J. Wysocki <[email protected]>
Sent: Thursday, October 29, 2020 7:06 AM
To: Mark Asselstine <[email protected]>
Cc: ACPI Devel Maling List <[email protected]>; Kaneda, Erik <[email protected]>; Moore, Robert <[email protected]>
Subject: Re: [PATCH] ACPICA: avoid double free when object already hasa
Quoted 20 lines
Quoted 20 lines
zero reference count
+Erik and Bob
Quoted 9 lines · Mark Asselstine
The first trip into
acpi_ut_update_ref_count()for an object where 'object->common.reference_count' is 1 and we are performing aREF_DECREMENTwill result in 'new_count' being 0 and thus the object is deleted viaacpi_ut_delete_internal_obj().Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
If for some reason we make a subsequent trip into
acpi_ut_update_ref_count()with the same object,object->common.reference_count' will be 0 and performing aREF_DECREMENTwill produce a warning msg "Reference Count isalready
Quoted 6 lines
Quoted 3 lines
Quoted 3 lines
zero, cannot decrement", '
new_count' will again be 0 and the already deleted object will be attempted to be deleted again viaacpi_ut_delete_internal_obj().Mark, Do you have an example of AML/ASL that you used to determine
this
Quoted 1 line
double free?
Unfortunately no. It is a rare occurance and a consequence of several actions taking place at the same time during boot, including a PCI rescan. Unfortunately due to circumstances I am sure you would rather not have to be
concerned about, we have so far had to focus our efforts on an older kernel which also has the preempt-rt patchset applied. It is for this reason I also didn't include the dmesg and eventual kernelBUG_ONin my submission.It is unclear at this time if the additional locking or other changes that have been merged since the kernel version we are on would prevent this from
occuring on an up to date kernel.I have reviewed the code in the latest linux kernel and as far as I can tell the deficiency is still present. If you do go into
acpi_ut_update_ref_count()with an object with a reference count of 0 and an action ofREF_DECREMENTthe
following may be called:
acpi_ut_update_ref_count
->acpi_ut_delete_internal_obj-> acpi_ut_delete_object_desc -> acpi_os_release_object -> kmem_cache_freeI completely understand if you have concerns about the change since I can't hand you a reproducer. I was hoping the merrits of the change would stand on
their own as there is no reason to callacpi_ut_delete_internal_obj()if we have already done so, even if the rest of the call chain was well behaved. In our case the eventual slab corruption ended up affecting the 'Acpi-Operand' dedicated cache. We see the corruption happen shortly after we see the msg warning that the reference count is already at zero.
Quoted 2 lines
Please let me know if there is anything else I can provide to help out and thanks for your time reviewing this change.
Since this is such a sensitive part of the codebase, I would like to see if it's possible to come up with ASL that reproduces this on our userspace interpreter (acpiexec).
I can try reproduce this from your explanations but it would be helpful to also get an acpidump and dmesg so that we can understand which named objects are causing this issue. This gives us a clue on how to reproduce this in user space.
Could you provide an acpidump of the machine and 2 dmesg logs? I need one normal dmesg that shows this error and a custom kernel built with CONFIG_ACPI_DEBUG=y and booting with the following commandline parameters:
acpi.debug_level=0x80000 acpi.debug_layer=0xffffffff
Thanks,
Erik
Quoted 73 lines
Thanks,
MarkQuoted 9 lines
Thanks,
ErikQuoted 5 lines
Quoted 5 lines
Since the object deletion doesn't NULL the object the calls to
acpi_ut_delete_internal_obj(),acpi_ut_delete_object_desc(),acpi_os_release_object(),kmem_cache_free()will operate on theobject
Quoted 15 lines
Quoted 15 lines
Quoted 15 lines
as if it hasn't been deleted. In many cases this can result in no issues, but if you are using the slab and a new object has been created with the same address this can be the cause slab corruption.
Adding a check if we are decrementing to 0 for the first time and only calling
acpi_ut_delete_internal_obj()in this case will prevent another attempt at deleting the object.Signed-off-by Mark Asselstine <[email protected]>---
Quoted 7 lines
Quoted 7 lines
drivers/acpi/acpica/utdelete.c
Quoted 7 lines
drivers/acpi/acpica/utdelete.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)drivers/acpi/acpica/utdelete.c +0
-0index
4c0d4e434196..c6b860fd9eb5Quoted 7 lines
Quoted 7 lines
drivers/acpi/acpica/utdelete.c acpi_ut_update_ref_count(union
Quoted 4 lines
100644
drivers/acpi/acpica/utdelete.c +0
-0@@ -421,9 +421,9 @@ acpi_ut_update_ref_count(union
acpi_operand_object*object, u32 action)Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
drivers/acpi/acpica/utdelete.c +0
-1ACPI_GET_FUNCTION_NAME , object, object->common.type, new_count)); -/* Actually delete the object on a reference count of zero */
+ /* If we haven't already, actually delete the
Quoted 17 lines
Quoted 17 lines
Quoted 1 line
object on a reference>
count of zero */
Quoted 14 lines
drivers/acpi/acpica/utdelete.c +1
-2-if (new_count ==0 ) { +if (new_count ==0 && original_count !=0 ) {acpi_ut_delete_internal_obj (object); } message ="Decrement" ; --2.17.1
Quoted 111 lines · Erik
Quoted 99 lines
-----Original Message-----
From: Mark Asselstine <[email protected]>
Sent: Monday, November 2, 2020 10:51 AM
To: Rafael J. Wysocki <[email protected]>; Kaneda, Erik <[email protected]>
Cc: ACPI Devel Maling List <[email protected]>; Moore, Robert <[email protected]>
Subject: Re: [PATCH] ACPICA: avoid double free when object already has a zero reference countQuoted 7 lines · Erik
Quoted 7 lines
-----Original Message-----
From: Rafael J. Wysocki <[email protected]>
Sent: Thursday, October 29, 2020 7:06 AM
To: Mark Asselstine <[email protected]>
Cc: ACPI Devel Maling List <[email protected]>; Kaneda, Erik <[email protected]>; Moore, Robert <[email protected]>
Subject: Re: [PATCH] ACPICA: avoid double free when object already hasa
Quoted 18 lines
Quoted 18 lines
zero reference count
+Erik and Bob
On Thu, Oct 29, 2020 at 3:05 AM Mark Asselstine
Quoted 7 lines · <[email protected]> wrote:
The first trip into
acpi_ut_update_ref_count()for an object where 'object->common.reference_count' is 1 and we are performing aREF_DECREMENTwill result in 'new_count' being 0 and thus the object is deleted viaacpi_ut_delete_internal_obj().Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
If for some reason we make a subsequent trip into
acpi_ut_update_ref_count()with the same object,object->common.reference_count' will be 0 and performing aREF_DECREMENTwill produce a warning msg "Reference Count isalready
Quoted 5 lines
Quoted 3 lines
Quoted 3 lines
zero, cannot decrement", '
new_count' will again be 0 and the already deleted object will be attempted to be deleted again viaacpi_ut_delete_internal_obj().Mark, Do you have an example of AML/ASL that you used to determine
this
Quoted 1 line
double free?
Unfortunately no. It is a rare occurance and a consequence of several actions taking place at the same time during boot, including a PCI rescan. Unfortunately due to circumstances I am sure you would rather not have to be
concerned about, we have so far had to focus our efforts on an older kernel
which also has the preempt-rt patchset applied. It is for this reason I also didn't include the dmesg and eventual kernelBUG_ONin my submission.It is unclear at this time if the additional locking or other changes that have been merged since the kernel version we are on would prevent this from
occuring on an up to date kernel.I have reviewed the code in the latest linux kernel and as far as I can tell the deficiency is still present. If you do go into
acpi_ut_update_ref_count()with an object with a reference count of 0 and an action ofREF_DECREMENTthe
following may be called:
acpi_ut_update_ref_count->
acpi_ut_delete_internal_obj-> acpi_ut_delete_object_desc -> acpi_os_release_object -> kmem_cache_freeI completely understand if you have concerns about the change since I can't
hand you a reproducer. I was hoping the merrits of the change would stand on
their own as there is no reason to callacpi_ut_delete_internal_obj()if we
have already done so, even if the rest of the call chain was well behaved. In our case the eventual slab corruption ended up affecting the 'Acpi-Operand' dedicated cache. We see the corruption happen shortly after we see the msg warning that the reference count is already at zero.Quoted 2 lines
Please let me know if there is anything else I can provide to help out and thanks for your time reviewing this change.
Since this is such a sensitive part of the codebase, I would like to see if it's possible to come up with ASL that reproduces this on our userspace interpreter (acpiexec).
I can try reproduce this from your explanations but it would be helpful to also get an acpidump and dmesg so that we can understand which named objects are causing this issue. This gives us a clue on how to reproduce this in user space.
Quoted 5 lines
Could you provide an acpidump of the machine and 2 dmesg logs? I need one normal dmesg that shows this error and a custom kernel built with
CONFIG_ACPI_DEBUG=yand booting with the following commandline parameters:acpi.debug_level=0x80000 acpi.debug_layer=0xffffffff
This should be possible. Give me a day or two to put something together. Again my intention, as when dealing with any maintainer, is to be sensitive of your time and given the kernel version skew and such I am not sure how well it will map to the latest kernel code. But if you can pull out some clues to help with this mapping I am not going to impede your offer of assistance.
Thanks,
MarkA
Quoted 72 lines
Thanks,
ErikQuoted 68 lines
Thanks,
MarkQuoted 6 lines
Thanks,
ErikQuoted 3 lines
Quoted 3 lines
Since the object deletion doesn't NULL the object the calls to
acpi_ut_delete_internal_obj(),acpi_ut_delete_object_desc(),acpi_os_release_object(),kmem_cache_free()will operate on theobject
Quoted 13 lines
Quoted 13 lines
Quoted 13 lines
as if it hasn't been deleted. In many cases this can result in no issues, but if you are using the slab and a new object has been created with the same address this can be the cause slab corruption.
Adding a check if we are decrementing to 0 for the first time and only
callingacpi_ut_delete_internal_obj()in this case will prevent another attempt at deleting the object.Quoted 8 lines
Quoted 8 lines
drivers/acpi/acpica/utdelete.c
Quoted 8 lines
Signed-off-by Mark Asselstine <[email protected]>---
drivers/acpi/acpica/utdelete.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)drivers/acpi/acpica/utdelete.c +0
-0index
4c0d4e434196..c6b860fd9eb5Quoted 7 lines
Quoted 7 lines
drivers/acpi/acpica/utdelete.c acpi_ut_update_ref_count(union
Quoted 4 lines
100644
drivers/acpi/acpica/utdelete.c +0
-0@@ -421,9 +421,9 @@ acpi_ut_update_ref_count(union
acpi_operand_object*object, u32 action)Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
drivers/acpi/acpica/utdelete.c +0
-1ACPI_GET_FUNCTION_NAME , object, object->common.type, new_count)); -/* Actually delete the object on a reference count of
zero */+ /* If we haven't already, actually delete the
Quoted 14 lines
Quoted 14 lines
Quoted 1 line
object on a reference>
count of zero */
Quoted 10 lines
drivers/acpi/acpica/utdelete.c +1
-2-if (new_count ==0 ) { +if (new_count ==0 && original_count !=0 ) {acpi_ut_delete_internal_obj (object); } message ="Decrement" ; --2.17.1
Quoted 127 lines · On Tuesday, November 3, 2020 1:52:28 P.M. EST Mark Asselstine wrote:
Quoted 126 lines · Erik
Quoted 108 lines
-----Original Message-----
From: Mark Asselstine <[email protected]>
Sent: Monday, November 2, 2020 10:51 AM
To: Rafael J. Wysocki <[email protected]>; Kaneda, Erik <[email protected]>
Cc: ACPI Devel Maling List <[email protected]>; Moore, Robert <[email protected]>
Subject: Re: [PATCH] ACPICA: avoid double free when object already has a zero reference countQuoted 9 lines · Erik
Quoted 9 lines
-----Original Message-----
From: Rafael J. Wysocki <[email protected]>
Sent: Thursday, October 29, 2020 7:06 AM
To: Mark Asselstine <[email protected]>
Cc: ACPI Devel Maling List <[email protected]>; Kaneda, Erik
<[email protected]>; Moore, Robert <[email protected]>
Subject: Re: [PATCH] ACPICA: avoid double free when object already hasa
Quoted 19 lines
Quoted 19 lines
zero reference count
+Erik and Bob
On Thu, Oct 29, 2020 at 3:05 AM Mark Asselstine
Quoted 8 lines · <[email protected]> wrote:
The first trip into
acpi_ut_update_ref_count()for an object where 'object->common.reference_count' is 1 and we are performing aREF_DECREMENTwill result in 'new_count' being 0 and thus the object
is deleted viaacpi_ut_delete_internal_obj().Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
If for some reason we make a subsequent trip into
acpi_ut_update_ref_count()with the same object,object->common.reference_count' will be 0 and performing aREF_DECREMENTwill produce a warning msg "Reference Count isalready
Quoted 6 lines
Quoted 4 lines
Quoted 4 lines
zero, cannot decrement", '
new_count' will again be 0 and the already
deleted object will be attempted to be deleted again viaacpi_ut_delete_internal_obj().Mark, Do you have an example of AML/ASL that you used to determine
this
Quoted 1 line
double free?
Unfortunately no. It is a rare occurance and a consequence of several actions taking place at the same time during boot, including a PCI rescan. Unfortunately due to circumstances I am sure you would rather not
have to be
concerned about, we have so far had to focus our efforts on an older kernel
which also has the preempt-rt patchset applied. It is for this reason I also didn't include the dmesg and eventual kernelBUG_ONin my submission.It is unclear at this time if the additional locking or other changes that
have been merged since the kernel version we are on would prevent this from
occuring on an up to date kernel.I have reviewed the code in the latest linux kernel and as far as I can tell the deficiency is still present. If you do go into
acpi_ut_update_ref_count()with an object with a reference count of 0 and
an action ofREF_DECREMENTthe
following may be called:
acpi_ut_update_ref_count->
acpi_ut_delete_internal_obj-> acpi_ut_delete_object_desc -> acpi_os_release_object -> kmem_cache_freeI completely understand if you have concerns about the change since I can't
hand you a reproducer. I was hoping the merrits of the change would stand
on
their own as there is no reason to callacpi_ut_delete_internal_obj()if we
have already done so, even if the rest of the call chain was well behaved.
In our case the eventual slab corruption ended up affecting the 'Acpi-Operand' dedicated cache. We see the corruption happen shortly after we see the msg warning that the reference count is already at zero.Quoted 3 lines
Please let me know if there is anything else I can provide to help out and
thanks for your time reviewing this change.Since this is such a sensitive part of the codebase, I would like to see if
it's possible to come up with ASL that reproduces this on our userspace interpreter (acpiexec).I can try reproduce this from your explanations but it would be helpful to also get an acpidump and dmesg so that we can understand which named objects are causing this issue. This gives us a clue on how to reproduce this in user space.
Could you provide an acpidump of the machine and 2 dmesg logs? I need one normal dmesg that shows this error and a custom kernel built with
CONFIG_ACPI_DEBUG=yand booting with the following commandline parameters:
Quoted 8 lines
Quoted 1 line
acpi.debug_level=0x80000 acpi.debug_layer=0xffffffff
This should be possible. Give me a day or two to put something together. Again my intention, as when dealing with any maintainer, is to be sensitive of your time and given the kernel version skew and such I am not sure how well it will map to the latest kernel code. But if you can pull out some clues to help with this mapping I am not going to impede your offer of assistance.
I have supplied what I could directly to the Intel folks. Unfortunately, as I had stated previously our reproduction case uses an older kernel so can't easily map to the current state of code in mainline.
I go back to my original assertion that the same condition which is being caught by the warning should be used to prevent a second attempt at deleting an already deleted object. Attempting a second deletion has been observed to be troublesome as the underlying memory could now be in used by a valid object and slab corruption is highly likely.
I have also supplied a github pull request to the acpica repo as this issue could also manifest itself negatively in other OSes.
https://github.com/acpica/acpica/pull/652
Mark
Quoted 76 lines
Thanks,
MarkAQuoted 72 lines
Thanks,
ErikQuoted 69 lines
Thanks,
MarkQuoted 6 lines
Thanks,
ErikQuoted 3 lines
Quoted 3 lines
Since the object deletion doesn't NULL the object the calls to
acpi_ut_delete_internal_obj(),acpi_ut_delete_object_desc(),acpi_os_release_object(),kmem_cache_free()will operate on theobject
Quoted 14 lines
Quoted 14 lines
Quoted 14 lines
as if it hasn't been deleted. In many cases this can result in no issues, but if you are using the slab and a new object has been created with the same address this can be the cause slab corruption.
Adding a check if we are decrementing to 0 for the first time and only
callingacpi_ut_delete_internal_obj()in this case will prevent another attempt at deleting the object.Quoted 8 lines
Quoted 8 lines
drivers/acpi/acpica/utdelete.c
Quoted 8 lines
Signed-off-by Mark Asselstine <[email protected]>---
drivers/acpi/acpica/utdelete.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)drivers/acpi/acpica/utdelete.c +0
-0index
4c0d4e434196..c6b860fd9eb5Quoted 7 lines
Quoted 7 lines
drivers/acpi/acpica/utdelete.c acpi_ut_update_ref_count(union
Quoted 4 lines
100644
drivers/acpi/acpica/utdelete.c +0
-0@@ -421,9 +421,9 @@ acpi_ut_update_ref_count(union
acpi_operand_object*object, u32 action)Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
drivers/acpi/acpica/utdelete.c +0
-1ACPI_GET_FUNCTION_NAME , object, object->common.type, new_count)); -/* Actually delete the object on a reference count of
zero */+ /* If we haven't already, actually delete the
Quoted 14 lines
Quoted 14 lines
Quoted 1 line
object on a reference>
count of zero */
Quoted 10 lines
drivers/acpi/acpica/utdelete.c +1
-2-if (new_count ==0 ) { +if (new_count ==0 && original_count !=0 ) {acpi_ut_delete_internal_obj (object); } message ="Decrement" ; --2.17.1