PCI/ASPM: fix UAF by disable ASPM for link when child function is removed
PATCH 1/1 PCI/ASPM: fix UAF by disable ASPM for link when child function is removed
If the Function 0 of a Multi-Function device is software removed, a freed downstream pointer will be left in struct pcie_link_state, and then when pcie_config_aspm_link() be invoked from any path, we will trigger use-after-free.
Based on the PCIe spec about ASPM Control (PCIe r6.0, sec 7.5.3.7), for Multi-Function Devices (including ARI Devices), it is recommended that software program the same value in all Functions. For ARI Devices, ASPM Control is determined solely by the setting in Function 0.
So we can just disable ASPM of the whole component if any child function is removed, the downstream pointer will be avoided from use-after-free, that will also avoid other potential corner cases.
--- drivers/pci/pcie/aspm.c | 15 +++++++-------- 1 file changed, 7 insertions(+), 8 deletions(-)
drivers/pci/pcie/aspm.c +7 -8
@@ -1010,18 +1010,17 @@ void pcie_aspm_exit_link_state(struct pci_dev *pdev)down_read (&pci_bus_sem);mutex_lock (&aspm_lock); -/* -* All PCIe functions are in one slot, remove one function will remove -* the whole slot, so just wait until we are the last function left. -*/ -if (!list_empty (&parent->subordinate->devices)) - goto out; link = parent->link_state; root = link->root; parent_link = link->parent; -/* All functions are removed, so just disable ASPM for the link */ +/* +* Any function is removed (including software removing), just +* disable ASPM for the link, in case we can not configure the same +* setting for all functions. +* See PCIe r6.0, sec 7.5.3.7. +*/ pcie_config_aspm_link (link,0 );list_del (&link->sibling);/* Clock PM is for endpoint device */
@@ -1032,7 +1031,7 @@ void pcie_aspm_exit_link_state(struct pci_dev *pdev)pcie_update_aspm_capable (root);pcie_config_aspm_path (parent_link); } -out: +mutex_unlock (&aspm_lock);up_read (&pci_bus_sem); }
Signature · 1 line
2.17.1
Hi,
On 5/4/23 5:34 AM, Ding Hui wrote:
Maybe you can use the following title?
"PCI/ASPM: Fix UAF by disabling ASPM for link when child function is removed
Quoted 6 lines
If the Function 0 of a Multi-Function device is software removed, a freed downstream pointer will be left in struct
pcie_link_state, and then whenpcie_config_aspm_link()be invoked from any path, we will trigger use-after-free.Based on the PCIe spec about ASPM Control (PCIe r6.0, sec 7.5.3.7),
As per PCIe spec r6.0, sec 7.5.3.7, it is recommended
Quoted 10 lines
for Multi-Function Devices (including ARI Devices), it is recommended that software program the same value in all Functions. For ARI Devices, ASPM Control is determined solely by the setting in Function 0.
So we can just disable ASPM of the whole component if any child function is removed, the downstream pointer will be avoided from use-after-free, that will also avoid other potential corner cases.
Fixes b5a0a9b59c81 ("PCI/ASPM: Read and set up L1 substate capabilities")Debugged-by Zongquan Qin <[email protected]>
Any bugzilla link with error log and reproduction steps?
Quoted 1 line
Suggestion-by Bjorn Helgaas <[email protected]>
Suggested-by?
Quoted 19 lines
Signed-off-by Ding Hui <[email protected]>--- drivers/pci/pcie/aspm.c | 15 +++++++-------- 1 file changed, 7 insertions(+), 8 deletions(-)drivers/pci/pcie/aspm.c +0
-6@@ -1010,18 +1010,17 @@ void pcie_aspm_exit_link_state(struct pci_dev *pdev)down_read (&pci_bus_sem);mutex_lock (&aspm_lock); -/* -* All PCIe functions are in one slot, remove one function will remove -* the whole slot, so just wait until we are the last function left. -*/ -if (!list_empty (&parent->subordinate->devices)) - goto out;
drivers/pci/pcie/aspm.c void pcie_aspm_exit_link_state(struct pci_dev *pdev) · line 1021
Quoted 10 lines
drivers/pci/pcie/aspm.c +4
-1link = parent->link_state; root = link->root; parent_link = link->parent; -/* All functions are removed, so just disable ASPM for the link */ +/* +* Any function is removed (including software removing), just +* disable ASPM for the link, in case we can not configure the same +* setting for all functions.
How about following?
/*
* For any function removed, disable ASPM for the link. See PCIe r6.0,
* sec 7.7.3.7 for details.
*/
Quoted 14 lines
drivers/pci/pcie/aspm.c +3
-1+* See PCIe r6.0, sec 7.5.3.7. +*/ pcie_config_aspm_link (link,0 );list_del (&link->sibling);/* Clock PM is for endpoint device */ @@ -1032,7 +1031,7 @@ void pcie_aspm_exit_link_state(struct pci_dev *pdev)pcie_update_aspm_capable (root);pcie_config_aspm_path (parent_link); } -out: +mutex_unlock (&aspm_lock);up_read (&pci_bus_sem); }
Signature · 2 lines
Sathyanarayanan Kuppuswamy Linux Kernel Developer
Quoted 8 lines · Sathyanarayanan Kuppuswamy
Hi,
On 5/4/23 5:34 AM, Ding Hui wrote:
Maybe you can use the following title?
"PCI/ASPM: Fix UAF by disabling ASPM for link when child function is removed
Thanks, I'll fix it in next patch.
Quoted 9 lines
Quoted 6 lines
If the Function 0 of a Multi-Function device is software removed, a freed downstream pointer will be left in struct
pcie_link_state, and then whenpcie_config_aspm_link()be invoked from any path, we will trigger use-after-free.Based on the PCIe spec about ASPM Control (PCIe r6.0, sec 7.5.3.7),
As per PCIe spec r6.0, sec 7.5.3.7, it is recommended
Thanks, I'll fix it in next patch.
Quoted 4 lines
Quoted 4 lines
for Multi-Function Devices (including ARI Devices), it is recommended that software program the same value in all Functions. For ARI Devices, ASPM Control is determined solely by the setting in Function 0.
Quoted 8 lines
Quoted 6 lines
So we can just disable ASPM of the whole component if any child function is removed, the downstream pointer will be avoided from use-after-free, that will also avoid other potential corner cases.
Fixes b5a0a9b59c81 ("PCI/ASPM: Read and set up L1 substate capabilities")Debugged-by Zongquan Qin <[email protected]>Any bugzilla link with error log and reproduction steps?
Yes, I should link previous
https://lore.kernel.org/lkml/[email protected]/
Since Bjorn think the ALL details is not necessary, so I'll add the reproducer and compact result in next patch.
Quoted 4 lines
Quoted 2 lines
Suggestion-by Bjorn Helgaas <[email protected]>Suggested-by?
Sorry, I'll fix it in next patch.
Quoted 33 lines
Quoted 20 lines
Signed-off-by Ding Hui <[email protected]>--- drivers/pci/pcie/aspm.c | 15 +++++++-------- 1 file changed, 7 insertions(+), 8 deletions(-)drivers/pci/pcie/aspm.c +0
-6@@ -1010,18 +1010,17 @@ void pcie_aspm_exit_link_state(struct pci_dev *pdev)down_read (&pci_bus_sem);mutex_lock (&aspm_lock); -/* -* All PCIe functions are in one slot, remove one function will remove -* the whole slot, so just wait until we are the last function left. -*/ -if (!list_empty (&parent->subordinate->devices)) - goto out;drivers/pci/pcie/aspm.c void pcie_aspm_exit_link_state(struct pci_dev *pdev) · line 1021
Quoted 10 lines
drivers/pci/pcie/aspm.c +4
-1link = parent->link_state; root = link->root; parent_link = link->parent; -/* All functions are removed, so just disable ASPM for the link */ +/* +* Any function is removed (including software removing), just +* disable ASPM for the link, in case we can not configure the same +* setting for all functions. How about following?
Quoted 5 lines
/*
* For any function removed, disable ASPM for the link. See PCIe r6.0,
* sec 7.7.3.7 for details.
*/
Thanks, it's better.
Quoted 15 lines
Quoted 14 lines
drivers/pci/pcie/aspm.c +3
-1+* See PCIe r6.0, sec 7.5.3.7. +*/ pcie_config_aspm_link (link,0 );list_del (&link->sibling);/* Clock PM is for endpoint device */ @@ -1032,7 +1031,7 @@ void pcie_aspm_exit_link_state(struct pci_dev *pdev)pcie_update_aspm_capable (root);pcie_config_aspm_path (parent_link); } -out: +mutex_unlock (&aspm_lock);up_read (&pci_bus_sem); }
Signature · 2 lines
Thanks, - Ding Hui
drivers/pci/pcie/aspm.c void pcie_aspm_exit_link_state(struct pci_dev *pdev)
Quoted 27 lines · Ding Hui
If the Function 0 of a Multi-Function device is software removed, a freed downstream pointer will be left in struct
pcie_link_state, and then whenpcie_config_aspm_link()be invoked from any path, we will trigger use-after-free.Based on the PCIe spec about ASPM Control (PCIe r6.0, sec 7.5.3.7), for Multi-Function Devices (including ARI Devices), it is recommended that software program the same value in all Functions. For ARI Devices, ASPM Control is determined solely by the setting in Function 0.
So we can just disable ASPM of the whole component if any child function is removed, the downstream pointer will be avoided from use-after-free, that will also avoid other potential corner cases.
Fixes b5a0a9b59c81 ("PCI/ASPM: Read and set up L1 substate capabilities")Debugged-by Zongquan Qin <[email protected]>Suggestion-by Bjorn Helgaas <[email protected]>Signed-off-by Ding Hui <[email protected]>--- drivers/pci/pcie/aspm.c | 15 +++++++-------- 1 file changed, 7 insertions(+), 8 deletions(-)drivers/pci/pcie/aspm.c +0
-0@@ -1010,18 +1010,17 @@ void pcie_aspm_exit_link_state(struct pci_dev *pdev)
Not directly related to your patch, but this looks racy to me:
void pcie_aspm_exit_link_state(struct pci_dev *pdev)
{
struct pci_dev *parent = pdev->bus->self; if (!parent || !parent->link_state)
return;
down_read(&pci_bus_sem);
mutex_lock(&aspm_lock);
link = parent->link_state;
root = link->root;
...
free_link_state(link);
link->pdev->link_state = NULL;
kfree(link);Since we check "parent->link_state" before acquiring the locks, I suspect that removing two functions at the same time could end up with a NULL pointer dereference:
pcie_aspm_exit_link_state(fn 0) pcie_aspm_exit_link_state(fn 1) parent = X parent = X parent->link_state != NULL parent->link_state != NULL
acquire locks
free_link_state(link)
link->pdev->link_state = NULL # aka parent->link_state kfree(link)
release locks
acquire locks
link = parent->link_state # now NULL
root = link->root # NULL ptrWhat do you think? I guess if this *is* a race, it should be fixed by a separate patch.
Quoted 35 lines
drivers/pci/pcie/aspm.c +7
-8down_read (&pci_bus_sem);mutex_lock (&aspm_lock); -/* -* All PCIe functions are in one slot, remove one function will remove -* the whole slot, so just wait until we are the last function left. -*/ -if (!list_empty (&parent->subordinate->devices)) - goto out; link = parent->link_state; root = link->root; parent_link = link->parent; -/* All functions are removed, so just disable ASPM for the link */ +/* +* Any function is removed (including software removing), just +* disable ASPM for the link, in case we can not configure the same +* setting for all functions. +* See PCIe r6.0, sec 7.5.3.7. +*/ pcie_config_aspm_link (link,0 );list_del (&link->sibling);/* Clock PM is for endpoint device */ @@ -1032,7 +1031,7 @@ void pcie_aspm_exit_link_state(struct pci_dev *pdev)pcie_update_aspm_capable (root);pcie_config_aspm_path (parent_link); } -out: +mutex_unlock (&aspm_lock);up_read (&pci_bus_sem); }Signature · 1 line
2.17.1
Quoted 66 lines · Bjorn Helgaas
drivers/pci/pcie/aspm.c void pcie_aspm_exit_link_state(struct pci_dev *pdev)
Quoted 27 lines · Ding Hui
If the Function 0 of a Multi-Function device is software removed, a freed downstream pointer will be left in struct
pcie_link_state, and then whenpcie_config_aspm_link()be invoked from any path, we will trigger use-after-free.Based on the PCIe spec about ASPM Control (PCIe r6.0, sec 7.5.3.7), for Multi-Function Devices (including ARI Devices), it is recommended that software program the same value in all Functions. For ARI Devices, ASPM Control is determined solely by the setting in Function 0.
So we can just disable ASPM of the whole component if any child function is removed, the downstream pointer will be avoided from use-after-free, that will also avoid other potential corner cases.
Fixes b5a0a9b59c81 ("PCI/ASPM: Read and set up L1 substate capabilities")Debugged-by Zongquan Qin <[email protected]>Suggestion-by Bjorn Helgaas <[email protected]>Signed-off-by Ding Hui <[email protected]>--- drivers/pci/pcie/aspm.c | 15 +++++++-------- 1 file changed, 7 insertions(+), 8 deletions(-)drivers/pci/pcie/aspm.c +0
-0@@ -1010,18 +1010,17 @@ void pcie_aspm_exit_link_state(struct pci_dev *pdev)Not directly related to your patch, but this looks racy to me:
void pcie_aspm_exit_link_state(struct pci_dev *pdev) { struct pci_dev *parent = pdev->bus->self;if (!parent || !parent->link_state) return; down_read(&pci_bus_sem); mutex_lock(&aspm_lock); link = parent->link_state; root = link->root; ... free_link_state(link); link->pdev->link_state = NULL; kfree(link);Since we check "
parent->link_state" before acquiring the locks, I suspect that removing two functions at the same time could end up with a NULL pointer dereference:pcie_aspm_exit_link_state(fn 0) pcie_aspm_exit_link_state(fn 1) parent = X parent = X parent->link_state != NULL parent->link_state != NULLacquire locks free_link_state(link) link->pdev->link_state = NULL # aka parent->link_state kfree(link) release locks acquire locks link = parent->link_state # now NULL root = link->root # NULL ptr
Quoted 3 lines
What do you think? I guess if this *is* a race, it should be fixed by a separate patch.
Maybe there's no need to worry about the race, when a pci device is removing, the pci_rescan_remove_lock should be acquired.
Anyway, double check would be safer.
Quoted 36 lines
Quoted 35 lines
drivers/pci/pcie/aspm.c +7
-8down_read (&pci_bus_sem);mutex_lock (&aspm_lock); -/* -* All PCIe functions are in one slot, remove one function will remove -* the whole slot, so just wait until we are the last function left. -*/ -if (!list_empty (&parent->subordinate->devices)) - goto out; link = parent->link_state; root = link->root; parent_link = link->parent; -/* All functions are removed, so just disable ASPM for the link */ +/* +* Any function is removed (including software removing), just +* disable ASPM for the link, in case we can not configure the same +* setting for all functions. +* See PCIe r6.0, sec 7.5.3.7. +*/ pcie_config_aspm_link (link,0 );list_del (&link->sibling);/* Clock PM is for endpoint device */ @@ -1032,7 +1031,7 @@ void pcie_aspm_exit_link_state(struct pci_dev *pdev)pcie_update_aspm_capable (root);pcie_config_aspm_path (parent_link); } -out: +mutex_unlock (&aspm_lock);up_read (&pci_bus_sem); }Signature · 1 line
2.17.1
Signature · 2 lines
Thanks, - Ding Hui