iommu/vt-d: Some misc tweaks in SVA
Subsystems (2)
Lu Baolu
maintainercover iommu/vt-d: Some misc tweaks in SVA
This includes some misc tweaks in the VT-d SVA implementation. I will plan them for v5.12 if no objections.
Best regards,
baolu
Lu Baolu (3):
iommu/vt-d: Add rate limited information when PRQ overflows iommu/vt-d: Allow devices to have more than 32 outstanding PRQ iommu/vt-d: Use INVALID response code instead of FAILURE
drivers/iommu/intel/iommu.c | 3 ++- drivers/iommu/intel/svm.c | 19 +++++++++---------- include/linux/intel-svm.h | 5 +++++ 3 files changed, 16 insertions(+), 11 deletions(-)
Signature · 6 lines
2.25.1 _______________________________________________ iommu mailing list [email protected] https://lists.linuxfoundation.org/mailman/listinfo/iommu
Lu Baolu
maintainerPATCH 1/3 iommu/vt-d: Add rate limited information when PRQ overflows
So that the uses could get chances to know what happened.
--- drivers/iommu/intel/svm.c | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-)
drivers/iommu/intel/svm.c +8 -2
@@ -895,6 +895,7 @@ static irqreturn_t prq_event_thread(int irq, void *d)struct intel_iommu *iommu = d;struct intel_svm *svm =NULL ;int head, tail, handled =0 ; +struct page_req_dsc *req;/* Clear PPR bit before reading head/tail registers, to * ensure that we get a new interrupt if needed. */
@@ -904,7 +905,6 @@ static irqreturn_t prq_event_thread(int irq, void *d) head =dmar_readq (iommu->reg +DMAR_PQH_REG ) &PRQ_RING_MASK ;while (head != tail) {struct vm_area_struct *vma; -struct page_req_dsc *req;struct qi_desc resp;int result;vm_fault_t ret;
@@ -1042,8 +1042,14 @@ static irqreturn_t prq_event_thread(int irq, void *d)* Clear the page request overflow bit and wake up all threads that * are waiting for the completion of this handling. */ -if (readl (iommu->reg +DMAR_PRS_REG ) &DMA_PRS_PRO ) +if (readl (iommu->reg +DMAR_PRS_REG ) &DMA_PRS_PRO ) { + head =dmar_readq (iommu->reg +DMAR_PQH_REG ) &PRQ_RING_MASK ; + req = &iommu->prq[head /sizeof (*req)]; +pr_warn_ratelimited ("IOMMU: %s: Page request overflow: HEAD: %08llx %08llx" , + iommu->name, ((unsigned long long *)req)[0 ], + ((unsigned long long *)req)[1 ]);writel (DMA_PRS_PRO , iommu->reg +DMAR_PRS_REG ); + }if (!completion_done (&iommu->prq_complete))complete (&iommu->prq_complete);
Signature · 6 lines
2.25.1 _______________________________________________ iommu mailing list [email protected] https://lists.linuxfoundation.org/mailman/listinfo/iommu
RE: [PATCH 1/3] iommu/vt-d: Add rate limited information when PRQ overflows
Quoted 40 lines
From: Lu Baolu <[email protected]>
Sent: Thursday, January 21, 2021 9:45 AMSo that the uses could get chances to know what happened.
Suggested-by Ashok Raj <[email protected]>Signed-off-by Lu Baolu <[email protected]>--- drivers/iommu/intel/svm.c | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-)drivers/iommu/intel/svm.c +1
-1@@ -895,6 +895,7 @@ static irqreturn_t prq_event_thread(int irq, void *d)struct intel_iommu *iommu = d;struct intel_svm *svm =NULL ;int head, tail, handled =0 ; +struct page_req_dsc *req;/* Clear PPR bit before reading head/tail registers, to * ensure that we get a new interrupt if needed. */ @@ -904,7 +905,6 @@ static irqreturn_t prq_event_thread(int irq, void *d) head =dmar_readq (iommu->reg +DMAR_PQH_REG ) &PRQ_RING_MASK ;while (head != tail) {struct vm_area_struct *vma; -struct page_req_dsc *req;struct qi_desc resp;int result;drivers/iommu/intel/svm.c +1
-1vm_fault_t ret;@@ -1042,8 +1042,14 @@ static irqreturn_t prq_event_thread(int irq, void*d) * Clear the page request overflow bit and wake up all threads that * are waiting for the completion of this handling. */ -if (readl (iommu->reg +DMAR_PRS_REG ) &DMA_PRS_PRO ) +if (readl (iommu->reg +DMAR_PRS_REG ) &DMA_PRS_PRO ) {
drivers/iommu/intel/svm.c static irqreturn_t prq_event_thread(int irq, void · line 1052
Quoted 10 lines
drivers/iommu/intel/svm.c +6
-0+ head =dmar_readq (iommu->reg +DMAR_PQH_REG ) &PRQ_RING_MASK ; + req = &iommu->prq[head /sizeof (*req)]; + pr_warn_ratelimited("IOMMU: %s: Page request overflow: HEAD: %08llx %08llx", + iommu->name, ((unsigned long long *)req)[0 ], + ((unsigned long long *)req)[1 ]);writel (DMA_PRS_PRO , iommu->reg +DMAR_PRS_REG ); + }
Not about rate limiting but I think we may have a problem in above logic. It is incorrect to always clear PRO when it's set w/o first checking whether the overflow condition has been cleared. This code assumes that if an overflow condition occurs it must have been cleared by earlier loop when hitting this check. However since this code runs in a threaded context, the overflow condition could occur even after you reset the head to the tail (under some extreme condition). To be sane I think we'd better read both head/tail again after seeing a pending PRO here and only clear PRO when it becomes a false indicator based on latest head/tail.
Thanks
Kevin
_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu
Lu Baolu
maintainerRe: [PATCH 1/3] iommu/vt-d: Add rate limited information when PRQ overflows
Hi Kevin,
Quoted 52 lines · Kevin
Quoted 40 lines
From: Lu Baolu <[email protected]>
Sent: Thursday, January 21, 2021 9:45 AMSo that the uses could get chances to know what happened.
Suggested-by Ashok Raj <[email protected]>Signed-off-by Lu Baolu <[email protected]>--- drivers/iommu/intel/svm.c | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-)drivers/iommu/intel/svm.c +1
-1@@ -895,6 +895,7 @@ static irqreturn_t prq_event_thread(int irq, void *d)struct intel_iommu *iommu = d;struct intel_svm *svm =NULL ;int head, tail, handled =0 ; +struct page_req_dsc *req;/* Clear PPR bit before reading head/tail registers, to * ensure that we get a new interrupt if needed. */ @@ -904,7 +905,6 @@ static irqreturn_t prq_event_thread(int irq, void *d) head =dmar_readq (iommu->reg +DMAR_PQH_REG ) &PRQ_RING_MASK ;while (head != tail) {struct vm_area_struct *vma; -struct page_req_dsc *req;struct qi_desc resp;int result;
vm_fault_tret;drivers/iommu/intel/svm.c +1
-1@@ -1042,8 +1042,14 @@ static irqreturn_t prq_event_thread(int irq, void*d) * Clear the page request overflow bit and wake up all threads that * are waiting for the completion of this handling. */ -if (readl (iommu->reg +DMAR_PRS_REG ) &DMA_PRS_PRO ) +if (readl (iommu->reg +DMAR_PRS_REG ) &DMA_PRS_PRO ) {drivers/iommu/intel/svm.c static irqreturn_t prq_event_thread(int irq, void · line 1052
Quoted 10 lines
drivers/iommu/intel/svm.c +6
-0+ head =dmar_readq (iommu->reg +DMAR_PQH_REG ) &PRQ_RING_MASK ; + req = &iommu->prq[head /sizeof (*req)]; + pr_warn_ratelimited("IOMMU: %s: Page request overflow: HEAD: %08llx %08llx", + iommu->name, ((unsigned long long *)req)[0 ], + ((unsigned long long *)req)[1 ]);writel (DMA_PRS_PRO , iommu->reg +DMAR_PRS_REG ); + }Not about rate limiting but I think we may have a problem in above
Quoted 9 lines
logic. It is incorrect to always clear PRO when it's set w/o first checking whether the overflow condition has been cleared. This code assumes that if an overflow condition occurs it must have been cleared by earlier loop when hitting this check. However since this code runs in a threaded context, the overflow condition could occur even after you reset the head to the tail (under some extreme condition). To be sane I think we'd better read both head/tail again after seeing a pending PRO here and only clear PRO when it becomes a false indicator based on latest head/tail.
Yes, agreed. We can check the head and tail and clear the overflow bit until the queue is empty. The finial code looks like:
/*
* Clear the page request overflow bit and wake up all threads that
* are waiting for the completion of this handling.
*/
if (readl(iommu->reg + DMAR_PRS_REG) & DMA_PRS_PRO) {
head = dmar_readq(iommu->reg + DMAR_PQH_REG) &
PRQ_RING_MASK;
tail = dmar_readq(iommu->reg + DMAR_PQT_REG) &
PRQ_RING_MASK;
if (head == tail) {
req = &iommu->prq[head / sizeof(*req)];
pr_warn_ratelimited("IOMMU: %s: Page request
overflow cleared: HEAD: %08llx %08llx",
iommu->name, ((unsigned
long long *)req)[0],
((unsigned long long
*)req)[1]);
writel(DMA_PRS_PRO, iommu->reg + DMAR_PRS_REG);
}
}Thought?
Quoted 3 lines
Thanks
Kevin
Best regards,
baolu
_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu
RE: [PATCH 1/3] iommu/vt-d: Add rate limited information when PRQ overflows
Quoted 95 lines
From: Lu Baolu <[email protected]>
Sent: Monday, January 25, 2021 2:29 PMHi Kevin,
Quoted 11 lines · Kevin
Quoted 11 lines
From: Lu Baolu <[email protected]>
Sent: Thursday, January 21, 2021 9:45 AMSo that the uses could get chances to know what happened.
Suggested-by Ashok Raj <[email protected]>Signed-off-by Lu Baolu <[email protected]>--- drivers/iommu/intel/svm.c | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-)Quoted 5 lines
drivers/iommu/intel/svm.c static irqreturn_t prq_event_thread(int irq, void
Quoted 5 lines
drivers/iommu/intel/svm.c +0
-0@@ -895,6 +895,7 @@ static irqreturn_t prq_event_thread(int irq, void*d)
Quoted 8 lines
drivers/iommu/intel/svm.c static irqreturn_t prq_event_thread(int irq, void
Quoted 8 lines
drivers/iommu/intel/svm.c +1
-0struct intel_iommu *iommu = d;struct intel_svm *svm =NULL ;int head, tail, handled =0 ; +struct page_req_dsc *req;/* Clear PPR bit before reading head/tail registers, to * ensure that we get a new interrupt if needed. */ @@ -904,7 +905,6 @@ static irqreturn_t prq_event_thread(int irq, void*d)
Quoted 9 lines
drivers/iommu/intel/svm.c static irqreturn_t prq_event_thread(int irq,
Quoted 9 lines
drivers/iommu/intel/svm.c +0
-1head =dmar_readq (iommu->reg +DMAR_PQH_REG ) &PRQ_RING_MASK ;while (head != tail) {struct vm_area_struct *vma; -struct page_req_dsc *req;struct qi_desc resp;int result;
vm_fault_tret;drivers/iommu/intel/svm.c +0
-0@@ -1042,8 +1042,14 @@ static irqreturn_t prq_event_thread(int irq,void
Quoted 19 lines
Quoted 7 lines
drivers/iommu/intel/svm.c +1
-1*d) * Clear the page request overflow bit and wake up all threads that * are waiting for the completion of this handling. */ -if (readl (iommu->reg +DMAR_PRS_REG ) &DMA_PRS_PRO ) +if (readl (iommu->reg +DMAR_PRS_REG ) &DMA_PRS_PRO ) {drivers/iommu/intel/svm.c static irqreturn_t prq_event_thread(int irq, void *d) · line 1052
Quoted 10 lines
drivers/iommu/intel/svm.c +6
-0+ head =dmar_readq (iommu->reg +DMAR_PQH_REG ) &PRQ_RING_MASK ; + req = &iommu->prq[head /sizeof (*req)]; + pr_warn_ratelimited("IOMMU: %s: Page request overflow: HEAD: %08llx %08llx", + iommu->name, ((unsigned long long *)req)[0 ], + ((unsigned long long *)req)[1 ]);writel (DMA_PRS_PRO , iommu->reg +DMAR_PRS_REG ); + }Not about rate limiting but I think we may have a problem in above
Quoted 9 lines
logic. It is incorrect to always clear PRO when it's set w/o first checking whether the overflow condition has been cleared. This code assumes that if an overflow condition occurs it must have been cleared by earlier loop when hitting this check. However since this code runs in a threaded context, the overflow condition could occur even after you reset the head to the tail (under some extreme condition). To be sane I think we'd better read both head/tail again after seeing a pending PRO here and only clear PRO when it becomes a false indicator based on latest head/tail.
Yes, agreed. We can check the head and tail and clear the overflow bit until the queue is empty. The finial code looks like:
/* * Clear the page request overflow bit and wake up all threads that * are waiting for the completion of this handling. */ if (readl(iommu->reg + DMAR_PRS_REG) & DMA_PRS_PRO) { head = dmar_readq(iommu->reg + DMAR_PQH_REG) & PRQ_RING_MASK; tail = dmar_readq(iommu->reg + DMAR_PQT_REG) & PRQ_RING_MASK; if (head == tail) { req = &iommu->prq[head / sizeof(*req)]; pr_warn_ratelimited("IOMMU: %s: Page request overflow cleared: HEAD: %08llx %08llx", iommu->name, ((unsigned long long *)req)[0], ((unsigned long long *)req)[1]); writel(DMA_PRS_PRO, iommu->reg + DMAR_PRS_REG); } }
Quoted 2 lines
Thought?
Just a small comment. Is it useful to also print a warning in the true overflow condition which has to wait for next interrupt to be cleared?
Thanks
Kevin
_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu
Lu Baolu
maintainerRe: [PATCH 1/3] iommu/vt-d: Add rate limited information when PRQ overflows
Hi Kevin,
Quoted 95 lines · Kevin
Quoted 95 lines
From: Lu Baolu <[email protected]>
Sent: Monday, January 25, 2021 2:29 PMHi Kevin,
Quoted 11 lines · Kevin
Quoted 11 lines
From: Lu Baolu <[email protected]>
Sent: Thursday, January 21, 2021 9:45 AMSo that the uses could get chances to know what happened.
Suggested-by Ashok Raj <[email protected]>Signed-off-by Lu Baolu <[email protected]>--- drivers/iommu/intel/svm.c | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-)Quoted 5 lines
drivers/iommu/intel/svm.c static irqreturn_t prq_event_thread(int irq, void
Quoted 5 lines
drivers/iommu/intel/svm.c +0
-0@@ -895,6 +895,7 @@ static irqreturn_t prq_event_thread(int irq, void*d)
Quoted 8 lines
drivers/iommu/intel/svm.c static irqreturn_t prq_event_thread(int irq, void
Quoted 8 lines
drivers/iommu/intel/svm.c +1
-0struct intel_iommu *iommu = d;struct intel_svm *svm =NULL ;int head, tail, handled =0 ; +struct page_req_dsc *req;/* Clear PPR bit before reading head/tail registers, to * ensure that we get a new interrupt if needed. */ @@ -904,7 +905,6 @@ static irqreturn_t prq_event_thread(int irq, void*d)
Quoted 9 lines
drivers/iommu/intel/svm.c static irqreturn_t prq_event_thread(int irq,
Quoted 9 lines
drivers/iommu/intel/svm.c +0
-1head =dmar_readq (iommu->reg +DMAR_PQH_REG ) &PRQ_RING_MASK ;while (head != tail) {struct vm_area_struct *vma; -struct page_req_dsc *req;struct qi_desc resp;int result;vm_fault_t ret;drivers/iommu/intel/svm.c +0
-0@@ -1042,8 +1042,14 @@ static irqreturn_t prq_event_thread(int irq,void
Quoted 19 lines
Quoted 7 lines
drivers/iommu/intel/svm.c +1
-1*d) * Clear the page request overflow bit and wake up all threads that * are waiting for the completion of this handling. */ -if (readl (iommu->reg +DMAR_PRS_REG ) &DMA_PRS_PRO ) +if (readl (iommu->reg +DMAR_PRS_REG ) &DMA_PRS_PRO ) {drivers/iommu/intel/svm.c static irqreturn_t prq_event_thread(int irq, void *d) · line 1052
Quoted 10 lines
drivers/iommu/intel/svm.c +6
-0+ head =dmar_readq (iommu->reg +DMAR_PQH_REG ) &PRQ_RING_MASK ; + req = &iommu->prq[head /sizeof (*req)]; + pr_warn_ratelimited("IOMMU: %s: Page request overflow: HEAD: %08llx %08llx", + iommu->name, ((unsigned long long *)req)[0 ], + ((unsigned long long *)req)[1 ]);writel (DMA_PRS_PRO , iommu->reg +DMAR_PRS_REG ); + }Not about rate limiting but I think we may have a problem in above
Quoted 9 lines
logic. It is incorrect to always clear PRO when it's set w/o first checking whether the overflow condition has been cleared. This code assumes that if an overflow condition occurs it must have been cleared by earlier loop when hitting this check. However since this code runs in a threaded context, the overflow condition could occur even after you reset the head to the tail (under some extreme condition). To be sane I think we'd better read both head/tail again after seeing a pending PRO here and only clear PRO when it becomes a false indicator based on latest head/tail.
Yes, agreed. We can check the head and tail and clear the overflow bit until the queue is empty. The finial code looks like:
/* * Clear the page request overflow bit and wake up all threads that * are waiting for the completion of this handling. */ if (readl(iommu->reg + DMAR_PRS_REG) & DMA_PRS_PRO) { head = dmar_readq(iommu->reg + DMAR_PQH_REG) & PRQ_RING_MASK; tail = dmar_readq(iommu->reg + DMAR_PQT_REG) & PRQ_RING_MASK; if (head == tail) { req = &iommu->prq[head / sizeof(*req)]; pr_warn_ratelimited("IOMMU: %s: Page request overflow cleared: HEAD: %08llx %08llx", iommu->name, ((unsigned long long *)req)[0], ((unsigned long long *)req)[1]); writel(DMA_PRS_PRO, iommu->reg + DMAR_PRS_REG); } }
Quoted 6 lines
Quoted 2 lines
Thought?
Just a small comment. Is it useful to also print a warning in the true overflow condition which has to wait for next interrupt to be cleared?
That's fine. :-)
Quoted 3 lines
Thanks
Kevin
Best regards,
baolu
_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu
Lu Baolu
maintainerPATCH 2/3 iommu/vt-d: Allow devices to have more than 32 outstanding PRQ
In reply to Lu BaoluThe minimum per-IOMMU PRQ queue size is one 4K page, this is more entries than the hardcoded limit of 32 in the current VT-d code. Some devices can support up to 512 outstanding PRQs but underutilized by this limit of 32. Although, 32 gives some rough fairness when multiple devices share the same IOMMU PRQ queue, but far from optimal for customized use cases.
--- drivers/iommu/intel/iommu.c | 3 ++- drivers/iommu/intel/svm.c | 4 ---- include/linux/intel-svm.h | 5 +++++ 3 files changed, 7 insertions(+), 5 deletions(-)
drivers/iommu/intel/iommu.c +2 -1
@@ -33,6 +33,7 @@#include <linux/iommu.h> #include <linux/dma-iommu.h> #include <linux/intel-iommu.h> +#include <linux/intel-svm.h> #include <linux/syscore_ops.h> #include <linux/tboot.h> #include <linux/dmi.h>
@@ -1527,7 +1528,7 @@ static void iommu_enable_dev_iotlb(struct device_domain_info *info)if (info->pri_supported && (info->pasid_enabled ?pci_prg_resp_pasid_required (pdev) :1 ) && - !pci_reset_pri (pdev) && !pci_enable_pri (pdev,32 )) + !pci_reset_pri (pdev) && !pci_enable_pri (pdev,PRQ_DEPTH )) info->pri_enabled =1 ;#endif if (info->ats_supported && pci_ats_page_aligned(pdev) &&
drivers/iommu/intel/svm.c +0 -4
@@ -26,8 +26,6 @@static irqreturn_t prq_event_thread (int irq,void *d);static void intel_svm_drain_prq (struct device *dev,u32 pasid); -#define PRQ_ORDER 0 -int intel_svm_enable_prq (struct intel_iommu *iommu) {struct page *pages;
@@ -722,8 +720,6 @@ struct page_req_dsc {u64 priv_data[2 ]; }; -#define PRQ_RING_MASK ((0x1000 << PRQ_ORDER) - 0x20) -static bool access_error (struct vm_area_struct *vma,struct page_req_dsc *req) {unsigned long requested =0 ;
include/linux/intel-svm.h +5 -0
@@ -21,6 +21,11 @@ struct svm_dev_ops {#define SVM_REQ_EXEC (1<<1)#define SVM_REQ_PRIV (1<<0) +/* Page Request Queue depth */ +#define PRQ_ORDER 0 +#define PRQ_RING_MASK ((0x1000 << PRQ_ORDER) - 0x20) +#define PRQ_DEPTH ((0x1000 << PRQ_ORDER) >> 5) +/* * The SVM_FLAG_PRIVATE_PASID flag requests a PASID which is *not* the "main" * PASID for the current process. Even if a PASID already exists, a new one
Signature · 6 lines
2.25.1 _______________________________________________ iommu mailing list [email protected] https://lists.linuxfoundation.org/mailman/listinfo/iommu
Lu Baolu
maintainerPATCH 3/3 iommu/vt-d: Use INVALID response code instead of FAILURE
In reply to Lu BaoluThe VT-d IOMMU response RESPONSE_FAILURE for a page request in below cases:
- When it gets a Page_Request with no PASID;
- When it gets a Page_Request with PASID that is not in use for this device.
This is allowed by the spec, but IOMMU driver doesn't support such cases today. When the device receives RESPONSE_FAILURE, it sends the device state machine to HALT state. Now if we try to unload the driver, it hangs since the device doesn't send any outbound transactions to host when the driver is trying to clear things up. The only possible responses would be for invalidation requests.
Let's use RESPONSE_INVALID instead for now, so that the device state machine doesn't enter HALT state.
--- drivers/iommu/intel/svm.c | 5 +---- 1 file changed, 1 insertion(+), 4 deletions(-)
drivers/iommu/intel/svm.c +1 -4
@@ -907,10 +907,8 @@ static irqreturn_t prq_event_thread(int irq, void *d)u64 address; handled =1 ; - req = &iommu->prq[head /sizeof (*req)]; - - result =QI_RESP_FAILURE ; + result =QI_RESP_INVALID ; address = (u64 )req->addr <<VTD_PAGE_SHIFT ;if (!req->pasid_present) { pr_err("%s: Page request without PASID: %08llx %08llx\n",
@@ -948,7 +946,6 @@ static irqreturn_t prq_event_thread(int irq, void *d)rcu_read_unlock (); } - result =QI_RESP_INVALID ;/* Since we're using init_mm.pgd directly, we should never take * any faults on kernel addresses. */ if (!svm->mm)
Signature · 6 lines
2.25.1 _______________________________________________ iommu mailing list [email protected] https://lists.linuxfoundation.org/mailman/listinfo/iommu