Skip to contentSkip to conversation

"iommu/vt-d: Delegate DMA domain to generic iommu" series breaks megaraid_sas

DISCUSSION Qian Cai 2 participants 7 messages Jun 7, 2019 – Jun 11, 2019 IOMMU

lore
Share & tools
lore

Qian Cai

author
#0 lore raw

The linux-next series "iommu/vt-d: Delegate DMA domain to generic iommu" [1] causes a system with the rootfs on megaraid_sas card unable to boot.

Reverted the whole series on the top of linux-next (next-20190607) fixed the issue.

The information regards this storage card is,

[  116.466810][  T324] megaraid_sas 0000:06:00.0: FW provided supportMaxExtLDs:
0 max_lds: 32
[  116.476052][  T324] megaraid_sas 0000:06:00.0: controller type :
iMR(0MB)
[  116.483646][  T324] megaraid_sas 0000:06:00.0: Online Controller Reset(OCR) : Enabled
[  116.492403][  T324] megaraid_sas 0000:06:00.0: Secure JBOD support :
Yes
[  116.499887][  T324] megaraid_sas 0000:06:00.0: NVMe passthru support :
No
[  116.507480][  T324] megaraid_sas 0000:06:00.0: FW provided
[  116.612523][  T324] megaraid_sas 0000:06:00.0: NVME page size : (0)
[  116.629991][  T324] megaraid_sas 0000:06:00.0: INIT adapter done
[  116.714789][  T324] megaraid_sas 0000:06:00.0: pci id :
(0x1000)/(0x0017)/(0x1d49)/(0x0500)
[  116.724228][  T324] megaraid_sas 0000:06:00.0: unevenspan support : no
[  116.731518][  T324] megaraid_sas 0000:06:00.0: firmware crash dump :
no
[  116.738981][  T324] megaraid_sas 0000:06:00.0: jbod sync map :
yes
[  116.787433][  T324] scsi host0: Avago SAS based MegaRAID driver [  117.081088][  T324] scsi 0:0:0:0: Direct-
Access     LENOVO   ST900MM0168      L587 PQ: 0 ANSI: 6

[1] https://lore.kernel.org/patchwork/cover/1078960/

#1 lore raw

Hi Qian,

I just posted some fix patches. I cc'ed them in your email inbox as well. Can you please check whether they happen to fix your issue? If not, do you mind posting more debug messages?

Best regards,
Baolu

Quoted 34 lines · Qian Cai

The linux-next series "iommu/vt-d: Delegate DMA domain to generic iommu" [1] causes a system with the rootfs on megaraid_sas card unable to boot.

Reverted the whole series on the top of linux-next (next-20190607) fixed the issue.

The information regards this storage card is,

[  116.466810][  T324] megaraid_sas 0000:06:00.0: FW provided supportMaxExtLDs:
0 max_lds: 32
[  116.476052][  T324] megaraid_sas 0000:06:00.0: controller type :
iMR(0MB)
[  116.483646][  T324] megaraid_sas 0000:06:00.0: Online Controller Reset(OCR) : Enabled
[  116.492403][  T324] megaraid_sas 0000:06:00.0: Secure JBOD support :
Yes
[  116.499887][  T324] megaraid_sas 0000:06:00.0: NVMe passthru support :
No
[  116.507480][  T324] megaraid_sas 0000:06:00.0: FW provided
[  116.612523][  T324] megaraid_sas 0000:06:00.0: NVME page size : (0)
[  116.629991][  T324] megaraid_sas 0000:06:00.0: INIT adapter done
[  116.714789][  T324] megaraid_sas 0000:06:00.0: pci id :
(0x1000)/(0x0017)/(0x1d49)/(0x0500)
[  116.724228][  T324] megaraid_sas 0000:06:00.0: unevenspan support : no
[  116.731518][  T324] megaraid_sas 0000:06:00.0: firmware crash dump :
no
[  116.738981][  T324] megaraid_sas 0000:06:00.0: jbod sync map :
yes
[  116.787433][  T324] scsi host0: Avago SAS based MegaRAID driver [  117.081088][  T324] scsi 0:0:0:0: Direct-
Access     LENOVO   ST900MM0168      L587 PQ: 0 ANSI: 6

[1] https://lore.kernel.org/patchwork/cover/1078960/

author
#2 lore raw
Quoted 5 lines · Lu Baolu

Hi Qian,

I just posted some fix patches. I cc'ed them in your email inbox as well. Can you please check whether they happen to fix your issue? If not, do you mind posting more debug messages?

Unfortunately, it does not work. Here is the dmesg.

https://raw.githubusercontent.com/cailca/tmp/master/dmesg?token=AMC35QKPIZBYUMFU
QKLW4ZC47ZPIK

_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu

author
#3 lore raw
Quoted 11 lines · Qian Cai
Quoted 5 lines · Lu Baolu

Hi Qian,

I just posted some fix patches. I cc'ed them in your email inbox as well. Can you please check whether they happen to fix your issue? If not, do you mind posting more debug messages?

Unfortunately, it does not work. Here is the dmesg.

https://raw.githubusercontent.com/cailca/tmp/master/dmesg?token=AMC35QKPIZBYUM
FUQKLW4ZC47ZPIK

This one should be good to view.

https://cailca.github.io/files/dmesg.txt
_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu

#4 lore raw

Ah, good catch!

The device failed to be attached by a DMA domain. Can you please try the attached fix patch?

[  101.885468] pci 0000:06:00.0: DMAR: Device is ineligible for IOMMU
domain attach due to platform RMRR requirement.  Contact your platform
vendor.
[  101.900801] pci 0000:06:00.0: Failed to add to iommu group 23: -1

Best regards,
Baolu

Quoted 8 lines · Qian Cai
Quoted 7 lines · Qian Cai
Quoted 5 lines · Lu Baolu

Hi Qian,

I just posted some fix patches. I cc'ed them in your email inbox as well. Can you please check whether they happen to fix your issue? If not, do you mind posting more debug messages?

Quoted 9 lines
Quoted 4 lines

Unfortunately, it does not work. Here is the dmesg.

https://raw.githubusercontent.com/cailca/tmp/master/dmesg?token=AMC35QKPIZBYUM
FUQKLW4ZC47ZPIK

This one should be good to view.

https://cailca.github.io/files/dmesg.txt

From ff0b1ae0d8fde0655392fde3a1090b03a7a35394 Mon Sep 17 00:00:00 2001
From: Lu Baolu <[email protected]>
Date: Tue, 11 Jun 2019 09:29:16 +0800
Subject: [PATCH 1/1] iommu/vt-d: Allow DMA domain attaching to rmrr locked
device

We don't allow a device to be assigned to user level when it is locked by any RMRR's. Hence, intel_iommu_attach_device() will return error if a domain of type IOMMU_DOMAIN_UNMANAGED is about to attach to a device locked by rmrr. But this doesn't apply to a domain of type other than IOMMU_DOMAIN_UNMANAGED. This adds a check to fix this.

Fixes fa954e6831789 ("iommu/vt-d: Delegate the dma domain to upper layer")
---
 drivers/iommu/intel-iommu.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
drivers/iommu/intel-iommu.c +2 -1
index 1dcb6365ddc4..38232220f6ff 100644
@@ -5281,7 +5281,8 @@ static int intel_iommu_attach_device(struct iommu_domain *domain,
 {
 	int ret;
 
-	if (device_is_rmrr_locked(dev)) {
+	if (domain->type == IOMMU_DOMAIN_UNMANAGED &&
+	    device_is_rmrr_locked(dev)) {
 		dev_warn(dev, "Device is ineligible for IOMMU domain attach due to platform RMRR requirement.  Contact your platform vendor.\n");
 		return -EPERM;
 	}
Signature · 1 line
2.17.1
author
#5 lore raw
Quoted 8 lines · Lu Baolu

Ah, good catch!

The device failed to be attached by a DMA domain. Can you please try the attached fix patch?

It works fine.

Quoted 25 lines
[  101.885468] pci 0000:06:00.0: DMAR: Device is ineligible for IOMMU
domain attach due to platform RMRR requirement.  Contact your platform
vendor.
[  101.900801] pci 0000:06:00.0: Failed to add to iommu group 23: -1

Best regards,
Baolu

Quoted 8 lines · Qian Cai
Quoted 7 lines · Qian Cai
Quoted 5 lines · Lu Baolu

Hi Qian,

I just posted some fix patches. I cc'ed them in your email inbox as well. Can you please check whether they happen to fix your issue? If not, do you mind posting more debug messages?

Quoted 6 lines
Quoted 4 lines

Unfortunately, it does not work. Here is the dmesg.

https://raw.githubusercontent.com/cailca/tmp/master/dmesg?token=AMC35QKPIZBYUM
FUQKLW4ZC47ZPIK

This one should be good to view.
https://cailca.github.io/files/dmesg.txt

<0001-iommu-vt-d-Allow-DMA-domain-attaching-to-rmrr-locked.patch>

#6 lore raw

Hi,

Quoted 9 lines · Qian Cai
Quoted 7 lines · Lu Baolu

Ah, good catch!

The device failed to be attached by a DMA domain. Can you please try the attached fix patch?

It works fine.

Thanks a lot for the report and verification.

Best regards,
Baolu