Skip to contentSkip to conversation

disable PCIE 'Enable No Snoop' bit by default

RFC In review Shaohua Li 4 participants 8 messages Sep 6, 2007 – Sep 11, 2007 PCI

lore
Share & tools
lore

Shaohua Li

author
#0 lore raw

RFC 1/1 disable PCIE 'Enable No Snoop' bit by default

PCIE 'Enable No Snoop' bit is set by default per PCIE spec, but OS assumes PCI DMA is snooped, which is legacy PCI device does. Enabling no snoop might cause potential DMA issues. This patch disables it, if a driver can work with no snoop, we can provide a helper to enable it.

I didn't see any real breaks, but did see some devices with the bit enabled

drivers/pci/probe.c +14 -0
Index: linux/drivers/pci/probe.c
===================================================================
@@ -694,6 +694,19 @@ static void pci_read_irq(struct pci_dev 
 
 #define LEGACY_IO_RESOURCE	(IORESOURCE_IO | IORESOURCE_PCI_FIXED)
 
+static void pcie_setup_device(struct pci_dev *dev)
+{
+	u16 reg16;
+	int pos;
+
+	pos = pci_find_capability(dev, PCI_CAP_ID_EXP);
+	if (pos) {
+		pci_read_config_word(dev, pos + PCI_EXP_DEVCTL, &reg16);
+		reg16 &= ~PCI_EXP_DEVCTL_NOSNOOP_EN;
+		pci_write_config_word(dev, pos + PCI_EXP_DEVCTL, reg16);
+	}
+}
+
 /**
  * pci_setup_device - fill in class and map information of a device
  * @dev: the device structure to fill
@@ -795,6 +808,7 @@ static int pci_setup_device(struct pci_d
 		dev->class = PCI_CLASS_NOT_DEFINED;
 	}
 
+	pcie_setup_device(dev);
 	/* We found a fine healthy device, go go go... */
 	return 0;
 }
#1 lore raw
Quoted 4 lines · Shaohua Li

PCIE 'Enable No Snoop' bit is set by default per PCIE spec, but OS assumes PCI DMA is snooped, which is legacy PCI device does. Enabling no snoop might cause potential DMA issues. This patch disables it, if a driver can work with no snoop, we can provide a helper to enable it.

I'm not sure your analysis is correct. Here's what my draft copy of the pcie 2.0 spec says:

Enble No Snoop ­ If this bit is Set, the Function is permitted to Set the No Snoop bit in the Requester Attributes of transactions it initiates that do not require hardware enforced cache coherency (see Section 2.2.6.5). Note that setting this bit to 1b should not cause a Function to Set the No Snoop attribute on all transactions that it initiates. Even when this bit is Set, a Function is only permitted to Set the No Snoop attribute on a transaction when it can guarantee that the address of the transaction is not stored in any cache in the system. This bit permitted to be hardwired to 0b if a Function would never Set the No Snoop attribute in transactions it initiates. Default value of this bit is 1b.

That implies that devices are only allowed to set it when it's safe to do so ... and we don't need to turn it off.

Signature · 4 lines
Intel are signing my paycheques ... these opinions are still mine
"Bill, look, we understand that you're interested in selling us this
operating system, but compare it to ours.  We can't possibly take such
a retrograde step."
#2 lore raw

From: Matthew Wilcox <[email protected]>
Date: Thu, 6 Sep 2007 05:57:31 -0600

Quoted 15 lines

I'm not sure your analysis is correct. Here's what my draft copy of the pcie 2.0 spec says:

Enble No Snoop ­ If this bit is Set, the Function is permitted to Set the No Snoop bit in the Requester Attributes of transactions it initiates that do not require hardware enforced cache coherency (see Section 2.2.6.5). Note that setting this bit to 1b should not cause a Function to Set the No Snoop attribute on all transactions that it initiates. Even when this bit is Set, a Function is only permitted to Set the No Snoop attribute on a transaction when it can guarantee that the address of the transaction is not stored in any cache in the system. This bit permitted to be hardwired to 0b if a Function would never Set the No Snoop attribute in transactions it initiates. Default value of this bit is 1b.

Quoted 2 lines

That implies that devices are only allowed to set it when it's safe to do so ... and we don't need to turn it off.

This is my understanding of this area of PCI-E as well, and I also agree that therefore we should not turn this bit off.

#3 lore raw
Quoted 18 lines · David Miller

From: Matthew Wilcox <[email protected]>
Date: Thu, 6 Sep 2007 05:57:31 -0600

Quoted 15 lines

I'm not sure your analysis is correct. Here's what my draft copy of the pcie 2.0 spec says:

Enble No Snoop ? If this bit is Set, the Function is permitted to Set the No Snoop bit in the Requester Attributes of transactions it initiates that do not require hardware enforced cache coherency (see Section 2.2.6.5). Note that setting this bit to 1b should not cause a Function to Set the No Snoop attribute on all transactions that it initiates. Even when this bit is Set, a Function is only permitted to Set the No Snoop attribute on a transaction when it can guarantee that the address of the transaction is not stored in any cache in the system. This bit permitted to be hardwired to 0b if a Function would never Set the No Snoop attribute in transactions it initiates. Default value of this bit is 1b.

Quoted 5 lines
Quoted 2 lines

That implies that devices are only allowed to set it when it's safe to do so ... and we don't need to turn it off.

This is my understanding of this area of PCI-E as well, and I also agree that therefore we should not turn this bit off.

I agree. But Shaohua, do you see any problems that this patch fixes?

thanks,

greg k-h

author
#4 lore raw
Quoted 19 lines · Greg KH
Quoted 18 lines · David Miller

From: Matthew Wilcox <[email protected]>
Date: Thu, 6 Sep 2007 05:57:31 -0600

Quoted 15 lines

I'm not sure your analysis is correct. Here's what my draft copy of the pcie 2.0 spec says:

Enble No Snoop ? If this bit is Set, the Function is permitted to Set the No Snoop bit in the Requester Attributes of transactions it initiates that do not require hardware enforced cache coherency (see Section 2.2.6.5). Note that setting this bit to 1b should not cause a Function to Set the No Snoop attribute on all transactions that it initiates. Even when this bit is Set, a Function is only permitted to Set the No Snoop attribute on a transaction when it can guarantee that the address of the transaction is not stored in any cache in the system. This bit permitted to be hardwired to 0b if a Function would never Set the No Snoop attribute in transactions it initiates. Default value of this bit is 1b.

Quoted 7 lines
Quoted 5 lines
Quoted 2 lines

That implies that devices are only allowed to set it when it's safe to do so ... and we don't need to turn it off.

This is my understanding of this area of PCI-E as well, and I also agree that therefore we should not turn this bit off.

I agree. But Shaohua, do you see any problems that this patch fixes?

No, I didn't see any breakage, just worry about it's a potential issue.

#5 lore raw
Quoted 20 lines · Shaohua Li
Quoted 19 lines · Greg KH
Quoted 18 lines · David Miller

From: Matthew Wilcox <[email protected]>
Date: Thu, 6 Sep 2007 05:57:31 -0600

Quoted 15 lines

I'm not sure your analysis is correct. Here's what my draft copy of the pcie 2.0 spec says:

Enble No Snoop ? If this bit is Set, the Function is permitted to Set the No Snoop bit in the Requester Attributes of transactions it initiates that do not require hardware enforced cache coherency (see Section 2.2.6.5). Note that setting this bit to 1b should not cause a Function to Set the No Snoop attribute on all transactions that it initiates. Even when this bit is Set, a Function is only permitted to Set the No Snoop attribute on a transaction when it can guarantee that the address of the transaction is not stored in any cache in the system. This bit permitted to be hardwired to 0b if a Function would never Set the No Snoop attribute in transactions it initiates. Default value of this bit is 1b.

Quoted 8 lines
Quoted 7 lines
Quoted 5 lines
Quoted 2 lines

That implies that devices are only allowed to set it when it's safe to do so ... and we don't need to turn it off.

This is my understanding of this area of PCI-E as well, and I also agree that therefore we should not turn this bit off.

I agree. But Shaohua, do you see any problems that this patch fixes?

No, I didn't see any breakage, just worry about it's a potential issue.

Hm, well, if you don't mind, I'd like to leave it as is for now, as no one is reporting any problems with this, and there seems to be some disagreement as to if it is really needed or not.

Is that ok?

thanks,

greg k-h

author
#6 lore raw
Quoted 27 lines · Greg KH
Quoted 26 lines · Shaohua Li
Quoted 25 lines · Greg KH
Quoted 23 lines · David Miller

From: Matthew Wilcox <[email protected]>
Date: Thu, 6 Sep 2007 05:57:31 -0600

Quoted 15 lines

I'm not sure your analysis is correct. Here's what my draft copy of the pcie 2.0 spec says:

Enble No Snoop ? If this bit is Set, the Function is permitted to Set the No Snoop bit in the Requester Attributes of transactions it initiates that do not require hardware enforced cache coherency (see Section 2.2.6.5). Note that setting this bit to 1b should not cause a Function to Set the No Snoop attribute on all transactions that it initiates. Even when this bit is Set, a Function is only permitted to Set the No Snoop attribute on a transaction when it can guarantee that the address of the transaction is not stored in any cache in the system. This bit permitted to be hardwired to 0b if a Function would never Set the No Snoop attribute in transactions it initiates. Default value of this bit is 1b.

Quoted 2 lines

That implies that devices are only allowed to set it when it's safe to do so ... and we don't need to turn it off.

This is my understanding of this area of PCI-E as well, and I also agree that therefore we should not turn this bit off.

Quoted 8 lines
Quoted 2 lines
Quoted 1 line

I agree. But Shaohua, do you see any problems that this patch fixes?

No, I didn't see any breakage, just worry about it's a potential issue.

Hm, well, if you don't mind, I'd like to leave it as is for now, as no one is reporting any problems with this, and there seems to be some disagreement as to if it is really needed or not.

Is that ok?

Ok.

Regards,
Shaohua

author
#7 lore raw
In reply to Matthew Wilcox
Quoted 21 lines · Matthew Wilcox
Quoted 4 lines · Shaohua Li

PCIE 'Enable No Snoop' bit is set by default per PCIE spec, but OS assumes PCI DMA is snooped, which is legacy PCI device does. Enabling no snoop might cause potential DMA issues. This patch disables it, if a driver can work with no snoop, we can provide a helper to enable it.

I'm not sure your analysis is correct. Here's what my draft copy of the pcie 2.0 spec says:

Enble No Snoop ­ If this bit is Set, the Function is permitted to Set the No Snoop bit in the Requester Attributes of transactions it initiates that do not require hardware enforced cache coherency (see Section 2.2.6.5). Note that setting this bit to 1b should not cause a Function to Set the No Snoop attribute on all transactions that it initiates. Even when this bit is Set, a Function is only permitted to Set the No Snoop attribute on a transaction when it can guarantee that the address of the transaction is not stored in any cache in the system. This bit permitted to be hardwired to 0b if a Function would never Set the No Snoop attribute in transactions it initiates. Default value of this bit is 1b.

Quoted 2 lines

That implies that devices are only allowed to set it when it's safe to do so ... and we don't need to turn it off.

Yes, I saw this in spec too. Devices don't know how OS handle DMA, so it's possible the bit is wrongly set. My point is setting the bit can be done by driver based. If a driver thinks it's really safe to do so, it can set the bit.

Thanks,
Shaohua