Skip to contentSkip to conversation

ata: call flush_dcache_page() around PIO data transfers in libata-aff.c

PATCH In review [email protected] 6 participants 31 messages Feb 2, 2010 – Feb 4, 2010 libata

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 ata: call flush_dcache_page() around PIO data transfers in libata-aff.c

From: Catalin Marinas <[email protected]>

Depending on the direction of the transfer, flush_dcache_page() must be called either before (ATA_TFLAG_WRITE) or after (!ATA_TFLAG_WRITE) the data copying to avoid D-cache aliasing with user space or I-D cache coherency issues (when reading data from an ATA device using PIO, the kernel dirties the D-cache but there is no flush_dcache_page() required on Harvard architectures).

This patch allows the ARM boards to use a rootfs on CompactFlash with the PATA platform driver.

As Anfei Zhou mentioned in a recent patch ("flush dcache before writing into page to avoid alias"), on some architectures there may be a performance benefit in differentiating the flush_dcache_page() calls based on whether the kernel or the user page needs flushing.

IMHO, we should differentiate based on the direction (kernel reading or writing from/to such page). In the ARM case with PIPT Harvard caches (newer processors), the kernel reading from a page that may be mapped in user space shouldn't need cache flushing. The kernel writing to such page would require D-cache flushing because of coherency with the I-cache. Currently on ARM, the latter happens in both cases.

---

 drivers/ata/libata-sff.c |    6 ++++++
 1 file changed, 6 insertions(+)
drivers/ata/libata-sff.c renamed from drivers/ata/libata-sff.c~ata-call-flush_dcache_page-around-pio-data-transfers-in-libata-affc +6 -0
diff -puN drivers/ata/libata-sff.c~ata-call-flush_dcache_page-around-pio-data-transfers-in-libata-affc drivers/ata/libata-sff.c
@@ -874,6 +874,9 @@ static void ata_pio_sector(struct ata_qu
 
 	DPRINTK("data %s\n", qc->tf.flags & ATA_TFLAG_WRITE ? "write" : "read");
 
+	if (do_write)
+		flush_dcache_page(page);
+
 	if (PageHighMem(page)) {
 		unsigned long flags;
 
@@ -893,6 +896,9 @@ static void ata_pio_sector(struct ata_qu
 				       do_write);
 	}
 
+	if (!do_write)
+		flush_dcache_page(page);
+
 	qc->curbytes += qc->sect_size;
 	qc->cursg_ofs += qc->sect_size;
 

_

#1 lore raw
Quoted 38 lines · On Tue, 2010-02-02 at 14:11 -0800, [email protected] wrote:

From: Catalin Marinas <[email protected]>

Depending on the direction of the transfer, flush_dcache_page() must be called either before (ATA_TFLAG_WRITE) or after (!ATA_TFLAG_WRITE) the data copying to avoid D-cache aliasing with user space or I-D cache coherency issues (when reading data from an ATA device using PIO, the kernel dirties the D-cache but there is no flush_dcache_page() required on Harvard architectures).

This patch allows the ARM boards to use a rootfs on CompactFlash with the PATA platform driver.

As Anfei Zhou mentioned in a recent patch ("flush dcache before writing into page to avoid alias"), on some architectures there may be a performance benefit in differentiating the flush_dcache_page() calls based on whether the kernel or the user page needs flushing.

IMHO, we should differentiate based on the direction (kernel reading or writing from/to such page). In the ARM case with PIPT Harvard caches (newer processors), the kernel reading from a page that may be mapped in user space shouldn't need cache flushing. The kernel writing to such page would require D-cache flushing because of coherency with the I-cache. Currently on ARM, the latter happens in both cases.

---

 drivers/ata/libata-sff.c |    6 ++++++
 1 file changed, 6 insertions(+)

diff -puN drivers/ata/libata-sff.c~ata-call-flush_dcache_page-around-pio-data-transfers-in-libata-affc drivers/ata/libata-sff.c

drivers/ata/libata-sff.c static void ata_pio_sector(struct ata_qu · line 879

Quoted 6 lines
drivers/ata/libata-sff.c renamed from drivers/ata/libata-sff.c~ata-call-flush_dcache_page-around-pio-data-transfers-in-libata-affc +3 -0
@@ -874,6 +874,9 @@ static void ata_pio_sector(struct ata_qu
 
 	DPRINTK("data %s\n", qc->tf.flags & ATA_TFLAG_WRITE ? "write" : "read");
 
+	if (do_write)
+		flush_dcache_page(page);
+

This looks wrong; the upper layers should already have made the page aliases coherent from user to kernel by calling flush_dcache_page (in __get_user_pages()), so the aliases should already be up to date and this flush is spurious.

drivers/ata/libata-sff.c static void ata_pio_sector(struct ata_qu · line 901

Quoted 10 lines
drivers/ata/libata-sff.c renamed from drivers/ata/libata-sff.c~ata-call-flush_dcache_page-around-pio-data-transfers-in-libata-affc +3 -0
 	if (PageHighMem(page)) {
 		unsigned long flags;
 
@@ -893,6 +896,9 @@ static void ata_pio_sector(struct ata_qu
 				       do_write);
 	}
 
+	if (!do_write)
+		flush_dcache_page(page);
+

OK, so this too looks wrong for two reasons

1. it's over flushing. Even after the write to the page by the kernel PIO, the only alias that is dirty should be the kernel, so this needs a flush_kernel_dcache_page() to empty the kernel alias. It is possible user space will have speculated over the user aliases, but there's stuff further up the block stack to bring this back into coherence.
2. If the page really is in highmem, the flush has to happen along the kernel alias, which you just lost because this flush is happening after the kunmap_atomic(), so it has to occur somewhere between the PIO operation and the kunmap.

drivers/ata/libata-sff.c static void ata_pio_sector(struct ata_qu · line 904

Quoted 3 lines
drivers/ata/libata-sff.c renamed from drivers/ata/libata-sff.c~ata-call-flush_dcache_page-around-pio-data-transfers-in-libata-affc +0 -0
 	qc->curbytes += qc->sect_size;
 	qc->cursg_ofs += qc->sect_size;
 

James

#2 lore raw
Quoted 42 lines · On Tue, 02 Feb 2010 16:58:38 -0600 James Bottomley <[email protected]>…
Quoted 41 lines · On Tue, 2010-02-02 at 14:11 -0800, [email protected] wrote:

From: Catalin Marinas <[email protected]>

Depending on the direction of the transfer, flush_dcache_page() must be called either before (ATA_TFLAG_WRITE) or after (!ATA_TFLAG_WRITE) the data copying to avoid D-cache aliasing with user space or I-D cache coherency issues (when reading data from an ATA device using PIO, the kernel dirties the D-cache but there is no flush_dcache_page() required on Harvard architectures).

This patch allows the ARM boards to use a rootfs on CompactFlash with the PATA platform driver.

As Anfei Zhou mentioned in a recent patch ("flush dcache before writing into page to avoid alias"), on some architectures there may be a performance benefit in differentiating the flush_dcache_page() calls based on whether the kernel or the user page needs flushing.

IMHO, we should differentiate based on the direction (kernel reading or writing from/to such page). In the ARM case with PIPT Harvard caches (newer processors), the kernel reading from a page that may be mapped in user space shouldn't need cache flushing. The kernel writing to such page would require D-cache flushing because of coherency with the I-cache. Currently on ARM, the latter happens in both cases.

---

 drivers/ata/libata-sff.c |    6 ++++++
 1 file changed, 6 insertions(+)

diff -puN drivers/ata/libata-sff.c~ata-call-flush_dcache_page-around-pio-data-transfers-in-libata-affc drivers/ata/libata-sff.c

drivers/ata/libata-sff.c renamed from drivers/ata/libata-sff.c~ata-call-flush_dcache_page-around-pio-data-transfers-in-libata-affc +0 -0
@@ -874,6 +874,9 @@ static void ata_pio_sector(struct ata_qu
 
 	DPRINTK("data %s\n", qc->tf.flags & ATA_TFLAG_WRITE ? "write" : "read");
 
Quoted 8 lines

drivers/ata/libata-sff.c static void ata_pio_sector(struct ata_qu · line 879

Quoted 3 lines
drivers/ata/libata-sff.c renamed from drivers/ata/libata-sff.c~ata-call-flush_dcache_page-around-pio-data-transfers-in-libata-affc +3 -0
+	if (do_write)
+		flush_dcache_page(page);
+

This looks wrong; the upper layers should already have made the page aliases coherent from user to kernel by calling flush_dcache_page (in __get_user_pages()), so the aliases should already be up to date and this flush is spurious.

The upper layers don't know that the CPU touched the data! If the driver did a DMA transfer then such a flush is unneeded, so we don't do it.

Quoted 30 lines

drivers/ata/libata-sff.c static void ata_pio_sector(struct ata_qu · line 901

Quoted 10 lines
drivers/ata/libata-sff.c renamed from drivers/ata/libata-sff.c~ata-call-flush_dcache_page-around-pio-data-transfers-in-libata-affc +3 -0
 	if (PageHighMem(page)) {
 		unsigned long flags;
 
@@ -893,6 +896,9 @@ static void ata_pio_sector(struct ata_qu
 				       do_write);
 	}
 
+	if (!do_write)
+		flush_dcache_page(page);
+

OK, so this too looks wrong for two reasons

1. it's over flushing. Even after the write to the page by the kernel PIO, the only alias that is dirty should be the kernel, so this needs a flush_kernel_dcache_page() to empty the kernel alias. It is possible user space will have speculated over the user aliases, but there's stuff further up the block stack to bring this back into coherence.
2. If the page really is in highmem, the flush has to happen along the kernel alias, which you just lost because this flush is happening after the kunmap_atomic(), so it has to occur somewhere between the PIO operation and the kunmap.

drivers/ata/libata-sff.c static void ata_pio_sector(struct ata_qu · line 904

Quoted 3 lines
drivers/ata/libata-sff.c renamed from drivers/ata/libata-sff.c~ata-call-flush_dcache_page-around-pio-data-transfers-in-libata-affc +0 -0
 	qc->curbytes += qc->sect_size;
 	qc->cursg_ofs += qc->sect_size;
 

James

#3 lore raw
Quoted 54 lines · Andrew Morton
Quoted 42 lines · On Tue, 02 Feb 2010 16:58:38 -0600 James Bottomley<[email protected]>…
Quoted 41 lines · On Tue, 2010-02-02 at 14:11 -0800, [email protected] wrote:

From: Catalin Marinas<[email protected]>

Depending on the direction of the transfer, flush_dcache_page() must be called either before (ATA_TFLAG_WRITE) or after (!ATA_TFLAG_WRITE) the data copying to avoid D-cache aliasing with user space or I-D cache coherency issues (when reading data from an ATA device using PIO, the kernel dirties the D-cache but there is no flush_dcache_page() required on Harvard architectures).

This patch allows the ARM boards to use a rootfs on CompactFlash with the PATA platform driver.

As Anfei Zhou mentioned in a recent patch ("flush dcache before writing into page to avoid alias"), on some architectures there may be a performance benefit in differentiating the flush_dcache_page() calls based on whether the kernel or the user page needs flushing.

IMHO, we should differentiate based on the direction (kernel reading or writing from/to such page). In the ARM case with PIPT Harvard caches (newer processors), the kernel reading from a page that may be mapped in user space shouldn't need cache flushing. The kernel writing to such page would require D-cache flushing because of coherency with the I-cache. Currently on ARM, the latter happens in both cases.

---

  drivers/ata/libata-sff.c |    6 ++++++
  1 file changed, 6 insertions(+)

diff -puN drivers/ata/libata-sff.c~ata-call-flush_dcache_page-around-pio-data-transfers-in-libata-affc drivers/ata/libata-sff.c

drivers/ata/libata-sff.c renamed from drivers/ata/libata-sff.c~ata-call-flush_dcache_page-around-pio-data-transfers-in-libata-affc +0 -0
@@ -874,6 +874,9 @@ static void ata_pio_sector(struct ata_qu
 
  	DPRINTK("data %s\n", qc->tf.flags&  ATA_TFLAG_WRITE ? "write" : "read");
 
Quoted 8 lines

drivers/ata/libata-sff.c static void ata_pio_sector(struct ata_qu · line 879

Quoted 3 lines
drivers/ata/libata-sff.c renamed from drivers/ata/libata-sff.c~ata-call-flush_dcache_page-around-pio-data-transfers-in-libata-affc +3 -0
+	if (do_write)
+		flush_dcache_page(page);
+

This looks wrong; the upper layers should already have made the page aliases coherent from user to kernel by calling flush_dcache_page (in __get_user_pages()), so the aliases should already be up to date and this flush is spurious.

Quoted 3 lines

The upper layers don't know that the CPU touched the data! If the driver did a DMA transfer then such a flush is unneeded, so we don't do it.

The patch in question only affects PIO transfers, not DMA. Data is transferred from a kernel buffer to hardware via out[bwl] via

	page data -> CPU register -> out[bwl]

or, data is transferred from hardware to a kernel buffer via

	in[bwl] -> CPU register -> page data

So what are the flushing rules given those conditions?

Jeff

#29 lore raw
In reply to Andrew Morton
Quoted 54 lines · Andrew Morton
Quoted 42 lines · On Tue, 02 Feb 2010 16:58:38 -0600 James Bottomley <[email protected]>…
Quoted 41 lines · On Tue, 2010-02-02 at 14:11 -0800, [email protected] wrote:

From: Catalin Marinas <[email protected]>

Depending on the direction of the transfer, flush_dcache_page() must be called either before (ATA_TFLAG_WRITE) or after (!ATA_TFLAG_WRITE) the data copying to avoid D-cache aliasing with user space or I-D cache coherency issues (when reading data from an ATA device using PIO, the kernel dirties the D-cache but there is no flush_dcache_page() required on Harvard architectures).

This patch allows the ARM boards to use a rootfs on CompactFlash with the PATA platform driver.

As Anfei Zhou mentioned in a recent patch ("flush dcache before writing into page to avoid alias"), on some architectures there may be a performance benefit in differentiating the flush_dcache_page() calls based on whether the kernel or the user page needs flushing.

IMHO, we should differentiate based on the direction (kernel reading or writing from/to such page). In the ARM case with PIPT Harvard caches (newer processors), the kernel reading from a page that may be mapped in user space shouldn't need cache flushing. The kernel writing to such page would require D-cache flushing because of coherency with the I-cache. Currently on ARM, the latter happens in both cases.

---

 drivers/ata/libata-sff.c |    6 ++++++
 1 file changed, 6 insertions(+)

diff -puN drivers/ata/libata-sff.c~ata-call-flush_dcache_page-around-pio-data-transfers-in-libata-affc drivers/ata/libata-sff.c

drivers/ata/libata-sff.c renamed from drivers/ata/libata-sff.c~ata-call-flush_dcache_page-around-pio-data-transfers-in-libata-affc +0 -0
@@ -874,6 +874,9 @@ static void ata_pio_sector(struct ata_qu
 
 	DPRINTK("data %s\n", qc->tf.flags & ATA_TFLAG_WRITE ? "write" : "read");
 
Quoted 8 lines

drivers/ata/libata-sff.c static void ata_pio_sector(struct ata_qu · line 879

Quoted 3 lines
drivers/ata/libata-sff.c renamed from drivers/ata/libata-sff.c~ata-call-flush_dcache_page-around-pio-data-transfers-in-libata-affc +3 -0
+	if (do_write)
+		flush_dcache_page(page);
+

This looks wrong; the upper layers should already have made the page aliases coherent from user to kernel by calling flush_dcache_page (in __get_user_pages()), so the aliases should already be up to date and this flush is spurious.

Quoted 1 line

The upper layers don't know that the CPU touched the data!

The upper layers know to flush the user aliases.

Quoted 3 lines

If the
driver did a DMA transfer then such a flush is unneeded, so we don't do it.

The point is that this is a write case (transfer from kernel to device) so the underlying data should remain stable. Once __get_user_pages() has cohered the aliases, nothing should decohere them because nothing should be altering the data in the page.

The danger case for VI caches is the read case because there you are altering data via the kernel aliases.

James

#4 lore raw
In reply to Jeff Garzik
Quoted 55 lines · Jeff Garzik
Quoted 54 lines · Andrew Morton
Quoted 50 lines · On Tue, 02 Feb 2010 16:58:38 -0600 James Bottomley<[email protected]>…
Quoted 38 lines · On Tue, 2010-02-02 at 14:11 -0800, [email protected] wrote:

From: Catalin Marinas<[email protected]>

Depending on the direction of the transfer, flush_dcache_page() must be called either before (ATA_TFLAG_WRITE) or after (!ATA_TFLAG_WRITE) the data copying to avoid D-cache aliasing with user space or I-D cache coherency issues (when reading data from an ATA device using PIO, the kernel dirties the D-cache but there is no flush_dcache_page() required on Harvard architectures).

This patch allows the ARM boards to use a rootfs on CompactFlash with the PATA platform driver.

As Anfei Zhou mentioned in a recent patch ("flush dcache before writing into page to avoid alias"), on some architectures there may be a performance benefit in differentiating the flush_dcache_page() calls based on whether the kernel or the user page needs flushing.

IMHO, we should differentiate based on the direction (kernel reading or writing from/to such page). In the ARM case with PIPT Harvard caches (newer processors), the kernel reading from a page that may be mapped in user space shouldn't need cache flushing. The kernel writing to such page would require D-cache flushing because of coherency with the I-cache. Currently on ARM, the latter happens in both cases.

---

  drivers/ata/libata-sff.c |    6 ++++++
  1 file changed, 6 insertions(+)

diff -puN drivers/ata/libata-sff.c~ata-call-flush_dcache_page-around-pio-data-transfers-in-libata-affc drivers/ata/libata-sff.c

drivers/ata/libata-sff.c static void ata_pio_sector(struct ata_qu · line 879

Quoted 6 lines
drivers/ata/libata-sff.c renamed from drivers/ata/libata-sff.c~ata-call-flush_dcache_page-around-pio-data-transfers-in-libata-affc +3 -0
@@ -874,6 +874,9 @@ static void ata_pio_sector(struct ata_qu
 
  	DPRINTK("data %s\n", qc->tf.flags&  ATA_TFLAG_WRITE ? "write" : "read");
 
+	if (do_write)
+		flush_dcache_page(page);
+

This looks wrong; the upper layers should already have made the page aliases coherent from user to kernel by calling flush_dcache_page (in __get_user_pages()), so the aliases should already be up to date and this flush is spurious.

Quoted 8 lines
Quoted 3 lines

The upper layers don't know that the CPU touched the data! If the driver did a DMA transfer then such a flush is unneeded, so we don't do it.

The patch in question only affects PIO transfers, not DMA. Data is transferred from a kernel buffer to hardware via out[bwl] via

	page data -> CPU register -> out[bwl]

No flush required here ... all the aliases should be up to date after __get_user_pages()

Quoted 3 lines

or, data is transferred from hardware to a kernel buffer via

	in[bwl] -> CPU register -> page data

This writes to the page via the kernel alias, so the kernel alias becomes dirty. In order to make the underlying page up to date, the kernel alias must be flushed this flush *must* occur while the page is mapped along the kernel alias. In order that the user aliases see the correct data, any speculation they may have done before the kernel alias was flushed must be invalidated (depending on architectures ... some architectures won't speculate reads in this case). This invalidation *should* be occurring further up the stack .. but if the architecture has implemented this invalidation in the dma ops, then flush_dcache_page() may be needed.

Quoted 1 line

So what are the flushing rules given those conditions?

See above.

James

#5 lore raw
In reply to Jeff Garzik

From: Jeff Garzik <[email protected]>
Date: Tue, 02 Feb 2010 18:14:40 -0500

Quoted 10 lines

The patch in question only affects PIO transfers, not DMA. Data is transferred from a kernel buffer to hardware via out[bwl] via

	page data -> CPU register -> out[bwl]

or, data is transferred from hardware to a kernel buffer via

	in[bwl] -> CPU register -> page data

So what are the flushing rules given those conditions?

Any time you touch a page cache page with the cpu, you have to get it purged from aliasing caches before it gets mapped into userspace or similar.

That's the problem on these machines.

Actually, sparc64 is probably susceptible to the same problem.

And it's not an issue in the IDE layer, know why? :-)

In the IDE layer we have arch specific ide_*() interfaces which is where all of this PIO dma flushing stuff deserves to live.

ATA should do something similar instead of randomly scattering flush_dcache_page() calls all over the place.

#6 lore raw
Quoted 22 lines · David Miller

From: Jeff Garzik <[email protected]>
Date: Tue, 02 Feb 2010 18:14:40 -0500

Quoted 10 lines

The patch in question only affects PIO transfers, not DMA. Data is transferred from a kernel buffer to hardware via out[bwl] via

	page data -> CPU register -> out[bwl]

or, data is transferred from hardware to a kernel buffer via

	in[bwl] -> CPU register -> page data

So what are the flushing rules given those conditions?

Any time you touch a page cache page with the cpu, you have to get it purged from aliasing caches before it gets mapped into userspace or similar.

That's the problem on these machines.

Actually, sparc64 is probably susceptible to the same problem.

Quoted 7 lines

And it's not an issue in the IDE layer, know why? :-)

In the IDE layer we have arch specific ide_*() interfaces which is where all of this PIO dma flushing stuff deserves to live.

ATA should do something similar instead of randomly scattering flush_dcache_page() calls all over the place.

Agree with this, except it shouldn't be ide specific ... any driver that does PIO (we have some examples in SCSI) is vulnerable to this issue and should have the same fix.

The ide interfaces are very low level .. they're the ide_in/out[sw] APIs which aren't necessarily the most useful (some of the SCSI "PIO" stuff actually gets fed in via memory transfers to registers rather than port in/out).

What it looks like we need is kmap_for_pio() and kunmap_for_pio() which take flags (or API extensions) indicating whether we're transferring to or from the device, which return the correct kernel address to the page (mapping it if it's in highmem) and perform the correct flushes/invalidates.

James

#28 lore raw
In reply to David Miller
Quoted 2 lines

ATA should do something similar instead of randomly scattering flush_dcache_page() calls all over the place.

ATA has an sff_data_xfer callback for standard style interfaces for exactly the same reason that IDE did. That should be the only place that needs touching for most controllers (either the callback point or in the callback).

The non-standard ones that do bits of PIO will need fixing in place because they are sufficiently weird they don't fit any standard format. Fortunately they are almost entirely platform specific and/or DMA.

Alan

#7 lore raw
In reply to James Bottomley

From: James Bottomley <[email protected]>
Date: Tue, 02 Feb 2010 17:32:48 -0600

Quoted 5 lines

What it looks like we need is kmap_for_pio() and kunmap_for_pio() which take flags (or API extensions) indicating whether we're transferring to or from the device, which return the correct kernel address to the page (mapping it if it's in highmem) and perform the correct flushes/invalidates.

That sounds like a good idea.

#30 lore raw
In reply to James Bottomley
Quoted 24 lines · James Bottomley
Quoted 23 lines · On Tue, 2010-02-02 at 14:11 -0800, [email protected] wrote:

From: Catalin Marinas <[email protected]>

Depending on the direction of the transfer, flush_dcache_page() must be called either before (ATA_TFLAG_WRITE) or after (!ATA_TFLAG_WRITE) the data copying to avoid D-cache aliasing with user space or I-D cache coherency issues (when reading data from an ATA device using PIO, the kernel dirties the D-cache but there is no flush_dcache_page() required on Harvard architectures).

This patch allows the ARM boards to use a rootfs on CompactFlash with the PATA platform driver.

As Anfei Zhou mentioned in a recent patch ("flush dcache before writing into page to avoid alias"), on some architectures there may be a performance benefit in differentiating the flush_dcache_page() calls based on whether the kernel or the user page needs flushing.

IMHO, we should differentiate based on the direction (kernel reading or writing from/to such page). In the ARM case with PIPT Harvard caches (newer processors), the kernel reading from a page that may be mapped in user space shouldn't need cache flushing. The kernel writing to such page would require D-cache flushing because of coherency with the I-cache. Currently on ARM, the latter happens in both cases.

[...]

Quoted 7 lines
Quoted 7 lines

diff -puN drivers/ata/libata-sff.c~ata-call-flush_dcache_page-around-pio-data-transfers-in-libata-affc drivers/ata/libata-sff.c

drivers/ata/libata-sff.c renamed from drivers/ata/libata-sff.c~ata-call-flush_dcache_page-around-pio-data-transfers-in-libata-affc +0 -0
@@ -874,6 +874,9 @@ static void ata_pio_sector(struct ata_qu
 
      DPRINTK("data %s\n", qc->tf.flags & ATA_TFLAG_WRITE ? "write" : "read");
 
Quoted 8 lines

drivers/ata/libata-sff.c static void ata_pio_sector(struct ata_qu · line 879

Quoted 3 lines
drivers/ata/libata-sff.c renamed from drivers/ata/libata-sff.c~ata-call-flush_dcache_page-around-pio-data-transfers-in-libata-affc +3 -0
+     if (do_write)
+             flush_dcache_page(page);
+

This looks wrong; the upper layers should already have made the page aliases coherent from user to kernel by calling flush_dcache_page (in __get_user_pages()), so the aliases should already be up to date and this flush is spurious.

In my case it was working fine even without this. I just added this for completeness following the cachetlb.txt document.

Quoted 13 lines

drivers/ata/libata-sff.c static void ata_pio_sector(struct ata_qu · line 901

Quoted 10 lines
drivers/ata/libata-sff.c renamed from drivers/ata/libata-sff.c~ata-call-flush_dcache_page-around-pio-data-transfers-in-libata-affc +3 -0
      if (PageHighMem(page)) {
              unsigned long flags;
 
@@ -893,6 +896,9 @@ static void ata_pio_sector(struct ata_qu
                                     do_write);
      }
 
+     if (!do_write)
+             flush_dcache_page(page);
+

OK, so this too looks wrong for two reasons

Quoted 6 lines

1. it's over flushing. Even after the write to the page by the kernel PIO, the only alias that is dirty should be the kernel, so this needs a flush_kernel_dcache_page() to empty the kernel alias. It is possible user space will have speculated over the user aliases, but there's stuff further up the block stack to bring this back into coherence.

In the ARM case, it's not necessarily the user D-cache aliasing (which on my current hardware does not exist) but the I and D cache coherency.

With non-aliasing D-cache on ARM, the flush_kernel_dcache_page() is a no-op. If we consider the I-cache as being an alias, this function would need to be implemented. So far this issue has been handled by flush_dcache_page() (lazily).

Quoted 4 lines

2. If the page really is in highmem, the flush has to happen along the kernel alias, which you just lost because this flush is happening after the kunmap_atomic(), so it has to occur somewhere between the PIO operation and the kunmap.

The kunmap_atomic(), at least in the ARM case, seems to do the cache flushing. My cache coherency issues show up with highmem disabled.

Signature · 1 line
Catalin
#8 lore raw
In reply to James Bottomley
Quoted 40 lines · James Bottomley
Quoted 22 lines · David Miller

From: Jeff Garzik <[email protected]>
Date: Tue, 02 Feb 2010 18:14:40 -0500

Quoted 10 lines

The patch in question only affects PIO transfers, not DMA. Data is transferred from a kernel buffer to hardware via out[bwl] via

    page data -> CPU register -> out[bwl]

or, data is transferred from hardware to a kernel buffer via

    in[bwl] -> CPU register -> page data

So what are the flushing rules given those conditions?

Any time you touch a page cache page with the cpu, you have to get it purged from aliasing caches before it gets mapped into userspace or similar.

That's the problem on these machines.

Actually, sparc64 is probably susceptible to the same problem.

Quoted 7 lines

And it's not an issue in the IDE layer, know why? :-)

In the IDE layer we have arch specific ide_*() interfaces which is where all of this PIO dma flushing stuff deserves to live.

ATA should do something similar instead of randomly scattering flush_dcache_page() calls all over the place.

Agree with this, except it shouldn't be ide specific ... any driver that does PIO (we have some examples in SCSI) is vulnerable to this issue and should have the same fix.

The ide interfaces are very low level .. they're the ide_in/out[sw] APIs which aren't necessarily the most useful (some of the SCSI "PIO" stuff actually gets fed in via memory transfers to registers rather than port in/out).

Quoted 5 lines

What it looks like we need is kmap_for_pio() and kunmap_for_pio() which take flags (or API extensions) indicating whether we're transferring to or from the device, which return the correct kernel address to the page (mapping it if it's in highmem) and perform the correct flushes/invalidates.

In a similar patch for dealing with a PIO USB HCD driver (ISP1760) I suggested something like pio_map_single/pio_unmap_single similar to the dma_* API. The problem with the HCD drivers is that the page information is lost when they are called. They only have access to a urb->transfer_buffer pointer and you can never know whether it is a highmem one or not. Hence I think an API that deals directly with pointers rather than page structures would be better.

If you are OK with the idea, I'm happy to propose a patch on linux-arch.

Thanks for the feedback so far.

Signature · 1 line
Catalin
#9 lore raw
Quoted 47 lines · Catalin Marinas
Quoted 40 lines · James Bottomley
Quoted 22 lines · David Miller

From: Jeff Garzik <[email protected]>
Date: Tue, 02 Feb 2010 18:14:40 -0500

Quoted 10 lines

The patch in question only affects PIO transfers, not DMA. Data is transferred from a kernel buffer to hardware via out[bwl] via

    page data -> CPU register -> out[bwl]

or, data is transferred from hardware to a kernel buffer via

    in[bwl] -> CPU register -> page data

So what are the flushing rules given those conditions?

Any time you touch a page cache page with the cpu, you have to get it purged from aliasing caches before it gets mapped into userspace or similar.

That's the problem on these machines.

Actually, sparc64 is probably susceptible to the same problem.

Quoted 7 lines

And it's not an issue in the IDE layer, know why? :-)

In the IDE layer we have arch specific ide_*() interfaces which is where all of this PIO dma flushing stuff deserves to live.

ATA should do something similar instead of randomly scattering flush_dcache_page() calls all over the place.

Agree with this, except it shouldn't be ide specific ... any driver that does PIO (we have some examples in SCSI) is vulnerable to this issue and should have the same fix.

The ide interfaces are very low level .. they're the ide_in/out[sw] APIs which aren't necessarily the most useful (some of the SCSI "PIO" stuff actually gets fed in via memory transfers to registers rather than port in/out).

Quoted 5 lines

What it looks like we need is kmap_for_pio() and kunmap_for_pio() which take flags (or API extensions) indicating whether we're transferring to or from the device, which return the correct kernel address to the page (mapping it if it's in highmem) and perform the correct flushes/invalidates.

Quoted 7 lines

In a similar patch for dealing with a PIO USB HCD driver (ISP1760) I suggested something like pio_map_single/pio_unmap_single similar to the dma_* API. The problem with the HCD drivers is that the page information is lost when they are called. They only have access to a urb->transfer_buffer pointer and you can never know whether it is a highmem one or not. Hence I think an API that deals directly with pointers rather than page structures would be better.

Actually, thinking about the semantics, the normal kmap has to have these flushes in the map and unmap anyway ... otherwise the page gets decohered if you do normal read/write on it. The only information we don't have in current kmap is whether we're mapping for read/write or both. So the API I'd propose is keep current kmap as is for the read and write case, and introduce a kmap_for_read and kmap_for_write (with corresponding umaps, and obviously atomics).

The fix to libata looks to be just that it should kmap all the time rather than trying to fiddle with the page to see if its higmem. For kmap on a normal page, we should just return the offset map address and do all the flushing.

James

Quoted 4 lines

If you are OK with the idea, I'm happy to propose a patch on linux-arch.

Thanks for the feedback so far.

#10 lore raw
Quoted 4 lines · James Bottomley

The fix to libata looks to be just that it should kmap all the time rather than trying to fiddle with the page to see if its higmem. For kmap on a normal page, we should just return the offset map address and do all the flushing.

libata tests PageHighMem() because it was measurably faster to do things the current way (which includes local_irq_save/restore, only for highmem) on boxes where it actually matters.

It seems more efficient to add a flush where necessary, than unconditionally punish everyone...

Jeff

#11 lore raw
Quoted 6 lines · On Wed, 03 Feb 2010 12:00:58 -0500 Jeff Garzik <[email protected]> wrote:
Quoted 4 lines · James Bottomley

The fix to libata looks to be just that it should kmap all the time rather than trying to fiddle with the page to see if its higmem. For kmap on a normal page, we should just return the offset map address and do all the flushing.

Quoted 6 lines

libata tests PageHighMem() because it was measurably faster to do things the current way (which includes local_irq_save/restore, only for highmem) on boxes where it actually matters.

It seems more efficient to add a flush where necessary, than unconditionally punish everyone...

kmap_atomic() tests PageHighMem() too - it's pretty lightweight for lowmem pages.

Anyway, I'd draw your attention to this claim in the changelog: "This patch allows the ARM boards to use a rootfs on CompactFlash with the PATA platform driver." Immediate-term, we should be looking for a small fix for this issue which is acceptable for 2.6.33 and 2.6.32 and earlier.

#27 lore raw
In reply to James Bottomley

From: James Bottomley <[email protected]>
Date: Wed, 03 Feb 2010 10:40:54 -0600

Quoted 7 lines

Actually, thinking about the semantics, the normal kmap has to have these flushes in the map and unmap anyway ... otherwise the page gets decohered if you do normal read/write on it. The only information we don't have in current kmap is whether we're mapping for read/write or both. So the API I'd propose is keep current kmap as is for the read and write case, and introduce a kmap_for_read and kmap_for_write (with corresponding umaps, and obviously atomics).

This would basically force non-highmem platforms to implement the kmap_*() interfaces.

I understand that MIPS does this already to deal with cache aliasing.

But that's currently a choice, and sparc64 for example has no need to do things that way. It would in fact be more expensive than how sparc64 currently handles cache aliasing.

So I'm more in support of a new interface. Platforms like sparc64 can implement it and then get rid of the ide_*() string op cache flushes and instead we put the new API calls in the right spots.

#12 lore raw
In reply to Andrew Morton
Quoted 12 lines · Andrew Morton
Quoted 10 lines · On Wed, 03 Feb 2010 12:00:58 -0500 Jeff Garzik<[email protected]> wrote:
Quoted 4 lines · James Bottomley

The fix to libata looks to be just that it should kmap all the time rather than trying to fiddle with the page to see if its higmem. For kmap on a normal page, we should just return the offset map address and do all the flushing.

libata tests PageHighMem() because it was measurably faster to do things the current way (which includes local_irq_save/restore, only for highmem) on boxes where it actually matters.

Quoted 5 lines
Quoted 2 lines

It seems more efficient to add a flush where necessary, than unconditionally punish everyone...

kmap_atomic() tests PageHighMem() too - it's pretty lightweight for lowmem pages.

Note the lack of local_irq_save/restore in our code, though... These PIO xfers are __slow__, from the perspective of a CPU manufactured in the past decade; you are definitely disabling local interrupts for a long time. I suppose we could do

if (high mem)
local irq save
kmap
xfer
kunmap
local irq restore
else
kmap
xfer
kunmap

does that solve the problem for ARM, for 2.6.33?

Quoted 4 lines

Anyway, I'd draw your attention to this claim in the changelog: "This patch allows the ARM boards to use a rootfs on CompactFlash with the PATA platform driver." Immediate-term, we should be looking for a small fix for this issue which is acceptable for 2.6.33 and 2.6.32 and earlier.

Sure... see above. hopefully one that does not punish -everybody- though. It would be sad to unconditionally slow down millions of volume platform (read: x86) users for some embedded boards.

Jeff

#13 lore raw
Quoted 35 lines · On Wed, 03 Feb 2010 12:15:46 -0500 Jeff Garzik <[email protected]> wrote:
Quoted 12 lines · Andrew Morton
Quoted 10 lines · On Wed, 03 Feb 2010 12:00:58 -0500 Jeff Garzik<[email protected]> wrote:
Quoted 4 lines · James Bottomley

The fix to libata looks to be just that it should kmap all the time rather than trying to fiddle with the page to see if its higmem. For kmap on a normal page, we should just return the offset map address and do all the flushing.

libata tests PageHighMem() because it was measurably faster to do things the current way (which includes local_irq_save/restore, only for highmem) on boxes where it actually matters.

Quoted 5 lines
Quoted 2 lines

It seems more efficient to add a flush where necessary, than unconditionally punish everyone...

kmap_atomic() tests PageHighMem() too - it's pretty lightweight for lowmem pages.

Note the lack of local_irq_save/restore in our code, though... These PIO xfers are __slow__, from the perspective of a CPU manufactured in the past decade; you are definitely disabling local interrupts for a long time. I suppose we could do

if (high mem)
local irq save
kmap
xfer
kunmap
local irq restore
else
kmap
xfer
kunmap

Quoted 2 lines

does that solve the problem for ARM, for 2.6.33?

It's unclear (to me) why that code is using KM_IRQ0 at all. Can't it use a non-irq kmap slot?

Quoted 8 lines
Quoted 4 lines

Anyway, I'd draw your attention to this claim in the changelog: "This patch allows the ARM boards to use a rootfs on CompactFlash with the PATA platform driver." Immediate-term, we should be looking for a small fix for this issue which is acceptable for 2.6.33 and 2.6.32 and earlier.

Sure... see above. hopefully one that does not punish -everybody- though. It would be sad to unconditionally slow down millions of volume platform (read: x86) users for some embedded boards.

Well. ata-call-flush_dcache_page-around-pio-data-transfers-in-libata-affc.patch is a no-op on x86. It only has an effect on architectures which implement flush_dcache_page(). And I expect flush_dcache_page() is pretty light on those architectures, when compared with a PIO-mode transfer.

#14 lore raw
Quoted 37 lines · Andrew Morton
Quoted 35 lines · On Wed, 03 Feb 2010 12:15:46 -0500 Jeff Garzik<[email protected]> wrote:
Quoted 12 lines · Andrew Morton
Quoted 10 lines · On Wed, 03 Feb 2010 12:00:58 -0500 Jeff Garzik<[email protected]> wrote:
Quoted 4 lines · James Bottomley

The fix to libata looks to be just that it should kmap all the time rather than trying to fiddle with the page to see if its higmem. For kmap on a normal page, we should just return the offset map address and do all the flushing.

libata tests PageHighMem() because it was measurably faster to do things the current way (which includes local_irq_save/restore, only for highmem) on boxes where it actually matters.

Quoted 5 lines
Quoted 2 lines

It seems more efficient to add a flush where necessary, than unconditionally punish everyone...

kmap_atomic() tests PageHighMem() too - it's pretty lightweight for lowmem pages.

Note the lack of local_irq_save/restore in our code, though... These PIO xfers are __slow__, from the perspective of a CPU manufactured in the past decade; you are definitely disabling local interrupts for a long time. I suppose we could do

if (high mem)
local irq save
kmap
xfer
kunmap
local irq restore
else
kmap
xfer
kunmap

Quoted 5 lines
Quoted 2 lines

does that solve the problem for ARM, for 2.6.33?

It's unclear (to me) why that code is using KM_IRQ0 at all. Can't it use a non-irq kmap slot?

libata may have to transfer data in response to an interrupt. That is normal interrupt-driven PIO -- although it should be noted that the code supports polled PIO as well.

Quoted 9 lines
Quoted 8 lines
Quoted 4 lines

Anyway, I'd draw your attention to this claim in the changelog: "This patch allows the ARM boards to use a rootfs on CompactFlash with the PATA platform driver." Immediate-term, we should be looking for a small fix for this issue which is acceptable for 2.6.33 and 2.6.32 and earlier.

Sure... see above. hopefully one that does not punish -everybody- though. It would be sad to unconditionally slow down millions of volume platform (read: x86) users for some embedded boards.

Quoted 6 lines

Well. ata-call-flush_dcache_page-around-pio-data-transfers-in-libata-affc.patch is a no-op on x86. It only has an effect on architectures which implement flush_dcache_page(). And I expect flush_dcache_page() is pretty light on those architectures, when compared with a PIO-mode transfer.

I don't object to the patch... as long as the arch people are happy. Arch people seem to be the ones complaining, though.

Jeff

#15 lore raw
Quoted 65 lines · Jeff Garzik
Quoted 37 lines · Andrew Morton
Quoted 35 lines · On Wed, 03 Feb 2010 12:15:46 -0500 Jeff Garzik<[email protected]> wrote:
Quoted 12 lines · Andrew Morton
Quoted 10 lines · On Wed, 03 Feb 2010 12:00:58 -0500 Jeff Garzik<[email protected]> wrote:
Quoted 4 lines · James Bottomley

The fix to libata looks to be just that it should kmap all the time rather than trying to fiddle with the page to see if its higmem. For kmap on a normal page, we should just return the offset map address and do all the flushing.

libata tests PageHighMem() because it was measurably faster to do things the current way (which includes local_irq_save/restore, only for highmem) on boxes where it actually matters.

Quoted 5 lines
Quoted 2 lines

It seems more efficient to add a flush where necessary, than unconditionally punish everyone...

kmap_atomic() tests PageHighMem() too - it's pretty lightweight for lowmem pages.

Note the lack of local_irq_save/restore in our code, though... These PIO xfers are __slow__, from the perspective of a CPU manufactured in the past decade; you are definitely disabling local interrupts for a long time. I suppose we could do

if (high mem)
local irq save
kmap
xfer
kunmap
local irq restore
else
kmap
xfer
kunmap

Quoted 5 lines
Quoted 2 lines

does that solve the problem for ARM, for 2.6.33?

It's unclear (to me) why that code is using KM_IRQ0 at all. Can't it use a non-irq kmap slot?

libata may have to transfer data in response to an interrupt. That is normal interrupt-driven PIO -- although it should be noted that the code supports polled PIO as well.

Quoted 9 lines
Quoted 8 lines
Quoted 4 lines

Anyway, I'd draw your attention to this claim in the changelog: "This patch allows the ARM boards to use a rootfs on CompactFlash with the PATA platform driver." Immediate-term, we should be looking for a small fix for this issue which is acceptable for 2.6.33 and 2.6.32 and earlier.

Sure... see above. hopefully one that does not punish -everybody- though. It would be sad to unconditionally slow down millions of volume platform (read: x86) users for some embedded boards.

Quoted 6 lines

Well. ata-call-flush_dcache_page-around-pio-data-transfers-in-libata-affc.patch is a no-op on x86. It only has an effect on architectures which implement flush_dcache_page(). And I expect flush_dcache_page() is pretty light on those architectures, when compared with a PIO-mode transfer.

Quoted 2 lines

I don't object to the patch... as long as the arch people are happy. Arch people seem to be the ones complaining, though

Apart from the contents, which is looking like sprinkle mainline with random flushes, I'm unhappy that something which could be better implemented by thinking about the problem is now being rammed through as a must have bug fix.

We got this piece of crap in the same way:

commit 2d4dc890b5c8fabd818a8586607e6843c4375e62

Author: Ilya Loginov <[email protected]>
Date:   Thu Nov 26 09:16:19 2009 +0100

block: add helpers to run flush_dcache_page() against a bio and a
request's pages

Which is another race to flush everywhere until my coherence problem goes away.

This scattershot approach to coherency is really damaging in the long term because all these random flushes are going to mask real problems we may or may not have in the arch APIs ... and worse, they'll mask mostly ... there'll likely be times when the masking is incomplete and we're left with incredibly hard to debug data loss or binary crashes.

James

#20 lore raw
In reply to Jeff Garzik
Quoted 6 lines

if (high mem)
local irq save
kmap
xfer
kunmap
local irq restore

At the very least you need to do

irq save
kmap
copy to local buffer
kunmap
local irq restore
xfer

and the mirror image for read.

Alan

#22 lore raw
Quoted 10 lines · Alan Cox

And indeed there was a patch I proposed in 2008 for this bounce buffer latency: See the archive

Date: Fri, 29 Feb 2008 13:51:06 +0000
From: Alan Cox<[email protected]>
To: [email protected], [email protected]
Subject: [RFC PATCH] libata: PIO via bounce buffer

although it doesn't deal with the dcache coherency issue and seems to need a little tweaking to apply due to other changes

Yeah, I definitely liked the idea. I wonder if we could do the allocation during port_start rather than at the time of bounce? One maximally-sized buffer per device should not be too punishing on the system, IMO.

In any case, it is a nice approach to pursue.

Jeff

#26 lore raw
In reply to Catalin Marinas
Quoted 5 lines · Catalin Marinas

No, it doesn't. If the page isn't a highmem one, the kunmap doesn't do anything to the caches (and I also have highmem disabled). My logic was based on a statement in cachetlb.txt that flush_dcache_page() should be called if the kernel writes to a page cache page, hence the patch (as it was pointed out, the first call to flush_dcache_page() isn't needed).

Well, if those are the API rules, libata certainly must follow them...

Jeff

#25 lore raw
In reply to Jeff Garzik
Quoted 35 lines · Jeff Garzik
Quoted 12 lines · Andrew Morton
Quoted 10 lines · On Wed, 03 Feb 2010 12:00:58 -0500 Jeff Garzik<[email protected]> wrote:
Quoted 4 lines · James Bottomley

The fix to libata looks to be just that it should kmap all the time rather than trying to fiddle with the page to see if its higmem. For kmap on a normal page, we should just return the offset map address and do all the flushing.

libata tests PageHighMem() because it was measurably faster to do things the current way (which includes local_irq_save/restore, only for highmem) on boxes where it actually matters.

Quoted 5 lines
Quoted 2 lines

It seems more efficient to add a flush where necessary, than unconditionally punish everyone...

kmap_atomic() tests PageHighMem() too - it's pretty lightweight for lowmem pages.

Note the lack of local_irq_save/restore in our code, though... These PIO xfers are __slow__, from the perspective of a CPU manufactured in the past decade; you are definitely disabling local interrupts for a long time. I suppose we could do

if (high mem)
local irq save
kmap
xfer
kunmap
local irq restore
else
kmap
xfer
kunmap

Quoted 1 line

does that solve the problem for ARM, for 2.6.33?

No, it doesn't. If the page isn't a highmem one, the kunmap doesn't do anything to the caches (and I also have highmem disabled). My logic was based on a statement in cachetlb.txt that flush_dcache_page() should be called if the kernel writes to a page cache page, hence the patch (as it was pointed out, the first call to flush_dcache_page() isn't needed).

Signature · 1 line
Catalin
#23 lore raw
In reply to Jeff Garzik
Quoted 9 lines · On Wed, 03 Feb 2010 12:52:34 -0500 Jeff Garzik <[email protected]> wrote:
Quoted 8 lines · Alan Cox

And indeed there was a patch I proposed in 2008 for this bounce buffer latency: See the archive

Date: Fri, 29 Feb 2008 13:51:06 +0000
From: Alan Cox<[email protected]>
To: [email protected], [email protected]
Subject: [RFC PATCH] libata: PIO via bounce buffer

Quoted 5 lines
Quoted 2 lines

although it doesn't deal with the dcache coherency issue and seems to need a little tweaking to apply due to other changes

Yeah, I definitely liked the idea. I wonder if we could do the allocation during port_start rather than at the time of bounce?

I actually got better numbers using kmalloc - no idea why - perhaps we get a hot page ?

#24 lore raw
Quoted 12 lines · Alan Cox
Quoted 9 lines · On Wed, 03 Feb 2010 12:52:34 -0500 Jeff Garzik<[email protected]> wrote:
Quoted 8 lines · Alan Cox

And indeed there was a patch I proposed in 2008 for this bounce buffer latency: See the archive

Date: Fri, 29 Feb 2008 13:51:06 +0000
From: Alan Cox<[email protected]>
To: [email protected], [email protected]
Subject: [RFC PATCH] libata: PIO via bounce buffer

Quoted 8 lines
Quoted 5 lines
Quoted 2 lines

although it doesn't deal with the dcache coherency issue and seems to need a little tweaking to apply due to other changes

Yeah, I definitely liked the idea. I wonder if we could do the allocation during port_start rather than at the time of bounce?

I actually got better numbers using kmalloc - no idea why - perhaps we get a hot page ?

Seems logical. It surprises me, though, that being lockless and completely avoiding the kmalloc infrastructure wasn't a bigger win. Was this measured on UP?

If your approach has the lowest cost, let's get it in... I like decisions informed by hard data. :)

Jeff

#16 lore raw
In reply to James Bottomley
Quoted 37 lines · James Bottomley
Quoted 19 lines · Jeff Garzik
Quoted 11 lines · Andrew Morton
Quoted 9 lines · On Wed, 03 Feb 2010 12:15:46 -0500 Jeff Garzik<[email protected]> wrote:
Quoted 4 lines · Andrew Morton

Anyway, I'd draw your attention to this claim in the changelog: "This patch allows the ARM boards to use a rootfs on CompactFlash with the PATA platform driver." Immediate-term, we should be looking for a small fix for this issue which is acceptable for 2.6.33 and 2.6.32 and earlier.

Sure... see above. hopefully one that does not punish -everybody- though. It would be sad to unconditionally slow down millions of volume platform (read: x86) users for some embedded boards.

Quoted 6 lines

Well. ata-call-flush_dcache_page-around-pio-data-transfers-in-libata-affc.patch is a no-op on x86. It only has an effect on architectures which implement flush_dcache_page(). And I expect flush_dcache_page() is pretty light on those architectures, when compared with a PIO-mode transfer.

Quoted 2 lines

I don't object to the patch... as long as the arch people are happy. Arch people seem to be the ones complaining, though

Apart from the contents, which is looking like sprinkle mainline with random flushes, I'm unhappy that something which could be better implemented by thinking about the problem is now being rammed through as a must have bug fix.

We got this piece of crap in the same way:

commit 2d4dc890b5c8fabd818a8586607e6843c4375e62

Author: Ilya Loginov <[email protected]>
Date:   Thu Nov 26 09:16:19 2009 +0100

block: add helpers to run flush_dcache_page() against a bio and a
request's pages

Quoted 8 lines

Which is another race to flush everywhere until my coherence problem goes away.

This scattershot approach to coherency is really damaging in the long term because all these random flushes are going to mask real problems we may or may not have in the arch APIs ... and worse, they'll mask mostly ... there'll likely be times when the masking is incomplete and we're left with incredibly hard to debug data loss or binary crashes.

I agree that this could be solved in a better way and I'm in favour of API improvements. The current API and description in cachetlb.txt suggest that flush_dcache_page() is the best approach when modifying page cache pages.

AFAICT, there are two main use cases of flush_dcache_page() (could be more but that's what affects ARM): (1) filesystems modifying page cache pages (NFS, cramfs etc.) and (2) drivers doing PIO (block, HCD) that may write directly to page cache pages. Point (2) is a newer addition (as the one above) to fix coherency problems on Harvard architectures.

As long as you use filesystems like cramfs that call flush_dcache_page(), the PIO block driver doesn't need to do any flushing. That's one of the reasons why MTD didn't need any for a long time. Once you start using filesystems like ext2, there is no flush_dcache_page() called by the VFS layer, so we thought about moving this into the driver (for a long time I had a dirty hack in mpage_end_io_read() to call flush_dcache_page()).

If we state that flush_dcache_page() should not be used in driver code, than we need additional API for this (like pio_map_page etc. to be in line with the dma_map_* functions). This would allow architectures to implement them however they like.

I can go on linux-arch with a proposed patch if this sounds acceptable.

An alternative may be to force this flushing in kunmap(). Is it guaranteed that page cache pages are only accessed via kmap/kunmap? There are a few cases in the kernel where the code checks whether the page is a highmem one before using these functions.

Regards.

Signature · 1 line
Catalin
#17 lore raw
Quoted 66 lines · Catalin Marinas
Quoted 37 lines · James Bottomley
Quoted 19 lines · Jeff Garzik
Quoted 11 lines · Andrew Morton
Quoted 9 lines · On Wed, 03 Feb 2010 12:15:46 -0500 Jeff Garzik<[email protected]> wrote:
Quoted 4 lines · Andrew Morton

Anyway, I'd draw your attention to this claim in the changelog: "This patch allows the ARM boards to use a rootfs on CompactFlash with the PATA platform driver." Immediate-term, we should be looking for a small fix for this issue which is acceptable for 2.6.33 and 2.6.32 and earlier.

Sure... see above. hopefully one that does not punish -everybody- though. It would be sad to unconditionally slow down millions of volume platform (read: x86) users for some embedded boards.

Quoted 6 lines

Well. ata-call-flush_dcache_page-around-pio-data-transfers-in-libata-affc.patch is a no-op on x86. It only has an effect on architectures which implement flush_dcache_page(). And I expect flush_dcache_page() is pretty light on those architectures, when compared with a PIO-mode transfer.

Quoted 2 lines

I don't object to the patch... as long as the arch people are happy. Arch people seem to be the ones complaining, though

Apart from the contents, which is looking like sprinkle mainline with random flushes, I'm unhappy that something which could be better implemented by thinking about the problem is now being rammed through as a must have bug fix.

We got this piece of crap in the same way:

commit 2d4dc890b5c8fabd818a8586607e6843c4375e62

Author: Ilya Loginov <[email protected]>
Date:   Thu Nov 26 09:16:19 2009 +0100

block: add helpers to run flush_dcache_page() against a bio and a
request's pages

Quoted 8 lines

Which is another race to flush everywhere until my coherence problem goes away.

This scattershot approach to coherency is really damaging in the long term because all these random flushes are going to mask real problems we may or may not have in the arch APIs ... and worse, they'll mask mostly ... there'll likely be times when the masking is incomplete and we're left with incredibly hard to debug data loss or binary crashes.

I agree that this could be solved in a better way and I'm in favour of API improvements. The current API and description in cachetlb.txt suggest that flush_dcache_page() is the best approach when modifying page cache pages.

AFAICT, there are two main use cases of flush_dcache_page() (could be more but that's what affects ARM): (1) filesystems modifying page cache pages (NFS, cramfs etc.) and (2) drivers doing PIO (block, HCD) that may write directly to page cache pages. Point (2) is a newer addition (as the one above) to fix coherency problems on Harvard architectures.

As long as you use filesystems like cramfs that call flush_dcache_page(), the PIO block driver doesn't need to do any flushing. That's one of the reasons why MTD didn't need any for a long time. Once you start using filesystems like ext2, there is no flush_dcache_page() called by the VFS layer, so we thought about moving this into the driver (for a long time I had a dirty hack in mpage_end_io_read() to call flush_dcache_page()).

Quoted 6 lines

If we state that flush_dcache_page() should not be used in driver code, than we need additional API for this (like pio_map_page etc. to be in line with the dma_map_* functions). This would allow architectures to implement them however they like.

I can go on linux-arch with a proposed patch if this sounds acceptable.

I think I'd prefer some type of kmap_ variant. The reason is that all the semantics are going to be the same as kmap ... down to having the slots for atomic. All we need is the extra flag giving the direction of transfer and, of course, what you get back is a virtual address pointer (all the dma operations return physical address handles).

Quoted 4 lines

An alternative may be to force this flushing in kunmap(). Is it guaranteed that page cache pages are only accessed via kmap/kunmap? There are a few cases in the kernel where the code checks whether the page is a highmem one before using these functions.

So code which checks for higmem really shouldn't. However DaveM makes the valid point that this would be problematic on platforms that have no highmem. But an additional variant like kmap_pio which all platforms can intercept looks more acceptable.

James

#18 lore raw
Quoted 72 lines · James Bottomley
Quoted 71 lines · Catalin Marinas
Quoted 37 lines · James Bottomley
Quoted 19 lines · Jeff Garzik
Quoted 11 lines · Andrew Morton
Quoted 9 lines · On Wed, 03 Feb 2010 12:15:46 -0500 Jeff Garzik<[email protected]> wrote:
Quoted 4 lines · Andrew Morton

Anyway, I'd draw your attention to this claim in the changelog: "This patch allows the ARM boards to use a rootfs on CompactFlash with the PATA platform driver." Immediate-term, we should be looking for a small fix for this issue which is acceptable for 2.6.33 and 2.6.32 and earlier.

Sure... see above. hopefully one that does not punish -everybody- though. It would be sad to unconditionally slow down millions of volume platform (read: x86) users for some embedded boards.

Quoted 6 lines

Well. ata-call-flush_dcache_page-around-pio-data-transfers-in-libata-affc.patch is a no-op on x86. It only has an effect on architectures which implement flush_dcache_page(). And I expect flush_dcache_page() is pretty light on those architectures, when compared with a PIO-mode transfer.

Quoted 2 lines

I don't object to the patch... as long as the arch people are happy. Arch people seem to be the ones complaining, though

Apart from the contents, which is looking like sprinkle mainline with random flushes, I'm unhappy that something which could be better implemented by thinking about the problem is now being rammed through as a must have bug fix.

We got this piece of crap in the same way:

commit 2d4dc890b5c8fabd818a8586607e6843c4375e62

Author: Ilya Loginov <[email protected]>
Date:   Thu Nov 26 09:16:19 2009 +0100

block: add helpers to run flush_dcache_page() against a bio and a
request's pages

Quoted 8 lines

Which is another race to flush everywhere until my coherence problem goes away.

This scattershot approach to coherency is really damaging in the long term because all these random flushes are going to mask real problems we may or may not have in the arch APIs ... and worse, they'll mask mostly ... there'll likely be times when the masking is incomplete and we're left with incredibly hard to debug data loss or binary crashes.

I agree that this could be solved in a better way and I'm in favour of API improvements. The current API and description in cachetlb.txt suggest that flush_dcache_page() is the best approach when modifying page cache pages.

AFAICT, there are two main use cases of flush_dcache_page() (could be more but that's what affects ARM): (1) filesystems modifying page cache pages (NFS, cramfs etc.) and (2) drivers doing PIO (block, HCD) that may write directly to page cache pages. Point (2) is a newer addition (as the one above) to fix coherency problems on Harvard architectures.

As long as you use filesystems like cramfs that call flush_dcache_page(), the PIO block driver doesn't need to do any flushing. That's one of the reasons why MTD didn't need any for a long time. Once you start using filesystems like ext2, there is no flush_dcache_page() called by the VFS layer, so we thought about moving this into the driver (for a long time I had a dirty hack in mpage_end_io_read() to call flush_dcache_page()).

If we state that flush_dcache_page() should not be used in driver code, than we need additional API for this (like pio_map_page etc. to be in line with the dma_map_* functions). This would allow architectures to implement them however they like.

Quoted 7 lines
Quoted 1 line

I can go on linux-arch with a proposed patch if this sounds acceptable.

I think I'd prefer some type of kmap_ variant. The reason is that all the semantics are going to be the same as kmap ... down to having the slots for atomic. All we need is the extra flag giving the direction of transfer and, of course, what you get back is a virtual address pointer (all the dma operations return physical address handles).

Would a kmap_pio API always require a page structure as argument?

There is a similar situation for HCD drivers and USB mass storage which I raised recently. The idea that the USB mass storage code should do the cache flushing was rejected as this has to be pushed down to the HCD driver. However, the HCD driver doesn't get page information, only a urb->transfer_buffer pointer to which it may do either DMA or PIO (both may actually happen in the same driver as an optimisation).

If we enforce the use of a kmap_pio API in the USB mass storage, it wouldn't be optimal since this code is not aware of whether the HCD driver would do PIO or DMA.

Looking at the HCD drivers, there are more similarities to the DMA API like pio_map_page, pio_map_single etc. I personally don't care much about how the API is named but rather where such API should be called from.

Regards.

Signature · 1 line
Catalin
#19 lore raw
Quoted 75 lines · Catalin Marinas
Quoted 74 lines · James Bottomley
Quoted 72 lines · Catalin Marinas
Quoted 37 lines · James Bottomley
Quoted 19 lines · Jeff Garzik
Quoted 11 lines · Andrew Morton
Quoted 9 lines · On Wed, 03 Feb 2010 12:15:46 -0500 Jeff Garzik<[email protected]> wrote:
Quoted 4 lines · Andrew Morton

Anyway, I'd draw your attention to this claim in the changelog: "This patch allows the ARM boards to use a rootfs on CompactFlash with the PATA platform driver." Immediate-term, we should be looking for a small fix for this issue which is acceptable for 2.6.33 and 2.6.32 and earlier.

Sure... see above. hopefully one that does not punish -everybody- though. It would be sad to unconditionally slow down millions of volume platform (read: x86) users for some embedded boards.

Quoted 6 lines

Well. ata-call-flush_dcache_page-around-pio-data-transfers-in-libata-affc.patch is a no-op on x86. It only has an effect on architectures which implement flush_dcache_page(). And I expect flush_dcache_page() is pretty light on those architectures, when compared with a PIO-mode transfer.

Quoted 2 lines

I don't object to the patch... as long as the arch people are happy. Arch people seem to be the ones complaining, though

Apart from the contents, which is looking like sprinkle mainline with random flushes, I'm unhappy that something which could be better implemented by thinking about the problem is now being rammed through as a must have bug fix.

We got this piece of crap in the same way:

commit 2d4dc890b5c8fabd818a8586607e6843c4375e62

Author: Ilya Loginov <[email protected]>
Date:   Thu Nov 26 09:16:19 2009 +0100

block: add helpers to run flush_dcache_page() against a bio and a
request's pages

Quoted 8 lines

Which is another race to flush everywhere until my coherence problem goes away.

This scattershot approach to coherency is really damaging in the long term because all these random flushes are going to mask real problems we may or may not have in the arch APIs ... and worse, they'll mask mostly ... there'll likely be times when the masking is incomplete and we're left with incredibly hard to debug data loss or binary crashes.

I agree that this could be solved in a better way and I'm in favour of API improvements. The current API and description in cachetlb.txt suggest that flush_dcache_page() is the best approach when modifying page cache pages.

AFAICT, there are two main use cases of flush_dcache_page() (could be more but that's what affects ARM): (1) filesystems modifying page cache pages (NFS, cramfs etc.) and (2) drivers doing PIO (block, HCD) that may write directly to page cache pages. Point (2) is a newer addition (as the one above) to fix coherency problems on Harvard architectures.

As long as you use filesystems like cramfs that call flush_dcache_page(), the PIO block driver doesn't need to do any flushing. That's one of the reasons why MTD didn't need any for a long time. Once you start using filesystems like ext2, there is no flush_dcache_page() called by the VFS layer, so we thought about moving this into the driver (for a long time I had a dirty hack in mpage_end_io_read() to call flush_dcache_page()).

If we state that flush_dcache_page() should not be used in driver code, than we need additional API for this (like pio_map_page etc. to be in line with the dma_map_* functions). This would allow architectures to implement them however they like.

I can go on linux-arch with a proposed patch if this sounds acceptable.

Quoted 7 lines
Quoted 5 lines

I think I'd prefer some type of kmap_ variant. The reason is that all the semantics are going to be the same as kmap ... down to having the slots for atomic. All we need is the extra flag giving the direction of transfer and, of course, what you get back is a virtual address pointer (all the dma operations return physical address handles).

Would a kmap_pio API always require a page structure as argument?

This seems a reasonable requirement.

Quoted 6 lines

There is a similar situation for HCD drivers and USB mass storage which I raised recently. The idea that the USB mass storage code should do the cache flushing was rejected as this has to be pushed down to the HCD driver. However, the HCD driver doesn't get page information, only a urb->transfer_buffer pointer to which it may do either DMA or PIO (both may actually happen in the same driver as an optimisation).

You'd want to do dma_map or kmap at a point after the DMA or PIO decision is made.

Quoted 3 lines

If we enforce the use of a kmap_pio API in the USB mass storage, it wouldn't be optimal since this code is not aware of whether the HCD driver would do PIO or DMA.

So, if the decision isn't made until the HCD driver, then surely this belongs there.

Quoted 4 lines

Looking at the HCD drivers, there are more similarities to the DMA API like pio_map_page, pio_map_single etc. I personally don't care much about how the API is named but rather where such API should be called from.

the API usage is similar to the dma one. The point I was making is the semantics are similar to kmap because of the slot and atomic/non atomic requirements.

James