Skip to contentSkip to conversation

don't udelay() in sn_mmiob

PATCH In review Jesse Barnes 4 participants 8 messages May 26, 2004 – May 29, 2004

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 don't udelay() in sn_mmiob

sn_mmiob is a lightweight way to ensure PCI write ordering, intended to be used as an alternative to doing a PIO read. Unfortunately, with the udelay() in there, it ends up being slower than a PCI read on small configurations, so remove it.

Thanks,
Jesse

===== arch/ia64/sn/io/machvec/iomv.c 1.8 vs edited =====

arch/ia64/sn/io/machvec/iomv.c +1 -1
@@ -71,6 +71,6 @@
 {
 	while ((((volatile unsigned long) (*pda->pio_write_status_addr)) & SH_PIO_WRITE_STATUS_0_PENDING_WRITE_COUNT_MASK) != 
 				SH_PIO_WRITE_STATUS_0_PENDING_WRITE_COUNT_MASK)
-		udelay(1);
+		;
 }
 EXPORT_SYMBOL(sn_mmiob);
#1 lore raw
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line · Jesse Barnes

Jesse> sn_mmiob is a lightweight way to ensure PCI write ordering, Jesse> intended to be used as an alternative to doing a PIO read. Jesse> Unfortunately, with the udelay() in there, it ends up being Jesse> slower than a PCI read on small configurations, so remove it.

Wouldn't you want at least a cpu_relax() in that loop?

	--david
#2 lore raw
In reply to Jesse Barnes
Quoted 1 line · David Mosberger

Wouldn't you want at least a cpu_relax() in that loop?

That'll cause the CPU to switch to the other thread if it's SMT so it won't spin waiting for the access to complete? If so, then yes. Here's an updated patch.

sn_mmiob is a lightweight way to ensure PCI write ordering, intended to be used as an alternative to doing a PIO read. Unfortunately, with the udelay() in there, it ends up being slower than a PCI read on small configurations, so remove it in favor of a simple cpu_relax() so that we're HT-friendly.

Thanks,
Jesse

===== arch/ia64/sn/io/machvec/iomv.c 1.8 vs edited =====

arch/ia64/sn/io/machvec/iomv.c +1 -1
@@ -71,6 +71,6 @@
 {
 	while ((((volatile unsigned long) (*pda->pio_write_status_addr)) & SH_PIO_WRITE_STATUS_0_PENDING_WRITE_COUNT_MASK) != 
 				SH_PIO_WRITE_STATUS_0_PENDING_WRITE_COUNT_MASK)
-		udelay(1);
+		cpu_relax();
 }
 EXPORT_SYMBOL(sn_mmiob);
#3 lore raw
In reply to Jesse Barnes
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line · Jesse Barnes
Quoted 1 line · David Mosberger
Quoted 1 line

Wouldn't you want at least a cpu_relax() in that loop?

Jesse> That'll cause the CPU to switch to the other thread if it's Jesse> SMT so it won't spin waiting for the access to complete? If Jesse> so, then yes.

It is simply good practice to always use cpu_relax() in busy-loops. It'll make sure the compiler doesn't get too clever (thanks to the implied barrier()) and ensures that the CPU doesn't go nuts on the busy-loop (e.g., a Pentium 4 core might overheat). In the case of ia64, cpu_relax() results in a "hint @pause" instruction which is specified as:

"Indicates to the processor that the currently executing stream is
waiting, spinning, or doing low priority tasks. This hint can be used by the processor to allocate more resources or time to another executing stream on the same processor."

	--david
#4 lore raw
In reply to Jesse Barnes
Quoted 6 lines · Jesse Barnes
Quoted 1 line · David Mosberger

Wouldn't you want at least a cpu_relax() in that loop?

That'll cause the CPU to switch to the other thread if it's SMT so it won't spin waiting for the access to complete? If so, then yes. Here's an updated patch.

I didn't think the code would actually spin.

sn_mmiob() is something like:

    1:	ld.acq r8=...
    	;;
	cmp.eq p8,p0=r8,r9

(p8) br 1b

The "load" is an uncached load. Won't the pipeline stall on the cmp waiting for data to arrive. I would not have expected that a cpu_relax() would have been needed here.

Data takes a few hundred nsec to be loaded from the chipset.

If we were spinning on a CACHED memory address, I agree that cpu_relax() is need.

Quoted 8 lines

sn_mmiob is a lightweight way to ensure PCI write ordering, intended to be used as an alternative to doing a PIO read. Unfortunately, with the udelay() in there, it ends up being slower than a PCI read on small configurations, so remove it in favor of a simple cpu_relax() so that we're HT-friendly.

Thanks,
Jesse

Signature · 4 lines
Thanks

Jack Steiner ([email protected])          651-683-5302
Principal Engineer                      SGI - Silicon Graphics, Inc.
#5 lore raw
In reply to Jesse Barnes
Quoted 7 lines · Jack Steiner
Quoted 6 lines · Jesse Barnes
Quoted 1 line · David Mosberger

Wouldn't you want at least a cpu_relax() in that loop?

That'll cause the CPU to switch to the other thread if it's SMT so it won't spin waiting for the access to complete? If so, then yes. Here's an updated patch.

Whoops, I posted this response to the wrong mail. I was replying to the question about whether the code should have a "cpu_relax()".

Quoted 44 lines

I didn't think the code would actually spin.

sn_mmiob() is something like:

    1:	ld.acq r8=...
    	;;
	cmp.eq p8,p0=r8,r9

(p8) br 1b

The "load" is an uncached load. Won't the pipeline stall on the cmp waiting for data to arrive. I would not have expected that a cpu_relax() would have been needed here.

Data takes a few hundred nsec to be loaded from the chipset.

If we were spinning on a CACHED memory address, I agree that cpu_relax() is need.

Quoted 8 lines

sn_mmiob is a lightweight way to ensure PCI write ordering, intended to be used as an alternative to doing a PIO read. Unfortunately, with the udelay() in there, it ends up being slower than a PCI read on small configurations, so remove it in favor of a simple cpu_relax() so that we're HT-friendly.

Thanks,
Jesse

Signature · 10 lines
Thanks

Jack Steiner ([email protected])          651-683-5302
Principal Engineer                      SGI - Silicon Graphics, Inc.


-
To unsubscribe from this list: send the line "unsubscribe linux-ia64" in
the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Signature · 4 lines
Thanks

Jack Steiner ([email protected])          651-683-5302
Principal Engineer                      SGI - Silicon Graphics, Inc.
#6 lore raw
In reply to Jesse Barnes

Jack Steiner <> wrote on Friday, May 28, 2004 10:35 AM:

Quoted 26 lines · Jack Steiner
Quoted 7 lines · Jesse Barnes
Quoted 6 lines · David Mosberger
Quoted 1 line

Wouldn't you want at least a cpu_relax() in that loop?

That'll cause the CPU to switch to the other thread if it's SMT so it won't spin waiting for the access to complete? If so, then yes. Here's an updated patch.

Whoops, I posted this response to the wrong mail. I was replying to the question about whether the code should have a "cpu_relax()".

Quoted 14 lines

I didn't think the code would actually spin.

sn_mmiob() is something like:

    1:	ld.acq r8=...
    	;;
	cmp.eq p8,p0=r8,r9

(p8) br 1b

The "load" is an uncached load. Won't the pipeline stall on the cmp waiting for data to arrive. I would not have expected that a cpu_relax() would have been needed here.

You are right that pipeline will stall on the cmp instruction. But if you could put the hint@pause between ld and cmp instructions then that would allow current execution stream to give up resources to other execution stream (if applicable) for the time that data takes to come from main memory.

-rohit

#7 lore raw
In reply to Jesse Barnes
Quoted 29 lines · Rohit

Jack Steiner <> wrote on Friday, May 28, 2004 10:35 AM:

Quoted 26 lines · Jack Steiner
Quoted 7 lines · Jesse Barnes
Quoted 6 lines · David Mosberger
Quoted 1 line

Wouldn't you want at least a cpu_relax() in that loop?

That'll cause the CPU to switch to the other thread if it's SMT so it won't spin waiting for the access to complete? If so, then yes. Here's an updated patch.

Whoops, I posted this response to the wrong mail. I was replying to the question about whether the code should have a "cpu_relax()".

Quoted 14 lines

I didn't think the code would actually spin.

sn_mmiob() is something like:

    1:	ld.acq r8=...
    	;;
	cmp.eq p8,p0=r8,r9

(p8) br 1b

The "load" is an uncached load. Won't the pipeline stall on the cmp waiting for data to arrive. I would not have expected that a cpu_relax() would have been needed here.

Quoted 7 lines

You are right that pipeline will stall on the cmp instruction. But if you could put the hint@pause between ld and cmp instructions then that would allow current execution stream to give up resources to other execution stream (if applicable) for the time that data takes to come from main memory.

-rohit

I thought the cpu would automatically switch on a cache miss & I assumed that a switch would occur on a UC ref. Or am I thinking of a different processor?

If you follow the logic that is cpu_relax is useful in the sn_mmiob() function, shouldn't there be a cpu_relax() after EVERY uncached load that is immediately (or closely) followed by an instruction that consumes the data.

What are the conditions that cause a switch between execution streams?

Signature · 4 lines
Thanks

Jack Steiner ([email protected])          651-683-5302
Principal Engineer                      SGI - Silicon Graphics, Inc.