Skip to contentSkip to conversation

MTD: Fix Orion NAND driver compilation with ARM OABI

PATCH In review Paulius Zaleckas 5 participants 14 messages Mar 25, 2010 – Apr 24, 2010

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

PATCH v2 1/1 MTD: Fix Orion NAND driver compilation with ARM OABI

We must tell GCC to use even register for variable passed to ldrd instruction. Without this patch GCC 4.2.1 puts this variable to r2/r3 on EABI and r3/r4 on OABI, so force it to r2/r3. This does not change anything when EABI and OABI compilation works OK.

Without this patch and with OABI I get:
CC drivers/mtd/nand/orion_nand.o
/tmp/ccMkwOCs.s: Assembler messages:
/tmp/ccMkwOCs.s:63: Error: first destination register must be even -- `ldrd r3,[ip]' make[5]: *** [drivers/mtd/nand/orion_nand.o] Error 1

---

 drivers/mtd/nand/orion_nand.c |    8 +++++++-
 1 files changed, 7 insertions(+), 1 deletions(-)
drivers/mtd/nand/orion_nand.c +7 -1
index f59c074..d60fc57 100644
@@ -60,7 +60,13 @@ static void orion_nand_read_buf(struct mtd_info *mtd, uint8_t *buf, int len)
 	}
 	buf64 = (uint64_t *)buf;
 	while (i < len/8) {
-		uint64_t x;
+		/*
+		 * Since GCC has no proper constraint (PR 43518)
+		 * force x variable to r2/r3 registers as ldrd instruction
+		 * requires first register to be even.
+		 */
+		register uint64_t x asm ("r2");
+
 		asm volatile ("ldrd\t%0, [%1]" : "=&r" (x) : "r" (io_base));
 		buf64[i++] = x;
 	}
Acked-by on v2 1/1
#1 lore raw
Quoted 6 lines · Paulius Zaleckas

We must tell GCC to use even register for variable passed to ldrd instruction. Without this patch GCC 4.2.1 puts this variable to r2/r3 on EABI and r3/r4 on OABI, so force it to r2/r3. This does not change anything when EABI and OABI compilation works OK.

Quoted 7 lines

Without this patch and with OABI I get:
CC drivers/mtd/nand/orion_nand.o
/tmp/ccMkwOCs.s: Assembler messages:
/tmp/ccMkwOCs.s:63: Error: first destination register must be even -- `ldrd r3,[ip]' make[5]: *** [drivers/mtd/nand/orion_nand.o] Error 1

Quoted 25 lines

---

 drivers/mtd/nand/orion_nand.c |    8 +++++++-
 1 files changed, 7 insertions(+), 1 deletions(-)
drivers/mtd/nand/orion_nand.c +7 -1
index f59c074..d60fc57 100644
@@ -60,7 +60,13 @@ static void orion_nand_read_buf(struct mtd_info *mtd, uint8_t *buf, int len)
 	}
 	buf64 = (uint64_t *)buf;
 	while (i < len/8) {
-		uint64_t x;
+		/*
+		 * Since GCC has no proper constraint (PR 43518)
+		 * force x variable to r2/r3 registers as ldrd instruction
+		 * requires first register to be even.
+		 */
+		register uint64_t x asm ("r2");
+
 		asm volatile ("ldrd\t%0, [%1]" : "=&r" (x) : "r" (io_base));
 		buf64[i++] = x;
 	}
#2 lore raw
Quoted 14 lines · Nicolas Pitre
Quoted 12 lines · Paulius Zaleckas

We must tell GCC to use even register for variable passed to ldrd instruction. Without this patch GCC 4.2.1 puts this variable to r2/r3 on EABI and r3/r4 on OABI, so force it to r2/r3. This does not change anything when EABI and OABI compilation works OK.

Without this patch and with OABI I get:
CC drivers/mtd/nand/orion_nand.o
/tmp/ccMkwOCs.s: Assembler messages:
/tmp/ccMkwOCs.s:63: Error: first destination register must be even -- `ldrd r3,[ip]' make[5]: *** [drivers/mtd/nand/orion_nand.o] Error 1

Quoted 3 lines
Quoted 1 line

David,

Will you take this patch?
Or you are waiting till I will add all Cc as Jamie suggested?

Quoted 26 lines
Quoted 26 lines

---

  drivers/mtd/nand/orion_nand.c |    8 +++++++-
  1 files changed, 7 insertions(+), 1 deletions(-)
drivers/mtd/nand/orion_nand.c +7 -1
index f59c074..d60fc57 100644
@@ -60,7 +60,13 @@ static void orion_nand_read_buf(struct mtd_info *mtd, uint8_t *buf, int len)
  	}
  	buf64 = (uint64_t *)buf;
  	while (i<  len/8) {
-		uint64_t x;
+		/*
+		 * Since GCC has no proper constraint (PR 43518)
+		 * force x variable to r2/r3 registers as ldrd instruction
+		 * requires first register to be even.
+		 */
+		register uint64_t x asm ("r2");
+
  		asm volatile ("ldrd\t%0, [%1]" : "=&r" (x) : "r" (io_base));
  		buf64[i++] = x;
  	}
#3 lore raw
Quoted 15 lines · Paulius Zaleckas
Quoted 14 lines · Nicolas Pitre
Quoted 12 lines · Paulius Zaleckas

We must tell GCC to use even register for variable passed to ldrd instruction. Without this patch GCC 4.2.1 puts this variable to r2/r3 on EABI and r3/r4 on OABI, so force it to r2/r3. This does not change anything when EABI and OABI compilation works OK.

Without this patch and with OABI I get:
CC drivers/mtd/nand/orion_nand.o
/tmp/ccMkwOCs.s: Assembler messages:
/tmp/ccMkwOCs.s:63: Error: first destination register must be even -- `ldrd r3,[ip]' make[5]: *** [drivers/mtd/nand/orion_nand.o] Error 1

Quoted 8 lines
Quoted 3 lines
Quoted 1 line

David,

Will you take this patch?
Or you are waiting till I will add all Cc as Jamie suggested?

Meanwhile, I've pushed your patch to my l2-mtd-2.6.git / dunno.

Signature · 2 lines
Best Regards,
Artem Bityutskiy (????? ????????)
#4 lore raw
Quoted 18 lines · Artem Bityutskiy
Quoted 17 lines · Paulius Zaleckas
Quoted 16 lines · Nicolas Pitre
Quoted 13 lines · Paulius Zaleckas

We must tell GCC to use even register for variable passed to ldrd instruction. Without this patch GCC 4.2.1 puts this variable to r2/r3 on EABI and r3/r4 on OABI, so force it to r2/r3. This does not change anything when EABI and OABI compilation works OK.

Without this patch and with OABI I get:
CC drivers/mtd/nand/orion_nand.o
/tmp/ccMkwOCs.s: Assembler messages:
/tmp/ccMkwOCs.s:63: Error: first destination register must be even -- `ldrd r3,[ip]' make[5]: *** [drivers/mtd/nand/orion_nand.o] Error 1

Quoted 8 lines
Quoted 6 lines
Quoted 1 line

David,

Will you take this patch?
Or you are waiting till I will add all Cc as Jamie suggested?

Meanwhile, I've pushed your patch to my l2-mtd-2.6.git / dunno.

I think it should go to mainline. It is not perfect, but still better than the current situation.

Nicolas

#5 lore raw
Quoted 24 lines · Nicolas Pitre
Quoted 22 lines · Artem Bityutskiy
Quoted 21 lines · Paulius Zaleckas
Quoted 14 lines · Nicolas Pitre
Quoted 12 lines · Paulius Zaleckas

We must tell GCC to use even register for variable passed to ldrd instruction. Without this patch GCC 4.2.1 puts this variable to r2/r3 on EABI and r3/r4 on OABI, so force it to r2/r3. This does not change anything when EABI and OABI compilation works OK.

Without this patch and with OABI I get:
CC drivers/mtd/nand/orion_nand.o
/tmp/ccMkwOCs.s: Assembler messages:
/tmp/ccMkwOCs.s:63: Error: first destination register must be even -- `ldrd r3,[ip]' make[5]: *** [drivers/mtd/nand/orion_nand.o] Error 1

Quoted 3 lines
Quoted 1 line

David,

Quoted 7 lines
Quoted 4 lines
Quoted 2 lines

Will you take this patch?
Or you are waiting till I will add all Cc as Jamie suggested?

Meanwhile, I've pushed your patch to my l2-mtd-2.6.git / dunno.

I think it should go to mainline. It is not perfect, but still better than the current situation.

Fine with me, but not up to me. But I guess Andrew could merge it.

Signature · 2 lines
Best Regards,
Artem Bityutskiy (????? ????????)
#6 lore raw
Quoted 28 lines · Artem Bityutskiy
Quoted 27 lines · Nicolas Pitre
Quoted 25 lines · Artem Bityutskiy
Quoted 23 lines · Paulius Zaleckas
Quoted 14 lines · Nicolas Pitre
Quoted 12 lines · Paulius Zaleckas

We must tell GCC to use even register for variable passed to ldrd instruction. Without this patch GCC 4.2.1 puts this variable to r2/r3 on EABI and r3/r4 on OABI, so force it to r2/r3. This does not change anything when EABI and OABI compilation works OK.

Without this patch and with OABI I get:
CC drivers/mtd/nand/orion_nand.o
/tmp/ccMkwOCs.s: Assembler messages:
/tmp/ccMkwOCs.s:63: Error: first destination register must be even -- `ldrd r3,[ip]' make[5]: *** [drivers/mtd/nand/orion_nand.o] Error 1

Quoted 3 lines
Quoted 1 line

David,

Will you take this patch?
Or you are waiting till I will add all Cc as Jamie suggested?

Quoted 6 lines
Quoted 4 lines
Quoted 1 line

Meanwhile, I've pushed your patch to my l2-mtd-2.6.git / dunno.

I think it should go to mainline. It is not perfect, but still better than the current situation.

Fine with me, but not up to me. But I guess Andrew could merge it.

The patch is providing a fix to my own code, and I acked it already. The root of the problem is a defficiency in gcc, and dwmw2 asked that a PR be filled for that and a reference to it added to the patch. This has been done.

Nicolas

#7 lore raw
Quoted 6 lines · On Fri, 23 Apr 2010 13:50:10 -0400 (EDT) Nicolas Pitre <[email protected]> wrote:
Quoted 1 line

Fine with me, but not up to me. But I guess Andrew could merge it.

The patch is providing a fix to my own code, and I acked it already. The root of the problem is a defficiency in gcc, and dwmw2 asked that a PR be filled for that and a reference to it added to the patch. This has been done.

Well, I merged it and can send it upstream. But Jamie's observations give me pause?

#8 lore raw
Quoted 5 lines · Andrew Morton
Quoted 2 lines · On Fri, 23 Apr 2010 13:50:10 -0400 (EDT) Nicolas Pitre <[email protected]> wrote:
Quoted 1 line

Fine with me, but not up to me. But I guess Andrew could merge it.

Quoted 7 lines
Quoted 4 lines

The patch is providing a fix to my own code, and I acked it already. The root of the problem is a defficiency in gcc, and dwmw2 asked that a PR be filled for that and a reference to it added to the patch. This has been done.

Well, I merged it and can send it upstream. But Jamie's observations give me pause?

What observation? Let me find that.

Nicolas

#9 lore raw
In reply to Paulius Zaleckas
Quoted 1 line · Paulius Zaleckas

It's probably worth including the people who weighed in on the discussion with 'Cc:' headers.

drivers/mtd/nand/orion_nand.c static void orion_nand_read_buf(struct mtd_info *mtd, uint8_t *buf, int len) · line 71

Quoted 10 lines
drivers/mtd/nand/orion_nand.c +7 -1
-		uint64_t x;
+		/*
+		 * Since GCC has no proper constraint (PR 43518)
+		 * force x variable to r2/r3 registers as ldrd instruction
+		 * requires first register to be even.
+		 */
+		register uint64_t x asm ("r2");
+
 		asm volatile ("ldrd\t%0, [%1]" : "=&r" (x) : "r" (io_base));
 		buf64[i++] = x;

The "register...asm" looks fine, but it occurs to me the constraints are too weak (and they were before), so GCC could optimise that to the wrong behaviour.

The "volatile" prevents GCC deleting the asm if it's output isn't used, but it doesn't stop GCC from reordering the asms, for example if it decides to unroll the loop. It probably won't reorder in that case, but it could. The result would be out of order values stored into buf[]. It could even move the ldrd earlier than the prior byte accesses, or after the later byte accesses.

Any one of these should fix it:

- Make io_base a pointer-to-volatile-u64 or cast it in the asm, and make sure to dereference it and use an "m" constraint (or tighter, such as "Q", if ldrd needs it). It must be u64, not pointer-to-void, to tell GCC the size. That tells GCC which memory the asm accesses, and the volatile dereference should tell GCC not to reorder them in principle (but the GCC manual doesn't make a specific promise about this for asms).

With a proper memory input with the correct size, in principle "asm volatile" can be changed to just "asm", but I'm not entirely convinced GCC will honour the volatile on the pointer, so I'd leave it on the asm too.

- Add "memory" to the asm's clobbers. Although it doesn't write, it does change the visible memory that *io_base sees, and anyway GCC's manual says to use "memory" clobber when the asm does unpredictable memory reads too. With that added, you still need the volatile keyword after asm, because the memory is not listed in the inputs or outputs (only the address is). The GCC manual explains that "asm volatile" is needed in that case.

This is slightly less good because it'd prevent reordering writes to buf[i++] if GCC unrolled the loop.

- Put barrier() before and after the asm, which is equivalent to adding a "memory" clobber (least good).

You aren't supposed to dereference pointers used with read{b,w,l} anyway. It doesn't matter in this driver because we "know" it's only used on an SoC where read{b,w,l} don't do any address translation. But will that always be true? I suppose the cleanest approach is to define readq, the 64-bit analogue of readl, and use that here. x86 already defines readq, so it's got precedent.

-- Jamie

#10 lore raw
Quoted 21 lines · Jamie Lokier
Quoted 1 line · Paulius Zaleckas

It's probably worth including the people who weighed in on the discussion with 'Cc:' headers.

drivers/mtd/nand/orion_nand.c static void orion_nand_read_buf(struct mtd_info *mtd, uint8_t *buf, int len) · line 71

Quoted 10 lines
drivers/mtd/nand/orion_nand.c +7 -1
-		uint64_t x;
+		/*
+		 * Since GCC has no proper constraint (PR 43518)
+		 * force x variable to r2/r3 registers as ldrd instruction
+		 * requires first register to be even.
+		 */
+		register uint64_t x asm ("r2");
+
 		asm volatile ("ldrd\t%0, [%1]" : "=&r" (x) : "r" (io_base));
 		buf64[i++] = x;

The "register...asm" looks fine, but it occurs to me the constraints are too weak (and they were before), so GCC could optimise that to the wrong behaviour.

Quoted 6 lines

The "volatile" prevents GCC deleting the asm if it's output isn't used, but it doesn't stop GCC from reordering the asms, for example if it decides to unroll the loop. It probably won't reorder in that case, but it could. The result would be out of order values stored into buf[]. It could even move the ldrd earlier than the prior byte accesses, or after the later byte accesses.

I don't see how that could happen. The store into buf[] puts a dependency on the output constraint of the inline asm statement. And by vertue of being volatile, gcc cannot cache the result of the output from the asm as if it was a pure function.

Quoted 9 lines

Any one of these should fix it:

- Make io_base a pointer-to-volatile-u64 or cast it in the asm, and make sure to dereference it and use an "m" constraint (or tighter, such as "Q", if ldrd needs it). It must be u64, not pointer-to-void, to tell GCC the size. That tells GCC which memory the asm accesses, and the volatile dereference should tell GCC not to reorder them in principle (but the GCC manual doesn't make a specific promise about this for asms).

The LDRD has special range constraints on its addressing mode which is not expressable with any of the available gcc memory constraints.

Quoted 6 lines

You aren't supposed to dereference pointers used with read{b,w,l} anyway. It doesn't matter in this driver because we "know" it's only used on an SoC where read{b,w,l} don't do any address translation. But will that always be true? I suppose the cleanest approach is to define readq, the 64-bit analogue of readl, and use that here. x86 already defines readq, so it's got precedent.

But yet it is not all ARM variants that can do 64-bit accesses. Anything pre ARMv5 doesn't have the LDRD instruction, and the equivalent LDM is not a possible substitute with regard to memory bus access either.

So I'd prefer to keep it as an obvious local exception that happens to exploit some specifics of the actual hardware implementation rather than something that was architecturally defined.

Nicolas

#11 lore raw
Quoted 15 lines · Nicolas Pitre
Quoted 7 lines · Jamie Lokier

drivers/mtd/nand/orion_nand.c static void orion_nand_read_buf(struct mtd_info *mtd, uint8_t *buf, int len) · line 71

Quoted 2 lines
drivers/mtd/nand/orion_nand.c +0 -0
 		asm volatile ("ldrd\t%0, [%1]" : "=&r" (x) : "r" (io_base));
 		buf64[i++] = x;

The "register...asm" looks fine, but it occurs to me the constraints are too weak (and they were before), so GCC could optimise that to the wrong behaviour.

Quoted 6 lines

The "volatile" prevents GCC deleting the asm if it's output isn't used, but it doesn't stop GCC from reordering the asms, for example if it decides to unroll the loop. It probably won't reorder in that case, but it could. The result would be out of order values stored into buf[]. It could even move the ldrd earlier than the prior byte accesses, or after the later byte accesses.

Quoted 4 lines

I don't see how that could happen. The store into buf[] puts a dependency on the output constraint of the inline asm statement. And by vertue of being volatile, gcc cannot cache the result of the output from the asm as if it was a pure function.

The store into buf[] dependency doesn't stop this, after unrolling:

    asm volatile ("ldrd\t%0, [%1]" : "=&r" (x) : "r" (io_base));
    buf64[i++] = x;
    asm volatile ("ldrd\t%0, [%1]" : "=&r" (x) : "r" (io_base));
    buf64[i++] = x;

from being reordered as this

    asm volatile ("ldrd\t%0, [%1]" : "=&r" (x2) : "r" (io_base));
    asm volatile ("ldrd\t%0, [%1]" : "=&r" (x1) : "r" (io_base));
    buf64[i++] = x1;
    buf64[i++] = x2;

because the asm doesn't depend on memory, just register inputs and outputs;

I'm not sure what you mean about the volatile stopping gcc from treating the asm as a pure function. Is that meaning of volatile in the asm documentation? (volatile on asm doesn't mean the same as volatile on a function, or volatile on a pointer).

Quoted 10 lines
Quoted 9 lines

Any one of these should fix it:

- Make io_base a pointer-to-volatile-u64 or cast it in the asm, and make sure to dereference it and use an "m" constraint (or tighter, such as "Q", if ldrd needs it). It must be u64, not pointer-to-void, to tell GCC the size. That tells GCC which memory the asm accesses, and the volatile dereference should tell GCC not to reorder them in principle (but the GCC manual doesn't make a specific promise about this for asms).

Quoted 2 lines

The LDRD has special range constraints on its addressing mode which is not expressable with any of the available gcc memory constraints.

'Q'
A memory reference where the exact address is in a single register (''m'' is preferable for 'asm' statements)

If 'r' is good enough for io_base, 'Q' should be good enough for *io_base.

-- Jamie

#12 lore raw
Quoted 12 lines · Jamie Lokier
Quoted 11 lines · Nicolas Pitre
Quoted 9 lines · Jamie Lokier

Any one of these should fix it:

- Make io_base a pointer-to-volatile-u64 or cast it in the asm, and make sure to dereference it and use an "m" constraint (or tighter, such as "Q", if ldrd needs it). It must be u64, not pointer-to-void, to tell GCC the size. That tells GCC which memory the asm accesses, and the volatile dereference should tell GCC not to reorder them in principle (but the GCC manual doesn't make a specific promise about this for asms).

Quoted 8 lines
Quoted 2 lines

The LDRD has special range constraints on its addressing mode which is not expressable with any of the available gcc memory constraints.

'Q'
A memory reference where the exact address is in a single register (''m'' is preferable for 'asm' statements)

If 'r' is good enough for io_base, 'Q' should be good enough for *io_base.

And if that doesn't work, it's possible to pass both the pointer, and the dereference, as separate input operands, and only use the pointer in the asm template. GCC will still treat the dereference input as a dependency.

-- Jamie

#13 lore raw
In reply to Jamie Lokier
Quoted 30 lines · Jamie Lokier
Quoted 15 lines · Nicolas Pitre
Quoted 7 lines · Jamie Lokier

drivers/mtd/nand/orion_nand.c static void orion_nand_read_buf(struct mtd_info *mtd, uint8_t *buf, int len) · line 71

Quoted 2 lines
drivers/mtd/nand/orion_nand.c +0 -0
 		asm volatile ("ldrd\t%0, [%1]" : "=&r" (x) : "r" (io_base));
 		buf64[i++] = x;

The "register...asm" looks fine, but it occurs to me the constraints are too weak (and they were before), so GCC could optimise that to the wrong behaviour.

Quoted 6 lines

The "volatile" prevents GCC deleting the asm if it's output isn't used, but it doesn't stop GCC from reordering the asms, for example if it decides to unroll the loop. It probably won't reorder in that case, but it could. The result would be out of order values stored into buf[]. It could even move the ldrd earlier than the prior byte accesses, or after the later byte accesses.

Quoted 4 lines

I don't see how that could happen. The store into buf[] puts a dependency on the output constraint of the inline asm statement. And by vertue of being volatile, gcc cannot cache the result of the output from the asm as if it was a pure function.

The store into buf[] dependency doesn't stop this, after unrolling:

    asm volatile ("ldrd\t%0, [%1]" : "=&r" (x) : "r" (io_base));
    buf64[i++] = x;
    asm volatile ("ldrd\t%0, [%1]" : "=&r" (x) : "r" (io_base));
    buf64[i++] = x;

from being reordered as this

Quoted 7 lines
    asm volatile ("ldrd\t%0, [%1]" : "=&r" (x2) : "r" (io_base));
    asm volatile ("ldrd\t%0, [%1]" : "=&r" (x1) : "r" (io_base));
    buf64[i++] = x1;
    buf64[i++] = x2;

because the asm doesn't depend on memory, just register inputs and outputs;

I disagree. The volatile tells gcc that the asm has side effects, and therefore 1) they can't be optimized away, and 2) can't be swapped with regards to each other like you do in your example.

Quoted 3 lines

'Q'
A memory reference where the exact address is in a single register (''m'' is preferable for 'asm' statements)

Hmmm... Is this something new? I must have missed it before.

Nicolas