ABI updates

arch/arm/kernel/setup.c static void __init cpuid_init_hwcaps(void) · line 409

Quoted 11 lines · On Friday 04 July 2014 20:52:17 Russell King wrote:

/*
* HWCAP_TLS is available only on 1136 r1p0 and later,
* see also kuser_get_tls_init.
*/

arch/arm/kernel/setup.c +4 -1

-       if ((((id >> 4) & 0xfff) == 0xb36) && (((id >> 20) & 3) == 0))
+       if (read_cpuid_part() == ARM_CPU_PART_ARM1136 &&
+           ((id >> 20) & 3) == 0) {
                elf_hwcap &= ~HWCAP_TLS;
+               return;
+       }

Would it make sense to tie this check to the ARMv6K architecture level?

I see that cpu_architecture currently reports the same value for plain ARMv6 (1136r0) and for ARMv6K (1136r1+ or any other ARM11), but we already distinguish them at compile time in a number of places and it seems you make the exact same distinction here at runtime.

Hmm, we need guidance from ARM people on that.

There may well be a better way to detect between ARMv6 and ARMv6K, which is given by the architecture spec. G7.3 of an early DDI0406 says that the MPIDR (mp affinity register) aliases to MIDR for ARMv6, but is of course implemented for ARMv6K.

This seems to be carried through to the latest ARM ARM. So it seems this would be a more correct way to tell ARMv6 from ARMv6K.

If so, we can certainly expand cpu_architecture() to detect between the two and add a CPU_ARCH_ARMv6K in there.

Let's see what Will has to say about that when he's next around... though I think it'll require another trawl through lots of documentation.

Quoted 13 lines · On Friday 04 July 2014 21:51:44 Russell King - ARM Linux wrote:

Hmm, we need guidance from ARM people on that.

There may well be a better way to detect between ARMv6 and ARMv6K, which is given by the architecture spec. G7.3 of an early DDI0406 says that the MPIDR (mp affinity register) aliases to MIDR for ARMv6, but is of course implemented for ARMv6K.

This seems to be carried through to the latest ARM ARM. So it seems this would be a more correct way to tell ARMv6 from ARMv6K.

If so, we can certainly expand cpu_architecture() to detect between the two and add a CPU_ARCH_ARMv6K in there.

Quoted 3 lines

Let's see what Will has to say about that when he's next around... though I think it'll require another trawl through lots of documentation.

I was thinking of a simpler check in __get_cpu_architecture, just checking if the CPUID is ARM1136r0 since that is the only ARMv6 CPU core we support. Anything else that we currently report as ARMv6 is actually ARMv6K.

Quoted 18 lines · Arnd Bergmann

Quoted 16 lines · On Friday 04 July 2014 21:51:44 Russell King - ARM Linux wrote:

Hmm, we need guidance from ARM people on that.

There may well be a better way to detect between ARMv6 and ARMv6K, which is given by the architecture spec. G7.3 of an early DDI0406 says that the MPIDR (mp affinity register) aliases to MIDR for ARMv6, but is of course implemented for ARMv6K.

This seems to be carried through to the latest ARM ARM. So it seems this would be a more correct way to tell ARMv6 from ARMv6K.

If so, we can certainly expand cpu_architecture() to detect between the two and add a CPU_ARCH_ARMv6K in there.

Let's see what Will has to say about that when he's next around... though I think it'll require another trawl through lots of documentation.

Quoted 4 lines

I was thinking of a simpler check in __get_cpu_architecture, just checking if the CPUID is ARM1136r0 since that is the only ARMv6 CPU core we support. Anything else that we currently report as ARMv6 is actually ARMv6K.

I'm not prepared to use that as the basis for detecting between V6 and V6K at the moment - we don't know that for sure. There seems to be some Samsung platforms which are V6 (at least they don't select V6K).

Either way, this isn't something we should rush into, because it has the capability to break all sorts of stuff. For example, merely deciding to add a CPU_ARCH_ARMv6K results in a change to /proc/cpuinfo (changing the "CPU architecture:" line) which is a user visible change.

We also have two ways to detect CPU_ARCH_ARMv6 - that needs careful thought as well. We also have a number of places which test for equality with ARMv6, rather than less than or greater-or-equal.

Then there's all the ifdefs on CPU_V6K...

This is not something I want to rush into, and I certainly don't want to make assumptions such as "ARMv6 === ARM1136 r0" - and I certainly don't want to get stuck in a corner by making some changes and then finding out that they're wrong later down the line when we can't then fix it without more user visible churn.

Quoted 13 lines · On Friday 04 July 2014 21:51:44 Russell King - ARM Linux wrote:

Hmm, we need guidance from ARM people on that.

There may well be a better way to detect between ARMv6 and ARMv6K, which is given by the architecture spec. G7.3 of an early DDI0406 says that the MPIDR (mp affinity register) aliases to MIDR for ARMv6, but is of course implemented for ARMv6K.

This seems to be carried through to the latest ARM ARM. So it seems this would be a more correct way to tell ARMv6 from ARMv6K.

If so, we can certainly expand cpu_architecture() to detect between the two and add a CPU_ARCH_ARMv6K in there.

Quoted 3 lines

Let's see what Will has to say about that when he's next around... though I think it'll require another trawl through lots of documentation.

I was thinking of a simpler check in __get_cpu_architecture, just checking if the CPUID is ARM1136r0 since that is the only ARMv6 CPU core we support. Anything else that we currently report as ARMv6 is actually ARMv6K.

I'm not sure that's the right approach. ARM added the CPUID scheme to allow the software to check specific features rather than guessing specific architecture versions. Basically there isn't a clear way to identify whether your CPU is ARMv6K or ARMv7 from a single ID register read (you may be able to infer by reading multiple ID regs).

I'm now trying to figure out whether the TPIDR registers actually have a dedicated ID. I think they only come as part of the VMSA version 7 as identified from ID_MMFR0. The ARM1136 r1p1 TRM states that ID_MMFR0[3:0] are 0x3 which mean VMSAv7.

Quoted 6 lines · Catalin Marinas

I'm not sure that's the right approach. ARM added the CPUID scheme to allow the software to check specific features rather than guessing specific architecture versions. Basically there isn't a clear way to identify whether your CPU is ARMv6K or ARMv7 from a single ID register read (you may be able to infer by reading multiple ID regs).

Quoted 4 lines

I'm now trying to figure out whether the TPIDR registers actually have a dedicated ID. I think they only come as part of the VMSA version 7 as identified from ID_MMFR0. The ARM1136 r1p1 TRM states that ID_MMFR0[3:0] are 0x3 which mean VMSAv7.

You can't rely on the CPUID registers on 1136, because it doesn't advertise them as being present - the architecture field of MIDR is 0x7, not 0xf.

So actually, 1136r0 is the architecturally correct version, and 1136r1 is the slightly cocked up non-standard version where we have to be careful how we treat it.

Quoted 12 lines · Russell King - ARM Linux

Quoted 10 lines · Catalin Marinas

I'm not sure that's the right approach. ARM added the CPUID scheme to allow the software to check specific features rather than guessing specific architecture versions. Basically there isn't a clear way to identify whether your CPU is ARMv6K or ARMv7 from a single ID register read (you may be able to infer by reading multiple ID regs).

I'm now trying to figure out whether the TPIDR registers actually have a dedicated ID. I think they only come as part of the VMSA version 7 as identified from ID_MMFR0. The ARM1136 r1p1 TRM states that ID_MMFR0[3:0] are 0x3 which mean VMSAv7.

Quoted 3 lines

You can't rely on the CPUID registers on 1136, because it doesn't advertise them as being present - the architecture field of MIDR is 0x7, not 0xf.

I was hoping they changed it with r1 but looking at the TRM that's not the case.

Quoted 3 lines

So actually, 1136r0 is the architecturally correct version, and 1136r1 is the slightly cocked up non-standard version where we have to be careful how we treat it.

Yes. Looking at ARM1176, it seems to be using the full CPUID scheme and reporting VMSAv7. So I guess we can safely assume TLS presence if VMSAv7 (actually what __get_cpu_architecture checks) or ARM1136 r1+.

Quoted 17 lines · Catalin Marinas

Quoted 15 lines · Russell King - ARM Linux

Quoted 10 lines · Catalin Marinas

I'm not sure that's the right approach. ARM added the CPUID scheme to allow the software to check specific features rather than guessing specific architecture versions. Basically there isn't a clear way to identify whether your CPU is ARMv6K or ARMv7 from a single ID register read (you may be able to infer by reading multiple ID regs).

I'm now trying to figure out whether the TPIDR registers actually have a dedicated ID. I think they only come as part of the VMSA version 7 as identified from ID_MMFR0. The ARM1136 r1p1 TRM states that ID_MMFR0[3:0] are 0x3 which mean VMSAv7.

You can't rely on the CPUID registers on 1136, because it doesn't advertise them as being present - the architecture field of MIDR is 0x7, not 0xf.

Quoted 2 lines

I was hoping they changed it with r1 but looking at the TRM that's not the case.

You misunderstand. r0 did not have the CPUID registers at all, so 0x7 is correct. r1 added the CPUID registers, but left the MIDR with the architecture field set to 0x7.

Quoted 7 lines

Quoted 3 lines

So actually, 1136r0 is the architecturally correct version, and 1136r1 is the slightly cocked up non-standard version where we have to be careful how we treat it.

Yes. Looking at ARM1176, it seems to be using the full CPUID scheme and reporting VMSAv7. So I guess we can safely assume TLS presence if VMSAv7 (actually what __get_cpu_architecture checks) or ARM1136 r1+.

*Not* ARM1136 r1, because there the CPUID registers are ignored because MIDR does not indicate their presence. So all ARM1136 are currently identified as ARMv6 by the kernel.

With my proposal, ARM1136 with MPIDR would be identified as ARMv6K, but ARM1136 r1 without MPIDR would remain identified as ARMv6.

Quoted 11 lines · Russell King - ARM Linux

Quoted 7 lines

Quoted 3 lines

So actually, 1136r0 is the architecturally correct version, and 1136r1 is the slightly cocked up non-standard version where we have to be careful how we treat it.

Yes. Looking at ARM1176, it seems to be using the full CPUID scheme and reporting VMSAv7. So I guess we can safely assume TLS presence if VMSAv7 (actually what __get_cpu_architecture checks) or ARM1136 r1+.

*Not* ARM1136 r1, because there the CPUID registers are ignored because MIDR does not indicate their presence. So all ARM1136 are currently identified as ARMv6 by the kernel.

I agree.

Quoted 2 lines

With my proposal, ARM1136 with MPIDR would be identified as ARMv6K, but ARM1136 r1 without MPIDR would remain identified as ARMv6.

The ARM1136 TRM states that r1 introduces ARMv6K features. However, I can't find any trace of MPIDR and it may actually just return MIDR. If that's the case, we don't have a way to classify ARMv6K here based on MPIDR.

My original point was to ignore the v6K classification and, for the TLS presence, just check the feature registers if full CPUID is present, otherwise let TLS enabled for ARM1136 r1 because we know it implements it (according to the TRM, special case without full CPUID).

Quoted 18 lines · Catalin Marinas

Quoted 11 lines · Russell King - ARM Linux

Quoted 7 lines

Quoted 3 lines

So actually, 1136r0 is the architecturally correct version, and 1136r1 is the slightly cocked up non-standard version where we have to be careful how we treat it.

Yes. Looking at ARM1176, it seems to be using the full CPUID scheme and reporting VMSAv7. So I guess we can safely assume TLS presence if VMSAv7 (actually what __get_cpu_architecture checks) or ARM1136 r1+.

*Not* ARM1136 r1, because there the CPUID registers are ignored because MIDR does not indicate their presence. So all ARM1136 are currently identified as ARMv6 by the kernel.

I agree.

Quoted 2 lines

With my proposal, ARM1136 with MPIDR would be identified as ARMv6K, but ARM1136 r1 without MPIDR would remain identified as ARMv6.

Quoted 4 lines

The ARM1136 TRM states that r1 introduces ARMv6K features. However, I can't find any trace of MPIDR and it may actually just return MIDR. If that's the case, we don't have a way to classify ARMv6K here based on MPIDR.

It is documented in all sorts of places, including the 1136 TRM, that when the MPIDR is not implemented, it returns the MIDR.

Quoted 4 lines

My original point was to ignore the v6K classification and, for the TLS presence, just check the feature registers if full CPUID is present, otherwise let TLS enabled for ARM1136 r1 because we know it implements it (according to the TRM, special case without full CPUID).

Can we please stop this pointless wandering off of the topic. We're not talking about TLS stuff here - that remains the same as it ever did.

We're talking about SWP and/or the exclusives.

Going on topic again, LDREXB is present in ARM1136 r1 but your changes would not detect it (and it's a different ID_ISAR3 format anyway). If we care about this case, you could either pretend that ARM1136 r1 is ARMv6K and link those decisions to v6K+ tests or check for individual features which for ARM1136 would require a different check from standard CPUID.

Quoted 5 lines · Catalin Marinas

Going on topic again, LDREXB is present in ARM1136 r1 but your changes would not detect it (and it's a different ID_ISAR3 format anyway). If we care about this case, you could either pretend that ARM1136 r1 is ARMv6K and link those decisions to v6K+ tests or check for individual features which for ARM1136 would require a different check from standard CPUID.

Ever since ARMv6 was first added, HWCAP_SWP has been indicated for all ARMv6 CPUs and their derivatives. Indeed, the SWP instruction is supported in hardware.

Hence, indicating HWCAP_SWP on ARMv6 whether or not there is LDREXB support is /not/ incorrect in any way.

ARM1136 r1 added LDREXB as part of the ARMv6K adoption, but it kept the SWP instruction (which can't be disabled.) So, indicating HWCAP_SWP also is /not/ incorrect.

However, when it comes to SMP, we would _prefer_ userspace to use the exclusives instructions, though it seems that the SWP behaviour on an ARMv6K SMP system with respect to other CPUs is not documented. All things being equal, we would prefer SWP not be used there because we don't know whether it's safe. Unfortunately, we can't turn the instruction off, so we can't emulate it.

However, we know that some userspace running there may well contain SWP instructions. So, as there aren't any bug reports, it's probably safe to assume that SWP is safe on ARMv6K SMP systems.

So, the upshot of this is:
* SWP on all ARMv6 is fine.
* Advertising HWCAP_SWP on all ARMv6 is fine too.
* We would prefer not to advertise HWCAP_SWP on ARMv6 with support for the exclusives, and we can do that trivially where we can check the CPUID values.
* ARM1136 is either ARMv6 or ARMv6K depending on its revision - and thus may have LDREXB, but LDREXB is not trivially detectable.

I believe what I'm doing in this patch is architecturally correct, I also believe it to be safe for the troublesome ARM1136 by way of defaulting to advertising HWCAP_SWP.

Quoted 13 lines · Arnd Bergmann

arch/arm/kernel/setup.c static void __init cpuid_init_hwcaps(void) · line 409

Quoted 11 lines · On Friday 04 July 2014 20:52:17 Russell King wrote:

/*
* HWCAP_TLS is available only on 1136 r1p0 and later,
* see also kuser_get_tls_init.
*/

arch/arm/kernel/setup.c +4 -1

-       if ((((id >> 4) & 0xfff) == 0xb36) && (((id >> 20) & 3) == 0))
+       if (read_cpuid_part() == ARM_CPU_PART_ARM1136 &&
+           ((id >> 20) & 3) == 0) {
                elf_hwcap &= ~HWCAP_TLS;
+               return;
+       }

Quoted 6 lines

Would it make sense to tie this check to the ARMv6K architecture level?

I see that cpu_architecture currently reports the same value for plain ARMv6 (1136r0) and for ARMv6K (1136r1+ or any other ARM11), but we already distinguish them at compile time in a number of places and it seems you make the exact same distinction here at runtime.

Hmm, we need guidance from ARM people on that.

There may well be a better way to detect between ARMv6 and ARMv6K, which is given by the architecture spec. G7.3 of an early DDI0406 says that the MPIDR (mp affinity register) aliases to MIDR for ARMv6, but is of course implemented for ARMv6K.

This seems to be carried through to the latest ARM ARM. So it seems this would be a more correct way to tell ARMv6 from ARMv6K.

If so, we can certainly expand cpu_architecture() to detect between the two and add a CPU_ARCH_ARMv6K in there.

Let's see what Will has to say about that when he's next around... though I think it'll require another trawl through lots of documentation.

Hi Russell,

Quoted 21 lines · Russell King - ARM Linux

Quoted 19 lines · Arnd Bergmann

arch/arm/kernel/setup.c static void __init cpuid_init_hwcaps(void) · line 409

Quoted 11 lines · On Friday 04 July 2014 20:52:17 Russell King wrote:

/*
* HWCAP_TLS is available only on 1136 r1p0 and later,
* see also kuser_get_tls_init.
*/

arch/arm/kernel/setup.c +4 -1

-       if ((((id >> 4) & 0xfff) == 0xb36) && (((id >> 20) & 3) == 0))
+       if (read_cpuid_part() == ARM_CPU_PART_ARM1136 &&
+           ((id >> 20) & 3) == 0) {
                elf_hwcap &= ~HWCAP_TLS;
+               return;
+       }

Would it make sense to tie this check to the ARMv6K architecture level?

I see that cpu_architecture currently reports the same value for plain ARMv6 (1136r0) and for ARMv6K (1136r1+ or any other ARM11), but we already distinguish them at compile time in a number of places and it seems you make the exact same distinction here at runtime.

Quoted 6 lines

Hmm, we need guidance from ARM people on that.

There may well be a better way to detect between ARMv6 and ARMv6K, which is given by the architecture spec. G7.3 of an early DDI0406 says that the MPIDR (mp affinity register) aliases to MIDR for ARMv6, but is of course implemented for ARMv6K.

I don't think MPIDR exists on any 1136 (i.e. probably UNDEFs).

Quoted 1 line · Will Deacon

I don't think MPIDR exists on any 1136 (i.e. probably UNDEFs).

There has long been a requirement (back to ARMv4 I believe) that instructions such as:

	mrc	p15, 0, rd, c0, c0, X

all return the MIDR unless they are implemented. This is certainly carried forward to the latest ARM ARMs. This is done to explicitly allow the presence of additional registers to be detected.

I would be very surprised if the 1136 did undef on this.

If I dig out my 1136 FPGA (which according to the docs would be r0p2) I can explicitly verify this... provided Integrator CP still works.

As a result of the recent discussions about android, several issues have been identified with existing kernels:

1. HWCAP_SWP does not properly reflect whether the SWP instruction should be used. Turn HWCAP_SWP off on ARMv6+ where the exclusives are available.

2. SWP is unsafe when running on a SMP CPU; there is no bus locking between the read and write parts of the instruction execution. Force SWP emulation on ARMv7+ where we can disable the SWP instruction.

3. Reporting of alignment faults - kernel mode faults are silent due to the requirements of the network stack, but we can note where the last one occurs.

Testing reveals that at least my OMAP4430 userspace (supplied from TI) executes lots of SWP instructions, which is something that wasn't known before this patch.

Many systems today are not configured with SWP emulation enabled on SMP. This means that the SWP instruction is available, but unsafe.

As a result of the recent discussions about android, several issues have been identified with existing kernels:

1. HWCAP_SWP does not properly reflect whether the SWP instruction should be used. Turn HWCAP_SWP off on ARMv6+ where the exclusives are available.

2. SWP is unsafe when running on a SMP CPU; there is no bus locking between the read and write parts of the instruction execution. Force SWP emulation on ARMv7+ where we can disable the SWP instruction.

3. Reporting of alignment faults - kernel mode faults are silent due to the requirements of the network stack, but we can note where the last one occurs.

Testing reveals that at least my OMAP4430 userspace (supplied from TI) executes lots of SWP instructions, which is something that wasn't known before this patch.

Many systems today are not configured with SWP emulation enabled on SMP. This means that the SWP instruction is available, but unsafe.

 arch/arm/kernel/setup.c       | 29 ++++++++++++++++++++++-------
 arch/arm/kernel/swp_emulate.c |  4 ++++
 arch/arm/mm/Kconfig           |  2 +-
 arch/arm/mm/alignment.c       |  4 +++-
 4 files changed, 30 insertions(+), 9 deletions(-)

Quoted 20 lines · Russell King - ARM Linux

As a result of the recent discussions about android, several issues have been identified with existing kernels:

1. HWCAP_SWP does not properly reflect whether the SWP instruction should be used. Turn HWCAP_SWP off on ARMv6+ where the exclusives are available.

2. SWP is unsafe when running on a SMP CPU; there is no bus locking between the read and write parts of the instruction execution. Force SWP emulation on ARMv7+ where we can disable the SWP instruction.

3. Reporting of alignment faults - kernel mode faults are silent due to the requirements of the network stack, but we can note where the last one occurs.

Testing reveals that at least my OMAP4430 userspace (supplied from TI) executes lots of SWP instructions, which is something that wasn't known before this patch.

Quoted 8 lines

Many systems today are not configured with SWP emulation enabled on SMP. This means that the SWP instruction is available, but unsafe.

 arch/arm/kernel/setup.c       | 29 ++++++++++++++++++++++-------
 arch/arm/kernel/swp_emulate.c |  4 ++++
 arch/arm/mm/Kconfig           |  2 +-
 arch/arm/mm/alignment.c       |  4 +++-
 4 files changed, 30 insertions(+), 9 deletions(-)

Tried testing this on omap2 which is ARMv6 with no K, but got this:

arch/arm/kernel/setup.c: In function ?elf_hwcap_fixup?:
arch/arm/kernel/setup.c:405:2: error: implicit declaration of function ?read_cpuid_part? [-Werror=implicit-function-declaration]

That should be read_cpuid_part_number instead. After fixing that, omap2 booted fine, so for ARMv6 with no K:

Quoted 5 lines · Tony Lindgren

Tried testing this on omap2 which is ARMv6 with no K, but got this:

arch/arm/kernel/setup.c: In function ?elf_hwcap_fixup?:
arch/arm/kernel/setup.c:405:2: error: implicit declaration of function ?read_cpuid_part? [-Werror=implicit-function-declaration]

That's because they rely upon my "ARM: make it easier to check the CPU part number correctly" patch in linux-next.

Quoted 2 lines

That should be read_cpuid_part_number instead. After fixing that, omap2 booted fine, so for ARMv6 with no K:

Great, I hope you checked /proc/cpuinfo came out correctly, as well as noticing that it merely booted.

As a result of the recent discussions about android, several issues have been identified with existing kernels:

1. HWCAP_SWP does not properly reflect whether the SWP instruction should be used. Turn HWCAP_SWP off on ARMv6+ where the exclusives are available.

2. SWP is unsafe when running on a SMP CPU; there is no bus locking between the read and write parts of the instruction execution. Force SWP emulation on ARMv7+ where we can disable the SWP instruction.

3. Reporting of alignment faults - kernel mode faults are silent due to the requirements of the network stack, but we can note where the last one occurs.

Testing reveals that at least my OMAP4430 userspace (supplied from TI) executes lots of SWP instructions, which is something that wasn't known before this patch.

Many systems today are not configured with SWP emulation enabled on SMP. This means that the SWP instruction is available, but unsafe.