arm64: Support the TSO memory model
Get series with b4
b4 am [email protected] change-id 20240411-tso-e86fdceb94b8Versions newest first
- v1 Latest4 patches 25 repliesIn review
Patches in v1 No applies recorded yet
| # | Subject and files | Size | Replies | Applied |
|---|---|---|---|---|
| 1/4 | prctl: Introduce PR_{SET,GET}_MEM_MODELinclude/ include/ kernel/ | +37 | 0 replies | |
| 2/4 | arm64: Implement PR_{GET,SET}_MEM_MODEL for always-TSO CPUsarch/ arch/ arch/ +4 more | +84 | 0 replies | |
| 3/4 | arm64: Introduce scaffolding to add ACTLR_EL1 to thread statearch/ arch/ arch/ +2 more | +44 | 0 replies | |
| 4/4 | arm64: Implement Apple IMPDEF TSO memory model controlarch/ arch/ arch/ +3 more | +65 | 0 replies |
Reviews trailers per patch
None of the 4 patches has a Reviewed-by or Acked-by of their own; the series has one for all of them.
| Patch | Neal Gompa |
|---|---|
| Series | R-b |
| 1/4 | |
| 2/4 | |
| 3/4 | |
| 4/4 |
Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes.
Cover letter [PATCH 0/4] arm64: Support the TSO memory model
x86 CPUs implement a stricter memory modern than ARM64 (TSO). For this reason, x86 emulation on baseline ARM64 systems requires very expensive memory model emulation. Having hardware that supports this natively is therefore very attractive. Such hardware, in fact, exists. This series adds support for userspace to identify when TSO is available and toggle it on, if supported.
Some ARM64 CPUs intrinsically implement the TSO memory model, while others expose is as an IMPDEF control. Apple Silicon SoCs are in the latter category. Using TSO for x86 emulation on chips that support it has been shown to provide a massive performance boost [1].
Patch 1 introduces the PR_{SET,GET}_MEM_MODEL userspace control, which is initially not implemented for any architectures.
Patch 2 implements it for CPUs which are known, to the best of my knowledge, to always implement the TSO memory model unconditionally. This uses the cpufeature mechanism to only enable this if *all* cores in the system meet the requirements.
Patch 3 adds the scaffolding necesasry to save/restore the ACTLR_EL1 register across context switches. This register contains IMPDEF flags related to CPU execution, and on Apple CPUs this is where the runtime TSO toggle bit is implemented. Other CPUs could conceivably benefit from this scaffolding if they also use ACTLR_EL1 for things that could ostensibly be runtime controlled and context-switched. For this to work, ACTLR_EL1 must have a uniform layout across all cores in the system.
Finally, patch 4 implements PR_{SET,GET}_MEM_MODEL for Apple CPUs by hooking it up to flip the appropriate ACTLR_EL1 bit when the Apple TSO feature is detected (on all CPUs, which also implies the uniform ACTLR_EL1 layout).
This series has been brewing in the downstream Asahi Linux tree for a while now, and ships to thousands of users. A subset have been using it with FEX-Emu, which already supports this feature. This rebase on v6.9-rc1 is only build-tested (all intermediate commits with and without the config enabled, on ARM64) but I'll update the downstream branch soon with this version and get it pushed out to users/testers.
The Apple support works on bare metal and *should* work exactly the same way on macOS VMs (as alluded to by Zayd in his independent submission [3]), though I haven't personally verified this. KVM support for this is left for a future patchset.
(Apologies for the large Cc: list; I want to make sure nobody who got Cced on Zayd's alternate take is left out of this one.)
[1] https://fex-emu.com/FEX-2306/
[2] https://github.com/AsahiLinux/linux/tree/bits/220-tso
[3] https://lore.kernel.org/lkml/[email protected]/
To: Catalin Marinas <[email protected]>
To: Will Deacon <[email protected]>
To: Marc Zyngier <[email protected]>
To: Mark Rutland <[email protected]>
Cc: Zayd Qumsieh <[email protected]>
Cc: Justin Lu <[email protected]>
Cc: Ryan Houdek <[email protected]>
Cc: Mark Brown <[email protected]>
Cc: Ard Biesheuvel <[email protected]>
Cc: Mateusz Guzik <[email protected]>
Cc: Anshuman Khandual <[email protected]>
Cc: Oliver Upton <[email protected]>
Cc: Miguel Luis <[email protected]>
Cc: Joey Gouly <[email protected]>
Cc: Christoph Paasch <[email protected]>
Cc: Kees Cook <[email protected]>
Cc: Sami Tolvanen <[email protected]>
Cc: Baoquan He <[email protected]>
Cc: Joel Granados <[email protected]>
Cc: Dawei Li <[email protected]>
Cc: Andrew Morton <[email protected]>
Cc: Florent Revest <[email protected]>
Cc: David Hildenbrand <[email protected]>
Cc: Stefan Roesch <[email protected]>
Cc: Andy Chiu <[email protected]>
Cc: Josh Triplett <[email protected]>
Cc: Oleg Nesterov <[email protected]>
Cc: Helge Deller <[email protected]>
Cc: Zev Weiss <[email protected]>
Cc: Ondrej Mosnacek <[email protected]>
Cc: Miguel Ojeda <[email protected]>
Cc: [email protected]
Cc: [email protected]
Cc: Asahi Linux <[email protected]>
---
Hector Martin (4):
prctl: Introduce PR_{SET,GET}_MEM_MODEL
arm64: Implement PR_{GET,SET}_MEM_MODEL for always-TSO CPUs
arm64: Introduce scaffolding to add ACTLR_EL1 to thread state
arm64: Implement Apple IMPDEF TSO memory model control
arch/arm64/Kconfig | 14 ++++++ arch/arm64/include/asm/apple_cpufeature.h | 15 +++++++ arch/arm64/include/asm/cpufeature.h | 10 +++++ arch/arm64/include/asm/processor.h | 3 ++ arch/arm64/kernel/Makefile | 3 +- arch/arm64/kernel/cpufeature.c | 11 ++--- arch/arm64/kernel/cpufeature_impdef.c | 61 ++++++++++++++++++++++++++ arch/arm64/kernel/process.c | 71 +++++++++++++++++++++++++++++++ arch/arm64/kernel/setup.c | 8 ++++ arch/arm64/tools/cpucaps | 2 + include/linux/memory_ordering_model.h | 11 +++++ include/uapi/linux/prctl.h | 5 +++ kernel/sys.c | 21 +++++++++ 13 files changed, 229 insertions(+), 6 deletions(-)
---
base-commit: 4cece764965020c22cff7665b18a012006359095
change-id: 20240411-tso-e86fdceb94b8
Best regards,
Signature · 1 line
Hector Martin <[email protected]>