arm64: implement virtual KASLR

This adds support for emitting PLTs at module load time for relative branches that are out of range. This is a prerequisite for KASLR, which may place the kernel and the modules anywhere in the vmalloc area, making it more likely that branch target offsets exceed the maximum range of +/- 128 MB.

Quoted 5 lines · Ard Biesheuvel

This adds support for emitting PLTs at module load time for relative branches that are out of range. This is a prerequisite for KASLR, which may place the kernel and the modules anywhere in the vmalloc area, making it more likely that branch target offsets exceed the maximum range of +/- 128 MB.

Any downside to trying to keep the kernel+modules coupled together so that we avoid the PLT?

Quoted 9 lines · Catalin Marinas

Quoted 5 lines · Ard Biesheuvel

This adds support for emitting PLTs at module load time for relative branches that are out of range. This is a prerequisite for KASLR, which may place the kernel and the modules anywhere in the vmalloc area, making it more likely that branch target offsets exceed the maximum range of +/- 128 MB.

Any downside to trying to keep the kernel+modules coupled together so that we avoid the PLT?

First of all, note that it is unlikely that the PLTs are ever required in practice, considering that either
a) the kernel is loaded at the default location right at the base of the vmalloc range, and in this case, the module space is reserved for modules only, or
b) the kernel is loaded at some random offset in the 240+ GB vmalloc space, and it is unlikely that all VMA space around the kernel happens to be given out to non-randomized vmalloc/ioremap allocations

So the bottom line is that it may be possible, but it is unlikely to be worth the effort imo

Quoted 11 lines · Ard Biesheuvel

Quoted 9 lines · Catalin Marinas

Quoted 5 lines · Ard Biesheuvel

This adds support for emitting PLTs at module load time for relative branches that are out of range. This is a prerequisite for KASLR, which may place the kernel and the modules anywhere in the vmalloc area, making it more likely that branch target offsets exceed the maximum range of +/- 128 MB.

Any downside to trying to keep the kernel+modules coupled together so that we avoid the PLT?

Quoted 8 lines

First of all, note that it is unlikely that the PLTs are ever required in practice, considering that either
a) the kernel is loaded at the default location right at the base of the vmalloc range, and in this case, the module space is reserved for modules only, or
b) the kernel is loaded at some random offset in the 240+ GB vmalloc space, and it is unlikely that all VMA space around the kernel happens to be given out to non-randomized vmalloc/ioremap allocations

My worry is that we merge some code that's rarely tested.

Quoted 2 lines

So the bottom line is that it may be possible, but it is unlikely to be worth the effort imo

The alternative of having our own allocator for modules isn't that appealing either, so let's stick with the PLTs.

--
Catalin

Quoted 21 lines · Catalin Marinas

Quoted 11 lines · Ard Biesheuvel

Quoted 9 lines · Catalin Marinas

Quoted 5 lines · Ard Biesheuvel

This adds support for emitting PLTs at module load time for relative branches that are out of range. This is a prerequisite for KASLR, which may place the kernel and the modules anywhere in the vmalloc area, making it more likely that branch target offsets exceed the maximum range of +/- 128 MB.

Any downside to trying to keep the kernel+modules coupled together so that we avoid the PLT?

Quoted 8 lines

First of all, note that it is unlikely that the PLTs are ever required in practice, considering that either
a) the kernel is loaded at the default location right at the base of the vmalloc range, and in this case, the module space is reserved for modules only, or
b) the kernel is loaded at some random offset in the 240+ GB vmalloc space, and it is unlikely that all VMA space around the kernel happens to be given out to non-randomized vmalloc/ioremap allocations

Quoted 1 line

My worry is that we merge some code that's rarely tested.

I understand. But unfortunately, having corner cases that are unlikely but not impossible comes with the territory of randomization.

Alternatively, we could take the performance hit if KASLR is in effect and allocate each module completely randomly as well. This way, the code is always exercised (for now), and we can always backpedal later if the performance is measurably worse.

Quoted 27 lines · Ard Biesheuvel

Quoted 21 lines · Catalin Marinas

Quoted 11 lines · Ard Biesheuvel

Quoted 9 lines · Catalin Marinas

Quoted 5 lines · Ard Biesheuvel

This adds support for emitting PLTs at module load time for relative branches that are out of range. This is a prerequisite for KASLR, which may place the kernel and the modules anywhere in the vmalloc area, making it more likely that branch target offsets exceed the maximum range of +/- 128 MB.

Any downside to trying to keep the kernel+modules coupled together so that we avoid the PLT?

Quoted 8 lines

First of all, note that it is unlikely that the PLTs are ever required in practice, considering that either
a) the kernel is loaded at the default location right at the base of the vmalloc range, and in this case, the module space is reserved for modules only, or
b) the kernel is loaded at some random offset in the 240+ GB vmalloc space, and it is unlikely that all VMA space around the kernel happens to be given out to non-randomized vmalloc/ioremap allocations

Quoted 1 line

My worry is that we merge some code that's rarely tested.

I understand. But unfortunately, having corner cases that are unlikely but not impossible comes with the territory of randomization.

Quoted 4 lines

Alternatively, we could take the performance hit if KASLR is in effect and allocate each module completely randomly as well. This way, the code is always exercised (for now), and we can always backpedal later if the performance is measurably worse.

I'm fine with this. You can post it as a separate patch that we can easily revert/modify later (like turning it into a config option).

Quoted 33 lines · Catalin Marinas

Quoted 31 lines · Ard Biesheuvel

Quoted 21 lines · Catalin Marinas

Quoted 11 lines · Ard Biesheuvel

Quoted 9 lines · Catalin Marinas

Quoted 5 lines · Ard Biesheuvel

This adds support for emitting PLTs at module load time for relative branches that are out of range. This is a prerequisite for KASLR, which may place the kernel and the modules anywhere in the vmalloc area, making it more likely that branch target offsets exceed the maximum range of +/- 128 MB.

Any downside to trying to keep the kernel+modules coupled together so that we avoid the PLT?

Quoted 8 lines

First of all, note that it is unlikely that the PLTs are ever required in practice, considering that either
a) the kernel is loaded at the default location right at the base of the vmalloc range, and in this case, the module space is reserved for modules only, or
b) the kernel is loaded at some random offset in the 240+ GB vmalloc space, and it is unlikely that all VMA space around the kernel happens to be given out to non-randomized vmalloc/ioremap allocations

Quoted 1 line

My worry is that we merge some code that's rarely tested.

I understand. But unfortunately, having corner cases that are unlikely but not impossible comes with the territory of randomization.

Alternatively, we could take the performance hit if KASLR is in effect and allocate each module completely randomly as well. This way, the code is always exercised (for now), and we can always backpedal later if the performance is measurably worse.

Quoted 2 lines

I'm fine with this. You can post it as a separate patch that we can easily revert/modify later (like turning it into a config option).

OK, I will hack something up

Quoted 34 lines · Ard Biesheuvel

Quoted 33 lines · Catalin Marinas

Quoted 31 lines · Ard Biesheuvel

Quoted 21 lines · Catalin Marinas

Quoted 11 lines · Ard Biesheuvel

Quoted 9 lines · Catalin Marinas

Quoted 5 lines · Ard Biesheuvel

This adds support for emitting PLTs at module load time for relative branches that are out of range. This is a prerequisite for KASLR, which may place the kernel and the modules anywhere in the vmalloc area, making it more likely that branch target offsets exceed the maximum range of +/- 128 MB.

Any downside to trying to keep the kernel+modules coupled together so that we avoid the PLT?

Quoted 8 lines

First of all, note that it is unlikely that the PLTs are ever required in practice, considering that either
a) the kernel is loaded at the default location right at the base of the vmalloc range, and in this case, the module space is reserved for modules only, or
b) the kernel is loaded at some random offset in the 240+ GB vmalloc space, and it is unlikely that all VMA space around the kernel happens to be given out to non-randomized vmalloc/ioremap allocations

Quoted 1 line

My worry is that we merge some code that's rarely tested.

I understand. But unfortunately, having corner cases that are unlikely but not impossible comes with the territory of randomization.

Alternatively, we could take the performance hit if KASLR is in effect and allocate each module completely randomly as well. This way, the code is always exercised (for now), and we can always backpedal later if the performance is measurably worse.

Quoted 4 lines

Quoted 2 lines

I'm fine with this. You can post it as a separate patch that we can easily revert/modify later (like turning it into a config option).

OK, I will hack something up

If it's simpler, you can just add a config option but defaulting to the full vmalloc space for modules.

Quoted 35 lines · Catalin Marinas

Quoted 34 lines · Ard Biesheuvel

Quoted 33 lines · Catalin Marinas

Quoted 31 lines · Ard Biesheuvel

Quoted 21 lines · Catalin Marinas

Quoted 11 lines · Ard Biesheuvel

Quoted 9 lines · Catalin Marinas

Quoted 5 lines · Ard Biesheuvel

This adds support for emitting PLTs at module load time for relative branches that are out of range. This is a prerequisite for KASLR, which may place the kernel and the modules anywhere in the vmalloc area, making it more likely that branch target offsets exceed the maximum range of +/- 128 MB.

Any downside to trying to keep the kernel+modules coupled together so that we avoid the PLT?

Quoted 8 lines

First of all, note that it is unlikely that the PLTs are ever required in practice, considering that either
a) the kernel is loaded at the default location right at the base of the vmalloc range, and in this case, the module space is reserved for modules only, or
b) the kernel is loaded at some random offset in the 240+ GB vmalloc space, and it is unlikely that all VMA space around the kernel happens to be given out to non-randomized vmalloc/ioremap allocations

Quoted 1 line

My worry is that we merge some code that's rarely tested.

I understand. But unfortunately, having corner cases that are unlikely but not impossible comes with the territory of randomization.

Alternatively, we could take the performance hit if KASLR is in effect and allocate each module completely randomly as well. This way, the code is always exercised (for now), and we can always backpedal later if the performance is measurably worse.

Quoted 7 lines

Quoted 4 lines

Quoted 2 lines

I'm fine with this. You can post it as a separate patch that we can easily revert/modify later (like turning it into a config option).

OK, I will hack something up

If it's simpler, you can just add a config option but defaulting to the full vmalloc space for modules.

What would be the simplest is to randomize the 128 MB module region as a whole, and either put it close to the kernel (which is what I am doing now), or put it at a random offset inside the vmalloc space, in which case all branches will be resolved via PLTs. My suggestion to randomize each module_alloc() call separately is actually not that straight-forward.

So what I propose now is to keep a single module_load_offset that randomizes the base of the region, and add a default-n config option that shrinks the interval it is chosen from so that PLTs are usually not needed.

Quoted 35 lines · Catalin Marinas

Quoted 34 lines · Ard Biesheuvel

Quoted 33 lines · Catalin Marinas

Quoted 31 lines · Ard Biesheuvel

Quoted 21 lines · Catalin Marinas

Quoted 11 lines · Ard Biesheuvel

Quoted 9 lines · Catalin Marinas

Quoted 5 lines · Ard Biesheuvel

This adds support for emitting PLTs at module load time for relative branches that are out of range. This is a prerequisite for KASLR, which may place the kernel and the modules anywhere in the vmalloc area, making it more likely that branch target offsets exceed the maximum range of +/- 128 MB.

Any downside to trying to keep the kernel+modules coupled together so that we avoid the PLT?

Quoted 8 lines

First of all, note that it is unlikely that the PLTs are ever required in practice, considering that either
a) the kernel is loaded at the default location right at the base of the vmalloc range, and in this case, the module space is reserved for modules only, or
b) the kernel is loaded at some random offset in the 240+ GB vmalloc space, and it is unlikely that all VMA space around the kernel happens to be given out to non-randomized vmalloc/ioremap allocations

Quoted 1 line

My worry is that we merge some code that's rarely tested.

I understand. But unfortunately, having corner cases that are unlikely but not impossible comes with the territory of randomization.

Alternatively, we could take the performance hit if KASLR is in effect and allocate each module completely randomly as well. This way, the code is always exercised (for now), and we can always backpedal later if the performance is measurably worse.

Quoted 8 lines

Quoted 4 lines

Quoted 2 lines

I'm fine with this. You can post it as a separate patch that we can easily revert/modify later (like turning it into a config option).

OK, I will hack something up

If it's simpler, you can just add a config option but defaulting to the full vmalloc space for modules.

What would be the simplest is to randomize the 128 MB module region as a whole, and either put it close to the kernel (which is what I am doing now), or put it at a random offset inside the vmalloc space, in which case all branches will be resolved via PLTs. My suggestion to randomize each module_alloc() call separately is actually not that straight-forward.

This adds the option to randomize the module region independently from the core kernel, and enables it by default. This makes it less likely that the location of core kernel data structures can be determined by an adversary, but causes all function calls from modules into the core kernel to be resolved via entries in the module PLTs.

Signed-off-by Ard Biesheuvel <[email protected]>

---
 arch/arm64/Kconfig              | 15 ++++++++
 arch/arm64/include/asm/module.h |  6 ++++
 arch/arm64/kernel/kaslr.c       | 36 +++++++++++++++-----
 arch/arm64/kernel/module.c      |  9 ++---
 4 files changed, 50 insertions(+), 16 deletions(-)

Quoted 6 lines · Ard Biesheuvel

This adds the option to randomize the module region independently from the core kernel, and enables it by default. This makes it less likely that the location of core kernel data structures can be determined by an adversary, but causes all function calls from modules into the core kernel to be resolved via entries in the module PLTs.

Quoted 7 lines

Signed-off-by Ard Biesheuvel <[email protected]>

---
 arch/arm64/Kconfig              | 15 ++++++++
 arch/arm64/include/asm/module.h |  6 ++++
 arch/arm64/kernel/kaslr.c       | 36 +++++++++++++++-----
 arch/arm64/kernel/module.c      |  9 ++---
 4 files changed, 50 insertions(+), 16 deletions(-)

With this patch I get an unhandled paging request, coming from kernel/module.c:2982 (the memset). The PC is wrongly attributed but it's in arch/arm64/lib/memset.S:

[    7.140606] Unable to handle kernel paging request at virtual address 00004000
[    7.147794] pgd = ffffffc060171000
[    7.151190] [00004000] *pgd=0000000000000000, *pud=0000000000000000
[    7.157447] Internal error: Oops: 96000045 [#1] PREEMPT SMP
[    7.162962] Modules linked in:
[    7.165995] CPU: 1 PID: 875 Comm: systemd-modules Not tainted 4.5.0-rc1+ #95
[    7.172976] Hardware name: Juno (DT)
[    7.176520] task: ffffffc9760bb000 ti: ffffffc079538000 task.ti: ffffffc079538000
[    7.183939] PC is at __efistub_memset+0x1ac/0x200
[    7.188601] LR is at load_module+0xfc8/0x1df8
[    7.192912] pc : [<ffffff8008336fac>] lr : [<ffffff8008120d88>] pstate: 40000145
[    7.200233] sp : ffffffc07953bd40
[    7.203514] x29: ffffffc07953bd40 x28: 0000000000002361
[    7.208791] x27: ffffff80086bb000 x26: ffffff8008f84aa0
[    7.214054] x25: 0000000000000111 x24: 000000000000006e
[    7.219317] x23: 0000007f7bc01918 x22: ffffff8008f0e100
[    7.224580] x21: ffffff8008f4d2c0 x20: 0000000000004000
[    7.229855] x19: ffffffc07953be70 x18: 0000000000000000
[    7.235127] x17: 0000000000000000 x16: 0000000000000002
[    7.240398] x15: ffffffffffffffff x14: ffffff0000000000
[    7.245667] x13: ffffffbdc3e55340 x12: 0000000000006fff
[    7.250934] x11: ffffffc97fed46a8 x10: 0000000000000010
[    7.256198] x9 : 0000000000000000 x8 : 0000000000004000
[    7.261462] x7 : 0000000000000000 x6 : 000000000000003f
[    7.266823] x5 : 0000000000000040 x4 : 0000000000000000 [    7.271219] systemd-journald[864]: Received request to flush runtime journal from PID 1

[    7.279835]
[    7.281487] x3 : 0000000000000004 x2 : 000000000000229e
[    7.286758] x1 : 0000000000000000 x0 : 0000000000004000
[    7.292019]
[    7.293495] Process systemd-modules (pid: 875, stack limit = 0xffffffc079538020)
[    7.300822] Stack: (0xffffffc07953bd40 to 0xffffffc07953c000)
[    7.306522] bd40: ffffffc07953be40 ffffff8008121de0 0000000000000000 0000000000000005
[    7.314276] bd60: 0000007f7bc01918 0000007f7bb24ad4 0000000080000000 0000000000000015
[    7.322029] bd80: 000000000000011e 0000000000000111 ffffff80086b0000 ffffffc079538000
[    7.329781] bda0: 0000000000000000 0000000000000005 0000007f7bc01918 0000007f7bb24ad4
[    7.337536] bdc0: ffffff8008f0e288 ffffff8008f84ae0 ffffff8008f0e2d8 ffffff8008f0d000
[    7.345288] bde0: ffff81a40000000f 0000000000000001 0000000000000000 0000000000077b20
[    7.353041] be00: 0000000056b8d7f8 00000000134c2b98 0000000056b8d7f8 000000001163e398
[    7.360793] be20: 0000000056b8d7f8 000000001163e398 0000000000001000 00000000000003c0
[    7.368545] be40: 0000000000000000 ffffff8008085d30 0000000000000000 0000000000000000
[    7.376298] be60: ffffffffffffffff 0000005571c2aa60 ffffff8008f0d000 0000000000077b20
[    7.384051] be80: ffffff8008f84120 ffffff8008f4b7af ffffff8008f4d2c0 0000000000001388
[    7.391803] bea0: 0000000000001dd8 0000000000000000 0000000000000000 0000002700000026
[    7.399555] bec0: 0000000000000011 000000000000000b 0000000000000005 0000007f7bc01918
[    7.407307] bee0: 0000000000000000 0000000000000005 0000000000000000 60ceffffffffffff
[    7.415060] bf00: ffffffffffffffff ffffffffffffffff 0000000000000111 0000000000000038
[    7.422812] bf20: 0101010101010101 0000000000000001 0000000000000000 ffffffffffff0000
[    7.430565] bf40: 0000007f7bc43000 0000007f7ba626b8 0000007f7bb24ab0 0000007f7bc132d8
[    7.438317] bf60: 0000005565850710 0000005571c2a8a0 0000000000000000 0000007f7bc01918
[    7.446069] bf80: 0000005571c2a920 0000000000020000 0000000000000000 0000000000000000
[    7.453821] bfa0: 0000005571c29330 0000000000000000 0000000000000000 0000007ff3bc1e80
[    7.461575] bfc0: 0000007f7bbfa1ac 0000007ff3bc1e80 0000007f7bb24ad4 0000000080000000
[    7.469327] bfe0: 0000000000000005 0000000000000111 f712e45f3fdb5baf 5d70fcf3d73b5fa3
[    7.477075] Call trace:
[    7.479494] Exception stack(0xffffffc07953bb80 to 0xffffffc07953bca0)
[    7.485871] bb80: ffffffc07953be70 0000000000004000 ffffffc07953bd40 ffffff8008336fac
[    7.493624] bba0: 0000000000400000 00000000024000c0 ffffffc975853300 00c8000000000713
[    7.501376] bbc0: ffffff80086bb000 0000000000002361 0000000000004000 0000000000000000
[    7.509128] bbe0: ffffffc07953bc60 ffffff80081885d8 ffffffc07953bca0 ffffff8008187fb8
[    7.516880] bc00: 0000000000000003 ffffffc975853480 00000000ffffffff 00000000024002c0
[    7.524631] bc20: 0000000000004000 0000000000000000 000000000000229e 0000000000000004
[    7.532383] bc40: 0000000000000000 0000000000000040 000000000000003f 0000000000000000
[    7.540135] bc60: 0000000000004000 0000000000000000 0000000000000010 ffffffc97fed46a8
[    7.547888] bc80: 0000000000006fff ffffffbdc3e55340 ffffff0000000000 ffffffffffffffff
[    7.555646] [<ffffff8008336fac>] __efistub_memset+0x1ac/0x200
[    7.561334] [<ffffff8008121de0>] SyS_finit_module+0xb0/0xc0
[    7.566852] [<ffffff8008085d30>] el0_svc_naked+0x24/0x28
[    7.572112] Code: 91010108 54ffff4a 8b040108 cb050042 (d50b7428)
[    7.578196] ---[ end trace 13bd770b734da68a ]---

--
Catalin

Quoted 6 lines · Ard Biesheuvel

This adds the option to randomize the module region independently from the core kernel, and enables it by default. This makes it less likely that the location of core kernel data structures can be determined by an adversary, but causes all function calls from modules into the core kernel to be resolved via entries in the module PLTs.

Quoted 7 lines

Signed-off-by Ard Biesheuvel <[email protected]>

---
 arch/arm64/Kconfig              | 15 ++++++++
 arch/arm64/include/asm/module.h |  6 ++++
 arch/arm64/kernel/kaslr.c       | 36 +++++++++++++++-----
 arch/arm64/kernel/module.c      |  9 ++---
 4 files changed, 50 insertions(+), 16 deletions(-)

With this patch I get an unhandled paging request, coming from kernel/module.c:2982 (the memset). The PC is wrongly attributed but it's in arch/arm64/lib/memset.S:

Quoted 15 lines · Catalin Marinas

Quoted 6 lines · Ard Biesheuvel

This adds the option to randomize the module region independently from the core kernel, and enables it by default. This makes it less likely that the location of core kernel data structures can be determined by an adversary, but causes all function calls from modules into the core kernel to be resolved via entries in the module PLTs.

Quoted 7 lines

Signed-off-by Ard Biesheuvel <[email protected]>

---
 arch/arm64/Kconfig              | 15 ++++++++
 arch/arm64/include/asm/module.h |  6 ++++
 arch/arm64/kernel/kaslr.c       | 36 +++++++++++++++-----
 arch/arm64/kernel/module.c      |  9 ++---
 4 files changed, 50 insertions(+), 16 deletions(-)

Quoted 4 lines

With this patch I get an unhandled paging request, coming from kernel/module.c:2982 (the memset). The PC is wrongly attributed but it's in arch/arm64/lib/memset.S:

This was due to a thinko in kaslr_early_init(). Folding the following hunk will fix it (or I can resend the patch if you prefer)

-------8<----------

arch/arm64/kernel/kaslr.c +0 -0

index f8ad30160026..d7eba99ab140 100644
@@ -82,6 +82,12 @@ u64 __init kaslr_early_init(u64 dt_phys)
        int size;
 
        /*

This adds support for emitting PLTs at module load time for relative branches that are out of range. This is a prerequisite for KASLR, which may place the kernel and the modules anywhere in the vmalloc area, making it more likely that branch target offsets exceed the maximum range of +/- 128 MB.

Signed-off-by Ard Biesheuvel <[email protected]>

---

In this version, I removed the distinction between relocations against .init executable sections and ordinary executable sections. The reason is that it is hardly worth the trouble, given that .init.text usually does not contain that many far branches, and this version now only reserves PLT entry space for jump and call relocations against undefined symbols (since symbols defined in the same module can be assumed to be within +/- 128 MB)

For example, the mac80211.ko module (which is fairly sizable at ~400 KB) built with -mcmodel=large gives the following relocation counts:

relocs branches unique !local
.text 3925 3347 518 219

  .init.text           11          8         7          1
  .exit.text            4          4         4          1

.text.unlikely 81 67 36 17

('unique' means branches to unique type/symbol/addend combos, of which !local is the subset referring to undefined symbols)

IOW, we are only emitting a single PLT entry for the .init sections, and we are better off just adding it to the core PLT section instead.

---
 arch/arm64/Kconfig              |   9 +
 arch/arm64/Makefile             |   6 +-
 arch/arm64/include/asm/module.h |  11 ++
 arch/arm64/kernel/Makefile      |   1 +
 arch/arm64/kernel/module-plts.c | 201 ++++++++++++++++++++
 arch/arm64/kernel/module.c      |  12 ++
 arch/arm64/kernel/module.lds    |   3 +
 7 files changed, 242 insertions(+), 1 deletion(-)

arch/arm64/kernel/module-plts.c new file +4 -0

+struct plt_entry {
+	/*
+	 * A program that conforms to the AArch64 Procedure Call Standard
+	 * (AAPCS64) must assume that a veneer that alters IP0 (x16) and/or

Hi Ard,

Quoted 40 lines · Ard Biesheuvel

This adds support for emitting PLTs at module load time for relative branches that are out of range. This is a prerequisite for KASLR, which may place the kernel and the modules anywhere in the vmalloc area, making it more likely that branch target offsets exceed the maximum range of +/- 128 MB.

Signed-off-by Ard Biesheuvel <[email protected]>

---

In this version, I removed the distinction between relocations against .init executable sections and ordinary executable sections. The reason is that it is hardly worth the trouble, given that .init.text usually does not contain that many far branches, and this version now only reserves PLT entry space for jump and call relocations against undefined symbols (since symbols defined in the same module can be assumed to be within +/- 128 MB)

For example, the mac80211.ko module (which is fairly sizable at ~400 KB) built with -mcmodel=large gives the following relocation counts:

relocs branches unique !local
.text 3925 3347 518 219

  .init.text           11          8         7          1
  .exit.text            4          4         4          1

.text.unlikely 81 67 36 17

('unique' means branches to unique type/symbol/addend combos, of which !local is the subset referring to undefined symbols)

IOW, we are only emitting a single PLT entry for the .init sections, and we are better off just adding it to the core PLT section instead.

---
 arch/arm64/Kconfig              |   9 +
 arch/arm64/Makefile             |   6 +-
 arch/arm64/include/asm/module.h |  11 ++
 arch/arm64/kernel/Makefile      |   1 +
 arch/arm64/kernel/module-plts.c | 201 ++++++++++++++++++++
 arch/arm64/kernel/module.c      |  12 ++
 arch/arm64/kernel/module.lds    |   3 +
 7 files changed, 242 insertions(+), 1 deletion(-)

[...]

Quoted 4 lines

arch/arm64/kernel/module-plts.c new file +4 -0

+struct plt_entry {
+     /*
+      * A program that conforms to the AArch64 Procedure Call Standard
+      * (AAPCS64) must assume that a veneer that alters IP0 (x16) and/or

arch/arm64/kernel/module-plts.c line 27

Quoted 10 lines

arch/arm64/kernel/module-plts.c new file +10 -0

+      * IP1 (x17) may be inserted at any branch instruction that is
+      * exposed to a relocation that supports long branches. Since that
+      * is exactly what we are dealing with here, we are free to use x16
+      * as a scratch register in the PLT veneers.
+      */
+     __le32  mov0;   /* movn x16, #0x....                    */
+     __le32  mov1;   /* movk x16, #0x...., lsl #16           */
+     __le32  mov2;   /* movk x16, #0x...., lsl #32           */
+     __le32  br;     /* br   x16                             */
+};

I'm worried about this code when CONFIG_ARM64_LSE_ATOMICS=y, but we don't detect them on the CPU at runtime. In this case, all atomic operations are moved out-of-line and called using a bl instruction from inline asm.

The out-of-line code is compiled with magic GCC options

to force the
explicit save/restore of all used registers (see arch/arm64/lib/Makefile), otherwise we'd have to clutter the inline asm with constraints that wouldn't be needed had we managed to patch the bl with an LSE atomic instruction.

If you're emitting a PLT, couldn't we end up with silent corruption of x16 for modules using out-of-line atomics like this?

Quoted 65 lines · Will Deacon

Hi Ard,

Quoted 40 lines · Ard Biesheuvel

This adds support for emitting PLTs at module load time for relative branches that are out of range. This is a prerequisite for KASLR, which may place the kernel and the modules anywhere in the vmalloc area, making it more likely that branch target offsets exceed the maximum range of +/- 128 MB.

Signed-off-by Ard Biesheuvel <[email protected]>

---

In this version, I removed the distinction between relocations against .init executable sections and ordinary executable sections. The reason is that it is hardly worth the trouble, given that .init.text usually does not contain that many far branches, and this version now only reserves PLT entry space for jump and call relocations against undefined symbols (since symbols defined in the same module can be assumed to be within +/- 128 MB)

For example, the mac80211.ko module (which is fairly sizable at ~400 KB) built with -mcmodel=large gives the following relocation counts:

relocs branches unique !local
.text 3925 3347 518 219

  .init.text           11          8         7          1
  .exit.text            4          4         4          1

.text.unlikely 81 67 36 17

('unique' means branches to unique type/symbol/addend combos, of which !local is the subset referring to undefined symbols)

IOW, we are only emitting a single PLT entry for the .init sections, and we are better off just adding it to the core PLT section instead.

---
 arch/arm64/Kconfig              |   9 +
 arch/arm64/Makefile             |   6 +-
 arch/arm64/include/asm/module.h |  11 ++
 arch/arm64/kernel/Makefile      |   1 +
 arch/arm64/kernel/module-plts.c | 201 ++++++++++++++++++++
 arch/arm64/kernel/module.c      |  12 ++
 arch/arm64/kernel/module.lds    |   3 +
 7 files changed, 242 insertions(+), 1 deletion(-)

[...]

Quoted 4 lines

arch/arm64/kernel/module-plts.c new file +4 -0

+struct plt_entry {
+     /*
+      * A program that conforms to the AArch64 Procedure Call Standard
+      * (AAPCS64) must assume that a veneer that alters IP0 (x16) and/or

arch/arm64/kernel/module-plts.c line 27

Quoted 10 lines

arch/arm64/kernel/module-plts.c new file +10 -0

+      * IP1 (x17) may be inserted at any branch instruction that is
+      * exposed to a relocation that supports long branches. Since that
+      * is exactly what we are dealing with here, we are free to use x16
+      * as a scratch register in the PLT veneers.
+      */
+     __le32  mov0;   /* movn x16, #0x....                    */
+     __le32  mov1;   /* movk x16, #0x...., lsl #16           */
+     __le32  mov2;   /* movk x16, #0x...., lsl #32           */
+     __le32  br;     /* br   x16                             */
+};

I'm worried about this code when CONFIG_ARM64_LSE_ATOMICS=y, but we don't detect them on the CPU at runtime. In this case, all atomic operations are moved out-of-line and called using a bl instruction from inline asm.

Quoted 1 line

The out-of-line code is compiled with magic GCC options

Which options are those exactly?

Quoted 6 lines

to force the
explicit save/restore of all used registers (see arch/arm64/lib/Makefile), otherwise we'd have to clutter the inline asm with constraints that wouldn't be needed had we managed to patch the bl with an LSE atomic instruction.

Quoted 3 lines

If you're emitting a PLT, couldn't we end up with silent corruption of x16 for modules using out-of-line atomics like this?

If you violate the AAPCS64 ABI, then obviously the claim above does not hold.

Quoted 20 lines · Will Deacon

Quoted 4 lines · Ard Biesheuvel

arch/arm64/kernel/module-plts.c new file +4 -0

+struct plt_entry {
+     /*
+      * A program that conforms to the AArch64 Procedure Call Standard
+      * (AAPCS64) must assume that a veneer that alters IP0 (x16) and/or

arch/arm64/kernel/module-plts.c line 27

Quoted 10 lines

arch/arm64/kernel/module-plts.c new file +10 -0

+      * IP1 (x17) may be inserted at any branch instruction that is
+      * exposed to a relocation that supports long branches. Since that
+      * is exactly what we are dealing with here, we are free to use x16
+      * as a scratch register in the PLT veneers.
+      */
+     __le32  mov0;   /* movn x16, #0x....                    */
+     __le32  mov1;   /* movk x16, #0x...., lsl #16           */
+     __le32  mov2;   /* movk x16, #0x...., lsl #32           */
+     __le32  br;     /* br   x16                             */
+};

I'm worried about this code when CONFIG_ARM64_LSE_ATOMICS=y, but we don't detect them on the CPU at runtime. In this case, all atomic operations are moved out-of-line and called using a bl instruction from inline asm.

Quoted 1 line

The out-of-line code is compiled with magic GCC options

Which options are those exactly?

Quoted 9 lines

to force the
explicit save/restore of all used registers (see arch/arm64/lib/Makefile), otherwise we'd have to clutter the inline asm with constraints that wouldn't be needed had we managed to patch the bl with an LSE atomic instruction.

If you're emitting a PLT, couldn't we end up with silent corruption of x16 for modules using out-of-line atomics like this?

If you violate the AAPCS64 ABI, then obviously the claim above does not hold.

Quoted 21 lines · Ard Biesheuvel

Quoted 20 lines · Will Deacon

Quoted 4 lines · Ard Biesheuvel

arch/arm64/kernel/module-plts.c new file +4 -0

+struct plt_entry {
+     /*
+      * A program that conforms to the AArch64 Procedure Call Standard
+      * (AAPCS64) must assume that a veneer that alters IP0 (x16) and/or

arch/arm64/kernel/module-plts.c line 27

Quoted 10 lines

arch/arm64/kernel/module-plts.c new file +10 -0

+      * IP1 (x17) may be inserted at any branch instruction that is
+      * exposed to a relocation that supports long branches. Since that
+      * is exactly what we are dealing with here, we are free to use x16
+      * as a scratch register in the PLT veneers.
+      */
+     __le32  mov0;   /* movn x16, #0x....                    */
+     __le32  mov1;   /* movk x16, #0x...., lsl #16           */
+     __le32  mov2;   /* movk x16, #0x...., lsl #32           */
+     __le32  br;     /* br   x16                             */
+};

I'm worried about this code when CONFIG_ARM64_LSE_ATOMICS=y, but we don't detect them on the CPU at runtime. In this case, all atomic operations are moved out-of-line and called using a bl instruction from inline asm.

Quoted 3 lines

Quoted 1 line

The out-of-line code is compiled with magic GCC options

Which options are those exactly?

# Tell the compiler to treat all general purpose registers as # callee-saved, which allows for efficient runtime patching of the bl

# instruction in the caller with an atomic instruction when supported by
# the CPU. Result and argument registers are handled correctly, based on
# the function prototype.
lib-$(CONFIG_ARM64_LSE_ATOMICS) += atomic_ll_sc.o
CFLAGS_atomic_ll_sc.o   := -fcall-used-x0 -ffixed-x1 -ffixed-x2         \
                   -ffixed-x3 -ffixed-x4 -ffixed-x5 -ffixed-x6          \
                   -ffixed-x7 -fcall-saved-x8 -fcall-saved-x9           \
                   -fcall-saved-x10 -fcall-saved-x11 -fcall-saved-x12   \
                   -fcall-saved-x13 -fcall-saved-x14 -fcall-saved-x15   \
                   -fcall-saved-x16 -fcall-saved-x17 -fcall-saved-x18

Quoted 10 lines

Quoted 9 lines

to force the
explicit save/restore of all used registers (see arch/arm64/lib/Makefile), otherwise we'd have to clutter the inline asm with constraints that wouldn't be needed had we managed to patch the bl with an LSE atomic instruction.

If you're emitting a PLT, couldn't we end up with silent corruption of x16 for modules using out-of-line atomics like this?

Quoted 1 line

If you violate the AAPCS64 ABI, then obviously the claim above does not hold.

Indeed, but this is what mainline is doing today and I'm not keen on breaking it. One way to fix it would be to generate a different type of plt for branches to the atomic functions that would use the stack instead of x16.

Quoted 28 lines · Will Deacon

Quoted 21 lines · Ard Biesheuvel

Quoted 20 lines · Will Deacon

Quoted 4 lines · Ard Biesheuvel

arch/arm64/kernel/module-plts.c new file +4 -0

+struct plt_entry {
+     /*
+      * A program that conforms to the AArch64 Procedure Call Standard
+      * (AAPCS64) must assume that a veneer that alters IP0 (x16) and/or

arch/arm64/kernel/module-plts.c line 27

Quoted 10 lines

arch/arm64/kernel/module-plts.c new file +10 -0

+      * IP1 (x17) may be inserted at any branch instruction that is
+      * exposed to a relocation that supports long branches. Since that
+      * is exactly what we are dealing with here, we are free to use x16
+      * as a scratch register in the PLT veneers.
+      */
+     __le32  mov0;   /* movn x16, #0x....                    */
+     __le32  mov1;   /* movk x16, #0x...., lsl #16           */
+     __le32  mov2;   /* movk x16, #0x...., lsl #32           */
+     __le32  br;     /* br   x16                             */
+};

I'm worried about this code when CONFIG_ARM64_LSE_ATOMICS=y, but we don't detect them on the CPU at runtime. In this case, all atomic operations are moved out-of-line and called using a bl instruction from inline asm.

Quoted 3 lines

Quoted 1 line

The out-of-line code is compiled with magic GCC options

Which options are those exactly?

# Tell the compiler to treat all general purpose registers as # callee-saved, which allows for efficient runtime patching of the bl

Quoted 11 lines

# instruction in the caller with an atomic instruction when supported by
# the CPU. Result and argument registers are handled correctly, based on
# the function prototype.
lib-$(CONFIG_ARM64_LSE_ATOMICS) += atomic_ll_sc.o
CFLAGS_atomic_ll_sc.o   := -fcall-used-x0 -ffixed-x1 -ffixed-x2         \
                   -ffixed-x3 -ffixed-x4 -ffixed-x5 -ffixed-x6          \
                   -ffixed-x7 -fcall-saved-x8 -fcall-saved-x9           \
                   -fcall-saved-x10 -fcall-saved-x11 -fcall-saved-x12   \
                   -fcall-saved-x13 -fcall-saved-x14 -fcall-saved-x15   \
                   -fcall-saved-x16 -fcall-saved-x17 -fcall-saved-x18

Yikes. Is that safe?

Quoted 10 lines

Quoted 10 lines

Quoted 9 lines

to force the
explicit save/restore of all used registers (see arch/arm64/lib/Makefile), otherwise we'd have to clutter the inline asm with constraints that wouldn't be needed had we managed to patch the bl with an LSE atomic instruction.

If you're emitting a PLT, couldn't we end up with silent corruption of x16 for modules using out-of-line atomics like this?

Quoted 7 lines

Quoted 1 line

If you violate the AAPCS64 ABI, then obviously the claim above does not hold.

Indeed, but this is what mainline is doing today and I'm not keen on breaking it. One way to fix it would be to generate a different type of plt for branches to the atomic functions that would use the stack instead of x16.

AFAIK, gcc never uses x18 (the platform register) so we may be able to use that instead. We'd need confirmation from the toolchain guys, though ...

Quoted 41 lines · Ard Biesheuvel

Quoted 28 lines · Will Deacon

Quoted 21 lines · Ard Biesheuvel

Quoted 20 lines · Will Deacon

Quoted 4 lines · Ard Biesheuvel

arch/arm64/kernel/module-plts.c new file +4 -0

+struct plt_entry {
+     /*
+      * A program that conforms to the AArch64 Procedure Call Standard
+      * (AAPCS64) must assume that a veneer that alters IP0 (x16) and/or

arch/arm64/kernel/module-plts.c line 27

Quoted 10 lines

arch/arm64/kernel/module-plts.c new file +10 -0

+      * IP1 (x17) may be inserted at any branch instruction that is
+      * exposed to a relocation that supports long branches. Since that
+      * is exactly what we are dealing with here, we are free to use x16
+      * as a scratch register in the PLT veneers.
+      */
+     __le32  mov0;   /* movn x16, #0x....                    */
+     __le32  mov1;   /* movk x16, #0x...., lsl #16           */
+     __le32  mov2;   /* movk x16, #0x...., lsl #32           */
+     __le32  br;     /* br   x16                             */
+};

I'm worried about this code when CONFIG_ARM64_LSE_ATOMICS=y, but we don't detect them on the CPU at runtime. In this case, all atomic operations are moved out-of-line and called using a bl instruction from inline asm.

Quoted 3 lines

Quoted 1 line

The out-of-line code is compiled with magic GCC options

Which options are those exactly?

# Tell the compiler to treat all general purpose registers as # callee-saved, which allows for efficient runtime patching of the bl

Quoted 11 lines

# instruction in the caller with an atomic instruction when supported by
# the CPU. Result and argument registers are handled correctly, based on
# the function prototype.
lib-$(CONFIG_ARM64_LSE_ATOMICS) += atomic_ll_sc.o
CFLAGS_atomic_ll_sc.o   := -fcall-used-x0 -ffixed-x1 -ffixed-x2         \
                   -ffixed-x3 -ffixed-x4 -ffixed-x5 -ffixed-x6          \
                   -ffixed-x7 -fcall-saved-x8 -fcall-saved-x9           \
                   -fcall-saved-x10 -fcall-saved-x11 -fcall-saved-x12   \
                   -fcall-saved-x13 -fcall-saved-x14 -fcall-saved-x15   \
                   -fcall-saved-x16 -fcall-saved-x17 -fcall-saved-x18

Quoted 1 line

Yikes. Is that safe?

It seems to work, and x86 uses a similar trick for it's hweight code.

Quoted 18 lines

Quoted 10 lines

Quoted 10 lines

Quoted 9 lines

to force the
explicit save/restore of all used registers (see arch/arm64/lib/Makefile), otherwise we'd have to clutter the inline asm with constraints that wouldn't be needed had we managed to patch the bl with an LSE atomic instruction.

If you're emitting a PLT, couldn't we end up with silent corruption of x16 for modules using out-of-line atomics like this?

Quoted 7 lines

Quoted 1 line

If you violate the AAPCS64 ABI, then obviously the claim above does not hold.

Indeed, but this is what mainline is doing today and I'm not keen on breaking it. One way to fix it would be to generate a different type of plt for branches to the atomic functions that would use the stack instead of x16.

Quoted 3 lines

AFAIK, gcc never uses x18 (the platform register) so we may be able to use that instead. We'd need confirmation from the toolchain guys, though ...

In fact, a better thing to do is probably for the atomic code to save/restore those register explicitly and then remove them from the cflags above.

I'll try hacking something together...

Quoted 9 lines · Will Deacon

Quoted 3 lines · Ard Biesheuvel

AFAIK, gcc never uses x18 (the platform register) so we may be able to use that instead. We'd need confirmation from the toolchain guys, though ...

In fact, a better thing to do is probably for the atomic code to save/restore those register explicitly and then remove them from the cflags above.

Quoted 2 lines

I'll try hacking something together...

That would be more correct, indeed. Also, the PLT can only use br <reg> so it will always enter the callee with the preserved value stacked, and so the callee needs to be modified in any case.

Quoted 13 lines · Ard Biesheuvel

Quoted 11 lines · Will Deacon

Quoted 3 lines · Ard Biesheuvel

AFAIK, gcc never uses x18 (the platform register) so we may be able to use that instead. We'd need confirmation from the toolchain guys, though ...

In fact, a better thing to do is probably for the atomic code to save/restore those register explicitly and then remove them from the cflags above.

I'll try hacking something together...

Quoted 3 lines

That would be more correct, indeed. Also, the PLT can only use br <reg> so it will always enter the callee with the preserved value stacked, and so the callee needs to be modified in any case.

Wait -- why does the callee need to be modified here? It has no idea about whether or not it was invoked via a PLT.

Also, can I rely on the PLT not clobbering the condition code?

Quoted 14 lines · Will Deacon

Quoted 13 lines · Ard Biesheuvel

Quoted 11 lines · Will Deacon

Quoted 3 lines · Ard Biesheuvel

AFAIK, gcc never uses x18 (the platform register) so we may be able to use that instead. We'd need confirmation from the toolchain guys, though ...

In fact, a better thing to do is probably for the atomic code to save/restore those register explicitly and then remove them from the cflags above.

I'll try hacking something together...

Quoted 7 lines

Quoted 3 lines

That would be more correct, indeed. Also, the PLT can only use br <reg> so it will always enter the callee with the preserved value stacked, and so the callee needs to be modified in any case.

Wait -- why does the callee need to be modified here? It has no idea about whether or not it was invoked via a PLT.

No, it doesn't, and that is exactly the problem: the PLT must end in a br <reg> instruction because that is the only branch instruction with unlimited range. That means you will always enter the callee with its address in some register rather than the value that should have been preserved. I don't see a way to manage that from the callee

Quoted 2 lines

Also, can I rely on the PLT not clobbering the condition code?

For my particular implementation, yes.

Quoted 23 lines · Ard Biesheuvel

Quoted 21 lines · Will Deacon

Quoted 10 lines · Ard Biesheuvel

Quoted 9 lines · Will Deacon

Quoted 3 lines · Ard Biesheuvel

AFAIK, gcc never uses x18 (the platform register) so we may be able to use that instead. We'd need confirmation from the toolchain guys, though ...

In fact, a better thing to do is probably for the atomic code to save/restore those register explicitly and then remove them from the cflags above.

Quoted 6 lines

Quoted 2 lines

I'll try hacking something together...

That would be more correct, indeed. Also, the PLT can only use br <reg> so it will always enter the callee with the preserved value stacked, and so the callee needs to be modified in any case.

Wait -- why does the callee need to be modified here? It has no idea about whether or not it was invoked via a PLT.

Quoted 6 lines

No, it doesn't, and that is exactly the problem: the PLT must end in a br <reg> instruction because that is the only branch instruction with unlimited range. That means you will always enter the callee with its address in some register rather than the value that should have been preserved. I don't see a way to manage that from the callee

*caller! dammit

Quoted 31 lines · Ard Biesheuvel

Quoted 29 lines · Ard Biesheuvel

Quoted 14 lines · Will Deacon

Quoted 13 lines · Ard Biesheuvel

Quoted 11 lines · Will Deacon

Quoted 3 lines · Ard Biesheuvel

AFAIK, gcc never uses x18 (the platform register) so we may be able to use that instead. We'd need confirmation from the toolchain guys, though ...

In fact, a better thing to do is probably for the atomic code to save/restore those register explicitly and then remove them from the cflags above.

I'll try hacking something together...

Quoted 7 lines

Quoted 3 lines

That would be more correct, indeed. Also, the PLT can only use br <reg> so it will always enter the callee with the preserved value stacked, and so the callee needs to be modified in any case.

Wait -- why does the callee need to be modified here? It has no idea about whether or not it was invoked via a PLT.

No, it doesn't, and that is exactly the problem: the PLT must end in a br <reg> instruction because that is the only branch instruction with unlimited range. That means you will always enter the callee with its address in some register rather than the value that should have been preserved. I don't see a way to manage that from the callee

Quoted 1 line

*caller! dammit

The caller can do:

save x16, 17
bl plt
restore x16, x17

the plt will do:

get_addr_of_callee_into_x16_and_clobber_x17_or_something br callee

then the callee will be compiled with all those weird options, but *not* the ones specifying x16 and x17. That means it can happily use those guys as scratch, because the caller will take care of them.

Quoted 5 lines · Will Deacon

the plt will do:

get_addr_of_callee_into_x16_and_clobber_x17_or_something br callee

Quoted 4 lines

then the callee will be compiled with all those weird options, but *not* the ones specifying x16 and x17. That means it can happily use those guys as scratch, because the caller will take care of them.

Yes, that makes sense. But if you don't relax that restriction, you only need the alternative __LL_SC_CALL for modules.

At the request of Catalin, this series has been split off from my series 'arm64: implement support for KASLR v4' [1]. This sub-series deals with creating a relocatable binary, and randomizing the virtual placement of the kernel itself, the modules and the linear region based on entropy supplied by the bootloader in the /chosen/kaslr-seed DT property.

Changes since v4:
- add randomization of the linear region, i.e., if the linear region is substantially larger than the space spanned by RAM, the mapping of RAM is moved to a random offset inside the linear region.
- dropped the CRC check in kaslr_early_init(), since the only code that may modify .data through __fixmap_remap_fdt() is instrumentation that should deal with that correctly (For instance, the branch profiling records taken branches in structs allocated in .data, and these counts will not be reset to 0 between the first and the second call to __mmap_switched())
- add Mark's ack to patch #4

Patch #1 adds support for resolving cross-module branches via veneers if the module is located too far from the target. This should not be necessary in the common case even with KASRL enabled, but since the space around the kernel image will no longer be strictly reserved for modules, there is a slight chance that the region may be exhausted, in which case modules will be moved further away from the kernel (and each other)

Patches #2 and #3 deal with the fact that relocations of 64-bit values are deferred to runtime when building a PIE executable, and some values are required before the relocation routine executes (either in the code, or as an externally visible constant in the Image header)

Patch #4 rehuffles asm/elf.h so we can use its preprocessor constants in head.S

Patch #5 and #6 implement building vmlinux as a PIE (Position Independent Executable) binary, which is emitted with relocation information that is processed by the program itself in the early boot code.

Patch #7 implements KASLR, i.e., CONFIG_RANDOMIZE_BASE for the core kernel and for the module region.

Patch #8 implements KASLR for the linear region.

[1] http://thread.gmane.org/gmane.linux.kernel/2135931

Ard Biesheuvel (8):
arm64: add support for module PLTs
arm64: avoid R_AARCH64_ABS64 relocations for Image header fields
arm64: avoid dynamic relocations in early boot code
arm64: make asm/elf.h available to asm files scripts/sortextable: add support for ET_DYN binaries
arm64: add support for building vmlinux as a relocatable PIE binary
arm64: add support for kernel ASLR
arm64: kaslr: randomize the linear region

 arch/arm64/Kconfig                 |  34 ++++
 arch/arm64/Makefile                |  10 +-
 arch/arm64/include/asm/assembler.h |  11 ++
 arch/arm64/include/asm/elf.h       |  24 ++-
 arch/arm64/include/asm/memory.h    |   5 +-
 arch/arm64/include/asm/module.h    |  11 ++
 arch/arm64/kernel/Makefile         |   2 +
 arch/arm64/kernel/efi-entry.S      |   2 +-
 arch/arm64/kernel/head.S           | 128 +++++++++++--
 arch/arm64/kernel/image.h          |  32 ++--
 arch/arm64/kernel/kaslr.c          | 143 ++++++++++++++
 arch/arm64/kernel/module-plts.c    | 201 ++++++++++++++++++++
 arch/arm64/kernel/module.c         |  20 +-
 arch/arm64/kernel/module.lds       |   3 +
 arch/arm64/kernel/setup.c          |  29 +++
 arch/arm64/kernel/vmlinux.lds.S    |  16 ++
 arch/arm64/mm/init.c               |  22 ++-
 arch/arm64/mm/mmu.c                |  33 +++-
 scripts/sortextable.c              |   8 +-
 19 files changed, 672 insertions(+), 62 deletions(-)
 create mode 100644 arch/arm64/kernel/kaslr.c
 create mode 100644 arch/arm64/kernel/module-plts.c
 create mode 100644 arch/arm64/kernel/module.lds

--
2.5.0

Quoted 6 lines · Ard Biesheuvel

At the request of Catalin, this series has been split off from my series 'arm64: implement support for KASLR v4' [1]. This sub-series deals with creating a relocatable binary, and randomizing the virtual placement of the kernel itself, the modules and the linear region based on entropy supplied by the bootloader in the /chosen/kaslr-seed DT property.

Quoted 11 lines

Changes since v4:
- add randomization of the linear region, i.e., if the linear region is substantially larger than the space spanned by RAM, the mapping of RAM is moved to a random offset inside the linear region.
- dropped the CRC check in kaslr_early_init(), since the only code that may modify .data through __fixmap_remap_fdt() is instrumentation that should deal with that correctly (For instance, the branch profiling records taken branches in structs allocated in .data, and these counts will not be reset to 0 between the first and the second call to __mmap_switched())
- add Mark's ack to patch #4

NOTE: I have omitted the relative kallsyms and relative extable patches, since they already queued in akpm's tree. However, while the kallsyms patch is merely an optimization, the extable patches are in fact required for correct operation, since the build time sorting does not work on absolute extables (this is due to the fact that the resolution of the relocations is deferred until runtime, and so the extable addresses are all zero at sorting time)

http://ozlabs.org/~akpm/mmotm/broken-out/extable-add-support-for-relative-extables-to-search-and-sort-routines.patch
http://ozlabs.org/~akpm/mmotm/broken-out/arm64-switch-to-relative-exception-tables.patch

Quoted 27 lines · Ard Biesheuvel

Quoted 6 lines · Ard Biesheuvel

At the request of Catalin, this series has been split off from my series 'arm64: implement support for KASLR v4' [1]. This sub-series deals with creating a relocatable binary, and randomizing the virtual placement of the kernel itself, the modules and the linear region based on entropy supplied by the bootloader in the /chosen/kaslr-seed DT property.

Quoted 11 lines

Changes since v4:
- add randomization of the linear region, i.e., if the linear region is substantially larger than the space spanned by RAM, the mapping of RAM is moved to a random offset inside the linear region.
- dropped the CRC check in kaslr_early_init(), since the only code that may modify .data through __fixmap_remap_fdt() is instrumentation that should deal with that correctly (For instance, the branch profiling records taken branches in structs allocated in .data, and these counts will not be reset to 0 between the first and the second call to __mmap_switched())
- add Mark's ack to patch #4

NOTE: I have omitted the relative kallsyms and relative extable patches, since they already queued in akpm's tree. However, while the kallsyms patch is merely an optimization, the extable patches are in fact required for correct operation, since the build time sorting does not work on absolute extables (this is due to the fact that the resolution of the relocations is deferred until runtime, and so the extable addresses are all zero at sorting time)

Quoted 2 lines

http://ozlabs.org/~akpm/mmotm/broken-out/extable-add-support-for-relative-extables-to-search-and-sort-routines.patch
http://ozlabs.org/~akpm/mmotm/broken-out/arm64-switch-to-relative-exception-tables.patch

I'm still trying to get my head around how we merge those. Since I assume akpm will push them during the merging window, part of your code cannot be tested before.

Can we deselect CONFIG_BUILDTIME_EXTABLE_SORT temporarily while we use absolute extable addresses? We patch them at boot-time and sort them at run-time. Once the above patches go in, we can revert to build-time extable sorting.

Quoted 28 lines · Catalin Marinas

Quoted 27 lines · Ard Biesheuvel

Quoted 6 lines · Ard Biesheuvel

At the request of Catalin, this series has been split off from my series 'arm64: implement support for KASLR v4' [1]. This sub-series deals with creating a relocatable binary, and randomizing the virtual placement of the kernel itself, the modules and the linear region based on entropy supplied by the bootloader in the /chosen/kaslr-seed DT property.

Quoted 11 lines

Changes since v4:
- add randomization of the linear region, i.e., if the linear region is substantially larger than the space spanned by RAM, the mapping of RAM is moved to a random offset inside the linear region.
- dropped the CRC check in kaslr_early_init(), since the only code that may modify .data through __fixmap_remap_fdt() is instrumentation that should deal with that correctly (For instance, the branch profiling records taken branches in structs allocated in .data, and these counts will not be reset to 0 between the first and the second call to __mmap_switched())
- add Mark's ack to patch #4

NOTE: I have omitted the relative kallsyms and relative extable patches, since they already queued in akpm's tree. However, while the kallsyms patch is merely an optimization, the extable patches are in fact required for correct operation, since the build time sorting does not work on absolute extables (this is due to the fact that the resolution of the relocations is deferred until runtime, and so the extable addresses are all zero at sorting time)

Quoted 6 lines

Quoted 2 lines

http://ozlabs.org/~akpm/mmotm/broken-out/extable-add-support-for-relative-extables-to-search-and-sort-routines.patch
http://ozlabs.org/~akpm/mmotm/broken-out/arm64-switch-to-relative-exception-tables.patch

I'm still trying to get my head around how we merge those. Since I assume akpm will push them during the merging window, part of your code cannot be tested before.

Actually, my original idea was for akpm to take them as a late merge after rebasing to -rc1, since they touch a variety of architectures, but I am not sure if that came across.

You could always take the series through your tree instead, I guess?

Quoted 4 lines

Can we deselect CONFIG_BUILDTIME_EXTABLE_SORT temporarily while we use absolute extable addresses? We patch them at boot-time and sort them at run-time. Once the above patches go in, we can revert to build-time extable sorting.

I suppose that would be possible, but I simply haven't tried. I noticed that ppc does not use build time extable sorting, that is why they don't need to scripts/extable patch while they do use PIE binaries.

Quoted 3 lines · Catalin Marinas

I'm still trying to get my head around how we merge those. Since I assume akpm will push them during the merging window, part of your code cannot be tested before.

Actually, my original idea was for akpm to take them as a late merge after rebasing to -rc1, since they touch a variety of architectures, but I am not sure if that came across.

You could always take the series through your tree instead, I guess?

Quoted 9 lines · Ard Biesheuvel

Quoted 3 lines · Catalin Marinas

I'm still trying to get my head around how we merge those. Since I assume akpm will push them during the merging window, part of your code cannot be tested before.

Actually, my original idea was for akpm to take them as a late merge after rebasing to -rc1, since they touch a variety of architectures, but I am not sure if that came across.

Quoted 1 line

You could always take the series through your tree instead, I guess?

Traditionally akpm will de-duplicate patches he's carrying that appear in another tree. I think it should be okay to carry them in both places. (Though I'm CCing akpm just to see if I'm talking crazy.)

Quoted 11 lines · Kees Cook

Quoted 9 lines · Ard Biesheuvel

Quoted 3 lines · Catalin Marinas

I'm still trying to get my head around how we merge those. Since I assume akpm will push them during the merging window, part of your code cannot be tested before.

Actually, my original idea was for akpm to take them as a late merge after rebasing to -rc1, since they touch a variety of architectures, but I am not sure if that came across.

Quoted 5 lines

Quoted 1 line

You could always take the series through your tree instead, I guess?

Traditionally akpm will de-duplicate patches he's carrying that appear in another tree. I think it should be okay to carry them in both places. (Though I'm CCing akpm just to see if I'm talking crazy.)

For now, I'll merge this series in the arm64 tree and push it to next:

http://lkml.kernel.org/r/1452007180-27411-1-git-send-email-ard.biesheuvel at linaro.org

If there are any objections, I can drop the patches and do the BUILDTIME_EXTABLE_SORT disabling trick until they end up in mainline.

Quoted 22 lines · Catalin Marinas

Quoted 11 lines · Kees Cook

Quoted 9 lines · Ard Biesheuvel

Quoted 3 lines · Catalin Marinas

I'm still trying to get my head around how we merge those. Since I assume akpm will push them during the merging window, part of your code cannot be tested before.

Actually, my original idea was for akpm to take them as a late merge after rebasing to -rc1, since they touch a variety of architectures, but I am not sure if that came across.

Quoted 5 lines

Quoted 1 line

You could always take the series through your tree instead, I guess?

Traditionally akpm will de-duplicate patches he's carrying that appear in another tree. I think it should be okay to carry them in both places. (Though I'm CCing akpm just to see if I'm talking crazy.)

For now, I'll merge this series in the arm64 tree and push it to next:

http://lkml.kernel.org/r/1452007180-27411-1-git-send-email-ard.biesheuvel at linaro.org

Quoted 2 lines

If there are any objections, I can drop the patches and do the BUILDTIME_EXTABLE_SORT disabling trick until they end up in mainline.

Latest version is here:
http://lkml.kernel.org/r/1453892123-17973-1-git-send-email-ard.biesheuvel at linaro.org (only difference is an ack from that Alpha maintainter/supporter to patches #1 and #2)

However, the arm64 patch (#6) now conflicts with futex.h in -rc3 after the PAN fix, not sure how to best address that ...

Quoted 23 lines · Ard Biesheuvel

Quoted 22 lines · Catalin Marinas

Quoted 11 lines · Kees Cook

Quoted 9 lines · Ard Biesheuvel

Quoted 3 lines · Catalin Marinas

I'm still trying to get my head around how we merge those. Since I assume akpm will push them during the merging window, part of your code cannot be tested before.

Actually, my original idea was for akpm to take them as a late merge after rebasing to -rc1, since they touch a variety of architectures, but I am not sure if that came across.

Quoted 5 lines

Quoted 1 line

You could always take the series through your tree instead, I guess?

Traditionally akpm will de-duplicate patches he's carrying that appear in another tree. I think it should be okay to carry them in both places. (Though I'm CCing akpm just to see if I'm talking crazy.)

For now, I'll merge this series in the arm64 tree and push it to next:

http://lkml.kernel.org/r/1452007180-27411-1-git-send-email-ard.biesheuvel at linaro.org

Quoted 7 lines

Quoted 2 lines

If there are any objections, I can drop the patches and do the BUILDTIME_EXTABLE_SORT disabling trick until they end up in mainline.

Latest version is here:
http://lkml.kernel.org/r/1453892123-17973-1-git-send-email-ard.biesheuvel at linaro.org (only difference is an ack from that Alpha maintainter/supporter to patches #1 and #2)

I applied the acks manually but I'll double-check to make sure I haven't missed anything.

Quoted 2 lines

However, the arm64 patch (#6) now conflicts with futex.h in -rc3 after the PAN fix, not sure how to best address that ...

I'll have a look.

Quoted 5 lines · Catalin Marinas

Quoted 2 lines · Ard Biesheuvel

However, the arm64 patch (#6) now conflicts with futex.h in -rc3 after the PAN fix, not sure how to best address that ...

I'll have a look.

Note that the fix is trivial, but I don't know who should be carrying the fix is all