selftests/vm/pkeys: Bug fixes and a new test
Subsystems (2)
cover selftests/vm/pkeys: Bug fixes and a new test
There has been a lot of activity on the x86 front around the XSAVE architecture which is used to context-switch processor state (among other things). In addition, AMD has recently joined the protection keys club by adding processor support for PKU.
The AMD implementation helped uncover a kernel bug around the PKRU "init state", which actually applied to Intel's implementation but was just harder to hit. This series adds a test which is expected to help find this class of bug both on AMD and Intel. All the work around pkeys on x86 also uncovered a few bugs in the selftest.
Any testing of this new code (especially from my powerpc friends) would be appreciated.
PATCH 1/4 selftests/vm/pkeys: Fix alloc_random_pkey() to make it really, really random
From: Dave Hansen <[email protected]>
The "random" pkey allocation code currently does the good old:
srand((unsigned int)time(NULL));
*But*, it unfortunately does this on every random pkey allocation.
There may be thousands of these a second. time() has a one second resolution. So, each time alloc_random_pkey() is called, the PRNG is *RESET* to time(). This is nasty. Normally, if you do:
srand(<ANYTHING>); foo = rand(); bar = rand();
You'll be quite guaranteed that 'foo' and 'bar' are different. But, if you do:
srand(1); foo = rand(); srand(1); bar = rand();
You are quite guaranteed that 'foo' and 'bar' are the *SAME*. The recent "fix" effectively forced the test case to use the same "random" pkey for the whole test, unless the test run crossed a second boundary.
Only run srand() once at program startup.
This explains some very odd and persistent test failures I've been seeing.
---
b/tools/testing/selftests/vm/protection_keys.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
tools/testing/selftests/vm/protection_keys.c renamed from tools/testing/selftests/vm/protection_keys.c~selftests_vm_pkeys_Fix_alloc_random_pkey_to_make_it_really_really_random-1 +2 -1
@@ -561,7 +561,6 @@ int alloc_random_pkey(void)int nr_alloced =0 ;int random_index;memset (alloced_pkeys,0 ,sizeof (alloced_pkeys)); -srand ((unsigned int )time (NULL ));/* allocate every possible key and make a note of which ones we got */ max_nr_pkey_allocs =NR_PKEYS ;
@@ -1552,6 +1551,8 @@ int main(void)int nr_iterations =22 ;int pkeys_supported =is_pkeys_supported (); +srand ((unsigned int )time (NULL )); +setup_handlers ();printf ("has pkeys: %d\n" , pkeys_supported);
_
PATCH 2/4 selftests/vm/pkeys: Handle negative sys_pkey_alloc() return code
In reply to Dave HansenFrom: Dave Hansen <[email protected]>
The alloc_pkey() sefltest function wraps the sys_pkey_alloc() system call. On success, it updates its "shadow" register value because sys_pkey_alloc() updates the real register.
But, the success check is wrong. pkey_alloc() considers any non-zero return code to indicate success where the pkey register will be modified. This fails to take negative return codes into account.
Consider only a positive return value as a successful call.
---
b/tools/testing/selftests/vm/protection_keys.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
tools/testing/selftests/vm/protection_keys.c renamed from tools/testing/selftests/vm/protection_keys.c~selftests_vm_pkeys_ret-code +1 -1
@@ -510,7 +510,7 @@ int alloc_pkey(void)" shadow: 0x%016llx\n" , __func__, __LINE__, ret,__read_pkey_reg (), shadow_pkey_reg); -if (ret) { +if (ret >0 ) {/* clear both the bits: */ shadow_pkey_reg =set_pkey_bits (shadow_pkey_reg, ret, ~PKEY_MASK );
_
PATCH 3/4 selftests/vm/pkeys: Refill shadow register after implicit kernel write
In reply to Dave HansenFrom: Dave Hansen <[email protected]>
The pkey test code keeps a "shadow" of the pkey register around. This ensures that any bugs which might write to the register can be caught more quickly.
Generally, userspace has a good idea when the kernel is going to write to the register. For instance, alloc_pkey() is passed a permission mask. The caller of alloc_pkey() can update the shadow based on the return value and the mask.
But, the kernel can also modify the pkey register in a more sneaky way. For mprotect(PROT_EXEC) mappings, the kernel will allocate a pkey and write the pkey register to create an execute-only mapping. The kernel never tells userspace what key it uses for this.
This can cause the test to fail with messages like:
protection_keys_64.2: pkey-helpers.h:132: _read_pkey_reg: Assertion `pkey_reg == shadow_pkey_reg' failed.
because the shadow was not updated with the new kernel-set value.
Forcibly update the shadow value immediately after an mprotect().
---
b/tools/testing/selftests/vm/protection_keys.c | 7 +++++++ 1 file changed, 7 insertions(+)
tools/testing/selftests/vm/protection_keys.c renamed from tools/testing/selftests/vm/protection_keys.c~selftests_vm_pkeys_Refill_shadow_register_after_implict_kernel_write-1 +7 -0
@@ -1448,6 +1448,13 @@ void test_implicit_mprotect_exec_only_me ret =mprotect (p1,PAGE_SIZE ,PROT_EXEC );pkey_assert (!ret); +/* +* Reset the shadow, assuming that the above mprotect() +* correctly changed PKRU, but to an unknown value since +* the actual alllocated pkey is unknown. +*/ + shadow_pkey_reg =__read_pkey_reg (); +dprintf2 ("pkey_reg: %016llx\n" ,read_pkey_reg ());/* Make sure this is an *instruction* fault */
_
PATCH 4/4 selftests/vm/pkeys: Exercise x86 XSAVE init state
In reply to Dave HansenFrom: Dave Hansen <[email protected]>
On x86, there is a set of instructions used to save and restore register state collectively known as the XSAVE architecture. There are about a dozen different features managed with XSAVE. The protection keys register, PKRU, is one of those features.
The hardware optimizes XSAVE by tracking when the state has not changed from its initial (init) state. In this case, it can avoid the cost of writing state to memory (it would usually just be a bunch of 0's).
When the pkey register is 0x0 the hardware optionally choose to track the register as being in the init state (optimize away the writes). AMD CPUs do this more aggressively compared to Intel.
On x86, PKRU is rarely in its (very permissive) init state. Instead, the value defaults to something very restrictive. It is not surprising that bugs have popped up in the rare cases when PKRU reaches its init state.
Add a protection key selftest which gets the protection keys register into its init state in a way that should work on Intel and AMD. Then, do a bunch of pkey register reads to watch for inadvertent changes.
This adds "-mxsave" to CFLAGS for all the x86 vm selftests in order to allow use of the XSAVE instruction __builtin functions. This will make the builtins available on all of the vm selftests, but is expected to be harmless.
---
b/tools/testing/selftests/vm/Makefile | 4 - b/tools/testing/selftests/vm/pkey-x86.h | 1 b/tools/testing/selftests/vm/protection_keys.c | 73 +++++++++++++++++++++++++ 3 files changed, 76 insertions(+), 2 deletions(-)
tools/testing/selftests/vm/Makefile renamed from tools/testing/selftests/vm/Makefile~selftests_vm_pkeys_Exercise_x86_XSAVE_init_state-1 +2 -2
@@ -100,7 +100,7 @@ $(1) $(1)_64: $(OUTPUT)/$(1)_64 endef ifeq ($(CAN_BUILD_I386),1) -$(BINARIES_32): CFLAGS += -m32 +$(BINARIES_32): CFLAGS += -m32 -mxsave $(BINARIES_32): LDLIBS += -lrt -ldl -lm $(BINARIES_32): $(OUTPUT)/%_32: %.c $(CC) $(CFLAGS) $(EXTRA_CFLAGS) $(notdir $^) $(LDLIBS) -o $@
@@ -108,7 +108,7 @@ $(foreach t,$(TARGETS),$(eval $(call gen endif ifeq ($(CAN_BUILD_X86_64),1) -$(BINARIES_64): CFLAGS += -m64 +$(BINARIES_64): CFLAGS += -m64 -mxsave $(BINARIES_64): LDLIBS += -lrt -ldl $(BINARIES_64): $(OUTPUT)/%_64: %.c $(CC) $(CFLAGS) $(EXTRA_CFLAGS) $(notdir $^) $(LDLIBS) -o $@
tools/testing/selftests/vm/pkey-x86.h renamed from tools/testing/selftests/vm/pkey-x86.h~selftests_vm_pkeys_Exercise_x86_XSAVE_init_state-1 +1 -0
@@ -126,6 +126,7 @@ static inline u32 pkey_bit_position(int#define XSTATE_PKEY_BIT (9)#define XSTATE_PKEY 0x200 +#define XSTATE_BV_OFFSET 512int pkey_reg_xstate_offset (void ) {
tools/testing/selftests/vm/protection_keys.c renamed from tools/testing/selftests/vm/protection_keys.c~selftests_vm_pkeys_Exercise_x86_XSAVE_init_state-1 +73 -0
85 lines not on this page · raw message
_
Quoted 5 lines · Dave Hansen
There has been a lot of activity on the x86 front around the XSAVE architecture which is used to context-switch processor state (among other things). In addition, AMD has recently joined the protection keys club by adding processor support for PKU.
Quoted 8 lines
The AMD implementation helped uncover a kernel bug around the PKRU "init state", which actually applied to Intel's implementation but was just harder to hit. This series adds a test which is expected to help find this class of bug both on AMD and Intel. All the work around pkeys on x86 also uncovered a few bugs in the selftest.
Any testing of this new code (especially from my powerpc friends) would be appreciated.
I tested this on ppc64.
Quoted 15 lines
Cc Thomas Gleixner <[email protected]>Cc Ram Pai <[email protected]>Cc Sandipan Das <[email protected]>Cc Andrew Morton <[email protected]>Cc Florian Weimer <[email protected]>Cc "Desnes A. Nunes do Rosario" <[email protected]>Cc Ingo Molnar <[email protected]>Cc Thiago Jung Bauermann <[email protected]>Cc "Aneesh Kumar K.V" <[email protected]>Cc Michael Ellerman <[email protected]>Cc Michal Hocko <[email protected]>Cc Michal Suchanek <[email protected]>Cc Shuah Khan <[email protected]>