powerpc/64s/hash: add stress_hpt kernel boot option to increase hash faults
Subsystems (2)
Nicholas Piggin
reviewerPATCH 1/1 powerpc/64s/hash: add stress_hpt kernel boot option to increase hash faults
This option increases the number of hash misses by limiting the number of kernel HPT entries, by keeping a per-CPU record of the last kernel HPTEs installed, and removing that from the hash table on the next hash insertion. A timer round-robins CPUs removing remaining kernel HPTEs and clearing the TLB (in the case of bare metal) to increase and slightly randomise kernel fault activity.
---
I tried this a while ago and it had some lockup issues, I think I fixed them. Deadlock if the same hpte group is used as the one to be cleared, solved by deferring the removal in that case. Livelock if CPUs are concurrently faulting in HPTEs and removing them, solved by limiting HPTE remove iterations. And thrashing / practical livelock due to insufficient TLB entries on pseries, solved by keeping the last 16 translations cached. Also added a periodic tlb flush timer to add a bit more randomness to faults.
This catches the irq return bug that the stress_code_patching.sh test was triggering.
Thanks,
Nick
.../admin-guide/kernel-parameters.txt | 5 + arch/powerpc/mm/book3s64/hash_4k.c | 5 + arch/powerpc/mm/book3s64/hash_64k.c | 10 ++ arch/powerpc/mm/book3s64/hash_utils.c | 129 +++++++++++++++++- arch/powerpc/mm/book3s64/internal.h | 11 ++ 5 files changed, 159 insertions(+), 1 deletion(-)
Documentation/admin-guide/kernel-parameters.txt +5 -0
@@ -1042,6 +1042,11 @@ them frequently to increase the rate of SLB faults on kernel addresses. + stress_hpt [PPC] + Limits the number of kernel HPT entries in the hash + page table to increase the rate of hash page table + faults on kernel addresses. + disable= [IPV6] See Documentation/networking/ipv6.rst.
arch/powerpc/mm/book3s64/hash_4k.c +5 -0
@@ -16,6 +16,8 @@#include <asm/machdep.h> #include <asm/mmu.h> +#include "internal.h" +int __hash_page_4K (unsigned long ea,unsigned long access,unsigned long vsid,pte_t *ptep,unsigned long trap,unsigned long flags,int ssize,int subpg_prot)
@@ -118,6 +120,9 @@ int __hash_page_4K(unsigned long ea, unsigned long access, unsigned long vsid, } new_pte = (new_pte & ~_PAGE_HPTEFLAGS) |H_PAGE_HASHPTE ; new_pte |=pte_set_hidx (ptep, rpte,0 , slot,PTRS_PER_PTE ); + +if (stress_hpt ()) +hpt_do_stress (ea, hpte_group); } *ptep =__pte (new_pte & ~H_PAGE_BUSY );return 0 ;
arch/powerpc/mm/book3s64/hash_64k.c +10 -0
@@ -16,6 +16,8 @@#include <asm/machdep.h> #include <asm/mmu.h> +#include "internal.h" +/* * Return true, if the entry has a slot value which * the software considers as invalid.
@@ -216,6 +218,9 @@ int __hash_page_4K(unsigned long ea, unsigned long access, unsigned long vsid, new_pte |=pte_set_hidx (ptep, rpte, subpg_index, slot,PTRS_PER_PTE ); new_pte |=H_PAGE_HASHPTE ; +if (stress_hpt ()) +hpt_do_stress (ea, hpte_group); + *ptep =__pte (new_pte & ~H_PAGE_BUSY );return 0 ; }
@@ -327,7 +332,12 @@ int __hash_page_64K(unsigned long ea, unsigned long access, new_pte = (new_pte & ~_PAGE_HPTEFLAGS) |H_PAGE_HASHPTE ; new_pte |=pte_set_hidx (ptep, rpte,0 , slot,PTRS_PER_PTE ); + +if (stress_hpt ()) +hpt_do_stress (ea, hpte_group); } + *ptep =__pte (new_pte & ~H_PAGE_BUSY ); +return 0 ; }
arch/powerpc/mm/book3s64/hash_utils.c +128 -1
159 lines not on this page · raw message
arch/powerpc/mm/book3s64/internal.h +11 -0
@@ -13,6 +13,17 @@ static inline bool stress_slb(void)return static_branch_unlikely (&stress_slb_key); } +extern bool stress_hpt_enabled; + +DECLARE_STATIC_KEY_FALSE (stress_hpt_key); + +static inline bool stress_hpt (void ) +{ +return static_branch_unlikely (&stress_hpt_key); +} + +void hpt_do_stress (unsigned long ea,unsigned long hpte_group); +void slb_setup_new_exec (void );void exit_lazy_flush_tlb (struct mm_struct *mm,bool always_flush);
Signature · 1 line
2.37.2
Michael Ellerman
AppliedQuoted 8 lines · Nicholas Piggin
This option increases the number of hash misses by limiting the number of kernel HPT entries, by keeping a per-CPU record of the last kernel HPTEs installed, and removing that from the hash table on the next hash insertion. A timer round-robins CPUs removing remaining kernel HPTEs and clearing the TLB (in the case of bare metal) to increase and slightly randomise kernel fault activity.
[...]
Applied to powerpc/next.
[1/1] powerpc/64s/hash: add stress_hpt kernel boot option to increase hash faults
https://git.kernel.org/powerpc/c/6b34a099faa123488b13caf704562f4dbe483fc4
cheers