powerpc/mm: Conditionally call H_BLOCK_REMOVE
Subsystems (2)
v3 cover powerpc/mm: Conditionally call H_BLOCK_REMOVE
Since the commit ba2dd8a26baa ("powerpc/pseries/mm: call H_BLOCK_REMOVE"), the call to H_BLOCK_REMOVE is always done if the feature is exhibited.
However, the hypervisor may not support all the block size for the hcall H_BLOCK_REMOVE depending on the segment base page size and actual page size.
When unsupported block size is used, the hcall H_BLOCK_REMOVE is returning H_PARAM, which is triggering a BUG_ON check leading to a panic like this:
kernel BUG at /home/srikar/work/linux.git/arch/powerpc/platforms/pseries/lpar.c:466!
Oops: Exception in kernel mode, sig: 5 [#1] BE PAGE_SIZE=64K MMU=Hash SMP NR_CPUS=2048 NUMA pSeries Modules linked in: CPU: 28 PID: 583 Comm: modprobe Not tainted 5.2.0-master #5 NIP: c0000000000be8dc LR: c0000000000be880 CTR: 0000000000000000 REGS: c0000007e77fb130 TRAP: 0700 Not tainted (5.2.0-master) MSR: 8000000000029032 <SF,EE,ME,IR,DR,RI> CR: 42224824 XER: 20000000
CFAR: c0000000000be8fc IRQMASK: 0
GPR00: 0000000022224828 c0000007e77fb3c0 c000000001434d00 0000000000000005
GPR04: 9000000004fa8c00 0000000000000000 0000000000000003 0000000000000001
GPR08: c0000007e77fb450 0000000000000000 0000000000000001 ffffffffffffffff
GPR12: c0000007e77fb450 c00000000edfcb80 0000cd7d3ea30000 c0000000016022b0
GPR16: 00000000000000b0 0000cd7d3ea30000 0000000000000001 c080001f04f00105
GPR20: 0000000000000003 0000000000000004 c000000fbeb05f58 c000000001602200
GPR24: 0000000000000000 0000000000000004 8800000000000000 c000000000c5d148
GPR28: c000000000000000 8000000000000000 a000000000000000 c0000007e77fb580 NIP [c0000000000be8dc] .call_block_remove+0x12c/0x220
LR [c0000000000be880] .call_block_remove+0xd0/0x220
Call Trace:
[c0000007e77fb3c0] [c000000fb8c00240] 0xc000000fb8c00240 (unreliable) [c0000007e77fb4e0] [c0000000000bfef8] .pSeries_lpar_flush_hash_range+0x578/0x670 [c0000007e77fb660] [c000000000073994] .flush_hash_range+0x44/0x100 [c0000007e77fb700] [c00000000007667c] .__flush_tlb_pending+0x3c/0xc0 [c0000007e77fb780] [c0000000002d3fdc] .zap_pte_range+0x7ec/0x830 [c0000007e77fb8d0] [c0000000002d4524] .unmap_page_range+0x3f4/0x540 [c0000007e77fb9f0] [c0000000002d4a04] .unmap_vmas+0x94/0x120 [c0000007e77fbab0] [c0000000002e0efc] .exit_mmap+0xac/0x1f0 [c0000007e77fbbc0] [c0000000000fae1c] .mmput+0x9c/0x1f0 [c0000007e77fbc40] [c000000000105738] .do_exit+0x388/0xd60 [c0000007e77fbd20] [c0000000001061b4] .do_group_exit+0x54/0x100 [c0000007e77fbdb0] [c000000000106274] .__se_sys_exit_group+0x14/0x20 [c0000007e77fbe20] [c00000000000b688] system_call+0x5c/0x70 Instruction dump:
39400001 38a00000 4800003c 60000000 60420000 7fa9e800 38e00000 419e0014 7d29d278 7d290074 7929d182 69270001 <0b070000> 7d495378 394a0001 7fa93040
The PAPR document specifies the TLB Block Invalidate Characteristics which tells for each pair of segment base page size, actual page size, the size of the block the hcall H_BLOCK_REMOVE is supporting (cf Power Architecture Platform Requirements +, Section 7.3.16.23, page 252).
Supporting various block sizes doesn't seem needed at that time since all systems I was able to play with was supporting an 8 addresses block size, which is the maximum through the hcall, or none at all. Supporting various size would complexify the algorithm in call_block_remove() so unless this is required, this is not done.
In the case of block size different from 8, a warning message is displayed at boot time and that block size will be ignored checking for the H_BLOCK_REMOVE support.
Due to the minimal amount of hardware showing a limited set of H_BLOCK_REMOVE supported page size, I don't think there is a need to push this series to the stable mailing list.
The first patch is reading the characteristic through the hcall ibm,get-system-parameter and record the supported block size for each page size. The second patch is changing the check used to detect the H_BLOCK_REMOVE availability to take care of the base page size and page size couple.
Changes since V2:
- Adressing Michael's comments
- Remove some unedeed logging
- Replace "hblkr" prefix/suffix to "hlkbrm"
- In pseries_lpar_read_hblkrm_characteristics()
- Remove unneeded buffer's initialisation
- Limit endianness trouble
- Prevent potential buffer overflow.
Changes since V1:
- Addressing Aneesh's comments
- Remove penc initialisation, this is already done in mmu_psize_set_default_penc()
- Add details on the TLB Block Invalidate Characteristics's buffer format
- Introduce #define instead of using direct numerical values
- Function reading the characteristics is now directly called from pSeries_setup_arch()
- The characteristics are now stored in a dedciated table static to lpar.c
Laurent Dufour (2):
powperc/mm: read TLB Block Invalidate Characteristics powerpc/mm: call H_BLOCK_REMOVE when supported
arch/powerpc/platforms/pseries/lpar.c | 162 ++++++++++++++++++++++- arch/powerpc/platforms/pseries/pseries.h | 1 + arch/powerpc/platforms/pseries/setup.c | 1 + 3 files changed, 162 insertions(+), 2 deletions(-)
Signature · 1 line
2.23.0
PATCH v3 1/2 powperc/mm: read TLB Block Invalidate Characteristics
The PAPR document specifies the TLB Block Invalidate Characteristics which tells for each pair of segment base page size, actual page size, the size of the block the hcall H_BLOCK_REMOVE supports.
These characteristics are loaded at boot time in a new table hblkr_size. The table is separate from the mmu_psize_def because this is specific to the pseries platform.
A new init function, pseries_lpar_read_hblkrm_characteristics() is added to read the characteristics. It is called from pSeries_setup_arch().
--- arch/powerpc/platforms/pseries/lpar.c | 139 +++++++++++++++++++++++ arch/powerpc/platforms/pseries/pseries.h | 1 + arch/powerpc/platforms/pseries/setup.c | 1 + 3 files changed, 141 insertions(+)
arch/powerpc/platforms/pseries/lpar.c +139 -0
@@ -56,6 +56,15 @@ EXPORT_SYMBOL(plpar_hcall);EXPORT_SYMBOL (plpar_hcall9);EXPORT_SYMBOL (plpar_hcall_norets); +/* +* H_BLOCK_REMOVE supported block size for this page size in segment who's base +* page size is that page size. +* +* The first index is the segment base page size, the second one is the actual +* page size. +*/ +static int hblkrm_size[MMU_PAGE_COUNT ][MMU_PAGE_COUNT ]__ro_after_init ; +#ifdef CONFIG_VIRT_CPU_ACCOUNTING_NATIVE static u8 dtl_mask =DTL_LOG_PREEMPT ;#else
@@ -1311,6 +1320,136 @@ static void do_block_remove(unsigned long number, struct ppc64_tlb_batch *batch, (void )call_block_remove (pix, param, true); } +/* +* TLB Block Invalidate Characteristics +* +* These characteristics define the size of the block the hcall H_BLOCK_REMOVE +* is able to process for each couple segment base page size, actual page size. +* +* The ibm,get-system-parameter properties is returning a buffer with the +* following layout: +* +* [ 2 bytes size of the RTAS buffer (excluding these 2 bytes) ] +* ----------------- +* TLB Block Invalidate Specifiers: +* [ 1 byte LOG base 2 of the TLB invalidate block size being specified ] +* [ 1 byte Number of page sizes (N) that are supported for the specified +* TLB invalidate block size ] +* [ 1 byte Encoded segment base page size and actual page size +* MSB=0 means 4k segment base page size and actual page size +* MSB=1 the penc value in mmu_psize_def ] +* ... +* ----------------- +* Next TLB Block Invalidate Specifiers... +* ----------------- +* [ 0 ] +*/ +static inline void set_hblkrm_bloc_size (int bpsize,int psize, +unsigned int block_size) +{ +if (block_size > hblkrm_size[bpsize][psize]) + hblkrm_size[bpsize][psize] = block_size; +} + +/* +* Decode the Encoded segment base page size and actual page size. +* PAPR specifies: +* - bit 7 is the L bit +* - bits 0-5 are the penc value +* If the L bit is 0, this means 4K segment base page size and actual page size +* otherwise the penc value should be read. +*/ +#define HBLKRM_L_MASK 0x80 +#define HBLKRM_PENC_MASK 0x3f +static inline void __init check_lp_set_hblkrm (unsigned int lp, +unsigned int block_size) +{ +unsigned int bpsize, psize; + +/* First, check the L bit, if not set, this means 4K */ +if ((lp &HBLKRM_L_MASK ) ==0 ) { +set_hblkrm_bloc_size (MMU_PAGE_4K ,MMU_PAGE_4K , block_size); +return ; + } + + lp &=HBLKRM_PENC_MASK ; +for (bpsize =0 ; bpsize <MMU_PAGE_COUNT ; bpsize++) { +struct mmu_psize_def *def = &mmu_psize_defs[bpsize]; + +for (psize =0 ; psize <MMU_PAGE_COUNT ; psize++) { +if (def->penc[psize] == lp) { +set_hblkrm_bloc_size (bpsize, psize, block_size); +return ; + } + } + } +} + +#define SPLPAR_TLB_BIC_TOKEN 50 +/* +* The size of the TLB Block Invalidate Characteristics is variable. But at the +* maximum it will be the number of possible page sizes *2 + 10 bytes. +* Currently MMU_PAGE_COUNT is 16, which means 42 bytes. Use a cache line size +* (128 bytes) for the buffer to get plenty of space. +*/ +#define SPLPAR_TLB_BIC_MAXLENGTH 128 +//#warning revoir ce define mpe voudrait que ce soit cach size aligned +void __init pseries_lpar_read_hblkrm_characteristics (void ) +{ +unsigned char local_buffer[SPLPAR_TLB_BIC_MAXLENGTH ]; +int call_status, len, idx, bpsize; + +spin_lock (&rtas_data_buf_lock); +memset (rtas_data_buf,0 ,RTAS_DATA_BUF_SIZE ); + call_status =rtas_call (rtas_token ("ibm,get-system-parameter" ),3 ,1 , +NULL , +SPLPAR_TLB_BIC_TOKEN , +__pa (rtas_data_buf), +RTAS_DATA_BUF_SIZE ); +memcpy (local_buffer, rtas_data_buf,SPLPAR_TLB_BIC_MAXLENGTH ); + local_buffer[SPLPAR_TLB_BIC_MAXLENGTH -1 ] ='\0' ; +spin_unlock (&rtas_data_buf_lock); + +if (call_status !=0 ) { +pr_warn ("%s %s Error calling get-system-parameter (0x%x)\n" , + __FILE__, __func__, call_status); +return ; + } + +/* +* The first two (2) bytes of the data in the buffer are the length of +* the returned data, not counting these first two (2) bytes. +*/ + len =be16_to_cpu (*((u16 *)local_buffer)) +2 ; +if (len >SPLPAR_TLB_BIC_MAXLENGTH ) { +pr_warn ("%s too large returned buffer %d" , __func__, len); +return ; + } + + idx =2 ; +while (idx < len) { +u8 block_shift = local_buffer[idx++]; +u32 block_size; +unsigned int npsize; + +if (!block_shift) +break ; + + block_size =1 << block_shift; + +for (npsize = local_buffer[idx++]; + npsize >0 && idx < len; npsize--) +check_lp_set_hblkrm ((unsigned int ) local_buffer[idx++], + block_size); + } + +for (bpsize =0 ; bpsize <MMU_PAGE_COUNT ; bpsize++) +for (idx =0 ; idx <MMU_PAGE_COUNT ; idx++) +if (hblkrm_size[bpsize][idx]) +pr_info ("H_BLOCK_REMOVE supports base psize:%d psize:%d block size:%d" , + bpsize, idx, hblkrm_size[bpsize][idx]); +} +/* * Take a spinlock around flushes to avoid bouncing the hypervisor tlbie * lock.
arch/powerpc/platforms/pseries/pseries.h +1 -0
@@ -112,5 +112,6 @@ static inline unsigned long cmo_get_page_size(void)int dlpar_workqueue_init (void );void pseries_setup_rfi_flush (void ); +void pseries_lpar_read_hblkrm_characteristics (void );#endif /* _PSERIES_PSERIES_H */
arch/powerpc/platforms/pseries/setup.c +1 -0
@@ -744,6 +744,7 @@ static void __init pSeries_setup_arch(void)pseries_setup_rfi_flush ();setup_stf_barrier (); +pseries_lpar_read_hblkrm_characteristics ();/* By default, only probe PCI (can be overridden by rtas_pci) */ pci_add_flags (PCI_PROBE_ONLY );
Signature · 1 line
2.23.0
Re: [PATCH v3 1/2] powperc/mm: read TLB Block Invalidate Characteristics
Quoted 8 lines · Laurent Dufour
The PAPR document specifies the TLB Block Invalidate Characteristics which tells for each pair of segment base page size, actual page size, the size of the block the hcall
H_BLOCK_REMOVEsupports.These characteristics are loaded at boot time in a new table
hblkr_size. The table is separate from themmu_psize_defbecause this is specific to the pseries platform.
Quoted 5 lines
A new init function,
pseries_lpar_read_hblkrm_characteristics()is added to read the characteristics. It is called frompSeries_setup_arch().Fixes ba2dd8a26baa ("powerpc/pseries/mm: call H_BLOCK_REMOVE")Signed-off-by Laurent Dufour <[email protected]>
Series applied to powerpc fixes, thanks.
https://git.kernel.org/powerpc/c/1211ee61b4a8e60d6dc77211cdcf01906915bfba
cheers
PATCH v3 2/2 powerpc/mm: call H_BLOCK_REMOVE when supported
In reply to Laurent DufourDepending on the hardware and the hypervisor, the hcall H_BLOCK_REMOVE may not be able to process all the page sizes for a segment base page size, as reported by the TLB Invalidate Characteristics.
For each pair of base segment page size and actual page size, this characteristic tells us the size of the block the hcall supports.
In the case, the hcall is not supporting a pair of base segment page size, actual page size, it is returning H_PARAM which leads to a panic like this:
kernel BUG at /home/srikar/work/linux.git/arch/powerpc/platforms/pseries/lpar.c:466!
Oops: Exception in kernel mode, sig: 5 [#1] BE PAGE_SIZE=64K MMU=Hash SMP NR_CPUS=2048 NUMA pSeries Modules linked in: CPU: 28 PID: 583 Comm: modprobe Not tainted 5.2.0-master #5 NIP: c0000000000be8dc LR: c0000000000be880 CTR: 0000000000000000 REGS: c0000007e77fb130 TRAP: 0700 Not tainted (5.2.0-master) MSR: 8000000000029032 <SF,EE,ME,IR,DR,RI> CR: 42224824 XER: 20000000
CFAR: c0000000000be8fc IRQMASK: 0
GPR00: 0000000022224828 c0000007e77fb3c0 c000000001434d00 0000000000000005
GPR04: 9000000004fa8c00 0000000000000000 0000000000000003 0000000000000001
GPR08: c0000007e77fb450 0000000000000000 0000000000000001 ffffffffffffffff
GPR12: c0000007e77fb450 c00000000edfcb80 0000cd7d3ea30000 c0000000016022b0
GPR16: 00000000000000b0 0000cd7d3ea30000 0000000000000001 c080001f04f00105
GPR20: 0000000000000003 0000000000000004 c000000fbeb05f58 c000000001602200
GPR24: 0000000000000000 0000000000000004 8800000000000000 c000000000c5d148
GPR28: c000000000000000 8000000000000000 a000000000000000 c0000007e77fb580 NIP [c0000000000be8dc] .call_block_remove+0x12c/0x220
LR [c0000000000be880] .call_block_remove+0xd0/0x220
Call Trace:
[c0000007e77fb3c0] [c000000fb8c00240] 0xc000000fb8c00240 (unreliable) [c0000007e77fb4e0] [c0000000000bfef8] .pSeries_lpar_flush_hash_range+0x578/0x670 [c0000007e77fb660] [c000000000073994] .flush_hash_range+0x44/0x100 [c0000007e77fb700] [c00000000007667c] .__flush_tlb_pending+0x3c/0xc0 [c0000007e77fb780] [c0000000002d3fdc] .zap_pte_range+0x7ec/0x830 [c0000007e77fb8d0] [c0000000002d4524] .unmap_page_range+0x3f4/0x540 [c0000007e77fb9f0] [c0000000002d4a04] .unmap_vmas+0x94/0x120 [c0000007e77fbab0] [c0000000002e0efc] .exit_mmap+0xac/0x1f0 [c0000007e77fbbc0] [c0000000000fae1c] .mmput+0x9c/0x1f0 [c0000007e77fbc40] [c000000000105738] .do_exit+0x388/0xd60 [c0000007e77fbd20] [c0000000001061b4] .do_group_exit+0x54/0x100 [c0000007e77fbdb0] [c000000000106274] .__se_sys_exit_group+0x14/0x20 [c0000007e77fbe20] [c00000000000b688] system_call+0x5c/0x70 Instruction dump:
39400001 38a00000 4800003c 60000000 60420000 7fa9e800 38e00000 419e0014 7d29d278 7d290074 7929d182 69270001 <0b070000> 7d495378 394a0001 7fa93040
The call to H_BLOCK_REMOVE should only be made for the supported pair of base segment page size, actual page size and using the correct maximum block size.
Due to the required complexity in do_block_remove() and call_block_remove(), and the fact that currently a block size of 8 is returned by the hypervisor, we are only supporting 8 size block to the H_BLOCK_REMOVE hcall.
In order to identify this limitation easily in the code,a local define HBLKR_SUPPORTED_SIZE defining the currently supported block size, and a dedicated checking helper is_supported_hlbkr() are introduced.
For regular pages and hugetlb, the assumption is made that the page size is equal to the base page size. For THP the page size is assumed to be 16M.
--- arch/powerpc/platforms/pseries/lpar.c | 23 +++++++++++++++++++++-- 1 file changed, 21 insertions(+), 2 deletions(-)
arch/powerpc/platforms/pseries/lpar.c +21 -2
@@ -65,6 +65,13 @@ EXPORT_SYMBOL(plpar_hcall_norets);*/ static int hblkrm_size[MMU_PAGE_COUNT ][MMU_PAGE_COUNT ]__ro_after_init ; +/* +* Due to the involved complexity, and that the current hypervisor is only +* returning this value or 0, we are limiting the support of the H_BLOCK_REMOVE +* buffer size to 8 size block. +*/ +#define HBLKRM_SUPPORTED_BLOCK_SIZE 8 +#ifdef CONFIG_VIRT_CPU_ACCOUNTING_NATIVE static u8 dtl_mask =DTL_LOG_PREEMPT ;#else
@@ -993,6 +1000,17 @@ static void pSeries_lpar_hpte_invalidate(unsigned long slot, unsigned long vpn,#define HBLKR_CTRL_ERRNOTFOUND 0x8800000000000000UL#define HBLKR_CTRL_ERRBUSY 0xa000000000000000UL +/* +* Returned true if we are supporting this block size for the specified segment +* base page size and actual page size. +* +* Currently, we only support 8 size block. +*/ +static inline bool is_supported_hlbkrm (int bpsize,int psize) +{ +return (hblkrm_size[bpsize][psize] ==HBLKRM_SUPPORTED_BLOCK_SIZE ); +} +/** * H_BLOCK_REMOVE caller. * @idx should point to the latest @param entry set with a PTEX.
@@ -1152,7 +1170,8 @@ static inline void __pSeries_lpar_hugepage_invalidate(unsigned long *slot,if (lock_tlbie)spin_lock_irqsave (&pSeries_lpar_tlbie_lock, flags); -if (firmware_has_feature (FW_FEATURE_BLOCK_REMOVE )) +/* Assuming THP size is 16M */ +if (is_supported_hlbkrm (psize,MMU_PAGE_16M ))hugepage_block_invalidate (slot, vpn, count, psize, ssize);else hugepage_bulk_invalidate (slot, vpn, count, psize, ssize);
@@ -1469,7 +1488,7 @@ static void pSeries_lpar_flush_hash_range(unsigned long number, int local)if (lock_tlbie)spin_lock_irqsave (&pSeries_lpar_tlbie_lock, flags); -if (firmware_has_feature (FW_FEATURE_BLOCK_REMOVE )) { +if (is_supported_hlbkrm (batch->psize, batch->psize)) {do_block_remove (number, batch, param); goto out; }
Signature · 1 line
2.23.0