vdso: Fixes for cache line sizes

PATCH Superseded Olof Johansson 3 participants 6 messages Nov 14, 2007 – Nov 20, 2007 PowerPC

lore

Share & tools

lore

Olof Johansson

author

2007-11-14 19:23 UTC #0 lore raw

PATCH 1/1 vdso: Fixes for cache line sizes

[POWERPC] vdso: Fixes for cache line sizes

Current VDSO implementation is hardcoded to 128 byte cachelines, which only works on IBM's 64-bit processors.

Convert it to get the line sizes out of vdso_data instead, similar to how the ppc64 in-kernel cache flush does it.

Signed-off-by Olof Johansson <[email protected]>

---
Paul, this is needed to make for example the IBM jvm run on pa6t. Please include as bugfix for 2.6.24.

arch/powerpc/kernel/asm-offsets.c +4 -0

index 2c8e756..02cfe9a 100644
@@ -284,6 +284,10 @@ int main(void)
 	DEFINE(CFG_SYSCALL_MAP32, offsetof(struct vdso_data, syscall_map_32));
 	DEFINE(WTOM_CLOCK_SEC, offsetof(struct vdso_data, wtom_clock_sec));
 	DEFINE(WTOM_CLOCK_NSEC, offsetof(struct vdso_data, wtom_clock_nsec));
+	DEFINE(CFG_ICACHE_LINESZ, offsetof(struct vdso_data, icache_line_size));
+	DEFINE(CFG_DCACHE_LINESZ, offsetof(struct vdso_data, dcache_line_size));
+	DEFINE(CFG_ICACHE_LOGLINESZ, offsetof(struct vdso_data, icache_log_line_size));
+	DEFINE(CFG_DCACHE_LOGLINESZ, offsetof(struct vdso_data, dcache_log_line_size));
 #ifdef CONFIG_PPC64
 	DEFINE(CFG_SYSCALL_MAP64, offsetof(struct vdso_data, syscall_map_64));
 	DEFINE(TVAL64_TV_SEC, offsetof(struct timeval, tv_sec));

arch/powerpc/kernel/vdso.c +7 -0

index 2322ba5..5a8ab23 100644
@@ -696,14 +696,21 @@ static int __init vdso_init(void)
 	vdso_data->physicalMemorySize = lmb_phys_mem_size();
 	vdso_data->dcache_size = ppc64_caches.dsize;
 	vdso_data->dcache_line_size = ppc64_caches.dline_size;
+	vdso_data->dcache_log_line_size = ppc64_caches.log_dline_size;
 	vdso_data->icache_size = ppc64_caches.isize;
 	vdso_data->icache_line_size = ppc64_caches.iline_size;
+	vdso_data->icache_log_line_size = ppc64_caches.log_iline_size;
 
 	/*
 	 * Calculate the size of the 64 bits vDSO
 	 */
 	vdso64_pages = (&vdso64_end - &vdso64_start) >> PAGE_SHIFT;
 	DBG("vdso64_kbase: %p, 0x%x pages\n", vdso64_kbase, vdso64_pages);
+#else
+	vdso_data->dcache_line_size = L1_CACHE_BYTES;
+	vdso_data->dcache_log_line_size = L1_CACHE_SHIFT;
+	vdso_data->icache_line_size = L1_CACHE_BYTES;
+	vdso_data->icache_log_line_size = L1_CACHE_SHIFT;
 #endif /* CONFIG_PPC64 */

arch/powerpc/kernel/vdso32/cacheflush.S +28 ~~-13~~

57 lines not on this page · raw message

arch/powerpc/kernel/vdso64/cacheflush.S +28 ~~-13~~

57 lines not on this page · raw message

include/asm-powerpc/vdso_datapage.h +6 -0

index 8a94f0e..e59d885 100644
@@ -77,6 +77,8 @@ struct vdso_data {
 	/* those additional ones don't have to be located anywhere
 	 * special as they were not part of the original systemcfg
 	 */
+	__u32 dcache_log_line_size;		/* L1 d-cache log line size */
+	__u32 icache_log_line_size;		/* L1 i-cache log line size */
 	__s32 wtom_clock_sec;			/* Wall to monotonic clock */
 	__s32 wtom_clock_nsec;
    	__u32 syscall_map_64[SYSCALL_MAP_SIZE]; /* map of syscalls  */

@@ -99,6 +101,10 @@ struct vdso_data {
 	__s32 wtom_clock_sec;			/* Wall to monotonic clock */
 	__s32 wtom_clock_nsec;
    	__u32 syscall_map_32[SYSCALL_MAP_SIZE]; /* map of syscalls */
+	__u32 dcache_line_size;		/* L1 d-cache line size	    */
+	__u32 icache_line_size;		/* L1 i-cache line size	    */
+	__u32 dcache_log_line_size;	/* L1 d-cache log line size */
+	__u32 icache_log_line_size;	/* L1 i-cache log line size */
 };
 
 #endif /* CONFIG_PPC64 */

Benjamin Herrenschmidt

2007-11-14 20:28 UTC #1 lore raw

Quoted 7 lines · Olof Johansson

[POWERPC] vdso: Fixes for cache line sizes
Current VDSO implementation is hardcoded to 128 byte cachelines, which only works on IBM's 64-bit processors.
Convert it to get the line sizes out of vdso_data instead, similar to how the ppc64 in-kernel cache flush does it.

Please call the fields "block size" not "line size". There are subtle differences and per architecture, it should really be block size.

Quoted 6 lines

Signed-off-by Olof Johansson <[email protected]>
---
Paul, this is needed to make for example the IBM jvm run on pa6t. Please include as bugfix for 2.6.24.

Heh, they use the vdso for flushes ? I didn't know that !

Ben.

Olof Johansson

author

2007-11-14 21:27 UTC #2 lore raw

PATCH v2 1/1 vdso: Fixes for cache block sizes

[POWERPC] vdso: Fixes for cache line sizes

Current VDSO implementation is hardcoded to 128 byte cache blocks, which are only used on IBM's 64-bit processors.

Convert it to get the blocks sizes out of vdso_data instead, similar to how the ppc64 in-kernel cache flush does it.

Signed-off-by Olof Johansson <[email protected]>

---
Paul, this is needed to make for example the IBM jvm run on pa6t. Please include as bugfix for 2.6.24.

Quoted 4 lines · Benjamin Herrenschmidt

Quoted 2 lines · Olof Johansson
[POWERPC] vdso: Fixes for cache line sizes

Quoted 8 lines

Quoted 5 lines
Current VDSO implementation is hardcoded to 128 byte cachelines, which only works on IBM's 64-bit processors.
Convert it to get the line sizes out of vdso_data instead, similar to how the ppc64 in-kernel cache flush does it.
Please call the fields "block size" not "line size". There are subtle differences and per architecture, it should really be block size.

Alright. I've added block sizes since I'm not sure whether the old systemcfg data is actually line or block sizes, and I guess it's something we need to stay compatible with. See separate new patch.

Also, ppc64_caches only has line sizes, so fill it with that for now. I'll submit a separate patch to complete ppc64_caches with block sizes, but that's a cleanup not a bug fix (i.e. target for that would be 2.6.25).

Quoted 4 lines

Quoted 2 lines
Paul, this is needed to make for example the IBM jvm run on pa6t. Please include as bugfix for 2.6.24.
Heh, they use the vdso for flushes ? I didn't know that !

Me neither, and the fact that the vdso was hardcoded for 128 had completely passed me by.

-Olof

arch/powerpc/kernel/asm-offsets.c +4 -0

index 2c8e756..d67bcd8 100644
@@ -284,6 +284,10 @@ int main(void)
 	DEFINE(CFG_SYSCALL_MAP32, offsetof(struct vdso_data, syscall_map_32));
 	DEFINE(WTOM_CLOCK_SEC, offsetof(struct vdso_data, wtom_clock_sec));
 	DEFINE(WTOM_CLOCK_NSEC, offsetof(struct vdso_data, wtom_clock_nsec));
+	DEFINE(CFG_ICACHE_BLOCKSZ, offsetof(struct vdso_data, icache_block_size));
+	DEFINE(CFG_DCACHE_BLOCKSZ, offsetof(struct vdso_data, dcache_block_size));
+	DEFINE(CFG_ICACHE_LOGBLOCKSZ, offsetof(struct vdso_data, icache_log_block_size));
+	DEFINE(CFG_DCACHE_LOGBLOCKSZ, offsetof(struct vdso_data, dcache_log_block_size));
 #ifdef CONFIG_PPC64
 	DEFINE(CFG_SYSCALL_MAP64, offsetof(struct vdso_data, syscall_map_64));
 	DEFINE(TVAL64_TV_SEC, offsetof(struct timeval, tv_sec));

arch/powerpc/kernel/vdso.c +11 -0

index 2322ba5..3702df7 100644
@@ -699,11 +699,22 @@ static int __init vdso_init(void)
 	vdso_data->icache_size = ppc64_caches.isize;
 	vdso_data->icache_line_size = ppc64_caches.iline_size;
 
+	/* XXXOJN: Blocks should be added to ppc64_caches and used instead */
+	vdso_data->dcache_block_size = ppc64_caches.dline_size;
+	vdso_data->icache_block_size = ppc64_caches.iline_size;
+	vdso_data->dcache_log_block_size = ppc64_caches.log_dline_size;
+	vdso_data->icache_log_block_size = ppc64_caches.log_iline_size;
+
 	/*
 	 * Calculate the size of the 64 bits vDSO
 	 */
 	vdso64_pages = (&vdso64_end - &vdso64_start) >> PAGE_SHIFT;
 	DBG("vdso64_kbase: %p, 0x%x pages\n", vdso64_kbase, vdso64_pages);
+#else
+	vdso_data->dcache_block_size = L1_CACHE_BYTES;
+	vdso_data->dcache_log_block_size = L1_CACHE_SHIFT;
+	vdso_data->icache_block_size = L1_CACHE_BYTES;
+	vdso_data->icache_log_block_size = L1_CACHE_SHIFT;
 #endif /* CONFIG_PPC64 */

arch/powerpc/kernel/vdso32/cacheflush.S +28 ~~-13~~

57 lines not on this page · raw message

arch/powerpc/kernel/vdso64/cacheflush.S +28 ~~-13~~

57 lines not on this page · raw message

include/asm-powerpc/vdso_datapage.h +8 -0

index 8a94f0e..f013932 100644
@@ -77,6 +77,10 @@ struct vdso_data {
 	/* those additional ones don't have to be located anywhere
 	 * special as they were not part of the original systemcfg
 	 */
+	__u32 dcache_block_size;		/* L1 d-cache block size     */
+	__u32 icache_block_size;		/* L1 i-cache block size     */
+	__u32 dcache_log_block_size;		/* L1 d-cache log block size */
+	__u32 icache_log_block_size;		/* L1 i-cache log block size */
 	__s32 wtom_clock_sec;			/* Wall to monotonic clock */
 	__s32 wtom_clock_nsec;
    	__u32 syscall_map_64[SYSCALL_MAP_SIZE]; /* map of syscalls  */

@@ -99,6 +103,10 @@ struct vdso_data {
 	__s32 wtom_clock_sec;			/* Wall to monotonic clock */
 	__s32 wtom_clock_nsec;
    	__u32 syscall_map_32[SYSCALL_MAP_SIZE]; /* map of syscalls */
+	__u32 dcache_block_size;	/* L1 d-cache block size     */
+	__u32 icache_block_size;	/* L1 i-cache block size     */
+	__u32 dcache_log_block_size;	/* L1 d-cache log block size */
+	__u32 icache_log_block_size;	/* L1 i-cache log block size */
 };
 
 #endif /* CONFIG_PPC64 */

Paul Mackerras

2007-11-19 04:07 UTC #3 lore raw

Re: [PATCH v2] [POWERPC] vdso: Fixes for cache block sizes

Quoted 7 lines · Olof Johansson

[POWERPC] vdso: Fixes for cache line sizes
Current VDSO implementation is hardcoded to 128 byte cache blocks, which are only used on IBM's 64-bit processors.
Convert it to get the blocks sizes out of vdso_data instead, similar to how the ppc64 in-kernel cache flush does it.

OK, but you have removed a "crclr cr0*4+so" instruction in a couple of places. They are there so that the functions follow the convention for system calls, where cr0.SO set on return indicates an error, and clear indicates no error. Was there any special reason why you removed them? If not, please put them back (after the last cmp or dot-form instruction).

Paul.

Olof Johansson

author

2007-11-19 04:23 UTC #4 lore raw

Re: [PATCH v2] [POWERPC] vdso: Fixes for cache block sizes

Quoted 10 lines · Paul Mackerras

Quoted 7 lines · Olof Johansson
[POWERPC] vdso: Fixes for cache line sizes
Current VDSO implementation is hardcoded to 128 byte cache blocks, which are only used on IBM's 64-bit processors.
Convert it to get the blocks sizes out of vdso_data instead, similar to how the ppc64 in-kernel cache flush does it.

Quoted 6 lines

OK, but you have removed a "crclr cr0*4+so" instruction in a couple of places. They are there so that the functions follow the convention for system calls, where cr0.SO set on return indicates an error, and clear indicates no error. Was there any special reason why you removed them? If not, please put them back (after the last cmp or dot-form instruction).

Good catch. They must have fallen off when I copied over the in-kernel versions. I'll repost tomorrow.

Thanks,

-Olof

Olof Johansson

author

2007-11-20 01:22 UTC #5 lore raw

PATCH v3 1/1 vdso: Fixes for cache block sizes

[POWERPC] vdso: Fixes for cache line sizes

Current VDSO implementation is hardcoded to 128 byte cache blocks, which are only used on IBM's 64-bit processors.

Convert it to get the blocks sizes out of vdso_data instead, similar to how the ppc64 in-kernel cache flush does it.

Signed-off-by Olof Johansson <[email protected]>

---

Added back the crclr instructions that got removed by mistake when moving over the in-kernel functions.

-Olof

arch/powerpc/kernel/asm-offsets.c +4 -0

index 2c8e756..d67bcd8 100644
@@ -284,6 +284,10 @@ int main(void)
 	DEFINE(CFG_SYSCALL_MAP32, offsetof(struct vdso_data, syscall_map_32));
 	DEFINE(WTOM_CLOCK_SEC, offsetof(struct vdso_data, wtom_clock_sec));
 	DEFINE(WTOM_CLOCK_NSEC, offsetof(struct vdso_data, wtom_clock_nsec));
+	DEFINE(CFG_ICACHE_BLOCKSZ, offsetof(struct vdso_data, icache_block_size));
+	DEFINE(CFG_DCACHE_BLOCKSZ, offsetof(struct vdso_data, dcache_block_size));
+	DEFINE(CFG_ICACHE_LOGBLOCKSZ, offsetof(struct vdso_data, icache_log_block_size));
+	DEFINE(CFG_DCACHE_LOGBLOCKSZ, offsetof(struct vdso_data, dcache_log_block_size));
 #ifdef CONFIG_PPC64
 	DEFINE(CFG_SYSCALL_MAP64, offsetof(struct vdso_data, syscall_map_64));
 	DEFINE(TVAL64_TV_SEC, offsetof(struct timeval, tv_sec));

arch/powerpc/kernel/vdso.c +11 -0

index 2322ba5..3702df7 100644
@@ -699,11 +699,22 @@ static int __init vdso_init(void)
 	vdso_data->icache_size = ppc64_caches.isize;
 	vdso_data->icache_line_size = ppc64_caches.iline_size;
 
+	/* XXXOJN: Blocks should be added to ppc64_caches and used instead */
+	vdso_data->dcache_block_size = ppc64_caches.dline_size;
+	vdso_data->icache_block_size = ppc64_caches.iline_size;
+	vdso_data->dcache_log_block_size = ppc64_caches.log_dline_size;
+	vdso_data->icache_log_block_size = ppc64_caches.log_iline_size;
+
 	/*
 	 * Calculate the size of the 64 bits vDSO
 	 */
 	vdso64_pages = (&vdso64_end - &vdso64_start) >> PAGE_SHIFT;
 	DBG("vdso64_kbase: %p, 0x%x pages\n", vdso64_kbase, vdso64_pages);
+#else
+	vdso_data->dcache_block_size = L1_CACHE_BYTES;
+	vdso_data->dcache_log_block_size = L1_CACHE_SHIFT;
+	vdso_data->icache_block_size = L1_CACHE_BYTES;
+	vdso_data->icache_log_block_size = L1_CACHE_SHIFT;
 #endif /* CONFIG_PPC64 */

arch/powerpc/kernel/vdso32/cacheflush.S +29 ~~-12~~

58 lines not on this page · raw message

arch/powerpc/kernel/vdso64/cacheflush.S +29 ~~-12~~

58 lines not on this page · raw message

include/asm-powerpc/vdso_datapage.h +8 -0

index 8a94f0e..f013932 100644
@@ -77,6 +77,10 @@ struct vdso_data {
 	/* those additional ones don't have to be located anywhere
 	 * special as they were not part of the original systemcfg
 	 */
+	__u32 dcache_block_size;		/* L1 d-cache block size     */
+	__u32 icache_block_size;		/* L1 i-cache block size     */
+	__u32 dcache_log_block_size;		/* L1 d-cache log block size */
+	__u32 icache_log_block_size;		/* L1 i-cache log block size */
 	__s32 wtom_clock_sec;			/* Wall to monotonic clock */
 	__s32 wtom_clock_nsec;
    	__u32 syscall_map_64[SYSCALL_MAP_SIZE]; /* map of syscalls  */

@@ -99,6 +103,10 @@ struct vdso_data {
 	__s32 wtom_clock_sec;			/* Wall to monotonic clock */
 	__s32 wtom_clock_nsec;
    	__u32 syscall_map_32[SYSCALL_MAP_SIZE]; /* map of syscalls */
+	__u32 dcache_block_size;	/* L1 d-cache block size     */
+	__u32 icache_block_size;	/* L1 i-cache block size     */
+	__u32 dcache_log_block_size;	/* L1 d-cache log block size */
+	__u32 icache_log_block_size;	/* L1 i-cache log block size */
 };
 
 #endif /* CONFIG_PPC64 */