Skip to contentSkip to conversation

powerpc: switch VDSO to C implementation

PATCH Superseded Christophe Leroy 2 participants 12 messages Aug 5, 2020 – Aug 6, 2020

Subsystems (2)

v10 of 13 Superseded Latest is v13 →

lore
Share & tools
lore
#0 lore raw

v10 cover powerpc: switch VDSO to C implementation

This is the tenth version of a series to switch powerpc VDSO to generic C implementation.

Changes since v10 are:
- Added a comment explaining the reason for the double stack frame
- Moved back .cfi_register lr next to mflr

Main changes since v9 are:
- Dropped the patches which put the VDSO datapage in front of VDSO text in the mapping
- Adds a second stack frame because the caller doesn't set one, at least on PPC64
- Saving the TOC pointer on PPC64 (is that really needed ?)

This series applies on today's powerpc/merge branch.

See the last patches for details on changes and performance.

Christophe Leroy (5):
powerpc/processor: Move cpu_relax() into asm/vdso/processor.h powerpc/vdso: Prepare for switching VDSO to generic C implementation. powerpc/vdso: Save and restore TOC pointer on PPC64 powerpc/vdso: Switch VDSO to generic C implementation. powerpc/vdso: Provide __kernel_clock_gettime64() on vdso32

 arch/powerpc/Kconfig                         |   2 +
 arch/powerpc/include/asm/clocksource.h       |   7 +
 arch/powerpc/include/asm/processor.h         |  13 +-
 arch/powerpc/include/asm/vdso/clocksource.h  |   7 +
 arch/powerpc/include/asm/vdso/gettimeofday.h | 197 ++++++++++++
 arch/powerpc/include/asm/vdso/processor.h    |  23 ++
 arch/powerpc/include/asm/vdso/vsyscall.h     |  25 ++
 arch/powerpc/include/asm/vdso_datapage.h     |  40 +--
 arch/powerpc/kernel/asm-offsets.c            |  49 +--
 arch/powerpc/kernel/time.c                   |  91 +-----
 arch/powerpc/kernel/vdso.c                   |   5 +-
 arch/powerpc/kernel/vdso32/Makefile          |  32 +-
 arch/powerpc/kernel/vdso32/config-fake32.h   |  34 +++
 arch/powerpc/kernel/vdso32/gettimeofday.S    | 300 +------------------
 arch/powerpc/kernel/vdso32/vdso32.lds.S      |   1 +
 arch/powerpc/kernel/vdso32/vgettimeofday.c   |  35 +++
 arch/powerpc/kernel/vdso64/Makefile          |  23 +-
 arch/powerpc/kernel/vdso64/gettimeofday.S    | 242 +--------------
 arch/powerpc/kernel/vdso64/vgettimeofday.c   |  29 ++
 19 files changed, 453 insertions(+), 702 deletions(-)
 create mode 100644 arch/powerpc/include/asm/clocksource.h
 create mode 100644 arch/powerpc/include/asm/vdso/clocksource.h
 create mode 100644 arch/powerpc/include/asm/vdso/gettimeofday.h
 create mode 100644 arch/powerpc/include/asm/vdso/processor.h
 create mode 100644 arch/powerpc/include/asm/vdso/vsyscall.h
 create mode 100644 arch/powerpc/kernel/vdso32/config-fake32.h
 create mode 100644 arch/powerpc/kernel/vdso32/vgettimeofday.c
 create mode 100644 arch/powerpc/kernel/vdso64/vgettimeofday.c
Signature · 1 line
2.25.0
#10 lore raw

PATCH v10 4/5 powerpc/vdso: Switch VDSO to generic C implementation.

For VDSO32 on PPC64, we create a fake 32 bits config, on the same principle as MIPS architecture, in order to get the correct parts of the different asm header files.

With the C VDSO, the performance is slightly lower, but it is worth it as it will ease maintenance and evolution, and also brings clocks that are not supported with the ASM VDSO.

On an 8xx at 132 MHz, vdsotest with the ASM VDSO:

gettimeofday:    vdso: 828 nsec/call
clock-getres-realtime-coarse:    vdso: 391 nsec/call
clock-gettime-realtime-coarse:    vdso: 614 nsec/call
clock-getres-realtime:    vdso: 460 nsec/call
clock-gettime-realtime:    vdso: 876 nsec/call
clock-getres-monotonic-coarse:    vdso: 399 nsec/call
clock-gettime-monotonic-coarse:    vdso: 691 nsec/call
clock-getres-monotonic:    vdso: 460 nsec/call
clock-gettime-monotonic:    vdso: 1026 nsec/call

On an 8xx at 132 MHz, vdsotest with the C VDSO:

gettimeofday:    vdso: 955 nsec/call
clock-getres-realtime-coarse:    vdso: 545 nsec/call
clock-gettime-realtime-coarse:    vdso: 592 nsec/call
clock-getres-realtime:    vdso: 545 nsec/call
clock-gettime-realtime:    vdso: 941 nsec/call
clock-getres-monotonic-coarse:    vdso: 545 nsec/call
clock-gettime-monotonic-coarse:    vdso: 591 nsec/call
clock-getres-monotonic:    vdso: 545 nsec/call
clock-gettime-monotonic:    vdso: 940 nsec/call

It is even better for gettime with monotonic clocks.

Unsupported clocks with ASM VDSO:

clock-gettime-boottime:    vdso: 3851 nsec/call
clock-gettime-tai:    vdso: 3852 nsec/call
clock-gettime-monotonic-raw:    vdso: 3396 nsec/call

Same clocks with C VDSO:

clock-gettime-tai:    vdso: 941 nsec/call
clock-gettime-monotonic-raw:    vdso: 1001 nsec/call
clock-gettime-monotonic-coarse:    vdso: 591 nsec/call

On an 8321E at 333 MHz, vdsotest with the ASM VDSO:

gettimeofday:    vdso: 220 nsec/call
clock-getres-realtime-coarse:    vdso: 102 nsec/call
clock-gettime-realtime-coarse:    vdso: 178 nsec/call
clock-getres-realtime:    vdso: 129 nsec/call
clock-gettime-realtime:    vdso: 235 nsec/call
clock-getres-monotonic-coarse:    vdso: 105 nsec/call
clock-gettime-monotonic-coarse:    vdso: 208 nsec/call
clock-getres-monotonic:    vdso: 129 nsec/call
clock-gettime-monotonic:    vdso: 274 nsec/call

On an 8321E at 333 MHz, vdsotest with the C VDSO:

gettimeofday:    vdso: 272 nsec/call
clock-getres-realtime-coarse:    vdso: 160 nsec/call
clock-gettime-realtime-coarse:    vdso: 184 nsec/call
clock-getres-realtime:    vdso: 166 nsec/call
clock-gettime-realtime:    vdso: 281 nsec/call
clock-getres-monotonic-coarse:    vdso: 160 nsec/call
clock-gettime-monotonic-coarse:    vdso: 184 nsec/call
clock-getres-monotonic:    vdso: 169 nsec/call
clock-gettime-monotonic:    vdso: 275 nsec/call

---
v9:
- Rebased (Impact on arch/powerpc/kernel/vdso??/Makefile

v7:
- Split out preparatory changes in a new preceding patch
- Added -fasynchronous-unwind-tables to CC flags.

v6:
- Added missing prototypes in asm/vdso/gettimeofday.h for __c_kernel_ functions.
- Using STACK_FRAME_OVERHEAD instead of INT_FRAME_SIZE
- Rebased on powerpc/merge as of 7 Apr 2020
- Fixed build failure with gcc 9
- Added a patch to create asm/vdso/processor.h and more cpu_relax() in it

---
 arch/powerpc/Kconfig                       |   2 +
 arch/powerpc/include/asm/vdso/vsyscall.h   |  25 ++
 arch/powerpc/include/asm/vdso_datapage.h   |  40 +--
 arch/powerpc/kernel/asm-offsets.c          |  49 +---
 arch/powerpc/kernel/time.c                 |  91 +------
 arch/powerpc/kernel/vdso.c                 |   5 +-
 arch/powerpc/kernel/vdso32/Makefile        |  32 ++-
 arch/powerpc/kernel/vdso32/config-fake32.h |  34 +++
 arch/powerpc/kernel/vdso32/gettimeofday.S  | 291 +--------------------
 arch/powerpc/kernel/vdso64/Makefile        |  23 +-
 arch/powerpc/kernel/vdso64/gettimeofday.S  | 242 +----------------
 11 files changed, 143 insertions(+), 691 deletions(-)
 create mode 100644 arch/powerpc/include/asm/vdso/vsyscall.h
 create mode 100644 arch/powerpc/kernel/vdso32/config-fake32.h
arch/powerpc/Kconfig +2 -0
index 8c7656cc10eb..9977ab939b42 100644
@@ -171,6 +171,7 @@ config PPC
 	select GENERIC_STRNCPY_FROM_USER
 	select GENERIC_STRNLEN_USER
 	select GENERIC_TIME_VSYSCALL
+	select GENERIC_GETTIMEOFDAY
 	select HAVE_ARCH_AUDITSYSCALL
 	select HAVE_ARCH_HUGE_VMAP		if PPC_BOOK3S_64 && PPC_RADIX_MMU
 	select HAVE_ARCH_JUMP_LABEL
@@ -202,6 +203,7 @@ config PPC
 	select HAVE_FUNCTION_GRAPH_TRACER
 	select HAVE_FUNCTION_TRACER
 	select HAVE_GCC_PLUGINS			if GCC_VERSION >= 50200   # plugin support on gcc <= 5.1 is buggy on PPC
+	select HAVE_GENERIC_VDSO
 	select HAVE_HW_BREAKPOINT		if PERF_EVENTS && (PPC_BOOK3S || PPC_8xx)
 	select HAVE_IDE
 	select HAVE_IOREMAP_PROT
arch/powerpc/include/asm/vdso/vsyscall.h new file +25 -0
new file mode 100644
index 000000000000..c56a030c0623
@@ -0,0 +1,25 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __ASM_VDSO_VSYSCALL_H
+#define __ASM_VDSO_VSYSCALL_H
+
+#ifndef __ASSEMBLY__
+
+#include <linux/timekeeper_internal.h>
+#include <asm/vdso_datapage.h>
+
+/*
+ * Update the vDSO data page to keep in sync with kernel timekeeping.
+ */
+static __always_inline
+struct vdso_data *__arch_get_k_vdso_data(void)
+{
+	return vdso_data->data;
+}
+#define __arch_get_k_vdso_data __arch_get_k_vdso_data
+
+/* The asm-generic header needs to be included after the definitions above */
+#include <asm-generic/vdso/vsyscall.h>
+
+#endif /* !__ASSEMBLY__ */
+
+#endif /* __ASM_VDSO_VSYSCALL_H */
arch/powerpc/include/asm/vdso_datapage.h +13 -27
index b9ef6cf50ea5..c4d320504d26 100644
@@ -36,6 +36,7 @@
 
 #include <linux/unistd.h>
 #include <linux/time.h>
+#include <vdso/datapage.h>
 
 #define SYSCALL_MAP_SIZE      ((NR_syscalls + 31) / 32)
 
@@ -45,7 +46,7 @@
 
 #ifdef CONFIG_PPC64
 
-struct vdso_data {
+struct vdso_arch_data {
 	__u8  eye_catcher[16];		/* Eyecatcher: SYSTEMCFG:PPC64	0x00 */
 	struct {			/* Systemcfg version numbers	     */
 		__u32 major;		/* Major number			0x10 */
@@ -59,13 +60,13 @@ struct vdso_data {
 	__u32 processor;		/* Processor type		0x1C */
 	__u64 processorCount;		/* # of physical processors	0x20 */
 	__u64 physicalMemorySize;	/* Size of real memory(B)	0x28 */
-	__u64 tb_orig_stamp;		/* Timebase at boot		0x30 */
+	__u64 tb_orig_stamp;		/* (NU) Timebase at boot	0x30 */
 	__u64 tb_ticks_per_sec;		/* Timebase tics / sec		0x38 */
-	__u64 tb_to_xs;			/* Inverse of TB to 2^20	0x40 */
-	__u64 stamp_xsec;		/*				0x48 */
-	__u64 tb_update_count;		/* Timebase atomicity ctr	0x50 */
-	__u32 tz_minuteswest;		/* Minutes west of Greenwich	0x58 */
-	__u32 tz_dsttime;		/* Type of dst correction	0x5C */
+	__u64 tb_to_xs;			/* (NU) Inverse of TB to 2^20	0x40 */
+	__u64 stamp_xsec;		/* (NU)				0x48 */
+	__u64 tb_update_count;		/* (NU) Timebase atomicity ctr	0x50 */
+	__u32 tz_minuteswest;		/* (NU) Min. west of Greenwich	0x58 */
+	__u32 tz_dsttime;		/* (NU) Type of dst correction	0x5C */
 	__u32 dcache_size;		/* L1 d-cache size		0x60 */
 	__u32 dcache_line_size;		/* L1 d-cache line size		0x64 */
 	__u32 icache_size;		/* L1 i-cache size		0x68 */
@@ -78,14 +79,10 @@ struct vdso_data {
 	__u32 icache_block_size;		/* L1 i-cache block size     */
 	__u32 dcache_log_block_size;		/* L1 d-cache log block size */
 	__u32 icache_log_block_size;		/* L1 i-cache log block size */
-	__u32 stamp_sec_fraction;		/* fractional seconds of stamp_xtime */
-	__s32 wtom_clock_nsec;			/* Wall to monotonic clock nsec */
-	__s64 wtom_clock_sec;			/* Wall to monotonic clock sec */
-	__s64 stamp_xtime_sec;			/* xtime secs as at tb_orig_stamp */
-	__s64 stamp_xtime_nsec;			/* xtime nsecs as at tb_orig_stamp */
-	__u32 hrtimer_res;			/* hrtimer resolution */
    	__u32 syscall_map_64[SYSCALL_MAP_SIZE]; /* map of syscalls  */
    	__u32 syscall_map_32[SYSCALL_MAP_SIZE]; /* map of syscalls */
+
+	struct vdso_data data[CS_BASES];
 };
 
 #else /* CONFIG_PPC64 */
@@ -93,26 +90,15 @@ struct vdso_data {
 /*
  * And here is the simpler 32 bits version
  */
-struct vdso_data {
-	__u64 tb_orig_stamp;		/* Timebase at boot		0x30 */
+struct vdso_arch_data {
 	__u64 tb_ticks_per_sec;		/* Timebase tics / sec		0x38 */
-	__u64 tb_to_xs;			/* Inverse of TB to 2^20	0x40 */
-	__u64 stamp_xsec;		/*				0x48 */
-	__u32 tb_update_count;		/* Timebase atomicity ctr	0x50 */
-	__u32 tz_minuteswest;		/* Minutes west of Greenwich	0x58 */
-	__u32 tz_dsttime;		/* Type of dst correction	0x5C */
-	__s32 wtom_clock_sec;			/* Wall to monotonic clock */
-	__s32 wtom_clock_nsec;
-	__s32 stamp_xtime_sec;		/* xtime seconds as at tb_orig_stamp */
-	__s32 stamp_xtime_nsec;		/* xtime nsecs as at tb_orig_stamp */
-	__u32 stamp_sec_fraction;	/* fractional seconds of stamp_xtime */
-	__u32 hrtimer_res;		/* hrtimer resolution */
    	__u32 syscall_map_32[SYSCALL_MAP_SIZE]; /* map of syscalls */
+	struct vdso_data data[CS_BASES];
 };
 
 #endif /* CONFIG_PPC64 */
 
-extern struct vdso_data *vdso_data;
+extern struct vdso_arch_data *vdso_data;
 
 #else /* __ASSEMBLY__ */
 
arch/powerpc/kernel/asm-offsets.c +9 -40

56 lines not on this page · raw message

arch/powerpc/kernel/time.c +1 -90

109 lines not on this page · raw message

arch/powerpc/kernel/vdso.c +3 -2
index 8dad44262e75..23208a051af5 100644
@@ -17,6 +17,7 @@
 #include <linux/elf.h>
 #include <linux/security.h>
 #include <linux/memblock.h>
+#include <vdso/datapage.h>
 
 #include <asm/processor.h>
 #include <asm/mmu.h>
@@ -70,10 +71,10 @@ static int vdso_ready;
  * with it, it will become dynamically allocated
  */
 static union {
-	struct vdso_data	data;
+	struct vdso_arch_data	data;
 	u8			page[PAGE_SIZE];
 } vdso_data_store __page_aligned_data;
-struct vdso_data *vdso_data = &vdso_data_store.data;
+struct vdso_arch_data *vdso_data = &vdso_data_store.data;
 
 /* Format of the patch table */
 struct vdso_patch_def
arch/powerpc/kernel/vdso32/Makefile +27 -5
index 87ab1152d5ce..b922044236dd 100644
@@ -2,7 +2,23 @@
 
 # List of files in the vdso, has to be asm only for now
 
-obj-vdso32 = sigtramp.o gettimeofday.o datapage.o cacheflush.o note.o getcpu.o
+ARCH_REL_TYPE_ABS := R_PPC_JUMP_SLOT|R_PPC_GLOB_DAT|R_PPC_ADDR32|R_PPC_ADDR24|R_PPC_ADDR16|R_PPC_ADDR16_LO|R_PPC_ADDR16_HI|R_PPC_ADDR16_HA|R_PPC_ADDR14|R_PPC_ADDR14_BRTAKEN|R_PPC_ADDR14_BRNTAKEN
+include $(srctree)/lib/vdso/Makefile
+
+obj-vdso32 = sigtramp.o datapage.o cacheflush.o note.o getcpu.o $(obj-vdso32-y)
+obj-vdso32 += gettimeofday.o
+
+ifneq ($(c-gettimeofday-y),)
+  ifdef CONFIG_PPC64
+    CFLAGS_vgettimeofday.o += -include $(srctree)/$(src)/config-fake32.h
+  endif
+  CFLAGS_vgettimeofday.o += -include $(c-gettimeofday-y)
+  CFLAGS_vgettimeofday.o += $(DISABLE_LATENT_ENTROPY_PLUGIN)
+  CFLAGS_vgettimeofday.o += $(call cc-option, -fno-stack-protector)
+  CFLAGS_vgettimeofday.o += -DDISABLE_BRANCH_PROFILING
+  CFLAGS_vgettimeofday.o += -ffreestanding -fasynchronous-unwind-tables
+  CFLAGS_REMOVE_vgettimeofday.o = $(CC_FLAGS_FTRACE)
+endif
 
 # Build rules
 
@@ -15,6 +31,7 @@ endif
 CC32FLAGS :=
 ifdef CONFIG_PPC64
 CC32FLAGS += -m32
+KBUILD_CFLAGS := $(filter-out -mcmodel=medium,$(KBUILD_CFLAGS))
 endif
 
 targets := $(obj-vdso32) vdso32.so vdso32.so.dbg
@@ -23,6 +40,7 @@ obj-vdso32 := $(addprefix $(obj)/, $(obj-vdso32))
 GCOV_PROFILE := n
 KCOV_INSTRUMENT := n
 UBSAN_SANITIZE := n
+KASAN_SANITIZE := n
 
 ccflags-y := -shared -fno-common -fno-builtin -nostdlib \
 	-Wl,-soname=linux-vdso32.so.1 -Wl,--hash-style=both
@@ -36,8 +54,8 @@ CPPFLAGS_vdso32.lds += -P -C -Upowerpc
 $(obj)/vdso32_wrapper.o : $(obj)/vdso32.so
 
 # link rule for the .so file, .lds has to be first
-$(obj)/vdso32.so.dbg: $(src)/vdso32.lds $(obj-vdso32) FORCE
-	$(call if_changed,vdso32ld)
+$(obj)/vdso32.so.dbg: $(src)/vdso32.lds $(obj-vdso32) $(obj)/vgettimeofday.o FORCE
+	$(call if_changed,vdso32ld_and_check)
 
 # strip rule for the .so file
 $(obj)/%.so: OBJCOPYFLAGS := -S
@@ -47,12 +65,16 @@ $(obj)/%.so: $(obj)/%.so.dbg FORCE
 # assembly rules for the .S files
 $(obj-vdso32): %.o: %.S FORCE
 	$(call if_changed_dep,vdso32as)
+$(obj)/vgettimeofday.o: %.o: %.c FORCE
+	$(call if_changed_dep,vdso32cc)
 
 # actual build commands
-quiet_cmd_vdso32ld = VDSO32L $@
-      cmd_vdso32ld = $(VDSOCC) $(c_flags) $(CC32FLAGS) -o $@ $(call cc-ldoption, -Wl$(comma)--orphan-handling=warn) -Wl,-T$(filter %.lds,$^) $(filter %.o,$^)
+quiet_cmd_vdso32ld_and_check = VDSO32L $@
+      cmd_vdso32ld_and_check = $(VDSOCC) $(c_flags) $(CC32FLAGS) -o $@ $(call cc-ldoption, -Wl$(comma)--orphan-handling=warn) -Wl,-T$(filter %.lds,$^) $(filter %.o,$^) ; $(cmd_vdso_check)
 quiet_cmd_vdso32as = VDSO32A $@
       cmd_vdso32as = $(VDSOCC) $(a_flags) $(CC32FLAGS) -c -o $@ $<
+quiet_cmd_vdso32cc = VDSO32C $@
+      cmd_vdso32cc = $(VDSOCC) $(c_flags) $(CC32FLAGS) -c -o $@ $<
 
 # install commands for the unstripped file
 quiet_cmd_vdso_install = INSTALL $@
arch/powerpc/kernel/vdso32/config-fake32.h new file +34 -0
new file mode 100644
index 000000000000..e16041fc15c9
@@ -0,0 +1,34 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * In case of a 32 bit VDSO for a 64 bit kernel fake a 32 bit kernel
+ * configuration.
+ */
+
+#undef CONFIG_PPC64
+#undef CONFIG_64BIT
+#define CONFIG_PPC32
+#define CONFIG_32BIT 1
+#define CONFIG_GENERIC_ATOMIC64 1
+
+#ifdef CONFIG_PPC_BOOK3S_64
+#undef CONFIG_PPC_BOOK3S_64
+#undef CONFIG_PPC_PSERIES
+#define CONFIG_PPC_BOOK3S_32
+#else
+#define CONFIG_PPC_MMU_NOHASH_32
+#define CONFIG_FSL_BOOKE
+#endif
+
+#define CONFIG_TASK_SIZE	0
+#undef CONFIG_MMIOWB
+#undef CONFIG_PPC_SPLPAR
+#undef CONFIG_SPARSEMEM
+#undef CONFIG_PGTABLE_LEVELS
+#define CONFIG_PGTABLE_LEVELS	2
+#undef CONFIG_TRANSPARENT_HUGEPAGE
+#undef CONFIG_SPARSEMEM_VMEMMAP
+#undef CONFIG_FLATMEM
+#define CONFIG_FLATMEM
+#undef CONFIG_PPC_INDIRECT_MMIO
+#undef CONFIG_PPC_INDIRECT_PIO
+#undef CONFIG_EEH
arch/powerpc/kernel/vdso32/gettimeofday.S +5 -286

319 lines not on this page · raw message

arch/powerpc/kernel/vdso64/Makefile +19 -4
index 38c317f25141..7890d889f9c5 100644
@@ -1,8 +1,20 @@
 # SPDX-License-Identifier: GPL-2.0
 # List of files in the vdso, has to be asm only for now
 
+ARCH_REL_TYPE_ABS := R_PPC_JUMP_SLOT|R_PPC_GLOB_DAT|R_PPC_ADDR32|R_PPC_ADDR24|R_PPC_ADDR16|R_PPC_ADDR16_LO|R_PPC_ADDR16_HI|R_PPC_ADDR16_HA|R_PPC_ADDR14|R_PPC_ADDR14_BRTAKEN|R_PPC_ADDR14_BRNTAKEN
+include $(srctree)/lib/vdso/Makefile
+
 obj-vdso64 = sigtramp.o gettimeofday.o datapage.o cacheflush.o note.o getcpu.o
 
+ifneq ($(c-gettimeofday-y),)
+  CFLAGS_vgettimeofday.o += -include $(c-gettimeofday-y)
+  CFLAGS_vgettimeofday.o += $(DISABLE_LATENT_ENTROPY_PLUGIN)
+  CFLAGS_vgettimeofday.o += $(call cc-option, -fno-stack-protector)
+  CFLAGS_vgettimeofday.o += -DDISABLE_BRANCH_PROFILING
+  CFLAGS_vgettimeofday.o += -ffreestanding -fasynchronous-unwind-tables
+  CFLAGS_REMOVE_vgettimeofday.o = $(CC_FLAGS_FTRACE)
+endif
+
 # Build rules
 
 targets := $(obj-vdso64) vdso64.so vdso64.so.dbg
@@ -11,6 +23,7 @@ obj-vdso64 := $(addprefix $(obj)/, $(obj-vdso64))
 GCOV_PROFILE := n
 KCOV_INSTRUMENT := n
 UBSAN_SANITIZE := n
+KASAN_SANITIZE := n
 
 ccflags-y := -shared -fno-common -fno-builtin -nostdlib \
 	-Wl,-soname=linux-vdso64.so.1 -Wl,--hash-style=both
@@ -20,12 +33,14 @@ obj-y += vdso64_wrapper.o
 extra-y += vdso64.lds
 CPPFLAGS_vdso64.lds += -P -C -U$(ARCH)
 
+$(obj)/vgettimeofday.o: %.o: %.c FORCE
+
 # Force dependency (incbin is bad)
 $(obj)/vdso64_wrapper.o : $(obj)/vdso64.so
 
 # link rule for the .so file, .lds has to be first
-$(obj)/vdso64.so.dbg: $(src)/vdso64.lds $(obj-vdso64) FORCE
-	$(call if_changed,vdso64ld)
+$(obj)/vdso64.so.dbg: $(src)/vdso64.lds $(obj-vdso64) $(obj)/vgettimeofday.o FORCE
+	$(call if_changed,vdso64ld_and_check)
 
 # strip rule for the .so file
 $(obj)/%.so: OBJCOPYFLAGS := -S
@@ -33,8 +48,8 @@ $(obj)/%.so: $(obj)/%.so.dbg FORCE
 	$(call if_changed,objcopy)
 
 # actual build commands
-quiet_cmd_vdso64ld = VDSO64L $@
-      cmd_vdso64ld = $(CC) $(c_flags) -o $@ -Wl,-T$(filter %.lds,$^) $(filter %.o,$^) $(call cc-ldoption, -Wl$(comma)--orphan-handling=warn)
+quiet_cmd_vdso64ld_and_check = VDSO64L $@
+      cmd_vdso64ld_and_check = $(CC) $(c_flags) -o $@ -Wl,-T$(filter %.lds,$^) $(filter %.o,$^) $(call cc-ldoption, -Wl$(comma)--orphan-handling=warn); $(cmd_vdso_check)
 
 # install commands for the unstripped file
 quiet_cmd_vdso_install = INSTALL $@
arch/powerpc/kernel/vdso64/gettimeofday.S +5 -237

270 lines not on this page · raw message

Signature · 1 line
2.25.0
#1 lore raw

PATCH v10 1/5 powerpc/processor: Move cpu_relax() into asm/vdso/processor.h

In reply to Christophe Leroy

cpu_relax() need to be in asm/vdso/processor.h to be used by the C VDSO generic library.

Move it there.

---
v9: Forgot to remove cpu_relax() from processor.h in v8

---
 arch/powerpc/include/asm/processor.h      | 13 ++-----------
 arch/powerpc/include/asm/vdso/processor.h | 23 +++++++++++++++++++++++
 2 files changed, 25 insertions(+), 11 deletions(-)
 create mode 100644 arch/powerpc/include/asm/vdso/processor.h
arch/powerpc/include/asm/processor.h +2 -11
index ed0d633ab5aa..c1ba9c8d9b90 100644
@@ -6,6 +6,8 @@
  * Copyright (C) 2001 PPC 64 Team, IBM Corp
  */
 
+#include <vdso/processor.h>
+
 #include <asm/reg.h>
 
 #ifdef CONFIG_VSX
@@ -63,14 +65,6 @@ extern int _chrp_type;
 
 #endif /* defined(__KERNEL__) && defined(CONFIG_PPC32) */
 
-/* Macros for adjusting thread priority (hardware multi-threading) */
-#define HMT_very_low()   asm volatile("or 31,31,31   # very low priority")
-#define HMT_low()	 asm volatile("or 1,1,1	     # low priority")
-#define HMT_medium_low() asm volatile("or 6,6,6      # medium low priority")
-#define HMT_medium()	 asm volatile("or 2,2,2	     # medium priority")
-#define HMT_medium_high() asm volatile("or 5,5,5      # medium high priority")
-#define HMT_high()	 asm volatile("or 3,3,3	     # high priority")
-
 #ifdef __KERNEL__
 
 #ifdef CONFIG_PPC64
@@ -350,7 +344,6 @@ static inline unsigned long __pack_fe01(unsigned int fpmode)
 }
 
 #ifdef CONFIG_PPC64
-#define cpu_relax()	do { HMT_low(); HMT_medium(); barrier(); } while (0)
 
 #define spin_begin()	HMT_low()
 
@@ -369,8 +362,6 @@ do {								\
 	}							\
 } while (0)
 
-#else
-#define cpu_relax()	barrier()
 #endif
 
 /* Check that a certain kernel stack pointer is valid in task_struct p */
arch/powerpc/include/asm/vdso/processor.h new file +23 -0
new file mode 100644
index 000000000000..39b9beace9ca
@@ -0,0 +1,23 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+#ifndef __ASM_VDSO_PROCESSOR_H
+#define __ASM_VDSO_PROCESSOR_H
+
+#ifndef __ASSEMBLY__
+
+/* Macros for adjusting thread priority (hardware multi-threading) */
+#define HMT_very_low()		asm volatile("or 31, 31, 31	# very low priority")
+#define HMT_low()		asm volatile("or 1, 1, 1	# low priority")
+#define HMT_medium_low()	asm volatile("or 6, 6, 6	# medium low priority")
+#define HMT_medium()		asm volatile("or 2, 2, 2	# medium priority")
+#define HMT_medium_high()	asm volatile("or 5, 5, 5	# medium high priority")
+#define HMT_high()		asm volatile("or 3, 3, 3	# high priority")
+
+#ifdef CONFIG_PPC64
+#define cpu_relax()	do { HMT_low(); HMT_medium(); barrier(); } while (0)
+#else
+#define cpu_relax()	barrier()
+#endif
+
+#endif /* __ASSEMBLY__ */
+
+#endif /* __ASM_VDSO_PROCESSOR_H */
Signature · 1 line
2.25.0
#9 lore raw

PATCH v10 3/5 powerpc/vdso: Save and restore TOC pointer on PPC64

In reply to Christophe Leroy

On PPC64, the TOC pointer needs to be saved and restored.

---
v9: New.

I'm not sure this is really needed, I can't see the VDSO C code doing anything with r2, at least on ppc64_defconfig.

So I let you decide whether you take it or not.

---
 arch/powerpc/include/asm/vdso/gettimeofday.h | 12 ++++++++++++
 1 file changed, 12 insertions(+)
arch/powerpc/include/asm/vdso/gettimeofday.h +12 -0
index e2c462796a22..8ed4329c9fb8 100644
@@ -19,10 +19,16 @@
   .cfi_register lr, r0
 	PPC_STLU	r1, -STACK_FRAME_OVERHEAD(r1)
 	PPC_STL		r0, STACK_FRAME_OVERHEAD + PPC_LR_STKOFF(r1)
+#ifdef CONFIG_PPC64
+	PPC_STL		r2, STACK_FRAME_OVERHEAD + STK_GOT(r1)
+#endif
 	get_datapage	r5, r0
 	addi		r5, r5, VDSO_DATA_OFFSET
 	bl		\funct
 	PPC_LL		r0, STACK_FRAME_OVERHEAD + PPC_LR_STKOFF(r1)
+#ifdef CONFIG_PPC64
+	PPC_LL		r2, STACK_FRAME_OVERHEAD + STK_GOT(r1)
+#endif
 	cmpwi		r3, 0
 	mtlr		r0
   .cfi_restore lr
@@ -42,10 +48,16 @@
   .cfi_register lr, r0
 	PPC_STLU	r1, -STACK_FRAME_OVERHEAD(r1)
 	PPC_STL		r0, STACK_FRAME_OVERHEAD + PPC_LR_STKOFF(r1)
+#ifdef CONFIG_PPC64
+	PPC_STL		r2, STACK_FRAME_OVERHEAD + STK_GOT(r1)
+#endif
 	get_datapage	r4, r0
 	addi		r4, r4, VDSO_DATA_OFFSET
 	bl		\funct
 	PPC_LL		r0, STACK_FRAME_OVERHEAD + PPC_LR_STKOFF(r1)
+#ifdef CONFIG_PPC64
+	PPC_LL		r2, STACK_FRAME_OVERHEAD + STK_GOT(r1)
+#endif
 	crclr		so
 	mtlr		r0
   .cfi_restore lr
Signature · 1 line
2.25.0
#11 lore raw

PATCH v10 5/5 powerpc/vdso: Provide __kernel_clock_gettime64() on vdso32

In reply to Christophe Leroy

Provides __kernel_clock_gettime64() on vdso32. This is the 64 bits version of __kernel_clock_gettime() which is y2038 compliant.

---
 arch/powerpc/kernel/vdso32/gettimeofday.S  | 9 +++++++++
 arch/powerpc/kernel/vdso32/vdso32.lds.S    | 1 +
 arch/powerpc/kernel/vdso32/vgettimeofday.c | 6 ++++++
 3 files changed, 16 insertions(+)
arch/powerpc/kernel/vdso32/gettimeofday.S +9 -0
index fd7b01c51281..a6e29f880e0e 100644
@@ -35,6 +35,15 @@ V_FUNCTION_BEGIN(__kernel_clock_gettime)
 	cvdso_call __c_kernel_clock_gettime
 V_FUNCTION_END(__kernel_clock_gettime)
 
+/*
+ * Exact prototype of clock_gettime64()
+ *
+ * int __kernel_clock_gettime64(clockid_t clock_id, struct __timespec64 *ts);
+ *
+ */
+V_FUNCTION_BEGIN(__kernel_clock_gettime64)
+	cvdso_call __c_kernel_clock_gettime64
+V_FUNCTION_END(__kernel_clock_gettime64)
 
 /*
  * Exact prototype of clock_getres()
arch/powerpc/kernel/vdso32/vdso32.lds.S +1 -0
index 4c985467a668..582c5b046cc9 100644
@@ -148,6 +148,7 @@ VERSION
 #ifndef CONFIG_PPC_BOOK3S_601
 		__kernel_gettimeofday;
 		__kernel_clock_gettime;
+		__kernel_clock_gettime64;
 		__kernel_clock_getres;
 		__kernel_time;
 		__kernel_get_tbfreq;
arch/powerpc/kernel/vdso32/vgettimeofday.c +6 -0
index 0b9ab4c22ef2..f7f71fecf4ed 100644
@@ -11,6 +11,12 @@ int __c_kernel_clock_gettime(clockid_t clock, struct old_timespec32 *ts,
 	return __cvdso_clock_gettime32_data(vd, clock, ts);
 }
 
+int __c_kernel_clock_gettime64(clockid_t clock, struct __kernel_timespec *ts,
+			       const struct vdso_data *vd)
+{
+	return __cvdso_clock_gettime_data(vd, clock, ts);
+}
+
 int __c_kernel_gettimeofday(struct __kernel_old_timeval *tv, struct timezone *tz,
 			    const struct vdso_data *vd)
 {
Signature · 1 line
2.25.0
#2 lore raw

PATCH v10 2/5 powerpc/vdso: Prepare for switching VDSO to generic C implementation.

In reply to Christophe Leroy

Prepare for switching VDSO to generic C implementation in following patch. Here, we:
- Prepare the helpers to call the C VDSO functions
- Prepare the required callbacks for the C VDSO functions
- Prepare the clocksource.h files to define VDSO_ARCH_CLOCKMODES
- Add the C trampolines to the generic C VDSO functions

powerpc is a bit special for VDSO as well as system calls in the way that it requires setting CR SO bit which cannot be done in C. Therefore, entry/exit needs to be performed in ASM.

Implementing __arch_get_vdso_data() would clobber the link register, requiring the caller to save it. As the ASM calling function already has to set a stack frame and saves the link register before calling the C vdso function, retriving the vdso data pointer there is lighter.

Implement __arch_vdso_capable() and:
- When the timebase is used, make it always return true.
- When the RTC clock is used, make it always return false.

Provide vdso_shift_ns(), as the generic x >> s gives the following bad result:

  18:	35 25 ff e0 	addic.  r9,r5,-32
  1c:	41 80 00 10 	blt     2c <shift+0x14>
  20:	7c 64 4c 30 	srw     r4,r3,r9
  24:	38 60 00 00 	li      r3,0
...
  2c:	54 69 08 3c 	rlwinm  r9,r3,1,0,30
  30:	21 45 00 1f 	subfic  r10,r5,31
  34:	7c 84 2c 30 	srw     r4,r4,r5
  38:	7d 29 50 30 	slw     r9,r9,r10
  3c:	7c 63 2c 30 	srw     r3,r3,r5
  40:	7d 24 23 78 	or      r4,r9,r4

In our case the shift is always <= 32. In addition, the upper 32 bits of the result are likely nul. Lets GCC know it, it also optimises the following calculations.

With the patch, we get:

   0:	21 25 00 20 	subfic  r9,r5,32
   4:	7c 69 48 30 	slw     r9,r3,r9
   8:	7c 84 2c 30 	srw     r4,r4,r5
   c:	7d 24 23 78 	or      r4,r9,r4
  10:	7c 63 2c 30 	srw     r3,r3,r5

---
v10:
- Added a comment to explain the reason for the two stack frames.
- Moved back the .cfi_register next to mflr

v9:
- No more modification of __get_datapage(). Offset is added after.
- Adding a second stack frame because the PPC VDSO ABI doesn't force the caller to set one.

v8:
- New, splitted out of last patch of the series

---
 arch/powerpc/include/asm/clocksource.h       |   7 +
 arch/powerpc/include/asm/vdso/clocksource.h  |   7 +
 arch/powerpc/include/asm/vdso/gettimeofday.h | 185 +++++++++++++++++++
 arch/powerpc/kernel/vdso32/vgettimeofday.c   |  29 +++
 arch/powerpc/kernel/vdso64/vgettimeofday.c   |  29 +++
 5 files changed, 257 insertions(+)
 create mode 100644 arch/powerpc/include/asm/clocksource.h
 create mode 100644 arch/powerpc/include/asm/vdso/clocksource.h
 create mode 100644 arch/powerpc/include/asm/vdso/gettimeofday.h
 create mode 100644 arch/powerpc/kernel/vdso32/vgettimeofday.c
 create mode 100644 arch/powerpc/kernel/vdso64/vgettimeofday.c
arch/powerpc/include/asm/clocksource.h new file +7 -0
new file mode 100644
index 000000000000..482185566b0c
@@ -0,0 +1,7 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _ASM_CLOCKSOURCE_H
+#define _ASM_CLOCKSOURCE_H
+
+#include <asm/vdso/clocksource.h>
+
+#endif
arch/powerpc/include/asm/vdso/clocksource.h new file +7 -0
new file mode 100644
index 000000000000..ec5d672d2569
@@ -0,0 +1,7 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __ASM_VDSOCLOCKSOURCE_H
+#define __ASM_VDSOCLOCKSOURCE_H
+
+#define VDSO_ARCH_CLOCKMODES	VDSO_CLOCKMODE_ARCHTIMER
+
+#endif
arch/powerpc/include/asm/vdso/gettimeofday.h new file +185 -0

185 lines not on this page · raw message

arch/powerpc/kernel/vdso32/vgettimeofday.c new file +29 -0
new file mode 100644
index 000000000000..0b9ab4c22ef2
@@ -0,0 +1,29 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Powerpc userspace implementations of gettimeofday() and similar.
+ */
+#include <linux/time.h>
+#include <linux/types.h>
+
+int __c_kernel_clock_gettime(clockid_t clock, struct old_timespec32 *ts,
+			     const struct vdso_data *vd)
+{
+	return __cvdso_clock_gettime32_data(vd, clock, ts);
+}
+
+int __c_kernel_gettimeofday(struct __kernel_old_timeval *tv, struct timezone *tz,
+			    const struct vdso_data *vd)
+{
+	return __cvdso_gettimeofday_data(vd, tv, tz);
+}
+
+int __c_kernel_clock_getres(clockid_t clock_id, struct old_timespec32 *res,
+			    const struct vdso_data *vd)
+{
+	return __cvdso_clock_getres_time32_data(vd, clock_id, res);
+}
+
+__kernel_old_time_t __c_kernel_time(__kernel_old_time_t *time, const struct vdso_data *vd)
+{
+	return __cvdso_time_data(vd, time);
+}
arch/powerpc/kernel/vdso64/vgettimeofday.c new file +29 -0
new file mode 100644
index 000000000000..5b5500058344
@@ -0,0 +1,29 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Powerpc userspace implementations of gettimeofday() and similar.
+ */
+#include <linux/time.h>
+#include <linux/types.h>
+
+int __c_kernel_clock_gettime(clockid_t clock, struct __kernel_timespec *ts,
+			     const struct vdso_data *vd)
+{
+	return __cvdso_clock_gettime_data(vd, clock, ts);
+}
+
+int __c_kernel_gettimeofday(struct __kernel_old_timeval *tv, struct timezone *tz,
+			    const struct vdso_data *vd)
+{
+	return __cvdso_gettimeofday_data(vd, tv, tz);
+}
+
+int __c_kernel_clock_getres(clockid_t clock_id, struct __kernel_timespec *res,
+			    const struct vdso_data *vd)
+{
+	return __cvdso_clock_getres_data(vd, clock_id, res);
+}
+
+__kernel_old_time_t __c_kernel_time(__kernel_old_time_t *time, const struct vdso_data *vd)
+{
+	return __cvdso_time_data(vd, time);
+}
Signature · 1 line
2.25.0
#3 lore raw

Re: [PATCH v10 2/5] powerpc/vdso: Prepare for switching VDSO to generic C implementation.

Hi!

Quoted 19 lines · Christophe Leroy

Provide vdso_shift_ns(), as the generic x >> s gives the following bad result:

  18:	35 25 ff e0 	addic.  r9,r5,-32
  1c:	41 80 00 10 	blt     2c <shift+0x14>
  20:	7c 64 4c 30 	srw     r4,r3,r9
  24:	38 60 00 00 	li      r3,0
...
  2c:	54 69 08 3c 	rlwinm  r9,r3,1,0,30
  30:	21 45 00 1f 	subfic  r10,r5,31
  34:	7c 84 2c 30 	srw     r4,r4,r5
  38:	7d 29 50 30 	slw     r9,r9,r10
  3c:	7c 63 2c 30 	srw     r3,r3,r5
  40:	7d 24 23 78 	or      r4,r9,r4

In our case the shift is always <= 32. In addition, the upper 32 bits of the result are likely nul. Lets GCC know it, it also optimises the following calculations.

Quoted 6 lines

With the patch, we get:

   0:	21 25 00 20 	subfic  r9,r5,32
   4:	7c 69 48 30 	slw     r9,r3,r9
   8:	7c 84 2c 30 	srw     r4,r4,r5
   c:	7d 24 23 78 	or      r4,r9,r4
  10:	7c 63 2c 30 	srw     r3,r3,r5

See below. Such code is valid on PowerPC for all shift < 64, and a future version of GCC will do that (it is on various TODO lists, it is bound to happen *some* day ;-), but it won't help you yet of course).

Quoted 5 lines

+/*
+ * The macros sets two stack frames, one for the caller and one for the callee
+ * because there are no requirement for the caller to set a stack frame when
+ * calling VDSO so it may have omitted to set one, especially on PPC64
+ */

If the caller follows the ABI, there always is a stack frame. So what is going on?

Quoted 9 lines

+/*
+ * powerpc specific delta calculation.
+ *
+ * This variant removes the masking of the subtraction because the
+ * clocksource mask of all VDSO capable clocksources on powerpc is U64_MAX
+ * which would result in a pointless operation. The compiler cannot
+ * optimize it away as the mask comes from the vdso data and is not compile
+ * time constant.
+ */

It cannot optimise it because it does not know shift < 32. The code below is incorrect for shift equal to 32, fwiw.

Quoted 9 lines
arch/powerpc/include/asm/vdso/gettimeofday.h new file +9 -0
+static __always_inline u64 vdso_calc_delta(u64 cycles, u64 last, u64 mask, u32 mult)
+{
+	return (cycles - last) * mult;
+}
+#define vdso_calc_delta vdso_calc_delta
+
+#ifndef __powerpc64__
+static __always_inline u64 vdso_shift_ns(u64 ns, unsigned long shift)
+{

arch/powerpc/include/asm/vdso/gettimeofday.h line 161

Quoted 10 lines
arch/powerpc/include/asm/vdso/gettimeofday.h new file +8 -0
+	u32 hi = ns >> 32;
+	u32 lo = ns;
+
+	lo >>= shift;
+	lo |= hi << (32 - shift);
+	hi >>= shift;
 
 
+	if (likely(hi == 0))
+		return lo;

Removing these two lines shouldn't change generated object code? Or not make it worse, at least.

arch/powerpc/include/asm/vdso/gettimeofday.h line 164

Quoted 2 lines
arch/powerpc/include/asm/vdso/gettimeofday.h new file +2 -0
+	return ((u64)hi << 32) | lo;
+}

What does the compiler do for just

static __always_inline u64 vdso_shift_ns(u64 ns, unsigned long shift)

	return ns >> (shift & 31);
}

?

Segher

#4 lore raw

Re: [PATCH v10 2/5] powerpc/vdso: Prepare for switching VDSO to generic C implementation.

Hi,

Quoted 13 lines · Segher Boessenkool

Hi!

Quoted 9 lines · Christophe Leroy

+/*
+ * powerpc specific delta calculation.
+ *
+ * This variant removes the masking of the subtraction because the
+ * clocksource mask of all VDSO capable clocksources on powerpc is U64_MAX
+ * which would result in a pointless operation. The compiler cannot
+ * optimize it away as the mask comes from the vdso data and is not compile
+ * time constant.
+ */

Quoted 2 lines

It cannot optimise it because it does not know shift < 32. The code below is incorrect for shift equal to 32, fwiw.

Is there a way to tell it ?

Quoted 18 lines
Quoted 16 lines
arch/powerpc/include/asm/vdso/gettimeofday.h new file +15 -0
+static __always_inline u64 vdso_calc_delta(u64 cycles, u64 last, u64 mask, u32 mult)
+{
+	return (cycles - last) * mult;
+}
+#define vdso_calc_delta vdso_calc_delta
+
+#ifndef __powerpc64__
+static __always_inline u64 vdso_shift_ns(u64 ns, unsigned long shift)
+{
+	u32 hi = ns >> 32;
+	u32 lo = ns;
+
+	lo >>= shift;
+	lo |= hi << (32 - shift);
+	hi >>= shift;
Quoted 5 lines

arch/powerpc/include/asm/vdso/gettimeofday.h line 161

Quoted 2 lines
arch/powerpc/include/asm/vdso/gettimeofday.h new file +2 -0
+	if (likely(hi == 0))
+		return lo;

Removing these two lines shouldn't change generated object code? Or not make it worse, at least.

I remember it made noticeable difference allthough I can't remember the details. See below with GCC 10.1. At least we see that with those two lines, GCC only sets a 16 bytes stack frame. Without those lines it sets a 32 bytes stack frame and seems to save some values for no reason.

With the two lines:

000006ac <__c_kernel_clock_gettime>:

 6ac:	28 03 00 0f 	cmplwi  r3,15
 6b0:	41 81 01 04 	bgt     7b4 <__c_kernel_clock_gettime+0x108>
 6b4:	39 40 00 01 	li      r10,1
 6b8:	7d 4a 18 30 	slw     r10,r10,r3
 6bc:	71 47 08 83 	andi.   r7,r10,2179
 6c0:	41 82 01 2c 	beq     7ec <__c_kernel_clock_gettime+0x140>
 6c4:	94 21 ff f0 	stwu    r1,-16(r1)
 6c8:	54 63 20 36 	rlwinm  r3,r3,4,0,27
 6cc:	93 e1 00 0c 	stw     r31,12(r1)
 6d0:	7d 85 1a 14 	add     r12,r5,r3
 6d4:	80 05 00 00 	lwz     r0,0(r5)
 6d8:	70 06 00 01 	andi.   r6,r0,1
 6dc:	40 82 00 d4 	bne     7b0 <__c_kernel_clock_gettime+0x104>
 6e0:	7d 4d 42 e6 	mftbu   r10
 6e4:	7d 6c 42 e6 	mftb    r11
 6e8:	7c ed 42 e6 	mftbu   r7
 6ec:	7c 0a 38 40 	cmplw   r10,r7
 6f0:	40 82 ff f0 	bne     6e0 <__c_kernel_clock_gettime+0x34>
 6f4:	80 e5 00 0c 	lwz     r7,12(r5)
 6f8:	80 65 00 08 	lwz     r3,8(r5)
 6fc:	7c e7 58 10 	subfc   r7,r7,r11
 700:	81 65 00 18 	lwz     r11,24(r5)
 704:	7d 43 51 10 	subfe   r10,r3,r10
 708:	7f e7 58 16 	mulhwu  r31,r7,r11
 70c:	7d 4a 59 d6 	mullw   r10,r10,r11
 710:	7c e7 59 d6 	mullw   r7,r7,r11
 714:	80 6c 00 2c 	lwz     r3,44(r12)
 718:	81 6c 00 28 	lwz     r11,40(r12)
 71c:	7c e7 18 14 	addc    r7,r7,r3
 720:	7d 4a fa 14 	add     r10,r10,r31
 724:	80 65 00 1c 	lwz     r3,28(r5)
 728:	7d 4a 59 14 	adde    r10,r10,r11
 72c:	7c e7 1c 30 	srw     r7,r7,r3
 730:	21 63 00 20 	subfic  r11,r3,32
 734:	7d 43 1c 31 	srw.    r3,r10,r3
 738:	7d 4a 58 30 	slw     r10,r10,r11
 73c:	7d 49 3b 78 	or      r9,r10,r7
 740:	39 00 00 00 	li      r8,0
 744:	40 82 00 84 	bne     7c8 <__c_kernel_clock_gettime+0x11c>
 748:	80 6c 00 24 	lwz     r3,36(r12)
 74c:	81 45 00 00 	lwz     r10,0(r5)
 750:	7c 00 50 40 	cmplw   r0,r10
 754:	40 a2 ff 80 	bne     6d4 <__c_kernel_clock_gettime+0x28>
 758:	2c 08 00 00 	cmpwi   r8,0
 75c:	41 82 00 7c 	beq     7d8 <__c_kernel_clock_gettime+0x12c>
 760:	3c e0 c4 65 	lis     r7,-15259
 764:	3c 00 3b 9a 	lis     r0,15258
 768:	60 e7 36 00 	ori     r7,r7,13824
 76c:	60 00 c9 ff 	ori     r0,r0,51711
 770:	7c a9 38 14 	addc    r5,r9,r7
 774:	7d 48 01 d4 	addme   r10,r8
 778:	2c 0a 00 00 	cmpwi   r10,0
 77c:	7d 48 53 78 	mr      r8,r10
 780:	7c a9 2b 78 	mr      r9,r5
 784:	38 c6 00 01 	addi    r6,r6,1
 788:	40 82 ff e8 	bne     770 <__c_kernel_clock_gettime+0xc4>
 78c:	7c 05 00 40 	cmplw   r5,r0
 790:	41 81 ff e0 	bgt     770 <__c_kernel_clock_gettime+0xc4>
 794:	7c 66 18 14 	addc    r3,r6,r3
 798:	90 64 00 00 	stw     r3,0(r4)
 79c:	91 24 00 04 	stw     r9,4(r4)
 7a0:	38 60 00 00 	li      r3,0
 7a4:	83 e1 00 0c 	lwz     r31,12(r1)
 7a8:	38 21 00 10 	addi    r1,r1,16
 7ac:	4e 80 00 20 	blr
 7b0:	4b ff ff 24 	b       6d4 <__c_kernel_clock_gettime+0x28>
 7b4:	38 00 00 f6 	li      r0,246
 7b8:	44 00 00 02 	sc
 7bc:	40 a3 00 08 	bns     7c4 <__c_kernel_clock_gettime+0x118>
 7c0:	7c 63 00 d0 	neg     r3,r3
 7c4:	4e 80 00 20 	blr
 7c8:	7d 2a 4b 78 	mr      r10,r9
 7cc:	7c 68 1b 78 	mr      r8,r3
 7d0:	7d 49 53 78 	mr      r9,r10
 7d4:	4b ff ff 74 	b       748 <__c_kernel_clock_gettime+0x9c>
 7d8:	3d 40 3b 9a 	lis     r10,15258
 7dc:	61 4a c9 ff 	ori     r10,r10,51711
 7e0:	7c 09 50 40 	cmplw   r9,r10
 7e4:	41 81 ff 7c 	bgt     760 <__c_kernel_clock_gettime+0xb4>
 7e8:	4b ff ff b0 	b       798 <__c_kernel_clock_gettime+0xec>
 7ec:	71 47 00 60 	andi.   r7,r10,96
 7f0:	54 69 20 36 	rlwinm  r9,r3,4,0,27
 7f4:	7d 25 4a 14 	add     r9,r5,r9
 7f8:	40 82 00 14 	bne     80c <__c_kernel_clock_gettime+0x160>
 7fc:	71 4a 00 10 	andi.   r10,r10,16
 800:	41 a2 ff b4 	beq     7b4 <__c_kernel_clock_gettime+0x108>
 804:	38 a5 00 f0 	addi    r5,r5,240
 808:	4b ff fe bc 	b       6c4 <__c_kernel_clock_gettime+0x18>
 80c:	81 05 00 00 	lwz     r8,0(r5)
 810:	71 0a 00 01 	andi.   r10,r8,1
 814:	40 a2 ff f8 	bne     80c <__c_kernel_clock_gettime+0x160>
 818:	80 69 00 24 	lwz     r3,36(r9)
 81c:	81 49 00 2c 	lwz     r10,44(r9)
 820:	80 e5 00 00 	lwz     r7,0(r5)
 824:	7c 08 38 40 	cmplw   r8,r7
 828:	40 a2 ff e4 	bne     80c <__c_kernel_clock_gettime+0x160>
 82c:	90 64 00 00 	stw     r3,0(r4)
 830:	91 44 00 04 	stw     r10,4(r4)
 834:	38 60 00 00 	li      r3,0

838: 4e 80 00 20 blr

Without the two lines:

000006ac <__c_kernel_clock_gettime>:

 6ac:	28 03 00 0f 	cmplwi  r3,15
 6b0:	41 81 01 14 	bgt     7c4 <__c_kernel_clock_gettime+0x118>
 6b4:	39 20 00 01 	li      r9,1
 6b8:	7d 29 18 30 	slw     r9,r9,r3
 6bc:	71 2a 08 83 	andi.   r10,r9,2179
 6c0:	41 82 01 2c 	beq     7ec <__c_kernel_clock_gettime+0x140>
 6c4:	94 21 ff e0 	stwu    r1,-32(r1)
 6c8:	54 63 20 36 	rlwinm  r3,r3,4,0,27
 6cc:	93 81 00 10 	stw     r28,16(r1)
 6d0:	93 a1 00 14 	stw     r29,20(r1)
 6d4:	93 c1 00 18 	stw     r30,24(r1)
 6d8:	93 e1 00 1c 	stw     r31,28(r1)
 6dc:	7c 65 1a 14 	add     r3,r5,r3
 6e0:	81 85 00 00 	lwz     r12,0(r5)
 6e4:	71 87 00 01 	andi.   r7,r12,1
 6e8:	40 82 00 d8 	bne     7c0 <__c_kernel_clock_gettime+0x114>
 6ec:	7d 2d 42 e6 	mftbu   r9
 6f0:	7c cc 42 e6 	mftb    r6
 6f4:	7d 4d 42 e6 	mftbu   r10
 6f8:	7c 09 50 40 	cmplw   r9,r10
 6fc:	40 82 ff f0 	bne     6ec <__c_kernel_clock_gettime+0x40>
 700:	83 83 00 28 	lwz     r28,40(r3)
 704:	83 a3 00 2c 	lwz     r29,44(r3)
 708:	81 65 00 08 	lwz     r11,8(r5)
 70c:	81 05 00 0c 	lwz     r8,12(r5)
 710:	83 c5 00 18 	lwz     r30,24(r5)
 714:	83 e5 00 1c 	lwz     r31,28(r5)
 718:	80 03 00 24 	lwz     r0,36(r3)
 71c:	81 45 00 00 	lwz     r10,0(r5)
 720:	7c 0c 50 40 	cmplw   r12,r10
 724:	40 a2 ff bc 	bne     6e0 <__c_kernel_clock_gettime+0x34>
 728:	7d 48 30 10 	subfc   r10,r8,r6
 72c:	7c cb 49 10 	subfe   r6,r11,r9
 730:	7c c6 f1 d6 	mullw   r6,r6,r30
 734:	7d 2a f0 16 	mulhwu  r9,r10,r30
 738:	7d 4a f1 d6 	mullw   r10,r10,r30
 73c:	7c c6 4a 14 	add     r6,r6,r9
 740:	7d 4a e8 14 	addc    r10,r10,r29
 744:	7c c6 e1 14 	adde    r6,r6,r28
 748:	7c c8 fc 30 	srw     r8,r6,r31
 74c:	2c 08 00 00 	cmpwi   r8,0
 750:	20 bf 00 20 	subfic  r5,r31,32
 754:	7d 4a fc 30 	srw     r10,r10,r31
 758:	7c c5 28 30 	slw     r5,r6,r5
 75c:	7c a9 53 78 	or      r9,r5,r10
 760:	41 82 00 78 	beq     7d8 <__c_kernel_clock_gettime+0x12c>
 764:	3c c0 c4 65 	lis     r6,-15259
 768:	3c 60 3b 9a 	lis     r3,15258
 76c:	60 c6 36 00 	ori     r6,r6,13824
 770:	60 63 c9 ff 	ori     r3,r3,51711
 774:	7c a9 30 14 	addc    r5,r9,r6
 778:	7d 48 01 d4 	addme   r10,r8
 77c:	2c 0a 00 00 	cmpwi   r10,0
 780:	7d 48 53 78 	mr      r8,r10
 784:	7c a9 2b 78 	mr      r9,r5
 788:	38 e7 00 01 	addi    r7,r7,1
 78c:	40 82 ff e8 	bne     774 <__c_kernel_clock_gettime+0xc8>
 790:	7c 05 18 40 	cmplw   r5,r3
 794:	41 81 ff e0 	bgt     774 <__c_kernel_clock_gettime+0xc8>
 798:	7c 07 00 14 	addc    r0,r7,r0
 79c:	90 04 00 00 	stw     r0,0(r4)
 7a0:	91 24 00 04 	stw     r9,4(r4)
 7a4:	38 60 00 00 	li      r3,0
 7a8:	83 81 00 10 	lwz     r28,16(r1)
 7ac:	83 a1 00 14 	lwz     r29,20(r1)
 7b0:	83 c1 00 18 	lwz     r30,24(r1)
 7b4:	83 e1 00 1c 	lwz     r31,28(r1)
 7b8:	38 21 00 20 	addi    r1,r1,32
 7bc:	4e 80 00 20 	blr
 7c0:	4b ff ff 20 	b       6e0 <__c_kernel_clock_gettime+0x34>
 7c4:	38 00 00 f6 	li      r0,246
 7c8:	44 00 00 02 	sc
 7cc:	40 a3 00 08 	bns     7d4 <__c_kernel_clock_gettime+0x128>
 7d0:	7c 63 00 d0 	neg     r3,r3
 7d4:	4e 80 00 20 	blr
 7d8:	3d 40 3b 9a 	lis     r10,15258
 7dc:	61 4a c9 ff 	ori     r10,r10,51711
 7e0:	7c 09 50 40 	cmplw   r9,r10
 7e4:	41 81 ff 80 	bgt     764 <__c_kernel_clock_gettime+0xb8>
 7e8:	4b ff ff b4 	b       79c <__c_kernel_clock_gettime+0xf0>
 7ec:	71 2a 00 60 	andi.   r10,r9,96
 7f0:	40 82 00 14 	bne     804 <__c_kernel_clock_gettime+0x158>
 7f4:	71 29 00 10 	andi.   r9,r9,16
 7f8:	41 a2 ff cc 	beq     7c4 <__c_kernel_clock_gettime+0x118>
 7fc:	38 a5 00 f0 	addi    r5,r5,240
 800:	4b ff fe c4 	b       6c4 <__c_kernel_clock_gettime+0x18>
 804:	54 69 20 36 	rlwinm  r9,r3,4,0,27
 808:	7d 25 4a 14 	add     r9,r5,r9
 80c:	81 05 00 00 	lwz     r8,0(r5)
 810:	71 0a 00 01 	andi.   r10,r8,1
 814:	40 82 00 28 	bne     83c <__c_kernel_clock_gettime+0x190>
 818:	80 09 00 24 	lwz     r0,36(r9)
 81c:	81 49 00 2c 	lwz     r10,44(r9)
 820:	80 e5 00 00 	lwz     r7,0(r5)
 824:	7c 08 38 40 	cmplw   r8,r7
 828:	40 a2 ff e4 	bne     80c <__c_kernel_clock_gettime+0x160>
 82c:	90 04 00 00 	stw     r0,0(r4)
 830:	91 44 00 04 	stw     r10,4(r4)
 834:	38 60 00 00 	li      r3,0

838: 4e 80 00 20 blr
83c: 4b ff ff d0 b 80c <__c_kernel_clock_gettime+0x160>

Quoted 11 lines

arch/powerpc/include/asm/vdso/gettimeofday.h line 164

Quoted 3 lines
arch/powerpc/include/asm/vdso/gettimeofday.h new file +2 -0
+	return ((u64)hi << 32) | lo;
+}

What does the compiler do for just

static __always_inline u64 vdso_shift_ns(u64 ns, unsigned long shift)

	return ns >> (shift & 31);
}

Worse:

000006ac <__c_kernel_clock_gettime>:

 6ac:	28 03 00 0f 	cmplwi  r3,15
 6b0:	41 81 01 30 	bgt     7e0 <__c_kernel_clock_gettime+0x134>
 6b4:	39 20 00 01 	li      r9,1
 6b8:	7d 29 18 30 	slw     r9,r9,r3
 6bc:	71 2a 08 83 	andi.   r10,r9,2179
 6c0:	41 82 01 48 	beq     808 <__c_kernel_clock_gettime+0x15c>
 6c4:	94 21 ff e0 	stwu    r1,-32(r1)
 6c8:	54 63 20 36 	rlwinm  r3,r3,4,0,27
 6cc:	93 81 00 10 	stw     r28,16(r1)
 6d0:	93 a1 00 14 	stw     r29,20(r1)
 6d4:	93 c1 00 18 	stw     r30,24(r1)
 6d8:	93 e1 00 1c 	stw     r31,28(r1)
 6dc:	7c 65 1a 14 	add     r3,r5,r3
 6e0:	80 c5 00 00 	lwz     r6,0(r5)
 6e4:	70 c7 00 01 	andi.   r7,r6,1
 6e8:	40 82 00 f4 	bne     7dc <__c_kernel_clock_gettime+0x130>
 6ec:	7d 2d 42 e6 	mftbu   r9
 6f0:	7d 0c 42 e6 	mftb    r8
 6f4:	7d 4d 42 e6 	mftbu   r10
 6f8:	7c 09 50 40 	cmplw   r9,r10
 6fc:	40 82 ff f0 	bne     6ec <__c_kernel_clock_gettime+0x40>
 700:	83 83 00 28 	lwz     r28,40(r3)
 704:	83 c3 00 2c 	lwz     r30,44(r3)
 708:	81 65 00 08 	lwz     r11,8(r5)
 70c:	81 45 00 0c 	lwz     r10,12(r5)
 710:	83 e5 00 18 	lwz     r31,24(r5)
 714:	81 85 00 1c 	lwz     r12,28(r5)
 718:	80 03 00 24 	lwz     r0,36(r3)
 71c:	83 a5 00 00 	lwz     r29,0(r5)
 720:	7c 06 e8 40 	cmplw   r6,r29
 724:	40 a2 ff bc 	bne     6e0 <__c_kernel_clock_gettime+0x34>
 728:	7d 0a 40 10 	subfc   r8,r10,r8
 72c:	7c cb 49 10 	subfe   r6,r11,r9
 730:	7c c6 f9 d6 	mullw   r6,r6,r31
 734:	7d 28 f8 16 	mulhwu  r9,r8,r31
 738:	7d 08 f9 d6 	mullw   r8,r8,r31
 73c:	55 8c 06 fe 	clrlwi  r12,r12,27
 740:	7f c8 f0 14 	addc    r30,r8,r30
 744:	7c c6 4a 14 	add     r6,r6,r9
 748:	7c c6 e1 14 	adde    r6,r6,r28
 74c:	34 6c ff e0 	addic.  r3,r12,-32
 750:	41 80 00 70 	blt     7c0 <__c_kernel_clock_gettime+0x114>
 754:	7c c9 1c 30 	srw     r9,r6,r3
 758:	39 00 00 00 	li      r8,0
 75c:	2c 08 00 00 	cmpwi   r8,0
 760:	41 82 00 94 	beq     7f4 <__c_kernel_clock_gettime+0x148>
 764:	3c c0 c4 65 	lis     r6,-15259
 768:	3c 60 3b 9a 	lis     r3,15258
 76c:	60 c6 36 00 	ori     r6,r6,13824
 770:	60 63 c9 ff 	ori     r3,r3,51711
 774:	7c a9 30 14 	addc    r5,r9,r6
 778:	7d 48 01 d4 	addme   r10,r8
 77c:	2c 0a 00 00 	cmpwi   r10,0
 780:	7d 48 53 78 	mr      r8,r10
 784:	7c a9 2b 78 	mr      r9,r5
 788:	38 e7 00 01 	addi    r7,r7,1
 78c:	40 82 ff e8 	bne     774 <__c_kernel_clock_gettime+0xc8>
 790:	7c 05 18 40 	cmplw   r5,r3
 794:	41 81 ff e0 	bgt     774 <__c_kernel_clock_gettime+0xc8>
 798:	7c 07 00 14 	addc    r0,r7,r0
 79c:	90 04 00 00 	stw     r0,0(r4)
 7a0:	91 24 00 04 	stw     r9,4(r4)
 7a4:	38 60 00 00 	li      r3,0
 7a8:	83 81 00 10 	lwz     r28,16(r1)
 7ac:	83 a1 00 14 	lwz     r29,20(r1)
 7b0:	83 c1 00 18 	lwz     r30,24(r1)
 7b4:	83 e1 00 1c 	lwz     r31,28(r1)
 7b8:	38 21 00 20 	addi    r1,r1,32
 7bc:	4e 80 00 20 	blr
 7c0:	54 c3 08 3c 	rlwinm  r3,r6,1,0,30
 7c4:	21 6c 00 1f 	subfic  r11,r12,31
 7c8:	7c 63 58 30 	slw     r3,r3,r11
 7cc:	7f c9 64 30 	srw     r9,r30,r12
 7d0:	7c 69 4b 78 	or      r9,r3,r9
 7d4:	7c c8 64 30 	srw     r8,r6,r12
 7d8:	4b ff ff 84 	b       75c <__c_kernel_clock_gettime+0xb0>
 7dc:	4b ff ff 04 	b       6e0 <__c_kernel_clock_gettime+0x34>
 7e0:	38 00 00 f6 	li      r0,246
 7e4:	44 00 00 02 	sc
 7e8:	40 a3 00 08 	bns     7f0 <__c_kernel_clock_gettime+0x144>
 7ec:	7c 63 00 d0 	neg     r3,r3
 7f0:	4e 80 00 20 	blr
 7f4:	3d 40 3b 9a 	lis     r10,15258
 7f8:	61 4a c9 ff 	ori     r10,r10,51711
 7fc:	7c 09 50 40 	cmplw   r9,r10
 800:	41 81 ff 64 	bgt     764 <__c_kernel_clock_gettime+0xb8>
 804:	4b ff ff 98 	b       79c <__c_kernel_clock_gettime+0xf0>
 808:	71 2a 00 60 	andi.   r10,r9,96
 80c:	40 82 00 14 	bne     820 <__c_kernel_clock_gettime+0x174>
 810:	71 29 00 10 	andi.   r9,r9,16
 814:	41 a2 ff cc 	beq     7e0 <__c_kernel_clock_gettime+0x134>
 818:	38 a5 00 f0 	addi    r5,r5,240
 81c:	4b ff fe a8 	b       6c4 <__c_kernel_clock_gettime+0x18>
 820:	54 69 20 36 	rlwinm  r9,r3,4,0,27
 824:	7d 25 4a 14 	add     r9,r5,r9
 828:	81 05 00 00 	lwz     r8,0(r5)
 82c:	71 0a 00 01 	andi.   r10,r8,1
 830:	40 82 00 28 	bne     858 <__c_kernel_clock_gettime+0x1ac>
 834:	80 09 00 24 	lwz     r0,36(r9)
 838:	81 49 00 2c 	lwz     r10,44(r9)
 83c:	80 e5 00 00 	lwz     r7,0(r5)
 840:	7c 08 38 40 	cmplw   r8,r7
 844:	40 a2 ff e4 	bne     828 <__c_kernel_clock_gettime+0x17c>
 848:	90 04 00 00 	stw     r0,0(r4)
 84c:	91 44 00 04 	stw     r10,4(r4)
 850:	38 60 00 00 	li      r3,0

854: 4e 80 00 20 blr
858: 4b ff ff d0 b 828 <__c_kernel_clock_gettime+0x17c>

Christophe

#7 lore raw

Re: [PATCH v10 2/5] powerpc/vdso: Prepare for switching VDSO to generic C implementation.

In reply to Segher Boessenkool

Hi Again,

Le 05/08/2020 à 16:03, Segher Boessenkool a écrit :

Quoted 11 lines

Hi!

Quoted 5 lines · Christophe Leroy

+/*
+ * The macros sets two stack frames, one for the caller and one for the callee
+ * because there are no requirement for the caller to set a stack frame when
+ * calling VDSO so it may have omitted to set one, especially on PPC64
+ */

If the caller follows the ABI, there always is a stack frame. So what is going on?

Looks like it is not the case. See discussion at https://patchwork.ozlabs.org/project/linuxppc-dev/patch/2a67c333893454868bbfda773ba4b01c20272a5d.1588079622.git.christophe.leroy@c-s.fr/

Seems like GCC uses the redzone and doesn't set a stack frame. I guess it doesn't know that the inline assembly contains a function call so it doesn't set the frame.

Christophe

#5 lore raw

Re: [PATCH v10 2/5] powerpc/vdso: Prepare for switching VDSO to generic C implementation.

In reply to Christophe Leroy

Hi!

Quoted 4 lines · Christophe Leroy
Quoted 2 lines

It cannot optimise it because it does not know shift < 32. The code below is incorrect for shift equal to 32, fwiw.

Is there a way to tell it ?

Sure, for example the &31 should work (but it doesn't, with the GCC version you used -- which version is that?)

Quoted 8 lines
Quoted 6 lines

What does the compiler do for just

static __always_inline u64 vdso_shift_ns(u64 ns, unsigned long shift)

	return ns >> (shift & 31);
}

Worse:

I cannot make heads or tails of all that branch spaghetti, sorry.

Quoted 6 lines
 73c:	55 8c 06 fe 	clrlwi  r12,r12,27
 740:	7f c8 f0 14 	addc    r30,r8,r30
 744:	7c c6 4a 14 	add     r6,r6,r9
 748:	7c c6 e1 14 	adde    r6,r6,r28
 74c:	34 6c ff e0 	addic.  r3,r12,-32
 750:	41 80 00 70 	blt     7c0 <__c_kernel_clock_gettime+0x114>

This branch is always true. Hrm.

Segher

#8 lore raw

Re: [PATCH v10 2/5] powerpc/vdso: Prepare for switching VDSO to generic C implementation.

In reply to Christophe Leroy

Hi!

Quoted 13 lines · Christophe Leroy

Le 05/08/2020 à 16:03, Segher Boessenkool a écrit :

Quoted 11 lines · Christophe Leroy
Quoted 7 lines

+/*
+ * The macros sets two stack frames, one for the caller and one for the callee
+ * because there are no requirement for the caller to set a stack frame when
+ * calling VDSO so it may have omitted to set one, especially on PPC64
+ */

If the caller follows the ABI, there always is a stack frame. So what is going on?

Quoted 6 lines

Looks like it is not the case. See discussion at
https://patchwork.ozlabs.org/project/linuxppc-dev/patch/2a67c333893454868bbfda773ba4b01c20272a5d.1588079622.git.christophe.leroy@c-s.fr/

Seems like GCC uses the redzone and doesn't set a stack frame. I guess it doesn't know that the inline assembly contains a function call so it doesn't set the frame.

Yes, that is the problem. See
https://gcc.gnu.org/onlinedocs/gcc-10.2.0/gcc/Extended-Asm.html#AssemblerTemplate
where this is (briefly) discussed:
"Accessing data from C programs without using input/output operands (such as by using global symbols directly from the assembler template) may not work as expected. Similarly, calling functions directly from an assembler template requires a detailed understanding of the target assembler and ABI."

I don't know of a good way to tell GCC some function needs a frame (that is, one that doesn't result in extra code other than to set up the frame). I'll think about it.

Segher

#6 lore raw

Re: [PATCH v10 2/5] powerpc/vdso: Prepare for switching VDSO to generic C implementation.

In reply to Segher Boessenkool

Hi,

Quoted 10 lines · Segher Boessenkool

Hi!

Quoted 4 lines · Christophe Leroy
Quoted 2 lines

It cannot optimise it because it does not know shift < 32. The code below is incorrect for shift equal to 32, fwiw.

Is there a way to tell it ?

Sure, for example the &31 should work (but it doesn't, with the GCC version you used -- which version is that?)

GCC 10.1

Quoted 12 lines
Quoted 9 lines
Quoted 7 lines

What does the compiler do for just

static __always_inline u64 vdso_shift_ns(u64 ns, unsigned long shift)

	return ns >> (shift & 31);
}

Worse:

I cannot make heads or tails of all that branch spaghetti, sorry.

Quoted 8 lines
Quoted 6 lines
  73c:	55 8c 06 fe 	clrlwi  r12,r12,27
  740:	7f c8 f0 14 	addc    r30,r8,r30
  744:	7c c6 4a 14 	add     r6,r6,r9
  748:	7c c6 e1 14 	adde    r6,r6,r28
  74c:	34 6c ff e0 	addic.  r3,r12,-32
  750:	41 80 00 70 	blt     7c0 <__c_kernel_clock_gettime+0x114>

This branch is always true. Hrm.

As a standalone function:

With your suggestion:

000006ac <vdso_shift_ns>:

 6ac:	54 a5 06 fe 	clrlwi  r5,r5,27
 6b0:	35 25 ff e0 	addic.  r9,r5,-32
 6b4:	41 80 00 10 	blt     6c4 <vdso_shift_ns+0x18>
 6b8:	7c 64 4c 30 	srw     r4,r3,r9
 6bc:	38 60 00 00 	li      r3,0
 6c0:	4e 80 00 20 	blr
 6c4:	54 69 08 3c 	rlwinm  r9,r3,1,0,30
 6c8:	21 45 00 1f 	subfic  r10,r5,31
 6cc:	7c 84 2c 30 	srw     r4,r4,r5
 6d0:	7d 29 50 30 	slw     r9,r9,r10
 6d4:	7c 63 2c 30 	srw     r3,r3,r5
 6d8:	7d 24 23 78 	or      r4,r9,r4

6dc: 4e 80 00 20 blr

With the version as is in my series:

000006ac <vdso_shift_ns>:

 6ac:	21 25 00 20 	subfic  r9,r5,32
 6b0:	7c 69 48 30 	slw     r9,r3,r9
 6b4:	7c 84 2c 30 	srw     r4,r4,r5
 6b8:	7d 24 23 78 	or      r4,r9,r4
 6bc:	7c 63 2c 30 	srw     r3,r3,r5

6c0: 4e 80 00 20 blr

Christophe