Skip to contentSkip to conversation

arch/x86/entry/vsyscall/vsyscall_gtod.c: remove __read_mostly from vclocks_used

PATCH In review Michael Rodin 3 participants 4 messages Jun 4, 2018 – Jun 23, 2018

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 arch/x86/entry/vsyscall/vsyscall_gtod.c: remove __read_mostly from vclocks_used

The variable "vclocks_used" doesn't appear to be "read mostly". Measurements of the access frequency with perf stat [1] and perf report show, that approximately half of the accesses to this variable are write accesses and happen in update_vsyscall() in the file arch/x86/entry/vsyscall/vsyscall_gtod.c.
The measurements were done with the kernel 4.13.0-43-generic used by ubuntu as well as with the stable kernel 4.16.7 with a custom config. I've used "perf bench sched" and iperf3 as workloads.

This was discovered during my master thesis in the CADOS project [2].

[1] perf stat --all-cpus --group --event="{mem:addr/4:rw:k,mem:addr/4:w:k}"
[2] https://www.sra.uni-hannover.de/Research/CADOS/

---
 arch/x86/entry/vsyscall/vsyscall_gtod.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
arch/x86/entry/vsyscall/vsyscall_gtod.c +1 -1
index e1216dd95c04..fc39f701fe4a 100644
@@ -17,7 +17,7 @@
 #include <asm/vgtod.h>
 #include <asm/vvar.h>
 
-int vclocks_used __read_mostly;
+int vclocks_used;
 
 DEFINE_VVAR(struct vsyscall_gtod_data, vsyscall_gtod_data);
 
Signature · 1 line
2.14.1
#1 lore raw

Michael,

Quoted 10 lines · Michael Rodin

The variable "vclocks_used" doesn't appear to be "read mostly". Measurements of the access frequency with perf stat [1] and perf report show, that approximately half of the accesses to this variable are write accesses and happen in update_vsyscall() in the file arch/x86/entry/vsyscall/vsyscall_gtod.c.
The measurements were done with the kernel 4.13.0-43-generic used by ubuntu as well as with the stable kernel 4.16.7 with a custom config. I've used "perf bench sched" and iperf3 as workloads.

This was discovered during my master thesis in the CADOS project [2].

Nice find, but ...

The point is that the content of that variable changes once in a blue moon, so the intent of marking it read_mostly is almost correct. Almost, because the unconditional write even with the ever repeating same value is not free. To the best of my knowledge there is no CPU implementation which supports optimizing silent stores, but I might be just not up to date or wrong as usual.

So there is a cost to the write and there are two things which have to be looked at before just doing the obvious knee jerk reaction 's/read_mostly//'.

1) Does it make sense to avoid the write if the value hasn't changed? That might be pretty much a free lunch because the variable has to be read in the first place in order to OR the supplied value to it before writing it back.

2) The variable is in a totally separate cache line than what the following stores target, i.e. vsyscall_gtod_data

So it might make sense just to do #1 but as vsyscall_gtod_data is anyway going to be dirtied it probably makes even more sense to move that vclocks_used variable into struct vsyscall_gtod_data right away to reduce the amount of cache lines touched by update_vsyscall().

Care to have a look at that?

Thanks,

tglx

maintainer
#2 lore raw
Quoted 14 lines · Thomas Gleixner

Michael,

Quoted 9 lines · Michael Rodin

The variable "vclocks_used" doesn't appear to be "read mostly". Measurements of the access frequency with perf stat [1] and perf report show, that approximately half of the accesses to this variable are write accesses and happen in update_vsyscall() in the file arch/x86/entry/vsyscall/vsyscall_gtod.c.
The measurements were done with the kernel 4.13.0-43-generic used by ubuntu as well as with the stable kernel 4.16.7 with a custom config. I've used "perf bench sched" and iperf3 as workloads.

Quoted 6 lines
Quoted 1 line

This was discovered during my master thesis in the CADOS project [2].

Nice find, but ...

The point is that the content of that variable changes once in a blue moon, so the intent of marking it read_mostly is almost correct.

I would propose a rather different fix. Add a an
arch_change_clocksource() function. Do:

static inline void arch_change_clocksource(struct clocksource *new_clocksource) { ... }
#define arch_change_clocksource arch_change_clocksource

and

#ifndef arch_change_clocksource
static inline void arch_change_clocksource(struct clocksource *new_clocksource) {}
#endif

in the generic header. In change_clocksource(), add a call to arch_change_clocksource() right after tk_setup_internals(). In x86's arch_change_clocksource, update vclocks_used.

Now it's genuinely read_mostly, and we don't need to touch that cacheline at all in the normal clock tick code. Everyone wins. (vclocks_used is actually rather rarely read. It's only used to prevent user code from accessing a never-used clocksource through the vvar area, which is a hardening measure. It's only referenced from the vvar fault handler code.)

--Andy

#3 lore raw
Quoted 37 lines · Andy Lutomirski
Quoted 11 lines · Thomas Gleixner
Quoted 9 lines · Michael Rodin

The variable "vclocks_used" doesn't appear to be "read mostly". Measurements of the access frequency with perf stat [1] and perf report show, that approximately half of the accesses to this variable are write accesses and happen in update_vsyscall() in the file arch/x86/entry/vsyscall/vsyscall_gtod.c.
The measurements were done with the kernel 4.13.0-43-generic used by ubuntu as well as with the stable kernel 4.16.7 with a custom config. I've used "perf bench sched" and iperf3 as workloads.

Quoted 6 lines
Quoted 1 line

This was discovered during my master thesis in the CADOS project [2].

Nice find, but ...

The point is that the content of that variable changes once in a blue moon, so the intent of marking it read_mostly is almost correct.

I would propose a rather different fix. Add a an
arch_change_clocksource() function. Do:

static inline void arch_change_clocksource(struct clocksource *new_clocksource) { ... }
#define arch_change_clocksource arch_change_clocksource

and

#ifndef arch_change_clocksource
static inline void arch_change_clocksource(struct clocksource *new_clocksource) {}
#endif

in the generic header. In change_clocksource(), add a call to arch_change_clocksource() right after tk_setup_internals(). In x86's arch_change_clocksource, update vclocks_used.

Quoted 6 lines

Now it's genuinely read_mostly, and we don't need to touch that cacheline at all in the normal clock tick code. Everyone wins. (vclocks_used is actually rather rarely read. It's only used to prevent user code from accessing a never-used clocksource through the vvar area, which is a hardening measure. It's only referenced from the vvar fault handler code.)

Agreed.

Thanks,

tglx