Skip to contentSkip to conversation

kprobe'ing conditionally executed instructions

RFC David Long 5 participants 9 messages Dec 11, 2015 – Dec 12, 2015

lore
Share & tools
lore

David Long

author
#0 lore raw

There is a moderate amount of code already in kprobes on ARM and the current ARMv8 patch to deal with conditional execution of instructions. One aspect of how this is handled is that instructions that fail their predicate and are not (technically) executed are also not treated as a hit kprobe. Steve Capper has suggested that the probe handling should still take place because we stepped through the instruction even if it was effectively a nop. This would be a significant change in how it currently works on 32-bit ARM, and a change in the patch for ARMv8 (although it's not likely to be much of a change in the kernel code).

I need input on this. Do people have opinions?

-dl

#1 lore raw
Quoted 11 lines · David Long

There is a moderate amount of code already in kprobes on ARM and the current ARMv8 patch to deal with conditional execution of instructions. One aspect of how this is handled is that instructions that fail their predicate and are not (technically) executed are also not treated as a hit kprobe. Steve Capper has suggested that the probe handling should still take place because we stepped through the instruction even if it was effectively a nop. This would be a significant change in how it currently works on 32-bit ARM, and a change in the patch for ARMv8 (although it's not likely to be much of a change in the kernel code).

I need input on this. Do people have opinions?

Hi David,
Thanks for posting this.

Just to clarify the reasoning behind my suggestion for kprobes always being hit was to achieve parity with x86.

I highlighted an example of discrepancy in behaviour between arm64 and x86 in the following email:
http://lists.infradead.org/pipermail/linux-arm-kernel/2015-August/364201.html

Cheers,
--
Steve

#2 lore raw
In reply to David Long
Quoted 8 lines · David Long

There is a moderate amount of code already in kprobes on ARM and the current ARMv8 patch to deal with conditional execution of instructions. One aspect of how this is handled is that instructions that fail their predicate and are not (technically) executed are also not treated as a hit kprobe. Steve Capper has suggested that the probe handling should still take place because we stepped through the instruction even if it was effectively a nop. This would be a significant change in how it currently works on 32-bit ARM

32-bit ARM uses undefined instructions for kprobe 'breakpoints' and the ARM ARM says it's implementation defined behaviour whether these generate exceptions or not, i.e. whether the kprobe handler will be called. You could say that we could always use unconditional breakpoints, but this doesn't work with thumb where the instruction could be in an IT block. So, the only way to have consistent behaviour on all platforms is to not call kprobe handlers if condition check fails. Which is the reason for the current implementation's design.

Also, if we change the current implementation as suggested, then looking at things from a source code point of view...

if (test)

	foo()
else
	bar();

If you put a probe on the call to foo() and the compiler uses a branch instruction for the test you're never going to hit the probe fortest==false. But if it decides to use conditional instructions it will (on some CPU implementations). And the choice between branch/conditional instructions probably varies between GCC version and kernel configs.

So again, IMO, the current kprobes implementation leads to consistency.

Signature · 1 line
Tixy 
#3 lore raw
Quoted 10 lines · Jon Medhurst (Tixy)
Quoted 8 lines · David Long

There is a moderate amount of code already in kprobes on ARM and the current ARMv8 patch to deal with conditional execution of instructions. One aspect of how this is handled is that instructions that fail their predicate and are not (technically) executed are also not treated as a hit kprobe. Steve Capper has suggested that the probe handling should still take place because we stepped through the instruction even if it was effectively a nop. This would be a significant change in how it currently works on 32-bit ARM

Quoted 4 lines

32-bit ARM uses undefined instructions for kprobe 'breakpoints' and the ARM ARM says it's implementation defined behaviour whether these generate exceptions or not, i.e. whether the kprobe handler will be called.

There are two classes of undefined instructions. There are those which fall into the above category, and there are those which are guaranteed to raise an exception. We should always be using the guaranteed ones, not the other set.

Signature · 3 lines
RMK's Patch system: http://www.arm.linux.org.uk/developer/patches/
FTTC broadband for 0.8mile line: currently at 9.6Mbps down 400kbps up
according to speedtest.net.
#4 lore raw
Quoted 16 lines · Russell King - ARM Linux
Quoted 10 lines · Jon Medhurst (Tixy)
Quoted 8 lines · David Long

There is a moderate amount of code already in kprobes on ARM and the current ARMv8 patch to deal with conditional execution of instructions. One aspect of how this is handled is that instructions that fail their predicate and are not (technically) executed are also not treated as a hit kprobe. Steve Capper has suggested that the probe handling should still take place because we stepped through the instruction even if it was effectively a nop. This would be a significant change in how it currently works on 32-bit ARM

Quoted 4 lines

32-bit ARM uses undefined instructions for kprobe 'breakpoints' and the ARM ARM says it's implementation defined behaviour whether these generate exceptions or not, i.e. whether the kprobe handler will be called.

Quoted 4 lines

There are two classes of undefined instructions. There are those which fall into the above category, and there are those which are guaranteed to raise an exception. We should always be using the guaranteed ones, not the other set.

I wonder if I'm going senile or have been subject to having the ARM ARM evolve under me. I could swear we used instructions that were defined as undefined (so to speak) and that conditional versions of undefined instructions were UNPREDICTABLE. However, checking the ARM ARM again I see those instruction encodings are for the BKPT instruction which says:

Breakpoint causes a software breakpoint to occur. Breakpoint is always unconditional, even when inside an IT block.

So, my previous statements about not being able to implement the proposed kprobe changes consistently aren't true.

Signature · 1 line
Tixy
#5 lore raw
In reply to David Long
Quoted 6 lines · David Long

There is a moderate amount of code already in kprobes on ARM and the current ARMv8 patch to deal with conditional execution of instructions. One aspect of how this is handled is that instructions that fail their predicate and are not (technically) executed are also not treated as a hit kprobe. Steve Capper has suggested that the probe handling should still take place because we stepped through the instruction even if it was effectively a nop. This would be a significant change in how it currently works on 32-bit ARM, and a change in the patch for ARMv8 (although it's not likely to be much of a change in the kernel code).

I need input on this. Do people have opinions?

-dl

Hi Dave,

Conditionally executing the kprobes would violate the assumptions made for perf and systemtap collecting data. Even if the instruction is predicated and treated as a NOP it should still reliably trigger the kprobe. However, for efficiency the simulation/emulation/single-step of the instruction could be skipped if the instruction is known to have no change on the machine state other than changing the program counter.

-Will Cohen

author
#6 lore raw
Quoted 5 lines · William Cohen
Quoted 4 lines · David Long

There is a moderate amount of code already in kprobes on ARM and the current ARMv8 patch to deal with conditional execution of instructions. One aspect of how this is handled is that instructions that fail their predicate and are not (technically) executed are also not treated as a hit kprobe. Steve Capper has suggested that the probe handling should still take place because we stepped through the instruction even if it was effectively a nop. This would be a significant change in how it currently works on 32-bit ARM, and a change in the patch for ARMv8 (although it's not likely to be much of a change in the kernel code).

I need input on this. Do people have opinions?

Quoted 9 lines
Quoted 2 lines

-dl

Hi Dave,

Conditionally executing the kprobes would violate the assumptions made for perf and systemtap collecting data. Even if the instruction is predicated and treated as a NOP it should still reliably trigger the kprobe. However, for efficiency the simulation/emulation/single-step of the instruction could be skipped if the instruction is known to have no change on the machine state other than changing the program counter.

-Will Cohen

I wonder if this might explain some systemtap failures.

-dl

#7 lore raw
Quoted 11 lines · David Long
Quoted 10 lines · William Cohen
Quoted 6 lines · David Long

There is a moderate amount of code already in kprobes on ARM and the current ARMv8 patch to deal with conditional execution of instructions. One aspect of how this is handled is that instructions that fail their predicate and are not (technically) executed are also not treated as a hit kprobe. Steve Capper has suggested that the probe handling should still take place because we stepped through the instruction even if it was effectively a nop. This would be a significant change in how it currently works on 32-bit ARM, and a change in the patch for ARMv8 (although it's not likely to be much of a change in the kernel code).

I need input on this. Do people have opinions?

-dl

Hi Dave,

Quoted 9 lines
Quoted 4 lines

Conditionally executing the kprobes would violate the assumptions made for perf and systemtap collecting data. Even if the instruction is predicated and treated as a NOP it should still reliably trigger the kprobe. However, for efficiency the simulation/emulation/single-step of the instruction could be skipped if the instruction is known to have no change on the machine state other than changing the program counter.

-Will Cohen

I wonder if this might explain some systemtap failures.

-dl

Hi Dave,

It is possible that some of the kprobes not triggering would cause testsuite failures, but I don't have a specific test to point to this is happening.

How much of change would it be to fix the kprobes to get the correct behavior. It might be easiest to fix it and compare the test results to find possible candidates where this problem occurs.

-Will

author
#8 lore raw
Quoted 22 lines · William Cohen
Quoted 11 lines · David Long
Quoted 10 lines · William Cohen
Quoted 6 lines · David Long

There is a moderate amount of code already in kprobes on ARM and the current ARMv8 patch to deal with conditional execution of instructions. One aspect of how this is handled is that instructions that fail their predicate and are not (technically) executed are also not treated as a hit kprobe. Steve Capper has suggested that the probe handling should still take place because we stepped through the instruction even if it was effectively a nop. This would be a significant change in how it currently works on 32-bit ARM, and a change in the patch for ARMv8 (although it's not likely to be much of a change in the kernel code).

I need input on this. Do people have opinions?

-dl

Hi Dave,

Quoted 9 lines
Quoted 4 lines

Conditionally executing the kprobes would violate the assumptions made for perf and systemtap collecting data. Even if the instruction is predicated and treated as a NOP it should still reliably trigger the kprobe. However, for efficiency the simulation/emulation/single-step of the instruction could be skipped if the instruction is known to have no change on the machine state other than changing the program counter.

-Will Cohen

I wonder if this might explain some systemtap failures.

-dl

Quoted 8 lines

Hi Dave,

It is possible that some of the kprobes not triggering would cause testsuite failures, but I don't have a specific test to point to this is happening.

How much of change would it be to fix the kprobes to get the correct behavior. It might be easiest to fix it and compare the test results to find possible candidates where this problem occurs.

-Will

I think it's a very simple change. I will come up with something for you to test shortly. It sounds like we may need the change permanently anyway, although clearly there is not a solid consensus yet.

I would think this would impact 32-bit behavior much more than 64-bit. I can see about producing something for testing that too.

-dl