Skip to contentSkip to conversation

i386 & x86_64: Live Patching Funcion on 2.6.11.7

PATCH In review Takashi Ikebe 4 participants 8 messages Apr 17, 2005 – Apr 23, 2005

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 i386 & x86_64: Live Patching Funcion on 2.6.11.7

Hello,
This patch add function called "Live patching" which is defined on OSDL's carrier grade linux requiremnt definition to linux 2.6.11.7 kernel. The live patching allows process to patch on-line (without restarting process) on i386 and x86_64 architectures, by overwriting jump assembly code on entry point of functions which you want to fix, to patched functions.
The live patching function is very common on high-availability system such as carrier system, and this patch realize it also on linux. (Patch & process restart time is very critical on such high-availability system, live patch allows you to milliseconds order process stopping time to apply new patch.)

The basis is below:
1. Live patch command loads the patch modules to target process's memory area,
2. Live patch command resolve patch symbol.
3. Live patch command overwrite jump code to the entry point of function which you want to fix, to the patch module's symbol.

Kernel patch and user mode tools are required, and both of them are available at http://pannus.sourceforge.net
Please take a look and give us comments!

This patch add following system calls and function.
o mmap3: maps patch to target process's memory area with security check.
o accesspvm: access(read/write) target process's memory area.
o init_pend: initialization of live patch sequence on target process.
o rt_handlereturn: run initialize root of each patch (same as signal handler).
o check_init: check that the initialization is finished or not.
o munmap3: unmap patch from target process's memory area.

---
Takashi Ikebe
NTT Network Service Systems Laboratories
9-11, Midori-Cho 3-Chome Musashino-Shi,
Tokyo 180-8585 Japan
Tel : +81 422 59 4246, Fax : +81 422 60 4012
e-mail : [email protected]

#1 lore raw

Takashi-san, have you ever investigated using kprobes to implement this feature? It seems a perfect fit, and would allow support on several architectures other than just x86 and x86_64.

If kprobes does not meet your needs completely, it could be trivially extended to do so.

I think implementing something like this from scratch is not a good idea when we have much of the needed logic and infrastructure already.

#2 lore raw
Quoted 6 lines · David S. Miller

Takashi-san, have you ever investigated using kprobes to implement this feature? It seems a perfect fit, and would allow support on several architectures other than just x86 and x86_64.

Quoted 6 lines

If kprobes does not meet your needs completely, it could be trivially extended to do so.

I think implementing something like this from scratch is not a good idea when we have much of the needed logic and infrastructure already.

Takashi-san's description was not very clear, but it sounds like it's a patching mechanism for userspace applications - not for kernel space. So kprobes would not be a good fit.

If I'm right, I'm not sure why some of the bits of it were done separately instead of via the existing ptrace mechanism. And GDB would appreciate a mechanism for mmap/munmap/mprotect in a debugged process, also.

Signature · 2 lines
Daniel Jacobowitz
CodeSourcery, LLC
#3 lore raw
Quoted 3 lines · On Sun, 17 Apr 2005 14:51:43 -0400 Daniel Jacobowitz <[email protected]> wrote:

Takashi-san's description was not very clear, but it sounds like it's a patching mechanism for userspace applications - not for kernel space. So kprobes would not be a good fit.

I saw the presentation of this stuff at the Linux Kernel conference last year in Tokyo. I'm pretty sure it's for the kernel. :-)

#4 lore raw

Daniel-san, David-san,

Pannus project has two targets.
One is user-mode application live patching, and the other one is kernel live patching. What we posted now is user-mode application live patching function.

Quoted 4 lines

If I'm right, I'm not sure why some of the bits of it were done separately instead of via the existing ptrace mechanism. And GDB would appreciate a mechanism for mmap/munmap/mprotect in a debugged process, also.

Daniel-san,
GDB based approach seems not fit to our requirements. GDB(ptrace) based functions are basically need to be done when target process is stopping. From our experience, sometimes patches became to dozens to hundreds at one patching, and in this case GDB based approach cause target process's availability descent.

Patch exceeds 50k, so I cut comments and separate architecture, and post as in line.

Quoted 16 lines · David S. Miller
Quoted 3 lines · On Sun, 17 Apr 2005 14:51:43 -0400 Daniel Jacobowitz <[email protected]> wrote:

Takashi-san's description was not very clear, but it sounds like it's a patching mechanism for userspace applications - not for kernel space. So kprobes would not be a good fit.

I saw the presentation of this stuff at the Linux Kernel conference last year in Tokyo. I'm pretty sure it's for the kernel. :-) -
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

Signature · 6 lines
Takashi Ikebe
NTT Network Service Systems Laboratories
9-11, Midori-Cho 3-Chome Musashino-Shi,
Tokyo 180-8585 Japan
Tel : +81 422 59 4246, Fax : +81 422 60 4012
e-mail : [email protected]
#5 lore raw
Quoted 6 lines · Takashi Ikebe

Daniel-san,
GDB based approach seems not fit to our requirements. GDB(ptrace) based functions are basically need to be done when target process is stopping. From our experience, sometimes patches became to dozens to hundreds at one patching, and in this case GDB based approach cause target process's availability descent.

That's right, it does require the target process be stopped. If it isn't stopped how do you know it isn't executing the same instruction you're currently patching?

Even with hundreds of kilobytes of patch, I have trouble imagining this takes a substantial amount of time.

Signature · 2 lines
Daniel Jacobowitz
CodeSourcery, LLC
#6 lore raw
Quoted 12 lines · Daniel Jacobowitz
Quoted 8 lines · Takashi Ikebe

Daniel-san,
GDB based approach seems not fit to our requirements. GDB(ptrace) based functions are basically need to be done when target process is stopping. From our experience, sometimes patches became to dozens to hundreds at one patching, and in this case GDB based approach cause target process's availability descent.

Quoted 8 lines

That's right, it does require the target process be stopped. If it isn't stopped how do you know it isn't executing the same instruction you're currently patching?

Even with hundreds of kilobytes of patch, I have trouble imagining this takes a substantial amount of time.

Pannus patch does not require target process stop while loading(mapping) patch module to target process memory,
but as you described, target process stopping is needed whenever check EIP not to conflict, and overwrite jump assembly code.(This makes only few milliseconds target process stopping. Even on hundreds, it only takes dozens mill-seconds yet.)
Typically telecoms application needs soft real time, and has timeout. This kind of framework can not stop target process so long(Should be dozens milliseconds at worst).
We want not to stop target process whenever patch module is loading.... we want not to stop target process as possible as.

Signature · 6 lines
Takashi Ikebe
NTT Network Service Systems Laboratories
9-11, Midori-Cho 3-Chome Musashino-Shi,
Tokyo 180-8585 Japan
Tel : +81 422 59 4246, Fax : +81 422 60 4012
e-mail : [email protected]
#7 lore raw
In reply to David S. Miller
Quoted 7 lines · David S. Miller

Takashi-san, have you ever investigated using kprobes to implement this feature? It seems a perfect fit, and would allow support on several architectures other than just x86 and x86_64.

If kprobes does not meet your needs completely, it could be trivially extended to do so.

kprobes would require an exception for each patchpoint because it uses an trap instruction.

Probably a bit too costly for commonly used functions. Especially on a P4 exceptions are quite costly.

But you could add lightweight kprobes that just use jmp
for it in the kprobes framework, that might be useful for other stuff too

-Andi