Skip to contentSkip to conversation

2.6.22.6 - Discrepancy between running and on-disk kernels

DISCUSSION Yan 2 participants 4 messages Mar 14, 2008

lore
Share & tools
lore

Yan

author
#0 lore raw

Hello,

I have been trying to compare the code from the on-disk compressed kernel that was booted and the running kernel extracted from /dev/kmem. I extracted the kernel's code from the disk image by stripping the head.S and similar and gunzipping it, and extracted the kernel from /dev/kmem by reading data between _text and _etext symbol offsets.

I then ran both through a disassembler and diff'ed the outputs. Predictably, the disassembly was similar, but not identical. Some instructions (e.g. bts) had a 'lock' prefix, where as others had a 'nop' in its place.

There were other differences with some instructions like mfence. Everything else matched just fine, the differences were mostly in memory-referencing instructions.

My question is, what can be changing the kernel between being on static storage and being loaded? Or am I just being fooled into thinking something changed it? I'm thinking that it had something to do with mutex locking and SMP, but I looked through boot code and tried googling and came up with nothing. Another possibility is me interpreting non-instructions as instructions.

Example:

The following pattern repeats a lot:

1325c1326,1327
< 1089: f0 0f b3 04 24 lock btr %eax,(%esp)
---

Quoted 2 lines

1089: 90 nop
108a: 0f b3 04 24 btr %eax,(%esp)

As well as other changes:

94383,94384c94832,94835

<    47bb3: f0 83 04 24 00          lock addl $0x0,(%esp)
<    47bb8: f0 ff 05 7c 30 80 c0    lock incl 0xc080307c

---

Quoted 4 lines
   47bb3: 0f ae f0                mfence
   47bb6: 89 f6                   mov    %esi,%esi
   47bb8: 90                      nop
   47bb9: ff 05 7c 30 80 c0       incl   0xc080307c

(I'm not subscribed; please cc any responses to me)

Thanks in advance,
Yan

#1 lore raw
Quoted 14 lines · On Fri, 14 Mar 2008 10:51:43 -0400 Yan <[email protected]> wrote:

Hello,

I have been trying to compare the code from the on-disk compressed kernel that was booted and the running kernel extracted from /dev/kmem. I extracted the kernel's code from the disk image by stripping the head.S and similar and gunzipping it, and extracted the kernel from /dev/kmem by reading data between _text and _etext symbol offsets.

I then ran both through a disassembler and diff'ed the outputs. Predictably, the disassembly was similar, but not identical. Some instructions (e.g. bts) had a 'lock' prefix, where as others had a 'nop' in its place.

Quoted 6 lines

There were other differences with some instructions like mfence. Everything else matched just fine, the differences were mostly in memory-referencing instructions.

My question is, what can be changing the kernel between being on static storage and being loaded?

the kernel code is self-patching, it gets modified to match your system during boot time. So you cannot assume that the kernel on disk and the kernel in memory are identical.
(Same goes for modules)

author
#2 lore raw

Can you point me to where in the source that self-patching happens? I tried grep'ing
through the source for anything relevant, and came out empty handed.

-yan

Quoted 29 lines · Arjan van de Ven
Quoted 14 lines · On Fri, 14 Mar 2008 10:51:43 -0400 Yan <[email protected]> wrote:

Hello,

I have been trying to compare the code from the on-disk compressed kernel that was booted and the running kernel extracted from /dev/kmem. I extracted the kernel's code from the disk image by stripping the head.S and similar and gunzipping it, and extracted the kernel from /dev/kmem by reading data between _text and _etext symbol offsets.

I then ran both through a disassembler and diff'ed the outputs. Predictably, the disassembly was similar, but not identical. Some instructions (e.g. bts) had a 'lock' prefix, where as others had a 'nop' in its place.

Quoted 6 lines

There were other differences with some instructions like mfence. Everything else matched just fine, the differences were mostly in memory-referencing instructions.

My question is, what can be changing the kernel between being on static storage and being loaded?

the kernel code is self-patching, it gets modified to match your system during boot time. So you cannot assume that the kernel on disk and the kernel in memory are identical.
(Same goes for modules)

#3 lore raw
Quoted 3 lines · On Fri, 14 Mar 2008 12:08:16 -0400 Yan <[email protected]> wrote:

Can you point me to where in the source that self-patching happens? I tried grep'ing
through the source for anything relevant, and came out empty handed.

in the apply_alternatives() function

Quoted 34 lines

-yan

Quoted 29 lines · Arjan van de Ven
Quoted 14 lines · On Fri, 14 Mar 2008 10:51:43 -0400 Yan <[email protected]> wrote:

Hello,

I have been trying to compare the code from the on-disk compressed kernel that was booted and the running kernel extracted from /dev/kmem. I extracted the kernel's code from the disk image by stripping the head.S and similar and gunzipping it, and extracted the kernel from /dev/kmem by reading data between _text and _etext symbol offsets.

I then ran both through a disassembler and diff'ed the outputs. Predictably, the disassembly was similar, but not identical. Some instructions (e.g. bts) had a 'lock' prefix, where as others had a 'nop' in its place.

Quoted 6 lines

There were other differences with some instructions like mfence. Everything else matched just fine, the differences were mostly in memory-referencing instructions.

My question is, what can be changing the kernel between being on static storage and being loaded?

the kernel code is self-patching, it gets modified to match your
system during boot time. So you cannot assume that the kernel on disk and the kernel in memory are identical.
(Same goes for modules)

Signature · 3 lines
If you want to reach me at my work email, use [email protected]
For development, discussion and tips for power savings, 
visit http://www.lesswatts.org