Skip to contentSkip to conversation

Implement per-processor data areas for i386.

RFC PATCH Chuck Ebbert 3 participants 6 messages Aug 30, 2006

lore
Share & tools
lore
#0 lore raw

In-Reply-To: <[email protected]>

Quoted 6 lines · Jeremy Fitzhardinge
Quoted 3 lines

This changes the ABI for signals and ptrace() and that seems like a bad idea to me.

I don't believe it does; it certainly shouldn't change the usermode ABI. How do you see it changing?

Nevermind. I thought because you changed struct pt_regs in ptrace_abi.h it meant a user ABI change.

Quoted 7 lines
Quoted 5 lines

And the way things are done now is so ingrained into the i386 kernel that I'm not sure it can be done. E.g. I found two open-coded implementations of current, one in kernel_fpu_begin() and one in math_state_restore().

That's OK. The current task will still be available in thread_info;

But they can get out of sync, e.g. when switch_to() restores the new task's esp, the PDA still contains the old pcurrent and they don't get synchronized until the write_pda() in __switch_to().

Quoted 2 lines

To be honest, I haven't looked at percpu.h in great detail. I was making assumptions about how it works, but it looks like they were wrong.

Would it make any sense to replace the 'cpu' field in thread_info with a pointer to a PDA-like structure? We could even embed the static per_cpu data directly into that struct instead of chasing pointers...

Signature · 1 line
Chuck
#1 lore raw
Quoted 8 lines · Chuck Ebbert

In-Reply-To: <[email protected]>

Quoted 4 lines · Jeremy Fitzhardinge
Quoted 3 lines

This changes the ABI for signals and ptrace() and that seems like a bad idea to me.

Quoted 5 lines
Quoted 2 lines

I don't believe it does; it certainly shouldn't change the usermode ABI. How do you see it changing?

Nevermind. I thought because you changed struct pt_regs in ptrace_abi.h it meant a user ABI change.

I think he broke the ptrace ABI actually in the first patch, but only by mistake and he promised to fix it :)

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

And the way things are done now is so ingrained into the i386 kernel that I'm not sure it can be done. E.g. I found two open-coded implementations of current, one in kernel_fpu_begin() and one in math_state_restore().

Perhaps those should be fixed? Is there a reason they are open coded?

Quoted 7 lines
Quoted 3 lines
Quoted 1 line

That's OK. The current task will still be available in thread_info;

But they can get out of sync, e.g. when switch_to() restores the new task's esp, the PDA still contains the old pcurrent and they don't get synchronized until the write_pda() in __switch_to().

But there is neither kernel_fpu_begin nor math_state_restore inbetween. And I think interrupts are off too.

Quoted 7 lines
Quoted 3 lines

To be honest, I haven't looked at percpu.h in great detail. I was making assumptions about how it works, but it looks like they were wrong.

Would it make any sense to replace the 'cpu' field in thread_info with a pointer to a PDA-like structure? We could even embed the static per_cpu data directly into that struct instead of chasing pointers...

I don't see what advantage it would have. %gs is clearly faster and shorter.

-Andi

#2 lore raw
In reply to Chuck Ebbert
Quoted 3 lines · Chuck Ebbert

Nevermind. I thought because you changed struct pt_regs in ptrace_abi.h it meant a user ABI change.

Yes, that header seems pretty badly misnamed. The user-visible structure is user_pt_regs.

Quoted 4 lines

But they can get out of sync, e.g. when switch_to() restores the new task's esp, the PDA still contains the old pcurrent and they don't get synchronized until the write_pda() in __switch_to().

Yes, that's true, but the window is fairly small. More importantly, the question of "what task is currently running" is fundamentally ill-defined while you're in the middle of a context switch, so I don't think this is a big issue. __switch_to runs on the new task's stack, so that's effectively where current_thread_info()->task changes value (aside from the few instructions in switch_to() between the %esp update and the jmp to __switch_to). I could put the write_pda() earlier in __switch_to so the window is smaller. But again, for it to make a difference, someone would want to be using current *within* __switch_to, which is just silly.

Quoted 7 lines
Quoted 2 lines

To be honest, I haven't looked at percpu.h in great detail. I was making assumptions about how it works, but it looks like they were wrong.

Would it make any sense to replace the 'cpu' field in thread_info with a pointer to a PDA-like structure? We could even embed the static per_cpu data directly into that struct instead of chasing pointers...

I don't think so. The whole point is to make the pda easily accessible with simple addressing modes based on %gs:. I have been wondering if we can modify the percpu mechanism to get the linker to construct the layout of the pda, so that all the existing percpu stuff can be transparently moved into the pda and accessed efficiently. I think it would be pretty tricky to get it all working though...

But the PDA isn't really intended for *all* per-cpu data; its mostly for stuff which is accessed often, or needs to be quickly and easily accessibly. My specific motivation is to use the PDA for easy access to Xen per-cpu data, which needs to be accessible with short instructions which can be inlined.

J

#3 lore raw
Quoted 8 lines

I don't think so. The whole point is to make the pda easily accessible
with simple addressing modes based on %gs:. I have been wondering if
we can modify the percpu mechanism to get the linker to construct the layout of the pda, so that all the existing percpu stuff can be transparently moved into the pda and accessed efficiently. I think it would be pretty tricky to get it all working though...

I tried that once on x86-64, but it wasn't possible because the linker is missing the right relocations. It has something on the first look similar for __thread data in user space, but it wasn't usable for the kernel.

Even with a single indirection it is still far more efficient than a array lookup.

-Andi

#4 lore raw
Quoted 3 lines · Andi Kleen

I tried that once on x86-64, but it wasn't possible because the linker
is missing the right relocations. It has something on the first look similar for __thread data in user space, but it wasn't usable for the kernel.

The other difficulty is that you can't take the addresses of things in the pda and pass them around, which happens a lot. It would be another matter if you could add an attribute to a pointer which told gcc "always use a %gs override for indirecting through this pointer", though that would still require some type qualifier on any pointer which holds the value.

(Hm, it would be interesting to see if we could possibly use the code generated by gcc for TLS variables...)

Quoted 1 line

Even with a single indirection it is still far more efficient than a array lookup.

Yes. Even in the Xen case it will be useful to have a pointer to the vcpu structure, at least until Xen can be convinced to put the vcpu structure in the PDA itself.

J

#5 lore raw
Quoted 7 lines · Jeremy Fitzhardinge
Quoted 4 lines · Andi Kleen

I tried that once on x86-64, but it wasn't possible because the linker is missing the right relocations. It has something on the first look similar for __thread data in user space, but it wasn't usable for the kernel.

The other difficulty is that you can't take the addresses of things in the pda and pass them around, which happens a lot.

The user space __thread works around this by always storing the address at offset 0. Kernel does it similar, except it's not at offset 0.

Quoted 2 lines

(Hm, it would be interesting to see if we could possibly use the code generated by gcc for TLS variables...)

I tried once on 64bit and it wasn't possible for various reasons.

-Andi