Skip to contentSkip to conversation

proposed interface change for setting the ldt

DISCUSSION Jeremy Fitzhardinge 4 participants 12 messages Aug 18, 2006 – Aug 21, 2006 virtio core

lore
Share & tools
lore
#0 lore raw

At the moment, all the places where the LDT are set in the kernel are of the form:

    set_ldt_desc(cpu, segments, count);
    load_LDT_desc();

(two instances in asm-i386/desc.h). set_ldt_desc() sets an LDT descriptor in the GDT, and load_LDT_desc() is basically just lldt. These map to the write_gdt_entry and load_ldt_desc paravirt ops.

This doesn't work well for Xen, because you set the ldt directly by passing the base+size into the hypervisor. In fact, it doesn't allow you to set an LDT-type descriptor into the GDT, so this current interface requires the Xen backend to decode the descriptor passed to write_gdt_entry, look to see if its an LDT; if so, store the base+size somewhere, and then when load_ldt_desc() is called, do the appropriate Xen hypercall.

A better interface for us would be simply:

	set_ldt(const struct desc_struct *ldt, int num_entries);

since this maps directly to the appropriate Xen hypercall. If you still want to implement it by plugging the LDT descriptor into the GDT and then lldt, then there's no reason you can't implement it that way.

Thoughts?

J

#1 lore raw
Quoted 2 lines · Jeremy Fitzhardinge
	set_ldt(const struct desc_struct *ldt, int num_entries);

Make that

	set_ldt(unsigned cpu, const struct desc_struct *ldt, int num_entries);

    J
#2 lore raw
Quoted 7 lines · Jeremy Fitzhardinge
Quoted 2 lines · Jeremy Fitzhardinge
    set_ldt(const struct desc_struct *ldt, int num_entries);

Make that

set_ldt(unsigned cpu, const struct desc_struct *ldt, int
num_entries);

Or maybe not. The current code assumes that cpu is the current cpu anyway; the implementation can use smp_processor_id() if it needs it anyway (Xen doesn't).

J

#3 lore raw
In reply to Jeremy Fitzhardinge
Quoted 9 lines · Jeremy Fitzhardinge

A better interface for us would be simply:

    set_ldt(const struct desc_struct *ldt, int num_entries);

since this maps directly to the appropriate Xen hypercall. If you still want to implement it by plugging the LDT descriptor into the GDT and then lldt, then there's no reason you can't implement it that way.

Thoughts?

This interface doesn't work for anything other than Xen. It is impossible to implement it without specific knowledge of kernel internals, since it doesn't provide the GDT selector for the LDT. Now everything that looks like real hardware needs to move the knowledge of the LDT structure into paravirt-ops, and it has no clear calling convention, so you've now got to reason about SMP preemption correctness inside the paravirt-op, instead of at the higher level where it should be done.

This is an example of one place where Xen has broken the x86 architecture in favor of a simpler implementation of the hypervisor, but a radical change to the kernel. It is one of the reasons why XenoLinux as implemented is unable to boot on native hardware. I would strongly prefer if we didn't introduce this ugliness into paravirt-ops.

Zach

#4 lore raw
Quoted 1 line · Zachary Amsden

This interface doesn't work for anything other than Xen.

It works OK for native. It's a very simple rolling together of two operations which always happen together anyway.

Quoted 2 lines

It is impossible to implement it without specific knowledge of kernel internals, since it doesn't provide the GDT selector for the LDT.

Neither does the previous interface. load_ldt_desc needs to have the specific LDT entry hardcoded into it.

Quoted 2 lines

Now everything that looks like real hardware needs to move the
knowledge of the LDT structure into paravirt-ops,

Do you mean the GDT structure?

Quoted 3 lines

and it has no clear calling convention, so you've now got to reason about SMP preemption correctness inside the paravirt-op, instead of at the higher level where it should be done.

The previous interface already required that preempt be disabled around those functions. In the previous interface, set_ldt_desc takes a cpu number, but it is required to equal the current cpu; load_ldt_desc always operates on the current CPU. It therefore requires that those two ops be paired with preempt disabled. The new interface is simpler, but still requires preempt disabled around it.

In general, the set_ldt interface is cleaner for the base kernel, and much cleaner for Xen, while being trivial to implement for native hardware or something which looks like it.

J

#5 lore raw
Quoted 9 lines · Jeremy Fitzhardinge
Quoted 1 line · Zachary Amsden

This interface doesn't work for anything other than Xen.

It works OK for native. It's a very simple rolling together of two operations which always happen together anyway.

Quoted 2 lines

It is impossible to implement it without specific knowledge of kernel internals, since it doesn't provide the GDT selector for the LDT.

Quoted 7 lines

Neither does the previous interface. load_ldt_desc needs to have the specific LDT entry hardcoded into it.

Quoted 2 lines

Now everything that looks like real hardware needs to move the
knowledge of the LDT structure into paravirt-ops,

Do you mean the GDT structure?

Yes.

Quoted 11 lines
Quoted 4 lines

and it has no clear calling convention, so you've now got to reason about SMP preemption correctness inside the paravirt-op, instead of at the higher level where it should be done.

The previous interface already required that preempt be disabled around those functions. In the previous interface, set_ldt_desc takes a cpu number, but it is required to equal the current cpu; load_ldt_desc always operates on the current CPU. It therefore requires that those two ops be paired with preempt disabled. The new interface is simpler, but still requires preempt disabled around it.

The paravirt-op just got a lot harder to implement, so there is a cost to the simpler interface.

Quoted 4 lines

In general, the set_ldt interface is cleaner for the base kernel, and much cleaner for Xen, while being trivial to implement for native hardware or something which looks like it.

I just think it's really weird to have LDT not described in the GDT, but LDT is weird anyways.

Zach

#6 lore raw
Quoted 2 lines · Zachary Amsden

The paravirt-op just got a lot harder to implement, so there is a cost to the simpler interface.

I don't see why it is a "lot" harder. It might take 5 lines of code rather than jumping straight into the ROM, but it doesn't seem like huge complexity.

Quoted 2 lines

I just think it's really weird to have LDT not described in the GDT, but LDT is weird anyways.

It gets excluded as part of a broader test which also prevents any table, but specifically the LDT, from referring to an LDT (on the grounds that its hard to reason about, and may be undefined).

J

#8 lore raw
Quoted 9 lines · Chris Wright
Quoted 3 lines · * Zachary Amsden ([email protected]) wrote:

The paravirt-op just got a lot harder to implement, so there is a cost to the simpler interface.

I'm missing why it's a lot harder. Seems reasonably straight forward. puzzled...

Before it could be a direct call for us. Now I am forced to write a wrapper function around it which does exactly the same work as the native code, them calls a ROM function. It is straight forward, but obviously undesirable.

Zach

#9 lore raw
Quoted 7 lines · Zachary Amsden
Quoted 6 lines · Chris Wright
Quoted 3 lines · * Zachary Amsden ([email protected]) wrote:

The paravirt-op just got a lot harder to implement, so there is a cost to the simpler interface.

Quoted 8 lines
Quoted 3 lines

I'm missing why it's a lot harder. Seems reasonably straight forward. puzzled...

Before it could be a direct call for us. Now I am forced to write a wrapper function around it which does exactly the same work as the native code, them calls a ROM function. It is straight forward, but obviously undesirable.

It sounds fine to me, although I'd like to see the patch. I don't have anything against higher-level abstractions, if it helps any hypervisor, as long as it doesn't warp the kernel code. And if most hypervisors and native break it down the same way, well, we can always create helpers.

Cheers,
Rusty.

Signature · 1 line
Help! Save Australia from the worst of the DMCA: http://linux.org.au/law
#10 lore raw
Quoted 5 lines · Rusty Russell

It sounds fine to me, although I'd like to see the patch. I don't have anything against higher-level abstractions, if it helps any hypervisor, as long as it doesn't warp the kernel code. And if most hypervisors and native break it down the same way, well, we can always create helpers.

Doing it this way does save a lot of complexity in the Xen case, which would otherwise have to:

1. unpack the descriptor
2. if it's an LDT, save it off in a per-cpu structure
3. when load_ldt_desc is called, look up the structure, extract the base+size of the ldt, and make a hypercall to set the ldt

vs

1. set the ldt

J

#11 lore raw
Quoted 10 lines · Jeremy Fitzhardinge
Quoted 5 lines · Rusty Russell

It sounds fine to me, although I'd like to see the patch. I don't have anything against higher-level abstractions, if it helps any hypervisor, as long as it doesn't warp the kernel code. And if most hypervisors and native break it down the same way, well, we can always create helpers.

Doing it this way does save a lot of complexity in the Xen case, which would otherwise have to:

Quoted 8 lines

1. unpack the descriptor
2. if it's an LDT, save it off in a per-cpu structure
3. when load_ldt_desc is called, look up the structure, extract the base+size of the ldt, and make a hypercall to set the ldt

vs

1. set the ldt

This way works for us as well, but my opinion is - Xen deliberately broke the x86 interface here, for no strong reason. In an ideal world, we shouldn't have to change Linux to work around that, nor should other hypervisors have to live with the consequences of that decision. So I believe the onus is on Xen to fix the problem in their layer, not the other way around. But I have no problem adapting to this way of doing things for now, since this is not performance critical, and LDT is barely used in Linux anyway, even then, mostly by unsupported applications in a paravirtualized world.

Zach