Skip to contentSkip to conversation

Kernel support for peer-to-peer protection models...

DISCUSSION Ivan Godard 3 participants 7 messages Mar 27, 2004 – Mar 29, 2004

lore
Share & tools
lore
#0 lore raw

We're a processor startup with a new architecture that we will be porting Linux to. The bulk of the port will be straightforward (well, you know what I mean), except for the protection model supported by the hardware. How would you extend/mod the kernel if you had hardware that:

1) had a large number of distinguishable address spaces
2) any running code had two of these (code and data environment) it could use arbitrarily, but access to addresses in others was arbitrarily protected
3) flat, unified virtual addresses (64 bit) so that pointers, including inter-space pointers, have the same representation in all spaces
4) no "supervisor mode"
5) inter-space references require grant of access (transitive) by the accessed space; grants can be entire space or any contiguous subspace
6) inter-space reference has same performance as intra-space
7) you can call across space boundaries in a way that changes the governing space and consequently the protection environment. Thus both code and data of a server process or DLL may be non-addressable by an app process, but the app can call a function (controllably) in the server/DLL and that function will run in the protection environment of the server/DLL, not that of the app, though it will run in the app's stack (in the app's address space) with no task switch. Return unwinds the change. This with full protection both ways - neither the app nor the server/DLL can klobber each other. Performance is the same as a conventional intra-space call/return.
8) OS calls are not traps, just inter-space calls, again with full protection.
9) Hardware interrupts are involuntary inter-space calls. They do not require locking (assuming the handler is re-entrant - and if not then only from themselves), nor task switch, nor disabling other interrupts. The handler runs in the stack of whoever got interrupted, which (depending on interrupt priorities) could be another interrupt, on an interrupt, ... on an app, all mutually protected.
10) Drivers can have their own individual space(s) distinct from those of the kernel and the apps. Buggy drivers cannot crash the kernel.

We could of course ignore the hardware model and emulate a conventional processor (and might, as a first step of the port), but this would discard significant reliability/performance improvements that the hardware could provide.

Is this model so alien to the existing Kernel that the best approach is to peel off a kernel tarball and create a new kernel, to be maintained in isolation forever? Or would this work fit into planned/expected kernel work dealing with protection models, interrupts, trap handling and the like? What about partitioning the kernel into disjoint (and mutually protected) components like IP stack, password/security, FS etc?

We will not be beginning this port for six months or so - the compiler has to come up first :-) So this question is more to get some sense of the lay of the land rather than any immediate help. We know that there is a lot of know-how and strong opinions concentrated in the kernel development crew, and we want to gain from that and also to contribute what is currently budgeted as some 10 engineer-years of work upcoming to the general improvement if it can be done. Alternatively we might be too wierd to be worth bothering with for the group, and so we should do it on our own and not try to fit with the "Linux way of doing things".

Comments please?

Ivan

#1 lore raw
Quoted 2 lines · Ivan Godard

3) flat, unified virtual addresses (64 bit) so that pointers, including inter-space pointers, have the same representation in all spaces

How are you going to implement fork() ?

Paul.

#2 lore raw

----- Original Message -----
From: "Paul Mackerras" <[email protected]>
To: "Ivan Godard" <[email protected]>
Cc: "Linux Kernel Mailing List" <[email protected]>
Sent: Sunday, March 28, 2004 4:17 PM
Subject: Re: Kernel support for peer-to-peer protection models...

Quoted 6 lines
Quoted 2 lines · Ivan Godard

3) flat, unified virtual addresses (64 bit) so that pointers, including inter-space pointers, have the same representation in all spaces

How are you going to implement fork() ?

The usual COW using the page tables. The child keeps the same code space but gets a new data space. I expect that specialized versions of fork will give explicit control over which space the child gets, but in comman usage no one cases just as no one cares which PID it gets.

Ivan

#3 lore raw
Quoted 13 lines · Ivan Godard

----- Original Message -----
From: "Paul Mackerras" <[email protected]>
To: "Ivan Godard" <[email protected]>
Cc: "Linux Kernel Mailing List" <[email protected]>
Sent: Sunday, March 28, 2004 4:17 PM
Subject: Re: Kernel support for peer-to-peer protection models...

Quoted 5 lines
Quoted 2 lines · Ivan Godard

3) flat, unified virtual addresses (64 bit) so that pointers, including inter-space pointers, have the same representation in all spaces

Quoted 6 lines
Quoted 1 line

How are you going to implement fork() ?

The usual COW using the page tables. The child keeps the same code space but gets a new data space. I expect that specialized versions of fork will give explicit control over which space the child gets, but in comman usage no one cases just as no one cares which PID it gets.

Uh?

int myexec(char const *cmd) {

	if (!fork()) {
		exit(exec(cmd));
	}
	...
}

- Davide

#4 lore raw

----- Original Message -----
From: "Davide Libenzi" <[email protected]>
To: "Ivan Godard" <[email protected]>
Cc: "Paul Mackerras" <[email protected]>; "Linux Kernel Mailing List" <[email protected]>
Sent: Sunday, March 28, 2004 7:48 PM
Subject: Re: Kernel support for peer-to-peer protection models...

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 1 line · Ivan Godard

3) flat, unified virtual addresses (64 bit) so that pointers,

including

Quoted 5 lines
Quoted 5 lines
Quoted 3 lines
Quoted 1 line

inter-space pointers, have the same representation in all spaces

How are you going to implement fork() ?

The usual COW using the page tables. The child keeps the same code space

but

Quoted 1 line
Quoted 1 line

gets a new data space. I expect that specialized versions of fork will

give

Quoted 1 line
Quoted 1 line

explicit control over which space the child gets, but in comman usage no

one

Quoted 4 lines
Quoted 1 line

cases just as no one cares which PID it gets.

Uh?

Quoted 8 lines

int myexec(char const *cmd) {

if (!fork()) {
exit(exec(cmd));
}
...
}

Ah! you wanted to know about exec, not fork. A true fork() is pretty rare these days anyway. Still, the answer is pretty much the same: the fork() gets you a new data space, retaining the old code space, and the exec() finds (or creates) the code space that cmd's code is in and switches the active code space to that space. Heritable data, such as file descriptors, won't have been in the old data space anyway, so the child references them through syscalls just as in a conventional.

Perhaps I'm missing your question here, but in general we see no problem with fork/exec in our model - it's one of the least changed things. You always get a new address space on a conventional, and you also do with a Mill; the only difference is that you don't have to shoot down the cache or TLB, so a fork/exec should be quite a bit faster.

Ivan

#5 lore raw
Quoted 8 lines · Ivan Godard

Ah! you wanted to know about exec, not fork. A true fork() is pretty rare these days anyway. Still, the answer is pretty much the same: the fork() gets you a new data space, retaining the old code space, and the exec() finds (or creates) the code space that cmd's code is in and switches the active code space to that space. Heritable data, such as file descriptors, won't have been in the old data space anyway, so the child references them through syscalls just as in a conventional.

Quoted 5 lines

Perhaps I'm missing your question here, but in general we see no problem with fork/exec in our model - it's one of the least changed things. You always get a new address space on a conventional, and you also do with a Mill; the only difference is that you don't have to shoot down the cache or TLB, so a fork/exec should be quite a bit faster.

No, sorry. Lossy email compression (sometimes being concise is a virtue but I usually fall way too far). Reading thru previous messages seems confusing to me. On one side you say that fork() uses a std COW, on the other side you say that the unified virtual address space combined with virtually tagged cache let you avoid cache flushes. As soon as you COW, you have one virtual address that refer to two different physical addresses. Does the virtual address have extra tag bits to identify the task?

- Davide

#6 lore raw

----- Original Message -----
From: "Davide Libenzi" <[email protected]>
To: "Ivan Godard" <[email protected]>
Cc: "Paul Mackerras" <[email protected]>; "Linux Kernel Mailing List" <[email protected]>
Sent: Monday, March 29, 2004 10:45 AM
Subject: Re: Kernel support for peer-to-peer protection models...

Quoted 3 lines · Ivan Godard
Quoted 2 lines

Ah! you wanted to know about exec, not fork. A true fork() is pretty

rare

Quoted 4 lines
Quoted 4 lines

these days anyway. Still, the answer is pretty much the same: the fork() gets you a new data space, retaining the old code space, and the exec() finds (or creates) the code space that cmd's code is in and switches the active code space to that space. Heritable data, such as file

descriptors,

Quoted 1 line
Quoted 1 line

won't have been in the old data space anyway, so the child references

them

Quoted 6 lines
Quoted 6 lines

through syscalls just as in a conventional.

Perhaps I'm missing your question here, but in general we see no problem with fork/exec in our model - it's one of the least changed things. You always get a new address space on a conventional, and you also do with a Mill; the only difference is that you don't have to shoot down the cache

or

Quoted 8 lines
Quoted 1 line

TLB, so a fork/exec should be quite a bit faster.

No, sorry. Lossy email compression (sometimes being concise is a virtue but I usually fall way too far). Reading thru previous messages seems confusing to me. On one side you say that fork() uses a std COW, on the other side you say that the unified virtual address space combined with virtually tagged cache let you avoid cache flushes. As soon as you COW, you have one virtual address that refer to two different physical

addresses.

Quoted 1 line

Does the virtual address have extra tag bits to identify the task?

No, but it has a mechanism (unfortunately I can't explain yet how it works) which amount to the same thing. In fact, the hardware can distinguish between the case where a particular address in the child is supposed to refer to the child, and the case where it's supposed to refer back into the parent. This is a novel feature that is not supportd by the conventional model. We expect to expose it through an API. For example a server can spawn worker processes such that each worker has its own space (and is protected from other workers) but can communicate with the server via data structures in the server, where the new worker COW comes up with the back-references already created. This gives the convenience of thread-workers with the protection of process-workers using a mmap communication region, but without the overhead of creating the region.

Ivan