Skip to contentSkip to conversation

Kernel support for peer-to-peer protection models...

DISCUSSION Pavel Machek 2 participants 7 messages Mar 27, 2004 – Mar 28, 2004 1 message not in the archive

lore
Share & tools
lore

A message that is not in the archive (sent off-list or not yet received): 048e01c413b3_3c3cae60_fc82c23f@pc21

#1 lore raw

Hi!

Quoted 5 lines

1) had a large number of distinguishable address spaces
2) any running code had two of these (code and data environment) it could use arbitrarily, but access to addresses in others was arbitrarily protected
3) flat, unified virtual addresses (64 bit) so that pointers, including inter-space pointers, have the same representation in all spaces

Hmm, will it be possible to have UML?

Quoted 1 line

4) no "supervisor mode"

Is all your i/o memory mapped?

Quoted 3 lines

5) inter-space references require grant of access (transitive) by the accessed space; grants can be entire space or any contiguous subspace
6) inter-space reference has same performance as intra-space

Huh? Does it mean that all the accesses are horibly slow?

Quoted 6 lines

9) Hardware interrupts are involuntary inter-space calls. They do not require locking (assuming the handler is re-entrant - and if not then only from themselves), nor task switch, nor disabling other interrupts. The handler runs in the stack of whoever got interrupted, which (depending on interrupt priorities) could be another interrupt, on an interrupt, ... on an app, all mutually protected.

How do you implement ptrace if apps are protected from kernel?

Quoted 2 lines

10) Drivers can have their own individual space(s) distinct from those of the kernel and the apps. Buggy drivers cannot crash the kernel.

Well... buggy drivers can usually program DMA to do crashing for them. How is your architecture called?

Quoted 3 lines

dealing with protection models, interrupts, trap handling and the like? What about partitioning the kernel into disjoint (and mutually protected) components like IP stack, password/security, FS etc?

That would be pretty big rewrite...

Anyway, I believe you *do* want linux on it, if only as a test load.

Signature · 1 line
64 bytes from 195.113.31.123: icmp_seq=28 ttl=51 time=448769.1 ms         
#2 lore raw

Interlinear

----- Original Message -----
From: "Pavel Machek" <[email protected]>
To: "Ivan Godard" <[email protected]>
Cc: "Linux Kernel Mailing List" <[email protected]>
Sent: Saturday, March 27, 2004 2:34 AM
Subject: Re: Kernel support for peer-to-peer protection models...

Quoted 4 lines

Hi!

Quoted 2 lines

1) had a large number of distinguishable address spaces
2) any running code had two of these (code and data environment) it

could

Quoted 1 line
Quoted 1 line

use arbitrarily, but access to addresses in others was arbitrarily

protected

Quoted 4 lines
Quoted 2 lines

3) flat, unified virtual addresses (64 bit) so that pointers, including inter-space pointers, have the same representation in all spaces

Hmm, will it be possible to have UML?

If by UML you mean Uniform Modelling Language, I don't understand where the protection model has any impact. UML models flow, permissions are somewhat superimposed, just like file permissions in a UML model on any machine.

Quoted 3 lines
Quoted 1 line

4) no "supervisor mode"

Is all your i/o memory mapped?

Yes, and also those few machine operations that are "priveledged".

Quoted 5 lines
Quoted 3 lines

5) inter-space references require grant of access (transitive) by the accessed space; grants can be entire space or any contiguous subspace
6) inter-space reference has same performance as intra-space

Huh? Does it mean that all the accesses are horibly slow?

No, they run at full regular memory speed, at all levels of the memory heirarchy. Because of the flat unified address space, all caches can be in virtual (this is common in 64-bit address systems) because a pointer means the same thing no matter who uses it. A consequence is that you don't have to cache scrub on task switch, which is a big time win.

Quoted 2 lines
Quoted 2 lines

9) Hardware interrupts are involuntary inter-space calls. They do not require locking (assuming the handler is re-entrant - and if not then

only

Quoted 2 lines
Quoted 2 lines

from themselves), nor task switch, nor disabling other interrupts. The handler runs in the stack of whoever got interrupted, which (depending

on

Quoted 1 line
Quoted 1 line

interrupt priorities) could be another interrupt, on an interrupt, ...

on an

Quoted 3 lines
Quoted 1 line

app, all mutually protected.

How do you implement ptrace if apps are protected from kernel?

Anybody can make all or part of themselves visible to anybody else. If you start up an app in your space, you can grant visibility to a debugger in another space. But this applies only to you. For example, suppose that your app calls a paranoid server DLL passing in a function, and the DLL in turn calls back your function. Then your stack will have a hunk of you (that you can see and expose to the debugger), then a hunk of DLL function activations (which are opaque to you AND the debugger unless you can talk the DLL into exposing itself), and then another hunk of you again (and again visible to you and the debugger). The DLL and you (and your debugger) are mutually protected.

To do this on a conventional system requires that the DLL runs as a server process, and getting it to do something for you requires a roundtrip through the dispatcher. For us it's a simple subroutine call, just as if the DLL were un-paranoid and had been linked into your code. Clearer?

Quoted 1 line
Quoted 1 line

10) Drivers can have their own individual space(s) distinct from those

of

Quoted 3 lines
Quoted 1 line

the kernel and the apps. Buggy drivers cannot crash the kernel.

Well... buggy drivers can usually program DMA to do crashing for them.

Nope. The DMA has the same permissions as the driver that starts it.

Quoted 1 line

How is your architecture called?

"Mill"

Quoted 1 line
Quoted 1 line

dealing with protection models, interrupts, trap handling and the like?

What

Quoted 6 lines
Quoted 2 lines

about partitioning the kernel into disjoint (and mutually protected) components like IP stack, password/security, FS etc?

That would be pretty big rewrite...

Anyway, I believe you *do* want linux on it, if only as a test load.

We definitely want Linux. The question is whether Linux will want the result of our port, or (in finer detail) what parts could we do in what way to be useful to other people.

Ivan

#3 lore raw

Hi!

Quoted 8 lines · Ivan Godard
Quoted 2 lines
Quoted 2 lines

1) had a large number of distinguishable address spaces
2) any running code had two of these (code and data environment) it

could

Quoted 1 line
Quoted 1 line

use arbitrarily, but access to addresses in others was arbitrarily

protected

Quoted 3 lines
Quoted 2 lines

3) flat, unified virtual addresses (64 bit) so that pointers, including inter-space pointers, have the same representation in all spaces

Quoted 6 lines
Quoted 1 line

Hmm, will it be possible to have UML?

If by UML you mean Uniform Modelling Language, I don't understand where the protection model has any impact. UML models flow, permissions are somewhat superimposed, just like file permissions in a UML model on any machine.

I meant "User Mode Linux" == linux running under linux. Someone probably has an URL.

Quoted 23 lines
Quoted 2 lines
Quoted 2 lines

9) Hardware interrupts are involuntary inter-space calls. They do not require locking (assuming the handler is re-entrant - and if not then

only

Quoted 2 lines
Quoted 2 lines

from themselves), nor task switch, nor disabling other interrupts. The handler runs in the stack of whoever got interrupted, which (depending

on

Quoted 1 line
Quoted 1 line

interrupt priorities) could be another interrupt, on an interrupt, ...

on an

Quoted 3 lines
Quoted 1 line

app, all mutually protected.

How do you implement ptrace if apps are protected from kernel?

Anybody can make all or part of themselves visible to anybody else. If you start up an app in your space, you can grant visibility to a debugger in another space. But this applies only to you. For example, suppose that your app calls a paranoid server DLL passing in a function, and the DLL in turn calls back your function. Then your stack will have a hunk of you (that you can see and expose to the debugger), then a hunk of DLL function activations (which are opaque to you AND the debugger unless you can talk the DLL into exposing itself), and then another hunk of you again (and again visible to you and the debugger). The DLL and you (and your debugger) are mutually protected.

Quoted 4 lines

To do this on a conventional system requires that the DLL runs as a server process, and getting it to do something for you requires a roundtrip through the dispatcher. For us it's a simple subroutine call, just as if the DLL were un-paranoid and had been linked into your code. Clearer?

Strange system.... If an application does not grant kernel access to its space, how is kernel supposed to do its job? For example, that "paranoid DLL" becomes unswappable, then?

If you have "enough" paranoid DLLs, you can then bring the machine down due to lack of real memory :-).

Quoted 7 lines
Quoted 1 line
Quoted 1 line

10) Drivers can have their own individual space(s) distinct from those

of

Quoted 3 lines
Quoted 1 line

the kernel and the apps. Buggy drivers cannot crash the kernel.

Well... buggy drivers can usually program DMA to do crashing for them.

Nope. The DMA has the same permissions as the driver that starts it.

So normal PCI cards are not allowed, or do you have some kind of IOMMU?

Quoted 7 lines
Quoted 3 lines

That would be pretty big rewrite...

Anyway, I believe you *do* want linux on it, if only as a test load.

We definitely want Linux. The question is whether Linux will want the result of our port, or (in finer detail) what parts could we do in what way to be useful to other people.

If most changes are in arch/, it should be acceptable...
Pavel

Signature · 2 lines
When do you have a heart between your knees?
[Johanka's followup: and *two* hearts?]
#4 lore raw

----- Original Message -----
From: "Pavel Machek" <[email protected]>
To: "Ivan Godard" <[email protected]>
Cc: "Linux Kernel Mailing List" <[email protected]>
Sent: Saturday, March 27, 2004 10:24 PM
Subject: Re: Kernel support for peer-to-peer protection models...

Quoted 9 lines

Hi!

Quoted 6 lines · Ivan Godard
Quoted 2 lines
Quoted 2 lines

1) had a large number of distinguishable address spaces
2) any running code had two of these (code and data environment) it

could

Quoted 1 line
Quoted 1 line

use arbitrarily, but access to addresses in others was arbitrarily

protected

Quoted 1 line
Quoted 1 line

3) flat, unified virtual addresses (64 bit) so that pointers,

including

Quoted 5 lines
Quoted 5 lines
Quoted 3 lines
Quoted 1 line

inter-space pointers, have the same representation in all spaces

Hmm, will it be possible to have UML?

If by UML you mean Uniform Modelling Language, I don't understand where

the

Quoted 1 line
Quoted 1 line

protection model has any impact. UML models flow, permissions are

somewhat

Quoted 5 lines
Quoted 2 lines

superimposed, just like file permissions in a UML model on any machine.

I meant "User Mode Linux" == linux running under linux. Someone probably has an URL.

Sorry - I plead ignorance :-) As the protection is recursive and transitive, I suppose that you could do this. When the UMK (user mode kernel) went to change the "real" machine it would get a protection fault that would be handled by the KMK, emulating the effect. Getting it right and also performant would be tricky though - is UML a necessary feature?

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

9) Hardware interrupts are involuntary inter-space calls. They do

not

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

require locking (assuming the handler is re-entrant - and if not

then

Quoted 2 lines
Quoted 2 lines

only

Quoted 1 line
Quoted 1 line

from themselves), nor task switch, nor disabling other interrupts.

The

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

handler runs in the stack of whoever got interrupted, which

(depending

Quoted 2 lines
Quoted 2 lines

on

Quoted 1 line
Quoted 1 line

interrupt priorities) could be another interrupt, on an interrupt,

...

Quoted 6 lines
Quoted 6 lines

on an

Quoted 3 lines
Quoted 1 line

app, all mutually protected.

How do you implement ptrace if apps are protected from kernel?

Anybody can make all or part of themselves visible to anybody else. If

you

Quoted 2 lines
Quoted 2 lines

start up an app in your space, you can grant visibility to a debugger in another space. But this applies only to you. For example, suppose that

your

Quoted 1 line
Quoted 1 line

app calls a paranoid server DLL passing in a function, and the DLL in

turn

Quoted 1 line
Quoted 1 line

calls back your function. Then your stack will have a hunk of you (that

you

Quoted 1 line
Quoted 1 line

can see and expose to the debugger), then a hunk of DLL function

activations

Quoted 1 line
Quoted 1 line

(which are opaque to you AND the debugger unless you can talk the DLL

into

Quoted 1 line
Quoted 1 line

exposing itself), and then another hunk of you again (and again visible

to

Quoted 4 lines
Quoted 4 lines

you and the debugger). The DLL and you (and your debugger) are mutually protected.

To do this on a conventional system requires that the DLL runs as a

server

Quoted 1 line
Quoted 1 line

process, and getting it to do something for you requires a roundtrip

through

Quoted 6 lines
Quoted 2 lines

the dispatcher. For us it's a simple subroutine call, just as if the DLL were un-paranoid and had been linked into your code. Clearer?

Strange system.... If an application does not grant kernel access to its space, how is kernel supposed to do its job? For example, that "paranoid DLL" becomes unswappable, then?

Pretection is in the *virtual* space, not physical. The physical-page manager (who has the TLB and underlying mapping tables in its space) can see and deal with any physical address, which in turn has the usual aliasing relationship with virtual addresses. Of course, physical is just one of the virtual spaces (and is distinguished solely by the one-to-one virtual-physical mapping). So the protection can be penetrated by anyone who can see the underlying physical page - but that's always true.

Quoted 2 lines

If you have "enough" paranoid DLLs, you can then bring the machine down due to lack of real memory :-).

No.

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

10) Drivers can have their own individual space(s) distinct from

those

Quoted 9 lines
Quoted 6 lines

of

Quoted 3 lines
Quoted 1 line

the kernel and the apps. Buggy drivers cannot crash the kernel.

Well... buggy drivers can usually program DMA to do crashing for them.

Nope. The DMA has the same permissions as the driver that starts it.

So normal PCI cards are not allowed, or do you have some kind of IOMMU?

Nope. Apertures in the memory controller. We looked at doing I/O in virtual, but the extra traffic on the pins was too expensive (although it did permit I/O onto paged memory). But it's easy to give each IO its own aperture base, just by adding a few more high-order bits to the address, and the controller can make sure that each DMA stays in its aperture. Sort of a poor-man's MMU :-)

Quoted 5 lines
Quoted 5 lines
Quoted 3 lines

That would be pretty big rewrite...

Anyway, I believe you *do* want linux on it, if only as a test load.

We definitely want Linux. The question is whether Linux will want the

result

Quoted 1 line
Quoted 1 line

of our port, or (in finer detail) what parts could we do in what way to

be

Quoted 3 lines
Quoted 1 line

useful to other people.

If most changes are in arch/, it should be acceptable...

I fear that it might be more extensive than that :-)

Ivan

#5 lore raw

Hi!

Quoted 8 lines
Quoted 2 lines

I meant "User Mode Linux" == linux running under linux. Someone probably has an URL.

Sorry - I plead ignorance :-) As the protection is recursive and transitive, I suppose that you could do this. When the UMK (user mode kernel) went to change the "real" machine it would get a protection fault that would be handled by the KMK, emulating the effect. Getting it right and also performant would be tricky though - is UML a necessary feature?

No. Its just "nice to have", and it does not support too many architectures.

Quoted 11 lines
Quoted 3 lines

Strange system.... If an application does not grant kernel access to its space, how is kernel supposed to do its job? For example, that "paranoid DLL" becomes unswappable, then?

Pretection is in the *virtual* space, not physical. The physical-page manager (who has the TLB and underlying mapping tables in its space) can see and deal with any physical address, which in turn has the usual aliasing relationship with virtual addresses. Of course, physical is just one of the virtual spaces (and is distinguished solely by the one-to-one virtual-physical mapping). So the protection can be penetrated by anyone who can see the underlying physical page - but that's always true.

Aha, so some part of kernel exist that has "absolute right". Ok, now I can imagine that it can work.

Quoted 3 lines
Quoted 1 line

If most changes are in arch/, it should be acceptable...

I fear that it might be more extensive than that :-)

Well, make patch and lets see... That means that 2.8 needs to be your target. If impact outside of arch is not "total rewrite", you might have a chance. If it is "total rewrite".... well you just need to be very clever.
Pavel

Signature · 2 lines
When do you have a heart between your knees?
[Johanka's followup: and *two* hearts?]
#6 lore raw

----- Original Message -----
From: "Pavel Machek" <[email protected]>
To: "Ivan Godard" <[email protected]>
Cc: "Linux Kernel Mailing List" <[email protected]>
Sent: Sunday, March 28, 2004 10:54 AM
Subject: Re: Kernel support for peer-to-peer protection models...

Quoted 8 lines

Hi!

Quoted 6 lines
Quoted 3 lines

Strange system.... If an application does not grant kernel access to its space, how is kernel supposed to do its job? For example, that "paranoid DLL" becomes unswappable, then?

Pretection is in the *virtual* space, not physical. The physical-page manager (who has the TLB and underlying mapping tables in its space) can

see

Quoted 2 lines
Quoted 2 lines

and deal with any physical address, which in turn has the usual aliasing relationship with virtual addresses. Of course, physical is just one of

the

Quoted 2 lines
Quoted 2 lines

virtual spaces (and is distinguished solely by the one-to-one virtual-physical mapping). So the protection can be penetrated by anyone

who

Quoted 4 lines
Quoted 1 line

can see the underlying physical page - but that's always true.

Aha, so some part of kernel exist that has "absolute right". Ok, now I can imagine that it can work.

The "boot process" comes up with unlimited access to everything and the virt2phys direct mapped. As it forks procesess it can arbitrarily restrict their vision, transitively, and set the translation tables any way it wants. What I've sketched is one model, where a particular virtual space is used to map physical and the kernel is broken up into distinct address spaces with protection boundaries between, and each driver and app in ots own space. But you could emulate a conventoional, with the kernel and the drivers all in one space (and mutually vulnerable), or others.

Quoted 8 lines
Quoted 3 lines
Quoted 1 line

If most changes are in arch/, it should be acceptable...

I fear that it might be more extensive than that :-)

Well, make patch and lets see... That means that 2.8 needs to be your target. If impact outside of arch is not "total rewrite", you might have a chance. If it is "total rewrite".... well you just need to be very clever.

How badly would the average driver break if it did not have direct data access to kernal data structures? Calls into the kernel and direct access by the called functions are OK.

Ivan

#7 lore raw

Hi!

Quoted 4 lines
Quoted 4 lines
Quoted 3 lines
Quoted 1 line

If most changes are in arch/, it should be acceptable...

I fear that it might be more extensive than that :-)

Quoted 8 lines
Quoted 4 lines

Well, make patch and lets see... That means that 2.8 needs to be your target. If impact outside of arch is not "total rewrite", you might have a chance. If it is "total rewrite".... well you just need to be very clever.

How badly would the average driver break if it did not have direct data access to kernal data structures? Calls into the kernel and direct access by the called functions are OK.

Kernel likes to pass it pointers to internal data structures. And drivers will walk over pointers in those structures pretty often.

Actually I'm not so sure. Perhaps for simple drivers something like that would be possible..

Pavel

Signature · 2 lines
When do you have a heart between your knees?
[Johanka's followup: and *two* hearts?]