Skip to contentSkip to conversation

Mapping memory regions on s390

DISCUSSION Dr. David Alan Gilbert 2 participants 2 messages Mar 1, 2019 – Apr 26, 2019 1 message not in the archive s390

lore
Share & tools
lore

A message that is not in the archive (sent off-list or not yet received): [email protected]

#1 lore raw
Quoted 31 lines · * David Hildenbrand ([email protected]) wrote:
Quoted 30 lines · Cornelia Huck
Quoted 26 lines · On Fri, 1 Mar 2019 14:18:21 +0100 (CET) Sebastian Ott <[email protected]>…
Quoted 11 lines · Cornelia Huck
Quoted 3 lines · On Fri, 1 Mar 2019 13:14:38 +0100 (CET) Sebastian Ott <[email protected]>…

The iomap, readb stuff is only usable in a PCI context on s390. But what is the problem here? virtio_ccw knows it's not a PCI driver - it doesn't have to use this stuff..

The device has to put the shared regions somewhere. If the virtio-pci transport is used, it's just a normal area within a BAR (IIUC), and should be handled just like normal pci memory.

Quoted 9 lines

The problem is where to put it if the virtio-ccw transport is used. The idea was to put it in its own region (a device-memory region) and allow the guest some way to discover that region (which is not within the normal system memory from the guest's POW, just as pci memory isn't). However, if we do that, we end up having the shared regions in different regions depending upon whether the pci or the ccw transport is used. Having pci use the special device memory on s390x does not really sound like a good idea, either. Nor does putting it into pci memory and accessing it from the ccw transport...

How should virtio_ccw access this memory? Only via CCW programs? In this case you don't have to put it in the guests address space. Or should standard memory instructions (lg,st) be used? In this case maybe the 1-to-1 mapping or the vmalloc space??

Quoted 8 lines
Quoted 6 lines

The idea was to use standard memory instructions (i.e. both host and guest can read from/write to the region at any time). There was also the idea to read/write this from the guest side only via ccws, for which we wouldn't need a new address space, but that would have serious drawbacks like not being able to read/write from an atomic context.

Or mapping into into user space, if that would ever be required.

The specific case this discussion started for was explicitly allowing it to be mapped all the way to user space; in virtio-fs we're mapping normal files into this space in the qemu and they're presented as DAX inside the guest kernel and eventually mapped through to the user process.

Dave

Quoted 5 lines
Signature · 3 lines
Thanks,

David / dhildenb

--
Dr. David Alan Gilbert / [email protected] / Manchester, UK

#2 lore raw
Quoted 31 lines · On Fri, 1 Mar 2019 14:50:14 +0100 David Hildenbrand <[email protected]> wrote:
Quoted 30 lines · Cornelia Huck
Quoted 26 lines · On Fri, 1 Mar 2019 14:18:21 +0100 (CET) Sebastian Ott <[email protected]>…
Quoted 11 lines · Cornelia Huck
Quoted 3 lines · On Fri, 1 Mar 2019 13:14:38 +0100 (CET) Sebastian Ott <[email protected]>…

The iomap, readb stuff is only usable in a PCI context on s390. But what is the problem here? virtio_ccw knows it's not a PCI driver - it doesn't have to use this stuff..

The device has to put the shared regions somewhere. If the virtio-pci transport is used, it's just a normal area within a BAR (IIUC), and should be handled just like normal pci memory.

Quoted 9 lines

The problem is where to put it if the virtio-ccw transport is used. The idea was to put it in its own region (a device-memory region) and allow the guest some way to discover that region (which is not within the normal system memory from the guest's POW, just as pci memory isn't). However, if we do that, we end up having the shared regions in different regions depending upon whether the pci or the ccw transport is used. Having pci use the special device memory on s390x does not really sound like a good idea, either. Nor does putting it into pci memory and accessing it from the ccw transport...

How should virtio_ccw access this memory? Only via CCW programs? In this case you don't have to put it in the guests address space. Or should standard memory instructions (lg,st) be used? In this case maybe the 1-to-1 mapping or the vmalloc space??

Quoted 9 lines
Quoted 6 lines

The idea was to use standard memory instructions (i.e. both host and guest can read from/write to the region at any time). There was also the idea to read/write this from the guest side only via ccws, for which we wouldn't need a new address space, but that would have serious drawbacks like not being able to read/write from an atomic context.

Or mapping into into user space, if that would ever be required.

Sorry about the thread necromancy here, but I'm still not quite sure how to proceed.

My understanding is that we can't use any other way to access this memory than normal read/write, or it would clash with at least one of the main use cases for this.

This leaves the problem that we support both virtio-ccw and virtio-pci devices on s390 guests. The three options I see are:

(1) use a special device memory area for ccw and normal pci memory for pci

(2) have pci use the special device memory area as well

(3) have ccw use pci memory

I don't really like any of these... (2) would make pci different on s390 than anywhere else (well, even more so than it already is), and
(3) feels wrong as well. (1) is probably most benign in practice, but would likely require us to configure devices differently depending on the transport used, which seems troublesome as well.

I also have no idea if the protected guest thing will bite us here in any way, especially as I have no access to the architecture.

Thoughts? Am I missing a fourth solution, or will we have to pick the least bad one?