Skip to contentSkip to conversation

Virtual Media

DISCUSSION Matt Spinler 6 participants 10 messages Jul 18, 2018 – Dec 10, 2018

lore
Share & tools
lore
#1 lore raw

Hi,

Does anyone know if supporting multiple virtual media instances at once is ever a required/useful customer use case? If not, we will just plan on supporting 1 at a time.

Thanks,

Matt

Quoted 15 lines · Matt Spinler

Hi,

We are preparing to start working on virtual media, as in remote mass storage.

I saw there was a discussion about it at the hackathon,
https://docs.google.com/document/d/1lvK2o4-mMHQ5R0rs1gaB71SKw0dCAlJjyxIqh5azm3k.

Has anyone done any more work on this yet, or plan to?

I hope to send out a high level design for RFC soon, or people can chime in now with any ideas they have for it.

Thanks!
Matt

#2 lore raw
In reply to Matt Spinler

Hi Matt & all,

Quoted 2 lines

I hope to send out a high level design for RFC soon, or people can chime in now with any ideas they have for it.

So, as part of working on that design, I've been doing some prototyping with the virtual media code, and just got some experiments working yesterday, that expose a file from my local browser to the host machine (as a USB mass-storage device).

The overall architecture looks like this:

- the BMC kernel has a network block device (nbd) driver; this provides a block device, where read & write operations on the block device end up being sent as requests over a socket interface.

- the nbd infrastructure allows us to use a unix domain socket (rather than a network socket) for that interface.

- so, I wrote a little proxy that does a some setup and then just forwards data between that unix domain socket and stdin/stdout.

- Using a websocket proxy, we can connect that process' stdio to a websocket.

- I wrote a little javascript nbd server to connect to that websocket from a browser session, and serve nbd read requests by reading from a local file. This gives us a block device on the BMC, that's backed by a file accessed by the browser.

- On the BMC, we create a USB mass-storage gadget (using the ASPEED UDC device), that uses the nbd device as the underlying storage.

Using that, I was able to forward an .iso image from my webbrowser on one side of the country to a machine in our lab on the other. Speed wasn't fantastic, but should be fine for most cases.

Let me know what you think. The main caveat of this is that this method isn't able to serve writeable images, as sensible browsers won't allow served javascript code to modify files.

If you're curious, here's the prototype code for the nbd channel (proxy
+ javascript server):

https://github.com/jk-ozlabs/jsnbd

This isn't necessarily the code that we'll want to run on OpenBMC, more a proof-of-concept to make sure that all the parts work.

In order to implement this on OpenBMC, we'd still need a few things:

- integrating the nbd-proxy into the proper http server framework

- integration into the actual OpenBMC web UI

- adding hooks to automatically start/stop the USB mass-storage gadget on nbd session setup

Cheers,

Jeremy

#3 lore raw
Quoted 4 lines · Jeremy Kerr

Hi Matt & all,

Quoted 2 lines

I hope to send out a high level design for RFC soon, or people can chime in now with any ideas they have for it.

In past BMC implementations, we offered up to four virtual devices to be mounted remotely. I can't remember if this included the one used for firmware updates or not. But we will probably need more than one.

Quoted 20 lines

So, as part of working on that design, I've been doing some prototyping with the virtual media code, and just got some experiments working yesterday, that expose a file from my local browser to the host machine (as a USB mass-storage device).

The overall architecture looks like this:

- the BMC kernel has a network block device (nbd) driver; this provides a block device, where read & write operations on the block device end up being sent as requests over a socket interface.

- the nbd infrastructure allows us to use a unix domain socket (rather than a network socket) for that interface.

- so, I wrote a little proxy that does a some setup and then just forwards data between that unix domain socket and stdin/stdout.

- Using a websocket proxy, we can connect that process' stdio to a websocket.

Quoted 7 lines

- I wrote a little javascript nbd server to connect to that websocket from a browser session, and serve nbd read requests by reading from a local file. This gives us a block device on the BMC, that's backed by a file accessed by the browser.

- On the BMC, we create a USB mass-storage gadget (using the ASPEED UDC device), that uses the nbd device as the underlying storage.

That sounds pretty good from a high level. Nice job using off-the-self parts.

Quoted 7 lines

Using that, I was able to forward an .iso image from my webbrowser on one side of the country to a machine in our lab on the other. Speed wasn't fantastic, but should be fine for most cases.

Let me know what you think. The main caveat of this is that this method isn't able to serve writeable images, as sensible browsers won't allow served javascript code to modify files.

RO files should really be the biggest use case for this -- host OS installs and such. I really don't know how much people will complain if we take their RW files away. We are also not requiring them to install Java, so they win one and lose one.

--Vernon

Quoted 21 lines

If you're curious, here's the prototype code for the nbd channel (proxy
+ javascript server):

https://github.com/jk-ozlabs/jsnbd

This isn't necessarily the code that we'll want to run on OpenBMC, more a proof-of-concept to make sure that all the parts work.

In order to implement this on OpenBMC, we'd still need a few things:

- integrating the nbd-proxy into the proper http server framework

- integration into the actual OpenBMC web UI

- adding hooks to automatically start/stop the USB mass-storage gadget on nbd session setup

Cheers,

Jeremy

#4 lore raw

Hi Vernon,

Quoted 1 line

In past BMC implementations, we offered up to four virtual devices to be mounted remotely. I can't remember if this included the one used for firmware updates or not. But we will probably need more than one.

That would definitely be possible with this design, we'd just need to parameterise the nbd device and USB endpoint.

However, I've personally never encountered a need for more than one virtual media device. As you say, most of this will be for host OS installs.

But if we do have a compelling case to implement multiple devices, that shouldn't be a hassle.

Quoted 1 line

That sounds pretty good from a high level. Nice job using off-the-self parts.

Thanks! Was nice to find that we didn't need to implement custom protocols & kernel interfaces for this.

Quoted 1 line

RO files should really be the biggest use case for this -- host OS installs and such. I really don't know how much people will complain if we take their RW files away.

Yes, I'd agree there. As above, I'm not sure how much this would be used. The one use-case I can imagine is a small writeable filesystem for saving debug logs (in our case, from petitboot). However, we also have actual host network available at that point...

It *could* be possible to do RW media in javascript by storing CoW data of modified blocks client-side (eg, in an IndexedDB). The UX isn't great for this though, as the user would need to explicitly "download" and save the modified file after use (and I guarantee that folks will forget to do that and lose data).

Quoted 1 line

We are also not requiring them to install Java,

Hooray! :)

Cheers,

Jeremy

#5 lore raw
Quoted 8 lines · Jeremy Kerr
Quoted 3 lines

RO files should really be the biggest use case for this -- host OS installs and such. I really don't know how much people will complain if we take their RW files away.

Yes, I'd agree there. As above, I'm not sure how much this would be used. The one use-case I can imagine is a small writeable filesystem for saving debug logs (in our case, from petitboot). However, we also have actual host network available at that point...

zmodem over the terminal right to the browser!

(only slightly not joking)

Signature · 2 lines
Stewart Smith
OPAL Architect, IBM.
#6 lore raw

Hi Stewart,

Quoted 6 lines
Quoted 4 lines

Yes, I'd agree there. As above, I'm not sure how much this would be used. The one use-case I can imagine is a small writeable filesystem for saving debug logs (in our case, from petitboot). However, we also have actual host network available at that point...

zmodem over the terminal right to the browser!

The browser still can't write to the local filesystem :P

Cheers,

Jeremy

#7 lore raw
Quoted 10 lines · Jeremy Kerr

Hi Stewart,

Quoted 6 lines
Quoted 4 lines

Yes, I'd agree there. As above, I'm not sure how much this would be used. The one use-case I can imagine is a small writeable filesystem for saving debug logs (in our case, from petitboot). However, we also have actual host network available at that point...

zmodem over the terminal right to the browser!

The browser still can't write to the local filesystem :P

But you can download the file and save it, which would solve the debug log problem.

Signature · 2 lines
Stewart Smith
OPAL Architect, IBM.
#8 lore raw
In reply to Jeremy Kerr

Hi Ed,

Quoted 5 lines

- the nbd infrastructure allows us to use a unix domain socket (rather than a network socket) for that interface.

- Using a websocket proxy, we can connect that process' stdio to a websocket.

As you pointed out, the Redfish schema for Virtual Media may not fit with this design, so wanted to get your thoughts on how/if we could somehow implement having the client request to mount the device via a websocket, like by calling the VirtualMedia/<device_id> api, or could we leverage some other schemas, or what would be your suggestion? Thanks!

Reference:
https://gerrit.openbmc-project.xyz/#/c/openbmc/bmcweb/+/16481/
https://gerrit.openbmc-project.xyz/#/c/openbmc/phosphor-rest-server/+/14752/

#9 lore raw
Quoted 14 lines · Adriana Kobylak

Hi Ed,

Quoted 5 lines

 - the nbd infrastructure allows us to use a unix domain socket (rather than a network socket) for that interface.

 - Using a websocket proxy, we can connect that process' stdio to a websocket.

As you pointed out, the Redfish schema for Virtual Media may not fit with this design, so wanted to get your thoughts on how/if we could somehow implement having the client request to mount the device via a websocket, like by calling the VirtualMedia/<device_id> api, or could we leverage some other schemas, or what would be your suggestion? Thanks!

Quoted 5 lines

Reference:
https://gerrit.openbmc-project.xyz/#/c/openbmc/bmcweb/+/16481/
https://gerrit.openbmc-project.xyz/#/c/openbmc/phosphor-rest-server/+/14752/

I think the root of the issue here is that the Virtual Media Redfish interface was designed to be able to mount a host from a network share, rather than proxying over a websocket.  I think in the long term, if we're going to support all the use cases we want to, we will eventually need both mechanisms, but for the moment, if we want to get something that works up quickly, and iterate, we're probably better off just continuing to make the websocket implementation work reliably.  In my head (i.e. I haven't backed this up with actual data) I see websocket as one implementation of a transport layer, with other transport layers (samba, NFS, ect) that could be supported at a later date.

I suspect the best approach here is to simply add a url handler under the existing name (/kvmws0) that does the websocket.  When and if someone goes to build out the Redfish interface for Virtual Media, we can go implement the redfish specific urls.  If you'd prefer to do it in the reverse order, that would work as well.

-Ed