Skip to content

device parts handling administration commands

Parav Pandit virtio-comment 5 versions – New

Get series with b4
b4 am [email protected]

Versions newest first · linked by author and title

  1. v5 Latest5 patches 0 repliesNew
  2. v48 patches 0 repliesSuperseded
  3. v38 patches 148 repliesSuperseded
Earlier versions · 2
VersionPostedPatchesRepliesStatus
v28 patches5 repliesSuperseded
v18 patches332 repliesSuperseded

Changes in v5 from the cover letter

- removed write recording from the device to future work
- removed device state (aka device context) tracking to future work
- simplified freeze and stopped mode, to a single mode stopped
- reworded from 'device context' to 'device parts'
- simplified the theory of operation description and command descriptions

Patches in v5 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/5admin: Add theory of operation for device partsadmin-cmds-capabilities.tex admin-cmds-device-parts.tex admin.tex+72 -00 replies
2/5admin: Extend resource objects for sr-iov group typeadmin-cmds-resource-objects.tex+11 -50 replies
3/5admin: Add admin commands for device partsadmin-cmds-device-parts.tex admin-cmds-resource-objects.tex admin.tex+252 -10 replies
4/5admin: Define common device partsadmin-cmds-device-parts.tex content.tex device-parts.tex +1 more+293 -00 replies
5/5admin: Add requirements of device parts commandsadmin-cmds-device-parts.tex conformance.tex+150 -00 replies

Reviews trailers per patch

No Reviewed-by, Acked-by or Tested-by trailers for v5 yet.

Read the v5 thread

Cover letter [PATCH v5 0/5] device parts handling administration commands

In some systems, there is a need to capture the state of all or part of a device and subsequently restore either the same device or a different one to this captured state. A group owner device can support administration commands to facilitate these get and set operations for the group member devices.

For example, a hypervisor can use the administration commands to capture parts of the device state and save the result as part of a VM snapshot. Later, the hypervisor can retrieve the snapshot and use the administration commands to restore parts of a device to resume VM operation.

As another example, these commands can be used to facilitate VM migration by the
hypervisor: one (source) hypervisor can get parts of a device and send the results to another (destination) hypervisor, which will in turn set (restore) parts of (another) device to resume the VM operation on the destination.

As always, these commands can also be used for debugging the device.

The device comprises many device parts which the driver can get and set. Administration commands are provided to either get and set all the device parts at once, or to get the device parts metadata that indicates which device parts are present, and later to get and set specific device parts.

To get and set the device parts or their metadata, the driver first creates a device parts resource object, indicating whether the object should handle get or set operations but not both simultaneously.

This series introduces administration commands for handling the device parts.

It takes inspiration from the similar idea presented at KVM Forum at [1].

Presently, administration commands are limited to the owner device handling device parts of the member device. However, in the future, this may be extended to a wider range of devices. For example, a member device may be able to handle parts of itself, or an administration device may handle parts of other devices.

Currently, this series does not define specific parts for individual device types. Such device parts will be added subsequent to this work.

This series depends on and uses the capabilities and resources infrastructure introduced several days ago in series [2].

[1] https://static.sched.com/hosted_files/kvmforum2022/3a/KVM22-Migratable-Vhost-vDPA.pdf
[2] https://lore.kernel.org/virtio-comment/[email protected]/T/#t

patch summary:
==============
patch-1 adds device parts theory of operation and overall flow patch-2 extends resource objects command for sriov group type patch-3 defines administration commands related to device parts patch-4 defines common device parts
patch-5 lists requirements for device parts administration commands

changelog:
==========
v4->v5:
- removed write recording from the device to future work
- removed device state (aka device context) tracking to future work
- simplified freeze and stopped mode, to a single mode stopped
- reworded from 'device context' to 'device parts'
- simplified the theory of operation description and command descriptions

v3->v4:
- updated commit message to be more precise for mapping
a PCI VF member device to guest virtual machine
- skipped adding msix table and pba table as they are mainly programmed by hypervisor which has the knowledge of vcpus and contains system specific encoded information which may be different on destination hypervisor
- moved vq enabled field to vq config struct so that it can be used for packed vq and split vq uniformly.
- reduce length in the field tlv from 64-bit to 32-bit to be realistic
- added missing num_entries in the feature bits tlv
- fixed copy paste error on structure for VIRTIO_DEV_CTX_VQ_SPLIT_DEV_OWN_DESC field
- corrected discard field type location and enum values
- enhanced queue context
- made device context read command more flexible for device implementations
- improved read command to be finish quickly during freeze mode to have lowest possible downtime
- improved description for query supported fields command and device context to report the length in it
- moved read context command dependency on output field
- added remaining context size bytes in read response so that after the device mode is changed to freeze, the driver can get most accurate information quickly to decide when to stop reading the context from the device
- relaxed device context formation and processing to simplify the device implementation
v2->v3:
- updated cover letter to reflect the use for passthrough and for only data path acceleration
- updated cover letter to utilize same infra for nested
- Addressed comments from Michael
- updated read context command to not depend on returned data length for closing the read context stream, instead depend on explicit read response with zero length
- fixed copy paste errors in write context command for fields description
- added device and driver normatives for {_START,_END}_MARKER fields
- wrote member VF device instead of VF device
v1->v2:
- Addressed comments from Michael and Jason
- replaced iova to page/physical address range in write recording commands
- several device specific requirements added to clarify, interaction of device reset, FLR, PCI PM and admin commands
- added device context fields query command to learn compatibility
- split device context field type range into generic and device specific
- added device context extension section to maintain backward and future compatibility
- several rewording in theory of operation
- added requirements to cover config space read/write interaction with device context commands
- added assumption about pci config space and msix table not present in device context, which can be added when hypervisor need them
v0->v1:
- enrich device context to cover device configuration layout, feature bits
- fixed alignment of device context fields
- added missing Sign-off for the joint work done with Satananda
- added link to the github issue

Parav Pandit (5):
admin: Add theory of operation for device parts
admin: Extend resource objects for sr-iov group type
admin: Add admin commands for device parts
admin: Define common device parts
admin: Add requirements of device parts commands

 admin-cmds-capabilities.tex     |  16 ++
 admin-cmds-device-parts.tex     | 469 ++++++++++++++++++++++++++++++++
 admin-cmds-resource-objects.tex |  29 +-
 admin.tex                       |  12 +-
 conformance.tex                 |   2 +
 content.tex                     |   1 +
 device-parts.tex                | 231 ++++++++++++++++
 newdevice.tex                   |  24 ++
 8 files changed, 778 insertions(+), 6 deletions(-)
 create mode 100644 admin-cmds-device-parts.tex
 create mode 100644 device-parts.tex
Signature · 1 line
2.34.1