blkif.h: Document protocol and existing extensions

Quoted 1 line

Quoted 1 line · Justin T. Gibbs

This patch series attempts to document the blkif PV interface and the various extensions to it that are out in the wild. I assembled this information by reviewing xend, various versions of the Linux blkif drivers, fixing bugs in the FreeBSD blkfront driver, and writing and testing a blkback driver for FreeBSD.

All but the last patch in the series is source compatible with existing drivers. This final patch adds FreeBSD's extension to allow an I/O to span multiple ring entries. The number of outstanding requests, and the max I/O size and number of segments per request, can all be negotiated. Drivers offering this extension are backwards compatible with existing drivers, but the definition of BLKIF_MAX_SEGMENTS_PER_REQUEST has been changed to reflect the new limit of 255. A global replacement of BLKIF_MAX_SEGMENTS_PER_REQUEST with BLKIF_MAX_SEGMENTS_PER_HEADER_BLOCK is all that's required to adjust drivers without the extension to this header change.

Quoted 1 line

Quoted 1 line · Justin T. Gibbs

xen/include/public/io/blkif.h +22 -0

@@ -393,6 +393,28 @@ DEFINE_RING_TYPES(blkif, struct blkif_re
 #define VDISK_REMOVABLE    0x2
 #define VDISK_READONLY     0x4
 
+/*
+ * Xen-defined major numbers for virtual block devices.
+ */
+#define XEN_IDE0_MAJOR          3
+#define XEN_IDE1_MAJOR          22
+#define XEN_SCSI_DISK0_MAJOR    8
+#define XEN_SCSI_DISK1_MAJOR    65
+#define XEN_SCSI_DISK2_MAJOR    66
+#define XEN_SCSI_DISK3_MAJOR    67
+#define XEN_SCSI_DISK4_MAJOR    68
+#define XEN_SCSI_DISK5_MAJOR    69
+#define XEN_SCSI_DISK6_MAJOR    70
+#define XEN_SCSI_DISK7_MAJOR    71
+#define XEN_SCSI_DISK8_MAJOR    128
+#define XEN_SCSI_DISK9_MAJOR    129
+#define XEN_SCSI_DISK10_MAJOR   130
+#define XEN_SCSI_DISK11_MAJOR   131
+#define XEN_SCSI_DISK12_MAJOR   132
+#define XEN_SCSI_DISK13_MAJOR   133
+#define XEN_SCSI_DISK14_MAJOR   134
+#define XEN_SCSI_DISK15_MAJOR   135
+
 #endif /* __XEN_PUBLIC_IO_BLKIF_H__ */
 
 /*

_______________________________________________
Xen-devel mailing list
[email protected]
http://lists.xensource.com/xen-devel

Quoted 1 line

Quoted 1 line · Justin T. Gibbs

xen/include/public/io/blkif.h | 106 ++++++++++++++++++++++++++++++++++++++++-

 1 files changed, 103 insertions(+), 3 deletions(-)

Note: The definition of BLKIF_MAX_SEGMENTS_PER_REQUEST has changed.
Drivers must be updated to, at minimum, use BLKIF_MAX_SEGMENTS_PER_HEADER_BLOCK, before being compatible with this header file.

This extension first appeared in FreeBSD.

Signed-off-by Justin T. Gibbs <[email protected]>

diff -r c3609ad53946 -r f5c2e866c661 xen/include/public/io/blkif.h

xen/include/public/io/blkif.h +23 -0

@@ -119,6 +119,29 @@
  *      The maximum supported size of the request ring buffer in units of
  *      machine pages.  The value must be a power of 2.
  *
+ * max-requests         <uint32_t>
+ *      Default Value:  BLKIF_MAX_RING_REQUESTS(PAGE_SIZE)
+ *      Maximum Value:  BLKIF_MAX_RING_REQUESTS(PAGE_SIZE * max-ring-pages)
+ *
+ *      The maximum number of concurrent requests supported by the backend.
+ *
+ * max-request-segments
+ *      Values:         <uint8_t>
+ *      Default Value:  BLKIF_MAX_SEGMENTS_PER_HEADER_BLOCK
+ *      Maximum Value:  255
+ *
+ *      The maximum value of blkif_request.nr_segments supported by
+ *      the backend.
+ *
+ * max-request-size
+ *      Values:         <uint32_t>
+ *      Default Value:  BLKIF_MAX_SEGMENTS_PER_HEADER_BLOCK * PAGE_SIZE
+ *      Maximum Value:  255 * PAGE_SIZE
+ *
+ *      The maximum amount of data, in bytes, that can be referenced by a
+ *      request type that accesses frontend memory (currently 
 BLKIF_OP_READ,
+ *      BLKIF_OP_WRITE, or BLKIF_OP_WRITE_BARRIER).
+ *
  *----------------------- Backend Device Identification -----------------------
  * mode

* Values: "r" (read only), "w" (writable)

xen/include/public/io/blkif.h +25 -0

@@ -235,6 +258,31 @@
  *      The size of the frontend allocated request ring buffer in units of
  *      machine pages.  The value must be a power of 2.
  *
+ * max-requests
+ *      Values:         <uint32_t>
+ *      Default Value:  BLKIF_MAX_RING_REQUESTS(PAGE_SIZE)
+ *      Maximum Value:  BLKIF_MAX_RING_REQUESTS(PAGE_SIZE * max-ring_pages)
+ *
+ *      The maximum number of concurrent requests that will be issued by
+ *      the frontend.
+ *
+ * max-request-segments
+ *      Values:         <uint8_t>
+ *      Default Value:  BLKIF_MAX_SEGMENTS_PER_HEADER_BLOCK
+ *      Maximum Value:  MIN(255, backend/max-request-segments)
+ *
+ *      The maximum value the frontend will set in the
+ *      blkif_request.nr_segments field.
+ *
+ * max-request-size
+ *      Values:         <uint32_t>
+ *      Default Value:  BLKIF_MAX_SEGMENTS_PER_HEADER_BLOCK * PAGE_SIZE
+ *      Maximum Value:  max-request-segments * PAGE_SIZE
+ *
+ *      The maximum amount of data, in bytes, that can be referenced by
+ *      a request type that accesses frontend memory (currently 
 BLKIF_OP_READ,
+ *      BLKIF_OP_WRITE, or BLKIF_OP_WRITE_BARRIER).
+ *
  *------------------------- Virtual Device Properties -------------------------
  *

* device-type

xen/include/public/io/blkif.h +1 -1

@@ -392,11 +440,21 @@
 #define BLKIF_OP_DISCARD           5
 
 /*
- * Maximum scatter/gather segments per request.
+ * Maximum scatter/gather segments associated with a request header block.
  * This is carefully chosen so that sizeof(blkif_ring_t) <= PAGE_SIZE.
  * NB. This could be 12 if the ring indexes weren't stored in the same 

page.
*/

xen/include/public/io/blkif.h +54 -2

-#define BLKIF_MAX_SEGMENTS_PER_REQUEST 11
+#define BLKIF_MAX_SEGMENTS_PER_HEADER_BLOCK  11
+
+/*
+ * Maximum scatter/gather segments associated with a segment block.
+ */
+#define BLKIF_MAX_SEGMENTS_PER_SEGMENT_BLOCK 14
+
+/*
+ * Maximum scatter/gather segments per request (header + segment blocks).
+ */
+#define BLKIF_MAX_SEGMENTS_PER_REQUEST 255
 
 /*
  * NB. first_sect and last_sect in blkif_request_segment, as well as

@@ -411,9 +469,25 @@ struct blkif_request_segment {
     /* @last_sect: last sector in frame to transfer (inclusive).     */
     uint8_t     first_sect, last_sect;
 };
+typedef struct blkif_request_segment blkif_request_segment_t;
 
 /*
  * Starting ring element for any I/O request.
+ *
+ * One or more segment blocks can be inserted into the request ring
+ * just after a blkif_request_t, allowing requests to operate on
+ * up to BLKIF_MAX_SEGMENTS_PER_REQUEST.
+ *
+ * BLKIF_SEGS_TO_BLOCKS() can be used on blkif_requst.nr_segments
+ * to determine the number of contiguous ring entries associated
+ * with this request.
+ *
+ * Note:  Due to the way Xen request rings operate, the producer and
+ *        consumer indices of the ring must be incremented by the
+ *        BLKIF_SEGS_TO_BLOCKS() value of the associated request.
+ *        (e.g. a response to a 3 ring entry request must also consume
+ *        3 entries in the ring, even though only the first ring entry
+ *        in the response has any data.)
  */
 struct blkif_request {
     uint8_t        operation;    /* BLKIF_OP_???                         */

@@ -421,11 +495,22 @@ struct blkif_request {
     blkif_vdev_t   handle;       /* only for read/write requests         */
     uint64_t       id;           /* private guest value, echoed in resp  */
     blkif_sector_t sector_number;/* start sector idx on disk (r/w only)  */
-    struct blkif_request_segment seg[BLKIF_MAX_SEGMENTS_PER_REQUEST];
+    blkif_request_segment_t seg[BLKIF_MAX_SEGMENTS_PER_HEADER_BLOCK];
 };
 typedef struct blkif_request blkif_request_t;
 
 /*
+ * A segment block is a ring request structure that contains only
+ * segment data.
+ *
+ * sizeof(struct blkif_segment_block) <= sizeof(struct blkif_request)
+ */
+struct blkif_segment_block {
+    blkif_request_segment_t seg[BLKIF_MAX_SEGMENTS_PER_SEGMENT_BLOCK];
+};
+typedef struct blkif_segment_block blkif_segment_block_t;
+
+/*
  * Cast to this structure when blkif_request.operation == BLKIF_OP_DISCARD
  * sizeof(struct blkif_request_discard) <= sizeof(struct blkif_request)
  */

@@ -462,6 +547,21 @@ typedef struct blkif_response blkif_resp
  */
 DEFINE_RING_TYPES(blkif, struct blkif_request, struct blkif_response);
 
+/*
+ * Index to, and treat as a segment block, an entry in the ring.
+ */
+#define BLKRING_GET_SEG_BLOCK(_r, _idx)                                 \
+    (((blkif_segment_block_t *)RING_GET_REQUEST(_r, _idx))->seg)
+
+/*
+ * The number of ring request blocks required to handle an I/O
+ * request containing _segs segments.
+ */
+#define BLKIF_SEGS_TO_BLOCKS(_segs)                                     \
+    ((((_segs - BLKIF_MAX_SEGMENTS_PER_HEADER_BLOCK)                    \
+     + (BLKIF_MAX_SEGMENTS_PER_SEGMENT_BLOCK - 1))                      \
+    / BLKIF_MAX_SEGMENTS_PER_SEGMENT_BLOCK) + /*header_block*/1)
+
 #define VDISK_CDROM        0x1
 #define VDISK_REMOVABLE    0x2
 #define VDISK_READONLY     0x4

_______________________________________________
Xen-devel mailing list
[email protected]
http://lists.xensource.com/xen-devel

xen/include/public/io/blkif.h line 222

Quoted 9 lines

Quoted 1 line

Quoted 1 line · Justin T. Gibbs

xen/include/public/io/blkif.h +4 -0

@@ -187,6 +216,25 @@
 *      The machine ABI rules governing the format of all ring request and
 *      response structures.
 *
+ * ring-page-order
+ *      Values:         <uint32_t>
+ *      Default Value:  0
+ *      Maximum Value:  MAX(ffs(max-ring-pages) - 1, max-ring-page-order)

Why not just max-ring-page-order. After all, the value is meaningless for a backend that only exposes max-ring-pages.

xen/include/public/io/blkif.h line 232

Quoted 10 lines

xen/include/public/io/blkif.h +10 -0

+ *      Notes:          1, 3
+ *
+ *      The size of the frontend allocated request ring buffer in units
+ *      of lb(machine pages). (e.g. 0 == 1 page, 1 = 2 pages, 2 == 4 pages,
+ *      etc.).
+ *
+ * ring-pages
+ *      Values:         <uint32_t>
+ *      Default Value:  1
+ *      Maximum Value:  MAX(max-ring-pages,(0x1 << max-ring-page-order))

Similarly here - just max-ring-pages should do.

Quoted 1 line

Quoted 1 line · Justin Gibbs

Quoted 24 lines · Jan Beulich

xen/include/public/io/blkif.h line 222

Quoted 9 lines

Quoted 1 line

Quoted 1 line · Justin T. Gibbs

xen/include/public/io/blkif.h +4 -0

@@ -187,6 +216,25 @@
 *      The machine ABI rules governing the format of all ring request and
 *      response structures.
 *
+ * ring-page-order
+ *      Values:         <uint32_t>
+ *      Default Value:  0
+ *      Maximum Value:  MAX(ffs(max-ring-pages) - 1, max-ring-page-order)

Why not just max-ring-page-order. After all, the value is meaningless for a backend that only exposes max-ring-pages.

Quoted 10 lines

xen/include/public/io/blkif.h +10 -0

+ *      Notes:          1, 3
+ *
+ *      The size of the frontend allocated request ring buffer in units
+ *      of lb(machine pages). (e.g. 0 == 1 page, 1 = 2 pages, 2 == 4 pages,
+ *      etc.).
+ *
+ * ring-pages
+ *      Values:         <uint32_t>
+ *      Default Value:  1
+ *      Maximum Value:  MAX(max-ring-pages,(0x1 << max-ring-page-order))

Quoted 1 line

Similarly here - just max-ring-pages should do.

This patch documents existing extensions. For a new driver to properly negotiate a
multi-page ring with a Dom0 on EC2 (ring-pages) or a Citrix Dom0/guest (ring-page-order),
you must use the XenBus node names as I've defined them here.

Quoted 27 lines

Quoted 1 line

Quoted 1 line · Justin Gibbs

Quoted 24 lines · Jan Beulich

xen/include/public/io/blkif.h line 222

Quoted 9 lines

Quoted 1 line

Quoted 1 line · Justin T. Gibbs

xen/include/public/io/blkif.h +4 -0

@@ -187,6 +216,25 @@
 *      The machine ABI rules governing the format of all ring request and
 *      response structures.
 *
+ * ring-page-order
+ *      Values:         <uint32_t>
+ *      Default Value:  0
+ *      Maximum Value:  MAX(ffs(max-ring-pages) - 1, max-ring-page-order)

Why not just max-ring-page-order. After all, the value is meaningless for a backend that only exposes max-ring-pages.

Quoted 10 lines

xen/include/public/io/blkif.h +10 -0

+ *      Notes:          1, 3
+ *
+ *      The size of the frontend allocated request ring buffer in units
+ *      of lb(machine pages). (e.g. 0 == 1 page, 1 = 2 pages, 2 == 4 pages,
+ *      etc.).
+ *
+ * ring-pages
+ *      Values:         <uint32_t>
+ *      Default Value:  1
+ *      Maximum Value:  MAX(max-ring-pages,(0x1 << max-ring-page-order))

Quoted 7 lines

Quoted 1 line

Similarly here - just max-ring-pages should do.

This patch documents existing extensions. For a new driver to properly negotiate a
multi-page ring with a Dom0 on EC2 (ring-pages) or a Citrix Dom0/guest (ring-page-order),
you must use the XenBus node names as I've defined them here.

That wasn't my point. I was exclusively asking the maximum values to get simplified.

Jan

Quoted 1 line

Quoted 1 line

Quoted 1 line · Justin T. Gibbs

These being mostly meaningless to non-Linux, I don't think they belong here.

Jan

Quoted 1 line

Quoted 1 line · Justin Gibbs

However,
if this is the rule, both types of "max ring size" values are "in effect" even if a back-end
does not provide them both. So how do you resolve the conflict? A fully interoperable
front should allocate the largest possible ring and advertise that size through both
mechanisms in a fully consistent manner. That's what I was trying to indicate by
writing the spec this way.

Quoted 1 line

NAK. No backwards incompatible changes allowed in public headers.

Sorry for the slow reply on this. I've been experimenting with ways to keep legacy
source compatibility. After trying lots of things, testing the impact on an existing blkfront
and blkback implementation, I think the best two options are:

1. Version the header file and require consumers to declare the interface
version
they are using. If the version isn't declared, the default, legacy,
"version 1.0" will
be in effect.

Positives: No change in or constant naming conventions. Data
structures and
constants for new features are properly hidden from
legacy implementations.
Negatives: Messy #ifdefs

+/*
+ * Xen-defined major numbers for virtual block devices.
+ */

o Document the XenBus nodes used in this protocol

+ * params
+ * Values: string

+ * virtual-device
+ * Values: <uint16_t> (XEN_*_MAJOR << 8 | Minor)

Quoted 9 lines

Quoted 1 line

Quoted 1 line · Justin T. Gibbs

xen/include/public/io/blkif.h | 106 ++++++++++++++++++++++++++++++++++++++++-

1 files changed, 103 insertions(+), 3 deletions(-)

Note: The definition of BLKIF_MAX_SEGMENTS_PER_REQUEST has changed.
Drivers must be updated to, at minimum, use BLKIF_MAX_SEGMENTS_PER_HEADER_BLOCK, before being compatible with this header file.

NAK. No backwards incompatible changes allowed in public headers.

Quoted 15 lines

Quoted 1 line

Quoted 1 line · Keir Fraser

Quoted 11 lines · Justin Gibbs

1. Version the header file and require consumers to declare the interface
version
they are using. If the version isn't declared, the default, legacy,
"version 1.0" will
be in effect.

Positives: No change in or constant naming conventions. Data
structures and
constants for new features are properly hidden from
legacy implementations.
Negatives: Messy #ifdefs

Quoted 3 lines

We already have this. See use of __XEN_INTERFACE_VERSION__/__XEN_LATEST_INTERFACE_VERSION__ in the public headers.

Hmm, I would think these should specifically not be used in the io/ subtree - those aren't definitions of the interface to Xen, but ones shared between the respective backends and frontends. Each interface is (apart from its relying on the ring definitions) entirely self contained.

Jan

Quoted 1 line

Quoted 1 line · Keir Fraser

Quoted 14 lines · Justin Gibbs

Quoted 1 line

NAK. No backwards incompatible changes allowed in public headers.

Sorry for the slow reply on this. I've been experimenting with ways to keep legacy
source compatibility. After trying lots of things, testing the impact on an existing blkfront
and blkback implementation, I think the best two options are:

1. Version the header file and require consumers to declare the interface
version
they are using. If the version isn't declared, the default, legacy,
"version 1.0" will
be in effect.

Quoted 5 lines

Positives: No change in or constant naming conventions. Data
structures and
constants for new features are properly hidden from
legacy implementations.
Negatives: Messy #ifdefs

We already have this. See use of __XEN_INTERFACE_VERSION__/__XEN_LATEST_INTERFACE_VERSION__ in the public headers.

Quoted 1 line

Quoted 1 line · Justin Gibbs

Quoted 2 lines · Jan Beulich

Quoted 2 lines

Quoted 1 line

Quoted 1 line · Justin T. Gibbs

xen/include/public/io/blkif.h | 106

++++++++++++++++++++++++++++++++++++++++-

Quoted 9 lines

Quoted 7 lines

1 files changed, 103 insertions(+), 3 deletions(-)

Note: The definition of BLKIF_MAX_SEGMENTS_PER_REQUEST has changed.
Drivers must be updated to, at minimum, use BLKIF_MAX_SEGMENTS_PER_HEADER_BLOCK, before being compatible with this header file.

NAK. No backwards incompatible changes allowed in public headers.

Sorry for the slow reply on this. I've been experimenting with ways to keep legacy
source compatibility. After trying lots of things, testing the impact on an existing blkfront
and blkback implementation, I think the best two options are:

1. Version the header file and require consumers to declare the interface
version
they are using. If the version isn't declared, the default, legacy,
"version 1.0" will
be in effect.

Positives: No change in or constant naming conventions. Data
structures and
constants for new features are properly hidden from
legacy implementations.
Negatives: Messy #ifdefs

2. Create a new constant BLKIF_MAX_SEGS_PER_REQUEST, have the two other new constants (for header and segment block counts) use this
convention, and
leave BLKIF_MAX_SEGMENTS_PER_REQUEST alone (but marked DEPRECATED).

Positives: No #ifdefs. Shorter names for these constants.
Negatives: Changes the existing constant naming convention for updated
drivers. Leaves BLKIF_MAX_SEGMENTS_PER_REQUEST, a
constant
that should no longer be used, visible.

Do you have a preference or a suggested alternative?

Quoted 1 line

Quoted 1 line · Keir Fraser

I guess I think of the whole header set under xen/include/public as one unit, versioned by __XEN_INTERFACE_VERSION__. And that's how users are generally syncing with our headers -- copy them in their entirety over the top of the old ones.

Quoted 8 lines · Justin Gibbs

However, if this is the rule, both types of "max ring size" values are "in effect" even if a back-end does not provide them both. So how do you resolve the conflict? A fully interoperable front should allocate the largest possible ring and advertise that size through both mechanisms in a fully consistent manner. That's what I was trying to indicate by writing the spec this way.

Hmm, I would think a fully compatible frontend should bail (revert to single page ring) on inconsistent max-ring-pages and max-ring-page-order, if both are provided by the backend. The limit for ring-pages should always be max-ring-pages, while the one for ring-page-order should always be max-ring-page-order. Any mixture is an error, unless both values are consistent with one another.

Jan

Justin T. Gibbs writes ("[Xen-devel] [PATCH 3 of 5] blkif.h: Add definitions for virtual block device major numbers"):

Quoted 3 lines

+/*
+ * Xen-defined major numbers for virtual block devices.
+ */

Isn't this the same thing as is in
docs/misc/vbd-interface.txt
only expressed differently ?

Justin T. Gibbs writes ("[Xen-devel] [PATCH 2 of 5] blkif.h: Provide more complete documentation of the blkif interface"):

Quoted 1 line

o Document the XenBus nodes used in this protocol

Awesome. But I don't think all of these are really supposed to be part of the guest interface, so you should probably mention which ones are.

For example,

Quoted 2 lines

+ * params
+ * Values: string

this is not intended for use by guests and they should not look at it.

Quoted 3 lines

+ * virtual-device
+ * Values: <uint16_t> (XEN_*_MAJOR << 8 | Minor)

This should be a reference to docs/misc/vbd-interface.txt. But isn't this also encoded in the device path in xenstore ?

Quoted 23 lines

Quoted 1 line

Quoted 1 line · Keir Fraser

Quoted 14 lines · Justin Gibbs

Quoted 1 line

NAK. No backwards incompatible changes allowed in public headers.

Sorry for the slow reply on this. I've been experimenting with ways to keep legacy
source compatibility. After trying lots of things, testing the impact on an existing blkfront
and blkback implementation, I think the best two options are:

1. Version the header file and require consumers to declare the interface version
they are using. If the version isn't declared, the default, legacy,
"version 1.0" will
be in effect.

Quoted 5 lines

Positives: No change in or constant naming conventions. Data
structures and
constants for new features are properly hidden from
legacy implementations.
Negatives: Messy #ifdefs

Quoted 3 lines

We already have this. See use of __XEN_INTERFACE_VERSION__/__XEN_LATEST_INTERFACE_VERSION__ in the public headers.

Hmm, I would think these should specifically not be used in the io/ subtree - those aren't definitions of the interface to Xen, but ones shared between the respective backends and frontends. Each interface is (apart from its relying on the ring definitions) entirely self contained.

Jan

Quoted 1 line

Quoted 1 line · Justin T. Gibbs

Quoted 9 lines · Jan Beulich

Quoted 8 lines · Justin Gibbs

However, if this is the rule, both types of "max ring size" values are "in effect" even if a back-end does not provide them both. So how do you resolve the conflict? A fully interoperable front should allocate the largest possible ring and advertise that size through both mechanisms in a fully consistent manner. That's what I was trying to indicate by writing the spec this way.

Quoted 8 lines

Hmm, I would think a fully compatible frontend should bail (revert to single page ring) on inconsistent max-ring-pages and max-ring-page-order, if both are provided by the backend. The limit for ring-pages should always be max-ring-pages, while the one for ring-page-order should always be max-ring-page-order. Any mixture is an error, unless both values are consistent with one another.

Jan

Mandating that a front-end publish inconsistent values to the XenStore would be a mistake. Having the front-end only publish the set of nodes that are recognized by the back-end just adds code complexity with no associated increase in functionality. You actually would lose the ability to tell that the driver supports both schemes.

To clarify what I mean by that, consider the current state of back-ends in the world. They fall into 1 of 4 camps:

1. No multi-page ring support
2. "Citrix style" multi-page ring support.
3. "Red Hat style" multi-page ring support.
4. "Both styles" multi-page ring support.

In cases 1-3, one or both of the max-ring-page* nodes is not present. Per the currently proposed spec, this means that these nodes have their default values. You seem to be advocating that the front-end write the default values into its corresponding node. For cases 2 and 3, this would lead to contradictory values in the XenStore (e.g. ring-page-order=0 and ring-pages=4). This will not confuse the back-end, but could confuse a human.

For case 4, the back-end must publish both node types even though the front-end may not look at both. Since we can't avoid having both nodes, it seems reasonable to allow a front-end to publish them both too. It also avoids the additional logic to test against back-end node presence.

With all that said, I believe the spec should indicate that a fully-interoperable implementation should publish both nodes with values that match the allocated ring size. The tolerance level for a back-end publishing inconsistent nodes should be left to the implementation.

Quoted 1 line

Quoted 1 line · Justin T. Gibbs

Quoted 27 lines · Jan Beulich

Quoted 23 lines

Quoted 1 line

Quoted 1 line · Keir Fraser

Quoted 14 lines · Justin Gibbs

Quoted 1 line

NAK. No backwards incompatible changes allowed in public headers.

Sorry for the slow reply on this. I've been experimenting with ways to keep legacy
source compatibility. After trying lots of things, testing the impact on an existing blkfront
and blkback implementation, I think the best two options are:

1. Version the header file and require consumers to declare the interface version
they are using. If the version isn't declared, the default, legacy,
"version 1.0" will
be in effect.

Quoted 5 lines

Positives: No change in or constant naming conventions. Data
structures and
constants for new features are properly hidden from
legacy implementations.
Negatives: Messy #ifdefs

Quoted 3 lines

We already have this. See use of __XEN_INTERFACE_VERSION__/__XEN_LATEST_INTERFACE_VERSION__ in the public headers.

Quoted 7 lines

Hmm, I would think these should specifically not be used in the io/ subtree - those aren't definitions of the interface to Xen, but ones shared between the respective backends and frontends. Each interface is (apart from its relying on the ring definitions) entirely self contained.

Jan

The versioning required allows a driver to declare, "I am compatible with any source compatibility breaking changes up to version X of the header file". Declaring support for the latest version does not require that a driver implement the new extensions. Just one constant needs to be renamed. So I don't see this as really altering the interface between front and backends (i.e. it is not a "blkif2")

If the xen-compat.h behavior is that you can safely import the public headers so long as you do not bump __XEN_INTERFACE_VERSION__ until after you have audited and adjusted for the #ifdef guarded changes in the header files, then that is exactly what is needed in blkif.h.

--
Justin

This patch series attempts to document the blkif PV interface and the various extensions to it that are out in the wild. I assembled this information by reviewing xend, various versions of the Linux blkif drivers, fixing bugs in the FreeBSD blkfront driver, and writing and testing a blkback driver for FreeBSD.

Even with this experience, given this interface was previously only documented in source code, I'm sure there are mistakes or ommissions in what I've done here. Corrections welcome.

All but the last patch in the series is source compatible with existing drivers. This final patch adds FreeBSD's extension to allow an I/O to span multiple ring entries. The number of outstanding requests, and the max I/O size and number of segments per request, can all be negotiated. Drivers offering this extension are backwards compatible with existing drivers, but the definition of BLKIF_MAX_SEGMENTS_PER_REQUEST has been changed to reflect the new limit of 255. A global replacement of BLKIF_MAX_SEGMENTS_PER_REQUEST with BLKIF_MAX_SEGMENTS_PER_HEADER_BLOCK is all that's required to adjust drivers without the extension to this header change.

--
Justin

 xen/include/public/io/blkif.h |    9 +-
 xen/include/public/io/blkif.h |  305 ++++++++++++++++++++++++++++++++++-------
 xen/include/public/io/blkif.h |   22 +++
 xen/include/public/io/blkif.h |  101 +++++++++++-
 xen/include/public/io/blkif.h |  106 ++++++++++++++-
 5 files changed, 466 insertions(+), 77 deletions(-)

_______________________________________________
Xen-devel mailing list
[email protected]
http://lists.xensource.com/xen-devel

xen/include/public/io/blkif.h | 9 ++++-----

 1 files changed, 4 insertions(+), 5 deletions(-)

o Remove trailing whitespace.
o Remove a blank line so that a comment block is adjacent to the code it documents.

No functional changes.

Signed-off-by Justin T. Gibbs <[email protected]>

Quoted 9 lines · Ian Jackson

Justin T. Gibbs writes ("[Xen-devel] [PATCH 2 of 5] blkif.h: Provide more complete documentation of the blkif interface"):

Quoted 1 line

o Document the XenBus nodes used in this protocol

Awesome. But I don't think all of these are really supposed to be part of the guest interface, so you should probably mention which ones are.

For example,

Quoted 4 lines

Quoted 2 lines

+ * params
+ * Values: string

this is not intended for use by guests and they should not look at it.

Guests can export back-end devices. Here at Spectra we do this all the time. :-)

It would be hard to implement a blkback driver without this information. The comment at the top of your email implies you're okay with it being in this file, but that I should mark "local-end use only" nodes?

Quoted 6 lines

Quoted 3 lines

+ * virtual-device
+ * Values: <uint16_t> (XEN_*_MAJOR << 8 | Minor)

This should be a reference to docs/misc/vbd-interface.txt. But isn't this also encoded in the device path in xenstore ?

It would appear so. Should this path naming convention be mandated by this file? Since "virtual-device" exists, it seems that front-ends should read it, not parse the path, to determine this information.

Quoted 5 lines · Jan Beulich

Quoted 1 line

Quoted 1 line

Quoted 1 line · Justin T. Gibbs

These being mostly meaningless to non-Linux, I don't think they belong here.

Quoted 1 line

Jan

[blkif major number #defines deleted]

Front and backends must use some common convention for communicating this virtual device mapping. Since Linux was the first Dom0, all blkif drivers that I'm aware of, regardless of OS platform, reference the Linux values.

The mapping really only matters for HVM guests and the hand-off between a QEMU emulated device and the PV driver. To avoid confusion (e.g. name of root mount device, device names in fstab), the PV drivers export alias devices named to match the devices QEMU emulates. The only way to do this successfully is to know the major number scheme used by Dom0/QEMU.

Quoted 24 lines · Jan Beulich

xen/include/public/io/blkif.h line 222

Quoted 9 lines

Quoted 1 line

Quoted 1 line · Justin T. Gibbs

xen/include/public/io/blkif.h +4 -0

@@ -187,6 +216,25 @@
 *      The machine ABI rules governing the format of all ring request and
 *      response structures.
 *
+ * ring-page-order
+ *      Values:         <uint32_t>
+ *      Default Value:  0
+ *      Maximum Value:  MAX(ffs(max-ring-pages) - 1, max-ring-page-order)

Why not just max-ring-page-order. After all, the value is meaningless for a backend that only exposes max-ring-pages.

Quoted 10 lines

xen/include/public/io/blkif.h +10 -0

+ *      Notes:          1, 3
+ *
+ *      The size of the frontend allocated request ring buffer in units
+ *      of lb(machine pages). (e.g. 0 == 1 page, 1 = 2 pages, 2 == 4 pages,
+ *      etc.).
+ *
+ * ring-pages
+ *      Values:         <uint32_t>
+ *      Default Value:  1
+ *      Maximum Value:  MAX(max-ring-pages,(0x1 << max-ring-page-order))

Quoted 1 line

Similarly here - just max-ring-pages should do.

This patch documents existing extensions. For a new driver to properly negotiate a multi-page ring with a Dom0 on EC2 (ring-pages) or a Citrix Dom0/guest (ring-page-order), you must use the XenBus node names as I've defined them here.

-----Original Message-----
QEMU emulates primary and secondary IDE master+slave, it has no concept of a major or minor number. Why does the dom0 numbering scheme matter?

(Does this relate to the old, or possibly current, xend misbehaviour of stat()ing the dom0 device while creating a domain?)

On Linux we treat d0-d3 (however they are specified) as aliasing those 4 IDE devices (although we don't actual present alias devices but instead use UUID or LABEL based fstab etc).

Whatever non-Linux does for weird compatibility reasons here I think the right place to document it would be in docs/misc/vbd-interface.txt following the precedent of the "Notes on Linux as a guest" in that document.

Ian.

_______________________________________________
Xen-devel mailing list
[email protected]
http://lists.xensource.com/xen-devel

Quoted 1 line

Quoted 1 line · Ian Campbell

I think the decision should made by the current state of mainline Linux & BSD front and backends, i.e. we should prefer what has been upstreamed rather than private forks if possible. Does anyone know what that state is?

Quoted 5 lines

Quoted 1 line

Quoted 1 line · Ian Campbell

I think the decision should made by the current state of mainline Linux & BSD front and backends, i.e. we should prefer what has been upstreamed rather than private forks if possible. Does anyone know what that state is?

Nothing at all is upstream on the Linux side.

The versioning required allows a driver to declare, "I am compatible with any source compatibility breaking changes up to version X of the header file". Declaring support for the latest version does not require that a driver implement the new extensions. Just one constant needs to be renamed. So I don't see this as really altering the interface between front and backends (i.e. it is not a "blkif2")

If the xen-compat.h behavior is that you can safely import the public headers so long as you do not bump __XEN_INTERFACE_VERSION__ until after you have audited and adjusted for the #ifdef guarded changes in the header files, then that is exactly what is needed in blkif.h.

Quoted 14 lines

Quoted 1 line

Quoted 1 line · Justin T. Gibbs

Quoted 8 lines · Jan Beulich

Hmm, I would think these should specifically not be used in the io/ subtree - those aren't definitions of the interface to Xen, but ones shared between the respective backends and frontends. Each interface is (apart from its relying on the ring definitions) entirely self contained.

Jan

Quoted 6 lines

The versioning required allows a driver to declare, "I am compatible with any source compatibility breaking changes up to version X of the header file". Declaring support for the latest version does not require that a driver implement the new extensions. Just one constant needs to be renamed. So I don't see this as really altering the interface between front and backends (i.e. it is not a "blkif2")

Sure. But pulling in header updates should not require *any* other source changes, so long as __XEN_INTERFACE_VERSION__ isn't bumped.

My point about not using __XEN_INTERFACE_VERSION__ under io/ is that these declare protocols that the hypervisor is completely unaware of, whereas the constant really is meant to control compatibility with hypervisor changes. In particular is this or any other change to the protocols under io/ entirely unrelated to the particular hypervisor version (and hence its interface revision).

It's also unclear to me why simply giving new constants new names (instead of changing the meaning of existing ones) is such a big deal - the most strait forward solution doubtlessly is not having any conditionals in that header, and simply add new things with new, unambiguous names.

Jan

Quoted 8 lines

If the xen-compat.h behavior is that you can safely import the public headers so long as you do not bump __XEN_INTERFACE_VERSION__ until after you have audited and adjusted for the #ifdef guarded changes in the header files, then that is exactly what is needed in blkif.h.

--
Justin

Quoted 40 lines

Quoted 1 line

Quoted 1 line · Justin T. Gibbs

Quoted 27 lines · Jan Beulich

Quoted 23 lines

Quoted 1 line

Quoted 1 line · Keir Fraser

Quoted 14 lines · Justin Gibbs

Quoted 1 line

NAK. No backwards incompatible changes allowed in public headers.

Sorry for the slow reply on this. I've been experimenting with ways to keep legacy
source compatibility. After trying lots of things, testing the impact on an existing blkfront
and blkback implementation, I think the best two options are:

1. Version the header file and require consumers to declare the interface version
they are using. If the version isn't declared, the default, legacy,
"version 1.0" will
be in effect.

Quoted 5 lines

Positives: No change in or constant naming conventions. Data
structures and
constants for new features are properly hidden from
legacy implementations.
Negatives: Messy #ifdefs

Quoted 3 lines

We already have this. See use of __XEN_INTERFACE_VERSION__/__XEN_LATEST_INTERFACE_VERSION__ in the public headers.

Quoted 7 lines

Hmm, I would think these should specifically not be used in the io/ subtree - those aren't definitions of the interface to Xen, but ones shared between the respective backends and frontends. Each interface is (apart from its relying on the ring definitions) entirely self contained.

Jan

Quoted 6 lines

The versioning required allows a driver to declare, "I am compatible with any source compatibility breaking changes up to version X of the header file". Declaring support for the latest version does not require that a driver implement the new extensions. Just one constant needs to be renamed. So I don't see this as really altering the interface between front and backends (i.e. it is not a "blkif2")

Sure. But pulling in header updates should not require *any* other source changes, so long as __XEN_INTERFACE_VERSION__ isn't bumped.

It's also unclear to me why simply giving new constants new names (instead of changing the meaning of existing ones) is such a big deal - the most strait forward solution doubtlessly is not having any conditionals in that header, and simply add new things with new, unambiguous names.

Quoted 36 lines

Quoted 1 line

Quoted 1 line · Justin T. Gibbs

Quoted 9 lines · Jan Beulich

Quoted 8 lines · Justin Gibbs

However, if this is the rule, both types of "max ring size" values are "in effect" even if a back-end does not provide them both. So how do you resolve the conflict? A fully interoperable front should allocate the largest possible ring and advertise that size through both mechanisms in a fully consistent manner. That's what I was trying to indicate by writing the spec this way.

Quoted 8 lines

Hmm, I would think a fully compatible frontend should bail (revert to single page ring) on inconsistent max-ring-pages and max-ring-page-order, if both are provided by the backend. The limit for ring-pages should always be max-ring-pages, while the one for ring-page-order should always be max-ring-page-order. Any mixture is an error, unless both values are consistent with one another.

Jan

Mandating that a front-end publish inconsistent values to the XenStore would be a mistake. Having the front-end only publish the set of nodes that are recognized by the back-end just adds code complexity with no associated increase in functionality. You actually would lose the ability to tell that the driver supports both schemes.

To clarify what I mean by that, consider the current state of back-ends in the world. They fall into 1 of 4 camps:

1. No multi-page ring support
2. "Citrix style" multi-page ring support.
3. "Red Hat style" multi-page ring support.
4. "Both styles" multi-page ring support.

Quoted 7 lines

In cases 1-3, one or both of the max-ring-page* nodes is not present. Per the currently proposed spec, this means that these nodes have their default values. You seem to be advocating that the front-end write the default values into its corresponding node. For cases 2 and 3, this would lead to contradictory values in the XenStore (e.g. ring-page-order=0 and ring-pages=4). This will not confuse the back-end, but could confuse a human.

No, that's not what I intended to propose. Instead, I'd see the frontend write consistent values (and best always write both node flavors) if at least one of the protocols is supported by the backend.

I'm only suggesting that the frontend should not use either multi- page rings at all if it finds both backend node flavors present yet having inconsistent values (i.e. absence of either [but not both] values does *not* count as inconsistent, and hence a single absent node not implicitly assuming its default value).

Quoted 26 lines · Justin Gibbs

Quoted 24 lines · Jan Beulich

xen/include/public/io/blkif.h line 222

Quoted 9 lines

Quoted 1 line

Quoted 1 line · Justin T. Gibbs

xen/include/public/io/blkif.h +4 -0

@@ -187,6 +216,25 @@
 *      The machine ABI rules governing the format of all ring request and
 *      response structures.
 *
+ * ring-page-order
+ *      Values:         <uint32_t>
+ *      Default Value:  0
+ *      Maximum Value:  MAX(ffs(max-ring-pages) - 1, max-ring-page-order)

Why not just max-ring-page-order. After all, the value is meaningless for a backend that only exposes max-ring-pages.

Quoted 10 lines

xen/include/public/io/blkif.h +10 -0

+ *      Notes:          1, 3
+ *
+ *      The size of the frontend allocated request ring buffer in units
+ *      of lb(machine pages). (e.g. 0 == 1 page, 1 = 2 pages, 2 == 4 pages,
+ *      etc.).
+ *
+ * ring-pages
+ *      Values:         <uint32_t>
+ *      Default Value:  1
+ *      Maximum Value:  MAX(max-ring-pages,(0x1 << max-ring-page-order))

Quoted 5 lines

Quoted 1 line

Similarly here - just max-ring-pages should do.

This patch documents existing extensions. For a new driver to properly negotiate a multi-page ring with a Dom0 on EC2 (ring-pages) or a Citrix Dom0/guest (ring-page-order), you must use the XenBus node names as I've defined them here.

Can we pick one and mark it as preferred and the other deprecated or similar? Perhaps backends will have to support both for the foreseeable future but we should make it possible for frontends to just pick one if that's what they want while nudging them in the direction of all picking the same one.

I think the decision should made by the current state of mainline Linux & BSD front and backends, i.e. we should prefer what has been upstreamed rather than private forks if possible. Does anyone know what that state is?

I wasn't aware of this file. After a brief look, it seems there is information missing from both vbd-interface.txt and blkif.h. Per vbd-interface.txt, there is also an error in blkif.h. The "virtual-device" node must tolerate 32bit integers.

I'll fix the size specification for the "virtual-device" node, but how should I reconcile blkif.h and vbd-interface.txt. I hate information duplication since one version is invariably stale.

Quoted 5 lines

Justin T. Gibbs writes ("[Xen-devel] [PATCH 2 of 5] blkif.h: Provide more complete documentation of the blkif interface"):

Quoted 2 lines

+ * params
+ * Values: string

this is not intended for use by guests and they should not look at it.

Guests can export back-end devices. Here at Spectra we do this all the time. :-)

It would be hard to implement a blkback driver without this information. The comment at the top of your email implies you're okay with it being in this file, but that I should mark "local-end use only" nodes?

Quoted 5 lines

Quoted 2 lines

+ * virtual-device
+ * Values: <uint16_t> (XEN_*_MAJOR << 8 | Minor)

This should be a reference to docs/misc/vbd-interface.txt. But isn't this also encoded in the device path in xenstore ?

It would appear so. Should this path naming convention be mandated by this file? Since "virtual-device" exists, it seems that front-ends should read it, not parse the path, to determine this information.

Justin T. Gibbs writes ("Re: [Xen-devel] [PATCH 3 of 5] blkif.h: Add definitions for virtual block device major numbers"):

Quoted 8 lines

I wasn't aware of this file. After a brief look, it seems there is information missing from both vbd-interface.txt and blkif.h. Per vbd-interface.txt, there is also an error in blkif.h. The "virtual-device" node must tolerate 32bit integers.

I'll fix the size specification for the "virtual-device" node, but how should I reconcile blkif.h and vbd-interface.txt. I hate information duplication since one version is invariably stale.

I wrote vbd-interface.txt and I think it's largely accurate. I wasn't aware of any duplication with blkif.h. If there are differences then we will have to discuss them here I think, or read the code.

Ian.

Quoted 4 lines

-----Original Message-----
QEMU emulates primary and secondary IDE master+slave, it has no concept of a major or minor number. Why does the dom0 numbering scheme matter?

For Windows PV drivers I parse the vbd number so I can extract the disk number which is then present as the scsi target id. I'd prefer if vbd-interface.txt remained the canonical place where the numbering scheme is specified though. Putting definitions into blkif.h just weakens that position.

The upstreamed Linux driver supports 15 SCSI major numbers. Are 13 of these the "deprecated high-numbered" ones, or were there more than 15 at some point?

Justin T. Gibbs writes ("Re: [Xen-devel] [PATCH 3 of 5] blkif.h: Add definitions for virtual block device major numbers"):

Quoted 3 lines

The upstreamed Linux driver supports 15 SCSI major numbers. Are 13 of these the "deprecated high-numbered" ones, or were there more than 15 at some point?

My understanding was that current upstream Linux does not ever "steal" scsi major numbers for xen vbds. If this is wrong then so is my document :-).

Quoted 33 lines · Ian Campbell

Quoted 26 lines · Justin Gibbs

Quoted 24 lines · Jan Beulich

xen/include/public/io/blkif.h line 222

Quoted 9 lines

Quoted 1 line

Quoted 1 line · Justin T. Gibbs

xen/include/public/io/blkif.h +4 -0

@@ -187,6 +216,25 @@
 *      The machine ABI rules governing the format of all ring request and
 *      response structures.
 *
+ * ring-page-order
+ *      Values:         <uint32_t>
+ *      Default Value:  0
+ *      Maximum Value:  MAX(ffs(max-ring-pages) - 1, max-ring-page-order)

Why not just max-ring-page-order. After all, the value is meaningless for a backend that only exposes max-ring-pages.

Quoted 10 lines

xen/include/public/io/blkif.h +10 -0

+ *      Notes:          1, 3
+ *
+ *      The size of the frontend allocated request ring buffer in units
+ *      of lb(machine pages). (e.g. 0 == 1 page, 1 = 2 pages, 2 == 4 pages,
+ *      etc.).
+ *
+ * ring-pages
+ *      Values:         <uint32_t>
+ *      Default Value:  1
+ *      Maximum Value:  MAX(max-ring-pages,(0x1 << max-ring-page-order))

Quoted 5 lines

Quoted 1 line

Similarly here - just max-ring-pages should do.

This patch documents existing extensions. For a new driver to properly negotiate a multi-page ring with a Dom0 on EC2 (ring-pages) or a Citrix Dom0/guest (ring-page-order), you must use the XenBus node names as I've defined them here.

Quoted 5 lines

Can we pick one and mark it as preferred and the other deprecated or similar? Perhaps backends will have to support both for the foreseeable future but we should make it possible for frontends to just pick one if that's what they want while nudging them in the direction of all picking the same one.

History says that back-ends are updated more slowly than front-ends. For example, my fixes to QEMU's serial port emulation that have been in Xen's mainline for ~2 years are still not widely deployed on EC2. Folks running FreeBSD there are using an ugly hack to FreeBSD's serial driver so they can get console output.

So, if you want your front-ends to have optimal performance regardless of what cloud or appliance they are running on, the two types will have to be supported for some time. Neither is better than the other, just different.

The opposite is true for back-ends. If your customer controls the guest environment and chooses not upgrade, you can't kill support for the variant they use.

Quoted 4 lines

I think the decision should made by the current state of mainline Linux & BSD front and backends, i.e. we should prefer what has been upstreamed rather than private forks if possible. Does anyone know what that state is?

The state is a bit embarrassing on the BSD side. FreeBSD has had a multi-page ring extension since October of 2010. Unfortunately, I wrote this extension before stumbling upon the Citrix blkif patch or the extension being used on EC2. It is closest to the EC2 implementation, but not quite compatible. The saving grace is that I don't know of any deployments of this back-end outside of Spectra, and we have not shipped it to customers, so I plan to upstream block front and back drivers that only implement the Citrix and EC2 style extension once blkif.h settles. A preliminary version of these changes went out for review a couple weeks ago.

Quoted 7 lines

The state is a bit embarrassing on the BSD side. FreeBSD has had a multi-page ring extension since October of 2010. Unfortunately, I wrote this extension before stumbling upon the Citrix blkif patch or the extension being used on EC2. It is closest to the EC2 implementation, but not quite compatible. The saving grace is that I don't know of any deployments of this back-end outside of Spectra, and we have not shipped it to customers, so I plan to upstream block front and back drivers that

Quoted 1 line

only implement the Citrix and EC2 style extension once blkif.h settles. A

Quoted 2 lines

preliminary version of these changes went out for review a couple weeks ago.

Cool, thanks.

Ian.

_______________________________________________
Xen-devel mailing list
[email protected]
http://lists.xensource.com/xen-devel

Quoted 7 lines

Quoted 7 lines

The state is a bit embarrassing on the BSD side. FreeBSD has had a multi-page ring extension since October of 2010. Unfortunately, I wrote this extension before stumbling upon the Citrix blkif patch or the extension being used on EC2. It is closest to the EC2 implementation, but not quite compatible. The saving grace is that I don't know of any deployments of this back-end outside of Spectra, and we have not shipped it to customers, so I plan to upstream block front and back drivers that

Excellent <crosses out another TODO on the "get done at some point list">

Quoted 1 line

Quoted 1 line

only implement the Citrix and EC2 style extension once blkif.h settles. A

So what is the Red Hat version of this extension? Where can I find it? Is it in the 2.6.18 hg tree?

Quoted 47 lines · Justin T. Gibbs

Quoted 33 lines · Ian Campbell

Quoted 26 lines · Justin Gibbs

Quoted 24 lines · Jan Beulich

xen/include/public/io/blkif.h line 222

Quoted 9 lines

Quoted 1 line

Quoted 1 line · Justin T. Gibbs

xen/include/public/io/blkif.h +4 -0

@@ -187,6 +216,25 @@
 *      The machine ABI rules governing the format of all ring request and
 *      response structures.
 *
+ * ring-page-order
+ *      Values:         <uint32_t>
+ *      Default Value:  0
+ *      Maximum Value:  MAX(ffs(max-ring-pages) - 1, max-ring-page-order)

Why not just max-ring-page-order. After all, the value is meaningless for a backend that only exposes max-ring-pages.

Quoted 10 lines

xen/include/public/io/blkif.h +10 -0

+ *      Notes:          1, 3
+ *
+ *      The size of the frontend allocated request ring buffer in units
+ *      of lb(machine pages). (e.g. 0 == 1 page, 1 = 2 pages, 2 == 4 pages,
+ *      etc.).
+ *
+ * ring-pages
+ *      Values:         <uint32_t>
+ *      Default Value:  1
+ *      Maximum Value:  MAX(max-ring-pages,(0x1 << max-ring-page-order))

Quoted 5 lines

Quoted 1 line

Similarly here - just max-ring-pages should do.

This patch documents existing extensions. For a new driver to properly negotiate a multi-page ring with a Dom0 on EC2 (ring-pages) or a Citrix Dom0/guest (ring-page-order), you must use the XenBus node names as I've defined them here.

Quoted 5 lines

Can we pick one and mark it as preferred and the other deprecated or similar? Perhaps backends will have to support both for the foreseeable future but we should make it possible for frontends to just pick one if that's what they want while nudging them in the direction of all picking the same one.

History says that back-ends are updated more slowly than front-ends. For example, my fixes to QEMU's serial port emulation that have been in Xen's mainline for ~2 years are still not widely deployed on EC2. Folks running FreeBSD there are using an ugly hack to FreeBSD's serial driver so they can get console output.

Quoted 7 lines

So, if you want your front-ends to have optimal performance regardless of what cloud or appliance they are running on, the two types will have to be supported for some time. Neither is better than the other, just different.

The opposite is true for back-ends. If your customer controls the guest environment and chooses not upgrade, you can't kill support for the variant they use.

I agree with all you say, the practicalities certainly mean we may well be stuck with both forever. I don't think there is any harm in documenting one of them as preferred though.

If someone has the freedom (or desire) to only implement one of the two in their code base then that provides guidance as to which it should be, at least then we will be encouraging the right trajectory. Also it gives some small amount of ammunition to people who are stuck with a provider (of either front or backend) of the wrong type for their environment and helps break the stalemate as to who should fix their end (however helpful that might be in practice).

Quoted 7 lines · Ian Campbell

If someone has the freedom (or desire) to only implement one of the two in their code base then that provides guidance as to which it should be, at least then we will be encouraging the right trajectory. Also it gives some small amount of ammunition to people who are stuck with a provider (of either front or backend) of the wrong type for their environment and helps break the stalemate as to who should fix their end (however helpful that might be in practice).

Anyone care to flip a coin?