Skip to contentSkip to conversation

Version 3 patch series for PCIe DOE for PCIe and CXL 2.0

RFC PATCH Superseded Chris Browy 2 participants 6 messages Mar 9, 2021 – Mar 12, 2021 Arm/Amlogic Meson SoC support

v3 of 6 Superseded Latest is v6 →

lore
Share & tools
lore
#0 lore raw

v3 cover Version 3 patch series for PCIe DOE for PCIe and CXL 2.0

Version 3 patch series for PCIe DOE for PCIe and CXL 2.0 implements all planned functionality.

Based on QEMU version:
https://gitlab.com/bwidawsk/qemu/-/tree/cxl-2.0v4

Summary:
1: PCIe DOE support for Discovery
- Support multiple DOE instances for each own protocol set
- MSI-X and polling supported
- Update error and interrupt status in DOE Status register
- Use static array to register callback function for DOE protocols
- Deprecate DOE_SUCCESS and DOE_DISCARD
- Add license headers
2: CXL DOE for CDAT and Compliance Mode.
- Device supports pre-defined CDAT or user-provided CDAT.
- Support on reading the iASL aml file via specifying "cdat=<filename.aml>" property to -device cxl-type3 skips over the ACPI header and writes only CDAT table entries
- Clean up CXL compliance structures
- DOE CDAT response returns one CDAT Structure instance based on request EntryHandle value.

Example cdat.dat file: (compile with iasl -G cdat.dat)
CDAT file may contain any mix and number of supported CDAT Structure types
----------------------
/* Header */
Signature : "CDAT"
Table Length : 00000000
Revision : 01
Checksum : 00
Oem ID : AVERY
Oem Table ID : 0
Oem Revision : 1
Asl Compiler ID : "INTL"
Asl Compiler Revision : 20160527

/* CDAT structures */
Label : DSMAS // Field Byte Length
UINT8 : 0 // Type 1
UINT8 : 00 // Reserved 1
UINT16 : 0018 // Length 2
UINT8 : 00 // DSMADHandle 1
UINT8 : 00 // Flags 1
UINT16 : 0000 // Reserved 2
UINT64 : 0000000000000000 // DPA Base 8
UINT64 : ffffffffffffffff // DPA Length 8

Label : DSLBIS // Field Byte Length
UINT8 : 01 // Type 1
UINT8 : 00 // Reserved 1
UINT16 : 0018 // Length 2
UINT8 : 00 // Handle 1
UINT8 : 00 // Flags 1
UINT8 : 00 // Data Type 1
UINT8 : 00 // Reserved 1
UINT64 : 0000000000000000 // Entry Base Unit 8
UINT16 : 0000 // Entry[0] 2
UINT16 : 0000 // Entry[1] 2
UINT16 : 0000 // Entry[2] 2
UINT16 : 0000 // Reserved 2

Label: DSMSCIS // Field Byte Length
UINT8 : 02 // Type 1
UINT8 : 00 // Reserved 1
UINT16 : 0014 // Length 2
UINT8 : 00 // DSMASHandle 1
UINT24 : 000000 // Reserved 3
UINT64 : 0000000000000000 // Memory Side Cache Size 8
UINT32 : 00000000 // Cache Attributes 4

Label : DSIS // Field Byte Length
UINT8 : 03 // Type 1
UINT8 : 00 // Reserved 1
UINT16 : 0008 // Length 2
UINT8 : 00 // Flags 1
UINT8 : 00 // Handle 1
UINT16 : 0000 // Reserved 2

Label : DSEMTS // Field Byte Length
UINT8 : 04 // Type 1
UINT8 : 00 // Reserved 1
UINT16 : 0018 // Length 2
UINT8 : 00 // DSMAS Handle 1
UINT8 : 00 // EFI Memory Type and Attribute 1
UINT16 : 0000 // Reserved 2
UINT64 : 0000000000000000 // DPA Offset 8
UINT64 : 0000000000000000 // DPA Length 8

Label : SSLBIS // Field Byte Length
UINT8 : 05 // Type 1
UINT8 : 00 // Reserved 1
UINT16 : 0020 // Length 2
UINT8 : 00 // Data Type 1
UINT24 : 000000 // Reserved 3
UINT64 : 0000000000000000 // Entry Base Unit 8
Label : SSLBE // SSLBE[0]
UINT16 : 0000 // Port X ID 2
UINT16 : 0000 // Port Y ID 2
UINT16 : 0000 // Latency or Bandwidth 2
UINT16 : BBBB // Reserved 2
Label : SSLBE // SSLBE[1]
UINT16 : 0000 // Port X ID 2
UINT16 : 0000 // Port Y ID 2
UINT16 : 0000 // Latency or Bandwidth 2
UINT16 : BBBC // Reserved 2
----

References:
1. CXL 2.0 specification
https://www.computeexpresslink.org/download-the-specification
2. PCI-SIG ECN: Data Object Exchange (DOE)
http://www.pcisig.com
3. Coherent Device Attribute Table CDAT 1.02
https://uefi.org/sites/default/files/resources/Coherent%20Device%20Attribute%20Table_1.02.pdf

---
Chris Browy (2):
Basic PCIe DOE support
CXL DOE support for CDAT and Compliance Mode

 MAINTAINERS                               |  49 +--
 hw/cxl/cxl-component-utils.c              |  93 +++++
 hw/mem/cxl_type3.c                        | 184 ++++++++++
 hw/pci/meson.build                        |   1 +
 hw/pci/pci.c                              |  13 +-
 hw/pci/pcie_doe.c                         | 416 ++++++++++++++++++++++
 include/hw/cxl/cxl_cdat.h                 | 127 +++++++
 include/hw/cxl/cxl_compl.h                | 252 +++++++++++++
 include/hw/cxl/cxl_component.h            |  74 ++++
 include/hw/cxl/cxl_device.h               |   4 +
 include/hw/cxl/cxl_pci.h                  |   2 +
 include/hw/pci/pci_ids.h                  |   5 +-
 include/hw/pci/pcie.h                     |   1 +
 include/hw/pci/pcie_doe.h                 | 142 ++++++++
 include/hw/pci/pcie_regs.h                |   4 +
 include/standard-headers/linux/pci_regs.h |   3 +-
 16 files changed, 1327 insertions(+), 43 deletions(-)
 create mode 100644 hw/pci/pcie_doe.c
 create mode 100644 include/hw/cxl/cxl_cdat.h
 create mode 100644 include/hw/cxl/cxl_compl.h
 create mode 100644 include/hw/pci/pcie_doe.h
Signature · 1 line
2.17.1
#1 lore raw

RFC PATCH v3 1/2 Basic PCIe DOE support

---
 MAINTAINERS                               |  49 +--
 hw/pci/meson.build                        |   1 +
 hw/pci/pci.c                              |  13 +-
 hw/pci/pcie_doe.c                         | 416 ++++++++++++++++++++++
 include/hw/pci/pci_ids.h                  |   5 +-
 include/hw/pci/pcie.h                     |   1 +
 include/hw/pci/pcie_doe.h                 | 142 ++++++++
 include/hw/pci/pcie_regs.h                |   4 +
 include/standard-headers/linux/pci_regs.h |   3 +-
 9 files changed, 591 insertions(+), 43 deletions(-)
 create mode 100644 hw/pci/pcie_doe.c
 create mode 100644 include/hw/pci/pcie_doe.h
MAINTAINERS +17 -32
index f9097ed9e7..8c5a9690a5 100644
@@ -87,7 +87,7 @@ S390 general architecture support
 M: Cornelia Huck <[email protected]>
 M: Thomas Huth <[email protected]>
 S: Supported
-F: default-configs/*/s390x-softmmu.mak
+F: default-configs/s390x-softmmu.mak
 F: gdb-xml/s390*.xml
 F: hw/char/sclp*.[hc]
 F: hw/char/terminal3270.c
@@ -188,15 +188,6 @@ F: include/hw/cris/
 F: tests/tcg/cris/
 F: disas/cris.c
 
-Hexagon TCG CPUs
-M: Taylor Simpson <[email protected]>
-S: Supported
-F: target/hexagon/
-F: linux-user/hexagon/
-F: tests/tcg/hexagon/
-F: disas/hexagon.c
-F: default-configs/targets/hexagon-linux-user.mak
-
 HPPA (PA-RISC) TCG CPUs
 M: Richard Henderson <[email protected]>
 S: Maintained
@@ -239,7 +230,7 @@ R: Jiaxun Yang <[email protected]>
 R: Aleksandar Rikalo <[email protected]>
 S: Odd Fixes
 F: target/mips/
-F: default-configs/*/*mips*
+F: default-configs/*mips*
 F: disas/mips.c
 F: docs/system/cpu-models-mips.rst.inc
 F: hw/intc/mips_gic.c
@@ -263,7 +254,7 @@ S: Maintained
 F: target/moxie/
 F: disas/moxie.c
 F: hw/moxie/
-F: default-configs/*/moxie-softmmu.mak
+F: default-configs/moxie-softmmu.mak
 
 NiosII TCG CPUs
 M: Chris Wulff <[email protected]>
@@ -272,7 +263,7 @@ S: Maintained
 F: target/nios2/
 F: hw/nios2/
 F: disas/nios2.c
-F: default-configs/*/nios2-softmmu.mak
+F: default-configs/nios2-softmmu.mak
 
 OpenRISC TCG CPUs
 M: Stafford Horne <[email protected]>
@@ -367,7 +358,7 @@ F: hw/xtensa/
 F: tests/tcg/xtensa/
 F: disas/xtensa.c
 F: include/hw/xtensa/xtensa-isa.h
-F: default-configs/*/xtensa*.mak
+F: default-configs/xtensa*.mak
 
 TriCore TCG CPUs
 M: Bastian Koppelmann <[email protected]>
@@ -1038,7 +1029,7 @@ AVR MCUs
 M: Michael Rolnik <[email protected]>
 R: Sarah Harris <[email protected]>
 S: Maintained
-F: default-configs/*/avr-softmmu.mak
+F: default-configs/avr-softmmu.mak
 F: hw/avr/
 F: include/hw/char/avr_usart.h
 F: hw/char/avr_usart.c
@@ -1067,7 +1058,7 @@ HP B160L
 M: Richard Henderson <[email protected]>
 R: Helge Deller <[email protected]>
 S: Odd Fixes
-F: default-configs/*/hppa-softmmu.mak
+F: default-configs/hppa-softmmu.mak
 F: hw/hppa/
 F: pc-bios/hppa-firmware.img
 
@@ -1183,7 +1174,6 @@ F: hw/intc/loongson_liointc.c
 F: hw/mips/loongson3_bootp.c
 F: hw/mips/loongson3_bootp.h
 F: hw/mips/loongson3_virt.c
-F: tests/acceptance/machine_mips_loongson3v.py
 
 Boston
 M: Paul Burton <[email protected]>
@@ -1373,15 +1363,6 @@ F: include/hw/misc/mchp_pfsoc_dmc.h
 F: include/hw/misc/mchp_pfsoc_ioscb.h
 F: include/hw/misc/mchp_pfsoc_sysreg.h
 
-SiFive Machines
-M: Alistair Francis <[email protected]>
-M: Bin Meng <[email protected]>
-M: Palmer Dabbelt <[email protected]>
-L: [email protected]
-S: Supported
-F: hw/*/*sifive*.c
-F: include/hw/*/*sifive*.h
-
 RX Machines
 -----------
 rx-gdbsim
@@ -1468,7 +1449,7 @@ F: hw/s390x/
 F: include/hw/s390x/
 F: hw/watchdog/wdt_diag288.c
 F: include/hw/watchdog/wdt_diag288.h
-F: default-configs/*/s390x-softmmu.mak
+F: default-configs/s390x-softmmu.mak
 F: tests/acceptance/machine_s390_ccw_virtio.py
 T: git https://gitlab.com/cohuck/qemu.git s390-next
 T: git https://github.com/borntraeger/qemu.git s390-next
@@ -1681,6 +1662,13 @@ F: docs/pci*
 F: docs/specs/*pci*
 F: default-configs/pci.mak
 
+PCIE DOE
+M: Huai-Cheng Kuo <[email protected]>
+M: Chris Browy <[email protected]>
+S: Supported
+F: include/hw/pci/pcie_doe.h
+F: hw/pci/pcie_doe.c
+
 ACPI/SMBIOS
 M: Michael S. Tsirkin <[email protected]>
 M: Igor Mammedov <[email protected]>
@@ -1764,7 +1752,6 @@ F: hw/ssi/xilinx_*
 
 SD (Secure Card)
 M: Philippe Mathieu-Daudé <[email protected]>
-M: Bin Meng <[email protected]>
 L: [email protected]
 S: Odd Fixes
 F: include/hw/sd/sd*
@@ -1859,7 +1846,6 @@ F: fsdev/
 F: docs/interop/virtfs-proxy-helper.rst
 F: tests/qtest/virtio-9p-test.c
 T: git https://gitlab.com/gkurz/qemu.git 9p-next
-T: git https://github.com/cschoenebeck/qemu.git 9p.next
 
 virtio-blk
 M: Stefan Hajnoczi <[email protected]>
@@ -2904,13 +2890,13 @@ F: accel/tcg/user-exec*.c
 BSD user
 S: Orphan
 F: bsd-user/
-F: default-configs/targets/*-bsd-user.mak
+F: default-configs/*-bsd-user.mak
 
 Linux user
 M: Laurent Vivier <[email protected]>
 S: Maintained
 F: linux-user/
-F: default-configs/targets/*linux-user.mak
+F: default-configs/*-linux-user.mak
 F: scripts/qemu-binfmt-conf.sh
 F: scripts/update-syscalltbl.sh
 F: scripts/update-mips-syscall-args.sh
@@ -2930,7 +2916,6 @@ S: Maintained
 F: docs/devel/tcg-plugins.rst
 F: plugins/
 F: tests/plugin/
-F: tests/acceptance/tcg_plugins.py
 F: contrib/plugins/
 
 AArch64 TCG target
hw/pci/meson.build +1 -0
index 5c4bbac817..115e50222f 100644
@@ -12,6 +12,7 @@ pci_ss.add(files(
 # allow plugging PCIe devices into PCI buses, include them even if
 # CONFIG_PCI_EXPRESS=n.
 pci_ss.add(files('pcie.c', 'pcie_aer.c'))
+pci_ss.add(files('pcie_doe.c'))
 softmmu_ss.add(when: 'CONFIG_PCI_EXPRESS', if_true: files('pcie_port.c', 'pcie_host.c'))
 softmmu_ss.add_all(when: 'CONFIG_PCI', if_true: pci_ss)
 
hw/pci/pci.c +5 -8
index 9d834facba..8d95c3fd25 100644
@@ -132,13 +132,8 @@ static void pci_bus_realize(BusState *qbus, Error **errp)
 static void pcie_bus_realize(BusState *qbus, Error **errp)
 {
     PCIBus *bus = PCI_BUS(qbus);
-    Error *local_err = NULL;
 
-    pci_bus_realize(qbus, &local_err);
-    if (local_err) {
-        error_propagate(errp, local_err);
-        return;
-    }
+    pci_bus_realize(qbus, errp);
 
     /*
      * A PCI-E bus can support extended config space if it's the root
@@ -2158,8 +2153,10 @@ static void pci_qdev_realize(DeviceState *qdev, Error **errp)
             pci_qdev_unrealize(DEVICE(pci_dev));
             return;
         }
-        if ((pci_dev->cap_present & QEMU_PCI_CAP_MULTIFUNCTION)
-            || (PCI_FUNC(pci_dev->devfn) != 0)) {
+        if (!(pci_dev->cap_present & QEMU_PCI_CAP_MULTIFUNCTION)
+            && (PCI_FUNC(pci_dev->devfn) == 0)) {
+            qdev->allow_unplug_during_migration = true;
+        } else {
             error_setg(errp, "failover: primary device must be in its own "
                               "PCI slot");
             pci_qdev_unrealize(DEVICE(pci_dev));
hw/pci/pcie_doe.c new file +416 -0

416 lines not on this page · raw message

include/hw/pci/pci_ids.h +3 -2
index 95f92d98e9..ffa9853247 100644
@@ -157,6 +157,8 @@
 
 /* Vendors and devices.  Sort key: vendor first, device next. */
 
+#define PCI_VENDOR_ID_PCI_SIG            0x0001
+
 #define PCI_VENDOR_ID_LSI_LOGIC          0x1000
 #define PCI_DEVICE_ID_LSI_53C810         0x0001
 #define PCI_DEVICE_ID_LSI_53C895A        0x0012
@@ -208,10 +210,9 @@
 #define PCI_DEVICE_ID_VIA_ISA_BRIDGE     0x0686
 #define PCI_DEVICE_ID_VIA_IDE            0x0571
 #define PCI_DEVICE_ID_VIA_UHCI           0x3038
-#define PCI_DEVICE_ID_VIA_82C686B_PM     0x3057
+#define PCI_DEVICE_ID_VIA_ACPI           0x3057
 #define PCI_DEVICE_ID_VIA_AC97           0x3058
 #define PCI_DEVICE_ID_VIA_MC97           0x3068
-#define PCI_DEVICE_ID_VIA_8231_PM        0x8235
 
 #define PCI_VENDOR_ID_MARVELL            0x11ab
 
include/hw/pci/pcie.h +1 -0
index 14c58ebdb6..47d6f66e52 100644
@@ -25,6 +25,7 @@
 #include "hw/pci/pcie_regs.h"
 #include "hw/pci/pcie_aer.h"
 #include "hw/hotplug.h"
+#include "hw/pci/pcie_doe.h"
 
 typedef enum {
     /* for attention and power indicator */
include/hw/pci/pcie_doe.h new file +142 -0

142 lines not on this page · raw message

include/hw/pci/pcie_regs.h +4 -0
index 1db86b0ec4..963dc2e170 100644
@@ -179,4 +179,8 @@ typedef enum PCIExpLinkWidth {
 #define PCI_ACS_VER                     0x1
 #define PCI_ACS_SIZEOF                  8
 
+/* DOE Capability Register Fields */
+#define PCI_DOE_VER                     0x1
+#define PCI_DOE_SIZEOF                  24
+
 #endif /* QEMU_PCIE_REGS_H */
include/standard-headers/linux/pci_regs.h +2 -1
index e709ae8235..4a7b7a426d 100644
@@ -730,7 +730,8 @@
 #define PCI_EXT_CAP_ID_DVSEC	0x23	/* Designated Vendor-Specific */
 #define PCI_EXT_CAP_ID_DLF	0x25	/* Data Link Feature */
 #define PCI_EXT_CAP_ID_PL_16GT	0x26	/* Physical Layer 16.0 GT/s */
-#define PCI_EXT_CAP_ID_MAX	PCI_EXT_CAP_ID_PL_16GT
+#define PCI_EXT_CAP_ID_DOE      0x2E    /* Data Object Exchange */
+#define PCI_EXT_CAP_ID_MAX	PCI_EXT_CAP_ID_DOE
 
 #define PCI_EXT_CAP_DSN_SIZEOF	12
 #define PCI_EXT_CAP_MCAST_ENDPOINT_SIZEOF 40
Signature · 1 line
2.17.1
#2 lore raw

Re: [RFC PATCH v3 cxl-2.0-doe 1/2] Basic PCIe DOE support

On Tue, 9 Mar 2021 16:03:55 -0500
Chris Browy <[email protected]> wrote:

Hi Chris, various specific comments inline but a few important ones to watch for next version.

1) Always read the final patch you are sending out. Last minute rebases can result in picking up a bunch of unrelated stuff like here. That basically gives a grumpy maintainer a reason not to read your patch. If I had maintainer hat on I'd have kicked this back to you unread.

2) Please add proper descriptions for each patch - I know it's an RFC but it feels like it is getting there so good to get that element in shape as well.

3) Split the patch up. Some stuff in here will be coming in via other paths (headers are picked up by a script in some cases).

4) To my eye far too much debug in here for code at this level of maturity.

From actual code point of view looks pretty good to me.
Drop CMA until you have it implemented - right now it's just unwanted noise in this patch.

Thanks,

Jonathan

Quoted 216 lines
---
 MAINTAINERS                               |  49 +--
 hw/pci/meson.build                        |   1 +
 hw/pci/pci.c                              |  13 +-
 hw/pci/pcie_doe.c                         | 416 ++++++++++++++++++++++
 include/hw/pci/pci_ids.h                  |   5 +-
 include/hw/pci/pcie.h                     |   1 +
 include/hw/pci/pcie_doe.h                 | 142 ++++++++
 include/hw/pci/pcie_regs.h                |   4 +
 include/standard-headers/linux/pci_regs.h |   3 +-
 9 files changed, 591 insertions(+), 43 deletions(-)
 create mode 100644 hw/pci/pcie_doe.c
 create mode 100644 include/hw/pci/pcie_doe.h
MAINTAINERS +17 -32
index f9097ed9e7..8c5a9690a5 100644
@@ -87,7 +87,7 @@ S390 general architecture support
 M: Cornelia Huck <[email protected]>
 M: Thomas Huth <[email protected]>
 S: Supported
-F: default-configs/*/s390x-softmmu.mak
+F: default-configs/s390x-softmmu.mak
 F: gdb-xml/s390*.xml
 F: hw/char/sclp*.[hc]
 F: hw/char/terminal3270.c
@@ -188,15 +188,6 @@ F: include/hw/cris/
 F: tests/tcg/cris/
 F: disas/cris.c
 
-Hexagon TCG CPUs
-M: Taylor Simpson <[email protected]>
-S: Supported
-F: target/hexagon/
-F: linux-user/hexagon/
-F: tests/tcg/hexagon/
-F: disas/hexagon.c
-F: default-configs/targets/hexagon-linux-user.mak
-
 HPPA (PA-RISC) TCG CPUs
 M: Richard Henderson <[email protected]>
 S: Maintained
@@ -239,7 +230,7 @@ R: Jiaxun Yang <[email protected]>
 R: Aleksandar Rikalo <[email protected]>
 S: Odd Fixes
 F: target/mips/
-F: default-configs/*/*mips*
+F: default-configs/*mips*
 F: disas/mips.c
 F: docs/system/cpu-models-mips.rst.inc
 F: hw/intc/mips_gic.c
@@ -263,7 +254,7 @@ S: Maintained
 F: target/moxie/
 F: disas/moxie.c
 F: hw/moxie/
-F: default-configs/*/moxie-softmmu.mak
+F: default-configs/moxie-softmmu.mak
 
 NiosII TCG CPUs
 M: Chris Wulff <[email protected]>
@@ -272,7 +263,7 @@ S: Maintained
 F: target/nios2/
 F: hw/nios2/
 F: disas/nios2.c
-F: default-configs/*/nios2-softmmu.mak
+F: default-configs/nios2-softmmu.mak
 
 OpenRISC TCG CPUs
 M: Stafford Horne <[email protected]>
@@ -367,7 +358,7 @@ F: hw/xtensa/
 F: tests/tcg/xtensa/
 F: disas/xtensa.c
 F: include/hw/xtensa/xtensa-isa.h
-F: default-configs/*/xtensa*.mak
+F: default-configs/xtensa*.mak
 
 TriCore TCG CPUs
 M: Bastian Koppelmann <[email protected]>
@@ -1038,7 +1029,7 @@ AVR MCUs
 M: Michael Rolnik <[email protected]>
 R: Sarah Harris <[email protected]>
 S: Maintained
-F: default-configs/*/avr-softmmu.mak
+F: default-configs/avr-softmmu.mak
 F: hw/avr/
 F: include/hw/char/avr_usart.h
 F: hw/char/avr_usart.c
@@ -1067,7 +1058,7 @@ HP B160L
 M: Richard Henderson <[email protected]>
 R: Helge Deller <[email protected]>
 S: Odd Fixes
-F: default-configs/*/hppa-softmmu.mak
+F: default-configs/hppa-softmmu.mak
 F: hw/hppa/
 F: pc-bios/hppa-firmware.img
 
@@ -1183,7 +1174,6 @@ F: hw/intc/loongson_liointc.c
 F: hw/mips/loongson3_bootp.c
 F: hw/mips/loongson3_bootp.h
 F: hw/mips/loongson3_virt.c
-F: tests/acceptance/machine_mips_loongson3v.py
 
 Boston
 M: Paul Burton <[email protected]>
@@ -1373,15 +1363,6 @@ F: include/hw/misc/mchp_pfsoc_dmc.h
 F: include/hw/misc/mchp_pfsoc_ioscb.h
 F: include/hw/misc/mchp_pfsoc_sysreg.h
 
-SiFive Machines
-M: Alistair Francis <[email protected]>
-M: Bin Meng <[email protected]>
-M: Palmer Dabbelt <[email protected]>
-L: [email protected]
-S: Supported
-F: hw/*/*sifive*.c
-F: include/hw/*/*sifive*.h
-
 RX Machines
 -----------
 rx-gdbsim
@@ -1468,7 +1449,7 @@ F: hw/s390x/
 F: include/hw/s390x/
 F: hw/watchdog/wdt_diag288.c
 F: include/hw/watchdog/wdt_diag288.h
-F: default-configs/*/s390x-softmmu.mak
+F: default-configs/s390x-softmmu.mak
 F: tests/acceptance/machine_s390_ccw_virtio.py
 T: git https://gitlab.com/cohuck/qemu.git s390-next
 T: git https://github.com/borntraeger/qemu.git s390-next
@@ -1681,6 +1662,13 @@ F: docs/pci*
 F: docs/specs/*pci*
 F: default-configs/pci.mak
 
+PCIE DOE
+M: Huai-Cheng Kuo <[email protected]>
+M: Chris Browy <[email protected]>
+S: Supported
+F: include/hw/pci/pcie_doe.h
+F: hw/pci/pcie_doe.c
+
 ACPI/SMBIOS
 M: Michael S. Tsirkin <[email protected]>
 M: Igor Mammedov <[email protected]>
@@ -1764,7 +1752,6 @@ F: hw/ssi/xilinx_*
 
 SD (Secure Card)
 M: Philippe Mathieu-Daudé <[email protected]>
-M: Bin Meng <[email protected]>
 L: [email protected]
 S: Odd Fixes
 F: include/hw/sd/sd*
@@ -1859,7 +1846,6 @@ F: fsdev/
 F: docs/interop/virtfs-proxy-helper.rst
 F: tests/qtest/virtio-9p-test.c
 T: git https://gitlab.com/gkurz/qemu.git 9p-next
-T: git https://github.com/cschoenebeck/qemu.git 9p.next
 
 virtio-blk
 M: Stefan Hajnoczi <[email protected]>
@@ -2904,13 +2890,13 @@ F: accel/tcg/user-exec*.c
 BSD user
 S: Orphan
 F: bsd-user/
-F: default-configs/targets/*-bsd-user.mak
+F: default-configs/*-bsd-user.mak
 
 Linux user
 M: Laurent Vivier <[email protected]>
 S: Maintained
 F: linux-user/
-F: default-configs/targets/*linux-user.mak
+F: default-configs/*-linux-user.mak
 F: scripts/qemu-binfmt-conf.sh
 F: scripts/update-syscalltbl.sh
 F: scripts/update-mips-syscall-args.sh
@@ -2930,7 +2916,6 @@ S: Maintained
 F: docs/devel/tcg-plugins.rst
 F: plugins/
 F: tests/plugin/
-F: tests/acceptance/tcg_plugins.py
 F: contrib/plugins/
 
 AArch64 TCG target
hw/pci/meson.build +1 -0
index 5c4bbac817..115e50222f 100644
@@ -12,6 +12,7 @@ pci_ss.add(files(
 # allow plugging PCIe devices into PCI buses, include them even if
 # CONFIG_PCI_EXPRESS=n.
 pci_ss.add(files('pcie.c', 'pcie_aer.c'))
+pci_ss.add(files('pcie_doe.c'))
 softmmu_ss.add(when: 'CONFIG_PCI_EXPRESS', if_true: files('pcie_port.c', 'pcie_host.c'))
 softmmu_ss.add_all(when: 'CONFIG_PCI', if_true: pci_ss)
 
hw/pci/pci.c +1 -6
index 9d834facba..8d95c3fd25 100644
@@ -132,13 +132,8 @@ static void pci_bus_realize(BusState *qbus, Error **errp)
 static void pcie_bus_realize(BusState *qbus, Error **errp)
 {
     PCIBus *bus = PCI_BUS(qbus);
-    Error *local_err = NULL;
 
-    pci_bus_realize(qbus, &local_err);
-    if (local_err) {
-        error_propagate(errp, local_err);
-        return;
-    }
+    pci_bus_realize(qbus, errp);
 
     /*
      * A PCI-E bus can support extended config space if it's the root

hw/pci/pci.c static void pci_qdev_realize(DeviceState *qdev, Error **errp) · line 2155

Quoted 3 lines
hw/pci/pci.c +0 -0
@@ -2158,8 +2153,10 @@ static void pci_qdev_realize(DeviceState *qdev, Error **errp)
             pci_qdev_unrealize(DEVICE(pci_dev));
             return;
         }

Please do a quick review of any patch before you send it out for unrelated stuff.

Quoted 27 lines
hw/pci/pci.c +4 -2
-        if ((pci_dev->cap_present & QEMU_PCI_CAP_MULTIFUNCTION)
-            || (PCI_FUNC(pci_dev->devfn) != 0)) {
+        if (!(pci_dev->cap_present & QEMU_PCI_CAP_MULTIFUNCTION)
+            && (PCI_FUNC(pci_dev->devfn) == 0)) {
+            qdev->allow_unplug_during_migration = true;
+        } else {
             error_setg(errp, "failover: primary device must be in its own "
                               "PCI slot");
             pci_qdev_unrealize(DEVICE(pci_dev));
hw/pci/pcie_doe.c new file +12 -0
new file mode 100644
index 0000000000..fab58eb897
@@ -0,0 +1,416 @@
+/*
+ * PCIe Data Object Exchange
+ *
+ * Copyright (C) 2020 Avery Design Systems, Inc.
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ */
+
+#include "qemu/osdep.h"
+#include "qemu/log.h"
+#include "qemu/error-report.h"

hw/pci/pcie_doe.c line 22

Quoted 10 lines
hw/pci/pcie_doe.c new file +10 -0
+#include "qapi/error.h"
+#include "qemu/range.h"
+#include "hw/pci/pci.h"
+#include "hw/pci/pcie.h"
+#include "hw/pci/pcie_doe.h"
+#include "hw/pci/msi.h"
+#include "hw/pci/msix.h"
+
+/*
+ * DOE Default Protocols (Discovery, CMA)

CMA is absolutely not a default protocol. If it is there, some other protocols cannot be.

Quoted 51 lines
hw/pci/pcie_doe.c new file +51 -0
+ */
+/* Discovery Request Object */
+struct doe_discovery {
+    DOEHeader header;
+    uint8_t index;
+    uint8_t reserved[3];
+} QEMU_PACKED;
+
+/* Discovery Response Object */
+struct doe_discovery_rsp {
+    DOEHeader header;
+    uint16_t vendor_id;
+    uint8_t doe_type;
+    uint8_t next_index;
+} QEMU_PACKED;
+
+/* Callback for Discovery */
+static bool pcie_doe_discovery_rsp(DOECap *doe_cap)
+{
+    struct doe_discovery *req = pcie_doe_get_req(doe_cap);
+    struct doe_discovery_rsp rsp;
+    uint8_t index = req->index;
+    DOEProtocol *prot;
+
+    /* Request length mismatch, discard */
+    if (pcie_doe_object_len(req) <
+        DIV_ROUND_UP(sizeof(struct doe_discovery), 4)) {
+        return false;
+    }
+
+    rsp.header = (DOEHeader) {
+        .vendor_id = PCI_VENDOR_ID_PCI_SIG,
+        .doe_type = PCI_SIG_DOE_DISCOVERY,
+        .length = DIV_ROUND_UP(sizeof(struct doe_discovery_rsp), 4),
+    };
+
+    /* Point to the requested protocol, index 0 must be Discovery */
+    if (index == 0) {
+        rsp.vendor_id = PCI_VENDOR_ID_PCI_SIG;
+        rsp.doe_type = PCI_SIG_DOE_DISCOVERY;
+    } else {
+        if (index < doe_cap->protocol_num) {
+            prot = &doe_cap->protocols[index - 1];
+        } else {
+            prot = NULL;
+        }
+
+        rsp.vendor_id = (prot) ? prot->vendor_id : 0xFFFF;
+        rsp.doe_type = (prot) ? prot->doe_type : 0xFF;
+    }
+

hw/pci/pcie_doe.c line 83

Quoted 10 lines
hw/pci/pcie_doe.c new file +10 -0
+    rsp.next_index = (index + 1) % doe_cap->protocol_num,
+
+    pcie_doe_set_rsp(doe_cap, &rsp);
+
+    return true;
+}
+
+/* Callback for CMA */
+bool pcie_doe_cma_rsp(DOECap *doe_cap)
+{

Umm. Really? Please don't leave a stub like this in place. Just don't support CMA until you are ready to do it fully.

If you can share plan on that it would be great though as it's on my list of things to track.

hw/pci/pcie_doe.c line 95

Quoted 12 lines
hw/pci/pcie_doe.c new file +12 -0
+    doe_cap->status.error = 1;
+
+    memset(doe_cap->read_mbox, 0,
+           PCI_DOE_MAX_DW_SIZE * sizeof(uint32_t));
+
+    doe_cap->write_mbox_len = 0;
+
+    return false;
+}
+
+/*
+ * DOE Utilities

Generic comments like this always rot (become out of date) over time, so don't bother.

hw/pci/pcie_doe.c line 108

Quoted 13 lines
hw/pci/pcie_doe.c new file +13 -0
+ */
+static void pcie_doe_reset_mbox(DOECap *st)
+{
+    st->read_mbox_idx = 0;
+    st->read_mbox_len = 0;
+    st->write_mbox_len = 0;
+
+    memset(st->read_mbox, 0, PCI_DOE_MAX_DW_SIZE * sizeof(uint32_t));
+    memset(st->write_mbox, 0, PCI_DOE_MAX_DW_SIZE * sizeof(uint32_t));
+}
+
+/*
+ * PCI DOE functions

Same with this one. Make it clear by naming of functions rather than a comment that will probably end up in the wrong place in a few years.

Quoted 119 lines
hw/pci/pcie_doe.c new file +119 -0
+ */
+/*
+ * Add DOE cap to a dev
+ * Initialize its protocol.
+ */
+void pcie_doe_init(PCIDevice *dev, DOECap *doe_cap, uint16_t offset,
+                   DOEProtocol *protocols, bool intr, uint16_t vec)
+{
+    pcie_add_capability(dev, PCI_EXT_CAP_ID_DOE, PCI_DOE_VER, offset,
+                        PCI_DOE_SIZEOF);
+
+    doe_cap->pdev = dev;
+    doe_cap->offset = offset;
+
+    /* Configure MSI/MSI-X */
+    if (intr && (msi_present(dev) || msix_present(dev))) {
+        doe_cap->cap.intr = intr;
+        doe_cap->cap.vec = vec;
+    }
+
+    /* Set up W/R Mailbox buffer */
+    doe_cap->write_mbox = g_malloc0(PCI_DOE_MAX_DW_SIZE * sizeof(uint32_t));
+    doe_cap->read_mbox = g_malloc0(PCI_DOE_MAX_DW_SIZE * sizeof(uint32_t));
+
+    pcie_doe_reset_mbox(doe_cap);
+
+    /* Register default discovery protocol */
+    doe_cap->protocols = protocols;
+    for (; protocols->vendor_id; protocols++) {
+        doe_cap->protocol_num++;
+    }
+    assert(doe_cap->protocol_num < PCI_DOE_PROTOCOL_MAX);
+
+    /* Add 1 for Discovery */
+    doe_cap->protocol_num++;
+}
+
+void pcie_doe_fini(DOECap *doe_cap)
+{
+    g_free(doe_cap->read_mbox);
+    g_free(doe_cap->write_mbox);
+    g_free(doe_cap);
+}
+
+uint32_t pcie_doe_build_protocol(DOEProtocol *p)
+{
+    return DATA_OBJ_BUILD_HEADER1(p->vendor_id, p->doe_type);
+}
+
+/* Return the pointer of DOE request in write mailbox buffer */
+void *pcie_doe_get_req(DOECap *doe_cap)
+{
+    return doe_cap->write_mbox;
+}
+
+/* Copy the response to read mailbox buffer */
+void pcie_doe_set_rsp(DOECap *doe_cap, void *rsp)
+{
+    uint32_t len = pcie_doe_object_len(rsp);
+
+    memcpy(doe_cap->read_mbox + doe_cap->read_mbox_len,
+           rsp, len * sizeof(uint32_t));
+    doe_cap->read_mbox_len += len;
+}
+
+/* Get Data Object length */
+uint32_t pcie_doe_object_len(void *obj)
+{
+    uint32_t len = (obj) ? ((DOEHeader *)obj)->length : 0;
+
+    return len & (PCI_DOE_MAX_DW_SIZE - 1);
+}
+
+static void pcie_doe_write_mbox(DOECap *doe_cap, uint32_t val)
+{
+    doe_cap->write_mbox[doe_cap->write_mbox_len] = val;
+
+    if (doe_cap->write_mbox_len == 1) {
+        DOE_DBG("(Capture DOE request DO length = %x)", val & 0x0003ffff);
+    }
+
+    doe_cap->write_mbox_len++;
+}
+
+static void pcie_doe_irq_assert(DOECap *doe_cap)
+{
+    PCIDevice *dev = doe_cap->pdev;
+
+    if (doe_cap->cap.intr && doe_cap->ctrl.intr) {
+        if (doe_cap->status.intr) {
+            return;
+        }
+        doe_cap->status.intr = 1;
+
+        /* Interrupt notify */
+        if (msix_enabled(dev)) {
+            msix_notify(dev, doe_cap->cap.vec);
+        } else if (msi_enabled(dev)) {
+            msi_notify(dev, doe_cap->cap.vec);
+        }
+        /* Not support legacy IRQ */
+    }
+}
+
+static void pcie_doe_set_ready(DOECap *doe_cap, bool rdy)
+{
+    doe_cap->status.ready = rdy;
+
+    if (rdy) {
+        pcie_doe_irq_assert(doe_cap);
+    }
+}
+
+static void pcie_doe_set_error(DOECap *doe_cap, bool err)
+{
+    doe_cap->status.error = err;
+
+    if (err) {
+        pcie_doe_irq_assert(doe_cap);

hw/pci/pcie_doe.c line 237

Quoted 10 lines
hw/pci/pcie_doe.c new file +10 -0
+    }
+}
+
+/*
+ * Check protocol the incoming request in write_mbox and
+ * memcpy the corresponding response to read_mbox.
+ *
+ * "discard" should be set up if the response callback
+ * function could not deal with request (e.g. length
+ * mismatch) or the protocol of request was not found.

I think this comment is out of date? What does "discard" being setup mean?

Quoted 118 lines
hw/pci/pcie_doe.c new file +118 -0
+ */
+static void pcie_doe_prepare_rsp(DOECap *doe_cap)
+{
+    bool success = false;
+    int p;
+
+    if (doe_cap->status.error) {
+        return;
+    }
+
+    if (doe_cap->write_mbox[0] ==
+        DATA_OBJ_BUILD_HEADER1(PCI_VENDOR_ID_PCI_SIG, PCI_SIG_DOE_DISCOVERY)) {
+        /* Discovery */
+        success = pcie_doe_discovery_rsp(doe_cap);
+    } else {
+        /* Other protocols */
+        for (p = 0; p < doe_cap->protocol_num - 1; p++) {
+            if (doe_cap->write_mbox[0] ==
+                pcie_doe_build_protocol(&doe_cap->protocols[p])) {
+                DOE_DBG("(protocol = %x)", doe_cap->write_mbox[0]);
+                /*
+                 * Spec:
+                 * If the number of DW transferred does not match the
+                 * indicated Length for a data object, then the
+                 * data object must be silently discarded.
+                 */
+                if (doe_cap->write_mbox_len ==
+                    pcie_doe_object_len(pcie_doe_get_req(doe_cap))) {
+                    success = doe_cap->protocols[p].set_rsp(doe_cap);
+                }
+                break;
+            }
+        }
+    }
+
+    /* Set DOE Ready */
+    if (success) {
+        pcie_doe_set_ready(doe_cap, 1);
+    } else {
+        pcie_doe_reset_mbox(doe_cap);
+    }
+}
+
+/*
+ * Read from DOE config space.
+ * Return false if the address doesn't hit the range.
+ */
+bool pcie_doe_read_config(DOECap *doe_cap, uint32_t addr, int size,
+                          uint32_t *buf)
+{
+    uint32_t shift, mask = 0xFFFFFFFF;
+    uint16_t doe_offset = doe_cap->offset;
+
+    if (!range_covers_byte(doe_offset + PCI_EXP_DOE_CAP,
+                           PCI_DOE_SIZEOF - 4, addr)) {
+        return false;
+    }
+
+    DOE_DBG("%s: addr=%x, size=%x\n", __func__, addr, size);
+    addr -= doe_offset;
+    *buf = 0;
+
+    if (range_covers_byte(PCI_EXP_DOE_CAP, sizeof(uint32_t), addr)) {
+        DOE_DBG("  CAP");
+        *buf = FIELD_DP32(*buf, PCI_DOE_CAP_REG, INTR_SUPP,
+                          doe_cap->cap.intr);
+        *buf = FIELD_DP32(*buf, PCI_DOE_CAP_REG, DOE_INTR_MSG_NUM,
+                          doe_cap->cap.vec);
+    } else if (range_covers_byte(PCI_EXP_DOE_CTRL, sizeof(uint32_t), addr)) {
+        DOE_DBG("  CONTROL");
+        /* Must return ABORT=0 and GO=0 */
+        *buf = FIELD_DP32(*buf, PCI_DOE_CAP_CONTROL, DOE_INTR_EN,
+                          doe_cap->ctrl.intr);
+    } else if (range_covers_byte(PCI_EXP_DOE_STATUS, sizeof(uint32_t), addr)) {
+        DOE_DBG("  STATUS");
+        *buf = FIELD_DP32(*buf, PCI_DOE_CAP_STATUS, DOE_BUSY,
+                          doe_cap->status.busy);
+        *buf = FIELD_DP32(*buf, PCI_DOE_CAP_STATUS, DOE_INTR_STATUS,
+                          doe_cap->status.intr);
+        *buf = FIELD_DP32(*buf, PCI_DOE_CAP_STATUS, DOE_ERROR,
+                          doe_cap->status.error);
+        *buf = FIELD_DP32(*buf, PCI_DOE_CAP_STATUS, DATA_OBJ_RDY,
+                          doe_cap->status.ready);
+    } else if (range_covers_byte(PCI_EXP_DOE_RD_DATA_MBOX, sizeof(uint32_t), addr)) {
+        DOE_DBG("  RD_MBOX");
+        if (doe_cap->status.ready && !doe_cap->status.error) {
+            DOE_DBG(" (ready, off=%x, len=%x)",
+                    doe_cap->read_mbox_idx, doe_cap->read_mbox_len);
+            *buf = doe_cap->read_mbox[doe_cap->read_mbox_idx];
+        }
+    } else if (range_covers_byte(PCI_EXP_DOE_WR_DATA_MBOX, sizeof(uint32_t), addr)) {
+        DOE_DBG("  WR_MBOX");
+    }
+    DOE_DBG(", val=%x\n", *buf);
+
+    /* Alignment */
+    shift = addr % sizeof(uint32_t);
+    *buf >>= shift * 8;
+    mask >>= (sizeof(uint32_t) - size) * 8;
+    *buf &= mask;
+
+    return true;
+}
+
+/*
+ * Write to DOE config space.
+ * Early return if the address doesn't hit the range or receives an abort signal.
+ */
+void pcie_doe_write_config(DOECap *doe_cap,
+                           uint32_t addr, uint32_t val, int size)
+{
+    uint16_t doe_offset = doe_cap->offset;
+    uint32_t shift;
+
+    if (!range_covers_byte(doe_offset + PCI_EXP_DOE_CAP,
+                           PCI_DOE_SIZEOF - 4, addr)) {
+        return;
+    }

hw/pci/pcie_doe.c line 365

Quoted 10 lines
hw/pci/pcie_doe.c new file +10 -0
+
+    DOE_DBG("%s: addr = %x, size = %x, val = %x\033[m\n", __func__, addr, size, val);
+    /* Alignment */
+    shift = addr % sizeof(uint32_t);
+    addr -= (doe_offset - shift);
+    val <<= shift * 8;
+
+    switch (addr) {
+    case PCI_EXP_DOE_CTRL:
+        DOE_DBG("  CONTROL");

The density of debug info in here feel too high for a code that is pretty close to ready for merging. I'd think hard about what actually matters - I'd suggest just unexpected error paths, but perhaps a little more if it's really useful.

It absolutely makes sense to have this debug in there whilst developing, but once you are done it's not really very useful + makes code less readable and bloated.

hw/pci/pcie_doe.c line 367

Quoted 2 lines
hw/pci/pcie_doe.c new file +2 -0
+        if (FIELD_EX32(val, PCI_DOE_CAP_CONTROL, DOE_ABORT)) {
+            /* If ABORT, clear status reg DOE Error and DOE Ready */

Fairly obvious comment and it doesn't mention the reset mailbox, so I'd drop it as unnecessary and incomplete.

Quoted 5 lines
hw/pci/pcie_doe.c new file +5 -0
+            DOE_DBG("-ABORT");
+            pcie_doe_set_ready(doe_cap, 0);
+            pcie_doe_set_error(doe_cap, 0);
+            pcie_doe_reset_mbox(doe_cap);
+            return;

hw/pci/pcie_doe.c line 382

Quoted 10 lines
hw/pci/pcie_doe.c new file +10 -0
+        }
+
+        if (FIELD_EX32(val, PCI_DOE_CAP_CONTROL, DOE_GO)) {
+            DOE_DBG("-GO");
+            pcie_doe_prepare_rsp(doe_cap);
+        }
+
+        if (FIELD_EX32(val, PCI_DOE_CAP_CONTROL, DOE_INTR_EN)) {
+            DOE_DBG("-INTR");
+            doe_cap->ctrl.intr = 1;

How to disable interrupt?

Quoted 19 lines
hw/pci/pcie_doe.c new file +19 -0
+        }
+        break;
+    case PCI_EXP_DOE_STATUS:
+        DOE_DBG("  STATUS");
+        if (FIELD_EX32(val, PCI_DOE_CAP_STATUS, DOE_INTR_STATUS)) {
+            DOE_DBG("-INTR");
+            doe_cap->status.intr = 0;
+        }
+        break;
+    case PCI_EXP_DOE_RD_DATA_MBOX:
+        DOE_DBG("  RD_MBOX");
+        doe_cap->read_mbox_idx += 1;
+        DOE_DBG(" (incr offset = %x)", doe_cap->read_mbox_idx);
+        if (doe_cap->read_mbox_idx == doe_cap->read_mbox_len) {
+            /* Last DW */
+            pcie_doe_reset_mbox(doe_cap);
+            pcie_doe_set_ready(doe_cap, 0);
+        } else if (doe_cap->read_mbox_idx > doe_cap->read_mbox_len) {
+            /* Underflow */

hw/pci/pcie_doe.c line 411

Quoted 10 lines
hw/pci/pcie_doe.c new file +10 -0
+            pcie_doe_set_error(doe_cap, 1);
+        }
+        break;
+    case PCI_EXP_DOE_WR_DATA_MBOX:
+        DOE_DBG("  WR_MBOX");
+        pcie_doe_write_mbox(doe_cap, val);
+        DOE_DBG(" (offset = %x)", doe_cap->write_mbox_len);
+        break;
+    case PCI_EXP_DOE_CAP:
+        DOE_DBG("  CAP (no effect)");

At least sometimes, qemu is built with checks that require explicit fallthrough markers

	   /* fallthrough */

This aids static checkers by making it clear you did this deliberately.

Quoted 19 lines
hw/pci/pcie_doe.c new file +5 -0
+    default:
+        break;
+    }
+    DOE_DBG("\n");
+}
include/hw/pci/pci_ids.h +2 -0
index 95f92d98e9..ffa9853247 100644
@@ -157,6 +157,8 @@
 
 /* Vendors and devices.  Sort key: vendor first, device next. */
 
+#define PCI_VENDOR_ID_PCI_SIG            0x0001
+
 #define PCI_VENDOR_ID_LSI_LOGIC          0x1000
 #define PCI_DEVICE_ID_LSI_53C810         0x0001
 #define PCI_DEVICE_ID_LSI_53C895A        0x0012

include/hw/pci/pci_ids.h line 211

Quoted 4 lines
include/hw/pci/pci_ids.h +0 -1
@@ -208,10 +210,9 @@
 #define PCI_DEVICE_ID_VIA_ISA_BRIDGE     0x0686
 #define PCI_DEVICE_ID_VIA_IDE            0x0571
 #define PCI_DEVICE_ID_VIA_UHCI           0x3038
-#define PCI_DEVICE_ID_VIA_82C686B_PM     0x3057

Please sanity check patches for merge conflicts. This just broke a bunch of other peoples drivers.

Quoted 59 lines
include/hw/pci/pci_ids.h +1 -1
+#define PCI_DEVICE_ID_VIA_ACPI           0x3057
 #define PCI_DEVICE_ID_VIA_AC97           0x3058
 #define PCI_DEVICE_ID_VIA_MC97           0x3068
-#define PCI_DEVICE_ID_VIA_8231_PM        0x8235
 
 #define PCI_VENDOR_ID_MARVELL            0x11ab
 
include/hw/pci/pcie.h +1 -0
index 14c58ebdb6..47d6f66e52 100644
@@ -25,6 +25,7 @@
 #include "hw/pci/pcie_regs.h"
 #include "hw/pci/pcie_aer.h"
 #include "hw/hotplug.h"
+#include "hw/pci/pcie_doe.h"
 
 typedef enum {
     /* for attention and power indicator */
include/hw/pci/pcie_doe.h new file +34 -0
new file mode 100644
index 0000000000..45fe2d3c88
@@ -0,0 +1,142 @@
+/*
+ * PCIe Data Object Exchange
+ *
+ * Copyright (C) 2020 Avery Design Systems, Inc.
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ */
+
+#ifndef PCIE_DOE_H
+#define PCIE_DOE_H
+
+#include "qemu/range.h"
+#include "qemu/typedefs.h"
+#include "hw/register.h"
+
+/*
+ * PCI DOE register defines 7.9.24
+ */
+/* Capabilities Register 7.9.24.2 */
+#define PCI_EXP_DOE_CAP             0x04
+REG32(PCI_DOE_CAP_REG, 0)
+    FIELD(PCI_DOE_CAP_REG, INTR_SUPP, 0, 1)
+    FIELD(PCI_DOE_CAP_REG, DOE_INTR_MSG_NUM, 1, 11)
+
+/* Control Register 7.9.24.3 */
+#define PCI_EXP_DOE_CTRL            0x08
+REG32(PCI_DOE_CAP_CONTROL, 0)
+    FIELD(PCI_DOE_CAP_CONTROL, DOE_ABORT, 0, 1)
+    FIELD(PCI_DOE_CAP_CONTROL, DOE_INTR_EN, 1, 1)
+    FIELD(PCI_DOE_CAP_CONTROL, DOE_GO, 31, 1)
+
+/* Status Register 7.9.24.4 */
+#define PCI_EXP_DOE_STATUS          0x0c

include/hw/pci/pcie_doe.h line 44

Quoted 10 lines
include/hw/pci/pcie_doe.h new file +10 -0
+REG32(PCI_DOE_CAP_STATUS, 0)
+    FIELD(PCI_DOE_CAP_STATUS, DOE_BUSY, 0, 1)
+    FIELD(PCI_DOE_CAP_STATUS, DOE_INTR_STATUS, 1, 1)
+    FIELD(PCI_DOE_CAP_STATUS, DOE_ERROR, 2, 1)
+    FIELD(PCI_DOE_CAP_STATUS, DATA_OBJ_RDY, 31, 1)
+
+/* Write Data Mailbox Register 7.9.24.5 */
+#define PCI_EXP_DOE_WR_DATA_MBOX    0x10
+
+/* Read Data Mailbox Register 7.9.24.6 */

As below, ECN references only as that's the finalized document.

include/hw/pci/pcie_doe.h line 47

Quoted 3 lines
include/hw/pci/pcie_doe.h new file +3 -0
+#define PCI_EXP_DOE_RD_DATA_MBOX    0x14
+
+/* Table 6-31 */

Not in a published spec I think, so use references to the ECNs that have been released only.

Quoted 16 lines
include/hw/pci/pcie_doe.h new file +16 -0
+#define PCI_SIG_DOE_DISCOVERY       0x00
+#define PCI_SIG_DOE_CMA             0x01
+
+#define DATA_OBJ_BUILD_HEADER1(v, p)  ((p << 16) | v)
+
+#define PCI_DOE_MAX_DW_SIZE (1 << 18)
+#define PCI_DOE_PROTOCOL_MAX 256
+
+/*
+ * DOE Protocol - Data Object
+ */
+typedef struct DOEHeader DOEHeader;
+typedef struct DOEProtocol DOEProtocol;
+typedef struct DOECap DOECap;
+
+struct DOEHeader {

include/hw/pci/pcie_doe.h line 73

Quoted 10 lines
include/hw/pci/pcie_doe.h new file +10 -0
+    uint16_t vendor_id;
+    uint8_t doe_type;
+    uint8_t reserved;
+    uint32_t length;
+} QEMU_PACKED;
+
+/* Protocol infos and rsp function callback */
+struct DOEProtocol {
+    uint16_t vendor_id;
+    uint8_t doe_type;

It's a "data object type", not a "data object exchange type", so something like do_type or data_obj_type.
Point being the DOE is the means of exchanging data objects, not the protocol for the objects being exchanged which is simply a "Data object protocol"
(nitpick but nice to get it all to line up with the spec)

include/hw/pci/pcie_doe.h line 84

Quoted 11 lines
include/hw/pci/pcie_doe.h new file +11 -0
+    bool (*set_rsp)(DOECap *);
+};
+
+struct DOECap {
+    /* Owner */
+    PCIDevice *pdev;
+
+    /* Capability offset */
+    uint16_t offset;
+
+    /* Capability */

Giving naming, this comment and similar ones below don't tell us anything and just look like opportunities for noise / bitrot in the future. I'd drop them. When a variable name is sufficient that's great!

Quoted 36 lines
include/hw/pci/pcie_doe.h new file +36 -0
+    struct {
+        bool intr;
+        uint16_t vec;
+    } cap;
+
+    /* Control */
+    struct {
+        bool abort;
+        bool intr;
+        bool go;
+    } ctrl;
+
+    /* Status */
+    struct {
+        bool busy;
+        bool intr;
+        bool error;
+        bool ready;
+    } status;
+
+    /* Mailbox buffer for device */
+    uint32_t *write_mbox;
+    uint32_t *read_mbox;
+
+    /* Mailbox position indicator */
+    uint32_t read_mbox_idx;
+    uint32_t read_mbox_len;
+    uint32_t write_mbox_len;
+
+    /* Protocols and its callback response */
+    DOEProtocol *protocols;
+    uint16_t protocol_num;
+};
+
+/* Basic DOE functions */
+void pcie_doe_init(PCIDevice *pdev, DOECap *doe_cap, uint16_t offset,

include/hw/pci/pcie_doe.h line 130

Quoted 10 lines
include/hw/pci/pcie_doe.h new file +10 -0
+                   DOEProtocol *protocols, bool intr, uint16_t vec);
+void pcie_doe_fini(DOECap *doe_cap);
+bool pcie_doe_read_config(DOECap *doe_cap, uint32_t addr, int size,
+                          uint32_t *buf);
+void pcie_doe_write_config(DOECap *doe_cap, uint32_t addr,
+                           uint32_t val, int size);
+
+/* Utility functions for DOEProtocol */
+uint32_t pcie_doe_build_protocol(DOEProtocol *p);
+bool pcie_doe_cma_rsp(DOECap *doe_cap);

cma is a user of DOE. Great to see it, but separate patch please. Absolutely fine for a DOE mailbox to not support the CMA protocol. More importantly the CMA ECN makes it clear that an CMA supporting mailbox may not support other protocols. 6.xx.3 CMA Rules That's then relaxed to allow IDE in the IDE ECN, but that's it currently.

Side question here. Any plans on OS side for CMA?

include/hw/pci/pcie_doe.h line 137

Quoted 7 lines
include/hw/pci/pcie_doe.h new file +7 -0
+void *pcie_doe_get_req(DOECap *doe_cap);
+void pcie_doe_set_rsp(DOECap *doe_cap, void *rsp);
+uint32_t pcie_doe_object_len(void *obj);
+
+#define DOE_DEBUG
+#ifdef DOE_DEBUG
+#define DOE_DBG(...) printf(__VA_ARGS__)

Drop the debug or move to debug more in keeping with qemu code. At very least copy what is in pcie_aer and print to standard error.

Quoted 10 lines
include/hw/pci/pcie_doe.h new file +5 -0
+#else
+#define DOE_DBG(...) {}
+#endif
+
+#endif /* PCIE_DOE_H */
include/hw/pci/pcie_regs.h +0 -0
index 1db86b0ec4..963dc2e170 100644

include/hw/pci/pcie_regs.h typedef enum PCIExpLinkWidth { · line 183

Quoted 5 lines
include/hw/pci/pcie_regs.h +2 -0
@@ -179,4 +179,8 @@ typedef enum PCIExpLinkWidth {
 #define PCI_ACS_VER                     0x1
 #define PCI_ACS_SIZEOF                  8
 
+/* DOE Capability Register Fields */
+#define PCI_DOE_VER                     0x1

Clearly precedence for this, but I'm a bit confused as to why it makes sense to have a define for a version number given any changes in future 'should' be backwards compatible anyway.

Note this approach isn't universal. ATS for example just puts the 0x1
in the call to pcie_add_capability().
I'd prefer that option myself.

include/hw/pci/pcie_regs.h typedef enum PCIExpLinkWidth { · line 184

Quoted 1 line
include/hw/pci/pcie_regs.h +1 -0
+#define PCI_DOE_SIZEOF                  24

Hohum. Seems size is often int eh standard-headers Perhaps I should probably add it there in the kernel patch.

include/standard-headers/linux/pci_regs.h line 734

Quoted 13 lines
include/hw/pci/pcie_regs.h +1 -0
+
 #endif /* QEMU_PCIE_REGS_H */
include/standard-headers/linux/pci_regs.h +2 -1
index e709ae8235..4a7b7a426d 100644
@@ -730,7 +730,8 @@
 #define PCI_EXT_CAP_ID_DVSEC	0x23	/* Designated Vendor-Specific */
 #define PCI_EXT_CAP_ID_DLF	0x25	/* Data Link Feature */
 #define PCI_EXT_CAP_ID_PL_16GT	0x26	/* Physical Layer 16.0 GT/s */
-#define PCI_EXT_CAP_ID_MAX	PCI_EXT_CAP_ID_PL_16GT
+#define PCI_EXT_CAP_ID_DOE      0x2E    /* Data Object Exchange */
+#define PCI_EXT_CAP_ID_MAX	PCI_EXT_CAP_ID_DOE

As mentioned before, this header will get pulled in from linux via a script. For purposes of reviewing patches etc, pull this out to a patch before this one, allowing it to be dropped once the header is available.

Quoted 3 lines
include/standard-headers/linux/pci_regs.h +0 -0
 #define PCI_EXT_CAP_DSN_SIZEOF	12
 #define PCI_EXT_CAP_MCAST_ENDPOINT_SIZEOF 40
#3 lore raw

RFC PATCH v3 2/2 CXL DOE support for CDAT and Compliance Mode

In reply to Chris Browy
---
 hw/cxl/cxl-component-utils.c   |  93 ++++++++++++
 hw/mem/cxl_type3.c             | 184 ++++++++++++++++++++++++
 include/hw/cxl/cxl_cdat.h      | 127 +++++++++++++++++
 include/hw/cxl/cxl_compl.h     | 252 +++++++++++++++++++++++++++++++++
 include/hw/cxl/cxl_component.h |  74 ++++++++++
 include/hw/cxl/cxl_device.h    |   4 +
 include/hw/cxl/cxl_pci.h       |   2 +
 7 files changed, 736 insertions(+)
 create mode 100644 include/hw/cxl/cxl_cdat.h
 create mode 100644 include/hw/cxl/cxl_compl.h
hw/cxl/cxl-component-utils.c +93 -0
index 41d36f802a..1a2408dc76 100644
@@ -11,6 +11,7 @@
 #include "qemu/log.h"
 #include "hw/pci/pci.h"
 #include "hw/cxl/cxl.h"
+#include "qapi/error.h"
 
 static uint64_t cxl_cache_mem_read_reg(void *opaque, hwaddr offset,
                                        unsigned size)
@@ -206,3 +207,95 @@ void cxl_component_create_dvsec(CXLComponentState *cxl, uint16_t length,
     range_init_nofail(&cxl->dvsecs[type], cxl->dvsec_offset, length);
     cxl->dvsec_offset += length;
 }
+
+static void cdat_len_check(struct cdat_sub_header *hdr, Error **errp)
+{
+    assert(hdr->length);
+    assert(hdr->reserved == 0);
+
+    switch (hdr->type) {
+    case CDAT_TYPE_DSMAS:
+        assert(hdr->length == sizeof(struct cdat_dsmas));
+        break;
+    case CDAT_TYPE_DSLBIS:
+        assert(hdr->length == sizeof(struct cdat_dslbis));
+        break;
+    case CDAT_TYPE_DSMSCIS:
+        assert(hdr->length == sizeof(struct cdat_dsmscis));
+        break;
+    case CDAT_TYPE_DSIS:
+        assert(hdr->length == sizeof(struct cdat_dsis));
+        break;
+    case CDAT_TYPE_DSEMTS:
+        assert(hdr->length == sizeof(struct cdat_dsemts));
+        break;
+    case CDAT_TYPE_SSLBIS:
+        assert(hdr->length >= sizeof(struct cdat_sslbis_header));
+        assert((hdr->length - sizeof(struct cdat_sslbis_header)) %
+               sizeof(struct cdat_sslbe) == 0);
+        break;
+    default:
+        error_setg(errp, "Type %d is reserved", hdr->type);
+    }
+}
+
+#define IASL_HEADER_LEN 0x24
+void cxl_doe_cdat_init(CXLComponentState *cxl_cstate, Error **errp)
+{
+    uint8_t sum = 0;
+    int i, curr = 0, ent = 0;
+    CDATStruct cdat_st[1024];
+    struct cdat_sub_header hdr;
+    FILE *fp;
+    uint8_t iasl_hdr[IASL_HEADER_LEN];
+    size_t rcount;
+
+    fp = fopen(cxl_cstate->cdat_filename, "r");
+
+    if (fp) {
+        /* Read iASL header */
+        rcount= fread(&iasl_hdr, sizeof(iasl_hdr), 1, fp);
+	if (rcount == 0)
+            error_setg(errp, "File read failed");
+	    
+
+        for (i = 0; i < IASL_HEADER_LEN; i++) {
+            sum += iasl_hdr[i];
+        }
+        sum = (~sum + 1) & 0xFF;
+
+        curr = ftell(fp);
+
+        /* Read CDAT structures */
+        while (fread(&hdr, sizeof(hdr), 1, fp)) {
+            cdat_len_check(&hdr, errp);
+
+            cdat_st[ent].base2 = curr;
+            cdat_st[ent].length = hdr.length;
+            ent++;
+
+            fseek(fp, curr + hdr.length, SEEK_SET);
+            curr = ftell(fp);
+        }
+        /* Check the last structure */
+        fseek(fp, 0, SEEK_END);
+        assert(curr == ftell(fp));
+    } else {
+        error_setg(errp, "Please specify the CDAT file by using ',cdat='");
+    }
+    cxl_cstate->cdat_file = fp;
+
+    cxl_cstate->cdat_ent_len = ent;
+    cxl_cstate->cdat_ent = g_malloc0(sizeof(CDATStruct) * ent);
+    memcpy(cxl_cstate->cdat_ent, cdat_st, sizeof(CDATStruct) * ent);
+
+    /* Set CDAT header, ent = 0 */
+    cxl_cstate->cdat_header.revision = CXL_CDAT_REV;
+    cxl_cstate->cdat_header.sequence = 0;
+    cxl_cstate->cdat_header.length += curr + sizeof(cxl_cstate->cdat_header);
+
+    sum += cxl_cstate->cdat_header.revision +
+           cxl_cstate->cdat_header.sequence + cxl_cstate->cdat_header.length;
+
+    cxl_cstate->cdat_header.checksum = ~sum + 1;
+}
hw/mem/cxl_type3.c +184 -0

226 lines not on this page · raw message

include/hw/cxl/cxl_cdat.h new file +127 -0

127 lines not on this page · raw message

include/hw/cxl/cxl_compl.h new file +252 -0

252 lines not on this page · raw message

include/hw/cxl/cxl_component.h +74 -0

96 lines not on this page · raw message

include/hw/cxl/cxl_device.h +4 -0
index 057c5b85c6..de006ff463 100644
@@ -236,6 +236,10 @@ typedef struct cxl_type3_dev {
     /* State */
     CXLComponentState cxl_cstate;
     CXLDeviceState cxl_dstate;
+
+    /* DOE */
+    DOECap doe_comp;
+    DOECap doe_cdat;
 } CXLType3Dev;
 
 #ifndef TYPE_CXL_TYPE3_DEV
include/hw/cxl/cxl_pci.h +2 -0
index e8235b10cc..a0df324a1f 100644
@@ -12,6 +12,8 @@
 
 #include "hw/pci/pci.h"
 #include "hw/pci/pcie.h"
+#include "hw/cxl/cxl_cdat.h"
+#include "hw/cxl/cxl_compl.h"
 
 #define CXL_VENDOR_ID 0x1e98
 
Signature · 1 line
2.17.1
#4 lore raw

Re: [RFC PATCH v3 cxl-2.0-doe 2/2] CXL DOE support for CDAT and Compliance Mode

On Tue, 9 Mar 2021 16:04:36 -0500
Chris Browy <[email protected]> wrote:

Hi Chris,

As for patch 1, description needed here. Let's move this series towards the form needed for a final submission.

2 features, 2 patches... If nothing else makes each one small enough to be suitable for review with morning coffee :)

Quoted 16 lines
---
 hw/cxl/cxl-component-utils.c   |  93 ++++++++++++
 hw/mem/cxl_type3.c             | 184 ++++++++++++++++++++++++
 include/hw/cxl/cxl_cdat.h      | 127 +++++++++++++++++
 include/hw/cxl/cxl_compl.h     | 252 +++++++++++++++++++++++++++++++++
 include/hw/cxl/cxl_component.h |  74 ++++++++++
 include/hw/cxl/cxl_device.h    |   4 +
 include/hw/cxl/cxl_pci.h       |   2 +
 7 files changed, 736 insertions(+)
 create mode 100644 include/hw/cxl/cxl_cdat.h
 create mode 100644 include/hw/cxl/cxl_compl.h
hw/cxl/cxl-component-utils.c +0 -0
index 41d36f802a..1a2408dc76 100644

...

Quoted 32 lines
hw/cxl/cxl-component-utils.c +32 -0
+void cxl_doe_cdat_init(CXLComponentState *cxl_cstate, Error **errp)
+{
+    uint8_t sum = 0;
+    int i, curr = 0, ent = 0;
+    CDATStruct cdat_st[1024];
+    struct cdat_sub_header hdr;
+    FILE *fp;
+    uint8_t iasl_hdr[IASL_HEADER_LEN];
+    size_t rcount;
+
+    fp = fopen(cxl_cstate->cdat_filename, "r");
+
+    if (fp) {
+        /* Read iASL header */
+        rcount= fread(&iasl_hdr, sizeof(iasl_hdr), 1, fp);
+	if (rcount == 0)
+            error_setg(errp, "File read failed");
+	    
+
+        for (i = 0; i < IASL_HEADER_LEN; i++) {
+            sum += iasl_hdr[i];
+        }
+        sum = (~sum + 1) & 0xFF;
+
+        curr = ftell(fp);
+
+        /* Read CDAT structures */
+        while (fread(&hdr, sizeof(hdr), 1, fp)) {
+            cdat_len_check(&hdr, errp);
+
+            cdat_st[ent].base2 = curr;
+            cdat_st[ent].length = hdr.length;

hw/cxl/cxl-component-utils.c void cxl_component_create_dvsec(CXLComponentState *cxl, uint16_t length, · line 284

Quoted 10 lines
hw/cxl/cxl-component-utils.c +10 -0
+            ent++;
+
+            fseek(fp, curr + hdr.length, SEEK_SET);
+            curr = ftell(fp);
+        }
+        /* Check the last structure */
+        fseek(fp, 0, SEEK_END);
+        assert(curr == ftell(fp));
+    } else {
+        error_setg(errp, "Please specify the CDAT file by using ',cdat='");

It definitely wants to be optional. Fine to not provide one. I'd hope we can generate one that makes 'sense' for whatever memory etc is configured though and have that as the fallback if one isn't provided. That will be a lot more convenient if people are working on other aspects of support and don't happen to care about exactly what is in CDAT on a given day.

Quoted 57 lines
hw/cxl/cxl-component-utils.c +17 -0
+    }
+    cxl_cstate->cdat_file = fp;
+
+    cxl_cstate->cdat_ent_len = ent;
+    cxl_cstate->cdat_ent = g_malloc0(sizeof(CDATStruct) * ent);
+    memcpy(cxl_cstate->cdat_ent, cdat_st, sizeof(CDATStruct) * ent);
+
+    /* Set CDAT header, ent = 0 */
+    cxl_cstate->cdat_header.revision = CXL_CDAT_REV;
+    cxl_cstate->cdat_header.sequence = 0;
+    cxl_cstate->cdat_header.length += curr + sizeof(cxl_cstate->cdat_header);
+
+    sum += cxl_cstate->cdat_header.revision +
+           cxl_cstate->cdat_header.sequence + cxl_cstate->cdat_header.length;
+
+    cxl_cstate->cdat_header.checksum = ~sum + 1;
+}
hw/mem/cxl_type3.c +32 -0
index bf33ddb915..33f571e1a5 100644
@@ -13,6 +13,156 @@
 #include "qemu/rcu.h"
 #include "sysemu/hostmem.h"
 #include "hw/cxl/cxl.h"
+#include "hw/pci/msi.h"
+#include "hw/pci/msix.h"
+
+uint32_t cxl_doe_compliance_init(DOECap *doe_cap)
+{
+    CXLComponentState *cxl_cstate = &CT3(doe_cap->pdev)->cxl_cstate;
+    uint32_t req;
+    uint32_t byte_cnt = 0;
+
+    DOE_DBG(">> %s\n",  __func__);
+
+    req = ((struct cxl_compliance_mode_cap *)pcie_doe_get_req(doe_cap))
+        ->req_header.req_code;
+    switch (req) {
+    case CXL_COMP_MODE_CAP:
+        byte_cnt = sizeof(struct cxl_compliance_mode_cap_rsp);
+        cxl_cstate->doe_resp.cap_rsp.header.vendor_id = CXL_VENDOR_ID;
+        cxl_cstate->doe_resp.cap_rsp.header.doe_type = CXL_DOE_COMPLIANCE;
+        cxl_cstate->doe_resp.cap_rsp.header.reserved = 0x0;
+        cxl_cstate->doe_resp.cap_rsp.header.length =
+            DIV_ROUND_UP(sizeof(struct cxl_compliance_mode_cap_rsp), 4);
+        cxl_cstate->doe_resp.cap_rsp.rsp_header.rsp_code = 0x0;
+        cxl_cstate->doe_resp.cap_rsp.rsp_header.version = 0x1;
+        cxl_cstate->doe_resp.cap_rsp.rsp_header.length = 0x1c;
+        cxl_cstate->doe_resp.cap_rsp.status = 0x0;
+        cxl_cstate->doe_resp.cap_rsp.available_cap_bitmask = 0x3;
+        cxl_cstate->doe_resp.cap_rsp.enabled_cap_bitmask = 0x3;
+        break;
+    case CXL_COMP_MODE_STATUS:
+        byte_cnt = sizeof(struct cxl_compliance_mode_status_rsp);
+        cxl_cstate->doe_resp.status_rsp.header.vendor_id = CXL_VENDOR_ID;
+        cxl_cstate->doe_resp.status_rsp.header.doe_type = CXL_DOE_COMPLIANCE;

hw/mem/cxl_type3.c line 57

Quoted 10 lines
hw/mem/cxl_type3.c +10 -0
+        cxl_cstate->doe_resp.status_rsp.header.reserved = 0x0;
+        cxl_cstate->doe_resp.status_rsp.header.length =
+            DIV_ROUND_UP(sizeof(struct cxl_compliance_mode_status_rsp), 4);
+        cxl_cstate->doe_resp.status_rsp.rsp_header.rsp_code = 0x1;
+        cxl_cstate->doe_resp.status_rsp.rsp_header.version = 0x1;
+        cxl_cstate->doe_resp.status_rsp.rsp_header.length = 0x14;
+        cxl_cstate->doe_resp.status_rsp.cap_bitfield = 0x3;
+        cxl_cstate->doe_resp.status_rsp.cache_size = 0;
+        cxl_cstate->doe_resp.status_rsp.cache_size_units = 0;
+        break;

Given there are only two types in reality. Can we get rid of all the others in the big union? Right now it looks like those aren't handled, but reality is they are the same as the ones here.

Quoted 71 lines
hw/mem/cxl_type3.c +71 -0
+    default:
+        break;
+    }
+
+    DOE_DBG("  REQ=%x, RSP BYTE_CNT=%d\n", req, byte_cnt);
+    DOE_DBG("<< %s\n",  __func__);
+    return byte_cnt;
+}
+
+
+bool cxl_doe_compliance_rsp(DOECap *doe_cap)
+{
+    CXLComponentState *cxl_cstate = &CT3(doe_cap->pdev)->cxl_cstate;
+    uint32_t byte_cnt;
+    uint32_t dw_cnt;
+
+    DOE_DBG(">> %s\n",  __func__);
+
+    byte_cnt = cxl_doe_compliance_init(doe_cap);
+    dw_cnt = byte_cnt / 4;
+    memcpy(doe_cap->read_mbox,
+           cxl_cstate->doe_resp.data_byte, byte_cnt);
+
+    doe_cap->read_mbox_len += dw_cnt;
+
+    DOE_DBG("  LEN=%x, RD MBOX[%d]=%x\n", dw_cnt - 1,
+            doe_cap->read_mbox_len,
+            *(doe_cap->read_mbox + dw_cnt - 1));
+
+    DOE_DBG("<< %s\n",  __func__);
+
+    return true;
+}
+
+bool cxl_doe_cdat_rsp(DOECap *doe_cap)
+{
+    CXLComponentState *cxl_cstate = &CT3(doe_cap->pdev)->cxl_cstate;
+    uint16_t ent;
+    uint32_t base = 0;
+    uint32_t len = 0 ;
+    size_t rcount;
+    struct cxl_cdat *req = pcie_doe_get_req(doe_cap);
+
+    ent = req->entry_handle;
+    if (ent == 0) {
+        len = sizeof(cxl_cstate->cdat_header);
+    } else if (ent <= cxl_cstate->cdat_ent_len) {
+        base = cxl_cstate->cdat_ent[ent - 1].base2;
+        len = cxl_cstate->cdat_ent[ent - 1].length;
+    }
+
+    struct cxl_cdat_rsp rsp = {
+        .header = {
+            .vendor_id = CXL_VENDOR_ID,
+            .doe_type = CXL_DOE_TABLE_ACCESS,
+            .reserved = 0x0,
+            .length = (sizeof(struct cxl_cdat_rsp) + len) / 4,
+        },
+        .req_code = CXL_DOE_TAB_RSP,
+        .table_type = CXL_DOE_TAB_TYPE_CDAT,
+        .entry_handle = (ent < cxl_cstate->cdat_ent_len) ? ent + 1 : CXL_DOE_TAB_ENT_MAX,
+    };
+
+    memcpy(doe_cap->read_mbox, &rsp, sizeof(rsp));
+
+    if (ent == 0) {
+        memcpy(doe_cap->read_mbox + sizeof(rsp) / 4,
+               &cxl_cstate->cdat_header, len);
+    } else if (ent <= cxl_cstate->cdat_ent_len) {
+        fseek(cxl_cstate->cdat_file, base, SEEK_SET);
+        rcount= fread(doe_cap->read_mbox + sizeof(rsp) / 4, len, 1, cxl_cstate->cdat_file);

hw/mem/cxl_type3.c line 138

Quoted 10 lines
hw/mem/cxl_type3.c +10 -0
+	if (rcount == 0)
+	    DOE_DBG("fread returned 0\n");
+    }
+
+    doe_cap->read_mbox_len += rsp.header.length;
+    DOE_DBG("%s: INCR RD_MBOX OFF=%x\n", __func__, doe_cap->read_mbox_len);
+
+    for (int i = 0; i < doe_cap->read_mbox_len; i++) {
+        DOE_DBG("  RD_MBOX[%d]=%08x\n", i, doe_cap->read_mbox[i]);
+    }

It's trivial to sanity check this isn't broken by just doing the read, so I'd drop the debug now you have it working.

Quoted 10 lines
hw/mem/cxl_type3.c +4 -0
+
+    return true;
+}
+
 
 
 
 static void build_dvsecs(CXLType3Dev *ct3d)
 {

hw/mem/cxl_type3.c static void ct3_finalize(Object *obj) · line 288

Quoted 5 lines
hw/mem/cxl_type3.c +2 -0
@@ -134,6 +284,8 @@ static void ct3_finalize(Object *obj)
 
     g_free((void *)regs->special_ops);
     g_free(ct3d->cxl_dstate.pmem);
+
+    fclose(cxl_cstate->cdat_file);

cdat will be small, perhaps we just read it in once and cache it?
That lets us not keep the file handle around or the file open once we have read it.

hw/mem/cxl_type3.c static MemoryRegion *cxl_md_get_memory_region(MemoryDeviceState *md, · line 361

Quoted 11 lines
hw/mem/cxl_type3.c +4 -0
 }
 
 static void cxl_setup_memory(CXLType3Dev *ct3d, Error **errp)
@@ -203,6 +355,19 @@ static MemoryRegion *cxl_md_get_memory_region(MemoryDeviceState *md,
     return ct3d->cxl_dstate.pmem;
 }
 
+static DOEProtocol doe_comp_prot[] = {
+    {PCI_VENDOR_ID_PCI_SIG, PCI_SIG_DOE_CMA, pcie_doe_cma_rsp},
+    {CXL_VENDOR_ID, CXL_DOE_COMPLIANCE, cxl_doe_compliance_rsp},
+    {CXL_VENDOR_ID, CXL_DOE_TABLE_ACCESS, cxl_doe_cdat_rsp},

This is kind of useful for exercising software paths, but I'm not sure why any real hardware would do this.

Generic software is likely to stop at the first DOE that supports what it wants, so it'll never reach the CDAT only one.

hw/mem/cxl_type3.c static MemoryRegion *cxl_md_get_memory_region(MemoryDeviceState *md, · line 366

Quoted 5 lines
hw/mem/cxl_type3.c +5 -0
+    { /* End of array */ },
+};
+
+static DOEProtocol doe_cdat_prot[] = {
+    {PCI_VENDOR_ID_PCI_SIG, PCI_SIG_DOE_CMA, pcie_doe_cma_rsp},

Not allowed to do that according to CMA ECN.

Quoted 60 lines
hw/mem/cxl_type3.c +23 -0
+    {CXL_VENDOR_ID, CXL_DOE_TABLE_ACCESS, cxl_doe_cdat_rsp},
+    { /* End of array */ },
+};
+
 static void ct3_realize(PCIDevice *pci_dev, Error **errp)
 {
     CXLType3Dev *ct3d = CT3(pci_dev);
@@ -210,6 +375,8 @@ static void ct3_realize(PCIDevice *pci_dev, Error **errp)
     ComponentRegisters *regs = &cxl_cstate->crb;
     MemoryRegion *mr = &regs->component_registers;
     uint8_t *pci_conf = pci_dev->config;
+    unsigned short msix_num = 2;
+    int i;
 
     if (!ct3d->cxl_dstate.pmem) {
         cxl_setup_memory(ct3d, errp);
@@ -239,6 +406,19 @@ static void ct3_realize(PCIDevice *pci_dev, Error **errp)
                      PCI_BASE_ADDRESS_SPACE_MEMORY |
                          PCI_BASE_ADDRESS_MEM_TYPE_64,
                      &ct3d->cxl_dstate.device_registers);
+
+    /* MSI(-X) Initailization */
+    msix_init_exclusive_bar(pci_dev, msix_num, 4, NULL);
+    for (i = 0; i < msix_num; i++) {
+        msix_vector_use(pci_dev, i);
+    }
+    /* msi_init(pci_dev, 0x60, 16, true, false, NULL); */
+
+    /* DOE Initailization */
+    pcie_doe_init(pci_dev, &ct3d->doe_comp, 0x160, doe_comp_prot, true, 0);
+    pcie_doe_init(pci_dev, &ct3d->doe_cdat, 0x190, doe_cdat_prot, true, 1);
+
+    cxl_doe_cdat_init(cxl_cstate, errp);
 }
 
 static uint64_t cxl_md_get_addr(const MemoryDeviceState *md)
@@ -275,6 +455,7 @@ static Property ct3_props[] = {
                      HostMemoryBackend *),
     DEFINE_PROP_LINK("lsa", CXLType3Dev, lsa, TYPE_MEMORY_BACKEND,
                      HostMemoryBackend *),
+    DEFINE_PROP_STRING("cdat", CXLType3Dev, cxl_cstate.cdat_filename),
     DEFINE_PROP_END_OF_LIST(),
 };
 
@@ -357,6 +538,9 @@ static void ct3_class_init(ObjectClass *oc, void *data)
     DeviceClass *dc = DEVICE_CLASS(oc);
     PCIDeviceClass *pc = PCI_DEVICE_CLASS(oc);
     MemoryDeviceClass *mdc = MEMORY_DEVICE_CLASS(oc);
+
+    pc->config_write = ct3d_config_write;
+    pc->config_read = ct3d_config_read;
     CXLType3Class *cvc = CXL_TYPE3_DEV_CLASS(oc);
 
     pc->realize = ct3_realize;
include/hw/cxl/cxl_cdat.h new file +0 -0
new file mode 100644
index 0000000000..edd994d410
@@ -0,0 +1,127 @@

...

include/hw/cxl/cxl_cdat.h line 35

Quoted 11 lines
include/hw/cxl/cxl_cdat.h new file +11 -0
+
+/* Read Entry Request, 8.1.11.1 Table 134 */
+struct cxl_cdat {
+    DOEHeader header;
+    uint8_t req_code;
+    uint8_t table_type;
+    uint16_t entry_handle;
+} QEMU_PACKED;
+
+/* Read Entry Response, 8.1.11.1 Table 135 */
+#define cxl_cdat_rsp    cxl_cdat    /* Same as request */

The header is the same as the request, but lots of data that this is implying doesn't exist. Also it doesn't have a request code so at very least just make that field code in the structure.

include/hw/cxl/cxl_cdat.h line 45

Quoted 10 lines
include/hw/cxl/cxl_cdat.h new file +10 -0
+
+/*
+ * CDAT Table Structure (CDAT Spec)
+ */
+#define CXL_CDAT_REV 1
+
+/* Data object header */
+struct cdat_table_header {
+    uint32_t length;    /* Length of table in bytes, including this header */
+    uint8_t revision;   /* ACPI Specification minor version number */

No. CDAT specification revision number. This isn't an ACPI table.

include/hw/cxl/cxl_cdat.h line 48

Quoted 3 lines
include/hw/cxl/cxl_cdat.h new file +3 -0
+    uint8_t checksum;   /* To make sum of entire table == 0 */
+    uint8_t reserved[6];
+    uint32_t sequence;  /* ASCII table signature */

Not what the sequence number is at all.

Quoted 2 lines
include/hw/cxl/cxl_cdat.h new file +2 -0
+} QEMU_PACKED;
+

...

include/hw/cxl/cxl_compl.h line 4

Quoted 11 lines
include/hw/cxl/cxl_cdat.h new file +1 -0
+#endif /* CXL_CDAT_H */
include/hw/cxl/cxl_compl.h new file +4 -0
new file mode 100644
index 0000000000..5edeaa5f27
@@ -0,0 +1,252 @@
+/*
+ * CXL Compliance Structure
+ *
+ * Copyright (C) 2020 Avery Design Systems, Inc.

Time for an update to include 2021.

Quoted 4 lines
include/hw/cxl/cxl_compl.h new file +4 -0
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ */

...

include/hw/cxl/cxl_compl.h line 56

Quoted 8 lines
include/hw/cxl/cxl_compl.h new file +8 -0
+
+struct status_rsp {
+    DOEHeader header;
+    CompRspHeader rsp_header;
+    uint8_t status;
+} QEMU_PACKED;
+
+struct len_rsvd_rsp {

Prefix these so they don't look like generic things.
Also for this one, the naming made me wonder if CXL really had defined a length field then marked it reserved (which would be odd). I would go with two versions of the header and name this differently.

struct compliance_rsp_header_no_len {
    uint8_t rsp_code;
    uint8_t version;
} QEMU_PACKED;

struct comp_no_data_resp {
DOEHeader header;
struct compliance_rsp_header_no_len resp_header; /* typedefs if you like are fine */ uint8_t reserved[6];
} QEMU_PACKED;

Quoted 6 lines
include/hw/cxl/cxl_compl.h new file +6 -0
+    DOEHeader header;
+    /* The length field in rsp_header is reserved. */
+    CompRspHeader rsp_header;
+    uint8_t reserved[5];
+} QEMU_PACKED;
+

...

include/hw/cxl/cxl_compl.h line 188

Quoted 6 lines
include/hw/cxl/cxl_compl.h new file +6 -0
+
+/* 14.16.4.10 */
+struct cxl_compliance_mode_toggle_cache_flush {
+    DOEHeader header;
+    CompReqHeader req_header;
+    uint8_t inj_flow_control;

?

Quoted 4 lines
include/hw/cxl/cxl_compl.h new file +4 -0
+} QEMU_PACKED;
+
+#define cxl_compliance_mode_toggle_cache_flush_rsp status_rsp
+

...

Quoted 15 lines
include/hw/cxl/cxl_compl.h new file +2 -0
+
+#endif /* CXL_COMPL_H */
include/hw/cxl/cxl_component.h +1 -0
index acc0730d96..1294d74d78 100644
@@ -18,6 +18,7 @@
 #include "qemu/range.h"
 #include "qemu/typedefs.h"
 #include "hw/register.h"
+#include "qapi/error.h"
 
 enum reg_type {
     CXL2_DEVICE,

include/hw/cxl/cxl_component.h typedef struct component_registers { · line 166

Quoted 6 lines
include/hw/cxl/cxl_component.h +3 -0
@@ -160,6 +161,11 @@ typedef struct component_registers {
     MemoryRegionOps *special_ops;
 } ComponentRegisters;
 
+typedef struct cdat_struct {
+    void *base;
+    uint32_t base2;

Given base vs base2 isn't obvious naming, add some comments or give it a more meaningful name.

Quoted 7 lines
include/hw/cxl/cxl_component.h +2 -0
+    uint32_t length;
+} CDATStruct;
 /*
  * A CXL component represents all entities in a CXL hierarchy. This includes,
  * host bridges, root ports, upstream/downstream switch ports, and devices
@@ -173,6 +179,70 @@ typedef struct cxl_component {
             struct PCIDevice *pdev;

include/hw/cxl/cxl_component.h typedef struct cxl_component { · line 189

Quoted 10 lines
include/hw/cxl/cxl_component.h +8 -0
         };
     };
+
+    /*
+     * SW write write mailbox and GO on last DW
+     * device sets READY of offset DW in DO types to copy
+     * to read mailbox register on subsequent cfg_read
+     * of read mailbox register and then on cfg_write to
+     * denote success read increments offset to next DW
+     */

Comment doesn't seem to have anything to do with any code around here, so drop it.

include/hw/cxl/cxl_component.h typedef struct cxl_component { · line 192

Quoted 3 lines
include/hw/cxl/cxl_component.h +3 -0
+
+    union doe_request_u {
+        /* Compliance DOE Data Objects Type=0*/

Space before */

Quoted 8 lines
include/hw/cxl/cxl_component.h +8 -0
+        struct cxl_compliance_mode_cap mode_cap;
+        struct cxl_compliance_mode_status mode_status;
+        struct cxl_compliance_mode_halt mode_halt;
+        struct cxl_compliance_mode_multiple_write_streaming
+            multiple_write_streaming;
+        struct cxl_compliance_mode_producer_consumer producer_consumer;
+        struct cxl_compliance_mode_inject_bogus_writes inject_bogus_writes;
+        struct cxl_compliance_mode_inject_poison inject_poison;

include/hw/cxl/cxl_component.h typedef struct cxl_component { · line 210

Quoted 10 lines
include/hw/cxl/cxl_component.h +10 -0
+        struct cxl_compliance_mode_inject_crc inject_crc;
+        struct cxl_compliance_mode_inject_flow_control inject_flow_control;
+        struct cxl_compliance_mode_toggle_cache_flush toggle_cache_flush;
+        struct cxl_compliance_mode_inject_mac_delay inject_mac_delay;
+        struct cxl_compliance_mode_insert_unexp_mac insert_unexp_mac;
+        struct cxl_compliance_mode_inject_viral inject_viral;
+        struct cxl_compliance_mode_inject_almp inject_almp;
+        struct cxl_compliance_mode_ignore_almp ignore_almp;
+        struct cxl_compliance_mode_inject_bit_error ignore_bit_error;
+        char data_byte[128];

We aren't accessing this as a byte anywhere, so just use a pointer to the whole structure. That will avoid padding this to a meaningless length. (closer inspection this one isn't even referenced, but resp below is)

include/hw/cxl/cxl_component.h typedef struct cxl_component { · line 214

Quoted 4 lines
include/hw/cxl/cxl_component.h +4 -0
+    } doe_request;
+
+    union doe_resp_u {
+        /* Compliance DOE Data Objects Type=0*/

Space before */

include/hw/cxl/cxl_component.h typedef struct cxl_component { · line 215

Quoted 1 line
include/hw/cxl/cxl_component.h +1 -0
+        struct cxl_compliance_mode_cap_rsp cap_rsp;

I wonder if we can make this all shorter by using an abbreviation?

cxl_cmpl_mode_stat_rsp

and similar?

It's nice to match spec names where possible, but sometimes they just end up as stupidly long/

Quoted 10 lines
include/hw/cxl/cxl_component.h +10 -0
+        struct cxl_compliance_mode_status_rsp status_rsp;
+        struct cxl_compliance_mode_halt_rsp halt_rsp;
+        struct cxl_compliance_mode_multiple_write_streaming_rsp
+            multiple_write_streaming_rsp;
+        struct cxl_compliance_mode_producer_consumer_rsp producer_consumer_rsp;
+        struct cxl_compliance_mode_inject_bogus_writes_rsp
+            inject_bogus_writes_rsp;
+        struct cxl_compliance_mode_inject_poison_rsp inject_poison_rsp;
+        struct cxl_compliance_mode_inject_crc_rsp inject_crc_rsp;
+        struct cxl_compliance_mode_inject_flow_control_rsp

include/hw/cxl/cxl_component.h typedef struct cxl_component { · line 235

Quoted 10 lines
include/hw/cxl/cxl_component.h +10 -0
+            inject_flow_control_rsp;
+        struct cxl_compliance_mode_toggle_cache_flush_rsp
+            toggle_cache_flush_rsp;
+        struct cxl_compliance_mode_inject_mac_delay_rsp inject_mac_delay_rsp;
+        struct cxl_compliance_mode_insert_unexp_mac_rsp insert_unexp_mac_rsp;
+        struct cxl_compliance_mode_inject_viral inject_viral_rsp;
+        struct cxl_compliance_mode_inject_almp_rsp inject_almp_rsp;
+        struct cxl_compliance_mode_ignore_almp_rsp ignore_almp_rsp;
+        struct cxl_compliance_mode_inject_bit_error_rsp ignore_bit_error_rsp;
+        char data_byte[520 * 4];

As above, I can't see a reason to have this. If individual structures can be variable length, put the maximums in them, not here.

Quoted 50 lines
include/hw/cxl/cxl_component.h +14 -0
+    } doe_resp;
+
+    /* Table entry types */
+    struct cdat_table_header cdat_header;
+
+    CDATStruct *cdat_ent;
+    int cdat_ent_len;
+
+    char *cdat_filename;
+    FILE *cdat_file;
 } CXLComponentState;
 
 void cxl_component_register_block_init(Object *obj,
@@ -184,4 +254,8 @@ void cxl_component_register_init_common(uint32_t *reg_state,
 void cxl_component_create_dvsec(CXLComponentState *cxl_cstate, uint16_t length,
                                 uint16_t type, uint8_t rev, uint8_t *body);
 
+uint32_t cxl_doe_compliance_init(DOECap *doe_cap);
+bool cxl_doe_compliance_rsp(DOECap *doe_cap);
+void cxl_doe_cdat_init(CXLComponentState *cxl_cstate, Error **errp);
+bool cxl_doe_cdat_rsp(DOECap *doe_cap);
 #endif
include/hw/cxl/cxl_device.h +4 -0
index 057c5b85c6..de006ff463 100644
@@ -236,6 +236,10 @@ typedef struct cxl_type3_dev {
     /* State */
     CXLComponentState cxl_cstate;
     CXLDeviceState cxl_dstate;
+
+    /* DOE */
+    DOECap doe_comp;
+    DOECap doe_cdat;
 } CXLType3Dev;
 
 #ifndef TYPE_CXL_TYPE3_DEV
include/hw/cxl/cxl_pci.h +2 -0
index e8235b10cc..a0df324a1f 100644
@@ -12,6 +12,8 @@
 
 #include "hw/pci/pci.h"
 #include "hw/pci/pcie.h"
+#include "hw/cxl/cxl_cdat.h"
+#include "hw/cxl/cxl_compl.h"
 
 #define CXL_VENDOR_ID 0x1e98
 
#5 lore raw
In reply to Chris Browy
Quoted 23 lines · On Tue, 9 Mar 2021 15:33:49 -0500 Chris Browy <[email protected]> wrote:

Version 3 patch series for PCIe DOE for PCIe and CXL 2.0 implements all planned functionality.

Based on QEMU version:
https://gitlab.com/bwidawsk/qemu/-/tree/cxl-2.0v4

Summary:
1: PCIe DOE support for Discovery
- Support multiple DOE instances for each own protocol set
- MSI-X and polling supported
- Update error and interrupt status in DOE Status register
- Use static array to register callback function for DOE protocols
- Deprecate DOE_SUCCESS and DOE_DISCARD
- Add license headers
2: CXL DOE for CDAT and Compliance Mode.
- Device supports pre-defined CDAT or user-provided CDAT.
- Support on reading the iASL aml file via specifying "cdat=<filename.aml>" property to -device cxl-type3 skips over the ACPI header and writes only CDAT table entries
- Clean up CXL compliance structures
- DOE CDAT response returns one CDAT Structure instance based on request EntryHandle value.

Quoted 13 lines

Example cdat.dat file: (compile with iasl -G cdat.dat)
CDAT file may contain any mix and number of supported CDAT Structure types
----------------------
/* Header */
Signature : "CDAT"
Table Length : 00000000
Revision : 01
Checksum : 00
Oem ID : AVERY
Oem Table ID : 0
Oem Revision : 1
Asl Compiler ID : "INTL"
Asl Compiler Revision : 20160527

I think this is a non starter because it's not an ACPI table and the header of CDAT just doesn't look like this.

We 'might' be able to add some special handling to iASL as a convenient tool for generating cdat, or we might just write something CDAT specific (it's pretty simple after all!)

Normally iASL will convert a static table into it's raw binary representation anyway, so we want to do something like that but end up with a valid CDAT table.

Jonathan

Quoted 107 lines

/* CDAT structures */
Label : DSMAS // Field Byte Length
UINT8 : 0 // Type 1
UINT8 : 00 // Reserved 1
UINT16 : 0018 // Length 2
UINT8 : 00 // DSMADHandle 1
UINT8 : 00 // Flags 1
UINT16 : 0000 // Reserved 2
UINT64 : 0000000000000000 // DPA Base 8
UINT64 : ffffffffffffffff // DPA Length 8

Label : DSLBIS // Field Byte Length
UINT8 : 01 // Type 1
UINT8 : 00 // Reserved 1
UINT16 : 0018 // Length 2
UINT8 : 00 // Handle 1
UINT8 : 00 // Flags 1
UINT8 : 00 // Data Type 1
UINT8 : 00 // Reserved 1
UINT64 : 0000000000000000 // Entry Base Unit 8
UINT16 : 0000 // Entry[0] 2
UINT16 : 0000 // Entry[1] 2
UINT16 : 0000 // Entry[2] 2
UINT16 : 0000 // Reserved 2

Label: DSMSCIS // Field Byte Length
UINT8 : 02 // Type 1
UINT8 : 00 // Reserved 1
UINT16 : 0014 // Length 2
UINT8 : 00 // DSMASHandle 1
UINT24 : 000000 // Reserved 3
UINT64 : 0000000000000000 // Memory Side Cache Size 8
UINT32 : 00000000 // Cache Attributes 4

Label : DSIS // Field Byte Length
UINT8 : 03 // Type 1
UINT8 : 00 // Reserved 1
UINT16 : 0008 // Length 2
UINT8 : 00 // Flags 1
UINT8 : 00 // Handle 1
UINT16 : 0000 // Reserved 2

Label : DSEMTS // Field Byte Length
UINT8 : 04 // Type 1
UINT8 : 00 // Reserved 1
UINT16 : 0018 // Length 2
UINT8 : 00 // DSMAS Handle 1
UINT8 : 00 // EFI Memory Type and Attribute 1
UINT16 : 0000 // Reserved 2
UINT64 : 0000000000000000 // DPA Offset 8
UINT64 : 0000000000000000 // DPA Length 8

Label : SSLBIS // Field Byte Length
UINT8 : 05 // Type 1
UINT8 : 00 // Reserved 1
UINT16 : 0020 // Length 2
UINT8 : 00 // Data Type 1
UINT24 : 000000 // Reserved 3
UINT64 : 0000000000000000 // Entry Base Unit 8
Label : SSLBE // SSLBE[0]
UINT16 : 0000 // Port X ID 2
UINT16 : 0000 // Port Y ID 2
UINT16 : 0000 // Latency or Bandwidth 2
UINT16 : BBBB // Reserved 2
Label : SSLBE // SSLBE[1]
UINT16 : 0000 // Port X ID 2
UINT16 : 0000 // Port Y ID 2
UINT16 : 0000 // Latency or Bandwidth 2
UINT16 : BBBC // Reserved 2
----

References:
1. CXL 2.0 specification
https://www.computeexpresslink.org/download-the-specification
2. PCI-SIG ECN: Data Object Exchange (DOE)
http://www.pcisig.com
3. Coherent Device Attribute Table CDAT 1.02
https://uefi.org/sites/default/files/resources/Coherent%20Device%20Attribute%20Table_1.02.pdf

---
Chris Browy (2):
Basic PCIe DOE support
CXL DOE support for CDAT and Compliance Mode

 MAINTAINERS                               |  49 +--
 hw/cxl/cxl-component-utils.c              |  93 +++++
 hw/mem/cxl_type3.c                        | 184 ++++++++++
 hw/pci/meson.build                        |   1 +
 hw/pci/pci.c                              |  13 +-
 hw/pci/pcie_doe.c                         | 416 ++++++++++++++++++++++
 include/hw/cxl/cxl_cdat.h                 | 127 +++++++
 include/hw/cxl/cxl_compl.h                | 252 +++++++++++++
 include/hw/cxl/cxl_component.h            |  74 ++++
 include/hw/cxl/cxl_device.h               |   4 +
 include/hw/cxl/cxl_pci.h                  |   2 +
 include/hw/pci/pci_ids.h                  |   5 +-
 include/hw/pci/pcie.h                     |   1 +
 include/hw/pci/pcie_doe.h                 | 142 ++++++++
 include/hw/pci/pcie_regs.h                |   4 +
 include/standard-headers/linux/pci_regs.h |   3 +-
 16 files changed, 1327 insertions(+), 43 deletions(-)
 create mode 100644 hw/pci/pcie_doe.c
 create mode 100644 include/hw/cxl/cxl_cdat.h
 create mode 100644 include/hw/cxl/cxl_compl.h
 create mode 100644 include/hw/pci/pcie_doe.h