CXL FMAPI interface over MCTP/I2C
cover CXL FMAPI interface over MCTP/I2C
A quick PoC to show the feasibility of emulating the CXL 2.0 Fabric Manager API over MCTP over I2C as possible path to standards based configuration of emulated CXL switches. Lots of missing parts (such as hooking this up to the upstream ports or making it actually do anything useful), but it's enough to establish this as a plausible approach. Purpose of sharing this is mostly to let people know I've been messing around with this (and that Klaus' work was really helpful :)
The end goal is to enable emulation of more sophisticated CXL switches that rely on an external agent (fabric manager) configuring them. One likely common way of doing this is to use the DMTF defined CXL FMAPI over MCTP binding (message type 7). Whilst we don't have multiple hosts in QEMU, that doesn't matter for exercising both the switch configuration elements, and the host handling of the resulting events (we can have multiple virtual hierarchies via multiple CXL root ports - similar to multiple hosts, but with a few additional software complexities). Note there are a load of other protocols that we will want to emulate in future (such as SPDM) but this one was a simple place to start. Note QEMU emulation of simple CXL switches is still under review (queued behind ARM support for CXL due to some fuzz in test cases).
https://lore.kernel.org/qemu-devel/[email protected]/
FMAPI allows:
* Management software to query switch state and capabilities (part of this is implemented in this RFC)
* Configuration of binding of physical ports to particular virtual hierarchies (PCIe topologies visible to the host).
* Configuration of the binding of MLD (Multiple Logical Device) LDs to particular virtual ports within a Virtual Hierarchy (how CXL 2.0 handles memory pooling).
* Lots of other stuff.
Details in the CXL 2.0 specification and ECNs.
FMAPI could be enabled via an external interface (and might be in some later patch set) but the recent RFC for I2C slave mode support
https://lore.kernel.org/qemu-devel/[email protected]/
provided the option to simply expose an MCTP interface to the guest. (normally it would be exported to a BMC or similar, but meh, that doesn't
matter for testing :) Ultimately emulation of all the common interfaces will be needed to provide a broad test platform.
Note this use of the above patch set is rather different to using it to communicate between two emulated machines, one of which is running firmware for an appropriate device. We could do that for CXL switches, but the tight coupling we will want to the other parts being emulated in QEMU would make such an approach challenging. Also the FMAPI is simple enough to lend itself to implementation as a state machine as part of the device emulation.
Given I wanted a simple test platform without separate firmware requirements I've added the aspeed-i2c controller to the arm virt image. There are some quirks around this such as the need for reset controller but overall the support is straight forward. A snag here is that for rest of CXL emulation I need to use ACPI and the aspeed-i2c driver in Linux is currently not very ACPI friendly and it may be controversial to make it so.
To bring up the connection:
* Use a kernel with appropriate i2c controller and mctp driver support. (5.18-rc1 or later should work - though there are some bugs that can be if you follow particular wrong configuration flows).
* Get mctp and mctpd from https://github.com/CodeConstruct/mctp
- modify the mctpd systemd init as provided and start the daemon.
- for 64 bit support, get latest version as a sizeof() bug was identified whilst testing this.
# Bring up the link
mctp link set mctpi2c0 up
# Assign an address to the aspeed-i2c controller
mctp addr add 50 dev mctpi2c0
# Assign a neetwork ID to the link (11)
mctp link set mctpi2c0 net 11
# Start the daemon that uses dbus for configuration.
systemctl start mctpd.service
# Assign an EID to the EP (hard coded I2C address is 0x4d)
busctl call xyz.openbmc_project.MCTP /xyz/openbmc_project/mctp au.com.CodeConstruct.MCTP AssignEndpoint say mctpi2c0 1 0x4d # Check it worked by dumping some state.
busctl introspect xyz.openbmc_project.MCTP /xyz/openbmc_project/mctp/11/8 xyz.openbmc_project.MCTP.Endpoint
At this point the control protocol has been exercised.
The following test program will query some information about the 'fake' CXL switch.
/*
* Trivial example program to exercise QEMU FMAPI Emulation over MCTP over I2C */
#include <sys/socket.h>
#include <stdio.h>
#include <string.h>
#include <stdint.h>
#include <stdlib.h>
#include "mctp.h"
#include "cxl_fmapi.h"
#define AF_MCTP 45
static int parse_physical_switch_identify_switch_device(void *buf, size_t buf_len)
{
struct cxl_fmapi_header *head = buf;
struct cxl_fmapi_ident_switch_dev_resp_pl *pl = (void *)(head + 1);
printf("Num total vppb %d\n", pl->num_total_vppb);
return 0;
}int query_physical_switch_info(int sd, struct sockaddr_mctp *addr, int *tag)
{
uint8_t buf[1024];
int rc;
ssize_t len;
struct sockaddr_mctp addrrx;
socklen_t addrlen = sizeof(addrrx);
struct cxl_fmapi_header req = {
.category = CXL_MCTP_CATEGORY_REQ,
.tag = *tag++,
.command = CXL_IDENTIFY_SWITCH_DEVICE,
.command_set = CXL_FM_API_CMD_SET_PHYSICAL_SWITCH,
.vendor_ext_status = 0xabcd,
};
len = sendto(sd, &req, sizeof(req), 0, (struct sockaddr *)addr, sizeof(*addr));
if (len != sizeof(req)) {
printf("Failed to send whole request\n");
return -1;
}
len = recvfrom(sd, buf, sizeof(buf), 0, (struct sockaddr *)&addrrx, &addrlen);
if (len < 0) {
printf("Failed to receive response\n");
return -1;
}
rc = parse_physical_switch_identify_switch_device(buf, len);
if (rc)
return -1;
return 0;
}static int parse_phys_port_state_rsp(void * buf, size_t buf_len, struct cxl_fmapi_header *head)
{
struct cxl_fmapi_header *rsp_head = buf;
struct cxl_fmapi_get_phys_port_state_resp_pl *pl = (void *)(rsp_head + 1);
uint32_t pl_length = rsp_head->pl_length[0] |
(rsp_head->pl_length[1] << 8) |
((rsp_head->pl_length[2] & 0xf) << 16);
if (rsp_head->category != CXL_MCTP_CATEGORY_RESP) {
printf("Message not a response\n");
return -1;
}
if (rsp_head->tag != head->tag) {
printf("Reply has wrong tag\n");
return -1;
}
if ((rsp_head->command != head->command) ||
(rsp_head->command_set != head->command_set)) {
printf("Response to wrong command\n");
return -1;
}
if (rsp_head->return_code != 0) {
printf("Error code in response %d\n", rsp_head);
return -1;
}
if (pl_length < 4 || pl_length < (pl->num_ports * sizeof(pl->ports[0]))) {
printf("too short\n");
return -1;
}
for (int i = 0; i < pl->num_ports; i++) {
struct cxl_fmapi_port_state_info_block *port = &pl->ports[i];
const char *port_states[] = {
[0x0] = "Disabled",
[0x1] = "Bind in progress",
[0x2] = "Unbind in progress",
[0x3] = "DSP",
[0x4] = "USP",
[0x5] = "Reserved",
//other values not present.
[0xf] = "Invalid Port ID"
};
const char *connected_device_modes[] = {
[0] = "Not CXL / connected",
[1] = "CXL 1.1",
[2] = "CXL 2.0",
};
const char *connected_device_type[] = {
[0] = "No device detected",
[1] = "PCIe device",
[2] = "CXL type 1 device",
[3] = "CXL type 2 device",
[4] = "CXL type 3 device",
[5] = "CXL type 3 pooled device",
[6] = "Reserved",
};
const char *ltssm_states[] = {
[0] = "Detect",
[1] = "Polling",
[2] = "Configuration",
[3] = "Recovery",
[4] = "L0",
[5] = "L0s",
[6] = "L1",
[7] = "L2",
[8] = "Disabled",
[9] = "Loop Back",
[10] = "Hot Reset",
};
printf("Port%02d:\n ", port->port_id);
printf("\tPort state: ");
if (port_states[port->config_state & 0xf])
printf("%s\n", port_states[port->config_state]);
else
printf("Unknown state\n");
if (port->config_state == 3) { /* DSP so device could be there */
printf("\tConnected Device CXL Version: ");
if (port->connected_device_cxl_version > 2)
printf("Unknown CXL Version\n");
else
printf("%s\n", connected_device_modes[port->connected_device_cxl_version]);
printf("\tConnected Device Type: ");
if (port->connected_device_type > 7)
printf("Unknown\n");
else
printf("%s\n", connected_device_type[port->connected_device_type]);
}
printf("\tSupported CXL Modes:");
if (port->port_cxl_version_bitmask & 0x1)
printf(" 1.1");
if (port->port_cxl_version_bitmask & 0x2)
printf(" 2.0");
printf("\n");
printf("\tMaximum Link Width: %d Negotiated Width %d\n",
port->max_link_width,
port->negotiated_link_width);
printf("\tSupported Speeds: ");
if (port->supported_link_speeds_vector & 0x1)
printf(" 2.5 GT/s");
if (port->supported_link_speeds_vector & 0x2)
printf(" 5.0 GT/s");
if (port->supported_link_speeds_vector & 0x4)
printf(" 8.0 GT/s");
if (port->supported_link_speeds_vector & 0x8)
printf(" 16.0 GT/s");
if (port->supported_link_speeds_vector & 0x10)
printf(" 32.0 GT/s");
if (port->supported_link_speeds_vector & 0x20)
printf(" 64.0 GT/s");
printf("\n");
printf("\tLTSSM: ");
if (port->ltssm_state >= sizeof(ltssm_states))
printf("Unkown\n");
else
printf("%s\n", ltssm_states[port->ltssm_state]);
}
}int query_ports(int sd, struct sockaddr_mctp *addr, int *tag)
{
uint8_t buf[1024];
ssize_t len;
int num_ports = 4;
int rc;
uint8_t port_list[4] = { 0, 3, 7, 4 };
struct sockaddr_mctp addrrx;
socklen_t addrlen = sizeof(addrrx);
struct cxl_fmapi_header *head;
struct cxl_fmapi_get_phys_port_state_req_pl *reqpl;
size_t req_sz = sizeof(*reqpl) + num_ports + sizeof(*head) ;
head = malloc(req_sz);
*head = (struct cxl_fmapi_header) {
.category = CXL_MCTP_CATEGORY_REQ,
.tag = *tag++,
.command = CXL_GET_PHYSICAL_PORT_STATE,
.command_set = CXL_FM_API_CMD_SET_PHYSICAL_SWITCH,
.pl_length = {
req_sz & 0xff,
(req_sz >> 8) & 0xff,
(req_sz >> 16) & 0xf },
.vendor_ext_status = 0x1234,
};
reqpl = (void *)(head + 1);
*reqpl = (struct cxl_fmapi_get_phys_port_state_req_pl) {
.num_ports = num_ports,
};
for (int j = 0; j < num_ports; j++)
reqpl->ports[j] = port_list[j];
len = sendto(sd, head, req_sz, 0,
(struct sockaddr *)addr, sizeof(*addr));
len = recvfrom(sd, buf, sizeof(buf), 0,
(struct sockaddr *)&addrrx, &addrlen);
if (len < sizeof(struct cxl_fmapi_header)) {
printf("Too short for header\n");
}
//TODO generic check of reply.
if (addrrx.smctp_type != 0x7) {
printf("Reply does not match expected message type\n");
}
rc = parse_phys_port_state_rsp(buf, len, head);
if (rc)
return rc;
return 0;
}int main()
{
int rc, sd;
int tag = 0; /* will increment on each use */
ssize_t len;
struct sockaddr_mctp addr = {
.smctp_family = AF_MCTP,
.smctp_network = 11,
.smctp_addr.s_addr = 8,
.smctp_type = 0x7, /* CXL FMAPI */
.smctp_tag = MCTP_TAG_OWNER,
};
struct sockaddr_mctp addrrx;
socklen_t addrlen = sizeof(addrrx);
sd = socket(AF_MCTP, SOCK_DGRAM, 0);
rc = bind(sd, (struct sockaddr *)&addr, sizeof(addr));
if (rc) {
printf("Bind failed\n");
return -1;
}
rc = query_physical_switch_info(sd, &addr, &tag);
if (rc)
return rc;
/* Next query some of the ports */
rc = query_ports(sd, &addr, &tag);
if (rc)
return rc;
return 0;
}All feedback welcome.
Enjoy,
Jonathan
Jonathan Cameron (2):
misc/i2c_mctp_cxl_fmapi: Initial device emulation arm/virt: Add aspeed-i2c controller and MCTP EP to enable MCTP testing
hw/arm/Kconfig | 1 + hw/arm/virt.c | 77 +++ hw/misc/Kconfig | 3 + hw/misc/i2c_mctp_cxl_fmapi.c | 759 +++++++++++++++++++++++++++ hw/misc/meson.build | 2 + include/hw/arm/virt.h | 2 + include/hw/cxl/cxl_fmapi.h | 158 ++++++ include/hw/misc/i2c_mctp_cxl_fmapi.h | 17 + 8 files changed, 1019 insertions(+) create mode 100644 hw/misc/i2c_mctp_cxl_fmapi.c create mode 100644 include/hw/cxl/cxl_fmapi.h create mode 100644 include/hw/misc/i2c_mctp_cxl_fmapi.h
Signature · 1 line
2.32.0
RFC PATCH 1/2 misc/i2c_mctp_cxl_fmapi: Initial device emulation
The Fabric Manager API is used to configure CXL switches and devices. DMTF has defined an MCTP binding specification to carry these messages. Then end goal of this work is to hook this up to an emulated CXL switch and allow control of the switch configuration. For now it's stand alone to show how the MCTP side of things works before hooking up the complexities of real switch emulation.
--- hw/misc/Kconfig | 3 + hw/misc/i2c_mctp_cxl_fmapi.c | 759 +++++++++++++++++++++++++++ hw/misc/meson.build | 2 + include/hw/cxl/cxl_fmapi.h | 158 ++++++ include/hw/misc/i2c_mctp_cxl_fmapi.h | 17 + 5 files changed, 939 insertions(+)
hw/misc/Kconfig +3 -0
@@ -174,4 +174,7 @@ config VIRT_CTRL config LASI bool +config I2C_MCTP_CXL_FMAPI + bool + source macio/Kconfig
hw/misc/i2c_mctp_cxl_fmapi.c new file +759 -0
759 lines not on this page · raw message
hw/misc/meson.build +2 -0
@@ -123,7 +123,9 @@ softmmu_ss.add(when: 'CONFIG_NRF51_SOC', if_true: files('nrf51_rng.c')) softmmu_ss.add(when: 'CONFIG_GRLIB', if_true: files('grlib_ahb_apb_pnp.c')) + softmmu_ss.add(when: 'CONFIG_I2C', if_true: files('i2c-echo.c')) +softmmu_ss.add(when: 'CONFIG_I2C_MCTP_CXL_FMAPI', if_true: files('i2c_mctp_cxl_fmapi.c')) specific_ss.add(when: 'CONFIG_AVR_POWER', if_true: files('avr_power.c'))
include/hw/cxl/cxl_fmapi.h new file +158 -0
158 lines not on this page · raw message
include/hw/misc/i2c_mctp_cxl_fmapi.h new file +17 -0
@@ -0,0 +1,17 @@ +/* +* SPDX-License-Identifier: GPL-2.0-or-later +* +* Emulation of a CXL Switch Fabric Management +* interface over MCTP over I2C. +* +* Copyright (c) 2022 Huawei Technologies. +*/ + +#ifndef I2C_MCTP_CXL_FMAPI_H +#define I2C_MCTP_CXL_FMAPI_H + +#include "qemu/osdep.h" + +#define TYPE_I2C_MCTP_CXL_SWITCH "i2c_mctp_cxl_switch" + +#endif
Signature · 1 line
2.32.0
RFC PATCH 2/2 arm/virt: Add aspeed-i2c controller and MCTP EP to enable MCTP testing
In reply to Jonathan CameronAs the only I2C emulation in QEMU that supports being both a master and a slave, suitable for MCTP over i2c is aspeed-i2c add this controller to the arm virt model and hook up our new i2c_mctp_cxl_fmapi device.
The current Linux driver for aspeed-i2c has a hard requirement on a reset controller. Throw down the simplest reset controller I could find so as to avoid need to make any chance to the kernel code.
Patch also builds appropriate device tree. Unfortunately for CXL we need to use ACPI (no DT bindings yet defined). Enabling this will either require appropriate support for MCTP on an i2c master that has ACPI bindings, or modifications of the kernel driver to support ACPI with aspeed-i2c (which might be a little controversial ;)
--- hw/arm/Kconfig | 1 + hw/arm/virt.c | 77 +++++++++++++++++++++++++++++++++++++++++++ include/hw/arm/virt.h | 2 ++ 3 files changed, 80 insertions(+)
hw/arm/Kconfig +1 -0
@@ -30,6 +30,7 @@ config ARM_VIRT select ACPI_VIOT select VIRTIO_MEM_SUPPORTED select ACPI_CXL + select I2C_MCTP_CXL_FMAPI config CHEETAH bool
hw/arm/virt.c +77 -0
107 lines not on this page · raw message
include/hw/arm/virt.h +2 -0
@@ -71,6 +71,8 @@ enum {VIRT_SMMU ,VIRT_UART ,VIRT_MMIO , +VIRT_I2C , +VIRT_RESET_FAKE ,VIRT_RTC ,VIRT_FW_CFG , VIRT_PCIE,
Signature · 1 line
2.32.0
Re: [RFC PATCH 2/2] arm/virt: Add aspeed-i2c controller and MCTP EP to enable MCTP testing
Quoted 9 lines · Jonathan Cameron
As the only I2C emulation in QEMU that supports being both a master and a slave, suitable for MCTP over i2c is aspeed-i2c add this controller to the arm virt model and hook up our new
i2c_mctp_cxl_fmapidevice.The current Linux driver for aspeed-i2c has a hard requirement on a reset controller. Throw down the simplest reset controller I could find so as to avoid need to make any chance to the kernel code.
s/chance/change
Quoted 6 lines
Patch also builds appropriate device tree. Unfortunately for CXL we need to use ACPI (no DT bindings yet defined). Enabling this will either require appropriate support for MCTP on an i2c master that has ACPI bindings, or modifications of the kernel driver to support ACPI with aspeed-i2c (which might be a little controversial ;)
I'm naive to what DT defines, but I assume what's there already is insufficient to make the bindings for CXL. I say this because I believe it wouldn't be too bad at all to make a cxl_dt.ko, and it's certainly less artificial than providing ACPI support for things which don't naturally have ACPI support.
Quoted 152 lines
Signed-off-by Jonathan Cameron <[email protected]>--- hw/arm/Kconfig | 1 + hw/arm/virt.c | 77 +++++++++++++++++++++++++++++++++++++++++++ include/hw/arm/virt.h | 2 ++ 3 files changed, 80 insertions(+)hw/arm/Kconfig +1
-0@@ -30,6 +30,7 @@ config ARM_VIRT select ACPI_VIOT select VIRTIO_MEM_SUPPORTED select ACPI_CXL + select I2C_MCTP_CXL_FMAPI config CHEETAH boolhw/arm/virt.c +77
-0@@ -80,6 +80,9 @@#include "hw/char/pl011.h" #include "hw/cxl/cxl.h" #include "qemu/guest-random.h" +#include "hw/i2c/i2c.h" +#include "hw/i2c/aspeed_i2c.h" +#include "hw/misc/i2c_mctp_cxl_fmapi.h" #define DEFINE_VIRT_MACHINE_LATEST (major, minor, latest) \ static void virt_##major##_##minor##_class_init(ObjectClass *oc, \@@ -156,6 +159,8 @@ static const MemMapEntry base_memmap[] = { [VIRT_PVTIME] = { 0x090a0000, 0x00010000 }, [VIRT_SECURE_GPIO] = { 0x090b0000, 0x00001000 }, [VIRT_MMIO] = { 0x0a000000, 0x00000200 }, + [VIRT_I2C] = { 0x0b000000, 0x00004000 }, + [VIRT_RESET_FAKE] = { 0x0b004000, 0x00000010 }, /* ...repeating for a total of NUM_VIRTIO_TRANSPORTS, each of that size */ [VIRT_PLATFORM_BUS] = { 0x0c000000, 0x02000000 }, [VIRT_SECURE_MEM] = { 0x0e000000, 0x01000000 },@@ -192,6 +197,7 @@ static const int a15irqmap[] = { [VIRT_GPIO] = 7, [VIRT_SECURE_UART] = 8, [VIRT_ACPI_GED] = 9, + [VIRT_I2C] = 10, [VIRT_MMIO] = 16, /* ...to 16 + NUM_VIRTIO_TRANSPORTS - 1 */ [VIRT_GIC_V2M] = 48, /* ...to 48 + NUM_GICV2M_SPIS - 1 */ [VIRT_SMMU] = 74, /* ...to 74 + NUM_SMMU_IRQS - 1 */@@ -1996,6 +2002,75 @@ static void virt_cpu_post_init(VirtMachineState *vms, MemoryRegion *sysmem) } } +static void create_mctp_test (MachineState *ms) +{ +VirtMachineState *vms =VIRT_MACHINE (ms); +MemoryRegion *sysmem =get_system_memory (); +AspeedI2CState *aspeedi2c; +struct DeviceState *dev; +char *nodename_i2c_master; +char *nodename_i2c_sub; +char *nodename_reset; +uint32_t clk_phandle, reset_phandle; +MemoryRegion *sysmem2; + + dev =qdev_new ("aspeed.i2c-ast2600" ); + aspeedi2c =ASPEED_I2C (dev); +object_property_set_link (OBJECT (dev),"dram" ,OBJECT (ms->ram), &error_fatal); +sysbus_realize_and_unref (SYS_BUS_DEVICE (dev), &error_fatal); +sysbus_mmio_map (SYS_BUS_DEVICE (dev),0 , vms->memmap[VIRT_I2C ].base); +sysbus_connect_irq (SYS_BUS_DEVICE (&aspeedi2c->busses[0 ]),0 ,qdev_get_gpio_in (vms->gic, vms->irqmap[VIRT_I2C ])); + +/* I2C bus DT */ + reset_phandle =qemu_fdt_alloc_phandle (ms->fdt); + nodename_reset =g_strdup_printf ("/reset@%" PRIx64, vms->memmap[VIRT_RESET_FAKE ].base); +qemu_fdt_add_subnode (ms->fdt, nodename_reset); +qemu_fdt_setprop_string (ms->fdt, nodename_reset,"compatible" ,"snps,dw-low-reset" ); +qemu_fdt_setprop_sized_cells (ms->fdt, nodename_reset,"reg" , +2 , vms->memmap[VIRT_RESET_FAKE ].base, +2 , vms->memmap[VIRT_RESET_FAKE ].size); +qemu_fdt_setprop_cell (ms->fdt, nodename_reset,"#reset-cells" ,0x1 ); +qemu_fdt_setprop_cell (ms->fdt, nodename_reset,"phandle" , reset_phandle); + sysmem2 =g_new (MemoryRegion,1 ); +memory_region_init_ram (sysmem2,NULL ,"reset" , vms->memmap[VIRT_RESET_FAKE ].size,NULL ); +memory_region_add_subregion (sysmem, vms->memmap[VIRT_RESET_FAKE ].base, sysmem2); + + clk_phandle =qemu_fdt_alloc_phandle (ms->fdt); + +qemu_fdt_add_subnode (ms->fdt,"/mclk" ); +qemu_fdt_setprop_string (ms->fdt,"/mclk" ,"compatible" ,"fixed-clock" ); +qemu_fdt_setprop_cell (ms->fdt,"/mclk" ,"#clock-cells" ,0x0 ); +qemu_fdt_setprop_cell (ms->fdt,"/mclk" ,"clock-frequency" ,24000 ); +qemu_fdt_setprop_string (ms->fdt,"/mclk" ,"clock-output-names" ,"bobsclk" ); +qemu_fdt_setprop_cell (ms->fdt,"/mclk" ,"phandle" , clk_phandle); + + nodename_i2c_master =g_strdup_printf ("/i2c@%" PRIx64, vms->memmap[VIRT_I2C ].base); +qemu_fdt_add_subnode (ms->fdt, nodename_i2c_master); +qemu_fdt_setprop_string (ms->fdt, nodename_i2c_master,"compatible" ,"aspeed,ast2600-i2c-bus" ); +qemu_fdt_setprop_cells (ms->fdt, nodename_i2c_master,"multi-master" ); +qemu_fdt_setprop_cell (ms->fdt, nodename_i2c_master,"#size-cells" ,0 ); +qemu_fdt_setprop_cell (ms->fdt, nodename_i2c_master,"#address-cells" ,1 ); +qemu_fdt_setprop_cell (ms->fdt, nodename_i2c_master,"clocks" , clk_phandle); +qemu_fdt_setprop_string (ms->fdt, nodename_i2c_master,"clock-names" ,"bobsclk" ); +qemu_fdt_setprop (ms->fdt, nodename_i2c_master,"mctp-controller" ,NULL ,0 ); +qemu_fdt_setprop_cells (ms->fdt, nodename_i2c_master,"interrupts" ,GIC_FDT_IRQ_TYPE_SPI , + vms->irqmap[VIRT_I2C ],GIC_FDT_IRQ_FLAGS_LEVEL_HI ); +/* Offset to the first bus is 0x80, next one at 0x100 etc */ +qemu_fdt_setprop_sized_cells (ms->fdt, nodename_i2c_master,"reg" , +2 , vms->memmap[VIRT_I2C ].base +0x80 , +2 ,0x80 ); +qemu_fdt_setprop_cells (ms->fdt, nodename_i2c_master,"resets" , reset_phandle,0 ); + + nodename_i2c_sub =g_strdup_printf ("/i2c@%" PRIx64"/mctp@%" PRIx64, vms->memmap[VIRT_I2C ].base,0x50l ); +qemu_fdt_add_subnode (ms->fdt, nodename_i2c_sub); +qemu_fdt_setprop_string (ms->fdt, nodename_i2c_sub,"compatible" ,"mctp-i2c-controller" ); +qemu_fdt_setprop_sized_cells (ms->fdt, nodename_i2c_sub,"reg" ,1 ,0x50 |0x40000000 ); + + +/* Slave device - linux doesn't use the presence of dt node for this so don't create one*/ +i2c_slave_create_simple (aspeed_i2c_get_bus (aspeedi2c,0 ),"i2c_mctp_cxl_switch" ,0x4d ); +} +static void machvirt_init (MachineState *machine) {VirtMachineState *vms =VIRT_MACHINE (machine);@@ -2289,6 +2364,8 @@ static void machvirt_init(MachineState *machine)create_gpio_devices (vms,VIRT_SECURE_GPIO , secure_sysmem); } +create_mctp_test (machine); +/* connect powerdown request */ vms->powerdown_notifier.notify = virt_powerdown_req;qemu_register_powerdown_notifier (&vms->powerdown_notifier);include/hw/arm/virt.h +2
-0@@ -71,6 +71,8 @@ enum {VIRT_SMMU ,VIRT_UART ,VIRT_MMIO , +VIRT_I2C , +VIRT_RESET_FAKE ,VIRT_RTC ,VIRT_FW_CFG , VIRT_PCIE,Signature · 1 line
2.32.0
Re: [RFC PATCH 2/2] arm/virt: Add aspeed-i2c controller and MCTP EP to enable MCTP testing
Quoted 6 lines · On Tue, 24 May 2022 09:36:44 -0700 Ben Widawsky <[email protected]> wrote:
Quoted 5 lines · Jonathan Cameron
As the only I2C emulation in QEMU that supports being both a master and a slave, suitable for MCTP over i2c is aspeed-i2c add this controller to the arm virt model and hook up our new
i2c_mctp_cxl_fmapidevice.
Quoted 6 lines
Quoted 4 lines
The current Linux driver for aspeed-i2c has a hard requirement on a reset controller. Throw down the simplest reset controller I could find so as to avoid need to make any chance to the kernel code.
s/chance/change
oops :)
Quoted 8 lines
Quoted 7 lines
Patch also builds appropriate device tree. Unfortunately for CXL we need to use ACPI (no DT bindings yet defined). Enabling this will either require appropriate support for MCTP on an i2c master that has ACPI bindings, or modifications of the kernel driver to support ACPI with aspeed-i2c (which might be a little controversial ;)
Quoted 4 lines
I'm naive to what DT defines, but I assume what's there already is insufficient to make the bindings for CXL. I say this because I believe it wouldn't be too bad at all to make a cxl_dt.ko, and it's certainly less artificial than providing ACPI support for things which don't naturally have ACPI support.
It wouldn't be that hard to work out a CXL dt binding, but it's not of sufficient interest to me that I'd want to do it (I'll review if someone else sends patches). Platforms I'm interested in CXL with are all strictly ACPI only.
The trick here I think, is going to be adding ACPI support for a suitable I2C controller which supports the requirement for supporting master and and I2C EP needed for MCTP. Either I find one that already has ACPI bindings and add enough emulation for this functionality, or work around the lack of ACPI bindings for the aspeed-i2c controller.
Based on a really quick check of I2C masters for which I have docs and that definitely have ACPI bindings, it's not a particularly common feature set.
Jonathan
Quoted 153 lines
Quoted 153 lines
Signed-off-by Jonathan Cameron <[email protected]>--- hw/arm/Kconfig | 1 + hw/arm/virt.c | 77 +++++++++++++++++++++++++++++++++++++++++++ include/hw/arm/virt.h | 2 ++ 3 files changed, 80 insertions(+)hw/arm/Kconfig +1
-0@@ -30,6 +30,7 @@ config ARM_VIRT select ACPI_VIOT select VIRTIO_MEM_SUPPORTED select ACPI_CXL + select I2C_MCTP_CXL_FMAPI config CHEETAH boolhw/arm/virt.c +77
-0@@ -80,6 +80,9 @@#include "hw/char/pl011.h" #include "hw/cxl/cxl.h" #include "qemu/guest-random.h" +#include "hw/i2c/i2c.h" +#include "hw/i2c/aspeed_i2c.h" +#include "hw/misc/i2c_mctp_cxl_fmapi.h" #define DEFINE_VIRT_MACHINE_LATEST (major, minor, latest) \ static void virt_##major##_##minor##_class_init(ObjectClass *oc, \@@ -156,6 +159,8 @@ static const MemMapEntry base_memmap[] = { [VIRT_PVTIME] = { 0x090a0000, 0x00010000 }, [VIRT_SECURE_GPIO] = { 0x090b0000, 0x00001000 }, [VIRT_MMIO] = { 0x0a000000, 0x00000200 }, + [VIRT_I2C] = { 0x0b000000, 0x00004000 }, + [VIRT_RESET_FAKE] = { 0x0b004000, 0x00000010 }, /* ...repeating for a total of NUM_VIRTIO_TRANSPORTS, each of that size */ [VIRT_PLATFORM_BUS] = { 0x0c000000, 0x02000000 }, [VIRT_SECURE_MEM] = { 0x0e000000, 0x01000000 },@@ -192,6 +197,7 @@ static const int a15irqmap[] = { [VIRT_GPIO] = 7, [VIRT_SECURE_UART] = 8, [VIRT_ACPI_GED] = 9, + [VIRT_I2C] = 10, [VIRT_MMIO] = 16, /* ...to 16 + NUM_VIRTIO_TRANSPORTS - 1 */ [VIRT_GIC_V2M] = 48, /* ...to 48 + NUM_GICV2M_SPIS - 1 */ [VIRT_SMMU] = 74, /* ...to 74 + NUM_SMMU_IRQS - 1 */@@ -1996,6 +2002,75 @@ static void virt_cpu_post_init(VirtMachineState *vms, MemoryRegion *sysmem) } } +static void create_mctp_test (MachineState *ms) +{ +VirtMachineState *vms =VIRT_MACHINE (ms); +MemoryRegion *sysmem =get_system_memory (); +AspeedI2CState *aspeedi2c; +struct DeviceState *dev; +char *nodename_i2c_master; +char *nodename_i2c_sub; +char *nodename_reset; +uint32_t clk_phandle, reset_phandle; +MemoryRegion *sysmem2; + + dev =qdev_new ("aspeed.i2c-ast2600" ); + aspeedi2c =ASPEED_I2C (dev); +object_property_set_link (OBJECT (dev),"dram" ,OBJECT (ms->ram), &error_fatal); +sysbus_realize_and_unref (SYS_BUS_DEVICE (dev), &error_fatal); +sysbus_mmio_map (SYS_BUS_DEVICE (dev),0 , vms->memmap[VIRT_I2C ].base); +sysbus_connect_irq (SYS_BUS_DEVICE (&aspeedi2c->busses[0 ]),0 ,qdev_get_gpio_in (vms->gic, vms->irqmap[VIRT_I2C ])); + +/* I2C bus DT */ + reset_phandle =qemu_fdt_alloc_phandle (ms->fdt); + nodename_reset =g_strdup_printf ("/reset@%" PRIx64, vms->memmap[VIRT_RESET_FAKE ].base); +qemu_fdt_add_subnode (ms->fdt, nodename_reset); +qemu_fdt_setprop_string (ms->fdt, nodename_reset,"compatible" ,"snps,dw-low-reset" ); +qemu_fdt_setprop_sized_cells (ms->fdt, nodename_reset,"reg" , +2 , vms->memmap[VIRT_RESET_FAKE ].base, +2 , vms->memmap[VIRT_RESET_FAKE ].size); +qemu_fdt_setprop_cell (ms->fdt, nodename_reset,"#reset-cells" ,0x1 ); +qemu_fdt_setprop_cell (ms->fdt, nodename_reset,"phandle" , reset_phandle); + sysmem2 =g_new (MemoryRegion,1 ); +memory_region_init_ram (sysmem2,NULL ,"reset" , vms->memmap[VIRT_RESET_FAKE ].size,NULL ); +memory_region_add_subregion (sysmem, vms->memmap[VIRT_RESET_FAKE ].base, sysmem2); + + clk_phandle =qemu_fdt_alloc_phandle (ms->fdt); + +qemu_fdt_add_subnode (ms->fdt,"/mclk" ); +qemu_fdt_setprop_string (ms->fdt,"/mclk" ,"compatible" ,"fixed-clock" ); +qemu_fdt_setprop_cell (ms->fdt,"/mclk" ,"#clock-cells" ,0x0 ); +qemu_fdt_setprop_cell (ms->fdt,"/mclk" ,"clock-frequency" ,24000 ); +qemu_fdt_setprop_string (ms->fdt,"/mclk" ,"clock-output-names" ,"bobsclk" ); +qemu_fdt_setprop_cell (ms->fdt,"/mclk" ,"phandle" , clk_phandle); + + nodename_i2c_master =g_strdup_printf ("/i2c@%" PRIx64, vms->memmap[VIRT_I2C ].base); +qemu_fdt_add_subnode (ms->fdt, nodename_i2c_master); +qemu_fdt_setprop_string (ms->fdt, nodename_i2c_master,"compatible" ,"aspeed,ast2600-i2c-bus" ); +qemu_fdt_setprop_cells (ms->fdt, nodename_i2c_master,"multi-master" ); +qemu_fdt_setprop_cell (ms->fdt, nodename_i2c_master,"#size-cells" ,0 ); +qemu_fdt_setprop_cell (ms->fdt, nodename_i2c_master,"#address-cells" ,1 ); +qemu_fdt_setprop_cell (ms->fdt, nodename_i2c_master,"clocks" , clk_phandle); +qemu_fdt_setprop_string (ms->fdt, nodename_i2c_master,"clock-names" ,"bobsclk" ); +qemu_fdt_setprop (ms->fdt, nodename_i2c_master,"mctp-controller" ,NULL ,0 ); +qemu_fdt_setprop_cells (ms->fdt, nodename_i2c_master,"interrupts" ,GIC_FDT_IRQ_TYPE_SPI , + vms->irqmap[VIRT_I2C ],GIC_FDT_IRQ_FLAGS_LEVEL_HI ); +/* Offset to the first bus is 0x80, next one at 0x100 etc */ +qemu_fdt_setprop_sized_cells (ms->fdt, nodename_i2c_master,"reg" , +2 , vms->memmap[VIRT_I2C ].base +0x80 , +2 ,0x80 ); +qemu_fdt_setprop_cells (ms->fdt, nodename_i2c_master,"resets" , reset_phandle,0 ); + + nodename_i2c_sub =g_strdup_printf ("/i2c@%" PRIx64"/mctp@%" PRIx64, vms->memmap[VIRT_I2C ].base,0x50l ); +qemu_fdt_add_subnode (ms->fdt, nodename_i2c_sub); +qemu_fdt_setprop_string (ms->fdt, nodename_i2c_sub,"compatible" ,"mctp-i2c-controller" ); +qemu_fdt_setprop_sized_cells (ms->fdt, nodename_i2c_sub,"reg" ,1 ,0x50 |0x40000000 ); + + +/* Slave device - linux doesn't use the presence of dt node for this so don't create one*/ +i2c_slave_create_simple (aspeed_i2c_get_bus (aspeedi2c,0 ),"i2c_mctp_cxl_switch" ,0x4d ); +} +static void machvirt_init (MachineState *machine) {VirtMachineState *vms =VIRT_MACHINE (machine);@@ -2289,6 +2364,8 @@ static void machvirt_init(MachineState *machine)create_gpio_devices (vms,VIRT_SECURE_GPIO , secure_sysmem); } +create_mctp_test (machine); +/* connect powerdown request */ vms->powerdown_notifier.notify = virt_powerdown_req;qemu_register_powerdown_notifier (&vms->powerdown_notifier);include/hw/arm/virt.h +2
-0@@ -71,6 +71,8 @@ enum {VIRT_SMMU ,VIRT_UART ,VIRT_MMIO , +VIRT_I2C , +VIRT_RESET_FAKE ,VIRT_RTC ,VIRT_FW_CFG , VIRT_PCIE,Signature · 1 line
2.32.0
Re: [RFC PATCH 2/2] arm/virt: Add aspeed-i2c controller and MCTP EP to enable MCTP testing
Quoted 34 lines · Jonathan Cameron
Quoted 6 lines · On Tue, 24 May 2022 09:36:44 -0700 Ben Widawsky <[email protected]> wrote:
Quoted 5 lines · Jonathan Cameron
As the only I2C emulation in QEMU that supports being both a master and a slave, suitable for MCTP over i2c is aspeed-i2c add this controller to the arm virt model and hook up our new
i2c_mctp_cxl_fmapidevice.Quoted 6 lines
Quoted 4 lines
The current Linux driver for aspeed-i2c has a hard requirement on a reset controller. Throw down the simplest reset controller I could find so as to avoid need to make any chance to the kernel code.
s/chance/change
oops :)
Quoted 8 lines
Quoted 7 lines
Patch also builds appropriate device tree. Unfortunately for CXL we need to use ACPI (no DT bindings yet defined). Enabling this will either require appropriate support for MCTP on an i2c master that has ACPI bindings, or modifications of the kernel driver to support ACPI with aspeed-i2c (which might be a little controversial ;)
Quoted 4 lines
I'm naive to what DT defines, but I assume what's there already is insufficient to make the bindings for CXL. I say this because I believe it wouldn't be too bad at all to make a cxl_dt.ko, and it's certainly less artificial than providing ACPI support for things which don't naturally have ACPI support.
It wouldn't be that hard to work out a CXL dt binding, but it's not of sufficient interest to me that I'd want to do it (I'll review if someone else sends patches). Platforms I'm interested in CXL with are all strictly ACPI only.
Quoted 8 lines
The trick here I think, is going to be adding ACPI support for a suitable I2C controller which supports the requirement for supporting master and and I2C EP needed for MCTP. Either I find one that already has ACPI bindings and add enough emulation for this functionality, or work around the lack of ACPI bindings for the aspeed-i2c controller.
Based on a really quick check of I2C masters for which I have docs and that definitely have ACPI bindings, it's not a particularly common feature set.
Ben already got my comments.
It's not common, but it's something you are using, so it obviously exists. What hardware do you have? The most natural thing might be to add your controller.
-corey
Quoted 159 lines
Jonathan
Quoted 153 lines
Quoted 153 lines
Signed-off-by Jonathan Cameron <[email protected]>--- hw/arm/Kconfig | 1 + hw/arm/virt.c | 77 +++++++++++++++++++++++++++++++++++++++++++ include/hw/arm/virt.h | 2 ++ 3 files changed, 80 insertions(+)hw/arm/Kconfig +1
-0@@ -30,6 +30,7 @@ config ARM_VIRT select ACPI_VIOT select VIRTIO_MEM_SUPPORTED select ACPI_CXL + select I2C_MCTP_CXL_FMAPI config CHEETAH boolhw/arm/virt.c +77
-0@@ -80,6 +80,9 @@#include "hw/char/pl011.h" #include "hw/cxl/cxl.h" #include "qemu/guest-random.h" +#include "hw/i2c/i2c.h" +#include "hw/i2c/aspeed_i2c.h" +#include "hw/misc/i2c_mctp_cxl_fmapi.h" #define DEFINE_VIRT_MACHINE_LATEST (major, minor, latest) \ static void virt_##major##_##minor##_class_init(ObjectClass *oc, \@@ -156,6 +159,8 @@ static const MemMapEntry base_memmap[] = { [VIRT_PVTIME] = { 0x090a0000, 0x00010000 }, [VIRT_SECURE_GPIO] = { 0x090b0000, 0x00001000 }, [VIRT_MMIO] = { 0x0a000000, 0x00000200 }, + [VIRT_I2C] = { 0x0b000000, 0x00004000 }, + [VIRT_RESET_FAKE] = { 0x0b004000, 0x00000010 }, /* ...repeating for a total of NUM_VIRTIO_TRANSPORTS, each of that size */ [VIRT_PLATFORM_BUS] = { 0x0c000000, 0x02000000 }, [VIRT_SECURE_MEM] = { 0x0e000000, 0x01000000 },@@ -192,6 +197,7 @@ static const int a15irqmap[] = { [VIRT_GPIO] = 7, [VIRT_SECURE_UART] = 8, [VIRT_ACPI_GED] = 9, + [VIRT_I2C] = 10, [VIRT_MMIO] = 16, /* ...to 16 + NUM_VIRTIO_TRANSPORTS - 1 */ [VIRT_GIC_V2M] = 48, /* ...to 48 + NUM_GICV2M_SPIS - 1 */ [VIRT_SMMU] = 74, /* ...to 74 + NUM_SMMU_IRQS - 1 */@@ -1996,6 +2002,75 @@ static void virt_cpu_post_init(VirtMachineState *vms, MemoryRegion *sysmem) } } +static void create_mctp_test (MachineState *ms) +{ +VirtMachineState *vms =VIRT_MACHINE (ms); +MemoryRegion *sysmem =get_system_memory (); +AspeedI2CState *aspeedi2c; +struct DeviceState *dev; +char *nodename_i2c_master; +char *nodename_i2c_sub; +char *nodename_reset; +uint32_t clk_phandle, reset_phandle; +MemoryRegion *sysmem2; + + dev =qdev_new ("aspeed.i2c-ast2600" ); + aspeedi2c =ASPEED_I2C (dev); +object_property_set_link (OBJECT (dev),"dram" ,OBJECT (ms->ram), &error_fatal); +sysbus_realize_and_unref (SYS_BUS_DEVICE (dev), &error_fatal); +sysbus_mmio_map (SYS_BUS_DEVICE (dev),0 , vms->memmap[VIRT_I2C ].base); +sysbus_connect_irq (SYS_BUS_DEVICE (&aspeedi2c->busses[0 ]),0 ,qdev_get_gpio_in (vms->gic, vms->irqmap[VIRT_I2C ])); + +/* I2C bus DT */ + reset_phandle =qemu_fdt_alloc_phandle (ms->fdt); + nodename_reset =g_strdup_printf ("/reset@%" PRIx64, vms->memmap[VIRT_RESET_FAKE ].base); +qemu_fdt_add_subnode (ms->fdt, nodename_reset); +qemu_fdt_setprop_string (ms->fdt, nodename_reset,"compatible" ,"snps,dw-low-reset" ); +qemu_fdt_setprop_sized_cells (ms->fdt, nodename_reset,"reg" , +2 , vms->memmap[VIRT_RESET_FAKE ].base, +2 , vms->memmap[VIRT_RESET_FAKE ].size); +qemu_fdt_setprop_cell (ms->fdt, nodename_reset,"#reset-cells" ,0x1 ); +qemu_fdt_setprop_cell (ms->fdt, nodename_reset,"phandle" , reset_phandle); + sysmem2 =g_new (MemoryRegion,1 ); +memory_region_init_ram (sysmem2,NULL ,"reset" , vms->memmap[VIRT_RESET_FAKE ].size,NULL ); +memory_region_add_subregion (sysmem, vms->memmap[VIRT_RESET_FAKE ].base, sysmem2); + + clk_phandle =qemu_fdt_alloc_phandle (ms->fdt); + +qemu_fdt_add_subnode (ms->fdt,"/mclk" ); +qemu_fdt_setprop_string (ms->fdt,"/mclk" ,"compatible" ,"fixed-clock" ); +qemu_fdt_setprop_cell (ms->fdt,"/mclk" ,"#clock-cells" ,0x0 ); +qemu_fdt_setprop_cell (ms->fdt,"/mclk" ,"clock-frequency" ,24000 ); +qemu_fdt_setprop_string (ms->fdt,"/mclk" ,"clock-output-names" ,"bobsclk" ); +qemu_fdt_setprop_cell (ms->fdt,"/mclk" ,"phandle" , clk_phandle); + + nodename_i2c_master =g_strdup_printf ("/i2c@%" PRIx64, vms->memmap[VIRT_I2C ].base); +qemu_fdt_add_subnode (ms->fdt, nodename_i2c_master); +qemu_fdt_setprop_string (ms->fdt, nodename_i2c_master,"compatible" ,"aspeed,ast2600-i2c-bus" ); +qemu_fdt_setprop_cells (ms->fdt, nodename_i2c_master,"multi-master" ); +qemu_fdt_setprop_cell (ms->fdt, nodename_i2c_master,"#size-cells" ,0 ); +qemu_fdt_setprop_cell (ms->fdt, nodename_i2c_master,"#address-cells" ,1 ); +qemu_fdt_setprop_cell (ms->fdt, nodename_i2c_master,"clocks" , clk_phandle); +qemu_fdt_setprop_string (ms->fdt, nodename_i2c_master,"clock-names" ,"bobsclk" ); +qemu_fdt_setprop (ms->fdt, nodename_i2c_master,"mctp-controller" ,NULL ,0 ); +qemu_fdt_setprop_cells (ms->fdt, nodename_i2c_master,"interrupts" ,GIC_FDT_IRQ_TYPE_SPI , + vms->irqmap[VIRT_I2C ],GIC_FDT_IRQ_FLAGS_LEVEL_HI ); +/* Offset to the first bus is 0x80, next one at 0x100 etc */ +qemu_fdt_setprop_sized_cells (ms->fdt, nodename_i2c_master,"reg" , +2 , vms->memmap[VIRT_I2C ].base +0x80 , +2 ,0x80 ); +qemu_fdt_setprop_cells (ms->fdt, nodename_i2c_master,"resets" , reset_phandle,0 ); + + nodename_i2c_sub =g_strdup_printf ("/i2c@%" PRIx64"/mctp@%" PRIx64, vms->memmap[VIRT_I2C ].base,0x50l ); +qemu_fdt_add_subnode (ms->fdt, nodename_i2c_sub); +qemu_fdt_setprop_string (ms->fdt, nodename_i2c_sub,"compatible" ,"mctp-i2c-controller" ); +qemu_fdt_setprop_sized_cells (ms->fdt, nodename_i2c_sub,"reg" ,1 ,0x50 |0x40000000 ); + + +/* Slave device - linux doesn't use the presence of dt node for this so don't create one*/ +i2c_slave_create_simple (aspeed_i2c_get_bus (aspeedi2c,0 ),"i2c_mctp_cxl_switch" ,0x4d ); +} +static void machvirt_init (MachineState *machine) {VirtMachineState *vms =VIRT_MACHINE (machine);@@ -2289,6 +2364,8 @@ static void machvirt_init(MachineState *machine)create_gpio_devices (vms,VIRT_SECURE_GPIO , secure_sysmem); } +create_mctp_test (machine); +/* connect powerdown request */ vms->powerdown_notifier.notify = virt_powerdown_req;qemu_register_powerdown_notifier (&vms->powerdown_notifier);include/hw/arm/virt.h +2
-0@@ -71,6 +71,8 @@ enum {VIRT_SMMU ,VIRT_UART ,VIRT_MMIO , +VIRT_I2C , +VIRT_RESET_FAKE ,VIRT_RTC ,VIRT_FW_CFG , VIRT_PCIE,Signature · 1 line
2.32.0
Re: [RFC PATCH 2/2] arm/virt: Add aspeed-i2c controller and MCTP EP to enable MCTP testing
Quoted 41 lines · On Tue, 24 May 2022 13:13:10 -0500 Corey Minyard <[email protected]> wrote:
Quoted 40 lines · Jonathan Cameron
Quoted 6 lines · On Tue, 24 May 2022 09:36:44 -0700 Ben Widawsky <[email protected]> wrote:
Quoted 5 lines · Jonathan Cameron
As the only I2C emulation in QEMU that supports being both a master and a slave, suitable for MCTP over i2c is aspeed-i2c add this controller to the arm virt model and hook up our new
i2c_mctp_cxl_fmapidevice.Quoted 6 lines
Quoted 4 lines
The current Linux driver for aspeed-i2c has a hard requirement on a reset controller. Throw down the simplest reset controller I could find so as to avoid need to make any chance to the kernel code.
s/chance/change
oops :)
Quoted 8 lines
Quoted 7 lines
Patch also builds appropriate device tree. Unfortunately for CXL we need to use ACPI (no DT bindings yet defined). Enabling this will either require appropriate support for MCTP on an i2c master that has ACPI bindings, or modifications of the kernel driver to support ACPI with aspeed-i2c (which might be a little controversial ;)
Quoted 4 lines
I'm naive to what DT defines, but I assume what's there already is insufficient to make the bindings for CXL. I say this because I believe it wouldn't be too bad at all to make a cxl_dt.ko, and it's certainly less artificial than providing ACPI support for things which don't naturally have ACPI support.
It wouldn't be that hard to work out a CXL dt binding, but it's not of sufficient interest to me that I'd want to do it (I'll review if someone else sends patches). Platforms I'm interested in CXL with are all strictly ACPI only.
The trick here I think, is going to be adding ACPI support for a suitable I2C controller which supports the requirement for supporting master and and I2C EP needed for MCTP. Either I find one that already has ACPI bindings and add enough emulation for this functionality, or work around the lack of ACPI bindings for the aspeed-i2c controller.
Quoted 9 lines
Quoted 2 lines
Based on a really quick check of I2C masters for which I have docs and that definitely have ACPI bindings, it's not a particularly common feature set.
Ben already got my comments.
It's not common, but it's something you are using, so it obviously exists. What hardware do you have? The most natural thing might be to add your controller.
Right now, no hardware. My interest here isn't really in the i2c mctp thing as opposed to running other stuff over MCTP - it's just a convenient path to emulating what a Fabric Manager (probably a BMC) would be doing in a CXL system. I'm just making us of the existing infrastructure rather than rolling a different interface (I could do MCTP over PCI VDM for example and may well do that in future).
Actual system might look something like:
- note that I don't need to model this to be able to exercise the OS support etc... (see later)
USP - normal upstream port
DSP - normal downstream port
vPPB - is sort of like a virtual downstream port. These
are dynamically bound to either a single logical devices (SLD) which has a whole physical CXL DSP to itself, or to one logical device in an Multilogical device (MLD)
What a host sees is either vPPB that has link down (nothing there) or a vPPB that has a device below it which it can discover via normal enumeration.
Example here has
Host 0 with two root ports
Each RP is connected to an upstream port on the switch with a small virtual hierarchy (VH0 and VH1) of two vPPB each As currently configured (and note it dynamically configuring this which is of interest) VH0 has one vPPB bound to a
_________________ ________________
| Host 0 | | Host 1 | ---------
| | | | | Fabric |
| CXLRP0 CXLRP1 | | CXLRP0 CXLRP1 | | Manager |
|_________________| |________________| |_________|
| | | | |
| CXL | CXL |CXL |CXL | I2C
___|________|____________|______|______________|______
| |
| USP USP USP USP FM-API |
| | | | | |
| | VH0 |VH1 |VH2 |VH3 |
| / \ / \ / \ / \ |
| / \ \ / \ \ |
|/ \ \ / \ \ |
|vPPB vPPB vPPB vPPB vPPB vPPB |
| | Unbound | Unbound | | |
| bound \ LD1 LD2 |
| | \Bound LD0 | / |
| | \ | / |
| DSP ----DSP-MLD --- |
|______________________________________________________|
| |
SLD Type 3 MLD memory devicememory device
However, to be able to poke all the OS interfaces and the fabric manager interfaces I just need the ability to model something more like.
_______________________________________________________
| Host 0 |
| FM Software |
| CXLRP0 CXLRP1 Manager |
|_______________________________________________________|
| | |
| CXL | CXL Not connected | I2C
___|________|__________________________________|______
| |
| USP USP USP USP FM-API |
| | | | | |
| | VH0 |VH1 |VH2 |VH3 |
| / \ / \ / \ / \ |
| / \ \ / \ \ |
|/ \ \ / \ \ |
|vPPB vPPB vPPB vPPB vPPB vPPB |
| | Unbound | Unbound | | |
| bound \ LD1 LD2 |
| | \Bound LD0 | / |
| | \ | / |
| DSP -----DSP-MLD --- |
|______________________________________________________|
| |
SLD Type 3 MLD memory devicememory device
The reason for jumping through the hoops of mctp over i2c is that the software support is there and standard today
+ this lets me use the existing DMTF MCTP bindings for CXL FM-API rather than wrapping them up in something custom. It's also nice and lightweight from the emulation point of view.
Clearly lots to do here yet though including...
1. Hooking the FM-API agent up to the parts of the switch (probably adding links to the USPs is sufficient)
2. Adding infrastructure for vPPBs.
3. Wire up everything to be able to trigger appropriate hotplug events to the host for CXL devices being bound to the vPPBs.
I'm not that bothered by the lack of suitable i2c master with ACPI bindings in the short term, it just makes testing a bit fiddlier as kernel will need patching. It'll be a trivial patch, particularly if I can get moving most of the aspeed-i2c kernel firmware handling over to the generic property.h infrastructure (and then use the wonder that is PRP0001, plus bodge the clk and reset).
Longer term I might just switch to the PCI VDM route but that's a bigger job in both the kernel and QEMU.
Right now I'm loving the short cut other people's work gave me ;)
Thanks,
Jonathan
Quoted 161 lines
-corey
Quoted 159 lines
Jonathan
Quoted 153 lines
Quoted 153 lines
Signed-off-by Jonathan Cameron <[email protected]>--- hw/arm/Kconfig | 1 + hw/arm/virt.c | 77 +++++++++++++++++++++++++++++++++++++++++++ include/hw/arm/virt.h | 2 ++ 3 files changed, 80 insertions(+)hw/arm/Kconfig +1
-0@@ -30,6 +30,7 @@ config ARM_VIRT select ACPI_VIOT select VIRTIO_MEM_SUPPORTED select ACPI_CXL + select I2C_MCTP_CXL_FMAPI config CHEETAH boolhw/arm/virt.c +77
-0@@ -80,6 +80,9 @@#include "hw/char/pl011.h" #include "hw/cxl/cxl.h" #include "qemu/guest-random.h" +#include "hw/i2c/i2c.h" +#include "hw/i2c/aspeed_i2c.h" +#include "hw/misc/i2c_mctp_cxl_fmapi.h" #define DEFINE_VIRT_MACHINE_LATEST (major, minor, latest) \ static void virt_##major##_##minor##_class_init(ObjectClass *oc, \@@ -156,6 +159,8 @@ static const MemMapEntry base_memmap[] = { [VIRT_PVTIME] = { 0x090a0000, 0x00010000 }, [VIRT_SECURE_GPIO] = { 0x090b0000, 0x00001000 }, [VIRT_MMIO] = { 0x0a000000, 0x00000200 }, + [VIRT_I2C] = { 0x0b000000, 0x00004000 }, + [VIRT_RESET_FAKE] = { 0x0b004000, 0x00000010 }, /* ...repeating for a total of NUM_VIRTIO_TRANSPORTS, each of that size */ [VIRT_PLATFORM_BUS] = { 0x0c000000, 0x02000000 }, [VIRT_SECURE_MEM] = { 0x0e000000, 0x01000000 },@@ -192,6 +197,7 @@ static const int a15irqmap[] = { [VIRT_GPIO] = 7, [VIRT_SECURE_UART] = 8, [VIRT_ACPI_GED] = 9, + [VIRT_I2C] = 10, [VIRT_MMIO] = 16, /* ...to 16 + NUM_VIRTIO_TRANSPORTS - 1 */ [VIRT_GIC_V2M] = 48, /* ...to 48 + NUM_GICV2M_SPIS - 1 */ [VIRT_SMMU] = 74, /* ...to 74 + NUM_SMMU_IRQS - 1 */@@ -1996,6 +2002,75 @@ static void virt_cpu_post_init(VirtMachineState *vms, MemoryRegion *sysmem) } } +static void create_mctp_test (MachineState *ms) +{ +VirtMachineState *vms =VIRT_MACHINE (ms); +MemoryRegion *sysmem =get_system_memory (); +AspeedI2CState *aspeedi2c; +struct DeviceState *dev; +char *nodename_i2c_master; +char *nodename_i2c_sub; +char *nodename_reset; +uint32_t clk_phandle, reset_phandle; +MemoryRegion *sysmem2; + + dev =qdev_new ("aspeed.i2c-ast2600" ); + aspeedi2c =ASPEED_I2C (dev); +object_property_set_link (OBJECT (dev),"dram" ,OBJECT (ms->ram), &error_fatal); +sysbus_realize_and_unref (SYS_BUS_DEVICE (dev), &error_fatal); +sysbus_mmio_map (SYS_BUS_DEVICE (dev),0 , vms->memmap[VIRT_I2C ].base); +sysbus_connect_irq (SYS_BUS_DEVICE (&aspeedi2c->busses[0 ]),0 ,qdev_get_gpio_in (vms->gic, vms->irqmap[VIRT_I2C ])); + +/* I2C bus DT */ + reset_phandle =qemu_fdt_alloc_phandle (ms->fdt); + nodename_reset =g_strdup_printf ("/reset@%" PRIx64, vms->memmap[VIRT_RESET_FAKE ].base); +qemu_fdt_add_subnode (ms->fdt, nodename_reset); +qemu_fdt_setprop_string (ms->fdt, nodename_reset,"compatible" ,"snps,dw-low-reset" ); +qemu_fdt_setprop_sized_cells (ms->fdt, nodename_reset,"reg" , +2 , vms->memmap[VIRT_RESET_FAKE ].base, +2 , vms->memmap[VIRT_RESET_FAKE ].size); +qemu_fdt_setprop_cell (ms->fdt, nodename_reset,"#reset-cells" ,0x1 ); +qemu_fdt_setprop_cell (ms->fdt, nodename_reset,"phandle" , reset_phandle); + sysmem2 =g_new (MemoryRegion,1 ); +memory_region_init_ram (sysmem2,NULL ,"reset" , vms->memmap[VIRT_RESET_FAKE ].size,NULL ); +memory_region_add_subregion (sysmem, vms->memmap[VIRT_RESET_FAKE ].base, sysmem2); + + clk_phandle =qemu_fdt_alloc_phandle (ms->fdt); + +qemu_fdt_add_subnode (ms->fdt,"/mclk" ); +qemu_fdt_setprop_string (ms->fdt,"/mclk" ,"compatible" ,"fixed-clock" ); +qemu_fdt_setprop_cell (ms->fdt,"/mclk" ,"#clock-cells" ,0x0 ); +qemu_fdt_setprop_cell (ms->fdt,"/mclk" ,"clock-frequency" ,24000 ); +qemu_fdt_setprop_string (ms->fdt,"/mclk" ,"clock-output-names" ,"bobsclk" ); +qemu_fdt_setprop_cell (ms->fdt,"/mclk" ,"phandle" , clk_phandle); + + nodename_i2c_master =g_strdup_printf ("/i2c@%" PRIx64, vms->memmap[VIRT_I2C ].base); +qemu_fdt_add_subnode (ms->fdt, nodename_i2c_master); +qemu_fdt_setprop_string (ms->fdt, nodename_i2c_master,"compatible" ,"aspeed,ast2600-i2c-bus" ); +qemu_fdt_setprop_cells (ms->fdt, nodename_i2c_master,"multi-master" ); +qemu_fdt_setprop_cell (ms->fdt, nodename_i2c_master,"#size-cells" ,0 ); +qemu_fdt_setprop_cell (ms->fdt, nodename_i2c_master,"#address-cells" ,1 ); +qemu_fdt_setprop_cell (ms->fdt, nodename_i2c_master,"clocks" , clk_phandle); +qemu_fdt_setprop_string (ms->fdt, nodename_i2c_master,"clock-names" ,"bobsclk" ); +qemu_fdt_setprop (ms->fdt, nodename_i2c_master,"mctp-controller" ,NULL ,0 ); +qemu_fdt_setprop_cells (ms->fdt, nodename_i2c_master,"interrupts" ,GIC_FDT_IRQ_TYPE_SPI , + vms->irqmap[VIRT_I2C ],GIC_FDT_IRQ_FLAGS_LEVEL_HI ); +/* Offset to the first bus is 0x80, next one at 0x100 etc */ +qemu_fdt_setprop_sized_cells (ms->fdt, nodename_i2c_master,"reg" , +2 , vms->memmap[VIRT_I2C ].base +0x80 , +2 ,0x80 ); +qemu_fdt_setprop_cells (ms->fdt, nodename_i2c_master,"resets" , reset_phandle,0 ); + + nodename_i2c_sub =g_strdup_printf ("/i2c@%" PRIx64"/mctp@%" PRIx64, vms->memmap[VIRT_I2C ].base,0x50l ); +qemu_fdt_add_subnode (ms->fdt, nodename_i2c_sub); +qemu_fdt_setprop_string (ms->fdt, nodename_i2c_sub,"compatible" ,"mctp-i2c-controller" ); +qemu_fdt_setprop_sized_cells (ms->fdt, nodename_i2c_sub,"reg" ,1 ,0x50 |0x40000000 ); + + +/* Slave device - linux doesn't use the presence of dt node for this so don't create one*/ +i2c_slave_create_simple (aspeed_i2c_get_bus (aspeedi2c,0 ),"i2c_mctp_cxl_switch" ,0x4d ); +} +static void machvirt_init (MachineState *machine) {VirtMachineState *vms =VIRT_MACHINE (machine);@@ -2289,6 +2364,8 @@ static void machvirt_init(MachineState *machine)create_gpio_devices (vms,VIRT_SECURE_GPIO , secure_sysmem); } +create_mctp_test (machine); +/* connect powerdown request */ vms->powerdown_notifier.notify = virt_powerdown_req;qemu_register_powerdown_notifier (&vms->powerdown_notifier);include/hw/arm/virt.h +2
-0@@ -71,6 +71,8 @@ enum {VIRT_SMMU ,VIRT_UART ,VIRT_MMIO , +VIRT_I2C , +VIRT_RESET_FAKE ,VIRT_RTC ,VIRT_FW_CFG , VIRT_PCIE,Signature · 1 line
2.32.0