IVSC support for IPU bridge
v2 cover IVSC support for IPU bridge
Hi,
I rebased the IVSC support patch on Hans de Goede's IPU bridge fixes and cleanups. I've tested it on MS Surface Go 2. I also added a patch to move the pre-existing exported symbol to the same namespace than the rest.
Sakari Ailus (1):
media: ipu-bridge: Make exported IPU bridge symbol GPL-only in a NS
Wentong Wu (1):
media: ipu-bridge: add IVSC support
drivers/media/pci/intel/ipu-bridge.c | 197 ++++++++++++++++++++++++++- include/media/ipu-bridge.h | 19 ++- 2 files changed, 208 insertions(+), 8 deletions(-)
Signature · 1 line
2.39.2
PATCH v2 1/2 media: ipu-bridge: add IVSC support
From: Wentong Wu <[email protected]>
Previously on ACPI platforms, sensors that are intended to be connected to a IPU device for use with the ipu3-cio2 driver lacking the necessary connection information in firmware. IPU bridge driver is to connect sensors to IPU device via software nodes.
Currently IVSC located between IPU device and sensors is available in existing commercial platforms from multiple OEMs. But the connection information between them in firmware is also not enough to build V4L2 connection graph. This patch parses the connection properties from the SSDB buffer in DSDT and build the connection using software nodes.
IVSC driver is based on MEI framework (previously known as HECI), it has two MEI clients, MEI CSI and MEI ACE. Both clients are used to communicate messages with IVSC firmware. Linux abstracts MEI client as a device, whose bus type is MEI. And the device is addressed by a GUID/UUID which is part of the device name of MEI client. After figured out MEI CSI via the UUID composed device name, this patch setup the connection between MEI CSI and IPU, and the connection between MEI CSI and sensor via software nodes.
--- drivers/media/pci/intel/ipu-bridge.c | 195 ++++++++++++++++++++++++++- include/media/ipu-bridge.h | 19 ++- 2 files changed, 207 insertions(+), 7 deletions(-)
drivers/media/pci/intel/ipu-bridge.c +189 -6
@@ -4,6 +4,8 @@#include <linux/acpi.h> #include <linux/device.h> #include <linux/i2c.h> +#include <linux/mei_cl_bus.h> +#include <linux/pci.h> #include <linux/pm_runtime.h> #include <linux/property.h> #include <linux/workqueue.h>
@@ -11,6 +13,16 @@#include <media/ipu-bridge.h> #include <media/v4l2-fwnode.h> +/* +* 92335fcf-3203-4472-af93-7b4453ac29da +* +* Used to build MEI CSI device name to lookup MEI CSI device by +* device_find_child_by_name(). +*/ +#define MEI_CSI_UUID \ + UUID_LE(0x92335FCF, 0x3203, 0x4472, \ + 0xAF, 0x93, 0x7b, 0x44, 0x53, 0xAC, 0x29, 0xDA) +/* * Extend this array with ACPI Hardware IDs of devices known to be working * plus the number of link-frequencies expected by their drivers, along with
@@ -64,6 +76,60 @@ static const char * const ipu_vcm_types[] = { "lc898212axb", }; +/* +* Used to figure out IVSC device by ipu_bridge_check_ivsc_dev() +* instead of device and driver match to probe IVSC device. +*/ +static const struct acpi_device_id ivsc_acpi_ids[] = { + {"INTC1059" }, + {"INTC1095" }, + {"INTC100A" }, + {"INTC10CF" }, +}; + +static int ipu_bridge_check_ivsc_dev (struct ipu_sensor *sensor, +struct acpi_device *sensor_adev) +{ +acpi_handle handle =acpi_device_handle (sensor_adev); +struct acpi_device *consumer, *adev; +uuid_le uuid =MEI_CSI_UUID ; +struct device *csi_dev; +unsigned int i; +char name[64 ]; + +for (i =0 ; i <ARRAY_SIZE (ivsc_acpi_ids); i++) { +const struct acpi_device_id *acpi_id = &ivsc_acpi_ids[i]; + +for_each_acpi_dev_match (adev, acpi_id->id,NULL ,-1 ) { +/* camera sensor depends on IVSC in DSDT */ +for_each_acpi_consumer_dev (adev, consumer) +if (consumer->handle == handle) +break ; + +if (!consumer) +continue ; + +snprintf (name,sizeof (name),"%s-%pUl" , +dev_name (&adev->dev), &uuid); + + csi_dev =device_find_child_by_name (&adev->dev, name); +if (!csi_dev) { +acpi_dev_put (adev); +dev_err (&adev->dev, +"Failed to find MEI CSI dev\n" ); +return -ENODEV ; + } + + sensor->csi_dev = csi_dev; + sensor->ivsc_adev = adev; + +return 0 ; + } + } + +return 0 ; +} +static int ipu_bridge_read_acpi_buffer (struct acpi_device *adev,char *id,void *data,u32 size) {
@@ -189,10 +255,48 @@ static void ipu_bridge_create_fwnode_properties(struct ipu_bridge *bridge, const struct ipu_sensor_config *cfg) { + struct ipu_property_names *names = &sensor->prop_names; +struct software_node *nodes = sensor->swnodes; + sensor->prop_names = prop_names; - sensor->local_ref[0 ] =SOFTWARE_NODE_REFERENCE (&sensor->swnodes[SWNODE_IPU_ENDPOINT ]); - sensor->remote_ref[0 ] =SOFTWARE_NODE_REFERENCE (&sensor->swnodes[SWNODE_SENSOR_ENDPOINT ]); +if (sensor->csi_dev) { + sensor->local_ref[0 ] = +SOFTWARE_NODE_REFERENCE (&nodes[SWNODE_IVSC_SENSOR_ENDPOINT ]); + sensor->remote_ref[0 ] = +SOFTWARE_NODE_REFERENCE (&nodes[SWNODE_IVSC_IPU_ENDPOINT ]); + sensor->ivsc_sensor_ref[0 ] = +SOFTWARE_NODE_REFERENCE (&nodes[SWNODE_SENSOR_ENDPOINT ]); + sensor->ivsc_ipu_ref[0 ] = +SOFTWARE_NODE_REFERENCE (&nodes[SWNODE_IPU_ENDPOINT ]); + + sensor->ivsc_sensor_ep_properties[0 ] = +PROPERTY_ENTRY_U32 (names->bus_type, +V4L2_FWNODE_BUS_TYPE_CSI2_DPHY ); + sensor->ivsc_sensor_ep_properties[1 ] = +PROPERTY_ENTRY_U32_ARRAY_LEN (names->data_lanes, + bridge->data_lanes, + sensor->lanes); + sensor->ivsc_sensor_ep_properties[2 ] = +PROPERTY_ENTRY_REF_ARRAY (names->remote_endpoint, + sensor->ivsc_sensor_ref); + + sensor->ivsc_ipu_ep_properties[0 ] = +PROPERTY_ENTRY_U32 (names->bus_type, +V4L2_FWNODE_BUS_TYPE_CSI2_DPHY ); + sensor->ivsc_ipu_ep_properties[1 ] = +PROPERTY_ENTRY_U32_ARRAY_LEN (names->data_lanes, + bridge->data_lanes, + sensor->lanes); + sensor->ivsc_ipu_ep_properties[2 ] = +PROPERTY_ENTRY_REF_ARRAY (names->remote_endpoint, + sensor->ivsc_ipu_ref); + }else { + sensor->local_ref[0 ] = +SOFTWARE_NODE_REFERENCE (&nodes[SWNODE_IPU_ENDPOINT ]); + sensor->remote_ref[0 ] = +SOFTWARE_NODE_REFERENCE (&nodes[SWNODE_SENSOR_ENDPOINT ]); + } sensor->dev_properties[0] = PROPERTY_ENTRY_U32( sensor->prop_names.clock_frequency,
@@ -205,7 +309,7 @@ static void ipu_bridge_create_fwnode_properties( sensor->orientation);if (sensor->vcm_type) { sensor->vcm_ref[0 ] = -SOFTWARE_NODE_REFERENCE (&sensor->swnodes[SWNODE_VCM ]); +SOFTWARE_NODE_REFERENCE (&nodes[SWNODE_VCM ]); sensor->dev_properties[3 ] =PROPERTY_ENTRY_REF_ARRAY ("lens-focus" , sensor->vcm_ref); }
@@ -245,11 +349,21 @@ static void ipu_bridge_init_swnode_names(struct ipu_sensor *sensor)snprintf (sensor->node_names.endpoint,sizeof (sensor->node_names.endpoint),SWNODE_GRAPH_ENDPOINT_NAME_FMT ,0 );/* And endpoint 0 */ +if (sensor->vcm_type) {/* append link to distinguish nodes with same model VCM */ snprintf (sensor->node_names.vcm,sizeof (sensor->node_names.vcm),"%s-%u" , sensor->vcm_type, sensor->link); } + +if (sensor->csi_dev) { +snprintf (sensor->node_names.ivsc_sensor_port, +sizeof (sensor->node_names.ivsc_sensor_port), +SWNODE_GRAPH_PORT_NAME_FMT ,0 ); +snprintf (sensor->node_names.ivsc_ipu_port, +sizeof (sensor->node_names.ivsc_ipu_port), +SWNODE_GRAPH_PORT_NAME_FMT ,1 ); + } }static void ipu_bridge_init_swnode_group (struct ipu_sensor *sensor)
@@ -261,13 +375,31 @@ static void ipu_bridge_init_swnode_group(struct ipu_sensor *sensor) sensor->group[SWNODE_SENSOR_ENDPOINT ] = &nodes[SWNODE_SENSOR_ENDPOINT ]; sensor->group[SWNODE_IPU_PORT ] = &nodes[SWNODE_IPU_PORT ]; sensor->group[SWNODE_IPU_ENDPOINT ] = &nodes[SWNODE_IPU_ENDPOINT ]; -if (sensor->vcm_type) - sensor->group[SWNODE_VCM ] = &nodes[SWNODE_VCM ]; + +if (sensor->csi_dev) { + sensor->group[SWNODE_IVSC_HID ] = + &nodes[SWNODE_IVSC_HID ]; + sensor->group[SWNODE_IVSC_SENSOR_PORT ] = + &nodes[SWNODE_IVSC_SENSOR_PORT ]; + sensor->group[SWNODE_IVSC_SENSOR_ENDPOINT ] = + &nodes[SWNODE_IVSC_SENSOR_ENDPOINT ]; + sensor->group[SWNODE_IVSC_IPU_PORT ] = + &nodes[SWNODE_IVSC_IPU_PORT ]; + sensor->group[SWNODE_IVSC_IPU_ENDPOINT ] = + &nodes[SWNODE_IVSC_IPU_ENDPOINT ]; + +if (sensor->vcm_type) + sensor->group[SWNODE_VCM ] = &nodes[SWNODE_VCM ]; + }else { +if (sensor->vcm_type) + sensor->group[SWNODE_IVSC_HID ] = &nodes[SWNODE_VCM ]; + } }static void ipu_bridge_create_connection_swnodes (struct ipu_bridge *bridge,struct ipu_sensor *sensor) { +struct ipu_node_names *names = &sensor->node_names;struct software_node *nodes = sensor->swnodes;ipu_bridge_init_swnode_names (sensor);
@@ -288,6 +420,28 @@ static void ipu_bridge_create_connection_swnodes(struct ipu_bridge *bridge, sensor->ipu_properties); nodes[SWNODE_VCM ] =NODE_VCM (sensor->node_names.vcm); +if (sensor->csi_dev) { +snprintf (sensor->ivsc_name,sizeof (sensor->ivsc_name),"%s-%u" , +acpi_device_hid (sensor->ivsc_adev), sensor->link); + + nodes[SWNODE_IVSC_HID ] =NODE_SENSOR (sensor->ivsc_name, + sensor->ivsc_properties); + nodes[SWNODE_IVSC_SENSOR_PORT ] = +NODE_PORT (names->ivsc_sensor_port, + &nodes[SWNODE_IVSC_HID ]); + nodes[SWNODE_IVSC_SENSOR_ENDPOINT ] = +NODE_ENDPOINT (names->endpoint, + &nodes[SWNODE_IVSC_SENSOR_PORT ], + sensor->ivsc_sensor_ep_properties); + nodes[SWNODE_IVSC_IPU_PORT ] = +NODE_PORT (names->ivsc_ipu_port, + &nodes[SWNODE_IVSC_HID ]); + nodes[SWNODE_IVSC_IPU_ENDPOINT ] = +NODE_ENDPOINT (names->endpoint, + &nodes[SWNODE_IVSC_IPU_PORT ], + sensor->ivsc_ipu_ep_properties); + } +ipu_bridge_init_swnode_group (sensor); }
@@ -397,6 +551,22 @@ int ipu_bridge_instantiate_vcm(struct device *sensor) }EXPORT_SYMBOL (ipu_bridge_instantiate_vcm); +static int ipu_bridge_instantiate_ivsc (struct ipu_sensor *sensor) +{ +struct fwnode_handle *fwnode; + +if (!sensor->csi_dev) +return 0 ; + + fwnode =software_node_fwnode (&sensor->swnodes[SWNODE_IVSC_HID ]); +if (!fwnode) +return -ENODEV ; + +set_secondary_fwnode (sensor->csi_dev, fwnode); + +return 0 ; +} +static void ipu_bridge_unregister_sensors (struct ipu_bridge *bridge) {struct ipu_sensor *sensor;
@@ -406,6 +576,8 @@ static void ipu_bridge_unregister_sensors(struct ipu_bridge *bridge) sensor = &bridge->sensors[i];software_node_unregister_node_group (sensor->group);acpi_dev_put (sensor->adev); +put_device (sensor->csi_dev); +acpi_dev_put (sensor->ivsc_adev); } }
@@ -436,12 +608,16 @@ static int ipu_bridge_connect_sensor(const struct ipu_sensor_config *cfg,snprintf (sensor->name,sizeof (sensor->name),"%s-%u" , cfg->hid, sensor->link); + ret =ipu_bridge_check_ivsc_dev (sensor, adev); +if (ret) + goto err_put_adev; +ipu_bridge_create_fwnode_properties (sensor, bridge, cfg);ipu_bridge_create_connection_swnodes (bridge, sensor); ret =software_node_register_node_group (sensor->group);if (ret) - goto err_put_adev; + goto err_put_ivsc; fwnode =software_node_fwnode (&sensor->swnodes[SWNODE_SENSOR_HID ]);
@@ -455,6 +631,10 @@ static int ipu_bridge_connect_sensor(const struct ipu_sensor_config *cfg, primary =acpi_fwnode_handle (adev); primary->secondary = fwnode; + ret =ipu_bridge_instantiate_ivsc (sensor); +if (ret) + goto err_free_swnodes; +dev_info (bridge->dev,"Found supported sensor %s\n" ,acpi_dev_name (adev));
@@ -465,6 +645,9 @@ static int ipu_bridge_connect_sensor(const struct ipu_sensor_config *cfg, err_free_swnodes:software_node_unregister_node_group (sensor->group); +err_put_ivsc: +put_device (sensor->csi_dev); +acpi_dev_put (sensor->ivsc_adev); err_put_adev:acpi_dev_put (adev);return ret;
include/media/ipu-bridge.h +18 -1
@@ -53,7 +53,12 @@ enum ipu_sensor_swnodes {SWNODE_SENSOR_ENDPOINT ,SWNODE_IPU_PORT ,SWNODE_IPU_ENDPOINT , -/* Must be last because it is optional / maybe empty */ +/* below are optional / maybe empty */ +SWNODE_IVSC_HID , +SWNODE_IVSC_SENSOR_PORT , +SWNODE_IVSC_SENSOR_ENDPOINT , +SWNODE_IVSC_IPU_PORT , +SWNODE_IVSC_IPU_ENDPOINT ,SWNODE_VCM ,SWNODE_COUNT };
@@ -100,6 +105,8 @@ struct ipu_property_names {struct ipu_node_names {char port[7 ]; +char ivsc_sensor_port[7 ]; +char ivsc_ipu_port[7 ];char endpoint[11 ];char remote_port[7 ];char vcm[16 ];
@@ -116,6 +123,10 @@ struct ipu_sensor {char name[ACPI_ID_LEN +4 ];struct acpi_device *adev; +struct device *csi_dev; +struct acpi_device *ivsc_adev; +char ivsc_name[ACPI_ID_LEN +4 ]; +/* SWNODE_COUNT + 1 for terminating NULL */ const struct software_node *group[SWNODE_COUNT +1 ];struct software_node swnodes[SWNODE_COUNT ];
@@ -132,9 +143,15 @@ struct ipu_sensor {struct property_entry ep_properties[5 ];struct property_entry dev_properties[5 ];struct property_entry ipu_properties[3 ]; +struct property_entry ivsc_properties[1 ]; +struct property_entry ivsc_sensor_ep_properties[4 ]; +struct property_entry ivsc_ipu_ep_properties[4 ]; +struct software_node_ref_args local_ref[1 ];struct software_node_ref_args remote_ref[1 ];struct software_node_ref_args vcm_ref[1 ]; +struct software_node_ref_args ivsc_sensor_ref[1 ]; +struct software_node_ref_args ivsc_ipu_ref[1 ]; };struct ipu_bridge {
Signature · 1 line
2.39.2
Re: [PATCH v2 1/2] media: ipu-bridge: add IVSC support
Quoted 22 lines · Sakari Ailus
From: Wentong Wu <[email protected]>
Previously on ACPI platforms, sensors that are intended to be connected to a IPU device for use with the ipu3-cio2 driver lacking the necessary connection information in firmware. IPU bridge driver is to connect sensors to IPU device via software nodes.
Currently IVSC located between IPU device and sensors is available in existing commercial platforms from multiple OEMs. But the connection information between them in firmware is also not enough to build V4L2 connection graph. This patch parses the connection properties from the SSDB buffer in DSDT and build the connection using software nodes.
IVSC driver is based on MEI framework (previously known as HECI), it has two MEI clients, MEI CSI and MEI ACE. Both clients are used to communicate messages with IVSC firmware. Linux abstracts MEI client as a device, whose bus type is MEI. And the device is addressed by a GUID/UUID which is part of the device name of MEI client. After figured out MEI CSI via the UUID composed device name, this patch setup the connection between MEI CSI and IPU, and the connection between MEI CSI and sensor via software nodes.
Quoted 3 lines
Signed-off-by Wentong Wu <[email protected]>Reviewed-by Andy Shevchenko <[email protected]>[Sakari Ailus: rebase on Hans de Goede's cleanups and fixes]
I believe we use past tense in these comment lines...
Quoted 1 line
Signed-off-by Sakari Ailus <[email protected]>
Signature · 2 lines
With Best Regards, Andy Shevchenko
Re: [PATCH v2 1/2] media: ipu-bridge: add IVSC support
Hi Andy,
Quoted 23 lines · Andy Shevchenko
Quoted 22 lines · Sakari Ailus
From: Wentong Wu <[email protected]>
Previously on ACPI platforms, sensors that are intended to be connected to a IPU device for use with the ipu3-cio2 driver lacking the necessary connection information in firmware. IPU bridge driver is to connect sensors to IPU device via software nodes.
Currently IVSC located between IPU device and sensors is available in existing commercial platforms from multiple OEMs. But the connection information between them in firmware is also not enough to build V4L2 connection graph. This patch parses the connection properties from the SSDB buffer in DSDT and build the connection using software nodes.
IVSC driver is based on MEI framework (previously known as HECI), it has two MEI clients, MEI CSI and MEI ACE. Both clients are used to communicate messages with IVSC firmware. Linux abstracts MEI client as a device, whose bus type is MEI. And the device is addressed by a GUID/UUID which is part of the device name of MEI client. After figured out MEI CSI via the UUID composed device name, this patch setup the connection between MEI CSI and IPU, and the connection between MEI CSI and sensor via software nodes.
Quoted 5 lines
Quoted 3 lines
Signed-off-by Wentong Wu <[email protected]>Reviewed-by Andy Shevchenko <[email protected]>[Sakari Ailus: rebase on Hans de Goede's cleanups and fixes]I believe we use past tense in these comment lines...
No objections, I'll add the missing "d" while applying if there are no other comments.
Quoted 3 lines
Quoted 2 lines
Signed-off-by Sakari Ailus <[email protected]>
Signature · 3 lines
Regards, Sakari Ailus
PATCH v2 2/2 media: ipu-bridge: Make exported IPU bridge symbol GPL-only in a NS
In reply to Sakari AilusSwitch to EXPORT_SYMBOL_NS_GPL() on ipu_bridge_instantiate_vcm(). The rest of the ipu bridge symbols are this way already.
--- drivers/media/pci/intel/ipu-bridge.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/media/pci/intel/ipu-bridge.c +1 -1
@@ -549,7 +549,7 @@ int ipu_bridge_instantiate_vcm(struct device *sensor)return 0 ; } -EXPORT_SYMBOL (ipu_bridge_instantiate_vcm); +EXPORT_SYMBOL_NS_GPL (ipu_bridge_instantiate_vcm,INTEL_IPU_BRIDGE );static int ipu_bridge_instantiate_ivsc (struct ipu_sensor *sensor) {
Signature · 1 line
2.39.2
Re: [PATCH v2 2/2] media: ipu-bridge: Make exported IPU bridge symbol GPL-only in a NS
Quoted 2 lines · Sakari Ailus
Switch to
EXPORT_SYMBOL_NS_GPL()onipu_bridge_instantiate_vcm(). The rest of the ipu bridge symbols are this way already.
Hmm... This actually changes the license, Why is it initially non-GPL and what are the consequences of this change?
...
Quoted 2 lines
drivers/media/pci/intel/ipu-bridge.c +1
-1-EXPORT_SYMBOL (ipu_bridge_instantiate_vcm); +EXPORT_SYMBOL_NS_GPL (ipu_bridge_instantiate_vcm,INTEL_IPU_BRIDGE );
Signature · 2 lines
With Best Regards, Andy Shevchenko
Re: [PATCH v2 2/2] media: ipu-bridge: Make exported IPU bridge symbol GPL-only in a NS
Hi Andy,
Quoted 6 lines · Andy Shevchenko
Quoted 2 lines · Sakari Ailus
Switch to
EXPORT_SYMBOL_NS_GPL()onipu_bridge_instantiate_vcm(). The rest of the ipu bridge symbols are this way already.Hmm... This actually changes the license, Why is it initially non-GPL and what are the consequences of this change?
This does not change the license. It's just that the function will be only available for GPL'd users. I'd think the reason why the plain EXPORT_SYMBOL() is used is plainly historical.
In any case, after Hans's set the other two functions required for using the IPU bridge are GPL-only.
Signature · 3 lines
Regards, Sakari Ailus
Re: [PATCH v2 2/2] media: ipu-bridge: Make exported IPU bridge symbol GPL-only in a NS
Hi,
Quoted 10 lines · Sakari Ailus
Hi Andy,
Quoted 6 lines · Andy Shevchenko
Quoted 2 lines · Sakari Ailus
Switch to
EXPORT_SYMBOL_NS_GPL()onipu_bridge_instantiate_vcm(). The rest of the ipu bridge symbols are this way already.Hmm... This actually changes the license, Why is it initially non-GPL and what are the consequences of this change?
Quoted 6 lines
This does not change the license. It's just that the function will be only available for GPL'd users. I'd think the reason why the plain
EXPORT_SYMBOL()is used is plainly historical.In any case, after Hans's set the other two functions required for using the IPU bridge are GPL-only.
Right I'll fix this in v3 of my series so that this symbol uses EXPORT_SYMBOL_NS_GPL() from the get go like the other 2 symbols.
Regards,
Hans