Skip to contentSkip to conversation

imx8-isi: Bug fixes and format support enhancements

PATCH Superseded Guoniu Zhou 4 participants 24 messages Jul 20 – Jul 22

Subsystems (4)

v2 of 5 Superseded Latest is v5 →

lore
Share & tools
lore
#0 lore raw

v2 cover imx8-isi: Bug fixes and format support enhancements

This series addresses critical bugs in the imx8-isi driver and extends format support for high-end sensors and Android requirements.

Patch 1 fixes a critical stream ID validation bug in the crossbar routing where the validation loop iterates over the wrong routing table, allowing userspace to bypass validation entirely and configure invalid routes.

Patch 2 adds additional stream ID validation to enforce hardware constraints (SOURCE stream must be 0).

Patch 3 fixes a stream reference counting bug in the crossbar that prevents multiple streams from different virtual channels on the same input pad from being enabled correctly. Only the first stream gets enabled in hardware, subsequent streams are silently ignored.

Patch 4 adds support for 16-bit raw Bayer formats (SBGGR16, SGBRG16, SGRBG16, SRGGB16) commonly used by high-end image sensors.

Patch 5 fixes incorrect color mapping for XBGR32 format in memory-to-memory mode (marked for stable backport).

Patch 6 extends RGB format support by adding BGRA32, RGBA32, BGRX32, RGBX32, and ARGB2101010 formats with full M2M capabilities to meet Android requirements.

---
Changes in v2:
- [1/6] Split v1 patch 1/5 into two patches: this patch fixes the core for_each_active_route() bug, next patch adds additional stream validation (Frank Li)
- [2/6] New patch split from v1 1/5: adds stream ID validation on top of for_each_active_route() fix (Frank Li)
- [2/6] Remove incorrect sink_stream validation
- [3/6] Use fixed-size array for enabled_count instead of dynamic allocation
- [3/6] Use BIT_ULL() macro for u64 bitmask operations
- [3/6] Use MXC_ISI_MAX_STREAMS (64) as loop boundary instead of num_sources
- [3/6] Remove mxc_isi_stream_counters_alloc/free functions
- [4/6] Add Reviewed-by tag from Frank Li
- [5/6] Reword commit description for clarity (Frank Li)
- [6/6] Add Reviewed-by tag from Frank Li
- Link to v1: https://lore.kernel.org/r/[email protected]

---
Guoniu Zhou (5):
media: nxp: imx8-isi: Fix stream ID validation bypass in crossbar routing
media: nxp: imx8-isi: Add stream ID validation for crossbar routing
media: nxp: imx8-isi: Fix per-stream reference counting for multiplexed streams
media: nxp: imx8-isi: Correct color map between V4L2 and ISI
media: nxp: imx8-isi: Add additional 32-bit RGB format support

Laurentiu Palcu (1):
media: nxp: imx8-isi: Add 16-bit raw Bayer format support

 .../media/platform/nxp/imx8-isi/imx8-isi-core.h    |  7 +-
 .../platform/nxp/imx8-isi/imx8-isi-crossbar.c      | 92 +++++++++++++++-----
 .../media/platform/nxp/imx8-isi/imx8-isi-pipe.c    | 24 ++++++
 .../media/platform/nxp/imx8-isi/imx8-isi-video.c   | 97 +++++++++++++++++++++-
 4 files changed, 196 insertions(+), 24 deletions(-)

---
base-commit: 06cb687a5132fcffe624c0070576ab852ac6b568
change-id: 20260626-isi-00f05b044ac9

Best regards,

Signature · 1 line
Guoniu Zhou <[email protected]>
#1 lore raw

PATCH v2 1/6 media: nxp: imx8-isi: Fix stream ID validation bypass in crossbar routing

The crossbar routing validation has a critical bug where it validates the wrong routing table, allowing userspace to bypass validation entirely.

The __mxc_isi_crossbar_set_routing() function is called to validate and apply a new routing table from userspace. However, the validation loop iterates over state->routing (the currently active routing table) instead of the routing parameter (the new table being validated):

    for_each_active_route(&state->routing, route) {

This means userspace can submit any invalid routing configuration and it will pass validation as long as the currently active routing is valid. This is a security issue as it allows userspace to configure routes that violate hardware constraints, potentially causing undefined hardware behavior.

Fix by validating the routing table that will actually be applied.

Fixes cf21f328fcaf ("media: nxp: Add i.MX8 ISI driver")

---
Changes in v2:
- Split v1 patch 1/5 into two patches: this patch fixes the core for_each_active_route() bug, next patch adds additional stream validation (Frank Li)

---
 drivers/media/platform/nxp/imx8-isi/imx8-isi-crossbar.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/media/platform/nxp/imx8-isi/imx8-isi-crossbar.c +1 -1
index c580c831972e..84871bceb31d 100644
@@ -107,7 +107,7 @@ static int __mxc_isi_crossbar_set_routing(struct v4l2_subdev *sd,
 		return ret;
 
 	/* The memory input can be routed to the first pipeline only. */
-	for_each_active_route(&state->routing, route) {
+	for_each_active_route(routing, route) {
 		if (route->sink_pad == xbar->num_sinks - 1 &&
 		    route->source_pad != xbar->num_sinks) {
 			dev_dbg(xbar->isi->dev,
Signature · 1 line
2.34.1
#2 lore raw

Re: [PATCH v2 1/6] media: nxp: imx8-isi: Fix stream ID validation bypass in crossbar routing

Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:

Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:

Pre-existing issues:
- [High] Undefined behavior from bit shift overflow when using BIT() macro on user-controlled stream IDs on 32-bit architectures.
- [High] Missing V4L2_SUBDEV_ROUTING_NO_SOURCE_STREAM_MIX flag in routing validation allows stream mixing, violating driver assumptions and causing pipeline state corruption. --

--- Patch [1]: [PATCH v2 1/6] media: nxp: imx8-isi: Fix stream ID validation bypass in crossbar routing --- commit c48b83ab851ce7535e8d8249fb85d35c57bb63e1
Author: Guoniu Zhou <[email protected]>

media: nxp: imx8-isi: Fix stream ID validation bypass in crossbar routing

The crossbar routing validation has a bug where it validates the wrong routing table, allowing userspace to bypass validation entirely. This commit fixes it by validating the routing table that will actually be applied.

drivers/media/platform/nxp/imx8-isi/imx8-isi-crossbar.c static int __mxc_isi_crossbar_set_routing(struct v4l2_subdev *sd, · line 107

Quoted 6 lines
drivers/media/platform/nxp/imx8-isi/imx8-isi-crossbar.c +0 -0
index c580c831972ec..84871bceb31db 100644
@@ -107,7 +107,7 @@ static int __mxc_isi_crossbar_set_routing(struct v4l2_subdev *sd,
 		return ret;

[Severity: High]
This isn't a bug introduced by this patch, but does the routing validation here miss the V4L2_SUBDEV_ROUTING_NO_SOURCE_STREAM_MIX flag?

The validation in __mxc_isi_crossbar_set_routing() does not pass this flag:

imx8-isi-crossbar.c:__mxc_isi_crossbar_set_routing() {
    ...
	ret = v4l2_subdev_routing_validate(sd, routing,
					   V4L2_SUBDEV_ROUTING_NO_N_TO_1);
    ...
}

This allows stream mixing, violating an explicit assumption in mxc_isi_crossbar_xlate_streams(), where the driver silently overwrites sink_pad for matching routes:

imx8-isi-crossbar.c:mxc_isi_crossbar_xlate_streams() {
    ...
	/*
	 * Translate the source pad and streams to the sink side. The routing
	 * validation forbids stream merging, so all matching entries in the
	 * routing table are guaranteed to have the same sink pad.
	 */
    ...
		sink_streams |= BIT(route->sink_stream);
		sink_pad = route->sink_pad;
    ...
}

If userspace configures a routing table via VIDIOC_SUBDEV_S_ROUTING with multiple routes from different sink pads ending at the same source pad, mxc_isi_crossbar_enable_streams() will only enable streams on the last matched remote subdevice, leaving others disabled and leading to hardware pipeline hangs.

drivers/media/platform/nxp/imx8-isi/imx8-isi-crossbar.c static int __mxc_isi_crossbar_set_routing(struct v4l2_subdev *sd, · line 110

Quoted 4 lines
drivers/media/platform/nxp/imx8-isi/imx8-isi-crossbar.c +1 -1
 	/* The memory input can be routed to the first pipeline only. */
-	for_each_active_route(&state->routing, route) {
+	for_each_active_route(routing, route) {

[Severity: High]
This is a pre-existing issue, but can using the BIT() macro on user-controlled stream IDs cause undefined behavior on 32-bit architectures?

The core framework limits stream IDs to 63 (V4L2_SUBDEV_MAX_STREAM_ID), but the driver uses BIT() on these IDs in mxc_isi_crossbar_xlate_streams():

imx8-isi-crossbar.c:mxc_isi_crossbar_xlate_streams() {
    ...
		if (route->source_pad != source_pad ||
		    !(source_streams & BIT(route->source_stream)))
			continue;

		sink_streams |= BIT(route->sink_stream);
    ...
}

On 32-bit architectures, unsigned long is 32 bits. If userspace provides a routing table via VIDIOC_SUBDEV_S_ROUTING with stream IDs >= 32, this causes a shift out of bounds. This can result in incorrect bitmasks, causing the driver to start or stop the wrong hardware streams and hang the pipeline.

Quoted 3 lines
drivers/media/platform/nxp/imx8-isi/imx8-isi-crossbar.c +0 -0
 		if (route->sink_pad == xbar->num_sinks - 1 &&
 		    route->source_pad != xbar->num_sinks) {
 			dev_dbg(xbar->isi->dev,
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1
Reviewed-by on v2 1/6
#3 lore raw

Re: [PATCH v2 1/6] media: nxp: imx8-isi: Fix stream ID validation bypass in crossbar routing

In reply to Guoniu Zhou
Quoted 16 lines · Guoniu Zhou

The crossbar routing validation has a critical bug where it validates the wrong routing table, allowing userspace to bypass validation entirely.

The __mxc_isi_crossbar_set_routing() function is called to validate and apply a new routing table from userspace. However, the validation loop iterates over state->routing (the currently active routing table) instead of the routing parameter (the new table being validated):

    for_each_active_route(&state->routing, route) {

This means userspace can submit any invalid routing configuration and it will pass validation as long as the currently active routing is valid. This is a security issue as it allows userspace to configure routes that violate hardware constraints, potentially causing undefined hardware behavior.

Quoted 6 lines

Fix by validating the routing table that will actually be applied.

Fixes cf21f328fcaf ("media: nxp: Add i.MX8 ISI driver")

---

Quoted 26 lines

Changes in v2:
- Split v1 patch 1/5 into two patches: this patch fixes the core for_each_active_route() bug, next patch adds additional stream validation (Frank Li)

---
 drivers/media/platform/nxp/imx8-isi/imx8-isi-crossbar.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/media/platform/nxp/imx8-isi/imx8-isi-crossbar.c +1 -2
index c580c831972e..84871bceb31d 100644
@@ -107,7 +107,7 @@ static int __mxc_isi_crossbar_set_routing(struct v4l2_subdev *sd,
 		return ret;
 
 	/* The memory input can be routed to the first pipeline only. */
-	for_each_active_route(&state->routing, route) {
+	for_each_active_route(routing, route) {
 		if (route->sink_pad == xbar->num_sinks - 1 &&
 		    route->source_pad != xbar->num_sinks) {
 			dev_dbg(xbar->isi->dev,
 
--

2.34.1

maintainer Reviewed-by on v2 1/6
#4 lore raw

Re: [PATCH v2 1/6] media: nxp: imx8-isi: Fix stream ID validation bypass in crossbar routing

In reply to Guoniu Zhou

Hi Guoniu,

Thank you for the patch.

Quoted 16 lines · Guoniu Zhou

The crossbar routing validation has a critical bug where it validates the wrong routing table, allowing userspace to bypass validation entirely.

The __mxc_isi_crossbar_set_routing() function is called to validate and apply a new routing table from userspace. However, the validation loop iterates over state->routing (the currently active routing table) instead of the routing parameter (the new table being validated):

    for_each_active_route(&state->routing, route) {

This means userspace can submit any invalid routing configuration and it will pass validation as long as the currently active routing is valid. This is a security issue as it allows userspace to configure routes that violate hardware constraints, potentially causing undefined hardware behavior.

Quoted 5 lines

Fix by validating the routing table that will actually be applied.

Fixes cf21f328fcaf ("media: nxp: Add i.MX8 ISI driver")
Quoted 23 lines

---
Changes in v2:
- Split v1 patch 1/5 into two patches: this patch fixes the core for_each_active_route() bug, next patch adds additional stream validation (Frank Li)

---
 drivers/media/platform/nxp/imx8-isi/imx8-isi-crossbar.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/media/platform/nxp/imx8-isi/imx8-isi-crossbar.c +1 -1
index c580c831972e..84871bceb31d 100644
@@ -107,7 +107,7 @@ static int __mxc_isi_crossbar_set_routing(struct v4l2_subdev *sd,
 		return ret;
 
 	/* The memory input can be routed to the first pipeline only. */
-	for_each_active_route(&state->routing, route) {
+	for_each_active_route(routing, route) {
 		if (route->sink_pad == xbar->num_sinks - 1 &&
 		    route->source_pad != xbar->num_sinks) {
 			dev_dbg(xbar->isi->dev,
Signature · 3 lines
Regards,

Laurent Pinchart
#5 lore raw

PATCH v2 2/6 media: nxp: imx8-isi: Add stream ID validation for crossbar routing

In reply to Guoniu Zhou

Add validation to enforce hardware constraints that were previously missing in the crossbar routing configuration:

- SOURCE stream must be 0 (ISI pipes are hardcoded to stream 0)

This check complements the existing memory input validation and ensures that all routing configurations respect hardware limitations.

---
Changes in v2:
- New patch split from v1 1/5: adds stream ID validation on top of for_each_active_route() fix (Frank Li)
- Remove incorrect sink_stream validation

---
 drivers/media/platform/nxp/imx8-isi/imx8-isi-crossbar.c | 14 +++++++++++++-
 1 file changed, 13 insertions(+), 1 deletion(-)
drivers/media/platform/nxp/imx8-isi/imx8-isi-crossbar.c +13 -1
index 84871bceb31d..328d08a278ea 100644
@@ -106,8 +106,20 @@ static int __mxc_isi_crossbar_set_routing(struct v4l2_subdev *sd,
 	if (ret)
 		return ret;
 
-	/* The memory input can be routed to the first pipeline only. */
+	/*
+	 * Validate routes against hardware constraints:
+	 * - SOURCE stream must be 0 (pipes are hardcoded to stream 0)
+	 * - Memory input can only route to the first pipeline
+	 */
 	for_each_active_route(routing, route) {
+		if (route->source_stream != 0) {
+			dev_dbg(xbar->isi->dev,
+				"route to pipe %u must use source_stream=0, got %u\n",
+				route->source_pad - xbar->num_sinks,
+				route->source_stream);
+			return -ENXIO;
+		}
+
 		if (route->sink_pad == xbar->num_sinks - 1 &&
 		    route->source_pad != xbar->num_sinks) {
 			dev_dbg(xbar->isi->dev,
Signature · 1 line
2.34.1
#6 lore raw

Re: [PATCH v2 2/6] media: nxp: imx8-isi: Add stream ID validation for crossbar routing

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:

Pre-existing issues:
- [High] The driver uses the BIT() macro on 64-bit stream masks (source_streams and sink_streams), which will cause undefined behavior on 32-bit platforms if the stream ID exceeds 31. --

--- Patch [2]: [PATCH v2 2/6] media: nxp: imx8-isi: Add stream ID validation for crossbar routing --- commit d756aed8e216cd683009378d45743e6351a90389
Author: Guoniu Zhou <[email protected]>

media: nxp: imx8-isi: Add stream ID validation for crossbar routing

This commit adds validation to enforce hardware constraints in the crossbar routing configuration. It ensures that the SOURCE stream is strictly set to 0, as ISI pipes are hardcoded to stream 0. It also ensures memory input respects routing limitations to the first pipeline.

Quoted 6 lines
drivers/media/platform/nxp/imx8-isi/imx8-isi-crossbar.c +0 -0
index 84871bceb31db..328d08a278eaf 100644
@@ -106,8 +106,20 @@ static int __mxc_isi_crossbar_set_routing(struct v4l2_subdev *sd,
 	if (ret)

drivers/media/platform/nxp/imx8-isi/imx8-isi-crossbar.c static int __mxc_isi_crossbar_set_routing(struct v4l2_subdev *sd, · line 115

Quoted 10 lines
drivers/media/platform/nxp/imx8-isi/imx8-isi-crossbar.c +6 -1
 		return ret;
 
-	/* The memory input can be routed to the first pipeline only. */
+	/*
+	 * Validate routes against hardware constraints:
+	 * - SOURCE stream must be 0 (pipes are hardcoded to stream 0)
+	 * - Memory input can only route to the first pipeline
+	 */
 	for_each_active_route(routing, route) {
+		if (route->source_stream != 0) {

[Severity: High]
This is a pre-existing issue, but while reviewing this stream ID validation, I noticed a potential bit shift overflow regarding stream IDs in the related mxc_isi_crossbar_xlate_streams() function:

drivers/media/platform/nxp/imx8-isi/imx8-isi-crossbar.c:mxc_isi_crossbar_xlate_streams() {
    ...
    if (!(source_streams & BIT(route->source_stream)))
        ...
    sink_streams |= BIT(route->sink_stream);
    ...
}

Does this code cause undefined behavior on 32-bit platforms if userspace provides a V4L2 routing configuration with a sink_stream or source_stream greater than or equal to 32?

Since the BIT() macro evaluates to a 32-bit value on 32-bit architectures, shifting it by 32 or more results in a bit shift overflow. This could allow the compiler to make unsafe optimization assumptions, potentially corrupting the 64-bit stream mask logic.

Should this use BIT_ULL() instead?

Quoted 10 lines
drivers/media/platform/nxp/imx8-isi/imx8-isi-crossbar.c +7 -0
+			dev_dbg(xbar->isi->dev,
+				"route to pipe %u must use source_stream=0, got %u\n",
+				route->source_pad - xbar->num_sinks,
+				route->source_stream);
+			return -ENXIO;
+		}
+
 		if (route->sink_pad == xbar->num_sinks - 1 &&
 		    route->source_pad != xbar->num_sinks) {
 			dev_dbg(xbar->isi->dev,

[ ... ]

Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=2
#7 lore raw

Re: [PATCH v2 2/6] media: nxp: imx8-isi: Add stream ID validation for crossbar routing

In reply to Guoniu Zhou
Quoted 25 lines · Guoniu Zhou

Add validation to enforce hardware constraints that were previously missing in the crossbar routing configuration:

- SOURCE stream must be 0 (ISI pipes are hardcoded to stream 0)

This check complements the existing memory input validation and ensures that all routing configurations respect hardware limitations.

---
Changes in v2:
- New patch split from v1 1/5: adds stream ID validation on top of for_each_active_route() fix (Frank Li)
- Remove incorrect sink_stream validation

---
 drivers/media/platform/nxp/imx8-isi/imx8-isi-crossbar.c | 14 +++++++++++++-
 1 file changed, 13 insertions(+), 1 deletion(-)
drivers/media/platform/nxp/imx8-isi/imx8-isi-crossbar.c +0 -0
index 84871bceb31d..328d08a278ea 100644
@@ -106,8 +106,20 @@ static int __mxc_isi_crossbar_set_routing(struct v4l2_subdev *sd,
 	if (ret)
 		return ret;

drivers/media/platform/nxp/imx8-isi/imx8-isi-crossbar.c static int __mxc_isi_crossbar_set_routing(struct v4l2_subdev *sd, · line 116

Quoted 10 lines
drivers/media/platform/nxp/imx8-isi/imx8-isi-crossbar.c +7 -1
 
-	/* The memory input can be routed to the first pipeline only. */
+	/*
+	 * Validate routes against hardware constraints:
+	 * - SOURCE stream must be 0 (pipes are hardcoded to stream 0)
+	 * - Memory input can only route to the first pipeline
+	 */
 	for_each_active_route(routing, route) {
+		if (route->source_stream != 0) {
+			dev_dbg(xbar->isi->dev,

it is one error, should be dev_err()?

Frank

Quoted 14 lines
drivers/media/platform/nxp/imx8-isi/imx8-isi-crossbar.c +6 -1
+				"route to pipe %u must use source_stream=0, got %u\n",
+				route->source_pad - xbar->num_sinks,
+				route->source_stream);
+			return -ENXIO;
+		}
+
 		if (route->sink_pad == xbar->num_sinks - 1 &&
 		    route->source_pad != xbar->num_sinks) {
 			dev_dbg(xbar->isi->dev,
 
--

2.34.1

maintainer Reviewed-by
#8 lore raw

Re: [PATCH v2 2/6] media: nxp: imx8-isi: Add stream ID validation for crossbar routing

Quoted 27 lines · Frank Li
Quoted 26 lines · Guoniu Zhou

Add validation to enforce hardware constraints that were previously missing in the crossbar routing configuration:

- SOURCE stream must be 0 (ISI pipes are hardcoded to stream 0)

This check complements the existing memory input validation and ensures that all routing configurations respect hardware limitations.

---
Changes in v2:
- New patch split from v1 1/5: adds stream ID validation on top of for_each_active_route() fix (Frank Li)
- Remove incorrect sink_stream validation

---
 drivers/media/platform/nxp/imx8-isi/imx8-isi-crossbar.c | 14 +++++++++++++-
 1 file changed, 13 insertions(+), 1 deletion(-)
drivers/media/platform/nxp/imx8-isi/imx8-isi-crossbar.c +0 -0
index 84871bceb31d..328d08a278ea 100644
@@ -106,8 +106,20 @@ static int __mxc_isi_crossbar_set_routing(struct v4l2_subdev *sd,
 	if (ret)
 		return ret;
 
Quoted 11 lines

drivers/media/platform/nxp/imx8-isi/imx8-isi-crossbar.c static int __mxc_isi_crossbar_set_routing(struct v4l2_subdev *sd, · line 116

Quoted 9 lines
drivers/media/platform/nxp/imx8-isi/imx8-isi-crossbar.c +7 -1
-	/* The memory input can be routed to the first pipeline only. */
+	/*
+	 * Validate routes against hardware constraints:
+	 * - SOURCE stream must be 0 (pipes are hardcoded to stream 0)
+	 * - Memory input can only route to the first pipeline
+	 */
 	for_each_active_route(routing, route) {
+		if (route->source_stream != 0) {
+			dev_dbg(xbar->isi->dev,

it is one error, should be dev_err()?

As this is an error that can be triggered by userspace, we prefer dev_dbg() to avoid giving unpriviledge userspace a way to flood the kernel log.

Quoted 9 lines
Quoted 9 lines
drivers/media/platform/nxp/imx8-isi/imx8-isi-crossbar.c +6 -0
+				"route to pipe %u must use source_stream=0, got %u\n",
+				route->source_pad - xbar->num_sinks,
+				route->source_stream);
+			return -ENXIO;
+		}
+
 		if (route->sink_pad == xbar->num_sinks - 1 &&
 		    route->source_pad != xbar->num_sinks) {
 			dev_dbg(xbar->isi->dev,
Signature · 3 lines
Regards,

Laurent Pinchart
#9 lore raw

PATCH v2 3/6 media: nxp: imx8-isi: Fix per-stream reference counting for multiplexed streams

In reply to Guoniu Zhou

The ISI crossbar fails to properly enable multiple streams from different virtual channels on the same input pad. Only the first stream gets enabled in hardware, subsequent streams are silently ignored.

The driver uses a single enable_count per input to track the input state. When enable_count is non-zero, the code assumes the input is already active and skips calling v4l2_subdev_enable_streams() for additional streams:

Call 1: enable_streams(stream 0)

    -> enable_count == 0, enable gasket and stream 0 in hardware
    -> enable_count = 1

Call 2: enable_streams(stream 1)

    -> enable_count == 1, skip hardware enable (BUG!)
    -> enable_count = 2
    -> stream 1 never gets enabled

Similarly on disable, when enable_count reaches zero, ALL streams are disabled regardless of which streams are actually still active.

Fix this by tracking per-stream state using:
- enabled_streams (u64 bitmask): tracks which streams are currently enabled
- enabled_count[] (array): per-stream reference counter to support the same stream being enabled/disabled multiple times

Now each stream is independently enabled/disabled in hardware based on the enabled_streams bitmask, while enabled_count[] provides reference counting for scenarios where the same stream is enabled multiple times, such as duplicate cases in the ISI stream.

Fixes cf21f328fcaf ("media: nxp: Add i.MX8 ISI driver")

---
Changes in v2:
- Use fixed-size array for enabled_count instead of dynamic allocation
- Use BIT_ULL() macro for u64 bitmask operations
- Use MXC_ISI_MAX_STREAMS (64) as loop boundary instead of num_sources
- Remove mxc_isi_stream_counters_alloc/free functions

---
 .../media/platform/nxp/imx8-isi/imx8-isi-core.h    |  7 +-
 .../platform/nxp/imx8-isi/imx8-isi-crossbar.c      | 76 ++++++++++++++++------
 2 files changed, 63 insertions(+), 20 deletions(-)
drivers/media/platform/nxp/imx8-isi/imx8-isi-core.h +6 -1
index 7547a6559d4c..9adbe2fe7cf8 100644
@@ -184,8 +184,13 @@ struct mxc_isi_dma_buffer {
 	dma_addr_t			dma;
 };
 
+/* V4L2 subdev max stream ID is 63, need 64 counters (0-63) */
+#define MXC_ISI_MAX_STREAMS		64
+
 struct mxc_isi_input {
-	unsigned int			enable_count;
+	u64				enabled_streams;
+	/* Per-stream reference counter */
+	unsigned int			enabled_count[MXC_ISI_MAX_STREAMS];
 };
 
 struct mxc_isi_crossbar {
drivers/media/platform/nxp/imx8-isi/imx8-isi-crossbar.c +57 -19

118 lines not on this page · raw message

Signature · 1 line
2.34.1
#10 lore raw

Re: [PATCH v2 3/6] media: nxp: imx8-isi: Fix per-stream reference counting for multiplexed streams

Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:

Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:

Pre-existing issues:
- [High] Unchecked return value from media_pad_remote_pad_first() causes a NULL pointer dereference if the sink pad has no enabled link.
- [Medium] Use of BIT() macro for 64-bit stream IDs causes undefined behavior and bitwise truncation on 32-bit platforms. --

--- Patch [3]: [PATCH v2 3/6] media: nxp: imx8-isi: Fix per-stream reference counting for multiplexed streams --- commit 976517cc78165ba12439fb004220b58271a0beea
Author: Guoniu Zhou <[email protected]>

media: nxp: imx8-isi: Fix per-stream reference counting for multiplexed streams

The driver uses a single enable_count per input to track the input state. This fails to properly enable multiple streams from different virtual channels on the same input pad. This patch tracks per-stream state using enabled_streams bitmask and enabled_count[] array.

Quoted 14 lines
drivers/media/platform/nxp/imx8-isi/imx8-isi-crossbar.c +2 -0
index 328d08a278eaf..64576f6bd45c2 100644
@@ -337,6 +337,8 @@ static int mxc_isi_crossbar_enable_streams(struct v4l2_subdev *sd,
 	struct mxc_isi_crossbar *xbar = to_isi_crossbar(sd);
 	struct v4l2_subdev *remote_sd;
 	struct mxc_isi_input *input;
+	u64 streams_to_enable;
+	unsigned long stream;
 	u64 sink_streams;
 	u32 sink_pad;
 	u32 remote_pad;

drivers/media/platform/nxp/imx8-isi/imx8-isi-crossbar.c static int mxc_isi_crossbar_enable_streams(struct v4l2_subdev *sd, · line 353

Quoted 2 lines
drivers/media/platform/nxp/imx8-isi/imx8-isi-crossbar.c +0 -0
@@ -350,30 +352,47 @@ static int mxc_isi_crossbar_enable_streams(struct v4l2_subdev *sd,
 
 	input = &xbar->inputs[sink_pad];

[Severity: High]
This is a pre-existing issue, but can this lead to a NULL pointer dereference if the remote link is disabled?

When userspace configures a V4L2 subdev route connecting a source pad to a sink pad on the crossbar, but leaves the incoming media link to that sink pad disabled, media_pad_remote_pad_first() will return NULL.

In drivers/media/platform/nxp/imx8-isi/imx8-isi-crossbar.c:

mxc_isi_crossbar_xlate_streams() {
    ...
	pad = media_pad_remote_pad_first(&xbar->pads[sink_pad]);
	sd = media_entity_to_v4l2_subdev(pad->entity);
	if (!sd) {
    ...
}

The media_entity_to_v4l2_subdev() macro immediately dereferences pad to access pad->entity before any NULL check is performed.

[Severity: Medium]
This is also a pre-existing issue, but might there be bitwise truncation and undefined behavior on 32-bit platforms when computing sink_streams?

When userspace configures a routing table using VIDIOC_SUBDEV_S_ROUTING with a sink_stream ID greater than or equal to 32, using BIT() will cause issues on 32-bit systems where BIT() operates on a 32-bit unsigned long.

In drivers/media/platform/nxp/imx8-isi/imx8-isi-crossbar.c:

mxc_isi_crossbar_xlate_streams() {
    ...
	for_each_active_route(&state->routing, route) {
		...
		sink_streams |= BIT(route->sink_stream);
		sink_pad = route->sink_pad;
	}
    ...
}

Could BIT_ULL() be used instead to safely construct the u64 sink_streams bitmask?

Quoted 4 lines
drivers/media/platform/nxp/imx8-isi/imx8-isi-crossbar.c +0 -4
-	/*
-	 * TODO: Track per-stream enable counts to support multiplexed
-	 * streams.
-	 */
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=3
#11 lore raw

Re: [PATCH v2 3/6] media: nxp: imx8-isi: Fix per-stream reference counting for multiplexed streams

In reply to Guoniu Zhou
Quoted 49 lines · Guoniu Zhou

The ISI crossbar fails to properly enable multiple streams from different virtual channels on the same input pad. Only the first stream gets enabled in hardware, subsequent streams are silently ignored.

The driver uses a single enable_count per input to track the input state. When enable_count is non-zero, the code assumes the input is already active and skips calling v4l2_subdev_enable_streams() for additional streams:

Call 1: enable_streams(stream 0)

    -> enable_count == 0, enable gasket and stream 0 in hardware
    -> enable_count = 1

Call 2: enable_streams(stream 1)

    -> enable_count == 1, skip hardware enable (BUG!)
    -> enable_count = 2
    -> stream 1 never gets enabled

Similarly on disable, when enable_count reaches zero, ALL streams are disabled regardless of which streams are actually still active.

Fix this by tracking per-stream state using:
- enabled_streams (u64 bitmask): tracks which streams are currently enabled
- enabled_count[] (array): per-stream reference counter to support the same stream being enabled/disabled multiple times

Now each stream is independently enabled/disabled in hardware based on the enabled_streams bitmask, while enabled_count[] provides reference counting for scenarios where the same stream is enabled multiple times, such as duplicate cases in the ISI stream.

Fixes cf21f328fcaf ("media: nxp: Add i.MX8 ISI driver")

---
Changes in v2:
- Use fixed-size array for enabled_count instead of dynamic allocation
- Use BIT_ULL() macro for u64 bitmask operations
- Use MXC_ISI_MAX_STREAMS (64) as loop boundary instead of num_sources
- Remove mxc_isi_stream_counters_alloc/free functions

---
 .../media/platform/nxp/imx8-isi/imx8-isi-core.h    |  7 +-
 .../platform/nxp/imx8-isi/imx8-isi-crossbar.c      | 76 ++++++++++++++++------
 2 files changed, 63 insertions(+), 20 deletions(-)
drivers/media/platform/nxp/imx8-isi/imx8-isi-core.h +0 -0
index 7547a6559d4c..9adbe2fe7cf8 100644

drivers/media/platform/nxp/imx8-isi/imx8-isi-core.h struct mxc_isi_dma_buffer { · line 188

Quoted 5 lines
drivers/media/platform/nxp/imx8-isi/imx8-isi-core.h +2 -0
@@ -184,8 +184,13 @@ struct mxc_isi_dma_buffer {
 	dma_addr_t			dma;
 };
 
+/* V4L2 subdev max stream ID is 63, need 64 counters (0-63) */
+#define MXC_ISI_MAX_STREAMS		64

Any existing Macro define max stream ID as 63? it'd better to use such macro

Quoted 8 lines
drivers/media/platform/nxp/imx8-isi/imx8-isi-core.h +4 -1
+
 struct mxc_isi_input {
-	unsigned int			enable_count;
+	u64				enabled_streams;
+	/* Per-stream reference counter */
+	unsigned int			enabled_count[MXC_ISI_MAX_STREAMS];
 };
 

...

drivers/media/platform/nxp/imx8-isi/imx8-isi-core.h static int mxc_isi_crossbar_disable_streams(struct v4l2_subdev *sd, · line 426

Quoted 12 lines
drivers/media/platform/nxp/imx8-isi/imx8-isi-core.h +7 -1
@@ -396,19 +417,36 @@ static int mxc_isi_crossbar_disable_streams(struct v4l2_subdev *sd,
 
 	input = &xbar->inputs[sink_pad];
 
-	input->enable_count--;
+	/*
+	 * Decrease the enable count for each stream. Only disable streams
+	 * whose count reaches zero.
+	 */
+	for (stream = 0; stream < MXC_ISI_MAX_STREAMS; stream++) {
+		if (!(sink_streams & BIT_ULL(stream)))
+			continue;

for_each_set_bit() ?

Frank

Quoted 39 lines
drivers/media/platform/nxp/imx8-isi/imx8-isi-core.h +19 -9
-	if (!input->enable_count) {
-		ret = v4l2_subdev_disable_streams(remote_sd, remote_pad,
-						  sink_streams);
-		if (ret)
-			dev_err(xbar->isi->dev,
-				"failed to disable streams 0x%llx on '%s':%u: %d\n",
-				sink_streams, remote_sd->name, remote_pad, ret);
+		if (!(input->enabled_streams & BIT_ULL(stream)))
+			continue;
 
-		mxc_isi_crossbar_gasket_disable(xbar, sink_pad);
+		if (--input->enabled_count[stream] == 0)
+			streams_to_disable |= BIT_ULL(stream);
 	}
 
+	if (!streams_to_disable)
+		return 0;
+
+	ret = v4l2_subdev_disable_streams(remote_sd, remote_pad,
+					  streams_to_disable);
+	if (ret)
+		dev_err(xbar->isi->dev,
+			"failed to disable streams 0x%llx on '%s':%u: %d\n",
+			streams_to_disable, remote_sd->name, remote_pad, ret);
+
+	input->enabled_streams &= ~streams_to_disable;
+
+	if (!input->enabled_streams)
+		mxc_isi_crossbar_gasket_disable(xbar, sink_pad);
+
 	return ret;
 }
 
 
--

2.34.1

maintainer
#12 lore raw

Re: [PATCH v2 3/6] media: nxp: imx8-isi: Fix per-stream reference counting for multiplexed streams

In reply to Guoniu Zhou
Quoted 25 lines · Guoniu Zhou

The ISI crossbar fails to properly enable multiple streams from different virtual channels on the same input pad. Only the first stream gets enabled in hardware, subsequent streams are silently ignored.

The driver uses a single enable_count per input to track the input state. When enable_count is non-zero, the code assumes the input is already active and skips calling v4l2_subdev_enable_streams() for additional streams:

Call 1: enable_streams(stream 0)

    -> enable_count == 0, enable gasket and stream 0 in hardware
    -> enable_count = 1

Call 2: enable_streams(stream 1)

    -> enable_count == 1, skip hardware enable (BUG!)
    -> enable_count = 2
    -> stream 1 never gets enabled

Similarly on disable, when enable_count reaches zero, ALL streams are disabled regardless of which streams are actually still active.

Fix this by tracking per-stream state using:
- enabled_streams (u64 bitmask): tracks which streams are currently enabled
- enabled_count[] (array): per-stream reference counter to support the same stream being enabled/disabled multiple times

Quoted 7 lines

Now each stream is independently enabled/disabled in hardware based on the enabled_streams bitmask, while enabled_count[] provides reference counting for scenarios where the same stream is enabled multiple times, such as duplicate cases in the ISI stream.

Fixes cf21f328fcaf ("media: nxp: Add i.MX8 ISI driver")

As far as I understand, only newer SoCs support multi-stream for ISI inputs (please tell me if that's incorrect). Those SoCs were not supported when the ISI driver was merged, so I don't think this should be backported to stable kernels. The subject line should then read "Implement per-stream ..." instead of "Fix per-stream ..." and the commit message adapted accordingly.

Quoted 18 lines

---
Changes in v2:
- Use fixed-size array for enabled_count instead of dynamic allocation
- Use BIT_ULL() macro for u64 bitmask operations
- Use MXC_ISI_MAX_STREAMS (64) as loop boundary instead of num_sources
- Remove mxc_isi_stream_counters_alloc/free functions

---
 .../media/platform/nxp/imx8-isi/imx8-isi-core.h    |  7 +-
 .../platform/nxp/imx8-isi/imx8-isi-crossbar.c      | 76 ++++++++++++++++------
 2 files changed, 63 insertions(+), 20 deletions(-)
drivers/media/platform/nxp/imx8-isi/imx8-isi-core.h +0 -0
index 7547a6559d4c..9adbe2fe7cf8 100644
@@ -184,8 +184,13 @@ struct mxc_isi_dma_buffer {
 	dma_addr_t			dma;

drivers/media/platform/nxp/imx8-isi/imx8-isi-core.h struct mxc_isi_dma_buffer { · line 193

Quoted 10 lines
drivers/media/platform/nxp/imx8-isi/imx8-isi-core.h +6 -1
 };
 
+/* V4L2 subdev max stream ID is 63, need 64 counters (0-63) */
+#define MXC_ISI_MAX_STREAMS		64
+
 struct mxc_isi_input {
-	unsigned int			enable_count;
+	u64				enabled_streams;
+	/* Per-stream reference counter */
+	unsigned int			enabled_count[MXC_ISI_MAX_STREAMS];

You can save some memory here by replacing unsigned int with u8. The counter can never go above the number of ISI outputs, as streams can be enabled once only.

drivers/media/platform/nxp/imx8-isi/imx8-isi-crossbar.c static int mxc_isi_crossbar_enable_streams(struct v4l2_subdev *sd, · line 341

Quoted 13 lines
drivers/media/platform/nxp/imx8-isi/imx8-isi-core.h +0 -0
 };
 
 struct mxc_isi_crossbar {
drivers/media/platform/nxp/imx8-isi/imx8-isi-crossbar.c +2 -0
index 328d08a278ea..64576f6bd45c 100644
@@ -337,6 +337,8 @@ static int mxc_isi_crossbar_enable_streams(struct v4l2_subdev *sd,
 	struct mxc_isi_crossbar *xbar = to_isi_crossbar(sd);
 	struct v4l2_subdev *remote_sd;
 	struct mxc_isi_input *input;
+	u64 streams_to_enable;
+	unsigned long stream;

unsigned int is enough, this is a counter, not a bitmask.

drivers/media/platform/nxp/imx8-isi/imx8-isi-crossbar.c static int mxc_isi_crossbar_enable_streams(struct v4l2_subdev *sd, · line 357

Quoted 12 lines
drivers/media/platform/nxp/imx8-isi/imx8-isi-crossbar.c +0 -5
 	u64 sink_streams;
 	u32 sink_pad;
 	u32 remote_pad;
@@ -350,30 +352,47 @@ static int mxc_isi_crossbar_enable_streams(struct v4l2_subdev *sd,
 
 	input = &xbar->inputs[sink_pad];
 
-	/*
-	 * TODO: Track per-stream enable counts to support multiplexed
-	 * streams.
-	 */
-	if (!input->enable_count) {

Let's keep a short comment here:

	/* Enable the gasket when the first stream is enabled for this input. */
Quoted 23 lines
drivers/media/platform/nxp/imx8-isi/imx8-isi-crossbar.c +10 -4
+	if (!input->enabled_streams) {
 		ret = mxc_isi_crossbar_gasket_enable(xbar, state, remote_sd,
 						     remote_pad, sink_pad);
 		if (ret)
 			return ret;
+	}
 
+	/*
+	 * Track per-stream enable counts to support multiplexed streams.
+	 * Only enable streams that are not already enabled.
+	 */
+	streams_to_enable = sink_streams & ~input->enabled_streams;
+
+	if (streams_to_enable) {
 		ret = v4l2_subdev_enable_streams(remote_sd, remote_pad,
-						 sink_streams);
+						 streams_to_enable);
 		if (ret) {
 			dev_err(xbar->isi->dev,
 				"failed to enable streams 0x%llx on '%s':%u: %d\n",
-				sink_streams, remote_sd->name, remote_pad, ret);
-			mxc_isi_crossbar_gasket_disable(xbar, sink_pad);
-			return ret;

drivers/media/platform/nxp/imx8-isi/imx8-isi-crossbar.c static int mxc_isi_crossbar_enable_streams(struct v4l2_subdev *sd, · line 382

Quoted 10 lines
drivers/media/platform/nxp/imx8-isi/imx8-isi-crossbar.c +6 -1
+				streams_to_enable, remote_sd->name, remote_pad, ret);
+			goto err_gasket_disable;
 		}
+
+		input->enabled_streams |= streams_to_enable;
 	}
 
-	input->enable_count++;
+	/* Increment reference count for all requested streams */
+	for (stream = 0; stream < MXC_ISI_MAX_STREAMS; stream++) {
	for (unsigned int stream = 0; stream < MXC_ISI_MAX_STREAMS; stream++) {

and drop the local variable above.

drivers/media/platform/nxp/imx8-isi/imx8-isi-crossbar.c static int mxc_isi_crossbar_enable_streams(struct v4l2_subdev *sd, · line 387

Quoted 5 lines
drivers/media/platform/nxp/imx8-isi/imx8-isi-crossbar.c +5 -0
+		if (!(sink_streams & BIT_ULL(stream)))
+			continue;
+
+		input->enabled_count[stream]++;
+	}

Each ISI pipe processes a single stream (the driver uses the V4L2_SUBDEV_ROUTING_NO_N_TO_1 validation flag, and patch 2/6 ensures that only stream 0 is accepted on the source side of the crossbar). This means that that, after translating streams_mask to the sink side, sink_streams should have a single bit set, and streams_to_enable will have either 0 or 1 bit set.

If that explanation is correct (please let me know if you disagree), then this loop can be replaced with

	input->enabled_count[__ffs64(sink_streams)]++;
Quoted 12 lines
drivers/media/platform/nxp/imx8-isi/imx8-isi-crossbar.c +6 -0
 	return 0;
+
+err_gasket_disable:
+	if (!input->enabled_streams)
+		mxc_isi_crossbar_gasket_disable(xbar, sink_pad);
+
+	return ret;
 }
 
 static int mxc_isi_crossbar_disable_streams(struct v4l2_subdev *sd,

drivers/media/platform/nxp/imx8-isi/imx8-isi-crossbar.c static int mxc_isi_crossbar_disable_streams(struct v4l2_subdev *sd, · line 406

Quoted 5 lines
drivers/media/platform/nxp/imx8-isi/imx8-isi-crossbar.c +2 -0
@@ -383,6 +402,8 @@ static int mxc_isi_crossbar_disable_streams(struct v4l2_subdev *sd,
 	struct mxc_isi_crossbar *xbar = to_isi_crossbar(sd);
 	struct v4l2_subdev *remote_sd;
 	struct mxc_isi_input *input;
+	u64 streams_to_disable = 0;
+	unsigned long stream;

unsigned int here too.

drivers/media/platform/nxp/imx8-isi/imx8-isi-crossbar.c static int mxc_isi_crossbar_disable_streams(struct v4l2_subdev *sd, · line 424

Quoted 13 lines
drivers/media/platform/nxp/imx8-isi/imx8-isi-crossbar.c +5 -1
 	u64 sink_streams;
 	u32 sink_pad;
 	u32 remote_pad;
@@ -396,19 +417,36 @@ static int mxc_isi_crossbar_disable_streams(struct v4l2_subdev *sd,
 
 	input = &xbar->inputs[sink_pad];
 
-	input->enable_count--;
+	/*
+	 * Decrease the enable count for each stream. Only disable streams
+	 * whose count reaches zero.
+	 */
+	for (stream = 0; stream < MXC_ISI_MAX_STREAMS; stream++) {
	for (unsigned int stream = 0; stream < MXC_ISI_MAX_STREAMS; stream++) {
Quoted 7 lines
drivers/media/platform/nxp/imx8-isi/imx8-isi-crossbar.c +2 -4
+		if (!(sink_streams & BIT_ULL(stream)))
+			continue;
 
-	if (!input->enable_count) {
-		ret = v4l2_subdev_disable_streams(remote_sd, remote_pad,
-						  sink_streams);
-		if (ret)

drivers/media/platform/nxp/imx8-isi/imx8-isi-crossbar.c static int mxc_isi_crossbar_disable_streams(struct v4l2_subdev *sd, · line 433

Quoted 10 lines
drivers/media/platform/nxp/imx8-isi/imx8-isi-crossbar.c +4 -4
-			dev_err(xbar->isi->dev,
-				"failed to disable streams 0x%llx on '%s':%u: %d\n",
-				sink_streams, remote_sd->name, remote_pad, ret);
+		if (!(input->enabled_streams & BIT_ULL(stream)))
+			continue;
 
-		mxc_isi_crossbar_gasket_disable(xbar, sink_pad);
+		if (--input->enabled_count[stream] == 0)
+			streams_to_disable |= BIT_ULL(stream);
 	}

Similarly here, the whole loop could be replaced by

	if (--input->enabled_count[__ffs64(stream)] == 0)
		streams_to_disable |= BIT_ULL(stream);

Another option is to store the information in the mxc_isi_pipe structure in the form of two new fields:

	unsigned int input;
	unsigned int input_stream;

The fields would be set to the input index and the sink stream mask in mxc_isi_crossbar_enable_streams():

	struct mxc_isi_pipe *pipe = &xbar->isi->pipes[pad - xbar->num_sources];

	...

	pipe->input = sink_pad;
	pipe->input_stream = sink_streams;

In this function you would then do (completely untested)

	struct mxc_isi_pipe *pipe = &xbar->isi->pipes[pad - xbar->num_sources];

	...

	pipe->input = 0;
	pipe->input_stream = 0;

	/*
	 * Check if any other pipe receives the same input stream. If so we
	 * can't disable it yet, so return immediately.
	 */
	for (i = 0; i < xbar->isi->pdata->num_channels; ++i) {
		struct mxc_isi_pipe *pipe = &xbar->isi->pipes[i];

		if (pipe->input == sink_pad &&
		    pipe->input_stream == sink_streams)
			return 0;
	}

	input->enabled_streams &= ~sink_streams;

The advantage of this is that we won't have to add a 64 entries array to each input. I'm not too concerned about the memory usage (even if reducing memory usage is always nice, more for the improvement in cache locality than for the memory saving itself), but the 64 entries feel a bit arbitrary.

What do you think is best ?

Quoted 18 lines
drivers/media/platform/nxp/imx8-isi/imx8-isi-crossbar.c +15 -0
+	if (!streams_to_disable)
+		return 0;
+
+	ret = v4l2_subdev_disable_streams(remote_sd, remote_pad,
+					  streams_to_disable);
+	if (ret)
+		dev_err(xbar->isi->dev,
+			"failed to disable streams 0x%llx on '%s':%u: %d\n",
+			streams_to_disable, remote_sd->name, remote_pad, ret);
+
+	input->enabled_streams &= ~streams_to_disable;
+
+	if (!input->enabled_streams)
+		mxc_isi_crossbar_gasket_disable(xbar, sink_pad);
+
 	return ret;
 }
 
Signature · 3 lines
Regards,

Laurent Pinchart
#13 lore raw

RE: [PATCH v2 3/6] media: nxp: imx8-isi: Fix per-stream reference counting for multiplexed streams

Hi Laurent,

Thanks for the review.

Quoted 171 lines

-----Original Message-----
From: Laurent Pinchart <[email protected]>
Sent: Tuesday, July 21, 2026 5:04 AM
To: G.N. Zhou (OSS) <[email protected]>
Cc: Mauro Carvalho Chehab <[email protected]>; Frank Li <[email protected]>; Sascha Hauer <[email protected]>; Pengutronix Kernel Team <[email protected]>; Fabio Estevam <[email protected]>; [email protected]; Stefan Riedmüller <[email protected]>; Jacopo Mondi <[email protected]>; Loic Poulain <[email protected]>; Bryan O'Donoghue <[email protected]>; Aisheng Dong <[email protected]>; G.N. Zhou <[email protected]>; [email protected]; [email protected]; linux-arm- [email protected]; [email protected]; [email protected]
Subject: Re: [PATCH v2 3/6] media: nxp: imx8-isi: Fix per-stream reference counting for multiplexed streams

Quoted 26 lines · Guoniu Zhou

The ISI crossbar fails to properly enable multiple streams from different virtual channels on the same input pad. Only the first stream gets enabled in hardware, subsequent streams are silently ignored.

The driver uses a single enable_count per input to track the input state. When enable_count is non-zero, the code assumes the input is already active and skips calling v4l2_subdev_enable_streams() for additional streams:

Call 1: enable_streams(stream 0)

    -> enable_count == 0, enable gasket and stream 0 in hardware
    -> enable_count = 1

Call 2: enable_streams(stream 1)

    -> enable_count == 1, skip hardware enable (BUG!)
    -> enable_count = 2
    -> stream 1 never gets enabled

Similarly on disable, when enable_count reaches zero, ALL streams are disabled regardless of which streams are actually still active.

Fix this by tracking per-stream state using:
- enabled_streams (u64 bitmask): tracks which streams are currently enabled
- enabled_count[] (array): per-stream reference counter to support the same stream being enabled/disabled multiple times

Quoted 7 lines

Now each stream is independently enabled/disabled in hardware based on the enabled_streams bitmask, while enabled_count[] provides reference counting for scenarios where the same stream is enabled multiple times, such as duplicate cases in the ISI stream.

Fixes cf21f328fcaf ("media: nxp: Add i.MX8 ISI driver")

As far as I understand, only newer SoCs support multi-stream for ISI inputs (please tell me if that's incorrect). Those SoCs were not supported when the ISI driver was merged, so I don't think this should be backported to stable kernels. The subject line should then read "Implement per-stream ..." instead of "Fix per-stream ..." and the commit message adapted accordingly.

Quoted 19 lines

---
Changes in v2:
- Use fixed-size array for enabled_count instead of dynamic allocation
- Use BIT_ULL() macro for u64 bitmask operations
- Use MXC_ISI_MAX_STREAMS (64) as loop boundary instead of num_sources
- Remove mxc_isi_stream_counters_alloc/free functions

---
 .../media/platform/nxp/imx8-isi/imx8-isi-core.h    |  7 +-
 .../platform/nxp/imx8-isi/imx8-isi-crossbar.c      | 76 ++++++++++++++++------
 2 files changed, 63 insertions(+), 20 deletions(-)
drivers/media/platform/nxp/imx8-isi/imx8-isi-core.h +0 -0
index 7547a6559d4c..9adbe2fe7cf8 100644
@@ -184,8 +184,13 @@ struct mxc_isi_dma_buffer {
 	dma_addr_t			dma;

drivers/media/platform/nxp/imx8-isi/imx8-isi-core.h struct mxc_isi_dma_buffer { · line 193

Quoted 10 lines
drivers/media/platform/nxp/imx8-isi/imx8-isi-core.h +6 -1
 };
 
+/* V4L2 subdev max stream ID is 63, need 64 counters (0-63) */
+#define MXC_ISI_MAX_STREAMS		64
+
 struct mxc_isi_input {
-	unsigned int			enable_count;
+	u64				enabled_streams;
+	/* Per-stream reference counter */
+	unsigned int
	enabled_count[MXC_ISI_MAX_STREAMS];

You can save some memory here by replacing unsigned int with u8. The counter can never go above the number of ISI outputs, as streams can be enabled once only.

drivers/media/platform/nxp/imx8-isi/imx8-isi-crossbar.c static int mxc_isi_crossbar_enable_streams(struct

Quoted 9 lines
drivers/media/platform/nxp/imx8-isi/imx8-isi-core.h +0 -0
 };
 
 struct mxc_isi_crossbar {
drivers/media/platform/nxp/imx8-isi/imx8-isi-crossbar.c +0 -0
index 328d08a278ea..64576f6bd45c 100644
@@ -337,6 +337,8 @@ static int mxc_isi_crossbar_enable_streams(struct

v4l2_subdev *sd,

drivers/media/platform/nxp/imx8-isi/imx8-isi-crossbar.c static int mxc_isi_crossbar_enable_streams(struct v4l2_subdev *sd, · line 341

Quoted 5 lines
drivers/media/platform/nxp/imx8-isi/imx8-isi-crossbar.c +2 -0
 	struct mxc_isi_crossbar *xbar = to_isi_crossbar(sd);
 	struct v4l2_subdev *remote_sd;
 	struct mxc_isi_input *input;
+	u64 streams_to_enable;
+	unsigned long stream;

unsigned int is enough, this is a counter, not a bitmask.

drivers/media/platform/nxp/imx8-isi/imx8-isi-crossbar.c static int · line 358

Quoted 13 lines
drivers/media/platform/nxp/imx8-isi/imx8-isi-crossbar.c +0 -5
 	u64 sink_streams;
 	u32 sink_pad;
 	u32 remote_pad;
@@ -350,30 +352,47 @@ static int
 mxc_isi_crossbar_enable_streams(struct v4l2_subdev *sd,
 
 	input = &xbar->inputs[sink_pad];
 
-	/*
-	 * TODO: Track per-stream enable counts to support multiplexed
-	 * streams.
-	 */
-	if (!input->enable_count) {

Let's keep a short comment here:

	/* Enable the gasket when the first stream is enabled for this input. */
Quoted 10 lines
drivers/media/platform/nxp/imx8-isi/imx8-isi-crossbar.c +5 -0
+	if (!input->enabled_streams) {
 		ret = mxc_isi_crossbar_gasket_enable(xbar, state, remote_sd,
 						     remote_pad, sink_pad);
 		if (ret)
 			return ret;
+	}
 
+	/*
+	 * Track per-stream enable counts to support multiplexed streams.
+	 * Only enable streams that are not already enabled.

drivers/media/platform/nxp/imx8-isi/imx8-isi-crossbar.c static int mxc_isi_crossbar_enable_streams(struct v4l2_subdev *sd,

Quoted 10 lines
drivers/media/platform/nxp/imx8-isi/imx8-isi-crossbar.c +5 -1
+	 */
+	streams_to_enable = sink_streams & ~input->enabled_streams;
+
+	if (streams_to_enable) {
 		ret = v4l2_subdev_enable_streams(remote_sd, remote_pad,
-						 sink_streams);
+						 streams_to_enable);
 		if (ret) {
 			dev_err(xbar->isi->dev,
 				"failed to enable streams 0x%llx on

'%s':%u: %d\n",

drivers/media/platform/nxp/imx8-isi/imx8-isi-crossbar.c static int · line 369

Quoted 1 line
drivers/media/platform/nxp/imx8-isi/imx8-isi-crossbar.c +0 -1
-				sink_streams, remote_sd->name, remote_pad,

ret);

drivers/media/platform/nxp/imx8-isi/imx8-isi-crossbar.c static int mxc_isi_crossbar_enable_streams(struct v4l2_subdev *sd,

Quoted 3 lines
drivers/media/platform/nxp/imx8-isi/imx8-isi-crossbar.c +1 -2
-			mxc_isi_crossbar_gasket_disable(xbar, sink_pad);
-			return ret;
+				streams_to_enable, remote_sd->name,

remote_pad, ret);

drivers/media/platform/nxp/imx8-isi/imx8-isi-crossbar.c static int mxc_isi_crossbar_enable_streams(struct v4l2_subdev *sd, · line 382

Quoted 9 lines
drivers/media/platform/nxp/imx8-isi/imx8-isi-crossbar.c +5 -1
+			goto err_gasket_disable;
 		}
+
+		input->enabled_streams |= streams_to_enable;
 	}
 
-	input->enable_count++;
+	/* Increment reference count for all requested streams */
+	for (stream = 0; stream < MXC_ISI_MAX_STREAMS; stream++) {
	for (unsigned int stream = 0; stream < MXC_ISI_MAX_STREAMS;

stream++) {

and drop the local variable above.

Quoted 11 lines

drivers/media/platform/nxp/imx8-isi/imx8-isi-crossbar.c static int mxc_isi_crossbar_enable_streams(struct v4l2_subdev *sd, · line 387

Quoted 5 lines
drivers/media/platform/nxp/imx8-isi/imx8-isi-crossbar.c +5 -0
+		if (!(sink_streams & BIT_ULL(stream)))
+			continue;
+
+		input->enabled_count[stream]++;
+	}

Each ISI pipe processes a single stream (the driver uses the V4L2_SUBDEV_ROUTING_NO_N_TO_1 validation flag, and patch 2/6 ensures that only stream 0 is accepted on the source side of the crossbar). This means that that, after translating streams_mask to the sink side, sink_streams should have a single bit set, and streams_to_enable will have either 0 or 1 bit set.

Yes, you are absolutely correct.

Quoted 112 lines

If that explanation is correct (please let me know if you disagree), then this loop can be replaced with

	input->enabled_count[__ffs64(sink_streams)]++;

drivers/media/platform/nxp/imx8-isi/imx8-isi-crossbar.c static int mxc_isi_crossbar_disable_streams(struct

Quoted 12 lines
drivers/media/platform/nxp/imx8-isi/imx8-isi-crossbar.c +6 -0
 	return 0;
+
+err_gasket_disable:
+	if (!input->enabled_streams)
+		mxc_isi_crossbar_gasket_disable(xbar, sink_pad);
+
+	return ret;
 }
 
 static int mxc_isi_crossbar_disable_streams(struct v4l2_subdev *sd,
@@ -383,6 +402,8 @@ static int mxc_isi_crossbar_disable_streams(struct

v4l2_subdev *sd,

drivers/media/platform/nxp/imx8-isi/imx8-isi-crossbar.c static int mxc_isi_crossbar_disable_streams(struct v4l2_subdev *sd, · line 406

Quoted 5 lines
drivers/media/platform/nxp/imx8-isi/imx8-isi-crossbar.c +2 -0
 	struct mxc_isi_crossbar *xbar = to_isi_crossbar(sd);
 	struct v4l2_subdev *remote_sd;
 	struct mxc_isi_input *input;
+	u64 streams_to_disable = 0;
+	unsigned long stream;

unsigned int here too.

Quoted 4 lines
drivers/media/platform/nxp/imx8-isi/imx8-isi-crossbar.c +0 -0
 	u64 sink_streams;
 	u32 sink_pad;
 	u32 remote_pad;

drivers/media/platform/nxp/imx8-isi/imx8-isi-crossbar.c static int · line 425

Quoted 10 lines
drivers/media/platform/nxp/imx8-isi/imx8-isi-crossbar.c +5 -1
@@ -396,19 +417,36 @@ static int
 mxc_isi_crossbar_disable_streams(struct v4l2_subdev *sd,
 
 	input = &xbar->inputs[sink_pad];
 
-	input->enable_count--;
+	/*
+	 * Decrease the enable count for each stream. Only disable streams
+	 * whose count reaches zero.
+	 */
+	for (stream = 0; stream < MXC_ISI_MAX_STREAMS; stream++) {
	for (unsigned int stream = 0; stream < MXC_ISI_MAX_STREAMS;

stream++) {

drivers/media/platform/nxp/imx8-isi/imx8-isi-crossbar.c static int mxc_isi_crossbar_disable_streams(struct v4l2_subdev *sd,

Quoted 9 lines
drivers/media/platform/nxp/imx8-isi/imx8-isi-crossbar.c +2 -6
+		if (!(sink_streams & BIT_ULL(stream)))
+			continue;
 
-	if (!input->enable_count) {
-		ret = v4l2_subdev_disable_streams(remote_sd, remote_pad,
-						  sink_streams);
-		if (ret)
-			dev_err(xbar->isi->dev,
-				"failed to disable streams 0x%llx on

'%s':%u: %d\n",

drivers/media/platform/nxp/imx8-isi/imx8-isi-crossbar.c static int · line 408

Quoted 1 line
drivers/media/platform/nxp/imx8-isi/imx8-isi-crossbar.c +0 -1
-				sink_streams, remote_sd->name, remote_pad,

ret);

drivers/media/platform/nxp/imx8-isi/imx8-isi-crossbar.c static int mxc_isi_crossbar_disable_streams(struct v4l2_subdev *sd, · line 433

Quoted 7 lines
drivers/media/platform/nxp/imx8-isi/imx8-isi-crossbar.c +4 -1
+		if (!(input->enabled_streams & BIT_ULL(stream)))
+			continue;
 
-		mxc_isi_crossbar_gasket_disable(xbar, sink_pad);
+		if (--input->enabled_count[stream] == 0)
+			streams_to_disable |= BIT_ULL(stream);
 	}

Similarly here, the whole loop could be replaced by

	if (--input->enabled_count[__ffs64(stream)] == 0)
		streams_to_disable |= BIT_ULL(stream);

Another option is to store the information in the mxc_isi_pipe structure in the form of two new fields:

	unsigned int input;
	unsigned int input_stream;

The fields would be set to the input index and the sink stream mask in mxc_isi_crossbar_enable_streams():

	struct mxc_isi_pipe *pipe = &xbar->isi->pipes[pad - xbar-

drivers/media/platform/nxp/imx8-isi/imx8-isi-crossbar.c static int · line 435

Quoted 1 line
drivers/media/platform/nxp/imx8-isi/imx8-isi-crossbar.c +0 -0
 num_sources];
	...

	pipe->input = sink_pad;
	pipe->input_stream = sink_streams;

In this function you would then do (completely untested)

	struct mxc_isi_pipe *pipe = &xbar->isi->pipes[pad - xbar-

drivers/media/platform/nxp/imx8-isi/imx8-isi-crossbar.c static int · line 436

Quoted 1 line
drivers/media/platform/nxp/imx8-isi/imx8-isi-crossbar.c +0 -0
 num_sources];
	...

	pipe->input = 0;
	pipe->input_stream = 0;

	/*
	 * Check if any other pipe receives the same input stream. If so we
	 * can't disable it yet, so return immediately.
	 */
	for (i = 0; i < xbar->isi->pdata->num_channels; ++i) {
		struct mxc_isi_pipe *pipe = &xbar->isi->pipes[i];

		if (pipe->input == sink_pad &&
		    pipe->input_stream == sink_streams)
			return 0;
	}

	input->enabled_streams &= ~sink_streams;
Quoted 6 lines

The advantage of this is that we won't have to add a 64 entries array to each input. I'm not too concerned about the memory usage (even if reducing memory usage is always nice, more for the improvement in cache locality than for the memory saving itself), but the 64 entries feel a bit arbitrary.

What do you think is best ?

Thank you for the detailed review and suggestions!

I agree that the second approach is better. Storing the input and input_stream directly in the mxc_isi_pipe structure is cleaner and avoids the arbitrary 64 entries array. The improved cache locality is also a nice benefit.

I will adopt this approach in the next version.

Thanks again for the suggestion!

Quoted 25 lines

drivers/media/platform/nxp/imx8-isi/imx8-isi-crossbar.c static int mxc_isi_crossbar_disable_streams(struct v4l2_subdev *sd,

Quoted 10 lines
drivers/media/platform/nxp/imx8-isi/imx8-isi-crossbar.c +9 -0
+	if (!streams_to_disable)
+		return 0;
+
+	ret = v4l2_subdev_disable_streams(remote_sd, remote_pad,
+					  streams_to_disable);
+	if (ret)
+		dev_err(xbar->isi->dev,
+			"failed to disable streams 0x%llx on '%s':%u: %d\n",
+			streams_to_disable, remote_sd->name, remote_pad,

ret);

Quoted 9 lines
drivers/media/platform/nxp/imx8-isi/imx8-isi-crossbar.c +6 -0
+
+	input->enabled_streams &= ~streams_to_disable;
+
+	if (!input->enabled_streams)
+		mxc_isi_crossbar_gasket_disable(xbar, sink_pad);
+
 	return ret;

}

--
Regards,

Laurent Pinchart

#14 lore raw

PATCH v2 4/6 media: nxp: imx8-isi: Add 16-bit raw Bayer format support

In reply to Guoniu Zhou

From: Laurentiu Palcu <[email protected]>

Add support for 16-bit raw Bayer formats (SBGGR16, SGBRG16, SGRBG16, SRGGB16) to both the pipeline subdev and video capture interface.

These formats are commonly used by high-end image sensors that output 16-bit raw data, enabling the ISI to process and capture full dynamic range from such sensors.

---
Changes in v2:
- Add Reviewed-by tag from Frank Li

---
 .../media/platform/nxp/imx8-isi/imx8-isi-pipe.c    | 24 +++++++++++++++
 .../media/platform/nxp/imx8-isi/imx8-isi-video.c   | 36 ++++++++++++++++++++++
 2 files changed, 60 insertions(+)
drivers/media/platform/nxp/imx8-isi/imx8-isi-pipe.c +24 -0
index 2d0843c86534..e58925d71164 100644
@@ -179,6 +179,30 @@ static const struct mxc_isi_bus_format_info mxc_isi_bus_formats[] = {
 		.pads		= BIT(MXC_ISI_PIPE_PAD_SINK)
 				| BIT(MXC_ISI_PIPE_PAD_SOURCE),
 		.encoding	= MXC_ISI_ENC_RAW,
+	}, {
+		.mbus_code	= MEDIA_BUS_FMT_SBGGR16_1X16,
+		.output		= MEDIA_BUS_FMT_SBGGR16_1X16,
+		.pads		= BIT(MXC_ISI_PIPE_PAD_SINK)
+				| BIT(MXC_ISI_PIPE_PAD_SOURCE),
+		.encoding	= MXC_ISI_ENC_RAW,
+	}, {
+		.mbus_code	= MEDIA_BUS_FMT_SGBRG16_1X16,
+		.output		= MEDIA_BUS_FMT_SGBRG16_1X16,
+		.pads		= BIT(MXC_ISI_PIPE_PAD_SINK)
+				| BIT(MXC_ISI_PIPE_PAD_SOURCE),
+		.encoding	= MXC_ISI_ENC_RAW,
+	}, {
+		.mbus_code	= MEDIA_BUS_FMT_SGRBG16_1X16,
+		.output		= MEDIA_BUS_FMT_SGRBG16_1X16,
+		.pads		= BIT(MXC_ISI_PIPE_PAD_SINK)
+				| BIT(MXC_ISI_PIPE_PAD_SOURCE),
+		.encoding	= MXC_ISI_ENC_RAW,
+	}, {
+		.mbus_code	= MEDIA_BUS_FMT_SRGGB16_1X16,
+		.output		= MEDIA_BUS_FMT_SRGGB16_1X16,
+		.pads		= BIT(MXC_ISI_PIPE_PAD_SINK)
+				| BIT(MXC_ISI_PIPE_PAD_SOURCE),
+		.encoding	= MXC_ISI_ENC_RAW,
 	},
 	/* JPEG */
 	{
drivers/media/platform/nxp/imx8-isi/imx8-isi-video.c +36 -0
index fe4adfa3a1f0..5eb448f4c26f 100644
@@ -356,6 +356,42 @@ static const struct mxc_isi_format_info mxc_isi_formats[] = {
 		.color_planes	= 1,
 		.depth		= { 16 },
 		.encoding	= MXC_ISI_ENC_RAW,
+	}, {
+		.mbus_code	= MEDIA_BUS_FMT_SBGGR16_1X16,
+		.fourcc		= V4L2_PIX_FMT_SBGGR16,
+		.type		= MXC_ISI_VIDEO_CAP,
+		.isi_out_format	= CHNL_IMG_CTRL_FORMAT_RAW16,
+		.mem_planes	= 1,
+		.color_planes	= 1,
+		.depth		= { 16 },
+		.encoding	= MXC_ISI_ENC_RAW,
+	}, {
+		.mbus_code	= MEDIA_BUS_FMT_SGBRG16_1X16,
+		.fourcc		= V4L2_PIX_FMT_SGBRG16,
+		.type		= MXC_ISI_VIDEO_CAP,
+		.isi_out_format	= CHNL_IMG_CTRL_FORMAT_RAW16,
+		.mem_planes	= 1,
+		.color_planes	= 1,
+		.depth		= { 16 },
+		.encoding	= MXC_ISI_ENC_RAW,
+	}, {
+		.mbus_code	= MEDIA_BUS_FMT_SGRBG16_1X16,
+		.fourcc		= V4L2_PIX_FMT_SGRBG16,
+		.type		= MXC_ISI_VIDEO_CAP,
+		.isi_out_format	= CHNL_IMG_CTRL_FORMAT_RAW16,
+		.mem_planes	= 1,
+		.color_planes	= 1,
+		.depth		= { 16 },
+		.encoding	= MXC_ISI_ENC_RAW,
+	}, {
+		.mbus_code	= MEDIA_BUS_FMT_SRGGB16_1X16,
+		.fourcc		= V4L2_PIX_FMT_SRGGB16,
+		.type		= MXC_ISI_VIDEO_CAP,
+		.isi_out_format	= CHNL_IMG_CTRL_FORMAT_RAW16,
+		.mem_planes	= 1,
+		.color_planes	= 1,
+		.depth		= { 16 },
+		.encoding	= MXC_ISI_ENC_RAW,
 	},
 	/* JPEG */
 	{
Signature · 1 line
2.34.1
maintainer Reviewed-by on v2 4/6
#15 lore raw

Re: [PATCH v2 4/6] media: nxp: imx8-isi: Add 16-bit raw Bayer format support

Hello Guoniu, Laurentiu,

Thank you for the patch.

Quoted 8 lines · On Mon, Jul 20, 2026 at 11:34:06AM +0800, [email protected] wrote:

From: Laurentiu Palcu <[email protected]>

Add support for 16-bit raw Bayer formats (SBGGR16, SGBRG16, SGRBG16, SRGGB16) to both the pipeline subdev and video capture interface.

These formats are commonly used by high-end image sensors that output 16-bit raw data, enabling the ISI to process and capture full dynamic range from such sensors.

It's not really high-end any more :-) Lots of sensors can output RAW16 today.

Quoted 16 lines

---
Changes in v2:
- Add Reviewed-by tag from Frank Li

---
 .../media/platform/nxp/imx8-isi/imx8-isi-pipe.c    | 24 +++++++++++++++
 .../media/platform/nxp/imx8-isi/imx8-isi-video.c   | 36 ++++++++++++++++++++++
 2 files changed, 60 insertions(+)
drivers/media/platform/nxp/imx8-isi/imx8-isi-pipe.c +0 -0
index 2d0843c86534..e58925d71164 100644

drivers/media/platform/nxp/imx8-isi/imx8-isi-pipe.c static const struct mxc_isi_bus_format_info mxc_isi_bus_formats[] = { · line 181

Quoted 3 lines
drivers/media/platform/nxp/imx8-isi/imx8-isi-pipe.c +0 -0
@@ -179,6 +179,30 @@ static const struct mxc_isi_bus_format_info mxc_isi_bus_formats[] = {
 		.pads		= BIT(MXC_ISI_PIPE_PAD_SINK)
 				| BIT(MXC_ISI_PIPE_PAD_SOURCE),
 		.encoding	= MXC_ISI_ENC_RAW,

While at it, could you please also add support for MEDIA_BUS_FMT_Y16_1X16, for 16-bit monochrome sensors (here and in mxc_isi_formats) ? With that,

Quoted 74 lines
drivers/media/platform/nxp/imx8-isi/imx8-isi-pipe.c +24 -0
+	}, {
+		.mbus_code	= MEDIA_BUS_FMT_SBGGR16_1X16,
+		.output		= MEDIA_BUS_FMT_SBGGR16_1X16,
+		.pads		= BIT(MXC_ISI_PIPE_PAD_SINK)
+				| BIT(MXC_ISI_PIPE_PAD_SOURCE),
+		.encoding	= MXC_ISI_ENC_RAW,
+	}, {
+		.mbus_code	= MEDIA_BUS_FMT_SGBRG16_1X16,
+		.output		= MEDIA_BUS_FMT_SGBRG16_1X16,
+		.pads		= BIT(MXC_ISI_PIPE_PAD_SINK)
+				| BIT(MXC_ISI_PIPE_PAD_SOURCE),
+		.encoding	= MXC_ISI_ENC_RAW,
+	}, {
+		.mbus_code	= MEDIA_BUS_FMT_SGRBG16_1X16,
+		.output		= MEDIA_BUS_FMT_SGRBG16_1X16,
+		.pads		= BIT(MXC_ISI_PIPE_PAD_SINK)
+				| BIT(MXC_ISI_PIPE_PAD_SOURCE),
+		.encoding	= MXC_ISI_ENC_RAW,
+	}, {
+		.mbus_code	= MEDIA_BUS_FMT_SRGGB16_1X16,
+		.output		= MEDIA_BUS_FMT_SRGGB16_1X16,
+		.pads		= BIT(MXC_ISI_PIPE_PAD_SINK)
+				| BIT(MXC_ISI_PIPE_PAD_SOURCE),
+		.encoding	= MXC_ISI_ENC_RAW,
 	},
 	/* JPEG */
 	{
drivers/media/platform/nxp/imx8-isi/imx8-isi-video.c +36 -0
index fe4adfa3a1f0..5eb448f4c26f 100644
@@ -356,6 +356,42 @@ static const struct mxc_isi_format_info mxc_isi_formats[] = {
 		.color_planes	= 1,
 		.depth		= { 16 },
 		.encoding	= MXC_ISI_ENC_RAW,
+	}, {
+		.mbus_code	= MEDIA_BUS_FMT_SBGGR16_1X16,
+		.fourcc		= V4L2_PIX_FMT_SBGGR16,
+		.type		= MXC_ISI_VIDEO_CAP,
+		.isi_out_format	= CHNL_IMG_CTRL_FORMAT_RAW16,
+		.mem_planes	= 1,
+		.color_planes	= 1,
+		.depth		= { 16 },
+		.encoding	= MXC_ISI_ENC_RAW,
+	}, {
+		.mbus_code	= MEDIA_BUS_FMT_SGBRG16_1X16,
+		.fourcc		= V4L2_PIX_FMT_SGBRG16,
+		.type		= MXC_ISI_VIDEO_CAP,
+		.isi_out_format	= CHNL_IMG_CTRL_FORMAT_RAW16,
+		.mem_planes	= 1,
+		.color_planes	= 1,
+		.depth		= { 16 },
+		.encoding	= MXC_ISI_ENC_RAW,
+	}, {
+		.mbus_code	= MEDIA_BUS_FMT_SGRBG16_1X16,
+		.fourcc		= V4L2_PIX_FMT_SGRBG16,
+		.type		= MXC_ISI_VIDEO_CAP,
+		.isi_out_format	= CHNL_IMG_CTRL_FORMAT_RAW16,
+		.mem_planes	= 1,
+		.color_planes	= 1,
+		.depth		= { 16 },
+		.encoding	= MXC_ISI_ENC_RAW,
+	}, {
+		.mbus_code	= MEDIA_BUS_FMT_SRGGB16_1X16,
+		.fourcc		= V4L2_PIX_FMT_SRGGB16,
+		.type		= MXC_ISI_VIDEO_CAP,
+		.isi_out_format	= CHNL_IMG_CTRL_FORMAT_RAW16,
+		.mem_planes	= 1,
+		.color_planes	= 1,
+		.depth		= { 16 },
+		.encoding	= MXC_ISI_ENC_RAW,
 	},
 	/* JPEG */
 	{
Signature · 3 lines
Regards,

Laurent Pinchart
#16 lore raw

PATCH v2 5/6 media: nxp: imx8-isi: Correct color map between V4L2 and ISI

In reply to Guoniu Zhou

Fix the ISI input format for the color map V4L2_PIX_FMT_XBGR32 in memory-to-memory mode.

Fixes cf21f328fcaf ("media: nxp: Add i.MX8 ISI driver")

---
Changes in v2:
- Reword commit description for clarity (Frank Li)

---
 drivers/media/platform/nxp/imx8-isi/imx8-isi-video.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/media/platform/nxp/imx8-isi/imx8-isi-video.c +1 -1
index 5eb448f4c26f..05b51b98344b 100644
@@ -151,7 +151,7 @@ static const struct mxc_isi_format_info mxc_isi_formats[] = {
 		.fourcc		= V4L2_PIX_FMT_XBGR32,
 		.type		= MXC_ISI_VIDEO_CAP | MXC_ISI_VIDEO_M2M_OUT
 				| MXC_ISI_VIDEO_M2M_CAP,
-		.isi_in_format	= CHNL_MEM_RD_CTRL_IMG_TYPE_XBGR8,
+		.isi_in_format	= CHNL_MEM_RD_CTRL_IMG_TYPE_XRGB8,
 		.isi_out_format	= CHNL_IMG_CTRL_FORMAT_XRGB888,
 		.mem_planes	= 1,
 		.color_planes	= 1,
Signature · 1 line
2.34.1
Reviewed-by on v2 5/6
#17 lore raw

Re: [PATCH v2 5/6] media: nxp: imx8-isi: Correct color map between V4L2 and ISI

Quoted 7 lines · Guoniu Zhou

Fix the ISI input format for the color map V4L2_PIX_FMT_XBGR32 in memory-to-memory mode.

Fixes cf21f328fcaf ("media: nxp: Add i.MX8 ISI driver")

---

Quoted 24 lines

Changes in v2:
- Reword commit description for clarity (Frank Li)

---
 drivers/media/platform/nxp/imx8-isi/imx8-isi-video.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/media/platform/nxp/imx8-isi/imx8-isi-video.c +1 -2
index 5eb448f4c26f..05b51b98344b 100644
@@ -151,7 +151,7 @@ static const struct mxc_isi_format_info mxc_isi_formats[] = {
 		.fourcc		= V4L2_PIX_FMT_XBGR32,
 		.type		= MXC_ISI_VIDEO_CAP | MXC_ISI_VIDEO_M2M_OUT
 				| MXC_ISI_VIDEO_M2M_CAP,
-		.isi_in_format	= CHNL_MEM_RD_CTRL_IMG_TYPE_XBGR8,
+		.isi_in_format	= CHNL_MEM_RD_CTRL_IMG_TYPE_XRGB8,
 		.isi_out_format	= CHNL_IMG_CTRL_FORMAT_XRGB888,
 		.mem_planes	= 1,
 		.color_planes	= 1,
 
--

2.34.1

maintainer Reviewed-by on v2 5/6
#18 lore raw

Re: [PATCH v2 5/6] media: nxp: imx8-isi: Correct color map between V4L2 and ISI

In reply to Guoniu Zhou

Hi Guoniu,

Quoted 6 lines · Guoniu Zhou

Fix the ISI input format for the color map V4L2_PIX_FMT_XBGR32 in memory-to-memory mode.

Fixes cf21f328fcaf ("media: nxp: Add i.MX8 ISI driver")

Tentatively,

I'm running a build to test this change.

Quoted 20 lines

---
Changes in v2:
- Reword commit description for clarity (Frank Li)

---
 drivers/media/platform/nxp/imx8-isi/imx8-isi-video.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/media/platform/nxp/imx8-isi/imx8-isi-video.c +1 -1
index 5eb448f4c26f..05b51b98344b 100644
@@ -151,7 +151,7 @@ static const struct mxc_isi_format_info mxc_isi_formats[] = {
 		.fourcc		= V4L2_PIX_FMT_XBGR32,
 		.type		= MXC_ISI_VIDEO_CAP | MXC_ISI_VIDEO_M2M_OUT
 				| MXC_ISI_VIDEO_M2M_CAP,
-		.isi_in_format	= CHNL_MEM_RD_CTRL_IMG_TYPE_XBGR8,
+		.isi_in_format	= CHNL_MEM_RD_CTRL_IMG_TYPE_XRGB8,
 		.isi_out_format	= CHNL_IMG_CTRL_FORMAT_XRGB888,
 		.mem_planes	= 1,
 		.color_planes	= 1,
Signature · 3 lines
Regards,

Laurent Pinchart
maintainer Tested-by
#19 lore raw

Re: [PATCH v2 5/6] media: nxp: imx8-isi: Correct color map between V4L2 and ISI

Quoted 8 lines · Laurent Pinchart
Quoted 6 lines · Guoniu Zhou

Fix the ISI input format for the color map V4L2_PIX_FMT_XBGR32 in memory-to-memory mode.

Fixes cf21f328fcaf ("media: nxp: Add i.MX8 ISI driver")
Quoted 5 lines

Tentatively,

I'm running a build to test this change.

Quoted 20 lines
Quoted 20 lines

---
Changes in v2:
- Reword commit description for clarity (Frank Li)

---
 drivers/media/platform/nxp/imx8-isi/imx8-isi-video.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/media/platform/nxp/imx8-isi/imx8-isi-video.c +1 -1
index 5eb448f4c26f..05b51b98344b 100644
@@ -151,7 +151,7 @@ static const struct mxc_isi_format_info mxc_isi_formats[] = {
 		.fourcc		= V4L2_PIX_FMT_XBGR32,
 		.type		= MXC_ISI_VIDEO_CAP | MXC_ISI_VIDEO_M2M_OUT
 				| MXC_ISI_VIDEO_M2M_CAP,
-		.isi_in_format	= CHNL_MEM_RD_CTRL_IMG_TYPE_XBGR8,
+		.isi_in_format	= CHNL_MEM_RD_CTRL_IMG_TYPE_XRGB8,
 		.isi_out_format	= CHNL_IMG_CTRL_FORMAT_XRGB888,
 		.mem_planes	= 1,
 		.color_planes	= 1,
Signature · 3 lines
Regards,

Laurent Pinchart
#20 lore raw

PATCH v2 6/6 media: nxp: imx8-isi: Add additional 32-bit RGB format support

In reply to Guoniu Zhou

Add support for additional 32-bit RGB pixel formats (BGRA32, RGBA32, BGRX32, RGBX32, ARGB2101010) and extend existing ABGR32 format with full memory-to-memory capabilities to meet Android requirements.

All formats support capture, M2M input, and M2M output operations, enabling complete format conversion pipelines.

---
Changes in v2:
- Add Reviewed-by tag from Frank Li

---
 .../media/platform/nxp/imx8-isi/imx8-isi-video.c   | 59 +++++++++++++++++++++-
 1 file changed, 58 insertions(+), 1 deletion(-)
drivers/media/platform/nxp/imx8-isi/imx8-isi-video.c +58 -1
index 05b51b98344b..ef638af350fe 100644
@@ -160,12 +160,69 @@ static const struct mxc_isi_format_info mxc_isi_formats[] = {
 	}, {
 		.mbus_code	= MEDIA_BUS_FMT_RGB888_1X24,
 		.fourcc		= V4L2_PIX_FMT_ABGR32,
-		.type		= MXC_ISI_VIDEO_CAP | MXC_ISI_VIDEO_M2M_CAP,
+		.type		= MXC_ISI_VIDEO_CAP | MXC_ISI_VIDEO_M2M_OUT
+				| MXC_ISI_VIDEO_M2M_CAP,
+		.isi_in_format	= CHNL_MEM_RD_CTRL_IMG_TYPE_XRGB8,
 		.isi_out_format	= CHNL_IMG_CTRL_FORMAT_ARGB8888,
 		.mem_planes	= 1,
 		.color_planes	= 1,
 		.depth		= { 32 },
 		.encoding	= MXC_ISI_ENC_RGB,
+	}, {
+		.mbus_code	= MEDIA_BUS_FMT_RGB888_1X24,
+		.fourcc		= V4L2_PIX_FMT_BGRA32,
+		.type		= MXC_ISI_VIDEO_CAP | MXC_ISI_VIDEO_M2M_OUT
+				| MXC_ISI_VIDEO_M2M_CAP,
+		.isi_in_format	= CHNL_MEM_RD_CTRL_IMG_TYPE_RGBX8,
+		.isi_out_format	= CHNL_IMG_CTRL_FORMAT_RGBA8888,
+		.mem_planes	= 1,
+		.color_planes	= 1,
+		.depth		= { 32 },
+		.encoding	= MXC_ISI_ENC_RGB,
+	}, {
+		.mbus_code	= MEDIA_BUS_FMT_RGB888_1X24,
+		.fourcc		= V4L2_PIX_FMT_RGBA32,
+		.type		= MXC_ISI_VIDEO_CAP | MXC_ISI_VIDEO_M2M_OUT
+				| MXC_ISI_VIDEO_M2M_CAP,
+		.isi_in_format	= CHNL_MEM_RD_CTRL_IMG_TYPE_XBGR8,
+		.isi_out_format	= CHNL_IMG_CTRL_FORMAT_ABGR8888,
+		.mem_planes	= 1,
+		.color_planes	= 1,
+		.depth		= { 32 },
+		.encoding	= MXC_ISI_ENC_RGB,
+	}, {
+		.mbus_code	= MEDIA_BUS_FMT_RGB888_1X24,
+		.fourcc		= V4L2_PIX_FMT_BGRX32,
+		.type		= MXC_ISI_VIDEO_CAP | MXC_ISI_VIDEO_M2M_OUT
+				| MXC_ISI_VIDEO_M2M_CAP,
+		.isi_in_format	= CHNL_MEM_RD_CTRL_IMG_TYPE_RGBX8,
+		.isi_out_format	= CHNL_IMG_CTRL_FORMAT_RGBX888,
+		.mem_planes	= 1,
+		.color_planes	= 1,
+		.depth		= { 32 },
+		.encoding	= MXC_ISI_ENC_RGB,
+	}, {
+		.mbus_code	= MEDIA_BUS_FMT_RGB888_1X24,
+		.fourcc		= V4L2_PIX_FMT_RGBX32,
+		.type		= MXC_ISI_VIDEO_CAP | MXC_ISI_VIDEO_M2M_OUT
+				| MXC_ISI_VIDEO_M2M_CAP,
+		.isi_in_format	= CHNL_MEM_RD_CTRL_IMG_TYPE_XBGR8,
+		.isi_out_format	= CHNL_IMG_CTRL_FORMAT_XBGR888,
+		.mem_planes	= 1,
+		.color_planes	= 1,
+		.depth		= { 32 },
+		.encoding	= MXC_ISI_ENC_RGB,
+	}, {
+		.mbus_code	= MEDIA_BUS_FMT_RGB888_1X24,
+		.fourcc		= V4L2_PIX_FMT_ARGB2101010,
+		.type		= MXC_ISI_VIDEO_CAP | MXC_ISI_VIDEO_M2M_OUT
+				| MXC_ISI_VIDEO_M2M_CAP,
+		.isi_in_format	= CHNL_MEM_RD_CTRL_IMG_TYPE_A2RGB10,
+		.isi_out_format	= CHNL_IMG_CTRL_FORMAT_A2RGB10,
+		.mem_planes	= 1,
+		.color_planes	= 1,
+		.depth		= { 32 },
+		.encoding	= MXC_ISI_ENC_RGB,
 	},
 	/*
 	 * RAW formats
Signature · 1 line
2.34.1
maintainer Tested-by on v2 6/6
#21 lore raw

Re: [PATCH v2 6/6] media: nxp: imx8-isi: Add additional 32-bit RGB format support

Hi Guoniu, Robert,

Thank you for the patch.

Quoted 3 lines · Guoniu Zhou

Add support for additional 32-bit RGB pixel formats (BGRA32, RGBA32, BGRX32, RGBX32, ARGB2101010) and extend existing ABGR32 format with full memory-to-memory capabilities to meet Android requirements.

What are those Android requirements ?

Quoted 18 lines

All formats support capture, M2M input, and M2M output operations, enabling complete format conversion pipelines.

---
Changes in v2:
- Add Reviewed-by tag from Frank Li

---
 .../media/platform/nxp/imx8-isi/imx8-isi-video.c   | 59 +++++++++++++++++++++-
 1 file changed, 58 insertions(+), 1 deletion(-)
drivers/media/platform/nxp/imx8-isi/imx8-isi-video.c +0 -0
index 05b51b98344b..ef638af350fe 100644

drivers/media/platform/nxp/imx8-isi/imx8-isi-video.c static const struct mxc_isi_format_info mxc_isi_formats[] = { · line 165

Quoted 7 lines
drivers/media/platform/nxp/imx8-isi/imx8-isi-video.c +3 -1
@@ -160,12 +160,69 @@ static const struct mxc_isi_format_info mxc_isi_formats[] = {
 	}, {
 		.mbus_code	= MEDIA_BUS_FMT_RGB888_1X24,
 		.fourcc		= V4L2_PIX_FMT_ABGR32,
-		.type		= MXC_ISI_VIDEO_CAP | MXC_ISI_VIDEO_M2M_CAP,
+		.type		= MXC_ISI_VIDEO_CAP | MXC_ISI_VIDEO_M2M_OUT
+				| MXC_ISI_VIDEO_M2M_CAP,
+		.isi_in_format	= CHNL_MEM_RD_CTRL_IMG_TYPE_XRGB8,

This doesn't seem right. The value is the same as for V4L2_PIX_FMT_XBGR32. As far as I understand, the ISI will ignore the alpha bits (MSBs in the 32-bit data).

This is why the V4L2_PIX_FMT_ABGR32 format doesn't set
MXC_ISI_VIDEO_M2M_OUT: to be supported as an output format (input to the ISI), we would need a hardware mode where the alpha bits are read from memory and used by the ISI. I recommend dropping this part of the patch.

drivers/media/platform/nxp/imx8-isi/imx8-isi-video.c static const struct mxc_isi_format_info mxc_isi_formats[] = { · line 176

Quoted 11 lines
drivers/media/platform/nxp/imx8-isi/imx8-isi-video.c +6 -0
 		.isi_out_format	= CHNL_IMG_CTRL_FORMAT_ARGB8888,
 		.mem_planes	= 1,
 		.color_planes	= 1,
 		.depth		= { 32 },
 		.encoding	= MXC_ISI_ENC_RGB,
+	}, {
+		.mbus_code	= MEDIA_BUS_FMT_RGB888_1X24,
+		.fourcc		= V4L2_PIX_FMT_BGRA32,
+		.type		= MXC_ISI_VIDEO_CAP | MXC_ISI_VIDEO_M2M_OUT
+				| MXC_ISI_VIDEO_M2M_CAP,
+		.isi_in_format	= CHNL_MEM_RD_CTRL_IMG_TYPE_RGBX8,

Same comment here, this format should not have MXC_ISI_VIDEO_M2M_OUT set.

drivers/media/platform/nxp/imx8-isi/imx8-isi-video.c static const struct mxc_isi_format_info mxc_isi_formats[] = { · line 187

Quoted 11 lines
drivers/media/platform/nxp/imx8-isi/imx8-isi-video.c +11 -0
+		.isi_out_format	= CHNL_IMG_CTRL_FORMAT_RGBA8888,
+		.mem_planes	= 1,
+		.color_planes	= 1,
+		.depth		= { 32 },
+		.encoding	= MXC_ISI_ENC_RGB,
+	}, {
+		.mbus_code	= MEDIA_BUS_FMT_RGB888_1X24,
+		.fourcc		= V4L2_PIX_FMT_RGBA32,
+		.type		= MXC_ISI_VIDEO_CAP | MXC_ISI_VIDEO_M2M_OUT
+				| MXC_ISI_VIDEO_M2M_CAP,
+		.isi_in_format	= CHNL_MEM_RD_CTRL_IMG_TYPE_XBGR8,

Same here too.

Quoted 28 lines
drivers/media/platform/nxp/imx8-isi/imx8-isi-video.c +28 -0
+		.isi_out_format	= CHNL_IMG_CTRL_FORMAT_ABGR8888,
+		.mem_planes	= 1,
+		.color_planes	= 1,
+		.depth		= { 32 },
+		.encoding	= MXC_ISI_ENC_RGB,
+	}, {
+		.mbus_code	= MEDIA_BUS_FMT_RGB888_1X24,
+		.fourcc		= V4L2_PIX_FMT_BGRX32,
+		.type		= MXC_ISI_VIDEO_CAP | MXC_ISI_VIDEO_M2M_OUT
+				| MXC_ISI_VIDEO_M2M_CAP,
+		.isi_in_format	= CHNL_MEM_RD_CTRL_IMG_TYPE_RGBX8,
+		.isi_out_format	= CHNL_IMG_CTRL_FORMAT_RGBX888,
+		.mem_planes	= 1,
+		.color_planes	= 1,
+		.depth		= { 32 },
+		.encoding	= MXC_ISI_ENC_RGB,
+	}, {
+		.mbus_code	= MEDIA_BUS_FMT_RGB888_1X24,
+		.fourcc		= V4L2_PIX_FMT_RGBX32,
+		.type		= MXC_ISI_VIDEO_CAP | MXC_ISI_VIDEO_M2M_OUT
+				| MXC_ISI_VIDEO_M2M_CAP,
+		.isi_in_format	= CHNL_MEM_RD_CTRL_IMG_TYPE_XBGR8,
+		.isi_out_format	= CHNL_IMG_CTRL_FORMAT_XBGR888,
+		.mem_planes	= 1,
+		.color_planes	= 1,
+		.depth		= { 32 },
+		.encoding	= MXC_ISI_ENC_RGB,
+	}, {

drivers/media/platform/nxp/imx8-isi/imx8-isi-video.c static const struct mxc_isi_format_info mxc_isi_formats[] = { · line 225

Quoted 10 lines
drivers/media/platform/nxp/imx8-isi/imx8-isi-video.c +10 -0
+		.mbus_code	= MEDIA_BUS_FMT_RGB888_1X24,
+		.fourcc		= V4L2_PIX_FMT_ARGB2101010,
+		.type		= MXC_ISI_VIDEO_CAP | MXC_ISI_VIDEO_M2M_OUT
+				| MXC_ISI_VIDEO_M2M_CAP,
+		.isi_in_format	= CHNL_MEM_RD_CTRL_IMG_TYPE_A2RGB10,
+		.isi_out_format	= CHNL_IMG_CTRL_FORMAT_A2RGB10,
+		.mem_planes	= 1,
+		.color_planes	= 1,
+		.depth		= { 32 },
+		.encoding	= MXC_ISI_ENC_RGB,

The rest looks good. I have tested all the new formats, both on the input and output side, and have not noticed any issue.

Quoted 3 lines
drivers/media/platform/nxp/imx8-isi/imx8-isi-video.c +0 -0
 	},
 	/*
 	 * RAW formats
Signature · 3 lines
Regards,

Laurent Pinchart
maintainer
#22 lore raw
In reply to Guoniu Zhou
Quoted 18 lines · Guoniu Zhou

This series addresses critical bugs in the imx8-isi driver and extends format support for high-end sensors and Android requirements.

Patch 1 fixes a critical stream ID validation bug in the crossbar routing where the validation loop iterates over the wrong routing table, allowing userspace to bypass validation entirely and configure invalid routes.

Patch 2 adds additional stream ID validation to enforce hardware constraints (SOURCE stream must be 0).

Patch 3 fixes a stream reference counting bug in the crossbar that prevents multiple streams from different virtual channels on the same input pad from being enabled correctly. Only the first stream gets enabled in hardware, subsequent streams are silently ignored.

Patch 4 adds support for 16-bit raw Bayer formats (SBGGR16, SGBRG16, SGRBG16, SRGGB16) commonly used by high-end image sensors.

Quoted 8 lines

Patch 5 fixes incorrect color mapping for XBGR32 format in memory-to-memory mode (marked for stable backport).

Patch 6 extends RGB format support by adding BGRA32, RGBA32, BGRX32, RGBX32, and ARGB2101010 formats with full M2M capabilities to meet Android requirements.

I have taken patches 1, 2 and 5 in my tree already ([1]). The CI pipeline is running ([2]). Once it completes successfully, I will send a pull request. You can base the next version of this series (patches 3, 4 and 6) on top of [1].

[1] https://gitlab.freedesktop.org/linux-media/users/pinchartl/-/commit/db30b841e534f6b97008583f6e63c93473f14afc
[2] https://gitlab.freedesktop.org/linux-media/users/pinchartl/-/pipelines/1710408

Quoted 36 lines

---
Changes in v2:
- [1/6] Split v1 patch 1/5 into two patches: this patch fixes the core for_each_active_route() bug, next patch adds additional stream validation (Frank Li)
- [2/6] New patch split from v1 1/5: adds stream ID validation on top of for_each_active_route() fix (Frank Li)
- [2/6] Remove incorrect sink_stream validation
- [3/6] Use fixed-size array for enabled_count instead of dynamic allocation
- [3/6] Use BIT_ULL() macro for u64 bitmask operations
- [3/6] Use MXC_ISI_MAX_STREAMS (64) as loop boundary instead of num_sources
- [3/6] Remove mxc_isi_stream_counters_alloc/free functions
- [4/6] Add Reviewed-by tag from Frank Li
- [5/6] Reword commit description for clarity (Frank Li)
- [6/6] Add Reviewed-by tag from Frank Li
- Link to v1: https://lore.kernel.org/r/[email protected]

---
Guoniu Zhou (5):
media: nxp: imx8-isi: Fix stream ID validation bypass in crossbar routing
media: nxp: imx8-isi: Add stream ID validation for crossbar routing
media: nxp: imx8-isi: Fix per-stream reference counting for multiplexed streams
media: nxp: imx8-isi: Correct color map between V4L2 and ISI
media: nxp: imx8-isi: Add additional 32-bit RGB format support

Laurentiu Palcu (1):
media: nxp: imx8-isi: Add 16-bit raw Bayer format support

 .../media/platform/nxp/imx8-isi/imx8-isi-core.h    |  7 +-
 .../platform/nxp/imx8-isi/imx8-isi-crossbar.c      | 92 +++++++++++++++-----
 .../media/platform/nxp/imx8-isi/imx8-isi-pipe.c    | 24 ++++++
 .../media/platform/nxp/imx8-isi/imx8-isi-video.c   | 97 +++++++++++++++++++++-
 4 files changed, 196 insertions(+), 24 deletions(-)

---
base-commit: 06cb687a5132fcffe624c0070576ab852ac6b568
change-id: 20260626-isi-00f05b044ac9

Signature · 3 lines
Regards,

Laurent Pinchart
maintainer
#23 lore raw
Quoted 28 lines · Laurent Pinchart
Quoted 18 lines · Guoniu Zhou

This series addresses critical bugs in the imx8-isi driver and extends format support for high-end sensors and Android requirements.

Patch 1 fixes a critical stream ID validation bug in the crossbar routing where the validation loop iterates over the wrong routing table, allowing userspace to bypass validation entirely and configure invalid routes.

Patch 2 adds additional stream ID validation to enforce hardware constraints (SOURCE stream must be 0).

Patch 3 fixes a stream reference counting bug in the crossbar that prevents multiple streams from different virtual channels on the same input pad from being enabled correctly. Only the first stream gets enabled in hardware, subsequent streams are silently ignored.

Patch 4 adds support for 16-bit raw Bayer formats (SBGGR16, SGBRG16, SGRBG16, SRGGB16) commonly used by high-end image sensors.

Quoted 8 lines

Patch 5 fixes incorrect color mapping for XBGR32 format in memory-to-memory mode (marked for stable backport).

Patch 6 extends RGB format support by adding BGRA32, RGBA32, BGRX32, RGBX32, and ARGB2101010 formats with full M2M capabilities to meet Android requirements.

Quoted 7 lines

I have taken patches 1, 2 and 5 in my tree already ([1]). The CI pipeline is running ([2]). Once it completes successfully, I will send a pull request. You can base the next version of this series (patches 3, 4 and 6) on top of [1].

[1] https://gitlab.freedesktop.org/linux-media/users/pinchartl/-/commit/db30b841e534f6b97008583f6e63c93473f14afc
[2] https://gitlab.freedesktop.org/linux-media/users/pinchartl/-/pipelines/1710408

This failed due to a mistake on my side. I've submitted a new pipeline:

https://gitlab.freedesktop.org/linux-media/users/pinchartl/-/pipelines/1710412

Quoted 36 lines
Quoted 36 lines

---
Changes in v2:
- [1/6] Split v1 patch 1/5 into two patches: this patch fixes the core for_each_active_route() bug, next patch adds additional stream validation (Frank Li)
- [2/6] New patch split from v1 1/5: adds stream ID validation on top of for_each_active_route() fix (Frank Li)
- [2/6] Remove incorrect sink_stream validation
- [3/6] Use fixed-size array for enabled_count instead of dynamic allocation
- [3/6] Use BIT_ULL() macro for u64 bitmask operations
- [3/6] Use MXC_ISI_MAX_STREAMS (64) as loop boundary instead of num_sources
- [3/6] Remove mxc_isi_stream_counters_alloc/free functions
- [4/6] Add Reviewed-by tag from Frank Li
- [5/6] Reword commit description for clarity (Frank Li)
- [6/6] Add Reviewed-by tag from Frank Li
- Link to v1: https://lore.kernel.org/r/[email protected]

---
Guoniu Zhou (5):
media: nxp: imx8-isi: Fix stream ID validation bypass in crossbar routing
media: nxp: imx8-isi: Add stream ID validation for crossbar routing
media: nxp: imx8-isi: Fix per-stream reference counting for multiplexed streams
media: nxp: imx8-isi: Correct color map between V4L2 and ISI
media: nxp: imx8-isi: Add additional 32-bit RGB format support

Laurentiu Palcu (1):
media: nxp: imx8-isi: Add 16-bit raw Bayer format support

 .../media/platform/nxp/imx8-isi/imx8-isi-core.h    |  7 +-
 .../platform/nxp/imx8-isi/imx8-isi-crossbar.c      | 92 +++++++++++++++-----
 .../media/platform/nxp/imx8-isi/imx8-isi-pipe.c    | 24 ++++++
 .../media/platform/nxp/imx8-isi/imx8-isi-video.c   | 97 +++++++++++++++++++++-
 4 files changed, 196 insertions(+), 24 deletions(-)

---
base-commit: 06cb687a5132fcffe624c0070576ab852ac6b568
change-id: 20260626-isi-00f05b044ac9

Signature · 3 lines
Regards,

Laurent Pinchart