Skip to contentSkip to conversation

media: intel/ipu6: Set the VC of the stream as the SOF event id

PATCH In review Hongju Wang 3 participants 7 messages Oct 24, 2024 – Nov 5, 2024

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 media: intel/ipu6: Set the VC of the stream as the SOF event id

In the virtual channel case, they should use v4l2_event.id to distinguish SOF events of different streams. Therefore, we set the virtual channel number of the stream as the SOF event ID. This number is unique.

---
 drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c | 1 +
 1 file changed, 1 insertion(+)
drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c +1 -0
index 051898ce53f4..5ad426afa0f0 100644
@@ -578,6 +578,7 @@ void ipu6_isys_csi2_sof_event_by_stream(struct ipu6_isys_stream *stream)
 		.type = V4L2_EVENT_FRAME_SYNC,
 	};
 
+	ev.id = stream->vc;
 	ev.u.frame_sync.frame_sequence = atomic_fetch_inc(&stream->sequence);
 	v4l2_event_queue(vdev, &ev);
 
Signature · 1 line
2.34.1
maintainer
#1 lore raw

Hi Hongju,

Quoted 14 lines · Hongju Wang

In the virtual channel case, they should use v4l2_event.id to distinguish SOF events of different streams. Therefore, we set the virtual channel number of the stream as the SOF event ID. This number is unique.

---
 drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c | 1 +
 1 file changed, 1 insertion(+)
drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c +0 -0
index 051898ce53f4..5ad426afa0f0 100644

drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c void ipu6_isys_csi2_sof_event_by_stream(struct ipu6_isys_stream *stream) · line 584

Quoted 7 lines
drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c +1 -0
@@ -578,6 +578,7 @@ void ipu6_isys_csi2_sof_event_by_stream(struct ipu6_isys_stream *stream)
 		.type = V4L2_EVENT_FRAME_SYNC,
 	};
 
+	ev.id = stream->vc;
 	ev.u.frame_sync.frame_sequence = atomic_fetch_inc(&stream->sequence);
 	v4l2_event_queue(vdev, &ev);
 

The id field in struct v4l2_event isn't used for the FRAME_SYNC event and also the virtual channel isn't communicated to the user space currently.

Doesn't the video device itself (and the routing configuration) associate this to a given sensor already?

Signature · 3 lines
Kind regards,

Sakari Ailus
#2 lore raw

Hi Sakari

Quoted 31 lines

-----Original Message-----
From: Sakari Ailus <[email protected]>
Sent: Thursday, October 24, 2024 6:31 PM
To: Wang, Hongju <[email protected]>
Cc: [email protected]; [email protected]
Subject: Re: [PATCH] media: intel/ipu6: Set the VC of the stream as the SOF event id

Hi Hongju,

Quoted 3 lines · Hongju Wang

In the virtual channel case, they should use v4l2_event.id to distinguish SOF events of different streams. Therefore, we set the virtual channel number of the stream as the SOF event ID. This number is

unique.

drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c void ipu6_isys_csi2_sof_event_by_stream(struct

Quoted 12 lines
---
 drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c | 1 +
 1 file changed, 1 insertion(+)
drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c +0 -0
index 051898ce53f4..5ad426afa0f0 100644
@@ -578,6 +578,7 @@ void ipu6_isys_csi2_sof_event_by_stream(struct

ipu6_isys_stream *stream)

Quoted 3 lines
drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c +0 -0
 		.type = V4L2_EVENT_FRAME_SYNC,
 	};
 
Quoted 7 lines

drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c void ipu6_isys_csi2_sof_event_by_stream(struct ipu6_isys_stream *stream) · line 584

Quoted 4 lines
drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c +1 -0
+	ev.id = stream->vc;
 	ev.u.frame_sync.frame_sequence = atomic_fetch_inc(&stream->sequence);
 	v4l2_event_queue(vdev, &ev);
 

The id field in struct v4l2_event isn't used for the FRAME_SYNC event and also the virtual channel isn't communicated to the user space currently.

How to use the id field? User space should use SOF to do FRAME_SYNC, but the v4l2_event doesn't distinguish different streams. Do you have any good suggestions?

Quoted 2 lines

Doesn't the video device itself (and the routing configuration) associate this to a given sensor already?

Yes, they are already connected. But we can't use the video device to do FRAME_SYNC, The CSI2 hardware interrupt has processing for FRAME_SYNC, we cannot pass it to the video device.

Quoted 5 lines

--
Kind regards,

Sakari Ailus

maintainer
#3 lore raw

Hi Hongju,

Quoted 41 lines · Hongju

Hi Sakari

Quoted 31 lines

-----Original Message-----
From: Sakari Ailus <[email protected]>
Sent: Thursday, October 24, 2024 6:31 PM
To: Wang, Hongju <[email protected]>
Cc: [email protected]; [email protected]
Subject: Re: [PATCH] media: intel/ipu6: Set the VC of the stream as the SOF event id

Hi Hongju,

Quoted 3 lines · Hongju Wang

In the virtual channel case, they should use v4l2_event.id to distinguish SOF events of different streams. Therefore, we set the virtual channel number of the stream as the SOF event ID. This number is

unique.

drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c void ipu6_isys_csi2_sof_event_by_stream(struct

Quoted 12 lines
---
 drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c | 1 +
 1 file changed, 1 insertion(+)
drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c +0 -0
index 051898ce53f4..5ad426afa0f0 100644
@@ -578,6 +578,7 @@ void ipu6_isys_csi2_sof_event_by_stream(struct

ipu6_isys_stream *stream)

Quoted 3 lines
drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c +0 -0
 		.type = V4L2_EVENT_FRAME_SYNC,
 	};
 
Quoted 7 lines

drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c void ipu6_isys_csi2_sof_event_by_stream(struct ipu6_isys_stream *stream) · line 584

Quoted 4 lines
drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c +1 -0
+	ev.id = stream->vc;
 	ev.u.frame_sync.frame_sequence = atomic_fetch_inc(&stream->sequence);
 	v4l2_event_queue(vdev, &ev);
 

The id field in struct v4l2_event isn't used for the FRAME_SYNC event and also the virtual channel isn't communicated to the user space currently.

Quoted 3 lines

How to use the id field? User space should use SOF to do FRAME_SYNC, but the v4l2_event doesn't distinguish different streams. Do you have any good suggestions?

Ah, now I think I do understand the problem. The events are produced by the CSI-2 receiver sub-device but there are multiple streams and the event isn't stream-aware?

It's nice that the id field of the FRAME_SYNC event isn't in use for any other purpose. This should be taken into account in event subscription, too.

I'll add a patch documenting this in the metadata series. Any thoughts?

Cc Laurent and Hans.

Signature · 3 lines
Kind regards,

Sakari Ailus
#4 lore raw
Quoted 37 lines · Sakari Ailus
Quoted 28 lines · Hongju
Quoted 19 lines · Sakari Ailus
Quoted 18 lines · Hongju Wang

In the virtual channel case, they should use v4l2_event.id to distinguish SOF events of different streams. Therefore, we set the virtual channel number of the stream as the SOF event ID. This number is unique.

---
 drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c | 1 +
 1 file changed, 1 insertion(+)
drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c +0 -0
index 051898ce53f4..5ad426afa0f0 100644
@@ -578,6 +578,7 @@ void ipu6_isys_csi2_sof_event_by_stream(struct ipu6_isys_stream *stream)
 		.type = V4L2_EVENT_FRAME_SYNC,
 	};
 
Quoted 7 lines

drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c void ipu6_isys_csi2_sof_event_by_stream(struct ipu6_isys_stream *stream) · line 584

Quoted 4 lines
drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c +1 -0
+	ev.id = stream->vc;
 	ev.u.frame_sync.frame_sequence = atomic_fetch_inc(&stream->sequence);
 	v4l2_event_queue(vdev, &ev);
 

The id field in struct v4l2_event isn't used for the FRAME_SYNC event and also the virtual channel isn't communicated to the user space currently.

Quoted 3 lines

How to use the id field? User space should use SOF to do FRAME_SYNC, but the v4l2_event doesn't distinguish different streams. Do you have any good suggestions?

Ah, now I think I do understand the problem. The events are produced by the CSI-2 receiver sub-device but there are multiple streams and the event isn't stream-aware?

Quoted 5 lines

It's nice that the id field of the FRAME_SYNC event isn't in use for any other purpose. This should be taken into account in event subscription, too.

I'll add a patch documenting this in the metadata series. Any thoughts?

I think we could use the id field for this purpose, but maybe not to report the VC. Userspace isn't VC-aware, we should instead convert the VC to a stream.

Quoted 1 line

Cc Laurent and Hans.

Signature · 3 lines
Regards,

Laurent Pinchart
#5 lore raw

Hi Laurent,

Quoted 57 lines

-----Original Message-----
From: Laurent Pinchart <[email protected]>
Sent: Monday, November 4, 2024 4:25 PM
To: Sakari Ailus <[email protected]>
Cc: Wang, Hongju <[email protected]>; [email protected]; [email protected]; [email protected]
Subject: Re: [PATCH] media: intel/ipu6: Set the VC of the stream as the SOF event id

Quoted 6 lines · Sakari Ailus
Quoted 5 lines · Hongju
Quoted 4 lines · Sakari Ailus
Quoted 3 lines · Hongju Wang

In the virtual channel case, they should use v4l2_event.id to distinguish SOF events of different streams. Therefore, we set the virtual channel number of the stream as the SOF event ID. This

number is unique.

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
---
 drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c | 1 +
 1 file changed, 1 insertion(+)
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines

drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c void ipu6_isys_csi2_sof_event_by_stream(struct

Quoted 6 lines
drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c +0 -0
index 051898ce53f4..5ad426afa0f0 100644
@@ -578,6 +578,7 @@ void ipu6_isys_csi2_sof_event_by_stream(struct

ipu6_isys_stream *stream)

Quoted 11 lines
Quoted 5 lines
Quoted 5 lines

drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c void ipu6_isys_csi2_sof_event_by_stream(struct ipu6_isys_stream *stream)

Quoted 5 lines
drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c +1 -0
 		.type = V4L2_EVENT_FRAME_SYNC,
 	};
 
+	ev.id = stream->vc;
 	ev.u.frame_sync.frame_sequence = atomic_fetch_inc(&stream-

sequence);

Quoted 5 lines
Quoted 5 lines

drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c void ipu6_isys_csi2_sof_event_by_stream(struct ipu6_isys_stream *stream) · line 584

Quoted 2 lines
drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c +0 -0
 	v4l2_event_queue(vdev, &ev);
 

The id field in struct v4l2_event isn't used for the FRAME_SYNC event and also the virtual channel isn't communicated to the user

space currently.

Quoted 3 lines
Quoted 3 lines

How to use the id field? User space should use SOF to do FRAME_SYNC, but the v4l2_event doesn't distinguish different streams. Do you

have

Quoted 11 lines
Quoted 1 line

any good suggestions?

Ah, now I think I do understand the problem. The events are produced by the
CSI-2 receiver sub-device but there are multiple streams and the event isn't stream-aware?

It's nice that the id field of the FRAME_SYNC event isn't in use for any other purpose. This should be taken into account in event subscription, too.

Quoted 6 lines
Quoted 1 line

I'll add a patch documenting this in the metadata series. Any thoughts?

I think we could use the id field for this purpose, but maybe not to report the VC. Userspace isn't VC-aware, we should instead convert the VC to a stream.

Do you mean that we use a stream id? But which stream is used isn't fixed. Only when video streams on, we select a stream for it. Which sensor uses Which VC is fixed by sensor driver.

Quoted 6 lines
Quoted 1 line

Cc Laurent and Hans.

--
Regards,

Laurent Pinchart

#6 lore raw
Quoted 45 lines · Hongju
Quoted 44 lines · Laurent Pinchart
Quoted 42 lines · Sakari Ailus
Quoted 28 lines · Hongju
Quoted 19 lines · Sakari Ailus
Quoted 18 lines · Hongju Wang

In the virtual channel case, they should use v4l2_event.id to distinguish SOF events of different streams. Therefore, we set
the virtual channel number of the stream as the SOF event ID. This number is unique.

---
 drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c | 1 +
 1 file changed, 1 insertion(+)
drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c +0 -0
index 051898ce53f4..5ad426afa0f0 100644
@@ -578,6 +578,7 @@ void ipu6_isys_csi2_sof_event_by_stream(struct ipu6_isys_stream *stream)
 		.type = V4L2_EVENT_FRAME_SYNC,
 	};
 
Quoted 7 lines

drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c void ipu6_isys_csi2_sof_event_by_stream(struct ipu6_isys_stream *stream) · line 584

Quoted 4 lines
drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c +1 -0
+	ev.id = stream->vc;
 	ev.u.frame_sync.frame_sequence = atomic_fetch_inc(&stream->sequence);
 	v4l2_event_queue(vdev, &ev);
 

The id field in struct v4l2_event isn't used for the FRAME_SYNC
event and also the virtual channel isn't communicated to the user space currently.

Quoted 3 lines

How to use the id field? User space should use SOF to do FRAME_SYNC, but the v4l2_event doesn't distinguish different streams. Do you have any good suggestions?

Ah, now I think I do understand the problem. The events are produced by the CSI-2 receiver sub-device but there are multiple streams and the event isn't stream-aware?

It's nice that the id field of the FRAME_SYNC event isn't in use for any other purpose. This should be taken into account in event subscription, too.

I'll add a patch documenting this in the metadata series. Any thoughts?

Quoted 7 lines
Quoted 3 lines

I think we could use the id field for this purpose, but maybe not to report the VC. Userspace isn't VC-aware, we should instead convert the VC to a stream.

Do you mean that we use a stream id? But which stream is used isn't fixed. Only when video streams on, we select a stream for it. Which sensor uses Which VC is fixed by sensor driver.

The event isn't generated until you start streaming, so by that time you should be able to convert the VC to a stream ID.

The VC can't easily be used by userspace, as there's no way for an application to map VCs to streams. To make this API really usable, we need something that applications can meaningfully use. I would like to see a userspace implementation (ideally in libcamera) to showcase that the API works as expected.

Quoted 1 line
Quoted 1 line
Quoted 1 line

Cc Laurent and Hans.

Signature · 3 lines
Regards,

Laurent Pinchart