Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D55F1C77B73 for ; Thu, 20 Apr 2023 14:49:13 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232042AbjDTOtM (ORCPT ); Thu, 20 Apr 2023 10:49:12 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41482 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230510AbjDTOtL (ORCPT ); Thu, 20 Apr 2023 10:49:11 -0400 Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5EA20423C for ; Thu, 20 Apr 2023 07:49:07 -0700 (PDT) Received: from pendragon.ideasonboard.com (133-32-181-51.west.xps.vectant.ne.jp [133.32.181.51]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id CBE79EF3; Thu, 20 Apr 2023 16:48:57 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1682002138; bh=p0QfeqXVnt/dasTklbvBnlHi/YLVM3/cDfk0XO3YHzk=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=D4TCeRgUUPJwtuX74G4wSxbWpjilHC3C/l9ejX18pWGPZhDJTbMVhJWpIwsL0bvKG onjOEjsr4OTr8UEFGdCuRMSEXFTX8kwZe8ts3BMku1H7abQxLya2q36fIC7g2/1bMb 9/v3YcMVLWyxW2EOtYDWYtLN8nvRR3XxLDKE9/zw= Date: Thu, 20 Apr 2023 17:49:17 +0300 From: Laurent Pinchart To: bingbu.cao@intel.com Cc: linux-media@vger.kernel.org, sakari.ailus@linux.intel.com, ilpo.jarvinen@linux.intel.com, tfiga@chromium.org, senozhatsky@chromium.org, hdegoede@redhat.com, bingbu.cao@linux.intel.com, tian.shu.qiu@intel.com, hongju.wang@intel.com, daniel.h.kang@intel.com Subject: Re: [RFC PATCH 13/14] Documentation: add Intel IPU6 ISYS driver admin-guide doc Message-ID: <20230420144917.GD17497@pendragon.ideasonboard.com> References: <20230413100429.919622-1-bingbu.cao@intel.com> <20230413100429.919622-14-bingbu.cao@intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: <20230413100429.919622-14-bingbu.cao@intel.com> Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org Hi Bingbu, Thank you for the patch. On Thu, Apr 13, 2023 at 06:04:28PM +0800, bingbu.cao@intel.com wrote: > From: Bingbu Cao > > This document mainly describe the functionality of IPU6 and > IPU6 isys driver, and gives an example that how user can do > imaging capture with tools. > > Signed-off-by: Bingbu Cao > --- > Documentation/admin-guide/media/ipu6-isys.rst | 128 +++++++ > .../admin-guide/media/ipu6_isys_graph.svg | 338 ++++++++++++++++++ > .../admin-guide/media/v4l-drivers.rst | 1 + > 3 files changed, 467 insertions(+) > create mode 100644 Documentation/admin-guide/media/ipu6-isys.rst > create mode 100644 Documentation/admin-guide/media/ipu6_isys_graph.svg > > diff --git a/Documentation/admin-guide/media/ipu6-isys.rst b/Documentation/admin-guide/media/ipu6-isys.rst > new file mode 100644 > index 000000000000..083b00449b86 > --- /dev/null > +++ b/Documentation/admin-guide/media/ipu6-isys.rst > @@ -0,0 +1,128 @@ > +.. SPDX-License-Identifier: GPL-2.0 > + > +.. include:: > + > +=============================================================== > +Intel Image Processing Unit 6 (IPU6) Input System driver > +=============================================================== > + > +Copyright |copy| 2023 Intel Corporation > + > +Introduction > +============ > + > +This file documents the Intel IPU6 (6th generation Image Processing Unit) > +Input System (MIPI CSI2 receiver) drivers located under > +drivers/media/pci/intel/ipu6. > + > + > +The Intel IPU6 can be found in certain Intel Chipsets but not in all SKUs: > + > +* TigerLake > +* JasperLake > +* AlderLake > +* RaptorLake > +* MeteorLake > + > +Intel IPU6 is made up of two components - Input System (ISYS) and Processing > +System (PSYS). > + > +The Input System mainly work as MIPI CSI2 receiver which receive and process the s/work/works/ s/receive/receives/ s/process/processes/ > +imaging data from the sensors and outputs the frames to memory. > + > +There are 2 driver modules - intel_ipu6 and intel_ipu6_isys. intel_ipu6 is IPU6 s/is IPU6/is an IPU6/ > +common driver which does PCI configuration, firmware loading and parsing, > +firmware authentication, DMA mapping and IPU-MMU (internal Memory mapping Unit) > +configuration. intel_ipu6_isys implements V4L2, Media Controller and V4L2 > +sub-device interfaces. The IPU6 ISYS driver supports camera sensors connected > +to the IPU6 ISYS through V4L2 sub-device sensor drivers. Will the intel_ipu6 driver also serve a future intel_ipu6_psys driver, or is it specific to the ISYS ? > + > +Input system driver > +=================== > + > +Input System driver mainly configure CSI2 DPHY, construct the firmware stream s/^Input/The Input/ s/configure/configures the/ s/construct/constructs/ > +configuration and send commands to firmware and get response from hardware and s/send/sends/ s/get response/gets responses/ > +firmware and then return buffers to user. s/return/returns/ So control of the ISYS goes through the firmware, the driver doesn't access the hardware directly ? > +The ISYS is represented as several V4L2 sub-devices - 'Intel IPU6 CSI2 $port', > +which provide V4L2 subdev interfaces to the user space, there are also several > +video nodes for each CSI-2 stream capture - 'Intel IPU6 ISYS capture $num' which > +provide interface to user to set formats, queue buffers and streaming. > + > +.. kernel-figure:: ipu6_isys_graph.svg > + :alt: ipu6 isys media graph I see in the figure 6 CSI-2 receivers and 4 video nodes. Are those CSI-2 receivers independent of each other, or do they share SoCs pins ? For instance, I'm thinking about a use case where you would have 4 data lanes and 2 clock lanes, and support capturing from 2x 2-lanes sensors with 2 CSI-2 RX, or 1x 4-lanes sensor with 1 CSI-2 RX. More generally, can you document the input options of the ISYS ? For DT-based systems I'd expect this information to be in the DT bindings, here it needs to be documented separately. On the video device node side, are there 4 independent hardware DMA engines that can be dynamically connected to the different CSI-2 receivers, or is that what the current firmware version provide, with the hardware able to capture more than 4 streams with firmware modifications ? > + > +Capturing frames by IPU6 ISYS > +------------------------------------ > + > +IPU6 ISYS is used to capture frames from the camera sensors connected to the > +CSI2 port. The supported input formats of ISYS are listed in table below: s/port/ports/ > + > +.. tabularcolumns:: |p{0.8cm}|p{4.0cm}|p{4.0cm}| > + > +.. flat-table:: > + :header-rows: 1 > + > + * - IPU6 ISYS supported input formats > + > + * - RGB565, RGB888 > + > + * - UYVY8, YUYV8 Unless I'm mistaken, CSI-2 doesn't have separate UYVY and YUYV formats. Am I missing something ? > + > + * - RAW8, RAW10, RAW12 Is embedded data capture supported ? More generically, can we capture different data types and/or virtual channels from the same CSI-2 input ? What's the maximum resolution supported by the ISYS ? Are the DMA engines independent, or would the maximum resolution be different if you capture from a single camera sensor or multiple camera sensors at the same time ? > + > +Here is an example of IPU6 ISYS raw capture on Dell XPS 9315 laptop. On this > +machine, ov01a10 sensor is connected to IPU ISYS CSI2 port 2, which can > +generate images at sBGGR10 with resolution 1280x800. > + > +Using the media controller APIs, we can configure ov01a10 sensor by > +media-ctl [#f1]_ and yavta [#f2]_ to transmit frames to IPU6 ISYS. > + > +.. code-block:: none > + > + # This example assumes /dev/media0 as the IPU ISYS media device > + export MDEV=/dev/media0 > + > + # Establish the link for the media devices using media-ctl > + media-ctl -d $MDEV -l "\"ov01a10 3-0036\":0 -> \"Intel IPU6 CSI2 2\":0[1]" > + > + # Set the format for the media devices > + media-ctl -d $MDEV -V "ov01a10:0 [fmt:SBGGR10/1280x800]" > + media-ctl -d $MDEV -V "Intel IPU6 CSI2 2:0 [fmt:SBGGR10/1280x800]" > + media-ctl -d $MDEV -V "Intel IPU6 CSI2 2:1 [fmt:SBGGR10/1280x800]" > + > + # Establish the link for the media devices using media-ctl > + media-ctl -d $MDEV -l "\"ov01a10 3-0036\":0 -> \"Intel IPU6 CSI2 2\":0[1]" > + media-ctl -d $MDEV -l "\"Intel IPU6 CSI2 2\":1 ->\"Intel IPU6 ISYS Capture 0\":0[5]" > + > +Once the media pipeline is configured, desired sensor specific settings > +(such as exposure and gain settings) can be set, using the yavta tool. > + > +e.g > + > +.. code-block:: none > + > + # and that ov01a10 sensor is connected to i2c bus 3 with address 0x36 > + export SDEV=$(media-ctl -d $MDEV -e "ov01a10 3-0036") > + > + yavta -w 0x009e0903 400 $SDEV > + yavta -w 0x009e0913 1000 $SDEV > + yavta -w 0x009e0911 2000 $SDEV > + > +Once the desired sensor settings are set, frame captures can be done as below. > + > +e.g > + > +.. code-block:: none > + > + yavta --data-prefix -u -c10 -n5 -I -s 1280x800 --file=/tmp/frame-#.bin \ > + -f SBGGR10 $(media-ctl -d $MDEV -e "Intel IPU6 ISYS Capture 0") > + > +With the above command, 10 frames are captured at 1280x800 resolution with > +sBGGR10 format. The captured frames are available as /tmp/frame-#.bin files. > + > + > +References > +========== > + > +.. [#f1] https://git.ideasonboard.org/?p=media-ctl.git;a=summary > +.. [#f2] https://git.ideasonboard.org/yavta.git > diff --git a/Documentation/admin-guide/media/ipu6_isys_graph.svg b/Documentation/admin-guide/media/ipu6_isys_graph.svg > new file mode 100644 > index 000000000000..661aee18dbe2 > --- /dev/null > +++ b/Documentation/admin-guide/media/ipu6_isys_graph.svg > @@ -0,0 +1,338 @@ > + > + + "http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd"> > + > + > + + viewBox="0.00 0.00 1270.00 288.00" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink"> > + > +board > + > + > + > +n00000001 > + > +Intel IPU6 ISYS Capture 0 > +/dev/video0 > + > + > + > +n00000005 > + > +Intel IPU6 ISYS Capture 1 > +/dev/video1 > + > + > + > +n00000009 > + > +Intel IPU6 ISYS Capture 2 > +/dev/video2 > + > + > + > +n0000000d > + > +Intel IPU6 ISYS Capture 3 > +/dev/video3 > + > + > + > +n00000011 > + > +0 > + > +Intel IPU6 CSI2 0 > +/dev/v4l-subdev0 > + > +1 > + > + > + > +n00000011:port1->n00000001 > + > + > + > + > + > +n00000011:port1->n00000005 > + > + > + > + > + > +n00000011:port1->n00000009 > + > + > + > + > + > +n00000011:port1->n0000000d > + > + > + > + > + > +n00000014 > + > +0 > + > +Intel IPU6 CSI2 1 > +/dev/v4l-subdev1 > + > +1 > + > + > + > +n00000014:port1->n00000001 > + > + > + > + > + > +n00000014:port1->n00000005 > + > + > + > + > + > +n00000014:port1->n00000009 > + > + > + > + > + > +n00000014:port1->n0000000d > + > + > + > + > + > +n00000017 > + > +0 > + > +Intel IPU6 CSI2 2 > +/dev/v4l-subdev2 > + > +1 > + > + > + > +n00000017:port1->n00000001 > + > + > + > + > + > +n00000017:port1->n00000005 > + > + > + > + > + > +n00000017:port1->n00000009 > + > + > + > + > + > +n00000017:port1->n0000000d > + > + > + > + > + > +n0000001a > + > +0 > + > +Intel IPU6 CSI2 3 > +/dev/v4l-subdev3 > + > +1 > + > + > + > +n0000001a:port1->n00000001 > + > + > + > + > + > +n0000001a:port1->n00000005 > + > + > + > + > + > +n0000001a:port1->n00000009 > + > + > + > + > + > +n0000001a:port1->n0000000d > + > + > + > + > + > +n0000001d > + > +0 > + > +Intel IPU6 CSI2 4 > +/dev/v4l-subdev4 > + > +1 > + > + > + > +n0000001d:port1->n00000001 > + > + > + > + > + > +n0000001d:port1->n00000005 > + > + > + > + > + > +n0000001d:port1->n00000009 > + > + > + > + > + > +n0000001d:port1->n0000000d > + > + > + > + > + > +n00000020 > + > +0 > + > +Intel IPU6 CSI2 5 > +/dev/v4l-subdev5 > + > +1 > + > + > + > +n00000020:port1->n00000001 > + > + > + > + > + > +n00000020:port1->n00000005 > + > + > + > + > + > +n00000020:port1->n00000009 > + > + > + > + > + > +n00000020:port1->n0000000d > + > + > + > + > + > +n00000023 > + > +0 > + > +Intel IPU6 CSI2 6 > +/dev/v4l-subdev6 > + > +1 > + > + > + > +n00000023:port1->n00000001 > + > + > + > + > + > +n00000023:port1->n00000005 > + > + > + > + > + > +n00000023:port1->n00000009 > + > + > + > + > + > +n00000023:port1->n0000000d > + > + > + > + > + > +n00000026 > + > +0 > + > +Intel IPU6 CSI2 7 > +/dev/v4l-subdev7 > + > +1 > + > + > + > +n00000026:port1->n00000001 > + > + > + > + > + > +n00000026:port1->n00000005 > + > + > + > + > + > +n00000026:port1->n00000009 > + > + > + > + > + > +n00000026:port1->n0000000d > + > + > + > + > + > +n00000069 > + > + > + > +ov01a10 3-0036 > +/dev/v4l-subdev8 > + > +0 > + > + > + > +n00000069:port0->n00000017:port0 > + > + > + > + > + > diff --git a/Documentation/admin-guide/media/v4l-drivers.rst b/Documentation/admin-guide/media/v4l-drivers.rst > index 1c41f87c3917..f6328a242cbe 100644 > --- a/Documentation/admin-guide/media/v4l-drivers.rst > +++ b/Documentation/admin-guide/media/v4l-drivers.rst > @@ -16,6 +16,7 @@ Video4Linux (V4L) driver-specific documentation > imx > imx7 > ipu3 > + ipu6-isys > ivtv > omap3isp > omap4_camera -- Regards, Laurent Pinchart