Skip to contentSkip to conversation

media: ti-vpe: Add the VIP driver

PATCH In review Benoit Parrot 3 participants 8 messages May 22, 2020 – May 29, 2020

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

cover media: ti-vpe: Add the VIP driver

This patch series add support for the TI VIP video capture engine. VIP stands for Video Input Port, it can be found on devices such as DRA7xx and provides a parallel interface to a video source such as a sensor or TV decoder. Each VIP can support two inputs (slices) and a SoC can be configured with a variable number of VIP's. Each slice can support two ports each connected to its own sub-device.

Here is the output of v4l2-compliance against this driver:

# v4l2-compliance -s -d 1
v4l2-compliance SHA: not available, 32 bits
Compliance test for vip device /dev/video1:
	Driver name      : vip
	Card type        : vip
	Bus info         : platform:vip1:vin2a:stream0
	Driver version   : 5.7.0
	Capabilities     : 0x85200001

Video Capture
Read/Write
Streaming
Extended Pix Format
Device Capabilities
Device Caps : 0x05200001
Video Capture
Read/Write
Streaming
Extended Pix Format

Required ioctls:
test VIDIOC_QUERYCAP: OK

Allow for multiple opens:
test second /dev/video1 open: OK
test VIDIOC_QUERYCAP: OK
test VIDIOC_G/S_PRIORITY: OK
test for unlimited opens: OK

Debug ioctls:
test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported) test VIDIOC_LOG_STATUS: OK

Input ioctls:
test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported) test VIDIOC_G/S_FREQUENCY: OK (Not Supported) test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported) test VIDIOC_ENUMAUDIO: OK (Not Supported)
test VIDIOC_G/S/ENUMINPUT: OK
test VIDIOC_G/S_AUDIO: OK (Not Supported)
Inputs: 1 Audio Inputs: 0 Tuners: 0

Output ioctls:
test VIDIOC_G/S_MODULATOR: OK (Not Supported) test VIDIOC_G/S_FREQUENCY: OK (Not Supported) test VIDIOC_ENUMAUDOUT: OK (Not Supported) test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported) test VIDIOC_G/S_AUDOUT: OK (Not Supported)
Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
test VIDIOC_ENUM/G/S/QUERY_STD: OK
test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported) test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported) test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls (Input 0):
test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK test VIDIOC_QUERYCTRL: OK
test VIDIOC_G/S_CTRL: OK
test VIDIOC_G/S/TRY_EXT_CTRLS: OK test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK test VIDIOC_G/S_JPEGCOMP: OK (Not Supported) Standard Controls: 5 Private Controls: 0

Format ioctls (Input 0):
test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK test VIDIOC_G/S_PARM: OK (Not Supported) test VIDIOC_G_FBUF: OK (Not Supported) test VIDIOC_G_FMT: OK
test VIDIOC_TRY_FMT: OK
test VIDIOC_S_FMT: OK
test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported) test Cropping: OK
test Composing: OK
test Scaling: OK (Not Supported)

Codec ioctls (Input 0):
test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported) test VIDIOC_G_ENC_INDEX: OK (Not Supported) test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls (Input 0):
test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK test VIDIOC_EXPBUF: OK
test Requests: OK (Not Supported)

Test input 0:

Streaming ioctls:
test read/write: OK
test blocking wait: OK
Video Capture: Frame #002 warn: v4l2-test-buffers.cpp(446): got sequence number 4, expected 3
Video Capture: Frame #005 warn: v4l2-test-buffers.cpp(446): got sequence number 8, expected 7
Video Capture: Frame #008 warn: v4l2-test-buffers.cpp(446): got sequence number 12, expected 11
Video Capture: Frame #011 warn: v4l2-test-buffers.cpp(446): got sequence number 16, expected 15
Video Capture: Frame #014 warn: v4l2-test-buffers.cpp(446): got sequence number 20, expected 19
Video Capture: Frame #017 warn: v4l2-test-buffers.cpp(446): got sequence number 24, expected 23
Video Capture: Frame #020 warn: v4l2-test-buffers.cpp(446): got sequence number 28, expected 27
Video Capture: Frame #023 warn: v4l2-test-buffers.cpp(446): got sequence number 32, expected 31
Video Capture: Frame #026 warn: v4l2-test-buffers.cpp(446): got sequence number 36, expected 35
Video Capture: Frame #029 warn: v4l2-test-buffers.cpp(446): got sequence number 40, expected 39
Video Capture: Frame #032 warn: v4l2-test-buffers.cpp(446): got sequence number 44, expected 43
Video Capture: Frame #035 warn: v4l2-test-buffers.cpp(446): got sequence number 48, expected 47
Video Capture: Frame #038 warn: v4l2-test-buffers.cpp(446): got sequence number 52, expected 51
Video Capture: Frame #041 warn: v4l2-test-buffers.cpp(446): got sequence number 56, expected 55
Video Capture: Frame #044 warn: v4l2-test-buffers.cpp(446): got sequence number 60, expected 59
Video Capture: Frame #047 warn: v4l2-test-buffers.cpp(446): got sequence number 64, expected 63
Video Capture: Frame #050 warn: v4l2-test-buffers.cpp(446): got sequence number 68, expected 67
Video Capture: Frame #053 warn: v4l2-test-buffers.cpp(446): got sequence number 72, expected 71
Video Capture: Frame #056 warn: v4l2-test-buffers.cpp(446): got sequence number 76, expected 75
test MMAP (no poll): OK
Video Capture: Frame #002 (select) warn: v4l2-test-buffers.cpp(446): got sequence number 4, expected 3
Video Capture: Frame #005 (select) warn: v4l2-test-buffers.cpp(446): got sequence number 8, expected 7
Video Capture: Frame #008 (select) warn: v4l2-test-buffers.cpp(446): got sequence number 12, expected 11
Video Capture: Frame #011 (select) warn: v4l2-test-buffers.cpp(446): got sequence number 16, expected 15
Video Capture: Frame #014 (select) warn: v4l2-test-buffers.cpp(446): got sequence number 20, expected 19
Video Capture: Frame #017 (select) warn: v4l2-test-buffers.cpp(446): got sequence number 24, expected 23
Video Capture: Frame #020 (select) warn: v4l2-test-buffers.cpp(446): got sequence number 28, expected 27
Video Capture: Frame #023 (select) warn: v4l2-test-buffers.cpp(446): got sequence number 32, expected 31
Video Capture: Frame #026 (select) warn: v4l2-test-buffers.cpp(446): got sequence number 36, expected 35
Video Capture: Frame #029 (select) warn: v4l2-test-buffers.cpp(446): got sequence number 40, expected 39
Video Capture: Frame #032 (select) warn: v4l2-test-buffers.cpp(446): got sequence number 44, expected 43
Video Capture: Frame #035 (select) warn: v4l2-test-buffers.cpp(446): got sequence number 48, expected 47
Video Capture: Frame #038 (select) warn: v4l2-test-buffers.cpp(446): got sequence number 52, expected 51
Video Capture: Frame #041 (select) warn: v4l2-test-buffers.cpp(446): got sequence number 56, expected 55
Video Capture: Frame #044 (select) warn: v4l2-test-buffers.cpp(446): got sequence number 60, expected 59
Video Capture: Frame #047 (select) warn: v4l2-test-buffers.cpp(446): got sequence number 64, expected 63
Video Capture: Frame #050 (select) warn: v4l2-test-buffers.cpp(446): got sequence number 68, expected 67
Video Capture: Frame #053 (select) warn: v4l2-test-buffers.cpp(446): got sequence number 72, expected 71
Video Capture: Frame #056 (select) warn: v4l2-test-buffers.cpp(446): got sequence number 76, expected 75
test MMAP (select): OK
Video Capture: Frame #002 (epoll) warn: v4l2-test-buffers.cpp(446): got sequence number 4, expected 3
Video Capture: Frame #005 (epoll) warn: v4l2-test-buffers.cpp(446): got sequence number 8, expected 7
Video Capture: Frame #008 (epoll) warn: v4l2-test-buffers.cpp(446): got sequence number 12, expected 11
Video Capture: Frame #011 (epoll) warn: v4l2-test-buffers.cpp(446): got sequence number 16, expected 15
Video Capture: Frame #014 (epoll) warn: v4l2-test-buffers.cpp(446): got sequence number 20, expected 19
Video Capture: Frame #017 (epoll) warn: v4l2-test-buffers.cpp(446): got sequence number 24, expected 23
Video Capture: Frame #020 (epoll) warn: v4l2-test-buffers.cpp(446): got sequence number 28, expected 27
Video Capture: Frame #023 (epoll) warn: v4l2-test-buffers.cpp(446): got sequence number 32, expected 31
Video Capture: Frame #026 (epoll) warn: v4l2-test-buffers.cpp(446): got sequence number 36, expected 35
Video Capture: Frame #029 (epoll) warn: v4l2-test-buffers.cpp(446): got sequence number 40, expected 39
Video Capture: Frame #032 (epoll) warn: v4l2-test-buffers.cpp(446): got sequence number 44, expected 43
Video Capture: Frame #035 (epoll) warn: v4l2-test-buffers.cpp(446): got sequence number 48, expected 47
Video Capture: Frame #038 (epoll) warn: v4l2-test-buffers.cpp(446): got sequence number 52, expected 51
Video Capture: Frame #041 (epoll) warn: v4l2-test-buffers.cpp(446): got sequence number 56, expected 55
Video Capture: Frame #044 (epoll) warn: v4l2-test-buffers.cpp(446): got sequence number 60, expected 59
Video Capture: Frame #047 (epoll) warn: v4l2-test-buffers.cpp(446): got sequence number 64, expected 63
Video Capture: Frame #050 (epoll) warn: v4l2-test-buffers.cpp(446): got sequence number 68, expected 67
Video Capture: Frame #053 (epoll) warn: v4l2-test-buffers.cpp(446): got sequence number 72, expected 71
Video Capture: Frame #056 (epoll) warn: v4l2-test-buffers.cpp(446): got sequence number 76, expected 75
test MMAP (epoll): OK
test USERPTR (no poll): OK (Not Supported) test USERPTR (select): OK (Not Supported) test DMABUF: Cannot test, specify --expbuf-device

Total for vip device /dev/video1: 51, Succeeded: 51, Failed: 0, Warnings: 57

Benoit Parrot (2):
dt-binbings: media: ti-vpe: Document the VIP driver
media: ti-vpe: Add the VIP driver

 .../devicetree/bindings/media/ti,vip.yaml     |  394 ++
 MAINTAINERS                                   |    1 +
 drivers/media/platform/Kconfig                |   13 +
 drivers/media/platform/ti-vpe/Makefile        |    2 +
 drivers/media/platform/ti-vpe/vip.c           | 4158 +++++++++++++++++
 drivers/media/platform/ti-vpe/vip.h           |  724 +++
 6 files changed, 5292 insertions(+)
 create mode 100644 Documentation/devicetree/bindings/media/ti,vip.yaml
 create mode 100644 drivers/media/platform/ti-vpe/vip.c
 create mode 100644 drivers/media/platform/ti-vpe/vip.h
Signature · 1 line
2.17.1
#1 lore raw

PATCH 1/2 dt-binbings: media: ti-vpe: Document the VIP driver

Device Tree bindings for the Video Input Port (VIP) driver.

---
 .../devicetree/bindings/media/ti,vip.yaml     | 394 ++++++++++++++++++
 MAINTAINERS                                   |   1 +
 2 files changed, 395 insertions(+)
 create mode 100644 Documentation/devicetree/bindings/media/ti,vip.yaml
Documentation/devicetree/bindings/media/ti,vip.yaml new file +394 -0
new file mode 100644
index 000000000000..8a9084e42329
@@ -0,0 +1,394 @@
+# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/media/ti,vip.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: Texas Instruments DRA7x VIDEO INPUT PORT (VIP) Device Tree Bindings
+
+maintainers:
+  - Benoit Parrot <[email protected]>
+
+description: |-
+  The Video Input Port (VIP) is a key component for image capture
+  applications. The capture module provides the system interface and the
+  processing capability to connect parallel image-sensor as well as
+  BT.656/1120 capable encoder chip to DRA7x device.
+
+  Each VIP instance supports 2 independently configurable external video
+  input capture slices (Slice 0 and Slice 1) each providing up to two video
+  input ports (Port A and Port B) where Port A can be configured as
+  24/16/8-bit port and Port B is fixed as 8-bit port.
+  Here these ports a represented as follows
+    port@0 -> Slice 0 Port A
+    port@1 -> Slice 0 Port B
+    port@2 -> Slice 1 Port A
+    port@3 -> Slice 1 Port B
+
+  Each camera port nodes should contain a 'port' child node with child
+  'endpoint' node. Please refer to the bindings defined in
+  Documentation/devicetree/bindings/media/video-interfaces.txt.
+
+properties:
+  compatible:
+    const: ti,dra7-vip
+
+  label:
+    description: Instance name
+
+  reg:
+    items:
+      - description: The VIP main register region
+      - description: Video Data Parser (PARSER) register region for Slice0
+      - description: Color Space Conversion (CSC) register region for Slice0
+      - description: Scaler (SC) register region for Slice0
+      - description: Video Data Parser (PARSER) register region for Slice1
+      - description: Color Space Conversion (CSC) register region for Slice1
+      - description: Scaler (SC) register region for Slice1
+      - description: Video Port Direct Memory Access (VPDMA) register region
+
+  reg-names:
+    items:
+      - const: vip
+      - const: parser0
+      - const: csc0
+      - const: sc0
+      - const: parser1
+      - const: csc1
+      - const: sc1
+      - const: vpdma
+
+  interrupts:
+    minItems: 2
+    description:
+      IRQ index 0 is used for Slice0 interrupts
+      IRQ index 1 is used for Slice1 interrupts
+
+  ti,vip-clk-polarity:
+    $ref: "/schemas/types.yaml#/definitions/phandle-array"
+    description:
+      phandle to the device control module. The 1st argument should
+      contain the register offset to the CTRL_CORE_SMA_SW_1 register.
+      2nd argument contains the bit field to slice 0 port A,
+      3rd argument contains the bit field to slice 0 port B,
+      4th argument contains the bit field to slice 1 port A,
+      5th argument contains the bit field to slice 1 port B.
+
+  # See ./video-interfaces.txt for details
+  ports:
+    type: object
+    additionalProperties: false
+
+    properties:
+      "#address-cells":
+        const: 1
+
+      "#size-cells":
+        const: 0
+
+      port@0:
+        type: object
+        additionalProperties: false
+
+        properties:
+          reg:
+            const: 0
+            description: Slice 0 Port A
+
+          label:
+            description: Port name. Usually the pin group name
+
+        patternProperties:
+          endpoint:
+            type: object
+            additionalProperties: false
+
+            properties:
+              hsync-active:
+                maxItems: 1
+
+              vsync-active:
+                maxItems: 1
+
+              pclk-sample:
+                maxItems: 1
+
+              bus-width:
+                maxItems: 1
+
+              ti,vip-pixel-mux:
+                type: boolean
+                description:
+                  In BT656/1120 mode, this enable pixel-muxing if
+                  the number of channels is either 1, 2 or 4. If this
+                  property is present then pixel-muxing is enabled
+                  otherwise it will use line-muxing.
+
+              ti,vip-channels:
+                $ref: "/schemas/types.yaml#definitions/uint8-array"
+                minItems: 1
+                maxItems: 16
+                description: |-
+                  In BT656/1120 mode, list of channel ids to be captured.
+                  If the property is not present then 1 channel is assumed.
+
+              remote-endpoint: true
+
+        required:
+          - reg
+          - label
+
+      port@1:
+        type: object
+        additionalProperties: false
+
+        properties:
+          reg:
+            const: 1
+            description: Slice 0 Port B
+
+          label:
+            description: Port name. Usually the pin group name
+
+        patternProperties:
+          endpoint:
+            type: object
+            additionalProperties: false
+
+            properties:
+              hsync-active:
+                maxItems: 1
+
+              vsync-active:
+                maxItems: 1
+
+              pclk-sample:
+                maxItems: 1
+
+              bus-width:
+                maxItems: 1
+
+              ti,vip-pixel-mux:
+                type: boolean
+                description:
+                  In BT656/1120 mode, this enable pixel-muxing if
+                  the number of channels is either 1, 2 or 4. If this
+                  property is present then pixel-muxing is enabled
+                  otherwise it will use line-muxing.
+
+              ti,vip-channels:
+                $ref: "/schemas/types.yaml#definitions/uint8-array"
+                minItems: 1
+                maxItems: 16
+                description:
+                  In BT656/1120 mode, list of channel ids to be captured.
+                  If the property is not present then 1 channel is assumed.
+
+              remote-endpoint: true
+
+        required:
+          - reg
+          - label
+
+      port@2:
+        type: object
+        additionalProperties: false
+
+        properties:
+          reg:
+            const: 2
+            description: Slice 1 Port A
+
+          label:
+            description: Port name. Usually the pin group name
+
+        patternProperties:
+          endpoint:
+            type: object
+            additionalProperties: false
+
+            properties:
+              hsync-active:
+                maxItems: 1
+
+              vsync-active:
+                maxItems: 1
+
+              pclk-sample:
+                maxItems: 1
+
+              bus-width:
+                maxItems: 1
+
+              ti,vip-pixel-mux:
+                type: boolean
+                description:
+                  In BT656/1120 mode, this enable pixel-muxing if
+                  the number of channels is either 1, 2 or 4. If this
+                  property is present then pixel-muxing is enabled
+                  otherwise it will use line-muxing.
+
+              ti,vip-channels:
+                $ref: "/schemas/types.yaml#definitions/uint8-array"
+                minItems: 1
+                maxItems: 16
+                description:
+                  In BT656/1120 mode, list of channel ids to be captured.
+                  If the property is not present then 1 channel is assumed.
+
+              remote-endpoint: true
+
+        required:
+          - reg
+          - label
+
+      port@3:
+        type: object
+        additionalProperties: false
+
+        properties:
+          reg:
+            const: 3
+            description: Slice 1 Port B
+
+          label:
+            description: Port name. Usually the pin group name
+
+        patternProperties:
+          endpoint:
+            type: object
+            additionalProperties: false
+
+            properties:
+              hsync-active:
+                maxItems: 1
+
+              vsync-active:
+                maxItems: 1
+
+              pclk-sample:
+                maxItems: 1
+
+              bus-width:
+                maxItems: 1
+
+              ti,vip-pixel-mux:
+                type: boolean
+                description:
+                  In BT656/1120 mode, this enable pixel-muxing if
+                  the number of channels is either 1, 2 or 4. If this
+                  property is present then pixel-muxing is enabled
+                  otherwise it will use line-muxing.
+
+              ti,vip-channels:
+                $ref: "/schemas/types.yaml#definitions/uint8-array"
+                minItems: 1
+                maxItems: 16
+                description:
+                  In BT656/1120 mode, list of channel ids to be captured.
+                  If the property is not present then 1 channel is assumed.
+
+              remote-endpoint: true
+
+        required:
+          - reg
+          - label
+
+    required:
+      - "#address-cells"
+      - "#size-cells"
+      - port@0
+
+required:
+  - compatible
+  - label
+  - reg
+  - reg-names
+  - interrupts
+  - ti,vip-clk-polarity
+
+additionalProperties: false
+
+examples:
+  - |
+    #include <dt-bindings/interrupt-controller/arm-gic.h>
+
+    vip1: vip@48970000 {
+        compatible = "ti,dra7-vip";
+        label = "vip1";
+        reg = <0x48970000 0x114>,
+              <0x48975500 0xD8>,
+              <0x48975700 0x18>,
+              <0x48975800 0x80>,
+              <0x48975a00 0xD8>,
+              <0x48975c00 0x18>,
+              <0x48975d00 0x80>,
+              <0x4897d000 0x400>;
+        reg-names = "vip",
+                    "parser0",
+                    "csc0",
+                    "sc0",
+                    "parser1",
+                    "csc1",
+                    "sc1",
+                    "vpdma";
+        interrupts = <GIC_SPI 351 IRQ_TYPE_LEVEL_HIGH>,
+                     <GIC_SPI 392 IRQ_TYPE_LEVEL_HIGH>;
+        ti,vip-clk-polarity = <&scm_conf 0x534 0x1 0x4 0x2 0x8>;
+
+        ports {
+              #address-cells = <1>;
+              #size-cells = <0>;
+
+              vin1a: port@0 {
+                    reg = <0>;
+                    label = "vin1a";
+
+                    vin1a_ep: endpoint {
+                           remote-endpoint = <&camera1>;
+                           hsync-active = <1>;
+                           vsync-active = <1>;
+                           pclk-sample = <0>;
+                           bus-width = <8>;
+                    };
+              };
+              vin1b: port@1 {
+                    reg = <1>;
+                    label = "vin1b";
+              };
+              vin2a: port@2 {
+                    reg = <2>;
+                    label = "vin2a";
+              };
+              vin2b: port@3 {
+                    reg = <3>;
+                    label = "vin2b";
+              };
+         };
+    };
+
+    i2c {
+        clock-frequency = <400000>;
+        #address-cells = <1>;
+        #size-cells = <0>;
+
+         camera@37 {
+              compatible = "ovti,ov10633";
+              reg = <0x37>;
+
+              clocks = <&fixed_clock>;
+              clocks-names = "xvclk";
+
+              port {
+                   camera1: endpoint {
+                           remote-endpoint = <&vin1a_ep>;
+                           hsync-active = <1>;
+                           vsync-active = <1>;
+                           pclk-sample = <0>;
+                           bus-width = <8>;
+                   };
+              };
+         };
+    };
+
+...
MAINTAINERS +1 -0
index 2e9a5f6e4ff7..06856d05b53b 100644
@@ -16947,6 +16947,7 @@ S:	Maintained
 W:	http://linuxtv.org/
 Q:	http://patchwork.linuxtv.org/project/linux-media/list/
 F:	Documentation/devicetree/bindings/media/ti,cal.yaml
+F:	Documentation/devicetree/bindings/media/ti,vip.yaml
 F:	Documentation/devicetree/bindings/media/ti,vpe.yaml
 F:	drivers/media/platform/ti-vpe/
 
Signature · 1 line
2.17.1
maintainer
#2 lore raw

Re: [Patch 1/2] dt-binbings: media: ti-vpe: Document the VIP driver

Quoted 1 line · Benoit Parrot

Device Tree bindings for the Video Input Port (VIP) driver.

Bindings document h/w, not drivers.

Quoted 41 lines
---
 .../devicetree/bindings/media/ti,vip.yaml     | 394 ++++++++++++++++++
 MAINTAINERS                                   |   1 +
 2 files changed, 395 insertions(+)
 create mode 100644 Documentation/devicetree/bindings/media/ti,vip.yaml
Documentation/devicetree/bindings/media/ti,vip.yaml new file +27 -0
new file mode 100644
index 000000000000..8a9084e42329
@@ -0,0 +1,394 @@
+# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/media/ti,vip.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: Texas Instruments DRA7x VIDEO INPUT PORT (VIP) Device Tree Bindings
+
+maintainers:
+  - Benoit Parrot <[email protected]>
+
+description: |-
+  The Video Input Port (VIP) is a key component for image capture
+  applications. The capture module provides the system interface and the
+  processing capability to connect parallel image-sensor as well as
+  BT.656/1120 capable encoder chip to DRA7x device.
+
+  Each VIP instance supports 2 independently configurable external video
+  input capture slices (Slice 0 and Slice 1) each providing up to two video
+  input ports (Port A and Port B) where Port A can be configured as
+  24/16/8-bit port and Port B is fixed as 8-bit port.
+  Here these ports a represented as follows
+    port@0 -> Slice 0 Port A
+    port@1 -> Slice 0 Port B
+    port@2 -> Slice 1 Port A
+    port@3 -> Slice 1 Port B
+

Documentation/devicetree/bindings/media/ti,vip.yaml line 37

Quoted 10 lines
Documentation/devicetree/bindings/media/ti,vip.yaml new file +10 -0
+  Each camera port nodes should contain a 'port' child node with child
+  'endpoint' node. Please refer to the bindings defined in
+  Documentation/devicetree/bindings/media/video-interfaces.txt.
+
+properties:
+  compatible:
+    const: ti,dra7-vip
+
+  label:
+    description: Instance name

Kind of odd for this type of binding. Are there a define set or pattern of values.

Quoted 61 lines
Documentation/devicetree/bindings/media/ti,vip.yaml new file +61 -0
+
+  reg:
+    items:
+      - description: The VIP main register region
+      - description: Video Data Parser (PARSER) register region for Slice0
+      - description: Color Space Conversion (CSC) register region for Slice0
+      - description: Scaler (SC) register region for Slice0
+      - description: Video Data Parser (PARSER) register region for Slice1
+      - description: Color Space Conversion (CSC) register region for Slice1
+      - description: Scaler (SC) register region for Slice1
+      - description: Video Port Direct Memory Access (VPDMA) register region
+
+  reg-names:
+    items:
+      - const: vip
+      - const: parser0
+      - const: csc0
+      - const: sc0
+      - const: parser1
+      - const: csc1
+      - const: sc1
+      - const: vpdma
+
+  interrupts:
+    minItems: 2
+    description:
+      IRQ index 0 is used for Slice0 interrupts
+      IRQ index 1 is used for Slice1 interrupts
+
+  ti,vip-clk-polarity:
+    $ref: "/schemas/types.yaml#/definitions/phandle-array"
+    description:
+      phandle to the device control module. The 1st argument should
+      contain the register offset to the CTRL_CORE_SMA_SW_1 register.
+      2nd argument contains the bit field to slice 0 port A,
+      3rd argument contains the bit field to slice 0 port B,
+      4th argument contains the bit field to slice 1 port A,
+      5th argument contains the bit field to slice 1 port B.
+
+  # See ./video-interfaces.txt for details
+  ports:
+    type: object
+    additionalProperties: false
+
+    properties:
+      "#address-cells":
+        const: 1
+
+      "#size-cells":
+        const: 0
+
+      port@0:
+        type: object
+        additionalProperties: false
+
+        properties:
+          reg:
+            const: 0
+            description: Slice 0 Port A
+
+          label:

Documentation/devicetree/bindings/media/ti,vip.yaml line 108

Quoted 10 lines
Documentation/devicetree/bindings/media/ti,vip.yaml new file +10 -0
+            description: Port name. Usually the pin group name
+
+        patternProperties:
+          endpoint:
+            type: object
+            additionalProperties: false
+
+            properties:
+              hsync-active:
+                maxItems: 1

Not an array. Just:

hsync-active: true

Documentation/devicetree/bindings/media/ti,vip.yaml line 117

Quoted 9 lines
Documentation/devicetree/bindings/media/ti,vip.yaml new file +9 -0
+
+              vsync-active:
+                maxItems: 1
+
+              pclk-sample:
+                maxItems: 1
+
+              bus-width:
+                maxItems: 1

Not an array. What subset of values are allowed?

Quoted 164 lines
Documentation/devicetree/bindings/media/ti,vip.yaml new file +164 -0
+
+              ti,vip-pixel-mux:
+                type: boolean
+                description:
+                  In BT656/1120 mode, this enable pixel-muxing if
+                  the number of channels is either 1, 2 or 4. If this
+                  property is present then pixel-muxing is enabled
+                  otherwise it will use line-muxing.
+
+              ti,vip-channels:
+                $ref: "/schemas/types.yaml#definitions/uint8-array"
+                minItems: 1
+                maxItems: 16
+                description: |-
+                  In BT656/1120 mode, list of channel ids to be captured.
+                  If the property is not present then 1 channel is assumed.
+
+              remote-endpoint: true
+
+        required:
+          - reg
+          - label
+
+      port@1:
+        type: object
+        additionalProperties: false
+
+        properties:
+          reg:
+            const: 1
+            description: Slice 0 Port B
+
+          label:
+            description: Port name. Usually the pin group name
+
+        patternProperties:
+          endpoint:
+            type: object
+            additionalProperties: false
+
+            properties:
+              hsync-active:
+                maxItems: 1
+
+              vsync-active:
+                maxItems: 1
+
+              pclk-sample:
+                maxItems: 1
+
+              bus-width:
+                maxItems: 1
+
+              ti,vip-pixel-mux:
+                type: boolean
+                description:
+                  In BT656/1120 mode, this enable pixel-muxing if
+                  the number of channels is either 1, 2 or 4. If this
+                  property is present then pixel-muxing is enabled
+                  otherwise it will use line-muxing.
+
+              ti,vip-channels:
+                $ref: "/schemas/types.yaml#definitions/uint8-array"
+                minItems: 1
+                maxItems: 16
+                description:
+                  In BT656/1120 mode, list of channel ids to be captured.
+                  If the property is not present then 1 channel is assumed.
+
+              remote-endpoint: true
+
+        required:
+          - reg
+          - label
+
+      port@2:
+        type: object
+        additionalProperties: false
+
+        properties:
+          reg:
+            const: 2
+            description: Slice 1 Port A
+
+          label:
+            description: Port name. Usually the pin group name
+
+        patternProperties:
+          endpoint:
+            type: object
+            additionalProperties: false
+
+            properties:
+              hsync-active:
+                maxItems: 1
+
+              vsync-active:
+                maxItems: 1
+
+              pclk-sample:
+                maxItems: 1
+
+              bus-width:
+                maxItems: 1
+
+              ti,vip-pixel-mux:
+                type: boolean
+                description:
+                  In BT656/1120 mode, this enable pixel-muxing if
+                  the number of channels is either 1, 2 or 4. If this
+                  property is present then pixel-muxing is enabled
+                  otherwise it will use line-muxing.
+
+              ti,vip-channels:
+                $ref: "/schemas/types.yaml#definitions/uint8-array"
+                minItems: 1
+                maxItems: 16
+                description:
+                  In BT656/1120 mode, list of channel ids to be captured.
+                  If the property is not present then 1 channel is assumed.
+
+              remote-endpoint: true
+
+        required:
+          - reg
+          - label
+
+      port@3:
+        type: object
+        additionalProperties: false
+
+        properties:
+          reg:
+            const: 3
+            description: Slice 1 Port B
+
+          label:
+            description: Port name. Usually the pin group name
+
+        patternProperties:
+          endpoint:
+            type: object
+            additionalProperties: false
+
+            properties:
+              hsync-active:
+                maxItems: 1
+
+              vsync-active:
+                maxItems: 1
+
+              pclk-sample:
+                maxItems: 1
+
+              bus-width:
+                maxItems: 1
+
+              ti,vip-pixel-mux:
+                type: boolean
+                description:
+                  In BT656/1120 mode, this enable pixel-muxing if
+                  the number of channels is either 1, 2 or 4. If this
+                  property is present then pixel-muxing is enabled
+                  otherwise it will use line-muxing.

Documentation/devicetree/bindings/media/ti,vip.yaml line 291

Quoted 10 lines
Documentation/devicetree/bindings/media/ti,vip.yaml new file +10 -0
+
+              ti,vip-channels:
+                $ref: "/schemas/types.yaml#definitions/uint8-array"
+                minItems: 1
+                maxItems: 16
+                description:
+                  In BT656/1120 mode, list of channel ids to be captured.
+                  If the property is not present then 1 channel is assumed.
+
+              remote-endpoint: true

If all the properties are the same across ports, then do a patternProperties with '^port@' and define them there. You'll still need 'port@0', etc. to define what each port is.

Quoted 118 lines
Documentation/devicetree/bindings/media/ti,vip.yaml new file +103 -0
+
+        required:
+          - reg
+          - label
+
+    required:
+      - "#address-cells"
+      - "#size-cells"
+      - port@0
+
+required:
+  - compatible
+  - label
+  - reg
+  - reg-names
+  - interrupts
+  - ti,vip-clk-polarity
+
+additionalProperties: false
+
+examples:
+  - |
+    #include <dt-bindings/interrupt-controller/arm-gic.h>
+
+    vip1: vip@48970000 {
+        compatible = "ti,dra7-vip";
+        label = "vip1";
+        reg = <0x48970000 0x114>,
+              <0x48975500 0xD8>,
+              <0x48975700 0x18>,
+              <0x48975800 0x80>,
+              <0x48975a00 0xD8>,
+              <0x48975c00 0x18>,
+              <0x48975d00 0x80>,
+              <0x4897d000 0x400>;
+        reg-names = "vip",
+                    "parser0",
+                    "csc0",
+                    "sc0",
+                    "parser1",
+                    "csc1",
+                    "sc1",
+                    "vpdma";
+        interrupts = <GIC_SPI 351 IRQ_TYPE_LEVEL_HIGH>,
+                     <GIC_SPI 392 IRQ_TYPE_LEVEL_HIGH>;
+        ti,vip-clk-polarity = <&scm_conf 0x534 0x1 0x4 0x2 0x8>;
+
+        ports {
+              #address-cells = <1>;
+              #size-cells = <0>;
+
+              vin1a: port@0 {
+                    reg = <0>;
+                    label = "vin1a";
+
+                    vin1a_ep: endpoint {
+                           remote-endpoint = <&camera1>;
+                           hsync-active = <1>;
+                           vsync-active = <1>;
+                           pclk-sample = <0>;
+                           bus-width = <8>;
+                    };
+              };
+              vin1b: port@1 {
+                    reg = <1>;
+                    label = "vin1b";
+              };
+              vin2a: port@2 {
+                    reg = <2>;
+                    label = "vin2a";
+              };
+              vin2b: port@3 {
+                    reg = <3>;
+                    label = "vin2b";
+              };
+         };
+    };
+
+    i2c {
+        clock-frequency = <400000>;
+        #address-cells = <1>;
+        #size-cells = <0>;
+
+         camera@37 {
+              compatible = "ovti,ov10633";
+              reg = <0x37>;
+
+              clocks = <&fixed_clock>;
+              clocks-names = "xvclk";
+
+              port {
+                   camera1: endpoint {
+                           remote-endpoint = <&vin1a_ep>;
+                           hsync-active = <1>;
+                           vsync-active = <1>;
+                           pclk-sample = <0>;
+                           bus-width = <8>;
+                   };
+              };
+         };
+    };
+
+...
MAINTAINERS +1 -0
index 2e9a5f6e4ff7..06856d05b53b 100644
@@ -16947,6 +16947,7 @@ S:	Maintained
 W:	http://linuxtv.org/
 Q:	http://patchwork.linuxtv.org/project/linux-media/list/
 F:	Documentation/devicetree/bindings/media/ti,cal.yaml
+F:	Documentation/devicetree/bindings/media/ti,vip.yaml
 F:	Documentation/devicetree/bindings/media/ti,vpe.yaml
 F:	drivers/media/platform/ti-vpe/
 
Signature · 1 line
2.17.1
#3 lore raw

Re: [Patch 1/2] dt-binbings: media: ti-vpe: Document the VIP driver

Hi Rob,

Thanks for the review.

Rob Herring <[email protected]> wrote on Thu [2020-May-28 17:39:33 -0600]:

Quoted 4 lines · Benoit Parrot
Quoted 1 line

Device Tree bindings for the Video Input Port (VIP) driver.

Bindings document h/w, not drivers.

I'll fix that.

Quoted 47 lines
Quoted 47 lines
---
 .../devicetree/bindings/media/ti,vip.yaml     | 394 ++++++++++++++++++
 MAINTAINERS                                   |   1 +
 2 files changed, 395 insertions(+)
 create mode 100644 Documentation/devicetree/bindings/media/ti,vip.yaml
Documentation/devicetree/bindings/media/ti,vip.yaml new file +32 -0
new file mode 100644
index 000000000000..8a9084e42329
@@ -0,0 +1,394 @@
+# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/media/ti,vip.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: Texas Instruments DRA7x VIDEO INPUT PORT (VIP) Device Tree Bindings
+
+maintainers:
+  - Benoit Parrot <[email protected]>
+
+description: |-
+  The Video Input Port (VIP) is a key component for image capture
+  applications. The capture module provides the system interface and the
+  processing capability to connect parallel image-sensor as well as
+  BT.656/1120 capable encoder chip to DRA7x device.
+
+  Each VIP instance supports 2 independently configurable external video
+  input capture slices (Slice 0 and Slice 1) each providing up to two video
+  input ports (Port A and Port B) where Port A can be configured as
+  24/16/8-bit port and Port B is fixed as 8-bit port.
+  Here these ports a represented as follows
+    port@0 -> Slice 0 Port A
+    port@1 -> Slice 0 Port B
+    port@2 -> Slice 1 Port A
+    port@3 -> Slice 1 Port B
+
+  Each camera port nodes should contain a 'port' child node with child
+  'endpoint' node. Please refer to the bindings defined in
+  Documentation/devicetree/bindings/media/video-interfaces.txt.
+
+properties:
Quoted 8 lines

Documentation/devicetree/bindings/media/ti,vip.yaml line 37

Quoted 5 lines
Documentation/devicetree/bindings/media/ti,vip.yaml new file +5 -0
+  compatible:
+    const: ti,dra7-vip
+
+  label:
+    description: Instance name

Kind of odd for this type of binding. Are there a define set or pattern of values.

The label here are used to 'name' each H/W instance. And so far there can be a up to 3 instances with the currently available devices, but that doesn't mean there couldn't be more in the future. So I did not want to limit it here.

Quoted 75 lines
Quoted 62 lines
Documentation/devicetree/bindings/media/ti,vip.yaml new file +61 -0
+
+  reg:
+    items:
+      - description: The VIP main register region
+      - description: Video Data Parser (PARSER) register region for Slice0
+      - description: Color Space Conversion (CSC) register region for Slice0
+      - description: Scaler (SC) register region for Slice0
+      - description: Video Data Parser (PARSER) register region for Slice1
+      - description: Color Space Conversion (CSC) register region for Slice1
+      - description: Scaler (SC) register region for Slice1
+      - description: Video Port Direct Memory Access (VPDMA) register region
+
+  reg-names:
+    items:
+      - const: vip
+      - const: parser0
+      - const: csc0
+      - const: sc0
+      - const: parser1
+      - const: csc1
+      - const: sc1
+      - const: vpdma
+
+  interrupts:
+    minItems: 2
+    description:
+      IRQ index 0 is used for Slice0 interrupts
+      IRQ index 1 is used for Slice1 interrupts
+
+  ti,vip-clk-polarity:
+    $ref: "/schemas/types.yaml#/definitions/phandle-array"
+    description:
+      phandle to the device control module. The 1st argument should
+      contain the register offset to the CTRL_CORE_SMA_SW_1 register.
+      2nd argument contains the bit field to slice 0 port A,
+      3rd argument contains the bit field to slice 0 port B,
+      4th argument contains the bit field to slice 1 port A,
+      5th argument contains the bit field to slice 1 port B.
+
+  # See ./video-interfaces.txt for details
+  ports:
+    type: object
+    additionalProperties: false
+
+    properties:
+      "#address-cells":
+        const: 1
+
+      "#size-cells":
+        const: 0
+
+      port@0:
+        type: object
+        additionalProperties: false
+
+        properties:
+          reg:
+            const: 0
+            description: Slice 0 Port A
+
+          label:

Documentation/devicetree/bindings/media/ti,vip.yaml line 108

Quoted 10 lines
Documentation/devicetree/bindings/media/ti,vip.yaml new file +10 -0
+            description: Port name. Usually the pin group name
+
+        patternProperties:
+          endpoint:
+            type: object
+            additionalProperties: false
+
+            properties:
+              hsync-active:
+                maxItems: 1

Not an array. Just:

Quoted 1 line

hsync-active: true

OK I'll fix all of those.

Quoted 12 lines

Documentation/devicetree/bindings/media/ti,vip.yaml line 117

Quoted 10 lines
Documentation/devicetree/bindings/media/ti,vip.yaml new file +9 -0
+
+              vsync-active:
+                maxItems: 1
+
+              pclk-sample:
+                maxItems: 1
+
+              bus-width:
+                maxItems: 1

Not an array. What subset of values are allowed?

So something like oneOf with a list then?

Quoted 171 lines
Quoted 171 lines
Documentation/devicetree/bindings/media/ti,vip.yaml new file +170 -0
+
+              ti,vip-pixel-mux:
+                type: boolean
+                description:
+                  In BT656/1120 mode, this enable pixel-muxing if
+                  the number of channels is either 1, 2 or 4. If this
+                  property is present then pixel-muxing is enabled
+                  otherwise it will use line-muxing.
+
+              ti,vip-channels:
+                $ref: "/schemas/types.yaml#definitions/uint8-array"
+                minItems: 1
+                maxItems: 16
+                description: |-
+                  In BT656/1120 mode, list of channel ids to be captured.
+                  If the property is not present then 1 channel is assumed.
+
+              remote-endpoint: true
+
+        required:
+          - reg
+          - label
+
+      port@1:
+        type: object
+        additionalProperties: false
+
+        properties:
+          reg:
+            const: 1
+            description: Slice 0 Port B
+
+          label:
+            description: Port name. Usually the pin group name
+
+        patternProperties:
+          endpoint:
+            type: object
+            additionalProperties: false
+
+            properties:
+              hsync-active:
+                maxItems: 1
+
+              vsync-active:
+                maxItems: 1
+
+              pclk-sample:
+                maxItems: 1
+
+              bus-width:
+                maxItems: 1
+
+              ti,vip-pixel-mux:
+                type: boolean
+                description:
+                  In BT656/1120 mode, this enable pixel-muxing if
+                  the number of channels is either 1, 2 or 4. If this
+                  property is present then pixel-muxing is enabled
+                  otherwise it will use line-muxing.
+
+              ti,vip-channels:
+                $ref: "/schemas/types.yaml#definitions/uint8-array"
+                minItems: 1
+                maxItems: 16
+                description:
+                  In BT656/1120 mode, list of channel ids to be captured.
+                  If the property is not present then 1 channel is assumed.
+
+              remote-endpoint: true
+
+        required:
+          - reg
+          - label
+
+      port@2:
+        type: object
+        additionalProperties: false
+
+        properties:
+          reg:
+            const: 2
+            description: Slice 1 Port A
+
+          label:
+            description: Port name. Usually the pin group name
+
+        patternProperties:
+          endpoint:
+            type: object
+            additionalProperties: false
+
+            properties:
+              hsync-active:
+                maxItems: 1
+
+              vsync-active:
+                maxItems: 1
+
+              pclk-sample:
+                maxItems: 1
+
+              bus-width:
+                maxItems: 1
+
+              ti,vip-pixel-mux:
+                type: boolean
+                description:
+                  In BT656/1120 mode, this enable pixel-muxing if
+                  the number of channels is either 1, 2 or 4. If this
+                  property is present then pixel-muxing is enabled
+                  otherwise it will use line-muxing.
+
+              ti,vip-channels:
+                $ref: "/schemas/types.yaml#definitions/uint8-array"
+                minItems: 1
+                maxItems: 16
+                description:
+                  In BT656/1120 mode, list of channel ids to be captured.
+                  If the property is not present then 1 channel is assumed.
+
+              remote-endpoint: true
+
+        required:
+          - reg
+          - label
+
+      port@3:
+        type: object
+        additionalProperties: false
+
+        properties:
+          reg:
+            const: 3
+            description: Slice 1 Port B
+
+          label:
+            description: Port name. Usually the pin group name
+
+        patternProperties:
+          endpoint:
+            type: object
+            additionalProperties: false
+
+            properties:
+              hsync-active:
+                maxItems: 1
+
+              vsync-active:
+                maxItems: 1
+
+              pclk-sample:
+                maxItems: 1
+
+              bus-width:
+                maxItems: 1
+
+              ti,vip-pixel-mux:
+                type: boolean
+                description:
+                  In BT656/1120 mode, this enable pixel-muxing if
+                  the number of channels is either 1, 2 or 4. If this
+                  property is present then pixel-muxing is enabled
+                  otherwise it will use line-muxing.
+
+              ti,vip-channels:
+                $ref: "/schemas/types.yaml#definitions/uint8-array"
+                minItems: 1
+                maxItems: 16
+                description:
Quoted 8 lines

Documentation/devicetree/bindings/media/ti,vip.yaml line 135

Quoted 4 lines
Documentation/devicetree/bindings/media/ti,vip.yaml new file +4 -0
+                  In BT656/1120 mode, list of channel ids to be captured.
+                  If the property is not present then 1 channel is assumed.
+
+              remote-endpoint: true

If all the properties are the same across ports, then do a patternProperties with '^port@' and define them there. You'll still need 'port@0', etc. to define what each port is.

Yeah I had tried to define something like that but I couldn't make pass dt-binding-check wihtout any errors/warnings.

Can you point me to a recent example using this syntax? Last time I check I couldn't find one.

Also I am not quite clear on the mechanism, you said to create a patterProperties and then I wouols also have to define the individual port themselves so I would have something like this:

port@0:
type: object
additionalProperties: false

properties:
reg:
const: 0
description: Slice 0 Port A

label:
description: Port name. Usually the pin group name

port@1:
type: object
additionalProperties: false

properties:
reg:
const: 1
description: Slice 0 Port B

label:
description: Port name. Usually the pin group name

port@2:
type: object
additionalProperties: false

properties:
reg:
const: 2
description: Slice 1 Port A

label:
description: Port name. Usually the pin group name

port@3:
type: object
additionalProperties: false

properties:
reg:
const: 3
description: Slice 1 Port B

label:
description: Port name. Usually the pin group name

And then in some way reference the patternProperties? Or just put it before the individual ports?

Quoted 116 lines
Quoted 116 lines
Documentation/devicetree/bindings/media/ti,vip.yaml new file +103 -0
+
+        required:
+          - reg
+          - label
+
+    required:
+      - "#address-cells"
+      - "#size-cells"
+      - port@0
+
+required:
+  - compatible
+  - label
+  - reg
+  - reg-names
+  - interrupts
+  - ti,vip-clk-polarity
+
+additionalProperties: false
+
+examples:
+  - |
+    #include <dt-bindings/interrupt-controller/arm-gic.h>
+
+    vip1: vip@48970000 {
+        compatible = "ti,dra7-vip";
+        label = "vip1";
+        reg = <0x48970000 0x114>,
+              <0x48975500 0xD8>,
+              <0x48975700 0x18>,
+              <0x48975800 0x80>,
+              <0x48975a00 0xD8>,
+              <0x48975c00 0x18>,
+              <0x48975d00 0x80>,
+              <0x4897d000 0x400>;
+        reg-names = "vip",
+                    "parser0",
+                    "csc0",
+                    "sc0",
+                    "parser1",
+                    "csc1",
+                    "sc1",
+                    "vpdma";
+        interrupts = <GIC_SPI 351 IRQ_TYPE_LEVEL_HIGH>,
+                     <GIC_SPI 392 IRQ_TYPE_LEVEL_HIGH>;
+        ti,vip-clk-polarity = <&scm_conf 0x534 0x1 0x4 0x2 0x8>;
+
+        ports {
+              #address-cells = <1>;
+              #size-cells = <0>;
+
+              vin1a: port@0 {
+                    reg = <0>;
+                    label = "vin1a";
+
+                    vin1a_ep: endpoint {
+                           remote-endpoint = <&camera1>;
+                           hsync-active = <1>;
+                           vsync-active = <1>;
+                           pclk-sample = <0>;
+                           bus-width = <8>;
+                    };
+              };
+              vin1b: port@1 {
+                    reg = <1>;
+                    label = "vin1b";
+              };
+              vin2a: port@2 {
+                    reg = <2>;
+                    label = "vin2a";
+              };
+              vin2b: port@3 {
+                    reg = <3>;
+                    label = "vin2b";
+              };
+         };
+    };
+
+    i2c {
+        clock-frequency = <400000>;
+        #address-cells = <1>;
+        #size-cells = <0>;
+
+         camera@37 {
+              compatible = "ovti,ov10633";
+              reg = <0x37>;
+
+              clocks = <&fixed_clock>;
+              clocks-names = "xvclk";
+
+              port {
+                   camera1: endpoint {
+                           remote-endpoint = <&vin1a_ep>;
+                           hsync-active = <1>;
+                           vsync-active = <1>;
+                           pclk-sample = <0>;
+                           bus-width = <8>;
+                   };
+              };
+         };
+    };
+
+...
MAINTAINERS +1 -0
index 2e9a5f6e4ff7..06856d05b53b 100644
@@ -16947,6 +16947,7 @@ S:	Maintained
 W:	http://linuxtv.org/
 Q:	http://patchwork.linuxtv.org/project/linux-media/list/
 F:	Documentation/devicetree/bindings/media/ti,cal.yaml
+F:	Documentation/devicetree/bindings/media/ti,vip.yaml
 F:	Documentation/devicetree/bindings/media/ti,vpe.yaml
 F:	drivers/media/platform/ti-vpe/
 
Quoted 3 lines
Quoted 3 lines
Signature · 1 line
2.17.1

Regards,
Benoit

#4 lore raw

PATCH 2/2 media: ti-vpe: Add the VIP driver

In reply to Benoit Parrot

VIP stands for Video Input Port, it can be found on devices such as DRA7xx and provides a parallel interface to a video source such as a sensor or TV decoder. Each VIP can support two inputs (slices) and a SoC can be configured with a variable number of VIP's. Each slice can supports two ports each connected to its own sub-device.

---
 drivers/media/platform/Kconfig         |   13 +
 drivers/media/platform/ti-vpe/Makefile |    2 +
 drivers/media/platform/ti-vpe/vip.c    | 4158 ++++++++++++++++++++++++
 drivers/media/platform/ti-vpe/vip.h    |  724 +++++
 4 files changed, 4897 insertions(+)
 create mode 100644 drivers/media/platform/ti-vpe/vip.c
 create mode 100644 drivers/media/platform/ti-vpe/vip.h
drivers/media/platform/Kconfig +13 -0
index c57ee78fa99d..f4100a1aad58 100644
@@ -168,6 +168,19 @@ config VIDEO_TI_CAL
 	  In TI Technical Reference Manual this module is referred as
 	  Camera Interface Subsystem (CAMSS).
 
+config VIDEO_TI_VIP
+	tristate "TI Video Input Port"
+	default n
+	depends on VIDEO_DEV && VIDEO_V4L2 && SOC_DRA7XX
+	depends on HAS_DMA
+	select VIDEOBUF2_DMA_CONTIG
+	select VIDEO_TI_VPDMA
+	select VIDEO_TI_SC
+	select VIDEO_TI_CSC
+	help
+	Driver support for VIP module on certain TI SoC's
+	VIP = Video Input Port.
+
 endif # V4L_PLATFORM_DRIVERS
 
 menuconfig V4L_MEM2MEM_DRIVERS
drivers/media/platform/ti-vpe/Makefile +2 -0
index 886ac5ec073f..cdbecadf7191 100644
@@ -3,11 +3,13 @@ obj-$(CONFIG_VIDEO_TI_VPE) += ti-vpe.o
 obj-$(CONFIG_VIDEO_TI_VPDMA) += ti-vpdma.o
 obj-$(CONFIG_VIDEO_TI_SC) += ti-sc.o
 obj-$(CONFIG_VIDEO_TI_CSC) += ti-csc.o
+obj-$(CONFIG_VIDEO_TI_VIP) += ti-vip.o
 
 ti-vpe-y := vpe.o
 ti-vpdma-y := vpdma.o
 ti-sc-y := sc.o
 ti-csc-y := csc.o
+ti-vip-y := vip.o
 
 ccflags-$(CONFIG_VIDEO_TI_VPE_DEBUG) += -DDEBUG
 
drivers/media/platform/ti-vpe/vip.c new file +4158 -0

2,968 lines not on this page · raw message

drivers/media/platform/ti-vpe/vip.h new file +724 -0

724 lines not on this page · raw message

The rest of this diff is not shown. View the raw message.

Signature · 1 line
2.17.1

Part of the diff is not shown here. View the raw message.

#5 lore raw
Quoted 121 lines · Benoit Parrot

VIP stands for Video Input Port, it can be found on devices such as DRA7xx and provides a parallel interface to a video source such as a sensor or TV decoder. Each VIP can support two inputs (slices) and a SoC can be configured with a variable number of VIP's. Each slice can supports two ports each connected to its own sub-device.

---
 drivers/media/platform/Kconfig         |   13 +
 drivers/media/platform/ti-vpe/Makefile |    2 +
 drivers/media/platform/ti-vpe/vip.c    | 4158 ++++++++++++++++++++++++
 drivers/media/platform/ti-vpe/vip.h    |  724 +++++
 4 files changed, 4897 insertions(+)
 create mode 100644 drivers/media/platform/ti-vpe/vip.c
 create mode 100644 drivers/media/platform/ti-vpe/vip.h
drivers/media/platform/Kconfig +13 -0
index c57ee78fa99d..f4100a1aad58 100644
@@ -168,6 +168,19 @@ config VIDEO_TI_CAL
 	  In TI Technical Reference Manual this module is referred as
 	  Camera Interface Subsystem (CAMSS).
 
+config VIDEO_TI_VIP
+	tristate "TI Video Input Port"
+	default n
+	depends on VIDEO_DEV && VIDEO_V4L2 && SOC_DRA7XX
+	depends on HAS_DMA
+	select VIDEOBUF2_DMA_CONTIG
+	select VIDEO_TI_VPDMA
+	select VIDEO_TI_SC
+	select VIDEO_TI_CSC
+	help
+	Driver support for VIP module on certain TI SoC's
+	VIP = Video Input Port.
+
 endif # V4L_PLATFORM_DRIVERS
 
 menuconfig V4L_MEM2MEM_DRIVERS
drivers/media/platform/ti-vpe/Makefile +2 -0
index 886ac5ec073f..cdbecadf7191 100644
@@ -3,11 +3,13 @@ obj-$(CONFIG_VIDEO_TI_VPE) += ti-vpe.o
 obj-$(CONFIG_VIDEO_TI_VPDMA) += ti-vpdma.o
 obj-$(CONFIG_VIDEO_TI_SC) += ti-sc.o
 obj-$(CONFIG_VIDEO_TI_CSC) += ti-csc.o
+obj-$(CONFIG_VIDEO_TI_VIP) += ti-vip.o
 
 ti-vpe-y := vpe.o
 ti-vpdma-y := vpdma.o
 ti-sc-y := sc.o
 ti-csc-y := csc.o
+ti-vip-y := vip.o
 
 ccflags-$(CONFIG_VIDEO_TI_VPE_DEBUG) += -DDEBUG
 
drivers/media/platform/ti-vpe/vip.c new file +55 -0
new file mode 100644
index 000000000000..307b01851a14
@@ -0,0 +1,4158 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * TI VIP capture driver
+ *
+ * Copyright (C) 2018 Texas Instruments Incorporated -  http://www.ti.com/
+ * David Griego, <[email protected]>
+ * Dale Farnsworth, <[email protected]>
+ * Nikhil Devshatwar, <[email protected]>
+ * Benoit Parrot, <[email protected]>
+ */
+
+#include <linux/clk.h>
+#include <linux/delay.h>
+#include <linux/dma-mapping.h>
+#include <linux/err.h>
+#include <linux/interrupt.h>
+#include <linux/module.h>
+#include <linux/workqueue.h>
+#include <linux/pm_runtime.h>
+#include <linux/sched.h>
+#include <linux/mfd/syscon.h>
+#include <linux/regmap.h>
+
+#include <linux/pinctrl/consumer.h>
+#include <linux/of_device.h>
+#include <linux/of_graph.h>
+
+#include "vip.h"
+
+#define VIP_MODULE_NAME "vip"
+
+static int debug;
+module_param(debug, int, 0644);
+MODULE_PARM_DESC(debug, "debug level (0-8)");
+
+/*
+ * Minimum and maximum frame sizes
+ */
+#define MIN_W		128
+#define MIN_H		128
+#define MAX_W		2048
+#define MAX_H		1536
+
+/*
+ * Required alignments
+ */
+#define S_ALIGN		0 /* multiple of 1 */
+#define H_ALIGN		1 /* multiple of 2 */
+#define W_ALIGN		1 /* multiple of 2 */
+#define L_ALIGN		7 /* multiple of 128, line stride, 16 bytes */
+
+/*
+ * Need a descriptor entry for each of up to 15 outputs,
+ * and up to 2 control transfers.
+ */

drivers/media/platform/ti-vpe/vip.c line 65

Quoted 10 lines
drivers/media/platform/ti-vpe/vip.c new file +10 -0
+#define VIP_DESC_LIST_SIZE	(17 * sizeof(struct vpdma_dtd))
+
+#define vip_dbg(level, dev, fmt, arg...)	\
+		v4l2_dbg(level, debug, dev, fmt, ##arg)
+#define vip_err(dev, fmt, arg...)	\
+		v4l2_err(dev, fmt, ##arg)
+#define vip_warn(dev, fmt, arg...)	\
+		v4l2_err(dev, fmt, ##arg)
+#define vip_info(dev, fmt, arg...)	\
+		v4l2_info(dev, fmt, ##arg)

These defines are just aliases. Any reason for not just using v4l2_info etc.?

Quoted 46 lines
drivers/media/platform/ti-vpe/vip.c new file +46 -0
+
+#define CTRL_CORE_SMA_SW_1      0x534
+/*
+ * The srce_info structure contains per-srce data.
+ */
+struct vip_srce_info {
+	u8	base_channel;	/* the VPDMA channel nummber */
+	u8	vb_index;	/* input frame f, f-1, f-2 index */
+	u8	vb_part;	/* identifies section of co-planar formats */
+};
+
+#define VIP_VPDMA_FIFO_SIZE	2
+#define VIP_DROPQ_SIZE		3
+
+/*
+ * Define indices into the srce_info tables
+ */
+
+#define VIP_SRCE_MULT_PORT		0
+#define VIP_SRCE_MULT_ANC		1
+#define VIP_SRCE_LUMA		2
+#define VIP_SRCE_CHROMA		3
+#define VIP_SRCE_RGB		4
+
+static struct vip_srce_info srce_info[5] = {
+	[VIP_SRCE_MULT_PORT] = {
+		.base_channel	= VIP1_CHAN_NUM_MULT_PORT_A_SRC0,
+		.vb_index	= 0,
+		.vb_part	= VIP_CHROMA,
+	},
+	[VIP_SRCE_MULT_ANC] = {
+		.base_channel	= VIP1_CHAN_NUM_MULT_ANC_A_SRC0,
+		.vb_index	= 0,
+		.vb_part	= VIP_LUMA,
+	},
+	[VIP_SRCE_LUMA] = {
+		.base_channel	= VIP1_CHAN_NUM_PORT_A_LUMA,
+		.vb_index	= 1,
+		.vb_part	= VIP_LUMA,
+	},
+	[VIP_SRCE_CHROMA] = {
+		.base_channel	= VIP1_CHAN_NUM_PORT_A_CHROMA,
+		.vb_index	= 1,
+		.vb_part	= VIP_CHROMA,
+	},
+	[VIP_SRCE_RGB] = {

drivers/media/platform/ti-vpe/vip.c line 121

Quoted 10 lines
drivers/media/platform/ti-vpe/vip.c new file +10 -0
+		.base_channel	= VIP1_CHAN_NUM_PORT_A_RGB,
+		.vb_part	= VIP_LUMA,
+	},
+};
+
+static struct vip_fmt vip_formats[VIP_MAX_ACTIVE_FMT] = {
+	{
+		.fourcc		= V4L2_PIX_FMT_NV12,
+		.code		= MEDIA_BUS_FMT_UYVY8_2X8,
+		.colorspace	= V4L2_COLORSPACE_SMPTE170M,

Colorspace information should come from the subdev (sensor or video receiver).

You really don't know what the colorspace is here.

Quoted 79 lines
drivers/media/platform/ti-vpe/vip.c new file +79 -0
+		.coplanar	= 1,
+		.vpdma_fmt	= { &vpdma_yuv_fmts[VPDMA_DATA_FMT_Y420],
+				    &vpdma_yuv_fmts[VPDMA_DATA_FMT_C420],
+				  },
+	},
+	{
+		.fourcc		= V4L2_PIX_FMT_UYVY,
+		.code		= MEDIA_BUS_FMT_UYVY8_2X8,
+		.colorspace	= V4L2_COLORSPACE_SMPTE170M,
+		.coplanar	= 0,
+		.vpdma_fmt	= { &vpdma_yuv_fmts[VPDMA_DATA_FMT_CBY422],
+				  },
+	},
+	{
+		.fourcc		= V4L2_PIX_FMT_YUYV,
+		.code		= MEDIA_BUS_FMT_UYVY8_2X8,
+		.colorspace	= V4L2_COLORSPACE_SMPTE170M,
+		.coplanar	= 0,
+		.vpdma_fmt	= { &vpdma_yuv_fmts[VPDMA_DATA_FMT_YCB422],
+				  },
+	},
+	{
+		.fourcc		= V4L2_PIX_FMT_VYUY,
+		.code		= MEDIA_BUS_FMT_UYVY8_2X8,
+		.colorspace	= V4L2_COLORSPACE_SMPTE170M,
+		.coplanar	= 0,
+		.vpdma_fmt	= { &vpdma_yuv_fmts[VPDMA_DATA_FMT_CRY422],
+				  },
+	},
+	{
+		.fourcc		= V4L2_PIX_FMT_YVYU,
+		.code		= MEDIA_BUS_FMT_UYVY8_2X8,
+		.colorspace	= V4L2_COLORSPACE_SMPTE170M,
+		.coplanar	= 0,
+		.vpdma_fmt	= { &vpdma_yuv_fmts[VPDMA_DATA_FMT_YCR422],
+				  },
+	},
+	{
+		.fourcc		= V4L2_PIX_FMT_RGB24,
+		.code		= MEDIA_BUS_FMT_UYVY8_2X8,
+		.colorspace	= V4L2_COLORSPACE_SRGB,
+		.coplanar	= 0,
+		.vpdma_fmt	= { &vpdma_rgb_fmts[VPDMA_DATA_FMT_RGB24],
+				  },
+	},
+	{
+		.fourcc		= V4L2_PIX_FMT_RGB32,
+		.code		= MEDIA_BUS_FMT_UYVY8_2X8,
+		.colorspace	= V4L2_COLORSPACE_SRGB,
+		.coplanar	= 0,
+		.vpdma_fmt	= { &vpdma_rgb_fmts[VPDMA_DATA_FMT_ARGB32],
+				  },
+	},
+	{
+		.fourcc		= V4L2_PIX_FMT_BGR24,
+		.code		= MEDIA_BUS_FMT_UYVY8_2X8,
+		.colorspace	= V4L2_COLORSPACE_SRGB,
+		.coplanar	= 0,
+		.vpdma_fmt	= { &vpdma_rgb_fmts[VPDMA_DATA_FMT_BGR24],
+				  },
+	},
+	{
+		.fourcc		= V4L2_PIX_FMT_BGR32,
+		.code		= MEDIA_BUS_FMT_UYVY8_2X8,
+		.colorspace	= V4L2_COLORSPACE_SRGB,
+		.coplanar	= 0,
+		.vpdma_fmt	= { &vpdma_rgb_fmts[VPDMA_DATA_FMT_ABGR32],
+				  },
+	},
+	{
+		.fourcc		= V4L2_PIX_FMT_RGB24,
+		.code		= MEDIA_BUS_FMT_RGB888_1X24,
+		.colorspace	= V4L2_COLORSPACE_SRGB,
+		.coplanar	= 0,
+		.vpdma_fmt	= { &vpdma_rgb_fmts[VPDMA_DATA_FMT_RGB24],
+				  },
+	},
+	{
+		.fourcc		= V4L2_PIX_FMT_RGB32,

drivers/media/platform/ti-vpe/vip.c line 210

Quoted 10 lines
drivers/media/platform/ti-vpe/vip.c new file +10 -0
+		.code		= MEDIA_BUS_FMT_ARGB8888_1X32,
+		.colorspace	= V4L2_COLORSPACE_SRGB,
+		.coplanar	= 0,
+		.vpdma_fmt	= { &vpdma_rgb_fmts[VPDMA_DATA_FMT_ARGB32],
+				  },
+	},
+	{
+		.fourcc		= V4L2_PIX_FMT_SBGGR8,
+		.code		= MEDIA_BUS_FMT_SBGGR8_1X8,
+		.colorspace	= V4L2_COLORSPACE_SMPTE170M,

SMPTE 170M for a Bayer format? That can't be right. But as mentioned above, this should come from the subdev.

Quoted 111 lines
drivers/media/platform/ti-vpe/vip.c new file +111 -0
+		.coplanar	= 0,
+		.vpdma_fmt	= { &vpdma_raw_fmts[VPDMA_DATA_FMT_RAW8],
+				  },
+	},
+	{
+		.fourcc		= V4L2_PIX_FMT_SGBRG8,
+		.code		= MEDIA_BUS_FMT_SGBRG8_1X8,
+		.colorspace	= V4L2_COLORSPACE_SMPTE170M,
+		.coplanar	= 0,
+		.vpdma_fmt	= { &vpdma_raw_fmts[VPDMA_DATA_FMT_RAW8],
+				  },
+	},
+	{
+		.fourcc		= V4L2_PIX_FMT_SGRBG8,
+		.code		= MEDIA_BUS_FMT_SGRBG8_1X8,
+		.colorspace	= V4L2_COLORSPACE_SMPTE170M,
+		.coplanar	= 0,
+		.vpdma_fmt	= { &vpdma_raw_fmts[VPDMA_DATA_FMT_RAW8],
+				  },
+	},
+	{
+		.fourcc		= V4L2_PIX_FMT_SRGGB8,
+		.code		= MEDIA_BUS_FMT_SRGGB8_1X8,
+		.colorspace	= V4L2_COLORSPACE_SMPTE170M,
+		.coplanar	= 0,
+		.vpdma_fmt	= { &vpdma_raw_fmts[VPDMA_DATA_FMT_RAW8],
+				  },
+	},
+	{
+		/* V4L2 currently only defines one 16 bit variant */
+		.fourcc		= V4L2_PIX_FMT_SBGGR16,
+		.code		= MEDIA_BUS_FMT_SBGGR16_1X16,
+		.colorspace	= V4L2_COLORSPACE_SMPTE170M,
+		.coplanar	= 0,
+		.vpdma_fmt	= { &vpdma_raw_fmts[VPDMA_DATA_FMT_RAW16],
+				  },
+	},
+};
+
+/* initialize  v4l2_format_info member in vip_formats array */
+static void vip_init_format_info(struct device *dev)
+{
+	struct vip_fmt *fmt;
+	int i;
+
+	for (i = 0; i < ARRAY_SIZE(vip_formats); i++) {
+		fmt = &vip_formats[i];
+		fmt->finfo = v4l2_format_info(fmt->fourcc);
+	}
+}
+
+/*  Print Four-character-code (FOURCC) */
+static char *fourcc_to_str(u32 fmt)
+{
+	static char code[5];
+
+	code[0] = (unsigned char)(fmt & 0xff);
+	code[1] = (unsigned char)((fmt >> 8) & 0xff);
+	code[2] = (unsigned char)((fmt >> 16) & 0xff);
+	code[3] = (unsigned char)((fmt >> 24) & 0xff);
+	code[4] = '\0';
+
+	return code;
+}
+
+/*
+ * Find our format description corresponding to the passed v4l2_format
+ */
+
+static struct vip_fmt *find_port_format_by_pix(struct vip_port *port,
+					       u32 pixelformat)
+{
+	struct vip_fmt *fmt;
+	unsigned int k;
+
+	for (k = 0; k < port->num_active_fmt; k++) {
+		fmt = port->active_fmt[k];
+		if (fmt->fourcc == pixelformat)
+			return fmt;
+	}
+
+	return NULL;
+}
+
+static struct vip_fmt *find_port_format_by_code(struct vip_port *port,
+						u32 code)
+{
+	struct vip_fmt *fmt;
+	unsigned int k;
+
+	for (k = 0; k < port->num_active_fmt; k++) {
+		fmt = port->active_fmt[k];
+		if (fmt->code == code)
+			return fmt;
+	}
+
+	return NULL;
+}
+
+static int vip_find_pad(struct v4l2_subdev *sd, int direction)
+{
+	unsigned int pad;
+
+	if (sd->entity.num_pads <= 1)
+		return 0;
+
+	for (pad = 0; pad < sd->entity.num_pads; pad++)
+		if (sd->entity.pads[pad].flags & direction)
+			return pad;
+
+	return -EINVAL;

drivers/media/platform/ti-vpe/vip.c line 331

Quoted 10 lines
drivers/media/platform/ti-vpe/vip.c new file +10 -0
+}
+
+inline struct vip_port *notifier_to_vip_port(struct v4l2_async_notifier *n)
+{
+	return container_of(n, struct vip_port, notifier);
+}
+
+static bool vip_is_mbuscode_yuv(u32 code)
+{
+	return ((code & 0xFF00) == 0x2000);

Use lower-case hex.

Quoted 713 lines
drivers/media/platform/ti-vpe/vip.c new file +713 -0
+}
+
+static bool vip_is_mbuscode_rgb(u32 code)
+{
+	return ((code & 0xFF00) == 0x1000);
+}
+
+static bool vip_is_mbuscode_raw(u32 code)
+{
+	return ((code & 0xFF00) == 0x3000);
+}
+
+/*
+ * This is not an accurate conversion but it is only used to
+ * assess if color conversion is needed.
+ */
+static u32 vip_mbus_code_to_fourcc(u32 code)
+{
+	if (vip_is_mbuscode_rgb(code))
+		return V4L2_PIX_FMT_RGB24;
+
+	if (vip_is_mbuscode_yuv(code))
+		return V4L2_PIX_FMT_UYVY;
+
+	return V4L2_PIX_FMT_SBGGR8;
+}
+
+static enum vip_csc_state
+vip_csc_direction(u32 src_code, const struct v4l2_format_info *dfinfo)
+{
+	if (vip_is_mbuscode_yuv(src_code) && v4l2_is_format_rgb(dfinfo))
+		return VIP_CSC_Y2R;
+	else if (vip_is_mbuscode_rgb(src_code) && v4l2_is_format_yuv(dfinfo))
+		return VIP_CSC_R2Y;
+	else
+		return VIP_CSC_NA;
+}
+
+/*
+ * port flag bits
+ */
+#define FLAG_FRAME_1D		BIT(0)
+#define FLAG_EVEN_LINE_SKIP	BIT(1)
+#define FLAG_ODD_LINE_SKIP	BIT(2)
+#define FLAG_MODE_TILED		BIT(3)
+#define FLAG_INTERLACED		BIT(4)
+#define FLAG_MULTIPLEXED	BIT(5)
+#define FLAG_MULT_PORT		BIT(6)
+#define FLAG_MULT_ANC		BIT(7)
+
+/*
+ * Function prototype declarations
+ */
+static int alloc_port(struct vip_dev *, int, const char *);
+static void free_port(struct vip_port *);
+static int vip_setup_parser(struct vip_port *port);
+static int vip_setup_scaler(struct vip_stream *stream);
+static void vip_enable_parser(struct vip_port *port, bool on);
+static void vip_reset_parser(struct vip_port *port, bool on);
+static void vip_parser_stop_imm(struct vip_port *port, bool on);
+static void stop_dma(struct vip_stream *stream, bool clear_list);
+static int vip_load_vpdma_list_fifo(struct vip_stream *stream);
+static inline bool is_scaler_available(struct vip_port *port);
+static inline bool allocate_scaler(struct vip_port *port);
+static inline void free_scaler(struct vip_port *port);
+static bool is_csc_available(struct vip_port *port);
+static bool allocate_csc(struct vip_port *port,
+				enum vip_csc_state csc_direction);
+static void free_csc(struct vip_port *port);
+
+#define reg_read(dev, offset) ioread32(dev->base + offset)
+#define reg_write(dev, offset, val) iowrite32(val, dev->base + offset)
+
+/*
+ * Insert a masked field into a 32-bit field
+ */
+static void insert_field(u32 *valp, u32 field, u32 mask, int shift)
+{
+	u32 val = *valp;
+
+	val &= ~(mask << shift);
+	val |= (field & mask) << shift;
+	*valp = val;
+}
+
+/*
+ * DMA address/data block for the shadow registers
+ */
+struct vip_mmr_adb {
+	struct vpdma_adb_hdr	sc_hdr0;
+	u32			sc_regs0[7];
+	u32			sc_pad0[1];
+	struct vpdma_adb_hdr	sc_hdr8;
+	u32			sc_regs8[6];
+	u32			sc_pad8[2];
+	struct vpdma_adb_hdr	sc_hdr17;
+	u32			sc_regs17[9];
+	u32			sc_pad17[3];
+	struct vpdma_adb_hdr	csc_hdr;
+	u32			csc_regs[6];
+	u32			csc_pad[2];
+};
+
+#define GET_OFFSET_TOP(port, obj, reg)	\
+	((obj)->res->start - port->dev->res->start + reg)
+
+#define VIP_SET_MMR_ADB_HDR(port, hdr, regs, offset_a)	\
+	VPDMA_SET_MMR_ADB_HDR(port->mmr_adb, vip_mmr_adb, hdr, regs, offset_a)
+
+/*
+ * Set the headers for all of the address/data block structures.
+ */
+static void init_adb_hdrs(struct vip_port *port)
+{
+	VIP_SET_MMR_ADB_HDR(port, sc_hdr0, sc_regs0,
+			    GET_OFFSET_TOP(port, port->dev->sc, CFG_SC0));
+	VIP_SET_MMR_ADB_HDR(port, sc_hdr8, sc_regs8,
+			    GET_OFFSET_TOP(port, port->dev->sc, CFG_SC8));
+	VIP_SET_MMR_ADB_HDR(port, sc_hdr17, sc_regs17,
+			    GET_OFFSET_TOP(port, port->dev->sc, CFG_SC17));
+	VIP_SET_MMR_ADB_HDR(port, csc_hdr, csc_regs,
+			    GET_OFFSET_TOP(port, port->dev->csc, CSC_CSC00));
+
+};
+
+/*
+ * These represent the module resets bit for slice 1
+ * Upon detecting slice2 we simply left shift by 1
+ */
+#define VIP_DP_RST	BIT(16)
+#define VIP_PARSER_RST	BIT(18)
+#define VIP_CSC_RST	BIT(20)
+#define VIP_SC_RST	BIT(22)
+#define VIP_DS0_RST	BIT(25)
+#define VIP_DS1_RST	BIT(27)
+
+static void vip_module_reset(struct vip_dev *dev, uint32_t module, bool on)
+{
+	u32 val = 0;
+
+	val = reg_read(dev, VIP_CLK_RESET);
+
+	if (dev->slice_id == VIP_SLICE2)
+		module <<= 1;
+
+	if (on)
+		val |= module;
+	else
+		val &= ~module;
+
+	reg_write(dev, VIP_CLK_RESET, val);
+}
+
+/*
+ * Enable or disable the VIP clocks
+ */
+static void vip_set_clock_enable(struct vip_dev *dev, bool on)
+{
+	u32 val = 0;
+
+	val = reg_read(dev, VIP_CLK_ENABLE);
+	if (on) {
+		val |= VIP_VPDMA_CLK_ENABLE;
+		if (dev->slice_id == VIP_SLICE1)
+			val |= VIP_VIP1_DATA_PATH_CLK_ENABLE;
+		else
+			val |= VIP_VIP2_DATA_PATH_CLK_ENABLE;
+	} else {
+		if (dev->slice_id == VIP_SLICE1)
+			val &= ~VIP_VIP1_DATA_PATH_CLK_ENABLE;
+		else
+			val &= ~VIP_VIP2_DATA_PATH_CLK_ENABLE;
+
+		/* Both VIP are disabled then shutdown VPDMA also */
+		if (!(val & (VIP_VIP1_DATA_PATH_CLK_ENABLE |
+			     VIP_VIP2_DATA_PATH_CLK_ENABLE)))
+			val = 0;
+	}
+
+	reg_write(dev, VIP_CLK_ENABLE, val);
+}
+
+/* This helper function is used to enable the clock early on to
+ * enable vpdma firmware loading before the slice device are created
+ */
+static void vip_shared_set_clock_enable(struct vip_shared *shared, bool on)
+{
+	u32 val = 0;
+
+	if (on)
+		val = VIP_VIP1_DATA_PATH_CLK_ENABLE | VIP_VPDMA_CLK_ENABLE;
+
+	reg_write(shared, VIP_CLK_ENABLE, val);
+}
+
+static void vip_top_reset(struct vip_dev *dev)
+{
+	u32 val = 0;
+
+	val = reg_read(dev, VIP_CLK_RESET);
+
+	if (dev->slice_id == VIP_SLICE1)
+		insert_field(&val, 1, VIP_DATA_PATH_CLK_RESET_MASK,
+			     VIP_VIP1_DATA_PATH_RESET_SHIFT);
+	else
+		insert_field(&val, 1, VIP_DATA_PATH_CLK_RESET_MASK,
+			     VIP_VIP2_DATA_PATH_RESET_SHIFT);
+
+	reg_write(dev, VIP_CLK_RESET, val);
+
+	usleep_range(200, 250);
+
+	val = reg_read(dev, VIP_CLK_RESET);
+
+	if (dev->slice_id == VIP_SLICE1)
+		insert_field(&val, 0, VIP_DATA_PATH_CLK_RESET_MASK,
+			     VIP_VIP1_DATA_PATH_RESET_SHIFT);
+	else
+		insert_field(&val, 0, VIP_DATA_PATH_CLK_RESET_MASK,
+			     VIP_VIP2_DATA_PATH_RESET_SHIFT);
+	reg_write(dev, VIP_CLK_RESET, val);
+}
+
+static void vip_top_vpdma_reset(struct vip_shared *shared)
+{
+	u32 val;
+
+	val = reg_read(shared, VIP_CLK_RESET);
+	insert_field(&val, 1, VIP_VPDMA_CLK_RESET_MASK,
+		     VIP_VPDMA_CLK_RESET_SHIFT);
+	reg_write(shared, VIP_CLK_RESET, val);
+
+	usleep_range(200, 250);
+
+	val = reg_read(shared, VIP_CLK_RESET);
+	insert_field(&val, 0, VIP_VPDMA_CLK_RESET_MASK,
+		     VIP_VPDMA_CLK_RESET_SHIFT);
+	reg_write(shared, VIP_CLK_RESET, val);
+}
+
+static void vip_set_pclk_invert(struct vip_port *port)
+{
+	struct vip_clk_polarity *pclk = port->dev->pclk_pol;
+	u32 index;
+
+	/*
+	 * When the VIP parser is configured to so that the pixel clock
+	 * is to be sampled at falling edge, the pixel clock needs to be
+	 * inverted before it is given to the VIP module. This is done
+	 * by setting a bit in the CTRL_CORE_SMA_SW1 register.
+	 */
+
+	index = 2 * port->dev->slice_id + port->port_id;
+	vip_dbg(3, port, "%s: slice%d:port%d -> index: %d\n", __func__,
+		port->dev->slice_id, port->port_id, index);
+
+	if (pclk->rm_pol)
+		regmap_update_bits(pclk->rm_pol,
+				   pclk->rm_offset,
+				   pclk->rm_bit_field[index],
+				   pclk->rm_bit_field[index]);
+}
+
+static void vip_clr_pclk_invert(struct vip_port *port)
+{
+	struct vip_clk_polarity *pclk = port->dev->pclk_pol;
+	u32 index;
+
+	index = 2 * port->dev->slice_id + port->port_id;
+	vip_dbg(3, port, "%s: slice%d:port%d -> index: %d\n", __func__,
+		port->dev->slice_id, port->port_id, index);
+
+	if (pclk->rm_pol)
+		regmap_update_bits(pclk->rm_pol, pclk->rm_offset,
+				   pclk->rm_bit_field[index], 0);
+}
+
+#define VIP_PARSER_PORT(p)	(VIP_PARSER_PORTA_0 + (p * 0x8U))
+#define VIP_PARSER_EXTRA_PORT(p)	(VIP_PARSER_PORTA_1 + (p * 0x8U))
+#define VIP_PARSER_CROP_H_PORT(p)	(VIP_PARSER_PORTA_EXTRA4 + (p * 0x10U))
+#define VIP_PARSER_CROP_V_PORT(p)	(VIP_PARSER_PORTA_EXTRA5 + (p * 0x10U))
+#define VIP_PARSER_STOP_IMM_PORT(p)	(VIP_PARSER_PORTA_EXTRA6 + (p * 0x4U))
+
+static void vip_set_data_interface(struct vip_port *port,
+				   enum data_interface_modes mode)
+{
+	u32 val = 0;
+
+	insert_field(&val, mode, VIP_DATA_INTERFACE_MODE_MASK,
+		     VIP_DATA_INTERFACE_MODE_SHFT);
+
+	reg_write(port->dev->parser, VIP_PARSER_MAIN_CFG, val);
+}
+
+static void vip_set_slice_path(struct vip_dev *dev,
+			       enum data_path_select data_path, u32 path_val)
+{
+	u32 val = 0;
+	int data_path_reg;
+
+	vip_dbg(3, dev, "%s:\n", __func__);
+
+	data_path_reg = VIP_VIP1_DATA_PATH_SELECT + 4 * dev->slice_id;
+
+	switch (data_path) {
+	case ALL_FIELDS_DATA_SELECT:
+		val |= path_val;
+		break;
+	case VIP_CSC_SRC_DATA_SELECT:
+		insert_field(&val, path_val, VIP_CSC_SRC_SELECT_MASK,
+			     VIP_CSC_SRC_SELECT_SHFT);
+		break;
+	case VIP_SC_SRC_DATA_SELECT:
+		insert_field(&val, path_val, VIP_SC_SRC_SELECT_MASK,
+			     VIP_SC_SRC_SELECT_SHFT);
+		break;
+	case VIP_RGB_SRC_DATA_SELECT:
+		val |= (path_val) ? VIP_RGB_SRC_SELECT : 0;
+		break;
+	case VIP_RGB_OUT_LO_DATA_SELECT:
+		val |= (path_val) ? VIP_RGB_OUT_LO_SRC_SELECT : 0;
+		break;
+	case VIP_RGB_OUT_HI_DATA_SELECT:
+		val |= (path_val) ? VIP_RGB_OUT_HI_SRC_SELECT : 0;
+		break;
+	case VIP_CHR_DS_1_SRC_DATA_SELECT:
+		insert_field(&val, path_val, VIP_DS1_SRC_SELECT_MASK,
+			     VIP_DS1_SRC_SELECT_SHFT);
+		break;
+	case VIP_CHR_DS_2_SRC_DATA_SELECT:
+		insert_field(&val, path_val, VIP_DS2_SRC_SELECT_MASK,
+			     VIP_DS2_SRC_SELECT_SHFT);
+		break;
+	case VIP_MULTI_CHANNEL_DATA_SELECT:
+		val |= (path_val) ? VIP_MULTI_CHANNEL_SELECT : 0;
+		break;
+	case VIP_CHR_DS_1_DATA_BYPASS:
+		val |= (path_val) ? VIP_DS1_BYPASS : 0;
+		break;
+	case VIP_CHR_DS_2_DATA_BYPASS:
+		val |= (path_val) ? VIP_DS2_BYPASS : 0;
+		break;
+	default:
+		vip_err(dev, "%s: data_path 0x%x is not valid\n",
+			__func__, data_path);
+		return;
+	}
+	insert_field(&val, data_path, VIP_DATAPATH_SELECT_MASK,
+		     VIP_DATAPATH_SELECT_SHFT);
+	reg_write(dev, data_path_reg, val);
+	vip_dbg(3, dev, "%s: DATA_PATH_SELECT(%08X): %08X\n", __func__,
+		data_path_reg, reg_read(dev, data_path_reg));
+}
+
+/*
+ * Return the vip_stream structure for a given struct file
+ */
+static inline struct vip_stream *file2stream(struct file *file)
+{
+	return video_drvdata(file);
+}
+
+/*
+ * Append a destination descriptor to the current descriptor list,
+ * setting up dma to the given srce.
+ */
+static int add_out_dtd(struct vip_stream *stream, int srce_type)
+{
+	struct vip_port *port = stream->port;
+	struct vip_dev *dev = port->dev;
+	struct vip_srce_info *sinfo = &srce_info[srce_type];
+	struct v4l2_rect *c_rect = &port->c_rect;
+	struct vip_fmt *fmt = port->fmt;
+	int channel, plane = 0;
+	int max_width, max_height;
+	dma_addr_t dma_addr;
+	u32 flags;
+	u32 width = stream->width;
+
+	channel = sinfo->base_channel;
+
+	switch (srce_type) {
+	case VIP_SRCE_MULT_PORT:
+	case VIP_SRCE_MULT_ANC:
+		if (port->port_id == VIP_PORTB)
+			channel += VIP_CHAN_MULT_PORTB_OFFSET;
+		channel += stream->stream_id;
+		flags = 0;
+		break;
+	case VIP_SRCE_CHROMA:
+		plane = 1;
+		/* fallthrough */
+	case VIP_SRCE_LUMA:
+		if (port->port_id == VIP_PORTB) {
+			if (port->scaler && !port->fmt->coplanar)
+				/*
+				 * In this case Port A Chroma channel
+				 * is used to carry Port B scaled YUV422
+				 */
+				channel += 1;
+			else
+				channel += VIP_CHAN_YUV_PORTB_OFFSET;
+		}
+		flags = port->flags;
+		break;
+	case VIP_SRCE_RGB:
+		if ((port->port_id == VIP_PORTB) ||
+		    ((port->port_id == VIP_PORTA) &&
+		     (port->csc == VIP_CSC_NA) &&
+		     v4l2_is_format_rgb(port->fmt->finfo)))
+			/*
+			 * RGB sensor only connect to Y_LO
+			 * channel i.e. port B channel.
+			 */
+			channel += VIP_CHAN_RGB_PORTB_OFFSET;
+		flags = port->flags;
+		break;
+	default:
+		vip_err(stream, "%s: srce_type 0x%x is not valid\n",
+			__func__, srce_type);
+		return -1;
+	}
+
+	if (dev->slice_id == VIP_SLICE2)
+		channel += VIP_CHAN_VIP2_OFFSET;
+
+	/* This is just for initialization purposes.
+	 * The actual dma_addr will be configured in vpdma_update_dma_addr
+	 */
+	dma_addr = 0;
+
+	if (port->fmt->vpdma_fmt[0] == &vpdma_raw_fmts[VPDMA_DATA_FMT_RAW8]) {
+		/*
+		 * Special case since we are faking a YUV422 16bit format
+		 * to have the vpdma perform the needed byte swap
+		 * we need to adjust the pixel width accordingly
+		 * otherwise the parser will attempt to collect more pixels
+		 * then available and the vpdma transfer will exceed the
+		 * allocated frame buffer.
+		 */
+		width >>= 1;
+		vip_dbg(1, stream, "%s: 8 bit raw detected, adjusting width to %d\n",
+			__func__, width);
+	}
+
+	/*
+	 * Use VPDMA_MAX_SIZE1 or VPDMA_MAX_SIZE2 register for slice0/1
+	 */
+
+	if (dev->slice_id == VIP_SLICE1) {
+		vpdma_set_max_size(dev->shared->vpdma, VPDMA_MAX_SIZE1,
+				   width, stream->height);
+
+		max_width = MAX_OUT_WIDTH_REG1;
+		max_height = MAX_OUT_HEIGHT_REG1;
+	} else {
+		vpdma_set_max_size(dev->shared->vpdma, VPDMA_MAX_SIZE2,
+				   width, stream->height);
+
+		max_width = MAX_OUT_WIDTH_REG2;
+		max_height = MAX_OUT_HEIGHT_REG2;
+	}
+
+	/*
+	 * Mark this channel to be cleared while cleaning up resources
+	 * This will make sure that an abort descriptor for this channel
+	 * would be submitted to VPDMA causing any ongoing  transaction to be
+	 * aborted and cleanup the VPDMA FSM for this channel
+	 */
+	stream->vpdma_channels[channel] = 1;
+
+	vpdma_rawchan_add_out_dtd(&stream->desc_list, c_rect->width,
+				  stream->bytesperline, c_rect,
+				  fmt->vpdma_fmt[plane], dma_addr,
+				  max_width, max_height, channel, flags);
+
+	return 0;
+}
+
+/*
+ * add_stream_dtds - prepares and starts DMA for pending transfers
+ */
+static void add_stream_dtds(struct vip_stream *stream)
+{
+	struct vip_port *port = stream->port;
+	int srce_type;
+
+	if (port->flags & FLAG_MULT_PORT)
+		srce_type = VIP_SRCE_MULT_PORT;
+	else if (port->flags & FLAG_MULT_ANC)
+		srce_type = VIP_SRCE_MULT_ANC;
+	else if (v4l2_is_format_rgb(port->fmt->finfo))
+		srce_type = VIP_SRCE_RGB;
+	else
+		srce_type = VIP_SRCE_LUMA;
+
+	add_out_dtd(stream, srce_type);
+
+	if (srce_type == VIP_SRCE_LUMA && port->fmt->coplanar)
+		add_out_dtd(stream, VIP_SRCE_CHROMA);
+}
+
+#define PARSER_IRQ_MASK (VIP_PORTA_OUTPUT_FIFO_YUV | \
+			 VIP_PORTB_OUTPUT_FIFO_YUV)
+
+static void enable_irqs(struct vip_dev *dev, int irq_num, int list_num)
+{
+	struct vip_parser_data *parser = dev->parser;
+	u32 reg_addr = VIP_INT0_ENABLE0_SET +
+			VIP_INTC_INTX_OFFSET * irq_num;
+	u32 irq_val = (1 << (list_num * 2)) |
+		      (VIP_VIP1_PARSER_INT << (irq_num * 1));
+
+	/* Enable Parser Interrupt */
+	reg_write(parser, VIP_PARSER_FIQ_MASK, ~PARSER_IRQ_MASK);
+
+	reg_write(dev->shared, reg_addr, irq_val);
+
+	vpdma_enable_list_complete_irq(dev->shared->vpdma,
+				       irq_num, list_num, true);
+}
+
+static void disable_irqs(struct vip_dev *dev, int irq_num, int list_num)
+{
+	struct vip_parser_data *parser = dev->parser;
+	u32 reg_addr = VIP_INT0_ENABLE0_CLR +
+			VIP_INTC_INTX_OFFSET * irq_num;
+	u32 irq_val = (1 << (list_num * 2)) |
+		      (VIP_VIP1_PARSER_INT << (irq_num * 1));
+
+	/* Disable all Parser Interrupt */
+	reg_write(parser, VIP_PARSER_FIQ_MASK, 0xffffffff);
+
+	reg_write(dev->shared, reg_addr, irq_val);
+
+	vpdma_enable_list_complete_irq(dev->shared->vpdma,
+				       irq_num, list_num, false);
+}
+
+static void clear_irqs(struct vip_dev *dev, int irq_num, int list_num)
+{
+	struct vip_parser_data *parser = dev->parser;
+	u32 reg_addr = VIP_INT0_STATUS0_CLR +
+			VIP_INTC_INTX_OFFSET * irq_num;
+	u32 irq_val = (1 << (list_num * 2)) |
+		      (VIP_VIP1_PARSER_INT << (irq_num * 1));
+
+	/* Clear all Parser Interrupt */
+	reg_write(parser, VIP_PARSER_FIQ_CLR, 0xffffffff);
+	reg_write(parser, VIP_PARSER_FIQ_CLR, 0x0);
+
+	reg_write(dev->shared, reg_addr, irq_val);
+
+	vpdma_clear_list_stat(dev->shared->vpdma, irq_num, dev->slice_id);
+}
+
+static void populate_desc_list(struct vip_stream *stream)
+{
+	struct vip_port *port = stream->port;
+	struct vip_dev *dev = port->dev;
+	unsigned int list_length;
+
+	stream->desc_next = stream->desc_list.buf.addr;
+	add_stream_dtds(stream);
+
+	list_length = stream->desc_next - stream->desc_list.buf.addr;
+	vpdma_map_desc_buf(dev->shared->vpdma, &stream->desc_list.buf);
+}
+
+/*
+ * start_dma - adds descriptors to the dma list and submits them.
+ * Should be called after a new vb is queued and on a vpdma list
+ * completion interrupt.
+ */
+static void start_dma(struct vip_stream *stream, struct vip_buffer *buf)
+{
+	struct vip_dev *dev = stream->port->dev;
+	struct vpdma_data *vpdma = dev->shared->vpdma;
+	int list_num = stream->list_num;
+	dma_addr_t dma_addr;
+	int drop_data;
+
+	if (vpdma_list_busy(vpdma, list_num)) {
+		vip_err(stream, "vpdma list busy, cannot post\n");
+		return;				/* nothing to do */
+	}
+
+	if (buf) {
+		dma_addr = vb2_dma_contig_plane_dma_addr(&buf->vb.vb2_buf, 0);
+		drop_data = 0;
+		vip_dbg(4, stream, "%s: vb2 buf idx:%d, dma_addr:%pad\n",
+			__func__, buf->vb.vb2_buf.index, &dma_addr);
+	} else {
+		dma_addr = 0;
+		drop_data = 1;
+		vip_dbg(4, stream, "%s: dropped\n", __func__);
+	}
+
+	vpdma_update_dma_addr(dev->shared->vpdma, &stream->desc_list,
+			      dma_addr, stream->write_desc, drop_data, 0);
+
+	if (stream->port->fmt->coplanar) {
+		dma_addr += stream->bytesperline * stream->height;
+		vpdma_update_dma_addr(dev->shared->vpdma, &stream->desc_list,
+				      dma_addr, stream->write_desc + 1,
+				      drop_data, 1);
+	}
+
+	vpdma_submit_descs(dev->shared->vpdma,
+			   &stream->desc_list, stream->list_num);
+}
+
+static void vip_schedule_next_buffer(struct vip_stream *stream)
+{
+	struct vip_dev *dev = stream->port->dev;
+	struct vip_buffer *buf;
+	unsigned long flags;
+
+	spin_lock_irqsave(&dev->slock, flags);
+	if (list_empty(&stream->vidq)) {
+		vip_dbg(4, stream, "Dropping frame\n");
+		if (list_empty(&stream->dropq)) {
+			vip_err(stream, "No dropq buffer left!");
+			spin_unlock_irqrestore(&dev->slock, flags);
+			return;
+		}
+		buf = list_entry(stream->dropq.next,
+				 struct vip_buffer, list);
+
+		buf->drop = true;
+		list_move_tail(&buf->list, &stream->post_bufs);
+		buf = NULL;
+	} else {
+		buf = list_entry(stream->vidq.next,
+				 struct vip_buffer, list);
+		buf->drop = false;
+		list_move_tail(&buf->list, &stream->post_bufs);
+		vip_dbg(4, stream, "added next buffer\n");
+	}
+
+	spin_unlock_irqrestore(&dev->slock, flags);
+	start_dma(stream, buf);
+}
+
+static void vip_process_buffer_complete(struct vip_stream *stream)
+{
+	struct vip_dev *dev = stream->port->dev;
+	struct vip_buffer *buf;
+	struct vb2_v4l2_buffer *vb = NULL;
+	unsigned long flags, fld;
+
+	buf = list_first_entry(&stream->post_bufs, struct vip_buffer, list);
+
+	if (stream->port->flags & FLAG_INTERLACED) {
+		vpdma_unmap_desc_buf(dev->shared->vpdma,
+				     &stream->desc_list.buf);
+
+		fld = dtd_get_field(stream->write_desc);
+		stream->field = fld ? V4L2_FIELD_BOTTOM : V4L2_FIELD_TOP;
+
+		vpdma_map_desc_buf(dev->shared->vpdma, &stream->desc_list.buf);
+	}
+
+	if (buf) {
+		vip_dbg(4, stream, "vip buffer complete 0x%x, 0x%x\n",
+			(unsigned int)buf, buf->drop);
+
+		vb = &buf->vb;
+		vb->field = stream->field;
+		vb->sequence = stream->sequence;
+		vb->vb2_buf.timestamp = ktime_get_ns();
+
+		if (buf->drop) {
+			spin_lock_irqsave(&dev->slock, flags);
+			list_move_tail(&buf->list, &stream->dropq);
+			spin_unlock_irqrestore(&dev->slock, flags);
+		} else {
+			spin_lock_irqsave(&dev->slock, flags);
+			list_del(&buf->list);
+			spin_unlock_irqrestore(&dev->slock, flags);
+			vb2_buffer_done(&vb->vb2_buf, VB2_BUF_STATE_DONE);
+		}
+	} else {
+		vip_err(stream, "%s: buf is null!!!\n", __func__);
+		return;
+	}
+
+	stream->sequence++;
+}
+
+static int vip_reset_vpdma(struct vip_stream *stream)
+{
+	struct vip_port *port = stream->port;
+	struct vip_dev *dev = port->dev;
+	struct vip_buffer *buf;
+	unsigned long flags;
+
+	stop_dma(stream, false);
+
+	spin_lock_irqsave(&dev->slock, flags);
+	/* requeue all active buffers in the opposite order */
+	while (!list_empty(&stream->post_bufs)) {
+		buf = list_last_entry(&stream->post_bufs,
+				      struct vip_buffer, list);
+		list_del(&buf->list);
+		if (buf->drop == 1) {
+			list_add_tail(&buf->list, &stream->dropq);
+			vip_dbg(4, stream, "requeueing drop buffer on dropq\n");
+		} else {
+			list_add(&buf->list, &stream->vidq);
+			vip_dbg(4, stream, "requeueing vb2 buf idx:%d on vidq\n",
+				buf->vb.vb2_buf.index);
+		}

drivers/media/platform/ti-vpe/vip.c line 1054

Quoted 10 lines
drivers/media/platform/ti-vpe/vip.c new file +10 -0
+	}
+	spin_unlock_irqrestore(&dev->slock, flags);
+
+	/* Make sure the desc_list is unmapped */
+	vpdma_unmap_desc_buf(dev->shared->vpdma, &stream->desc_list.buf);
+
+	return 0;
+}
+
+static void vip_overflow_recovery_work(struct work_struct *work)

This function requires some additional comments about when it should be called and what it does. Recovery is always tricky, and so it is worthwhile spending additional time on commenting it.

Quoted 298 lines
drivers/media/platform/ti-vpe/vip.c new file +298 -0
+{
+	struct vip_stream *stream = container_of(work, struct vip_stream,
+						 recovery_work);
+	struct vip_port *port = stream->port;
+	struct vip_dev *dev = port->dev;
+
+	vip_err(stream, "%s: Port %c\n", __func__,
+		port->port_id == VIP_PORTA ? 'A' : 'B');
+
+	disable_irqs(dev, dev->slice_id, stream->list_num);
+	clear_irqs(dev, dev->slice_id, stream->list_num);
+
+	/* 1.	Set VIP_XTRA6_PORT_A[31:16] YUV_SRCNUM_STOP_IMMEDIATELY */
+	/* 2.	Set VIP_XTRA6_PORT_A[15:0] ANC_SRCNUM_STOP_IMMEDIATELY */
+	vip_parser_stop_imm(port, 1);
+
+	/* 3.	Clear VIP_PORT_A[8] ENABLE */
+	/*
+	 * 4.	Set VIP_PORT_A[7] CLR_ASYNC_FIFO_RD
+	 *      Set VIP_PORT_A[6] CLR_ASYNC_FIFO_WR
+	 */
+	vip_enable_parser(port, false);
+
+	/* 5.	Set VIP_PORT_A[23] SW_RESET */
+	vip_reset_parser(port, 1);
+
+	/*
+	 * 6.	Reset other VIP modules
+	 *	For each module used downstream of VIP_PARSER, write 1 to the
+	 *      bit location of the VIP_CLKC_RST register which is connected
+	 *      to VIP_PARSER
+	 */
+	vip_module_reset(dev, VIP_DP_RST, true);
+
+	usleep_range(200, 250);
+
+	/*
+	 * 7.	Abort VPDMA channels
+	 *	Write to list attribute to stop list 0
+	 *	Write to list address register location of abort list
+	 *	Write to list attribute register list 0 and size of abort list
+	 */
+	vip_reset_vpdma(stream);
+
+	/* 8.	Clear VIP_PORT_A[23] SW_RESET */
+	vip_reset_parser(port, 0);
+
+	/*
+	 * 9.	Un-reset other VIP modules
+	 *	For each module used downstream of VIP_PARSER, write 0 to
+	 *	the bit location of the VIP_CLKC_RST register which is
+	 *	connected to VIP_PARSER
+	 */
+	vip_module_reset(dev, VIP_DP_RST, false);
+
+	/* 10.	(Delay) */
+	/* 11.	SC coeff downloaded (if VIP_SCALER is being used) */
+	vip_setup_scaler(stream);
+
+	/* 12.	(Delay) */
+		/* the above are not needed here yet */
+
+	populate_desc_list(stream);
+	stream->num_recovery++;
+	if (stream->num_recovery < 5) {
+		/* Reload the vpdma */
+		vip_load_vpdma_list_fifo(stream);
+
+		enable_irqs(dev, dev->slice_id, stream->list_num);
+		vip_schedule_next_buffer(stream);
+
+		/* 13.	Clear VIP_XTRA6_PORT_A[31:16] YUV_SRCNUM_STOP_IMM */
+		/* 14.	Clear VIP_XTRA6_PORT_A[15:0] ANC_SRCNUM_STOP_IMM */
+
+		vip_parser_stop_imm(port, 0);
+
+		/* 15.	Set VIP_PORT_A[8] ENABLE */
+		/*
+		 * 16.	Clear VIP_PORT_A[7] CLR_ASYNC_FIFO_RD
+		 *	Clear VIP_PORT_A[6] CLR_ASYNC_FIFO_WR
+		 */
+		vip_enable_parser(port, true);
+	} else {
+		vip_err(stream, "%s: num_recovery limit exceeded leaving disabled\n",
+			__func__);
+	}
+}
+
+static void handle_parser_irqs(struct vip_dev *dev)
+{
+	struct vip_parser_data *parser = dev->parser;
+	struct vip_port *porta = dev->ports[VIP_PORTA];
+	struct vip_port *portb = dev->ports[VIP_PORTB];
+	struct vip_stream *stream = NULL;
+	u32 irq_stat = reg_read(parser, VIP_PARSER_FIQ_STATUS);
+	int i;
+
+	vip_dbg(3, dev, "%s: FIQ_STATUS: 0x%08x\n", __func__, irq_stat);
+
+	/* Clear all Parser Interrupt */
+	reg_write(parser, VIP_PARSER_FIQ_CLR, irq_stat);
+	reg_write(parser, VIP_PARSER_FIQ_CLR, 0x0);
+
+	if (irq_stat & VIP_PORTA_VDET)
+		vip_dbg(3, dev, "VIP_PORTA_VDET\n");
+	if (irq_stat & VIP_PORTB_VDET)
+		vip_dbg(3, dev, "VIP_PORTB_VDET\n");
+	if (irq_stat & VIP_PORTA_ASYNC_FIFO_OF)
+		vip_err(dev, "VIP_PORTA_ASYNC_FIFO_OF\n");
+	if (irq_stat & VIP_PORTB_ASYNC_FIFO_OF)
+		vip_err(dev, "VIP_PORTB_ASYNC_FIFO_OF\n");
+	if (irq_stat & VIP_PORTA_OUTPUT_FIFO_YUV)
+		vip_err(dev, "VIP_PORTA_OUTPUT_FIFO_YUV\n");
+	if (irq_stat & VIP_PORTA_OUTPUT_FIFO_ANC)
+		vip_err(dev, "VIP_PORTA_OUTPUT_FIFO_ANC\n");
+	if (irq_stat & VIP_PORTB_OUTPUT_FIFO_YUV)
+		vip_err(dev, "VIP_PORTB_OUTPUT_FIFO_YUV\n");
+	if (irq_stat & VIP_PORTB_OUTPUT_FIFO_ANC)
+		vip_err(dev, "VIP_PORTB_OUTPUT_FIFO_ANC\n");
+	if (irq_stat & VIP_PORTA_CONN)
+		vip_dbg(3, dev, "VIP_PORTA_CONN\n");
+	if (irq_stat & VIP_PORTA_DISCONN)
+		vip_dbg(3, dev, "VIP_PORTA_DISCONN\n");
+	if (irq_stat & VIP_PORTB_CONN)
+		vip_dbg(3, dev, "VIP_PORTB_CONN\n");
+	if (irq_stat & VIP_PORTB_DISCONN)
+		vip_dbg(3, dev, "VIP_PORTB_DISCONN\n");
+	if (irq_stat & VIP_PORTA_SRC0_SIZE)
+		vip_dbg(3, dev, "VIP_PORTA_SRC0_SIZE\n");
+	if (irq_stat & VIP_PORTB_SRC0_SIZE)
+		vip_dbg(3, dev, "VIP_PORTB_SRC0_SIZE\n");
+	if (irq_stat & VIP_PORTA_YUV_PROTO_VIOLATION)
+		vip_dbg(3, dev, "VIP_PORTA_YUV_PROTO_VIOLATION\n");
+	if (irq_stat & VIP_PORTA_ANC_PROTO_VIOLATION)
+		vip_dbg(3, dev, "VIP_PORTA_ANC_PROTO_VIOLATION\n");
+	if (irq_stat & VIP_PORTB_YUV_PROTO_VIOLATION)
+		vip_dbg(3, dev, "VIP_PORTB_YUV_PROTO_VIOLATION\n");
+	if (irq_stat & VIP_PORTB_ANC_PROTO_VIOLATION)
+		vip_dbg(3, dev, "VIP_PORTB_ANC_PROTO_VIOLATION\n");
+	if (irq_stat & VIP_PORTA_CFG_DISABLE_COMPLETE)
+		vip_dbg(3, dev, "VIP_PORTA_CFG_DISABLE_COMPLETE\n");
+	if (irq_stat & VIP_PORTB_CFG_DISABLE_COMPLETE)
+		vip_dbg(3, dev, "VIP_PORTB_CFG_DISABLE_COMPLETE\n");
+
+	if (irq_stat & (VIP_PORTA_ASYNC_FIFO_OF |
+			VIP_PORTA_OUTPUT_FIFO_YUV |
+			VIP_PORTA_OUTPUT_FIFO_ANC)) {
+		for (i = 0; i < VIP_CAP_STREAMS_PER_PORT; i++) {
+			if (porta->cap_streams[i] &&
+			    porta->cap_streams[i]->port->port_id ==
+			    porta->port_id) {
+				stream = porta->cap_streams[i];
+				break;
+			}
+		}
+		if (stream) {
+			disable_irqs(dev, dev->slice_id,
+				     stream->list_num);
+			schedule_work(&stream->recovery_work);
+			return;
+		}
+	}
+	if (irq_stat & (VIP_PORTB_ASYNC_FIFO_OF |
+			VIP_PORTB_OUTPUT_FIFO_YUV |
+			VIP_PORTB_OUTPUT_FIFO_ANC)) {
+		for (i = 0; i < VIP_CAP_STREAMS_PER_PORT; i++) {
+			if (portb->cap_streams[i] &&
+			    portb->cap_streams[i]->port->port_id ==
+			    portb->port_id) {
+				stream = portb->cap_streams[i];
+				break;
+			}
+		}
+		if (stream) {
+			disable_irqs(dev, dev->slice_id,
+				     stream->list_num);
+			schedule_work(&stream->recovery_work);
+			return;
+		}
+	}
+}
+
+static irqreturn_t vip_irq(int irq_vip, void *data)
+{
+	struct vip_dev *dev = (struct vip_dev *)data;
+	struct vpdma_data *vpdma;
+	struct vip_stream *stream;
+	int list_num;
+	int irq_num = dev->slice_id;
+	u32 irqst, irqst_saved, reg_addr;
+
+	if (!dev->shared)
+		return IRQ_HANDLED;
+
+	vpdma = dev->shared->vpdma;
+	reg_addr = VIP_INT0_STATUS0 +
+			VIP_INTC_INTX_OFFSET * irq_num;
+	irqst_saved = reg_read(dev->shared, reg_addr);
+	irqst = irqst_saved;
+
+	vip_dbg(8, dev, "IRQ %d VIP_INT%d_STATUS0 0x%x\n",
+		irq_vip, irq_num, irqst);
+	if (irqst) {
+		if (irqst & (VIP_VIP1_PARSER_INT << (irq_num * 1))) {
+			irqst &= ~(VIP_VIP1_PARSER_INT << (irq_num * 1));
+			handle_parser_irqs(dev);
+		}
+
+		for (list_num = 0; irqst && (list_num < 8);  list_num++) {
+			/* Check for LIST_COMPLETE IRQ */
+			if (!(irqst & (1 << list_num * 2)))
+				continue;
+
+			vip_dbg(8, dev, "IRQ %d: handling LIST%d_COMPLETE\n",
+				irq_num, list_num);
+
+			stream = vpdma_hwlist_get_priv(vpdma, list_num);
+			if (!stream || stream->list_num != list_num) {
+				vip_err(dev, "IRQ occurred for unused list");
+				continue;
+			}
+
+			vpdma_clear_list_stat(vpdma, irq_num, list_num);
+
+			vip_process_buffer_complete(stream);
+
+			vip_schedule_next_buffer(stream);
+
+			irqst &= ~((1 << list_num * 2));
+		}
+	}
+
+	/* Acknowledge that we are done with all interrupts */
+	reg_write(dev->shared, VIP_INTC_E0I, 1 << irq_num);
+
+	/* Clear handled events from status register */
+	reg_addr = VIP_INT0_STATUS0_CLR +
+		   VIP_INTC_INTX_OFFSET * irq_num;
+	reg_write(dev->shared, reg_addr, irqst_saved);
+
+	return IRQ_HANDLED;
+}
+
+/*
+ * video ioctls
+ */
+static int vip_querycap(struct file *file, void *priv,
+			struct v4l2_capability *cap)
+{
+	struct vip_stream *stream = file2stream(file);
+	struct vip_port *port = stream->port;
+	struct vip_dev *dev = port->dev;
+
+	strscpy(cap->driver, VIP_MODULE_NAME, sizeof(cap->driver));
+	strscpy(cap->card, VIP_MODULE_NAME, sizeof(cap->card));
+
+	snprintf(cap->bus_info, sizeof(cap->bus_info),
+		 "platform:%s:%s:stream%1d", dev->shared->name, port->name,
+		 stream->stream_id);
+	return 0;
+}
+
+static int vip_enuminput(struct file *file, void *priv,
+			 struct v4l2_input *inp)
+{
+	struct vip_stream *stream = file2stream(file);
+
+	if (inp->index)
+		return -EINVAL;
+
+	inp->type = V4L2_INPUT_TYPE_CAMERA;
+	inp->std = stream->vfd->tvnorms;
+	snprintf(inp->name, sizeof(inp->name), "camera %u", stream->vfd->num);
+
+	return 0;
+}
+
+static int vip_g_input(struct file *file, void *priv, unsigned int *i)
+{
+	*i = 0;
+	return 0;
+}
+
+static int vip_s_input(struct file *file, void *priv, unsigned int i)
+{
+	if (i != 0)
+		return -EINVAL;
+	return 0;
+}
+
+static int vip_querystd(struct file *file, void *fh, v4l2_std_id *std)
+{
+	struct vip_stream *stream = file2stream(file);
+	struct vip_port *port = stream->port;
+
+	*std = stream->vfd->tvnorms;
+	v4l2_subdev_call(port->subdev, video, querystd, std);
+	vip_dbg(1, stream, "querystd: 0x%lx\n", (unsigned long)*std);

drivers/media/platform/ti-vpe/vip.c line 1362

Quoted 10 lines
drivers/media/platform/ti-vpe/vip.c new file +10 -0
+	return 0;
+}
+
+static int vip_g_std(struct file *file, void *fh, v4l2_std_id *std)
+{
+	struct vip_stream *stream = file2stream(file);
+	struct vip_port *port = stream->port;
+
+	*std = stream->vfd->tvnorms;
+	v4l2_subdev_call(port->subdev, video, g_std_output, std);

g_std_output? Shouldn't this just be g_std? The g_std_output op is for video transmitters.

Quoted 9 lines
drivers/media/platform/ti-vpe/vip.c new file +9 -0
+	vip_dbg(1, stream, "g_std: 0x%lx\n", (unsigned long)*std);
+
+	return 0;
+}
+
+static int vip_s_std(struct file *file, void *fh, v4l2_std_id std)
+{
+	struct vip_stream *stream = file2stream(file);
+	struct vip_port *port = stream->port;

drivers/media/platform/ti-vpe/vip.c line 1381

Quoted 10 lines
drivers/media/platform/ti-vpe/vip.c new file +10 -0
+
+	vip_dbg(1, stream, "s_std: 0x%lx\n", (unsigned long)std);
+
+	if (!(std & stream->vfd->tvnorms)) {
+		vip_dbg(1, stream, "s_std after check: 0x%lx\n",
+			(unsigned long)std);
+		return -EINVAL;
+	}
+
+	v4l2_subdev_call(port->subdev, video, s_std_output, std);

Ditto.

Quoted 8 lines
drivers/media/platform/ti-vpe/vip.c new file +8 -0
+	return 0;
+}
+
+static int vip_enum_fmt_vid_cap(struct file *file, void *priv,
+				struct v4l2_fmtdesc *f)
+{
+	struct vip_stream *stream = file2stream(file);
+	struct vip_port *port = stream->port;

drivers/media/platform/ti-vpe/vip.c line 1399

Quoted 10 lines
drivers/media/platform/ti-vpe/vip.c new file +10 -0
+	struct vip_fmt *fmt;
+
+	vip_dbg(3, stream, "enum_fmt index:%d\n", f->index);
+	if (f->index >= port->num_active_fmt)
+		return -EINVAL;
+
+	fmt = port->active_fmt[f->index];
+
+	f->pixelformat = fmt->fourcc;
+	f->type = V4L2_BUF_TYPE_VIDEO_CAPTURE;

No need to set the type field.

drivers/media/platform/ti-vpe/vip.c line 1401

Quoted 2 lines
drivers/media/platform/ti-vpe/vip.c new file +2 -0
+	vip_dbg(3, stream, "enum_fmt fourcc:%s\n",
+		fourcc_to_str(f->pixelformat));

Excessive debugging.

Quoted 16 lines
drivers/media/platform/ti-vpe/vip.c new file +16 -0
+
+	return 0;
+}
+
+static int vip_enum_framesizes(struct file *file, void *priv,
+			       struct v4l2_frmsizeenum *f)
+{
+	struct vip_stream *stream = file2stream(file);
+	struct vip_port *port = stream->port;
+	struct vip_fmt *fmt;
+	struct v4l2_subdev_frame_size_enum fse;
+	int ret;
+
+	fmt = find_port_format_by_pix(port, f->pixel_format);
+	if (!fmt)
+		return -EINVAL;

drivers/media/platform/ti-vpe/vip.c line 1427

Quoted 10 lines
drivers/media/platform/ti-vpe/vip.c new file +10 -0
+
+	fse.index = f->index;
+	fse.pad = port->source_pad;
+	fse.code = fmt->code;
+	fse.which = V4L2_SUBDEV_FORMAT_ACTIVE;
+	ret = v4l2_subdev_call(port->subdev, pad, enum_frame_size, NULL, &fse);
+	if (ret == -ENOIOCTLCMD && !f->index) {
+		/*
+		 * if subdev does not support enum_frame_size
+		 * then use get_fmt

I don't think that's right. If the subdev doesn't support this, then this ioctl should be disabled altogether. Typically this ioctl is only valid for sensor subdevs, not for video receivers.

Use v4l2_subdev_has_op() and v4l2_disable_ioctl().

Quoted 42 lines
drivers/media/platform/ti-vpe/vip.c new file +42 -0
+		 */
+		struct v4l2_subdev_format format = {
+			.which = V4L2_SUBDEV_FORMAT_ACTIVE,
+			.pad = port->source_pad,
+		};
+		ret = v4l2_subdev_call(port->subdev, pad, get_fmt, NULL,
+				       &format);
+		if (ret)
+			return ret;
+
+		fse.max_width = format.format.width;
+		fse.max_height = format.format.height;
+	} else if (ret) {
+		return -EINVAL;
+	}
+
+	vip_dbg(1, stream, "%s: index: %d code: %x W:[%d,%d] H:[%d,%d]\n",
+		__func__, fse.index, fse.code, fse.min_width, fse.max_width,
+		fse.min_height, fse.max_height);
+
+	f->type = V4L2_FRMSIZE_TYPE_DISCRETE;
+	f->discrete.width = fse.max_width;
+	f->discrete.height = fse.max_height;
+
+	return 0;
+}
+
+static int vip_enum_frameintervals(struct file *file, void *priv,
+				   struct v4l2_frmivalenum *f)
+{
+	struct vip_stream *stream = file2stream(file);
+	struct vip_port *port = stream->port;
+	struct vip_fmt *fmt;
+	struct v4l2_subdev_frame_interval_enum fie = {
+		.index = f->index,
+		.width = f->width,
+		.height = f->height,
+		.which = V4L2_SUBDEV_FORMAT_ACTIVE,
+	};
+	int ret;
+
+	fmt = find_port_format_by_pix(port, f->pixel_format);

drivers/media/platform/ti-vpe/vip.c line 1479

Quoted 10 lines
drivers/media/platform/ti-vpe/vip.c new file +10 -0
+	if (!fmt)
+		return -EINVAL;
+
+	fie.code = fmt->code;
+	ret = v4l2_subdev_call(port->subdev, pad, enum_frame_interval,
+			       NULL, &fie);
+	if (ret)
+		return ret;
+	f->type = V4L2_FRMIVAL_TYPE_DISCRETE;
+	f->discrete = fie.interval;

Same here: disable this ioctl if the subdev doesn't support enum_frame_interval.

Quoted 10 lines
drivers/media/platform/ti-vpe/vip.c new file +10 -0
+
+	return 0;
+}
+
+static int vip_g_parm(struct file *file, void *priv,
+		      struct v4l2_streamparm *parm)
+{
+	struct vip_stream *stream = file2stream(file);
+	struct vip_port *port = stream->port;
+

drivers/media/platform/ti-vpe/vip.c line 1499

Quoted 10 lines
drivers/media/platform/ti-vpe/vip.c new file +10 -0
+	return v4l2_g_parm_cap(stream->vfd, port->subdev, parm);
+}
+
+static int vip_s_parm(struct file *file, void *priv,
+		      struct v4l2_streamparm *parm)
+{
+	struct vip_stream *stream = file2stream(file);
+	struct vip_port *port = stream->port;
+
+	return v4l2_s_parm_cap(stream->vfd, port->subdev, parm);

Same here: typically this is not supported for video receivers.

Quoted 33 lines
drivers/media/platform/ti-vpe/vip.c new file +33 -0
+}
+
+static int vip_calc_format_size(struct vip_port *port,
+				struct vip_fmt *fmt,
+				struct v4l2_format *f)
+{
+	enum v4l2_field *field;
+	unsigned int stride;
+
+	if (!fmt) {
+		vip_dbg(2, port,
+			"no vip_fmt format provided!\n");
+		return -EINVAL;
+	}
+
+	field = &f->fmt.pix.field;
+	if (*field == V4L2_FIELD_ANY)
+		*field = V4L2_FIELD_NONE;
+	else if (V4L2_FIELD_NONE != *field && V4L2_FIELD_ALTERNATE != *field)
+		return -EINVAL;
+
+	v4l_bound_align_image(&f->fmt.pix.width, MIN_W, MAX_W, W_ALIGN,
+			      &f->fmt.pix.height, MIN_H, MAX_H, H_ALIGN,
+			      S_ALIGN);
+
+	stride = f->fmt.pix.width * (fmt->vpdma_fmt[0]->depth >> 3);
+	if (stride > f->fmt.pix.bytesperline)
+		f->fmt.pix.bytesperline = stride;
+
+	f->fmt.pix.bytesperline = clamp_t(u32, f->fmt.pix.bytesperline,
+					  stride, VPDMA_MAX_STRIDE);
+	f->fmt.pix.bytesperline = ALIGN(f->fmt.pix.bytesperline,
+					VPDMA_STRIDE_ALIGN);

drivers/media/platform/ti-vpe/vip.c line 1542

Quoted 10 lines
drivers/media/platform/ti-vpe/vip.c new file +10 -0
+
+	f->fmt.pix.sizeimage = f->fmt.pix.height * f->fmt.pix.bytesperline;
+	if (fmt->coplanar) {
+		f->fmt.pix.sizeimage += f->fmt.pix.height *
+					f->fmt.pix.bytesperline *
+					fmt->vpdma_fmt[VIP_CHROMA]->depth >> 3;
+	}
+
+	f->fmt.pix.colorspace = fmt->colorspace;
+	f->fmt.pix.priv = 0;

No need to set this.

Quoted 544 lines
drivers/media/platform/ti-vpe/vip.c new file +544 -0
+
+	vip_dbg(3, port, "calc_format_size: fourcc:%s size: %dx%d bpl:%d img_size:%d\n",
+		fourcc_to_str(f->fmt.pix.pixelformat),
+		f->fmt.pix.width, f->fmt.pix.height,
+		f->fmt.pix.bytesperline, f->fmt.pix.sizeimage);
+
+	return 0;
+}
+
+static inline bool vip_is_size_dma_aligned(u32 bpp, u32 width)
+{
+	return ((width * bpp) == ALIGN(width * bpp, VPDMA_STRIDE_ALIGN));
+}
+
+static int vip_try_fmt_vid_cap(struct file *file, void *priv,
+			       struct v4l2_format *f)
+{
+	struct vip_stream *stream = file2stream(file);
+	struct vip_port *port = stream->port;
+	struct v4l2_subdev_frame_size_enum fse;
+	struct vip_fmt *fmt;
+	u32 best_width, best_height, largest_width, largest_height;
+	int ret, found;
+	enum vip_csc_state csc_direction;
+
+	vip_dbg(3, stream, "try_fmt fourcc:%s size: %dx%d\n",
+		fourcc_to_str(f->fmt.pix.pixelformat),
+		f->fmt.pix.width, f->fmt.pix.height);
+
+	fmt = find_port_format_by_pix(port, f->fmt.pix.pixelformat);
+	if (!fmt) {
+		vip_dbg(2, stream,
+			"Fourcc format (0x%08x) not found.\n",
+			f->fmt.pix.pixelformat);
+
+		/* Just get the first one enumerated */
+		fmt = port->active_fmt[0];
+		f->fmt.pix.pixelformat = fmt->fourcc;
+	}
+
+	csc_direction =  vip_csc_direction(fmt->code, fmt->finfo);
+	if (csc_direction != VIP_CSC_NA) {
+		if (!is_csc_available(port)) {
+			vip_dbg(2, stream,
+				"CSC not available for Fourcc format (0x%08x).\n",
+				f->fmt.pix.pixelformat);
+
+			/* Just get the first one enumerated */
+			fmt = port->active_fmt[0];
+			f->fmt.pix.pixelformat = fmt->fourcc;
+			/* re-evaluate the csc_direction here */
+			csc_direction =  vip_csc_direction(fmt->code,
+							   fmt->finfo);
+		} else {
+			vip_dbg(3, stream, "CSC active on Port %c: going %s\n",
+				port->port_id == VIP_PORTA ? 'A' : 'B',
+				(csc_direction == VIP_CSC_Y2R) ? "Y2R" : "R2Y");
+		}
+	}
+
+	/*
+	 * Given that sensors might support multiple mbus code we need
+	 * to use the one that matches the requested pixel format
+	 */
+	port->try_mbus_framefmt = port->mbus_framefmt;
+	port->try_mbus_framefmt.code = fmt->code;
+
+	/* check for/find a valid width/height */
+	ret = 0;
+	found = false;
+	best_width = 0;
+	best_height = 0;
+	largest_width = 0;
+	largest_height = 0;
+	fse.pad = port->source_pad;
+	fse.code = fmt->code;
+	fse.which = V4L2_SUBDEV_FORMAT_ACTIVE;
+	for (fse.index = 0; ; fse.index++) {
+		u32 bpp = fmt->vpdma_fmt[0]->depth >> 3;
+
+		ret = v4l2_subdev_call(port->subdev, pad,
+				       enum_frame_size, NULL, &fse);
+		if (ret == -ENOIOCTLCMD) {
+			/*
+			 * if subdev does not support enum_frame_size
+			 * then just try to set_fmt directly
+			 */
+			struct v4l2_subdev_format format = {
+				.which = V4L2_SUBDEV_FORMAT_TRY,
+			};
+			struct v4l2_subdev_pad_config *pad_cfg;
+
+			pad_cfg = v4l2_subdev_alloc_pad_config(port->subdev);
+			if (!pad_cfg)
+				return -ENOMEM;
+
+			v4l2_fill_mbus_format(&format.format, &f->fmt.pix,
+					      fmt->code);
+			ret = v4l2_subdev_call(port->subdev, pad, set_fmt,
+					       pad_cfg, &format);
+			if (ret)
+				/* here regardless of the reason we give up */
+				break;
+
+			if (f->fmt.pix.width == format.format.width &&
+			    f->fmt.pix.height == format.format.height) {
+				found = true;
+				vip_dbg(3, stream, "try_fmt loop:%d found direct match: %dx%d\n",
+					fse.index, format.format.width,
+					format.format.height);
+			}
+			largest_width = format.format.width;
+			largest_height = format.format.height;
+			best_width = format.format.width;
+			best_height = format.format.height;
+
+			v4l2_subdev_free_pad_config(pad_cfg);
+			break;
+
+		} else if (ret) {
+			break;
+		}
+
+		vip_dbg(3, stream, "try_fmt loop:%d fourcc:%s size: %dx%d\n",
+			fse.index, fourcc_to_str(f->fmt.pix.pixelformat),
+			fse.max_width, fse.max_height);
+
+		if (!vip_is_size_dma_aligned(bpp, fse.max_width))
+			continue;
+
+		if ((fse.max_width >= largest_width) &&
+		    (fse.max_height >= largest_height)) {
+			vip_dbg(3, stream, "try_fmt loop:%d found new larger: %dx%d\n",
+				fse.index, fse.max_width, fse.max_height);
+			largest_width = fse.max_width;
+			largest_height = fse.max_height;
+		}
+
+		if ((fse.max_width >= f->fmt.pix.width) &&
+		    (fse.max_height >= f->fmt.pix.height)) {
+			vip_dbg(3, stream, "try_fmt loop:%d found at least larger: %dx%d\n",
+				fse.index, fse.max_width, fse.max_height);
+
+			if (!best_width ||
+			    ((abs(best_width - f->fmt.pix.width) >=
+			      abs(fse.max_width - f->fmt.pix.width)) &&
+			     (abs(best_height - f->fmt.pix.height) >=
+			      abs(fse.max_height - f->fmt.pix.height)))) {
+				best_width = fse.max_width;
+				best_height = fse.max_height;
+				vip_dbg(3, stream, "try_fmt loop:%d found new best: %dx%d\n",
+					fse.index, fse.max_width,
+					fse.max_height);
+			}
+		}
+
+		if ((f->fmt.pix.width == fse.max_width) &&
+		    (f->fmt.pix.height == fse.max_height)) {
+			found = true;
+			vip_dbg(3, stream, "try_fmt loop:%d found direct match: %dx%d\n",
+				fse.index, fse.max_width,
+				fse.max_height);
+			break;
+		}
+
+		if ((f->fmt.pix.width >= fse.min_width) &&
+		    (f->fmt.pix.width <= fse.max_width) &&
+		    (f->fmt.pix.height >= fse.min_height) &&
+		    (f->fmt.pix.height <= fse.max_height)) {
+			found = true;
+			vip_dbg(3, stream, "try_fmt loop:%d found direct range match: %dx%d\n",
+				fse.index, fse.max_width,
+				fse.max_height);
+			break;
+		}
+	}
+
+	if (found) {
+		port->try_mbus_framefmt.width = f->fmt.pix.width;
+		port->try_mbus_framefmt.height = f->fmt.pix.height;
+		/* No need to check for scaling */
+		goto calc_size;
+	} else if (largest_width && f->fmt.pix.width > largest_width) {
+		port->try_mbus_framefmt.width = largest_width;
+		port->try_mbus_framefmt.height = largest_height;
+	} else if (best_width) {
+		port->try_mbus_framefmt.width = best_width;
+		port->try_mbus_framefmt.height = best_height;
+	} else {
+		/* use existing values as default */
+	}
+
+	vip_dbg(3, stream, "try_fmt best subdev size: %dx%d\n",
+		port->try_mbus_framefmt.width,
+		port->try_mbus_framefmt.height);
+
+	if (is_scaler_available(port) &&
+	    csc_direction != VIP_CSC_Y2R &&
+	    !vip_is_mbuscode_raw(fmt->code) &&
+	    f->fmt.pix.height <= port->try_mbus_framefmt.height &&
+	    port->try_mbus_framefmt.height <= SC_MAX_PIXEL_HEIGHT &&
+	    port->try_mbus_framefmt.width <= SC_MAX_PIXEL_WIDTH) {
+		/*
+		 * Scaler is only accessible if the dst colorspace is YUV.
+		 * As the input to the scaler must be in YUV mode only.
+		 *
+		 * Scaling up is allowed only horizontally.
+		 */
+		unsigned int hratio, vratio, width_align, height_align;
+		u32 bpp = fmt->vpdma_fmt[0]->depth >> 3;
+
+		vip_dbg(3, stream, "Scaler active on Port %c: requesting %dx%d\n",
+			port->port_id == VIP_PORTA ? 'A' : 'B',
+			f->fmt.pix.width, f->fmt.pix.height);
+
+		/* Just make sure everything is properly aligned */
+		width_align = ALIGN(f->fmt.pix.width * bpp, VPDMA_STRIDE_ALIGN);
+		width_align /= bpp;
+		height_align = ALIGN(f->fmt.pix.height, 2);
+
+		f->fmt.pix.width = width_align;
+		f->fmt.pix.height = height_align;
+
+		hratio = f->fmt.pix.width * 1000 /
+			 port->try_mbus_framefmt.width;
+		vratio = f->fmt.pix.height * 1000 /
+			 port->try_mbus_framefmt.height;
+		if (hratio < 125) {
+			f->fmt.pix.width = port->try_mbus_framefmt.width / 8;
+			vip_dbg(3, stream, "Horizontal scaling ratio out of range adjusting -> %d\n",
+				f->fmt.pix.width);
+		}
+
+		if (vratio < 188) {
+			f->fmt.pix.height = port->try_mbus_framefmt.height / 4;
+			vip_dbg(3, stream, "Vertical scaling ratio out of range adjusting -> %d\n",
+				f->fmt.pix.height);
+		}
+		vip_dbg(3, stream, "Scaler: got %dx%d\n",
+			f->fmt.pix.width, f->fmt.pix.height);
+	} else {
+		/* use existing values as default */
+		f->fmt.pix.width = port->try_mbus_framefmt.width;
+		f->fmt.pix.height = port->try_mbus_framefmt.height;
+	}
+
+calc_size:
+	/* That we have a fmt calculate imagesize and bytesperline */
+	return vip_calc_format_size(port, fmt, f);
+}
+
+static int vip_g_fmt_vid_cap(struct file *file, void *priv,
+			     struct v4l2_format *f)
+{
+	struct vip_stream *stream = file2stream(file);
+	struct vip_port *port = stream->port;
+	struct vip_fmt *fmt = port->fmt;
+
+	/* Use last known values or defaults */
+	f->fmt.pix.width	= stream->width;
+	f->fmt.pix.height	= stream->height;
+	f->fmt.pix.pixelformat	= port->fmt->fourcc;
+	f->fmt.pix.field	= stream->sup_field;
+	f->fmt.pix.colorspace	= port->fmt->colorspace;
+	f->fmt.pix.bytesperline	= stream->bytesperline;
+	f->fmt.pix.sizeimage	= stream->sizeimage;
+
+	vip_dbg(3, stream,
+		"g_fmt fourcc:%s code: %04x size: %dx%d bpl:%d img_size:%d\n",
+		fourcc_to_str(f->fmt.pix.pixelformat),
+		fmt->code,
+		f->fmt.pix.width, f->fmt.pix.height,
+		f->fmt.pix.bytesperline, f->fmt.pix.sizeimage);
+	vip_dbg(3, stream, "g_fmt vpdma data type: 0x%02X\n",
+		port->fmt->vpdma_fmt[0]->data_type);
+
+	return 0;
+}
+
+static int vip_s_fmt_vid_cap(struct file *file, void *priv,
+			     struct v4l2_format *f)
+{
+	struct vip_stream *stream = file2stream(file);
+	struct vip_port *port = stream->port;
+	struct v4l2_subdev_format sfmt;
+	struct v4l2_mbus_framefmt *mf;
+	enum vip_csc_state csc_direction;
+	int ret;
+
+	vip_dbg(3, stream, "s_fmt input fourcc:%s size: %dx%d bpl:%d img_size:%d\n",
+		fourcc_to_str(f->fmt.pix.pixelformat),
+		f->fmt.pix.width, f->fmt.pix.height,
+		f->fmt.pix.bytesperline, f->fmt.pix.sizeimage);
+
+	ret = vip_try_fmt_vid_cap(file, priv, f);
+	if (ret)
+		return ret;
+
+	vip_dbg(3, stream, "s_fmt try_fmt fourcc:%s size: %dx%d bpl:%d img_size:%d\n",
+		fourcc_to_str(f->fmt.pix.pixelformat),
+		f->fmt.pix.width, f->fmt.pix.height,
+		f->fmt.pix.bytesperline, f->fmt.pix.sizeimage);
+
+	if (vb2_is_busy(&stream->vb_vidq)) {
+		vip_err(stream, "%s queue busy\n", __func__);
+		return -EBUSY;
+	}
+
+	/*
+	 * Check if we need the scaler or not
+	 *
+	 * Since on previous S_FMT call the scaler might have been
+	 * allocated if it is not needed in this instance we will
+	 * attempt to free it just in case.
+	 *
+	 * free_scaler() is harmless unless the current port
+	 * allocated it.
+	 */
+	if (f->fmt.pix.width == port->try_mbus_framefmt.width &&
+	    f->fmt.pix.height == port->try_mbus_framefmt.height)
+		free_scaler(port);
+	else
+		allocate_scaler(port);
+
+	port->fmt = find_port_format_by_pix(port,
+					    f->fmt.pix.pixelformat);
+	stream->width		= f->fmt.pix.width;
+	stream->height		= f->fmt.pix.height;
+	stream->bytesperline	= f->fmt.pix.bytesperline;
+	stream->sizeimage	= f->fmt.pix.sizeimage;
+	stream->sup_field	= f->fmt.pix.field;
+	stream->field		= f->fmt.pix.field;
+
+	port->c_rect.left	= 0;
+	port->c_rect.top	= 0;
+	port->c_rect.width	= stream->width;
+	port->c_rect.height	= stream->height;
+
+	/*
+	 * Check if we need the csc unit or not
+	 *
+	 * Since on previous S_FMT call, the csc might have been
+	 * allocated if it is not needed in this instance we will
+	 * attempt to free it just in case.
+	 *
+	 * free_csc() is harmless unless the current port
+	 * allocated it.
+	 */
+	csc_direction =  vip_csc_direction(port->fmt->code, port->fmt->finfo);
+	if (csc_direction == VIP_CSC_NA)
+		free_csc(port);
+	else
+		allocate_csc(port, csc_direction);
+
+	if (stream->sup_field == V4L2_FIELD_ALTERNATE)
+		port->flags |= FLAG_INTERLACED;
+	else
+		port->flags &= ~FLAG_INTERLACED;
+
+	vip_dbg(3, stream, "s_fmt fourcc:%s size: %dx%d bpl:%d img_size:%d\n",
+		fourcc_to_str(f->fmt.pix.pixelformat),
+		f->fmt.pix.width, f->fmt.pix.height,
+		f->fmt.pix.bytesperline, f->fmt.pix.sizeimage);
+
+	mf = &sfmt.format;
+	v4l2_fill_mbus_format(mf, &f->fmt.pix, port->fmt->code);
+	/* Make sure to use the subdev size found in the try_fmt */
+	mf->width = port->try_mbus_framefmt.width;
+	mf->height = port->try_mbus_framefmt.height;
+
+	vip_dbg(3, stream, "s_fmt pix_to_mbus mbus_code: %04X size: %dx%d\n",
+		mf->code,
+		mf->width, mf->height);
+
+	sfmt.which = V4L2_SUBDEV_FORMAT_ACTIVE;
+	sfmt.pad = port->source_pad;
+	ret = v4l2_subdev_call(port->subdev, pad, set_fmt, NULL, &sfmt);
+	if (ret) {
+		vip_dbg(1, stream, "set_fmt failed in subdev\n");
+		return ret;
+	}
+
+	/* Save it */
+	port->mbus_framefmt = *mf;
+
+	vip_dbg(3, stream, "s_fmt subdev fmt mbus_code: %04X size: %dx%d\n",
+		port->mbus_framefmt.code,
+		port->mbus_framefmt.width, port->mbus_framefmt.height);
+	vip_dbg(3, stream, "s_fmt vpdma data type: 0x%02X\n",
+		port->fmt->vpdma_fmt[0]->data_type);
+
+	return 0;
+}
+
+/*
+ * Does the exact opposite of set_fmt_params
+ * It makes sure the DataPath register is sane after tear down
+ */
+static void unset_fmt_params(struct vip_stream *stream)
+{
+	struct vip_dev *dev = stream->port->dev;
+	struct vip_port *port = stream->port;
+
+	stream->sequence = 0;
+	if (stream->port->flags & FLAG_INTERLACED)
+		stream->field = V4L2_FIELD_TOP;
+
+	if (port->csc == VIP_CSC_Y2R) {
+		if (port->port_id == VIP_PORTA) {
+			vip_set_slice_path(dev, VIP_CSC_SRC_DATA_SELECT, 0);
+			vip_set_slice_path(dev,
+					   VIP_MULTI_CHANNEL_DATA_SELECT, 0);
+			vip_set_slice_path(dev, VIP_RGB_OUT_HI_DATA_SELECT, 0);
+			vip_set_slice_path(dev, VIP_RGB_SRC_DATA_SELECT, 0);
+		} else {
+			vip_set_slice_path(dev, VIP_CSC_SRC_DATA_SELECT, 0);
+			vip_set_slice_path(dev,
+					   VIP_MULTI_CHANNEL_DATA_SELECT, 0);
+			vip_set_slice_path(dev, VIP_RGB_OUT_LO_DATA_SELECT, 0);
+		}
+		/* We are done */
+		return;
+	} else if (port->csc == VIP_CSC_R2Y) {
+		if (port->scaler && port->fmt->coplanar) {
+			if (port->port_id == VIP_PORTA) {
+				vip_set_slice_path(dev,
+						   VIP_CSC_SRC_DATA_SELECT, 0);
+				vip_set_slice_path(dev,
+						   VIP_SC_SRC_DATA_SELECT, 0);
+				vip_set_slice_path(dev,
+						   VIP_CHR_DS_1_SRC_DATA_SELECT,
+						   0);
+				vip_set_slice_path(dev,
+						   VIP_CHR_DS_1_DATA_BYPASS, 0);
+				vip_set_slice_path(dev,
+						   VIP_RGB_OUT_HI_DATA_SELECT,
+						   0);
+			}
+		} else if (port->scaler) {
+			if (port->port_id == VIP_PORTA) {
+				vip_set_slice_path(dev,
+						   VIP_CSC_SRC_DATA_SELECT, 0);
+				vip_set_slice_path(dev,
+						   VIP_SC_SRC_DATA_SELECT, 0);
+				vip_set_slice_path(dev,
+						   VIP_CHR_DS_1_SRC_DATA_SELECT,
+						   0);
+				vip_set_slice_path(dev,
+						   VIP_CHR_DS_1_DATA_BYPASS, 0);
+				vip_set_slice_path(dev,
+						   VIP_RGB_OUT_HI_DATA_SELECT,
+						   0);
+			}
+		} else if (port->fmt->coplanar) {
+			if (port->port_id == VIP_PORTA) {
+				vip_set_slice_path(dev,
+						   VIP_CSC_SRC_DATA_SELECT, 0);
+				vip_set_slice_path(dev,
+						   VIP_CHR_DS_1_SRC_DATA_SELECT,
+						   0);
+				vip_set_slice_path(dev,
+						   VIP_CHR_DS_1_DATA_BYPASS, 0);
+				vip_set_slice_path(dev,
+						   VIP_RGB_OUT_HI_DATA_SELECT,
+						   0);
+			}
+		} else {
+			if (port->port_id == VIP_PORTA) {
+				vip_set_slice_path(dev,
+						   VIP_CSC_SRC_DATA_SELECT, 0);
+				vip_set_slice_path(dev,
+						   VIP_CHR_DS_1_SRC_DATA_SELECT,
+						   0);
+				vip_set_slice_path(dev,
+						   VIP_CHR_DS_1_DATA_BYPASS, 0);
+				vip_set_slice_path(dev,
+						   VIP_RGB_OUT_HI_DATA_SELECT,
+						   0);
+			}
+		}
+		/* We are done */
+		return;
+	} else if (v4l2_is_format_rgb(port->fmt->finfo)) {
+		if (port->port_id == VIP_PORTA) {
+			vip_set_slice_path(dev,
+					   VIP_MULTI_CHANNEL_DATA_SELECT, 0);
+			vip_set_slice_path(dev, VIP_RGB_OUT_LO_DATA_SELECT, 0);
+		}
+		/* We are done */
+		return;
+	}
+
+	if (port->scaler && port->fmt->coplanar) {
+		if (port->port_id == VIP_PORTA) {
+			vip_set_slice_path(dev, VIP_SC_SRC_DATA_SELECT, 0);
+			vip_set_slice_path(dev,
+					   VIP_CHR_DS_1_SRC_DATA_SELECT, 0);
+			vip_set_slice_path(dev, VIP_CHR_DS_1_DATA_BYPASS, 0);
+			vip_set_slice_path(dev, VIP_RGB_OUT_HI_DATA_SELECT, 0);
+		} else {
+			vip_set_slice_path(dev, VIP_SC_SRC_DATA_SELECT, 0);
+			vip_set_slice_path(dev,
+					   VIP_CHR_DS_2_SRC_DATA_SELECT, 0);
+			vip_set_slice_path(dev, VIP_CHR_DS_1_DATA_BYPASS, 0);
+			vip_set_slice_path(dev, VIP_RGB_OUT_LO_DATA_SELECT, 0);
+			vip_set_slice_path(dev,
+					   VIP_MULTI_CHANNEL_DATA_SELECT, 0);
+		}
+	} else if (port->scaler) {
+		if (port->port_id == VIP_PORTA) {
+			vip_set_slice_path(dev, VIP_SC_SRC_DATA_SELECT, 0);
+			vip_set_slice_path(dev,
+					   VIP_CHR_DS_1_SRC_DATA_SELECT, 0);
+			vip_set_slice_path(dev, VIP_CHR_DS_1_DATA_BYPASS, 0);
+			vip_set_slice_path(dev, VIP_RGB_OUT_HI_DATA_SELECT, 0);
+		} else {
+			vip_set_slice_path(dev, VIP_SC_SRC_DATA_SELECT, 0);
+			vip_set_slice_path(dev,
+					   VIP_CHR_DS_2_SRC_DATA_SELECT, 0);
+			vip_set_slice_path(dev, VIP_CHR_DS_1_DATA_BYPASS, 0);
+			vip_set_slice_path(dev, VIP_CHR_DS_2_DATA_BYPASS, 0);
+			vip_set_slice_path(dev, VIP_RGB_OUT_HI_DATA_SELECT, 0);
+		}
+	} else if (port->fmt->coplanar) {
+		if (port->port_id == VIP_PORTA) {
+			vip_set_slice_path(dev,
+					   VIP_CHR_DS_1_SRC_DATA_SELECT, 0);
+			vip_set_slice_path(dev, VIP_CHR_DS_1_DATA_BYPASS, 0);
+			vip_set_slice_path(dev, VIP_RGB_OUT_HI_DATA_SELECT, 0);
+		} else {
+			vip_set_slice_path(dev,
+					   VIP_CHR_DS_2_SRC_DATA_SELECT, 0);
+			vip_set_slice_path(dev, VIP_CHR_DS_2_DATA_BYPASS, 0);
+			vip_set_slice_path(dev,
+					   VIP_MULTI_CHANNEL_DATA_SELECT, 0);
+			vip_set_slice_path(dev, VIP_RGB_OUT_LO_DATA_SELECT, 0);
+		}
+	} else {
+		/*
+		 * We undo all data path setting except for the multi
+		 * stream case.
+		 * Because we cannot disrupt other on-going capture if only
+		 * one stream is terminated the other might still be going
+		 */

drivers/media/platform/ti-vpe/vip.c line 2096

Quoted 10 lines
drivers/media/platform/ti-vpe/vip.c new file +10 -0
+		vip_set_slice_path(dev, VIP_MULTI_CHANNEL_DATA_SELECT, 1);
+		vip_set_slice_path(dev, VIP_RGB_OUT_LO_DATA_SELECT, 0);
+	}
+}
+
+/*
+ * Set the registers that are modified when the video format changes.
+ */
+static void set_fmt_params(struct vip_stream *stream)
+{

Hmm, this is a *very* long function. Perhaps this could be split up a bit, or reorganized?

Quoted 12 lines
drivers/media/platform/ti-vpe/vip.c new file +12 -0
+	struct vip_dev *dev = stream->port->dev;
+	struct vip_port *port = stream->port;
+
+	stream->sequence = 0;
+	if (stream->port->flags & FLAG_INTERLACED)
+		stream->field = V4L2_FIELD_TOP;
+
+	if (port->csc == VIP_CSC_Y2R) {
+		port->flags &= ~FLAG_MULT_PORT;
+		/* Set alpha component in background color */
+		vpdma_set_bg_color(dev->shared->vpdma,
+				   (struct vpdma_data_format *)

drivers/media/platform/ti-vpe/vip.c line 2118

Quoted 10 lines
drivers/media/platform/ti-vpe/vip.c new file +10 -0
+				   port->fmt->vpdma_fmt[0],
+				   0xff);
+		if (port->port_id == VIP_PORTA) {
+			/*
+			 * Input A: YUV422
+			 * Output: Y_UP/UV_UP: RGB
+			 * CSC_SRC_SELECT       = 1
+			 * RGB_OUT_HI_SELECT    = 1
+			 * RGB_SRC_SELECT       = 1
+			 * MULTI_CHANNEL_SELECT = 0

It's a bit pointless to comment what the register values should be when you set them in the code below. I'd drop that part, it will make the code shorter.

drivers/media/platform/ti-vpe/vip.c line 2122

Quoted 4 lines
drivers/media/platform/ti-vpe/vip.c new file +4 -0
+			 */
+			vip_set_slice_path(dev, VIP_CSC_SRC_DATA_SELECT, 1);
+			vip_set_slice_path(dev,
+					   VIP_MULTI_CHANNEL_DATA_SELECT, 0);

For readability purposes I think it is better to keep this on one line. Same for the other vip_set_slice_path calls.

Quoted 282 lines
drivers/media/platform/ti-vpe/vip.c new file +282 -0
+			vip_set_slice_path(dev, VIP_RGB_OUT_HI_DATA_SELECT, 1);
+			vip_set_slice_path(dev, VIP_RGB_SRC_DATA_SELECT, 1);
+		} else {
+			/*
+			 * Input B: YUV422
+			 * Output: Y_UP/UV_UP: RGB
+			 * CSC_SRC_SELECT       = 2
+			 * RGB_OUT_LO_SELECT    = 1
+			 * MULTI_CHANNEL_SELECT = 0
+			 */
+			vip_set_slice_path(dev, VIP_CSC_SRC_DATA_SELECT, 2);
+			vip_set_slice_path(dev,
+					   VIP_MULTI_CHANNEL_DATA_SELECT, 0);
+			vip_set_slice_path(dev, VIP_RGB_OUT_LO_DATA_SELECT, 1);
+		}
+		/* We are done */
+		return;
+	} else if (port->csc == VIP_CSC_R2Y) {
+		port->flags &= ~FLAG_MULT_PORT;
+		if (port->scaler && port->fmt->coplanar) {
+			if (port->port_id == VIP_PORTA) {
+				/*
+				 * Input A: RGB
+				 * Output: Y_UP/UV_UP: Scaled YUV420
+				 * CSC_SRC_SELECT       = 4
+				 * SC_SRC_SELECT        = 1
+				 * CHR_DS_1_SRC_SELECT  = 1
+				 * CHR_DS_1_BYPASS      = 0
+				 * RGB_OUT_HI_SELECT    = 0
+				 */
+				vip_set_slice_path(dev,
+						   VIP_CSC_SRC_DATA_SELECT, 4);
+				vip_set_slice_path(dev,
+						   VIP_SC_SRC_DATA_SELECT, 1);
+				vip_set_slice_path(dev,
+						   VIP_CHR_DS_1_SRC_DATA_SELECT,
+						   1);
+				vip_set_slice_path(dev,
+						   VIP_CHR_DS_1_DATA_BYPASS, 0);
+				vip_set_slice_path(dev,
+						   VIP_RGB_OUT_HI_DATA_SELECT,
+						   0);
+			} else {
+				vip_err(stream, "RGB sensor can only be on Port A\n");
+			}
+		} else if (port->scaler) {
+			if (port->port_id == VIP_PORTA) {
+				/*
+				 * Input A: RGB
+				 * Output: Y_UP: Scaled YUV422
+				 * CSC_SRC_SELECT       = 4
+				 * SC_SRC_SELECT        = 1
+				 * CHR_DS_1_SRC_SELECT  = 1
+				 * CHR_DS_1_BYPASS      = 1
+				 * RGB_OUT_HI_SELECT    = 0
+				 */
+				vip_set_slice_path(dev,
+						   VIP_CSC_SRC_DATA_SELECT, 4);
+				vip_set_slice_path(dev,
+						   VIP_SC_SRC_DATA_SELECT, 1);
+				vip_set_slice_path(dev,
+						   VIP_CHR_DS_1_SRC_DATA_SELECT,
+						   1);
+				vip_set_slice_path(dev,
+						   VIP_CHR_DS_1_DATA_BYPASS, 1);
+				vip_set_slice_path(dev,
+						   VIP_RGB_OUT_HI_DATA_SELECT,
+						   0);
+			} else {
+				vip_err(stream, "RGB sensor can only be on Port A\n");
+			}
+		} else if (port->fmt->coplanar) {
+			if (port->port_id == VIP_PORTA) {
+				/*
+				 * Input A: RGB
+				 * Output: Y_UP/UV_UP: YUV420
+				 * CSC_SRC_SELECT       = 4
+				 * CHR_DS_1_SRC_SELECT  = 2
+				 * CHR_DS_1_BYPASS      = 0
+				 * RGB_OUT_HI_SELECT    = 0
+				 */
+				vip_set_slice_path(dev,
+						   VIP_CSC_SRC_DATA_SELECT, 4);
+				vip_set_slice_path(dev,
+						   VIP_CHR_DS_1_SRC_DATA_SELECT,
+						   2);
+				vip_set_slice_path(dev,
+						   VIP_CHR_DS_1_DATA_BYPASS, 0);
+				vip_set_slice_path(dev,
+						   VIP_RGB_OUT_HI_DATA_SELECT,
+						   0);
+			} else {
+				vip_err(stream, "RGB sensor can only be on Port A\n");
+			}
+		} else {
+			if (port->port_id == VIP_PORTA) {
+				/*
+				 * Input A: RGB
+				 * Output: Y_UP/UV_UP: YUV420
+				 * CSC_SRC_SELECT       = 4
+				 * CHR_DS_1_SRC_SELECT  = 2
+				 * CHR_DS_1_BYPASS      = 1
+				 * RGB_OUT_HI_SELECT    = 0
+				 */
+				vip_set_slice_path(dev,
+						   VIP_CSC_SRC_DATA_SELECT, 4);
+				vip_set_slice_path(dev,
+						   VIP_CHR_DS_1_SRC_DATA_SELECT,
+						   2);
+				vip_set_slice_path(dev,
+						   VIP_CHR_DS_1_DATA_BYPASS, 1);
+				vip_set_slice_path(dev,
+						   VIP_RGB_OUT_HI_DATA_SELECT,
+						   0);
+			} else {
+				vip_err(stream, "RGB sensor can only be on Port A\n");
+			}
+		}
+		/* We are done */
+		return;
+	} else if (v4l2_is_format_rgb(port->fmt->finfo)) {
+		port->flags &= ~FLAG_MULT_PORT;
+		/* Set alpha component in background color */
+		vpdma_set_bg_color(dev->shared->vpdma,
+				   (struct vpdma_data_format *)
+				   port->fmt->vpdma_fmt[0],
+				   0xff);
+		if (port->port_id == VIP_PORTA) {
+			/*
+			 * Input A: RGB
+			 * Output: Y_LO/UV_LO: RGB
+			 * RGB_OUT_LO_SELECT    = 1
+			 * MULTI_CHANNEL_SELECT = 1
+			 */
+			vip_set_slice_path(dev,
+					   VIP_MULTI_CHANNEL_DATA_SELECT, 1);
+			vip_set_slice_path(dev, VIP_RGB_OUT_LO_DATA_SELECT, 1);
+		} else {
+			vip_err(stream, "RGB sensor can only be on Port A\n");
+		}
+		/* We are done */
+		return;
+	}
+
+	if (port->scaler && port->fmt->coplanar) {
+		port->flags &= ~FLAG_MULT_PORT;
+		if (port->port_id == VIP_PORTA) {
+			/*
+			 * Input A: YUV422
+			 * Output: Y_UP/UV_UP: Scaled YUV420
+			 * SC_SRC_SELECT        = 2
+			 * CHR_DS_1_SRC_SELECT  = 1
+			 * CHR_DS_1_BYPASS      = 0
+			 * RGB_OUT_HI_SELECT    = 0
+			 */
+			vip_set_slice_path(dev, VIP_SC_SRC_DATA_SELECT, 2);
+			vip_set_slice_path(dev,
+					   VIP_CHR_DS_1_SRC_DATA_SELECT, 1);
+			vip_set_slice_path(dev, VIP_CHR_DS_1_DATA_BYPASS, 0);
+			vip_set_slice_path(dev, VIP_RGB_OUT_HI_DATA_SELECT, 0);
+		} else {
+			/*
+			 * Input B: YUV422
+			 * Output: Y_LO/UV_LO: Scaled YUV420
+			 * SC_SRC_SELECT        = 3
+			 * CHR_DS_2_SRC_SELECT  = 1
+			 * RGB_OUT_LO_SELECT    = 0
+			 * MULTI_CHANNEL_SELECT = 0
+			 */
+			vip_set_slice_path(dev, VIP_SC_SRC_DATA_SELECT, 3);
+			vip_set_slice_path(dev,
+					   VIP_CHR_DS_2_SRC_DATA_SELECT, 1);
+			vip_set_slice_path(dev, VIP_CHR_DS_1_DATA_BYPASS, 0);
+			vip_set_slice_path(dev, VIP_RGB_OUT_LO_DATA_SELECT, 0);
+			vip_set_slice_path(dev,
+					   VIP_MULTI_CHANNEL_DATA_SELECT, 0);
+		}
+	} else if (port->scaler) {
+		port->flags &= ~FLAG_MULT_PORT;
+		if (port->port_id == VIP_PORTA) {
+			/*
+			 * Input A: YUV422
+			 * Output: Y_UP: Scaled YUV422
+			 * SC_SRC_SELECT        = 2
+			 * CHR_DS_1_SRC_SELECT  = 1
+			 * CHR_DS_1_BYPASS      = 1
+			 * RGB_OUT_HI_SELECT    = 0
+			 */
+			vip_set_slice_path(dev, VIP_SC_SRC_DATA_SELECT, 2);
+			vip_set_slice_path(dev,
+					   VIP_CHR_DS_1_SRC_DATA_SELECT, 1);
+			vip_set_slice_path(dev, VIP_CHR_DS_1_DATA_BYPASS, 1);
+			vip_set_slice_path(dev, VIP_RGB_OUT_HI_DATA_SELECT, 0);
+		} else {
+			/*
+			 * Input B: YUV422
+			 * Output: UV_UP: Scaled YUV422
+			 * SC_SRC_SELECT        = 3
+			 * CHR_DS_2_SRC_SELECT  = 1
+			 * CHR_DS_1_BYPASS      = 1
+			 * CHR_DS_2_BYPASS      = 1
+			 * RGB_OUT_HI_SELECT    = 0
+			 */
+			vip_set_slice_path(dev, VIP_SC_SRC_DATA_SELECT, 3);
+			vip_set_slice_path(dev,
+					   VIP_CHR_DS_2_SRC_DATA_SELECT, 1);
+			vip_set_slice_path(dev, VIP_CHR_DS_1_DATA_BYPASS, 1);
+			vip_set_slice_path(dev, VIP_CHR_DS_2_DATA_BYPASS, 1);
+			vip_set_slice_path(dev, VIP_RGB_OUT_HI_DATA_SELECT, 0);
+		}
+	} else if (port->fmt->coplanar) {
+		port->flags &= ~FLAG_MULT_PORT;
+		if (port->port_id == VIP_PORTA) {
+			/*
+			 * Input A: YUV422
+			 * Output: Y_UP/UV_UP: YUV420
+			 * CHR_DS_1_SRC_SELECT  = 3
+			 * CHR_DS_1_BYPASS      = 0
+			 * RGB_OUT_HI_SELECT    = 0
+			 */
+			vip_set_slice_path(dev,
+					   VIP_CHR_DS_1_SRC_DATA_SELECT, 3);
+			vip_set_slice_path(dev, VIP_CHR_DS_1_DATA_BYPASS, 0);
+			vip_set_slice_path(dev, VIP_RGB_OUT_HI_DATA_SELECT, 0);
+		} else {
+			/*
+			 * Input B: YUV422
+			 * Output: Y_LO/UV_LO: YUV420
+			 * CHR_DS_2_SRC_SELECT  = 4
+			 * CHR_DS_2_BYPASS      = 0
+			 * RGB_OUT_LO_SELECT    = 0
+			 * MULTI_CHANNEL_SELECT = 0
+			 */
+			vip_set_slice_path(dev,
+					   VIP_CHR_DS_2_SRC_DATA_SELECT, 4);
+			vip_set_slice_path(dev, VIP_CHR_DS_2_DATA_BYPASS, 0);
+			vip_set_slice_path(dev,
+					   VIP_MULTI_CHANNEL_DATA_SELECT, 0);
+			vip_set_slice_path(dev, VIP_RGB_OUT_LO_DATA_SELECT, 0);
+		}
+	} else {
+		port->flags |= FLAG_MULT_PORT;
+		/*
+		 * Input A/B: YUV422
+		 * Output: Y_LO: YUV422 - UV_LO: YUV422
+		 * MULTI_CHANNEL_SELECT = 1
+		 * RGB_OUT_LO_SELECT    = 0
+		 */
+		vip_set_slice_path(dev, VIP_MULTI_CHANNEL_DATA_SELECT, 1);
+		vip_set_slice_path(dev, VIP_RGB_OUT_LO_DATA_SELECT, 0);
+	}
+}
+
+static int vip_g_selection(struct file *file, void *fh,
+			   struct v4l2_selection *s)
+{
+	struct vip_stream *stream = file2stream(file);
+
+	if (s->type != V4L2_BUF_TYPE_VIDEO_CAPTURE)
+		return -EINVAL;
+
+	switch (s->target) {
+	case V4L2_SEL_TGT_COMPOSE_DEFAULT:
+	case V4L2_SEL_TGT_COMPOSE_BOUNDS:
+	case V4L2_SEL_TGT_CROP_BOUNDS:
+	case V4L2_SEL_TGT_CROP_DEFAULT:
+		s->r.left = 0;
+		s->r.top = 0;
+		s->r.width = stream->width;
+		s->r.height = stream->height;
+		return 0;
+
+	case V4L2_SEL_TGT_COMPOSE:
+	case V4L2_SEL_TGT_CROP:
+		s->r = stream->port->c_rect;
+		return 0;
+	}
+
+	return -EINVAL;
+}
+
+static int enclosed_rectangle(struct v4l2_rect *a, struct v4l2_rect *b)

drivers/media/platform/ti-vpe/vip.c line 2414

Quoted 10 lines
drivers/media/platform/ti-vpe/vip.c new file +10 -0
+{
+	if (a->left < b->left || a->top < b->top)
+		return 0;
+	if (a->left + a->width > b->left + b->width)
+		return 0;
+	if (a->top + a->height > b->top + b->height)
+		return 0;
+
+	return 1;
+}

There are helper functions in include/media/v4l2-rect.h, it would make sense to add this one to that header.

drivers/media/platform/ti-vpe/vip.c line 2427

Quoted 13 lines
drivers/media/platform/ti-vpe/vip.c new file +13 -0
+
+static int vip_s_selection(struct file *file, void *fh,
+			   struct v4l2_selection *s)
+{
+	struct vip_stream *stream = file2stream(file);
+	struct v4l2_rect r = s->r;
+
+	if (s->type != V4L2_BUF_TYPE_VIDEO_CAPTURE)
+		return -EINVAL;
+
+	switch (s->target) {
+	case V4L2_SEL_TGT_COMPOSE:
+	case V4L2_SEL_TGT_CROP:

Why both crop and compose when it is the same c_rect? That makes no sense.

Quoted 37 lines
drivers/media/platform/ti-vpe/vip.c new file +37 -0
+		v4l_bound_align_image(&r.width, 0, stream->width, 0,
+				      &r.height, 0, stream->height, 0, 0);
+
+		r.left = clamp_t(unsigned int, r.left, 0,
+				 stream->width - r.width);
+		r.top  = clamp_t(unsigned int, r.top, 0,
+				 stream->height - r.height);
+
+		if (s->flags & V4L2_SEL_FLAG_LE &&
+		    !enclosed_rectangle(&r, &s->r))
+			return -ERANGE;
+
+		if (s->flags & V4L2_SEL_FLAG_GE &&
+		    !enclosed_rectangle(&s->r, &r))
+			return -ERANGE;
+
+		s->r = r;
+		stream->port->c_rect = r;
+
+		vip_dbg(1, stream, "cropped (%d,%d)/%dx%d of %dx%d\n",
+			r.left, r.top, r.width, r.height,
+			stream->width, stream->height);
+
+			s->r = stream->port->c_rect;
+		return 0;
+	default:
+		return -EINVAL;
+	}
+
+	return 0;
+}
+
+static long vip_ioctl_default(struct file *file, void *fh, bool valid_prio,
+			      unsigned int cmd, void *arg)
+{
+	struct vip_stream *stream = file2stream(file);
+

drivers/media/platform/ti-vpe/vip.c line 2474

Quoted 10 lines
drivers/media/platform/ti-vpe/vip.c new file +10 -0
+	if (!valid_prio) {
+		vip_err(stream, "%s device busy\n", __func__);
+		return -EBUSY;
+	}
+
+	switch (cmd) {
+	default:
+		return -ENOTTY;
+	}
+}

Huh?

Quoted 271 lines
drivers/media/platform/ti-vpe/vip.c new file +271 -0
+
+static const struct v4l2_ioctl_ops vip_ioctl_ops = {
+	.vidioc_querycap	= vip_querycap,
+	.vidioc_enum_input	= vip_enuminput,
+	.vidioc_g_input		= vip_g_input,
+	.vidioc_s_input		= vip_s_input,
+
+	.vidioc_querystd	= vip_querystd,
+	.vidioc_g_std		= vip_g_std,
+	.vidioc_s_std		= vip_s_std,
+
+	.vidioc_enum_fmt_vid_cap = vip_enum_fmt_vid_cap,
+	.vidioc_g_fmt_vid_cap	= vip_g_fmt_vid_cap,
+	.vidioc_try_fmt_vid_cap	= vip_try_fmt_vid_cap,
+	.vidioc_s_fmt_vid_cap	= vip_s_fmt_vid_cap,
+
+	.vidioc_enum_frameintervals	= vip_enum_frameintervals,
+	.vidioc_enum_framesizes		= vip_enum_framesizes,
+	.vidioc_s_parm			= vip_s_parm,
+	.vidioc_g_parm			= vip_g_parm,
+	.vidioc_g_selection	= vip_g_selection,
+	.vidioc_s_selection	= vip_s_selection,
+	.vidioc_reqbufs		= vb2_ioctl_reqbufs,
+	.vidioc_create_bufs	= vb2_ioctl_create_bufs,
+	.vidioc_prepare_buf	= vb2_ioctl_prepare_buf,
+	.vidioc_querybuf	= vb2_ioctl_querybuf,
+	.vidioc_qbuf		= vb2_ioctl_qbuf,
+	.vidioc_dqbuf		= vb2_ioctl_dqbuf,
+	.vidioc_expbuf		= vb2_ioctl_expbuf,
+
+	.vidioc_streamon	= vb2_ioctl_streamon,
+	.vidioc_streamoff	= vb2_ioctl_streamoff,
+	.vidioc_log_status	= v4l2_ctrl_log_status,
+	.vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
+	.vidioc_unsubscribe_event = v4l2_event_unsubscribe,
+	.vidioc_default		= vip_ioctl_default,
+};
+
+/*
+ * Videobuf operations
+ */
+static int vip_queue_setup(struct vb2_queue *vq,
+			   unsigned int *nbuffers, unsigned int *nplanes,
+			   unsigned int sizes[], struct device *alloc_devs[])
+{
+	struct vip_stream *stream = vb2_get_drv_priv(vq);
+	unsigned int size = stream->sizeimage;
+
+	if (vq->num_buffers + *nbuffers < 3)
+		*nbuffers = 3 - vq->num_buffers;
+
+	if (*nplanes) {
+		if (sizes[0] < size)
+			return -EINVAL;
+		size = sizes[0];
+	}
+
+	*nplanes = 1;
+	sizes[0] = size;
+
+	vip_dbg(1, stream, "get %d buffer(s) of size %d each.\n",
+		*nbuffers, sizes[0]);
+
+	return 0;
+}
+
+static int vip_buf_prepare(struct vb2_buffer *vb)
+{
+	struct vip_stream *stream = vb2_get_drv_priv(vb->vb2_queue);
+
+	if (vb2_plane_size(vb, 0) < stream->sizeimage) {
+		vip_dbg(1, stream,
+			"%s data will not fit into plane (%lu < %lu)\n",
+			__func__, vb2_plane_size(vb, 0),
+			(long)stream->sizeimage);
+		return -EINVAL;
+	}
+
+	vb2_set_plane_payload(vb, 0, stream->sizeimage);
+
+	return 0;
+}
+
+static void vip_buf_queue(struct vb2_buffer *vb)
+{
+	struct vip_stream *stream = vb2_get_drv_priv(vb->vb2_queue);
+	struct vip_dev *dev = stream->port->dev;
+	struct vip_buffer *buf = container_of(vb, struct vip_buffer,
+					      vb.vb2_buf);
+	unsigned long flags;
+
+	spin_lock_irqsave(&dev->slock, flags);
+	list_add_tail(&buf->list, &stream->vidq);
+	spin_unlock_irqrestore(&dev->slock, flags);
+}
+
+static int vip_setup_scaler(struct vip_stream *stream)
+{
+	struct vip_port *port = stream->port;
+	struct vip_dev *dev = port->dev;
+	struct sc_data *sc = dev->sc;
+	struct csc_data *csc = dev->csc;
+	struct vpdma_data *vpdma = dev->shared->vpdma;
+	struct vip_mmr_adb *mmr_adb = port->mmr_adb.addr;
+	int list_num = stream->list_num;
+	int timeout = 500;
+	struct v4l2_format dst_f;
+	struct v4l2_format src_f;
+
+	memset(&src_f, 0, sizeof(src_f));
+	src_f.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
+	v4l2_fill_pix_format(&src_f.fmt.pix, &port->mbus_framefmt);
+	src_f.fmt.pix.pixelformat = vip_mbus_code_to_fourcc(port->fmt->code);
+
+	dst_f = src_f;
+	dst_f.fmt.pix.pixelformat = port->fmt->fourcc;
+	dst_f.fmt.pix.width = stream->width;
+	dst_f.fmt.pix.height = stream->height;
+
+	/* if scaler not associated with this port then skip */
+	if (port->scaler) {
+		sc_set_hs_coeffs(sc, port->sc_coeff_h.addr,
+				 port->mbus_framefmt.width,
+				 port->c_rect.width);
+		sc_set_vs_coeffs(sc, port->sc_coeff_v.addr,
+				 port->mbus_framefmt.height,
+				 port->c_rect.height);
+		sc_config_scaler(sc, &mmr_adb->sc_regs0[0],
+				 &mmr_adb->sc_regs8[0], &mmr_adb->sc_regs17[0],
+				 port->mbus_framefmt.width,
+				 port->mbus_framefmt.height,
+				 port->c_rect.width,
+				 port->c_rect.height);
+		port->load_mmrs = true;
+	}
+
+	/* if csc not associated with this port then skip */
+	if (port->csc) {
+		csc_set_coeff(csc, &mmr_adb->csc_regs[0],
+			      &src_f, &dst_f);
+
+		port->load_mmrs = true;
+	}
+
+	/* If coeff are already loaded then skip */
+	if (!sc->load_coeff_v && !sc->load_coeff_h && !port->load_mmrs)
+		return 0;
+
+	if (vpdma_list_busy(vpdma, list_num)) {
+		vip_dbg(3, stream, "%s: List %d is busy\n",
+			__func__, list_num);
+	}
+
+	/* Make sure we start with a clean list */
+	vpdma_reset_desc_list(&stream->desc_list);
+
+	/* config descriptors */
+	if (port->load_mmrs) {
+		vpdma_map_desc_buf(vpdma, &port->mmr_adb);
+		vpdma_add_cfd_adb(&stream->desc_list, CFD_MMR_CLIENT,
+				  &port->mmr_adb);
+
+		port->load_mmrs = false;
+		vip_dbg(3, stream, "Added mmr_adb config desc\n");
+	}
+
+	if (sc->loaded_coeff_h != port->sc_coeff_h.dma_addr ||
+	    sc->load_coeff_h) {
+		vpdma_map_desc_buf(vpdma, &port->sc_coeff_h);
+		vpdma_add_cfd_block(&stream->desc_list,
+				    VIP_SLICE1_CFD_SC_CLIENT + dev->slice_id,
+				    &port->sc_coeff_h, 0);
+
+		sc->loaded_coeff_h = port->sc_coeff_h.dma_addr;
+		sc->load_coeff_h = false;
+		vip_dbg(3, stream, "Added sc_coeff_h config desc\n");
+	}
+
+	if (sc->loaded_coeff_v != port->sc_coeff_v.dma_addr ||
+	    sc->load_coeff_v) {
+		vpdma_map_desc_buf(vpdma, &port->sc_coeff_v);
+		vpdma_add_cfd_block(&stream->desc_list,
+				    VIP_SLICE1_CFD_SC_CLIENT + dev->slice_id,
+				    &port->sc_coeff_v, SC_COEF_SRAM_SIZE >> 4);
+
+		sc->loaded_coeff_v = port->sc_coeff_v.dma_addr;
+		sc->load_coeff_v = false;
+		vip_dbg(3, stream, "Added sc_coeff_v config desc\n");
+	}
+	vip_dbg(3, stream, "CFD_SC_CLIENT %d slice_id: %d\n",
+		VIP_SLICE1_CFD_SC_CLIENT + dev->slice_id, dev->slice_id);
+
+	vpdma_map_desc_buf(vpdma, &stream->desc_list.buf);
+	vip_dbg(3, stream, "Submitting desc on list# %d\n", list_num);
+	vpdma_submit_descs(vpdma, &stream->desc_list, list_num);
+
+	while (vpdma_list_busy(vpdma, list_num) && timeout--)
+		usleep_range(1000, 1100);
+
+	vpdma_unmap_desc_buf(dev->shared->vpdma, &port->mmr_adb);
+	vpdma_unmap_desc_buf(dev->shared->vpdma, &port->sc_coeff_h);
+	vpdma_unmap_desc_buf(dev->shared->vpdma, &port->sc_coeff_v);
+	vpdma_unmap_desc_buf(dev->shared->vpdma, &stream->desc_list.buf);
+
+	vpdma_reset_desc_list(&stream->desc_list);
+
+	if (timeout <= 0) {
+		vip_err(stream, "Timed out setting up scaler through VPDMA list\n");
+		return -EBUSY;
+	}
+
+	return 0;
+}
+
+static int vip_load_vpdma_list_fifo(struct vip_stream *stream)
+{
+	struct vip_port *port = stream->port;
+	struct vip_dev *dev = port->dev;
+	struct vpdma_data *vpdma = dev->shared->vpdma;
+	int list_num = stream->list_num;
+	struct vip_buffer *buf;
+	unsigned long flags;
+	int timeout, i;
+
+	if (vpdma_list_busy(dev->shared->vpdma, stream->list_num))
+		return -EBUSY;
+
+	for (i = 0; i < VIP_VPDMA_FIFO_SIZE; i++) {
+		spin_lock_irqsave(&dev->slock, flags);
+		if (list_empty(&stream->vidq)) {
+			vip_err(stream, "No buffer left!");
+			spin_unlock_irqrestore(&dev->slock, flags);
+			return -EINVAL;
+		}
+
+		buf = list_entry(stream->vidq.next,
+				 struct vip_buffer, list);
+		buf->drop = false;
+
+		list_move_tail(&buf->list, &stream->post_bufs);
+		spin_unlock_irqrestore(&dev->slock, flags);
+
+		vip_dbg(2, stream, "%s: start_dma vb2 buf idx:%d\n",
+			__func__, buf->vb.vb2_buf.index);
+		start_dma(stream, buf);
+
+		timeout = 500;
+		while (vpdma_list_busy(vpdma, list_num) && timeout--)
+			usleep_range(1000, 1100);
+
+		if (timeout <= 0) {
+			vip_err(stream, "Timed out loading VPDMA list fifo\n");
+			return -EBUSY;
+		}
+	}
+	return 0;
+}
+
+static int vip_start_streaming(struct vb2_queue *vq, unsigned int count)
+{
+	struct vip_stream *stream = vb2_get_drv_priv(vq);
+	struct vip_port *port = stream->port;
+	struct vip_dev *dev = port->dev;
+	int ret;
+
+	vip_setup_scaler(stream);
+
+	/*
+	 * Make sure the scaler is configured before the datapath is
+	 * enabled. The scaler can only load the coefficient
+	 * parameters when it is idle. If the scaler path is enabled

drivers/media/platform/ti-vpe/vip.c line 2755

Quoted 10 lines
drivers/media/platform/ti-vpe/vip.c new file +10 -0
+	 * and video data is being received then the VPDMA transfer will
+	 * stall indefinetely.
+	 */
+	set_fmt_params(stream);
+	vip_setup_parser(port);
+
+	if (port->subdev) {
+		ret = v4l2_subdev_call(port->subdev, video, s_stream, 1);
+		if (ret < 0 && ret != -ENOIOCTLCMD) {
+			vip_dbg(1, stream, "stream on failed in subdev\n");

On error, all pending buffers need to be returned to vb2 with vb2_buffer_done and state VB2_BUF_STATE_QUEUED.

Quoted 721 lines
drivers/media/platform/ti-vpe/vip.c new file +721 -0
+			return ret;
+		}
+	}
+
+	stream->sequence = 0;
+	if (stream->port->flags & FLAG_INTERLACED)
+		stream->field = V4L2_FIELD_TOP;
+	populate_desc_list(stream);
+
+	ret = vip_load_vpdma_list_fifo(stream);
+	if (ret)
+		return ret;
+
+	stream->num_recovery = 0;
+
+	clear_irqs(dev, dev->slice_id, stream->list_num);
+	enable_irqs(dev, dev->slice_id, stream->list_num);
+	vip_schedule_next_buffer(stream);
+	vip_parser_stop_imm(port, false);
+	vip_enable_parser(port, true);
+
+	return 0;
+}
+
+/*
+ * Abort streaming and wait for last buffer
+ */
+static void vip_stop_streaming(struct vb2_queue *vq)
+{
+	struct vip_stream *stream = vb2_get_drv_priv(vq);
+	struct vip_port *port = stream->port;
+	struct vip_dev *dev = port->dev;
+	struct vip_buffer *buf;
+	int ret;
+
+	vip_dbg(2, stream, "%s:\n", __func__);
+
+	vip_parser_stop_imm(port, true);
+	vip_enable_parser(port, false);
+	unset_fmt_params(stream);
+
+	disable_irqs(dev, dev->slice_id, stream->list_num);
+	clear_irqs(dev, dev->slice_id, stream->list_num);
+
+	if (port->subdev) {
+		ret = v4l2_subdev_call(port->subdev, video, s_stream, 0);
+		if (ret < 0 && ret != -ENOIOCTLCMD)
+			vip_dbg(1, stream, "stream on failed in subdev\n");
+	}
+
+	stop_dma(stream, true);
+
+	/* release all active buffers */
+	while (!list_empty(&stream->post_bufs)) {
+		buf = list_entry(stream->post_bufs.next,
+				 struct vip_buffer, list);
+		list_del(&buf->list);
+		if (buf->drop == 1)
+			list_add_tail(&buf->list, &stream->dropq);
+		else
+			vb2_buffer_done(&buf->vb.vb2_buf, VB2_BUF_STATE_ERROR);
+	}
+	while (!list_empty(&stream->vidq)) {
+		buf = list_entry(stream->vidq.next, struct vip_buffer, list);
+		list_del(&buf->list);
+		vb2_buffer_done(&buf->vb.vb2_buf, VB2_BUF_STATE_ERROR);
+	}
+
+	if (!vb2_is_streaming(vq))
+		return;
+
+	vpdma_unmap_desc_buf(dev->shared->vpdma, &stream->desc_list.buf);
+	vpdma_reset_desc_list(&stream->desc_list);
+}
+
+static const struct vb2_ops vip_video_qops = {
+	.queue_setup		= vip_queue_setup,
+	.buf_prepare		= vip_buf_prepare,
+	.buf_queue		= vip_buf_queue,
+	.start_streaming	= vip_start_streaming,
+	.stop_streaming		= vip_stop_streaming,
+	.wait_prepare		= vb2_ops_wait_prepare,
+	.wait_finish		= vb2_ops_wait_finish,
+};
+
+/*
+ * File operations
+ */
+
+static int vip_init_dev(struct vip_dev *dev)
+{
+	if (dev->num_ports != 0)
+		goto done;
+
+	vip_set_clock_enable(dev, 1);
+	vip_module_reset(dev, VIP_SC_RST, false);
+	vip_module_reset(dev, VIP_CSC_RST, false);
+done:
+	dev->num_ports++;
+
+	return 0;
+}
+
+static inline bool is_scaler_available(struct vip_port *port)
+{
+	if (port->endpoint.bus_type == V4L2_MBUS_PARALLEL)
+		if (port->dev->sc_assigned == VIP_NOT_ASSIGNED ||
+		    port->dev->sc_assigned == port->port_id)
+			return true;
+	return false;
+}
+
+static inline bool allocate_scaler(struct vip_port *port)
+{
+	if (is_scaler_available(port)) {
+		if (port->dev->sc_assigned == VIP_NOT_ASSIGNED ||
+		    port->dev->sc_assigned == port->port_id) {
+			port->dev->sc_assigned = port->port_id;
+			port->scaler = true;
+			return true;
+		}
+	}
+	return false;
+}
+
+static inline void free_scaler(struct vip_port *port)
+{
+	if (port->dev->sc_assigned == port->port_id) {
+		port->dev->sc_assigned = VIP_NOT_ASSIGNED;
+		port->scaler = false;
+	}
+}
+
+static bool is_csc_available(struct vip_port *port)
+{
+	if (port->endpoint.bus_type == V4L2_MBUS_PARALLEL)
+		if (port->dev->csc_assigned == VIP_NOT_ASSIGNED ||
+		    port->dev->csc_assigned == port->port_id)
+			return true;
+	return false;
+}
+
+static bool allocate_csc(struct vip_port *port,
+				enum vip_csc_state csc_direction)
+{
+	/* Is CSC needed? */
+	if (csc_direction != VIP_CSC_NA) {
+		if (is_csc_available(port)) {
+			port->dev->csc_assigned = port->port_id;
+			port->csc = csc_direction;
+			vip_dbg(1, port, "%s: csc allocated: dir: %d\n",
+				__func__, csc_direction);
+			return true;
+		}
+	}
+	return false;
+}
+
+static void free_csc(struct vip_port *port)
+{
+	if (port->dev->csc_assigned == port->port_id) {
+		port->dev->csc_assigned = VIP_NOT_ASSIGNED;
+		port->csc = VIP_CSC_NA;
+		vip_dbg(1, port, "%s: csc freed\n",
+			__func__);
+	}
+}
+
+static int vip_init_port(struct vip_port *port)
+{
+	int ret;
+	struct vip_fmt *fmt;
+	struct v4l2_subdev_format sd_fmt;
+	struct v4l2_mbus_framefmt *mbus_fmt = &sd_fmt.format;
+
+	if (port->num_streams != 0)
+		goto done;
+
+	ret = vip_init_dev(port->dev);
+	if (ret)
+		goto done;
+
+	/* Get subdevice current frame format */
+	sd_fmt.which = V4L2_SUBDEV_FORMAT_ACTIVE;
+	sd_fmt.pad = port->source_pad;
+	ret = v4l2_subdev_call(port->subdev, pad, get_fmt, NULL, &sd_fmt);
+	if (ret)
+		vip_dbg(1, port, "init_port get_fmt failed in subdev: (%d)\n",
+			ret);
+
+	/* try to find one that matches */
+	fmt = find_port_format_by_code(port, mbus_fmt->code);
+	if (!fmt) {
+		vip_dbg(1, port, "subdev default mbus_fmt %04x is not matched.\n",
+			mbus_fmt->code);
+		/* if all else fails just pick the first one */
+		fmt = port->active_fmt[0];
+
+		mbus_fmt->code = fmt->code;
+		sd_fmt.which = V4L2_SUBDEV_FORMAT_ACTIVE;
+		sd_fmt.pad = port->source_pad;
+		ret = v4l2_subdev_call(port->subdev, pad, set_fmt,
+				       NULL, &sd_fmt);
[… more lines in the raw message]

Lines of this file are not shown. View the raw message.

The rest of this diff is not shown. View the raw message.

drivers/media/platform/ti-vpe/vip.c line 2968

Quoted 10 lines
drivers/media/platform/ti-vpe/vip.c new file +10 -0
+		if (ret)
+			vip_dbg(1, port, "init_port set_fmt failed in subdev: (%d)\n",
+				ret);
+	}
+
+	/* Assign current format */
+	port->fmt = fmt;
+	port->mbus_framefmt = *mbus_fmt;
+
+	vip_dbg(3, port, "%s: g_mbus_fmt subdev mbus_code: %04X fourcc:%s size: %dx%d\n",
[… more lines in the raw message]

Shouldn't this be done before the call to video_register_device()?

drivers/media/platform/ti-vpe/vip.h

Quoted 1,403 lines
drivers/media/platform/ti-vpe/vip.c new file +672 -0

Lines of this file are not shown. View the raw message.

drivers/media/platform/ti-vpe/vip.h new file +724 -0
new file mode 100644
index 000000000000..f078a16a85b7

Lines of this file are not shown. View the raw message.

The rest of this diff is not shown. View the raw message.

Regards,

Hans

Part of the diff is not shown here. View the raw message.

#6 lore raw

Hans,

Thanks for the review.

Hans Verkuil <[email protected]> wrote on Tue [2020-May-26 13:48:35 +0200]:

Quoted 126 lines · Benoit Parrot
Quoted 125 lines

VIP stands for Video Input Port, it can be found on devices such as DRA7xx and provides a parallel interface to a video source such as a sensor or TV decoder. Each VIP can support two inputs (slices) and a SoC can be configured with a variable number of VIP's. Each slice can supports two ports each connected to its own sub-device.

---
 drivers/media/platform/Kconfig         |   13 +
 drivers/media/platform/ti-vpe/Makefile |    2 +
 drivers/media/platform/ti-vpe/vip.c    | 4158 ++++++++++++++++++++++++
 drivers/media/platform/ti-vpe/vip.h    |  724 +++++
 4 files changed, 4897 insertions(+)
 create mode 100644 drivers/media/platform/ti-vpe/vip.c
 create mode 100644 drivers/media/platform/ti-vpe/vip.h
drivers/media/platform/Kconfig +13 -0
index c57ee78fa99d..f4100a1aad58 100644
@@ -168,6 +168,19 @@ config VIDEO_TI_CAL
 	  In TI Technical Reference Manual this module is referred as
 	  Camera Interface Subsystem (CAMSS).
 
+config VIDEO_TI_VIP
+	tristate "TI Video Input Port"
+	default n
+	depends on VIDEO_DEV && VIDEO_V4L2 && SOC_DRA7XX
+	depends on HAS_DMA
+	select VIDEOBUF2_DMA_CONTIG
+	select VIDEO_TI_VPDMA
+	select VIDEO_TI_SC
+	select VIDEO_TI_CSC
+	help
+	Driver support for VIP module on certain TI SoC's
+	VIP = Video Input Port.
+
 endif # V4L_PLATFORM_DRIVERS
 
 menuconfig V4L_MEM2MEM_DRIVERS
drivers/media/platform/ti-vpe/Makefile +2 -0
index 886ac5ec073f..cdbecadf7191 100644
@@ -3,11 +3,13 @@ obj-$(CONFIG_VIDEO_TI_VPE) += ti-vpe.o
 obj-$(CONFIG_VIDEO_TI_VPDMA) += ti-vpdma.o
 obj-$(CONFIG_VIDEO_TI_SC) += ti-sc.o
 obj-$(CONFIG_VIDEO_TI_CSC) += ti-csc.o
+obj-$(CONFIG_VIDEO_TI_VIP) += ti-vip.o
 
 ti-vpe-y := vpe.o
 ti-vpdma-y := vpdma.o
 ti-sc-y := sc.o
 ti-csc-y := csc.o
+ti-vip-y := vip.o
 
 ccflags-$(CONFIG_VIDEO_TI_VPE_DEBUG) += -DDEBUG
 
drivers/media/platform/ti-vpe/vip.c new file +59 -0
new file mode 100644
index 000000000000..307b01851a14
@@ -0,0 +1,4158 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * TI VIP capture driver
+ *
+ * Copyright (C) 2018 Texas Instruments Incorporated -  http://www.ti.com/
+ * David Griego, <[email protected]>
+ * Dale Farnsworth, <[email protected]>
+ * Nikhil Devshatwar, <[email protected]>
+ * Benoit Parrot, <[email protected]>
+ */
+
+#include <linux/clk.h>
+#include <linux/delay.h>
+#include <linux/dma-mapping.h>
+#include <linux/err.h>
+#include <linux/interrupt.h>
+#include <linux/module.h>
+#include <linux/workqueue.h>
+#include <linux/pm_runtime.h>
+#include <linux/sched.h>
+#include <linux/mfd/syscon.h>
+#include <linux/regmap.h>
+
+#include <linux/pinctrl/consumer.h>
+#include <linux/of_device.h>
+#include <linux/of_graph.h>
+
+#include "vip.h"
+
+#define VIP_MODULE_NAME "vip"
+
+static int debug;
+module_param(debug, int, 0644);
+MODULE_PARM_DESC(debug, "debug level (0-8)");
+
+/*
+ * Minimum and maximum frame sizes
+ */
+#define MIN_W		128
+#define MIN_H		128
+#define MAX_W		2048
+#define MAX_H		1536
+
+/*
+ * Required alignments
+ */
+#define S_ALIGN		0 /* multiple of 1 */
+#define H_ALIGN		1 /* multiple of 2 */
+#define W_ALIGN		1 /* multiple of 2 */
+#define L_ALIGN		7 /* multiple of 128, line stride, 16 bytes */
+
+/*
+ * Need a descriptor entry for each of up to 15 outputs,
+ * and up to 2 control transfers.
+ */
+#define VIP_DESC_LIST_SIZE	(17 * sizeof(struct vpdma_dtd))
+
+#define vip_dbg(level, dev, fmt, arg...)	\
+		v4l2_dbg(level, debug, dev, fmt, ##arg)
Quoted 8 lines

drivers/media/platform/ti-vpe/vip.c line 65

Quoted 6 lines
drivers/media/platform/ti-vpe/vip.c new file +6 -0
+#define vip_err(dev, fmt, arg...)	\
+		v4l2_err(dev, fmt, ##arg)
+#define vip_warn(dev, fmt, arg...)	\
+		v4l2_err(dev, fmt, ##arg)
+#define vip_info(dev, fmt, arg...)	\
+		v4l2_info(dev, fmt, ##arg)

These defines are just aliases. Any reason for not just using v4l2_info etc.?

Yeah at some point the macros were abstracting the &dev->v4l2_dev but not any longeri since I figured out they are only using the ->name member, so I guess I could remove them.

Quoted 60 lines
Quoted 47 lines
drivers/media/platform/ti-vpe/vip.c new file +46 -0
+
+#define CTRL_CORE_SMA_SW_1      0x534
+/*
+ * The srce_info structure contains per-srce data.
+ */
+struct vip_srce_info {
+	u8	base_channel;	/* the VPDMA channel nummber */
+	u8	vb_index;	/* input frame f, f-1, f-2 index */
+	u8	vb_part;	/* identifies section of co-planar formats */
+};
+
+#define VIP_VPDMA_FIFO_SIZE	2
+#define VIP_DROPQ_SIZE		3
+
+/*
+ * Define indices into the srce_info tables
+ */
+
+#define VIP_SRCE_MULT_PORT		0
+#define VIP_SRCE_MULT_ANC		1
+#define VIP_SRCE_LUMA		2
+#define VIP_SRCE_CHROMA		3
+#define VIP_SRCE_RGB		4
+
+static struct vip_srce_info srce_info[5] = {
+	[VIP_SRCE_MULT_PORT] = {
+		.base_channel	= VIP1_CHAN_NUM_MULT_PORT_A_SRC0,
+		.vb_index	= 0,
+		.vb_part	= VIP_CHROMA,
+	},
+	[VIP_SRCE_MULT_ANC] = {
+		.base_channel	= VIP1_CHAN_NUM_MULT_ANC_A_SRC0,
+		.vb_index	= 0,
+		.vb_part	= VIP_LUMA,
+	},
+	[VIP_SRCE_LUMA] = {
+		.base_channel	= VIP1_CHAN_NUM_PORT_A_LUMA,
+		.vb_index	= 1,
+		.vb_part	= VIP_LUMA,
+	},
+	[VIP_SRCE_CHROMA] = {
+		.base_channel	= VIP1_CHAN_NUM_PORT_A_CHROMA,
+		.vb_index	= 1,
+		.vb_part	= VIP_CHROMA,
+	},
+	[VIP_SRCE_RGB] = {

drivers/media/platform/ti-vpe/vip.c line 121

Quoted 10 lines
drivers/media/platform/ti-vpe/vip.c new file +10 -0
+		.base_channel	= VIP1_CHAN_NUM_PORT_A_RGB,
+		.vb_part	= VIP_LUMA,
+	},
+};
+
+static struct vip_fmt vip_formats[VIP_MAX_ACTIVE_FMT] = {
+	{
+		.fourcc		= V4L2_PIX_FMT_NV12,
+		.code		= MEDIA_BUS_FMT_UYVY8_2X8,
+		.colorspace	= V4L2_COLORSPACE_SMPTE170M,

Colorspace information should come from the subdev (sensor or video receiver).

Quoted 1 line

You really don't know what the colorspace is here.

Ah yes, this has been here for a long time...
I'll clean it up.

Quoted 85 lines
Quoted 85 lines
drivers/media/platform/ti-vpe/vip.c new file +84 -0
+		.coplanar	= 1,
+		.vpdma_fmt	= { &vpdma_yuv_fmts[VPDMA_DATA_FMT_Y420],
+				    &vpdma_yuv_fmts[VPDMA_DATA_FMT_C420],
+				  },
+	},
+	{
+		.fourcc		= V4L2_PIX_FMT_UYVY,
+		.code		= MEDIA_BUS_FMT_UYVY8_2X8,
+		.colorspace	= V4L2_COLORSPACE_SMPTE170M,
+		.coplanar	= 0,
+		.vpdma_fmt	= { &vpdma_yuv_fmts[VPDMA_DATA_FMT_CBY422],
+				  },
+	},
+	{
+		.fourcc		= V4L2_PIX_FMT_YUYV,
+		.code		= MEDIA_BUS_FMT_UYVY8_2X8,
+		.colorspace	= V4L2_COLORSPACE_SMPTE170M,
+		.coplanar	= 0,
+		.vpdma_fmt	= { &vpdma_yuv_fmts[VPDMA_DATA_FMT_YCB422],
+				  },
+	},
+	{
+		.fourcc		= V4L2_PIX_FMT_VYUY,
+		.code		= MEDIA_BUS_FMT_UYVY8_2X8,
+		.colorspace	= V4L2_COLORSPACE_SMPTE170M,
+		.coplanar	= 0,
+		.vpdma_fmt	= { &vpdma_yuv_fmts[VPDMA_DATA_FMT_CRY422],
+				  },
+	},
+	{
+		.fourcc		= V4L2_PIX_FMT_YVYU,
+		.code		= MEDIA_BUS_FMT_UYVY8_2X8,
+		.colorspace	= V4L2_COLORSPACE_SMPTE170M,
+		.coplanar	= 0,
+		.vpdma_fmt	= { &vpdma_yuv_fmts[VPDMA_DATA_FMT_YCR422],
+				  },
+	},
+	{
+		.fourcc		= V4L2_PIX_FMT_RGB24,
+		.code		= MEDIA_BUS_FMT_UYVY8_2X8,
+		.colorspace	= V4L2_COLORSPACE_SRGB,
+		.coplanar	= 0,
+		.vpdma_fmt	= { &vpdma_rgb_fmts[VPDMA_DATA_FMT_RGB24],
+				  },
+	},
+	{
+		.fourcc		= V4L2_PIX_FMT_RGB32,
+		.code		= MEDIA_BUS_FMT_UYVY8_2X8,
+		.colorspace	= V4L2_COLORSPACE_SRGB,
+		.coplanar	= 0,
+		.vpdma_fmt	= { &vpdma_rgb_fmts[VPDMA_DATA_FMT_ARGB32],
+				  },
+	},
+	{
+		.fourcc		= V4L2_PIX_FMT_BGR24,
+		.code		= MEDIA_BUS_FMT_UYVY8_2X8,
+		.colorspace	= V4L2_COLORSPACE_SRGB,
+		.coplanar	= 0,
+		.vpdma_fmt	= { &vpdma_rgb_fmts[VPDMA_DATA_FMT_BGR24],
+				  },
+	},
+	{
+		.fourcc		= V4L2_PIX_FMT_BGR32,
+		.code		= MEDIA_BUS_FMT_UYVY8_2X8,
+		.colorspace	= V4L2_COLORSPACE_SRGB,
+		.coplanar	= 0,
+		.vpdma_fmt	= { &vpdma_rgb_fmts[VPDMA_DATA_FMT_ABGR32],
+				  },
+	},
+	{
+		.fourcc		= V4L2_PIX_FMT_RGB24,
+		.code		= MEDIA_BUS_FMT_RGB888_1X24,
+		.colorspace	= V4L2_COLORSPACE_SRGB,
+		.coplanar	= 0,
+		.vpdma_fmt	= { &vpdma_rgb_fmts[VPDMA_DATA_FMT_RGB24],
+				  },
+	},
+	{
+		.fourcc		= V4L2_PIX_FMT_RGB32,
+		.code		= MEDIA_BUS_FMT_ARGB8888_1X32,
+		.colorspace	= V4L2_COLORSPACE_SRGB,
+		.coplanar	= 0,
+		.vpdma_fmt	= { &vpdma_rgb_fmts[VPDMA_DATA_FMT_ARGB32],
+				  },
Quoted 8 lines

drivers/media/platform/ti-vpe/vip.c line 210

Quoted 5 lines
drivers/media/platform/ti-vpe/vip.c new file +5 -0
+	},
+	{
+		.fourcc		= V4L2_PIX_FMT_SBGGR8,
+		.code		= MEDIA_BUS_FMT_SBGGR8_1X8,
+		.colorspace	= V4L2_COLORSPACE_SMPTE170M,

SMPTE 170M for a Bayer format? That can't be right. But as mentioned above, this should come from the subdev.

Same here.

Quoted 116 lines
Quoted 116 lines
drivers/media/platform/ti-vpe/vip.c new file +115 -0
+		.coplanar	= 0,
+		.vpdma_fmt	= { &vpdma_raw_fmts[VPDMA_DATA_FMT_RAW8],
+				  },
+	},
+	{
+		.fourcc		= V4L2_PIX_FMT_SGBRG8,
+		.code		= MEDIA_BUS_FMT_SGBRG8_1X8,
+		.colorspace	= V4L2_COLORSPACE_SMPTE170M,
+		.coplanar	= 0,
+		.vpdma_fmt	= { &vpdma_raw_fmts[VPDMA_DATA_FMT_RAW8],
+				  },
+	},
+	{
+		.fourcc		= V4L2_PIX_FMT_SGRBG8,
+		.code		= MEDIA_BUS_FMT_SGRBG8_1X8,
+		.colorspace	= V4L2_COLORSPACE_SMPTE170M,
+		.coplanar	= 0,
+		.vpdma_fmt	= { &vpdma_raw_fmts[VPDMA_DATA_FMT_RAW8],
+				  },
+	},
+	{
+		.fourcc		= V4L2_PIX_FMT_SRGGB8,
+		.code		= MEDIA_BUS_FMT_SRGGB8_1X8,
+		.colorspace	= V4L2_COLORSPACE_SMPTE170M,
+		.coplanar	= 0,
+		.vpdma_fmt	= { &vpdma_raw_fmts[VPDMA_DATA_FMT_RAW8],
+				  },
+	},
+	{
+		/* V4L2 currently only defines one 16 bit variant */
+		.fourcc		= V4L2_PIX_FMT_SBGGR16,
+		.code		= MEDIA_BUS_FMT_SBGGR16_1X16,
+		.colorspace	= V4L2_COLORSPACE_SMPTE170M,
+		.coplanar	= 0,
+		.vpdma_fmt	= { &vpdma_raw_fmts[VPDMA_DATA_FMT_RAW16],
+				  },
+	},
+};
+
+/* initialize  v4l2_format_info member in vip_formats array */
+static void vip_init_format_info(struct device *dev)
+{
+	struct vip_fmt *fmt;
+	int i;
+
+	for (i = 0; i < ARRAY_SIZE(vip_formats); i++) {
+		fmt = &vip_formats[i];
+		fmt->finfo = v4l2_format_info(fmt->fourcc);
+	}
+}
+
+/*  Print Four-character-code (FOURCC) */
+static char *fourcc_to_str(u32 fmt)
+{
+	static char code[5];
+
+	code[0] = (unsigned char)(fmt & 0xff);
+	code[1] = (unsigned char)((fmt >> 8) & 0xff);
+	code[2] = (unsigned char)((fmt >> 16) & 0xff);
+	code[3] = (unsigned char)((fmt >> 24) & 0xff);
+	code[4] = '\0';
+
+	return code;
+}
+
+/*
+ * Find our format description corresponding to the passed v4l2_format
+ */
+
+static struct vip_fmt *find_port_format_by_pix(struct vip_port *port,
+					       u32 pixelformat)
+{
+	struct vip_fmt *fmt;
+	unsigned int k;
+
+	for (k = 0; k < port->num_active_fmt; k++) {
+		fmt = port->active_fmt[k];
+		if (fmt->fourcc == pixelformat)
+			return fmt;
+	}
+
+	return NULL;
+}
+
+static struct vip_fmt *find_port_format_by_code(struct vip_port *port,
+						u32 code)
+{
+	struct vip_fmt *fmt;
+	unsigned int k;
+
+	for (k = 0; k < port->num_active_fmt; k++) {
+		fmt = port->active_fmt[k];
+		if (fmt->code == code)
+			return fmt;
+	}
+
+	return NULL;
+}
+
+static int vip_find_pad(struct v4l2_subdev *sd, int direction)
+{
+	unsigned int pad;
+
+	if (sd->entity.num_pads <= 1)
+		return 0;
+
+	for (pad = 0; pad < sd->entity.num_pads; pad++)
+		if (sd->entity.pads[pad].flags & direction)
+			return pad;
+
+	return -EINVAL;
+}
+
+inline struct vip_port *notifier_to_vip_port(struct v4l2_async_notifier *n)
+{
Quoted 8 lines

drivers/media/platform/ti-vpe/vip.c line 331

Quoted 6 lines
drivers/media/platform/ti-vpe/vip.c new file +6 -0
+	return container_of(n, struct vip_port, notifier);
+}
+
+static bool vip_is_mbuscode_yuv(u32 code)
+{
+	return ((code & 0xFF00) == 0x2000);

Use lower-case hex.

Will do.

Quoted 720 lines
Quoted 720 lines
drivers/media/platform/ti-vpe/vip.c new file +719 -0
+}
+
+static bool vip_is_mbuscode_rgb(u32 code)
+{
+	return ((code & 0xFF00) == 0x1000);
+}
+
+static bool vip_is_mbuscode_raw(u32 code)
+{
+	return ((code & 0xFF00) == 0x3000);
+}
+
+/*
+ * This is not an accurate conversion but it is only used to
+ * assess if color conversion is needed.
+ */
+static u32 vip_mbus_code_to_fourcc(u32 code)
+{
+	if (vip_is_mbuscode_rgb(code))
+		return V4L2_PIX_FMT_RGB24;
+
+	if (vip_is_mbuscode_yuv(code))
+		return V4L2_PIX_FMT_UYVY;
+
+	return V4L2_PIX_FMT_SBGGR8;
+}
+
+static enum vip_csc_state
+vip_csc_direction(u32 src_code, const struct v4l2_format_info *dfinfo)
+{
+	if (vip_is_mbuscode_yuv(src_code) && v4l2_is_format_rgb(dfinfo))
+		return VIP_CSC_Y2R;
+	else if (vip_is_mbuscode_rgb(src_code) && v4l2_is_format_yuv(dfinfo))
+		return VIP_CSC_R2Y;
+	else
+		return VIP_CSC_NA;
+}
+
+/*
+ * port flag bits
+ */
+#define FLAG_FRAME_1D		BIT(0)
+#define FLAG_EVEN_LINE_SKIP	BIT(1)
+#define FLAG_ODD_LINE_SKIP	BIT(2)
+#define FLAG_MODE_TILED		BIT(3)
+#define FLAG_INTERLACED		BIT(4)
+#define FLAG_MULTIPLEXED	BIT(5)
+#define FLAG_MULT_PORT		BIT(6)
+#define FLAG_MULT_ANC		BIT(7)
+
+/*
+ * Function prototype declarations
+ */
+static int alloc_port(struct vip_dev *, int, const char *);
+static void free_port(struct vip_port *);
+static int vip_setup_parser(struct vip_port *port);
+static int vip_setup_scaler(struct vip_stream *stream);
+static void vip_enable_parser(struct vip_port *port, bool on);
+static void vip_reset_parser(struct vip_port *port, bool on);
+static void vip_parser_stop_imm(struct vip_port *port, bool on);
+static void stop_dma(struct vip_stream *stream, bool clear_list);
+static int vip_load_vpdma_list_fifo(struct vip_stream *stream);
+static inline bool is_scaler_available(struct vip_port *port);
+static inline bool allocate_scaler(struct vip_port *port);
+static inline void free_scaler(struct vip_port *port);
+static bool is_csc_available(struct vip_port *port);
+static bool allocate_csc(struct vip_port *port,
+				enum vip_csc_state csc_direction);
+static void free_csc(struct vip_port *port);
+
+#define reg_read(dev, offset) ioread32(dev->base + offset)
+#define reg_write(dev, offset, val) iowrite32(val, dev->base + offset)
+
+/*
+ * Insert a masked field into a 32-bit field
+ */
+static void insert_field(u32 *valp, u32 field, u32 mask, int shift)
+{
+	u32 val = *valp;
+
+	val &= ~(mask << shift);
+	val |= (field & mask) << shift;
+	*valp = val;
+}
+
+/*
+ * DMA address/data block for the shadow registers
+ */
+struct vip_mmr_adb {
+	struct vpdma_adb_hdr	sc_hdr0;
+	u32			sc_regs0[7];
+	u32			sc_pad0[1];
+	struct vpdma_adb_hdr	sc_hdr8;
+	u32			sc_regs8[6];
+	u32			sc_pad8[2];
+	struct vpdma_adb_hdr	sc_hdr17;
+	u32			sc_regs17[9];
+	u32			sc_pad17[3];
+	struct vpdma_adb_hdr	csc_hdr;
+	u32			csc_regs[6];
+	u32			csc_pad[2];
+};
+
+#define GET_OFFSET_TOP(port, obj, reg)	\
+	((obj)->res->start - port->dev->res->start + reg)
+
+#define VIP_SET_MMR_ADB_HDR(port, hdr, regs, offset_a)	\
+	VPDMA_SET_MMR_ADB_HDR(port->mmr_adb, vip_mmr_adb, hdr, regs, offset_a)
+
+/*
+ * Set the headers for all of the address/data block structures.
+ */
+static void init_adb_hdrs(struct vip_port *port)
+{
+	VIP_SET_MMR_ADB_HDR(port, sc_hdr0, sc_regs0,
+			    GET_OFFSET_TOP(port, port->dev->sc, CFG_SC0));
+	VIP_SET_MMR_ADB_HDR(port, sc_hdr8, sc_regs8,
+			    GET_OFFSET_TOP(port, port->dev->sc, CFG_SC8));
+	VIP_SET_MMR_ADB_HDR(port, sc_hdr17, sc_regs17,
+			    GET_OFFSET_TOP(port, port->dev->sc, CFG_SC17));
+	VIP_SET_MMR_ADB_HDR(port, csc_hdr, csc_regs,
+			    GET_OFFSET_TOP(port, port->dev->csc, CSC_CSC00));
+
+};
+
+/*
+ * These represent the module resets bit for slice 1
+ * Upon detecting slice2 we simply left shift by 1
+ */
+#define VIP_DP_RST	BIT(16)
+#define VIP_PARSER_RST	BIT(18)
+#define VIP_CSC_RST	BIT(20)
+#define VIP_SC_RST	BIT(22)
+#define VIP_DS0_RST	BIT(25)
+#define VIP_DS1_RST	BIT(27)
+
+static void vip_module_reset(struct vip_dev *dev, uint32_t module, bool on)
+{
+	u32 val = 0;
+
+	val = reg_read(dev, VIP_CLK_RESET);
+
+	if (dev->slice_id == VIP_SLICE2)
+		module <<= 1;
+
+	if (on)
+		val |= module;
+	else
+		val &= ~module;
+
+	reg_write(dev, VIP_CLK_RESET, val);
+}
+
+/*
+ * Enable or disable the VIP clocks
+ */
+static void vip_set_clock_enable(struct vip_dev *dev, bool on)
+{
+	u32 val = 0;
+
+	val = reg_read(dev, VIP_CLK_ENABLE);
+	if (on) {
+		val |= VIP_VPDMA_CLK_ENABLE;
+		if (dev->slice_id == VIP_SLICE1)
+			val |= VIP_VIP1_DATA_PATH_CLK_ENABLE;
+		else
+			val |= VIP_VIP2_DATA_PATH_CLK_ENABLE;
+	} else {
+		if (dev->slice_id == VIP_SLICE1)
+			val &= ~VIP_VIP1_DATA_PATH_CLK_ENABLE;
+		else
+			val &= ~VIP_VIP2_DATA_PATH_CLK_ENABLE;
+
+		/* Both VIP are disabled then shutdown VPDMA also */
+		if (!(val & (VIP_VIP1_DATA_PATH_CLK_ENABLE |
+			     VIP_VIP2_DATA_PATH_CLK_ENABLE)))
+			val = 0;
+	}
+
+	reg_write(dev, VIP_CLK_ENABLE, val);
+}
+
+/* This helper function is used to enable the clock early on to
+ * enable vpdma firmware loading before the slice device are created
+ */
+static void vip_shared_set_clock_enable(struct vip_shared *shared, bool on)
+{
+	u32 val = 0;
+
+	if (on)
+		val = VIP_VIP1_DATA_PATH_CLK_ENABLE | VIP_VPDMA_CLK_ENABLE;
+
+	reg_write(shared, VIP_CLK_ENABLE, val);
+}
+
+static void vip_top_reset(struct vip_dev *dev)
+{
+	u32 val = 0;
+
+	val = reg_read(dev, VIP_CLK_RESET);
+
+	if (dev->slice_id == VIP_SLICE1)
+		insert_field(&val, 1, VIP_DATA_PATH_CLK_RESET_MASK,
+			     VIP_VIP1_DATA_PATH_RESET_SHIFT);
+	else
+		insert_field(&val, 1, VIP_DATA_PATH_CLK_RESET_MASK,
+			     VIP_VIP2_DATA_PATH_RESET_SHIFT);
+
+	reg_write(dev, VIP_CLK_RESET, val);
+
+	usleep_range(200, 250);
+
+	val = reg_read(dev, VIP_CLK_RESET);
+
+	if (dev->slice_id == VIP_SLICE1)
+		insert_field(&val, 0, VIP_DATA_PATH_CLK_RESET_MASK,
+			     VIP_VIP1_DATA_PATH_RESET_SHIFT);
+	else
+		insert_field(&val, 0, VIP_DATA_PATH_CLK_RESET_MASK,
+			     VIP_VIP2_DATA_PATH_RESET_SHIFT);
+	reg_write(dev, VIP_CLK_RESET, val);
+}
+
+static void vip_top_vpdma_reset(struct vip_shared *shared)
+{
+	u32 val;
+
+	val = reg_read(shared, VIP_CLK_RESET);
+	insert_field(&val, 1, VIP_VPDMA_CLK_RESET_MASK,
+		     VIP_VPDMA_CLK_RESET_SHIFT);
+	reg_write(shared, VIP_CLK_RESET, val);
+
+	usleep_range(200, 250);
+
+	val = reg_read(shared, VIP_CLK_RESET);
+	insert_field(&val, 0, VIP_VPDMA_CLK_RESET_MASK,
+		     VIP_VPDMA_CLK_RESET_SHIFT);
+	reg_write(shared, VIP_CLK_RESET, val);
+}
+
+static void vip_set_pclk_invert(struct vip_port *port)
+{
+	struct vip_clk_polarity *pclk = port->dev->pclk_pol;
+	u32 index;
+
+	/*
+	 * When the VIP parser is configured to so that the pixel clock
+	 * is to be sampled at falling edge, the pixel clock needs to be
+	 * inverted before it is given to the VIP module. This is done
+	 * by setting a bit in the CTRL_CORE_SMA_SW1 register.
+	 */
+
+	index = 2 * port->dev->slice_id + port->port_id;
+	vip_dbg(3, port, "%s: slice%d:port%d -> index: %d\n", __func__,
+		port->dev->slice_id, port->port_id, index);
+
+	if (pclk->rm_pol)
+		regmap_update_bits(pclk->rm_pol,
+				   pclk->rm_offset,
+				   pclk->rm_bit_field[index],
+				   pclk->rm_bit_field[index]);
+}
+
+static void vip_clr_pclk_invert(struct vip_port *port)
+{
+	struct vip_clk_polarity *pclk = port->dev->pclk_pol;
+	u32 index;
+
+	index = 2 * port->dev->slice_id + port->port_id;
+	vip_dbg(3, port, "%s: slice%d:port%d -> index: %d\n", __func__,
+		port->dev->slice_id, port->port_id, index);
+
+	if (pclk->rm_pol)
+		regmap_update_bits(pclk->rm_pol, pclk->rm_offset,
+				   pclk->rm_bit_field[index], 0);
+}
+
+#define VIP_PARSER_PORT(p)	(VIP_PARSER_PORTA_0 + (p * 0x8U))
+#define VIP_PARSER_EXTRA_PORT(p)	(VIP_PARSER_PORTA_1 + (p * 0x8U))
+#define VIP_PARSER_CROP_H_PORT(p)	(VIP_PARSER_PORTA_EXTRA4 + (p * 0x10U))
+#define VIP_PARSER_CROP_V_PORT(p)	(VIP_PARSER_PORTA_EXTRA5 + (p * 0x10U))
+#define VIP_PARSER_STOP_IMM_PORT(p)	(VIP_PARSER_PORTA_EXTRA6 + (p * 0x4U))
+
+static void vip_set_data_interface(struct vip_port *port,
+				   enum data_interface_modes mode)
+{
+	u32 val = 0;
+
+	insert_field(&val, mode, VIP_DATA_INTERFACE_MODE_MASK,
+		     VIP_DATA_INTERFACE_MODE_SHFT);
+
+	reg_write(port->dev->parser, VIP_PARSER_MAIN_CFG, val);
+}
+
+static void vip_set_slice_path(struct vip_dev *dev,
+			       enum data_path_select data_path, u32 path_val)
+{
+	u32 val = 0;
+	int data_path_reg;
+
+	vip_dbg(3, dev, "%s:\n", __func__);
+
+	data_path_reg = VIP_VIP1_DATA_PATH_SELECT + 4 * dev->slice_id;
+
+	switch (data_path) {
+	case ALL_FIELDS_DATA_SELECT:
+		val |= path_val;
+		break;
+	case VIP_CSC_SRC_DATA_SELECT:
+		insert_field(&val, path_val, VIP_CSC_SRC_SELECT_MASK,
+			     VIP_CSC_SRC_SELECT_SHFT);
+		break;
+	case VIP_SC_SRC_DATA_SELECT:
+		insert_field(&val, path_val, VIP_SC_SRC_SELECT_MASK,
+			     VIP_SC_SRC_SELECT_SHFT);
+		break;
+	case VIP_RGB_SRC_DATA_SELECT:
+		val |= (path_val) ? VIP_RGB_SRC_SELECT : 0;
+		break;
+	case VIP_RGB_OUT_LO_DATA_SELECT:
+		val |= (path_val) ? VIP_RGB_OUT_LO_SRC_SELECT : 0;
+		break;
+	case VIP_RGB_OUT_HI_DATA_SELECT:
+		val |= (path_val) ? VIP_RGB_OUT_HI_SRC_SELECT : 0;
+		break;
+	case VIP_CHR_DS_1_SRC_DATA_SELECT:
+		insert_field(&val, path_val, VIP_DS1_SRC_SELECT_MASK,
+			     VIP_DS1_SRC_SELECT_SHFT);
+		break;
+	case VIP_CHR_DS_2_SRC_DATA_SELECT:
+		insert_field(&val, path_val, VIP_DS2_SRC_SELECT_MASK,
+			     VIP_DS2_SRC_SELECT_SHFT);
+		break;
+	case VIP_MULTI_CHANNEL_DATA_SELECT:
+		val |= (path_val) ? VIP_MULTI_CHANNEL_SELECT : 0;
+		break;
+	case VIP_CHR_DS_1_DATA_BYPASS:
+		val |= (path_val) ? VIP_DS1_BYPASS : 0;
+		break;
+	case VIP_CHR_DS_2_DATA_BYPASS:
+		val |= (path_val) ? VIP_DS2_BYPASS : 0;
+		break;
+	default:
+		vip_err(dev, "%s: data_path 0x%x is not valid\n",
+			__func__, data_path);
+		return;
+	}
+	insert_field(&val, data_path, VIP_DATAPATH_SELECT_MASK,
+		     VIP_DATAPATH_SELECT_SHFT);
+	reg_write(dev, data_path_reg, val);
+	vip_dbg(3, dev, "%s: DATA_PATH_SELECT(%08X): %08X\n", __func__,
+		data_path_reg, reg_read(dev, data_path_reg));
+}
+
+/*
+ * Return the vip_stream structure for a given struct file
+ */
+static inline struct vip_stream *file2stream(struct file *file)
+{
+	return video_drvdata(file);
+}
+
+/*
+ * Append a destination descriptor to the current descriptor list,
+ * setting up dma to the given srce.
+ */
+static int add_out_dtd(struct vip_stream *stream, int srce_type)
+{
+	struct vip_port *port = stream->port;
+	struct vip_dev *dev = port->dev;
+	struct vip_srce_info *sinfo = &srce_info[srce_type];
+	struct v4l2_rect *c_rect = &port->c_rect;
+	struct vip_fmt *fmt = port->fmt;
+	int channel, plane = 0;
+	int max_width, max_height;
+	dma_addr_t dma_addr;
+	u32 flags;
+	u32 width = stream->width;
+
+	channel = sinfo->base_channel;
+
+	switch (srce_type) {
+	case VIP_SRCE_MULT_PORT:
+	case VIP_SRCE_MULT_ANC:
+		if (port->port_id == VIP_PORTB)
+			channel += VIP_CHAN_MULT_PORTB_OFFSET;
+		channel += stream->stream_id;
+		flags = 0;
+		break;
+	case VIP_SRCE_CHROMA:
+		plane = 1;
+		/* fallthrough */
+	case VIP_SRCE_LUMA:
+		if (port->port_id == VIP_PORTB) {
+			if (port->scaler && !port->fmt->coplanar)
+				/*
+				 * In this case Port A Chroma channel
+				 * is used to carry Port B scaled YUV422
+				 */
+				channel += 1;
+			else
+				channel += VIP_CHAN_YUV_PORTB_OFFSET;
+		}
+		flags = port->flags;
+		break;
+	case VIP_SRCE_RGB:
+		if ((port->port_id == VIP_PORTB) ||
+		    ((port->port_id == VIP_PORTA) &&
+		     (port->csc == VIP_CSC_NA) &&
+		     v4l2_is_format_rgb(port->fmt->finfo)))
+			/*
+			 * RGB sensor only connect to Y_LO
+			 * channel i.e. port B channel.
+			 */
+			channel += VIP_CHAN_RGB_PORTB_OFFSET;
+		flags = port->flags;
+		break;
+	default:
+		vip_err(stream, "%s: srce_type 0x%x is not valid\n",
+			__func__, srce_type);
+		return -1;
+	}
+
+	if (dev->slice_id == VIP_SLICE2)
+		channel += VIP_CHAN_VIP2_OFFSET;
+
+	/* This is just for initialization purposes.
+	 * The actual dma_addr will be configured in vpdma_update_dma_addr
+	 */
+	dma_addr = 0;
+
+	if (port->fmt->vpdma_fmt[0] == &vpdma_raw_fmts[VPDMA_DATA_FMT_RAW8]) {
+		/*
+		 * Special case since we are faking a YUV422 16bit format
+		 * to have the vpdma perform the needed byte swap
+		 * we need to adjust the pixel width accordingly
+		 * otherwise the parser will attempt to collect more pixels
+		 * then available and the vpdma transfer will exceed the
+		 * allocated frame buffer.
+		 */
+		width >>= 1;
+		vip_dbg(1, stream, "%s: 8 bit raw detected, adjusting width to %d\n",
+			__func__, width);
+	}
+
+	/*
+	 * Use VPDMA_MAX_SIZE1 or VPDMA_MAX_SIZE2 register for slice0/1
+	 */
+
+	if (dev->slice_id == VIP_SLICE1) {
+		vpdma_set_max_size(dev->shared->vpdma, VPDMA_MAX_SIZE1,
+				   width, stream->height);
+
+		max_width = MAX_OUT_WIDTH_REG1;
+		max_height = MAX_OUT_HEIGHT_REG1;
+	} else {
+		vpdma_set_max_size(dev->shared->vpdma, VPDMA_MAX_SIZE2,
+				   width, stream->height);
+
+		max_width = MAX_OUT_WIDTH_REG2;
+		max_height = MAX_OUT_HEIGHT_REG2;
+	}
+
+	/*
+	 * Mark this channel to be cleared while cleaning up resources
+	 * This will make sure that an abort descriptor for this channel
+	 * would be submitted to VPDMA causing any ongoing  transaction to be
+	 * aborted and cleanup the VPDMA FSM for this channel
+	 */
+	stream->vpdma_channels[channel] = 1;
+
+	vpdma_rawchan_add_out_dtd(&stream->desc_list, c_rect->width,
+				  stream->bytesperline, c_rect,
+				  fmt->vpdma_fmt[plane], dma_addr,
+				  max_width, max_height, channel, flags);
+
+	return 0;
+}
+
+/*
+ * add_stream_dtds - prepares and starts DMA for pending transfers
+ */
+static void add_stream_dtds(struct vip_stream *stream)
+{
+	struct vip_port *port = stream->port;
+	int srce_type;
+
+	if (port->flags & FLAG_MULT_PORT)
+		srce_type = VIP_SRCE_MULT_PORT;
+	else if (port->flags & FLAG_MULT_ANC)
+		srce_type = VIP_SRCE_MULT_ANC;
+	else if (v4l2_is_format_rgb(port->fmt->finfo))
+		srce_type = VIP_SRCE_RGB;
+	else
+		srce_type = VIP_SRCE_LUMA;
+
+	add_out_dtd(stream, srce_type);
+
+	if (srce_type == VIP_SRCE_LUMA && port->fmt->coplanar)
+		add_out_dtd(stream, VIP_SRCE_CHROMA);
+}
+
+#define PARSER_IRQ_MASK (VIP_PORTA_OUTPUT_FIFO_YUV | \
+			 VIP_PORTB_OUTPUT_FIFO_YUV)
+
+static void enable_irqs(struct vip_dev *dev, int irq_num, int list_num)
+{
+	struct vip_parser_data *parser = dev->parser;
+	u32 reg_addr = VIP_INT0_ENABLE0_SET +
+			VIP_INTC_INTX_OFFSET * irq_num;
+	u32 irq_val = (1 << (list_num * 2)) |
+		      (VIP_VIP1_PARSER_INT << (irq_num * 1));
+
+	/* Enable Parser Interrupt */
+	reg_write(parser, VIP_PARSER_FIQ_MASK, ~PARSER_IRQ_MASK);
+
+	reg_write(dev->shared, reg_addr, irq_val);
+
+	vpdma_enable_list_complete_irq(dev->shared->vpdma,
+				       irq_num, list_num, true);
+}
+
+static void disable_irqs(struct vip_dev *dev, int irq_num, int list_num)
+{
+	struct vip_parser_data *parser = dev->parser;
+	u32 reg_addr = VIP_INT0_ENABLE0_CLR +
+			VIP_INTC_INTX_OFFSET * irq_num;
+	u32 irq_val = (1 << (list_num * 2)) |
+		      (VIP_VIP1_PARSER_INT << (irq_num * 1));
+
+	/* Disable all Parser Interrupt */
+	reg_write(parser, VIP_PARSER_FIQ_MASK, 0xffffffff);
+
+	reg_write(dev->shared, reg_addr, irq_val);
+
+	vpdma_enable_list_complete_irq(dev->shared->vpdma,
+				       irq_num, list_num, false);
+}
+
+static void clear_irqs(struct vip_dev *dev, int irq_num, int list_num)
+{
+	struct vip_parser_data *parser = dev->parser;
+	u32 reg_addr = VIP_INT0_STATUS0_CLR +
+			VIP_INTC_INTX_OFFSET * irq_num;
+	u32 irq_val = (1 << (list_num * 2)) |
+		      (VIP_VIP1_PARSER_INT << (irq_num * 1));
+
+	/* Clear all Parser Interrupt */
+	reg_write(parser, VIP_PARSER_FIQ_CLR, 0xffffffff);
+	reg_write(parser, VIP_PARSER_FIQ_CLR, 0x0);
+
+	reg_write(dev->shared, reg_addr, irq_val);
+
+	vpdma_clear_list_stat(dev->shared->vpdma, irq_num, dev->slice_id);
+}
+
+static void populate_desc_list(struct vip_stream *stream)
+{
+	struct vip_port *port = stream->port;
+	struct vip_dev *dev = port->dev;
+	unsigned int list_length;
+
+	stream->desc_next = stream->desc_list.buf.addr;
+	add_stream_dtds(stream);
+
+	list_length = stream->desc_next - stream->desc_list.buf.addr;
+	vpdma_map_desc_buf(dev->shared->vpdma, &stream->desc_list.buf);
+}
+
+/*
+ * start_dma - adds descriptors to the dma list and submits them.
+ * Should be called after a new vb is queued and on a vpdma list
+ * completion interrupt.
+ */
+static void start_dma(struct vip_stream *stream, struct vip_buffer *buf)
+{
+	struct vip_dev *dev = stream->port->dev;
+	struct vpdma_data *vpdma = dev->shared->vpdma;
+	int list_num = stream->list_num;
+	dma_addr_t dma_addr;
+	int drop_data;
+
+	if (vpdma_list_busy(vpdma, list_num)) {
+		vip_err(stream, "vpdma list busy, cannot post\n");
+		return;				/* nothing to do */
+	}
+
+	if (buf) {
+		dma_addr = vb2_dma_contig_plane_dma_addr(&buf->vb.vb2_buf, 0);
+		drop_data = 0;
+		vip_dbg(4, stream, "%s: vb2 buf idx:%d, dma_addr:%pad\n",
+			__func__, buf->vb.vb2_buf.index, &dma_addr);
+	} else {
+		dma_addr = 0;
+		drop_data = 1;
+		vip_dbg(4, stream, "%s: dropped\n", __func__);
+	}
+
+	vpdma_update_dma_addr(dev->shared->vpdma, &stream->desc_list,
+			      dma_addr, stream->write_desc, drop_data, 0);
+
+	if (stream->port->fmt->coplanar) {
+		dma_addr += stream->bytesperline * stream->height;
+		vpdma_update_dma_addr(dev->shared->vpdma, &stream->desc_list,
+				      dma_addr, stream->write_desc + 1,
+				      drop_data, 1);
+	}
+
+	vpdma_submit_descs(dev->shared->vpdma,
+			   &stream->desc_list, stream->list_num);
+}
+
+static void vip_schedule_next_buffer(struct vip_stream *stream)
+{
+	struct vip_dev *dev = stream->port->dev;
+	struct vip_buffer *buf;
+	unsigned long flags;
+
+	spin_lock_irqsave(&dev->slock, flags);
+	if (list_empty(&stream->vidq)) {
+		vip_dbg(4, stream, "Dropping frame\n");
+		if (list_empty(&stream->dropq)) {
+			vip_err(stream, "No dropq buffer left!");
+			spin_unlock_irqrestore(&dev->slock, flags);
+			return;
+		}
+		buf = list_entry(stream->dropq.next,
+				 struct vip_buffer, list);
+
+		buf->drop = true;
+		list_move_tail(&buf->list, &stream->post_bufs);
+		buf = NULL;
+	} else {
+		buf = list_entry(stream->vidq.next,
+				 struct vip_buffer, list);
+		buf->drop = false;
+		list_move_tail(&buf->list, &stream->post_bufs);
+		vip_dbg(4, stream, "added next buffer\n");
+	}
+
+	spin_unlock_irqrestore(&dev->slock, flags);
+	start_dma(stream, buf);
+}
+
+static void vip_process_buffer_complete(struct vip_stream *stream)
+{
+	struct vip_dev *dev = stream->port->dev;
+	struct vip_buffer *buf;
+	struct vb2_v4l2_buffer *vb = NULL;
+	unsigned long flags, fld;
+
+	buf = list_first_entry(&stream->post_bufs, struct vip_buffer, list);
+
+	if (stream->port->flags & FLAG_INTERLACED) {
+		vpdma_unmap_desc_buf(dev->shared->vpdma,
+				     &stream->desc_list.buf);
+
+		fld = dtd_get_field(stream->write_desc);
+		stream->field = fld ? V4L2_FIELD_BOTTOM : V4L2_FIELD_TOP;
+
+		vpdma_map_desc_buf(dev->shared->vpdma, &stream->desc_list.buf);
+	}
+
+	if (buf) {
+		vip_dbg(4, stream, "vip buffer complete 0x%x, 0x%x\n",
+			(unsigned int)buf, buf->drop);
+
+		vb = &buf->vb;
+		vb->field = stream->field;
+		vb->sequence = stream->sequence;
+		vb->vb2_buf.timestamp = ktime_get_ns();
+
+		if (buf->drop) {
+			spin_lock_irqsave(&dev->slock, flags);
+			list_move_tail(&buf->list, &stream->dropq);
+			spin_unlock_irqrestore(&dev->slock, flags);
+		} else {
+			spin_lock_irqsave(&dev->slock, flags);
+			list_del(&buf->list);
+			spin_unlock_irqrestore(&dev->slock, flags);
+			vb2_buffer_done(&vb->vb2_buf, VB2_BUF_STATE_DONE);
+		}
+	} else {
+		vip_err(stream, "%s: buf is null!!!\n", __func__);
+		return;
+	}
+
+	stream->sequence++;
+}
+
+static int vip_reset_vpdma(struct vip_stream *stream)
+{
+	struct vip_port *port = stream->port;
+	struct vip_dev *dev = port->dev;
+	struct vip_buffer *buf;
+	unsigned long flags;
+
+	stop_dma(stream, false);
+
+	spin_lock_irqsave(&dev->slock, flags);
+	/* requeue all active buffers in the opposite order */
+	while (!list_empty(&stream->post_bufs)) {
+		buf = list_last_entry(&stream->post_bufs,
+				      struct vip_buffer, list);
+		list_del(&buf->list);
+		if (buf->drop == 1) {
+			list_add_tail(&buf->list, &stream->dropq);
+			vip_dbg(4, stream, "requeueing drop buffer on dropq\n");
+		} else {
+			list_add(&buf->list, &stream->vidq);
+			vip_dbg(4, stream, "requeueing vb2 buf idx:%d on vidq\n",
+				buf->vb.vb2_buf.index);
+		}
+	}
+	spin_unlock_irqrestore(&dev->slock, flags);
+
+	/* Make sure the desc_list is unmapped */
+	vpdma_unmap_desc_buf(dev->shared->vpdma, &stream->desc_list.buf);
+
Quoted 8 lines

drivers/media/platform/ti-vpe/vip.c line 1054

Quoted 4 lines
drivers/media/platform/ti-vpe/vip.c new file +4 -0
+	return 0;
+}
+
+static void vip_overflow_recovery_work(struct work_struct *work)

This function requires some additional comments about when it should be called and what it does. Recovery is always tricky, and so it is worthwhile spending additional time on commenting it.

Ok I'll add a comments and also a reference to the related TRM section.

Quoted 304 lines
Quoted 304 lines
drivers/media/platform/ti-vpe/vip.c new file +303 -0
+{
+	struct vip_stream *stream = container_of(work, struct vip_stream,
+						 recovery_work);
+	struct vip_port *port = stream->port;
+	struct vip_dev *dev = port->dev;
+
+	vip_err(stream, "%s: Port %c\n", __func__,
+		port->port_id == VIP_PORTA ? 'A' : 'B');
+
+	disable_irqs(dev, dev->slice_id, stream->list_num);
+	clear_irqs(dev, dev->slice_id, stream->list_num);
+
+	/* 1.	Set VIP_XTRA6_PORT_A[31:16] YUV_SRCNUM_STOP_IMMEDIATELY */
+	/* 2.	Set VIP_XTRA6_PORT_A[15:0] ANC_SRCNUM_STOP_IMMEDIATELY */
+	vip_parser_stop_imm(port, 1);
+
+	/* 3.	Clear VIP_PORT_A[8] ENABLE */
+	/*
+	 * 4.	Set VIP_PORT_A[7] CLR_ASYNC_FIFO_RD
+	 *      Set VIP_PORT_A[6] CLR_ASYNC_FIFO_WR
+	 */
+	vip_enable_parser(port, false);
+
+	/* 5.	Set VIP_PORT_A[23] SW_RESET */
+	vip_reset_parser(port, 1);
+
+	/*
+	 * 6.	Reset other VIP modules
+	 *	For each module used downstream of VIP_PARSER, write 1 to the
+	 *      bit location of the VIP_CLKC_RST register which is connected
+	 *      to VIP_PARSER
+	 */
+	vip_module_reset(dev, VIP_DP_RST, true);
+
+	usleep_range(200, 250);
+
+	/*
+	 * 7.	Abort VPDMA channels
+	 *	Write to list attribute to stop list 0
+	 *	Write to list address register location of abort list
+	 *	Write to list attribute register list 0 and size of abort list
+	 */
+	vip_reset_vpdma(stream);
+
+	/* 8.	Clear VIP_PORT_A[23] SW_RESET */
+	vip_reset_parser(port, 0);
+
+	/*
+	 * 9.	Un-reset other VIP modules
+	 *	For each module used downstream of VIP_PARSER, write 0 to
+	 *	the bit location of the VIP_CLKC_RST register which is
+	 *	connected to VIP_PARSER
+	 */
+	vip_module_reset(dev, VIP_DP_RST, false);
+
+	/* 10.	(Delay) */
+	/* 11.	SC coeff downloaded (if VIP_SCALER is being used) */
+	vip_setup_scaler(stream);
+
+	/* 12.	(Delay) */
+		/* the above are not needed here yet */
+
+	populate_desc_list(stream);
+	stream->num_recovery++;
+	if (stream->num_recovery < 5) {
+		/* Reload the vpdma */
+		vip_load_vpdma_list_fifo(stream);
+
+		enable_irqs(dev, dev->slice_id, stream->list_num);
+		vip_schedule_next_buffer(stream);
+
+		/* 13.	Clear VIP_XTRA6_PORT_A[31:16] YUV_SRCNUM_STOP_IMM */
+		/* 14.	Clear VIP_XTRA6_PORT_A[15:0] ANC_SRCNUM_STOP_IMM */
+
+		vip_parser_stop_imm(port, 0);
+
+		/* 15.	Set VIP_PORT_A[8] ENABLE */
+		/*
+		 * 16.	Clear VIP_PORT_A[7] CLR_ASYNC_FIFO_RD
+		 *	Clear VIP_PORT_A[6] CLR_ASYNC_FIFO_WR
+		 */
+		vip_enable_parser(port, true);
+	} else {
+		vip_err(stream, "%s: num_recovery limit exceeded leaving disabled\n",
+			__func__);
+	}
+}
+
+static void handle_parser_irqs(struct vip_dev *dev)
+{
+	struct vip_parser_data *parser = dev->parser;
+	struct vip_port *porta = dev->ports[VIP_PORTA];
+	struct vip_port *portb = dev->ports[VIP_PORTB];
+	struct vip_stream *stream = NULL;
+	u32 irq_stat = reg_read(parser, VIP_PARSER_FIQ_STATUS);
+	int i;
+
+	vip_dbg(3, dev, "%s: FIQ_STATUS: 0x%08x\n", __func__, irq_stat);
+
+	/* Clear all Parser Interrupt */
+	reg_write(parser, VIP_PARSER_FIQ_CLR, irq_stat);
+	reg_write(parser, VIP_PARSER_FIQ_CLR, 0x0);
+
+	if (irq_stat & VIP_PORTA_VDET)
+		vip_dbg(3, dev, "VIP_PORTA_VDET\n");
+	if (irq_stat & VIP_PORTB_VDET)
+		vip_dbg(3, dev, "VIP_PORTB_VDET\n");
+	if (irq_stat & VIP_PORTA_ASYNC_FIFO_OF)
+		vip_err(dev, "VIP_PORTA_ASYNC_FIFO_OF\n");
+	if (irq_stat & VIP_PORTB_ASYNC_FIFO_OF)
+		vip_err(dev, "VIP_PORTB_ASYNC_FIFO_OF\n");
+	if (irq_stat & VIP_PORTA_OUTPUT_FIFO_YUV)
+		vip_err(dev, "VIP_PORTA_OUTPUT_FIFO_YUV\n");
+	if (irq_stat & VIP_PORTA_OUTPUT_FIFO_ANC)
+		vip_err(dev, "VIP_PORTA_OUTPUT_FIFO_ANC\n");
+	if (irq_stat & VIP_PORTB_OUTPUT_FIFO_YUV)
+		vip_err(dev, "VIP_PORTB_OUTPUT_FIFO_YUV\n");
+	if (irq_stat & VIP_PORTB_OUTPUT_FIFO_ANC)
+		vip_err(dev, "VIP_PORTB_OUTPUT_FIFO_ANC\n");
+	if (irq_stat & VIP_PORTA_CONN)
+		vip_dbg(3, dev, "VIP_PORTA_CONN\n");
+	if (irq_stat & VIP_PORTA_DISCONN)
+		vip_dbg(3, dev, "VIP_PORTA_DISCONN\n");
+	if (irq_stat & VIP_PORTB_CONN)
+		vip_dbg(3, dev, "VIP_PORTB_CONN\n");
+	if (irq_stat & VIP_PORTB_DISCONN)
+		vip_dbg(3, dev, "VIP_PORTB_DISCONN\n");
+	if (irq_stat & VIP_PORTA_SRC0_SIZE)
+		vip_dbg(3, dev, "VIP_PORTA_SRC0_SIZE\n");
+	if (irq_stat & VIP_PORTB_SRC0_SIZE)
+		vip_dbg(3, dev, "VIP_PORTB_SRC0_SIZE\n");
+	if (irq_stat & VIP_PORTA_YUV_PROTO_VIOLATION)
+		vip_dbg(3, dev, "VIP_PORTA_YUV_PROTO_VIOLATION\n");
+	if (irq_stat & VIP_PORTA_ANC_PROTO_VIOLATION)
+		vip_dbg(3, dev, "VIP_PORTA_ANC_PROTO_VIOLATION\n");
+	if (irq_stat & VIP_PORTB_YUV_PROTO_VIOLATION)
+		vip_dbg(3, dev, "VIP_PORTB_YUV_PROTO_VIOLATION\n");
+	if (irq_stat & VIP_PORTB_ANC_PROTO_VIOLATION)
+		vip_dbg(3, dev, "VIP_PORTB_ANC_PROTO_VIOLATION\n");
+	if (irq_stat & VIP_PORTA_CFG_DISABLE_COMPLETE)
+		vip_dbg(3, dev, "VIP_PORTA_CFG_DISABLE_COMPLETE\n");
+	if (irq_stat & VIP_PORTB_CFG_DISABLE_COMPLETE)
+		vip_dbg(3, dev, "VIP_PORTB_CFG_DISABLE_COMPLETE\n");
+
+	if (irq_stat & (VIP_PORTA_ASYNC_FIFO_OF |
+			VIP_PORTA_OUTPUT_FIFO_YUV |
+			VIP_PORTA_OUTPUT_FIFO_ANC)) {
+		for (i = 0; i < VIP_CAP_STREAMS_PER_PORT; i++) {
+			if (porta->cap_streams[i] &&
+			    porta->cap_streams[i]->port->port_id ==
+			    porta->port_id) {
+				stream = porta->cap_streams[i];
+				break;
+			}
+		}
+		if (stream) {
+			disable_irqs(dev, dev->slice_id,
+				     stream->list_num);
+			schedule_work(&stream->recovery_work);
+			return;
+		}
+	}
+	if (irq_stat & (VIP_PORTB_ASYNC_FIFO_OF |
+			VIP_PORTB_OUTPUT_FIFO_YUV |
+			VIP_PORTB_OUTPUT_FIFO_ANC)) {
+		for (i = 0; i < VIP_CAP_STREAMS_PER_PORT; i++) {
+			if (portb->cap_streams[i] &&
+			    portb->cap_streams[i]->port->port_id ==
+			    portb->port_id) {
+				stream = portb->cap_streams[i];
+				break;
+			}
+		}
+		if (stream) {
+			disable_irqs(dev, dev->slice_id,
+				     stream->list_num);
+			schedule_work(&stream->recovery_work);
+			return;
+		}
+	}
+}
+
+static irqreturn_t vip_irq(int irq_vip, void *data)
+{
+	struct vip_dev *dev = (struct vip_dev *)data;
+	struct vpdma_data *vpdma;
+	struct vip_stream *stream;
+	int list_num;
+	int irq_num = dev->slice_id;
+	u32 irqst, irqst_saved, reg_addr;
+
+	if (!dev->shared)
+		return IRQ_HANDLED;
+
+	vpdma = dev->shared->vpdma;
+	reg_addr = VIP_INT0_STATUS0 +
+			VIP_INTC_INTX_OFFSET * irq_num;
+	irqst_saved = reg_read(dev->shared, reg_addr);
+	irqst = irqst_saved;
+
+	vip_dbg(8, dev, "IRQ %d VIP_INT%d_STATUS0 0x%x\n",
+		irq_vip, irq_num, irqst);
+	if (irqst) {
+		if (irqst & (VIP_VIP1_PARSER_INT << (irq_num * 1))) {
+			irqst &= ~(VIP_VIP1_PARSER_INT << (irq_num * 1));
+			handle_parser_irqs(dev);
+		}
+
+		for (list_num = 0; irqst && (list_num < 8);  list_num++) {
+			/* Check for LIST_COMPLETE IRQ */
+			if (!(irqst & (1 << list_num * 2)))
+				continue;
+
+			vip_dbg(8, dev, "IRQ %d: handling LIST%d_COMPLETE\n",
+				irq_num, list_num);
+
+			stream = vpdma_hwlist_get_priv(vpdma, list_num);
+			if (!stream || stream->list_num != list_num) {
+				vip_err(dev, "IRQ occurred for unused list");
+				continue;
+			}
+
+			vpdma_clear_list_stat(vpdma, irq_num, list_num);
+
+			vip_process_buffer_complete(stream);
+
+			vip_schedule_next_buffer(stream);
+
+			irqst &= ~((1 << list_num * 2));
+		}
+	}
+
+	/* Acknowledge that we are done with all interrupts */
+	reg_write(dev->shared, VIP_INTC_E0I, 1 << irq_num);
+
+	/* Clear handled events from status register */
+	reg_addr = VIP_INT0_STATUS0_CLR +
+		   VIP_INTC_INTX_OFFSET * irq_num;
+	reg_write(dev->shared, reg_addr, irqst_saved);
+
+	return IRQ_HANDLED;
+}
+
+/*
+ * video ioctls
+ */
+static int vip_querycap(struct file *file, void *priv,
+			struct v4l2_capability *cap)
+{
+	struct vip_stream *stream = file2stream(file);
+	struct vip_port *port = stream->port;
+	struct vip_dev *dev = port->dev;
+
+	strscpy(cap->driver, VIP_MODULE_NAME, sizeof(cap->driver));
+	strscpy(cap->card, VIP_MODULE_NAME, sizeof(cap->card));
+
+	snprintf(cap->bus_info, sizeof(cap->bus_info),
+		 "platform:%s:%s:stream%1d", dev->shared->name, port->name,
+		 stream->stream_id);
+	return 0;
+}
+
+static int vip_enuminput(struct file *file, void *priv,
+			 struct v4l2_input *inp)
+{
+	struct vip_stream *stream = file2stream(file);
+
+	if (inp->index)
+		return -EINVAL;
+
+	inp->type = V4L2_INPUT_TYPE_CAMERA;
+	inp->std = stream->vfd->tvnorms;
+	snprintf(inp->name, sizeof(inp->name), "camera %u", stream->vfd->num);
+
+	return 0;
+}
+
+static int vip_g_input(struct file *file, void *priv, unsigned int *i)
+{
+	*i = 0;
+	return 0;
+}
+
+static int vip_s_input(struct file *file, void *priv, unsigned int i)
+{
+	if (i != 0)
+		return -EINVAL;
+	return 0;
+}
+
+static int vip_querystd(struct file *file, void *fh, v4l2_std_id *std)
+{
+	struct vip_stream *stream = file2stream(file);
+	struct vip_port *port = stream->port;
+
+	*std = stream->vfd->tvnorms;
+	v4l2_subdev_call(port->subdev, video, querystd, std);
+	vip_dbg(1, stream, "querystd: 0x%lx\n", (unsigned long)*std);
+	return 0;
+}
+
+static int vip_g_std(struct file *file, void *fh, v4l2_std_id *std)
+{
Quoted 8 lines

drivers/media/platform/ti-vpe/vip.c line 1362

Quoted 5 lines
drivers/media/platform/ti-vpe/vip.c new file +5 -0
+	struct vip_stream *stream = file2stream(file);
+	struct vip_port *port = stream->port;
+
+	*std = stream->vfd->tvnorms;
+	v4l2_subdev_call(port->subdev, video, g_std_output, std);

g_std_output? Shouldn't this just be g_std? The g_std_output op is for video transmitters.

Yes, I'll fix those.

Quoted 14 lines
Quoted 14 lines
drivers/media/platform/ti-vpe/vip.c new file +13 -0
+	vip_dbg(1, stream, "g_std: 0x%lx\n", (unsigned long)*std);
+
+	return 0;
+}
+
+static int vip_s_std(struct file *file, void *fh, v4l2_std_id std)
+{
+	struct vip_stream *stream = file2stream(file);
+	struct vip_port *port = stream->port;
+
+	vip_dbg(1, stream, "s_std: 0x%lx\n", (unsigned long)std);
+
+	if (!(std & stream->vfd->tvnorms)) {
Quoted 8 lines

drivers/media/platform/ti-vpe/vip.c line 1381

Quoted 6 lines
drivers/media/platform/ti-vpe/vip.c new file +6 -0
+		vip_dbg(1, stream, "s_std after check: 0x%lx\n",
+			(unsigned long)std);
+		return -EINVAL;
+	}
+
+	v4l2_subdev_call(port->subdev, video, s_std_output, std);

Ditto.

Yes, I'll fix those.

Quoted 13 lines
Quoted 13 lines
drivers/media/platform/ti-vpe/vip.c new file +12 -0
+	return 0;
+}
+
+static int vip_enum_fmt_vid_cap(struct file *file, void *priv,
+				struct v4l2_fmtdesc *f)
+{
+	struct vip_stream *stream = file2stream(file);
+	struct vip_port *port = stream->port;
+	struct vip_fmt *fmt;
+
+	vip_dbg(3, stream, "enum_fmt index:%d\n", f->index);
+	if (f->index >= port->num_active_fmt)
Quoted 8 lines

drivers/media/platform/ti-vpe/vip.c line 1399

Quoted 6 lines
drivers/media/platform/ti-vpe/vip.c new file +6 -0
+		return -EINVAL;
+
+	fmt = port->active_fmt[f->index];
+
+	f->pixelformat = fmt->fourcc;
+	f->type = V4L2_BUF_TYPE_VIDEO_CAPTURE;

No need to set the type field.

Ok.

Quoted 5 lines

drivers/media/platform/ti-vpe/vip.c line 1401

Quoted 3 lines
drivers/media/platform/ti-vpe/vip.c new file +2 -0
+	vip_dbg(3, stream, "enum_fmt fourcc:%s\n",
+		fourcc_to_str(f->pixelformat));

Excessive debugging.

Why excessive?

Quoted 32 lines
Quoted 17 lines
drivers/media/platform/ti-vpe/vip.c new file +16 -0
+
+	return 0;
+}
+
+static int vip_enum_framesizes(struct file *file, void *priv,
+			       struct v4l2_frmsizeenum *f)
+{
+	struct vip_stream *stream = file2stream(file);
+	struct vip_port *port = stream->port;
+	struct vip_fmt *fmt;
+	struct v4l2_subdev_frame_size_enum fse;
+	int ret;
+
+	fmt = find_port_format_by_pix(port, f->pixel_format);
+	if (!fmt)
+		return -EINVAL;

drivers/media/platform/ti-vpe/vip.c line 1427

Quoted 10 lines
drivers/media/platform/ti-vpe/vip.c new file +10 -0
+
+	fse.index = f->index;
+	fse.pad = port->source_pad;
+	fse.code = fmt->code;
+	fse.which = V4L2_SUBDEV_FORMAT_ACTIVE;
+	ret = v4l2_subdev_call(port->subdev, pad, enum_frame_size, NULL, &fse);
+	if (ret == -ENOIOCTLCMD && !f->index) {
+		/*
+		 * if subdev does not support enum_frame_size
+		 * then use get_fmt

I don't think that's right. If the subdev doesn't support this, then this ioctl should be disabled altogether. Typically this ioctl is only valid for sensor subdevs, not for video receivers.

Quoted 1 line

Use v4l2_subdev_has_op() and v4l2_disable_ioctl().

You mean to check if the subdev support this ioctl and if not disable it for the current video device only, correct?

Quoted 116 lines
Quoted 43 lines
drivers/media/platform/ti-vpe/vip.c new file +42 -0
+		 */
+		struct v4l2_subdev_format format = {
+			.which = V4L2_SUBDEV_FORMAT_ACTIVE,
+			.pad = port->source_pad,
+		};
+		ret = v4l2_subdev_call(port->subdev, pad, get_fmt, NULL,
+				       &format);
+		if (ret)
+			return ret;
+
+		fse.max_width = format.format.width;
+		fse.max_height = format.format.height;
+	} else if (ret) {
+		return -EINVAL;
+	}
+
+	vip_dbg(1, stream, "%s: index: %d code: %x W:[%d,%d] H:[%d,%d]\n",
+		__func__, fse.index, fse.code, fse.min_width, fse.max_width,
+		fse.min_height, fse.max_height);
+
+	f->type = V4L2_FRMSIZE_TYPE_DISCRETE;
+	f->discrete.width = fse.max_width;
+	f->discrete.height = fse.max_height;
+
+	return 0;
+}
+
+static int vip_enum_frameintervals(struct file *file, void *priv,
+				   struct v4l2_frmivalenum *f)
+{
+	struct vip_stream *stream = file2stream(file);
+	struct vip_port *port = stream->port;
+	struct vip_fmt *fmt;
+	struct v4l2_subdev_frame_interval_enum fie = {
+		.index = f->index,
+		.width = f->width,
+		.height = f->height,
+		.which = V4L2_SUBDEV_FORMAT_ACTIVE,
+	};
+	int ret;
+
+	fmt = find_port_format_by_pix(port, f->pixel_format);

drivers/media/platform/ti-vpe/vip.c line 1479

Quoted 10 lines
drivers/media/platform/ti-vpe/vip.c new file +10 -0
+	if (!fmt)
+		return -EINVAL;
+
+	fie.code = fmt->code;
+	ret = v4l2_subdev_call(port->subdev, pad, enum_frame_interval,
+			       NULL, &fie);
+	if (ret)
+		return ret;
+	f->type = V4L2_FRMIVAL_TYPE_DISCRETE;
+	f->discrete = fie.interval;

Same here: disable this ioctl if the subdev doesn't support enum_frame_interval.

Quoted 10 lines
drivers/media/platform/ti-vpe/vip.c new file +10 -0
+
+	return 0;
+}
+
+static int vip_g_parm(struct file *file, void *priv,
+		      struct v4l2_streamparm *parm)
+{
+	struct vip_stream *stream = file2stream(file);
+	struct vip_port *port = stream->port;
+

drivers/media/platform/ti-vpe/vip.c line 1499

Quoted 10 lines
drivers/media/platform/ti-vpe/vip.c new file +10 -0
+	return v4l2_g_parm_cap(stream->vfd, port->subdev, parm);
+}
+
+static int vip_s_parm(struct file *file, void *priv,
+		      struct v4l2_streamparm *parm)
+{
+	struct vip_stream *stream = file2stream(file);
+	struct vip_port *port = stream->port;
+
+	return v4l2_s_parm_cap(stream->vfd, port->subdev, parm);

Same here: typically this is not supported for video receivers.

Quoted 37 lines
drivers/media/platform/ti-vpe/vip.c new file +37 -0
+}
+
+static int vip_calc_format_size(struct vip_port *port,
+				struct vip_fmt *fmt,
+				struct v4l2_format *f)
+{
+	enum v4l2_field *field;
+	unsigned int stride;
+
+	if (!fmt) {
+		vip_dbg(2, port,
+			"no vip_fmt format provided!\n");
+		return -EINVAL;
+	}
+
+	field = &f->fmt.pix.field;
+	if (*field == V4L2_FIELD_ANY)
+		*field = V4L2_FIELD_NONE;
+	else if (V4L2_FIELD_NONE != *field && V4L2_FIELD_ALTERNATE != *field)
+		return -EINVAL;
+
+	v4l_bound_align_image(&f->fmt.pix.width, MIN_W, MAX_W, W_ALIGN,
+			      &f->fmt.pix.height, MIN_H, MAX_H, H_ALIGN,
+			      S_ALIGN);
+
+	stride = f->fmt.pix.width * (fmt->vpdma_fmt[0]->depth >> 3);
+	if (stride > f->fmt.pix.bytesperline)
+		f->fmt.pix.bytesperline = stride;
+
+	f->fmt.pix.bytesperline = clamp_t(u32, f->fmt.pix.bytesperline,
+					  stride, VPDMA_MAX_STRIDE);
+	f->fmt.pix.bytesperline = ALIGN(f->fmt.pix.bytesperline,
+					VPDMA_STRIDE_ALIGN);
+
+	f->fmt.pix.sizeimage = f->fmt.pix.height * f->fmt.pix.bytesperline;
+	if (fmt->coplanar) {
+		f->fmt.pix.sizeimage += f->fmt.pix.height *
Quoted 8 lines

drivers/media/platform/ti-vpe/vip.c line 1542

Quoted 6 lines
drivers/media/platform/ti-vpe/vip.c new file +6 -0
+					f->fmt.pix.bytesperline *
+					fmt->vpdma_fmt[VIP_CHROMA]->depth >> 3;
+	}
+
+	f->fmt.pix.colorspace = fmt->colorspace;
+	f->fmt.pix.priv = 0;

No need to set this.

You mean pix.priv? I thought I remember v4l2-compliance complaining about something like this?

Quoted 550 lines
Quoted 550 lines
drivers/media/platform/ti-vpe/vip.c new file +549 -0
+
+	vip_dbg(3, port, "calc_format_size: fourcc:%s size: %dx%d bpl:%d img_size:%d\n",
+		fourcc_to_str(f->fmt.pix.pixelformat),
+		f->fmt.pix.width, f->fmt.pix.height,
+		f->fmt.pix.bytesperline, f->fmt.pix.sizeimage);
+
+	return 0;
+}
+
+static inline bool vip_is_size_dma_aligned(u32 bpp, u32 width)
+{
+	return ((width * bpp) == ALIGN(width * bpp, VPDMA_STRIDE_ALIGN));
+}
+
+static int vip_try_fmt_vid_cap(struct file *file, void *priv,
+			       struct v4l2_format *f)
+{
+	struct vip_stream *stream = file2stream(file);
+	struct vip_port *port = stream->port;
+	struct v4l2_subdev_frame_size_enum fse;
+	struct vip_fmt *fmt;
+	u32 best_width, best_height, largest_width, largest_height;
+	int ret, found;
+	enum vip_csc_state csc_direction;
+
+	vip_dbg(3, stream, "try_fmt fourcc:%s size: %dx%d\n",
+		fourcc_to_str(f->fmt.pix.pixelformat),
+		f->fmt.pix.width, f->fmt.pix.height);
+
+	fmt = find_port_format_by_pix(port, f->fmt.pix.pixelformat);
+	if (!fmt) {
+		vip_dbg(2, stream,
+			"Fourcc format (0x%08x) not found.\n",
+			f->fmt.pix.pixelformat);
+
+		/* Just get the first one enumerated */
+		fmt = port->active_fmt[0];
+		f->fmt.pix.pixelformat = fmt->fourcc;
+	}
+
+	csc_direction =  vip_csc_direction(fmt->code, fmt->finfo);
+	if (csc_direction != VIP_CSC_NA) {
+		if (!is_csc_available(port)) {
+			vip_dbg(2, stream,
+				"CSC not available for Fourcc format (0x%08x).\n",
+				f->fmt.pix.pixelformat);
+
+			/* Just get the first one enumerated */
+			fmt = port->active_fmt[0];
+			f->fmt.pix.pixelformat = fmt->fourcc;
+			/* re-evaluate the csc_direction here */
+			csc_direction =  vip_csc_direction(fmt->code,
+							   fmt->finfo);
+		} else {
+			vip_dbg(3, stream, "CSC active on Port %c: going %s\n",
+				port->port_id == VIP_PORTA ? 'A' : 'B',
+				(csc_direction == VIP_CSC_Y2R) ? "Y2R" : "R2Y");
+		}
+	}
+
+	/*
+	 * Given that sensors might support multiple mbus code we need
+	 * to use the one that matches the requested pixel format
+	 */
+	port->try_mbus_framefmt = port->mbus_framefmt;
+	port->try_mbus_framefmt.code = fmt->code;
+
+	/* check for/find a valid width/height */
+	ret = 0;
+	found = false;
+	best_width = 0;
+	best_height = 0;
+	largest_width = 0;
+	largest_height = 0;
+	fse.pad = port->source_pad;
+	fse.code = fmt->code;
+	fse.which = V4L2_SUBDEV_FORMAT_ACTIVE;
+	for (fse.index = 0; ; fse.index++) {
+		u32 bpp = fmt->vpdma_fmt[0]->depth >> 3;
+
+		ret = v4l2_subdev_call(port->subdev, pad,
+				       enum_frame_size, NULL, &fse);
+		if (ret == -ENOIOCTLCMD) {
+			/*
+			 * if subdev does not support enum_frame_size
+			 * then just try to set_fmt directly
+			 */
+			struct v4l2_subdev_format format = {
+				.which = V4L2_SUBDEV_FORMAT_TRY,
+			};
+			struct v4l2_subdev_pad_config *pad_cfg;
+
+			pad_cfg = v4l2_subdev_alloc_pad_config(port->subdev);
+			if (!pad_cfg)
+				return -ENOMEM;
+
+			v4l2_fill_mbus_format(&format.format, &f->fmt.pix,
+					      fmt->code);
+			ret = v4l2_subdev_call(port->subdev, pad, set_fmt,
+					       pad_cfg, &format);
+			if (ret)
+				/* here regardless of the reason we give up */
+				break;
+
+			if (f->fmt.pix.width == format.format.width &&
+			    f->fmt.pix.height == format.format.height) {
+				found = true;
+				vip_dbg(3, stream, "try_fmt loop:%d found direct match: %dx%d\n",
+					fse.index, format.format.width,
+					format.format.height);
+			}
+			largest_width = format.format.width;
+			largest_height = format.format.height;
+			best_width = format.format.width;
+			best_height = format.format.height;
+
+			v4l2_subdev_free_pad_config(pad_cfg);
+			break;
+
+		} else if (ret) {
+			break;
+		}
+
+		vip_dbg(3, stream, "try_fmt loop:%d fourcc:%s size: %dx%d\n",
+			fse.index, fourcc_to_str(f->fmt.pix.pixelformat),
+			fse.max_width, fse.max_height);
+
+		if (!vip_is_size_dma_aligned(bpp, fse.max_width))
+			continue;
+
+		if ((fse.max_width >= largest_width) &&
+		    (fse.max_height >= largest_height)) {
+			vip_dbg(3, stream, "try_fmt loop:%d found new larger: %dx%d\n",
+				fse.index, fse.max_width, fse.max_height);
+			largest_width = fse.max_width;
+			largest_height = fse.max_height;
+		}
+
+		if ((fse.max_width >= f->fmt.pix.width) &&
+		    (fse.max_height >= f->fmt.pix.height)) {
+			vip_dbg(3, stream, "try_fmt loop:%d found at least larger: %dx%d\n",
+				fse.index, fse.max_width, fse.max_height);
+
+			if (!best_width ||
+			    ((abs(best_width - f->fmt.pix.width) >=
+			      abs(fse.max_width - f->fmt.pix.width)) &&
+			     (abs(best_height - f->fmt.pix.height) >=
+			      abs(fse.max_height - f->fmt.pix.height)))) {
+				best_width = fse.max_width;
+				best_height = fse.max_height;
+				vip_dbg(3, stream, "try_fmt loop:%d found new best: %dx%d\n",
+					fse.index, fse.max_width,
+					fse.max_height);
+			}
+		}
+
+		if ((f->fmt.pix.width == fse.max_width) &&
+		    (f->fmt.pix.height == fse.max_height)) {
+			found = true;
+			vip_dbg(3, stream, "try_fmt loop:%d found direct match: %dx%d\n",
+				fse.index, fse.max_width,
+				fse.max_height);
+			break;
+		}
+
+		if ((f->fmt.pix.width >= fse.min_width) &&
+		    (f->fmt.pix.width <= fse.max_width) &&
+		    (f->fmt.pix.height >= fse.min_height) &&
+		    (f->fmt.pix.height <= fse.max_height)) {
+			found = true;
+			vip_dbg(3, stream, "try_fmt loop:%d found direct range match: %dx%d\n",
+				fse.index, fse.max_width,
+				fse.max_height);
+			break;
+		}
+	}
+
+	if (found) {
+		port->try_mbus_framefmt.width = f->fmt.pix.width;
+		port->try_mbus_framefmt.height = f->fmt.pix.height;
+		/* No need to check for scaling */
+		goto calc_size;
+	} else if (largest_width && f->fmt.pix.width > largest_width) {
+		port->try_mbus_framefmt.width = largest_width;
+		port->try_mbus_framefmt.height = largest_height;
+	} else if (best_width) {
+		port->try_mbus_framefmt.width = best_width;
+		port->try_mbus_framefmt.height = best_height;
+	} else {
+		/* use existing values as default */
+	}
+
+	vip_dbg(3, stream, "try_fmt best subdev size: %dx%d\n",
+		port->try_mbus_framefmt.width,
+		port->try_mbus_framefmt.height);
+
+	if (is_scaler_available(port) &&
+	    csc_direction != VIP_CSC_Y2R &&
+	    !vip_is_mbuscode_raw(fmt->code) &&
+	    f->fmt.pix.height <= port->try_mbus_framefmt.height &&
+	    port->try_mbus_framefmt.height <= SC_MAX_PIXEL_HEIGHT &&
+	    port->try_mbus_framefmt.width <= SC_MAX_PIXEL_WIDTH) {
+		/*
+		 * Scaler is only accessible if the dst colorspace is YUV.
+		 * As the input to the scaler must be in YUV mode only.
+		 *
+		 * Scaling up is allowed only horizontally.
+		 */
+		unsigned int hratio, vratio, width_align, height_align;
+		u32 bpp = fmt->vpdma_fmt[0]->depth >> 3;
+
+		vip_dbg(3, stream, "Scaler active on Port %c: requesting %dx%d\n",
+			port->port_id == VIP_PORTA ? 'A' : 'B',
+			f->fmt.pix.width, f->fmt.pix.height);
+
+		/* Just make sure everything is properly aligned */
+		width_align = ALIGN(f->fmt.pix.width * bpp, VPDMA_STRIDE_ALIGN);
+		width_align /= bpp;
+		height_align = ALIGN(f->fmt.pix.height, 2);
+
+		f->fmt.pix.width = width_align;
+		f->fmt.pix.height = height_align;
+
+		hratio = f->fmt.pix.width * 1000 /
+			 port->try_mbus_framefmt.width;
+		vratio = f->fmt.pix.height * 1000 /
+			 port->try_mbus_framefmt.height;
+		if (hratio < 125) {
+			f->fmt.pix.width = port->try_mbus_framefmt.width / 8;
+			vip_dbg(3, stream, "Horizontal scaling ratio out of range adjusting -> %d\n",
+				f->fmt.pix.width);
+		}
+
+		if (vratio < 188) {
+			f->fmt.pix.height = port->try_mbus_framefmt.height / 4;
+			vip_dbg(3, stream, "Vertical scaling ratio out of range adjusting -> %d\n",
+				f->fmt.pix.height);
+		}
+		vip_dbg(3, stream, "Scaler: got %dx%d\n",
+			f->fmt.pix.width, f->fmt.pix.height);
+	} else {
+		/* use existing values as default */
+		f->fmt.pix.width = port->try_mbus_framefmt.width;
+		f->fmt.pix.height = port->try_mbus_framefmt.height;
+	}
+
+calc_size:
+	/* That we have a fmt calculate imagesize and bytesperline */
+	return vip_calc_format_size(port, fmt, f);
+}
+
+static int vip_g_fmt_vid_cap(struct file *file, void *priv,
+			     struct v4l2_format *f)
+{
+	struct vip_stream *stream = file2stream(file);
+	struct vip_port *port = stream->port;
+	struct vip_fmt *fmt = port->fmt;
+
+	/* Use last known values or defaults */
+	f->fmt.pix.width	= stream->width;
+	f->fmt.pix.height	= stream->height;
+	f->fmt.pix.pixelformat	= port->fmt->fourcc;
+	f->fmt.pix.field	= stream->sup_field;
+	f->fmt.pix.colorspace	= port->fmt->colorspace;
+	f->fmt.pix.bytesperline	= stream->bytesperline;
+	f->fmt.pix.sizeimage	= stream->sizeimage;
+
+	vip_dbg(3, stream,
+		"g_fmt fourcc:%s code: %04x size: %dx%d bpl:%d img_size:%d\n",
+		fourcc_to_str(f->fmt.pix.pixelformat),
+		fmt->code,
+		f->fmt.pix.width, f->fmt.pix.height,
+		f->fmt.pix.bytesperline, f->fmt.pix.sizeimage);
+	vip_dbg(3, stream, "g_fmt vpdma data type: 0x%02X\n",
+		port->fmt->vpdma_fmt[0]->data_type);
+
+	return 0;
+}
+
+static int vip_s_fmt_vid_cap(struct file *file, void *priv,
+			     struct v4l2_format *f)
+{
+	struct vip_stream *stream = file2stream(file);
+	struct vip_port *port = stream->port;
+	struct v4l2_subdev_format sfmt;
+	struct v4l2_mbus_framefmt *mf;
+	enum vip_csc_state csc_direction;
+	int ret;
+
+	vip_dbg(3, stream, "s_fmt input fourcc:%s size: %dx%d bpl:%d img_size:%d\n",
+		fourcc_to_str(f->fmt.pix.pixelformat),
+		f->fmt.pix.width, f->fmt.pix.height,
+		f->fmt.pix.bytesperline, f->fmt.pix.sizeimage);
+
+	ret = vip_try_fmt_vid_cap(file, priv, f);
+	if (ret)
+		return ret;
+
+	vip_dbg(3, stream, "s_fmt try_fmt fourcc:%s size: %dx%d bpl:%d img_size:%d\n",
+		fourcc_to_str(f->fmt.pix.pixelformat),
+		f->fmt.pix.width, f->fmt.pix.height,
+		f->fmt.pix.bytesperline, f->fmt.pix.sizeimage);
+
+	if (vb2_is_busy(&stream->vb_vidq)) {
+		vip_err(stream, "%s queue busy\n", __func__);
+		return -EBUSY;
+	}
+
+	/*
+	 * Check if we need the scaler or not
+	 *
+	 * Since on previous S_FMT call the scaler might have been
+	 * allocated if it is not needed in this instance we will
+	 * attempt to free it just in case.
+	 *
+	 * free_scaler() is harmless unless the current port
+	 * allocated it.
+	 */
+	if (f->fmt.pix.width == port->try_mbus_framefmt.width &&
+	    f->fmt.pix.height == port->try_mbus_framefmt.height)
+		free_scaler(port);
+	else
+		allocate_scaler(port);
+
+	port->fmt = find_port_format_by_pix(port,
+					    f->fmt.pix.pixelformat);
+	stream->width		= f->fmt.pix.width;
+	stream->height		= f->fmt.pix.height;
+	stream->bytesperline	= f->fmt.pix.bytesperline;
+	stream->sizeimage	= f->fmt.pix.sizeimage;
+	stream->sup_field	= f->fmt.pix.field;
+	stream->field		= f->fmt.pix.field;
+
+	port->c_rect.left	= 0;
+	port->c_rect.top	= 0;
+	port->c_rect.width	= stream->width;
+	port->c_rect.height	= stream->height;
+
+	/*
+	 * Check if we need the csc unit or not
+	 *
+	 * Since on previous S_FMT call, the csc might have been
+	 * allocated if it is not needed in this instance we will
+	 * attempt to free it just in case.
+	 *
+	 * free_csc() is harmless unless the current port
+	 * allocated it.
+	 */
+	csc_direction =  vip_csc_direction(port->fmt->code, port->fmt->finfo);
+	if (csc_direction == VIP_CSC_NA)
+		free_csc(port);
+	else
+		allocate_csc(port, csc_direction);
+
+	if (stream->sup_field == V4L2_FIELD_ALTERNATE)
+		port->flags |= FLAG_INTERLACED;
+	else
+		port->flags &= ~FLAG_INTERLACED;
+
+	vip_dbg(3, stream, "s_fmt fourcc:%s size: %dx%d bpl:%d img_size:%d\n",
+		fourcc_to_str(f->fmt.pix.pixelformat),
+		f->fmt.pix.width, f->fmt.pix.height,
+		f->fmt.pix.bytesperline, f->fmt.pix.sizeimage);
+
+	mf = &sfmt.format;
+	v4l2_fill_mbus_format(mf, &f->fmt.pix, port->fmt->code);
+	/* Make sure to use the subdev size found in the try_fmt */
+	mf->width = port->try_mbus_framefmt.width;
+	mf->height = port->try_mbus_framefmt.height;
+
+	vip_dbg(3, stream, "s_fmt pix_to_mbus mbus_code: %04X size: %dx%d\n",
+		mf->code,
+		mf->width, mf->height);
+
+	sfmt.which = V4L2_SUBDEV_FORMAT_ACTIVE;
+	sfmt.pad = port->source_pad;
+	ret = v4l2_subdev_call(port->subdev, pad, set_fmt, NULL, &sfmt);
+	if (ret) {
+		vip_dbg(1, stream, "set_fmt failed in subdev\n");
+		return ret;
+	}
+
+	/* Save it */
+	port->mbus_framefmt = *mf;
+
+	vip_dbg(3, stream, "s_fmt subdev fmt mbus_code: %04X size: %dx%d\n",
+		port->mbus_framefmt.code,
+		port->mbus_framefmt.width, port->mbus_framefmt.height);
+	vip_dbg(3, stream, "s_fmt vpdma data type: 0x%02X\n",
+		port->fmt->vpdma_fmt[0]->data_type);
+
+	return 0;
+}
+
+/*
+ * Does the exact opposite of set_fmt_params
+ * It makes sure the DataPath register is sane after tear down
+ */
+static void unset_fmt_params(struct vip_stream *stream)
+{
+	struct vip_dev *dev = stream->port->dev;
+	struct vip_port *port = stream->port;
+
+	stream->sequence = 0;
+	if (stream->port->flags & FLAG_INTERLACED)
+		stream->field = V4L2_FIELD_TOP;
+
+	if (port->csc == VIP_CSC_Y2R) {
+		if (port->port_id == VIP_PORTA) {
+			vip_set_slice_path(dev, VIP_CSC_SRC_DATA_SELECT, 0);
+			vip_set_slice_path(dev,
+					   VIP_MULTI_CHANNEL_DATA_SELECT, 0);
+			vip_set_slice_path(dev, VIP_RGB_OUT_HI_DATA_SELECT, 0);
+			vip_set_slice_path(dev, VIP_RGB_SRC_DATA_SELECT, 0);
+		} else {
+			vip_set_slice_path(dev, VIP_CSC_SRC_DATA_SELECT, 0);
+			vip_set_slice_path(dev,
+					   VIP_MULTI_CHANNEL_DATA_SELECT, 0);
+			vip_set_slice_path(dev, VIP_RGB_OUT_LO_DATA_SELECT, 0);
+		}
+		/* We are done */
+		return;
+	} else if (port->csc == VIP_CSC_R2Y) {
+		if (port->scaler && port->fmt->coplanar) {
+			if (port->port_id == VIP_PORTA) {
+				vip_set_slice_path(dev,
+						   VIP_CSC_SRC_DATA_SELECT, 0);
+				vip_set_slice_path(dev,
+						   VIP_SC_SRC_DATA_SELECT, 0);
+				vip_set_slice_path(dev,
+						   VIP_CHR_DS_1_SRC_DATA_SELECT,
+						   0);
+				vip_set_slice_path(dev,
+						   VIP_CHR_DS_1_DATA_BYPASS, 0);
+				vip_set_slice_path(dev,
+						   VIP_RGB_OUT_HI_DATA_SELECT,
+						   0);
+			}
+		} else if (port->scaler) {
+			if (port->port_id == VIP_PORTA) {
+				vip_set_slice_path(dev,
+						   VIP_CSC_SRC_DATA_SELECT, 0);
+				vip_set_slice_path(dev,
+						   VIP_SC_SRC_DATA_SELECT, 0);
+				vip_set_slice_path(dev,
+						   VIP_CHR_DS_1_SRC_DATA_SELECT,
+						   0);
+				vip_set_slice_path(dev,
+						   VIP_CHR_DS_1_DATA_BYPASS, 0);
+				vip_set_slice_path(dev,
+						   VIP_RGB_OUT_HI_DATA_SELECT,
+						   0);
+			}
+		} else if (port->fmt->coplanar) {
+			if (port->port_id == VIP_PORTA) {
+				vip_set_slice_path(dev,
+						   VIP_CSC_SRC_DATA_SELECT, 0);
+				vip_set_slice_path(dev,
+						   VIP_CHR_DS_1_SRC_DATA_SELECT,
+						   0);
+				vip_set_slice_path(dev,
+						   VIP_CHR_DS_1_DATA_BYPASS, 0);
+				vip_set_slice_path(dev,
+						   VIP_RGB_OUT_HI_DATA_SELECT,
+						   0);
+			}
+		} else {
+			if (port->port_id == VIP_PORTA) {
+				vip_set_slice_path(dev,
+						   VIP_CSC_SRC_DATA_SELECT, 0);
+				vip_set_slice_path(dev,
+						   VIP_CHR_DS_1_SRC_DATA_SELECT,
+						   0);
+				vip_set_slice_path(dev,
+						   VIP_CHR_DS_1_DATA_BYPASS, 0);
+				vip_set_slice_path(dev,
+						   VIP_RGB_OUT_HI_DATA_SELECT,
+						   0);
+			}
+		}
+		/* We are done */
+		return;
+	} else if (v4l2_is_format_rgb(port->fmt->finfo)) {
+		if (port->port_id == VIP_PORTA) {
+			vip_set_slice_path(dev,
+					   VIP_MULTI_CHANNEL_DATA_SELECT, 0);
+			vip_set_slice_path(dev, VIP_RGB_OUT_LO_DATA_SELECT, 0);
+		}
+		/* We are done */
+		return;
+	}
+
+	if (port->scaler && port->fmt->coplanar) {
+		if (port->port_id == VIP_PORTA) {
+			vip_set_slice_path(dev, VIP_SC_SRC_DATA_SELECT, 0);
+			vip_set_slice_path(dev,
+					   VIP_CHR_DS_1_SRC_DATA_SELECT, 0);
+			vip_set_slice_path(dev, VIP_CHR_DS_1_DATA_BYPASS, 0);
+			vip_set_slice_path(dev, VIP_RGB_OUT_HI_DATA_SELECT, 0);
+		} else {
+			vip_set_slice_path(dev, VIP_SC_SRC_DATA_SELECT, 0);
+			vip_set_slice_path(dev,
+					   VIP_CHR_DS_2_SRC_DATA_SELECT, 0);
+			vip_set_slice_path(dev, VIP_CHR_DS_1_DATA_BYPASS, 0);
+			vip_set_slice_path(dev, VIP_RGB_OUT_LO_DATA_SELECT, 0);
+			vip_set_slice_path(dev,
+					   VIP_MULTI_CHANNEL_DATA_SELECT, 0);
+		}
+	} else if (port->scaler) {
+		if (port->port_id == VIP_PORTA) {
+			vip_set_slice_path(dev, VIP_SC_SRC_DATA_SELECT, 0);
+			vip_set_slice_path(dev,
+					   VIP_CHR_DS_1_SRC_DATA_SELECT, 0);
+			vip_set_slice_path(dev, VIP_CHR_DS_1_DATA_BYPASS, 0);
+			vip_set_slice_path(dev, VIP_RGB_OUT_HI_DATA_SELECT, 0);
+		} else {
+			vip_set_slice_path(dev, VIP_SC_SRC_DATA_SELECT, 0);
+			vip_set_slice_path(dev,
+					   VIP_CHR_DS_2_SRC_DATA_SELECT, 0);
+			vip_set_slice_path(dev, VIP_CHR_DS_1_DATA_BYPASS, 0);
+			vip_set_slice_path(dev, VIP_CHR_DS_2_DATA_BYPASS, 0);
+			vip_set_slice_path(dev, VIP_RGB_OUT_HI_DATA_SELECT, 0);
+		}
+	} else if (port->fmt->coplanar) {
+		if (port->port_id == VIP_PORTA) {
+			vip_set_slice_path(dev,
+					   VIP_CHR_DS_1_SRC_DATA_SELECT, 0);
+			vip_set_slice_path(dev, VIP_CHR_DS_1_DATA_BYPASS, 0);
+			vip_set_slice_path(dev, VIP_RGB_OUT_HI_DATA_SELECT, 0);
+		} else {
+			vip_set_slice_path(dev,
+					   VIP_CHR_DS_2_SRC_DATA_SELECT, 0);
+			vip_set_slice_path(dev, VIP_CHR_DS_2_DATA_BYPASS, 0);
+			vip_set_slice_path(dev,
+					   VIP_MULTI_CHANNEL_DATA_SELECT, 0);
+			vip_set_slice_path(dev, VIP_RGB_OUT_LO_DATA_SELECT, 0);
+		}
+	} else {
+		/*
+		 * We undo all data path setting except for the multi
+		 * stream case.
+		 * Because we cannot disrupt other on-going capture if only
+		 * one stream is terminated the other might still be going
+		 */
+		vip_set_slice_path(dev, VIP_MULTI_CHANNEL_DATA_SELECT, 1);
+		vip_set_slice_path(dev, VIP_RGB_OUT_LO_DATA_SELECT, 0);
+	}
+}
+
Quoted 8 lines

drivers/media/platform/ti-vpe/vip.c line 2096

Quoted 5 lines
drivers/media/platform/ti-vpe/vip.c new file +5 -0
+/*
+ * Set the registers that are modified when the video format changes.
+ */
+static void set_fmt_params(struct vip_stream *stream)
+{

Hmm, this is a *very* long function. Perhaps this could be split up a bit, or reorganized?

Yeah, I'll start by removing the extra comment lines and reformat it.

Quoted 19 lines
Quoted 19 lines
drivers/media/platform/ti-vpe/vip.c new file +18 -0
+	struct vip_dev *dev = stream->port->dev;
+	struct vip_port *port = stream->port;
+
+	stream->sequence = 0;
+	if (stream->port->flags & FLAG_INTERLACED)
+		stream->field = V4L2_FIELD_TOP;
+
+	if (port->csc == VIP_CSC_Y2R) {
+		port->flags &= ~FLAG_MULT_PORT;
+		/* Set alpha component in background color */
+		vpdma_set_bg_color(dev->shared->vpdma,
+				   (struct vpdma_data_format *)
+				   port->fmt->vpdma_fmt[0],
+				   0xff);
+		if (port->port_id == VIP_PORTA) {
+			/*
+			 * Input A: YUV422
+			 * Output: Y_UP/UV_UP: RGB
Quoted 8 lines

drivers/media/platform/ti-vpe/vip.c line 2118

Quoted 4 lines
drivers/media/platform/ti-vpe/vip.c new file +4 -0
+			 * CSC_SRC_SELECT       = 1
+			 * RGB_OUT_HI_SELECT    = 1
+			 * RGB_SRC_SELECT       = 1
+			 * MULTI_CHANNEL_SELECT = 0

It's a bit pointless to comment what the register values should be when you set them in the code below. I'd drop that part, it will make the code shorter.

Ok.

Quoted 8 lines

drivers/media/platform/ti-vpe/vip.c line 2122

Quoted 5 lines
drivers/media/platform/ti-vpe/vip.c new file +4 -0
+			 */
+			vip_set_slice_path(dev, VIP_CSC_SRC_DATA_SELECT, 1);
+			vip_set_slice_path(dev,
+					   VIP_MULTI_CHANNEL_DATA_SELECT, 0);

For readability purposes I think it is better to keep this on one line. Same for the other vip_set_slice_path calls.

Ok.

Quoted 288 lines
Quoted 288 lines
drivers/media/platform/ti-vpe/vip.c new file +287 -0
+			vip_set_slice_path(dev, VIP_RGB_OUT_HI_DATA_SELECT, 1);
+			vip_set_slice_path(dev, VIP_RGB_SRC_DATA_SELECT, 1);
+		} else {
+			/*
+			 * Input B: YUV422
+			 * Output: Y_UP/UV_UP: RGB
+			 * CSC_SRC_SELECT       = 2
+			 * RGB_OUT_LO_SELECT    = 1
+			 * MULTI_CHANNEL_SELECT = 0
+			 */
+			vip_set_slice_path(dev, VIP_CSC_SRC_DATA_SELECT, 2);
+			vip_set_slice_path(dev,
+					   VIP_MULTI_CHANNEL_DATA_SELECT, 0);
+			vip_set_slice_path(dev, VIP_RGB_OUT_LO_DATA_SELECT, 1);
+		}
+		/* We are done */
+		return;
+	} else if (port->csc == VIP_CSC_R2Y) {
+		port->flags &= ~FLAG_MULT_PORT;
+		if (port->scaler && port->fmt->coplanar) {
+			if (port->port_id == VIP_PORTA) {
+				/*
+				 * Input A: RGB
+				 * Output: Y_UP/UV_UP: Scaled YUV420
+				 * CSC_SRC_SELECT       = 4
+				 * SC_SRC_SELECT        = 1
+				 * CHR_DS_1_SRC_SELECT  = 1
+				 * CHR_DS_1_BYPASS      = 0
+				 * RGB_OUT_HI_SELECT    = 0
+				 */
+				vip_set_slice_path(dev,
+						   VIP_CSC_SRC_DATA_SELECT, 4);
+				vip_set_slice_path(dev,
+						   VIP_SC_SRC_DATA_SELECT, 1);
+				vip_set_slice_path(dev,
+						   VIP_CHR_DS_1_SRC_DATA_SELECT,
+						   1);
+				vip_set_slice_path(dev,
+						   VIP_CHR_DS_1_DATA_BYPASS, 0);
+				vip_set_slice_path(dev,
+						   VIP_RGB_OUT_HI_DATA_SELECT,
+						   0);
+			} else {
+				vip_err(stream, "RGB sensor can only be on Port A\n");
+			}
+		} else if (port->scaler) {
+			if (port->port_id == VIP_PORTA) {
+				/*
+				 * Input A: RGB
+				 * Output: Y_UP: Scaled YUV422
+				 * CSC_SRC_SELECT       = 4
+				 * SC_SRC_SELECT        = 1
+				 * CHR_DS_1_SRC_SELECT  = 1
+				 * CHR_DS_1_BYPASS      = 1
+				 * RGB_OUT_HI_SELECT    = 0
+				 */
+				vip_set_slice_path(dev,
+						   VIP_CSC_SRC_DATA_SELECT, 4);
+				vip_set_slice_path(dev,
+						   VIP_SC_SRC_DATA_SELECT, 1);
+				vip_set_slice_path(dev,
+						   VIP_CHR_DS_1_SRC_DATA_SELECT,
+						   1);
+				vip_set_slice_path(dev,
+						   VIP_CHR_DS_1_DATA_BYPASS, 1);
+				vip_set_slice_path(dev,
+						   VIP_RGB_OUT_HI_DATA_SELECT,
+						   0);
+			} else {
+				vip_err(stream, "RGB sensor can only be on Port A\n");
+			}
+		} else if (port->fmt->coplanar) {
+			if (port->port_id == VIP_PORTA) {
+				/*
+				 * Input A: RGB
+				 * Output: Y_UP/UV_UP: YUV420
+				 * CSC_SRC_SELECT       = 4
+				 * CHR_DS_1_SRC_SELECT  = 2
+				 * CHR_DS_1_BYPASS      = 0
+				 * RGB_OUT_HI_SELECT    = 0
+				 */
+				vip_set_slice_path(dev,
+						   VIP_CSC_SRC_DATA_SELECT, 4);
+				vip_set_slice_path(dev,
+						   VIP_CHR_DS_1_SRC_DATA_SELECT,
+						   2);
+				vip_set_slice_path(dev,
+						   VIP_CHR_DS_1_DATA_BYPASS, 0);
+				vip_set_slice_path(dev,
+						   VIP_RGB_OUT_HI_DATA_SELECT,
+						   0);
+			} else {
+				vip_err(stream, "RGB sensor can only be on Port A\n");
+			}
+		} else {
+			if (port->port_id == VIP_PORTA) {
+				/*
+				 * Input A: RGB
+				 * Output: Y_UP/UV_UP: YUV420
+				 * CSC_SRC_SELECT       = 4
+				 * CHR_DS_1_SRC_SELECT  = 2
+				 * CHR_DS_1_BYPASS      = 1
+				 * RGB_OUT_HI_SELECT    = 0
+				 */
+				vip_set_slice_path(dev,
+						   VIP_CSC_SRC_DATA_SELECT, 4);
+				vip_set_slice_path(dev,
+						   VIP_CHR_DS_1_SRC_DATA_SELECT,
+						   2);
+				vip_set_slice_path(dev,
+						   VIP_CHR_DS_1_DATA_BYPASS, 1);
+				vip_set_slice_path(dev,
+						   VIP_RGB_OUT_HI_DATA_SELECT,
+						   0);
+			} else {
+				vip_err(stream, "RGB sensor can only be on Port A\n");
+			}
+		}
+		/* We are done */
+		return;
+	} else if (v4l2_is_format_rgb(port->fmt->finfo)) {
+		port->flags &= ~FLAG_MULT_PORT;
+		/* Set alpha component in background color */
+		vpdma_set_bg_color(dev->shared->vpdma,
+				   (struct vpdma_data_format *)
+				   port->fmt->vpdma_fmt[0],
+				   0xff);
+		if (port->port_id == VIP_PORTA) {
+			/*
+			 * Input A: RGB
+			 * Output: Y_LO/UV_LO: RGB
+			 * RGB_OUT_LO_SELECT    = 1
+			 * MULTI_CHANNEL_SELECT = 1
+			 */
+			vip_set_slice_path(dev,
+					   VIP_MULTI_CHANNEL_DATA_SELECT, 1);
+			vip_set_slice_path(dev, VIP_RGB_OUT_LO_DATA_SELECT, 1);
+		} else {
+			vip_err(stream, "RGB sensor can only be on Port A\n");
+		}
+		/* We are done */
+		return;
+	}
+
+	if (port->scaler && port->fmt->coplanar) {
+		port->flags &= ~FLAG_MULT_PORT;
+		if (port->port_id == VIP_PORTA) {
+			/*
+			 * Input A: YUV422
+			 * Output: Y_UP/UV_UP: Scaled YUV420
+			 * SC_SRC_SELECT        = 2
+			 * CHR_DS_1_SRC_SELECT  = 1
+			 * CHR_DS_1_BYPASS      = 0
+			 * RGB_OUT_HI_SELECT    = 0
+			 */
+			vip_set_slice_path(dev, VIP_SC_SRC_DATA_SELECT, 2);
+			vip_set_slice_path(dev,
+					   VIP_CHR_DS_1_SRC_DATA_SELECT, 1);
+			vip_set_slice_path(dev, VIP_CHR_DS_1_DATA_BYPASS, 0);
+			vip_set_slice_path(dev, VIP_RGB_OUT_HI_DATA_SELECT, 0);
+		} else {
+			/*
+			 * Input B: YUV422
+			 * Output: Y_LO/UV_LO: Scaled YUV420
+			 * SC_SRC_SELECT        = 3
+			 * CHR_DS_2_SRC_SELECT  = 1
+			 * RGB_OUT_LO_SELECT    = 0
+			 * MULTI_CHANNEL_SELECT = 0
+			 */
+			vip_set_slice_path(dev, VIP_SC_SRC_DATA_SELECT, 3);
+			vip_set_slice_path(dev,
+					   VIP_CHR_DS_2_SRC_DATA_SELECT, 1);
+			vip_set_slice_path(dev, VIP_CHR_DS_1_DATA_BYPASS, 0);
+			vip_set_slice_path(dev, VIP_RGB_OUT_LO_DATA_SELECT, 0);
+			vip_set_slice_path(dev,
+					   VIP_MULTI_CHANNEL_DATA_SELECT, 0);
+		}
+	} else if (port->scaler) {
+		port->flags &= ~FLAG_MULT_PORT;
+		if (port->port_id == VIP_PORTA) {
+			/*
+			 * Input A: YUV422
+			 * Output: Y_UP: Scaled YUV422
+			 * SC_SRC_SELECT        = 2
+			 * CHR_DS_1_SRC_SELECT  = 1
+			 * CHR_DS_1_BYPASS      = 1
+			 * RGB_OUT_HI_SELECT    = 0
+			 */
+			vip_set_slice_path(dev, VIP_SC_SRC_DATA_SELECT, 2);
+			vip_set_slice_path(dev,
+					   VIP_CHR_DS_1_SRC_DATA_SELECT, 1);
+			vip_set_slice_path(dev, VIP_CHR_DS_1_DATA_BYPASS, 1);
+			vip_set_slice_path(dev, VIP_RGB_OUT_HI_DATA_SELECT, 0);
+		} else {
+			/*
+			 * Input B: YUV422
+			 * Output: UV_UP: Scaled YUV422
+			 * SC_SRC_SELECT        = 3
+			 * CHR_DS_2_SRC_SELECT  = 1
+			 * CHR_DS_1_BYPASS      = 1
+			 * CHR_DS_2_BYPASS      = 1
+			 * RGB_OUT_HI_SELECT    = 0
+			 */
+			vip_set_slice_path(dev, VIP_SC_SRC_DATA_SELECT, 3);
+			vip_set_slice_path(dev,
+					   VIP_CHR_DS_2_SRC_DATA_SELECT, 1);
+			vip_set_slice_path(dev, VIP_CHR_DS_1_DATA_BYPASS, 1);
+			vip_set_slice_path(dev, VIP_CHR_DS_2_DATA_BYPASS, 1);
+			vip_set_slice_path(dev, VIP_RGB_OUT_HI_DATA_SELECT, 0);
+		}
+	} else if (port->fmt->coplanar) {
+		port->flags &= ~FLAG_MULT_PORT;
+		if (port->port_id == VIP_PORTA) {
+			/*
+			 * Input A: YUV422
+			 * Output: Y_UP/UV_UP: YUV420
+			 * CHR_DS_1_SRC_SELECT  = 3
+			 * CHR_DS_1_BYPASS      = 0
+			 * RGB_OUT_HI_SELECT    = 0
+			 */
+			vip_set_slice_path(dev,
+					   VIP_CHR_DS_1_SRC_DATA_SELECT, 3);
+			vip_set_slice_path(dev, VIP_CHR_DS_1_DATA_BYPASS, 0);
+			vip_set_slice_path(dev, VIP_RGB_OUT_HI_DATA_SELECT, 0);
+		} else {
+			/*
+			 * Input B: YUV422
+			 * Output: Y_LO/UV_LO: YUV420
+			 * CHR_DS_2_SRC_SELECT  = 4
+			 * CHR_DS_2_BYPASS      = 0
+			 * RGB_OUT_LO_SELECT    = 0
+			 * MULTI_CHANNEL_SELECT = 0
+			 */
+			vip_set_slice_path(dev,
+					   VIP_CHR_DS_2_SRC_DATA_SELECT, 4);
+			vip_set_slice_path(dev, VIP_CHR_DS_2_DATA_BYPASS, 0);
+			vip_set_slice_path(dev,
+					   VIP_MULTI_CHANNEL_DATA_SELECT, 0);
+			vip_set_slice_path(dev, VIP_RGB_OUT_LO_DATA_SELECT, 0);
+		}
+	} else {
+		port->flags |= FLAG_MULT_PORT;
+		/*
+		 * Input A/B: YUV422
+		 * Output: Y_LO: YUV422 - UV_LO: YUV422
+		 * MULTI_CHANNEL_SELECT = 1
+		 * RGB_OUT_LO_SELECT    = 0
+		 */
+		vip_set_slice_path(dev, VIP_MULTI_CHANNEL_DATA_SELECT, 1);
+		vip_set_slice_path(dev, VIP_RGB_OUT_LO_DATA_SELECT, 0);
+	}
+}
+
+static int vip_g_selection(struct file *file, void *fh,
+			   struct v4l2_selection *s)
+{
+	struct vip_stream *stream = file2stream(file);
+
+	if (s->type != V4L2_BUF_TYPE_VIDEO_CAPTURE)
+		return -EINVAL;
+
+	switch (s->target) {
+	case V4L2_SEL_TGT_COMPOSE_DEFAULT:
+	case V4L2_SEL_TGT_COMPOSE_BOUNDS:
+	case V4L2_SEL_TGT_CROP_BOUNDS:
+	case V4L2_SEL_TGT_CROP_DEFAULT:
+		s->r.left = 0;
+		s->r.top = 0;
+		s->r.width = stream->width;
+		s->r.height = stream->height;
+		return 0;
+
+	case V4L2_SEL_TGT_COMPOSE:
+	case V4L2_SEL_TGT_CROP:
+		s->r = stream->port->c_rect;
+		return 0;
+	}
+
+	return -EINVAL;
+}
+
+static int enclosed_rectangle(struct v4l2_rect *a, struct v4l2_rect *b)
+{
+	if (a->left < b->left || a->top < b->top)
+		return 0;
+	if (a->left + a->width > b->left + b->width)
+		return 0;
Quoted 8 lines

drivers/media/platform/ti-vpe/vip.c line 2414

Quoted 5 lines
drivers/media/platform/ti-vpe/vip.c new file +5 -0
+	if (a->top + a->height > b->top + b->height)
+		return 0;
+
+	return 1;
+}

There are helper functions in include/media/v4l2-rect.h, it would make sense to add this one to that header.

I'll check that out.

Quoted 16 lines
Quoted 4 lines
drivers/media/platform/ti-vpe/vip.c new file +3 -0
+
+static int vip_s_selection(struct file *file, void *fh,
+			   struct v4l2_selection *s)

drivers/media/platform/ti-vpe/vip.c line 2427

Quoted 10 lines
drivers/media/platform/ti-vpe/vip.c new file +10 -0
+{
+	struct vip_stream *stream = file2stream(file);
+	struct v4l2_rect r = s->r;
+
+	if (s->type != V4L2_BUF_TYPE_VIDEO_CAPTURE)
+		return -EINVAL;
+
+	switch (s->target) {
+	case V4L2_SEL_TGT_COMPOSE:
+	case V4L2_SEL_TGT_CROP:

Why both crop and compose when it is the same c_rect? That makes no sense.

Yeah, this is always puzzling to me. When to use which and what not. I'll catch you on IRC sometime to chat about this.

Quoted 42 lines
Quoted 42 lines
drivers/media/platform/ti-vpe/vip.c new file +41 -0
+		v4l_bound_align_image(&r.width, 0, stream->width, 0,
+				      &r.height, 0, stream->height, 0, 0);
+
+		r.left = clamp_t(unsigned int, r.left, 0,
+				 stream->width - r.width);
+		r.top  = clamp_t(unsigned int, r.top, 0,
+				 stream->height - r.height);
+
+		if (s->flags & V4L2_SEL_FLAG_LE &&
+		    !enclosed_rectangle(&r, &s->r))
+			return -ERANGE;
+
+		if (s->flags & V4L2_SEL_FLAG_GE &&
+		    !enclosed_rectangle(&s->r, &r))
+			return -ERANGE;
+
+		s->r = r;
+		stream->port->c_rect = r;
+
+		vip_dbg(1, stream, "cropped (%d,%d)/%dx%d of %dx%d\n",
+			r.left, r.top, r.width, r.height,
+			stream->width, stream->height);
+
+			s->r = stream->port->c_rect;
+		return 0;
+	default:
+		return -EINVAL;
+	}
+
+	return 0;
+}
+
+static long vip_ioctl_default(struct file *file, void *fh, bool valid_prio,
+			      unsigned int cmd, void *arg)
+{
+	struct vip_stream *stream = file2stream(file);
+
+	if (!valid_prio) {
+		vip_err(stream, "%s device busy\n", __func__);
+		return -EBUSY;
+	}
Quoted 8 lines

drivers/media/platform/ti-vpe/vip.c line 2474

Quoted 6 lines
drivers/media/platform/ti-vpe/vip.c new file +6 -0
+
+	switch (cmd) {
+	default:
+		return -ENOTTY;
+	}
+}

Huh?

Yeah, I don't when this was added or for what purpose. I should probably just remove it.

Quoted 277 lines
Quoted 277 lines
drivers/media/platform/ti-vpe/vip.c new file +276 -0
+
+static const struct v4l2_ioctl_ops vip_ioctl_ops = {
+	.vidioc_querycap	= vip_querycap,
+	.vidioc_enum_input	= vip_enuminput,
+	.vidioc_g_input		= vip_g_input,
+	.vidioc_s_input		= vip_s_input,
+
+	.vidioc_querystd	= vip_querystd,
+	.vidioc_g_std		= vip_g_std,
+	.vidioc_s_std		= vip_s_std,
+
+	.vidioc_enum_fmt_vid_cap = vip_enum_fmt_vid_cap,
+	.vidioc_g_fmt_vid_cap	= vip_g_fmt_vid_cap,
+	.vidioc_try_fmt_vid_cap	= vip_try_fmt_vid_cap,
+	.vidioc_s_fmt_vid_cap	= vip_s_fmt_vid_cap,
+
+	.vidioc_enum_frameintervals	= vip_enum_frameintervals,
+	.vidioc_enum_framesizes		= vip_enum_framesizes,
+	.vidioc_s_parm			= vip_s_parm,
+	.vidioc_g_parm			= vip_g_parm,
+	.vidioc_g_selection	= vip_g_selection,
+	.vidioc_s_selection	= vip_s_selection,
+	.vidioc_reqbufs		= vb2_ioctl_reqbufs,
+	.vidioc_create_bufs	= vb2_ioctl_create_bufs,
+	.vidioc_prepare_buf	= vb2_ioctl_prepare_buf,
+	.vidioc_querybuf	= vb2_ioctl_querybuf,
+	.vidioc_qbuf		= vb2_ioctl_qbuf,
+	.vidioc_dqbuf		= vb2_ioctl_dqbuf,
+	.vidioc_expbuf		= vb2_ioctl_expbuf,
+
+	.vidioc_streamon	= vb2_ioctl_streamon,
+	.vidioc_streamoff	= vb2_ioctl_streamoff,
+	.vidioc_log_status	= v4l2_ctrl_log_status,
+	.vidioc_subscribe_event = v4l2_ctrl_subscribe_event,
+	.vidioc_unsubscribe_event = v4l2_event_unsubscribe,
+	.vidioc_default		= vip_ioctl_default,
+};
+
+/*
+ * Videobuf operations
+ */
+static int vip_queue_setup(struct vb2_queue *vq,
+			   unsigned int *nbuffers, unsigned int *nplanes,
+			   unsigned int sizes[], struct device *alloc_devs[])
+{
+	struct vip_stream *stream = vb2_get_drv_priv(vq);
+	unsigned int size = stream->sizeimage;
+
+	if (vq->num_buffers + *nbuffers < 3)
+		*nbuffers = 3 - vq->num_buffers;
+
+	if (*nplanes) {
+		if (sizes[0] < size)
+			return -EINVAL;
+		size = sizes[0];
+	}
+
+	*nplanes = 1;
+	sizes[0] = size;
+
+	vip_dbg(1, stream, "get %d buffer(s) of size %d each.\n",
+		*nbuffers, sizes[0]);
+
+	return 0;
+}
+
+static int vip_buf_prepare(struct vb2_buffer *vb)
+{
+	struct vip_stream *stream = vb2_get_drv_priv(vb->vb2_queue);
+
+	if (vb2_plane_size(vb, 0) < stream->sizeimage) {
+		vip_dbg(1, stream,
+			"%s data will not fit into plane (%lu < %lu)\n",
+			__func__, vb2_plane_size(vb, 0),
+			(long)stream->sizeimage);
+		return -EINVAL;
+	}
+
+	vb2_set_plane_payload(vb, 0, stream->sizeimage);
+
+	return 0;
+}
+
+static void vip_buf_queue(struct vb2_buffer *vb)
+{
+	struct vip_stream *stream = vb2_get_drv_priv(vb->vb2_queue);
+	struct vip_dev *dev = stream->port->dev;
+	struct vip_buffer *buf = container_of(vb, struct vip_buffer,
+					      vb.vb2_buf);
+	unsigned long flags;
+
+	spin_lock_irqsave(&dev->slock, flags);
+	list_add_tail(&buf->list, &stream->vidq);
+	spin_unlock_irqrestore(&dev->slock, flags);
+}
+
+static int vip_setup_scaler(struct vip_stream *stream)
+{
+	struct vip_port *port = stream->port;
+	struct vip_dev *dev = port->dev;
+	struct sc_data *sc = dev->sc;
+	struct csc_data *csc = dev->csc;
+	struct vpdma_data *vpdma = dev->shared->vpdma;
+	struct vip_mmr_adb *mmr_adb = port->mmr_adb.addr;
+	int list_num = stream->list_num;
+	int timeout = 500;
+	struct v4l2_format dst_f;
+	struct v4l2_format src_f;
+
+	memset(&src_f, 0, sizeof(src_f));
+	src_f.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
+	v4l2_fill_pix_format(&src_f.fmt.pix, &port->mbus_framefmt);
+	src_f.fmt.pix.pixelformat = vip_mbus_code_to_fourcc(port->fmt->code);
+
+	dst_f = src_f;
+	dst_f.fmt.pix.pixelformat = port->fmt->fourcc;
+	dst_f.fmt.pix.width = stream->width;
+	dst_f.fmt.pix.height = stream->height;
+
+	/* if scaler not associated with this port then skip */
+	if (port->scaler) {
+		sc_set_hs_coeffs(sc, port->sc_coeff_h.addr,
+				 port->mbus_framefmt.width,
+				 port->c_rect.width);
+		sc_set_vs_coeffs(sc, port->sc_coeff_v.addr,
+				 port->mbus_framefmt.height,
+				 port->c_rect.height);
+		sc_config_scaler(sc, &mmr_adb->sc_regs0[0],
+				 &mmr_adb->sc_regs8[0], &mmr_adb->sc_regs17[0],
+				 port->mbus_framefmt.width,
+				 port->mbus_framefmt.height,
+				 port->c_rect.width,
+				 port->c_rect.height);
+		port->load_mmrs = true;
+	}
+
+	/* if csc not associated with this port then skip */
+	if (port->csc) {
+		csc_set_coeff(csc, &mmr_adb->csc_regs[0],
+			      &src_f, &dst_f);
+
+		port->load_mmrs = true;
+	}
+
+	/* If coeff are already loaded then skip */
+	if (!sc->load_coeff_v && !sc->load_coeff_h && !port->load_mmrs)
+		return 0;
+
+	if (vpdma_list_busy(vpdma, list_num)) {
+		vip_dbg(3, stream, "%s: List %d is busy\n",
+			__func__, list_num);
+	}
+
+	/* Make sure we start with a clean list */
+	vpdma_reset_desc_list(&stream->desc_list);
+
+	/* config descriptors */
+	if (port->load_mmrs) {
+		vpdma_map_desc_buf(vpdma, &port->mmr_adb);
+		vpdma_add_cfd_adb(&stream->desc_list, CFD_MMR_CLIENT,
+				  &port->mmr_adb);
+
+		port->load_mmrs = false;
+		vip_dbg(3, stream, "Added mmr_adb config desc\n");
+	}
+
+	if (sc->loaded_coeff_h != port->sc_coeff_h.dma_addr ||
+	    sc->load_coeff_h) {
+		vpdma_map_desc_buf(vpdma, &port->sc_coeff_h);
+		vpdma_add_cfd_block(&stream->desc_list,
+				    VIP_SLICE1_CFD_SC_CLIENT + dev->slice_id,
+				    &port->sc_coeff_h, 0);
+
+		sc->loaded_coeff_h = port->sc_coeff_h.dma_addr;
+		sc->load_coeff_h = false;
+		vip_dbg(3, stream, "Added sc_coeff_h config desc\n");
+	}
+
+	if (sc->loaded_coeff_v != port->sc_coeff_v.dma_addr ||
+	    sc->load_coeff_v) {
+		vpdma_map_desc_buf(vpdma, &port->sc_coeff_v);
+		vpdma_add_cfd_block(&stream->desc_list,
+				    VIP_SLICE1_CFD_SC_CLIENT + dev->slice_id,
+				    &port->sc_coeff_v, SC_COEF_SRAM_SIZE >> 4);
+
+		sc->loaded_coeff_v = port->sc_coeff_v.dma_addr;
+		sc->load_coeff_v = false;
+		vip_dbg(3, stream, "Added sc_coeff_v config desc\n");
+	}
+	vip_dbg(3, stream, "CFD_SC_CLIENT %d slice_id: %d\n",
+		VIP_SLICE1_CFD_SC_CLIENT + dev->slice_id, dev->slice_id);
+
+	vpdma_map_desc_buf(vpdma, &stream->desc_list.buf);
+	vip_dbg(3, stream, "Submitting desc on list# %d\n", list_num);
+	vpdma_submit_descs(vpdma, &stream->desc_list, list_num);
+
+	while (vpdma_list_busy(vpdma, list_num) && timeout--)
+		usleep_range(1000, 1100);
+
+	vpdma_unmap_desc_buf(dev->shared->vpdma, &port->mmr_adb);
+	vpdma_unmap_desc_buf(dev->shared->vpdma, &port->sc_coeff_h);
+	vpdma_unmap_desc_buf(dev->shared->vpdma, &port->sc_coeff_v);
+	vpdma_unmap_desc_buf(dev->shared->vpdma, &stream->desc_list.buf);
+
+	vpdma_reset_desc_list(&stream->desc_list);
+
+	if (timeout <= 0) {
+		vip_err(stream, "Timed out setting up scaler through VPDMA list\n");
+		return -EBUSY;
+	}
+
+	return 0;
+}
+
+static int vip_load_vpdma_list_fifo(struct vip_stream *stream)
+{
+	struct vip_port *port = stream->port;
+	struct vip_dev *dev = port->dev;
+	struct vpdma_data *vpdma = dev->shared->vpdma;
+	int list_num = stream->list_num;
+	struct vip_buffer *buf;
+	unsigned long flags;
+	int timeout, i;
+
+	if (vpdma_list_busy(dev->shared->vpdma, stream->list_num))
+		return -EBUSY;
+
+	for (i = 0; i < VIP_VPDMA_FIFO_SIZE; i++) {
+		spin_lock_irqsave(&dev->slock, flags);
+		if (list_empty(&stream->vidq)) {
+			vip_err(stream, "No buffer left!");
+			spin_unlock_irqrestore(&dev->slock, flags);
+			return -EINVAL;
+		}
+
+		buf = list_entry(stream->vidq.next,
+				 struct vip_buffer, list);
+		buf->drop = false;
+
+		list_move_tail(&buf->list, &stream->post_bufs);
+		spin_unlock_irqrestore(&dev->slock, flags);
+
+		vip_dbg(2, stream, "%s: start_dma vb2 buf idx:%d\n",
+			__func__, buf->vb.vb2_buf.index);
+		start_dma(stream, buf);
+
+		timeout = 500;
+		while (vpdma_list_busy(vpdma, list_num) && timeout--)
+			usleep_range(1000, 1100);
+
+		if (timeout <= 0) {
+			vip_err(stream, "Timed out loading VPDMA list fifo\n");
+			return -EBUSY;
+		}
+	}
+	return 0;
+}
+
+static int vip_start_streaming(struct vb2_queue *vq, unsigned int count)
+{
+	struct vip_stream *stream = vb2_get_drv_priv(vq);
+	struct vip_port *port = stream->port;
+	struct vip_dev *dev = port->dev;
+	int ret;
+
+	vip_setup_scaler(stream);
+
+	/*
+	 * Make sure the scaler is configured before the datapath is
+	 * enabled. The scaler can only load the coefficient
+	 * parameters when it is idle. If the scaler path is enabled
+	 * and video data is being received then the VPDMA transfer will
+	 * stall indefinetely.
+	 */
+	set_fmt_params(stream);
+	vip_setup_parser(port);
Quoted 8 lines

drivers/media/platform/ti-vpe/vip.c line 2755

Quoted 5 lines
drivers/media/platform/ti-vpe/vip.c new file +5 -0
+
+	if (port->subdev) {
+		ret = v4l2_subdev_call(port->subdev, video, s_stream, 1);
+		if (ret < 0 && ret != -ENOIOCTLCMD) {
+			vip_dbg(1, stream, "stream on failed in subdev\n");

On error, all pending buffers need to be returned to vb2 with vb2_buffer_done and state VB2_BUF_STATE_QUEUED.

Ah yes missed this.

Quoted 726 lines
Quoted 726 lines
drivers/media/platform/ti-vpe/vip.c new file +725 -0
+			return ret;
+		}
+	}
+
+	stream->sequence = 0;
+	if (stream->port->flags & FLAG_INTERLACED)
+		stream->field = V4L2_FIELD_TOP;
+	populate_desc_list(stream);
+
+	ret = vip_load_vpdma_list_fifo(stream);
+	if (ret)
+		return ret;
+
+	stream->num_recovery = 0;
+
+	clear_irqs(dev, dev->slice_id, stream->list_num);
+	enable_irqs(dev, dev->slice_id, stream->list_num);
+	vip_schedule_next_buffer(stream);
+	vip_parser_stop_imm(port, false);
+	vip_enable_parser(port, true);
+
+	return 0;
+}
+
+/*
+ * Abort streaming and wait for last buffer
+ */
+static void vip_stop_streaming(struct vb2_queue *vq)
+{
+	struct vip_stream *stream = vb2_get_drv_priv(vq);
+	struct vip_port *port = stream->port;
+	struct vip_dev *dev = port->dev;
+	struct vip_buffer *buf;
+	int ret;
+
+	vip_dbg(2, stream, "%s:\n", __func__);
+
+	vip_parser_stop_imm(port, true);
+	vip_enable_parser(port, false);
+	unset_fmt_params(stream);
+
+	disable_irqs(dev, dev->slice_id, stream->list_num);
+	clear_irqs(dev, dev->slice_id, stream->list_num);
+
+	if (port->subdev) {
+		ret = v4l2_subdev_call(port->subdev, video, s_stream, 0);
+		if (ret < 0 && ret != -ENOIOCTLCMD)
+			vip_dbg(1, stream, "stream on failed in subdev\n");
+	}
+
+	stop_dma(stream, true);
+
+	/* release all active buffers */
+	while (!list_empty(&stream->post_bufs)) {
+		buf = list_entry(stream->post_bufs.next,
+				 struct vip_buffer, list);
+		list_del(&buf->list);
+		if (buf->drop == 1)
+			list_add_tail(&buf->list, &stream->dropq);
+		else
+			vb2_buffer_done(&buf->vb.vb2_buf, VB2_BUF_STATE_ERROR);
+	}
+	while (!list_empty(&stream->vidq)) {
+		buf = list_entry(stream->vidq.next, struct vip_buffer, list);
+		list_del(&buf->list);
+		vb2_buffer_done(&buf->vb.vb2_buf, VB2_BUF_STATE_ERROR);
+	}
+
+	if (!vb2_is_streaming(vq))
+		return;
+
+	vpdma_unmap_desc_buf(dev->shared->vpdma, &stream->desc_list.buf);
+	vpdma_reset_desc_list(&stream->desc_list);
+}
+
+static const struct vb2_ops vip_video_qops = {
+	.queue_setup		= vip_queue_setup,
+	.buf_prepare		= vip_buf_prepare,
+	.buf_queue		= vip_buf_queue,
+	.start_streaming	= vip_start_streaming,
+	.stop_streaming		= vip_stop_streaming,
+	.wait_prepare		= vb2_ops_wait_prepare,
+	.wait_finish		= vb2_ops_wait_finish,
+};
+
+/*
+ * File operations
+ */
+
+static int vip_init_dev(struct vip_dev *dev)
+{
+	if (dev->num_ports != 0)
+		goto done;
+
+	vip_set_clock_enable(dev, 1);
+	vip_module_reset(dev, VIP_SC_RST, false);
+	vip_module_reset(dev, VIP_CSC_RST, false);
+done:
+	dev->num_ports++;
+
+	return 0;
+}
+
+static inline bool is_scaler_available(struct vip_port *port)
+{
+	if (port->endpoint.bus_type == V4L2_MBUS_PARALLEL)
+		if (port->dev->sc_assigned == VIP_NOT_ASSIGNED ||
+		    port->dev->sc_assigned == port->port_id)
+			return true;
+	return false;
+}
+
+static inline bool allocate_scaler(struct vip_port *port)
+{
+	if (is_scaler_available(port)) {
+		if (port->dev->sc_assigned == VIP_NOT_ASSIGNED ||
+		    port->dev->sc_assigned == port->port_id) {
+			port->dev->sc_assigned = port->port_id;
+			port->scaler = true;
+			return true;
+		}
+	}
+	return false;
+}
+
+static inline void free_scaler(struct vip_port *port)
+{
+	if (port->dev->sc_assigned == port->port_id) {
+		port->dev->sc_assigned = VIP_NOT_ASSIGNED;
+		port->scaler = false;
+	}
+}
+
+static bool is_csc_available(struct vip_port *port)
+{
+	if (port->endpoint.bus_type == V4L2_MBUS_PARALLEL)
+		if (port->dev->csc_assigned == VIP_NOT_ASSIGNED ||
+		    port->dev->csc_assigned == port->port_id)
+			return true;
+	return false;
+}
+
+static bool allocate_csc(struct vip_port *port,
+				enum vip_csc_state csc_direction)
+{
+	/* Is CSC needed? */
+	if (csc_direction != VIP_CSC_NA) {
+		if (is_csc_available(port)) {
+			port->dev->csc_assigned = port->port_id;
+			port->csc = csc_direction;
+			vip_dbg(1, port, "%s: csc allocated: dir: %d\n",
+				__func__, csc_direction);
+			return true;
+		}
+	}
+	return false;
+}
+
+static void free_csc(struct vip_port *port)
+{
+	if (port->dev->csc_assigned == port->port_id) {
+		port->dev->csc_assigned = VIP_NOT_ASSIGNED;
+		port->csc = VIP_CSC_NA;
+		vip_dbg(1, port, "%s: csc freed\n",
+			__func__);
+	}
+}
+
+static int vip_init_port(struct vip_port *port)
+{
+	int ret;
+	struct vip_fmt *fmt;
+	struct v4l2_subdev_format sd_fmt;
+	struct v4l2_mbus_framefmt *mbus_fmt = &sd_fmt.format;
+
+	if (port->num_streams != 0)
+		goto done;
+
+	ret = vip_init_dev(port->dev);
+	if (ret)
+		goto done;
+
+	/* Get subdevice current frame format */
+	sd_fmt.which = V4L2_SUBDEV_FORMAT_ACTIVE;
+	sd_fmt.pad = port->source_pad;
+	ret = v4l2_subdev_call(port->subdev, pad, get_fmt, NULL, &sd_fmt);
+	if (ret)
+		vip_dbg(1, port, "init_port get_fmt failed in subdev: (%d)\n",
+			ret);
+
+	/* try to find one that matches */
+	fmt = find_port_format_by_code(port, mbus_fmt->code);
+	if (!fmt) {
+		vip_dbg(1, port, "subdev default mbus_fmt %04x is not matched.\n",
+			mbus_fmt->code);
+		/* if all else fails just pick the first one */
+		fmt = port->active_fmt[0];
+
+		mbus_fmt->code = fmt->code;
+		sd_fmt.which = V4L2_SUBDEV_FORMAT_ACTIVE;
+		sd_fmt.pad = port->source_pad;
+		ret = v4l2_subdev_call(port->subdev, pad, set_fmt,
+				       NULL, &sd_fmt);
+		if (ret)
+			vip_dbg(1, port, "init_port set_fmt failed in subdev: (%d)\n",
+				ret);
+	}
+
+	/* Assign current format */
+	port->fmt = fmt;
+	port->mbus_framefmt = *mbus_fmt;
+
+	vip_dbg(3, port, "%s: g_mbus_fmt subdev mbus_code: %04X fourcc:%s size: %dx%d\n",
[… more lines in the raw message]

Lines of this file are not shown. View the raw message.

The rest of this diff is not shown. View the raw message.

Quoted 8 lines

drivers/media/platform/ti-vpe/vip.c

Quoted 6 lines
drivers/media/platform/ti-vpe/vip.c new file +6 -0

Lines of this file are not shown. View the raw message.

The rest of this diff is not shown. View the raw message.

Shouldn't this be done before the call to video_register_device()?

Yep most likely.

Quoted 1,407 lines

drivers/media/platform/ti-vpe/vip.h

Quoted 1,404 lines
drivers/media/platform/ti-vpe/vip.c new file +672 -0

Lines of this file are not shown. View the raw message.

drivers/media/platform/ti-vpe/vip.h new file +724 -0
new file mode 100644
index 000000000000..f078a16a85b7

Lines of this file are not shown. View the raw message.

The rest of this diff is not shown. View the raw message.

Regards,

Quoted 1 line

Hans

Regards,
Benoit

Part of the diff is not shown here. View the raw message.

#7 lore raw
Quoted 24 lines · Benoit Parrot

Hans,

Thanks for the review.

Hans Verkuil <[email protected]> wrote on Tue [2020-May-26 13:48:35 +0200]:

Quoted 19 lines · Benoit Parrot
Quoted 18 lines

VIP stands for Video Input Port, it can be found on devices such as DRA7xx and provides a parallel interface to a video source such as a sensor or TV decoder. Each VIP can support two inputs (slices) and a SoC can be configured with a variable number of VIP's. Each slice can supports two ports each connected to its own sub-device.

---
 drivers/media/platform/Kconfig         |   13 +
 drivers/media/platform/ti-vpe/Makefile |    2 +
 drivers/media/platform/ti-vpe/vip.c    | 4158 ++++++++++++++++++++++++
 drivers/media/platform/ti-vpe/vip.h    |  724 +++++
 4 files changed, 4897 insertions(+)
 create mode 100644 drivers/media/platform/ti-vpe/vip.c
 create mode 100644 drivers/media/platform/ti-vpe/vip.h

<snip>

Quoted 21 lines
Quoted 9 lines
Quoted 9 lines
drivers/media/platform/ti-vpe/vip.c new file +9 -0
+static int vip_enum_fmt_vid_cap(struct file *file, void *priv,
+				struct v4l2_fmtdesc *f)
+{
+	struct vip_stream *stream = file2stream(file);
+	struct vip_port *port = stream->port;
+	struct vip_fmt *fmt;
+
+	vip_dbg(3, stream, "enum_fmt index:%d\n", f->index);
+	if (f->index >= port->num_active_fmt)
Quoted 8 lines

drivers/media/platform/ti-vpe/vip.c line 1399

Quoted 6 lines
drivers/media/platform/ti-vpe/vip.c new file +6 -0
+		return -EINVAL;
+
+	fmt = port->active_fmt[f->index];
+
+	f->pixelformat = fmt->fourcc;
+	f->type = V4L2_BUF_TYPE_VIDEO_CAPTURE;

No need to set the type field.

Ok.

Quoted 1 line
Quoted 6 lines
Quoted 4 lines

drivers/media/platform/ti-vpe/vip.c line 1401

Quoted 2 lines
drivers/media/platform/ti-vpe/vip.c new file +2 -0
+	vip_dbg(3, stream, "enum_fmt fourcc:%s\n",
+		fourcc_to_str(f->pixelformat));

Excessive debugging.

Why excessive?

Two debug messages for a simple function like this seems overkill.

Besides, the v4l2 core already has debugging support for ioctl calls:

https://linuxtv.org/downloads/v4l-dvb-apis-new/driver-api/v4l2-dev.html#video-device-debugging

Quoted 33 lines
Quoted 33 lines
Quoted 18 lines
drivers/media/platform/ti-vpe/vip.c new file +16 -0
+
+	return 0;
+}
+
+static int vip_enum_framesizes(struct file *file, void *priv,
+			       struct v4l2_frmsizeenum *f)
+{
+	struct vip_stream *stream = file2stream(file);
+	struct vip_port *port = stream->port;
+	struct vip_fmt *fmt;
+	struct v4l2_subdev_frame_size_enum fse;
+	int ret;
+
+	fmt = find_port_format_by_pix(port, f->pixel_format);
+	if (!fmt)
+		return -EINVAL;

drivers/media/platform/ti-vpe/vip.c line 1427

Quoted 10 lines
drivers/media/platform/ti-vpe/vip.c new file +10 -0
+
+	fse.index = f->index;
+	fse.pad = port->source_pad;
+	fse.code = fmt->code;
+	fse.which = V4L2_SUBDEV_FORMAT_ACTIVE;
+	ret = v4l2_subdev_call(port->subdev, pad, enum_frame_size, NULL, &fse);
+	if (ret == -ENOIOCTLCMD && !f->index) {
+		/*
+		 * if subdev does not support enum_frame_size
+		 * then use get_fmt

I don't think that's right. If the subdev doesn't support this, then this ioctl should be disabled altogether. Typically this ioctl is only valid for sensor subdevs, not for video receivers.

Quoted 4 lines
Quoted 1 line

Use v4l2_subdev_has_op() and v4l2_disable_ioctl().

You mean to check if the subdev support this ioctl and if not disable it for the current video device only, correct?

Correct.

There are several other drivers that do this, just grep for them.

Quoted 44 lines
Quoted 35 lines
Quoted 35 lines
drivers/media/platform/ti-vpe/vip.c new file +35 -0
+static int vip_calc_format_size(struct vip_port *port,
+				struct vip_fmt *fmt,
+				struct v4l2_format *f)
+{
+	enum v4l2_field *field;
+	unsigned int stride;
+
+	if (!fmt) {
+		vip_dbg(2, port,
+			"no vip_fmt format provided!\n");
+		return -EINVAL;
+	}
+
+	field = &f->fmt.pix.field;
+	if (*field == V4L2_FIELD_ANY)
+		*field = V4L2_FIELD_NONE;
+	else if (V4L2_FIELD_NONE != *field && V4L2_FIELD_ALTERNATE != *field)
+		return -EINVAL;
+
+	v4l_bound_align_image(&f->fmt.pix.width, MIN_W, MAX_W, W_ALIGN,
+			      &f->fmt.pix.height, MIN_H, MAX_H, H_ALIGN,
+			      S_ALIGN);
+
+	stride = f->fmt.pix.width * (fmt->vpdma_fmt[0]->depth >> 3);
+	if (stride > f->fmt.pix.bytesperline)
+		f->fmt.pix.bytesperline = stride;
+
+	f->fmt.pix.bytesperline = clamp_t(u32, f->fmt.pix.bytesperline,
+					  stride, VPDMA_MAX_STRIDE);
+	f->fmt.pix.bytesperline = ALIGN(f->fmt.pix.bytesperline,
+					VPDMA_STRIDE_ALIGN);
+
+	f->fmt.pix.sizeimage = f->fmt.pix.height * f->fmt.pix.bytesperline;
+	if (fmt->coplanar) {
+		f->fmt.pix.sizeimage += f->fmt.pix.height *
Quoted 8 lines

drivers/media/platform/ti-vpe/vip.c line 1542

Quoted 6 lines
drivers/media/platform/ti-vpe/vip.c new file +6 -0
+					f->fmt.pix.bytesperline *
+					fmt->vpdma_fmt[VIP_CHROMA]->depth >> 3;
+	}
+
+	f->fmt.pix.colorspace = fmt->colorspace;
+	f->fmt.pix.priv = 0;

No need to set this.

Quoted 2 lines

You mean pix.priv? I thought I remember v4l2-compliance complaining about something like this?

Yes, pix.priv. It used to complain a long time ago, but the v4l2 core should handle this. Basically drivers shouldn't touch pix.priv.

Quoted 919 lines
Quoted 551 lines
Quoted 551 lines
drivers/media/platform/ti-vpe/vip.c new file +549 -0
+
+	vip_dbg(3, port, "calc_format_size: fourcc:%s size: %dx%d bpl:%d img_size:%d\n",
+		fourcc_to_str(f->fmt.pix.pixelformat),
+		f->fmt.pix.width, f->fmt.pix.height,
+		f->fmt.pix.bytesperline, f->fmt.pix.sizeimage);
+
+	return 0;
+}
+
+static inline bool vip_is_size_dma_aligned(u32 bpp, u32 width)
+{
+	return ((width * bpp) == ALIGN(width * bpp, VPDMA_STRIDE_ALIGN));
+}
+
+static int vip_try_fmt_vid_cap(struct file *file, void *priv,
+			       struct v4l2_format *f)
+{
+	struct vip_stream *stream = file2stream(file);
+	struct vip_port *port = stream->port;
+	struct v4l2_subdev_frame_size_enum fse;
+	struct vip_fmt *fmt;
+	u32 best_width, best_height, largest_width, largest_height;
+	int ret, found;
+	enum vip_csc_state csc_direction;
+
+	vip_dbg(3, stream, "try_fmt fourcc:%s size: %dx%d\n",
+		fourcc_to_str(f->fmt.pix.pixelformat),
+		f->fmt.pix.width, f->fmt.pix.height);
+
+	fmt = find_port_format_by_pix(port, f->fmt.pix.pixelformat);
+	if (!fmt) {
+		vip_dbg(2, stream,
+			"Fourcc format (0x%08x) not found.\n",
+			f->fmt.pix.pixelformat);
+
+		/* Just get the first one enumerated */
+		fmt = port->active_fmt[0];
+		f->fmt.pix.pixelformat = fmt->fourcc;
+	}
+
+	csc_direction =  vip_csc_direction(fmt->code, fmt->finfo);
+	if (csc_direction != VIP_CSC_NA) {
+		if (!is_csc_available(port)) {
+			vip_dbg(2, stream,
+				"CSC not available for Fourcc format (0x%08x).\n",
+				f->fmt.pix.pixelformat);
+
+			/* Just get the first one enumerated */
+			fmt = port->active_fmt[0];
+			f->fmt.pix.pixelformat = fmt->fourcc;
+			/* re-evaluate the csc_direction here */
+			csc_direction =  vip_csc_direction(fmt->code,
+							   fmt->finfo);
+		} else {
+			vip_dbg(3, stream, "CSC active on Port %c: going %s\n",
+				port->port_id == VIP_PORTA ? 'A' : 'B',
+				(csc_direction == VIP_CSC_Y2R) ? "Y2R" : "R2Y");
+		}
+	}
+
+	/*
+	 * Given that sensors might support multiple mbus code we need
+	 * to use the one that matches the requested pixel format
+	 */
+	port->try_mbus_framefmt = port->mbus_framefmt;
+	port->try_mbus_framefmt.code = fmt->code;
+
+	/* check for/find a valid width/height */
+	ret = 0;
+	found = false;
+	best_width = 0;
+	best_height = 0;
+	largest_width = 0;
+	largest_height = 0;
+	fse.pad = port->source_pad;
+	fse.code = fmt->code;
+	fse.which = V4L2_SUBDEV_FORMAT_ACTIVE;
+	for (fse.index = 0; ; fse.index++) {
+		u32 bpp = fmt->vpdma_fmt[0]->depth >> 3;
+
+		ret = v4l2_subdev_call(port->subdev, pad,
+				       enum_frame_size, NULL, &fse);
+		if (ret == -ENOIOCTLCMD) {
+			/*
+			 * if subdev does not support enum_frame_size
+			 * then just try to set_fmt directly
+			 */
+			struct v4l2_subdev_format format = {
+				.which = V4L2_SUBDEV_FORMAT_TRY,
+			};
+			struct v4l2_subdev_pad_config *pad_cfg;
+
+			pad_cfg = v4l2_subdev_alloc_pad_config(port->subdev);
+			if (!pad_cfg)
+				return -ENOMEM;
+
+			v4l2_fill_mbus_format(&format.format, &f->fmt.pix,
+					      fmt->code);
+			ret = v4l2_subdev_call(port->subdev, pad, set_fmt,
+					       pad_cfg, &format);
+			if (ret)
+				/* here regardless of the reason we give up */
+				break;
+
+			if (f->fmt.pix.width == format.format.width &&
+			    f->fmt.pix.height == format.format.height) {
+				found = true;
+				vip_dbg(3, stream, "try_fmt loop:%d found direct match: %dx%d\n",
+					fse.index, format.format.width,
+					format.format.height);
+			}
+			largest_width = format.format.width;
+			largest_height = format.format.height;
+			best_width = format.format.width;
+			best_height = format.format.height;
+
+			v4l2_subdev_free_pad_config(pad_cfg);
+			break;
+
+		} else if (ret) {
+			break;
+		}
+
+		vip_dbg(3, stream, "try_fmt loop:%d fourcc:%s size: %dx%d\n",
+			fse.index, fourcc_to_str(f->fmt.pix.pixelformat),
+			fse.max_width, fse.max_height);
+
+		if (!vip_is_size_dma_aligned(bpp, fse.max_width))
+			continue;
+
+		if ((fse.max_width >= largest_width) &&
+		    (fse.max_height >= largest_height)) {
+			vip_dbg(3, stream, "try_fmt loop:%d found new larger: %dx%d\n",
+				fse.index, fse.max_width, fse.max_height);
+			largest_width = fse.max_width;
+			largest_height = fse.max_height;
+		}
+
+		if ((fse.max_width >= f->fmt.pix.width) &&
+		    (fse.max_height >= f->fmt.pix.height)) {
+			vip_dbg(3, stream, "try_fmt loop:%d found at least larger: %dx%d\n",
+				fse.index, fse.max_width, fse.max_height);
+
+			if (!best_width ||
+			    ((abs(best_width - f->fmt.pix.width) >=
+			      abs(fse.max_width - f->fmt.pix.width)) &&
+			     (abs(best_height - f->fmt.pix.height) >=
+			      abs(fse.max_height - f->fmt.pix.height)))) {
+				best_width = fse.max_width;
+				best_height = fse.max_height;
+				vip_dbg(3, stream, "try_fmt loop:%d found new best: %dx%d\n",
+					fse.index, fse.max_width,
+					fse.max_height);
+			}
+		}
+
+		if ((f->fmt.pix.width == fse.max_width) &&
+		    (f->fmt.pix.height == fse.max_height)) {
+			found = true;
+			vip_dbg(3, stream, "try_fmt loop:%d found direct match: %dx%d\n",
+				fse.index, fse.max_width,
+				fse.max_height);
+			break;
+		}
+
+		if ((f->fmt.pix.width >= fse.min_width) &&
+		    (f->fmt.pix.width <= fse.max_width) &&
+		    (f->fmt.pix.height >= fse.min_height) &&
+		    (f->fmt.pix.height <= fse.max_height)) {
+			found = true;
+			vip_dbg(3, stream, "try_fmt loop:%d found direct range match: %dx%d\n",
+				fse.index, fse.max_width,
+				fse.max_height);
+			break;
+		}
+	}
+
+	if (found) {
+		port->try_mbus_framefmt.width = f->fmt.pix.width;
+		port->try_mbus_framefmt.height = f->fmt.pix.height;
+		/* No need to check for scaling */
+		goto calc_size;
+	} else if (largest_width && f->fmt.pix.width > largest_width) {
+		port->try_mbus_framefmt.width = largest_width;
+		port->try_mbus_framefmt.height = largest_height;
+	} else if (best_width) {
+		port->try_mbus_framefmt.width = best_width;
+		port->try_mbus_framefmt.height = best_height;
+	} else {
+		/* use existing values as default */
+	}
+
+	vip_dbg(3, stream, "try_fmt best subdev size: %dx%d\n",
+		port->try_mbus_framefmt.width,
+		port->try_mbus_framefmt.height);
+
+	if (is_scaler_available(port) &&
+	    csc_direction != VIP_CSC_Y2R &&
+	    !vip_is_mbuscode_raw(fmt->code) &&
+	    f->fmt.pix.height <= port->try_mbus_framefmt.height &&
+	    port->try_mbus_framefmt.height <= SC_MAX_PIXEL_HEIGHT &&
+	    port->try_mbus_framefmt.width <= SC_MAX_PIXEL_WIDTH) {
+		/*
+		 * Scaler is only accessible if the dst colorspace is YUV.
+		 * As the input to the scaler must be in YUV mode only.
+		 *
+		 * Scaling up is allowed only horizontally.
+		 */
+		unsigned int hratio, vratio, width_align, height_align;
+		u32 bpp = fmt->vpdma_fmt[0]->depth >> 3;
+
+		vip_dbg(3, stream, "Scaler active on Port %c: requesting %dx%d\n",
+			port->port_id == VIP_PORTA ? 'A' : 'B',
+			f->fmt.pix.width, f->fmt.pix.height);
+
+		/* Just make sure everything is properly aligned */
+		width_align = ALIGN(f->fmt.pix.width * bpp, VPDMA_STRIDE_ALIGN);
+		width_align /= bpp;
+		height_align = ALIGN(f->fmt.pix.height, 2);
+
+		f->fmt.pix.width = width_align;
+		f->fmt.pix.height = height_align;
+
+		hratio = f->fmt.pix.width * 1000 /
+			 port->try_mbus_framefmt.width;
+		vratio = f->fmt.pix.height * 1000 /
+			 port->try_mbus_framefmt.height;
+		if (hratio < 125) {
+			f->fmt.pix.width = port->try_mbus_framefmt.width / 8;
+			vip_dbg(3, stream, "Horizontal scaling ratio out of range adjusting -> %d\n",
+				f->fmt.pix.width);
+		}
+
+		if (vratio < 188) {
+			f->fmt.pix.height = port->try_mbus_framefmt.height / 4;
+			vip_dbg(3, stream, "Vertical scaling ratio out of range adjusting -> %d\n",
+				f->fmt.pix.height);
+		}
+		vip_dbg(3, stream, "Scaler: got %dx%d\n",
+			f->fmt.pix.width, f->fmt.pix.height);
+	} else {
+		/* use existing values as default */
+		f->fmt.pix.width = port->try_mbus_framefmt.width;
+		f->fmt.pix.height = port->try_mbus_framefmt.height;
+	}
+
+calc_size:
+	/* That we have a fmt calculate imagesize and bytesperline */
+	return vip_calc_format_size(port, fmt, f);
+}
+
+static int vip_g_fmt_vid_cap(struct file *file, void *priv,
+			     struct v4l2_format *f)
+{
+	struct vip_stream *stream = file2stream(file);
+	struct vip_port *port = stream->port;
+	struct vip_fmt *fmt = port->fmt;
+
+	/* Use last known values or defaults */
+	f->fmt.pix.width	= stream->width;
+	f->fmt.pix.height	= stream->height;
+	f->fmt.pix.pixelformat	= port->fmt->fourcc;
+	f->fmt.pix.field	= stream->sup_field;
+	f->fmt.pix.colorspace	= port->fmt->colorspace;
+	f->fmt.pix.bytesperline	= stream->bytesperline;
+	f->fmt.pix.sizeimage	= stream->sizeimage;
+
+	vip_dbg(3, stream,
+		"g_fmt fourcc:%s code: %04x size: %dx%d bpl:%d img_size:%d\n",
+		fourcc_to_str(f->fmt.pix.pixelformat),
+		fmt->code,
+		f->fmt.pix.width, f->fmt.pix.height,
+		f->fmt.pix.bytesperline, f->fmt.pix.sizeimage);
+	vip_dbg(3, stream, "g_fmt vpdma data type: 0x%02X\n",
+		port->fmt->vpdma_fmt[0]->data_type);
+
+	return 0;
+}
+
+static int vip_s_fmt_vid_cap(struct file *file, void *priv,
+			     struct v4l2_format *f)
+{
+	struct vip_stream *stream = file2stream(file);
+	struct vip_port *port = stream->port;
+	struct v4l2_subdev_format sfmt;
+	struct v4l2_mbus_framefmt *mf;
+	enum vip_csc_state csc_direction;
+	int ret;
+
+	vip_dbg(3, stream, "s_fmt input fourcc:%s size: %dx%d bpl:%d img_size:%d\n",
+		fourcc_to_str(f->fmt.pix.pixelformat),
+		f->fmt.pix.width, f->fmt.pix.height,
+		f->fmt.pix.bytesperline, f->fmt.pix.sizeimage);
+
+	ret = vip_try_fmt_vid_cap(file, priv, f);
+	if (ret)
+		return ret;
+
+	vip_dbg(3, stream, "s_fmt try_fmt fourcc:%s size: %dx%d bpl:%d img_size:%d\n",
+		fourcc_to_str(f->fmt.pix.pixelformat),
+		f->fmt.pix.width, f->fmt.pix.height,
+		f->fmt.pix.bytesperline, f->fmt.pix.sizeimage);
+
+	if (vb2_is_busy(&stream->vb_vidq)) {
+		vip_err(stream, "%s queue busy\n", __func__);
+		return -EBUSY;
+	}
+
+	/*
+	 * Check if we need the scaler or not
+	 *
+	 * Since on previous S_FMT call the scaler might have been
+	 * allocated if it is not needed in this instance we will
+	 * attempt to free it just in case.
+	 *
+	 * free_scaler() is harmless unless the current port
+	 * allocated it.
+	 */
+	if (f->fmt.pix.width == port->try_mbus_framefmt.width &&
+	    f->fmt.pix.height == port->try_mbus_framefmt.height)
+		free_scaler(port);
+	else
+		allocate_scaler(port);
+
+	port->fmt = find_port_format_by_pix(port,
+					    f->fmt.pix.pixelformat);
+	stream->width		= f->fmt.pix.width;
+	stream->height		= f->fmt.pix.height;
+	stream->bytesperline	= f->fmt.pix.bytesperline;
+	stream->sizeimage	= f->fmt.pix.sizeimage;
+	stream->sup_field	= f->fmt.pix.field;
+	stream->field		= f->fmt.pix.field;
+
+	port->c_rect.left	= 0;
+	port->c_rect.top	= 0;
+	port->c_rect.width	= stream->width;
+	port->c_rect.height	= stream->height;
+
+	/*
+	 * Check if we need the csc unit or not
+	 *
+	 * Since on previous S_FMT call, the csc might have been
+	 * allocated if it is not needed in this instance we will
+	 * attempt to free it just in case.
+	 *
+	 * free_csc() is harmless unless the current port
+	 * allocated it.
+	 */
+	csc_direction =  vip_csc_direction(port->fmt->code, port->fmt->finfo);
+	if (csc_direction == VIP_CSC_NA)
+		free_csc(port);
+	else
+		allocate_csc(port, csc_direction);
+
+	if (stream->sup_field == V4L2_FIELD_ALTERNATE)
+		port->flags |= FLAG_INTERLACED;
+	else
+		port->flags &= ~FLAG_INTERLACED;
+
+	vip_dbg(3, stream, "s_fmt fourcc:%s size: %dx%d bpl:%d img_size:%d\n",
+		fourcc_to_str(f->fmt.pix.pixelformat),
+		f->fmt.pix.width, f->fmt.pix.height,
+		f->fmt.pix.bytesperline, f->fmt.pix.sizeimage);
+
+	mf = &sfmt.format;
+	v4l2_fill_mbus_format(mf, &f->fmt.pix, port->fmt->code);
+	/* Make sure to use the subdev size found in the try_fmt */
+	mf->width = port->try_mbus_framefmt.width;
+	mf->height = port->try_mbus_framefmt.height;
+
+	vip_dbg(3, stream, "s_fmt pix_to_mbus mbus_code: %04X size: %dx%d\n",
+		mf->code,
+		mf->width, mf->height);
+
+	sfmt.which = V4L2_SUBDEV_FORMAT_ACTIVE;
+	sfmt.pad = port->source_pad;
+	ret = v4l2_subdev_call(port->subdev, pad, set_fmt, NULL, &sfmt);
+	if (ret) {
+		vip_dbg(1, stream, "set_fmt failed in subdev\n");
+		return ret;
+	}
+
+	/* Save it */
+	port->mbus_framefmt = *mf;
+
+	vip_dbg(3, stream, "s_fmt subdev fmt mbus_code: %04X size: %dx%d\n",
+		port->mbus_framefmt.code,
+		port->mbus_framefmt.width, port->mbus_framefmt.height);
+	vip_dbg(3, stream, "s_fmt vpdma data type: 0x%02X\n",
+		port->fmt->vpdma_fmt[0]->data_type);
+
+	return 0;
+}
+
+/*
+ * Does the exact opposite of set_fmt_params
+ * It makes sure the DataPath register is sane after tear down
+ */
+static void unset_fmt_params(struct vip_stream *stream)
+{
+	struct vip_dev *dev = stream->port->dev;
+	struct vip_port *port = stream->port;
+
+	stream->sequence = 0;
+	if (stream->port->flags & FLAG_INTERLACED)
+		stream->field = V4L2_FIELD_TOP;
+
+	if (port->csc == VIP_CSC_Y2R) {
+		if (port->port_id == VIP_PORTA) {
+			vip_set_slice_path(dev, VIP_CSC_SRC_DATA_SELECT, 0);
+			vip_set_slice_path(dev,
+					   VIP_MULTI_CHANNEL_DATA_SELECT, 0);
+			vip_set_slice_path(dev, VIP_RGB_OUT_HI_DATA_SELECT, 0);
+			vip_set_slice_path(dev, VIP_RGB_SRC_DATA_SELECT, 0);
+		} else {
+			vip_set_slice_path(dev, VIP_CSC_SRC_DATA_SELECT, 0);
+			vip_set_slice_path(dev,
+					   VIP_MULTI_CHANNEL_DATA_SELECT, 0);
+			vip_set_slice_path(dev, VIP_RGB_OUT_LO_DATA_SELECT, 0);
+		}
+		/* We are done */
+		return;
+	} else if (port->csc == VIP_CSC_R2Y) {
+		if (port->scaler && port->fmt->coplanar) {
+			if (port->port_id == VIP_PORTA) {
+				vip_set_slice_path(dev,
+						   VIP_CSC_SRC_DATA_SELECT, 0);
+				vip_set_slice_path(dev,
+						   VIP_SC_SRC_DATA_SELECT, 0);
+				vip_set_slice_path(dev,
+						   VIP_CHR_DS_1_SRC_DATA_SELECT,
+						   0);
+				vip_set_slice_path(dev,
+						   VIP_CHR_DS_1_DATA_BYPASS, 0);
+				vip_set_slice_path(dev,
+						   VIP_RGB_OUT_HI_DATA_SELECT,
+						   0);
+			}
+		} else if (port->scaler) {
+			if (port->port_id == VIP_PORTA) {
+				vip_set_slice_path(dev,
+						   VIP_CSC_SRC_DATA_SELECT, 0);
+				vip_set_slice_path(dev,
+						   VIP_SC_SRC_DATA_SELECT, 0);
+				vip_set_slice_path(dev,
+						   VIP_CHR_DS_1_SRC_DATA_SELECT,
+						   0);
+				vip_set_slice_path(dev,
+						   VIP_CHR_DS_1_DATA_BYPASS, 0);
+				vip_set_slice_path(dev,
+						   VIP_RGB_OUT_HI_DATA_SELECT,
+						   0);
+			}
+		} else if (port->fmt->coplanar) {
+			if (port->port_id == VIP_PORTA) {
+				vip_set_slice_path(dev,
+						   VIP_CSC_SRC_DATA_SELECT, 0);
+				vip_set_slice_path(dev,
+						   VIP_CHR_DS_1_SRC_DATA_SELECT,
+						   0);
+				vip_set_slice_path(dev,
+						   VIP_CHR_DS_1_DATA_BYPASS, 0);
+				vip_set_slice_path(dev,
+						   VIP_RGB_OUT_HI_DATA_SELECT,
+						   0);
+			}
+		} else {
+			if (port->port_id == VIP_PORTA) {
+				vip_set_slice_path(dev,
+						   VIP_CSC_SRC_DATA_SELECT, 0);
+				vip_set_slice_path(dev,
+						   VIP_CHR_DS_1_SRC_DATA_SELECT,
+						   0);
+				vip_set_slice_path(dev,
+						   VIP_CHR_DS_1_DATA_BYPASS, 0);
+				vip_set_slice_path(dev,
+						   VIP_RGB_OUT_HI_DATA_SELECT,
+						   0);
+			}
+		}
+		/* We are done */
+		return;
+	} else if (v4l2_is_format_rgb(port->fmt->finfo)) {
+		if (port->port_id == VIP_PORTA) {
+			vip_set_slice_path(dev,
+					   VIP_MULTI_CHANNEL_DATA_SELECT, 0);
+			vip_set_slice_path(dev, VIP_RGB_OUT_LO_DATA_SELECT, 0);
+		}
+		/* We are done */
+		return;
+	}
+
+	if (port->scaler && port->fmt->coplanar) {
+		if (port->port_id == VIP_PORTA) {
+			vip_set_slice_path(dev, VIP_SC_SRC_DATA_SELECT, 0);
+			vip_set_slice_path(dev,
+					   VIP_CHR_DS_1_SRC_DATA_SELECT, 0);
+			vip_set_slice_path(dev, VIP_CHR_DS_1_DATA_BYPASS, 0);
+			vip_set_slice_path(dev, VIP_RGB_OUT_HI_DATA_SELECT, 0);
+		} else {
+			vip_set_slice_path(dev, VIP_SC_SRC_DATA_SELECT, 0);
+			vip_set_slice_path(dev,
+					   VIP_CHR_DS_2_SRC_DATA_SELECT, 0);
+			vip_set_slice_path(dev, VIP_CHR_DS_1_DATA_BYPASS, 0);
+			vip_set_slice_path(dev, VIP_RGB_OUT_LO_DATA_SELECT, 0);
+			vip_set_slice_path(dev,
+					   VIP_MULTI_CHANNEL_DATA_SELECT, 0);
+		}
+	} else if (port->scaler) {
+		if (port->port_id == VIP_PORTA) {
+			vip_set_slice_path(dev, VIP_SC_SRC_DATA_SELECT, 0);
+			vip_set_slice_path(dev,
+					   VIP_CHR_DS_1_SRC_DATA_SELECT, 0);
+			vip_set_slice_path(dev, VIP_CHR_DS_1_DATA_BYPASS, 0);
+			vip_set_slice_path(dev, VIP_RGB_OUT_HI_DATA_SELECT, 0);
+		} else {
+			vip_set_slice_path(dev, VIP_SC_SRC_DATA_SELECT, 0);
+			vip_set_slice_path(dev,
+					   VIP_CHR_DS_2_SRC_DATA_SELECT, 0);
+			vip_set_slice_path(dev, VIP_CHR_DS_1_DATA_BYPASS, 0);
+			vip_set_slice_path(dev, VIP_CHR_DS_2_DATA_BYPASS, 0);
+			vip_set_slice_path(dev, VIP_RGB_OUT_HI_DATA_SELECT, 0);
+		}
+	} else if (port->fmt->coplanar) {
+		if (port->port_id == VIP_PORTA) {
+			vip_set_slice_path(dev,
+					   VIP_CHR_DS_1_SRC_DATA_SELECT, 0);
+			vip_set_slice_path(dev, VIP_CHR_DS_1_DATA_BYPASS, 0);
+			vip_set_slice_path(dev, VIP_RGB_OUT_HI_DATA_SELECT, 0);
+		} else {
+			vip_set_slice_path(dev,
+					   VIP_CHR_DS_2_SRC_DATA_SELECT, 0);
+			vip_set_slice_path(dev, VIP_CHR_DS_2_DATA_BYPASS, 0);
+			vip_set_slice_path(dev,
+					   VIP_MULTI_CHANNEL_DATA_SELECT, 0);
+			vip_set_slice_path(dev, VIP_RGB_OUT_LO_DATA_SELECT, 0);
+		}
+	} else {
+		/*
+		 * We undo all data path setting except for the multi
+		 * stream case.
+		 * Because we cannot disrupt other on-going capture if only
+		 * one stream is terminated the other might still be going
+		 */
+		vip_set_slice_path(dev, VIP_MULTI_CHANNEL_DATA_SELECT, 1);
+		vip_set_slice_path(dev, VIP_RGB_OUT_LO_DATA_SELECT, 0);
+	}
+}
+
Quoted 8 lines

drivers/media/platform/ti-vpe/vip.c line 2096

Quoted 5 lines
drivers/media/platform/ti-vpe/vip.c new file +5 -0
+/*
+ * Set the registers that are modified when the video format changes.
+ */
+static void set_fmt_params(struct vip_stream *stream)
+{

Hmm, this is a *very* long function. Perhaps this could be split up a bit, or reorganized?

Yeah, I'll start by removing the extra comment lines and reformat it.

Quoted 19 lines
Quoted 19 lines
drivers/media/platform/ti-vpe/vip.c new file +18 -0
+	struct vip_dev *dev = stream->port->dev;
+	struct vip_port *port = stream->port;
+
+	stream->sequence = 0;
+	if (stream->port->flags & FLAG_INTERLACED)
+		stream->field = V4L2_FIELD_TOP;
+
+	if (port->csc == VIP_CSC_Y2R) {
+		port->flags &= ~FLAG_MULT_PORT;
+		/* Set alpha component in background color */
+		vpdma_set_bg_color(dev->shared->vpdma,
+				   (struct vpdma_data_format *)
+				   port->fmt->vpdma_fmt[0],
+				   0xff);
+		if (port->port_id == VIP_PORTA) {
+			/*
+			 * Input A: YUV422
+			 * Output: Y_UP/UV_UP: RGB
Quoted 8 lines

drivers/media/platform/ti-vpe/vip.c line 2118

Quoted 4 lines
drivers/media/platform/ti-vpe/vip.c new file +4 -0
+			 * CSC_SRC_SELECT       = 1
+			 * RGB_OUT_HI_SELECT    = 1
+			 * RGB_SRC_SELECT       = 1
+			 * MULTI_CHANNEL_SELECT = 0

It's a bit pointless to comment what the register values should be when you set them in the code below. I'd drop that part, it will make the code shorter.

Ok.

Quoted 8 lines

drivers/media/platform/ti-vpe/vip.c line 2122

Quoted 5 lines
drivers/media/platform/ti-vpe/vip.c new file +4 -0
+			 */
+			vip_set_slice_path(dev, VIP_CSC_SRC_DATA_SELECT, 1);
+			vip_set_slice_path(dev,
+					   VIP_MULTI_CHANNEL_DATA_SELECT, 0);

For readability purposes I think it is better to keep this on one line. Same for the other vip_set_slice_path calls.

Ok.

Quoted 288 lines
Quoted 288 lines
drivers/media/platform/ti-vpe/vip.c new file +287 -0
+			vip_set_slice_path(dev, VIP_RGB_OUT_HI_DATA_SELECT, 1);
+			vip_set_slice_path(dev, VIP_RGB_SRC_DATA_SELECT, 1);
+		} else {
+			/*
+			 * Input B: YUV422
+			 * Output: Y_UP/UV_UP: RGB
+			 * CSC_SRC_SELECT       = 2
+			 * RGB_OUT_LO_SELECT    = 1
+			 * MULTI_CHANNEL_SELECT = 0
+			 */
+			vip_set_slice_path(dev, VIP_CSC_SRC_DATA_SELECT, 2);
+			vip_set_slice_path(dev,
+					   VIP_MULTI_CHANNEL_DATA_SELECT, 0);
+			vip_set_slice_path(dev, VIP_RGB_OUT_LO_DATA_SELECT, 1);
+		}
+		/* We are done */
+		return;
+	} else if (port->csc == VIP_CSC_R2Y) {
+		port->flags &= ~FLAG_MULT_PORT;
+		if (port->scaler && port->fmt->coplanar) {
+			if (port->port_id == VIP_PORTA) {
+				/*
+				 * Input A: RGB
+				 * Output: Y_UP/UV_UP: Scaled YUV420
+				 * CSC_SRC_SELECT       = 4
+				 * SC_SRC_SELECT        = 1
+				 * CHR_DS_1_SRC_SELECT  = 1
+				 * CHR_DS_1_BYPASS      = 0
+				 * RGB_OUT_HI_SELECT    = 0
+				 */
+				vip_set_slice_path(dev,
+						   VIP_CSC_SRC_DATA_SELECT, 4);
+				vip_set_slice_path(dev,
+						   VIP_SC_SRC_DATA_SELECT, 1);
+				vip_set_slice_path(dev,
+						   VIP_CHR_DS_1_SRC_DATA_SELECT,
+						   1);
+				vip_set_slice_path(dev,
+						   VIP_CHR_DS_1_DATA_BYPASS, 0);
+				vip_set_slice_path(dev,
+						   VIP_RGB_OUT_HI_DATA_SELECT,
+						   0);
+			} else {
+				vip_err(stream, "RGB sensor can only be on Port A\n");
+			}
+		} else if (port->scaler) {
+			if (port->port_id == VIP_PORTA) {
+				/*
+				 * Input A: RGB
+				 * Output: Y_UP: Scaled YUV422
+				 * CSC_SRC_SELECT       = 4
+				 * SC_SRC_SELECT        = 1
+				 * CHR_DS_1_SRC_SELECT  = 1
+				 * CHR_DS_1_BYPASS      = 1
+				 * RGB_OUT_HI_SELECT    = 0
+				 */
+				vip_set_slice_path(dev,
+						   VIP_CSC_SRC_DATA_SELECT, 4);
+				vip_set_slice_path(dev,
+						   VIP_SC_SRC_DATA_SELECT, 1);
+				vip_set_slice_path(dev,
+						   VIP_CHR_DS_1_SRC_DATA_SELECT,
+						   1);
+				vip_set_slice_path(dev,
+						   VIP_CHR_DS_1_DATA_BYPASS, 1);
+				vip_set_slice_path(dev,
+						   VIP_RGB_OUT_HI_DATA_SELECT,
+						   0);
+			} else {
+				vip_err(stream, "RGB sensor can only be on Port A\n");
+			}
+		} else if (port->fmt->coplanar) {
+			if (port->port_id == VIP_PORTA) {
+				/*
+				 * Input A: RGB
+				 * Output: Y_UP/UV_UP: YUV420
+				 * CSC_SRC_SELECT       = 4
+				 * CHR_DS_1_SRC_SELECT  = 2
+				 * CHR_DS_1_BYPASS      = 0
+				 * RGB_OUT_HI_SELECT    = 0
+				 */
+				vip_set_slice_path(dev,
+						   VIP_CSC_SRC_DATA_SELECT, 4);
+				vip_set_slice_path(dev,
+						   VIP_CHR_DS_1_SRC_DATA_SELECT,
+						   2);
+				vip_set_slice_path(dev,
+						   VIP_CHR_DS_1_DATA_BYPASS, 0);
+				vip_set_slice_path(dev,
+						   VIP_RGB_OUT_HI_DATA_SELECT,
+						   0);
+			} else {
+				vip_err(stream, "RGB sensor can only be on Port A\n");
+			}
+		} else {
+			if (port->port_id == VIP_PORTA) {
+				/*
+				 * Input A: RGB
+				 * Output: Y_UP/UV_UP: YUV420
+				 * CSC_SRC_SELECT       = 4
+				 * CHR_DS_1_SRC_SELECT  = 2
+				 * CHR_DS_1_BYPASS      = 1
+				 * RGB_OUT_HI_SELECT    = 0
+				 */
+				vip_set_slice_path(dev,
+						   VIP_CSC_SRC_DATA_SELECT, 4);
+				vip_set_slice_path(dev,
+						   VIP_CHR_DS_1_SRC_DATA_SELECT,
+						   2);
+				vip_set_slice_path(dev,
+						   VIP_CHR_DS_1_DATA_BYPASS, 1);
+				vip_set_slice_path(dev,
+						   VIP_RGB_OUT_HI_DATA_SELECT,
+						   0);
+			} else {
+				vip_err(stream, "RGB sensor can only be on Port A\n");
+			}
+		}
+		/* We are done */
+		return;
+	} else if (v4l2_is_format_rgb(port->fmt->finfo)) {
+		port->flags &= ~FLAG_MULT_PORT;
+		/* Set alpha component in background color */
+		vpdma_set_bg_color(dev->shared->vpdma,
+				   (struct vpdma_data_format *)
+				   port->fmt->vpdma_fmt[0],
+				   0xff);
+		if (port->port_id == VIP_PORTA) {
+			/*
+			 * Input A: RGB
+			 * Output: Y_LO/UV_LO: RGB
+			 * RGB_OUT_LO_SELECT    = 1
+			 * MULTI_CHANNEL_SELECT = 1
+			 */
+			vip_set_slice_path(dev,
+					   VIP_MULTI_CHANNEL_DATA_SELECT, 1);
+			vip_set_slice_path(dev, VIP_RGB_OUT_LO_DATA_SELECT, 1);
+		} else {
+			vip_err(stream, "RGB sensor can only be on Port A\n");
+		}
+		/* We are done */
+		return;
+	}
+
+	if (port->scaler && port->fmt->coplanar) {
+		port->flags &= ~FLAG_MULT_PORT;
+		if (port->port_id == VIP_PORTA) {
+			/*
+			 * Input A: YUV422
+			 * Output: Y_UP/UV_UP: Scaled YUV420
+			 * SC_SRC_SELECT        = 2
+			 * CHR_DS_1_SRC_SELECT  = 1
+			 * CHR_DS_1_BYPASS      = 0
+			 * RGB_OUT_HI_SELECT    = 0
+			 */
+			vip_set_slice_path(dev, VIP_SC_SRC_DATA_SELECT, 2);
+			vip_set_slice_path(dev,
+					   VIP_CHR_DS_1_SRC_DATA_SELECT, 1);
+			vip_set_slice_path(dev, VIP_CHR_DS_1_DATA_BYPASS, 0);
+			vip_set_slice_path(dev, VIP_RGB_OUT_HI_DATA_SELECT, 0);
+		} else {
+			/*
+			 * Input B: YUV422
+			 * Output: Y_LO/UV_LO: Scaled YUV420
+			 * SC_SRC_SELECT        = 3
+			 * CHR_DS_2_SRC_SELECT  = 1
+			 * RGB_OUT_LO_SELECT    = 0
+			 * MULTI_CHANNEL_SELECT = 0
+			 */
+			vip_set_slice_path(dev, VIP_SC_SRC_DATA_SELECT, 3);
+			vip_set_slice_path(dev,
+					   VIP_CHR_DS_2_SRC_DATA_SELECT, 1);
+			vip_set_slice_path(dev, VIP_CHR_DS_1_DATA_BYPASS, 0);
+			vip_set_slice_path(dev, VIP_RGB_OUT_LO_DATA_SELECT, 0);
+			vip_set_slice_path(dev,
+					   VIP_MULTI_CHANNEL_DATA_SELECT, 0);
+		}
+	} else if (port->scaler) {
+		port->flags &= ~FLAG_MULT_PORT;
+		if (port->port_id == VIP_PORTA) {
+			/*
+			 * Input A: YUV422
+			 * Output: Y_UP: Scaled YUV422
+			 * SC_SRC_SELECT        = 2
+			 * CHR_DS_1_SRC_SELECT  = 1
+			 * CHR_DS_1_BYPASS      = 1
+			 * RGB_OUT_HI_SELECT    = 0
+			 */
+			vip_set_slice_path(dev, VIP_SC_SRC_DATA_SELECT, 2);
+			vip_set_slice_path(dev,
+					   VIP_CHR_DS_1_SRC_DATA_SELECT, 1);
+			vip_set_slice_path(dev, VIP_CHR_DS_1_DATA_BYPASS, 1);
+			vip_set_slice_path(dev, VIP_RGB_OUT_HI_DATA_SELECT, 0);
+		} else {
+			/*
+			 * Input B: YUV422
+			 * Output: UV_UP: Scaled YUV422
+			 * SC_SRC_SELECT        = 3
+			 * CHR_DS_2_SRC_SELECT  = 1
+			 * CHR_DS_1_BYPASS      = 1
+			 * CHR_DS_2_BYPASS      = 1
+			 * RGB_OUT_HI_SELECT    = 0
+			 */
+			vip_set_slice_path(dev, VIP_SC_SRC_DATA_SELECT, 3);
+			vip_set_slice_path(dev,
+					   VIP_CHR_DS_2_SRC_DATA_SELECT, 1);
+			vip_set_slice_path(dev, VIP_CHR_DS_1_DATA_BYPASS, 1);
+			vip_set_slice_path(dev, VIP_CHR_DS_2_DATA_BYPASS, 1);
+			vip_set_slice_path(dev, VIP_RGB_OUT_HI_DATA_SELECT, 0);
+		}
+	} else if (port->fmt->coplanar) {
+		port->flags &= ~FLAG_MULT_PORT;
+		if (port->port_id == VIP_PORTA) {
+			/*
+			 * Input A: YUV422
+			 * Output: Y_UP/UV_UP: YUV420
+			 * CHR_DS_1_SRC_SELECT  = 3
+			 * CHR_DS_1_BYPASS      = 0
+			 * RGB_OUT_HI_SELECT    = 0
+			 */
+			vip_set_slice_path(dev,
+					   VIP_CHR_DS_1_SRC_DATA_SELECT, 3);
+			vip_set_slice_path(dev, VIP_CHR_DS_1_DATA_BYPASS, 0);
+			vip_set_slice_path(dev, VIP_RGB_OUT_HI_DATA_SELECT, 0);
+		} else {
+			/*
+			 * Input B: YUV422
+			 * Output: Y_LO/UV_LO: YUV420
+			 * CHR_DS_2_SRC_SELECT  = 4
+			 * CHR_DS_2_BYPASS      = 0
+			 * RGB_OUT_LO_SELECT    = 0
+			 * MULTI_CHANNEL_SELECT = 0
+			 */
+			vip_set_slice_path(dev,
+					   VIP_CHR_DS_2_SRC_DATA_SELECT, 4);
+			vip_set_slice_path(dev, VIP_CHR_DS_2_DATA_BYPASS, 0);
+			vip_set_slice_path(dev,
+					   VIP_MULTI_CHANNEL_DATA_SELECT, 0);
+			vip_set_slice_path(dev, VIP_RGB_OUT_LO_DATA_SELECT, 0);
+		}
+	} else {
+		port->flags |= FLAG_MULT_PORT;
+		/*
+		 * Input A/B: YUV422
+		 * Output: Y_LO: YUV422 - UV_LO: YUV422
+		 * MULTI_CHANNEL_SELECT = 1
+		 * RGB_OUT_LO_SELECT    = 0
+		 */
+		vip_set_slice_path(dev, VIP_MULTI_CHANNEL_DATA_SELECT, 1);
+		vip_set_slice_path(dev, VIP_RGB_OUT_LO_DATA_SELECT, 0);
+	}
+}
+
+static int vip_g_selection(struct file *file, void *fh,
+			   struct v4l2_selection *s)
+{
+	struct vip_stream *stream = file2stream(file);
+
+	if (s->type != V4L2_BUF_TYPE_VIDEO_CAPTURE)
+		return -EINVAL;
+
+	switch (s->target) {
+	case V4L2_SEL_TGT_COMPOSE_DEFAULT:
+	case V4L2_SEL_TGT_COMPOSE_BOUNDS:
+	case V4L2_SEL_TGT_CROP_BOUNDS:
+	case V4L2_SEL_TGT_CROP_DEFAULT:
+		s->r.left = 0;
+		s->r.top = 0;
+		s->r.width = stream->width;
+		s->r.height = stream->height;
+		return 0;
+
+	case V4L2_SEL_TGT_COMPOSE:
+	case V4L2_SEL_TGT_CROP:
+		s->r = stream->port->c_rect;
+		return 0;
+	}
+
+	return -EINVAL;
+}
+
+static int enclosed_rectangle(struct v4l2_rect *a, struct v4l2_rect *b)
+{
+	if (a->left < b->left || a->top < b->top)
+		return 0;
+	if (a->left + a->width > b->left + b->width)
+		return 0;
Quoted 8 lines

drivers/media/platform/ti-vpe/vip.c line 2414

Quoted 5 lines
drivers/media/platform/ti-vpe/vip.c new file +5 -0
+	if (a->top + a->height > b->top + b->height)
+		return 0;
+
+	return 1;
+}

There are helper functions in include/media/v4l2-rect.h, it would make sense to add this one to that header.

I'll check that out.

Quoted 16 lines
Quoted 4 lines
drivers/media/platform/ti-vpe/vip.c new file +3 -0
+
+static int vip_s_selection(struct file *file, void *fh,
+			   struct v4l2_selection *s)

drivers/media/platform/ti-vpe/vip.c line 2427

Quoted 10 lines
drivers/media/platform/ti-vpe/vip.c new file +10 -0
+{
+	struct vip_stream *stream = file2stream(file);
+	struct v4l2_rect r = s->r;
+
+	if (s->type != V4L2_BUF_TYPE_VIDEO_CAPTURE)
+		return -EINVAL;
+
+	switch (s->target) {
+	case V4L2_SEL_TGT_COMPOSE:
+	case V4L2_SEL_TGT_CROP:

Why both crop and compose when it is the same c_rect? That makes no sense.

Quoted 2 lines

Yeah, this is always puzzling to me. When to use which and what not. I'll catch you on IRC sometime to chat about this.

For video capture devices cropping occurs before the DMA step, usually in the sensor. Composing affects the DMA engine: it composes the (possibly cropped) video frame into a buffer. E.g. the buffer might be sized for 1920x1080, but the video is 1280x720 and you want to have it DMAed to the center of the 1920x1080-sized buffer.

This requires that the DMA engine supports this, which is uncommon for video capture drivers. So typically video capture drivers just support cropping.

Quoted 25 lines
Quoted 25 lines
Quoted 25 lines
drivers/media/platform/ti-vpe/vip.c new file +23 -0
+		v4l_bound_align_image(&r.width, 0, stream->width, 0,
+				      &r.height, 0, stream->height, 0, 0);
+
+		r.left = clamp_t(unsigned int, r.left, 0,
+				 stream->width - r.width);
+		r.top  = clamp_t(unsigned int, r.top, 0,
+				 stream->height - r.height);
+
+		if (s->flags & V4L2_SEL_FLAG_LE &&
+		    !enclosed_rectangle(&r, &s->r))
+			return -ERANGE;
+
+		if (s->flags & V4L2_SEL_FLAG_GE &&
+		    !enclosed_rectangle(&s->r, &r))
+			return -ERANGE;
+
+		s->r = r;
+		stream->port->c_rect = r;
+
+		vip_dbg(1, stream, "cropped (%d,%d)/%dx%d of %dx%d\n",
+			r.left, r.top, r.width, r.height,
+			stream->width, stream->height);
+
Quoted 8 lines
Quoted 8 lines

drivers/media/platform/ti-vpe/vip.c line 2458

Quoted 8 lines
drivers/media/platform/ti-vpe/vip.c new file +8 -0
+			s->r = stream->port->c_rect;
+		return 0;
+	default:
+		return -EINVAL;
+	}
+
+	return 0;
+}

Regards,

Hans