Allwinner A31/A83T MIPI CSI-2 and A31 ISP / ISP Driver
Subsystems (6)
v7 cover Allwinner A31/A83T MIPI CSI-2 and A31 ISP / ISP Driver
This part only concerns the introduction of the new ISP driver and related adaptation of the CSI driver.
Most non-dt patches still need reviewing but should be pretty straightforward. Since this multi-part series has been going on for a while, it would be great to see it merged soon!
Changes since v6:
- Added a per-compatible check for the required port in dt binding;
- Reworded ISP output port description in dt binding;
- Reversed ISP detection order to have fwnode first;
- Removed info print when ISP link is detected;
- Added warn print when ISP is linked but not enabled in config;
- Fixed sun6i_csi_isp_detect return type;
- Removed useless initialization in sun6i_csi_isp_detect;
- Fixed typo in sun6i_csi_isp_detect;
- Added collected tags;
Changes since v5:
- Rebased on latest media tree;
- Added collected tag;
- Switched to using media_pad_remote_pad_first;
- Switched to using media_pad_remote_pad_unique.
Changes since v4:
- Fixed device-tree binding indent-align;
- Added collected tag;
- Rebased on latest media tree;
Changes since v3:
- Removed the v4l2 controls handler from the driver;
- Added variant structure for table sizes;
- Removed the info message about video device registration;
- Removed comments in uAPI header;
- Used '/schemas/graph.yaml#/properties/port' whenever possible in bindings;
- Added CSI patches dependent on the ISP driver;
- Rebased on the latest media tree;
Changes since all-in-one v2:
- Updated Kconfig to follow the latest media-wide changes;
- Reworked async subdev handling with a dedicated structure holding the corresponding source to avoid matching in the driver;
- Switched to clock-managed regmap mmio;
- Used helper to get a single enabled link for an entity's pad, to replace source selection at link_validate time and select the remote source at stream on time instead;
- Added mutex for mbus format serialization;
- Used endpoint-base instead of video-interface for "internal" endpoints in device-tree schema;
- Added TODO with unstaging requirements;
- Various cosmetic cleanups;
- Updated copyright years;
Paul Kocialkowski (6):
dt-bindings: media: Add Allwinner A31 ISP bindings documentation
dt-bindings: media: sun6i-a31-csi: Add internal output port to the ISP
staging: media: Add support for the Allwinner A31 ISP
MAINTAINERS: Add entry for the Allwinner A31 ISP driver
media: sun6i-csi: Detect the availability of the ISP
media: sun6i-csi: Add support for hooking to the isp devices
.../media/allwinner,sun6i-a31-csi.yaml | 4 + .../media/allwinner,sun6i-a31-isp.yaml | 101 +++ MAINTAINERS | 9 + .../platform/sunxi/sun6i-csi/sun6i_csi.c | 75 +- .../platform/sunxi/sun6i-csi/sun6i_csi.h | 10 + .../sunxi/sun6i-csi/sun6i_csi_bridge.c | 32 +- .../sunxi/sun6i-csi/sun6i_csi_capture.c | 19 +- .../sunxi/sun6i-csi/sun6i_csi_capture.h | 1 + drivers/staging/media/sunxi/Kconfig | 1 + drivers/staging/media/sunxi/Makefile | 1 + drivers/staging/media/sunxi/sun6i-isp/Kconfig | 15 + .../staging/media/sunxi/sun6i-isp/Makefile | 4 + .../staging/media/sunxi/sun6i-isp/TODO.txt | 6 + .../staging/media/sunxi/sun6i-isp/sun6i_isp.c | 555 +++++++++++++ .../staging/media/sunxi/sun6i-isp/sun6i_isp.h | 90 +++ .../media/sunxi/sun6i-isp/sun6i_isp_capture.c | 742 ++++++++++++++++++ .../media/sunxi/sun6i-isp/sun6i_isp_capture.h | 78 ++ .../media/sunxi/sun6i-isp/sun6i_isp_params.c | 566 +++++++++++++ .../media/sunxi/sun6i-isp/sun6i_isp_params.h | 52 ++ .../media/sunxi/sun6i-isp/sun6i_isp_proc.c | 577 ++++++++++++++ .../media/sunxi/sun6i-isp/sun6i_isp_proc.h | 66 ++ .../media/sunxi/sun6i-isp/sun6i_isp_reg.h | 275 +++++++ .../sunxi/sun6i-isp/uapi/sun6i-isp-config.h | 43 + 23 files changed, 3309 insertions(+), 13 deletions(-) create mode 100644 Documentation/devicetree/bindings/media/allwinner,sun6i-a31-isp.yaml create mode 100644 drivers/staging/media/sunxi/sun6i-isp/Kconfig create mode 100644 drivers/staging/media/sunxi/sun6i-isp/Makefile create mode 100644 drivers/staging/media/sunxi/sun6i-isp/TODO.txt create mode 100644 drivers/staging/media/sunxi/sun6i-isp/sun6i_isp.c create mode 100644 drivers/staging/media/sunxi/sun6i-isp/sun6i_isp.h create mode 100644 drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_capture.c create mode 100644 drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_capture.h create mode 100644 drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_params.c create mode 100644 drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_params.h create mode 100644 drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_proc.c create mode 100644 drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_proc.h create mode 100644 drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_reg.h create mode 100644 drivers/staging/media/sunxi/sun6i-isp/uapi/sun6i-isp-config.h
Signature · 1 line
2.37.3
PATCH v7 1/6 dt-bindings: media: Add Allwinner A31 ISP bindings documentation
This introduces YAML bindings documentation for the Allwinner A31 Image Signal Processor (ISP).
--- .../media/allwinner,sun6i-a31-isp.yaml | 101 ++++++++++++++++++ 1 file changed, 101 insertions(+) create mode 100644 Documentation/devicetree/bindings/media/allwinner,sun6i-a31-isp.yaml
Documentation/devicetree/bindings/media/allwinner,sun6i-a31-isp.yaml new file +101 -0
@@ -0,0 +1,101 @@ +# SPDX-License-Identifier: (GPL-2.0 OR BSD-2-Clause) +%YAML 1.2 +--- +$id: http://devicetree.org/schemas/media/allwinner,sun6i-a31-isp.yaml# +$schema: http://devicetree.org/meta-schemas/core.yaml# + +title: Allwinner A31 Image Signal Processor Driver (ISP) Device Tree Bindings + +maintainers: + - Paul Kocialkowski <[email protected]> + +properties: + compatible: + enum: + - allwinner,sun6i-a31-isp + - allwinner,sun8i-v3s-isp + + reg: + maxItems: 1 + + interrupts: + maxItems: 1 + + clocks: + items: + - description: Bus Clock + - description: Module Clock + - description: DRAM Clock + + clock-names: + items: + - const: bus + - const: mod + - const: ram + + resets: + maxItems: 1 + + ports: + $ref: /schemas/graph.yaml#/properties/ports + + properties: + port@0: + $ref: /schemas/graph.yaml#/properties/port + description: CSI0 input port + + port@1: + $ref: /schemas/graph.yaml#/properties/port + description: CSI1 input port + + if: + properties: + compatible: + contains: + enum: + - allwinner,sun8i-v3s-isp + then: + required: + - port@0 + +required: + - compatible + - reg + - interrupts + - clocks + - clock-names + - resets + +additionalProperties: false + +examples: + - | + #include <dt-bindings/interrupt-controller/arm-gic.h> + #include <dt-bindings/clock/sun8i-v3s-ccu.h> + #include <dt-bindings/reset/sun8i-v3s-ccu.h> + + isp: isp@1cb8000 { + compatible = "allwinner,sun8i-v3s-isp"; + reg = <0x01cb8000 0x1000>; + interrupts = <GIC_SPI 83 IRQ_TYPE_LEVEL_HIGH>; + clocks = <&ccu CLK_BUS_CSI>, + <&ccu CLK_CSI1_SCLK>, + <&ccu CLK_DRAM_CSI>; + clock-names = "bus", "mod", "ram"; + resets = <&ccu RST_BUS_CSI>; + + ports { + #address-cells = <1>; + #size-cells = <0>; + + port@0 { + reg = <0>; + + isp_in_csi0: endpoint { + remote-endpoint = <&csi0_out_isp>; + }; + }; + }; + }; + +...
Signature · 1 line
2.37.3
PATCH v7 2/6 dt-bindings: media: sun6i-a31-csi: Add internal output port to the ISP
In reply to Paul KocialkowskiSome Allwinner devices come with an Image Signal Processor (ISP) that allows processing camera data to produce good-looking images, especially from raw bayer representations.
The ISP does not have a dedicated capture path: it is fed directly by one of the CSI controllers, which can be selected at run-time.
Represent this possibility as a graph connection between the CSI controller and the ISP in the device-tree bindings.
--- .../devicetree/bindings/media/allwinner,sun6i-a31-csi.yaml | 4 ++++ 1 file changed, 4 insertions(+)
Documentation/devicetree/bindings/media/allwinner,sun6i-a31-csi.yaml +4 -0
@@ -73,6 +73,10 @@ properties: $ref: /schemas/graph.yaml#/properties/port description: MIPI CSI-2 bridge input port + port@2: + $ref: /schemas/graph.yaml#/properties/port + description: Internal output port to the ISP + anyOf: - required: - port@0
Signature · 1 line
2.37.3
PATCH v7 3/6 staging: media: Add support for the Allwinner A31 ISP
In reply to Paul KocialkowskiSome Allwinner platforms come with an Image Signal Processor, which supports various features in order to enhance and transform data received by image sensors into good-looking pictures. In most cases, the data is raw bayer, which gets internally converted to RGB and finally YUV, which is what the hardware produces.
This driver supports ISPs that are similar to the A31 ISP, which was the first standalone ISP found in Allwinner platforms. Simpler ISP blocks were found in the A10 and A20, where they are tied to a CSI controller. Newer generations of Allwinner SoCs (starting with the H6, H616, etc) come with a new camera subsystem and revised ISP. Even though these previous and next-generation ISPs are somewhat similar to the A31 ISP, they have enough significant differences to be out of the scope of this driver.
While the ISP supports many features, including 3A and many enhancement blocks, this implementation is limited to the following:
- V3s (V3/S3) platform support;
- Bayer media bus formats as input;
- Semi-planar YUV (NV12/NV21) as output;
- Debayering with per-component gain and offset configuration;
- 2D noise filtering with configurable coefficients.
Since many features are missing from the associated uAPI, the driver is aimed to integrate staging until all features are properly described.
On the technical side, it uses the v4l2 and media controller APIs, with a video node for capture, a processor subdev and a video node for parameters submission. A specific uAPI structure and associated v4l2 meta format are used to configure parameters of the supported modules.
One particular thing about the hardware is that configuration for module registers needs to be stored in a DMA buffer and gets copied to actual registers by the hardware at the next vsync, when instructed by a flag. This is handled by the "state" mechanism in the driver.
--- drivers/staging/media/sunxi/Kconfig | 1 + drivers/staging/media/sunxi/Makefile | 1 + drivers/staging/media/sunxi/sun6i-isp/Kconfig | 15 + .../staging/media/sunxi/sun6i-isp/Makefile | 4 + .../staging/media/sunxi/sun6i-isp/TODO.txt | 6 + .../staging/media/sunxi/sun6i-isp/sun6i_isp.c | 555 +++++++++++++ .../staging/media/sunxi/sun6i-isp/sun6i_isp.h | 90 +++ .../media/sunxi/sun6i-isp/sun6i_isp_capture.c | 742 ++++++++++++++++++ .../media/sunxi/sun6i-isp/sun6i_isp_capture.h | 78 ++ .../media/sunxi/sun6i-isp/sun6i_isp_params.c | 566 +++++++++++++ .../media/sunxi/sun6i-isp/sun6i_isp_params.h | 52 ++ .../media/sunxi/sun6i-isp/sun6i_isp_proc.c | 577 ++++++++++++++ .../media/sunxi/sun6i-isp/sun6i_isp_proc.h | 66 ++ .../media/sunxi/sun6i-isp/sun6i_isp_reg.h | 275 +++++++ .../sunxi/sun6i-isp/uapi/sun6i-isp-config.h | 43 + 15 files changed, 3071 insertions(+) create mode 100644 drivers/staging/media/sunxi/sun6i-isp/Kconfig create mode 100644 drivers/staging/media/sunxi/sun6i-isp/Makefile create mode 100644 drivers/staging/media/sunxi/sun6i-isp/TODO.txt create mode 100644 drivers/staging/media/sunxi/sun6i-isp/sun6i_isp.c create mode 100644 drivers/staging/media/sunxi/sun6i-isp/sun6i_isp.h create mode 100644 drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_capture.c create mode 100644 drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_capture.h create mode 100644 drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_params.c create mode 100644 drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_params.h create mode 100644 drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_proc.c create mode 100644 drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_proc.h create mode 100644 drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_reg.h create mode 100644 drivers/staging/media/sunxi/sun6i-isp/uapi/sun6i-isp-config.h
drivers/staging/media/sunxi/Kconfig +1 -0
@@ -12,5 +12,6 @@ config VIDEO_SUNXI if VIDEO_SUNXI source "drivers/staging/media/sunxi/cedrus/Kconfig" +source "drivers/staging/media/sunxi/sun6i-isp/Kconfig" endif
drivers/staging/media/sunxi/Makefile +1 -0
@@ -1,2 +1,3 @@ # SPDX-License-Identifier: GPL-2.0 obj-$(CONFIG_VIDEO_SUNXI_CEDRUS) += cedrus/ +obj-$(CONFIG_VIDEO_SUN6I_ISP) += sun6i-isp/
drivers/staging/media/sunxi/sun6i-isp/Kconfig new file +15 -0
@@ -0,0 +1,15 @@ +# SPDX-License-Identifier: GPL-2.0-only +config VIDEO_SUN6I_ISP + tristate "Allwinner A31 Image Signal Processor (ISP) Driver" + depends on V4L_PLATFORM_DRIVERS && VIDEO_DEV + depends on ARCH_SUNXI || COMPILE_TEST + depends on PM && COMMON_CLK && HAS_DMA + select MEDIA_CONTROLLER + select VIDEO_V4L2_SUBDEV_API + select VIDEOBUF2_DMA_CONTIG + select VIDEOBUF2_VMALLOC + select V4L2_FWNODE + select REGMAP_MMIO + help + Support for the Allwinner A31 Image Signal Processor (ISP), also + found on other platforms such as the A80, A83T or V3/V3s.
drivers/staging/media/sunxi/sun6i-isp/Makefile new file +4 -0
@@ -0,0 +1,4 @@ +# SPDX-License-Identifier: GPL-2.0-only +sun6i-isp-y += sun6i_isp.o sun6i_isp_proc.o sun6i_isp_capture.o sun6i_isp_params.o + +obj-$(CONFIG_VIDEO_SUN6I_ISP) += sun6i-isp.o
drivers/staging/media/sunxi/sun6i-isp/TODO.txt new file +6 -0
@@ -0,0 +1,6 @@ +Unstaging requirements: +- Add uAPI support and documentation for the configuration of all the hardware + modules and description of the statistics data structures; +- Add support for statistics reporting; +- Add userspace support in libcamera which demonstrates the ability to receive + statistics and adapt hardware modules configuration accordingly;
drivers/staging/media/sunxi/sun6i-isp/sun6i_isp.c new file +555 -0
555 lines not on this page · raw message
drivers/staging/media/sunxi/sun6i-isp/sun6i_isp.h new file +90 -0
90 lines not on this page · raw message
drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_capture.c new file +742 -0
742 lines not on this page · raw message
drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_capture.h new file +78 -0
78 lines not on this page · raw message
drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_params.c new file +566 -0
566 lines not on this page · raw message
drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_params.h new file +52 -0
52 lines not on this page · raw message
drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_proc.c new file +577 -0
577 lines not on this page · raw message
drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_proc.h new file +66 -0
66 lines not on this page · raw message
drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_reg.h new file +275 -0
240 lines not on this page · raw message
drivers/staging/media/sunxi/sun6i-isp/uapi/sun6i-isp-config.h new file +43 -0
43 lines not on this page · raw message
The rest of this diff is not shown. View the raw message.
Signature · 1 line
2.37.3
Part of the diff is not shown here. View the raw message.
PATCH v7 4/6 MAINTAINERS: Add entry for the Allwinner A31 ISP driver
In reply to Paul KocialkowskiAdd myself as maintainer of the Allwinner A31 ISP media driver.
--- MAINTAINERS | 9 +++++++++ 1 file changed, 9 insertions(+)
MAINTAINERS +9 -0
@@ -787,6 +787,15 @@ T: git git://linuxtv.org/media_tree.git F: Documentation/devicetree/bindings/media/allwinner,sun6i-a31-csi.yaml F: drivers/media/platform/sunxi/sun6i-csi/ +ALLWINNER A31 ISP DRIVER +M: Paul Kocialkowski <[email protected]> +L: [email protected] +S: Maintained +T: git git://linuxtv.org/media_tree.git +F: Documentation/devicetree/bindings/media/allwinner,sun6i-a31-isp.yaml +F: drivers/staging/media/sunxi/sun6i-isp/ +F: drivers/staging/media/sunxi/sun6i-isp/uapi/sun6i-isp-config.h + ALLWINNER A31 MIPI CSI-2 BRIDGE DRIVER M: Paul Kocialkowski <[email protected]> L: [email protected]
Signature · 1 line
2.37.3
PATCH v7 5/6 media: sun6i-csi: Detect the availability of the ISP
In reply to Paul KocialkowskiAdd a helper to detect whether the ISP is available and connected and store the indication in the driver-specific device structure.
--- .../platform/sunxi/sun6i-csi/sun6i_csi.c | 34 +++++++++++++++++++ .../platform/sunxi/sun6i-csi/sun6i_csi.h | 3 ++ 2 files changed, 37 insertions(+)
drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.c +34 -0
@@ -24,6 +24,36 @@#include "sun6i_csi_capture.h" #include "sun6i_csi_reg.h" +/* ISP */ + +static int sun6i_csi_isp_detect (struct sun6i_csi_device *csi_dev) +{ +struct device *dev = csi_dev->dev; +struct fwnode_handle *handle; + +/* +* ISP is not available if not connected via fwnode graph. +* This will also check that the remote parent node is available. +*/ + handle =fwnode_graph_get_endpoint_by_id (dev_fwnode (dev), +SUN6I_CSI_PORT_ISP ,0 , +FWNODE_GRAPH_ENDPOINT_NEXT ); +if (!handle) +return 0 ; + +fwnode_handle_put (handle); + +if (!IS_ENABLED (CONFIG_VIDEO_SUN6I_ISP )) { +dev_warn (dev, +"ISP link is detected but not enabled in kernel config!" ); +return 0 ; + } + + csi_dev->isp_available = true; + +return 0 ; +} +/* Media */ static const struct media_device_ops sun6i_csi_media_ops = {
@@ -290,6 +320,10 @@ static int sun6i_csi_probe(struct platform_device *platform_dev)if (ret)return ret; + ret =sun6i_csi_isp_detect (csi_dev); +if (ret) + goto error_resources; + ret =sun6i_csi_v4l2_setup (csi_dev);if (ret) goto error_resources;
drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.h +3 -0
@@ -21,6 +21,7 @@enum sun6i_csi_port {SUN6I_CSI_PORT_PARALLEL =0 ,SUN6I_CSI_PORT_MIPI_CSI2 =1 , +SUN6I_CSI_PORT_ISP =2 , };struct sun6i_csi_buffer {
@@ -44,6 +45,8 @@ struct sun6i_csi_device {struct clk *clock_mod;struct clk *clock_ram;struct reset_control *reset; + +bool isp_available; };struct sun6i_csi_variant {
Signature · 1 line
2.37.3
PATCH v7 6/6 media: sun6i-csi: Add support for hooking to the isp devices
In reply to Paul KocialkowskiIn order to use the isp and csi together, both devices need to be parented to the same v4l2 and media devices. We use the isp as top-level device and let the csi code hook to its v4l2 and media devices when async subdev registration takes place.
As a result v4l2/media device setup is only called when the ISP is missing and the capture device is registered after the devices are hooked. The bridge subdev and its notifier are registered without any device when the ISP is available. Top-level pointers for the devices are introduced to either redirect to the hooked ones (isp available) or the registered ones (isp missing).
Also keep track of whether the capture node was setup or not to avoid cleaning up resources when it wasn't.
--- .../platform/sunxi/sun6i-csi/sun6i_csi.c | 45 +++++++++++++++---- .../platform/sunxi/sun6i-csi/sun6i_csi.h | 7 +++ .../sunxi/sun6i-csi/sun6i_csi_bridge.c | 32 +++++++++++-- .../sunxi/sun6i-csi/sun6i_csi_capture.c | 19 ++++++-- .../sunxi/sun6i-csi/sun6i_csi_capture.h | 1 + 5 files changed, 89 insertions(+), 15 deletions(-)
drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.c +37 -8
@@ -26,6 +26,18 @@/* ISP */ +int sun6i_csi_isp_complete (struct sun6i_csi_device *csi_dev, +struct v4l2_device *v4l2_dev) +{ +if (csi_dev->v4l2_dev && csi_dev->v4l2_dev != v4l2_dev) +return -EINVAL ; + + csi_dev->v4l2_dev = v4l2_dev; + csi_dev->media_dev = v4l2_dev->mdev; + +return sun6i_csi_capture_setup (csi_dev); +} +static int sun6i_csi_isp_detect (struct sun6i_csi_device *csi_dev) {struct device *dev = csi_dev->dev;
@@ -96,6 +108,9 @@ static int sun6i_csi_v4l2_setup(struct sun6i_csi_device *csi_dev) goto error_media; } + csi_dev->v4l2_dev = v4l2_dev; + csi_dev->media_dev = media_dev; +return 0 ; error_media:
@@ -324,17 +339,27 @@ static int sun6i_csi_probe(struct platform_device *platform_dev)if (ret) goto error_resources; - ret =sun6i_csi_v4l2_setup (csi_dev); -if (ret) - goto error_resources; +/* +* Register our own v4l2 and media devices when there is no ISP around. +* Otherwise the ISP will use async subdev registration with our bridge, +* which will provide v4l2 and media devices that are used to register +* the video interface. +*/ +if (!csi_dev->isp_available) { + ret =sun6i_csi_v4l2_setup (csi_dev); +if (ret) + goto error_resources; + } ret =sun6i_csi_bridge_setup (csi_dev);if (ret) goto error_v4l2; - ret =sun6i_csi_capture_setup (csi_dev); -if (ret) - goto error_bridge; +if (!csi_dev->isp_available) { + ret =sun6i_csi_capture_setup (csi_dev); +if (ret) + goto error_bridge; + }return 0 ;
@@ -342,7 +367,8 @@ static int sun6i_csi_probe(struct platform_device *platform_dev)sun6i_csi_bridge_cleanup (csi_dev); error_v4l2: -sun6i_csi_v4l2_cleanup (csi_dev); +if (!csi_dev->isp_available) +sun6i_csi_v4l2_cleanup (csi_dev); error_resources:sun6i_csi_resources_cleanup (csi_dev);
@@ -356,7 +382,10 @@ static int sun6i_csi_remove(struct platform_device *pdev)sun6i_csi_capture_cleanup (csi_dev);sun6i_csi_bridge_cleanup (csi_dev); -sun6i_csi_v4l2_cleanup (csi_dev); + +if (!csi_dev->isp_available) +sun6i_csi_v4l2_cleanup (csi_dev); +sun6i_csi_resources_cleanup (csi_dev);return 0 ;
drivers/media/platform/sunxi/sun6i-csi/sun6i_csi.h +7 -0
@@ -36,6 +36,8 @@ struct sun6i_csi_v4l2 {struct sun6i_csi_device {struct device *dev; +struct v4l2_device *v4l2_dev; +struct media_device *media_dev;struct sun6i_csi_v4l2 v4l2;struct sun6i_csi_bridge bridge;
@@ -53,4 +55,9 @@ struct sun6i_csi_variant { unsigned long clock_mod_rate; }; +/* ISP */ + +int sun6i_csi_isp_complete (struct sun6i_csi_device *csi_dev, +struct v4l2_device *v4l2_dev); +#endif
drivers/media/platform/sunxi/sun6i-csi/sun6i_csi_bridge.c +28 -4
@@ -653,6 +653,7 @@ sun6i_csi_bridge_notifier_bound(struct v4l2_async_notifier *notifier,struct sun6i_csi_bridge *bridge = &csi_dev->bridge;struct sun6i_csi_bridge_source *source = bridge_async_subdev->source;bool enabled; +int ret;switch (source->endpoint.base.port) {case SUN6I_CSI_PORT_PARALLEL :
@@ -667,6 +668,16 @@ sun6i_csi_bridge_notifier_bound(struct v4l2_async_notifier *notifier, source->subdev = remote_subdev; +if (csi_dev->isp_available) { +/* +* Hook to the first available remote subdev to get v4l2 and +* media devices and register the capture device then. +*/ + ret =sun6i_csi_isp_complete (csi_dev, remote_subdev->v4l2_dev); +if (ret) +return ret; + } +return sun6i_csi_bridge_link (csi_dev,SUN6I_CSI_BRIDGE_PAD_SINK , remote_subdev, enabled); }
@@ -679,6 +690,9 @@ sun6i_csi_bridge_notifier_complete(struct v4l2_async_notifier *notifier) bridge.notifier);struct v4l2_device *v4l2_dev = &csi_dev->v4l2.v4l2_dev; +if (csi_dev->isp_available) +return 0 ; +return v4l2_device_register_subdev_nodes (v4l2_dev); }
@@ -752,7 +766,7 @@ int sun6i_csi_bridge_setup(struct sun6i_csi_device *csi_dev) {struct device *dev = csi_dev->dev;struct sun6i_csi_bridge *bridge = &csi_dev->bridge; -struct v4l2_device *v4l2_dev = &csi_dev->v4l2.v4l2_dev; +struct v4l2_device *v4l2_dev = csi_dev->v4l2_dev;struct v4l2_subdev *subdev = &bridge->subdev;struct v4l2_async_notifier *notifier = &bridge->notifier;struct media_pad *pads = bridge->pads;
@@ -793,7 +807,11 @@ int sun6i_csi_bridge_setup(struct sun6i_csi_device *csi_dev)/* V4L2 Subdev */ - ret =v4l2_device_register_subdev (v4l2_dev, subdev); +if (csi_dev->isp_available) + ret =v4l2_async_register_subdev (subdev); +else + ret =v4l2_device_register_subdev (v4l2_dev, subdev); +if (ret) {dev_err (dev,"failed to register v4l2 subdev: %d\n" , ret); goto error_media_entity;
@@ -810,7 +828,10 @@ int sun6i_csi_bridge_setup(struct sun6i_csi_device *csi_dev)sun6i_csi_bridge_source_setup (csi_dev, &bridge->source_mipi_csi2,SUN6I_CSI_PORT_MIPI_CSI2 ,NULL ); - ret =v4l2_async_nf_register (v4l2_dev, notifier); +if (csi_dev->isp_available) + ret =v4l2_async_subdev_nf_register (subdev, notifier); +else + ret =v4l2_async_nf_register (v4l2_dev, notifier);if (ret) {dev_err (dev,"failed to register v4l2 async notifier: %d\n" , ret);
@@ -822,7 +843,10 @@ int sun6i_csi_bridge_setup(struct sun6i_csi_device *csi_dev) error_v4l2_async_notifier:v4l2_async_nf_cleanup (notifier); -v4l2_device_unregister_subdev (subdev); +if (csi_dev->isp_available) +v4l2_async_unregister_subdev (subdev); +else +v4l2_device_unregister_subdev (subdev); error_media_entity:media_entity_cleanup (&subdev->entity);
drivers/media/platform/sunxi/sun6i-csi/sun6i_csi_capture.c +16 -3
@@ -570,7 +570,7 @@ static int sun6i_csi_capture_buffer_prepare(struct vb2_buffer *buffer) {struct sun6i_csi_device *csi_dev =vb2_get_drv_priv (buffer->vb2_queue);struct sun6i_csi_capture *capture = &csi_dev->capture; -struct v4l2_device *v4l2_dev = &csi_dev->v4l2.v4l2_dev; +struct v4l2_device *v4l2_dev = csi_dev->v4l2_dev;struct vb2_v4l2_buffer *v4l2_buffer =to_vb2_v4l2_buffer (buffer);unsigned long size = capture->format.fmt.pix.sizeimage;
@@ -889,7 +889,7 @@ static int sun6i_csi_capture_link_validate(struct media_link *link)struct video_device *video_dev =media_entity_to_video_device (link->sink->entity);struct sun6i_csi_device *csi_dev =video_get_drvdata (video_dev); -struct v4l2_device *v4l2_dev = &csi_dev->v4l2.v4l2_dev; +struct v4l2_device *v4l2_dev = csi_dev->v4l2_dev;const struct sun6i_csi_capture_format *capture_format;const struct sun6i_csi_bridge_format *bridge_format;unsigned int capture_width, capture_height;
@@ -971,7 +971,7 @@ int sun6i_csi_capture_setup(struct sun6i_csi_device *csi_dev) {struct sun6i_csi_capture *capture = &csi_dev->capture;struct sun6i_csi_capture_state *state = &capture->state; -struct v4l2_device *v4l2_dev = &csi_dev->v4l2.v4l2_dev; +struct v4l2_device *v4l2_dev = csi_dev->v4l2_dev;struct v4l2_subdev *bridge_subdev = &csi_dev->bridge.subdev;struct video_device *video_dev = &capture->video_dev;struct vb2_queue *queue = &capture->queue;
@@ -980,6 +980,10 @@ int sun6i_csi_capture_setup(struct sun6i_csi_device *csi_dev)struct v4l2_pix_format *pix_format = &format->fmt.pix;int ret; +/* This may happen with multiple bridge notifier bound calls. */ +if (state->setup) +return 0 ; +/* State */ INIT_LIST_HEAD (&state->queue);
@@ -1055,6 +1059,7 @@ int sun6i_csi_capture_setup(struct sun6i_csi_device *csi_dev) ret =media_create_pad_link (&bridge_subdev->entity,SUN6I_CSI_BRIDGE_PAD_SOURCE , &video_dev->entity,0 , + csi_dev->isp_available ?0 :MEDIA_LNK_FL_ENABLED |MEDIA_LNK_FL_IMMUTABLE );if (ret <0 ) {
@@ -1065,6 +1070,8 @@ int sun6i_csi_capture_setup(struct sun6i_csi_device *csi_dev) goto error_video_device; } + state->setup = true; +return 0 ; error_video_device:
@@ -1083,7 +1090,13 @@ void sun6i_csi_capture_cleanup(struct sun6i_csi_device *csi_dev)struct sun6i_csi_capture *capture = &csi_dev->capture;struct video_device *video_dev = &capture->video_dev; +/* This may happen if async registration failed to complete. */ +if (!capture->state.setup) +return ; +vb2_video_unregister_device (video_dev);media_entity_cleanup (&video_dev->entity);mutex_destroy (&capture->lock); + + capture->state.setup = false; }
drivers/media/platform/sunxi/sun6i-csi/sun6i_csi_capture.h +1 -0
@@ -45,6 +45,7 @@ struct sun6i_csi_capture_state {unsigned int sequence;bool streaming; +bool setup; };struct sun6i_csi_capture {
Signature · 1 line
2.37.3
Hi,
Quoted 2 lines · Paul Kocialkowski
This part only concerns the introduction of the new ISP driver and related adaptation of the CSI driver.
I don't think there is any significant issue preventing this series from being merged in Linux 6.2 at this point.
Could we move forward on it? It's been around for quite some time now.
Thanks!
Paul
Quoted 100 lines
Most non-dt patches still need reviewing but should be pretty straightforward. Since this multi-part series has been going on for a while, it would be great to see it merged soon!
Changes since v6:
- Added a per-compatible check for the required port in dt binding;
- Reworded ISP output port description in dt binding;
- Reversed ISP detection order to have fwnode first;
- Removed info print when ISP link is detected;
- Added warn print when ISP is linked but not enabled in config;
- Fixedsun6i_csi_isp_detectreturn type;
- Removed useless initialization insun6i_csi_isp_detect;
- Fixed typo insun6i_csi_isp_detect;
- Added collected tags;Changes since v5:
- Rebased on latest media tree;
- Added collected tag;
- Switched to usingmedia_pad_remote_pad_first;
- Switched to usingmedia_pad_remote_pad_unique.Changes since v4:
- Fixed device-tree binding indent-align;
- Added collected tag;
- Rebased on latest media tree;Changes since v3:
- Removed the v4l2 controls handler from the driver;
- Added variant structure for table sizes;
- Removed the info message about video device registration;
- Removed comments in uAPI header;
- Used '/schemas/graph.yaml#/properties/port' whenever possible in bindings;
- Added CSI patches dependent on the ISP driver;
- Rebased on the latest media tree;Changes since all-in-one v2:
- UpdatedKconfigto follow the latest media-wide changes;
- Reworked async subdev handling with a dedicated structure holding the corresponding source to avoid matching in the driver;
- Switched to clock-managed regmap mmio;
- Used helper to get a single enabled link for an entity's pad, to replace source selection atlink_validatetime and select the remote source at stream on time instead;
- Added mutex for mbus format serialization;
- Used endpoint-base instead of video-interface for "internal" endpoints in device-tree schema;
- Added TODO with unstaging requirements;
- Various cosmetic cleanups;
- Updated copyright years;Paul Kocialkowski (6):
dt-bindings: media: Add Allwinner A31 ISP bindings documentation
dt-bindings: media: sun6i-a31-csi: Add internal output port to the ISP
staging: media: Add support for the Allwinner A31 ISP
MAINTAINERS: Add entry for the Allwinner A31 ISP driver
media: sun6i-csi: Detect the availability of the ISP
media: sun6i-csi: Add support for hooking to the isp devices.../media/allwinner,sun6i-a31-csi.yaml | 4 + .../media/allwinner,sun6i-a31-isp.yaml | 101 +++ MAINTAINERS | 9 + .../platform/sunxi/sun6i-csi/sun6i_csi.c | 75 +- .../platform/sunxi/sun6i-csi/sun6i_csi.h | 10 + .../sunxi/sun6i-csi/sun6i_csi_bridge.c | 32 +- .../sunxi/sun6i-csi/sun6i_csi_capture.c | 19 +- .../sunxi/sun6i-csi/sun6i_csi_capture.h | 1 + drivers/staging/media/sunxi/Kconfig | 1 + drivers/staging/media/sunxi/Makefile | 1 + drivers/staging/media/sunxi/sun6i-isp/Kconfig | 15 + .../staging/media/sunxi/sun6i-isp/Makefile | 4 + .../staging/media/sunxi/sun6i-isp/TODO.txt | 6 + .../staging/media/sunxi/sun6i-isp/sun6i_isp.c | 555 +++++++++++++ .../staging/media/sunxi/sun6i-isp/sun6i_isp.h | 90 +++ .../media/sunxi/sun6i-isp/sun6i_isp_capture.c | 742 ++++++++++++++++++ .../media/sunxi/sun6i-isp/sun6i_isp_capture.h | 78 ++ .../media/sunxi/sun6i-isp/sun6i_isp_params.c | 566 +++++++++++++ .../media/sunxi/sun6i-isp/sun6i_isp_params.h | 52 ++ .../media/sunxi/sun6i-isp/sun6i_isp_proc.c | 577 ++++++++++++++ .../media/sunxi/sun6i-isp/sun6i_isp_proc.h | 66 ++ .../media/sunxi/sun6i-isp/sun6i_isp_reg.h | 275 +++++++ .../sunxi/sun6i-isp/uapi/sun6i-isp-config.h | 43 + 23 files changed, 3309 insertions(+), 13 deletions(-) create mode 100644 Documentation/devicetree/bindings/media/allwinner,sun6i-a31-isp.yaml create mode 100644 drivers/staging/media/sunxi/sun6i-isp/Kconfig create mode 100644 drivers/staging/media/sunxi/sun6i-isp/Makefile create mode 100644 drivers/staging/media/sunxi/sun6i-isp/TODO.txt create mode 100644 drivers/staging/media/sunxi/sun6i-isp/sun6i_isp.c create mode 100644 drivers/staging/media/sunxi/sun6i-isp/sun6i_isp.h create mode 100644 drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_capture.c create mode 100644 drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_capture.h create mode 100644 drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_params.c create mode 100644 drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_params.h create mode 100644 drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_proc.c create mode 100644 drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_proc.h create mode 100644 drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_reg.h create mode 100644 drivers/staging/media/sunxi/sun6i-isp/uapi/sun6i-isp-config.hSignature · 1 line
2.37.3
Signature · 3 lines
Paul Kocialkowski, Bootlin Embedded Linux and kernel engineering https://bootlin.com
Jernej Škrabec
maintainerHi Paul,
Dne petek, 21. oktober 2022 ob 16:14:26 CEST je Paul Kocialkowski napisal(a):
Quoted 10 lines
Hi,
Quoted 2 lines · Paul Kocialkowski
This part only concerns the introduction of the new ISP driver and related adaptation of the CSI driver.
I don't think there is any significant issue preventing this series from being merged in Linux 6.2 at this point.
Could we move forward on it? It's been around for quite some time now.
While I'm no expert in ISP drivers, I can take a look (for other series too) this week.
Could you attach output of v4l2-complience output (for both)?
Best regards,
Jernej
Quoted 111 lines
Thanks!
Paul
Quoted 106 lines
Most non-dt patches still need reviewing but should be pretty straightforward. Since this multi-part series has been going on for a while, it would be great to see it merged soon!
Changes since v6:
- Added a per-compatible check for the required port in dt binding;
- Reworded ISP output port description in dt binding;
- Reversed ISP detection order to have fwnode first;
- Removed info print when ISP link is detected;
- Added warn print when ISP is linked but not enabled in config;
- Fixedsun6i_csi_isp_detectreturn type;
- Removed useless initialization insun6i_csi_isp_detect;
- Fixed typo insun6i_csi_isp_detect;
- Added collected tags;Changes since v5:
- Rebased on latest media tree;
- Added collected tag;
- Switched to usingmedia_pad_remote_pad_first;
- Switched to usingmedia_pad_remote_pad_unique.Changes since v4:
- Fixed device-tree binding indent-align;
- Added collected tag;
- Rebased on latest media tree;Changes since v3:
- Removed the v4l2 controls handler from the driver;
- Added variant structure for table sizes;
- Removed the info message about video device registration;
- Removed comments in uAPI header;
- Used '/schemas/graph.yaml#/properties/port' whenever possible in bindings; - Added CSI patches dependent on the ISP driver;
- Rebased on the latest media tree;Changes since all-in-one v2:
- UpdatedKconfigto follow the latest media-wide changes;
- Reworked async subdev handling with a dedicated structure holding thecorresponding source to avoid matching in the driver;
- Switched to clock-managed regmap mmio;
- Used helper to get a single enabled link for an entity's pad, to replacesource selection at
link_validatetime and select the remote source at stream on time instead;- Added mutex for mbus format serialization;
- Used endpoint-base instead of video-interface for "internal" endpointsin device-tree schema;
- Added TODO with unstaging requirements;
- Various cosmetic cleanups;
- Updated copyright years;Paul Kocialkowski (6):
dt-bindings: media: Add Allwinner A31 ISP bindings documentation
dt-bindings: media: sun6i-a31-csi: Add internal output port to the ISP
staging: media: Add support for the Allwinner A31 ISP
MAINTAINERS: Add entry for the Allwinner A31 ISP driver
media: sun6i-csi: Detect the availability of the ISP
media: sun6i-csi: Add support for hooking to the isp devices.../media/allwinner,sun6i-a31-csi.yaml | 4 + .../media/allwinner,sun6i-a31-isp.yaml | 101 +++ MAINTAINERS | 9 + .../platform/sunxi/sun6i-csi/sun6i_csi.c | 75 +- .../platform/sunxi/sun6i-csi/sun6i_csi.h | 10 + .../sunxi/sun6i-csi/sun6i_csi_bridge.c | 32 +- .../sunxi/sun6i-csi/sun6i_csi_capture.c | 19 +- .../sunxi/sun6i-csi/sun6i_csi_capture.h | 1 + drivers/staging/media/sunxi/Kconfig | 1 + drivers/staging/media/sunxi/Makefile | 1 + drivers/staging/media/sunxi/sun6i-isp/Kconfig | 15 + .../staging/media/sunxi/sun6i-isp/Makefile | 4 + .../staging/media/sunxi/sun6i-isp/TODO.txt | 6 + .../staging/media/sunxi/sun6i-isp/sun6i_isp.c | 555 +++++++++++++ .../staging/media/sunxi/sun6i-isp/sun6i_isp.h | 90 +++ .../media/sunxi/sun6i-isp/sun6i_isp_capture.c | 742 ++++++++++++++++++ .../media/sunxi/sun6i-isp/sun6i_isp_capture.h | 78 ++ .../media/sunxi/sun6i-isp/sun6i_isp_params.c | 566 +++++++++++++ .../media/sunxi/sun6i-isp/sun6i_isp_params.h | 52 ++ .../media/sunxi/sun6i-isp/sun6i_isp_proc.c | 577 ++++++++++++++ .../media/sunxi/sun6i-isp/sun6i_isp_proc.h | 66 ++ .../media/sunxi/sun6i-isp/sun6i_isp_reg.h | 275 +++++++ .../sunxi/sun6i-isp/uapi/sun6i-isp-config.h | 43 + 23 files changed, 3309 insertions(+), 13 deletions(-) create mode 100644 Documentation/devicetree/bindings/media/allwinner,sun6i-a31-isp.yaml create mode 100644 drivers/staging/media/sunxi/sun6i-isp/Kconfig create mode 100644 drivers/staging/media/sunxi/sun6i-isp/Makefile create mode 100644 drivers/staging/media/sunxi/sun6i-isp/TODO.txt create mode 100644 drivers/staging/media/sunxi/sun6i-isp/sun6i_isp.c create mode 100644 drivers/staging/media/sunxi/sun6i-isp/sun6i_isp.h create mode 100644 drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_capture.c create mode 100644 drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_capture.h create mode 100644 drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_params.c create mode 100644 drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_params.h create mode 100644 drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_proc.c create mode 100644 drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_proc.h create mode 100644 drivers/staging/media/sunxi/sun6i-isp/sun6i_isp_reg.h create mode 100644
drivers/staging/media/sunxi/sun6i-isp/uapi/sun6i-isp-config.h