S3C244X/S3C64XX SoC camera host interface driver

Hi all,

This patch adds a driver for Samsung S3C244X/S3C64XX SoC series camera host interface. My intention was to create a V4L2 driver that would work with standard applications like Gstreamer or mplayer, and yet exposing possibly all features available in the hardware.

It took me several weeks to do this work in my (limited) spare time. Finally I've got something that is functional and I think might be useful for others, so I'm publishing this initial version. It hopefully doesn't need much tweaking or corrections, at least as far as S3C244X is concerned. It has not been tested on S3C64XX SoCs, as I don't have the hardware. However, the driver has been designed with covering S3C64XX as well in mind, and I've already taken care of some differences between S3C2444X and S3C64XX. Mem-to-mem features are not yet supported, but these are quite separate issue and could be easily added as a next step.

The patch to follow only adds the CAMIF driver, the other two required for the camera on Mini2440 board to work (OV9650 sensor driver and the board file patch) can be found in branch s3c-camif-v3.5 in git repository:

git://github.com/snawrocki/linux.git

Gitweb: https://github.com/snawrocki/linux/commits/s3c-camif-v3.5

These patches are based off of stable v3.5 kernel.

The S3C-CAMIF driver exposes one v4l2 subdev and two capture video nodes
- for the "codec" and "preview" data paths. For minimum functionality there is no need to touch the subdev device node in user space. However it is recommended for best results.

For my tests I used nice Mini2440 BSP from Pengutronix, which already contains various applications, like Gstreamer, v4l2-ctl or even media-ctl. It's available at http://www.pengutronix.de/oselas/bsp/pengutronix/index_en.html.

I've tested the driver with mplayer and Gstreamer, also simultaneous capture from the "preview" and "codec" video nodes. The codec path camera preview at framebuffer can be started, for example, with following command:

# gst-launch v4l2src device=/dev/video0 ! video/x-raw-yuv,width=320,height=
  240 ! ffmpegcolorspace ! fbdevsink

In order to select the preview video node (which supports only RGB pixel formats) /dev/video0 need to be replaced with /dev/video1, e.g.

# gst-launch v4l2src device=/dev/video1 ! video/x-raw-rgb,bpp=32,endianness=
  4321,depth=32,width=320,height=240 ! ffmpegcolorspace ! fbdevsink

In this case I'm getting slightly incorrect color representation, still need to figure out why this happens.

The supported pixel formats are listed in the attached supported_pixel_formats.txt file.

The camera test pattern is exposed through a private control at the subdev node, all supported controls are listed in attached supported_controls.txt file. The test pattern can be enabled e.g. with command:

# v4l2-ctl -d /dev/v4l-subdev1 --set-ctrl=test_pattern=1

A note about the driver's internals

The S3C-CAMIF driver sets at the camera input ("catchcam") (default) pixel format as retrieved from the sensor subdev. This happens during driver's initialization, so there is no need to touch the subdev in user space to capture video from the camera. If pixel resolution selected at /dev/video? differs from the one set at camera input (S3C-CAMIF subdev pad 0), the image frames will be resized accordingly, taking into account the resizer's capabilities.

To change pixel format at the sensor subdev and the camif input, so those better match our target capture resolution, following commands can be used:

media-ctl --set-v4l2 '"OV9650":0 [fmt: YUYV2X8/640x640]' media-ctl --set-v4l2 '"S3C-CAMIF":0 [fmt: YUYV2X8/640x640]'

The only requirement is that these formats are same, otherwise it won't be possible to start streaming and VIDIOC_STREAMON will fail wit -EPIPE errno.

The video node numbering might be different, if there are other V4L2 drivers in the system. It can be easily checked with media-ctl utility (media-ctl -p), my configuration was as in attached camif_media_info.txt log.

I've run v4l2-compliance on both video nodes, the results can be found in file v4l2_compliance_log.txt.

A graph depicting device topology can be generated from attached camif_graph.dot file (which was created with 'media-ctl --print-dot'), with following command:

# cat camif_graph.dot | dot -Tpdf > camif_graph.pdf
# evince camif_graph.pdf

There is still some more work needed to make the OV9650 sensor driver ready for the mainline, I'm planning to take care of it in near future. As for the CAMIF driver, I might try to push it upstream if it doesn't take too much of my time and there is enough interest from the users side.

Feature requests or bug reports are welcome.

Regards,
Sylwester

---

Sylwester Nawrocki (1):
V4L: Add driver for S3C244X/S3C64XX SoC series camera interface

 drivers/media/video/Kconfig                   |   12 +
 drivers/media/video/Makefile                  |    1 +
 drivers/media/video/s3c-camif/Makefile        |    5 +
 drivers/media/video/s3c-camif/camif-capture.c | 1602 +++++++++++++++++++++
 drivers/media/video/s3c-camif/camif-core.c    |  625 ++++++++
 drivers/media/video/s3c-camif/camif-core.h    |  375 ++++
 drivers/media/video/s3c-camif/camif-regs.c    |  497 ++++++
 drivers/media/video/s3c-camif/camif-regs.h    |  262 ++
 include/media/s3c_camif.h                     |   36 +
 9 files changed, 3415 insertions(+), 0 deletions(-)
 create mode 100644 drivers/media/video/s3c-camif/Makefile
 create mode 100644 drivers/media/video/s3c-camif/camif-capture.c
 create mode 100644 drivers/media/video/s3c-camif/camif-core.c
 create mode 100644 drivers/media/video/s3c-camif/camif-core.h
 create mode 100644 drivers/media/video/s3c-camif/camif-regs.c
 create mode 100644 drivers/media/video/s3c-camif/camif-regs.h
 create mode 100644 include/media/s3c_camif.h

root@mini2440:~ v4l2-compliance -d /dev/video0
Driver Info:

        Driver name   : s3c-camif
        Card type     : s3c-camif
        Bus info      : platform
        Driver version: 3.5.0
        Capabilities  : 0x84000001
                Video Capture
                Streaming
                Device Capabilities
        Device Caps   : 0x04000001
                Video Capture
                Streaming

Compliance test for device /dev/video0 (not using libv4l2):

Required ioctls:
test VIDIOC_QUERYCAP: OK

Allow for multiple opens:
test second video open: OK
test VIDIOC_QUERYCAP: OK
test VIDIOC_G/S_PRIORITY: OK

Debug ioctls:
test VIDIOC_DBG_G_CHIP_IDENT: Not Supported test VIDIOC_DBG_G/S_REGISTER: Not Supported test VIDIOC_LOG_STATUS: OK

Input ioctls:
test VIDIOC_G/S_TUNER: Not Supported
test VIDIOC_G/S_FREQUENCY: Not Supported
test VIDIOC_S_HW_FREQ_SEEK: Not Supported
test VIDIOC_ENUMAUDIO: Not Supported
test VIDIOC_G/S/ENUMINPUT: OK
test VIDIOC_G/S_AUDIO: Not Supported
Inputs: 1 Audio Inputs: 0 Tuners: 0

Output ioctls:
test VIDIOC_G/S_MODULATOR: Not Supported
test VIDIOC_G/S_FREQUENCY: Not Supported
test VIDIOC_ENUMAUDOUT: Not Supported
test VIDIOC_G/S/ENUMOUTPUT: Not Supported
test VIDIOC_G/S_AUDOUT: Not Supported
Outputs: 0 Audio Outputs: 0 Modulators: 0

Control ioctls:
test VIDIOC_QUERYCTRL/MENU: OK
test VIDIOC_G/S_CTRL: OK
test VIDIOC_G/S/TRY_EXT_CTRLS: OK test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK test VIDIOC_G/S_JPEGCOMP: Not Supported Standard Controls: 3 Private Controls: 0

Input/Output configuration ioctls:
test VIDIOC_ENUM/G/S/QUERY_STD: Not Supported test VIDIOC_ENUM/G/S/QUERY_DV_PRESETS: Not Supported test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: Not Supported test VIDIOC_DV_TIMINGS_CAP: Not Supported

Format ioctls:
test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
fail: v4l2-test-formats.cpp(558): cap->readbuffers
test VIDIOC_G/S_PARM: FAIL
test VIDIOC_G_FBUF: Not Supported
test VIDIOC_G_FMT: OK
test VIDIOC_G_SLICED_VBI_CAP: Not Supported
Buffer ioctls:
test VIDIOC_REQBUFS/CREATE_BUFS: OK
test read/write: OK
Total: 34, Succeeded: 33, Failed: 1, Warnings: 0

root@mini2440:~ v4l2-compliance -d /dev/video1
Driver Info:

        Driver name   : s3c-camif
        Card type     : s3c-camif
        Bus info      : platform
        Driver version: 3.5.0
        Capabilities  : 0x84000001
                Video Capture
                Streaming
                Device Capabilities
        Device Caps   : 0x04000001
                Video Capture
                Streaming

Compliance test for device /dev/video1 (not using libv4l2):

Required ioctls:
test VIDIOC_QUERYCAP: OK

Allow for multiple opens:
test second video open: OK
test VIDIOC_QUERYCAP: OK
test VIDIOC_G/S_PRIORITY: OK

Debug ioctls:
test VIDIOC_DBG_G_CHIP_IDENT: Not Supported test VIDIOC_DBG_G/S_REGISTER: Not Supported test VIDIOC_LOG_STATUS: OK

Input ioctls:
test VIDIOC_G/S_TUNER: Not Supported
test VIDIOC_G/S_FREQUENCY: Not Supported
test VIDIOC_S_HW_FREQ_SEEK: Not Supported
test VIDIOC_ENUMAUDIO: Not Supported
test VIDIOC_G/S/ENUMINPUT: OK
test VIDIOC_G/S_AUDIO: Not Supported
Inputs: 1 Audio Inputs: 0 Tuners: 0

Output ioctls:
test VIDIOC_G/S_MODULATOR: Not Supported
test VIDIOC_G/S_FREQUENCY: Not Supported
test VIDIOC_ENUMAUDOUT: Not Supported
test VIDIOC_G/S/ENUMOUTPUT: Not Supported
test VIDIOC_G/S_AUDOUT: Not Supported
Outputs: 0 Audio Outputs: 0 Modulators: 0

Control ioctls:
test VIDIOC_QUERYCTRL/MENU: OK
test VIDIOC_G/S_CTRL: OK
test VIDIOC_G/S/TRY_EXT_CTRLS: OK test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK test VIDIOC_G/S_JPEGCOMP: Not Supported Standard Controls: 3 Private Controls: 0

Input/Output configuration ioctls:
test VIDIOC_ENUM/G/S/QUERY_STD: Not Supported test VIDIOC_ENUM/G/S/QUERY_DV_PRESETS: Not Supported test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: Not Supported test VIDIOC_DV_TIMINGS_CAP: Not Supported

Format ioctls:
test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
fail: v4l2-test-formats.cpp(558): cap->readbuffers
test VIDIOC_G/S_PARM: FAIL
test VIDIOC_G_FBUF: Not Supported
test VIDIOC_G_FMT: OK
test VIDIOC_G_SLICED_VBI_CAP: Not Supported
Buffer ioctls:
test VIDIOC_REQBUFS/CREATE_BUFS: OK
test read/write: OK
Total: 34, Succeeded: 33, Failed: 1, Warnings: 0

root@mini2440:~ media-ctl -p
Opening media device /dev/media0
Enumerating entities
Found 4 entities
Enumerating pads and links
Media controller API version 0.0.0

Media device information
------------------------
driver s3c-camif
model SAMSUNG S3C244X CAMIF
serial

bus info        platform
hw revision     0x20
driver version  1.0.0

Device topology
- entity 1: OV9650 (1 pad, 1 link)
type V4L2 subdev subtype Sensor device node name /dev/v4l-subdev0
pad0: Source
[fmt:YUYV2X8/640x480]
-> "S3C-CAMIF":0 [ENABLED,IMMUTABLE]

- entity 2: S3C-CAMIF (3 pads, 3 links)

            type V4L2 subdev subtype Unknown
            device node name /dev/v4l-subdev1
        pad0: Sink
                [fmt:YUYV2X8/640x480
                 crop.bounds:(0,0)/640x480
                 crop:(0,0)/640x480]
                <- "OV9650":0 [ENABLED,IMMUTABLE]
        pad1: Source
                [fmt:YUYV2X8/640x480]
                -> "camif-codec":0 [ENABLED,IMMUTABLE]
        pad2: Source
                [fmt:YUYV2X8/640x480]
                -> "camif-preview":0 [ENABLED,IMMUTABLE]

- entity 3: camif-codec (1 pad, 1 link)
type Node subtype V4L
device node name /dev/video0
pad0: Sink
<- "S3C-CAMIF":1 [ENABLED,IMMUTABLE]

- entity 4: camif-preview (1 pad, 1 link) type Node subtype V4L
device node name /dev/video1
pad0: Sink
<- "S3C-CAMIF":2 [ENABLED,IMMUTABLE]

root@mini2440:~ v4l2-ctl -d /dev/video0 --list-formats
ioctl: VIDIOC_ENUM_FMT
Index : 0
Type : Video Capture
Pixel Format: '422P'
Name : YUV 4:2:2 planar, Y/Cb/Cr

Index : 1
Type : Video Capture
Pixel Format: 'YU12'
Name : YUV 4:2:0 planar, Y/Cb/Cr

root@mini2440:~
root@mini2440:~
root@mini2440:~
root@mini2440:~
root@mini2440:~
root@mini2440:~ v4l2-ctl -d /dev/video1 --list-formats
ioctl: VIDIOC_ENUM_FMT
Index : 0
Type : Video Capture
Pixel Format: 'RGBR'
Name : RGB565, 16 bpp

Index : 1
Type : Video Capture
Pixel Format: 'RGB4'
Name : XRGB8888, 32 bpp

root@mini2440:~ v4l2-ctl -d /dev/video0 --list-ctrls

User Controls

horizontal_flip (bool) : default=0 value=0
vertical_flip (bool) : default=0 value=0

root@mini2440:~ v4l2-ctl -d /dev/video1 --list-ctrls

User Controls

horizontal_flip (bool) : default=0 value=0
vertical_flip (bool) : default=0 value=0

root@mini2440:~ v4l2-ctl -d /dev/v4l-subdev1 --list-ctrls
VIDIOC_QUERYCAP: failed: Inappropriate ioctl for device

User Controls

test_pattern (int) : min=0 max=3 step=1 default=0 value=0

This patch adds V4L2 driver for Samsung S3C244X/S3C64XX SoC series. It exposes a subdev device node for CAMIF pixel resolution and crop control and two video nodes - for the "codec" and "preview" data paths. This patch has been tested only on S3C2440 SoC based system (Mini2440 board). Some minor corrections may be needed for capture functionality on S3C64XX. mem-to-mem features available in S3C64XX SoC series are not supported yet.

---
 drivers/media/video/Kconfig                   |   12 +
 drivers/media/video/Makefile                  |    1 +
 drivers/media/video/s3c-camif/Makefile        |    5 +
 drivers/media/video/s3c-camif/camif-capture.c | 1602 +++++++++++++++++++++++++
 drivers/media/video/s3c-camif/camif-core.c    |  625 ++++++++++
 drivers/media/video/s3c-camif/camif-core.h    |  375 ++++++
 drivers/media/video/s3c-camif/camif-regs.c    |  497 ++++++++
 drivers/media/video/s3c-camif/camif-regs.h    |  262 ++++
 include/media/s3c_camif.h                     |   36 +
 9 files changed, 3415 insertions(+), 0 deletions(-)
 create mode 100644 drivers/media/video/s3c-camif/Makefile
 create mode 100644 drivers/media/video/s3c-camif/camif-capture.c
 create mode 100644 drivers/media/video/s3c-camif/camif-core.c
 create mode 100644 drivers/media/video/s3c-camif/camif-core.h
 create mode 100644 drivers/media/video/s3c-camif/camif-regs.c
 create mode 100644 drivers/media/video/s3c-camif/camif-regs.h
 create mode 100644 include/media/s3c_camif.h

index 99937c9..1f445a0 100644
@@ -1151,6 +1151,18 @@ config VIDEO_ATMEL_ISI
 	  This module makes the ATMEL Image Sensor Interface available
 	  as a v4l2 device.
 
+config VIDEO_S3C_CAMIF
+	tristate "Samsung S3C244X/S3C64XX SoC Camera Interface driver"
+	depends on VIDEO_V4L2 && I2C && VIDEO_V4L2_SUBDEV_API
+	depends on (PLAT_S3C64XX || PLAT_S3C24XX)
+	select VIDEOBUF2_DMA_CONTIG
+	---help---
+	  This is a v4l2 driver for s3c244x and s3c64xx SoC series camera
+	  host interface (CAMIF).
+
+	  To compile this driver as a module, choose M here: the module
+	  will be called s3c-camif.
+
 source "drivers/media/video/s5p-fimc/Kconfig"
 source "drivers/media/video/s5p-tv/Kconfig"
 

index d209de0..f5f6086 100644
@@ -188,6 +188,7 @@ obj-$(CONFIG_VIDEO_ATMEL_ISI)		+= atmel-isi.o
 
 obj-$(CONFIG_VIDEO_MX2_EMMAPRP)		+= mx2_emmaprp.o
 
+obj-$(CONFIG_VIDEO_S3C_CAMIF) 		+= s3c-camif/
 obj-$(CONFIG_VIDEO_SAMSUNG_S5P_FIMC) 	+= s5p-fimc/
 obj-$(CONFIG_VIDEO_SAMSUNG_S5P_JPEG)	+= s5p-jpeg/
 obj-$(CONFIG_VIDEO_SAMSUNG_S5P_MFC)	+= s5p-mfc/

new file mode 100644
index 0000000..50bf8c5
@@ -0,0 +1,5 @@
+# Makefile for s3c244x/s3c64xx CAMIF driver
+
+s3c-camif-objs := camif-core.o camif-capture.o camif-regs.o
+
+obj-$(CONFIG_VIDEO_S3C_CAMIF) += s3c-camif.o

new file mode 100644
index 0000000..dd89124

--
1.7.4.1

Part of the diff is not shown here. View the raw message.

Hi,

I will try to test it on S3C6410 in some reasonably near future and report any needed corrections.

Currently I am using a modified s5p-fimc driver in my project, but it supports only the codec path of S3C64xx and any needed stream duplication and rescaling is done in later processing, so it might be wise to migrate to yours.

Best regards,
Tomasz Figa

Sounds great, thanks.

Yeah, the s3c camif is quite different from the s5p one. Thus the s3c-camif driver might be a much better starting point for S3C6410. I could test any changes for s3c6410 back on s3c2440 and incorporate the verified ones into some common stable git branch.

--

Regards,
Sylwester

I have not tested the driver yet, but I am looking through the code and it seems like S3C6410 (at least according to the documentation) supports far more pixel formats than defined in the driver.

Both preview and scaler paths are supposed to support 420/422 planar, 422 interleaved and 565/666/888 formats.

Also two distinct planar 420 formats exist that simply differ by plane order YUV420 (currently supported in your driver) and YVU420 (with Cb and Cr planes swapped). It should be pretty straightforward to add support for the latter.

Best regards,
Tomasz Figa

Hi Tomasz,

Indeed, somehow I missed that s3c64xx supports most of the pixel formats on both: the preview and the codec data paths. I've updated the pixel format definitions to reflect that, but it still needs to be verified with proper tests. I just didn't add YUV422 packed format, I expect it to be done by someone who actually verifies that it works, after checking/updating camif-regs.c as well.

Yeah, thanks for the suggestion. I've added support for YVU420 - it yields more options where setting up gstreamer pipelines, along with a few fixes for issues I've found after some more testing, including LKM build. It can be pulled from following git tree:

git://linuxtv.org/snawrocki/media.git s3c-camif-devel

It's based off of staging/for_v3.7 branch (3.6-rc3). I consider it more or less stable. The final branch for mainline is 's3c-camif', the difference is only that the fixes were squashed to a single commit adding the whole driver there.

--

Thanks,
Sylwester