Skip to contentSkip to conversation

media: adv748x: add support for HDMI audio

PATCH Superseded Alex Riesen 2 participants 14 messages Mar 26, 2020 – Apr 2, 2020

Subsystems (5)

v4 of 5 Superseded Latest is v5 →

lore
Share & tools
lore
#0 lore raw

v4 cover media: adv748x: add support for HDMI audio

This adds minimal support for accessing the HDMI audio provided through the I2S port available on ADV7481 and ADV7482 decoder devices by ADI. The port carries audio signal from the decoded HDMI stream.

Currently, the driver only supports I2S in TDM, 8 channels a 24bit at 48kHz. Furthermore, only left-justified, 8 slots, 32bit/slot TDM, at 256fs has been ever tried.

An ADV7482 on the Renesas Salvator-X ES1.1 (R8A77950 SoC) was used during development of this code.

Changes since v3:
- use clk_hw instead of clk

- formatting improvements and use const where possible

- removed implementation of log_status and EDID setting ioctls, those will be submitted as separate patches.

Changes since v2:
- prepare/enable the clock when it is used, as it seems nothing else does this otherwise

- give the clock a unique name to ensure it can be registered if there are multiple adv748x devices in the system

- remove optionality note from clock cell description to ensure the device description matches the real device (the line is always present, even if not used)

Changes since v1:
- Add ssi4_ctrl pin group to the sound pins. The pins are responsible for SCK4 (sample clock) WS4 and (word boundary input), and are required for SSI audio input over I2S.

- Removed the audio clock C from the list of clocks of adv748x, it is exactly the other way around.

- Add an instance of (currently) fixed rate I2S master clock (MCLK), connected to the audio_clk_c line of the R-Car SoC. Explicitly declare the device a clock producer and add it to the list of clocks used by the audio system of the Salvator-X board.

- The implementation of DAI driver has been moved in a separate file and modified to activate audio decoding and I2S streaming using snd_soc_dai_... interfaces. This allows the driver to be used with just ALSA interfaces.

- The ioctls for selecting audio output and muting have been removed, as not applicable.

I have left implementation of the QUERYCAP in, as it seems to be required by v4l-ctl to support loading of EDID for this node. And setting the EDID is one feature I desperately need: there are devices which plainly refuse to talk to the sink if it does not provide EDID they like.

- A device tree configuration without audio port will disable the audio code altogether, supporting integrations where the port is not connected.

- The patches have been re-arranged, starting with the generic changes and changes not related to audio directly. Those will be probably sent as a separate series later.

- The whole series has been rebased on top of v5.6-rc6

Alex Riesen (9):
media: adv748x: fix end-of-line terminators in diagnostic statements
media: adv748x: include everything adv748x.h needs into the file
media: adv748x: reduce amount of code for bitwise modifications of
device registers
media: adv748x: add definitions for audio output related registers
media: adv748x: add support for HDMI audio
media: adv748x: prepare/enable mclk when the audio is used
media: adv748x: only activate DAI if it is described in device tree
dt-bindings: adv748x: add information about serial audio interface

    (I2S/TDM)

arm64: dts: renesas: salvator: add a connection from adv748x codec
(HDMI input) to the R-Car SoC

 .../devicetree/bindings/media/i2c/adv748x.txt |  16 +-
 .../boot/dts/renesas/r8a77950-salvator-x.dts  |   3 +-
 .../boot/dts/renesas/salvator-common.dtsi     |  47 ++-
 drivers/media/i2c/adv748x/Makefile            |   3 +-
 drivers/media/i2c/adv748x/adv748x-afe.c       |   6 +-
 drivers/media/i2c/adv748x/adv748x-core.c      |  45 +--
 drivers/media/i2c/adv748x/adv748x-csi2.c      |   8 +-
 drivers/media/i2c/adv748x/adv748x-dai.c       | 278 ++++++++++++++++++
 drivers/media/i2c/adv748x/adv748x-hdmi.c      |   6 +-
 drivers/media/i2c/adv748x/adv748x.h           |  65 +++-
 10 files changed, 435 insertions(+), 42 deletions(-)
 create mode 100644 drivers/media/i2c/adv748x/adv748x-dai.c
Signature · 1 line
2.25.1.25.g9ecbe7eb18
#1 lore raw

PATCH v4 1/9 media: adv748x: fix end-of-line terminators in diagnostic statements

---
 drivers/media/i2c/adv748x/adv748x-core.c | 24 ++++++++++++------------
 drivers/media/i2c/adv748x/adv748x-csi2.c |  2 +-
 2 files changed, 13 insertions(+), 13 deletions(-)
drivers/media/i2c/adv748x/adv748x-core.c +12 -12
index 23e02ff27b17..c3fb113cef62 100644
@@ -623,11 +623,11 @@ static int adv748x_parse_dt(struct adv748x_state *state)
 
 	for_each_endpoint_of_node(state->dev->of_node, ep_np) {
 		of_graph_parse_endpoint(ep_np, &ep);
-		adv_info(state, "Endpoint %pOF on port %d", ep.local_node,
+		adv_info(state, "Endpoint %pOF on port %d\n", ep.local_node,
 			 ep.port);
 
 		if (ep.port >= ADV748X_PORT_MAX) {
-			adv_err(state, "Invalid endpoint %pOF on port %d",
+			adv_err(state, "Invalid endpoint %pOF on port %d\n",
 				ep.local_node, ep.port);
 
 			continue;
@@ -635,7 +635,7 @@ static int adv748x_parse_dt(struct adv748x_state *state)
 
 		if (state->endpoints[ep.port]) {
 			adv_err(state,
-				"Multiple port endpoints are not supported");
+				"Multiple port endpoints are not supported\n");
 			continue;
 		}
 
@@ -702,62 +702,62 @@ static int adv748x_probe(struct i2c_client *client)
 	/* Discover and process ports declared by the Device tree endpoints */
 	ret = adv748x_parse_dt(state);
 	if (ret) {
-		adv_err(state, "Failed to parse device tree");
+		adv_err(state, "Failed to parse device tree\n");
 		goto err_free_mutex;
 	}
 
 	/* Configure IO Regmap region */
 	ret = adv748x_configure_regmap(state, ADV748X_PAGE_IO);
 	if (ret) {
-		adv_err(state, "Error configuring IO regmap region");
+		adv_err(state, "Error configuring IO regmap region\n");
 		goto err_cleanup_dt;
 	}
 
 	ret = adv748x_identify_chip(state);
 	if (ret) {
-		adv_err(state, "Failed to identify chip");
+		adv_err(state, "Failed to identify chip\n");
 		goto err_cleanup_dt;
 	}
 
 	/* Configure remaining pages as I2C clients with regmap access */
 	ret = adv748x_initialise_clients(state);
 	if (ret) {
-		adv_err(state, "Failed to setup client regmap pages");
+		adv_err(state, "Failed to setup client regmap pages\n");
 		goto err_cleanup_clients;
 	}
 
 	/* SW reset ADV748X to its default values */
 	ret = adv748x_reset(state);
 	if (ret) {
-		adv_err(state, "Failed to reset hardware");
+		adv_err(state, "Failed to reset hardware\n");
 		goto err_cleanup_clients;
 	}
 
 	/* Initialise HDMI */
 	ret = adv748x_hdmi_init(&state->hdmi);
 	if (ret) {
-		adv_err(state, "Failed to probe HDMI");
+		adv_err(state, "Failed to probe HDMI\n");
 		goto err_cleanup_clients;
 	}
 
 	/* Initialise AFE */
 	ret = adv748x_afe_init(&state->afe);
 	if (ret) {
-		adv_err(state, "Failed to probe AFE");
+		adv_err(state, "Failed to probe AFE\n");
 		goto err_cleanup_hdmi;
 	}
 
 	/* Initialise TXA */
 	ret = adv748x_csi2_init(state, &state->txa);
 	if (ret) {
-		adv_err(state, "Failed to probe TXA");
+		adv_err(state, "Failed to probe TXA\n");
 		goto err_cleanup_afe;
 	}
 
 	/* Initialise TXB */
 	ret = adv748x_csi2_init(state, &state->txb);
 	if (ret) {
-		adv_err(state, "Failed to probe TXB");
+		adv_err(state, "Failed to probe TXB\n");
 		goto err_cleanup_txa;
 	}
 
drivers/media/i2c/adv748x/adv748x-csi2.c +1 -1
index 2091cda50935..c43ce5d78723 100644
@@ -72,7 +72,7 @@ static int adv748x_csi2_registered(struct v4l2_subdev *sd)
 	struct adv748x_state *state = tx->state;
 	int ret;
 
-	adv_dbg(state, "Registered %s (%s)", is_txa(tx) ? "TXA":"TXB",
+	adv_dbg(state, "Registered %s (%s)\n", is_txa(tx) ? "TXA":"TXB",
 			sd->name);
 
 	/*
Signature · 1 line
2.25.1.25.g9ecbe7eb18
#2 lore raw

PATCH v4 2/9 media: adv748x: include everything adv748x.h needs into the file

In reply to Alex Riesen

To follow the established practice of not depending on others to pull everything in. While at it, make sure it stays like this.

---
 drivers/media/i2c/adv748x/adv748x-afe.c  | 6 ++----
 drivers/media/i2c/adv748x/adv748x-core.c | 6 ++----
 drivers/media/i2c/adv748x/adv748x-csi2.c | 6 ++----
 drivers/media/i2c/adv748x/adv748x-hdmi.c | 6 ++----
 drivers/media/i2c/adv748x/adv748x.h      | 2 ++
 5 files changed, 10 insertions(+), 16 deletions(-)
drivers/media/i2c/adv748x/adv748x-afe.c +2 -4
index dbbb1e4d6363..5a25d1fbe25f 100644
@@ -6,18 +6,16 @@
  * Copyright (C) 2017 Renesas Electronics Corp.
  */
 
+#include "adv748x.h"
+
 #include <linux/delay.h>
 #include <linux/module.h>
 #include <linux/mutex.h>
 #include <linux/v4l2-dv-timings.h>
 
-#include <media/v4l2-ctrls.h>
-#include <media/v4l2-device.h>
 #include <media/v4l2-dv-timings.h>
 #include <media/v4l2-ioctl.h>
 
-#include "adv748x.h"
-
 /* -----------------------------------------------------------------------------
  * SDP
  */
drivers/media/i2c/adv748x/adv748x-core.c +2 -4
index c3fb113cef62..5c59aad319d1 100644
@@ -10,6 +10,8 @@
  *	Kieran Bingham <[email protected]>
  */
 
+#include "adv748x.h"
+
 #include <linux/delay.h>
 #include <linux/errno.h>
 #include <linux/i2c.h>
@@ -20,14 +22,10 @@
 #include <linux/slab.h>
 #include <linux/v4l2-dv-timings.h>
 
-#include <media/v4l2-ctrls.h>
-#include <media/v4l2-device.h>
 #include <media/v4l2-dv-timings.h>
 #include <media/v4l2-fwnode.h>
 #include <media/v4l2-ioctl.h>
 
-#include "adv748x.h"
-
 /* -----------------------------------------------------------------------------
  * Register manipulation
  */
drivers/media/i2c/adv748x/adv748x-csi2.c +2 -4
index c43ce5d78723..c00d4f347d95 100644
@@ -5,15 +5,13 @@
  * Copyright (C) 2017 Renesas Electronics Corp.
  */
 
+#include "adv748x.h"
+
 #include <linux/module.h>
 #include <linux/mutex.h>
 
-#include <media/v4l2-ctrls.h>
-#include <media/v4l2-device.h>
 #include <media/v4l2-ioctl.h>
 
-#include "adv748x.h"
-
 static int adv748x_csi2_set_virtual_channel(struct adv748x_csi2 *tx,
 					    unsigned int vc)
 {
drivers/media/i2c/adv748x/adv748x-hdmi.c +2 -4
index c557f8fdf11a..f598acec3b5c 100644
@@ -5,18 +5,16 @@
  * Copyright (C) 2017 Renesas Electronics Corp.
  */
 
+#include "adv748x.h"
+
 #include <linux/module.h>
 #include <linux/mutex.h>
 
-#include <media/v4l2-ctrls.h>
-#include <media/v4l2-device.h>
 #include <media/v4l2-dv-timings.h>
 #include <media/v4l2-ioctl.h>
 
 #include <uapi/linux/v4l2-dv-timings.h>
 
-#include "adv748x.h"
-
 /* -----------------------------------------------------------------------------
  * HDMI and CP
  */
drivers/media/i2c/adv748x/adv748x.h +2 -0
index fccb388ce179..09aab4138c3f 100644
@@ -19,6 +19,8 @@
  */
 
 #include <linux/i2c.h>
+#include <media/v4l2-ctrls.h>
+#include <media/v4l2-device.h>
 
 #ifndef _ADV748X_H_
 #define _ADV748X_H_
Signature · 1 line
2.25.1.25.g9ecbe7eb18
#3 lore raw

PATCH v4 3/9 media: adv748x: reduce amount of code for bitwise modifications of device registers

In reply to Alex Riesen

The regmap provides a convenient utility for this.
The hdmi_* and dpll_* register modification macros added for symmetry with the existing operations (io_*, sdp_*).

--
v3: remove _update name in favor of existing _clrset

---
 drivers/media/i2c/adv748x/adv748x-core.c |  6 ++++++
 drivers/media/i2c/adv748x/adv748x.h      | 14 +++++++++++---
 2 files changed, 17 insertions(+), 3 deletions(-)
drivers/media/i2c/adv748x/adv748x-core.c +6 -0
index 5c59aad319d1..8580e6624276 100644
@@ -133,6 +133,12 @@ static int adv748x_write_check(struct adv748x_state *state, u8 page, u8 reg,
 	return *error;
 }
 
+int adv748x_update_bits(struct adv748x_state *state, u8 page, u8 reg, u8 mask,
+			u8 value)
+{
+	return regmap_update_bits(state->regmap[page], reg, mask, value);
+}
+
 /* adv748x_write_block(): Write raw data with a maximum of I2C_SMBUS_BLOCK_MAX
  * size to one or more registers.
  *
drivers/media/i2c/adv748x/adv748x.h +11 -3
index 09aab4138c3f..0a9d78c2870b 100644
@@ -393,25 +393,33 @@ int adv748x_write(struct adv748x_state *state, u8 page, u8 reg, u8 value);
 int adv748x_write_block(struct adv748x_state *state, int client_page,
 			unsigned int init_reg, const void *val,
 			size_t val_len);
+int adv748x_update_bits(struct adv748x_state *state, u8 page, u8 reg,
+			u8 mask, u8 value);
 
 #define io_read(s, r) adv748x_read(s, ADV748X_PAGE_IO, r)
 #define io_write(s, r, v) adv748x_write(s, ADV748X_PAGE_IO, r, v)
-#define io_clrset(s, r, m, v) io_write(s, r, (io_read(s, r) & ~(m)) | (v))
+#define io_clrset(s, r, m, v) adv748x_update_bits(s, ADV748X_PAGE_IO, r, m, v)
 
 #define hdmi_read(s, r) adv748x_read(s, ADV748X_PAGE_HDMI, r)
 #define hdmi_read16(s, r, m) (((hdmi_read(s, r) << 8) | hdmi_read(s, (r)+1)) & (m))
 #define hdmi_write(s, r, v) adv748x_write(s, ADV748X_PAGE_HDMI, r, v)
+#define hdmi_clrset(s, r, m, v) \
+	adv748x_update_bits(s, ADV748X_PAGE_HDMI, r, m, v)
+
+#define dpll_read(s, r) adv748x_read(s, ADV748X_PAGE_DPLL, r)
+#define dpll_clrset(s, r, m, v) \
+	adv748x_update_bits(s, ADV748X_PAGE_DPLL, r, m, v)
 
 #define repeater_read(s, r) adv748x_read(s, ADV748X_PAGE_REPEATER, r)
 #define repeater_write(s, r, v) adv748x_write(s, ADV748X_PAGE_REPEATER, r, v)
 
 #define sdp_read(s, r) adv748x_read(s, ADV748X_PAGE_SDP, r)
 #define sdp_write(s, r, v) adv748x_write(s, ADV748X_PAGE_SDP, r, v)
-#define sdp_clrset(s, r, m, v) sdp_write(s, r, (sdp_read(s, r) & ~(m)) | (v))
+#define sdp_clrset(s, r, m, v) adv748x_update_bits(s, ADV748X_PAGE_SDP, r, m, v)
 
 #define cp_read(s, r) adv748x_read(s, ADV748X_PAGE_CP, r)
 #define cp_write(s, r, v) adv748x_write(s, ADV748X_PAGE_CP, r, v)
-#define cp_clrset(s, r, m, v) cp_write(s, r, (cp_read(s, r) & ~(m)) | (v))
+#define cp_clrset(s, r, m, v) adv748x_update_bits(s, ADV748X_PAGE_CP, r, m, v)
 
 #define tx_read(t, r) adv748x_read(t->state, t->page, r)
 #define tx_write(t, r, v) adv748x_write(t->state, t->page, r, v)
Signature · 1 line
2.25.1.25.g9ecbe7eb18
#4 lore raw

PATCH v4 4/9 media: adv748x: add definitions for audio output related registers

In reply to Alex Riesen
---
 drivers/media/i2c/adv748x/adv748x.h | 32 +++++++++++++++++++++++++++++
 1 file changed, 32 insertions(+)
drivers/media/i2c/adv748x/adv748x.h +32 -0
index 0a9d78c2870b..1a1ea70086c6 100644
@@ -226,6 +226,11 @@ struct adv748x_state {
 
 #define ADV748X_IO_VID_STD		0x05
 
+#define ADV748X_IO_PAD_CONTROLS		0x0e
+#define ADV748X_IO_PAD_CONTROLS_TRI_AUD	BIT(5)
+#define ADV748X_IO_PAD_CONTROLS_PDN_AUD	BIT(1)
+#define ADV748X_IO_PAD_CONTROLS1	0x1d
+
 #define ADV748X_IO_10			0x10	/* io_reg_10 */
 #define ADV748X_IO_10_CSI4_EN		BIT(7)
 #define ADV748X_IO_10_CSI1_EN		BIT(6)
@@ -248,7 +253,21 @@ struct adv748x_state {
 #define ADV748X_IO_REG_FF		0xff
 #define ADV748X_IO_REG_FF_MAIN_RESET	0xff
 
+/* DPLL Map */
+#define ADV748X_DPLL_MCLK_FS		0xb5
+#define ADV748X_DPLL_MCLK_FS_N_MASK	GENMASK(2, 0)
+
 /* HDMI RX Map */
+#define ADV748X_HDMI_I2S		0x03	/* I2S mode and width */
+#define ADV748X_HDMI_I2SBITWIDTH_MASK	GENMASK(4, 0)
+#define ADV748X_HDMI_I2SOUTMODE_SHIFT	5
+#define ADV748X_HDMI_I2SOUTMODE_MASK	\
+	GENMASK(6, ADV748X_HDMI_I2SOUTMODE_SHIFT)
+#define ADV748X_I2SOUTMODE_I2S 0
+#define ADV748X_I2SOUTMODE_RIGHT_J 1
+#define ADV748X_I2SOUTMODE_LEFT_J 2
+#define ADV748X_I2SOUTMODE_SPDIF 3
+
 #define ADV748X_HDMI_LW1		0x07	/* line width_1 */
 #define ADV748X_HDMI_LW1_VERT_FILTER	BIT(7)
 #define ADV748X_HDMI_LW1_DE_REGEN	BIT(5)
@@ -260,6 +279,16 @@ struct adv748x_state {
 #define ADV748X_HDMI_F1H1		0x0b	/* field1 height_1 */
 #define ADV748X_HDMI_F1H1_INTERLACED	BIT(5)
 
+#define ADV748X_HDMI_MUTE_CTRL		0x1a
+#define ADV748X_HDMI_MUTE_CTRL_MUTE_AUDIO BIT(4)
+#define ADV748X_HDMI_MUTE_CTRL_WAIT_UNMUTE_MASK	GENMASK(3, 1)
+#define ADV748X_HDMI_MUTE_CTRL_NOT_AUTO_UNMUTE	BIT(0)
+
+#define ADV748X_HDMI_AUDIO_MUTE_SPEED	0x0f
+#define ADV748X_HDMI_AUDIO_MUTE_SPEED_MASK	GENMASK(4, 0)
+#define ADV748X_MAN_AUDIO_DL_BYPASS BIT(7)
+#define ADV748X_AUDIO_DELAY_LINE_BYPASS BIT(6)
+
 #define ADV748X_HDMI_HFRONT_PORCH	0x20	/* hsync_front_porch_1 */
 #define ADV748X_HDMI_HFRONT_PORCH_MASK	0x1fff
 
@@ -281,6 +310,9 @@ struct adv748x_state {
 #define ADV748X_HDMI_TMDS_1		0x51	/* hdmi_reg_51 */
 #define ADV748X_HDMI_TMDS_2		0x52	/* hdmi_reg_52 */
 
+#define ADV748X_HDMI_REG_6D		0x6d	/* hdmi_reg_6d */
+#define ADV748X_I2S_TDM_MODE_ENABLE BIT(7)
+
 /* HDMI RX Repeater Map */
 #define ADV748X_REPEATER_EDID_SZ	0x70	/* primary_edid_size */
 #define ADV748X_REPEATER_EDID_SZ_SHIFT	4
Signature · 1 line
2.25.1.25.g9ecbe7eb18
#5 lore raw

PATCH v4 5/9 media: adv748x: add support for HDMI audio

In reply to Alex Riesen

This adds an implemention of SoC DAI driver which provides access to the I2S port of the device.

--

v3: fix clock registration in case of multiple adv748x devices

v4: use clk_hw instead of clk

v4: use const for capture snd_soc_pcm_stream instance

---
 drivers/media/i2c/adv748x/Makefile       |   3 +-
 drivers/media/i2c/adv748x/adv748x-core.c |   9 +-
 drivers/media/i2c/adv748x/adv748x-dai.c  | 261 +++++++++++++++++++++++
 drivers/media/i2c/adv748x/adv748x.h      |  17 +-
 4 files changed, 287 insertions(+), 3 deletions(-)
 create mode 100644 drivers/media/i2c/adv748x/adv748x-dai.c
drivers/media/i2c/adv748x/Makefile +2 -1
index 93844f14cb10..6e7a302ef199 100644
@@ -3,6 +3,7 @@ adv748x-objs	:= \
 		adv748x-afe.o \
 		adv748x-core.o \
 		adv748x-csi2.o \
-		adv748x-hdmi.o
+		adv748x-hdmi.o \
+		adv748x-dai.o
 
 obj-$(CONFIG_VIDEO_ADV748X)	+= adv748x.o
drivers/media/i2c/adv748x/adv748x-core.c +8 -1
index 8580e6624276..3513ca138e53 100644
@@ -765,8 +765,14 @@ static int adv748x_probe(struct i2c_client *client)
 		goto err_cleanup_txa;
 	}
 
+	ret = adv748x_dai_init(&state->dai);
+	if (ret) {
+		adv_err(state, "Failed to probe DAI\n");
+		goto err_cleanup_txb;
+	}
 	return 0;
-
+err_cleanup_txb:
+	adv748x_csi2_cleanup(&state->txb);
 err_cleanup_txa:
 	adv748x_csi2_cleanup(&state->txa);
 err_cleanup_afe:
@@ -787,6 +793,7 @@ static int adv748x_remove(struct i2c_client *client)
 {
 	struct adv748x_state *state = i2c_get_clientdata(client);
 
+	adv748x_dai_cleanup(&state->dai);
 	adv748x_afe_cleanup(&state->afe);
 	adv748x_hdmi_cleanup(&state->hdmi);
 
drivers/media/i2c/adv748x/adv748x-dai.c new file +261 -0

261 lines not on this page · raw message

drivers/media/i2c/adv748x/adv748x.h +16 -1
index 1a1ea70086c6..454f97ff7b54 100644
@@ -19,6 +19,7 @@
  */
 
 #include <linux/i2c.h>
+#include <sound/soc.h>
 #include <media/v4l2-ctrls.h>
 #include <media/v4l2-device.h>
 
@@ -63,7 +64,8 @@ enum adv748x_ports {
 	ADV748X_PORT_TTL = 9,
 	ADV748X_PORT_TXA = 10,
 	ADV748X_PORT_TXB = 11,
-	ADV748X_PORT_MAX = 12,
+	ADV748X_PORT_I2S = 12,
+	ADV748X_PORT_MAX = 13,
 };
 
 enum adv748x_csi2_pads {
@@ -166,6 +168,13 @@ struct adv748x_afe {
 	container_of(ctrl->handler, struct adv748x_afe, ctrl_hdl)
 #define adv748x_sd_to_afe(sd) container_of(sd, struct adv748x_afe, sd)
 
+struct adv748x_dai {
+	struct snd_soc_dai_driver drv;
+	struct clk_hw *mclk_hw;
+	char *mclk_name;
+	unsigned int freq, fmt, tdm;
+};
+
 /**
  * struct adv748x_state - State of ADV748X
  * @dev:		(OF) device
@@ -182,6 +191,7 @@ struct adv748x_afe {
  * @afe:		state of AFE receiver context
  * @txa:		state of TXA transmitter context
  * @txb:		state of TXB transmitter context
+ * @mclk:		MCLK clock of the I2S port
  */
 struct adv748x_state {
 	struct device *dev;
@@ -197,10 +207,12 @@ struct adv748x_state {
 	struct adv748x_afe afe;
 	struct adv748x_csi2 txa;
 	struct adv748x_csi2 txb;
+	struct adv748x_dai dai;
 };
 
 #define adv748x_hdmi_to_state(h) container_of(h, struct adv748x_state, hdmi)
 #define adv748x_afe_to_state(a) container_of(a, struct adv748x_state, afe)
+#define adv748x_dai_to_state(p) container_of(p, struct adv748x_state, dai)
 
 #define adv_err(a, fmt, arg...)	dev_err(a->dev, fmt, ##arg)
 #define adv_info(a, fmt, arg...) dev_info(a->dev, fmt, ##arg)
@@ -484,4 +496,7 @@ int adv748x_csi2_set_pixelrate(struct v4l2_subdev *sd, s64 rate);
 int adv748x_hdmi_init(struct adv748x_hdmi *hdmi);
 void adv748x_hdmi_cleanup(struct adv748x_hdmi *hdmi);
 
+int adv748x_dai_init(struct adv748x_dai *);
+void adv748x_dai_cleanup(struct adv748x_dai *);
+
 #endif /* _ADV748X_H_ */
Signature · 1 line
2.25.1.25.g9ecbe7eb18
#6 lore raw

PATCH v4 6/9 media: adv748x: prepare/enable mclk when the audio is used

In reply to Alex Riesen

As there is nothing else (the consumers are supposed to do that) which enables the clock, do it in the driver.

--

v3: added

---
 drivers/media/i2c/adv748x/adv748x-dai.c | 14 +++++++++++++-
 1 file changed, 13 insertions(+), 1 deletion(-)
drivers/media/i2c/adv748x/adv748x-dai.c +13 -1
index c9191f8f1ca8..185f78023e91 100644
@@ -117,11 +117,22 @@ static int adv748x_dai_set_fmt(struct snd_soc_dai *dai, unsigned int fmt)
 
 static int adv748x_dai_startup(struct snd_pcm_substream *sub, struct snd_soc_dai *dai)
 {
+	int ret;
 	struct adv748x_state *state = state_of(dai);
 
 	if (sub->stream != SNDRV_PCM_STREAM_CAPTURE)
 		return -EINVAL;
-	return set_audio_pads_state(state, 1);
+	ret = set_audio_pads_state(state, 1);
+	if (ret)
+		goto fail;
+	ret = clk_prepare_enable(mclk_of(state));
+	if (ret)
+		goto fail_pwdn;
+	return 0;
+fail_pwdn:
+	set_audio_pads_state(state, 0);
+fail:
+	return ret;
 }
 
 static int adv748x_dai_hw_params(struct snd_pcm_substream *sub,
@@ -174,6 +185,7 @@ static void adv748x_dai_shutdown(struct snd_pcm_substream *sub, struct snd_soc_d
 {
 	struct adv748x_state *state = state_of(dai);
 
+	clk_disable_unprepare(mclk_of(state));
 	set_audio_pads_state(state, 0);
 }
 
Signature · 1 line
2.25.1.25.g9ecbe7eb18
#7 lore raw

PATCH v4 7/9 media: adv748x: only activate DAI if it is described in device tree

In reply to Alex Riesen

To avoid setting it up even if the hardware is not actually connected to anything physically.

Besides, the bindings explicitly notes that port definitions are "optional if they are not connected to anything at the hardware level".

---
 drivers/media/i2c/adv748x/adv748x-dai.c | 5 +++++
 1 file changed, 5 insertions(+)
drivers/media/i2c/adv748x/adv748x-dai.c +5 -0
index 185f78023e91..f9cc47fa9ad1 100644
@@ -216,6 +216,11 @@ int adv748x_dai_init(struct adv748x_dai *dai)
 	int ret;
 	struct adv748x_state *state = adv748x_dai_to_state(dai);
 
+	if (!state->endpoints[ADV748X_PORT_I2S]) {
+		adv_info(state, "no I2S port, DAI disabled\n");
+		ret = 0;
+		goto fail;
+	}
 	dai->mclk_name = kasprintf(GFP_KERNEL, "%s.%s-i2s-mclk",
 				   state->dev->driver->name,
 				   dev_name(state->dev));
Signature · 1 line
2.25.1.25.g9ecbe7eb18
#8 lore raw

PATCH v4 8/9 dt-bindings: adv748x: add information about serial audio interface (I2S/TDM)

In reply to Alex Riesen

As the driver has some support for the audio interface of the device, the bindings file should mention it.

--

v3: remove optionality off MCLK clock cell to ensure the description
matches the hardware no matter if the line is connected.

---
 .../devicetree/bindings/media/i2c/adv748x.txt    | 16 +++++++++++++++-
 1 file changed, 15 insertions(+), 1 deletion(-)
Documentation/devicetree/bindings/media/i2c/adv748x.txt +15 -1
index 4f91686e54a6..50a753189b81 100644
@@ -2,7 +2,9 @@
 
 The ADV7481 and ADV7482 are multi format video decoders with an integrated
 HDMI receiver. They can output CSI-2 on two independent outputs TXA and TXB
-from three input sources HDMI, analog and TTL.
+from three input sources HDMI, analog and TTL. There is also support for an
+I2S-compatible interface connected to the audio processor of the HDMI decoder.
+The interface has TDM capability (8 slots, 32 bits, left or right justified).
 
 Required Properties:
 
@@ -16,6 +18,8 @@ Required Properties:
     slave device on the I2C bus. The main address is mandatory, others are
     optional and remain at default values if not specified.
 
+  - #clock-cells: must be <0>
+
 Optional Properties:
 
   - interrupt-names: Should specify the interrupts as "intrq1", "intrq2" and/or
@@ -47,6 +51,7 @@ are numbered as follows.
 	  TTL		sink		9
 	  TXA		source		10
 	  TXB		source		11
+	  I2S		source		12
 
 The digital output port nodes, when present, shall contain at least one
 endpoint. Each of those endpoints shall contain the data-lanes property as
@@ -72,6 +77,7 @@ Example:
 
 		#address-cells = <1>;
 		#size-cells = <0>;
+		#clock-cells = <0>;
 
 		interrupt-parent = <&gpio6>;
 		interrupt-names = "intrq1", "intrq2";
@@ -113,4 +119,12 @@ Example:
 				remote-endpoint = <&csi20_in>;
 			};
 		};
+
+		port@c {
+			reg = <12>;
+
+			adv7482_i2s: endpoint {
+				remote-endpoint = <&i2s_in>;
+			};
+		};
 	};
Signature · 1 line
2.25.1.25.g9ecbe7eb18
#9 lore raw

PATCH v4 9/9 arm64: dts: renesas: salvator: add a connection from adv748x codec (HDMI input) to the R-Car SoC

In reply to Alex Riesen

As all known variants of the Salvator board have the HDMI decoder chip (the ADV7482) connected to the SSI4 on R-Car SoC, the ADV7482 endpoint and the connection definitions are placed in the common board file.

For the same reason, the CLK_C clock line and I2C configuration (similar to the ak4613, on the same interface) are added into the common file.

--

v2: Also add ssi4_ctrl pin group in the sound pins. The pins are
responsible for SCK4 (sample clock) WS4 and (word boundary input), and are required for SSI audio input over I2S.

The adv748x shall provide its own implementation of the output clock (MCLK), connected to the audio_clk_c line of the R-Car SoC.

If the frequency of the ADV748x MCLK were fixed, the clock implementation were not necessary, but it does not seem so: the MCLK depends on the value in a speed multiplier register and the input sample rate (48kHz).

Remove audio clock C from the clocks of adv7482.

The clocks property of the video-receiver node lists the input clocks of the device, which is quite the opposite from the original intention: the adv7482 on Salvator X boards is a provide of the MCLK clock for I2S audio output.

Remove old definition of &sound_card.dais and reduce size of changes in the Salvator-X specific device tree source.

Declare video-receiver a clock producer, as the adv748x driver implements the master clock used I2S audio output.

v2: The driver provides only MCLK clock, not the SCLK and LRCLK,
which are part of the I2S protocol.

---
 .../boot/dts/renesas/r8a77950-salvator-x.dts  |  3 +-
 .../boot/dts/renesas/salvator-common.dtsi     | 47 +++++++++++++++++--
 2 files changed, 44 insertions(+), 6 deletions(-)
arch/arm64/boot/dts/renesas/r8a77950-salvator-x.dts +2 -1
index 2438825c9b22..e16c146808b6 100644
@@ -146,7 +146,8 @@ &sata {
 &sound_card {
 	dais = <&rsnd_port0	/* ak4613 */
 		&rsnd_port1	/* HDMI0  */
-		&rsnd_port2>;	/* HDMI1  */
+		&rsnd_port2	/* HDMI1  */
+		&rsnd_port3>;	/* adv7482 hdmi-in  */
 };
 
 &usb2_phy2 {
arch/arm64/boot/dts/renesas/salvator-common.dtsi +42 -5

90 lines not on this page · raw message

Signature · 1 line
2.25.1.25.g9ecbe7eb18
#10 lore raw

Re: [PATCH v4 9/9] arm64: dts: renesas: salvator: add a connection from adv748x codec (HDMI input) to the R-Car SoC

Hi Alex,

Quoted 9 lines · Alex Riesen

As all known variants of the Salvator board have the HDMI decoder chip (the ADV7482) connected to the SSI4 on R-Car SoC, the ADV7482 endpoint and the connection definitions are placed in the common board file.

For the same reason, the CLK_C clock line and I2C configuration (similar to the ak4613, on the same interface) are added into the common file.

Thanks for your patch!

Quoted 22 lines
arch/arm64/boot/dts/renesas/salvator-common.dtsi +2 -1
@@ -460,7 +460,7 @@ pca9654: gpio@20 {
                #gpio-cells = <2>;
        };
 
-       video-receiver@70 {
+       adv7482_hdmi_in: video-receiver@70 {
                compatible = "adi,adv7482";
                reg = <0x70 0x71 0x72 0x73 0x74 0x75
                       0x60 0x61 0x62 0x63 0x64 0x65>;
@@ -469,6 +469,7 @@ video-receiver@70 {
 
                #address-cells = <1>;
                #size-cells = <0>;
+               #clock-cells = <0>; /* the MCLK for I2S output */
 
                interrupt-parent = <&gpio6>;
                interrupt-names = "intrq1", "intrq2";
@@ -510,6 +511,15 @@ adv7482_txb: endpoint {
                                remote-endpoint = <&csi20_in>;
                        };

arch/arm64/boot/dts/renesas/salvator-common.dtsi adv7482_txb: endpoint { · line 522

Quoted 10 lines
arch/arm64/boot/dts/renesas/salvator-common.dtsi +9 -0
                };
+
+               port@c {
+                       reg = <12>;
+
+                       adv7482_i2s: endpoint {
+                               remote-endpoint = <&rsnd_endpoint3>;
+                               system-clock-direction-out;
+                       };
+               };

As the adv748x driver just ignores "invalid" endpoints...

Quoted 12 lines
arch/arm64/boot/dts/renesas/salvator-common.dtsi +2 -2
@@ -733,8 +744,8 @@ &rcar_sound {
        pinctrl-0 = <&sound_pins &sound_clk_pins>;
        pinctrl-names = "default";
 
-       /* Single DAI */
-       #sound-dai-cells = <0>;
+       /* multi DAI */
+       #sound-dai-cells = <1>;
 
        /* audio_clkout0/1/2/3 */
        #clock-cells = <1>;

arch/arm64/boot/dts/renesas/salvator-common.dtsi &rcar_sound { · line 773

Quoted 6 lines
arch/arm64/boot/dts/renesas/salvator-common.dtsi +1 -1
@@ -758,8 +769,19 @@ &rcar_sound {
                 <&cpg CPG_MOD 1020>, <&cpg CPG_MOD 1021>,
                 <&cpg CPG_MOD 1019>, <&cpg CPG_MOD 1018>,
                 <&audio_clk_a>, <&cs2000>,
-                <&audio_clk_c>,
+                <&adv7482_hdmi_in>,
                 <&cpg CPG_CORE CPG_AUDIO_CLK_I>;

... and the rsnd driver ignores nonexistent-clocks, the DT change has no hard dependency on the driver change, and won't introduce regressions when included, right?

Quoted 11 lines
arch/arm64/boot/dts/renesas/salvator-common.dtsi +7 -0
@@ -777,6 +799,21 @@ rsnd_endpoint0: endpoint {
                                capture  = <&ssi1 &src1 &dvc1>;
                        };
                };
+               rsnd_port3: port@3 {
+                       reg = <3>;
+                       rsnd_endpoint3: endpoint {
+                               remote-endpoint = <&adv7482_i2s>;
+
+                               dai-tdm-slot-num = <8>;
+                               dai-tdm-slot-width = <32>;

arch/arm64/boot/dts/renesas/salvator-common.dtsi rsnd_endpoint0: endpoint { · line 818

Quoted 10 lines
arch/arm64/boot/dts/renesas/salvator-common.dtsi +8 -0
+                               dai-format = "left_j";
+                               mclk-fs = <256>;
+                               bitclock-master = <&adv7482_i2s>;
+                               frame-master = <&adv7482_i2s>;
+
+                               capture = <&ssi4>;
+                       };
+               };
        };
 };

However, as salvator-common.dtsi is shared by all Salvator-X(S) variants, you'll have to add a dummy ssi4 node to r8a77961.dtsi first.

Gr{oetje,eeting}s,

Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- [email protected]

In personal conversations with technical people, I call myself a hacker. But when I'm talking to journalists I just say "programmer" or something like that.
-- Linus Torvalds

#11 lore raw

Re: [PATCH v4 9/9] arm64: dts: renesas: salvator: add a connection from adv748x codec (HDMI input) to the R-Car SoC

Hi Geert,

I'm sorry for late reply. Some unrelated happenings here in south Germany.

Geert Uytterhoeven, Mon, Mar 30, 2020 10:32:47 +0200:

Quoted 19 lines · Alex Riesen
Quoted 5 lines
arch/arm64/boot/dts/renesas/salvator-common.dtsi +0 -0
@@ -510,6 +511,15 @@ adv7482_txb: endpoint {
                                remote-endpoint = <&csi20_in>;
                        };

arch/arm64/boot/dts/renesas/salvator-common.dtsi adv7482_txb: endpoint { · line 522

Quoted 10 lines
arch/arm64/boot/dts/renesas/salvator-common.dtsi +9 -0
                };
+
+               port@c {
+                       reg = <12>;
+
+                       adv7482_i2s: endpoint {
+                               remote-endpoint = <&rsnd_endpoint3>;
+                               system-clock-direction-out;
+                       };
+               };

As the adv748x driver just ignores "invalid" endpoints...

Quoted 11 lines

arch/arm64/boot/dts/renesas/salvator-common.dtsi &rcar_sound { · line 773

Quoted 7 lines
arch/arm64/boot/dts/renesas/salvator-common.dtsi +1 -1
@@ -758,8 +769,19 @@ &rcar_sound {
                 <&cpg CPG_MOD 1020>, <&cpg CPG_MOD 1021>,
                 <&cpg CPG_MOD 1019>, <&cpg CPG_MOD 1018>,
                 <&audio_clk_a>, <&cs2000>,
-                <&audio_clk_c>,
+                <&adv7482_hdmi_in>,
                 <&cpg CPG_CORE CPG_AUDIO_CLK_I>;

... and the rsnd driver ignores nonexistent-clocks, the DT change has no hard dependency on the driver change, and won't introduce regressions when included, right?

Well, it maybe won't, but isn't it a little ... implicit?
And I'm no haste to include the changes, if you mean I can (or should) submit the device tree patch separately.

Quoted 16 lines
Quoted 16 lines
arch/arm64/boot/dts/renesas/salvator-common.dtsi +12 -0
@@ -777,6 +799,21 @@ rsnd_endpoint0: endpoint {
                                capture  = <&ssi1 &src1 &dvc1>;
                        };
                };
+               rsnd_port3: port@3 {
+                       reg = <3>;
+                       rsnd_endpoint3: endpoint {
+                               remote-endpoint = <&adv7482_i2s>;
+
+                               dai-tdm-slot-num = <8>;
+                               dai-tdm-slot-width = <32>;
+                               dai-format = "left_j";
+                               mclk-fs = <256>;
+                               bitclock-master = <&adv7482_i2s>;
+                               frame-master = <&adv7482_i2s>;
+
Quoted 8 lines

arch/arm64/boot/dts/renesas/salvator-common.dtsi rsnd_endpoint0: endpoint { · line 818

Quoted 5 lines
arch/arm64/boot/dts/renesas/salvator-common.dtsi +3 -0
+                               capture = <&ssi4>;
+                       };
+               };
        };
 };

However, as salvator-common.dtsi is shared by all Salvator-X(S) variants, you'll have to add a dummy ssi4 node to r8a77961.dtsi first.

I see. There are even two dummy SSI nodes already. I would prefer to submit the change together with other Salvator device tree changes. Is that alright?

Regards,
Alex

#12 lore raw

Re: [PATCH v4 9/9] arm64: dts: renesas: salvator: add a connection from adv748x codec (HDMI input) to the R-Car SoC

Hi Alex,

Quoted 32 lines · Alex Riesen

Geert Uytterhoeven, Mon, Mar 30, 2020 10:32:47 +0200:

Quoted 19 lines · Alex Riesen
Quoted 5 lines
arch/arm64/boot/dts/renesas/salvator-common.dtsi +0 -0
@@ -510,6 +511,15 @@ adv7482_txb: endpoint {
                                remote-endpoint = <&csi20_in>;
                        };

arch/arm64/boot/dts/renesas/salvator-common.dtsi adv7482_txb: endpoint { · line 522

Quoted 10 lines
arch/arm64/boot/dts/renesas/salvator-common.dtsi +9 -0
                };
+
+               port@c {
+                       reg = <12>;
+
+                       adv7482_i2s: endpoint {
+                               remote-endpoint = <&rsnd_endpoint3>;
+                               system-clock-direction-out;
+                       };
+               };

As the adv748x driver just ignores "invalid" endpoints...

Quoted 11 lines

arch/arm64/boot/dts/renesas/salvator-common.dtsi &rcar_sound { · line 773

Quoted 7 lines
arch/arm64/boot/dts/renesas/salvator-common.dtsi +1 -1
@@ -758,8 +769,19 @@ &rcar_sound {
                 <&cpg CPG_MOD 1020>, <&cpg CPG_MOD 1021>,
                 <&cpg CPG_MOD 1019>, <&cpg CPG_MOD 1018>,
                 <&audio_clk_a>, <&cs2000>,
-                <&audio_clk_c>,
+                <&adv7482_hdmi_in>,
                 <&cpg CPG_CORE CPG_AUDIO_CLK_I>;

... and the rsnd driver ignores nonexistent-clocks, the DT change has no hard dependency on the driver change, and won't introduce regressions when included, right?

Quoted 3 lines

Well, it maybe won't, but isn't it a little ... implicit?
And I'm no haste to include the changes, if you mean I can (or should) submit the device tree patch separately.

OK, fine for me to postpone (that'll be for v5.9, I guess?).

Quoted 25 lines
Quoted 16 lines
Quoted 16 lines
arch/arm64/boot/dts/renesas/salvator-common.dtsi +12 -0
@@ -777,6 +799,21 @@ rsnd_endpoint0: endpoint {
                                capture  = <&ssi1 &src1 &dvc1>;
                        };
                };
+               rsnd_port3: port@3 {
+                       reg = <3>;
+                       rsnd_endpoint3: endpoint {
+                               remote-endpoint = <&adv7482_i2s>;
+
+                               dai-tdm-slot-num = <8>;
+                               dai-tdm-slot-width = <32>;
+                               dai-format = "left_j";
+                               mclk-fs = <256>;
+                               bitclock-master = <&adv7482_i2s>;
+                               frame-master = <&adv7482_i2s>;
+
Quoted 8 lines

arch/arm64/boot/dts/renesas/salvator-common.dtsi rsnd_endpoint0: endpoint { · line 818

Quoted 5 lines
arch/arm64/boot/dts/renesas/salvator-common.dtsi +3 -0
+                               capture = <&ssi4>;
+                       };
+               };
        };
 };

However, as salvator-common.dtsi is shared by all Salvator-X(S) variants, you'll have to add a dummy ssi4 node to r8a77961.dtsi first.

Quoted 2 lines

I see. There are even two dummy SSI nodes already. I would prefer to submit the change together with other Salvator device tree changes. Is that alright?

Fine for me.

Gr{oetje,eeting}s,

Geert

Signature · 5 lines
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- [email protected]

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
                                -- Linus Torvalds
#13 lore raw

Re: [PATCH v4 9/9] arm64: dts: renesas: salvator: add a connection from adv748x codec (HDMI input) to the R-Car SoC

Geert Uytterhoeven, Thu, Apr 02, 2020 17:26:15 +0200:

Quoted 33 lines · Alex Riesen
Quoted 32 lines

Geert Uytterhoeven, Mon, Mar 30, 2020 10:32:47 +0200:

Quoted 30 lines · Alex Riesen
Quoted 5 lines
arch/arm64/boot/dts/renesas/salvator-common.dtsi +0 -0
@@ -510,6 +511,15 @@ adv7482_txb: endpoint {
                                remote-endpoint = <&csi20_in>;
                        };

arch/arm64/boot/dts/renesas/salvator-common.dtsi adv7482_txb: endpoint { · line 522

Quoted 10 lines
arch/arm64/boot/dts/renesas/salvator-common.dtsi +9 -0
                };
+
+               port@c {
+                       reg = <12>;
+
+                       adv7482_i2s: endpoint {
+                               remote-endpoint = <&rsnd_endpoint3>;
+                               system-clock-direction-out;
+                       };
+               };

As the adv748x driver just ignores "invalid" endpoints...

arch/arm64/boot/dts/renesas/salvator-common.dtsi &rcar_sound { · line 773

Quoted 7 lines
arch/arm64/boot/dts/renesas/salvator-common.dtsi +1 -1
@@ -758,8 +769,19 @@ &rcar_sound {
                 <&cpg CPG_MOD 1020>, <&cpg CPG_MOD 1021>,
                 <&cpg CPG_MOD 1019>, <&cpg CPG_MOD 1018>,
                 <&audio_clk_a>, <&cs2000>,
-                <&audio_clk_c>,
+                <&adv7482_hdmi_in>,
                 <&cpg CPG_CORE CPG_AUDIO_CLK_I>;

... and the rsnd driver ignores nonexistent-clocks, the DT change has no hard dependency on the driver change, and won't introduce regressions when included, right?

Quoted 6 lines
Quoted 3 lines

Well, it maybe won't, but isn't it a little ... implicit?
And I'm no haste to include the changes, if you mean I can (or should) submit the device tree patch separately.

OK, fine for me to postpone (that'll be for v5.9, I guess?).

Looks scary :)
But yes, fine with me too.

v5 with ssi4 dummy in a moment.

Regards,
Alex