Skip to contentSkip to conversation

NVIDIA Tegra Video Decoder driver improvements

PATCH Superseded Dmitry Osipenko 3 participants 11 messages Jun 18, 2019

Subsystems (5)

v2 of 3 Superseded Latest is v3 →

lore
Share & tools
lore
#0 lore raw

v2 cover NVIDIA Tegra Video Decoder driver improvements

Hello,

This series cleans up some of minor shortcomings that are caused by checkpatch recommendations that are not very applicable for the driver. Then IOMMU support is added to the driver and now it can handle sparse memory buffers that GPU hands to VDE in a default kernel configuration on Tegra30+.

Changelog:

v2: - Fixed use-after-free bug and uninitialized-variable warning that
were reported by smatch and pointed out by Hans Verkuil in the review comment to v1.

- Fixed build failure when CONFIG_ARM_DMA_USE_IOMMU=y and tested this case properly (multi-platform kernel config).

- Made some extra minor changes, prettifying code a tad more.

Dmitry Osipenko (4):
staging: media: tegra-vde: Remove BIT() macro from UAPI header
staging: media: tegra-vde: Manually pack UAPI structures
staging: media: tegra-vde: Add IOMMU support
staging: media: tegra-vde: Defer dmabuf's unmapping

 drivers/staging/media/tegra-vde/Kconfig       |   1 +
 drivers/staging/media/tegra-vde/Makefile      |   1 +
 .../staging/media/tegra-vde/dmabuf-cache.c    | 226 ++++++++++++++++++
 drivers/staging/media/tegra-vde/iommu.c       | 149 ++++++++++++
 drivers/staging/media/tegra-vde/trace.h       |   2 +
 drivers/staging/media/tegra-vde/uapi.h        |  48 ++--
 .../media/tegra-vde/{tegra-vde.c => vde.c}    | 212 ++++++----------
 drivers/staging/media/tegra-vde/vde.h         | 105 ++++++++
 8 files changed, 581 insertions(+), 163 deletions(-)
 create mode 100644 drivers/staging/media/tegra-vde/dmabuf-cache.c
 create mode 100644 drivers/staging/media/tegra-vde/iommu.c
 rename drivers/staging/media/tegra-vde/{tegra-vde.c => vde.c} (88%)
 create mode 100644 drivers/staging/media/tegra-vde/vde.h
Signature · 1 line
2.22.0
#1 lore raw

PATCH v2 1/1 ARM: dts: tegra30: Connect SMMU with Video Decoder Engine

Enable IOMMU support for the video decoder.

---
 arch/arm/boot/dts/tegra30.dtsi | 1 +
 1 file changed, 1 insertion(+)
arch/arm/boot/dts/tegra30.dtsi +1 -0
index 934caa83c8db..ce162125e7bf 100644
@@ -424,6 +424,7 @@
 		clocks = <&tegra_car TEGRA30_CLK_VDE>;
 		reset-names = "vde", "mc";
 		resets = <&tegra_car 61>, <&mc TEGRA30_MC_RESET_VDE>;
+		iommus = <&mc TEGRA_SWGROUP_VDE>;
 	};
 
 	apbmisc@70000800 {
Signature · 1 line
2.22.0
#2 lore raw

PATCH v2 1/1 ARM: tegra: Enable Tegra VDE driver in tegra_defconfig

In reply to Dmitry Osipenko

The video decoder driver was tested by time and works absolutely fine. The reason why it is in staging is because it doesn't provide common V4L interface yet, this shouldn't stop driver enabling in the defconfig since our userspace (libvdpau-tegra) provides combined acceleration of decoding and displaying without use of V4L.

---
 arch/arm/configs/tegra_defconfig | 2 ++
 1 file changed, 2 insertions(+)
arch/arm/configs/tegra_defconfig +2 -0
index 8f5c6a5b444c..a27592d3b1fa 100644
@@ -250,6 +250,8 @@ CONFIG_KEYBOARD_NVEC=y
 CONFIG_SERIO_NVEC_PS2=y
 CONFIG_NVEC_POWER=y
 CONFIG_NVEC_PAZ00=y
+CONFIG_STAGING_MEDIA=y
+CONFIG_TEGRA_VDE=y
 CONFIG_TEGRA_IOMMU_GART=y
 CONFIG_TEGRA_IOMMU_SMMU=y
 CONFIG_ARCH_TEGRA_2x_SOC=y
Signature · 1 line
2.22.0
#3 lore raw

PATCH v2 1/1 media: dt: bindings: tegra-vde: Document new optional IOMMU property

In reply to Dmitry Osipenko

All NVIDIA Tegra SoC generations provide IOMMU support for the video decoder engine. Document new optional device-tree property that connects VDE with the IOMMU provider.

---
 Documentation/devicetree/bindings/media/nvidia,tegra-vde.txt | 2 ++
 1 file changed, 2 insertions(+)
Documentation/devicetree/bindings/media/nvidia,tegra-vde.txt +2 -0
index 7302e949e662..602169b8aa19 100644
@@ -35,6 +35,7 @@ Optional properties:
 - resets : Must contain an entry for each entry in reset-names.
 - reset-names : Must include the following entries:
   - mc
+- iommus: Must contain phandle to the IOMMU device node.
 
 Example:
 
@@ -59,4 +60,5 @@ video-codec@6001a000 {
 	clocks = <&tegra_car TEGRA20_CLK_VDE>;
 	reset-names = "vde", "mc";
 	resets = <&tegra_car 61>, <&mc TEGRA20_MC_RESET_VDE>;
+	iommus = <&mc TEGRA_SWGROUP_VDE>;
 };
Signature · 1 line
2.22.0
#4 lore raw

PATCH v2 1/4 staging: media: tegra-vde: Remove BIT() macro from UAPI header

In reply to Dmitry Osipenko

The BIT macro isn't available in userspace. Checkpatch complains about shifts being used instead of the macro and people are starting to send patches without realizing that it's a UAPI header file. Hence let's replace the BIT macro with a hex values to make everyone happy.

---
 drivers/staging/media/tegra-vde/uapi.h | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
drivers/staging/media/tegra-vde/uapi.h +2 -2
index a0dad1ed94ef..dd3e4a8c9f7e 100644
@@ -6,8 +6,8 @@
 #include <linux/types.h>
 #include <asm/ioctl.h>
 
-#define FLAG_B_FRAME		BIT(0)
-#define FLAG_REFERENCE		BIT(1)
+#define FLAG_B_FRAME		0x1
+#define FLAG_REFERENCE		0x2
 
 struct tegra_vde_h264_frame {
 	__s32 y_fd;
Signature · 1 line
2.22.0
#5 lore raw

Re: [PATCH v2 1/4] staging: media: tegra-vde: Remove BIT() macro from UAPI header

Quoted 15 lines · Dmitry Osipenko

The BIT macro isn't available in userspace. Checkpatch complains about shifts being used instead of the macro and people are starting to send patches without realizing that it's a UAPI header file. Hence let's replace the BIT macro with a hex values to make everyone happy.

---
 drivers/staging/media/tegra-vde/uapi.h | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
drivers/staging/media/tegra-vde/uapi.h +0 -0
index a0dad1ed94ef..dd3e4a8c9f7e 100644

drivers/staging/media/tegra-vde/uapi.h line 13

Quoted 10 lines
drivers/staging/media/tegra-vde/uapi.h +2 -2
@@ -6,8 +6,8 @@
 #include <linux/types.h>
 #include <asm/ioctl.h>
 
-#define FLAG_B_FRAME		BIT(0)
-#define FLAG_REFERENCE		BIT(1)
+#define FLAG_B_FRAME		0x1
+#define FLAG_REFERENCE		0x2
 
 struct tegra_vde_h264_frame {
 	__s32 y_fd;

The BIT macro is already used quite a bit in uapi headers. I imagine all those need fixup...

$ git grep -P '\bBIT\s*\(' include/uapi
include/uapi/linux/bpf.h:#define BPF_FIB_LOOKUP_DIRECT  BIT(0)
include/uapi/linux/bpf.h:#define BPF_FIB_LOOKUP_OUTPUT  BIT(1)
include/uapi/linux/coresight-stm.h:#define STM_FLAG_TIMESTAMPED   BIT(3)
include/uapi/linux/coresight-stm.h:#define STM_FLAG_GUARANTEED    BIT(7)
include/uapi/linux/nl80211.h: * bitmask of BIT(NL80211_BAND_*) as described in %enum
include/uapi/linux/pkt_sched.h:#define TC_ETF_DEADLINE_MODE_ON  BIT(0)
include/uapi/linux/pkt_sched.h:#define TC_ETF_OFFLOAD_ON        BIT(1)
include/uapi/linux/psci.h:#define PSCI_1_0_OS_INITIATED                 BIT(0)
include/uapi/linux/serio.h:#define SERIO_TIMEOUT        BIT(0)
include/uapi/linux/serio.h:#define SERIO_PARITY BIT(1)
include/uapi/linux/serio.h:#define SERIO_FRAME  BIT(2)
include/uapi/linux/serio.h:#define SERIO_OOB_DATA       BIT(3)
include/uapi/linux/tc_act/tc_ctinfo.h:  CTINFO_MODE_DSCP        = BIT(0),
include/uapi/linux/tc_act/tc_ctinfo.h:  CTINFO_MODE_CPMARK      = BIT(1)
#6 lore raw

Re: [PATCH v2 1/4] staging: media: tegra-vde: Remove BIT() macro from UAPI header

Quoted 4 lines · Joe Perches

include/uapi/linux/nl80211.h: * bitmask of BIT(NL80211_BAND_*) as described in %enum

That's a comment :P

johannes

#7 lore raw

Re: [PATCH v2 1/4] staging: media: tegra-vde: Remove BIT() macro from UAPI header

Quoted 5 lines · Johannes Berg
Quoted 3 lines · Joe Perches

include/uapi/linux/nl80211.h: * bitmask of BIT(NL80211_BAND_*) as described in %enum

That's a comment :P

That's true and that's nice.

It was a casual grep and the comment should
probably be improved anyway.

<insert appropriate emoji here>

cheers, Joe

#8 lore raw

PATCH v2 2/4 staging: media: tegra-vde: Manually pack UAPI structures

In reply to Dmitry Osipenko

The __packed macro isn't available in userspace with the kernel headers. Checkpatch asks to use the macro, which is unwanted in a case of a UAPI header. There is no much benefit in a tight packing of the structures, hence let's pack them manually to cleanup things a tad. Note that there is no old-stable userspace that will suffer from this change, hence it's fine to change the ABI. In a result also more space is reserved for a possible future expansion of the UAPI as it was already shown that more fields will be needed for a later SoC generations.

---
 drivers/staging/media/tegra-vde/tegra-vde.c | 13 +++++-
 drivers/staging/media/tegra-vde/uapi.h      | 44 +++++++++++----------
 2 files changed, 34 insertions(+), 23 deletions(-)
drivers/staging/media/tegra-vde/tegra-vde.c +11 -2
index a5020dbf6eef..cc4244da2705 100644
@@ -795,7 +795,7 @@ static int tegra_vde_ioctl_decode_h264(struct tegra_vde *vde,
 {
 	struct device *dev = vde->miscdev.parent;
 	struct tegra_vde_h264_decoder_ctx ctx;
-	struct tegra_vde_h264_frame frames[17];
+	struct tegra_vde_h264_frame *frames;
 	struct tegra_vde_h264_frame __user *frames_user;
 	struct video_frame *dpb_frames;
 	struct dma_buf_attachment *bitstream_data_dmabuf_attachment;
@@ -830,11 +830,17 @@ static int tegra_vde_ioctl_decode_h264(struct tegra_vde *vde,
 	if (ret)
 		return ret;
 
+	frames = kmalloc_array(ctx.dpb_frames_nb, sizeof(*frames), GFP_KERNEL);
+	if (!frames) {
+		ret = -ENOMEM;
+		goto release_bitstream_dmabuf;
+	}
+
 	dpb_frames = kcalloc(ctx.dpb_frames_nb, sizeof(*dpb_frames),
 			     GFP_KERNEL);
 	if (!dpb_frames) {
 		ret = -ENOMEM;
-		goto release_bitstream_dmabuf;
+		goto free_frames;
 	}
 
 	macroblocks_nb = ctx.pic_width_in_mbs * ctx.pic_height_in_mbs;
@@ -955,6 +961,9 @@ static int tegra_vde_ioctl_decode_h264(struct tegra_vde *vde,
 free_dpb_frames:
 	kfree(dpb_frames);
 
+free_frames:
+	kfree(frames);
+
 release_bitstream_dmabuf:
 	tegra_vde_detach_and_put_dmabuf(bitstream_data_dmabuf_attachment,
 					bitstream_sgt, DMA_TO_DEVICE);
drivers/staging/media/tegra-vde/uapi.h +23 -21

63 lines not on this page · raw message

Signature · 1 line
2.22.0
#9 lore raw

PATCH v2 3/4 staging: media: tegra-vde: Add IOMMU support

In reply to Dmitry Osipenko

All Tegra's could provide memory isolation for the video decoder hardware using IOMMU, it is also required for Tegra30+ in order to handle sparse dmabuf's which GPU exports in a default kernel configuration.

---
 drivers/staging/media/tegra-vde/Kconfig       |   1 +
 drivers/staging/media/tegra-vde/Makefile      |   1 +
 drivers/staging/media/tegra-vde/iommu.c       | 151 ++++++++++++++
 drivers/staging/media/tegra-vde/trace.h       |   2 +
 .../media/tegra-vde/{tegra-vde.c => vde.c}    | 188 +++++++++---------
 drivers/staging/media/tegra-vde/vde.h         |  89 +++++++++
 6 files changed, 339 insertions(+), 93 deletions(-)
 create mode 100644 drivers/staging/media/tegra-vde/iommu.c
 rename drivers/staging/media/tegra-vde/{tegra-vde.c => vde.c} (91%)
 create mode 100644 drivers/staging/media/tegra-vde/vde.h
drivers/staging/media/tegra-vde/Kconfig +1 -0
index ff8e846cd15d..2e7f644ae591 100644
@@ -3,6 +3,7 @@ config TEGRA_VDE
 	tristate "NVIDIA Tegra Video Decoder Engine driver"
 	depends on ARCH_TEGRA || COMPILE_TEST
 	select DMA_SHARED_BUFFER
+	select IOMMU_IOVA if IOMMU_SUPPORT
 	select SRAM
 	help
 	    Say Y here to enable support for the NVIDIA Tegra video decoder
drivers/staging/media/tegra-vde/Makefile +1 -0
index 7f9020e634f3..c11867e28233 100644
@@ -1,2 +1,3 @@
 # SPDX-License-Identifier: GPL-2.0
+tegra-vde-y := vde.o iommu.o
 obj-$(CONFIG_TEGRA_VDE)	+= tegra-vde.o
drivers/staging/media/tegra-vde/iommu.c new file +151 -0

151 lines not on this page · raw message

drivers/staging/media/tegra-vde/trace.h +2 -0
index 85e2f7e2d4d0..e5714107db58 100644
@@ -8,6 +8,8 @@
 
 #include <linux/tracepoint.h>
 
+#include "vde.h"
+
 DECLARE_EVENT_CLASS(register_access,
 	TP_PROTO(struct tegra_vde *vde, void __iomem *base,
 		 u32 offset, u32 value),
drivers/staging/media/tegra-vde/vde.c renamed from drivers/staging/media/tegra-vde/tegra-vde.c +95 -93

354 lines not on this page · raw message

drivers/staging/media/tegra-vde/vde.h new file +89 -0

89 lines not on this page · raw message

Signature · 1 line
2.22.0
#10 lore raw

PATCH v2 4/4 staging: media: tegra-vde: Defer dmabuf's unmapping

In reply to Dmitry Osipenko

Frequent IOMMU remappings take about 50% of CPU usage because there is quite a lot to remap. Defer dmabuf's unmapping by 5 seconds in order to mitigate the mapping overhead which goes away completely and driver works as fast as in a case of a disabled IOMMU. The case of a disabled IOMMU should also benefit a tad from the caching since CPU cache maintenance that happens on dmabuf's attaching takes some resources.

---
 drivers/staging/media/tegra-vde/Makefile      |   2 +-
 .../staging/media/tegra-vde/dmabuf-cache.c    | 226 ++++++++++++++++++
 drivers/staging/media/tegra-vde/iommu.c       |   2 -
 drivers/staging/media/tegra-vde/vde.c         | 143 +++--------
 drivers/staging/media/tegra-vde/vde.h         |  18 +-
 5 files changed, 276 insertions(+), 115 deletions(-)
 create mode 100644 drivers/staging/media/tegra-vde/dmabuf-cache.c
drivers/staging/media/tegra-vde/Makefile +1 -1
index c11867e28233..2827f7601de8 100644
@@ -1,3 +1,3 @@
 # SPDX-License-Identifier: GPL-2.0
-tegra-vde-y := vde.o iommu.o
+tegra-vde-y := vde.o iommu.o dmabuf-cache.o
 obj-$(CONFIG_TEGRA_VDE)	+= tegra-vde.o
drivers/staging/media/tegra-vde/dmabuf-cache.c new file +226 -0

226 lines not on this page · raw message

drivers/staging/media/tegra-vde/iommu.c +0 -2
index ddc281b82e5c..1ee46a8d1cc7 100644
@@ -19,7 +19,6 @@
 int tegra_vde_iommu_map(struct tegra_vde *vde,
 			struct sg_table *sgt,
 			struct iova **iovap,
-			dma_addr_t *addrp,
 			size_t size)
 {
 	struct iova *iova;
@@ -45,7 +44,6 @@ int tegra_vde_iommu_map(struct tegra_vde *vde,
 	}
 
 	*iovap = iova;
-	*addrp = addr;
 
 	return 0;
 }
drivers/staging/media/tegra-vde/vde.c +32 -111

276 lines not on this page · raw message

drivers/staging/media/tegra-vde/vde.h +17 -1
index ef9616b24e08..4b901aeed6e9 100644
@@ -9,16 +9,20 @@
 #define TEGRA_VDE_H
 
 #include <linux/completion.h>
+#include <linux/dma-direction.h>
+#include <linux/list.h>
 #include <linux/miscdevice.h>
 #include <linux/mutex.h>
 #include <linux/types.h>
 #include <linux/iova.h>
 
 struct clk;
+struct dma_buf;
 struct gen_pool;
 struct iommu_group;
 struct iommu_domain;
 struct reset_control;
+struct dma_buf_attachment;
 
 struct tegra_vde {
 	void __iomem *sxe;
@@ -31,6 +35,8 @@ struct tegra_vde {
 	void __iomem *vdma;
 	void __iomem *frameid;
 	struct mutex lock;
+	struct mutex map_lock;
+	struct list_head map_list;
 	struct miscdevice miscdev;
 	struct reset_control *rst;
 	struct reset_control *rst_mc;
@@ -49,10 +55,20 @@ void tegra_vde_iommu_deinit(struct tegra_vde *vde);
 int tegra_vde_iommu_map(struct tegra_vde *vde,
 			struct sg_table *sgt,
 			struct iova **iovap,
-			dma_addr_t *addrp,
 			size_t size);
 void tegra_vde_iommu_unmap(struct tegra_vde *vde, struct iova *iova);
 
+int tegra_vde_dmabuf_cache_map(struct tegra_vde *vde,
+			       struct dma_buf *dmabuf,
+			       enum dma_data_direction dma_dir,
+			       struct dma_buf_attachment **ap,
+			       dma_addr_t *addrp);
+void tegra_vde_dmabuf_cache_unmap(struct tegra_vde *vde,
+				  struct dma_buf_attachment *a,
+				  bool release);
+void tegra_vde_dmabuf_cache_unmap_sync(struct tegra_vde *vde);
+void tegra_vde_dmabuf_cache_unmap_all(struct tegra_vde *vde);
+
 static __maybe_unused char const *
 tegra_vde_reg_base_name(struct tegra_vde *vde, void __iomem *base)
 {
Signature · 1 line
2.22.0