Skip to contentSkip to conversation

Vfio-on-arm: the device tree does not contain the DMA device

DISCUSSION Shannon Zhao 4 participants 33 messages Jun 18, 2014 – Jun 25, 2014 33 messages not in the archive

lore
Share & tools
lore
#0 lore raw

Hi Antonios and all,

	Recently, I want to test vfio on ARM and refer to this document "http://www.virtualopensystems.com/en/solutions/guides/vfio-on-arm/" .
	Following is my test steps.

	1. Get kernel src
		git clone git://github.com/virtualopensystems/linux-kvm-arm.git
		cd linux-kvm-arm
		git checkout origin/vfio-platform-v4

	2. Get kernel config
		http://www.virtualopensystems.com/downloads/guides/kvm_on_arm/kernel-config > .config

	3. Make menuconfig the kernel
		enable the KVM, SMMU and VFIO configuration options for the host kernel

	4. Compile the kernel
		CROSS_COMPILE=arm-linux-gnueabi- ARCH=arm make zImage

	5. Build a device tree
		There isn't rtsm_ve-cortex_a15x4_a7x4.dts in the directory "arch/arm/boot/dts/".
		So I download it from this git "git://github.com/virtualopensystems/arm-dts.git".
		./scripts/dtc/dtc -O dtb -o host-a15.dtb ../arm-dts/fast_models/rtsm_ve-cortex_a15x4_a7x4.dts
		But the rtsm_ve-cortex_a15x4_a7x4.dts doesn't contain the DMA or SMMU device.

	6. Launch the model
		Following is the boot_model.sh.
			#!/bin/bash

			MODEL_PATH='/root/ARM/FastModelsPortfolio_8.2/examples/FVP_VE/Build_Cortex-A15x4-A7x4-MMU400-DMA330/Linux64-Debug-GCC-4.4'

			$MODEL_PATH/isim_system \
			-a coretile.cluster0.*=/home/vfio/linux-system-semi.axf \
			-a coretile.cluster1.*=/home/vfio/linux-system-semi.axf \
			-C motherboard.smsc_91c111.enabled=1 \
			-C motherboard.hostbridge.interfaceName="ARMroot" \
			-C coretile.cluster0.cpu0.semihosting-enable=1 \
			-C coretile.dmac0_int_base=64 \
			-C coretile.dmac1_int_base=96 \
			-C coretile.dmac2_int_base=128 \
			-C coretile.dmac3_int_base=168 \
			-C coretile.mmu400_dma330.dmac0.p_irq_nsecure="true" \
			-C coretile.mmu400_dma330.dmac1.p_irq_nsecure="true" \
			-C coretile.mmu400_dma330.dmac2.p_irq_nsecure="true" \
			-C coretile.mmu400_dma330.dmac3.p_irq_nsecure="true" \
			-C coretile.cluster0.cpu0.semihosting-cmd_line="\
			--kernel /home/vfio/zImage \
			--initrd /home/vfio/filesystem.cpio.gz \
			--dtb host-a15.dtb  -- earlyprintk console=ttyAMA0 mem=2048M "
		It launched successfully. There is "vfio_platform" in the directory, but I can't see any DMA decive in the directory "/sys/devices".

	PS: I also try to enable the kernel configuration option for DMA. "DMA Engine support --> DMA API Driver for PL330".
	Is there something I'm wrong? Is there any way I can get the right device tree and kernel config?
	Look forward to your reply.
Signature · 1 line
Shannon
#2 lore raw
In reply to a message not in the archive

Hi Shannon,

Thank you for reporting this issue, the device tree file was indeed missing. To solve the problem, you can apply the following patch to the repository you mentioned
(git://github.com/virtualopensystems/arm-dts.git).

For your information, soon we will update the guide with a newer version of VFIO with this issue fixed.

Thank you,
alvise

---
 fast_models/rtsm_ve-cortex_a15x4_a7x4.dts | 100
+++++++++++++++++++++++++++++-
 1 file changed, 97 insertions(+), 3 deletions(-)
(quoted diff) +0 -0

b/fast_models/rtsm_ve-cortex_a15x4_a7x4.dts
index ef1275b..79c6af2 100644

fast_models/rtsm_ve-cortex_a15x4_a7x4.dts +97 -3

121 lines not on this page · raw message

Signature · 1 line
1.9.1
Quoted 55 lines · Shannon Zhao

Hi Antonios and all,

	Recently, I want to test vfio on ARM and refer to this document "http://www.virtualopensystems.com/en/solutions/guides/vfio-on-arm/" .
	Following is my test steps.

	1. Get kernel src
		git clone git://github.com/virtualopensystems/linux-kvm-arm.git
		cd linux-kvm-arm
		git checkout origin/vfio-platform-v4

	2. Get kernel config
		http://www.virtualopensystems.com/downloads/guides/kvm_on_arm/kernel-config > .config

	3. Make menuconfig the kernel
		enable the KVM, SMMU and VFIO configuration options for the host kernel

	4. Compile the kernel
		CROSS_COMPILE=arm-linux-gnueabi- ARCH=arm make zImage

	5. Build a device tree
		There isn't rtsm_ve-cortex_a15x4_a7x4.dts in the directory "arch/arm/boot/dts/".
		So I download it from this git "git://github.com/virtualopensystems/arm-dts.git".
		./scripts/dtc/dtc -O dtb -o host-a15.dtb ../arm-dts/fast_models/rtsm_ve-cortex_a15x4_a7x4.dts
		But the rtsm_ve-cortex_a15x4_a7x4.dts doesn't contain the DMA or SMMU device.

	6. Launch the model
		Following is the boot_model.sh.
			#!/bin/bash

			MODEL_PATH='/root/ARM/FastModelsPortfolio_8.2/examples/FVP_VE/Build_Cortex-A15x4-A7x4-MMU400-DMA330/Linux64-Debug-GCC-4.4'

			$MODEL_PATH/isim_system \
			-a coretile.cluster0.*=/home/vfio/linux-system-semi.axf \
			-a coretile.cluster1.*=/home/vfio/linux-system-semi.axf \
			-C motherboard.smsc_91c111.enabled=1 \
			-C motherboard.hostbridge.interfaceName="ARMroot" \
			-C coretile.cluster0.cpu0.semihosting-enable=1 \
			-C coretile.dmac0_int_base=64 \
			-C coretile.dmac1_int_base=96 \
			-C coretile.dmac2_int_base=128 \
			-C coretile.dmac3_int_base=168 \
			-C coretile.mmu400_dma330.dmac0.p_irq_nsecure="true" \
			-C coretile.mmu400_dma330.dmac1.p_irq_nsecure="true" \
			-C coretile.mmu400_dma330.dmac2.p_irq_nsecure="true" \
			-C coretile.mmu400_dma330.dmac3.p_irq_nsecure="true" \
			-C coretile.cluster0.cpu0.semihosting-cmd_line="\
			--kernel /home/vfio/zImage \
			--initrd /home/vfio/filesystem.cpio.gz \
			--dtb host-a15.dtb  -- earlyprintk console=ttyAMA0 mem=2048M "
		It launched successfully. There is "vfio_platform" in the directory, but I can't see any DMA decive in the directory "/sys/devices".

	PS: I also try to enable the kernel configuration option for DMA. "DMA Engine support --> DMA API Driver for PL330".
	Is there something I'm wrong? Is there any way I can get the right device tree and kernel config?
	Look forward to your reply.
#4 lore raw
In reply to a message not in the archive

Hi Alvise,

Thanks for your reply.
I have apply the patch and could see the DMA devices.
But there is no vosys_dmatest int the directory "/sys/kernel/debug/" after mounting debugfs. I found that the branch vfio-platform-v4 doesn't contain the module vosys_dmatest. Then I port it from the branch pl330_dmatest and run vosys_dmatest successfully.

	echo 1 > /sys/kernel/debug/vosys_dmatest/start

	[ 6145.343419] dmaengine: __dma_request_channel: success (dma0chan0)
	[ 6145.344000] vosys dmatest: callback: transfer completed
	[ 6145.344147] vosys dmatest: tx completed
	[ 6145.344589] vosys dmatest: source and destination match.

Download the qemu and use following command to start a guest:

	./qemu-system-arm -enable-kvm -M vexpress-a15 -cpu cortex-a15 \
        	-device pl330,vfio_group="/dev/vfio/0",vfio_device="2c0a0000.dma",\
        	vfio_container="/dev/vfio/vfio" \
        	-append "earlyprintk ip=dhcp console=ttyAMA0 rootwait" \
        	-kernel zImage -dtb guest.dtb -initrd filesystem.cpio.gz  -nographic

An error reported:
qemu-system-arm: -device pl330,vfio_group=/dev/vfio/0,vfio_device=2c0a0000.dma,:
could not open disk image vfio_container=/dev/vfio:
Could not open 'vfio_container=/dev/vfio': No such file or directory

Is there something I'm wrong or do I miss some steps?

Thank you,
Shannon

Quoted 149 lines · Alvise Rigo

Hi Shannon,

Thank you for reporting this issue, the device tree file was indeed missing. To solve the problem, you can apply the following patch to the repository you mentioned
(git://github.com/virtualopensystems/arm-dts.git).

For your information, soon we will update the guide with a newer version of VFIO with this issue fixed.

Thank you,
alvise

---
 fast_models/rtsm_ve-cortex_a15x4_a7x4.dts | 100
+++++++++++++++++++++++++++++-
 1 file changed, 97 insertions(+), 3 deletions(-)
fast_models/rtsm_ve-cortex_a15x4_a7x4.dts +97 -3
index ef1275b..79c6af2 100644
@@ -142,7 +142,7 @@
 		cpu7: cpu@7 {
 			device_type = "cpu";
 			compatible = "arm,cortex-a7";
-			reg = <0x104>;
+			reg = <0x103>;
 			cluster = <&cluster1>;
 			core = <&core7>;
 		};
@@ -159,7 +159,7 @@
 		#address-cells = <0>;
 		interrupt-controller;
 		reg = <0 0x2c001000 0 0x1000>,
-		      <0 0x2c002000 0 0x2000>,
+		      <0 0x2c002000 0 0x1000>,
 		      <0 0x2c004000 0 0x2000>,
 		      <0 0x2c006000 0 0x2000>;
 		interrupts = <1 9 0xf04>;
@@ -183,6 +183,101 @@
 			     <1 10 0xf08>;
 	};
 
+	smmu@2c0b0000 {
+                compatible = "arm,smmu-v2";
+                reg = <0x0 0x2c0b0000 0x0 0x10000>;
+                #global-interrupts = <2>;
+                interrupts = <0 58 4>,
+                             <0 59 4>,
+                             <0 60 4>,
+                             <0 60 4>,
+                             <0 60 4>,
+                             <0 60 4>,
+                             <0 60 4>,
+                             <0 60 4>,
+                             <0 60 4>;
+                mmu-masters = <&dma0 0x0 0x8>,
+                              <&dma1 0x1 0x9>,
+                              <&dma2 0x2 0xa>,
+                              <&dma3 0x3 0xb>;
+        };
+
+	dma0: dma@2c0a0000 {
+		compatible = "arm,pl330";
+		reg = <0 0x2c0a0000 0 0x1000>;
+		interrupts = <0 64 4>,
+			     <0 65 4>,
+			     <0 66 4>,
+			     <0 67 4>,
+			     <0 68 4>,
+			     <0 69 4>,
+			     <0 70 4>,
+			     <0 71 4>;
+		#dma-cells = <1>;
+		#stream-id-cells = <2>;
+		clocks = <&oscclk6a>;
+		clock-names = "apb_pclk";
+	};
+
+	dma1: dma@2c0a1000 {
+		compatible = "arm,pl330", "arm,primecell";
+		reg = <0 0x2c0a1000 0 0x1000>;
+		interrupts = <0 96 4>,
+			     <0 97 4>,
+			     <0 98 4>,
+			     <0 99 4>,
+			     <0 100 4>,
+			     <0 101 4>,
+			     <0 102 4>,
+			     <0 103 4>;
+		#dma-cells = <1>;
+		#stream-id-cells = <2>;
+		clocks = <&oscclk6a>;
+		clock-names = "apb_pclk";
+	};
+
+	dma2: dma@2c0a2000 {
+		compatible = "arm,pl330", "arm,primecell";
+		reg = <0 0x2c0a2000 0 0x1000>;
+		interrupts = <0 128 4>,
+			     <0 129 4>,
+			     <0 130 4>,
+			     <0 131 4>,
+			     <0 132 4>,
+			     <0 133 4>,
+			     <0 134 4>,
+			     <0 135 4>;
+		#dma-cells = <1>;
+		#stream-id-cells = <2>;
+		clocks = <&oscclk6a>;
+		clock-names = "apb_pclk";
+	};
+
+	dma3: dma@2c0a3000 {
+		compatible = "arm,pl330", "arm,primecell";
+		reg = <0 0x2c0a3000 0 0x1000>;
+		interrupts = <0 160 4>,
+			     <0 161 4>,
+			     <0 162 4>,
+			     <0 163 4>,
+			     <0 164 4>,
+			     <0 165 4>,
+			     <0 166 4>,
+			     <0 167 4>;
+		#dma-cells = <1>;
+		#stream-id-cells = <2>;
+		clocks = <&oscclk6a>;
+		clock-names = "apb_pclk";
+	};
+
+	oscclk6a: oscclk6a {
+		/* Reference 24MHz clock */
+		compatible = "fixed-clock";
+		#clock-cells = <0>;
+		clock-frequency = <24000000>;
+		clock-output-names = "oscclk6a";
+	};
+
 	smb {
 		compatible = "simple-bus";
 
@@ -244,4 +339,3 @@
 		/include/ "rtsm_ve-motherboard.dtsi"
 	};
 };
-
Signature · 1 line
Shannon
#6 lore raw
In reply to a message not in the archive
Quoted 28 lines

-----Original Message-----
From: [email protected] [mailto:kvmarm- [email protected]] On Behalf Of Shannon Zhao
Sent: Thursday, June 19, 2014 2:45 PM
To: Alvise Rigo; a.motakis-lrHrjnjw1UfHK3s98zE1ajGjJy/[email protected]; iommu-cunTk1MwBs/[email protected] foundation.org; [email protected]
Subject: Re: Vfio-on-arm: the device tree does not contain the DMA device

Hi Alvise,

Thanks for your reply.
I have apply the patch and could see the DMA devices.
But there is no vosys_dmatest int the directory "/sys/kernel/debug/" after mounting debugfs.
I found that the branch vfio-platform-v4 doesn't contain the module vosys_dmatest.
Then I port it from the branch pl330_dmatest and run vosys_dmatest successfully.

	echo 1 > /sys/kernel/debug/vosys_dmatest/start

	[ 6145.343419] dmaengine: __dma_request_channel: success (dma0chan0)
	[ 6145.344000] vosys dmatest: callback: transfer completed
	[ 6145.344147] vosys dmatest: tx completed
	[ 6145.344589] vosys dmatest: source and destination match.

Download the qemu and use following command to start a guest:

Quoted 8 lines
	./qemu-system-arm -enable-kvm -M vexpress-a15 -cpu cortex-a15 \
        	-device pl330,vfio_group="/dev/vfio/0",vfio_device="2c0a0000.dma",\
        	vfio_container="/dev/vfio/vfio" \
        	-append "earlyprintk ip=dhcp console=ttyAMA0 rootwait" \
        	-kernel zImage -dtb guest.dtb -initrd filesystem.cpio.gz  -

nographic An error reported:
qemu-system-arm: -device
pl330,vfio_group=/dev/vfio/0,vfio_device=2c0a0000.dma,:

Why do we need to give "vfio_group=/dev/vfio/0" ?

Quoted 2 lines

could not open disk image vfio_container=/dev/vfio:
Could not open 'vfio_container=/dev/vfio': No such file or directory

Are you unbinding "2c0a0000.dma" from its driver and binding with "vfio-playform" driver ?

Thanks
-Bharat

Quoted 164 lines

Is there something I'm wrong or do I miss some steps?

Thank you,
Shannon

Quoted 145 lines · Alvise Rigo

Hi Shannon,

Thank you for reporting this issue, the device tree file was indeed missing. To solve the problem, you can apply the following patch to the repository you mentioned
(git://github.com/virtualopensystems/arm-dts.git).

For your information, soon we will update the guide with a newer version of VFIO with this issue fixed.

Thank you,
alvise

---
 fast_models/rtsm_ve-cortex_a15x4_a7x4.dts | 100
+++++++++++++++++++++++++++++-
 1 file changed, 97 insertions(+), 3 deletions(-)
fast_models/rtsm_ve-cortex_a15x4_a7x4.dts +97 -2
index ef1275b..79c6af2 100644
@@ -142,7 +142,7 @@
 		cpu7: cpu@7 {
 			device_type = "cpu";
 			compatible = "arm,cortex-a7";
-			reg = <0x104>;
+			reg = <0x103>;
 			cluster = <&cluster1>;
 			core = <&core7>;
 		};
@@ -159,7 +159,7 @@
 		#address-cells = <0>;
 		interrupt-controller;
 		reg = <0 0x2c001000 0 0x1000>,
-		      <0 0x2c002000 0 0x2000>,
+		      <0 0x2c002000 0 0x1000>,
 		      <0 0x2c004000 0 0x2000>,
 		      <0 0x2c006000 0 0x2000>;
 		interrupts = <1 9 0xf04>;
@@ -183,6 +183,101 @@
 			     <1 10 0xf08>;
 	};
 
+	smmu@2c0b0000 {
+                compatible = "arm,smmu-v2";
+                reg = <0x0 0x2c0b0000 0x0 0x10000>;
+                #global-interrupts = <2>;
+                interrupts = <0 58 4>,
+                             <0 59 4>,
+                             <0 60 4>,
+                             <0 60 4>,
+                             <0 60 4>,
+                             <0 60 4>,
+                             <0 60 4>,
+                             <0 60 4>,
+                             <0 60 4>;
+                mmu-masters = <&dma0 0x0 0x8>,
+                              <&dma1 0x1 0x9>,
+                              <&dma2 0x2 0xa>,
+                              <&dma3 0x3 0xb>;
+        };
+
+	dma0: dma@2c0a0000 {
+		compatible = "arm,pl330";
+		reg = <0 0x2c0a0000 0 0x1000>;
+		interrupts = <0 64 4>,
+			     <0 65 4>,
+			     <0 66 4>,
+			     <0 67 4>,
+			     <0 68 4>,
+			     <0 69 4>,
+			     <0 70 4>,
+			     <0 71 4>;
+		#dma-cells = <1>;
+		#stream-id-cells = <2>;
+		clocks = <&oscclk6a>;
+		clock-names = "apb_pclk";
+	};
+
+	dma1: dma@2c0a1000 {
+		compatible = "arm,pl330", "arm,primecell";
+		reg = <0 0x2c0a1000 0 0x1000>;
+		interrupts = <0 96 4>,
+			     <0 97 4>,
+			     <0 98 4>,
+			     <0 99 4>,
+			     <0 100 4>,
+			     <0 101 4>,
+			     <0 102 4>,
+			     <0 103 4>;
+		#dma-cells = <1>;
+		#stream-id-cells = <2>;
+		clocks = <&oscclk6a>;
+		clock-names = "apb_pclk";
+	};
+
+	dma2: dma@2c0a2000 {
+		compatible = "arm,pl330", "arm,primecell";
+		reg = <0 0x2c0a2000 0 0x1000>;
+		interrupts = <0 128 4>,
+			     <0 129 4>,
+			     <0 130 4>,
+			     <0 131 4>,
+			     <0 132 4>,
+			     <0 133 4>,
+			     <0 134 4>,
+			     <0 135 4>;
+		#dma-cells = <1>;
+		#stream-id-cells = <2>;
+		clocks = <&oscclk6a>;
+		clock-names = "apb_pclk";
+	};
+
+	dma3: dma@2c0a3000 {
+		compatible = "arm,pl330", "arm,primecell";
+		reg = <0 0x2c0a3000 0 0x1000>;
+		interrupts = <0 160 4>,
+			     <0 161 4>,
+			     <0 162 4>,
+			     <0 163 4>,
+			     <0 164 4>,
+			     <0 165 4>,
+			     <0 166 4>,
+			     <0 167 4>;
+		#dma-cells = <1>;
+		#stream-id-cells = <2>;
+		clocks = <&oscclk6a>;
+		clock-names = "apb_pclk";
+	};
+
+	oscclk6a: oscclk6a {
+		/* Reference 24MHz clock */
+		compatible = "fixed-clock";
+		#clock-cells = <0>;
+		clock-frequency = <24000000>;
+		clock-output-names = "oscclk6a";
+	};
+
 	smb {
 		compatible = "simple-bus";
 

fast_models/rtsm_ve-cortex_a15x4_a7x4.dts line 247

Quoted 4 lines
fast_models/rtsm_ve-cortex_a15x4_a7x4.dts +0 -1
@@ -244,4 +339,3 @@
 		/include/ "rtsm_ve-motherboard.dtsi"
 	};
 };
-

--
Shannon

_______________________________________________
kvmarm mailing list [email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm

#8 lore raw
In reply to a message not in the archive
Quoted 40 lines
Quoted 14 lines

-----Original Message-----
From: [email protected] [mailto:kvmarm- [email protected]] On Behalf Of Shannon Zhao
Sent: Thursday, June 19, 2014 2:45 PM
To: Alvise Rigo; a.motakis-lrHrjnjw1UfHK3s98zE1ajGjJy/[email protected]; iommu-cunTk1MwBs/[email protected] foundation.org; [email protected]
Subject: Re: Vfio-on-arm: the device tree does not contain the DMA
device

Hi Alvise,

Quoted 7 lines

Thanks for your reply.
I have apply the patch and could see the DMA devices.
But there is no vosys_dmatest int the directory "/sys/kernel/debug/" after mounting debugfs.
I found that the branch vfio-platform-v4 doesn't contain the module vosys_dmatest.
Then I port it from the branch pl330_dmatest and run vosys_dmatest

successfully.

Quoted 9 lines
	echo 1 > /sys/kernel/debug/vosys_dmatest/start

	[ 6145.343419] dmaengine: __dma_request_channel: success (dma0chan0)
	[ 6145.344000] vosys dmatest: callback: transfer completed
	[ 6145.344147] vosys dmatest: tx completed
	[ 6145.344589] vosys dmatest: source and destination match.

Download the qemu and use following command to start a guest:

Quoted 8 lines
	./qemu-system-arm -enable-kvm -M vexpress-a15 -cpu cortex-a15 \
        	-device pl330,vfio_group="/dev/vfio/0",vfio_device="2c0a0000.dma",\
        	vfio_container="/dev/vfio/vfio" \
        	-append "earlyprintk ip=dhcp console=ttyAMA0 rootwait" \
        	-kernel zImage -dtb guest.dtb -initrd filesystem.cpio.gz  -

nographic An error reported:
qemu-system-arm: -device
pl330,vfio_group=/dev/vfio/0,vfio_device=2c0a0000.dma,:

Quoted 5 lines

Why do we need to give "vfio_group=/dev/vfio/0" ?

Quoted 2 lines

could not open disk image vfio_container=/dev/vfio:
Could not open 'vfio_container=/dev/vfio': No such file or directory

Btw it should be looking for "/dev/vfio/vfio" and /dev/vfio/<iommu_group-number>, no ?

Thanks
-Bharat

Quoted 175 lines

Are you unbinding "2c0a0000.dma" from its driver and binding with "vfio- playform" driver ?

Thanks
-Bharat

Quoted 160 lines

Is there something I'm wrong or do I miss some steps?

Thank you,
Shannon

Quoted 145 lines · Alvise Rigo

Hi Shannon,

Thank you for reporting this issue, the device tree file was indeed missing. To solve the problem, you can apply the following patch to the repository you mentioned
(git://github.com/virtualopensystems/arm-dts.git).

For your information, soon we will update the guide with a newer version of VFIO with this issue fixed.

Thank you,
alvise

---
 fast_models/rtsm_ve-cortex_a15x4_a7x4.dts | 100
+++++++++++++++++++++++++++++-
 1 file changed, 97 insertions(+), 3 deletions(-)
fast_models/rtsm_ve-cortex_a15x4_a7x4.dts +97 -2
index ef1275b..79c6af2 100644
@@ -142,7 +142,7 @@
 		cpu7: cpu@7 {
 			device_type = "cpu";
 			compatible = "arm,cortex-a7";
-			reg = <0x104>;
+			reg = <0x103>;
 			cluster = <&cluster1>;
 			core = <&core7>;
 		};
@@ -159,7 +159,7 @@
 		#address-cells = <0>;
 		interrupt-controller;
 		reg = <0 0x2c001000 0 0x1000>,
-		      <0 0x2c002000 0 0x2000>,
+		      <0 0x2c002000 0 0x1000>,
 		      <0 0x2c004000 0 0x2000>,
 		      <0 0x2c006000 0 0x2000>;
 		interrupts = <1 9 0xf04>;
@@ -183,6 +183,101 @@
 			     <1 10 0xf08>;
 	};
 
+	smmu@2c0b0000 {
+                compatible = "arm,smmu-v2";
+                reg = <0x0 0x2c0b0000 0x0 0x10000>;
+                #global-interrupts = <2>;
+                interrupts = <0 58 4>,
+                             <0 59 4>,
+                             <0 60 4>,
+                             <0 60 4>,
+                             <0 60 4>,
+                             <0 60 4>,
+                             <0 60 4>,
+                             <0 60 4>,
+                             <0 60 4>;
+                mmu-masters = <&dma0 0x0 0x8>,
+                              <&dma1 0x1 0x9>,
+                              <&dma2 0x2 0xa>,
+                              <&dma3 0x3 0xb>;
+        };
+
+	dma0: dma@2c0a0000 {
+		compatible = "arm,pl330";
+		reg = <0 0x2c0a0000 0 0x1000>;
+		interrupts = <0 64 4>,
+			     <0 65 4>,
+			     <0 66 4>,
+			     <0 67 4>,
+			     <0 68 4>,
+			     <0 69 4>,
+			     <0 70 4>,
+			     <0 71 4>;
+		#dma-cells = <1>;
+		#stream-id-cells = <2>;
+		clocks = <&oscclk6a>;
+		clock-names = "apb_pclk";
+	};
+
+	dma1: dma@2c0a1000 {
+		compatible = "arm,pl330", "arm,primecell";
+		reg = <0 0x2c0a1000 0 0x1000>;
+		interrupts = <0 96 4>,
+			     <0 97 4>,
+			     <0 98 4>,
+			     <0 99 4>,
+			     <0 100 4>,
+			     <0 101 4>,
+			     <0 102 4>,
+			     <0 103 4>;
+		#dma-cells = <1>;
+		#stream-id-cells = <2>;
+		clocks = <&oscclk6a>;
+		clock-names = "apb_pclk";
+	};
+
+	dma2: dma@2c0a2000 {
+		compatible = "arm,pl330", "arm,primecell";
+		reg = <0 0x2c0a2000 0 0x1000>;
+		interrupts = <0 128 4>,
+			     <0 129 4>,
+			     <0 130 4>,
+			     <0 131 4>,
+			     <0 132 4>,
+			     <0 133 4>,
+			     <0 134 4>,
+			     <0 135 4>;
+		#dma-cells = <1>;
+		#stream-id-cells = <2>;
+		clocks = <&oscclk6a>;
+		clock-names = "apb_pclk";
+	};
+
+	dma3: dma@2c0a3000 {
+		compatible = "arm,pl330", "arm,primecell";
+		reg = <0 0x2c0a3000 0 0x1000>;
+		interrupts = <0 160 4>,
+			     <0 161 4>,
+			     <0 162 4>,
+			     <0 163 4>,
+			     <0 164 4>,
+			     <0 165 4>,
+			     <0 166 4>,
+			     <0 167 4>;
+		#dma-cells = <1>;
+		#stream-id-cells = <2>;
+		clocks = <&oscclk6a>;
+		clock-names = "apb_pclk";
+	};
+
+	oscclk6a: oscclk6a {
+		/* Reference 24MHz clock */
+		compatible = "fixed-clock";
+		#clock-cells = <0>;
+		clock-frequency = <24000000>;
+		clock-output-names = "oscclk6a";
+	};
+
 	smb {
 		compatible = "simple-bus";
 

fast_models/rtsm_ve-cortex_a15x4_a7x4.dts line 247

Quoted 4 lines
fast_models/rtsm_ve-cortex_a15x4_a7x4.dts +0 -1
@@ -244,4 +339,3 @@
 		/include/ "rtsm_ve-motherboard.dtsi"
 	};
 };
-

--
Shannon

Quoted 4 lines

_______________________________________________
kvmarm mailing list [email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm

_______________________________________________
iommu mailing list iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

#9 lore raw
In reply to a message not in the archive

Hi Bharat,

Quoted 37 lines · On 2014/6/19 17:19, [email protected]…
Quoted 28 lines

-----Original Message-----
From: [email protected] [mailto:kvmarm- [email protected]] On Behalf Of Shannon Zhao
Sent: Thursday, June 19, 2014 2:45 PM
To: Alvise Rigo; a.motakis-lrHrjnjw1UfHK3s98zE1ajGjJy/[email protected]; iommu-cunTk1MwBs/[email protected] foundation.org; [email protected]
Subject: Re: Vfio-on-arm: the device tree does not contain the DMA device

Hi Alvise,

Thanks for your reply.
I have apply the patch and could see the DMA devices.
But there is no vosys_dmatest int the directory "/sys/kernel/debug/" after mounting debugfs.
I found that the branch vfio-platform-v4 doesn't contain the module vosys_dmatest.
Then I port it from the branch pl330_dmatest and run vosys_dmatest successfully.

	echo 1 > /sys/kernel/debug/vosys_dmatest/start

	[ 6145.343419] dmaengine: __dma_request_channel: success (dma0chan0)
	[ 6145.344000] vosys dmatest: callback: transfer completed
	[ 6145.344147] vosys dmatest: tx completed
	[ 6145.344589] vosys dmatest: source and destination match.

Download the qemu and use following command to start a guest:

Quoted 8 lines
	./qemu-system-arm -enable-kvm -M vexpress-a15 -cpu cortex-a15 \
        	-device pl330,vfio_group="/dev/vfio/0",vfio_device="2c0a0000.dma",\
        	vfio_container="/dev/vfio/vfio" \
        	-append "earlyprintk ip=dhcp console=ttyAMA0 rootwait" \
        	-kernel zImage -dtb guest.dtb -initrd filesystem.cpio.gz  -

nographic An error reported:
qemu-system-arm: -device
pl330,vfio_group=/dev/vfio/0,vfio_device=2c0a0000.dma,:

Quoted 1 line

Why do we need to give "vfio_group=/dev/vfio/0" ?

I refer to the url "http://www.virtualopensystems.com/en/solutions/guides/vfio-on-arm/".

Quoted 6 lines
Quoted 4 lines

could not open disk image vfio_container=/dev/vfio:
Could not open 'vfio_container=/dev/vfio': No such file or directory

Are you unbinding "2c0a0000.dma" from its driver and binding with "vfio-playform" driver ?

How to unbind "2c0a0000.dma" from its driver?
I have tried following cmd to bind "2c0a0000.dma" with "vfio-playform" driver.
echo 2c0a0000.dma > /sys/bus/platform/drivers/vfio-platform/vfio_bind
But there is no /dev/vfio/0 or /dev/vfio/vfio .

Quoted 171 lines

Thanks
-Bharat

Quoted 160 lines

Is there something I'm wrong or do I miss some steps?

Thank you,
Shannon

Quoted 145 lines · Alvise Rigo

Hi Shannon,

Thank you for reporting this issue, the device tree file was indeed missing. To solve the problem, you can apply the following patch to the repository you mentioned
(git://github.com/virtualopensystems/arm-dts.git).

For your information, soon we will update the guide with a newer version of VFIO with this issue fixed.

Thank you,
alvise

---
 fast_models/rtsm_ve-cortex_a15x4_a7x4.dts | 100
+++++++++++++++++++++++++++++-
 1 file changed, 97 insertions(+), 3 deletions(-)
fast_models/rtsm_ve-cortex_a15x4_a7x4.dts +97 -2
index ef1275b..79c6af2 100644
@@ -142,7 +142,7 @@
 		cpu7: cpu@7 {
 			device_type = "cpu";
 			compatible = "arm,cortex-a7";
-			reg = <0x104>;
+			reg = <0x103>;
 			cluster = <&cluster1>;
 			core = <&core7>;
 		};
@@ -159,7 +159,7 @@
 		#address-cells = <0>;
 		interrupt-controller;
 		reg = <0 0x2c001000 0 0x1000>,
-		      <0 0x2c002000 0 0x2000>,
+		      <0 0x2c002000 0 0x1000>,
 		      <0 0x2c004000 0 0x2000>,
 		      <0 0x2c006000 0 0x2000>;
 		interrupts = <1 9 0xf04>;
@@ -183,6 +183,101 @@
 			     <1 10 0xf08>;
 	};
 
+	smmu@2c0b0000 {
+                compatible = "arm,smmu-v2";
+                reg = <0x0 0x2c0b0000 0x0 0x10000>;
+                #global-interrupts = <2>;
+                interrupts = <0 58 4>,
+                             <0 59 4>,
+                             <0 60 4>,
+                             <0 60 4>,
+                             <0 60 4>,
+                             <0 60 4>,
+                             <0 60 4>,
+                             <0 60 4>,
+                             <0 60 4>;
+                mmu-masters = <&dma0 0x0 0x8>,
+                              <&dma1 0x1 0x9>,
+                              <&dma2 0x2 0xa>,
+                              <&dma3 0x3 0xb>;
+        };
+
+	dma0: dma@2c0a0000 {
+		compatible = "arm,pl330";
+		reg = <0 0x2c0a0000 0 0x1000>;
+		interrupts = <0 64 4>,
+			     <0 65 4>,
+			     <0 66 4>,
+			     <0 67 4>,
+			     <0 68 4>,
+			     <0 69 4>,
+			     <0 70 4>,
+			     <0 71 4>;
+		#dma-cells = <1>;
+		#stream-id-cells = <2>;
+		clocks = <&oscclk6a>;
+		clock-names = "apb_pclk";
+	};
+
+	dma1: dma@2c0a1000 {
+		compatible = "arm,pl330", "arm,primecell";
+		reg = <0 0x2c0a1000 0 0x1000>;
+		interrupts = <0 96 4>,
+			     <0 97 4>,
+			     <0 98 4>,
+			     <0 99 4>,
+			     <0 100 4>,
+			     <0 101 4>,
+			     <0 102 4>,
+			     <0 103 4>;
+		#dma-cells = <1>;
+		#stream-id-cells = <2>;
+		clocks = <&oscclk6a>;
+		clock-names = "apb_pclk";
+	};
+
+	dma2: dma@2c0a2000 {
+		compatible = "arm,pl330", "arm,primecell";
+		reg = <0 0x2c0a2000 0 0x1000>;
+		interrupts = <0 128 4>,
+			     <0 129 4>,
+			     <0 130 4>,
+			     <0 131 4>,
+			     <0 132 4>,
+			     <0 133 4>,
+			     <0 134 4>,
+			     <0 135 4>;
+		#dma-cells = <1>;
+		#stream-id-cells = <2>;
+		clocks = <&oscclk6a>;
+		clock-names = "apb_pclk";
+	};
+
+	dma3: dma@2c0a3000 {
+		compatible = "arm,pl330", "arm,primecell";
+		reg = <0 0x2c0a3000 0 0x1000>;
+		interrupts = <0 160 4>,
+			     <0 161 4>,
+			     <0 162 4>,
+			     <0 163 4>,
+			     <0 164 4>,
+			     <0 165 4>,
+			     <0 166 4>,
+			     <0 167 4>;
+		#dma-cells = <1>;
+		#stream-id-cells = <2>;
+		clocks = <&oscclk6a>;
+		clock-names = "apb_pclk";
+	};
+
+	oscclk6a: oscclk6a {
+		/* Reference 24MHz clock */
+		compatible = "fixed-clock";
+		#clock-cells = <0>;
+		clock-frequency = <24000000>;
+		clock-output-names = "oscclk6a";
+	};
+
 	smb {
 		compatible = "simple-bus";
 

fast_models/rtsm_ve-cortex_a15x4_a7x4.dts line 247

Quoted 4 lines
fast_models/rtsm_ve-cortex_a15x4_a7x4.dts +0 -1
@@ -244,4 +339,3 @@
 		/include/ "rtsm_ve-motherboard.dtsi"
 	};
 };
-

--
Shannon

Quoted 4 lines

_______________________________________________
kvmarm mailing list [email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm

.

Signature · 1 line
Shannon
#11 lore raw
In reply to a message not in the archive
Quoted 44 lines · Shannon Zhao

Hi Bharat,

Quoted 37 lines · On 2014/6/19 17:19, [email protected]…
Quoted 28 lines

-----Original Message-----
From: [email protected] [mailto:kvmarm- [email protected]] On Behalf Of Shannon Zhao
Sent: Thursday, June 19, 2014 2:45 PM
To: Alvise Rigo; a.motakis-lrHrjnjw1UfHK3s98zE1ajGjJy/[email protected]; iommu-cunTk1MwBs/[email protected] foundation.org; [email protected]
Subject: Re: Vfio-on-arm: the device tree does not contain the DMA device

Hi Alvise,

Thanks for your reply.
I have apply the patch and could see the DMA devices.
But there is no vosys_dmatest int the directory "/sys/kernel/debug/" after mounting debugfs.
I found that the branch vfio-platform-v4 doesn't contain the module vosys_dmatest.
Then I port it from the branch pl330_dmatest and run vosys_dmatest successfully.

	echo 1 > /sys/kernel/debug/vosys_dmatest/start

	[ 6145.343419] dmaengine: __dma_request_channel: success (dma0chan0)
	[ 6145.344000] vosys dmatest: callback: transfer completed
	[ 6145.344147] vosys dmatest: tx completed
	[ 6145.344589] vosys dmatest: source and destination match.

Download the qemu and use following command to start a guest:

Quoted 8 lines
	./qemu-system-arm -enable-kvm -M vexpress-a15 -cpu cortex-a15 \
        	-device pl330,vfio_group="/dev/vfio/0",vfio_device="2c0a0000.dma",\
        	vfio_container="/dev/vfio/vfio" \
        	-append "earlyprintk ip=dhcp console=ttyAMA0 rootwait" \
        	-kernel zImage -dtb guest.dtb -initrd filesystem.cpio.gz  -

nographic An error reported:
qemu-system-arm: -device
pl330,vfio_group=/dev/vfio/0,vfio_device=2c0a0000.dma,:

Quoted 1 line

Why do we need to give "vfio_group=/dev/vfio/0" ?

I refer to the url "http://www.virtualopensystems.com/en/solutions/guides/vfio-on-arm/".

Quoted 2 lines
Quoted 8 lines
Quoted 4 lines
Quoted 2 lines

could not open disk image vfio_container=/dev/vfio:
Could not open 'vfio_container=/dev/vfio': No such file or directory

Are you unbinding "2c0a0000.dma" from its driver and binding with "vfio-playform" driver ?

How to unbind "2c0a0000.dma" from its driver?
I have tried following cmd to bind "2c0a0000.dma" with "vfio-playform" driver.
echo 2c0a0000.dma > /sys/bus/platform/drivers/vfio-platform/vfio_bind
But there is no /dev/vfio/0 or /dev/vfio/vfio .

Hi Shannon,

you may try following cmd to unbind the native dma driver echo 2c0a0000.dma | sudo tee -a /sys/bus/platform/devices/2c0a0000.dma/driver/unbind

and then the following one to bing the meta-driver to the device echo 2c0a0000.dma | sudo tee -a /sys/bus/platform/drivers/vfio-platform/vfio_bind

Best Regards

Eric

Quoted 173 lines
Quoted 172 lines

Thanks
-Bharat

Quoted 160 lines

Is there something I'm wrong or do I miss some steps?

Thank you,
Shannon

Quoted 145 lines · Alvise Rigo

Hi Shannon,

Thank you for reporting this issue, the device tree file was indeed missing. To solve the problem, you can apply the following patch to the repository you mentioned
(git://github.com/virtualopensystems/arm-dts.git).

For your information, soon we will update the guide with a newer version of VFIO with this issue fixed.

Thank you,
alvise

---
 fast_models/rtsm_ve-cortex_a15x4_a7x4.dts | 100
+++++++++++++++++++++++++++++-
 1 file changed, 97 insertions(+), 3 deletions(-)
fast_models/rtsm_ve-cortex_a15x4_a7x4.dts +97 -2
index ef1275b..79c6af2 100644
@@ -142,7 +142,7 @@
 		cpu7: cpu@7 {
 			device_type = "cpu";
 			compatible = "arm,cortex-a7";
-			reg = <0x104>;
+			reg = <0x103>;
 			cluster = <&cluster1>;
 			core = <&core7>;
 		};
@@ -159,7 +159,7 @@
 		#address-cells = <0>;
 		interrupt-controller;
 		reg = <0 0x2c001000 0 0x1000>,
-		      <0 0x2c002000 0 0x2000>,
+		      <0 0x2c002000 0 0x1000>,
 		      <0 0x2c004000 0 0x2000>,
 		      <0 0x2c006000 0 0x2000>;
 		interrupts = <1 9 0xf04>;
@@ -183,6 +183,101 @@
 			     <1 10 0xf08>;
 	};
 
+	smmu@2c0b0000 {
+                compatible = "arm,smmu-v2";
+                reg = <0x0 0x2c0b0000 0x0 0x10000>;
+                #global-interrupts = <2>;
+                interrupts = <0 58 4>,
+                             <0 59 4>,
+                             <0 60 4>,
+                             <0 60 4>,
+                             <0 60 4>,
+                             <0 60 4>,
+                             <0 60 4>,
+                             <0 60 4>,
+                             <0 60 4>;
+                mmu-masters = <&dma0 0x0 0x8>,
+                              <&dma1 0x1 0x9>,
+                              <&dma2 0x2 0xa>,
+                              <&dma3 0x3 0xb>;
+        };
+
+	dma0: dma@2c0a0000 {
+		compatible = "arm,pl330";
+		reg = <0 0x2c0a0000 0 0x1000>;
+		interrupts = <0 64 4>,
+			     <0 65 4>,
+			     <0 66 4>,
+			     <0 67 4>,
+			     <0 68 4>,
+			     <0 69 4>,
+			     <0 70 4>,
+			     <0 71 4>;
+		#dma-cells = <1>;
+		#stream-id-cells = <2>;
+		clocks = <&oscclk6a>;
+		clock-names = "apb_pclk";
+	};
+
+	dma1: dma@2c0a1000 {
+		compatible = "arm,pl330", "arm,primecell";
+		reg = <0 0x2c0a1000 0 0x1000>;
+		interrupts = <0 96 4>,
+			     <0 97 4>,
+			     <0 98 4>,
+			     <0 99 4>,
+			     <0 100 4>,
+			     <0 101 4>,
+			     <0 102 4>,
+			     <0 103 4>;
+		#dma-cells = <1>;
+		#stream-id-cells = <2>;
+		clocks = <&oscclk6a>;
+		clock-names = "apb_pclk";
+	};
+
+	dma2: dma@2c0a2000 {
+		compatible = "arm,pl330", "arm,primecell";
+		reg = <0 0x2c0a2000 0 0x1000>;
+		interrupts = <0 128 4>,
+			     <0 129 4>,
+			     <0 130 4>,
+			     <0 131 4>,
+			     <0 132 4>,
+			     <0 133 4>,
+			     <0 134 4>,
+			     <0 135 4>;
+		#dma-cells = <1>;
+		#stream-id-cells = <2>;
+		clocks = <&oscclk6a>;
+		clock-names = "apb_pclk";
+	};
+
+	dma3: dma@2c0a3000 {
+		compatible = "arm,pl330", "arm,primecell";
+		reg = <0 0x2c0a3000 0 0x1000>;
+		interrupts = <0 160 4>,
+			     <0 161 4>,
+			     <0 162 4>,
+			     <0 163 4>,
+			     <0 164 4>,
+			     <0 165 4>,
+			     <0 166 4>,
+			     <0 167 4>;
+		#dma-cells = <1>;
+		#stream-id-cells = <2>;
+		clocks = <&oscclk6a>;
+		clock-names = "apb_pclk";
+	};
+
+	oscclk6a: oscclk6a {
+		/* Reference 24MHz clock */
+		compatible = "fixed-clock";
+		#clock-cells = <0>;
+		clock-frequency = <24000000>;
+		clock-output-names = "oscclk6a";
+	};
+
 	smb {
 		compatible = "simple-bus";
 

fast_models/rtsm_ve-cortex_a15x4_a7x4.dts line 247

Quoted 4 lines
fast_models/rtsm_ve-cortex_a15x4_a7x4.dts +0 -1
@@ -244,4 +339,3 @@
 		/include/ "rtsm_ve-motherboard.dtsi"
 	};
 };
-

--
Shannon

Quoted 4 lines

_______________________________________________
kvmarm mailing list [email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm

.

#13 lore raw
In reply to a message not in the archive
Quoted 69 lines

-----Original Message-----
From: Eric Auger [mailto:[email protected]]
Sent: Thursday, June 19, 2014 3:41 PM
To: Shannon Zhao; Bhushan Bharat-R65777; Alvise Rigo; a.motakis-lrHrjnjw1UfHK3s98zE1ajGjJy/[email protected]; iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org; [email protected]
Subject: Re: Vfio-on-arm: the device tree does not contain the DMA device

Quoted 22 lines · Shannon Zhao

Hi Bharat,

Quoted 19 lines · On 2014/6/19 17:19, [email protected]…
Quoted 19 lines

-----Original Message-----
From: [email protected] [mailto:kvmarm- [email protected]] On Behalf Of Shannon Zhao
Sent: Thursday, June 19, 2014 2:45 PM
To: Alvise Rigo; a.motakis-lrHrjnjw1UfHK3s98zE1ajGjJy/[email protected]; iommu-cunTk1MwBs/[email protected] foundation.org; [email protected]
Subject: Re: Vfio-on-arm: the device tree does not contain the DMA device

Hi Alvise,

Thanks for your reply.
I have apply the patch and could see the DMA devices.
But there is no vosys_dmatest int the directory "/sys/kernel/debug/" after mounting debugfs.
I found that the branch vfio-platform-v4 doesn't contain the module vosys_dmatest.
Then I port it from the branch pl330_dmatest and run vosys_dmatest

successfully.

Quoted 18 lines
Quoted 18 lines
Quoted 17 lines
	echo 1 > /sys/kernel/debug/vosys_dmatest/start

	[ 6145.343419] dmaengine: __dma_request_channel: success (dma0chan0)
	[ 6145.344000] vosys dmatest: callback: transfer completed
	[ 6145.344147] vosys dmatest: tx completed
	[ 6145.344589] vosys dmatest: source and destination match.

Download the qemu and use following command to start a guest:

	./qemu-system-arm -enable-kvm -M vexpress-a15 -cpu cortex-a15 \
        	-device pl330,vfio_group="/dev/vfio/0",vfio_device="2c0a0000.dma",\
        	vfio_container="/dev/vfio/vfio" \
        	-append "earlyprintk ip=dhcp console=ttyAMA0 rootwait" \
        	-kernel zImage -dtb guest.dtb -initrd filesystem.cpio.gz  -

nographic An error reported:
qemu-system-arm: -device
pl330,vfio_group=/dev/vfio/0,vfio_device=2c0a0000.dma,:

Quoted 2 lines
Quoted 1 line

Why do we need to give "vfio_group=/dev/vfio/0" ?

I refer to the url

"http://www.virtualopensystems.com/en/solutions/guides/vfio-on-arm/".

Quoted 6 lines
Quoted 6 lines
Quoted 4 lines

could not open disk image vfio_container=/dev/vfio:
Could not open 'vfio_container=/dev/vfio': No such file or directory

Are you unbinding "2c0a0000.dma" from its driver and binding with "vfio-

playform" driver ?

Quoted 4 lines

How to unbind "2c0a0000.dma" from its driver?
I have tried following cmd to bind "2c0a0000.dma" with "vfio-playform" driver.
echo 2c0a0000.dma > /sys/bus/platform/drivers/vfio-platform/vfio_bind
But there is no /dev/vfio/0 or /dev/vfio/vfio .

Hi Shannon,

Quoted 7 lines

you may try following cmd to unbind the native dma driver echo 2c0a0000.dma | sudo tee -a /sys/bus/platform/devices/2c0a0000.dma/driver/unbind

and then the following one to bing the meta-driver to the device echo 2c0a0000.dma | sudo tee -a /sys/bus/platform/drivers/vfio-platform/vfio_bind

Should not we use "driver-override" mechanism for binding with vfio-platform driver?

Thanks
-Bharat

Quoted 179 lines

Best Regards

Eric

Quoted 8 lines
Quoted 8 lines

Thanks
-Bharat

Quoted 3 lines

Is there something I'm wrong or do I miss some steps?

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines

Thank you,
Shannon

Quoted 3 lines · Alvise Rigo

Hi Shannon,

Thank you for reporting this issue, the device tree file was indeed

missing.

Quoted 158 lines
Quoted 157 lines
Quoted 150 lines
Quoted 142 lines

To solve the problem, you can apply the following patch to the repository you mentioned
(git://github.com/virtualopensystems/arm-dts.git).

For your information, soon we will update the guide with a newer version of VFIO with this issue fixed.

Thank you,
alvise

---
 fast_models/rtsm_ve-cortex_a15x4_a7x4.dts | 100
+++++++++++++++++++++++++++++-
 1 file changed, 97 insertions(+), 3 deletions(-)
fast_models/rtsm_ve-cortex_a15x4_a7x4.dts +97 -2
index ef1275b..79c6af2 100644
@@ -142,7 +142,7 @@
 		cpu7: cpu@7 {
 			device_type = "cpu";
 			compatible = "arm,cortex-a7";
-			reg = <0x104>;
+			reg = <0x103>;
 			cluster = <&cluster1>;
 			core = <&core7>;
 		};
@@ -159,7 +159,7 @@
 		#address-cells = <0>;
 		interrupt-controller;
 		reg = <0 0x2c001000 0 0x1000>,
-		      <0 0x2c002000 0 0x2000>,
+		      <0 0x2c002000 0 0x1000>,
 		      <0 0x2c004000 0 0x2000>,
 		      <0 0x2c006000 0 0x2000>;
 		interrupts = <1 9 0xf04>;
@@ -183,6 +183,101 @@
 			     <1 10 0xf08>;
 	};
 
+	smmu@2c0b0000 {
+                compatible = "arm,smmu-v2";
+                reg = <0x0 0x2c0b0000 0x0 0x10000>;
+                #global-interrupts = <2>;
+                interrupts = <0 58 4>,
+                             <0 59 4>,
+                             <0 60 4>,
+                             <0 60 4>,
+                             <0 60 4>,
+                             <0 60 4>,
+                             <0 60 4>,
+                             <0 60 4>,
+                             <0 60 4>;
+                mmu-masters = <&dma0 0x0 0x8>,
+                              <&dma1 0x1 0x9>,
+                              <&dma2 0x2 0xa>,
+                              <&dma3 0x3 0xb>;
+        };
+
+	dma0: dma@2c0a0000 {
+		compatible = "arm,pl330";
+		reg = <0 0x2c0a0000 0 0x1000>;
+		interrupts = <0 64 4>,
+			     <0 65 4>,
+			     <0 66 4>,
+			     <0 67 4>,
+			     <0 68 4>,
+			     <0 69 4>,
+			     <0 70 4>,
+			     <0 71 4>;
+		#dma-cells = <1>;
+		#stream-id-cells = <2>;
+		clocks = <&oscclk6a>;
+		clock-names = "apb_pclk";
+	};
+
+	dma1: dma@2c0a1000 {
+		compatible = "arm,pl330", "arm,primecell";
+		reg = <0 0x2c0a1000 0 0x1000>;
+		interrupts = <0 96 4>,
+			     <0 97 4>,
+			     <0 98 4>,
+			     <0 99 4>,
+			     <0 100 4>,
+			     <0 101 4>,
+			     <0 102 4>,
+			     <0 103 4>;
+		#dma-cells = <1>;
+		#stream-id-cells = <2>;
+		clocks = <&oscclk6a>;
+		clock-names = "apb_pclk";
+	};
+
+	dma2: dma@2c0a2000 {
+		compatible = "arm,pl330", "arm,primecell";
+		reg = <0 0x2c0a2000 0 0x1000>;
+		interrupts = <0 128 4>,
+			     <0 129 4>,
+			     <0 130 4>,
+			     <0 131 4>,
+			     <0 132 4>,
+			     <0 133 4>,
+			     <0 134 4>,
+			     <0 135 4>;
+		#dma-cells = <1>;
+		#stream-id-cells = <2>;
+		clocks = <&oscclk6a>;
+		clock-names = "apb_pclk";
+	};
+
+	dma3: dma@2c0a3000 {
+		compatible = "arm,pl330", "arm,primecell";
+		reg = <0 0x2c0a3000 0 0x1000>;
+		interrupts = <0 160 4>,
+			     <0 161 4>,
+			     <0 162 4>,
+			     <0 163 4>,
+			     <0 164 4>,
+			     <0 165 4>,
+			     <0 166 4>,
+			     <0 167 4>;
+		#dma-cells = <1>;
+		#stream-id-cells = <2>;
+		clocks = <&oscclk6a>;
+		clock-names = "apb_pclk";
+	};
+
+	oscclk6a: oscclk6a {
+		/* Reference 24MHz clock */
+		compatible = "fixed-clock";
+		#clock-cells = <0>;
+		clock-frequency = <24000000>;
+		clock-output-names = "oscclk6a";
+	};
+
 	smb {
 		compatible = "simple-bus";
 

fast_models/rtsm_ve-cortex_a15x4_a7x4.dts line 247

Quoted 4 lines
fast_models/rtsm_ve-cortex_a15x4_a7x4.dts +0 -1
@@ -244,4 +339,3 @@
 		/include/ "rtsm_ve-motherboard.dtsi"
 	};
 };
-

--
Shannon

Quoted 4 lines

_______________________________________________
kvmarm mailing list [email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm

.

#15 lore raw
In reply to a message not in the archive
Quoted 73 lines · On 06/19/2014 12:15 PM, [email protected]…
Quoted 73 lines

-----Original Message-----
From: Eric Auger [mailto:[email protected]]
Sent: Thursday, June 19, 2014 3:41 PM
To: Shannon Zhao; Bhushan Bharat-R65777; Alvise Rigo; a.motakis-lrHrjnjw1UfHK3s98zE1ajGjJy/[email protected]; iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org; [email protected]
Subject: Re: Vfio-on-arm: the device tree does not contain the DMA device

Quoted 22 lines · Shannon Zhao

Hi Bharat,

Quoted 19 lines · On 2014/6/19 17:19, [email protected]…
Quoted 19 lines

-----Original Message-----
From: [email protected] [mailto:kvmarm- [email protected]] On Behalf Of Shannon Zhao
Sent: Thursday, June 19, 2014 2:45 PM
To: Alvise Rigo; a.motakis-lrHrjnjw1UfHK3s98zE1ajGjJy/[email protected]; iommu-cunTk1MwBs/[email protected] foundation.org; [email protected]
Subject: Re: Vfio-on-arm: the device tree does not contain the DMA device

Hi Alvise,

Thanks for your reply.
I have apply the patch and could see the DMA devices.
But there is no vosys_dmatest int the directory "/sys/kernel/debug/" after mounting debugfs.
I found that the branch vfio-platform-v4 doesn't contain the module vosys_dmatest.
Then I port it from the branch pl330_dmatest and run vosys_dmatest

successfully.

Quoted 18 lines
Quoted 18 lines
Quoted 17 lines
	echo 1 > /sys/kernel/debug/vosys_dmatest/start

	[ 6145.343419] dmaengine: __dma_request_channel: success (dma0chan0)
	[ 6145.344000] vosys dmatest: callback: transfer completed
	[ 6145.344147] vosys dmatest: tx completed
	[ 6145.344589] vosys dmatest: source and destination match.

Download the qemu and use following command to start a guest:

	./qemu-system-arm -enable-kvm -M vexpress-a15 -cpu cortex-a15 \
        	-device pl330,vfio_group="/dev/vfio/0",vfio_device="2c0a0000.dma",\
        	vfio_container="/dev/vfio/vfio" \
        	-append "earlyprintk ip=dhcp console=ttyAMA0 rootwait" \
        	-kernel zImage -dtb guest.dtb -initrd filesystem.cpio.gz  -

nographic An error reported:
qemu-system-arm: -device
pl330,vfio_group=/dev/vfio/0,vfio_device=2c0a0000.dma,:

Quoted 2 lines
Quoted 1 line

Why do we need to give "vfio_group=/dev/vfio/0" ?

I refer to the url

"http://www.virtualopensystems.com/en/solutions/guides/vfio-on-arm/".

Quoted 6 lines
Quoted 6 lines
Quoted 4 lines

could not open disk image vfio_container=/dev/vfio:
Could not open 'vfio_container=/dev/vfio': No such file or directory

Are you unbinding "2c0a0000.dma" from its driver and binding with "vfio-

playform" driver ?

Quoted 4 lines

How to unbind "2c0a0000.dma" from its driver?
I have tried following cmd to bind "2c0a0000.dma" with "vfio-playform" driver.
echo 2c0a0000.dma > /sys/bus/platform/drivers/vfio-platform/vfio_bind
But there is no /dev/vfio/0 or /dev/vfio/vfio .

Hi Shannon,

you may try following cmd to unbind the native dma driver echo 2c0a0000.dma | sudo tee -a /sys/bus/platform/devices/2c0a0000.dma/driver/unbind

Quoted 8 lines
Quoted 3 lines

and then the following one to bing the meta-driver to the device echo 2c0a0000.dma | sudo tee -a /sys/bus/platform/drivers/vfio-platform/vfio_bind

Should not we use "driver-override" mechanism for binding with vfio-platform driver?

Thanks
-Bharat

Hi Bharat,

I guess Shannon is using an older version of the kernel. With vfio-platform v4 this is the old unbind/bin mechanism that is in place, if I am not wrong.

Best Regards

Eric

Quoted 181 lines
Quoted 180 lines

Best Regards

Eric

Quoted 8 lines
Quoted 8 lines

Thanks
-Bharat

Quoted 3 lines

Is there something I'm wrong or do I miss some steps?

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines

Thank you,
Shannon

Quoted 3 lines · Alvise Rigo

Hi Shannon,

Thank you for reporting this issue, the device tree file was indeed

missing.

Quoted 158 lines
Quoted 157 lines
Quoted 150 lines
Quoted 142 lines

To solve the problem, you can apply the following patch to the repository you mentioned
(git://github.com/virtualopensystems/arm-dts.git).

For your information, soon we will update the guide with a newer version of VFIO with this issue fixed.

Thank you,
alvise

---
 fast_models/rtsm_ve-cortex_a15x4_a7x4.dts | 100
+++++++++++++++++++++++++++++-
 1 file changed, 97 insertions(+), 3 deletions(-)
fast_models/rtsm_ve-cortex_a15x4_a7x4.dts +97 -2
index ef1275b..79c6af2 100644
@@ -142,7 +142,7 @@
 		cpu7: cpu@7 {
 			device_type = "cpu";
 			compatible = "arm,cortex-a7";
-			reg = <0x104>;
+			reg = <0x103>;
 			cluster = <&cluster1>;
 			core = <&core7>;
 		};
@@ -159,7 +159,7 @@
 		#address-cells = <0>;
 		interrupt-controller;
 		reg = <0 0x2c001000 0 0x1000>,
-		      <0 0x2c002000 0 0x2000>,
+		      <0 0x2c002000 0 0x1000>,
 		      <0 0x2c004000 0 0x2000>,
 		      <0 0x2c006000 0 0x2000>;
 		interrupts = <1 9 0xf04>;
@@ -183,6 +183,101 @@
 			     <1 10 0xf08>;
 	};
 
+	smmu@2c0b0000 {
+                compatible = "arm,smmu-v2";
+                reg = <0x0 0x2c0b0000 0x0 0x10000>;
+                #global-interrupts = <2>;
+                interrupts = <0 58 4>,
+                             <0 59 4>,
+                             <0 60 4>,
+                             <0 60 4>,
+                             <0 60 4>,
+                             <0 60 4>,
+                             <0 60 4>,
+                             <0 60 4>,
+                             <0 60 4>;
+                mmu-masters = <&dma0 0x0 0x8>,
+                              <&dma1 0x1 0x9>,
+                              <&dma2 0x2 0xa>,
+                              <&dma3 0x3 0xb>;
+        };
+
+	dma0: dma@2c0a0000 {
+		compatible = "arm,pl330";
+		reg = <0 0x2c0a0000 0 0x1000>;
+		interrupts = <0 64 4>,
+			     <0 65 4>,
+			     <0 66 4>,
+			     <0 67 4>,
+			     <0 68 4>,
+			     <0 69 4>,
+			     <0 70 4>,
+			     <0 71 4>;
+		#dma-cells = <1>;
+		#stream-id-cells = <2>;
+		clocks = <&oscclk6a>;
+		clock-names = "apb_pclk";
+	};
+
+	dma1: dma@2c0a1000 {
+		compatible = "arm,pl330", "arm,primecell";
+		reg = <0 0x2c0a1000 0 0x1000>;
+		interrupts = <0 96 4>,
+			     <0 97 4>,
+			     <0 98 4>,
+			     <0 99 4>,
+			     <0 100 4>,
+			     <0 101 4>,
+			     <0 102 4>,
+			     <0 103 4>;
+		#dma-cells = <1>;
+		#stream-id-cells = <2>;
+		clocks = <&oscclk6a>;
+		clock-names = "apb_pclk";
+	};
+
+	dma2: dma@2c0a2000 {
+		compatible = "arm,pl330", "arm,primecell";
+		reg = <0 0x2c0a2000 0 0x1000>;
+		interrupts = <0 128 4>,
+			     <0 129 4>,
+			     <0 130 4>,
+			     <0 131 4>,
+			     <0 132 4>,
+			     <0 133 4>,
+			     <0 134 4>,
+			     <0 135 4>;
+		#dma-cells = <1>;
+		#stream-id-cells = <2>;
+		clocks = <&oscclk6a>;
+		clock-names = "apb_pclk";
+	};
+
+	dma3: dma@2c0a3000 {
+		compatible = "arm,pl330", "arm,primecell";
+		reg = <0 0x2c0a3000 0 0x1000>;
+		interrupts = <0 160 4>,
+			     <0 161 4>,
+			     <0 162 4>,
+			     <0 163 4>,
+			     <0 164 4>,
+			     <0 165 4>,
+			     <0 166 4>,
+			     <0 167 4>;
+		#dma-cells = <1>;
+		#stream-id-cells = <2>;
+		clocks = <&oscclk6a>;
+		clock-names = "apb_pclk";
+	};
+
+	oscclk6a: oscclk6a {
+		/* Reference 24MHz clock */
+		compatible = "fixed-clock";
+		#clock-cells = <0>;
+		clock-frequency = <24000000>;
+		clock-output-names = "oscclk6a";
+	};
+
 	smb {
 		compatible = "simple-bus";
 

fast_models/rtsm_ve-cortex_a15x4_a7x4.dts line 247

Quoted 4 lines
fast_models/rtsm_ve-cortex_a15x4_a7x4.dts +0 -1
@@ -244,4 +339,3 @@
 		/include/ "rtsm_ve-motherboard.dtsi"
 	};
 };
-

--
Shannon

Quoted 4 lines

_______________________________________________
kvmarm mailing list [email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm

.

#18 lore raw
In reply to a message not in the archive

Hi Eric,

Thanks for your reply.
There is no such directory "/sys/bus/platform/devices/2c0a0000.dma/driver/".

Quoted 245 lines · Eric Auger
Quoted 44 lines · Shannon Zhao

Hi Bharat,

Quoted 37 lines · On 2014/6/19 17:19, [email protected]…
Quoted 28 lines

-----Original Message-----
From: [email protected] [mailto:kvmarm- [email protected]] On Behalf Of Shannon Zhao
Sent: Thursday, June 19, 2014 2:45 PM
To: Alvise Rigo; a.motakis-lrHrjnjw1UfHK3s98zE1ajGjJy/[email protected]; iommu-cunTk1MwBs/[email protected] foundation.org; [email protected]
Subject: Re: Vfio-on-arm: the device tree does not contain the DMA device

Hi Alvise,

Thanks for your reply.
I have apply the patch and could see the DMA devices.
But there is no vosys_dmatest int the directory "/sys/kernel/debug/" after mounting debugfs.
I found that the branch vfio-platform-v4 doesn't contain the module vosys_dmatest.
Then I port it from the branch pl330_dmatest and run vosys_dmatest successfully.

	echo 1 > /sys/kernel/debug/vosys_dmatest/start

	[ 6145.343419] dmaengine: __dma_request_channel: success (dma0chan0)
	[ 6145.344000] vosys dmatest: callback: transfer completed
	[ 6145.344147] vosys dmatest: tx completed
	[ 6145.344589] vosys dmatest: source and destination match.

Download the qemu and use following command to start a guest:

Quoted 8 lines
	./qemu-system-arm -enable-kvm -M vexpress-a15 -cpu cortex-a15 \
        	-device pl330,vfio_group="/dev/vfio/0",vfio_device="2c0a0000.dma",\
        	vfio_container="/dev/vfio/vfio" \
        	-append "earlyprintk ip=dhcp console=ttyAMA0 rootwait" \
        	-kernel zImage -dtb guest.dtb -initrd filesystem.cpio.gz  -

nographic An error reported:
qemu-system-arm: -device
pl330,vfio_group=/dev/vfio/0,vfio_device=2c0a0000.dma,:

Quoted 1 line

Why do we need to give "vfio_group=/dev/vfio/0" ?

I refer to the url "http://www.virtualopensystems.com/en/solutions/guides/vfio-on-arm/".

Quoted 2 lines
Quoted 8 lines
Quoted 4 lines
Quoted 2 lines

could not open disk image vfio_container=/dev/vfio:
Could not open 'vfio_container=/dev/vfio': No such file or directory

Are you unbinding "2c0a0000.dma" from its driver and binding with "vfio-playform" driver ?

How to unbind "2c0a0000.dma" from its driver?
I have tried following cmd to bind "2c0a0000.dma" with "vfio-playform" driver.
echo 2c0a0000.dma > /sys/bus/platform/drivers/vfio-platform/vfio_bind
But there is no /dev/vfio/0 or /dev/vfio/vfio .

Hi Shannon,

you may try following cmd to unbind the native dma driver echo 2c0a0000.dma | sudo tee -a /sys/bus/platform/devices/2c0a0000.dma/driver/unbind

and then the following one to bing the meta-driver to the device echo 2c0a0000.dma | sudo tee -a /sys/bus/platform/drivers/vfio-platform/vfio_bind

Best Regards

Eric

Quoted 165 lines
Quoted 165 lines

Thanks
-Bharat

Quoted 160 lines

Is there something I'm wrong or do I miss some steps?

Thank you,
Shannon

Quoted 145 lines · Alvise Rigo

Hi Shannon,

Thank you for reporting this issue, the device tree file was indeed missing. To solve the problem, you can apply the following patch to the repository you mentioned
(git://github.com/virtualopensystems/arm-dts.git).

For your information, soon we will update the guide with a newer version of VFIO with this issue fixed.

Thank you,
alvise

---
 fast_models/rtsm_ve-cortex_a15x4_a7x4.dts | 100
+++++++++++++++++++++++++++++-
 1 file changed, 97 insertions(+), 3 deletions(-)
fast_models/rtsm_ve-cortex_a15x4_a7x4.dts +97 -2
index ef1275b..79c6af2 100644
@@ -142,7 +142,7 @@
 		cpu7: cpu@7 {
 			device_type = "cpu";
 			compatible = "arm,cortex-a7";
-			reg = <0x104>;
+			reg = <0x103>;
 			cluster = <&cluster1>;
 			core = <&core7>;
 		};
@@ -159,7 +159,7 @@
 		#address-cells = <0>;
 		interrupt-controller;
 		reg = <0 0x2c001000 0 0x1000>,
-		      <0 0x2c002000 0 0x2000>,
+		      <0 0x2c002000 0 0x1000>,
 		      <0 0x2c004000 0 0x2000>,
 		      <0 0x2c006000 0 0x2000>;
 		interrupts = <1 9 0xf04>;
@@ -183,6 +183,101 @@
 			     <1 10 0xf08>;
 	};
 
+	smmu@2c0b0000 {
+                compatible = "arm,smmu-v2";
+                reg = <0x0 0x2c0b0000 0x0 0x10000>;
+                #global-interrupts = <2>;
+                interrupts = <0 58 4>,
+                             <0 59 4>,
+                             <0 60 4>,
+                             <0 60 4>,
+                             <0 60 4>,
+                             <0 60 4>,
+                             <0 60 4>,
+                             <0 60 4>,
+                             <0 60 4>;
+                mmu-masters = <&dma0 0x0 0x8>,
+                              <&dma1 0x1 0x9>,
+                              <&dma2 0x2 0xa>,
+                              <&dma3 0x3 0xb>;
+        };
+
+	dma0: dma@2c0a0000 {
+		compatible = "arm,pl330";
+		reg = <0 0x2c0a0000 0 0x1000>;
+		interrupts = <0 64 4>,
+			     <0 65 4>,
+			     <0 66 4>,
+			     <0 67 4>,
+			     <0 68 4>,
+			     <0 69 4>,
+			     <0 70 4>,
+			     <0 71 4>;
+		#dma-cells = <1>;
+		#stream-id-cells = <2>;
+		clocks = <&oscclk6a>;
+		clock-names = "apb_pclk";
+	};
+
+	dma1: dma@2c0a1000 {
+		compatible = "arm,pl330", "arm,primecell";
+		reg = <0 0x2c0a1000 0 0x1000>;
+		interrupts = <0 96 4>,
+			     <0 97 4>,
+			     <0 98 4>,
+			     <0 99 4>,
+			     <0 100 4>,
+			     <0 101 4>,
+			     <0 102 4>,
+			     <0 103 4>;
+		#dma-cells = <1>;
+		#stream-id-cells = <2>;
+		clocks = <&oscclk6a>;
+		clock-names = "apb_pclk";
+	};
+
+	dma2: dma@2c0a2000 {
+		compatible = "arm,pl330", "arm,primecell";
+		reg = <0 0x2c0a2000 0 0x1000>;
+		interrupts = <0 128 4>,
+			     <0 129 4>,
+			     <0 130 4>,
+			     <0 131 4>,
+			     <0 132 4>,
+			     <0 133 4>,
+			     <0 134 4>,
+			     <0 135 4>;
+		#dma-cells = <1>;
+		#stream-id-cells = <2>;
+		clocks = <&oscclk6a>;
+		clock-names = "apb_pclk";
+	};
+
+	dma3: dma@2c0a3000 {
+		compatible = "arm,pl330", "arm,primecell";
+		reg = <0 0x2c0a3000 0 0x1000>;
+		interrupts = <0 160 4>,
+			     <0 161 4>,
+			     <0 162 4>,
+			     <0 163 4>,
+			     <0 164 4>,
+			     <0 165 4>,
+			     <0 166 4>,
+			     <0 167 4>;
+		#dma-cells = <1>;
+		#stream-id-cells = <2>;
+		clocks = <&oscclk6a>;
+		clock-names = "apb_pclk";
+	};
+
+	oscclk6a: oscclk6a {
+		/* Reference 24MHz clock */
+		compatible = "fixed-clock";
+		#clock-cells = <0>;
+		clock-frequency = <24000000>;
+		clock-output-names = "oscclk6a";
+	};
+
 	smb {
 		compatible = "simple-bus";
 

fast_models/rtsm_ve-cortex_a15x4_a7x4.dts line 247

Quoted 4 lines
fast_models/rtsm_ve-cortex_a15x4_a7x4.dts +0 -1
@@ -244,4 +339,3 @@
 		/include/ "rtsm_ve-motherboard.dtsi"
 	};
 };
-

--
Shannon

Quoted 8 lines
Quoted 7 lines
Quoted 4 lines

_______________________________________________
kvmarm mailing list [email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm

.

.

Signature · 1 line
Shannon
#19 lore raw
In reply to a message not in the archive

Hi,

Il 19/06/2014 11:38, Shannon Zhao ha scritto:

Quoted 44 lines

Hi Bharat,

Quoted 37 lines · On 2014/6/19 17:19, [email protected]…
Quoted 28 lines

-----Original Message-----
From: [email protected] [mailto:kvmarm- [email protected]] On Behalf Of Shannon Zhao
Sent: Thursday, June 19, 2014 2:45 PM
To: Alvise Rigo; a.motakis-lrHrjnjw1UfHK3s98zE1ajGjJy/[email protected]; iommu-cunTk1MwBs/[email protected] foundation.org; [email protected]
Subject: Re: Vfio-on-arm: the device tree does not contain the DMA device

Hi Alvise,

Thanks for your reply.
I have apply the patch and could see the DMA devices.
But there is no vosys_dmatest int the directory "/sys/kernel/debug/" after mounting debugfs.
I found that the branch vfio-platform-v4 doesn't contain the module vosys_dmatest.
Then I port it from the branch pl330_dmatest and run vosys_dmatest successfully.

	echo 1 > /sys/kernel/debug/vosys_dmatest/start

	[ 6145.343419] dmaengine: __dma_request_channel: success (dma0chan0)
	[ 6145.344000] vosys dmatest: callback: transfer completed
	[ 6145.344147] vosys dmatest: tx completed
	[ 6145.344589] vosys dmatest: source and destination match.

Download the qemu and use following command to start a guest:

Quoted 8 lines
	./qemu-system-arm -enable-kvm -M vexpress-a15 -cpu cortex-a15 \
        	-device pl330,vfio_group="/dev/vfio/0",vfio_device="2c0a0000.dma",\
        	vfio_container="/dev/vfio/vfio" \
        	-append "earlyprintk ip=dhcp console=ttyAMA0 rootwait" \
        	-kernel zImage -dtb guest.dtb -initrd filesystem.cpio.gz  -

nographic An error reported:
qemu-system-arm: -device
pl330,vfio_group=/dev/vfio/0,vfio_device=2c0a0000.dma,:

Quoted 1 line

Why do we need to give "vfio_group=/dev/vfio/0" ?

I refer to the url "http://www.virtualopensystems.com/en/solutions/guides/vfio-on-arm/".

Quoted 2 lines
Quoted 8 lines
Quoted 4 lines
Quoted 2 lines

could not open disk image vfio_container=/dev/vfio:
Could not open 'vfio_container=/dev/vfio': No such file or directory

Are you unbinding "2c0a0000.dma" from its driver and binding with "vfio-playform" driver ?

How to unbind "2c0a0000.dma" from its driver?
I have tried following cmd to bind "2c0a0000.dma" with "vfio-playform" driver.
echo 2c0a0000.dma > /sys/bus/platform/drivers/vfio-platform/vfio_bind
But there is no /dev/vfio/0 or /dev/vfio/vfio .

Can you please verify that while configuring the host you enabled:
- Device Drivers -> IOMMU Hardware Support -> ARM Ltd. System MMU (SMMU) Support
and
- Device Drivers -> VFIO Non-Privileged userspace driver framework -> VFIO support for platform devices

Then try to update the pl330-vfio-dev branch and run QEMU with the following command:

./qemu-system-arm -enable-kvm -M vexpress-a15 -cpu cortex-a15 \
    -device pl330,vfio_group="/dev/vfio/0",vfio_device="2c0a0000.dma",vfio_container="/dev/vfio/vfio" \
    -append "earlyprintk ip=dhcp console=ttyAMA0 rootwait" \
    -kernel zImage -dtb guest.dtb -initrd filesystem.cpio.gz

that means without breaking the "-device pl330" options.

Regards,
alvise

Quoted 173 lines
Quoted 172 lines

Thanks
-Bharat

Quoted 160 lines

Is there something I'm wrong or do I miss some steps?

Thank you,
Shannon

Quoted 145 lines · Alvise Rigo

Hi Shannon,

Thank you for reporting this issue, the device tree file was indeed missing. To solve the problem, you can apply the following patch to the repository you mentioned
(git://github.com/virtualopensystems/arm-dts.git).

For your information, soon we will update the guide with a newer version of VFIO with this issue fixed.

Thank you,
alvise

---
 fast_models/rtsm_ve-cortex_a15x4_a7x4.dts | 100
+++++++++++++++++++++++++++++-
 1 file changed, 97 insertions(+), 3 deletions(-)
fast_models/rtsm_ve-cortex_a15x4_a7x4.dts +97 -2
index ef1275b..79c6af2 100644
@@ -142,7 +142,7 @@
 		cpu7: cpu@7 {
 			device_type = "cpu";
 			compatible = "arm,cortex-a7";
-			reg = <0x104>;
+			reg = <0x103>;
 			cluster = <&cluster1>;
 			core = <&core7>;
 		};
@@ -159,7 +159,7 @@
 		#address-cells = <0>;
 		interrupt-controller;
 		reg = <0 0x2c001000 0 0x1000>,
-		      <0 0x2c002000 0 0x2000>,
+		      <0 0x2c002000 0 0x1000>,
 		      <0 0x2c004000 0 0x2000>,
 		      <0 0x2c006000 0 0x2000>;
 		interrupts = <1 9 0xf04>;
@@ -183,6 +183,101 @@
 			     <1 10 0xf08>;
 	};
 
+	smmu@2c0b0000 {
+                compatible = "arm,smmu-v2";
+                reg = <0x0 0x2c0b0000 0x0 0x10000>;
+                #global-interrupts = <2>;
+                interrupts = <0 58 4>,
+                             <0 59 4>,
+                             <0 60 4>,
+                             <0 60 4>,
+                             <0 60 4>,
+                             <0 60 4>,
+                             <0 60 4>,
+                             <0 60 4>,
+                             <0 60 4>;
+                mmu-masters = <&dma0 0x0 0x8>,
+                              <&dma1 0x1 0x9>,
+                              <&dma2 0x2 0xa>,
+                              <&dma3 0x3 0xb>;
+        };
+
+	dma0: dma@2c0a0000 {
+		compatible = "arm,pl330";
+		reg = <0 0x2c0a0000 0 0x1000>;
+		interrupts = <0 64 4>,
+			     <0 65 4>,
+			     <0 66 4>,
+			     <0 67 4>,
+			     <0 68 4>,
+			     <0 69 4>,
+			     <0 70 4>,
+			     <0 71 4>;
+		#dma-cells = <1>;
+		#stream-id-cells = <2>;
+		clocks = <&oscclk6a>;
+		clock-names = "apb_pclk";
+	};
+
+	dma1: dma@2c0a1000 {
+		compatible = "arm,pl330", "arm,primecell";
+		reg = <0 0x2c0a1000 0 0x1000>;
+		interrupts = <0 96 4>,
+			     <0 97 4>,
+			     <0 98 4>,
+			     <0 99 4>,
+			     <0 100 4>,
+			     <0 101 4>,
+			     <0 102 4>,
+			     <0 103 4>;
+		#dma-cells = <1>;
+		#stream-id-cells = <2>;
+		clocks = <&oscclk6a>;
+		clock-names = "apb_pclk";
+	};
+
+	dma2: dma@2c0a2000 {
+		compatible = "arm,pl330", "arm,primecell";
+		reg = <0 0x2c0a2000 0 0x1000>;
+		interrupts = <0 128 4>,
+			     <0 129 4>,
+			     <0 130 4>,
+			     <0 131 4>,
+			     <0 132 4>,
+			     <0 133 4>,
+			     <0 134 4>,
+			     <0 135 4>;
+		#dma-cells = <1>;
+		#stream-id-cells = <2>;
+		clocks = <&oscclk6a>;
+		clock-names = "apb_pclk";
+	};
+
+	dma3: dma@2c0a3000 {
+		compatible = "arm,pl330", "arm,primecell";
+		reg = <0 0x2c0a3000 0 0x1000>;
+		interrupts = <0 160 4>,
+			     <0 161 4>,
+			     <0 162 4>,
+			     <0 163 4>,
+			     <0 164 4>,
+			     <0 165 4>,
+			     <0 166 4>,
+			     <0 167 4>;
+		#dma-cells = <1>;
+		#stream-id-cells = <2>;
+		clocks = <&oscclk6a>;
+		clock-names = "apb_pclk";
+	};
+
+	oscclk6a: oscclk6a {
+		/* Reference 24MHz clock */
+		compatible = "fixed-clock";
+		#clock-cells = <0>;
+		clock-frequency = <24000000>;
+		clock-output-names = "oscclk6a";
+	};
+
 	smb {
 		compatible = "simple-bus";
 

fast_models/rtsm_ve-cortex_a15x4_a7x4.dts line 247

Quoted 4 lines
fast_models/rtsm_ve-cortex_a15x4_a7x4.dts +0 -1
@@ -244,4 +339,3 @@
 		/include/ "rtsm_ve-motherboard.dtsi"
 	};
 };
-

--
Shannon

Quoted 4 lines

_______________________________________________
kvmarm mailing list [email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm

.

#17 lore raw
In reply to a message not in the archive

Yes, that is an old guide based on VFIO v4. We will update it soon.

Regards,
alvise

Il 19/06/2014 12:18, Eric Auger ha scritto:

Quoted 274 lines · On 06/19/2014 12:15 PM, [email protected]…
Quoted 73 lines
Quoted 73 lines

-----Original Message-----
From: Eric Auger [mailto:[email protected]]
Sent: Thursday, June 19, 2014 3:41 PM
To: Shannon Zhao; Bhushan Bharat-R65777; Alvise Rigo; a.motakis-lrHrjnjw1UfHK3s98zE1ajGjJy/[email protected]; iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org; [email protected]
Subject: Re: Vfio-on-arm: the device tree does not contain the DMA device

Quoted 22 lines · Shannon Zhao

Hi Bharat,

Quoted 19 lines · On 2014/6/19 17:19, [email protected]…
Quoted 19 lines

-----Original Message-----
From: [email protected] [mailto:kvmarm- [email protected]] On Behalf Of Shannon Zhao
Sent: Thursday, June 19, 2014 2:45 PM
To: Alvise Rigo; a.motakis-lrHrjnjw1UfHK3s98zE1ajGjJy/[email protected]; iommu-cunTk1MwBs/[email protected] foundation.org; [email protected]
Subject: Re: Vfio-on-arm: the device tree does not contain the DMA device

Hi Alvise,

Thanks for your reply.
I have apply the patch and could see the DMA devices.
But there is no vosys_dmatest int the directory "/sys/kernel/debug/" after mounting debugfs.
I found that the branch vfio-platform-v4 doesn't contain the module vosys_dmatest.
Then I port it from the branch pl330_dmatest and run vosys_dmatest

successfully.

Quoted 18 lines
Quoted 18 lines
Quoted 17 lines
	echo 1 > /sys/kernel/debug/vosys_dmatest/start

	[ 6145.343419] dmaengine: __dma_request_channel: success (dma0chan0)
	[ 6145.344000] vosys dmatest: callback: transfer completed
	[ 6145.344147] vosys dmatest: tx completed
	[ 6145.344589] vosys dmatest: source and destination match.

Download the qemu and use following command to start a guest:

	./qemu-system-arm -enable-kvm -M vexpress-a15 -cpu cortex-a15 \
        	-device pl330,vfio_group="/dev/vfio/0",vfio_device="2c0a0000.dma",\
        	vfio_container="/dev/vfio/vfio" \
        	-append "earlyprintk ip=dhcp console=ttyAMA0 rootwait" \
        	-kernel zImage -dtb guest.dtb -initrd filesystem.cpio.gz  -

nographic An error reported:
qemu-system-arm: -device
pl330,vfio_group=/dev/vfio/0,vfio_device=2c0a0000.dma,:

Quoted 2 lines
Quoted 1 line

Why do we need to give "vfio_group=/dev/vfio/0" ?

I refer to the url

"http://www.virtualopensystems.com/en/solutions/guides/vfio-on-arm/".

Quoted 6 lines
Quoted 6 lines
Quoted 4 lines

could not open disk image vfio_container=/dev/vfio:
Could not open 'vfio_container=/dev/vfio': No such file or directory

Are you unbinding "2c0a0000.dma" from its driver and binding with "vfio-

playform" driver ?

Quoted 4 lines

How to unbind "2c0a0000.dma" from its driver?
I have tried following cmd to bind "2c0a0000.dma" with "vfio-playform" driver.
echo 2c0a0000.dma > /sys/bus/platform/drivers/vfio-platform/vfio_bind
But there is no /dev/vfio/0 or /dev/vfio/vfio .

Hi Shannon,

you may try following cmd to unbind the native dma driver echo 2c0a0000.dma | sudo tee -a /sys/bus/platform/devices/2c0a0000.dma/driver/unbind

Quoted 8 lines
Quoted 3 lines

and then the following one to bing the meta-driver to the device echo 2c0a0000.dma | sudo tee -a /sys/bus/platform/drivers/vfio-platform/vfio_bind

Should not we use "driver-override" mechanism for binding with vfio-platform driver?

Thanks
-Bharat

Hi Bharat,

I guess Shannon is using an older version of the kernel. With vfio-platform v4 this is the old unbind/bin mechanism that is in place, if I am not wrong.

Best Regards

Eric

Quoted 181 lines
Quoted 180 lines

Best Regards

Eric

Quoted 8 lines
Quoted 8 lines

Thanks
-Bharat

Quoted 3 lines

Is there something I'm wrong or do I miss some steps?

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines

Thank you,
Shannon

Quoted 3 lines · Alvise Rigo

Hi Shannon,

Thank you for reporting this issue, the device tree file was indeed

missing.

Quoted 158 lines
Quoted 157 lines
Quoted 150 lines
Quoted 142 lines

To solve the problem, you can apply the following patch to the repository you mentioned
(git://github.com/virtualopensystems/arm-dts.git).

For your information, soon we will update the guide with a newer version of VFIO with this issue fixed.

Thank you,
alvise

---
 fast_models/rtsm_ve-cortex_a15x4_a7x4.dts | 100
+++++++++++++++++++++++++++++-
 1 file changed, 97 insertions(+), 3 deletions(-)
fast_models/rtsm_ve-cortex_a15x4_a7x4.dts +97 -2
index ef1275b..79c6af2 100644
@@ -142,7 +142,7 @@
 		cpu7: cpu@7 {
 			device_type = "cpu";
 			compatible = "arm,cortex-a7";
-			reg = <0x104>;
+			reg = <0x103>;
 			cluster = <&cluster1>;
 			core = <&core7>;
 		};
@@ -159,7 +159,7 @@
 		#address-cells = <0>;
 		interrupt-controller;
 		reg = <0 0x2c001000 0 0x1000>,
-		      <0 0x2c002000 0 0x2000>,
+		      <0 0x2c002000 0 0x1000>,
 		      <0 0x2c004000 0 0x2000>,
 		      <0 0x2c006000 0 0x2000>;
 		interrupts = <1 9 0xf04>;
@@ -183,6 +183,101 @@
 			     <1 10 0xf08>;
 	};
 
+	smmu@2c0b0000 {
+                compatible = "arm,smmu-v2";
+                reg = <0x0 0x2c0b0000 0x0 0x10000>;
+                #global-interrupts = <2>;
+                interrupts = <0 58 4>,
+                             <0 59 4>,
+                             <0 60 4>,
+                             <0 60 4>,
+                             <0 60 4>,
+                             <0 60 4>,
+                             <0 60 4>,
+                             <0 60 4>,
+                             <0 60 4>;
+                mmu-masters = <&dma0 0x0 0x8>,
+                              <&dma1 0x1 0x9>,
+                              <&dma2 0x2 0xa>,
+                              <&dma3 0x3 0xb>;
+        };
+
+	dma0: dma@2c0a0000 {
+		compatible = "arm,pl330";
+		reg = <0 0x2c0a0000 0 0x1000>;
+		interrupts = <0 64 4>,
+			     <0 65 4>,
+			     <0 66 4>,
+			     <0 67 4>,
+			     <0 68 4>,
+			     <0 69 4>,
+			     <0 70 4>,
+			     <0 71 4>;
+		#dma-cells = <1>;
+		#stream-id-cells = <2>;
+		clocks = <&oscclk6a>;
+		clock-names = "apb_pclk";
+	};
+
+	dma1: dma@2c0a1000 {
+		compatible = "arm,pl330", "arm,primecell";
+		reg = <0 0x2c0a1000 0 0x1000>;
+		interrupts = <0 96 4>,
+			     <0 97 4>,
+			     <0 98 4>,
+			     <0 99 4>,
+			     <0 100 4>,
+			     <0 101 4>,
+			     <0 102 4>,
+			     <0 103 4>;
+		#dma-cells = <1>;
+		#stream-id-cells = <2>;
+		clocks = <&oscclk6a>;
+		clock-names = "apb_pclk";
+	};
+
+	dma2: dma@2c0a2000 {
+		compatible = "arm,pl330", "arm,primecell";
+		reg = <0 0x2c0a2000 0 0x1000>;
+		interrupts = <0 128 4>,
+			     <0 129 4>,
+			     <0 130 4>,
+			     <0 131 4>,
+			     <0 132 4>,
+			     <0 133 4>,
+			     <0 134 4>,
+			     <0 135 4>;
+		#dma-cells = <1>;
+		#stream-id-cells = <2>;
+		clocks = <&oscclk6a>;
+		clock-names = "apb_pclk";
+	};
+
+	dma3: dma@2c0a3000 {
+		compatible = "arm,pl330", "arm,primecell";
+		reg = <0 0x2c0a3000 0 0x1000>;
+		interrupts = <0 160 4>,
+			     <0 161 4>,
+			     <0 162 4>,
+			     <0 163 4>,
+			     <0 164 4>,
+			     <0 165 4>,
+			     <0 166 4>,
+			     <0 167 4>;
+		#dma-cells = <1>;
+		#stream-id-cells = <2>;
+		clocks = <&oscclk6a>;
+		clock-names = "apb_pclk";
+	};
+
+	oscclk6a: oscclk6a {
+		/* Reference 24MHz clock */
+		compatible = "fixed-clock";
+		#clock-cells = <0>;
+		clock-frequency = <24000000>;
+		clock-output-names = "oscclk6a";
+	};
+
 	smb {
 		compatible = "simple-bus";
 

fast_models/rtsm_ve-cortex_a15x4_a7x4.dts line 247

Quoted 4 lines
fast_models/rtsm_ve-cortex_a15x4_a7x4.dts +0 -1
@@ -244,4 +339,3 @@
 		/include/ "rtsm_ve-motherboard.dtsi"
 	};
 };
-

--
Shannon

Quoted 4 lines

_______________________________________________
kvmarm mailing list [email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm

.

#21 lore raw
In reply to a message not in the archive
Quoted 57 lines · Alvise Rigo

Hi,

Il 19/06/2014 11:38, Shannon Zhao ha scritto:

Quoted 44 lines

Hi Bharat,

Quoted 37 lines · On 2014/6/19 17:19, [email protected]…
Quoted 28 lines

-----Original Message-----
From: [email protected] [mailto:kvmarm- [email protected]] On Behalf Of Shannon Zhao
Sent: Thursday, June 19, 2014 2:45 PM
To: Alvise Rigo; a.motakis-lrHrjnjw1UfHK3s98zE1ajGjJy/[email protected]; iommu-cunTk1MwBs/[email protected] foundation.org; [email protected]
Subject: Re: Vfio-on-arm: the device tree does not contain the DMA device

Hi Alvise,

Thanks for your reply.
I have apply the patch and could see the DMA devices.
But there is no vosys_dmatest int the directory "/sys/kernel/debug/" after mounting debugfs.
I found that the branch vfio-platform-v4 doesn't contain the module vosys_dmatest.
Then I port it from the branch pl330_dmatest and run vosys_dmatest successfully.

	echo 1 > /sys/kernel/debug/vosys_dmatest/start

	[ 6145.343419] dmaengine: __dma_request_channel: success (dma0chan0)
	[ 6145.344000] vosys dmatest: callback: transfer completed
	[ 6145.344147] vosys dmatest: tx completed
	[ 6145.344589] vosys dmatest: source and destination match.

Download the qemu and use following command to start a guest:

Quoted 8 lines
	./qemu-system-arm -enable-kvm -M vexpress-a15 -cpu cortex-a15 \
        	-device pl330,vfio_group="/dev/vfio/0",vfio_device="2c0a0000.dma",\
        	vfio_container="/dev/vfio/vfio" \
        	-append "earlyprintk ip=dhcp console=ttyAMA0 rootwait" \
        	-kernel zImage -dtb guest.dtb -initrd filesystem.cpio.gz  -

nographic An error reported:
qemu-system-arm: -device
pl330,vfio_group=/dev/vfio/0,vfio_device=2c0a0000.dma,:

Quoted 1 line

Why do we need to give "vfio_group=/dev/vfio/0" ?

I refer to the url "http://www.virtualopensystems.com/en/solutions/guides/vfio-on-arm/".

Quoted 2 lines
Quoted 8 lines
Quoted 4 lines
Quoted 2 lines

could not open disk image vfio_container=/dev/vfio:
Could not open 'vfio_container=/dev/vfio': No such file or directory

Are you unbinding "2c0a0000.dma" from its driver and binding with "vfio-playform" driver ?

How to unbind "2c0a0000.dma" from its driver?
I have tried following cmd to bind "2c0a0000.dma" with "vfio-playform" driver.
echo 2c0a0000.dma > /sys/bus/platform/drivers/vfio-platform/vfio_bind
But there is no /dev/vfio/0 or /dev/vfio/vfio .

Quoted 5 lines

Can you please verify that while configuring the host you enabled:
- Device Drivers -> IOMMU Hardware Support -> ARM Ltd. System MMU (SMMU) Support
and
- Device Drivers -> VFIO Non-Privileged userspace driver framework -> VFIO support for platform devices

Hi Alvise,

I have checked the two configuration are enabled.

Then try the following cmd, but another error reported:
qemu-system-arm: -device pl330,vfio_group=/dev/vfio/0,vfio_device=2c0a0000.dma,vfio_container=/dev/vfio/vfio: Parameter 'driver' expects pluggable device type

Do I need to unbind the "2c0a0000.dma" from its driver and binding with "vfio-platform" driver as Bharat mentioned ?

Quoted 188 lines

Then try to update the pl330-vfio-dev branch and run QEMU with the following command:

./qemu-system-arm -enable-kvm -M vexpress-a15 -cpu cortex-a15 \
    -device pl330,vfio_group="/dev/vfio/0",vfio_device="2c0a0000.dma",vfio_container="/dev/vfio/vfio" \
    -append "earlyprintk ip=dhcp console=ttyAMA0 rootwait" \
    -kernel zImage -dtb guest.dtb -initrd filesystem.cpio.gz

that means without breaking the "-device pl330" options.

Regards,
alvise

Quoted 165 lines
Quoted 165 lines

Thanks
-Bharat

Quoted 160 lines

Is there something I'm wrong or do I miss some steps?

Thank you,
Shannon

Quoted 145 lines · Alvise Rigo

Hi Shannon,

Thank you for reporting this issue, the device tree file was indeed missing. To solve the problem, you can apply the following patch to the repository you mentioned
(git://github.com/virtualopensystems/arm-dts.git).

For your information, soon we will update the guide with a newer version of VFIO with this issue fixed.

Thank you,
alvise

---
 fast_models/rtsm_ve-cortex_a15x4_a7x4.dts | 100
+++++++++++++++++++++++++++++-
 1 file changed, 97 insertions(+), 3 deletions(-)
fast_models/rtsm_ve-cortex_a15x4_a7x4.dts +97 -2
index ef1275b..79c6af2 100644
@@ -142,7 +142,7 @@
 		cpu7: cpu@7 {
 			device_type = "cpu";
 			compatible = "arm,cortex-a7";
-			reg = <0x104>;
+			reg = <0x103>;
 			cluster = <&cluster1>;
 			core = <&core7>;
 		};
@@ -159,7 +159,7 @@
 		#address-cells = <0>;
 		interrupt-controller;
 		reg = <0 0x2c001000 0 0x1000>,
-		      <0 0x2c002000 0 0x2000>,
+		      <0 0x2c002000 0 0x1000>,
 		      <0 0x2c004000 0 0x2000>,
 		      <0 0x2c006000 0 0x2000>;
 		interrupts = <1 9 0xf04>;
@@ -183,6 +183,101 @@
 			     <1 10 0xf08>;
 	};
 
+	smmu@2c0b0000 {
+                compatible = "arm,smmu-v2";
+                reg = <0x0 0x2c0b0000 0x0 0x10000>;
+                #global-interrupts = <2>;
+                interrupts = <0 58 4>,
+                             <0 59 4>,
+                             <0 60 4>,
+                             <0 60 4>,
+                             <0 60 4>,
+                             <0 60 4>,
+                             <0 60 4>,
+                             <0 60 4>,
+                             <0 60 4>;
+                mmu-masters = <&dma0 0x0 0x8>,
+                              <&dma1 0x1 0x9>,
+                              <&dma2 0x2 0xa>,
+                              <&dma3 0x3 0xb>;
+        };
+
+	dma0: dma@2c0a0000 {
+		compatible = "arm,pl330";
+		reg = <0 0x2c0a0000 0 0x1000>;
+		interrupts = <0 64 4>,
+			     <0 65 4>,
+			     <0 66 4>,
+			     <0 67 4>,
+			     <0 68 4>,
+			     <0 69 4>,
+			     <0 70 4>,
+			     <0 71 4>;
+		#dma-cells = <1>;
+		#stream-id-cells = <2>;
+		clocks = <&oscclk6a>;
+		clock-names = "apb_pclk";
+	};
+
+	dma1: dma@2c0a1000 {
+		compatible = "arm,pl330", "arm,primecell";
+		reg = <0 0x2c0a1000 0 0x1000>;
+		interrupts = <0 96 4>,
+			     <0 97 4>,
+			     <0 98 4>,
+			     <0 99 4>,
+			     <0 100 4>,
+			     <0 101 4>,
+			     <0 102 4>,
+			     <0 103 4>;
+		#dma-cells = <1>;
+		#stream-id-cells = <2>;
+		clocks = <&oscclk6a>;
+		clock-names = "apb_pclk";
+	};
+
+	dma2: dma@2c0a2000 {
+		compatible = "arm,pl330", "arm,primecell";
+		reg = <0 0x2c0a2000 0 0x1000>;
+		interrupts = <0 128 4>,
+			     <0 129 4>,
+			     <0 130 4>,
+			     <0 131 4>,
+			     <0 132 4>,
+			     <0 133 4>,
+			     <0 134 4>,
+			     <0 135 4>;
+		#dma-cells = <1>;
+		#stream-id-cells = <2>;
+		clocks = <&oscclk6a>;
+		clock-names = "apb_pclk";
+	};
+
+	dma3: dma@2c0a3000 {
+		compatible = "arm,pl330", "arm,primecell";
+		reg = <0 0x2c0a3000 0 0x1000>;
+		interrupts = <0 160 4>,
+			     <0 161 4>,
+			     <0 162 4>,
+			     <0 163 4>,
+			     <0 164 4>,
+			     <0 165 4>,
+			     <0 166 4>,
+			     <0 167 4>;
+		#dma-cells = <1>;
+		#stream-id-cells = <2>;
+		clocks = <&oscclk6a>;
+		clock-names = "apb_pclk";
+	};
+
+	oscclk6a: oscclk6a {
+		/* Reference 24MHz clock */
+		compatible = "fixed-clock";
+		#clock-cells = <0>;
+		clock-frequency = <24000000>;
+		clock-output-names = "oscclk6a";
+	};
+
 	smb {
 		compatible = "simple-bus";
 

fast_models/rtsm_ve-cortex_a15x4_a7x4.dts line 247

Quoted 4 lines
fast_models/rtsm_ve-cortex_a15x4_a7x4.dts +0 -1
@@ -244,4 +339,3 @@
 		/include/ "rtsm_ve-motherboard.dtsi"
 	};
 };
-

--
Shannon

Quoted 8 lines
Quoted 7 lines
Quoted 4 lines

_______________________________________________
kvmarm mailing list [email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm

.

.

Signature · 1 line
Shannon
#23 lore raw
In reply to a message not in the archive

Il 19/06/2014 13:11, Shannon Zhao ha scritto:

Quoted 65 lines · Alvise Rigo
Quoted 57 lines

Hi,

Il 19/06/2014 11:38, Shannon Zhao ha scritto:

Quoted 44 lines

Hi Bharat,

Quoted 37 lines · On 2014/6/19 17:19, [email protected]…
Quoted 28 lines

-----Original Message-----
From: [email protected] [mailto:kvmarm- [email protected]] On Behalf Of Shannon Zhao
Sent: Thursday, June 19, 2014 2:45 PM
To: Alvise Rigo; a.motakis-lrHrjnjw1UfHK3s98zE1ajGjJy/[email protected]; iommu-cunTk1MwBs/[email protected] foundation.org; [email protected]
Subject: Re: Vfio-on-arm: the device tree does not contain the DMA device

Hi Alvise,

Thanks for your reply.
I have apply the patch and could see the DMA devices.
But there is no vosys_dmatest int the directory "/sys/kernel/debug/" after mounting debugfs.
I found that the branch vfio-platform-v4 doesn't contain the module vosys_dmatest.
Then I port it from the branch pl330_dmatest and run vosys_dmatest successfully.

	echo 1 > /sys/kernel/debug/vosys_dmatest/start

	[ 6145.343419] dmaengine: __dma_request_channel: success (dma0chan0)
	[ 6145.344000] vosys dmatest: callback: transfer completed
	[ 6145.344147] vosys dmatest: tx completed
	[ 6145.344589] vosys dmatest: source and destination match.

Download the qemu and use following command to start a guest:

Quoted 8 lines
	./qemu-system-arm -enable-kvm -M vexpress-a15 -cpu cortex-a15 \
        	-device pl330,vfio_group="/dev/vfio/0",vfio_device="2c0a0000.dma",\
        	vfio_container="/dev/vfio/vfio" \
        	-append "earlyprintk ip=dhcp console=ttyAMA0 rootwait" \
        	-kernel zImage -dtb guest.dtb -initrd filesystem.cpio.gz  -

nographic An error reported:
qemu-system-arm: -device
pl330,vfio_group=/dev/vfio/0,vfio_device=2c0a0000.dma,:

Quoted 1 line

Why do we need to give "vfio_group=/dev/vfio/0" ?

I refer to the url "http://www.virtualopensystems.com/en/solutions/guides/vfio-on-arm/".

Quoted 2 lines
Quoted 8 lines
Quoted 4 lines
Quoted 2 lines

could not open disk image vfio_container=/dev/vfio:
Could not open 'vfio_container=/dev/vfio': No such file or directory

Are you unbinding "2c0a0000.dma" from its driver and binding with "vfio-playform" driver ?

How to unbind "2c0a0000.dma" from its driver?
I have tried following cmd to bind "2c0a0000.dma" with "vfio-playform" driver.
echo 2c0a0000.dma > /sys/bus/platform/drivers/vfio-platform/vfio_bind
But there is no /dev/vfio/0 or /dev/vfio/vfio .

Quoted 5 lines

Can you please verify that while configuring the host you enabled:
- Device Drivers -> IOMMU Hardware Support -> ARM Ltd. System MMU (SMMU) Support
and
- Device Drivers -> VFIO Non-Privileged userspace driver framework -> VFIO support for platform devices

Quoted 8 lines

Hi Alvise,

I have checked the two configuration are enabled.

Then try the following cmd, but another error reported:
qemu-system-arm: -device pl330,vfio_group=/dev/vfio/0,vfio_device=2c0a0000.dma,vfio_container=/dev/vfio/vfio: Parameter 'driver' expects pluggable device type

Do I need to unbind the "2c0a0000.dma" from its driver and binding with "vfio-platform" driver as Bharat mentioned ?

Yes, the 2c0a0000.dma should not be bound to any driver at boot, so you just need to:
echo 2c0a0000.dma > /sys/bus/platform/drivers/vfio-platform/vfio_bind to bind the device to the VFIO driver.

Regards,
alvise

Quoted 190 lines
Quoted 189 lines

Then try to update the pl330-vfio-dev branch and run QEMU with the following command:

./qemu-system-arm -enable-kvm -M vexpress-a15 -cpu cortex-a15 \
    -device pl330,vfio_group="/dev/vfio/0",vfio_device="2c0a0000.dma",vfio_container="/dev/vfio/vfio" \
    -append "earlyprintk ip=dhcp console=ttyAMA0 rootwait" \
    -kernel zImage -dtb guest.dtb -initrd filesystem.cpio.gz

that means without breaking the "-device pl330" options.

Regards,
alvise

Quoted 165 lines
Quoted 165 lines

Thanks
-Bharat

Quoted 160 lines

Is there something I'm wrong or do I miss some steps?

Thank you,
Shannon

Quoted 145 lines · Alvise Rigo

Hi Shannon,

Thank you for reporting this issue, the device tree file was indeed missing. To solve the problem, you can apply the following patch to the repository you mentioned
(git://github.com/virtualopensystems/arm-dts.git).

For your information, soon we will update the guide with a newer version of VFIO with this issue fixed.

Thank you,
alvise

---
 fast_models/rtsm_ve-cortex_a15x4_a7x4.dts | 100
+++++++++++++++++++++++++++++-
 1 file changed, 97 insertions(+), 3 deletions(-)
fast_models/rtsm_ve-cortex_a15x4_a7x4.dts +97 -2
index ef1275b..79c6af2 100644
@@ -142,7 +142,7 @@
 		cpu7: cpu@7 {
 			device_type = "cpu";
 			compatible = "arm,cortex-a7";
-			reg = <0x104>;
+			reg = <0x103>;
 			cluster = <&cluster1>;
 			core = <&core7>;
 		};
@@ -159,7 +159,7 @@
 		#address-cells = <0>;
 		interrupt-controller;
 		reg = <0 0x2c001000 0 0x1000>,
-		      <0 0x2c002000 0 0x2000>,
+		      <0 0x2c002000 0 0x1000>,
 		      <0 0x2c004000 0 0x2000>,
 		      <0 0x2c006000 0 0x2000>;
 		interrupts = <1 9 0xf04>;
@@ -183,6 +183,101 @@
 			     <1 10 0xf08>;
 	};
 
+	smmu@2c0b0000 {
+                compatible = "arm,smmu-v2";
+                reg = <0x0 0x2c0b0000 0x0 0x10000>;
+                #global-interrupts = <2>;
+                interrupts = <0 58 4>,
+                             <0 59 4>,
+                             <0 60 4>,
+                             <0 60 4>,
+                             <0 60 4>,
+                             <0 60 4>,
+                             <0 60 4>,
+                             <0 60 4>,
+                             <0 60 4>;
+                mmu-masters = <&dma0 0x0 0x8>,
+                              <&dma1 0x1 0x9>,
+                              <&dma2 0x2 0xa>,
+                              <&dma3 0x3 0xb>;
+        };
+
+	dma0: dma@2c0a0000 {
+		compatible = "arm,pl330";
+		reg = <0 0x2c0a0000 0 0x1000>;
+		interrupts = <0 64 4>,
+			     <0 65 4>,
+			     <0 66 4>,
+			     <0 67 4>,
+			     <0 68 4>,
+			     <0 69 4>,
+			     <0 70 4>,
+			     <0 71 4>;
+		#dma-cells = <1>;
+		#stream-id-cells = <2>;
+		clocks = <&oscclk6a>;
+		clock-names = "apb_pclk";
+	};
+
+	dma1: dma@2c0a1000 {
+		compatible = "arm,pl330", "arm,primecell";
+		reg = <0 0x2c0a1000 0 0x1000>;
+		interrupts = <0 96 4>,
+			     <0 97 4>,
+			     <0 98 4>,
+			     <0 99 4>,
+			     <0 100 4>,
+			     <0 101 4>,
+			     <0 102 4>,
+			     <0 103 4>;
+		#dma-cells = <1>;
+		#stream-id-cells = <2>;
+		clocks = <&oscclk6a>;
+		clock-names = "apb_pclk";
+	};
+
+	dma2: dma@2c0a2000 {
+		compatible = "arm,pl330", "arm,primecell";
+		reg = <0 0x2c0a2000 0 0x1000>;
+		interrupts = <0 128 4>,
+			     <0 129 4>,
+			     <0 130 4>,
+			     <0 131 4>,
+			     <0 132 4>,
+			     <0 133 4>,
+			     <0 134 4>,
+			     <0 135 4>;
+		#dma-cells = <1>;
+		#stream-id-cells = <2>;
+		clocks = <&oscclk6a>;
+		clock-names = "apb_pclk";
+	};
+
+	dma3: dma@2c0a3000 {
+		compatible = "arm,pl330", "arm,primecell";
+		reg = <0 0x2c0a3000 0 0x1000>;
+		interrupts = <0 160 4>,
+			     <0 161 4>,
+			     <0 162 4>,
+			     <0 163 4>,
+			     <0 164 4>,
+			     <0 165 4>,
+			     <0 166 4>,
+			     <0 167 4>;
+		#dma-cells = <1>;
+		#stream-id-cells = <2>;
+		clocks = <&oscclk6a>;
+		clock-names = "apb_pclk";
+	};
+
+	oscclk6a: oscclk6a {
+		/* Reference 24MHz clock */
+		compatible = "fixed-clock";
+		#clock-cells = <0>;
+		clock-frequency = <24000000>;
+		clock-output-names = "oscclk6a";
+	};
+
 	smb {
 		compatible = "simple-bus";
 

fast_models/rtsm_ve-cortex_a15x4_a7x4.dts line 247

Quoted 4 lines
fast_models/rtsm_ve-cortex_a15x4_a7x4.dts +0 -1
@@ -244,4 +339,3 @@
 		/include/ "rtsm_ve-motherboard.dtsi"
 	};
 };
-

--
Shannon

Quoted 8 lines
Quoted 7 lines
Quoted 4 lines

_______________________________________________
kvmarm mailing list [email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm

.

.

#25 lore raw
In reply to a message not in the archive

Hi Alvise,

I run following cmds :

	echo 2c0a0000.dma > /sys/bus/platform/drivers/vfio-platform/vfio_bind
	./qemu-system-arm -enable-kvm -M vexpress-a15 -cpu cortex-a15 \
        	-device pl330,vfio_group="/dev/vfio/0",vfio_device="2c0a0000.dma",vfio_container="/dev/vfio/vfio" \
        	-append "earlyprintk ip=dhcp console=ttyAMA0 rootwait" \
        	-kernel zImage -dtb guest-a15.dtb -initrd filesystem.cpio.gz  -nographic

But the error still exists.
qemu-system-arm: -device pl330,vfio_group=/dev/vfio/0,vfio_device=2c0a0000.dma,vfio_container=/dev/vfio/vfio: Parameter 'driver' expects pluggable device type

Thanks,
Shannon

Quoted 279 lines · Alvise Rigo

Il 19/06/2014 13:11, Shannon Zhao ha scritto:

Quoted 65 lines · Alvise Rigo
Quoted 57 lines

Hi,

Il 19/06/2014 11:38, Shannon Zhao ha scritto:

Quoted 44 lines

Hi Bharat,

Quoted 37 lines · On 2014/6/19 17:19, [email protected]…
Quoted 28 lines

-----Original Message-----
From: [email protected] [mailto:kvmarm- [email protected]] On Behalf Of Shannon Zhao
Sent: Thursday, June 19, 2014 2:45 PM
To: Alvise Rigo; a.motakis-lrHrjnjw1UfHK3s98zE1ajGjJy/[email protected]; iommu-cunTk1MwBs/[email protected] foundation.org; [email protected]
Subject: Re: Vfio-on-arm: the device tree does not contain the DMA device

Hi Alvise,

Thanks for your reply.
I have apply the patch and could see the DMA devices.
But there is no vosys_dmatest int the directory "/sys/kernel/debug/" after mounting debugfs.
I found that the branch vfio-platform-v4 doesn't contain the module vosys_dmatest.
Then I port it from the branch pl330_dmatest and run vosys_dmatest successfully.

	echo 1 > /sys/kernel/debug/vosys_dmatest/start

	[ 6145.343419] dmaengine: __dma_request_channel: success (dma0chan0)
	[ 6145.344000] vosys dmatest: callback: transfer completed
	[ 6145.344147] vosys dmatest: tx completed
	[ 6145.344589] vosys dmatest: source and destination match.

Download the qemu and use following command to start a guest:

Quoted 8 lines
	./qemu-system-arm -enable-kvm -M vexpress-a15 -cpu cortex-a15 \
        	-device pl330,vfio_group="/dev/vfio/0",vfio_device="2c0a0000.dma",\
        	vfio_container="/dev/vfio/vfio" \
        	-append "earlyprintk ip=dhcp console=ttyAMA0 rootwait" \
        	-kernel zImage -dtb guest.dtb -initrd filesystem.cpio.gz  -

nographic An error reported:
qemu-system-arm: -device
pl330,vfio_group=/dev/vfio/0,vfio_device=2c0a0000.dma,:

Quoted 1 line

Why do we need to give "vfio_group=/dev/vfio/0" ?

I refer to the url "http://www.virtualopensystems.com/en/solutions/guides/vfio-on-arm/".

Quoted 2 lines
Quoted 8 lines
Quoted 4 lines
Quoted 2 lines

could not open disk image vfio_container=/dev/vfio:
Could not open 'vfio_container=/dev/vfio': No such file or directory

Are you unbinding "2c0a0000.dma" from its driver and binding with "vfio-playform" driver ?

How to unbind "2c0a0000.dma" from its driver?
I have tried following cmd to bind "2c0a0000.dma" with "vfio-playform" driver.
echo 2c0a0000.dma > /sys/bus/platform/drivers/vfio-platform/vfio_bind
But there is no /dev/vfio/0 or /dev/vfio/vfio .

Quoted 5 lines

Can you please verify that while configuring the host you enabled:
- Device Drivers -> IOMMU Hardware Support -> ARM Ltd. System MMU (SMMU) Support
and
- Device Drivers -> VFIO Non-Privileged userspace driver framework -> VFIO support for platform devices

Quoted 8 lines

Hi Alvise,

I have checked the two configuration are enabled.

Then try the following cmd, but another error reported:
qemu-system-arm: -device pl330,vfio_group=/dev/vfio/0,vfio_device=2c0a0000.dma,vfio_container=/dev/vfio/vfio: Parameter 'driver' expects pluggable device type

Do I need to unbind the "2c0a0000.dma" from its driver and binding with "vfio-platform" driver as Bharat mentioned ?

Yes, the 2c0a0000.dma should not be bound to any driver at boot, so you just need to:
echo 2c0a0000.dma > /sys/bus/platform/drivers/vfio-platform/vfio_bind to bind the device to the VFIO driver.

Regards,
alvise

Quoted 183 lines
Quoted 183 lines

Then try to update the pl330-vfio-dev branch and run QEMU with the following command:

./qemu-system-arm -enable-kvm -M vexpress-a15 -cpu cortex-a15 \
    -device pl330,vfio_group="/dev/vfio/0",vfio_device="2c0a0000.dma",vfio_container="/dev/vfio/vfio" \
    -append "earlyprintk ip=dhcp console=ttyAMA0 rootwait" \
    -kernel zImage -dtb guest.dtb -initrd filesystem.cpio.gz

that means without breaking the "-device pl330" options.

Regards,
alvise

Quoted 170 lines
Quoted 170 lines

Thanks
-Bharat

Quoted 164 lines

Is there something I'm wrong or do I miss some steps?

Thank you,
Shannon

Quoted 145 lines · Alvise Rigo

Hi Shannon,

Thank you for reporting this issue, the device tree file was indeed missing. To solve the problem, you can apply the following patch to the repository you mentioned
(git://github.com/virtualopensystems/arm-dts.git).

For your information, soon we will update the guide with a newer version of VFIO with this issue fixed.

Thank you,
alvise

---
 fast_models/rtsm_ve-cortex_a15x4_a7x4.dts | 100
+++++++++++++++++++++++++++++-
 1 file changed, 97 insertions(+), 3 deletions(-)
fast_models/rtsm_ve-cortex_a15x4_a7x4.dts +97 -2
index ef1275b..79c6af2 100644
@@ -142,7 +142,7 @@
 		cpu7: cpu@7 {
 			device_type = "cpu";
 			compatible = "arm,cortex-a7";
-			reg = <0x104>;
+			reg = <0x103>;
 			cluster = <&cluster1>;
 			core = <&core7>;
 		};
@@ -159,7 +159,7 @@
 		#address-cells = <0>;
 		interrupt-controller;
 		reg = <0 0x2c001000 0 0x1000>,
-		      <0 0x2c002000 0 0x2000>,
+		      <0 0x2c002000 0 0x1000>,
 		      <0 0x2c004000 0 0x2000>,
 		      <0 0x2c006000 0 0x2000>;
 		interrupts = <1 9 0xf04>;
@@ -183,6 +183,101 @@
 			     <1 10 0xf08>;
 	};
 
+	smmu@2c0b0000 {
+                compatible = "arm,smmu-v2";
+                reg = <0x0 0x2c0b0000 0x0 0x10000>;
+                #global-interrupts = <2>;
+                interrupts = <0 58 4>,
+                             <0 59 4>,
+                             <0 60 4>,
+                             <0 60 4>,
+                             <0 60 4>,
+                             <0 60 4>,
+                             <0 60 4>,
+                             <0 60 4>,
+                             <0 60 4>;
+                mmu-masters = <&dma0 0x0 0x8>,
+                              <&dma1 0x1 0x9>,
+                              <&dma2 0x2 0xa>,
+                              <&dma3 0x3 0xb>;
+        };
+
+	dma0: dma@2c0a0000 {
+		compatible = "arm,pl330";
+		reg = <0 0x2c0a0000 0 0x1000>;
+		interrupts = <0 64 4>,
+			     <0 65 4>,
+			     <0 66 4>,
+			     <0 67 4>,
+			     <0 68 4>,
+			     <0 69 4>,
+			     <0 70 4>,
+			     <0 71 4>;
+		#dma-cells = <1>;
+		#stream-id-cells = <2>;
+		clocks = <&oscclk6a>;
+		clock-names = "apb_pclk";
+	};
+
+	dma1: dma@2c0a1000 {
+		compatible = "arm,pl330", "arm,primecell";
+		reg = <0 0x2c0a1000 0 0x1000>;
+		interrupts = <0 96 4>,
+			     <0 97 4>,
+			     <0 98 4>,
+			     <0 99 4>,
+			     <0 100 4>,
+			     <0 101 4>,
+			     <0 102 4>,
+			     <0 103 4>;
+		#dma-cells = <1>;
+		#stream-id-cells = <2>;
+		clocks = <&oscclk6a>;
+		clock-names = "apb_pclk";
+	};
+
+	dma2: dma@2c0a2000 {
+		compatible = "arm,pl330", "arm,primecell";
+		reg = <0 0x2c0a2000 0 0x1000>;
+		interrupts = <0 128 4>,
+			     <0 129 4>,
+			     <0 130 4>,
+			     <0 131 4>,
+			     <0 132 4>,
+			     <0 133 4>,
+			     <0 134 4>,
+			     <0 135 4>;
+		#dma-cells = <1>;
+		#stream-id-cells = <2>;
+		clocks = <&oscclk6a>;
+		clock-names = "apb_pclk";
+	};
+
+	dma3: dma@2c0a3000 {
+		compatible = "arm,pl330", "arm,primecell";
+		reg = <0 0x2c0a3000 0 0x1000>;
+		interrupts = <0 160 4>,
+			     <0 161 4>,
+			     <0 162 4>,
+			     <0 163 4>,
+			     <0 164 4>,
+			     <0 165 4>,
+			     <0 166 4>,
+			     <0 167 4>;
+		#dma-cells = <1>;
+		#stream-id-cells = <2>;
+		clocks = <&oscclk6a>;
+		clock-names = "apb_pclk";
+	};
+
+	oscclk6a: oscclk6a {
+		/* Reference 24MHz clock */
+		compatible = "fixed-clock";
+		#clock-cells = <0>;
+		clock-frequency = <24000000>;
+		clock-output-names = "oscclk6a";
+	};
+
 	smb {
 		compatible = "simple-bus";
 

fast_models/rtsm_ve-cortex_a15x4_a7x4.dts line 247

Quoted 4 lines
fast_models/rtsm_ve-cortex_a15x4_a7x4.dts +0 -1
@@ -244,4 +339,3 @@
 		/include/ "rtsm_ve-motherboard.dtsi"
 	};
 };
-

--
Shannon

_______________________________________________
kvmarm mailing list [email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm

Quoted 7 lines
Quoted 6 lines
Quoted 3 lines
Quoted 2 lines

.

.

.

Signature · 1 line
Shannon
#27 lore raw
In reply to a message not in the archive
Quoted 9 lines

-----Original Message-----
From: Shannon Zhao [mailto:[email protected]]
Sent: Thursday, June 19, 2014 5:38 PM
To: Alvise Rigo; Bhushan Bharat-R65777; a.motakis-lrHrjnjw1UfHK3s98zE1ajGjJy/[email protected]; iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org; [email protected]
Subject: Re: Vfio-on-arm: the device tree does not contain the DMA device

Quoted 4 lines

Hi Alvise,

I run following cmds :
echo 2c0a0000.dma > /sys/bus/platform/drivers/vfio-platform/vfio_bind

run "ls /dev/vfio/" before and after the above command

Thanks
-Bharat

Quoted 314 lines
	./qemu-system-arm -enable-kvm -M vexpress-a15 -cpu cortex-a15 \
        	-device

pl330,vfio_group="/dev/vfio/0",vfio_device="2c0a0000.dma",vfio_container="/dev/v fio/vfio" \
-append "earlyprintk ip=dhcp console=ttyAMA0 rootwait" \
-kernel zImage -dtb guest-a15.dtb -initrd filesystem.cpio.gz -
nographic But the error still exists.
qemu-system-arm: -device
pl330,vfio_group=/dev/vfio/0,vfio_device=2c0a0000.dma,vfio_container=/dev/vfio/v
fio: Parameter 'driver' expects pluggable device type

Thanks,
Shannon

Quoted 32 lines · Alvise Rigo

Il 19/06/2014 13:11, Shannon Zhao ha scritto:

Quoted 29 lines · Alvise Rigo
Quoted 26 lines

Hi,

Il 19/06/2014 11:38, Shannon Zhao ha scritto:

Quoted 23 lines

Hi Bharat,

Quoted 20 lines · On 2014/6/19 17:19, [email protected]…
Quoted 20 lines

-----Original Message-----
From: [email protected] [mailto:kvmarm- [email protected]] On Behalf Of Shannon Zhao
Sent: Thursday, June 19, 2014 2:45 PM
To: Alvise Rigo; a.motakis-lrHrjnjw1UfHK3s98zE1ajGjJy/[email protected]; iommu-cunTk1MwBs/[email protected] foundation.org; [email protected]
Subject: Re: Vfio-on-arm: the device tree does not contain the
DMA device

Hi Alvise,

Thanks for your reply.
I have apply the patch and could see the DMA devices. But there is no vosys_dmatest int the directory "/sys/kernel/debug/" after mounting debugfs.
I found that the branch vfio-platform-v4 doesn't contain the module vosys_dmatest.
Then I port it from the branch pl330_dmatest and run vosys_dmatest

successfully.

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
	echo 1 > /sys/kernel/debug/vosys_dmatest/start

	[ 6145.343419] dmaengine: __dma_request_channel: success

(dma0chan0)

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
	[ 6145.344000] vosys dmatest: callback: transfer completed
	[ 6145.344147] vosys dmatest: tx completed
	[ 6145.344589] vosys dmatest: source and destination match.

Download the qemu and use following command to start a guest:

	./qemu-system-arm -enable-kvm -M vexpress-a15 -cpu cortex-a15 \
        	-device

pl330,vfio_group="/dev/vfio/0",vfio_device="2c0a0000.dma",\

Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 8 lines
Quoted 6 lines
        	vfio_container="/dev/vfio/vfio" \
        	-append "earlyprintk ip=dhcp console=ttyAMA0 rootwait" \
        	-kernel zImage -dtb guest.dtb -initrd filesystem.cpio.gz

- nographic An error reported:
qemu-system-arm: -device
pl330,vfio_group=/dev/vfio/0,vfio_device=2c0a0000.dma,:

Why do we need to give "vfio_group=/dev/vfio/0" ?

I refer to the url

"http://www.virtualopensystems.com/en/solutions/guides/vfio-on-arm/".

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 5 lines

could not open disk image vfio_container=/dev/vfio:
Could not open 'vfio_container=/dev/vfio': No such file or
directory

Are you unbinding "2c0a0000.dma" from its driver and binding with "vfio-

playform" driver ?

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

How to unbind "2c0a0000.dma" from its driver?
I have tried following cmd to bind "2c0a0000.dma" with "vfio-playform"

driver.

Quoted 13 lines
Quoted 13 lines
Quoted 11 lines
Quoted 3 lines
	echo 2c0a0000.dma >

/sys/bus/platform/drivers/vfio-platform/vfio_bind
But there is no /dev/vfio/0 or /dev/vfio/vfio .

Can you please verify that while configuring the host you enabled:
- Device Drivers -> IOMMU Hardware Support -> ARM Ltd. System MMU (SMMU) Support and
- Device Drivers -> VFIO Non-Privileged userspace driver framework -> VFIO support for platform devices

Hi Alvise,

Quoted 8 lines
Quoted 8 lines

I have checked the two configuration are enabled.

Then try the following cmd, but another error reported:
qemu-system-arm: -device
pl330,vfio_group=/dev/vfio/0,vfio_device=2c0a0000.dma,vfio_container= /dev/vfio/vfio: Parameter 'driver' expects pluggable device type

Do I need to unbind the "2c0a0000.dma" from its driver and binding with

"vfio-platform" driver as Bharat mentioned ?

Quoted 12 lines

Yes, the 2c0a0000.dma should not be bound to any driver at boot, so you just need to:
echo 2c0a0000.dma > /sys/bus/platform/drivers/vfio-platform/vfio_bind to bind the device to the VFIO driver.

Regards,
alvise

Quoted 2 lines
Quoted 2 lines

Then try to update the pl330-vfio-dev branch and run QEMU with the following

command:

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

./qemu-system-arm -enable-kvm -M vexpress-a15 -cpu cortex-a15 \
-device

pl330,vfio_group="/dev/vfio/0",vfio_device="2c0a0000.dma",vfio_container="/dev/v fio/vfio" \

Quoted 16 lines
Quoted 16 lines
Quoted 16 lines

-append "earlyprintk ip=dhcp console=ttyAMA0 rootwait" \
-kernel zImage -dtb guest.dtb -initrd filesystem.cpio.gz

that means without breaking the "-device pl330" options.

Regards,
alvise

Quoted 8 lines
Quoted 8 lines

Thanks
-Bharat

Quoted 3 lines

Is there something I'm wrong or do I miss some steps?

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines

Thank you,
Shannon

Quoted 3 lines · Alvise Rigo

Hi Shannon,

Thank you for reporting this issue, the device tree file was indeed

missing.

Quoted 159 lines
Quoted 159 lines
Quoted 159 lines
Quoted 158 lines
Quoted 157 lines
Quoted 150 lines
Quoted 142 lines

To solve the problem, you can apply the following patch to the repository you mentioned
(git://github.com/virtualopensystems/arm-dts.git).

For your information, soon we will update the guide with a newer version of VFIO with this issue fixed.

Thank you,
alvise

---
 fast_models/rtsm_ve-cortex_a15x4_a7x4.dts | 100
+++++++++++++++++++++++++++++-
 1 file changed, 97 insertions(+), 3 deletions(-)
fast_models/rtsm_ve-cortex_a15x4_a7x4.dts +97 -2
index ef1275b..79c6af2 100644
@@ -142,7 +142,7 @@
 		cpu7: cpu@7 {
 			device_type = "cpu";
 			compatible = "arm,cortex-a7";
-			reg = <0x104>;
+			reg = <0x103>;
 			cluster = <&cluster1>;
 			core = <&core7>;
 		};
@@ -159,7 +159,7 @@
 		#address-cells = <0>;
 		interrupt-controller;
 		reg = <0 0x2c001000 0 0x1000>,
-		      <0 0x2c002000 0 0x2000>,
+		      <0 0x2c002000 0 0x1000>,
 		      <0 0x2c004000 0 0x2000>,
 		      <0 0x2c006000 0 0x2000>;
 		interrupts = <1 9 0xf04>;
@@ -183,6 +183,101 @@
 			     <1 10 0xf08>;
 	};
 
+	smmu@2c0b0000 {
+                compatible = "arm,smmu-v2";
+                reg = <0x0 0x2c0b0000 0x0 0x10000>;
+                #global-interrupts = <2>;
+                interrupts = <0 58 4>,
+                             <0 59 4>,
+                             <0 60 4>,
+                             <0 60 4>,
+                             <0 60 4>,
+                             <0 60 4>,
+                             <0 60 4>,
+                             <0 60 4>,
+                             <0 60 4>;
+                mmu-masters = <&dma0 0x0 0x8>,
+                              <&dma1 0x1 0x9>,
+                              <&dma2 0x2 0xa>,
+                              <&dma3 0x3 0xb>;
+        };
+
+	dma0: dma@2c0a0000 {
+		compatible = "arm,pl330";
+		reg = <0 0x2c0a0000 0 0x1000>;
+		interrupts = <0 64 4>,
+			     <0 65 4>,
+			     <0 66 4>,
+			     <0 67 4>,
+			     <0 68 4>,
+			     <0 69 4>,
+			     <0 70 4>,
+			     <0 71 4>;
+		#dma-cells = <1>;
+		#stream-id-cells = <2>;
+		clocks = <&oscclk6a>;
+		clock-names = "apb_pclk";
+	};
+
+	dma1: dma@2c0a1000 {
+		compatible = "arm,pl330", "arm,primecell";
+		reg = <0 0x2c0a1000 0 0x1000>;
+		interrupts = <0 96 4>,
+			     <0 97 4>,
+			     <0 98 4>,
+			     <0 99 4>,
+			     <0 100 4>,
+			     <0 101 4>,
+			     <0 102 4>,
+			     <0 103 4>;
+		#dma-cells = <1>;
+		#stream-id-cells = <2>;
+		clocks = <&oscclk6a>;
+		clock-names = "apb_pclk";
+	};
+
+	dma2: dma@2c0a2000 {
+		compatible = "arm,pl330", "arm,primecell";
+		reg = <0 0x2c0a2000 0 0x1000>;
+		interrupts = <0 128 4>,
+			     <0 129 4>,
+			     <0 130 4>,
+			     <0 131 4>,
+			     <0 132 4>,
+			     <0 133 4>,
+			     <0 134 4>,
+			     <0 135 4>;
+		#dma-cells = <1>;
+		#stream-id-cells = <2>;
+		clocks = <&oscclk6a>;
+		clock-names = "apb_pclk";
+	};
+
+	dma3: dma@2c0a3000 {
+		compatible = "arm,pl330", "arm,primecell";
+		reg = <0 0x2c0a3000 0 0x1000>;
+		interrupts = <0 160 4>,
+			     <0 161 4>,
+			     <0 162 4>,
+			     <0 163 4>,
+			     <0 164 4>,
+			     <0 165 4>,
+			     <0 166 4>,
+			     <0 167 4>;
+		#dma-cells = <1>;
+		#stream-id-cells = <2>;
+		clocks = <&oscclk6a>;
+		clock-names = "apb_pclk";
+	};
+
+	oscclk6a: oscclk6a {
+		/* Reference 24MHz clock */
+		compatible = "fixed-clock";
+		#clock-cells = <0>;
+		clock-frequency = <24000000>;
+		clock-output-names = "oscclk6a";
+	};
+
 	smb {
 		compatible = "simple-bus";
 

fast_models/rtsm_ve-cortex_a15x4_a7x4.dts line 247

Quoted 4 lines
fast_models/rtsm_ve-cortex_a15x4_a7x4.dts +0 -1
@@ -244,4 +339,3 @@
 		/include/ "rtsm_ve-motherboard.dtsi"
 	};
 };
-

--
Shannon

Quoted 4 lines

_______________________________________________
kvmarm mailing list [email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm

.

Quoted 6 lines
Quoted 3 lines
Quoted 2 lines

.

.

--
Shannon

#29 lore raw
In reply to a message not in the archive
Quoted 9 lines · On 2014/6/19 20:12, [email protected]…
Quoted 9 lines

-----Original Message-----
From: Shannon Zhao [mailto:[email protected]]
Sent: Thursday, June 19, 2014 5:38 PM
To: Alvise Rigo; Bhushan Bharat-R65777; a.motakis-lrHrjnjw1UfHK3s98zE1ajGjJy/[email protected]; iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org; [email protected]
Subject: Re: Vfio-on-arm: the device tree does not contain the DMA device

Quoted 6 lines
Quoted 4 lines

Hi Alvise,

I run following cmds :
echo 2c0a0000.dma > /sys/bus/platform/drivers/vfio-platform/vfio_bind

run "ls /dev/vfio/" before and after the above command

Hi Bharat,

Nothing appeared either before or after the cmd "echo 2c0a0000.dma > /sys/bus/platform/drivers/vfio-platform/vfio_bind". There is only /dev/vfio.

Quoted 323 lines

Thanks
-Bharat

Quoted 309 lines
	./qemu-system-arm -enable-kvm -M vexpress-a15 -cpu cortex-a15 \
        	-device

pl330,vfio_group="/dev/vfio/0",vfio_device="2c0a0000.dma",vfio_container="/dev/v fio/vfio" \
-append "earlyprintk ip=dhcp console=ttyAMA0 rootwait" \
-kernel zImage -dtb guest-a15.dtb -initrd filesystem.cpio.gz -
nographic But the error still exists.
qemu-system-arm: -device
pl330,vfio_group=/dev/vfio/0,vfio_device=2c0a0000.dma,vfio_container=/dev/vfio/v
fio: Parameter 'driver' expects pluggable device type

Thanks,
Shannon

Quoted 32 lines · Alvise Rigo

Il 19/06/2014 13:11, Shannon Zhao ha scritto:

Quoted 29 lines · Alvise Rigo
Quoted 26 lines

Hi,

Il 19/06/2014 11:38, Shannon Zhao ha scritto:

Quoted 23 lines

Hi Bharat,

Quoted 20 lines · On 2014/6/19 17:19, [email protected]…
Quoted 20 lines

-----Original Message-----
From: [email protected] [mailto:kvmarm- [email protected]] On Behalf Of Shannon Zhao
Sent: Thursday, June 19, 2014 2:45 PM
To: Alvise Rigo; a.motakis-lrHrjnjw1UfHK3s98zE1ajGjJy/[email protected]; iommu-cunTk1MwBs/[email protected] foundation.org; [email protected]
Subject: Re: Vfio-on-arm: the device tree does not contain the
DMA device

Hi Alvise,

Thanks for your reply.
I have apply the patch and could see the DMA devices. But there is no vosys_dmatest int the directory "/sys/kernel/debug/" after mounting debugfs.
I found that the branch vfio-platform-v4 doesn't contain the module vosys_dmatest.
Then I port it from the branch pl330_dmatest and run vosys_dmatest

successfully.

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
	echo 1 > /sys/kernel/debug/vosys_dmatest/start

	[ 6145.343419] dmaengine: __dma_request_channel: success

(dma0chan0)

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
	[ 6145.344000] vosys dmatest: callback: transfer completed
	[ 6145.344147] vosys dmatest: tx completed
	[ 6145.344589] vosys dmatest: source and destination match.

Download the qemu and use following command to start a guest:

	./qemu-system-arm -enable-kvm -M vexpress-a15 -cpu cortex-a15 \
        	-device

pl330,vfio_group="/dev/vfio/0",vfio_device="2c0a0000.dma",\

Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 8 lines
Quoted 6 lines
        	vfio_container="/dev/vfio/vfio" \
        	-append "earlyprintk ip=dhcp console=ttyAMA0 rootwait" \
        	-kernel zImage -dtb guest.dtb -initrd filesystem.cpio.gz

- nographic An error reported:
qemu-system-arm: -device
pl330,vfio_group=/dev/vfio/0,vfio_device=2c0a0000.dma,:

Why do we need to give "vfio_group=/dev/vfio/0" ?

I refer to the url

"http://www.virtualopensystems.com/en/solutions/guides/vfio-on-arm/".

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 5 lines

could not open disk image vfio_container=/dev/vfio:
Could not open 'vfio_container=/dev/vfio': No such file or
directory

Are you unbinding "2c0a0000.dma" from its driver and binding with "vfio-

playform" driver ?

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

How to unbind "2c0a0000.dma" from its driver?
I have tried following cmd to bind "2c0a0000.dma" with "vfio-playform"

driver.

Quoted 13 lines
Quoted 13 lines
Quoted 11 lines
Quoted 3 lines
	echo 2c0a0000.dma >

/sys/bus/platform/drivers/vfio-platform/vfio_bind
But there is no /dev/vfio/0 or /dev/vfio/vfio .

Can you please verify that while configuring the host you enabled:
- Device Drivers -> IOMMU Hardware Support -> ARM Ltd. System MMU (SMMU) Support and
- Device Drivers -> VFIO Non-Privileged userspace driver framework -> VFIO support for platform devices

Hi Alvise,

Quoted 8 lines
Quoted 8 lines

I have checked the two configuration are enabled.

Then try the following cmd, but another error reported:
qemu-system-arm: -device
pl330,vfio_group=/dev/vfio/0,vfio_device=2c0a0000.dma,vfio_container= /dev/vfio/vfio: Parameter 'driver' expects pluggable device type

Do I need to unbind the "2c0a0000.dma" from its driver and binding with

"vfio-platform" driver as Bharat mentioned ?

Quoted 12 lines

Yes, the 2c0a0000.dma should not be bound to any driver at boot, so you just need to:
echo 2c0a0000.dma > /sys/bus/platform/drivers/vfio-platform/vfio_bind to bind the device to the VFIO driver.

Regards,
alvise

Quoted 2 lines
Quoted 2 lines

Then try to update the pl330-vfio-dev branch and run QEMU with the following

command:

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

./qemu-system-arm -enable-kvm -M vexpress-a15 -cpu cortex-a15 \
-device

pl330,vfio_group="/dev/vfio/0",vfio_device="2c0a0000.dma",vfio_container="/dev/v fio/vfio" \

Quoted 16 lines
Quoted 16 lines
Quoted 16 lines

-append "earlyprintk ip=dhcp console=ttyAMA0 rootwait" \
-kernel zImage -dtb guest.dtb -initrd filesystem.cpio.gz

that means without breaking the "-device pl330" options.

Regards,
alvise

Quoted 8 lines
Quoted 8 lines

Thanks
-Bharat

Quoted 3 lines

Is there something I'm wrong or do I miss some steps?

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines

Thank you,
Shannon

Quoted 3 lines · Alvise Rigo

Hi Shannon,

Thank you for reporting this issue, the device tree file was indeed

missing.

Quoted 163 lines
Quoted 162 lines
Quoted 161 lines
Quoted 150 lines
Quoted 150 lines
Quoted 150 lines
Quoted 142 lines

To solve the problem, you can apply the following patch to the repository you mentioned
(git://github.com/virtualopensystems/arm-dts.git).

For your information, soon we will update the guide with a newer version of VFIO with this issue fixed.

Thank you,
alvise

---
 fast_models/rtsm_ve-cortex_a15x4_a7x4.dts | 100
+++++++++++++++++++++++++++++-
 1 file changed, 97 insertions(+), 3 deletions(-)
fast_models/rtsm_ve-cortex_a15x4_a7x4.dts +97 -2
index ef1275b..79c6af2 100644
@@ -142,7 +142,7 @@
 		cpu7: cpu@7 {
 			device_type = "cpu";
 			compatible = "arm,cortex-a7";
-			reg = <0x104>;
+			reg = <0x103>;
 			cluster = <&cluster1>;
 			core = <&core7>;
 		};
@@ -159,7 +159,7 @@
 		#address-cells = <0>;
 		interrupt-controller;
 		reg = <0 0x2c001000 0 0x1000>,
-		      <0 0x2c002000 0 0x2000>,
+		      <0 0x2c002000 0 0x1000>,
 		      <0 0x2c004000 0 0x2000>,
 		      <0 0x2c006000 0 0x2000>;
 		interrupts = <1 9 0xf04>;
@@ -183,6 +183,101 @@
 			     <1 10 0xf08>;
 	};
 
+	smmu@2c0b0000 {
+                compatible = "arm,smmu-v2";
+                reg = <0x0 0x2c0b0000 0x0 0x10000>;
+                #global-interrupts = <2>;
+                interrupts = <0 58 4>,
+                             <0 59 4>,
+                             <0 60 4>,
+                             <0 60 4>,
+                             <0 60 4>,
+                             <0 60 4>,
+                             <0 60 4>,
+                             <0 60 4>,
+                             <0 60 4>;
+                mmu-masters = <&dma0 0x0 0x8>,
+                              <&dma1 0x1 0x9>,
+                              <&dma2 0x2 0xa>,
+                              <&dma3 0x3 0xb>;
+        };
+
+	dma0: dma@2c0a0000 {
+		compatible = "arm,pl330";
+		reg = <0 0x2c0a0000 0 0x1000>;
+		interrupts = <0 64 4>,
+			     <0 65 4>,
+			     <0 66 4>,
+			     <0 67 4>,
+			     <0 68 4>,
+			     <0 69 4>,
+			     <0 70 4>,
+			     <0 71 4>;
+		#dma-cells = <1>;
+		#stream-id-cells = <2>;
+		clocks = <&oscclk6a>;
+		clock-names = "apb_pclk";
+	};
+
+	dma1: dma@2c0a1000 {
+		compatible = "arm,pl330", "arm,primecell";
+		reg = <0 0x2c0a1000 0 0x1000>;
+		interrupts = <0 96 4>,
+			     <0 97 4>,
+			     <0 98 4>,
+			     <0 99 4>,
+			     <0 100 4>,
+			     <0 101 4>,
+			     <0 102 4>,
+			     <0 103 4>;
+		#dma-cells = <1>;
+		#stream-id-cells = <2>;
+		clocks = <&oscclk6a>;
+		clock-names = "apb_pclk";
+	};
+
+	dma2: dma@2c0a2000 {
+		compatible = "arm,pl330", "arm,primecell";
+		reg = <0 0x2c0a2000 0 0x1000>;
+		interrupts = <0 128 4>,
+			     <0 129 4>,
+			     <0 130 4>,
+			     <0 131 4>,
+			     <0 132 4>,
+			     <0 133 4>,
+			     <0 134 4>,
+			     <0 135 4>;
+		#dma-cells = <1>;
+		#stream-id-cells = <2>;
+		clocks = <&oscclk6a>;
+		clock-names = "apb_pclk";
+	};
+
+	dma3: dma@2c0a3000 {
+		compatible = "arm,pl330", "arm,primecell";
+		reg = <0 0x2c0a3000 0 0x1000>;
+		interrupts = <0 160 4>,
+			     <0 161 4>,
+			     <0 162 4>,
+			     <0 163 4>,
+			     <0 164 4>,
+			     <0 165 4>,
+			     <0 166 4>,
+			     <0 167 4>;
+		#dma-cells = <1>;
+		#stream-id-cells = <2>;
+		clocks = <&oscclk6a>;
+		clock-names = "apb_pclk";
+	};
+
+	oscclk6a: oscclk6a {
+		/* Reference 24MHz clock */
+		compatible = "fixed-clock";
+		#clock-cells = <0>;
+		clock-frequency = <24000000>;
+		clock-output-names = "oscclk6a";
+	};
+
 	smb {
 		compatible = "simple-bus";
 

fast_models/rtsm_ve-cortex_a15x4_a7x4.dts line 247

Quoted 4 lines
fast_models/rtsm_ve-cortex_a15x4_a7x4.dts +0 -1
@@ -244,4 +339,3 @@
 		/include/ "rtsm_ve-motherboard.dtsi"
 	};
 };
-

--
Shannon

Quoted 8 lines
Quoted 7 lines
Quoted 4 lines

_______________________________________________
kvmarm mailing list [email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm

.

.

Quoted 5 lines
Quoted 2 lines

.

--
Shannon

.

Signature · 1 line
Shannon
#31 lore raw
In reply to a message not in the archive
Quoted 29 lines

-----Original Message-----
From: Shannon Zhao [mailto:[email protected]]
Sent: Thursday, June 19, 2014 5:47 PM
To: Bhushan Bharat-R65777; Alvise Rigo; a.motakis-lrHrjnjw1UfHK3s98zE1ajGjJy/[email protected]; iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org; [email protected]
Subject: Re: Vfio-on-arm: the device tree does not contain the DMA device

Quoted 17 lines · On 2014/6/19 20:12, [email protected]…
Quoted 16 lines

-----Original Message-----
From: Shannon Zhao [mailto:[email protected]]
Sent: Thursday, June 19, 2014 5:38 PM
To: Alvise Rigo; Bhushan Bharat-R65777;
a.motakis-lrHrjnjw1UfHK3s98zE1ajGjJy/[email protected]; iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org; [email protected]
Subject: Re: Vfio-on-arm: the device tree does not contain the DMA
device

Hi Alvise,

I run following cmds :

	echo 2c0a0000.dma >

/sys/bus/platform/drivers/vfio-platform/vfio_bind

Quoted 6 lines
Quoted 1 line

run "ls /dev/vfio/" before and after the above command

Hi Bharat,

Nothing appeared either before or after the cmd "echo 2c0a0000.dma > /sys/bus/platform/drivers/vfio-platform/vfio_bind".
There is only /dev/vfio.

I am not sure, but may be you are using pretty old rootfs with broken udev rules. Try to fetch a newer rootfs form somewhere.

Thanks
-Bharat

Quoted 330 lines
Quoted 8 lines

Thanks
-Bharat

Quoted 3 lines
	./qemu-system-arm -enable-kvm -M vexpress-a15 -cpu cortex-a15 \
        	-device

pl330,vfio_group="/dev/vfio/0",vfio_device="2c0a0000.dma",vfio_container="/dev/v

Quoted 6 lines
Quoted 6 lines

fio/vfio" \
-append "earlyprintk ip=dhcp console=ttyAMA0 rootwait" \
-kernel zImage -dtb guest-a15.dtb -initrd filesystem.cpio.gz -
nographic But the error still exists.
qemu-system-arm: -device

pl330,vfio_group=/dev/vfio/0,vfio_device=2c0a0000.dma,vfio_container=/dev/vfio/v

Quoted 102 lines
Quoted 102 lines

fio: Parameter 'driver' expects pluggable device type

Thanks,
Shannon

Quoted 32 lines · Alvise Rigo

Il 19/06/2014 13:11, Shannon Zhao ha scritto:

Quoted 29 lines · Alvise Rigo
Quoted 26 lines

Hi,

Il 19/06/2014 11:38, Shannon Zhao ha scritto:

Quoted 23 lines

Hi Bharat,

Quoted 20 lines · On 2014/6/19 17:19, [email protected]…
Quoted 20 lines

-----Original Message-----
From: [email protected] [mailto:kvmarm- [email protected]] On Behalf Of Shannon Zhao
Sent: Thursday, June 19, 2014 2:45 PM
To: Alvise Rigo; a.motakis-lrHrjnjw1UfHK3s98zE1ajGjJy/[email protected]; iommu-cunTk1MwBs/[email protected] foundation.org; [email protected]
Subject: Re: Vfio-on-arm: the device tree does not contain the
DMA device

Hi Alvise,

Thanks for your reply.
I have apply the patch and could see the DMA devices. But there is no vosys_dmatest int the directory "/sys/kernel/debug/" after mounting debugfs.
I found that the branch vfio-platform-v4 doesn't contain the module vosys_dmatest.
Then I port it from the branch pl330_dmatest and run vosys_dmatest

successfully.

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
	echo 1 > /sys/kernel/debug/vosys_dmatest/start

	[ 6145.343419] dmaengine: __dma_request_channel: success

(dma0chan0)

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
	[ 6145.344000] vosys dmatest: callback: transfer completed
	[ 6145.344147] vosys dmatest: tx completed
	[ 6145.344589] vosys dmatest: source and destination match.

Download the qemu and use following command to start a guest:

	./qemu-system-arm -enable-kvm -M vexpress-a15 -cpu cortex-a15 \
        	-device

pl330,vfio_group="/dev/vfio/0",vfio_device="2c0a0000.dma",\

Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 8 lines
Quoted 6 lines
        	vfio_container="/dev/vfio/vfio" \
        	-append "earlyprintk ip=dhcp console=ttyAMA0 rootwait" \
        	-kernel zImage -dtb guest.dtb -initrd filesystem.cpio.gz

- nographic An error reported:
qemu-system-arm: -device
pl330,vfio_group=/dev/vfio/0,vfio_device=2c0a0000.dma,:

Why do we need to give "vfio_group=/dev/vfio/0" ?

I refer to the url

"http://www.virtualopensystems.com/en/solutions/guides/vfio-on-arm/".

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 5 lines

could not open disk image vfio_container=/dev/vfio:
Could not open 'vfio_container=/dev/vfio': No such file or
directory

Are you unbinding "2c0a0000.dma" from its driver and binding with "vfio-

playform" driver ?

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

How to unbind "2c0a0000.dma" from its driver?
I have tried following cmd to bind "2c0a0000.dma" with "vfio-playform"

driver.

Quoted 13 lines
Quoted 13 lines
Quoted 11 lines
Quoted 3 lines
	echo 2c0a0000.dma >

/sys/bus/platform/drivers/vfio-platform/vfio_bind
But there is no /dev/vfio/0 or /dev/vfio/vfio .

Can you please verify that while configuring the host you enabled:
- Device Drivers -> IOMMU Hardware Support -> ARM Ltd. System MMU (SMMU) Support and
- Device Drivers -> VFIO Non-Privileged userspace driver framework -> VFIO support for platform devices

Hi Alvise,

Quoted 8 lines
Quoted 8 lines

I have checked the two configuration are enabled.

Then try the following cmd, but another error reported:
qemu-system-arm: -device
pl330,vfio_group=/dev/vfio/0,vfio_device=2c0a0000.dma,vfio_container= /dev/vfio/vfio: Parameter 'driver' expects pluggable device type

Do I need to unbind the "2c0a0000.dma" from its driver and binding with

"vfio-platform" driver as Bharat mentioned ?

Quoted 7 lines

Yes, the 2c0a0000.dma should not be bound to any driver at boot, so you just need to:
echo 2c0a0000.dma > /sys/bus/platform/drivers/vfio-platform/vfio_bind to bind the device to the VFIO driver.

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

Regards,
alvise

Quoted 2 lines
Quoted 2 lines

Then try to update the pl330-vfio-dev branch and run QEMU with the

following

Quoted 5 lines
Quoted 5 lines

command:

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

./qemu-system-arm -enable-kvm -M vexpress-a15 -cpu cortex-a15 \
-device

pl330,vfio_group="/dev/vfio/0",vfio_device="2c0a0000.dma",vfio_container="/dev/v

Quoted 188 lines
Quoted 188 lines

fio/vfio" \

Quoted 16 lines
Quoted 16 lines
Quoted 16 lines

-append "earlyprintk ip=dhcp console=ttyAMA0 rootwait" \
-kernel zImage -dtb guest.dtb -initrd filesystem.cpio.gz

that means without breaking the "-device pl330" options.

Regards,
alvise

Quoted 8 lines
Quoted 8 lines

Thanks
-Bharat

Quoted 3 lines

Is there something I'm wrong or do I miss some steps?

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines

Thank you,
Shannon

Quoted 3 lines · Alvise Rigo

Hi Shannon,

Thank you for reporting this issue, the device tree file was indeed

missing.

Quoted 163 lines
Quoted 162 lines
Quoted 161 lines
Quoted 150 lines
Quoted 150 lines
Quoted 150 lines
Quoted 142 lines

To solve the problem, you can apply the following patch to the repository you mentioned
(git://github.com/virtualopensystems/arm-dts.git).

For your information, soon we will update the guide with a newer version of VFIO with this issue fixed.

Thank you,
alvise

---
 fast_models/rtsm_ve-cortex_a15x4_a7x4.dts | 100
+++++++++++++++++++++++++++++-
 1 file changed, 97 insertions(+), 3 deletions(-)
fast_models/rtsm_ve-cortex_a15x4_a7x4.dts +97 -2
index ef1275b..79c6af2 100644
@@ -142,7 +142,7 @@
 		cpu7: cpu@7 {
 			device_type = "cpu";
 			compatible = "arm,cortex-a7";
-			reg = <0x104>;
+			reg = <0x103>;
 			cluster = <&cluster1>;
 			core = <&core7>;
 		};
@@ -159,7 +159,7 @@
 		#address-cells = <0>;
 		interrupt-controller;
 		reg = <0 0x2c001000 0 0x1000>,
-		      <0 0x2c002000 0 0x2000>,
+		      <0 0x2c002000 0 0x1000>,
 		      <0 0x2c004000 0 0x2000>,
 		      <0 0x2c006000 0 0x2000>;
 		interrupts = <1 9 0xf04>;
@@ -183,6 +183,101 @@
 			     <1 10 0xf08>;
 	};
 
+	smmu@2c0b0000 {
+                compatible = "arm,smmu-v2";
+                reg = <0x0 0x2c0b0000 0x0 0x10000>;
+                #global-interrupts = <2>;
+                interrupts = <0 58 4>,
+                             <0 59 4>,
+                             <0 60 4>,
+                             <0 60 4>,
+                             <0 60 4>,
+                             <0 60 4>,
+                             <0 60 4>,
+                             <0 60 4>,
+                             <0 60 4>;
+                mmu-masters = <&dma0 0x0 0x8>,
+                              <&dma1 0x1 0x9>,
+                              <&dma2 0x2 0xa>,
+                              <&dma3 0x3 0xb>;
+        };
+
+	dma0: dma@2c0a0000 {
+		compatible = "arm,pl330";
+		reg = <0 0x2c0a0000 0 0x1000>;
+		interrupts = <0 64 4>,
+			     <0 65 4>,
+			     <0 66 4>,
+			     <0 67 4>,
+			     <0 68 4>,
+			     <0 69 4>,
+			     <0 70 4>,
+			     <0 71 4>;
+		#dma-cells = <1>;
+		#stream-id-cells = <2>;
+		clocks = <&oscclk6a>;
+		clock-names = "apb_pclk";
+	};
+
+	dma1: dma@2c0a1000 {
+		compatible = "arm,pl330", "arm,primecell";
+		reg = <0 0x2c0a1000 0 0x1000>;
+		interrupts = <0 96 4>,
+			     <0 97 4>,
+			     <0 98 4>,
+			     <0 99 4>,
+			     <0 100 4>,
+			     <0 101 4>,
+			     <0 102 4>,
+			     <0 103 4>;
+		#dma-cells = <1>;
+		#stream-id-cells = <2>;
+		clocks = <&oscclk6a>;
+		clock-names = "apb_pclk";
+	};
+
+	dma2: dma@2c0a2000 {
+		compatible = "arm,pl330", "arm,primecell";
+		reg = <0 0x2c0a2000 0 0x1000>;
+		interrupts = <0 128 4>,
+			     <0 129 4>,
+			     <0 130 4>,
+			     <0 131 4>,
+			     <0 132 4>,
+			     <0 133 4>,
+			     <0 134 4>,
+			     <0 135 4>;
+		#dma-cells = <1>;
+		#stream-id-cells = <2>;
+		clocks = <&oscclk6a>;
+		clock-names = "apb_pclk";
+	};
+
+	dma3: dma@2c0a3000 {
+		compatible = "arm,pl330", "arm,primecell";
+		reg = <0 0x2c0a3000 0 0x1000>;
+		interrupts = <0 160 4>,
+			     <0 161 4>,
+			     <0 162 4>,
+			     <0 163 4>,
+			     <0 164 4>,
+			     <0 165 4>,
+			     <0 166 4>,
+			     <0 167 4>;
+		#dma-cells = <1>;
+		#stream-id-cells = <2>;
+		clocks = <&oscclk6a>;
+		clock-names = "apb_pclk";
+	};
+
+	oscclk6a: oscclk6a {
+		/* Reference 24MHz clock */
+		compatible = "fixed-clock";
+		#clock-cells = <0>;
+		clock-frequency = <24000000>;
+		clock-output-names = "oscclk6a";
+	};
+
 	smb {
 		compatible = "simple-bus";
 

fast_models/rtsm_ve-cortex_a15x4_a7x4.dts line 247

Quoted 4 lines
fast_models/rtsm_ve-cortex_a15x4_a7x4.dts +0 -1
@@ -244,4 +339,3 @@
 		/include/ "rtsm_ve-motherboard.dtsi"
 	};
 };
-

--
Shannon

Quoted 8 lines
Quoted 7 lines
Quoted 4 lines

_______________________________________________
kvmarm mailing list [email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm

.

.

Quoted 9 lines
Quoted 5 lines
Quoted 2 lines

.

--
Shannon

.

--
Shannon

#33 lore raw
In reply to a message not in the archive

Hi Bharat,

Thanks for your advice .

I'm using the rootfs getting from the url "http://www.virtualopensystems.com/downloads/guides/kvm_on_arm/fs-alip-armel.cramfs".

Can you give me another link to get a newer rootfs?

Thank you
Shannon

Quoted 376 lines · On 2014/6/19 20:20, [email protected]…
Quoted 29 lines

-----Original Message-----
From: Shannon Zhao [mailto:[email protected]]
Sent: Thursday, June 19, 2014 5:47 PM
To: Bhushan Bharat-R65777; Alvise Rigo; a.motakis-lrHrjnjw1UfHK3s98zE1ajGjJy/[email protected]; iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org; [email protected]
Subject: Re: Vfio-on-arm: the device tree does not contain the DMA device

Quoted 17 lines · On 2014/6/19 20:12, [email protected]…
Quoted 16 lines

-----Original Message-----
From: Shannon Zhao [mailto:[email protected]]
Sent: Thursday, June 19, 2014 5:38 PM
To: Alvise Rigo; Bhushan Bharat-R65777;
a.motakis-lrHrjnjw1UfHK3s98zE1ajGjJy/[email protected]; iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org; [email protected]
Subject: Re: Vfio-on-arm: the device tree does not contain the DMA
device

Hi Alvise,

I run following cmds :

	echo 2c0a0000.dma >

/sys/bus/platform/drivers/vfio-platform/vfio_bind

Quoted 6 lines
Quoted 1 line

run "ls /dev/vfio/" before and after the above command

Hi Bharat,

Nothing appeared either before or after the cmd "echo 2c0a0000.dma > /sys/bus/platform/drivers/vfio-platform/vfio_bind".
There is only /dev/vfio.

I am not sure, but may be you are using pretty old rootfs with broken udev rules. Try to fetch a newer rootfs form somewhere.

Thanks
-Bharat

Quoted 325 lines
Quoted 8 lines

Thanks
-Bharat

Quoted 3 lines
	./qemu-system-arm -enable-kvm -M vexpress-a15 -cpu cortex-a15 \
        	-device

pl330,vfio_group="/dev/vfio/0",vfio_device="2c0a0000.dma",vfio_container="/dev/v

Quoted 6 lines
Quoted 6 lines

fio/vfio" \
-append "earlyprintk ip=dhcp console=ttyAMA0 rootwait" \
-kernel zImage -dtb guest-a15.dtb -initrd filesystem.cpio.gz -
nographic But the error still exists.
qemu-system-arm: -device

pl330,vfio_group=/dev/vfio/0,vfio_device=2c0a0000.dma,vfio_container=/dev/vfio/v

Quoted 102 lines
Quoted 102 lines

fio: Parameter 'driver' expects pluggable device type

Thanks,
Shannon

Quoted 32 lines · Alvise Rigo

Il 19/06/2014 13:11, Shannon Zhao ha scritto:

Quoted 29 lines · Alvise Rigo
Quoted 26 lines

Hi,

Il 19/06/2014 11:38, Shannon Zhao ha scritto:

Quoted 23 lines

Hi Bharat,

Quoted 20 lines · On 2014/6/19 17:19, [email protected]…
Quoted 20 lines

-----Original Message-----
From: [email protected] [mailto:kvmarm- [email protected]] On Behalf Of Shannon Zhao
Sent: Thursday, June 19, 2014 2:45 PM
To: Alvise Rigo; a.motakis-lrHrjnjw1UfHK3s98zE1ajGjJy/[email protected]; iommu-cunTk1MwBs/[email protected] foundation.org; [email protected]
Subject: Re: Vfio-on-arm: the device tree does not contain the
DMA device

Hi Alvise,

Thanks for your reply.
I have apply the patch and could see the DMA devices. But there is no vosys_dmatest int the directory "/sys/kernel/debug/" after mounting debugfs.
I found that the branch vfio-platform-v4 doesn't contain the module vosys_dmatest.
Then I port it from the branch pl330_dmatest and run vosys_dmatest

successfully.

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
	echo 1 > /sys/kernel/debug/vosys_dmatest/start

	[ 6145.343419] dmaengine: __dma_request_channel: success

(dma0chan0)

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
	[ 6145.344000] vosys dmatest: callback: transfer completed
	[ 6145.344147] vosys dmatest: tx completed
	[ 6145.344589] vosys dmatest: source and destination match.

Download the qemu and use following command to start a guest:

	./qemu-system-arm -enable-kvm -M vexpress-a15 -cpu cortex-a15 \
        	-device

pl330,vfio_group="/dev/vfio/0",vfio_device="2c0a0000.dma",\

Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 8 lines
Quoted 6 lines
        	vfio_container="/dev/vfio/vfio" \
        	-append "earlyprintk ip=dhcp console=ttyAMA0 rootwait" \
        	-kernel zImage -dtb guest.dtb -initrd filesystem.cpio.gz

- nographic An error reported:
qemu-system-arm: -device
pl330,vfio_group=/dev/vfio/0,vfio_device=2c0a0000.dma,:

Why do we need to give "vfio_group=/dev/vfio/0" ?

I refer to the url

"http://www.virtualopensystems.com/en/solutions/guides/vfio-on-arm/".

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 5 lines

could not open disk image vfio_container=/dev/vfio:
Could not open 'vfio_container=/dev/vfio': No such file or
directory

Are you unbinding "2c0a0000.dma" from its driver and binding with "vfio-

playform" driver ?

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

How to unbind "2c0a0000.dma" from its driver?
I have tried following cmd to bind "2c0a0000.dma" with "vfio-playform"

driver.

Quoted 13 lines
Quoted 13 lines
Quoted 11 lines
Quoted 3 lines
	echo 2c0a0000.dma >

/sys/bus/platform/drivers/vfio-platform/vfio_bind
But there is no /dev/vfio/0 or /dev/vfio/vfio .

Can you please verify that while configuring the host you enabled:
- Device Drivers -> IOMMU Hardware Support -> ARM Ltd. System MMU (SMMU) Support and
- Device Drivers -> VFIO Non-Privileged userspace driver framework -> VFIO support for platform devices

Hi Alvise,

Quoted 8 lines
Quoted 8 lines

I have checked the two configuration are enabled.

Then try the following cmd, but another error reported:
qemu-system-arm: -device
pl330,vfio_group=/dev/vfio/0,vfio_device=2c0a0000.dma,vfio_container= /dev/vfio/vfio: Parameter 'driver' expects pluggable device type

Do I need to unbind the "2c0a0000.dma" from its driver and binding with

"vfio-platform" driver as Bharat mentioned ?

Quoted 7 lines

Yes, the 2c0a0000.dma should not be bound to any driver at boot, so you just need to:
echo 2c0a0000.dma > /sys/bus/platform/drivers/vfio-platform/vfio_bind to bind the device to the VFIO driver.

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

Regards,
alvise

Quoted 2 lines
Quoted 2 lines

Then try to update the pl330-vfio-dev branch and run QEMU with the

following

Quoted 5 lines
Quoted 5 lines

command:

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

./qemu-system-arm -enable-kvm -M vexpress-a15 -cpu cortex-a15 \
-device

pl330,vfio_group="/dev/vfio/0",vfio_device="2c0a0000.dma",vfio_container="/dev/v

Quoted 195 lines
Quoted 193 lines

fio/vfio" \

Quoted 16 lines
Quoted 16 lines
Quoted 16 lines

-append "earlyprintk ip=dhcp console=ttyAMA0 rootwait" \
-kernel zImage -dtb guest.dtb -initrd filesystem.cpio.gz

that means without breaking the "-device pl330" options.

Regards,
alvise

Quoted 8 lines
Quoted 8 lines

Thanks
-Bharat

Quoted 3 lines

Is there something I'm wrong or do I miss some steps?

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines

Thank you,
Shannon

Quoted 3 lines · Alvise Rigo

Hi Shannon,

Thank you for reporting this issue, the device tree file was indeed

missing.

Quoted 159 lines
Quoted 159 lines
Quoted 159 lines
Quoted 158 lines
Quoted 157 lines
Quoted 150 lines
Quoted 142 lines

To solve the problem, you can apply the following patch to the repository you mentioned
(git://github.com/virtualopensystems/arm-dts.git).

For your information, soon we will update the guide with a newer version of VFIO with this issue fixed.

Thank you,
alvise

---
 fast_models/rtsm_ve-cortex_a15x4_a7x4.dts | 100
+++++++++++++++++++++++++++++-
 1 file changed, 97 insertions(+), 3 deletions(-)
fast_models/rtsm_ve-cortex_a15x4_a7x4.dts +97 -2
index ef1275b..79c6af2 100644
@@ -142,7 +142,7 @@
 		cpu7: cpu@7 {
 			device_type = "cpu";
 			compatible = "arm,cortex-a7";
-			reg = <0x104>;
+			reg = <0x103>;
 			cluster = <&cluster1>;
 			core = <&core7>;
 		};
@@ -159,7 +159,7 @@
 		#address-cells = <0>;
 		interrupt-controller;
 		reg = <0 0x2c001000 0 0x1000>,
-		      <0 0x2c002000 0 0x2000>,
+		      <0 0x2c002000 0 0x1000>,
 		      <0 0x2c004000 0 0x2000>,
 		      <0 0x2c006000 0 0x2000>;
 		interrupts = <1 9 0xf04>;
@@ -183,6 +183,101 @@
 			     <1 10 0xf08>;
 	};
 
+	smmu@2c0b0000 {
+                compatible = "arm,smmu-v2";
+                reg = <0x0 0x2c0b0000 0x0 0x10000>;
+                #global-interrupts = <2>;
+                interrupts = <0 58 4>,
+                             <0 59 4>,
+                             <0 60 4>,
+                             <0 60 4>,
+                             <0 60 4>,
+                             <0 60 4>,
+                             <0 60 4>,
+                             <0 60 4>,
+                             <0 60 4>;
+                mmu-masters = <&dma0 0x0 0x8>,
+                              <&dma1 0x1 0x9>,
+                              <&dma2 0x2 0xa>,
+                              <&dma3 0x3 0xb>;
+        };
+
+	dma0: dma@2c0a0000 {
+		compatible = "arm,pl330";
+		reg = <0 0x2c0a0000 0 0x1000>;
+		interrupts = <0 64 4>,
+			     <0 65 4>,
+			     <0 66 4>,
+			     <0 67 4>,
+			     <0 68 4>,
+			     <0 69 4>,
+			     <0 70 4>,
+			     <0 71 4>;
+		#dma-cells = <1>;
+		#stream-id-cells = <2>;
+		clocks = <&oscclk6a>;
+		clock-names = "apb_pclk";
+	};
+
+	dma1: dma@2c0a1000 {
+		compatible = "arm,pl330", "arm,primecell";
+		reg = <0 0x2c0a1000 0 0x1000>;
+		interrupts = <0 96 4>,
+			     <0 97 4>,
+			     <0 98 4>,
+			     <0 99 4>,
+			     <0 100 4>,
+			     <0 101 4>,
+			     <0 102 4>,
+			     <0 103 4>;
+		#dma-cells = <1>;
+		#stream-id-cells = <2>;
+		clocks = <&oscclk6a>;
+		clock-names = "apb_pclk";
+	};
+
+	dma2: dma@2c0a2000 {
+		compatible = "arm,pl330", "arm,primecell";
+		reg = <0 0x2c0a2000 0 0x1000>;
+		interrupts = <0 128 4>,
+			     <0 129 4>,
+			     <0 130 4>,
+			     <0 131 4>,
+			     <0 132 4>,
+			     <0 133 4>,
+			     <0 134 4>,
+			     <0 135 4>;
+		#dma-cells = <1>;
+		#stream-id-cells = <2>;
+		clocks = <&oscclk6a>;
+		clock-names = "apb_pclk";
+	};
+
+	dma3: dma@2c0a3000 {
+		compatible = "arm,pl330", "arm,primecell";
+		reg = <0 0x2c0a3000 0 0x1000>;
+		interrupts = <0 160 4>,
+			     <0 161 4>,
+			     <0 162 4>,
+			     <0 163 4>,
+			     <0 164 4>,
+			     <0 165 4>,
+			     <0 166 4>,
+			     <0 167 4>;
+		#dma-cells = <1>;
+		#stream-id-cells = <2>;
+		clocks = <&oscclk6a>;
+		clock-names = "apb_pclk";
+	};
+
+	oscclk6a: oscclk6a {
+		/* Reference 24MHz clock */
+		compatible = "fixed-clock";
+		#clock-cells = <0>;
+		clock-frequency = <24000000>;
+		clock-output-names = "oscclk6a";
+	};
+
 	smb {
 		compatible = "simple-bus";
 

fast_models/rtsm_ve-cortex_a15x4_a7x4.dts line 247

Quoted 4 lines
fast_models/rtsm_ve-cortex_a15x4_a7x4.dts +0 -1
@@ -244,4 +339,3 @@
 		/include/ "rtsm_ve-motherboard.dtsi"
 	};
 };
-

--
Shannon

Quoted 4 lines

_______________________________________________
kvmarm mailing list [email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm

.

Quoted 6 lines
Quoted 3 lines
Quoted 2 lines

.

.

--
Shannon

Quoted 5 lines
Quoted 2 lines

.

--
Shannon

.

Signature · 1 line
Shannon
#35 lore raw
In reply to a message not in the archive

Hi Shannon,

If you follow the steps in the section "Creating the File System" of the guide you should be able to obtain a working fs.

Regards,
alvise

Il 20/06/2014 04:57, Shannon Zhao ha scritto:

Quoted 389 lines

Hi Bharat,

Thanks for your advice .

I'm using the rootfs getting from the url "http://www.virtualopensystems.com/downloads/guides/kvm_on_arm/fs-alip-armel.cramfs".

Can you give me another link to get a newer rootfs?

Thank you
Shannon

Quoted 376 lines · On 2014/6/19 20:20, [email protected]…
Quoted 29 lines

-----Original Message-----
From: Shannon Zhao [mailto:[email protected]]
Sent: Thursday, June 19, 2014 5:47 PM
To: Bhushan Bharat-R65777; Alvise Rigo; a.motakis-lrHrjnjw1UfHK3s98zE1ajGjJy/[email protected]; iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org; [email protected]
Subject: Re: Vfio-on-arm: the device tree does not contain the DMA device

Quoted 17 lines · On 2014/6/19 20:12, [email protected]…
Quoted 16 lines

-----Original Message-----
From: Shannon Zhao [mailto:[email protected]]
Sent: Thursday, June 19, 2014 5:38 PM
To: Alvise Rigo; Bhushan Bharat-R65777;
a.motakis-lrHrjnjw1UfHK3s98zE1ajGjJy/[email protected]; iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org; [email protected]
Subject: Re: Vfio-on-arm: the device tree does not contain the DMA
device

Hi Alvise,

I run following cmds :

	echo 2c0a0000.dma >

/sys/bus/platform/drivers/vfio-platform/vfio_bind

Quoted 6 lines
Quoted 1 line

run "ls /dev/vfio/" before and after the above command

Hi Bharat,

Nothing appeared either before or after the cmd "echo 2c0a0000.dma > /sys/bus/platform/drivers/vfio-platform/vfio_bind".
There is only /dev/vfio.

I am not sure, but may be you are using pretty old rootfs with broken udev rules. Try to fetch a newer rootfs form somewhere.

Thanks
-Bharat

Quoted 325 lines
Quoted 8 lines

Thanks
-Bharat

Quoted 3 lines
	./qemu-system-arm -enable-kvm -M vexpress-a15 -cpu cortex-a15 \
        	-device

pl330,vfio_group="/dev/vfio/0",vfio_device="2c0a0000.dma",vfio_container="/dev/v

Quoted 6 lines
Quoted 6 lines

fio/vfio" \
-append "earlyprintk ip=dhcp console=ttyAMA0 rootwait" \
-kernel zImage -dtb guest-a15.dtb -initrd filesystem.cpio.gz -
nographic But the error still exists.
qemu-system-arm: -device

pl330,vfio_group=/dev/vfio/0,vfio_device=2c0a0000.dma,vfio_container=/dev/vfio/v

Quoted 102 lines
Quoted 102 lines

fio: Parameter 'driver' expects pluggable device type

Thanks,
Shannon

Quoted 32 lines · Alvise Rigo

Il 19/06/2014 13:11, Shannon Zhao ha scritto:

Quoted 29 lines · Alvise Rigo
Quoted 26 lines

Hi,

Il 19/06/2014 11:38, Shannon Zhao ha scritto:

Quoted 23 lines

Hi Bharat,

Quoted 20 lines · On 2014/6/19 17:19, [email protected]…
Quoted 20 lines

-----Original Message-----
From: [email protected] [mailto:kvmarm- [email protected]] On Behalf Of Shannon Zhao
Sent: Thursday, June 19, 2014 2:45 PM
To: Alvise Rigo; a.motakis-lrHrjnjw1UfHK3s98zE1ajGjJy/[email protected]; iommu-cunTk1MwBs/[email protected] foundation.org; [email protected]
Subject: Re: Vfio-on-arm: the device tree does not contain the
DMA device

Hi Alvise,

Thanks for your reply.
I have apply the patch and could see the DMA devices. But there is no vosys_dmatest int the directory "/sys/kernel/debug/" after mounting debugfs.
I found that the branch vfio-platform-v4 doesn't contain the module vosys_dmatest.
Then I port it from the branch pl330_dmatest and run vosys_dmatest

successfully.

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
	echo 1 > /sys/kernel/debug/vosys_dmatest/start

	[ 6145.343419] dmaengine: __dma_request_channel: success

(dma0chan0)

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
	[ 6145.344000] vosys dmatest: callback: transfer completed
	[ 6145.344147] vosys dmatest: tx completed
	[ 6145.344589] vosys dmatest: source and destination match.

Download the qemu and use following command to start a guest:

	./qemu-system-arm -enable-kvm -M vexpress-a15 -cpu cortex-a15 \
        	-device

pl330,vfio_group="/dev/vfio/0",vfio_device="2c0a0000.dma",\

Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 8 lines
Quoted 6 lines
        	vfio_container="/dev/vfio/vfio" \
        	-append "earlyprintk ip=dhcp console=ttyAMA0 rootwait" \
        	-kernel zImage -dtb guest.dtb -initrd filesystem.cpio.gz

- nographic An error reported:
qemu-system-arm: -device
pl330,vfio_group=/dev/vfio/0,vfio_device=2c0a0000.dma,:

Why do we need to give "vfio_group=/dev/vfio/0" ?

I refer to the url

"http://www.virtualopensystems.com/en/solutions/guides/vfio-on-arm/".

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 5 lines

could not open disk image vfio_container=/dev/vfio:
Could not open 'vfio_container=/dev/vfio': No such file or
directory

Are you unbinding "2c0a0000.dma" from its driver and binding with "vfio-

playform" driver ?

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

How to unbind "2c0a0000.dma" from its driver?
I have tried following cmd to bind "2c0a0000.dma" with "vfio-playform"

driver.

Quoted 13 lines
Quoted 13 lines
Quoted 11 lines
Quoted 3 lines
	echo 2c0a0000.dma >

/sys/bus/platform/drivers/vfio-platform/vfio_bind
But there is no /dev/vfio/0 or /dev/vfio/vfio .

Can you please verify that while configuring the host you enabled:
- Device Drivers -> IOMMU Hardware Support -> ARM Ltd. System MMU (SMMU) Support and
- Device Drivers -> VFIO Non-Privileged userspace driver framework -> VFIO support for platform devices

Hi Alvise,

Quoted 8 lines
Quoted 8 lines

I have checked the two configuration are enabled.

Then try the following cmd, but another error reported:
qemu-system-arm: -device
pl330,vfio_group=/dev/vfio/0,vfio_device=2c0a0000.dma,vfio_container= /dev/vfio/vfio: Parameter 'driver' expects pluggable device type

Do I need to unbind the "2c0a0000.dma" from its driver and binding with

"vfio-platform" driver as Bharat mentioned ?

Quoted 7 lines

Yes, the 2c0a0000.dma should not be bound to any driver at boot, so you just need to:
echo 2c0a0000.dma > /sys/bus/platform/drivers/vfio-platform/vfio_bind to bind the device to the VFIO driver.

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

Regards,
alvise

Quoted 2 lines
Quoted 2 lines

Then try to update the pl330-vfio-dev branch and run QEMU with the

following

Quoted 5 lines
Quoted 5 lines

command:

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

./qemu-system-arm -enable-kvm -M vexpress-a15 -cpu cortex-a15 \
-device

pl330,vfio_group="/dev/vfio/0",vfio_device="2c0a0000.dma",vfio_container="/dev/v

Quoted 195 lines
Quoted 193 lines

fio/vfio" \

Quoted 16 lines
Quoted 16 lines
Quoted 16 lines

-append "earlyprintk ip=dhcp console=ttyAMA0 rootwait" \
-kernel zImage -dtb guest.dtb -initrd filesystem.cpio.gz

that means without breaking the "-device pl330" options.

Regards,
alvise

Quoted 8 lines
Quoted 8 lines

Thanks
-Bharat

Quoted 3 lines

Is there something I'm wrong or do I miss some steps?

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines

Thank you,
Shannon

Quoted 3 lines · Alvise Rigo

Hi Shannon,

Thank you for reporting this issue, the device tree file was indeed

missing.

Quoted 159 lines
Quoted 159 lines
Quoted 159 lines
Quoted 158 lines
Quoted 157 lines
Quoted 150 lines
Quoted 142 lines

To solve the problem, you can apply the following patch to the repository you mentioned
(git://github.com/virtualopensystems/arm-dts.git).

For your information, soon we will update the guide with a newer version of VFIO with this issue fixed.

Thank you,
alvise

---
 fast_models/rtsm_ve-cortex_a15x4_a7x4.dts | 100
+++++++++++++++++++++++++++++-
 1 file changed, 97 insertions(+), 3 deletions(-)
fast_models/rtsm_ve-cortex_a15x4_a7x4.dts +97 -2
index ef1275b..79c6af2 100644
@@ -142,7 +142,7 @@
 		cpu7: cpu@7 {
 			device_type = "cpu";
 			compatible = "arm,cortex-a7";
-			reg = <0x104>;
+			reg = <0x103>;
 			cluster = <&cluster1>;
 			core = <&core7>;
 		};
@@ -159,7 +159,7 @@
 		#address-cells = <0>;
 		interrupt-controller;
 		reg = <0 0x2c001000 0 0x1000>,
-		      <0 0x2c002000 0 0x2000>,
+		      <0 0x2c002000 0 0x1000>,
 		      <0 0x2c004000 0 0x2000>,
 		      <0 0x2c006000 0 0x2000>;
 		interrupts = <1 9 0xf04>;
@@ -183,6 +183,101 @@
 			     <1 10 0xf08>;
 	};
 
+	smmu@2c0b0000 {
+                compatible = "arm,smmu-v2";
+                reg = <0x0 0x2c0b0000 0x0 0x10000>;
+                #global-interrupts = <2>;
+                interrupts = <0 58 4>,
+                             <0 59 4>,
+                             <0 60 4>,
+                             <0 60 4>,
+                             <0 60 4>,
+                             <0 60 4>,
+                             <0 60 4>,
+                             <0 60 4>,
+                             <0 60 4>;
+                mmu-masters = <&dma0 0x0 0x8>,
+                              <&dma1 0x1 0x9>,
+                              <&dma2 0x2 0xa>,
+                              <&dma3 0x3 0xb>;
+        };
+
+	dma0: dma@2c0a0000 {
+		compatible = "arm,pl330";
+		reg = <0 0x2c0a0000 0 0x1000>;
+		interrupts = <0 64 4>,
+			     <0 65 4>,
+			     <0 66 4>,
+			     <0 67 4>,
+			     <0 68 4>,
+			     <0 69 4>,
+			     <0 70 4>,
+			     <0 71 4>;
+		#dma-cells = <1>;
+		#stream-id-cells = <2>;
+		clocks = <&oscclk6a>;
+		clock-names = "apb_pclk";
+	};
+
+	dma1: dma@2c0a1000 {
+		compatible = "arm,pl330", "arm,primecell";
+		reg = <0 0x2c0a1000 0 0x1000>;
+		interrupts = <0 96 4>,
+			     <0 97 4>,
+			     <0 98 4>,
+			     <0 99 4>,
+			     <0 100 4>,
+			     <0 101 4>,
+			     <0 102 4>,
+			     <0 103 4>;
+		#dma-cells = <1>;
+		#stream-id-cells = <2>;
+		clocks = <&oscclk6a>;
+		clock-names = "apb_pclk";
+	};
+
+	dma2: dma@2c0a2000 {
+		compatible = "arm,pl330", "arm,primecell";
+		reg = <0 0x2c0a2000 0 0x1000>;
+		interrupts = <0 128 4>,
+			     <0 129 4>,
+			     <0 130 4>,
+			     <0 131 4>,
+			     <0 132 4>,
+			     <0 133 4>,
+			     <0 134 4>,
+			     <0 135 4>;
+		#dma-cells = <1>;
+		#stream-id-cells = <2>;
+		clocks = <&oscclk6a>;
+		clock-names = "apb_pclk";
+	};
+
+	dma3: dma@2c0a3000 {
+		compatible = "arm,pl330", "arm,primecell";
+		reg = <0 0x2c0a3000 0 0x1000>;
+		interrupts = <0 160 4>,
+			     <0 161 4>,
+			     <0 162 4>,
+			     <0 163 4>,
+			     <0 164 4>,
+			     <0 165 4>,
+			     <0 166 4>,
+			     <0 167 4>;
+		#dma-cells = <1>;
+		#stream-id-cells = <2>;
+		clocks = <&oscclk6a>;
+		clock-names = "apb_pclk";
+	};
+
+	oscclk6a: oscclk6a {
+		/* Reference 24MHz clock */
+		compatible = "fixed-clock";
+		#clock-cells = <0>;
+		clock-frequency = <24000000>;
+		clock-output-names = "oscclk6a";
+	};
+
 	smb {
 		compatible = "simple-bus";
 

fast_models/rtsm_ve-cortex_a15x4_a7x4.dts line 247

Quoted 4 lines
fast_models/rtsm_ve-cortex_a15x4_a7x4.dts +0 -1
@@ -244,4 +339,3 @@
 		/include/ "rtsm_ve-motherboard.dtsi"
 	};
 };
-

--
Shannon

Quoted 4 lines

_______________________________________________
kvmarm mailing list [email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm

.

Quoted 6 lines
Quoted 3 lines
Quoted 2 lines

.

.

--
Shannon

Quoted 5 lines
Quoted 2 lines

.

--
Shannon

.

#37 lore raw
In reply to a message not in the archive

Hi Alvise,

I'm so sorry that I can't use qemu-debootstrap because my environment can't access some links using proxy. Are there other ways to get a proper fs?

Thanks
Shannon

Quoted 401 lines · Alvise Rigo

Hi Shannon,

If you follow the steps in the section "Creating the File System" of the guide you should be able to obtain a working fs.

Regards,
alvise

Il 20/06/2014 04:57, Shannon Zhao ha scritto:

Quoted 4 lines

Hi Bharat,

Thanks for your advice .

Quoted 385 lines

I'm using the rootfs getting from the url "http://www.virtualopensystems.com/downloads/guides/kvm_on_arm/fs-alip-armel.cramfs".

Can you give me another link to get a newer rootfs?

Thank you
Shannon

Quoted 376 lines · On 2014/6/19 20:20, [email protected]…
Quoted 29 lines

-----Original Message-----
From: Shannon Zhao [mailto:[email protected]]
Sent: Thursday, June 19, 2014 5:47 PM
To: Bhushan Bharat-R65777; Alvise Rigo; a.motakis-lrHrjnjw1UfHK3s98zE1ajGjJy/[email protected]; iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org; [email protected]
Subject: Re: Vfio-on-arm: the device tree does not contain the DMA device

Quoted 17 lines · On 2014/6/19 20:12, [email protected]…
Quoted 16 lines

-----Original Message-----
From: Shannon Zhao [mailto:[email protected]]
Sent: Thursday, June 19, 2014 5:38 PM
To: Alvise Rigo; Bhushan Bharat-R65777;
a.motakis-lrHrjnjw1UfHK3s98zE1ajGjJy/[email protected]; iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org; [email protected]
Subject: Re: Vfio-on-arm: the device tree does not contain the DMA
device

Hi Alvise,

I run following cmds :

	echo 2c0a0000.dma >

/sys/bus/platform/drivers/vfio-platform/vfio_bind

Quoted 6 lines
Quoted 1 line

run "ls /dev/vfio/" before and after the above command

Hi Bharat,

Nothing appeared either before or after the cmd "echo 2c0a0000.dma > /sys/bus/platform/drivers/vfio-platform/vfio_bind".
There is only /dev/vfio.

I am not sure, but may be you are using pretty old rootfs with broken udev rules. Try to fetch a newer rootfs form somewhere.

Thanks
-Bharat

Quoted 325 lines
Quoted 8 lines

Thanks
-Bharat

Quoted 3 lines
	./qemu-system-arm -enable-kvm -M vexpress-a15 -cpu cortex-a15 \
        	-device

pl330,vfio_group="/dev/vfio/0",vfio_device="2c0a0000.dma",vfio_container="/dev/v

Quoted 6 lines
Quoted 6 lines

fio/vfio" \
-append "earlyprintk ip=dhcp console=ttyAMA0 rootwait" \
-kernel zImage -dtb guest-a15.dtb -initrd filesystem.cpio.gz -
nographic But the error still exists.
qemu-system-arm: -device

pl330,vfio_group=/dev/vfio/0,vfio_device=2c0a0000.dma,vfio_container=/dev/vfio/v

Quoted 102 lines
Quoted 102 lines

fio: Parameter 'driver' expects pluggable device type

Thanks,
Shannon

Quoted 32 lines · Alvise Rigo

Il 19/06/2014 13:11, Shannon Zhao ha scritto:

Quoted 29 lines · Alvise Rigo
Quoted 26 lines

Hi,

Il 19/06/2014 11:38, Shannon Zhao ha scritto:

Quoted 23 lines

Hi Bharat,

Quoted 20 lines · On 2014/6/19 17:19, [email protected]…
Quoted 20 lines

-----Original Message-----
From: [email protected] [mailto:kvmarm- [email protected]] On Behalf Of Shannon Zhao
Sent: Thursday, June 19, 2014 2:45 PM
To: Alvise Rigo; a.motakis-lrHrjnjw1UfHK3s98zE1ajGjJy/[email protected]; iommu-cunTk1MwBs/[email protected] foundation.org; [email protected]
Subject: Re: Vfio-on-arm: the device tree does not contain the
DMA device

Hi Alvise,

Thanks for your reply.
I have apply the patch and could see the DMA devices. But there is no vosys_dmatest int the directory "/sys/kernel/debug/" after mounting debugfs.
I found that the branch vfio-platform-v4 doesn't contain the module vosys_dmatest.
Then I port it from the branch pl330_dmatest and run vosys_dmatest

successfully.

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
	echo 1 > /sys/kernel/debug/vosys_dmatest/start

	[ 6145.343419] dmaengine: __dma_request_channel: success

(dma0chan0)

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
	[ 6145.344000] vosys dmatest: callback: transfer completed
	[ 6145.344147] vosys dmatest: tx completed
	[ 6145.344589] vosys dmatest: source and destination match.

Download the qemu and use following command to start a guest:

	./qemu-system-arm -enable-kvm -M vexpress-a15 -cpu cortex-a15 \
        	-device

pl330,vfio_group="/dev/vfio/0",vfio_device="2c0a0000.dma",\

Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 8 lines
Quoted 6 lines
        	vfio_container="/dev/vfio/vfio" \
        	-append "earlyprintk ip=dhcp console=ttyAMA0 rootwait" \
        	-kernel zImage -dtb guest.dtb -initrd filesystem.cpio.gz

- nographic An error reported:
qemu-system-arm: -device
pl330,vfio_group=/dev/vfio/0,vfio_device=2c0a0000.dma,:

Why do we need to give "vfio_group=/dev/vfio/0" ?

I refer to the url

"http://www.virtualopensystems.com/en/solutions/guides/vfio-on-arm/".

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 5 lines

could not open disk image vfio_container=/dev/vfio:
Could not open 'vfio_container=/dev/vfio': No such file or
directory

Are you unbinding "2c0a0000.dma" from its driver and binding with "vfio-

playform" driver ?

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

How to unbind "2c0a0000.dma" from its driver?
I have tried following cmd to bind "2c0a0000.dma" with "vfio-playform"

driver.

Quoted 13 lines
Quoted 13 lines
Quoted 11 lines
Quoted 3 lines
	echo 2c0a0000.dma >

/sys/bus/platform/drivers/vfio-platform/vfio_bind
But there is no /dev/vfio/0 or /dev/vfio/vfio .

Can you please verify that while configuring the host you enabled:
- Device Drivers -> IOMMU Hardware Support -> ARM Ltd. System MMU (SMMU) Support and
- Device Drivers -> VFIO Non-Privileged userspace driver framework -> VFIO support for platform devices

Hi Alvise,

Quoted 8 lines
Quoted 8 lines

I have checked the two configuration are enabled.

Then try the following cmd, but another error reported:
qemu-system-arm: -device
pl330,vfio_group=/dev/vfio/0,vfio_device=2c0a0000.dma,vfio_container= /dev/vfio/vfio: Parameter 'driver' expects pluggable device type

Do I need to unbind the "2c0a0000.dma" from its driver and binding with

"vfio-platform" driver as Bharat mentioned ?

Quoted 7 lines

Yes, the 2c0a0000.dma should not be bound to any driver at boot, so you just need to:
echo 2c0a0000.dma > /sys/bus/platform/drivers/vfio-platform/vfio_bind to bind the device to the VFIO driver.

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

Regards,
alvise

Quoted 2 lines
Quoted 2 lines

Then try to update the pl330-vfio-dev branch and run QEMU with the

following

Quoted 5 lines
Quoted 5 lines

command:

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

./qemu-system-arm -enable-kvm -M vexpress-a15 -cpu cortex-a15 \
-device

pl330,vfio_group="/dev/vfio/0",vfio_device="2c0a0000.dma",vfio_container="/dev/v

Quoted 195 lines
Quoted 193 lines

fio/vfio" \

Quoted 16 lines
Quoted 16 lines
Quoted 16 lines

-append "earlyprintk ip=dhcp console=ttyAMA0 rootwait" \
-kernel zImage -dtb guest.dtb -initrd filesystem.cpio.gz

that means without breaking the "-device pl330" options.

Regards,
alvise

Quoted 8 lines
Quoted 8 lines

Thanks
-Bharat

Quoted 3 lines

Is there something I'm wrong or do I miss some steps?

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines

Thank you,
Shannon

Quoted 3 lines · Alvise Rigo

Hi Shannon,

Thank you for reporting this issue, the device tree file was indeed

missing.

Quoted 159 lines
Quoted 159 lines
Quoted 159 lines
Quoted 158 lines
Quoted 157 lines
Quoted 150 lines
Quoted 146 lines

To solve the problem, you can apply the following patch to the
repository you mentioned
(git://github.com/virtualopensystems/arm-dts.git).

For your information, soon we will update the guide with a newer
version of VFIO with this issue fixed.

Thank you,
alvise

---
fast_models/rtsm_ve-cortex_a15x4_a7x4.dts | 100
+++++++++++++++++++++++++++++-
1 file changed, 97 insertions(+), 3 deletions(-)

diff --git a/fast_models/rtsm_ve-cortex_a15x4_a7x4.dts
b/fast_models/rtsm_ve-cortex_a15x4_a7x4.dts
index ef1275b..79c6af2 100644
--- a/fast_models/rtsm_ve-cortex_a15x4_a7x4.dts
+++ b/fast_models/rtsm_ve-cortex_a15x4_a7x4.dts
@@ -142,7 +142,7 @@
cpu7: cpu@7 {
device_type = "cpu";
compatible = "arm,cortex-a7";
- reg = <0x104>;
+ reg = <0x103>;
cluster = <&cluster1>;
core = <&core7>;
};
@@ -159,7 +159,7 @@
#address-cells = <0>;
interrupt-controller;
reg = <0 0x2c001000 0 0x1000>,
- <0 0x2c002000 0 0x2000>,
+ <0 0x2c002000 0 0x1000>,
<0 0x2c004000 0 0x2000>,
<0 0x2c006000 0 0x2000>;
interrupts = <1 9 0xf04>;
@@ -183,6 +183,101 @@
<1 10 0xf08>;
};

+ smmu@2c0b0000 {
+ compatible = "arm,smmu-v2";
+ reg = <0x0 0x2c0b0000 0x0 0x10000>;
+ #global-interrupts = <2>;
+ interrupts = <0 58 4>,
+ <0 59 4>,
+ <0 60 4>,
+ <0 60 4>,
+ <0 60 4>,
+ <0 60 4>,
+ <0 60 4>,
+ <0 60 4>,
+ <0 60 4>;
+ mmu-masters = <&dma0 0x0 0x8>,
+ <&dma1 0x1 0x9>,
+ <&dma2 0x2 0xa>,
+ <&dma3 0x3 0xb>;
+ };
+
+ dma0: dma@2c0a0000 {
+ compatible = "arm,pl330";
+ reg = <0 0x2c0a0000 0 0x1000>;
+ interrupts = <0 64 4>,
+ <0 65 4>,
+ <0 66 4>,
+ <0 67 4>,
+ <0 68 4>,
+ <0 69 4>,
+ <0 70 4>,
+ <0 71 4>;
+ #dma-cells = <1>;
+ #stream-id-cells = <2>;
+ clocks = <&oscclk6a>;
+ clock-names = "apb_pclk";
+ };
+
+ dma1: dma@2c0a1000 {
+ compatible = "arm,pl330", "arm,primecell";
+ reg = <0 0x2c0a1000 0 0x1000>;
+ interrupts = <0 96 4>,
+ <0 97 4>,
+ <0 98 4>,
+ <0 99 4>,
+ <0 100 4>,
+ <0 101 4>,
+ <0 102 4>,
+ <0 103 4>;
+ #dma-cells = <1>;
+ #stream-id-cells = <2>;
+ clocks = <&oscclk6a>;
+ clock-names = "apb_pclk";
+ };
+
+ dma2: dma@2c0a2000 {
+ compatible = "arm,pl330", "arm,primecell";
+ reg = <0 0x2c0a2000 0 0x1000>;
+ interrupts = <0 128 4>,
+ <0 129 4>,
+ <0 130 4>,
+ <0 131 4>,
+ <0 132 4>,
+ <0 133 4>,
+ <0 134 4>,
+ <0 135 4>;
+ #dma-cells = <1>;
+ #stream-id-cells = <2>;
+ clocks = <&oscclk6a>;
+ clock-names = "apb_pclk";
+ };
+
+ dma3: dma@2c0a3000 {
+ compatible = "arm,pl330", "arm,primecell";
+ reg = <0 0x2c0a3000 0 0x1000>;
+ interrupts = <0 160 4>,
+ <0 161 4>,
+ <0 162 4>,
+ <0 163 4>,
+ <0 164 4>,
+ <0 165 4>,
+ <0 166 4>,
+ <0 167 4>;
+ #dma-cells = <1>;
+ #stream-id-cells = <2>;
+ clocks = <&oscclk6a>;
+ clock-names = "apb_pclk";
+ };
+
+ oscclk6a: oscclk6a {
+ /* Reference 24MHz clock */
+ compatible = "fixed-clock";
+ #clock-cells = <0>;
+ clock-frequency = <24000000>;
+ clock-output-names = "oscclk6a";
+ };
+
smb {
compatible = "simple-bus";

@@ -244,4 +339,3 @@
/include/ "rtsm_ve-motherboard.dtsi"
};
};
-

--
Shannon

Quoted 4 lines

_______________________________________________
kvmarm mailing list [email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm

.

Quoted 6 lines
Quoted 3 lines
Quoted 2 lines

.

.

--
Shannon

Quoted 5 lines
Quoted 2 lines

.

--
Shannon

.

.

Signature · 1 line
Shannon
#39 lore raw
In reply to a message not in the archive

Linaro has a great set of pre-built images here http://releases.linaro.org/.

For example these should work fine:
http://releases.linaro.org/13.12/ubuntu/saucy-images

alvise

Il 20/06/2014 10:23, Shannon Zhao ha scritto:

Quoted 411 lines

Hi Alvise,

I'm so sorry that I can't use qemu-debootstrap because my environment can't access some links using proxy. Are there other ways to get a proper fs?

Thanks
Shannon

Quoted 401 lines · Alvise Rigo

Hi Shannon,

If you follow the steps in the section "Creating the File System" of the guide you should be able to obtain a working fs.

Regards,
alvise

Il 20/06/2014 04:57, Shannon Zhao ha scritto:

Quoted 4 lines

Hi Bharat,

Thanks for your advice .

Quoted 385 lines

I'm using the rootfs getting from the url "http://www.virtualopensystems.com/downloads/guides/kvm_on_arm/fs-alip-armel.cramfs".

Can you give me another link to get a newer rootfs?

Thank you
Shannon

Quoted 376 lines · On 2014/6/19 20:20, [email protected]…
Quoted 29 lines

-----Original Message-----
From: Shannon Zhao [mailto:[email protected]]
Sent: Thursday, June 19, 2014 5:47 PM
To: Bhushan Bharat-R65777; Alvise Rigo; a.motakis-lrHrjnjw1UfHK3s98zE1ajGjJy/[email protected]; iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org; [email protected]
Subject: Re: Vfio-on-arm: the device tree does not contain the DMA device

Quoted 17 lines · On 2014/6/19 20:12, [email protected]…
Quoted 16 lines

-----Original Message-----
From: Shannon Zhao [mailto:[email protected]]
Sent: Thursday, June 19, 2014 5:38 PM
To: Alvise Rigo; Bhushan Bharat-R65777;
a.motakis-lrHrjnjw1UfHK3s98zE1ajGjJy/[email protected]; iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org; [email protected]
Subject: Re: Vfio-on-arm: the device tree does not contain the DMA
device

Hi Alvise,

I run following cmds :

	echo 2c0a0000.dma >

/sys/bus/platform/drivers/vfio-platform/vfio_bind

Quoted 6 lines
Quoted 1 line

run "ls /dev/vfio/" before and after the above command

Hi Bharat,

Nothing appeared either before or after the cmd "echo 2c0a0000.dma > /sys/bus/platform/drivers/vfio-platform/vfio_bind".
There is only /dev/vfio.

I am not sure, but may be you are using pretty old rootfs with broken udev rules. Try to fetch a newer rootfs form somewhere.

Thanks
-Bharat

Quoted 325 lines
Quoted 8 lines

Thanks
-Bharat

Quoted 3 lines
	./qemu-system-arm -enable-kvm -M vexpress-a15 -cpu cortex-a15 \
        	-device

pl330,vfio_group="/dev/vfio/0",vfio_device="2c0a0000.dma",vfio_container="/dev/v

Quoted 6 lines
Quoted 6 lines

fio/vfio" \
-append "earlyprintk ip=dhcp console=ttyAMA0 rootwait" \
-kernel zImage -dtb guest-a15.dtb -initrd filesystem.cpio.gz -
nographic But the error still exists.
qemu-system-arm: -device

pl330,vfio_group=/dev/vfio/0,vfio_device=2c0a0000.dma,vfio_container=/dev/vfio/v

Quoted 102 lines
Quoted 102 lines

fio: Parameter 'driver' expects pluggable device type

Thanks,
Shannon

Quoted 32 lines · Alvise Rigo

Il 19/06/2014 13:11, Shannon Zhao ha scritto:

Quoted 29 lines · Alvise Rigo
Quoted 26 lines

Hi,

Il 19/06/2014 11:38, Shannon Zhao ha scritto:

Quoted 23 lines

Hi Bharat,

Quoted 20 lines · On 2014/6/19 17:19, [email protected]…
Quoted 20 lines

-----Original Message-----
From: [email protected] [mailto:kvmarm-
[email protected]] On Behalf Of Shannon Zhao
Sent: Thursday, June 19, 2014 2:45 PM
To: Alvise Rigo; a.motakis-lrHrjnjw1UfHK3s98zE1ajGjJy/[email protected];
iommu-cunTk1MwBs/[email protected] foundation.org; [email protected]
Subject: Re: Vfio-on-arm: the device tree does not contain the
DMA device

Hi Alvise,

Thanks for your reply.
I have apply the patch and could see the DMA devices.
But there is no vosys_dmatest int the directory
"/sys/kernel/debug/" after mounting debugfs.
I found that the branch vfio-platform-v4 doesn't contain the
module vosys_dmatest.
Then I port it from the branch pl330_dmatest and run vosys_dmatest

successfully.

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

echo 1 > /sys/kernel/debug/vosys_dmatest/start

[ 6145.343419] dmaengine: __dma_request_channel: success

(dma0chan0)

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines

[ 6145.344000] vosys dmatest: callback: transfer completed
[ 6145.344147] vosys dmatest: tx completed
[ 6145.344589] vosys dmatest: source and destination match.

Download the qemu and use following command to start a guest:
./qemu-system-arm -enable-kvm -M vexpress-a15 -cpu cortex-a15 \
-device

pl330,vfio_group="/dev/vfio/0",vfio_device="2c0a0000.dma",\

Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 8 lines
Quoted 6 lines

vfio_container="/dev/vfio/vfio" \
-append "earlyprintk ip=dhcp console=ttyAMA0 rootwait" \
-kernel zImage -dtb guest.dtb -initrd filesystem.cpio.gz
- nographic An error reported:
qemu-system-arm: -device
pl330,vfio_group=/dev/vfio/0,vfio_device=2c0a0000.dma,:

Why do we need to give "vfio_group=/dev/vfio/0" ?

I refer to the url

"http://www.virtualopensystems.com/en/solutions/guides/vfio-on-arm/".

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 5 lines

could not open disk image vfio_container=/dev/vfio:
Could not open 'vfio_container=/dev/vfio': No such file or
directory

Are you unbinding "2c0a0000.dma" from its driver and binding with "vfio-

playform" driver ?

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

How to unbind "2c0a0000.dma" from its driver?
I have tried following cmd to bind "2c0a0000.dma" with "vfio-playform"

driver.

Quoted 13 lines
Quoted 13 lines
Quoted 11 lines
Quoted 3 lines
	echo 2c0a0000.dma >

/sys/bus/platform/drivers/vfio-platform/vfio_bind
But there is no /dev/vfio/0 or /dev/vfio/vfio .

Can you please verify that while configuring the host you enabled:
- Device Drivers -> IOMMU Hardware Support -> ARM Ltd. System MMU (SMMU) Support and
- Device Drivers -> VFIO Non-Privileged userspace driver framework -> VFIO support for platform devices

Hi Alvise,

Quoted 8 lines
Quoted 8 lines

I have checked the two configuration are enabled.

Then try the following cmd, but another error reported:
qemu-system-arm: -device
pl330,vfio_group=/dev/vfio/0,vfio_device=2c0a0000.dma,vfio_container= /dev/vfio/vfio: Parameter 'driver' expects pluggable device type

Do I need to unbind the "2c0a0000.dma" from its driver and binding with

"vfio-platform" driver as Bharat mentioned ?

Quoted 7 lines

Yes, the 2c0a0000.dma should not be bound to any driver at boot, so you just need to:
echo 2c0a0000.dma > /sys/bus/platform/drivers/vfio-platform/vfio_bind to bind the device to the VFIO driver.

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

Regards,
alvise

Quoted 2 lines
Quoted 2 lines

Then try to update the pl330-vfio-dev branch and run QEMU with the

following

Quoted 5 lines
Quoted 5 lines

command:

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

./qemu-system-arm -enable-kvm -M vexpress-a15 -cpu cortex-a15 \
-device

pl330,vfio_group="/dev/vfio/0",vfio_device="2c0a0000.dma",vfio_container="/dev/v

Quoted 195 lines
Quoted 193 lines

fio/vfio" \

Quoted 16 lines
Quoted 16 lines
Quoted 16 lines

-append "earlyprintk ip=dhcp console=ttyAMA0 rootwait" \
-kernel zImage -dtb guest.dtb -initrd filesystem.cpio.gz

that means without breaking the "-device pl330" options.

Regards,
alvise

Quoted 8 lines
Quoted 8 lines

Thanks
-Bharat

Quoted 3 lines

Is there something I'm wrong or do I miss some steps?

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines

Thank you,
Shannon

On 2014/6/18 21:24, Alvise Rigo wrote:
Hi Shannon,

Thank you for reporting this issue, the device tree file was indeed

missing.

Quoted 159 lines
Quoted 159 lines
Quoted 159 lines
Quoted 158 lines
Quoted 157 lines
Quoted 154 lines

To solve the problem, you can apply the following patch to the
repository you mentioned
(git://github.com/virtualopensystems/arm-dts.git).

For your information, soon we will update the guide with a newer
version of VFIO with this issue fixed.

Thank you,
alvise

---
fast_models/rtsm_ve-cortex_a15x4_a7x4.dts | 100
+++++++++++++++++++++++++++++-
1 file changed, 97 insertions(+), 3 deletions(-)

diff --git a/fast_models/rtsm_ve-cortex_a15x4_a7x4.dts
b/fast_models/rtsm_ve-cortex_a15x4_a7x4.dts
index ef1275b..79c6af2 100644
--- a/fast_models/rtsm_ve-cortex_a15x4_a7x4.dts
+++ b/fast_models/rtsm_ve-cortex_a15x4_a7x4.dts
@@ -142,7 +142,7 @@
cpu7: cpu@7 {
device_type = "cpu";
compatible = "arm,cortex-a7";
- reg = <0x104>;
+ reg = <0x103>;
cluster = <&cluster1>;
core = <&core7>;
};
@@ -159,7 +159,7 @@
#address-cells = <0>;
interrupt-controller;
reg = <0 0x2c001000 0 0x1000>,
- <0 0x2c002000 0 0x2000>,
+ <0 0x2c002000 0 0x1000>,
<0 0x2c004000 0 0x2000>,
<0 0x2c006000 0 0x2000>;
interrupts = <1 9 0xf04>;
@@ -183,6 +183,101 @@
<1 10 0xf08>;
};

+ smmu@2c0b0000 {
+ compatible = "arm,smmu-v2";
+ reg = <0x0 0x2c0b0000 0x0 0x10000>;
+ #global-interrupts = <2>;
+ interrupts = <0 58 4>,
+ <0 59 4>,
+ <0 60 4>,
+ <0 60 4>,
+ <0 60 4>,
+ <0 60 4>,
+ <0 60 4>,
+ <0 60 4>,
+ <0 60 4>;
+ mmu-masters = <&dma0 0x0 0x8>,
+ <&dma1 0x1 0x9>,
+ <&dma2 0x2 0xa>,
+ <&dma3 0x3 0xb>;
+ };
+
+ dma0: dma@2c0a0000 {
+ compatible = "arm,pl330";
+ reg = <0 0x2c0a0000 0 0x1000>;
+ interrupts = <0 64 4>,
+ <0 65 4>,
+ <0 66 4>,
+ <0 67 4>,
+ <0 68 4>,
+ <0 69 4>,
+ <0 70 4>,
+ <0 71 4>;
+ #dma-cells = <1>;
+ #stream-id-cells = <2>;
+ clocks = <&oscclk6a>;
+ clock-names = "apb_pclk";
+ };
+
+ dma1: dma@2c0a1000 {
+ compatible = "arm,pl330", "arm,primecell";
+ reg = <0 0x2c0a1000 0 0x1000>;
+ interrupts = <0 96 4>,
+ <0 97 4>,
+ <0 98 4>,
+ <0 99 4>,
+ <0 100 4>,
+ <0 101 4>,
+ <0 102 4>,
+ <0 103 4>;
+ #dma-cells = <1>;
+ #stream-id-cells = <2>;
+ clocks = <&oscclk6a>;
+ clock-names = "apb_pclk";
+ };
+
+ dma2: dma@2c0a2000 {
+ compatible = "arm,pl330", "arm,primecell";
+ reg = <0 0x2c0a2000 0 0x1000>;
+ interrupts = <0 128 4>,
+ <0 129 4>,
+ <0 130 4>,
+ <0 131 4>,
+ <0 132 4>,
+ <0 133 4>,
+ <0 134 4>,
+ <0 135 4>;
+ #dma-cells = <1>;
+ #stream-id-cells = <2>;
+ clocks = <&oscclk6a>;
+ clock-names = "apb_pclk";
+ };
+
+ dma3: dma@2c0a3000 {
+ compatible = "arm,pl330", "arm,primecell";
+ reg = <0 0x2c0a3000 0 0x1000>;
+ interrupts = <0 160 4>,
+ <0 161 4>,
+ <0 162 4>,
+ <0 163 4>,
+ <0 164 4>,
+ <0 165 4>,
+ <0 166 4>,
+ <0 167 4>;
+ #dma-cells = <1>;
+ #stream-id-cells = <2>;
+ clocks = <&oscclk6a>;
+ clock-names = "apb_pclk";
+ };
+
+ oscclk6a: oscclk6a {
+ /* Reference 24MHz clock */
+ compatible = "fixed-clock";
+ #clock-cells = <0>;
+ clock-frequency = <24000000>;
+ clock-output-names = "oscclk6a";
+ };
+
smb {
compatible = "simple-bus";

@@ -244,4 +339,3 @@
/include/ "rtsm_ve-motherboard.dtsi"
};
};
-

--
Shannon

_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm

.

Quoted 6 lines
Quoted 3 lines
Quoted 2 lines

.

.

--
Shannon

Quoted 5 lines
Quoted 2 lines

.

--
Shannon

.

.

#41 lore raw
In reply to a message not in the archive

Hi Alvise,

Thanks for your help.
Now I can see /dev/vfio/vfio and /dev/vfio/0 after executing "echo 2c0a0000.dma > /sys/bus/platform/drivers/vfio-platform/vfio_bind". But when using qemu to start guest, there is an error :

-nographic can not be used with -daemonize
char device redirected to /dev/pts/3 (label pty0-base)
PL330_VFIO: === VFIO device file descriptor 13 === Device has 1 region(s):
PL330_VFIO: - Region 0: size=0x1000 offset=0x2c0a0000 flags=0x7
PL330_VFIO: - IRQ 0: range of 1, flags=0x7

------------[ cut here ]------------
kernel BUG at mm/slub.c:3386!
Internal error: Oops - BUG: 0 [#1] SMP ARM
Modules linked in:
CPU: 2 PID: 1746 Comm: qemu-system-arm Tainted: G        W    3.14.0-rc1+ #19
task: ee665580 ti: ed0b8000 task.ti: ed0b8000
PC is at kfree+0x128/0x134
LR is at vfio_set_trigger.isra.5+0x40/0x104
pc : [<800d9e3c>]    lr : [<80319094>]    psr: 400f0013

sp : ed0b9e64 ip : 000800f3 fp : 000014ac
r10: 00000000 r9 : ed0b8000 r8 : edeb3b80
r7 : 0000000e r6 : 80620584 r5 : 00000000 r4 : eea0b000 r3 : 00000400 r2 : 800f38b0 r1 : eea0ce60 r0 : 00000400
Flags: nZcv IRQs on FIQs on Mode SVC_32 ISA ARM Segment user
Control: 30c53c7d Table: eddac080 DAC: 00000000 Process qemu-system-arm (pid: 1746, stack limit = 0xed0b8240)

Stack: (0xed0b9e64 to 0xed0ba000)

9e60: ee603700 00000000 ee603700 0000000e edeb3b80 00000000 80319094
9e80: ee603700 00003b6e 00000000 edeb3b80 ee603700 00003b6e 0000000d 80319208
9ea0: 0000000e edeb3b80 80492a18 edeb3b80 ee57f780 80319690 00000024 008b2074
9ec0: 008b2074 8031886c 00000001 008b2074 00000018 00000024 00000000 00000000
9ee0: 00000001 00000004 00000001 00000001 ee088e70 008b2060 ee57f780 803155d0
9f00: 803155b0 800ef430 80477a58 00080802 ed0b8000 800e1358 00000020 ed0b9f50
9f20: ed0b9f60 00000003 8061e808 00080802 ed0b9f60 800e14f0 00000000 806522f8
9f40: 80477a58 8061e808 ede4c1c0 8011af18 00000000 00000009 8053bbc0 00000000
9f60: ee683610 00000001 ee57f780 008b2060 00003b6e 0000000d ed0b8000 00000000
9f80: 000014ac 800ef9ac 0000000e 00000001 008b2060 00429eb0 0000000d 00000036
9fa0: 8001c464 8001c2e0 008b2060 00429eb0 0000000d 00003b6e 008b2060 00000001
9fc0: 008b2060 00429eb0 0000000d 00000036 00000000 7ea99040 0089a220 000014ac
9fe0: 003fa474 7ea98fc4 000bc5c8 76ceb2ec 200f0010 0000000d 00000000 8062c770 [<800d9e3c>] (kfree) from [<80319094>] (vfio_set_trigger.isra.5+0x40/0x104) [<80319094>] (vfio_set_trigger.isra.5) from [<80319208>] (vfio_platform_set_irq_trigger+0xb0/0x144) [<80319208>] (vfio_platform_set_irq_trigger) from [<80319690>] (vfio_platform_set_irqs_ioctl+0x44/0x58) [<80319690>] (vfio_platform_set_irqs_ioctl) from [<8031886c>] (vfio_platform_ioctl+0x2e0/0x314) [<8031886c>] (vfio_platform_ioctl) from [<803155d0>] (vfio_device_fops_unl_ioctl+0x20/0x2c) [<803155d0>] (vfio_device_fops_unl_ioctl) from [<800ef430>] (do_vfs_ioctl+0x78/0x5b8) [<800ef430>] (do_vfs_ioctl) from [<800ef9ac>] (SyS_ioctl+0x3c/0x5c)
[<800ef9ac>] (SyS_ioctl) from [<8001c2e0>] (ret_fast_syscall+0x0/0x30)

Code: 15913038 e1a01003 e8bd45f0 eaff28e5 (e7f001f2)

I'm using branch vfio-platform-v4 for kernel and pl330-vfio-dev for qemu. Following is qemu cmdline:

	./qemu-system-arm -enable-kvm -M vexpress-a15 -cpu cortex-a15 \
       	-chardev pty,id=pty0,mux=on -monitor chardev:pty0 \
       	-serial chardev:pty0 -daemonize \
       	-device pl330,vfio_group="/dev/vfio/0",vfio_device="2c0a0000.dma",vfio_container="/dev/vfio/vfio" \
       	-append "earlyprintk ip=dhcp console=ttyAMA0 rootwait" \
       	-kernel zImage -dtb guest-a15.dtb -initrd filesystem.cpio.gz  -nographic

If I change to branch vfio-platform-v6 for kernel, there is "/dev/vfio/vfio". But no "/sys/bus/platform/drivers/vfio-platform/vfio_bind". And if I use above qemu cmdline to start guest, there is also an error :
PL330_VFIO: Group is not viable (not all devices bound for vfio)
qemu-system-arm: Error while probing real device.

qemu-system-arm: Initialization of device pl330 failed

The usage about v4 and v6 has changed ?
And would you tell me when you will update the guideline about vfio-on-arm?

Thanks,
Shannon

Quoted 422 lines · Alvise Rigo

Linaro has a great set of pre-built images here http://releases.linaro.org/.

For example these should work fine:
http://releases.linaro.org/13.12/ubuntu/saucy-images

alvise

Il 20/06/2014 10:23, Shannon Zhao ha scritto:

Quoted 411 lines

Hi Alvise,

I'm so sorry that I can't use qemu-debootstrap because my environment can't access some links using proxy. Are there other ways to get a proper fs?

Thanks
Shannon

Quoted 401 lines · Alvise Rigo

Hi Shannon,

If you follow the steps in the section "Creating the File System" of the guide you should be able to obtain a working fs.

Regards,
alvise

Il 20/06/2014 04:57, Shannon Zhao ha scritto:

Quoted 4 lines

Hi Bharat,

Thanks for your advice .

Quoted 385 lines

I'm using the rootfs getting from the url "http://www.virtualopensystems.com/downloads/guides/kvm_on_arm/fs-alip-armel.cramfs".

Can you give me another link to get a newer rootfs?

Thank you
Shannon

Quoted 376 lines · On 2014/6/19 20:20, [email protected]…
Quoted 29 lines

-----Original Message-----
From: Shannon Zhao [mailto:[email protected]]
Sent: Thursday, June 19, 2014 5:47 PM
To: Bhushan Bharat-R65777; Alvise Rigo; a.motakis-lrHrjnjw1UfHK3s98zE1ajGjJy/[email protected]; iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org; [email protected]
Subject: Re: Vfio-on-arm: the device tree does not contain the DMA device

Quoted 17 lines · On 2014/6/19 20:12, [email protected]…
Quoted 16 lines

-----Original Message-----
From: Shannon Zhao [mailto:[email protected]]
Sent: Thursday, June 19, 2014 5:38 PM
To: Alvise Rigo; Bhushan Bharat-R65777;
a.motakis-lrHrjnjw1UfHK3s98zE1ajGjJy/[email protected]; iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org; [email protected]
Subject: Re: Vfio-on-arm: the device tree does not contain the DMA
device

Hi Alvise,

I run following cmds :

	echo 2c0a0000.dma >

/sys/bus/platform/drivers/vfio-platform/vfio_bind

Quoted 6 lines
Quoted 1 line

run "ls /dev/vfio/" before and after the above command

Hi Bharat,

Nothing appeared either before or after the cmd "echo 2c0a0000.dma > /sys/bus/platform/drivers/vfio-platform/vfio_bind".
There is only /dev/vfio.

I am not sure, but may be you are using pretty old rootfs with broken udev rules. Try to fetch a newer rootfs form somewhere.

Thanks
-Bharat

Quoted 325 lines
Quoted 8 lines

Thanks
-Bharat

Quoted 3 lines
	./qemu-system-arm -enable-kvm -M vexpress-a15 -cpu cortex-a15 \
        	-device

pl330,vfio_group="/dev/vfio/0",vfio_device="2c0a0000.dma",vfio_container="/dev/v

Quoted 6 lines
Quoted 6 lines

fio/vfio" \
-append "earlyprintk ip=dhcp console=ttyAMA0 rootwait" \
-kernel zImage -dtb guest-a15.dtb -initrd filesystem.cpio.gz -
nographic But the error still exists.
qemu-system-arm: -device

pl330,vfio_group=/dev/vfio/0,vfio_device=2c0a0000.dma,vfio_container=/dev/vfio/v

Quoted 102 lines
Quoted 102 lines

fio: Parameter 'driver' expects pluggable device type

Thanks,
Shannon

Quoted 32 lines · Alvise Rigo

Il 19/06/2014 13:11, Shannon Zhao ha scritto:

Quoted 29 lines · Alvise Rigo
Quoted 26 lines

Hi,

Il 19/06/2014 11:38, Shannon Zhao ha scritto:

Quoted 23 lines

Hi Bharat,

Quoted 20 lines · On 2014/6/19 17:19, [email protected]…

-----Original Message-----
From: [email protected] [mailto:kvmarm-
[email protected]] On Behalf Of Shannon Zhao
Sent: Thursday, June 19, 2014 2:45 PM
To: Alvise Rigo; a.motakis-lrHrjnjw1UfHK3s98zE1ajGjJy/[email protected];
iommu-cunTk1MwBs/[email protected] foundation.org; [email protected]
Subject: Re: Vfio-on-arm: the device tree does not contain the
DMA device

Hi Alvise,

Thanks for your reply.
I have apply the patch and could see the DMA devices.
But there is no vosys_dmatest int the directory
"/sys/kernel/debug/" after mounting debugfs.
I found that the branch vfio-platform-v4 doesn't contain the
module vosys_dmatest.
Then I port it from the branch pl330_dmatest and run vosys_dmatest

successfully.

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

echo 1 > /sys/kernel/debug/vosys_dmatest/start

[ 6145.343419] dmaengine: __dma_request_channel: success

(dma0chan0)

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines

[ 6145.344000] vosys dmatest: callback: transfer completed
[ 6145.344147] vosys dmatest: tx completed
[ 6145.344589] vosys dmatest: source and destination match.

Download the qemu and use following command to start a guest:
./qemu-system-arm -enable-kvm -M vexpress-a15 -cpu cortex-a15 \
-device

pl330,vfio_group="/dev/vfio/0",vfio_device="2c0a0000.dma",\

Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 8 lines

vfio_container="/dev/vfio/vfio" \
-append "earlyprintk ip=dhcp console=ttyAMA0 rootwait" \
-kernel zImage -dtb guest.dtb -initrd filesystem.cpio.gz
- nographic An error reported:
qemu-system-arm: -device
pl330,vfio_group=/dev/vfio/0,vfio_device=2c0a0000.dma,:

Why do we need to give "vfio_group=/dev/vfio/0" ?

I refer to the url

"http://www.virtualopensystems.com/en/solutions/guides/vfio-on-arm/".

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines

could not open disk image vfio_container=/dev/vfio:
Could not open 'vfio_container=/dev/vfio': No such file or
directory

Are you unbinding "2c0a0000.dma" from its driver and binding with "vfio-

playform" driver ?

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

How to unbind "2c0a0000.dma" from its driver?
I have tried following cmd to bind "2c0a0000.dma" with "vfio-playform"

driver.

Quoted 13 lines
Quoted 13 lines
Quoted 11 lines
Quoted 3 lines
	echo 2c0a0000.dma >

/sys/bus/platform/drivers/vfio-platform/vfio_bind
But there is no /dev/vfio/0 or /dev/vfio/vfio .

Can you please verify that while configuring the host you enabled:
- Device Drivers -> IOMMU Hardware Support -> ARM Ltd. System MMU (SMMU) Support and
- Device Drivers -> VFIO Non-Privileged userspace driver framework -> VFIO support for platform devices

Hi Alvise,

Quoted 8 lines
Quoted 8 lines

I have checked the two configuration are enabled.

Then try the following cmd, but another error reported:
qemu-system-arm: -device
pl330,vfio_group=/dev/vfio/0,vfio_device=2c0a0000.dma,vfio_container= /dev/vfio/vfio: Parameter 'driver' expects pluggable device type

Do I need to unbind the "2c0a0000.dma" from its driver and binding with

"vfio-platform" driver as Bharat mentioned ?

Quoted 7 lines

Yes, the 2c0a0000.dma should not be bound to any driver at boot, so you just need to:
echo 2c0a0000.dma > /sys/bus/platform/drivers/vfio-platform/vfio_bind to bind the device to the VFIO driver.

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

Regards,
alvise

Quoted 2 lines
Quoted 2 lines

Then try to update the pl330-vfio-dev branch and run QEMU with the

following

Quoted 5 lines
Quoted 5 lines

command:

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

./qemu-system-arm -enable-kvm -M vexpress-a15 -cpu cortex-a15 \
-device

pl330,vfio_group="/dev/vfio/0",vfio_device="2c0a0000.dma",vfio_container="/dev/v

Quoted 195 lines
Quoted 193 lines

fio/vfio" \

Quoted 16 lines
Quoted 16 lines
Quoted 16 lines

-append "earlyprintk ip=dhcp console=ttyAMA0 rootwait" \
-kernel zImage -dtb guest.dtb -initrd filesystem.cpio.gz

that means without breaking the "-device pl330" options.

Regards,
alvise

Quoted 8 lines
Quoted 8 lines

Thanks
-Bharat

Is there something I'm wrong or do I miss some steps?

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines

Thank you,
Shannon

On 2014/6/18 21:24, Alvise Rigo wrote:
Hi Shannon,

Thank you for reporting this issue, the device tree file was indeed

missing.

Quoted 159 lines
Quoted 159 lines
Quoted 159 lines
Quoted 158 lines
Quoted 157 lines

To solve the problem, you can apply the following patch to the
repository you mentioned
(git://github.com/virtualopensystems/arm-dts.git).

For your information, soon we will update the guide with a newer
version of VFIO with this issue fixed.

Thank you,
alvise

---
fast_models/rtsm_ve-cortex_a15x4_a7x4.dts | 100
+++++++++++++++++++++++++++++-
1 file changed, 97 insertions(+), 3 deletions(-)

diff --git a/fast_models/rtsm_ve-cortex_a15x4_a7x4.dts
b/fast_models/rtsm_ve-cortex_a15x4_a7x4.dts
index ef1275b..79c6af2 100644
--- a/fast_models/rtsm_ve-cortex_a15x4_a7x4.dts
+++ b/fast_models/rtsm_ve-cortex_a15x4_a7x4.dts
@@ -142,7 +142,7 @@
cpu7: cpu@7 {
device_type = "cpu";
compatible = "arm,cortex-a7";
- reg = <0x104>;
+ reg = <0x103>;
cluster = <&cluster1>;
core = <&core7>;
};
@@ -159,7 +159,7 @@
#address-cells = <0>;
interrupt-controller;
reg = <0 0x2c001000 0 0x1000>,
- <0 0x2c002000 0 0x2000>,
+ <0 0x2c002000 0 0x1000>,
<0 0x2c004000 0 0x2000>,
<0 0x2c006000 0 0x2000>;
interrupts = <1 9 0xf04>;
@@ -183,6 +183,101 @@
<1 10 0xf08>;
};

+ smmu@2c0b0000 {
+ compatible = "arm,smmu-v2";
+ reg = <0x0 0x2c0b0000 0x0 0x10000>;
+ #global-interrupts = <2>;
+ interrupts = <0 58 4>,
+ <0 59 4>,
+ <0 60 4>,
+ <0 60 4>,
+ <0 60 4>,
+ <0 60 4>,
+ <0 60 4>,
+ <0 60 4>,
+ <0 60 4>;
+ mmu-masters = <&dma0 0x0 0x8>,
+ <&dma1 0x1 0x9>,
+ <&dma2 0x2 0xa>,
+ <&dma3 0x3 0xb>;
+ };
+
+ dma0: dma@2c0a0000 {
+ compatible = "arm,pl330";
+ reg = <0 0x2c0a0000 0 0x1000>;
+ interrupts = <0 64 4>,
+ <0 65 4>,
+ <0 66 4>,
+ <0 67 4>,
+ <0 68 4>,
+ <0 69 4>,
+ <0 70 4>,
+ <0 71 4>;
+ #dma-cells = <1>;
+ #stream-id-cells = <2>;
+ clocks = <&oscclk6a>;
+ clock-names = "apb_pclk";
+ };
+
+ dma1: dma@2c0a1000 {
+ compatible = "arm,pl330", "arm,primecell";
+ reg = <0 0x2c0a1000 0 0x1000>;
+ interrupts = <0 96 4>,
+ <0 97 4>,
+ <0 98 4>,
+ <0 99 4>,
+ <0 100 4>,
+ <0 101 4>,
+ <0 102 4>,
+ <0 103 4>;
+ #dma-cells = <1>;
+ #stream-id-cells = <2>;
+ clocks = <&oscclk6a>;
+ clock-names = "apb_pclk";
+ };
+
+ dma2: dma@2c0a2000 {
+ compatible = "arm,pl330", "arm,primecell";
+ reg = <0 0x2c0a2000 0 0x1000>;
+ interrupts = <0 128 4>,
+ <0 129 4>,
+ <0 130 4>,
+ <0 131 4>,
+ <0 132 4>,
+ <0 133 4>,
+ <0 134 4>,
+ <0 135 4>;
+ #dma-cells = <1>;
+ #stream-id-cells = <2>;
+ clocks = <&oscclk6a>;
+ clock-names = "apb_pclk";
+ };
+
+ dma3: dma@2c0a3000 {
+ compatible = "arm,pl330", "arm,primecell";
+ reg = <0 0x2c0a3000 0 0x1000>;
+ interrupts = <0 160 4>,
+ <0 161 4>,
+ <0 162 4>,
+ <0 163 4>,
+ <0 164 4>,
+ <0 165 4>,
+ <0 166 4>,
+ <0 167 4>;
+ #dma-cells = <1>;
+ #stream-id-cells = <2>;
+ clocks = <&oscclk6a>;
+ clock-names = "apb_pclk";
+ };
+
+ oscclk6a: oscclk6a {
+ /* Reference 24MHz clock */
+ compatible = "fixed-clock";
+ #clock-cells = <0>;
+ clock-frequency = <24000000>;
+ clock-output-names = "oscclk6a";
+ };
+
smb {
compatible = "simple-bus";

@@ -244,4 +339,3 @@
/include/ "rtsm_ve-motherboard.dtsi"
};
};
-

--
Shannon

_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm

.

Quoted 6 lines
Quoted 3 lines
Quoted 2 lines

.

.

--
Shannon

Quoted 5 lines
Quoted 2 lines

.

--
Shannon

.

.

.

Signature · 1 line
Shannon
#43 lore raw
In reply to a message not in the archive
Quoted 74 lines

-----Original Message-----
From: Shannon Zhao [mailto:[email protected]]
Sent: Tuesday, June 24, 2014 12:30 PM
To: Alvise Rigo; Bhushan Bharat-R65777; a.motakis-lrHrjnjw1UfHK3s98zE1ajGjJy/[email protected]; iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org; [email protected]; [email protected]
Subject: Re: Vfio-on-arm: the device tree does not contain the DMA device

Hi Alvise,

Thanks for your help.
Now I can see /dev/vfio/vfio and /dev/vfio/0 after executing "echo 2c0a0000.dma

Quoted 1 line

/sys/bus/platform/drivers/vfio-platform/vfio_bind".

But when using qemu to start guest, there is an error :

-nographic can not be used with -daemonize
char device redirected to /dev/pts/3 (label pty0-base)
PL330_VFIO: === VFIO device file descriptor 13 === Device has 1 region(s):
PL330_VFIO: - Region 0: size=0x1000 offset=0x2c0a0000 flags=0x7
PL330_VFIO: - IRQ 0: range of 1, flags=0x7

------------[ cut here ]------------
kernel BUG at mm/slub.c:3386!
Internal error: Oops - BUG: 0 [#1] SMP ARM
Modules linked in:
CPU: 2 PID: 1746 Comm: qemu-system-arm Tainted: G        W    3.14.0-rc1+ #19
task: ee665580 ti: ed0b8000 task.ti: ed0b8000
PC is at kfree+0x128/0x134
LR is at vfio_set_trigger.isra.5+0x40/0x104
pc : [<800d9e3c>]    lr : [<80319094>]    psr: 400f0013

sp : ed0b9e64 ip : 000800f3 fp : 000014ac
r10: 00000000 r9 : ed0b8000 r8 : edeb3b80
r7 : 0000000e r6 : 80620584 r5 : 00000000 r4 : eea0b000 r3 : 00000400 r2 : 800f38b0 r1 : eea0ce60 r0 : 00000400
Flags: nZcv IRQs on FIQs on Mode SVC_32 ISA ARM Segment user
Control: 30c53c7d Table: eddac080 DAC: 00000000 Process qemu-system-arm (pid: 1746, stack limit = 0xed0b8240)

Stack: (0xed0b9e64 to 0xed0ba000)

9e60: ee603700 00000000 ee603700 0000000e edeb3b80 00000000 80319094
9e80: ee603700 00003b6e 00000000 edeb3b80 ee603700 00003b6e 0000000d 80319208 9ea0: 0000000e edeb3b80 80492a18 edeb3b80 ee57f780 80319690 00000024 008b2074 9ec0: 008b2074 8031886c 00000001 008b2074 00000018 00000024 00000000 00000000 9ee0: 00000001 00000004 00000001 00000001 ee088e70 008b2060 ee57f780 803155d0 9f00: 803155b0 800ef430 80477a58 00080802 ed0b8000 800e1358 00000020 ed0b9f50 9f20: ed0b9f60 00000003 8061e808 00080802 ed0b9f60 800e14f0 00000000 806522f8 9f40: 80477a58 8061e808 ede4c1c0 8011af18 00000000 00000009 8053bbc0 00000000 9f60: ee683610 00000001 ee57f780 008b2060 00003b6e 0000000d ed0b8000 00000000 9f80: 000014ac 800ef9ac 0000000e 00000001 008b2060 00429eb0 0000000d 00000036 9fa0: 8001c464 8001c2e0 008b2060 00429eb0 0000000d 00003b6e 008b2060 00000001 9fc0: 008b2060 00429eb0 0000000d 00000036 00000000 7ea99040 0089a220 000014ac 9fe0: 003fa474 7ea98fc4 000bc5c8 76ceb2ec 200f0010 0000000d 00000000 8062c770 [<800d9e3c>] (kfree) from [<80319094>] (vfio_set_trigger.isra.5+0x40/0x104) [<80319094>] (vfio_set_trigger.isra.5) from [<80319208>] (vfio_platform_set_irq_trigger+0xb0/0x144)
[<80319208>] (vfio_platform_set_irq_trigger) from [<80319690>] (vfio_platform_set_irqs_ioctl+0x44/0x58)
[<80319690>] (vfio_platform_set_irqs_ioctl) from [<8031886c>] (vfio_platform_ioctl+0x2e0/0x314)
[<8031886c>] (vfio_platform_ioctl) from [<803155d0>] (vfio_device_fops_unl_ioctl+0x20/0x2c)
[<803155d0>] (vfio_device_fops_unl_ioctl) from [<800ef430>] (do_vfs_ioctl+0x78/0x5b8)
[<800ef430>] (do_vfs_ioctl) from [<800ef9ac>] (SyS_ioctl+0x3c/0x5c) [<800ef9ac>] (SyS_ioctl) from [<8001c2e0>] (ret_fast_syscall+0x0/0x30)

Code: 15913038 e1a01003 e8bd45f0 eaff28e5 (e7f001f2)

I'm using branch vfio-platform-v4 for kernel and pl330-vfio-dev for qemu. Following is qemu cmdline:

	./qemu-system-arm -enable-kvm -M vexpress-a15 -cpu cortex-a15 \
       	-chardev pty,id=pty0,mux=on -monitor chardev:pty0 \
       	-serial chardev:pty0 -daemonize \
       	-device
Quoted 8 lines

pl330,vfio_group="/dev/vfio/0",vfio_device="2c0a0000.dma",vfio_container="/dev/v fio/vfio" \
-append "earlyprintk ip=dhcp console=ttyAMA0 rootwait" \
-kernel zImage -dtb guest-a15.dtb -initrd filesystem.cpio.gz -
nographic

If I change to branch vfio-platform-v6 for kernel, there is "/dev/vfio/vfio". But no "/sys/bus/platform/drivers/vfio-platform/vfio_bind".

Can you Check if you have driver_override at "/sys/bus/platform/devices/DEVICE-XXX/ ? If yes then you should use driver_override mechanism to bind that device to vfio-platform driver.

Thanks
-Bharat

Quoted 453 lines

And if I use above qemu cmdline to start guest, there is also an error :
PL330_VFIO: Group is not viable (not all devices bound for vfio)
qemu-system-arm: Error while probing real device.

qemu-system-arm: Initialization of device pl330 failed

The usage about v4 and v6 has changed ?
And would you tell me when you will update the guideline about vfio-on-arm?

Thanks,
Shannon

Quoted 11 lines · Alvise Rigo

Linaro has a great set of pre-built images here http://releases.linaro.org/.

For example these should work fine:
http://releases.linaro.org/13.12/ubuntu/saucy-images

alvise

Il 20/06/2014 10:23, Shannon Zhao ha scritto:

Quoted 3 lines

Hi Alvise,

I'm so sorry that I can't use qemu-debootstrap because my environment can't

access some links using proxy.

Quoted 14 lines
Quoted 14 lines

Are there other ways to get a proper fs?

Thanks
Shannon

Quoted 8 lines · Alvise Rigo

Hi Shannon,

If you follow the steps in the section "Creating the File System" of the guide you should be able to obtain a working fs.

Regards,
alvise

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines

Il 20/06/2014 04:57, Shannon Zhao ha scritto:

Quoted 5 lines

Hi Bharat,

Thanks for your advice .

I'm using the rootfs getting from the url

"http://www.virtualopensystems.com/downloads/guides/kvm_on_arm/fs-alip- armel.cramfs".

Quoted 36 lines
Quoted 36 lines
Quoted 36 lines
Quoted 36 lines

Can you give me another link to get a newer rootfs?

Thank you
Shannon

Quoted 29 lines · On 2014/6/19 20:20, [email protected]…
Quoted 29 lines

-----Original Message-----
From: Shannon Zhao [mailto:[email protected]]
Sent: Thursday, June 19, 2014 5:47 PM
To: Bhushan Bharat-R65777; Alvise Rigo; a.motakis-lrHrjnjw1UfHK3s98zE1ajGjJy/[email protected]; iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org; [email protected]
Subject: Re: Vfio-on-arm: the device tree does not contain the DMA device

Quoted 17 lines · On 2014/6/19 20:12, [email protected]…
Quoted 16 lines

-----Original Message-----
From: Shannon Zhao [mailto:[email protected]]
Sent: Thursday, June 19, 2014 5:38 PM
To: Alvise Rigo; Bhushan Bharat-R65777;
a.motakis-lrHrjnjw1UfHK3s98zE1ajGjJy/[email protected]; iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org; [email protected]
Subject: Re: Vfio-on-arm: the device tree does not contain the DMA
device

Hi Alvise,

I run following cmds :

	echo 2c0a0000.dma >

/sys/bus/platform/drivers/vfio-platform/vfio_bind

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 6 lines
Quoted 1 line

run "ls /dev/vfio/" before and after the above command

Hi Bharat,

Nothing appeared either before or after the cmd "echo 2c0a0000.dma > /sys/bus/platform/drivers/vfio-platform/vfio_bind".
There is only /dev/vfio.

I am not sure, but may be you are using pretty old rootfs with broken udev

rules. Try to fetch a newer rootfs form somewhere.

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines

Thanks
-Bharat

Quoted 1 line
Quoted 1 line
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 7 lines

Thanks
-Bharat

Quoted 3 lines
	./qemu-system-arm -enable-kvm -M vexpress-a15 -cpu cortex-a15 \
        	-device

pl330,vfio_group="/dev/vfio/0",vfio_device="2c0a0000.dma",vfio_container="/dev/v

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

fio/vfio" \
-append "earlyprintk ip=dhcp console=ttyAMA0 rootwait" \
-kernel zImage -dtb guest-a15.dtb -initrd filesystem.cpio.gz

-

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 3 lines
Quoted 3 lines

nographic But the error still exists.
qemu-system-arm: -device

pl330,vfio_group=/dev/vfio/0,vfio_device=2c0a0000.dma,vfio_container=/dev/vfio/v

Quoted 31 lines
Quoted 31 lines
Quoted 31 lines
Quoted 31 lines
Quoted 31 lines
Quoted 31 lines
Quoted 31 lines
Quoted 31 lines

fio: Parameter 'driver' expects pluggable device type

Thanks,
Shannon

Quoted 25 lines · Alvise Rigo

Il 19/06/2014 13:11, Shannon Zhao ha scritto:

Quoted 22 lines · Alvise Rigo
Quoted 19 lines

Hi,

Il 19/06/2014 11:38, Shannon Zhao ha scritto:

Quoted 16 lines

Hi Bharat,

On 2014/6/19 17:19, [email protected] wrote:

-----Original Message-----
From: [email protected] [mailto:kvmarm-
[email protected]] On Behalf Of Shannon Zhao
Sent: Thursday, June 19, 2014 2:45 PM
To: Alvise Rigo; a.motakis-lrHrjnjw1UfHK3s98zE1ajGjJy/[email protected];
iommu-cunTk1MwBs/[email protected] foundation.org; [email protected]
Subject: Re: Vfio-on-arm: the device tree does not contain the
DMA device

Hi Alvise,

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines

Thanks for your reply.
I have apply the patch and could see the DMA devices.
But there is no vosys_dmatest int the directory
"/sys/kernel/debug/" after mounting debugfs.
I found that the branch vfio-platform-v4 doesn't contain the
module vosys_dmatest.
Then I port it from the branch pl330_dmatest and run

vosys_dmatest

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

successfully.

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

echo 1 > /sys/kernel/debug/vosys_dmatest/start

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

[ 6145.343419] dmaengine: __dma_request_channel: success

(dma0chan0)

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines

[ 6145.344000] vosys dmatest: callback: transfer completed
[ 6145.344147] vosys dmatest: tx completed
[ 6145.344589] vosys dmatest: source and destination match.

Download the qemu and use following command to start a guest:
./qemu-system-arm -enable-kvm -M vexpress-a15 -cpu cortex-

a15 \

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

-device

pl330,vfio_group="/dev/vfio/0",vfio_device="2c0a0000.dma",\

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

vfio_container="/dev/vfio/vfio" \
-append "earlyprintk ip=dhcp console=ttyAMA0 rootwait"

\

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

-kernel zImage -dtb guest.dtb -initrd

filesystem.cpio.gz

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines

- nographic An error reported:
qemu-system-arm: -device
pl330,vfio_group=/dev/vfio/0,vfio_device=2c0a0000.dma,:

Why do we need to give "vfio_group=/dev/vfio/0" ?
I refer to the url

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines

"http://www.virtualopensystems.com/en/solutions/guides/vfio-on-arm/".

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines

could not open disk image vfio_container=/dev/vfio:
Could not open 'vfio_container=/dev/vfio': No such file or
directory

Are you unbinding "2c0a0000.dma" from its driver and binding with

"vfio-

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

playform" driver ?

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

How to unbind "2c0a0000.dma" from its driver?
I have tried following cmd to bind "2c0a0000.dma" with "vfio-

playform"

Quoted 14 lines
Quoted 14 lines
Quoted 14 lines
Quoted 14 lines
Quoted 14 lines
Quoted 14 lines
Quoted 14 lines
Quoted 14 lines

driver.

Quoted 13 lines
Quoted 13 lines
Quoted 11 lines
Quoted 3 lines

echo 2c0a0000.dma >
/sys/bus/platform/drivers/vfio-platform/vfio_bind
But there is no /dev/vfio/0 or /dev/vfio/vfio .

Can you please verify that while configuring the host you enabled:
- Device Drivers -> IOMMU Hardware Support -> ARM Ltd. System MMU (SMMU) Support and
- Device Drivers -> VFIO Non-Privileged userspace driver framework -> VFIO support for platform devices

Hi Alvise,

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines

I have checked the two configuration are enabled.

Then try the following cmd, but another error reported:
qemu-system-arm: -device
pl330,vfio_group=/dev/vfio/0,vfio_device=2c0a0000.dma,vfio_container= /dev/vfio/vfio: Parameter 'driver' expects pluggable device type

Do I need to unbind the "2c0a0000.dma" from its driver and binding

with

Quoted 12 lines
Quoted 12 lines
Quoted 12 lines
Quoted 12 lines
Quoted 12 lines
Quoted 12 lines
Quoted 12 lines
Quoted 12 lines

"vfio-platform" driver as Bharat mentioned ?

Quoted 11 lines

Yes, the 2c0a0000.dma should not be bound to any driver at boot, so you just need to:
echo 2c0a0000.dma > /sys/bus/platform/drivers/vfio-platform/vfio_bind to bind the device to the VFIO driver.

Regards,
alvise

Quoted 1 line
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

Then try to update the pl330-vfio-dev branch and run QEMU with the

following

Quoted 5 lines
Quoted 5 lines

command:

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

./qemu-system-arm -enable-kvm -M vexpress-a15 -cpu cortex-a15 \
-device

pl330,vfio_group="/dev/vfio/0",vfio_device="2c0a0000.dma",vfio_container="/dev/v

Quoted 17 lines
Quoted 17 lines
Quoted 17 lines
Quoted 17 lines
Quoted 17 lines
Quoted 17 lines
Quoted 17 lines
Quoted 17 lines

fio/vfio" \

Quoted 16 lines
Quoted 16 lines
Quoted 16 lines

-append "earlyprintk ip=dhcp console=ttyAMA0 rootwait" \
-kernel zImage -dtb guest.dtb -initrd filesystem.cpio.gz

that means without breaking the "-device pl330" options.

Regards,
alvise

Quoted 8 lines

Thanks
-Bharat

Is there something I'm wrong or do I miss some steps?

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines

Thank you,
Shannon

On 2014/6/18 21:24, Alvise Rigo wrote:
Hi Shannon,

Thank you for reporting this issue, the device tree file was

indeed

Quoted 182 lines
Quoted 182 lines
Quoted 182 lines
Quoted 181 lines
Quoted 180 lines
Quoted 171 lines
Quoted 171 lines
Quoted 169 lines

missing.

Quoted 159 lines
Quoted 159 lines
Quoted 159 lines
Quoted 158 lines

To solve the problem, you can apply the following patch to the
repository you mentioned
(git://github.com/virtualopensystems/arm-dts.git).

For your information, soon we will update the guide with a newer
version of VFIO with this issue fixed.

Thank you,
alvise

---
fast_models/rtsm_ve-cortex_a15x4_a7x4.dts | 100
+++++++++++++++++++++++++++++-
1 file changed, 97 insertions(+), 3 deletions(-)

diff --git a/fast_models/rtsm_ve-cortex_a15x4_a7x4.dts
b/fast_models/rtsm_ve-cortex_a15x4_a7x4.dts
index ef1275b..79c6af2 100644
--- a/fast_models/rtsm_ve-cortex_a15x4_a7x4.dts
+++ b/fast_models/rtsm_ve-cortex_a15x4_a7x4.dts
@@ -142,7 +142,7 @@
cpu7: cpu@7 {
device_type = "cpu";
compatible = "arm,cortex-a7";
- reg = <0x104>;
+ reg = <0x103>;
cluster = <&cluster1>;
core = <&core7>;
};
@@ -159,7 +159,7 @@
#address-cells = <0>;
interrupt-controller;
reg = <0 0x2c001000 0 0x1000>,
- <0 0x2c002000 0 0x2000>,
+ <0 0x2c002000 0 0x1000>,
<0 0x2c004000 0 0x2000>,
<0 0x2c006000 0 0x2000>;
interrupts = <1 9 0xf04>;
@@ -183,6 +183,101 @@
<1 10 0xf08>;
};

+ smmu@2c0b0000 {
+ compatible = "arm,smmu-v2";
+ reg = <0x0 0x2c0b0000 0x0 0x10000>;
+ #global-interrupts = <2>;
+ interrupts = <0 58 4>,
+ <0 59 4>,
+ <0 60 4>,
+ <0 60 4>,
+ <0 60 4>,
+ <0 60 4>,
+ <0 60 4>,
+ <0 60 4>,
+ <0 60 4>;
+ mmu-masters = <&dma0 0x0 0x8>,
+ <&dma1 0x1 0x9>,
+ <&dma2 0x2 0xa>,
+ <&dma3 0x3 0xb>;
+ };
+
+ dma0: dma@2c0a0000 {
+ compatible = "arm,pl330";
+ reg = <0 0x2c0a0000 0 0x1000>;
+ interrupts = <0 64 4>,
+ <0 65 4>,
+ <0 66 4>,
+ <0 67 4>,
+ <0 68 4>,
+ <0 69 4>,
+ <0 70 4>,
+ <0 71 4>;
+ #dma-cells = <1>;
+ #stream-id-cells = <2>;
+ clocks = <&oscclk6a>;
+ clock-names = "apb_pclk";
+ };
+
+ dma1: dma@2c0a1000 {
+ compatible = "arm,pl330", "arm,primecell";
+ reg = <0 0x2c0a1000 0 0x1000>;
+ interrupts = <0 96 4>,
+ <0 97 4>,
+ <0 98 4>,
+ <0 99 4>,
+ <0 100 4>,
+ <0 101 4>,
+ <0 102 4>,
+ <0 103 4>;
+ #dma-cells = <1>;
+ #stream-id-cells = <2>;
+ clocks = <&oscclk6a>;
+ clock-names = "apb_pclk";
+ };
+
+ dma2: dma@2c0a2000 {
+ compatible = "arm,pl330", "arm,primecell";
+ reg = <0 0x2c0a2000 0 0x1000>;
+ interrupts = <0 128 4>,
+ <0 129 4>,
+ <0 130 4>,
+ <0 131 4>,
+ <0 132 4>,
+ <0 133 4>,
+ <0 134 4>,
+ <0 135 4>;
+ #dma-cells = <1>;
+ #stream-id-cells = <2>;
+ clocks = <&oscclk6a>;
+ clock-names = "apb_pclk";
+ };
+
+ dma3: dma@2c0a3000 {
+ compatible = "arm,pl330", "arm,primecell";
+ reg = <0 0x2c0a3000 0 0x1000>;
+ interrupts = <0 160 4>,
+ <0 161 4>,
+ <0 162 4>,
+ <0 163 4>,
+ <0 164 4>,
+ <0 165 4>,
+ <0 166 4>,
+ <0 167 4>;
+ #dma-cells = <1>;
+ #stream-id-cells = <2>;
+ clocks = <&oscclk6a>;
+ clock-names = "apb_pclk";
+ };
+
+ oscclk6a: oscclk6a {
+ /* Reference 24MHz clock */
+ compatible = "fixed-clock";
+ #clock-cells = <0>;
+ clock-frequency = <24000000>;
+ clock-output-names = "oscclk6a";
+ };
+
smb {
compatible = "simple-bus";

@@ -244,4 +339,3 @@
/include/ "rtsm_ve-motherboard.dtsi"
};
};
-

--
Shannon

_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm

.

Quoted 6 lines
Quoted 3 lines
Quoted 2 lines

.

.

--
Shannon

Quoted 5 lines
Quoted 2 lines

.

--
Shannon

.

Quoted 6 lines
Quoted 3 lines
Quoted 2 lines

.

.

--
Shannon

#45 lore raw
In reply to a message not in the archive

Hi Bharat,

Thanks for your help.

I executed the following cmd:
echo vfio-platform > /sys/bus/platform/devices/2c0a0000.dma/driver_override
But there is no /dev/vfio/0 and error reported starting guest

-nographic can not be used with -daemonize
char device redirected to /dev/pts/3 (label pty0-base)
PL330_VFIO: Group is not viable (not all devices bound for vfio)
qemu-system-arm: Error while probing real device.

qemu-system-arm: Initialization of device pl330 failed
Is there something wrong?

Thank you very much.
Shannon

Quoted 543 lines · On 2014/6/24 15:05, [email protected]…
Quoted 74 lines

-----Original Message-----
From: Shannon Zhao [mailto:[email protected]]
Sent: Tuesday, June 24, 2014 12:30 PM
To: Alvise Rigo; Bhushan Bharat-R65777; a.motakis-lrHrjnjw1UfHK3s98zE1ajGjJy/[email protected]; iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org; [email protected]; [email protected]
Subject: Re: Vfio-on-arm: the device tree does not contain the DMA device

Hi Alvise,

Thanks for your help.
Now I can see /dev/vfio/vfio and /dev/vfio/0 after executing "echo 2c0a0000.dma

Quoted 1 line

/sys/bus/platform/drivers/vfio-platform/vfio_bind".

But when using qemu to start guest, there is an error :

-nographic can not be used with -daemonize
char device redirected to /dev/pts/3 (label pty0-base)
PL330_VFIO: === VFIO device file descriptor 13 === Device has 1 region(s):
PL330_VFIO: - Region 0: size=0x1000 offset=0x2c0a0000 flags=0x7
PL330_VFIO: - IRQ 0: range of 1, flags=0x7

------------[ cut here ]------------
kernel BUG at mm/slub.c:3386!
Internal error: Oops - BUG: 0 [#1] SMP ARM
Modules linked in:
CPU: 2 PID: 1746 Comm: qemu-system-arm Tainted: G        W    3.14.0-rc1+ #19
task: ee665580 ti: ed0b8000 task.ti: ed0b8000
PC is at kfree+0x128/0x134
LR is at vfio_set_trigger.isra.5+0x40/0x104
pc : [<800d9e3c>]    lr : [<80319094>]    psr: 400f0013

sp : ed0b9e64 ip : 000800f3 fp : 000014ac
r10: 00000000 r9 : ed0b8000 r8 : edeb3b80
r7 : 0000000e r6 : 80620584 r5 : 00000000 r4 : eea0b000 r3 : 00000400 r2 : 800f38b0 r1 : eea0ce60 r0 : 00000400
Flags: nZcv IRQs on FIQs on Mode SVC_32 ISA ARM Segment user
Control: 30c53c7d Table: eddac080 DAC: 00000000 Process qemu-system-arm (pid: 1746, stack limit = 0xed0b8240)

Stack: (0xed0b9e64 to 0xed0ba000)

9e60: ee603700 00000000 ee603700 0000000e edeb3b80 00000000 80319094
9e80: ee603700 00003b6e 00000000 edeb3b80 ee603700 00003b6e 0000000d 80319208 9ea0: 0000000e edeb3b80 80492a18 edeb3b80 ee57f780 80319690 00000024 008b2074 9ec0: 008b2074 8031886c 00000001 008b2074 00000018 00000024 00000000 00000000 9ee0: 00000001 00000004 00000001 00000001 ee088e70 008b2060 ee57f780 803155d0 9f00: 803155b0 800ef430 80477a58 00080802 ed0b8000 800e1358 00000020 ed0b9f50 9f20: ed0b9f60 00000003 8061e808 00080802 ed0b9f60 800e14f0 00000000 806522f8 9f40: 80477a58 8061e808 ede4c1c0 8011af18 00000000 00000009 8053bbc0 00000000 9f60: ee683610 00000001 ee57f780 008b2060 00003b6e 0000000d ed0b8000 00000000 9f80: 000014ac 800ef9ac 0000000e 00000001 008b2060 00429eb0 0000000d 00000036 9fa0: 8001c464 8001c2e0 008b2060 00429eb0 0000000d 00003b6e 008b2060 00000001 9fc0: 008b2060 00429eb0 0000000d 00000036 00000000 7ea99040 0089a220 000014ac 9fe0: 003fa474 7ea98fc4 000bc5c8 76ceb2ec 200f0010 0000000d 00000000 8062c770 [<800d9e3c>] (kfree) from [<80319094>] (vfio_set_trigger.isra.5+0x40/0x104) [<80319094>] (vfio_set_trigger.isra.5) from [<80319208>] (vfio_platform_set_irq_trigger+0xb0/0x144)
[<80319208>] (vfio_platform_set_irq_trigger) from [<80319690>] (vfio_platform_set_irqs_ioctl+0x44/0x58)
[<80319690>] (vfio_platform_set_irqs_ioctl) from [<8031886c>] (vfio_platform_ioctl+0x2e0/0x314)
[<8031886c>] (vfio_platform_ioctl) from [<803155d0>] (vfio_device_fops_unl_ioctl+0x20/0x2c)
[<803155d0>] (vfio_device_fops_unl_ioctl) from [<800ef430>] (do_vfs_ioctl+0x78/0x5b8)
[<800ef430>] (do_vfs_ioctl) from [<800ef9ac>] (SyS_ioctl+0x3c/0x5c) [<800ef9ac>] (SyS_ioctl) from [<8001c2e0>] (ret_fast_syscall+0x0/0x30)

Code: 15913038 e1a01003 e8bd45f0 eaff28e5 (e7f001f2)

I'm using branch vfio-platform-v4 for kernel and pl330-vfio-dev for qemu. Following is qemu cmdline:

	./qemu-system-arm -enable-kvm -M vexpress-a15 -cpu cortex-a15 \
       	-chardev pty,id=pty0,mux=on -monitor chardev:pty0 \
       	-serial chardev:pty0 -daemonize \
       	-device
Quoted 8 lines

pl330,vfio_group="/dev/vfio/0",vfio_device="2c0a0000.dma",vfio_container="/dev/v fio/vfio" \
-append "earlyprintk ip=dhcp console=ttyAMA0 rootwait" \
-kernel zImage -dtb guest-a15.dtb -initrd filesystem.cpio.gz -
nographic

If I change to branch vfio-platform-v6 for kernel, there is "/dev/vfio/vfio". But no "/sys/bus/platform/drivers/vfio-platform/vfio_bind".

Can you Check if you have driver_override at "/sys/bus/platform/devices/DEVICE-XXX/ ? If yes then you should use driver_override mechanism to bind that device to vfio-platform driver.

Thanks
-Bharat

Quoted 453 lines

And if I use above qemu cmdline to start guest, there is also an error :
PL330_VFIO: Group is not viable (not all devices bound for vfio)
qemu-system-arm: Error while probing real device.

qemu-system-arm: Initialization of device pl330 failed

The usage about v4 and v6 has changed ?
And would you tell me when you will update the guideline about vfio-on-arm?

Thanks,
Shannon

Quoted 11 lines · Alvise Rigo

Linaro has a great set of pre-built images here http://releases.linaro.org/.

For example these should work fine:
http://releases.linaro.org/13.12/ubuntu/saucy-images

alvise

Il 20/06/2014 10:23, Shannon Zhao ha scritto:

Quoted 3 lines

Hi Alvise,

I'm so sorry that I can't use qemu-debootstrap because my environment can't

access some links using proxy.

Quoted 14 lines
Quoted 14 lines

Are there other ways to get a proper fs?

Thanks
Shannon

Quoted 8 lines · Alvise Rigo

Hi Shannon,

If you follow the steps in the section "Creating the File System" of the guide you should be able to obtain a working fs.

Regards,
alvise

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines

Il 20/06/2014 04:57, Shannon Zhao ha scritto:

Quoted 5 lines

Hi Bharat,

Thanks for your advice .

I'm using the rootfs getting from the url

"http://www.virtualopensystems.com/downloads/guides/kvm_on_arm/fs-alip- armel.cramfs".

Quoted 36 lines
Quoted 36 lines
Quoted 36 lines
Quoted 36 lines

Can you give me another link to get a newer rootfs?

Thank you
Shannon

Quoted 29 lines · On 2014/6/19 20:20, [email protected]…
Quoted 29 lines

-----Original Message-----
From: Shannon Zhao [mailto:[email protected]]
Sent: Thursday, June 19, 2014 5:47 PM
To: Bhushan Bharat-R65777; Alvise Rigo; a.motakis-lrHrjnjw1UfHK3s98zE1ajGjJy/[email protected]; iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org; [email protected]
Subject: Re: Vfio-on-arm: the device tree does not contain the DMA device

Quoted 17 lines · On 2014/6/19 20:12, [email protected]…
Quoted 16 lines

-----Original Message-----
From: Shannon Zhao [mailto:[email protected]]
Sent: Thursday, June 19, 2014 5:38 PM
To: Alvise Rigo; Bhushan Bharat-R65777;
a.motakis-lrHrjnjw1UfHK3s98zE1ajGjJy/[email protected]; iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org; [email protected]
Subject: Re: Vfio-on-arm: the device tree does not contain the DMA
device

Hi Alvise,

I run following cmds :

	echo 2c0a0000.dma >

/sys/bus/platform/drivers/vfio-platform/vfio_bind

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 6 lines
Quoted 1 line

run "ls /dev/vfio/" before and after the above command

Hi Bharat,

Nothing appeared either before or after the cmd "echo 2c0a0000.dma > /sys/bus/platform/drivers/vfio-platform/vfio_bind".
There is only /dev/vfio.

I am not sure, but may be you are using pretty old rootfs with broken udev

rules. Try to fetch a newer rootfs form somewhere.

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines

Thanks
-Bharat

Quoted 1 line
Quoted 1 line
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 7 lines

Thanks
-Bharat

Quoted 3 lines
	./qemu-system-arm -enable-kvm -M vexpress-a15 -cpu cortex-a15 \
        	-device

pl330,vfio_group="/dev/vfio/0",vfio_device="2c0a0000.dma",vfio_container="/dev/v

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

fio/vfio" \
-append "earlyprintk ip=dhcp console=ttyAMA0 rootwait" \
-kernel zImage -dtb guest-a15.dtb -initrd filesystem.cpio.gz

-

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 3 lines
Quoted 3 lines

nographic But the error still exists.
qemu-system-arm: -device

pl330,vfio_group=/dev/vfio/0,vfio_device=2c0a0000.dma,vfio_container=/dev/vfio/v

Quoted 31 lines
Quoted 31 lines
Quoted 31 lines
Quoted 31 lines
Quoted 31 lines
Quoted 31 lines
Quoted 31 lines
Quoted 31 lines

fio: Parameter 'driver' expects pluggable device type

Thanks,
Shannon

Quoted 25 lines · Alvise Rigo

Il 19/06/2014 13:11, Shannon Zhao ha scritto:

Quoted 22 lines · Alvise Rigo
Quoted 19 lines

Hi,

Il 19/06/2014 11:38, Shannon Zhao ha scritto:
Hi Bharat,

On 2014/6/19 17:19, [email protected] wrote:

-----Original Message-----
From: [email protected] [mailto:kvmarm-
[email protected]] On Behalf Of Shannon Zhao
Sent: Thursday, June 19, 2014 2:45 PM
To: Alvise Rigo; a.motakis-lrHrjnjw1UfHK3s98zE1ajGjJy/[email protected];
iommu-cunTk1MwBs/[email protected] foundation.org; [email protected]
Subject: Re: Vfio-on-arm: the device tree does not contain the
DMA device

Hi Alvise,

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines

Thanks for your reply.
I have apply the patch and could see the DMA devices.
But there is no vosys_dmatest int the directory
"/sys/kernel/debug/" after mounting debugfs.
I found that the branch vfio-platform-v4 doesn't contain the
module vosys_dmatest.
Then I port it from the branch pl330_dmatest and run

vosys_dmatest

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

successfully.

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

echo 1 > /sys/kernel/debug/vosys_dmatest/start

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line

[ 6145.343419] dmaengine: __dma_request_channel: success

(dma0chan0)

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines

[ 6145.344000] vosys dmatest: callback: transfer completed
[ 6145.344147] vosys dmatest: tx completed
[ 6145.344589] vosys dmatest: source and destination match.

Download the qemu and use following command to start a guest:
./qemu-system-arm -enable-kvm -M vexpress-a15 -cpu cortex-

a15 \

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line

-device

pl330,vfio_group="/dev/vfio/0",vfio_device="2c0a0000.dma",\

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

vfio_container="/dev/vfio/vfio" \
-append "earlyprintk ip=dhcp console=ttyAMA0 rootwait"

\

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

-kernel zImage -dtb guest.dtb -initrd

filesystem.cpio.gz

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines

- nographic An error reported:
qemu-system-arm: -device
pl330,vfio_group=/dev/vfio/0,vfio_device=2c0a0000.dma,:

Why do we need to give "vfio_group=/dev/vfio/0" ?
I refer to the url

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines

"http://www.virtualopensystems.com/en/solutions/guides/vfio-on-arm/".

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines

could not open disk image vfio_container=/dev/vfio:
Could not open 'vfio_container=/dev/vfio': No such file or
directory

Are you unbinding "2c0a0000.dma" from its driver and binding with

"vfio-

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

playform" driver ?

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

How to unbind "2c0a0000.dma" from its driver?
I have tried following cmd to bind "2c0a0000.dma" with "vfio-

playform"

Quoted 14 lines
Quoted 14 lines
Quoted 14 lines
Quoted 14 lines
Quoted 14 lines
Quoted 14 lines
Quoted 14 lines
Quoted 14 lines

driver.

Quoted 13 lines
Quoted 13 lines
Quoted 11 lines

echo 2c0a0000.dma >
/sys/bus/platform/drivers/vfio-platform/vfio_bind
But there is no /dev/vfio/0 or /dev/vfio/vfio .

Can you please verify that while configuring the host you enabled:
- Device Drivers -> IOMMU Hardware Support -> ARM Ltd. System MMU
(SMMU) Support and
- Device Drivers -> VFIO Non-Privileged userspace driver framework
-> VFIO support for platform devices

Hi Alvise,

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines

I have checked the two configuration are enabled.

Then try the following cmd, but another error reported:
qemu-system-arm: -device
pl330,vfio_group=/dev/vfio/0,vfio_device=2c0a0000.dma,vfio_container= /dev/vfio/vfio: Parameter 'driver' expects pluggable device type

Do I need to unbind the "2c0a0000.dma" from its driver and binding

with

Quoted 12 lines
Quoted 12 lines
Quoted 12 lines
Quoted 12 lines
Quoted 12 lines
Quoted 12 lines
Quoted 12 lines
Quoted 12 lines

"vfio-platform" driver as Bharat mentioned ?

Quoted 11 lines

Yes, the 2c0a0000.dma should not be bound to any driver at boot, so you just need to:
echo 2c0a0000.dma > /sys/bus/platform/drivers/vfio-platform/vfio_bind to bind the device to the VFIO driver.

Regards,
alvise

Quoted 1 line
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

Then try to update the pl330-vfio-dev branch and run QEMU with the

following

Quoted 5 lines
Quoted 5 lines

command:

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

./qemu-system-arm -enable-kvm -M vexpress-a15 -cpu cortex-a15 \
-device

pl330,vfio_group="/dev/vfio/0",vfio_device="2c0a0000.dma",vfio_container="/dev/v

Quoted 17 lines
Quoted 17 lines
Quoted 17 lines
Quoted 17 lines
Quoted 17 lines
Quoted 17 lines
Quoted 17 lines
Quoted 17 lines

fio/vfio" \

Quoted 16 lines
Quoted 16 lines
Quoted 16 lines

-append "earlyprintk ip=dhcp console=ttyAMA0 rootwait" \
-kernel zImage -dtb guest.dtb -initrd filesystem.cpio.gz

that means without breaking the "-device pl330" options.

Regards,
alvise

Thanks
-Bharat

Is there something I'm wrong or do I miss some steps?

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines

Thank you,
Shannon

On 2014/6/18 21:24, Alvise Rigo wrote:
Hi Shannon,

Thank you for reporting this issue, the device tree file was

indeed

Quoted 182 lines
Quoted 182 lines
Quoted 182 lines
Quoted 181 lines
Quoted 180 lines
Quoted 171 lines
Quoted 171 lines
Quoted 169 lines

missing.

Quoted 159 lines
Quoted 159 lines
Quoted 159 lines

To solve the problem, you can apply the following patch to the
repository you mentioned
(git://github.com/virtualopensystems/arm-dts.git).

For your information, soon we will update the guide with a newer
version of VFIO with this issue fixed.

Thank you,
alvise

---
fast_models/rtsm_ve-cortex_a15x4_a7x4.dts | 100
+++++++++++++++++++++++++++++-
1 file changed, 97 insertions(+), 3 deletions(-)

diff --git a/fast_models/rtsm_ve-cortex_a15x4_a7x4.dts
b/fast_models/rtsm_ve-cortex_a15x4_a7x4.dts
index ef1275b..79c6af2 100644
--- a/fast_models/rtsm_ve-cortex_a15x4_a7x4.dts
+++ b/fast_models/rtsm_ve-cortex_a15x4_a7x4.dts
@@ -142,7 +142,7 @@
cpu7: cpu@7 {
device_type = "cpu";
compatible = "arm,cortex-a7";
- reg = <0x104>;
+ reg = <0x103>;
cluster = <&cluster1>;
core = <&core7>;
};
@@ -159,7 +159,7 @@
#address-cells = <0>;
interrupt-controller;
reg = <0 0x2c001000 0 0x1000>,
- <0 0x2c002000 0 0x2000>,
+ <0 0x2c002000 0 0x1000>,
<0 0x2c004000 0 0x2000>,
<0 0x2c006000 0 0x2000>;
interrupts = <1 9 0xf04>;
@@ -183,6 +183,101 @@
<1 10 0xf08>;
};

+ smmu@2c0b0000 {
+ compatible = "arm,smmu-v2";
+ reg = <0x0 0x2c0b0000 0x0 0x10000>;
+ #global-interrupts = <2>;
+ interrupts = <0 58 4>,
+ <0 59 4>,
+ <0 60 4>,
+ <0 60 4>,
+ <0 60 4>,
+ <0 60 4>,
+ <0 60 4>,
+ <0 60 4>,
+ <0 60 4>;
+ mmu-masters = <&dma0 0x0 0x8>,
+ <&dma1 0x1 0x9>,
+ <&dma2 0x2 0xa>,
+ <&dma3 0x3 0xb>;
+ };
+
+ dma0: dma@2c0a0000 {
+ compatible = "arm,pl330";
+ reg = <0 0x2c0a0000 0 0x1000>;
+ interrupts = <0 64 4>,
+ <0 65 4>,
+ <0 66 4>,
+ <0 67 4>,
+ <0 68 4>,
+ <0 69 4>,
+ <0 70 4>,
+ <0 71 4>;
+ #dma-cells = <1>;
+ #stream-id-cells = <2>;
+ clocks = <&oscclk6a>;
+ clock-names = "apb_pclk";
+ };
+
+ dma1: dma@2c0a1000 {
+ compatible = "arm,pl330", "arm,primecell";
+ reg = <0 0x2c0a1000 0 0x1000>;
+ interrupts = <0 96 4>,
+ <0 97 4>,
+ <0 98 4>,
+ <0 99 4>,
+ <0 100 4>,
+ <0 101 4>,
+ <0 102 4>,
+ <0 103 4>;
+ #dma-cells = <1>;
+ #stream-id-cells = <2>;
+ clocks = <&oscclk6a>;
+ clock-names = "apb_pclk";
+ };
+
+ dma2: dma@2c0a2000 {
+ compatible = "arm,pl330", "arm,primecell";
+ reg = <0 0x2c0a2000 0 0x1000>;
+ interrupts = <0 128 4>,
+ <0 129 4>,
+ <0 130 4>,
+ <0 131 4>,
+ <0 132 4>,
+ <0 133 4>,
+ <0 134 4>,
+ <0 135 4>;
+ #dma-cells = <1>;
+ #stream-id-cells = <2>;
+ clocks = <&oscclk6a>;
+ clock-names = "apb_pclk";
+ };
+
+ dma3: dma@2c0a3000 {
+ compatible = "arm,pl330", "arm,primecell";
+ reg = <0 0x2c0a3000 0 0x1000>;
+ interrupts = <0 160 4>,
+ <0 161 4>,
+ <0 162 4>,
+ <0 163 4>,
+ <0 164 4>,
+ <0 165 4>,
+ <0 166 4>,
+ <0 167 4>;
+ #dma-cells = <1>;
+ #stream-id-cells = <2>;
+ clocks = <&oscclk6a>;
+ clock-names = "apb_pclk";
+ };
+
+ oscclk6a: oscclk6a {
+ /* Reference 24MHz clock */
+ compatible = "fixed-clock";
+ #clock-cells = <0>;
+ clock-frequency = <24000000>;
+ clock-output-names = "oscclk6a";
+ };
+
smb {
compatible = "simple-bus";

@@ -244,4 +339,3 @@
/include/ "rtsm_ve-motherboard.dtsi"
};
};
-

--
Shannon

_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm

.

Quoted 6 lines
Quoted 3 lines
Quoted 2 lines

.

.

--
Shannon

Quoted 5 lines
Quoted 2 lines

.

--
Shannon

.

Quoted 6 lines
Quoted 3 lines
Quoted 2 lines

.

.

--
Shannon

Signature · 1 line
Shannon
#47 lore raw
In reply to a message not in the archive
Quoted 10 lines

-----Original Message-----
From: Shannon Zhao [mailto:[email protected]]
Sent: Tuesday, June 24, 2014 1:16 PM
To: Bhushan Bharat-R65777; Alvise Rigo; a.motakis-lrHrjnjw1UfHK3s98zE1ajGjJy/[email protected]; iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org; [email protected]; [email protected]
Subject: Re: Vfio-on-arm: the device tree does not contain the DMA device

Quoted 7 lines

Hi Bharat,

Thanks for your help.

I executed the following cmd:
echo vfio-platform >
/sys/bus/platform/devices/2c0a0000.dma/driver_override

You have to bind the device with vfio-platform driver after the above command.

Quoted 5 lines

But there is no /dev/vfio/0 and error reported starting guest

-nographic can not be used with -daemonize
char device redirected to /dev/pts/3 (label pty0-base)
PL330_VFIO: Group is not viable (not all devices bound for vfio)

A bit surprised; it should be failing when opening /dev/vfio/0 but it is failing in VFIO_GROUP_GET_STATUS ioctl.

Also looks like there are more device attached to the said iommu-group. Can check by running below command:
#ls -l /sys/bus/pci/devices/2c0a0000.dma/iommu_group/devices

Thanks
-Bharat

Quoted 594 lines

qemu-system-arm: Error while probing real device.

qemu-system-arm: Initialization of device pl330 failed Is there something
wrong?

Thank you very much.
Shannon

Quoted 93 lines · On 2014/6/24 15:05, [email protected]…
Quoted 93 lines

-----Original Message-----
From: Shannon Zhao [mailto:[email protected]]
Sent: Tuesday, June 24, 2014 12:30 PM
To: Alvise Rigo; Bhushan Bharat-R65777;
a.motakis-lrHrjnjw1UfHK3s98zE1ajGjJy/[email protected]; iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org; [email protected]; [email protected]
Subject: Re: Vfio-on-arm: the device tree does not contain the DMA
device

Hi Alvise,

Thanks for your help.
Now I can see /dev/vfio/vfio and /dev/vfio/0 after executing "echo 2c0a0000.dma

Quoted 1 line

/sys/bus/platform/drivers/vfio-platform/vfio_bind".

But when using qemu to start guest, there is an error :

-nographic can not be used with -daemonize char device redirected to /dev/pts/3 (label pty0-base)
PL330_VFIO: === VFIO device file descriptor 13 === Device has 1 region(s):
PL330_VFIO: - Region 0: size=0x1000 offset=0x2c0a0000 flags=0x7
PL330_VFIO: - IRQ 0: range of 1, flags=0x7 ------------[ cut here ]------------ kernel BUG at mm/slub.c:3386!

Internal error: Oops - BUG: 0 [#1] SMP ARM Modules linked in:
CPU: 2 PID: 1746 Comm: qemu-system-arm Tainted: G        W    3.14.0-rc1+ #19
task: ee665580 ti: ed0b8000 task.ti: ed0b8000 PC is at
kfree+0x128/0x134 LR is at vfio_set_trigger.isra.5+0x40/0x104
pc : [<800d9e3c>]    lr : [<80319094>]    psr: 400f0013

sp : ed0b9e64 ip : 000800f3 fp : 000014ac
r10: 00000000 r9 : ed0b8000 r8 : edeb3b80
r7 : 0000000e r6 : 80620584 r5 : 00000000 r4 : eea0b000 r3 : 00000400 r2 : 800f38b0 r1 : eea0ce60 r0 : 00000400
Flags: nZcv IRQs on FIQs on Mode SVC_32 ISA ARM Segment user
Control: 30c53c7d Table: eddac080 DAC: 00000000 Process qemu-system-arm (pid: 1746, stack limit = 0xed0b8240)

Stack: (0xed0b9e64 to 0xed0ba000)

9e60: ee603700 00000000 ee603700 0000000e edeb3b80 00000000 80319094
9e80: ee603700 00003b6e 00000000 edeb3b80 ee603700 00003b6e 0000000d 80319208
9ea0: 0000000e edeb3b80 80492a18 edeb3b80 ee57f780 80319690 00000024 008b2074
9ec0: 008b2074 8031886c 00000001 008b2074 00000018 00000024 00000000 00000000
9ee0: 00000001 00000004 00000001 00000001 ee088e70 008b2060 ee57f780 803155d0
9f00: 803155b0 800ef430 80477a58 00080802 ed0b8000 800e1358 00000020 ed0b9f50
9f20: ed0b9f60 00000003 8061e808 00080802 ed0b9f60 800e14f0 00000000 806522f8
9f40: 80477a58 8061e808 ede4c1c0 8011af18 00000000 00000009 8053bbc0 00000000
9f60: ee683610 00000001 ee57f780 008b2060 00003b6e 0000000d ed0b8000 00000000
9f80: 000014ac 800ef9ac 0000000e 00000001 008b2060 00429eb0 0000000d 00000036
9fa0: 8001c464 8001c2e0 008b2060 00429eb0 0000000d 00003b6e 008b2060 00000001
9fc0: 008b2060 00429eb0 0000000d 00000036 00000000 7ea99040 0089a220 000014ac
9fe0: 003fa474 7ea98fc4 000bc5c8 76ceb2ec 200f0010 0000000d 00000000 8062c770 [<800d9e3c>] (kfree) from [<80319094>] (vfio_set_trigger.isra.5+0x40/0x104)
[<80319094>] (vfio_set_trigger.isra.5) from [<80319208>] (vfio_platform_set_irq_trigger+0xb0/0x144)
[<80319208>] (vfio_platform_set_irq_trigger) from [<80319690>] (vfio_platform_set_irqs_ioctl+0x44/0x58)
[<80319690>] (vfio_platform_set_irqs_ioctl) from [<8031886c>] (vfio_platform_ioctl+0x2e0/0x314)
[<8031886c>] (vfio_platform_ioctl) from [<803155d0>] (vfio_device_fops_unl_ioctl+0x20/0x2c)
[<803155d0>] (vfio_device_fops_unl_ioctl) from [<800ef430>] (do_vfs_ioctl+0x78/0x5b8)
[<800ef430>] (do_vfs_ioctl) from [<800ef9ac>] (SyS_ioctl+0x3c/0x5c) [<800ef9ac>] (SyS_ioctl) from [<8001c2e0>] (ret_fast_syscall+0x0/0x30)

Code: 15913038 e1a01003 e8bd45f0 eaff28e5 (e7f001f2)

I'm using branch vfio-platform-v4 for kernel and pl330-vfio-dev for qemu. Following is qemu cmdline:

	./qemu-system-arm -enable-kvm -M vexpress-a15 -cpu cortex-a15 \
       	-chardev pty,id=pty0,mux=on -monitor chardev:pty0 \
       	-serial chardev:pty0 -daemonize \
       	-device

pl330,vfio_group="/dev/vfio/0",vfio_device="2c0a0000.dma",vfio_contai ner="/dev/v
fio/vfio" \
-append "earlyprintk ip=dhcp console=ttyAMA0 rootwait" \
-kernel zImage -dtb guest-a15.dtb -initrd filesystem.cpio.gz
- nographic

Quoted 4 lines
Quoted 2 lines

If I change to branch vfio-platform-v6 for kernel, there is "/dev/vfio/vfio". But no "/sys/bus/platform/drivers/vfio-platform/vfio_bind".

Can you Check if you have driver_override at

"/sys/bus/platform/devices/DEVICE-XXX/ ?

Quoted 1 line

If yes then you should use driver_override mechanism to bind that device to

vfio-platform driver.

Quoted 482 lines

Thanks
-Bharat

Quoted 477 lines

And if I use above qemu cmdline to start guest, there is also an error :
PL330_VFIO: Group is not viable (not all devices bound for vfio)
qemu-system-arm: Error while probing real device.

qemu-system-arm: Initialization of device pl330 failed

The usage about v4 and v6 has changed ?
And would you tell me when you will update the guideline about vfio-on-arm?

Thanks,
Shannon

Quoted 12 lines · Alvise Rigo

Linaro has a great set of pre-built images here http://releases.linaro.org/.

For example these should work fine:
http://releases.linaro.org/13.12/ubuntu/saucy-images

alvise

Il 20/06/2014 10:23, Shannon Zhao ha scritto:

Quoted 4 lines

Hi Alvise,

I'm so sorry that I can't use qemu-debootstrap because my environment can't

access some links using proxy.

Quoted 14 lines
Quoted 14 lines

Are there other ways to get a proper fs?

Thanks
Shannon

Quoted 8 lines · Alvise Rigo

Hi Shannon,

If you follow the steps in the section "Creating the File System" of the guide you should be able to obtain a working fs.

Regards,
alvise

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines

Il 20/06/2014 04:57, Shannon Zhao ha scritto:

Quoted 5 lines

Hi Bharat,

Thanks for your advice .

I'm using the rootfs getting from the url

"http://www.virtualopensystems.com/downloads/guides/kvm_on_arm/fs-ali p-
armel.cramfs".

Quoted 40 lines
Quoted 40 lines
Quoted 40 lines
Quoted 40 lines

Can you give me another link to get a newer rootfs?

Thank you
Shannon

Quoted 33 lines · On 2014/6/19 20:20, [email protected]…
Quoted 33 lines

-----Original Message-----
From: Shannon Zhao [mailto:[email protected]]
Sent: Thursday, June 19, 2014 5:47 PM
To: Bhushan Bharat-R65777; Alvise Rigo;
a.motakis-lrHrjnjw1UfHK3s98zE1ajGjJy/[email protected]; iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org; [email protected]
Subject: Re: Vfio-on-arm: the device tree does not contain the
DMA device

Quoted 19 lines · On 2014/6/19 20:12, [email protected]…
Quoted 12 lines

-----Original Message-----
From: Shannon Zhao [mailto:[email protected]]
Sent: Thursday, June 19, 2014 5:38 PM
To: Alvise Rigo; Bhushan Bharat-R65777; a.motakis-lrHrjnjw1UfHK3s98zE1ajGjJy/[email protected]; iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org; [email protected]
Subject: Re: Vfio-on-arm: the device tree does not contain
the DMA device

Quoted 5 lines

Hi Alvise,

I run following cmds :

	echo 2c0a0000.dma >

/sys/bus/platform/drivers/vfio-platform/vfio_bind

run "ls /dev/vfio/" before and after the above command

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 5 lines

Hi Bharat,

Nothing appeared either before or after the cmd "echo 2c0a0000.dma > /sys/bus/platform/drivers/vfio-platform/vfio_bind". There is only /dev/vfio.

I am not sure, but may be you are using pretty old rootfs with broken udev

rules. Try to fetch a newer rootfs form somewhere.

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines

Thanks
-Bharat

Quoted 1 line
Quoted 1 line
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 7 lines

Thanks
-Bharat

Quoted 3 lines
	./qemu-system-arm -enable-kvm -M vexpress-a15 -cpu cortex-a15 \
        	-device

pl330,vfio_group="/dev/vfio/0",vfio_device="2c0a0000.dma",vfio_contai ner="/dev/v

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

fio/vfio" \
-append "earlyprintk ip=dhcp console=ttyAMA0 rootwait" \
-kernel zImage -dtb guest-a15.dtb -initrd
filesystem.cpio.gz

-

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 3 lines
Quoted 3 lines

nographic But the error still exists.
qemu-system-arm: -device

pl330,vfio_group=/dev/vfio/0,vfio_device=2c0a0000.dma,vfio_container= /dev/vfio/v

Quoted 33 lines
Quoted 33 lines
Quoted 33 lines
Quoted 33 lines
Quoted 33 lines
Quoted 33 lines
Quoted 33 lines
Quoted 33 lines

fio: Parameter 'driver' expects pluggable device type

Thanks,
Shannon

Quoted 27 lines · Alvise Rigo

Il 19/06/2014 13:11, Shannon Zhao ha scritto:

Quoted 24 lines · Alvise Rigo

Hi,

Il 19/06/2014 11:38, Shannon Zhao ha scritto:
Hi Bharat,

On 2014/6/19 17:19, [email protected] wrote:

-----Original Message-----
From: [email protected]
[mailto:kvmarm- [email protected]] On
Behalf Of Shannon Zhao
Sent: Thursday, June 19, 2014 2:45 PM
To: Alvise Rigo; a.motakis-lrHrjnjw1UfHK3s98zE1ajGjJy/[email protected];
iommu-cunTk1MwBs/[email protected] foundation.org;
[email protected]
Subject: Re: Vfio-on-arm: the device tree does not
contain the DMA device

Hi Alvise,

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines

Thanks for your reply.
I have apply the patch and could see the DMA devices.
But there is no vosys_dmatest int the directory
"/sys/kernel/debug/" after mounting debugfs.
I found that the branch vfio-platform-v4 doesn't
contain the module vosys_dmatest.
Then I port it from the branch pl330_dmatest and run

vosys_dmatest

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines

successfully.

Quoted 5 lines
Quoted 5 lines

echo 1 > /sys/kernel/debug/vosys_dmatest/start

[ 6145.343419] dmaengine: __dma_request_channel:
success

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines

(dma0chan0)

Quoted 7 lines
Quoted 7 lines

[ 6145.344000] vosys dmatest: callback: transfer completed
[ 6145.344147] vosys dmatest: tx completed
[ 6145.344589] vosys dmatest: source and destination match.

Download the qemu and use following command to start a guest:
./qemu-system-arm -enable-kvm -M vexpress-a15 -cpu
cortex-

a15 \

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 1 line
Quoted 1 line

-device

pl330,vfio_group="/dev/vfio/0",vfio_device="2c0a0000.dma",\

Quoted 2 lines
Quoted 2 lines

vfio_container="/dev/vfio/vfio" \
-append "earlyprintk ip=dhcp console=ttyAMA0 rootwait"

\

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

-kernel zImage -dtb guest.dtb -initrd

filesystem.cpio.gz

Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 6 lines
Quoted 6 lines

- nographic An error reported:
qemu-system-arm: -device
pl330,vfio_group=/dev/vfio/0,vfio_device=2c0a0000.dma,:

Why do we need to give "vfio_group=/dev/vfio/0" ?
I refer to the url

"http://www.virtualopensystems.com/en/solutions/guides/vfio-on-arm/".

Quoted 2 lines
Quoted 2 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines

could not open disk image vfio_container=/dev/vfio:
Could not open 'vfio_container=/dev/vfio': No such
file or directory

Are you unbinding "2c0a0000.dma" from its driver and
binding with

"vfio-

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

playform" driver ?

Quoted 3 lines
Quoted 3 lines

How to unbind "2c0a0000.dma" from its driver?
I have tried following cmd to bind "2c0a0000.dma" with
"vfio-

playform"

Quoted 25 lines
Quoted 25 lines
Quoted 25 lines
Quoted 25 lines
Quoted 25 lines
Quoted 25 lines
Quoted 25 lines
Quoted 25 lines

driver.

Quoted 24 lines
Quoted 24 lines

echo 2c0a0000.dma >
/sys/bus/platform/drivers/vfio-platform/vfio_bind
But there is no /dev/vfio/0 or /dev/vfio/vfio .

Can you please verify that while configuring the host you enabled:
- Device Drivers -> IOMMU Hardware Support -> ARM Ltd.
System MMU
(SMMU) Support and
- Device Drivers -> VFIO Non-Privileged userspace driver
framework
-> VFIO support for platform devices

Hi Alvise,

I have checked the two configuration are enabled.

Then try the following cmd, but another error reported:
qemu-system-arm: -device
pl330,vfio_group=/dev/vfio/0,vfio_device=2c0a0000.dma,vfio_
container=
/dev/vfio/vfio: Parameter 'driver' expects pluggable device
type

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

Do I need to unbind the "2c0a0000.dma" from its driver and
binding

with

Quoted 15 lines
Quoted 15 lines
Quoted 15 lines
Quoted 15 lines
Quoted 15 lines
Quoted 15 lines
Quoted 15 lines
Quoted 15 lines

"vfio-platform" driver as Bharat mentioned ?

Quoted 14 lines

Yes, the 2c0a0000.dma should not be bound to any driver at boot, so you just need to:
echo 2c0a0000.dma > /sys/bus/platform/drivers/vfio-platform/vfio_bind to bind the device to the VFIO driver.

Regards,
alvise

Quoted 3 lines

Then try to update the pl330-vfio-dev branch and run QEMU
with the

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines

following

Quoted 5 lines
Quoted 5 lines

command:

Quoted 3 lines
Quoted 3 lines

./qemu-system-arm -enable-kvm -M vexpress-a15 -cpu cortex-a15 \
-device

pl330,vfio_group="/dev/vfio/0",vfio_device="2c0a0000.dma",vfio_contai ner="/dev/v

Quoted 18 lines
Quoted 18 lines
Quoted 18 lines
Quoted 18 lines
Quoted 18 lines
Quoted 18 lines
Quoted 18 lines
Quoted 18 lines

fio/vfio" \

Quoted 17 lines
Quoted 17 lines

-append "earlyprintk ip=dhcp console=ttyAMA0 rootwait" \
-kernel zImage -dtb guest.dtb -initrd
filesystem.cpio.gz

that means without breaking the "-device pl330" options.

Regards,
alvise

Thanks
-Bharat

Is there something I'm wrong or do I miss some steps?

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines

Thank you,
Shannon

On 2014/6/18 21:24, Alvise Rigo wrote:
Hi Shannon,

Thank you for reporting this issue, the device tree
file was

indeed

Quoted 181 lines
Quoted 181 lines
Quoted 181 lines
Quoted 180 lines
Quoted 179 lines
Quoted 170 lines
Quoted 170 lines
Quoted 168 lines

missing.

Quoted 158 lines
Quoted 158 lines

To solve the problem, you can apply the following
patch to the repository you mentioned
(git://github.com/virtualopensystems/arm-dts.git).

For your information, soon we will update the guide
with a newer version of VFIO with this issue fixed.

Thank you,
alvise

---
fast_models/rtsm_ve-cortex_a15x4_a7x4.dts | 100
+++++++++++++++++++++++++++++-
1 file changed, 97 insertions(+), 3 deletions(-)

diff --git a/fast_models/rtsm_ve-cortex_a15x4_a7x4.dts
b/fast_models/rtsm_ve-cortex_a15x4_a7x4.dts
index ef1275b..79c6af2 100644
--- a/fast_models/rtsm_ve-cortex_a15x4_a7x4.dts
+++ b/fast_models/rtsm_ve-cortex_a15x4_a7x4.dts
@@ -142,7 +142,7 @@
cpu7: cpu@7 {
device_type = "cpu";
compatible = "arm,cortex-a7";
- reg = <0x104>;
+ reg = <0x103>;
cluster = <&cluster1>;
core = <&core7>;
};
@@ -159,7 +159,7 @@
#address-cells = <0>;
interrupt-controller;
reg = <0 0x2c001000 0 0x1000>,
- <0 0x2c002000 0 0x2000>,
+ <0 0x2c002000 0 0x1000>,
<0 0x2c004000 0 0x2000>,
<0 0x2c006000 0 0x2000>;
interrupts = <1 9 0xf04>; @@ -183,6 +183,101 @@
<1 10 0xf08>;
};

+ smmu@2c0b0000 {
+ compatible = "arm,smmu-v2";
+ reg = <0x0 0x2c0b0000 0x0 0x10000>;
+ #global-interrupts = <2>;
+ interrupts = <0 58 4>,
+ <0 59 4>,
+ <0 60 4>,
+ <0 60 4>,
+ <0 60 4>,
+ <0 60 4>,
+ <0 60 4>,
+ <0 60 4>,
+ <0 60 4>;
+ mmu-masters = <&dma0 0x0 0x8>,
+ <&dma1 0x1 0x9>,
+ <&dma2 0x2 0xa>,
+ <&dma3 0x3 0xb>;
+ };
+
+ dma0: dma@2c0a0000 {
+ compatible = "arm,pl330";
+ reg = <0 0x2c0a0000 0 0x1000>;
+ interrupts = <0 64 4>,
+ <0 65 4>,
+ <0 66 4>,
+ <0 67 4>,
+ <0 68 4>,
+ <0 69 4>,
+ <0 70 4>,
+ <0 71 4>;
+ #dma-cells = <1>;
+ #stream-id-cells = <2>;
+ clocks = <&oscclk6a>;
+ clock-names = "apb_pclk";
+ };
+
+ dma1: dma@2c0a1000 {
+ compatible = "arm,pl330", "arm,primecell";
+ reg = <0 0x2c0a1000 0 0x1000>;
+ interrupts = <0 96 4>,
+ <0 97 4>,
+ <0 98 4>,
+ <0 99 4>,
+ <0 100 4>,
+ <0 101 4>,
+ <0 102 4>,
+ <0 103 4>;
+ #dma-cells = <1>;
+ #stream-id-cells = <2>;
+ clocks = <&oscclk6a>;
+ clock-names = "apb_pclk";
+ };
+
+ dma2: dma@2c0a2000 {
+ compatible = "arm,pl330", "arm,primecell";
+ reg = <0 0x2c0a2000 0 0x1000>;
+ interrupts = <0 128 4>,
+ <0 129 4>,
+ <0 130 4>,
+ <0 131 4>,
+ <0 132 4>,
+ <0 133 4>,
+ <0 134 4>,
+ <0 135 4>;
+ #dma-cells = <1>;
+ #stream-id-cells = <2>;
+ clocks = <&oscclk6a>;
+ clock-names = "apb_pclk";
+ };
+
+ dma3: dma@2c0a3000 {
+ compatible = "arm,pl330", "arm,primecell";
+ reg = <0 0x2c0a3000 0 0x1000>;
+ interrupts = <0 160 4>,
+ <0 161 4>,
+ <0 162 4>,
+ <0 163 4>,
+ <0 164 4>,
+ <0 165 4>,
+ <0 166 4>,
+ <0 167 4>;
+ #dma-cells = <1>;
+ #stream-id-cells = <2>;
+ clocks = <&oscclk6a>;
+ clock-names = "apb_pclk";
+ };
+
+ oscclk6a: oscclk6a {
+ /* Reference 24MHz clock */
+ compatible = "fixed-clock";
+ #clock-cells = <0>;
+ clock-frequency = <24000000>;
+ clock-output-names = "oscclk6a";
+ };
+
smb {
compatible = "simple-bus";

@@ -244,4 +339,3 @@
/include/ "rtsm_ve-motherboard.dtsi"
};
};
-

--
Shannon

_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm

.

Quoted 6 lines
Quoted 3 lines

.

.

--
Shannon

Quoted 5 lines
Quoted 2 lines

.

--
Shannon

.

Quoted 6 lines
Quoted 3 lines
Quoted 2 lines

.

.

--
Shannon

--
Shannon

#65 lore raw
In reply to a message not in the archive

Hi,

Quoted 50 lines · Shannon Zhao

Hi Alvise,

Thanks for your help.
Now I can see /dev/vfio/vfio and /dev/vfio/0 after executing "echo 2c0a0000.dma > /sys/bus/platform/drivers/vfio-platform/vfio_bind". But when using qemu to start guest, there is an error :

-nographic can not be used with -daemonize
char device redirected to /dev/pts/3 (label pty0-base)
PL330_VFIO: === VFIO device file descriptor 13 === Device has 1 region(s):
PL330_VFIO: - Region 0: size=0x1000 offset=0x2c0a0000 flags=0x7
PL330_VFIO: - IRQ 0: range of 1, flags=0x7

------------[ cut here ]------------
kernel BUG at mm/slub.c:3386!
Internal error: Oops - BUG: 0 [#1] SMP ARM
Modules linked in:
CPU: 2 PID: 1746 Comm: qemu-system-arm Tainted: G        W    3.14.0-rc1+ #19
task: ee665580 ti: ed0b8000 task.ti: ed0b8000
PC is at kfree+0x128/0x134
LR is at vfio_set_trigger.isra.5+0x40/0x104
pc : [<800d9e3c>]    lr : [<80319094>]    psr: 400f0013

sp : ed0b9e64 ip : 000800f3 fp : 000014ac
r10: 00000000 r9 : ed0b8000 r8 : edeb3b80
r7 : 0000000e r6 : 80620584 r5 : 00000000 r4 : eea0b000 r3 : 00000400 r2 : 800f38b0 r1 : eea0ce60 r0 : 00000400
Flags: nZcv IRQs on FIQs on Mode SVC_32 ISA ARM Segment user
Control: 30c53c7d Table: eddac080 DAC: 00000000 Process qemu-system-arm (pid: 1746, stack limit = 0xed0b8240)

Stack: (0xed0b9e64 to 0xed0ba000)

9e60: ee603700 00000000 ee603700 0000000e edeb3b80 00000000 80319094
9e80: ee603700 00003b6e 00000000 edeb3b80 ee603700 00003b6e 0000000d 80319208
9ea0: 0000000e edeb3b80 80492a18 edeb3b80 ee57f780 80319690 00000024 008b2074
9ec0: 008b2074 8031886c 00000001 008b2074 00000018 00000024 00000000 00000000
9ee0: 00000001 00000004 00000001 00000001 ee088e70 008b2060 ee57f780 803155d0
9f00: 803155b0 800ef430 80477a58 00080802 ed0b8000 800e1358 00000020 ed0b9f50
9f20: ed0b9f60 00000003 8061e808 00080802 ed0b9f60 800e14f0 00000000 806522f8
9f40: 80477a58 8061e808 ede4c1c0 8011af18 00000000 00000009 8053bbc0 00000000
9f60: ee683610 00000001 ee57f780 008b2060 00003b6e 0000000d ed0b8000 00000000
9f80: 000014ac 800ef9ac 0000000e 00000001 008b2060 00429eb0 0000000d 00000036
9fa0: 8001c464 8001c2e0 008b2060 00429eb0 0000000d 00003b6e 008b2060 00000001
9fc0: 008b2060 00429eb0 0000000d 00000036 00000000 7ea99040 0089a220 000014ac
9fe0: 003fa474 7ea98fc4 000bc5c8 76ceb2ec 200f0010 0000000d 00000000 8062c770 [<800d9e3c>] (kfree) from [<80319094>] (vfio_set_trigger.isra.5+0x40/0x104) [<80319094>] (vfio_set_trigger.isra.5) from [<80319208>] (vfio_platform_set_irq_trigger+0xb0/0x144) [<80319208>] (vfio_platform_set_irq_trigger) from [<80319690>] (vfio_platform_set_irqs_ioctl+0x44/0x58) [<80319690>] (vfio_platform_set_irqs_ioctl) from [<8031886c>] (vfio_platform_ioctl+0x2e0/0x314) [<8031886c>] (vfio_platform_ioctl) from [<803155d0>] (vfio_device_fops_unl_ioctl+0x20/0x2c) [<803155d0>] (vfio_device_fops_unl_ioctl) from [<800ef430>] (do_vfs_ioctl+0x78/0x5b8) [<800ef430>] (do_vfs_ioctl) from [<800ef9ac>] (SyS_ioctl+0x3c/0x5c)
[<800ef9ac>] (SyS_ioctl) from [<8001c2e0>] (ret_fast_syscall+0x0/0x30)

Quoted 1 line
Code: 15913038 e1a01003 e8bd45f0 eaff28e5 (e7f001f2)

I'm investigating what may have failed here.

Quoted 10 lines

I'm using branch vfio-platform-v4 for kernel and pl330-vfio-dev for qemu. Following is qemu cmdline:

        ./qemu-system-arm -enable-kvm -M vexpress-a15 -cpu cortex-a15 \
        -chardev pty,id=pty0,mux=on -monitor chardev:pty0 \
        -serial chardev:pty0 -daemonize \
        -device pl330,vfio_group="/dev/vfio/0",vfio_device="2c0a0000.dma",vfio_container="/dev/vfio/vfio" \
        -append "earlyprintk ip=dhcp console=ttyAMA0 rootwait" \
        -kernel zImage -dtb guest-a15.dtb -initrd filesystem.cpio.gz  -nographic
Quoted 8 lines

If I change to branch vfio-platform-v6 for kernel, there is "/dev/vfio/vfio". But no "/sys/bus/platform/drivers/vfio-platform/vfio_bind". And if I use above qemu cmdline to start guest, there is also an error :
PL330_VFIO: Group is not viable (not all devices bound for vfio)
qemu-system-arm: Error while probing real device.

qemu-system-arm: Initialization of device pl330 failed

The usage about v4 and v6 has changed ?

Yes, the usage it a bit changed. The guide is suppose to work with the v4 version of VFIO.

Quoted 1 line

And would you tell me when you will update the guideline about vfio-on-arm?

The new guide should be published this week and it will be based on a more recent release of VFIO
and QEMU.

alvise

Quoted 430 lines

Thanks,
Shannon

Quoted 416 lines · Alvise Rigo

Linaro has a great set of pre-built images here http://releases.linaro.org/.

For example these should work fine:
http://releases.linaro.org/13.12/ubuntu/saucy-images

alvise

Il 20/06/2014 10:23, Shannon Zhao ha scritto:

Quoted 408 lines

Hi Alvise,

I'm so sorry that I can't use qemu-debootstrap because my environment can't access some links using proxy. Are there other ways to get a proper fs?

Thanks
Shannon

Quoted 399 lines · Alvise Rigo

Hi Shannon,

If you follow the steps in the section "Creating the File System" of the guide you should be able to obtain a working fs.

Regards,
alvise

Il 20/06/2014 04:57, Shannon Zhao ha scritto:

Quoted 389 lines

Hi Bharat,

Thanks for your advice .

I'm using the rootfs getting from the url "http://www.virtualopensystems.com/downloads/guides/kvm_on_arm/fs-alip-armel.cramfs".

Can you give me another link to get a newer rootfs?

Thank you
Shannon

Quoted 376 lines · On 2014/6/19 20:20, [email protected]…
Quoted 29 lines

-----Original Message-----
From: Shannon Zhao [mailto:[email protected]]
Sent: Thursday, June 19, 2014 5:47 PM
To: Bhushan Bharat-R65777; Alvise Rigo; a.motakis-lrHrjnjw1UfHK3s98zE1ajGjJy/[email protected]; iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org; [email protected]
Subject: Re: Vfio-on-arm: the device tree does not contain the DMA device

Quoted 17 lines · On 2014/6/19 20:12, [email protected]…
Quoted 16 lines

-----Original Message-----
From: Shannon Zhao [mailto:[email protected]]
Sent: Thursday, June 19, 2014 5:38 PM
To: Alvise Rigo; Bhushan Bharat-R65777;
a.motakis-lrHrjnjw1UfHK3s98zE1ajGjJy/[email protected]; iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org; [email protected]
Subject: Re: Vfio-on-arm: the device tree does not contain the DMA
device

Hi Alvise,

I run following cmds :

       echo 2c0a0000.dma >

/sys/bus/platform/drivers/vfio-platform/vfio_bind

Quoted 6 lines
Quoted 1 line

run "ls /dev/vfio/" before and after the above command

Hi Bharat,

Nothing appeared either before or after the cmd "echo 2c0a0000.dma > /sys/bus/platform/drivers/vfio-platform/vfio_bind".
There is only /dev/vfio.

I am not sure, but may be you are using pretty old rootfs with broken udev rules. Try to fetch a newer rootfs form somewhere.

Thanks
-Bharat

Quoted 325 lines
Quoted 8 lines

Thanks
-Bharat

Quoted 3 lines
       ./qemu-system-arm -enable-kvm -M vexpress-a15 -cpu cortex-a15 \
               -device

pl330,vfio_group="/dev/vfio/0",vfio_device="2c0a0000.dma",vfio_container="/dev/v

Quoted 6 lines
Quoted 6 lines

fio/vfio" \
-append "earlyprintk ip=dhcp console=ttyAMA0 rootwait" \
-kernel zImage -dtb guest-a15.dtb -initrd filesystem.cpio.gz -
nographic But the error still exists.
qemu-system-arm: -device

pl330,vfio_group=/dev/vfio/0,vfio_device=2c0a0000.dma,vfio_container=/dev/vfio/v

Quoted 102 lines
Quoted 102 lines

fio: Parameter 'driver' expects pluggable device type

Thanks,
Shannon

Quoted 32 lines · Alvise Rigo

Il 19/06/2014 13:11, Shannon Zhao ha scritto:

Quoted 29 lines · Alvise Rigo
Quoted 26 lines

Hi,

Il 19/06/2014 11:38, Shannon Zhao ha scritto:

Quoted 23 lines

Hi Bharat,

On 2014/6/19 17:19, [email protected] wrote:

-----Original Message-----
From: [email protected] [mailto:kvmarm-
[email protected]] On Behalf Of Shannon Zhao
Sent: Thursday, June 19, 2014 2:45 PM
To: Alvise Rigo; a.motakis-lrHrjnjw1UfHK3s98zE1ajGjJy/[email protected];
iommu-cunTk1MwBs/[email protected] foundation.org; [email protected]
Subject: Re: Vfio-on-arm: the device tree does not contain the
DMA device

Hi Alvise,

Thanks for your reply.
I have apply the patch and could see the DMA devices.
But there is no vosys_dmatest int the directory
"/sys/kernel/debug/" after mounting debugfs.
I found that the branch vfio-platform-v4 doesn't contain the
module vosys_dmatest.
Then I port it from the branch pl330_dmatest and run vosys_dmatest

successfully.

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

echo 1 > /sys/kernel/debug/vosys_dmatest/start

[ 6145.343419] dmaengine: __dma_request_channel: success

(dma0chan0)

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines

[ 6145.344000] vosys dmatest: callback: transfer completed
[ 6145.344147] vosys dmatest: tx completed
[ 6145.344589] vosys dmatest: source and destination match.

Download the qemu and use following command to start a guest:
./qemu-system-arm -enable-kvm -M vexpress-a15 -cpu cortex-a15 \
-device

pl330,vfio_group="/dev/vfio/0",vfio_device="2c0a0000.dma",\

Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines

vfio_container="/dev/vfio/vfio" \
-append "earlyprintk ip=dhcp console=ttyAMA0 rootwait" \
-kernel zImage -dtb guest.dtb -initrd filesystem.cpio.gz
- nographic An error reported:
qemu-system-arm: -device
pl330,vfio_group=/dev/vfio/0,vfio_device=2c0a0000.dma,:

Why do we need to give "vfio_group=/dev/vfio/0" ?
I refer to the url

"http://www.virtualopensystems.com/en/solutions/guides/vfio-on-arm/".

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines

could not open disk image vfio_container=/dev/vfio:
Could not open 'vfio_container=/dev/vfio': No such file or
directory

Are you unbinding "2c0a0000.dma" from its driver and binding with "vfio-

playform" driver ?

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

How to unbind "2c0a0000.dma" from its driver?
I have tried following cmd to bind "2c0a0000.dma" with "vfio-playform"

driver.

Quoted 13 lines
Quoted 13 lines
Quoted 11 lines
Quoted 3 lines

echo 2c0a0000.dma >
/sys/bus/platform/drivers/vfio-platform/vfio_bind
But there is no /dev/vfio/0 or /dev/vfio/vfio .

Can you please verify that while configuring the host you enabled:
- Device Drivers -> IOMMU Hardware Support -> ARM Ltd. System MMU (SMMU) Support and
- Device Drivers -> VFIO Non-Privileged userspace driver framework -> VFIO support for platform devices

Hi Alvise,

Quoted 8 lines
Quoted 8 lines

I have checked the two configuration are enabled.

Then try the following cmd, but another error reported:
qemu-system-arm: -device
pl330,vfio_group=/dev/vfio/0,vfio_device=2c0a0000.dma,vfio_container= /dev/vfio/vfio: Parameter 'driver' expects pluggable device type

Do I need to unbind the "2c0a0000.dma" from its driver and binding with

"vfio-platform" driver as Bharat mentioned ?

Quoted 7 lines

Yes, the 2c0a0000.dma should not be bound to any driver at boot, so you just need to:
echo 2c0a0000.dma > /sys/bus/platform/drivers/vfio-platform/vfio_bind to bind the device to the VFIO driver.

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

Regards,
alvise

Quoted 2 lines
Quoted 2 lines

Then try to update the pl330-vfio-dev branch and run QEMU with the

following

Quoted 5 lines
Quoted 5 lines

command:

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

./qemu-system-arm -enable-kvm -M vexpress-a15 -cpu cortex-a15 \
-device

pl330,vfio_group="/dev/vfio/0",vfio_device="2c0a0000.dma",vfio_container="/dev/v

Quoted 195 lines
Quoted 193 lines

fio/vfio" \

Quoted 16 lines
Quoted 16 lines
Quoted 16 lines

-append "earlyprintk ip=dhcp console=ttyAMA0 rootwait" \
-kernel zImage -dtb guest.dtb -initrd filesystem.cpio.gz

that means without breaking the "-device pl330" options.

Regards,
alvise

Quoted 8 lines

Thanks
-Bharat

Is there something I'm wrong or do I miss some steps?

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines

Thank you,
Shannon

On 2014/6/18 21:24, Alvise Rigo wrote:
Hi Shannon,

Thank you for reporting this issue, the device tree file was indeed

missing.

Quoted 159 lines
Quoted 159 lines
Quoted 159 lines
Quoted 158 lines

To solve the problem, you can apply the following patch to the
repository you mentioned
(git://github.com/virtualopensystems/arm-dts.git).

For your information, soon we will update the guide with a newer
version of VFIO with this issue fixed.

Thank you,
alvise

---
fast_models/rtsm_ve-cortex_a15x4_a7x4.dts | 100
+++++++++++++++++++++++++++++-
1 file changed, 97 insertions(+), 3 deletions(-)

diff --git a/fast_models/rtsm_ve-cortex_a15x4_a7x4.dts
b/fast_models/rtsm_ve-cortex_a15x4_a7x4.dts
index ef1275b..79c6af2 100644
--- a/fast_models/rtsm_ve-cortex_a15x4_a7x4.dts
+++ b/fast_models/rtsm_ve-cortex_a15x4_a7x4.dts
@@ -142,7 +142,7 @@
cpu7: cpu@7 {
device_type = "cpu";
compatible = "arm,cortex-a7";
- reg = <0x104>;
+ reg = <0x103>;
cluster = <&cluster1>;
core = <&core7>;
};
@@ -159,7 +159,7 @@
#address-cells = <0>;
interrupt-controller;
reg = <0 0x2c001000 0 0x1000>,
- <0 0x2c002000 0 0x2000>,
+ <0 0x2c002000 0 0x1000>,
<0 0x2c004000 0 0x2000>,
<0 0x2c006000 0 0x2000>;
interrupts = <1 9 0xf04>;
@@ -183,6 +183,101 @@
<1 10 0xf08>;
};

+ smmu@2c0b0000 {
+ compatible = "arm,smmu-v2";
+ reg = <0x0 0x2c0b0000 0x0 0x10000>;
+ #global-interrupts = <2>;
+ interrupts = <0 58 4>,
+ <0 59 4>,
+ <0 60 4>,
+ <0 60 4>,
+ <0 60 4>,
+ <0 60 4>,
+ <0 60 4>,
+ <0 60 4>,
+ <0 60 4>;
+ mmu-masters = <&dma0 0x0 0x8>,
+ <&dma1 0x1 0x9>,
+ <&dma2 0x2 0xa>,
+ <&dma3 0x3 0xb>;
+ };
+
+ dma0: dma@2c0a0000 {
+ compatible = "arm,pl330";
+ reg = <0 0x2c0a0000 0 0x1000>;
+ interrupts = <0 64 4>,
+ <0 65 4>,
+ <0 66 4>,
+ <0 67 4>,
+ <0 68 4>,
+ <0 69 4>,
+ <0 70 4>,
+ <0 71 4>;
+ #dma-cells = <1>;
+ #stream-id-cells = <2>;
+ clocks = <&oscclk6a>;
+ clock-names = "apb_pclk";
+ };
+
+ dma1: dma@2c0a1000 {
+ compatible = "arm,pl330", "arm,primecell";
+ reg = <0 0x2c0a1000 0 0x1000>;
+ interrupts = <0 96 4>,
+ <0 97 4>,
+ <0 98 4>,
+ <0 99 4>,
+ <0 100 4>,
+ <0 101 4>,
+ <0 102 4>,
+ <0 103 4>;
+ #dma-cells = <1>;
+ #stream-id-cells = <2>;
+ clocks = <&oscclk6a>;
+ clock-names = "apb_pclk";
+ };
+
+ dma2: dma@2c0a2000 {
+ compatible = "arm,pl330", "arm,primecell";
+ reg = <0 0x2c0a2000 0 0x1000>;
+ interrupts = <0 128 4>,
+ <0 129 4>,
+ <0 130 4>,
+ <0 131 4>,
+ <0 132 4>,
+ <0 133 4>,
+ <0 134 4>,
+ <0 135 4>;
+ #dma-cells = <1>;
+ #stream-id-cells = <2>;
+ clocks = <&oscclk6a>;
+ clock-names = "apb_pclk";
+ };
+
+ dma3: dma@2c0a3000 {
+ compatible = "arm,pl330", "arm,primecell";
+ reg = <0 0x2c0a3000 0 0x1000>;
+ interrupts = <0 160 4>,
+ <0 161 4>,
+ <0 162 4>,
+ <0 163 4>,
+ <0 164 4>,
+ <0 165 4>,
+ <0 166 4>,
+ <0 167 4>;
+ #dma-cells = <1>;
+ #stream-id-cells = <2>;
+ clocks = <&oscclk6a>;
+ clock-names = "apb_pclk";
+ };
+
+ oscclk6a: oscclk6a {
+ /* Reference 24MHz clock */
+ compatible = "fixed-clock";
+ #clock-cells = <0>;
+ clock-frequency = <24000000>;
+ clock-output-names = "oscclk6a";
+ };
+
smb {
compatible = "simple-bus";

@@ -244,4 +339,3 @@
/include/ "rtsm_ve-motherboard.dtsi"
};
};
-

--
Shannon

_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm

.

Quoted 6 lines
Quoted 3 lines
Quoted 2 lines

.

.

--
Shannon

Quoted 5 lines
Quoted 2 lines

.

--
Shannon

.

Quoted 6 lines
Quoted 3 lines
Quoted 2 lines

.

.

--
Shannon

#49 lore raw
In reply to a message not in the archive

Hi Bharat,

ls -l /sys/bus/platform/devices/2c0a0000.dma/iommu_group/devices
total 0
lrwxrwxrwx 1 root root 0 Jan 1 02:58 2c0a0000.dma -> ../../../../devices/2c0a0000.dma

Then I run "echo 2c0a0000.dma > /sys/bus/platform/drivers/vfio-platform/bind". There is /dev/vfio/0.

But there is also an error reported.

	-nographic can not be used with -daemonize
	char device redirected to /dev/pts/3 (label pty0-base)
	PL330_VFIO: === VFIO device file descriptor 13 ===
	Device has 1 region(s):
	PL330_VFIO: - Region 0: size=0x1000 offset=0x0 flags=0x7
	PL330_VFIO: - IRQ 0: range of 1, flags=0x7
	PL330_VFIO: - IRQ 1: range of 1, flags=0x7
	PL330_VFIO: - IRQ 2: range of 1, flags=0x7
	PL330_VFIO: - IRQ 3: range of 1, flags=0x7
	PL330_VFIO: - IRQ 4: range of 1, flags=0x7
	qemu-system-arm: -device pl330,vfio_group=/dev/vfio/0,vfio_device=2c0a0000.dma,vfio_container=/dev/vfio/vfio: Property '.vfio_group' not found

Thanks,
Shannon

Quoted 630 lines · On 2014/6/24 16:08, [email protected]…
Quoted 10 lines

-----Original Message-----
From: Shannon Zhao [mailto:[email protected]]
Sent: Tuesday, June 24, 2014 1:16 PM
To: Bhushan Bharat-R65777; Alvise Rigo; a.motakis-lrHrjnjw1UfHK3s98zE1ajGjJy/[email protected]; iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org; [email protected]; [email protected]
Subject: Re: Vfio-on-arm: the device tree does not contain the DMA device

Quoted 7 lines

Hi Bharat,

Thanks for your help.

I executed the following cmd:
echo vfio-platform >
/sys/bus/platform/devices/2c0a0000.dma/driver_override

You have to bind the device with vfio-platform driver after the above command.

Quoted 5 lines

But there is no /dev/vfio/0 and error reported starting guest

-nographic can not be used with -daemonize
char device redirected to /dev/pts/3 (label pty0-base)
PL330_VFIO: Group is not viable (not all devices bound for vfio)

A bit surprised; it should be failing when opening /dev/vfio/0 but it is failing in VFIO_GROUP_GET_STATUS ioctl.

Also looks like there are more device attached to the said iommu-group. Can check by running below command:
#ls -l /sys/bus/pci/devices/2c0a0000.dma/iommu_group/devices

Thanks
-Bharat

Quoted 594 lines

qemu-system-arm: Error while probing real device.

qemu-system-arm: Initialization of device pl330 failed Is there something
wrong?

Thank you very much.
Shannon

Quoted 93 lines · On 2014/6/24 15:05, [email protected]…
Quoted 93 lines

-----Original Message-----
From: Shannon Zhao [mailto:[email protected]]
Sent: Tuesday, June 24, 2014 12:30 PM
To: Alvise Rigo; Bhushan Bharat-R65777;
a.motakis-lrHrjnjw1UfHK3s98zE1ajGjJy/[email protected]; iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org; [email protected]; [email protected]
Subject: Re: Vfio-on-arm: the device tree does not contain the DMA
device

Hi Alvise,

Thanks for your help.
Now I can see /dev/vfio/vfio and /dev/vfio/0 after executing "echo 2c0a0000.dma

Quoted 1 line

/sys/bus/platform/drivers/vfio-platform/vfio_bind".

But when using qemu to start guest, there is an error :

-nographic can not be used with -daemonize char device redirected to /dev/pts/3 (label pty0-base)
PL330_VFIO: === VFIO device file descriptor 13 === Device has 1 region(s):
PL330_VFIO: - Region 0: size=0x1000 offset=0x2c0a0000 flags=0x7
PL330_VFIO: - IRQ 0: range of 1, flags=0x7 ------------[ cut here ]------------ kernel BUG at mm/slub.c:3386!

Internal error: Oops - BUG: 0 [#1] SMP ARM Modules linked in:
CPU: 2 PID: 1746 Comm: qemu-system-arm Tainted: G        W    3.14.0-rc1+ #19
task: ee665580 ti: ed0b8000 task.ti: ed0b8000 PC is at
kfree+0x128/0x134 LR is at vfio_set_trigger.isra.5+0x40/0x104
pc : [<800d9e3c>]    lr : [<80319094>]    psr: 400f0013

sp : ed0b9e64 ip : 000800f3 fp : 000014ac
r10: 00000000 r9 : ed0b8000 r8 : edeb3b80
r7 : 0000000e r6 : 80620584 r5 : 00000000 r4 : eea0b000 r3 : 00000400 r2 : 800f38b0 r1 : eea0ce60 r0 : 00000400
Flags: nZcv IRQs on FIQs on Mode SVC_32 ISA ARM Segment user
Control: 30c53c7d Table: eddac080 DAC: 00000000 Process qemu-system-arm (pid: 1746, stack limit = 0xed0b8240)

Stack: (0xed0b9e64 to 0xed0ba000)

9e60: ee603700 00000000 ee603700 0000000e edeb3b80 00000000 80319094
9e80: ee603700 00003b6e 00000000 edeb3b80 ee603700 00003b6e 0000000d 80319208
9ea0: 0000000e edeb3b80 80492a18 edeb3b80 ee57f780 80319690 00000024 008b2074
9ec0: 008b2074 8031886c 00000001 008b2074 00000018 00000024 00000000 00000000
9ee0: 00000001 00000004 00000001 00000001 ee088e70 008b2060 ee57f780 803155d0
9f00: 803155b0 800ef430 80477a58 00080802 ed0b8000 800e1358 00000020 ed0b9f50
9f20: ed0b9f60 00000003 8061e808 00080802 ed0b9f60 800e14f0 00000000 806522f8
9f40: 80477a58 8061e808 ede4c1c0 8011af18 00000000 00000009 8053bbc0 00000000
9f60: ee683610 00000001 ee57f780 008b2060 00003b6e 0000000d ed0b8000 00000000
9f80: 000014ac 800ef9ac 0000000e 00000001 008b2060 00429eb0 0000000d 00000036
9fa0: 8001c464 8001c2e0 008b2060 00429eb0 0000000d 00003b6e 008b2060 00000001
9fc0: 008b2060 00429eb0 0000000d 00000036 00000000 7ea99040 0089a220 000014ac
9fe0: 003fa474 7ea98fc4 000bc5c8 76ceb2ec 200f0010 0000000d 00000000 8062c770 [<800d9e3c>] (kfree) from [<80319094>] (vfio_set_trigger.isra.5+0x40/0x104)
[<80319094>] (vfio_set_trigger.isra.5) from [<80319208>] (vfio_platform_set_irq_trigger+0xb0/0x144)
[<80319208>] (vfio_platform_set_irq_trigger) from [<80319690>] (vfio_platform_set_irqs_ioctl+0x44/0x58)
[<80319690>] (vfio_platform_set_irqs_ioctl) from [<8031886c>] (vfio_platform_ioctl+0x2e0/0x314)
[<8031886c>] (vfio_platform_ioctl) from [<803155d0>] (vfio_device_fops_unl_ioctl+0x20/0x2c)
[<803155d0>] (vfio_device_fops_unl_ioctl) from [<800ef430>] (do_vfs_ioctl+0x78/0x5b8)
[<800ef430>] (do_vfs_ioctl) from [<800ef9ac>] (SyS_ioctl+0x3c/0x5c) [<800ef9ac>] (SyS_ioctl) from [<8001c2e0>] (ret_fast_syscall+0x0/0x30)

Code: 15913038 e1a01003 e8bd45f0 eaff28e5 (e7f001f2)

I'm using branch vfio-platform-v4 for kernel and pl330-vfio-dev for qemu. Following is qemu cmdline:

	./qemu-system-arm -enable-kvm -M vexpress-a15 -cpu cortex-a15 \
       	-chardev pty,id=pty0,mux=on -monitor chardev:pty0 \
       	-serial chardev:pty0 -daemonize \
       	-device

pl330,vfio_group="/dev/vfio/0",vfio_device="2c0a0000.dma",vfio_contai ner="/dev/v
fio/vfio" \
-append "earlyprintk ip=dhcp console=ttyAMA0 rootwait" \
-kernel zImage -dtb guest-a15.dtb -initrd filesystem.cpio.gz
- nographic

Quoted 4 lines
Quoted 2 lines

If I change to branch vfio-platform-v6 for kernel, there is "/dev/vfio/vfio". But no "/sys/bus/platform/drivers/vfio-platform/vfio_bind".

Can you Check if you have driver_override at

"/sys/bus/platform/devices/DEVICE-XXX/ ?

Quoted 1 line

If yes then you should use driver_override mechanism to bind that device to

vfio-platform driver.

Quoted 482 lines

Thanks
-Bharat

Quoted 477 lines

And if I use above qemu cmdline to start guest, there is also an error :
PL330_VFIO: Group is not viable (not all devices bound for vfio)
qemu-system-arm: Error while probing real device.

qemu-system-arm: Initialization of device pl330 failed

The usage about v4 and v6 has changed ?
And would you tell me when you will update the guideline about vfio-on-arm?

Thanks,
Shannon

Quoted 12 lines · Alvise Rigo

Linaro has a great set of pre-built images here http://releases.linaro.org/.

For example these should work fine:
http://releases.linaro.org/13.12/ubuntu/saucy-images

alvise

Il 20/06/2014 10:23, Shannon Zhao ha scritto:

Quoted 4 lines

Hi Alvise,

I'm so sorry that I can't use qemu-debootstrap because my environment can't

access some links using proxy.

Quoted 14 lines
Quoted 14 lines

Are there other ways to get a proper fs?

Thanks
Shannon

Quoted 8 lines · Alvise Rigo

Hi Shannon,

If you follow the steps in the section "Creating the File System" of the guide you should be able to obtain a working fs.

Regards,
alvise

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines

Il 20/06/2014 04:57, Shannon Zhao ha scritto:

Quoted 5 lines

Hi Bharat,

Thanks for your advice .

I'm using the rootfs getting from the url

"http://www.virtualopensystems.com/downloads/guides/kvm_on_arm/fs-ali p-
armel.cramfs".

Quoted 40 lines
Quoted 40 lines
Quoted 40 lines
Quoted 40 lines

Can you give me another link to get a newer rootfs?

Thank you
Shannon

Quoted 33 lines · On 2014/6/19 20:20, [email protected]…
Quoted 33 lines

-----Original Message-----
From: Shannon Zhao [mailto:[email protected]]
Sent: Thursday, June 19, 2014 5:47 PM
To: Bhushan Bharat-R65777; Alvise Rigo;
a.motakis-lrHrjnjw1UfHK3s98zE1ajGjJy/[email protected]; iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org; [email protected]
Subject: Re: Vfio-on-arm: the device tree does not contain the
DMA device

Quoted 19 lines · On 2014/6/19 20:12, [email protected]…
Quoted 12 lines

-----Original Message-----
From: Shannon Zhao [mailto:[email protected]]
Sent: Thursday, June 19, 2014 5:38 PM
To: Alvise Rigo; Bhushan Bharat-R65777; a.motakis-lrHrjnjw1UfHK3s98zE1ajGjJy/[email protected]; iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org; [email protected]
Subject: Re: Vfio-on-arm: the device tree does not contain
the DMA device

Quoted 5 lines

Hi Alvise,

I run following cmds :

	echo 2c0a0000.dma >

/sys/bus/platform/drivers/vfio-platform/vfio_bind

run "ls /dev/vfio/" before and after the above command

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 5 lines

Hi Bharat,

Nothing appeared either before or after the cmd "echo 2c0a0000.dma > /sys/bus/platform/drivers/vfio-platform/vfio_bind". There is only /dev/vfio.

I am not sure, but may be you are using pretty old rootfs with broken udev

rules. Try to fetch a newer rootfs form somewhere.

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines

Thanks
-Bharat

Quoted 1 line
Quoted 1 line
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 7 lines

Thanks
-Bharat

Quoted 3 lines
	./qemu-system-arm -enable-kvm -M vexpress-a15 -cpu cortex-a15 \
        	-device

pl330,vfio_group="/dev/vfio/0",vfio_device="2c0a0000.dma",vfio_contai ner="/dev/v

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

fio/vfio" \
-append "earlyprintk ip=dhcp console=ttyAMA0 rootwait" \
-kernel zImage -dtb guest-a15.dtb -initrd
filesystem.cpio.gz

-

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 3 lines
Quoted 3 lines

nographic But the error still exists.
qemu-system-arm: -device

pl330,vfio_group=/dev/vfio/0,vfio_device=2c0a0000.dma,vfio_container= /dev/vfio/v

Quoted 33 lines
Quoted 33 lines
Quoted 33 lines
Quoted 33 lines
Quoted 33 lines
Quoted 33 lines
Quoted 33 lines
Quoted 33 lines

fio: Parameter 'driver' expects pluggable device type

Thanks,
Shannon

Quoted 27 lines · Alvise Rigo

Il 19/06/2014 13:11, Shannon Zhao ha scritto:

On 2014/6/19 18:47, Alvise Rigo wrote:
Hi,

Il 19/06/2014 11:38, Shannon Zhao ha scritto:
Hi Bharat,

On 2014/6/19 17:19, [email protected] wrote:

-----Original Message-----
From: [email protected]
[mailto:kvmarm- [email protected]] On
Behalf Of Shannon Zhao
Sent: Thursday, June 19, 2014 2:45 PM
To: Alvise Rigo; a.motakis-lrHrjnjw1UfHK3s98zE1ajGjJy/[email protected];
iommu-cunTk1MwBs/[email protected] foundation.org;
[email protected]
Subject: Re: Vfio-on-arm: the device tree does not
contain the DMA device

Hi Alvise,

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines

Thanks for your reply.
I have apply the patch and could see the DMA devices.
But there is no vosys_dmatest int the directory
"/sys/kernel/debug/" after mounting debugfs.
I found that the branch vfio-platform-v4 doesn't
contain the module vosys_dmatest.
Then I port it from the branch pl330_dmatest and run

vosys_dmatest

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines

successfully.

Quoted 5 lines

echo 1 > /sys/kernel/debug/vosys_dmatest/start

[ 6145.343419] dmaengine: __dma_request_channel:
success

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines

(dma0chan0)

Quoted 7 lines

[ 6145.344000] vosys dmatest: callback: transfer completed
[ 6145.344147] vosys dmatest: tx completed
[ 6145.344589] vosys dmatest: source and destination match.

Download the qemu and use following command to start a guest:
./qemu-system-arm -enable-kvm -M vexpress-a15 -cpu
cortex-

a15 \

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 1 line

-device

pl330,vfio_group="/dev/vfio/0",vfio_device="2c0a0000.dma",\

Quoted 2 lines

vfio_container="/dev/vfio/vfio" \
-append "earlyprintk ip=dhcp console=ttyAMA0 rootwait"

\

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

-kernel zImage -dtb guest.dtb -initrd

filesystem.cpio.gz

Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 6 lines

- nographic An error reported:
qemu-system-arm: -device
pl330,vfio_group=/dev/vfio/0,vfio_device=2c0a0000.dma,:

Why do we need to give "vfio_group=/dev/vfio/0" ?
I refer to the url

"http://www.virtualopensystems.com/en/solutions/guides/vfio-on-arm/".

Quoted 2 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines

could not open disk image vfio_container=/dev/vfio:
Could not open 'vfio_container=/dev/vfio': No such
file or directory

Are you unbinding "2c0a0000.dma" from its driver and
binding with

"vfio-

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

playform" driver ?

Quoted 3 lines

How to unbind "2c0a0000.dma" from its driver?
I have tried following cmd to bind "2c0a0000.dma" with
"vfio-

playform"

Quoted 25 lines
Quoted 25 lines
Quoted 25 lines
Quoted 25 lines
Quoted 25 lines
Quoted 25 lines
Quoted 25 lines
Quoted 25 lines

driver.

Quoted 24 lines

echo 2c0a0000.dma >
/sys/bus/platform/drivers/vfio-platform/vfio_bind
But there is no /dev/vfio/0 or /dev/vfio/vfio .

Can you please verify that while configuring the host you enabled:
- Device Drivers -> IOMMU Hardware Support -> ARM Ltd.
System MMU
(SMMU) Support and
- Device Drivers -> VFIO Non-Privileged userspace driver
framework
-> VFIO support for platform devices

Hi Alvise,

I have checked the two configuration are enabled.

Then try the following cmd, but another error reported:
qemu-system-arm: -device
pl330,vfio_group=/dev/vfio/0,vfio_device=2c0a0000.dma,vfio_
container=
/dev/vfio/vfio: Parameter 'driver' expects pluggable device
type

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

Do I need to unbind the "2c0a0000.dma" from its driver and
binding

with

Quoted 15 lines
Quoted 15 lines
Quoted 15 lines
Quoted 15 lines
Quoted 15 lines
Quoted 15 lines
Quoted 15 lines
Quoted 15 lines

"vfio-platform" driver as Bharat mentioned ?

Quoted 14 lines

Yes, the 2c0a0000.dma should not be bound to any driver at
boot, so you just need to:
echo 2c0a0000.dma >
/sys/bus/platform/drivers/vfio-platform/vfio_bind
to bind the device to the VFIO driver.

Regards,
alvise

Then try to update the pl330-vfio-dev branch and run QEMU
with the

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines

following

Quoted 5 lines
Quoted 5 lines

command:

Quoted 3 lines

./qemu-system-arm -enable-kvm -M vexpress-a15 -cpu cortex-a15 \
-device

pl330,vfio_group="/dev/vfio/0",vfio_device="2c0a0000.dma",vfio_contai ner="/dev/v

Quoted 18 lines
Quoted 18 lines
Quoted 18 lines
Quoted 18 lines
Quoted 18 lines
Quoted 18 lines
Quoted 18 lines
Quoted 18 lines

fio/vfio" \

Quoted 17 lines

-append "earlyprintk ip=dhcp console=ttyAMA0 rootwait" \
-kernel zImage -dtb guest.dtb -initrd
filesystem.cpio.gz

that means without breaking the "-device pl330" options.

Regards,
alvise

Thanks
-Bharat

Is there something I'm wrong or do I miss some steps?

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines

Thank you,
Shannon

On 2014/6/18 21:24, Alvise Rigo wrote:
Hi Shannon,

Thank you for reporting this issue, the device tree
file was

indeed

Quoted 181 lines
Quoted 181 lines
Quoted 181 lines
Quoted 180 lines
Quoted 179 lines
Quoted 170 lines
Quoted 170 lines
Quoted 168 lines

missing.

Quoted 164 lines

To solve the problem, you can apply the following
patch to the repository you mentioned
(git://github.com/virtualopensystems/arm-dts.git).

For your information, soon we will update the guide
with a newer version of VFIO with this issue fixed.

Thank you,
alvise

---
fast_models/rtsm_ve-cortex_a15x4_a7x4.dts | 100
+++++++++++++++++++++++++++++-
1 file changed, 97 insertions(+), 3 deletions(-)

diff --git a/fast_models/rtsm_ve-cortex_a15x4_a7x4.dts
b/fast_models/rtsm_ve-cortex_a15x4_a7x4.dts
index ef1275b..79c6af2 100644
--- a/fast_models/rtsm_ve-cortex_a15x4_a7x4.dts
+++ b/fast_models/rtsm_ve-cortex_a15x4_a7x4.dts
@@ -142,7 +142,7 @@
cpu7: cpu@7 {
device_type = "cpu";
compatible = "arm,cortex-a7";
- reg = <0x104>;
+ reg = <0x103>;
cluster = <&cluster1>;
core = <&core7>;
};
@@ -159,7 +159,7 @@
#address-cells = <0>;
interrupt-controller;
reg = <0 0x2c001000 0 0x1000>,
- <0 0x2c002000 0 0x2000>,
+ <0 0x2c002000 0 0x1000>,
<0 0x2c004000 0 0x2000>,
<0 0x2c006000 0 0x2000>;
interrupts = <1 9 0xf04>; @@ -183,6 +183,101 @@
<1 10 0xf08>;
};

+ smmu@2c0b0000 {
+ compatible = "arm,smmu-v2";
+ reg = <0x0 0x2c0b0000 0x0 0x10000>;
+ #global-interrupts = <2>;
+ interrupts = <0 58 4>,
+ <0 59 4>,
+ <0 60 4>,
+ <0 60 4>,
+ <0 60 4>,
+ <0 60 4>,
+ <0 60 4>,
+ <0 60 4>,
+ <0 60 4>;
+ mmu-masters = <&dma0 0x0 0x8>,
+ <&dma1 0x1 0x9>,
+ <&dma2 0x2 0xa>,
+ <&dma3 0x3 0xb>;
+ };
+
+ dma0: dma@2c0a0000 {
+ compatible = "arm,pl330";
+ reg = <0 0x2c0a0000 0 0x1000>;
+ interrupts = <0 64 4>,
+ <0 65 4>,
+ <0 66 4>,
+ <0 67 4>,
+ <0 68 4>,
+ <0 69 4>,
+ <0 70 4>,
+ <0 71 4>;
+ #dma-cells = <1>;
+ #stream-id-cells = <2>;
+ clocks = <&oscclk6a>;
+ clock-names = "apb_pclk";
+ };
+
+ dma1: dma@2c0a1000 {
+ compatible = "arm,pl330", "arm,primecell";
+ reg = <0 0x2c0a1000 0 0x1000>;
+ interrupts = <0 96 4>,
+ <0 97 4>,
+ <0 98 4>,
+ <0 99 4>,
+ <0 100 4>,
+ <0 101 4>,
+ <0 102 4>,
+ <0 103 4>;
+ #dma-cells = <1>;
+ #stream-id-cells = <2>;
+ clocks = <&oscclk6a>;
+ clock-names = "apb_pclk";
+ };
+
+ dma2: dma@2c0a2000 {
+ compatible = "arm,pl330", "arm,primecell";
+ reg = <0 0x2c0a2000 0 0x1000>;
+ interrupts = <0 128 4>,
+ <0 129 4>,
+ <0 130 4>,
+ <0 131 4>,
+ <0 132 4>,
+ <0 133 4>,
+ <0 134 4>,
+ <0 135 4>;
+ #dma-cells = <1>;
+ #stream-id-cells = <2>;
+ clocks = <&oscclk6a>;
+ clock-names = "apb_pclk";
+ };
+
+ dma3: dma@2c0a3000 {
+ compatible = "arm,pl330", "arm,primecell";
+ reg = <0 0x2c0a3000 0 0x1000>;
+ interrupts = <0 160 4>,
+ <0 161 4>,
+ <0 162 4>,
+ <0 163 4>,
+ <0 164 4>,
+ <0 165 4>,
+ <0 166 4>,
+ <0 167 4>;
+ #dma-cells = <1>;
+ #stream-id-cells = <2>;
+ clocks = <&oscclk6a>;
+ clock-names = "apb_pclk";
+ };
+
+ oscclk6a: oscclk6a {
+ /* Reference 24MHz clock */
+ compatible = "fixed-clock";
+ #clock-cells = <0>;
+ clock-frequency = <24000000>;
+ clock-output-names = "oscclk6a";
+ };
+
smb {
compatible = "simple-bus";

@@ -244,4 +339,3 @@
/include/ "rtsm_ve-motherboard.dtsi"
};
};
-

--
Shannon

_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm

.

.

.

--
Shannon

Quoted 5 lines
Quoted 2 lines

.

--
Shannon

.

Quoted 6 lines
Quoted 3 lines
Quoted 2 lines

.

.

--
Shannon

--
Shannon

Signature · 1 line
Shannon
#51 lore raw
In reply to a message not in the archive

This is a QEMU error. Are you using the pl330-vfio-dev branch? Is it up-to-date?

alvise

Il 24/06/2014 11:00, Shannon Zhao ha scritto:

Quoted 657 lines

Hi Bharat,

ls -l /sys/bus/platform/devices/2c0a0000.dma/iommu_group/devices
total 0
lrwxrwxrwx 1 root root 0 Jan 1 02:58 2c0a0000.dma -> ../../../../devices/2c0a0000.dma

Then I run "echo 2c0a0000.dma > /sys/bus/platform/drivers/vfio-platform/bind". There is /dev/vfio/0.

But there is also an error reported.

	-nographic can not be used with -daemonize
	char device redirected to /dev/pts/3 (label pty0-base)
	PL330_VFIO: === VFIO device file descriptor 13 ===
	Device has 1 region(s):
	PL330_VFIO: - Region 0: size=0x1000 offset=0x0 flags=0x7
	PL330_VFIO: - IRQ 0: range of 1, flags=0x7
	PL330_VFIO: - IRQ 1: range of 1, flags=0x7
	PL330_VFIO: - IRQ 2: range of 1, flags=0x7
	PL330_VFIO: - IRQ 3: range of 1, flags=0x7
	PL330_VFIO: - IRQ 4: range of 1, flags=0x7
	qemu-system-arm: -device pl330,vfio_group=/dev/vfio/0,vfio_device=2c0a0000.dma,vfio_container=/dev/vfio/vfio: Property '.vfio_group' not found

Thanks,
Shannon

Quoted 630 lines · On 2014/6/24 16:08, [email protected]…
Quoted 10 lines

-----Original Message-----
From: Shannon Zhao [mailto:[email protected]]
Sent: Tuesday, June 24, 2014 1:16 PM
To: Bhushan Bharat-R65777; Alvise Rigo; a.motakis-lrHrjnjw1UfHK3s98zE1ajGjJy/[email protected]; iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org; [email protected]; [email protected]
Subject: Re: Vfio-on-arm: the device tree does not contain the DMA device

Quoted 7 lines

Hi Bharat,

Thanks for your help.

I executed the following cmd:
echo vfio-platform >
/sys/bus/platform/devices/2c0a0000.dma/driver_override

You have to bind the device with vfio-platform driver after the above command.

Quoted 5 lines

But there is no /dev/vfio/0 and error reported starting guest

-nographic can not be used with -daemonize
char device redirected to /dev/pts/3 (label pty0-base)
PL330_VFIO: Group is not viable (not all devices bound for vfio)

A bit surprised; it should be failing when opening /dev/vfio/0 but it is failing in VFIO_GROUP_GET_STATUS ioctl.

Also looks like there are more device attached to the said iommu-group. Can check by running below command:
#ls -l /sys/bus/pci/devices/2c0a0000.dma/iommu_group/devices

Thanks
-Bharat

Quoted 594 lines

qemu-system-arm: Error while probing real device.

qemu-system-arm: Initialization of device pl330 failed Is there something
wrong?

Thank you very much.
Shannon

Quoted 93 lines · On 2014/6/24 15:05, [email protected]…
Quoted 93 lines

-----Original Message-----
From: Shannon Zhao [mailto:[email protected]]
Sent: Tuesday, June 24, 2014 12:30 PM
To: Alvise Rigo; Bhushan Bharat-R65777;
a.motakis-lrHrjnjw1UfHK3s98zE1ajGjJy/[email protected]; iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org; [email protected]; [email protected]
Subject: Re: Vfio-on-arm: the device tree does not contain the DMA
device

Hi Alvise,

Thanks for your help.
Now I can see /dev/vfio/vfio and /dev/vfio/0 after executing "echo 2c0a0000.dma

Quoted 1 line

/sys/bus/platform/drivers/vfio-platform/vfio_bind".

But when using qemu to start guest, there is an error :

-nographic can not be used with -daemonize char device redirected to /dev/pts/3 (label pty0-base)
PL330_VFIO: === VFIO device file descriptor 13 === Device has 1 region(s):
PL330_VFIO: - Region 0: size=0x1000 offset=0x2c0a0000 flags=0x7
PL330_VFIO: - IRQ 0: range of 1, flags=0x7 ------------[ cut here ]------------ kernel BUG at mm/slub.c:3386!

Internal error: Oops - BUG: 0 [#1] SMP ARM Modules linked in:
CPU: 2 PID: 1746 Comm: qemu-system-arm Tainted: G        W    3.14.0-rc1+ #19
task: ee665580 ti: ed0b8000 task.ti: ed0b8000 PC is at
kfree+0x128/0x134 LR is at vfio_set_trigger.isra.5+0x40/0x104
pc : [<800d9e3c>]    lr : [<80319094>]    psr: 400f0013

sp : ed0b9e64 ip : 000800f3 fp : 000014ac
r10: 00000000 r9 : ed0b8000 r8 : edeb3b80
r7 : 0000000e r6 : 80620584 r5 : 00000000 r4 : eea0b000 r3 : 00000400 r2 : 800f38b0 r1 : eea0ce60 r0 : 00000400
Flags: nZcv IRQs on FIQs on Mode SVC_32 ISA ARM Segment user
Control: 30c53c7d Table: eddac080 DAC: 00000000 Process qemu-system-arm (pid: 1746, stack limit = 0xed0b8240)

Stack: (0xed0b9e64 to 0xed0ba000)

9e60: ee603700 00000000 ee603700 0000000e edeb3b80 00000000 80319094
9e80: ee603700 00003b6e 00000000 edeb3b80 ee603700 00003b6e 0000000d 80319208
9ea0: 0000000e edeb3b80 80492a18 edeb3b80 ee57f780 80319690 00000024 008b2074
9ec0: 008b2074 8031886c 00000001 008b2074 00000018 00000024 00000000 00000000
9ee0: 00000001 00000004 00000001 00000001 ee088e70 008b2060 ee57f780 803155d0
9f00: 803155b0 800ef430 80477a58 00080802 ed0b8000 800e1358 00000020 ed0b9f50
9f20: ed0b9f60 00000003 8061e808 00080802 ed0b9f60 800e14f0 00000000 806522f8
9f40: 80477a58 8061e808 ede4c1c0 8011af18 00000000 00000009 8053bbc0 00000000
9f60: ee683610 00000001 ee57f780 008b2060 00003b6e 0000000d ed0b8000 00000000
9f80: 000014ac 800ef9ac 0000000e 00000001 008b2060 00429eb0 0000000d 00000036
9fa0: 8001c464 8001c2e0 008b2060 00429eb0 0000000d 00003b6e 008b2060 00000001
9fc0: 008b2060 00429eb0 0000000d 00000036 00000000 7ea99040 0089a220 000014ac
9fe0: 003fa474 7ea98fc4 000bc5c8 76ceb2ec 200f0010 0000000d 00000000 8062c770 [<800d9e3c>] (kfree) from [<80319094>] (vfio_set_trigger.isra.5+0x40/0x104)
[<80319094>] (vfio_set_trigger.isra.5) from [<80319208>] (vfio_platform_set_irq_trigger+0xb0/0x144)
[<80319208>] (vfio_platform_set_irq_trigger) from [<80319690>] (vfio_platform_set_irqs_ioctl+0x44/0x58)
[<80319690>] (vfio_platform_set_irqs_ioctl) from [<8031886c>] (vfio_platform_ioctl+0x2e0/0x314)
[<8031886c>] (vfio_platform_ioctl) from [<803155d0>] (vfio_device_fops_unl_ioctl+0x20/0x2c)
[<803155d0>] (vfio_device_fops_unl_ioctl) from [<800ef430>] (do_vfs_ioctl+0x78/0x5b8)
[<800ef430>] (do_vfs_ioctl) from [<800ef9ac>] (SyS_ioctl+0x3c/0x5c) [<800ef9ac>] (SyS_ioctl) from [<8001c2e0>] (ret_fast_syscall+0x0/0x30)

Code: 15913038 e1a01003 e8bd45f0 eaff28e5 (e7f001f2)

I'm using branch vfio-platform-v4 for kernel and pl330-vfio-dev for qemu. Following is qemu cmdline:

	./qemu-system-arm -enable-kvm -M vexpress-a15 -cpu cortex-a15 \
       	-chardev pty,id=pty0,mux=on -monitor chardev:pty0 \
       	-serial chardev:pty0 -daemonize \
       	-device

pl330,vfio_group="/dev/vfio/0",vfio_device="2c0a0000.dma",vfio_contai ner="/dev/v
fio/vfio" \
-append "earlyprintk ip=dhcp console=ttyAMA0 rootwait" \
-kernel zImage -dtb guest-a15.dtb -initrd filesystem.cpio.gz
- nographic

Quoted 4 lines
Quoted 2 lines

If I change to branch vfio-platform-v6 for kernel, there is "/dev/vfio/vfio". But no "/sys/bus/platform/drivers/vfio-platform/vfio_bind".

Can you Check if you have driver_override at

"/sys/bus/platform/devices/DEVICE-XXX/ ?

Quoted 1 line

If yes then you should use driver_override mechanism to bind that device to

vfio-platform driver.

Quoted 482 lines

Thanks
-Bharat

Quoted 477 lines

And if I use above qemu cmdline to start guest, there is also an error :
PL330_VFIO: Group is not viable (not all devices bound for vfio)
qemu-system-arm: Error while probing real device.

qemu-system-arm: Initialization of device pl330 failed

The usage about v4 and v6 has changed ?
And would you tell me when you will update the guideline about vfio-on-arm?

Thanks,
Shannon

Quoted 12 lines · Alvise Rigo

Linaro has a great set of pre-built images here http://releases.linaro.org/.

For example these should work fine:
http://releases.linaro.org/13.12/ubuntu/saucy-images

alvise

Il 20/06/2014 10:23, Shannon Zhao ha scritto:

Quoted 4 lines

Hi Alvise,

I'm so sorry that I can't use qemu-debootstrap because my environment can't

access some links using proxy.

Quoted 14 lines
Quoted 14 lines

Are there other ways to get a proper fs?

Thanks
Shannon

Quoted 8 lines · Alvise Rigo

Hi Shannon,

If you follow the steps in the section "Creating the File System" of the guide you should be able to obtain a working fs.

Regards,
alvise

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines

Il 20/06/2014 04:57, Shannon Zhao ha scritto:

Quoted 5 lines

Hi Bharat,

Thanks for your advice .

I'm using the rootfs getting from the url

"http://www.virtualopensystems.com/downloads/guides/kvm_on_arm/fs-ali p-
armel.cramfs".

Quoted 40 lines
Quoted 40 lines
Quoted 40 lines
Quoted 40 lines

Can you give me another link to get a newer rootfs?

Thank you
Shannon

Quoted 33 lines · On 2014/6/19 20:20, [email protected]…
Quoted 33 lines

-----Original Message-----
From: Shannon Zhao [mailto:[email protected]]
Sent: Thursday, June 19, 2014 5:47 PM
To: Bhushan Bharat-R65777; Alvise Rigo;
a.motakis-lrHrjnjw1UfHK3s98zE1ajGjJy/[email protected]; iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org; [email protected]
Subject: Re: Vfio-on-arm: the device tree does not contain the
DMA device

Quoted 19 lines · On 2014/6/19 20:12, [email protected]…
Quoted 17 lines

-----Original Message-----
From: Shannon Zhao [mailto:[email protected]]
Sent: Thursday, June 19, 2014 5:38 PM
To: Alvise Rigo; Bhushan Bharat-R65777;
a.motakis-lrHrjnjw1UfHK3s98zE1ajGjJy/[email protected];
iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org;
[email protected]
Subject: Re: Vfio-on-arm: the device tree does not contain
the DMA device

Hi Alvise,

I run following cmds :
echo 2c0a0000.dma >
/sys/bus/platform/drivers/vfio-platform/vfio_bind

run "ls /dev/vfio/" before and after the above command

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 5 lines

Hi Bharat,

Nothing appeared either before or after the cmd "echo 2c0a0000.dma > /sys/bus/platform/drivers/vfio-platform/vfio_bind". There is only /dev/vfio.

I am not sure, but may be you are using pretty old rootfs with broken udev

rules. Try to fetch a newer rootfs form somewhere.

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines

Thanks
-Bharat

Quoted 1 line
Quoted 1 line
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 7 lines

Thanks
-Bharat

Quoted 3 lines

./qemu-system-arm -enable-kvm -M vexpress-a15 -cpu cortex-a15 \
-device

pl330,vfio_group="/dev/vfio/0",vfio_device="2c0a0000.dma",vfio_contai ner="/dev/v

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

fio/vfio" \
-append "earlyprintk ip=dhcp console=ttyAMA0 rootwait" \
-kernel zImage -dtb guest-a15.dtb -initrd
filesystem.cpio.gz

-

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 3 lines
Quoted 3 lines

nographic But the error still exists.
qemu-system-arm: -device

pl330,vfio_group=/dev/vfio/0,vfio_device=2c0a0000.dma,vfio_container= /dev/vfio/v

Quoted 33 lines
Quoted 33 lines
Quoted 33 lines
Quoted 33 lines
Quoted 33 lines
Quoted 33 lines
Quoted 33 lines
Quoted 33 lines

fio: Parameter 'driver' expects pluggable device type

Thanks,
Shannon

On 2014/6/19 19:26, Alvise Rigo wrote:

Il 19/06/2014 13:11, Shannon Zhao ha scritto:

On 2014/6/19 18:47, Alvise Rigo wrote:
Hi,

Il 19/06/2014 11:38, Shannon Zhao ha scritto:
Hi Bharat,

On 2014/6/19 17:19, [email protected] wrote:

-----Original Message-----
From: [email protected]
[mailto:kvmarm- [email protected]] On
Behalf Of Shannon Zhao
Sent: Thursday, June 19, 2014 2:45 PM
To: Alvise Rigo; a.motakis-lrHrjnjw1UfHK3s98zE1ajGjJy/[email protected];
iommu-cunTk1MwBs/[email protected] foundation.org;
[email protected]
Subject: Re: Vfio-on-arm: the device tree does not
contain the DMA device

Hi Alvise,

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines

Thanks for your reply.
I have apply the patch and could see the DMA devices.
But there is no vosys_dmatest int the directory
"/sys/kernel/debug/" after mounting debugfs.
I found that the branch vfio-platform-v4 doesn't
contain the module vosys_dmatest.
Then I port it from the branch pl330_dmatest and run

vosys_dmatest

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines

successfully.

echo 1 > /sys/kernel/debug/vosys_dmatest/start

[ 6145.343419] dmaengine: __dma_request_channel:
success

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines

(dma0chan0)
[ 6145.344000] vosys dmatest: callback: transfer completed
[ 6145.344147] vosys dmatest: tx completed
[ 6145.344589] vosys dmatest: source and destination match.

Download the qemu and use following command to start a guest:
./qemu-system-arm -enable-kvm -M vexpress-a15 -cpu
cortex-

a15 \

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

-device
pl330,vfio_group="/dev/vfio/0",vfio_device="2c0a0000.dma",\
vfio_container="/dev/vfio/vfio" \
-append "earlyprintk ip=dhcp console=ttyAMA0 rootwait"

\

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

-kernel zImage -dtb guest.dtb -initrd

filesystem.cpio.gz

Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines

- nographic An error reported:
qemu-system-arm: -device
pl330,vfio_group=/dev/vfio/0,vfio_device=2c0a0000.dma,:

Why do we need to give "vfio_group=/dev/vfio/0" ?
I refer to the url
"http://www.virtualopensystems.com/en/solutions/guides/vfio-on-arm/".

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines

could not open disk image vfio_container=/dev/vfio:
Could not open 'vfio_container=/dev/vfio': No such
file or directory

Are you unbinding "2c0a0000.dma" from its driver and
binding with

"vfio-

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

playform" driver ?
How to unbind "2c0a0000.dma" from its driver?
I have tried following cmd to bind "2c0a0000.dma" with
"vfio-

playform"

Quoted 25 lines
Quoted 25 lines
Quoted 25 lines
Quoted 25 lines
Quoted 25 lines
Quoted 25 lines
Quoted 25 lines
Quoted 25 lines

driver.
echo 2c0a0000.dma >
/sys/bus/platform/drivers/vfio-platform/vfio_bind
But there is no /dev/vfio/0 or /dev/vfio/vfio .

Can you please verify that while configuring the host you enabled:
- Device Drivers -> IOMMU Hardware Support -> ARM Ltd.
System MMU
(SMMU) Support and
- Device Drivers -> VFIO Non-Privileged userspace driver
framework
-> VFIO support for platform devices

Hi Alvise,

I have checked the two configuration are enabled.

Then try the following cmd, but another error reported:
qemu-system-arm: -device
pl330,vfio_group=/dev/vfio/0,vfio_device=2c0a0000.dma,vfio_
container=
/dev/vfio/vfio: Parameter 'driver' expects pluggable device
type

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

Do I need to unbind the "2c0a0000.dma" from its driver and
binding

with

Quoted 15 lines
Quoted 15 lines
Quoted 15 lines
Quoted 15 lines
Quoted 15 lines
Quoted 15 lines
Quoted 15 lines
Quoted 15 lines

"vfio-platform" driver as Bharat mentioned ?

Yes, the 2c0a0000.dma should not be bound to any driver at
boot, so you just need to:
echo 2c0a0000.dma >
/sys/bus/platform/drivers/vfio-platform/vfio_bind
to bind the device to the VFIO driver.

Regards,
alvise

Then try to update the pl330-vfio-dev branch and run QEMU
with the

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines

following

Quoted 5 lines
Quoted 5 lines

command:

./qemu-system-arm -enable-kvm -M vexpress-a15 -cpu cortex-a15 \
-device

pl330,vfio_group="/dev/vfio/0",vfio_device="2c0a0000.dma",vfio_contai ner="/dev/v

Quoted 18 lines
Quoted 18 lines
Quoted 18 lines
Quoted 18 lines
Quoted 18 lines
Quoted 18 lines
Quoted 18 lines
Quoted 18 lines

fio/vfio" \
-append "earlyprintk ip=dhcp console=ttyAMA0 rootwait" \
-kernel zImage -dtb guest.dtb -initrd
filesystem.cpio.gz

that means without breaking the "-device pl330" options.

Regards,
alvise

Thanks
-Bharat

Is there something I'm wrong or do I miss some steps?

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines

Thank you,
Shannon

On 2014/6/18 21:24, Alvise Rigo wrote:
Hi Shannon,

Thank you for reporting this issue, the device tree
file was

indeed

Quoted 181 lines
Quoted 181 lines
Quoted 181 lines
Quoted 180 lines
Quoted 179 lines
Quoted 170 lines
Quoted 170 lines
Quoted 168 lines

missing.
To solve the problem, you can apply the following
patch to the repository you mentioned
(git://github.com/virtualopensystems/arm-dts.git).

For your information, soon we will update the guide
with a newer version of VFIO with this issue fixed.

Thank you,
alvise

---
fast_models/rtsm_ve-cortex_a15x4_a7x4.dts | 100
+++++++++++++++++++++++++++++-
1 file changed, 97 insertions(+), 3 deletions(-)

diff --git a/fast_models/rtsm_ve-cortex_a15x4_a7x4.dts
b/fast_models/rtsm_ve-cortex_a15x4_a7x4.dts
index ef1275b..79c6af2 100644
--- a/fast_models/rtsm_ve-cortex_a15x4_a7x4.dts
+++ b/fast_models/rtsm_ve-cortex_a15x4_a7x4.dts
@@ -142,7 +142,7 @@
cpu7: cpu@7 {
device_type = "cpu";
compatible = "arm,cortex-a7";
- reg = <0x104>;
+ reg = <0x103>;
cluster = <&cluster1>;
core = <&core7>;
};
@@ -159,7 +159,7 @@
#address-cells = <0>;
interrupt-controller;
reg = <0 0x2c001000 0 0x1000>,
- <0 0x2c002000 0 0x2000>,
+ <0 0x2c002000 0 0x1000>,
<0 0x2c004000 0 0x2000>,
<0 0x2c006000 0 0x2000>;
interrupts = <1 9 0xf04>; @@ -183,6 +183,101 @@
<1 10 0xf08>;
};

+ smmu@2c0b0000 {
+ compatible = "arm,smmu-v2";
+ reg = <0x0 0x2c0b0000 0x0 0x10000>;
+ #global-interrupts = <2>;
+ interrupts = <0 58 4>,
+ <0 59 4>,
+ <0 60 4>,
+ <0 60 4>,
+ <0 60 4>,
+ <0 60 4>,
+ <0 60 4>,
+ <0 60 4>,
+ <0 60 4>;
+ mmu-masters = <&dma0 0x0 0x8>,
+ <&dma1 0x1 0x9>,
+ <&dma2 0x2 0xa>,
+ <&dma3 0x3 0xb>;
+ };
+
+ dma0: dma@2c0a0000 {
+ compatible = "arm,pl330";
+ reg = <0 0x2c0a0000 0 0x1000>;
+ interrupts = <0 64 4>,
+ <0 65 4>,
+ <0 66 4>,
+ <0 67 4>,
+ <0 68 4>,
+ <0 69 4>,
+ <0 70 4>,
+ <0 71 4>;
+ #dma-cells = <1>;
+ #stream-id-cells = <2>;
+ clocks = <&oscclk6a>;
+ clock-names = "apb_pclk";
+ };
+
+ dma1: dma@2c0a1000 {
+ compatible = "arm,pl330", "arm,primecell";
+ reg = <0 0x2c0a1000 0 0x1000>;
+ interrupts = <0 96 4>,
+ <0 97 4>,
+ <0 98 4>,
+ <0 99 4>,
+ <0 100 4>,
+ <0 101 4>,
+ <0 102 4>,
+ <0 103 4>;
+ #dma-cells = <1>;
+ #stream-id-cells = <2>;
+ clocks = <&oscclk6a>;
+ clock-names = "apb_pclk";
+ };
+
+ dma2: dma@2c0a2000 {
+ compatible = "arm,pl330", "arm,primecell";
+ reg = <0 0x2c0a2000 0 0x1000>;
+ interrupts = <0 128 4>,
+ <0 129 4>,
+ <0 130 4>,
+ <0 131 4>,
+ <0 132 4>,
+ <0 133 4>,
+ <0 134 4>,
+ <0 135 4>;
+ #dma-cells = <1>;
+ #stream-id-cells = <2>;
+ clocks = <&oscclk6a>;
+ clock-names = "apb_pclk";
+ };
+
+ dma3: dma@2c0a3000 {
+ compatible = "arm,pl330", "arm,primecell";
+ reg = <0 0x2c0a3000 0 0x1000>;
+ interrupts = <0 160 4>,
+ <0 161 4>,
+ <0 162 4>,
+ <0 163 4>,
+ <0 164 4>,
+ <0 165 4>,
+ <0 166 4>,
+ <0 167 4>;
+ #dma-cells = <1>;
+ #stream-id-cells = <2>;
+ clocks = <&oscclk6a>;
+ clock-names = "apb_pclk";
+ };
+
+ oscclk6a: oscclk6a {
+ /* Reference 24MHz clock */
+ compatible = "fixed-clock";
+ #clock-cells = <0>;
+ clock-frequency = <24000000>;
+ clock-output-names = "oscclk6a";
+ };
+
smb {
compatible = "simple-bus";

@@ -244,4 +339,3 @@
/include/ "rtsm_ve-motherboard.dtsi"
};
};
-

--
Shannon

_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm

.

.

.

--
Shannon

Quoted 5 lines
Quoted 2 lines

.

--
Shannon

.

Quoted 6 lines
Quoted 3 lines
Quoted 2 lines

.

.

--
Shannon

--
Shannon

#53 lore raw
In reply to a message not in the archive

Hi Alvise,

Thanks for your help.
I'm using pl330-vfio-dev branch but it's not up-to-date. Then I update the qemu code.
Now I can start up the guest.

I run the following cmd in guest :

echo 1 > /sys/kernel/debug/vosys_dmatest/start
dmaengine: dma_sync_wait: timeout! vosys dmatest: tx error
vosys dmatest: src and destination do not match
dmesg | grep vosys
vosys dmatest: vosys dmatest module init vosys dmatest: tx error
vosys dmatest: tx completed
And at the terminal running isim_system there is an error :

ERROR: DMAC330 channel thread abort.
ERROR: DMAC330: Instruction fetch ERROR, thread set to FAULTING state

Is there something wrong?

Thanks,
Shannon

Quoted 666 lines · Alvise Rigo

This is a QEMU error. Are you using the pl330-vfio-dev branch? Is it up-to-date?

alvise

Il 24/06/2014 11:00, Shannon Zhao ha scritto:

Quoted 23 lines

Hi Bharat,

ls -l /sys/bus/platform/devices/2c0a0000.dma/iommu_group/devices
total 0
lrwxrwxrwx 1 root root 0 Jan 1 02:58 2c0a0000.dma -> ../../../../devices/2c0a0000.dma

Then I run "echo 2c0a0000.dma > /sys/bus/platform/drivers/vfio-platform/bind". There is /dev/vfio/0.

But there is also an error reported.

	-nographic can not be used with -daemonize
	char device redirected to /dev/pts/3 (label pty0-base)
	PL330_VFIO: === VFIO device file descriptor 13 ===
	Device has 1 region(s):
	PL330_VFIO: - Region 0: size=0x1000 offset=0x0 flags=0x7
	PL330_VFIO: - IRQ 0: range of 1, flags=0x7
	PL330_VFIO: - IRQ 1: range of 1, flags=0x7
	PL330_VFIO: - IRQ 2: range of 1, flags=0x7
	PL330_VFIO: - IRQ 3: range of 1, flags=0x7
	PL330_VFIO: - IRQ 4: range of 1, flags=0x7
	qemu-system-arm: -device pl330,vfio_group=/dev/vfio/0,vfio_device=2c0a0000.dma,vfio_container=/dev/vfio/vfio: Property '.vfio_group' not found
Quoted 634 lines

Thanks,
Shannon

Quoted 630 lines · On 2014/6/24 16:08, [email protected]…
Quoted 10 lines

-----Original Message-----
From: Shannon Zhao [mailto:[email protected]]
Sent: Tuesday, June 24, 2014 1:16 PM
To: Bhushan Bharat-R65777; Alvise Rigo; a.motakis-lrHrjnjw1UfHK3s98zE1ajGjJy/[email protected]; iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org; [email protected]; [email protected]
Subject: Re: Vfio-on-arm: the device tree does not contain the DMA device

Quoted 7 lines

Hi Bharat,

Thanks for your help.

I executed the following cmd:
echo vfio-platform >
/sys/bus/platform/devices/2c0a0000.dma/driver_override

You have to bind the device with vfio-platform driver after the above command.

Quoted 5 lines

But there is no /dev/vfio/0 and error reported starting guest

-nographic can not be used with -daemonize
char device redirected to /dev/pts/3 (label pty0-base)
PL330_VFIO: Group is not viable (not all devices bound for vfio)

A bit surprised; it should be failing when opening /dev/vfio/0 but it is failing in VFIO_GROUP_GET_STATUS ioctl.

Also looks like there are more device attached to the said iommu-group. Can check by running below command:
#ls -l /sys/bus/pci/devices/2c0a0000.dma/iommu_group/devices

Thanks
-Bharat

Quoted 594 lines

qemu-system-arm: Error while probing real device.

qemu-system-arm: Initialization of device pl330 failed Is there something
wrong?

Thank you very much.
Shannon

Quoted 93 lines · On 2014/6/24 15:05, [email protected]…
Quoted 93 lines

-----Original Message-----
From: Shannon Zhao [mailto:[email protected]]
Sent: Tuesday, June 24, 2014 12:30 PM
To: Alvise Rigo; Bhushan Bharat-R65777;
a.motakis-lrHrjnjw1UfHK3s98zE1ajGjJy/[email protected]; iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org; [email protected]; [email protected]
Subject: Re: Vfio-on-arm: the device tree does not contain the DMA
device

Hi Alvise,

Thanks for your help.
Now I can see /dev/vfio/vfio and /dev/vfio/0 after executing "echo 2c0a0000.dma

Quoted 1 line

/sys/bus/platform/drivers/vfio-platform/vfio_bind".

But when using qemu to start guest, there is an error :

-nographic can not be used with -daemonize char device redirected to /dev/pts/3 (label pty0-base)
PL330_VFIO: === VFIO device file descriptor 13 === Device has 1 region(s):
PL330_VFIO: - Region 0: size=0x1000 offset=0x2c0a0000 flags=0x7
PL330_VFIO: - IRQ 0: range of 1, flags=0x7 ------------[ cut here ]------------ kernel BUG at mm/slub.c:3386!

Internal error: Oops - BUG: 0 [#1] SMP ARM Modules linked in:
CPU: 2 PID: 1746 Comm: qemu-system-arm Tainted: G        W    3.14.0-rc1+ #19
task: ee665580 ti: ed0b8000 task.ti: ed0b8000 PC is at
kfree+0x128/0x134 LR is at vfio_set_trigger.isra.5+0x40/0x104
pc : [<800d9e3c>]    lr : [<80319094>]    psr: 400f0013

sp : ed0b9e64 ip : 000800f3 fp : 000014ac
r10: 00000000 r9 : ed0b8000 r8 : edeb3b80
r7 : 0000000e r6 : 80620584 r5 : 00000000 r4 : eea0b000 r3 : 00000400 r2 : 800f38b0 r1 : eea0ce60 r0 : 00000400
Flags: nZcv IRQs on FIQs on Mode SVC_32 ISA ARM Segment user
Control: 30c53c7d Table: eddac080 DAC: 00000000 Process qemu-system-arm (pid: 1746, stack limit = 0xed0b8240)

Stack: (0xed0b9e64 to 0xed0ba000)

9e60: ee603700 00000000 ee603700 0000000e edeb3b80 00000000 80319094
9e80: ee603700 00003b6e 00000000 edeb3b80 ee603700 00003b6e 0000000d 80319208
9ea0: 0000000e edeb3b80 80492a18 edeb3b80 ee57f780 80319690 00000024 008b2074
9ec0: 008b2074 8031886c 00000001 008b2074 00000018 00000024 00000000 00000000
9ee0: 00000001 00000004 00000001 00000001 ee088e70 008b2060 ee57f780 803155d0
9f00: 803155b0 800ef430 80477a58 00080802 ed0b8000 800e1358 00000020 ed0b9f50
9f20: ed0b9f60 00000003 8061e808 00080802 ed0b9f60 800e14f0 00000000 806522f8
9f40: 80477a58 8061e808 ede4c1c0 8011af18 00000000 00000009 8053bbc0 00000000
9f60: ee683610 00000001 ee57f780 008b2060 00003b6e 0000000d ed0b8000 00000000
9f80: 000014ac 800ef9ac 0000000e 00000001 008b2060 00429eb0 0000000d 00000036
9fa0: 8001c464 8001c2e0 008b2060 00429eb0 0000000d 00003b6e 008b2060 00000001
9fc0: 008b2060 00429eb0 0000000d 00000036 00000000 7ea99040 0089a220 000014ac
9fe0: 003fa474 7ea98fc4 000bc5c8 76ceb2ec 200f0010 0000000d 00000000 8062c770 [<800d9e3c>] (kfree) from [<80319094>] (vfio_set_trigger.isra.5+0x40/0x104)
[<80319094>] (vfio_set_trigger.isra.5) from [<80319208>] (vfio_platform_set_irq_trigger+0xb0/0x144)
[<80319208>] (vfio_platform_set_irq_trigger) from [<80319690>] (vfio_platform_set_irqs_ioctl+0x44/0x58)
[<80319690>] (vfio_platform_set_irqs_ioctl) from [<8031886c>] (vfio_platform_ioctl+0x2e0/0x314)
[<8031886c>] (vfio_platform_ioctl) from [<803155d0>] (vfio_device_fops_unl_ioctl+0x20/0x2c)
[<803155d0>] (vfio_device_fops_unl_ioctl) from [<800ef430>] (do_vfs_ioctl+0x78/0x5b8)
[<800ef430>] (do_vfs_ioctl) from [<800ef9ac>] (SyS_ioctl+0x3c/0x5c) [<800ef9ac>] (SyS_ioctl) from [<8001c2e0>] (ret_fast_syscall+0x0/0x30)

Code: 15913038 e1a01003 e8bd45f0 eaff28e5 (e7f001f2)

I'm using branch vfio-platform-v4 for kernel and pl330-vfio-dev for qemu. Following is qemu cmdline:

	./qemu-system-arm -enable-kvm -M vexpress-a15 -cpu cortex-a15 \
       	-chardev pty,id=pty0,mux=on -monitor chardev:pty0 \
       	-serial chardev:pty0 -daemonize \
       	-device

pl330,vfio_group="/dev/vfio/0",vfio_device="2c0a0000.dma",vfio_contai ner="/dev/v
fio/vfio" \
-append "earlyprintk ip=dhcp console=ttyAMA0 rootwait" \
-kernel zImage -dtb guest-a15.dtb -initrd filesystem.cpio.gz
- nographic

Quoted 4 lines
Quoted 2 lines

If I change to branch vfio-platform-v6 for kernel, there is "/dev/vfio/vfio". But no "/sys/bus/platform/drivers/vfio-platform/vfio_bind".

Can you Check if you have driver_override at

"/sys/bus/platform/devices/DEVICE-XXX/ ?

Quoted 1 line

If yes then you should use driver_override mechanism to bind that device to

vfio-platform driver.

Quoted 482 lines

Thanks
-Bharat

Quoted 477 lines

And if I use above qemu cmdline to start guest, there is also an error :
PL330_VFIO: Group is not viable (not all devices bound for vfio)
qemu-system-arm: Error while probing real device.

qemu-system-arm: Initialization of device pl330 failed

The usage about v4 and v6 has changed ?
And would you tell me when you will update the guideline about vfio-on-arm?

Thanks,
Shannon

Quoted 12 lines · Alvise Rigo

Linaro has a great set of pre-built images here http://releases.linaro.org/.

For example these should work fine:
http://releases.linaro.org/13.12/ubuntu/saucy-images

alvise

Il 20/06/2014 10:23, Shannon Zhao ha scritto:

Quoted 4 lines

Hi Alvise,

I'm so sorry that I can't use qemu-debootstrap because my environment can't

access some links using proxy.

Quoted 14 lines
Quoted 14 lines

Are there other ways to get a proper fs?

Thanks
Shannon

Quoted 8 lines · Alvise Rigo

Hi Shannon,

If you follow the steps in the section "Creating the File System" of the guide you should be able to obtain a working fs.

Regards,
alvise

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines

Il 20/06/2014 04:57, Shannon Zhao ha scritto:

Quoted 5 lines

Hi Bharat,

Thanks for your advice .

I'm using the rootfs getting from the url

"http://www.virtualopensystems.com/downloads/guides/kvm_on_arm/fs-ali p-
armel.cramfs".

Quoted 40 lines
Quoted 40 lines
Quoted 40 lines
Quoted 40 lines

Can you give me another link to get a newer rootfs?

Thank you
Shannon

Quoted 33 lines · On 2014/6/19 20:20, [email protected]…
Quoted 33 lines

-----Original Message-----
From: Shannon Zhao [mailto:[email protected]]
Sent: Thursday, June 19, 2014 5:47 PM
To: Bhushan Bharat-R65777; Alvise Rigo;
a.motakis-lrHrjnjw1UfHK3s98zE1ajGjJy/[email protected]; iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org; [email protected]
Subject: Re: Vfio-on-arm: the device tree does not contain the
DMA device

Quoted 19 lines · On 2014/6/19 20:12, [email protected]…

-----Original Message-----
From: Shannon Zhao [mailto:[email protected]]
Sent: Thursday, June 19, 2014 5:38 PM
To: Alvise Rigo; Bhushan Bharat-R65777;
a.motakis-lrHrjnjw1UfHK3s98zE1ajGjJy/[email protected];
iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org;
[email protected]
Subject: Re: Vfio-on-arm: the device tree does not contain
the DMA device

Hi Alvise,

I run following cmds :
echo 2c0a0000.dma >
/sys/bus/platform/drivers/vfio-platform/vfio_bind

run "ls /dev/vfio/" before and after the above command

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 5 lines

Hi Bharat,

Nothing appeared either before or after the cmd "echo 2c0a0000.dma > /sys/bus/platform/drivers/vfio-platform/vfio_bind". There is only /dev/vfio.

I am not sure, but may be you are using pretty old rootfs with broken udev

rules. Try to fetch a newer rootfs form somewhere.

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines

Thanks
-Bharat

Quoted 1 line
Quoted 1 line
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 7 lines

Thanks
-Bharat

./qemu-system-arm -enable-kvm -M vexpress-a15 -cpu cortex-a15 \
-device

pl330,vfio_group="/dev/vfio/0",vfio_device="2c0a0000.dma",vfio_contai ner="/dev/v

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

fio/vfio" \
-append "earlyprintk ip=dhcp console=ttyAMA0 rootwait" \
-kernel zImage -dtb guest-a15.dtb -initrd
filesystem.cpio.gz

-

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 3 lines

nographic But the error still exists.
qemu-system-arm: -device

pl330,vfio_group=/dev/vfio/0,vfio_device=2c0a0000.dma,vfio_container= /dev/vfio/v

Quoted 33 lines
Quoted 33 lines
Quoted 33 lines
Quoted 33 lines
Quoted 33 lines
Quoted 33 lines
Quoted 33 lines

fio: Parameter 'driver' expects pluggable device type

Thanks,
Shannon

On 2014/6/19 19:26, Alvise Rigo wrote:

Il 19/06/2014 13:11, Shannon Zhao ha scritto:

On 2014/6/19 18:47, Alvise Rigo wrote:
Hi,

Il 19/06/2014 11:38, Shannon Zhao ha scritto:
Hi Bharat,

On 2014/6/19 17:19, [email protected] wrote:

-----Original Message-----
From: [email protected]
[mailto:kvmarm- [email protected]] On
Behalf Of Shannon Zhao
Sent: Thursday, June 19, 2014 2:45 PM
To: Alvise Rigo; a.motakis-lrHrjnjw1UfHK3s98zE1ajGjJy/[email protected];
iommu-cunTk1MwBs/[email protected] foundation.org;
[email protected]
Subject: Re: Vfio-on-arm: the device tree does not
contain the DMA device

Hi Alvise,

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines

Thanks for your reply.
I have apply the patch and could see the DMA devices.
But there is no vosys_dmatest int the directory
"/sys/kernel/debug/" after mounting debugfs.
I found that the branch vfio-platform-v4 doesn't
contain the module vosys_dmatest.
Then I port it from the branch pl330_dmatest and run

vosys_dmatest

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines

successfully.

echo 1 > /sys/kernel/debug/vosys_dmatest/start

[ 6145.343419] dmaengine: __dma_request_channel:
success

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines

(dma0chan0)
[ 6145.344000] vosys dmatest: callback: transfer completed
[ 6145.344147] vosys dmatest: tx completed
[ 6145.344589] vosys dmatest: source and destination match.

Download the qemu and use following command to start a guest:
./qemu-system-arm -enable-kvm -M vexpress-a15 -cpu
cortex-

a15 \

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

-device
pl330,vfio_group="/dev/vfio/0",vfio_device="2c0a0000.dma",\
vfio_container="/dev/vfio/vfio" \
-append "earlyprintk ip=dhcp console=ttyAMA0 rootwait"

\

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

-kernel zImage -dtb guest.dtb -initrd

filesystem.cpio.gz

Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines

- nographic An error reported:
qemu-system-arm: -device
pl330,vfio_group=/dev/vfio/0,vfio_device=2c0a0000.dma,:

Why do we need to give "vfio_group=/dev/vfio/0" ?
I refer to the url
"http://www.virtualopensystems.com/en/solutions/guides/vfio-on-arm/".

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines

could not open disk image vfio_container=/dev/vfio:
Could not open 'vfio_container=/dev/vfio': No such
file or directory

Are you unbinding "2c0a0000.dma" from its driver and
binding with

"vfio-

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

playform" driver ?
How to unbind "2c0a0000.dma" from its driver?
I have tried following cmd to bind "2c0a0000.dma" with
"vfio-

playform"

Quoted 25 lines
Quoted 25 lines
Quoted 25 lines
Quoted 25 lines
Quoted 25 lines
Quoted 25 lines
Quoted 25 lines

driver.
echo 2c0a0000.dma >
/sys/bus/platform/drivers/vfio-platform/vfio_bind
But there is no /dev/vfio/0 or /dev/vfio/vfio .

Can you please verify that while configuring the host you enabled:
- Device Drivers -> IOMMU Hardware Support -> ARM Ltd.
System MMU
(SMMU) Support and
- Device Drivers -> VFIO Non-Privileged userspace driver
framework
-> VFIO support for platform devices

Hi Alvise,

I have checked the two configuration are enabled.

Then try the following cmd, but another error reported:
qemu-system-arm: -device
pl330,vfio_group=/dev/vfio/0,vfio_device=2c0a0000.dma,vfio_
container=
/dev/vfio/vfio: Parameter 'driver' expects pluggable device
type

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

Do I need to unbind the "2c0a0000.dma" from its driver and
binding

with

Quoted 15 lines
Quoted 15 lines
Quoted 15 lines
Quoted 15 lines
Quoted 15 lines
Quoted 15 lines
Quoted 15 lines

"vfio-platform" driver as Bharat mentioned ?

Yes, the 2c0a0000.dma should not be bound to any driver at
boot, so you just need to:
echo 2c0a0000.dma >
/sys/bus/platform/drivers/vfio-platform/vfio_bind
to bind the device to the VFIO driver.

Regards,
alvise

Then try to update the pl330-vfio-dev branch and run QEMU
with the

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines

following

Quoted 5 lines

command:

./qemu-system-arm -enable-kvm -M vexpress-a15 -cpu cortex-a15 \
-device

pl330,vfio_group="/dev/vfio/0",vfio_device="2c0a0000.dma",vfio_contai ner="/dev/v

Quoted 18 lines
Quoted 18 lines
Quoted 18 lines
Quoted 18 lines
Quoted 18 lines
Quoted 18 lines
Quoted 18 lines

fio/vfio" \
-append "earlyprintk ip=dhcp console=ttyAMA0 rootwait" \
-kernel zImage -dtb guest.dtb -initrd
filesystem.cpio.gz

that means without breaking the "-device pl330" options.

Regards,
alvise

Thanks
-Bharat

Is there something I'm wrong or do I miss some steps?

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines

Thank you,
Shannon

On 2014/6/18 21:24, Alvise Rigo wrote:
Hi Shannon,

Thank you for reporting this issue, the device tree
file was

indeed

Quoted 181 lines
Quoted 181 lines
Quoted 181 lines
Quoted 180 lines
Quoted 179 lines
Quoted 170 lines
Quoted 170 lines

missing.
To solve the problem, you can apply the following
patch to the repository you mentioned
(git://github.com/virtualopensystems/arm-dts.git).

For your information, soon we will update the guide
with a newer version of VFIO with this issue fixed.

Thank you,
alvise

---
fast_models/rtsm_ve-cortex_a15x4_a7x4.dts | 100
+++++++++++++++++++++++++++++-
1 file changed, 97 insertions(+), 3 deletions(-)

diff --git a/fast_models/rtsm_ve-cortex_a15x4_a7x4.dts
b/fast_models/rtsm_ve-cortex_a15x4_a7x4.dts
index ef1275b..79c6af2 100644
--- a/fast_models/rtsm_ve-cortex_a15x4_a7x4.dts
+++ b/fast_models/rtsm_ve-cortex_a15x4_a7x4.dts
@@ -142,7 +142,7 @@
cpu7: cpu@7 {
device_type = "cpu";
compatible = "arm,cortex-a7";
- reg = <0x104>;
+ reg = <0x103>;
cluster = <&cluster1>;
core = <&core7>;
};
@@ -159,7 +159,7 @@
#address-cells = <0>;
interrupt-controller;
reg = <0 0x2c001000 0 0x1000>,
- <0 0x2c002000 0 0x2000>,
+ <0 0x2c002000 0 0x1000>,
<0 0x2c004000 0 0x2000>,
<0 0x2c006000 0 0x2000>;
interrupts = <1 9 0xf04>; @@ -183,6 +183,101 @@
<1 10 0xf08>;
};

+ smmu@2c0b0000 {
+ compatible = "arm,smmu-v2";
+ reg = <0x0 0x2c0b0000 0x0 0x10000>;
+ #global-interrupts = <2>;
+ interrupts = <0 58 4>,
+ <0 59 4>,
+ <0 60 4>,
+ <0 60 4>,
+ <0 60 4>,
+ <0 60 4>,
+ <0 60 4>,
+ <0 60 4>,
+ <0 60 4>;
+ mmu-masters = <&dma0 0x0 0x8>,
+ <&dma1 0x1 0x9>,
+ <&dma2 0x2 0xa>,
+ <&dma3 0x3 0xb>;
+ };
+
+ dma0: dma@2c0a0000 {
+ compatible = "arm,pl330";
+ reg = <0 0x2c0a0000 0 0x1000>;
+ interrupts = <0 64 4>,
+ <0 65 4>,
+ <0 66 4>,
+ <0 67 4>,
+ <0 68 4>,
+ <0 69 4>,
+ <0 70 4>,
+ <0 71 4>;
+ #dma-cells = <1>;
+ #stream-id-cells = <2>;
+ clocks = <&oscclk6a>;
+ clock-names = "apb_pclk";
+ };
+
+ dma1: dma@2c0a1000 {
+ compatible = "arm,pl330", "arm,primecell";
+ reg = <0 0x2c0a1000 0 0x1000>;
+ interrupts = <0 96 4>,
+ <0 97 4>,
+ <0 98 4>,
+ <0 99 4>,
+ <0 100 4>,
+ <0 101 4>,
+ <0 102 4>,
+ <0 103 4>;
+ #dma-cells = <1>;
+ #stream-id-cells = <2>;
+ clocks = <&oscclk6a>;
+ clock-names = "apb_pclk";
+ };
+
+ dma2: dma@2c0a2000 {
+ compatible = "arm,pl330", "arm,primecell";
+ reg = <0 0x2c0a2000 0 0x1000>;
+ interrupts = <0 128 4>,
+ <0 129 4>,
+ <0 130 4>,
+ <0 131 4>,
+ <0 132 4>,
+ <0 133 4>,
+ <0 134 4>,
+ <0 135 4>;
+ #dma-cells = <1>;
+ #stream-id-cells = <2>;
+ clocks = <&oscclk6a>;
+ clock-names = "apb_pclk";
+ };
+
+ dma3: dma@2c0a3000 {
+ compatible = "arm,pl330", "arm,primecell";
+ reg = <0 0x2c0a3000 0 0x1000>;
+ interrupts = <0 160 4>,
+ <0 161 4>,
+ <0 162 4>,
+ <0 163 4>,
+ <0 164 4>,
+ <0 165 4>,
+ <0 166 4>,
+ <0 167 4>;
+ #dma-cells = <1>;
+ #stream-id-cells = <2>;
+ clocks = <&oscclk6a>;
+ clock-names = "apb_pclk";
+ };
+
+ oscclk6a: oscclk6a {
+ /* Reference 24MHz clock */
+ compatible = "fixed-clock";
+ #clock-cells = <0>;
+ clock-frequency = <24000000>;
+ clock-output-names = "oscclk6a";
+ };
+
smb {
compatible = "simple-bus";

@@ -244,4 +339,3 @@
/include/ "rtsm_ve-motherboard.dtsi"
};
};
-

--
Shannon

_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm

.

.

.

--
Shannon

Quoted 5 lines
Quoted 2 lines

.

--
Shannon

.

Quoted 6 lines
Quoted 3 lines
Quoted 2 lines

.

.

--
Shannon

--
Shannon

.

Signature · 1 line
Shannon
#56 lore raw
In reply to a message not in the archive

Hello,

This is a known error, caused by using the last version of VFIO with an old version of QEMU.
Try to use the vfio-platform-v4 kernel, if it still doesn't work, use the vfio-platform-v5.

Regards,
alvise

Il 24/06/2014 15:19, Shannon Zhao ha scritto:

Quoted 696 lines

Hi Alvise,

Thanks for your help.
I'm using pl330-vfio-dev branch but it's not up-to-date. Then I update the qemu code.
Now I can start up the guest.

I run the following cmd in guest :

echo 1 > /sys/kernel/debug/vosys_dmatest/start
dmaengine: dma_sync_wait: timeout! vosys dmatest: tx error
vosys dmatest: src and destination do not match
dmesg | grep vosys
vosys dmatest: vosys dmatest module init vosys dmatest: tx error
vosys dmatest: tx completed
And at the terminal running isim_system there is an error :

ERROR: DMAC330 channel thread abort.
ERROR: DMAC330: Instruction fetch ERROR, thread set to FAULTING state

Is there something wrong?

Thanks,
Shannon

Quoted 666 lines · Alvise Rigo

This is a QEMU error. Are you using the pl330-vfio-dev branch? Is it up-to-date?

alvise

Il 24/06/2014 11:00, Shannon Zhao ha scritto:

Quoted 23 lines

Hi Bharat,

ls -l /sys/bus/platform/devices/2c0a0000.dma/iommu_group/devices
total 0
lrwxrwxrwx 1 root root 0 Jan 1 02:58 2c0a0000.dma -> ../../../../devices/2c0a0000.dma

Then I run "echo 2c0a0000.dma > /sys/bus/platform/drivers/vfio-platform/bind". There is /dev/vfio/0.

But there is also an error reported.

	-nographic can not be used with -daemonize
	char device redirected to /dev/pts/3 (label pty0-base)
	PL330_VFIO: === VFIO device file descriptor 13 ===
	Device has 1 region(s):
	PL330_VFIO: - Region 0: size=0x1000 offset=0x0 flags=0x7
	PL330_VFIO: - IRQ 0: range of 1, flags=0x7
	PL330_VFIO: - IRQ 1: range of 1, flags=0x7
	PL330_VFIO: - IRQ 2: range of 1, flags=0x7
	PL330_VFIO: - IRQ 3: range of 1, flags=0x7
	PL330_VFIO: - IRQ 4: range of 1, flags=0x7
	qemu-system-arm: -device pl330,vfio_group=/dev/vfio/0,vfio_device=2c0a0000.dma,vfio_container=/dev/vfio/vfio: Property '.vfio_group' not found
Quoted 634 lines

Thanks,
Shannon

Quoted 630 lines · On 2014/6/24 16:08, [email protected]…
Quoted 10 lines

-----Original Message-----
From: Shannon Zhao [mailto:[email protected]]
Sent: Tuesday, June 24, 2014 1:16 PM
To: Bhushan Bharat-R65777; Alvise Rigo; a.motakis-lrHrjnjw1UfHK3s98zE1ajGjJy/[email protected]; iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org; [email protected]; [email protected]
Subject: Re: Vfio-on-arm: the device tree does not contain the DMA device

Quoted 7 lines

Hi Bharat,

Thanks for your help.

I executed the following cmd:
echo vfio-platform >
/sys/bus/platform/devices/2c0a0000.dma/driver_override

You have to bind the device with vfio-platform driver after the above command.

Quoted 5 lines

But there is no /dev/vfio/0 and error reported starting guest

-nographic can not be used with -daemonize
char device redirected to /dev/pts/3 (label pty0-base)
PL330_VFIO: Group is not viable (not all devices bound for vfio)

A bit surprised; it should be failing when opening /dev/vfio/0 but it is failing in VFIO_GROUP_GET_STATUS ioctl.

Also looks like there are more device attached to the said iommu-group. Can check by running below command:
#ls -l /sys/bus/pci/devices/2c0a0000.dma/iommu_group/devices

Thanks
-Bharat

Quoted 594 lines

qemu-system-arm: Error while probing real device.

qemu-system-arm: Initialization of device pl330 failed Is there something
wrong?

Thank you very much.
Shannon

Quoted 93 lines · On 2014/6/24 15:05, [email protected]…
Quoted 93 lines

-----Original Message-----
From: Shannon Zhao [mailto:[email protected]]
Sent: Tuesday, June 24, 2014 12:30 PM
To: Alvise Rigo; Bhushan Bharat-R65777;
a.motakis-lrHrjnjw1UfHK3s98zE1ajGjJy/[email protected]; iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org; [email protected]; [email protected]
Subject: Re: Vfio-on-arm: the device tree does not contain the DMA
device

Hi Alvise,

Thanks for your help.
Now I can see /dev/vfio/vfio and /dev/vfio/0 after executing "echo 2c0a0000.dma

Quoted 1 line

/sys/bus/platform/drivers/vfio-platform/vfio_bind".

But when using qemu to start guest, there is an error :

-nographic can not be used with -daemonize char device redirected to /dev/pts/3 (label pty0-base)
PL330_VFIO: === VFIO device file descriptor 13 === Device has 1 region(s):
PL330_VFIO: - Region 0: size=0x1000 offset=0x2c0a0000 flags=0x7
PL330_VFIO: - IRQ 0: range of 1, flags=0x7 ------------[ cut here ]------------ kernel BUG at mm/slub.c:3386!

Internal error: Oops - BUG: 0 [#1] SMP ARM Modules linked in:
CPU: 2 PID: 1746 Comm: qemu-system-arm Tainted: G        W    3.14.0-rc1+ #19
task: ee665580 ti: ed0b8000 task.ti: ed0b8000 PC is at
kfree+0x128/0x134 LR is at vfio_set_trigger.isra.5+0x40/0x104
pc : [<800d9e3c>]    lr : [<80319094>]    psr: 400f0013

sp : ed0b9e64 ip : 000800f3 fp : 000014ac
r10: 00000000 r9 : ed0b8000 r8 : edeb3b80
r7 : 0000000e r6 : 80620584 r5 : 00000000 r4 : eea0b000 r3 : 00000400 r2 : 800f38b0 r1 : eea0ce60 r0 : 00000400
Flags: nZcv IRQs on FIQs on Mode SVC_32 ISA ARM Segment user
Control: 30c53c7d Table: eddac080 DAC: 00000000 Process qemu-system-arm (pid: 1746, stack limit = 0xed0b8240)

Stack: (0xed0b9e64 to 0xed0ba000)

9e60: ee603700 00000000 ee603700 0000000e edeb3b80 00000000 80319094
9e80: ee603700 00003b6e 00000000 edeb3b80 ee603700 00003b6e 0000000d 80319208
9ea0: 0000000e edeb3b80 80492a18 edeb3b80 ee57f780 80319690 00000024 008b2074
9ec0: 008b2074 8031886c 00000001 008b2074 00000018 00000024 00000000 00000000
9ee0: 00000001 00000004 00000001 00000001 ee088e70 008b2060 ee57f780 803155d0
9f00: 803155b0 800ef430 80477a58 00080802 ed0b8000 800e1358 00000020 ed0b9f50
9f20: ed0b9f60 00000003 8061e808 00080802 ed0b9f60 800e14f0 00000000 806522f8
9f40: 80477a58 8061e808 ede4c1c0 8011af18 00000000 00000009 8053bbc0 00000000
9f60: ee683610 00000001 ee57f780 008b2060 00003b6e 0000000d ed0b8000 00000000
9f80: 000014ac 800ef9ac 0000000e 00000001 008b2060 00429eb0 0000000d 00000036
9fa0: 8001c464 8001c2e0 008b2060 00429eb0 0000000d 00003b6e 008b2060 00000001
9fc0: 008b2060 00429eb0 0000000d 00000036 00000000 7ea99040 0089a220 000014ac
9fe0: 003fa474 7ea98fc4 000bc5c8 76ceb2ec 200f0010 0000000d 00000000 8062c770 [<800d9e3c>] (kfree) from [<80319094>] (vfio_set_trigger.isra.5+0x40/0x104)
[<80319094>] (vfio_set_trigger.isra.5) from [<80319208>] (vfio_platform_set_irq_trigger+0xb0/0x144)
[<80319208>] (vfio_platform_set_irq_trigger) from [<80319690>] (vfio_platform_set_irqs_ioctl+0x44/0x58)
[<80319690>] (vfio_platform_set_irqs_ioctl) from [<8031886c>] (vfio_platform_ioctl+0x2e0/0x314)
[<8031886c>] (vfio_platform_ioctl) from [<803155d0>] (vfio_device_fops_unl_ioctl+0x20/0x2c)
[<803155d0>] (vfio_device_fops_unl_ioctl) from [<800ef430>] (do_vfs_ioctl+0x78/0x5b8)
[<800ef430>] (do_vfs_ioctl) from [<800ef9ac>] (SyS_ioctl+0x3c/0x5c) [<800ef9ac>] (SyS_ioctl) from [<8001c2e0>] (ret_fast_syscall+0x0/0x30)

Code: 15913038 e1a01003 e8bd45f0 eaff28e5 (e7f001f2)

I'm using branch vfio-platform-v4 for kernel and pl330-vfio-dev for qemu. Following is qemu cmdline:

	./qemu-system-arm -enable-kvm -M vexpress-a15 -cpu cortex-a15 \
       	-chardev pty,id=pty0,mux=on -monitor chardev:pty0 \
       	-serial chardev:pty0 -daemonize \
       	-device

pl330,vfio_group="/dev/vfio/0",vfio_device="2c0a0000.dma",vfio_contai ner="/dev/v
fio/vfio" \
-append "earlyprintk ip=dhcp console=ttyAMA0 rootwait" \
-kernel zImage -dtb guest-a15.dtb -initrd filesystem.cpio.gz
- nographic

Quoted 4 lines
Quoted 2 lines

If I change to branch vfio-platform-v6 for kernel, there is "/dev/vfio/vfio". But no "/sys/bus/platform/drivers/vfio-platform/vfio_bind".

Can you Check if you have driver_override at

"/sys/bus/platform/devices/DEVICE-XXX/ ?

Quoted 1 line

If yes then you should use driver_override mechanism to bind that device to

vfio-platform driver.

Quoted 482 lines

Thanks
-Bharat

Quoted 477 lines

And if I use above qemu cmdline to start guest, there is also an error :
PL330_VFIO: Group is not viable (not all devices bound for vfio)
qemu-system-arm: Error while probing real device.

qemu-system-arm: Initialization of device pl330 failed

The usage about v4 and v6 has changed ?
And would you tell me when you will update the guideline about vfio-on-arm?

Thanks,
Shannon

Quoted 12 lines · Alvise Rigo

Linaro has a great set of pre-built images here http://releases.linaro.org/.

For example these should work fine:
http://releases.linaro.org/13.12/ubuntu/saucy-images

alvise

Il 20/06/2014 10:23, Shannon Zhao ha scritto:

Quoted 4 lines

Hi Alvise,

I'm so sorry that I can't use qemu-debootstrap because my environment can't

access some links using proxy.

Quoted 14 lines
Quoted 14 lines

Are there other ways to get a proper fs?

Thanks
Shannon

Quoted 8 lines · Alvise Rigo

Hi Shannon,

If you follow the steps in the section "Creating the File System" of the guide you should be able to obtain a working fs.

Regards,
alvise

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines

Il 20/06/2014 04:57, Shannon Zhao ha scritto:

Quoted 5 lines

Hi Bharat,

Thanks for your advice .

I'm using the rootfs getting from the url

"http://www.virtualopensystems.com/downloads/guides/kvm_on_arm/fs-ali p-
armel.cramfs".

Quoted 40 lines
Quoted 40 lines
Quoted 40 lines
Quoted 40 lines

Can you give me another link to get a newer rootfs?

Thank you
Shannon

Quoted 33 lines · On 2014/6/19 20:20, [email protected]…
Quoted 33 lines

-----Original Message-----
From: Shannon Zhao [mailto:[email protected]]
Sent: Thursday, June 19, 2014 5:47 PM
To: Bhushan Bharat-R65777; Alvise Rigo;
a.motakis-lrHrjnjw1UfHK3s98zE1ajGjJy/[email protected];
iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org; [email protected]
Subject: Re: Vfio-on-arm: the device tree does not contain the
DMA device

On 2014/6/19 20:12, [email protected] wrote:

-----Original Message-----
From: Shannon Zhao [mailto:[email protected]]
Sent: Thursday, June 19, 2014 5:38 PM
To: Alvise Rigo; Bhushan Bharat-R65777;
a.motakis-lrHrjnjw1UfHK3s98zE1ajGjJy/[email protected];
iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org;
[email protected]
Subject: Re: Vfio-on-arm: the device tree does not contain
the DMA device

Hi Alvise,

I run following cmds :
echo 2c0a0000.dma >
/sys/bus/platform/drivers/vfio-platform/vfio_bind

run "ls /dev/vfio/" before and after the above command

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 5 lines

Hi Bharat,

Nothing appeared either before or after the cmd "echo
2c0a0000.dma > /sys/bus/platform/drivers/vfio-platform/vfio_bind".
There is only /dev/vfio.

I am not sure, but may be you are using pretty old rootfs with broken udev

rules. Try to fetch a newer rootfs form somewhere.

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines

Thanks
-Bharat

Quoted 1 line
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines

Thanks
-Bharat

./qemu-system-arm -enable-kvm -M vexpress-a15 -cpu cortex-a15 \
-device

pl330,vfio_group="/dev/vfio/0",vfio_device="2c0a0000.dma",vfio_contai ner="/dev/v

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

fio/vfio" \
-append "earlyprintk ip=dhcp console=ttyAMA0 rootwait" \
-kernel zImage -dtb guest-a15.dtb -initrd
filesystem.cpio.gz

-

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

nographic But the error still exists.
qemu-system-arm: -device

pl330,vfio_group=/dev/vfio/0,vfio_device=2c0a0000.dma,vfio_container= /dev/vfio/v

Quoted 33 lines
Quoted 33 lines
Quoted 33 lines
Quoted 33 lines
Quoted 33 lines
Quoted 33 lines

fio: Parameter 'driver' expects pluggable device type

Thanks,
Shannon

On 2014/6/19 19:26, Alvise Rigo wrote:

Il 19/06/2014 13:11, Shannon Zhao ha scritto:

On 2014/6/19 18:47, Alvise Rigo wrote:
Hi,

Il 19/06/2014 11:38, Shannon Zhao ha scritto:
Hi Bharat,

On 2014/6/19 17:19, [email protected] wrote:

-----Original Message-----
From: [email protected]
[mailto:kvmarm- [email protected]] On
Behalf Of Shannon Zhao
Sent: Thursday, June 19, 2014 2:45 PM
To: Alvise Rigo; a.motakis-lrHrjnjw1UfHK3s98zE1ajGjJy/[email protected];
iommu-cunTk1MwBs/[email protected] foundation.org;
[email protected]
Subject: Re: Vfio-on-arm: the device tree does not
contain the DMA device

Hi Alvise,

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines

Thanks for your reply.
I have apply the patch and could see the DMA devices.
But there is no vosys_dmatest int the directory
"/sys/kernel/debug/" after mounting debugfs.
I found that the branch vfio-platform-v4 doesn't
contain the module vosys_dmatest.
Then I port it from the branch pl330_dmatest and run

vosys_dmatest

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines

successfully.

echo 1 > /sys/kernel/debug/vosys_dmatest/start

[ 6145.343419] dmaengine: __dma_request_channel:
success

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines

(dma0chan0)
[ 6145.344000] vosys dmatest: callback: transfer completed
[ 6145.344147] vosys dmatest: tx completed
[ 6145.344589] vosys dmatest: source and destination match.

Download the qemu and use following command to start a guest:
./qemu-system-arm -enable-kvm -M vexpress-a15 -cpu
cortex-

a15 \

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

-device
pl330,vfio_group="/dev/vfio/0",vfio_device="2c0a0000.dma",\
vfio_container="/dev/vfio/vfio" \
-append "earlyprintk ip=dhcp console=ttyAMA0 rootwait"

\

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

-kernel zImage -dtb guest.dtb -initrd

filesystem.cpio.gz

Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines

- nographic An error reported:
qemu-system-arm: -device
pl330,vfio_group=/dev/vfio/0,vfio_device=2c0a0000.dma,:

Why do we need to give "vfio_group=/dev/vfio/0" ?
I refer to the url
"http://www.virtualopensystems.com/en/solutions/guides/vfio-on-arm/".

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines

could not open disk image vfio_container=/dev/vfio:
Could not open 'vfio_container=/dev/vfio': No such
file or directory

Are you unbinding "2c0a0000.dma" from its driver and
binding with

"vfio-

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

playform" driver ?
How to unbind "2c0a0000.dma" from its driver?
I have tried following cmd to bind "2c0a0000.dma" with
"vfio-

playform"

Quoted 25 lines
Quoted 25 lines
Quoted 25 lines
Quoted 25 lines
Quoted 25 lines
Quoted 25 lines

driver.
echo 2c0a0000.dma >
/sys/bus/platform/drivers/vfio-platform/vfio_bind
But there is no /dev/vfio/0 or /dev/vfio/vfio .

Can you please verify that while configuring the host you enabled:
- Device Drivers -> IOMMU Hardware Support -> ARM Ltd.
System MMU
(SMMU) Support and
- Device Drivers -> VFIO Non-Privileged userspace driver
framework
-> VFIO support for platform devices

Hi Alvise,

I have checked the two configuration are enabled.

Then try the following cmd, but another error reported:
qemu-system-arm: -device
pl330,vfio_group=/dev/vfio/0,vfio_device=2c0a0000.dma,vfio_
container=
/dev/vfio/vfio: Parameter 'driver' expects pluggable device
type

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

Do I need to unbind the "2c0a0000.dma" from its driver and
binding

with

Quoted 15 lines
Quoted 15 lines
Quoted 15 lines
Quoted 15 lines
Quoted 15 lines
Quoted 15 lines

"vfio-platform" driver as Bharat mentioned ?

Yes, the 2c0a0000.dma should not be bound to any driver at
boot, so you just need to:
echo 2c0a0000.dma >
/sys/bus/platform/drivers/vfio-platform/vfio_bind
to bind the device to the VFIO driver.

Regards,
alvise

Then try to update the pl330-vfio-dev branch and run QEMU
with the

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines

following
command:

./qemu-system-arm -enable-kvm -M vexpress-a15 -cpu cortex-a15 \
-device

pl330,vfio_group="/dev/vfio/0",vfio_device="2c0a0000.dma",vfio_contai ner="/dev/v

Quoted 18 lines
Quoted 18 lines
Quoted 18 lines
Quoted 18 lines
Quoted 18 lines
Quoted 18 lines

fio/vfio" \
-append "earlyprintk ip=dhcp console=ttyAMA0 rootwait" \
-kernel zImage -dtb guest.dtb -initrd
filesystem.cpio.gz

that means without breaking the "-device pl330" options.

Regards,
alvise

Thanks
-Bharat

Is there something I'm wrong or do I miss some steps?

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines

Thank you,
Shannon

On 2014/6/18 21:24, Alvise Rigo wrote:
Hi Shannon,

Thank you for reporting this issue, the device tree
file was

indeed

Quoted 181 lines
Quoted 181 lines
Quoted 181 lines
Quoted 180 lines
Quoted 179 lines
Quoted 175 lines

missing.
To solve the problem, you can apply the following
patch to the repository you mentioned
(git://github.com/virtualopensystems/arm-dts.git).

For your information, soon we will update the guide
with a newer version of VFIO with this issue fixed.

Thank you,
alvise

---
fast_models/rtsm_ve-cortex_a15x4_a7x4.dts | 100
+++++++++++++++++++++++++++++-
1 file changed, 97 insertions(+), 3 deletions(-)

diff --git a/fast_models/rtsm_ve-cortex_a15x4_a7x4.dts
b/fast_models/rtsm_ve-cortex_a15x4_a7x4.dts
index ef1275b..79c6af2 100644
--- a/fast_models/rtsm_ve-cortex_a15x4_a7x4.dts
+++ b/fast_models/rtsm_ve-cortex_a15x4_a7x4.dts
@@ -142,7 +142,7 @@
cpu7: cpu@7 {
device_type = "cpu";
compatible = "arm,cortex-a7";
- reg = <0x104>;
+ reg = <0x103>;
cluster = <&cluster1>;
core = <&core7>;
};
@@ -159,7 +159,7 @@
#address-cells = <0>;
interrupt-controller;
reg = <0 0x2c001000 0 0x1000>,
- <0 0x2c002000 0 0x2000>,
+ <0 0x2c002000 0 0x1000>,
<0 0x2c004000 0 0x2000>,
<0 0x2c006000 0 0x2000>;
interrupts = <1 9 0xf04>; @@ -183,6 +183,101 @@
<1 10 0xf08>;
};

+ smmu@2c0b0000 {
+ compatible = "arm,smmu-v2";
+ reg = <0x0 0x2c0b0000 0x0 0x10000>;
+ #global-interrupts = <2>;
+ interrupts = <0 58 4>,
+ <0 59 4>,
+ <0 60 4>,
+ <0 60 4>,
+ <0 60 4>,
+ <0 60 4>,
+ <0 60 4>,
+ <0 60 4>,
+ <0 60 4>;
+ mmu-masters = <&dma0 0x0 0x8>,
+ <&dma1 0x1 0x9>,
+ <&dma2 0x2 0xa>,
+ <&dma3 0x3 0xb>;
+ };
+
+ dma0: dma@2c0a0000 {
+ compatible = "arm,pl330";
+ reg = <0 0x2c0a0000 0 0x1000>;
+ interrupts = <0 64 4>,
+ <0 65 4>,
+ <0 66 4>,
+ <0 67 4>,
+ <0 68 4>,
+ <0 69 4>,
+ <0 70 4>,
+ <0 71 4>;
+ #dma-cells = <1>;
+ #stream-id-cells = <2>;
+ clocks = <&oscclk6a>;
+ clock-names = "apb_pclk";
+ };
+
+ dma1: dma@2c0a1000 {
+ compatible = "arm,pl330", "arm,primecell";
+ reg = <0 0x2c0a1000 0 0x1000>;
+ interrupts = <0 96 4>,
+ <0 97 4>,
+ <0 98 4>,
+ <0 99 4>,
+ <0 100 4>,
+ <0 101 4>,
+ <0 102 4>,
+ <0 103 4>;
+ #dma-cells = <1>;
+ #stream-id-cells = <2>;
+ clocks = <&oscclk6a>;
+ clock-names = "apb_pclk";
+ };
+
+ dma2: dma@2c0a2000 {
+ compatible = "arm,pl330", "arm,primecell";
+ reg = <0 0x2c0a2000 0 0x1000>;
+ interrupts = <0 128 4>,
+ <0 129 4>,
+ <0 130 4>,
+ <0 131 4>,
+ <0 132 4>,
+ <0 133 4>,
+ <0 134 4>,
+ <0 135 4>;
+ #dma-cells = <1>;
+ #stream-id-cells = <2>;
+ clocks = <&oscclk6a>;
+ clock-names = "apb_pclk";
+ };
+
+ dma3: dma@2c0a3000 {
+ compatible = "arm,pl330", "arm,primecell";
+ reg = <0 0x2c0a3000 0 0x1000>;
+ interrupts = <0 160 4>,
+ <0 161 4>,
+ <0 162 4>,
+ <0 163 4>,
+ <0 164 4>,
+ <0 165 4>,
+ <0 166 4>,
+ <0 167 4>;
+ #dma-cells = <1>;
+ #stream-id-cells = <2>;
+ clocks = <&oscclk6a>;
+ clock-names = "apb_pclk";
+ };
+
+ oscclk6a: oscclk6a {
+ /* Reference 24MHz clock */
+ compatible = "fixed-clock";
+ #clock-cells = <0>;
+ clock-frequency = <24000000>;
+ clock-output-names = "oscclk6a";
+ };
+
smb {
compatible = "simple-bus";

@@ -244,4 +339,3 @@
/include/ "rtsm_ve-motherboard.dtsi"
};
};
-

--
Shannon

_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm

.

.

.

--
Shannon

.

--
Shannon

.

Quoted 6 lines
Quoted 3 lines
Quoted 2 lines

.

.

--
Shannon

--
Shannon

.

#60 lore raw
In reply to a message not in the archive

Hi Alvise ,

I have tried to use vfio-platform-v4 and vfio-platform-v5 kernel, but it doesn't work. Kernel Panic as mentioned before :

 ------------[ cut here ]------------
kernel BUG at mm/slub.c:3392!
Internal error: Oops - BUG: 0 [#1] SMP ARM
Modules linked in:
CPU: 1 PID: 1711 Comm: qemu-system-arm Tainted: G        W    3.14.0-rc4+ #2
task: edd76780 ti: edfcc000 task.ti: edfcc000
PC is at kfree+0x128/0x134
LR is at vfio_set_trigger.isra.5+0x40/0x104
pc : [<800da170>]    lr : [<803197cc>]    psr: 400b0013

sp : edfcde64 ip : 00080385 fp : 7e800100
r10: 00000000 r9 : edfcc000 r8 : ee5c4e80
r7 : 0000000e r6 : 80620584 r5 : 00000000 r4 : eea0b000 r3 : 00000400 r2 : 80385db0 r1 : eea120a0 r0 : 00000400
Flags: nZcv IRQs on FIQs on Mode SVC_32 ISA ARM Segment user
Control: 30c53c7d Table: edf93fc0 DAC: 00000000 Process qemu-system-arm (pid: 1711, stack limit = 0xedfcc240)

Stack: (0xedfcde64 to 0xedfce000)

de60: eddb3c00 00000000 eddb3c00 0000000e ee5c4e80 00000000 803197cc
de80: eddb3c00 00003b6e 00000000 ee5c4e80 eddb3c00 00003b6e 0000000d 80319940
dea0: 0000000e ee5c4e80 804929d8 ee5c4e80 eddee780 80319dc8 00000024 009331e4
dec0: 009331e4 803190ac 00000001 009331e4 00000018 00000024 00000000 00000000
dee0: 00000001 edd76780 edd00480 80614840 ee06a2a0 009331d0 eddee780 80315bfc
df00: 80315bdc 800ef7e8 80477a18 00080802 edfcc000 800e1674 00000020 edfcdf50
df20: edfcdf60 00000003 8061e808 00080802 edfcdf60 800e180c 00000000 80652438
df40: 80477a18 8061e808 edf93e00 8011b3dc 00000000 00000009 8053bb0c 00000000
df60: ee501190 00000001 eddee780 009331d0 00003b6e 0000000d edfcc000 00000000
df80: 7e800100 800efd64 0000000e 00000001 009331d0 00429ed8 0000000d 00000036
dfa0: 8001c4a4 8001c320 009331d0 00429ed8 0000000d 00003b6e 009331d0 00000001
dfc0: 009331d0 00429ed8 0000000d 00000036 00000000 0094fa60 00000000 7e800100
dfe0: 003fa3fc 7e80008c 000bc5d0 76d552ec 200b0010 0000000d 00000000 8062c870 [<800da170>] (kfree) from [<803197cc>] (vfio_set_trigger.isra.5+0x40/0x104) [<803197cc>] (vfio_set_trigger.isra.5) from [<80319940>] (vfio_platform_set_irq_trigger+0xb0/0x144) [<80319940>] (vfio_platform_set_irq_trigger) from [<80319dc8>] (vfio_platform_set_irqs_ioctl+0x44/0x58) [<80319dc8>] (vfio_platform_set_irqs_ioctl) from [<803190ac>] (vfio_platform_ioctl+0x2e8/0x31c) [<803190ac>] (vfio_platform_ioctl) from [<80315bfc>] (vfio_device_fops_unl_ioctl+0x20/0x2c) [<80315bfc>] (vfio_device_fops_unl_ioctl) from [<800ef7e8>] (do_vfs_ioctl+0x78/0x5b8) [<800ef7e8>] (do_vfs_ioctl) from [<800efd64>] (SyS_ioctl+0x3c/0x5c)
[<800efd64>] (SyS_ioctl) from [<8001c320>] (ret_fast_syscall+0x0/0x30)

As you say, if using vfio-platform-v6 kernel and pl330-vfio-dev-new qemu, the vosys_dmatest should be successful? Is the qemu cmdline to start guest different? I got the following error:

qemu-system-arm: -device pl330,vfio_group=/dev/vfio/0,vfio_device=2c0a0000.dma,vfio_container=/dev/vfio/vfio: Parameter 'driver' expects pluggable device type
Thanks,
Shannon

Quoted 710 lines · Alvise Rigo

Hello,

This is a known error, caused by using the last version of VFIO with an old version of QEMU.
Try to use the vfio-platform-v4 kernel, if it still doesn't work, use the vfio-platform-v5.

Regards,
alvise

Il 24/06/2014 15:19, Shannon Zhao ha scritto:

Quoted 22 lines

Hi Alvise,

Thanks for your help.
I'm using pl330-vfio-dev branch but it's not up-to-date. Then I update the qemu code.
Now I can start up the guest.

I run the following cmd in guest :

echo 1 > /sys/kernel/debug/vosys_dmatest/start
dmaengine: dma_sync_wait: timeout! vosys dmatest: tx error
vosys dmatest: src and destination do not match
dmesg | grep vosys
vosys dmatest: vosys dmatest module init vosys dmatest: tx error
vosys dmatest: tx completed
And at the terminal running isim_system there is an error :

ERROR: DMAC330 channel thread abort.
ERROR: DMAC330: Instruction fetch ERROR, thread set to FAULTING state

Quoted 674 lines

Is there something wrong?

Thanks,
Shannon

Quoted 666 lines · Alvise Rigo

This is a QEMU error. Are you using the pl330-vfio-dev branch? Is it up-to-date?

alvise

Il 24/06/2014 11:00, Shannon Zhao ha scritto:

Quoted 23 lines

Hi Bharat,

ls -l /sys/bus/platform/devices/2c0a0000.dma/iommu_group/devices
total 0
lrwxrwxrwx 1 root root 0 Jan 1 02:58 2c0a0000.dma -> ../../../../devices/2c0a0000.dma

Then I run "echo 2c0a0000.dma > /sys/bus/platform/drivers/vfio-platform/bind". There is /dev/vfio/0.

But there is also an error reported.

	-nographic can not be used with -daemonize
	char device redirected to /dev/pts/3 (label pty0-base)
	PL330_VFIO: === VFIO device file descriptor 13 ===
	Device has 1 region(s):
	PL330_VFIO: - Region 0: size=0x1000 offset=0x0 flags=0x7
	PL330_VFIO: - IRQ 0: range of 1, flags=0x7
	PL330_VFIO: - IRQ 1: range of 1, flags=0x7
	PL330_VFIO: - IRQ 2: range of 1, flags=0x7
	PL330_VFIO: - IRQ 3: range of 1, flags=0x7
	PL330_VFIO: - IRQ 4: range of 1, flags=0x7
	qemu-system-arm: -device pl330,vfio_group=/dev/vfio/0,vfio_device=2c0a0000.dma,vfio_container=/dev/vfio/vfio: Property '.vfio_group' not found
Quoted 634 lines

Thanks,
Shannon

Quoted 630 lines · On 2014/6/24 16:08, [email protected]…
Quoted 10 lines

-----Original Message-----
From: Shannon Zhao [mailto:[email protected]]
Sent: Tuesday, June 24, 2014 1:16 PM
To: Bhushan Bharat-R65777; Alvise Rigo; a.motakis-lrHrjnjw1UfHK3s98zE1ajGjJy/[email protected]; iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org; [email protected]; [email protected]
Subject: Re: Vfio-on-arm: the device tree does not contain the DMA device

Quoted 7 lines

Hi Bharat,

Thanks for your help.

I executed the following cmd:
echo vfio-platform >
/sys/bus/platform/devices/2c0a0000.dma/driver_override

You have to bind the device with vfio-platform driver after the above command.

Quoted 5 lines

But there is no /dev/vfio/0 and error reported starting guest

-nographic can not be used with -daemonize
char device redirected to /dev/pts/3 (label pty0-base)
PL330_VFIO: Group is not viable (not all devices bound for vfio)

A bit surprised; it should be failing when opening /dev/vfio/0 but it is failing in VFIO_GROUP_GET_STATUS ioctl.

Also looks like there are more device attached to the said iommu-group. Can check by running below command:
#ls -l /sys/bus/pci/devices/2c0a0000.dma/iommu_group/devices

Thanks
-Bharat

Quoted 594 lines

qemu-system-arm: Error while probing real device.

qemu-system-arm: Initialization of device pl330 failed Is there something
wrong?

Thank you very much.
Shannon

Quoted 93 lines · On 2014/6/24 15:05, [email protected]…
Quoted 93 lines

-----Original Message-----
From: Shannon Zhao [mailto:[email protected]]
Sent: Tuesday, June 24, 2014 12:30 PM
To: Alvise Rigo; Bhushan Bharat-R65777;
a.motakis-lrHrjnjw1UfHK3s98zE1ajGjJy/[email protected]; iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org; [email protected]; [email protected]
Subject: Re: Vfio-on-arm: the device tree does not contain the DMA
device

Hi Alvise,

Thanks for your help.
Now I can see /dev/vfio/vfio and /dev/vfio/0 after executing "echo 2c0a0000.dma

Quoted 1 line

/sys/bus/platform/drivers/vfio-platform/vfio_bind".

But when using qemu to start guest, there is an error :

-nographic can not be used with -daemonize char device redirected to /dev/pts/3 (label pty0-base)
PL330_VFIO: === VFIO device file descriptor 13 === Device has 1 region(s):
PL330_VFIO: - Region 0: size=0x1000 offset=0x2c0a0000 flags=0x7
PL330_VFIO: - IRQ 0: range of 1, flags=0x7 ------------[ cut here ]------------ kernel BUG at mm/slub.c:3386!

Internal error: Oops - BUG: 0 [#1] SMP ARM Modules linked in:
CPU: 2 PID: 1746 Comm: qemu-system-arm Tainted: G        W    3.14.0-rc1+ #19
task: ee665580 ti: ed0b8000 task.ti: ed0b8000 PC is at
kfree+0x128/0x134 LR is at vfio_set_trigger.isra.5+0x40/0x104
pc : [<800d9e3c>]    lr : [<80319094>]    psr: 400f0013

sp : ed0b9e64 ip : 000800f3 fp : 000014ac
r10: 00000000 r9 : ed0b8000 r8 : edeb3b80
r7 : 0000000e r6 : 80620584 r5 : 00000000 r4 : eea0b000 r3 : 00000400 r2 : 800f38b0 r1 : eea0ce60 r0 : 00000400
Flags: nZcv IRQs on FIQs on Mode SVC_32 ISA ARM Segment user
Control: 30c53c7d Table: eddac080 DAC: 00000000 Process qemu-system-arm (pid: 1746, stack limit = 0xed0b8240)

Stack: (0xed0b9e64 to 0xed0ba000)

9e60: ee603700 00000000 ee603700 0000000e edeb3b80 00000000 80319094
9e80: ee603700 00003b6e 00000000 edeb3b80 ee603700 00003b6e 0000000d 80319208
9ea0: 0000000e edeb3b80 80492a18 edeb3b80 ee57f780 80319690 00000024 008b2074
9ec0: 008b2074 8031886c 00000001 008b2074 00000018 00000024 00000000 00000000
9ee0: 00000001 00000004 00000001 00000001 ee088e70 008b2060 ee57f780 803155d0
9f00: 803155b0 800ef430 80477a58 00080802 ed0b8000 800e1358 00000020 ed0b9f50
9f20: ed0b9f60 00000003 8061e808 00080802 ed0b9f60 800e14f0 00000000 806522f8
9f40: 80477a58 8061e808 ede4c1c0 8011af18 00000000 00000009 8053bbc0 00000000
9f60: ee683610 00000001 ee57f780 008b2060 00003b6e 0000000d ed0b8000 00000000
9f80: 000014ac 800ef9ac 0000000e 00000001 008b2060 00429eb0 0000000d 00000036
9fa0: 8001c464 8001c2e0 008b2060 00429eb0 0000000d 00003b6e 008b2060 00000001
9fc0: 008b2060 00429eb0 0000000d 00000036 00000000 7ea99040 0089a220 000014ac
9fe0: 003fa474 7ea98fc4 000bc5c8 76ceb2ec 200f0010 0000000d 00000000 8062c770 [<800d9e3c>] (kfree) from [<80319094>] (vfio_set_trigger.isra.5+0x40/0x104)
[<80319094>] (vfio_set_trigger.isra.5) from [<80319208>] (vfio_platform_set_irq_trigger+0xb0/0x144)
[<80319208>] (vfio_platform_set_irq_trigger) from [<80319690>] (vfio_platform_set_irqs_ioctl+0x44/0x58)
[<80319690>] (vfio_platform_set_irqs_ioctl) from [<8031886c>] (vfio_platform_ioctl+0x2e0/0x314)
[<8031886c>] (vfio_platform_ioctl) from [<803155d0>] (vfio_device_fops_unl_ioctl+0x20/0x2c)
[<803155d0>] (vfio_device_fops_unl_ioctl) from [<800ef430>] (do_vfs_ioctl+0x78/0x5b8)
[<800ef430>] (do_vfs_ioctl) from [<800ef9ac>] (SyS_ioctl+0x3c/0x5c) [<800ef9ac>] (SyS_ioctl) from [<8001c2e0>] (ret_fast_syscall+0x0/0x30)

Code: 15913038 e1a01003 e8bd45f0 eaff28e5 (e7f001f2)

I'm using branch vfio-platform-v4 for kernel and pl330-vfio-dev for qemu. Following is qemu cmdline:

	./qemu-system-arm -enable-kvm -M vexpress-a15 -cpu cortex-a15 \
       	-chardev pty,id=pty0,mux=on -monitor chardev:pty0 \
       	-serial chardev:pty0 -daemonize \
       	-device

pl330,vfio_group="/dev/vfio/0",vfio_device="2c0a0000.dma",vfio_contai ner="/dev/v
fio/vfio" \
-append "earlyprintk ip=dhcp console=ttyAMA0 rootwait" \
-kernel zImage -dtb guest-a15.dtb -initrd filesystem.cpio.gz
- nographic

Quoted 4 lines
Quoted 2 lines

If I change to branch vfio-platform-v6 for kernel, there is "/dev/vfio/vfio". But no "/sys/bus/platform/drivers/vfio-platform/vfio_bind".

Can you Check if you have driver_override at

"/sys/bus/platform/devices/DEVICE-XXX/ ?

Quoted 1 line

If yes then you should use driver_override mechanism to bind that device to

vfio-platform driver.

Quoted 482 lines

Thanks
-Bharat

Quoted 477 lines

And if I use above qemu cmdline to start guest, there is also an error :
PL330_VFIO: Group is not viable (not all devices bound for vfio)
qemu-system-arm: Error while probing real device.

qemu-system-arm: Initialization of device pl330 failed

The usage about v4 and v6 has changed ?
And would you tell me when you will update the guideline about vfio-on-arm?

Thanks,
Shannon

Quoted 12 lines · Alvise Rigo

Linaro has a great set of pre-built images here http://releases.linaro.org/.

For example these should work fine:
http://releases.linaro.org/13.12/ubuntu/saucy-images

alvise

Il 20/06/2014 10:23, Shannon Zhao ha scritto:

Quoted 4 lines

Hi Alvise,

I'm so sorry that I can't use qemu-debootstrap because my environment can't

access some links using proxy.

Quoted 14 lines
Quoted 14 lines

Are there other ways to get a proper fs?

Thanks
Shannon

Quoted 8 lines · Alvise Rigo

Hi Shannon,

If you follow the steps in the section "Creating the File System" of the guide you should be able to obtain a working fs.

Regards,
alvise

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines

Il 20/06/2014 04:57, Shannon Zhao ha scritto:

Quoted 5 lines

Hi Bharat,

Thanks for your advice .

I'm using the rootfs getting from the url

"http://www.virtualopensystems.com/downloads/guides/kvm_on_arm/fs-ali p-
armel.cramfs".

Quoted 40 lines
Quoted 40 lines
Quoted 40 lines
Quoted 40 lines

Can you give me another link to get a newer rootfs?

Thank you
Shannon

Quoted 33 lines · On 2014/6/19 20:20, [email protected]…

-----Original Message-----
From: Shannon Zhao [mailto:[email protected]]
Sent: Thursday, June 19, 2014 5:47 PM
To: Bhushan Bharat-R65777; Alvise Rigo;
a.motakis-lrHrjnjw1UfHK3s98zE1ajGjJy/[email protected];
iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org; [email protected]
Subject: Re: Vfio-on-arm: the device tree does not contain the
DMA device

On 2014/6/19 20:12, [email protected] wrote:

-----Original Message-----
From: Shannon Zhao [mailto:[email protected]]
Sent: Thursday, June 19, 2014 5:38 PM
To: Alvise Rigo; Bhushan Bharat-R65777;
a.motakis-lrHrjnjw1UfHK3s98zE1ajGjJy/[email protected];
iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org;
[email protected]
Subject: Re: Vfio-on-arm: the device tree does not contain
the DMA device

Hi Alvise,

I run following cmds :
echo 2c0a0000.dma >
/sys/bus/platform/drivers/vfio-platform/vfio_bind

run "ls /dev/vfio/" before and after the above command

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines

Hi Bharat,

Nothing appeared either before or after the cmd "echo
2c0a0000.dma > /sys/bus/platform/drivers/vfio-platform/vfio_bind".
There is only /dev/vfio.

I am not sure, but may be you are using pretty old rootfs with
broken udev

rules. Try to fetch a newer rootfs form somewhere.

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines

Thanks
-Bharat

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines

Thanks
-Bharat

./qemu-system-arm -enable-kvm -M vexpress-a15 -cpu cortex-a15 \
-device

pl330,vfio_group="/dev/vfio/0",vfio_device="2c0a0000.dma",vfio_contai ner="/dev/v

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

fio/vfio" \
-append "earlyprintk ip=dhcp console=ttyAMA0 rootwait" \
-kernel zImage -dtb guest-a15.dtb -initrd
filesystem.cpio.gz

-

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

nographic But the error still exists.
qemu-system-arm: -device

pl330,vfio_group=/dev/vfio/0,vfio_device=2c0a0000.dma,vfio_container= /dev/vfio/v

Quoted 33 lines
Quoted 33 lines
Quoted 33 lines
Quoted 33 lines
Quoted 33 lines

fio: Parameter 'driver' expects pluggable device type

Thanks,
Shannon

On 2014/6/19 19:26, Alvise Rigo wrote:

Il 19/06/2014 13:11, Shannon Zhao ha scritto:

On 2014/6/19 18:47, Alvise Rigo wrote:
Hi,

Il 19/06/2014 11:38, Shannon Zhao ha scritto:
Hi Bharat,

On 2014/6/19 17:19, [email protected] wrote:

-----Original Message-----
From: [email protected]
[mailto:kvmarm- [email protected]] On
Behalf Of Shannon Zhao
Sent: Thursday, June 19, 2014 2:45 PM
To: Alvise Rigo; a.motakis-lrHrjnjw1UfHK3s98zE1ajGjJy/[email protected];
iommu-cunTk1MwBs/[email protected] foundation.org;
[email protected]
Subject: Re: Vfio-on-arm: the device tree does not
contain the DMA device

Hi Alvise,

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines

Thanks for your reply.
I have apply the patch and could see the DMA devices.
But there is no vosys_dmatest int the directory
"/sys/kernel/debug/" after mounting debugfs.
I found that the branch vfio-platform-v4 doesn't
contain the module vosys_dmatest.
Then I port it from the branch pl330_dmatest and run

vosys_dmatest

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines

successfully.

echo 1 > /sys/kernel/debug/vosys_dmatest/start

[ 6145.343419] dmaengine: __dma_request_channel:
success

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines

(dma0chan0)
[ 6145.344000] vosys dmatest: callback: transfer completed
[ 6145.344147] vosys dmatest: tx completed
[ 6145.344589] vosys dmatest: source and destination match.

Download the qemu and use following command to start a guest:
./qemu-system-arm -enable-kvm -M vexpress-a15 -cpu
cortex-

a15 \

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

-device
pl330,vfio_group="/dev/vfio/0",vfio_device="2c0a0000.dma",\
vfio_container="/dev/vfio/vfio" \
-append "earlyprintk ip=dhcp console=ttyAMA0 rootwait"

\

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

-kernel zImage -dtb guest.dtb -initrd

filesystem.cpio.gz

Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines

- nographic An error reported:
qemu-system-arm: -device
pl330,vfio_group=/dev/vfio/0,vfio_device=2c0a0000.dma,:

Why do we need to give "vfio_group=/dev/vfio/0" ?
I refer to the url
"http://www.virtualopensystems.com/en/solutions/guides/vfio-on-arm/".

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines

could not open disk image vfio_container=/dev/vfio:
Could not open 'vfio_container=/dev/vfio': No such
file or directory

Are you unbinding "2c0a0000.dma" from its driver and
binding with

"vfio-

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

playform" driver ?
How to unbind "2c0a0000.dma" from its driver?
I have tried following cmd to bind "2c0a0000.dma" with
"vfio-

playform"

Quoted 25 lines
Quoted 25 lines
Quoted 25 lines
Quoted 25 lines
Quoted 25 lines

driver.
echo 2c0a0000.dma >
/sys/bus/platform/drivers/vfio-platform/vfio_bind
But there is no /dev/vfio/0 or /dev/vfio/vfio .

Can you please verify that while configuring the host you enabled:
- Device Drivers -> IOMMU Hardware Support -> ARM Ltd.
System MMU
(SMMU) Support and
- Device Drivers -> VFIO Non-Privileged userspace driver
framework
-> VFIO support for platform devices

Hi Alvise,

I have checked the two configuration are enabled.

Then try the following cmd, but another error reported:
qemu-system-arm: -device
pl330,vfio_group=/dev/vfio/0,vfio_device=2c0a0000.dma,vfio_
container=
/dev/vfio/vfio: Parameter 'driver' expects pluggable device
type

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

Do I need to unbind the "2c0a0000.dma" from its driver and
binding

with

Quoted 15 lines
Quoted 15 lines
Quoted 15 lines
Quoted 15 lines
Quoted 15 lines

"vfio-platform" driver as Bharat mentioned ?

Yes, the 2c0a0000.dma should not be bound to any driver at
boot, so you just need to:
echo 2c0a0000.dma >
/sys/bus/platform/drivers/vfio-platform/vfio_bind
to bind the device to the VFIO driver.

Regards,
alvise

Then try to update the pl330-vfio-dev branch and run QEMU
with the

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines

following
command:

./qemu-system-arm -enable-kvm -M vexpress-a15 -cpu cortex-a15 \
-device

pl330,vfio_group="/dev/vfio/0",vfio_device="2c0a0000.dma",vfio_contai ner="/dev/v

Quoted 18 lines
Quoted 18 lines
Quoted 18 lines
Quoted 18 lines
Quoted 18 lines

fio/vfio" \
-append "earlyprintk ip=dhcp console=ttyAMA0 rootwait" \
-kernel zImage -dtb guest.dtb -initrd
filesystem.cpio.gz

that means without breaking the "-device pl330" options.

Regards,
alvise

Thanks
-Bharat

Is there something I'm wrong or do I miss some steps?

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines

Thank you,
Shannon

On 2014/6/18 21:24, Alvise Rigo wrote:
Hi Shannon,

Thank you for reporting this issue, the device tree
file was

indeed

Quoted 181 lines
Quoted 181 lines
Quoted 181 lines
Quoted 180 lines
Quoted 179 lines

missing.
To solve the problem, you can apply the following
patch to the repository you mentioned
(git://github.com/virtualopensystems/arm-dts.git).

For your information, soon we will update the guide
with a newer version of VFIO with this issue fixed.

Thank you,
alvise

---
fast_models/rtsm_ve-cortex_a15x4_a7x4.dts | 100
+++++++++++++++++++++++++++++-
1 file changed, 97 insertions(+), 3 deletions(-)

diff --git a/fast_models/rtsm_ve-cortex_a15x4_a7x4.dts
b/fast_models/rtsm_ve-cortex_a15x4_a7x4.dts
index ef1275b..79c6af2 100644
--- a/fast_models/rtsm_ve-cortex_a15x4_a7x4.dts
+++ b/fast_models/rtsm_ve-cortex_a15x4_a7x4.dts
@@ -142,7 +142,7 @@
cpu7: cpu@7 {
device_type = "cpu";
compatible = "arm,cortex-a7";
- reg = <0x104>;
+ reg = <0x103>;
cluster = <&cluster1>;
core = <&core7>;
};
@@ -159,7 +159,7 @@
#address-cells = <0>;
interrupt-controller;
reg = <0 0x2c001000 0 0x1000>,
- <0 0x2c002000 0 0x2000>,
+ <0 0x2c002000 0 0x1000>,
<0 0x2c004000 0 0x2000>,
<0 0x2c006000 0 0x2000>;
interrupts = <1 9 0xf04>; @@ -183,6 +183,101 @@
<1 10 0xf08>;
};

+ smmu@2c0b0000 {
+ compatible = "arm,smmu-v2";
+ reg = <0x0 0x2c0b0000 0x0 0x10000>;
+ #global-interrupts = <2>;
+ interrupts = <0 58 4>,
+ <0 59 4>,
+ <0 60 4>,
+ <0 60 4>,
+ <0 60 4>,
+ <0 60 4>,
+ <0 60 4>,
+ <0 60 4>,
+ <0 60 4>;
+ mmu-masters = <&dma0 0x0 0x8>,
+ <&dma1 0x1 0x9>,
+ <&dma2 0x2 0xa>,
+ <&dma3 0x3 0xb>;
+ };
+
+ dma0: dma@2c0a0000 {
+ compatible = "arm,pl330";
+ reg = <0 0x2c0a0000 0 0x1000>;
+ interrupts = <0 64 4>,
+ <0 65 4>,
+ <0 66 4>,
+ <0 67 4>,
+ <0 68 4>,
+ <0 69 4>,
+ <0 70 4>,
+ <0 71 4>;
+ #dma-cells = <1>;
+ #stream-id-cells = <2>;
+ clocks = <&oscclk6a>;
+ clock-names = "apb_pclk";
+ };
+
+ dma1: dma@2c0a1000 {
+ compatible = "arm,pl330", "arm,primecell";
+ reg = <0 0x2c0a1000 0 0x1000>;
+ interrupts = <0 96 4>,
+ <0 97 4>,
+ <0 98 4>,
+ <0 99 4>,
+ <0 100 4>,
+ <0 101 4>,
+ <0 102 4>,
+ <0 103 4>;
+ #dma-cells = <1>;
+ #stream-id-cells = <2>;
+ clocks = <&oscclk6a>;
+ clock-names = "apb_pclk";
+ };
+
+ dma2: dma@2c0a2000 {
+ compatible = "arm,pl330", "arm,primecell";
+ reg = <0 0x2c0a2000 0 0x1000>;
+ interrupts = <0 128 4>,
+ <0 129 4>,
+ <0 130 4>,
+ <0 131 4>,
+ <0 132 4>,
+ <0 133 4>,
+ <0 134 4>,
+ <0 135 4>;
+ #dma-cells = <1>;
+ #stream-id-cells = <2>;
+ clocks = <&oscclk6a>;
+ clock-names = "apb_pclk";
+ };
+
+ dma3: dma@2c0a3000 {
+ compatible = "arm,pl330", "arm,primecell";
+ reg = <0 0x2c0a3000 0 0x1000>;
+ interrupts = <0 160 4>,
+ <0 161 4>,
+ <0 162 4>,
+ <0 163 4>,
+ <0 164 4>,
+ <0 165 4>,
+ <0 166 4>,
+ <0 167 4>;
+ #dma-cells = <1>;
+ #stream-id-cells = <2>;
+ clocks = <&oscclk6a>;
+ clock-names = "apb_pclk";
+ };
+
+ oscclk6a: oscclk6a {
+ /* Reference 24MHz clock */
+ compatible = "fixed-clock";
+ #clock-cells = <0>;
+ clock-frequency = <24000000>;
+ clock-output-names = "oscclk6a";
+ };
+
smb {
compatible = "simple-bus";

@@ -244,4 +339,3 @@
/include/ "rtsm_ve-motherboard.dtsi"
};
};
-

--
Shannon

_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm

.

.

.

--
Shannon

.

--
Shannon

.

Quoted 6 lines
Quoted 3 lines
Quoted 2 lines

.

.

--
Shannon

--
Shannon

.

.

Signature · 1 line
Shannon
#64 lore raw
In reply to a message not in the archive

Hi Shannon,

Quoted 3 lines · Shannon Zhao

Hi Alvise ,

I have tried to use vfio-platform-v4 and vfio-platform-v5 kernel, but it doesn't work.

Strange, I tried to replicate it with both v4 and v5 and it worked fine.

Quoted 41 lines

Kernel Panic as mentioned before :

 ------------[ cut here ]------------
kernel BUG at mm/slub.c:3392!
Internal error: Oops - BUG: 0 [#1] SMP ARM
Modules linked in:
CPU: 1 PID: 1711 Comm: qemu-system-arm Tainted: G        W    3.14.0-rc4+ #2
task: edd76780 ti: edfcc000 task.ti: edfcc000
PC is at kfree+0x128/0x134
LR is at vfio_set_trigger.isra.5+0x40/0x104
pc : [<800da170>]    lr : [<803197cc>]    psr: 400b0013

sp : edfcde64 ip : 00080385 fp : 7e800100
r10: 00000000 r9 : edfcc000 r8 : ee5c4e80
r7 : 0000000e r6 : 80620584 r5 : 00000000 r4 : eea0b000 r3 : 00000400 r2 : 80385db0 r1 : eea120a0 r0 : 00000400
Flags: nZcv IRQs on FIQs on Mode SVC_32 ISA ARM Segment user
Control: 30c53c7d Table: edf93fc0 DAC: 00000000 Process qemu-system-arm (pid: 1711, stack limit = 0xedfcc240)

Stack: (0xedfcde64 to 0xedfce000)

de60: eddb3c00 00000000 eddb3c00 0000000e ee5c4e80 00000000 803197cc
de80: eddb3c00 00003b6e 00000000 ee5c4e80 eddb3c00 00003b6e 0000000d 80319940
dea0: 0000000e ee5c4e80 804929d8 ee5c4e80 eddee780 80319dc8 00000024 009331e4
dec0: 009331e4 803190ac 00000001 009331e4 00000018 00000024 00000000 00000000
dee0: 00000001 edd76780 edd00480 80614840 ee06a2a0 009331d0 eddee780 80315bfc
df00: 80315bdc 800ef7e8 80477a18 00080802 edfcc000 800e1674 00000020 edfcdf50
df20: edfcdf60 00000003 8061e808 00080802 edfcdf60 800e180c 00000000 80652438
df40: 80477a18 8061e808 edf93e00 8011b3dc 00000000 00000009 8053bb0c 00000000
df60: ee501190 00000001 eddee780 009331d0 00003b6e 0000000d edfcc000 00000000
df80: 7e800100 800efd64 0000000e 00000001 009331d0 00429ed8 0000000d 00000036
dfa0: 8001c4a4 8001c320 009331d0 00429ed8 0000000d 00003b6e 009331d0 00000001
dfc0: 009331d0 00429ed8 0000000d 00000036 00000000 0094fa60 00000000 7e800100
dfe0: 003fa3fc 7e80008c 000bc5d0 76d552ec 200b0010 0000000d 00000000 8062c870 [<800da170>] (kfree) from [<803197cc>] (vfio_set_trigger.isra.5+0x40/0x104) [<803197cc>] (vfio_set_trigger.isra.5) from [<80319940>] (vfio_platform_set_irq_trigger+0xb0/0x144) [<80319940>] (vfio_platform_set_irq_trigger) from [<80319dc8>] (vfio_platform_set_irqs_ioctl+0x44/0x58) [<80319dc8>] (vfio_platform_set_irqs_ioctl) from [<803190ac>] (vfio_platform_ioctl+0x2e8/0x31c) [<803190ac>] (vfio_platform_ioctl) from [<80315bfc>] (vfio_device_fops_unl_ioctl+0x20/0x2c) [<80315bfc>] (vfio_device_fops_unl_ioctl) from [<800ef7e8>] (do_vfs_ioctl+0x78/0x5b8) [<800ef7e8>] (do_vfs_ioctl) from [<800efd64>] (SyS_ioctl+0x3c/0x5c)
[<800efd64>] (SyS_ioctl) from [<8001c320>] (ret_fast_syscall+0x0/0x30)

Quoted 4 lines

As you say, if using vfio-platform-v6 kernel and pl330-vfio-dev-new qemu, the vosys_dmatest should be successful? Is the qemu cmdline to start guest different? I got the following error:

qemu-system-arm: -device pl330,vfio_group=/dev/vfio/0,vfio_device=2c0a0000.dma,vfio_container=/dev/vfio/vfio: Parameter 'driver' expects pluggable device type> Thanks,

Yes, with that branch of QEMU the starting command is totally different. For this, I suggest you to wait the new version of the guide that will be released tomorrow.

Regards,
alvise

Quoted 714 lines

Shannon

Quoted 710 lines · Alvise Rigo

Hello,

This is a known error, caused by using the last version of VFIO with an old version of QEMU.
Try to use the vfio-platform-v4 kernel, if it still doesn't work, use the vfio-platform-v5.

Regards,
alvise

Il 24/06/2014 15:19, Shannon Zhao ha scritto:

Quoted 22 lines

Hi Alvise,

Thanks for your help.
I'm using pl330-vfio-dev branch but it's not up-to-date. Then I update the qemu code.
Now I can start up the guest.

I run the following cmd in guest :

echo 1 > /sys/kernel/debug/vosys_dmatest/start
dmaengine: dma_sync_wait: timeout! vosys dmatest: tx error
vosys dmatest: src and destination do not match
dmesg | grep vosys
vosys dmatest: vosys dmatest module init vosys dmatest: tx error
vosys dmatest: tx completed
And at the terminal running isim_system there is an error :

ERROR: DMAC330 channel thread abort.
ERROR: DMAC330: Instruction fetch ERROR, thread set to FAULTING state

Quoted 674 lines

Is there something wrong?

Thanks,
Shannon

Quoted 666 lines · Alvise Rigo

This is a QEMU error. Are you using the pl330-vfio-dev branch? Is it up-to-date?

alvise

Il 24/06/2014 11:00, Shannon Zhao ha scritto:

Quoted 23 lines

Hi Bharat,

ls -l /sys/bus/platform/devices/2c0a0000.dma/iommu_group/devices
total 0
lrwxrwxrwx 1 root root 0 Jan 1 02:58 2c0a0000.dma -> ../../../../devices/2c0a0000.dma

Then I run "echo 2c0a0000.dma > /sys/bus/platform/drivers/vfio-platform/bind". There is /dev/vfio/0.

But there is also an error reported.

	-nographic can not be used with -daemonize
	char device redirected to /dev/pts/3 (label pty0-base)
	PL330_VFIO: === VFIO device file descriptor 13 ===
	Device has 1 region(s):
	PL330_VFIO: - Region 0: size=0x1000 offset=0x0 flags=0x7
	PL330_VFIO: - IRQ 0: range of 1, flags=0x7
	PL330_VFIO: - IRQ 1: range of 1, flags=0x7
	PL330_VFIO: - IRQ 2: range of 1, flags=0x7
	PL330_VFIO: - IRQ 3: range of 1, flags=0x7
	PL330_VFIO: - IRQ 4: range of 1, flags=0x7
	qemu-system-arm: -device pl330,vfio_group=/dev/vfio/0,vfio_device=2c0a0000.dma,vfio_container=/dev/vfio/vfio: Property '.vfio_group' not found
Quoted 634 lines

Thanks,
Shannon

Quoted 630 lines · On 2014/6/24 16:08, [email protected]…
Quoted 10 lines

-----Original Message-----
From: Shannon Zhao [mailto:[email protected]]
Sent: Tuesday, June 24, 2014 1:16 PM
To: Bhushan Bharat-R65777; Alvise Rigo; a.motakis-lrHrjnjw1UfHK3s98zE1ajGjJy/[email protected]; iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org; [email protected]; [email protected]
Subject: Re: Vfio-on-arm: the device tree does not contain the DMA device

Quoted 7 lines

Hi Bharat,

Thanks for your help.

I executed the following cmd:
echo vfio-platform >
/sys/bus/platform/devices/2c0a0000.dma/driver_override

You have to bind the device with vfio-platform driver after the above command.

Quoted 5 lines

But there is no /dev/vfio/0 and error reported starting guest

-nographic can not be used with -daemonize
char device redirected to /dev/pts/3 (label pty0-base)
PL330_VFIO: Group is not viable (not all devices bound for vfio)

A bit surprised; it should be failing when opening /dev/vfio/0 but it is failing in VFIO_GROUP_GET_STATUS ioctl.

Also looks like there are more device attached to the said iommu-group. Can check by running below command:
#ls -l /sys/bus/pci/devices/2c0a0000.dma/iommu_group/devices

Thanks
-Bharat

Quoted 594 lines

qemu-system-arm: Error while probing real device.

qemu-system-arm: Initialization of device pl330 failed Is there something
wrong?

Thank you very much.
Shannon

Quoted 93 lines · On 2014/6/24 15:05, [email protected]…
Quoted 93 lines

-----Original Message-----
From: Shannon Zhao [mailto:[email protected]]
Sent: Tuesday, June 24, 2014 12:30 PM
To: Alvise Rigo; Bhushan Bharat-R65777;
a.motakis-lrHrjnjw1UfHK3s98zE1ajGjJy/[email protected]; iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org; [email protected]; [email protected]
Subject: Re: Vfio-on-arm: the device tree does not contain the DMA
device

Hi Alvise,

Thanks for your help.
Now I can see /dev/vfio/vfio and /dev/vfio/0 after executing "echo 2c0a0000.dma

Quoted 1 line

/sys/bus/platform/drivers/vfio-platform/vfio_bind".

But when using qemu to start guest, there is an error :

-nographic can not be used with -daemonize char device redirected to /dev/pts/3 (label pty0-base)
PL330_VFIO: === VFIO device file descriptor 13 === Device has 1 region(s):
PL330_VFIO: - Region 0: size=0x1000 offset=0x2c0a0000 flags=0x7
PL330_VFIO: - IRQ 0: range of 1, flags=0x7 ------------[ cut here ]------------ kernel BUG at mm/slub.c:3386!

Internal error: Oops - BUG: 0 [#1] SMP ARM Modules linked in:
CPU: 2 PID: 1746 Comm: qemu-system-arm Tainted: G        W    3.14.0-rc1+ #19
task: ee665580 ti: ed0b8000 task.ti: ed0b8000 PC is at
kfree+0x128/0x134 LR is at vfio_set_trigger.isra.5+0x40/0x104
pc : [<800d9e3c>]    lr : [<80319094>]    psr: 400f0013

sp : ed0b9e64 ip : 000800f3 fp : 000014ac
r10: 00000000 r9 : ed0b8000 r8 : edeb3b80
r7 : 0000000e r6 : 80620584 r5 : 00000000 r4 : eea0b000 r3 : 00000400 r2 : 800f38b0 r1 : eea0ce60 r0 : 00000400
Flags: nZcv IRQs on FIQs on Mode SVC_32 ISA ARM Segment user
Control: 30c53c7d Table: eddac080 DAC: 00000000 Process qemu-system-arm (pid: 1746, stack limit = 0xed0b8240)

Stack: (0xed0b9e64 to 0xed0ba000)

9e60: ee603700 00000000 ee603700 0000000e edeb3b80 00000000 80319094
9e80: ee603700 00003b6e 00000000 edeb3b80 ee603700 00003b6e 0000000d 80319208
9ea0: 0000000e edeb3b80 80492a18 edeb3b80 ee57f780 80319690 00000024 008b2074
9ec0: 008b2074 8031886c 00000001 008b2074 00000018 00000024 00000000 00000000
9ee0: 00000001 00000004 00000001 00000001 ee088e70 008b2060 ee57f780 803155d0
9f00: 803155b0 800ef430 80477a58 00080802 ed0b8000 800e1358 00000020 ed0b9f50
9f20: ed0b9f60 00000003 8061e808 00080802 ed0b9f60 800e14f0 00000000 806522f8
9f40: 80477a58 8061e808 ede4c1c0 8011af18 00000000 00000009 8053bbc0 00000000
9f60: ee683610 00000001 ee57f780 008b2060 00003b6e 0000000d ed0b8000 00000000
9f80: 000014ac 800ef9ac 0000000e 00000001 008b2060 00429eb0 0000000d 00000036
9fa0: 8001c464 8001c2e0 008b2060 00429eb0 0000000d 00003b6e 008b2060 00000001
9fc0: 008b2060 00429eb0 0000000d 00000036 00000000 7ea99040 0089a220 000014ac
9fe0: 003fa474 7ea98fc4 000bc5c8 76ceb2ec 200f0010 0000000d 00000000 8062c770 [<800d9e3c>] (kfree) from [<80319094>] (vfio_set_trigger.isra.5+0x40/0x104)
[<80319094>] (vfio_set_trigger.isra.5) from [<80319208>] (vfio_platform_set_irq_trigger+0xb0/0x144)
[<80319208>] (vfio_platform_set_irq_trigger) from [<80319690>] (vfio_platform_set_irqs_ioctl+0x44/0x58)
[<80319690>] (vfio_platform_set_irqs_ioctl) from [<8031886c>] (vfio_platform_ioctl+0x2e0/0x314)
[<8031886c>] (vfio_platform_ioctl) from [<803155d0>] (vfio_device_fops_unl_ioctl+0x20/0x2c)
[<803155d0>] (vfio_device_fops_unl_ioctl) from [<800ef430>] (do_vfs_ioctl+0x78/0x5b8)
[<800ef430>] (do_vfs_ioctl) from [<800ef9ac>] (SyS_ioctl+0x3c/0x5c) [<800ef9ac>] (SyS_ioctl) from [<8001c2e0>] (ret_fast_syscall+0x0/0x30)

Code: 15913038 e1a01003 e8bd45f0 eaff28e5 (e7f001f2)

I'm using branch vfio-platform-v4 for kernel and pl330-vfio-dev for qemu. Following is qemu cmdline:

	./qemu-system-arm -enable-kvm -M vexpress-a15 -cpu cortex-a15 \
       	-chardev pty,id=pty0,mux=on -monitor chardev:pty0 \
       	-serial chardev:pty0 -daemonize \
       	-device

pl330,vfio_group="/dev/vfio/0",vfio_device="2c0a0000.dma",vfio_contai ner="/dev/v
fio/vfio" \
-append "earlyprintk ip=dhcp console=ttyAMA0 rootwait" \
-kernel zImage -dtb guest-a15.dtb -initrd filesystem.cpio.gz
- nographic

Quoted 4 lines
Quoted 2 lines

If I change to branch vfio-platform-v6 for kernel, there is "/dev/vfio/vfio". But no "/sys/bus/platform/drivers/vfio-platform/vfio_bind".

Can you Check if you have driver_override at

"/sys/bus/platform/devices/DEVICE-XXX/ ?

Quoted 1 line

If yes then you should use driver_override mechanism to bind that device to

vfio-platform driver.

Quoted 482 lines

Thanks
-Bharat

Quoted 477 lines

And if I use above qemu cmdline to start guest, there is also an error :
PL330_VFIO: Group is not viable (not all devices bound for vfio)
qemu-system-arm: Error while probing real device.

qemu-system-arm: Initialization of device pl330 failed

The usage about v4 and v6 has changed ?
And would you tell me when you will update the guideline about vfio-on-arm?

Thanks,
Shannon

Quoted 12 lines · Alvise Rigo

Linaro has a great set of pre-built images here http://releases.linaro.org/.

For example these should work fine:
http://releases.linaro.org/13.12/ubuntu/saucy-images

alvise

Il 20/06/2014 10:23, Shannon Zhao ha scritto:

Quoted 4 lines

Hi Alvise,

I'm so sorry that I can't use qemu-debootstrap because my environment can't

access some links using proxy.

Quoted 14 lines
Quoted 14 lines

Are there other ways to get a proper fs?

Thanks
Shannon

Quoted 8 lines · Alvise Rigo

Hi Shannon,

If you follow the steps in the section "Creating the File System" of the guide you should be able to obtain a working fs.

Regards,
alvise

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines

Il 20/06/2014 04:57, Shannon Zhao ha scritto:

Quoted 5 lines

Hi Bharat,

Thanks for your advice .

I'm using the rootfs getting from the url

"http://www.virtualopensystems.com/downloads/guides/kvm_on_arm/fs-ali p-
armel.cramfs".

Quoted 40 lines
Quoted 40 lines
Quoted 40 lines
Quoted 40 lines

Can you give me another link to get a newer rootfs?

Thank you
Shannon

On 2014/6/19 20:20, [email protected] wrote:

-----Original Message-----
From: Shannon Zhao [mailto:[email protected]]
Sent: Thursday, June 19, 2014 5:47 PM
To: Bhushan Bharat-R65777; Alvise Rigo;
a.motakis-lrHrjnjw1UfHK3s98zE1ajGjJy/[email protected];
iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org; [email protected]
Subject: Re: Vfio-on-arm: the device tree does not contain the
DMA device

On 2014/6/19 20:12, [email protected] wrote:

-----Original Message-----
From: Shannon Zhao [mailto:[email protected]]
Sent: Thursday, June 19, 2014 5:38 PM
To: Alvise Rigo; Bhushan Bharat-R65777;
a.motakis-lrHrjnjw1UfHK3s98zE1ajGjJy/[email protected];
iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org;
[email protected]
Subject: Re: Vfio-on-arm: the device tree does not contain
the DMA device

Hi Alvise,

I run following cmds :
echo 2c0a0000.dma >
/sys/bus/platform/drivers/vfio-platform/vfio_bind

run "ls /dev/vfio/" before and after the above command

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines

Hi Bharat,

Nothing appeared either before or after the cmd "echo
2c0a0000.dma > /sys/bus/platform/drivers/vfio-platform/vfio_bind".
There is only /dev/vfio.

I am not sure, but may be you are using pretty old rootfs with
broken udev

rules. Try to fetch a newer rootfs form somewhere.

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines

Thanks
-Bharat

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines

Thanks
-Bharat

./qemu-system-arm -enable-kvm -M vexpress-a15 -cpu cortex-a15 \
-device

pl330,vfio_group="/dev/vfio/0",vfio_device="2c0a0000.dma",vfio_contai ner="/dev/v

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

fio/vfio" \
-append "earlyprintk ip=dhcp console=ttyAMA0 rootwait" \
-kernel zImage -dtb guest-a15.dtb -initrd
filesystem.cpio.gz

-

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

nographic But the error still exists.
qemu-system-arm: -device

pl330,vfio_group=/dev/vfio/0,vfio_device=2c0a0000.dma,vfio_container= /dev/vfio/v

Quoted 33 lines
Quoted 33 lines
Quoted 33 lines
Quoted 33 lines

fio: Parameter 'driver' expects pluggable device type

Thanks,
Shannon

On 2014/6/19 19:26, Alvise Rigo wrote:

Il 19/06/2014 13:11, Shannon Zhao ha scritto:

On 2014/6/19 18:47, Alvise Rigo wrote:
Hi,

Il 19/06/2014 11:38, Shannon Zhao ha scritto:
Hi Bharat,

On 2014/6/19 17:19, [email protected] wrote:

-----Original Message-----
From: [email protected]
[mailto:kvmarm- [email protected]] On
Behalf Of Shannon Zhao
Sent: Thursday, June 19, 2014 2:45 PM
To: Alvise Rigo; a.motakis-lrHrjnjw1UfHK3s98zE1ajGjJy/[email protected];
iommu-cunTk1MwBs/[email protected] foundation.org;
[email protected]
Subject: Re: Vfio-on-arm: the device tree does not
contain the DMA device

Hi Alvise,

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines

Thanks for your reply.
I have apply the patch and could see the DMA devices.
But there is no vosys_dmatest int the directory
"/sys/kernel/debug/" after mounting debugfs.
I found that the branch vfio-platform-v4 doesn't
contain the module vosys_dmatest.
Then I port it from the branch pl330_dmatest and run

vosys_dmatest

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines

successfully.

echo 1 > /sys/kernel/debug/vosys_dmatest/start

[ 6145.343419] dmaengine: __dma_request_channel:
success

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines

(dma0chan0)
[ 6145.344000] vosys dmatest: callback: transfer completed
[ 6145.344147] vosys dmatest: tx completed
[ 6145.344589] vosys dmatest: source and destination match.

Download the qemu and use following command to start a guest:
./qemu-system-arm -enable-kvm -M vexpress-a15 -cpu
cortex-

a15 \

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

-device
pl330,vfio_group="/dev/vfio/0",vfio_device="2c0a0000.dma",\
vfio_container="/dev/vfio/vfio" \
-append "earlyprintk ip=dhcp console=ttyAMA0 rootwait"

\

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

-kernel zImage -dtb guest.dtb -initrd

filesystem.cpio.gz

Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines

- nographic An error reported:
qemu-system-arm: -device
pl330,vfio_group=/dev/vfio/0,vfio_device=2c0a0000.dma,:

Why do we need to give "vfio_group=/dev/vfio/0" ?
I refer to the url
"http://www.virtualopensystems.com/en/solutions/guides/vfio-on-arm/".

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines

could not open disk image vfio_container=/dev/vfio:
Could not open 'vfio_container=/dev/vfio': No such
file or directory

Are you unbinding "2c0a0000.dma" from its driver and
binding with

"vfio-

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

playform" driver ?
How to unbind "2c0a0000.dma" from its driver?
I have tried following cmd to bind "2c0a0000.dma" with
"vfio-

playform"

Quoted 25 lines
Quoted 25 lines
Quoted 25 lines
Quoted 25 lines

driver.
echo 2c0a0000.dma >
/sys/bus/platform/drivers/vfio-platform/vfio_bind
But there is no /dev/vfio/0 or /dev/vfio/vfio .

Can you please verify that while configuring the host you enabled:
- Device Drivers -> IOMMU Hardware Support -> ARM Ltd.
System MMU
(SMMU) Support and
- Device Drivers -> VFIO Non-Privileged userspace driver
framework
-> VFIO support for platform devices

Hi Alvise,

I have checked the two configuration are enabled.

Then try the following cmd, but another error reported:
qemu-system-arm: -device
pl330,vfio_group=/dev/vfio/0,vfio_device=2c0a0000.dma,vfio_
container=
/dev/vfio/vfio: Parameter 'driver' expects pluggable device
type

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

Do I need to unbind the "2c0a0000.dma" from its driver and
binding

with

Quoted 15 lines
Quoted 15 lines
Quoted 15 lines
Quoted 15 lines

"vfio-platform" driver as Bharat mentioned ?

Yes, the 2c0a0000.dma should not be bound to any driver at
boot, so you just need to:
echo 2c0a0000.dma >
/sys/bus/platform/drivers/vfio-platform/vfio_bind
to bind the device to the VFIO driver.

Regards,
alvise

Then try to update the pl330-vfio-dev branch and run QEMU
with the

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines

following
command:

./qemu-system-arm -enable-kvm -M vexpress-a15 -cpu cortex-a15 \
-device

pl330,vfio_group="/dev/vfio/0",vfio_device="2c0a0000.dma",vfio_contai ner="/dev/v

Quoted 18 lines
Quoted 18 lines
Quoted 18 lines
Quoted 18 lines

fio/vfio" \
-append "earlyprintk ip=dhcp console=ttyAMA0 rootwait" \
-kernel zImage -dtb guest.dtb -initrd
filesystem.cpio.gz

that means without breaking the "-device pl330" options.

Regards,
alvise

Thanks
-Bharat

Is there something I'm wrong or do I miss some steps?

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines

Thank you,
Shannon

On 2014/6/18 21:24, Alvise Rigo wrote:
Hi Shannon,

Thank you for reporting this issue, the device tree
file was

indeed

Quoted 181 lines
Quoted 181 lines
Quoted 181 lines
Quoted 180 lines

missing.
To solve the problem, you can apply the following
patch to the repository you mentioned
(git://github.com/virtualopensystems/arm-dts.git).

For your information, soon we will update the guide
with a newer version of VFIO with this issue fixed.

Thank you,
alvise

---
fast_models/rtsm_ve-cortex_a15x4_a7x4.dts | 100
+++++++++++++++++++++++++++++-
1 file changed, 97 insertions(+), 3 deletions(-)

diff --git a/fast_models/rtsm_ve-cortex_a15x4_a7x4.dts
b/fast_models/rtsm_ve-cortex_a15x4_a7x4.dts
index ef1275b..79c6af2 100644
--- a/fast_models/rtsm_ve-cortex_a15x4_a7x4.dts
+++ b/fast_models/rtsm_ve-cortex_a15x4_a7x4.dts
@@ -142,7 +142,7 @@
cpu7: cpu@7 {
device_type = "cpu";
compatible = "arm,cortex-a7";
- reg = <0x104>;
+ reg = <0x103>;
cluster = <&cluster1>;
core = <&core7>;
};
@@ -159,7 +159,7 @@
#address-cells = <0>;
interrupt-controller;
reg = <0 0x2c001000 0 0x1000>,
- <0 0x2c002000 0 0x2000>,
+ <0 0x2c002000 0 0x1000>,
<0 0x2c004000 0 0x2000>,
<0 0x2c006000 0 0x2000>;
interrupts = <1 9 0xf04>; @@ -183,6 +183,101 @@
<1 10 0xf08>;
};

+ smmu@2c0b0000 {
+ compatible = "arm,smmu-v2";
+ reg = <0x0 0x2c0b0000 0x0 0x10000>;
+ #global-interrupts = <2>;
+ interrupts = <0 58 4>,
+ <0 59 4>,
+ <0 60 4>,
+ <0 60 4>,
+ <0 60 4>,
+ <0 60 4>,
+ <0 60 4>,
+ <0 60 4>,
+ <0 60 4>;
+ mmu-masters = <&dma0 0x0 0x8>,
+ <&dma1 0x1 0x9>,
+ <&dma2 0x2 0xa>,
+ <&dma3 0x3 0xb>;
+ };
+
+ dma0: dma@2c0a0000 {
+ compatible = "arm,pl330";
+ reg = <0 0x2c0a0000 0 0x1000>;
+ interrupts = <0 64 4>,
+ <0 65 4>,
+ <0 66 4>,
+ <0 67 4>,
+ <0 68 4>,
+ <0 69 4>,
+ <0 70 4>,
+ <0 71 4>;
+ #dma-cells = <1>;
+ #stream-id-cells = <2>;
+ clocks = <&oscclk6a>;
+ clock-names = "apb_pclk";
+ };
+
+ dma1: dma@2c0a1000 {
+ compatible = "arm,pl330", "arm,primecell";
+ reg = <0 0x2c0a1000 0 0x1000>;
+ interrupts = <0 96 4>,
+ <0 97 4>,
+ <0 98 4>,
+ <0 99 4>,
+ <0 100 4>,
+ <0 101 4>,
+ <0 102 4>,
+ <0 103 4>;
+ #dma-cells = <1>;
+ #stream-id-cells = <2>;
+ clocks = <&oscclk6a>;
+ clock-names = "apb_pclk";
+ };
+
+ dma2: dma@2c0a2000 {
+ compatible = "arm,pl330", "arm,primecell";
+ reg = <0 0x2c0a2000 0 0x1000>;
+ interrupts = <0 128 4>,
+ <0 129 4>,
+ <0 130 4>,
+ <0 131 4>,
+ <0 132 4>,
+ <0 133 4>,
+ <0 134 4>,
+ <0 135 4>;
+ #dma-cells = <1>;
+ #stream-id-cells = <2>;
+ clocks = <&oscclk6a>;
+ clock-names = "apb_pclk";
+ };
+
+ dma3: dma@2c0a3000 {
+ compatible = "arm,pl330", "arm,primecell";
+ reg = <0 0x2c0a3000 0 0x1000>;
+ interrupts = <0 160 4>,
+ <0 161 4>,
+ <0 162 4>,
+ <0 163 4>,
+ <0 164 4>,
+ <0 165 4>,
+ <0 166 4>,
+ <0 167 4>;
+ #dma-cells = <1>;
+ #stream-id-cells = <2>;
+ clocks = <&oscclk6a>;
+ clock-names = "apb_pclk";
+ };
+
+ oscclk6a: oscclk6a {
+ /* Reference 24MHz clock */
+ compatible = "fixed-clock";
+ #clock-cells = <0>;
+ clock-frequency = <24000000>;
+ clock-output-names = "oscclk6a";
+ };
+
smb {
compatible = "simple-bus";

@@ -244,4 +339,3 @@
/include/ "rtsm_ve-motherboard.dtsi"
};
};
-

--
Shannon

_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm

.

.

.

--
Shannon

.

--
Shannon

.

Quoted 6 lines
Quoted 3 lines
Quoted 2 lines

.

.

--
Shannon

--
Shannon

.

.