Skip to contentSkip to conversation

CSA patch v3

PATCH In review Monk Liu 2 participants 9 messages Jan 11, 2017 6 messages not in the archive

Subsystems (2)
lore
Share & tools
lore

Monk Liu

author
#0 lore raw

cover CSA patch v3

v2:
drop invoking gem_va and update CSA address in CS routines, v3:
reserve CSA bo and call vm_bo_rmv() to remove CSA mappings prior to vm_fini()

Monk Liu (3):
drm/amdgpu:new field members for SRIOV
drm/amdgpu:add new file for SRIOV
drm/amdgpu:invoke CSA functions

 drivers/gpu/drm/amd/amdgpu/Makefile        |  2 +-
 drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c     | 14 +++++
 drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 12 ++++
 drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c    | 14 +++++
 drivers/gpu/drm/amd/amdgpu/amdgpu_virt.c   | 95 ++++++++++++++++++++++++++++++
 drivers/gpu/drm/amd/amdgpu/amdgpu_virt.h   | 12 ++++
 drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h     |  3 +
 drivers/gpu/drm/amd/amdgpu/vi.c            |  3 +
 8 files changed, 154 insertions(+), 1 deletion(-)
 create mode 100644 drivers/gpu/drm/amd/amdgpu/amdgpu_virt.c
Signature · 6 lines
2.7.4

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

A message that is not in the archive (sent off-list or not yet received): 1484131406-19893-1-git-send-email-Monk.Liu-5C7GfCeVMHo@public.gmane.org

author
#2 lore raw

PATCH 1/3 drm/amdgpu:new field members for SRIOV

and implement CSA functions in this file

Change-Id Ife0eff7b13b8b5946f005a39f6ecb8db1cb72c38
---
 drivers/gpu/drm/amd/amdgpu/amdgpu_virt.h | 8 ++++++++
 drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h   | 3 +++
 2 files changed, 11 insertions(+)
drivers/gpu/drm/amd/amdgpu/amdgpu_virt.h +8 -0
index 0d821d9..b65bedc 100644
@@ -28,11 +28,19 @@
 #define AMDGPU_SRIOV_CAPS_ENABLE_IOV   (1 << 1) /* sr-iov is enabled on this GPU */
 #define AMDGPU_SRIOV_CAPS_IS_VF        (1 << 2) /* this GPU is a virtual function */
 #define AMDGPU_PASSTHROUGH_MODE        (1 << 3) /* thw whole GPU is pass through for VM */
+
 /* GPU virtualization */
 struct amdgpu_virt {
 	uint32_t caps;
+	uint32_t csa_size;
+	struct amdgpu_bo *csa_obj;
+	uint64_t csa_vmid0_addr;
+	uint64_t gds_vmid0_addr;
 };
 
+#define AMDGPU_CSA_SIZE    (8 * 1024)
+#define AMDGPU_CSA_VADDR   (AMDGPU_VA_RESERVED_SIZE - AMDGPU_CSA_SIZE)
+
 #define amdgpu_sriov_enabled(adev) \
 ((adev)->virt.caps & AMDGPU_SRIOV_CAPS_ENABLE_IOV)
 
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h +3 -0
index 42a629b..42f1ec1 100644
@@ -111,6 +111,9 @@ struct amdgpu_vm {
 
 	/* client id */
 	u64                     client_id;
+	/* each VM will map on CSA */
+	struct ttm_validate_buffer csa_tv;
+	struct amdgpu_bo_va *csa_bo_va;
 };
 
 struct amdgpu_vm_id {
Signature · 6 lines
2.7.4

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

A message that is not in the archive (sent off-list or not yet received): 1484131406-19893-2-git-send-email-Monk.Liu-5C7GfCeVMHo@public.gmane.org

#4 lore raw

Re: [PATCH 1/3] drm/amdgpu:new field members for SRIOV

Am 11.01.2017 um 11:43 schrieb Monk Liu:

Quoted 15 lines

and implement CSA functions in this file

Change-Id Ife0eff7b13b8b5946f005a39f6ecb8db1cb72c38
---
  drivers/gpu/drm/amd/amdgpu/amdgpu_virt.h | 8 ++++++++
  drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h   | 3 +++
  2 files changed, 11 insertions(+)
drivers/gpu/drm/amd/amdgpu/amdgpu_virt.h +0 -0
index 0d821d9..b65bedc 100644
@@ -28,11 +28,19 @@
  #define AMDGPU_SRIOV_CAPS_ENABLE_IOV   (1 << 1) /* sr-iov is enabled on this GPU */

drivers/gpu/drm/amd/amdgpu/amdgpu_virt.h line 38

Quoted 10 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_virt.h +5 -0
  #define AMDGPU_SRIOV_CAPS_IS_VF        (1 << 2) /* this GPU is a virtual function */
  #define AMDGPU_PASSTHROUGH_MODE        (1 << 3) /* thw whole GPU is pass through for VM */
+
  /* GPU virtualization */
  struct amdgpu_virt {
  	uint32_t caps;
+	uint32_t csa_size;
+	struct amdgpu_bo *csa_obj;
+	uint64_t csa_vmid0_addr;
+	uint64_t gds_vmid0_addr;

Similar to the GDS address inside the VMs I think the code actually using it should calculate it from the BO address.

Quoted 13 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_virt.h +3 -0
  };
 
+#define AMDGPU_CSA_SIZE    (8 * 1024)
+#define AMDGPU_CSA_VADDR   (AMDGPU_VA_RESERVED_SIZE - AMDGPU_CSA_SIZE)
+
  #define amdgpu_sriov_enabled(adev) \
  ((adev)->virt.caps & AMDGPU_SRIOV_CAPS_ENABLE_IOV)
 
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h +0 -0
index 42a629b..42f1ec1 100644

drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h struct amdgpu_vm { · line 115

Quoted 5 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h +2 -0
@@ -111,6 +111,9 @@ struct amdgpu_vm {
 
  	/* client id */
  	u64                     client_id;
+	/* each VM will map on CSA */
+	struct ttm_validate_buffer csa_tv;

The validate buffer is only temporarily used during mapping the CSA into the VM.

So please put that one the stack in that function.

Apart from that the patch looks good to me,
Christian.

drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h struct amdgpu_vm { · line 119

Quoted 4 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h +1 -0
+	struct amdgpu_bo_va *csa_bo_va;
  };
 
  struct amdgpu_vm_id {

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

A message that is not in the archive (sent off-list or not yet received): 1560cfc2-f700-fedd-c557-0b121c2f3cca-ANTagKRnAhcb1SvskN2V4Q@public.gmane.org

author
#6 lore raw

RE: [PATCH 1/3] drm/amdgpu:new field members for SRIOV

Quoted 1 line

Similar to the GDS address inside the VMs I think the code actually using it should calculate it from the BO address.

[ML] do you mean If some where want to use vmid0_gds_address I should just use vmid0_csa_addr + 4096 ?

Quoted 1 line

The validate buffer is only temporarily used during mapping the CSA into the VM.

So please put that one the stack in that function.

[ML] okay

Can I get your RB after above two place modified in patch ?

BR Monk

-----Original Message-----
From: amd-gfx [mailto:[email protected]] On Behalf Of Christian K?nig
Sent: Wednesday, January 11, 2017 8:46 PM
To: Liu, Monk <[email protected]>; [email protected]
Subject: Re: [PATCH 1/3] drm/amdgpu:new field members for SRIOV

Am 11.01.2017 um 11:43 schrieb Monk Liu:

Quoted 16 lines

and implement CSA functions in this file

Change-Id Ife0eff7b13b8b5946f005a39f6ecb8db1cb72c38
---
  drivers/gpu/drm/amd/amdgpu/amdgpu_virt.h | 8 ++++++++
  drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h   | 3 +++
  2 files changed, 11 insertions(+)
drivers/gpu/drm/amd/amdgpu/amdgpu_virt.h +0 -0
index 0d821d9..b65bedc 100644
@@ -28,11 +28,19 @@
  #define AMDGPU_SRIOV_CAPS_ENABLE_IOV   (1 << 1) /* sr-iov is enabled on this GPU */

drivers/gpu/drm/amd/amdgpu/amdgpu_virt.h line 38

Quoted 10 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_virt.h +5 -0
  #define AMDGPU_SRIOV_CAPS_IS_VF        (1 << 2) /* this GPU is a virtual function */
  #define AMDGPU_PASSTHROUGH_MODE        (1 << 3) /* thw whole GPU is pass through for VM */
+
  /* GPU virtualization */
  struct amdgpu_virt {
  	uint32_t caps;
+	uint32_t csa_size;
+	struct amdgpu_bo *csa_obj;
+	uint64_t csa_vmid0_addr;
+	uint64_t gds_vmid0_addr;

Similar to the GDS address inside the VMs I think the code actually using it should calculate it from the BO address.

Quoted 14 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_virt.h +3 -0
  };
 
+#define AMDGPU_CSA_SIZE    (8 * 1024)
+#define AMDGPU_CSA_VADDR   (AMDGPU_VA_RESERVED_SIZE - AMDGPU_CSA_SIZE)
+
  #define amdgpu_sriov_enabled(adev) \
  ((adev)->virt.caps & AMDGPU_SRIOV_CAPS_ENABLE_IOV)
 
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h +0 -0
index 42a629b..42f1ec1 100644

drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h struct amdgpu_vm { · line 115

Quoted 5 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h +2 -0
@@ -111,6 +111,9 @@ struct amdgpu_vm {
 
  	/* client id */
  	u64                     client_id;
+	/* each VM will map on CSA */
+	struct ttm_validate_buffer csa_tv;

The validate buffer is only temporarily used during mapping the CSA into the VM.

So please put that one the stack in that function.

Apart from that the patch looks good to me, Christian.

drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h struct amdgpu_vm { · line 119

Quoted 4 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h +1 -0
+	struct amdgpu_bo_va *csa_bo_va;
  };
 
  struct amdgpu_vm_id {

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

A message that is not in the archive (sent off-list or not yet received): BY2PR1201MB11106625F28ACE98D9AF99E984660-O28G1zQ8oGliQkyLPkmea2rFom/aUZj6nBOFsp37pqbUKgpGm//BTAC/[email protected]

#8 lore raw

Re: [PATCH 1/3] drm/amdgpu:new field members for SRIOV

Quoted 1 line

[ML] do you mean If some where want to use vmid0_gds_address I should just use vmid0_csa_addr + 4096 ?

Yes.

Quoted 1 line

Can I get your RB after above two place modified in patch ?

Yeah, with that fixed the patch is Reviewed-by: Christian König <[email protected]>.

Regards,
Christian.

Am 11.01.2017 um 15:49 schrieb Liu, Monk:

Quoted 76 lines
Quoted 1 line

Similar to the GDS address inside the VMs I think the code actually using it should calculate it from the BO address.

[ML] do you mean If some where want to use vmid0_gds_address I should just use vmid0_csa_addr + 4096 ?

Quoted 1 line

The validate buffer is only temporarily used during mapping the CSA into the VM.

So please put that one the stack in that function.

[ML] okay

Can I get your RB after above two place modified in patch ?

BR Monk

-----Original Message-----
From: amd-gfx [mailto:[email protected]] On Behalf Of Christian K?nig
Sent: Wednesday, January 11, 2017 8:46 PM
To: Liu, Monk <[email protected]>; [email protected]
Subject: Re: [PATCH 1/3] drm/amdgpu:new field members for SRIOV

Am 11.01.2017 um 11:43 schrieb Monk Liu:

Quoted 16 lines

and implement CSA functions in this file

Change-Id Ife0eff7b13b8b5946f005a39f6ecb8db1cb72c38
---
   drivers/gpu/drm/amd/amdgpu/amdgpu_virt.h | 8 ++++++++
   drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h   | 3 +++
   2 files changed, 11 insertions(+)
drivers/gpu/drm/amd/amdgpu/amdgpu_virt.h +0 -0
index 0d821d9..b65bedc 100644
@@ -28,11 +28,19 @@
   #define AMDGPU_SRIOV_CAPS_ENABLE_IOV   (1 << 1) /* sr-iov is enabled on this GPU */

drivers/gpu/drm/amd/amdgpu/amdgpu_virt.h line 38

Quoted 10 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_virt.h +5 -0
   #define AMDGPU_SRIOV_CAPS_IS_VF        (1 << 2) /* this GPU is a virtual function */
   #define AMDGPU_PASSTHROUGH_MODE        (1 << 3) /* thw whole GPU is pass through for VM */
+
   /* GPU virtualization */
   struct amdgpu_virt {
   	uint32_t caps;
+	uint32_t csa_size;
+	struct amdgpu_bo *csa_obj;
+	uint64_t csa_vmid0_addr;
+	uint64_t gds_vmid0_addr;

Similar to the GDS address inside the VMs I think the code actually using it should calculate it from the BO address.

Quoted 14 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_virt.h +3 -0
   };
 
+#define AMDGPU_CSA_SIZE    (8 * 1024)
+#define AMDGPU_CSA_VADDR   (AMDGPU_VA_RESERVED_SIZE - AMDGPU_CSA_SIZE)
+
   #define amdgpu_sriov_enabled(adev) \
   ((adev)->virt.caps & AMDGPU_SRIOV_CAPS_ENABLE_IOV)
 
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h +0 -0
index 42a629b..42f1ec1 100644

drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h struct amdgpu_vm { · line 115

Quoted 5 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h +2 -0
@@ -111,6 +111,9 @@ struct amdgpu_vm {
 
   	/* client id */
   	u64                     client_id;
+	/* each VM will map on CSA */
+	struct ttm_validate_buffer csa_tv;

The validate buffer is only temporarily used during mapping the CSA into the VM.

So please put that one the stack in that function.

Apart from that the patch looks good to me, Christian.

Quoted 3 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h +1 -0
+	struct amdgpu_bo_va *csa_bo_va;
   };
 
Quoted 10 lines

drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h struct amdgpu_vm { · line 119

Quoted 1 line
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h +0 -0
   struct amdgpu_vm_id {

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

author
#9 lore raw

PATCH 2/3 drm/amdgpu:add new file for SRIOV

In reply to a message not in the archive

for SRIOV usage, CSA is only used per device and each VM will map on it.

Change-Id Ibc63d87be59d7d89b96117e4d1b7e970a872580d
---
 drivers/gpu/drm/amd/amdgpu/Makefile      |  2 +-
 drivers/gpu/drm/amd/amdgpu/amdgpu_virt.c | 95 ++++++++++++++++++++++++++++++++
 drivers/gpu/drm/amd/amdgpu/amdgpu_virt.h |  4 ++
 3 files changed, 100 insertions(+), 1 deletion(-)
 create mode 100644 drivers/gpu/drm/amd/amdgpu/amdgpu_virt.c
drivers/gpu/drm/amd/amdgpu/Makefile +1 -1
index 4185b03..0b8e470 100644
@@ -30,7 +30,7 @@ amdgpu-y += amdgpu_device.o amdgpu_kms.o \
 	atombios_encoders.o amdgpu_sa.o atombios_i2c.o \
 	amdgpu_prime.o amdgpu_vm.o amdgpu_ib.o amdgpu_pll.o \
 	amdgpu_ucode.o amdgpu_bo_list.o amdgpu_ctx.o amdgpu_sync.o \
-	amdgpu_gtt_mgr.o amdgpu_vram_mgr.o
+	amdgpu_gtt_mgr.o amdgpu_vram_mgr.o amdgpu_virt.o
 
 # add asic specific block
 amdgpu-$(CONFIG_DRM_AMDGPU_CIK)+= cik.o cik_ih.o kv_smc.o kv_dpm.o \
drivers/gpu/drm/amd/amdgpu/amdgpu_virt.c new file +95 -0

95 lines not on this page · raw message

drivers/gpu/drm/amd/amdgpu/amdgpu_virt.h +4 -0
index b65bedc..a59b404 100644
@@ -62,4 +62,8 @@ static inline bool is_virtual_machine(void)
 #endif
 }
 
+struct amdgpu_vm;
+int amdgpu_allocate_static_csa(struct amdgpu_device *adev);
+int amdgpu_map_static_csa(struct amdgpu_device *adev, struct amdgpu_vm *vm);
+
 #endif
Signature · 6 lines
2.7.4

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

A message that is not in the archive (sent off-list or not yet received): 1484131406-19893-3-git-send-email-Monk.Liu-5C7GfCeVMHo@public.gmane.org

#11 lore raw

Re: [PATCH 2/3] drm/amdgpu:add new file for SRIOV

Am 11.01.2017 um 11:43 schrieb Monk Liu:

Quoted 76 lines

for SRIOV usage, CSA is only used per device and each VM will map on it.

Change-Id Ibc63d87be59d7d89b96117e4d1b7e970a872580d
---
  drivers/gpu/drm/amd/amdgpu/Makefile      |  2 +-
  drivers/gpu/drm/amd/amdgpu/amdgpu_virt.c | 95 ++++++++++++++++++++++++++++++++
  drivers/gpu/drm/amd/amdgpu/amdgpu_virt.h |  4 ++
  3 files changed, 100 insertions(+), 1 deletion(-)

create mode 100644 drivers/gpu/drm/amd/amdgpu/amdgpu_virt.c

drivers/gpu/drm/amd/amdgpu/Makefile +1 -1
index 4185b03..0b8e470 100644
@@ -30,7 +30,7 @@ amdgpu-y += amdgpu_device.o amdgpu_kms.o \
  	atombios_encoders.o amdgpu_sa.o atombios_i2c.o \
  	amdgpu_prime.o amdgpu_vm.o amdgpu_ib.o amdgpu_pll.o \
  	amdgpu_ucode.o amdgpu_bo_list.o amdgpu_ctx.o amdgpu_sync.o \
-	amdgpu_gtt_mgr.o amdgpu_vram_mgr.o
+	amdgpu_gtt_mgr.o amdgpu_vram_mgr.o amdgpu_virt.o
 
  # add asic specific block
  amdgpu-$(CONFIG_DRM_AMDGPU_CIK)+= cik.o cik_ih.o kv_smc.o kv_dpm.o \
drivers/gpu/drm/amd/amdgpu/amdgpu_virt.c new file +45 -0
new file mode 100644
index 0000000..0247a2e
@@ -0,0 +1,95 @@
+/*
+ * Copyright 2016 Advanced Micro Devices, Inc.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO EVENT SHALL
+ * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
+ * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
+ * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
+ * OTHER DEALINGS IN THE SOFTWARE.
+ *
+ */
+
+#include "amdgpu.h"
+
+int amdgpu_allocate_static_csa(struct amdgpu_device *adev)
+{
+	int r;
+	void *ptr;
+
+	r = amdgpu_bo_create_kernel(adev, adev->virt.csa_size, PAGE_SIZE,
+				AMDGPU_GEM_DOMAIN_VRAM, &adev->virt.csa_obj,
+				&adev->virt.csa_vmid0_addr, &ptr);
+	if (r)
+		return r;
+
+	/* gds part take the bottom half buffer */
+	adev->virt.gds_vmid0_addr = adev->virt.csa_vmid0_addr + PAGE_SIZE;
+	memset(ptr, 0, adev->virt.csa_size);
+	return 0;
+}
+
+/*
+ * amdgpu_map_static_csa should be called during amdgpu_vm_init
+ * it maps virtual address "AMDGPU_VA_RESERVED_SIZE - adev->virt.csa_size"

drivers/gpu/drm/amd/amdgpu/amdgpu_virt.c line 55

Quoted 10 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_virt.c new file +10 -0
+ * to this VM, and each command submission of GFX should use this virtual
+ * address within META_DATA init package to support SRIOV gfx preemption.
+ */
+
+int amdgpu_map_static_csa(struct amdgpu_device *adev, struct amdgpu_vm *vm)
+{
+	int r;
+	struct amdgpu_bo_va *bo_va;
+	struct ww_acquire_ctx ticket;
+	struct list_head list, duplicates;

Only a nit pick, but no need for the duplicates list here.

drivers/gpu/drm/amd/amdgpu/amdgpu_virt.c line 60

Quoted 5 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_virt.c new file +5 -0
+	struct amdgpu_bo_list_entry pd;
+
+	INIT_LIST_HEAD(&list);
+	INIT_LIST_HEAD(&duplicates);
+	INIT_LIST_HEAD(&vm->csa_tv.head);

As said on the other patch as well the csa_tv is only used temporary here, please move that to the stack as well.

With those two comments fixed the patch is Reviewed-by: Christian König <[email protected]>.

Regards,
Christian.

Quoted 40 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_virt.c new file +35 -0
+	vm->csa_tv.bo = &adev->virt.csa_obj->tbo;
+	vm->csa_tv.shared = true;
+
+	list_add(&vm->csa_tv.head, &list);
+	amdgpu_vm_get_pd_bo(vm, &list, &pd);
+
+	r = ttm_eu_reserve_buffers(&ticket, &list, true, &duplicates);
+	if (r) {
+		DRM_ERROR("failed to reserve CSA,PD BOs: err=%d\n", r);
+		return r;
+	}
+
+	bo_va = amdgpu_vm_bo_add(adev, vm, adev->virt.csa_obj);
+	if (!bo_va) {
+		ttm_eu_backoff_reservation(&ticket, &list);
+		DRM_ERROR("failed to create bo_va for static CSA\n");
+		return -ENOMEM;
+	}
+
+	r = amdgpu_vm_bo_map(adev, bo_va, AMDGPU_CSA_VADDR, 0,AMDGPU_CSA_SIZE,
+						AMDGPU_PTE_READABLE | AMDGPU_PTE_WRITEABLE |
+						AMDGPU_PTE_EXECUTABLE);
+
+	if (r) {
+		DRM_ERROR("failed to do bo_map on static CSA, err=%d\n", r);
+		amdgpu_vm_bo_rmv(adev, bo_va);
+		ttm_eu_backoff_reservation(&ticket, &list);
+		kfree(bo_va);
+		return r;
+	}
+
+	vm->csa_bo_va = bo_va;
+	ttm_eu_backoff_reservation(&ticket, &list);
+	return 0;
+}
drivers/gpu/drm/amd/amdgpu/amdgpu_virt.h +0 -0
index b65bedc..a59b404 100644

drivers/gpu/drm/amd/amdgpu/amdgpu_virt.h static inline bool is_virtual_machine(void) · line 69

Quoted 8 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_virt.h +4 -0
@@ -62,4 +62,8 @@ static inline bool is_virtual_machine(void)
  #endif
  }
 
+struct amdgpu_vm;
+int amdgpu_allocate_static_csa(struct amdgpu_device *adev);
+int amdgpu_map_static_csa(struct amdgpu_device *adev, struct amdgpu_vm *vm);
+
  #endif

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

author
#12 lore raw

PATCH 3/3 drm/amdgpu:invoke CSA functions

In reply to a message not in the archive
Change-Id I528c2f324830aaa21ab8d8250bc80a2a6bab33bd
---
 drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c     | 14 ++++++++++++++
 drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 12 ++++++++++++
 drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c    | 14 ++++++++++++++
 drivers/gpu/drm/amd/amdgpu/vi.c            |  3 +++
 4 files changed, 43 insertions(+)
drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c +14 -0
index 6159afc..328f1c7 100644
@@ -771,6 +771,20 @@ static int amdgpu_bo_vm_update_pte(struct amdgpu_cs_parser *p,
 	if (r)
 		return r;
 
+	if (amdgpu_sriov_vf(adev)) {
+		struct fence *f;
+		bo_va = vm->csa_bo_va;
+		BUG_ON(!bo_va);
+		r = amdgpu_vm_bo_update(adev, bo_va, false);
+		if (r)
+			return r;
+
+		f = bo_va->last_pt_update;
+		r = amdgpu_sync_fence(adev, &p->job->sync, f);
+		if (r)
+			return r;
+	}
+
 	if (p->bo_list) {
 		for (i = 0; i < p->bo_list->num_entries; i++) {
 			struct fence *f;
drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +12 -0
index f82919d..7c73dee 100644
@@ -1384,6 +1384,15 @@ static int amdgpu_init(struct amdgpu_device *adev)
 				return r;
 			}
 			adev->ip_blocks[i].status.hw = true;
+
+			/* right after GMC hw init, we create CSA */
+			if (amdgpu_sriov_vf(adev)) {
+				r = amdgpu_allocate_static_csa(adev);
+				if (r) {
+					DRM_ERROR("allocate CSA failed %d\n", r);
+					return r;
+				}
+			}
 		}
 	}
 
@@ -1517,6 +1526,9 @@ static int amdgpu_fini(struct amdgpu_device *adev)
 		adev->ip_blocks[i].status.late_initialized = false;
 	}
 
+	if (amdgpu_sriov_vf(adev))
+		amdgpu_bo_free_kernel(&adev->virt.csa_obj, &adev->virt.csa_vmid0_addr, NULL);
+
 	return 0;
 }
 
drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c +14 -0
index 47bc8e1..1421a9d 100644
@@ -650,6 +650,12 @@ int amdgpu_driver_open_kms(struct drm_device *dev, struct drm_file *file_priv)
 		goto out_suspend;
 	}
 
+	if (amdgpu_sriov_vf(adev)) {
+		r = amdgpu_map_static_csa(adev, &fpriv->vm);
+		if (r)
+			goto out_suspend;
+	}
+
 	mutex_init(&fpriv->bo_list_lock);
 	idr_init(&fpriv->bo_list_handles);
 
@@ -688,6 +694,14 @@ void amdgpu_driver_postclose_kms(struct drm_device *dev,
 	amdgpu_uvd_free_handles(adev, file_priv);
 	amdgpu_vce_free_handles(adev, file_priv);
 
+	if (amdgpu_sriov_vf(adev)) {
+		/* TODO: how to handle reserve failure */
+		BUG_ON(amdgpu_bo_reserve(adev->virt.csa_obj, false));
+		amdgpu_vm_bo_rmv(adev, fpriv->vm.csa_bo_va);
+		fpriv->vm.csa_bo_va = NULL;
+		amdgpu_bo_unreserve(adev->virt.csa_obj);
+	}
+
 	amdgpu_vm_fini(adev, &fpriv->vm);
 
 	idr_for_each_entry(&fpriv->bo_list_handles, list, handle)
drivers/gpu/drm/amd/amdgpu/vi.c +3 -0
index 7350a8f..1f307c3 100644
@@ -455,6 +455,9 @@ static void vi_detect_hw_virtualization(struct amdgpu_device *adev)
 		if (is_virtual_machine()) /* passthrough mode exclus sr-iov mode */
 			adev->virt.caps |= AMDGPU_PASSTHROUGH_MODE;
 	}
+
+	if (amdgpu_sriov_vf(adev))
+		adev->virt.csa_size = AMDGPU_CSA_SIZE; /* two page now for VI and AI */
 }
 
 static const struct amdgpu_allowed_register_entry tonga_allowed_read_registers[] = {
Signature · 6 lines
2.7.4

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

A message that is not in the archive (sent off-list or not yet received): 1484131406-19893-4-git-send-email-Monk.Liu-5C7GfCeVMHo@public.gmane.org

#14 lore raw

Re: [PATCH 3/3] drm/amdgpu:invoke CSA functions

Am 11.01.2017 um 11:43 schrieb Monk Liu:

Quoted 14 lines
Change-Id I528c2f324830aaa21ab8d8250bc80a2a6bab33bd
---
  drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c     | 14 ++++++++++++++
  drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 12 ++++++++++++
  drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c    | 14 ++++++++++++++
  drivers/gpu/drm/amd/amdgpu/vi.c            |  3 +++
  4 files changed, 43 insertions(+)
drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c +0 -0
index 6159afc..328f1c7 100644

drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c static int amdgpu_bo_vm_update_pte(struct amdgpu_cs_parser *p, · line 775

Quoted 5 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c +2 -0
@@ -771,6 +771,20 @@ static int amdgpu_bo_vm_update_pte(struct amdgpu_cs_parser *p,
  	if (r)
  		return r;
 
+	if (amdgpu_sriov_vf(adev)) {
+		struct fence *f;

A new line between declaration and code.

Quoted 22 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c +12 -0
+		bo_va = vm->csa_bo_va;
+		BUG_ON(!bo_va);
+		r = amdgpu_vm_bo_update(adev, bo_va, false);
+		if (r)
+			return r;
+
+		f = bo_va->last_pt_update;
+		r = amdgpu_sync_fence(adev, &p->job->sync, f);
+		if (r)
+			return r;
+	}
+
  	if (p->bo_list) {
  		for (i = 0; i < p->bo_list->num_entries; i++) {
  			struct fence *f;
drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +0 -0
index f82919d..7c73dee 100644
@@ -1384,6 +1384,15 @@ static int amdgpu_init(struct amdgpu_device *adev)
  				return r;
  			}

drivers/gpu/drm/amd/amdgpu/amdgpu_device.c static int amdgpu_init(struct amdgpu_device *adev) · line 1395

Quoted 10 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +9 -0
  			adev->ip_blocks[i].status.hw = true;
+
+			/* right after GMC hw init, we create CSA */
+			if (amdgpu_sriov_vf(adev)) {
+				r = amdgpu_allocate_static_csa(adev);
+				if (r) {
+					DRM_ERROR("allocate CSA failed %d\n", r);
+					return r;
+				}
+			}

We sooner or later need to fix those extras in the init order, but that is clearly not topic of this patch.

Quoted 31 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +3 -0
  		}
  	}
 
@@ -1517,6 +1526,9 @@ static int amdgpu_fini(struct amdgpu_device *adev)
  		adev->ip_blocks[i].status.late_initialized = false;
  	}
 
+	if (amdgpu_sriov_vf(adev))
+		amdgpu_bo_free_kernel(&adev->virt.csa_obj, &adev->virt.csa_vmid0_addr, NULL);
+
  	return 0;
  }
 
drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c +6 -0
index 47bc8e1..1421a9d 100644
@@ -650,6 +650,12 @@ int amdgpu_driver_open_kms(struct drm_device *dev, struct drm_file *file_priv)
  		goto out_suspend;
  	}
 
+	if (amdgpu_sriov_vf(adev)) {
+		r = amdgpu_map_static_csa(adev, &fpriv->vm);
+		if (r)
+			goto out_suspend;
+	}
+
  	mutex_init(&fpriv->bo_list_lock);
  	idr_init(&fpriv->bo_list_handles);
 

drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c void amdgpu_driver_postclose_kms(struct drm_device *dev, · line 699

Quoted 6 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c +3 -0
@@ -688,6 +694,14 @@ void amdgpu_driver_postclose_kms(struct drm_device *dev,
  	amdgpu_uvd_free_handles(adev, file_priv);
  	amdgpu_vce_free_handles(adev, file_priv);
 
+	if (amdgpu_sriov_vf(adev)) {
+		/* TODO: how to handle reserve failure */
+		BUG_ON(amdgpu_bo_reserve(adev->virt.csa_obj, false));

Reservation should never fail as long as you don't set the interruptible paramter to true, so using BUG_ON() should be ok here.

But I'm not 100% sure if BUG_ON() doesn't works like assert(), e.g. on a not debug build the compiler could optimize the call to amdgpu_bo_reserve away.

So better do:
r = amdgpu_bo_reserve(adev->virt.csa_obj, false);
BUG_ON(!r);

With that fixed the patch is Reviewed-by: Christian König <[email protected]>.

Regards,
Christian.

Quoted 13 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c +5 -0
+		amdgpu_vm_bo_rmv(adev, fpriv->vm.csa_bo_va);
+		fpriv->vm.csa_bo_va = NULL;
+		amdgpu_bo_unreserve(adev->virt.csa_obj);
+	}
+
  	amdgpu_vm_fini(adev, &fpriv->vm);
 
  	idr_for_each_entry(&fpriv->bo_list_handles, list, handle)
drivers/gpu/drm/amd/amdgpu/vi.c +0 -0
index 7350a8f..1f307c3 100644

drivers/gpu/drm/amd/amdgpu/vi.c static void vi_detect_hw_virtualization(struct amdgpu_device *adev) · line 463

Quoted 9 lines
drivers/gpu/drm/amd/amdgpu/vi.c +3 -0
@@ -455,6 +455,9 @@ static void vi_detect_hw_virtualization(struct amdgpu_device *adev)
  		if (is_virtual_machine()) /* passthrough mode exclus sr-iov mode */
  			adev->virt.caps |= AMDGPU_PASSTHROUGH_MODE;
  	}
+
+	if (amdgpu_sriov_vf(adev))
+		adev->virt.csa_size = AMDGPU_CSA_SIZE; /* two page now for VI and AI */
  }
 
  static const struct amdgpu_allowed_register_entry tonga_allowed_read_registers[] = {

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx