GART table recovery
Subsystems (2)
cover GART table recovery
gart table is stored in one bo which must be ready before gart init, but the shadow bo must be created after gart is ready, so they cannot be created at a same time. shado bo itself aslo is included in gart table, So shadow bo needs a synchronization after device init. After sync, the contents of bo and shadwo bo will be same, and be updated at a same time. Then we will be able to recover gart table from shadow bo when gpu full reset.
patch10 is a fix for memory leak.
Chunming Zhou (10):
drm/amdgpu: make need_backup generic
drm/amdgpu: implement gart late_init/fini drm/amdgpu: add gart_late_init/fini to gmc V7/8 drm/amdgpu: abstract amdgpu_bo_create_shadow drm/amdgpu: shadow gart table support
drm/amdgpu: make recover_bo_from_shadow be generic drm/amdgpu: implement gart recovery
drm/amdgpu: recover gart table first when full reset drm/amdgpu: sync gart table before initialization completed drm/amdgpu: fix memory leak of sched fence
drivers/gpu/drm/amd/amdgpu/amdgpu.h | 9 ++ drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 2 + drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c | 139 +++++++++++++++++++++++++++++ drivers/gpu/drm/amd/amdgpu/amdgpu_job.c | 2 +- drivers/gpu/drm/amd/amdgpu/amdgpu_object.c | 80 ++++++++++++++--- drivers/gpu/drm/amd/amdgpu/amdgpu_object.h | 9 ++ drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 50 ++--------- drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c | 39 +++++++- drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c | 40 ++++++++- 9 files changed, 304 insertions(+), 66 deletions(-)
Signature · 6 lines
1.9.1 _______________________________________________ amd-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/amd-gfx
A message that is not in the archive (sent off-list or not yet received): 1470124840-26170-1-git-send-email-David1.Zhou-5C7GfCeVMHo@public.gmane.org
PATCH 1/10 drm/amdgpu: make need_backup generic
It will be used other place.
--- drivers/gpu/drm/amd/amdgpu/amdgpu.h | 1 + drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 2 +- 2 files changed, 2 insertions(+), 1 deletion(-)
drivers/gpu/drm/amd/amdgpu/amdgpu.h +1 -0
@@ -975,6 +975,7 @@ void amdgpu_vm_manager_init(struct amdgpu_device *adev);void amdgpu_vm_manager_fini (struct amdgpu_device *adev);int amdgpu_vm_init (struct amdgpu_device *adev,struct amdgpu_vm *vm);void amdgpu_vm_fini (struct amdgpu_device *adev,struct amdgpu_vm *vm); +bool amdgpu_vm_need_backup (struct amdgpu_device *adev);void amdgpu_vm_get_pd_bo (struct amdgpu_vm *vm,struct list_head *validated,struct amdgpu_bo_list_entry *entry);
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c +1 -1
@@ -112,7 +112,7 @@ void amdgpu_vm_get_pd_bo(struct amdgpu_vm *vm,list_add (&entry->tv.head, validated); } -static bool amdgpu_vm_need_backup (struct amdgpu_device *adev) +bool amdgpu_vm_need_backup (struct amdgpu_device *adev) {if (adev->flags &AMD_IS_APU )return false;
Signature · 6 lines
1.9.1 _______________________________________________ amd-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/amd-gfx
PATCH 2/10 drm/amdgpu: implement gart late_init/fini
In reply to a message not in the archiveadd recovery entity to gart.
--- drivers/gpu/drm/amd/amdgpu/amdgpu.h | 3 +++ drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c | 18 ++++++++++++++++++ 2 files changed, 21 insertions(+)
drivers/gpu/drm/amd/amdgpu/amdgpu.h +3 -0
@@ -646,6 +646,7 @@ struct amdgpu_gart {#endif bool ready;const struct amdgpu_gart_funcs *gart_funcs; +struct amd_sched_entity recover_entity; };int amdgpu_gart_table_ram_alloc (struct amdgpu_device *adev);
@@ -656,6 +657,8 @@ int amdgpu_gart_table_vram_pin(struct amdgpu_device *adev);void amdgpu_gart_table_vram_unpin (struct amdgpu_device *adev);int amdgpu_gart_init (struct amdgpu_device *adev);void amdgpu_gart_fini (struct amdgpu_device *adev); +int amdgpu_gart_late_init (struct amdgpu_device *adev); +void amdgpu_gart_late_fini (struct amdgpu_device *adev);void amdgpu_gart_unbind (struct amdgpu_device *adev,unsigned offset,int pages); int amdgpu_gart_bind(struct amdgpu_device *adev, unsigned offset,
drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c +18 -0
@@ -363,3 +363,21 @@ void amdgpu_gart_fini(struct amdgpu_device *adev)#endif amdgpu_dummy_page_fini (adev); } + +int amdgpu_gart_late_init (struct amdgpu_device *adev) +{ +struct amd_sched_rq *rq; +struct amdgpu_ring *ring = adev->mman.buffer_funcs_ring; + + rq = &ring->sched.sched_rq[AMD_SCHED_PRIORITY_RECOVER ]; +return amd_sched_entity_init (&ring->sched, &adev->gart.recover_entity, + rq, amdgpu_sched_jobs); + +} + +void amdgpu_gart_late_fini (struct amdgpu_device *adev) +{ +struct amdgpu_ring *ring = adev->mman.buffer_funcs_ring; + +amd_sched_entity_fini (&ring->sched, &adev->gart.recover_entity); +}
Signature · 6 lines
1.9.1 _______________________________________________ amd-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/amd-gfx
PATCH 3/10 drm/amdgpu: add gart_late_init/fini to gmc V7/8
In reply to a message not in the archive--- drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c | 24 ++++++++++++++++++++---- drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c | 24 ++++++++++++++++++++---- 2 files changed, 40 insertions(+), 8 deletions(-)
drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c +20 -4
@@ -887,11 +887,26 @@ static int gmc_v7_0_early_init(void *handle)static int gmc_v7_0_late_init (void *handle) {struct amdgpu_device *adev = (struct amdgpu_device *)handle; +int r; -if (amdgpu_vm_fault_stop !=AMDGPU_VM_FAULT_STOP_ALWAYS ) -return amdgpu_irq_get (adev, &adev->mc.vm_fault,0 ); -else -return 0 ; + r =amdgpu_gart_late_init (adev); +if (r) +return r; + +if (amdgpu_vm_fault_stop !=AMDGPU_VM_FAULT_STOP_ALWAYS ) { + r =amdgpu_irq_get (adev, &adev->mc.vm_fault,0 ); +if (r) +amdgpu_gart_late_fini (adev); + } + +return r; +} + +static void gmc_v7_0_late_fini (void *handle) +{ +struct amdgpu_device *adev = (struct amdgpu_device *)handle; + +amdgpu_gart_late_fini (adev); }static int gmc_v7_0_sw_init (void *handle)
@@ -1242,6 +1257,7 @@ const struct amd_ip_funcs gmc_v7_0_ip_funcs = { .name = "gmc_v7_0", .early_init = gmc_v7_0_early_init, .late_init = gmc_v7_0_late_init, + .late_fini = gmc_v7_0_late_fini, .sw_init = gmc_v7_0_sw_init, .sw_fini = gmc_v7_0_sw_fini, .hw_init = gmc_v7_0_hw_init,
drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c +20 -4
@@ -876,11 +876,26 @@ static int gmc_v8_0_early_init(void *handle)static int gmc_v8_0_late_init (void *handle) {struct amdgpu_device *adev = (struct amdgpu_device *)handle; +int r; -if (amdgpu_vm_fault_stop !=AMDGPU_VM_FAULT_STOP_ALWAYS ) -return amdgpu_irq_get (adev, &adev->mc.vm_fault,0 ); -else -return 0 ; + r =amdgpu_gart_late_init (adev); +if (r) +return r; + +if (amdgpu_vm_fault_stop !=AMDGPU_VM_FAULT_STOP_ALWAYS ) { + r =amdgpu_irq_get (adev, &adev->mc.vm_fault,0 ); +if (r) +amdgpu_gart_late_fini (adev); + } + +return r; +} + +static void gmc_v8_0_late_fini (void *handle) +{ +struct amdgpu_device *adev = (struct amdgpu_device *)handle; + +amdgpu_gart_late_fini (adev); }#define mmMC_SEQ_MISC0_FIJI 0xA71
@@ -1434,6 +1449,7 @@ const struct amd_ip_funcs gmc_v8_0_ip_funcs = { .name = "gmc_v8_0", .early_init = gmc_v8_0_early_init, .late_init = gmc_v8_0_late_init, + .late_fini = gmc_v8_0_late_fini, .sw_init = gmc_v8_0_sw_init, .sw_fini = gmc_v8_0_sw_fini, .hw_init = gmc_v8_0_hw_init,
Signature · 6 lines
1.9.1 _______________________________________________ amd-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/amd-gfx
PATCH 4/10 drm/amdgpu: abstract amdgpu_bo_create_shadow
In reply to a message not in the archive--- drivers/gpu/drm/amd/amdgpu/amdgpu_object.c | 40 ++++++++++++++++++++---------- drivers/gpu/drm/amd/amdgpu/amdgpu_object.h | 3 +++ 2 files changed, 30 insertions(+), 13 deletions(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_object.c +27 -13
@@ -380,6 +380,32 @@ fail_free:return r; } +int amdgpu_bo_create_shadow (struct amdgpu_device *adev, +unsigned long size,int byte_align, +struct amdgpu_bo *bo) +{ +struct ttm_placement placement = {0 }; +struct ttm_place placements[AMDGPU_GEM_DOMAIN_MAX +1 ]; + +if (bo->shadow) +return 0 ; + + bo->flags |=AMDGPU_GEM_CREATE_SHADOW ; +memset (&placements,0 , + (AMDGPU_GEM_DOMAIN_MAX +1 ) *sizeof (struct ttm_place )); + +amdgpu_ttm_placement_init (adev, &placement, + placements,AMDGPU_GEM_DOMAIN_GTT , +AMDGPU_GEM_CREATE_CPU_GTT_USWC ); + +return amdgpu_bo_create_restricted (adev, size, byte_align, true, +AMDGPU_GEM_DOMAIN_GTT , +AMDGPU_GEM_CREATE_CPU_GTT_USWC , +NULL , &placement, + bo->tbo.resv, + &bo->shadow); +} + int amdgpu_bo_create(struct amdgpu_device *adev, unsigned long size, int byte_align, bool kernel, u32 domain, u64 flags,
@@ -404,19 +430,7 @@ int amdgpu_bo_create(struct amdgpu_device *adev,return r;if (flags &AMDGPU_GEM_CREATE_SHADOW ) { -memset (&placements,0 , - (AMDGPU_GEM_DOMAIN_MAX +1 ) *sizeof (struct ttm_place )); - -amdgpu_ttm_placement_init (adev, &placement, - placements,AMDGPU_GEM_DOMAIN_GTT , -AMDGPU_GEM_CREATE_CPU_GTT_USWC ); - - r =amdgpu_bo_create_restricted (adev, size, byte_align, kernel, -AMDGPU_GEM_DOMAIN_GTT , -AMDGPU_GEM_CREATE_CPU_GTT_USWC , -NULL , &placement, - (*bo_ptr)->tbo.resv, - &(*bo_ptr)->shadow); + r =amdgpu_bo_create_shadow (adev, size, byte_align, (*bo_ptr));if (r)amdgpu_bo_unref (bo_ptr); }else
drivers/gpu/drm/amd/amdgpu/amdgpu_object.h +3 -0
@@ -117,6 +117,9 @@ int amdgpu_bo_create(struct amdgpu_device *adev,struct sg_table *sg, struct reservation_object *resv, struct amdgpu_bo **bo_ptr); +int amdgpu_bo_create_shadow (struct amdgpu_device *adev, +unsigned long size,int byte_align, +struct amdgpu_bo *bo); int amdgpu_bo_create_restricted(struct amdgpu_device *adev, unsigned long size, int byte_align, bool kernel, u32 domain, u64 flags,
Signature · 6 lines
1.9.1 _______________________________________________ amd-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/amd-gfx
PATCH 5/10 drm/amdgpu: shadow gart table support
In reply to a message not in the archiveallocate gart shadow bo, and using shadow bo to backup gart table.
--- drivers/gpu/drm/amd/amdgpu/amdgpu.h | 3 ++ drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c | 47 +++++++++++++++++++++++++++++++- drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c | 15 ++++++++++ drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c | 16 +++++++++++ 4 files changed, 80 insertions(+), 1 deletion(-)
drivers/gpu/drm/amd/amdgpu/amdgpu.h +3 -0
@@ -638,6 +638,7 @@ struct amdgpu_gart {dma_addr_t table_addr;struct amdgpu_bo *robj;void *ptr; +void *shadow_ptr;unsigned num_gpu_pages;unsigned num_cpu_pages;unsigned table_size;
@@ -655,6 +656,8 @@ int amdgpu_gart_table_vram_alloc(struct amdgpu_device *adev);void amdgpu_gart_table_vram_free (struct amdgpu_device *adev);int amdgpu_gart_table_vram_pin (struct amdgpu_device *adev);void amdgpu_gart_table_vram_unpin (struct amdgpu_device *adev); +int amdgpu_gart_table_vram_shadow_pin (struct amdgpu_device *adev); +void amdgpu_gart_table_vram_shadow_unpin (struct amdgpu_device *adev);int amdgpu_gart_init (struct amdgpu_device *adev);void amdgpu_gart_fini (struct amdgpu_device *adev);int amdgpu_gart_late_init (struct amdgpu_device *adev);
drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c +46 -1
71 lines not on this page · raw message
drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c +15 -0
@@ -589,7 +589,21 @@ static int gmc_v7_0_gart_enable(struct amdgpu_device *adev) (unsigned )(adev->mc.gtt_size >>20 ), (unsigned long long)adev->gart.table_addr); adev->gart.ready = true; +if (amdgpu_vm_need_backup (adev) && adev->gart.robj) { + r =amdgpu_bo_create_shadow (adev, adev->gart.table_size, +PAGE_SIZE , adev->gart.robj); +if (r) + goto err; + r =amdgpu_gart_table_vram_shadow_pin (adev); +if (r) + goto err; + } +return 0 ; +err: +amdgpu_gart_table_vram_unpin (adev); + +return r; }static int gmc_v7_0_gart_init (struct amdgpu_device *adev)
@@ -634,6 +648,7 @@ static void gmc_v7_0_gart_disable(struct amdgpu_device *adev)WREG32 (mmVM_L2_CNTL, tmp);WREG32 (mmVM_L2_CNTL2,0 );amdgpu_gart_table_vram_unpin (adev); +amdgpu_gart_table_vram_shadow_unpin (adev); }/**
drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c +16 -0
@@ -704,7 +704,22 @@ static int gmc_v8_0_gart_enable(struct amdgpu_device *adev) (unsigned )(adev->mc.gtt_size >>20 ), (unsigned long long)adev->gart.table_addr); adev->gart.ready = true; + +if (amdgpu_vm_need_backup (adev) && adev->gart.robj) { + r =amdgpu_bo_create_shadow (adev, adev->gart.table_size, +PAGE_SIZE , adev->gart.robj); +if (r) + goto err; + r =amdgpu_gart_table_vram_shadow_pin (adev); +if (r) + goto err; + } +return 0 ; +err: +amdgpu_gart_table_vram_unpin (adev); + +return r; }static int gmc_v8_0_gart_init (struct amdgpu_device *adev)
@@ -749,6 +764,7 @@ static void gmc_v8_0_gart_disable(struct amdgpu_device *adev)WREG32 (mmVM_L2_CNTL, tmp);WREG32 (mmVM_L2_CNTL2,0 );amdgpu_gart_table_vram_unpin (adev); +amdgpu_gart_table_vram_shadow_unpin (adev); }/**
Signature · 6 lines
1.9.1 _______________________________________________ amd-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/amd-gfx
PATCH 6/10 drm/amdgpu: make recover_bo_from_shadow be generic
In reply to a message not in the archive--- drivers/gpu/drm/amd/amdgpu/amdgpu_object.c | 40 +++++++++++++++++++++++++ drivers/gpu/drm/amd/amdgpu/amdgpu_object.h | 6 ++++ drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 48 ++++-------------------------- 3 files changed, 51 insertions(+), 43 deletions(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_object.c +40 -0
@@ -439,6 +439,46 @@ int amdgpu_bo_create(struct amdgpu_device *adev,return r; } +int amdgpu_bo_recover_bo_from_shadow (struct amdgpu_device *adev, +struct amdgpu_ring *ring, +struct amd_sched_entity *entity, +struct amdgpu_bo *bo, +struct reservation_object *resv, +struct fence **fence) + +{ +int r; +uint64_t vram_addr, gtt_addr; + + r =amdgpu_bo_pin (bo,AMDGPU_GEM_DOMAIN_VRAM , &vram_addr); +if (r) { +DRM_ERROR ("Failed to pin bo object\n" ); + goto err1; + } + r =amdgpu_bo_pin (bo->shadow,AMDGPU_GEM_DOMAIN_GTT , >t_addr); +if (r) { +DRM_ERROR ("Failed to pin bo shadow object\n" ); + goto err2; + } + + r =reservation_object_reserve_shared (bo->tbo.resv); +if (r) + goto err3; + + r =amdgpu_copy_buffer (ring, entity, gtt_addr, + vram_addr,amdgpu_bo_size (bo), resv, fence); +if (!r) +amdgpu_bo_fence (bo, *fence, true); + +err3: +amdgpu_bo_unpin (bo->shadow); +err2: +amdgpu_bo_unpin (bo); +err1: + +return r; +} +int amdgpu_bo_kmap (struct amdgpu_bo *bo,void **ptr) {bool is_iomem;
drivers/gpu/drm/amd/amdgpu/amdgpu_object.h +6 -0
@@ -158,6 +158,12 @@ int amdgpu_bo_fault_reserve_notify(struct ttm_buffer_object *bo);void amdgpu_bo_fence (struct amdgpu_bo *bo,struct fence *fence,bool shared);u64 amdgpu_bo_gpu_offset (struct amdgpu_bo *bo); +int amdgpu_bo_recover_bo_from_shadow (struct amdgpu_device *adev, +struct amdgpu_ring *ring, +struct amd_sched_entity *entity, +struct amdgpu_bo *bo, +struct reservation_object *resv, +struct fence **fence);/* * sub allocation
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c +5 -43
66 lines not on this page · raw message
Signature · 6 lines
1.9.1 _______________________________________________ amd-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/amd-gfx
PATCH 7/10 drm/amdgpu: implement gart recovery
In reply to a message not in the archiverecover gart bo from its shadow bo.
--- drivers/gpu/drm/amd/amdgpu/amdgpu.h | 1 + drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c | 31 +++++++++++++++++++++++++++++++ 2 files changed, 32 insertions(+)
drivers/gpu/drm/amd/amdgpu/amdgpu.h +1 -0
@@ -667,6 +667,7 @@ void amdgpu_gart_unbind(struct amdgpu_device *adev, unsigned offset,int amdgpu_gart_bind (struct amdgpu_device *adev,unsigned offset,int pages,struct page **pagelist,dma_addr_t *dma_addr,uint32_t flags); +int amdgpu_gart_table_recover_from_shadow (struct amdgpu_device *adev);/* * GPU MC structures, functions & helpers
drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c +31 -0
@@ -135,6 +135,37 @@ int amdgpu_gart_table_vram_alloc(struct amdgpu_device *adev)return 0 ; } +int amdgpu_gart_table_recover_from_shadow (struct amdgpu_device *adev) +{ +struct amdgpu_ring *ring = adev->mman.buffer_funcs_ring; +struct fence *fence; +int r; + +if (!amdgpu_vm_need_backup (adev)) +return 0 ; +/* bo and shadow use same resv, so reserve one time */ + r =amdgpu_bo_reserve (adev->gart.robj, false); +if (unlikely (r !=0 )) +return r; + + r =amdgpu_bo_recover_bo_from_shadow (adev, ring, + &adev->gart.recover_entity, + adev->gart.robj, +NULL , &fence); +amdgpu_bo_unreserve (adev->gart.robj); +if (r) { +DRM_ERROR ("recover page table failed!\n" ); + goto err; + } + +if (fence) + r =fence_wait (fence, false); +fence_put (fence); + +err: +return r; +} +/** * amdgpu_gart_table_vram_pin - pin gart page table in vram *
Signature · 6 lines
1.9.1 _______________________________________________ amd-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/amd-gfx
PATCH 8/10 drm/amdgpu: recover gart table first when full reset
In reply to a message not in the archive--- drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 2 ++ 1 file changed, 2 insertions(+)
drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +2 -0
@@ -2191,6 +2191,8 @@ retry:kthread_unpark (ring->sched.thread); unpark_bits |=1 << ring->idx; } +DRM_INFO ("recover gart table first\n" ); +amdgpu_gart_table_recover_from_shadow (adev);spin_lock (&adev->vm_list_lock);list_for_each_entry_safe (vm, tmp, &adev->vm_list, list) {
Signature · 6 lines
1.9.1 _______________________________________________ amd-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/amd-gfx
PATCH 9/10 drm/amdgpu: sync gart table before initialization completed
In reply to a message not in the archiveSince the shadow is in GTT, shadow itslef pte isn't in shadow, We need to do sync before initialization is completed
--- drivers/gpu/drm/amd/amdgpu/amdgpu.h | 1 + drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c | 47 +++++++++++++++++++++++++++++++- 2 files changed, 47 insertions(+), 1 deletion(-)
drivers/gpu/drm/amd/amdgpu/amdgpu.h +1 -0
@@ -648,6 +648,7 @@ struct amdgpu_gart {bool ready;const struct amdgpu_gart_funcs *gart_funcs;struct amd_sched_entity recover_entity; +u64 shadow_gpu_addr; };int amdgpu_gart_table_ram_alloc (struct amdgpu_device *adev);
drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c +46 -1
66 lines not on this page · raw message
Signature · 6 lines
1.9.1 _______________________________________________ amd-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/amd-gfx
PATCH 10/10 drm/amdgpu: fix memory leak of sched fence
In reply to a message not in the archiveamdgpu_job_free_resources is already called by submit_job.
If it is called in run_job, the sched fence could be got twice in sa bo free, then memory leak happens.
--- drivers/gpu/drm/amd/amdgpu/amdgpu_job.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_job.c +1 -1
@@ -183,7 +183,7 @@ static struct fence *amdgpu_job_run(struct amd_sched_job *sched_job)/* if gpu reset, hw fence will be replaced here */ fence_put (job->fence); job->fence =fence_get (fence); -amdgpu_job_free_resources (job); +return fence; }
Signature · 6 lines
1.9.1 _______________________________________________ amd-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/amd-gfx
Well you have been hardworking during my vacation :)
Looks pretty good to me, but hope that I can get a closer look tomorrow.
Is there any particular order the three sets must be applied?
Regards,
Christian.
Am 02.08.2016 um 10:00 schrieb Chunming Zhou:
Quoted 16 lines
gart table is stored in one bo which must be ready before gart init, but the shadow bo must be created after gart is ready, so they cannot be created at a same time. shado bo itself aslo is included in gart table, So shadow bo needs a synchronization after device init. After sync, the contents of bo and shadwo bo will be same, and be updated at a same time. Then we will be able to recover gart table from shadow bo when gpu full reset.
patch10 is a fix for memory leak.
Chunming Zhou (10):
drm/amdgpu: makeneed_backupgeneric
drm/amdgpu: implement gart late_init/fini drm/amdgpu: add gart_late_init/fini to gmc V7/8 drm/amdgpu: abstractamdgpu_bo_create_shadowdrm/amdgpu: shadow gart table support
drm/amdgpu: makerecover_bo_from_shadowbe generic drm/amdgpu: implement gart recovery
drm/amdgpu: recover gart table first when full reset drm/amdgpu: sync gart table before initialization completed drm/amdgpu: fix memory leak of sched fence
Quoted 11 lines
drivers/gpu/drm/amd/amdgpu/amdgpu.h | 9 ++ drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 2 + drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c | 139 +++++++++++++++++++++++++++++ drivers/gpu/drm/amd/amdgpu/amdgpu_job.c | 2 +- drivers/gpu/drm/amd/amdgpu/amdgpu_object.c | 80 ++++++++++++++--- drivers/gpu/drm/amd/amdgpu/amdgpu_object.h | 9 ++ drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 50 ++--------- drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c | 39 +++++++- drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c | 40 ++++++++- 9 files changed, 304 insertions(+), 66 deletions(-)
_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
A message that is not in the archive (sent off-list or not yet received): f1b6c786-7e9c-ff61-1de9-299bc4daed15-ANTagKRnAhcb1SvskN2V4Q@public.gmane.org
Quoted 5 lines · Christian König
Well you have been hardworking during my vacation :)
Looks pretty good to me, but hope that I can get a closer look tomorrow.
Is there any particular order the three sets must be applied?
they are depending on my development order:
1. [PATCH 00/13] shadow page table support
2. [PATCH 00/11] add recovery entity and run queue
3. [PATCH 00/10] GART table recovery
Thanks,
David Zhou
Quoted 34 lines
Regards,
Christian.Am 02.08.2016 um 10:00 schrieb Chunming Zhou:
Quoted 28 lines
gart table is stored in one bo which must be ready before gart init, but the shadow bo must be created after gart is ready, so they cannot be created at a same time. shado bo itself aslo is included in gart table, So shadow bo needs a synchronization after device init. After sync, the contents of bo and shadwo bo will be same, and be updated at a same time. Then we will be able to recover gart table from shadow bo when gpu full reset.
patch10 is a fix for memory leak.
Chunming Zhou (10):
drm/amdgpu: makeneed_backupgeneric
drm/amdgpu: implement gart late_init/fini drm/amdgpu: add gart_late_init/fini to gmc V7/8 drm/amdgpu: abstractamdgpu_bo_create_shadowdrm/amdgpu: shadow gart table support
drm/amdgpu: makerecover_bo_from_shadowbe generic drm/amdgpu: implement gart recovery
drm/amdgpu: recover gart table first when full reset drm/amdgpu: sync gart table before initialization completed drm/amdgpu: fix memory leak of sched fencedrivers/gpu/drm/amd/amdgpu/amdgpu.h | 9 ++ drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 2 + drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c | 139 +++++++++++++++++++++++++++++ drivers/gpu/drm/amd/amdgpu/amdgpu_job.c | 2 +- drivers/gpu/drm/amd/amdgpu/amdgpu_object.c | 80 ++++++++++++++--- drivers/gpu/drm/amd/amdgpu/amdgpu_object.h | 9 ++ drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 50 ++--------- drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c | 39 +++++++- drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c | 40 ++++++++- 9 files changed, 304 insertions(+), 66 deletions(-)
_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
Well patch #10 is incorrect. The SA BO will be set to NULL by amdgpu_sa_bo_free(), so it can't be freed twice and so you can't reference the fence twice.
Additional to that the whole approach here of restoring the GART from the backup using the SDMA won't work either. For the SDMA to work you need the GART to access the ring buffer.
So you run into a chicken and egg problem here, for the ring buffer to work you need the GART and for the GART backup to work you need the ring buffer.
We should just restore the GART content from the housekeeping structure instead. Going to evaluate if and how that might be possible.
Regards,
Christian.
Am 02.08.2016 um 10:00 schrieb Chunming Zhou:
Quoted 16 lines
gart table is stored in one bo which must be ready before gart init, but the shadow bo must be created after gart is ready, so they cannot be created at a same time. shado bo itself aslo is included in gart table, So shadow bo needs a synchronization after device init. After sync, the contents of bo and shadwo bo will be same, and be updated at a same time. Then we will be able to recover gart table from shadow bo when gpu full reset.
patch10 is a fix for memory leak.
Chunming Zhou (10):
drm/amdgpu: makeneed_backupgeneric
drm/amdgpu: implement gart late_init/fini drm/amdgpu: add gart_late_init/fini to gmc V7/8 drm/amdgpu: abstractamdgpu_bo_create_shadowdrm/amdgpu: shadow gart table support
drm/amdgpu: makerecover_bo_from_shadowbe generic drm/amdgpu: implement gart recovery
drm/amdgpu: recover gart table first when full reset drm/amdgpu: sync gart table before initialization completed drm/amdgpu: fix memory leak of sched fence
Quoted 11 lines
drivers/gpu/drm/amd/amdgpu/amdgpu.h | 9 ++ drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 2 + drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c | 139 +++++++++++++++++++++++++++++ drivers/gpu/drm/amd/amdgpu/amdgpu_job.c | 2 +- drivers/gpu/drm/amd/amdgpu/amdgpu_object.c | 80 ++++++++++++++--- drivers/gpu/drm/amd/amdgpu/amdgpu_object.h | 9 ++ drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 50 ++--------- drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c | 39 +++++++- drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c | 40 ++++++++- 9 files changed, 304 insertions(+), 66 deletions(-)
_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
A message that is not in the archive (sent off-list or not yet received): 54bb3255-2dda-f6ad-3682-8e4396ec932a-ANTagKRnAhcb1SvskN2V4Q@public.gmane.org
Quoted 1 line · Christian König
Well patch #10 is incorrect. The SA BO will be set to NULL by
amdgpu_sa_bo_free(), so it can't be freed twice and so you can't reference the fence twice.
I see.
But amdgpu_job_free_resources still shouldn't be called twice, right? That's an obvious duplication although it seems no effect now. Is there any other reason?
Quoted 4 lines
Additional to that the whole approach here of restoring the GART from the backup using the SDMA won't work either. For the SDMA to work you need the GART to access the ring buffer.
So you run into a chicken and egg problem here, for the ring buffer to work you need the GART and for the GART backup to work you need the ring buffer.
Good catch, ring buffer is a GTT buffer as well.
Then Can we use memcpy to copy GTT to VRAM? Fortunately, the GART bo is only one bo.
Regards,
David Zhou
Quoted 37 lines
We should just restore the GART content from the housekeeping structure instead. Going to evaluate if and how that might be possible.
Regards,
Christian.Am 02.08.2016 um 10:00 schrieb Chunming Zhou:
Quoted 16 lines
gart table is stored in one bo which must be ready before gart init, but the shadow bo must be created after gart is ready, so they cannot be created at a same time. shado bo itself aslo is included in gart table, So shadow bo needs a synchronization after device init. After sync, the contents of bo and shadwo bo will be same, and be updated at a same time. Then we will be able to recover gart table from shadow bo when gpu full reset.
patch10 is a fix for memory leak.
Chunming Zhou (10):
drm/amdgpu: makeneed_backupgeneric
drm/amdgpu: implement gart late_init/fini drm/amdgpu: add gart_late_init/fini to gmc V7/8 drm/amdgpu: abstractamdgpu_bo_create_shadowdrm/amdgpu: shadow gart table support
drm/amdgpu: makerecover_bo_from_shadowbe generic drm/amdgpu: implement gart recovery
drm/amdgpu: recover gart table first when full reset drm/amdgpu: sync gart table before initialization completed drm/amdgpu: fix memory leak of sched fenceQuoted 12 lines
drivers/gpu/drm/amd/amdgpu/amdgpu.h | 9 ++ drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 2 + drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c | 139 +++++++++++++++++++++++++++++ drivers/gpu/drm/amd/amdgpu/amdgpu_job.c | 2 +- drivers/gpu/drm/amd/amdgpu/amdgpu_object.c | 80 ++++++++++++++--- drivers/gpu/drm/amd/amdgpu/amdgpu_object.h | 9 ++ drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 50 ++--------- drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c | 39 +++++++- drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c | 40 ++++++++- 9 files changed, 304 insertions(+), 66 deletions(-)
Quoted 4 lines
_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
A message that is not in the archive (sent off-list or not yet received): [email protected]
Am 04.08.2016 um 05:35 schrieb zhoucm1:
Quoted 6 lines · Christian König
Quoted 1 line
Well patch #10 is incorrect. The SA BO will be set to NULL by
amdgpu_sa_bo_free(), so it can't be freed twice and so you can't reference the fence twice.I see.
Butamdgpu_job_free_resourcesstill shouldn't be called twice, right? That's an obvious duplication although it seems no effect now. Is there any other reason?
It's actually called from a couple of different locations:
1. From the CS path in amdgpu_cs.c as soon as we have a scheduler fence.
2. From the amdgpu_job_submit() path as soon as we have a scheduler fence.
3. From amdgpu_job_run() after submitting the job to the hardware ring.
4. From amdgpu_job_free(), this is for direct submissions or for freeing the job when something went wrong.
Thinking about it you could be right and we could probably drop the one in amdgpu_job_run(), because amdgpu_job_submit() should have already taken care of that. But I'm not 100% sure of that.
Quoted 8 lines
Quoted 5 lines
Additional to that the whole approach here of restoring the GART from the backup using the SDMA won't work either. For the SDMA to work you need the GART to access the ring buffer.
So you run into a chicken and egg problem here, for the ring buffer to work you need the GART and for the GART backup to work you need the ring buffer.
Good catch, ring buffer is a GTT buffer as well.
Then Can we use memcpy to copy GTT to VRAM? Fortunately, the GART bo is only one bo.
Yeah that is what we did with radeon as well. Unfortunately the double housekeeping costs quite a bunch of memory.
And actually we have the exactly same information in the TTM MM as well, we would just need to bind all BOs again.
Give me a day or two to double check that. Might be that the solution is rather simple.
Regards,
Christian.
Quoted 46 lines
Regards,
David ZhouQuoted 41 lines
We should just restore the GART content from the housekeeping structure instead. Going to evaluate if and how that might be possible.
Regards,
Christian.Am 02.08.2016 um 10:00 schrieb Chunming Zhou:
Quoted 16 lines
gart table is stored in one bo which must be ready before gart init, but the shadow bo must be created after gart is ready, so they cannot be created at a same time. shado bo itself aslo is included in gart table, So shadow bo needs a synchronization after device init. After sync, the contents of bo and shadwo bo will be same, and be updated at a same time. Then we will be able to recover gart table from shadow bo when gpu full reset.
patch10 is a fix for memory leak.
Chunming Zhou (10):
drm/amdgpu: makeneed_backupgeneric
drm/amdgpu: implement gart late_init/fini drm/amdgpu: add gart_late_init/fini to gmc V7/8 drm/amdgpu: abstractamdgpu_bo_create_shadowdrm/amdgpu: shadow gart table support
drm/amdgpu: makerecover_bo_from_shadowbe generic drm/amdgpu: implement gart recovery
drm/amdgpu: recover gart table first when full reset drm/amdgpu: sync gart table before initialization completed drm/amdgpu: fix memory leak of sched fenceQuoted 12 lines
drivers/gpu/drm/amd/amdgpu/amdgpu.h | 9 ++ drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 2 + drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c | 139 +++++++++++++++++++++++++++++ drivers/gpu/drm/amd/amdgpu/amdgpu_job.c | 2 +- drivers/gpu/drm/amd/amdgpu/amdgpu_object.c | 80 ++++++++++++++--- drivers/gpu/drm/amd/amdgpu/amdgpu_object.h | 9 ++ drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 50 ++--------- drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c | 39 +++++++- drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c | 40 ++++++++- 9 files changed, 304 insertions(+), 66 deletions(-)_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
A message that is not in the archive (sent off-list or not yet received): 077bb11d-957d-c6f2-2f87-248fbc19304a-ANTagKRnAhcb1SvskN2V4Q@public.gmane.org
Quoted 23 lines · Christian König
Am 04.08.2016 um 05:35 schrieb zhoucm1:
Quoted 6 lines · Christian König
Quoted 1 line
Well patch #10 is incorrect. The SA BO will be set to NULL by
amdgpu_sa_bo_free(), so it can't be freed twice and so you can't reference the fence twice.I see.
Butamdgpu_job_free_resourcesstill shouldn't be called twice, right? That's an obvious duplication although it seems no effect now. Is there any other reason?It's actually called from a couple of different locations:
1. From the CS path inamdgpu_cs.cas soon as we have a scheduler fence.
2. From theamdgpu_job_submit()path as soon as we have a scheduler fence.
3. Fromamdgpu_job_run()after submitting the job to the hardware ring.
4. Fromamdgpu_job_free(), this is for direct submissions or for freeing the job when something went wrong.Thinking about it you could be right and we could probably drop the one in
amdgpu_job_run(), becauseamdgpu_job_submit()should have already taken care of that. But I'm not 100% sure of that.Quoted 7 lines
Quoted 5 lines
Additional to that the whole approach here of restoring the GART from the backup using the SDMA won't work either. For the SDMA to work you need the GART to access the ring buffer.
So you run into a chicken and egg problem here, for the ring buffer to work you need the GART and for the GART backup to work you need the ring buffer.
Good catch, ring buffer is a GTT buffer as well.
Quoted 7 lines
Quoted 1 line
Then Can we use memcpy to copy GTT to VRAM? Fortunately, the GART bo is only one bo.
Yeah that is what we did with radeon as well. Unfortunately the double housekeeping costs quite a bunch of memory.
And actually we have the exactly same information in the TTM MM as well, we would just need to bind all BOs again.
Give me a day or two to double check that. Might be that the solution is rather simple.
How about this? Do you have any better idea for it? or just change ring buffer to VRAM bo?
Regards,
David Zhou
Quoted 51 lines
Regards,
Christian.Quoted 46 lines
Regards,
David ZhouQuoted 41 lines
We should just restore the GART content from the housekeeping structure instead. Going to evaluate if and how that might be possible.
Regards,
Christian.Am 02.08.2016 um 10:00 schrieb Chunming Zhou:
Quoted 16 lines
gart table is stored in one bo which must be ready before gart init, but the shadow bo must be created after gart is ready, so they cannot be created at a same time. shado bo itself aslo is included in gart table, So shadow bo needs a synchronization after device init. After sync, the contents of bo and shadwo bo will be same, and be updated at a same time. Then we will be able to recover gart table from shadow bo when gpu full reset.
patch10 is a fix for memory leak.
Chunming Zhou (10):
drm/amdgpu: makeneed_backupgeneric
drm/amdgpu: implement gart late_init/fini drm/amdgpu: add gart_late_init/fini to gmc V7/8 drm/amdgpu: abstractamdgpu_bo_create_shadowdrm/amdgpu: shadow gart table support
drm/amdgpu: makerecover_bo_from_shadowbe generic drm/amdgpu: implement gart recovery
drm/amdgpu: recover gart table first when full reset drm/amdgpu: sync gart table before initialization completed drm/amdgpu: fix memory leak of sched fenceQuoted 12 lines
drivers/gpu/drm/amd/amdgpu/amdgpu.h | 9 ++ drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 2 + drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c | 139 +++++++++++++++++++++++++++++ drivers/gpu/drm/amd/amdgpu/amdgpu_job.c | 2 +- drivers/gpu/drm/amd/amdgpu/amdgpu_object.c | 80 ++++++++++++++--- drivers/gpu/drm/amd/amdgpu/amdgpu_object.h | 9 ++ drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 50 ++--------- drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c | 39 +++++++- drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c | 40 ++++++++- 9 files changed, 304 insertions(+), 66 deletions(-)_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
A message that is not in the archive (sent off-list or not yet received): [email protected]
Am 18.08.2016 um 10:50 schrieb zhoucm1:
Quoted 26 lines · Christian König
Quoted 23 lines
Am 04.08.2016 um 05:35 schrieb zhoucm1:
Quoted 6 lines · Christian König
Quoted 1 line
Well patch #10 is incorrect. The SA BO will be set to NULL by
amdgpu_sa_bo_free(), so it can't be freed twice and so you can't reference the fence twice.I see.
Butamdgpu_job_free_resourcesstill shouldn't be called twice, right? That's an obvious duplication although it seems no effect now. Is there any other reason?It's actually called from a couple of different locations:
1. From the CS path inamdgpu_cs.cas soon as we have a scheduler fence.
2. From theamdgpu_job_submit()path as soon as we have a scheduler fence.
3. Fromamdgpu_job_run()after submitting the job to the hardware ring.
4. Fromamdgpu_job_free(), this is for direct submissions or for freeing the job when something went wrong.Thinking about it you could be right and we could probably drop the one in
amdgpu_job_run(), becauseamdgpu_job_submit()should have already taken care of that. But I'm not 100% sure of that.Quoted 7 lines
Quoted 5 lines
Additional to that the whole approach here of restoring the GART from the backup using the SDMA won't work either. For the SDMA to work you need the GART to access the ring buffer.
So you run into a chicken and egg problem here, for the ring buffer to work you need the GART and for the GART backup to work you need the ring buffer.
Good catch, ring buffer is a GTT buffer as well.
Quoted 8 lines
Quoted 7 lines
Quoted 1 line
Then Can we use memcpy to copy GTT to VRAM? Fortunately, the GART bo is only one bo.
Yeah that is what we did with radeon as well. Unfortunately the double housekeeping costs quite a bunch of memory.
And actually we have the exactly same information in the TTM MM as well, we would just need to bind all BOs again.
Give me a day or two to double check that. Might be that the solution is rather simple.
How about this? Do you have any better idea for it?
Sorry for not answering earlier, but you know humans have only one head and two hands to type :)
The informations needed to restore the GART is already stored in the amdgpu_ttm_tt structures.
We just need to link them together in amdgpu_ttm_backend_bind() and unlink them in amdgpu_ttm_backend_unbind() to be able to restore the GART table after a reset.
Quoted 1 line
or just change ring buffer to VRAM bo?
Interesting idea, but considering how much overhead it has to write to VRAM from the CPU it clearly wouldn't be such a good idea to do in general.
Regards,
Christian.
Quoted 55 lines
Regards,
David ZhouQuoted 51 lines
Regards,
Christian.Quoted 46 lines
Regards,
David ZhouQuoted 41 lines
We should just restore the GART content from the housekeeping structure instead. Going to evaluate if and how that might be possible.
Regards,
Christian.Am 02.08.2016 um 10:00 schrieb Chunming Zhou:
Quoted 16 lines
gart table is stored in one bo which must be ready before gart init, but the shadow bo must be created after gart is ready, so they cannot be created at a same time. shado bo itself aslo is included in gart table, So shadow bo needs a synchronization after device init. After sync, the contents of bo and shadwo bo will be same, and be updated at a same time. Then we will be able to recover gart table from shadow bo when gpu full reset.
patch10 is a fix for memory leak.
Chunming Zhou (10):
drm/amdgpu: makeneed_backupgeneric
drm/amdgpu: implement gart late_init/fini drm/amdgpu: add gart_late_init/fini to gmc V7/8 drm/amdgpu: abstractamdgpu_bo_create_shadowdrm/amdgpu: shadow gart table support
drm/amdgpu: makerecover_bo_from_shadowbe generic drm/amdgpu: implement gart recovery
drm/amdgpu: recover gart table first when full reset drm/amdgpu: sync gart table before initialization completed drm/amdgpu: fix memory leak of sched fenceQuoted 12 lines
drivers/gpu/drm/amd/amdgpu/amdgpu.h | 9 ++ drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 2 + drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c | 139 +++++++++++++++++++++++++++++ drivers/gpu/drm/amd/amdgpu/amdgpu_job.c | 2 +- drivers/gpu/drm/amd/amdgpu/amdgpu_object.c | 80 ++++++++++++++--- drivers/gpu/drm/amd/amdgpu/amdgpu_object.h | 9 ++ drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 50 ++--------- drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c | 39 +++++++- drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c | 40 ++++++++- 9 files changed, 304 insertions(+), 66 deletions(-)_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
Quoted 4 lines
_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx