radv: Initial implementation of VK_EXT_queue_global_priority
PATCH 1/1 radv: Initial implementation of VK_EXT_queue_global_priority
This patch series implements VK_EXT_queue_global_priority, a vulkan extension that focuses on providing scheduling guarantees.
For further information on the use case refer to:
https://lists.freedesktop.org/archives/amd-gfx/2016-December/004068.html
For some perf data on this initial implementation:
https://lists.freedesktop.org/archives/amd-gfx/2016-December/004257.html
Test apps available at:
https://github.com/lostgoat/Vulkan
Related amdgpu and libdrm_amdgpu patches can be found on the amd-gfx mailing list.
Additionaly, results for vk-cts -n *semaphore* on an RX480:
Test run totals:
Passed: 1821/4046 (45.0%)
Failed: 0/4046 (0.0%)
Not supported: 2225/4046 (55.0%)
Warnings: 0/4046 (0.0%)
_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
A message that is not in the archive (sent off-list or not yet received): 20170104022229.7747-1-andresx7-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org
PATCH 1/2 radv: use a winsys context per-queue, instead of per device
Queues are independent execution streams. The vulkan spec provides no ordering guarantees for different queues.
By using a single context for all queues, we are forcing all commands into an unecessary FIFO ordering.
This change is a preparation step to allow our-of-ordering scheduling of certain work tasks.
As a side effect, vkQueueWaitIdle will be marginally faster. Previously due to the shared context, vkQueueWaitIdle was equivalent to vkDeviceWaitIdle.
--- src/amd/vulkan/radv_device.c | 35 ++++++++++++++++++++--------------- src/amd/vulkan/radv_private.h | 2 +- src/amd/vulkan/radv_wsi.c | 2 +- 3 files changed, 22 insertions(+), 17 deletions(-)
src/amd/vulkan/radv_device.c +20 -15
@@ -651,7 +651,7 @@ void radv_GetPhysicalDeviceMemoryProperties( }; } -static void +static int radv_queue_init (struct radv_device *device,struct radv_queue *queue,int queue_family_index,int idx) {
@@ -659,11 +659,19 @@ radv_queue_init(struct radv_device *device, struct radv_queue *queue, queue->device = device; queue->queue_family_index = queue_family_index; queue->queue_idx = idx; + + queue->hw_ctx = device->ws->ctx_create (device->ws); +if (!queue->hw_ctx) +return VK_ERROR_OUT_OF_HOST_MEMORY ; + +return VK_SUCCESS ; }static void radv_queue_finish (struct radv_queue *queue) { +if (queue->hw_ctx) + queue->device->ws->ctx_destroy (queue->hw_ctx); } VkResult radv_CreateDevice(
@@ -718,23 +726,21 @@ VkResult radv_CreateDevice( goto fail; } - device->queue_count[qfi] = queue_create->queueCount; +memset (device->queues[qfi],0 , queue_create->queueCount *sizeof (struct radv_queue )); -for (unsigned q =0 ; q < queue_create->queueCount; q++) -radv_queue_init (device, &device->queues[qfi][q], qfi, q); - } + device->queue_count[qfi] = queue_create->queueCount; - device->hw_ctx = device->ws->ctx_create (device->ws); -if (!device->hw_ctx) { - result =VK_ERROR_OUT_OF_HOST_MEMORY ; - goto fail; +for (unsigned q =0 ; q < queue_create->queueCount; q++) { + result =radv_queue_init (device, &device->queues[qfi][q], qfi, q); +if (result !=VK_SUCCESS ) + goto fail; + } } result =radv_device_init_meta (device); -if (result !=VK_SUCCESS ) { - device->ws->ctx_destroy (device->hw_ctx); +if (result !=VK_SUCCESS ) goto fail; - } + device->allow_fast_clears =env_var_as_boolean ("RADV_FAST_CLEARS" , false); device->allow_dcc = !env_var_as_boolean ("RADV_DCC_DISABLE" , false); device->shader_stats_dump =env_var_as_boolean ("RADV_SHADER_STATS" , false);
@@ -780,7 +786,6 @@ void radv_DestroyDevice( {RADV_FROM_HANDLE (radv_device, device, _device); - device->ws->ctx_destroy (device->hw_ctx);for (unsigned i =0 ; i <RADV_MAX_QUEUE_FAMILIES ; i++) {for (unsigned q =0 ; q < device->queue_count[i]; q++)radv_queue_finish (&device->queues[i][q]);
@@ -878,7 +883,7 @@ VkResult radv_QueueSubmit(RADV_FROM_HANDLE (radv_queue, queue, _queue);RADV_FROM_HANDLE (radv_fence, fence, _fence);struct radeon_winsys_fence *base_fence = fence ? fence->fence :NULL ; -struct radeon_winsys_ctx *ctx = queue->device->hw_ctx; +struct radeon_winsys_ctx *ctx = queue->hw_ctx;int ret;for (uint32_t i =0 ; i < submitCount; i++) {
@@ -929,7 +934,7 @@ VkResult radv_QueueWaitIdle( {RADV_FROM_HANDLE (radv_queue, queue, _queue); - queue->device->ws->ctx_wait_idle (queue->device->hw_ctx, + queue->device->ws->ctx_wait_idle (queue->hw_ctx,radv_queue_family_to_ring (queue->queue_family_index), queue->queue_idx);return VK_SUCCESS ;
src/amd/vulkan/radv_private.h +1 -1
@@ -456,6 +456,7 @@ enum ring_type radv_queue_family_to_ring(int f);struct radv_queue {VK_LOADER_DATA _loader_data;struct radv_device * device; +struct radeon_winsys_ctx *hw_ctx;int queue_family_index;int queue_idx; };
@@ -467,7 +468,6 @@ struct radv_device {struct radv_instance * instance;struct radeon_winsys *ws; -struct radeon_winsys_ctx *hw_ctx;struct radv_meta_state meta_state;
src/amd/vulkan/radv_wsi.c +1 -1
@@ -364,7 +364,7 @@ VkResult radv_QueuePresentKHR(RADV_FROM_HANDLE (radv_fence, fence, swapchain->fences[0 ]);struct radeon_winsys_fence *base_fence = fence->fence; -struct radeon_winsys_ctx *ctx = queue->device->hw_ctx; +struct radeon_winsys_ctx *ctx = queue->hw_ctx; queue->device->ws->cs_submit(ctx, queue->queue_idx, &queue->device->empty_cs[queue->queue_family_index], 1,
Signature · 6 lines
2.9.3 _______________________________________________ amd-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/amd-gfx
PATCH 2/2 radv: define and implement VK_EXT_queue_global_priority
In reply to a message not in the archiveAdd a new extension VK_EXT_queue_global_priority, which allows the caller to elevate a queue's priority above all other queues in the system.
This extension aims to provide a mechanism for compositors to have achieve a guaranteed quality of service, even when the hardware may be loaded by a game or application.
--- docs/specs/VK_EXT_queue_global_priority.txt | 97 +++++++++++++++++++++++++++ include/vulkan/vk_ext_queue_global_priority.h | 72 ++++++++++++++++++++ src/amd/vulkan/radv_device.c | 31 ++++++++- src/amd/vulkan/radv_private.h | 12 ++++ src/amd/vulkan/radv_radeon_winsys.h | 8 ++- src/amd/vulkan/winsys/amdgpu/radv_amdgpu_cs.c | 20 +++++- 6 files changed, 234 insertions(+), 6 deletions(-) create mode 100644 docs/specs/VK_EXT_queue_global_priority.txt create mode 100644 include/vulkan/vk_ext_queue_global_priority.h
docs/specs/VK_EXT_queue_global_priority.txt new file +97 -0
@@ -0,0 +1,97 @@ +Name Strings + + VK_EXT_queue_global_priority + +Extension Type + + Queue Extension + +Registered Extension Number + + Draft + +Status + + Draft + +Version + + 0 (Early Draft) + +Last Modified Date + + See git log. + +IP Status + + No known IP claims. + +Dependencies + + This extension is written against the Vulkan 1.0.32 specification [1]. + +Contributors + + Andres Rodriguez, Valve Software <[email protected]> + Pierre-Loup Griffais, Valve Software <[email protected]> + +Contact + + Andres Rodriguez, Valve Software <[email protected]> + +Overview + + In Vulkan 1.0.32 users can specify device-scope queue priorities. In + some cases it may be useful to extend this concept to a system-wide scope. + + This extension provides a mechanism for caller's to set their system-wide + priority. The default queue priority is VK_QUEUE_GLOBAL_PRIORITY_NORMAL. + + TODO: privileges required for priority escalation and note about system + resource starvation. + +New Object Types + + VkQueueGlobalPriorityEXT + VK_STRUCTURE_TYPE_QUEUE_GLOBAL_PRIORITY_EXT + +New Enum Constants + + None + +New Enums + + VkQueueGlobalPriority + VK_QUEUE_GLOBAL_PRIORITY_HIGH + VK_QUEUE_GLOBAL_PRIORITY_NORMAL + +New Structs + + typedef struct VkQueueGlobalPriorityEXT { + VkStructureType sType; + void* pNext; + VkQueueGlobalPriority priority; + } VkQueueGlobalPriorityEXT; + +New Functions + + None + +Description + + TODO + +Issues + + TODO + +References + + [1]: https://github.com/KhronosGroup/Vulkan-Docs/tree/v1.0-core-20161025 + +Version History + + See git log for full history. + + - Revision 1, 2017-01-01 (Andres Rodriguez) + - Initial draft
include/vulkan/vk_ext_queue_global_priority.h new file +72 -0
72 lines not on this page · raw message
src/amd/vulkan/radv_device.c +28 -3
@@ -651,16 +651,41 @@ void radv_GetPhysicalDeviceMemoryProperties( }; } +static enum ctx_priority +radv_get_queue_global_priority (VkQueueGlobalPriorityEXT *pObj) +{ +switch (pObj->priority) { +case VK_QUEUE_GLOBAL_PRIORITY_HIGH : +return CTX_PRIORITY_HIGH ; +default : +return CTX_PRIORITY_NORMAL ; + } +} +static int radv_queue_init (struct radv_device *device,struct radv_queue *queue, -int queue_family_index,int idx) +int queue_family_index,int idx,const VkDeviceQueueCreateInfo *create_info) { queue->_loader_data.loaderMagic =ICD_LOADER_MAGIC ; queue->device = device; queue->queue_family_index = queue_family_index; queue->queue_idx = idx; + queue->priority =CTX_PRIORITY_NORMAL ; + + radv_foreach_ext_obj(create_info, next_obj) { +switch (next_obj->sType) { +case VK_STRUCTURE_TYPE_QUEUE_GLOBAL_PRIORITY_EXT : + queue->priority =radv_get_queue_global_priority ( + (VkQueueGlobalPriorityEXT *)next_obj); +break ; +default : +radv_finishme ("Unsupported queue extension VkStructureType %d\n" , + next_obj->sType); +break ; + } + } - queue->hw_ctx = device->ws->ctx_create (device->ws); + queue->hw_ctx = device->ws->ctx_create (device->ws, queue->priority);if (!queue->hw_ctx)return VK_ERROR_OUT_OF_HOST_MEMORY ;
@@ -731,7 +756,7 @@ VkResult radv_CreateDevice( device->queue_count[qfi] = queue_create->queueCount;for (unsigned q =0 ; q < queue_create->queueCount; q++) { - result =radv_queue_init (device, &device->queues[qfi][q], qfi, q); + result =radv_queue_init (device, &device->queues[qfi][q], qfi, q, queue_create);if (result !=VK_SUCCESS ) goto fail; }
src/amd/vulkan/radv_private.h +12 -0
@@ -66,6 +66,7 @@ typedef uint32_t xcb_window_t;#include <vulkan/vulkan.h> #include <vulkan/vulkan_intel.h> +#include <vulkan/vk_ext_queue_global_priority.h> #include <vulkan/vk_icd.h> #include "radv_entrypoints.h"
@@ -457,6 +458,7 @@ struct radv_queue {VK_LOADER_DATA _loader_data;struct radv_device * device;struct radeon_winsys_ctx *hw_ctx; +enum ctx_priority priority;int queue_family_index;int queue_idx; };
@@ -1325,4 +1327,14 @@ RADV_DEFINE_NONDISP_HANDLE_CASTS(radv_shader_module, VkShaderModule) return (const __VkType *) __radv_obj; \ } +/** +* Used to iterate over Vk structs that support extension types +* +* Iterate over the __obj->pNext list, keeping the current object in __varname +*/ +#define radv_foreach_ext_obj (__obj, __varname) \ +/*Note: Any type with sType/pNext works here*/ \ + for ( VkSubmitInfo *__varname = (VkSubmitInfo*)__obj->pNext;\ + __varname != NULL;\ + __varname = (VkSubmitInfo*)__varname->pNext)#endif /* RADV_PRIVATE_H */
src/amd/vulkan/radv_radeon_winsys.h +7 -1
@@ -64,6 +64,11 @@ enum ring_type { RING_LAST, }; +enum ctx_priority { +CTX_PRIORITY_HIGH =0 , +CTX_PRIORITY_NORMAL +}; +struct radeon_winsys_cs {unsigned cdw;/* Number of used dwords. */ unsigned max_dw;/* Maximum number of dwords. */
@@ -284,7 +289,8 @@ struct radeon_winsys {void (*buffer_set_metadata)(struct radeon_winsys_bo *bo,struct radeon_bo_metadata *md); -struct radeon_winsys_ctx *(*ctx_create)(struct radeon_winsys *ws); +struct radeon_winsys_ctx *(*ctx_create)(struct radeon_winsys *ws, +enum ctx_priority priority);void (*ctx_destroy)(struct radeon_winsys_ctx *ctx); bool (*ctx_wait_idle)(struct radeon_winsys_ctx *ctx,
src/amd/vulkan/winsys/amdgpu/radv_amdgpu_cs.c +18 -2
@@ -775,15 +775,31 @@ static int radv_amdgpu_winsys_cs_submit(struct radeon_winsys_ctx *_ctx,return ret; } -static struct radeon_winsys_ctx *radv_amdgpu_ctx_create (struct radeon_winsys *_ws) +static uint32_t radv_to_amdgpu_priority(enum ctx_priority priority radv_priority) +{ +switch (radv_priority) { +case AMDGPU_CTX_PRIORITY_HIGH : +return AMDGPU_CTX_PRIORITY_HIGH ; +case AMDGPU_CTX_PRIORITY_NORMAL : +return AMDGPU_CTX_PRIORITY_NORMAL ; +default : +radv_loge ("Invalid radv priority %d\n, defualting to normal" , radv_priority); +return AMDGPU_CTX_PRIORITY_NORMAL ; + } +} + +static struct radeon_winsys_ctx *radv_amdgpu_ctx_create (struct radeon_winsys *_ws, +enum ctx_priority priority) {struct radv_amdgpu_winsys *ws =radv_amdgpu_winsys (_ws);struct radv_amdgpu_ctx *ctx =CALLOC_STRUCT (radv_amdgpu_ctx); +uint32_t amdgpu_priority =radv_to_amdgpu_priority (priority);int r;if (!ctx)return NULL ; - r =amdgpu_cs_ctx_create (ws->dev, &ctx->ctx); + + r =amdgpu_cs_ctx_create2 (ws->dev, amdgpu_priority, &ctx->ctx);if (r) {fprintf (stderr,"amdgpu: radv_amdgpu_cs_ctx_create failed. (%i)\n" , r); goto error_create;
Signature · 6 lines
2.9.3 _______________________________________________ amd-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/amd-gfx
Ooops sent this to the wrong list, sorry.
________________________________________
From: amd-gfx [[email protected]] on behalf of Andres Rodriguez [[email protected]]
Sent: Tuesday, January 03, 2017 9:22 PM
To: [email protected]
Subject: [PATCH] radv: Initial implementation of VK_EXT_queue_global_priority
This patch series implements VK_EXT_queue_global_priority, a vulkan extension that focuses on providing scheduling guarantees.
For further information on the use case refer to:
https://lists.freedesktop.org/archives/amd-gfx/2016-December/004068.html
For some perf data on this initial implementation:
https://lists.freedesktop.org/archives/amd-gfx/2016-December/004257.html
Test apps available at:
https://github.com/lostgoat/Vulkan
Related amdgpu and libdrm_amdgpu patches can be found on the amd-gfx mailing list.
Additionaly, results for vk-cts -n *semaphore* on an RX480:
Test run totals:
Passed: 1821/4046 (45.0%)
Failed: 0/4046 (0.0%)
Not supported: 2225/4046 (55.0%)
Warnings: 0/4046 (0.0%)
_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
Hey Guys,
One of my first non-trivial contributions to mesa here. Let me know if I've messed up anything, bad coding style or some patch validation script I should've run or any other kind of pitfall a newbie might make.
I tried to follow the guidelines over at:
http://www.mesa3d.org/devinfo.html
Although some of them seen to not apply anymore (radv code was tabs instead of 3 spaces).
Thanks for taking the time to review :)
Regards,
Andres
Quoted 16 lines · Andres Rodriguez
This patch series implements
VK_EXT_queue_global_priority, a vulkan extension
that focuses on providing scheduling guarantees.For further information on the use case refer to:
https://lists.freedesktop.org/archives/amd-gfx/2016-December/004068.htmlFor some perf data on this initial implementation:
https://lists.freedesktop.org/archives/amd-gfx/2016-December/004257.htmlTest apps available at:
https://github.com/lostgoat/VulkanRelated amdgpu and
libdrm_amdgpupatches can be found on the amd-gfx mailing list.
Quoted 9 lines
Additionaly, results for vk-cts -n *semaphore* on an RX480:
Test run totals:
Passed: 1821/4046 (45.0%)
Failed: 0/4046 (0.0%)
Not supported: 2225/4046 (55.0%)
Warnings: 0/4046 (0.0%)
_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx