gpudev: use CPU map functionality in comm list
PATCH 1/1 gpudev: use CPU map functionality in comm list
From: Elena Agostini <[email protected]>
rte_gpu_mem_cpu_map() exposes a GPU memory area to the CPU. In gpudev communication list this is useful to store the status flag.
A communication list status flag allocated on GPU memory and mapped for CPU visibility can be updated by CPU and polled by a GPU workload.
The polling operation is more frequent than the CPU update operation. Having the status flag in GPU memory reduces the GPU workload polling latency.
If CPU mapping feature is not enabled, status flag resides in CPU memory registered so it's visible from the GPU.
To facilitate the interaction with the status flag, this patch provides also the set/get functions for it.
--- app/test-gpudev/main.c | 8 ++- doc/guides/prog_guide/gpudev.rst | 2 +- lib/gpudev/gpudev.c | 116 +++++++++++++++++++++++++++++-- lib/gpudev/rte_gpudev.h | 49 ++++++++++++- lib/gpudev/version.map | 2 + 5 files changed, 166 insertions(+), 11 deletions(-)
app/test-gpudev/main.c +7 -1
@@ -324,7 +324,13 @@ simulate_gpu_task(struct rte_gpu_comm_list *comm_list_item, int num_pkts)* consume(comm_list_item->pkt_list[idx].addr); */ } - comm_list_item->status =RTE_GPU_COMM_LIST_DONE ; +/* +* A real GPU workload function can't directly call rte_gpu_comm_set_status +* because it's a CPU-only function. +* A real GPU workload should implement the content +* of rte_gpu_comm_set_status() in GPU specific code. +*/ +rte_gpu_comm_set_status (comm_list_item,RTE_GPU_COMM_LIST_DONE );return 0 ; }
doc/guides/prog_guide/gpudev.rst +1 -1
@@ -216,7 +216,7 @@ about how to use functions in this library in case of a CUDA application. /* GPU kernel keeps checking this flag to know if it has to quit or wait for more packets. */ while (*quit_flag_ptr == 0) { - if (comm_list[comm_list_index]->status != RTE_GPU_COMM_LIST_READY) + if (comm_list[comm_list_index]->status_d[0] != RTE_GPU_COMM_LIST_READY) continue; if (threadIdx.x < comm_list[comm_list_index]->num_pkts)
lib/gpudev/gpudev.c +110 -6
178 lines not on this page · raw message
lib/gpudev/rte_gpudev.h +46 -3
70 lines not on this page · raw message
lib/gpudev/version.map +2 -0
@@ -12,8 +12,10 @@ EXPERIMENTAL { rte_gpu_comm_destroy_flag; rte_gpu_comm_destroy_list; rte_gpu_comm_get_flag_value; + rte_gpu_comm_get_status; rte_gpu_comm_populate_list_pkts; rte_gpu_comm_set_flag; + rte_gpu_comm_set_status; rte_gpu_count_avail; rte_gpu_find_next; rte_gpu_info_get;
Signature · 1 line
2.17.1
PATCH v2 1/1 gpudev: use CPU map functionality in comm list
From: Elena Agostini <[email protected]>
rte_gpu_mem_cpu_map() exposes a GPU memory area to the CPU. In gpudev communication list this is useful to store the status flag.
A communication list status flag allocated on GPU memory and mapped for CPU visibility can be updated by CPU and polled by a GPU workload.
The polling operation is more frequent than the CPU update operation. Having the status flag in GPU memory reduces the GPU workload polling latency.
If CPU mapping feature is not enabled, status flag resides in CPU memory registered so it's visible from the GPU.
To facilitate the interaction with the status flag, this patch provides also the set/get functions for it.
----
Changelog:
- Address checkpatch issues
--- app/test-gpudev/main.c | 8 ++- doc/guides/prog_guide/gpudev.rst | 2 +- lib/gpudev/gpudev.c | 114 +++++++++++++++++++++++++++++-- lib/gpudev/rte_gpudev.h | 49 ++++++++++++- lib/gpudev/version.map | 2 + 5 files changed, 165 insertions(+), 10 deletions(-)
app/test-gpudev/main.c +7 -1
@@ -324,7 +324,13 @@ simulate_gpu_task(struct rte_gpu_comm_list *comm_list_item, int num_pkts)* consume(comm_list_item->pkt_list[idx].addr); */ } - comm_list_item->status =RTE_GPU_COMM_LIST_DONE ; +/* +* A real GPU workload function can't directly call rte_gpu_comm_set_status +* because it's a CPU-only function. +* A real GPU workload should implement the content +* of rte_gpu_comm_set_status() in GPU specific code. +*/ +rte_gpu_comm_set_status (comm_list_item,RTE_GPU_COMM_LIST_DONE );return 0 ; }
doc/guides/prog_guide/gpudev.rst +1 -1
@@ -216,7 +216,7 @@ about how to use functions in this library in case of a CUDA application. /* GPU kernel keeps checking this flag to know if it has to quit or wait for more packets. */ while (*quit_flag_ptr == 0) { - if (comm_list[comm_list_index]->status != RTE_GPU_COMM_LIST_READY) + if (comm_list[comm_list_index]->status_d[0] != RTE_GPU_COMM_LIST_READY) continue; if (threadIdx.x < comm_list[comm_list_index]->num_pkts)
lib/gpudev/gpudev.c +109 -5
176 lines not on this page · raw message
lib/gpudev/rte_gpudev.h +46 -3
70 lines not on this page · raw message
lib/gpudev/version.map +2 -0
@@ -12,8 +12,10 @@ EXPERIMENTAL { rte_gpu_comm_destroy_flag; rte_gpu_comm_destroy_list; rte_gpu_comm_get_flag_value; + rte_gpu_comm_get_status; rte_gpu_comm_populate_list_pkts; rte_gpu_comm_set_flag; + rte_gpu_comm_set_status; rte_gpu_count_avail; rte_gpu_find_next; rte_gpu_info_get;
Signature · 1 line
2.17.1
Thomas Monjalon
Applied22/02/2022 01:22, [email protected]:
Quoted 21 lines
From: Elena Agostini <[email protected]>
rte_gpu_mem_cpu_map()exposes a GPU memory area to the CPU. In gpudev communication list this is useful to store the status flag.A communication list status flag allocated on GPU memory and mapped for CPU visibility can be updated by CPU and polled by a GPU workload.
The polling operation is more frequent than the CPU update operation. Having the status flag in GPU memory reduces the GPU workload polling latency.
If CPU mapping feature is not enabled, status flag resides in CPU memory registered so it's visible from the GPU.
To facilitate the interaction with the status flag, this patch provides also the set/get functions for it.
Signed-off-by Elena Agostini <[email protected]>
[...]
Quoted 3 lines
* @param ptr
- * Pointer to the GPU memory area to be unmapped.
+ * Pointer to the memory area to be unmapped.
rebase issue: that reverts another commit sent the same day. I will skip
[...]
Quoted 2 lines
#endif /*
RTE_GPUDEV_H*/
+
Will remove lines added at the end of 2 files.
Applied, thanks.