Skip to contentSkip to conversation

gpudev: annotate memory allocation

PATCH Superseded Stephen Hemminger 2 participants 8 messages Oct 17, 2024 – Nov 9, 2024

lore
Share & tools
lore
#0 lore raw

cover gpudev: annotate memory allocation

Use function attributes to catch misuse of GPU memory at compile time.

Stephen Hemminger (2):
test-gpudev: avoid use-after-free and free-non-heap warnings
gpudev: add malloc annotations to rte_gpu_mem_alloc

 app/test-gpudev/main.c  | 10 ++++++++-
 lib/gpudev/rte_gpudev.h | 46 +++++++++++++++++++++--------------------
 2 files changed, 33 insertions(+), 23 deletions(-)
Signature · 1 line
2.45.2
#1 lore raw

PATCH 1/2 test-gpudev: avoid use-after-free and free-non-heap warnings

This test intentionally frees a bad pointer in GPU memory, and GCC sees that at compile time. Suppress the warnings to allow the test to run.

---
 app/test-gpudev/main.c | 10 +++++++++-
 1 file changed, 9 insertions(+), 1 deletion(-)
app/test-gpudev/main.c +9 -1
index f065e6cd81..14df14d528 100644
@@ -96,6 +96,13 @@ alloc_gpu_memory(uint16_t gpu_id)
 		goto error;
 	}
 
+	/* GCC 11 or later, is able to detect use-after-free and calling free on no-pointer
+	 * at compilation, but want to allow these tests to still work.
+	 */
+#pragma GCC diagnostic push
+#pragma GCC diagnostic ignored "-Wuse-after-free"
+#pragma GCC diagnostic ignored "-Wfree-nonheap-object"
+
 	ret = rte_gpu_mem_free(gpu_id, (uint8_t *)(ptr_1)+0x700);
 	if (ret < 0) {
 		printf("GPU memory 0x%p NOT freed: GPU driver didn't find this memory address internally.\n",
@@ -128,6 +135,7 @@ alloc_gpu_memory(uint16_t gpu_id)
 	rte_gpu_mem_free(gpu_id, ptr_1);
 	rte_gpu_mem_free(gpu_id, ptr_2);
 
+#pragma GCC diagnostic pop
 	printf("\n=======> TEST: FAILED\n");
 	return -1;
 }
@@ -228,12 +236,12 @@ gpu_mem_cpu_map(uint16_t gpu_id)
 	}
 	printf("GPU memory CPU unmapped, 0x%p not valid anymore\n", ptr_cpu);
 
+	printf("GPU memory 0x%p freeing\n", ptr_gpu);
 	ret = rte_gpu_mem_free(gpu_id, ptr_gpu);
 	if (ret < 0) {
 		fprintf(stderr, "rte_gpu_mem_free returned error %d\n", ret);
 		goto error;
 	}
-	printf("GPU memory 0x%p freed\n", ptr_gpu);
 
 	printf("\n=======> TEST: PASSED\n");
 	return 0;
Signature · 1 line
2.45.2
#2 lore raw

PATCH 2/2 gpudev: add malloc annotations to rte_gpu_mem_alloc

In reply to Stephen Hemminger

Add function attributes that allow detecting use after free, and calling free on bad pointer at compile time.

---
 lib/gpudev/rte_gpudev.h | 46 +++++++++++++++++++++--------------------
 1 file changed, 24 insertions(+), 22 deletions(-)
lib/gpudev/rte_gpudev.h +24 -22
index 0a94a6abc4..7a35cf85d1 100644
@@ -357,6 +357,28 @@ int rte_gpu_callback_unregister(int16_t dev_id, enum rte_gpu_event event,
 __rte_experimental
 int rte_gpu_info_get(int16_t dev_id, struct rte_gpu_info *info);
 
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice.
+ *
+ * Deallocate a chunk of memory allocated with rte_gpu_mem_alloc().
+ *
+ * @param dev_id
+ *   Reference device ID.
+ * @param ptr
+ *   Pointer to the memory area to be deallocated.
+ *   NULL is a no-op accepted value.
+ *
+ * @return
+ *   0 on success, -rte_errno otherwise:
+ *   - ENODEV if invalid dev_id
+ *   - ENOTSUP if operation not supported by the driver
+ *   - EPERM if driver error
+ */
+__rte_experimental
+int rte_gpu_mem_free(int16_t dev_id, void *ptr);
+
 /**
  * @warning
  * @b EXPERIMENTAL: this API may change without prior notice.
@@ -385,28 +407,8 @@ int rte_gpu_info_get(int16_t dev_id, struct rte_gpu_info *info);
  */
 __rte_experimental
 void *rte_gpu_mem_alloc(int16_t dev_id, size_t size, unsigned int align)
-__rte_alloc_size(2);
-
-/**
- * @warning
- * @b EXPERIMENTAL: this API may change without prior notice.
- *
- * Deallocate a chunk of memory allocated with rte_gpu_mem_alloc().
- *
- * @param dev_id
- *   Reference device ID.
- * @param ptr
- *   Pointer to the memory area to be deallocated.
- *   NULL is a no-op accepted value.
- *
- * @return
- *   0 on success, -rte_errno otherwise:
- *   - ENODEV if invalid dev_id
- *   - ENOTSUP if operation not supported by the driver
- *   - EPERM if driver error
- */
-__rte_experimental
-int rte_gpu_mem_free(int16_t dev_id, void *ptr);
+	__rte_alloc_size(2) __rte_alloc_align(3)
+	__rte_malloc __rte_dealloc(rte_gpu_mem_free, 2);
 
 /**
  * @warning
Signature · 1 line
2.45.2
#3 lore raw

v2 cover gpudev: annotate memory allocation functions

In reply to Stephen Hemminger

Use function attributes to catch misuse of GPU memory at compile time.

v2 - remove test case where invalid pointer was being passed.
The test is now caught at compile time, making a runtime test no longer necessary.

Stephen Hemminger (2):
test-gpudev: avoid use-after-free and free-non-heap warnings
gpudev: add malloc annotations to rte_gpu_mem_alloc

 app/test-gpudev/main.c  | 16 +++-----------
 lib/gpudev/rte_gpudev.h | 46 +++++++++++++++++++++--------------------
 2 files changed, 27 insertions(+), 35 deletions(-)
Signature · 1 line
2.45.2
#4 lore raw

PATCH v2 1/2 test-gpudev: avoid use-after-free and free-non-heap warnings

Remove the test intentionally frees a bad pointer in GPU memory, and GCC sees that at compile time now.

Fix use after free in printf.

---
 app/test-gpudev/main.c | 16 +++-------------
 1 file changed, 3 insertions(+), 13 deletions(-)
app/test-gpudev/main.c +3 -13
index f065e6cd81..30fe573bf9 100644
@@ -96,29 +96,19 @@ alloc_gpu_memory(uint16_t gpu_id)
 		goto error;
 	}
 
-	ret = rte_gpu_mem_free(gpu_id, (uint8_t *)(ptr_1)+0x700);
-	if (ret < 0) {
-		printf("GPU memory 0x%p NOT freed: GPU driver didn't find this memory address internally.\n",
-				(uint8_t *)(ptr_1)+0x700);
-	} else {
-		fprintf(stderr, "ERROR: rte_gpu_mem_free freed GPU memory 0x%p\n",
-				(uint8_t *)(ptr_1)+0x700);
-		goto error;
-	}
-
+	printf("GPU memory 0x%p free\n", ptr_2);
 	ret = rte_gpu_mem_free(gpu_id, ptr_2);
 	if (ret < 0) {
 		fprintf(stderr, "rte_gpu_mem_free returned error %d\n", ret);
 		goto error;
 	}
-	printf("GPU memory 0x%p freed\n", ptr_2);
 
+	printf("GPU memory 0x%p free\n", ptr_1);
 	ret = rte_gpu_mem_free(gpu_id, ptr_1);
 	if (ret < 0) {
 		fprintf(stderr, "rte_gpu_mem_free returned error %d\n", ret);
 		goto error;
 	}
-	printf("GPU memory 0x%p freed\n", ptr_1);
 
 	printf("\n=======> TEST: PASSED\n");
 	return 0;
@@ -228,12 +218,12 @@ gpu_mem_cpu_map(uint16_t gpu_id)
 	}
 	printf("GPU memory CPU unmapped, 0x%p not valid anymore\n", ptr_cpu);
 
+	printf("GPU memory 0x%p freeing\n", ptr_gpu);
 	ret = rte_gpu_mem_free(gpu_id, ptr_gpu);
 	if (ret < 0) {
 		fprintf(stderr, "rte_gpu_mem_free returned error %d\n", ret);
 		goto error;
 	}
-	printf("GPU memory 0x%p freed\n", ptr_gpu);
 
 	printf("\n=======> TEST: PASSED\n");
 	return 0;
Signature · 1 line
2.45.2
#5 lore raw

PATCH v2 2/2 gpudev: add malloc annotations to rte_gpu_mem_alloc

In reply to Stephen Hemminger

Add function attributes that allow detecting use after free, and calling free on bad pointer at compile time.

---
 lib/gpudev/rte_gpudev.h | 46 +++++++++++++++++++++--------------------
 1 file changed, 24 insertions(+), 22 deletions(-)
lib/gpudev/rte_gpudev.h +24 -22
index 0a94a6abc4..7a35cf85d1 100644
@@ -357,6 +357,28 @@ int rte_gpu_callback_unregister(int16_t dev_id, enum rte_gpu_event event,
 __rte_experimental
 int rte_gpu_info_get(int16_t dev_id, struct rte_gpu_info *info);
 
+
+/**
+ * @warning
+ * @b EXPERIMENTAL: this API may change without prior notice.
+ *
+ * Deallocate a chunk of memory allocated with rte_gpu_mem_alloc().
+ *
+ * @param dev_id
+ *   Reference device ID.
+ * @param ptr
+ *   Pointer to the memory area to be deallocated.
+ *   NULL is a no-op accepted value.
+ *
+ * @return
+ *   0 on success, -rte_errno otherwise:
+ *   - ENODEV if invalid dev_id
+ *   - ENOTSUP if operation not supported by the driver
+ *   - EPERM if driver error
+ */
+__rte_experimental
+int rte_gpu_mem_free(int16_t dev_id, void *ptr);
+
 /**
  * @warning
  * @b EXPERIMENTAL: this API may change without prior notice.
@@ -385,28 +407,8 @@ int rte_gpu_info_get(int16_t dev_id, struct rte_gpu_info *info);
  */
 __rte_experimental
 void *rte_gpu_mem_alloc(int16_t dev_id, size_t size, unsigned int align)
-__rte_alloc_size(2);
-
-/**
- * @warning
- * @b EXPERIMENTAL: this API may change without prior notice.
- *
- * Deallocate a chunk of memory allocated with rte_gpu_mem_alloc().
- *
- * @param dev_id
- *   Reference device ID.
- * @param ptr
- *   Pointer to the memory area to be deallocated.
- *   NULL is a no-op accepted value.
- *
- * @return
- *   0 on success, -rte_errno otherwise:
- *   - ENODEV if invalid dev_id
- *   - ENOTSUP if operation not supported by the driver
- *   - EPERM if driver error
- */
-__rte_experimental
-int rte_gpu_mem_free(int16_t dev_id, void *ptr);
+	__rte_alloc_size(2) __rte_alloc_align(3)
+	__rte_malloc __rte_dealloc(rte_gpu_mem_free, 2);
 
 /**
  * @warning
Signature · 1 line
2.45.2
#6 lore raw

Re: [PATCH v2 0/2] gpudev: annotate memory allocation functions

In reply to Stephen Hemminger
Quoted 8 lines · Stephen Hemminger

Use function attributes to catch misuse of GPU memory at compile time.

v2 - remove test case where invalid pointer was being passed.
The test is now caught at compile time, making a runtime test no longer necessary.

Quoted 7 lines

Stephen Hemminger (2):
test-gpudev: avoid use-after-free and free-non-heap warnings
gpudev: add malloc annotations to rte_gpu_mem_alloc

 app/test-gpudev/main.c  | 16 +++-----------
 lib/gpudev/rte_gpudev.h | 46 +++++++++++++++++++++--------------------
 2 files changed, 27 insertions(+), 35 deletions(-)

There is a build error due to referencing experimental symbol in annotation.

Signature · 1 line
David Marchand
#7 lore raw
In reply to Stephen Hemminger
Quoted 11 lines · On Thu, 17 Oct 2024 15:58:02 -0700 Stephen Hemminger…

Use function attributes to catch misuse of GPU memory at compile time.

Stephen Hemminger (2):
test-gpudev: avoid use-after-free and free-non-heap warnings
gpudev: add malloc annotations to rte_gpu_mem_alloc

 app/test-gpudev/main.c  | 10 ++++++++-
 lib/gpudev/rte_gpudev.h | 46 +++++++++++++++++++++--------------------
 2 files changed, 33 insertions(+), 23 deletions(-)

The problem is that include checker can't handle this.
####################################################################################
#### [Begin job log] "ubuntu-22.04-gcc-debug+doc+examples+tests" at step Build and test
####################################################################################
from buildtools/chkincs/chkincs.p/gpudev_driver.c:1:
/home/runner/work/dpdk/dpdk/lib/gpudev/rte_gpudev.h:411:9: error: ‘rte_gpu_mem_free’ is deprecated: Symbol is not yet part of stable ABI [-Werror=deprecated-declarations]

  411 |         __rte_malloc __rte_dealloc(rte_gpu_mem_free, 2);
      |         ^~~~~~~~~~~~
/home/runner/work/dpdk/dpdk/lib/gpudev/rte_gpudev.h:380:5: note: declared here
  380 | int rte_gpu_mem_free(int16_t dev_id, void *ptr);
      |     ^~~~~~~~~~~~~~~~

Either include checker needs to be able to handle experimental symbols or maybe it is time for gpudev to be moved out experimental status for 25.03?