Skip to contentSkip to conversation

dma-buf: heaps: Set allocation orders for larger page sizes

PATCH Superseded Juan Yescas 5 participants 9 messages Apr 15, 2025 – Apr 17, 2025

Subsystems (3)

v1 of 3 Superseded Latest is v3 →

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 dma-buf: heaps: Set allocation orders for larger page sizes

This change sets the allocation orders for the different page sizes (4k, 16k, 64k) based on PAGE_SHIFT. Before this change, the orders for large page sizes were calculated incorrectly, this caused system heap to allocate from 2% to 4% more memory on 16KiB page size kernels.

This change was tested on 4k/16k page size kernels.

---
 drivers/dma-buf/heaps/system_heap.c | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)
drivers/dma-buf/heaps/system_heap.c +8 -1
index 26d5dc89ea16..54674c02dcb4 100644
@@ -50,8 +50,15 @@ static gfp_t order_flags[] = {HIGH_ORDER_GFP, HIGH_ORDER_GFP, LOW_ORDER_GFP};
  * to match with the sizes often found in IOMMUs. Using order 4 pages instead
  * of order 0 pages can significantly improve the performance of many IOMMUs
  * by reducing TLB pressure and time spent updating page tables.
+ *
+ * Note: When the order is 0, the minimum allocation is PAGE_SIZE. The possible
+ * page sizes for ARM devices could be 4K, 16K and 64K.
  */
-static const unsigned int orders[] = {8, 4, 0};
+#define ORDER_1M (20 - PAGE_SHIFT)
+#define ORDER_64K (16 - PAGE_SHIFT)
+#define ORDER_FOR_PAGE_SIZE (0)
+static const unsigned int orders[] = {ORDER_1M, ORDER_64K, ORDER_FOR_PAGE_SIZE};
+
 #define NUM_ORDERS ARRAY_SIZE(orders)
 
 static struct sg_table *dup_sg_table(struct sg_table *table)
Signature · 1 line
2.49.0.604.gff1f9ca942-goog
reviewer Acked-by on v1 1/1
#1 lore raw
Quoted 9 lines · Juan Yescas

This change sets the allocation orders for the different page sizes (4k, 16k, 64k) based on PAGE_SHIFT. Before this change, the orders for large page sizes were calculated incorrectly, this caused system heap to allocate from 2% to 4% more memory on 16KiB page size kernels.

This change was tested on 4k/16k page size kernels.

Seems reasonable to me.

thanks
-john

reviewer Reviewed-by on v1 1/1
#2 lore raw
In reply to Juan Yescas
Quoted 9 lines · Juan Yescas

This change sets the allocation orders for the different page sizes (4k, 16k, 64k) based on PAGE_SHIFT. Before this change, the orders for large page sizes were calculated incorrectly, this caused system heap to allocate from 2% to 4% more memory on 16KiB page size kernels.

This change was tested on 4k/16k page size kernels.

I think "dma-buf: system_heap:" would be better for the subject since this is specific to the system heap.

Would you mind cleaning up the extra space on line 321 too? @@ -318,7 +318,7 @@ static struct page *alloc_largest_available(unsigned long size,

        int i;

        for (i = 0; i < NUM_ORDERS; i++) {

- if (size < (PAGE_SIZE << orders[i]))
+ if (size < (PAGE_SIZE << orders[i]))

With that,

Fixes d963ab0f15fb ("dma-buf: system_heap: Allocate higher order

pages if available") is also probably a good idea.

Quoted 28 lines
---
 drivers/dma-buf/heaps/system_heap.c | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)
drivers/dma-buf/heaps/system_heap.c +8 -2
index 26d5dc89ea16..54674c02dcb4 100644
@@ -50,8 +50,15 @@ static gfp_t order_flags[] = {HIGH_ORDER_GFP, HIGH_ORDER_GFP, LOW_ORDER_GFP};
  * to match with the sizes often found in IOMMUs. Using order 4 pages instead
  * of order 0 pages can significantly improve the performance of many IOMMUs
  * by reducing TLB pressure and time spent updating page tables.
+ *
+ * Note: When the order is 0, the minimum allocation is PAGE_SIZE. The possible
+ * page sizes for ARM devices could be 4K, 16K and 64K.
  */
-static const unsigned int orders[] = {8, 4, 0};
+#define ORDER_1M (20 - PAGE_SHIFT)
+#define ORDER_64K (16 - PAGE_SHIFT)
+#define ORDER_FOR_PAGE_SIZE (0)
+static const unsigned int orders[] = {ORDER_1M, ORDER_64K, ORDER_FOR_PAGE_SIZE};
+
 #define NUM_ORDERS ARRAY_SIZE(orders)
 
 static struct sg_table *dup_sg_table(struct sg_table *table)
--

2.49.0.604.gff1f9ca942-goog

#3 lore raw
Quoted 43 lines

-----Original Message-----
From: T.J. Mercier [mailto:[email protected]]
Sent: Wednesday, April 16, 2025 5:57 AM
To: Juan Yescas <[email protected]>
Cc: Sumit Semwal <[email protected]>; Benjamin Gaignard <[email protected]>; Brian Starkey <[email protected]>; John Stultz <[email protected]>; Christian König <[email protected]>; [email protected]; dri- [email protected]; [email protected]; linux- [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]
Subject: Re: [PATCH] dma-buf: heaps: Set allocation orders for larger page sizes

Quoted 9 lines · Juan Yescas

This change sets the allocation orders for the different page sizes (4k, 16k, 64k) based on PAGE_SHIFT. Before this change, the orders for large page sizes were calculated incorrectly, this caused system heap to allocate from 2% to 4% more memory on 16KiB page size kernels.

This change was tested on 4k/16k page size kernels.

I think "dma-buf: system_heap:" would be better for the subject since this is specific to the system heap.

Would you mind cleaning up the extra space on line 321 too?

(quoted diff) +1 -1
@@ -318,7 +318,7 @@ static struct page
 *alloc_largest_available(unsigned long size,
        int i;
 
        for (i = 0; i < NUM_ORDERS; i++) {
-               if (size <  (PAGE_SIZE << orders[i]))
+               if (size < (PAGE_SIZE << orders[i]))
 
 With that,
Quoted 3 lines
Fixes d963ab0f15fb ("dma-buf: system_heap: Allocate higher order pages if

available") is also probably a good idea.

Hi Juan.

Yes. This system_heap change should be changed for 16KB page. Actually, we may need to check other drivers using page order number. I guess gpu drivers may be one of them.

Quoted 34 lines

drivers/dma-buf/heaps/system_heap.c static gfp_t order_flags[] = {HIGH_ORDER_GFP,

Quoted 10 lines
---
 drivers/dma-buf/heaps/system_heap.c | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)
drivers/dma-buf/heaps/system_heap.c +0 -0
index 26d5dc89ea16..54674c02dcb4 100644
@@ -50,8 +50,15 @@ static gfp_t order_flags[] = {HIGH_ORDER_GFP,

HIGH_ORDER_GFP, LOW_ORDER_GFP};

drivers/dma-buf/heaps/system_heap.c static gfp_t order_flags[] = {HIGH_ORDER_GFP, · line 50

Quoted 1 line
drivers/dma-buf/heaps/system_heap.c +0 -0
  * to match with the sizes often found in IOMMUs. Using order 4 pages

instead

drivers/dma-buf/heaps/system_heap.c static gfp_t order_flags[] = {HIGH_ORDER_GFP, · line 51

Quoted 1 line
drivers/dma-buf/heaps/system_heap.c +0 -0
  * of order 0 pages can significantly improve the performance of many

IOMMUs

Quoted 19 lines
drivers/dma-buf/heaps/system_heap.c +10 -2
  * by reducing TLB pressure and time spent updating page tables.
+ *
+ * Note: When the order is 0, the minimum allocation is PAGE_SIZE.
+ The possible
+ * page sizes for ARM devices could be 4K, 16K and 64K.
  */
-static const unsigned int orders[] = {8, 4, 0};
+#define ORDER_1M (20 - PAGE_SHIFT)
+#define ORDER_64K (16 - PAGE_SHIFT)
+#define ORDER_FOR_PAGE_SIZE (0)
+static const unsigned int orders[] = {ORDER_1M, ORDER_64K,
+ORDER_FOR_PAGE_SIZE};
+
 #define NUM_ORDERS ARRAY_SIZE(orders)
 
 static struct sg_table *dup_sg_table(struct sg_table *table)
--

2.49.0.604.gff1f9ca942-goog

#4 lore raw
Quoted 51 lines · 김재원
Quoted 44 lines

-----Original Message-----
From: T.J. Mercier [mailto:[email protected]]
Sent: Wednesday, April 16, 2025 5:57 AM
To: Juan Yescas <[email protected]>
Cc: Sumit Semwal <[email protected]>; Benjamin Gaignard <[email protected]>; Brian Starkey <[email protected]>; John Stultz <[email protected]>; Christian König <[email protected]>; [email protected]; dri- [email protected]; [email protected]; linux- [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]
Subject: Re: [PATCH] dma-buf: heaps: Set allocation orders for larger page sizes

Quoted 9 lines · Juan Yescas

This change sets the allocation orders for the different page sizes (4k, 16k, 64k) based on PAGE_SHIFT. Before this change, the orders for large page sizes were calculated incorrectly, this caused system heap to allocate from 2% to 4% more memory on 16KiB page size kernels.

This change was tested on 4k/16k page size kernels.

I think "dma-buf: system_heap:" would be better for the subject since this is specific to the system heap.

Would you mind cleaning up the extra space on line 321 too?

(quoted diff) +1 -1
@@ -318,7 +318,7 @@ static struct page
 *alloc_largest_available(unsigned long size,
        int i;
 
        for (i = 0; i < NUM_ORDERS; i++) {
-               if (size <  (PAGE_SIZE << orders[i]))
+               if (size < (PAGE_SIZE << orders[i]))
 
 With that,
Quoted 3 lines
Fixes d963ab0f15fb ("dma-buf: system_heap: Allocate higher order pages if

available") is also probably a good idea.

Hi Juan.

Quoted 4 lines

Yes. This system_heap change should be changed for 16KB page. Actually, we may need to check other drivers using page order number. I guess gpu drivers may be one of them.

Thanks Jaewon for pointing it out. We'll take a look at the GPU drivers to make sure that they are using the proper page order.

Quoted 36 lines
Quoted 34 lines

drivers/dma-buf/heaps/system_heap.c static gfp_t order_flags[] = {HIGH_ORDER_GFP,

Quoted 10 lines
---
 drivers/dma-buf/heaps/system_heap.c | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)
drivers/dma-buf/heaps/system_heap.c +0 -0
index 26d5dc89ea16..54674c02dcb4 100644
@@ -50,8 +50,15 @@ static gfp_t order_flags[] = {HIGH_ORDER_GFP,

HIGH_ORDER_GFP, LOW_ORDER_GFP};

drivers/dma-buf/heaps/system_heap.c static gfp_t order_flags[] = {HIGH_ORDER_GFP, · line 50

Quoted 1 line
drivers/dma-buf/heaps/system_heap.c +0 -0
  * to match with the sizes often found in IOMMUs. Using order 4 pages

instead

drivers/dma-buf/heaps/system_heap.c static gfp_t order_flags[] = {HIGH_ORDER_GFP, · line 51

Quoted 1 line
drivers/dma-buf/heaps/system_heap.c +0 -0
  * of order 0 pages can significantly improve the performance of many

IOMMUs

Quoted 19 lines
drivers/dma-buf/heaps/system_heap.c +10 -2
  * by reducing TLB pressure and time spent updating page tables.
+ *
+ * Note: When the order is 0, the minimum allocation is PAGE_SIZE.
+ The possible
+ * page sizes for ARM devices could be 4K, 16K and 64K.
  */
-static const unsigned int orders[] = {8, 4, 0};
+#define ORDER_1M (20 - PAGE_SHIFT)
+#define ORDER_64K (16 - PAGE_SHIFT)
+#define ORDER_FOR_PAGE_SIZE (0)
+static const unsigned int orders[] = {ORDER_1M, ORDER_64K,
+ORDER_FOR_PAGE_SIZE};
+
 #define NUM_ORDERS ARRAY_SIZE(orders)
 
 static struct sg_table *dup_sg_table(struct sg_table *table)
--

2.49.0.604.gff1f9ca942-goog

maintainer
#5 lore raw
In reply to Juan Yescas

Am 15.04.25 um 19:19 schrieb Juan Yescas:

Quoted 20 lines

This change sets the allocation orders for the different page sizes (4k, 16k, 64k) based on PAGE_SHIFT. Before this change, the orders for large page sizes were calculated incorrectly, this caused system heap to allocate from 2% to 4% more memory on 16KiB page size kernels.

This change was tested on 4k/16k page size kernels.

---
 drivers/dma-buf/heaps/system_heap.c | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)
drivers/dma-buf/heaps/system_heap.c +0 -0
index 26d5dc89ea16..54674c02dcb4 100644
@@ -50,8 +50,15 @@ static gfp_t order_flags[] = {HIGH_ORDER_GFP, HIGH_ORDER_GFP, LOW_ORDER_GFP};
  * to match with the sizes often found in IOMMUs. Using order 4 pages instead
  * of order 0 pages can significantly improve the performance of many IOMMUs
  * by reducing TLB pressure and time spent updating page tables.

drivers/dma-buf/heaps/system_heap.c static gfp_t order_flags[] = {HIGH_ORDER_GFP, HIGH_ORDER_GFP, LOW_ORDER_GFP}; · line 61

Quoted 10 lines
drivers/dma-buf/heaps/system_heap.c +8 -1
+ *
+ * Note: When the order is 0, the minimum allocation is PAGE_SIZE. The possible
+ * page sizes for ARM devices could be 4K, 16K and 64K.
  */
-static const unsigned int orders[] = {8, 4, 0};
+#define ORDER_1M (20 - PAGE_SHIFT)
+#define ORDER_64K (16 - PAGE_SHIFT)
+#define ORDER_FOR_PAGE_SIZE (0)
+static const unsigned int orders[] = {ORDER_1M, ORDER_64K, ORDER_FOR_PAGE_SIZE};
+#

Good catch, but I think the defines are just overkill.

What you should do instead is to subtract page shift when using the array.

Apart from that using 1M, 64K and then falling back to 4K just sounds random to me. We have especially pushed back on 64K more than once because it is actually not beneficial in almost all cases.

I suggest to fix the code in system_heap_allocate to not over allocate instead and just try the available orders like TTM does. This has proven to be working architecture independent.

Regards,
Christian.

Quoted 3 lines
drivers/dma-buf/heaps/system_heap.c +0 -0
 #define NUM_ORDERS ARRAY_SIZE(orders)
 
 static struct sg_table *dup_sg_table(struct sg_table *table)
#6 lore raw
Quoted 37 lines · Christian König

Am 15.04.25 um 19:19 schrieb Juan Yescas:

Quoted 20 lines

This change sets the allocation orders for the different page sizes (4k, 16k, 64k) based on PAGE_SHIFT. Before this change, the orders for large page sizes were calculated incorrectly, this caused system heap to allocate from 2% to 4% more memory on 16KiB page size kernels.

This change was tested on 4k/16k page size kernels.

---
 drivers/dma-buf/heaps/system_heap.c | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)
drivers/dma-buf/heaps/system_heap.c +0 -0
index 26d5dc89ea16..54674c02dcb4 100644
@@ -50,8 +50,15 @@ static gfp_t order_flags[] = {HIGH_ORDER_GFP, HIGH_ORDER_GFP, LOW_ORDER_GFP};
  * to match with the sizes often found in IOMMUs. Using order 4 pages instead
  * of order 0 pages can significantly improve the performance of many IOMMUs
  * by reducing TLB pressure and time spent updating page tables.

drivers/dma-buf/heaps/system_heap.c static gfp_t order_flags[] = {HIGH_ORDER_GFP, HIGH_ORDER_GFP, LOW_ORDER_GFP}; · line 61

Quoted 10 lines
drivers/dma-buf/heaps/system_heap.c +8 -1
+ *
+ * Note: When the order is 0, the minimum allocation is PAGE_SIZE. The possible
+ * page sizes for ARM devices could be 4K, 16K and 64K.
  */
-static const unsigned int orders[] = {8, 4, 0};
+#define ORDER_1M (20 - PAGE_SHIFT)
+#define ORDER_64K (16 - PAGE_SHIFT)
+#define ORDER_FOR_PAGE_SIZE (0)
+static const unsigned int orders[] = {ORDER_1M, ORDER_64K, ORDER_FOR_PAGE_SIZE};
+#

Good catch, but I think the defines are just overkill.

Quoted 2 lines

What you should do instead is to subtract page shift when using the array.

There are several occurrences of orders in the file and I think it is better to do the calculations up front in the array. Would you be ok if we get rid of the defines as per your suggestion and make the calculations during the definition of the array. Something like this:

static const unsigned int orders[] = {20 - PAGE_SHIFT, 16 - PAGE_SHIFT, 0};

Quoted 2 lines

Apart from that using 1M, 64K and then falling back to 4K just sounds random to me. We have especially pushed back on 64K more than once because it is actually not beneficial in almost all cases.

In the hardware where the driver is used, the 64K is beneficial for:

Arm SMMUv3 (4KB granule size):
64KB (contiguous bit groups 16 4KB pages)

SysMMU benefits from 64KB (“large” page) on 4k/16k page sizes.

Quoted 2 lines

I suggest to fix the code in system_heap_allocate to not over allocate instead and just try the available orders like TTM does. This has proven to be working architecture independent.

Do you mean to have an implementation similar to __ttm_pool_alloc()?

https://web.git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/gpu/drm/ttm/ttm_pool.c?h=v6.15-rc2#n728

If that is the case, we can try the change, run some benchmarks and submit the patch if we see positive results.

Thanks
Juan

Quoted 7 lines

Regards,
Christian.

Quoted 3 lines
drivers/dma-buf/heaps/system_heap.c +0 -0
 #define NUM_ORDERS ARRAY_SIZE(orders)
 
 static struct sg_table *dup_sg_table(struct sg_table *table)
maintainer
#7 lore raw

Am 16.04.25 um 23:51 schrieb Juan Yescas:

Quoted 37 lines
Quoted 35 lines · Christian König

Am 15.04.25 um 19:19 schrieb Juan Yescas:

Quoted 20 lines

This change sets the allocation orders for the different page sizes (4k, 16k, 64k) based on PAGE_SHIFT. Before this change, the orders for large page sizes were calculated incorrectly, this caused system heap to allocate from 2% to 4% more memory on 16KiB page size kernels.

This change was tested on 4k/16k page size kernels.

---
 drivers/dma-buf/heaps/system_heap.c | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)
drivers/dma-buf/heaps/system_heap.c +0 -0
index 26d5dc89ea16..54674c02dcb4 100644
@@ -50,8 +50,15 @@ static gfp_t order_flags[] = {HIGH_ORDER_GFP, HIGH_ORDER_GFP, LOW_ORDER_GFP};
  * to match with the sizes often found in IOMMUs. Using order 4 pages instead
  * of order 0 pages can significantly improve the performance of many IOMMUs
  * by reducing TLB pressure and time spent updating page tables.

drivers/dma-buf/heaps/system_heap.c static gfp_t order_flags[] = {HIGH_ORDER_GFP, HIGH_ORDER_GFP, LOW_ORDER_GFP}; · line 61

Quoted 10 lines
drivers/dma-buf/heaps/system_heap.c +8 -1
+ *
+ * Note: When the order is 0, the minimum allocation is PAGE_SIZE. The possible
+ * page sizes for ARM devices could be 4K, 16K and 64K.
  */
-static const unsigned int orders[] = {8, 4, 0};
+#define ORDER_1M (20 - PAGE_SHIFT)
+#define ORDER_64K (16 - PAGE_SHIFT)
+#define ORDER_FOR_PAGE_SIZE (0)
+static const unsigned int orders[] = {ORDER_1M, ORDER_64K, ORDER_FOR_PAGE_SIZE};
+#

Good catch, but I think the defines are just overkill.

Quoted 8 lines
Quoted 2 lines

What you should do instead is to subtract page shift when using the array.

There are several occurrences of orders in the file and I think it is better to do the calculations up front in the array. Would you be ok if we get rid of the defines as per your suggestion and make the calculations during the definition of the array. Something like this:

static const unsigned int orders[] = {20 - PAGE_SHIFT, 16 - PAGE_SHIFT, 0};

Yeah that totally works for me as well. Just make sure that a code comment nearby explains why 20, 16 and 0.

Quoted 8 lines
Quoted 2 lines

Apart from that using 1M, 64K and then falling back to 4K just sounds random to me. We have especially pushed back on 64K more than once because it is actually not beneficial in almost all cases.

In the hardware where the driver is used, the 64K is beneficial for:

Arm SMMUv3 (4KB granule size):
64KB (contiguous bit groups 16 4KB pages)

SysMMU benefits from 64KB (“large” page) on 4k/16k page sizes.

Question could this HW also work with pages larger than 64K? (I strongly expect yes).

Quoted 3 lines
Quoted 2 lines

I suggest to fix the code in system_heap_allocate to not over allocate instead and just try the available orders like TTM does. This has proven to be working architecture independent.

Do you mean to have an implementation similar to __ttm_pool_alloc()?

Yeah something like that.

Quoted 4 lines

https://web.git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/gpu/drm/ttm/ttm_pool.c?h=v6.15-rc2#n728

If that is the case, we can try the change, run some benchmarks and submit the patch if we see positive results.

Could be that this doesn't matter for your platform, but it's minimal extra overhead asking for larger chunks first and it just avoids fragmenting everything into 64K chunks.

That is kind of important since the code in DMA-heaps should be platform neutral if possible.

Regards,
Christian.

Quoted 10 lines

Thanks
Juan

Quoted 6 lines

Regards,
Christian.

Quoted 3 lines
drivers/dma-buf/heaps/system_heap.c +0 -0
 #define NUM_ORDERS ARRAY_SIZE(orders)
 
 static struct sg_table *dup_sg_table(struct sg_table *table)
#8 lore raw
Quoted 49 lines · Christian König

Am 16.04.25 um 23:51 schrieb Juan Yescas:

On Wed, Apr 16, 2025 at 4:34 AM Christian König
<[email protected]> wrote:

Am 15.04.25 um 19:19 schrieb Juan Yescas:

This change sets the allocation orders for the different page sizes (4k, 16k, 64k) based on PAGE_SHIFT. Before this change, the orders for large page sizes were calculated incorrectly, this caused system heap to allocate from 2% to 4% more memory on 16KiB page size kernels.

This change was tested on 4k/16k page size kernels.

---
 drivers/dma-buf/heaps/system_heap.c | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)
drivers/dma-buf/heaps/system_heap.c +8 -1
index 26d5dc89ea16..54674c02dcb4 100644
@@ -50,8 +50,15 @@ static gfp_t order_flags[] = {HIGH_ORDER_GFP, HIGH_ORDER_GFP, LOW_ORDER_GFP};
  * to match with the sizes often found in IOMMUs. Using order 4 pages instead
  * of order 0 pages can significantly improve the performance of many IOMMUs
  * by reducing TLB pressure and time spent updating page tables.
+ *
+ * Note: When the order is 0, the minimum allocation is PAGE_SIZE. The possible
+ * page sizes for ARM devices could be 4K, 16K and 64K.
  */
-static const unsigned int orders[] = {8, 4, 0};
+#define ORDER_1M (20 - PAGE_SHIFT)
+#define ORDER_64K (16 - PAGE_SHIFT)
+#define ORDER_FOR_PAGE_SIZE (0)
+static const unsigned int orders[] = {ORDER_1M, ORDER_64K, ORDER_FOR_PAGE_SIZE};
+#
 

Good catch, but I think the defines are just overkill.

What you should do instead is to subtract page shift when using the array.

There are several occurrences of orders in the file and I think it is better to do the calculations up front in the array. Would you be ok if we get rid of the defines as per your suggestion and make the calculations during the definition of the array. Something like this:

Quoted 5 lines

static const unsigned int orders[] = {20 - PAGE_SHIFT, 16 - PAGE_SHIFT, 0};

Yeah that totally works for me as well. Just make sure that a code comment nearby explains why 20, 16 and 0.

Thanks, the changes were added in the [PATCH v3].

Quoted 4 lines

Apart from that using 1M, 64K and then falling back to 4K just sounds random to me. We have especially pushed back on 64K more than once because it is actually not beneficial in almost all cases.

In the hardware where the driver is used, the 64K is beneficial for:

Quoted 8 lines

Arm SMMUv3 (4KB granule size):
64KB (contiguous bit groups 16 4KB pages)

SysMMU benefits from 64KB (“large” page) on 4k/16k page sizes.

Question could this HW also work with pages larger than 64K? (I strongly expect yes).

Yes, if the page sizes in the SMMUv3 are:
- 4k, we can have 2MiB pages
- 16k, we can have 32MiB pages

Quoted 9 lines

I suggest to fix the code in system_heap_allocate to not over allocate instead and just try the available orders like TTM does. This has proven to be working architecture independent.

Do you mean to have an implementation similar to __ttm_pool_alloc()?

Yeah something like that.

https://web.git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/gpu/drm/ttm/ttm_pool.c?h=v6.15-rc2#n728

Quoted 7 lines

If that is the case, we can try the change, run some benchmarks and submit the patch if we see positive results.

Could be that this doesn't matter for your platform, but it's minimal extra overhead asking for larger chunks first and it just avoids fragmenting everything into 64K chunks.

That is kind of important since the code in DMA-heaps should be platform neutral if possible.

I agree, I'll make the change in another patch.

Thanks
Juan

Quoted 16 lines

Regards,
Christian.

Thanks
Juan

Regards,
Christian.

#define NUM_ORDERS ARRAY_SIZE(orders)

static struct sg_table *dup_sg_table(struct sg_table *table)