fix xnheap_alloc rounding

PATCH In review Jan Kiszka 3 participants 6 messages Jul 11, 2006 – Jul 15, 2006

lore

Share & tools

lore

Jan Kiszka

author

2006-07-11 14:02 UTC #0 lore raw

PATCH 1/1 fix xnheap_alloc rounding

Hi,

playing a stupid rt_heap user (actually I didn't just play this...), I stumbled over this undocumented oddity:

rt_heap_create(&heap, name, 10000, H_PRIO|H_MAPPABLE); rt_heap_alloc(&heap, 10000, TM_NONBLOCK, &ptr);

Creation is successful, allocation fails. The reason: while during creation the net heap size is rounded down to page boundaries, the allocation of memory > PAGE_SIZE is rounded up. One could add H_SINGLE to the flags, but this may even result in allocating less memory than the user expected, causing severe problems later.

How to resolve this best? I thought about rounding twice in rt_head_create (one time the net size, the second time including the overhead), but this encodes characteristics of the underlying heap allocator into the skin (I have a generic heap allocator framework in mind for 2.3). So I decided to do this rounding in xnheap_overhead() instead, see attached patch. Hope I didn't skewed up any calculation. At least the scenario above now works fine.

Jan

nucleus/heap.h +8 -2

Index: include/nucleus/heap.h
===================================================================
@@ -109,8 +109,14 @@ extern xnheap_t kheap;
 #define xnheap_used_mem(heap)        ((heap)->ubytes)
 #define xnheap_max_contiguous(heap)  ((heap)->maxcont)
 #define xnheap_overhead(hsize,psize) \
-((sizeof(xnextent_t) + (((hsize) - sizeof(xnextent_t)) / (psize)) + \
- XNHEAP_MINALIGNSZ - 1) & ~(XNHEAP_MINALIGNSZ - 1))
+({ \
+    u_long rounded_hsize = (hsize + psize - 1) & ~(psize - 1); \
+    u_long overhead = ((sizeof(xnextent_t) + \
+	(((rounded_hsize) - sizeof(xnextent_t)) / (psize)) + \
+	XNHEAP_MINALIGNSZ - 1) & ~(XNHEAP_MINALIGNSZ - 1)); \
+    overhead += rounded_hsize - hsize; \
+    overhead; \
+})
 
 #define xnmalloc(size)     xnheap_alloc(&kheap,size)
 #define xnfree(ptr)        xnheap_free(&kheap,ptr)

Gilles Chanteperdrix

2006-07-12 12:26 UTC #1 lore raw

Quoted 14 lines · Jan Kiszka

Hi,
playing a stupid rt_heap user (actually I didn't just play this...), I stumbled over this undocumented oddity:
rt_heap_create(&heap, name, 10000, H_PRIO|H_MAPPABLE); rt_heap_alloc(&heap, 10000, TM_NONBLOCK, &ptr);
Creation is successful, allocation fails. The reason: while during creation the net heap size is rounded down to page boundaries, the allocation of memory > PAGE_SIZE is rounded up. One could add H_SINGLE to the flags, but this may even result in allocating less memory than the user expected, causing severe problems later.

Quoted 7 lines

How to resolve this best? I thought about rounding twice in rt_head_create (one time the net size, the second time including the overhead), but this encodes characteristics of the underlying heap allocator into the skin (I have a generic heap allocator framework in mind for 2.3). So I decided to do this rounding in xnheap_overhead() instead, see attached patch. Hope I didn't skewed up any calculation. At least the scenario above now works fine.

The problem I see with this patch is that you are counting (rounded_hsize - hsize) as part of the overhead, whereas you would like to count it as free space. Would not it make more sense to do the rounding in xnheap_init_mapped ?

Signature · 1 line

					    Gilles Chanteperdrix.

Jan Kiszka

author

2006-07-12 12:38 UTC #2 lore raw

Quoted 23 lines · Gilles Chanteperdrix

Quoted 14 lines · Jan Kiszka
Hi,
playing a stupid rt_heap user (actually I didn't just play this...), I stumbled over this undocumented oddity:
rt_heap_create(&heap, name, 10000, H_PRIO|H_MAPPABLE); rt_heap_alloc(&heap, 10000, TM_NONBLOCK, &ptr);
Creation is successful, allocation fails. The reason: while during creation the net heap size is rounded down to page boundaries, the allocation of memory > PAGE_SIZE is rounded up. One could add H_SINGLE to the flags, but this may even result in allocating less memory than the user expected, causing severe problems later.
Quoted 7 lines
How to resolve this best? I thought about rounding twice in rt_head_create (one time the net size, the second time including the overhead), but this encodes characteristics of the underlying heap allocator into the skin (I have a generic heap allocator framework in mind for 2.3). So I decided to do this rounding in xnheap_overhead() instead, see attached patch. Hope I didn't skewed up any calculation. At least the scenario above now works fine.

Quoted 5 lines

The problem I see with this patch is that you are counting (rounded_hsize - hsize) as part of the overhead, whereas you would like to count it as free space. Would not it make more sense to do the rounding in xnheap_init_mapped ?

I was thinking of the open-coded

allocate(heap_size + xnheap_overhead)
xnheap_init()

in many skins as well.

Jan

Philippe Gerum

2006-07-15 01:48 UTC #3 lore raw

In reply to Jan Kiszka

Quoted 14 lines · Jan Kiszka

Hi,
playing a stupid rt_heap user (actually I didn't just play this...), I stumbled over this undocumented oddity:
rt_heap_create(&heap, name, 10000, H_PRIO|H_MAPPABLE); rt_heap_alloc(&heap, 10000, TM_NONBLOCK, &ptr);
Creation is successful, allocation fails. The reason: while during creation the net heap size is rounded down to page boundaries, the allocation of memory > PAGE_SIZE is rounded up. One could add H_SINGLE to the flags, but this may even result in allocating less memory than the user expected, causing severe problems later.

Quoted 5 lines

How to resolve this best? I thought about rounding twice in rt_head_create (one time the net size, the second time including the overhead), but this encodes characteristics of the underlying heap allocator into the skin (I have a generic heap allocator framework in mind for 2.3).

The nucleus heap expects the caller to pass it the memory which is going to be used to fulfill allocation requests, so there is already a builtin dependency from any client heap manager on the nucleus implementation. Double rounding is therefore the way to go, since callers need to be fixed so that they provide enough memory to the xnheap manager when specializing its behaviour.

Quoted 31 lines

So I decided to do this rounding in xnheap_overhead()
instead, see attached patch. Hope I didn't skewed up any calculation. At least the scenario above now works fine.
Jan
plain text document attachment (xnheap_overhead-fix-rounding.patch)
Index: include/nucleus/heap.h
===================================================================
nucleus/heap.h +8 -2
@@ -109,8 +109,14 @@ extern xnheap_t kheap;
 #define xnheap_used_mem(heap)        ((heap)->ubytes)
 #define xnheap_max_contiguous(heap)  ((heap)->maxcont)
 #define xnheap_overhead(hsize,psize) \
-((sizeof(xnextent_t) + (((hsize) - sizeof(xnextent_t)) / (psize)) + \
- XNHEAP_MINALIGNSZ - 1) & ~(XNHEAP_MINALIGNSZ - 1))
+({ \
+    u_long rounded_hsize = (hsize + psize - 1) & ~(psize - 1); \
+    u_long overhead = ((sizeof(xnextent_t) + \
+	(((rounded_hsize) - sizeof(xnextent_t)) / (psize)) + \
+	XNHEAP_MINALIGNSZ - 1) & ~(XNHEAP_MINALIGNSZ - 1)); \
+    overhead += rounded_hsize - hsize; \
+    overhead; \
+})
 
 #define xnmalloc(size)     xnheap_alloc(&kheap,size)
 #define xnfree(ptr)        xnheap_free(&kheap,ptr)
_______________________________________________
Xenomai-core mailing list
[email protected]
https://mail.gna.org/listinfo/xenomai-core

Signature · 1 line

Philippe.

Jan Kiszka

author

2006-07-15 08:07 UTC #4 lore raw

Quoted 21 lines · Philippe Gerum

Quoted 14 lines · Jan Kiszka
Hi,
playing a stupid rt_heap user (actually I didn't just play this...), I stumbled over this undocumented oddity:
rt_heap_create(&heap, name, 10000, H_PRIO|H_MAPPABLE); rt_heap_alloc(&heap, 10000, TM_NONBLOCK, &ptr);
Creation is successful, allocation fails. The reason: while during creation the net heap size is rounded down to page boundaries, the allocation of memory > PAGE_SIZE is rounded up. One could add H_SINGLE to the flags, but this may even result in allocating less memory than the user expected, causing severe problems later.
Quoted 5 lines
How to resolve this best? I thought about rounding twice in rt_head_create (one time the net size, the second time including the overhead), but this encodes characteristics of the underlying heap allocator into the skin (I have a generic heap allocator framework in mind for 2.3).

Quoted 6 lines

The nucleus heap expects the caller to pass it the memory which is going to be used to fulfill allocation requests, so there is already a builtin dependency from any client heap manager on the nucleus implementation. Double rounding is therefore the way to go, since callers need to be fixed so that they provide enough memory to the xnheap manager when specializing its behaviour.

If this means to do the rounding in the clients without the help of some well-defined xnheap-API, then I disagree. I have an alternative nucleus heap manager in mind for the future that will have a different rounding strategy for allocations. So we should cleanly hide such property behind a xnheap service - either in the overhead calculation or in some additional rounding service. But I do not see a need for splitting this up yet.

Quoted 32 lines

Quoted 32 lines
So I decided to do this rounding in xnheap_overhead()
instead, see attached patch. Hope I didn't skewed up any calculation. At least the scenario above now works fine.
Jan
plain text document attachment (xnheap_overhead-fix-rounding.patch)
Index: include/nucleus/heap.h
===================================================================
nucleus/heap.h +8 -2
@@ -109,8 +109,14 @@ extern xnheap_t kheap;
 #define xnheap_used_mem(heap)        ((heap)->ubytes)
 #define xnheap_max_contiguous(heap)  ((heap)->maxcont)
 #define xnheap_overhead(hsize,psize) \
-((sizeof(xnextent_t) + (((hsize) - sizeof(xnextent_t)) / (psize)) + \
- XNHEAP_MINALIGNSZ - 1) & ~(XNHEAP_MINALIGNSZ - 1))
+({ \
+    u_long rounded_hsize = (hsize + psize - 1) & ~(psize - 1); \
+    u_long overhead = ((sizeof(xnextent_t) + \
+	(((rounded_hsize) - sizeof(xnextent_t)) / (psize)) + \
+	XNHEAP_MINALIGNSZ - 1) & ~(XNHEAP_MINALIGNSZ - 1)); \
+    overhead += rounded_hsize - hsize; \
+    overhead; \
+})
 
 #define xnmalloc(size)     xnheap_alloc(&kheap,size)
 #define xnfree(ptr)        xnheap_free(&kheap,ptr)
_______________________________________________
Xenomai-core mailing list
[email protected]
https://mail.gna.org/listinfo/xenomai-core

Philippe Gerum

2006-07-15 08:19 UTC #5 lore raw

Quoted 29 lines · Jan Kiszka

Quoted 21 lines · Philippe Gerum
Quoted 14 lines · Jan Kiszka
Hi,
playing a stupid rt_heap user (actually I didn't just play this...), I stumbled over this undocumented oddity:
rt_heap_create(&heap, name, 10000, H_PRIO|H_MAPPABLE); rt_heap_alloc(&heap, 10000, TM_NONBLOCK, &ptr);
Creation is successful, allocation fails. The reason: while during creation the net heap size is rounded down to page boundaries, the allocation of memory > PAGE_SIZE is rounded up. One could add H_SINGLE to the flags, but this may even result in allocating less memory than the user expected, causing severe problems later.
Quoted 5 lines
How to resolve this best? I thought about rounding twice in rt_head_create (one time the net size, the second time including the overhead), but this encodes characteristics of the underlying heap allocator into the skin (I have a generic heap allocator framework in mind for 2.3).
Quoted 6 lines
The nucleus heap expects the caller to pass it the memory which is going to be used to fulfill allocation requests, so there is already a builtin dependency from any client heap manager on the nucleus implementation. Double rounding is therefore the way to go, since callers need to be fixed so that they provide enough memory to the xnheap manager when specializing its behaviour.

Quoted 7 lines

If this means to do the rounding in the clients without the help of some well-defined xnheap-API, then I disagree. I have an alternative nucleus heap manager in mind for the future that will have a different rounding strategy for allocations. So we should cleanly hide such property behind a xnheap service - either in the overhead calculation or in some additional rounding service. But I do not see a need for splitting this up yet.

xnheap_overhead() needs to remain that way, i.e. compute the exact number of additional bytes used for housekeeping purposes by the heap manager. Fixing the clients by providing them the required support to perform proper rounding without resorting to open-coded calculations is another issue, and is indeed the way to go. I'll handle that.

Quoted 35 lines

Quoted 33 lines
Quoted 33 lines
So I decided to do this rounding in xnheap_overhead()
instead, see attached patch. Hope I didn't skewed up any calculation. At least the scenario above now works fine.
Jan
plain text document attachment (xnheap_overhead-fix-rounding.patch)
Index: include/nucleus/heap.h
===================================================================
nucleus/heap.h +8 -2
@@ -109,8 +109,14 @@ extern xnheap_t kheap;
 #define xnheap_used_mem(heap)        ((heap)->ubytes)
 #define xnheap_max_contiguous(heap)  ((heap)->maxcont)
 #define xnheap_overhead(hsize,psize) \
-((sizeof(xnextent_t) + (((hsize) - sizeof(xnextent_t)) / (psize)) + \
- XNHEAP_MINALIGNSZ - 1) & ~(XNHEAP_MINALIGNSZ - 1))
+({ \
+    u_long rounded_hsize = (hsize + psize - 1) & ~(psize - 1); \
+    u_long overhead = ((sizeof(xnextent_t) + \
+	(((rounded_hsize) - sizeof(xnextent_t)) / (psize)) + \
+	XNHEAP_MINALIGNSZ - 1) & ~(XNHEAP_MINALIGNSZ - 1)); \
+    overhead += rounded_hsize - hsize; \
+    overhead; \
+})
 
 #define xnmalloc(size)     xnheap_alloc(&kheap,size)
 #define xnfree(ptr)        xnheap_free(&kheap,ptr)
_______________________________________________
Xenomai-core mailing list
[email protected]
https://mail.gna.org/listinfo/xenomai-core

Signature · 1 line

Philippe.