Why is the kfree() argument const?
Hi,
is there any reason why kfree() takes a const pointer just to degrade it with the call to slab_free()/__cache_free() again? The promise that the pointee is not modified is just bogus in this case, anyway, isn't it?
Hannes
Quoted 3 lines · Johannes Weiner
is there any reason why
kfree()takes a const pointer just to degrade it with the call to slab_free()/__cache_free() again? The promise that the pointee is not modified is just bogus in this case, anyway, isn't it?
The object is modified in various cases f.e. because of poisoning or the need to store the free pointer. So its bogus, yes. Pekka?
Quoted 8 lines · Christoph Lameter
Quoted 3 lines · Johannes Weiner
is there any reason why
kfree()takes a const pointer just to degrade it with the call to slab_free()/__cache_free() again? The promise that the pointee is not modified is just bogus in this case, anyway, isn't it?The object is modified in various cases f.e. because of poisoning or the need to store the free pointer. So its bogus, yes. Pekka?
Technically one should be able to pass a "const $type *" (which may have been a "non-const $type *" before but at some point in time it became "const $type *") to kfree().
The (formerly) constant contents as such vanishes IMHO (and it is not really "modified").
Poisoning and free memory handling is IMHO internal stuff to the free memory management subsystem and basically unrelated to the "life" of the pointered contents before it's death with kfree().
Bernd
Signature · 3 lines
Firmix Software GmbH http://www.firmix.at/
mobil: +43 664 4416156 fax: +43 1 7890849-55
Embedded Linux Development and ServicesHi,
Quoted 3 lines · Johannes Weiner
Quoted 3 lines
is there any reason why
kfree()takes a const pointer just to degrade it with the call to slab_free()/__cache_free() again? The promise that the pointee is not modified is just bogus in this case, anyway, isn't it?
Quoted 2 lines · Christoph Lameter
The object is modified in various cases f.e. because of poisoning or the need to store the free pointer. So its bogus, yes. Pekka?
Yeah, bogus, and has been that way for a long time according to git. I'm ok with removing that (which would make it consistent with the user-space equivalent free(3) function btw).
Pekka
Quoted 4 lines · Johannes Weiner
is there any reason why
kfree()takes a const pointer just to degrade it with the call to slab_free()/__cache_free() again? The promise that the pointee is not modified is just bogus in this case, anyway, isn't it?
"const" has *never* been about the thing not being modified. Forget all that claptrap. C does not have such a notion.
"const" is a pointer type issue, and is meant to make certain mis-uses more visible at compile time. It has *no* other meaning, and anybody who thinks it has is just setting himself up for problems.
In the particular case of kfree(), the pointer argument *should* be const, and the fact that the C library gets this wrong is not the kernels problem, it's a problem with the C library.
Why?
- From a very obvious and very *real* caller perspective, "free()" really doesn't change the thing the pointer points to. It does something totally different: it makes the *pointer* itself invalid.
In other words, if you think that "kfree()" changed the thing you free'd, you're simply wrong. It did no such thing. The memory is 100% the same, it's just that you cannot access it any more, and if you try, you'll get somebody elses memory.
In other words, "kfree()" can be const.
- Anything that *can* take a const pointer should always do so.
Why? Because we want the types to be as tight as possible, and normal code should need as few casts as possible.
Here's a question for you: let's say that you have a structure that has a member that is never changed. To make that obvious, and to allow the compiler to warn about mis-use of a pointer, the structure should look something like
struct mystruct {
const char *name;
..and let's look at what happens if the allocation of that const thing is dynamic.
The *correct* way to do that is:
char *name = kmalloc(...) /* Fill it in */ snprintf(name, ...) mystruct->name = name;
and there are no casts anywhere, and you get exactly the semantics you
want: "name" itself isn't constant (it's obviously modified), but at the same time the type system makes it very clear that trying to change it through that mystruct member pointer is wrong.
How do you free it?
That's right, you do:
kfree(mystruct->name);
and this is why "kfree()" should take a const pointer. If it doesn't, you have to add an *incorrect* and totally useless cast to code that was correct.
So never believe that "const" is some guarantee that the memory under the pointer doesn't change. That is *never* true. It has never been true in C, since there can be arbitrary pointer aliases to that memory that aren't actually const. If you think "const *p" means that the memory behind "p" is immutable, you're simply wrong.
Anybody who thinks that kfree() cannot (or should not) be const doesn't understand the C type system.
Linus
Hi,
Linus Torvalds <[email protected]> writes:
[...]
Quoted 3 lines
"const" is a pointer type issue, and is meant to make certain mis-uses more visible at compile time. It has *no* other meaning, and anybody who thinks it has is just setting himself up for problems.
[...]
Quoted 10 lines
- From a very obvious and very *real* caller perspective, "
free()" really doesn't change the thing the pointer points to. It does something totally different: it makes the *pointer* itself invalid.In other words, if you think that "
kfree()" changed the thing you free'd, you're simply wrong. It did no such thing. The memory is 100% the same, it's just that you cannot access it any more, and if you try, you'll get somebody elses memory.In other words, "
kfree()" can be const.
[...]
Quoted 5 lines
So never believe that "const" is some guarantee that the memory under the pointer doesn't change. That is *never* true. It has never been true in C, since there can be arbitrary pointer aliases to that memory that aren't actually const. If you think "const *p" means that the memory behind "p" is immutable, you're simply wrong.
Okay, I understood that now. A const qualifier just forbids modifying the underlying memory _through this particular pointer_, right?
In the case of slub's kfree(), which takes a const pointer, you pass it on to slab_free() which actually _DOES_ modification of the underlying memory when linking the object to the freelist (as far as I understood the code).
So if I got it right and you actually modify the memory you only got a const pointer to, you reach a level where you _have to_ break this policy and cast to a non-const pointer, as it is currently done in kfree(). No?
Hannes
Quoted 4 lines · Johannes Weiner
So if I got it right and you actually modify the memory you only got a const pointer to, you reach a level where you
_haveto_break this policy and cast to a non-const pointer, as it is currently done inkfree(). No?
Correct and we have gcc 4.2 currently spitting out warnings because of casting to non const. Any idea how to convince gcc that this is okay?
Hi,
Quoted 9 lines · Christoph Lameter
Quoted 4 lines · Johannes Weiner
So if I got it right and you actually modify the memory you only got a const pointer to, you reach a level where you
_haveto_break this policy and cast to a non-const pointer, as it is currently done inkfree(). No?Correct and we have gcc 4.2 currently spitting out warnings because of casting to non const. Any idea how to convince gcc that this is okay?
Two dirty hacks where gcc at least does not complain:
void *y = (void *)x;
and then pass y, or passing
*(void **)&x
directly.
Both approaches seem just too ugly to silence a bogus warning.
Hannes
Hi,
Quoted 9 lines · Christoph Lameter
Quoted 4 lines · Johannes Weiner
So if I got it right and you actually modify the memory you only got a const pointer to, you reach a level where you
_haveto_break this policy and cast to a non-const pointer, as it is currently done inkfree(). No?Correct and we have gcc 4.2 currently spitting out warnings because of casting to non const. Any idea how to convince gcc that this is okay?
Just for the record, this really seems to be a gcc bug, I can not explain this otherwise:
$ cat test.c #include <stdio.h>
static void test(void *p)
{
char *v = p;
puts(v);
}int main(void)
{
const char foo[] = "foo";
test((void *)foo);
return 0;
}
$ gcc -Wall test.c -o test
$ gcc -Wall -O test.c -o test
test.c: In function 'main':
test.c:12: warning: passing argument 1 of 'test' discards qualifiers from pointer target type
$ gcc -Wall -O -fno-inline test.c -o test
$gcc is version 4.2.2.
Hannes
Quoted 3 lines · Christoph Lameter
Correct and we have gcc 4.2 currently spitting out warnings because of casting to non const. Any idea how to convince gcc that this is okay?
Either don't use a broken compiler (casting a const pointer to a non-const is definitely not a bug), or cast to "unsigned long" (if it still complains, now the compiler is not just stupid, it's broken).
The whole point of memory management is that we know how pointers work, and understand that they have a *bit* representation, not just the C semantics.
Linus
Quoted 3 lines · Johannes Weiner
Okay, I understood that now. A const qualifier just forbids modifying the underlying memory
_throughthis particularpointer_, right?
Yes, exactly.
Quoted 4 lines
In the case of slub's
kfree(), which takes a const pointer, you pass it on toslab_free()which actually _DOES_ modification of the underlying memory when linking the object to the freelist (as far as I understood the code).
First off, the whole concept of "underlying memory" is all about the fact that pointers point not to separate objects with its own existence, but point to a shared resource ("memory") that can be accessed many different ways.
So immediately when you talk about "underlying memory", you need to realize that now you're not talking about "that pointer" any more, but you are talking about the stuff that *other* pointers can access.
And the whole (and *only*) point of a memory allocator is turning that amorphous notion of "infinitely aliasable underlying memory" model into more of a "C pointer to separate objects" model.
That *is* what a memory allocator does.
Quoted 2 lines
So if I got it right and you actually modify the memory you only got a const pointer to
No. YOU DO NOT.
You *invalidate* the pointer. Then, the memory allocator releases that portion of the "infinitely aliasable underlying memory", but that means that THAT POINTER NO LONGER EXISTS ON A C LEVEL.
So no, you don't "modify the memory you only got a const pointer to".
What you do is that the memory allocator
- keeps track of the NON-const memory that it always had
- it sees the const pointer you gave it, and uses that pointer to look up ITS OWN DATA STRUCTURES.
- it then accesses those memory locations using its own pointers.
The "const pointer" you passed to kfree() simply no longer exists. The object it pointed to has gone away. That particular pointer simply isn't valid any more.
But the memory management code, which maintains its own structures for what it has allocated and not allocated, can (and will) generally use the knowledge *it* has about its own pointers to modify its own data structures.
The fact that they (obviously) alias in memory is irrelevant. It has no meaning for the "const void *" you passed in. That pointer is not usable for the caller any more.
And no, this is not just a semantic argument. It's a real issue. The pointer you pass in to kfree() is obviously used to *find* the data structures that the memory allocator maintains, and there is almost inevitably a rather direct mapping between the passed-in pointers and the data structures that a memory manager maintains itself, but they really are different.
So to take a kfree() a pointer, the memory manager will use the "binary value" of that pointer to find its own data structures, and sometimes the values are so closely related that it ends up being a direct cast (possibly with an offset), but the end result of that direct cast really is now the memory management internal data, not the original pointer.
A real example of this is the actual kfree() implementation in the kernel (let's take the one from SLAB):
struct kmem_cache *c; .. c = virt_to_cache(objp);
which then actually ends up doing a
struct page *page = virt_to_head_page(obj); return page_get_cache(page);
which literally does math on the binary representation of the pointer to look up the backing store values (because the memory manager by definition is the thing that knows how the low-level representation of pointers actually works).
So now the kernel has turned that user-supplied pointer into ITS OWN data structures, that contain the NON-CONST aliases for the data.
(It will actually use the const pointer to further get offsets into those things, of course, so it's several levels of indirection off that pointer)
And yes, then it will write to those non-const aliases, but that's really no different from the same old thing: the pointer *you* passed kmalloc() may be "const", but the memory manager itself has its own data structures to keep track of *its* aliases to the same memory, and those are the ones that are actually used to re-use the memory for others (and poison it etc).
So the particular pointer you passed in is "const". But that never *ever* guarantees that there aren't other non-const pointers around to be used that can modify the memory. And the memory manegement definitely has all those aliased pointers around - it's the whole *point* of memory management.
Linus
Quoted 8 lines · Linus Torvalds
Quoted 4 lines · Johannes Weiner
is there any reason why
kfree()takes a const pointer just to degrade it with the call to slab_free()/__cache_free() again? The promise that the pointee is not modified is just bogus in this case, anyway, isn't it?
Quoted 6 lines
"const" has *never* been about the thing not being modified. Forget all that claptrap. C does not have such a notion.
"const" is a pointer type issue, and is meant to make certain mis-uses more visible at compile time. It has *no* other meaning, and anybody who thinks it has is just setting himself up for problems.
I totally agree with the above.
Quoted 4 lines
In the particular case of
kfree(), the pointer argument *should* be const, and the fact that the C library gets this wrong is not the kernels problem, it's a problem with the C library.
Here, I'm not so sure.
Quoted 6 lines
Why?
- From a very obvious and very *real* caller perspective, "
free()" really doesn't change the thing the pointer points to. It does something totally different: it makes the *pointer* itself invalid.
I'm OK with this.
Quoted 5 lines
In other words, if you think that "
kfree()" changed the thing you free'd, you're simply wrong. It did no such thing. The memory is 100% the same, it's just that you cannot access it any more, and if you try, you'll get somebody elses memory.
This too.
Quoted 2 lines
In other words, "
kfree()" can be const.
Err, not sure.
Quoted 5 lines
- Anything that *can* take a const pointer should always do so.
Why? Because we want the types to be as tight as possible, and normal code should need as few casts as possible.
OK
Quoted 25 lines
Here's a question for you: let's say that you have a structure that has a member that is never changed. To make that obvious, and to allow the compiler to warn about mis-use of a pointer, the structure should look something like
struct mystruct { const char *name; ..and let's look at what happens if the allocation of that const thing is dynamic.
The *correct* way to do that is:
char *name = kmalloc(...) /* Fill it in */ snprintf(name, ...) mystruct->name = name;and there are no casts anywhere, and you get exactly the semantics you
want: "name" itself isn't constant (it's obviously modified), but at the same time the type system makes it very clear that trying to change it through that mystruct member pointer is wrong.
Quoted 5 lines
How do you free it?
That's right, you do:
kfree(mystruct->name);
This is where I disagree. If a struct has a constant pointer to it, then the usage of that pointer by the struct should never modify it. If I need to allocate memory for a name to a struct, I would not expect that struct to ever free it.
Let's use your example. I'll assume that the struct was created by some constructor and the destructor freed it. I'd argue the correct way would be to have the kfree with a typecast.
Why?
- const pointers (especially strings) should be able to point to
static data. One thing that we would like to avoid is:
mystruct->name = "myobj";
...
kfree(mystruct->name);- really, kfree should match kmalloc for types. What kmalloc returns should be what kfree accepts. Passing in a const pointer to kfree *should* be a red flag that something might not be right.
Quoted 4 lines
and this is why "
kfree()" should take a const pointer. If it doesn't, you have to add an *incorrect* and totally useless cast to code that was correct.
char *name = kmalloc(...);
...
mystruct->name = name; /* this is an implicit cast */
So adding a cast to kfree isn't incorrect. C automatically casts name to a const pointer, which means if we want to free it, then we should cast it back.
Quoted 6 lines
So never believe that "const" is some guarantee that the memory under the pointer doesn't change. That is *never* true. It has never been true in C, since there can be arbitrary pointer aliases to that memory that aren't actually const. If you think "const *p" means that the memory behind "p" is immutable, you're simply wrong.
Again, I totally agree with the above.
Quoted 3 lines
Anybody who thinks that
kfree()cannot (or should not) be const doesn't understand the C type system.
OK, I think that kfree should not be const, but not because of the explanation that you gave, but because of the C type system in general.
kfree should match the kmalloc type. We don't declare
const void *kmalloc(...)
so we shouldn't do the same with kfree. If you assign a const pointer to something from kmalloc, C implicitly does the cast. This doesn't mean that we should ignore doing the cast back in kfree. Especially since this could help us avoid the kfree("mystring") issue.
Just my $0.02
-- Steve