Skip to contentSkip to conversation

fix qemu_malloc() error check for size==0

PATCH In review Eduardo Habkost 5 participants 33 messages May 18, 2009 – May 20, 2009

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 fix qemu_malloc() error check for size==0

This patch is similar to a previous qemu_realloc() fix (commit 322691a5c9f1c8531554148d47c078b5be590805), but for qemu_malloc().

malloc(0) may correctly return NULL if size==0. We don't want to abort qemu on this case.

---
 qemu-malloc.c |   11 ++++-------
 1 files changed, 4 insertions(+), 7 deletions(-)
qemu-malloc.c +4 -7
index 6761857..2c60969 100644
@@ -24,9 +24,9 @@
 #include "qemu-common.h"
 #include <stdlib.h>
 
-static void *oom_check(void *ptr)
+static void *oom_check(size_t size, void *ptr)
 {
-    if (ptr == NULL)
+    if (size != 0 && ptr == NULL)
         abort();
     return ptr;
 }
@@ -43,15 +43,12 @@ void qemu_free(void *ptr)
 
 void *qemu_malloc(size_t size)
 {
-    return oom_check(malloc(size));
+    return oom_check(size, malloc(size));
 }
 
 void *qemu_realloc(void *ptr, size_t size)
 {
-    if (size)
-        return oom_check(realloc(ptr, size));
-    else
-        return realloc(ptr, size);
+    return oom_check(size, realloc(ptr, size));
 }
 
 void *qemu_mallocz(size_t size)
Signature · 1 line
1.6.3.rc4.29.g8146
#1 lore raw
Quoted 5 lines · Eduardo Habkost

This patch is similar to a previous qemu_realloc() fix (commit 322691a5c9f1c8531554148d47c078b5be590805), but for qemu_malloc().

malloc(0) may correctly return NULL if size==0. We don't want to abort qemu on this case.

Only it wouldn't (on Linux):

$ cat malloc.c
#include <stdlib.h>

int main (void)

{
    printf ("%p\n", malloc (0));
    return 0;
}
$ gcc malloc.c
$ ./a.out 
0x10011008

Standard (in 7.20.3) says that malloc's behaviour in case of size being zero is implementation defined.

Try `git show 63c75dcd669d011f438421980b4379827da4bb1c'.

The best(only?) thing to do is to check size passed to qemu_malloc[z] and abort the program if this situation is encountered.

Signature · 1 line
mailto:[email protected]
#2 lore raw
Quoted 22 lines · malc
Quoted 5 lines · Eduardo Habkost

This patch is similar to a previous qemu_realloc() fix (commit 322691a5c9f1c8531554148d47c078b5be590805), but for qemu_malloc().

malloc(0) may correctly return NULL if size==0. We don't want to abort qemu on this case.

Only it wouldn't (on Linux):

$ cat malloc.c
#include <stdlib.h>

int main (void)

{
    printf ("%p\n", malloc (0));
    return 0;
}
$ gcc malloc.c
$ ./a.out 
0x10011008
Quoted 7 lines

Standard (in 7.20.3) says that malloc's behaviour in case of size being zero is implementation defined.

Try `git show 63c75dcd669d011f438421980b4379827da4bb1c'.

The best(only?) thing to do is to check size passed to qemu_malloc[z] and abort the program if this situation is encountered.

Why? malloc(0) is as valid as realloc(p, 0). It will either return NULL or a pointer, and on any case the value can be safely passed to free() later.

Signature · 1 line
Eduardo
#3 lore raw
Quoted 26 lines · Eduardo Habkost
Quoted 25 lines · malc
Quoted 5 lines · Eduardo Habkost

This patch is similar to a previous qemu_realloc() fix (commit 322691a5c9f1c8531554148d47c078b5be590805), but for qemu_malloc().

malloc(0) may correctly return NULL if size==0. We don't want to abort qemu on this case.

Only it wouldn't (on Linux):

$ cat malloc.c
#include <stdlib.h>

int main (void)

{
    printf ("%p\n", malloc (0));
    return 0;
}
$ gcc malloc.c
$ ./a.out 
0x10011008

Standard (in 7.20.3) says that malloc's behaviour in case of size being zero is implementation defined.

Quoted 8 lines
Quoted 4 lines

Try `git show 63c75dcd669d011f438421980b4379827da4bb1c'.

The best(only?) thing to do is to check size passed to qemu_malloc[z] and abort the program if this situation is encountered.

Why? malloc(0) is as valid as realloc(p, 0). It will either return NULL or a pointer, and on any case the value can be safely passed to free() later.

I believe you haven't examined the commit that i referenced. Thing is existing code used to, i'd venture a guess accidentaly, rely on the behaviour that current GLIBC provides and consequently failed to operate on AIX where malloc(0) returns NULL, IOW making qemu_malloc[z] return whatever the underlying system returns is just hiding the bugs, the code becomes unportable.

Signature · 1 line
mailto:[email protected]
#4 lore raw
Quoted 11 lines · malc
Quoted 9 lines · Eduardo Habkost
Quoted 8 lines · malc
Quoted 5 lines · Eduardo Habkost

This patch is similar to a previous qemu_realloc() fix (commit 322691a5c9f1c8531554148d47c078b5be590805), but for qemu_malloc().

malloc(0) may correctly return NULL if size==0. We don't want to abort qemu on this case.

[...]

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

Standard (in 7.20.3) says that malloc's behaviour in case of size being zero is implementation defined.

Try `git show 63c75dcd669d011f438421980b4379827da4bb1c'.

Quoted 8 lines
Quoted 6 lines
Quoted 2 lines

The best(only?) thing to do is to check size passed to qemu_malloc[z] and abort the program if this situation is encountered.

Why? malloc(0) is as valid as realloc(p, 0). It will either return NULL or a pointer, and on any case the value can be safely passed to free() later.

I believe you haven't examined the commit that i referenced. Thing is

Well, I just did, but I don't get it.

    diff --git a/block-qcow2.c b/block-qcow2.c
    index 9aa7261..d4556ef 100644
    --- a/block-qcow2.c
    +++ b/block-qcow2.c
    @@ -1809,6 +1809,12 @@ static int qcow_read_snapshots(BlockDriverState *bs)
         int64_t offset;
         uint32_t extra_data_size;

    +    if (!s->nb_snapshots) {
    +        s->snapshots = NULL;
    +        s->snapshots_size = 0;
    +        return 0;
    +    }
    +
         offset = s->snapshots_offset;
         s->snapshots = qemu_mallocz(s->nb_snapshots * sizeof(QCowSnapshot));
         if (!s->snapshots)

Can't see what this hunk accomplishes. If we remove it, the loop rejects, and we thus execute:

    offset = s->snapshots_offset;
    s->snapshots = qemu_mallocz(s->nb_snapshots * sizeof(QCowSnapshot));
    s->snapshots_size = offset - s->snapshots_offset;
    return 0;

Next hunk:

@@ -2023,8 +2029,10 @@ static int qcow_snapshot_create(BlockDriverState *bs,
snapshots1 = qemu_malloc((s->nb_snapshots + 1) * sizeof(QCowSnapshot)); if (!snapshots1)
goto fail;
- memcpy(snapshots1, s->snapshots, s->nb_snapshots * sizeof(QCowSnapshot));
- qemu_free(s->snapshots);
+ if (s->snapshots) {
+ memcpy(snapshots1, s->snapshots, s->nb_snapshots * sizeof(QCowSnapshot)
);
+ qemu_free(s->snapshots);
+ }
s->snapshots = snapshots1;
s->snapshots[s->nb_snapshots++] = *sn;

Again, I can't see the point. if !s->snapshots, then s->nb_snapshots == 0, doesn't it? memcpy(snapshots1, NULL, 0) does nothing. free(NULL) does nothing.

I'm sure you didn't patch this for nothing, so I must miss something. Could you enlighten me?

Quoted 3 lines

existing code used to, i'd venture a guess accidentaly, rely on the behaviour that current GLIBC provides and consequently failed to operate on AIX where malloc(0) returns NULL,

Common error.

Quoted 3 lines

IOW making qemu_malloc[z]
return whatever the underlying system returns is just hiding the bugs, the code becomes unportable.

Matter of taste.

1. Deal with the implementation-definedness. Every caller that could pass zero needs to take care not to confuse empty allocation with an out of memory condition.

This is easier than it sounds when you check for out of memory in just one place, like we do.

2. Remove the implementation-definedness. Easiest way is to detect zero size in a wrapper (for us: qemu_malloc()) and bump it to one.

qemu_realloc() currently uses 1.

realloc(NULL, sz) is specified to be equivalent to malloc(sz). It would be kind of nice to keep that for qemu_realloc() and qemu_malloc().

#5 lore raw
Quoted 22 lines · Markus Armbruster
Quoted 11 lines · malc
Quoted 9 lines · Eduardo Habkost
Quoted 8 lines · malc
Quoted 5 lines · Eduardo Habkost

This patch is similar to a previous qemu_realloc() fix (commit 322691a5c9f1c8531554148d47c078b5be590805), but for qemu_malloc().

malloc(0) may correctly return NULL if size==0. We don't want to abort qemu on this case.

[...]

Quoted 8 lines
Quoted 8 lines
Quoted 7 lines

Standard (in 7.20.3) says that malloc's behaviour in case of size being zero is implementation defined.

Try `git show 63c75dcd669d011f438421980b4379827da4bb1c'.

The best(only?) thing to do is to check size passed to qemu_malloc[z] and abort the program if this situation is encountered.

Quoted 7 lines
Quoted 5 lines
Quoted 3 lines

Why? malloc(0) is as valid as realloc(p, 0). It will either return NULL or a pointer, and on any case the value can be safely passed to free() later.

I believe you haven't examined the commit that i referenced. Thing is

Well, I just did, but I don't get it.

Pretty good sign that something is wrong, no?

Quoted 19 lines
    diff --git a/block-qcow2.c b/block-qcow2.c
    index 9aa7261..d4556ef 100644
    --- a/block-qcow2.c
    +++ b/block-qcow2.c
    @@ -1809,6 +1809,12 @@ static int qcow_read_snapshots(BlockDriverState *bs)
         int64_t offset;
         uint32_t extra_data_size;

    +    if (!s->nb_snapshots) {
    +        s->snapshots = NULL;
    +        s->snapshots_size = 0;
    +        return 0;
    +    }
    +
         offset = s->snapshots_offset;
         s->snapshots = qemu_mallocz(s->nb_snapshots * sizeof(QCowSnapshot));
         if (!s->snapshots)
Quoted 7 lines

Can't see what this hunk accomplishes. If we remove it, the loop rejects, and we thus execute:

    offset = s->snapshots_offset;
    s->snapshots = qemu_mallocz(s->nb_snapshots * sizeof(QCowSnapshot));
    s->snapshots_size = offset - s->snapshots_offset;
    return 0;

Uh, no.

$ git cat-file -p 63c75dcd669d011f438421980b4379827da4bb1c^:block-qcow2.c | sed -n 1811,1815p
    offset = s->snapshots_offset;
    s->snapshots = qemu_mallocz(s->nb_snapshots * sizeof(QCowSnapshot));
    if (!s->snapshots)
        goto fail;

So before the patch qemu_mallocz(0) call would return NULL, goto fail would be executed and `qcow_read_snapshots' will return -1 and QEMU will fail to start with very helpful message:
qemu: could not open disk image <image name>

Quoted 17 lines

Next hunk:

@@ -2023,8 +2029,10 @@ static int qcow_snapshot_create(BlockDriverState *bs,
snapshots1 = qemu_malloc((s->nb_snapshots + 1) * sizeof(QCowSnapshot)); if (!snapshots1)
goto fail;
- memcpy(snapshots1, s->snapshots, s->nb_snapshots * sizeof(QCowSnapshot));
- qemu_free(s->snapshots);
+ if (s->snapshots) {
+ memcpy(snapshots1, s->snapshots, s->nb_snapshots * sizeof(QCowSnapshot)
);
+ qemu_free(s->snapshots);
+ }
s->snapshots = snapshots1;
s->snapshots[s->nb_snapshots++] = *sn;

Quoted 3 lines

Again, I can't see the point. if !s->snapshots, then s->nb_snapshots == 0, doesn't it? memcpy(snapshots1, NULL, 0) does nothing. free(NULL) does nothing.

Wish i had your confidence and analysis skills, after skimming over 2666 lines of C in block-qcow2.c i can safely assert that this is indeed the only condition under which s->snapshots can be NULL, hence this hunk.

Quoted 3 lines

I'm sure you didn't patch this for nothing, so I must miss something. Could you enlighten me?

I hope that the above explanation is good enough, please do tell if it's not.

Also please notice how throwing second implemenation-defined behaviour of malloc in to the mix led to the hunk #2, in which you see no point.

Quoted 5 lines
Quoted 3 lines

existing code used to, i'd venture a guess accidentaly, rely on the behaviour that current GLIBC provides and consequently failed to operate on AIX where malloc(0) returns NULL,

Common error.

Made by a very skilled programmer i might add.

Quoted 10 lines
Quoted 3 lines

IOW making qemu_malloc[z]
return whatever the underlying system returns is just hiding the bugs, the code becomes unportable.

Matter of taste.

1. Deal with the implementation-definedness. Every caller that could pass zero needs to take care not to confuse empty allocation with an out of memory condition.

Quoted 5 lines

This is easier than it sounds when you check for out of memory in just one place, like we do.

2. Remove the implementation-definedness. Easiest way is to detect zero size in a wrapper (for us: qemu_malloc()) and bump it to one.

And mine:
3. Abort the program if somebody tries it. Because so far history thought me that nobody does 1.

Quoted 6 lines

qemu_realloc() currently uses 1.

realloc(NULL, sz) is specified to be equivalent to malloc(sz). It would be kind of nice to keep that for qemu_realloc() and qemu_malloc().

qemu_realloc shouldn't be called qemu_realloc if doesn't do that. The part about qemu_malloc escapes me.

Signature · 1 line
mailto:[email protected]
#6 lore raw
Quoted 28 lines · malc
Quoted 26 lines · Markus Armbruster
Quoted 11 lines · malc
Quoted 9 lines · Eduardo Habkost
Quoted 8 lines · malc
Quoted 5 lines · Eduardo Habkost

This patch is similar to a previous qemu_realloc() fix (commit 322691a5c9f1c8531554148d47c078b5be590805), but for qemu_malloc().

malloc(0) may correctly return NULL if size==0. We don't want to abort qemu on this case.

[...]

Quoted 12 lines
Quoted 11 lines
Quoted 7 lines

Standard (in 7.20.3) says that malloc's behaviour in case of size being zero is implementation defined.

Try `git show 63c75dcd669d011f438421980b4379827da4bb1c'.

The best(only?) thing to do is to check size passed to qemu_malloc[z] and abort the program if this situation is encountered.

Why? malloc(0) is as valid as realloc(p, 0). It will either return NULL or a pointer, and on any case the value can be safely passed to free() later.

Quoted 5 lines
Quoted 3 lines
Quoted 1 line

I believe you haven't examined the commit that i referenced. Thing is

Well, I just did, but I don't get it.

Pretty good sign that something is wrong, no?

That's why I'm asking :)

Quoted 34 lines
Quoted 18 lines
    diff --git a/block-qcow2.c b/block-qcow2.c
    index 9aa7261..d4556ef 100644
    --- a/block-qcow2.c
    +++ b/block-qcow2.c
    @@ -1809,6 +1809,12 @@ static int qcow_read_snapshots(BlockDriverState *bs)
         int64_t offset;
         uint32_t extra_data_size;

    +    if (!s->nb_snapshots) {
    +        s->snapshots = NULL;
    +        s->snapshots_size = 0;
    +        return 0;
    +    }
    +
         offset = s->snapshots_offset;
         s->snapshots = qemu_mallocz(s->nb_snapshots * sizeof(QCowSnapshot));
         if (!s->snapshots)
Quoted 7 lines

Can't see what this hunk accomplishes. If we remove it, the loop rejects, and we thus execute:

    offset = s->snapshots_offset;
    s->snapshots = qemu_mallocz(s->nb_snapshots * sizeof(QCowSnapshot));
    s->snapshots_size = offset - s->snapshots_offset;
    return 0;

Uh, no.

$ git cat-file -p 63c75dcd669d011f438421980b4379827da4bb1c^:block-qcow2.c | sed -n 1811,1815p
    offset = s->snapshots_offset;
    s->snapshots = qemu_mallocz(s->nb_snapshots * sizeof(QCowSnapshot));
    if (!s->snapshots)
        goto fail;
Quoted 4 lines

So before the patch qemu_mallocz(0) call would return NULL, goto fail would be executed and `qcow_read_snapshots' will return -1 and QEMU will fail to start with very helpful message:
qemu: could not open disk image <image name>

The test went away in commit 3ec88e80. The hunk became pointless only then.

Quoted 25 lines
Quoted 16 lines

Next hunk:

@@ -2023,8 +2029,10 @@ static int qcow_snapshot_create(BlockDriverState *bs,
snapshots1 = qemu_malloc((s->nb_snapshots + 1) * sizeof(QCowSnapshot)); if (!snapshots1)
goto fail;
- memcpy(snapshots1, s->snapshots, s->nb_snapshots * sizeof(QCowSnapshot));
- qemu_free(s->snapshots);
+ if (s->snapshots) {
+ memcpy(snapshots1, s->snapshots, s->nb_snapshots * sizeof(QCowSnapshot)
);
+ qemu_free(s->snapshots);
+ }
s->snapshots = snapshots1;
s->snapshots[s->nb_snapshots++] = *sn;

Quoted 3 lines

Again, I can't see the point. if !s->snapshots, then s->nb_snapshots == 0, doesn't it? memcpy(snapshots1, NULL, 0) does nothing. free(NULL) does nothing.

Wish i had your confidence and analysis skills, after skimming over 2666 lines of C in block-qcow2.c i can safely assert that this is indeed the only condition under which s->snapshots can be NULL, hence this hunk.

Quoted 5 lines
Quoted 3 lines

I'm sure you didn't patch this for nothing, so I must miss something. Could you enlighten me?

I hope that the above explanation is good enough, please do tell if it's not.

Thanks.

Quoted 10 lines

Also please notice how throwing second implemenation-defined behaviour of malloc in to the mix led to the hunk #2, in which you see no point.

Quoted 5 lines
Quoted 3 lines

existing code used to, i'd venture a guess accidentaly, rely on the behaviour that current GLIBC provides and consequently failed to operate on AIX where malloc(0) returns NULL,

Common error.

Made by a very skilled programmer i might add.

I've fallen into this trap myself.

Quoted 13 lines
Quoted 13 lines
Quoted 3 lines

IOW making qemu_malloc[z]
return whatever the underlying system returns is just hiding the bugs, the code becomes unportable.

Matter of taste.

1. Deal with the implementation-definedness. Every caller that could pass zero needs to take care not to confuse empty allocation with an out of memory condition.

This is easier than it sounds when you check for out of memory in just one place, like we do.

Quoted 6 lines
Quoted 2 lines

2. Remove the implementation-definedness. Easiest way is to detect zero size in a wrapper (for us: qemu_malloc()) and bump it to one.

And mine:
3. Abort the program if somebody tries it. Because so far history thought me that nobody does 1.

Tries what? Passing zero to qemu_malloc()? That's legitimate. And with allocation functions that cannot return failure, it's hardly dangerous, isn't it?

Quoted 8 lines
Quoted 5 lines

qemu_realloc() currently uses 1.

realloc(NULL, sz) is specified to be equivalent to malloc(sz). It would be kind of nice to keep that for qemu_realloc() and qemu_malloc().

qemu_realloc shouldn't be called qemu_realloc if doesn't do that. The part about qemu_malloc escapes me.

qemu_malloc() & friends never fail. Checking their value for failure is pointless. Therefore, 1. is practical.

2. is certainly practical as well.

3. is like 2, with the (size ? size : 1) pushed into callers. I find that mildly annoying.

I don't care whether we pick 1., 2. or 3., but I'd prefer we pick the same for qemu_malloc() and qemu_realloc(). We currently use 1. for qemu_realloc(). My point is: if you pick something else for qemu_malloc(), please consider changing qemu_realloc(). That's all.

#7 lore raw
Quoted 2 lines · Markus Armbruster

malc <[email protected]> writes:

[..snip..]

Quoted 18 lines
Quoted 18 lines
Quoted 18 lines
    diff --git a/block-qcow2.c b/block-qcow2.c
    index 9aa7261..d4556ef 100644
    --- a/block-qcow2.c
    +++ b/block-qcow2.c
    @@ -1809,6 +1809,12 @@ static int qcow_read_snapshots(BlockDriverState *bs)
         int64_t offset;
         uint32_t extra_data_size;

    +    if (!s->nb_snapshots) {
    +        s->snapshots = NULL;
    +        s->snapshots_size = 0;
    +        return 0;
    +    }
    +
         offset = s->snapshots_offset;
         s->snapshots = qemu_mallocz(s->nb_snapshots * sizeof(QCowSnapshot));
         if (!s->snapshots)
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

Can't see what this hunk accomplishes. If we remove it, the loop rejects, and we thus execute:

Once again, on Linux/GLIBC it will, on AIX it wont.

And FWIW despite behaviour of malloc(0) being marked as implementation defined i have sa far was unable to find any documentaiton (Linux man pages, GLIBC info files) witht the actual definition, unlike on AIX where man pages make it crystal clear what happens.

Quoted 13 lines
Quoted 13 lines
Quoted 4 lines
    offset = s->snapshots_offset;
    s->snapshots = qemu_mallocz(s->nb_snapshots * sizeof(QCowSnapshot));
    s->snapshots_size = offset - s->snapshots_offset;
    return 0;

Uh, no.

$ git cat-file -p 63c75dcd669d011f438421980b4379827da4bb1c^:block-qcow2.c | sed -n 1811,1815p
    offset = s->snapshots_offset;
    s->snapshots = qemu_mallocz(s->nb_snapshots * sizeof(QCowSnapshot));
    if (!s->snapshots)
        goto fail;
Quoted 7 lines
Quoted 4 lines

So before the patch qemu_mallocz(0) call would return NULL, goto fail would be executed and `qcow_read_snapshots' will return -1 and QEMU will fail to start with very helpful message:
qemu: could not open disk image <image name>

The test went away in commit 3ec88e80. The hunk became pointless only then.

Again, it's pointless only with your proposed addition, otherwise instead of 'could not open disk image' one would get an out of memory error.

[..snip..]

Quoted 30 lines
Quoted 11 lines

Also please notice how throwing second implemenation-defined behaviour of malloc in to the mix led to the hunk #2, in which you see no point.

Quoted 5 lines
Quoted 3 lines

existing code used to, i'd venture a guess accidentaly, rely on the behaviour that current GLIBC provides and consequently failed to operate on AIX where malloc(0) returns NULL,

Common error.

Made by a very skilled programmer i might add.

I've fallen into this trap myself.

Quoted 16 lines
Quoted 15 lines
Quoted 3 lines

IOW making qemu_malloc[z]
return whatever the underlying system returns is just hiding the bugs, the code becomes unportable.

Matter of taste.

1. Deal with the implementation-definedness. Every caller that could pass zero needs to take care not to confuse empty allocation with an out of memory condition.

This is easier than it sounds when you check for out of memory in just one place, like we do.

2. Remove the implementation-definedness. Easiest way is to detect zero size in a wrapper (for us: qemu_malloc()) and bump it to one.

Quoted 7 lines
Quoted 3 lines

And mine:
3. Abort the program if somebody tries it. Because so far history thought me that nobody does 1.

Tries what? Passing zero to qemu_malloc()? That's legitimate. And with allocation functions that cannot return failure, it's hardly dangerous, isn't it?

That's legitimate only if one writes unportable code targeting single system and knowing how it was defined. As for being dangerous, yes it
is: dereferencing the returned pointer, while UB, doesn't trigger a SEGFAULT on, at least, this machine with Linux.

Quoted 1 line
Quoted 1 line
Quoted 1 line

qemu_realloc() currently uses 1.

void *qemu_realloc(void *ptr, size_t size)

{
    if (size)
        return oom_check(realloc(ptr, size));
    else
        return realloc(ptr, size);
}

There is nothing implementation defined about realloc(whatever, 0), it has a defined meaning in POSIX:
http://opengroup.org/onlinepubs/007908775/xsh/realloc.html

So it doesn't use 1.

Quoted 8 lines
Quoted 7 lines
Quoted 4 lines

realloc(NULL, sz) is specified to be equivalent to malloc(sz). It would be kind of nice to keep that for qemu_realloc() and qemu_malloc().

qemu_realloc shouldn't be called qemu_realloc if doesn't do that. The part about qemu_malloc escapes me.

Quoted 7 lines

qemu_malloc() & friends never fail. Checking their value for failure is pointless. Therefore, 1. is practical.

2. is certainly practical as well.

3. is like 2, with the (size ? size : 1) pushed into callers. I find that mildly annoying.

Huh, that's not at all what i proposed. What i had in mind is:

void *qemu_malloc(size_t size)

{
    if (!size) abort();
    return oom_check(malloc(size));
}
Quoted 5 lines

I don't care whether we pick 1., 2. or 3., but I'd prefer we pick the same for qemu_malloc() and qemu_realloc(). We currently use 1. for qemu_realloc(). My point is: if you pick something else for qemu_malloc(), please consider changing qemu_realloc(). That's all.

I see.

Signature · 1 line
mailto:[email protected]
#8 lore raw
Quoted 25 lines · malc
Quoted 2 lines · Markus Armbruster

malc <[email protected]> writes:

[..snip..]

Quoted 18 lines
Quoted 18 lines
Quoted 18 lines
    diff --git a/block-qcow2.c b/block-qcow2.c
    index 9aa7261..d4556ef 100644
    --- a/block-qcow2.c
    +++ b/block-qcow2.c
    @@ -1809,6 +1809,12 @@ static int qcow_read_snapshots(BlockDriverState *bs)
         int64_t offset;
         uint32_t extra_data_size;

    +    if (!s->nb_snapshots) {
    +        s->snapshots = NULL;
    +        s->snapshots_size = 0;
    +        return 0;
    +    }
    +
         offset = s->snapshots_offset;
         s->snapshots = qemu_mallocz(s->nb_snapshots * sizeof(QCowSnapshot));
         if (!s->snapshots)
Quoted 5 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

Can't see what this hunk accomplishes. If we remove it, the loop rejects, and we thus execute:

Once again, on Linux/GLIBC it will, on AIX it wont.

Why not? It will. If nb_snapshots is 0, it won't enter the loop. The problem with that code was the "if (!s->snapshots)" check, not the qemu_mallocz(0) call.

Quoted 5 lines

And FWIW despite behaviour of malloc(0) being marked as implementation defined i have sa far was unable to find any documentaiton (Linux man pages, GLIBC info files) witht the actual definition, unlike on AIX where man pages make it crystal clear what happens.

You don't need to have the exact behavior defined, as long as:

1) You call free(p) later
2) You don't dereference the returned pointer (just like you can't dereference p[n] on a malloc(n) block)
3) You don't assume anything about the returned value when size==0

My point is that this is valid malloc() usage, and there may be existing qemu code relying on that, and I don't see any reason to put a trap for code that would be valid malloc()/free() usage.

Quoted 2 lines

<snip>

Quoted 7 lines
Quoted 4 lines

Tries what? Passing zero to qemu_malloc()? That's legitimate. And with allocation functions that cannot return failure, it's hardly dangerous, isn't it?

That's legitimate only if one writes unportable code targeting single system and knowing how it was defined.

No, that's legitimate and portable. You just can't assume anything about the returned value.

Quoted 14 lines

As for being dangerous, yes it
is: dereferencing the returned pointer, while UB, doesn't trigger a SEGFAULT on, at least, this machine with Linux.

Quoted 1 line
Quoted 1 line
Quoted 1 line

qemu_realloc() currently uses 1.

void *qemu_realloc(void *ptr, size_t size)

{
    if (size)
        return oom_check(realloc(ptr, size));
    else
        return realloc(ptr, size);
}
Quoted 6 lines

There is nothing implementation defined about realloc(whatever, 0), it has a defined meaning in POSIX:
http://opengroup.org/onlinepubs/007908775/xsh/realloc.html

So it doesn't use 1.

realloc() return value is specified exactly the same way malloc() is:

"If size is 0, either a null pointer or a unique pointer that can be successfully passed to free() is returned."

Quoted 16 lines
Quoted 8 lines
Quoted 7 lines
Quoted 4 lines

realloc(NULL, sz) is specified to be equivalent to malloc(sz). It would be kind of nice to keep that for qemu_realloc() and qemu_malloc().

qemu_realloc shouldn't be called qemu_realloc if doesn't do that. The part about qemu_malloc escapes me.

Quoted 7 lines

qemu_malloc() & friends never fail. Checking their value for failure is pointless. Therefore, 1. is practical.

2. is certainly practical as well.

3. is like 2, with the (size ? size : 1) pushed into callers. I find that mildly annoying.

Quoted 7 lines

Huh, that's not at all what i proposed. What i had in mind is:

void *qemu_malloc(size_t size)

{
    if (!size) abort();
    return oom_check(malloc(size));
}

Understood. And that's exactly what I think we should not do.

Signature · 1 line
Eduardo
#9 lore raw
Quoted 30 lines · Eduardo Habkost
Quoted 29 lines · malc
Quoted 2 lines · Markus Armbruster

malc <[email protected]> writes:

[..snip..]

Quoted 21 lines
Quoted 21 lines
Quoted 21 lines
    diff --git a/block-qcow2.c b/block-qcow2.c
    index 9aa7261..d4556ef 100644
    --- a/block-qcow2.c
    +++ b/block-qcow2.c
    @@ -1809,6 +1809,12 @@ static int qcow_read_snapshots(BlockDriverState *bs)
         int64_t offset;
         uint32_t extra_data_size;

    +    if (!s->nb_snapshots) {
    +        s->snapshots = NULL;
    +        s->snapshots_size = 0;
    +        return 0;
    +    }
    +
         offset = s->snapshots_offset;
         s->snapshots = qemu_mallocz(s->nb_snapshots * sizeof(QCowSnapshot));
         if (!s->snapshots)

Can't see what this hunk accomplishes. If we remove it, the loop rejects, and we thus execute:

Quoted 5 lines
Quoted 1 line

Once again, on Linux/GLIBC it will, on AIX it wont.

Why not? It will. If nb_snapshots is 0, it won't enter the loop. The problem with that code was the "if (!s->snapshots)" check, not the qemu_mallocz(0) call.

Because qemu_mallocz on AIX will be terminated by oom_check.

Quoted 9 lines
Quoted 7 lines

And FWIW despite behaviour of malloc(0) being marked as implementation defined i have sa far was unable to find any documentaiton (Linux man pages, GLIBC info files) witht the actual definition, unlike on AIX where man pages make it crystal clear what happens.

You don't need to have the exact behavior defined, as long as:

I certainly don't, the standard certainly says the implementation is obliged to document it, that's what seprates implementation-defined from unspecified behaviour.

Quoted 8 lines

1) You call free(p) later
2) You don't dereference the returned pointer (just like you can't dereference p[n] on a malloc(n) block)
3) You don't assume anything about the returned value when size==0

My point is that this is valid malloc() usage, and there may be existing qemu code relying on that, and I don't see any reason to put a trap for code that would be valid malloc()/free() usage.

Okay, at this point we both expressed our points of view and have to agree to disagree.

Quoted 35 lines
Quoted 3 lines

<snip>

Quoted 7 lines
Quoted 4 lines

Tries what? Passing zero to qemu_malloc()? That's legitimate. And with allocation functions that cannot return failure, it's hardly dangerous, isn't it?

That's legitimate only if one writes unportable code targeting single system and knowing how it was defined.

No, that's legitimate and portable. You just can't assume anything about the returned value.

Quoted 18 lines

As for being dangerous, yes it
is: dereferencing the returned pointer, while UB, doesn't trigger a SEGFAULT on, at least, this machine with Linux.

Quoted 1 line
Quoted 1 line
Quoted 1 line

qemu_realloc() currently uses 1.

void *qemu_realloc(void *ptr, size_t size)

{
    if (size)
        return oom_check(realloc(ptr, size));
    else
        return realloc(ptr, size);
}

There is nothing implementation defined about realloc(whatever, 0), it has a defined meaning in POSIX:
http://opengroup.org/onlinepubs/007908775/xsh/realloc.html

Quoted 7 lines
Quoted 2 lines

So it doesn't use 1.

realloc() return value is specified exactly the same way malloc() is:

"If size is 0, either a null pointer or a unique pointer that can be successfully passed to free() is returned."

Nope, quoting from above page:

If size is 0 and ptr is not a null pointer, the object pointed to is freed.

Quoted 29 lines
Quoted 18 lines
Quoted 10 lines
Quoted 9 lines
Quoted 6 lines

realloc(NULL, sz) is specified to be equivalent to malloc(sz). It would be kind of nice to keep that for qemu_realloc() and qemu_malloc().

qemu_realloc shouldn't be called qemu_realloc if doesn't do that. The part about qemu_malloc escapes me.

Quoted 7 lines

qemu_malloc() & friends never fail. Checking their value for failure is pointless. Therefore, 1. is practical.

2. is certainly practical as well.

3. is like 2, with the (size ? size : 1) pushed into callers. I find that mildly annoying.

Quoted 7 lines

Huh, that's not at all what i proposed. What i had in mind is:

void *qemu_malloc(size_t size)

{
    if (!size) abort();
    return oom_check(malloc(size));
}

Understood. And that's exactly what I think we should not do.

Signature · 1 line
mailto:[email protected]
#10 lore raw

On Tue, May 19, 2009 at 06:48:25PM +0400, malc wrote:
<snip>

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

Can't see what this hunk accomplishes. If we remove it, the loop rejects, and we thus execute:

Quoted 7 lines
Quoted 5 lines
Quoted 1 line

Once again, on Linux/GLIBC it will, on AIX it wont.

Why not? It will. If nb_snapshots is 0, it won't enter the loop. The problem with that code was the "if (!s->snapshots)" check, not the qemu_mallocz(0) call.

Because qemu_mallocz on AIX will be terminated by oom_check.

That's exactly what the patch prevents from happening.

Quoted 18 lines
Quoted 8 lines
Quoted 6 lines

And FWIW despite behaviour of malloc(0) being marked as implementation defined i have sa far was unable to find any documentaiton (Linux man pages, GLIBC info files) witht the actual definition, unlike on AIX where man pages make it crystal clear what happens.

You don't need to have the exact behavior defined, as long as:

I certainly don't, the standard certainly says the implementation is obliged to document it, that's what seprates implementation-defined from unspecified behaviour.

Quoted 5 lines

1) You call free(p) later
2) You don't dereference the returned pointer (just like you can't dereference p[n] on a malloc(n) block)
3) You don't assume anything about the returned value when size==0

Quoted 6 lines
Quoted 3 lines

My point is that this is valid malloc() usage, and there may be existing qemu code relying on that, and I don't see any reason to put a trap for code that would be valid malloc()/free() usage.

Okay, at this point we both expressed our points of view and have to agree to disagree.

Agreed.

<snip>

Quoted 10 lines
Quoted 10 lines
Quoted 7 lines

There is nothing implementation defined about realloc(whatever, 0), it has a defined meaning in POSIX:
http://opengroup.org/onlinepubs/007908775/xsh/realloc.html

So it doesn't use 1.

realloc() return value is specified exactly the same way malloc() is:

Quoted 7 lines
Quoted 2 lines

"If size is 0, either a null pointer or a unique pointer that can be successfully passed to free() is returned."

Nope, quoting from above page:

If size is 0 and ptr is not a null pointer, the object pointed to is freed.

I quoted the above from exactly the same page.

I really hope you are not proposing to make qemu_realloc(p, 0) work but qemu_malloc(0) fail, because you would be breaking lots of realloc()/malloc() equivalency assumptions.

Signature · 1 line
Eduardo
#11 lore raw
Quoted 9 lines · Eduardo Habkost

On Tue, May 19, 2009 at 06:48:25PM +0400, malc wrote:
<snip>

Quoted 7 lines
Quoted 7 lines
Quoted 6 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

Can't see what this hunk accomplishes. If we remove it, the loop rejects, and we thus execute:

Once again, on Linux/GLIBC it will, on AIX it wont.

Quoted 7 lines
Quoted 5 lines
Quoted 3 lines

Why not? It will. If nb_snapshots is 0, it won't enter the loop. The problem with that code was the "if (!s->snapshots)" check, not the qemu_mallocz(0) call.

Because qemu_mallocz on AIX will be terminated by oom_check.

That's exactly what the patch prevents from happening.

And i said as much:

<quote>
Again, it's pointless only with your proposed addition, otherwise instead of 'could not open disk image' one would get an out of memory error.
</quote>

Quoted 46 lines
Quoted 20 lines
Quoted 10 lines
Quoted 8 lines

And FWIW despite behaviour of malloc(0) being marked as implementation defined i have sa far was unable to find any documentaiton (Linux man pages, GLIBC info files) witht the actual definition, unlike on AIX where man pages make it crystal clear what happens.

You don't need to have the exact behavior defined, as long as:

I certainly don't, the standard certainly says the implementation is obliged to document it, that's what seprates implementation-defined from unspecified behaviour.

Quoted 5 lines

1) You call free(p) later
2) You don't dereference the returned pointer (just like you can't dereference p[n] on a malloc(n) block)
3) You don't assume anything about the returned value when size==0

Quoted 6 lines
Quoted 3 lines

My point is that this is valid malloc() usage, and there may be existing qemu code relying on that, and I don't see any reason to put a trap for code that would be valid malloc()/free() usage.

Okay, at this point we both expressed our points of view and have to agree to disagree.

Agreed.

<snip>

Quoted 15 lines
Quoted 5 lines
Quoted 5 lines

There is nothing implementation defined about realloc(whatever, 0), it has a defined meaning in POSIX:
http://opengroup.org/onlinepubs/007908775/xsh/realloc.html

Quoted 7 lines
Quoted 2 lines

So it doesn't use 1.

realloc() return value is specified exactly the same way malloc() is:

"If size is 0, either a null pointer or a unique pointer that can be successfully passed to free() is returned."

Nope, quoting from above page:

Quoted 8 lines
Quoted 2 lines

If size is 0 and ptr is not a null pointer, the object pointed to is freed.

I quoted the above from exactly the same page.

I really hope you are not proposing to make qemu_realloc(p, 0) work but qemu_malloc(0) fail, because you would be breaking lots of realloc()/malloc() equivalency assumptions.

That's exactly what i'm proposing.

Signature · 1 line
mailto:[email protected]
#12 lore raw
Quoted 19 lines · malc
Quoted 9 lines · Eduardo Habkost

On Tue, May 19, 2009 at 06:48:25PM +0400, malc wrote:
<snip>

Quoted 7 lines
Quoted 7 lines
Quoted 6 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

Can't see what this hunk accomplishes. If we remove it, the loop rejects, and we thus execute:

Once again, on Linux/GLIBC it will, on AIX it wont.

Quoted 7 lines
Quoted 5 lines
Quoted 3 lines

Why not? It will. If nb_snapshots is 0, it won't enter the loop. The problem with that code was the "if (!s->snapshots)" check, not the qemu_mallocz(0) call.

Because qemu_mallocz on AIX will be terminated by oom_check.

That's exactly what the patch prevents from happening.

Quoted 7 lines

And i said as much:

<quote>
Again, it's pointless only with your proposed addition, otherwise instead of 'could not open disk image' one would get an out of memory error.
</quote>

We are already running in circles. I will stop arguing about that until somebody else chimes in.

Quoted 1 line

<snip>

Quoted 11 lines
Quoted 11 lines
Quoted 10 lines
Quoted 5 lines

realloc() return value is specified exactly the same way malloc() is:

"If size is 0, either a null pointer or a unique pointer that can be successfully passed to free() is returned."

Nope, quoting from above page:

If size is 0 and ptr is not a null pointer, the object pointed to is freed.

Quoted 7 lines
Quoted 5 lines

I quoted the above from exactly the same page.

I really hope you are not proposing to make qemu_realloc(p, 0) work but qemu_malloc(0) fail, because you would be breaking lots of realloc()/malloc() equivalency assumptions.

That's exactly what i'm proposing.

Now, _that_ sounds like a really bad idea. realloc(NULL, n) is specified to be equivalent to malloc(n).

I can't prevent you from inventing a new malloc/free API that works differently from every malloc/free implementation out there. All I can say is that this sounds like a really bad idea.

Signature · 1 line
Eduardo
#13 lore raw
Quoted 2 lines · Eduardo Habkost

Now, _that_ sounds like a really bad idea. realloc(NULL, n) is specified to be equivalent to malloc(n).

No it isn't. You can't make that substitution.

In the case where n == 0, realloc(NULL, n) is guaranteed to not allocate anything and return NULL, whereas malloc(n) does not guarantee that and in fact doesn't do that on a lot of implementations.

-- Jamie

#14 lore raw

(Replying only now because I wasn't CCed on the reply)

Excerpts from Jamie Lokier's message of Tue May 19 17:32:23 -0300 2009:

Quoted 9 lines
Quoted 2 lines · Eduardo Habkost

Now, _that_ sounds like a really bad idea. realloc(NULL, n) is specified to be equivalent to malloc(n).

No it isn't. You can't make that substitution.

In the case where n == 0, realloc(NULL, n) is guaranteed to not allocate anything and return NULL, whereas malloc(n) does not guarantee that and in fact doesn't do that on a lot of implementations.

http://opengroup.org/onlinepubs/007908775/xsh/realloc.html

"If ptr is a null pointer, realloc() behaves like malloc() for the specified size."

"If size is 0, either a null pointer or a unique pointer that can be successfully passed to free() is returned."

Signature · 1 line
Eduardo
#15 lore raw
Quoted 11 lines · Eduardo Habkost

Excerpts from Jamie Lokier's message of Tue May 19 17:32:23 -0300 2009:

Quoted 9 lines
Quoted 2 lines · Eduardo Habkost

Now, _that_ sounds like a really bad idea. realloc(NULL, n) is specified to be equivalent to malloc(n).

No it isn't. You can't make that substitution.

In the case where n == 0, realloc(NULL, n) is guaranteed to not allocate anything and return NULL, whereas malloc(n) does not guarantee that and in fact doesn't do that on a lot of implementations.

Quoted 7 lines

http://opengroup.org/onlinepubs/007908775/xsh/realloc.html

"If ptr is a null pointer, realloc() behaves like malloc() for the specified size."

"If size is 0, either a null pointer or a unique pointer that can be successfully passed to free() is returned."

Oh. Thanks! I stand corrected; sorry for propagating misinformation.

The relevant part in the standard which makes the above not contradict realloc's freeing behaviour, is "if size is 0 and __ptr is not a null pointer__, the object pointed to is freed."

All this creates a different problem, unfortunately:

If you do:

p = malloc(n) /* Arbitrary n, could be zero. */
free(p)

it works, but

p = realloc(oldp, n) /* Arbitrary n, could be zero. */ realloc(p, 0)

is not guaranteed to free the allocated block, if n == 0 && p == NULL.

realloc(p, 0) is not equivalent to free(p) in this problem cases.

Which is IMHO another reason to either forbid n == 0, or warn about it, or change it unambiguously to n == 1 in the qemu_ wrappers.

While we're here, the C language FAQ adds:

Q: Is it legal to pass a null pointer as the first argument to
realloc? Why would you want to?

A: ANSI C sanctions this usage (and the related realloc(..., 0), which
frees), although several earlier implementations do not support it, so it may not be fully portable.

-- Jamie

#16 lore raw
Quoted 14 lines · Jamie Lokier
Quoted 13 lines · Eduardo Habkost

Excerpts from Jamie Lokier's message of Tue May 19 17:32:23 -0300 2009:

Quoted 9 lines
Quoted 2 lines · Eduardo Habkost

Now, _that_ sounds like a really bad idea. realloc(NULL, n) is specified to be equivalent to malloc(n).

No it isn't. You can't make that substitution.

In the case where n == 0, realloc(NULL, n) is guaranteed to not allocate anything and return NULL, whereas malloc(n) does not guarantee that and in fact doesn't do that on a lot of implementations.

http://opengroup.org/onlinepubs/007908775/xsh/realloc.html

Quoted 7 lines
Quoted 5 lines

"If ptr is a null pointer, realloc() behaves like malloc() for the specified size."

"If size is 0, either a null pointer or a unique pointer that can be successfully passed to free() is returned."

Oh. Thanks! I stand corrected; sorry for propagating misinformation.

:-)

Quoted 12 lines

The relevant part in the standard which makes the above not contradict realloc's freeing behaviour, is "if size is 0 and __ptr is not a null pointer__, the object pointed to is freed."

All this creates a different problem, unfortunately:

If you do:

p = malloc(n) /* Arbitrary n, could be zero. */
free(p)

Quoted 6 lines

it works, but

p = realloc(oldp, n) /* Arbitrary n, could be zero. */ realloc(p, 0)

is not guaranteed to free the allocated block, if n == 0 && p == NULL.

It is a problem only if you drop the return value of the second realloc() call. You shouldn't do that.

Quoted 2 lines

realloc(p, 0) is not equivalent to free(p) in this problem cases.

Yes. realloc(p, 0) frees p, but it is not exactly equivalent to free(p) because it does more than freeing p: it returns a new value you need to free later. You still need to save the returned value and eventually pass it to free() (or maybe another realloc() call, whose return value will need to be saved, and so on).

Quoted 3 lines

Which is IMHO another reason to either forbid n == 0, or warn about it, or change it unambiguously to n == 1 in the qemu_ wrappers.

The example you have shown ignores the return value of realloc() and that's the cause of the problem, not the possiblity of n==0.

That's the beauty of it: n==0 and realloc(NULL, n) don't need to be treated like a special case by the caller, it Just Works. The only thing you need to be careful about is not checking for NULL if you call realloc(p, 0) or malloc(0). And, obviously, it has the same requirements that apply for any other allocations, such as freeing it eventually, and not making any out-of-bounds dereference of the returned pointer.

Quoted 9 lines

While we're here, the C language FAQ adds:

Q: Is it legal to pass a null pointer as the first argument to
realloc? Why would you want to?

A: ANSI C sanctions this usage (and the related realloc(..., 0), which
frees), although several earlier implementations do not support it, so it may not be fully portable.

Old implementations, I guess. Is there any implementation we care about that does that?

Signature · 1 line
Eduardo
#17 lore raw
In reply to Eduardo Habkost
Quoted 3 lines · Eduardo Habkost

I really hope you are not proposing to make qemu_realloc(p, 0) work but qemu_malloc(0) fail, because you would be breaking lots of realloc()/malloc() equivalency assumptions.

Careful.

realloc(p, 0) *always* frees p (or does nothing if p == NULL), on all systems complying with ISO C, and always returns NULL.

malloc(0) on the other hand does not guarantee this, so malloc(0) is *not* a substitute for realloc(NULL, 0).

-- Jamie

#18 lore raw
In reply to malc
Quoted 3 lines · malc
Quoted 1 line · Markus Armbruster

malc <[email protected]> writes:

[...]

Quoted 6 lines

Once again, on Linux/GLIBC it will, on AIX it wont.

And FWIW despite behaviour of malloc(0) being marked as implementation defined i have sa far was unable to find any documentaiton (Linux man pages, GLIBC info files) witht the actual definition, unlike on AIX where man pages make it crystal clear what happens.

malloc(3) says:

If size is 0, then malloc() returns either NULL, or a unique pointer value that can later be successfully passed to free().

Matches ISO C. It's all you need to know to write portable programs.

[...]

Quoted 20 lines
Quoted 20 lines
Quoted 19 lines
Quoted 10 lines
Quoted 3 lines

IOW making qemu_malloc[z]
return whatever the underlying system returns is just hiding the bugs, the code becomes unportable.

Matter of taste.

1. Deal with the implementation-definedness. Every caller that could pass zero needs to take care not to confuse empty allocation with an out of memory condition.

Quoted 5 lines

This is easier than it sounds when you check for out of memory in just one place, like we do.

2. Remove the implementation-definedness. Easiest way is to detect zero size in a wrapper (for us: qemu_malloc()) and bump it to one.

And mine:
3. Abort the program if somebody tries it. Because so far history thought me that nobody does 1.

Quoted 8 lines
Quoted 3 lines

Tries what? Passing zero to qemu_malloc()? That's legitimate. And with allocation functions that cannot return failure, it's hardly dangerous, isn't it?

That's legitimate only if one writes unportable code targeting single system and knowing how it was defined. As for being dangerous, yes it
is: dereferencing the returned pointer, while UB, doesn't trigger a SEGFAULT on, at least, this machine with Linux.

malloc(0) is perfectly portable. The fact that dereferencing the return value may or may not fault doesn't change that. What happens on dereferencing of an invalid pointer is never portable.

Quoted 10 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line

qemu_realloc() currently uses 1.

void *qemu_realloc(void *ptr, size_t size)

{
    if (size)
        return oom_check(realloc(ptr, size));
    else
        return realloc(ptr, size);
}
Quoted 5 lines

There is nothing implementation defined about realloc(whatever, 0), it has a defined meaning in POSIX:
http://opengroup.org/onlinepubs/007908775/xsh/realloc.html

So it doesn't use 1.

Quoting from there:

If ptr is a null pointer, realloc() behaves like malloc() for the specified size.

So, realloc(whatever, 0) uses 1. when malloc(0) does.

Quoted 15 lines
Quoted 7 lines
Quoted 6 lines
Quoted 3 lines

realloc(NULL, sz) is specified to be equivalent to malloc(sz). It would be kind of nice to keep that for qemu_realloc() and qemu_malloc().

qemu_realloc shouldn't be called qemu_realloc if doesn't do that. The part about qemu_malloc escapes me.

Quoted 7 lines

qemu_malloc() & friends never fail. Checking their value for failure is pointless. Therefore, 1. is practical.

2. is certainly practical as well.

3. is like 2, with the (size ? size : 1) pushed into callers. I find that mildly annoying.

Quoted 7 lines

Huh, that's not at all what i proposed. What i had in mind is:

void *qemu_malloc(size_t size)

{
    if (!size) abort();
    return oom_check(malloc(size));
}

Exactly. Callers who used the fact that malloc(size) is perfectly portable for size>=0 then have to use qemu_malloc(size ? size : 1).

Quoted 6 lines
Quoted 4 lines

I don't care whether we pick 1., 2. or 3., but I'd prefer we pick the same for qemu_malloc() and qemu_realloc(). We currently use 1. for qemu_realloc(). My point is: if you pick something else for qemu_malloc(), please consider changing qemu_realloc(). That's all.

I see.

Good.

#19 lore raw
In reply to malc
Quoted 1 line · malc

On Tue, 19 May 2009, Markus Armbruster wrote:

<snip>

Quoted 13 lines
Quoted 13 lines
Quoted 3 lines

IOW making qemu_malloc[z]
return whatever the underlying system returns is just hiding the bugs, the code becomes unportable.

Matter of taste.

1. Deal with the implementation-definedness. Every caller that could pass zero needs to take care not to confuse empty allocation with an out of memory condition.

This is easier than it sounds when you check for out of memory in just one place, like we do.

Quoted 6 lines
Quoted 2 lines

2. Remove the implementation-definedness. Easiest way is to detect zero size in a wrapper (for us: qemu_malloc()) and bump it to one.

And mine:
3. Abort the program if somebody tries it. Because so far history thought me that nobody does 1.

Are you sure about that? There may be cases where qemu_malloc(0) is called correctly, without the wrong assumptions about the returned value.

You are proposing to make the qemu_malloc() API behavior diverge from the standard C malloc() behavior and prevent usage that is valid for malloc()/free() usage. Do you volunteer to audit all Qemu code to make sure the new behavior is safe? ;)

Signature · 1 line
Eduardo
#20 lore raw
Quoted 23 lines · Eduardo Habkost
Quoted 1 line · malc

On Tue, 19 May 2009, Markus Armbruster wrote:

<snip>

Quoted 13 lines
Quoted 13 lines
Quoted 3 lines

IOW making qemu_malloc[z]
return whatever the underlying system returns is just hiding the bugs, the code becomes unportable.

Matter of taste.

1. Deal with the implementation-definedness. Every caller that could pass zero needs to take care not to confuse empty allocation with an out of memory condition.

This is easier than it sounds when you check for out of memory in just one place, like we do.

Quoted 6 lines
Quoted 2 lines

2. Remove the implementation-definedness. Easiest way is to detect zero size in a wrapper (for us: qemu_malloc()) and bump it to one.

And mine:
3. Abort the program if somebody tries it. Because so far history thought me that nobody does 1.

Quoted 8 lines

Are you sure about that? There may be cases where qemu_malloc(0) is called correctly, without the wrong assumptions about the returned value.

You are proposing to make the qemu_malloc() API behavior diverge from the standard C malloc() behavior and prevent usage that is valid for malloc()/free() usage. Do you volunteer to audit all Qemu code to make sure the new behavior is safe? ;)

That's the problem standard C does _not_ define the behaviour, and leaves that to implementation. As for audit, that's precisely what aborting on zero will try (and fail) to accomplish the offenders will be (given unlimited time) caught. oom_check was added despite the fact that there were places that correctly handled malloc's returning NULL. And i for one do not know if there are/were places that called qemu_malloc with zero and expected Linux behaviour.

Signature · 1 line
mailto:[email protected]
#21 lore raw
Quoted 29 lines · malc
Quoted 27 lines · Eduardo Habkost
Quoted 1 line · malc

On Tue, 19 May 2009, Markus Armbruster wrote:

<snip>

Quoted 13 lines
Quoted 13 lines
Quoted 3 lines

IOW making qemu_malloc[z]
return whatever the underlying system returns is just hiding the bugs, the code becomes unportable.

Matter of taste.

1. Deal with the implementation-definedness. Every caller that could pass zero needs to take care not to confuse empty allocation with an out of memory condition.

This is easier than it sounds when you check for out of memory in just one place, like we do.

Quoted 6 lines
Quoted 2 lines

2. Remove the implementation-definedness. Easiest way is to detect zero size in a wrapper (for us: qemu_malloc()) and bump it to one.

And mine:
3. Abort the program if somebody tries it. Because so far history thought me that nobody does 1.

Are you sure about that? There may be cases where qemu_malloc(0) is called correctly, without the wrong assumptions about the returned value.

Quoted 7 lines
Quoted 4 lines

You are proposing to make the qemu_malloc() API behavior diverge from the standard C malloc() behavior and prevent usage that is valid for malloc()/free() usage. Do you volunteer to audit all Qemu code to make sure the new behavior is safe? ;)

That's the problem standard C does _not_ define the behaviour, and leaves that to implementation.

The only thing it doesn't define is either the returned pointer is NULL or not, and that doesn't make malloc(0) automatically unportable, because all the rest is perfectly defined:

1) You can't dereference the pointer (just like you can't dereference p[n] on a malloc(n) block)
2) You should pass the returned pointer to free() later

Quoted 3 lines

As for audit, that's precisely what aborting on
zero will try (and fail) to accomplish the offenders will be (given unlimited time) caught.

My point is that malloc(0) is not a bug, and I don't see a reason to make it an offense and diverge from standard malloc() and free().

Quoted 4 lines

oom_check was added despite the fact that there
were places that correctly handled malloc's returning NULL. And i for one do not know if there are/were places that called qemu_malloc with zero and expected Linux behaviour.

I agree that expecting the Linux behaviour (non-NULL) is a bug. My point is that there is no reason to consider malloc(0) a bug.

Signature · 1 line
Eduardo
#22 lore raw
Quoted 38 lines · Eduardo Habkost
Quoted 29 lines · malc
Quoted 27 lines · Eduardo Habkost
Quoted 1 line · malc

On Tue, 19 May 2009, Markus Armbruster wrote:

<snip>

Quoted 13 lines
Quoted 13 lines
Quoted 3 lines

IOW making qemu_malloc[z]
return whatever the underlying system returns is just hiding the bugs, the code becomes unportable.

Matter of taste.

1. Deal with the implementation-definedness. Every caller that could pass zero needs to take care not to confuse empty allocation with an out of memory condition.

This is easier than it sounds when you check for out of memory in just one place, like we do.

Quoted 6 lines
Quoted 2 lines

2. Remove the implementation-definedness. Easiest way is to detect zero size in a wrapper (for us: qemu_malloc()) and bump it to one.

And mine:
3. Abort the program if somebody tries it. Because so far history thought me that nobody does 1.

Are you sure about that? There may be cases where qemu_malloc(0) is called correctly, without the wrong assumptions about the returned value.

Quoted 7 lines
Quoted 4 lines

You are proposing to make the qemu_malloc() API behavior diverge from the standard C malloc() behavior and prevent usage that is valid for malloc()/free() usage. Do you volunteer to audit all Qemu code to make sure the new behavior is safe? ;)

That's the problem standard C does _not_ define the behaviour, and leaves that to implementation.

Quoted 8 lines

The only thing it doesn't define is either the returned pointer is NULL or not, and that doesn't make malloc(0) automatically unportable, because all the rest is perfectly defined:

1) You can't dereference the pointer (just like you can't dereference p[n] on a malloc(n) block)
2) You should pass the returned pointer to free() later

Alas your list is not exhaustive:

3) Test the returned value against NULL

[Which is precisely what the qcow2 code did]

And

4) Accidentally derefence it

And here one would get away with it on certain subset of systems.

Quoted 8 lines
Quoted 4 lines

As for audit, that's precisely what aborting on
zero will try (and fail) to accomplish the offenders will be (given unlimited time) caught.

My point is that malloc(0) is not a bug, and I don't see a reason to make it an offense and diverge from standard malloc() and free().

Quoted 7 lines
Quoted 4 lines

oom_check was added despite the fact that there
were places that correctly handled malloc's returning NULL. And i for one do not know if there are/were places that called qemu_malloc with zero and expected Linux behaviour.

I agree that expecting the Linux behaviour (non-NULL) is a bug. My point is that there is no reason to consider malloc(0) a bug.

There is, due to the possibility of performing a 3) and a hard time catching that (unless someone solves halting problem or subset applicable to QEMU thereof)

Signature · 1 line
mailto:[email protected]
#23 lore raw

PATCH 1/1 Make qemu_alloc()/qemu_realloc() return NULL for size==0 (was Re: [Qemu-devel] [PATCH] fix qemu_malloc() error check for size==0)

On Tue, May 19, 2009 at 06:55:11PM +0400, malc wrote:
<snip>

Quoted 19 lines
Quoted 4 lines
Quoted 3 lines

That's the problem standard C does _not_ define the behaviour, and leaves that to implementation.

Quoted 8 lines

The only thing it doesn't define is either the returned pointer is NULL or not, and that doesn't make malloc(0) automatically unportable, because all the rest is perfectly defined:

1) You can't dereference the pointer (just like you can't dereference p[n] on a malloc(n) block)
2) You should pass the returned pointer to free() later

Alas your list is not exhaustive:

3) Test the returned value against NULL

[Which is precisely what the qcow2 code did]

[...]

Quoted 7 lines
Quoted 3 lines

I agree that expecting the Linux behaviour (non-NULL) is a bug. My point is that there is no reason to consider malloc(0) a bug.

There is, due to the possibility of performing a 3) and a hard time catching that (unless someone solves halting problem or subset applicable to QEMU thereof)

This is probably the only of your points which I agree with. What about the following, then?

That would catch the cases you are worried about, but won't break existing cases where malloc(0) is used correctly, and we won't be creating a new malloc/free API that is incompabible from every other malloc/free API out there.

-----------

Quoted 1 line

From c2e0dd58e3eb7e2fd749bcf4999f067b1ba19083 Mon Sep 17 00:00:00 2001

From: Eduardo Habkost <[email protected]>
Date: Tue, 19 May 2009 13:24:47 -0300
Subject: [PATCH] Make qemu_alloc()/qemu_realloc() return NULL for size==0

This is an alternative to the fix I've proposed at:

Message-Id: <[email protected]>
http://marc.info/?l=qemu-devel&m=124267871605176

Instead of returning whatever realloc(p, 0) or malloc(0) of the system we are running on returns, always return NULL on those cases.

This should make it easier to catch cases where code is incorrectly checking the qemu_malloc() or qemu_realloc() return value for NULL when when size==0.

---
 qemu-malloc.c |   13 +++++++++----
 1 files changed, 9 insertions(+), 4 deletions(-)
qemu-malloc.c +9 -4
index 6761857..5346ece 100644
@@ -43,15 +43,20 @@ void qemu_free(void *ptr)
 
 void *qemu_malloc(size_t size)
 {
+    if (!size)
+        return NULL;
+
     return oom_check(malloc(size));
 }
 
 void *qemu_realloc(void *ptr, size_t size)
 {
-    if (size)
-        return oom_check(realloc(ptr, size));
-    else
-        return realloc(ptr, size);
+    if (!size) {
+        free(ptr);
+        return NULL;
+    }
+
+    return oom_check(realloc(ptr, size));
 }
 
 void *qemu_mallocz(size_t size)
Signature · 5 lines
1.6.3.rc4.29.g8146


-- 
Eduardo
#24 lore raw

Re: [PATCH] Make qemu_alloc()/qemu_realloc() return NULL for size==0 (was Re: [Qemu-devel] [PATCH] fix qemu_malloc() error check for size==0)

Quoted 30 lines · Eduardo Habkost

On Tue, May 19, 2009 at 06:55:11PM +0400, malc wrote:
<snip>

Quoted 19 lines
Quoted 4 lines
Quoted 3 lines

That's the problem standard C does _not_ define the behaviour, and leaves that to implementation.

Quoted 8 lines

The only thing it doesn't define is either the returned pointer is NULL or not, and that doesn't make malloc(0) automatically unportable, because all the rest is perfectly defined:

1) You can't dereference the pointer (just like you can't dereference p[n] on a malloc(n) block)
2) You should pass the returned pointer to free() later

Alas your list is not exhaustive:

3) Test the returned value against NULL

[Which is precisely what the qcow2 code did]

[...]

Quoted 7 lines
Quoted 3 lines

I agree that expecting the Linux behaviour (non-NULL) is a bug. My point is that there is no reason to consider malloc(0) a bug.

There is, due to the possibility of performing a 3) and a hard time catching that (unless someone solves halting problem or subset applicable to QEMU thereof)

Quoted 7 lines

This is probably the only of your points which I agree with. What about the following, then?

That would catch the cases you are worried about, but won't break existing cases where malloc(0) is used correctly, and we won't be creating a new malloc/free API that is incompabible from every other malloc/free API out there.

Thanks for an attempt, but i don't like it either, since it sortof breaks the (unspoken?) qemu_malloc/realloc contract that those will never return NULL. I've commited the thing i had in mind.

[..snip..]

P.S. It probably says something about my experience in the field, but the
only time i've encountered real-life usage of malloc(0) is in that qcow2 case, the code was produced by a programmer who among (numerous) other things wrote full C compiler and won IOCCC twice, yet even his code failed to work.

Signature · 1 line
mailto:[email protected]
#25 lore raw

Re: [PATCH] Make qemu_alloc()/qemu_realloc() return NULL for size==0 (was Re: [Qemu-devel] [PATCH] fix qemu_malloc() error check for size==0)

Quoted 35 lines · malc
Quoted 33 lines · Eduardo Habkost

On Tue, May 19, 2009 at 06:55:11PM +0400, malc wrote:
<snip>

Quoted 19 lines
Quoted 4 lines
Quoted 3 lines

That's the problem standard C does _not_ define the behaviour, and leaves that to implementation.

Quoted 8 lines

The only thing it doesn't define is either the returned pointer is NULL or not, and that doesn't make malloc(0) automatically unportable, because all the rest is perfectly defined:

1) You can't dereference the pointer (just like you can't dereference p[n] on a malloc(n) block)
2) You should pass the returned pointer to free() later

Alas your list is not exhaustive:

3) Test the returned value against NULL

[Which is precisely what the qcow2 code did]

[...]

Quoted 7 lines
Quoted 3 lines

I agree that expecting the Linux behaviour (non-NULL) is a bug. My point is that there is no reason to consider malloc(0) a bug.

There is, due to the possibility of performing a 3) and a hard time catching that (unless someone solves halting problem or subset applicable to QEMU thereof)

This is probably the only of your points which I agree with. What about the following, then?

Quoted 8 lines
Quoted 4 lines

That would catch the cases you are worried about, but won't break existing cases where malloc(0) is used correctly, and we won't be creating a new malloc/free API that is incompabible from every other malloc/free API out there.

Thanks for an attempt, but i don't like it either, since it sortof breaks the (unspoken?) qemu_malloc/realloc contract that those will never return NULL. I've commited the thing i had in mind.

Asking for feedback before committing wouldn't hurt.

Now:

- Every caller that could pass 0 to qemu_malloc() have to make it an special case.
- Every caller that uses qemu_realloc() will have to add special cases, if size==0 is possible.
- We are not sure where those callers are.
- Qemu's API is incompatible with every other malloc/free API out there, but is not documented

Signature · 1 line
Eduardo
#26 lore raw

Re: [PATCH] Make qemu_alloc()/qemu_realloc() return NULL for size==0 (was Re: [Qemu-devel] [PATCH] fix qemu_malloc() error check for size==0)

In reply to malc
Quoted 3 lines · malc

Thanks for an attempt, but i don't like it either, since it sortof breaks the (unspoken?) qemu_malloc/realloc contract that those will never return NULL. I've commited the thing i had in mind.

realloc returns NULL if the size is zero, and frees the object if the pointer is non-NULL.

-- Jamie

#27 lore raw

Re: [PATCH] Make qemu_alloc()/qemu_realloc() return NULL for size==0 (was Re: [Qemu-devel] [PATCH] fix qemu_malloc() error check for size==0)

In reply to malc
Quoted 3 lines · malc

Thanks for an attempt, but i don't like it either, since it sortof breaks the (unspoken?) qemu_malloc/realloc contract that those will never return NULL. I've commited the thing i had in mind.

I don't know of such a contract. If you want to have it, you should add a comment which specifies what is guaranteed and what not. And best post the patch to the list first this time.

Btw, even with your patch realloc(ptr, 0) with ptr != NULL still can return NULL. And if you now go ahead and forbid size = 0 entirely, I'm almost sure we have a regression as I didn't do the original patch to allow this just for fun but because it's used...

Kevin

#28 lore raw

[Qemu-devel] Re: [PATCH] Make qemu_alloc()/qemu_realloc() return NULL for size==0

In reply to malc

malc <[email protected]> writes:

[Debate snipped...]

Quoted 1 line

never return NULL. I've commited the thing i had in mind.

Arguing this case has been a waste of time, for which I apologize. I'll try not to do that again.

Quoted 7 lines

[..snip..]

P.S. It probably says something about my experience in the field, but the
only time i've encountered real-life usage of malloc(0) is in that qcow2 case, the code was produced by a programmer who among (numerous) other things wrote full C compiler and won IOCCC twice, yet even his code failed to work.

Read more code then, and more varied code.

#29 lore raw

[Qemu-devel] Re: [PATCH] Make qemu_alloc()/qemu_realloc() return NULL for size==0

Quoted 10 lines · Markus Armbruster

malc <[email protected]> writes:

[Debate snipped...]

Quoted 1 line

never return NULL. I've commited the thing i had in mind.

Arguing this case has been a waste of time, for which I apologize. I'll try not to do that again.

Quoted 2 lines

[..snip..]

Quoted 7 lines
Quoted 5 lines

P.S. It probably says something about my experience in the field, but the
only time i've encountered real-life usage of malloc(0) is in that qcow2 case, the code was produced by a programmer who among (numerous) other things wrote full C compiler and won IOCCC twice, yet even his code failed to work.

Read more code then, and more varied code.

Any suggested reading?

Signature · 1 line
mailto:[email protected]
#30 lore raw
In reply to malc
Quoted 1 line · malc

3) Test the returned value against NULL

Indeed, that makes malloc(0) a bad idea, and why GNU/Linux sensibly makes it return a non-NULL pointer, to minimise unexpected behaviour of programs.

Quoted 3 lines

4) Accidentally derefence it

And here one would get away with it on certain subset of systems.

That's not in the list of reliable things, though.

You mustn't accidentally dereference p[n] after calling p = malloc(n) either.

-- Jamie

#31 lore raw
In reply to malc
Quoted 37 lines · malc
Quoted 26 lines · Eduardo Habkost
Quoted 25 lines · malc
Quoted 5 lines · Eduardo Habkost

This patch is similar to a previous qemu_realloc() fix (commit 322691a5c9f1c8531554148d47c078b5be590805), but for qemu_malloc().

malloc(0) may correctly return NULL if size==0. We don't want to abort qemu on this case.

Only it wouldn't (on Linux):

$ cat malloc.c
#include <stdlib.h>

int main (void)

{
    printf ("%p\n", malloc (0));
    return 0;
}
$ gcc malloc.c
$ ./a.out 
0x10011008

Standard (in 7.20.3) says that malloc's behaviour in case of size being zero is implementation defined.

Quoted 8 lines
Quoted 4 lines

Try `git show 63c75dcd669d011f438421980b4379827da4bb1c'.

The best(only?) thing to do is to check size passed to qemu_malloc[z] and abort the program if this situation is encountered.

Why? malloc(0) is as valid as realloc(p, 0). It will either return NULL or a pointer, and on any case the value can be safely passed to free() later.

Quoted 6 lines

I believe you haven't examined the commit that i referenced. Thing is existing code used to, i'd venture a guess accidentaly, rely on the behaviour that current GLIBC provides and consequently failed to operate on AIX where malloc(0) returns NULL, IOW making qemu_malloc[z] return whatever the underlying system returns is just hiding the bugs, the code becomes unportable.

The assumption that malloc(0) will return anything (either NULL or not-NULL) is not portable. That's exactly the point of my patch: not making any assumption about the returned value when size==0.

But calling malloc(0) is perfectly valid, as long as you call free() on the returned value later. I don't see any reason to make the qemu_malloc() behavior from the standard malloc() behavior. The sequence "p=malloc(0);free(p)" is valid and works. Why would we prevent "p=qemu_malloc(0);qemu_free(p)" from working?

Yes, we may have broken code that assumes that qemu_malloc(0) is not NULL, and _that_ code is broken and must be fixed. But why would we break the cases where qemu_malloc(0) is called and handled correctly?

Signature · 1 line
Eduardo
#32 lore raw
Quoted 45 lines · Eduardo Habkost
Quoted 37 lines · malc
Quoted 26 lines · Eduardo Habkost
Quoted 25 lines · malc
Quoted 5 lines · Eduardo Habkost

This patch is similar to a previous qemu_realloc() fix (commit 322691a5c9f1c8531554148d47c078b5be590805), but for qemu_malloc().

malloc(0) may correctly return NULL if size==0. We don't want to abort qemu on this case.

Only it wouldn't (on Linux):

$ cat malloc.c
#include <stdlib.h>

int main (void)

{
    printf ("%p\n", malloc (0));
    return 0;
}
$ gcc malloc.c
$ ./a.out 
0x10011008

Standard (in 7.20.3) says that malloc's behaviour in case of size being zero is implementation defined.

Quoted 8 lines
Quoted 4 lines

Try `git show 63c75dcd669d011f438421980b4379827da4bb1c'.

The best(only?) thing to do is to check size passed to qemu_malloc[z] and abort the program if this situation is encountered.

Why? malloc(0) is as valid as realloc(p, 0). It will either return NULL or a pointer, and on any case the value can be safely passed to free() later.

Quoted 6 lines

I believe you haven't examined the commit that i referenced. Thing is existing code used to, i'd venture a guess accidentaly, rely on the behaviour that current GLIBC provides and consequently failed to operate on AIX where malloc(0) returns NULL, IOW making qemu_malloc[z] return whatever the underlying system returns is just hiding the bugs, the code becomes unportable.

Quoted 3 lines

The assumption that malloc(0) will return anything (either NULL or not-NULL) is not portable. That's exactly the point of my patch: not making any assumption about the returned value when size==0.

Yes i got that, i just disagree with the outcome of the test.

Quoted 10 lines

But calling malloc(0) is perfectly valid, as long as you call free() on the returned value later. I don't see any reason to make the qemu_malloc() behavior from the standard malloc() behavior. The sequence "p=malloc(0);free(p)" is valid and works. Why would we prevent "p=qemu_malloc(0);qemu_free(p)" from working?

Yes, we may have broken code that assumes that qemu_malloc(0) is not NULL, and _that_ code is broken and must be fixed. But why would we break the cases where qemu_malloc(0) is called and handled correctly?

To reiterate, there was code that handled qemu_malloc(non_zero) failures gracefuly yet the oom_check was introduced. Different tradeoffs i guess.

Signature · 1 line
mailto:[email protected]