glibc detected *** groupadd: malloc(): memory corruption

DISCUSSION Robert Yang 3 participants 6 messages Jun 18, 2012 – Jun 23, 2012

Share & tools

Robert Yang

author

2012-06-18 09:28 UTC #0 lore raw

Hi Experts:

I've met a strange issue, when set the length of builddir to 266 (Why I did this is that I need to know how long the tmpdir that oe-core can support):

NOTE: package dbus-1.4.20-r3.0: task do_install: Started
*** glibc detected *** groupadd: malloc(): memory corruption: 0x000000000101da70 *** *** glibc detected *** groupadd: malloc(): memory corruption: 0x000000000101da70 ***

Then the build would hang, this is caused by the command:

PSEUDO_PREFIX=/too/long/path/tmp/sysroots/x86_64-linux/usr PSEUDO_LOCALSTATEDIR=... PSEUDO_PASSWD=... PSEUDO_NOSYMLINKEXP=1 PSEUDO_DISABLED=0 PSEUDO_LOCALSTATEDIR ... /too/long/path/to/tmp/sysroots/x86_64-linux/usr/bin/pseudo groupadd --root /too/long/path/to/tmp/sysroots/qemux86 -r netdev -f

1) I had looked into the code of groupadd, and found that this would happen when it used the glibc function which needs malloc(for example, the access()), so it seemed this was caused by the glibc or pseudo, but I didn't know why it only happened to groupadd/useradd.

2) I had tried not to use pseudo, it worked well:

sudo groupadd --root /too/long/path/to/tmp/sysroots/qemux86 -r netdev -f

From this, it seemed that the glibc was ok

3) I had tried to write a small piece of code which used the access(), and used pseudo to run it, it worked well:

/too/long/path/for/pseudo/settings/and/then/run/pseudo my_app

From this, it seemed that both pseudo and glibc were OK.

These 3 steps make me puzzle, maybe we can think that the tmpdir can not be too long, and limit it to a proper length, please see this:

http://lists.linuxtogo.org/pipermail/openembedded-core/2012-May/022112.html

BTW. the "argument list too long" error has been fixed.

Any suggestion is appreciated.

Signature · 3 lines

Thanks

Robert

Mark Hatle

2012-06-18 17:58 UTC #1 lore raw

We're not away of any size limitations within pseudo, other then PATH_MAX which is typically 4096...

--Mark

Quoted 46 lines · Robert Yang

Hi Experts:
I've met a strange issue, when set the length of builddir to 266 (Why I did this is that I need to know how long the tmpdir that oe-core can support):
NOTE: package dbus-1.4.20-r3.0: task do_install: Started
*** glibc detected *** groupadd: malloc(): memory corruption: 0x000000000101da70 *** *** glibc detected *** groupadd: malloc(): memory corruption: 0x000000000101da70 ***
Then the build would hang, this is caused by the command:
PSEUDO_PREFIX=/too/long/path/tmp/sysroots/x86_64-linux/usr PSEUDO_LOCALSTATEDIR=... PSEUDO_PASSWD=... PSEUDO_NOSYMLINKEXP=1 PSEUDO_DISABLED=0 PSEUDO_LOCALSTATEDIR ... /too/long/path/to/tmp/sysroots/x86_64-linux/usr/bin/pseudo groupadd --root /too/long/path/to/tmp/sysroots/qemux86 -r netdev -f
1) I had looked into the code of groupadd, and found that this would happen when it used the glibc function which needs malloc(for example, the access()), so it seemed this was caused by the glibc or pseudo, but I didn't know why it only happened to groupadd/useradd.
2) I had tried not to use pseudo, it worked well:
sudo groupadd --root /too/long/path/to/tmp/sysroots/qemux86 -r netdev -f
From this, it seemed that the glibc was ok
3) I had tried to write a small piece of code which used the access(), and used pseudo to run it, it worked well:
/too/long/path/for/pseudo/settings/and/then/run/pseudo my_app
From this, it seemed that both pseudo and glibc were OK.
These 3 steps make me puzzle, maybe we can think that the tmpdir can not be too long, and limit it to a proper length, please see this:
http://lists.linuxtogo.org/pipermail/openembedded-core/2012-May/022112.html
BTW. the "argument list too long" error has been fixed.
Any suggestion is appreciated.

Robert Yang

author

2012-06-20 06:18 UTC #2 lore raw

Thanks Mark, the rpm.real also has the same problem when the length of tmpdir is 210, it seems that this is because of the glibc.

And also other problems found:
When use PACKAGE_CLASSES = "package_deb", and the length of tmpdir is 177, the error is:

/too/long/path/totmp/sysroots/x86_64-linux/usr/bin/dpkg-scanpackages: bad interpreter: No such file or directory

The interpreter is perl, and it does exist, it seems that we should limit the length of tmpdir to a smaller value then 177 rather than fix these strange problems. I will go on working on it.

// Robert

Quoted 62 lines · Mark Hatle

We're not away of any size limitations within pseudo, other then PATH_MAX which is typically 4096...
--Mark
Quoted 39 lines · Robert Yang
Hi Experts:
I've met a strange issue, when set the length of builddir to 266 (Why I did this is that I need to know how long the tmpdir that oe-core can support):
NOTE: package dbus-1.4.20-r3.0: task do_install: Started *** glibc detected *** groupadd: malloc(): memory corruption:
0x000000000101da70 ***
*** glibc detected *** groupadd: malloc(): memory corruption:
0x000000000101da70 ***
Then the build would hang, this is caused by the command:
PSEUDO_PREFIX=/too/long/path/tmp/sysroots/x86_64-linux/usr PSEUDO_LOCALSTATEDIR=... PSEUDO_PASSWD=... PSEUDO_NOSYMLINKEXP=1 PSEUDO_DISABLED=0 PSEUDO_LOCALSTATEDIR ... /too/long/path/to/tmp/sysroots/x86_64-linux/usr/bin/pseudo groupadd --root /too/long/path/to/tmp/sysroots/qemux86 -r netdev -f
1) I had looked into the code of groupadd, and found that this would happen when it used the glibc function which needs malloc(for example, the access()), so it seemed this was caused by the glibc or pseudo, but I didn't know why it only happened to groupadd/useradd.
2) I had tried not to use pseudo, it worked well:
sudo groupadd --root /too/long/path/to/tmp/sysroots/qemux86 -r netdev -f
From this, it seemed that the glibc was ok
3) I had tried to write a small piece of code which used the access(), and used pseudo to run it, it worked well:
/too/long/path/for/pseudo/settings/and/then/run/pseudo my_app
From this, it seemed that both pseudo and glibc were OK.
Quoted 9 lines
These 3 steps make me puzzle, maybe we can think that the tmpdir can not be too long, and limit it to a proper length, please see this:
http://lists.linuxtogo.org/pipermail/openembedded-core/2012-May/022112.html
BTW. the "argument list too long" error has been fixed.
Any suggestion is appreciated.
_______________________________________________
Openembedded-core mailing list
[email protected]
http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core

Saul Wold

2012-06-21 15:57 UTC #3 lore raw

Quoted 10 lines · Robert Yang

Thanks Mark, the rpm.real also has the same problem when the length of tmpdir
is 210, it seems that this is because of the glibc.
And also other problems found:
When use PACKAGE_CLASSES = "package_deb", and the length of tmpdir is 177, the error is:

Quoted 8 lines

/too/long/path/totmp/sysroots/x86_64-linux/usr/bin/dpkg-scanpackages:
bad interpreter: No such file or directory
The interpreter is perl, and it does exist, it seems that we should limit the length of tmpdir to a smaller value then 177 rather than fix these strange
problems. I will go on working on it.

Robert,

A path length of 175 or short is not ideal, we should file bugs against these issues. We should also add the sanity check you suggested as a WARNING for lenghts over 175. That should not be the final solution though, we need to address the issues and get them fixed up stream.

The groupadd one sounds like it might be a buffer overflow issue which could have larger ramifications

Thanks

Sau!

Quoted 75 lines

// Robert
Quoted 55 lines · Mark Hatle
We're not away of any size limitations within pseudo, other then PATH_MAX which
is typically 4096...
--Mark
Quoted 48 lines · Robert Yang
Hi Experts:
I've met a strange issue, when set the length of builddir to 266 (Why I did this is that I need to know how long the tmpdir that oe-core can support):
NOTE: package dbus-1.4.20-r3.0: task do_install: Started *** glibc detected *** groupadd: malloc(): memory corruption:
0x000000000101da70 ***
*** glibc detected *** groupadd: malloc(): memory corruption:
0x000000000101da70 ***
Then the build would hang, this is caused by the command:
PSEUDO_PREFIX=/too/long/path/tmp/sysroots/x86_64-linux/usr PSEUDO_LOCALSTATEDIR=... PSEUDO_PASSWD=... PSEUDO_NOSYMLINKEXP=1 PSEUDO_DISABLED=0 PSEUDO_LOCALSTATEDIR ... /too/long/path/to/tmp/sysroots/x86_64-linux/usr/bin/pseudo groupadd --root /too/long/path/to/tmp/sysroots/qemux86 -r netdev -f
1) I had looked into the code of groupadd, and found that this would happen when it used the glibc function which needs malloc(for example, the access()), so it seemed this was caused by the glibc or pseudo, but I didn't know why it only happened to groupadd/useradd.
2) I had tried not to use pseudo, it worked well:
sudo groupadd --root /too/long/path/to/tmp/sysroots/qemux86 -r netdev -f
From this, it seemed that the glibc was ok
3) I had tried to write a small piece of code which used the access(), and
used pseudo to run it, it worked well:
/too/long/path/for/pseudo/settings/and/then/run/pseudo my_app
From this, it seemed that both pseudo and glibc were OK.
These 3 steps make me puzzle, maybe we can think that the tmpdir can not be too long, and limit it to a proper length, please see this:
http://lists.linuxtogo.org/pipermail/openembedded-core/2012-May/022112.html
BTW. the "argument list too long" error has been fixed.
Quoted 10 lines
Quoted 2 lines
Any suggestion is appreciated.
_______________________________________________
Openembedded-core mailing list
[email protected]
http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
_______________________________________________
Openembedded-core mailing list
[email protected]
http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core

Mark Hatle

2012-06-21 16:08 UTC #4 lore raw

Quoted 7 lines · Saul Wold

Quoted 6 lines · Robert Yang
Thanks Mark, the rpm.real also has the same problem when the length of tmpdir
is 210, it seems that this is because of the glibc.

Quoted 6 lines

Quoted 6 lines
And also other problems found:
When use PACKAGE_CLASSES = "package_deb", and the length of tmpdir is 177, the error is:
/too/long/path/totmp/sysroots/x86_64-linux/usr/bin/dpkg-scanpackages:
bad interpreter: No such file or directory

There is a limit to the length of the interpreter line. The only solution I have ever found for this is to replace explicit interpreter paths with:

#! /usr/bin/env <interpreter>

This does require that the items be in the PATH.

We should scan scripts for that syntax and replace them w/ the /usr/bin/env syntax, assuming it will work as expected.

That may still leave issues with other components however.

(and I agree with Saul, 175 characters is unacceptable, it's -very- easy to hit that..)

--Mark

Quoted 100 lines

Quoted 5 lines
The interpreter is perl, and it does exist, it seems that we should limit the length of tmpdir to a smaller value then 177 rather than fix these strange
problems. I will go on working on it.
Robert,
A path length of 175 or short is not ideal, we should file bugs against these issues. We should also add the sanity check you suggested as a WARNING for lenghts over 175. That should not be the final solution though, we need to address the issues and get them fixed up stream.
The groupadd one sounds like it might be a buffer overflow issue which could have larger ramifications
Thanks
Sau!
Quoted 69 lines
// Robert
Quoted 55 lines · Mark Hatle
We're not away of any size limitations within pseudo, other then PATH_MAX which
is typically 4096...
--Mark
Quoted 48 lines · Robert Yang
Hi Experts:
I've met a strange issue, when set the length of builddir to 266 (Why I did this is that I need to know how long the tmpdir that oe-core can support):
NOTE: package dbus-1.4.20-r3.0: task do_install: Started *** glibc detected *** groupadd: malloc(): memory corruption:
0x000000000101da70 ***
*** glibc detected *** groupadd: malloc(): memory corruption:
0x000000000101da70 ***
Then the build would hang, this is caused by the command:
PSEUDO_PREFIX=/too/long/path/tmp/sysroots/x86_64-linux/usr PSEUDO_LOCALSTATEDIR=... PSEUDO_PASSWD=... PSEUDO_NOSYMLINKEXP=1 PSEUDO_DISABLED=0 PSEUDO_LOCALSTATEDIR ... /too/long/path/to/tmp/sysroots/x86_64-linux/usr/bin/pseudo groupadd --root /too/long/path/to/tmp/sysroots/qemux86 -r netdev -f
1) I had looked into the code of groupadd, and found that this would happen when it used the glibc function which needs malloc(for example, the access()), so it seemed this was caused by the glibc or pseudo, but I didn't know why it only happened to groupadd/useradd.
2) I had tried not to use pseudo, it worked well:
sudo groupadd --root /too/long/path/to/tmp/sysroots/qemux86 -r netdev -f
From this, it seemed that the glibc was ok
3) I had tried to write a small piece of code which used the access(), and
used pseudo to run it, it worked well:
/too/long/path/for/pseudo/settings/and/then/run/pseudo my_app
From this, it seemed that both pseudo and glibc were OK.
These 3 steps make me puzzle, maybe we can think that the tmpdir can not be too long, and limit it to a proper length, please see this:
http://lists.linuxtogo.org/pipermail/openembedded-core/2012-May/022112.html
BTW. the "argument list too long" error has been fixed.
Quoted 10 lines
Quoted 2 lines
Any suggestion is appreciated.
_______________________________________________
Openembedded-core mailing list
[email protected]
http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
Quoted 6 lines
_______________________________________________
Openembedded-core mailing list
[email protected]
http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
_______________________________________________
Openembedded-core mailing list
[email protected]
http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core

Robert Yang

author

2012-06-23 03:20 UTC #5 lore raw

In reply to Saul Wold

Quoted 14 lines · Saul Wold

Quoted 13 lines · Robert Yang
Thanks Mark, the rpm.real also has the same problem when the length of tmpdir
is 210, it seems that this is because of the glibc.
And also other problems found:
When use PACKAGE_CLASSES = "package_deb", and the length of tmpdir is 177, the error is:
/too/long/path/totmp/sysroots/x86_64-linux/usr/bin/dpkg-scanpackages:
bad interpreter: No such file or directory

Quoted 8 lines

Quoted 5 lines
The interpreter is perl, and it does exist, it seems that we should limit the length of tmpdir to a smaller value then 177 rather than fix these strange
problems. I will go on working on it.
Robert,
A path length of 175 or short is not ideal, we should file bugs against these

Thanks Saul, the bug is:

https://bugzilla.yoctoproject.org/show_bug.cgi?id=2639

I'd like to try to fix it next week, I will send a patch to show the WARNING as you suggested if I can't fix it, but the value maybe shorter than 175, I've realised that even 175 is a very short value, so this should be fixed:-)

// Robert

Quoted 88 lines

issues. We should also add the sanity check you suggested as a WARNING for lenghts over 175. That should not be the final solution though, we need to address the issues and get them fixed up stream.
The groupadd one sounds like it might be a buffer overflow issue which could have larger ramifications
Thanks
Sau!
Quoted 75 lines
// Robert
Quoted 55 lines · Mark Hatle
We're not away of any size limitations within pseudo, other then PATH_MAX which
is typically 4096...
--Mark
Quoted 48 lines · Robert Yang
Hi Experts:
I've met a strange issue, when set the length of builddir to 266 (Why I did this is that I need to know how long the tmpdir that oe-core can support):
NOTE: package dbus-1.4.20-r3.0: task do_install: Started *** glibc detected *** groupadd: malloc(): memory corruption:
0x000000000101da70 ***
*** glibc detected *** groupadd: malloc(): memory corruption:
0x000000000101da70 ***
Then the build would hang, this is caused by the command:
PSEUDO_PREFIX=/too/long/path/tmp/sysroots/x86_64-linux/usr PSEUDO_LOCALSTATEDIR=... PSEUDO_PASSWD=... PSEUDO_NOSYMLINKEXP=1 PSEUDO_DISABLED=0 PSEUDO_LOCALSTATEDIR ... /too/long/path/to/tmp/sysroots/x86_64-linux/usr/bin/pseudo groupadd --root /too/long/path/to/tmp/sysroots/qemux86 -r netdev -f
1) I had looked into the code of groupadd, and found that this would happen when it used the glibc function which needs malloc(for example, the access()), so it seemed this was caused by the glibc or pseudo, but I didn't know why it only happened to groupadd/useradd.
2) I had tried not to use pseudo, it worked well:
sudo groupadd --root /too/long/path/to/tmp/sysroots/qemux86 -r netdev -f
From this, it seemed that the glibc was ok
3) I had tried to write a small piece of code which used the access(), and
used pseudo to run it, it worked well:
/too/long/path/for/pseudo/settings/and/then/run/pseudo my_app
From this, it seemed that both pseudo and glibc were OK.
These 3 steps make me puzzle, maybe we can think that the tmpdir can not be too long, and limit it to a proper length, please see this:
http://lists.linuxtogo.org/pipermail/openembedded-core/2012-May/022112.html
BTW. the "argument list too long" error has been fixed.
Quoted 10 lines
Quoted 2 lines
Any suggestion is appreciated.
_______________________________________________
Openembedded-core mailing list
[email protected]
http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
_______________________________________________
Openembedded-core mailing list
[email protected]
http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core