apt 0.7.14: runtime error fix when tmpdir is very long
cover apt 0.7.14: runtime error fix when tmpdir is very long
The following changes since commit a6b6df1350149c116050cb93c3c7b4802c709d31:
task-core-tools-debug: Added openssh-sftp-server. (2012-07-03 14:52:38 +0100)
are available in the git repository at:
git://git.pokylinux.org/poky-contrib robert/apt
http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=robert/apt
Robert Yang (2):
apt 0.7.14: runtime error: filename too long (tmpdir length) apt 0.7.14: runtime error: Method file has died unexpectedly
.../apt/apt-0.7.14/allocate-larger-memory.patch | 75 ++++++++++++++++++++ .../apt/apt-0.7.14/truncate-filename.patch | 35 +++++++++ meta/recipes-devtools/apt/apt-native_0.7.14.bb | 2 +- meta/recipes-devtools/apt/apt.inc | 2 + meta/recipes-devtools/apt/apt_0.7.14.bb | 2 +- 5 files changed, 114 insertions(+), 2 deletions(-) create mode 100644 meta/recipes-devtools/apt/apt-0.7.14/allocate-larger-memory.patch create mode 100644 meta/recipes-devtools/apt/apt-0.7.14/truncate-filename.patch
PATCH 1/2 apt 0.7.14: runtime error: filename too long (tmpdir length)
when the tmpdir dir is longer than 220, there is no files saved in tmp/sysroots/x86_64-linux/var/lib/apt/lists/ after run apt-get update, this is because apt-get uses the path as the file name, but the file name can't be longer than 255 according to /usr/include/linux/limits.h.
[YOCTO #2688]
--- .../apt/apt-0.7.14/truncate-filename.patch | 35 ++++++++++++++++++++ meta/recipes-devtools/apt/apt-native_0.7.14.bb | 2 +- meta/recipes-devtools/apt/apt.inc | 1 + meta/recipes-devtools/apt/apt_0.7.14.bb | 2 +- 4 files changed, 38 insertions(+), 2 deletions(-) create mode 100644 meta/recipes-devtools/apt/apt-0.7.14/truncate-filename.patch
meta/recipes-devtools/apt/apt-0.7.14/truncate-filename.patch new file +35 -0
@@ -0,0 +1,35 @@ +strutl.cc: the filename can't be longer than 255 + +The URItoFileName translates the path into the filename, but the +filename can't be longer than 255 according to +/usr/include/linux/limits.h. + +Truncate it when it is longer than 240 (leave some spaces for +".Packages" and "._Release" suffix) + +Upstream-Status: Pending +Signed-off-by: Robert Yang <[email protected]> +--- + apt-pkg/contrib/strutl.cc | 7 ++++++- + 1 file changed, 6 insertions(+), 1 deletion(-) + +diff --git a/apt-pkg/contrib/strutl.cc b/apt-pkg/contrib/strutl.cc +--- a/apt-pkg/contrib/strutl.cc ++++ b/apt-pkg/contrib/strutl.cc +@@ -399,7 +399,12 @@ string URItoFileName(const string &URI) + // "\x00-\x20{}|\\\\^\\[\\]<>\"\x7F-\xFF"; + string NewURI = QuoteString(U,"\\|{}[]<>\"^~_=!@#$%^&*"); + replace(NewURI.begin(),NewURI.end(),'/','_'); +- return NewURI; ++ ++ // Truncate from the head when it is longer than 240 ++ if(NewURI.length() > 240) ++ return NewURI.substr(NewURI.length() - 240, NewURI.length() - 1); ++ else ++ return NewURI; + } + /*}}}*/ + // Base64Encode - Base64 Encoding routine for short strings /*{{{*/ +-- +1.7.10.4 +
meta/recipes-devtools/apt/apt-native_0.7.14.bb +1 -1
@@ -1,6 +1,6 @@ require apt-native.inc -PR = "r8" +PR = "r9" SRC_URI += "file://nodoc.patch \ file://noconfigure.patch \
meta/recipes-devtools/apt/apt.inc +1 -0
@@ -8,6 +8,7 @@ SRC_URI = "${DEBIAN_MIRROR}/main/a/apt/apt_${PV}.tar.gz \ file://localefixes.patch \ file://makerace.patch \ file://remove-redeclaration.patch \ + file://truncate-filename.patch \ " inherit autotools gettext
meta/recipes-devtools/apt/apt_0.7.14.bb +1 -1
@@ -3,7 +3,7 @@ RDEPENDS_${PN} = "dpkg" LIC_FILES_CHKSUM = "file://COPYING.GPL;md5=0636e73ff0215e8d672dc4c32c317bb3" require apt.inc -PR = "r13" +PR = "r14" SRC_URI += "file://nodoc.patch \ file://includes-fix.patch "
Signature · 1 line
1.7.1
Re: [PATCH 1/2] apt 0.7.14: runtime error: filename too long (tmpdir length)
Quoted 41 lines · Robert Yang
when the tmpdir dir is longer than 220, there is no files saved in tmp/sysroots/x86_64-linux/var/lib/apt/lists/ after run apt-get update, this is because apt-get uses the path as the file name, but the file name can't be longer than 255 according to
/usr/include/linux/limits.h.[YOCTO #2688]
Signed-off-by Robert Yang <[email protected]>--- .../apt/apt-0.7.14/truncate-filename.patch | 35 ++++++++++++++++++++ meta/recipes-devtools/apt/apt-native_0.7.14.bb | 2 +- meta/recipes-devtools/apt/apt.inc | 1 + meta/recipes-devtools/apt/apt_0.7.14.bb | 2 +- 4 files changed, 38 insertions(+), 2 deletions(-)create mode 100644
meta/recipes-devtools/apt/apt-0.7.14/truncate-filename.patchmeta/recipes-devtools/apt/apt-0.7.14/truncate-filename.patch new file +19
-0@@ -0,0 +1,35 @@ +strutl.cc: the filename can't be longer than 255 + +The URItoFileName translates the path into the filename, but the +filename can't be longer than 255 according to +/usr/include/linux/limits.h. + +Truncate it when it is longer than 240 (leave some spaces for +".Packages" and "._Release" suffix) + +Upstream-Status: Pending +Signed-off-by: Robert Yang <[email protected]> +--- + apt-pkg/contrib/strutl.cc | 7 ++++++- + 1 file changed, 6 insertions(+), 1 deletion(-) + +diff --git a/apt-pkg/contrib/strutl.cc b/apt-pkg/contrib/strutl.cc +--- a/apt-pkg/contrib/strutl.cc ++++ b/apt-pkg/contrib/strutl.cc +@@ -399,7 +399,12 @@ string URItoFileName(const string &URI)
meta/recipes-devtools/apt/apt-0.7.14/truncate-filename.patch line 29
Quoted 10 lines
meta/recipes-devtools/apt/apt-0.7.14/truncate-filename.patch new file +10
-0+ // "\x00-\x20{}|\\\\^\\[\\]<>\"\x7F-\xFF"; + string NewURI = QuoteString(U,"\\|{}[]<>\"^~_=!@#$%^&*"); + replace(NewURI.begin(),NewURI.end(),'/','_'); +- return NewURI; ++ ++ // Truncate from the head when it is longer than 240 ++ if(NewURI.length() > 240) ++ return NewURI.substr(NewURI.length() - 240, NewURI.length() - 1); ++ else ++ return NewURI;
Robert, I am not super familiar with this part of the DPKG code, what uses the URItoFileName() and what effect does the truncation have?
Sau!
Quoted 44 lines
meta/recipes-devtools/apt/apt-0.7.14/truncate-filename.patch new file +6
-0+ } + /*}}}*/ + // Base64Encode - Base64 Encoding routine for short strings /*{{{*/ +-- +1.7.10.4 +meta/recipes-devtools/apt/apt-native_0.7.14.bb +1
-1@@ -1,6 +1,6 @@ require apt-native.inc -PR = "r8" +PR = "r9" SRC_URI += "file://nodoc.patch \ file://noconfigure.patch \meta/recipes-devtools/apt/apt.inc +1
-0@@ -8,6 +8,7 @@ SRC_URI = "${DEBIAN_MIRROR}/main/a/apt/apt_${PV}.tar.gz \ file://localefixes.patch \ file://makerace.patch \ file://remove-redeclaration.patch \ + file://truncate-filename.patch \ " inherit autotools gettextmeta/recipes-devtools/apt/apt_0.7.14.bb +1
-1@@ -3,7 +3,7 @@ RDEPENDS_${PN} = "dpkg" LIC_FILES_CHKSUM = "file://COPYING.GPL;md5=0636e73ff0215e8d672dc4c32c317bb3" require apt.inc -PR = "r13" +PR = "r14" SRC_URI += "file://nodoc.patch \ file://includes-fix.patch "
Re: [PATCH 1/2] apt 0.7.14: runtime error: filename too long (tmpdir length)
Quoted 48 lines · Saul Wold
Quoted 47 lines · Robert Yang
when the tmpdir dir is longer than 220, there is no files saved in tmp/sysroots/x86_64-linux/var/lib/apt/lists/ after run apt-get update, this is because apt-get uses the path as the file name, but the file name can't be longer than 255 according to
/usr/include/linux/limits.h.[YOCTO #2688]
Signed-off-by Robert Yang <[email protected]>--- .../apt/apt-0.7.14/truncate-filename.patch | 35 ++++++++++++++++++++ meta/recipes-devtools/apt/apt-native_0.7.14.bb | 2 +- meta/recipes-devtools/apt/apt.inc | 1 + meta/recipes-devtools/apt/apt_0.7.14.bb | 2 +- 4 files changed, 38 insertions(+), 2 deletions(-)create mode 100644
meta/recipes-devtools/apt/apt-0.7.14/truncate-filename.patchmeta/recipes-devtools/apt/apt-0.7.14/truncate-filename.patch new file +24
-0@@ -0,0 +1,35 @@ +strutl.cc: the filename can't be longer than 255 + +The URItoFileName translates the path into the filename, but the +filename can't be longer than 255 according to +/usr/include/linux/limits.h. + +Truncate it when it is longer than 240 (leave some spaces for +".Packages" and "._Release" suffix) + +Upstream-Status: Pending +Signed-off-by: Robert Yang <[email protected]> +--- + apt-pkg/contrib/strutl.cc | 7 ++++++- + 1 file changed, 6 insertions(+), 1 deletion(-) + +diff --git a/apt-pkg/contrib/strutl.cc b/apt-pkg/contrib/strutl.cc +--- a/apt-pkg/contrib/strutl.cc ++++ b/apt-pkg/contrib/strutl.cc +@@ -399,7 +399,12 @@ string URItoFileName(const string &URI) + // "\x00-\x20{}|\\\\^\\[\\]<>\"\x7F-\xFF"; + string NewURI = QuoteString(U,"\\|{}[]<>\"^~_=!@#$%^&*"); + replace(NewURI.begin(),NewURI.end(),'/','_'); +- return NewURI; ++
Quoted 9 lines
meta/recipes-devtools/apt/apt-0.7.14/truncate-filename.patch line 29
Quoted 5 lines
meta/recipes-devtools/apt/apt-0.7.14/truncate-filename.patch new file +5
-0++ // Truncate from the head when it is longer than 240 ++ if(NewURI.length() > 240) ++ return NewURI.substr(NewURI.length() - 240, NewURI.length() - 1); ++ else ++ return NewURI;Robert, I am not super familiar with this part of the DPKG code, what uses the
URItoFileName()and what effect does the truncation have?
Hi Saul,
The filename which is used /var/lib/apt/lists/ is from URItoFileName(), for example, there is a line in my /etc/apt/sources.list:
deb http://mirrors.163.com/ubuntu/ maverick main restricted universe multiverse
Then the URItoFileName will change this line to a filename:
[snip] mirrors.163.com_ubuntu_dists_maverick_main_binary-i386_Packages mirrors.163.com_ubuntu_dists_maverick_restricted_binary-i386_Packages [snip]
When the filename is longer than 240, truncate the head of them, e.g.:
ubuntu_dists_maverick_main_binary-i386_Packages ubuntu_dists_maverick_restricted_binary-i386_Packages
I think this is fine for us since the end part of the path is uniq. Another solution is that use the md5sum for the filename no matter how long it is, or use the "translated_path+part_of_the_md5sum", this is easy to do, but it is a little big change to apt, and we don't need.
// Robert
Quoted 52 lines
Sau!
Quoted 47 lines
meta/recipes-devtools/apt/apt-0.7.14/truncate-filename.patch new file +6
-0+ } + /*}}}*/ + // Base64Encode - Base64 Encoding routine for short strings /*{{{*/ +-- +1.7.10.4 +meta/recipes-devtools/apt/apt-native_0.7.14.bb +1
-1@@ -1,6 +1,6 @@ require apt-native.inc -PR = "r8" +PR = "r9" SRC_URI += "file://nodoc.patch \ file://noconfigure.patch \meta/recipes-devtools/apt/apt.inc +1
-0@@ -8,6 +8,7 @@ SRC_URI = "${DEBIAN_MIRROR}/main/a/apt/apt_${PV}.tar.gz \ file://localefixes.patch \ file://makerace.patch \ file://remove-redeclaration.patch \ + file://truncate-filename.patch \ " inherit autotools gettextmeta/recipes-devtools/apt/apt_0.7.14.bb +1
-1@@ -3,7 +3,7 @@ RDEPENDS_${PN} = "dpkg" LIC_FILES_CHKSUM = "file://COPYING.GPL;md5=0636e73ff0215e8d672dc4c32c317bb3" require apt.inc -PR = "r13" +PR = "r14" SRC_URI += "file://nodoc.patch \ file://includes-fix.patch "
PATCH 2/2 apt 0.7.14: runtime error: Method file has died unexpectedly
In reply to Robert YangWhen the length of the tmpdir is longer than 400, there is an error when run "apt-get update":
Method file has died unexpectedly!
This is because the "char S[1024]" is not enough for long URI, S[2048] would be enough.
[YOCTO #2689]
--- .../apt/apt-0.7.14/allocate-larger-memory.patch | 75 ++++++++++++++++++++ meta/recipes-devtools/apt/apt-native_0.7.14.bb | 2 +- meta/recipes-devtools/apt/apt.inc | 1 + meta/recipes-devtools/apt/apt_0.7.14.bb | 2 +- 4 files changed, 78 insertions(+), 2 deletions(-) create mode 100644 meta/recipes-devtools/apt/apt-0.7.14/allocate-larger-memory.patch
meta/recipes-devtools/apt/apt-0.7.14/allocate-larger-memory.patch new file +75 -0
@@ -0,0 +1,75 @@ +Method file has died unexpectedly + +"Method file has died unexpectedly!", this is because the "char S[1024]" +is not enough for the long the URI, "char S[2048]" would be enough. + +It would be boring if we use malloc here since we can't know how much +memory is needed except strelen() every component of it. So similarly +use "char S[2048]" as it did before. + +Upstream-Status: Pending +Signed-off-by: Robert Yang <[email protected]> +--- + apt-pkg/acquire-method.cc | 12 ++++++------ + 1 file changed, 6 insertions(+), 6 deletions(-) + +diff --git a/apt-pkg/acquire-method.cc b/apt-pkg/acquire-method.cc +--- a/apt-pkg/acquire-method.cc ++++ b/apt-pkg/acquire-method.cc +@@ -95,7 +95,7 @@ void pkgAcqMethod::Fail(string Err,bool Transient) + *I = ' '; + } + +- char S[1024]; ++ char S[2048]; + if (Queue != 0) + { + snprintf(S,sizeof(S)-50,"400 URI Failure\nURI: %s\n" +@@ -132,7 +132,7 @@ void pkgAcqMethod::URIStart(FetchResult &Res) + if (Queue == 0) + abort(); + +- char S[1024] = ""; ++ char S[2048] = ""; + char *End = S; + + End += snprintf(S,sizeof(S),"200 URI Start\nURI: %s\n",Queue->Uri.c_str()); +@@ -160,7 +160,7 @@ void pkgAcqMethod::URIDone(FetchResult &Res, FetchResult *Alt) + if (Queue == 0) + abort(); + +- char S[1024] = ""; ++ char S[2048] = ""; + char *End = S; + + End += snprintf(S,sizeof(S),"201 URI Done\nURI: %s\n",Queue->Uri.c_str()); +@@ -242,7 +242,7 @@ void pkgAcqMethod::URIDone(FetchResult &Res, FetchResult *Alt) + to be ackd */ + bool pkgAcqMethod::MediaFail(string Required,string Drive) + { +- char S[1024]; ++ char S[2048]; + snprintf(S,sizeof(S),"403 Media Failure\nMedia: %s\nDrive: %s\n\n", + Required.c_str(),Drive.c_str()); + +@@ -411,7 +411,7 @@ void pkgAcqMethod::Log(const char *Format,...) + va_start(args,Format); + + // sprintf the description +- char S[1024]; ++ char S[2048]; + unsigned int Len = snprintf(S,sizeof(S)-4,"101 Log\nURI: %s\n" + "Message: ",CurrentURI.c_str()); + +@@ -435,7 +435,7 @@ void pkgAcqMethod::Status(const char *Format,...) + va_start(args,Format); + + // sprintf the description +- char S[1024]; ++ char S[2048]; + unsigned int Len = snprintf(S,sizeof(S)-4,"102 Status\nURI: %s\n" + "Message: ",CurrentURI.c_str()); + +-- +1.7.10.4 +
meta/recipes-devtools/apt/apt-native_0.7.14.bb +1 -1
@@ -1,6 +1,6 @@ require apt-native.inc -PR = "r9" +PR = "r10" SRC_URI += "file://nodoc.patch \ file://noconfigure.patch \
meta/recipes-devtools/apt/apt.inc +1 -0
@@ -9,6 +9,7 @@ SRC_URI = "${DEBIAN_MIRROR}/main/a/apt/apt_${PV}.tar.gz \ file://makerace.patch \ file://remove-redeclaration.patch \ file://truncate-filename.patch \ + file://allocate-larger-memory.patch \ " inherit autotools gettext
meta/recipes-devtools/apt/apt_0.7.14.bb +1 -1
@@ -3,7 +3,7 @@ RDEPENDS_${PN} = "dpkg" LIC_FILES_CHKSUM = "file://COPYING.GPL;md5=0636e73ff0215e8d672dc4c32c317bb3" require apt.inc -PR = "r14" +PR = "r15" SRC_URI += "file://nodoc.patch \ file://includes-fix.patch "
Signature · 1 line
1.7.1
Saul Wold
AppliedQuoted 18 lines · Robert Yang
The following changes since commit
a6b6df1350149c116050cb93c3c7b4802c709d31:task-core-tools-debug: Added openssh-sftp-server. (2012-07-03 14:52:38 +0100)are available in the git repository at:
git://git.pokylinux.org/poky-contribrobert/apt
http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=robert/aptRobert Yang (2):
apt 0.7.14: runtime error: filename too long (tmpdir length) apt 0.7.14: runtime error: Method file has died unexpectedly.../apt/apt-0.7.14/allocate-larger-memory.patch | 75 ++++++++++++++++++++ .../apt/apt-0.7.14/truncate-filename.patch | 35 +++++++++ meta/recipes-devtools/apt/apt-native_0.7.14.bb | 2 +- meta/recipes-devtools/apt/apt.inc | 2 + meta/recipes-devtools/apt/apt_0.7.14.bb | 2 +- 5 files changed, 114 insertions(+), 2 deletions(-)
Quoted 9 lines
create mode 100644
meta/recipes-devtools/apt/apt-0.7.14/allocate-larger-memory.patchcreate mode 100644meta/recipes-devtools/apt/apt-0.7.14/truncate-filename.patch_______________________________________________
Openembedded-core mailing list
[email protected]
http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
Merged into OE-Core
Thanks
Sau!