Skip to contentSkip to conversation

scanpypi: use archive file name to specify the extraction folder

PATCH Superseded yegorslists at googlemail.com 4 participants 5 messages Nov 2, 2018 – Nov 14, 2018

lore
Share & tools
lore

yegorslists at googlemail.com

author
#0 lore raw

PATCH v2 1/1 scanpypi: use archive file name to specify the extraction folder

From: Yegor Yefremov <[email protected]>

Some packages have archive name that is different from package name. For example websocket-client's archive name is websocket_client-*.tar.gz. scanpypi expects the temporary extract folder to be:

/tmp-folder/BR-package-name/PyPI-packagename-and-version

In the case of websocket-client package the real extraction folder will be different from the expected one because of the '_' in the archive file name.

Use archive file name instead of package name to specify the extraction folder. As the version is already part of this file, we don't need to specify it.

Bonus: remove obsolete "return None, None" as the function doesn't return anything. OSError class doesn't provide "message" member, so replace it with "strerror".

Fixes:
https://bugs.busybox.net/show_bug.cgi?id=11251

Signed-off-by Yegor Yefremov <[email protected]>

---
Changes v1->v2:
- fix typos
- add fixes tag

 utils/scanpypi | 19 ++++++++++---------
 1 file changed, 10 insertions(+), 9 deletions(-)
utils/scanpypi +10 -9
index 3983be1ad1..3d3e604435 100755
@@ -243,13 +243,14 @@ class BuildrootPackage():
                     os.makedirs(tmp_pkg)
                 except OSError as exception:
                     if exception.errno != errno.EEXIST:
-                        print("ERROR: ", exception.message, file=sys.stderr)
-                        return None, None
-                    print('WARNING:', exception.message, file=sys.stderr)
+                        print("ERROR: ", exception.strerror, file=sys.stderr)
+                        return
+                    print('WARNING:', exception.strerror, file=sys.stderr)
                     print('Removing {pkg}...'.format(pkg=tmp_pkg))
                     shutil.rmtree(tmp_pkg)
                     os.makedirs(tmp_pkg)
                 as_zipfile.extractall(tmp_pkg)
+                pkg_filename = self.filename.split(".zip")[0]
         else:
             with tarfile.open(fileobj=as_file) as as_tarfile:
                 tmp_pkg = os.path.join(tmp_path, self.buildroot_name)

@@ -257,19 +258,19 @@ class BuildrootPackage():
                     os.makedirs(tmp_pkg)
                 except OSError as exception:
                     if exception.errno != errno.EEXIST:
-                        print("ERROR: ", exception.message, file=sys.stderr)
-                        return None, None
-                    print('WARNING:', exception.message, file=sys.stderr)
+                        print("ERROR: ", exception.strerror, file=sys.stderr)
+                        return
+                    print('WARNING:', exception.strerror, file=sys.stderr)
                     print('Removing {pkg}...'.format(pkg=tmp_pkg))
                     shutil.rmtree(tmp_pkg)
                     os.makedirs(tmp_pkg)
                 as_tarfile.extractall(tmp_pkg)
+                pkg_filename = self.filename.split(".tar")[0]
 
-        tmp_extract = '{folder}/{name}-{version}'
+        tmp_extract = '{folder}/{name}'
         self.tmp_extract = tmp_extract.format(
             folder=tmp_pkg,
-            name=self.metadata_name,
-            version=self.version)
+            name=pkg_filename)
 
     def load_setup(self):
         """
Signature · 1 line
2.17.0
#1 lore raw

PATCH v2 1/1 scanpypi: use archive file name to specify the extraction folder

Quoted 2 lines · On Fri, Nov 2, 2018 at 3:45 PM <[email protected]> wrote:

From: Yegor Yefremov <[email protected]>

Reviewed-by Asaf Kahlon <[email protected]>
Quoted 86 lines

Some packages have archive name that is different from package name. For example websocket-client's archive name is websocket_client-*.tar.gz. scanpypi expects the temporary extract folder to be:

/tmp-folder/BR-package-name/PyPI-packagename-and-version

In the case of websocket-client package the real extraction folder will be different from the expected one because of the '_' in the archive file name.

Use archive file name instead of package name to specify the extraction folder. As the version is already part of this file, we don't need to specify it.

Bonus: remove obsolete "return None, None" as the function doesn't return anything. OSError class doesn't provide "message" member, so replace it with "strerror".

Fixes:
https://bugs.busybox.net/show_bug.cgi?id=11251

Signed-off-by Yegor Yefremov <[email protected]>

---
Changes v1->v2:
- fix typos
- add fixes tag

 utils/scanpypi | 19 ++++++++++---------
 1 file changed, 10 insertions(+), 9 deletions(-)
utils/scanpypi +10 -10
index 3983be1ad1..3d3e604435 100755
@@ -243,13 +243,14 @@ class BuildrootPackage():
                     os.makedirs(tmp_pkg)
                 except OSError as exception:
                     if exception.errno != errno.EEXIST:
-                        print("ERROR: ", exception.message, file=sys.stderr)
-                        return None, None
-                    print('WARNING:', exception.message, file=sys.stderr)
+                        print("ERROR: ", exception.strerror, file=sys.stderr)
+                        return
+                    print('WARNING:', exception.strerror, file=sys.stderr)
                     print('Removing {pkg}...'.format(pkg=tmp_pkg))
                     shutil.rmtree(tmp_pkg)
                     os.makedirs(tmp_pkg)
                 as_zipfile.extractall(tmp_pkg)
+                pkg_filename = self.filename.split(".zip")[0]
         else:
             with tarfile.open(fileobj=as_file) as as_tarfile:
                 tmp_pkg = os.path.join(tmp_path, self.buildroot_name)

@@ -257,19 +258,19 @@ class BuildrootPackage():
                     os.makedirs(tmp_pkg)
                 except OSError as exception:
                     if exception.errno != errno.EEXIST:
-                        print("ERROR: ", exception.message, file=sys.stderr)
-                        return None, None
-                    print('WARNING:', exception.message, file=sys.stderr)
+                        print("ERROR: ", exception.strerror, file=sys.stderr)
+                        return
+                    print('WARNING:', exception.strerror, file=sys.stderr)
                     print('Removing {pkg}...'.format(pkg=tmp_pkg))
                     shutil.rmtree(tmp_pkg)
                     os.makedirs(tmp_pkg)
                 as_tarfile.extractall(tmp_pkg)
+                pkg_filename = self.filename.split(".tar")[0]
 
-        tmp_extract = '{folder}/{name}-{version}'
+        tmp_extract = '{folder}/{name}'
         self.tmp_extract = tmp_extract.format(
             folder=tmp_pkg,
-            name=self.metadata_name,
-            version=self.version)
+            name=pkg_filename)
 
     def load_setup(self):
         """
--

2.17.0

_______________________________________________
buildroot mailing list
buildroot at busybox.net
http://lists.busybox.net/mailman/listinfo/buildroot

#2 lore raw

PATCH v2 1/1 scanpypi: use archive file name to specify the extraction folder

In reply to yegorslists at googlemail.com

Hello,

Quoted 12 lines · yegorslists at googlemail.com

From: Yegor Yefremov <[email protected]>

Some packages have archive name that is different from package name. For example websocket-client's archive name is websocket_client-*.tar.gz. scanpypi expects the temporary extract folder to be:

/tmp-folder/BR-package-name/PyPI-packagename-and-version

In the case of websocket-client package the real extraction folder will be different from the expected one because of the '_' in the archive file name.

Quoted 7 lines

Use archive file name instead of package name to specify the extraction folder. As the version is already part of this file, we don't need to specify it.

Bonus: remove obsolete "return None, None" as the function doesn't return anything. OSError class doesn't provide "message" member, so replace it with "strerror".

I've applied to master, but I believe it should perhaps have been two separate patches. Indeed, I don't see the relationship between the use of the archive file name, and the change around "return None, None".

Thanks!

Thomas

Signature · 3 lines
Thomas Petazzoni, CTO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com
author
#3 lore raw

PATCH v2 1/1 scanpypi: use archive file name to specify the extraction folder

Quoted 20 lines · Thomas Petazzoni

Hello,

Quoted 16 lines · yegorslists at googlemail.com

From: Yegor Yefremov <[email protected]>

Some packages have archive name that is different from package name. For example websocket-client's archive name is websocket_client-*.tar.gz. scanpypi expects the temporary extract folder to be:

/tmp-folder/BR-package-name/PyPI-packagename-and-version

In the case of websocket-client package the real extraction folder will be different from the expected one because of the '_' in the archive file name.

Use archive file name instead of package name to specify the extraction folder. As the version is already part of this file, we don't need to specify it.

Quoted 7 lines
Quoted 3 lines

Bonus: remove obsolete "return None, None" as the function doesn't return anything. OSError class doesn't provide "message" member, so replace it with "strerror".

I've applied to master, but I believe it should perhaps have been two separate patches. Indeed, I don't see the relationship between the use of the archive file name, and the change around "return None, None".

You're right. Will do next time.

Yegor

#4 lore raw

PATCH v2 1/1 scanpypi: use archive file name to specify the extraction folder

In reply to yegorslists at googlemail.com
Quoted 21 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

"yegorslists" == yegorslists <[email protected]> writes:

From: Yegor Yefremov <[email protected]>
Some packages have archive name that is different from package name. For example websocket-client's archive name is websocket_client-*.tar.gz. scanpypi expects the temporary extract folder to be:

/tmp-folder/BR-package-name/PyPI-packagename-and-version

In the case of websocket-client package the real extraction folder will be different from the expected one because of the '_' in the archive file name.

Use archive file name instead of package name to specify the extraction folder. As the version is already part of this file, we don't need to specify it.

Bonus: remove obsolete "return None, None" as the function doesn't return anything. OSError class doesn't provide "message" member, so replace it with "strerror".

Quoted 8 lines

Fixes:
https://bugs.busybox.net/show_bug.cgi?id=11251

Signed-off-by Yegor Yefremov <[email protected]>

---
Changes v1->v2:
- fix typos
- add fixes tag

Committed to 2018.02.x and 2018.08.x, thanks.

Signature · 1 line
Bye, Peter Korsgaard