New: Can't download newer revisions of package from PyPI

https://bugs.busybox.net/show_bug.cgi?id=9706

Bug ID: 9706
Summary: Can't download newer revisions of package from PyPI
Product: buildroot
Version: 2016.11
Hardware: All
OS: Linux
Status: NEW
Severity: normal
Priority: P5
Component: Outdated package
Assignee: unassigned at buildroot.uclibc.org
Reporter: jem at kamama.com
CC: buildroot at uclibc.org
Target Milestone: ---

Newer versions of python projects (in my case I was attempting to update supervisor to supervisor-3.3.1) don't download with the URL scheme currently used. Not sure if this is only a supervisor problem or might affect other modules from PyPI. Recommendation from PyPI is to use their API

API documentation is at https://warehouse.pypa.io/api-reference/legacy/
It looks like buildroot should first make a request against
/simple/supervisor/ and fetch links from the response.

For example if I hit https://pypi.python.org/simple/supervisor/ I get:

<!DOCTYPE html><html><head><title>Links for
supervisor</title></head><body><h1>Links for supervisor</h1><a href="../../packages/2e/3b/f28afd29920be27c948923b276d1f131276880f6b4f9391323064b87b15b/supervisor-3.0a7.tar.gz#md5=d2e6c491fcb2606e4fd0afe4ebfd4b13" rel="internal">supervisor-3.0a7.tar.gz</a><br/>
<a href="../../packages/44/80/d28047d120bfcc8158b4e41127706731ee6a3419c661e0a858fb0e7c4b2d/supervisor-3.3.0.tar.gz#md5=46bac00378d1eddb616752b990c67416" rel="internal">supervisor-3.3.0.tar.gz</a><br/>
<a href="../../packages/4b/7d/cb3d91d6c65f73fd253c376fc0dbbc48b83e6606f61d9961d9ac7cf293f5/supervisor-3.0a9.tar.gz#md5=81d192acab20eef3578856651af7e851" rel="internal">supervisor-3.0a9.tar.gz</a><br/>

...etc.

You are receiving this mail because:
You are on the CC list for the bug.

https://bugs.busybox.net/show_bug.cgi?id=9706

Arnout Vandecappelle <[email protected]> changed:

What |Removed |Added
----------------------------------------------------------------------------

             Status|NEW                         |RESOLVED
         Resolution|---                         |INVALID

--- Comment #1 from Arnout Vandecappelle <[email protected]> --- When bumping the package, SUPERVISOR_SITE should change from the current

http://pypi.python.org/packages/source/s/supervisor

to

https://pypi.python.org/packages/44/80/d28047d120bfcc8158b4e41127706731ee6a3419c661e0a858fb0e7c4b2d

but it's not unusual for the _SITE to change when the version is bumped.

I don't see how this is a bug, so I'm closing as invalid. If you disagree, feel free to reopen.

PS if you do bump the package, don't forget to send the patch to the Buildroot mailing list for upstream inclusion.

You are receiving this mail because:
You are on the CC list for the bug.

Hi Arnout,

I've posted an RFC [1] concerning this issue.

[1] https://patchwork.ozlabs.org/patch/708384/

Yegor

https://bugs.busybox.net/show_bug.cgi?id=9706

--- Comment #2 from Arnout Vandecappelle <[email protected]> --- Correction, the URL for version 3.3.1 is:

https://pypi.python.org/packages/80/37/964c0d53cbd328796b1aeb7abea4c0f7b0e8c7197ea9b0b9967b7d004def

You are receiving this mail because:
You are on the CC list for the bug.

https://bugs.busybox.net/show_bug.cgi?id=9706

--- Comment #3 from J. Eric Mason <[email protected]> --- Semantically buildroot could (I won't go as far to say should yet) be using the simple repository protocol to find download URLs and hashes for a package generically https://www.python.org/dev/peps/pep-0503/ as the URLs could change over time. If I have the bandwidth I'll see if I can propose a changelist to do this resolution so this wouldn't be a problem in the future.

You are receiving this mail because:
You are on the CC list for the bug.

https://bugs.busybox.net/show_bug.cgi?id=9706

--- Comment #4 from Thomas Petazzoni <[email protected]> --- I think what you're proposing is exactly what Yegor Yefremov has implemented in
https://patchwork.ozlabs.org/patch/708384/.

You are receiving this mail because:
You are on the CC list for the bug.