classes/archiver: fix tarball content in "original" mode
PATCH RESEND v2 1/1 classes/archiver: fix tarball content in "original" mode
When ARCHIVER_MODE[src] is set to "original", a tarball is generated with the unpacked sources of a given recipe. Prior to it, a tmpdir (i.e. from tempfile.mkdtemp()) is created and the sources are unpacked inside:
(...)/vim/9.2.0340/archiver-work/tmp8pu2nsck
`-- vim-9.2.0340
|-- READMEdir
|-- ci
|-- langThis tmpdir, however, ends up being shipped in the deployed tarball:
$ tar -tf vim-9.2.0340-r0.tar.xz (...) tmp8pu2nsck/vim-9.2.0340/ci/config.mk.clang-12.sed tmp8pu2nsck/vim-9.2.0340/LICENSE tmp8pu2nsck/vim-9.2.0340/.git-blame-ignore-revs (...)
This can be avoided by pointing to the subdir where the sources are unpacked, rather than to tmpdir, when calling create_tarball().
The subdir's name can be inferred from 'BP', so we will use 'tmp8pu2nsck/vim-9.2.0340/.' instead of 'tmp8pu2nsck/.' as input.
e.g. and in the result:
./
./vim-9.2.0304/./vim-9.2.0304/SECURITY.md./vim-9.2.0304/.codecov.yml
(...)
--- meta/classes/archiver.bbclass | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
meta/classes/archiver.bbclass +1 -1
@@ -230,7 +230,7 @@ python do_ar_original() { else: bb.fatal("Cannot determine archive names for original source because 'name=' URL parameter '%s' is used twice. Make it unique in: %s %s" % (tarball_suffix[name], url)) tarball_suffix[name] = url - create_tarball(d, tmpdir + '/.', name, ar_outdir) + create_tarball(d, os.path.join(tmpdir, d.getVar('BP'), '.'), name, ar_outdir) # Emit patch series files for 'original' bb.note('Writing patch series files...')
Signature · 1 line
2.55.0
Hello,
Quoted 43 lines · João Marcos Costa
When ARCHIVER_MODE[src] is set to "original", a tarball is generated with the unpacked sources of a given recipe. Prior to it, a tmpdir (i.e. from
tempfile.mkdtemp()) is created and the sources are unpacked inside:(...)/vim/9.2.0340/archiver-work/tmp8pu2nsck
`-- vim-9.2.0340|-- READMEdir |-- ci |-- langThis tmpdir, however, ends up being shipped in the deployed tarball:
$ tar -tf vim-9.2.0340-r0.tar.xz (...) tmp8pu2nsck/vim-9.2.0340/ci/config.mk.clang-12.sed tmp8pu2nsck/vim-9.2.0340/LICENSE tmp8pu2nsck/vim-9.2.0340/.git-blame-ignore-revs (...)This can be avoided by pointing to the subdir where the sources are unpacked, rather than to tmpdir, when calling
create_tarball().The subdir's name can be inferred from 'BP', so we will use 'tmp8pu2nsck/vim-9.2.0340/.' instead of 'tmp8pu2nsck/.' as input.
e.g. and in the result:
./
./vim-9.2.0304/./vim-9.2.0304/SECURITY.md./vim-9.2.0304/.codecov.yml
(...)Signed-off-by João Marcos Costa <[email protected]>--- meta/classes/archiver.bbclass | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)meta/classes/archiver.bbclass +0
-0
meta/classes/archiver.bbclass python do_ar_original() { · line 236
Quoted 8 lines
meta/classes/archiver.bbclass +1
-1@@ -230,7 +230,7 @@ python do_ar_original() { else: bb.fatal("Cannot determine archive names for original source because 'name=' URL parameter '%s' is used twice. Make it unique in: %s %s" % (tarball_suffix[name], url)) tarball_suffix[name] = url - create_tarball(d, tmpdir + '/.', name, ar_outdir) + create_tarball(d, os.path.join(tmpdir, d.getVar('BP'), '.'), name, ar_outdir) # Emit patch series files for 'original' bb.note('Writing patch series files...')
Gentle ping: would there be any updates on this one? Thanks!
Signature · 2 lines
Best regards, João Marcos Costa
Quoted 52 lines · Joao Marcos Costa
Hello,
Quoted 49 lines · João Marcos Costa
When ARCHIVER_MODE[src] is set to "original", a tarball is generated with the unpacked sources of a given recipe. Prior to it, a tmpdir (i.e. from
tempfile.mkdtemp()) is created and the sources are unpacked inside:(...)/vim/9.2.0340/archiver-work/tmp8pu2nsck
`-- vim-9.2.0340|-- READMEdir |-- ci |-- langThis tmpdir, however, ends up being shipped in the deployed tarball:
$ tar -tf vim-9.2.0340-r0.tar.xz (...) tmp8pu2nsck/vim-9.2.0340/ci/config.mk.clang-12.sed tmp8pu2nsck/vim-9.2.0340/LICENSE tmp8pu2nsck/vim-9.2.0340/.git-blame-ignore-revs (...)This can be avoided by pointing to the subdir where the sources are unpacked, rather than to tmpdir, when calling
create_tarball().The subdir's name can be inferred from 'BP', so we will use 'tmp8pu2nsck/vim-9.2.0340/.' instead of 'tmp8pu2nsck/.' as input.
e.g. and in the result:
./
./vim-9.2.0304/./vim-9.2.0304/SECURITY.md./vim-9.2.0304/.codecov.yml
(...)Signed-off-by João Marcos Costa <[email protected]>--- meta/classes/archiver.bbclass | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)meta/classes/archiver.bbclass +1
-1@@ -230,7 +230,7 @@ python do_ar_original() { else: bb.fatal("Cannot determine archive names for original source because 'name=' URL parameter '%s' is used twice. Make it unique in: %s %s" % (tarball_suffix[name], url)) tarball_suffix[name] = url - create_tarball(d, tmpdir + '/.', name, ar_outdir) + create_tarball(d, os.path.join(tmpdir, d.getVar('BP'), '.'), name, ar_outdir)
Quoted 4 lines
meta/classes/archiver.bbclass python do_ar_original() { · line 236
Quoted 2 lines
meta/classes/archiver.bbclass +0
-0# Emit patch series files for 'original' bb.note('Writing patch series files...')Gentle ping: would there be any updates on this one? Thanks!
Hi João,
In my reply to the v1 patch [1], I asked why we're creating a tmpdir under ${ARCHIVER_WORKDIR}, why not just unpack the source directly under the archiver workdir?
The archiver already works in an archiver-work subdirectory, so there isn't any risk of the unpacked sources clashing with ${S} used by other recipe tasks. We may be able to change the code so it doesn't use an additional directory layer.
[1]: https://lore.kernel.org/all/[email protected]/
Best regards,
Signature · 1 line
Paul Barker
Hello, Paul
Thanks for your reply.
Quoted 64 lines · Paul Barker
Quoted 52 lines · Joao Marcos Costa
Hello,
Quoted 49 lines · João Marcos Costa
When ARCHIVER_MODE[src] is set to "original", a tarball is generated with the unpacked sources of a given recipe. Prior to it, a tmpdir (i.e. from
tempfile.mkdtemp()) is created and the sources are unpacked inside:(...)/vim/9.2.0340/archiver-work/tmp8pu2nsck
`-- vim-9.2.0340|-- READMEdir |-- ci |-- langThis tmpdir, however, ends up being shipped in the deployed tarball:
$ tar -tf vim-9.2.0340-r0.tar.xz (...) tmp8pu2nsck/vim-9.2.0340/ci/config.mk.clang-12.sed tmp8pu2nsck/vim-9.2.0340/LICENSE tmp8pu2nsck/vim-9.2.0340/.git-blame-ignore-revs (...)This can be avoided by pointing to the subdir where the sources are unpacked, rather than to tmpdir, when calling
create_tarball().The subdir's name can be inferred from 'BP', so we will use 'tmp8pu2nsck/vim-9.2.0340/.' instead of 'tmp8pu2nsck/.' as input.
e.g. and in the result:
./
./vim-9.2.0304/./vim-9.2.0304/SECURITY.md./vim-9.2.0304/.codecov.yml
(...)Signed-off-by João Marcos Costa <[email protected]>--- meta/classes/archiver.bbclass | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)meta/classes/archiver.bbclass +1
-1@@ -230,7 +230,7 @@ python do_ar_original() { else: bb.fatal("Cannot determine archive names for original source because 'name=' URL parameter '%s' is used twice. Make it unique in: %s %s" % (tarball_suffix[name], url)) tarball_suffix[name] = url - create_tarball(d, tmpdir + '/.', name, ar_outdir) + create_tarball(d, os.path.join(tmpdir, d.getVar('BP'), '.'), name, ar_outdir)Quoted 4 lines
meta/classes/archiver.bbclass python do_ar_original() { · line 236
Quoted 2 lines
meta/classes/archiver.bbclass +0
-0# Emit patch series files for 'original' bb.note('Writing patch series files...')Gentle ping: would there be any updates on this one? Thanks!
Hi João,
In my reply to the v1 patch [1], I asked why we're creating a tmpdir under ${ARCHIVER_WORKDIR}, why not just unpack the source directly under the archiver workdir?
Quoted 8 lines
The archiver already works in an
archiver-worksubdirectory, so there isn't any risk of the unpacked sources clashing with ${S} used by other recipe tasks. We may be able to change the code so it doesn't use an additional directory layer.[1]: https://lore.kernel.org/all/[email protected]/
Best regards,
There could be something off with my mailer, Idk, but I replied to your comment(s) back then:
"I do believe that tmpdir was created to handle the case where multiple source controlled URLs are used in SRC_URI. If I fake a second git for Vim, the result will be a couple of tmpdirs, then 2 separate tarballs, each ending with the suffix defined in SRCREV_FORMAT."
So I believe the tmpdir is needed, and the cleaner way to handle this is to change the call site in do_ar_original() as you suggested, thus my v2. What do you think?
Signature · 2 lines
Best regards, João Marcos Costa
Quoted 81 lines · Joao Marcos Costa
Hello, Paul
Thanks for your reply.
Quoted 64 lines · Paul Barker
Quoted 52 lines · Joao Marcos Costa
Hello,
Quoted 49 lines · João Marcos Costa
When ARCHIVER_MODE[src] is set to "original", a tarball is generated with the unpacked sources of a given recipe. Prior to it, a tmpdir (i.e. from
tempfile.mkdtemp()) is created and the sources are unpacked inside:(...)/vim/9.2.0340/archiver-work/tmp8pu2nsck
`-- vim-9.2.0340|-- READMEdir |-- ci |-- langThis tmpdir, however, ends up being shipped in the deployed tarball:
$ tar -tf vim-9.2.0340-r0.tar.xz (...) tmp8pu2nsck/vim-9.2.0340/ci/config.mk.clang-12.sed tmp8pu2nsck/vim-9.2.0340/LICENSE tmp8pu2nsck/vim-9.2.0340/.git-blame-ignore-revs (...)This can be avoided by pointing to the subdir where the sources are unpacked, rather than to tmpdir, when calling
create_tarball().The subdir's name can be inferred from 'BP', so we will use 'tmp8pu2nsck/vim-9.2.0340/.' instead of 'tmp8pu2nsck/.' as input.
e.g. and in the result:
./
./vim-9.2.0304/./vim-9.2.0304/SECURITY.md./vim-9.2.0304/.codecov.yml
(...)Signed-off-by João Marcos Costa <[email protected]>--- meta/classes/archiver.bbclass | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)meta/classes/archiver.bbclass +1
-1@@ -230,7 +230,7 @@ python do_ar_original() { else: bb.fatal("Cannot determine archive names for original source because 'name=' URL parameter '%s' is used twice. Make it unique in: %s %s" % (tarball_suffix[name], url)) tarball_suffix[name] = url - create_tarball(d, tmpdir + '/.', name, ar_outdir) + create_tarball(d, os.path.join(tmpdir, d.getVar('BP'), '.'), name, ar_outdir)Quoted 4 lines
meta/classes/archiver.bbclass python do_ar_original() { · line 236
Quoted 2 lines
meta/classes/archiver.bbclass +0
-0# Emit patch series files for 'original' bb.note('Writing patch series files...')Gentle ping: would there be any updates on this one? Thanks!
Hi João,
In my reply to the v1 patch [1], I asked why we're creating a tmpdir under ${ARCHIVER_WORKDIR}, why not just unpack the source directly under the archiver workdir?
Quoted 8 lines
The archiver already works in an
archiver-worksubdirectory, so there isn't any risk of the unpacked sources clashing with ${S} used by other recipe tasks. We may be able to change the code so it doesn't use an additional directory layer.[1]: https://lore.kernel.org/all/[email protected]/
Best regards,
There could be something off with my mailer, Idk, but I replied to your comment(s) back then:
Quoted 8 lines
"I do believe that tmpdir was created to handle the case where multiple source controlled URLs are used in
SRC_URI. If I fake a second git for Vim, the result will be a couple of tmpdirs, then 2 separate tarballs, each ending with the suffix defined inSRCREV_FORMAT."So I believe the tmpdir is needed, and the cleaner way to handle this is to change the call site in
do_ar_original()as you suggested, thus my v2. What do you think?
Hi João,
This does explain the tmpdir a little more. In which case v2 is probably good, I'll see if we can move this forward in the next patch review call.
Thanks,
Signature · 1 line
Paul Barker
Hello, Paul and Yoann
On 10/16/26 3:40 PM, Paul Barker wrote:
(...)
Quoted 8 lines
Hi João,
This does explain the tmpdir a little more. In which case v2 is probably good, I'll see if we can move this forward in the next patch review call.
Thanks,
While I see this as a bugfix, some might see as an "improvement". What do you think? Should this fix be backported to blacksail/wrynose/scarthgap? If so, let me know and I'll prepare the patches.
Signature · 2 lines
Best regards, João Marcos Costa