Skip to contentSkip to conversation

classes/archiver: fix tarball content in "original" mode

PATCH RESEND In review João Marcos Costa 2 participants 6 messages Sep 18 – Oct 20

lore
Share & tools
lore
#0 lore raw

PATCH RESEND v2 1/1 classes/archiver: fix tarball content in "original" mode

When ARCHIVER_MODE[src] is set to "original", a tarball is generated with the unpacked sources of a given recipe. Prior to it, a tmpdir (i.e. from tempfile.mkdtemp()) is created and the sources are unpacked inside:

(...)/vim/9.2.0340/archiver-work/tmp8pu2nsck
`-- vim-9.2.0340

    |-- READMEdir
    |-- ci
    |-- lang

This tmpdir, however, ends up being shipped in the deployed tarball:

$ tar -tf vim-9.2.0340-r0.tar.xz
(...)
tmp8pu2nsck/vim-9.2.0340/ci/config.mk.clang-12.sed
tmp8pu2nsck/vim-9.2.0340/LICENSE
tmp8pu2nsck/vim-9.2.0340/.git-blame-ignore-revs
(...)

This can be avoided by pointing to the subdir where the sources are unpacked, rather than to tmpdir, when calling create_tarball().

The subdir's name can be inferred from 'BP', so we will use 'tmp8pu2nsck/vim-9.2.0340/.' instead of 'tmp8pu2nsck/.' as input.

e.g. and in the result:

./
./vim-9.2.0304/
./vim-9.2.0304/SECURITY.md
./vim-9.2.0304/.codecov.yml
(...)

---
 meta/classes/archiver.bbclass | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
meta/classes/archiver.bbclass +1 -1
index b274c53e79..c824718a1d 100644
@@ -230,7 +230,7 @@ python do_ar_original() {
                     else:
                         bb.fatal("Cannot determine archive names for original source because 'name=' URL parameter '%s' is used twice. Make it unique in: %s %s" % (tarball_suffix[name], url))
             tarball_suffix[name] = url
-            create_tarball(d, tmpdir + '/.', name, ar_outdir)
+            create_tarball(d, os.path.join(tmpdir, d.getVar('BP'), '.'), name, ar_outdir)
 
     # Emit patch series files for 'original'
     bb.note('Writing patch series files...')
Signature · 1 line
2.55.0
#1 lore raw

Hello,

Quoted 43 lines · João Marcos Costa

When ARCHIVER_MODE[src] is set to "original", a tarball is generated with the unpacked sources of a given recipe. Prior to it, a tmpdir (i.e. from tempfile.mkdtemp()) is created and the sources are unpacked inside:

(...)/vim/9.2.0340/archiver-work/tmp8pu2nsck
`-- vim-9.2.0340

     |-- READMEdir
     |-- ci
     |-- lang

This tmpdir, however, ends up being shipped in the deployed tarball:

$ tar -tf vim-9.2.0340-r0.tar.xz
(...)
tmp8pu2nsck/vim-9.2.0340/ci/config.mk.clang-12.sed
tmp8pu2nsck/vim-9.2.0340/LICENSE
tmp8pu2nsck/vim-9.2.0340/.git-blame-ignore-revs
(...)

This can be avoided by pointing to the subdir where the sources are unpacked, rather than to tmpdir, when calling create_tarball().

The subdir's name can be inferred from 'BP', so we will use 'tmp8pu2nsck/vim-9.2.0340/.' instead of 'tmp8pu2nsck/.' as input.

e.g. and in the result:

./
./vim-9.2.0304/
./vim-9.2.0304/SECURITY.md
./vim-9.2.0304/.codecov.yml
(...)

---
  meta/classes/archiver.bbclass | 2 +-
  1 file changed, 1 insertion(+), 1 deletion(-)
meta/classes/archiver.bbclass +0 -0
index b274c53e79..c824718a1d 100644

meta/classes/archiver.bbclass python do_ar_original() { · line 236

Quoted 8 lines
meta/classes/archiver.bbclass +1 -1
@@ -230,7 +230,7 @@ python do_ar_original() {
                      else:
                          bb.fatal("Cannot determine archive names for original source because 'name=' URL parameter '%s' is used twice. Make it unique in: %s %s" % (tarball_suffix[name], url))
              tarball_suffix[name] = url
-            create_tarball(d, tmpdir + '/.', name, ar_outdir)
+            create_tarball(d, os.path.join(tmpdir, d.getVar('BP'), '.'), name, ar_outdir)
 
      # Emit patch series files for 'original'
      bb.note('Writing patch series files...')

Gentle ping: would there be any updates on this one? Thanks!

Signature · 2 lines
Best regards,
João Marcos Costa
#2 lore raw
Quoted 52 lines · Joao Marcos Costa

Hello,

Quoted 49 lines · João Marcos Costa

When ARCHIVER_MODE[src] is set to "original", a tarball is generated with the unpacked sources of a given recipe. Prior to it, a tmpdir (i.e. from tempfile.mkdtemp()) is created and the sources are unpacked inside:

(...)/vim/9.2.0340/archiver-work/tmp8pu2nsck
`-- vim-9.2.0340

     |-- READMEdir
     |-- ci
     |-- lang

This tmpdir, however, ends up being shipped in the deployed tarball:

$ tar -tf vim-9.2.0340-r0.tar.xz
(...)
tmp8pu2nsck/vim-9.2.0340/ci/config.mk.clang-12.sed
tmp8pu2nsck/vim-9.2.0340/LICENSE
tmp8pu2nsck/vim-9.2.0340/.git-blame-ignore-revs
(...)

This can be avoided by pointing to the subdir where the sources are unpacked, rather than to tmpdir, when calling create_tarball().

The subdir's name can be inferred from 'BP', so we will use 'tmp8pu2nsck/vim-9.2.0340/.' instead of 'tmp8pu2nsck/.' as input.

e.g. and in the result:

./
./vim-9.2.0304/
./vim-9.2.0304/SECURITY.md
./vim-9.2.0304/.codecov.yml
(...)

---
  meta/classes/archiver.bbclass | 2 +-
  1 file changed, 1 insertion(+), 1 deletion(-)
meta/classes/archiver.bbclass +1 -1
index b274c53e79..c824718a1d 100644
@@ -230,7 +230,7 @@ python do_ar_original() {
                      else:
                          bb.fatal("Cannot determine archive names for original source because 'name=' URL parameter '%s' is used twice. Make it unique in: %s %s" % (tarball_suffix[name], url))
              tarball_suffix[name] = url
-            create_tarball(d, tmpdir + '/.', name, ar_outdir)
+            create_tarball(d, os.path.join(tmpdir, d.getVar('BP'), '.'), name, ar_outdir)
 
Quoted 4 lines

meta/classes/archiver.bbclass python do_ar_original() { · line 236

Quoted 2 lines
meta/classes/archiver.bbclass +0 -0
      # Emit patch series files for 'original'
      bb.note('Writing patch series files...')

Gentle ping: would there be any updates on this one? Thanks!

Hi João,

In my reply to the v1 patch [1], I asked why we're creating a tmpdir under ${ARCHIVER_WORKDIR}, why not just unpack the source directly under the archiver workdir?

The archiver already works in an archiver-work subdirectory, so there isn't any risk of the unpacked sources clashing with ${S} used by other recipe tasks. We may be able to change the code so it doesn't use an additional directory layer.

[1]: https://lore.kernel.org/all/[email protected]/

Best regards,

Signature · 1 line
Paul Barker
#3 lore raw

Hello, Paul

Thanks for your reply.

Quoted 64 lines · Paul Barker
Quoted 52 lines · Joao Marcos Costa

Hello,

Quoted 49 lines · João Marcos Costa

When ARCHIVER_MODE[src] is set to "original", a tarball is generated with the unpacked sources of a given recipe. Prior to it, a tmpdir (i.e. from tempfile.mkdtemp()) is created and the sources are unpacked inside:

(...)/vim/9.2.0340/archiver-work/tmp8pu2nsck
`-- vim-9.2.0340

      |-- READMEdir
      |-- ci
      |-- lang

This tmpdir, however, ends up being shipped in the deployed tarball:

$ tar -tf vim-9.2.0340-r0.tar.xz
(...)
tmp8pu2nsck/vim-9.2.0340/ci/config.mk.clang-12.sed
tmp8pu2nsck/vim-9.2.0340/LICENSE
tmp8pu2nsck/vim-9.2.0340/.git-blame-ignore-revs
(...)

This can be avoided by pointing to the subdir where the sources are unpacked, rather than to tmpdir, when calling create_tarball().

The subdir's name can be inferred from 'BP', so we will use 'tmp8pu2nsck/vim-9.2.0340/.' instead of 'tmp8pu2nsck/.' as input.

e.g. and in the result:

./
./vim-9.2.0304/
./vim-9.2.0304/SECURITY.md
./vim-9.2.0304/.codecov.yml
(...)

---
   meta/classes/archiver.bbclass | 2 +-
   1 file changed, 1 insertion(+), 1 deletion(-)
meta/classes/archiver.bbclass +1 -1
index b274c53e79..c824718a1d 100644
@@ -230,7 +230,7 @@ python do_ar_original() {
                       else:
                           bb.fatal("Cannot determine archive names for original source because 'name=' URL parameter '%s' is used twice. Make it unique in: %s %s" % (tarball_suffix[name], url))
               tarball_suffix[name] = url
-            create_tarball(d, tmpdir + '/.', name, ar_outdir)
+            create_tarball(d, os.path.join(tmpdir, d.getVar('BP'), '.'), name, ar_outdir)
 
Quoted 4 lines

meta/classes/archiver.bbclass python do_ar_original() { · line 236

Quoted 2 lines
meta/classes/archiver.bbclass +0 -0
       # Emit patch series files for 'original'
       bb.note('Writing patch series files...')

Gentle ping: would there be any updates on this one? Thanks!

Hi João,

In my reply to the v1 patch [1], I asked why we're creating a tmpdir under ${ARCHIVER_WORKDIR}, why not just unpack the source directly under the archiver workdir?

Quoted 8 lines

The archiver already works in an archiver-work subdirectory, so there isn't any risk of the unpacked sources clashing with ${S} used by other recipe tasks. We may be able to change the code so it doesn't use an additional directory layer.

[1]: https://lore.kernel.org/all/[email protected]/

Best regards,

There could be something off with my mailer, Idk, but I replied to your comment(s) back then:

"I do believe that tmpdir was created to handle the case where multiple source controlled URLs are used in SRC_URI. If I fake a second git for Vim, the result will be a couple of tmpdirs, then 2 separate tarballs, each ending with the suffix defined in SRCREV_FORMAT."

So I believe the tmpdir is needed, and the cleaner way to handle this is to change the call site in do_ar_original() as you suggested, thus my v2. What do you think?

Signature · 2 lines
Best regards,
João Marcos Costa
#4 lore raw
Quoted 81 lines · Joao Marcos Costa

Hello, Paul

Thanks for your reply.

Quoted 64 lines · Paul Barker
Quoted 52 lines · Joao Marcos Costa

Hello,

Quoted 49 lines · João Marcos Costa

When ARCHIVER_MODE[src] is set to "original", a tarball is generated with the unpacked sources of a given recipe. Prior to it, a tmpdir (i.e. from tempfile.mkdtemp()) is created and the sources are unpacked inside:

(...)/vim/9.2.0340/archiver-work/tmp8pu2nsck
`-- vim-9.2.0340

      |-- READMEdir
      |-- ci
      |-- lang

This tmpdir, however, ends up being shipped in the deployed tarball:

$ tar -tf vim-9.2.0340-r0.tar.xz
(...)
tmp8pu2nsck/vim-9.2.0340/ci/config.mk.clang-12.sed
tmp8pu2nsck/vim-9.2.0340/LICENSE
tmp8pu2nsck/vim-9.2.0340/.git-blame-ignore-revs
(...)

This can be avoided by pointing to the subdir where the sources are unpacked, rather than to tmpdir, when calling create_tarball().

The subdir's name can be inferred from 'BP', so we will use 'tmp8pu2nsck/vim-9.2.0340/.' instead of 'tmp8pu2nsck/.' as input.

e.g. and in the result:

./
./vim-9.2.0304/
./vim-9.2.0304/SECURITY.md
./vim-9.2.0304/.codecov.yml
(...)

---
   meta/classes/archiver.bbclass | 2 +-
   1 file changed, 1 insertion(+), 1 deletion(-)
meta/classes/archiver.bbclass +1 -1
index b274c53e79..c824718a1d 100644
@@ -230,7 +230,7 @@ python do_ar_original() {
                       else:
                           bb.fatal("Cannot determine archive names for original source because 'name=' URL parameter '%s' is used twice. Make it unique in: %s %s" % (tarball_suffix[name], url))
               tarball_suffix[name] = url
-            create_tarball(d, tmpdir + '/.', name, ar_outdir)
+            create_tarball(d, os.path.join(tmpdir, d.getVar('BP'), '.'), name, ar_outdir)
 
Quoted 4 lines

meta/classes/archiver.bbclass python do_ar_original() { · line 236

Quoted 2 lines
meta/classes/archiver.bbclass +0 -0
       # Emit patch series files for 'original'
       bb.note('Writing patch series files...')

Gentle ping: would there be any updates on this one? Thanks!

Hi João,

In my reply to the v1 patch [1], I asked why we're creating a tmpdir under ${ARCHIVER_WORKDIR}, why not just unpack the source directly under the archiver workdir?

Quoted 8 lines

The archiver already works in an archiver-work subdirectory, so there isn't any risk of the unpacked sources clashing with ${S} used by other recipe tasks. We may be able to change the code so it doesn't use an additional directory layer.

[1]: https://lore.kernel.org/all/[email protected]/

Best regards,

There could be something off with my mailer, Idk, but I replied to your comment(s) back then:

Quoted 8 lines

"I do believe that tmpdir was created to handle the case where multiple source controlled URLs are used in SRC_URI. If I fake a second git for Vim, the result will be a couple of tmpdirs, then 2 separate tarballs, each ending with the suffix defined in SRCREV_FORMAT."

So I believe the tmpdir is needed, and the cleaner way to handle this is to change the call site in do_ar_original() as you suggested, thus my v2. What do you think?

Hi João,

This does explain the tmpdir a little more. In which case v2 is probably good, I'll see if we can move this forward in the next patch review call.

Thanks,

Signature · 1 line
Paul Barker
#5 lore raw

Hello, Paul and Yoann

On 10/16/26 3:40 PM, Paul Barker wrote:
(...)

Quoted 8 lines

Hi João,

This does explain the tmpdir a little more. In which case v2 is probably good, I'll see if we can move this forward in the next patch review call.

Thanks,

While I see this as a bugfix, some might see as an "improvement". What do you think? Should this fix be backported to blacksail/wrynose/scarthgap? If so, let me know and I'll prepare the patches.

Signature · 2 lines
Best regards,
João Marcos Costa