Skip to contentSkip to conversation

systemtap: 3.3 -> 4.0

PATCH In review Victor Kamensky 5 participants 13 messages Oct 15, 2018 – Oct 26, 2018

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 systemtap: 3.3 -> 4.0

Upgrade systemtap from 3.3 to 4.0: Removed backported patch.

Very short summary of major changes from SystemTap 4.0 announcement by Frank Ch. Eigler <[email protected]>:

Quoted 5 lines

prometheus exporter network service; ebpf support extensions including strings and implementation of traditional log(), sprintf() functions; rebuilt rich tapset coverage for 4.17+ syscalls and for tracepoint-based syscalls; script language tweaks for supporting machine-generated scripts

Fixes [YOCTO #12950]

---
Hi Guys,

I'll let you to decide whether you want to accept it for 2.6 at this point or push it for the next release. On one hand after move to 4.18 kernel SystemTap 3.3 fails to trace any system call on x86_64 - it is major functionality breakage. Look at:

https://bugzilla.redhat.com/show_bug.cgi?id=1596456

On other hand it is quite late to do major version updates.

I did go through limitted set of tests on all
supported QEMU targets. As far as these tests
concerned proposed 4.0 matches 3.3 results.

Thanks,
Victor

 ...entrypc-avoid-usage-of-uninitialized.patch | 46 -------------------
 .../systemtap/systemtap_git.inc               |  5 +-
 2 files changed, 2 insertions(+), 49 deletions(-)
 delete mode 100644 meta/recipes-kernel/systemtap/systemtap/0001-dwflpp-function_entrypc-avoid-usage-of-uninitialized.patch
meta/recipes-kernel/systemtap/systemtap/0001-dwflpp-function_entrypc-avoid-usage-of-uninitialized.patch deleted +0 -46
deleted file mode 100644
index d0082a1094..0000000000
@@ -1,46 +0,0 @@
-From 8466fca2a074323a235ef38d425f994a2ff7e64f Mon Sep 17 00:00:00 2001
-From: Victor Kamensky <[email protected]>
-Date: Mon, 9 Jul 2018 09:31:19 -0700
-Subject: [PATCH] dwflpp::function_entrypc avoid usage of uninitialized memory
-
-Failure on 3.3 release was observed. Failure was elusive and
-disappeared after seemingly random configure option change, or when
-code was compiled with -O1 or -O0 (vs default -O2). Running failing
-test case under valgrind memcheck pointed to couple places where
-'Conditional jump or move depends on uninitialised value(s)' occured.
-
-After addressing these in two places in dwflpp::function_entrypc,
-valgrind memcheck run is clean and original issue got fixed.
-
-Upstream-Status: Backport
-Signed-off-by: Victor Kamensky <[email protected]>
----
- dwflpp.cxx | 6 +++++-
- 1 file changed, 5 insertions(+), 1 deletion(-)
-
-diff --git a/dwflpp.cxx b/dwflpp.cxx
-index bfbb6b096..2172e705a 100644
---- a/dwflpp.cxx
-+++ b/dwflpp.cxx
-@@ -2465,13 +2465,17 @@ bool
- dwflpp::function_entrypc (Dwarf_Addr * addr)
- {
-   assert (function);
-+
-+  // assign default value
-+  *addr = 0;
-+
-   // PR10574: reject 0, which tends to be eliminated COMDAT
-   if (dwarf_entrypc (function, addr) == 0 && *addr != 0)
-     return true;
- 
-   /* Assume the entry pc is the base address, or (if zero)
-      the first address of the ranges covering this DIE.  */
--  Dwarf_Addr start, end;
-+  Dwarf_Addr start = 0, end;
-   if (dwarf_ranges (function, 0, addr, &start, &end) >= 0)
-     {
-       if (*addr == 0)
--- 
-2.17.1
-
meta/recipes-kernel/systemtap/systemtap_git.inc +2 -3
index 06924fc240..274fcde5c1 100644
@@ -1,7 +1,7 @@
 LICENSE = "GPLv2"
 LIC_FILES_CHKSUM = "file://COPYING;md5=b234ee4d69f5fce4486a80fdaf4a4263"
-SRCREV = "48867d1cface9445d58e3c6e14497770b7eb77e1"
-PV = "3.3"
+SRCREV = "428f84e9e656bce71018e8902e4edb8aacafcc0e"
+PV = "4.0"
 
 SRC_URI = "git://sourceware.org/git/systemtap.git \
            file://configure-allow-to-disable-libvirt.patch \
@@ -11,7 +11,6 @@ SRC_URI = "git://sourceware.org/git/systemtap.git \
            file://0001-Do-not-let-configure-write-a-python-location-into-th.patch \
            file://0001-Install-python-modules-to-correct-library-dir.patch \
            file://0001-staprun-stapbpf-don-t-support-installing-a-non-root.patch \
-           file://0001-dwflpp-function_entrypc-avoid-usage-of-uninitialized.patch \
            "
 
 COMPATIBLE_HOST = '(x86_64|i.86|powerpc|arm|aarch64|microblazeel|mips).*-linux'
Signature · 1 line
2.17.1
#1 lore raw
Quoted 17 lines · Victor Kamensky

Upgrade systemtap from 3.3 to 4.0: Removed backported patch.

Very short summary of major changes from SystemTap 4.0 announcement by Frank Ch. Eigler <[email protected]>:

Quoted 5 lines

prometheus exporter network service; ebpf support extensions including strings and implementation of traditional log(), sprintf() functions; rebuilt rich tapset coverage for 4.17+ syscalls and for tracepoint-based syscalls; script language tweaks for supporting machine-generated scripts

Fixes [YOCTO #12950]

---
Hi Guys,

Quoted 7 lines

I'll let you to decide whether you want to accept it for 2.6 at this point or push it for the next release. On one hand after move to 4.18 kernel SystemTap 3.3 fails to trace any system call on x86_64 - it is major functionality breakage. Look at:

https://bugzilla.redhat.com/show_bug.cgi?id=1596456

Ouch!

Quoted 3 lines

On other hand it is quite late to do major version updates.

It is but nothing depends on systemtap so I think we don't have much choice and we should do it unless there are autobuilder tests that regress.

A quick read through the YP mega-manual suggests that we don't have to change any of the text. Victor,
Could you take a look and confirm that please?

I noticed that the link in section:
20.3.3. Documentation
for:
http://sourceware.org/systemtap/langref/
is dead. There is a pdf:
http://sourceware.org/systemtap/langref.pdf
Perhaps this is temporary due to the recent release.

Quoted 7 lines

I did go through limited set of tests on all
supported QEMU targets. As far as these tests
concerned proposed 4.0 matches 3.3 results.

Thanks,
Victor

Thanks Victor.

Talk about hot off the presses, 4.0 was released at:
13-Oct-2018 18:54

$ git log --oneline release-3.3..release-4.0 | wc -l
252
$ git diff release-3.3..release-4.0 | diffstat | tail -1
 1574 files changed, 50756 insertions(+), 23604 deletions(-)

Lots of changes to docs, testsuite but also quite a bit to core code and tapsets:

$ git diff release-3.3..release-4.0 | diffstat  | cut -d"/" -f-2 | uniq -c |grep -v "   [1-5] "
    770  b/doc
     16  b/httpd
      8  b/man
      9  b/po
     18  b/runtime
     15  b/stap-exporter
      6  b/staprun
    440  b/tapset
    223  b/testsuite

../Randy

Quoted 81 lines
  ...entrypc-avoid-usage-of-uninitialized.patch | 46 -------------------
  .../systemtap/systemtap_git.inc               |  5 +-
  2 files changed, 2 insertions(+), 49 deletions(-)

delete mode 100644 meta/recipes-kernel/systemtap/systemtap/0001-dwflpp-function_entrypc-avoid-usage-of-uninitialized.patch

meta/recipes-kernel/systemtap/systemtap/0001-dwflpp-function_entrypc-avoid-usage-of-uninitialized.patch deleted +0 -46
deleted file mode 100644
index d0082a1094..0000000000
@@ -1,46 +0,0 @@
-From 8466fca2a074323a235ef38d425f994a2ff7e64f Mon Sep 17 00:00:00 2001
-From: Victor Kamensky <[email protected]>
-Date: Mon, 9 Jul 2018 09:31:19 -0700
-Subject: [PATCH] dwflpp::function_entrypc avoid usage of uninitialized memory
-
-Failure on 3.3 release was observed. Failure was elusive and
-disappeared after seemingly random configure option change, or when
-code was compiled with -O1 or -O0 (vs default -O2). Running failing
-test case under valgrind memcheck pointed to couple places where
-'Conditional jump or move depends on uninitialised value(s)' occured.
-
-After addressing these in two places in dwflpp::function_entrypc,
-valgrind memcheck run is clean and original issue got fixed.
-
-Upstream-Status: Backport
-Signed-off-by: Victor Kamensky <[email protected]>
----
- dwflpp.cxx | 6 +++++-
- 1 file changed, 5 insertions(+), 1 deletion(-)
-
-diff --git a/dwflpp.cxx b/dwflpp.cxx
-index bfbb6b096..2172e705a 100644
---- a/dwflpp.cxx
-+++ b/dwflpp.cxx
-@@ -2465,13 +2465,17 @@ bool
- dwflpp::function_entrypc (Dwarf_Addr * addr)
- {
-   assert (function);
-+
-+  // assign default value
-+  *addr = 0;
-+
-   // PR10574: reject 0, which tends to be eliminated COMDAT
-   if (dwarf_entrypc (function, addr) == 0 && *addr != 0)
-     return true;
-
-   /* Assume the entry pc is the base address, or (if zero)
-      the first address of the ranges covering this DIE.  */
--  Dwarf_Addr start, end;
-+  Dwarf_Addr start = 0, end;
-   if (dwarf_ranges (function, 0, addr, &start, &end) >= 0)
-     {
-       if (*addr == 0)
---
-2.17.1
-
meta/recipes-kernel/systemtap/systemtap_git.inc +2 -3
index 06924fc240..274fcde5c1 100644
@@ -1,7 +1,7 @@
  LICENSE = "GPLv2"
  LIC_FILES_CHKSUM = "file://COPYING;md5=b234ee4d69f5fce4486a80fdaf4a4263"
-SRCREV = "48867d1cface9445d58e3c6e14497770b7eb77e1"
-PV = "3.3"
+SRCREV = "428f84e9e656bce71018e8902e4edb8aacafcc0e"
+PV = "4.0"
 
  SRC_URI = "git://sourceware.org/git/systemtap.git \
             file://configure-allow-to-disable-libvirt.patch \
@@ -11,7 +11,6 @@ SRC_URI = "git://sourceware.org/git/systemtap.git \
             file://0001-Do-not-let-configure-write-a-python-location-into-th.patch \
             file://0001-Install-python-modules-to-correct-library-dir.patch \
             file://0001-staprun-stapbpf-don-t-support-installing-a-non-root.patch \
-           file://0001-dwflpp-function_entrypc-avoid-usage-of-uninitialized.patch \
             "
 
  COMPATIBLE_HOST = '(x86_64|i.86|powerpc|arm|aarch64|microblazeel|mips).*-linux'
Signature · 2 lines
# Randy MacLeod
# Wind River Linux
#2 lore raw

Hi Randy,

Quoted 29 lines · Randy MacLeod
Quoted 17 lines · Victor Kamensky

Upgrade systemtap from 3.3 to 4.0: Removed backported patch.

Very short summary of major changes from SystemTap 4.0 announcement by Frank Ch. Eigler <[email protected]>:

Quoted 5 lines

prometheus exporter network service; ebpf support extensions including strings and implementation of traditional log(), sprintf() functions; rebuilt rich tapset coverage for 4.17+ syscalls and for tracepoint-based syscalls; script language tweaks for supporting machine-generated scripts

Fixes [YOCTO #12950]

---
Hi Guys,

Quoted 7 lines

I'll let you to decide whether you want to accept it for 2.6 at this point or push it for the next release. On one hand after move to 4.18 kernel SystemTap 3.3 fails to trace any system call on x86_64 - it is major functionality breakage. Look at:

https://bugzilla.redhat.com/show_bug.cgi?id=1596456

Ouch!

Quoted 1 line
Quoted 6 lines
Quoted 2 lines

On other hand it is quite late to do major version updates.

It is but nothing depends on systemtap so I think we don't have much choice and we should do it unless there are autobuilder tests that regress.

Personally, I agree. I had similar thoughts, since it is leaf project risk is quite contained.

Quoted 6 lines

A quick read through the YP mega-manual suggests that we don't have to change any of the text. Victor,
Could you take a look and confirm that please?

Quoted 7 lines

I noticed that the link in section:
20.3.3. Documentation
for:
http://sourceware.org/systemtap/langref/
is dead. There is a pdf:
http://sourceware.org/systemtap/langref.pdf
Perhaps this is temporary due to the recent release.

Yes, I agree. It is better to change link to langref.pdf as you suggested. But I don't know how to do that. Is there somewhere git that holds YP mega-manual sources?

Quoted 5 lines
Quoted 5 lines

I did go through limited set of tests on all
supported QEMU targets. As far as these tests
concerned proposed 4.0 matches 3.3 results.

Quoted 7 lines
Quoted 2 lines

Thanks,
Victor

Thanks Victor.

Talk about hot off the presses, 4.0 was released at:
13-Oct-2018 18:54

Yes :), I was looking how to fix [YOCTO #12950] and was about to write email to SystemTap guys asking when the next release is planned and got email from Frank on SystemTap mailing list announcing 4.0 release. It was very good timing.

My understanding is that FC28 and FC27 will get SystemTap 4.0 very soon since they moved to 4.18 kernel too.

Thanks,
Victor

Quoted 116 lines
$ git log --oneline release-3.3..release-4.0 | wc -l
252
$ git diff release-3.3..release-4.0 | diffstat | tail -1
1574 files changed, 50756 insertions(+), 23604 deletions(-)

Lots of changes to docs, testsuite but also quite a bit to core code and tapsets:

$ git diff release-3.3..release-4.0 | diffstat  | cut -d"/" -f-2 | uniq -c |grep -v "   [1-5] "
   770  b/doc
    16  b/httpd
     8  b/man
     9  b/po
    18  b/runtime
    15  b/stap-exporter
     6  b/staprun
   440  b/tapset
   223  b/testsuite

../Randy

Quoted 87 lines
  ...entrypc-avoid-usage-of-uninitialized.patch | 46 -------------------
  .../systemtap/systemtap_git.inc               |  5 +-
  2 files changed, 2 insertions(+), 49 deletions(-)

delete mode 100644
meta/recipes-kernel/systemtap/systemtap/0001-dwflpp-function_entrypc-avoid-usage-of-uninitialized.patch

meta/recipes-kernel/systemtap/systemtap/0001-dwflpp-function_entrypc-avoid-usage-of-uninitialized.patch deleted +0 -1
deleted file mode 100644
index d0082a1094..0000000000
--- 

a/meta/recipes-kernel/systemtap/systemtap/0001-dwflpp-function_entrypc-avoid-usage-of-uninitialized.patch +++ /dev/null

meta/recipes-kernel/systemtap/systemtap/0001-dwflpp-function_entrypc-avoid-usage-of-uninitialized.patch deleted +0 -4
@@ -1,46 +0,0 @@
-From 8466fca2a074323a235ef38d425f994a2ff7e64f Mon Sep 17 00:00:00 2001
-From: Victor Kamensky <[email protected]>
-Date: Mon, 9 Jul 2018 09:31:19 -0700
-Subject: [PATCH] dwflpp::function_entrypc avoid usage of uninitialized 

memory

meta/recipes-kernel/systemtap/systemtap/0001-dwflpp-function_entrypc-avoid-usage-of-uninitialized.patch deleted +0 -42
-
-Failure on 3.3 release was observed. Failure was elusive and
-disappeared after seemingly random configure option change, or when
-code was compiled with -O1 or -O0 (vs default -O2). Running failing
-test case under valgrind memcheck pointed to couple places where
-'Conditional jump or move depends on uninitialised value(s)' occured.
-
-After addressing these in two places in dwflpp::function_entrypc,
-valgrind memcheck run is clean and original issue got fixed.
-
-Upstream-Status: Backport
-Signed-off-by: Victor Kamensky <[email protected]>
----
- dwflpp.cxx | 6 +++++-
- 1 file changed, 5 insertions(+), 1 deletion(-)
-
-diff --git a/dwflpp.cxx b/dwflpp.cxx
-index bfbb6b096..2172e705a 100644
---- a/dwflpp.cxx
-+++ b/dwflpp.cxx
-@@ -2465,13 +2465,17 @@ bool
- dwflpp::function_entrypc (Dwarf_Addr * addr)
- {
-   assert (function);
-+
-+  // assign default value
-+  *addr = 0;
-+
-   // PR10574: reject 0, which tends to be eliminated COMDAT
-   if (dwarf_entrypc (function, addr) == 0 && *addr != 0)
-     return true;
-
-   /* Assume the entry pc is the base address, or (if zero)
-      the first address of the ranges covering this DIE.  */
--  Dwarf_Addr start, end;
-+  Dwarf_Addr start = 0, end;
-   if (dwarf_ranges (function, 0, addr, &start, &end) >= 0)
-     {
-       if (*addr == 0)
---
-2.17.1
-
meta/recipes-kernel/systemtap/systemtap_git.inc +2 -3
index 06924fc240..274fcde5c1 100644
@@ -1,7 +1,7 @@
  LICENSE = "GPLv2"
  LIC_FILES_CHKSUM = "file://COPYING;md5=b234ee4d69f5fce4486a80fdaf4a4263"
-SRCREV = "48867d1cface9445d58e3c6e14497770b7eb77e1"
-PV = "3.3"
+SRCREV = "428f84e9e656bce71018e8902e4edb8aacafcc0e"
+PV = "4.0"
    SRC_URI = "git://sourceware.org/git/systemtap.git \
             file://configure-allow-to-disable-libvirt.patch \
@@ -11,7 +11,6 @@ SRC_URI = "git://sourceware.org/git/systemtap.git \
             file://0001-Do-not-let-configure-write-a-python-location-into-th.patch 
 \
             file://0001-Install-python-modules-to-correct-library-dir.patch 
 \
             file://0001-staprun-stapbpf-don-t-support-installing-a-non-root.patch 
 \
- 
file://0001-dwflpp-function_entrypc-avoid-usage-of-uninitialized.patch \
             "
    COMPATIBLE_HOST = 

'(x86_64|i.86|powerpc|arm|aarch64|microblazeel|mips).*-linux'

Signature · 2 lines
# Randy MacLeod
# Wind River Linux
#3 lore raw
In reply to Victor Kamensky
Quoted 1 line · Victor Kamensky

Upgrade systemtap from 3.3 to 4.0: Removed backported patch.

WARNING: systemtap-4.0-r0 do_package_qa: QA Issue: systemtap:
/systemtap/etc/stap-exporter/EXAMPLE is owned by uid 1000, which is the same as the user running bitbake. This may be due to host contamination
systemtap: /systemtap/etc/stap-exporter/EXAMPLE_NOERROR is owned by uid 1000, which is the same as the user running bitbake. This may be due to host contamination
systemtap: /systemtap/etc/stap-exporter/errno.stp is owned by uid 1000, which is the same as the user running bitbake. This may be due to host contamination
systemtap: /systemtap/etc/stap-exporter/syscalllatency.stp is owned by uid 1000, which is the same as the user running bitbake. This may be due to host contamination
systemtap: /systemtap/etc/stap-exporter/timeout.stp is owned by uid 1000, which is the same as the user running bitbake. This may be due to host contamination
systemtap: /systemtap/etc/stap-exporter/topsys.stp is owned by uid 1000, which is the same as the user running bitbake. This may be due to host contamination [host-user-contaminated]

#4 lore raw
Quoted 16 lines · Ross
Quoted 1 line · Victor Kamensky

Upgrade systemtap from 3.3 to 4.0: Removed backported patch.

WARNING: systemtap-4.0-r0 do_package_qa: QA Issue: systemtap:
/systemtap/etc/stap-exporter/EXAMPLE is owned by uid 1000, which is the same as the user running bitbake. This may be due to host contamination
systemtap: /systemtap/etc/stap-exporter/EXAMPLE_NOERROR is owned by uid 1000, which is the same as the user running bitbake. This may be due to host contamination
systemtap: /systemtap/etc/stap-exporter/errno.stp is owned by uid 1000, which is the same as the user running bitbake. This may be due to host contamination
systemtap: /systemtap/etc/stap-exporter/syscalllatency.stp is owned by

Quoted 8 lines

uid 1000, which is the same as the user running bitbake. This may be due to host contamination
systemtap: /systemtap/etc/stap-exporter/timeout.stp is owned by uid 1000, which is the same as the user running bitbake. This may be due to host contamination
systemtap: /systemtap/etc/stap-exporter/topsys.stp is owned by uid 1000, which is the same as the user running bitbake. This may be due to host contamination [host-user-contaminated]

There are also other issues from the addition of systemd unit files in 4.0. I've a patch in -next which tweaks both issues which I'll put in for testing.

Cheers,

Richard

#5 lore raw
Quoted 26 lines · Richard Purdie
Quoted 16 lines · Ross
Quoted 1 line · Victor Kamensky

Upgrade systemtap from 3.3 to 4.0: Removed backported patch.

WARNING: systemtap-4.0-r0 do_package_qa: QA Issue: systemtap:
/systemtap/etc/stap-exporter/EXAMPLE is owned by uid 1000, which is the same as the user running bitbake. This may be due to host contamination
systemtap: /systemtap/etc/stap-exporter/EXAMPLE_NOERROR is owned by uid 1000, which is the same as the user running bitbake. This may be due to host contamination
systemtap: /systemtap/etc/stap-exporter/errno.stp is owned by uid 1000, which is the same as the user running bitbake. This may be due to host contamination
systemtap: /systemtap/etc/stap-exporter/syscalllatency.stp is owned by

Quoted 8 lines

uid 1000, which is the same as the user running bitbake. This may be due to host contamination
systemtap: /systemtap/etc/stap-exporter/timeout.stp is owned by uid 1000, which is the same as the user running bitbake. This may be due to host contamination
systemtap: /systemtap/etc/stap-exporter/topsys.stp is owned by uid 1000, which is the same as the user running bitbake. This may be due to host contamination [host-user-contaminated]

Quoted 3 lines

There are also other issues from the addition of systemd unit files in 4.0. I've a patch in -next which tweaks both issues which I'll put in for testing.

Richard, thank you very prompt help.

In the long term it would be beneficial that those issues be fixed in SystemTap itself, correct? If so, I can look at creating proper patches and pushing them into SystemTap.

Also on my todo list is to attempt to push few existing OE systemtap patches into SystemTap upstream source. I believe they look like quite legit issues and I hope they could be upstreamed.

Thanks,
Victor

Quoted 5 lines

Cheers,

Richard

#6 lore raw
Quoted 3 lines · Victor Kamensky

In the long term it would be beneficial that those issues be fixed in SystemTap itself, correct? If so, I can look at creating proper patches and pushing them into SystemTap.

Yes, fixing those upstream would be very helpful, probably through adding the ability to configure the systemd unit path and fixing the file installation method.

Quoted 4 lines

Also on my todo list is to attempt to push few existing OE systemtap patches into SystemTap upstream source. I believe they look like quite legit issues and I hope they could be upstreamed.

That kind of work is very much appreciated and makes everyone's life easier in the future!

Cheers,

Richard

#7 lore raw

There seems to be one more issue in current master (for which I haven't seen a fix in master-next or ML):

ERROR: systemtap-4.0-r0 do_package: QA Issue: systemtap:
Files/directories were installed but not shipped in any package:
/lib
/lib/systemd
/lib/systemd/system /lib/systemd/system/stap-exporter.service
Please set FILES such that these items are packaged. Alternatively if they are unneeded, avoid installing them or delete them within do_install.
systemtap: 4 installed and not shipped files. [installed-vs-shipped]

Quoted 27 lines · On Wed, Oct 17, 2018 at 1:59 PM <[email protected]> wrote:
Quoted 3 lines · Victor Kamensky

In the long term it would be beneficial that those issues be fixed in SystemTap itself, correct? If so, I can look at creating proper patches and pushing them into SystemTap.

Yes, fixing those upstream would be very helpful, probably through adding the ability to configure the systemd unit path and fixing the file installation method.

Quoted 4 lines

Also on my todo list is to attempt to push few existing OE systemtap patches into SystemTap upstream source. I believe they look like quite legit issues and I hope they could be upstreamed.

That kind of work is very much appreciated and makes everyone's life easier in the future!

Cheers,

Richard

--
_______________________________________________
Openembedded-core mailing list
[email protected]
http://lists.openembedded.org/mailman/listinfo/openembedded-core

#8 lore raw
Quoted 13 lines · Martin Jansa

There seems to be one more issue in current master (for which I haven't seen a fix in master-next or ML):

ERROR: systemtap-4.0-r0 do_package: QA Issue: systemtap:
Files/directories were installed but not shipped in any package:
/lib
/lib/systemd
/lib/systemd/system /lib/systemd/system/stap-exporter.service
Please set FILES such that these items are packaged. Alternatively if they are unneeded, avoid installing them or delete them within do_install.
systemtap: 4 installed and not shipped files. [installed-vs-shipped]

http://git.yoctoproject.org/cgit.cgi/poky/commit/?id=3b77e7b7852549dcfbc426d4ce258e6e857c0acd
was supposed to fix that :(

Cheers,

Richard

#9 lore raw

systemd.bbclass won't include that .service file in FILES_<pkg> without setting SYSTEMD_SERVICE_<pkg>.

I'll send fix for that in few minutes.

Quoted 24 lines · On Wed, Oct 17, 2018 at 5:07 PM <[email protected]> wrote:
Quoted 13 lines · Martin Jansa

There seems to be one more issue in current master (for which I haven't seen a fix in master-next or ML):

ERROR: systemtap-4.0-r0 do_package: QA Issue: systemtap:
Files/directories were installed but not shipped in any package:
/lib
/lib/systemd
/lib/systemd/system /lib/systemd/system/stap-exporter.service
Please set FILES such that these items are packaged. Alternatively if they are unneeded, avoid installing them or delete them within do_install.
systemtap: 4 installed and not shipped files. [installed-vs-shipped]

http://git.yoctoproject.org/cgit.cgi/poky/commit/?id=3b77e7b7852549dcfbc426d4ce258e6e857c0acd
was supposed to fix that :(

Cheers,

Richard

#10 lore raw

PATCH 1/1 systemtap: fix QA issue

ERROR: systemtap-4.0-r0 do_package: QA Issue: systemtap:
Files/directories were installed but not shipped in any package:
/lib
/lib/systemd
/lib/systemd/system /lib/systemd/system/stap-exporter.service
Please set FILES such that these items are packaged. Alternatively if they are unneeded, avoid installing them or delete them within do_install.
systemtap: 4 installed and not shipped files. [installed-vs-shipped]

---
 meta/recipes-kernel/systemtap/systemtap_git.bb | 2 ++
 1 file changed, 2 insertions(+)
meta/recipes-kernel/systemtap/systemtap_git.bb +2 -0
index 3c45bfa937..88942a2362 100644
@@ -27,6 +27,8 @@ PACKAGECONFIG[python3-probes] = "--with-python3-probes,--without-python3-probes,
 
 inherit autotools gettext pkgconfig distutils3-base systemd
 
+SYSTEMD_SERVICE_${PN} = "stap-exporter.service"
+
 do_configure_prepend () {
     # Improve reproducibility for c++ object files
     reltivepath="${@os.path.relpath(d.getVar('STAGING_INCDIR'), d.getVar('S'))}"
Signature · 1 line
2.17.1
#11 lore raw
In reply to Martin Jansa
Quoted 2 lines · Martin Jansa

systemd.bbclass won't include that .service file in FILES_<pkg> without setting SYSTEMD_SERVICE_<pkg>. I'll send fix for that in few minutes.

Martin, Richard, thank you for the help. I've reproduced the issue reported by Martin in systemd enabled image, and verified that his fix solves the issue.

The lesson I learnt:

o Look for WARNINGs

o Test systemd enabled image, along with default sysvinit

stap-exporter service is a new feature in SystemTap 4.0, I've never used it before. I'll dig in and will try to make sure that it is functional in resulting OE image.

Thanks,
Victor

Quoted 25 lines · On Wed, Oct 17, 2018 at 5:07 PM <[email protected]> wrote:

On Wed, 2018-10-17 at 16:50 +0200, Martin Jansa wrote:
> There seems to be one more issue in current master (for which I > haven't seen a fix in master-next or ML):
>
> ERROR: systemtap-4.0-r0 do_package: QA Issue: systemtap:
> Files/directories were installed but not shipped in any package:
>   /lib
>   /lib/systemd
>   /lib/systemd/system
>   /lib/systemd/system/stap-exporter.service
> Please set FILES such that these items are packaged. Alternatively if > they are unneeded, avoid installing them or delete them within > do_install.
> systemtap: 4 installed and not shipped files. [installed-vs-shipped]

http://git.yoctoproject.org/cgit.cgi/poky/commit/?id=3b77e7b7852549dcfbc426d4ce258e6e857c0acd
was supposed to fix that :(

Cheers,

Richard

#12 lore raw

Hi Guys,

Quoted 3 lines · Victor Kamensky

stap-exporter service is a new feature in SystemTap 4.0, I've never used it before. I'll dig in and will try to make sure that it is functional in resulting OE image.

I've had a chance to look at stap-exporter and its service more closely.

It looks like it is simple HTTP server that allows to run stap scripts on target and relay its information to Prometheus monitoring system. Please check.

https://www.mankier.com/8/stap-exporter

https://prometheus.io

Since it is HTTP server, and its man page says that it is not supposed
to run in untrusted environment I am quite hesitant to have it enabled
by default. Moreover, it comes with some default example scripts that
actually fail to compile in OE core-image-lsb-sdk because it does not
include kernel-vmlinux package by default. stap-exporter is not
prepared to handle script compilation failure and tries to compile it
in the loop, so in core-image-lsb-sdk systemd image where stap-exporter.service would be enabled by default it would trash the system.

Considering all that I suggest stap-exporter to be moved in separate systemtap-exporter package, so it can be installed explcitely. I posted patch for it at

http://lists.openembedded.org/pipermail/openembedded-core/2018-October/157073.html

BTW in FC it is separate systemtap-exporter package.

I've tested systemtap-exporter and stap-exporter, if installed along with proper prerequisites (kernel-vmlinux) is functioning OK with given default example as far as my manual http testing with curl shows. I did not test it with prometheus.

Thanks,
Victor