Skip to contentSkip to conversation

multipath-tools: replace perl with sed in libdmmp/Makefile for pseudo

PATCH In review Geoff Parker 3 participants 4 messages Oct 9, 2020 1 message not in the archive

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 multipath-tools: replace perl with sed in libdmmp/Makefile for pseudo

From: Geoff Parker <[email protected]>

The multipath-tools libdmmp/Makefile install target uses 'perl -i' instead of 'sed -i' for string substitutions. The perl method creates a temporary file and overwrites the original which changes the inodes and corrupts the pseudo db. Changes to pseduo cause a build abort rather than allow possible bad ownership or permissions settings on the files. 'sed -i' is compatible with pseudo.

see https://wiki.yoctoproject.org/wiki/Pseudo_Abort

---
 ...-replace-perl-with-sed-in-install-ta.patch | 40 +++++++++++++++++++
 .../multipath-tools/multipath-tools_0.8.4.bb  |  1 +
 2 files changed, 41 insertions(+)
 create mode 100644 meta-oe/recipes-support/multipath-tools/files/0032-libdmmp-Makefile-replace-perl-with-sed-in-install-ta.patch
meta-oe/recipes-support/multipath-tools/files/0032-libdmmp-Makefile-replace-perl-with-sed-in-install-ta.patch new file +40 -0
new file mode 100644
index 000000000..b35d67cf5
@@ -0,0 +1,40 @@
+From 35421d03898ac1b38f97a76241e6a29086d0c9f1 Mon Sep 17 00:00:00 2001
+From: Geoff Parker <[email protected]>
+Date: Fri, 9 Oct 2020 17:48:47 +0000
+Subject: [PATCH] libdmmp/Makefile: replace perl with sed in install target to
+ work with pseudo
+
+The multipath-tools libdmmp/Makefile install target uses 'perl -i' instead
+of 'sed -i' for string substitutions. The perl method creates a temporary file
+and overwrites the original which changes the inodes and corrupts the pseudo
+db. Changes to pseduo cause a build abort rather than allow possible bad
+ownership or permissions settings on the files. 'sed -i' is compatible
+with pseudo.
+
+Signed-off-by: Geoff Parker <[email protected]>
+---
+ libdmmp/Makefile | 6 +++---
+ 1 file changed, 3 insertions(+), 3 deletions(-)
+
+diff --git a/libdmmp/Makefile b/libdmmp/Makefile
+index 1dd3f340..8166075c 100644
+--- a/libdmmp/Makefile
++++ b/libdmmp/Makefile
+@@ -34,11 +34,11 @@ install:
+ 	$(LN) $(LIBS) $(DESTDIR)$(usrlibdir)/$(DEVLIB)
+ 	$(INSTALL_PROGRAM) -m 644 -D \
+ 		$(PKGFILE).in $(DESTDIR)$(pkgconfdir)/$(PKGFILE)
+-	perl -i -pe 's|__VERSION__|$(LIBDMMP_VERSION)|g' \
++	sed -i 's|__VERSION__|$(LIBDMMP_VERSION)|g' \
+ 		$(DESTDIR)$(pkgconfdir)/$(PKGFILE)
+-	perl -i -pe 's|__LIBDIR__|$(usrlibdir)|g' \
++	sed -i 's|__LIBDIR__|$(usrlibdir)|g' \
+ 		$(DESTDIR)$(pkgconfdir)/$(PKGFILE)
+-	perl -i -pe 's|__INCLUDEDIR__|$(includedir)|g' \
++	sed -i 's|__INCLUDEDIR__|$(includedir)|g' \
+ 		$(DESTDIR)$(pkgconfdir)/$(PKGFILE)
+ 	@for file in docs/man/*.3.gz; do \
+ 		$(INSTALL_PROGRAM) -m 644 -D \
+-- 
+2.20.1
+
meta-oe/recipes-support/multipath-tools/multipath-tools_0.8.4.bb +1 -0
index 48a348d80..331c0cd01 100644
@@ -43,6 +43,7 @@ SRC_URI = "git://git.opensvc.com/multipath-tools/.git;protocol=http \
            file://0029-multipath-tools-modify-Makefile.inc-for-cross-compil.patch \
            file://0030-Always-use-devmapper.patch \
            file://0031-Always-use-devmapper-for-kpartx.patch \
+           file://0032-libdmmp-Makefile-replace-perl-with-sed-in-install-ta.patch \
            file://0001-fix-bug-of-do_compile-and-do_install.patch \
            file://0001-add-explicit-dependency-on-libraries.patch \
            file://0001-fix-boolean-value-with-json-c-0.14.patch \
Signature · 1 line
2.20.1
#1 lore raw
Quoted 9 lines · Geoff Parker

From: Geoff Parker <[email protected]>

The multipath-tools libdmmp/Makefile install target uses 'perl -i' instead of 'sed -i' for string substitutions. The perl method creates a temporary file and overwrites the original which changes the inodes and corrupts the pseudo db. Changes to pseduo cause a build abort rather than allow possible bad ownership or permissions settings on the files. 'sed -i' is compatible with pseudo.

while I think this patch is ok, I am not sure about the reasoning, when perl is running under pseudo contect perl -i -pe should be generating all sys calls under pseudo context, so how does this gets missed. is it something peculiar to perl ? I ask this because perl is commonly used to do sed like operations so do we have this issue in all those packages too ?

Quoted 72 lines

see https://wiki.yoctoproject.org/wiki/Pseudo_Abort

---
  ...-replace-perl-with-sed-in-install-ta.patch | 40 +++++++++++++++++++
  .../multipath-tools/multipath-tools_0.8.4.bb  |  1 +
  2 files changed, 41 insertions(+)

create mode 100644 meta-oe/recipes-support/multipath-tools/files/0032-libdmmp-Makefile-replace-perl-with-sed-in-install-ta.patch

meta-oe/recipes-support/multipath-tools/files/0032-libdmmp-Makefile-replace-perl-with-sed-in-install-ta.patch new file +40 -0
new file mode 100644
index 000000000..b35d67cf5
@@ -0,0 +1,40 @@
+From 35421d03898ac1b38f97a76241e6a29086d0c9f1 Mon Sep 17 00:00:00 2001
+From: Geoff Parker <[email protected]>
+Date: Fri, 9 Oct 2020 17:48:47 +0000
+Subject: [PATCH] libdmmp/Makefile: replace perl with sed in install target to
+ work with pseudo
+
+The multipath-tools libdmmp/Makefile install target uses 'perl -i' instead
+of 'sed -i' for string substitutions. The perl method creates a temporary file
+and overwrites the original which changes the inodes and corrupts the pseudo
+db. Changes to pseduo cause a build abort rather than allow possible bad
+ownership or permissions settings on the files. 'sed -i' is compatible
+with pseudo.
+
+Signed-off-by: Geoff Parker <[email protected]>
+---
+ libdmmp/Makefile | 6 +++---
+ 1 file changed, 3 insertions(+), 3 deletions(-)
+
+diff --git a/libdmmp/Makefile b/libdmmp/Makefile
+index 1dd3f340..8166075c 100644
+--- a/libdmmp/Makefile
++++ b/libdmmp/Makefile
+@@ -34,11 +34,11 @@ install:
+ 	$(LN) $(LIBS) $(DESTDIR)$(usrlibdir)/$(DEVLIB)
+ 	$(INSTALL_PROGRAM) -m 644 -D \
+ 		$(PKGFILE).in $(DESTDIR)$(pkgconfdir)/$(PKGFILE)
+-	perl -i -pe 's|__VERSION__|$(LIBDMMP_VERSION)|g' \
++	sed -i 's|__VERSION__|$(LIBDMMP_VERSION)|g' \
+ 		$(DESTDIR)$(pkgconfdir)/$(PKGFILE)
+-	perl -i -pe 's|__LIBDIR__|$(usrlibdir)|g' \
++	sed -i 's|__LIBDIR__|$(usrlibdir)|g' \
+ 		$(DESTDIR)$(pkgconfdir)/$(PKGFILE)
+-	perl -i -pe 's|__INCLUDEDIR__|$(includedir)|g' \
++	sed -i 's|__INCLUDEDIR__|$(includedir)|g' \
+ 		$(DESTDIR)$(pkgconfdir)/$(PKGFILE)
+ 	@for file in docs/man/*.3.gz; do \
+ 		$(INSTALL_PROGRAM) -m 644 -D \
+--
+2.20.1
+
meta-oe/recipes-support/multipath-tools/multipath-tools_0.8.4.bb +1 -0
index 48a348d80..331c0cd01 100644
@@ -43,6 +43,7 @@ SRC_URI = "git://git.opensvc.com/multipath-tools/.git;protocol=http \
             file://0029-multipath-tools-modify-Makefile.inc-for-cross-compil.patch \
             file://0030-Always-use-devmapper.patch \
             file://0031-Always-use-devmapper-for-kpartx.patch \
+           file://0032-libdmmp-Makefile-replace-perl-with-sed-in-install-ta.patch \
             file://0001-fix-bug-of-do_compile-and-do_install.patch \
             file://0001-add-explicit-dependency-on-libraries.patch \
             file://0001-fix-boolean-value-with-json-c-0.14.patch \
#2 lore raw

Is the issue reproducible when you use perl from perlnative?

I've built multipath-tools at least couple times on ubuntu 18.04 and 20.04 since the pseudo changes were merged and it doesn't abort() for me, so I guess it's something special your host perl is doing and it would be useful to find out what.

Quoted 111 lines · Khem Raj
Quoted 3 lines · Geoff Parker

From: Geoff Parker <[email protected]>

The multipath-tools libdmmp/Makefile install target uses 'perl -i'

instead

Quoted 1 line

of 'sed -i' for string substitutions. The perl method creates a

temporary file

Quoted 1 line

and overwrites the original which changes the inodes and corrupts the

pseudo

Quoted 4 lines

db. Changes to pseduo cause a build abort rather than allow possible bad ownership or permissions settings on the files. 'sed -i' is compatible with pseudo.

while I think this patch is ok, I am not sure about the reasoning, when perl is running under pseudo contect perl -i -pe should be generating all sys calls under pseudo context, so how does this gets missed. is it something peculiar to perl ? I ask this because perl is commonly used to do sed like operations so do we have this issue in all those packages too ?

Quoted 8 lines

see https://wiki.yoctoproject.org/wiki/Pseudo_Abort

---
  ...-replace-perl-with-sed-in-install-ta.patch | 40 +++++++++++++++++++
  .../multipath-tools/multipath-tools_0.8.4.bb  |  1 +
  2 files changed, 41 insertions(+)

create mode 100644

meta-oe/recipes-support/multipath-tools/files/0032-libdmmp-Makefile-replace-perl-with-sed-in-install-ta.patch

Quoted 2 lines

diff --git

a/meta-oe/recipes-support/multipath-tools/files/0032-libdmmp-Makefile-replace-perl-with-sed-in-install-ta.patch b/meta-oe/recipes-support/multipath-tools/files/0032-libdmmp-Makefile-replace-perl-with-sed-in-install-ta.patch

Quoted 4 lines

new file mode 100644
index 000000000..b35d67cf5
--- /dev/null
+++

b/meta-oe/recipes-support/multipath-tools/files/0032-libdmmp-Makefile-replace-perl-with-sed-in-install-ta.patch

meta-oe/recipes-support/multipath-tools/files/0032-libdmmp-Makefile-replace-perl-with-sed-in-install-ta.patch line 4

Quoted 5 lines
meta-oe/recipes-support/multipath-tools/files/0032-libdmmp-Makefile-replace-perl-with-sed-in-install-ta.patch new file +4 -0
@@ -0,0 +1,40 @@
+From 35421d03898ac1b38f97a76241e6a29086d0c9f1 Mon Sep 17 00:00:00 2001
+From: Geoff Parker <[email protected]>
+Date: Fri, 9 Oct 2020 17:48:47 +0000
+Subject: [PATCH] libdmmp/Makefile: replace perl with sed in install

target to

meta-oe/recipes-support/multipath-tools/files/0032-libdmmp-Makefile-replace-perl-with-sed-in-install-ta.patch

Quoted 3 lines
meta-oe/recipes-support/multipath-tools/files/0032-libdmmp-Makefile-replace-perl-with-sed-in-install-ta.patch new file +3 -0
+ work with pseudo
+
+The multipath-tools libdmmp/Makefile install target uses 'perl -i'

instead

Review line 8

Quoted 1 line
(quoted diff) +1 -0
+of 'sed -i' for string substitutions. The perl method creates a

temporary file

Review line 9

Quoted 1 line
(quoted diff) +1 -0
+and overwrites the original which changes the inodes and corrupts the

pseudo

Review

Quoted 32 lines
meta-oe/recipes-support/multipath-tools/files/0032-libdmmp-Makefile-replace-perl-with-sed-in-install-ta.patch new file +31 -0
+db. Changes to pseduo cause a build abort rather than allow possible bad
+ownership or permissions settings on the files. 'sed -i' is compatible
+with pseudo.
+
+Signed-off-by: Geoff Parker <[email protected]>
+---
+ libdmmp/Makefile | 6 +++---
+ 1 file changed, 3 insertions(+), 3 deletions(-)
+
+diff --git a/libdmmp/Makefile b/libdmmp/Makefile
+index 1dd3f340..8166075c 100644
+--- a/libdmmp/Makefile
++++ b/libdmmp/Makefile
+@@ -34,11 +34,11 @@ install:
+     $(LN) $(LIBS) $(DESTDIR)$(usrlibdir)/$(DEVLIB)
+     $(INSTALL_PROGRAM) -m 644 -D \
+             $(PKGFILE).in $(DESTDIR)$(pkgconfdir)/$(PKGFILE)
+-    perl -i -pe 's|__VERSION__|$(LIBDMMP_VERSION)|g' \
++    sed -i 's|__VERSION__|$(LIBDMMP_VERSION)|g' \
+             $(DESTDIR)$(pkgconfdir)/$(PKGFILE)
+-    perl -i -pe 's|__LIBDIR__|$(usrlibdir)|g' \
++    sed -i 's|__LIBDIR__|$(usrlibdir)|g' \
+             $(DESTDIR)$(pkgconfdir)/$(PKGFILE)
+-    perl -i -pe 's|__INCLUDEDIR__|$(includedir)|g' \
++    sed -i 's|__INCLUDEDIR__|$(includedir)|g' \
+             $(DESTDIR)$(pkgconfdir)/$(PKGFILE)
+     @for file in docs/man/*.3.gz; do \
+             $(INSTALL_PROGRAM) -m 644 -D \
+--
+2.20.1
+
(quoted diff) +0 -0

multipath-tools_0.8.4.bb b/meta-oe/recipes-support/multipath-tools/ multipath-tools_0.8.4.bb

meta-oe/recipes-support/multipath-tools/multipath-tools_0.8.4.bb SRC_URI = "git://

Quoted 4 lines

index 48a348d80..331c0cd01 100644

meta-oe/recipes-support/multipath-tools/multipath-tools_0.8.4.bb +0 -0
@@ -43,6 +43,7 @@ SRC_URI = "git://

git.opensvc.com/multipath-tools/.git;protocol=http \

Quoted 1 line

file://0029-multipath-tools-modify-Makefile.inc-for-cross-compil.patch \

Quoted 3 lines
             file://0030-Always-use-devmapper.patch \
             file://0031-Always-use-devmapper-for-kpartx.patch \

+

file://0032-libdmmp-Makefile-replace-perl-with-sed-in-install-ta.patch \

Quoted 8 lines
             file://0001-fix-bug-of-do_compile-and-do_install.patch \
             file://0001-add-explicit-dependency-on-libraries.patch \
             file://0001-fix-boolean-value-with-json-c-0.14.patch \

A message that is not in the archive (sent off-list or not yet received): BY5PR11MB4307A8928D1C4C5C1F1930A9E7080@BY5PR11MB4307.namprd11.prod.outlook.com

#4 lore raw

Can you try to add

inherit perlnative to test different perl? If it doesn't fail then enabling pseudo debug and comparing with perlnative and perl from Deb 10 might show what's the difference.

export PSEUDO_DEBUG="nfoPcvdDyerpswikVx"

in multiplath-tools.do_install is what RP recommended to me when we were debugging a similar case.

Quoted 156 lines · Geoff Parker

Hi Martin and Khem,

If I run bitbake -e, I see that perl is added to HOSTTOOLS by meta/conf/bitbake.conf:485. So I'm using a standard configuration.

Since the pseudo changes a few days ago, it’s been broken on our production Debian 10 build systems, my dev Deb 10 system, and I think with a few developers who build with Ubuntu 20.04.

This is the do_install error I get:

perl -i -pe 's|__VERSION__|0.2.0|g' \

|
/home/geoff/yocto/poky/build-nanoscope/tmp/work/cortexa9t2hf-neon-oe-linux-gnueabi/multipath-tools/0.8.4-r0/image//lib/pkgconfig/libdmmp.pc

| abort()ing pseudi client by server request. See
https://wiki.yoctoproject.org/wiki/Pseudo_Abort for more details on this.

| make[1]: *** [Makefile:35: install] Aborted

Looking at the pseudo/pseudo.log of the multipath-tools workspace,
path mismatch [1 link]: ino 4238971 db 'NAMELESS FILE' req '/home/geoff/yocto/poky/build-nanoscope/tmp/work/cortexa9t2hf-neon-oe-linux-gnueabi/multipath-tools/0.8.4-r0/image/lib/pkgconfig/XX6mDoYB'.

looking at the image/lib/pkgconfig/ dir, I see the original libdmmp.pc file and the XX6mD0YB file that perl created. I assume that it was in the process of overwriting libdmmpm.pc with XX6mDOYB.

From: Martin Jansa <[email protected]>
Sent: Friday, October 9, 2020 12:01 PM
To: Khem Raj <[email protected]>
Cc: Geoff Parker <[email protected]>; openembedded-devel < [email protected]>; Geoff Parker <[email protected]>
Subject: Re: [oe] [PATCH] multipath-tools: replace perl with sed in libdmmp/Makefile for pseudo

Is the issue reproducible when you use perl from perlnative?

I've built multipath-tools at least couple times on ubuntu 18.04 and 20.04 since the pseudo changes were merged and it doesn't abort() for me, so I guess it's something special your host perl is doing and it would be useful to find out what.

On Fri, Oct 9, 2020 at 8:48 PM Khem Raj <[email protected]> wrote:

Quoted 3 lines · Geoff Parker

From: Geoff Parker <[email protected]>

The multipath-tools libdmmp/Makefile install target uses 'perl -i'

instead

Quoted 1 line

of 'sed -i' for string substitutions. The perl method creates a

temporary file

Quoted 1 line

and overwrites the original which changes the inodes and corrupts the

pseudo

Quoted 4 lines

db. Changes to pseduo cause a build abort rather than allow possible bad ownership or permissions settings on the files. 'sed -i' is compatible with pseudo.

while I think this patch is ok, I am not sure about the reasoning, when perl is running under pseudo contect perl -i -pe should be generating all sys calls under pseudo context, so how does this gets missed. is it something peculiar to perl ? I ask this because perl is commonly used to do sed like operations so do we have this issue in all those packages too ?

Quoted 8 lines

see https://wiki.yoctoproject.org/wiki/Pseudo_Abort

---
  ...-replace-perl-with-sed-in-install-ta.patch | 40 +++++++++++++++++++
  .../multipath-tools/multipath-tools_0.8.4.bb  |  1 +
  2 files changed, 41 insertions(+)

create mode 100644

meta-oe/recipes-support/multipath-tools/files/0032-libdmmp-Makefile-replace-perl-with-sed-in-install-ta.patch

Quoted 2 lines

diff --git

a/meta-oe/recipes-support/multipath-tools/files/0032-libdmmp-Makefile-replace-perl-with-sed-in-install-ta.patch b/meta-oe/recipes-support/multipath-tools/files/0032-libdmmp-Makefile-replace-perl-with-sed-in-install-ta.patch

Quoted 4 lines

new file mode 100644
index 000000000..b35d67cf5
--- /dev/null
+++

b/meta-oe/recipes-support/multipath-tools/files/0032-libdmmp-Makefile-replace-perl-with-sed-in-install-ta.patch

meta-oe/recipes-support/multipath-tools/files/0032-libdmmp-Makefile-replace-perl-with-sed-in-install-ta.patch line 4

Quoted 5 lines
meta-oe/recipes-support/multipath-tools/files/0032-libdmmp-Makefile-replace-perl-with-sed-in-install-ta.patch new file +4 -0
@@ -0,0 +1,40 @@
+From 35421d03898ac1b38f97a76241e6a29086d0c9f1 Mon Sep 17 00:00:00 2001
+From: Geoff Parker <[email protected]>
+Date: Fri, 9 Oct 2020 17:48:47 +0000
+Subject: [PATCH] libdmmp/Makefile: replace perl with sed in install

target to

meta-oe/recipes-support/multipath-tools/files/0032-libdmmp-Makefile-replace-perl-with-sed-in-install-ta.patch

Quoted 3 lines
meta-oe/recipes-support/multipath-tools/files/0032-libdmmp-Makefile-replace-perl-with-sed-in-install-ta.patch new file +3 -0
+ work with pseudo
+
+The multipath-tools libdmmp/Makefile install target uses 'perl -i'

instead

Review line 8

Quoted 1 line
(quoted diff) +1 -0
+of 'sed -i' for string substitutions. The perl method creates a

temporary file

Review line 9

Quoted 1 line
(quoted diff) +1 -0
+and overwrites the original which changes the inodes and corrupts the

pseudo

Review

Quoted 32 lines
meta-oe/recipes-support/multipath-tools/files/0032-libdmmp-Makefile-replace-perl-with-sed-in-install-ta.patch new file +31 -0
+db. Changes to pseduo cause a build abort rather than allow possible bad
+ownership or permissions settings on the files. 'sed -i' is compatible
+with pseudo.
+
+Signed-off-by: Geoff Parker <[email protected]>
+---
+ libdmmp/Makefile | 6 +++---
+ 1 file changed, 3 insertions(+), 3 deletions(-)
+
+diff --git a/libdmmp/Makefile b/libdmmp/Makefile
+index 1dd3f340..8166075c 100644
+--- a/libdmmp/Makefile
++++ b/libdmmp/Makefile
+@@ -34,11 +34,11 @@ install:
+     $(LN) $(LIBS) $(DESTDIR)$(usrlibdir)/$(DEVLIB)
+     $(INSTALL_PROGRAM) -m 644 -D \
+             $(PKGFILE).in $(DESTDIR)$(pkgconfdir)/$(PKGFILE)
+-    perl -i -pe 's|__VERSION__|$(LIBDMMP_VERSION)|g' \
++    sed -i 's|__VERSION__|$(LIBDMMP_VERSION)|g' \
+             $(DESTDIR)$(pkgconfdir)/$(PKGFILE)
+-    perl -i -pe 's|__LIBDIR__|$(usrlibdir)|g' \
++    sed -i 's|__LIBDIR__|$(usrlibdir)|g' \
+             $(DESTDIR)$(pkgconfdir)/$(PKGFILE)
+-    perl -i -pe 's|__INCLUDEDIR__|$(includedir)|g' \
++    sed -i 's|__INCLUDEDIR__|$(includedir)|g' \
+             $(DESTDIR)$(pkgconfdir)/$(PKGFILE)
+     @for file in docs/man/*.3.gz; do \
+             $(INSTALL_PROGRAM) -m 644 -D \
+--
+2.20.1
+
(quoted diff) +0 -0

multipath-tools_0.8.4.bb b/meta-oe/recipes-support/multipath-tools/ multipath-tools_0.8.4.bb

meta-oe/recipes-support/multipath-tools/multipath-tools_0.8.4.bb SRC_URI = "git://

Quoted 4 lines

index 48a348d80..331c0cd01 100644

meta-oe/recipes-support/multipath-tools/multipath-tools_0.8.4.bb +0 -0
@@ -43,6 +43,7 @@ SRC_URI = "git://

git.opensvc.com/multipath-tools/.git;protocol=http \

Quoted 1 line

file://0029-multipath-tools-modify-Makefile.inc-for-cross-compil.patch \

Quoted 3 lines
             file://0030-Always-use-devmapper.patch \
             file://0031-Always-use-devmapper-for-kpartx.patch \

+

file://0032-libdmmp-Makefile-replace-perl-with-sed-in-install-ta.patch \

Quoted 8 lines
             file://0001-fix-bug-of-do_compile-and-do_install.patch \
             file://0001-add-explicit-dependency-on-libraries.patch \
             file://0001-fix-boolean-value-with-json-c-0.14.patch \