Skip to contentSkip to conversation

Add pigz as gzip-native alternative.

PATCH In review Björn Stenberg 5 participants 12 messages Mar 13, 2012 – Mar 21, 2012

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 Add pigz as gzip-native alternative.

pigz, which stands for parallel implementation of gzip, is a fully functional replacement for gzip that exploits multiple processors and multiple cores to the hilt when compressing data. pigz was written by Mark Adler, and uses the zlib and pthread libraries.

This recipe adds pigz as an alternative provider of gzip-native. Pigz is much faster than gzip on multi-core machines.

Upstream-status: Pending

---
 meta/recipes-extended/pigz/files/ldflags.patch |   11 +++++++++++
 meta/recipes-extended/pigz/pigz.inc            |   21 +++++++++++++++++++++
 meta/recipes-extended/pigz/pigz_2.2.4.bb       |    9 +++++++++
 3 files changed, 41 insertions(+), 0 deletions(-)
 create mode 100644 meta/recipes-extended/pigz/files/ldflags.patch
 create mode 100644 meta/recipes-extended/pigz/pigz.inc
 create mode 100644 meta/recipes-extended/pigz/pigz_2.2.4.bb
meta/recipes-extended/pigz/files/ldflags.patch new file +11 -0
new file mode 100644
index 0000000..534d280
@@ -0,0 +1,11 @@
+--- pigz-2.2.4/Makefile.orig	2012-01-01 02:46:58.000000000 +0100
++++ pigz-2.2.4/Makefile	2012-03-09 08:47:40.025094863 +0100
+@@ -2,7 +2,7 @@
+ CFLAGS=-O3 -Wall -Wextra
+ 
+ pigz: pigz.o yarn.o
+-	$(CC) -o pigz pigz.o yarn.o -lpthread -lz
++	$(CC) $(LDFLAGS) -o pigz pigz.o yarn.o -lpthread -lz
+ 	ln -f pigz unpigz
+ 
+ pigz.o: pigz.c yarn.h
meta/recipes-extended/pigz/pigz.inc new file +21 -0
new file mode 100644
index 0000000..b501d7a
@@ -0,0 +1,21 @@
+SUMMARY = "A parallel implementation of gzip"
+DESCRIPTION = "pigz, which stands for parallel implementation of gzip, is a \
+fully functional replacement for gzip that exploits multiple processors and \
+multiple cores to the hilt when compressing data. pigz was written by Mark \
+Adler, and uses the zlib and pthread libraries."
+HOMEPAGE = "http://zlib.net/pigz/"
+SECTION = "console/utils"
+LICENSE = "Zlib"
+
+SRC_URI = "http://zlib.net/pigz/pigz-${PV}.tar.gz \
+           file://ldflags.patch"
+
+PROVIDES += "gzip-native"
+
+DEPENDS_virtclass-native = "zlib-native"
+
+do_install () {
+        install -d ${D}${bindir}
+        install ${B}/pigz ${D}${bindir}/gzip
+        install ${B}/unpigz ${D}${bindir}/gunzip
+}
meta/recipes-extended/pigz/pigz_2.2.4.bb new file +9 -0
new file mode 100644
index 0000000..598bbe3
@@ -0,0 +1,9 @@
+require pigz.inc
+
+PR = "r0"
+
+BBCLASSEXTEND = "native"
+
+SRC_URI[md5sum] = "9df2a3c742524446fa4e797c17e8fd85"
+SRC_URI[sha256sum] = "7e7967c47f66c07be97cbfa25c8816b72f9d35258466ea31eddc7666c914160e"
+LIC_FILES_CHKSUM = "file://pigz.c;md5=53f47c097bbf11b937a421b1816a8dc5"
Signature · 1 line
1.7.5.4
#1 lore raw

Hi Björn,

Quoted 9 lines · On Tuesday 13 March 2012 15:52:51 Björn Stenberg wrote:

pigz, which stands for parallel implementation of gzip, is a fully functional replacement for gzip that exploits multiple processors and multiple cores to the hilt when compressing data. pigz was written by Mark Adler, and uses the zlib and pthread libraries.

This recipe adds pigz as an alternative provider of gzip-native. Pigz is much faster than gzip on multi-core machines.

Upstream-status: Pending

This needs to be moved to the inner ldflags.patch, and..

Quoted 1 line

this should be copied to there. The patch should also start with an explanation of what the patch does (in this case it's trivial, but still worth having in there.)

Thanks,
Paul

Signature · 2 lines
Paul Eggleton
Intel Open Source Technology Centre
#2 lore raw

PATCH 1/1 Add pigz as gzip-native alternative.

pigz, which stands for parallel implementation of gzip, is a fully functional replacement for gzip that exploits multiple processors and multiple cores to the hilt when compressing data. pigz was written by Mark Adler, and uses the zlib and pthread libraries.

This recipe adds pigz as an alternative gzip-native implementation only.

---
 meta/recipes-extended/pigz/files/ldflags.patch |   17 +++++++++++++++++
 meta/recipes-extended/pigz/pigz.inc            |   21 +++++++++++++++++++++
 meta/recipes-extended/pigz/pigz_2.2.4.bb       |    9 +++++++++
 3 files changed, 47 insertions(+), 0 deletions(-)
 create mode 100644 meta/recipes-extended/pigz/files/ldflags.patch
 create mode 100644 meta/recipes-extended/pigz/pigz.inc
 create mode 100644 meta/recipes-extended/pigz/pigz_2.2.4.bb
meta/recipes-extended/pigz/files/ldflags.patch new file +17 -0
new file mode 100644
index 0000000..928c3a5
@@ -0,0 +1,17 @@
+Link using the bitbake LDFLAGS.
+
+Signed-off-by: Björn Stenberg <[email protected]>
+
+Upstream-Status: Pending
+
+--- pigz-2.2.4/Makefile.orig	2012-01-01 02:46:58.000000000 +0100
++++ pigz-2.2.4/Makefile	2012-03-09 08:47:40.025094863 +0100
+@@ -2,7 +2,7 @@
+ CFLAGS=-O3 -Wall -Wextra
+ 
+ pigz: pigz.o yarn.o
+-	$(CC) -o pigz pigz.o yarn.o -lpthread -lz
++	$(CC) $(LDFLAGS) -o pigz pigz.o yarn.o -lpthread -lz
+ 	ln -f pigz unpigz
+ 
+ pigz.o: pigz.c yarn.h
meta/recipes-extended/pigz/pigz.inc new file +21 -0
new file mode 100644
index 0000000..b501d7a
@@ -0,0 +1,21 @@
+SUMMARY = "A parallel implementation of gzip"
+DESCRIPTION = "pigz, which stands for parallel implementation of gzip, is a \
+fully functional replacement for gzip that exploits multiple processors and \
+multiple cores to the hilt when compressing data. pigz was written by Mark \
+Adler, and uses the zlib and pthread libraries."
+HOMEPAGE = "http://zlib.net/pigz/"
+SECTION = "console/utils"
+LICENSE = "Zlib"
+
+SRC_URI = "http://zlib.net/pigz/pigz-${PV}.tar.gz \
+           file://ldflags.patch"
+
+PROVIDES += "gzip-native"
+
+DEPENDS_virtclass-native = "zlib-native"
+
+do_install () {
+        install -d ${D}${bindir}
+        install ${B}/pigz ${D}${bindir}/gzip
+        install ${B}/unpigz ${D}${bindir}/gunzip
+}
meta/recipes-extended/pigz/pigz_2.2.4.bb new file +9 -0
new file mode 100644
index 0000000..598bbe3
@@ -0,0 +1,9 @@
+require pigz.inc
+
+PR = "r0"
+
+BBCLASSEXTEND = "native"
+
+SRC_URI[md5sum] = "9df2a3c742524446fa4e797c17e8fd85"
+SRC_URI[sha256sum] = "7e7967c47f66c07be97cbfa25c8816b72f9d35258466ea31eddc7666c914160e"
+LIC_FILES_CHKSUM = "file://pigz.c;md5=53f47c097bbf11b937a421b1816a8dc5"
Signature · 1 line
1.7.5.4
#3 lore raw
Quoted 18 lines · On Wednesday 14 March 2012 16:59:31 Björn Stenberg wrote:

pigz, which stands for parallel implementation of gzip, is a fully functional replacement for gzip that exploits multiple processors and multiple cores to the hilt when compressing data. pigz was written by Mark Adler, and uses the zlib and pthread libraries.

This recipe adds pigz as an alternative gzip-native implementation only.
...

(quoted diff) +0 -0

b/meta/recipes-extended/pigz/pigz.inc new file mode 100644 index 0000000..b501d7a

meta/recipes-extended/pigz/pigz.inc new file +2 -0
@@ -0,0 +1,21 @@
+SUMMARY = "A parallel implementation of gzip"
+DESCRIPTION = "pigz, which stands for parallel implementation of gzip, is a

\ +fully functional replacement for gzip that exploits multiple processors and \ +multiple cores to the hilt when compressing data. pigz was written by Mark \ +Adler, and uses the zlib and pthread libraries."

meta/recipes-extended/pigz/pigz.inc line 13

Quoted 8 lines
meta/recipes-extended/pigz/pigz.inc new file +8 -0
+HOMEPAGE = "http://zlib.net/pigz/"
+SECTION = "console/utils"
+LICENSE = "Zlib"
+
+SRC_URI = "http://zlib.net/pigz/pigz-${PV}.tar.gz \
+           file://ldflags.patch"
+
+PROVIDES += "gzip-native"

Sorry for not picking up on this earlier, but I think this needs to be PROVIDES_virtclass-native, otherwise the target recipe will also provide gzip- native.

Cheers,
Paul

Signature · 2 lines
Paul Eggleton
Intel Open Source Technology Centre
#4 lore raw

PATCH 1/1 Add pigz as gzip-native alternative.

pigz, which stands for parallel implementation of gzip, is a fully functional replacement for gzip that exploits multiple processors and multiple cores to the hilt when compressing data. pigz was written by Mark Adler, and uses the zlib and pthread libraries.

This recipe adds pigz as an alternative gzip-native implementation only.

---
 meta/recipes-extended/pigz/files/ldflags.patch |   17 +++++++++++++++++
 meta/recipes-extended/pigz/pigz.inc            |   21 +++++++++++++++++++++
 meta/recipes-extended/pigz/pigz_2.2.4.bb       |    9 +++++++++
 3 files changed, 47 insertions(+), 0 deletions(-)
 create mode 100644 meta/recipes-extended/pigz/files/ldflags.patch
 create mode 100644 meta/recipes-extended/pigz/pigz.inc
 create mode 100644 meta/recipes-extended/pigz/pigz_2.2.4.bb
meta/recipes-extended/pigz/files/ldflags.patch new file +17 -0
new file mode 100644
index 0000000..928c3a5
@@ -0,0 +1,17 @@
+Link using the bitbake LDFLAGS.
+
+Signed-off-by: Björn Stenberg <[email protected]>
+
+Upstream-Status: Pending
+
+--- pigz-2.2.4/Makefile.orig	2012-01-01 02:46:58.000000000 +0100
++++ pigz-2.2.4/Makefile	2012-03-09 08:47:40.025094863 +0100
+@@ -2,7 +2,7 @@
+ CFLAGS=-O3 -Wall -Wextra
+ 
+ pigz: pigz.o yarn.o
+-	$(CC) -o pigz pigz.o yarn.o -lpthread -lz
++	$(CC) $(LDFLAGS) -o pigz pigz.o yarn.o -lpthread -lz
+ 	ln -f pigz unpigz
+ 
+ pigz.o: pigz.c yarn.h
meta/recipes-extended/pigz/pigz.inc new file +21 -0
new file mode 100644
index 0000000..acf1b71
@@ -0,0 +1,21 @@
+SUMMARY = "A parallel implementation of gzip"
+DESCRIPTION = "pigz, which stands for parallel implementation of gzip, is a \
+fully functional replacement for gzip that exploits multiple processors and \
+multiple cores to the hilt when compressing data. pigz was written by Mark \
+Adler, and uses the zlib and pthread libraries."
+HOMEPAGE = "http://zlib.net/pigz/"
+SECTION = "console/utils"
+LICENSE = "Zlib"
+
+SRC_URI = "http://zlib.net/pigz/pigz-${PV}.tar.gz \
+           file://ldflags.patch"
+
+PROVIDES_virtclass-native += "gzip-native"
+
+DEPENDS_virtclass-native = "zlib-native"
+
+do_install () {
+        install -d ${D}${bindir}
+        install ${B}/pigz ${D}${bindir}/gzip
+        install ${B}/unpigz ${D}${bindir}/gunzip
+}
meta/recipes-extended/pigz/pigz_2.2.4.bb new file +9 -0
new file mode 100644
index 0000000..598bbe3
@@ -0,0 +1,9 @@
+require pigz.inc
+
+PR = "r0"
+
+BBCLASSEXTEND = "native"
+
+SRC_URI[md5sum] = "9df2a3c742524446fa4e797c17e8fd85"
+SRC_URI[sha256sum] = "7e7967c47f66c07be97cbfa25c8816b72f9d35258466ea31eddc7666c914160e"
+LIC_FILES_CHKSUM = "file://pigz.c;md5=53f47c097bbf11b937a421b1816a8dc5"
Signature · 1 line
1.7.5.4
#5 lore raw

Op 15 mrt. 2012, om 09:25 heeft Björn Stenberg het volgende geschreven:

Quoted 14 lines

+++ b/meta/recipes-extended/pigz/pigz.inc

meta/recipes-extended/pigz/pigz.inc new file +11 -0
@@ -0,0 +1,21 @@
+SUMMARY = "A parallel implementation of gzip"
+DESCRIPTION = "pigz, which stands for parallel implementation of gzip, is a \
+fully functional replacement for gzip that exploits multiple processors and \
+multiple cores to the hilt when compressing data. pigz was written by Mark \
+Adler, and uses the zlib and pthread libraries."
+HOMEPAGE = "http://zlib.net/pigz/"
+SECTION = "console/utils"
+LICENSE = "Zlib"
+
+SRC_URI = "http://zlib.net/pigz/pigz-${PV}.tar.gz \
+           file://ldflags.patch"

meta/recipes-extended/pigz/pigz.inc line 21

Quoted 10 lines
meta/recipes-extended/pigz/pigz.inc new file +10 -0
+
+PROVIDES_virtclass-native += "gzip-native"
+
+DEPENDS_virtclass-native = "zlib-native"
+
+do_install () {
+        install -d ${D}${bindir}
+        install ${B}/pigz ${D}${bindir}/gzip
+        install ${B}/unpigz ${D}${bindir}/gunzip
+}

This is either missing update-alternatives entries or it is missing RREPLACES/RCONFLICTS with gzip/busybox/etc. Or both.

regards,

Koen

#6 lore raw
Quoted 5 lines · Koen Kooi

Op 15 mrt. 2012, om 09:25 heeft Björn Stenberg het volgende geschreven:

Quoted 1 line

+PROVIDES_virtclass-native += "gzip-native"

This is either missing update-alternatives entries or it is missing RREPLACES/RCONFLICTS with gzip/busybox/etc. Or both.

My intention with the patch was to provide a native-only recipe for pigz, to improve distro build speed.

I'm not sure there is much interest in using pigz on target.

Signature · 1 line
Björn
#7 lore raw

Op 19 mrt. 2012, om 16:58 heeft Björn Stenberg het volgende geschreven:

Quoted 8 lines
Quoted 5 lines · Koen Kooi

Op 15 mrt. 2012, om 09:25 heeft Björn Stenberg het volgende geschreven:

Quoted 1 line

+PROVIDES_virtclass-native += "gzip-native"

This is either missing update-alternatives entries or it is missing RREPLACES/RCONFLICTS with gzip/busybox/etc. Or both.

My intention with the patch was to provide a native-only recipe for pigz, to improve distro build speed.

But it went in as a regular recipe and hence needs to play well with others.

Quoted 10 lines

I'm not sure there is much interest in using pigz on target.

Signature · 6 lines
Björn

_______________________________________________
Openembedded-core mailing list
[email protected]
http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
#8 lore raw
Quoted 4 lines · Koen Kooi
Quoted 2 lines

My intention with the patch was to provide a native-only recipe for pigz, to improve distro build speed.

But it went in as a regular recipe and hence needs to play well with others.

I thought I made it native-only by only providing gzip-native?

Signature · 1 line
Björn
#9 lore raw

Op 20 mrt. 2012, om 15:12 heeft Björn Stenberg het volgende geschreven:

Quoted 7 lines
Quoted 4 lines · Koen Kooi
Quoted 2 lines

My intention with the patch was to provide a native-only recipe for pigz, to improve distro build speed.

But it went in as a regular recipe and hence needs to play well with others.

I thought I made it native-only by only providing gzip-native?

try 'bitbake pigz' :)

#10 lore raw

PATCH 1/1 pigz: Add update-altneratives support since this otherwise conflicts with busybox/gzip

---

meta/recipes-extended/pigz/pigz.inc +14 -3
index 5fda8ca..e1cbc8b 100644
@@ -15,7 +15,18 @@ PROVIDES_virtclass-native += "gzip-native"
 DEPENDS = "zlib"
 
 do_install () {
-        install -d ${D}${bindir}
-        install ${B}/pigz ${D}${bindir}/gzip
-        install ${B}/unpigz ${D}${bindir}/gunzip
+	if [ "${PN}" = "${BPN}" ] ; then
+		# Install files into /bin (FHS), which is typical place for gzip
+	        install -d ${D}${base_bindir}	        
+		install ${B}/pigz ${D}${base_bindir}/gzip
+	        install ${B}/unpigz ${D}${base_bindir}/gunzip
+	else
+	        install -d ${D}${bindir}
+	        install ${B}/pigz ${D}${bindir}/gzip
+	        install ${B}/unpigz ${D}${bindir}/gunzip
+	fi
 }
+
+ALTERNATIVE_LINKS = "${base_bindir}/gzip ${base_bindir}/gunzip"
+ALTERNATIVE_PRIORITY = "80"
+
meta/recipes-extended/pigz/pigz_2.2.4.bb +1 -1
index eaf00c6..e76c49c 100644
@@ -1,6 +1,6 @@
 require pigz.inc
 
-PR = "r1"
+PR = "r2"
 
 NATIVE_PACKAGE_PATH_SUFFIX = "/${PN}"
 
#11 lore raw
In reply to Björn Stenberg
Quoted 65 lines · Björn Stenberg

pigz, which stands for parallel implementation of gzip, is a fully functional replacement for gzip that exploits multiple processors and multiple cores to the hilt when compressing data. pigz was written by Mark Adler, and uses the zlib and pthread libraries.

This recipe adds pigz as an alternative provider of gzip-native. Pigz is much faster than gzip on multi-core machines.

Upstream-status: Pending

---
  meta/recipes-extended/pigz/files/ldflags.patch |   11 +++++++++++
  meta/recipes-extended/pigz/pigz.inc            |   21 +++++++++++++++++++++
  meta/recipes-extended/pigz/pigz_2.2.4.bb       |    9 +++++++++
  3 files changed, 41 insertions(+), 0 deletions(-)

create mode 100644 meta/recipes-extended/pigz/files/ldflags.patch create mode 100644 meta/recipes-extended/pigz/pigz.inc create mode 100644 meta/recipes-extended/pigz/pigz_2.2.4.bb

meta/recipes-extended/pigz/files/ldflags.patch new file +11 -0
new file mode 100644
index 0000000..534d280
@@ -0,0 +1,11 @@
+--- pigz-2.2.4/Makefile.orig	2012-01-01 02:46:58.000000000 +0100
++++ pigz-2.2.4/Makefile	2012-03-09 08:47:40.025094863 +0100
+@@ -2,7 +2,7 @@
+ CFLAGS=-O3 -Wall -Wextra
+
+ pigz: pigz.o yarn.o
+-	$(CC) -o pigz pigz.o yarn.o -lpthread -lz
++	$(CC) $(LDFLAGS) -o pigz pigz.o yarn.o -lpthread -lz
+ 	ln -f pigz unpigz
+
+ pigz.o: pigz.c yarn.h
meta/recipes-extended/pigz/pigz.inc new file +21 -0
new file mode 100644
index 0000000..b501d7a
@@ -0,0 +1,21 @@
+SUMMARY = "A parallel implementation of gzip"
+DESCRIPTION = "pigz, which stands for parallel implementation of gzip, is a \
+fully functional replacement for gzip that exploits multiple processors and \
+multiple cores to the hilt when compressing data. pigz was written by Mark \
+Adler, and uses the zlib and pthread libraries."
+HOMEPAGE = "http://zlib.net/pigz/"
+SECTION = "console/utils"
+LICENSE = "Zlib"
+
+SRC_URI = "http://zlib.net/pigz/pigz-${PV}.tar.gz \
+           file://ldflags.patch"
+
+PROVIDES += "gzip-native"
+
+DEPENDS_virtclass-native = "zlib-native"
+
+do_install () {
+        install -d ${D}${bindir}
+        install ${B}/pigz ${D}${bindir}/gzip
+        install ${B}/unpigz ${D}${bindir}/gunzip
+}
meta/recipes-extended/pigz/pigz_2.2.4.bb new file +0 -0
new file mode 100644
index 0000000..598bbe3

meta/recipes-extended/pigz/pigz_2.2.4.bb line 9

Quoted 9 lines
meta/recipes-extended/pigz/pigz_2.2.4.bb new file +9 -0
@@ -0,0 +1,9 @@
+require pigz.inc
+
+PR = "r0"
+
+BBCLASSEXTEND = "native"
+
+SRC_URI[md5sum] = "9df2a3c742524446fa4e797c17e8fd85"
+SRC_URI[sha256sum] = "7e7967c47f66c07be97cbfa25c8816b72f9d35258466ea31eddc7666c914160e"
+LIC_FILES_CHKSUM = "file://pigz.c;md5=53f47c097bbf11b937a421b1816a8dc5"

Merged and enabled in OE-Core

Thanks
Sau!