Skip to contentSkip to conversation

Add pigz as gzip-native alternative.

PATCH In review Björn Stenberg 2 participants 3 messages Mar 12, 2012 – Mar 13, 2012

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 Add pigz as gzip-native alternative.

pigz, which stands for parallel implementation of gzip, is a fully functional replacement for gzip that exploits multiple processors and multiple cores to the hilt when compressing data. pigz was written by Mark Adler, and uses the zlib and pthread libraries.

This recipe adds pigz as an alternative provider of gzip-native. Pigz is much faster than gzip on multi-core machines.

---
 meta/recipes-extended/pigz/files/ldflags.patch |   11 +++++++++++
 meta/recipes-extended/pigz/pigz.inc            |   21 +++++++++++++++++++++
 meta/recipes-extended/pigz/pigz_2.2.4.bb       |    9 +++++++++
 3 files changed, 41 insertions(+), 0 deletions(-)
 create mode 100644 meta/recipes-extended/pigz/files/ldflags.patch
 create mode 100644 meta/recipes-extended/pigz/pigz.inc
 create mode 100644 meta/recipes-extended/pigz/pigz_2.2.4.bb
meta/recipes-extended/pigz/files/ldflags.patch new file +11 -0
new file mode 100644
index 0000000..534d280
@@ -0,0 +1,11 @@
+--- pigz-2.2.3/Makefile.orig	2012-01-01 02:46:58.000000000 +0100
++++ pigz-2.2.3/Makefile	2012-03-09 08:47:40.025094863 +0100
+@@ -2,7 +2,7 @@
+ CFLAGS=-O3 -Wall -Wextra
+ 
+ pigz: pigz.o yarn.o
+-	$(CC) -o pigz pigz.o yarn.o -lpthread -lz
++	$(CC) $(LDFLAGS) -o pigz pigz.o yarn.o -lpthread -lz
+ 	ln -f pigz unpigz
+ 
+ pigz.o: pigz.c yarn.h
meta/recipes-extended/pigz/pigz.inc new file +21 -0
new file mode 100644
index 0000000..b501d7a
@@ -0,0 +1,21 @@
+SUMMARY = "A parallel implementation of gzip"
+DESCRIPTION = "pigz, which stands for parallel implementation of gzip, is a \
+fully functional replacement for gzip that exploits multiple processors and \
+multiple cores to the hilt when compressing data. pigz was written by Mark \
+Adler, and uses the zlib and pthread libraries."
+HOMEPAGE = "http://zlib.net/pigz/"
+SECTION = "console/utils"
+LICENSE = "Zlib"
+
+SRC_URI = "http://zlib.net/pigz/pigz-${PV}.tar.gz \
+           file://ldflags.patch"
+
+PROVIDES += "gzip-native"
+
+DEPENDS_virtclass-native = "zlib-native"
+
+do_install () {
+        install -d ${D}${bindir}
+        install ${B}/pigz ${D}${bindir}/gzip
+        install ${B}/unpigz ${D}${bindir}/gunzip
+}
meta/recipes-extended/pigz/pigz_2.2.4.bb new file +9 -0
new file mode 100644
index 0000000..598bbe3
@@ -0,0 +1,9 @@
+require pigz.inc
+
+PR = "r0"
+
+BBCLASSEXTEND = "native"
+
+SRC_URI[md5sum] = "9df2a3c742524446fa4e797c17e8fd85"
+SRC_URI[sha256sum] = "7e7967c47f66c07be97cbfa25c8816b72f9d35258466ea31eddc7666c914160e"
+LIC_FILES_CHKSUM = "file://pigz.c;md5=53f47c097bbf11b937a421b1816a8dc5"
Signature · 1 line
1.7.5.4
#1 lore raw

I thought about doing this - can you share any performance numbers?

-M

Quoted 80 lines · Björn Stenberg

pigz, which stands for parallel implementation of gzip, is a fully functional replacement for gzip that exploits multiple processors and multiple cores to the hilt when compressing data. pigz was written by Mark Adler, and uses the zlib and pthread libraries.

This recipe adds pigz as an alternative provider of gzip-native. Pigz is much faster than gzip on multi-core machines.

---
 meta/recipes-extended/pigz/files/ldflags.patch |   11 +++++++++++ meta/recipes-extended/pigz/pigz.inc            |   21 +++++++++++++++++++++ meta/recipes-extended/pigz/pigz_2.2.4.bb       |    9 +++++++++

 3 files changed, 41 insertions(+), 0 deletions(-)

 create mode 100644 meta/recipes-extended/pigz/files/ldflags.patch create mode 100644 meta/recipes-extended/pigz/pigz.inc create mode 100644 meta/recipes-extended/pigz/pigz_2.2.4.bb

meta/recipes-extended/pigz/files/ldflags.patch new file +11 -0
new file mode 100644
index 0000000..534d280
@@ -0,0 +1,11 @@
+--- pigz-2.2.3/Makefile.orig   2012-01-01 02:46:58.000000000 +0100
++++ pigz-2.2.3/Makefile        2012-03-09 08:47:40.025094863 +0100
+@@ -2,7 +2,7 @@
+ CFLAGS=-O3 -Wall -Wextra
+
+ pigz: pigz.o yarn.o
+-      $(CC) -o pigz pigz.o yarn.o -lpthread -lz
++      $(CC) $(LDFLAGS) -o pigz pigz.o yarn.o -lpthread -lz
+       ln -f pigz unpigz
+
+ pigz.o: pigz.c yarn.h
meta/recipes-extended/pigz/pigz.inc new file +21 -0
new file mode 100644
index 0000000..b501d7a
@@ -0,0 +1,21 @@
+SUMMARY = "A parallel implementation of gzip"
+DESCRIPTION = "pigz, which stands for parallel implementation of gzip, is a \
+fully functional replacement for gzip that exploits multiple processors and \
+multiple cores to the hilt when compressing data. pigz was written by Mark \
+Adler, and uses the zlib and pthread libraries."
+HOMEPAGE = "http://zlib.net/pigz/"
+SECTION = "console/utils"
+LICENSE = "Zlib"
+
+SRC_URI = "http://zlib.net/pigz/pigz-${PV}.tar.gz \
+           file://ldflags.patch"
+
+PROVIDES += "gzip-native"
+
+DEPENDS_virtclass-native = "zlib-native"
+
+do_install () {
+        install -d ${D}${bindir}
+        install ${B}/pigz ${D}${bindir}/gzip
+        install ${B}/unpigz ${D}${bindir}/gunzip
+}
meta/recipes-extended/pigz/pigz_2.2.4.bb new file +9 -1
new file mode 100644
index 0000000..598bbe3
@@ -0,0 +1,9 @@
+require pigz.inc
+
+PR = "r0"
+
+BBCLASSEXTEND = "native"
+
+SRC_URI[md5sum] = "9df2a3c742524446fa4e797c17e8fd85"
+SRC_URI[sha256sum] = "7e7967c47f66c07be97cbfa25c8816b72f9d35258466ea31eddc7666c914160e"
+LIC_FILES_CHKSUM = "file://pigz.c;md5=53f47c097bbf11b937a421b1816a8dc5"
--

1.7.5.4

_______________________________________________
poky mailing list
[email protected]
https://lists.yoctoproject.org/listinfo/poky

#2 lore raw
Quoted 1 line · McClintock Matthew-B29882 wrote:

I thought about doing this - can you share any performance numbers?

pigz seems to scale fairly well. I compared the time it takes to compress the final rootfs tarball in do_rootfs for one of my images.

gzip:

real    1m33.523s
user    1m33.146s
sys     0m0.368s

pigz:

real    0m2.239s
user    1m40.554s
sys     0m0.716s

This is on a 64-core machine.

Signature · 1 line
Björn