Skip to contentSkip to conversation

lighttpd: Add patch for reuse large memory chunks

PATCH Superseded Purushottam Choudhary 2 participants 2 messages Aug 18, 2021

v5 of 6 Superseded Latest is v6 →

lore
Share & tools
lore
#0 lore raw

PATCH v5 1/1 lighttpd: Add patch for reuse large memory chunks

Added 0001-core-reuse-large-mem-chunks-fix-mem-usage-fixes-3033.patch to fix large memory usage for large file downloads
from dynamic backends reuse or release large memory chunks.

This issue is caused by a bug in the lighttpd 1.4.55 version and has been fixed in lighttpd 1.4.58. Hence, it is not needed for master and hardknott branch because lighttpd has 1.4.59 version.

---
 ...fig-for-pcre-dependency-instead-of-config.patch |  10 +-
 ...large-mem-chunks-fix-mem-usage-fixes-3033.patch | 221 +++++++++++++++++++++
 meta/recipes-extended/lighttpd/lighttpd_1.4.55.bb  |   1 +
 3 files changed, 226 insertions(+), 6 deletions(-)
 create mode 100644 meta/recipes-extended/lighttpd/lighttpd/0001-core-reuse-large-mem-chunks-fix-mem-usage-fixes-3033.patch
meta/recipes-extended/lighttpd/lighttpd/0001-Use-pkg-config-for-pcre-dependency-instead-of-config.patch +4 -6
index f17bdce..edf2a07 100644
@@ -1,4 +1,4 @@
-From 22afc5d9aaa215c3c87ba21c77d47da44ab3b113 Mon Sep 17 00:00:00 2001
+From 4744f5b1cde08c056986ebaf900ee99141084a0d Mon Sep 17 00:00:00 2001
 From: Alexander Kanavin <[email protected]>
 Date: Fri, 26 Aug 2016 18:20:32 +0300
 Subject: [PATCH] Use pkg-config for pcre dependency instead of -config script.
@@ -6,15 +6,16 @@ Subject: [PATCH] Use pkg-config for pcre dependency instead of -config script.
 RP 2014/5/22
 Upstream-Status: Pending
 Signed-off-by: Alexander Kanavin <[email protected]>
+
 ---
  configure.ac | 16 ++++++++++++----
  1 file changed, 12 insertions(+), 4 deletions(-)
 
 diff --git a/configure.ac b/configure.ac
-index 5383cec..c29a902 100644
+index dbddfb9..62cf17f 100644
 --- a/configure.ac
 +++ b/configure.ac
-@@ -651,10 +651,18 @@ AC_ARG_WITH([pcre],
+@@ -748,10 +748,18 @@ AC_ARG_WITH([pcre],
  )
  AC_MSG_RESULT([$WITH_PCRE])
  
@@ -37,6 +38,3 @@ index 5383cec..c29a902 100644
    else
      AC_PATH_PROG([PCRECONFIG], [pcre-config])
      if test -n "$PCRECONFIG"; then
--- 
-2.15.0
-
meta/recipes-extended/lighttpd/lighttpd/0001-core-reuse-large-mem-chunks-fix-mem-usage-fixes-3033.patch new file +221 -0

221 lines not on this page · raw message

meta/recipes-extended/lighttpd/lighttpd_1.4.55.bb +1 -0
index 35a268a..5329465 100644
@@ -18,6 +18,7 @@ SRC_URI = "http://download.lighttpd.net/lighttpd/releases-1.4.x/lighttpd-${PV}.t
         file://lighttpd.conf \
         file://lighttpd \
         file://0001-Use-pkg-config-for-pcre-dependency-instead-of-config.patch \
+        file://0001-core-reuse-large-mem-chunks-fix-mem-usage-fixes-3033.patch \
         "
 
 SRC_URI[md5sum] = "be4bda2c28bcbdac6eb941528f6edf03"
Signature · 1 line
2.7.4
#1 lore raw

Hello

Quoted 37 lines · Purushottam Choudhary

From: Glenn Strauss <[email protected]>

meta/recipes-extended/lighttpd/lighttpd/0001-core-reuse-large-mem-chunks-fix-mem-usage-fixes-3033.patch new file +12 -0
+Date: Wed, 23 Dec 2020 23:14:47 -0500
+Subject: [PATCH] reuse large mem chunks (fix mem usage) (fixes #3033)
+
+(cherry picked from commit 7ba521ffb4959f6f74a609d5d4acafc29a038337)
+
+(thx flynn)
+
+fix large memory usage for large file downloads from dynamic backends
+
+reuse or release large memory chunks
+Upstream-Status: Backport
+

https://redmine.lighttpd.net/projects/lighttpd/repository/14/revisions/7ba521ffb4959f6f74a609d5d4acafc29a038337

meta/recipes-extended/lighttpd/lighttpd/0001-core-reuse-large-mem-chunks-fix-mem-usage-fixes-3033.patch new file +22 -1
+Signed-off-by: Purushottam Choudhary <[email protected]>
+x-ref:
+  "Memory Growth with PUT and full buffered streams"
+  https://redmine.lighttpd.net/issues/3033
+
+---
+ src/chunk.c            | 99 ++++++++++++++++++++++++++++++++++++++++-
---------
+ src/chunk.h            |  2 +
+ src/http-header-glue.c |  2 +-
+ 3 files changed, 82 insertions(+), 21 deletions(-)
+
+diff --git a/src/chunk.c b/src/chunk.c
+index 09dd3f1..ffb3795 100644
+--- a/src/chunk.c
++++ b/src/chunk.c
+@@ -28,16 +28,20 @@
+ static size_t chunk_buf_sz = 4096;
+ static chunk *chunks, *chunks_oversized;
+ static chunk *chunk_buffers;
++static int chunks_oversized_n;
+ static array *chunkqueue_default_tempdirs = NULL;
+ static off_t chunkqueue_default_tempfile_size =
Quoted 8 lines

DEFAULT_TEMPFILE_SIZE;
+
+ void chunkqueue_set_chunk_size (size_t sz)
+ {
+-    chunk_buf_sz = sz > 0 ? ((sz + 1023) & ~1023uL) : 4096; ++    size_t x = 1024;
++    while (x < sz && x < (1u << 30)) x <<= 1;
++    chunk_buf_sz = sz > 0 ? x : 4096;

This doesn't look like a clean backport and you seem to have changed the chunk size to 4k from 8k in the patch. This should be mentioned.

Quoted 5 lines

+ }
+
+ void chunkqueue_set_tempdirs_default_reset (void)
+ {
++    chunk_buf_sz = 8192;

While here it's being reset to 8k. This looks wrong ... Perhaps you should consider taking an earlier commit as well that's changing the size to 8k. Also, is the aim of this patch is to fix

https://redmine.lighttpd.net/issues/3033

? If yes, you probably need some other patches as well:

https://redmine.lighttpd.net/issues/3033?tab=changesets

Thanks,

Anuj