memory usage tracing update

PATCH In review [email protected] 2 participants 7 messages Jan 17, 2013 – Jan 22, 2013 4 messages not in the archive

lore

Share & tools

lore

[email protected]

author

2013-01-17 08:55 UTC #0 lore raw

v3 cover memory usage tracing update

Hi,

This is the v3 patch set. There's several fixes per Harald's comments.

Please review.

Thanks
Dave

[email protected]

author

2013-01-17 08:55 UTC #1 lore raw

PATCH v3 1/2 add function getargnum

For cmdline argument with numeric value, add a new function getargnum It will get proper value with default value as $1, min value as $2, max value as $3, and param name as $4. valid result will be echo to stdout. for nul or value not valid it will just echo the default value.
Note: The values should be >=0

[v1->v2]: add arg <minval>
[v2->v3]: do not use bash string match =~

Signed-off-by Dave Young <[email protected]>

---
 modules.d/99base/dracut-lib.sh |   29 +++++++++++++++++++++++++++++
 1 file changed, 29 insertions(+)

modules.d/99base/dracut-lib.sh +29 -0

@@ -167,6 +167,35 @@ getargbool() {
     return 0
 }
 
+isdigit() {
+    case "$1" in
+        *[!0-9]*|"") return 1;;
+    esac
+
+    return 0
+}
+
+# getargnum <defaultval> <minval> <maxval> <arg>
+# Will echo the arg if it's in range [minval - maxval].
+# If it's not set or it's not valid, will set it <defaultval>.
+# Note all values are required to be >= 0 here.
+# <defaultval> should be with [minval -maxval].
+getargnum() {
+    local _b
+    unset _b
+    local _default _min _max
+    _default=$1; shift
+    _min=$1; shift
+    _max=$1; shift
+    _b=$(getarg "$1")
+    [ $? -ne 0 -a -z "$_b" ] && _b=$_default
+    if [ -n "$_b" ]; then
+        isdigit "$_b" && _b=$(($_b)) && \
+        [[ $_b -ge $_min && $_b -le $_max ]] && echo $_b && return
+    fi
+    echo $_default
+}
+
 _dogetargs() {
     debug_off
     local _o _found _key

A message that is not in the archive (sent off-list or not yet received): [email protected]

Harald Hoyer

2013-01-22 14:32 UTC #3 lore raw

Re: [PATCH 1/2 V3] add function getargnum

Am 17.01.2013 09:55, schrieb [email protected]:

Quoted 10 lines

For cmdline argument with numeric value, add a new function getargnum It will get proper value with default value as $1, min value as $2, max value as $3, and param name as $4. valid result will be echo to stdout. for nul or value not valid it will just echo the default value.
Note: The values should be >=0
[v1->v2]: add arg <minval>
[v2->v3]: do not use bash string match =~
Signed-off-by Dave Young <[email protected]>

merged

[email protected]

author

2013-01-17 08:55 UTC #4 lore raw

PATCH v3 2/2 Add memory usage trace to diffrent hook points

In reply to [email protected]

Port mkdumprd memory trace functions wrote by [email protected] For kdump, memory usage tracing is important because there's limited
memory in kdump 2nd kernel.

Add a rd.memdebug cmdline for this. set rd.memdebug=<X> will set the debug level to X which is the debug verbose level.

the format of cmdline is like below:
<level>[+]:<type>
<level> is the debug level
[+] means debug level >= <level>
<type> is the debug info type, as for this patch I added mem, iomem, slab mem is for /proc/meminfo, iomem is for /proc/iomem, slab is for /proc/slabinfo Also shortmem is the stripped /proc/meminfo which only includes 3 lines of Memfree, Cached and Slab, for example:

MemFree:         6327176 kB
Cached:           741916 kB
Slab:              77284 kB

I added several trace point to the begin of several init hooks At cmdline hooks I'm adding trace of "1+:mem 1+:iomem 3+:slab" For other hooks I'm adding trace of "1:shortmem 2+:mem 3+:slab"

This means:
rd.memdebug=1)
cmdline hook: print mem and iomem
other hooks: print shortmem
rd.memdebug=2)
cmdline hook: print mem and iomem
other hooks: print mem
rd.memdebug=3):
cmdline hook: print mem iomem, and slabinfo
other hooks: print mem and slabinfo
*):
do not print any mem debug info

[v1->v2]: update to use getargnum with <minval> as argument
print iomem info at cmdline hook as well
[v2->v3]: harald: use $() instead of ``
use bash string match instead of grep and pipe.

Signed-off-by Dave Young <[email protected]>

---
 dracut.cmdline.7.asc                      |    4 +
 modules.d/98systemd/dracut-cmdline.sh     |    1 
 modules.d/98systemd/dracut-initqueue.sh   |    1 
 modules.d/98systemd/dracut-pre-pivot.sh   |    1 
 modules.d/98systemd/dracut-pre-trigger.sh |    1 
 modules.d/98systemd/dracut-pre-udev.sh    |    1 
 modules.d/99base/dracut-lib.sh            |   91 ++++++++++++++++++++++++++++++
 7 files changed, 100 insertions(+)

modules.d/98systemd/dracut-cmdline.sh +1 -0

@@ -18,6 +18,7 @@ type getarg >/dev/null 2>&1 || . /lib/dr
 
 source_conf /etc/conf.d
 
+make_trace_mem "hook cmdline" 1+:mem 1+:iomem 3+:slab
 # run scriptlets to parse the command line
 getarg 'rd.break=cmdline' -d 'rdbreak=cmdline' && emergency_shell -n cmdline "Break before cmdline"
 source_hook cmdline

modules.d/98systemd/dracut-initqueue.sh +1 -0

@@ -10,6 +10,7 @@ type getarg >/dev/null 2>&1 || . /lib/dr
 
 source_conf /etc/conf.d
 
+make_trace_mem "hook initqueue" 1:shortmem 2+:mem 3+:slab
 getarg 'rd.break=initqueue' -d 'rdbreak=initqueue' && emergency_shell -n initqueue "Break before initqueue"
 
 RDRETRY=$(getarg rd.retry -d 'rd_retry=')

modules.d/98systemd/dracut-pre-pivot.sh +1 -0

@@ -10,6 +10,7 @@ type getarg >/dev/null 2>&1 || . /lib/dr
 
 source_conf /etc/conf.d
 
+make_trace_mem "hook pre-pivot" 1:shortmem 2+:mem 3+:slab
 # pre pivot scripts are sourced just before we doing cleanup and switch over
 # to the new root.
 getarg 'rd.break=pre-pivot' 'rdbreak=pre-pivot' && emergency_shell -n pre-pivot "Break pre-pivot"

modules.d/98systemd/dracut-pre-trigger.sh +1 -0

@@ -10,6 +10,7 @@ type getarg >/dev/null 2>&1 || . /lib/dr
 
 source_conf /etc/conf.d
 
+make_trace_mem "hook pre-trigger" 1:shortmem 2+:mem 3+:slab
 getargbool 0 rd.udev.info -n -y rdudevinfo && udevadm control --log-priority=info
 getargbool 0 rd.udev.debug -n -y rdudevdebug && udevadm control --log-priority=debug
 udevproperty "hookdir=$hookdir"

modules.d/98systemd/dracut-pre-udev.sh +1 -0

@@ -9,6 +9,7 @@ type getarg >/dev/null 2>&1 || . /lib/dr
 
 source_conf /etc/conf.d
 
+make_trace_mem "hook pre-udev" 1:shortmem 2+:mem 3+:slab
 # pre pivot scripts are sourced just before we doing cleanup and switch over
 # to the new root.
 getarg 'rd.break=pre-udev' 'rdbreak=pre-udev' && emergency_shell -n pre-udev "Break pre-udev"

dracut.cmdline.7.asc +4 -0

@@ -143,6 +143,10 @@ It should be attached to any report abou
     If systemd is not active, the logs are written to dmesg and _/run/initramfs/init.log_.
     If "quiet" is set, it also logs to the console.
 
+**rd.memdebug=[0-3]**::
+    print memory usage debug info, set the verbose level from 1 to 3
+    print nothing when set rd.memdebug=0
+
 **rd.break**::
     drop to a shell at the end

modules.d/99base/dracut-lib.sh +91 -0

94 lines not on this page · raw message

A message that is not in the archive (sent off-list or not yet received): [email protected]

Harald Hoyer

2013-01-17 13:43 UTC #6 lore raw

Re: [PATCH 2/2 V3] Add memory usage trace to diffrent hook points

Am 17.01.2013 09:55, schrieb [email protected]:

Quoted 52 lines

Port mkdumprd memory trace functions wrote by [email protected] For kdump, memory usage tracing is important because there's limited
memory in kdump 2nd kernel.
Add a rd.memdebug cmdline for this. set rd.memdebug=<X> will set the debug level to X which is the debug verbose level.
the format of cmdline is like below:
<level>[+]:<type>
<level> is the debug level
[+] means debug level >= <level>
<type> is the debug info type, as for this patch I added mem, iomem, slab mem is for /proc/meminfo, iomem is for /proc/iomem, slab is for /proc/slabinfo Also shortmem is the stripped /proc/meminfo which only includes 3 lines of Memfree, Cached and Slab, for example:
MemFree:         6327176 kB
Cached:           741916 kB
Slab:              77284 kB
I added several trace point to the begin of several init hooks At cmdline hooks I'm adding trace of "1+:mem 1+:iomem 3+:slab" For other hooks I'm adding trace of "1:shortmem 2+:mem 3+:slab"
This means:
rd.memdebug=1)
cmdline hook: print mem and iomem
other hooks: print shortmem
rd.memdebug=2)
cmdline hook: print mem and iomem
other hooks: print mem
rd.memdebug=3):
cmdline hook: print mem iomem, and slabinfo
other hooks: print mem and slabinfo
*):
do not print any mem debug info
[v1->v2]: update to use getargnum with <minval> as argument
print iomem info at cmdline hook as well
[v2->v3]: harald: use $() instead of ``
use bash string match instead of grep and pipe.
Signed-off-by Dave Young <[email protected]>
---
 dracut.cmdline.7.asc                      |    4 +
 modules.d/98systemd/dracut-cmdline.sh     |    1 
 modules.d/98systemd/dracut-initqueue.sh   |    1 
 modules.d/98systemd/dracut-pre-pivot.sh   |    1 
 modules.d/98systemd/dracut-pre-trigger.sh |    1 
 modules.d/98systemd/dracut-pre-udev.sh    |    1 
 modules.d/99base/dracut-lib.sh            |   91 ++++++++++++++++++++++++++++++
 7 files changed, 100 insertions(+)

...

Quoted 2 lines

modules.d/99base/dracut-lib.sh +0 -0

...

modules.d/99base/dracut-lib.sh listlist() { · line 1008

Quoted 3 lines

modules.d/99base/dracut-lib.sh +3 -0
+# parameters: msg [trace_level:trace]...
+function make_trace_mem()
+{

local msg

modules.d/99base/dracut-lib.sh listlist() { · line 1018

Quoted 10 lines

modules.d/99base/dracut-lib.sh +10 -0

+    msg=$1
+    shift
+    if [ "$DEBUG_MEM_LEVEL" -gt 0 ]; then
+        make_trace show_memstats $DEBUG_MEM_LEVEL "[debug_mem]" "$msg" "$@"
+    fi
+}
+
+# parameters: func log_level prefix msg [trace_level:trace]...
+function make_trace()
+{

local func
local log_level
local prefix
.....

Quoted 51 lines

modules.d/99base/dracut-lib.sh +51 -0

+    func=$1
+    shift
+
+    log_level=$1
+    shift
+
+    prefix=$1
+    shift
+
+    msg=$1
+    shift
+
+    if [ -z "$log_level" ]; then
+        return
+    fi
+
+    msg=$(echo $msg)
+
+    msg_printed=0
+    while [ $# -gt 0 ]; do
+        trace=${1%%:*}
+        trace_level=${trace%%+}
+        [ "$trace" != "$trace_level" ] && trace_in_higher_levels="yes"
+        trace=${1##*:}
+
+        if [ -z "$trace_level" ]; then
+            trace_level=0
+        fi
+
+        insert_trace=0
+        if [ -n "$trace_in_higher_levels" ]; then
+            if [ "$log_level" -ge "$trace_level" ]; then
+                insert_trace=1
+            fi
+        else
+            if [ "$log_level" -eq "$trace_level" ]; then
+                insert_trace=1
+            fi
+        fi
+
+        if [ $insert_trace -eq 1 ]; then
+            if [ $msg_printed -eq 0 ]; then
+                echo "$prefix $msg"
+                msg_printed=1
+            fi
+            $func $trace
+        fi
+        shift
+    done
+}
+

A message that is not in the archive (sent off-list or not yet received): [email protected]

Dave Young

author

2013-01-18 01:59 UTC #8 lore raw

PATCH v4 2/2 Add memory usage trace to diffrent hook points

Hi, Here is an update of this patch, fixed the local variable issue.
---

Add memory usage trace to diffrent hook points

Port mkdumprd memory trace functions wrote by [email protected] For kdump, memory usage tracing is important because there's limited
memory in kdump 2nd kernel.

Add a rd.memdebug cmdline for this. set rd.memdebug=<X> will set the debug level to X which is the debug verbose level.

MemFree:         6327176 kB
Cached:           741916 kB
Slab:              77284 kB

I added several trace point to the begin of several init hooks At cmdline hooks I'm adding trace of "1+:mem 1+:iomem 3+:slab" For other hooks I'm adding trace of "1:shortmem 2+:mem 3+:slab"

[v1->v2]: update to use getargnum with <minval> as argument
print iomem info at cmdline hook as well
[v2->v3]: harald: use $() instead of ``
use bash string match instead of grep and pipe.
[v3->v4]: harald: use local variables in function.

Signed-off-by Dave Young <[email protected]>

---
 dracut.cmdline.7.asc                      |    4 +
 modules.d/98systemd/dracut-cmdline.sh     |    1 
 modules.d/98systemd/dracut-initqueue.sh   |    1 
 modules.d/98systemd/dracut-pre-pivot.sh   |    1 
 modules.d/98systemd/dracut-pre-trigger.sh |    1 
 modules.d/98systemd/dracut-pre-udev.sh    |    1 
 modules.d/99base/dracut-lib.sh            |   94 ++++++++++++++++++++++++++++++
 7 files changed, 103 insertions(+)

modules.d/98systemd/dracut-cmdline.sh +1 -0

@@ -18,6 +18,7 @@ type getarg >/dev/null 2>&1 || . /lib/dr
 
 source_conf /etc/conf.d
 
+make_trace_mem "hook cmdline" 1+:mem 1+:iomem 3+:slab
 # run scriptlets to parse the command line
 getarg 'rd.break=cmdline' -d 'rdbreak=cmdline' && emergency_shell -n cmdline "Break before cmdline"
 source_hook cmdline

modules.d/98systemd/dracut-initqueue.sh +1 -0

@@ -10,6 +10,7 @@ type getarg >/dev/null 2>&1 || . /lib/dr
 
 source_conf /etc/conf.d
 
+make_trace_mem "hook initqueue" 1:shortmem 2+:mem 3+:slab
 getarg 'rd.break=initqueue' -d 'rdbreak=initqueue' && emergency_shell -n initqueue "Break before initqueue"
 
 RDRETRY=$(getarg rd.retry -d 'rd_retry=')

modules.d/98systemd/dracut-pre-pivot.sh +1 -0

@@ -10,6 +10,7 @@ type getarg >/dev/null 2>&1 || . /lib/dr
 
 source_conf /etc/conf.d
 
+make_trace_mem "hook pre-pivot" 1:shortmem 2+:mem 3+:slab
 # pre pivot scripts are sourced just before we doing cleanup and switch over
 # to the new root.
 getarg 'rd.break=pre-pivot' 'rdbreak=pre-pivot' && emergency_shell -n pre-pivot "Break pre-pivot"

modules.d/98systemd/dracut-pre-trigger.sh +1 -0

@@ -10,6 +10,7 @@ type getarg >/dev/null 2>&1 || . /lib/dr
 
 source_conf /etc/conf.d
 
+make_trace_mem "hook pre-trigger" 1:shortmem 2+:mem 3+:slab
 getargbool 0 rd.udev.info -n -y rdudevinfo && udevadm control --log-priority=info
 getargbool 0 rd.udev.debug -n -y rdudevdebug && udevadm control --log-priority=debug
 udevproperty "hookdir=$hookdir"

modules.d/98systemd/dracut-pre-udev.sh +1 -0

@@ -9,6 +9,7 @@ type getarg >/dev/null 2>&1 || . /lib/dr
 
 source_conf /etc/conf.d
 
+make_trace_mem "hook pre-udev" 1:shortmem 2+:mem 3+:slab
 # pre pivot scripts are sourced just before we doing cleanup and switch over
 # to the new root.
 getarg 'rd.break=pre-udev' 'rdbreak=pre-udev' && emergency_shell -n pre-udev "Break pre-udev"

dracut.cmdline.7.asc +4 -0

@@ -143,6 +143,10 @@ It should be attached to any report abou
     If systemd is not active, the logs are written to dmesg and _/run/initramfs/init.log_.
     If "quiet" is set, it also logs to the console.
 
+**rd.memdebug=[0-3]**::
+    print memory usage debug info, set the verbose level from 1 to 3
+    print nothing when set rd.memdebug=0
+
 **rd.break**::
     drop to a shell at the end

modules.d/99base/dracut-lib.sh +94 -0

97 lines not on this page · raw message

A message that is not in the archive (sent off-list or not yet received): [email protected]

Harald Hoyer

2013-01-22 14:32 UTC #10 lore raw

Re: [PATCH 2/2 V4] Add memory usage trace to diffrent hook points

Am 18.01.2013 02:59, schrieb Dave Young:

Quoted 42 lines

Hi, Here is an update of this patch, fixed the local variable issue.
---
Add memory usage trace to diffrent hook points
Port mkdumprd memory trace functions wrote by [email protected] For kdump, memory usage tracing is important because there's limited
memory in kdump 2nd kernel.
Add a rd.memdebug cmdline for this. set rd.memdebug=<X> will set the debug level to X which is the debug verbose level.
the format of cmdline is like below:
<level>[+]:<type>
<level> is the debug level
[+] means debug level >= <level>
<type> is the debug info type, as for this patch I added mem, iomem, slab mem is for /proc/meminfo, iomem is for /proc/iomem, slab is for /proc/slabinfo Also shortmem is the stripped /proc/meminfo which only includes 3 lines of Memfree, Cached and Slab, for example:
MemFree:         6327176 kB
Cached:           741916 kB
Slab:              77284 kB
I added several trace point to the begin of several init hooks At cmdline hooks I'm adding trace of "1+:mem 1+:iomem 3+:slab" For other hooks I'm adding trace of "1:shortmem 2+:mem 3+:slab"
This means:
rd.memdebug=1)
cmdline hook: print mem and iomem
other hooks: print shortmem
rd.memdebug=2)
cmdline hook: print mem and iomem
other hooks: print mem
rd.memdebug=3):
cmdline hook: print mem iomem, and slabinfo
other hooks: print mem and slabinfo
*):
do not print any mem debug info

Quoted 7 lines

[v1->v2]: update to use getargnum with <minval> as argument
print iomem info at cmdline hook as well
[v2->v3]: harald: use $() instead of ``
use bash string match instead of grep and pipe.
[v3->v4]: harald: use local variables in function.
Signed-off-by Dave Young <[email protected]>

merged