kconfig: simplification of scripts/extract-ikconfig
Subsystems (2)
PATCH 1/1 kconfig: simplification of scripts/extract-ikconfig
I've rewritten the extract-ikconfig script to extract the kernel configuration from a kernel compiled with CONFIG_IKCONFIG. The main motivation for the rewrite was to remove the dependency on the external C program binoffset.c, which is compiled on the initial run.
The binoffset executable is invoked with a relative path, which means that the old script can only be run from the top of the kernel tree, and only when you have write permission in the scripts directory. The new script uses echo/grep/tail/zcat only, and can be invoked from anywhere. The binoffset.c program has been removed.
---
Documentation/dontdiff | 1 - scripts/.gitignore | 1 - scripts/binoffset.c | 163 ---------------------------------------------- scripts/extract-ikconfig | 108 +++++++++++------------------- 4 files changed, 39 insertions(+), 234 deletions(-)
Documentation/dontdiff +0 -1
@@ -68,7 +68,6 @@ autoconf.h* bbootsect bin2c binkernel.spec -binoffset bootsect bounds.h bsetup
scripts/.gitignore +0 -1
@@ -6,4 +6,3 @@ kallsyms pnmtologo bin2c unifdef -binoffset
scripts/binoffset.c deleted +0 -163
163 lines not on this page · raw message
scripts/extract-ikconfig +39 -69
117 lines not on this page · raw message
Quoted 12 lines · Dick Streefland
I've rewritten the extract-ikconfig script to extract the kernel configuration from a kernel compiled with
CONFIG_IKCONFIG. The main motivation for the rewrite was to remove the dependency on the external C programbinoffset.c, which is compiled on the initial run.The binoffset executable is invoked with a relative path, which means that the old script can only be run from the top of the kernel tree, and only when you have write permission in the scripts directory. The new script uses echo/grep/tail/zcat only, and can be invoked from anywhere. The
binoffset.cprogram has been removed.Signed-off-by Dick Streefland <[email protected]>
<snip>
scripts/extract-ikconfig line 16
Quoted 13 lines
scripts/extract-ikconfig +4
-0+gzip_header=`/bin/echo -en '\x1f\x8b\x08'` +ikconfig_header="IKCFG_ST$gzip_header" + +dump_config(){
scripts/extract-ikconfig +1
-7- echo " usage: extract-ikconfig [b]zImage_filename" -} - -clean_up() -{ - if [ "$TMPFILE" != "" ]; then - rm -f $TMPFILE + if pos=`grep -abo "$ikconfig_header" "$1"`
Are you sure this 'grep' works well?
My tests on a binary file told me no.
Quoted 44 lines
scripts/extract-ikconfig +4
-0+ then + pos=${pos%%:*} + tail -c+$(($pos+8+1)) "$1" | zcat -q + exit 0fi
}scripts/extract-ikconfig +4
-1-if [ $# -lt 1 ] +# Check invocation: +me=${0##*/} +img=$1 +if [ $# -ne 1 -o ! -s "$img" ]then
scripts/extract-ikconfig +2
-2- usage - exit 1 + echo "Usage: $me <kernel-image>" >&2 + exit 2fi
scripts/extract-ikconfig +0
-25-TMPFILE=`mktemp -t ikconfig-XXXXXX` || exit 1 -image="$1" - -# vmlinux: Attempt to dump the configuration from the file directly -dump_config "$image" - -GZHDR1="0x1f 0x8b 0x08 0x00" -GZHDR2="0x1f 0x8b 0x08 0x08" - -# vmlinux.gz: Check for a compressed images -off=`$binoffset "$image" $GZHDR1 2>/dev/null` -[ "$?" != "0" ] && off="-1" -if [ "$off" -eq "-1" ]; then - off=`$binoffset "$image" $GZHDR2 2>/dev/null` - [ "$?" != "0" ] && off="-1" -fi -if [ "$off" -eq "0" ]; then - zcat <"$image" >"$TMPFILE" - dump_config "$TMPFILE" -elif [ "$off" -ne "-1" ]; then - (dd ibs="$off" skip=1 count=0 && dd bs=512k) <"$image" 2>/dev/null | \ - zcat >"$TMPFILE" - dump_config "$TMPFILE" -fi -
scripts/extract-ikconfig line 38
Quoted 10 lines
scripts/extract-ikconfig +6
-4-echo "ERROR: Unable to extract kernel configuration information." -echo " This kernel image may not have the config info." - -clean_up +# Initial attempt for uncompressed images or objects: +dump_config "$img" + +# That didn't work, so decompress and try again: +tmp=/tmp/ikconfig$$ +trap "rm -f $tmp" 0
Why do you use trap instead of doing this explicitly on exit?
Quoted 1 line
+for pos in
grep -abo "$gzip_header" "$img"
Also here...
Can we have more than one $gzip_header? But even if yes, grep only reports the first one I am afraid, or at least, the first one in one "line".
Quoted 9 lines
scripts/extract-ikconfig +8
-0+do + pos=${pos%%:*} + tail -c+$(($pos+1)) "$img" | zcat -q > $tmp + dump_config $tmp +done + +# Bail out: +echo "$me: Cannot find kernel config." >&2exit 1
On Wednesday 2009-06-24 10:15, Amerigo Wang wrote:
| On Wed, Jun 24, 2009 at 12:52:45AM +0200, Dick Streefland wrote: | >+ if pos=`grep -abo "$ikconfig_header" "$1"` | | Are you sure this 'grep' works well? | | My tests on a binary file told me no.
The version of grep on the machines I tested on (GNU grep 2.5.3) does work, but I now see that this combination of options doesn't work on older versions like 2.5.1. Bummer!
| >+tmp=/tmp/ikconfig$$ | >+trap "rm -f $tmp" 0 | | Why do you use trap instead of doing this explicitly | on exit?
Because it is simpler when you have multiple exit points, and also to make sure the file is removed when the script is terminated prematurely, for instance by a signal.
| Can we have more than one $gzip_header? But even if yes, | grep only reports the first one I am afraid, or at least, | the first one in one "line".
With grep 2.5.3, you will find every occurrence of the pattern, not just a single one per "line".
Because I don't want to rely on a particular version of grep, I've modified the script to use perl instead. I hope that's OK. Below is a new patch.
======================================================================
I've rewritten the extract-ikconfig script to extract the kernel configuration from a kernel compiled with CONFIG_IKCONFIG. The main motivation for the rewrite was to remove the dependency on the external C program binoffset.c, which is compiled on the initial run.
The binoffset executable is invoked with a relative path, which means that the old script can only be run from the top of the kernel tree, and only when you have write permission in the scripts directory. The new script uses perl/tail/zcat only, and can be invoked from anywhere. The binoffset.c program has been removed.
---
Documentation/dontdiff | 1 - scripts/.gitignore | 1 - scripts/binoffset.c | 163 ---------------------------------------------- scripts/extract-ikconfig | 112 ++++++++++++------------------- 4 files changed, 43 insertions(+), 234 deletions(-)
Documentation/dontdiff +0 -1
@@ -68,7 +68,6 @@ autoconf.h* bbootsect bin2c binkernel.spec -binoffset bootsect bounds.h bsetup
scripts/.gitignore +0 -1
@@ -6,4 +6,3 @@ kallsyms pnmtologo bin2c unifdef -binoffset
scripts/binoffset.c deleted +0 -163
163 lines not on this page · raw message
scripts/extract-ikconfig +43 -69
121 lines not on this page · raw message
On Wednesday 2009-06-24 13:46, Dick Streefland wrote:
| Because I don't want to rely on a particular version of grep, I've | modified the script to use perl instead. I hope that's OK. Below is | a new patch.
<snip>
I found a way to work around the differences in behavior of grep. Older versions of grep report the byte offset of the start of the line instead of the byte offset of the pattern. By using tr to make sure the pattern is always at the start of a "line", both old and new versions of grep can be used.
Below is a third version of the patch without a dependency on perl, which may be undesirable in some situations.
======================================================================
I've rewritten the extract-ikconfig script to extract the kernel configuration from a kernel compiled with CONFIG_IKCONFIG. The main motivation for the rewrite was to remove the dependency on the external C program binoffset.c, which is compiled on the initial run.
The binoffset executable is invoked with a relative path, which means that the old script can only be run from the top of the kernel tree, and only when you have write permission in the scripts directory. The new script uses tr/grep/tail/zcat only, and can be invoked from anywhere. The binoffset.c program has been removed.
---
Documentation/dontdiff | 1 - scripts/.gitignore | 1 - scripts/binoffset.c | 163 ---------------------------------------------- scripts/extract-ikconfig | 113 ++++++++++++------------------- 4 files changed, 44 insertions(+), 234 deletions(-)
Documentation/dontdiff +0 -1
@@ -68,7 +68,6 @@ autoconf.h* bbootsect bin2c binkernel.spec -binoffset bootsect bounds.h bsetup
scripts/.gitignore +0 -1
@@ -6,4 +6,3 @@ kallsyms pnmtologo bin2c unifdef -binoffset
scripts/binoffset.c deleted +0 -163
163 lines not on this page · raw message
scripts/extract-ikconfig +44 -69
122 lines not on this page · raw message
Quoted 5 lines · Dick Streefland
On Wednesday 2009-06-24 13:46, Dick Streefland wrote:
| Because I don't want to rely on a particular version of grep, I've | modified the script to use perl instead. I hope that's OK. Below is | a new patch.<snip>
Sorry for the delay, I missed this patch...
Quoted 6 lines
I found a way to work around the differences in behavior of grep. Older versions of grep report the byte offset of the start of the line instead of the byte offset of the pattern. By using tr to make sure the pattern is always at the start of a "line", both old and new versions of grep can be used.
This is very tricky...
<snip>
Quoted 8 lines
scripts/extract-ikconfig +7
-0+ +gz1='\037\213\010' +gz2='01' +cf1='IKCFG_ST\037\213\010' +cf2='0123456789' + +dump_config(){
scripts/extract-ikconfig line 21
Quoted 8 lines
scripts/extract-ikconfig +1
-7- echo " usage: extract-ikconfig [b]zImage_filename" -} - -clean_up() -{ - if [ "$TMPFILE" != "" ]; then - rm -f $TMPFILE + if pos=`tr "$cf1\n$cf2" "\n$cf2=" < "$1" | grep -abo "^$cf2"`
Does this work?
My quick test shows no...
I still can't get the right offset number with this trick.
P.S. My grep is 2.5.1.
On Monday 2009-06-29 10:51, Amerigo Wang wrote:
| Does this work? | My quick test shows no... | | I still can't get the right offset number with this trick. | | P.S. My grep is 2.5.1.
I've tested the script with grep 2.5.1, and it works for me. Can you be more specific about what you think is wrong with the offsets? You need to be aware of the fact that because "tr" replaces the first character of the pattern by a newline, an offset will be 1 higher than the start of the original pattern. The resulting offsets are therefore 1-based instead of 0-based, which is exactly what "tail" expects for the "-c" option.
Signature · 1 line
Dick
Amerigo Wang
Tested-byQuoted 8 lines · Dick Streefland
On Monday 2009-06-29 10:51, Amerigo Wang wrote:
| Does this work? | My quick test shows no... | | I still can't get the right offset number with this trick. | | P.S. My grep is 2.5.1.
Quoted 7 lines
I've tested the script with grep 2.5.1, and it works for me. Can you be more specific about what you think is wrong with the offsets? You need to be aware of the fact that because "tr" replaces the first character of the pattern by a newline, an offset will be 1 higher than the start of the original pattern. The resulting offsets are therefore 1-based instead of 0-based, which is exactly what "tail" expects for the "-c" option.
Hi,
I appologize for my delay, I totally missed this thread, sorry.
I just tested it, it works! So,
But, I still don't understand the behavior of 'grep -abo', please try the following commands:
echo -e '\x01\x02\x03\x04' > test.bin
od -x test.bin
tr $'\x01\x02\x03\x04' '1234' < test.bin | grep -abo '4'
Am I missing something here??
Thanks.
On Thursday 2009-07-09 10:38, Amerigo Wang wrote:
| I appologize for my delay, I totally missed this thread, sorry. | | I just tested it, it works! So, | | Tested-by: WANG Cong <[email protected]>
Great, thanks.
| But, I still don't understand the behavior of 'grep -abo', | please try the following commands: | | echo -e '\x01\x02\x03\x04' > test.bin | od -x test.bin | tr $'\x01\x02\x03\x04' '1234' < test.bin | grep -abo '4' | | Am I missing something here??
The output with grep-2.5.3 is 3:4, which means that the string "4" was found at offset 3 (0-based). With grep-2.5.1, you get 0:4, which means that the line containing the string "4" has offset 0. The "tr" commands in the script ensure that the patterns we are looking for are always at the start of a line, so that it doesn't matter which version of grep is installed.
Signature · 1 line
Dick
Quoted 18 lines · Dick Streefland
On Thursday 2009-07-09 10:38, Amerigo Wang wrote:
| I appologize for my delay, I totally missed this thread, sorry. | | I just tested it, it works! So, | | Tested-by: WANG Cong <[email protected]>Great, thanks.
| But, I still don't understand the behavior of 'grep -abo', | please try the following commands: | | echo -e '\x01\x02\x03\x04' > test.bin | od -x test.bin | tr $'\x01\x02\x03\x04' '1234' < test.bin | grep -abo '4' | | Am I missing something here??
Quoted 6 lines
The output with grep-2.5.3 is 3:4, which means that the string "4" was found at offset 3 (0-based). With grep-2.5.1, you get 0:4, which means that the line containing the string "4" has offset 0. The "tr" commands in the script ensure that the patterns we are looking for are always at the start of a line, so that it doesn't matter which version of grep is installed.
Hmm, I see, it is really a _big_ change. :)
Have you checked other version of grep? I mean lower version, of course, does your script work with them too?
Thank you!
On Friday 2009-07-10 15:55, Amerigo Wang wrote:
| Have you checked other version of grep? I mean lower version, of course, | does your script work with them too?
The script works with all grep versions since 2.5, released 2002-03-13, where the -o option was added. The config extraction feature was introduced in Linux 2.6.8, released 2004-08-14, so I don't think this dependency will be a problem.
Signature · 1 line
Dick
Quoted 8 lines · Dick Streefland
On Friday 2009-07-10 15:55, Amerigo Wang wrote:
| Have you checked other version of grep? I mean lower version, of course, | does your script work with them too?The script works with all grep versions since 2.5, released 2002-03-13, where the -o option was added. The config extraction feature was introduced in Linux 2.6.8, released 2004-08-14, so I don't think this dependency will be a problem.
Dick - I lost the original patch.
Can you please resend.
In the changelog please include the information that you tested this with relevant grep versions.
Sam
On Friday 2009-07-17 23:17, Sam Ravnborg wrote:
| Dick - I lost the original patch. | Can you please resend. | | In the changelog please include the information that | you tested this with relevant grep versions.
Sure, here it is:
======================================================================
I've rewritten the extract-ikconfig script to extract the kernel configuration from a kernel compiled with CONFIG_IKCONFIG. The main motivation for the rewrite was to remove the dependency on the external C program binoffset.c, which is compiled on the initial run.
The binoffset executable is invoked with a relative path, which means that the old script can only be run from the top of the kernel tree, and only when you have write permission in the scripts directory. The new script uses tr/grep/tail/zcat only, and can be invoked from anywhere. The binoffset.c program has been removed. This script requires GNU grep 2.5 (released 2002-03-13) or higher, because the -o option was introduced in that version.
---
Documentation/dontdiff | 1 - scripts/.gitignore | 1 - scripts/binoffset.c | 163 ---------------------------------------------- scripts/extract-ikconfig | 113 ++++++++++++------------------- 4 files changed, 44 insertions(+), 234 deletions(-)
Documentation/dontdiff +0 -1
@@ -68,7 +68,6 @@ autoconf.h* bbootsect bin2c binkernel.spec -binoffset bootsect bounds.h bsetup
scripts/.gitignore +0 -1
@@ -6,5 +6,4 @@ kallsyms pnmtologo bin2c unifdef -binoffset ihex2fw
scripts/binoffset.c deleted +0 -163
163 lines not on this page · raw message
scripts/extract-ikconfig +44 -69
122 lines not on this page · raw message
Quoted 24 lines · Dick Streefland
On Friday 2009-07-17 23:17, Sam Ravnborg wrote:
| Dick - I lost the original patch. | Can you please resend. | | In the changelog please include the information that | you tested this with relevant grep versions.Sure, here it is:
======================================================================
I've rewritten the extract-ikconfig script to extract the kernel configuration from a kernel compiled with
CONFIG_IKCONFIG. The main motivation for the rewrite was to remove the dependency on the external C programbinoffset.c, which is compiled on the initial run.The binoffset executable is invoked with a relative path, which means that the old script can only be run from the top of the kernel tree, and only when you have write permission in the scripts directory. The new script uses tr/grep/tail/zcat only, and can be invoked from anywhere. The
binoffset.cprogram has been removed. This script requires GNU grep 2.5 (released 2002-03-13) or higher, because the -o option was introduced in that version.
Quoted 1 line
Signed-off-by Dick Streefland <[email protected]>
Applied to kbuild-next.git.
Sam