Skip to contentSkip to conversation

don't diff generated firmware files

path New Arjan van de Ven 6 participants 11 messages Sep 1, 2008 – Sep 16, 2008 1 message not in the archive

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

With the new firmware infrastructure in 2.6.27, some files are generated and shouldn't be diffed; add these 2 to the "dontdiff" file

Documentation/dontdiff +2 -0
diff -purN --exclude-from=linux.trees.git/Documentation/dontdiff /home/arjan/linux.trees.git/Documentation/dontdiff linux.trees.git/Documentation/dontdiff
@@ -5,6 +5,8 @@
 *.css
 *.dvi
 *.eps
+*.fw.gen.S
+*.fw
 *.gif
 *.grep
 *.grp
#1 lore raw

Documentation/dontdiff line 9

Quoted 11 lines · Arjan van de Ven

With the new firmware infrastructure in 2.6.27, some files are generated and shouldn't be diffed; add these 2 to the "dontdiff" file

Documentation/dontdiff +2 -0
@@ -5,6 +5,8 @@
 *.css
 *.dvi
 *.eps
+*.fw.gen.S
+*.fw

Dontdiff is obsoleted because of .gitignore and should probably be auto-generated from them.

#2 lore raw
Quoted 4 lines · On Tue, 2 Sep 2008 03:14:19 +0400 Alexey Dobriyan wrote:
Quoted 2 lines · Arjan van de Ven

With the new firmware infrastructure in 2.6.27, some files are generated and shouldn't be diffed; add these 2 to the "dontdiff" file

Quoted 11 lines

Documentation/dontdiff line 9

Quoted 8 lines
Documentation/dontdiff +2 -0
@@ -5,6 +5,8 @@
 *.css
 *.dvi
 *.eps
+*.fw.gen.S
+*.fw

Dontdiff is obsoleted because of .gitignore and should probably be auto-generated from them.

Let's please not assume that everyone uses git nor force that.

---
~Randy
Linux Plumbers Conference, 17-19 September 2008, Portland, Oregon USA
http://linuxplumbersconf.org/

#3 lore raw

PATCH 1/1 dontdiff: more updates to be closer to gitignore

In reply to Alexey Dobriyan

defkeymap.c_shipped should be diffed if it is changed.

COPYING, CREDITS, .mailmap should be diffed if they are changed. keywords.c_shipped & lex.c_shipped should be diffed when changed. parse.[ch]_shipped should be diffed when changed.

vsyscall* updates from a .gitignore patch by "Denis V. Lunev" <[email protected]>.

*.so.dbg from a .gitignore patch by Thomas Gleixner <[email protected]>.

binoffset from a .gitignore patch by Uwe Kleine-K=F6nig <[email protected]>.

Module.markers from a .gitignore patch by Matthew Wilcox <[email protected]>.

vmlinux*.lds* should be diffed if changed.

vmlinux.lds from a .gitignore patch by Daniel Guilak <[email protected]>.

*.scr should be diffed if changed.

Lots of updates from http://lkml.org/lkml/2008/5/20/32

Use ncscope.* instead of *cscope* since the latter may catch too many files.

Add *.elf, from a .gitignore patch by Eduard - Gabriel Munteanu <[email protected]>.

Make firmware entries match .gitignore entries.

Make some entries less greedy by removing trailing '*'.

Remove "make_times_h" (no such file).
Remove "filelist" (no such file).
Remove "dummy_sym.c" (no such file).
Remove "gen-kdb_cmds.c" (no such file).
Remove "gentbl" (no such file).
Remove "kconfig.tk" (no such file).
Remove "tkparse" (no such file).
Remove "sim710_d.h" (no such file).
Remove "53c8xx_d.h" (no such file).
Add "syscalltab.h" (generated file).

---
 Documentation/dontdiff |   59 +++++++++++++++++++++++++++----------------------
 1 file changed, 33 insertions(+), 26 deletions(-)
Documentation/dontdiff +33 -26
@@ -2,11 +2,13 @@
 *.aux
 *.bin
 *.cpio
-*.css
+*.csp
+*.dsp
 *.dvi
+*.elf
 *.eps
-*.fw.gen.S
 *.fw
+*.gen.S
 *.gif
 *.grep
 *.grp
@@ -30,6 +32,7 @@
 *.s
 *.sgml
 *.so
+*.so.dbg
 *.symtypes
 *.tab.c
 *.tab.h
@@ -38,24 +41,17 @@
 *.xml
 *_MODULES
 *_vga16.c
-*cscope*
 *~
 *.9
 *.9.gz
 .*
-.cscope
-.gitignore
-.mailmap
 .mm
 53c700_d.h
-53c8xx_d.h*
-COPYING
-CREDITS
 CVS
 ChangeSet
 Image
 Kerntypes
-MODS.txt
+Module.markers
 Module.symvers
 PENDING
 SCCS
@@ -73,7 +69,9 @@ autoconf.h*
 bbootsect
 bin2c
 binkernel.spec
+binoffset
 bootsect
+bounds.h
 bsetup
 btfixupprep
 build
@@ -89,39 +87,36 @@ config_data.h*
 config_data.gz*
 conmakehash
 consolemap_deftbl.c*
+cpustr.h
 crc32table.h*
 cscope.*
-defkeymap.c*
+defkeymap.c
 devlist.h*
 docproc
-dummy_sym.c*
 elf2ecoff
 elfconfig.h*
-filelist
 fixdep
 fore200e_mkfirm
 fore200e_pca_fw.c*
 gconf
 gen-devlist
-gen-kdb_cmds.c*
 gen_crc32table
 gen_init_cpio
 genksyms
-gentbl
 *_gray256.c
+ihex2fw
 ikconfig.h*
 initramfs_data.cpio
 initramfs_data.cpio.gz
 initramfs_list
 kallsyms
 kconfig
-kconfig.tk
-keywords.c*
+keywords.c
 ksym.c*
 ksym.h*
 kxgettext
 lkc_defs.h
-lex.c*
+lex.c
 lex.*.c
 logo_*.c
 logo_*_clut224.c
@@ -130,7 +125,6 @@ lxdialog
 mach-types
 mach-types.h
 machtypes.h
-make_times_h
 map
 maui_boot.h
 mconf
@@ -138,6 +132,7 @@ miboot*
 mk_elfconfig
 mkboot
 mkbugboot
+mkcpustr
 mkdep
 mkprep
 mktables
@@ -145,11 +140,12 @@ mktree
 modpost
 modules.order
 modversions.h*
+ncscope.*
 offset.h
 offsets.h
 oui.c*
-parse.c*
-parse.h*
+parse.c
+parse.h
 patches*
 pca200e.bin
 pca200e_ecd.bin2
@@ -157,7 +153,7 @@ piggy.gz
 piggyback
 pnmtologo
 ppc_defs.h*
-promcon_tbl.c*
+promcon_tbl.c
 pss_boot.h
 qconf
 raid6altivec*.c
@@ -168,27 +164,38 @@ series
 setup
 setup.bin
 setup.elf
-sim710_d.h*
 sImage
 sm_tbl*
 split-include
+syscalltab.h
 tags
 tftpboot.img
 timeconst.h
 times.h*
-tkparse
 trix_boot.h
 utsrelease.h*
+vdso-syms.lds
 vdso.lds
+vdso32-int80-syms.lds
+vdso32-syms.lds
+vdso32-syscall-syms.lds
+vdso32-sysenter-syms.lds
+vdso32.lds
+vdso32.so.dbg
+vdso64.lds
+vdso64.so.dbg
 version.h*
 vmlinux
 vmlinux-*
 vmlinux.aout
-vmlinux*.lds*
-vmlinux*.scr
+vmlinux.lds
 vsyscall.lds
+vsyscall_32.lds
 wanxlfw.inc
 uImage
 unifdef
+wakeup.bin
+wakeup.elf
+wakeup.lds
 zImage*
 zconf.hash.c
#4 lore raw

[RFC/PATCH] dontdiff: generate from gitignore

In reply to Alexey Dobriyan

From: Randy Dunlap <[email protected]>

Generate the "dontdiff" file from the .gitignore files. Save it in scripts/dontdiff.

---
 Makefile           |   10 +++++++++-
 scripts/.gitignore |    1 +
 scripts/mkdontdiff |   23 +++++++++++++++++++++++
 3 files changed, 33 insertions(+), 1 deletion(-)
Makefile +9 -1
@@ -398,7 +398,7 @@ endif
 no-dot-config-targets := clean mrproper distclean \
 			 cscope TAGS tags help %docs check% \
 			 include/linux/version.h headers_% \
-			 kernelrelease kernelversion
+			 kernelrelease kernelversion dontdiff
 
 config-targets := 0
 mixed-targets  := 0
@@ -1251,6 +1251,7 @@ help:
 	@echo  '  dir/            - Build all files in dir and below'
 	@echo  '  dir/file.[ois]  - Build specified target only'
 	@echo  '  dir/file.ko     - Build module including final link'
+	@echo  '  dontdiff        - Create a "dontdiff" file'
 	@echo  '  prepare         - Set up for building external modules'
 	@echo  '  tags/TAGS	  - Generate tags file for editors'
 	@echo  '  cscope	  - Generate cscope index'
@@ -1519,6 +1520,13 @@ endef
 tags: FORCE
 	$(call cmd,tags)
 
+quiet_cmd_dontdiff = MAKE    dontdiff
+      cmd_dontdiff = $(shell $(CONFIG_SHELL) \
+	                  $(srctree)/scripts/mkdontdiff $(srctree))
+
+dontdiff: FORCE
+	$(call cmd,dontdiff)
+
 
 # Scripts to check various things for consistency
 # ---------------------------------------------------------------------------
scripts/mkdontdiff new file +23 -0
@@ -0,0 +1,23 @@
+#! /bin/sh
+set -f
+
+TMPFILE=`mktemp ./.tmpdd.XXXXXX`
+srctree=$1
+DDFILE=$srctree/scripts/dontdiff
+
+find . -name .gitignore | xargs cat | grep -v '^[#!]' | grep -v '^[:space:]*$' | sort | uniq > $TMPFILE
+
+rm -f $DDFILE
+touch $DDFILE
+
+while read PATTERN ; do
+	if echo "$PATTERN" | grep -q '/' ; then
+		filename=`basename $PATTERN`
+	else
+		filename="$PATTERN"
+	fi
+
+	echo $filename >> $DDFILE
+done < $TMPFILE
+
+rm $TMPFILE
scripts/.gitignore +1 -0
@@ -2,6 +2,7 @@
 # Generated files
 #
 conmakehash
+dontdiff
 kallsyms
 pnmtologo
 bin2c
#5 lore raw

Re: [RFC/PATCH] dontdiff: generate from gitignore

Quoted 2 lines · Randy Dunlap

Generate the "dontdiff" file from the .gitignore files.

t's not quite as simple as that, though.

The .gitignore files are fundamentally location hierarchy-aware, so a ignore entry in one subdirectory only affects that subdirectory (recursively), so if you flatten then, you should take that into account.

As far as I can tell, your script will generate lots of incorrect entries due to this. Eg, it will generate

	parse.[ch]

as a dontdiff pattern, because scripts/genksyms/.gitignore has that, but that means that now it will ignore parse.c in all the _other_ places, where it *isn't* a generated file (ie mm/parse.c).

Linus

#6 lore raw

Re: [RFC/PATCH] dontdiff: generate from gitignore

Quoted 10 lines · Linus Torvalds
Quoted 1 line · Randy Dunlap

Generate the "dontdiff" file from the .gitignore files.

t's not quite as simple as that, though.

The .gitignore files are fundamentally location hierarchy-aware, so a ignore entry in one subdirectory only affects that subdirectory (recursively), so if you flatten then, you should take that into account.

Quoted 8 lines

As far as I can tell, your script will generate lots of incorrect entries due to this. Eg, it will generate

	parse.[ch]

as a dontdiff pattern, because scripts/genksyms/.gitignore has that, but that means that now it will ignore parse.c in all the _other_ places, where it *isn't* a generated file (ie mm/parse.c).

Yes, that's one of the reasons that it contains "RFC" in $subject. There's also no handling of !pattern .gitignore lines....

AFAIK, diff with a dontdiff file does not allow/support full pathname hierarchies like .gitignore does, so this is probably a futile exercise. ??

Signature · 3 lines
~Randy
Linux Plumbers Conference, 17-19 September 2008, Portland, Oregon USA
http://linuxplumbersconf.org/
#7 lore raw

Re: [RFC/PATCH] dontdiff: generate from gitignore

Quoted 20 lines · Randy Dunlap
Quoted 10 lines · Linus Torvalds
Quoted 1 line · Randy Dunlap

Generate the "dontdiff" file from the .gitignore files.

t's not quite as simple as that, though.

The .gitignore files are fundamentally location hierarchy-aware, so a ignore entry in one subdirectory only affects that subdirectory (recursively), so if you flatten then, you should take that into account.

Quoted 8 lines

As far as I can tell, your script will generate lots of incorrect entries due to this. Eg, it will generate

      parse.[ch]

as a dontdiff pattern, because scripts/genksyms/.gitignore has that, but that means that now it will ignore parse.c in all the _other_ places, where it *isn't* a generated file (ie mm/parse.c).

Quoted 5 lines

Yes, that's one of the reasons that it contains "RFC" in $subject. There's also no handling of !pattern .gitignore lines....

AFAIK, diff with a dontdiff file does not allow/support full pathname hierarchies like .gitignore does, so this is probably a futile exercise. ??

I'm presuming you're talking about this diff option?

`-X FILE'
`--exclude-from=FILE'
When comparing directories, ignore files and subdirectories whose basenames match any pattern contained in FILE. *Note Comparing Directories::.

Indeed: it clearly says "basename" here.

If it weren't for that, you could just use git-ls-files --ignore to find them ...

A message that is not in the archive (sent off-list or not yet received): [email protected]

#9 lore raw

RFC PATCH 1/1 Script for generating Documentation/dontdiff from .gitignore files.

.gitignore files get more testing than Documentation/dontdiff, since most developers follow Git workflow when sending patches. The latter tends to stay outdated and needs to be synchronized every now and then. This patch provides a script that scans all .gitignore files and generates a Documentation/dontdiff to stdout.

---
 scripts/gen_dontdiff.sh |   21 +++++++++++++++++++++
 1 files changed, 21 insertions(+), 0 deletions(-)
 create mode 100755 scripts/gen_dontdiff.sh
scripts/gen_dontdiff.sh new file +21 -0
new file mode 100755
index 0000000..cd94747
@@ -0,0 +1,21 @@
+#!/bin/bash
+
+# Copyright (C) 2008 Eduard - Gabriel Munteanu
+#
+# This file is released under GPL version 2.
+
+echo "### Generated by gen_dontdiff.sh ###"
+
+grep -Ev "^([[:space:]]*[[#]]*|$)" .gitignore
+
+find . -mindepth 2 -name .gitignore | sed -e "s/^\.\///g" | while read FILE
+do
+	CURR_DIR=`echo "$FILE" | sed -e "s/\.gitignore$//g"`
+	echo "# $FILE"
+	grep -Ev "^([[:space:]]*[[#]]*|$)" $FILE | while read LINE
+	do
+		echo $CURR_DIR$LINE
+	done
+	echo ""
+done
+
Signature · 1 line
1.5.6.1
#10 lore raw

Re: [RFC PATCH] Script for generating Documentation/dontdiff from .gitignore files.

Quoted 7 lines · On Sun, 14 Sep 2008 21:23:57 +0300 Eduard - Gabriel Munteanu wrote:

.gitignore files get more testing than Documentation/dontdiff, since most developers follow Git workflow when sending patches. The latter tends to stay outdated and needs to be synchronized every now and then. This patch provides a script that scans all .gitignore files and generates a Documentation/dontdiff to stdout.

Hi,
You probably missed (& should read) this thread:
http://lkml.org/lkml/2008/9/12/154

Quoted 23 lines
---
 scripts/gen_dontdiff.sh |   21 +++++++++++++++++++++
 1 files changed, 21 insertions(+), 0 deletions(-)
 create mode 100755 scripts/gen_dontdiff.sh
scripts/gen_dontdiff.sh new file +11 -0
new file mode 100755
index 0000000..cd94747
@@ -0,0 +1,21 @@
+#!/bin/bash
+
+# Copyright (C) 2008 Eduard - Gabriel Munteanu
+#
+# This file is released under GPL version 2.
+
+echo "### Generated by gen_dontdiff.sh ###"
+
+grep -Ev "^([[:space:]]*[[#]]*|$)" .gitignore
+
+find . -mindepth 2 -name .gitignore | sed -e "s/^\.\///g" | while read FILE

scripts/gen_dontdiff.sh line 21

Quoted 10 lines
scripts/gen_dontdiff.sh new file +10 -0
+do
+	CURR_DIR=`echo "$FILE" | sed -e "s/\.gitignore$//g"`
+	echo "# $FILE"
+	grep -Ev "^([[:space:]]*[[#]]*|$)" $FILE | while read LINE
+	do
+		echo $CURR_DIR$LINE
+	done
+	echo ""
+done
+

---
~Randy
Linux Plumbers Conference, 17-19 September 2008, Portland, Oregon USA
http://linuxplumbersconf.org/

#11 lore raw

Re: [RFC PATCH] Script for generating Documentation/dontdiff from .gitignore files. (was Re: [RFC/PATCH] dontdiff: generate from gitignore)

Quoted 8 lines · Randy Dunlap
Quoted 6 lines · On Sun, 14 Sep 2008 21:23:57 +0300 Eduard - Gabriel Munteanu wrote:

.gitignore files get more testing than Documentation/dontdiff, since most developers follow Git workflow when sending patches. The latter tends to stay outdated and needs to be synchronized every now and then. This patch provides a script that scans all .gitignore files and generates a Documentation/dontdiff to stdout.

Quoted 7 lines
Quoted 1 line

Hi,
You probably missed (& should read) this thread:
http://lkml.org/lkml/2008/9/12/154

Thanks, it makes sense.

It seems like diff can't handle such scenarios at all. I suppose Linux should supply its own diff tool (which uses regular diff, but implements proper file matching).

Maybe git-diff alone could be included in the tarballs, I'll look into it.

Cheers,
Eduard

Quoted 38 lines
Quoted 23 lines
---
 scripts/gen_dontdiff.sh |   21 +++++++++++++++++++++
 1 files changed, 21 insertions(+), 0 deletions(-)
 create mode 100755 scripts/gen_dontdiff.sh
scripts/gen_dontdiff.sh new file +11 -0
new file mode 100755
index 0000000..cd94747
@@ -0,0 +1,21 @@
+#!/bin/bash
+
+# Copyright (C) 2008 Eduard - Gabriel Munteanu
+#
+# This file is released under GPL version 2.
+
+echo "### Generated by gen_dontdiff.sh ###"
+
+grep -Ev "^([[:space:]]*[[#]]*|$)" .gitignore
+
+find . -mindepth 2 -name .gitignore | sed -e "s/^\.\///g" | while read FILE

scripts/gen_dontdiff.sh line 21

Quoted 10 lines
scripts/gen_dontdiff.sh new file +10 -0
+do
+	CURR_DIR=`echo "$FILE" | sed -e "s/\.gitignore$//g"`
+	echo "# $FILE"
+	grep -Ev "^([[:space:]]*[[#]]*|$)" $FILE | while read LINE
+	do
+		echo $CURR_DIR$LINE
+	done
+	echo ""
+done
+

---
~Randy
Linux Plumbers Conference, 17-19 September 2008, Portland, Oregon USA
http://linuxplumbersconf.org/