Skip to contentSkip to conversation

case-duplicated filenames

DISCUSSION Tony Mantler 5 participants 8 messages Aug 11, 2009

lore
Share & tools
lore
#0 lore raw

A co-worker reminded me this afternoon that the linux source tree does not get along well with case-insensitive filesystems. Ever the curious and bored sort, and in the middle of an sqlite bender, I decided to cook up a quick script to find exactly what files those are.

The results (in lowercase) are as follows:
linux-source-2.6.30/include/linux/netfilter/xt_mark.h linux-source-2.6.30/include/linux/netfilter/xt_connmark.h linux-source-2.6.30/include/linux/netfilter/xt_dscp.h linux-source-2.6.30/include/linux/netfilter/xt_rateest.h linux-source-2.6.30/include/linux/netfilter/xt_tcpmss.h linux-source-2.6.30/include/linux/netfilter_ipv6/ip6t_mark.h linux-source-2.6.30/include/linux/netfilter_ipv6/ip6t_hl.h linux-source-2.6.30/include/linux/netfilter_ipv4/ipt_ttl.h linux-source-2.6.30/include/linux/netfilter_ipv4/ipt_tos.h linux-source-2.6.30/include/linux/netfilter_ipv4/ipt_ecn.h linux-source-2.6.30/include/linux/netfilter_ipv4/ipt_connmark.h linux-source-2.6.30/include/linux/netfilter_ipv4/ipt_dscp.h linux-source-2.6.30/include/linux/netfilter_ipv4/ipt_tcpmss.h linux-source-2.6.30/include/linux/netfilter_ipv4/ipt_mark.h linux-source-2.6.30/net/netfilter/xt_tcpmss.c linux-source-2.6.30/net/netfilter/xt_connmark.c linux-source-2.6.30/net/netfilter/xt_mark.c linux-source-2.6.30/net/netfilter/xt_dscp.c linux-source-2.6.30/net/netfilter/xt_rateest.c linux-source-2.6.30/net/netfilter/xt_hl.c linux-source-2.6.30/net/ipv4/netfilter/ipt_ecn.c linux-source-2.6.30/documentation/io-mapping.txt

And the script, if you'd like to run it yourself, is as follows (hereby released GPL2):
#!/bin/bash

#set -x

SQLITE3=which sqlite3
DBNAME=filenames.db
SQLFILE=filenames.sql

DIRECTORY=linux-source-2.6.30

$SQLITE3 $DBNAME "CREATE TABLE filenames ( id INTEGER PRIMARY KEY, name TEXT );"

echo "PRAGMA synchronous=OFF;" >$SQLFILE
echo "BEGIN TRANSACTION;" >>$SQLFILE
find $DIRECTORY | tr '[:upper:]' '[:lower:]' | sed 's/^\(.*\)/INSERT INTO filenames (name) VALUES ("\1");/' >>$SQLFILE
echo "COMMIT;" >>$SQLFILE

$SQLITE3 $DBNAME <$SQLFILE

$SQLITE3 $DBNAME "SELECT a.name FROM filenames a, filenames b WHERE a.name == b.name AND a.id < b.id;"

rm $SQLFILE
rm $DBNAME

#1 lore raw
Quoted 5 lines · Tony Mantler

A co-worker reminded me this afternoon that the linux source tree does not get along well with case-insensitive filesystems. Ever the curious and bored sort, and in the middle of an sqlite bender, I decided to cook up a quick script to find exactly what files those are.

Such an extravagant process: you're just being silly. There appears to be a line missing:

find $DIRECTORY | sed 's/^\(.*\)/INSERT INTO filenames (name) VALUES ("\1");/' >>$SQLFILE

#2 lore raw
Quoted 8 lines · David Newall
Quoted 5 lines · Tony Mantler

A co-worker reminded me this afternoon that the linux source tree does not get along well with case-insensitive filesystems. Ever the curious and bored sort, and in the middle of an sqlite bender, I decided to cook up a quick script to find exactly what files those are.

Such an extravagant process: you're just being silly.

Well I did say I was bored. ;)

Quoted 3 lines

There appears to be a line missing:

find $DIRECTORY | sed 's/^\(.*\)/INSERT INTO filenames (name) VALUES ("\1");/' >>$SQLFILE

?

#3 lore raw
Quoted 9 lines · Tony Mantler
Quoted 8 lines · David Newall
Quoted 6 lines · Tony Mantler

A co-worker reminded me this afternoon that the linux source tree does not
get along well with case-insensitive filesystems. Ever the curious and bored sort, and in the middle of an sqlite bender, I decided to cook up a quick script to find exactly what files those are.

Quoted 3 lines
Quoted 1 line

Such an extravagant process: you're just being silly.

Well I did say I was bored. ;)

Performance aside, it's a one-liner in shell* ;).

find | sort | uniq -ic | grep -v " 1 "

Alan

[*] Posix compliance not guaranteed. I don't know how standard uniq -i is.

#4 lore raw
Quoted 12 lines · Alan Jenkins
Quoted 11 lines · Tony Mantler
Quoted 9 lines · David Newall
Quoted 6 lines · Tony Mantler

A co-worker reminded me this afternoon that the linux source tree does not
get along well with case-insensitive filesystems. Ever the curious and bored sort, and in the middle of an sqlite bender, I decided to cook up a quick script to find exactly what files those are.

Such an extravagant process: you're just being silly.

Quoted 7 lines
Quoted 1 line

Well I did say I was bored. ;)

Performance aside, it's a one-liner in shell* ;).

find | sort | uniq -ic | grep -v " 1 "

[*] Posix compliance not guaranteed. I don't know how standard uniq -i is.

It's not standard. You're also assuming case-insensitive sort. Using sort -f increases the chances of this doing what you want, but it still depends on the locale settings.

Signature · 2 lines
Måns Rullgård
[email protected]
#5 lore raw
In reply to Tony Mantler
Quoted 7 lines · Tony Mantler
Quoted 4 lines · David Newall

There appears to be a line missing:

find $DIRECTORY | sed 's/^\(.*\)/INSERT INTO filenames (name) VALUES ("\1");/' >>$SQLFILE

?

Sorry. I just now realised what it does.

#6 lore raw
In reply to Tony Mantler
Quoted 11 lines · Tony Mantler

A co-worker reminded me this afternoon that the linux source tree does not get along well with case-insensitive filesystems. Ever the curious and bored sort, and in the middle of an sqlite bender, I decided to cook up a quick script to find exactly what files those are.

The results (in lowercase) are as follows:
[22 files]

And the script, if you'd like to run it yourself, is as follows (hereby released GPL2):
[SQL]

Everybody know the shell is the hackers best friend:

$ git ls-files | tr A-Z a-z | sort | uniq -d
documentation/io-mapping.txt
include/linux/netfilter/xt_connmark.h
include/linux/netfilter/xt_dscp.h
include/linux/netfilter/xt_mark.h
include/linux/netfilter/xt_rateest.h
include/linux/netfilter/xt_tcpmss.h
include/linux/netfilter_ipv4/ipt_connmark.h
include/linux/netfilter_ipv4/ipt_dscp.h
include/linux/netfilter_ipv4/ipt_ecn.h
include/linux/netfilter_ipv4/ipt_mark.h
include/linux/netfilter_ipv4/ipt_tcpmss.h
include/linux/netfilter_ipv4/ipt_tos.h
include/linux/netfilter_ipv4/ipt_ttl.h
include/linux/netfilter_ipv6/ip6t_hl.h
include/linux/netfilter_ipv6/ip6t_mark.h
net/ipv4/netfilter/ipt_ecn.c
net/netfilter/xt_connmark.c
net/netfilter/xt_dscp.c
net/netfilter/xt_hl.c
net/netfilter/xt_mark.c
net/netfilter/xt_rateest.c
net/netfilter/xt_tcpmss.c

That runs in about 0.03 seconds here.

Signature · 2 lines
Måns Rullgård
[email protected]
#7 lore raw
In reply to Tony Mantler
Quoted 5 lines · Tony Mantler

A co-worker reminded me this afternoon that the linux source tree does not get along well with case-insensitive filesystems. Ever the curious and bored sort, and in the middle of an sqlite bender, I decided to cook up a quick script to find exactly what files those are.

This one-liner does it too:

find linux-2.6.30.4 |tr '[:upper:]' '[:lower:]' | sort | uniq -d