Skip to contentSkip to conversation

rename_rev.pl script for reviewing renames

DISCUSSION Dan Carpenter 3 participants 6 messages Feb 3, 2011 – Feb 7, 2011

lore
Share & tools
lore
#0 lore raw

There are a lot of refactoring patches where people change camel case names to kernel style names etc. I've written a script to make it easier to review them. It's attached.

For example the linked patch is about 7900 lines long. It renames A_BOOL to bool, TRUE to true and FALSE to false.
http://driverdev.linuxdriverproject.org/pipermail/devel/2011-February/011810.html

cat email.txt | ./rename_rev.pl A_BOOL bool TRUE true FALSE false | less

The resulting diff is 78 lines long (99% reduced). Woohoo!

regards,
dan carpenter

#!/usr/bin/perl

use File::Temp qw/ :mktemp /;

sub usage() {
    print "cat diff | transform.pl old new old new old new...\n";
    die;
}

my @subs;

sub filter($) {
    my $line = shift();

    foreach my $sub (@subs) {
	$line =~ s/$sub->[0]/$sub->[1]/g;
    }

    # white space at the end of lines
    $line =~ s/ *$//g;
    $line =~ s/\t*$//g;

    # remove the first char
    $line =~ s/^[ +-]//;

    # tabs to spaces
    $line =~ s/\ {8}/\t/g;

    return $line;
}

($oldfh, $oldfile) = mkstemp("/tmp/oldXXXXX");
($newfh, $newfile) = mkstemp("/tmp/newXXXXX");

while (my $param1 = shift()) {
    my $param2 = shift;

    if ($param2 =~ /^$/) {
	usage();
    }
    push @subs, [$param1, $param2];
}
while (<>) {
    my $line = $_;

    if ($line =~ /^---/) {
	next;
    }
    if ($line =~ /^\+\+\+/) {
	next;
    }

    my $output = filter($line);
    if ($line =~ /^-/) {
	print $oldfh $output;
	next;
    }
    if ($line =~ /^\+/) {
	print $newfh $output;
	next;
    }
    print $oldfh $output;
    print $newfh $output;
}

system("diff -u $oldfile $newfile");

unlink($oldfile);
unlink($newfile);

#1 lore raw
Quoted 3 lines · Dan Carpenter

There are a lot of refactoring patches where people change camel case names to kernel style names etc. I've written a script to make it easier to review them. It's attached.

Cool, thanks for sharing. I guess my comments won't matter much, here they are anyway :)

Quoted 2 lines
sub usage() {
    print "cat diff | transform.pl old new old new old new...\n";

Filename of the tool does not match.

Quoted 3 lines
    # white space at the end of lines
    $line =~ s/ *$//g;
    $line =~ s/\t*$//g;

Character class here as well? Will also get mixtures of the two.

Quoted 2 lines
while (<>) {
    my $line = $_;

You could work here with plain $_, but I assume you don't do on purpose.

Quoted 7 lines
    if ($line =~ /^---/) {
	next;
    }
    if ($line =~ /^\+\+\+/) {
	next;
    }

Use an alternation in the regexp?

Quoted 9 lines
    my $output = filter($line);
    if ($line =~ /^-/) {
	print $oldfh $output;
	next;
    }
    if ($line =~ /^\+/) {
	print $newfh $output;
	next;
    }

Ditto.

Regards,

Wolfram

Signature · 2 lines
Pengutronix e.K.                           | Wolfram Sang                |
Industrial Linux Solutions                 | http://www.pengutronix.de/  |
#2 lore raw
Quoted 6 lines · Wolfram Sang
Quoted 3 lines
    # white space at the end of lines
    $line =~ s/ *$//g;
    $line =~ s/\t*$//g;

Character class here as well? Will also get mixtures of the two.

Actually it's better to let diff to take care of white space probably. I changed the diff at the end to "diff -uw" and it seems to work beautifully.

regards,
dan carpenter

#3 lore raw
In reply to Wolfram Sang
Quoted 8 lines · Wolfram Sang
Quoted 3 lines · Dan Carpenter

There are a lot of refactoring patches where people change camel case names to kernel style names etc. I've written a script to make it easier to review them. It's attached.

Cool, thanks for sharing. I guess my comments won't matter much, here they are anyway :)

Thanks for the review. I made the changes.

regards,
dan carpenter

#!/usr/bin/perl

use File::Temp qw/ :mktemp /;

sub usage() {
    print "usage: cat diff | $0 old new old new old new...\n";
    exit(1);
}

my @subs;

sub filter($) {
    my $line = shift();

    foreach my $sub (@subs) {
	$line =~ s/$sub->[0]/$sub->[1]/g;
    }

    # remove the first char
    $line =~ s/^[ +-]//;

    return $line;
}

($oldfh, $oldfile) = mkstemp("/tmp/oldXXXXX");
($newfh, $newfile) = mkstemp("/tmp/newXXXXX");

while (my $param1 = shift()) {
    my $param2 = shift;

    if ($param2 =~ /^$/) {
	usage();
    }
    push @subs, [$param1, $param2];
}
while (<>) {
    my $line = $_;

    if ($line =~ /^(---|\+\+\+)/) {
	next;
    }

    my $output = filter($line);
    if ($line =~ /^-/) {
	print $oldfh $output;
	next;
    }
    if ($line =~ /^\+/) {
	print $newfh $output;
	next;
    }
    print $oldfh $output;
    print $newfh $output;
}

system("diff -uw $oldfile $newfile");

unlink($oldfile);
unlink($newfile);

#4 lore raw

I've added the some flags:
-e : gets executed on old lines
-ea: executed on everything
-nc: strips one line comments

So say you have a patch that adds parenthesis around a bunch of macros like this:
http://marc.info/?l=linux-kernel&m=129685142419550&w=2

Then you can just type:

cat /home/dcarpenter/tmp/html23/parens.patch | \
	./rename_rev.pl -e 's/(define \w+) (.*)/$1 ($2)/'

Or if you have a camel case script that changes "ThisVariable" to "this_variable". Then the command would be:
git show c659c38 | ./rename_rev.pl -ea '$_ = lc' -ea 's/_//g' Which changes everything to lower case and strips out all the underscores. You might want to combine it with some other flags:

git show c659c38 | ./rename_rev.pl -nc \
	-e 's/TLanPrivateInfo/struct tlan_priv/' \
	-e 's/TLanList/struct tlan_list/' \
	-ea '$_ = lc' -ea 's/_//g'

What I would like is if there was some way to ignore changes which just introduced new lines, but didn't affect runtime behavior. I'm not sure how to do that.

regards,
dan carpenter

#!/usr/bin/perl

use strict;
use File::Temp qw/ :mktemp /;

sub usage() {
    print "usage: cat diff | $0 old new old new old new...\n";
    print "   or: cat diff | $0 -e 's/old/new/g'\n"; 
    print " -e : execute on old lines\n";
    print " -ea: execute on all lines\n";
    print " -nc: strip one line comments\n"; 
    exit(1);
}

my @subs;
my @cmds;
my $strip_comments;

sub filter($) {
    my $_ = shift();

    my $old = 0;
    if ($_ =~ /^-/) {
        $old = 1;
    }

    # remove the first char
    s/^[ +-]//;

    if ($strip_comments) {
        s/\/\*.*?\*\///g;
        s/\/\/.*//;
    }

    foreach my $cmd (@cmds) {
        if ($old || $cmd->[0] =~ /^-ea$/) {
		eval $cmd->[1];
	}
    }

    foreach my $sub (@subs) {
	if ($old) {
		s/$sub->[0]/$sub->[1]/g;
	}
    }

    return $_;
}

my ($oldfh, $oldfile) = mkstemp("/tmp/oldXXXXX");
my ($newfh, $newfile) = mkstemp("/tmp/newXXXXX");

while (my $param1 = shift()) {
    if ($param1 =~ /^-nc$/) {
        $strip_comments = 1;
        next;
    }

    my $param2 = shift;
    if ($param2 =~ /^$/) {
	usage();
    }

    if ($param1 =~ /^-e(a|)$/) {
        push @cmds, [$param1, $param2];
	next;
    }
    push @subs, [$param1, $param2];
}
while (<>) {
    my $line = $_;

    if ($line =~ /^(---|\+\+\+)/) {
	next;
    }

    my $output = filter($line);
    if ($line =~ /^-/) {
	print $oldfh $output;
	next;
    }
    if ($line =~ /^\+/) {
	print $newfh $output;
	next;
    }
    print $oldfh $output;
    print $newfh $output;
}

system("diff -uw $oldfile $newfile");

unlink($oldfile);
unlink($newfile);

#5 lore raw
Quoted 9 lines · Dan Carpenter

Or if you have a camel case script that changes "ThisVariable" to "this_variable". Then the command would be:
git show c659c38 | ./rename_rev.pl -ea '$_ = lc' -ea 's/_//g' Which changes everything to lower case and strips out all the underscores. You might want to combine it with some other flags:

git show c659c38 | ./rename_rev.pl -nc \
	-e 's/TLanPrivateInfo/struct tlan_priv/' \
	-e 's/TLanList/struct tlan_list/' \
	-ea '$_ = lc' -ea 's/_//g'

Hi Dan.

I think you'll need to add '\b$1\b' to your tests otherwise your first example will check things like "TLanPrivateInfoType" as well.

Quoted 3 lines

What I would like is if there was some way to ignore changes which just introduced new lines, but didn't affect runtime behavior. I'm not sure how to do that.

Any object change will affect runtime behavior.