diff: simplify cpp funcname regex
The current funcname matcher for C files requires one or more words before the function name, like:
static int foo(int arg)
{However, some coding styles look like this:
static int
foo(int arg)
{and we do not match, even though the default regex would.
This patch simplifies the regex significantly. Rather than trying to handle all the variants of keywords and return types, we simply look for an identifier at the start of the line that contains a "(", meaning it is either a function definition or a function call, and then not containing ";" which would indicate it is a call or declaration.
This can be fooled by something like:
if (foo)
but it is unlikely to find that at the very start of a line with no identation (and without having a complete set of keywords, we cannot distinguish that from a function called "if" taking "foo" anyway).
We had no tests at all for .c files, so this attempts to add a few obvious cases (including the one we are fixing here).
---
I tried accommodating this one case in the current regex, but it just kept getting more complicated and unreadable. Maybe I am being naive to think that this much simpler version can work. We don't have a lot of tests or a known-good data set to check.
I did try "git log -1000 -p" before and after on git.git, diff'd the results and manually inspected. The results were a mix of strict improvement (mostly instances of the style I was trying to fix here) and cases where there is no good answer. For example, for top-level changes outside functions, we might find:
N_("some text that is long"
that is part of:
const char *foo =
N_("some text that is long"
"and spans multiple lines");
Before this change, we would skip past it (using the cpp regex, that is; the default one tends to find the same line) and either report nothing, or whatever random function was before us. So it's a behavior change, but the existing behavior is really no better.
t/t4018-diff-funcname.sh | 36 ++++++++++++++++++++++++++++++++++++ userdiff.c | 2 +- 2 files changed, 37 insertions(+), 1 deletion(-)
t/t4018-diff-funcname.sh +36 -0
@@ -93,6 +93,29 @@ sed -e ' s/song;/song();/ ' <Beer.perl >Beer-correct.perl +cat >Beer.c <<\EOF +static int foo(void) +{ +label: + int x = old; +} + +struct foo; /* catch failure below */ +static int +gnu(int arg) +{ + int x = old; +} + +struct foo; /* catch failure below */ +int multiline(int arg, + char *arg2) +{ + int x = old; +} +EOF +sed s/old/new/ <Beer.c >Beer-correct.c + test_expect_funcname () { lang=${2-java} test_expect_code 1 git diff --no-index -U1 \
@@ -127,6 +150,7 @@ test_expect_success 'set up .gitattributes declaring drivers to test' ' cat >.gitattributes <<-\EOF *.java diff=java *.perl diff=perl + *.c diff=cpp EOF '
@@ -158,6 +182,18 @@ test_expect_success 'perl pattern is not distracted by forward declaration' ' test_expect_funcname "package Beer;\$" perl ' +test_expect_success 'c pattern skips labels' ' + test_expect_funcname "static int foo(void)" c +' + +test_expect_success 'c pattern matches GNU-style functions' ' + test_expect_funcname "gnu(int arg)\$" c +' + +test_expect_success 'c pattern matches multiline functions' ' + test_expect_funcname "int multiline(int arg,\$" c +' + test_expect_success 'custom pattern' ' test_config diff.java.funcname "!static !String
userdiff.c +1 -1
@@ -127,7 +127,7 @@/* Jump targets or access declarations */ "!^[ \t]*[A-Za-z_][A-Za-z_0-9]*:.*$\n" /* C/++ functions/methods at top level */ -"^([A-Za-z_][A-Za-z_0-9]*([ \t*]+[A-Za-z_][A-Za-z_0-9]*([ \t]*::[ \t]*[^[:space:]]+)?){1,}[ \t]*\\([^;]*)$\n" +"^([A-Za-z_].*\\([^;]*)$\n" /* compound type at top level */ "^((struct|class|enum)[^;]*)$",/* -- */
Signature · 1 line
1.8.5.2.500.g8060133
Am 3/5/2014 1:36, schrieb Jeff King:
Quoted 12 lines
The current funcname matcher for C files requires one or more words before the function name, like:
static int foo(int arg) {However, some coding styles look like this:
static int foo(int arg) {
Quoted 8 lines
and we do not match, even though the default regex would.
This patch simplifies the regex significantly. Rather than trying to handle all the variants of keywords and return types, we simply look for an identifier at the start of the line that contains a "(", meaning it is either a function definition or a function call, and then not containing ";" which would indicate it is a call or declaration.
Here is a patch that I'm carrying around since... a while. What do you think?
The pattern I chose also catches variable definition, not just functions. That is what I need, but it hurts grep --function-context That's the reason I didn't forward the patch, yet.
--- 8< ---
From: Johannes Sixt <[email protected]>
Date: Tue, 25 Sep 2012 14:08:02 +0200
Subject: [PATCH] userdiff: have 'cpp' hunk header pattern catch more C++ anchor points
The hunk header pattern 'cpp' is intended for C and C++ source code, but it is actually not very useful for the latter, and even hurts some use-cases for the former.
The parts of the pattern have the following flaws:
- The first part matches an identifier followed immediately by a colon and arbitrary text and is intended to reject goto labels and C++ access specifiers (public, private, protected). But this pattern also rejects C++ constructs, which look like this:
MyClass::MyClass()
MyClass::~MyClass()
MyClass::Item MyClass::Find(...- The second part matches an identifier followed by a list of qualified names (i.e. identifiers separated by the C++ scope operator '::') separated by space or '*' followed by an opening parenthesis (with space between the tokens). It matches function declarations like
struct item* get_head(...
int Outer::Inner::Func(...
Since the pattern requires at least two identifiers, GNU-style function definitions are ignored:
void
func(...Moreover, since the pattern does not allow punctuation other than '*', the following C++ constructs are not recognized:
. template definitions:
template<class T> int func(T arg)
. functions returning references:
const string& get_message()
. functions returning templated types:
vector<int> foo()
. operator definitions:
Value operator+(Value l, Value r)
- The third part of the pattern finally matches compound definitions. But it forgets about unions and namespaces, and also skips single-line definitions
struct random_iterator_tag {};because no semicolon can occur on the line.
Change the first pattern to require a colon at the end of the line (except for trailing space and comments), so that it does not reject constructor or destructor definitions.
Notice that all interesting anchor points begin with an identifier or keyword. But since there is a large variety of syntactical constructs after the first "word", the simplest is to require only this word and accept everything else. Therefore, this boils down to a line that begins with a letter or underscore (optionally preceded by the C++ scope operator '::' to accept functions returning a type anchored at the global namespace). Replace the second and third part by a single pattern that picks such a line.
This has the following desirable consequence:
- All constructs mentioned above are recognized.
and the following likely desirable consequences:
- Definitions of global variables and typedefs are recognized:
int num_entries = 0;
extern const char* help_text;
typedef basic_string<wchar_t> wstring;- Commonly used marco-ized boilerplate code is recognized:
BEGIN_MESSAGE_MAP(CCanvas,CWnd)
Q_DECLARE_METATYPE(MyStruct)
PATTERNS("tex",...)(The last one is from this very patch.)
but also the following possibly undesirable consequence:
- When a label is not on a line by itself (except for a comment) it is no longer rejected, but can appear as a hunk header if it occurs at the beginning of a line:
next:;
IMO, the benefits of the change outweigh the (possible) regressions by a large margin.
--- userdiff.c | 8 +++----- 13 files changed, 3 insertions(+), 17 deletions(-)
userdiff.c +3 -5
@@ -125,11 +125,9 @@ PATTERNS("tex", "^(\\\\((sub)*section|chapter|part)\\*{0,1}\\{.*)$", "\\\\[a-zA-Z@]+|\\\\.|[a-zA-Z0-9\x80-\xff]+"),PATTERNS ("cpp" ,/* Jump targets or access declarations */ -"!^[ \t]*[A-Za-z_][A-Za-z_0-9]*:.*$\n" -/* C/++ functions/methods at top level */ -"^([A-Za-z_][A-Za-z_0-9]*([ \t*]+[A-Za-z_][A-Za-z_0-9]*([ \t]*::[ \t]*[^[:space:]]+)?){1,}[ \t]*\\([^;]*)$\n" -/* compound type at top level */ -"^((struct|class|enum)[^;]*)$" , +"!^[ \t]*[A-Za-z_][A-Za-z_0-9]*:[[:space:]]*($|/[/*])\n" +/* functions/methods, variables, and compounds at top level */ +"^((::[[:space:]]*)?[A-Za-z_].*)$" ,/* -- */ "[a-zA-Z_][a-zA-Z0-9_]*" "|[-+0-9.e]+[fFlL]?|0[xXbB]?[0-9a-fA-F]+[lL]?"
Signature · 1 line
1.9.0.1398.g59a4f1b
Quoted 6 lines · Johannes Sixt
Here is a patch that I'm carrying around since... a while. What do you think?
The pattern I chose also catches variable definition, not just functions. That is what I need, but it hurts grep --function-context That's the reason I didn't forward the patch, yet.
If by variable definition you mean:
struct foo bar = {
- old
+ new
};
I'd think that would be covered by the existing "struct|class|enum". Though I think we'd want to also allow keywords in front of it, like "static". I suspect the original was more meant to find:
struct foo {
-old
+new
};Quoted 10 lines
The parts of the pattern have the following flaws:
- The first part matches an identifier followed immediately by a colon and arbitrary text and is intended to reject goto labels and C++ access specifiers (public, private, protected). But this pattern also rejects C++ constructs, which look like this:
MyClass::MyClass() MyClass::~MyClass() MyClass::Item MyClass::Find(...
Makes sense. I noticed your fix is to look for end-of-line or comments afterwards. Would it be simpler to just check for a non-colon, like:
!^[ \t]*[A-Za-z_][A-Za-z_0-9]*:($|[^:])
Quoted 3 lines
- The second part matches an identifier followed by a list of qualified names (i.e. identifiers separated by the C++ scope operator '::')
[...]
A tried to keep the "looks like a function definition" bit in mine, and yours loosens this quite a bit more. I think that may be OK. That is, I do not think there is any reason for somebody to do:
void foo() {
call_to_bar();
-old
+new
}That is, nobody would put a function _call_ without indentation. If something has alphanumerics at the left-most column, then it is probably interesting no matter what.
Quoted 7 lines
- The third part of the pattern finally matches compound definitions. But it forgets about unions and namespaces, and also skips single-line definitions
struct random_iterator_tag {};because no semicolon can occur on the line.
I don't see how that is an interesting line. The point is to find a block that is surrounding the changes, but that is not surrounding the lines below.
Quoted 8 lines
Notice that all interesting anchor points begin with an identifier or keyword. But since there is a large variety of syntactical constructs after the first "word", the simplest is to require only this word and accept everything else. Therefore, this boils down to a line that begins with a letter or underscore (optionally preceded by the C++ scope operator '::' to accept functions returning a type anchored at the global namespace). Replace the second and third part by a single pattern that picks such a line.
Yeah, this bit makes sense to me.
Both yours and mine will find the first line here in things like:
void foo(void); -void bar(void); +void bar(int arg);
but I think that is OK. There _isn't_ any interesting surrounding context here. The current code will sometimes come up with an empty funcline (which is good), but it may just as easily come up with a totally bogus funcline in a case like:
void unrelated(void)
{
}
void foo(void);
-void bar(void);
+void bar(int arg);So trying to be very restrictive and say "that doesn't look like a function" does not really buy us anything (and it creates tons of false negatives, as you documented, because C++ syntax has all kinds of crazy stuff).
_If_ the backwards search learned to terminate (e.g., seeing the "^}" line and saying "well, we can't be inside a function"), then such negative lines might be useful for coming up with an empty funcname rather than the bogus "void foo(void);". But we do not do that currently, and I do not think it is that useful (the funcname above is arguably just as or more useful than an empty one).
-Peff
Am 3/6/2014 22:28, schrieb Jeff King:
Quoted 12 lines · Johannes Sixt
Quoted 3 lines
The pattern I chose also catches variable definition, not just functions. That is what I need, but it hurts grep --function-context That's the reason I didn't forward the patch, yet.
If by variable definition you mean:
struct foo bar = {- old
+ new
};
Quoted 8 lines
I'd think that would be covered by the existing "struct|class|enum". Though I think we'd want to also allow keywords in front of it, like "static". I suspect the original was more meant to find:
struct foo { -old +new };
No, I meant lines like
static double var;
-static int old;
+static int new;
The motivation is to show hints where in a file the change is located:
Anything that could be of significance for the author should be picked up.
But that does not necessarily help grep --function-context. For example, when there is a longish block of global variable definitions and there is a match in the middle, then --function-context would provide no context because the line itself would be regarded as the beginning of a "function", i.e., the context, and the next line (which also matches the pattern) would be the beginning of the *next* function, and would not be in the context anymore.
Quoted 8 lines
Quoted 8 lines
The parts of the pattern have the following flaws:
- The first part matches an identifier followed immediately by a colon and arbitrary text and is intended to reject goto labels and C++ access specifiers (public, private, protected). But this pattern also rejects C++ constructs, which look like this:
Quoted 8 lines
Quoted 3 lines
MyClass::MyClass() MyClass::~MyClass() MyClass::Item MyClass::Find(...Makes sense. I noticed your fix is to look for end-of-line or comments afterwards. Would it be simpler to just check for a non-colon, like:
!^[ \t]*[A-Za-z_][A-Za-z_0-9]*:($|[^:])
I want to match [[:space:]] after the label's colon, because I have lot's of C++ files with CRLF, and I need to match the CR. Your more liberal pattern would fit as well, although it would pick up a bit field as in
struct foo {
unsigned
flag: 1;
-old
+newI would not mind ignoring this case ("garbage in, garbage out" ;-).
Quoted 22 lines
Quoted 3 lines
- The second part matches an identifier followed by a list of qualified names (i.e. identifiers separated by the C++ scope operator '::')
[...]A tried to keep the "looks like a function definition" bit in mine, and yours loosens this quite a bit more. I think that may be OK. That is, I do not think there is any reason for somebody to do:
void foo() { call_to_bar(); -old +new }That is, nobody would put a function _call_ without indentation. If something has alphanumerics at the left-most column, then it is probably interesting no matter what.
Quoted 4 lines
- The third part of the pattern finally matches compound definitions. But it forgets about unions and namespaces, and also skips single-line definitions
Quoted 7 lines
Quoted 3 lines
struct random_iterator_tag {};because no semicolon can occur on the line.
I don't see how that is an interesting line. The point is to find a block that is surrounding the changes, but that is not surrounding the lines below.
I more often than not want to have an answer to the question "where?", not "wherein?" Then anything that helps locate a hunk is useful.
(The particular example, an empty struct, looks strange for C programmers, of course, but it's a common idiom in C++ when it comes to template meta-programming.)
Quoted 35 lines
Quoted 8 lines
Notice that all interesting anchor points begin with an identifier or keyword. But since there is a large variety of syntactical constructs after the first "word", the simplest is to require only this word and accept everything else. Therefore, this boils down to a line that begins with a letter or underscore (optionally preceded by the C++ scope operator '::' to accept functions returning a type anchored at the global namespace). Replace the second and third part by a single pattern that picks such a line.
Yeah, this bit makes sense to me.
Both yours and mine will find the first line here in things like:
void foo(void); -void bar(void); +void bar(int arg);but I think that is OK. There _isn't_ any interesting surrounding context here. The current code will sometimes come up with an empty funcline (which is good), but it may just as easily come up with a totally bogus funcline in a case like:
void unrelated(void) { } void foo(void); -void bar(void); +void bar(int arg);So trying to be very restrictive and say "that doesn't look like a function" does not really buy us anything (and it creates tons of false negatives, as you documented, because C++ syntax has all kinds of crazy stuff).
Quoted 6 lines
_If_ the backwards search learned to terminate (e.g., seeing the "^}" line and saying "well, we can't be inside a function"), then such negative lines might be useful for coming up with an empty funcname rather than the bogus "void foo(void);". But we do not do that currently, and I do not think it is that useful (the funcname above is arguably just as or more useful than an empty one).
As I said, my motivation is not so much to find a "container", but rather a clue to help locate a change while reading the patch text. I can speak for myself, but I have no idea what is more important for the majority.
-- Hannes
Quoted 8 lines · Johannes Sixt
No, I meant lines like
static double var; -static int old; +static int new;The motivation is to show hints where in a file the change is located:
Anything that could be of significance for the author should be picked up.
I see. Coupled with what you said below:
Quoted 3 lines
As I said, my motivation is not so much to find a "container", but rather a clue to help locate a change while reading the patch text. I can speak for myself, but I have no idea what is more important for the majority.
your proposal makes a lot more sense to me, and I think that is really at the center of our discussion. I do not have a gut feeling for which is "more right" (i.e., "container" versus "context").
But given that most of the cases we are discussing are ones where the "diff -p" default regex gets it right (or at least better than) compared to the C regex, I am tempted to say that we should be erring in the direction of simplicity, and just finding interesting lines without worrying about containers (i.e., it argues for your patch).
Quoted 9 lines
Quoted 4 lines
Makes sense. I noticed your fix is to look for end-of-line or comments afterwards. Would it be simpler to just check for a non-colon, like:
!^[ \t]*[A-Za-z_][A-Za-z_0-9]*:($|[^:])
I want to match [[:space:]] after the label's colon, because I have lot's of C++ files with CRLF, and I need to match the CR. Your more liberal pattern would fit as well, although it would pick up a bit field as in
Quoted 5 lines
struct foo {
unsigned
flag: 1;
-old
+newThanks, I was having trouble thinking of another good use of a colon, but bitfields are what I was missing. Your pattern is probably better here.
So I am leaning towards your patch, but I'm not sure if I understand all of the implications for "git grep". Can you give some concrete examples?
-Peff
Am 3/14/2014 4:54, schrieb Jeff King:
Quoted 21 lines · Johannes Sixt
Quoted 9 lines
No, I meant lines like
static double var; -static int old; +static int new;The motivation is to show hints where in a file the change is located:
Anything that could be of significance for the author should be picked up.I see. Coupled with what you said below:
Quoted 3 lines
As I said, my motivation is not so much to find a "container", but rather a clue to help locate a change while reading the patch text. I can speak for myself, but I have no idea what is more important for the majority.
your proposal makes a lot more sense to me, and I think that is really at the center of our discussion. I do not have a gut feeling for which is "more right" (i.e., "container" versus "context").
Quoted 5 lines
But given that most of the cases we are discussing are ones where the "diff -p" default regex gets it right (or at least better than) compared to the C regex, I am tempted to say that we should be erring in the direction of simplicity, and just finding interesting lines without worrying about containers (i.e., it argues for your patch).
We are in agreement here. So, let's base a decision on the implications on git grep.
Quoted 2 lines
... but I'm not sure if I understand all
of the implications for "git grep". Can you give some concrete examples?
Consider this code:
void above()
{}
static int Y;
static int A;
int bar()
{
return X;
}
void below()
{}When you 'git grep --function-context X', then you get this output with the current pattern, you proposal, and my proposal (file name etc omitted for brevity):
int bar()
{
return X;
}because we start the context at the last hunk header pattern above *or including the matching line* (and write it in the output), and we stop at the next hunk header pattern below the match (and do not write it in the output).
When you 'git grep --function-context Y', what do you want to see? With the current pattern, and with your pattern that forbids semicolon we get:
void above()
{}
static int Y;
static int A;
and with my simple pattern, which allows semicolon, we get merely
static int Y;
because the line itself is a hunk header (and we do not look back any further) and the next line is as well. That is not exactly "function context", and that is what I'm a bit worried about.
-- Hannes
Quoted 12 lines · Johannes Sixt
Consider this code:
void above() {} static int Y; static int A; int bar() { return X; } void below() {}
Thanks, this example is very helpful.
Quoted 8 lines
When you 'git grep --function-context X', then you get this output with the current pattern, you proposal, and my proposal (file name etc omitted for brevity):
int bar() { return X; }
Right, that makes sense to me.
Quoted 8 lines
When you 'git grep --function-context Y', what do you want to see? With the current pattern, and with your pattern that forbids semicolon we get:
void
above()
{}
static int Y;
static int A;
Quoted 7 lines
and with my simple pattern, which allows semicolon, we get merely
static int Y;
because the line itself is a hunk header (and we do not look back any further) and the next line is as well. That is not exactly "function context", and that is what I'm a bit worried about.
Hmm. To be honest, I do not see yours as all that bad. Is "above()" or "A" actually interesting here? I'm not sure that they are. But then I do not use --function-context myself.
I guess it violates the "show things that are vaguely nearby, rather than a container" view of context that we discussed earlier. But somehow that seems less important to me with "--function-context".
So I dunno. I kind of like your version.
-Peff
Cc René; do you have any comments regarding grep --function-context?
Am 3/18/2014 6:24, schrieb Jeff King:
Quoted 48 lines · Johannes Sixt
Quoted 13 lines
Consider this code:
void above() {} static int Y; static int A; int bar() { return X; } void below() {}Thanks, this example is very helpful.
Quoted 8 lines
When you 'git grep --function-context X', then you get this output with the current pattern, you proposal, and my proposal (file name etc omitted for brevity):
int bar() { return X; }Right, that makes sense to me.
Quoted 8 lines
When you 'git grep --function-context Y', what do you want to see? With the current pattern, and with your pattern that forbids semicolon we get:
void
above()
{}
static int Y;
static int A;Quoted 7 lines
and with my simple pattern, which allows semicolon, we get merely
static int Y;
because the line itself is a hunk header (and we do not look back any further) and the next line is as well. That is not exactly "function context", and that is what I'm a bit worried about.
Hmm. To be honest, I do not see yours as all that bad. Is "
above()" or "A" actually interesting here? I'm not sure that they are. But then I do not use --function-context myself.
Quoted 5 lines
I guess it violates the "show things that are vaguely nearby, rather than a container" view of context that we discussed earlier. But somehow that seems less important to me with "--function-context".
So I dunno. I kind of like your version.
Then I'll polish my patch series (it also rewrites the test framework) and post it.
-- Hannes
Am 18.03.2014 09:02, schrieb Johannes Sixt:
Quoted 46 lines
Cc René; do you have any comments regarding grep --function-context?
Am 3/18/2014 6:24, schrieb Jeff King:
Quoted 43 lines · Johannes Sixt
Quoted 13 lines
Consider this code:
void above() {} static int Y; static int A; int bar() { return X; } void below() {}Thanks, this example is very helpful.
Quoted 8 lines
When you 'git grep --function-context X', then you get this output with the current pattern, you proposal, and my proposal (file name etc omitted for brevity):
int bar() { return X; }Right, that makes sense to me.
Quoted 15 lines
When you 'git grep --function-context Y', what do you want to see? With the current pattern, and with your pattern that forbids semicolon we get:
void above() {} static int Y; static int A;and with my simple pattern, which allows semicolon, we get merely
static int Y;because the line itself is a hunk header (and we do not look back any further) and the next line is as well. That is not exactly "function context", and that is what I'm a bit worried about.
In global context there is no "function context", of course, so the latter makes sense.
"grep --function-context" is about useful context and its implementation piggy-backs on the hunk header definitions. If those are useful then the grep output should be fine as well. IAW: No worries, go ahead. :)
However, I only use the defaults heuristic (which shows just the Y-line as well) and don't know C++, so I my opinion on this matter isn't worth that much.
René
cover userdiff: cpp pattern simplification and test framework
Here is a series that makes the hunk header pattern for C and C++ even simpler than suggested by Peff in [1] to catch a lot more C++ functions and two more C patterns.
As a preparatory work, the test cases are totally rewritten to make it a lot simpler to drop in new tests. There was an earlier attempt to change the infrastructure [2], and it is the reason for the widened Cc list.
Two patches also extend the word regexp.
[1] http://article.gmane.org/gmane.comp.version-control.git/243408
[2] http://thread.gmane.org/gmane.comp.version-control.git/187269/focus=187497
Johannes Sixt (10):
userdiff: support C++ ->* and .* operators in the word regexp
userdiff: support unsigned and long long suffixes of integer constants
t4018: an infrastructure to test hunk headers
t4018: convert perl pattern tests to the new infrastructure
t4018: convert java pattern test to the new infrastructure
t4018: convert custom pattern test to the new infrastructure
t4018: reduce test files for pattern compilation tests
t4018: test cases for the built-in cpp pattern
t4018: test cases showing that the cpp pattern misses many anchor
points
userdiff: have 'cpp' hunk header pattern catch more C++ anchor points
t/t4018-diff-funcname.sh | 230 ++++++++++------------------- t/t4018/README | 18 +++ t/t4018/cpp-c++-function | 4 + t/t4018/cpp-class-constructor | 4 + t/t4018/cpp-class-constructor-mem-init | 5 + t/t4018/cpp-class-definition | 4 + t/t4018/cpp-class-definition-derived | 5 + t/t4018/cpp-class-destructor | 4 + t/t4018/cpp-function-returning-global-type | 4 + t/t4018/cpp-function-returning-nested | 5 + t/t4018/cpp-function-returning-pointer | 4 + t/t4018/cpp-function-returning-reference | 4 + t/t4018/cpp-gnu-style-function | 5 + t/t4018/cpp-namespace-definition | 4 + t/t4018/cpp-operator-definition | 4 + t/t4018/cpp-skip-access-specifiers | 8 + t/t4018/cpp-skip-comment-block | 9 ++ t/t4018/cpp-skip-labels | 8 + t/t4018/cpp-struct-definition | 9 ++ t/t4018/cpp-struct-single-line | 7 + t/t4018/cpp-template-function-definition | 4 + t/t4018/cpp-union-definition | 4 + t/t4018/cpp-void-c-function | 4 + t/t4018/custom1-pattern | 17 +++ t/t4018/custom2-match-to-end-of-line | 8 + t/t4018/custom3-alternation-in-pattern | 17 +++ t/t4018/java-class-member-function | 8 + t/t4018/perl-skip-end-of-heredoc | 8 + t/t4018/perl-skip-forward-decl | 10 ++ t/t4018/perl-skip-sub-in-pod | 18 +++ t/t4018/perl-sub-definition | 4 + t/t4018/perl-sub-definition-kr-brace | 4 + userdiff.c | 12 +- 33 files changed, 303 insertions(+), 160 deletions(-) create mode 100644 t/t4018/README create mode 100644 t/t4018/cpp-c++-function create mode 100644 t/t4018/cpp-class-constructor create mode 100644 t/t4018/cpp-class-constructor-mem-init create mode 100644 t/t4018/cpp-class-definition create mode 100644 t/t4018/cpp-class-definition-derived create mode 100644 t/t4018/cpp-class-destructor create mode 100644 t/t4018/cpp-function-returning-global-type create mode 100644 t/t4018/cpp-function-returning-nested create mode 100644 t/t4018/cpp-function-returning-pointer create mode 100644 t/t4018/cpp-function-returning-reference create mode 100644 t/t4018/cpp-gnu-style-function create mode 100644 t/t4018/cpp-namespace-definition create mode 100644 t/t4018/cpp-operator-definition create mode 100644 t/t4018/cpp-skip-access-specifiers create mode 100644 t/t4018/cpp-skip-comment-block create mode 100644 t/t4018/cpp-skip-labels create mode 100644 t/t4018/cpp-struct-definition create mode 100644 t/t4018/cpp-struct-single-line create mode 100644 t/t4018/cpp-template-function-definition create mode 100644 t/t4018/cpp-union-definition create mode 100644 t/t4018/cpp-void-c-function create mode 100644 t/t4018/custom1-pattern create mode 100644 t/t4018/custom2-match-to-end-of-line create mode 100644 t/t4018/custom3-alternation-in-pattern create mode 100644 t/t4018/java-class-member-function create mode 100644 t/t4018/perl-skip-end-of-heredoc create mode 100644 t/t4018/perl-skip-forward-decl create mode 100644 t/t4018/perl-skip-sub-in-pod create mode 100644 t/t4018/perl-sub-definition create mode 100644 t/t4018/perl-sub-definition-kr-brace
Signature · 1 line
1.8.5.2.244.g9fb3fb1
PATCH 1/10 userdiff: support C++ ->* and .* operators in the word regexp
The character sequences ->* and .* are valid C++ operators. Keep them together in --word-diff mode.
--- userdiff.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
userdiff.c +1 -1
@@ -133,7 +133,7 @@/* -- */ "[a-zA-Z_][a-zA-Z0-9_]*" "|[-+0-9.e]+[fFlL]?|0[xXbB]?[0-9a-fA-F]+[lL]?" - "|[-+*/<>%&^|=!]=|--|\\+\\+|<<=?|>>=?|&&|\\|\\||::|->"), + "|[-+*/<>%&^|=!]=|--|\\+\\+|<<=?|>>=?|&&|\\|\\||::|->\\*?|\\.\\*"),PATTERNS ("csharp" ,/* Keywords */ "!^[ \t]*(do|while|for|if|else|instanceof|new|return|switch|case|throw|catch|using)\n"
Signature · 1 line
1.8.5.2.244.g9fb3fb1
PATCH 2/10 userdiff: support unsigned and long long suffixes of integer constants
In reply to Johannes SixtDo not split constants such as 123U, 456ll, 789UL at the first U or second L.
--- userdiff.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
userdiff.c +1 -1
@@ -132,7 +132,7 @@ "^((struct|class|enum)[^;]*)$",/* -- */ "[a-zA-Z_][a-zA-Z0-9_]*" - "|[-+0-9.e]+[fFlL]?|0[xXbB]?[0-9a-fA-F]+[lL]?" + "|[-+0-9.e]+[fFlL]?|0[xXbB]?[0-9a-fA-F]+[lLuU]*" "|[-+*/<>%&^|=!]=|--|\\+\\+|<<=?|>>=?|&&|\\|\\||::|->\\*?|\\.\\*"), PATTERNS("csharp",/* Keywords */
Signature · 1 line
1.8.5.2.244.g9fb3fb1
PATCH 3/10 t4018: an infrastructure to test hunk headers
In reply to Johannes SixtAdd an infrastructure that simplifies adding new tests of the hunk header regular expressions.
To add new tests, a file with the syntax to test can be dropped in the directory t4018. The README file explains how a test file must contain; the README itself tests the default behavior.
--- t/t4018-diff-funcname.sh | 60 +++++++++++++++++++++++++++++++++++++++++++----- t/t4018/README | 18 +++++++++++++++ 2 files changed, 72 insertions(+), 6 deletions(-) create mode 100644 t/t4018/README
t/t4018-diff-funcname.sh +54 -6
@@ -100,7 +100,25 @@ test_expect_funcname () { grep "^@@.*@@ $1" diff } -for p in ada bibtex cpp csharp fortran html java matlab objc pascal perl php python ruby tex +diffpatterns=" + ada + bibtex + cpp + csharp + fortran + html + java + matlab + objc + pascal + perl + php + python + ruby + tex +" + +for p in $diffpatterns do test_expect_success "builtin $p pattern compiles" ' echo "*.java diff=$p" >.gitattributes &&
@@ -118,11 +136,6 @@ do ' done -test_expect_success 'default behaviour' ' - rm -f .gitattributes && - test_expect_funcname "public class Beer\$" -' - test_expect_success 'set up .gitattributes declaring drivers to test' ' cat >.gitattributes <<-\EOF *.java diff=java
@@ -182,4 +195,39 @@ test_expect_success 'alternation in pattern' ' test_expect_funcname "public static void main(" ' +test_expect_success 'setup hunk header tests' ' + for i in $diffpatterns + do + echo "$i-* diff=$i" + done > .gitattributes && + + # add all test files to the index + ( + cd "$TEST_DIRECTORY"/t4018 && + git --git-dir="$TRASH_DIRECTORY/.git" add . + ) && + + # place modified files in the worktree + for i in $(git ls-files) + do + sed -e "s/ChangeMe/IWasChanged/" <"$TEST_DIRECTORY/t4018/$i" >"$i" || return 1 + done +' + +# check each individual file +for i in $(git ls-files) +do + if grep broken "$i" >/dev/null 2>&1 + then + result=failure + else + result=success + fi + test_expect_$result "hunk header: $i" " + test_when_finished 'cat actual' && # for debugging only + git diff -U1 $i >actual && + grep '@@ .* @@.*RIGHT' actual + " +done + test_done
t/t4018/README new file +18 -0
@@ -0,0 +1,18 @@ +How to write RIGHT test cases +============================= + +Insert the word "ChangeMe" (exactly this form) at a distance of +at least two lines from the line that must appear in the hunk header. + +The text that must appear in the hunk header must contain the word +"right", but in all upper-case, like in the title above. + +To mark a test case that highlights a malfunction, insert the word +BROKEN in all lower-case somewhere in the file. + +This text is a bit twisted and out of order, but it is itself a +test case for the default hunk header pattern. Know what you are doing +if you change it. + +BTW, this tests that the head line goes to the hunk header, not the line +of equal signs.
Signature · 1 line
1.8.5.2.244.g9fb3fb1
Re: [PATCH 03/10] t4018: an infrastructure to test hunk headers
Quoted 5 lines · Johannes Sixt
Add an infrastructure that simplifies adding new tests of the hunk header regular expressions.
To add new tests, a file with the syntax to test can be dropped in the directory t4018. The README file explains how a test file must contain;
s/how/what/, or "how a test file must be written" you mean?
Quoted 1 line
the README itself tests the default behavior.
Thanks. Looks like a reasonable way to mark what must be found.
Quoted 28 lines
Signed-off-by Johannes Sixt <[email protected]>--- t/t4018-diff-funcname.sh | 60 +++++++++++++++++++++++++++++++++++++++++++----- t/t4018/README | 18 +++++++++++++++ 2 files changed, 72 insertions(+), 6 deletions(-) create mode 100644 t/t4018/READMEt/t4018-diff-funcname.sh +12
-1@@ -100,7 +100,25 @@ test_expect_funcname () { grep "^@@.*@@ $1" diff } -for p in ada bibtex cpp csharp fortran html java matlab objc pascal perl php python ruby tex +diffpatterns=" + ada + bibtex + cpp + csharp + fortran + html + java + matlab + objc + pascal + perl
t/t4018-diff-funcname.sh test_expect_funcname () { · line 124
Quoted 10 lines
t/t4018-diff-funcname.sh +7
-0+ php + python + ruby + tex +" + +for p in $diffpatterns do test_expect_success "builtin $p pattern compiles" ' echo "*.java diff=$p" >.gitattributes &&
I always found this "Let's apply rules for language $p to these *.java files" strange. I have wonder if it makes sense to further change the framework to read the name of the rule to be applied from the file in t/t4018/ directory, instead of using filename that is the same as the name of the rule? That way, you can list the files in t/t4018/ directory to come up with the above list, without having to maintain the list of rules separately like the above.
Quoted 13 lines
t/t4018-diff-funcname.sh +0
-5@@ -118,11 +136,6 @@ do ' done -test_expect_success 'default behaviour' ' - rm -f .gitattributes && - test_expect_funcname "public class Beer\$" -' - test_expect_success 'set up .gitattributes declaring drivers to test' ' cat >.gitattributes <<-\EOF *.java diff=java
t/t4018-diff-funcname.sh test_expect_success 'alternation in pattern' ' · line 202
Quoted 8 lines
t/t4018-diff-funcname.sh +5
-0@@ -182,4 +195,39 @@ test_expect_success 'alternation in pattern' ' test_expect_funcname "public static void main(" ' +test_expect_success 'setup hunk header tests' ' + for i in $diffpatterns + do + echo "$i-* diff=$i" + done > .gitattributes &&
I like that you can have more than one test for each language/rule this way, allowing you to test one kind of breakage without getting affected by lines prepared for other tests in the same file.
Quoted 54 lines
t/t4018-diff-funcname.sh +29
-0+ # add all test files to the index + ( + cd "$TEST_DIRECTORY"/t4018 && + git --git-dir="$TRASH_DIRECTORY/.git" add . + ) && + + # place modified files in the worktree + for i in $(git ls-files) + do + sed -e "s/ChangeMe/IWasChanged/" <"$TEST_DIRECTORY/t4018/$i" >"$i" || return 1 + done +' + +# check each individual file +for i in $(git ls-files) +do + if grep broken "$i" >/dev/null 2>&1 + then + result=failure + else + result=success + fi + test_expect_$result "hunk header: $i" " + test_when_finished 'cat actual' && # for debugging only + git diff -U1 $i >actual && + grep '@@ .* @@.*RIGHT' actual + " +done + test_donet/t4018/README new file +18
-0@@ -0,0 +1,18 @@ +How to write RIGHT test cases +============================= + +Insert the word "ChangeMe" (exactly this form) at a distance of +at least two lines from the line that must appear in the hunk header. + +The text that must appear in the hunk header must contain the word +"right", but in all upper-case, like in the title above. + +To mark a test case that highlights a malfunction, insert the word +BROKEN in all lower-case somewhere in the file. + +This text is a bit twisted and out of order, but it is itself a +test case for the default hunk header pattern. Know what you are doing +if you change it. + +BTW, this tests that the head line goes to the hunk header, not the line +of equal signs.
Re: [PATCH 03/10] t4018: an infrastructure to test hunk headers
Am 21.03.2014 23:00, schrieb Junio C Hamano:
Quoted 9 lines
Quoted 5 lines · Johannes Sixt
Add an infrastructure that simplifies adding new tests of the hunk header regular expressions.
To add new tests, a file with the syntax to test can be dropped in the directory t4018. The README file explains how a test file must contain;
s/how/what/, or "how a test file must be written" you mean?
I do mean one of those, preferably the latter ;)
Quoted 24 lines
Quoted 13 lines
t/t4018-diff-funcname.sh +12
-1-for p in ada bibtex cpp csharp fortran html java matlab objc pascal perl php python ruby tex +diffpatterns=" + ada + bibtex + cpp + csharp + fortran + html + java + matlab + objc + pascal + perlt/t4018-diff-funcname.sh test_expect_funcname () { · line 124
Quoted 10 lines
t/t4018-diff-funcname.sh +7
-0+ php + python + ruby + tex +" + +for p in $diffpatterns do test_expect_success "builtin $p pattern compiles" ' echo "*.java diff=$p" >.gitattributes &&
Quoted 7 lines
I always found this "Let's apply rules for language $p to these *.java files" strange. I have wonder if it makes sense to further change the framework to read the name of the rule to be applied from the file in t/t4018/ directory, instead of using filename that is the same as the name of the rule? That way, you can list the files in t/t4018/ directory to come up with the above list, without having to maintain the list of rules separately like the above.
I see what you mean. Notice, however, that we also test whether the regular expressions can be compiled successfully, and for this it is desirable to have the complete list of userdiff drivers. Until we have at least one test-case for each driver, we wouldn't get the complete list. In summary, I do agree that a bit more automation is also desirable, but we are not there, yet.
-- Hannes
Re: [PATCH 03/10] t4018: an infrastructure to test hunk headers
Quoted 5 lines · Johannes Sixt
I see what you mean. Notice, however, that we also test whether the regular expressions can be compiled successfully, and for this it is desirable to have the complete list of userdiff drivers. Until we have at least one test-case for each driver, we wouldn't get the complete list.
I missed the fact that we lack "here is a pair of test files in this lang $p; go find diff between them and make sure we have the right hunk header" for many $p.
So the kind of improvements I wondered about are allowed to come later, but shouldn't be done in this series, especiallly not at this step in the series.
Thanks.
Re: [PATCH 03/10] t4018: an infrastructure to test hunk headers
In reply to Johannes SixtQuoted 6 lines · Johannes Sixt
Add an infrastructure that simplifies adding new tests of the hunk header regular expressions.
To add new tests, a file with the syntax to test can be dropped in the directory t4018. The README file explains how a test file must contain; the README itself tests the default behavior.
I really like the cleanups you've done in t4018. I noticed how messy it was when I modified it recently, but I didn't take the time to clean it.
t/t4018/README line 5
Quoted 11 lines
t/t4018/README new file +5
-0@@ -0,0 +1,18 @@ +How to write RIGHT test cases +============================= + +Insert the word "ChangeMe" (exactly this form) at a distance of +at least two lines from the line that must appear in the hunk header.
The existing tests use -U1 to make writing cases simpler. Is there a reason not to continue that (or if you found that porting the existing cases was not a chore with -U3, I can buy that argument, too)?
t/t4018/README line 11
Quoted 5 lines
t/t4018/README new file +5
-0+The text that must appear in the hunk header must contain the word +"right", but in all upper-case, like in the title above. + +To mark a test case that highlights a malfunction, insert the word +BROKEN in all lower-case somewhere in the file.
I wondered why you wouldn't write them in the case you are indicating, when...
t/t4018/README line 15
Quoted 3 lines
t/t4018/README new file +3
-0+This text is a bit twisted and out of order, but it is itself a +test case for the default hunk header pattern. Know what you are doing +if you change it.
Ah. Clever. :)
-Peff
Re: [PATCH 03/10] t4018: an infrastructure to test hunk headers
Quoted 9 lines · Jeff King
t/t4018/README line 5
Quoted 5 lines
t/t4018/README new file +5
-0+How to write RIGHT test cases +============================= + +Insert the word "ChangeMe" (exactly this form) at a distance of +at least two lines from the line that must appear in the hunk header.The existing tests use -U1 to make writing cases simpler. Is there a reason not to continue that (or if you found that porting the existing cases was not a chore with -U3, I can buy that argument, too)?
I take it back. You did keep "-U1" in the result. Is this "two lines" rule necessary, then?
-Peff
Re: [PATCH 03/10] t4018: an infrastructure to test hunk headers
Am 24.03.2014 22:39, schrieb Jeff King:
Quoted 12 lines · Jeff King
Quoted 10 lines
t/t4018/README line 5
Quoted 6 lines
t/t4018/README new file +5
-0+How to write RIGHT test cases +============================= + +Insert the word "ChangeMe" (exactly this form) at a distance of +at least two lines from the line that must appear in the hunk header.The existing tests use -U1 to make writing cases simpler. Is there a reason not to continue that (or if you found that porting the existing cases was not a chore with -U3, I can buy that argument, too)?
Quoted 2 lines
I take it back. You did keep "-U1" in the result. Is this "two lines" rule necessary, then?
When we have
one
two
three
how would you describe the distance between "one" and "three"?
Or do you have a wrong expectation how we determine the hunk header? The hunk header is searched in the text *before* the pre-context. Therefore, we need at least as many lines between the intended hunk header line (with tag RIGHT) and the changed line (ChangeMe) as we request with -U for the context.
So: yes, we need the rule.
-- Hannes
Re: [PATCH 03/10] t4018: an infrastructure to test hunk headers
Quoted 16 lines · Johannes Sixt
Am 24.03.2014 22:39, schrieb Jeff King:
Quoted 12 lines · Jeff King
Quoted 10 lines
t/t4018/README line 5
Quoted 6 lines
t/t4018/README new file +5
-0+How to write RIGHT test cases +============================= + +Insert the word "ChangeMe" (exactly this form) at a distance of +at least two lines from the line that must appear in the hunk header.The existing tests use -U1 to make writing cases simpler. Is there a reason not to continue that (or if you found that porting the existing cases was not a chore with -U3, I can buy that argument, too)?
Quoted 2 lines
I take it back. You did keep "-U1" in the result. Is this "two lines" rule necessary, then?
Quoted 7 lines
When we have
one
two
threehow would you describe the distance between "one" and "three"?
I read it as "two lines between". Though that would be appropriate for -U2, so I think I probably had an off-by-one in my head, then, too.
So it is probably fine as-is, and this was mostly just me confusing myself. Sorry for the noise.
-Peff
PATCH 4/10 t4018: convert perl pattern tests to the new infrastructure
In reply to Johannes SixtThere is one subtlety: The old test case 'perl pattern gets full line of POD header' does not have its own new test case, but the feature is tested nevertheless by placing the RIGHT tag at the end of the expected hunk header in t4018/perl-skip-sub-in-pod.
--- t/t4018-diff-funcname.sh | 88 ------------------------------------ t/t4018/perl-skip-end-of-heredoc | 8 ++++ t/t4018/perl-skip-forward-decl | 10 ++++ t/t4018/perl-skip-sub-in-pod | 18 ++++++++ t/t4018/perl-sub-definition | 4 ++ t/t4018/perl-sub-definition-kr-brace | 4 ++ 6 files changed, 44 insertions(+), 88 deletions(-) create mode 100644 t/t4018/perl-skip-end-of-heredoc create mode 100644 t/t4018/perl-skip-forward-decl create mode 100644 t/t4018/perl-skip-sub-in-pod create mode 100644 t/t4018/perl-sub-definition create mode 100644 t/t4018/perl-sub-definition-kr-brace
t/t4018-diff-funcname.sh +0 -88
@@ -29,69 +29,6 @@ public class Beer } EOF sed 's/beer\\/beer,\\/' <Beer.java >Beer-correct.java -cat >Beer.perl <<\EOT -package Beer; - -use strict; -use warnings; -use parent qw(Exporter); -our @EXPORT_OK = qw(round finalround); - -sub other; # forward declaration - -# hello - -sub round { - my ($n) = @_; - print "$n bottles of beer on the wall "; - print "$n bottles of beer\n"; - print "Take one down, pass it around, "; - $n = $n - 1; - print "$n bottles of beer on the wall.\n"; -} - -sub finalround -{ - print "Go to the store, buy some more\n"; - print "99 bottles of beer on the wall.\n"); -} - -sub withheredocument { - print <<"EOF" -decoy here-doc -EOF - # some lines of context - # to pad it out - print "hello\n"; -} - -__END__ - -=head1 NAME - -Beer - subroutine to output fragment of a drinking song - -=head1 SYNOPSIS - - use Beer qw(round finalround); - - sub song { - for (my $i = 99; $i > 0; $i--) { - round $i; - } - finalround; - } - - song; - -=cut -EOT -sed -e ' - s/hello/goodbye/ - s/beer\\/beer,\\/ - s/more\\/more,\\/ - s/song;/song();/ -' <Beer.perl >Beer-correct.perl test_expect_funcname () { lang=${2-java}
@@ -139,7 +76,6 @@ done test_expect_success 'set up .gitattributes declaring drivers to test' ' cat >.gitattributes <<-\EOF *.java diff=java - *.perl diff=perl EOF '
@@ -147,30 +83,6 @@ test_expect_success 'preset java pattern' ' test_expect_funcname "public static void main(" ' -test_expect_success 'preset perl pattern' ' - test_expect_funcname "sub round {\$" perl -' - -test_expect_success 'perl pattern accepts K&R style brace placement, too' ' - test_expect_funcname "sub finalround\$" perl -' - -test_expect_success 'but is not distracted by end of <<here document' ' - test_expect_funcname "sub withheredocument {\$" perl -' - -test_expect_success 'perl pattern is not distracted by sub within POD' ' - test_expect_funcname "=head" perl -' - -test_expect_success 'perl pattern gets full line of POD header' ' - test_expect_funcname "=head1 SYNOPSIS\$" perl -' - -test_expect_success 'perl pattern is not distracted by forward declaration' ' - test_expect_funcname "package Beer;\$" perl -' - test_expect_success 'custom pattern' ' test_config diff.java.funcname "!static !String
t/t4018/perl-skip-end-of-heredoc new file +8 -0
@@ -0,0 +1,8 @@ +sub RIGHTwithheredocument { + print <<"EOF" +decoy here-doc +EOF + # some lines of context + # to pad it out + print "ChangeMe\n"; +}
t/t4018/perl-skip-forward-decl new file +10 -0
@@ -0,0 +1,10 @@ +package RIGHT; + +use strict; +use warnings; +use parent qw(Exporter); +our @EXPORT_OK = qw(round finalround); + +sub other; # forward declaration + +# ChangeMe
t/t4018/perl-skip-sub-in-pod new file +18 -0
@@ -0,0 +1,18 @@ +=head1 NAME + +Beer - subroutine to output fragment of a drinking song + +=head1 SYNOPSIS_RIGHT + + use Beer qw(round finalround); + + sub song { + for (my $i = 99; $i > 0; $i--) { + round $i; + } + finalround; + } + + ChangeMe; + +=cut
t/t4018/perl-sub-definition new file +4 -0
@@ -0,0 +1,4 @@ +sub RIGHT { + my ($n) = @_; + print "ChangeMe"; +}
t/t4018/perl-sub-definition-kr-brace new file +4 -0
@@ -0,0 +1,4 @@ +sub RIGHT +{ + print "ChangeMe\n"; +}
Signature · 1 line
1.8.5.2.244.g9fb3fb1
PATCH 5/10 t4018: convert java pattern test to the new infrastructure
In reply to Johannes Sixt--- t/t4018-diff-funcname.sh | 4 ---- t/t4018/java-class-member-function | 8 ++++++++ 2 files changed, 8 insertions(+), 4 deletions(-) create mode 100644 t/t4018/java-class-member-function
t/t4018-diff-funcname.sh +0 -4
@@ -79,10 +79,6 @@ test_expect_success 'set up .gitattributes declaring drivers to test' ' EOF ' -test_expect_success 'preset java pattern' ' - test_expect_funcname "public static void main(" -' - test_expect_success 'custom pattern' ' test_config diff.java.funcname "!static !String
t/t4018/java-class-member-function new file +8 -0
@@ -0,0 +1,8 @@ +public class Beer +{ + int special; + public static void main(String RIGHT[]) + { + System.out.print("ChangeMe"); + } +}
Signature · 1 line
1.8.5.2.244.g9fb3fb1
PATCH 6/10 t4018: convert custom pattern test to the new infrastructure
In reply to Johannes SixtFor the test case "matches to end of line", extend the pattern by a few wildcards so that the pattern captures the "RIGHT" token, which is needed for verification, without mentioning it in the pattern.
--- t/t4018-diff-funcname.sh | 40 ++++++++++++++-------------------- t/t4018/custom1-pattern | 17 +++++++++++++++ t/t4018/custom2-match-to-end-of-line | 8 +++++++ t/t4018/custom3-alternation-in-pattern | 17 +++++++++++++++ 4 files changed, 58 insertions(+), 24 deletions(-) create mode 100644 t/t4018/custom1-pattern create mode 100644 t/t4018/custom2-match-to-end-of-line create mode 100644 t/t4018/custom3-alternation-in-pattern
t/t4018-diff-funcname.sh +16 -24
@@ -30,12 +30,19 @@ public class Beer EOF sed 's/beer\\/beer,\\/' <Beer.java >Beer-correct.java -test_expect_funcname () { - lang=${2-java} - test_expect_code 1 git diff --no-index -U1 \ - "Beer.$lang" "Beer-correct.$lang" >diff && - grep "^@@.*@@ $1" diff -} +test_expect_success 'setup' ' + # a non-trivial custom pattern + git config diff.custom1.funcname "!static +!String +[^ ].*s.*" && + + # a custom pattern which matches to end of line + git config diff.custom2.funcname "......Beer\$" && + + # alternation in pattern + git config diff.custom3.funcname "Beer$" && + git config diff.custom3.xfuncname "^[ ]*((public|static).*)$" +' diffpatterns=" ada
@@ -53,6 +60,9 @@ diffpatterns=" python ruby tex + custom1 + custom2 + custom3 " for p in $diffpatterns
@@ -79,30 +89,12 @@ test_expect_success 'set up .gitattributes declaring drivers to test' ' EOF ' -test_expect_success 'custom pattern' ' - test_config diff.java.funcname "!static -!String -[^ ].*s.*" && - test_expect_funcname "int special;\$" -' - test_expect_success 'last regexp must not be negated' ' test_config diff.java.funcname "!static" && test_expect_code 128 git diff --no-index Beer.java Beer-correct.java 2>msg && grep ": Last expression must not be negated:" msg ' -test_expect_success 'pattern which matches to end of line' ' - test_config diff.java.funcname "Beer\$" && - test_expect_funcname "Beer\$" -' - -test_expect_success 'alternation in pattern' ' - test_config diff.java.funcname "Beer$" && - test_config diff.java.xfuncname "^[ ]*((public|static).*)$" && - test_expect_funcname "public static void main(" -' - test_expect_success 'setup hunk header tests' ' for i in $diffpatterns do
t/t4018/custom1-pattern new file +17 -0
@@ -0,0 +1,17 @@ +public class Beer +{ + int special, RIGHT; + public static void main(String args[]) + { + String s=" "; + for(int x = 99; x > 0; x--) + { + System.out.print(x + " bottles of beer on the wall " + + x + " bottles of beer\n" // ChangeMe + + "Take one down, pass it around, " + (x - 1) + + " bottles of beer on the wall.\n"); + } + System.out.print("Go to the store, buy some more,\n" + + "99 bottles of beer on the wall.\n"); + } +}
t/t4018/custom2-match-to-end-of-line new file +8 -0
@@ -0,0 +1,8 @@ +public class RIGHT_Beer +{ + int special; + public static void main(String args[]) + { + System.out.print("ChangeMe"); + } +}
t/t4018/custom3-alternation-in-pattern new file +17 -0
@@ -0,0 +1,17 @@ +public class Beer +{ + int special; + public static void main(String RIGHT[]) + { + String s=" "; + for(int x = 99; x > 0; x--) + { + System.out.print(x + " bottles of beer on the wall " + + x + " bottles of beer\n" // ChangeMe + + "Take one down, pass it around, " + (x - 1) + + " bottles of beer on the wall.\n"); + } + System.out.print("Go to the store, buy some more,\n" + + "99 bottles of beer on the wall.\n"); + } +}
Signature · 1 line
1.8.5.2.244.g9fb3fb1
PATCH 7/10 t4018: reduce test files for pattern compilation tests
In reply to Johannes SixtAll test cases that need a file with specific text patterns have been converted to utilize texts in the t4018/ directory. The remaining tests in the test script deal only with the validity of the regular expressions. These tests do not depend on the contents of files that 'git diff' is invoked on. Remove the largish here-document and use only tiny files.
While we are touching these tests, convert grep to test_i18ngrep as the texts checked for may undergo translation in the future.
--- t/t4018-diff-funcname.sh | 52 +++++++++++++----------------------------------- 1 file changed, 14 insertions(+), 38 deletions(-)
t/t4018-diff-funcname.sh +14 -38
@@ -7,29 +7,6 @@ test_description='Test custom diff function name patterns' . ./test-lib.sh -LF=' -' -cat >Beer.java <<\EOF -public class Beer -{ - int special; - public static void main(String args[]) - { - String s=" "; - for(int x = 99; x > 0; x--) - { - System.out.print(x + " bottles of beer on the wall " - + x + " bottles of beer\n" - + "Take one down, pass it around, " + (x - 1) - + " bottles of beer on the wall.\n"); - } - System.out.print("Go to the store, buy some more,\n" - + "99 bottles of beer on the wall.\n"); - } -} -EOF -sed 's/beer\\/beer,\\/' <Beer.java >Beer-correct.java - test_expect_success 'setup' ' # a non-trivial custom pattern git config diff.custom1.funcname "!static
@@ -41,7 +18,11 @@ test_expect_success 'setup' ' # alternation in pattern git config diff.custom3.funcname "Beer$" && - git config diff.custom3.xfuncname "^[ ]*((public|static).*)$" + git config diff.custom3.xfuncname "^[ ]*((public|static).*)$" && + + # for regexp compilation tests + echo A >A.java && + echo B >B.java ' diffpatterns="
@@ -70,29 +51,24 @@ do test_expect_success "builtin $p pattern compiles" ' echo "*.java diff=$p" >.gitattributes && test_expect_code 1 git diff --no-index \ - Beer.java Beer-correct.java 2>msg && - ! grep fatal msg && - ! grep error msg + A.java B.java 2>msg && + ! test_i18ngrep fatal msg && + ! test_i18ngrep error msg ' test_expect_success "builtin $p wordRegex pattern compiles" ' echo "*.java diff=$p" >.gitattributes && test_expect_code 1 git diff --no-index --word-diff \ - Beer.java Beer-correct.java 2>msg && - ! grep fatal msg && - ! grep error msg + A.java B.java 2>msg && + ! test_i18ngrep fatal msg && + ! test_i18ngrep error msg ' done -test_expect_success 'set up .gitattributes declaring drivers to test' ' - cat >.gitattributes <<-\EOF - *.java diff=java - EOF -' - test_expect_success 'last regexp must not be negated' ' + echo "*.java diff=java" >.gitattributes && test_config diff.java.funcname "!static" && - test_expect_code 128 git diff --no-index Beer.java Beer-correct.java 2>msg && - grep ": Last expression must not be negated:" msg + test_expect_code 128 git diff --no-index A.java B.java 2>msg && + test_i18ngrep ": Last expression must not be negated:" msg ' test_expect_success 'setup hunk header tests' '
Signature · 1 line
1.8.5.2.244.g9fb3fb1
PATCH 8/10 t4018: test cases for the built-in cpp pattern
In reply to Johannes SixtA later patch changes the built-in cpp pattern. These test cases demonstrate aspects of the pattern that we do not want to change.
--- t/t4018/cpp-c++-function | 4 ++++ t/t4018/cpp-class-definition | 4 ++++ t/t4018/cpp-class-definition-derived | 5 +++++ t/t4018/cpp-function-returning-pointer | 4 ++++ t/t4018/cpp-skip-access-specifiers | 8 ++++++++ t/t4018/cpp-skip-comment-block | 9 +++++++++ t/t4018/cpp-skip-labels | 8 ++++++++ t/t4018/cpp-struct-definition | 9 +++++++++ t/t4018/cpp-void-c-function | 4 ++++ 9 files changed, 55 insertions(+) create mode 100644 t/t4018/cpp-c++-function create mode 100644 t/t4018/cpp-class-definition create mode 100644 t/t4018/cpp-class-definition-derived create mode 100644 t/t4018/cpp-function-returning-pointer create mode 100644 t/t4018/cpp-skip-access-specifiers create mode 100644 t/t4018/cpp-skip-comment-block create mode 100644 t/t4018/cpp-skip-labels create mode 100644 t/t4018/cpp-struct-definition create mode 100644 t/t4018/cpp-void-c-function
t/t4018/cpp-c++-function new file +4 -0
@@ -0,0 +1,4 @@ +Item RIGHT::DoSomething( Args with_spaces ) +{ + ChangeMe; +}
t/t4018/cpp-class-definition new file +4 -0
@@ -0,0 +1,4 @@ +class RIGHT +{ + int ChangeMe; +};
t/t4018/cpp-class-definition-derived new file +5 -0
@@ -0,0 +1,5 @@ +class RIGHT : + public Baseclass +{ + int ChangeMe; +};
t/t4018/cpp-function-returning-pointer new file +4 -0
@@ -0,0 +1,4 @@ +const char *get_it_RIGHT(char *ptr) +{ + ChangeMe; +}
t/t4018/cpp-skip-access-specifiers new file +8 -0
@@ -0,0 +1,8 @@ +class RIGHT : public Baseclass +{ +public: +protected: +private: + void DoSomething(); + int ChangeMe; +};
t/t4018/cpp-skip-comment-block new file +9 -0
@@ -0,0 +1,9 @@ +struct item RIGHT(int i) +// Do not +// pick up +/* these +** comments. +*/ +{ + ChangeMe; +}
t/t4018/cpp-skip-labels new file +8 -0
@@ -0,0 +1,8 @@ +void RIGHT (void) +{ +repeat: // C++ comment +next: /* C comment */ + do_something(); + + ChangeMe; +}
t/t4018/cpp-struct-definition new file +9 -0
@@ -0,0 +1,9 @@ +struct RIGHT { + unsigned + /* this bit field looks like a label and should not be picked up */ + decoy_bitfield: 2, + more : 1; + int filler; + + int ChangeMe; +};
t/t4018/cpp-void-c-function new file +4 -0
@@ -0,0 +1,4 @@ +void RIGHT (void) +{ + ChangeMe; +}
Signature · 1 line
1.8.5.2.244.g9fb3fb1
PATCH 9/10 t4018: test cases showing that the cpp pattern misses many anchor points
In reply to Johannes SixtMost of the tests show C++ code, but there is also a union definition and a GNU style function definition that are not recognized.
--- t/t4018/cpp-class-constructor | 5 +++++ t/t4018/cpp-class-constructor-mem-init | 6 ++++++ t/t4018/cpp-class-destructor | 5 +++++ t/t4018/cpp-function-returning-global-type | 5 +++++ t/t4018/cpp-function-returning-nested | 6 ++++++ t/t4018/cpp-function-returning-reference | 5 +++++ t/t4018/cpp-gnu-style-function | 6 ++++++ t/t4018/cpp-namespace-definition | 5 +++++ t/t4018/cpp-operator-definition | 5 +++++ t/t4018/cpp-struct-single-line | 8 ++++++++ t/t4018/cpp-template-function-definition | 5 +++++ t/t4018/cpp-union-definition | 5 +++++ 12 files changed, 66 insertions(+) create mode 100644 t/t4018/cpp-class-constructor create mode 100644 t/t4018/cpp-class-constructor-mem-init create mode 100644 t/t4018/cpp-class-destructor create mode 100644 t/t4018/cpp-function-returning-global-type create mode 100644 t/t4018/cpp-function-returning-nested create mode 100644 t/t4018/cpp-function-returning-reference create mode 100644 t/t4018/cpp-gnu-style-function create mode 100644 t/t4018/cpp-namespace-definition create mode 100644 t/t4018/cpp-operator-definition create mode 100644 t/t4018/cpp-struct-single-line create mode 100644 t/t4018/cpp-template-function-definition create mode 100644 t/t4018/cpp-union-definition
t/t4018/cpp-class-constructor new file +5 -0
@@ -0,0 +1,5 @@ +Item::Item(int RIGHT) +{ + ChangeMe; + broken; +}
t/t4018/cpp-class-constructor-mem-init new file +6 -0
@@ -0,0 +1,6 @@ +Item::Item(int RIGHT) : + member(0) +{ + ChangeMe; + broken; +}
t/t4018/cpp-class-destructor new file +5 -0
@@ -0,0 +1,5 @@ +RIGHT::~RIGHT() +{ + ChangeMe; + broken; +}
t/t4018/cpp-function-returning-global-type new file +5 -0
@@ -0,0 +1,5 @@ +::Item get::it::RIGHT() +{ + ChangeMe; + broken; +}
t/t4018/cpp-function-returning-nested new file +6 -0
@@ -0,0 +1,6 @@ +get::Item get::it::RIGHT() +{ + ChangeMe; + broken; +} +
t/t4018/cpp-function-returning-reference new file +5 -0
@@ -0,0 +1,5 @@ +string& get::it::RIGHT(char *ptr) +{ + ChangeMe; + broken; +}
t/t4018/cpp-gnu-style-function new file +6 -0
@@ -0,0 +1,6 @@ +const char * +RIGHT(int arg) +{ + ChangeMe; + broken; +}
t/t4018/cpp-namespace-definition new file +5 -0
@@ -0,0 +1,5 @@ +namespace RIGHT +{ + ChangeMe; + broken; +}
t/t4018/cpp-operator-definition new file +5 -0
@@ -0,0 +1,5 @@ +Value operator+(Value LEFT, Value RIGHT) +{ + ChangeMe; + broken; +}
t/t4018/cpp-struct-single-line new file +8 -0
@@ -0,0 +1,8 @@ +void wrong() +{ +} + +struct RIGHT_iterator_tag {}; + +int ChangeMe; +// broken
t/t4018/cpp-template-function-definition new file +5 -0
@@ -0,0 +1,5 @@ +template<class T> int RIGHT(T arg) +{ + ChangeMe; + broken; +}
t/t4018/cpp-union-definition new file +5 -0
@@ -0,0 +1,5 @@ +union RIGHT { + double v; + int ChangeMe; + broken; +};
Signature · 1 line
1.8.5.2.244.g9fb3fb1
PATCH 10/10 userdiff: have 'cpp' hunk header pattern catch more C++ anchor points
In reply to Johannes SixtThe hunk header pattern 'cpp' is intended for C and C++ source code, but it is actually not particularly useful for the latter, and even misses some use-cases for the former.
The parts of the pattern have the following flaws:
- The first part matches an identifier followed immediately by a colon and arbitrary text and is intended to reject goto labels and C++ access specifiers (public, private, protected). But this pattern also rejects C++ constructs, which look like this:
MyClass::MyClass()
MyClass::~MyClass()
MyClass::Item MyClass::Find(...- The second part matches an identifier followed by a list of qualified names (i.e. identifiers separated by the C++ scope operator '::') separated by space or '*' followed by an opening parenthesis (with space between the tokens). It matches function declarations like
struct item* get_head(...
int Outer::Inner::Func(...
Since the pattern requires at least two identifiers, GNU-style function definitions are ignored:
void
func(...Moreover, since the pattern does not allow punctuation other than '*', the following C++ constructs are not recognized:
. template definitions:
template<class T> int func(T arg)
. functions returning references:
const string& get_message()
. functions returning templated types:
vector<int> foo()
. operator definitions:
Value operator+(Value l, Value r)
- The third part of the pattern finally matches compound definitions. But it forgets about unions and namespaces, and also skips single-line definitions
struct random_iterator_tag {};because no semicolon can occur on the line.
Change the first pattern to require a colon at the end of the line (except for trailing space and comments), so that it does not reject constructor or destructor definitions.
Notice that all interesting anchor points begin with an identifier or keyword. But since there is a large variety of syntactical constructs after the first "word", the simplest is to require only this word and accept everything else. Therefore, this boils down to a line that begins with a letter or underscore (optionally preceded by the C++ scope operator '::' to accept functions returning a type anchored at the global namespace). Replace the second and third part by a single pattern that picks such a line.
This has the following desirable consequence:
- All constructs mentioned above are recognized.
and the following likely desirable consequences:
- Definitions of global variables and typedefs are recognized:
int num_entries = 0;
extern const char* help_text;
typedef basic_string<wchar_t> wstring;- Commonly used marco-ized boilerplate code is recognized:
BEGIN_MESSAGE_MAP(CCanvas,CWnd)
Q_DECLARE_METATYPE(MyStruct)
PATTERNS("tex",...)(The last one is from this very patch.)
but also the following possibly undesirable consequence:
- When a label is not on a line by itself (except for a comment) it is no longer rejected, but can appear as a hunk header if it occurs at the beginning of a line:
next:;
IMO, the benefits of the change outweigh the (possible) regressions by a large margin.
--- t/t4018/cpp-class-constructor | 1 - t/t4018/cpp-class-constructor-mem-init | 1 - t/t4018/cpp-class-destructor | 1 - t/t4018/cpp-function-returning-global-type | 1 - t/t4018/cpp-function-returning-nested | 1 - t/t4018/cpp-function-returning-reference | 1 - t/t4018/cpp-gnu-style-function | 1 - t/t4018/cpp-namespace-definition | 1 - t/t4018/cpp-operator-definition | 1 - t/t4018/cpp-struct-single-line | 1 - t/t4018/cpp-template-function-definition | 1 - t/t4018/cpp-union-definition | 1 - userdiff.c | 8 +++----- 13 files changed, 3 insertions(+), 17 deletions(-)
t/t4018/cpp-class-constructor +0 -1
@@ -1,5 +1,4 @@ Item::Item(int RIGHT) { ChangeMe; - broken; }
t/t4018/cpp-class-constructor-mem-init +0 -1
@@ -2,5 +2,4 @@ Item::Item(int RIGHT) : member(0) { ChangeMe; - broken; }
t/t4018/cpp-class-destructor +0 -1
@@ -1,5 +1,4 @@ RIGHT::~RIGHT() { ChangeMe; - broken; }
t/t4018/cpp-function-returning-global-type +0 -1
@@ -1,5 +1,4 @@ ::Item get::it::RIGHT() { ChangeMe; - broken; }
t/t4018/cpp-function-returning-nested +0 -1
@@ -1,6 +1,5 @@ get::Item get::it::RIGHT() { ChangeMe; - broken; }
t/t4018/cpp-function-returning-reference +0 -1
@@ -1,5 +1,4 @@ string& get::it::RIGHT(char *ptr) { ChangeMe; - broken; }
t/t4018/cpp-gnu-style-function +0 -1
@@ -2,5 +2,4 @@ const char * RIGHT(int arg) { ChangeMe; - broken; }
t/t4018/cpp-namespace-definition +0 -1
@@ -1,5 +1,4 @@ namespace RIGHT { ChangeMe; - broken; }
t/t4018/cpp-operator-definition +0 -1
@@ -1,5 +1,4 @@ Value operator+(Value LEFT, Value RIGHT) { ChangeMe; - broken; }
t/t4018/cpp-struct-single-line +0 -1
@@ -5,4 +5,3 @@ void wrong() struct RIGHT_iterator_tag {}; int ChangeMe; -// broken
t/t4018/cpp-template-function-definition +0 -1
@@ -1,5 +1,4 @@ template<class T> int RIGHT(T arg) { ChangeMe; - broken; }
t/t4018/cpp-union-definition +0 -1
@@ -1,5 +1,4 @@ union RIGHT { double v; int ChangeMe; - broken; };
userdiff.c +3 -5
@@ -125,11 +125,9 @@ "\\\\[a-zA-Z@]+|\\\\.|[a-zA-Z0-9\x80-\xff]+"),PATTERNS ("cpp" ,/* Jump targets or access declarations */ -"!^[ \t]*[A-Za-z_][A-Za-z_0-9]*:.*$\n" -/* C/++ functions/methods at top level */ -"^([A-Za-z_][A-Za-z_0-9]*([ \t*]+[A-Za-z_][A-Za-z_0-9]*([ \t]*::[ \t]*[^[:space:]]+)?){1,}[ \t]*\\([^;]*)$\n" -/* compound type at top level */ -"^((struct|class|enum)[^;]*)$" , +"!^[ \t]*[A-Za-z_][A-Za-z_0-9]*:[[:space:]]*($|/[/*])\n" +/* functions/methods, variables, and compounds at top level */ +"^((::[[:space:]]*)?[A-Za-z_].*)$" ,/* -- */ "[a-zA-Z_][a-zA-Z0-9_]*" "|[-+0-9.e]+[fFlL]?|0[xXbB]?[0-9a-fA-F]+[lLuU]*"
Signature · 1 line
1.8.5.2.244.g9fb3fb1
Re: [PATCH 00/10] userdiff: cpp pattern simplification and test framework
In reply to Johannes SixtThanks; will replace jk/diff-funcname-cpp-regex with this series.
Re: [PATCH 00/10] userdiff: cpp pattern simplification and test framework
In reply to Johannes SixtQuoted 8 lines · Johannes Sixt
Here is a series that makes the hunk header pattern for C and C++ even simpler than suggested by Peff in [1] to catch a lot more C++ functions and two more C patterns.
As a preparatory work, the test cases are totally rewritten to make it a lot simpler to drop in new tests. There was an earlier attempt to change the infrastructure [2], and it is the reason for the widened Cc list.
Thanks. This looks sane overall, and I am especially happy about the increased test coverage. I do not know if your 10/10 misses some cases, but at least now we can collect and reason about a set of examples going forward.
-Peff
Quoted 4 lines · Jeff King
types, we simply look for an identifier at the start of the line that contains a "(", meaning it is either a function definition or a function call, and then not containing ";" which would indicate it is a call or declaration.
It is not worth worrying about:
foo(arg,
another);
that is not indented, so I think that simplicity is good.
Quoted 10 lines
For example, for top-level changes
outside functions, we might find:N_("some text that is long"
that is part of:
const char *foo =
N_("some text that is long"
"and spans multiple lines");
Unfortunate, but cannot be avoided.
Quoted 5 lines
Before this change, we would skip past it (using the cpp regex, that is; the default one tends to find the same line) and either report nothing, or whatever random function was before us. So it's a behavior change, but the existing behavior is really no better.
True.