userdiff: add buildin diff driver for JavaScript language
cover userdiff: add buildin diff driver for JavaScript language
I have already searched the git public-inbox don't find any active patch about userdiff build-in driver for JavaScript(there is an unfinished patch about three years ago). So I pick this as my GSoC micro project.
Below are typical function patterns from JavaScript, and also has an example that matches the corresponding pattern. These examples come from many popular JavaScript projects. Because I want to make sure the hunk header would work well on real-world projects.
Common function's pattern for JavaScript
1. normal function
^[\t ]*((export[\t ]+)?((async|get|set)[\t ]+)?function[\t ]*([\t ]*\\*[\t ]*|[\t ]*)?[$_[:alpha:]][$_[:alnum:]]*[\t ]*\\(.*) example:
1. https://github.com/mozilla/pdf.js/blob/85ff7b117e04471c550914b7a13193ab7f7ecc6a/src/display/canvas.js#L648
2. https://github.com/mozilla/pdf.js/blob/ad4b2ce021277ff7cea8ec7e32775c65d74ee673/test/unit/evaluator_spec.js#L40
2. JavaScript variable declaration with a lambda expression
`^^[\t ]*((const|let|var)[\t ]*[$_[:alpha:]][$_[:alnum:]]*[\t ]*=[\t ]*"
"(\\(.*\\)|[$_[:alpha:]][$_[:alnum:]]*[\t ])[\t ]*=>[\t ]*\\{?)` example:
1. https://github.com/webpack/webpack/blob/2279c5a2105ea1498b83a4854919aefe1a28c553/lib/ChunkGraph.js#L91
2. https://github.com/webpack/webpack/blob/2279c5a2105ea1498b83a4854919aefe1a28c553/lib/ChunkGraph.js#L122
I found sometimes would define function on this way. But this should only match the top level? Because it may match inside the function, and the below code would match the wrong function[1].
3. exports methods by assigning an anonymous function
^[\t ]*(exports\\.[$_[:alpha:]][$_[:alnum:]]*[\t ]*=[\t ]*(\\(.*\\)|[$_[:alpha:]][$_[:alnum:]]*)[\t ]*=>.*)
example:
1. https://github.com/webpack/webpack/blob/c181294865dca01b28e6e316636fef5f2aad4eb6/lib/dependencies/DynamicExports.js#L17
2. https://github.com/ajaxorg/ace/blob/d95725983b363a616c584237013dfd36eaadbba4/lib/ace/lib/dom.js#L37
4. expression about assign function to LHS
^(.*=[\t ]*function[\t ]*([$_[:alpha:]][$_[:alnum:]]*)?[\t ]*\\(.*)
example:
1. https://github.com/ajaxorg/ace/blob/94422a4a892495564c56089af85019a8f8f24673/lib/ace/anchor.js#L102
2. https://github.com/ajaxorg/ace/blob/d95725983b363a616c584237013dfd36eaadbba4/lib/ace/lib/dom.js#L37
3. https://github.com/ajaxorg/ace/blob/4257621787b4253d6d493611f4ec5a37829da323/lib/ace/search.js#L350
4. https://github.com/mozilla/pdf.js/blob/85ff7b117e04471c550914b7a13193ab7f7ecc6a/src/display/canvas.js#L299
Maybe this should only match on the 0,1,2 indent level? Because JavaScript may match the function assignment in another function.
5. normal function in object literal
^[\t ]*([$_[:alpha:]][$_[:alnum:]]*[\t ]*:[\t ]*function[\t ].*)
1. https://github.com/jquery/jquery/blob/de5398a6ad088dc006b46c6a870a2a053f4cd663/src/core.js#L201
2. https://github.com/mozilla/pdf.js/blob/85ff7b117e04471c550914b7a13193ab7f7ecc6a/src/display/canvas.js#L242
6. function in class
^[\t ]*((static[\t ]+)?((async|get|set)[\t ]+)?[$_[:alpha:]][$_[:alnum:]]*[\t ]*\\(.*)
This regex is tricky because the class's function is no function keyword in JavaScript.
If you write the regex to match them, it will match many non-function declaration things!!! Like examples below:
1. the non-function matches
1. https://github.com/ajaxorg/ace/blob/94422a4a892495564c56089af85019a8f8f24673/lib/ace/anchor.js#L58
2. https://github.com/ajaxorg/ace/blob/d95725983b363a616c584237013dfd36eaadbba4/lib/ace/lib/dom.js#L132
2. the function in class
1. https://github.com/mozilla/pdf.js/blob/85ff7b117e04471c550914b7a13193ab7f7ecc6a/src/display/canvas.js#L1929
2. https://github.com/webpack/webpack/blob/ccecc17c01af96edddb931a76e7a3b21ef2969d8/lib/Chunk.js#L179
3. https://github.com/webpack/webpack/blob/612de998f186a9bb2fe8769a91678df689a0541e/lib/Module.js#L242
4. https://github.com/mozilla/pdf.js/blob/5cf116a958548f6596674bf8d5ca0fe64aa2df3c/web/view_history.js#L75
5. https://github.com/mozilla/pdf.js/blob/5cf116a958548f6596674bf8d5ca0fe64aa2df3c/web/view_history.js#L89
My solution is to add some negation rules, and one rule is skipping the keywords that may have characters immediately following them in the parenthesis, rule is !^[ \t]*(if|do|while|for|with|switch|catch|import|return). Another negation rule is only before this 「function in class」 regex, that skips the line's indent level more than one because most of the function in class has one indent level(the class is on top-level). The negation rule is !^(\t{2,}|[ ]{5,}).
But this is not enough, because maybe has function call on one indent level. So need an negation rule for skipping statement. The negation rule is !^.*;[ \t]*. But the bad news is JavaScript's statement can end without a semicolon. So it still has an opportunity to recognize function call as the function declaration if the code's statement does not end with semicolons.
Word's pattern for JavaScript
In this part, I reference the formal ECMA specification heavily[2].
JavaScript has some special syntax, such as numbers can be separated with an underscore for readability[3]. And has BigInt literal, which is number end with a 「n」 character. So the number-related regex would be some differences with another language.
In the last, I had a fork git project on Github. And has the CI's result, the all test cases pass[4].
[1] https://github.com/webpack/webpack/blob/2279c5a2105ea1498b83a4854919aefe1a28c553/lib/ChunkGraph.js#L279
[2] https://262.ecma-international.org/12.0/#sec-ecmascript-language-lexical-grammar
[3] https://v8.dev/features/numeric-separators
[4] https://github.com/a97410985/git/actions/runs/1933091300
xing zhi jiang (1):
Add a diff driver for JavaScript languages.
.gitignore | 1 + Documentation/gitattributes.txt | 2 + ...avascript-assignment-of-anonymous-function | 4 ++ .../javascript-assignment-of-arrow-function | 4 ++ .../javascript-assignment-of-named-function | 4 ++ t/t4018/javascript-async-function | 4 ++ t/t4018/javascript-export-async-function | 4 ++ t/t4018/javascript-export-function | 4 ++ t/t4018/javascript-exports-anomyous-function | 4 ++ .../javascript-exports-anomyous-function-2 | 4 ++ t/t4018/javascript-exports-function | 4 ++ t/t4018/javascript-function | 4 ++ t/t4018/javascript-function-2 | 10 ++++ t/t4018/javascript-function-belong-to-IIFE | 6 +++ t/t4018/javascript-function-in-class | 6 +++ t/t4018/javascript-function-in-class-2 | 11 ++++ t/t4018/javascript-function-in-object-literal | 7 +++ t/t4018/javascript-generator-function | 4 ++ t/t4018/javascript-generator-function-2 | 4 ++ t/t4018/javascript-getter-function-in-class | 6 +++ t/t4018/javascript-setter-function-in-class | 6 +++ .../javascript-skip-function-call-statement | 7 +++ t/t4018/javascript-skip-keywords | 34 ++++++++++++ t/t4018/javascript-static-function-in-class | 6 +++ t/t4034-diff-words.sh | 1 + t/t4034/javascript/expect | 52 +++++++++++++++++++ t/t4034/javascript/post | 32 ++++++++++++ t/t4034/javascript/pre | 32 ++++++++++++ userdiff.c | 38 ++++++++++++++ 29 files changed, 305 insertions(+) create mode 100644 t/t4018/javascript-assignment-of-anonymous-function create mode 100644 t/t4018/javascript-assignment-of-arrow-function create mode 100644 t/t4018/javascript-assignment-of-named-function create mode 100644 t/t4018/javascript-async-function create mode 100644 t/t4018/javascript-export-async-function create mode 100644 t/t4018/javascript-export-function create mode 100644 t/t4018/javascript-exports-anomyous-function create mode 100644 t/t4018/javascript-exports-anomyous-function-2 create mode 100644 t/t4018/javascript-exports-function create mode 100644 t/t4018/javascript-function create mode 100644 t/t4018/javascript-function-2 create mode 100644 t/t4018/javascript-function-belong-to-IIFE create mode 100644 t/t4018/javascript-function-in-class create mode 100644 t/t4018/javascript-function-in-class-2 create mode 100644 t/t4018/javascript-function-in-object-literal create mode 100644 t/t4018/javascript-generator-function create mode 100644 t/t4018/javascript-generator-function-2 create mode 100644 t/t4018/javascript-getter-function-in-class create mode 100644 t/t4018/javascript-setter-function-in-class create mode 100644 t/t4018/javascript-skip-function-call-statement create mode 100644 t/t4018/javascript-skip-keywords create mode 100644 t/t4018/javascript-static-function-in-class create mode 100644 t/t4034/javascript/expect create mode 100644 t/t4034/javascript/post create mode 100644 t/t4034/javascript/pre
base-commit: e6ebfd0e8cbbd10878070c8a356b5ad1b3ca464e
Signature · 1 line
2.35.1.273.ge6ebfd0e8c.dirty
PATCH 1/1 Add a diff driver for JavaScript languages.
In the xfunction part that matches normal functions,
a variable declaration with an assignment of function, the function declaration in the class, and also the function is object literal's property.
And in the word regex part, that matches numbers, punctuations, and also the JavaScript identifier.
This part heavily references the formal ECMA sepcification[1].
[1]https://262.ecma-international.org/12.0/#sec-ecmascript-language-lexical-grammar
--- .gitignore | 1 + Documentation/gitattributes.txt | 2 + ...avascript-assignment-of-anonymous-function | 4 ++ .../javascript-assignment-of-arrow-function | 4 ++ .../javascript-assignment-of-named-function | 4 ++ t/t4018/javascript-async-function | 4 ++ t/t4018/javascript-export-async-function | 4 ++ t/t4018/javascript-export-function | 4 ++ t/t4018/javascript-exports-anomyous-function | 4 ++ .../javascript-exports-anomyous-function-2 | 4 ++ t/t4018/javascript-exports-function | 4 ++ t/t4018/javascript-function | 4 ++ t/t4018/javascript-function-2 | 10 ++++ t/t4018/javascript-function-belong-to-IIFE | 6 +++ t/t4018/javascript-function-in-class | 6 +++ t/t4018/javascript-function-in-class-2 | 11 ++++ t/t4018/javascript-function-in-object-literal | 7 +++ t/t4018/javascript-generator-function | 4 ++ t/t4018/javascript-generator-function-2 | 4 ++ t/t4018/javascript-getter-function-in-class | 6 +++ t/t4018/javascript-setter-function-in-class | 6 +++ .../javascript-skip-function-call-statement | 7 +++ t/t4018/javascript-skip-keywords | 34 ++++++++++++ t/t4018/javascript-static-function-in-class | 6 +++ t/t4034-diff-words.sh | 1 + t/t4034/javascript/expect | 52 +++++++++++++++++++ t/t4034/javascript/post | 32 ++++++++++++ t/t4034/javascript/pre | 32 ++++++++++++ userdiff.c | 38 ++++++++++++++ 29 files changed, 305 insertions(+) create mode 100644 t/t4018/javascript-assignment-of-anonymous-function create mode 100644 t/t4018/javascript-assignment-of-arrow-function create mode 100644 t/t4018/javascript-assignment-of-named-function create mode 100644 t/t4018/javascript-async-function create mode 100644 t/t4018/javascript-export-async-function create mode 100644 t/t4018/javascript-export-function create mode 100644 t/t4018/javascript-exports-anomyous-function create mode 100644 t/t4018/javascript-exports-anomyous-function-2 create mode 100644 t/t4018/javascript-exports-function create mode 100644 t/t4018/javascript-function create mode 100644 t/t4018/javascript-function-2 create mode 100644 t/t4018/javascript-function-belong-to-IIFE create mode 100644 t/t4018/javascript-function-in-class create mode 100644 t/t4018/javascript-function-in-class-2 create mode 100644 t/t4018/javascript-function-in-object-literal create mode 100644 t/t4018/javascript-generator-function create mode 100644 t/t4018/javascript-generator-function-2 create mode 100644 t/t4018/javascript-getter-function-in-class create mode 100644 t/t4018/javascript-setter-function-in-class create mode 100644 t/t4018/javascript-skip-function-call-statement create mode 100644 t/t4018/javascript-skip-keywords create mode 100644 t/t4018/javascript-static-function-in-class create mode 100644 t/t4034/javascript/expect create mode 100644 t/t4034/javascript/post create mode 100644 t/t4034/javascript/pre
.gitignore +1 -0
@@ -244,3 +244,4 @@ Release/ /git.VC.db *.dSYM /contrib/buildsystems/out +/.cache
Documentation/gitattributes.txt +2 -0
@@ -828,6 +828,8 @@ patterns are available: - `java` suitable for source code in the Java language. +- `javascript suitable for source code in the JavaScript language. + - `markdown` suitable for Markdown documents. - `matlab` suitable for source code in the MATLAB and Octave languages.
t/t4018/javascript-assignment-of-anonymous-function new file +4 -0
@@ -0,0 +1,4 @@ +const RIGHT = function (a, b) { + + return a + b; // ChangeMe +};
t/t4018/javascript-assignment-of-arrow-function new file +4 -0
@@ -0,0 +1,4 @@ +const RIGHT = (a, b) => { + + return a + b; // ChangeMe +};
t/t4018/javascript-assignment-of-named-function new file +4 -0
@@ -0,0 +1,4 @@ +const RIGHT = function test (a, b) { + + return a + b; // ChangeMe +};
t/t4018/javascript-async-function new file +4 -0
@@ -0,0 +1,4 @@ +async function RIGHT(a, b) { + + return a + b; // ChangeMe +}
t/t4018/javascript-export-async-function new file +4 -0
@@ -0,0 +1,4 @@ +export async function RIGHT(a, b) { + + return a + b; // ChangeMe +}
t/t4018/javascript-export-function new file +4 -0
@@ -0,0 +1,4 @@ +export function RIGHT(a, b) { + + return a + b; // ChangeMe +}
t/t4018/javascript-exports-anomyous-function new file +4 -0
@@ -0,0 +1,4 @@ +exports.setFlagged = RIGHT => { + + return ChangeMe; +};
t/t4018/javascript-exports-anomyous-function-2 new file +4 -0
@@ -0,0 +1,4 @@ +exports.RIGHT = (a, b, runtime) => { + + return ChangeMe; +};
t/t4018/javascript-exports-function new file +4 -0
@@ -0,0 +1,4 @@ +exports.RIGHT = function(document) { + + return ChangeMe +}
t/t4018/javascript-function new file +4 -0
@@ -0,0 +1,4 @@ +function RIGHT(a, b) { + + return a + b; // ChangeMe +}
t/t4018/javascript-function-2 new file +10 -0
@@ -0,0 +1,10 @@ +function test(a, b) { + return { + RIGHT: function () { + currentUpdateRemovedChunks.forEach(function (chunkId) { + delete $installedChunks$[chunkId]; + }); + currentUpdateRemovedChunks = ChangeMe; + } + } +}
t/t4018/javascript-function-belong-to-IIFE new file +6 -0
@@ -0,0 +1,6 @@ +(function () { + this.$RIGHT = function (needle, modifier) { + let a = 5; + return ChangeMe; + }; +}).call(aaaa.prototype);
t/t4018/javascript-function-in-class new file +6 -0
@@ -0,0 +1,6 @@ +class Test { + RIGHT() { + let a = 4; + let b = ChangeMe; + } +}
t/t4018/javascript-function-in-class-2 new file +11 -0
@@ -0,0 +1,11 @@ +class Test { + RIGHT( + aaaaaaaaaa, + bbbbbbbbbb, + cccccccccc, + dddddddddd + ) { + let a = 4; + let b = ChangeMe; + } +}
t/t4018/javascript-function-in-object-literal new file +7 -0
@@ -0,0 +1,7 @@ +const obj = { + RIGHT: function (elems, callback, arg) { + var length, value; + // ... + return ChangeMe + } +}
t/t4018/javascript-generator-function new file +4 -0
@@ -0,0 +1,4 @@ +function* RIGHT(a, b) { + + return a + b; // ChangeMe +}
t/t4018/javascript-generator-function-2 new file +4 -0
@@ -0,0 +1,4 @@ +function *RIGHT(a, b) { + + return a + b; // ChangeMe +}
t/t4018/javascript-getter-function-in-class new file +6 -0
@@ -0,0 +1,6 @@ +class Test { + get RIGHT() { + let a = 4; + let b = ChangeMe; + } +}
t/t4018/javascript-setter-function-in-class new file +6 -0
@@ -0,0 +1,6 @@ +class Test { + set RIGHT() { + let a = 4; + let b = ChangeMe; + } +}
t/t4018/javascript-skip-function-call-statement new file +7 -0
@@ -0,0 +1,7 @@ +class Test { + static RIGHT() { + haha(); + haha2() + let b = ChangeMe; + } +}
t/t4018/javascript-skip-keywords new file +34 -0
@@ -0,0 +1,34 @@ +function RIGHT(a, b) { + import("./async1") + if (a > 1) { + // ... + } + do { + // ... + } while (i < 5); + for (const element of array1) { + console.log(element) + } + with(o) { + console.log(x) + } + switch (expr) { + case 'a': + // ... + break; + case 'b': + // ... + break; + default: + // ... + } + try { + // ... + return (a + c) + } + catch (error) { + // ... + } + + return a + b; // ChangeMe +}
t/t4018/javascript-static-function-in-class new file +6 -0
@@ -0,0 +1,6 @@ +class Test { + static RIGHT() { + let a = 4; + let b = ChangeMe; + } +}
t/t4034-diff-words.sh +1 -0
@@ -324,6 +324,7 @@ test_language_driver dts test_language_driver fortran test_language_driver html test_language_driver java +test_language_driver javascript test_language_driver matlab test_language_driver objc test_language_driver pascal
t/t4034/javascript/expect new file +52 -0
52 lines not on this page · raw message
t/t4034/javascript/post new file +32 -0
@@ -0,0 +1,32 @@ +// DecimalLiteral +124 +0.124 +0.123e-5 +0.123E-5 +0.123E5 +12222222222222222233344445n +// HexIntegerLiteral +0x11 +0X5Fa1 +0x123_756 +0x1234182989812f1289bn +// OctalIntegerLiteral +06 +0o5 +0O4 +0511_567 +0o424242424242424242424242424242666667n +// BinaryIntegerLiteral +0b1101 +0B0010 +0b0001_1101_0011 +0b11111111111111000011111111111111111n +// punctuations +{b} (b) +b; +[2,3] +[ ...params_v2 ] +a=2 a=2 a=2 a=2 a=2 a=2 a=2 a=2 +a-=b a%=b a&&=b a|=b +b-c a++ a<<b a>>>=b a<<=b +a&b a|b a??=b
t/t4034/javascript/pre new file +32 -0
@@ -0,0 +1,32 @@ +// DecimalLiteral +123 +0.123 +0.123e+5 +0.123E+5 +0.123e5 +1222222222222222223334444n +// HexIntegerLiteral +0x10 +0X6Fa1 +0x123_456 +0x1234182989812f1289an +// OctalIntegerLiteral +05 +0o6 +0O7 +0512_567 +0o424242424242424242424242424242666666n +// BinaryIntegerLiteral +0b1001 +0B0110 +0b0001_1001_0011 +0b1111111111111111111111111111111111111n +// punctuations +{a} (a) +a; +[1,2] +[ 1, 2, ...params ] +a<=2 a>=2 a==2 a!=2 a===2 a!==2 a^=2 a=>2 +a+=b a*=b a**=b a||=b +b+c a-- a>>b a>>>b a>>=b +a&&b a||b a&&=b
userdiff.c +38 -0
@@ -168,6 +168,44 @@ PATTERNS("java","|[-+0-9.e]+[fFlL]?|0[xXbB]?[0-9a-fA-F]+[lL]?" "|[-+*/<>%&^|=!]=" "|--|\\+\\+|<<=?|>>>?=?|&&|\\|\\|"), + +PATTERNS ("javascript" , +/* don't match the expression may contain parenthesis, because it is not a function declaration */ +"!^[ \t]*(if|do|while|for|with|switch|catch|import|return)\n" +/* don't match statement */ +"!^.*;[ \t]*\n" +/* match normal function */ +"^[\t ]*((export[\t ]+)?(async[\t ]+)?function[\t ]*([\t ]*\\*[\t ]*|[\t ]*)?[$_[:alpha:]][$_[:alnum:]]*[\t ]*\\(.*)\n" +/* match JavaScript variable declaration with a lambda expression */ +"^[\t ]*((const|let|var)[\t ]*[$_[:alpha:]][$_[:alnum:]]*[\t ]*=[\t ]*" +"(\\(.*\\)|[$_[:alpha:]][$_[:alnum:]]*[\t ])[\t ]*=>[\t ]*\\{?)\n" +/* match exports for anonymous fucntion */ +"^[\t ]*(exports\\.[$_[:alpha:]][$_[:alnum:]]*[\t ]*=[\t ]*(\\(.*\\)|[$_[:alpha:]][$_[:alnum:]]*)[\t ]*=>.*)\n" +/* match assign function to LHS */ +"^(.*=[\t ]*function[\t ]*([$_[:alpha:]][$_[:alnum:]]*)?[\t ]*\\(.*)\n" +/* match normal function in object literal */ +"^[\t ]*([$_[:alpha:]][$_[:alnum:]]*[\t ]*:[\t ]*function[\t ].*)\n" +/* don't match the function in class, which has more than one ident level */ +"!^(\t{2,}|[ ]{5,})\n" +/* match function in class */ +"^[\t ]*((static[\t ]+)?((async|get|set)[\t ]+)?[$_[:alpha:]][$_[:alnum:]]*[\t ]*\\(.*)" , +/* word regex */ +/* hexIntegerLiteral and bigHexIntegerLiteral*/ +"0[xX][0-9a-fA-F][_0-9a-fA-F]*n?|" +/* octalIntegerLiteral and bigOctalIntegerLiteral */ +"0[oO]?[0-7][_0-7]*n?|" +/* binaryIntegerLiteral and bigBinaryIntegerLiteral */ +"(0[bB][01][_01]*n?)|" +/* decimalLiteral */ +"(0|[1-9][_0-9]*)?\\.?[0-9][_0-9]*([eE][+-]?[_0-9]+)?|" +/* bigDecimalLiteral */ +"(0|[1-9][_0-9]*)n|" +/* punctuations */ +"\\{|\\}|\\(|\\)|\\.|\\.{3}|;|,|<|>|<=|>=|==|!=|={3}|!==|\\+|-|\\*|/|%|\\*{2}|" +"\\+{2}|--|<<|>>|>>>|&|\\||\\^|!|~|&&|\\|{2}|\\?{1,2}|:|=|\\+=|-=|\\*=|%=|\\*{2}=|" +"<<=|>>=|>>>=|&=|\\|=|\\^=|&&=|\\|{2}=|\\?{2}=|=>|" +/* identifiers */ +"[$_[:alpha:]][$_[:alnum:]]*" ), PATTERNS("markdown", "^ {0,3}#{1,6}[ \t].*",/* -- */
Signature · 1 line
2.35.1.273.ge6ebfd0e8c.dirty
Re: [GSoC][PATCH 1/1] Add a diff driver for JavaScript languages.
Thank you for your contribution!
Am 04.03.22 um 14:08 schrieb xing zhi jiang:
Quoted 3 lines
In the xfunction part that matches normal functions,
a variable declaration with an assignment of function, the function declaration in the class, and also the function is object literal's property.
On the first read, I stumbled over the last half-sentence. Is this JavaScript-lingo for the construct
x = { foo: function() { ... } }
A parenthetical note in this regard would be helpful.
Quoted 5 lines
And in the word regex part, that matches numbers, punctuations, and also the JavaScript identifier.
This part heavily references the formal ECMA sepcification[1].[1]https://262.ecma-international.org/12.0/#sec-ecmascript-language-lexical-grammar
After having seen the regex, to follow the syntax specification is unnecessarily tight. If you follow my suggestions, the note should probably be rewritten, but keeping the link to the language reference is certainly helpful.
Quoted 64 lines
Signed-off-by xing zhi jiang <[email protected]>--- .gitignore | 1 + Documentation/gitattributes.txt | 2 + ...avascript-assignment-of-anonymous-function | 4 ++ .../javascript-assignment-of-arrow-function | 4 ++ .../javascript-assignment-of-named-function | 4 ++ t/t4018/javascript-async-function | 4 ++ t/t4018/javascript-export-async-function | 4 ++ t/t4018/javascript-export-function | 4 ++ t/t4018/javascript-exports-anomyous-function | 4 ++ .../javascript-exports-anomyous-function-2 | 4 ++ t/t4018/javascript-exports-function | 4 ++ t/t4018/javascript-function | 4 ++ t/t4018/javascript-function-2 | 10 ++++ t/t4018/javascript-function-belong-to-IIFE | 6 +++ t/t4018/javascript-function-in-class | 6 +++ t/t4018/javascript-function-in-class-2 | 11 ++++ t/t4018/javascript-function-in-object-literal | 7 +++ t/t4018/javascript-generator-function | 4 ++ t/t4018/javascript-generator-function-2 | 4 ++ t/t4018/javascript-getter-function-in-class | 6 +++ t/t4018/javascript-setter-function-in-class | 6 +++ .../javascript-skip-function-call-statement | 7 +++ t/t4018/javascript-skip-keywords | 34 ++++++++++++ t/t4018/javascript-static-function-in-class | 6 +++ t/t4034-diff-words.sh | 1 + t/t4034/javascript/expect | 52 +++++++++++++++++++ t/t4034/javascript/post | 32 ++++++++++++ t/t4034/javascript/pre | 32 ++++++++++++ userdiff.c | 38 ++++++++++++++ 29 files changed, 305 insertions(+) create mode 100644 t/t4018/javascript-assignment-of-anonymous-function create mode 100644 t/t4018/javascript-assignment-of-arrow-function create mode 100644 t/t4018/javascript-assignment-of-named-function create mode 100644 t/t4018/javascript-async-function create mode 100644 t/t4018/javascript-export-async-function create mode 100644 t/t4018/javascript-export-function create mode 100644 t/t4018/javascript-exports-anomyous-function create mode 100644 t/t4018/javascript-exports-anomyous-function-2 create mode 100644 t/t4018/javascript-exports-function create mode 100644 t/t4018/javascript-function create mode 100644 t/t4018/javascript-function-2 create mode 100644 t/t4018/javascript-function-belong-to-IIFE create mode 100644 t/t4018/javascript-function-in-class create mode 100644 t/t4018/javascript-function-in-class-2 create mode 100644 t/t4018/javascript-function-in-object-literal create mode 100644 t/t4018/javascript-generator-function create mode 100644 t/t4018/javascript-generator-function-2 create mode 100644 t/t4018/javascript-getter-function-in-class create mode 100644 t/t4018/javascript-setter-function-in-class create mode 100644 t/t4018/javascript-skip-function-call-statement create mode 100644 t/t4018/javascript-skip-keywords create mode 100644 t/t4018/javascript-static-function-in-class create mode 100644 t/t4034/javascript/expect create mode 100644 t/t4034/javascript/post create mode 100644 t/t4034/javascript/pre.gitignore +0
-0
.gitignore Release/ · line 247
Quoted 5 lines
.gitignore +1
-0@@ -244,3 +244,4 @@ Release/ /git.VC.db *.dSYM /contrib/buildsystems/out +/.cache
Do not include this change. It does not belong to this commit.
Documentation/gitattributes.txt patterns are available: · line 831
Quoted 9 lines
Documentation/gitattributes.txt +1
-0@@ -828,6 +828,8 @@ patterns are available: - `java` suitable for source code in the Java language. +- `javascript suitable for source code in the JavaScript language.
Please do not forget the closing quote.
Quoted 10 lines
Documentation/gitattributes.txt +1
-0+ - `markdown` suitable for Markdown documents. - `matlab` suitable for source code in the MATLAB and Octave languages.t/t4018/javascript-assignment-of-anonymous-function new file +0
-0
t/t4018/javascript-assignment-of-anonymous-function line 4
Quoted 5 lines
t/t4018/javascript-assignment-of-anonymous-function new file +4
-0@@ -0,0 +1,4 @@ +const RIGHT = function (a, b) { + + return a + b; // ChangeMe +};
Notice this "No newline at end of file". Please complete the last line of the file, i.e. do add the last line break. Same for all other new files introduced in this patch.
Quoted 83 lines
t/t4018/javascript-assignment-of-arrow-function new file +4
-0@@ -0,0 +1,4 @@ +const RIGHT = (a, b) => { + + return a + b; // ChangeMe +};t/t4018/javascript-assignment-of-named-function new file +4
-0@@ -0,0 +1,4 @@ +const RIGHT = function test (a, b) { + + return a + b; // ChangeMe +};t/t4018/javascript-async-function new file +4
-0@@ -0,0 +1,4 @@ +async function RIGHT(a, b) { + + return a + b; // ChangeMe +}t/t4018/javascript-export-async-function new file +4
-0@@ -0,0 +1,4 @@ +export async function RIGHT(a, b) { + + return a + b; // ChangeMe +}t/t4018/javascript-export-function new file +4
-0@@ -0,0 +1,4 @@ +export function RIGHT(a, b) { + + return a + b; // ChangeMe +}t/t4018/javascript-exports-anomyous-function new file +4
-0@@ -0,0 +1,4 @@ +exports.setFlagged = RIGHT => { + + return ChangeMe; +};t/t4018/javascript-exports-anomyous-function-2 new file +4
-0@@ -0,0 +1,4 @@ +exports.RIGHT = (a, b, runtime) => { + + return ChangeMe; +};t/t4018/javascript-exports-function new file +0
-0
t/t4018/javascript-exports-function line 4
Quoted 4 lines
t/t4018/javascript-exports-function new file +4
-0@@ -0,0 +1,4 @@ +exports.RIGHT = function(document) { + + return ChangeMe +}
Not a particularly important point, but the correct syntax requires a semicolon here, I guess.
Quoted 18 lines
\ No newline at end of file
t/t4018/javascript-function new file +4
-0@@ -0,0 +1,4 @@ +function RIGHT(a, b) { + + return a + b; // ChangeMe +}t/t4018/javascript-function-2 new file +0
-0
t/t4018/javascript-function-2 line 4
Quoted 4 lines
t/t4018/javascript-function-2 new file +4
-0@@ -0,0 +1,10 @@ +function test(a, b) { + return { + RIGHT: function () { + currentUpdateRemovedChunks.forEach(function (chunkId) {
This line is a decoy that is not picked up. Nice.
Quoted 41 lines
t/t4018/javascript-function-2 new file +6
-0+ delete $installedChunks$[chunkId]; + }); + currentUpdateRemovedChunks = ChangeMe; + } + } +}t/t4018/javascript-function-belong-to-IIFE new file +6
-0@@ -0,0 +1,6 @@ +(function () { + this.$RIGHT = function (needle, modifier) { + let a = 5; + return ChangeMe; + }; +}).call(aaaa.prototype);t/t4018/javascript-function-in-class new file +6
-0@@ -0,0 +1,6 @@ +class Test { + RIGHT() { + let a = 4; + let b = ChangeMe; + } +}t/t4018/javascript-function-in-class-2 new file +2
-0@@ -0,0 +1,11 @@ +class Test { + RIGHT(
t/t4018/javascript-function-in-class-2 line 11
Quoted 10 lines
t/t4018/javascript-function-in-class-2 new file +9
-0+ aaaaaaaaaa, + bbbbbbbbbb, + cccccccccc, + dddddddddd + ) { + let a = 4; + let b = ChangeMe; + } +}
In the above two, we see class member functions. Is there a test case where the function parameter is on the same line as the function name or is that one of the difficult cases?
Quoted 68 lines
t/t4018/javascript-function-in-object-literal new file +7
-0@@ -0,0 +1,7 @@ +const obj = { + RIGHT: function (elems, callback, arg) { + var length, value; + // ... + return ChangeMe + } +}t/t4018/javascript-generator-function new file +4
-0@@ -0,0 +1,4 @@ +function* RIGHT(a, b) { + + return a + b; // ChangeMe +}t/t4018/javascript-generator-function-2 new file +4
-0@@ -0,0 +1,4 @@ +function *RIGHT(a, b) { + + return a + b; // ChangeMe +}t/t4018/javascript-getter-function-in-class new file +6
-0@@ -0,0 +1,6 @@ +class Test { + get RIGHT() { + let a = 4; + let b = ChangeMe; + } +}t/t4018/javascript-setter-function-in-class new file +6
-0@@ -0,0 +1,6 @@ +class Test { + set RIGHT() { + let a = 4; + let b = ChangeMe; + } +}t/t4018/javascript-skip-function-call-statement new file +0
-0
t/t4018/javascript-skip-function-call-statement line 7
Quoted 7 lines
t/t4018/javascript-skip-function-call-statement new file +7
-0@@ -0,0 +1,7 @@ +class Test { + static RIGHT() { + haha(); + haha2() + let b = ChangeMe; + } +}
Good call to include this test case!
Quoted 31 lines
\ No newline at end of file
t/t4018/javascript-skip-keywords new file +24
-0@@ -0,0 +1,34 @@ +function RIGHT(a, b) { + import("./async1") + if (a > 1) { + // ... + } + do { + // ... + } while (i < 5); + for (const element of array1) { + console.log(element) + } + with(o) { + console.log(x) + } + switch (expr) { + case 'a': + // ... + break; + case 'b': + // ... + break; + default: + // ... + }
t/t4018/javascript-skip-keywords line 34
Quoted 10 lines
t/t4018/javascript-skip-keywords new file +10
-0+ try { + // ... + return (a + c) + } + catch (error) { + // ... + } + + return a + b; // ChangeMe +}
Very well!
Quoted 74 lines
\ No newline at end of file
t/t4018/javascript-static-function-in-class new file +6
-0@@ -0,0 +1,6 @@ +class Test { + static RIGHT() { + let a = 4; + let b = ChangeMe; + } +}t/t4034-diff-words.sh +1
-0@@ -324,6 +324,7 @@ test_language_driver dts test_language_driver fortran test_language_driver html test_language_driver java +test_language_driver javascript test_language_driver matlab test_language_driver objc test_language_driver pascalt/t4034/javascript/expect new file +42
-0@@ -0,0 +1,52 @@ +<BOLD>diff --git a/pre b/post<RESET> +<BOLD>index b72238f..8bc3e3a 100644<RESET> +<BOLD>--- a/pre<RESET> +<BOLD>+++ b/post<RESET> +<CYAN>@@ -1,32 +1,32 @@<RESET> +// DecimalLiteral<RESET> +<RED>123<RESET> +<RED>0.123<RESET> +<RED>0.123e+5<RESET> +<RED>0.123E+5<RESET> +<RED>0.123e5<RESET> +<RED>1222222222222222223334444n<RESET><GREEN>124<RESET> +<GREEN>0.124<RESET> +<GREEN>0.123e-5<RESET> +<GREEN>0.123E-5<RESET> +<GREEN>0.123E5<RESET> +<GREEN>12222222222222222233344445n<RESET> +// HexIntegerLiteral<RESET> +<RED>0x10<RESET> +<RED>0X6Fa1<RESET> +<RED>0x123_456<RESET> +<RED>0x1234182989812f1289an<RESET><GREEN>0x11<RESET> +<GREEN>0X5Fa1<RESET> +<GREEN>0x123_756<RESET> +<GREEN>0x1234182989812f1289bn<RESET> +// OctalIntegerLiteral<RESET> +<RED>05<RESET> +<RED>0o6<RESET> +<RED>0O7<RESET> +<RED>0512_567<RESET> +<RED>0o424242424242424242424242424242666666n<RESET><GREEN>06<RESET> +<GREEN>0o5<RESET> +<GREEN>0O4<RESET> +<GREEN>0511_567<RESET> +<GREEN>0o424242424242424242424242424242666667n<RESET> +// BinaryIntegerLiteral<RESET> +<RED>0b1001<RESET> +<RED>0B0110<RESET> +<RED>0b0001_1001_0011<RESET> +<RED>0b1111111111111111111111111111111111111n<RESET><GREEN>0b1101<RESET> +<GREEN>0B0010<RESET> +<GREEN>0b0001_1101_0011<RESET>
t/t4034/javascript/expect line 52
Quoted 10 lines
t/t4034/javascript/expect new file +10
-0+<GREEN>0b11111111111111000011111111111111111n<RESET> +// punctuations<RESET> +{<RED>a<RESET><GREEN>b<RESET>} (<RED>a<RESET><GREEN>b<RESET>) +<RED>a<RESET><GREEN>b<RESET>; +[<RED>1,<RESET>2<GREEN>,3<RESET>] +[<RED>1, 2,<RESET> ...<RED>params<RESET><GREEN>params_v2<RESET> ] +a<RED><=<RESET><GREEN>=<RESET>2 a<RED>>=<RESET><GREEN>=<RESET>2 a<RED>==<RESET><GREEN>=<RESET>2 a<RED>!=<RESET><GREEN>=<RESET>2 a<RED>===<RESET><GREEN>=<RESET>2 a<RED>!==<RESET><GREEN>=<RESET>2 a<RED>^=<RESET><GREEN>=<RESET>2 a<RED>=><RESET><GREEN>=<RESET>2 +a<RED>+=<RESET><GREEN>-=<RESET>b a<RED>*=<RESET><GREEN>%=<RESET>b a<RED>**=<RESET><GREEN>&&=<RESET>b a<RED>||=<RESET><GREEN>|=<RESET>b +b<RED>+<RESET><GREEN>-<RESET>c a<RED>--<RESET><GREEN>++<RESET> a<RED>>><RESET><GREEN><<<RESET>b a<RED>>>><RESET><GREEN>>>>=<RESET>b a<RED>>>=<RESET><GREEN><<=<RESET>b +a<RED>&&<RESET><GREEN>&<RESET>b a<RED>||<RESET><GREEN>|<RESET>b a<RED>&&=<RESET><GREEN>??=<RESET>b
This looks good! I see many changes in operators being tested.
Quoted 83 lines
t/t4034/javascript/post new file +32
-0@@ -0,0 +1,32 @@ +// DecimalLiteral +124 +0.124 +0.123e-5 +0.123E-5 +0.123E5 +12222222222222222233344445n +// HexIntegerLiteral +0x11 +0X5Fa1 +0x123_756 +0x1234182989812f1289bn +// OctalIntegerLiteral +06 +0o5 +0O4 +0511_567 +0o424242424242424242424242424242666667n +// BinaryIntegerLiteral +0b1101 +0B0010 +0b0001_1101_0011 +0b11111111111111000011111111111111111n +// punctuations +{b} (b) +b; +[2,3] +[ ...params_v2 ] +a=2 a=2 a=2 a=2 a=2 a=2 a=2 a=2 +a-=b a%=b a&&=b a|=b +b-c a++ a<<b a>>>=b a<<=b +a&b a|b a??=bt/t4034/javascript/pre new file +32
-0@@ -0,0 +1,32 @@ +// DecimalLiteral +123 +0.123 +0.123e+5 +0.123E+5 +0.123e5 +1222222222222222223334444n +// HexIntegerLiteral +0x10 +0X6Fa1 +0x123_456 +0x1234182989812f1289an +// OctalIntegerLiteral +05 +0o6 +0O7 +0512_567 +0o424242424242424242424242424242666666n +// BinaryIntegerLiteral +0b1001 +0B0110 +0b0001_1001_0011 +0b1111111111111111111111111111111111111n +// punctuations +{a} (a) +a; +[1,2] +[ 1, 2, ...params ] +a<=2 a>=2 a==2 a!=2 a===2 a!==2 a^=2 a=>2 +a+=b a*=b a**=b a||=b +b+c a-- a>>b a>>>b a>>=b +a&&b a||b a&&=buserdiff.c +0
-0
userdiff.c PATTERNS("java", · line 174
Quoted 7 lines
userdiff.c +4
-0@@ -168,6 +168,44 @@ PATTERNS("java","|[-+0-9.e]+[fFlL]?|0[xXbB]?[0-9a-fA-F]+[lL]?" "|[-+*/<>%&^|=!]=" "|--|\\+\\+|<<=?|>>>?=?|&&|\\|\\|"), + +PATTERNS ("javascript" , +/* don't match the expression may contain parenthesis, because it is not a function declaration */ + "!^[ \t]*(if|do|while|for|with|switch|catch|import|return)\n"
These will not match
}while (expr)
note the absent blank before the keyword, but that is an acceptable trade-off to keep things simple. Good.
userdiff.c PATTERNS("java", · line 176
Quoted 2 lines
userdiff.c +2
-0+/* don't match statement */ +"!^.*;[ \t]*\n"
This regexp can be reduced to
"!;\n"
no?
userdiff.c PATTERNS("java", · line 178
Quoted 2 lines
userdiff.c +2
-0+/* match normal function */ +"^[\t ]*((export[\t ]+)?(async[\t ]+)?function[\t ]*([\t ]*\\*[\t ]*|[\t ]*)?[$_[:alpha:]][$_[:alnum:]]*[\t ]*\\(.*)\n"
Good. One note though: keyword "function" can optionally be followed by an asterisk '*'. You can probably simplify the middle part to
...function[\t *]*...identifier...
userdiff.c PATTERNS("java", · line 181
Quoted 3 lines
userdiff.c +3
-0+/* match JavaScript variable declaration with a lambda expression */ +"^[\t ]*((const|let|var)[\t ]*[$_[:alpha:]][$_[:alnum:]]*[\t ]*=[\t ]*" +"(\\(.*\\)|[$_[:alpha:]][$_[:alnum:]]*[\t ])[\t ]*=>[\t ]*\\{?)\n"
Let's break this down:
"^[\t ]*"
"("
"(const|let|var)[\t ]*"
"[$_[:alpha:]][$_[:alnum:]]*[\t ]*"
"=[\t ]*"
"("
"\\(.*\\)"
"|"
"[$_[:alpha:]][$_[:alnum:]]*[\t ]"
")[\t ]*"
"=>[\t ]*"
"\\{?"
")\n"Can you not have
var f = foo=>{because I see that whitespace is required between the identifier and "=>"?
userdiff.c PATTERNS("java", · line 183
Quoted 2 lines
userdiff.c +2
-0+/* match exports for anonymous fucntion */ +"^[\t ]*(exports\\.[$_[:alpha:]][$_[:alnum:]]*[\t ]*=[\t ]*(\\(.*\\)|[$_[:alpha:]][$_[:alnum:]]*)[\t ]*=>.*)\n"
Here, whitespace is not required. Is the above an oversight?
BTW, can keyword "exports" be used for something other than functions?
userdiff.c PATTERNS("java", · line 185
Quoted 2 lines
userdiff.c +2
-0+/* match assign function to LHS */ +"^(.*=[\t ]*function[\t ]*([$_[:alpha:]][$_[:alnum:]]*)?[\t ]*\\(.*)\n"
This makes me think that whenever keyword "function" appears, then we see the beginning of a function. This would allow a simple pattern upfront that picks out all functions defined with this keyword, and all other patterns need only be concerned with the exceptional cases.
/* "function" is first non-space token */ "^[\t ]*function[\t ].*)\n" /* "function" is not first token */ "^.*[^$_[:alnum:]]function[\t ].*\n"
userdiff.c PATTERNS("java", · line 189
Quoted 4 lines
userdiff.c +4
-0+/* match normal function in object literal */ +"^[\t ]*([$_[:alpha:]][$_[:alnum:]]*[\t ]*:[\t ]*function[\t ].*)\n" +/* don't match the function in class, which has more than one ident level */ +"!^(\t{2,}|[ ]{5,})\n"
For some, hopefully universally agreed upon in the JavaScript community, definition of indentation level. Ok...
userdiff.c PATTERNS("java", · line 191
Quoted 2 lines
userdiff.c +2
-0+/* match function in class */ + "^[\t ]*((static[\t ]+)?((async|get|set)[\t ]+)?[$_[:alpha:]][$_[:alnum:]]*[\t ]*\\(.*)",
Aren't "get" and "set" as universal identifiers of functions or can they occur on other contexts? Thinking of it, they can occur in comments everywhere, so they would pick up too many matches if treated like my "function" proposal above.
userdiff.c PATTERNS("java", · line 202
Quoted 11 lines
userdiff.c +11
-0+/* word regex */ +/* hexIntegerLiteral and bigHexIntegerLiteral*/ +"0[xX][0-9a-fA-F][_0-9a-fA-F]*n?|" +/* octalIntegerLiteral and bigOctalIntegerLiteral */ +"0[oO]?[0-7][_0-7]*n?|" +/* binaryIntegerLiteral and bigBinaryIntegerLiteral */ +"(0[bB][01][_01]*n?)|" +/* decimalLiteral */ +"(0|[1-9][_0-9]*)?\\.?[0-9][_0-9]*([eE][+-]?[_0-9]+)?|" +/* bigDecimalLiteral */ +"(0|[1-9][_0-9]*)n|"
You do not have to make the word-regex so tight that it excludes incorrect literals because you can assume that incorrect literals will not occur. In particular integers beginning with a 0 need not be treated specially. You can fold the octal and decimal integers into a single expression:
"[0-9][oO]?[_0-9.]*)([eE][+-]?[_0-9]+)?n?"
Are floatingpoint literals beginning with a decimal point like .5 not permitted?
Please follow the custom to place the alternation character "|" at the beginning of the next line, not at the end of the previous.
userdiff.c PATTERNS("java", · line 206
Quoted 4 lines
userdiff.c +4
-0+/* punctuations */ +"\\{|\\}|\\(|\\)|\\.|\\.{3}|;|,|<|>|<=|>=|==|!=|={3}|!==|\\+|-|\\*|/|%|\\*{2}|" +"\\+{2}|--|<<|>>|>>>|&|\\||\\^|!|~|&&|\\|{2}|\\?{1,2}|:|=|\\+=|-=|\\*=|%=|\\*{2}=|" +"<<=|>>=|>>>=|&=|\\|=|\\^=|&&=|\\|{2}=|\\?{2}=|=>|"
You could collaps many of the operators into single alternatives, but if you prefer it this way, it is fine, too. One example:
">{1,3}="But please remove the single-character operators from the list, because there is an implicit single-character alternative that is not visible in the code here.
userdiff.c PATTERNS("java", · line 211
Quoted 5 lines
userdiff.c +2
-0+/* identifiers */ + "[$_[:alpha:]][$_[:alnum:]]*"), PATTERNS("markdown", "^ {0,3}#{1,6}[ \t].*",/* -- */
-- Hannes
Re: [GSoC][PATCH 1/1] Add a diff driver for JavaScript languages.
Thanks for your review and give much good advice !!!
I will apply much of your advice to my v2 patch. But before I write the v2 patch,
there is still something to check and discuss.
Quoted 19 lines · Johannes Sixt
Thank you for your contribution!
Am 04.03.22 um 14:08 schrieb xing zhi jiang:
Quoted 3 lines
In the xfunction part that matches normal functions,
a variable declaration with an assignment of function, the function declaration in the class, and also the function is object literal's property.On the first read, I stumbled over the last half-sentence. Is this JavaScript-lingo for the construct
x = { foo: function() { ... } }A parenthetical note in this regard would be helpful.
Quoted 4 lines
And in the word regex part, that matches numbers, punctuations, and also the JavaScript identifier.
This part heavily references the formal ECMA sepcification[1].
Quoted 6 lines
After having seen the regex, to follow the syntax specification is unnecessarily tight. If you follow my suggestions, the note should probably be rewritten, but keeping the link to the language reference is certainly helpful.
OK, I will adjust the commit message but remain the reference link.
Quoted 59 lines
Quoted 59 lines
Signed-off-by xing zhi jiang <[email protected]>--- .gitignore | 1 + Documentation/gitattributes.txt | 2 + ...avascript-assignment-of-anonymous-function | 4 ++ .../javascript-assignment-of-arrow-function | 4 ++ .../javascript-assignment-of-named-function | 4 ++ t/t4018/javascript-async-function | 4 ++ t/t4018/javascript-export-async-function | 4 ++ t/t4018/javascript-export-function | 4 ++ t/t4018/javascript-exports-anomyous-function | 4 ++ .../javascript-exports-anomyous-function-2 | 4 ++ t/t4018/javascript-exports-function | 4 ++ t/t4018/javascript-function | 4 ++ t/t4018/javascript-function-2 | 10 ++++ t/t4018/javascript-function-belong-to-IIFE | 6 +++ t/t4018/javascript-function-in-class | 6 +++ t/t4018/javascript-function-in-class-2 | 11 ++++ t/t4018/javascript-function-in-object-literal | 7 +++ t/t4018/javascript-generator-function | 4 ++ t/t4018/javascript-generator-function-2 | 4 ++ t/t4018/javascript-getter-function-in-class | 6 +++ t/t4018/javascript-setter-function-in-class | 6 +++ .../javascript-skip-function-call-statement | 7 +++ t/t4018/javascript-skip-keywords | 34 ++++++++++++ t/t4018/javascript-static-function-in-class | 6 +++ t/t4034-diff-words.sh | 1 + t/t4034/javascript/expect | 52 +++++++++++++++++++ t/t4034/javascript/post | 32 ++++++++++++ t/t4034/javascript/pre | 32 ++++++++++++ userdiff.c | 38 ++++++++++++++ 29 files changed, 305 insertions(+) create mode 100644 t/t4018/javascript-assignment-of-anonymous-function create mode 100644 t/t4018/javascript-assignment-of-arrow-function create mode 100644 t/t4018/javascript-assignment-of-named-function create mode 100644 t/t4018/javascript-async-function create mode 100644 t/t4018/javascript-export-async-function create mode 100644 t/t4018/javascript-export-function create mode 100644 t/t4018/javascript-exports-anomyous-function create mode 100644 t/t4018/javascript-exports-anomyous-function-2 create mode 100644 t/t4018/javascript-exports-function create mode 100644 t/t4018/javascript-function create mode 100644 t/t4018/javascript-function-2 create mode 100644 t/t4018/javascript-function-belong-to-IIFE create mode 100644 t/t4018/javascript-function-in-class create mode 100644 t/t4018/javascript-function-in-class-2 create mode 100644 t/t4018/javascript-function-in-object-literal create mode 100644 t/t4018/javascript-generator-function create mode 100644 t/t4018/javascript-generator-function-2 create mode 100644 t/t4018/javascript-getter-function-in-class create mode 100644 t/t4018/javascript-setter-function-in-class create mode 100644 t/t4018/javascript-skip-function-call-statement create mode 100644 t/t4018/javascript-skip-keywords create mode 100644 t/t4018/javascript-static-function-in-class create mode 100644 t/t4034/javascript/expect create mode 100644 t/t4034/javascript/post create mode 100644 t/t4034/javascript/pre
Quoted 12 lines
.gitignore Release/ · line 247
Quoted 10 lines
.gitignore +1
-0@@ -244,3 +244,4 @@ Release/ /git.VC.db *.dSYM /contrib/buildsystems/out +/.cacheDo not include this change. It does not belong to this commit.
OK
Quoted 11 lines
Documentation/gitattributes.txt patterns are available: · line 831
Quoted 9 lines
Documentation/gitattributes.txt +1
-0@@ -828,6 +828,8 @@ patterns are available: - `java` suitable for source code in the Java language. +- `javascript suitable for source code in the JavaScript language.Please do not forget the closing quote.
I will fix it in v2 patch.
Quoted 19 lines
Quoted 10 lines
Documentation/gitattributes.txt +1
-0+ - `markdown` suitable for Markdown documents. - `matlab` suitable for source code in the MATLAB and Octave languages.t/t4018/javascript-assignment-of-anonymous-function new file +0
-0t/t4018/javascript-assignment-of-anonymous-function line 4
Quoted 5 lines
t/t4018/javascript-assignment-of-anonymous-function new file +4
-0@@ -0,0 +1,4 @@ +const RIGHT = function (a, b) { + + return a + b; // ChangeMe +};Notice this "No newline at end of file". Please complete the last line of the file, i.e. do add the last line break. Same for all other new files introduced in this patch.
I will fix it in v2 patch.
Quoted 82 lines
Quoted 82 lines
t/t4018/javascript-assignment-of-arrow-function new file +4
-0@@ -0,0 +1,4 @@ +const RIGHT = (a, b) => { + + return a + b; // ChangeMe +};t/t4018/javascript-assignment-of-named-function new file +4
-0@@ -0,0 +1,4 @@ +const RIGHT = function test (a, b) { + + return a + b; // ChangeMe +};t/t4018/javascript-async-function new file +4
-0@@ -0,0 +1,4 @@ +async function RIGHT(a, b) { + + return a + b; // ChangeMe +}t/t4018/javascript-export-async-function new file +4
-0@@ -0,0 +1,4 @@ +export async function RIGHT(a, b) { + + return a + b; // ChangeMe +}t/t4018/javascript-export-function new file +4
-0@@ -0,0 +1,4 @@ +export function RIGHT(a, b) { + + return a + b; // ChangeMe +}t/t4018/javascript-exports-anomyous-function new file +4
-0@@ -0,0 +1,4 @@ +exports.setFlagged = RIGHT => { + + return ChangeMe; +};t/t4018/javascript-exports-anomyous-function-2 new file +4
-0@@ -0,0 +1,4 @@ +exports.RIGHT = (a, b, runtime) => { + + return ChangeMe; +};t/t4018/javascript-exports-function new file +0
-0
Quoted 9 lines
t/t4018/javascript-exports-function line 4
Quoted 5 lines
t/t4018/javascript-exports-function new file +4
-0@@ -0,0 +1,4 @@ +exports.RIGHT = function(document) { + + return ChangeMe +}Not a particularly important point, but the correct syntax requires a semicolon here, I guess.
Yes, I forgot the semicolon here.
Quoted 47 lines
Quoted 47 lines
t/t4018/javascript-exports-function new file +6
-0+ delete $installedChunks$[chunkId]; + }); + currentUpdateRemovedChunks = ChangeMe; + } + } +}t/t4018/javascript-function-belong-to-IIFE new file +6
-0@@ -0,0 +1,6 @@ +(function () { + this.$RIGHT = function (needle, modifier) { + let a = 5; + return ChangeMe; + }; +}).call(aaaa.prototype);t/t4018/javascript-function-in-class new file +6
-0@@ -0,0 +1,6 @@ +class Test { + RIGHT() { + let a = 4; + let b = ChangeMe; + } +}t/t4018/javascript-function-in-class-2 new file +8
-0@@ -0,0 +1,11 @@ +class Test { + RIGHT( + aaaaaaaaaa, + bbbbbbbbbb, + cccccccccc, + dddddddddd + ) { + let a = 4;
Quoted 8 lines
t/t4018/javascript-function-in-class-2 line 11
Quoted 4 lines
t/t4018/javascript-function-in-class-2 new file +3
-0+ let b = ChangeMe; + } +}In the above two, we see class member functions. Is there a test case where the function parameter is on the same line as the function name or is that one of the difficult cases?
I can't get what you mean, but the second Test case wants to make sure we can match the function with a long parameter list, and its parameters are formatted to multiline.
Quoted 93 lines
Quoted 83 lines
t/t4034/javascript/post new file +32
-0@@ -0,0 +1,32 @@ +// DecimalLiteral +124 +0.124 +0.123e-5 +0.123E-5 +0.123E5 +12222222222222222233344445n +// HexIntegerLiteral +0x11 +0X5Fa1 +0x123_756 +0x1234182989812f1289bn +// OctalIntegerLiteral +06 +0o5 +0O4 +0511_567 +0o424242424242424242424242424242666667n +// BinaryIntegerLiteral +0b1101 +0B0010 +0b0001_1101_0011 +0b11111111111111000011111111111111111n +// punctuations +{b} (b) +b; +[2,3] +[ ...params_v2 ] +a=2 a=2 a=2 a=2 a=2 a=2 a=2 a=2 +a-=b a%=b a&&=b a|=b +b-c a++ a<<b a>>>=b a<<=b +a&b a|b a??=bt/t4034/javascript/pre new file +32
-0@@ -0,0 +1,32 @@ +// DecimalLiteral +123 +0.123 +0.123e+5 +0.123E+5 +0.123e5 +1222222222222222223334444n +// HexIntegerLiteral +0x10 +0X6Fa1 +0x123_456 +0x1234182989812f1289an +// OctalIntegerLiteral +05 +0o6 +0O7 +0512_567 +0o424242424242424242424242424242666666n +// BinaryIntegerLiteral +0b1001 +0B0110 +0b0001_1001_0011 +0b1111111111111111111111111111111111111n +// punctuations +{a} (a) +a; +[1,2] +[ 1, 2, ...params ] +a<=2 a>=2 a==2 a!=2 a===2 a!==2 a^=2 a=>2 +a+=b a*=b a**=b a||=b +b+c a-- a>>b a>>>b a>>=b +a&&b a||b a&&=buserdiff.c +0
-0userdiff.c PATTERNS("java", · line 174
Quoted 7 lines
userdiff.c +4
-0@@ -168,6 +168,44 @@ PATTERNS("java","|[-+0-9.e]+[fFlL]?|0[xXbB]?[0-9a-fA-F]+[lL]?" "|[-+*/<>%&^|=!]=" "|--|\\+\\+|<<=?|>>>?=?|&&|\\|\\|"), + +PATTERNS ("javascript" , +/* don't match the expression may contain parenthesis, because it is not a function declaration */ + "!^[ \t]*(if|do|while|for|with|switch|catch|import|return)\n"These will not match
Quoted 5 lines
}while (expr)note the absent blank before the keyword, but that is an acceptable trade-off to keep things simple. Good.
I thought not to match with }while (expr) is OK because this rule is mainly to prevent matching too many
wrong things about the regex for 「the function in the class」.
And before keywords maybe need blanks, because some javascript formatting style is using space
instead of tab.
Quoted 7 lines
userdiff.c PATTERNS("java", · line 176
Quoted 2 lines
userdiff.c +2
-0+/* don't match statement */ +"!^.*;[ \t]*\n" This regexp can be reduced to
"!;\n"
good advice, I would apply it.
Quoted 7 lines
userdiff.c PATTERNS("java", · line 178
Quoted 2 lines
userdiff.c +2
-0+/* match normal function */ +"^[\t ]*((export[\t ]+)?(async[\t ]+)?function[\t ]*([\t ]*\\*[\t ]*|[\t ]*)?[$_[:alpha:]][$_[:alnum:]]*[\t ]*\\(.*)\n" Good. One note though: keyword "function" can optionally be followed by an asterisk '*'. You can probably simplify the middle part to
...function[\t *]*...identifier...
good advice, I would apply it.
Quoted 20 lines
userdiff.c PATTERNS("java", · line 181
Quoted 3 lines
userdiff.c +3
-0+/* match JavaScript variable declaration with a lambda expression */ +"^[\t ]*((const|let|var)[\t ]*[$_[:alpha:]][$_[:alnum:]]*[\t ]*=[\t ]*" +"(\\(.*\\)|[$_[:alpha:]][$_[:alnum:]]*[\t ])[\t ]*=>[\t ]*\\{?)\n" Let's break this down:
"^[\t ]*" "(" "(const|let|var)[\t ]*" "[$_[:alpha:]][$_[:alnum:]]*[\t ]*" "=[\t ]*" "(" "\\(.*\\)" "|" "[$_[:alpha:]][$_[:alnum:]]*[\t ]" ")[\t ]*" "=>[\t ]*" "\\{?" ")\n"
Quoted 5 lines
Can you not have
var f = foo=>{because I see that whitespace is required between the identifier and "=>"?
It is my mistake. I should not match tab or space between identifier and =>. Thanks for pointing it out. I will fix it.
Quoted 7 lines
userdiff.c PATTERNS("java", · line 183
Quoted 2 lines
userdiff.c +2
-0+/* match exports for anonymous fucntion */ +"^[\t ]*(exports\\.[$_[:alpha:]][$_[:alnum:]]*[\t ]*=[\t ]*(\\(.*\\)|[$_[:alpha:]][$_[:alnum:]]*)[\t ]*=>.*)\n" Here, whitespace is not required. Is the above an oversight?
BTW, can keyword "exports" be used for something other than functions?
「Whitespace is not required」 is only referred to 「^[\t ]*(exports」? I would remove the [\t ]* before exports
keyword because exports must be top-level.
The exports keyword can be used for exporting many things, such as string. ex: exports.SimpleMessage = 'Hello world';
, So I define a more precise regex for only matching anonymous functions.
Quoted 8 lines
userdiff.c PATTERNS("java", · line 185
Quoted 2 lines
userdiff.c +2
-0+/* match assign function to LHS */ +"^(.*=[\t ]*function[\t ]*([$_[:alpha:]][$_[:alnum:]]*)?[\t ]*\\(.*)\n" This makes me think that whenever the keyword "function" appears, then we see the beginning of a function. This would allow a simple pattern upfront that picks out all functions defined with this keyword, and all other patterns need only be concerned with the exceptional cases.
Quoted 4 lines
/* "function" is first non-space token */
"^[\t ]*function[\t ].*)\n"
/* "function" is not first token */
"^.*[^$_[:alnum:]]function[\t ].*\n"This is good advice, I would try.
Quoted 7 lines
userdiff.c PATTERNS("java", · line 189
Quoted 4 lines
userdiff.c +4
-0+/* match normal function in object literal */ +"^[\t ]*([$_[:alpha:]][$_[:alnum:]]*[\t ]*:[\t ]*function[\t ].*)\n" +/* don't match the function in class, which has more than one ident level */ +"!^(\t{2,}|[ ]{5,})\n" For some, hopefully universally agreed upon in the JavaScript community, definition of indentation level. Ok...
I had searched the web, the javascript's style about indentation level has two kinds -
ident with 2 or 4 spaces. So I don't match the line, which begins with more than four
spaces. If javascript must end with a semicolon, it doesn't need this rule :(
Quoted 7 lines
userdiff.c PATTERNS("java", · line 191
Quoted 2 lines
userdiff.c +2
-0+/* match function in class */ + "^[\t ]*((static[\t ]+)?((async|get|set)[\t ]+)?[$_[:alpha:]][$_[:alnum:]]*[\t ]*\\(.*)",Aren't "get" and "set" as universal identifiers of functions or can they occur on other contexts? Thinking of it, they can occur in comments everywhere, so they would pick up too many matches if treated like my "function" proposal above.
"get" and "set" are not javascript keywords, we could write code like "let get = 5;"
In my view, it wouldn't match the comment, because the comment has "//" or "/*. But I would think about it.
Quoted 18 lines
userdiff.c PATTERNS("java", · line 202
Quoted 11 lines
userdiff.c +11
-0+/* word regex */ +/* hexIntegerLiteral and bigHexIntegerLiteral*/ +"0[xX][0-9a-fA-F][_0-9a-fA-F]*n?|" +/* octalIntegerLiteral and bigOctalIntegerLiteral */ +"0[oO]?[0-7][_0-7]*n?|" +/* binaryIntegerLiteral and bigBinaryIntegerLiteral */ +"(0[bB][01][_01]*n?)|" +/* decimalLiteral */ +"(0|[1-9][_0-9]*)?\\.?[0-9][_0-9]*([eE][+-]?[_0-9]+)?|" +/* bigDecimalLiteral */ +"(0|[1-9][_0-9]*)n|" You do not have to make the word-regex so tight that it excludes incorrect literals because you can assume that incorrect literals will not occur. In particular integers beginning with a 0 need not be treated specially. You can fold the octal and decimal integers into a single expression:
Quoted 4 lines
"[0-9][oO]?[_0-9.]*)([eE][+-]?[_0-9]+)?n?"Are floatingpoint literals beginning with a decimal point like .5 not permitted?
This is permitted. ex: "let a = .5" is valid.
And I would follow your advice to combine multiple regexes.
Quoted 2 lines
Please follow the custom to place the alternation character "|" at the beginning of the next line, not at the end of the previous.
OK.
Quoted 14 lines
userdiff.c PATTERNS("java", · line 206
Quoted 4 lines
userdiff.c +4
-0+/* punctuations */ +"\\{|\\}|\\(|\\)|\\.|\\.{3}|;|,|<|>|<=|>=|==|!=|={3}|!==|\\+|-|\\*|/|%|\\*{2}|" +"\\+{2}|--|<<|>>|>>>|&|\\||\\^|!|~|&&|\\|{2}|\\?{1,2}|:|=|\\+=|-=|\\*=|%=|\\*{2}=|" +"<<=|>>=|>>>=|&=|\\|=|\\^=|&&=|\\|{2}=|\\?{2}=|=>|" You could collaps many of the operators into single alternatives, but if you prefer it this way, it is fine, too. One example:
">{1,3}="But please remove the single-character operators from the list, because there is an implicit single-character alternative that is not visible in the code here.
Quoted 7 lines
userdiff.c PATTERNS("java", · line 211
Quoted 5 lines
userdiff.c +2
-0+/* identifiers */ + "[$_[:alpha:]][$_[:alnum:]]*"), PATTERNS("markdown", "^ {0,3}#{1,6}[ \t].*",/* -- */ -- Hannes
OK.
In the end, I am very much grateful for your review.
Best regards.
Re: [GSoC][PATCH 1/1] Add a diff driver for JavaScript languages.
Am 07.03.22 um 16:10 schrieb xing-zhi jiang:
Quoted 24 lines · Johannes Sixt
Quoted 15 lines
Am 04.03.22 um 14:08 schrieb xing zhi jiang:
Quoted 14 lines
t/t4018/javascript-function-in-class-2 new file +8
-0@@ -0,0 +1,11 @@ +class Test { + RIGHT( + aaaaaaaaaa, + bbbbbbbbbb, + cccccccccc, + dddddddddd + ) { + let a = 4;Quoted 8 lines
t/t4018/javascript-function-in-class-2 line 11
Quoted 4 lines
t/t4018/javascript-function-in-class-2 new file +3
-0+ let b = ChangeMe; + } +}In the above two, we see class member functions. Is there a test case where the function parameter is on the same line as the function name or is that one of the difficult cases?
Quoted 3 lines
I can't get what you mean, but the second Test case wants to make sure we can match the function with a long parameter list, and its parameters are formatted to multiline.
I meant a multi-line function call like so, where the first argument is on the same line with the function name:
RIGHT(aaaaaaaaaa,
bbbbbbbbbb,
cccccccccc,
dddddddddd
) {
...Quoted 20 lines
Quoted 15 lines
userdiff.c PATTERNS("java", · line 174
Quoted 12 lines
userdiff.c +4
-0@@ -168,6 +168,44 @@ PATTERNS("java","|[-+0-9.e]+[fFlL]?|0[xXbB]?[0-9a-fA-F]+[lL]?" "|[-+*/<>%&^|=!]=" "|--|\\+\\+|<<=?|>>>?=?|&&|\\|\\|"), + +PATTERNS ("javascript" , +/* don't match the expression may contain parenthesis, because it is not a function declaration */ + "!^[ \t]*(if|do|while|for|with|switch|catch|import|return)\n"These will not match
Quoted 5 lines
}while (expr)note the absent blank before the keyword, but that is an acceptable trade-off to keep things simple. Good.
Quoted 6 lines
I thought not to match with
}while (expr)is OK because this rule is mainly to prevent matching too many
wrong things about the regex for 「the function in the class」.
And before keywords maybe need blanks, because some javascript formatting style is using space
instead of tab.
I don't recall why I emphasize the absent blank between '}' and "while". I must have misread something. At any rate, the regex would not match, either, if there is whitespace between the two, so, it is totally OK.
Quoted 8 lines
Quoted 5 lines
Can you not have
var f = foo=>{because I see that whitespace is required between the identifier and "=>"?
It is my mistake. I should not match tab or space between identifier and =>. Thanks for pointing it out. I will fix it.
Quoted 8 lines
Quoted 7 lines
userdiff.c PATTERNS("java", · line 183
Quoted 2 lines
userdiff.c +2
-0+/* match exports for anonymous fucntion */ +"^[\t ]*(exports\\.[$_[:alpha:]][$_[:alnum:]]*[\t ]*=[\t ]*(\\(.*\\)|[$_[:alpha:]][$_[:alnum:]]*)[\t ]*=>.*)\n" Here, whitespace is not required. Is the above an oversight?
BTW, can keyword "exports" be used for something other than functions?
「Whitespace is not required」 is only referred to 「^[\t ]*(exports」?
No, I refer to the whitespace around "=>" that is not required here, but was required in the previous expression.
Quoted 6 lines
I
would remove the [\t ]* before exports
keyword because exports must be top-level.
The exports keyword can be used for exporting many things, such as string. ex: exports.SimpleMessage = 'Hello world';
, So I define a more precise regex for only matching anonymous functions.
Understood. "exports" is a keyword only in certain contexts. Since it can be used as (or like?) a variable name, it cannot be used as match that always identifies a function header.
-- Hannes
Re: [GSoC][PATCH 1/1] Add a diff driver for JavaScript languages.
Quoted 9 lines · Johannes Sixt
I meant a multi-line function call like so, where the first argument is on the same line with the function name:
RIGHT(aaaaaaaaaa,
bbbbbbbbbb,
cccccccccc,
dddddddddd
) {
...
OK, I got it. I thought it is not a difficult case, but I add a new test case for this.
Quoted 2 lines
<No, I refer to the whitespace around "=>" that is not required here, but was required in the previous expression.
I think the space is required, ex:
https://github.com/webpack/webpack/blob/c181294865dca01b28e6e316636fef5f2aad4eb6/lib/dependencies/DynamicExports.js#L17
. The link above it has space before 「=>」.
Best regards.
Am 04.03.22 um 14:08 schrieb xing zhi jiang:
Quoted 9 lines
I have already searched the git public-inbox don't find any active patch about userdiff build-in driver for JavaScript(there is an unfinished patch about three years ago). So I pick this as my GSoC micro project.
Below are typical function patterns from JavaScript, and also has an example that matches the corresponding pattern. These examples come from many popular JavaScript projects. Because I want to make sure the hunk header would work well on real-world projects.
Thank you for the thorough research. I've already responded to the patch about the technical solution. Here are some general thoughts.
Quoted 9 lines
Common function's pattern for JavaScript
1. normal function
^[\t ]*((export[\t ]+)?((async|get|set)[\t ]+)?function[\t ]*([\t ]*\\*[\t ]*|[\t ]*)?[$_[:alpha:]][$_[:alnum:]]*[\t ]*\\(.*)example:
1. https://github.com/mozilla/pdf.js/blob/85ff7b117e04471c550914b7a13193ab7f7ecc6a/src/display/canvas.js#L648
2. https://github.com/mozilla/pdf.js/blob/ad4b2ce021277ff7cea8ec7e32775c65d74ee673/test/unit/evaluator_spec.js#L40
2. JavaScript variable declaration with a lambda expression
`^^[\t ]*((const|let|var)[\t ]*[$_[:alpha:]][$_[:alnum:]]*[\t ]*=[\t ]*"
Quoted 7 lines
"(\\(.*\\)|[$_[:alpha:]][$_[:alnum:]]*[\t ])[\t ]*=>[\t ]*\\{?)`example:
1. https://github.com/webpack/webpack/blob/2279c5a2105ea1498b83a4854919aefe1a28c553/lib/ChunkGraph.js#L91
2. https://github.com/webpack/webpack/blob/2279c5a2105ea1498b83a4854919aefe1a28c553/lib/ChunkGraph.js#L122I found sometimes would define function on this way. But this should only match the top level? Because it may match inside the function, and the below code would match the wrong function[1].
Functions nested in other functions in general lead to surprising results with a stateless parser. There is not much you can do if the nesting level cannot be determined. Using the indentation as a proxy may work, but it depends on that the indentation width is agreed on in the community.
Note that the function header patterns are used for these tasks:
1. For hunk headers in patch text
2. To find the context in diff and grep --function-context
3. To find the lines of interest in log -L:function_name:file.name
For the first, mismatching the function is secondary because it is (IMO) just a hint for the reader about where the hunk should apply.
For the others, finding the actual beginning and end of a function would be desirable. This does not work as intended when patterns match nested function headers. But a general solution to this problem basically means to implement a parser for the language.
Quoted 41 lines
3. exports methods by assigning an anonymous function
^[\t ]*(exports\\.[$_[:alpha:]][$_[:alnum:]]*[\t ]*=[\t ]*(\\(.*\\)|[$_[:alpha:]][$_[:alnum:]]*)[\t ]*=>.*)
example:
1. https://github.com/webpack/webpack/blob/c181294865dca01b28e6e316636fef5f2aad4eb6/lib/dependencies/DynamicExports.js#L17
2. https://github.com/ajaxorg/ace/blob/d95725983b363a616c584237013dfd36eaadbba4/lib/ace/lib/dom.js#L37
4. expression about assign function to LHS
^(.*=[\t ]*function[\t ]*([$_[:alpha:]][$_[:alnum:]]*)?[\t ]*\\(.*)
example:
1. https://github.com/ajaxorg/ace/blob/94422a4a892495564c56089af85019a8f8f24673/lib/ace/anchor.js#L102
2. https://github.com/ajaxorg/ace/blob/d95725983b363a616c584237013dfd36eaadbba4/lib/ace/lib/dom.js#L37
3. https://github.com/ajaxorg/ace/blob/4257621787b4253d6d493611f4ec5a37829da323/lib/ace/search.js#L350
4. https://github.com/mozilla/pdf.js/blob/85ff7b117e04471c550914b7a13193ab7f7ecc6a/src/display/canvas.js#L299Maybe this should only match on the 0,1,2 indent level? Because JavaScript may match the function assignment in another function.
5. normal function in object literal
^[\t ]*([$_[:alpha:]][$_[:alnum:]]*[\t ]*:[\t ]*function[\t ].*)
1. https://github.com/jquery/jquery/blob/de5398a6ad088dc006b46c6a870a2a053f4cd663/src/core.js#L201
2. https://github.com/mozilla/pdf.js/blob/85ff7b117e04471c550914b7a13193ab7f7ecc6a/src/display/canvas.js#L242
6. function in class
^[\t ]*((static[\t ]+)?((async|get|set)[\t ]+)?[$_[:alpha:]][$_[:alnum:]]*[\t ]*\\(.*)This regex is tricky because the class's function is no function keyword in JavaScript.
If you write the regex to match them, it will match many non-function declaration things!!! Like examples below:
1. the non-function matches
1. https://github.com/ajaxorg/ace/blob/94422a4a892495564c56089af85019a8f8f24673/lib/ace/anchor.js#L58
2. https://github.com/ajaxorg/ace/blob/d95725983b363a616c584237013dfd36eaadbba4/lib/ace/lib/dom.js#L132
2. the function in class
1. https://github.com/mozilla/pdf.js/blob/85ff7b117e04471c550914b7a13193ab7f7ecc6a/src/display/canvas.js#L1929
2. https://github.com/webpack/webpack/blob/ccecc17c01af96edddb931a76e7a3b21ef2969d8/lib/Chunk.js#L179
3. https://github.com/webpack/webpack/blob/612de998f186a9bb2fe8769a91678df689a0541e/lib/Module.js#L242
4. https://github.com/mozilla/pdf.js/blob/5cf116a958548f6596674bf8d5ca0fe64aa2df3c/web/view_history.js#L75
5. https://github.com/mozilla/pdf.js/blob/5cf116a958548f6596674bf8d5ca0fe64aa2df3c/web/view_history.js#L89My solution is to add some negation rules, and one rule is skipping the keywords that may have characters immediately following them in the parenthesis, rule is
!^[ \t]*(if|do|while|for|with|switch|catch|import|return). Another negation rule is only before this 「function in class」 regex, that skips the line's indent level more than one because most of the function in class has one indent level(the class is on top-level). The negation rule is!^(\t{2,}|[ ]{5,}).
Quoted 4 lines
But this is not enough, because maybe has function call on one indent level. So need an negation rule for skipping statement. The negation rule is
!^.*;[ \t]*. But the bad news is JavaScript's statement can end without a semicolon. So it still has an opportunity to recognize function call as the function declaration if the code's statement does not end with semicolons.
The patterns need not be perfect, but "only" reasonably useful, i.e., suitable for the majority of the actual codebase. If it turns out that too many lines match in practice, the patterns can be tweaked.
Quoted 74 lines
Word's pattern for JavaScript
In this part, I reference the formal ECMA specification heavily[2].
JavaScript has some special syntax, such as numbers can be separated with an underscore for readability[3]. And has BigInt literal, which is number end with a 「n」 character. So the number-related regex would be some differences with another language.In the last, I had a fork git project on Github. And has the CI's result, the all test cases pass[4].
[1] https://github.com/webpack/webpack/blob/2279c5a2105ea1498b83a4854919aefe1a28c553/lib/ChunkGraph.js#L279
[2] https://262.ecma-international.org/12.0/#sec-ecmascript-language-lexical-grammar
[3] https://v8.dev/features/numeric-separators
[4] https://github.com/a97410985/git/actions/runs/1933091300xing zhi jiang (1):
Add a diff driver for JavaScript languages..gitignore | 1 + Documentation/gitattributes.txt | 2 + ...avascript-assignment-of-anonymous-function | 4 ++ .../javascript-assignment-of-arrow-function | 4 ++ .../javascript-assignment-of-named-function | 4 ++ t/t4018/javascript-async-function | 4 ++ t/t4018/javascript-export-async-function | 4 ++ t/t4018/javascript-export-function | 4 ++ t/t4018/javascript-exports-anomyous-function | 4 ++ .../javascript-exports-anomyous-function-2 | 4 ++ t/t4018/javascript-exports-function | 4 ++ t/t4018/javascript-function | 4 ++ t/t4018/javascript-function-2 | 10 ++++ t/t4018/javascript-function-belong-to-IIFE | 6 +++ t/t4018/javascript-function-in-class | 6 +++ t/t4018/javascript-function-in-class-2 | 11 ++++ t/t4018/javascript-function-in-object-literal | 7 +++ t/t4018/javascript-generator-function | 4 ++ t/t4018/javascript-generator-function-2 | 4 ++ t/t4018/javascript-getter-function-in-class | 6 +++ t/t4018/javascript-setter-function-in-class | 6 +++ .../javascript-skip-function-call-statement | 7 +++ t/t4018/javascript-skip-keywords | 34 ++++++++++++ t/t4018/javascript-static-function-in-class | 6 +++ t/t4034-diff-words.sh | 1 + t/t4034/javascript/expect | 52 +++++++++++++++++++ t/t4034/javascript/post | 32 ++++++++++++ t/t4034/javascript/pre | 32 ++++++++++++ userdiff.c | 38 ++++++++++++++ 29 files changed, 305 insertions(+) create mode 100644 t/t4018/javascript-assignment-of-anonymous-function create mode 100644 t/t4018/javascript-assignment-of-arrow-function create mode 100644 t/t4018/javascript-assignment-of-named-function create mode 100644 t/t4018/javascript-async-function create mode 100644 t/t4018/javascript-export-async-function create mode 100644 t/t4018/javascript-export-function create mode 100644 t/t4018/javascript-exports-anomyous-function create mode 100644 t/t4018/javascript-exports-anomyous-function-2 create mode 100644 t/t4018/javascript-exports-function create mode 100644 t/t4018/javascript-function create mode 100644 t/t4018/javascript-function-2 create mode 100644 t/t4018/javascript-function-belong-to-IIFE create mode 100644 t/t4018/javascript-function-in-class create mode 100644 t/t4018/javascript-function-in-class-2 create mode 100644 t/t4018/javascript-function-in-object-literal create mode 100644 t/t4018/javascript-generator-function create mode 100644 t/t4018/javascript-generator-function-2 create mode 100644 t/t4018/javascript-getter-function-in-class create mode 100644 t/t4018/javascript-setter-function-in-class create mode 100644 t/t4018/javascript-skip-function-call-statement create mode 100644 t/t4018/javascript-skip-keywords create mode 100644 t/t4018/javascript-static-function-in-class create mode 100644 t/t4034/javascript/expect create mode 100644 t/t4034/javascript/post create mode 100644 t/t4034/javascript/pre
Quoted 1 line
base-commit:
e6ebfd0e8cbbd10878070c8a356b5ad1b3ca464e
-- Hannes
PATCH v2 1/1 Add a diff driver for JavaScript languages.
In reply to xing zhi jiangIn the xfunction part that matches normal functions,
a variable declaration with an assignment of function, the function declaration in the class, and also the function is object literal's property[1].
And in the word regex part, that matches numbers, punctuations, and also the JavaScript identifier.
This part reference the formal ECMA specification[2].
[1]https://github.com/jquery/jquery/blob/de5398a6ad088dc006b46c6a870a2a053f4cd663/src/core.js#L201 [2]https://262.ecma-international.org/12.0/#sec-ecmascript-language-lexical-grammar
---
Range-diff against v1:
1: 7f764f97cf ! 1: 3b326bd2b6 Add a diff driver for JavaScript languages.
@@ Commit message
In the xfunction part that matches normal functions,
a variable declaration with an assignment of function, the function declaration
- in the class, and also the function is object literal's property.
+ in the class, and also the function is object literal's property[1].
And in the word regex part, that matches numbers, punctuations, and also the
JavaScript identifier.
- This part heavily references the formal ECMA sepcification[1].
+ This part reference the formal ECMA specification[2].
- [1]https://262.ecma-international.org/12.0/#sec-ecmascript-language-lexical-grammar
+ [1]https://github.com/jquery/jquery/blob/de5398a6ad088dc006b46c6a870a2a053f4cd663/src/core.js#L201
+ [2]https://262.ecma-international.org/12.0/#sec-ecmascript-language-lexical-grammar
Signed-off-by: xing zhi jiang <[email protected]>
- ## .gitignore ##
-@@ .gitignore: Release/
- /git.VC.db
- *.dSYM
- /contrib/buildsystems/out
-+/.cache
- \ No newline at end of file
-
## Documentation/gitattributes.txt ##
@@ Documentation/gitattributes.txt: patterns are available:
- `java` suitable for source code in the Java language.
-+- `javascript suitable for source code in the JavaScript language.
++- `javascript` suitable for source code in the JavaScript language.
+
- `markdown` suitable for Markdown documents.
@@ t/t4018/javascript-assignment-of-anonymous-function (new)
+
+ return a + b; // ChangeMe
+};
- \ No newline at end of file
## t/t4018/javascript-assignment-of-arrow-function (new) ##
@@
@@ t/t4018/javascript-assignment-of-arrow-function (new)
+
+ return a + b; // ChangeMe
+};
- \ No newline at end of file
+
+ ## t/t4018/javascript-assignment-of-arrow-function-2 (new) ##
+@@
++const RIGHT = (a, b)=>{
++
++ return a + b; // ChangeMe
++};
+
+ ## t/t4018/javascript-assignment-of-arrow-function-3 (new) ##
+@@
++const RIGHT=test=>{
++
++ return test + 1; // ChangeMe
++};
## t/t4018/javascript-assignment-of-named-function (new) ##
@@
@@ t/t4018/javascript-assignment-of-named-function (new)
+
+ return a + b; // ChangeMe
+};
- \ No newline at end of file
## t/t4018/javascript-async-function (new) ##
@@
@@ t/t4018/javascript-async-function (new)
+
+ return a + b; // ChangeMe
+}
- \ No newline at end of file
## t/t4018/javascript-export-async-function (new) ##
@@
@@ t/t4018/javascript-export-async-function (new)
+
+ return a + b; // ChangeMe
+}
- \ No newline at end of file
## t/t4018/javascript-export-function (new) ##
@@
@@ t/t4018/javascript-export-function (new)
+
+ return a + b; // ChangeMe
+}
- \ No newline at end of file
## t/t4018/javascript-exports-anomyous-function (new) ##
@@
@@ t/t4018/javascript-exports-anomyous-function (new)
+
+ return ChangeMe;
+};
- \ No newline at end of file
## t/t4018/javascript-exports-anomyous-function-2 (new) ##
@@
@@ t/t4018/javascript-exports-anomyous-function-2 (new)
+
+ return ChangeMe;
+};
- \ No newline at end of file
## t/t4018/javascript-exports-function (new) ##
@@
+exports.RIGHT = function(document) {
+
-+ return ChangeMe
++ return ChangeMe;
+}
- \ No newline at end of file
## t/t4018/javascript-function (new) ##
@@
@@ t/t4018/javascript-function (new)
+
+ return a + b; // ChangeMe
+}
- \ No newline at end of file
## t/t4018/javascript-function-2 (new) ##
@@
@@ t/t4018/javascript-function-2 (new)
+ }
+ }
+}
- \ No newline at end of file
## t/t4018/javascript-function-belong-to-IIFE (new) ##
@@
@@ t/t4018/javascript-function-belong-to-IIFE (new)
+ return ChangeMe;
+ };
+}).call(aaaa.prototype);
- \ No newline at end of file
## t/t4018/javascript-function-in-class (new) ##
@@
@@ t/t4018/javascript-function-in-class (new)
+ let b = ChangeMe;
+ }
+}
- \ No newline at end of file
## t/t4018/javascript-function-in-class-2 (new) ##
@@
@@ t/t4018/javascript-function-in-class-2 (new)
+ let b = ChangeMe;
+ }
+}
- \ No newline at end of file
+
+ ## t/t4018/javascript-function-in-class-3 (new) ##
+@@
++class Test {
++ RIGHT(aaaaaaaaaa,
++ bbbbbbbbbb,
++ cccccccccc,
++ dddddddddd
++ ) {
++ let a = 4;
++ let b = ChangeMe;
++ }
++}
## t/t4018/javascript-function-in-object-literal (new) ##
@@
@@ t/t4018/javascript-function-in-object-literal (new)
+ return ChangeMe
+ }
+}
- \ No newline at end of file
## t/t4018/javascript-generator-function (new) ##
@@
@@ t/t4018/javascript-generator-function (new)
+
+ return a + b; // ChangeMe
+}
- \ No newline at end of file
## t/t4018/javascript-generator-function-2 (new) ##
@@
@@ t/t4018/javascript-generator-function-2 (new)
+
+ return a + b; // ChangeMe
+}
- \ No newline at end of file
## t/t4018/javascript-getter-function-in-class (new) ##
@@
@@ t/t4018/javascript-getter-function-in-class (new)
+ let b = ChangeMe;
+ }
+}
- \ No newline at end of file
## t/t4018/javascript-setter-function-in-class (new) ##
@@
@@ t/t4018/javascript-setter-function-in-class (new)
+ let b = ChangeMe;
+ }
+}
- \ No newline at end of file
## t/t4018/javascript-skip-function-call-statement (new) ##
@@
@@ t/t4018/javascript-skip-function-call-statement (new)
+ let b = ChangeMe;
+ }
+}
- \ No newline at end of file
## t/t4018/javascript-skip-keywords (new) ##
@@
@@ t/t4018/javascript-skip-keywords (new)
+
+ return a + b; // ChangeMe
+}
- \ No newline at end of file
## t/t4018/javascript-static-function-in-class (new) ##
@@
@@ t/t4018/javascript-static-function-in-class (new)
+ let b = ChangeMe;
+ }
+}
- \ No newline at end of file
## t/t4034-diff-words.sh ##
@@ t/t4034-diff-words.sh: test_language_driver dts
@@ t/t4034-diff-words.sh: test_language_driver dts
## t/t4034/javascript/expect (new) ##
@@
+<BOLD>diff --git a/pre b/post<RESET>
-+<BOLD>index b72238f..8bc3e3a 100644<RESET>
++<BOLD>index 18f4796..46f9b62 100644<RESET>
+<BOLD>--- a/pre<RESET>
+<BOLD>+++ b/post<RESET>
-+<CYAN>@@ -1,32 +1,32 @@<RESET>
++<CYAN>@@ -1,33 +1,33 @@<RESET>
+// DecimalLiteral<RESET>
+<RED>123<RESET>
+<RED>0.123<RESET>
++<RED>.123<RESET>
+<RED>0.123e+5<RESET>
+<RED>0.123E+5<RESET>
+<RED>0.123e5<RESET>
+<RED>1222222222222222223334444n<RESET><GREEN>124<RESET>
+<GREEN>0.124<RESET>
++<GREEN>.124<RESET>
+<GREEN>0.123e-5<RESET>
+<GREEN>0.123E-5<RESET>
+<GREEN>0.123E5<RESET>
@@ t/t4034/javascript/post (new)
+// DecimalLiteral
+124
+0.124
++.124
+0.123e-5
+0.123E-5
+0.123E5
@@ t/t4034/javascript/post (new)
+a-=b a%=b a&&=b a|=b
+b-c a++ a<<b a>>>=b a<<=b
+a&b a|b a??=b
- \ No newline at end of file
## t/t4034/javascript/pre (new) ##
@@
+// DecimalLiteral
+123
+0.123
++.123
+0.123e+5
+0.123E+5
+0.123e5
@@ t/t4034/javascript/pre (new)
+a+=b a*=b a**=b a||=b
+b+c a-- a>>b a>>>b a>>=b
+a&&b a||b a&&=b
- \ No newline at end of file
## userdiff.c ##
@@ userdiff.c: PATTERNS("java",
@@ userdiff.c: PATTERNS("java",
+ /* don't match the expression may contain parenthesis, because it is not a function declaration */
+ "!^[ \t]*(if|do|while|for|with|switch|catch|import|return)\n"
+ /* don't match statement */
-+ "!^.*;[ \t]*\n"
++ "!;\n"
+ /* match normal function */
-+ "^[\t ]*((export[\t ]+)?(async[\t ]+)?function[\t ]*([\t ]*\\*[\t ]*|[\t ]*)?[$_[:alpha:]][$_[:alnum:]]*[\t ]*\\(.*)\n"
++ "^((export[\t ]+)?(async[\t ]+)?function[\t ]*[\t *]*[$_[:alpha:]][$_[:alnum:]]*[\t ]*\\(.*)\n"
+ /* match JavaScript variable declaration with a lambda expression */
+ "^[\t ]*((const|let|var)[\t ]*[$_[:alpha:]][$_[:alnum:]]*[\t ]*=[\t ]*"
-+ "(\\(.*\\)|[$_[:alpha:]][$_[:alnum:]]*[\t ])[\t ]*=>[\t ]*\\{?)\n"
++ "(\\(.*\\)|[$_[:alpha:]][$_[:alnum:]]*)[\t ]*=>[\t ]*\\{?)\n"
+ /* match exports for anonymous fucntion */
-+ "^[\t ]*(exports\\.[$_[:alpha:]][$_[:alnum:]]*[\t ]*=[\t ]*(\\(.*\\)|[$_[:alpha:]][$_[:alnum:]]*)[\t ]*=>.*)\n"
++ "^(exports\\.[$_[:alpha:]][$_[:alnum:]]*[\t ]*=[\t ]*(\\(.*\\)|[$_[:alpha:]][$_[:alnum:]]*)[\t ]*=>.*)\n"
+ /* match assign function to LHS */
+ "^(.*=[\t ]*function[\t ]*([$_[:alpha:]][$_[:alnum:]]*)?[\t ]*\\(.*)\n"
+ /* match normal function in object literal */
@@ userdiff.c: PATTERNS("java",
+ /* match function in class */
+ "^[\t ]*((static[\t ]+)?((async|get|set)[\t ]+)?[$_[:alpha:]][$_[:alnum:]]*[\t ]*\\(.*)",
+ /* word regex */
-+ /* hexIntegerLiteral and bigHexIntegerLiteral*/
-+ "0[xX][0-9a-fA-F][_0-9a-fA-F]*n?|"
-+ /* octalIntegerLiteral and bigOctalIntegerLiteral */
-+ "0[oO]?[0-7][_0-7]*n?|"
-+ /* binaryIntegerLiteral and bigBinaryIntegerLiteral */
-+ "(0[bB][01][_01]*n?)|"
-+ /* decimalLiteral */
-+ "(0|[1-9][_0-9]*)?\\.?[0-9][_0-9]*([eE][+-]?[_0-9]+)?|"
-+ /* bigDecimalLiteral */
-+ "(0|[1-9][_0-9]*)n|"
++ /* hexIntegerLiteral, octalIntegerLiteral, binaryIntegerLiteral, DecimalLiteral and its big version */
++ "(0[xXoObB])?[0-9a-fA-F][_0-9a-fA-F]*n?"
++ /* DecimalLiteral may be float */
++ "|(0|[1-9][_0-9]*)?\\.?[0-9][_0-9]*([eE][+-]?[_0-9]+)?"
+ /* punctuations */
-+ "\\{|\\}|\\(|\\)|\\.|\\.{3}|;|,|<|>|<=|>=|==|!=|={3}|!==|\\+|-|\\*|/|%|\\*{2}|"
-+ "\\+{2}|--|<<|>>|>>>|&|\\||\\^|!|~|&&|\\|{2}|\\?{1,2}|:|=|\\+=|-=|\\*=|%=|\\*{2}=|"
-+ "<<=|>>=|>>>=|&=|\\|=|\\^=|&&=|\\|{2}=|\\?{2}=|=>|"
++ "|\\.{3}|<=|>=|==|!=|={3}|!==|\\*{2}|\\+{2}|--|<<|>>"
++ "|>>>|&&|\\|{2}|\\?{2}|\\+=|-=|\\*=|%=|\\*{2}="
++ "|<<=|>>=|>>>=|&=|\\|=|\\^=|&&=|\\|{2}=|\\?{2}=|=>"
+ /* identifiers */
-+ "[$_[:alpha:]][$_[:alnum:]]*"),
++ "|[$_[:alpha:]][$_[:alnum:]]*"),
PATTERNS("markdown",
"^ {0,3}#{1,6}[ \t].*",
/* -- */Documentation/gitattributes.txt | 2 + ...avascript-assignment-of-anonymous-function | 4 ++ .../javascript-assignment-of-arrow-function | 4 ++ .../javascript-assignment-of-arrow-function-2 | 4 ++ .../javascript-assignment-of-arrow-function-3 | 4 ++ .../javascript-assignment-of-named-function | 4 ++ t/t4018/javascript-async-function | 4 ++ t/t4018/javascript-export-async-function | 4 ++ t/t4018/javascript-export-function | 4 ++ t/t4018/javascript-exports-anomyous-function | 4 ++ .../javascript-exports-anomyous-function-2 | 4 ++ t/t4018/javascript-exports-function | 4 ++ t/t4018/javascript-function | 4 ++ t/t4018/javascript-function-2 | 10 ++++ t/t4018/javascript-function-belong-to-IIFE | 6 +++ t/t4018/javascript-function-in-class | 6 +++ t/t4018/javascript-function-in-class-2 | 11 ++++ t/t4018/javascript-function-in-class-3 | 10 ++++ t/t4018/javascript-function-in-object-literal | 7 +++ t/t4018/javascript-generator-function | 4 ++ t/t4018/javascript-generator-function-2 | 4 ++ t/t4018/javascript-getter-function-in-class | 6 +++ t/t4018/javascript-setter-function-in-class | 6 +++ .../javascript-skip-function-call-statement | 7 +++ t/t4018/javascript-skip-keywords | 34 ++++++++++++ t/t4018/javascript-static-function-in-class | 6 +++ t/t4034-diff-words.sh | 1 + t/t4034/javascript/expect | 54 +++++++++++++++++++ t/t4034/javascript/post | 33 ++++++++++++ t/t4034/javascript/pre | 33 ++++++++++++ userdiff.c | 32 +++++++++++ 31 files changed, 320 insertions(+) create mode 100644 t/t4018/javascript-assignment-of-anonymous-function create mode 100644 t/t4018/javascript-assignment-of-arrow-function create mode 100644 t/t4018/javascript-assignment-of-arrow-function-2 create mode 100644 t/t4018/javascript-assignment-of-arrow-function-3 create mode 100644 t/t4018/javascript-assignment-of-named-function create mode 100644 t/t4018/javascript-async-function create mode 100644 t/t4018/javascript-export-async-function create mode 100644 t/t4018/javascript-export-function create mode 100644 t/t4018/javascript-exports-anomyous-function create mode 100644 t/t4018/javascript-exports-anomyous-function-2 create mode 100644 t/t4018/javascript-exports-function create mode 100644 t/t4018/javascript-function create mode 100644 t/t4018/javascript-function-2 create mode 100644 t/t4018/javascript-function-belong-to-IIFE create mode 100644 t/t4018/javascript-function-in-class create mode 100644 t/t4018/javascript-function-in-class-2 create mode 100644 t/t4018/javascript-function-in-class-3 create mode 100644 t/t4018/javascript-function-in-object-literal create mode 100644 t/t4018/javascript-generator-function create mode 100644 t/t4018/javascript-generator-function-2 create mode 100644 t/t4018/javascript-getter-function-in-class create mode 100644 t/t4018/javascript-setter-function-in-class create mode 100644 t/t4018/javascript-skip-function-call-statement create mode 100644 t/t4018/javascript-skip-keywords create mode 100644 t/t4018/javascript-static-function-in-class create mode 100644 t/t4034/javascript/expect create mode 100644 t/t4034/javascript/post create mode 100644 t/t4034/javascript/pre
Documentation/gitattributes.txt +2 -0
@@ -828,6 +828,8 @@ patterns are available: - `java` suitable for source code in the Java language. +- `javascript` suitable for source code in the JavaScript language. + - `markdown` suitable for Markdown documents. - `matlab` suitable for source code in the MATLAB and Octave languages.
t/t4018/javascript-assignment-of-anonymous-function new file +4 -0
@@ -0,0 +1,4 @@ +const RIGHT = function (a, b) { + + return a + b; // ChangeMe +};
t/t4018/javascript-assignment-of-arrow-function new file +4 -0
@@ -0,0 +1,4 @@ +const RIGHT = (a, b) => { + + return a + b; // ChangeMe +};
t/t4018/javascript-assignment-of-arrow-function-2 new file +4 -0
@@ -0,0 +1,4 @@ +const RIGHT = (a, b)=>{ + + return a + b; // ChangeMe +};
t/t4018/javascript-assignment-of-arrow-function-3 new file +4 -0
@@ -0,0 +1,4 @@ +const RIGHT=test=>{ + + return test + 1; // ChangeMe +};
t/t4018/javascript-assignment-of-named-function new file +4 -0
@@ -0,0 +1,4 @@ +const RIGHT = function test (a, b) { + + return a + b; // ChangeMe +};
t/t4018/javascript-async-function new file +4 -0
@@ -0,0 +1,4 @@ +async function RIGHT(a, b) { + + return a + b; // ChangeMe +}
t/t4018/javascript-export-async-function new file +4 -0
@@ -0,0 +1,4 @@ +export async function RIGHT(a, b) { + + return a + b; // ChangeMe +}
t/t4018/javascript-export-function new file +4 -0
@@ -0,0 +1,4 @@ +export function RIGHT(a, b) { + + return a + b; // ChangeMe +}
t/t4018/javascript-exports-anomyous-function new file +4 -0
@@ -0,0 +1,4 @@ +exports.setFlagged = RIGHT => { + + return ChangeMe; +};
t/t4018/javascript-exports-anomyous-function-2 new file +4 -0
@@ -0,0 +1,4 @@ +exports.RIGHT = (a, b, runtime) => { + + return ChangeMe; +};
t/t4018/javascript-exports-function new file +4 -0
@@ -0,0 +1,4 @@ +exports.RIGHT = function(document) { + + return ChangeMe; +}
t/t4018/javascript-function new file +4 -0
@@ -0,0 +1,4 @@ +function RIGHT(a, b) { + + return a + b; // ChangeMe +}
t/t4018/javascript-function-2 new file +10 -0
@@ -0,0 +1,10 @@ +function test(a, b) { + return { + RIGHT: function () { + currentUpdateRemovedChunks.forEach(function (chunkId) { + delete $installedChunks$[chunkId]; + }); + currentUpdateRemovedChunks = ChangeMe; + } + } +}
t/t4018/javascript-function-belong-to-IIFE new file +6 -0
@@ -0,0 +1,6 @@ +(function () { + this.$RIGHT = function (needle, modifier) { + let a = 5; + return ChangeMe; + }; +}).call(aaaa.prototype);
t/t4018/javascript-function-in-class new file +6 -0
@@ -0,0 +1,6 @@ +class Test { + RIGHT() { + let a = 4; + let b = ChangeMe; + } +}
t/t4018/javascript-function-in-class-2 new file +11 -0
@@ -0,0 +1,11 @@ +class Test { + RIGHT( + aaaaaaaaaa, + bbbbbbbbbb, + cccccccccc, + dddddddddd + ) { + let a = 4; + let b = ChangeMe; + } +}
t/t4018/javascript-function-in-class-3 new file +10 -0
@@ -0,0 +1,10 @@ +class Test { + RIGHT(aaaaaaaaaa, + bbbbbbbbbb, + cccccccccc, + dddddddddd + ) { + let a = 4; + let b = ChangeMe; + } +}
t/t4018/javascript-function-in-object-literal new file +7 -0
@@ -0,0 +1,7 @@ +const obj = { + RIGHT: function (elems, callback, arg) { + var length, value; + // ... + return ChangeMe + } +}
t/t4018/javascript-generator-function new file +4 -0
@@ -0,0 +1,4 @@ +function* RIGHT(a, b) { + + return a + b; // ChangeMe +}
t/t4018/javascript-generator-function-2 new file +4 -0
@@ -0,0 +1,4 @@ +function *RIGHT(a, b) { + + return a + b; // ChangeMe +}
t/t4018/javascript-getter-function-in-class new file +6 -0
@@ -0,0 +1,6 @@ +class Test { + get RIGHT() { + let a = 4; + let b = ChangeMe; + } +}
t/t4018/javascript-setter-function-in-class new file +6 -0
@@ -0,0 +1,6 @@ +class Test { + set RIGHT() { + let a = 4; + let b = ChangeMe; + } +}
t/t4018/javascript-skip-function-call-statement new file +7 -0
@@ -0,0 +1,7 @@ +class Test { + static RIGHT() { + haha(); + haha2() + let b = ChangeMe; + } +}
t/t4018/javascript-skip-keywords new file +34 -0
@@ -0,0 +1,34 @@ +function RIGHT(a, b) { + import("./async1") + if (a > 1) { + // ... + } + do { + // ... + } while (i < 5); + for (const element of array1) { + console.log(element) + } + with(o) { + console.log(x) + } + switch (expr) { + case 'a': + // ... + break; + case 'b': + // ... + break; + default: + // ... + } + try { + // ... + return (a + c) + } + catch (error) { + // ... + } + + return a + b; // ChangeMe +}
t/t4018/javascript-static-function-in-class new file +6 -0
@@ -0,0 +1,6 @@ +class Test { + static RIGHT() { + let a = 4; + let b = ChangeMe; + } +}
t/t4034-diff-words.sh +1 -0
@@ -324,6 +324,7 @@ test_language_driver dts test_language_driver fortran test_language_driver html test_language_driver java +test_language_driver javascript test_language_driver matlab test_language_driver objc test_language_driver pascal
t/t4034/javascript/expect new file +54 -0
54 lines not on this page · raw message
t/t4034/javascript/post new file +33 -0
@@ -0,0 +1,33 @@ +// DecimalLiteral +124 +0.124 +.124 +0.123e-5 +0.123E-5 +0.123E5 +12222222222222222233344445n +// HexIntegerLiteral +0x11 +0X5Fa1 +0x123_756 +0x1234182989812f1289bn +// OctalIntegerLiteral +06 +0o5 +0O4 +0511_567 +0o424242424242424242424242424242666667n +// BinaryIntegerLiteral +0b1101 +0B0010 +0b0001_1101_0011 +0b11111111111111000011111111111111111n +// punctuations +{b} (b) +b; +[2,3] +[ ...params_v2 ] +a=2 a=2 a=2 a=2 a=2 a=2 a=2 a=2 +a-=b a%=b a&&=b a|=b +b-c a++ a<<b a>>>=b a<<=b +a&b a|b a??=b
t/t4034/javascript/pre new file +33 -0
@@ -0,0 +1,33 @@ +// DecimalLiteral +123 +0.123 +.123 +0.123e+5 +0.123E+5 +0.123e5 +1222222222222222223334444n +// HexIntegerLiteral +0x10 +0X6Fa1 +0x123_456 +0x1234182989812f1289an +// OctalIntegerLiteral +05 +0o6 +0O7 +0512_567 +0o424242424242424242424242424242666666n +// BinaryIntegerLiteral +0b1001 +0B0110 +0b0001_1001_0011 +0b1111111111111111111111111111111111111n +// punctuations +{a} (a) +a; +[1,2] +[ 1, 2, ...params ] +a<=2 a>=2 a==2 a!=2 a===2 a!==2 a^=2 a=>2 +a+=b a*=b a**=b a||=b +b+c a-- a>>b a>>>b a>>=b +a&&b a||b a&&=b
userdiff.c +32 -0
@@ -168,6 +168,38 @@ PATTERNS("java","|[-+0-9.e]+[fFlL]?|0[xXbB]?[0-9a-fA-F]+[lL]?" "|[-+*/<>%&^|=!]=" "|--|\\+\\+|<<=?|>>>?=?|&&|\\|\\|"), + +PATTERNS ("javascript" , +/* don't match the expression may contain parenthesis, because it is not a function declaration */ +"!^[ \t]*(if|do|while|for|with|switch|catch|import|return)\n" +/* don't match statement */ +"!;\n" +/* match normal function */ +"^((export[\t ]+)?(async[\t ]+)?function[\t ]*[\t *]*[$_[:alpha:]][$_[:alnum:]]*[\t ]*\\(.*)\n" +/* match JavaScript variable declaration with a lambda expression */ +"^[\t ]*((const|let|var)[\t ]*[$_[:alpha:]][$_[:alnum:]]*[\t ]*=[\t ]*" +"(\\(.*\\)|[$_[:alpha:]][$_[:alnum:]]*)[\t ]*=>[\t ]*\\{?)\n" +/* match exports for anonymous fucntion */ +"^(exports\\.[$_[:alpha:]][$_[:alnum:]]*[\t ]*=[\t ]*(\\(.*\\)|[$_[:alpha:]][$_[:alnum:]]*)[\t ]*=>.*)\n" +/* match assign function to LHS */ +"^(.*=[\t ]*function[\t ]*([$_[:alpha:]][$_[:alnum:]]*)?[\t ]*\\(.*)\n" +/* match normal function in object literal */ +"^[\t ]*([$_[:alpha:]][$_[:alnum:]]*[\t ]*:[\t ]*function[\t ].*)\n" +/* don't match the function in class, which has more than one ident level */ +"!^(\t{2,}|[ ]{5,})\n" +/* match function in class */ +"^[\t ]*((static[\t ]+)?((async|get|set)[\t ]+)?[$_[:alpha:]][$_[:alnum:]]*[\t ]*\\(.*)" , +/* word regex */ +/* hexIntegerLiteral, octalIntegerLiteral, binaryIntegerLiteral, DecimalLiteral and its big version */ +"(0[xXoObB])?[0-9a-fA-F][_0-9a-fA-F]*n?" +/* DecimalLiteral may be float */ +"|(0|[1-9][_0-9]*)?\\.?[0-9][_0-9]*([eE][+-]?[_0-9]+)?" +/* punctuations */ +"|\\.{3}|<=|>=|==|!=|={3}|!==|\\*{2}|\\+{2}|--|<<|>>" +"|>>>|&&|\\|{2}|\\?{2}|\\+=|-=|\\*=|%=|\\*{2}=" +"|<<=|>>=|>>>=|&=|\\|=|\\^=|&&=|\\|{2}=|\\?{2}=|=>" +/* identifiers */ +"|[$_[:alpha:]][$_[:alnum:]]*" ), PATTERNS("markdown", "^ {0,3}#{1,6}[ \t].*",/* -- */
Signature · 1 line
2.35.1.273.ge6ebfd0e8c.dirty
Re: [GSoC][PATCH v2] Add a diff driver for JavaScript languages.
When you send a new iteration of a patch or patch set, it is customary on this list to include everyone who took part in the earlier rounds in the Cc: list.
Am 12.03.22 um 17:48 schrieb xing zhi jiang:
Quoted 23 lines
In the xfunction part that matches normal functions,
a variable declaration with an assignment of function, the function declaration in the class, and also the function is object literal's property[1].And in the word regex part, that matches numbers, punctuations, and also the JavaScript identifier.
This part reference the formal ECMA specification[2].[1]https://github.com/jquery/jquery/blob/de5398a6ad088dc006b46c6a870a2a053f4cd663/src/core.js#L201 [2]https://262.ecma-international.org/12.0/#sec-ecmascript-language-lexical-grammar
Signed-off-by xing zhi jiang <[email protected]>---
userdiff.c +1
-0@@ -168,6 +168,38 @@ PATTERNS("java","|[-+0-9.e]+[fFlL]?|0[xXbB]?[0-9a-fA-F]+[lL]?" "|[-+*/<>%&^|=!]=" "|--|\\+\\+|<<=?|>>>?=?|&&|\\|\\|"), +
userdiff.c PATTERNS("java", · line 181
Quoted 10 lines
userdiff.c +10
-0+PATTERNS ("javascript" , +/* don't match the expression may contain parenthesis, because it is not a function declaration */ +"!^[ \t]*(if|do|while|for|with|switch|catch|import|return)\n" +/* don't match statement */ +"!;\n" +/* match normal function */ +"^((export[\t ]+)?(async[\t ]+)?function[\t ]*[\t *]*[$_[:alpha:]][$_[:alnum:]]*[\t ]*\\(.*)\n" +/* match JavaScript variable declaration with a lambda expression */ +"^[\t ]*((const|let|var)[\t ]*[$_[:alpha:]][$_[:alnum:]]*[\t ]*=[\t ]*" + "(\\(.*\\)|[$_[:alpha:]][$_[:alnum:]]*)[\t ]*=>[\t ]*\\{?)\n"
It would help readability if this second line of this regex were indented because it is a continuation of the first line.
userdiff.c PATTERNS("java", · line 185
Quoted 4 lines
userdiff.c +4
-0+/* match exports for anonymous fucntion */ +"^(exports\\.[$_[:alpha:]][$_[:alnum:]]*[\t ]*=[\t ]*(\\(.*\\)|[$_[:alpha:]][$_[:alnum:]]*)[\t ]*=>.*)\n" +/* match assign function to LHS */ +"^(.*=[\t ]*function[\t ]*([$_[:alpha:]][$_[:alnum:]]*)?[\t ]*\\(.*)\n"
This should be written as
"^(.*=[\t ]*function[\t ]*([$_[:alpha:]][$_[:alnum:]]*[\t ]*)?\\(.*)\n"
Notice that the whitespace after the identifier can only appear when there is actually an identifier. The point is to reduce the different matches permitted by the sub-expression "[\t ]*[\t ]*" when there is no identifier in the text.
Can the keyword function ever be followed by a number? I guess not. Then [$_[:alpha:]][$_[:alnum:]]* could be reduced to [$_[:alnum:]]+
userdiff.c PATTERNS("java", · line 196
Quoted 10 lines
userdiff.c +10
-0+/* match normal function in object literal */ +"^[\t ]*([$_[:alpha:]][$_[:alnum:]]*[\t ]*:[\t ]*function[\t ].*)\n" +/* don't match the function in class, which has more than one ident level */ +"!^(\t{2,}|[ ]{5,})\n" +/* match function in class */ + "^[\t ]*((static[\t ]+)?((async|get|set)[\t ]+)?[$_[:alpha:]][$_[:alnum:]]*[\t ]*\\(.*)",> + /* word regex */ +/* hexIntegerLiteral, octalIntegerLiteral, binaryIntegerLiteral, DecimalLiteral and its big version */ +"(0[xXoObB])?[0-9a-fA-F][_0-9a-fA-F]*n?" +/* DecimalLiteral may be float */ +"|(0|[1-9][_0-9]*)?\\.?[0-9][_0-9]*([eE][+-]?[_0-9]+)?"
Having alternatives that begin with an optional part make the regex evaluation comparatively inefficient. In particular, both alternatives above match a decimal integer. I suggest to have the first alternative only for hex, octal, and binary integers, and the second for all decimal numbers including floatingpoint:
/* hexIntegerLiteral, octalIntegerLiteral, binaryIntegerLiteral, and their big versions */ "0[xXoObB][_0-9a-fA-F]+n?" /* DecimalLiteral may be float */ "|[0-9][_0-9]*(\\.[_0-9]*|n)?([eE][+-]?[_0-9]+)?"
and if floating point literals can begin with a decimal point, then we also need
"|\\.[0-9][_0-9]*([eE][+-]?[_0-9]+)?"
userdiff.c PATTERNS("java", · line 205
Quoted 9 lines
userdiff.c +6
-0+/* punctuations */ +"|\\.{3}|<=|>=|==|!=|={3}|!==|\\*{2}|\\+{2}|--|<<|>>" +"|>>>|&&|\\|{2}|\\?{2}|\\+=|-=|\\*=|%=|\\*{2}=" +"|<<=|>>=|>>>=|&=|\\|=|\\^=|&&=|\\|{2}=|\\?{2}=|=>" +/* identifiers */ + "|[$_[:alpha:]][$_[:alnum:]]*"), PATTERNS("markdown", "^ {0,3}#{1,6}[ \t].*",/* -- */
-- Hannes
Re: [GSoC][PATCH v2] Add a diff driver for JavaScript languages.
Sorry for the late reply. I've been busy the last two weeks.
Quoted 16 lines · Johannes Sixt
Am 12.03.22 um 17:48 schrieb xing zhi jiang:
Quoted 14 lines
userdiff.c +6
-0@@ -168,6 +168,38 @@ PATTERNS("java","|[-+0-9.e]+[fFlL]?|0[xXbB]?[0-9a-fA-F]+[lL]?" "|[-+*/<>%&^|=!]=" "|--|\\+\\+|<<=?|>>>?=?|&&|\\|\\|"), + +PATTERNS ("javascript" , +/* don't match the expression may contain parenthesis, because it is not a function declaration */ +"!^[ \t]*(if|do|while|for|with|switch|catch|import|return)\n" +/* don't match statement */ + "!;\n"
Quoted 9 lines
userdiff.c PATTERNS("java", · line 181
Quoted 5 lines
userdiff.c +5
-0+/* match normal function */ +"^((export[\t ]+)?(async[\t ]+)?function[\t ]*[\t *]*[$_[:alpha:]][$_[:alnum:]]*[\t ]*\\(.*)\n" +/* match JavaScript variable declaration with a lambda expression */ +"^[\t ]*((const|let|var)[\t ]*[$_[:alpha:]][$_[:alnum:]]*[\t ]*=[\t ]*" +"(\\(.*\\)|[$_[:alpha:]][$_[:alnum:]]*)[\t ]*=>[\t ]*\\{?)\n" It would help readability if this second line of this regex were indented because it is a continuation of the first line.
This will be fixed in the v3 patch.
Quoted 9 lines
userdiff.c PATTERNS("java", · line 185
Quoted 4 lines
userdiff.c +4
-0+/* match exports for anonymous fucntion */ +"^(exports\\.[$_[:alpha:]][$_[:alnum:]]*[\t ]*=[\t ]*(\\(.*\\)|[$_[:alpha:]][$_[:alnum:]]*)[\t ]*=>.*)\n" +/* match assign function to LHS */ +"^(.*=[\t ]*function[\t ]*([$_[:alpha:]][$_[:alnum:]]*)?[\t ]*\\(.*)\n" This should be written as
"^(.*=[\t ]*function[\t ]*([$_[:alpha:]][$_[:alnum:]]*[\t ]*)?\\(.*)\n"
Quoted 7 lines
Notice that the whitespace after the identifier can only appear when there is actually an identifier. The point is to reduce the different matches permitted by the sub-expression "[\t ]*[\t ]*" when there is no identifier in the text.
Can the keyword function ever be followed by a number? I guess not. Then [$_[:alpha:]][$_[:alnum:]]* could be reduced to [$_[:alnum:]]+
This will be fixed in the v3 patch.
Quoted 23 lines
userdiff.c PATTERNS("java", · line 196
Quoted 10 lines
userdiff.c +10
-0+/* match normal function in object literal */ +"^[\t ]*([$_[:alpha:]][$_[:alnum:]]*[\t ]*:[\t ]*function[\t ].*)\n" +/* don't match the function in class, which has more than one ident level */ +"!^(\t{2,}|[ ]{5,})\n" +/* match function in class */ + "^[\t ]*((static[\t ]+)?((async|get|set)[\t ]+)?[$_[:alpha:]][$_[:alnum:]]*[\t ]*\\(.*)",> + /* word regex */ +/* hexIntegerLiteral, octalIntegerLiteral, binaryIntegerLiteral, DecimalLiteral and its big version */ +"(0[xXoObB])?[0-9a-fA-F][_0-9a-fA-F]*n?" +/* DecimalLiteral may be float */ +"|(0|[1-9][_0-9]*)?\\.?[0-9][_0-9]*([eE][+-]?[_0-9]+)?" Having alternatives that begin with an optional part make the regex evaluation comparatively inefficient. In particular, both alternatives above match a decimal integer. I suggest to have the first alternative only for hex, octal, and binary integers, and the second for all decimal numbers including floatingpoint:
/* hexIntegerLiteral, octalIntegerLiteral, binaryIntegerLiteral, and their big versions */ "0[xXoObB][_0-9a-fA-F]+n?" /* DecimalLiteral may be float */ "|[0-9][_0-9]*(\\.[_0-9]*|n)?([eE][+-]?[_0-9]+)?"
Quoted 5 lines
and if floating point literals can begin with a decimal point, then we also need
"|\\.[0-9][_0-9]*([eE][+-]?[_0-9]+)?"
I agree on separate decimal regex from others. And in JavaScript floating numbers can start with a dot. So need add new regex - "|\\.[0-9][_0-9]*([eE][+-]?[_0-9]+)?".
This will be fixed in the v3 patch.
Re: [GSoC][PATCH v2] Add a diff driver for JavaScript languages.
In reply to xing zhi jiangWelcome to the Git project :) I never realized that we didn't include a built in JS diff driver, so this would be quite welcome.
I'm not entirely familiar with this part of the system or our regex style, so I won't comment on those. I'll only comment on the patterns we are trying to detect and whether or not we want them.
I'm not 100% clear on the style around diff drivers e.g. how do we decide that we want a pattern or not? I'd appreciate any pointers to docs or commits.
Quoted 36 lines · xing zhi jiang
t/t4018/javascript-assignment-of-anonymous-function new file +4
-0@@ -0,0 +1,4 @@ +const RIGHT = function (a, b) { + + return a + b; // ChangeMe +};t/t4018/javascript-assignment-of-arrow-function new file +4
-0@@ -0,0 +1,4 @@ +const RIGHT = (a, b) => { + + return a + b; // ChangeMe +};t/t4018/javascript-assignment-of-arrow-function-2 new file +4
-0@@ -0,0 +1,4 @@ +const RIGHT = (a, b)=>{ + + return a + b; // ChangeMe +};t/t4018/javascript-assignment-of-arrow-function-3 new file +0
-0
t/t4018/javascript-assignment-of-arrow-function-3 line 4
Quoted 4 lines
t/t4018/javascript-assignment-of-arrow-function-3 new file +4
-0@@ -0,0 +1,4 @@ +const RIGHT=test=>{ + + return test + 1; // ChangeMe +};
These are all variable assignments of anonymous functions, so we won't 'technically' be showing the function name in the diff, but as a practical matter, they are _probably_ referred to by this name consistently. So including them makes sense.
Quoted 36 lines
t/t4018/javascript-assignment-of-named-function new file +4
-0@@ -0,0 +1,4 @@ +const RIGHT = function test (a, b) { + + return a + b; // ChangeMe +};t/t4018/javascript-async-function new file +4
-0@@ -0,0 +1,4 @@ +async function RIGHT(a, b) { + + return a + b; // ChangeMe +}t/t4018/javascript-export-async-function new file +4
-0@@ -0,0 +1,4 @@ +export async function RIGHT(a, b) { + + return a + b; // ChangeMe +}t/t4018/javascript-export-function new file +0
-0
t/t4018/javascript-export-function line 4
Quoted 4 lines
t/t4018/javascript-export-function new file +4
-0@@ -0,0 +1,4 @@ +export function RIGHT(a, b) { + + return a + b; // ChangeMe +}
These look good; the 'export' statements are part of the ES modules feature. I'm not sure if it makes sense to explicitly test these cases unless we have reason to believe that the 'export' keyword will affect the matching.
Quoted 26 lines
t/t4018/javascript-exports-anomyous-function new file +4
-0@@ -0,0 +1,4 @@ +exports.setFlagged = RIGHT => { + + return ChangeMe; +};t/t4018/javascript-exports-anomyous-function-2 new file +4
-0@@ -0,0 +1,4 @@ +exports.RIGHT = (a, b, runtime) => { + + return ChangeMe; +};t/t4018/javascript-exports-function new file +0
-0
t/t4018/javascript-exports-function line 4
Quoted 4 lines
t/t4018/javascript-exports-function new file +4
-0@@ -0,0 +1,4 @@ +exports.RIGHT = function(document) { + + return ChangeMe; +}
I don't think we should include 'exports.foo = bar'. To my knowledge, this is _not_ a standard ES feature, but rather the CommonJS module system popularized by Node.js [1] and other frameworks. To confirm this, I searched the ES spec and did not find any reference to exports.* [2].
Even if we wanted to support nonstandard 'language features' (and this label is tenuous at best, CommonJS is not trying to replace the ES modules standard AFAIK), Node.js is also starting to support ES modules, so I don't think this is a good long term direction for Git.
[1] https://nodejs.org/api/modules.html
[2] https://262.ecma-international.org/12.0/#sec-exports
Quoted 16 lines
t/t4018/javascript-function new file +4
-0@@ -0,0 +1,4 @@ +function RIGHT(a, b) { + + return a + b; // ChangeMe +}t/t4018/javascript-function-2 new file +0
-0
t/t4018/javascript-function-2 line 10
Quoted 10 lines
t/t4018/javascript-function-2 new file +10
-0@@ -0,0 +1,10 @@ +function test(a, b) { + return { + RIGHT: function () { + currentUpdateRemovedChunks.forEach(function (chunkId) { + delete $installedChunks$[chunkId]; + }); + currentUpdateRemovedChunks = ChangeMe; + } + } +}
There is also the ES2015 'method shorthand' syntax [3], e.g. bar in:
const foo = {
bar() {
console.log('hi');
}
}[3] https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Functions/Method_definitions
t/t4018/javascript-function-belong-to-IIFE line 6
Quoted 12 lines
t/t4018/javascript-function-belong-to-IIFE new file +6
-0@@ -0,0 +1,6 @@ +(function () { + this.$RIGHT = function (needle, modifier) { + let a = 5; + return ChangeMe; + }; +}).call(aaaa.prototype);
Does the IIFE matter in this case? This line:
this.$RIGHT = function (needle, modifier) {looks extremely similar to the previous test of foo = function bar().
Or perhaps this is meant to demonstrate the edge case of "matching in a complicated construct"? If so, perhaps we should test other edge cases like:
function WRONG() {
let RIGHT = function (needle, modifier) {
let a = 5;
return ChangeMe;
};
}Quoted 161 lines
t/t4018/javascript-function-in-class new file +6
-0@@ -0,0 +1,6 @@ +class Test { + RIGHT() { + let a = 4; + let b = ChangeMe; + } +}t/t4018/javascript-function-in-class-2 new file +11
-0@@ -0,0 +1,11 @@ +class Test { + RIGHT( + aaaaaaaaaa, + bbbbbbbbbb, + cccccccccc, + dddddddddd + ) { + let a = 4; + let b = ChangeMe; + } +}t/t4018/javascript-function-in-class-3 new file +10
-0@@ -0,0 +1,10 @@ +class Test { + RIGHT(aaaaaaaaaa, + bbbbbbbbbb, + cccccccccc, + dddddddddd + ) { + let a = 4; + let b = ChangeMe; + } +}t/t4018/javascript-function-in-object-literal new file +7
-0@@ -0,0 +1,7 @@ +const obj = { + RIGHT: function (elems, callback, arg) { + var length, value; + // ... + return ChangeMe + } +}t/t4018/javascript-generator-function new file +4
-0@@ -0,0 +1,4 @@ +function* RIGHT(a, b) { + + return a + b; // ChangeMe +}t/t4018/javascript-generator-function-2 new file +4
-0@@ -0,0 +1,4 @@ +function *RIGHT(a, b) { + + return a + b; // ChangeMe +}t/t4018/javascript-getter-function-in-class new file +6
-0@@ -0,0 +1,6 @@ +class Test { + get RIGHT() { + let a = 4; + let b = ChangeMe; + } +}t/t4018/javascript-setter-function-in-class new file +6
-0@@ -0,0 +1,6 @@ +class Test { + set RIGHT() { + let a = 4; + let b = ChangeMe; + } +}t/t4018/javascript-skip-function-call-statement new file +7
-0@@ -0,0 +1,7 @@ +class Test { + static RIGHT() { + haha(); + haha2() + let b = ChangeMe; + } +}t/t4018/javascript-skip-keywords new file +34
-0@@ -0,0 +1,34 @@ +function RIGHT(a, b) { + import("./async1") + if (a > 1) { + // ... + } + do { + // ... + } while (i < 5); + for (const element of array1) { + console.log(element) + } + with(o) { + console.log(x) + } + switch (expr) { + case 'a': + // ... + break; + case 'b': + // ... + break; + default: + // ... + } + try { + // ... + return (a + c) + } + catch (error) { + // ... + } + + return a + b; // ChangeMe +}t/t4018/javascript-static-function-in-class new file +0
-0
t/t4018/javascript-static-function-in-class line 6
Quoted 6 lines
t/t4018/javascript-static-function-in-class new file +6
-0@@ -0,0 +1,6 @@ +class Test { + static RIGHT() { + let a = 4; + let b = ChangeMe; + } +}
The rest of the test cases look good.
Re: [GSoC][PATCH v2] Add a diff driver for JavaScript languages.
Am 14.03.22 um 18:20 schrieb Glen Choo:
Quoted 17 lines
t/t4018/javascript-exports-function line 4
Quoted 10 lines · xing zhi jiang
t/t4018/javascript-exports-function new file +4
-0@@ -0,0 +1,4 @@ +exports.RIGHT = function(document) { + + return ChangeMe; +}I don't think we should include 'exports.foo = bar'. To my knowledge, this is _not_ a standard ES feature, but rather the CommonJS module system popularized by Node.js [1] and other frameworks. To confirm this, I searched the ES spec and did not find any reference to exports.* [2].
Quoted 4 lines
Even if we wanted to support nonstandard 'language features' (and this label is tenuous at best, CommonJS is not trying to replace the ES modules standard AFAIK), Node.js is also starting to support ES modules, so I don't think this is a good long term direction for Git.
It is not a priority to model hunk header regular expressions after some standard and to ignore stuff that is outside the standard. The goal is to make them useful in a majority of cases. If there exists a noticable chunk of code that uses non-standard constructs, then that is worth being supported.
Quoted 3 lines
[1] https://nodejs.org/api/modules.html
[2] https://262.ecma-international.org/12.0/#sec-exports
-- Hannes
Re: [GSoC][PATCH v2] Add a diff driver for JavaScript languages.
Quoted 22 lines · Johannes Sixt
Am 14.03.22 um 18:20 schrieb Glen Choo:
Quoted 16 lines
t/t4018/javascript-exports-function line 3
Quoted 9 lines · xing zhi jiang
t/t4018/javascript-exports-function new file +3
-0@@ -0,0 +1,4 @@ +exports.RIGHT = function(document) { + + return ChangeMe; >>> +}I don't think we should include 'exports.foo = bar'. To my knowledge, this is _not_ a standard ES feature, but rather the CommonJS module system popularized by Node.js [1] and other frameworks. To confirm this, I searched the ES spec and did not find any reference to exports.* [2].
Quoted 4 lines
Even if we wanted to support nonstandard 'language features' (and this label is tenuous at best, CommonJS is not trying to replace the ES modules standard AFAIK), Node.js is also starting to support ES modules, so I don't think this is a good long term direction for Git.
Quoted 5 lines
It is not a priority to model hunk header regular expressions after some standard and to ignore stuff that is outside the standard. The goal is to make them useful in a majority of cases. If there exists a noticable chunk of code that uses non-standard constructs, then that is worth being supported.
Interesting, I'll take note. I'm still personally not keen on supporting CommonJS-only patterns when we are purportedly trying to show diffs for JavaScript, but if we think this fits the style, I'm happy to oblige.
So the question becomes "Is there a significant amount of code that uses this pattern?" Probably - this is a fairly common pattern in Node.js after all. But in my experience,
module.exports.RIGHT = function(document) {
return ChangeMe;}
is even more common. The difference between 'module.exports' and 'exports' isn't worth going into (StackOverflow has all the answers, for the curious), but if we're taking the approach of supporting CommonJS, I'd like to be consistent and also support 'module.exports', i.e. perhaps change:
"^(exports\\.[$_[:alpha:]][$_[:alnum:]]*[\t ]*=[\t ]*(\\(.*\\)|[$_[:alpha:]][$_[:alnum:]]*)[\t ]*=>.*)\n"
to something like:
"^((module.)?exports\\.[$_[:alpha:]][$_[:alnum:]]*[\t ]*=[\t ]*(\\(.*\\)|[$_[:alpha:]][$_[:alnum:]]*)[\t ]*=>.*)\n"
Re: [GSoC][PATCH v2] Add a diff driver for JavaScript languages.
Quoted 3 lines · Glen Choo
Interesting, I'll take note. I'm still personally not keen on supporting CommonJS-only patterns when we are purportedly trying to show diffs for JavaScript, but if we think this fits the style, I'm happy to oblige.
The question is, with these patterns that are aware of CommonJS convention, would your bog-standard-and-boring vanilla JS code be detected incorrectly? Becoming aware of popular conventions without hurting others would be a good thing.
And the "popular conventions" does not have to be limited to CommonJS/Node.
Re: [GSoC][PATCH v2] Add a diff driver for JavaScript languages.
Quoted 6 lines · Junio C Hamano
Quoted 3 lines · Glen Choo
Interesting, I'll take note. I'm still personally not keen on supporting CommonJS-only patterns when we are purportedly trying to show diffs for JavaScript, but if we think this fits the style, I'm happy to oblige.
Quoted 7 lines
The question is, with these patterns that are aware of CommonJS convention, would your bog-standard-and-boring vanilla JS code be detected incorrectly? Becoming aware of popular conventions without hurting others would be a good thing.
And the "popular conventions" does not have to be limited to CommonJS/Node.
From the perspective of "'exports' is a special name", yes, we could detect vanilla JS code 'incorrectly' because, in vanilla JS, the names 'exports' or 'module.exports' are not special. So perhaps, one could imagine a browser-side script that deals with "imports" and "exports" as part of their business:
const exports = {
quantity: 1,
type: 'boxes',
};
exports.getQuantity = () => {
foo();
};This diff driver would mistakenly detect exports.getQuantity = () => {.
Although, the more I think about it, the spirit of this patch seems to be "we want to show headers whenever we think we are in a function", so we don't actually need to treat 'exports' or 'module.exports' specially at all, e.g. this case should also pass our diff driver tests:
const foo = {};
foo.RIGHT = () => {ChangeMe();
};
and if we do this, we will correctly handle 'exports' and 'module.exports' anyway by virtue of them being plain old JS objects.
Re: [GSoC][PATCH v2] Add a diff driver for JavaScript languages.
Quoted 39 lines · Glen Choo
Quoted 6 lines · Junio C Hamano
Quoted 3 lines · Glen Choo
Interesting, I'll take note. I'm still personally not keen on supporting CommonJS-only patterns when we are purportedly trying to show diffs for JavaScript, but if we think this fits the style, I'm happy to oblige.
Quoted 7 lines
The question is, with these patterns that are aware of CommonJS convention, would your bog-standard-and-boring vanilla JS code be detected incorrectly? Becoming aware of popular conventions without hurting others would be a good thing.
And the "popular conventions" does not have to be limited to CommonJS/Node.
From the perspective of "'exports' is a special name", yes, we could detect vanilla JS code 'incorrectly' because, in vanilla JS, the names 'exports' or 'module.exports' are not special. So perhaps, one could imagine a browser-side script that deals with "imports" and "exports" as part of their business:
const exports = { quantity: 1, type: 'boxes', }; exports.getQuantity = () => { foo(); };This diff driver would mistakenly detect
exports.getQuantity = () => {.Although, the more I think about it, the spirit of this patch seems to be "we want to show headers whenever we think we are in a function", so we don't actually need to treat 'exports' or 'module.exports' specially at all, e.g. this case should also pass our diff driver tests:
Quoted 8 lines
const foo = {}; foo.RIGHT = () => {ChangeMe();};
and if we do this, we will correctly handle 'exports' and 'module.exports' anyway by virtue of them being plain old JS objects.
The spirit of this patch is to show headers when we are in the function body(which has a large code block). Because this can help users understand the context.
And prevent mismatch non-related function. ex:
function WRONG() {
// ...
}
const foo = {};
foo.RIGHT = () => {
ChangeMe();
};if we don't match "foo.RIGHT = () => {". It may match the "function WRONG() {". This would be very misleading.
So the v3 patch. I do not treat exports as a special keyword. And can match the code like "foo.RIGHT = () => {".
The regex would be "((module\\.)?[$_[:alpha:]][$_[:alnum:]]*\\.[$_[:alpha:]][$_[:alnum:]]*[\t ]*=[\t ]*(\\(.*\\)|[$_[:alpha:]][$_[:alnum:]]*)[\t ]*=>.*)"
Re: [GSoC][PATCH v2] Add a diff driver for JavaScript languages.
In reply to Glen ChooQuoted 17 lines · Glen Choo
Johannes Sixt <[email protected]> writes:
So the question becomes "Is there a significant amount of code that uses this pattern?" Probably - this is a fairly common pattern in Node.js after all. But in my experience,module.exports.RIGHT = function(document) {
return ChangeMe;}
is even more common. The difference between 'module.exports' and 'exports' isn't worth going into (StackOverflow has all the answers, for the curious), but if we're taking the approach of supporting CommonJS, I'd like to be consistent and also support 'module.exports', i.e. perhaps change:
Quoted 5 lines
"^(exports\\.[$_[:alpha:]][$_[:alnum:]]*[\t ]*=[\t ]*(\\(.*\\)|[$_[:alpha:]][$_[:alnum:]]*)[\t ]*=>.*)\n"
to something like:
"^((module.)?exports\\.[$_[:alpha:]][$_[:alnum:]]*[\t ]*=[\t ]*(\\(.*\\)|[$_[:alpha:]][$_[:alnum:]]*)[\t ]*=>.*)\n"
I agree on also support the "module.exports". This will be applied on the v3 patch
Re: [GSoC][PATCH v2] Add a diff driver for JavaScript languages.
In reply to Glen ChooSorry for the late reply. Because I was busy last two weeks. And grateful for your review.
Quoted 30 lines · Glen Choo
Quoted 17 lines
t/t4018/javascript-function new file +4
-0@@ -0,0 +1,4 @@ +function RIGHT(a, b) { + + return a + b; // ChangeMe +}t/t4018/javascript-function-2 new file +0
-0t/t4018/javascript-function-2 line 10
Quoted 10 lines
t/t4018/javascript-function-2 new file +10
-0@@ -0,0 +1,10 @@ +function test(a, b) { + return { + RIGHT: function () { + currentUpdateRemovedChunks.forEach(function (chunkId) { + delete $installedChunks$[chunkId]; + }); + currentUpdateRemovedChunks = ChangeMe; + } + } +}There is also the ES2015 'method shorthand' syntax [3], e.g.
barin:
Quoted 8 lines
const foo = { bar() { console.log('hi'); } }[3] https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Functions/Method_definitions
The ES2015 method shorthand is already matched by the regex for function in class
"^[\t ]*((static[\t ]+)?((async|get|set)[\t ]+)?[$_[:alpha:]][$_[:alnum:]]*[\t ]*\\(.*)"
So I update the comment for this regex in the v3 patch. The comment is "/* match function in class and ES5 method shorthand */"
Quoted 23 lines
t/t4018/javascript-function-belong-to-IIFE line 6
Quoted 12 lines
t/t4018/javascript-function-belong-to-IIFE new file +6
-0@@ -0,0 +1,6 @@ +(function () { + this.$RIGHT = function (needle, modifier) { + let a = 5; + return ChangeMe; + }; +}).call(aaaa.prototype);Does the IIFE matter in this case? This line:
this.$RIGHT = function (needle, modifier) {looks extremely similar to the previous test of
foo = function bar().Or perhaps this is meant to demonstrate the edge case of "matching in a complicated construct"? If so, perhaps we should test other edge cases like:
Quoted 7 lines
function WRONG() {
let RIGHT = function (needle, modifier) {
let a = 5;
return ChangeMe;
};
}Currently, I realize this test case is redundant. Because I trust the function keyword. So the regex "^(.*=[\t ]*function[\t ]*([$_[:alnum:]]+[\t ]*)?\\(.*)\n" is loose for LHS.
I will remove this case in the v3 patch.
PATCH v3 1/1 Add a diff driver for JavaScript languages.
In reply to xing zhi jiangIn the xfunction part that matches normal functions,
a variable declaration with an assignment of function, the function declaration in the class, named export for function in ECMA2015, CommonJS exports for named function, and also the function is object literal's property[1]. The special match is this would match the popular framework test case pattern(most is same as regex for matching function in class).
And in the word regex part, that matches numbers, punctuations, and also the JavaScript identifier.
This part reference the formal ECMA specification[2].
[1]https://github.com/jquery/jquery/blob/de5398a6ad088dc006b46c6a870a2a053f4cd663/src/core.js#L201 [2]https://262.ecma-international.org/12.0/#sec-ecmascript-language-lexical-grammar
---
I write a simple python script to help me observe the diff's results easier. And the result is in https://github.com/a97410985/user_diff_observe_tool.
I observe the six popular JavaScript projects - Jquery, NodeJS, React, TensorflowJS, Vue, and Webpack. The result is not perfect but acceptable.
Most of the mismatch is related to the function in class. Ex:
https://github.com/a97410985/user_diff_observe_tool/blob/092bd14c849b9a3f61bb872d730e38b58cc9c303/diff_result/jquery_result.diff#L1094.
But this is inevitable.
And besides, apply the code reviews suggestions. I add some new regex.
1. "!^[\t ]*[$_[:alpha:]][$_[:alnum:]][\t ]*\\(.*\\)\\.\n"
this is for the case below
`
function RIGHT(a, b) {
WRONG(spy.calls.count()).toBe(1)
return a + b; // ChangeMe
} `
if we don't skip chained function call. This would happen mismatch.
2. "^[\t ]*(QUnit.test\\(.*)\n"
I oberve there are many test case pattern in JavaScript. And most of popular JavaScript unit test framework's test case pattern is function call can be matched by the regex, which match "function in class". So it is no problem. But QUnit framework don't matched by that. Then add this regex into JavaScript userdiff.
And the reason includes test case pattern has two
1. This is inevitable. Because we can't distinguish that with function in class.
2. This can help the user understand what change is happening in which test case. And if don't match the test case, it has the large possibility to match no related function.
And I move the regex, which match normal function in object literal, under the regex ""!^(\t{2,}|[ ]{5,})\n".
Because I found it would many mismatch if function in object literal has too many indent level( the function body usually just few lines of code, and the below has large of code. If has change on below, it would happen mismatch).
And there are two example for hunk header that match test case pattern.
Range-diff against v2:
1: 3b326bd2b6 ! 1: 2791a7b6b8 Add a diff driver for JavaScript languages.
@@ Commit message
In the xfunction part that matches normal functions,
a variable declaration with an assignment of function, the function declaration
- in the class, and also the function is object literal's property[1].
+ in the class, named export for function in ECMA2015, CommonJS exports for
+ named function, and also the function is object literal's property[1].
+ The special match is this would match the popular framework test case
+ pattern(most is same as regex for matching function in class).
And in the word regex part, that matches numbers, punctuations, and also the
JavaScript identifier.
@@ t/t4018/javascript-async-function (new)
+async function RIGHT(a, b) {
+
+ return a + b; // ChangeMe
++}
+
+ ## t/t4018/javascript-es5-method-shorthand (new) ##
+@@
++const foo = {
++ RIGHT() {
++
++ console.log('ChangeMe');
++ }
+}
## t/t4018/javascript-export-async-function (new) ##
@@ t/t4018/javascript-exports-anomyous-function-2 (new)
+exports.RIGHT = (a, b, runtime) => {
+
+ return ChangeMe;
++};
+
+ ## t/t4018/javascript-exports-anomyous-function-3 (new) ##
+@@
++module.exports.RIGHT = (a, b, runtime) => {
++
++ return ChangeMe;
++};
+
+ ## t/t4018/javascript-exports-anomyous-function-4 (new) ##
+@@
++module.exports = ( RIGHT ) => {
++
++ return ChangeMe;
+};
## t/t4018/javascript-exports-function (new) ##
@@ t/t4018/javascript-function (new)
+ return a + b; // ChangeMe
+}
- ## t/t4018/javascript-function-2 (new) ##
-@@
-+function test(a, b) {
-+ return {
-+ RIGHT: function () {
-+ currentUpdateRemovedChunks.forEach(function (chunkId) {
-+ delete $installedChunks$[chunkId];
-+ });
-+ currentUpdateRemovedChunks = ChangeMe;
-+ }
-+ }
-+}
-
- ## t/t4018/javascript-function-belong-to-IIFE (new) ##
-@@
-+(function () {
-+ this.$RIGHT = function (needle, modifier) {
-+ let a = 5;
-+ return ChangeMe;
-+ };
-+}).call(aaaa.prototype);
-
## t/t4018/javascript-function-in-class (new) ##
@@
+class Test {
@@ t/t4018/javascript-setter-function-in-class (new)
+ let a = 4;
+ let b = ChangeMe;
+ }
++}
+
+ ## t/t4018/javascript-skip-chained-function (new) ##
+@@
++function RIGHT(a, b) {
++
++ expect(spy.calls.count()).toBe(1)
++ return a + b; // ChangeMe
+}
## t/t4018/javascript-skip-function-call-statement (new) ##
@@ userdiff.c: PATTERNS("java",
+ "!^[ \t]*(if|do|while|for|with|switch|catch|import|return)\n"
+ /* don't match statement */
+ "!;\n"
-+ /* match normal function */
++ /* match normal function or named export for function in ECMA2015 */
+ "^((export[\t ]+)?(async[\t ]+)?function[\t ]*[\t *]*[$_[:alpha:]][$_[:alnum:]]*[\t ]*\\(.*)\n"
-+ /* match JavaScript variable declaration with a lambda expression */
-+ "^[\t ]*((const|let|var)[\t ]*[$_[:alpha:]][$_[:alnum:]]*[\t ]*=[\t ]*"
-+ "(\\(.*\\)|[$_[:alpha:]][$_[:alnum:]]*)[\t ]*=>[\t ]*\\{?)\n"
-+ /* match exports for anonymous fucntion */
-+ "^(exports\\.[$_[:alpha:]][$_[:alnum:]]*[\t ]*=[\t ]*(\\(.*\\)|[$_[:alpha:]][$_[:alnum:]]*)[\t ]*=>.*)\n"
-+ /* match assign function to LHS */
-+ "^(.*=[\t ]*function[\t ]*([$_[:alpha:]][$_[:alnum:]]*)?[\t ]*\\(.*)\n"
-+ /* match normal function in object literal */
-+ "^[\t ]*([$_[:alpha:]][$_[:alnum:]]*[\t ]*:[\t ]*function[\t ].*)\n"
-+ /* don't match the function in class, which has more than one ident level */
++ /* match JavaScript variable declaration with a lambda expression at top level */
++ "^((const|let|var)[\t ]*[$_[:alpha:]][$_[:alnum:]]*[\t ]*=[\t ]*"
++ "(\\(.*\\)|[$_[:alpha:]][$_[:alnum:]]*)[\t ]*=>[\t ]*\\{?)\n"
++ /* match object's property assignment by anonymous function and CommonJS exports for named function */
++ "^((module\\.)?[$_[:alpha:]][$_[:alnum:]]*\\.[$_[:alpha:]][$_[:alnum:]]*[\t ]*=[\t ]*(async[\t ]+)?(\\(.*\\)|[$_[:alpha:]][$_[:alnum:]]*)[\t ]*=>.*)\n"
++ /* match assign function to LHS with explicit function keyword */
++ "^(.*=[\t ]*function[\t ]*([$_[:alnum:]]+[\t ]*)?\\(.*)\n"
++ /* popular unit testing framework test case pattern. Most of framework pattern is match by regex for "function in class" */
++ "^[\t ]*(QUnit.test\\(.*)\n"
++ /* don't match the function in class or in object literal, which has more than one ident level */
+ "!^(\t{2,}|[ ]{5,})\n"
-+ /* match function in class */
++ /* match normal function in object literal */
++ "^[\t ]*([$_[:alpha:]][$_[:alnum:]]*[\t ]*:[\t ]*function.*)\n"
++ /* don't match chained method call */
++ "!^[\t ]*[$_[:alpha:]][$_[:alnum:]][\t ]*\\(.*\\)\\.\n"
++ /* match function in class and ES5 method shorthand */
+ "^[\t ]*((static[\t ]+)?((async|get|set)[\t ]+)?[$_[:alpha:]][$_[:alnum:]]*[\t ]*\\(.*)",
+ /* word regex */
-+ /* hexIntegerLiteral, octalIntegerLiteral, binaryIntegerLiteral, DecimalLiteral and its big version */
-+ "(0[xXoObB])?[0-9a-fA-F][_0-9a-fA-F]*n?"
-+ /* DecimalLiteral may be float */
-+ "|(0|[1-9][_0-9]*)?\\.?[0-9][_0-9]*([eE][+-]?[_0-9]+)?"
++ /* hexIntegerLiteral, octalIntegerLiteral, binaryIntegerLiteral, and its big version */
++ "0[xXoObB][_0-9a-fA-F]+n?"
++ /* DecimalLiteral and its big version*/
++ "|[0-9][_0-9]*(\\.[0-9][_0-9]*|n)?([eE][+-]?[_0-9]+)?"
++ "|\\.[0-9][_0-9]*([eE][+-]?[_0-9]+)?"
+ /* punctuations */
+ "|\\.{3}|<=|>=|==|!=|={3}|!==|\\*{2}|\\+{2}|--|<<|>>"
+ "|>>>|&&|\\|{2}|\\?{2}|\\+=|-=|\\*=|%=|\\*{2}="Documentation/gitattributes.txt | 2 + ...avascript-assignment-of-anonymous-function | 4 ++ .../javascript-assignment-of-arrow-function | 4 ++ .../javascript-assignment-of-arrow-function-2 | 4 ++ .../javascript-assignment-of-arrow-function-3 | 4 ++ .../javascript-assignment-of-named-function | 4 ++ t/t4018/javascript-async-function | 4 ++ t/t4018/javascript-es5-method-shorthand | 6 +++ t/t4018/javascript-export-async-function | 4 ++ t/t4018/javascript-export-function | 4 ++ t/t4018/javascript-exports-anomyous-function | 4 ++ .../javascript-exports-anomyous-function-2 | 4 ++ .../javascript-exports-anomyous-function-3 | 4 ++ .../javascript-exports-anomyous-function-4 | 4 ++ t/t4018/javascript-exports-function | 4 ++ t/t4018/javascript-function | 4 ++ t/t4018/javascript-function-in-class | 6 +++ t/t4018/javascript-function-in-class-2 | 11 ++++ t/t4018/javascript-function-in-class-3 | 10 ++++ t/t4018/javascript-function-in-object-literal | 7 +++ t/t4018/javascript-generator-function | 4 ++ t/t4018/javascript-generator-function-2 | 4 ++ t/t4018/javascript-getter-function-in-class | 6 +++ t/t4018/javascript-setter-function-in-class | 6 +++ t/t4018/javascript-skip-chained-function | 5 ++ .../javascript-skip-function-call-statement | 7 +++ t/t4018/javascript-skip-keywords | 34 ++++++++++++ t/t4018/javascript-static-function-in-class | 6 +++ t/t4034-diff-words.sh | 1 + t/t4034/javascript/expect | 54 +++++++++++++++++++ t/t4034/javascript/post | 33 ++++++++++++ t/t4034/javascript/pre | 33 ++++++++++++ userdiff.c | 37 +++++++++++++ 33 files changed, 328 insertions(+) create mode 100644 t/t4018/javascript-assignment-of-anonymous-function create mode 100644 t/t4018/javascript-assignment-of-arrow-function create mode 100644 t/t4018/javascript-assignment-of-arrow-function-2 create mode 100644 t/t4018/javascript-assignment-of-arrow-function-3 create mode 100644 t/t4018/javascript-assignment-of-named-function create mode 100644 t/t4018/javascript-async-function create mode 100644 t/t4018/javascript-es5-method-shorthand create mode 100644 t/t4018/javascript-export-async-function create mode 100644 t/t4018/javascript-export-function create mode 100644 t/t4018/javascript-exports-anomyous-function create mode 100644 t/t4018/javascript-exports-anomyous-function-2 create mode 100644 t/t4018/javascript-exports-anomyous-function-3 create mode 100644 t/t4018/javascript-exports-anomyous-function-4 create mode 100644 t/t4018/javascript-exports-function create mode 100644 t/t4018/javascript-function create mode 100644 t/t4018/javascript-function-in-class create mode 100644 t/t4018/javascript-function-in-class-2 create mode 100644 t/t4018/javascript-function-in-class-3 create mode 100644 t/t4018/javascript-function-in-object-literal create mode 100644 t/t4018/javascript-generator-function create mode 100644 t/t4018/javascript-generator-function-2 create mode 100644 t/t4018/javascript-getter-function-in-class create mode 100644 t/t4018/javascript-setter-function-in-class create mode 100644 t/t4018/javascript-skip-chained-function create mode 100644 t/t4018/javascript-skip-function-call-statement create mode 100644 t/t4018/javascript-skip-keywords create mode 100644 t/t4018/javascript-static-function-in-class create mode 100644 t/t4034/javascript/expect create mode 100644 t/t4034/javascript/post create mode 100644 t/t4034/javascript/pre
Documentation/gitattributes.txt +2 -0
@@ -828,6 +828,8 @@ patterns are available: - `java` suitable for source code in the Java language. +- `javascript` suitable for source code in the JavaScript language. + - `markdown` suitable for Markdown documents. - `matlab` suitable for source code in the MATLAB and Octave languages.
t/t4018/javascript-assignment-of-anonymous-function new file +4 -0
@@ -0,0 +1,4 @@ +const RIGHT = function (a, b) { + + return a + b; // ChangeMe +};
t/t4018/javascript-assignment-of-arrow-function new file +4 -0
@@ -0,0 +1,4 @@ +const RIGHT = (a, b) => { + + return a + b; // ChangeMe +};
t/t4018/javascript-assignment-of-arrow-function-2 new file +4 -0
@@ -0,0 +1,4 @@ +const RIGHT = (a, b)=>{ + + return a + b; // ChangeMe +};
t/t4018/javascript-assignment-of-arrow-function-3 new file +4 -0
@@ -0,0 +1,4 @@ +const RIGHT=test=>{ + + return test + 1; // ChangeMe +};
t/t4018/javascript-assignment-of-named-function new file +4 -0
@@ -0,0 +1,4 @@ +const RIGHT = function test (a, b) { + + return a + b; // ChangeMe +};
t/t4018/javascript-async-function new file +4 -0
@@ -0,0 +1,4 @@ +async function RIGHT(a, b) { + + return a + b; // ChangeMe +}
t/t4018/javascript-es5-method-shorthand new file +6 -0
@@ -0,0 +1,6 @@ +const foo = { + RIGHT() { + + console.log('ChangeMe'); + } +}
t/t4018/javascript-export-async-function new file +4 -0
@@ -0,0 +1,4 @@ +export async function RIGHT(a, b) { + + return a + b; // ChangeMe +}
t/t4018/javascript-export-function new file +4 -0
@@ -0,0 +1,4 @@ +export function RIGHT(a, b) { + + return a + b; // ChangeMe +}
t/t4018/javascript-exports-anomyous-function new file +4 -0
@@ -0,0 +1,4 @@ +exports.setFlagged = RIGHT => { + + return ChangeMe; +};
t/t4018/javascript-exports-anomyous-function-2 new file +4 -0
@@ -0,0 +1,4 @@ +exports.RIGHT = (a, b, runtime) => { + + return ChangeMe; +};
t/t4018/javascript-exports-anomyous-function-3 new file +4 -0
@@ -0,0 +1,4 @@ +module.exports.RIGHT = (a, b, runtime) => { + + return ChangeMe; +};
t/t4018/javascript-exports-anomyous-function-4 new file +4 -0
@@ -0,0 +1,4 @@ +module.exports = ( RIGHT ) => { + + return ChangeMe; +};
t/t4018/javascript-exports-function new file +4 -0
@@ -0,0 +1,4 @@ +exports.RIGHT = function(document) { + + return ChangeMe; +}
t/t4018/javascript-function new file +4 -0
@@ -0,0 +1,4 @@ +function RIGHT(a, b) { + + return a + b; // ChangeMe +}
t/t4018/javascript-function-in-class new file +6 -0
@@ -0,0 +1,6 @@ +class Test { + RIGHT() { + let a = 4; + let b = ChangeMe; + } +}
t/t4018/javascript-function-in-class-2 new file +11 -0
@@ -0,0 +1,11 @@ +class Test { + RIGHT( + aaaaaaaaaa, + bbbbbbbbbb, + cccccccccc, + dddddddddd + ) { + let a = 4; + let b = ChangeMe; + } +}
t/t4018/javascript-function-in-class-3 new file +10 -0
@@ -0,0 +1,10 @@ +class Test { + RIGHT(aaaaaaaaaa, + bbbbbbbbbb, + cccccccccc, + dddddddddd + ) { + let a = 4; + let b = ChangeMe; + } +}
t/t4018/javascript-function-in-object-literal new file +7 -0
@@ -0,0 +1,7 @@ +const obj = { + RIGHT: function (elems, callback, arg) { + var length, value; + // ... + return ChangeMe + } +}
t/t4018/javascript-generator-function new file +4 -0
@@ -0,0 +1,4 @@ +function* RIGHT(a, b) { + + return a + b; // ChangeMe +}
t/t4018/javascript-generator-function-2 new file +4 -0
@@ -0,0 +1,4 @@ +function *RIGHT(a, b) { + + return a + b; // ChangeMe +}
t/t4018/javascript-getter-function-in-class new file +6 -0
@@ -0,0 +1,6 @@ +class Test { + get RIGHT() { + let a = 4; + let b = ChangeMe; + } +}
t/t4018/javascript-setter-function-in-class new file +6 -0
@@ -0,0 +1,6 @@ +class Test { + set RIGHT() { + let a = 4; + let b = ChangeMe; + } +}
t/t4018/javascript-skip-chained-function new file +5 -0
@@ -0,0 +1,5 @@ +function RIGHT(a, b) { + + expect(spy.calls.count()).toBe(1) + return a + b; // ChangeMe +}
t/t4018/javascript-skip-function-call-statement new file +7 -0
@@ -0,0 +1,7 @@ +class Test { + static RIGHT() { + haha(); + haha2() + let b = ChangeMe; + } +}
t/t4018/javascript-skip-keywords new file +34 -0
@@ -0,0 +1,34 @@ +function RIGHT(a, b) { + import("./async1") + if (a > 1) { + // ... + } + do { + // ... + } while (i < 5); + for (const element of array1) { + console.log(element) + } + with(o) { + console.log(x) + } + switch (expr) { + case 'a': + // ... + break; + case 'b': + // ... + break; + default: + // ... + } + try { + // ... + return (a + c) + } + catch (error) { + // ... + } + + return a + b; // ChangeMe +}
t/t4018/javascript-static-function-in-class new file +6 -0
@@ -0,0 +1,6 @@ +class Test { + static RIGHT() { + let a = 4; + let b = ChangeMe; + } +}
t/t4034-diff-words.sh +1 -0
@@ -324,6 +324,7 @@ test_language_driver dts test_language_driver fortran test_language_driver html test_language_driver java +test_language_driver javascript test_language_driver matlab test_language_driver objc test_language_driver pascal
t/t4034/javascript/expect new file +54 -0
54 lines not on this page · raw message
t/t4034/javascript/post new file +33 -0
@@ -0,0 +1,33 @@ +// DecimalLiteral +124 +0.124 +.124 +0.123e-5 +0.123E-5 +0.123E5 +12222222222222222233344445n +// HexIntegerLiteral +0x11 +0X5Fa1 +0x123_756 +0x1234182989812f1289bn +// OctalIntegerLiteral +06 +0o5 +0O4 +0511_567 +0o424242424242424242424242424242666667n +// BinaryIntegerLiteral +0b1101 +0B0010 +0b0001_1101_0011 +0b11111111111111000011111111111111111n +// punctuations +{b} (b) +b; +[2,3] +[ ...params_v2 ] +a=2 a=2 a=2 a=2 a=2 a=2 a=2 a=2 +a-=b a%=b a&&=b a|=b +b-c a++ a<<b a>>>=b a<<=b +a&b a|b a??=b
t/t4034/javascript/pre new file +33 -0
@@ -0,0 +1,33 @@ +// DecimalLiteral +123 +0.123 +.123 +0.123e+5 +0.123E+5 +0.123e5 +1222222222222222223334444n +// HexIntegerLiteral +0x10 +0X6Fa1 +0x123_456 +0x1234182989812f1289an +// OctalIntegerLiteral +05 +0o6 +0O7 +0512_567 +0o424242424242424242424242424242666666n +// BinaryIntegerLiteral +0b1001 +0B0110 +0b0001_1001_0011 +0b1111111111111111111111111111111111111n +// punctuations +{a} (a) +a; +[1,2] +[ 1, 2, ...params ] +a<=2 a>=2 a==2 a!=2 a===2 a!==2 a^=2 a=>2 +a+=b a*=b a**=b a||=b +b+c a-- a>>b a>>>b a>>=b +a&&b a||b a&&=b
userdiff.c +37 -0
@@ -168,6 +168,43 @@ PATTERNS("java","|[-+0-9.e]+[fFlL]?|0[xXbB]?[0-9a-fA-F]+[lL]?" "|[-+*/<>%&^|=!]=" "|--|\\+\\+|<<=?|>>>?=?|&&|\\|\\|"), + +PATTERNS ("javascript" , +/* don't match the expression may contain parenthesis, because it is not a function declaration */ +"!^[ \t]*(if|do|while|for|with|switch|catch|import|return)\n" +/* don't match statement */ +"!;\n" +/* match normal function or named export for function in ECMA2015 */ +"^((export[\t ]+)?(async[\t ]+)?function[\t ]*[\t *]*[$_[:alpha:]][$_[:alnum:]]*[\t ]*\\(.*)\n" +/* match JavaScript variable declaration with a lambda expression at top level */ +"^((const|let|var)[\t ]*[$_[:alpha:]][$_[:alnum:]]*[\t ]*=[\t ]*" +"(\\(.*\\)|[$_[:alpha:]][$_[:alnum:]]*)[\t ]*=>[\t ]*\\{?)\n" +/* match object's property assignment by anonymous function and CommonJS exports for named function */ +"^((module\\.)?[$_[:alpha:]][$_[:alnum:]]*\\.[$_[:alpha:]][$_[:alnum:]]*[\t ]*=[\t ]*(async[\t ]+)?(\\(.*\\)|[$_[:alpha:]][$_[:alnum:]]*)[\t ]*=>.*)\n" +/* match assign function to LHS with explicit function keyword */ +"^(.*=[\t ]*function[\t ]*([$_[:alnum:]]+[\t ]*)?\\(.*)\n" +/* popular unit testing framework test case pattern. Most of framework pattern is match by regex for "function in class" */ +"^[\t ]*(QUnit.test\\(.*)\n" +/* don't match the function in class or in object literal, which has more than one ident level */ +"!^(\t{2,}|[ ]{5,})\n" +/* match normal function in object literal */ +"^[\t ]*([$_[:alpha:]][$_[:alnum:]]*[\t ]*:[\t ]*function.*)\n" +/* don't match chained method call */ +"!^[\t ]*[$_[:alpha:]][$_[:alnum:]][\t ]*\\(.*\\)\\.\n" +/* match function in class and ES5 method shorthand */ +"^[\t ]*((static[\t ]+)?((async|get|set)[\t ]+)?[$_[:alpha:]][$_[:alnum:]]*[\t ]*\\(.*)" , +/* word regex */ +/* hexIntegerLiteral, octalIntegerLiteral, binaryIntegerLiteral, and its big version */ +"0[xXoObB][_0-9a-fA-F]+n?" +/* DecimalLiteral and its big version*/ +"|[0-9][_0-9]*(\\.[0-9][_0-9]*|n)?([eE][+-]?[_0-9]+)?" +"|\\.[0-9][_0-9]*([eE][+-]?[_0-9]+)?" +/* punctuations */ +"|\\.{3}|<=|>=|==|!=|={3}|!==|\\*{2}|\\+{2}|--|<<|>>" +"|>>>|&&|\\|{2}|\\?{2}|\\+=|-=|\\*=|%=|\\*{2}=" +"|<<=|>>=|>>>=|&=|\\|=|\\^=|&&=|\\|{2}=|\\?{2}=|=>" +/* identifiers */ +"|[$_[:alpha:]][$_[:alnum:]]*" ), PATTERNS("markdown", "^ {0,3}#{1,6}[ \t].*",/* -- */
Signature · 1 line
2.35.1.273.ge6ebfd0e8c.dirty
Re: [GSoC][PATCH v3] Add a diff driver for JavaScript languages.
Am 03.04.22 um 15:25 schrieb xing zhi jiang:
Quoted 27 lines
In the xfunction part that matches normal functions,
a variable declaration with an assignment of function, the function declaration in the class, named export for function in ECMA2015, CommonJS exports for named function, and also the function is object literal's property[1]. The special match is this would match the popular framework test case pattern(most is same as regex for matching function in class).And in the word regex part, that matches numbers, punctuations, and also the JavaScript identifier.
This part reference the formal ECMA specification[2].[1]https://github.com/jquery/jquery/blob/de5398a6ad088dc006b46c6a870a2a053f4cd663/src/core.js#L201 [2]https://262.ecma-international.org/12.0/#sec-ecmascript-language-lexical-grammar
Signed-off-by xing zhi jiang <[email protected]>---
I write a simple python script to help me observe the diff's results easier. And the result is in https://github.com/a97410985/user_diff_observe_tool.
I observe the six popular JavaScript projects - Jquery, NodeJS, React, TensorflowJS, Vue, and Webpack. The result is not perfect but acceptable.
Most of the mismatch is related to the function in class. Ex:
https://github.com/a97410985/user_diff_observe_tool/blob/092bd14c849b9a3f61bb872d730e38b58cc9c303/diff_result/jquery_result.diff#L1094.
But this is inevitable.
And besides, apply the code reviews suggestions. I add some new regex.
1. "!^[\t ]*[$_[:alpha:]][$_[:alnum:]][\t ]*\\(.*\\)\\.\n"
this is for the case below
`
Quoted 7 lines
function RIGHT(a, b) { WRONG(spy.calls.count()).toBe(1) return a + b; // ChangeMe }`
if we don't skip chained function call. This would happen mismatch.
OK, but see below.
Quoted 9 lines
2. "^[\t ]*(QUnit.test\\(.*)\n"
I oberve there are many test case pattern in JavaScript. And most of popular JavaScript unit test framework's test case pattern is function call can be matched by the regex, which match "function in class". So it is no problem. ButQUnit frameworkdon't matched by that. Then add this regex into JavaScript userdiff.
And the reason includes test case pattern has two
1. This is inevitable. Because we can't distinguish that with function in class.
2. This can help the user understand what change is happening in which test case. And if don't match the test case, it has the large possibility to match no related function.
Good. I consider this like matching a language keyword. If additional patterns of this kind occur, they can be added later, but we must make sure that this does not proliferate.
Quoted 7 lines
And I move the regex, which match normal function in object literal, under the regex ""!^(\t{2,}|[ ]{5,})\n".
Because I found it would many mismatch if function in object literal has too many indent level( the function body usually just few lines of code, and the below has large of code. If has change on below, it would happen mismatch).
And there are two example for hunk header that match test case pattern.
OK.
t/t4018/javascript-skip-chained-function line 5
Quoted 11 lines
t/t4018/javascript-skip-chained-function new file +5
-0@@ -0,0 +1,5 @@ +function RIGHT(a, b) { + + expect(spy.calls.count()).toBe(1) + return a + b; // ChangeMe +}
This is the test case that attempts to make sure that the chained function calls are not matched. But since there is no line between ChangeMe and the not-to-be-matched line, this test case would not detect such an incorrect match. You have to insert a line between expect... and return...ChangeMe
Quoted 23 lines
userdiff.c +15
-0@@ -168,6 +168,43 @@ PATTERNS("java","|[-+0-9.e]+[fFlL]?|0[xXbB]?[0-9a-fA-F]+[lL]?" "|[-+*/<>%&^|=!]=" "|--|\\+\\+|<<=?|>>>?=?|&&|\\|\\|"), + +PATTERNS ("javascript" , +/* don't match the expression may contain parenthesis, because it is not a function declaration */ +"!^[ \t]*(if|do|while|for|with|switch|catch|import|return)\n" +/* don't match statement */ +"!;\n" +/* match normal function or named export for function in ECMA2015 */ +"^((export[\t ]+)?(async[\t ]+)?function[\t ]*[\t *]*[$_[:alpha:]][$_[:alnum:]]*[\t ]*\\(.*)\n" +/* match JavaScript variable declaration with a lambda expression at top level */ +"^((const|let|var)[\t ]*[$_[:alpha:]][$_[:alnum:]]*[\t ]*=[\t ]*" +"(\\(.*\\)|[$_[:alpha:]][$_[:alnum:]]*)[\t ]*=>[\t ]*\\{?)\n" +/* match object's property assignment by anonymous function and CommonJS exports for named function */ +"^((module\\.)?[$_[:alpha:]][$_[:alnum:]]*\\.[$_[:alpha:]][$_[:alnum:]]*[\t ]*=[\t ]*(async[\t ]+)?(\\(.*\\)|[$_[:alpha:]][$_[:alnum:]]*)[\t ]*=>.*)\n" +/* match assign function to LHS with explicit function keyword */ + "^(.*=[\t ]*function[\t ]*([$_[:alnum:]]+[\t ]*)?\\(.*)\n"
userdiff.c PATTERNS("java", · line 195
Quoted 10 lines
userdiff.c +10
-0+/* popular unit testing framework test case pattern. Most of framework pattern is match by regex for "function in class" */ +"^[\t ]*(QUnit.test\\(.*)\n" +/* don't match the function in class or in object literal, which has more than one ident level */ +"!^(\t{2,}|[ ]{5,})\n" +/* match normal function in object literal */ +"^[\t ]*([$_[:alpha:]][$_[:alnum:]]*[\t ]*:[\t ]*function.*)\n" +/* don't match chained method call */ +"!^[\t ]*[$_[:alpha:]][$_[:alnum:]][\t ]*\\(.*\\)\\.\n" +/* match function in class and ES5 method shorthand */ + "^[\t ]*((static[\t ]+)?((async|get|set)[\t ]+)?[$_[:alpha:]][$_[:alnum:]]*[\t ]*\\(.*)",
No objections to these expressions.
userdiff.c PATTERNS("java", · line 207
Quoted 12 lines
userdiff.c +12
-0+/* word regex */ +/* hexIntegerLiteral, octalIntegerLiteral, binaryIntegerLiteral, and its big version */ +"0[xXoObB][_0-9a-fA-F]+n?" +/* DecimalLiteral and its big version*/ +"|[0-9][_0-9]*(\\.[0-9][_0-9]*|n)?([eE][+-]?[_0-9]+)?" +"|\\.[0-9][_0-9]*([eE][+-]?[_0-9]+)?" +/* punctuations */ +"|\\.{3}|<=|>=|==|!=|={3}|!==|\\*{2}|\\+{2}|--|<<|>>" +"|>>>|&&|\\|{2}|\\?{2}|\\+=|-=|\\*=|%=|\\*{2}=" +"|<<=|>>=|>>>=|&=|\\|=|\\^=|&&=|\\|{2}=|\\?{2}=|=>" +/* identifiers */ + "|[$_[:alpha:]][$_[:alnum:]]*"),
These word-regexs look good.
userdiff.c PATTERNS("java", · line 210
Quoted 3 lines
userdiff.c +0
-0PATTERNS("markdown", "^ {0,3}#{1,6}[ \t].*",/* -- */
This round looks quite good. With the one issue I noted above fixed, this should be good to go, IMO.
-- Hannes
Re: [GSoC][PATCH v3] Add a diff driver for JavaScript languages.
In reply to xing zhi jiangOn Sun, Apr 03 2022, xing zhi jiang wrote:
Aside from what Johannes Sixt mentioned:
Quoted 29 lines
userdiff.c +29
-0+PATTERNS ("javascript" , +/* don't match the expression may contain parenthesis, because it is not a function declaration */ +"!^[ \t]*(if|do|while|for|with|switch|catch|import|return)\n" +/* don't match statement */ +"!;\n" +/* match normal function or named export for function in ECMA2015 */ +"^((export[\t ]+)?(async[\t ]+)?function[\t ]*[\t *]*[$_[:alpha:]][$_[:alnum:]]*[\t ]*\\(.*)\n" +/* match JavaScript variable declaration with a lambda expression at top level */ +"^((const|let|var)[\t ]*[$_[:alpha:]][$_[:alnum:]]*[\t ]*=[\t ]*" +"(\\(.*\\)|[$_[:alpha:]][$_[:alnum:]]*)[\t ]*=>[\t ]*\\{?)\n" +/* match object's property assignment by anonymous function and CommonJS exports for named function */ +"^((module\\.)?[$_[:alpha:]][$_[:alnum:]]*\\.[$_[:alpha:]][$_[:alnum:]]*[\t ]*=[\t ]*(async[\t ]+)?(\\(.*\\)|[$_[:alpha:]][$_[:alnum:]]*)[\t ]*=>.*)\n" +/* match assign function to LHS with explicit function keyword */ +"^(.*=[\t ]*function[\t ]*([$_[:alnum:]]+[\t ]*)?\\(.*)\n" +/* popular unit testing framework test case pattern. Most of framework pattern is match by regex for "function in class" */ +"^[\t ]*(QUnit.test\\(.*)\n" +/* don't match the function in class or in object literal, which has more than one ident level */ +"!^(\t{2,}|[ ]{5,})\n" +/* match normal function in object literal */ +"^[\t ]*([$_[:alpha:]][$_[:alnum:]]*[\t ]*:[\t ]*function.*)\n" +/* don't match chained method call */ +"!^[\t ]*[$_[:alpha:]][$_[:alnum:]][\t ]*\\(.*\\)\\.\n" +/* match function in class and ES5 method shorthand */ +"^[\t ]*((static[\t ]+)?((async|get|set)[\t ]+)?[$_[:alpha:]][$_[:alnum:]]*[\t ]*\\(.*)" , +/* word regex */ +/* hexIntegerLiteral, octalIntegerLiteral, binaryIntegerLiteral, and its big version */ +"0[xXoObB][_0-9a-fA-F]+n?" +/* DecimalLiteral and its big version*/ + "|[0-9][_0-9]*(\\.[0-9][_0-9]*|n)?([eE][+-]?[_0-9]+)?"
userdiff.c PATTERNS("java",
Quoted 10 lines
userdiff.c +7
-0+"|\\.[0-9][_0-9]*([eE][+-]?[_0-9]+)?" +/* punctuations */ +"|\\.{3}|<=|>=|==|!=|={3}|!==|\\*{2}|\\+{2}|--|<<|>>" +"|>>>|&&|\\|{2}|\\?{2}|\\+=|-=|\\*=|%=|\\*{2}=" +"|<<=|>>=|>>>=|&=|\\|=|\\^=|&&=|\\|{2}=|\\?{2}=|=>" +/* identifiers */ + "|[$_[:alpha:]][$_[:alnum:]]*"), PATTERNS("markdown", "^ {0,3}#{1,6}[ \t].*", /* -- */<
While we don't use helper macros for these currently there's no reason we can't, I thin the above might be more readable with e.g.:
#define JS_AA "[$_[:alpha:]][$_[:alnum:]]"
Which would make this:
+PATTERNS("javascript",
+ /* don't match the expression may contain parenthesis, because it is not a function declaration */
+ "!^[ \t]*(if|do|while|for|with|switch|catch|import|return)\n"
+ /* don't match statement */
+ "!;\n"
+ /* match normal function or named export for function in ECMA2015 */
+ "^((export[\t ]+)?(async[\t ]+)?function[\t ]*[\t *]*" JS_AA "*[\t ]*\\(.*)\n"
+ /* match JavaScript variable declaration with a lambda expression at top level */
+ "^((const|let|var)[\t ]*" JS_AA "*[\t ]*=[\t ]*"
+ "(\\(.*\\)|" JS_AA "*)[\t ]*=>[\t ]*\\{?)\n"
+ /* match object's property assignment by anonymous function and CommonJS exports for named function */
+ "^((module\\.)?" JS_AA "*\\." JS_AA "*[\t ]*=[\t ]*(async[\t ]+)?(\\(.*\\)|" JS_AA "*)[\t ]*=>.*)\n"
+ /* match assign function to LHS with explicit function keyword */
+ "^(.*=[\t ]*function[\t ]*([$_[:alnum:]]+[\t ]*)?\\(.*)\n"
+ /* popular unit testing framework test case pattern. Most of framework pattern is match by regex for "function in class" */Wry try to stick to wrapping at 80 characters, so some of these comments should really be wrapped (see CodingGuidelines for the multi-line comment style we use).
+ "^[\t ]*(QUnit.test\\(.*)\n"
+ /* don't match the function in class or in object literal, which has more than one ident level */
+ "!^(\t{2,}|[ ]{5,})\n"
+ /* match normal function in object literal */
+ "^[\t ]*(" JS_AA "*[\t ]*:[\t ]*function.*)\n"
+ /* don't match chained method call */
+ "!^[\t ]*" JS_AA "[\t ]*\\(.*\\)\\.\n"
+ /* match function in class and ES5 method shorthand */
+ "^[\t ]*((static[\t ]+)?((async|get|set)[\t ]+)?" JS_AA "*[\t ]*\\(.*)",
+ /* word regex */
+ /* hexIntegerLiteral, octalIntegerLiteral, binaryIntegerLiteral, and its big version */
+ "0[xXoObB][_0-9a-fA-F]+n?"
+ /* DecimalLiteral and its big version*/
+ "|[0-9][_0-9]*(\\.[0-9][_0-9]*|n)?([eE][+-]?[_0-9]+)?"
+ "|\\.[0-9][_0-9]*([eE][+-]?[_0-9]+)?"
+ /* punctuations */
+ "|\\.{3}|<=|>=|==|!=|={3}|!==|\\*{2}|\\+{2}|--|<<|>>"
+ "|>>>|&&|\\|{2}|\\?{2}|\\+=|-=|\\*=|%=|\\*{2}="
+ "|<<=|>>=|>>>=|&=|\\|=|\\^=|&&=|\\|{2}=|\\?{2}=|=>"
+ /* identifiers */
+ "|" JS_AA "*"),Just a thought, I wonder how much line-noisy we could make this thing in general if we defined some common patterns with such helpers.
Anyway, insted of :alnum:and :alpha: don't you really mean [a-zA-Z0-9] and [a-zA-Z]. I.e. do you *really* want to have this different depending on the user's locale?
I haven't tested, but see the LC_CTYPE in gettext.c, so I'm fairly sure that'll happen...
Re: [GSoC][PATCH v3] Add a diff driver for JavaScript languages.
Am 04.04.22 um 09:12 schrieb Ævar Arnfjörð Bjarmason:
Quoted 4 lines
While we don't use helper macros for these currently there's no reason we can't, I thin the above might be more readable with e.g.:
#define JS_AA "[$_[:alpha:]][$_[:alnum:]]"
Please consider including "identifier" somehow in the macro name. And add the trailing '*', which...
Quoted 22 lines
Which would make this:
+PATTERNS("javascript", + /* don't match the expression may contain parenthesis, because it is not a function declaration */ + "!^[ \t]*(if|do|while|for|with|switch|catch|import|return)\n" + /* don't match statement */ + "!;\n" + /* match normal function or named export for function in ECMA2015 */ + "^((export[\t ]+)?(async[\t ]+)?function[\t ]*[\t *]*" JS_AA "*[\t ]*\\(.*)\n" + /* match JavaScript variable declaration with a lambda expression at top level */ + "^((const|let|var)[\t ]*" JS_AA "*[\t ]*=[\t ]*" + "(\\(.*\\)|" JS_AA "*)[\t ]*=>[\t ]*\\{?)\n" + /* match object's property assignment by anonymous function and CommonJS exports for named function */ + "^((module\\.)?" JS_AA "*\\." JS_AA "*[\t ]*=[\t ]*(async[\t ]+)?(\\(.*\\)|" JS_AA "*)[\t ]*=>.*)\n" + /* match assign function to LHS with explicit function keyword */ + "^(.*=[\t ]*function[\t ]*([$_[:alnum:]]+[\t ]*)?\\(.*)\n" + /* popular unit testing framework test case pattern. Most of framework pattern is match by regex for "function in class" */Wry try to stick to wrapping at 80 characters, so some of these comments should really be wrapped (see CodingGuidelines for the multi-line comment style we use).
Quoted 7 lines
+ "^[\t ]*(QUnit.test\\(.*)\n"
+ /* don't match the function in class or in object literal, which has more than one ident level */
+ "!^(\t{2,}|[ ]{5,})\n"
+ /* match normal function in object literal */
+ "^[\t ]*(" JS_AA "*[\t ]*:[\t ]*function.*)\n"
+ /* don't match chained method call */
+ "!^[\t ]*" JS_AA "[\t ]*\\(.*\\)\\.\n"... which makes me wonder why it is not present here. If that's an
oversight: nice catch!
Quoted 15 lines
+ /* match function in class and ES5 method shorthand */
+ "^[\t ]*((static[\t ]+)?((async|get|set)[\t ]+)?" JS_AA "*[\t ]*\\(.*)",
+ /* word regex */
+ /* hexIntegerLiteral, octalIntegerLiteral, binaryIntegerLiteral, and its big version */
+ "0[xXoObB][_0-9a-fA-F]+n?"
+ /* DecimalLiteral and its big version*/
+ "|[0-9][_0-9]*(\\.[0-9][_0-9]*|n)?([eE][+-]?[_0-9]+)?"
+ "|\\.[0-9][_0-9]*([eE][+-]?[_0-9]+)?"
+ /* punctuations */
+ "|\\.{3}|<=|>=|==|!=|={3}|!==|\\*{2}|\\+{2}|--|<<|>>"
+ "|>>>|&&|\\|{2}|\\?{2}|\\+=|-=|\\*=|%=|\\*{2}="
+ "|<<=|>>=|>>>=|&=|\\|=|\\^=|&&=|\\|{2}=|\\?{2}=|=>"
+ /* identifiers */
+ "|" JS_AA "*"),Quoted 6 lines
Just a thought, I wonder how much line-noisy we could make this thing in general if we defined some common patterns with such helpers.
Anyway, insted of :alnum:and :alpha: don't you really mean [a-zA-Z0-9] and [a-zA-Z]. I.e. do you *really* want to have this different depending on the user's locale?
That's worth considering.
Quoted 4 lines
I haven't tested, but see the
LC_CTYPEingettext.c, so I'm fairly sure that'll happen...
-- Hannes
Re: [GSoC][PATCH v3] Add a diff driver for JavaScript languages.
Quoted 11 lines · Johannes Sixt
Am 04.04.22 um 09:12 schrieb Ævar Arnfjörð Bjarmason:
Quoted 4 lines
While we don't use helper macros for these currently there's no reason we can't, I thin the above might be more readable with e.g.:
#define JS_AA "[$_[:alpha:]][$_[:alnum:]]"Please consider including "identifier" somehow in the macro name. And add the trailing '*', which...
... which makes me wonder why it is not present here. If that's an
oversight: nice catch!
Yeah, and taken together with the locale worry, we shouldn't play cute and use :alpha: and :alnum:, but spell them out, i.e.
#define JS_IDENT "[$_a-zA-Z][$_a-zA-Z0-9]"
probably.
Re: [GSoC][PATCH v3] Add a diff driver for JavaScript languages.
In reply to Johannes SixtQuoted 8 lines · Johannes Sixt
Am 04.04.22 um 09:12 schrieb Ævar Arnfjörð Bjarmason:
Quoted 4 lines
While we don't use helper macros for these currently there's no reason we can't, I thin the above might be more readable with e.g.:
#define JS_AA "[$_[:alpha:]][$_[:alnum:]]"Please consider including "identifier" somehow in the macro name. And add the trailing '*', which...
Indeed, although for something like this a cute short name is probably OK, and we can just #undef it right afer.
Quoted 30 lines
Quoted 22 lines
Which would make this:
+PATTERNS("javascript", + /* don't match the expression may contain parenthesis, because it is not a function declaration */ + "!^[ \t]*(if|do|while|for|with|switch|catch|import|return)\n" + /* don't match statement */ + "!;\n" + /* match normal function or named export for function in ECMA2015 */ + "^((export[\t ]+)?(async[\t ]+)?function[\t ]*[\t *]*" JS_AA "*[\t ]*\\(.*)\n" + /* match JavaScript variable declaration with a lambda expression at top level */ + "^((const|let|var)[\t ]*" JS_AA "*[\t ]*=[\t ]*" + "(\\(.*\\)|" JS_AA "*)[\t ]*=>[\t ]*\\{?)\n" + /* match object's property assignment by anonymous function and CommonJS exports for named function */ + "^((module\\.)?" JS_AA "*\\." JS_AA "*[\t ]*=[\t ]*(async[\t ]+)?(\\(.*\\)|" JS_AA "*)[\t ]*=>.*)\n" + /* match assign function to LHS with explicit function keyword */ + "^(.*=[\t ]*function[\t ]*([$_[:alnum:]]+[\t ]*)?\\(.*)\n" + /* popular unit testing framework test case pattern. Most of framework pattern is match by regex for "function in class" */Wry try to stick to wrapping at 80 characters, so some of these comments should really be wrapped (see CodingGuidelines for the multi-line comment style we use).
Quoted 7 lines
+ "^[\t ]*(QUnit.test\\(.*)\n" + /* don't match the function in class or in object literal, which has more than one ident level */ + "!^(\t{2,}|[ ]{5,})\n" + /* match normal function in object literal */ + "^[\t ]*(" JS_AA "*[\t ]*:[\t ]*function.*)\n" + /* don't match chained method call */ + "!^[\t ]*" JS_AA "[\t ]*\\(.*\\)\\.\n"
Quoted 2 lines
... which makes me wonder why it is not present here. If that's an
oversight: nice catch!
*Nod*, I just did a dumb search replace and didn't notice that myself,
but it's clearly making things easier to read.
Asanother thing I noticed: shouldn't that '.' in QUnit.test be escaped? Presumably we don't want QUnitXtest or whatever.
Quoted 15 lines
Quoted 15 lines
+ /* match function in class and ES5 method shorthand */ + "^[\t ]*((static[\t ]+)?((async|get|set)[\t ]+)?" JS_AA "*[\t ]*\\(.*)", + /* word regex */ + /* hexIntegerLiteral, octalIntegerLiteral, binaryIntegerLiteral, and its big version */ + "0[xXoObB][_0-9a-fA-F]+n?" + /* DecimalLiteral and its big version*/ + "|[0-9][_0-9]*(\\.[0-9][_0-9]*|n)?([eE][+-]?[_0-9]+)?" + "|\\.[0-9][_0-9]*([eE][+-]?[_0-9]+)?" + /* punctuations */ + "|\\.{3}|<=|>=|==|!=|={3}|!==|\\*{2}|\\+{2}|--|<<|>>" + "|>>>|&&|\\|{2}|\\?{2}|\\+=|-=|\\*=|%=|\\*{2}=" + "|<<=|>>=|>>>=|&=|\\|=|\\^=|&&=|\\|{2}=|\\?{2}=|=>" + /* identifiers */ + "|" JS_AA "*"),
Quoted 8 lines
Quoted 6 lines
Just a thought, I wonder how much line-noisy we could make this thing in general if we defined some common patterns with such helpers.
Anyway, insted of :alnum:and :alpha: don't you really mean [a-zA-Z0-9] and [a-zA-Z]. I.e. do you *really* want to have this different depending on the user's locale?
That's worth considering.
If it's intentional it makes sense to add some locale-stressing tests to the tests, i.e. if we really mean to match non-ASCII identifiers etc.
Quoted 6 lines
Quoted 4 lines
I haven't tested, but see the
LC_CTYPEingettext.c, so I'm fairly sure that'll happen...-- Hannes
Re: [GSoC][PATCH v3] Add a diff driver for JavaScript languages.
In reply to xing zhi jiangI'm pretty happy to see follow up on this patch, I think it'll be very useful. Thanks! I think this patch has a fairly consistent direction now; though I'm undecided on whether or not I agree with the direction, I won't block this from going through.
So treat these comments as suggestions and additional JS info for the folks who actually matter :)
Quoted 6 lines · xing zhi jiang
In the xfunction part that matches normal functions,
a variable declaration with an assignment of function, the function declaration in the class, named export for function in ECMA2015, CommonJS exports for named function, and also the function is object literal's property[1]. The special match is this would match the popular framework test case pattern(most is same as regex for matching function in class).
With this paragraph and..
Quoted 1 line
2. "^[\t ]*(QUnit.test\\(.*)\n"
this pattern, it looks like the direction we've chosen to take is "support popular libraries, even if this might detect the wrong thing in vanilla JS", which sounds reasonable enough to me. I think the experience would be a bit inconsistent for vanilla JS folks, but the worst thing I can imagine is that we are a bit overeager to detect headers.
As an aside, this is my first time hearing of QUnit, but I guess the JS landscape is huge and changes too often for me to keep track.
Quoted 8 lines
I oberve there are many test case pattern in JavaScript. And most of popular JavaScript unit test framework's test case pattern is function call can be matched by the regex, which match "function in class". So it is no problem. But
QUnit frameworkdon't matched by that. Then add this regex into JavaScript userdiff.
And the reason includes test case pattern has two
1. This is inevitable. Because we can't distinguish that with function in class.
2. This can help the user understand what change is happening in which test case. And if don't match the test case, it has the large possibility to match no related function.
Nit: since we're adding a pattern just for QUnit, it would be nice to have a test for it. This would also have the nice benefit of showing why QUnit can't make use of the other patterns.
userdiff.c PATTERNS("java", · line 187
Quoted 2 lines
userdiff.c +2
-0+/* popular unit testing framework test case pattern. Most of framework pattern is match by regex for "function in class" */ +"^[\t ]*(QUnit.test\\(.*)\n"
This wasn't entirely clear to me on its own, but reading your --- description, I believe you mean that other unit testing frameworks are covered by another regex.
Wording suggestion, perhaps:
/*
* QUnit (popular unit testing framework) test case; most of the other
* frameworks are matched by "function in class".
*/
As a technicality, there is no guarantee that the user will import the QUnit library as the variable QUnit, though most users will probably use QUnit out of convention, so I don't think this a problem [1].
[1] In case some additional context becomes useful to other reviewers:
most JavaScript module systems do not enforce variable names when importing a module, so there is no guarantee that QUnit will be imported as the token 'QUnit', though this is often true by _convention_ because library authors often stick to a single name in their documentation.
In 'old school JavaScript in the browser', libraries are often exported as single global variables. Those do have well-enforced names, because otherwise the globals would be unusable.
In native ES modules (aka standardized, modern JavaScript supported by modern browsers and most frameworks), the library author can suggest a name using named exports, but with the caveats 1. the importer can alias the variable to a different name and 2. many authors use anonymous exports - either using a default export or asking users to import the entire library as a single, anonymous object.
In CommonJS (typically used in NodeJS), one cannot name exports IIRC; an export is always a single, anonymous object.