t/perf: correctly align non-ASCII descriptions in output
PATCH 1/1 t/perf: correctly align non-ASCII descriptions in output
Change the test descriptions from being treated as binary blobs by perl to being treated as UTF-8. This ensures that e.g. a test description like "æ" is counted as 1 character, not 2.
I have WIP performance tests for non-ASCII grep patterns on another topic that are affected by this.
Now instead of:
$ ./run p0000-perf-lib-sanity.sh
[...]
0000.4: export a weird var 0.00(0.00+0.00)
0000.5: éḿíẗ ńöń-ÁŚĆÍÍ ćḧáŕáćẗéŕś 0.00(0.00+0.00)
0000.7: important variables available in subshells 0.00(0.00+0.00)
[...]We emit:
[...]
0000.4: export a weird var 0.00(0.00+0.00)
0000.5: éḿíẗ ńöń-ÁŚĆÍÍ ćḧáŕáćẗéŕś 0.00(0.00+0.00)
0000.7: important variables available in subshells 0.00(0.00+0.00)
[...]Fixes code originally added in 342e9ef2d9 ("Introduce a performance testing framework", 2012-02-17).
--- t/perf/aggregate.perl | 3 +++ t/perf/p0000-perf-lib-sanity.sh | 2 ++ 2 files changed, 5 insertions(+)
t/perf/aggregate.perl +3 -0
@@ -88,6 +88,7 @@ for my $t (@tests) { sub read_descr { my $name = shift; open my $fh, "<", $name or return "<error reading description>"; + binmode $fh, ":utf8" or die "PANIC on binmode: $!"; my $line = <$fh>; close $fh or die "cannot close $name"; chomp $line;
@@ -147,6 +148,8 @@ for my $t (@subtests) { my $totalwidth = 3*@dirs+$descrlen; $totalwidth += $_ for (@colwidth); +binmode STDOUT, ":utf8" or die "PANIC on binmode: $!"; + printf "%-${descrlen}s", "Test"; for my $i (0..$#dirs) { my $d = $dirs[$i];
t/perf/p0000-perf-lib-sanity.sh +2 -0
@@ -33,6 +33,8 @@ test_perf 'export a weird var' ' test_export bar ' +test_perf 'éḿíẗ ńöń-ÁŚĆÍÍ ćḧáŕáćẗéŕś' 'true' + test_expect_success 'test_export works with weird vars' ' echo "$bar" && test "$bar" = "weird # variable"
Signature · 1 line
2.11.0
Quoted 6 lines · Ævar Arnfjörð Bjarmason
Change the test descriptions from being treated as binary blobs by perl to being treated as UTF-8. This ensures that e.g. a test description like "æ" is counted as 1 character, not 2.
I have WIP performance tests for non-ASCII grep patterns on another topic that are affected by this.
Makes sense. As this is purely about test titles in our project, choosing utf8 as the only encoding is quite sensible.
t/perf/aggregate.perl for my $t (@tests) { · line 91
Quoted 9 lines
t/perf/aggregate.perl +1
-0@@ -88,6 +88,7 @@ for my $t (@tests) { sub read_descr { my $name = shift; open my $fh, "<", $name or return "<error reading description>"; + binmode $fh, ":utf8" or die "PANIC on binmode: $!";
I thought there was some "use" flag we could set to just make all of our handles utf8. But all I could come up with was stuff like PERLIO and "perl -C". Using binmode isn't too bad, though (I think you could just do it as part of the open, too, but I'm not sure if antique versions of perl support that).
-Peff
Quoted 11 lines · Jeff King
Quoted 6 lines · Ævar Arnfjörð Bjarmason
Change the test descriptions from being treated as binary blobs by perl to being treated as UTF-8. This ensures that e.g. a test description like "æ" is counted as 1 character, not 2.
I have WIP performance tests for non-ASCII grep patterns on another topic that are affected by this.
Makes sense. As this is purely about test titles in our project, choosing utf8 as the only encoding is quite sensible.
*Nod*
Quoted 7 lines
Quoted 7 lines
t/perf/aggregate.perl +0
-0@@ -88,6 +88,7 @@ for my $t (@tests) { sub read_descr { my $name = shift;
Quoted 8 lines
t/perf/aggregate.perl for my $t (@tests) { · line 91
Quoted 2 lines
t/perf/aggregate.perl +1
-0open my $fh, "<", $name or return "<error reading description>"; + binmode $fh, ":utf8" or die "PANIC on binmode: $!";I thought there was some "use" flag we could set to just make all of our handles utf8. But all I could come up with was stuff like PERLIO and "perl -C". Using binmode isn't too bad, though (I think you could just do it as part of the open, too, but I'm not sure if antique versions of perl support that).
[Debugging perl encoding issues is one of the many perks of my dayjob]
Using binmode like this is about as straightforward as you can get, the former occurrence could be equivalently replaced by:
utf8::decode(my $line = <$fh>);
But better just to mark the handle as utf8. There's a fancier way to do it as part of the three-arg-open syntax, but I couldn't remember whether all the perl versions we support have it.
About the "use" flag, you're probably thinking of the confusingly named "use utf8", but that's to set your source code to utf8, not your handles, e.g.:
$ perl -CA -MDevel::Peek -wE 'use utf8; my $日本語 = shift; Dump $日本語' æ
SV = PV(0x12cc090) at 0x12cded8
REFCNT = 1
FLAGS = (PADMY,POK,pPOK,UTF8)
PV = 0x12de460 "\303\246"\0 [UTF8 "\x{e6}"]
CUR = 2
LEN = 16As you can see people got a bit overexcited about Unicode in the 90s.
Quoted 8 lines · Ævar Arnfjörð Bjarmason
Quoted 5 lines
I thought there was some "use" flag we could set to just make all of our handles utf8. But all I could come up with was stuff like PERLIO and "perl -C". Using binmode isn't too bad, though (I think you could just do it as part of the open, too, but I'm not sure if antique versions of perl support that).
[Debugging perl encoding issues is one of the many perks of my dayjob]
Quoted 8 lines
Using binmode like this is about as straightforward as you can get, the former occurrence could be equivalently replaced by:
utf8::decode(my $line = <$fh>);But better just to mark the handle as utf8. There's a fancier way to do it as part of the three-arg-open syntax, but I couldn't remember whether all the perl versions we support have it.
Yeah, I agree marking the handle is better. binmode is pretty straightforward, but we'd have to remember to manually set it if we add any other handles. That's probably not a big deal in this particular script, though, which is pretty short.
Quoted 12 lines
About the "use" flag, you're probably thinking of the confusingly named "use utf8", but that's to set your source code to utf8, not your handles, e.g.:
$ perl -CA -MDevel::Peek -wE 'use utf8; my $日本語 = shift; Dump $日本語' æ SV = PV(0x12cc090) at 0x12cded8 REFCNT = 1 FLAGS = (PADMY,POK,pPOK,UTF8) PV = 0x12de460 "\303\246"\0 [UTF8 "\x{e6}"] CUR = 2 LEN = 16
Quoted 1 line
As you can see people got a bit overexcited about Unicode in the 90s.
Yeah, I know "use utf8" doesn't work for that, but I was thinking there was some other trick. Digging...ah, here it is:
use open ':encoding(utf8)'
No clue how portable that is. For such a small script it may be better to just stick with vanilla binmode().
-Peff
Quoted 38 lines · Jeff King
Quoted 8 lines · Ævar Arnfjörð Bjarmason
Quoted 5 lines
I thought there was some "use" flag we could set to just make all of our handles utf8. But all I could come up with was stuff like PERLIO and "perl -C". Using binmode isn't too bad, though (I think you could just do it as part of the open, too, but I'm not sure if antique versions of perl support that).
[Debugging perl encoding issues is one of the many perks of my dayjob]
Quoted 8 lines
Using binmode like this is about as straightforward as you can get, the former occurrence could be equivalently replaced by:
utf8::decode(my $line = <$fh>);But better just to mark the handle as utf8. There's a fancier way to do it as part of the three-arg-open syntax, but I couldn't remember whether all the perl versions we support have it.
Yeah, I agree marking the handle is better. binmode is pretty straightforward, but we'd have to remember to manually set it if we add any other handles. That's probably not a big deal in this particular script, though, which is pretty short.
Quoted 12 lines
About the "use" flag, you're probably thinking of the confusingly named "use utf8", but that's to set your source code to utf8, not your handles, e.g.:
$ perl -CA -MDevel::Peek -wE 'use utf8; my $日本語 = shift; Dump $日本語' æ SV = PV(0x12cc090) at 0x12cded8 REFCNT = 1 FLAGS = (PADMY,POK,pPOK,UTF8) PV = 0x12de460 "\303\246"\0 [UTF8 "\x{e6}"] CUR = 2 LEN = 16Quoted 1 line
As you can see people got a bit overexcited about Unicode in the 90s.
Quoted 7 lines
Yeah, I know "use utf8" doesn't work for that, but I was thinking there was some other trick. Digging...ah, here it is:
use open ':encoding(utf8)'
No clue how portable that is. For such a small script it may be better to just stick with vanilla
binmode().
Yeah that would work, but doesn't work on 5.8.0, which is the lowest version we support.
Quoted 10 lines · Ævar Arnfjörð Bjarmason
Quoted 7 lines
Yeah, I know "use utf8" doesn't work for that, but I was thinking there was some other trick. Digging...ah, here it is:
use open ':encoding(utf8)'
No clue how portable that is. For such a small script it may be better to just stick with vanilla
binmode().Yeah that would work, but doesn't work on 5.8.0, which is the lowest version we support.
Thanks for looking that up.
For the benefit of the maintainer, I'll summarize: Ævar's original patch is fine as-is.
-Peff