Skip to contentSkip to conversation

gitweb: Make feed title valid utf8

PATCH In review Jürgen Kreileder 2 participants 6 messages Jun 15, 2016

lore
Share & tools
lore
#0 lore raw

PATCH 2/4 gitweb: Make feed title valid utf8

Properly encode site and project names for RSS and Atom feeds.

---
 gitweb/gitweb.perl |    2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
gitweb/gitweb.perl +1 -1
index 9cfe5b5..09294eb 100755
@@ -8056,7 +8056,7 @@ sub git_feed {
 	return if ($cgi->request_method() eq 'HEAD');
 
 	# header variables
-	my $title = "$site_name - $project/$action";
+	my $title = to_utf8($site_name) . " - " . to_utf8($project) . "/$action";
 	my $feed_type = 'log';
 	if (defined $hash) {
 		$title .= " - '$hash'";
Signature · 1 line
1.7.10.4
#1 lore raw
Quoted 13 lines · Jürgen Kreileder

Properly encode site and project names for RSS and Atom feeds.

---
 gitweb/gitweb.perl |    2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
gitweb/gitweb.perl +0 -0
index 9cfe5b5..09294eb 100755

gitweb/gitweb.perl sub git_feed { · line 8062

Quoted 8 lines
gitweb/gitweb.perl +1 -1
@@ -8056,7 +8056,7 @@ sub git_feed {
 	return if ($cgi->request_method() eq 'HEAD');
 
 	# header variables
-	my $title = "$site_name - $project/$action";
+	my $title = to_utf8($site_name) . " - " . to_utf8($project) . "/$action";
 	my $feed_type = 'log';
 	if (defined $hash) {
 		$title .= " - '$hash'";

Was this patch triggered by some bug?

Because the above is not necessary, as git_feed() has

	$title = esc_html($title);

a bit later, which does to_utf8() internally.

Signature · 1 line
Jakub Narębski
#2 lore raw
Quoted 16 lines · Jakub Narębski
Quoted 14 lines · Jürgen Kreileder

Properly encode site and project names for RSS and Atom feeds.

---
 gitweb/gitweb.perl |    2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
gitweb/gitweb.perl +0 -0
index 9cfe5b5..09294eb 100755
@@ -8056,7 +8056,7 @@ sub git_feed {
 	return if ($cgi->request_method() eq 'HEAD');
 
Quoted 9 lines

gitweb/gitweb.perl sub git_feed { · line 8062

Quoted 7 lines
gitweb/gitweb.perl +1 -1
 	# header variables
-	my $title = "$site_name - $project/$action";
+	my $title = to_utf8($site_name) . " - " . to_utf8($project) . "/$action";
 	my $feed_type = 'log';
 	if (defined $hash) {
 		$title .= " - '$hash'";

Was this patch triggered by some bug?

Yes, I actually see broken encoding with the old code, e.g on
https://git.blackdown.de/old.cgi?p=contactalbum.git;a=rss
my first name is messed up in the title tag.

New version: https://git.blackdown.de/?p=contactalbum.git;a=rss

Quoted 5 lines

Because the above is not necessary, as git_feed() has

	$title = esc_html($title);

a bit later, which does to_utf8() internally.

Good point. But it doesn't fix the string in question:
It looks like to_utf8("$a $b") != (to_utf8($a) . " " . to_utf($b)).

Juergen

#3 lore raw
Quoted 16 lines · Jürgen Kreileder
Quoted 8 lines · Jakub Narębski

gitweb/gitweb.perl sub git_feed { · line 8059

Quoted 4 lines · Jürgen Kreileder

Properly encode site and project names for RSS and Atom feeds.

gitweb/gitweb.perl +1 -1
-	my $title = "$site_name - $project/$action";
+	my $title = to_utf8($site_name) . " - " . to_utf8($project) . "/$action";

Was this patch triggered by some bug?

Yes, I actually see broken encoding with the old code, e.g on
https://git.blackdown.de/old.cgi?p=contactalbum.git;a=rss
my first name is messed up in the title tag.

New version: https://git.blackdown.de/?p=contactalbum.git;a=rss

Quoted 8 lines
Quoted 5 lines

Because the above is not necessary, as git_feed() has

	$title = esc_html($title);

a bit later, which does to_utf8() internally.

Good point. But it doesn't fix the string in question:
It looks like to_utf8("$a $b") != (to_utf8($a) . " " . to_utf8($b)).

Strange. I wonder if the bug is in our to_utf8() implementation, or in Encode, or in Perl... and whether this bug can be triggered anywhere else in gitweb.

What Perl version and Encode module version do you use?

Signature · 1 line
Jakub Narębski
#4 lore raw
Quoted 21 lines · Jakub Narębski
Quoted 20 lines · Jürgen Kreileder
Quoted 8 lines · Jakub Narębski

gitweb/gitweb.perl sub git_feed { · line 8059

Quoted 4 lines · Jürgen Kreileder

Properly encode site and project names for RSS and Atom feeds.

gitweb/gitweb.perl +1 -1
-	my $title = "$site_name - $project/$action";
+	my $title = to_utf8($site_name) . " - " . to_utf8($project) . "/$action";

Was this patch triggered by some bug?

Yes, I actually see broken encoding with the old code, e.g on
https://git.blackdown.de/old.cgi?p=contactalbum.git;a=rss
my first name is messed up in the title tag.

New version: https://git.blackdown.de/?p=contactalbum.git;a=rss

Quoted 4 lines

Because the above is not necessary, as git_feed() has

	$title = esc_html($title);
Quoted 8 lines
Quoted 4 lines
Quoted 1 line

a bit later, which does to_utf8() internally.

Good point. But it doesn't fix the string in question:
It looks like to_utf8("$a $b") != (to_utf8($a) . " " . to_utf8($b)).

Strange. I wonder if the bug is in our to_utf8() implementation, or in Encode, or in Perl... and whether this bug can be triggered anywhere else in gitweb.

I don't think it's a bug, more like a consequence of concatenating utf8 and non-utf8 strings:

    my $a = "ü";
    my $b = "ü";
    my $c = "$a - $b";
    print "$c -> ". to_utf8($c) . ": " . (utf8::is_utf8($c) ? "utf8" : "not utf8") . "\n"; # GOOD
    $b = to_utf8($b);
    $c = "$a - $b";
    print "$c -> ". to_utf8($c) . ": " . (utf8::is_utf8($c) ? "utf8" : "not utf8") . "\n"; # GOOD

yields (hopefully the broken encoding shows up correctly here):

    ü - ü -> ü - ü: not utf8
    ü - ü -> ü - ü: utf8

In gitweb we have the bad case:

   my $title = "$site_name - $project/$action";

$project and $action are apparently utf8 already but $site_name isn't. The resulting string is marked as utf8 - although the encoding of $site_name was never fixed. The to_utf8() in esc_html() returns the string without fixing anything because of that.

Quoted 1 line

What Perl version and Encode module version do you use?

5.14.2 and 2.42_01 on Ubuntu. Same results with 5.12.4 and 2.39 on OS X.

Juergen

#5 lore raw
Quoted 25 lines · Jürgen Kreileder
Quoted 4 lines · Jakub Narębski
Quoted 3 lines · Jürgen Kreileder
Quoted 2 lines · Jakub Narębski

Jürgen Kreileder wrote:

Quoted 8 lines
Quoted 4 lines
Quoted 1 line
Quoted 1 line

Properly encode site and project names for RSS and Atom feeds.

Good point. But it doesn't fix the string in question:
It looks like to_utf8("$a $b") != (to_utf8($a) . " " . to_utf8($b)).

Strange. I wonder if the bug is in our to_utf8() implementation, or in Encode, or in Perl... and whether this bug can be triggered anywhere else in gitweb.

I don't think it's a bug, more like a consequence of concatenating utf8 and non-utf8 strings:

    my $a = "ü";
    my $b = "ü";
    my $c = "$a - $b";
    print "$c -> ". to_utf8($c) . ": " . (utf8::is_utf8($c) ? "utf8" : "not utf8") . "\n"; # GOOD
    $b = to_utf8($b);
    $c = "$a - $b";
    print "$c -> ". to_utf8($c) . ": " . (utf8::is_utf8($c) ? "utf8" : "not utf8") . "\n"; # GOOD
Quoted 4 lines

yields (hopefully the broken encoding shows up correctly here):

    ü - ü -> ü - ü: not utf8
    ü - ü -> ü - ü: utf8

Ah, so it looks like it is misfeature of the way Perl handles Unicode; concatenating adds 'UTF8' flag if either of concatenates strings has it to the result.

[Which I have checked using Devel::Peek with
perl -MDevel::Peek -E '
my $a = "ż"; my $b = "\x{17c}";
Dump $a; Dump $b; Dump "$b - $a"'
]

Quoted 5 lines

In gitweb we have the bad case:

   my $title = "$site_name - $project/$action";

$project and $action are apparently utf8 already but $site_name isn't.

$project and $action are taken from URL, and we have to run decode_utf8 (at least for query params) for gitweb to work correctly.

$site_name is usually taken from config file, and gitweb doesn't have "use utf8" pragma.

Quoted 3 lines

The resulting string is marked as utf8 - although the encoding of $site_name was never fixed. The to_utf8() in esc_html() returns the string without fixing anything because of that.

O.K.

_Maybe_ it would be worth adding explanation of this to commit message (and I see I should audit gitweb for similar problems elsewhere), but anyway

Signature · 1 line
Jakub Narębski