tip: New "Link:" tag to replace "LKML-Reference:"
Hi all,
At Linus' request, I have modified the tip-bot to recognize a new form of email archive references, which should work as clickable links. Instead of:
LKML-Reference: <message-id>
use, instead:
The entire bit "Link: http://lkml.kernel.org/r/" is a literal; the rest is the message-id from the original message, *without* angle brackets. This points to a redirection service on kernel.org, which currently bounces to marc.info, but which we can redirect in the future.
I changed the name from LKML-Reference: to Link: to make the width of the string tolerable, and since the link itself contains an explicit reference to the specific archive used.
Please note that message-ids sometimes contain characters which have to be escaped to fit in valid URI syntax! The characters @ and $ which are very common in message-ids do not, however, need to be escaped in this particular context (even though they are not part of the URI safe character set, and so most library functions will escape them.) The characters # ; & % as well as whitespace are definitely unsafe either for use in an URI or for programs to recognize them as part of one, and should definitely be escaped.
The tip-bot will recognize the *exact* form above (including URI escaping) and should do the message threading correctly. To keep people from injecting malicious links, however, there is sanity filtering on the unescaped content, so in theory a legal-but-weird message-id might be rejected.
-hpa
Quoted 30 lines · H. Peter Anvin
Hi all,
At Linus' request, I have modified the tip-bot to recognize a new form of email archive references, which should work as clickable links. Instead of:
LKML-Reference: <message-id>
use, instead:
The entire bit "Link: http://lkml.kernel.org/r/" is a literal; the rest is the message-id from the original message, *without* angle brackets. This points to a redirection service on kernel.org, which currently bounces to marc.info, but which we can redirect in the future.
I changed the name from LKML-Reference: to Link: to make the width of the string tolerable, and since the link itself contains an explicit reference to the specific archive used.
Please note that message-ids sometimes contain characters which have to be escaped to fit in valid URI syntax! The characters @ and $ which are very common in message-ids do not, however, need to be escaped in this particular context (even though they are not part of the URI safe character set, and so most library functions will escape them.) The characters # ; & % as well as whitespace are definitely unsafe either for use in an URI or for programs to recognize them as part of one, and should definitely be escaped.
Quoted 5 lines
The tip-bot will recognize the *exact* form above (including URI escaping) and should do the message threading correctly. To keep people from injecting malicious links, however, there is sanity filtering on the unescaped content, so in theory a legal-but-weird message-id might be rejected.
OK, so I fixed my scripts to match this and it all seems to work, except for the:
LKML-Reference: <new-submission>
case, where the tip-bot would mail it out for me. While I try not to use it too much its convenient for quick fixes etc. Will the absence of a Link tag suffice or is there more to it?
Quoted 10 lines · Peter Zijlstra
OK, so I fixed my scripts to match this and it all seems to work, except for the:
LKML-Reference: <new-submission>
case, where the tip-bot would mail it out for me. While I try not to use it too much its convenient for quick fixes etc. Will the absence of a Link tag suffice or is there more to it?
OK, I have asked for a long time that we do not use <new-submission>.
The recommended pattern has been:
LKML-Reference: <tip-*@git.kernel.org>
... which at least indicates to the user how the message-ID is to be generated (* being a placeholder for the commit SHA1).
The only think I can think of for how to make Link: work would be to recognize a pattern that the tip-bot would use as its own Message-ID *instead* of the commit SHA1 pattern that it would normally use. There are disadvantages to every approach, of course.
We have a couple of alternatives:
a) Continue to use "LKML-Reference: <tip-*@git.kernel.org>"
+ At least gives a hint how to find the newly-formed LKML thread
- Not a clickable link
- Somewhat noisy
b) Just Cc: everything to LKML regardless of tag
+ Easy
- No way to *not* post everything to LKML
- No way to locate the thread without knowing the magic
c) A recognizable pattern with a unique pattern *generated by the committer*, something like:
+ Clickable link
- Long (the above is 86 characters wide)
- Message-IDs of a nonstandard form
- Bad things will happen if someone re-uses an identifier
- String has to be generated by machine at commit time
[Using ranpwd instead of uuidgen allows for a more compact random string by using a wider character set. For example:
... has the same entropy but is 74 characters wide.]
Thoughts?
-hpa
Signature · 2 lines
H. Peter Anvin, Intel Open Source Technology Center I work for Intel. I don't speak on their behalf.
Quoted 12 lines · H. Peter Anvin
Quoted 10 lines · Peter Zijlstra
OK, so I fixed my scripts to match this and it all seems to work, except for the:
LKML-Reference: <new-submission>
case, where the tip-bot would mail it out for me. While I try not to use it too much its convenient for quick fixes etc. Will the absence of a Link tag suffice or is there more to it?
Quoted 8 lines
OK, I have asked for a long time that we do not use <new-submission>.
The recommended pattern has been:
LKML-Reference: <tip-*@git.kernel.org>
... which at least indicates to the user how the message-ID is to be generated (* being a placeholder for the commit SHA1).
Oh, I never heard of that, it would've been easy to make my scripts do that if that were the preferred form, a well.
Quoted 24 lines
The only think I can think of for how to make Link: work would be to recognize a pattern that the tip-bot would use as its own Message-ID *instead* of the commit SHA1 pattern that it would normally use. There are disadvantages to every approach, of course.
We have a couple of alternatives:
a) Continue to use "LKML-Reference: <tip-*@git.kernel.org>"
+ At least gives a hint how to find the newly-formed LKML thread
- Not a clickable link
- Somewhat noisy
b) Just Cc: everything to LKML regardless of tag
+ Easy
- No way to *not* post everything to LKML
- No way to locate the thread without knowing the magic
c) A recognizable pattern with a unique pattern *generated by the committer*, something like:Link http://lkml.kernel.org/r/tip-`uuidgen -r`@git.kernel.org+ Clickable link
- Long (the above is 86 characters wide)
- Message-IDs of a nonstandard form
- Bad things will happen if someone re-uses an identifier
- String has to be generated by machine at commit time
Quoted 6 lines
[Using ranpwd instead of uuidgen allows for a more compact random string by using a wider character set. For example:
Link http://lkml.kernel.org/r/tip-`ranpwd -l 24`@git.kernel.org... has the same entropy but is 74 characters wide.]
Can't we use a prepare-commit-msg or commit-msg hook to re-write the commit message to include a Link tag when one is missing, using something like:
And have the post-commit hook that sends the messages recognize this form and send the message using the correct msgid.
That way the commit includes a correct and clickable link, doesn't require magic knowledge and doesn't need the committer to do extra work.
Quoted 5 lines · Peter Zijlstra
Can't we use a prepare-commit-msg or commit-msg hook to re-write the commit message to include a Link tag when one is missing, using something like:
Quoted 8 lines
And have the post-commit hook that sends the messages recognize this form and send the message using the correct msgid.
That way the commit includes a correct and clickable link, doesn't require magic knowledge and doesn't need the committer to do extra work.
Think about it... hint: the SHA1 is a cryptographically secure signature of the commit object.
-hpa
Signature · 2 lines
H. Peter Anvin, Intel Open Source Technology Center I work for Intel. I don't speak on their behalf.
Quoted 11 lines · H. Peter Anvin
Quoted 10 lines · Peter Zijlstra
Can't we use a prepare-commit-msg or commit-msg hook to re-write the commit message to include a Link tag when one is missing, using something like:
And have the post-commit hook that sends the messages recognize this form and send the message using the correct msgid.
Quoted 6 lines
Quoted 3 lines
That way the commit includes a correct and clickable link, doesn't require magic knowledge and doesn't need the committer to do extra work.
Think about it... hint: the SHA1 is a cryptographically secure signature of the commit object.
D'0h ok.. but the scheme would work with any generated msgid, all we need is to generate one for commits that lack a Link tag and insert it using the prepare-commit-msg/commit-msg hooks and for the post-commit hook to recognize it and use that actual msgid.
The only thing to avoid is msgid collision, but given that MUAs already have this problem all we need to do is borrow some of their logic.
Quoted 5 lines · Peter Zijlstra
Can't we use a prepare-commit-msg or commit-msg hook to re-write the commit message to include a Link tag when one is missing, using something like:
Quoted 8 lines
And have the post-commit hook that sends the messages recognize this form and send the message using the correct msgid.
That way the commit includes a correct and clickable link, doesn't require magic knowledge and doesn't need the committer to do extra work.
To summarize the problem I hinted at earlier:
1. The link that ends up in the commit message needs to be unique, or it is not going to be a working future-clickable link.
2. The link that ends up in the commit message CANNOT use the SHA1 of the commit itself, since the commit SHA1 is derived from the entire commit object including the message;
3. As a result, the only option is for the robot to change the Message-ID to something other than the SHA1 of the commit. It is a bit unfortunate, but probably not disastrous.
-hpa
Signature · 2 lines
H. Peter Anvin, Intel Open Source Technology Center I work for Intel. I don't speak on their behalf.
Quoted 9 lines · Peter Zijlstra
D'0h ok.. but the scheme would work with any generated msgid, all we need is to generate one for commits that lack a Link tag and insert it using the prepare-commit-msg/commit-msg hooks and for the post-commit hook to recognize it and use that actual msgid.
The only thing to avoid is msgid collision, but given that MUAs already have this problem all we need to do is borrow some of their logic.
OK, this doesn't really need any hooks at all (the tip-bot is not a hook). The committer is, and *has to be* responsible for getting the unique string into the Link: tag, through whatever means they feel is appropriate.
Based on the previous message, I'm going to modify the bot to recognize the pattern:
... to override the Message-ID. It seems like the least ugly of a lot of ugly options.
-hpa
Signature · 2 lines
H. Peter Anvin, Intel Open Source Technology Center I work for Intel. I don't speak on their behalf.
Quoted 6 lines · H. Peter Anvin
OK, this doesn't really need any hooks at all (the tip-bot is not a hook). The committer is, and *has to be* responsible for getting the unique string into the Link: tag, through whatever means they feel is appropriate.
Quoted 8 lines
Based on the previous message, I'm going to modify the bot to recognize
the pattern:Link http://lkml.kernel.org/r/tip-`ranpwd -l 24`@git.kernel.org... to override the Message-ID. It seems like the least ugly of a lot of ugly options.
OK, but to qualify, it needn't be the committer, that is, Ingo usually commits the stuff for me, but I usually provide him with complete patches including LKML-Reference, now Link, tag.
I'll change my script to:
Use an existing Link tag provided it doesn't match the above pattern, if there's no such tag, use Message-id, lacking that, generate one using the above pattern.
Thanks!
Quoted 11 lines · H. Peter Anvin
Hi all,
At Linus' request, I have modified the tip-bot to recognize a new form of email archive references, which should work as clickable links. Instead of:
LKML-Reference: <message-id>
use, instead:
If use of this grows outside plain linux-kernel development a more generic name could make sense.
Suggestion:
http://mid.kernel.org/r/message-id
mid for message-id.
Sam
Quoted 10 lines · Sam Ravnborg
If use of this grows outside plain linux-kernel development a more generic name could make sense.
Suggestion:
http://mid.kernel.org/r/message-id
mid for message-id.
I explicitly did not want to do that, since I believe it is important information "which mailing list does this come from". Otherwise we may need a database on kernel.org knowing exactly which mailing list any particular message-id belongs to.
-hpa
Signature · 2 lines
H. Peter Anvin, Intel Open Source Technology Center I work for Intel. I don't speak on their behalf.
Quoted 11 lines · H. Peter Anvin
Hi all,
At Linus' request, I have modified the tip-bot to recognize a new form of email archive references, which should work as clickable links. Instead of:
LKML-Reference: <message-id>
use, instead:
Really cool idea. I assume you'll mention this in Documentation/SubmittingPatches or somewhere similar.
Bjorn
Quoted 7 lines · Bjorn Helgaas
Quoted 6 lines · H. Peter Anvin
Hi all,
At Linus' request, I have modified the tip-bot to recognize a new form of email archive references, which should work as clickable links. Instead of:
Quoted 9 lines
Quoted 5 lines
LKML-Reference: <message-id>
use, instead:
Really cool idea. I assume you'll mention this in
Documentation/SubmittingPatchesor somewhere similar.
Well, it is better done by the committers rather than the patch submitters, so it's mostly an issue for people using git as opposed to submitting patches by email.
-hpa
Link: tags for new submission -- UPDATE
In reply to H. Peter AnvinQuoted 16 lines · H. Peter Anvin
Quoted 9 lines · Peter Zijlstra
D'0h ok.. but the scheme would work with any generated msgid, all we need is to generate one for commits that lack a Link tag and insert it using the prepare-commit-msg/commit-msg hooks and for the post-commit hook to recognize it and use that actual msgid.
The only thing to avoid is msgid collision, but given that MUAs already have this problem all we need to do is borrow some of their logic.
OK, this doesn't really need any hooks at all (the tip-bot is not a hook). The committer is, and *has to be* responsible for getting the unique string into the Link: tag, through whatever means they feel is appropriate.
Quoted 8 lines
Based on the previous message, I'm going to modify the bot to recognize the pattern:
Link http://lkml.kernel.org/r/tip-`ranpwd -l 24`@git.kernel.org... to override the Message-ID. It seems like the least ugly of a lot of ugly options.
<<<< IMPORTANT CHANGE >>>>
Please use the format:
^
... with an "n" for "new" instead of "r" for "reference" instead.
This allows the bot to distinguish a "new submission" link from a *reference* to a message that was created as a result of a new submission!
-hpa