extra search flags and params? (ispatch, replycount, ...)
Would they be useful?
It's not currently possible to quickly search for whether or not a term (e.g. patchid:) is present in a Xapian document. Having the ability to do so would make it easier to find non-patch messages, or easily filter down to cover letters, bot replies, etc...
Thus adding Xapian terms is probably necessary for that.
replycount would be useful for implementing search for ANSWERED/UNANSWERED in IMAP. However, proper indexing/reindex could be a PITA due to out-of-order message reception.
I don't think any of these would be required to get "lei rediff" working on entire patchsets, though (it only does individual messages, currently).
Quoted 6 lines · Eric Wong
Would they be useful?
It's not currently possible to quickly search for whether or not a term (e.g. patchid:) is present in a Xapian document. Having the ability to do so would make it easier to find non-patch messages, or easily filter down to cover letters, bot replies, etc...
I understand the reasoning, but I'm not sure we should be trying too hard to make public-inbox a patch tracking platform. What makes lei great is ability to automatically find and retrieve entire threads -- I feel like we should leave series tracking to other platforms that already exist (patchwork, patchew, etc).
Quoted 3 lines
I don't think any of these would be required to get "lei rediff" working on entire patchsets, though (it only does individual messages, currently).
Incidentally, I've recently discovered that relying on git-patch-id to match commits to message archives has some important flaws. Linus was actually the one who caused this when he recommended that maintainers switch to using the "histogram" diff algorithm instead of the default ("myers").
This made me realize that there's actually a multitude of ways the same patch can be represented (diff-algorithm, number of context lines, etc) that would cause git-patch-id to return a different value for the exact same commit.
So, while I know that Linus doesn't want Link: entries in commits that just go to the series, using the message-id remains the only mechanism to reliably link commits to the series discussion.
-K
Quoted 8 lines · Konstantin Ryabitsev
Quoted 6 lines · Eric Wong
Would they be useful?
It's not currently possible to quickly search for whether or not a term (e.g. patchid:) is present in a Xapian document. Having the ability to do so would make it easier to find non-patch messages, or easily filter down to cover letters, bot replies, etc...
Quoted 5 lines
I understand the reasoning, but I'm not sure we should be trying too hard to make public-inbox a patch tracking platform. What makes lei great is ability to automatically find and retrieve entire threads -- I feel like we should leave series tracking to other platforms that already exist (patchwork, patchew, etc).
I was thinking more along the lines of readers just trying to find trying to find non-patch discussions. I'm not really interested in the tracking part, more just being able to quickly find discussion related to a commit.
Quoted 8 lines
Quoted 3 lines
I don't think any of these would be required to get "lei rediff" working on entire patchsets, though (it only does individual messages, currently).
Incidentally, I've recently discovered that relying on git-patch-id to match commits to message archives has some important flaws. Linus was actually the one who caused this when he recommended that maintainers switch to using the "histogram" diff algorithm instead of the default ("myers").
Yeah, -cindex was actually built to support joins on pre or post-image blob OIDs, too, just need to clamp to a 7 char hex abbreviation. Even Subjects <=> commit titles could be made to work with the way our indices are setup.
Quoted 3 lines
This made me realize that there's actually a multitude of ways the same patch can be represented (diff-algorithm, number of context lines, etc) that would cause git-patch-id to return a different value for the exact same commit.
Yeah, post-image blob abbreviations are probably the way to go.
Fwiw, solver only uses post-image blob abbreviations and the filename as a hint. I rolled it out a few hours ago on yhbt.net/lore and it seems to be solving kernel blobs just fine, but the debug log is choosing random git URLs.
(Solver is the thing that powers `lei rediff' and the linkified hunk headers on public-inbox.org/git since 2019, and now yhbt.net/lore)
Quoted 8 lines · Eric Wong
Quoted 5 lines
I understand the reasoning, but I'm not sure we should be trying too hard to make public-inbox a patch tracking platform. What makes lei great is ability to automatically find and retrieve entire threads -- I feel like we should leave series tracking to other platforms that already exist (patchwork, patchew, etc).
I was thinking more along the lines of readers just trying to find trying to find non-patch discussions.
Ah. I think here is enough to just say "s:* AND NOT s:PATCH" without introducing additional xapian indexing parameters. Though, perhaps the web interface can also gain a "collapse threads" view?
Quoted 10 lines
Quoted 3 lines
This made me realize that there's actually a multitude of ways the same patch can be represented (diff-algorithm, number of context lines, etc) that would cause git-patch-id to return a different value for the exact same commit.
Yeah, post-image blob abbreviations are probably the way to go.
Fwiw, solver only uses post-image blob abbreviations and the filename as a hint. I rolled it out a few hours ago on yhbt.net/lore and it seems to be solving kernel blobs just fine, but the debug log is choosing random git URLs.
Ah, neat! That said, what happens if a series was applied with "git am -3" and the post-image blob abbreviations are necessarily different? (I may be misunderstanding the approach, please correct me if I do.)
-K
Quoted 7 lines · Konstantin Ryabitsev
Quoted 6 lines · Eric Wong
Quoted 5 lines
I understand the reasoning, but I'm not sure we should be trying too hard to make public-inbox a patch tracking platform. What makes lei great is ability to automatically find and retrieve entire threads -- I feel like we should leave series tracking to other platforms that already exist (patchwork, patchew, etc).
Quoted 6 lines
Quoted 2 lines
I was thinking more along the lines of readers just trying to find trying to find non-patch discussions.
Ah. I think here is enough to just say "s:* AND NOT s:PATCH" without introducing additional xapian indexing parameters. Though, perhaps the web interface can also gain a "collapse threads" view?
topics_new.html / topics_active.html endpoints? Also, '&t' is a weird accident that happens to work:
https://yhbt.net/lore/git/?q=s:*+AND+NOT+s:PATCH&t
I suppose that's OK for the majority of cases.
Though being able to find unanswered threads could be helpful.
Quoted 6 lines
Quoted 6 lines
Quoted 3 lines
This made me realize that there's actually a multitude of ways the same patch can be represented (diff-algorithm, number of context lines, etc) that would cause git-patch-id to return a different value for the exact same commit.
Yeah, post-image blob abbreviations are probably the way to go.
Quoted 8 lines
Quoted 4 lines
Fwiw, solver only uses post-image blob abbreviations and the filename as a hint. I rolled it out a few hours ago on yhbt.net/lore and it seems to be solving kernel blobs just fine, but the debug log is choosing random git URLs.
Ah, neat! That said, what happens if a series was applied with "git am -3" and the post-image blob abbreviations are necessarily different? (I may be misunderstanding the approach, please correct me if I do.)
The resulting blob will be retrievable directly via git once it's pushed to a mirrored repo. solver tries to look for a blob in git, first, before trying to search+apply patches[1].
Right now I'm only clamping to 3 coderepos for performance reasons, but that limitation is going away. But I need a nap, first :x
[1] solver could reverse the order to look for the mails first if we wanted to see the fullest email history of a blob, but it's much slower because it has to do a lot more work (hitting Xapian and runing git apply+ls-files for every single patch).
Quoted 10 lines · Eric Wong
Quoted 3 lines
Ah. I think here is enough to just say "s:* AND NOT s:PATCH" without introducing additional xapian indexing parameters. Though, perhaps the web interface can also gain a "collapse threads" view?
topics_new.html / topics_active.html endpoints? Also, '&t' is a weird accident that happens to work:
https://yhbt.net/lore/git/?q=s:*+AND+NOT+s:PATCH&t
I suppose that's OK for the majority of cases.
Nice!
Quoted 1 line
Though being able to find unanswered threads could be helpful.
Note, I'm not saying it's not a cool feature. :) However, I imagine people would be more interested in searching for something like "show me all threads mentioning $foo to which *I* haven't replied yet". It's not quite the same thing as "nobody has replied yet."
I have no idea how hard this would be.
-K
Quoted 9 lines · Konstantin Ryabitsev
Quoted 1 line · Eric Wong
Though being able to find unanswered threads could be helpful.
Note, I'm not saying it's not a cool feature. :) However, I imagine people would be more interested in searching for something like "show me all threads mentioning $foo to which *I* haven't replied yet". It's not quite the same thing as "nobody has replied yet."
I have no idea how hard this would be.
I think notmuch can do it:
https://public-inbox.org/meta/20230412000654.M9394@dcvr/
https://public-inbox.org/meta/[email protected]/
So can we, since I've started using C++ in xap_helper and will introduce RLIMIT_* support soonish to keep performance in check. Going to see how much punishment the ancient (but fanless :>) CPU on yhbt.net/lore can handle...
Quoted 7 lines · Konstantin Ryabitsev
Quoted 5 lines · Eric Wong
Quoted 5 lines
I understand the reasoning, but I'm not sure we should be trying too hard to make public-inbox a patch tracking platform. What makes lei great is ability to automatically find and retrieve entire threads -- I feel like we should leave series tracking to other platforms that already exist (patchwork, patchew, etc).
patch tracking platforms might want to use public-inbox to get the patches in the first place.
Quoted 3 lines
Quoted 3 lines
I was thinking more along the lines of readers just trying to find trying to find non-patch discussions.
I do this time to time to find things I miss. Since I have patch tracking, I don't miss patches.
Quoted 3 lines
Ah. I think here is enough to just say "s:* AND NOT s:PATCH" without introducing additional xapian indexing parameters. Though, perhaps the web interface can also gain a "collapse threads" view?
There's also [RFC 1/N], [PATCHv5], or just [vN], so:
"s:* AND NOT (s:PATCH OR s:RFC OR s:v1 OR s:v2 OR s:v3...)"
But when someone does "[RFC] Things I want to discuss" or "blah blah patch blah blah" it won't work. It's fragile and inexact.
We already have find certain patches with dfn:, why not find all patches?
Rob