Skip to contentSkip to conversation

CFT: imaps://public-inbox.org/INBOX.comp.version-control.git.7

DISCUSSION Eric Wong 2 participants 6 messages Jun 10, 2020 – Jun 23, 2020

lore
Share & tools
lore

Eric Wong

author
#0 lore raw

Hey all, I've been busy with public-inbox work this year. I started working on a read-only IMAP/IMAPS server, and I expect kernel.org to eventually host a v2 copy, too; and hopefully other folks get interested in mirroring.

For now, you can access git@vger archives through 8 IMAP mailboxes sliced into ~50K messages to not overload clients. They go from 0..7, with 7 being the most recent:

imaps://public-inbox.org/INBOX.comp.version-control.git.0 imaps://public-inbox.org/INBOX.comp.version-control.git.1 imaps://public-inbox.org/INBOX.comp.version-control.git.2 imaps://public-inbox.org/INBOX.comp.version-control.git.3 imaps://public-inbox.org/INBOX.comp.version-control.git.4 imaps://public-inbox.org/INBOX.comp.version-control.git.5 imaps://public-inbox.org/INBOX.comp.version-control.git.6 imaps://public-inbox.org/INBOX.comp.version-control.git.7

When more messages show up, imaps://public-inbox.org/INBOX.comp.version-control.git.8 should exist.

I've tested with mutt, offlineimap and mbsync (isync).

It's not yet complete (when is software ever?), but it's a usable start, I think. Some more info about it over on [email protected]:
https://public-inbox.org/meta/20200609113442.GA16856@dcvr/
https://public-inbox.org/meta/[email protected]/
And maybe JMAP is coming, too...

#1 lore raw

Hi Eric,

Quoted 22 lines · Eric Wong

Hey all, I've been busy with public-inbox work this year. I started working on a read-only IMAP/IMAPS server, and I expect kernel.org to eventually host a v2 copy, too; and hopefully other folks get interested in mirroring.

For now, you can access git@vger archives through 8 IMAP mailboxes sliced into ~50K messages to not overload clients. They go from 0..7, with 7 being the most recent:

imaps://public-inbox.org/INBOX.comp.version-control.git.0 imaps://public-inbox.org/INBOX.comp.version-control.git.1 imaps://public-inbox.org/INBOX.comp.version-control.git.2 imaps://public-inbox.org/INBOX.comp.version-control.git.3 imaps://public-inbox.org/INBOX.comp.version-control.git.4 imaps://public-inbox.org/INBOX.comp.version-control.git.5 imaps://public-inbox.org/INBOX.comp.version-control.git.6 imaps://public-inbox.org/INBOX.comp.version-control.git.7

When more messages show up, imaps://public-inbox.org/INBOX.comp.version-control.git.8 should exist.

Quoted 8 lines

I've tested with mutt, offlineimap and mbsync (isync).

It's not yet complete (when is software ever?), but it's a usable start, I think. Some more info about it over on [email protected]:
https://public-inbox.org/meta/20200609113442.GA16856@dcvr/
https://public-inbox.org/meta/[email protected]/
And maybe JMAP is coming, too...

Exciting news! Since I'm not actually subscribed to the mailing list, public-inbox has been integral in my workflow by letting me reply to messages that I'm not CC'd on. The IMAP gateway is a great development.

I fetched your message via IMAP just now ;)

-Denton

author
#2 lore raw
Quoted 5 lines · Denton Liu

Exciting news! Since I'm not actually subscribed to the mailing list, public-inbox has been integral in my workflow by letting me reply to messages that I'm not CC'd on. The IMAP gateway is a great development.

I fetched your message via IMAP just now ;)

Hi Denton, great to know you appreciate it :)

It's still in the early stages and there still seems to be a problem where it still interacts badly with mutt's header cache. Maybe checking over the old HTTPS/NNTPS endpoints once in a while if things appear too quiet is a good idea while I shake out some bugs.

I'm dreading the cost of ~100K RAM/storage overhead per-client connection to support sequence numbers properly...

author
#3 lore raw
Quoted 9 lines · Eric Wong
Quoted 5 lines · Denton Liu

Exciting news! Since I'm not actually subscribed to the mailing list, public-inbox has been integral in my workflow by letting me reply to messages that I'm not CC'd on. The IMAP gateway is a great development.

I fetched your message via IMAP just now ;)

Hi Denton, great to know you appreciate it :)

Quoted 8 lines

It's still in the early stages and there still seems to be a problem where it still interacts badly with mutt's header cache. Maybe checking over the old HTTPS/NNTPS endpoints once in a while if things appear too quiet is a good idea while I shake out some bugs.

I'm dreading the cost of ~100K RAM/storage overhead per-client connection to support sequence numbers properly...

OK, that ought to be fixed and I found what seems to be an acceptable solution w.r.t. server-side overhead:
https://public-inbox.org/meta/20200612234924.GA31809@dcvr/

I've also just deployed a recursive descent parser for which is hopefully compliant with the way IMAP search queries are parsed:
https://public-inbox.org/meta/[email protected]/

One of the annoyances of read-only IMAP is MUAs (unlike newsreaders) lack the ability to store status flags for new/unread messages. I've got some ideas to deal with that on the client-side...

#4 lore raw

Hi Eric,

Quoted 7 lines · Eric Wong
Quoted 2 lines · Eric Wong

I'm dreading the cost of ~100K RAM/storage overhead per-client connection to support sequence numbers properly...

OK, that ought to be fixed and I found what seems to be an acceptable solution w.r.t. server-side overhead:
https://public-inbox.org/meta/20200612234924.GA31809@dcvr/

Aha, I noticed last night when I was using it that it seemed much snappier! I can also confirm on my end that your patch works and it makes the experience much better.

Thanks again!

author
#5 lore raw

CFT: imaps://public-inbox.org/INBOX.comp.version-control.git.8

In reply to Eric Wong

+Cc: Christian for Git Rev News 64

Quoted 2 lines · Eric Wong

When more messages show up, imaps://public-inbox.org/INBOX.comp.version-control.git.8

They've showed up, and .8 now exists, bringing to total up to 9.