AI review agent
Hello Everyone,
it has been a long journey, but we can happily announce that our AI review agent is up and running.
This is something new also for us. We worked hard in the past years to keep the quality of tests as high as possible with a tiny team of ~4 people actively reviewing patches.
And despite the number of patches per day, which is still around 10 per day, the amount of work kept increasing. Everyone has a job, private life and sometimes it's hard to keep up reviewing kernel tests.
For this reason, we started to explore the possibility of using LLMs for reviewing LTP patches, ending up with an agent that was good enough to review most of the common errors and reducing workload on maintainers shoulders.
Hopefully, this will help developers to write high quality patches, without feeling left behind in the review process that might be slow sometimes.
There are still improvements to be made, but the reviews are pretty good at the moment, with a minimum amount of hallucinations.
You can find the source code of the agent at the following link:
https://github.com/acerv/ltp-agent
Best Regards,
--
Andrea Cervesato
SUSE QE Automation Engineer Linux
[email protected]
Signature · 1 line
Mailing list info: https://lists.linux.it/listinfo/ltp
Hi!
Quoted 2 lines
it has been a long journey, but we can happily announce that our AI review agent is up and running.
Thanks a lot!
Signature · 5 lines
Cyril Hrubis [email protected] -- Mailing list info: https://lists.linux.it/listinfo/ltp
Hi Andrea,
Thanks for the nice AI review tool.
I noticed that each automated AI review email starts with the following disclaimer:
Quoted 4 lines
Our agent completed the review of the patch.
The agent can sometimes produce false positives although often its findings are genuine. If you find issues with the review, please comment this email or ignore the suggestions.
From my view, it consumes a little bit of energy to read or scroll past this note at the beginning of every email.
Maybe we could just put it at the bottom of the email as a note?
For example:
[... AI Patch Review ...]
Note: Our agent completed the review of this patch. The agent can sometimes produce false positives, although often its findings are genuine. If you find issues
with the review, please reply to this email or ignore the suggestions.
I think this would make the automated emails much easier to read at a glance.
Signature · 5 lines
Regards, Li Wang -- Mailing list info: https://lists.linux.it/listinfo/ltp
Hi Li,
Thanks!
Quoted 8 lines
Note: Our agent completed the review of this patch. The agent can sometimes produce false positives, although often its findings are genuine. If you find issues
with the review, please reply to this email or ignore the suggestions.I think this would make the automated emails much easier to read at a glance.
It's a good idea, feel free to send a PR on https://github.com/acerv/ltp-agent
Regards,
--
Andrea Cervesato
SUSE QE Automation Engineer Linux
[email protected]
Signature · 1 line
Mailing list info: https://lists.linux.it/listinfo/ltp
Quoted 1 line
It's a good idea, feel free to send a PR on https://github.com/acerv/ltp-agent
PR opened: https://github.com/acerv/ltp-agent/pull/2/
Signature · 5 lines
Regards, Li Wang -- Mailing list info: https://lists.linux.it/listinfo/ltp
Quoted 11 lines · Li Wang
Hi Andrea,
Thanks for the nice AI review tool.
I noticed that each automated AI review email starts with the following disclaimer:Quoted 4 lines
Our agent completed the review of the patch.
The agent can sometimes produce false positives although often its findings are genuine. If you find issues with the review, please comment this email or ignore the suggestions.
Quoted 4 lines
From my view, it consumes a little bit of energy to read or scroll
past this note at the beginning of every email.Maybe we could just put it at the bottom of the email as a note?
Hi,
I think the e-mail should still start with some short notice that the review was AI generated, with the longer disclaimer at the bottom. For example:
Hi $sender,
this patch review was generated by AI. See disclaimer at the bottom.
[... AI Patch Review ...]
Disclaimer: Our agent completed the review of this patch. The agent can sometimes produce false positives, although often its findings are genuine. If you find issues with the review, please reply to this email or ignore the suggestions.
Signature · 10 lines
Martin Doucha [email protected] SW Quality Engineer SUSE LINUX, s.r.o. CORSO IIa Krizikova 148/34 186 00 Prague 8 Czech Republic -- Mailing list info: https://lists.linux.it/listinfo/ltp
Quoted 12 lines · Martin Doucha
Quoted 11 lines · Li Wang
Hi Andrea,
Thanks for the nice AI review tool.
I noticed that each automated AI review email starts with the following disclaimer:Quoted 4 lines
Our agent completed the review of the patch.
The agent can sometimes produce false positives although often its findings are genuine. If you find issues with the review, please comment this email or ignore the suggestions.
Quoted 8 lines
Quoted 4 lines
From my view, it consumes a little bit of energy to read or scroll
past this note at the beginning of every email.Maybe we could just put it at the bottom of the email as a note?
Hi,
I think the e-mail should still start with some short notice that the review was AI generated, with the longer disclaimer at the bottom. For example:
Maybe yes, I see Andrea already merged the PR, so let's see how it looks, if necessary, then we can follow up this suggestion. Thanks!
Signature · 5 lines
Regards, Li Wang -- Mailing list info: https://lists.linux.it/listinfo/ltp
The repo has been moved under the linux-test-project ownership:
https://github.com/linux-test-project/ltp-agent
Kind regards,
--
Andrea Cervesato
SUSE QE Automation Engineer Linux
[email protected]
Signature · 1 line
Mailing list info: https://lists.linux.it/listinfo/ltp