Skip to contentSkip to conversation

AI review agent

LTP Andrea Cervesato via ltp 4 participants 8 messages Apr 13 – Apr 14

lore
Share & tools
lore
#0 lore raw

Hello Everyone,

it has been a long journey, but we can happily announce that our AI review agent is up and running.

This is something new also for us. We worked hard in the past years to keep the quality of tests as high as possible with a tiny team of ~4 people actively reviewing patches.

And despite the number of patches per day, which is still around 10 per day, the amount of work kept increasing. Everyone has a job, private life and sometimes it's hard to keep up reviewing kernel tests.

For this reason, we started to explore the possibility of using LLMs for reviewing LTP patches, ending up with an agent that was good enough to review most of the common errors and reducing workload on maintainers shoulders.

Hopefully, this will help developers to write high quality patches, without feeling left behind in the review process that might be slow sometimes.

There are still improvements to be made, but the reviews are pretty good at the moment, with a minimum amount of hallucinations.

You can find the source code of the agent at the following link:

https://github.com/acerv/ltp-agent

Best Regards,
--
Andrea Cervesato
SUSE QE Automation Engineer Linux
[email protected]

Signature · 1 line
Mailing list info: https://lists.linux.it/listinfo/ltp
#1 lore raw

Hi!

Quoted 2 lines

it has been a long journey, but we can happily announce that our AI review agent is up and running.

Thanks a lot!

Signature · 5 lines
Cyril Hrubis
[email protected]

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp
#2 lore raw
In reply to Andrea Cervesato via ltp

Hi Andrea,

Thanks for the nice AI review tool.
I noticed that each automated AI review email starts with the following disclaimer:

Quoted 4 lines

Our agent completed the review of the patch.
The agent can sometimes produce false positives although often its findings are genuine. If you find issues with the review, please comment this email or ignore the suggestions.

From my view, it consumes a little bit of energy to read or scroll past this note at the beginning of every email.

Maybe we could just put it at the bottom of the email as a note?

For example:

[... AI Patch Review ...]

Note: Our agent completed the review of this patch. The agent can sometimes produce false positives, although often its findings are genuine. If you find issues
with the review, please reply to this email or ignore the suggestions.

I think this would make the automated emails much easier to read at a glance.

Signature · 5 lines
Regards,
Li Wang

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp
#3 lore raw

Hi Li,

Thanks!

Quoted 8 lines

Note: Our agent completed the review of this patch. The agent can sometimes produce false positives, although often its findings are genuine. If you find issues
with the review, please reply to this email or ignore the suggestions.

I think this would make the automated emails much easier to read at a glance.

It's a good idea, feel free to send a PR on https://github.com/acerv/ltp-agent

Regards,
--
Andrea Cervesato
SUSE QE Automation Engineer Linux
[email protected]

Signature · 1 line
Mailing list info: https://lists.linux.it/listinfo/ltp
#5 lore raw
In reply to Li Wang
Quoted 11 lines · Li Wang

Hi Andrea,

Thanks for the nice AI review tool.
I noticed that each automated AI review email starts with the following disclaimer:

Quoted 4 lines

Our agent completed the review of the patch.
The agent can sometimes produce false positives although often its findings are genuine. If you find issues with the review, please comment this email or ignore the suggestions.

Quoted 4 lines

From my view, it consumes a little bit of energy to read or scroll
past this note at the beginning of every email.

Maybe we could just put it at the bottom of the email as a note?

Hi,
I think the e-mail should still start with some short notice that the review was AI generated, with the longer disclaimer at the bottom. For example:

Hi $sender,
this patch review was generated by AI. See disclaimer at the bottom.

[... AI Patch Review ...]

Disclaimer: Our agent completed the review of this patch. The agent can sometimes produce false positives, although often its findings are genuine. If you find issues with the review, please reply to this email or ignore the suggestions.

Signature · 10 lines
Martin Doucha   [email protected]
SW Quality Engineer
SUSE LINUX, s.r.o.
CORSO IIa
Krizikova 148/34
186 00 Prague 8
Czech Republic

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp
#6 lore raw
Quoted 12 lines · Martin Doucha
Quoted 11 lines · Li Wang

Hi Andrea,

Thanks for the nice AI review tool.
I noticed that each automated AI review email starts with the following disclaimer:

Quoted 4 lines

Our agent completed the review of the patch.
The agent can sometimes produce false positives although often its findings are genuine. If you find issues with the review, please comment this email or ignore the suggestions.

Quoted 8 lines
Quoted 4 lines

From my view, it consumes a little bit of energy to read or scroll
past this note at the beginning of every email.

Maybe we could just put it at the bottom of the email as a note?

Hi,
I think the e-mail should still start with some short notice that the review was AI generated, with the longer disclaimer at the bottom. For example:

Maybe yes, I see Andrea already merged the PR, so let's see how it looks, if necessary, then we can follow up this suggestion. Thanks!

Signature · 5 lines
Regards,
Li Wang

-- 
Mailing list info: https://lists.linux.it/listinfo/ltp