netdev development stats for 7.0 (and some AI review thoughts)

Hi!

Intro
-----

As is tradition here are the development statistics based on mailing list traffic on netdev@vger.

These stats are somewhat like LWN stats: https://lwn.net/Articles/1004998/
but more focused on mailing list participation. And by participation we mean reviewing code more than producing patches.

In particular "review score" tries to capture the balance between reviewing other people's code vs posting patches. It's roughly number of patches reviewed minus number of patches posted. Those who post more than they review will have a negative score.

Previous 3 reports:
- for 6.17: https://lore.kernel.org/[email protected]
- for 6.18: https://lore.kernel.org/[email protected]
- for 6.19: https://lore.kernel.org/[email protected]

General stats
-------------

This was a significantly slower development cycle compared to the previous one for us, with 14.5% less ML traffic and 20.9% fewer commits. The ratio of cross-company review has slipped by 2%. The lower traffic and commit volume can be partially explained by Christmas / winter break, linux-next as a whole was also 13.2% smaller at the beginning of this merge window compared to the previous one.

AI reviews
----------

Notably this was the first cycle in which we had AI code reviews integrated fully and executed on every submission. I hope the slight decline in human reviews isn't associated with AI integration. Discouraging human participation is my main concern when it comes to the AI reviews. The data supports the intuition that it is now harder to get code merged (since AI catches more issues). The average number of revisions for a series went up from 1.96 to 2.24. That said (the average number of revisions for a single-patch postings dropped slightly from 1.37 to 1.32.)

The AI integration has gone relatively smoothly from my perspective. Of course there are some false positives and cases where AI sends submitters down the wrong path. On balance, however, AI catches a lot of valid issues (especially in dreaded error paths). This is in no small part due to work Chris Mason continues to invest into improving the AI prompts and workflow, and to netdev maintainers who try to pre-screen the reviews. Eyeballing the internal graphs we have in NIPA the AI reviews are completely wrong or not worth reporting about 13% of the time.

The main point of frustration with the AI reviews so far has been that they are nondeterministic. AI may find new or different issues on each posting. My understanding is that this is largely a question of efficiency. AI can invent infinite potential issues to investigate, and it explores them in somewhat random order. So each time we run the review a different set of problems gets explored. While we can't change the "order of exploration", optimizing the efficiency of the agent lets us broaden the set of problems for every single review. Chris has reworked the agent over the last month significantly, to use sub-agents and to prepare the context of review a priori, without the agent having to go back and forth fetching information. This helps a lot for large code submissions.

Looking into the future I'd particularly like the AI reviews to focus on validation of how interfaces are used or implemented, on top of catching coding mistakes (and typos) which it does today. Until now the agents seem to rarely consult Documentation/ or even kdoc of invoked functions. I'm hoping that the evolution of these systems will let us invest more time in improving Documentation/ now that something (even if not someone) will finally read it. The future of writing instructions for bots ("CLAUDE.md") _in addition_ to documentation for humans is not particularly appealing. Chris has already started working on pushing the AI workflow more towards consulting Documentation/. We hope to see results soon.

Testing
-------

The percentage of commits adding selftest has crept up slightly but still remains under 10%.

Top contributors of selftests:
1 [ 15] Jakub Kicinski
2 [ 13] Matthieu Baerts

   3 [  9] Bobby Eshleman
   4 [  7] Xu Du
   5 [  5] Felix Maurer
   6 [  5] David Wei
   7 [  5] Ido Schimmel
   8 [  4] Gal Pressman
   9 [  4] Victor Nogueira
  10 [  3] Daniel Zahka

One more note on testing, we have completed the migration of NIPA to netdev foundation machines. Due to DRAM shortage, however, the shipment of additional machines is delayed.

Developer rankings
------------------

Top reviewers (cs): Top reviewers (msg):

   1 (   ) [27] Jakub Kicinski          1 (   ) [69] Jakub Kicinski      
   2 (   ) [15] Simon Horman            2 (   ) [28] Andrew Lunn         
   3 (   ) [14] Andrew Lunn             3 (   ) [24] Simon Horman        
   4 (   ) [11] Paolo Abeni             4 (   ) [17] Paolo Abeni         
   5 ( +1) [ 6] Eric Dumazet            5 ( +2) [13] Aleksandr Loktionov 
   6 ( +2) [ 4] Aleksandr Loktionov     6 ( +2) [12] Eric Dumazet        
   7 (   ) [ 4] Russell King            7 ( -1) [12] Russell King        
   8 ( +9) [ 4] Willem de Bruijn        8 (+11) [10] Willem de Bruijn    
   9 ( +7) [ 3] Rob Herring             9 ( +7) [ 9] Vladimir Oltean     
  10 ( +2) [ 3] Vadim Fedorenko        10 ( -1) [ 9] Michael S. Tsirkin  
  11 ( +3) [ 3] Vladimir Oltean        11 ( -6) [ 6] Maxime Chevallier   
  12 ( -7) [ 3] Maxime Chevallier      12 ( +8) [ 6] Rob Herring         
  13 ( -4) [ 3] Jacob Keller           13 ( -3) [ 6] Stefano Garzarella  
  14 ( +1) [ 3] Paul Menzel            14 (+18) [ 5] Krzysztof Kozlowski 
  15 (+13) [ 2] Krzysztof Kozlowski    15 ( -2) [ 5] Kuniyuki Iwashima   

Slight fluctuation within the top reviewer rankings. Most of the names should be familiar. In addition to the maintainers, we have Aleksandr and Jake from Intel reviewing Ethernet drivers. Russell, Maxime and Vladimir focusing on embedded networking. Willem and Kuniyuki reviewing core code. Rob and Krzysztof reviewing device tree changes. And Stefano reviewing vsock patches. Paul Menzel reviews Intel driver patches. Huge thanks to those who help with patch reviews!

Top authors (cs): Top authors (msg):

   1 (   ) [8] Eric Dumazet             1 ( +1) [20] Russell King        
   2 (***) [4] Ethan Nelson-Moore       2 ( +5) [16] Daniel Golle        
   3 (   ) [3] Jakub Kicinski           3 (***) [15] Ratheesh Kannoth    
   4 ( -2) [3] Russell King             4 (+21) [15] Chia-Yu Chang       
   5 (+30) [3] David Yang               5 ( +3) [15] Eric Dumazet        
   6 ( -2) [2] Tariq Toukan             6 (***) [13] Menglong Dong       
   7 (+27) [2] Daniel Golle             7 ( -4) [12] Tariq Toukan        
   8 ( -3) [1] Heiner Kallweit          8 (+15) [12] Wei Fang            
   9 ( +3) [1] Lorenzo Bianconi         9 (***) [10] Ivan Vecera         

10 (+26) [1] Raju Rangoju 10 (+21) [10] Frederic Weisbecker

Top scores (positive): Top scores (negative):

   1 (   ) [462] Jakub Kicinski         1 (***) [60] Ratheesh Kannoth    
   2 ( +1) [212] Andrew Lunn            2 (+12) [59] Chia-Yu Chang       
   3 ( -1) [205] Simon Horman           3 ( +2) [54] Daniel Golle        
   4 (   ) [133] Paolo Abeni            4 (***) [51] Menglong Dong       
   5 (   ) [ 78] Aleksandr Loktionov    5 (+18) [43] Wei Fang            
   6 ( +3) [ 64] Willem de Bruijn       6 (+12) [41] Frederic Weisbecker 
   7 ( +1) [ 53] Rob Herring            7 (+45) [41] Jacky Chou          

Frequent re-posters of large series dominate the negative review score axis. Ratheesh reposting a new (copy of an) Octeon driver. Chia-Yu Chang submitted AccECN patches which were successfully merged. Daniel Golle contributes a lot of small embedded series (OpenWRT?).

Company rankings
----------------

Top reviewers (cs): Top reviewers (msg):

   1 ( +1) [31] Meta                    1 ( +1) [88] Meta                
   2 ( -1) [29] RedHat                  2 ( -1) [71] RedHat              
   3 (   ) [14] Intel                   3 (   ) [42] Intel               
   4 (   ) [14] Andrew Lunn             4 ( +1) [29] Google              
   5 (   ) [10] Google                  5 ( -1) [28] Andrew Lunn         
   6 (   ) [ 8] nVidia                  6 (   ) [20] nVidia              
   7 (   ) [ 6] Oracle                  7 (   ) [15] Oracle              

Top authors (cs): Top authors (msg):

   1 ( +2) [12] Google                  1 ( +1) [69] RedHat              
   2 (   ) [ 9] RedHat                  2 ( -1) [40] Meta                
   3 ( -2) [ 8] Meta                    3 (   ) [35] nVidia              
   4 ( +2) [ 5] nVidia                  4 ( +2) [28] Google              
   5 ( -1) [ 4] Intel                   5 ( +2) [28] Oracle              
   6 (***) [ 4] Ethan Nelson-Moore      6 ( +3) [23] NXP                 
   7 ( -2) [ 4] Oracle                  7 ( +3) [23] Huawei              

Top scores (positive): Top scores (negative):

   1 ( +2) [433] Meta                   1 ( +3) [71] Huawei              
   2 ( -1) [218] RedHat                 2 ( +8) [59] Nokia               
   3 ( -1) [212] Andrew Lunn            3 (+22) [56] Marvell             
   4 (   ) [162] Intel                  4 ( -2) [54] Daniel Golle        
   5 ( +2) [ 79] Google                 5 (***) [52] Tencent             
   6 ( -1) [ 55] ARM                    6 (+14) [45] Isovalent           
   7 ( +4) [ 34] Max-Planck             7 (***) [45] Kylin Software      

Huawei, Nokia and Marvell are in top 3 biggest net-consumers of reviews. Nokia was contributing the AccECN support so perhaps it's a temporary spike. The other two are constant on the negative side.

Code: https://github.com/kuba-moo/ml-stat
Raw output: https://netdev.bots.linux.dev/ml-stats/stats-7.0

<snip>

<snip>

Just something I noticed that might be connected to this is that the links to "Raw output" in these announcements broke for each previous message you posted.

It is possible to use the new link convention/format:

Raw output: https://netdev.bots.linux.dev/ml-stats/stats-7.0

to go back as far as 6.18 stats e.g.:

	https://netdev.bots.linux.dev/ml-stats/stats-6.18

but beyond that (6.17 and older) it doesn't work. The "old" nipa links:

Raw output: https://netdev.bots.linux.dev/static/nipa/stats-6.19/stdout

are completely broken.

P.S. I am using a freshly setup email client on a new machine so apologies if the formatting is wrong on my end. I did send this message to myself for testing and I think it's okay, but I might be missing something :)

Kind regards,
Dawid

Yup, this is due to the NIPA migration from Meta systems to netdev foundation machines. I can regenerate the outputs if you need them (modulo the JSON format changing since), LMK if and how far back.

I did copy the "time based" stats from old machines (3mo, 6mo etc) FWIW:
https://netdev.bots.linux.dev/ml-stats/
which I suspect to be more useful for actionable tracking of activity than the release-by-release. Release stats are mostly an excuse to send an email to the list and thank people :)

Looks fine :)

I don't need them, just noticed the gap and reported it, as I thought you might've not been aware.

Understood, in that case my response above still holds true. For myself I don't need anything more, but other folks might want/need them for some purpose. If they do I guess they will let you know one way or another :)

Thanks for confirming!
Dawid