Skip to contentSkip to conversation

DAMON Yearly Retrospect (2024)

DISCUSSION SeongJae Park 1 participant 1 message Feb 16

Subsystems (2)
lore
Share & tools
lore

SeongJae Park

author maintainer
#0 lore raw

After DAMON has upstreamed in 2021, I shared yearly retrospects for 2022 [1] and 2023 [2]. And somehow I forgot to do that for 2024 and 2025. Let me share the retrospect of 2024, before my memory goes away.

Summary
=======

2024 was a year that DAMON has been more publicly and widely adopted. AWS published their usage of DAMON as a VLDB paper. SK hynix developed and upstreamed DAMON features for memory tiering. Debian kernel team has enabled DAMON build config on their kernels. Citations of DAMON papers have also increased from 10 in 2023 to 17 in 2024.

DAMON's status and future plans have presented and discussed on multiple conferences, including OSSummit NA, LSF/MM/BPF, OSSummit EU, and LPC.

Number of development participants and commits have reduced compared to 2023, though. Hopefully that reflects how DAMON continues being stabilized.

Events
======

In January, Linux 6.8-rc1 has released, with feedback-driven DAMOS aggressiveness auto-tuning [3] feature. The feature allows users simply feed DAMOS rewards for current aggressiveness, and DAMOS can automatically tune the aggressiveness to convince the user.

In March, Linux 6.9-rc1 has released, with DAMOS aggressiveness self-tuning [4] feature. It makes the aggressiveness auto-tuning can work itself, without user's feedback.

In April, DAMON was presented [4] at OSSummit North America. The presentation was focusing on the feedback-driven DAMOS auto/self-tuning.

In May, DAMON updates and future plans were discussed [5] at LSF/MM/BPF. The plan was including DAMON-based memory tiering and holistic access-aware memory autoscaling for cloud providers.

In July, Linux 6.11-rc1 has released, with DAMON changes [6] for CXL memory tiering. The change is developed by SK hynix for their heterogeneous memory software development kit.

In July, AWS published a paper [7] introducing their DAMON usage as a part of it, at VLDB. The paper introduces AWS Aurora Serverless V2, which uses DAMON as its core part of memory auto-scaling. Specifically, they use DAMON for proactively reclaiming cold pages inside guests.

In August, DAMON project repositories on GitHub has moved [8] from awslabs organization to damonitor organization.

In September, DAMON use cases were presented [9] at OSSummit EU. The talk was mainly covering DAMON usages from AWS and SK hynix. Honggyu Kim, an SK hynix engineer who developed DAMON-based memory tiering, has shared their use case on the talk.

In September, DAMON long term development plan was presented [10] at LPC Kernel Memory Management Micro-conference. The plan was about how DAMON will be flexible but also just works.

In September, Debian enabled [11] DAMON build configuration on their kernels.

In December, DAMON monitoring intervals tuning guide has published [12]. The guide includes the monitoring and tuning results on real world server workloads.

Features
========

In 2024, a few new DAMON features have landed on the mainline. Among those, below two features are noticeable to me.

DAMOS aggressiveness auto-tuning feature have landed in two parts. The first part [3] on v6.8, and the second part [4] on v6.9. This feature makes access aware system operations nearly self-driving.

DAMON changes for CXL memory tiering has developed by SK hynix and landed [6] on v6.11. The feature allows users to setup DAMOS for migrating hot and cold pages for appropriate tiers.

Statistics
==========

In 2024, 20 people made 157 commits for DAMON on the mainline. I got the numbers with name of the contributors using my script [13] as below.

    $ ./lazybox/version_control/authors.py ./linux --skip_merge_commits \
            --since 2024-01-01 --until 2024-12-31 --linux_subsystems DAMON
    1. SeongJae Park <[email protected]>: 133 commits
    2. Zheng Yejian <[email protected]>: 2 commits
    3. Anna-Maria Behnsen <[email protected]>: 2 commits
    4. Honggyu Kim <[email protected]>: 2 commits
    5. Hyeongtak Ji <[email protected]>: 2 commits
    6. Barry Song <[email protected]>: 2 commits
    7. Akinobu Mita <[email protected]>: 1 commits
    8. Maximilian Heyne <[email protected]>: 1 commits
    9. Andrew Paniakin <[email protected]>: 1 commits
    10. James Houghton <[email protected]>: 1 commits
    11. Ba Jing <[email protected]>: 1 commits
    12. Leo Stone <[email protected]>: 1 commits
    13. Jinjie Ruan <[email protected]>: 1 commits
    14. Diederik de Haas <[email protected]>: 1 commits
    15. Liam R. Howlett <[email protected]>: 1 commits
    16. Peng Hao <[email protected]>: 1 commits
    17. Christophe Leroy <[email protected]>: 1 commits
    18. Alex Rusuf <[email protected]>: 1 commits
    19. Javier Carrasco <[email protected]>: 1 commits
    20. Vincenzo Mezzela <[email protected]>: 1 commits
    # 20 authors, 157 commits in total

Huge appreciation to all the developers. It was a grateful year to work with you.

The last line of the output for 2023 was like below:

    $ ./lazybox/version_control/authors.py ./linux --skip_merge_commits \
            --since 2023-01-01 --until 2023-12-31 --linux_subsystems DAMON
    [...]
    # 25 authors, 180 commits in total

So, both the number of developers (25 -> 20) and commits (180 -> 157) have reduced. Hopefully this means that DAMON is becoming more stabilized and reliable.

DAMON User-space Tool
---------------------

DAMON user-space tool is also an important part of DAMON project. In 2024, eight people made 1,031 commits for DAMON user-space tool. The list of the developers are as below:

    $ ./lazybox/version_control/authors.py ./damo --skip_merge_commits \
            --since 2024-01-01 --until 2024-12-31
    1. SeongJae Park <[email protected]>: 753 commits
    2. SeongJae Park <[email protected]>: 264 commits
    3. Mithun Veluri <[email protected]>: 4 commits
    4. Honggyu Kim <[email protected]>: 3 commits
    5. m8 <[email protected]>: 2 commits
    6. Yunjeong Mun <[email protected]>: 2 commits
    7. Akinobu Mita <[email protected]>: 1 commits
    8. Alex Rusuf <[email protected]>: 1 commits
    9. Piyush Thange <[email protected]>: 1 commits
    # 9 authors, 1031 commits in total

Note that I was counted twice due to different email address usages.

The tool's output for 2023 was like below:

    $ ./lazybox/version_control/authors.py ./damo --skip_merge_commits \
            --since 2023-01-01 --until 2023-12-31
    1. SeongJae Park <[email protected]>: 1793 commits
    2. SeongJae Park <[email protected]>: 58 commits
    3. Honggyu Kim <[email protected]>: 8 commits
    4. sjpark <[email protected]>: 5 commits
    5. Michel Alexandre Salim <[email protected]>: 4 commits
    6. SeongJae Park <[email protected]>: 3 commits
    7. Honggyu Kim <[email protected]>: 2 commits
    8. Andrew Paniakin <[email protected]>: 1 commits
    9. fdu <[email protected]>: 1 commits
    10. Puranjay Mohan <[email protected]>: 1 commits
    11. Puranjay Mohan <[email protected]>: 1 commits
    # 11 authors, 1877 commits in total

Note that I was counted four times, and Puranjay was counted twice. So, seven people made 1,877 commits for DAMON user-space tool in 2023. The number of contributors have increased, while the number of total commits have quite reduced. The reduction of the commits was mainly from my side. I was making ~1,800 commits in 2023, but only ~1,000 commits in 2024. I actually feel DAMON user-space tool is being more stabilized. I also spent quite amount of my time that I used to use for DAMON user-space tool for another tool, hkml [16].

Paper Citations
---------------

Yet another interesting statistics about the number of DAMON paper citations. I published two academic papers introducing DAMON in 2019 [14] and 2022 [15].

The number of citations for the two papers have quite increased, as below.

2019-published paper: 6 (2023) -> 10 (2024)
2022-published paper: 4 (2023) -> 7 (2024)

Conclusion
==========

So, 2024 was a year DAMON has more publicly and widely adopted, and extended itself for more use cases.

References
==========

[1] 2022 retrospect: https://lore.kernel.org/[email protected]/
[2] 2023 retrospect: https://lore.kernel.org/[email protected]/
[3] DAMOS auto-tune: https://git.kernel.org/torvalds/c/9294a037c01564786
[4] DAMOS self-tune: https://git.kernel.org/torvalds/c/78f2f60377ee43b7f
[5] OSSummit NA talk: https://ossna2024.sched.com/event/1aBOg
[6] DAMOS memory-tiering: https://git.kernel.org/torvalds/c/a00ce85af2a1be49
[7] VLDB AWS paper: https://www.amazon.science/publications/resource-management-in-aurora-serverless
[8] awslabs to damonitor GitHub repos move: https://lore.kernel.org/all/[email protected]/
[9] OSSummit EU talk: https://osseu2024.sched.com/event/1ej2S
[10] LPC talk: https://lpc.events/event/18/contributions/1768/
[11] Debian DAMON enablement: https://salsa.debian.org/kernel-team/linux/-/merge_requests/1172
[12] DAMON tuning guide: https://lore.kernel.org/all/[email protected]/
[13] https://github.com/sjp38/lazybox/blob/master/version_control/authors.py
[14] 2019 DAMON paper: https://dl.acm.org/doi/abs/10.1145/3366626.3368125
[15] 2022 DAMON paper: https://dl.acm.org/doi/abs/10.1145/3502181.3531466
[16] hkml: https://github.com/sjp38/hackermail