Skip to contentSkip to conversation

A few plans for DAMON sysfs extensions

RFC SeongJae Park 3 participants 5 messages Jun 2, 2022 – Jul 21, 2022 DAMON

lore
Share & tools
lore

SeongJae Park

author maintainer
#0 lore raw

Hello all,

I have discussed with several people about how they want to use DAMON for their specific use cases. For supporting those in perfectly optimal way, we might need to make yet another user interface. However, IMHO, almost all of the use cases could be covered by extending DAMON sysfs interface[1]. Hence, I'd like to share my rough and humble ideas here. It would be great if you could give some comments.

[1] https://docs.kernel.org/admin-guide/mm/damon/usage.html#sysfs-interface

Non-root Users
--------------

Someone told me they want to use DAMON from non-root user, for the user's processes. My DAMON sysfs interface extension idea for this usage is adding a special directory under the DAMON sysfs root (/sys/kernel/mm/damon/) as a sibling of the admin/ directory. The name would be, say, non-root-users/ (please blame my bad naming sense and suggest something better). The non-root users who want to use DAMON would be able to populate a directory under the directory. Then, the non-root user populated directory will have its files hierarchy same to that of the admin/ directory. The files will work same to those under admin/ directory but fails when the monitoring target is out of the user's permission. An example usage would be something like below:

    $ ls /sys/kernel/mm/damon
    admin non-root-users
    $ cd /sys/kernel/mm/damon/non-root-users
    $ mkdir for_my_damon_usage; cd for_my_damon_usage/kdamonds/
    $ echo 1 > nr_kdamonds; cd 0/
    $ echo 1 > /contexts/nr_contexts
    $ echo vaddr > 0/contexts/0/operations
    $ echo 1 > 0/contexts/0/targets/nr_targets
    $ echo $(pidof my_process) > 0/contexts/0/targets/0/pid_target
    $ echo on > state

The files tree would be something like below:

/sys/kernel/mm/damon/
│ non-root-users/ <- This is the new directory
│ │ for_my_damon_usage/
│ │ │ kdamonds/nr_kdamonds
│ │ │ │ 0/state,pid
│ │ │ ...
│ admin/
│ │ kdamonds/nr_kdamonds
│ │ │ 0/state,pid
│ │ │ │ contexts/nr_contexts
│ │ │ │ │ 0/avail_operations,operations
│ │ │ │ │ │ monitoring_attrs/
│ │ │ │ │ │ │ intervals/sample_us,aggr_us,update_us │ │ │ │ │ │ │ nr_regions/min,max
│ │ │ │ │ │ targets/nr_targets
│ │ │ │ │ │ │ 0/pid_target
│ │ │ │ │ │ │ │ regions/nr_regions
│ │ │ │ │ │ │ │ │ 0/start,end
│ │ │ │ │ │ │ │ │ ...
│ │ │ │ │ │ │ ...
│ │ │ │ │ │ ...
│ │ │ │ │ ...
│ │ │ ...

No kdamond
----------

Currently, DAMON creates worker kernel threads called kdamond and does requested works on those. There was a request asking the work to be done on the requester's process for easier resource usage charging. For this, we could add a new directory under the user-specific DAMON sysfs root directory, called, say, self-service/ (again, blame my bad naming sense please). It will have files similar to the contexts/ directory. Then, DAMON code will run on the process that writes 'on' to the 'state' file under the self_service/ directory.

For an example, below files tree is imaginable.

/sys/kernel/mm/damon/
│ non-root-users/
│ │ for_my_damon_usage/
│ │ │ self_service/ <- This is the new directory
│ │ │ │ state
│ │ │ │ contexts/nr_contexts
│ │ │ │ │ 0/avail_operations,operations
│ │ │ │ │ ...
│ │ │ kdamonds/nr_kdamonds
│ │ │ │ 0/state,pid
│ │ │ ...
│ admin/
│ │ self_service/ <- This is the new directory
│ │ │ state
│ │ │ contexts/nr_contexts
│ │ │ │ 0/avail_operations,operations
│ │ │ │ ...
│ │ kdamonds/nr_kdamonds
│ │ │ 0/state,pid
│ │ │ │ contexts/nr_contexts
│ │ │ │ │ 0/avail_operations,operations
│ │ │ │ │ │ monitoring_attrs/
│ │ │ │ │ │ │ intervals/sample_us,aggr_us,update_us │ │ │ │ │ │ │ nr_regions/min,max
│ │ │ │ │ │ targets/nr_targets
│ │ │ │ │ │ │ 0/pid_target
│ │ │ │ │ │ │ │ regions/nr_regions
│ │ │ │ │ │ │ │ │ 0/start,end
│ │ │ │ │ │ │ │ │ ...
│ │ │ │ │ │ │ ...
│ │ │ │ │ │ ...
│ │ │ │ │ ...
│ │ │ ...

Efficient Monitoring Results Collection
---------------------------------------

Currently, damon:damon_aggregated tracepoint is the only way to get the DAMON monitoring results in full. I heard some users don't want to get the full results, but only the monitoring results of specific address range having special access pattern. For the use case, the tracepoint is inefficient.

DAMON sysfs already provides some of the results via the stats/ directory under scheme directory, though. We could further extend it for more efficient and more detailed results collection. The idea is, adding a file under each scheme directory, say, report_the_detail. Writing Y to the file will create a directory called, say, 'report'. The file will have files similar to that under 'targets/<N>/regions/', and have contents that represents monitoring results of the address ranges that fulfills the scheme's target access pattern.

For example, below hierarchy is imaginable.

/sys/kernel/mm/damon/admin
│ kdamonds/nr_kdamonds
│ │ 0/state,pid
│ │ │ contexts/nr_contexts
│ │ │ │ 0/avail_operations,operations
│ │ │ │ │ monitoring_attrs/
│ │ │ │ │ │ intervals/sample_us,aggr_us,update_us
│ │ │ │ │ │ nr_regions/min,max
│ │ │ │ │ targets/nr_targets
│ │ │ │ │ │ 0/pid_target
│ │ │ │ │ │ │ regions/nr_regions
│ │ │ │ │ │ │ │ 0/start,end
│ │ │ │ │ │ │ │ ...
│ │ │ │ │ │ ...
│ │ │ │ │ schemes/nr_schemes
│ │ │ │ │ │ 0/action
│ │ │ │ │ │ │ report_the_detail <- this is the new file
│ │ │ │ │ │ │ report/ <- this is the new directory
│ │ │ │ │ │ │ │ regions/nr_regions <- nr_regions would be read-only │ │ │ │ │ │ │ │ │ 0/start,end,nr_accesses,age
│ │ │ │ │ │ │ │ │ ...
│ │ │ │ │ │ │ access_pattern/
│ │ │ │ │ │ │ │ sz/min,max
│ │ │ │ │ │ │ │ nr_accesses/min,max
│ │ │ │ │ │ │ │ age/min,max
│ │ │ │ │ │ │ quotas/ms,bytes,reset_interval_ms
│ │ │ │ │ │ │ │ weights/sz_permil,nr_accesses_permil,age_permil │ │ │ │ │ │ │ watermarks/metric,interval_us,high,mid,low
│ │ │ │ │ │ │ stats/nr_tried,sz_tried,nr_applied,sz_applied,qt_exceeds │ │ │ │ │ │ ...
│ │ │ │ ...
│ │ ...

As I told at the beginning, these are just my rough and humble ideas, and there could be some important things I'm missing. Also, I don't have ETAs for each of the plans, but currently have biggest interest in the efficient monitoring results collection. All are in my TODO list, though. All questions, comments, and suggestons are welcome as always!

Thanks,
SJ

#1 lore raw

Hi, SJ

Quoted 129 lines · SeongJae Park

Hello all,

I have discussed with several people about how they want to use DAMON for their specific use cases. For supporting those in perfectly optimal way, we might need to make yet another user interface. However, IMHO, almost all of the use cases could be covered by extending DAMON sysfs interface[1]. Hence, I'd like to share my rough and humble ideas here. It would be great if you could give some comments.

[1] https://docs.kernel.org/admin-guide/mm/damon/usage.html#sysfs-interface

Non-root Users
--------------

Someone told me they want to use DAMON from non-root user, for the user's processes. My DAMON sysfs interface extension idea for this usage is adding a special directory under the DAMON sysfs root (/sys/kernel/mm/damon/) as a sibling of the admin/ directory. The name would be, say, non-root-users/ (please blame my bad naming sense and suggest something better). The non-root users who want to use DAMON would be able to populate a directory under the directory. Then, the non-root user populated directory will have its files hierarchy same to that of the admin/ directory. The files will work same to those under admin/ directory but fails when the monitoring target is out of the user's permission. An example usage would be something like below:

     $ ls /sys/kernel/mm/damon
     admin non-root-users
     $ cd /sys/kernel/mm/damon/non-root-users
     $ mkdir for_my_damon_usage; cd for_my_damon_usage/kdamonds/
     $ echo 1 > nr_kdamonds; cd 0/
     $ echo 1 > /contexts/nr_contexts
     $ echo vaddr > 0/contexts/0/operations
     $ echo 1 > 0/contexts/0/targets/nr_targets
     $ echo $(pidof my_process) > 0/contexts/0/targets/0/pid_target
     $ echo on > state

The files tree would be something like below:

/sys/kernel/mm/damon/
│ non-root-users/ <- This is the new directory
│ │ for_my_damon_usage/
│ │ │ kdamonds/nr_kdamonds
│ │ │ │ 0/state,pid
│ │ │ ...
│ admin/
│ │ kdamonds/nr_kdamonds
│ │ │ 0/state,pid
│ │ │ │ contexts/nr_contexts
│ │ │ │ │ 0/avail_operations,operations
│ │ │ │ │ │ monitoring_attrs/
│ │ │ │ │ │ │ intervals/sample_us,aggr_us,update_us │ │ │ │ │ │ │ nr_regions/min,max
│ │ │ │ │ │ targets/nr_targets
│ │ │ │ │ │ │ 0/pid_target
│ │ │ │ │ │ │ │ regions/nr_regions
│ │ │ │ │ │ │ │ │ 0/start,end
│ │ │ │ │ │ │ │ │ ...
│ │ │ │ │ │ │ ...
│ │ │ │ │ │ ...
│ │ │ │ │ ...
│ │ │ ...

No kdamond
----------

Currently, DAMON creates worker kernel threads called kdamond and does requested works on those. There was a request asking the work to be done on the requester's process for easier resource usage charging. For this, we could add a new directory under the user-specific DAMON sysfs root directory, called, say, self-service/ (again, blame my bad naming sense please). It will have files similar to the contexts/ directory. Then, DAMON code will run on the process that writes 'on' to the 'state' file under the self_service/ directory.

For an example, below files tree is imaginable.

/sys/kernel/mm/damon/
│ non-root-users/
│ │ for_my_damon_usage/
│ │ │ self_service/ <- This is the new directory
│ │ │ │ state
│ │ │ │ contexts/nr_contexts
│ │ │ │ │ 0/avail_operations,operations
│ │ │ │ │ ...
│ │ │ kdamonds/nr_kdamonds
│ │ │ │ 0/state,pid
│ │ │ ...
│ admin/
│ │ self_service/ <- This is the new directory
│ │ │ state
│ │ │ contexts/nr_contexts
│ │ │ │ 0/avail_operations,operations
│ │ │ │ ...
│ │ kdamonds/nr_kdamonds
│ │ │ 0/state,pid
│ │ │ │ contexts/nr_contexts
│ │ │ │ │ 0/avail_operations,operations
│ │ │ │ │ │ monitoring_attrs/
│ │ │ │ │ │ │ intervals/sample_us,aggr_us,update_us │ │ │ │ │ │ │ nr_regions/min,max
│ │ │ │ │ │ targets/nr_targets
│ │ │ │ │ │ │ 0/pid_target
│ │ │ │ │ │ │ │ regions/nr_regions
│ │ │ │ │ │ │ │ │ 0/start,end
│ │ │ │ │ │ │ │ │ ...
│ │ │ │ │ │ │ ...
│ │ │ │ │ │ ...
│ │ │ │ │ ...
│ │ │ ...

Efficient Monitoring Results Collection
---------------------------------------

Currently, damon:damon_aggregated tracepoint is the only way to get the DAMON monitoring results in full. I heard some users don't want to get the full results, but only the monitoring results of specific address range having special access pattern. For the use case, the tracepoint is inefficient.

DAMON sysfs already provides some of the results via the stats/ directory under scheme directory, though. We could further extend it for more efficient and more detailed results collection. The idea is, adding a file under each scheme directory, say, report_the_detail. Writing Y to the file will create a directory called, say, 'report'. The file will have files similar to that under 'targets/<N>/regions/', and have contents that represents monitoring results of the address ranges that fulfills the scheme's target access pattern.

For example, below hierarchy is imaginable.

Quoted 13 lines

/sys/kernel/mm/damon/admin
│ kdamonds/nr_kdamonds
│ │ 0/state,pid
│ │ │ contexts/nr_contexts
│ │ │ │ 0/avail_operations,operations
│ │ │ │ │ monitoring_attrs/
│ │ │ │ │ │ intervals/sample_us,aggr_us,update_us │ │ │ │ │ │ nr_regions/min,max
│ │ │ │ │ targets/nr_targets
│ │ │ │ │ │ 0/pid_target
│ │ │ │ │ │ │ regions/nr_regions
│ │ │ │ │ │ │ │ 0/start,end
│ │ │ │ │ │ │ │ ...

It's good that the damon interface is getting more and more detailed, but what my concern is that if we have too many interface to operate, for example to check the region interface to know every region info,  it will lead to high overhead for kdamond thread  and user-mode tools process,  especially when we monitor many processes in one kdamond thread.

Quoted 29 lines
     │ │ │ │ │ │ ...
     │ │ │ │ │ schemes/nr_schemes
     │ │ │ │ │ │ 0/action
     │ │ │ │ │ │ │ report_the_detail	<- this is the new file
     │ │ │ │ │ │ │ report/		<- this is the new directory
     │ │ │ │ │ │ │ │ regions/nr_regions  <- nr_regions would be read-only
     │ │ │ │ │ │ │ │ │ 0/start,end,nr_accesses,age
     │ │ │ │ │ │ │ │ │ ...
     │ │ │ │ │ │ │ access_pattern/
     │ │ │ │ │ │ │ │ sz/min,max
     │ │ │ │ │ │ │ │ nr_accesses/min,max
     │ │ │ │ │ │ │ │ age/min,max
     │ │ │ │ │ │ │ quotas/ms,bytes,reset_interval_ms
     │ │ │ │ │ │ │ │ weights/sz_permil,nr_accesses_permil,age_permil
     │ │ │ │ │ │ │ watermarks/metric,interval_us,high,mid,low
     │ │ │ │ │ │ │ stats/nr_tried,sz_tried,nr_applied,sz_applied,qt_exceeds
     │ │ │ │ │ │ ...
     │ │ │ │ ...
     │ │ ...

As I told at the beginning, these are just my rough and humble ideas, and there could be some important things I'm missing. Also, I don't have ETAs for each of the plans, but currently have biggest interest in the efficient monitoring results collection. All are in my TODO list, though. All questions, comments, and suggestons are welcome as always!

Thanks,
SJ

Signature · 2 lines
Best Regards!
Xin Hao
author maintainer
#2 lore raw

Hi Xin,

Quoted 15 lines · On Mon, 6 Jun 2022 19:37:56 +0800 [email protected] wrote:

Hi, SJ

Quoted 12 lines · SeongJae Park

Hello all,

I have discussed with several people about how they want to use DAMON for their specific use cases. For supporting those in perfectly optimal way, we might need to make yet another user interface. However, IMHO, almost all of the use cases could be covered by extending DAMON sysfs interface[1]. Hence, I'd like to share my rough and humble ideas here. It would be great if you could give some comments.

[1] https://docs.kernel.org/admin-guide/mm/damon/usage.html#sysfs-interface

[...]

Quoted 32 lines
Quoted 18 lines

Efficient Monitoring Results Collection
---------------------------------------

Currently, damon:damon_aggregated tracepoint is the only way to get the DAMON monitoring results in full. I heard some users don't want to get the full results, but only the monitoring results of specific address range having special access pattern. For the use case, the tracepoint is inefficient.

DAMON sysfs already provides some of the results via the stats/ directory under scheme directory, though. We could further extend it for more efficient and more detailed results collection. The idea is, adding a file under each scheme directory, say, report_the_detail. Writing Y to the file will create a directory called, say, 'report'. The file will have files similar to that under 'targets/<N>/regions/', and have contents that represents monitoring results of the address ranges that fulfills the scheme's target access pattern.

For example, below hierarchy is imaginable.

Quoted 13 lines

/sys/kernel/mm/damon/admin
│ kdamonds/nr_kdamonds
│ │ 0/state,pid
│ │ │ contexts/nr_contexts
│ │ │ │ 0/avail_operations,operations
│ │ │ │ │ monitoring_attrs/
│ │ │ │ │ │ intervals/sample_us,aggr_us,update_us │ │ │ │ │ │ nr_regions/min,max
│ │ │ │ │ targets/nr_targets
│ │ │ │ │ │ 0/pid_target
│ │ │ │ │ │ │ regions/nr_regions
│ │ │ │ │ │ │ │ 0/start,end
│ │ │ │ │ │ │ │ ...

Quoted 5 lines

It's good that the damon interface is getting more and more detailed, but what my concern is that if we have too many interface to operate, for example to check the region interface to know every region info, it will lead to high overhead for kdamond thread and user-mode tools process, especially when we monitor many processes in one kdamond thread.

Good point, thank you for letting me know your concern! I think your concern makes sense, but in my humble opinion, the overhead could be lower than the tracepoint in many cases and at least be under control, as the users could still set the maximum number of regions, and the scheme's target access pattern. Also, users can decide when to write 'Y' to the 'report_the_deatail'
file. If it turns out the control is not so easy, I think we could imagine
adding a sort of time quota for the report/ directory population task. Of course, the user space app which will read the directory will need to control its overhead on its own.

Thanks,
SJ

Quoted 35 lines
Quoted 30 lines
     │ │ │ │ │ │ ...
     │ │ │ │ │ schemes/nr_schemes
     │ │ │ │ │ │ 0/action
     │ │ │ │ │ │ │ report_the_detail	<- this is the new file
     │ │ │ │ │ │ │ report/		<- this is the new directory
     │ │ │ │ │ │ │ │ regions/nr_regions  <- nr_regions would be read-only
     │ │ │ │ │ │ │ │ │ 0/start,end,nr_accesses,age
     │ │ │ │ │ │ │ │ │ ...
     │ │ │ │ │ │ │ access_pattern/
     │ │ │ │ │ │ │ │ sz/min,max
     │ │ │ │ │ │ │ │ nr_accesses/min,max
     │ │ │ │ │ │ │ │ age/min,max
     │ │ │ │ │ │ │ quotas/ms,bytes,reset_interval_ms
     │ │ │ │ │ │ │ │ weights/sz_permil,nr_accesses_permil,age_permil
     │ │ │ │ │ │ │ watermarks/metric,interval_us,high,mid,low
     │ │ │ │ │ │ │ stats/nr_tried,sz_tried,nr_applied,sz_applied,qt_exceeds
     │ │ │ │ │ │ ...
     │ │ │ │ ...
     │ │ ...

As I told at the beginning, these are just my rough and humble ideas, and there could be some important things I'm missing. Also, I don't have ETAs for each of the plans, but currently have biggest interest in the efficient monitoring results collection. All are in my TODO list, though. All questions, comments, and suggestons are welcome as always!

Thanks,
SJ

Signature · 2 lines
Best Regards!
Xin Hao
#3 lore raw
In reply to SeongJae Park
Quoted 14 lines · SeongJae Park

Non-root Users
--------------

Someone told me they want to use DAMON from non-root user, for the user's processes. My DAMON sysfs interface extension idea for this usage is adding a special directory under the DAMON sysfs root (/sys/kernel/mm/damon/) as a sibling of the admin/ directory. The name would be, say, non-root-users/ (please blame my bad naming sense and suggest something better). The non-root users who want to use DAMON would be able to populate a directory under the directory. Then, the non-root user populated directory will have its files hierarchy same to that of the admin/ directory. The files will work same to those under admin/ directory but fails when the monitoring target is out of the user's permission. An example usage would be something like below:

Quoted 11 lines
    $ ls /sys/kernel/mm/damon
    admin non-root-users
    $ cd /sys/kernel/mm/damon/non-root-users
    $ mkdir for_my_damon_usage; cd for_my_damon_usage/kdamonds/
    $ echo 1 > nr_kdamonds; cd 0/
    $ echo 1 > /contexts/nr_contexts
    $ echo vaddr > 0/contexts/0/operations
    $ echo 1 > 0/contexts/0/targets/nr_targets
    $ echo $(pidof my_process) > 0/contexts/0/targets/0/pid_target
    $ echo on > state

Hi SeongJae,

Thanks very much for looping us in on this.

One concern would be the ability of non-root userspace to be able to go off and mkdir under this hierarchy. It seems like for this to work as described above that root has created a non-root-users directory that anybody can create a directory under and then provision work for a kdamond thread to do.

Much better might be to require that the admin has to do any mkdir here. It can mkdir a non-root directory and then chown it to the user so that the non-root user can control it (much like a "user subcontainer" for a cgroup hierarchy).

Quoted 29 lines

The files tree would be something like below:

/sys/kernel/mm/damon/
│ non-root-users/ <- This is the new directory
│ │ for_my_damon_usage/
│ │ │ kdamonds/nr_kdamonds
│ │ │ │ 0/state,pid
│ │ │ ...
│ admin/
│ │ kdamonds/nr_kdamonds
│ │ │ 0/state,pid
│ │ │ │ contexts/nr_contexts
│ │ │ │ │ 0/avail_operations,operations
│ │ │ │ │ │ monitoring_attrs/
│ │ │ │ │ │ │ intervals/sample_us,aggr_us,update_us │ │ │ │ │ │ │ nr_regions/min,max
│ │ │ │ │ │ targets/nr_targets
│ │ │ │ │ │ │ 0/pid_target
│ │ │ │ │ │ │ │ regions/nr_regions
│ │ │ │ │ │ │ │ │ 0/start,end
│ │ │ │ │ │ │ │ │ ...
│ │ │ │ │ │ │ ...
│ │ │ │ │ │ ...
│ │ │ │ │ ...
│ │ │ ...

No kdamond
----------

Quoted 8 lines

Currently, DAMON creates worker kernel threads called kdamond and does requested works on those. There was a request asking the work to be done on the requester's process for easier resource usage charging. For this, we could add a new directory under the user-specific DAMON sysfs root directory, called, say, self-service/ (again, blame my bad naming sense please). It will have files similar to the contexts/ directory. Then, DAMON code will run on the process that writes 'on' to the 'state' file under the self_service/ directory.

For this, the sysfs interface might be getting too complex. We can imagine hundreds of files that userspace must maintain to implement the full scheme of how it wants to use DAMON. This allows it to be very configurable, but at the downside of a lot of complexity that must be managed not only in the interface but also in how kdamond must be implemented to support the various combinations.

So I'd definitely recommend that all possible tunables and configurations are indeed necessary and have real-world use cases behind them that are *generally* useful to everybody. In other words, hold a high standard for trying to add more tunables under these directories because it becomes very hard, if not impossible, to deprecate this support once the sysfs interface provides them and userspace depend on them. Something to keep in mind when extending this interface.

Quoted 37 lines

For an example, below files tree is imaginable.

/sys/kernel/mm/damon/
│ non-root-users/
│ │ for_my_damon_usage/
│ │ │ self_service/ <- This is the new directory
│ │ │ │ state
│ │ │ │ contexts/nr_contexts
│ │ │ │ │ 0/avail_operations,operations
│ │ │ │ │ ...
│ │ │ kdamonds/nr_kdamonds
│ │ │ │ 0/state,pid
│ │ │ ...
│ admin/
│ │ self_service/ <- This is the new directory
│ │ │ state
│ │ │ contexts/nr_contexts
│ │ │ │ 0/avail_operations,operations
│ │ │ │ ...
│ │ kdamonds/nr_kdamonds
│ │ │ 0/state,pid
│ │ │ │ contexts/nr_contexts
│ │ │ │ │ 0/avail_operations,operations
│ │ │ │ │ │ monitoring_attrs/
│ │ │ │ │ │ │ intervals/sample_us,aggr_us,update_us │ │ │ │ │ │ │ nr_regions/min,max
│ │ │ │ │ │ targets/nr_targets
│ │ │ │ │ │ │ 0/pid_target
│ │ │ │ │ │ │ │ regions/nr_regions
│ │ │ │ │ │ │ │ │ 0/start,end
│ │ │ │ │ │ │ │ │ ...
│ │ │ │ │ │ │ ...
│ │ │ │ │ │ ...
│ │ │ │ │ ...
│ │ │ ...

author maintainer
#4 lore raw

Hello David,

Quoted 38 lines · On Wed, 20 Jul 2022 18:41:29 -0700 (PDT) David Rientjes <[email protected]>…
Quoted 14 lines · SeongJae Park

Non-root Users
--------------

Someone told me they want to use DAMON from non-root user, for the user's processes. My DAMON sysfs interface extension idea for this usage is adding a special directory under the DAMON sysfs root (/sys/kernel/mm/damon/) as a sibling of the admin/ directory. The name would be, say, non-root-users/ (please blame my bad naming sense and suggest something better). The non-root users who want to use DAMON would be able to populate a directory under the directory. Then, the non-root user populated directory will have its files hierarchy same to that of the admin/ directory. The files will work same to those under admin/ directory but fails when the monitoring target is out of the user's permission. An example usage would be something like below:

Quoted 11 lines
    $ ls /sys/kernel/mm/damon
    admin non-root-users
    $ cd /sys/kernel/mm/damon/non-root-users
    $ mkdir for_my_damon_usage; cd for_my_damon_usage/kdamonds/
    $ echo 1 > nr_kdamonds; cd 0/
    $ echo 1 > /contexts/nr_contexts
    $ echo vaddr > 0/contexts/0/operations
    $ echo 1 > 0/contexts/0/targets/nr_targets
    $ echo $(pidof my_process) > 0/contexts/0/targets/0/pid_target
    $ echo on > state

Hi SeongJae,

Thanks very much for looping us in on this.

One concern would be the ability of non-root userspace to be able to go off and mkdir under this hierarchy. It seems like for this to work as described above that root has created a non-root-users directory that anybody can create a directory under and then provision work for a kdamond thread to do.

Quoted 4 lines

Much better might be to require that the admin has to do any mkdir here. It can mkdir a non-root directory and then chown it to the user so that the non-root user can control it (much like a "user subcontainer" for a cgroup hierarchy).

Agreed, that sounds making much more sense and provides better user experience.

Quoted 46 lines
Quoted 30 lines

The files tree would be something like below:

/sys/kernel/mm/damon/
│ non-root-users/ <- This is the new directory
│ │ for_my_damon_usage/
│ │ │ kdamonds/nr_kdamonds
│ │ │ │ 0/state,pid
│ │ │ ...
│ admin/
│ │ kdamonds/nr_kdamonds
│ │ │ 0/state,pid
│ │ │ │ contexts/nr_contexts
│ │ │ │ │ 0/avail_operations,operations
│ │ │ │ │ │ monitoring_attrs/
│ │ │ │ │ │ │ intervals/sample_us,aggr_us,update_us │ │ │ │ │ │ │ nr_regions/min,max
│ │ │ │ │ │ targets/nr_targets
│ │ │ │ │ │ │ 0/pid_target
│ │ │ │ │ │ │ │ regions/nr_regions
│ │ │ │ │ │ │ │ │ 0/start,end
│ │ │ │ │ │ │ │ │ ...
│ │ │ │ │ │ │ ...
│ │ │ │ │ │ ...
│ │ │ │ │ ...
│ │ │ ...

No kdamond
----------

Quoted 8 lines

Currently, DAMON creates worker kernel threads called kdamond and does requested works on those. There was a request asking the work to be done on the requester's process for easier resource usage charging. For this, we could add a new directory under the user-specific DAMON sysfs root directory, called, say, self-service/ (again, blame my bad naming sense please). It will have files similar to the contexts/ directory. Then, DAMON code will run on the process that writes 'on' to the 'state' file under the self_service/ directory.

For this, the sysfs interface might be getting too complex. We can imagine hundreds of files that userspace must maintain to implement the full scheme of how it wants to use DAMON. This allows it to be very configurable, but at the downside of a lot of complexity that must be managed not only in the interface but also in how kdamond must be implemented to support the various combinations.

Quoted 7 lines

So I'd definitely recommend that all possible tunables and configurations are indeed necessary and have real-world use cases behind them that are *generally* useful to everybody. In other words, hold a high standard for trying to add more tunables under these directories because it becomes very hard, if not impossible, to deprecate this support once the sysfs interface provides them and userspace depend on them. Something to keep in mind when extending this interface.

Again, fully agreed. Ideas here are only in brain storming level at the moment, and would need more discussion with various party to justify the benefit of the added complexity. To expedite such discussions, I'm planning an open, regular, and informal virtual meeting series. I will announce that soon as soon as prepared.

Off the topic. I'm thinking about a sort of auto-tuning mechanism for reducing the tuning complexity. For example, TMO[1] idea-based DAMON_RECLAIM autotuning. My experiments running a very simplified TMO[2] with DAMON_LRU_SORT in background reduces some-memory-PSI about 10%.

[1] https://www.pdl.cmu.edu/ftp/NVM/tmo_asplos22.pdf
[2] https://github.com/awslabs/damon-tests/blob/next/perf/runners/back/0009_ttmo.sh

Thanks,
SJ

Quoted 39 lines
Quoted 38 lines

For an example, below files tree is imaginable.

/sys/kernel/mm/damon/
│ non-root-users/
│ │ for_my_damon_usage/
│ │ │ self_service/ <- This is the new directory
│ │ │ │ state
│ │ │ │ contexts/nr_contexts
│ │ │ │ │ 0/avail_operations,operations
│ │ │ │ │ ...
│ │ │ kdamonds/nr_kdamonds
│ │ │ │ 0/state,pid
│ │ │ ...
│ admin/
│ │ self_service/ <- This is the new directory
│ │ │ state
│ │ │ contexts/nr_contexts
│ │ │ │ 0/avail_operations,operations
│ │ │ │ ...
│ │ kdamonds/nr_kdamonds
│ │ │ 0/state,pid
│ │ │ │ contexts/nr_contexts
│ │ │ │ │ 0/avail_operations,operations
│ │ │ │ │ │ monitoring_attrs/
│ │ │ │ │ │ │ intervals/sample_us,aggr_us,update_us │ │ │ │ │ │ │ nr_regions/min,max
│ │ │ │ │ │ targets/nr_targets
│ │ │ │ │ │ │ 0/pid_target
│ │ │ │ │ │ │ │ regions/nr_regions
│ │ │ │ │ │ │ │ │ 0/start,end
│ │ │ │ │ │ │ │ │ ...
│ │ │ │ │ │ │ ...
│ │ │ │ │ │ ...
│ │ │ │ │ ...
│ │ │ ...