Skip to contentSkip to conversation

Containers: css_put() dilemma

DISCUSSION Balbir Singh 5 participants 26 messages Jul 16, 2007 – Jul 19, 2007

lore
Share & tools
lore
#0 lore raw

Hi, Paul,

I've run into a strange problem with css_put(). After the changes for notify_on_release(), the css_put() routine can now block and it blocks on the container_mutex. This implies that css_put() cannot be called if

1. We cannot block
2. We already hold the container_mutex

The problem I have is that of preventing the destruction of my container (when the user does rmdir). If the user migrates away all tasks and does an rmdir, the only way to prevent the container from going away is through css_get() references. In my case, some pages have been allocated from the container and hence I do not want it to go away, until all the pages charged to it are freed. When I use css_get/put() to prevent destruction I am blocked by the limitations of css_put() listed above.

Do you have any recommendations for a cleaner solution? I suspect we'll need can_destroy() callbacks (similar to can_attach()).

Signature · 4 lines
	Warm Regards,
	Balbir Singh
	Linux Technology Center
	IBM, ISTL
#1 lore raw
Quoted 16 lines · Balbir Singh

Hi, Paul,

I've run into a strange problem with css_put(). After the changes for notify_on_release(), the css_put() routine can now block and it blocks on the container_mutex. This implies that css_put() cannot be called if

1. We cannot block
2. We already hold the container_mutex

The problem I have is that of preventing the destruction of my container (when the user does rmdir). If the user migrates away all tasks and does an rmdir, the only way to prevent the container from going away is through css_get() references. In my case, some pages have been allocated from the container and hence I do not want it to go away, until all the pages charged to it are freed. When I use css_get/put() to prevent destruction I am blocked by the limitations of css_put() listed above.

Quoted 2 lines

Do you have any recommendations for a cleaner solution? I suspect we'll need can_destroy() callbacks (similar to can_attach()).

I think moving the release_list synchronization inside a separate spinlock, and thus not requiring container_mutex to be held for check_for_release(), is the simplest solution. I'll do that. I'm hoping to get a new set of patches to Andrew today or tomorrow.

Adding a can_destroy() callback is possible, but since I envisage that most subsystems that would want to implement it would basically be doing reference counting anyway, it seems worth having a generic reference counting mechanism in the framework. In particular, since once the container does become releasable due to all the subsystem-specific refcounts being released, we want to be able to invoke the release agent, we'll end up with the same synchronization problems that we have now if we just pushed everything into a can_destroy() method. (Unless the framework polled all can_destroy() methods for potentially-removable containers, which seems a bit nasty).

We can add can_destroy() if we encounter a situation that can't be handled by generic reference counting.

Paul

#2 lore raw
Quoted 19 lines · Paul (??) Menage
Quoted 18 lines · Balbir Singh

Hi, Paul,

I've run into a strange problem with css_put(). After the changes for notify_on_release(), the css_put() routine can now block and it blocks on the container_mutex. This implies that css_put() cannot be called if

1. We cannot block
2. We already hold the container_mutex

The problem I have is that of preventing the destruction of my container (when the user does rmdir). If the user migrates away all tasks and does an rmdir, the only way to prevent the container from going away is through
css_get() references. In my case, some pages have been allocated from the container and hence I do not want it to go away, until all the pages charged to it are freed. When I use css_get/put() to prevent destruction I am blocked by the limitations of css_put() listed above.

Quoted 8 lines
Quoted 2 lines

Do you have any recommendations for a cleaner solution? I suspect we'll need can_destroy() callbacks (similar to can_attach()).

I think moving the release_list synchronization inside a separate spinlock, and thus not requiring container_mutex to be held for check_for_release(), is the simplest solution. I'll do that. I'm hoping to get a new set of patches to Andrew today or tomorrow.

That sounds good to me. But I worry about having to do release synchronization on every css_put(). The current patch I have, but does not work 100% does the following (WARNING: white spaces ahead, do not use the patch directly)

- if (notify_on_release(cont)) {

+       if (atomic_dec_and_test(&css->refcnt) && notify_on_release(cont)) {
                mutex_lock(&container_mutex);
                set_bit(CONT_RELEASABLE, &cont->flags);
-               if (atomic_dec_and_test(&css->refcnt)) {
-                       check_for_release(cont);
-               }
+               check_for_release(cont);
                mutex_unlock(&container_mutex);

That way we set the CONT_RELEASABLE bit only when the ref count drops to zero.

Quoted 12 lines

Adding a can_destroy() callback is possible, but since I envisage that most subsystems that would want to implement it would basically be doing reference counting anyway, it seems worth having a generic reference counting mechanism in the framework. In particular, since once the container does become releasable due to all the subsystem-specific refcounts being released, we want to be able to invoke the release agent, we'll end up with the same synchronization problems that we have now if we just pushed everything into a can_destroy() method. (Unless the framework polled all can_destroy() methods for potentially-removable containers, which seems a bit nasty).

Quoted 3 lines

We can add can_destroy() if we encounter a situation that can't be handled by generic reference counting.

Yes, that is correct, the advantage is that with can_destroy() we don't need to go through release synchronization each time we do a css_put(). May be the patch above will fix the problem along with your release locking proposal.

Quoted 5 lines

Paul
_______________________________________________
Containers mailing list Containers-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org
https://lists.linux-foundation.org/mailman/listinfo/containers

Signature · 4 lines
	Warm Regards,
	Balbir Singh
	Linux Technology Center
	IBM, ISTL
#3 lore raw

Review

Quoted 3 lines · Balbir Singh
(quoted diff) +1 -1
-       if (notify_on_release(cont)) {
+       if (atomic_dec_and_test(&css->refcnt) && notify_on_release(cont)) {

This seems like a good idea, as long as atomic_dec_and_test() isn't noticeably more expensive than atomic_dec(). I assume it shouldn't need to be, since the bus locking operations are presumably the same in each case.

Quoted 11 lines
                mutex_lock(&container_mutex);
                set_bit(CONT_RELEASABLE, &cont->flags);
(quoted diff) +1 -3
-               if (atomic_dec_and_test(&css->refcnt)) {
-                       check_for_release(cont);
-               }
+               check_for_release(cont);
                mutex_unlock(&container_mutex);

That way we set the CONT_RELEASABLE bit only when the ref count drops to zero.

That's probably a good idea, in conjunction with another part of my patch for this that frees container objects under RCU - as soon as you do the atomic_dec_and_test(), then in theory some other thread could delete the container (since we're no longer going to be taking container_mutex in this function). But as long as the container object remains valid until synchronize_rcu() completes, then we can safely set the CONT_RELEASABLE bit on it.

Quoted 4 lines

Yes, that is correct, the advantage is that with can_destroy() we don't need to go through release synchronization each time we do a css_put().

I think the amount of release synchronization *needed* is going to be the same whether you have the refcounting done in the subsystem or in the framework. But I agree that right now we're doing one more atomic op than we strictly need to, and can remove it.

Paul

#4 lore raw
Quoted 35 lines · Paul (宝瑠) Menage

Review

Quoted 3 lines · Balbir Singh
(quoted diff) +1 -1
-       if (notify_on_release(cont)) {
+       if (atomic_dec_and_test(&css->refcnt) && notify_on_release(cont)) {

This seems like a good idea, as long as atomic_dec_and_test() isn't noticeably more expensive than atomic_dec(). I assume it shouldn't need to be, since the bus locking operations are presumably the same in each case.

Quoted 11 lines
               mutex_lock(&container_mutex);
               set_bit(CONT_RELEASABLE, &cont->flags);
(quoted diff) +1 -3
-               if (atomic_dec_and_test(&css->refcnt)) {
-                       check_for_release(cont);
-               }
+               check_for_release(cont);
               mutex_unlock(&container_mutex);

That way we set the CONT_RELEASABLE bit only when the ref count drops to zero.

That's probably a good idea, in conjunction with another part of my patch for this that frees container objects under RCU - as soon as you do the atomic_dec_and_test(), then in theory some other thread could delete the container (since we're no longer going to be taking container_mutex in this function). But as long as the container object remains valid until synchronize_rcu() completes, then we can safely set the CONT_RELEASABLE bit on it.

Quoted 4 lines

Yes, that is correct, the advantage is that with can_destroy() we don't need to go through release synchronization each time we do a css_put().

Quoted 6 lines

I think the amount of release synchronization *needed* is going to be the same whether you have the refcounting done in the subsystem or in the framework. But I agree that right now we're doing one more atomic op than we strictly need to, and can remove it.

Paul

Hi, Paul/Andrew

Would you accept this fix, while we wait for the complete solution. It worked for me quite well.

Description

Stop checking if the container can be released every time we do css_put(). A better solution that avoids container_mutex has been suggested by Paul, but meanwhile, to get containers working correctly, this fix would be very useful.

Signed-off-by <[email protected]>

---

 kernel/container.c |    8 ++------
 1 file changed, 2 insertions(+), 6 deletions(-)
kernel/container.c renamed from kernel/container.c~container-css-put-on-refcount-zero +2 -6
diff -puN kernel/container.c~container-css-put-on-refcount-zero kernel/container.c
@@ -2515,15 +2515,11 @@ static void check_for_release(struct con
 void css_put(struct container_subsys_state *css)
 {
 	struct container *cont = css->container;
-	if (notify_on_release(cont)) {
+	if (atomic_dec_and_test(&css->refcnt) && notify_on_release(cont)) {
 		mutex_lock(&container_mutex);
 		set_bit(CONT_RELEASABLE, &cont->flags);
-		if (atomic_dec_and_test(&css->refcnt)) {
-			check_for_release(cont);
-		}
+		check_for_release(cont);
 		mutex_unlock(&container_mutex);
-	} else {
-		atomic_dec(&css->refcnt);
 	}
 }
 

_

Signature · 4 lines
	Warm Regards,
	Balbir Singh
	Linux Technology Center
	IBM, ISTL
#5 lore raw
Quoted 9 lines · Balbir Singh
Quoted 9 lines

kernel/container.c static void check_for_release(struct con · line 2522

Quoted 9 lines
               mutex_lock(&container_mutex);
               set_bit(CONT_RELEASABLE, &cont->flags);
kernel/container.c renamed from kernel/container.c~container-css-put-on-refcount-zero +1 -3
-               if (atomic_dec_and_test(&css->refcnt)) {
-                       check_for_release(cont);
-               }
+               check_for_release(cont);
               mutex_unlock(&container_mutex);

I think that this isn't safe as it stands, without a synchronize_rcu() in container_diput() prior to the kfree(). Also, it will break if anyone tries to use a release agent on a hierarchy that has your memory controller bound to it.

Paul

#6 lore raw
Quoted 11 lines · Paul (??) Menage
Quoted 9 lines · Balbir Singh
Quoted 9 lines

kernel/container.c static void check_for_release(struct con · line 2522

Quoted 9 lines
               mutex_lock(&container_mutex);
               set_bit(CONT_RELEASABLE, &cont->flags);
kernel/container.c renamed from kernel/container.c~container-css-put-on-refcount-zero +1 -3
-               if (atomic_dec_and_test(&css->refcnt)) {
-                       check_for_release(cont);
-               }
+               check_for_release(cont);
               mutex_unlock(&container_mutex);
Quoted 5 lines

I think that this isn't safe as it stands, without a synchronize_rcu() in container_diput() prior to the kfree(). Also, it will break if anyone tries to use a release agent on a hierarchy that has your memory controller bound to it.

Isn't the code functionally the same as before? We still do atomic_test_and_dec() as before. We still set_bit() CONT_RELEASABLE, we take the container_mutex and check_for_release(). I am not sure I understand what changed?

Could you please elaborate as to why using a release agent is broken when the memory controller is attached to it? I am not quite sure why we need the synchronize_rcu() either in container_diput().

Quoted 5 lines

Paul
_______________________________________________
Containers mailing list
[email protected]
https://lists.linux-foundation.org/mailman/listinfo/containers

Signature · 4 lines
	Warm Regards,
	Balbir Singh
	Linux Technology Center
	IBM, ISTL
#7 lore raw
Quoted 19 lines · Balbir Singh
Quoted 11 lines · Paul (??) Menage
Quoted 9 lines · Balbir Singh
Quoted 9 lines

kernel/container.c static void check_for_release(struct con · line 2522

Quoted 9 lines
               mutex_lock(&container_mutex);
               set_bit(CONT_RELEASABLE, &cont->flags);
kernel/container.c renamed from kernel/container.c~container-css-put-on-refcount-zero +1 -3
-               if (atomic_dec_and_test(&css->refcnt)) {
-                       check_for_release(cont);
-               }
+               check_for_release(cont);
               mutex_unlock(&container_mutex);
Quoted 5 lines

I think that this isn't safe as it stands, without a synchronize_rcu() in container_diput() prior to the kfree(). Also, it will break if anyone tries to use a release agent on a hierarchy that has your memory controller bound to it.

Quoted 3 lines

Isn't the code functionally the same as before? We still do atomic_test_and_dec() as before. We still set_bit() CONT_RELEASABLE, we take the container_mutex and check_for_release(). I am not sure I understand what changed?

Because as soon as you do the atomic_dec_and_test() on css->refcnt and the refcnt hits zero, then theoretically someone other thread (that already holds container_mutex) could check that the refcount is zero and free the container structure.

Adding a synchronize_rcu in container_diput() guarantees that the container structure won't be freed while someone may still be accessing it.

Quoted 3 lines

Could you please elaborate as to why using a release agent is broken when the memory controller is attached to it?

Because then it will try to take container_mutex in css_put() if it drops the last reference to a container, which is the thing that you said you had to avoid since you called css_put() in contexts that couldn't sleep.

Paul

#8 lore raw
Quoted 4 lines · Paul (宝瑠) Menage

Because as soon as you do the atomic_dec_and_test() on css->refcnt and the refcnt hits zero, then theoretically someone other thread (that already holds container_mutex) could check that the refcount is zero and free the container structure.

Then that other task had a reference and itself should have bumped the count, and the other user would never have seen it hit zero.

Even if there are still pages attached to the container, why not just have those take a reference, and don't bother actually freeing the container until the last true reference is dropped?

Does it matter if the destruction callbacks don't happen until well after an attempt to destroy the container is made?

-- Dave

#9 lore raw
Quoted 8 lines · Dave Hansen
Quoted 4 lines · Paul (宝瑠) Menage

Because as soon as you do the atomic_dec_and_test() on css->refcnt and the refcnt hits zero, then theoretically someone other thread (that already holds container_mutex) could check that the refcount is zero and free the container structure.

Then that other task had a reference and itself should have bumped the count, and the other user would never have seen it hit zero.

Nope. The container could have been empty (of tasks) and hence had a zero count.

The liveness model used by containers is that when the refcount hits zero, the container isn't immediately destroyed (since it can contain useful historical usage data, etc) but simply becomes eligible for destruction by userspace via an rmdir().

Quoted 4 lines

Even if there are still pages attached to the container, why not just have those take a reference, and don't bother actually freeing the container until the last true reference is dropped?

Yes, we could potentially just use the main count variable rather than having separate per-subsystem extra refcounts. The main reasons to do it this way are:

- the root subsystem state for a subsystem can shift between different "struct container" objects if it was previously inactive and gets mounted as part of a hierarchy (or similarly, gets unmounted and goes inactive). Possibly we could get around this by simply saying not doing refcounting on the subsys states attached to root containers since they can never be freed anyway.

- At some point I'd like to be able to support shifting subsystems between active hierarchies, at least in limited cases such as where the hierarchies are isomorphic, or binding/unbinding subsystems to/from active hierarchies. At that point we definitely need to be able to split out the different subsystem state refcounts from one another in the same hierarchy.

- we'd still have the issue that Balbir wants to be able to drop a reference in a non-sleeping context, and we want to avoid doing excessive synchronization in the normal case when the css_put() doesn't put the reference count to zero.

Quoted 3 lines

Does it matter if the destruction callbacks don't happen until well after an attempt to destroy the container is made?

Well that's sort of the point of putting a synchronize_rcu() in container_diput() - it ensures that the actual destruction of the object doesn't occur until sufficiently after the destruction attempt is initiated that no one is still using the reference.

The alternative would be something that polls to spot whether refcounts have reached zero and if so runs the userspace helper. That doesn't seem particularly palatable when you have large numbers of containers, if we can avoid it easily.

Paul

#10 lore raw
In reply to Paul (宝瑠) Menage
Quoted 4 lines

Because as soon as you do the atomic_dec_and_test() on css->refcnt and the refcnt hits zero, then theoretically someone other thread (that already holds container_mutex) could check that the refcount is zero and free the container structure.

Not just theory ... I've debugged crashes from live customer systems that were basically this race.

Signature · 3 lines
                  I won't rest till it's the best ...
                  Programmer, Linux Scalability
                  Paul Jackson <[email protected]> 1.925.600.0401
#11 lore raw
In reply to Paul (宝瑠) Menage
Quoted 5 lines · Paul (??) Menage

Because as soon as you do the atomic_dec_and_test() on css->refcnt and the refcnt hits zero, then theoretically someone other thread (that already holds container_mutex) could check that the refcount is zero and free the container structure.

Hi, Paul,

That sounds correct. I wonder now if the solution should be some form of delegation for deletion of unreferenced containers (HINT: work queue or kernel threads).

Quoted 4 lines

Adding a synchronize_rcu in container_diput() guarantees that the container structure won't be freed while someone may still be accessing it.

Do we take rcu_read_lock() in css_put() path or use call_rcu() to free the container?

Quoted 10 lines
Quoted 3 lines

Could you please elaborate as to why using a release agent is broken when the memory controller is attached to it?

Because then it will try to take container_mutex in css_put() if it drops the last reference to a container, which is the thing that you said you had to avoid since you called css_put() in contexts that couldn't sleep.

Paul

Signature · 4 lines
	Warm Regards,
	Balbir Singh
	Linux Technology Center
	IBM, ISTL
#12 lore raw
Quoted 4 lines · Balbir Singh

That sounds correct. I wonder now if the solution should be some form of delegation for deletion of unreferenced containers (HINT: work queue or kernel threads).

What a great idea. In fact, that's exactly what the release agent patch already does.

Quoted 8 lines
Quoted 5 lines

Adding a synchronize_rcu in container_diput() guarantees that the container structure won't be freed while someone may still be accessing it.

Do we take rcu_read_lock() in css_put() path or use call_rcu() to free the container?

Good point, we ought to add rcu_read_lock() (even though it doesn't actually do anything on architectures other than alpha, right?)

Using call_rcu to do the container kfree rather than synchronize_rcu() would be a possible future optimization, yes.

Paul

#23 lore raw
In reply to Balbir Singh
Quoted 3 lines

That sounds correct. I wonder now if the solution should be some form of delegation for deletion of unreferenced containers (HINT: work queue or kernel threads).

At least for cpusets (the mother of all containers), notify on release is part of the user visible API of cpusets. The kernel does not remove cpusets; it runs a user program, /sbin/cpuset_release_agent. That program might choose to rmdir the cpuset directory, and/or do other actions, like notify a batch scheduler that one of its cpusets was released.

That API is not as sacrosanct as some, but it is working, and I wouldn't want to break it without good reason.

Signature · 3 lines
                  I won't rest till it's the best ...
                  Programmer, Linux Scalability
                  Paul Jackson <[email protected]> 1.925.600.0401
#13 lore raw
In reply to Paul (宝瑠) Menage
Quoted 2 lines · Paul M

In fact, that's exactly what the release agent
patch already does.

I'm feeling lazy ;) What's the Subject of that
patch, for my easy searching?

Signature · 3 lines
                  I won't rest till it's the best ...
                  Programmer, Linux Scalability
                  Paul Jackson <[email protected]> 1.925.600.0401
#24 lore raw
In reply to Paul Jackson
Quoted 7 lines · Paul Jackson

At least for cpusets (the mother of all containers), notify on release is part of the user visible API of cpusets. The kernel does not remove cpusets; it runs a user program, /sbin/cpuset_release_agent. That program might choose to rmdir the cpuset directory, and/or do other actions, like notify a batch scheduler that one of its cpusets was released.

Right, that's what the release agent patch for process containers does
- there's a file in the hierarchy root called "release_agent" that contains the path to the program to run if a notify_on_release container goes idle. When you mount the "cpuset" filesystem, it automatically populates that file with "/sbin/cpuset_release_agent". A workqueue task is used to actually do the notifications so that references can be dropped without having to potentially run a userspace helper.

Paul

#14 lore raw
In reply to Paul Jackson
Quoted 6 lines · Paul Jackson
Quoted 2 lines · Paul M

In fact, that's exactly what the release agent
patch already does.

I'm feeling lazy ;) What's the Subject of that
patch, for my easy searching?

"Support for automatic userspace release agents"

Paul

#25 lore raw
In reply to Paul (宝瑠) Menage
Quoted 5 lines · Paul M

Right, that's what the release agent patch for process containers does
- there's a file in the hierarchy root called "release_agent" that contains the path to the program to run if a notify_on_release container goes idle. When you mount the "cpuset" filesystem, it automatically populates that file with "/sbin/cpuset_release_agent".

Cool -- thanks. Sounds good.

Nevermind my other "what's the Subject" query, if you haven't already answered it. You just nicely answered my questions.

Signature · 3 lines
                  I won't rest till it's the best ...
                  Programmer, Linux Scalability
                  Paul Jackson <pj-sJ/[email protected]> 1.925.600.0401
#15 lore raw
In reply to Paul (宝瑠) Menage
Quoted 8 lines · Paul (??) Menage
Quoted 4 lines · Balbir Singh

That sounds correct. I wonder now if the solution should be some form of delegation for deletion of unreferenced containers (HINT: work queue or kernel threads).

What a great idea. In fact, that's exactly what the release agent patch already does.

:-) I should have seen that. I am a little lost thinking that container_rmdir() and the release agent check_for_release() work without too much knowledge of each other. BTW, what are the semantics of css_put() is it expected to free the container/run the release agent when the reference count of the container_subsys_state drops to zero?

Signature · 4 lines
	Warm Regards,
	Balbir Singh
	Linux Technology Center
	IBM, ISTL
#16 lore raw
Quoted 4 lines · Balbir Singh

without too much knowledge of each other. BTW, what are the semantics of css_put() is it expected to free the container/run the release agent when the reference count of the container_subsys_state drops to zero?

If you css_put() the last reference on a subsystem state object and the associated container is marked as notify_on_release, then check_for_release() is called which does a more full check of whether the container is releasable. If it is, a workqueue task is scheduled to run the userspace release agent, which can then do anything it wants, including potentially deleting the empty container.

Paul

#17 lore raw
Quoted 6 lines · Paul (??) Menage
Quoted 4 lines · Balbir Singh

without too much knowledge of each other. BTW, what are the semantics of css_put() is it expected to free the container/run the release agent when the reference count of the container_subsys_state drops to zero?

Quoted 7 lines

If you css_put() the last reference on a subsystem state object and the associated container is marked as notify_on_release, then check_for_release() is called which does a more full check of whether the container is releasable. If it is, a workqueue task is scheduled to run the userspace release agent, which can then do anything it wants, including potentially deleting the empty container.

Ok.. so my problem still remains, how do I get a non-blocking atomic reference increment/decrement routine, that would prevent my container from being deleted?

I don't find cpusets using css_put(). I was hoping that we could alter css_* would provide the functionality I need.

Signature · 4 lines
	Warm Regards,
	Balbir Singh
	Linux Technology Center
	IBM, ISTL
#18 lore raw
Quoted 14 lines · Balbir Singh
Quoted 5 lines · Paul (??) Menage
Quoted 4 lines · Balbir Singh

without too much knowledge of each other. BTW, what are the semantics of css_put() is it expected to free the container/run the release agent when the reference count of the container_subsys_state drops to zero?

Quoted 7 lines

If you css_put() the last reference on a subsystem state object and the associated container is marked as notify_on_release, then check_for_release() is called which does a more full check of whether the container is releasable. If it is, a workqueue task is scheduled to run the userspace release agent, which can then do anything it wants, including potentially deleting the empty container.

Quoted 8 lines

Ok.. so my problem still remains, how do I get a non-blocking atomic reference increment/decrement routine, that would prevent my container from being deleted?

I don't find cpusets using css_put(). I was hoping that we could alter css_* would provide the functionality I need.

Thinking out loud again, can we add can_destory() callbacks?

Signature · 4 lines
	Warm Regards,
	Balbir Singh
	Linux Technology Center
	IBM, ISTL
#22 lore raw
In reply to Balbir Singh
Quoted 4 lines · Balbir Singh

Ok.. so my problem still remains, how do I get a non-blocking atomic reference increment/decrement routine, that would prevent my container from being deleted?

css_put() in my new patchset will be non-blocking.

Quoted 2 lines

I don't find cpusets using css_put().

Cpusets never needs to keep a non-process reference on a cpuset object, so the framework handles everything.

Paul

#20 lore raw
In reply to Balbir Singh
Quoted 3 lines · Balbir Singh

Thinking out loud again, can we add can_destroy() callbacks?

What would the exact semantics of such a callback be?

Since for proper interaction with release agents we need the subsystem to notify the framework when a subsystem object becomes releasable (currently as part of css_put()), what would a can_destroy() callback let you do that you couldn't do just by taking an extra css refcount to prevent destruction and releasing that refcount to allow destruction?

Paul

#21 lore raw
Quoted 5 lines · Paul Menage
Quoted 3 lines · Balbir Singh

Thinking out loud again, can we add can_destroy() callbacks?

Quoted 8 lines

What would the exact semantics of such a callback be?

Since for proper interaction with release agents we need the subsystem to notify the framework when a subsystem object becomes releasable (currently as part of css_put()), what would a can_destroy() callback let you do that you couldn't do just by taking an extra css refcount to prevent destruction and releasing that refcount to allow destruction?

I was thinking along those lines before you mentioned that the next version of css_put() will not block. The advantage I see of can_destory() is that it allows subsystems to do their own reference counting and decide whether they are ready to be deleted or not. The other advantage I see is that it can act like a prepare to be deleted phase for the controller, the controller might decide to take some action in the can_destroy() phase, like the memory controller could decide to start reclaiming all the remaining page cache pages.

BTW, do you know upfront as to when the next set of container enhancement patches will be ready? The css_put() issue is blocking us currently.

Balbir