xfs/log: protect xc_cil in xlog_cil_push()
Subsystems (2)
PATCH 1/1 xfs/log: protect xc_cil in xlog_cil_push()
xlog_cil_push() is the reader and writer of xc_cil, and should be protected against xlog_cil_insert_items().
To: [email protected]
Cc: [email protected]
--- fs/xfs/xfs_log_cil.c | 2 ++ 1 file changed, 2 insertions(+)
fs/xfs/xfs_log_cil.c +2 -0
@@ -723,6 +723,7 @@ xlog_cil_push(*/ lv =NULL ; num_iovecs =0 ; +spin_lock (&cil->xc_cil_lock);while (!list_empty (&cil->xc_cil)) {struct xfs_log_item *item;
@@ -737,6 +738,7 @@ xlog_cil_push( item->li_lv =NULL ; num_iovecs += lv->lv_niovecs; } +spin_unlock (&cil->xc_cil_lock);/* * initialise the new context and attach it to the CIL. Then attach
Signature · 1 line
2.7.5
Quoted 24 lines · Pingfan Liu
xlog_cil_push()is the reader and writer ofxc_cil, and should be protected againstxlog_cil_insert_items().Signed-off-by Pingfan Liu <[email protected]>Cc "Darrick J. Wong" <[email protected]>To: [email protected]
Cc: [email protected]--- fs/xfs/xfs_log_cil.c | 2 ++ 1 file changed, 2 insertions(+)fs/xfs/xfs_log_cil.c +1
-0@@ -723,6 +723,7 @@ xlog_cil_push(*/ lv =NULL ; num_iovecs =0 ; +spin_lock (&cil->xc_cil_lock);while (!list_empty (&cil->xc_cil)) {struct xfs_log_item *item;
fs/xfs/xfs_log_cil.c xlog_cil_push( · line 741
Quoted 4 lines
fs/xfs/xfs_log_cil.c +1
-0@@ -737,6 +738,7 @@ xlog_cil_push( item->li_lv =NULL ; num_iovecs += lv->lv_niovecs; } +spin_unlock (&cil->xc_cil_lock);
The majority of this function executes under exclusive ->xc_ctx_lock. xlog_cil_insert_items() runs with the ->xc_ctx_lock taken in read mode. The ->xc_cil_lock spinlock is used in the latter case to protect the list under concurrent transaction commits.
Brian
Quoted 6 lines
fs/xfs/xfs_log_cil.c +0
-0/* * initialise the new context and attach it to the CIL. Then attach Signature · 1 line
2.7.5
Quoted 24 lines · Brian Foster
Quoted 23 lines · Pingfan Liu
xlog_cil_push()is the reader and writer ofxc_cil, and should be protected againstxlog_cil_insert_items().Signed-off-by Pingfan Liu <[email protected]>Cc "Darrick J. Wong" <[email protected]>To: [email protected]
Cc: [email protected]--- fs/xfs/xfs_log_cil.c | 2 ++ 1 file changed, 2 insertions(+)fs/xfs/xfs_log_cil.c +1
-0@@ -723,6 +723,7 @@ xlog_cil_push(*/ lv =NULL ; num_iovecs =0 ; +spin_lock (&cil->xc_cil_lock);while (!list_empty (&cil->xc_cil)) {struct xfs_log_item *item;
Quoted 11 lines
fs/xfs/xfs_log_cil.c xlog_cil_push( · line 741
Quoted 5 lines
fs/xfs/xfs_log_cil.c +1
-0@@ -737,6 +738,7 @@ xlog_cil_push( item->li_lv =NULL ; num_iovecs += lv->lv_niovecs; } +spin_unlock (&cil->xc_cil_lock);The majority of this function executes under exclusive ->xc_ctx_lock.
xlog_cil_insert_items()runs with the ->xc_ctx_lock taken in read mode. The ->xc_cil_lock spinlock is used in the latter case to protect the list under concurrent transaction commits.
I think the logic of xc_ctx_lock should be at a higher level of file system. But on the fundamental level, reader and writer should be protected against each other. And there is no protection for the list ops here.
BTW, even spinlock is not enough to protect integrity of a trans, and I think another patch should be involved. I will send the extra patch, which is applied on this one.
Thanks for your kindly review.
Regards,
Pingfan
PATCH 1/1 xfs/log: protect the logging content under xc_ctx_lock
xc_cil_lock is not enough to protect the integrity of a trans logging. Taking the scenario:
cpuA cpuB cpuC
xlog_cil_insert_format_items()
spin_lock(&cil->xc_cil_lock)
link transA's items to xc_cil,
including item1
spin_unlock(&cil->xc_cil_lock)
xlog_cil_push() fetches transA's item under xc_cil_lock
issue transB, modify item1
xlog_write(), but now, item1 contains content from transB and we have a broken transA
Survive this race issue by putting under the protection of xc_ctx_lock. Meanwhile the xc_cil_lock can be dropped as xc_ctx_lock does it against xlog_cil_insert_items()
To: [email protected]
Cc: [email protected]
--- fs/xfs/xfs_log_cil.c | 35 +++++++++++++++++++---------------- 1 file changed, 19 insertions(+), 16 deletions(-)
fs/xfs/xfs_log_cil.c +19 -16
@@ -723,22 +723,6 @@ xlog_cil_push(*/ lv =NULL ; num_iovecs =0 ; -spin_lock (&cil->xc_cil_lock); -while (!list_empty (&cil->xc_cil)) { -struct xfs_log_item *item; - - item =list_first_entry (&cil->xc_cil, - struct xfs_log_item, li_cil); -list_del_init (&item->li_cil); -if (!ctx->lv_chain) - ctx->lv_chain = item->li_lv; -else - lv->lv_next = item->li_lv; - lv = item->li_lv; - item->li_lv =NULL ; - num_iovecs += lv->lv_niovecs; - } -spin_unlock (&cil->xc_cil_lock);/* * initialise the new context and attach it to the CIL. Then attach
@@ -783,6 +767,25 @@ xlog_cil_push(up_write (&cil->xc_ctx_lock);/* +* cil->xc_cil_lock around this loop can be dropped, since xc_ctx_lock +* protects us against xlog_cil_insert_items(). +*/ +while (!list_empty (&cil->xc_cil)) { +struct xfs_log_item *item; + + item =list_first_entry (&cil->xc_cil, + struct xfs_log_item, li_cil); +list_del_init (&item->li_cil); +if (!ctx->lv_chain) + ctx->lv_chain = item->li_lv; +else + lv->lv_next = item->li_lv; + lv = item->li_lv; + item->li_lv =NULL ; + num_iovecs += lv->lv_niovecs; + } + +/* * Build a checkpoint transaction header and write it to the log to * begin the transaction. We need to account for the space used by the * transaction header here as it is not accounted for in xlog_write().
Signature · 1 line
2.7.5
Re: [PATCH] xfs/log: protect the logging content under xc_ctx_lock
Quoted 14 lines · Pingfan Liu
xc_cil_lockis not enough to protect the integrity of a trans logging. Taking the scenario:
cpuA cpuB cpuC
xlog_cil_insert_format_items()
spin_lock(&cil->xc_cil_lock)
link transA's items toxc_cil,
including item1
spin_unlock(&cil->xc_cil_lock)
xlog_cil_push()fetches transA's item underxc_cil_lock
issue transB, modify item1
xlog_write(), but now, item1 contains content from transB and we have a broken transA
Quoted 3 lines
Survive this race issue by putting under the protection of
xc_ctx_lock. Meanwhile thexc_cil_lockcan be dropped asxc_ctx_lockdoes it againstxlog_cil_insert_items()
How did you trigger this race? Is there a test case to reproduce, or did you figure this out via code inspection?
--D
Quoted 66 lines
Signed-off-by Pingfan Liu <[email protected]>Cc "Darrick J. Wong" <[email protected]>Cc Brian Foster <[email protected]>To: [email protected]
Cc: [email protected]--- fs/xfs/xfs_log_cil.c | 35 +++++++++++++++++++---------------- 1 file changed, 19 insertions(+), 16 deletions(-)fs/xfs/xfs_log_cil.c +19
-16@@ -723,22 +723,6 @@ xlog_cil_push(*/ lv =NULL ; num_iovecs =0 ; -spin_lock (&cil->xc_cil_lock); -while (!list_empty (&cil->xc_cil)) { -struct xfs_log_item *item; - - item =list_first_entry (&cil->xc_cil, - struct xfs_log_item, li_cil); -list_del_init (&item->li_cil); -if (!ctx->lv_chain) - ctx->lv_chain = item->li_lv; -else - lv->lv_next = item->li_lv; - lv = item->li_lv; - item->li_lv =NULL ; - num_iovecs += lv->lv_niovecs; - } -spin_unlock (&cil->xc_cil_lock);/* * initialise the new context and attach it to the CIL. Then attach @@ -783,6 +767,25 @@ xlog_cil_push(up_write (&cil->xc_ctx_lock);/* +* cil->xc_cil_lock around this loop can be dropped, since xc_ctx_lock +* protects us against xlog_cil_insert_items(). +*/ +while (!list_empty (&cil->xc_cil)) { +struct xfs_log_item *item; + + item =list_first_entry (&cil->xc_cil, + struct xfs_log_item, li_cil); +list_del_init (&item->li_cil); +if (!ctx->lv_chain) + ctx->lv_chain = item->li_lv; +else + lv->lv_next = item->li_lv; + lv = item->li_lv; + item->li_lv =NULL ; + num_iovecs += lv->lv_niovecs; + } + +/* * Build a checkpoint transaction header and write it to the log to * begin the transaction. We need to account for the space used by the * transaction header here as it is not accounted for in xlog_write(). Signature · 1 line
2.7.5
Re: [PATCH] xfs/log: protect the logging content under xc_ctx_lock
Quoted 15 lines · Darrick J. Wong
Quoted 14 lines · Pingfan Liu
xc_cil_lockis not enough to protect the integrity of a trans logging. Taking the scenario:
cpuA cpuB cpuC
xlog_cil_insert_format_items()
spin_lock(&cil->xc_cil_lock)
link transA's items toxc_cil,
including item1
spin_unlock(&cil->xc_cil_lock)
xlog_cil_push()fetches transA's item underxc_cil_lock
issue transB, modify item1
xlog_write(), but now, item1 contains content from transB and we have a broken transA
Quoted 7 lines
Quoted 3 lines
Survive this race issue by putting under the protection of
xc_ctx_lock. Meanwhile thexc_cil_lockcan be dropped asxc_ctx_lockdoes it againstxlog_cil_insert_items()How did you trigger this race? Is there a test case to reproduce, or did you figure this out via code inspection?
Via code inspection. To hit this bug, the condition is hard to meet:
a broken transA is written to disk, then system encounters a failure before transB is written. Only if this happens, the recovery will bring us to a broken context.
Regards,
Pingfan
Re: [PATCH] xfs/log: protect the logging content under xc_ctx_lock
In reply to Pingfan LiuDropped linux-fsdevel from cc. There's no reason to spam -fsdevel with low level XFS patches.
Quoted 10 lines · Pingfan Liu
xc_cil_lockis not enough to protect the integrity of a trans logging. Taking the scenario:
cpuA cpuB cpuC
xlog_cil_insert_format_items()
spin_lock(&cil->xc_cil_lock)
link transA's items toxc_cil,
including item1
spin_unlock(&cil->xc_cil_lock)
So you commit a transaction, item1 ends up on the CIL.
Quoted 1 line
xlog_cil_push()fetches transA's item underxc_cil_lock
xlog_cil_push() doesn't use ->xc_cil_lock, so I'm not sure what this means. This sequence executes under ->xc_ctx_lock in write mode, which locks out all transaction committers.
Quoted 1 line
issue transB, modify item1
So presumably transB joins item1 while it is on the CIL from trans A and commits.
Quoted 1 line
xlog_write(), but now, item1 contains content from transB and we have a broken transA
I'm not following how this is possible. The CIL push above, under exclusive lock, removes each log item from ->xc_cil and pulls the log vectors off of the log items to form the lv chain on the CIL context. This means that the transB commit either updates the lv attached to the log item from transA with the latest in-core version or uses the new shadow buffer allocated in the commit path of transB. Either way is fine because there is no guarantee of per-transaction granularity in the on-disk log. The purpose of the on-disk log is to guarantee filesystem consistency after a crash.
All in all, I can't really tell what problem you're describing here. If you believe there's an issue in this code, I'd suggest to either try and instrument it manually to reproduce a demonstrable problem and/or provide far more detailed of a description to explain it.
Quoted 11 lines
Survive this race issue by putting under the protection of
xc_ctx_lock. Meanwhile thexc_cil_lockcan be dropped asxc_ctx_lockdoes it againstxlog_cil_insert_items()Signed-off-by Pingfan Liu <[email protected]>Cc "Darrick J. Wong" <[email protected]>Cc Brian Foster <[email protected]>To: [email protected]
Cc: [email protected]
---
FYI, this patch also doesn't apply to for-next. I'm guessing because it's based on your previous patch to add the spinlock around the loop.
fs/xfs/xfs_log_cil.c xlog_cil_push( · line 727
Quoted 13 lines
fs/xfs/xfs_log_cil.c | 35 +++++++++++++++++++---------------- 1 file changed, 19 insertions(+), 16 deletions(-)fs/xfs/xfs_log_cil.c +0
-2@@ -723,22 +723,6 @@ xlog_cil_push(*/ lv =NULL ; num_iovecs =0 ; -spin_lock (&cil->xc_cil_lock); -while (!list_empty (&cil->xc_cil)) {
There's a comment just above that documents this loop that isn't moved/modified.
Quoted 29 lines
fs/xfs/xfs_log_cil.c +8
-14-struct xfs_log_item *item; - - item =list_first_entry (&cil->xc_cil, - struct xfs_log_item, li_cil); -list_del_init (&item->li_cil); -if (!ctx->lv_chain) - ctx->lv_chain = item->li_lv; -else - lv->lv_next = item->li_lv; - lv = item->li_lv; - item->li_lv =NULL ; - num_iovecs += lv->lv_niovecs; - } -spin_unlock (&cil->xc_cil_lock);/* * initialise the new context and attach it to the CIL. Then attach @@ -783,6 +767,25 @@ xlog_cil_push(up_write (&cil->xc_ctx_lock);/* +* cil->xc_cil_lock around this loop can be dropped, since xc_ctx_lock +* protects us against xlog_cil_insert_items(). +*/ +while (!list_empty (&cil->xc_cil)) { +struct xfs_log_item *item; + + item =list_first_entry (&cil->xc_cil, + struct xfs_log_item, li_cil);
fs/xfs/xfs_log_cil.c xlog_cil_push( · line 787
Quoted 10 lines
fs/xfs/xfs_log_cil.c +10
-0+list_del_init (&item->li_cil); +if (!ctx->lv_chain) + ctx->lv_chain = item->li_lv; +else + lv->lv_next = item->li_lv; + lv = item->li_lv; + item->li_lv =NULL ; + num_iovecs += lv->lv_niovecs; + } +
This places the associated loop outside of ->xc_ctx_lock, which means we can now race modifying ->xc_cil during a CIL push and a transaction commit. Have you tested this?
Brian
Quoted 7 lines
fs/xfs/xfs_log_cil.c +1
-0+/* * Build a checkpoint transaction header and write it to the log to * begin the transaction. We need to account for the space used by the * transaction header here as it is not accounted for in xlog_write(). Signature · 1 line
2.7.5
Re: [PATCH] xfs/log: protect the logging content under xc_ctx_lock
Quoted 4 lines · Brian Foster
Dropped linux-fsdevel from cc. There's no reason to spam -fsdevel with low level XFS patches.
On Wed, Oct 30, 2019 at 09:37:11PM +0800, Pingfan Liu wrote:
[...]
Quoted 2 lines
I'm not following how this is possible. The CIL push above, under
It turns out not to be a bug as I replied to Dave's mail in this thread.
Quoted 6 lines
exclusive lock, removes each log item from ->xc_cil and pulls the log vectors off of the log items to form the lv chain on the CIL context. This means that the transB commit either updates the lv attached to the log item from transA with the latest in-core version or uses the new shadow buffer allocated in the commit path of transB. Either way is fine because there is no guarantee of per-transaction granularity in the
Yes, no guarantee of per-transaction granularity, but there is a boundary placed on several effect-merged transactions. That is what xc_ctx_lock and private chain ctx->lv_chain guarantee.
Quoted 8 lines
on-disk log. The purpose of the on-disk log is to guarantee filesystem consistency after a crash.
All in all, I can't really tell what problem you're describing here. If you believe there's an issue in this code, I'd suggest to either try and instrument it manually to reproduce a demonstrable problem and/or provide far more detailed of a description to explain it.
Sorry that I raise a false alarm. But thank you all for helping me to figure out through this.
Regards,
Pingfan
Re: [PATCH] xfs/log: protect the logging content under xc_ctx_lock
In reply to Pingfan LiuQuoted 6 lines · Pingfan Liu
xc_cil_lockis not enough to protect the integrity of a trans logging. Taking the scenario:
cpuA cpuB cpuC
xlog_cil_insert_format_items()
Quoted 7 lines
spin_lock(&cil->xc_cil_lock)
link transA's items toxc_cil,
including item1
spin_unlock(&cil->xc_cil_lock)
xlog_cil_push()fetches transA's item underxc_cil_lock
issue transB, modify item1
xlog_write(), but now, item1 contains content from transB and we have a broken transA
TL;DR: 1. log vectors. 2. CIL context lock exclusion.
When CPU A formats the item during commit, it copies all the changes into a list of log vectors, and that is attached to the log item and the item is added to the CIL. The item is then unlocked. This is done with the CIL context lock held excluding CIL pushes.
When CPU C pushes on the CIL, it detatches the -log vectors- from the log item and removes the item from the CIL. This is done hold the CIL context lock, excluding transaction commits from modifying the CIL log vector list. It then formats the -log vectors- into the journal by passing them to xlog_write(). It does not use log items for this, and because the log vector list has been isolated and is now private to the push context, we don't need to hold any locks anymore to call xlog_write....
When CPU B modifies item1, it modifies the item and logs the new changes to the log item. It does not modify the log vector that might be attached to the log item from a previous change. The log vector is only updated during transaction commit, so the changes being made in transaction on CPU B are private to that transaction until they are committed, formatted into log vectors and inserted into the CIL under the CIL context lock.
Quoted 41 lines
Survive this race issue by putting under the protection of
xc_ctx_lock. Meanwhile thexc_cil_lockcan be dropped asxc_ctx_lockdoes it againstxlog_cil_insert_items()Signed-off-by Pingfan Liu <[email protected]>Cc "Darrick J. Wong" <[email protected]>Cc Brian Foster <[email protected]>To: [email protected]
Cc: [email protected]--- fs/xfs/xfs_log_cil.c | 35 +++++++++++++++++++---------------- 1 file changed, 19 insertions(+), 16 deletions(-)fs/xfs/xfs_log_cil.c +0
-16@@ -723,22 +723,6 @@ xlog_cil_push(*/ lv =NULL ; num_iovecs =0 ; -spin_lock (&cil->xc_cil_lock); -while (!list_empty (&cil->xc_cil)) { -struct xfs_log_item *item; - - item =list_first_entry (&cil->xc_cil, - struct xfs_log_item, li_cil); -list_del_init (&item->li_cil); -if (!ctx->lv_chain) - ctx->lv_chain = item->li_lv; -else - lv->lv_next = item->li_lv; - lv = item->li_lv; - item->li_lv =NULL ; - num_iovecs += lv->lv_niovecs; - } -spin_unlock (&cil->xc_cil_lock);/* * initialise the new context and attach it to the CIL. Then attach
fs/xfs/xfs_log_cil.c xlog_cil_push( · line 767
Quoted 1 line
fs/xfs/xfs_log_cil.c +0
-0@@ -783,6 +767,25 @@ xlog_cil_push(up_write (&cil->xc_ctx_lock);
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
We don't hold the CIL context lock anymore....
Quoted 9 lines
fs/xfs/xfs_log_cil.c +7
-0/* +* cil->xc_cil_lock around this loop can be dropped, since xc_ctx_lock +* protects us against xlog_cil_insert_items(). +*/ +while (!list_empty (&cil->xc_cil)) { +struct xfs_log_item *item; + + item = list_first_entry(&cil->xc_cil,
fs/xfs/xfs_log_cil.c xlog_cil_push( · line 786
Quoted 10 lines
fs/xfs/xfs_log_cil.c +10
-0+ struct xfs_log_item, li_cil); +list_del_init (&item->li_cil); +if (!ctx->lv_chain) + ctx->lv_chain = item->li_lv; +else + lv->lv_next = item->li_lv; + lv = item->li_lv; + item->li_lv =NULL ; + num_iovecs += lv->lv_niovecs; + }
So this is completely unserialised now. i.e. even if there was a problem like you suggest, this modification doesn't do what you say it does.
Cheers,
Dave.
Signature · 2 lines
Dave Chinner [email protected]
Re: [PATCH] xfs/log: protect the logging content under xc_ctx_lock
Quoted 22 lines · Dave Chinner
Quoted 6 lines · Pingfan Liu
xc_cil_lockis not enough to protect the integrity of a trans logging. Taking the scenario:
cpuA cpuB cpuC
xlog_cil_insert_format_items()Quoted 7 lines
spin_lock(&cil->xc_cil_lock)
link transA's items toxc_cil,
including item1
spin_unlock(&cil->xc_cil_lock)
xlog_cil_push()fetches transA's item underxc_cil_lock
issue transB, modify item1
xlog_write(), but now, item1 contains content from transB and we have a broken transATL;DR: 1. log vectors. 2. CIL context lock exclusion.
When CPU A formats the item during commit, it copies all the changes into a list of log vectors, and that is attached to the log item and the item is added to the CIL. The item is then unlocked. This is done with the CIL context lock held excluding CIL pushes.
Quoted 8 lines
When CPU C pushes on the CIL, it detatches the -log vectors- from the log item and removes the item from the CIL. This is done hold the CIL context lock, excluding transaction commits from modifying the CIL log vector list. It then formats the -log vectors- into the journal by passing them to
xlog_write(). It does not use log items for this, and because the log vector list has been isolated and is now private to the push context, we don't need to hold any locks anymore to callxlog_write....
Yes. I failed to realize it. The critical "item->li_lv = NULL" in xlog_cil_push(), which isolates the vectors and free of new modification even after releasing xc_ctx_lock.
[...]
Quoted 7 lines
fs/xfs/xfs_log_cil.c xlog_cil_push( · line 767
Quoted 3 lines
fs/xfs/xfs_log_cil.c +0
-0* initialise the new context and attach it to the CIL. Then attach @@ -783,6 +767,25 @@ xlog_cil_push(up_write (&cil->xc_ctx_lock);^^^^^^^^^^^^^^^^^^^^^^^^^^^^We don't hold the CIL context lock anymore....
Doze on it, make a mistaken reverse recognition of the up/down meaning.
Thank you for very patient and detailed explain. I get a full understanding now.
Regards,
Pingfan
Quoted 36 lines · Pingfan Liu
Quoted 24 lines · Brian Foster
Quoted 23 lines · Pingfan Liu
xlog_cil_push()is the reader and writer ofxc_cil, and should be protected againstxlog_cil_insert_items().Signed-off-by Pingfan Liu <[email protected]>Cc "Darrick J. Wong" <[email protected]>To: [email protected]
Cc: [email protected]--- fs/xfs/xfs_log_cil.c | 2 ++ 1 file changed, 2 insertions(+)fs/xfs/xfs_log_cil.c +1
-0@@ -723,6 +723,7 @@ xlog_cil_push(*/ lv =NULL ; num_iovecs =0 ; +spin_lock (&cil->xc_cil_lock);while (!list_empty (&cil->xc_cil)) {struct xfs_log_item *item;Quoted 11 lines
fs/xfs/xfs_log_cil.c xlog_cil_push( · line 741
Quoted 5 lines
fs/xfs/xfs_log_cil.c +1
-0@@ -737,6 +738,7 @@ xlog_cil_push( item->li_lv =NULL ; num_iovecs += lv->lv_niovecs; } +spin_unlock (&cil->xc_cil_lock);The majority of this function executes under exclusive ->xc_ctx_lock.
xlog_cil_insert_items()runs with the ->xc_ctx_lock taken in read mode. The ->xc_cil_lock spinlock is used in the latter case to protect the list under concurrent transaction commits.
Quoted 4 lines
I think the logic of
xc_ctx_lockshould be at a higher level of file system. But on the fundamental level, reader and writer should be protected against each other. And there is no protection for the list ops here.
Yes there is. The locking here is complex and unique, so takes some understanding.
These are two different sets of operations that are being serialised
- high level operation is that transaction commits can run concurrently (and must for performance), while CIL pushes must run exclusively (for correctness).
So, yes, there is only one data structure we are accessing here and it has two locks protecting it. They _nest_ to provide different levels of exclusion: multiple producers vs single consumer via a rwsem, and producer vs producer via a spin lock inside the shared rwsem context. i.e.:
commit 1 commit 2 push
down_read(ctx_lock)
down_read(ctx_lock)
down_write(ctx_lock) <blocks>spin_lock(cil_lock)
add to CIL spin_lock(cil_lock)
<spins>
spin_unlock(cil_lock)
<gets cil_lock)
add to CILup_read(ctx_lock)
spin_unlock(cil_lock) up_read(ctx_lock)
<gets ctx_lock> Processes CIL
As you can see, the CIL can only be accessed by a single thread at a time, despite the fact there are multiple locks involved.
And the implied unlock->lock memory barriers ensure that list state does not leak incorrectly between the different contexts....
Cheers,
Dave.
Signature · 2 lines
Dave Chinner [email protected]