xfs: grant lock scaling and removal V3
This series addresses the log grant lock contention seen by 8-way fs_mark workloads. The log grant lock protects:
- reserve grant head and wait queue
- write grant head and wait queue
- tail lsn
- last sync lsn
While they are all currently protected by a single lock, there is no reason that they need to all be under the same lock. As a result, one option for scaling was simply to split the grant lock into three locks - one for each of the above groups. However, this would mean that we'd need to nest locks inside each other and it ignores the fact that we really only use the lock on the tail and last sync lsn variables to protect against concurrent updates.
Hence we can make the tail and last sync LSN variables independent of the grant lock by making them atomic variables. This means that when we are moving the tail of the log, we can avoid all locking except when there are waiters queued on the grant heads.
Making the grant heads scale better is a bit more of a challenge. Just replacing the grant lock with reserve and write grant locks doesn't really help improve scalability because we'd still need to take both locks in the hot xfs_log_reserve() path. To improve scalability, we really need to make this path lock free.
The first steps aree to clean up some of the code. We convert the ticket queues to use the common list_head infrastructure, factor out some common debug code, refactor and rearrange the grant head calculations code and convert all the users of the sv_t wait mechanisms to use wait queues directly.
The second step to acheiving this is to encode the grant heads as a 64 bit variable and then convert it to an atomic variable. The tail/last sync LSNs also get converted to atomic variables, and this means we can read the grant heads without holding locks and that allows tail pushing calculations and available log space calculations to operate lock free.
The next step is to introduce a lock per grant queue that is used exclusively to protect queue manpulations. With the use of list_empty_careful() we can check whether the queue has waiters without holding the queue lock. Hence in the case where the queues are empty we do not need to take the queue locks in the fast path.
Finally, we need to make the grant head space calculations lockless. With the grant heads already being atomic variables, we can change the calculation algorithm to a lockless cmpxchg algorithm. This means we no longer need any spinlocks in the transaction reserve fast path and hence the scalability of this path should be significantly improved.
There is one down side to this change - the xlog_verify_head() debug code can no longer be reliably used to detect accounting problems in the grant space allocations as it requires an atomic sample of both grant heads. However, the tail verification and the xlog_space_left() verification still works without problems, so we still have some debug checking on the grant head locations.
Version 3:
- dropped cleanup of xlog_grant_log_space() and xlog_regrant_log_write_space().
- split grant head aggregation into multiple patches
- split out xlog_verify_tail() function
- factor grant head calculations and drop wrappers
- combine grant heads and add wrappers to crack/combine grant heads.
- removed intermediate grant head "_lsn" suffix name.
- folded all sv_t removal patches into one.
- don't pass tail and last sync lsn into xlog_grant_push_ail().
- ensure that shutdown checks in ticket queue processing are done consistently before sleeping.
- removed xlog_grant_verify_head()
- folded grant lock removal into patch that converts grant head manipulations to lockless algorithms.
- added a couple of tracepoints for when the log tail is moved and queued tickets are woken to aid debugging.
Version 2:
- split into lots of patches
- clean up the code and comments
- add patches to clean up sv_t usage at the end of the series
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
PATCH 1/12 xfs: convert log grant ticket queues to list heads
From: Dave Chinner <[email protected]>
The grant write and reserve queues use a roll-your-own double linked list, so convert it to a standard list_head structure and convert all the list traversals to use list_for_each_entry(). We can also get rid of the XLOG_TIC_IN_Q flag as we can use the list_empty() check to tell if the ticket is in a list or not.
--- fs/xfs/linux-2.6/xfs_trace.h | 12 ++-- fs/xfs/xfs_log.c | 123 ++++++++++++++---------------------------- fs/xfs/xfs_log_priv.h | 11 ++--- 3 files changed, 51 insertions(+), 95 deletions(-)
fs/xfs/linux-2.6/xfs_trace.h +6 -6
@@ -766,8 +766,8 @@ DECLARE_EVENT_CLASS(xfs_loggrant_class, __field(int, curr_res) __field(int, unit_res) __field(unsigned int, flags) - __field(void *, reserve_headq) - __field(void *, write_headq) + __field(void *, reserveq) + __field(void *, writeq) __field(int, grant_reserve_cycle) __field(int, grant_reserve_bytes) __field(int, grant_write_cycle)
@@ -784,8 +784,8 @@ DECLARE_EVENT_CLASS(xfs_loggrant_class, __entry->curr_res = tic->t_curr_res; __entry->unit_res = tic->t_unit_res; __entry->flags = tic->t_flags; - __entry->reserve_headq = log->l_reserve_headq; - __entry->write_headq = log->l_write_headq; + __entry->reserveq = log->l_reserveq.next; + __entry->writeq = log->l_writeq.next; __entry->grant_reserve_cycle = log->l_grant_reserve_cycle; __entry->grant_reserve_bytes = log->l_grant_reserve_bytes; __entry->grant_write_cycle = log->l_grant_write_cycle;
@@ -807,8 +807,8 @@ DECLARE_EVENT_CLASS(xfs_loggrant_class, __entry->curr_res, __entry->unit_res,__print_flags (__entry->flags,"|" ,XLOG_TIC_FLAGS ), - __entry->reserve_headq, - __entry->write_headq, + __entry->reserveq, + __entry->writeq, __entry->grant_reserve_cycle, __entry->grant_reserve_bytes, __entry->grant_write_cycle,
fs/xfs/xfs_log.c +41 -82
243 lines not on this page · raw message
fs/xfs/xfs_log_priv.h +4 -7
@@ -133,12 +133,10 @@ static inline uint xlog_get_client_id(__be32 i)*/ #define XLOG_TIC_INITED 0x1/* has been initialized */ #define XLOG_TIC_PERM_RESERV 0x2/* permanent reservation */ -#define XLOG_TIC_IN_Q 0x4#define XLOG_TIC_FLAGS \ { XLOG_TIC_INITED, "XLOG_TIC_INITED" }, \ - { XLOG_TIC_PERM_RESERV, "XLOG_TIC_PERM_RESERV" }, \ - { XLOG_TIC_IN_Q, "XLOG_TIC_IN_Q" } + { XLOG_TIC_PERM_RESERV, "XLOG_TIC_PERM_RESERV" }#endif /* __KERNEL__ */
@@ -245,8 +243,7 @@ typedef struct xlog_res { typedef struct xlog_ticket { sv_t t_wait; /* ticket wait queue : 20 */ - struct xlog_ticket *t_next; /* :4|8 */ - struct xlog_ticket *t_prev; /* :4|8 */ + struct list_head t_queue; /* reserve/write queue */ xlog_tid_t t_tid; /* transaction identifier : 4 */ atomic_t t_ref; /* ticket reference count : 4 */ int t_curr_res; /* current reservation in bytes : 4 */
@@ -520,8 +517,8 @@ typedef struct log {/* The following block of fields are changed while holding grant_lock */ spinlock_t l_grant_lock____cacheline_aligned_in_smp ; -xlog_ticket_t *l_reserve_headq; -xlog_ticket_t *l_write_headq; +struct list_head l_reserveq; +struct list_head l_writeq;int l_grant_reserve_cycle;int l_grant_reserve_bytes;int l_grant_write_cycle;
Signature · 6 lines
1.7.2.3 _______________________________________________ xfs mailing list [email protected] http://oss.sgi.com/mailman/listinfo/xfs
Re: [PATCH 01/12] xfs: convert log grant ticket queues to list heads
fs/xfs/linux-2.6/xfs_trace.h DECLARE_EVENT_CLASS(xfs_loggrant_class, · line 770
Quoted 4 lines · Dave Chinner
fs/xfs/linux-2.6/xfs_trace.h +2
-2- __field(void *, reserve_headq) -__field (void *, write_headq) + __field(void *, reserveq) +__field (void *, writeq)
To repeat my question from the last review: what's the point in logging this at all? There's not much we can do with it from trace-cmd / perf output. What might be more useful is a list_empty() boolean flag.
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
Re: [PATCH 01/12] xfs: convert log grant ticket queues to list heads
Quoted 9 lines · Christoph Hellwig
fs/xfs/linux-2.6/xfs_trace.h DECLARE_EVENT_CLASS(xfs_loggrant_class, · line 770
Quoted 4 lines · Dave Chinner
fs/xfs/linux-2.6/xfs_trace.h +2
-2- __field(void *, reserve_headq) -__field (void *, write_headq) + __field(void *, reserveq) +__field (void *, writeq)To repeat my question from the last review: what's the point in logging this at all? There's not much we can do with it from trace-cmd / perf output. What might be more useful is a
list_empty()boolean flag.
Oh, I missed that when running through all the comments. Thanks for pointing it out again - I'll fix it this time.
Cheers,
Dave.
Signature · 7 lines
Dave Chinner [email protected] _______________________________________________ xfs mailing list [email protected] http://oss.sgi.com/mailman/listinfo/xfs
PATCH 2/12 xfs: fact out common grant head/log tail verification code
In reply to Dave ChinnerFrom: Dave Chinner <[email protected]>
Factor repeated debug code out of grant head manipulation functions into a separate function. This removes ifdef DEBUG spagetti from the code and makes the code easier to follow.
--- fs/xfs/xfs_log.c | 51 ++++++++++++++++++++++----------------------------- 1 files changed, 22 insertions(+), 29 deletions(-)
fs/xfs/xfs_log.c +22 -29
@@ -82,6 +82,7 @@ STATIC void xlog_ungrant_log_space(xlog_t *log,#if defined(DEBUG ) STATIC void xlog_verify_dest_ptr(xlog_t *log, char *ptr); STATIC void xlog_verify_grant_head(xlog_t *log, int equals); +STATIC void xlog_verify_grant_tail(struct log *log); STATIC void xlog_verify_iclog(xlog_t *log, xlog_in_core_t *iclog,int count,boolean_t syncing); STATIC void xlog_verify_tail_lsn(xlog_t *log, xlog_in_core_t *iclog,
@@ -89,6 +90,7 @@ STATIC void xlog_verify_tail_lsn(xlog_t *log, xlog_in_core_t *iclog,#else #define xlog_verify_dest_ptr (a,b)#define xlog_verify_grant_head (a,b) +#define xlog_verify_grant_tail (a)#define xlog_verify_iclog (a,b,c,d)#define xlog_verify_tail_lsn (a,b,c)#endif
@@ -2503,10 +2505,6 @@ xlog_grant_log_space(xlog_t *log, {int free_bytes;int need_bytes; -#ifdef DEBUG -xfs_lsn_t tail_lsn; -#endif -#ifdef DEBUG if (log->l_flags &XLOG_ACTIVE_RECOVERY )
@@ -2577,21 +2575,9 @@ redo:/* we've got enough space */ xlog_grant_add_space (log, need_bytes); -#ifdef DEBUG - tail_lsn = log->l_tail_lsn; -/* -* Check to make sure the grant write head didn't just over lap the -* tail. If the cycles are the same, we can't be overlapping. -* Otherwise, make sure that the cycles differ by exactly one and -* check the byte count. -*/ -if (CYCLE_LSN (tail_lsn) != log->l_grant_write_cycle) { -ASSERT (log->l_grant_write_cycle-1 ==CYCLE_LSN (tail_lsn)); -ASSERT (log->l_grant_write_bytes <=BBTOB (BLOCK_LSN (tail_lsn))); - } -#endif trace_xfs_log_grant_exit (log, tic);xlog_verify_grant_head (log,1 ); +xlog_verify_grant_tail (log);spin_unlock (&log->l_grant_lock);return 0 ;
@@ -2621,9 +2607,6 @@ xlog_regrant_write_log_space(xlog_t *log, xlog_ticket_t *tic) {int free_bytes, need_bytes; -#ifdef DEBUG -xfs_lsn_t tail_lsn; -#endif tic->t_curr_res = tic->t_unit_res;xlog_tic_reset_res (tic);
@@ -2719,17 +2702,9 @@ redo:/* we've got enough space */ xlog_grant_add_space_write (log, need_bytes); -#ifdef DEBUG - tail_lsn = log->l_tail_lsn; -if (CYCLE_LSN (tail_lsn) != log->l_grant_write_cycle) { -ASSERT (log->l_grant_write_cycle-1 ==CYCLE_LSN (tail_lsn)); -ASSERT (log->l_grant_write_bytes <=BBTOB (BLOCK_LSN (tail_lsn))); - } -#endif -trace_xfs_log_regrant_write_exit (log, tic); -xlog_verify_grant_head (log,1 ); +xlog_verify_grant_tail (log);spin_unlock (&log->l_grant_lock);return 0 ;
@@ -3465,6 +3440,24 @@ xlog_verify_grant_head(xlog_t *log, int equals) } }/* xlog_verify_grant_head */ +STATIC void +xlog_verify_grant_tail ( +struct log *log) +{ +xfs_lsn_t tail_lsn = log->l_tail_lsn; + +/* +* Check to make sure the grant write head didn't just over lap the +* tail. If the cycles are the same, we can't be overlapping. +* Otherwise, make sure that the cycles differ by exactly one and +* check the byte count. +*/ +if (CYCLE_LSN (tail_lsn) != log->l_grant_write_cycle) { +ASSERT (log->l_grant_write_cycle -1 ==CYCLE_LSN (tail_lsn)); +ASSERT (log->l_grant_write_bytes <=BBTOB (BLOCK_LSN (tail_lsn))); + } +} +/* check if it will fit */ STATIC void xlog_verify_tail_lsn(xlog_t *log,
Signature · 6 lines
1.7.2.3 _______________________________________________ xfs mailing list [email protected] http://oss.sgi.com/mailman/listinfo/xfs
Re: [PATCH 02/12] xfs: fact out common grant head/log tail verification code
Quoted 5 lines · Dave Chinner
From: Dave Chinner <[email protected]>
Factor repeated debug code out of grant head manipulation functions into a separate function. This removes ifdef DEBUG spagetti from the code and makes the code easier to follow.
Looks good,
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
PATCH 3/12 xfs: rework log grant space calculations
In reply to Dave ChinnerFrom: Dave Chinner <[email protected]>
The log grant space calculations are repeated for both write and reserve grant heads. To make it simpler to convert the calculations toa different algorithm, factor them so both the gratn heads use the same calculation functions. Once this is done we can drop the wrappers that are used in only a couple of place to update both grant heads at once as they don't provide any particular value.
--- fs/xfs/xfs_log.c | 95 +++++++++++++++++++++++++++-------------------------- 1 files changed, 48 insertions(+), 47 deletions(-)
fs/xfs/xfs_log.c +48 -47
@@ -98,53 +98,34 @@ STATIC void xlog_verify_tail_lsn(xlog_t *log, xlog_in_core_t *iclog, STATIC int xlog_iclogs_empty(xlog_t *log);static void -xlog_grant_sub_space (struct log *log,int bytes) -{ - log->l_grant_write_bytes -= bytes; -if (log->l_grant_write_bytes <0 ) { - log->l_grant_write_bytes += log->l_logsize; - log->l_grant_write_cycle--; - } - - log->l_grant_reserve_bytes -= bytes; -if ((log)->l_grant_reserve_bytes <0 ) { - log->l_grant_reserve_bytes += log->l_logsize; - log->l_grant_reserve_cycle--; - } - -} - -static void -xlog_grant_add_space_write (struct log *log,int bytes) +xlog_grant_sub_space ( +struct log *log, +int *cycle, +int *space, +int bytes) { -int tmp = log->l_logsize - log->l_grant_write_bytes; -if (tmp > bytes) - log->l_grant_write_bytes += bytes; -else { - log->l_grant_write_cycle++; - log->l_grant_write_bytes = bytes - tmp; + *space -= bytes; +if (*space <0 ) { + *space += log->l_logsize; + (*cycle)--; } }static void -xlog_grant_add_space_reserve (struct log *log,int bytes) +xlog_grant_add_space ( +struct log *log, +int *cycle, +int *space, +int bytes) { -int tmp = log->l_logsize - log->l_grant_reserve_bytes; +int tmp = log->l_logsize - *space;if (tmp > bytes) - log->l_grant_reserve_bytes += bytes; + *space += bytes;else { - log->l_grant_reserve_cycle++; - log->l_grant_reserve_bytes = bytes - tmp; + *space = bytes - tmp; + (*cycle)++; } } - -static inline void -xlog_grant_add_space (struct log *log,int bytes) -{ -xlog_grant_add_space_write (log, bytes); -xlog_grant_add_space_reserve (log, bytes); -} -static void xlog_tic_reset_res (xlog_ticket_t *tic) {
@@ -1344,7 +1325,10 @@ xlog_sync(xlog_t *log,/* move grant heads by roundoff in sync */ spin_lock (&log->l_grant_lock); -xlog_grant_add_space (log, roundoff); +xlog_grant_add_space (log, &log->l_grant_reserve_cycle, + &log->l_grant_reserve_bytes, roundoff); +xlog_grant_add_space (log, &log->l_grant_write_cycle, + &log->l_grant_write_bytes, roundoff);spin_unlock (&log->l_grant_lock);/* put cycle number in every block */
@@ -2574,7 +2558,10 @@ redo:list_del_init (&tic->t_queue);/* we've got enough space */ -xlog_grant_add_space (log, need_bytes); +xlog_grant_add_space (log, &log->l_grant_reserve_cycle, + &log->l_grant_reserve_bytes, need_bytes); +xlog_grant_add_space (log, &log->l_grant_write_cycle, + &log->l_grant_write_bytes, need_bytes);trace_xfs_log_grant_exit (log, tic);xlog_verify_grant_head (log,1 );xlog_verify_grant_tail (log);
@@ -2701,7 +2688,8 @@ redo:list_del_init (&tic->t_queue);/* we've got enough space */ -xlog_grant_add_space_write (log, need_bytes); +xlog_grant_add_space (log, &log->l_grant_write_cycle, + &log->l_grant_write_bytes, need_bytes);trace_xfs_log_regrant_write_exit (log, tic);xlog_verify_grant_head (log,1 );xlog_verify_grant_tail (log);
@@ -2742,7 +2730,12 @@ xlog_regrant_reserve_log_space(xlog_t *log, ticket->t_cnt--;spin_lock (&log->l_grant_lock); -xlog_grant_sub_space (log, ticket->t_curr_res); +xlog_grant_sub_space (log, &log->l_grant_reserve_cycle, + &log->l_grant_reserve_bytes, + ticket->t_curr_res); +xlog_grant_sub_space (log, &log->l_grant_write_cycle, + &log->l_grant_write_bytes, + ticket->t_curr_res); ticket->t_curr_res = ticket->t_unit_res;xlog_tic_reset_res (ticket);
@@ -2756,7 +2749,9 @@ xlog_regrant_reserve_log_space(xlog_t *log,return ; } -xlog_grant_add_space_reserve (log, ticket->t_unit_res); +xlog_grant_add_space (log, &log->l_grant_reserve_cycle, + &log->l_grant_reserve_bytes, + ticket->t_unit_res);trace_xfs_log_regrant_reserve_exit (log, ticket);
@@ -2785,24 +2780,30 @@ STATIC voidxlog_ungrant_log_space (xlog_t *log, xlog_ticket_t *ticket) { +int bytes; +if (ticket->t_cnt >0 ) ticket->t_cnt--;spin_lock (&log->l_grant_lock);trace_xfs_log_ungrant_enter (log, ticket); - -xlog_grant_sub_space (log, ticket->t_curr_res); -trace_xfs_log_ungrant_sub (log, ticket); -/* If this is a permanent reservation ticket, we may be able to free +/* +* If this is a permanent reservation ticket, we may be able to free * up more space based on the remaining count. */ + bytes = ticket->t_curr_res;if (ticket->t_cnt >0 ) {ASSERT (ticket->t_flags &XLOG_TIC_PERM_RESERV ); -xlog_grant_sub_space (log, ticket->t_unit_res*ticket->t_cnt); + bytes += ticket->t_unit_res*ticket->t_cnt; } +xlog_grant_sub_space (log, &log->l_grant_reserve_cycle, + &log->l_grant_reserve_bytes, bytes); +xlog_grant_sub_space (log, &log->l_grant_write_cycle, + &log->l_grant_write_bytes, bytes); +trace_xfs_log_ungrant_exit (log, ticket);xlog_verify_grant_head (log,1 );
Signature · 6 lines
1.7.2.3 _______________________________________________ xfs mailing list [email protected] http://oss.sgi.com/mailman/listinfo/xfs
Re: [PATCH 03/12] xfs: rework log grant space calculations
Looks good,
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
PATCH 4/12 xfs: combine grant heads into a single 64 bit integer
In reply to Dave ChinnerFrom: Dave Chinner <[email protected]>
Prepare for switching the grant heads to atomic variables by combining the two 32 bit values that make up the grant head into a single 64 bit variable. Provide wrapper functions to combine and split the grant heads appropriately for calculations and use them as necessary.
--- fs/xfs/linux-2.6/xfs_trace.h | 10 ++- fs/xfs/xfs_log.c | 166 ++++++++++++++++++++++-------------------- fs/xfs/xfs_log_priv.h | 26 ++++++- fs/xfs/xfs_log_recover.c | 8 +- 4 files changed, 119 insertions(+), 91 deletions(-)
fs/xfs/linux-2.6/xfs_trace.h +6 -4
@@ -786,10 +786,12 @@ DECLARE_EVENT_CLASS(xfs_loggrant_class, __entry->flags = tic->t_flags; __entry->reserveq = log->l_reserveq.next; __entry->writeq = log->l_writeq.next; - __entry->grant_reserve_cycle = log->l_grant_reserve_cycle; - __entry->grant_reserve_bytes = log->l_grant_reserve_bytes; - __entry->grant_write_cycle = log->l_grant_write_cycle; - __entry->grant_write_bytes = log->l_grant_write_bytes; +xlog_crack_grant_head (&log->l_grant_reserve_head, + &__entry->grant_reserve_cycle, + &__entry->grant_reserve_bytes); +xlog_crack_grant_head (&log->l_grant_write_head, + &__entry->grant_write_cycle, + &__entry->grant_write_bytes); __entry->curr_cycle = log->l_curr_cycle; __entry->curr_block = log->l_curr_block; __entry->tail_lsn = log->l_tail_lsn;
fs/xfs/xfs_log.c +87 -79
315 lines not on this page · raw message
fs/xfs/xfs_log_priv.h +22 -4
@@ -519,10 +519,8 @@ typedef struct log {spinlock_t l_grant_lock____cacheline_aligned_in_smp ;struct list_head l_reserveq;struct list_head l_writeq; -int l_grant_reserve_cycle; -int l_grant_reserve_bytes; -int l_grant_write_cycle; -int l_grant_write_bytes; +int64_t l_grant_reserve_head; +int64_t l_grant_write_head;/* The following field are used for debugging; need to hold icloglock */ #ifdef DEBUG
@@ -559,6 +557,26 @@ int xlog_write(struct log *log, struct xfs_log_vec *log_vector, xlog_in_core_t **commit_iclog, uint flags);/* +* When we crack the grrant head, we sample it first so that the value will not +* change while we are cracking it into the component values. This means we +* will always get consistent component values to work from. +*/ +static inline void +xlog_crack_grant_head (int64_t *head,int *cycle,int *space) +{ +int64_t val = *head; + + *cycle = val >>32 ; + *space = val &0xffffffff ; +} + +static inline void +xlog_assign_grant_head (int64_t *head,int cycle,int space) +{ + *head = ((int64_t )cycle <<32 ) | space; +} + +/* * Committed Item List interfaces */ int xlog_cil_init (struct log *log);
fs/xfs/xfs_log_recover.c +4 -4
@@ -927,10 +927,10 @@ xlog_find_tail( log->l_curr_cycle++; log->l_tail_lsn =be64_to_cpu (rhead->h_tail_lsn); log->l_last_sync_lsn =be64_to_cpu (rhead->h_lsn); - log->l_grant_reserve_cycle = log->l_curr_cycle; - log->l_grant_reserve_bytes =BBTOB (log->l_curr_block); - log->l_grant_write_cycle = log->l_curr_cycle; - log->l_grant_write_bytes =BBTOB (log->l_curr_block); +xlog_assign_grant_head (&log->l_grant_reserve_head, log->l_curr_cycle, +BBTOB (log->l_curr_block)); +xlog_assign_grant_head (&log->l_grant_write_head, log->l_curr_cycle, +BBTOB (log->l_curr_block));/* * Look for unmount record. If we find it, then we know there
Signature · 6 lines
1.7.2.3 _______________________________________________ xfs mailing list [email protected] http://oss.sgi.com/mailman/listinfo/xfs
Re: [PATCH 04/12] xfs: combine grant heads into a single 64 bit integer
Quoted 9 lines · Dave Chinner
From: Dave Chinner <[email protected]>
Prepare for switching the grant heads to atomic variables by combining the two 32 bit values that make up the grant head into a single 64 bit variable. Provide wrapper functions to combine and split the grant heads appropriately for calculations and use them as necessary.
Signed-off-by Dave Chinner <[email protected]>
Looks good,
Minor nitpick:
fs/xfs/xfs_log.c xlog_assign_tail_lsn(xfs_mount_t *mp) · line 840
Quoted 3 lines
fs/xfs/xfs_log.c +3
-0+if ((tail_cycle == head_cycle) && (head_bytes >= tail_bytes)) { + free_bytes = log->l_logsize - (head_bytes - tail_bytes); + }else if ((tail_cycle +1 ) < head_cycle) {
Lots of superflous braces in the if statements.
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
PATCH 5/12 xfs: use wait queues directly for the log wait queues
In reply to Dave ChinnerFrom: Dave Chinner <[email protected]>
The log grant queues are one of the few places left using sv_t constructs for waiting. Given we are touching this code, we should convert them to plain wait queues. While there, convert all the other sv_t users in the log code as well.
Seeing as this removes the last users of the sv_t type, remove the header file defining the wrapper and the fragments that still reference it.
--- fs/xfs/linux-2.6/sv.h | 59 -------------------------------------- fs/xfs/linux-2.6/xfs_linux.h | 1 - fs/xfs/quota/xfs_dquot.c | 1 - fs/xfs/xfs_log.c | 64 ++++++++++++++++++----------------------- fs/xfs/xfs_log_cil.c | 8 ++-- fs/xfs/xfs_log_priv.h | 25 +++++++++++++--- 6 files changed, 52 insertions(+), 106 deletions(-) delete mode 100644 fs/xfs/linux-2.6/sv.h
fs/xfs/linux-2.6/sv.h deleted +0 -59
@@ -1,59 +0,0 @@ -/* -* Copyright (c) 2000-2002,2005 Silicon Graphics, Inc. -* All Rights Reserved. -* -* This program is free software; you can redistribute it and/or -* modify it under the terms of the GNU General Public License as -* published by the Free Software Foundation. -* -* This program is distributed in the hope that it would be useful, -* but WITHOUT ANY WARRANTY; without even the implied warranty of -* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -* GNU General Public License for more details. -* -* You should have received a copy of the GNU General Public License -* along with this program; if not, write the Free Software Foundation, -* Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA -*/ -#ifndef __XFS_SUPPORT_SV_H__ -#define __XFS_SUPPORT_SV_H__ - -#include <linux/wait.h> -#include <linux/sched.h> -#include <linux/spinlock.h> - -/* -* Synchronisation variables. -* -* (Parameters "pri", "svf" and "rts" are not implemented) -*/ - -typedef struct sv_s { -wait_queue_head_t waiters; -}sv_t ; - -static inline void _sv_wait (sv_t *sv,spinlock_t *lock) -{ -DECLARE_WAITQUEUE (wait, current); - -add_wait_queue_exclusive (&sv->waiters, &wait); -__set_current_state (TASK_UNINTERRUPTIBLE ); -spin_unlock (lock); - -schedule (); - -remove_wait_queue (&sv->waiters, &wait); -} - -#define sv_init (sv,flag,name) \ - init_waitqueue_head(&(sv)->waiters) -#define sv_destroy (sv) \ -/*NOTHING*/ -#define sv_wait (sv, pri, lock, s) \ - _sv_wait(sv, lock) -#define sv_signal (sv) \ - wake_up(&(sv)->waiters) -#define sv_broadcast (sv) \ - wake_up_all(&(sv)->waiters) - -#endif /* __XFS_SUPPORT_SV_H__ */
fs/xfs/linux-2.6/xfs_linux.h +0 -1
@@ -37,7 +37,6 @@#include <kmem.h> #include <mrlock.h> -#include <sv.h> #include <time.h> #include <support/debug.h>
fs/xfs/quota/xfs_dquot.c +0 -1
@@ -149,7 +149,6 @@ xfs_qm_dqdestroy(ASSERT (list_empty (&dqp->q_freelist));mutex_destroy (&dqp->q_qlock); -sv_destroy (&dqp->q_pinwait);kmem_zone_free (xfs_Gqm->qm_dqzone, dqp);atomic_dec (&xfs_Gqm->qm_totaldquots);
fs/xfs/xfs_log.c +28 -36
206 lines not on this page · raw message
fs/xfs/xfs_log_cil.c +4 -4
@@ -61,7 +61,7 @@ xlog_cil_init(INIT_LIST_HEAD (&cil->xc_committing);spin_lock_init (&cil->xc_cil_lock);init_rwsem (&cil->xc_ctx_lock); -sv_init (&cil->xc_commit_wait,SV_DEFAULT ,"cilwait" ); +init_waitqueue_head (&cil->xc_commit_wait);INIT_LIST_HEAD (&ctx->committing);INIT_LIST_HEAD (&ctx->busy_extents);
@@ -563,7 +563,7 @@ restart:* It is still being pushed! Wait for the push to * complete, then start again from the beginning. */ -sv_wait (&cil->xc_commit_wait,0 , &cil->xc_cil_lock,0 ); +xlog_wait (&cil->xc_commit_wait, &cil->xc_cil_lock); goto restart; } }
@@ -587,7 +587,7 @@ restart:*/ spin_lock (&cil->xc_cil_lock); ctx->commit_lsn = commit_lsn; -sv_broadcast (&cil->xc_commit_wait); +wake_up_all (&cil->xc_commit_wait);spin_unlock (&cil->xc_cil_lock);/* release the hounds! */
@@ -752,7 +752,7 @@ restart:* It is still being pushed! Wait for the push to * complete, then start again from the beginning. */ -sv_wait (&cil->xc_commit_wait,0 , &cil->xc_cil_lock,0 ); +xlog_wait (&cil->xc_commit_wait, &cil->xc_cil_lock); goto restart; }if (ctx->sequence != sequence)
fs/xfs/xfs_log_priv.h +20 -5
@@ -242,7 +242,7 @@ typedef struct xlog_res { } xlog_res_t; typedef struct xlog_ticket { - sv_t t_wait; /* ticket wait queue : 20 */ + wait_queue_head_t t_wait; /* ticket wait queue */ struct list_head t_queue; /* reserve/write queue */ xlog_tid_t t_tid; /* transaction identifier : 4 */ atomic_t t_ref; /* ticket reference count : 4 */
@@ -350,8 +350,8 @@ typedef union xlog_in_core2 {* and move everything else out to subsequent cachelines. */ typedef struct xlog_in_core { - sv_t ic_force_wait; - sv_t ic_write_wait; + wait_queue_head_t ic_force_wait; + wait_queue_head_t ic_write_wait; struct xlog_in_core *ic_next; struct xlog_in_core *ic_prev; struct xfs_buf *ic_bp;
@@ -418,7 +418,7 @@ struct xfs_cil {struct xfs_cil_ctx *xc_ctx;struct rw_semaphore xc_ctx_lock;struct list_head xc_committing; -sv_t xc_commit_wait; +wait_queue_head_t xc_commit_wait;xfs_lsn_t xc_current_sequence; };
@@ -500,7 +500,7 @@ typedef struct log {int l_logBBsize;/* size of log in BB chunks */ /* The following block of fields are changed while holding icloglock */ -sv_t l_flush_wait____cacheline_aligned_in_smp ; +wait_queue_head_t l_flush_wait____cacheline_aligned_in_smp ;/* waiting for iclog flush */ int l_covered_state;/* state of "covering disk * log entries" */
@@ -600,6 +600,21 @@ xlog_cil_force(struct log *log)*/ #define XLOG_UNMOUNT_REC_TYPE (-1U) +/* +* Wrapper function for waiting on a wait queue serialised against wakeups +* by a spinlock. This matches the semantics of all the wait queues used in the +* log code. +*/ +static inline void xlog_wait (wait_queue_head_t *wq,spinlock_t *lock) +{ +DECLARE_WAITQUEUE (wait, current); + +add_wait_queue_exclusive (wq, &wait); +__set_current_state (TASK_UNINTERRUPTIBLE ); +spin_unlock (lock); +schedule (); +remove_wait_queue (wq, &wait); +}#endif /* __KERNEL__ */ #endif /* __XFS_LOG_PRIV_H__ */
Signature · 6 lines
1.7.2.3 _______________________________________________ xfs mailing list [email protected] http://oss.sgi.com/mailman/listinfo/xfs
Re: [PATCH 05/12] xfs: use wait queues directly for the log wait queues
Quoted 10 lines · Dave Chinner
From: Dave Chinner <[email protected]>
The log grant queues are one of the few places left using
sv_tconstructs for waiting. Given we are touching this code, we should convert them to plain wait queues. While there, convert all the othersv_tusers in the log code as well.Seeing as this removes the last users of the
sv_ttype, remove the header file defining the wrapper and the fragments that still reference it.
Looks good,
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
PATCH 6/12 xfs: make AIL tail pushing independent of the grant lock
In reply to Dave ChinnerFrom: Dave Chinner <[email protected]>
The xlog_grant_push_ail() currently takes the grant lock internally to sample the tail lsn, last sync lsn and the reserve grant head. Most of the callers already hold the grant lock but have to drop it before calling xlog_grant_push_ail(). This is a left over from when the AIL tail pushing was done in line and hence xlog_grant_push_ail had to drop the grant lock. AIL push is now done in another thread and hence we can safely hold the grant lock over the entire xlog_grant_push_ail call.
Push the grant lock outside of xlog_grant_push_ail() to simplify the locking and synchronisation needed for tail pushing. This will reduce traffic on the grant lock by itself, but this is only one step in preparing for the complete removal of the grant lock.
While there, clean up the formatting of xlog_grant_push_ail() to match the rest of the XFS code.
--- fs/xfs/xfs_log.c | 111 ++++++++++++++++++++++++++--------------------------- 1 files changed, 54 insertions(+), 57 deletions(-)
fs/xfs/xfs_log.c +54 -57
@@ -70,7 +70,7 @@ STATIC void xlog_state_want_sync(xlog_t *log, xlog_in_core_t *iclog);/* local functions to manipulate grant head */ STATIC int xlog_grant_log_space(xlog_t *log,xlog_ticket_t *xtic); -STATIC void xlog_grant_push_ail(xfs_mount_t *mp, +STATIC void xlog_grant_push_ail(struct log *log,int need_bytes); STATIC void xlog_regrant_reserve_log_space(xlog_t *log,xlog_ticket_t *ticket);
@@ -318,7 +318,9 @@ xfs_log_reserve(trace_xfs_log_reserve (log, internal_ticket); -xlog_grant_push_ail (mp, internal_ticket->t_unit_res); +spin_lock (&log->l_grant_lock); +xlog_grant_push_ail (log, internal_ticket->t_unit_res); +spin_unlock (&log->l_grant_lock); retval =xlog_regrant_write_log_space (log, internal_ticket); }else {/* may sleep if need to allocate more tickets */
@@ -332,9 +334,11 @@ xfs_log_reserve(trace_xfs_log_reserve (log, internal_ticket); -xlog_grant_push_ail (mp, +spin_lock (&log->l_grant_lock); +xlog_grant_push_ail (log, (internal_ticket->t_unit_res * internal_ticket->t_cnt)); +spin_unlock (&log->l_grant_lock); retval =xlog_grant_log_space (log, internal_ticket); }
@@ -1185,59 +1189,58 @@ xlog_commit_record(* water mark. In this manner, we would be creating a low water mark. */ STATIC void -xlog_grant_push_ail (xfs_mount_t *mp, -int need_bytes) +xlog_grant_push_ail ( +struct log *log, +int need_bytes) { -xlog_t *log = mp->m_log;/* pointer to the log */ -xfs_lsn_t tail_lsn;/* lsn of the log tail */ -xfs_lsn_t threshold_lsn =0 ;/* lsn we'd like to be at */ -int free_blocks;/* free blocks left to write to */ -int free_bytes;/* free bytes left to write to */ -int threshold_block;/* block in lsn we'd like to be at */ -int threshold_cycle;/* lsn cycle we'd like to be at */ -int free_threshold; - -ASSERT (BTOBB (need_bytes) < log->l_logBBsize); - -spin_lock (&log->l_grant_lock); - free_bytes =xlog_space_left (log, &log->l_grant_reserve_head); - tail_lsn = log->l_tail_lsn; - free_blocks =BTOBBT (free_bytes); - -/* -* Set the threshold for the minimum number of free blocks in the -* log to the maximum of what the caller needs, one quarter of the -* log, and 256 blocks. -*/ - free_threshold =BTOBB (need_bytes); - free_threshold =MAX (free_threshold, (log->l_logBBsize >>2 )); - free_threshold =MAX (free_threshold,256 ); -if (free_blocks < free_threshold) { +xfs_lsn_t threshold_lsn =0 ; +xfs_lsn_t tail_lsn; +int free_blocks; +int free_bytes; +int threshold_block; +int threshold_cycle; +int free_threshold; + +ASSERT (BTOBB (need_bytes) < log->l_logBBsize); + + tail_lsn = log->l_tail_lsn; + free_bytes =xlog_space_left (log, &log->l_grant_reserve_head); + free_blocks =BTOBBT (free_bytes); + +/* +* Set the threshold for the minimum number of free blocks in the +* log to the maximum of what the caller needs, one quarter of the +* log, and 256 blocks. +*/ + free_threshold =BTOBB (need_bytes); + free_threshold =MAX (free_threshold, (log->l_logBBsize >>2 )); + free_threshold =MAX (free_threshold,256 ); +if (free_blocks >= free_threshold) +return ; + threshold_block =BLOCK_LSN (tail_lsn) + free_threshold; threshold_cycle =CYCLE_LSN (tail_lsn);if (threshold_block >= log->l_logBBsize) { - threshold_block -= log->l_logBBsize; - threshold_cycle +=1 ; + threshold_block -= log->l_logBBsize; + threshold_cycle +=1 ; } - threshold_lsn =xlog_assign_lsn (threshold_cycle, threshold_block); - -/* Don't pass in an lsn greater than the lsn of the last + threshold_lsn =xlog_assign_lsn (threshold_cycle, + threshold_block); +/* +* Don't pass in an lsn greater than the lsn of the last * log record known to be on disk. */ if (XFS_LSN_CMP (threshold_lsn, log->l_last_sync_lsn) >0 ) - threshold_lsn = log->l_last_sync_lsn; - } -spin_unlock (&log->l_grant_lock); - -/* -* Get the transaction layer to kick the dirty buffers out to -* disk asynchronously. No point in trying to do this if -* the filesystem is shutting down. -*/ -if (threshold_lsn && - !XLOG_FORCED_SHUTDOWN (log)) -xfs_trans_ail_push (log->l_ailp, threshold_lsn); -}/* xlog_grant_push_ail */ + threshold_lsn = log->l_last_sync_lsn; + +/* +* Get the transaction layer to kick the dirty buffers out to +* disk asynchronously. No point in trying to do this if +* the filesystem is shutting down. +*/ +if (!XLOG_FORCED_SHUTDOWN (log)) +xfs_trans_ail_push (log->l_ailp, threshold_lsn); +}/* * The bdstrat callback function for log bufs. This gives us a central
@@ -2543,9 +2546,7 @@ redo:trace_xfs_log_grant_sleep2 (log, tic); -spin_unlock (&log->l_grant_lock); -xlog_grant_push_ail (log->l_mp, need_bytes); -spin_lock (&log->l_grant_lock); +xlog_grant_push_ail (log, need_bytes);XFS_STATS_INC (xs_sleep_logspace);xlog_wait (&tic->t_wait, &log->l_grant_lock);
@@ -2641,9 +2642,7 @@ xlog_regrant_write_log_space(xlog_t *log,trace_xfs_log_regrant_write_sleep1 (log, tic); -spin_unlock (&log->l_grant_lock); -xlog_grant_push_ail (log->l_mp, need_bytes); -spin_lock (&log->l_grant_lock); +xlog_grant_push_ail (log, need_bytes);XFS_STATS_INC (xs_sleep_logspace);xlog_wait (&tic->t_wait, &log->l_grant_lock);
@@ -2666,9 +2665,7 @@ redo:if (free_bytes < need_bytes) {if (list_empty (&tic->t_queue))list_add_tail (&tic->t_queue, &log->l_writeq); -spin_unlock (&log->l_grant_lock); -xlog_grant_push_ail (log->l_mp, need_bytes); -spin_lock (&log->l_grant_lock); +xlog_grant_push_ail (log, need_bytes);XFS_STATS_INC (xs_sleep_logspace);trace_xfs_log_regrant_write_sleep2 (log, tic);
Signature · 6 lines
1.7.2.3 _______________________________________________ xfs mailing list [email protected] http://oss.sgi.com/mailman/listinfo/xfs
Re: [PATCH 06/12] xfs: make AIL tail pushing independent of the grant lock
Looks good,
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
PATCH 7/12 xfs: convert l_last_sync_lsn to an atomic variable
In reply to Dave ChinnerFrom: Dave Chinner <[email protected]>
log->l_last_sync_lsn is updated in only one critical spot - log buffer Io completion - and is protected by the grant lock here. This requires the grant lock to be taken for every log buffer IO completion. Converting the l_last_sync_lsn variable to an atomic64_t means that we do not need to take the grant lock in log buffer IO completion to update it.
This also removes the need for explicitly holding a spinlock to read the l_last_sync_lsn on 32 bit platforms.
--- fs/xfs/xfs_log.c | 55 +++++++++++++++++++++------------------------- fs/xfs/xfs_log_priv.h | 9 ++++++- fs/xfs/xfs_log_recover.c | 6 ++-- 3 files changed, 36 insertions(+), 34 deletions(-)
fs/xfs/xfs_log.c +25 -30
@@ -675,12 +675,8 @@ xfs_log_move_tail(xfs_mount_t *mp,if (XLOG_FORCED_SHUTDOWN (log))return ; -if (tail_lsn ==0 ) { -/* needed since sync_lsn is 64 bits */ -spin_lock (&log->l_icloglock); - tail_lsn = log->l_last_sync_lsn; -spin_unlock (&log->l_icloglock); - } +if (tail_lsn ==0 ) + tail_lsn =atomic64_read (&log->l_last_sync_lsn);spin_lock (&log->l_grant_lock);
@@ -800,11 +796,9 @@ xlog_assign_tail_lsn(xfs_mount_t *mp) tail_lsn =xfs_trans_ail_tail (mp->m_ail);spin_lock (&log->l_grant_lock); -if (tail_lsn !=0 ) { - log->l_tail_lsn = tail_lsn; - }else { - tail_lsn = log->l_tail_lsn = log->l_last_sync_lsn; - } +if (!tail_lsn) + tail_lsn =atomic64_read (&log->l_last_sync_lsn); + log->l_tail_lsn = tail_lsn;spin_unlock (&log->l_grant_lock);return tail_lsn;
@@ -1014,9 +1008,9 @@ xlog_alloc_log(xfs_mount_t *mp, log->l_flags |=XLOG_ACTIVE_RECOVERY ; log->l_prev_block =-1 ; - log->l_tail_lsn =xlog_assign_lsn (1 ,0 );/* log->l_tail_lsn = 0x100000000LL; cycle = 1; current block = 0 */ - log->l_last_sync_lsn = log->l_tail_lsn; + log->l_tail_lsn =xlog_assign_lsn (1 ,0 ); +atomic64_set (&log->l_last_sync_lsn,xlog_assign_lsn (1 ,0 )); log->l_curr_cycle =1 ;/* 0 is bad since this is initial value */ xlog_assign_grant_head (&log->l_grant_reserve_head,1 ,0 );xlog_assign_grant_head (&log->l_grant_write_head,1 ,0 );
@@ -1194,6 +1188,7 @@ xlog_grant_push_ail( int need_bytes) {xfs_lsn_t threshold_lsn =0 ; +xfs_lsn_t last_sync_lsn;xfs_lsn_t tail_lsn;int free_blocks;int free_bytes;
@@ -1228,10 +1223,12 @@ xlog_grant_push_ail( threshold_block);/* * Don't pass in an lsn greater than the lsn of the last -* log record known to be on disk. +* log record known to be on disk. Use a snapshot of the last sync lsn +* so that it doesn't change between the compare and the set. */ -if (XFS_LSN_CMP (threshold_lsn, log->l_last_sync_lsn) >0 ) - threshold_lsn = log->l_last_sync_lsn; + last_sync_lsn =atomic64_read (&log->l_last_sync_lsn); +if (XFS_LSN_CMP (threshold_lsn, last_sync_lsn) >0 ) + threshold_lsn = last_sync_lsn;/* * Get the transaction layer to kick the dirty buffers out to
@@ -2194,7 +2191,7 @@ xlog_state_do_callback( lowest_lsn =xlog_get_lowest_lsn (log);if (lowest_lsn &&XFS_LSN_CMP (lowest_lsn, -be64_to_cpu (iclog->ic_header.h_lsn)) <0 ) { +be64_to_cpu (iclog->ic_header.h_lsn)) <0 ) { iclog = iclog->ic_next;continue ;/* Leave this iclog for * another thread */
@@ -2202,23 +2199,21 @@ xlog_state_do_callback( iclog->ic_state =XLOG_STATE_CALLBACK ; -spin_unlock (&log->l_icloglock); -/* l_last_sync_lsn field protected by -* l_grant_lock. Don't worry about iclog's lsn. -* No one else can be here except us. +/* +* update the last_sync_lsn before we drop the +* icloglock to ensure we are the only one that +* can update it. */ -spin_lock (&log->l_grant_lock); -ASSERT (XFS_LSN_CMP (log->l_last_sync_lsn, -be64_to_cpu (iclog->ic_header.h_lsn)) <=0 ); - log->l_last_sync_lsn = -be64_to_cpu (iclog->ic_header.h_lsn); -spin_unlock (&log->l_grant_lock); +ASSERT (XFS_LSN_CMP (atomic64_read (&log->l_last_sync_lsn), +be64_to_cpu (iclog->ic_header.h_lsn)) <=0 ); +atomic64_set (&log->l_last_sync_lsn, +be64_to_cpu (iclog->ic_header.h_lsn)); - }else { -spin_unlock (&log->l_icloglock); + }else ioerrors++; - } + +spin_unlock (&log->l_icloglock);/* * Keep processing entries in the callback list until
fs/xfs/xfs_log_priv.h +8 -1
@@ -508,7 +508,6 @@ typedef struct log {spinlock_t l_icloglock;/* grab to change iclog state */ xfs_lsn_t l_tail_lsn;/* lsn of 1st LR with unflushed * buffers */ -xfs_lsn_t l_last_sync_lsn;/* lsn of last LR on disk */ int l_curr_cycle;/* Cycle number of log writes */ int l_prev_cycle;/* Cycle number before last * block increment */
@@ -522,6 +521,14 @@ typedef struct log {int64_t l_grant_reserve_head;int64_t l_grant_write_head; +/* +* l_last_sync_lsn is an atomic so it can be set and read without +* needing to hold specific locks. To avoid operations contending with +* other hot objects, place it on a separate cacheline. +*/ +/* lsn of last LR on disk */ +atomic64_t l_last_sync_lsn____cacheline_aligned_in_smp ; +/* The following field are used for debugging; need to hold icloglock */ #ifdef DEBUG char *l_iclog_bak[XLOG_MAX_ICLOGS ];
fs/xfs/xfs_log_recover.c +3 -3
@@ -926,7 +926,7 @@ xlog_find_tail(if (found ==2 ) log->l_curr_cycle++; log->l_tail_lsn =be64_to_cpu (rhead->h_tail_lsn); - log->l_last_sync_lsn =be64_to_cpu (rhead->h_lsn); +atomic64_set (&log->l_last_sync_lsn,be64_to_cpu (rhead->h_lsn));xlog_assign_grant_head (&log->l_grant_reserve_head, log->l_curr_cycle,BBTOB (log->l_curr_block)); xlog_assign_grant_head(&log->l_grant_write_head, log->l_curr_cycle,
@@ -978,9 +978,9 @@ xlog_find_tail( log->l_tail_lsn =xlog_assign_lsn (log->l_curr_cycle, after_umount_blk); - log->l_last_sync_lsn = +atomic64_set (&log->l_last_sync_lsn,xlog_assign_lsn (log->l_curr_cycle, - after_umount_blk); + after_umount_blk)); *tail_blk = after_umount_blk;/*
Signature · 6 lines
1.7.2.3 _______________________________________________ xfs mailing list [email protected] http://oss.sgi.com/mailman/listinfo/xfs
PATCH 8/12 xfs: convert l_tail_lsn to an atomic variable.
In reply to Dave ChinnerFrom: Dave Chinner <[email protected]>
log->l_tail_lsn is currently protected by the log grant lock. The lock is only needed for serialising readers against writers, so we don't really need the lock if we make the l_tail_lsn variable an atomic. Converting the l_tail_lsn variable to an atomic64_t means we can start to peel back the grant lock from various operations.
Also, provide functions to safely crack an atomic LSN variable into it's component pieces and to recombined the components into an atomic variable. Use them where appropriate.
This also removes the need for explicitly holding a spinlock to read the l_tail_lsn on 32 bit platforms.
--- fs/xfs/linux-2.6/xfs_trace.h | 2 +- fs/xfs/xfs_log.c | 56 ++++++++++++++++++----------------------- fs/xfs/xfs_log_priv.h | 37 +++++++++++++++++++++++---- fs/xfs/xfs_log_recover.c | 14 ++++------ 4 files changed, 63 insertions(+), 46 deletions(-)
fs/xfs/linux-2.6/xfs_trace.h +1 -1
@@ -794,7 +794,7 @@ DECLARE_EVENT_CLASS(xfs_loggrant_class, &__entry->grant_write_bytes); __entry->curr_cycle = log->l_curr_cycle; __entry->curr_block = log->l_curr_block; - __entry->tail_lsn = log->l_tail_lsn; + __entry->tail_lsn =atomic64_read (&log->l_tail_lsn); ),TP_printk ("dev %d:%d type %s t_ocnt %u t_cnt %u t_curr_res %u " "t_unit_res %u t_flags %s reserve_headq 0x%p "
fs/xfs/xfs_log.c +25 -31
126 lines not on this page · raw message
fs/xfs/xfs_log_priv.h +31 -6
@@ -54,7 +54,6 @@ struct xfs_mount;BTOBB (XLOG_MAX_ICLOGS << (xfs_sb_version_haslogv2 (&log->l_mp->m_sb) ? \ XLOG_MAX_RECORD_BSHIFT : XLOG_BIG_RECORD_BSHIFT)) -static inline xfs_lsn_t xlog_assign_lsn (uint cycle,uint block) {return ((xfs_lsn_t )cycle <<32 ) | block;
@@ -506,8 +505,6 @@ typedef struct log { * log entries" */ xlog_in_core_t *l_iclog;/* head log queue */ spinlock_t l_icloglock;/* grab to change iclog state */ -xfs_lsn_t l_tail_lsn;/* lsn of 1st LR with unflushed -* buffers */ int l_curr_cycle;/* Cycle number of log writes */ int l_prev_cycle;/* Cycle number before last * block increment */
@@ -522,12 +519,15 @@ typedef struct log {int64_t l_grant_write_head;/* -* l_last_sync_lsn is an atomic so it can be set and read without -* needing to hold specific locks. To avoid operations contending with -* other hot objects, place it on a separate cacheline. +* l_last_sync_lsn and l_tail_lsn are atomics so they can be set and +* read without needing to hold specific locks. To avoid operations +* contending with other hot objects, place each of them on a separate +* cacheline. */ /* lsn of last LR on disk */ atomic64_t l_last_sync_lsn____cacheline_aligned_in_smp ; +/* lsn of 1st LR with unflushed * buffers */ +atomic64_t l_tail_lsn____cacheline_aligned_in_smp ;/* The following field are used for debugging; need to hold icloglock */ #ifdef DEBUG
@@ -564,6 +564,31 @@ int xlog_write(struct log *log, struct xfs_log_vec *log_vector, xlog_in_core_t **commit_iclog, uint flags);/* +* When we crack an atomic LSN, we sample it first so that the value will not +* change while we are cracking it into the component values. This means we +* will always get consistent component values to work from. This should always +* be used to smaple and crack LSNs taht are stored and updated in atomic +* variables. +*/ +static inline void +xlog_crack_atomic_lsn (atomic64_t *lsn,uint *cycle,uint *block) +{ +xfs_lsn_t val =atomic64_read (lsn); + + *cycle =CYCLE_LSN (val); + *block =BLOCK_LSN (val); +} + +/* +* Calculate and assign a value to an atomic LSN variable from component pieces. +*/ +static inline void +xlog_assign_atomic_lsn (atomic64_t *lsn,uint cycle,uint block) +{ +atomic64_set (lsn,xlog_assign_lsn (cycle, block)); +} + +/* * When we crack the grrant head, we sample it first so that the value will not * change while we are cracking it into the component values. This means we * will always get consistent component values to work from.
fs/xfs/xfs_log_recover.c +6 -8
@@ -925,7 +925,7 @@ xlog_find_tail( log->l_curr_cycle =be32_to_cpu (rhead->h_cycle);if (found ==2 ) log->l_curr_cycle++; - log->l_tail_lsn =be64_to_cpu (rhead->h_tail_lsn); +atomic64_set (&log->l_tail_lsn,be64_to_cpu (rhead->h_tail_lsn));atomic64_set (&log->l_last_sync_lsn,be64_to_cpu (rhead->h_lsn));xlog_assign_grant_head (&log->l_grant_reserve_head, log->l_curr_cycle,BBTOB (log->l_curr_block));
@@ -960,7 +960,7 @@ xlog_find_tail( } after_umount_blk = (i + hblks + (int )BTOBB (be32_to_cpu (rhead->h_len))) % log->l_logBBsize; - tail_lsn = log->l_tail_lsn; + tail_lsn =atomic64_read (&log->l_tail_lsn);if (*head_blk == after_umount_blk &&be32_to_cpu (rhead->h_num_logops) ==1 ) { umount_data_blk = (i + hblks) % log->l_logBBsize;
@@ -975,12 +975,10 @@ xlog_find_tail(* log records will point recovery to after the * current unmount record. */ - log->l_tail_lsn = -xlog_assign_lsn (log->l_curr_cycle, - after_umount_blk); -atomic64_set (&log->l_last_sync_lsn, -xlog_assign_lsn (log->l_curr_cycle, - after_umount_blk)); +xlog_assign_atomic_lsn (&log->l_tail_lsn, + log->l_curr_cycle, after_umount_blk); +xlog_assign_atomic_lsn (&log->l_last_sync_lsn, + log->l_curr_cycle, after_umount_blk); *tail_blk = after_umount_blk;/*
Signature · 6 lines
1.7.2.3 _______________________________________________ xfs mailing list [email protected] http://oss.sgi.com/mailman/listinfo/xfs
Re: [PATCH 08/12] xfs: convert l_tail_lsn to an atomic variable.
Quoted 8 lines · Dave Chinner
From: Dave Chinner <[email protected]>
log->l_tail_lsnis currently protected by the log grant lock. The lock is only needed for serialising readers against writers, so we don't really need the lock if we make thel_tail_lsnvariable an atomic. Converting thel_tail_lsnvariable to anatomic64_tmeans we can start to peel back the grant lock from various operations.
Quoted 6 lines
Also, provide functions to safely crack an atomic LSN variable into it's component pieces and to recombined the components into an atomic variable. Use them where appropriate.
This also removes the need for explicitly holding a spinlock to read the
l_tail_lsnon 32 bit platforms.
I know I suggested this, but I think the atomic read of l_tail_lsn in xlog_space_left might be problemetic for the call from xlog_grant_push_ail, where we read it twice now. Maybe split xlog_space_left into a __xlog_space_left that gets the already cracked values, and xlog_space_left as a wrapper around it?
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
Re: [PATCH 08/12] xfs: convert l_tail_lsn to an atomic variable.
Quoted 13 lines · Christoph Hellwig
Quoted 12 lines · Dave Chinner
From: Dave Chinner <[email protected]>
log->l_tail_lsnis currently protected by the log grant lock. The lock is only needed for serialising readers against writers, so we don't really need the lock if we make thel_tail_lsnvariable an atomic. Converting thel_tail_lsnvariable to anatomic64_tmeans we can start to peel back the grant lock from various operations.Also, provide functions to safely crack an atomic LSN variable into it's component pieces and to recombined the components into an atomic variable. Use them where appropriate.
Quoted 8 lines
Quoted 2 lines
This also removes the need for explicitly holding a spinlock to read the
l_tail_lsnon 32 bit platforms.I know I suggested this, but I think the atomic read of
l_tail_lsninxlog_space_leftmight be problemetic for the call fromxlog_grant_push_ail, where we read it twice now. Maybe splitxlog_space_leftinto a__xlog_space_leftthat gets the already cracked values, andxlog_space_leftas a wrapper around it?
I'd convinced myself that it wouldn't be a problem. That is, once we have a value for the tail_lsn in xlog_grant_push_ail(), the threshold that we will push to is effectively fixed. The only thing that will change is the amount of log space currently available, which can only increase if the tail moves.
Hence we'll either get:
a) not enough log space and have to push, in which case the value of the tail lsn seen in xlog_space_left() is irrelevant to the threshold lsn we calculate, or
b) we'll have enough log space and not need to push in which case we don't need to use the tail_lsn at all because we don't need to push.
So it seems to me that the double sample of the tail_lsn doesn't matter at all for this code. Is there a hole in my logic here?
Cheers,
Dave.
Signature · 7 lines
Dave Chinner [email protected] _______________________________________________ xfs mailing list [email protected] http://oss.sgi.com/mailman/listinfo/xfs
PATCH 9/12 xfs: convert log grant heads to atomic variables
In reply to Dave ChinnerFrom: Dave Chinner <[email protected]>
Convert the log grant heads to atomic64_t types in preparation for converting the accounting algorithms to atomic operations. his patch just converts the variables; the algorithmic changes are in a separate patch for clarity.
--- fs/xfs/xfs_log.c | 8 ++++---- fs/xfs/xfs_log_priv.h | 12 ++++++------ 2 files changed, 10 insertions(+), 10 deletions(-)
fs/xfs/xfs_log.c +4 -4
@@ -47,7 +47,7 @@ STATIC xlog_t * xlog_alloc_log(xfs_mount_t *mp,xfs_buftarg_t *log_target, xfs_daddr_t blk_offset, int num_bblks); -STATIC int xlog_space_left(struct log *log, int64_t *head); +STATIC int xlog_space_left(struct log *log, atomic64_t *head); STATIC int xlog_sync(xlog_t *log, xlog_in_core_t *iclog); STATIC void xlog_dealloc_log(xlog_t *log);
@@ -100,7 +100,7 @@ STATIC int xlog_iclogs_empty(xlog_t *log);static void xlog_grant_sub_space (struct log *log, -int64_t *head, +atomic64_t *head,int bytes) {int cycle, space;
@@ -119,7 +119,7 @@ xlog_grant_sub_space(static void xlog_grant_add_space (struct log *log, -int64_t *head, +atomic64_t *head,int bytes) {int tmp;
@@ -816,7 +816,7 @@ xlog_assign_tail_lsn( STATIC intxlog_space_left (struct log *log, -int64_t *head) +atomic64_t *head) {int free_bytes;int tail_bytes;
fs/xfs/xfs_log_priv.h +6 -6
@@ -515,8 +515,8 @@ typedef struct log {spinlock_t l_grant_lock____cacheline_aligned_in_smp ;struct list_head l_reserveq;struct list_head l_writeq; -int64_t l_grant_reserve_head; -int64_t l_grant_write_head; +atomic64_t l_grant_reserve_head; +atomic64_t l_grant_write_head;/* * l_last_sync_lsn and l_tail_lsn are atomics so they can be set and
@@ -594,18 +594,18 @@ xlog_assign_atomic_lsn(atomic64_t *lsn, uint cycle, uint block)* will always get consistent component values to work from. */ static inline void -xlog_crack_grant_head (int64_t *head,int *cycle,int *space) +xlog_crack_grant_head (atomic64_t *head,int *cycle,int *space) { -int64_t val = *head; +int64_t val =atomic64_read (head); *cycle = val >>32 ; *space = val &0xffffffff ; }static inline void -xlog_assign_grant_head (int64_t *head,int cycle,int space) +xlog_assign_grant_head (atomic64_t *head,int cycle,int space) { - *head = ((int64_t )cycle <<32 ) | space; +atomic64_set (head, ((int64_t )cycle <<32 ) | space); }/*
Signature · 6 lines
1.7.2.3 _______________________________________________ xfs mailing list [email protected] http://oss.sgi.com/mailman/listinfo/xfs
PATCH 10/12 xfs: introduce new locks for the log grant ticket wait queues
In reply to Dave ChinnerFrom: Dave Chinner <[email protected]>
The log grant ticket wait queues are currently protected by the log grant lock. However, the queues are functionally independent from each other, and operations on them only require serialisation against other queue operations now that all of the other log variables they use are atomic values.
Hence, we can make them independent of the grant lock by introducing new locks just to protect the lists operations. because the lists are independent, we can use a lock per list and ensure that reserve and write head queuing do not contend.
To ensure forced shutdowns work correctly in conjunction with the new fast paths, ensure that we check whether the log has been shut down in the grant functions once we hold the relevant spin locks but before we go to sleep. This is needed to co-ordinate correctly with the wakeups that are issued on the ticket queues so we don't leave any processes sleeping on the queues during a shutdown.
--- fs/xfs/linux-2.6/xfs_trace.h | 2 + fs/xfs/xfs_log.c | 139 +++++++++++++++++++++++++----------------- fs/xfs/xfs_log_priv.h | 16 ++++- 3 files changed, 97 insertions(+), 60 deletions(-)
fs/xfs/linux-2.6/xfs_trace.h +2 -0
@@ -837,6 +837,7 @@ DEFINE_LOGGRANT_EVENT(xfs_log_grant_sleep1);DEFINE_LOGGRANT_EVENT (xfs_log_grant_wake1);DEFINE_LOGGRANT_EVENT (xfs_log_grant_sleep2);DEFINE_LOGGRANT_EVENT (xfs_log_grant_wake2); +DEFINE_LOGGRANT_EVENT (xfs_log_grant_wake_up);DEFINE_LOGGRANT_EVENT (xfs_log_regrant_write_enter);DEFINE_LOGGRANT_EVENT (xfs_log_regrant_write_exit);DEFINE_LOGGRANT_EVENT (xfs_log_regrant_write_error);
@@ -844,6 +845,7 @@ DEFINE_LOGGRANT_EVENT(xfs_log_regrant_write_sleep1);DEFINE_LOGGRANT_EVENT (xfs_log_regrant_write_wake1);DEFINE_LOGGRANT_EVENT (xfs_log_regrant_write_sleep2);DEFINE_LOGGRANT_EVENT (xfs_log_regrant_write_wake2); +DEFINE_LOGGRANT_EVENT (xfs_log_regrant_write_wake_up);DEFINE_LOGGRANT_EVENT (xfs_log_regrant_reserve_enter);DEFINE_LOGGRANT_EVENT (xfs_log_regrant_reserve_exit);DEFINE_LOGGRANT_EVENT (xfs_log_regrant_reserve_sub);
fs/xfs/xfs_log.c +83 -56
327 lines not on this page · raw message
fs/xfs/xfs_log_priv.h +12 -4
@@ -513,10 +513,6 @@ typedef struct log {/* The following block of fields are changed while holding grant_lock */ spinlock_t l_grant_lock____cacheline_aligned_in_smp ; -struct list_head l_reserveq; -struct list_head l_writeq; -atomic64_t l_grant_reserve_head; -atomic64_t l_grant_write_head;/* * l_last_sync_lsn and l_tail_lsn are atomics so they can be set and
@@ -529,6 +525,18 @@ typedef struct log {/* lsn of 1st LR with unflushed * buffers */ atomic64_t l_tail_lsn____cacheline_aligned_in_smp ; +/* +* ticket grant locks, queues and accounting have their own cachlines +* as these are quite hot and can be operated on concurrently. +*/ +spinlock_t l_grant_reserve_lock____cacheline_aligned_in_smp ; +struct list_head l_reserveq; +atomic64_t l_grant_reserve_head; + +spinlock_t l_grant_write_lock____cacheline_aligned_in_smp ; +struct list_head l_writeq; +atomic64_t l_grant_write_head; +/* The following field are used for debugging; need to hold icloglock */ #ifdef DEBUG char *l_iclog_bak[XLOG_MAX_ICLOGS ];
Signature · 6 lines
1.7.2.3 _______________________________________________ xfs mailing list [email protected] http://oss.sgi.com/mailman/listinfo/xfs
Re: [PATCH 10/12] xfs: introduce new locks for the log grant ticket wait queues
Quoted 13 lines · Dave Chinner
From: Dave Chinner <[email protected]>
The log grant ticket wait queues are currently protected by the log grant lock. However, the queues are functionally independent from each other, and operations on them only require serialisation against other queue operations now that all of the other log variables they use are atomic values.
Hence, we can make them independent of the grant lock by introducing new locks just to protect the lists operations. because the lists are independent, we can use a lock per list and ensure that reserve and write head queuing do not contend.
Quoted 8 lines
To ensure forced shutdowns work correctly in conjunction with the new fast paths, ensure that we check whether the log has been shut down in the grant functions once we hold the relevant spin locks but before we go to sleep. This is needed to co-ordinate correctly with the wakeups that are issued on the ticket queues so we don't leave any processes sleeping on the queues during a shutdown.
Signed-off-by Dave Chinner <[email protected]>
Looks good,
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
PATCH 11/12 xfs: convert grant head manipulations to lockless algorithm
In reply to Dave ChinnerFrom: Dave Chinner <[email protected]>
The only thing that the grant lock remains to protect is the grant head manipulations when adding or removing space from the log. These calculations are already based on atomic variables, so we can already update them safely without locks. However, the grant head manpulations require atomic multi-step calculations to be executed, which the algorithms currently don't allow.
To make these multi-step calculations atomic, convert the algorithms to compare-and-exchange loops on the atomic variables. That is, we sample the old value, perform the calculation and use atomic64_cmpxchg() to attempt to update the head with the new value. If the head has not changed since we sampled it, it will succeed and we are done. Otherwise, we rerun the calculation again from a new sample of the head.
This allows us to remove the grant lock from around all the grant head space manipulations, and that effectively removes the grant lock from the log completely. Hence we can remove the grant lock completely from the log at this point.
--- fs/xfs/xfs_log.c | 103 ++++++++++++++++--------------------------------- fs/xfs/xfs_log_priv.h | 23 +++++++---- 2 files changed, 49 insertions(+), 77 deletions(-)
fs/xfs/xfs_log.c +34 -69
@@ -81,7 +81,6 @@ STATIC void xlog_ungrant_log_space(xlog_t *log,#if defined(DEBUG ) STATIC void xlog_verify_dest_ptr(xlog_t *log, char *ptr); -STATIC void xlog_verify_grant_head(xlog_t *log, int equals); STATIC void xlog_verify_grant_tail(struct log *log); STATIC void xlog_verify_iclog(xlog_t *log, xlog_in_core_t *iclog,int count,boolean_t syncing);
@@ -89,7 +88,6 @@ STATIC void xlog_verify_tail_lsn(xlog_t *log, xlog_in_core_t *iclog, xfs_lsn_t tail_lsn);#else #define xlog_verify_dest_ptr (a,b) -#define xlog_verify_grant_head (a,b)#define xlog_verify_grant_tail (a)#define xlog_verify_iclog (a,b,c,d)#define xlog_verify_tail_lsn (a,b,c)
@@ -103,17 +101,24 @@ xlog_grant_sub_space(atomic64_t *head, int bytes) { - int cycle, space; + int64_t head_val = atomic64_read(head); +int64_t new, old; -xlog_crack_grant_head (head, &cycle, &space); +do { +int cycle, space; - space -= bytes; -if (space <0 ) { - space += log->l_logsize; - cycle--; - } +xlog_crack_grant_head_val (head_val, &cycle, &space); -xlog_assign_grant_head (head, cycle, space); + space -= bytes; +if (space <0 ) { + space += log->l_logsize; + cycle--; + } + + old = head_val; + new =xlog_assign_grant_head_val (cycle, space); + head_val =atomic64_cmpxchg (head, old, new); + }while (head_val != old); } static void
@@ -122,20 +127,27 @@ xlog_grant_add_space(atomic64_t *head, int bytes) { - int tmp; -int cycle, space; + int64_t head_val = atomic64_read(head); +int64_t new, old; -xlog_crack_grant_head (head, &cycle, &space); +do { +int tmp; +int cycle, space; - tmp = log->l_logsize - space; -if (tmp > bytes) - space += bytes; -else { - space = bytes - tmp; - cycle++; - } +xlog_crack_grant_head_val (head_val, &cycle, &space); -xlog_assign_grant_head (head, cycle, space); + tmp = log->l_logsize - space; +if (tmp > bytes) + space += bytes; +else { + space = bytes - tmp; + cycle++; + } + + old = head_val; + new =xlog_assign_grant_head_val (cycle, space); + head_val =atomic64_cmpxchg (head, old, new); + }while (head_val != old); } static void
@@ -318,9 +330,7 @@ xfs_log_reserve(trace_xfs_log_reserve (log, internal_ticket); -spin_lock (&log->l_grant_lock);xlog_grant_push_ail (log, internal_ticket->t_unit_res); -spin_unlock (&log->l_grant_lock); retval =xlog_regrant_write_log_space (log, internal_ticket); }else {/* may sleep if need to allocate more tickets */
@@ -334,11 +344,9 @@ xfs_log_reserve(trace_xfs_log_reserve (log, internal_ticket); -spin_lock (&log->l_grant_lock);xlog_grant_push_ail (log, (internal_ticket->t_unit_res * internal_ticket->t_cnt)); -spin_unlock (&log->l_grant_lock); retval =xlog_grant_log_space (log, internal_ticket); }
@@ -1057,7 +1065,6 @@ xlog_alloc_log(xfs_mount_t *mp, log->l_xbuf = bp;spin_lock_init (&log->l_icloglock); -spin_lock_init (&log->l_grant_lock);init_waitqueue_head (&log->l_flush_wait);/* log record size must be multiple of BBSIZE; see xlog_rec_header_t */
@@ -1135,7 +1142,6 @@ out_free_iclog:kmem_free (iclog); }spinlock_destroy (&log->l_icloglock); -spinlock_destroy (&log->l_grant_lock);xfs_buf_free (log->l_xbuf); out_free_log:kmem_free (log);
@@ -1331,10 +1337,8 @@ xlog_sync(xlog_t *log, roundoff < BBTOB(1)));/* move grant heads by roundoff in sync */ -spin_lock (&log->l_grant_lock);xlog_grant_add_space (log, &log->l_grant_reserve_head, roundoff);xlog_grant_add_space (log, &log->l_grant_write_head, roundoff); -spin_unlock (&log->l_grant_lock);/* put cycle number in every block */ xlog_pack_data (log, iclog, roundoff);
@@ -1455,7 +1459,6 @@ xlog_dealloc_log(xlog_t *log) iclog = next_iclog; }spinlock_destroy (&log->l_icloglock); -spinlock_destroy (&log->l_grant_lock);xfs_buf_free (log->l_xbuf); log->l_mp->m_log =NULL ;
@@ -2574,13 +2577,10 @@ redo: }/* we've got enough space */ -spin_lock (&log->l_grant_lock);xlog_grant_add_space (log, &log->l_grant_reserve_head, need_bytes);xlog_grant_add_space (log, &log->l_grant_write_head, need_bytes);trace_xfs_log_grant_exit (log, tic); -xlog_verify_grant_head (log,1 );xlog_verify_grant_tail (log); -spin_unlock (&log->l_grant_lock);return 0 ; error_return_unlocked:
@@ -2694,12 +2694,9 @@ redo: }/* we've got enough space */ -spin_lock (&log->l_grant_lock);xlog_grant_add_space (log, &log->l_grant_write_head, need_bytes);trace_xfs_log_regrant_write_exit (log, tic); -xlog_verify_grant_head (log,1 );xlog_verify_grant_tail (log); -spin_unlock (&log->l_grant_lock);return 0 ;
@@ -2737,7 +2734,6 @@ xlog_regrant_reserve_log_space(xlog_t *log,if (ticket->t_cnt >0 ) ticket->t_cnt--; -spin_lock (&log->l_grant_lock);xlog_grant_sub_space (log, &log->l_grant_reserve_head, ticket->t_curr_res); xlog_grant_sub_space(log, &log->l_grant_write_head,
@@ -2747,21 +2743,15 @@ xlog_regrant_reserve_log_space(xlog_t *log,trace_xfs_log_regrant_reserve_sub (log, ticket); -xlog_verify_grant_head (log,1 ); -/* just return if we still have some of the pre-reserved space */ -if (ticket->t_cnt >0 ) { -spin_unlock (&log->l_grant_lock); +if (ticket->t_cnt >0 )return ; - }xlog_grant_add_space (log, &log->l_grant_reserve_head, ticket->t_unit_res);trace_xfs_log_regrant_reserve_exit (log, ticket); -xlog_verify_grant_head (log,0 ); -spin_unlock (&log->l_grant_lock); ticket->t_curr_res = ticket->t_unit_res;xlog_tic_reset_res (ticket); }/* xlog_regrant_reserve_log_space */
@@ -2790,7 +2780,6 @@ xlog_ungrant_log_space(xlog_t *log,if (ticket->t_cnt >0 ) ticket->t_cnt--; -spin_lock (&log->l_grant_lock);trace_xfs_log_ungrant_enter (log, ticket);trace_xfs_log_ungrant_sub (log, ticket);
@@ -2809,8 +2798,6 @@ xlog_ungrant_log_space(xlog_t *log,trace_xfs_log_ungrant_exit (log, ticket); -xlog_verify_grant_head (log,1 ); -spin_unlock (&log->l_grant_lock);xfs_log_move_tail (log->l_mp,1 ); }/* xlog_ungrant_log_space */
@@ -3429,28 +3416,6 @@ xlog_verify_dest_ptr( } STATIC void -xlog_verify_grant_head (xlog_t *log,int equals) -{ -int reserve_cycle, reserve_space; -int write_cycle, write_space; - -xlog_crack_grant_head (&log->l_grant_reserve_head, - &reserve_cycle, &reserve_space); -xlog_crack_grant_head (&log->l_grant_write_head, - &write_cycle, &write_space); - -if (reserve_cycle == write_cycle) { -if (equals) -ASSERT (reserve_space >= write_space); -else -ASSERT (reserve_space > write_space); - }else { -ASSERT (reserve_cycle -1 == write_cycle); -ASSERT (write_space >= reserve_space); - } -} - -STATIC voidxlog_verify_grant_tail (struct log *log) {
fs/xfs/xfs_log_priv.h +15 -8
@@ -511,9 +511,6 @@ typedef struct log {int l_curr_block;/* current logical log block */ int l_prev_block;/* previous logical log block */ -/* The following block of fields are changed while holding grant_lock */ -spinlock_t l_grant_lock____cacheline_aligned_in_smp ; -/* * l_last_sync_lsn and l_tail_lsn are atomics so they can be set and * read without needing to hold specific locks. To avoid operations
@@ -597,23 +594,33 @@ xlog_assign_atomic_lsn(atomic64_t *lsn, uint cycle, uint block) }/* -* When we crack the grrant head, we sample it first so that the value will not +* When we crack the grant head, we sample it first so that the value will not * change while we are cracking it into the component values. This means we * will always get consistent component values to work from. */ static inline void -xlog_crack_grant_head (atomic64_t *head,int *cycle,int *space) +xlog_crack_grant_head_val (int64_t val,int *cycle,int *space) { -int64_t val =atomic64_read (head); - *cycle = val >>32 ; *space = val &0xffffffff ; }static inline void +xlog_crack_grant_head (atomic64_t *head,int *cycle,int *space) +{ +xlog_crack_grant_head_val (atomic64_read (head), cycle, space); +} + +static inline int64_t +xlog_assign_grant_head_val (int cycle,int space) +{ +return ((int64_t )cycle <<32 ) | space; +} + +static inline void xlog_assign_grant_head (atomic64_t *head,int cycle,int space) { -atomic64_set (head, ((int64_t )cycle <<32 ) | space); +atomic64_set (head,xlog_assign_grant_head_val (cycle, space)); }/*
Signature · 6 lines
1.7.2.3 _______________________________________________ xfs mailing list [email protected] http://oss.sgi.com/mailman/listinfo/xfs
Re: [PATCH 11/12] xfs: convert grant head manipulations to lockless algorithm
Looks good,
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
PATCH 12/12 xfs: kill useless spinlock_destroy macro
In reply to Dave ChinnerFrom: Dave Chinner <[email protected]>
It is only used in 2 places in the log code, and is an empty macro, so get rid of it.
--- fs/xfs/linux-2.6/xfs_linux.h | 2 -- fs/xfs/xfs_log.c | 2 -- 2 files changed, 0 insertions(+), 4 deletions(-)
fs/xfs/linux-2.6/xfs_linux.h +0 -2
@@ -113,8 +113,6 @@#define current_restore_flags_nested (sp, f) \ (current->flags = ((current->flags & ~(f)) | (*(sp) & (f)))) -#define spinlock_destroy (lock) -#define NBBY 8/* number of bits per byte */ /*
fs/xfs/xfs_log.c +0 -2
@@ -1141,7 +1141,6 @@ out_free_iclog:xfs_buf_free (iclog->ic_bp);kmem_free (iclog); } -spinlock_destroy (&log->l_icloglock);xfs_buf_free (log->l_xbuf); out_free_log:kmem_free (log);
@@ -1458,7 +1457,6 @@ xlog_dealloc_log(xlog_t *log)kmem_free (iclog); iclog = next_iclog; } -spinlock_destroy (&log->l_icloglock);xfs_buf_free (log->l_xbuf); log->l_mp->m_log =NULL ;
Signature · 6 lines
1.7.2.3 _______________________________________________ xfs mailing list [email protected] http://oss.sgi.com/mailman/listinfo/xfs