xfs: fix log space reservation calculation if log stripe unit is specified
PATCH 1/1 xfs: fix log space reservation calculation if log stripe unit is specified
Hello,
About two weeks ago, Dave has found an issue by running xfstests/297.
http://oss.sgi.com/archives/xfs/2013-03/msg00273.html
According to our previous discussion, if the log stripe unit is configured, we should take it into account as it will dynamically increase the log reservation twice of it per ticket.
This patch is trying to fix it by checking the given log space against the maximum request among those transactions(this procedure is implemented similar to xfsprogs/mkfs/maxres.c), because the fundamental limit is that no single transaction can be larger than half of the log. Also, looks at least another two log stripe unit should be added when calculating the minimum log space, or else I can simply trigger a DEAD LOOP via create large number of files, I think I need some time to digest Dave's comments posted on original bug ticket, i.e.
Quoted 4 lines
Quoted 4 lines
The question is this: how much space do we need to reserve. I'm thinking a minimum of 4*lsu - 2*lsu for the existing CIL context, and another 2*lsu for any queued ticket waiting for space to come available.
Put simply, with this fix, mount a partition with an improper log space setup vs log stripe unit will failed although mkfs still works. Ah, maybe we can improve the user space xfs_mkfs with some pre-checkup similar to the implementation inside kernel? Besides that, it will drop a warning to syslog and the suggested log space for the given log stripe unit is shown there, which looks like the following:
# mkfs.xfs -f -b size=512 -d agcount=16,su=256k,sw=12 -l su=256k,size=2560b /dev/sdb1
meta-data=/dev/sdb1 isize=256 agcount=16, agsize=524288 blks
= sectsz=512 attr=2, projid32bit=0
data = bsize=512 blocks=8388608, imaxpct=25
= sunit=512 swidth=6144 blks
naming =version 2 bsize=4096 ascii-ci=0
log =internal log bsize=512 blocks=2560, version=2
= sectsz=512 sunit=512 blks, lazy-count=1
realtime =none extsz=4096 blocks=0, rtextents=0# mount /dev/sdb1 /xfs1
mount: wrong fs type, bad option, bad superblock on /dev/sdb1,
missing codepage or helper program, or other error
In some cases useful info is found in syslog - try
dmesg | tail or so# dmesg:
.......
XFS (sdb1): log space of 2560 blocks too small, minimum request 6656 XFS (sdb1): log space validation failed
XFS (sdb1): log mount failed
Tests:
Ran some cases in xfstests as well as a few self-defined Bonnie++/FIO tests with above configuration(6656 log blocks), looks the current fix works, at least no crash to me. :)
But I have not yet dig into the detailed of how the suggested minimum log space would affect the performance, given that the AIL push thresholds is defined to 25% of the log space, a small logs might introduce IO overheads for pushing AIL too frequently. In addition, considering the backgroup CIL commit threshold is 1/8 of the log, this would also impact the log IO throughput IMHO. Maybe we can figure out an optimized log space combine those two cases and drop it to syslog along with the minimum size?
To Dave,
Sorry for the delay in drop this patch since I have mentioned that I'll post a fix last night. However, I have ran into an issue when testing it by creating/removing a tons of files in parallel at that time :(
--- fs/xfs/xfs_log.c | 130 ++++++++++++++++++++++++++++++++++------- fs/xfs/xfs_log.h | 3 + fs/xfs/xfs_mount.c | 1 + fs/xfs/xfs_mount.h | 61 +++++++++++--------- fs/xfs/xfs_trans.c | 163 +++++++++++++++++++++++++++++++++++++++++++--------- fs/xfs/xfs_trans.h | 61 ++++++++++---------- 6 files changed, 314 insertions(+), 105 deletions(-)
fs/xfs/xfs_log.c +108 -20
@@ -598,6 +598,64 @@ xfs_log_release_iclog( }/* +* Check if the specified log space is sufficient for a file system +* with a given log strip unit. +*/ +STATIC int +xfs_mount_validate_log_size ( +struct xfs_mount *mp) +{ +struct xlog *log = mp->m_log; +struct xfs_trans_res tres; +int unit_bytes; +int min_lblks, lsu =0 ; + +xfs_max_trans_res_by_mount (mp, &tres); + +/* +* Figure out the total space needed for the maximum transaction +* log space reservation by adding some extra spaces which should +* be taken into account. +*/ + unit_bytes =xlog_ticket_unit_res (log, &tres); +if (tres.tr_cnt >1 ) + unit_bytes = unit_bytes * tres.tr_cnt; + + min_lblks =BTOBB (unit_bytes); +/* +* FIXME: why we should add another 2 log strip units if it +* is specified? As per my tryout, creat a dozens dirs/files +* on a partition without another 2 log strip units will +* cause DEAD LOOP, it's fine if taken this into account. +* +* As per Dave's comments: +* I'm thinking a minimum of 4*lsu - 2*lsu for the existing +* CIL context, and another 2*lsu for any queued ticket +* waiting for space to come available. +*/ +if (xfs_sb_version_haslogv2 (&log->l_mp->m_sb) && + log->l_mp->m_sb.sb_logsunit >1 ) + lsu =BTOBB (log->l_mp->m_sb.sb_logsunit); + +/* +* The fundamental limit is that no single transaction can be +* larger than half the size of the log space, take another +* two log strip unit account as well. +*/ +if ((log->l_logBBsize >>1 ) < (min_lblks + lsu)) { +xfs_warn (mp, +"log space of %d blocks too small, minimum request %d" , + log->l_logBBsize, +roundup ((int )min_lblks <<1 , (int )lsu) + +2 * lsu); + +return XFS_ERROR (EINVAL ); + } + +return 0 ; +} + +/* * Mount a log filesystem * * mp - ubiquitous xfs mount point structure
@@ -630,6 +688,12 @@ xfs_log_mount( goto out; } + error =xfs_mount_validate_log_size (mp); +if (error) { +xfs_warn (mp,"log space validation failed" ); + goto out_free_log; + } +/* * Initialize the AIL now we have a log. */
@@ -3377,24 +3441,23 @@ xfs_log_ticket_get( }/* -* Allocate and initialise a new log ticket. +* Figure out how many bytes would be reserved totally per ticket. +* Especially, take log strip unit into account if it is specified. +* +* FIXME: this is totally copied from xlog_ticket_alloc(), it's better +* to introduce a new helper to calculate the extra space reservation +* that can be shared with xlog_ticket_alloc() if the current though +* is reasonable. */ -struct xlog_ticket * -xlog_ticket_alloc ( -struct xlog *log, -int unit_bytes, -int cnt, -char client, -bool permanent, -xfs_km_flags_t alloc_flags) +int +xlog_ticket_unit_res ( +struct xlog *log, +struct xfs_trans_res *tres) { -struct xlog_ticket *tic; -uint num_headers; -int iclog_space; - - tic =kmem_zone_zalloc (xfs_log_ticket_zone, alloc_flags); -if (!tic) -return NULL ; +uint unit_bytes = tres->tr_res; +int total_bytes = unit_bytes; +int iclog_space; +uint num_headers;/* * Permanent reservations have up to 'cnt'-1 active log operations
@@ -3459,8 +3522,8 @@ xlog_ticket_alloc(/* add extra header reservations if we overrun */ while (!num_headers || -howmany (unit_bytes, iclog_space) > num_headers) { - unit_bytes +=sizeof (xlog_op_header_t); +howmany (total_bytes, iclog_space) > num_headers) { + total_bytes +=sizeof (xlog_op_header_t); num_headers++; } unit_bytes += log->l_iclog_hsize * num_headers;
@@ -3478,11 +3541,38 @@ xlog_ticket_alloc( unit_bytes +=2 *BBSIZE ; } + return unit_bytes; +} + +/* +* Allocate and initialise a new log ticket. +*/ +struct xlog_ticket * +xlog_ticket_alloc ( +struct xlog *log, +int unit_bytes, +int cnt, +char client, +bool permanent, +xfs_km_flags_t alloc_flags) +{ +struct xlog_ticket *tic; +struct xfs_trans_res tres; +int unit_res; + + tic =kmem_zone_zalloc (xfs_log_ticket_zone, alloc_flags); +if (!tic) +return NULL ; + + tres.tr_res = unit_bytes; + tres.tr_cnt = cnt; + unit_res =xlog_ticket_unit_res (log, &tres); +atomic_set (&tic->t_ref,1 ); tic->t_task = current;INIT_LIST_HEAD (&tic->t_queue); - tic->t_unit_res = unit_bytes; - tic->t_curr_res = unit_bytes; + tic->t_unit_res = unit_res; + tic->t_curr_res = unit_res; tic->t_cnt = cnt; tic->t_ocnt = cnt; tic->t_tid =random32 ();
fs/xfs/xfs_log.h +3 -0
@@ -119,11 +119,13 @@ typedef struct xfs_log_callback {#ifdef __KERNEL__/* Log manager interfaces */ struct xfs_mount ; +struct xlog ;struct xlog_in_core ;struct xlog_ticket ;struct xfs_log_item ;struct xfs_item_ops ;struct xfs_trans ; +struct xfs_trans_res ; void xfs_log_item_init(struct xfs_mount *mp, struct xfs_log_item *item,
@@ -184,6 +186,7 @@ bool xfs_log_item_in_current_chkpt(struct xfs_log_item *lip);void xfs_log_work_queue (struct xfs_mount *mp);void xfs_log_worker (struct work_struct *work);void xfs_log_quiesce (struct xfs_mount *mp); +int xlog_ticket_unit_res (struct xlog *log,struct xfs_trans_res *tres);#endif #endif /* __XFS_LOG_H__ */
fs/xfs/xfs_mount.c +1 -0
@@ -20,6 +20,7 @@#include "xfs_types.h" #include "xfs_bit.h" #include "xfs_log.h" +#include "xfs_log_priv.h" #include "xfs_inum.h" #include "xfs_trans.h" #include "xfs_trans_priv.h"
fs/xfs/xfs_mount.h +32 -28
66 lines not on this page · raw message
fs/xfs/xfs_trans.c +135 -27
172 lines not on this page · raw message
fs/xfs/xfs_trans.h +32 -28
77 lines not on this page · raw message
Signature · 6 lines
1.7.9.5 _______________________________________________ xfs mailing list [email protected] http://oss.sgi.com/mailman/listinfo/xfs
Quoted 26 lines · Jeff Liu
Hello,
About two weeks ago, Dave has found an issue by running xfstests/297.
http://oss.sgi.com/archives/xfs/2013-03/msg00273.htmlAccording to our previous discussion, if the log stripe unit is configured, we should take it into account as it will dynamically increase the log reservation twice of it per ticket.
This patch is trying to fix it by checking the given log space against the maximum request among those transactions(this procedure is implemented similar to
xfsprogs/mkfs/maxres.c), because the fundamental limit is that no single transaction can be larger than half of the log. Also, looks at least another two log stripe unit should be added when calculating the minimum log space, or else I can simply trigger a DEAD LOOP via create large number of files, I think I need some time to digest Dave's comments posted on original bug ticket, i.e.Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
The question is this: how much space do we need to reserve. I'm thinking a minimum of 4*lsu - 2*lsu for the existing CIL context, and another 2*lsu for any queued ticket waiting for space to come available.
Put simply, with this fix, mount a partition with an improper log space setup vs log stripe unit will failed although mkfs still works. Ah, maybe we can improve the user space
xfs_mkfswith some pre-checkup similar to the implementation inside kernel? Besides that, it will drop a warning to syslog and the suggested log space for the given log stripe unit is shown there, which looks like the following:
Quoted 10 lines
# mkfs.xfs -f -b size=512 -d agcount=16,su=256k,sw=12 -l su=256k,size=2560b /dev/sdb1
meta-data=/dev/sdb1 isize=256 agcount=16, agsize=524288 blks
= sectsz=512 attr=2, projid32bit=0
data = bsize=512 blocks=8388608, imaxpct=25
= sunit=512 swidth=6144 blks
naming =version 2 bsize=4096 ascii-ci=0
log =internal log bsize=512 blocks=2560, version=2
= sectsz=512 sunit=512 blks, lazy-count=1
realtime =none extsz=4096 blocks=0, rtextents=0Shouldn't mkfs.xfs also know it is building a filesystem that cannot be mounted?
When mkfs.xfs is given a log stripe unit is greater than 256KB, should we divide the specified log stripe unit by 2 until it is under 256KB rather than reset to 32KB?
Quoted 17 lines
# mount /dev/sdb1 /xfs1 mount: wrong fs type, bad option, bad superblock on /dev/sdb1, missing codepage or helper program, or other error In some cases useful info is found in syslog - try dmesg | tail or so# dmesg:
.......
XFS (sdb1): log space of 2560 blocks too small, minimum request 6656 XFS (sdb1): log space validation failed
XFS (sdb1): log mount failedTests:
Ran some cases in xfstests as well as a few self-defined Bonnie++/FIO tests with above configuration(6656 log blocks), looks the current fix works, at least no crash to me.:)
Quoted 7 lines
But I have not yet dig into the detailed of how the suggested minimum log space would affect the performance, given that the AIL push thresholds is defined to 25% of the log space, a small logs might introduce IO overheads for pushing AIL too frequently. In addition, considering the backgroup CIL commit threshold is 1/8 of the log, this would also impact the log IO throughput IMHO. Maybe we can figure out an optimized log space combine those two cases and drop it to syslog along with the minimum size?
I think 1 MB is the smallest log size before we soft hang even without stripe units define.
Quoted 6 lines
To Dave,
Sorry for the delay in drop this patch since I have mentioned that I'll post a fix last night. However, I have ran into an issue when testing it by creating/removing a tons of files in parallel at that time:(
The iclog buffers have to be a multiple of the log stripe unit or we start punching the lsn in places that it should not. I think the idea that was mentioned is to remove the power of two on the iclog buffer size and replace with multiple of log stripe unit.
http://oss.sgi.com/archives/xfs/2013-03/msg00039.html
--Mark.
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
Quoted 38 lines · Mark Tinguely
Quoted 26 lines · Jeff Liu
Hello,
About two weeks ago, Dave has found an issue by running xfstests/297.
http://oss.sgi.com/archives/xfs/2013-03/msg00273.htmlAccording to our previous discussion, if the log stripe unit is configured, we should take it into account as it will dynamically increase the log reservation twice of it per ticket.
This patch is trying to fix it by checking the given log space against the maximum request among those transactions(this procedure is implemented similar to
xfsprogs/mkfs/maxres.c), because the fundamental limit is that no single transaction can be larger than half of the log. Also, looks at least another two log stripe unit should be added when calculating the minimum log space, or else I can simply trigger a DEAD LOOP via create large number of files, I think I need some time to digest Dave's comments posted on original bug ticket, i.e.Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
The question is this: how much space do we need to reserve. I'm thinking a minimum of 4*lsu - 2*lsu for the existing CIL context, and another 2*lsu for any queued ticket waiting for space to come available.
Put simply, with this fix, mount a partition with an improper log space setup vs log stripe unit will failed although mkfs still works. Ah, maybe we can improve the user space
xfs_mkfswith some pre-checkup similar to the implementation inside kernel? Besides that, it will drop a warning to syslog and the suggested log space for the given log stripe unit is shown there, which looks like the following:Quoted 10 lines
# mkfs.xfs -f -b size=512 -d agcount=16,su=256k,sw=12 -l su=256k,size=2560b /dev/sdb1 meta-data=/dev/sdb1 isize=256 agcount=16, agsize=524288 blks = sectsz=512 attr=2, projid32bit=0 data = bsize=512 blocks=8388608, imaxpct=25 = sunit=512 swidth=6144 blks naming =version 2 bsize=4096 ascii-ci=0 log =internal log bsize=512 blocks=2560, version=2 = sectsz=512 sunit=512 blks, lazy-count=1 realtime =none extsz=4096 blocks=0, rtextents=0
Quoted 2 lines
Shouldn't mkfs.xfs also know it is building a filesystem that cannot be mounted?
Yes, it should - this is what mkfs/maxtrres.c does. It looks like some of this patch came from that code.....
Quoted 3 lines
When mkfs.xfs is given a log stripe unit is greater than 256KB, should we divide the specified log stripe unit by 2 until it is under 256KB rather than reset to 32KB?
I think if it is specified on the command line, it shoul dbe rejected. If it's automatically determined from the data device sunit, then the divide-by-2-until-in-range algorithm seems fine to me.
Quoted 14 lines
Quoted 14 lines
# mount /dev/sdb1 /xfs1 mount: wrong fs type, bad option, bad superblock on /dev/sdb1, missing codepage or helper program, or other error In some cases useful info is found in syslog - try dmesg | tail or so# dmesg:
.......
XFS (sdb1): log space of 2560 blocks too small, minimum request 6656 XFS (sdb1): log space validation failed
XFS (sdb1): log mount failed
Quoted 7 lines
Quoted 7 lines
Tests:
Ran some cases in xfstests as well as a few self-defined Bonnie++/FIO tests with above configuration(6656 log blocks), looks the current fix works, at least no crash to me.:)But I have not yet dig into the detailed of how the suggested minimum log space would affect the performance, given that the AIL push thresholds is defined to 25% of the log space, a small logs might introduce IO overheads for pushing AIL too frequently.
That's already a problem of using small logs. That's not something we need to worry about when trying to determine the minimum valid log size....
Quoted 3 lines
Quoted 3 lines
In addition, considering the backgroup CIL commit threshold is 1/8 of the log, this would also impact the log IO throughput IMHO. Maybe we can figure out an optimized log space combine those two cases and drop it to syslog along with the minimum size?
Anyone who cares about metadata performance on their small filesystems is not going to use a minimum sized log. As it is, on any filesystem larger than about 50GB using a default log size (about 25MB for a default mkfs), the log stripe unit simply isn't an issue...
Quoted 2 lines
I think 1 MB is the smallest log size before we soft hang even without stripe units define.
$ sudo mkfs.xfs -f -l size=256b -dsize=1g /dev/vdc log size 256 blocks too small, minimum size is 512 blocks $
mkfs won't allow a log size smaller than 2MB for default configurations.
Quoted 6 lines
Quoted 5 lines
To Dave,
Sorry for the delay in drop this patch since I have mentioned that I'll post a fix last night. However, I have ran into an issue when testing it by creating/removing a tons of files in parallel at that time:(
Quoted 6 lines
The iclog buffers have to be a multiple of the log stripe unit or we start punching the lsn in places that it should not. I think the idea that was mentioned is to remove the power of two on the iclog buffer size and replace with multiple of log stripe unit.
Right:
| Personally, I'd prefer that logbsize be limited to power-of-2 | multiples of the lsunit or XLOG_MIN_RECORD_BSIZE (if lsunit = 0) as | allowing arbitrary values to be specified by users leads to a | testing and bug triage nightmare.
ie I think that the valid values for logbsize are:
32k <= logbsize <= 256k
logbsize = logsunit * 2^N for N that does not violate the first rule.
Cheers,
Dave.
Signature · 7 lines
Dave Chinner [email protected] _______________________________________________ xfs mailing list [email protected] http://oss.sgi.com/mailman/listinfo/xfs
Quoted 39 lines · Dave Chinner
Quoted 38 lines · Mark Tinguely
Quoted 36 lines · Jeff Liu
Hello,
About two weeks ago, Dave has found an issue by running xfstests/297.
http://oss.sgi.com/archives/xfs/2013-03/msg00273.htmlAccording to our previous discussion, if the log stripe unit is configured, we should take it into account as it will dynamically increase the log reservation twice of it per ticket.
This patch is trying to fix it by checking the given log space against the maximum request among those transactions(this procedure is implemented similar to
xfsprogs/mkfs/maxres.c), because the fundamental limit is that no single transaction can be larger than half of the log. Also, looks at least another two log stripe unit should be added when calculating the minimum log space, or else I can simply trigger a DEAD LOOP via create large number of files, I think I need some time to digest Dave's comments posted on original bug ticket, i.e.Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
The question is this: how much space do we need to reserve. I'm thinking a minimum of 4*lsu - 2*lsu for the existing CIL context, and another 2*lsu for any queued ticket waiting for space to come available.
Put simply, with this fix, mount a partition with an improper log space setup vs log stripe unit will failed although mkfs still works. Ah, maybe we can improve the user space
xfs_mkfswith some pre-checkup similar to the implementation inside kernel? Besides that, it will drop a warning to syslog and the suggested log space for the given log stripe unit is shown there, which looks like the following:# mkfs.xfs -f -b size=512 -d agcount=16,su=256k,sw=12 -l su=256k,size=2560b /dev/sdb1 meta-data=/dev/sdb1 isize=256 agcount=16, agsize=524288 blks = sectsz=512 attr=2, projid32bit=0 data = bsize=512 blocks=8388608, imaxpct=25 = sunit=512 swidth=6144 blks naming =version 2 bsize=4096 ascii-ci=0 log =internal log bsize=512 blocks=2560, version=2 = sectsz=512 sunit=512 blks, lazy-count=1 realtime =none extsz=4096 blocks=0, rtextents=0
Quoted 5 lines
Quoted 2 lines
Shouldn't mkfs.xfs also know it is building a filesystem that cannot be mounted?
Yes, it should - this is what
mkfs/maxtrres.cdoes. It looks like some of this patch came from that code.....
Yes, especially for figuring out the transaction with maximum log space reservation. I'm going to sync up kernel changes to user space from the next round post.
Quoted 40 lines
Quoted 4 lines
When mkfs.xfs is given a log stripe unit is greater than 256KB, should we divide the specified log stripe unit by 2 until it is under 256KB rather than reset to 32KB?
I think if it is specified on the command line, it shoul dbe rejected. If it's automatically determined from the data device sunit, then the divide-by-2-until-in-range algorithm seems fine to me.
Quoted 14 lines
Quoted 14 lines
# mount /dev/sdb1 /xfs1 mount: wrong fs type, bad option, bad superblock on /dev/sdb1, missing codepage or helper program, or other error In some cases useful info is found in syslog - try dmesg | tail or so# dmesg:
.......
XFS (sdb1): log space of 2560 blocks too small, minimum request 6656 XFS (sdb1): log space validation failed
XFS (sdb1): log mount failedQuoted 7 lines
Quoted 7 lines
Tests:
Ran some cases in xfstests as well as a few self-defined Bonnie++/FIO tests with above configuration(6656 log blocks), looks the current fix works, at least no crash to me.:)But I have not yet dig into the detailed of how the suggested minimum log space would affect the performance, given that the AIL push thresholds is defined to 25% of the log space, a small logs might introduce IO overheads for pushing AIL too frequently.
That's already a problem of using small logs. That's not something we need to worry about when trying to determine the minimum valid log size....
Quoted 3 lines
Quoted 3 lines
In addition, considering the backgroup CIL commit threshold is 1/8 of the log, this would also impact the log IO throughput IMHO. Maybe we can figure out an optimized log space combine those two cases and drop it to syslog along with the minimum size?
Quoted 5 lines
Anyone who cares about metadata performance on their small filesystems is not going to use a minimum sized log. As it is, on any filesystem larger than about 50GB using a default log size (about 25MB for a default mkfs), the log stripe unit simply isn't an issue...
Thanks for the clarification.
Quoted 31 lines
Quoted 3 lines
I think 1 MB is the smallest log size before we soft hang even without stripe units define.
$ sudo mkfs.xfs -f -l size=256b -dsize=1g /dev/vdc log size 256 blocks too small, minimum size is 512 blocks $mkfs won't allow a log size smaller than 2MB for default configurations.
Quoted 6 lines
Quoted 5 lines
To Dave,
Sorry for the delay in drop this patch since I have mentioned that I'll post a fix last night. However, I have ran into an issue when testing it by creating/removing a tons of files in parallel at that time:(
Quoted 6 lines
The iclog buffers have to be a multiple of the log stripe unit or we start punching the lsn in places that it should not. I think the idea that was mentioned is to remove the power of two on the iclog buffer size and replace with multiple of log stripe unit.
Right:
| Personally, I'd prefer that logbsize be limited to power-of-2 | multiples of the lsunit or XLOG_MIN_RECORD_BSIZE (if lsunit = 0) as | allowing arbitrary values to be specified by users leads to a | testing and bug triage nightmare.
Quoted 4 lines
ie I think that the valid values for logbsize are:
32k <= logbsize <= 256k
logbsize = logsunit * 2^N for N that does not violate the first rule.
So I'll take this into account for validating log stripe unit.
Thanks,
-Jeff
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
Quoted 4 lines · Jeff Liu
Hello,
About two weeks ago, Dave has found an issue by running xfstests/297.
http://oss.sgi.com/archives/xfs/2013-03/msg00273.html
...
Quoted 9 lines
--- fs/xfs/xfs_log.c | 130 ++++++++++++++++++++++++++++++++++------- fs/xfs/xfs_log.h | 3 + fs/xfs/xfs_mount.c | 1 + fs/xfs/xfs_mount.h | 61 +++++++++++--------- fs/xfs/xfs_trans.c | 163 +++++++++++++++++++++++++++++++++++++++++++--------- fs/xfs/xfs_trans.h | 61 ++++++++++---------- 6 files changed, 314 insertions(+), 105 deletions(-)
Hmmmm. That's a lot more change that I expected.....
Quoted 10 lines
fs/xfs/xfs_log.c +3
-0@@ -598,6 +598,64 @@ xfs_log_release_iclog( }/* +* Check if the specified log space is sufficient for a file system +* with a given log strip unit. +*/
fs/xfs/xfs_log.c xfs_log_release_iclog( · line 612
Quoted 10 lines
fs/xfs/xfs_log.c +10
-0+STATIC int +xfs_mount_validate_log_size ( +struct xfs_mount *mp) +{ +struct xlog *log = mp->m_log; +struct xfs_trans_res tres; +int unit_bytes; +int min_lblks, lsu =0 ; + +xfs_max_trans_res_by_mount (mp, &tres);
Ok, that's been copied from mkfs, right?
What I'd suggest we need to do here is separate out all this reservation/validation code into it's own file so we can easily share it with libxfs in userspace.
fs/xfs/xfs_log.c xfs_log_release_iclog( · line 621
Quoted 9 lines
fs/xfs/xfs_log.c +9
-0+ +/* +* Figure out the total space needed for the maximum transaction +* log space reservation by adding some extra spaces which should +* be taken into account. +*/ + unit_bytes =xlog_ticket_unit_res (log, &tres); +if (tres.tr_cnt >1 ) + unit_bytes = unit_bytes * tres.tr_cnt;
Hmmmm - it's a bit different to userspace - there's no count in the userspace code. But yes, we do need to take into account the permanent log reservations...
Quoted 13 lines
fs/xfs/xfs_log.c +13
-0+ + min_lblks =BTOBB (unit_bytes); +/* +* FIXME: why we should add another 2 log strip units if it +* is specified? As per my tryout, creat a dozens dirs/files +* on a partition without another 2 log strip units will +* cause DEAD LOOP, it's fine if taken this into account. +* +* As per Dave's comments: +* I'm thinking a minimum of 4*lsu - 2*lsu for the existing +* CIL context, and another 2*lsu for any queued ticket +* waiting for space to come available. +*/
fs/xfs/xfs_log.c xfs_log_release_iclog( · line 644
Quoted 10 lines
fs/xfs/xfs_log.c +10
-0+if (xfs_sb_version_haslogv2 (&log->l_mp->m_sb) && + log->l_mp->m_sb.sb_logsunit >1 ) + lsu =BTOBB (log->l_mp->m_sb.sb_logsunit); + +/* +* The fundamental limit is that no single transaction can be +* larger than half the size of the log space, take another +* two log strip unit account as well. +*/ +if ((log->l_logBBsize >>1 ) < (min_lblks + lsu)) {
A transaction requires 2 LSU for the reservation because there are two log writes that can require padding - the transaction data and the commit record are written separately and both can require padding to the LSU.
And as per my comments above, we can have an active CIL reservation (holding 2*LSU), but the CIL is not over a push threshold. If we don't have space for at one new transaction, which includes *another* 2*LSU in the reservation, that's when we have problems. So, the log size needs to be able to contain two maximally sized and padded transactions, which is (2 * (2 * LSU + maxtrres)). Hence if you are comparing this against half the log size (i.e. maximum transaction size), it needs to be (2 * (2 * LSU + maxtrres)) / 2. i.e. (minlblks + 2 * lsu)
fs/xfs/xfs_log.c xfs_log_release_iclog( · line 651
Quoted 7 lines
fs/xfs/xfs_log.c +7
-0+xfs_warn (mp, +"log space of %d blocks too small, minimum request %d" , + log->l_logBBsize, +roundup ((int )min_lblks <<1 , (int )lsu) + +2 * lsu); + +return XFS_ERROR (EINVAL );
But, we can't just reject the mount if this fails. This would mean that people would have to downgrade their kernel just to remedy the situation as there is no way to grow the log (short of black magic surgery with xfs_db).
So this should just remain a warning message, though I'd make it of "xfs_crit" level (i.e. critical) so people notice it as well as making the message a little more informative.
fs/xfs/xfs_log.c xfs_log_ticket_get( · line 3449
Quoted 11 lines
fs/xfs/xfs_log.c +7
-1@@ -3377,24 +3441,23 @@ xfs_log_ticket_get( }/* -* Allocate and initialise a new log ticket. +* Figure out how many bytes would be reserved totally per ticket. +* Especially, take log strip unit into account if it is specified. +* +* FIXME: this is totally copied from xlog_ticket_alloc(), it's better +* to introduce a new helper to calculate the extra space reservation +* that can be shared with xlog_ticket_alloc() if the current though +* is reasonable.
That FIXME looks like you've already fixed it ;)
Quoted 28 lines
fs/xfs/xfs_log.c +8
-15*/ -struct xlog_ticket * -xlog_ticket_alloc ( -struct xlog *log, -int unit_bytes, -int cnt, -char client, -bool permanent, -xfs_km_flags_t alloc_flags) +int +xlog_ticket_unit_res ( +struct xlog *log, +struct xfs_trans_res *tres) { -struct xlog_ticket *tic; -uint num_headers; -int iclog_space; - - tic =kmem_zone_zalloc (xfs_log_ticket_zone, alloc_flags); -if (!tic) -return NULL ; +uint unit_bytes = tres->tr_res; +int total_bytes = unit_bytes; +int iclog_space; +uint num_headers;/* * Permanent reservations have up to 'cnt'-1 active log operations
fs/xfs/xfs_log.c xlog_ticket_alloc( · line 3528
Quoted 9 lines
fs/xfs/xfs_log.c +2
-2@@ -3459,8 +3522,8 @@ xlog_ticket_alloc(/* add extra header reservations if we overrun */ while (!num_headers || -howmany (unit_bytes, iclog_space) > num_headers) { - unit_bytes +=sizeof (xlog_op_header_t); +howmany (total_bytes, iclog_space) > num_headers) { + total_bytes +=sizeof (xlog_op_header_t); num_headers++; } unit_bytes += log->l_iclog_hsize * num_headers;
What is the reason for using total_bytes here? We've got to take into account the size of the xlog_op_header_t headers in the ticket reservation, so adding them to unit_bytes is correct AFAICT....
fs/xfs/xfs_log.c xlog_ticket_alloc( · line 3544
Quoted 5 lines
fs/xfs/xfs_log.c +1
-0@@ -3478,11 +3541,38 @@ xlog_ticket_alloc( unit_bytes +=2 *BBSIZE ; } + return unit_bytes; +}
This patch hunk is broken.
Quoted 14 lines
fs/xfs/xfs_log.c +14
-0+ +/* +* Allocate and initialise a new log ticket. +*/ +struct xlog_ticket * +xlog_ticket_alloc ( +struct xlog *log, +int unit_bytes, +int cnt, +char client, +bool permanent, +xfs_km_flags_t alloc_flags) +{ +struct xlog_ticket *tic;
fs/xfs/xfs_log.c xlog_ticket_alloc( · line 3568
Quoted 10 lines
fs/xfs/xfs_log.c +10
-0+struct xfs_trans_res tres; +int unit_res; + + tic =kmem_zone_zalloc (xfs_log_ticket_zone, alloc_flags); +if (!tic) +return NULL ; + + tres.tr_res = unit_bytes; + tres.tr_cnt = cnt; + unit_res =xlog_ticket_unit_res (log, &tres);
Ok, I'm starting to see where this tres stuff is going. More on that later....
Quoted 29 lines
fs/xfs/xfs_log.c +3
-2+atomic_set (&tic->t_ref,1 ); tic->t_task = current;INIT_LIST_HEAD (&tic->t_queue); - tic->t_unit_res = unit_bytes; - tic->t_curr_res = unit_bytes; + tic->t_unit_res = unit_res; + tic->t_curr_res = unit_res; tic->t_cnt = cnt; tic->t_ocnt = cnt; tic->t_tid =random32 ();fs/xfs/xfs_log.h +2
-0@@ -119,11 +119,13 @@ typedef struct xfs_log_callback {#ifdef __KERNEL__/* Log manager interfaces */ struct xfs_mount ; +struct xlog ;struct xlog_in_core ;struct xlog_ticket ;struct xfs_log_item ;struct xfs_item_ops ;struct xfs_trans ; +struct xfs_trans_res ; void xfs_log_item_init(struct xfs_mount *mp, struct xfs_log_item *item,
fs/xfs/xfs_log.h bool xfs_log_item_in_current_chkpt(struct xfs_log_item *lip); · line 189
Quoted 4 lines
fs/xfs/xfs_log.h +1
-0@@ -184,6 +186,7 @@ bool xfs_log_item_in_current_chkpt(struct xfs_log_item *lip);void xfs_log_work_queue (struct xfs_mount *mp);void xfs_log_worker (struct work_struct *work);void xfs_log_quiesce (struct xfs_mount *mp); +int xlog_ticket_unit_res (struct xlog *log,struct xfs_trans_res *tres);
We generally name the log external functions as "xfs_log_..." and pass a struct xfs_mount around with them. i.e.:
int xfs_log_ticket_unit_res(struct xfs_mount *mp, struct xfs_trans_res *tres);
As it is, I'm not sure what that means from the name of the function. It has nothing to do with log tickets, but it's calculating the unit reservation for the ticket. Perhaps a better name is something like xfs_log_calc_unit_res()?
fs/xfs/xfs_mount.c line 23
Quoted 11 lines
fs/xfs/xfs_log.h +0
-0#endif #endif /* __XFS_LOG_H__ */ fs/xfs/xfs_mount.c +1
-0@@ -20,6 +20,7 @@#include "xfs_types.h" #include "xfs_bit.h" #include "xfs_log.h" +#include "xfs_log_priv.h"
stray include?
Quoted 8 lines
fs/xfs/xfs_mount.c +0
-0#include "xfs_inum.h" #include "xfs_trans.h" #include "xfs_trans_priv.h" fs/xfs/xfs_mount.h +0
-0
fs/xfs/xfs_mount.h line 23
Quoted 6 lines
fs/xfs/xfs_mount.h +3
-0@@ -18,35 +18,40 @@#ifndef __XFS_MOUNT_H__#define __XFS_MOUNT_H__ +typedef struct xfs_trans_res { +uint tr_res; +int tr_cnt; +} xfs_tr_t;
No new typedefs, please.
fs/xfs/xfs_trans.c line 46
Quoted 9 lines
fs/xfs/xfs_trans.c +1
-0@@ -43,6 +43,7 @@#include "xfs_inode_item.h" #include "xfs_log_priv.h" #include "xfs_buf_item.h" +#include "xfs_attr_leaf.h"
Another stray include?
Quoted 31 lines
fs/xfs/xfs_trans.c +0
-25#include "xfs_trace.h" kmem_zone_t *xfs_trans_zone;@@ -645,34 +646,140 @@ xfs_trans_init( {struct xfs_trans_reservations *resp = &mp->m_reservations; - resp->tr_write =xfs_calc_write_reservation (mp); - resp->tr_itruncate =xfs_calc_itruncate_reservation (mp); - resp->tr_rename =xfs_calc_rename_reservation (mp); - resp->tr_link =xfs_calc_link_reservation (mp); - resp->tr_remove =xfs_calc_remove_reservation (mp); - resp->tr_symlink =xfs_calc_symlink_reservation (mp); - resp->tr_create =xfs_calc_create_reservation (mp); - resp->tr_mkdir =xfs_calc_mkdir_reservation (mp); - resp->tr_ifree =xfs_calc_ifree_reservation (mp); - resp->tr_ichange =xfs_calc_ichange_reservation (mp); - resp->tr_growdata =xfs_calc_growdata_reservation (mp); - resp->tr_swrite =xfs_calc_swrite_reservation (mp); - resp->tr_writeid =xfs_calc_writeid_reservation (mp); - resp->tr_addafork =xfs_calc_addafork_reservation (mp); - resp->tr_attrinval =xfs_calc_attrinval_reservation (mp); - resp->tr_attrsetm =xfs_calc_attrsetm_reservation (mp); - resp->tr_attrsetrt =xfs_calc_attrsetrt_reservation (mp); - resp->tr_attrrm =xfs_calc_attrrm_reservation (mp); - resp->tr_clearagi =xfs_calc_clear_agi_bucket_reservation (mp); - resp->tr_growrtalloc =xfs_calc_growrtalloc_reservation (mp); - resp->tr_growrtzero =xfs_calc_growrtzero_reservation (mp); - resp->tr_growrtfree =xfs_calc_growrtfree_reservation (mp); - resp->tr_qm_sbchange =xfs_calc_qm_sbchange_reservation (mp); - resp->tr_qm_setqlim =xfs_calc_qm_setqlim_reservation (mp); - resp->tr_qm_dqalloc =xfs_calc_qm_dqalloc_reservation (mp); - resp->tr_qm_quotaoff =xfs_calc_qm_quotaoff_reservation (mp);
fs/xfs/xfs_trans.c xfs_trans_init( · line 656
Quoted 10 lines
fs/xfs/xfs_trans.c +8
-2- resp->tr_qm_equotaoff =xfs_calc_qm_quotaoff_end_reservation (mp); - resp->tr_sb =xfs_calc_sb_reservation (mp); + resp->tr_write.tr_res =xfs_calc_write_reservation (mp); + resp->tr_write.tr_cnt =XFS_WRITE_LOG_COUNT ; + + resp->tr_itruncate.tr_res =xfs_calc_itruncate_reservation (mp); + resp->tr_itruncate.tr_cnt =XFS_ITRUNCATE_LOG_COUNT ; + + resp->tr_rename.tr_res =xfs_calc_rename_reservation (mp); + resp->tr_rename.tr_cnt =XFS_RENAME_LOG_COUNT ;
.....
I like the idea, but I don't think you've carried it through far enough. :)
i.e. This patch leaves us with having multiple places where this information has to be maintained (here and the xfs_trans_reserve() calls). What I think is the best way to approach this is to separate out this table change into a separate patch (i.e. without all the other code that uses it), and then change the xfs_trans_reserve() interface to take a struct xfs_trans_res *.
At that point, we can then do:
xfs_trans_reserve(tp, &mp->m_reservations.tr_rename, blockres, rtblockres, flags);
and now we can propagate the logspace/logcount through xfs_log_reserve(), xlog_ticket_alloc() and so on via the reservation structure.
That leaves us with a single place that we set up and maintain log space reservations, makes the transaction reservation calls cleaner (no more messy macros everywhere), and if we rename mp->m_reservations to mp->m_resv there's a whole lot less typing, too.
We could potentially also add the XFS_TRANS_PERM_LOG_RES flag to the struct xfs_trans_res, so most xfs_trans_reserve() calls don't need to pass a flag in at all (even cleaner!).
Quoted 11 lines
fs/xfs/xfs_trans.c +11
-0+STATIC void +xfs_max_attrsetm_trans_res_adjust ( +struct xfs_mount *mp) +{ +int local; +int size; +int nblks; +int res; + +/* +* Determine space the maximal sized attribute will use,
fs/xfs/xfs_trans.c xfs_trans_init( · line 754
Quoted 10 lines
fs/xfs/xfs_trans.c +10
-0+* to calculate the largest reservatoin size needed. +*/ + size =xfs_attr_leaf_newentsize (MAXNAMELEN ,64 *1024 , + mp->m_sb.sb_blocksize, &local); +ASSERT (!local); + nblks =XFS_DAENTER_SPACE_RES (mp,XFS_ATTR_FORK ); + nblks +=XFS_B_TO_FSB (mp, size); + nblks +=XFS_NEXTENTADD_SPACE_RES (mp, size,XFS_ATTR_FORK ); + res =XFS_ATTRSETM_LOG_RES (mp) +XFS_ATTRSETRT_LOG_RES (mp) * nblks; + mp->m_reservations.tr_attrsetm.tr_res = res;
That's copied from mkfs, right? I need to look a bit closer, but I don't think this is correct - large attributes end up out of line and not logged, while this assumes that the full 64k of the remote attribute is logged. Over estimating is fine, though, for the moment.
Quoted 8 lines
fs/xfs/xfs_trans.c +8
-0+} + +/* +* Figure out the total log space a transaction would required in terms +* of the pre-calculated values which are done at mount time, then find +* out and return the maximum reservation among them. +*/ +void
fs/xfs/xfs_trans.c xfs_trans_init( · line 772
Quoted 10 lines
fs/xfs/xfs_trans.c +10
-0+xfs_max_trans_res_by_mount ( + struct xfs_mount *mp, + struct xfs_trans_res *mres) +{ + struct xfs_trans_reservations *resp = &mp->m_reservations; +struct xfs_trans_res *p, *tres =NULL ; +int res; + +for (res =0 , p = (struct xfs_trans_res *)resp; + p < (struct xfs_trans_res *)(resp +1 ); p++) {
I don't really like the pointer arithmetic here. Something like
res = 0;
for (i = 0; i < ARRAY_SIZE(mp->m_reservations); i++) {
p = &mp->m_reservations[i];is a much neater way of iterating an array....
fs/xfs/xfs_trans.c xfs_trans_init( · line 783
Quoted 11 lines
fs/xfs/xfs_trans.c +10
-0+int tmp = p->tr_cnt >1 ? p->tr_res * p->tr_cnt : + p->tr_res; +if (res < tmp) { + res = tmp; + tres = p; + } + } + +ASSERT (tres !=NULL ); + *mres = *tres; }
All these changes look to me like something we shoul dbe sharing with libxfs in userspace so that mkfs can re-use the code without modifications....
Quoted 33 lines
fs/xfs/xfs_trans.c +0
-0/* fs/xfs/xfs_trans.h +6
-14@@ -19,6 +19,7 @@#define __XFS_TRANS_H__struct xfs_log_item ; +struct xfs_trans_res ;/* * This is the structure written in the log at the head of @@ -232,39 +233,39 @@ struct xfs_log_item_desc { XFS_DAENTER_BMAPS(mp, XFS_DATA_FORK) + 1) -#define XFS_WRITE_LOG_RES(mp) ((mp)->m_reservations.tr_write) -#define XFS_ITRUNCATE_LOG_RES (mp) ((mp)->m_reservations.tr_itruncate) -#define XFS_RENAME_LOG_RES (mp) ((mp)->m_reservations.tr_rename) -#define XFS_LINK_LOG_RES (mp) ((mp)->m_reservations.tr_link) -#define XFS_REMOVE_LOG_RES (mp) ((mp)->m_reservations.tr_remove) -#define XFS_SYMLINK_LOG_RES (mp) ((mp)->m_reservations.tr_symlink) -#define XFS_CREATE_LOG_RES (mp) ((mp)->m_reservations.tr_create) -#define XFS_MKDIR_LOG_RES (mp) ((mp)->m_reservations.tr_mkdir) -#define XFS_IFREE_LOG_RES (mp) ((mp)->m_reservations.tr_ifree) -#define XFS_ICHANGE_LOG_RES (mp) ((mp)->m_reservations.tr_ichange) -#define XFS_GROWDATA_LOG_RES (mp) ((mp)->m_reservations.tr_growdata) -#define XFS_GROWRTALLOC_LOG_RES (mp) ((mp)->m_reservations.tr_growrtalloc) -#define XFS_GROWRTZERO_LOG_RES (mp) ((mp)->m_reservations.tr_growrtzero) -#define XFS_GROWRTFREE_LOG_RES (mp) ((mp)->m_reservations.tr_growrtfree) -#define XFS_SWRITE_LOG_RES (mp) ((mp)->m_reservations.tr_swrite) +#define XFS_WRITE_LOG_RES (mp) ((mp)->m_reservations.tr_write.tr_res) +#define XFS_ITRUNCATE_LOG_RES (mp) ((mp)->m_reservations.tr_itruncate.tr_res) +#define XFS_RENAME_LOG_RES (mp) ((mp)->m_reservations.tr_rename.tr_res) +#define XFS_LINK_LOG_RES (mp) ((mp)->m_reservations.tr_link.tr_res) +#define XFS_REMOVE_LOG_RES (mp) ((mp)->m_reservations.tr_remove.tr_res)
fs/xfs/xfs_trans.h struct xfs_log_item_desc { · line 249
Quoted 10 lines
fs/xfs/xfs_trans.h +10
-0+#define XFS_SYMLINK_LOG_RES (mp) ((mp)->m_reservations.tr_symlink.tr_res) +#define XFS_CREATE_LOG_RES (mp) ((mp)->m_reservations.tr_create.tr_res) +#define XFS_MKDIR_LOG_RES (mp) ((mp)->m_reservations.tr_mkdir.tr_res) +#define XFS_IFREE_LOG_RES (mp) ((mp)->m_reservations.tr_ifree.tr_res) +#define XFS_ICHANGE_LOG_RES (mp) ((mp)->m_reservations.tr_ichange.tr_res) +#define XFS_GROWDATA_LOG_RES (mp) ((mp)->m_reservations.tr_growdata.tr_res) +#define XFS_GROWRTALLOC_LOG_RES (mp) ((mp)->m_reservations.tr_growrtalloc.tr_res) +#define XFS_GROWRTZERO_LOG_RES (mp) ((mp)->m_reservations.tr_growrtzero.tr_res) +#define XFS_GROWRTFREE_LOG_RES (mp) ((mp)->m_reservations.tr_growrtfree.tr_res) +#define XFS_SWRITE_LOG_RES (mp) ((mp)->m_reservations.tr_swrite.tr_res)
If we do the "pass xfs_trans_res to xfs_trans_reserve(), all these macros could go away....
Cheers,
Dave.
Signature · 7 lines
Dave Chinner [email protected] _______________________________________________ xfs mailing list [email protected] http://oss.sgi.com/mailman/listinfo/xfs
Quoted 16 lines · Dave Chinner
Quoted 4 lines · Jeff Liu
Hello,
About two weeks ago, Dave has found an issue by running xfstests/297.
http://oss.sgi.com/archives/xfs/2013-03/msg00273.html...
Quoted 9 lines
--- fs/xfs/xfs_log.c | 130 ++++++++++++++++++++++++++++++++++------- fs/xfs/xfs_log.h | 3 + fs/xfs/xfs_mount.c | 1 + fs/xfs/xfs_mount.h | 61 +++++++++++--------- fs/xfs/xfs_trans.c | 163 +++++++++++++++++++++++++++++++++++++++++++--------- fs/xfs/xfs_trans.h | 61 ++++++++++---------- 6 files changed, 314 insertions(+), 105 deletions(-)
Quoted 1 line
Hmmmm. That's a lot more change that I expected.....
I don't want to make more changes as well, but....
Quoted 15 lines
Quoted 15 lines
fs/xfs/xfs_log.c +7
-0@@ -598,6 +598,64 @@ xfs_log_release_iclog( }/* +* Check if the specified log space is sufficient for a file system +* with a given log strip unit. +*/ +STATIC int +xfs_mount_validate_log_size ( +struct xfs_mount *mp) +{
Quoted 8 lines
fs/xfs/xfs_log.c xfs_log_release_iclog( · line 612
Quoted 6 lines
fs/xfs/xfs_log.c +6
-0+struct xlog *log = mp->m_log; +struct xfs_trans_res tres; +int unit_bytes; +int min_lblks, lsu =0 ; + +xfs_max_trans_res_by_mount (mp, &tres);Ok, that's been copied from mkfs, right?
Right.
Quoted 4 lines
What I'd suggest we need to do here is separate out all this reservation/validation code into it's own file so we can easily share it with libxfs in userspace.
I think so.
Quoted 44 lines
fs/xfs/xfs_log.c xfs_log_release_iclog( · line 621
Quoted 10 lines
fs/xfs/xfs_log.c +9
-0+ +/* +* Figure out the total space needed for the maximum transaction +* log space reservation by adding some extra spaces which should +* be taken into account. +*/ + unit_bytes =xlog_ticket_unit_res (log, &tres); +if (tres.tr_cnt >1 ) + unit_bytes = unit_bytes * tres.tr_cnt;Hmmmm - it's a bit different to userspace - there's no count in the userspace code. But yes, we do need to take into account the permanent log reservations...
Quoted 13 lines
fs/xfs/xfs_log.c +13
-0+ + min_lblks =BTOBB (unit_bytes); +/* +* FIXME: why we should add another 2 log strip units if it +* is specified? As per my tryout, creat a dozens dirs/files +* on a partition without another 2 log strip units will +* cause DEAD LOOP, it's fine if taken this into account. +* +* As per Dave's comments: +* I'm thinking a minimum of 4*lsu - 2*lsu for the existing +* CIL context, and another 2*lsu for any queued ticket +* waiting for space to come available. +*/ fs/xfs/xfs_log.c xfs_log_release_iclog( · line 644
Quoted 10 lines
fs/xfs/xfs_log.c +10
-0+if (xfs_sb_version_haslogv2 (&log->l_mp->m_sb) && + log->l_mp->m_sb.sb_logsunit >1 ) + lsu =BTOBB (log->l_mp->m_sb.sb_logsunit); + +/* +* The fundamental limit is that no single transaction can be +* larger than half the size of the log space, take another +* two log strip unit account as well. +*/ +if ((log->l_logBBsize >>1 ) < (min_lblks + lsu)) {A transaction requires 2 LSU for the reservation because there are two log writes that can require padding - the transaction data and the commit record are written separately and both can require padding to the LSU.
Quoted 9 lines
And as per my comments above, we can have an active CIL reservation (holding 2*LSU), but the CIL is not over a push threshold. If we don't have space for at one new transaction, which includes *another* 2*LSU in the reservation, that's when we have problems. So, the log size needs to be able to contain two maximally sized and padded transactions, which is (2 * (2 * LSU + maxtrres)). Hence if you are comparing this against half the log size (i.e. maximum transaction size), it needs to be (2 * (2 * LSU + maxtrres)) / 2. i.e. (minlblks + 2 * lsu)
Thanks for pointing me out, this question puzzled me for a long time.
Quoted 15 lines
fs/xfs/xfs_log.c xfs_log_release_iclog( · line 651
Quoted 9 lines
fs/xfs/xfs_log.c +7
-0+xfs_warn (mp, +"log space of %d blocks too small, minimum request %d" , + log->l_logBBsize, +roundup ((int )min_lblks <<1 , (int )lsu) + +2 * lsu); + +return XFS_ERROR (EINVAL );But, we can't just reject the mount if this fails. This would mean that people would have to downgrade their kernel just to remedy the situation as there is no way to grow the log (short of black magic surgery with
xfs_db).
Quoted 3 lines
So this should just remain a warning message, though I'd make it of "
xfs_crit" level (i.e. critical) so people notice it as well as making the message a little more informative.
Yes, fair enough.
Quoted 14 lines
fs/xfs/xfs_log.c xfs_log_ticket_get( · line 3449
Quoted 12 lines
fs/xfs/xfs_log.c +7
-1@@ -3377,24 +3441,23 @@ xfs_log_ticket_get( }/* -* Allocate and initialise a new log ticket. +* Figure out how many bytes would be reserved totally per ticket. +* Especially, take log strip unit into account if it is specified. +* +* FIXME: this is totally copied from xlog_ticket_alloc(), it's better +* to introduce a new helper to calculate the extra space reservation +* that can be shared with xlog_ticket_alloc() if the current though +* is reasonable. That FIXME looks like you've already fixed it ;)
... forgot removing it. :(
Quoted 34 lines
Quoted 34 lines
fs/xfs/xfs_log.c +9
-17*/ -struct xlog_ticket * -xlog_ticket_alloc ( -struct xlog *log, -int unit_bytes, -int cnt, -char client, -bool permanent, -xfs_km_flags_t alloc_flags) +int +xlog_ticket_unit_res ( +struct xlog *log, +struct xfs_trans_res *tres) { -struct xlog_ticket *tic; -uint num_headers; -int iclog_space; - - tic =kmem_zone_zalloc (xfs_log_ticket_zone, alloc_flags); -if (!tic) -return NULL ; +uint unit_bytes = tres->tr_res; +int total_bytes = unit_bytes; +int iclog_space; +uint num_headers;/* * Permanent reservations have up to 'cnt'-1 active log operations @@ -3459,8 +3522,8 @@ xlog_ticket_alloc(/* add extra header reservations if we overrun */ while (!num_headers || -howmany (unit_bytes, iclog_space) > num_headers) { - unit_bytes +=sizeof (xlog_op_header_t); +howmany (total_bytes, iclog_space) > num_headers) {
Quoted 8 lines
fs/xfs/xfs_log.c xlog_ticket_alloc( · line 3528
Quoted 4 lines
fs/xfs/xfs_log.c +1
-0+ total_bytes +=sizeof (xlog_op_header_t); num_headers++; } unit_bytes += log->l_iclog_hsize * num_headers;What is the reason for using
total_byteshere? We've got to take into account the size of thexlog_op_header_theaders in the ticket reservation, so adding them tounit_bytesis correct AFAICT....
You are right, this is a typo when I refine this procedure out of xfs_ticket_alloc().
Quoted 8 lines
fs/xfs/xfs_log.c xlog_ticket_alloc( · line 3544
Quoted 6 lines
fs/xfs/xfs_log.c +1
-0@@ -3478,11 +3541,38 @@ xlog_ticket_alloc( unit_bytes +=2 *BBSIZE ; } + return unit_bytes; +}This patch hunk is broken.
This will be fixed.
Quoted 66 lines
Quoted 15 lines
fs/xfs/xfs_log.c +14
-0+ +/* +* Allocate and initialise a new log ticket. +*/ +struct xlog_ticket * +xlog_ticket_alloc ( +struct xlog *log, +int unit_bytes, +int cnt, +char client, +bool permanent, +xfs_km_flags_t alloc_flags) +{ +struct xlog_ticket *tic;fs/xfs/xfs_log.c xlog_ticket_alloc( · line 3568
Quoted 10 lines
fs/xfs/xfs_log.c +10
-0+struct xfs_trans_res tres; +int unit_res; + + tic =kmem_zone_zalloc (xfs_log_ticket_zone, alloc_flags); +if (!tic) +return NULL ; + + tres.tr_res = unit_bytes; + tres.tr_cnt = cnt; + unit_res =xlog_ticket_unit_res (log, &tres);Ok, I'm starting to see where this tres stuff is going. More on that later....
Quoted 29 lines
fs/xfs/xfs_log.c +3
-2+atomic_set (&tic->t_ref,1 ); tic->t_task = current;INIT_LIST_HEAD (&tic->t_queue); - tic->t_unit_res = unit_bytes; - tic->t_curr_res = unit_bytes; + tic->t_unit_res = unit_res; + tic->t_curr_res = unit_res; tic->t_cnt = cnt; tic->t_ocnt = cnt; tic->t_tid =random32 ();fs/xfs/xfs_log.h +2
-0@@ -119,11 +119,13 @@ typedef struct xfs_log_callback {#ifdef __KERNEL__/* Log manager interfaces */ struct xfs_mount ; +struct xlog ;struct xlog_in_core ;struct xlog_ticket ;struct xfs_log_item ;struct xfs_item_ops ;struct xfs_trans ; +struct xfs_trans_res ; void xfs_log_item_init(struct xfs_mount *mp, struct xfs_log_item *item,fs/xfs/xfs_log.h bool xfs_log_item_in_current_chkpt(struct xfs_log_item *lip); · line 189
Quoted 4 lines
fs/xfs/xfs_log.h +1
-0@@ -184,6 +186,7 @@ bool xfs_log_item_in_current_chkpt(struct xfs_log_item *lip);void xfs_log_work_queue (struct xfs_mount *mp);void xfs_log_worker (struct work_struct *work);void xfs_log_quiesce (struct xfs_mount *mp); +int xlog_ticket_unit_res (struct xlog *log,struct xfs_trans_res *tres);We generally name the log external functions as "
xfs_log_..." and pass a structxfs_mountaround with them. i.e.:
Quoted 6 lines
int
xfs_log_ticket_unit_res(struct xfs_mount *mp, struct xfs_trans_res *tres);As it is, I'm not sure what that means from the name of the function. It has nothing to do with log tickets, but it's calculating the unit reservation for the ticket. Perhaps a better name is something like
xfs_log_calc_unit_res()?
The suggested name is nice to me as we already have xfs_calc_buf_res(), therefore we can follow up this naming convention for such kind of routines.
Quoted 14 lines
fs/xfs/xfs_mount.c line 23
Quoted 12 lines
fs/xfs/xfs_log.h +0
-0#endif #endif /* __XFS_LOG_H__ */ fs/xfs/xfs_mount.c +1
-0@@ -20,6 +20,7 @@#include "xfs_types.h" #include "xfs_bit.h" #include "xfs_log.h" +#include "xfs_log_priv.h" stray include?
Yes.
Quoted 17 lines
Quoted 9 lines
fs/xfs/xfs_mount.c +0
-0#include "xfs_inum.h" #include "xfs_trans.h" #include "xfs_trans_priv.h" fs/xfs/xfs_mount.h +0
-0fs/xfs/xfs_mount.h line 23
Quoted 6 lines
fs/xfs/xfs_mount.h +3
-0@@ -18,35 +18,40 @@#ifndef __XFS_MOUNT_H__#define __XFS_MOUNT_H__ +typedef struct xfs_trans_res { +uint tr_res; +int tr_cnt; +} xfs_tr_t;No new typedefs, please.
Ok.
Quoted 12 lines
fs/xfs/xfs_trans.c line 46
Quoted 10 lines
fs/xfs/xfs_trans.c +1
-0@@ -43,6 +43,7 @@#include "xfs_inode_item.h" #include "xfs_log_priv.h" #include "xfs_buf_item.h" +#include "xfs_attr_leaf.h" Another stray include?
No, this is required for xfs_attr_leaf_newentsize().
Quoted 47 lines
Quoted 32 lines
fs/xfs/xfs_trans.c +0
-25#include "xfs_trace.h" kmem_zone_t *xfs_trans_zone;@@ -645,34 +646,140 @@ xfs_trans_init( {struct xfs_trans_reservations *resp = &mp->m_reservations; - resp->tr_write =xfs_calc_write_reservation (mp); - resp->tr_itruncate =xfs_calc_itruncate_reservation (mp); - resp->tr_rename =xfs_calc_rename_reservation (mp); - resp->tr_link =xfs_calc_link_reservation (mp); - resp->tr_remove =xfs_calc_remove_reservation (mp); - resp->tr_symlink =xfs_calc_symlink_reservation (mp); - resp->tr_create =xfs_calc_create_reservation (mp); - resp->tr_mkdir =xfs_calc_mkdir_reservation (mp); - resp->tr_ifree =xfs_calc_ifree_reservation (mp); - resp->tr_ichange =xfs_calc_ichange_reservation (mp); - resp->tr_growdata =xfs_calc_growdata_reservation (mp); - resp->tr_swrite =xfs_calc_swrite_reservation (mp); - resp->tr_writeid =xfs_calc_writeid_reservation (mp); - resp->tr_addafork =xfs_calc_addafork_reservation (mp); - resp->tr_attrinval =xfs_calc_attrinval_reservation (mp); - resp->tr_attrsetm =xfs_calc_attrsetm_reservation (mp); - resp->tr_attrsetrt =xfs_calc_attrsetrt_reservation (mp); - resp->tr_attrrm =xfs_calc_attrrm_reservation (mp); - resp->tr_clearagi =xfs_calc_clear_agi_bucket_reservation (mp); - resp->tr_growrtalloc =xfs_calc_growrtalloc_reservation (mp); - resp->tr_growrtzero =xfs_calc_growrtzero_reservation (mp); - resp->tr_growrtfree =xfs_calc_growrtfree_reservation (mp); - resp->tr_qm_sbchange =xfs_calc_qm_sbchange_reservation (mp); - resp->tr_qm_setqlim =xfs_calc_qm_setqlim_reservation (mp); - resp->tr_qm_dqalloc =xfs_calc_qm_dqalloc_reservation (mp); - resp->tr_qm_quotaoff =xfs_calc_qm_quotaoff_reservation (mp);fs/xfs/xfs_trans.c xfs_trans_init( · line 656
Quoted 10 lines
fs/xfs/xfs_trans.c +8
-2- resp->tr_qm_equotaoff =xfs_calc_qm_quotaoff_end_reservation (mp); - resp->tr_sb =xfs_calc_sb_reservation (mp); + resp->tr_write.tr_res =xfs_calc_write_reservation (mp); + resp->tr_write.tr_cnt =XFS_WRITE_LOG_COUNT ; + + resp->tr_itruncate.tr_res =xfs_calc_itruncate_reservation (mp); + resp->tr_itruncate.tr_cnt =XFS_ITRUNCATE_LOG_COUNT ; + + resp->tr_rename.tr_res =xfs_calc_rename_reservation (mp); + resp->tr_rename.tr_cnt =XFS_RENAME_LOG_COUNT ;.....
I like the idea, but I don't think you've carried it through far enough. :)
Quoted 6 lines
i.e. This patch leaves us with having multiple places where this information has to be maintained (here and the
xfs_trans_reserve()calls). What I think is the best way to approach this is to separate out this table change into a separate patch (i.e. without all the other code that uses it), and then change thexfs_trans_reserve()interface to take a structxfs_trans_res*.
Exactly.
Quoted 16 lines
At that point, we can then do:
xfs_trans_reserve(tp, &mp->m_reservations.tr_rename, blockres, rtblockres, flags);and now we can propagate the logspace/logcount through
xfs_log_reserve(),xlog_ticket_alloc()and so on via the reservation structure.That leaves us with a single place that we set up and maintain log space reservations, makes the transaction reservation calls cleaner (no more messy macros everywhere), and if we rename
mp->m_reservationstomp->m_resvthere's a whole lot less typing, too.
Quoted 3 lines
We could potentially also add the
XFS_TRANS_PERM_LOG_RESflag to the structxfs_trans_res, so mostxfs_trans_reserve()calls don't need to pass a flag in at all (even cleaner!).
Well, that'll be very nice! By this means we can make the transaction log res info well encapsulated and reduce the stack push by two arguments on each call.
Quoted 20 lines
Quoted 20 lines
fs/xfs/xfs_trans.c +19
-0+STATIC void +xfs_max_attrsetm_trans_res_adjust ( +struct xfs_mount *mp) +{ +int local; +int size; +int nblks; +int res; + +/* +* Determine space the maximal sized attribute will use, +* to calculate the largest reservatoin size needed. +*/ + size =xfs_attr_leaf_newentsize (MAXNAMELEN ,64 *1024 , + mp->m_sb.sb_blocksize, &local); +ASSERT (!local); + nblks =XFS_DAENTER_SPACE_RES (mp,XFS_ATTR_FORK ); + nblks +=XFS_B_TO_FSB (mp, size); + nblks +=XFS_NEXTENTADD_SPACE_RES (mp, size,XFS_ATTR_FORK );
Quoted 8 lines
fs/xfs/xfs_trans.c xfs_trans_init( · line 754
Quoted 2 lines
fs/xfs/xfs_trans.c +2
-0+ res =XFS_ATTRSETM_LOG_RES (mp) +XFS_ATTRSETRT_LOG_RES (mp) * nblks; + mp->m_reservations.tr_attrsetm.tr_res = res;That's copied from mkfs, right? I need to look a bit closer, but I don't think this is correct - large attributes end up out of line and not logged, while this assumes that the full 64k of the remote attribute is logged. Over estimating is fine, though, for the moment.
Right. So for me, the next step is to verify whether the default attr transaction would affect the minimum log space calculation or not.
Quoted 22 lines
Quoted 9 lines
fs/xfs/xfs_trans.c +8
-0+} + +/* +* Figure out the total log space a transaction would required in terms +* of the pre-calculated values which are done at mount time, then find +* out and return the maximum reservation among them. +*/ +void fs/xfs/xfs_trans.c xfs_trans_init( · line 772
Quoted 10 lines
fs/xfs/xfs_trans.c +10
-0+xfs_max_trans_res_by_mount ( + struct xfs_mount *mp, + struct xfs_trans_res *mres) +{ + struct xfs_trans_reservations *resp = &mp->m_reservations; +struct xfs_trans_res *p, *tres =NULL ; +int res; + +for (res =0 , p = (struct xfs_trans_res *)resp; + p < (struct xfs_trans_res *)(resp +1 ); p++) {I don't really like the pointer arithmetic here. Something like
Quoted 5 lines
res = 0; for (i = 0; i < ARRAY_SIZE(mp->m_reservations); i++) { p = &mp->m_reservations[i];is a much neater way of iterating an array....
This is basically copied from mkfs/maxtrres.c, but yes, the above is the best approach.
Quoted 17 lines
fs/xfs/xfs_trans.c xfs_trans_init( · line 783
Quoted 13 lines
fs/xfs/xfs_trans.c +10
-0+int tmp = p->tr_cnt >1 ? p->tr_res * p->tr_cnt : + p->tr_res; +if (res < tmp) { + res = tmp; + tres = p; + } + } + +ASSERT (tres !=NULL ); + *mres = *tres; }All these changes look to me like something we shoul dbe sharing with libxfs in userspace so that mkfs can re-use the code without modifications....
So maybe it's better to fix the method of iterating array in user space as well.
Quoted 39 lines
Quoted 39 lines
fs/xfs/xfs_trans.c +0
-0/* fs/xfs/xfs_trans.h +11
-14@@ -19,6 +19,7 @@#define __XFS_TRANS_H__struct xfs_log_item ; +struct xfs_trans_res ;/* * This is the structure written in the log at the head of @@ -232,39 +233,39 @@ struct xfs_log_item_desc { XFS_DAENTER_BMAPS(mp, XFS_DATA_FORK) + 1) -#define XFS_WRITE_LOG_RES(mp) ((mp)->m_reservations.tr_write) -#define XFS_ITRUNCATE_LOG_RES (mp) ((mp)->m_reservations.tr_itruncate) -#define XFS_RENAME_LOG_RES (mp) ((mp)->m_reservations.tr_rename) -#define XFS_LINK_LOG_RES (mp) ((mp)->m_reservations.tr_link) -#define XFS_REMOVE_LOG_RES (mp) ((mp)->m_reservations.tr_remove) -#define XFS_SYMLINK_LOG_RES (mp) ((mp)->m_reservations.tr_symlink) -#define XFS_CREATE_LOG_RES (mp) ((mp)->m_reservations.tr_create) -#define XFS_MKDIR_LOG_RES (mp) ((mp)->m_reservations.tr_mkdir) -#define XFS_IFREE_LOG_RES (mp) ((mp)->m_reservations.tr_ifree) -#define XFS_ICHANGE_LOG_RES (mp) ((mp)->m_reservations.tr_ichange) -#define XFS_GROWDATA_LOG_RES (mp) ((mp)->m_reservations.tr_growdata) -#define XFS_GROWRTALLOC_LOG_RES (mp) ((mp)->m_reservations.tr_growrtalloc) -#define XFS_GROWRTZERO_LOG_RES (mp) ((mp)->m_reservations.tr_growrtzero) -#define XFS_GROWRTFREE_LOG_RES (mp) ((mp)->m_reservations.tr_growrtfree) -#define XFS_SWRITE_LOG_RES (mp) ((mp)->m_reservations.tr_swrite) +#define XFS_WRITE_LOG_RES (mp) ((mp)->m_reservations.tr_write.tr_res) +#define XFS_ITRUNCATE_LOG_RES (mp) ((mp)->m_reservations.tr_itruncate.tr_res) +#define XFS_RENAME_LOG_RES (mp) ((mp)->m_reservations.tr_rename.tr_res) +#define XFS_LINK_LOG_RES (mp) ((mp)->m_reservations.tr_link.tr_res) +#define XFS_REMOVE_LOG_RES (mp) ((mp)->m_reservations.tr_remove.tr_res) +#define XFS_SYMLINK_LOG_RES (mp) ((mp)->m_reservations.tr_symlink.tr_res) +#define XFS_CREATE_LOG_RES (mp) ((mp)->m_reservations.tr_create.tr_res) +#define XFS_MKDIR_LOG_RES (mp) ((mp)->m_reservations.tr_mkdir.tr_res) +#define XFS_IFREE_LOG_RES (mp) ((mp)->m_reservations.tr_ifree.tr_res) +#define XFS_ICHANGE_LOG_RES (mp) ((mp)->m_reservations.tr_ichange.tr_res)
Quoted 8 lines
fs/xfs/xfs_trans.h struct xfs_log_item_desc { · line 249
Quoted 5 lines
fs/xfs/xfs_trans.h +5
-0+#define XFS_GROWDATA_LOG_RES (mp) ((mp)->m_reservations.tr_growdata.tr_res) +#define XFS_GROWRTALLOC_LOG_RES (mp) ((mp)->m_reservations.tr_growrtalloc.tr_res) +#define XFS_GROWRTZERO_LOG_RES (mp) ((mp)->m_reservations.tr_growrtzero.tr_res) +#define XFS_GROWRTFREE_LOG_RES (mp) ((mp)->m_reservations.tr_growrtfree.tr_res) +#define XFS_SWRITE_LOG_RES (mp) ((mp)->m_reservations.tr_swrite.tr_res)If we do the "pass
xfs_trans_restoxfs_trans_reserve(), all these macros could go away....
Absoultely. :)
Thanks,
-Jeff
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs