Skip to contentSkip to conversation

xfsprogs 2.x vs 3.x logsize changed

DISCUSSION Łukasz Oleś 4 participants 5 messages Nov 17, 2010 – Nov 19, 2010 1 message not in the archive

lore
Share & tools
lore
#0 lore raw

Hi,

I'm upgrading xfsprogs from 2.10.1 to the latest 3.1.4 version. I noticed that when I'm creating large lvm volume (2T) the log size is almost 1G in the old version it was 128M.
I know I can manipulate this value with -lsize option, but I'm wondering why this difference is so huge?

On this volume I have one sparse file which is exported by iSCSI Target. I have script which calculates for me "seek" value for dd command and now it returns me wrong values.
Can I stay with the old log size or maybe there are some good reasons to use new values?

Regards,
Łukasz Oleś

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs

#1 lore raw
Quoted 2 lines · On Mittwoch, 17. November 2010 Łukasz Oleś wrote:

Can I stay with the old log size or maybe there are some good reasons to use new values?

I believe the new defaults are there because it results in faster metadata operations when there are lots of them. I'd guess if you only ever have one writer it wouldn't matter much, but in case you have several parallel writers it could make a difference. But I'm not a dev, maybe they can confirm or clarify.

Signature · 11 lines
mit freundlichen Grüssen,
Michael Monnerie, Ing. BSc

it-management Internet Services: Protéger
http://proteger.at [gesprochen: Prot-e-schee]
Tel: +43 660 / 415 6531

// ****** Radiointerview zum Thema Spam ******
// http://www.it-podcast.at/archiv.html#podcast-100716
// 
// Haus zu verkaufen: http://zmi.at/langegg/

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs

A message that is not in the archive (sent off-list or not yet received): [email protected]

#3 lore raw
Quoted 61 lines · On Wednesday 17 of November 2010 18:38:29 Eric Sandeen wrote:
Quoted 8 lines · Łukasz Oleś

Hi,

I'm upgrading xfsprogs from 2.10.1 to the latest 3.1.4 version. I noticed that when I'm creating large lvm volume (2T) the log size is almost 1G in the old version it was 128M.
I know I can manipulate this value with -lsize option, but I'm wondering why this difference is so huge?

Quoted 8 lines

On this volume I have one sparse file which is exported by iSCSI Target. I have script which calculates for me "seek" value for dd command and now it returns me wrong values.
Can I stay with the old log size or maybe there are some good reasons to use new values?

Regards,
Łukasz Oleś

commit a6634fba3dec4a92f0a2c4e30c80b634c0576ad5

Author: David Chinner <[email protected]>
Date:   Sun May 10 21:31:22 2009 +0200

mkfs: allow to make larger logs

Increase the maximum log size supported by mkfs.

The log size can be increased easily in mkfs by changing a few defines and a couple of types to allow the log size to increase to just under 2GB. Theoretically, the log size can be made much, much larger than this (up to 2^32 sectors), but going beyond 2^31 *bytes* causes integer overflow issues in the kernel log code.

We use a maximum size of just under 2GB as exactly 2GB will cause overflow issues on the first transaction reservation.

Maximum log size is now reached at a fs size of ~4TB.

did:

(quoted diff) +4 -4
@@ -1737,10 +1737,10 @@ _("size %s specified for log subvolume is too
 large, maximum is %lld blocks\n"), logblocks = 0;
        } else if (loginternal && !logsize) {
                /*
-                * logblocks grows from min_logblocks to XFS_MAX_LOG_BLOCKS
-                * at 128GB
-                *
-                * 2048 = 128GB / MAX_LOG_BYTES
+                * With a 2GB max log size, default to maximum size
+                * at 4TB. This keeps the same ratio from the older
+                * max log size of 128M at 256GB fs size. IOWs,
+                * the ratio of fs size to log size is 2048:1.
                 */
                logblocks = (dblocks << blocklog) / 2048;
                logblocks = logblocks >> blocklog;

old MAX was:

Quoted 8 lines

-#define XFS_MAX_LOG_BYTES (128 * 1024 * 1024)

In your case ... I'm not sure a larger log is going to help, I think keeping it smaller is fine if you wish.

Dave may have a different opinion though :)

-Eric

Ok, so it's this commit http://git.kernel.org/?p=fs/xfs/xfsprogs-
dev.git;a=commitdiff;h=a6634fba3dec4a92f0a2c4e30c80b634c0576ad5

Would it be safe if I just revert this commit? It looks safe for me, but maybe other things depends on it and I will break something.

Regards,
Łukasz Oleś

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs

#4 lore raw
Quoted 5 lines · Łukasz Oleś

Ok, so it's this commit http://git.kernel.org/?p=fs/xfs/xfsprogs-
dev.git;a=commitdiff;h=a6634fba3dec4a92f0a2c4e30c80b634c0576ad5

Would it be safe if I just revert this commit? It looks safe for me, but maybe other things depends on it and I will break something.

I would just specify a smaller log on the commandline if that is what you prefer, rather than changing the code.

-Eric

Quoted 3 lines

Regards,
Łukasz Oleś

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs

#5 lore raw
In reply to Łukasz Oleś
Quoted 7 lines · Łukasz Oleś

Hi,

I'm upgrading xfsprogs from 2.10.1 to the latest 3.1.4 version. I noticed that when I'm creating large lvm volume (2T) the log size is almost 1G in the old version it was 128M.
I know I can manipulate this value with -lsize option, but I'm wondering why this difference is so huge?

Many workloads were demonstrated to have substantially better performance with larger logs, even on small filesystems. At 4TB, most people are using RAID of some kind, so larger logs are quite beneficial here.

Quoted 5 lines

On this volume I have one sparse file which is exported by iSCSI Target. I have script which calculates for me "seek" value for dd command and now it returns me wrong values.
Can I stay with the old log size or maybe there are some good reasons to use new values?

Staying with the old log size is just fine - it'll behave exactly the same as it does now. There are two main things hat make a larger log size attractive:

1. log size determines maximum transaction parallelism, so so smaller logs may limit operational concurrency. A 128MB log typically allows ~250 concurrent transactions on a 1TB, 4k block size filesystem.
2. larger logs allow the filesystem to soak up larger burst of metadata modifications without needing to write back dirty metadata.

The downside ot a larger log is that recovery can take longer after a crash.

Anyway, if you are having no problems at 128MB, then just use that...

Cheers,

Dave.

Signature · 7 lines
Dave Chinner
[email protected]

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs