Skip to contentSkip to conversation

hfsplus: implement functionality for HFSPLUS_JOURNAL_NEED_INIT flag

PATCH Superseded Vyacheslav Dubeyko 2 participants 4 messages Feb 12, 2014

Subsystems (2)

v3 of 4 Superseded Latest is v4 →

lore
Share & tools
lore

Vyacheslav Dubeyko

author maintainer
#0 lore raw

PATCH v3 4/15 hfsplus: implement functionality for HFSPLUS_JOURNAL_NEED_INIT flag

From: Vyacheslav Dubeyko <[email protected]>
Subject: [PATCH v3 04/15] hfsplus: implement functionality for HFSPLUS_JOURNAL_NEED_INIT flag

This patch implements functionality of creation of journal header, journal buffer and initialization of them.

CC Al Viro <[email protected]>
CC Christoph Hellwig <[email protected]>
CC Hin-Tak Leung <[email protected]>
---
 fs/hfsplus/journal.c |  178 +++++++++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 176 insertions(+), 2 deletions(-)
fs/hfsplus/journal.c +176 -2
index 89dea39..0a4bfc4 100644
@@ -8,6 +8,23 @@
 
 #include "hfsplus_fs.h"
 
+#define HFSPLUS_1KB_SHIFT		10
+#define HFSPLUS_1MB_SHIFT		20
+#define HFSPLUS_1GB_SHIFT		30
+#define HFSPLUS_1TB_SHIFT		40
+
+#define HFSPLUS_JOURNAL_MIN_SIZE	(512 * 1024)
+#define HFSPLUS_JOURNAL_DEFAULT_SIZE	(8 * (1 << HFSPLUS_1MB_SHIFT))
+#define HFSPLUS_JOURNAL_MAX_SIZE	(512 * (1 << HFSPLUS_1MB_SHIFT))
+
+#define HFSPLUS_VOLUME_GRAIN		100 /* 100 GB*/
+#define HFSPLUS_JSCALE_MAX		64
+
+#define HFSPLUS_VOLUME_MIN_THRESHOLD	(128 * (1 << HFSPLUS_1MB_SHIFT))
+#define HFSPLUS_DEFAULT_JHDR_SIZE	(4 * 1024)
+#define HFSPLUS_BLHDR_SIZE_MIN		(4 * 1024)
+#define HFSPLUS_BLHDR_SIZE_MAX		(16 * 1024)
+
 #define HFSPLUS_HAS_JOURNAL(sb) \
 	(HFSPLUS_SB(sb)->s_vhdr->attributes & \
 	 cpu_to_be32(HFSPLUS_VOL_JOURNALED))
@@ -25,11 +42,168 @@
 	((sector_t)(blk) << \
 	 (HFSPLUS_SB(sb)->alloc_blksz_shift - HFSPLUS_SECTOR_SHIFT))
 
+#define JHDR_SIZE(jh) \
+	(le32_to_cpu(((struct hfsplus_journal_header *)(jh))->jhdr_size))
+
+/*
+ * We want at least 8 megs of journal for each 100 gigs of
+ * disk space.  We cap the size at 512 megs (64x default), unless
+ * the allocation block size is larger, in which case we use one
+ * allocation block.
+ *
+ * Volumes that are 128 megs or less in size have such
+ * a small bitmap (one 4k-block) and inherhently such
+ * a small btree that we can get by with a much smaller
+ * journal. Even in a worst case scenario of a catalog
+ * filled with very long korean file names we should
+ * never touch more than 256k of meta-data for a single
+ * transaction.  therefore we'll make the journal 512k
+ * which is safe and doesn't waste much space.
+ */
+static u64 calc_journal_size(struct super_block *sb)
+{
+	u64 journal_size;
+	u64 jscale;
+	sector_t sect_count;
+	sector_t secs_in_1gb;
+	sector_t volume_grain;
+	u64 volume_size;
+
+	sect_count = HFSPLUS_SB(sb)->sect_count;
+	secs_in_1gb = ((sector_t)1 << HFSPLUS_1GB_SHIFT) >>
+			HFSPLUS_SECTOR_SHIFT;
+	volume_grain = secs_in_1gb * HFSPLUS_VOLUME_GRAIN;
+
+	jscale = div64_ul(sect_count, volume_grain);
+
+	if (jscale > HFSPLUS_JSCALE_MAX)
+		jscale = HFSPLUS_JSCALE_MAX;
+
+	journal_size = HFSPLUS_JOURNAL_DEFAULT_SIZE * (jscale + 1);
+
+	if (journal_size > HFSPLUS_JOURNAL_MAX_SIZE)
+		journal_size = HFSPLUS_JOURNAL_MAX_SIZE;
+
+	volume_size = sect_count * HFSPLUS_SECTOR_SIZE;
+
+	if (volume_size < HFSPLUS_VOLUME_MIN_THRESHOLD)
+		journal_size = HFSPLUS_JOURNAL_MIN_SIZE;
+
+	return journal_size;
+}
+
+/*
+ * Technical Note TN1150
+ *
+ * Checksums can be verified as part of a basic consistency check.
+ * To verify the checksum, temporarily set the checksum field to zero
+ * and then call the calc_checksum routine with the address and size of
+ * the header being checksummed. The function result should equal the
+ * original value of the checksum field.
+ *
+ * static int
+ * calc_checksum(unsigned char *ptr, int len)
+ * {
+ *    int i, cksum=0;
+ *
+ *    for(i=0; i < len; i++, ptr++) {
+ *        cksum = (cksum << 8) ^ (cksum + *ptr);
+ *    }
+ *
+ *    return (~cksum);
+ * }
+*/
+static u32 calc_internal_checksum(u32 seed, unsigned char *ptr, int len)
+{
+	int i;
+	u32 chksum = seed;
+
+	for (i = 0; i < len; i++, ptr++)
+		chksum = (chksum << 8) ^ (chksum + *ptr);
+
+	return chksum;
+}
+
+#define HFSPLUS_CHECKSUM(internal_checksum) \
+	(cpu_to_le32(~(u32)(internal_checksum)))
+
+#define HFSPLUS_CALC_CHECKSUM(ptr, len) \
+	HFSPLUS_CHECKSUM(calc_internal_checksum(0, ptr, len))
+
+/*
+ * hfsplus_create_journal - create journal
+ *
+ * @sb: superblock
+ *
+ * Create journal header, journal buffer and initialize them.
+ * It is assumed that presence of journal is already verified.
+ */
 static int hfsplus_create_journal(struct super_block *sb,
 				  struct hfsplus_journal *jnl)
 {
-	/* TODO: implement */
-	return -EINVAL;
+	u64 offset = be64_to_cpu(jnl->jib->offset);
+	int blksz_bits = HFSPLUS_SB(sb)->alloc_blksz_shift;
+	u32 blksz = HFSPLUS_SB(sb)->alloc_blksz;
+	u64 journal_size = be64_to_cpu(jnl->jib->size);
+	u32 journal_size_rem = 0;
+	u64 calculated_journal_size;
+
+	hfs_dbg(JOURNAL, "create HFS+ journal: blocksize %u, jib_hdr->offset %llu\n",
+		blksz, offset);
+
+	if (((offset >> blksz_bits) << blksz_bits) != offset) {
+		pr_err("journal size is not aligned\n");
+		return -EIO;
+	}
+
+	if (journal_size < HFSPLUS_JOURNAL_MIN_SIZE ||
+	    journal_size > HFSPLUS_JOURNAL_MAX_SIZE) {
+		pr_err("journal size %llu looks bogus\n", journal_size);
+		return -EIO;
+	}
+
+	div_u64_rem(journal_size, blksz, &journal_size_rem);
+	if (journal_size_rem != 0) {
+		pr_err("journal size %llu is not an even multiple of block size %u\n",
+			journal_size, blksz);
+		return -EIO;
+	}
+
+	calculated_journal_size = calc_journal_size(sb);
+	if (journal_size != calculated_journal_size) {
+		pr_warn("journal size %llu is different from calculated journal size %llu\n",
+			journal_size, calculated_journal_size);
+	}
+
+	jnl->jh->magic = cpu_to_le32(HFSPLUS_JOURNAL_HEADER_MAGIC);
+	jnl->jh->endian = cpu_to_le32(HFSPLUS_JOURNAL_HEADER_ENDIAN);
+
+	/*
+	 * The jhdr_size is a 512 byte for volumes are less than 1TB.
+	 * Otherwise, jhdr_size is 4096 bytes for 1+TB volumes.
+	 */
+	if (HFSPLUS_SB(sb)->sect_count >
+	    (((u64)1 << HFSPLUS_1TB_SHIFT) >> HFSPLUS_SECTOR_SHIFT))
+		jnl->jh->jhdr_size = cpu_to_le32(HFSPLUS_DEFAULT_JHDR_SIZE);
+	else
+		jnl->jh->jhdr_size = cpu_to_le32(HFSPLUS_SECTOR_SIZE);
+
+	if (blksz < HFSPLUS_BLHDR_SIZE_MIN)
+		jnl->jh->blhdr_size = cpu_to_le32(HFSPLUS_BLHDR_SIZE_MIN);
+	else if (blksz > HFSPLUS_BLHDR_SIZE_MAX)
+		jnl->jh->blhdr_size = cpu_to_le32(HFSPLUS_BLHDR_SIZE_MAX);
+	else
+		jnl->jh->blhdr_size = cpu_to_le32(sb->s_blocksize);
+
+	jnl->jh->start = cpu_to_le64(JHDR_SIZE(jnl->jh));
+	jnl->jh->end = cpu_to_le64(JHDR_SIZE(jnl->jh));
+	jnl->jh->size = cpu_to_le64(journal_size);
+
+	jnl->jh->checksum = 0;
+	jnl->jh->checksum = HFSPLUS_CALC_CHECKSUM((unsigned char *)(jnl->jh),
+						  sizeof(*(jnl->jh)));
+
+	return 0;
 }
 
 /*
Signature · 1 line
1.7.9.5
#1 lore raw
Quoted 5 lines · Vyacheslav Dubeyko

From: Vyacheslav Dubeyko <[email protected]>
Subject: [PATCH v3 04/15] hfsplus: implement functionality for HFSPLUS_JOURNAL_NEED_INIT flag

This patch implements functionality of creation of journal header, journal buffer and initialization of them.

Why initialize journal if you are not going to journal changes? It is probably better and safer to leave the "need initialize" flag. An implementation that really does journaling will initialize it on mount.

Quoted 167 lines
CC Al Viro <[email protected]>
CC Christoph Hellwig <[email protected]>
CC Hin-Tak Leung <[email protected]>
---
 fs/hfsplus/journal.c |  178 +++++++++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 176 insertions(+), 2 deletions(-)
fs/hfsplus/journal.c +139 -2
index 89dea39..0a4bfc4 100644
@@ -8,6 +8,23 @@
 
 #include "hfsplus_fs.h"
 
+#define HFSPLUS_1KB_SHIFT              10
+#define HFSPLUS_1MB_SHIFT              20
+#define HFSPLUS_1GB_SHIFT              30
+#define HFSPLUS_1TB_SHIFT              40
+
+#define HFSPLUS_JOURNAL_MIN_SIZE       (512 * 1024)
+#define HFSPLUS_JOURNAL_DEFAULT_SIZE   (8 * (1 << HFSPLUS_1MB_SHIFT))
+#define HFSPLUS_JOURNAL_MAX_SIZE       (512 * (1 << HFSPLUS_1MB_SHIFT))
+
+#define HFSPLUS_VOLUME_GRAIN           100 /* 100 GB*/
+#define HFSPLUS_JSCALE_MAX             64
+
+#define HFSPLUS_VOLUME_MIN_THRESHOLD   (128 * (1 << HFSPLUS_1MB_SHIFT))
+#define HFSPLUS_DEFAULT_JHDR_SIZE      (4 * 1024)
+#define HFSPLUS_BLHDR_SIZE_MIN         (4 * 1024)
+#define HFSPLUS_BLHDR_SIZE_MAX         (16 * 1024)
+
 #define HFSPLUS_HAS_JOURNAL(sb) \
        (HFSPLUS_SB(sb)->s_vhdr->attributes & \
         cpu_to_be32(HFSPLUS_VOL_JOURNALED))
@@ -25,11 +42,168 @@
        ((sector_t)(blk) << \
         (HFSPLUS_SB(sb)->alloc_blksz_shift - HFSPLUS_SECTOR_SHIFT))
 
+#define JHDR_SIZE(jh) \
+       (le32_to_cpu(((struct hfsplus_journal_header *)(jh))->jhdr_size))
+
+/*
+ * We want at least 8 megs of journal for each 100 gigs of
+ * disk space.  We cap the size at 512 megs (64x default), unless
+ * the allocation block size is larger, in which case we use one
+ * allocation block.
+ *
+ * Volumes that are 128 megs or less in size have such
+ * a small bitmap (one 4k-block) and inherhently such
+ * a small btree that we can get by with a much smaller
+ * journal. Even in a worst case scenario of a catalog
+ * filled with very long korean file names we should
+ * never touch more than 256k of meta-data for a single
+ * transaction.  therefore we'll make the journal 512k
+ * which is safe and doesn't waste much space.
+ */
+static u64 calc_journal_size(struct super_block *sb)
+{
+       u64 journal_size;
+       u64 jscale;
+       sector_t sect_count;
+       sector_t secs_in_1gb;
+       sector_t volume_grain;
+       u64 volume_size;
+
+       sect_count = HFSPLUS_SB(sb)->sect_count;
+       secs_in_1gb = ((sector_t)1 << HFSPLUS_1GB_SHIFT) >>
+                       HFSPLUS_SECTOR_SHIFT;
+       volume_grain = secs_in_1gb * HFSPLUS_VOLUME_GRAIN;
+
+       jscale = div64_ul(sect_count, volume_grain);
+
+       if (jscale > HFSPLUS_JSCALE_MAX)
+               jscale = HFSPLUS_JSCALE_MAX;
+
+       journal_size = HFSPLUS_JOURNAL_DEFAULT_SIZE * (jscale + 1);
+
+       if (journal_size > HFSPLUS_JOURNAL_MAX_SIZE)
+               journal_size = HFSPLUS_JOURNAL_MAX_SIZE;
+
+       volume_size = sect_count * HFSPLUS_SECTOR_SIZE;
+
+       if (volume_size < HFSPLUS_VOLUME_MIN_THRESHOLD)
+               journal_size = HFSPLUS_JOURNAL_MIN_SIZE;
+
+       return journal_size;
+}
+
+/*
+ * Technical Note TN1150
+ *
+ * Checksums can be verified as part of a basic consistency check.
+ * To verify the checksum, temporarily set the checksum field to zero
+ * and then call the calc_checksum routine with the address and size of
+ * the header being checksummed. The function result should equal the
+ * original value of the checksum field.
+ *
+ * static int
+ * calc_checksum(unsigned char *ptr, int len)
+ * {
+ *    int i, cksum=0;
+ *
+ *    for(i=0; i < len; i++, ptr++) {
+ *        cksum = (cksum << 8) ^ (cksum + *ptr);
+ *    }
+ *
+ *    return (~cksum);
+ * }
+*/
+static u32 calc_internal_checksum(u32 seed, unsigned char *ptr, int len)
+{
+       int i;
+       u32 chksum = seed;
+
+       for (i = 0; i < len; i++, ptr++)
+               chksum = (chksum << 8) ^ (chksum + *ptr);
+
+       return chksum;
+}
+
+#define HFSPLUS_CHECKSUM(internal_checksum) \
+       (cpu_to_le32(~(u32)(internal_checksum)))
+
+#define HFSPLUS_CALC_CHECKSUM(ptr, len) \
+       HFSPLUS_CHECKSUM(calc_internal_checksum(0, ptr, len))
+
+/*
+ * hfsplus_create_journal - create journal
+ *
+ * @sb: superblock
+ *
+ * Create journal header, journal buffer and initialize them.
+ * It is assumed that presence of journal is already verified.
+ */
 static int hfsplus_create_journal(struct super_block *sb,
                                  struct hfsplus_journal *jnl)
 {
-       /* TODO: implement */
-       return -EINVAL;
+       u64 offset = be64_to_cpu(jnl->jib->offset);
+       int blksz_bits = HFSPLUS_SB(sb)->alloc_blksz_shift;
+       u32 blksz = HFSPLUS_SB(sb)->alloc_blksz;
+       u64 journal_size = be64_to_cpu(jnl->jib->size);
+       u32 journal_size_rem = 0;
+       u64 calculated_journal_size;
+
+       hfs_dbg(JOURNAL, "create HFS+ journal: blocksize %u, jib_hdr->offset %llu\n",
+               blksz, offset);
+
+       if (((offset >> blksz_bits) << blksz_bits) != offset) {
+               pr_err("journal size is not aligned\n");
+               return -EIO;
+       }
+
+       if (journal_size < HFSPLUS_JOURNAL_MIN_SIZE ||
+           journal_size > HFSPLUS_JOURNAL_MAX_SIZE) {
+               pr_err("journal size %llu looks bogus\n", journal_size);
+               return -EIO;
+       }
+
+       div_u64_rem(journal_size, blksz, &journal_size_rem);
+       if (journal_size_rem != 0) {
+               pr_err("journal size %llu is not an even multiple of block size %u\n",
+                       journal_size, blksz);
+               return -EIO;

fs/hfsplus/journal.c line 179

Quoted 10 lines
fs/hfsplus/journal.c +10 -0
+       }
+
+       calculated_journal_size = calc_journal_size(sb);
+       if (journal_size != calculated_journal_size) {
+               pr_warn("journal size %llu is different from calculated journal size %llu\n",
+                       journal_size, calculated_journal_size);
+       }
+
+       jnl->jh->magic = cpu_to_le32(HFSPLUS_JOURNAL_HEADER_MAGIC);
+       jnl->jh->endian = cpu_to_le32(HFSPLUS_JOURNAL_HEADER_ENDIAN);

It is not very important, but using native endianness (so that the journal will be either LE or BE) leads to shorter and more effective jounaling code (which does not exist at the moment though).

Quoted 42 lines
fs/hfsplus/journal.c +27 -1
+
+       /*
+        * The jhdr_size is a 512 byte for volumes are less than 1TB.
+        * Otherwise, jhdr_size is 4096 bytes for 1+TB volumes.
+        */
+       if (HFSPLUS_SB(sb)->sect_count >
+           (((u64)1 << HFSPLUS_1TB_SHIFT) >> HFSPLUS_SECTOR_SHIFT))
+               jnl->jh->jhdr_size = cpu_to_le32(HFSPLUS_DEFAULT_JHDR_SIZE);
+       else
+               jnl->jh->jhdr_size = cpu_to_le32(HFSPLUS_SECTOR_SIZE);
+
+       if (blksz < HFSPLUS_BLHDR_SIZE_MIN)
+               jnl->jh->blhdr_size = cpu_to_le32(HFSPLUS_BLHDR_SIZE_MIN);
+       else if (blksz > HFSPLUS_BLHDR_SIZE_MAX)
+               jnl->jh->blhdr_size = cpu_to_le32(HFSPLUS_BLHDR_SIZE_MAX);
+       else
+               jnl->jh->blhdr_size = cpu_to_le32(sb->s_blocksize);
+
+       jnl->jh->start = cpu_to_le64(JHDR_SIZE(jnl->jh));
+       jnl->jh->end = cpu_to_le64(JHDR_SIZE(jnl->jh));
+       jnl->jh->size = cpu_to_le64(journal_size);
+
+       jnl->jh->checksum = 0;
+       jnl->jh->checksum = HFSPLUS_CALC_CHECKSUM((unsigned char *)(jnl->jh),
+                                                 sizeof(*(jnl->jh)));
+
+       return 0;
 }
 
 /*
--

1.7.9.5

--
To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

author maintainer
#2 lore raw
Quoted 4 lines · Sergei Antonov
Quoted 3 lines · Vyacheslav Dubeyko

From: Vyacheslav Dubeyko <[email protected]>
Subject: [PATCH v3 04/15] hfsplus: implement functionality for HFSPLUS_JOURNAL_NEED_INIT flag

Quoted 8 lines
Quoted 2 lines

This patch implements functionality of creation of journal header, journal buffer and initialization of them.

Why initialize journal if you are not going to journal changes? It is probably better and safer to leave the "need initialize" flag. An implementation that really does journaling will initialize it on mount.

Technical Note TN1150
"kJIJournalNeedInitMask
This bit is set to indicate that the journal header is invalid and needs to be initialized. This bit is typically set when the journal is first created, and the space has been allocated; the first mount of the journaled volume typically initializes the journal header and clears this bit. When this bit is set, there are no valid transactions in the journal."

Quoted 9 lines

fs/hfsplus/journal.c line 179

Quoted 4 lines
fs/hfsplus/journal.c +3 -0
+
+       jnl->jh->magic = cpu_to_le32(HFSPLUS_JOURNAL_HEADER_MAGIC);
+       jnl->jh->endian = cpu_to_le32(HFSPLUS_JOURNAL_HEADER_ENDIAN);

It is not very important, but using native endianness (so that the journal will be either LE or BE) leads to shorter and more effective jounaling code (which does not exist at the moment though).

Could you describe in more details your suggestion?

Thanks,
Vyacheslav Dubeyko.

#3 lore raw
Quoted 16 lines · Vyacheslav Dubeyko
Quoted 4 lines · Sergei Antonov
Quoted 3 lines · Vyacheslav Dubeyko

From: Vyacheslav Dubeyko <[email protected]>
Subject: [PATCH v3 04/15] hfsplus: implement functionality for HFSPLUS_JOURNAL_NEED_INIT flag

Quoted 8 lines
Quoted 2 lines

This patch implements functionality of creation of journal header, journal buffer and initialization of them.

Why initialize journal if you are not going to journal changes? It is probably better and safer to leave the "need initialize" flag. An implementation that really does journaling will initialize it on mount.

Quoted 3 lines

Technical Note TN1150
"kJIJournalNeedInitMask
This bit is set to indicate that the journal header is invalid and needs to be initialized. This bit is typically set when the journal is first created, and the space has been allocated; the first mount of the journaled volume typically initializes the journal header and clears this bit. When this bit is set, there are no valid transactions in the journal."

Which does not mean initialization is necessary. It is, of course, necessary for drivers that write to journal because they need the journal to write transactions to. But in this driver you initialize it and ... what? Nothing. It will not be used.

Quoted 6 lines
Quoted 6 lines
Quoted 5 lines
fs/hfsplus/journal.c +3 -0
+
+       jnl->jh->magic = cpu_to_le32(HFSPLUS_JOURNAL_HEADER_MAGIC);
+       jnl->jh->endian = cpu_to_le32(HFSPLUS_JOURNAL_HEADER_ENDIAN);
Quoted 6 lines
Quoted 4 lines

It is not very important, but using native endianness (so that the journal will be either LE or BE) leads to shorter and more effective jounaling code (which does not exist at the moment though).

Could you describe in more details your suggestion?

jnl->jh->magic = HFSPLUS_JOURNAL_HEADER_MAGIC;
jnl->jh->endian = HFSPLUS_JOURNAL_HEADER_ENDIAN;

or do not initialze at all.