Set nodatacow per file?

Hello,
is it possible to set nodatacow on a per-file basis? I couldn't find anything.
If not, wouldn't that be a great feature to get around the performance issues with VM and database storage? Of course cloning should still cause COW.

Thanks,
Ralf-Peter

anything.

e issues with VM and database storage? Of course cloning should still c= ause COW.

IIRC this is already a feature in btrfs. I didn't catch the whole talk, but Chris mentioned something like this at Scale 10x. I also remember seeing a patch for it a while back (I think it was from liu bo) that does this.

" in

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" = in
the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

You're right, and I've made the prog patches which is against linux-ulit(chattr/lsattr):

http://www.spinics.net/lists/linux-btrfs/msg09604.html
http://www.spinics.net/lists/linux-btrfs/msg09605.html

but they are not merged yet.

thanks,
liubo

Hello Ralf-Peter,

Actually it is possible. Check out David's response to my question from some time ago:
http://permalink.gmane.org/gmane.comp.file-systems.btrfs/14227

The nocow.c script he attached does just the thing you want. The script is really working. I needed nocow for different purpose but it did not occur to me to try it on VM image and see if the performance would improve. Sounds like a great idea. If you get around to try it, pls. post your impressions here.

best
~dima

Thanks for the link, this is indeed interesting.

I made a couple of small changes, i.e. I wanted a way to unset nocow and to check that changing flags really worked.

gcc -o /usr/local/bin/nocow nocow.c
ln -sf /usr/local/bin/nocow /usr/local/bin/cow

Perhaps the support for setting this flag should be added to the 'btrfs' utility.

With respect,
Roman

~~~~~~~~~~~~~~~~~~~~~~~~~~~
"Stallman had a printer,
with code he could not see.
So he began to tinker,
and set the software free."

#include <fcntl.h>
#include <stdlib.h>
#include <sys/ioctl.h>
#include <string.h>

#include <linux/types.h>
#include <stdio.h>
#include <errno.h>

#ifndef FS_IOC_SETFLAGS
#define FS_IOC_SETFLAGS _IOW('f', 2, long)
#warning defining SETFLAGS locally
#endif

#ifndef FS_IOC_GETFLAGS
#define FS_IOC_GETFLAGS _IOR('f', 1, long)
#warning defining GETFLAGS locally
#endif

#ifndef FS_NOCOW_FL
#define FS_NOCOW_FL 0x00800000 /* Do not cow file */
#endif

int main(int argc, char **argv)

{
        int fd;
        int r;

long flags;

  if (argc < 2) {
    printf("usage: %s file\n", argv[0]);
    exit(1);
  }

  fd = open(argv[1], O_RDONLY);
  if (fd == -1) {
    perror("open()");
    return 1;
  }
  printf("GETFLAGS ioctl 0x%x\n", FS_IOC_GETFLAGS);
  printf("SETFLAGS ioctl 0x%x\n", FS_IOC_SETFLAGS);

  r = ioctl(fd, FS_IOC_GETFLAGS, &flags);
  if (r == -1) {
    perror("ioctl(GETFLAGS)");
    return 1;
  } else {
    printf("file flags before: 0x%lx\n", flags);
  }

  if(strcmp(argv[0], "cow")==0) {
    printf("Remove NOCOW flag for %s\n", argv[1]);
    flags ^= FS_NOCOW_FL;
  } else {
    printf("Set NOCOW flag for %s\n", argv[1]);    
    flags |= FS_NOCOW_FL;
  }
  r = ioctl(fd, FS_IOC_SETFLAGS, &flags);
  printf("ioctl returned: %d\n", r);
  if (r == -1) {
    perror("ioctl()");
    return 1;
  }

  r = ioctl(fd, FS_IOC_GETFLAGS, &flags);
  if (r == -1) {
    perror("ioctl(GETFLAGS)");
    return 1;
  } else {
    printf("file flags after:  0x%lx\n", flags);
  }  

return 0;
}

Cool. Thanks Roma!
I really wanted the feature to 'unset' the nocow and check the current state of flags. Will check it out today. I also think that it should definitely be included into the userspace btrfs utilities. best
~dima

Hello Roman,
I don't seem to be able to 'unset' the NOCOW flag. Looking at the code I would guess that it is supposed to alternate between 'cow' and 'nocow' states, but the condition

    printf("Remove NOCOW flag for %s\n", argv[1]);

never shows for me. What should I do to make it working?
Maybe it would be nice to have a switch to just check if nocow is set on file without actually changing the flag.

thanks
~dima

If you place it in /usr/local/bin and also make a symlink "nocow -> cow" as described in the quoted part above, you can then just run:

# cow ./filename

and the program will instead unset the NOCOW flag.

Of course it remains to be a very basic program, I'm not sure if it's worth developing it further, or to add this to 'btrfs', as Liu Bo in the adjacent thread said that there are patches to chattr/lsattr for using the COW attribute.

With respect,
Roman

~~~~~~~~~~~~~~~~~~~~~~~~~~~
"Stallman had a printer,
with code he could not see.
So he began to tinker,
and set the software free."

Thanks Roma! I overlooked the symlink. Works!
~dima

Hi,

this was a quick aid, please see attached file for an updated tool to set the file flags, now added 'z' for NOCOMPRESS flag, and supports chattr syntax plus all of the standard file flags.

Setting and unsetting nocow is done like 'fileflags +C file' or -C for unseting. Without any + or - options it prints current state.

david

#include <errno.h>
#include <fcntl.h>
#include <inttypes.h>
#include <linux/fs.h>
#include <linux/types.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <sys/ioctl.h>

#ifndef FS_NOCOMP_FL
#define FS_NOCOMP_FL 0x00000400 /* Don't compress */
#endif

#ifndef FS_NOCOW_FL
#define FS_NOCOW_FL 0x00800000 /* Do not cow file */
#endif

struct flags_name {
	unsigned long	flag;
	const char	short_name;
	const char	*long_name;
};

static const struct flags_name flags[]={
	/* new */
	{ FS_NOCOW_FL, 'C', "NOCOW" },
	{ FS_NOCOMP_FL, 'z', "Not_Compressed" },
	/* current */
	{ FS_SECRM_FL, 's', "Secure_Deletion" },
	{ FS_UNRM_FL, 'u' , "Undelete" },
	{ FS_SYNC_FL, 'S', "Synchronous_Updates" },
	{ FS_DIRSYNC_FL, 'D', "Synchronous_Directory_Updates" },
	{ FS_IMMUTABLE_FL, 'i', "Immutable" },
	{ FS_APPEND_FL, 'a', "Append_Only" },
	{ FS_NODUMP_FL, 'd', "No_Dump" },
	{ FS_NOATIME_FL, 'A', "No_Atime" },
	{ FS_COMPR_FL, 'c', "Compression_Requested" },
	{ FS_COMPRBLK_FL, 'B', "Compressed_File" },
	{ FS_DIRTY_FL, 'Z', "Compressed_Dirty_File" },
	{ FS_ECOMPR_FL, 'E', "Compression_Error" },
	{ FS_JOURNAL_DATA_FL, 'j', "Journaled_Data" },
	{ FS_INDEX_FL, 'I', "Indexed_directory" },
	{ FS_NOTAIL_FL, 't', "No_Tailmerging" },
	{ FS_TOPDIR_FL, 'T', "Top_of_Directory_Hierarchies" },
	/* { EXT4_EXTENTS_FL, 'e', "Extents" }, */
	/* { EXT4_HUGE_FILE_FL, 'h', "Huge_file" }, */
};

static unsigned long to_set, to_unset;

unsigned long c2val(char c)

{
	int i;

	for(i=0;i<sizeof(flags)/sizeof(flags[0]);i++) {
		if(flags[i].short_name==c)
			return flags[i].flag;
	}

	printf("Warning: flag '%c' not found\n", c);
	return 0;
}
void list_flags(unsigned long fflags)
{
	int i;

	printf("Flags:\n");
	for(i=0;i<sizeof(flags)/sizeof(flags[0]);i++) {
		if(fflags & flags[i].flag) {
			printf(" %c %s\n",
				flags[i].short_name,
				flags[i].long_name);
		}
	}
}

void set_flag(const char* in)

{
	int i;

	for(i=0;i<strlen(in);i++) {
		to_set |= c2val(in[i]);
		printf(" set %c\n", in[i]);
	}
}

void unset_flag(const char* in)

{
	int i;

	for(i=0;i<strlen(in);i++) {
		to_unset |= c2val(in[i]);
		printf(" unset %c\n", in[i]);
	}
}

int main(int argc, char **argv)

{
        int ret;
	int optind;
	int modify=0;

	if (argc < 2) {
		printf("usage: %s [options] [--] file...\n", argv[0]);
		exit(1);
	}
	to_set = 0;
	to_unset = 0;
	for(optind=1;optind<argc;optind++) {
		char *o=argv[optind];
		if(o[0]=='-' && o[1]=='-') {
			optind++;
			break;
		} else if(o[0]=='-') {
			unset_flag(o+1);
			modify=1;
		} else if(o[0]=='+') {
			set_flag(o+1);
			modify=1;
		} else break;
	}

	for(;optind<argc;optind++) {
		unsigned long fflags;
		int fd = -1;

		if(fd!=-1) close(fd);
		fd = open(argv[optind], O_RDONLY);
		if (fd == -1) {
			perror("open()");
			continue;
		}
		ret = ioctl(fd, FS_IOC_GETFLAGS, &fflags);
		if (ret == -1) {
			perror("ioctl(GETFLAGS)");
			continue;
		}
		if(modify) {
			fflags |= to_set;
			fflags &= ~to_unset;
			ret = ioctl(fd, FS_IOC_SETFLAGS, &fflags);
			if (ret == -1) {
				perror("ioctl(SETFLAGS)");
				continue;
			}
		}
		printf("File: %s\n", argv[optind]);
		list_flags(fflags);
		putchar('\n');
	}

        return 0;
}

The fileflags utility works great!
Thanks!

Am 13.02.2012 15:10, schrieb David Sterba:

Thanks David. Perfect! This would be a great help. ~dima

Hello,

Please be aware that there seems to be a possible problem with using NOCOW flag on files situated on a filesystem mounted with compress-force(=lzo, in my case).

Since experimenting with NOCOW, I started regularly hitting this BUG at extent-tree.c:5813

  5813                 BUG_ON(!(flags & BTRFS_BLOCK_FLAG_FULL_BACKREF));

I was unable to make netconsole work over a bridged interface, so can only post screenshots of this OOPS:
http://romanrm.ru/pics/2012/2012-02-16-btrfs-bug-1.jpg
http://romanrm.ru/pics/2012/2012-02-16-btrfs-bug-2.jpg

This happened four times already, and always on snapshot creation (but not every case). I have hourly snapshots in crontab, and only one case out of about ten fails with this problem. Did not try to deliberately reproduce it yet by manually making snapshots very often, etc.

With respect,
Roman

~~~~~~~~~~~~~~~~~~~~~~~~~~~
"Stallman had a printer,
with code he could not see.
So he began to tinker,
and set the software free."

I've hit a different bug with

* a process doing random reads/writes to a file
* looped toggling of NOCOW flag on the file every 10 seconds
* snapshotting of the fs every 10 seconds

and it triggered after a few minutes. It's an ENOSPC at

920 btrfs_i_size_write(parent_inode, parent_inode->i_size +
921 dentry->d_name.len * 2);
922 ret = btrfs_update_inode(trans, parent_root, parent_inode);
923 BUG_ON(ret);

[49220.539979] ------------[ cut here ]------------
[49220.543580] kernel BUG at fs/btrfs/transaction.c:923!
[49220.543580] invalid opcode: 0000 [#1] SMP
[49220.543580] CPU 0
[49220.543580] Modules linked in: btrfs aoe [last unloaded: btrfs]
[49220.543580]
[49220.543580] Pid: 12646, comm: btrfs Not tainted 3.2.0-default+ #114 Intel Corporation Santa Rosa platform/Matanzas
[49220.543580] RIP: 0010:[<ffffffffa01086da>]  [<ffffffffa01086da>] create_pending_snapshot+0x4ca/0x4d0 [btrfs]
[49220.543580] RSP: 0018:ffff880000869b48  EFLAGS: 00010286
[49220.543580] RAX: 00000000ffffffe4 RBX: ffff8800191afdf8 RCX: ffff88000e8679d8
[49220.543580] RDX: ffff88006cddd440 RSI: ffffffff8187d42d RDI: 0000000000000246
[49220.543580] RBP: ffff880000869c08 R08: 0000000000000000 R09: 0000000000000000
[49220.543580] R10: 0000000000000000 R11: 0000000000000000 R12: ffff8800347b6740
[49220.543580] R13: ffff880054eb5180 R14: ffff880062b56000 R15: ffff880078f45510
[49220.543580] FS:  00007f6637f5c740(0000) GS:ffff88007dc00000(0000) knlGS:0000000000000000
[49220.543580] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[49220.543580] CR2: 00007f80a26c7000 CR3: 000000001edad000 CR4: 00000000000006f0
[49220.543580] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[49220.543580] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[49220.543580] Process btrfs (pid: 12646, threadinfo ffff880000868000, task ffff88006cddd440)
[49220.543580] Stack:
[49220.543580]  ffff880000000002 0000000000000083 ffff880078394000 ffff8800105d8390
[49220.543580]  ffff88001016bcd0 ffff880062b56000 ffff88007a15e070 ffff8800105d8390
[49220.543580]  ffff880079c14000 ffff8800347b6758 0000000000000175 ffffffffffffff84
[49220.543580] Call Trace:
[49220.543580]  [<ffffffffa00e65fa>] ? btrfs_free_path+0x2a/0x40 [btrfs]
[49220.543580]  [<ffffffffa0108726>] create_pending_snapshots+0x46/0x70 [btrfs]
[49220.543580]  [<ffffffffa0109a57>] btrfs_commit_transaction+0x3e7/0x8d0 [btrfs]
[49220.543580]  [<ffffffff81077aa0>] ? wake_up_bit+0x40/0x40
[49220.543580]  [<ffffffff81357eee>] ? do_raw_spin_unlock+0x5e/0xb0
[49220.543580]  [<ffffffffa0139e48>] btrfs_mksubvol+0x358/0x360 [btrfs]
[49220.543580]  [<ffffffffa0139f50>] btrfs_ioctl_snap_create_transid+0x100/0x160 [btrfs]
[49220.543580]  [<ffffffff81113f73>] ? might_fault+0x53/0xb0
[49220.543580]  [<ffffffffa013a12d>] btrfs_ioctl_snap_create_v2.clone.0+0xfd/0x110 [btrfs]
[49220.543580]  [<ffffffffa013ba47>] btrfs_ioctl+0x4a7/0x1270 [btrfs]
[49220.543580]  [<ffffffff81883400>] ? do_page_fault+0x2d0/0x580
[49220.543580]  [<ffffffff8187f150>] ? _raw_spin_unlock_irq+0x30/0x50
[49220.543580]  [<ffffffff8103ff23>] ? finish_task_switch+0x83/0xf0
[49220.543580]  [<ffffffff8103fee6>] ? finish_task_switch+0x46/0xf0
[49220.543580]  [<ffffffff8114d5e8>] do_vfs_ioctl+0x98/0x560
[49220.543580]  [<ffffffff8108b7a9>] ? trace_hardirqs_off_caller+0x29/0xc0
[49220.543580]  [<ffffffff8187f919>] ? retint_swapgs+0x13/0x1b
[49220.543580]  [<ffffffff8114daff>] sys_ioctl+0x4f/0x80
[49220.543580]  [<ffffffff81887c02>] system_call_fastpath+0x16/0x1b
[49220.543580] Code: 75 fd ff ff 0f 1f 44 00 00 0f b7 16 83 e8 02 48 83 c6 02 66 89 17 48 83 c7 02 e9 64 fd ff ff 0f 0b 0f 0b 0f 0b 0f 0b 0f 0b 0f 0b <0f> 0b 0f 1f 40 00 55 48 89 e5 41 56 41 55 41 54 53 66 66 66 66
[49220.543580] RIP  [<ffffffffa01086da>] create_pending_snapshot+0x4ca/0x4d0 [btrfs]
[49220.543580]  RSP <ffff880000869b48>
[49220.937724] ---[ end trace 8f8b8a074f8e75d4 ]---

Interesting, NOCOW and compression don't really mix. We always cow for compression. I'll try to reproduce it.

-chris

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

I hit this one today without nocow or compression. The only thing non-default was that I mounted with -ossd. The backing store was a 1GB non-sparse loopback file on tmpfs.

I had kdump enabled and with 16GB, I wasn't waiting around for the dump to complete. If it happens again, I'll have a full stack trace. My test case was filling the disk while making snapshots.

- -Jeff

- --
Jeff Mahoney
SUSE Labs
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.18 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iQIcBAEBAgAGBQJPX424AAoJEB57S2MheeWybq0P/1xv09dKpBfuvK/2vpyriebN 3RsPtHVZgAL9y9XeLVw0KPlRKTGM+PIyc+AEloIiYQgULifA625nPa6+DXlNqzCp 0jKVleAE5RLCOtBDC91GX8JO/55fszjTQHSXAHjSRr4vr/4bZEf4tLBTwbW7Nfax YAUPQ6cu+FQlCDWppXMLswKmLsmnewTnKrZp0YQTOTVGY3dajsvGTR3vR0VyxMwY FZ13hvGOfP+UXsZC1qijxA3g8CrWwu/dhiU/dnCYwhcCF+0ONiHHHhB3krDAkNP8 yVAZgoSGJsIUdDiqeKXaSm8RDT6vGQwnJicFQSkUqGiHXyi+/fXVUULNSbgj39EC R2jHLlXtMMaRyoqiM/wOfZUOr5MGQhk0duXB1NjNGAaffAsjvBY0c8y4yvGJVhtR E2EknQLHr2jBWF8KCpLe0YYLPjcB3Gp3SPUhyGZbg4ATUAv2amMcw13deI8I17gI v9dJSCWyi+r5c0d26rgtsS9SpfM8qHz5A/EvqVtn99DgS+O49o4+3F8M+2WioDjM mHndWGXqg4fBIwrpxvo5RtBmQ8OYgZzxZXoZU3XP/eRS4c6z8OJ9U92/pUFP64Id 6tpqU1E9XxEijTWwcIyvkSzu6bjOLQucePBPCKkWLtTy9XrbNMTmvcGozKNCTVKK km1EWb9RM2m8JPrEc+JA
=ywbL
-----END PGP SIGNATURE-----

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Well that didn't take long.

[  626.100684] ------------[ cut here ]------------
[  626.104053] kernel BUG at
/usr/src/packages/BUILD/kernel-default-3.0.23/linux-3.0/fs/btrfs/extent-tree.c:6091!
[  626.104053] invalid opcode: 0000 [#1] SMP
[  626.104053] CPU 8
[  626.104053] Modules linked in: btrfs zlib_deflate crc32c libcrc32c

autofs4 edd nfs lockd fscache auth_rpcgss nfs_acl sunrpc ipv6 ipv6_lib af_packet cpufreq_conservative cpufreq_userspace cpufreq_powersave powernow_k8 mperf microcode fuse loop dm_mod igb i2c_piix4 i2c_core k10temp sg dca rtc_cmos pcspkr button serio_raw ext3 jbd mbcache ohci_hcd ehci_hcd usbcore sd_mod usb_common crc_t10dif processor thermal_sys hwmon ata_generic ahci libahci pata_atiixp libata scsi_mod

[  626.104053] Supported: Yes
[  626.104053]
[  626.104053] Pid: 14214, comm: btrfs Not tainted
3.0.23-0.0.0.0.4dd40bc-default #1 HP ProLiant DL165 G7
[  626.104053] RIP: 0010:[<ffffffffa03cd623>]  [<ffffffffa03cd623>]
alloc_reserved_tree_block+0x1e3/0x1f0 [btrfs]
[  626.104053] RSP: 0018:ffff88020766fae8  EFLAGS: 00010246
[  626.104053] RAX: ffff880434b76000 RBX: 0000000000000e2c RCX:
0000000000000000
[  626.104053] RDX: ffff880000000000 RSI: 0000000000000000 RDI:
ffff880404690678
[  626.104053] RBP: ffff880436e7f670 R08: ffff88020766faa8 R09:
0000000000001000
[  626.104053] R10: 0000000000000e2b R11: b000000000000000 R12:
ffff880404690678
[  626.104053] R13: 0000000000000d9d R14: ffff880432ac15a0 R15:
ffff880437a01c80
[  626.104053] FS:  00007fabeaf13740(0000) GS:ffff88043fc00000(0000)
knlGS:0000000000000000
[  626.104053] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[  626.104053] CR2: 00007fabea61bb50 CR3: 0000000402dc1000 CR4:
00000000000006e0
[  626.104053] DR0: 0000000000000000 DR1: 0000000000000000 DR2:
0000000000000000
[  626.104053] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7:
0000000000000400
[  626.104053] Process btrfs (pid: 14214, threadinfo ffff88020766e000,
task ffff8802389260c0)
[  626.104053] Stack:
[  626.104053]  ffff88020766fb28 0000000000000000 0000000000000000
0000000004d3e000
[  626.104053]  ffff8800b220c000 ffff8804357bd000 0000003300000000
ffff88043977c540
[  626.104053]  ffff880437a01c80 ffff880432ac15a0 ffff8800b220c000
0000000000000001
[  626.104053] Call Trace:
[  626.104053]  [<ffffffffa03cdecc>] run_delayed_tree_ref+0xfc/0x150
[btrfs]
[  626.104053]  [<ffffffffa03ce21e>] run_clustered_refs+0xce/0x310 [btrfs]
[  626.104053]  [<ffffffffa03ce599>]
btrfs_run_delayed_refs+0x139/0x2e0 [btrfs]
[  626.104053]  [<ffffffffa03de553>]
btrfs_commit_transaction+0x433/0x8a0 [btrfs]
[  626.104053]  [<ffffffffa040a361>] create_snapshot+0x1a1/0x1c0 [btrfs]
[  626.104053]  [<ffffffffa040aa60>] btrfs_mksubvol+0x150/0x1e0 [btrfs]
[  626.104053]  [<ffffffffa040ac5b>]
btrfs_ioctl_snap_create_transid+0x16b/0x1a0 [btrfs]
[  626.104053]  [<ffffffffa040ad98>]
btrfs_ioctl_snap_create_v2+0x108/0x110 [btrfs]
[  626.104053]  [<ffffffffa040fbb7>] btrfs_ioctl+0x697/0x7d0 [btrfs]
[  626.104053]  [<ffffffff81162b7b>] do_vfs_ioctl+0x8b/0x3b0
[  626.104053]  [<ffffffff81162f41>] sys_ioctl+0xa1/0xb0
[  626.104053]  [<ffffffff8144ad12>] system_call_fastpath+0x16/0x1b
[  626.104053] DWARF2 unwinder stuck at system_call_fastpath+0x16/0x1b
[  626.104053]
[  626.104053] Leftover inexact backtrace:
[  626.104053]
[  626.104053] Code: 4c 89 e7 e8 50 76 02 00 e9 6b ff ff ff 48 8b 44
24 78 48 c7 c7 30 94 43 a0 48 8b 50 09 48 8b 30 31 c0 e8 47 2b 07 e1
0f 0b eb fe <0f> 0b eb fe 66 0f 1f 84 00 00 00 00 00 41 57 41 56 41 55
41 54
[  626.104053] RIP  [<ffffffffa03cd623>]
alloc_reserved_tree_block+0x1e3/0x1f0 [btrfs]
[  626.104053]  RSP <ffff88020766fae8>

- --
Jeff Mahoney
SUSE Labs
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.18 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iQIcBAEBAgAGBQJPX5OQAAoJEB57S2MheeWyxisP/37vZAl8f9N9OWwXKHiqybla 6nKCLN2DN0XaNNP/2hmjXh/V8AdU1ZQOR4apS1qrLIHXCZOoF3kJlTR5vrEwUxtW GbkWr6t5TfSpdit+eSceADbGM9Yy7pi6AY830D7qj059dliLfbKHIw7k4gCbWQXa STnh2ukeFYO75a9w7RbxxlK4I2Lt7Nf5Qmd8XZkyAJyvNqYrMhu/SXkpwz8rym3U S0VH44zlKsLKSMNvn0dQYwKDZ5z1TbNgna9Wg1CpRYihVfAIjva58YXobkXHgZAb eIuOoBuPEvdw+ZNSfKlB1a/TblcHUvgSteJYfcJu/EzgGIBKlzzIrR+TQZDegVnR b4I/Jad/2Xm+1L9OZQHhJAaRetiY15aDtJ+R6+KewsQgYYILNXlIOMjA9+X646Wm ypWQ6q6XjJ4XMEEGzXGEBfrJpUXeuWEXpQkRfxI7xvGmrXT8/r4yiEqrUVbWOoUa EKMeN5u61HTXcUCdB0HtjAppO2mqFnuAVH/A4CkO3gRxVaOjPTFKIZszuxE/aJT5 Rs1VZMUoJtbLPu+npQpg2OLMUI8tdyt3RshSWEdynZaAk0utlH0lZa37shkniwe1 cK+tZwZJLQL8YLnBpF+BV9xn9dJTM82acjnOA5akSqyCXgrJcU5jrHYbiyqe44L6 I5rH1gzrESJKc9qU59Pz
=SpJv
-----END PGP SIGNATURE-----

I get the following errors when running fileflags on large (>2GB) database files:

open(): No such file or directory

open(): Value too large for defined data type

http://www.gnu.org/software/coreutils/faq/#Value-too-large-for-defined-data-type

"""The message "Value too large for defined data type" is a system error message reported when an operation on a large file is attempted using a non-large file data type. Large files are defined as anything larger than a signed 32-bit integer, or stated differently, larger than 2GB.

Many system calls that deal with files return values in a "long int" data type. On 32-bit hardware a long int is 32-bits and therefore this imposes a 2GB limit on the size of files. When this was invented that was HUGE and it was hard to conceive of needing anything that large. Time has passed and files can be much larger today. On native 64-bit systems the file size limit is usually 2GB * 2GB. Which we will again think is huge."""

Hello,
Going back to the original question from Ralf I wanted to share my experience.

Yesterday I set up KVM+qemu and set -z -C with David's 'fileflags' utility for the VM image file.
I was very pleased with results - Redhat 6 Minimal installation was installed in 10 minutes whereas it was taking 'forever' the last time I tried it some 4 months ago. Writes during installation were very moderate. Performance of VM is excellent. Installing some big packages with yum inside VM goes very quickly with the speed indistinguishable from that of bare metal installs.

I am not quite sure should this improvement be attributed to the nocow and nocompress flags or to the overall improvement of btrfs (I am on 3.3-rc4 kernel) but KVM is definitely more than usable on btrfs now.

I am yet to test the install speed and performance without those flags set.

best
~dima

Hello,

Since several people asked to post the results, here they are.
I tried raw virtio disk with and without -z -C set and also qcow2 virtio disk without -z -C set and did not notice any difference in performance at all - Redhat 6.2 Minimal installs in 10 minutes in each case. The "abysmal" performance as it was some several months ago (like 10 minutes just for virtual disk formatting) under the same conditions is no more at least on 3.3.0-rc5.

best
~dima

tio

ce at

ysmal"

or

t on

Just to make sure, this is a _new_ virtual disk right? I can barely contain my excitement right now. This is amazing progress.

d

nce

e I

es

e

ow

gs

s" in

" in

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" = in
the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

Yes, it is a newly created virtual disk. By the way, one thing that slipped out from my message - in case of raw I did pre-allocation of the entire image, but in case of qcow2 I unchecked this box in virt-manager and the disk was "growing" as the system was installing. Nevertheless I did not notice performance degradation during the install.

best
~dima

But still Virtualbox is faster. It installs Redhat 6 Minimal in just about 5 minutes. I observe that yum installations work faster.
WindowsXP install under KVM was a hell lasting for about 3 hours and the performance is awful with host HD indicator flashing all the time. There was no change whatsoever whether the image was with +C +z or not. On Virtualbox WinXP install completed in about 25 minutes and the performance is excellent even w/o guest additions applied.
I wonder why such a big difference.