Skip to contentSkip to conversation

Enhance btrfs fi df

PATCH In review Goffredo Baroncelli 8 participants 30 messages Feb 13, 2014 – Feb 22, 2014

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

PATCH v4 1/1 Enhance btrfs fi df

Hi all,

This is the 4th attempt of my patches related to show how the data are stored in a btrfs filesystem. I rebased all the patches on the v3.13 btrfs-progs.

These patches update the btrfs fi df command and add two new commands:
- btrfs filesystem disk-usage <path>
- btrfs device disk-usage <path>

The command "btrfs filesystem df" now shows only the disk usage/available.

$ sudo btrfs filesystem df /mnt/btrfs1/
Disk size:		 400.00GB
Disk allocated:		   8.04GB
Disk unallocated:	 391.97GB
Used:			  11.29MB
Free (Estimated):	 250.45GB	(Max: 396.99GB, min: 201.00GB)
Data to disk ratio:	     63 %

The "Free (Estimated)" tries to give an estimation of the free space on the basis of the chunks usage. Max and min are the maximum allowable space (if the next chunk are allocated as SINGLE) or the minimum one ( if the next chunks are allocated as DUP/RAID1/RAID10).

The other two commands show the chunks in the disks.

$ sudo btrfs filesystem disk-usage /mnt/btrfs1/
Data,Single: Size:8.00MB, Used:0.00
   /dev/vdb	    8.00MB

Data,RAID6: Size:2.00GB, Used:11.25MB

   /dev/vdb	    1.00GB
   /dev/vdc	    1.00GB
   /dev/vdd	    1.00GB
   /dev/vde	    1.00GB

Metadata,Single: Size:8.00MB, Used:0.00
/dev/vdb 8.00MB

Metadata,RAID5: Size:3.00GB, Used:36.00KB

   /dev/vdb	    1.00GB
   /dev/vdc	    1.00GB
   /dev/vdd	    1.00GB
   /dev/vde	    1.00GB

System,Single: Size:4.00MB, Used:0.00
/dev/vdb 4.00MB

System,RAID5: Size:12.00MB, Used:4.00KB

   /dev/vdb	    4.00MB
   /dev/vdc	    4.00MB
   /dev/vdd	    4.00MB
   /dev/vde	    4.00MB

Unallocated:

   /dev/vdb	   97.98GB
   /dev/vdc	   98.00GB
   /dev/vdd	   98.00GB
   /dev/vde	   98.00GB

or in tabular format

$ sudo ./btrfs filesystem disk-usage -t /mnt/btrfs1/
         Data   Data    Metadata Metadata System System             
         Single RAID6   Single   RAID5    Single RAID5   Unallocated
/dev/vdb 8.00MB  1.00GB   8.00MB   1.00GB 4.00MB  4.00MB     97.98GB
/dev/vdc      -  1.00GB        -   1.00GB      -  4.00MB     98.00GB
/dev/vdd      -  1.00GB        -   1.00GB      -  4.00MB     98.00GB
/dev/vde      -  1.00GB        -   1.00GB      -  4.00MB     98.00GB
         ====== ======= ======== ======== ====== ======= ===========

Total 8.00MB 2.00GB 8.00MB 3.00GB 4.00MB 12.00MB 391.97GB
Used 0.00 11.25MB 0.00 36.00KB 0.00 4.00KB

These are the most complete output, where it is possible to know which disk a chunk uses and the usage of every chunk.

Finally the last command shows which chunks a disk hosts:

$ sudo ./btrfs device disk-usage /mnt/btrfs1/
/dev/vdb	  100.00GB
   Data,Single:              8.00MB
   Data,RAID6:               1.00GB
   Metadata,Single:          8.00MB
   Metadata,RAID5:           1.00GB
   System,Single:            4.00MB
   System,RAID5:             4.00MB
   Unallocated:             97.98GB

/dev/vdc 100.00GB

   Data,RAID6:               1.00GB
   Metadata,RAID5:           1.00GB
   System,RAID5:             4.00MB

Unallocated: 98.00GB

/dev/vdd 100.00GB

   Data,RAID6:               1.00GB
   Metadata,RAID5:           1.00GB
   System,RAID5:             4.00MB

Unallocated: 98.00GB

/dev/vde 100.00GB

   Data,RAID6:               1.00GB
   Metadata,RAID5:           1.00GB
   System,RAID5:             4.00MB

Unallocated: 98.00GB

More or less are the same information above, only grouped by disk. Unfortunately I don't have any information about the chunk usage per disk basis.

Comments are welcome.
The code is pullable from
https://github.com/kreijack/btrfs-progs.git
branch
dfdu

Finally I have to point out that the command btrfs fi df previous didn't require the root capability, now with my patches it is required, because I need to know some info about the chunks so I need to use the "BTRFS_IOC_TREE_SEARCH".

I think that there are the following possibilities:
1) accept this regresssion
2) remove the command "btrfs fi df" and leave only "btrfs fi disk-usage" and "btrfs dev disk-usage"
3) adding a new ioctl which could be used without root capability. Of course this ioctl would return only a subset of the BTRFS_IOC_TREE_SEARCH info

I think that the 3) would be the "long term" solution. I am not happy about the 1), so as "short term solution" I think that we should go with the 2). What do you think ?

Below the description of the patches.

--

BR
G.Baroncelli

Changelog:
v1 2013/02/18 First issue
v2 2013/02/23 Fix uncorrct "not enough memory" handling in patch #1.
Thanks to Zac to highlight it.
v3 2013/03/10 - Rebased on the latest mason git repository
- Small fix in the comment
- Increase the buffer in the function df_pretty_sizes() to avoid buffer pverwflow (thanks to Zach to highlight it)
- adding the sla_pretty_sizes() function to remove the call of string_list_add() from df_pretty_sizes.
v4 2014/02/13 - Rebased to v3.12
- Replace MB/GB/TB with MiB/GiB/TiB
- correct some small bug related to mixing same raid level with different disk number
-removing the string_list_add() function

Signature · 2 lines
gpg @keyserver.linux.it: Goffredo Baroncelli (kreijackATinwind.it>
Key fingerprint BBF5 1610 0B64 DAC6 5F7D  17B2 0EDA 9B37 8B82 E0B5
#1 lore raw

PATCH 1/8 Enhance the command btrfs filesystem df.

Enhance the command "btrfs filesystem df" to show space usage information for a mount point(s). It shows also an estimation of the space available, on the basis of the current one used.

---
 Makefile             |   3 +-
 cmds-fi-disk_usage.c | 516 +++++++++++++++++++++++++++++++++++++++++++++++++++
 cmds-fi-disk_usage.h |  25 +++
 cmds-filesystem.c    |  91 +--------
 ctree.h              |  17 +-
 utils.c              |  13 ++
 utils.h              |   2 +
 7 files changed, 575 insertions(+), 92 deletions(-)
 create mode 100644 cmds-fi-disk_usage.c
 create mode 100644 cmds-fi-disk_usage.h
Makefile +2 -1
index 0874a41..db470b6 100644
@@ -13,7 +13,8 @@ objects = ctree.o disk-io.o radix-tree.o extent-tree.o print-tree.o \
 cmds_objects = cmds-subvolume.o cmds-filesystem.o cmds-device.o cmds-scrub.o \
 	       cmds-inspect.o cmds-balance.o cmds-send.o cmds-receive.o \
 	       cmds-quota.o cmds-qgroup.o cmds-replace.o cmds-check.o \
-	       cmds-restore.o cmds-rescue.o chunk-recover.o super-recover.o
+	       cmds-restore.o cmds-rescue.o chunk-recover.o super-recover.o \
+	       cmds-fi-disk_usage.o
 libbtrfs_objects = send-stream.o send-utils.o rbtree.o btrfs-list.o crc32c.o \
 		   uuid-tree.o
 libbtrfs_headers = send-stream.h send-utils.h send.h rbtree.h btrfs-list.h \
cmds-fi-disk_usage.c new file +516 -0

516 lines not on this page · raw message

cmds-fi-disk_usage.h new file +25 -0
new file mode 100644
index 0000000..9f68bb3
@@ -0,0 +1,25 @@
+/*
+ * Copyright (C) 2007 Oracle.  All rights reserved.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public
+ * License v2 as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public
+ * License along with this program; if not, write to the
+ * Free Software Foundation, Inc., 59 Temple Place - Suite 330,
+ * Boston, MA 021110-1307, USA.
+ */
+
+#ifndef __CMDS_FI_DISK_USAGE__
+#define __CMDS_FI_DISK_USAGE__
+
+extern const char * const cmd_filesystem_df_usage[];
+int cmd_filesystem_df(int argc, char **argv);
+
+#endif
cmds-filesystem.c +2 -89

115 lines not on this page · raw message

ctree.h +15 -2
index 2117374..ff7ffee 100644
@@ -830,9 +830,22 @@ struct btrfs_csum_item {
 #define BTRFS_BLOCK_GROUP_RAID1		(1ULL << 4)
 #define BTRFS_BLOCK_GROUP_DUP		(1ULL << 5)
 #define BTRFS_BLOCK_GROUP_RAID10	(1ULL << 6)
-#define BTRFS_BLOCK_GROUP_RAID5    (1ULL << 7)
-#define BTRFS_BLOCK_GROUP_RAID6    (1ULL << 8)
+#define BTRFS_BLOCK_GROUP_RAID5    	(1ULL << 7)
+#define BTRFS_BLOCK_GROUP_RAID6    	(1ULL << 8)
 #define BTRFS_BLOCK_GROUP_RESERVED	BTRFS_AVAIL_ALLOC_BIT_SINGLE
+#define BTRFS_NR_RAID_TYPES             7
+
+#define BTRFS_BLOCK_GROUP_TYPE_MASK     (BTRFS_BLOCK_GROUP_DATA |    \
+					 BTRFS_BLOCK_GROUP_SYSTEM |  \
+					 BTRFS_BLOCK_GROUP_METADATA)
+
+#define BTRFS_BLOCK_GROUP_PROFILE_MASK  (BTRFS_BLOCK_GROUP_RAID0 |   \
+					 BTRFS_BLOCK_GROUP_RAID1 |   \
+					 BTRFS_BLOCK_GROUP_RAID5 |   \
+					 BTRFS_BLOCK_GROUP_RAID6 |   \
+					 BTRFS_BLOCK_GROUP_DUP |     \
+					 BTRFS_BLOCK_GROUP_RAID10)
+
 
 #define BTRFS_BLOCK_GROUP_TYPE_MASK	(BTRFS_BLOCK_GROUP_DATA |    \
 					 BTRFS_BLOCK_GROUP_SYSTEM |  \
utils.c +13 -0
index f499023..760db6b 100644
@@ -38,6 +38,8 @@
 #include <linux/kdev_t.h>
 #include <limits.h>
 #include <blkid/blkid.h>
+#include <sys/vfs.h>
+
 #include "kerncompat.h"
 #include "radix-tree.h"
 #include "ctree.h"
@@ -2108,3 +2110,14 @@ int lookup_ino_rootid(int fd, u64 *rootid)
 
 	return 0;
 }
+
+u64 disk_size(char *path)
+{
+	struct statfs	sfs;
+
+	if (statfs(path, &sfs) < 0)
+		return 0;
+	else
+		return sfs.f_bsize * sfs.f_blocks;
+
+}
utils.h +2 -0
index 6f4b10c..0f7939c 100644
@@ -20,6 +20,7 @@
 #define __UTILS__
 
 #include <sys/stat.h>
+#include "kerncompat.h"
 #include "ctree.h"
 #include <dirent.h>
 
@@ -95,4 +96,5 @@ int lookup_ino_rootid(int fd, u64 *rootid);
 int btrfs_scan_lblkid(int update_kernel);
 int get_btrfs_mount(const char *dev, char *mp, size_t mp_size);
 
+u64 disk_size(char *path);
 #endif
Signature · 7 lines
1.8.5.3



-- 
gpg @keyserver.linux.it: Goffredo Baroncelli (kreijackATinwind.it>
Key fingerprint BBF5 1610 0B64 DAC6 5F7D  17B2 0EDA 9B37 8B82 E0B5
#2 lore raw

PATCH 2/8 Create the man page entry for the command btrfs fi df

In reply to Goffredo Baroncelli
---
 man/btrfs.8.in | 52 ++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 52 insertions(+)
man/btrfs.8.in +52 -0
index b620348..4acf16d 100644
@@ -35,6 +35,9 @@ btrfs \- control a btrfs filesystem
 .PP
 \fBbtrfs\fP \fBfilesystem label\fP [\fI<device>\fP|\fI<mount_point>\fP] [\fI<newlabel>\fP]
 .PP
+\fBbtrfs\fP \fBfilesystem df\fP\fI [-b] \fIpath [path..]\fR\fP
+.PP
+\fBbtrfs\fP \fBfilesystem balance\fP\fI <path> \fP
 .PP
 \fBbtrfs\fP \fB[filesystem] balance start\fP [\fIoptions\fP] \fI<path>\fP
 .PP
@@ -370,6 +373,55 @@ force reducing of metadata integrity
 
 \fB[filesystem] balance pause\fR\fI <path>\fR
 Pause running balance.
+
+.RE
+.TP
+
+\fBfilesystem df\fP [-b] \fIpath [path..]\fR
+
+Show space usage information for a mount point.
+
+\fB-b\fP Set byte as unit
+
+The command \fBbtrfs filesystem df\fP is used to query how many space on the 
+disk(s) are used and an estimation of the free
+space of the filesystem.
+The output of the command \fBbtrfs filesystem df\fP shows:
+
+.RS
+.IP \fBDisk\ size\fP
+the total size of the disks which compose the filesystem.
+
+.IP \fBDisk\ allocated\fP
+the size of the area of the disks used by the chunks.
+
+.IP \fBDisk\ unallocated\fP 
+the size of the area of the disks which is free (i.e.
+the differences of the values above).
+
+.IP \fBUsed\fP
+the portion of the logical space used by the file and metadata.
+
+.IP \fBFree\ (estimated)\fP
+the estimated free space available: i.e. how many space can be used
+by the user. The evaluation 
+cannot be rigorous because it depends by the allocation policy (DUP, Single,
+RAID1...) of the metadata and data chunks. If every chunk is stored as
+"Single" the sum of the \fBfree (estimated)\fP space and the \fBused\fP 
+space  is equal to the \fBdisk size\fP.
+Otherwise if all the chunk are mirrored (raid1 or raid10) or duplicated
+the sum of the \fBfree (estimated)\fP space and the \fBused\fP space is
+half of the \fBdisk size\fP. Normally the \fBfree (estimated)\fP is between
+these two limits.
+
+.IP \fBData\ to\ disk\ ratio\fP
+the ratio betwen the \fBlogical size\fP (i.e. the space available by
+the chunks) and the \fBdisk allocated\fP (by the chunks). Normally it is 
+lower than 100% because the metadata is duplicated for security reasons.
+If all the data and metadata are duplicated (or have a profile like RAID1)
+the \fBData\ to\ disk\ ratio\fP could be 50%.
+
+.RE
 .TP
 
 \fB[filesystem] balance cancel\fR\fI <path>\fR
Signature · 7 lines
1.8.5.3



-- 
gpg @keyserver.linux.it: Goffredo Baroncelli (kreijackATinwind.it>
Key fingerprint BBF5 1610 0B64 DAC6 5F7D  17B2 0EDA 9B37 8B82 E0B5
#3 lore raw

PATCH 3/8 Add helpers functions to handle the printing of data in tabular format.

In reply to Goffredo Baroncelli

This patch adds some functions to manage the printing of the data in tabular format.

The function
struct string_table *table_create(int columns, int rows)
creates an (empty) table.

The functions
char *table_printf(struct string_table *tab, int column,
int row, char *fmt, ...)
char *table_vprintf(struct string_table *tab, int column,
int row, char *fmt, va_list ap)
populate the table with text. To align the text to the left, the text shall be prefixed with '<', otherwise the text shall be prefixed by a '>'. If the first character is a '=', the the text is replace by a sequence of '=' to fill the column width.

The function
void table_free(struct string_table *)
frees all the data associated to the table.

The function
void table_dump(struct string_table *tab)
prints the table on stdout.

---
 Makefile       |   2 +-
 string_table.c | 156 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 string_table.h |  36 +++++++++++++
 3 files changed, 193 insertions(+), 1 deletion(-)
 create mode 100644 string_table.c
 create mode 100644 string_table.h
Makefile +1 -1
index db470b6..5738379 100644
@@ -9,7 +9,7 @@ CFLAGS = -g -O1 -fno-strict-aliasing
 objects = ctree.o disk-io.o radix-tree.o extent-tree.o print-tree.o \
 	  root-tree.o dir-item.o file-item.o inode-item.o inode-map.o \
 	  extent-cache.o extent_io.o volumes.o utils.o repair.o \
-	  qgroup.o raid6.o free-space-cache.o list_sort.o
+	  qgroup.o raid6.o free-space-cache.o list_sort.o string_table.o 
 cmds_objects = cmds-subvolume.o cmds-filesystem.o cmds-device.o cmds-scrub.o \
 	       cmds-inspect.o cmds-balance.o cmds-send.o cmds-receive.o \
 	       cmds-quota.o cmds-qgroup.o cmds-replace.o cmds-check.o \
string_table.c new file +156 -0

156 lines not on this page · raw message

string_table.h new file +36 -0
new file mode 100644
index 0000000..83c4425
@@ -0,0 +1,36 @@
+/*
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public
+ * License v2 as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public
+ * License along with this program; if not, write to the
+ * Free Software Foundation, Inc., 59 Temple Place - Suite 330,
+ * Boston, MA 021110-1307, USA.
+ */
+
+#ifndef STRING_TABLE_H
+#define STRING_TABLE_H
+
+struct string_table {
+
+	int	ncols, nrows;
+	char	*cells[];
+
+};
+
+
+struct string_table *table_create(int columns, int rows);
+char *table_printf(struct string_table *tab, int column, int row,
+			  char *fmt, ...);
+char *table_vprintf(struct string_table *tab, int column, int row,
+			  char *fmt, va_list ap);
+void table_dump(struct string_table *tab);
+void table_free(struct string_table *);
+
+#endif
Signature · 7 lines
1.8.5.3



-- 
gpg @keyserver.linux.it: Goffredo Baroncelli (kreijackATinwind.it>
Key fingerprint BBF5 1610 0B64 DAC6 5F7D  17B2 0EDA 9B37 8B82 E0B5
#4 lore raw

PATCH 4/8 Allow use of get_device_info()

In reply to Goffredo Baroncelli

Allow the use of get_device_info() for different units.

---
 utils.c | 2 +-
 utils.h | 2 ++
 2 files changed, 3 insertions(+), 1 deletion(-)
utils.c +1 -1
index 760db6b..69295b5 100644
@@ -1591,7 +1591,7 @@ void close_file_or_dir(int fd, DIR *dirstream)
 		close(fd);
 }
 
-static int get_device_info(int fd, u64 devid,
+int get_device_info(int fd, u64 devid,
 		struct btrfs_ioctl_dev_info_args *di_args)
 {
 	int ret;
utils.h +2 -0
index 0f7939c..58fb4a7 100644
@@ -97,4 +97,6 @@ int btrfs_scan_lblkid(int update_kernel);
 int get_btrfs_mount(const char *dev, char *mp, size_t mp_size);
 
 u64 disk_size(char *path);
+int get_device_info(int fd, u64 devid,
+		struct btrfs_ioctl_dev_info_args *di_args);
 #endif
Signature · 7 lines
1.8.5.3



-- 
gpg @keyserver.linux.it: Goffredo Baroncelli (kreijackATinwind.it>
Key fingerprint BBF5 1610 0B64 DAC6 5F7D  17B2 0EDA 9B37 8B82 E0B5
#6 lore raw

PATCH 5/8 Add command btrfs filesystem disk-usage

In reply to Goffredo Baroncelli
---
 cmds-fi-disk_usage.c | 428 +++++++++++++++++++++++++++++++++++++++++++++++++++
 cmds-fi-disk_usage.h |   3 +
 cmds-filesystem.c    |   3 +
 utils.c              |  63 ++++++++
 utils.h              |   3 +
 5 files changed, 500 insertions(+)
cmds-fi-disk_usage.c +428 -0
index 4012c78..16b3ab2 100644
@@ -20,10 +20,12 @@
 #include <unistd.h>
 #include <sys/ioctl.h>
 #include <errno.h>
+#include <stdarg.h>
 
 #include "utils.h"
 #include "kerncompat.h"
 #include "ctree.h"
+#include "string_table.h"
 
 #include "commands.h"
 
@@ -44,6 +46,13 @@ struct chunk_info {
 	u64	num_stripes;
 };
 
+/* to store information about the disks */
+struct disk_info {
+	u64	devid;
+	char	path[BTRFS_DEVICE_PATH_NAME_MAX];
+	u64	size;
+};
+
 /*
  * Pretty print the size
  * PAY ATTENTION: it return a statically buffer
@@ -514,3 +523,422 @@ int cmd_filesystem_df(int argc, char **argv)
 	return 0;
 }
 
+/*
+ *  Helper to sort the disk_info structure
+ */
+static int cmp_disk_info(const void *a, const void *b)
+{
+	return strcmp(((struct disk_info *)a)->path,
+			((struct disk_info *)b)->path);
+}
+
+/*
+ *  This function load the disk_info structure and put them in an array
+ */
+static int load_disks_info(int fd,
+			   struct disk_info **disks_info_ptr,
+			   int *disks_info_count)
+{
+
+	int ret, i, ndevs;
+	struct btrfs_ioctl_fs_info_args fi_args;
+	struct btrfs_ioctl_dev_info_args dev_info;
+	struct disk_info *info;
+
+	*disks_info_count = 0;
+	*disks_info_ptr = 0;
+
+	ret = ioctl(fd, BTRFS_IOC_FS_INFO, &fi_args);
+	if (ret < 0) {
+		fprintf(stderr, "ERROR: cannot get filesystem info\n");
+		return -1;
+	}
+
+	info = malloc(sizeof(struct disk_info) * fi_args.num_devices);
+	if (!info) {
+		fprintf(stderr, "ERROR: not enough memory\n");
+		return -1;
+	}
+
+	for (i = 0, ndevs = 0 ; i <= fi_args.max_id ; i++) {
+
+		BUG_ON(ndevs >= fi_args.num_devices);
+		ret = get_device_info(fd, i, &dev_info);
+
+		if (ret == -ENODEV)
+			continue;
+		if (ret) {
+			fprintf(stderr,
+			    "ERROR: cannot get info about device devid=%d\n",
+			    i);
+			free(info);
+			return -1;
+		}
+
+		info[ndevs].devid = dev_info.devid;
+		strcpy(info[ndevs].path, (char *)dev_info.path);
+		info[ndevs].size = get_partition_size((char *)dev_info.path);
+		++ndevs;
+	}
+
+	BUG_ON(ndevs != fi_args.num_devices);
+	qsort(info, fi_args.num_devices,
+		sizeof(struct disk_info), cmp_disk_info);
+
+	*disks_info_count = fi_args.num_devices;
+	*disks_info_ptr = info;
+
+	return 0;
+
+}
+
+/*
+ *  This function computes the size of a chunk in a disk
+ */
+static u64 calc_chunk_size(struct chunk_info *ci)
+{
+	if (ci->type & BTRFS_BLOCK_GROUP_RAID0)
+		return ci->size / ci->num_stripes;
+	else if (ci->type & BTRFS_BLOCK_GROUP_RAID1)
+		return ci->size ;
+	else if (ci->type & BTRFS_BLOCK_GROUP_DUP)
+		return ci->size ;
+	else if (ci->type & BTRFS_BLOCK_GROUP_RAID5)
+		return ci->size / (ci->num_stripes -1);
+	else if (ci->type & BTRFS_BLOCK_GROUP_RAID6)
+		return ci->size / (ci->num_stripes -2);
+	else if (ci->type & BTRFS_BLOCK_GROUP_RAID10)
+		return ci->size / ci->num_stripes;
+	return ci->size;
+}
+
+/*
+ *  This function print the results of the command btrfs fi disk-usage
+ *  in tabular format
+ */
+static void _cmd_filesystem_disk_usage_tabular(int mode,
+					struct btrfs_ioctl_space_args *sargs,
+					struct chunk_info *chunks_info_ptr,
+					int chunks_info_count,
+					struct disk_info *disks_info_ptr,
+					int disks_info_count)
+{
+	int i;
+	u64 total_unused = 0;
+	struct string_table *matrix = 0;
+	int  ncols, nrows;
+
+	ncols = sargs->total_spaces + 2;
+	nrows = 2 + 1 + disks_info_count + 1 + 2;
+
+	matrix = table_create(ncols, nrows);
+	if (!matrix) {
+		fprintf(stderr, "ERROR: not enough memory\n");
+		return;
+	}
+
+	/* header */
+	for (i = 0; i < sargs->total_spaces; i++) {
+		const char *description;
+
+		u64 flags = sargs->spaces[i].flags;
+		description = group_type_str(flags);
+
+		table_printf(matrix, 1+i, 0, "<%s", description);
+	}
+
+	for (i = 0; i < sargs->total_spaces; i++) {
+		const char *r_mode;
+
+		u64 flags = sargs->spaces[i].flags;
+		r_mode = group_profile_str(flags);
+
+		table_printf(matrix, 1+i, 1, "<%s", r_mode);
+	}
+
+	table_printf(matrix, 1+sargs->total_spaces, 1, "<Unallocated");
+
+	/* body */
+	for (i = 0 ; i < disks_info_count ; i++) {
+		int k, col;
+		char *p;
+
+		u64  total_allocated = 0, unused;
+
+		p = strrchr(disks_info_ptr[i].path, '/');
+		if (!p)
+			p = disks_info_ptr[i].path;
+		else
+			p++;
+
+		table_printf(matrix, 0, i+3, "<%s",
+				disks_info_ptr[i].path);
+
+		for (col = 1, k = 0 ; k < sargs->total_spaces ; k++)  {
+			u64	flags = sargs->spaces[k].flags;
+			u64 devid = disks_info_ptr[i].devid;
+			int	j;
+			u64 size = 0;
+
+			for (j = 0 ; j < chunks_info_count ; j++) {
+				if (chunks_info_ptr[j].type != flags )
+						continue;
+				if (chunks_info_ptr[j].devid != devid)
+						continue;
+
+				size += calc_chunk_size(chunks_info_ptr+j);
+			}
+
+			if (size)
+				table_printf(matrix, col, i+3,
+					">%s", df_pretty_sizes(size, mode));
+			else
+				table_printf(matrix, col, i+3, ">-");
+
+			total_allocated += size;
+			col++;
+		}
+
+		unused = get_partition_size(disks_info_ptr[i].path) -
+				total_allocated;
+
+		table_printf(matrix, sargs->total_spaces + 1, i + 3,
+			       ">%s", df_pretty_sizes(unused, mode));
+		total_unused += unused;
+
+	}
+
+	for (i = 0; i <= sargs->total_spaces; i++)
+		table_printf(matrix, i + 1, disks_info_count + 3, "=");
+
+
+	/* footer */
+	table_printf(matrix, 0, disks_info_count + 4, "<Total");
+	for (i = 0; i < sargs->total_spaces; i++)
+		table_printf(matrix, 1 + i, disks_info_count + 4,
+			">%s",
+			df_pretty_sizes(sargs->spaces[i].total_bytes, mode));
+
+	table_printf(matrix, sargs->total_spaces+1, disks_info_count+4,
+		">%s", df_pretty_sizes(total_unused, mode));
+
+	table_printf(matrix, 0, disks_info_count+5, "<Used");
+	for (i = 0; i < sargs->total_spaces; i++)
+		table_printf(matrix, 1+i, disks_info_count+5, ">%s",
+			df_pretty_sizes(sargs->spaces[i].used_bytes, mode));
+
+
+	table_dump(matrix);
+	table_free(matrix);
+
+}
+
+/*
+ *  This function prints the unused space per every disk
+ */
+static void print_unused(struct chunk_info *info_ptr,
+			  int info_count,
+			  struct disk_info *disks_info_ptr,
+			  int disks_info_count,
+			  int mode)
+{
+	int i;
+	for (i = 0 ; i < disks_info_count ; i++) {
+
+		int	j;
+		u64	total = 0;
+
+		for (j = 0 ; j < info_count ; j++)
+			if (info_ptr[j].devid == disks_info_ptr[i].devid)
+				total += calc_chunk_size(info_ptr+j);
+
+		printf("   %s\t%10s\n",
+			disks_info_ptr[i].path,
+			df_pretty_sizes(disks_info_ptr[i].size - total, mode));
+
+	}
+
+}
+
+/*
+ *  This function prints the allocated chunk per every disk
+ */
+static void print_chunk_disks(u64 chunk_type,
+				struct chunk_info *chunks_info_ptr,
+				int chunks_info_count,
+				struct disk_info *disks_info_ptr,
+				int disks_info_count,
+				int mode)
+{
+	int i;
+
+	for (i = 0 ; i < disks_info_count ; i++) {
+
+		int	j;
+		u64	total = 0;
+
+		for (j = 0 ; j < chunks_info_count ; j++) {
+
+			if (chunks_info_ptr[j].type != chunk_type)
+				continue;
+			if (chunks_info_ptr[j].devid != disks_info_ptr[i].devid)
+				continue;
+
+			total += calc_chunk_size(&(chunks_info_ptr[j]));
+			//total += chunks_info_ptr[j].size;
+		}
+
+		if (total > 0)
+			printf("   %s\t%10s\n",
+				disks_info_ptr[i].path,
+				df_pretty_sizes(total, mode));
+	}
+}
+
+/*
+ *  This function print the results of the command btrfs fi disk-usage
+ *  in linear format
+ */
+static void _cmd_filesystem_disk_usage_linear(int mode,
+					struct btrfs_ioctl_space_args *sargs,
+					struct chunk_info *info_ptr,
+					int info_count,
+					struct disk_info *disks_info_ptr,
+					int disks_info_count)
+{
+	int i;
+
+	for (i = 0; i < sargs->total_spaces; i++) {
+		const char *description;
+		const char *r_mode;
+
+		u64 flags = sargs->spaces[i].flags;
+		description= group_type_str(flags);
+		r_mode = group_profile_str(flags);
+
+		printf("%s,%s: Size:%s, ",
+			description,
+			r_mode,
+			df_pretty_sizes(sargs->spaces[i].total_bytes ,
+			    mode));
+		printf("Used:%s\n",
+			df_pretty_sizes(sargs->spaces[i].used_bytes,
+					mode));
+		print_chunk_disks(flags, info_ptr, info_count,
+				disks_info_ptr, disks_info_count,
+				mode);
+		printf("\n");
+
+	}
+
+	printf("Unallocated:\n");
+	print_unused(info_ptr, info_count,
+			disks_info_ptr, disks_info_count,
+			mode);
+
+
+
+}
+
+static int _cmd_filesystem_disk_usage(int fd, char *path, int mode, int tabular)
+{
+	struct btrfs_ioctl_space_args *sargs = 0;
+	int info_count = 0;
+	struct chunk_info *info_ptr = 0;
+	struct disk_info *disks_info_ptr = 0;
+	int disks_info_count = 0;
+	int ret = 0;
+
+	if (load_chunk_info(fd, &info_ptr, &info_count) ||
+	    load_disks_info(fd, &disks_info_ptr, &disks_info_count)) {
+		ret = -1;
+		goto exit;
+	}
+
+	if ((sargs = load_space_info(fd, path)) == NULL) {
+		ret = -1;
+		goto exit;
+	}
+
+	if (tabular)
+		_cmd_filesystem_disk_usage_tabular(mode, sargs,
+					info_ptr, info_count,
+					disks_info_ptr, disks_info_count);
+	else
+		_cmd_filesystem_disk_usage_linear(mode, sargs,
+					info_ptr, info_count,
+					disks_info_ptr, disks_info_count);
+
+exit:
+
+	if (sargs)
+		free(sargs);
+	if (disks_info_ptr)
+		free(disks_info_ptr);
+	if (info_ptr)
+		free(info_ptr);
+
+	return ret;
+}
+
+const char * const cmd_filesystem_disk_usage_usage[] = {
+	"btrfs filesystem disk-usage [-b][-t] <path> [<path>..]",
+	"Show in which disk the chunks are allocated.",
+	"",
+	"-b\tSet byte as unit",
+	"-t\tShow data in tabular format",
+	NULL
+};
+
+int cmd_filesystem_disk_usage(int argc, char **argv)
+{
+
+	int	flags =	DF_HUMAN_UNIT;
+	int	i, more_than_one = 0;
+	int	tabular = 0;
+
+	optind = 1;
+	while (1) {
+		char	c = getopt(argc, argv, "bt");
+		if (c < 0)
+			break;
+		switch (c) {
+		case 'b':
+			flags &= ~DF_HUMAN_UNIT;
+			break;
+		case 't':
+			tabular = 1;
+			break;
+		default:
+			usage(cmd_filesystem_disk_usage_usage);
+		}
+	}
+
+	if (check_argc_min(argc - optind, 1)) {
+		usage(cmd_filesystem_disk_usage_usage);
+		return 21;
+	}
+
+	for (i = optind; i < argc ; i++) {
+		int r, fd;
+		DIR	*dirstream = NULL;
+		if (more_than_one)
+			printf("\n");
+
+		fd = open_file_or_dir(argv[i], &dirstream);
+		if (fd < 0) {
+			fprintf(stderr, "ERROR: can't access to '%s'\n",
+				argv[1]);
+			return 12;
+		}
+		r = _cmd_filesystem_disk_usage(fd, argv[i], flags, tabular);
+		close_file_or_dir(fd, dirstream);
+
+		if (r)
+			return r;
+		more_than_one = 1;
+
+	}
+
+	return 0;
+}
cmds-fi-disk_usage.h +3 -0
index 9f68bb3..c7459b1 100644
@@ -22,4 +22,7 @@
 extern const char * const cmd_filesystem_df_usage[];
 int cmd_filesystem_df(int argc, char **argv);
 
+extern const char * const cmd_filesystem_disk_usage_usage[];
+int cmd_filesystem_disk_usage(int argc, char **argv);
+
 #endif
cmds-filesystem.c +3 -0
index fc85eef..d4cab63 100644
@@ -798,6 +798,9 @@ const struct cmd_group filesystem_cmd_group = {
 		{ "balance", cmd_balance, NULL, &balance_cmd_group, 1 },
 		{ "resize", cmd_resize, cmd_resize_usage, NULL, 0 },
 		{ "label", cmd_label, cmd_label_usage, NULL, 0 },
+		{ "disk-usage", cmd_filesystem_disk_usage,
+			cmd_filesystem_disk_usage_usage, NULL, 0 },
+
 		NULL_CMD_STRUCT
 	}
 };
utils.c +63 -0

66 lines not on this page · raw message

utils.h +3 -0
index 58fb4a7..8469d4a 100644
@@ -99,4 +99,7 @@ int get_btrfs_mount(const char *dev, char *mp, size_t mp_size);
 u64 disk_size(char *path);
 int get_device_info(int fd, u64 devid,
 		struct btrfs_ioctl_dev_info_args *di_args);
+u64 get_partition_size(char *dev);
+const char * group_type_str(u64 flags);
+const char * group_profile_str(u64 flags);
 #endif
Signature · 7 lines
1.8.5.3



-- 
gpg @keyserver.linux.it: Goffredo Baroncelli (kreijackATinwind.it>
Key fingerprint BBF5 1610 0B64 DAC6 5F7D  17B2 0EDA 9B37 8B82 E0B5
#18 lore raw

PATCH 6/8 Create entry in man page for btrfs filesystem disk-usage

In reply to Goffredo Baroncelli
---
 man/btrfs.8.in | 14 ++++++++++++++
 1 file changed, 14 insertions(+)
man/btrfs.8.in +14 -0
index 4acf16d..4b7e771 100644
@@ -35,6 +35,9 @@ btrfs \- control a btrfs filesystem
 .PP
 \fBbtrfs\fP \fBfilesystem label\fP [\fI<device>\fP|\fI<mount_point>\fP] [\fI<newlabel>\fP]
 .PP
+\fBbtrfs\fP \fBfilesystem filesystem disk-usage [-t][-b]\fP\fI <path> 
+[path..]\fP
+.PP
 \fBbtrfs\fP \fBfilesystem df\fP\fI [-b] \fIpath [path..]\fR\fP
 .PP
 \fBbtrfs\fP \fBfilesystem balance\fP\fI <path> \fP
@@ -341,6 +344,17 @@ the same starting disk cylinder as before.
 \fBfilesystem label\fP [\fI<dev>\fP|\fI<mount_point>\fP] [\fInewlabel\fP]\fP
 Show or update the label of a filesystem. \fI[<device>|<mountpoint>]\fR is used
 to identify the filesystem. 
+.TP
+\fBfilesystem disk-usage\fP [-t][-b] \fIpath [path..]\fR
+
+Show in which disk the chunks are allocated.
+
+\fB-b\fP Set byte as unit
+
+\fB-t\fP Show data in tabular format
+
+.TP
+
 If a \fInewlabel\fR optional argument is passed, the label is changed. The
 following constraints exist for a label:
 .IP
Signature · 7 lines
1.8.5.3



-- 
gpg @keyserver.linux.it: Goffredo Baroncelli (kreijackATinwind.it>
Key fingerprint BBF5 1610 0B64 DAC6 5F7D  17B2 0EDA 9B37 8B82 E0B5
#19 lore raw

PATCH 7/8 Add btrfs device disk-usage command

In reply to Goffredo Baroncelli
---
 cmds-device.c        |   3 ++
 cmds-fi-disk_usage.c | 136 +++++++++++++++++++++++++++++++++++++++++++++++++++
 cmds-fi-disk_usage.h |   3 ++
 3 files changed, 142 insertions(+)
cmds-device.c +3 -0
index bc4a8dc..f25dbfa 100644
@@ -28,6 +28,7 @@
 #include "ctree.h"
 #include "ioctl.h"
 #include "utils.h"
+#include "cmds-fi-disk_usage.h"
 
 #include "commands.h"
 
@@ -401,6 +402,8 @@ const struct cmd_group device_cmd_group = {
 		{ "scan", cmd_scan_dev, cmd_scan_dev_usage, NULL, 0 },
 		{ "ready", cmd_ready_dev, cmd_ready_dev_usage, NULL, 0 },
 		{ "stats", cmd_dev_stats, cmd_dev_stats_usage, NULL, 0 },
+		{ "disk-usage", cmd_device_disk_usage,
+			cmd_device_disk_usage_usage, NULL, 0 },
 		NULL_CMD_STRUCT
 	}
 };
cmds-fi-disk_usage.c +136 -0

139 lines not on this page · raw message

cmds-fi-disk_usage.h +3 -0
index c7459b1..c315004 100644
@@ -25,4 +25,7 @@ int cmd_filesystem_df(int argc, char **argv);
 extern const char * const cmd_filesystem_disk_usage_usage[];
 int cmd_filesystem_disk_usage(int argc, char **argv);
 
+extern const char * const cmd_device_disk_usage_usage[];
+int cmd_device_disk_usage(int argc, char **argv);
+
 #endif
Signature · 7 lines
1.8.5.3



-- 
gpg @keyserver.linux.it: Goffredo Baroncelli (kreijackATinwind.it>
Key fingerprint BBF5 1610 0B64 DAC6 5F7D  17B2 0EDA 9B37 8B82 E0B5
#22 lore raw

PATCH 8/8 Create a new entry in btrfs man page for btrfs device disk-usage.

In reply to Goffredo Baroncelli
---
 man/btrfs.8.in | 11 +++++++++++
 1 file changed, 11 insertions(+)
man/btrfs.8.in +11 -0
index 4b7e771..0984504 100644
@@ -59,6 +59,9 @@ btrfs \- control a btrfs filesystem
 .PP
 \fBbtrfs\fP \fBdevice scan\fP [\fI--all-devices\fP|\fI<device> \P[\fI<device>...\fP]
 .PP
+\fBbtrfs\fP \fBdevice disk-usage\fP\fI [-b] <path> [<path>...] \fP
+.PP
+.PP
 \fBbtrfs\fP \fBdevice ready\fP\fI <device>\fP
 .PP
 \fBbtrfs\fP \fBdevice stats\fP [-z] {\fI<path>\fP|\fI<device>\fP}
@@ -474,6 +477,14 @@ Remove device(s) from a filesystem identified by \fI<path>\fR.
 .TP
 
 \fBdevice scan\fR [--all-devices|\fI<device> \fP[\fI<device>...\fP]\fR
+.TP
+
+\fBdevice disk-usage\fR\fI [-b] <path> [<path>..] <path>\fR
+Show which chunks are in a device.
+
+\fB-b\fP set byte as unit.
+.TP
+
 If one or more devices are passed, these are scanned for a btrfs filesystem. 
 If no devices are passed, \fBbtrfs\fR uses block devices containing btrfs
 filesystem as listed by blkid.
Signature · 7 lines
1.8.5.3



-- 
gpg @keyserver.linux.it: Goffredo Baroncelli (kreijackATinwind.it>
Key fingerprint BBF5 1610 0B64 DAC6 5F7D  17B2 0EDA 9B37 8B82 E0B5
#20 lore raw

Re: [PATCH 7/8] Add btrfs device disk-usage command

In reply to Goffredo Baroncelli
Quoted 21 lines · On Thu, 13 Feb 2014 20:20:12 +0100 Goffredo Baroncelli <[email protected]>…
---
 cmds-device.c        |   3 ++
 cmds-fi-disk_usage.c | 136 +++++++++++++++++++++++++++++++++++++++++++++++++++
 cmds-fi-disk_usage.h |   3 ++
 3 files changed, 142 insertions(+)
cmds-device.c +1 -0
index bc4a8dc..f25dbfa 100644
@@ -28,6 +28,7 @@
 #include "ctree.h"
 #include "ioctl.h"
 #include "utils.h"
+#include "cmds-fi-disk_usage.h"
 
 #include "commands.h"
 

cmds-device.c const struct cmd_group device_cmd_group = { · line 407

Quoted 6 lines
cmds-device.c +2 -0
@@ -401,6 +402,8 @@ const struct cmd_group device_cmd_group = {
 		{ "scan", cmd_scan_dev, cmd_scan_dev_usage, NULL, 0 },
 		{ "ready", cmd_ready_dev, cmd_ready_dev_usage, NULL, 0 },
 		{ "stats", cmd_dev_stats, cmd_dev_stats_usage, NULL, 0 },
+		{ "disk-usage", cmd_device_disk_usage,
+			cmd_device_disk_usage_usage, NULL, 0 },
 		NULL_CMD_STRUCT

Why not just name it "du"? Think about the current handy feature of shortening commands by only specifying parts of the keywords. "btrfs dev du" would seem more intuitive than calling it as "btrfs dev disk" or "btrfs dev disk-u".

Or simply "usage", because "...device disk..." is redundant, device is already a disk, so it can easily be "btrfs device usage".

Signature · 2 lines
With respect,
Roman
#7 lore raw

Re: [PATCH 5/8] Add command btrfs filesystem disk-usage

In reply to Goffredo Baroncelli
Quoted 480 lines · On Thu, 13 Feb 2014 20:19:50 +0100 Goffredo Baroncelli <[email protected]>…
---
 cmds-fi-disk_usage.c | 428 +++++++++++++++++++++++++++++++++++++++++++++++++++
 cmds-fi-disk_usage.h |   3 +
 cmds-filesystem.c    |   3 +
 utils.c              |  63 ++++++++
 utils.h              |   3 +
 5 files changed, 500 insertions(+)
cmds-fi-disk_usage.c +428 -0
index 4012c78..16b3ab2 100644
@@ -20,10 +20,12 @@
 #include <unistd.h>
 #include <sys/ioctl.h>
 #include <errno.h>
+#include <stdarg.h>
 
 #include "utils.h"
 #include "kerncompat.h"
 #include "ctree.h"
+#include "string_table.h"
 
 #include "commands.h"
 
@@ -44,6 +46,13 @@ struct chunk_info {
 	u64	num_stripes;
 };
 
+/* to store information about the disks */
+struct disk_info {
+	u64	devid;
+	char	path[BTRFS_DEVICE_PATH_NAME_MAX];
+	u64	size;
+};
+
 /*
  * Pretty print the size
  * PAY ATTENTION: it return a statically buffer
@@ -514,3 +523,422 @@ int cmd_filesystem_df(int argc, char **argv)
 	return 0;
 }
 
+/*
+ *  Helper to sort the disk_info structure
+ */
+static int cmp_disk_info(const void *a, const void *b)
+{
+	return strcmp(((struct disk_info *)a)->path,
+			((struct disk_info *)b)->path);
+}
+
+/*
+ *  This function load the disk_info structure and put them in an array
+ */
+static int load_disks_info(int fd,
+			   struct disk_info **disks_info_ptr,
+			   int *disks_info_count)
+{
+
+	int ret, i, ndevs;
+	struct btrfs_ioctl_fs_info_args fi_args;
+	struct btrfs_ioctl_dev_info_args dev_info;
+	struct disk_info *info;
+
+	*disks_info_count = 0;
+	*disks_info_ptr = 0;
+
+	ret = ioctl(fd, BTRFS_IOC_FS_INFO, &fi_args);
+	if (ret < 0) {
+		fprintf(stderr, "ERROR: cannot get filesystem info\n");
+		return -1;
+	}
+
+	info = malloc(sizeof(struct disk_info) * fi_args.num_devices);
+	if (!info) {
+		fprintf(stderr, "ERROR: not enough memory\n");
+		return -1;
+	}
+
+	for (i = 0, ndevs = 0 ; i <= fi_args.max_id ; i++) {
+
+		BUG_ON(ndevs >= fi_args.num_devices);
+		ret = get_device_info(fd, i, &dev_info);
+
+		if (ret == -ENODEV)
+			continue;
+		if (ret) {
+			fprintf(stderr,
+			    "ERROR: cannot get info about device devid=%d\n",
+			    i);
+			free(info);
+			return -1;
+		}
+
+		info[ndevs].devid = dev_info.devid;
+		strcpy(info[ndevs].path, (char *)dev_info.path);
+		info[ndevs].size = get_partition_size((char *)dev_info.path);
+		++ndevs;
+	}
+
+	BUG_ON(ndevs != fi_args.num_devices);
+	qsort(info, fi_args.num_devices,
+		sizeof(struct disk_info), cmp_disk_info);
+
+	*disks_info_count = fi_args.num_devices;
+	*disks_info_ptr = info;
+
+	return 0;
+
+}
+
+/*
+ *  This function computes the size of a chunk in a disk
+ */
+static u64 calc_chunk_size(struct chunk_info *ci)
+{
+	if (ci->type & BTRFS_BLOCK_GROUP_RAID0)
+		return ci->size / ci->num_stripes;
+	else if (ci->type & BTRFS_BLOCK_GROUP_RAID1)
+		return ci->size ;
+	else if (ci->type & BTRFS_BLOCK_GROUP_DUP)
+		return ci->size ;
+	else if (ci->type & BTRFS_BLOCK_GROUP_RAID5)
+		return ci->size / (ci->num_stripes -1);
+	else if (ci->type & BTRFS_BLOCK_GROUP_RAID6)
+		return ci->size / (ci->num_stripes -2);
+	else if (ci->type & BTRFS_BLOCK_GROUP_RAID10)
+		return ci->size / ci->num_stripes;
+	return ci->size;
+}
+
+/*
+ *  This function print the results of the command btrfs fi disk-usage
+ *  in tabular format
+ */
+static void _cmd_filesystem_disk_usage_tabular(int mode,
+					struct btrfs_ioctl_space_args *sargs,
+					struct chunk_info *chunks_info_ptr,
+					int chunks_info_count,
+					struct disk_info *disks_info_ptr,
+					int disks_info_count)
+{
+	int i;
+	u64 total_unused = 0;
+	struct string_table *matrix = 0;
+	int  ncols, nrows;
+
+	ncols = sargs->total_spaces + 2;
+	nrows = 2 + 1 + disks_info_count + 1 + 2;
+
+	matrix = table_create(ncols, nrows);
+	if (!matrix) {
+		fprintf(stderr, "ERROR: not enough memory\n");
+		return;
+	}
+
+	/* header */
+	for (i = 0; i < sargs->total_spaces; i++) {
+		const char *description;
+
+		u64 flags = sargs->spaces[i].flags;
+		description = group_type_str(flags);
+
+		table_printf(matrix, 1+i, 0, "<%s", description);
+	}
+
+	for (i = 0; i < sargs->total_spaces; i++) {
+		const char *r_mode;
+
+		u64 flags = sargs->spaces[i].flags;
+		r_mode = group_profile_str(flags);
+
+		table_printf(matrix, 1+i, 1, "<%s", r_mode);
+	}
+
+	table_printf(matrix, 1+sargs->total_spaces, 1, "<Unallocated");
+
+	/* body */
+	for (i = 0 ; i < disks_info_count ; i++) {
+		int k, col;
+		char *p;
+
+		u64  total_allocated = 0, unused;
+
+		p = strrchr(disks_info_ptr[i].path, '/');
+		if (!p)
+			p = disks_info_ptr[i].path;
+		else
+			p++;
+
+		table_printf(matrix, 0, i+3, "<%s",
+				disks_info_ptr[i].path);
+
+		for (col = 1, k = 0 ; k < sargs->total_spaces ; k++)  {
+			u64	flags = sargs->spaces[k].flags;
+			u64 devid = disks_info_ptr[i].devid;
+			int	j;
+			u64 size = 0;
+
+			for (j = 0 ; j < chunks_info_count ; j++) {
+				if (chunks_info_ptr[j].type != flags )
+						continue;
+				if (chunks_info_ptr[j].devid != devid)
+						continue;
+
+				size += calc_chunk_size(chunks_info_ptr+j);
+			}
+
+			if (size)
+				table_printf(matrix, col, i+3,
+					">%s", df_pretty_sizes(size, mode));
+			else
+				table_printf(matrix, col, i+3, ">-");
+
+			total_allocated += size;
+			col++;
+		}
+
+		unused = get_partition_size(disks_info_ptr[i].path) -
+				total_allocated;
+
+		table_printf(matrix, sargs->total_spaces + 1, i + 3,
+			       ">%s", df_pretty_sizes(unused, mode));
+		total_unused += unused;
+
+	}
+
+	for (i = 0; i <= sargs->total_spaces; i++)
+		table_printf(matrix, i + 1, disks_info_count + 3, "=");
+
+
+	/* footer */
+	table_printf(matrix, 0, disks_info_count + 4, "<Total");
+	for (i = 0; i < sargs->total_spaces; i++)
+		table_printf(matrix, 1 + i, disks_info_count + 4,
+			">%s",
+			df_pretty_sizes(sargs->spaces[i].total_bytes, mode));
+
+	table_printf(matrix, sargs->total_spaces+1, disks_info_count+4,
+		">%s", df_pretty_sizes(total_unused, mode));
+
+	table_printf(matrix, 0, disks_info_count+5, "<Used");
+	for (i = 0; i < sargs->total_spaces; i++)
+		table_printf(matrix, 1+i, disks_info_count+5, ">%s",
+			df_pretty_sizes(sargs->spaces[i].used_bytes, mode));
+
+
+	table_dump(matrix);
+	table_free(matrix);
+
+}
+
+/*
+ *  This function prints the unused space per every disk
+ */
+static void print_unused(struct chunk_info *info_ptr,
+			  int info_count,
+			  struct disk_info *disks_info_ptr,
+			  int disks_info_count,
+			  int mode)
+{
+	int i;
+	for (i = 0 ; i < disks_info_count ; i++) {
+
+		int	j;
+		u64	total = 0;
+
+		for (j = 0 ; j < info_count ; j++)
+			if (info_ptr[j].devid == disks_info_ptr[i].devid)
+				total += calc_chunk_size(info_ptr+j);
+
+		printf("   %s\t%10s\n",
+			disks_info_ptr[i].path,
+			df_pretty_sizes(disks_info_ptr[i].size - total, mode));
+
+	}
+
+}
+
+/*
+ *  This function prints the allocated chunk per every disk
+ */
+static void print_chunk_disks(u64 chunk_type,
+				struct chunk_info *chunks_info_ptr,
+				int chunks_info_count,
+				struct disk_info *disks_info_ptr,
+				int disks_info_count,
+				int mode)
+{
+	int i;
+
+	for (i = 0 ; i < disks_info_count ; i++) {
+
+		int	j;
+		u64	total = 0;
+
+		for (j = 0 ; j < chunks_info_count ; j++) {
+
+			if (chunks_info_ptr[j].type != chunk_type)
+				continue;
+			if (chunks_info_ptr[j].devid != disks_info_ptr[i].devid)
+				continue;
+
+			total += calc_chunk_size(&(chunks_info_ptr[j]));
+			//total += chunks_info_ptr[j].size;
+		}
+
+		if (total > 0)
+			printf("   %s\t%10s\n",
+				disks_info_ptr[i].path,
+				df_pretty_sizes(total, mode));
+	}
+}
+
+/*
+ *  This function print the results of the command btrfs fi disk-usage
+ *  in linear format
+ */
+static void _cmd_filesystem_disk_usage_linear(int mode,
+					struct btrfs_ioctl_space_args *sargs,
+					struct chunk_info *info_ptr,
+					int info_count,
+					struct disk_info *disks_info_ptr,
+					int disks_info_count)
+{
+	int i;
+
+	for (i = 0; i < sargs->total_spaces; i++) {
+		const char *description;
+		const char *r_mode;
+
+		u64 flags = sargs->spaces[i].flags;
+		description= group_type_str(flags);
+		r_mode = group_profile_str(flags);
+
+		printf("%s,%s: Size:%s, ",
+			description,
+			r_mode,
+			df_pretty_sizes(sargs->spaces[i].total_bytes ,
+			    mode));
+		printf("Used:%s\n",
+			df_pretty_sizes(sargs->spaces[i].used_bytes,
+					mode));
+		print_chunk_disks(flags, info_ptr, info_count,
+				disks_info_ptr, disks_info_count,
+				mode);
+		printf("\n");
+
+	}
+
+	printf("Unallocated:\n");
+	print_unused(info_ptr, info_count,
+			disks_info_ptr, disks_info_count,
+			mode);
+
+
+
+}
+
+static int _cmd_filesystem_disk_usage(int fd, char *path, int mode, int tabular)
+{
+	struct btrfs_ioctl_space_args *sargs = 0;
+	int info_count = 0;
+	struct chunk_info *info_ptr = 0;
+	struct disk_info *disks_info_ptr = 0;
+	int disks_info_count = 0;
+	int ret = 0;
+
+	if (load_chunk_info(fd, &info_ptr, &info_count) ||
+	    load_disks_info(fd, &disks_info_ptr, &disks_info_count)) {
+		ret = -1;
+		goto exit;
+	}
+
+	if ((sargs = load_space_info(fd, path)) == NULL) {
+		ret = -1;
+		goto exit;
+	}
+
+	if (tabular)
+		_cmd_filesystem_disk_usage_tabular(mode, sargs,
+					info_ptr, info_count,
+					disks_info_ptr, disks_info_count);
+	else
+		_cmd_filesystem_disk_usage_linear(mode, sargs,
+					info_ptr, info_count,
+					disks_info_ptr, disks_info_count);
+
+exit:
+
+	if (sargs)
+		free(sargs);
+	if (disks_info_ptr)
+		free(disks_info_ptr);
+	if (info_ptr)
+		free(info_ptr);
+
+	return ret;
+}
+
+const char * const cmd_filesystem_disk_usage_usage[] = {
+	"btrfs filesystem disk-usage [-b][-t] <path> [<path>..]",
+	"Show in which disk the chunks are allocated.",
+	"",
+	"-b\tSet byte as unit",
+	"-t\tShow data in tabular format",
+	NULL
+};
+
+int cmd_filesystem_disk_usage(int argc, char **argv)
+{
+
+	int	flags =	DF_HUMAN_UNIT;
+	int	i, more_than_one = 0;
+	int	tabular = 0;
+
+	optind = 1;
+	while (1) {
+		char	c = getopt(argc, argv, "bt");
+		if (c < 0)
+			break;
+		switch (c) {
+		case 'b':
+			flags &= ~DF_HUMAN_UNIT;
+			break;
+		case 't':
+			tabular = 1;
+			break;
+		default:
+			usage(cmd_filesystem_disk_usage_usage);
+		}
+	}
+
+	if (check_argc_min(argc - optind, 1)) {
+		usage(cmd_filesystem_disk_usage_usage);
+		return 21;
+	}
+
+	for (i = optind; i < argc ; i++) {
+		int r, fd;
+		DIR	*dirstream = NULL;
+		if (more_than_one)
+			printf("\n");
+
+		fd = open_file_or_dir(argv[i], &dirstream);
+		if (fd < 0) {
+			fprintf(stderr, "ERROR: can't access to '%s'\n",
+				argv[1]);
+			return 12;
+		}
+		r = _cmd_filesystem_disk_usage(fd, argv[i], flags, tabular);
+		close_file_or_dir(fd, dirstream);
+
+		if (r)
+			return r;
+		more_than_one = 1;
+
+	}
+
+	return 0;
+}
cmds-fi-disk_usage.h +3 -0
index 9f68bb3..c7459b1 100644
@@ -22,4 +22,7 @@
 extern const char * const cmd_filesystem_df_usage[];
 int cmd_filesystem_df(int argc, char **argv);
 
+extern const char * const cmd_filesystem_disk_usage_usage[];
+int cmd_filesystem_disk_usage(int argc, char **argv);
+
 #endif
cmds-filesystem.c +0 -0
index fc85eef..d4cab63 100644

cmds-filesystem.c const struct cmd_group filesystem_cmd_group = { · line 803

Quoted 6 lines
cmds-filesystem.c +3 -0
@@ -798,6 +798,9 @@ const struct cmd_group filesystem_cmd_group = {
 		{ "balance", cmd_balance, NULL, &balance_cmd_group, 1 },
 		{ "resize", cmd_resize, cmd_resize_usage, NULL, 0 },
 		{ "label", cmd_label, cmd_label_usage, NULL, 0 },
+		{ "disk-usage", cmd_filesystem_disk_usage,
+			cmd_filesystem_disk_usage_usage, NULL, 0 },
+

Same here, I'd suggest "du" or "usage".
As currently in the patch it's not shorthand-friendly, you basically propose that people wishing a shorter invocation use it as "btrfs fi di" or "btrfs fi disk", which seems awkward and not intuitive.

Signature · 2 lines
With respect,
Roman
#21 lore raw

Re: [PATCH 7/8] Add btrfs device disk-usage command

In reply to Roman Mamedov

Hi Roman,

Quoted 6 lines · Roman Mamedov
Quoted 3 lines · On Thu, 13 Feb 2014 20:20:12 +0100 Goffredo Baroncelli <[email protected]>…

---

[...]

Quoted 7 lines

Why not just name it "du"? Think about the current handy feature of shortening commands by only specifying parts of the keywords. "btrfs dev du" would seem more intuitive than calling it as "btrfs dev disk" or "btrfs dev disk-u".

Or simply "usage", because "...device disk..." is redundant, device is already a disk, so it can easily be "btrfs device usage".

Yes, I agree about "usage"
GB

Signature · 2 lines
gpg @keyserver.linux.it: Goffredo Baroncelli (kreijackATinwind.it>
Key fingerprint BBF5 1610 0B64 DAC6 5F7D  17B2 0EDA 9B37 8B82 E0B5
#8 lore raw

Re: [PATCH 5/8] Add command btrfs filesystem disk-usage

In reply to Roman Mamedov

Hi Roman

Quoted 3 lines · Roman Mamedov

On Thu, 13 Feb 2014 20:19:50 +0100
Goffredo Baroncelli <[email protected]> wrote:

[...]

Quoted 8 lines
Quoted 8 lines
cmds-filesystem.c +0 -0
index fc85eef..d4cab63 100644
@@ -798,6 +798,9 @@ const struct cmd_group filesystem_cmd_group = {
 		{ "balance", cmd_balance, NULL, &balance_cmd_group, 1 },
 		{ "resize", cmd_resize, cmd_resize_usage, NULL, 0 },
 		{ "label", cmd_label, cmd_label_usage, NULL, 0 },
Quoted 8 lines

cmds-filesystem.c const struct cmd_group filesystem_cmd_group = { · line 803

Quoted 3 lines
cmds-filesystem.c +3 -0
+		{ "disk-usage", cmd_filesystem_disk_usage,
+			cmd_filesystem_disk_usage_usage, NULL, 0 },
+

Same here, I'd suggest "du" or "usage".
As currently in the patch it's not shorthand-friendly, you basically propose that people wishing a shorter invocation use it as "btrfs fi di" or "btrfs fi disk", which seems awkward and not intuitive.

Thanks for the comments, however I don't like du not usage; but you are right
when you don't like "disk-usage". What about "btrfs filesystem chunk-usage" ? The short form would be

btrfs fi chunk

which is coherent with the data displayed

Regards
Goffredo

P.S.
Please the next time, cut the email lines which are not relevant to your replay

Signature · 2 lines
gpg @keyserver.linux.it: Goffredo Baroncelli (kreijackATinwind.it>
Key fingerprint BBF5 1610 0B64 DAC6 5F7D  17B2 0EDA 9B37 8B82 E0B5
#9 lore raw

Re: [PATCH 5/8] Add command btrfs filesystem disk-usage

Goffredo Baroncelli posted on Thu, 13 Feb 2014 20:49:08 +0100 as excerpted:

Quoted 7 lines

Thanks for the comments, however I don't like du not usage; but you are right when you don't like "disk-usage". What about "btrfs filesystem chunk-usage" ? The short form would be

btrfs fi chunk

which is coherent with the data displayed

That makes much more sense from a different perspective as well. "Disk" sounds so 20th-century spinning rust..., and at least to me, "du" isn't as intuitive or commonly used as "df", so while "df" seems natural to me, while I understand where "du" comes from, it still makes me go "huh?" as I wouldn't expect to see it in the btrfs context.

So btrfs filesystem chunk(-usage), please. =:^)

Signature · 3 lines
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman
#10 lore raw

Re: [PATCH 5/8] Add command btrfs filesystem disk-usage

In reply to Goffredo Baroncelli
Quoted 2 lines · On Thu, 13 Feb 2014 20:49:08 +0100 Goffredo Baroncelli <[email protected]>…

Thanks for the comments, however I don't like du not usage; but you are right when you don't like "disk-usage". What about "btrfs filesystem chunk-usage" ?

Personally I don't see the point of being super-pedantic here, i.e. "look this is not just filesystem usage, this is filesystem CHUNK usage"... Consistency of having a matching "dev usage" and "fi usage" would have been nicer.

Signature · 2 lines
With respect,
Roman
#11 lore raw

Re: [PATCH 5/8] Add command btrfs filesystem disk-usage

Quoted 9 lines · Roman Mamedov
Quoted 2 lines · On Thu, 13 Feb 2014 20:49:08 +0100 Goffredo Baroncelli <[email protected]>…

Thanks for the comments, however I don't like du not usage; but you are right when you don't like "disk-usage". What about "btrfs filesystem chunk-usage" ?

Personally I don't see the point of being super-pedantic here, i.e. "look this is not just filesystem usage, this is filesystem CHUNK usage"... Consistency of having a matching "dev usage" and "fi usage" would have been nicer.

What about "btrfs filesystem chunk-usage" ?

Signature · 2 lines
gpg @keyserver.linux.it: Goffredo Baroncelli (kreijackATinwind.it>
Key fingerprint BBF5 1610 0B64 DAC6 5F7D  17B2 0EDA 9B37 8B82 E0B5
#12 lore raw

Re: [PATCH 5/8] Add command btrfs filesystem disk-usage

Quoted 7 lines · On Fri, 14 Feb 2014 18:57:03 +0100 Goffredo Baroncelli <[email protected]>…
Quoted 6 lines · Roman Mamedov
Quoted 2 lines · On Thu, 13 Feb 2014 20:49:08 +0100 Goffredo Baroncelli <[email protected]>…

Thanks for the comments, however I don't like du not usage; but you are right when you don't like "disk-usage". What about "btrfs filesystem chunk-usage" ?

Quoted 6 lines
Quoted 3 lines

Personally I don't see the point of being super-pedantic here, i.e. "look this is not just filesystem usage, this is filesystem CHUNK usage"... Consistency of having a matching "dev usage" and "fi usage" would have been nicer.

What about "btrfs filesystem chunk-usage" ?

Uhm? Had to reread this several times, but it looks like you're repeating exactly the same question that I was already answering in the quoted part.

To clarify even more, personally I'd like if there would have been "btrfs dev usage" and "btrfs fi usage". Do not see the need to specifically make the 2nd one "chunk-usage" instead of simply "usage".

Signature · 2 lines
With respect,
Roman
#13 lore raw

Re: [PATCH 5/8] Add command btrfs filesystem disk-usage

Quoted 15 lines · Roman Mamedov
Quoted 12 lines · On Fri, 14 Feb 2014 18:57:03 +0100 Goffredo Baroncelli <[email protected]>…
Quoted 9 lines · Roman Mamedov
Quoted 2 lines · On Thu, 13 Feb 2014 20:49:08 +0100 Goffredo Baroncelli <[email protected]>…

Thanks for the comments, however I don't like du not usage; but you are right when you don't like "disk-usage". What about "btrfs filesystem chunk-usage" ?

Personally I don't see the point of being super-pedantic here, i.e. "look this is not just filesystem usage, this is filesystem CHUNK usage"... Consistency of having a matching "dev usage" and "fi usage" would have been nicer.

Quoted 8 lines
Quoted 1 line

What about "btrfs filesystem chunk-usage" ?

Uhm? Had to reread this several times, but it looks like you're repeating exactly the same question that I was already answering in the quoted part.

To clarify even more, personally I'd like if there would have been "btrfs dev usage" and "btrfs fi usage". Do not see the need to specifically make the 2nd one "chunk-usage" instead of simply "usage".

I don't like "usage" because it to me seems to be too much generic. Because both "btrfs filesystem disk-usage" and "btrfs device disk-usage" report about chunk (and/or block group) infos, I am investigating about
- btrfs filesystem chunk-usage
- btrfs device chunk-usage

Regards
GB

Signature · 2 lines
gpg @keyserver.linux.it: Goffredo Baroncelli (kreijackATinwind.it>
Key fingerprint BBF5 1610 0B64 DAC6 5F7D  17B2 0EDA 9B37 8B82 E0B5
#14 lore raw

Re: [PATCH 5/8] Add command btrfs filesystem disk-usage

Quoted 25 lines · Goffredo Baroncelli
Quoted 15 lines · Roman Mamedov
Quoted 12 lines · On Fri, 14 Feb 2014 18:57:03 +0100 Goffredo Baroncelli <[email protected]>…
Quoted 9 lines · Roman Mamedov
Quoted 2 lines · On Thu, 13 Feb 2014 20:49:08 +0100 Goffredo Baroncelli <[email protected]>…

Thanks for the comments, however I don't like du not usage; but you are right when you don't like "disk-usage". What about "btrfs filesystem chunk-usage" ?

Personally I don't see the point of being super-pedantic here, i.e. "look this is not just filesystem usage, this is filesystem CHUNK usage"... Consistency of having a matching "dev usage" and "fi usage" would have been nicer.

Quoted 8 lines
Quoted 1 line

What about "btrfs filesystem chunk-usage" ?

Uhm? Had to reread this several times, but it looks like you're repeating exactly the same question that I was already answering in the quoted part.

To clarify even more, personally I'd like if there would have been "btrfs dev usage" and "btrfs fi usage". Do not see the need to specifically make the 2nd one "chunk-usage" instead of simply "usage".

Quoted 6 lines

I don't like "usage" because it to me seems to be too much generic. Because both "btrfs filesystem disk-usage" and "btrfs device disk-usage" report about chunk (and/or block group) infos, I am investigating about
- btrfs filesystem chunk-usage
- btrfs device chunk-usage

Most people aren't going to know (or care) what a chunk is. I'm
much happier with Roman's suggestion of btrfs {fi,dev} usage.

Hugo.

Signature · 3 lines
=== Hugo Mills: hugo@... carfax.org.uk | darksatanic.net | lug.org.uk ===
  PGP key: 65E74AC0 from wwwkeys.eu.pgp.net or http://www.carfax.org.uk
               --- Nostalgia isn't what it used to be. ---               
#15 lore raw

Re: [PATCH 5/8] Add command btrfs filesystem disk-usage

Quoted 33 lines · Hugo Mills
Quoted 25 lines · Goffredo Baroncelli
Quoted 15 lines · Roman Mamedov
Quoted 12 lines · On Fri, 14 Feb 2014 18:57:03 +0100 Goffredo Baroncelli <[email protected]>…
Quoted 9 lines · Roman Mamedov
Quoted 2 lines · On Thu, 13 Feb 2014 20:49:08 +0100 Goffredo Baroncelli <[email protected]>…

Thanks for the comments, however I don't like du not usage; but you are right when you don't like "disk-usage". What about "btrfs filesystem chunk-usage" ?

Personally I don't see the point of being super-pedantic here, i.e. "look this is not just filesystem usage, this is filesystem CHUNK usage"... Consistency of having a matching "dev usage" and "fi usage" would have been nicer.

Quoted 8 lines
Quoted 1 line

What about "btrfs filesystem chunk-usage" ?

Uhm? Had to reread this several times, but it looks like you're repeating exactly the same question that I was already answering in the quoted part.

To clarify even more, personally I'd like if there would have been "btrfs dev usage" and "btrfs fi usage". Do not see the need to specifically make the 2nd one "chunk-usage" instead of simply "usage".

Quoted 6 lines

I don't like "usage" because it to me seems to be too much generic. Because both "btrfs filesystem disk-usage" and "btrfs device disk-usage" report about chunk (and/or block group) infos, I am investigating about
- btrfs filesystem chunk-usage
- btrfs device chunk-usage

Quoted 2 lines

Most people aren't going to know (or care) what a chunk is. I'm
much happier with Roman's suggestion of btrfs {fi,dev} usage.

Or btrfs filesystem examine, or btrfs filesystem detail, which are semi-consistent with mdadm for obtaining similar data.

Chris Murphy

#16 lore raw

Re: [PATCH 5/8] Add command btrfs filesystem disk-usage

Quoted 36 lines · Chris Murphy
Quoted 33 lines · Hugo Mills
Quoted 31 lines · Goffredo Baroncelli
Quoted 15 lines · Roman Mamedov
Quoted 12 lines · On Fri, 14 Feb 2014 18:57:03 +0100 Goffredo Baroncelli <[email protected]>…
Quoted 9 lines · Roman Mamedov
Quoted 2 lines · On Thu, 13 Feb 2014 20:49:08 +0100 Goffredo Baroncelli <[email protected]>…

Thanks for the comments, however I don't like du not usage; but you are right when you don't like "disk-usage". What about "btrfs filesystem chunk-usage" ?

Personally I don't see the point of being super-pedantic here, i.e. "look this is not just filesystem usage, this is filesystem CHUNK usage"... Consistency of having a matching "dev usage" and "fi usage" would have been nicer.

Quoted 8 lines
Quoted 1 line

What about "btrfs filesystem chunk-usage" ?

Uhm? Had to reread this several times, but it looks like you're repeating exactly the same question that I was already answering in the quoted part.

To clarify even more, personally I'd like if there would have been "btrfs dev usage" and "btrfs fi usage". Do not see the need to specifically make the 2nd one "chunk-usage" instead of simply "usage".

I don't like "usage" because it to me seems to be too much generic. Because both "btrfs filesystem disk-usage" and "btrfs device disk-usage" report about chunk (and/or block group) infos, I am investigating about
- btrfs filesystem chunk-usage
- btrfs device chunk-usage

Quoted 6 lines
Quoted 2 lines

Most people aren't going to know (or care) what a chunk is. I'm
much happier with Roman's suggestion of btrfs {fi,dev} usage.

Or btrfs filesystem examine, or btrfs filesystem detail, which are semi-consistent with mdadm for obtaining similar data.

I have to agree with Chris: looking at the output of "btrfs fi disk-usage"

$ sudo ./btrfs filesystem disk-usage -t /mnt/btrfs1/
         Data   Data    Metadata Metadata System System             
         Single RAID6   Single   RAID5    Single RAID5   Unallocated
/dev/vdb 8.00MB  1.00GB   8.00MB   1.00GB 4.00MB  4.00MB     97.98GB
/dev/vdc      -  1.00GB        -   1.00GB      -  4.00MB     98.00GB
/dev/vdd      -  1.00GB        -   1.00GB      -  4.00MB     98.00GB
/dev/vde      -  1.00GB        -   1.00GB      -  4.00MB     98.00GB
         ====== ======= ======== ======== ====== ======= ===========

Total 8.00MB 2.00GB 8.00MB 3.00GB 4.00MB 12.00MB 391.97GB
Used 0.00 11.25MB 0.00 36.00KB 0.00 4.00KB

it is hard to tell that this can be named "filesystem usage". I think that "details" or "examine" is a better name.

Regarding "btrfs device usage", it seems to me more coherent. But as reported before consistency also matters, so now I am inclined to use "detail" (or examine) also for "btrfs device"

Quoted 3 lines

Chris Murphy

Regards
Goffredo

Signature · 2 lines
gpg @keyserver.linux.it: Goffredo Baroncelli (kreijackATinwind.it>
Key fingerprint BBF5 1610 0B64 DAC6 5F7D  17B2 0EDA 9B37 8B82 E0B5
#23 lore raw
In reply to Goffredo Baroncelli
Quoted 3 lines · Goffredo Baroncelli

This is the 4th attempt of my patches related to show how the data are stored in a btrfs filesystem. I rebased all the patches on the v3.13 btrfs-progs.

FYI, I've added this series as-is into the -next part of the integration branch so we have something to test, let the review and comments phase continue.

#24 lore raw
Quoted 3 lines · David Sterba

series as-is into the -next part of the integration
branch so we have something to test, let the review and comments phase continue.

Hi David,

many thanks

BR
G.Baroncelli

Signature · 2 lines
gpg @keyserver.linux.it: Goffredo Baroncelli (kreijackATinwind.it>
Key fingerprint BBF5 1610 0B64 DAC6 5F7D  17B2 0EDA 9B37 8B82 E0B5
#17 lore raw

Re: [PATCH 5/8] Add command btrfs filesystem disk-usage

In reply to Goffredo Baroncelli
Quoted 14 lines · Goffredo Baroncelli

I have to agree with Chris: looking at the output of "btrfs fi disk-usage"

$ sudo ./btrfs filesystem disk-usage -t /mnt/btrfs1/
         Data   Data    Metadata Metadata System System             
         Single RAID6   Single   RAID5    Single RAID5   Unallocated
/dev/vdb 8.00MB  1.00GB   8.00MB   1.00GB 4.00MB  4.00MB     97.98GB
/dev/vdc      -  1.00GB        -   1.00GB      -  4.00MB     98.00GB
/dev/vdd      -  1.00GB        -   1.00GB      -  4.00MB     98.00GB
/dev/vde      -  1.00GB        -   1.00GB      -  4.00MB     98.00GB
         ====== ======= ======== ======== ====== ======= ===========

Total 8.00MB 2.00GB 8.00MB 3.00GB 4.00MB 12.00MB 391.97GB
Used 0.00 11.25MB 0.00 36.00KB 0.00 4.00KB

Quoted 6 lines

it is hard to tell that this can be named "filesystem usage". I think that "details" or "examine" is a better name.

Regarding "btrfs device usage", it seems to me more coherent. But as reported before consistency also matters, so now I am inclined to use "detail" (or examine) also for "btrfs device"

I'm for 'btrfs filesystem usage' and 'btrfs device usage', IMHO intuitive, easy to type and remember.

#25 lore raw
In reply to Goffredo Baroncelli
Quoted 8 lines · Goffredo Baroncelli

space (if the next chunk are allocated as SINGLE) or the minimum one ( if the next chunks are allocated as DUP/RAID1/RAID10).

The other two commands show the chunks in the disks.

$ sudo btrfs filesystem disk-usage /mnt/btrfs1/
Data,Single: Size:8.00MB, Used:0.00
   /dev/vdb	    8.00MB

The information about per-device usage can be enhanced and there's enough space to print that:

* allocated in chunks (the number above)
* actually used (simiar to what 'btrfs fi show' prints as 'used')

I don't see a reason why it would not fit here, nor any other place where this can be obtained.

There is the cumulative number of 'Used' for all devices, but I'd like to see it per-device as well.

Quoted 11 lines

or in tabular format

$ sudo ./btrfs filesystem disk-usage -t /mnt/btrfs1/
         Data   Data    Metadata Metadata System System             
         Single RAID6   Single   RAID5    Single RAID5   Unallocated
/dev/vdb 8.00MB  1.00GB   8.00MB   1.00GB 4.00MB  4.00MB     97.98GB
/dev/vdc      -  1.00GB        -   1.00GB      -  4.00MB     98.00GB
/dev/vdd      -  1.00GB        -   1.00GB      -  4.00MB     98.00GB
/dev/vde      -  1.00GB        -   1.00GB      -  4.00MB     98.00GB
         ====== ======= ======== ======== ====== ======= ===========
Quoted 5 lines

Total 8.00MB 2.00GB 8.00MB 3.00GB 4.00MB 12.00MB 391.97GB
Used 0.00 11.25MB 0.00 36.00KB 0.00 4.00KB

These are the most complete output, where it is possible to know which disk a chunk uses and the usage of every chunk.

Though not per-device, similar to the above, but the tabular output is limited compared to the sequential output. Not sure what to do here.

Quoted 24 lines

Finally the last command shows which chunks a disk hosts:

$ sudo ./btrfs device disk-usage /mnt/btrfs1/
/dev/vdb	  100.00GB
   Data,Single:              8.00MB
   Data,RAID6:               1.00GB
   Metadata,Single:          8.00MB
   Metadata,RAID5:           1.00GB
   System,Single:            4.00MB
   System,RAID5:             4.00MB
   Unallocated:             97.98GB

/dev/vdc 100.00GB

   Data,RAID6:               1.00GB
   Metadata,RAID5:           1.00GB
   System,RAID5:             4.00MB

Unallocated: 98.00GB

/dev/vdd 100.00GB

   Data,RAID6:               1.00GB
   Metadata,RAID5:           1.00GB
   System,RAID5:             4.00MB

Unallocated: 98.00GB

Quoted 7 lines

/dev/vde 100.00GB

   Data,RAID6:               1.00GB
   Metadata,RAID5:           1.00GB
   System,RAID5:             4.00MB

Unallocated: 98.00GB

More or less are the same information above, only grouped by disk.

Ie. it's only a variant of the 'filesystem usage' where it is grouped by blockgroup type.

Why doesn't 'btrfs device usage' take a device instead of the whole filesystem? This seems counterintuitive. It should be possible to ask for a device by it or path.

Also, I'd like to see all useful information about the device:

* id, path, uuid, ... whatever
* physical device size
* size visible by the filesystem
* space allocated in chunks
* space actually used

Quoted 1 line

Unfortunately I don't have any information about the chunk usage per disk basis.

And I'm missing it. Is it a fundamental problem or just not addressed in this patchset?

Quoted 12 lines

Finally I have to point out that the command btrfs fi df previous didn't require the root capability, now with my patches it is required, because I need to know some info about the chunks so I need to use the "BTRFS_IOC_TREE_SEARCH".

I think that there are the following possibilities:
1) accept this regresssion
2) remove the command "btrfs fi df" and leave only "btrfs fi disk-usage" and "btrfs dev disk-usage"
3) adding a new ioctl which could be used without root capability. Of course this ioctl would return only a subset of the BTRFS_IOC_TREE_SEARCH info

Quoted 3 lines

I think that the 3) would be the "long term" solution. I am not happy about the 1), so as "short term solution" I think that we should go with the 2). What do you think ?

No sorry, 1) is not acceptable. We can live with this limitation only during development so we're not blocked by some new ioctl development.

No for 2), 'fi df' is useful as and widely used in existing scripts.

Yes for 3), we may also export the information through the existing ioctls if possible (eg. IOC_FS_INFO).

#5 lore raw

Re: [PATCH 4/8] Allow use of get_device_info()

In reply to Goffredo Baroncelli
Quoted 2 lines · Goffredo Baroncelli

Allow the use of get_device_info() for different units.

FYI, I'll move this patch to integration base, it's a simple and isolated change.

#26 lore raw
In reply to David Sterba

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Quoted 106 lines · David Sterba
Quoted 7 lines · Goffredo Baroncelli

space (if the next chunk are allocated as SINGLE) or the minimum one ( if the next chunks are allocated as DUP/RAID1/RAID10).

The other two commands show the chunks in the disks.

$ sudo btrfs filesystem disk-usage /mnt/btrfs1/ Data,Single:
Size:8.00MB, Used:0.00 /dev/vdb	    8.00MB

The information about per-device usage can be enhanced and there's enough space to print that:

* allocated in chunks (the number above) * actually used (simiar to what 'btrfs fi show' prints as 'used')

I don't see a reason why it would not fit here, nor any other place where this can be obtained.

There is the cumulative number of 'Used' for all devices, but I'd like to see it per-device as well.

Quoted 10 lines

or in tabular format

$ sudo ./btrfs filesystem disk-usage -t /mnt/btrfs1/ Data   Data
Metadata Metadata System System Single RAID6   Single   RAID5
Single RAID5   Unallocated

/dev/vdb 8.00MB 1.00GB 8.00MB 1.00GB 4.00MB 4.00MB

97.98GB /dev/vdc      -  1.00GB        -   1.00GB      -  4.00MB
98.00GB /dev/vdd      -  1.00GB        -   1.00GB      -  4.00MB
98.00GB /dev/vde      -  1.00GB        -   1.00GB      -  4.00MB
Quoted 7 lines

98.00GB ====== ======= ======== ======== ====== =======
=========== Total 8.00MB 2.00GB 8.00MB 3.00GB 4.00MB
12.00MB 391.97GB Used 0.00 11.25MB 0.00 36.00KB
0.00 4.00KB

These are the most complete output, where it is possible to know which disk a chunk uses and the usage of every chunk.

Though not per-device, similar to the above, but the tabular output is limited compared to the sequential output. Not sure what to do here.

Quoted 16 lines

Finally the last command shows which chunks a disk hosts:

$ sudo ./btrfs device disk-usage /mnt/btrfs1/ /dev/vdb
100.00GB Data,Single:              8.00MB Data,RAID6:
1.00GB Metadata,Single:          8.00MB Metadata,RAID5:
1.00GB System,Single:            4.00MB System,RAID5:
4.00MB Unallocated:             97.98GB

/dev/vdc 100.00GB Data,RAID6: 1.00GB
Metadata,RAID5: 1.00GB System,RAID5:
4.00MB Unallocated: 98.00GB

/dev/vdd 100.00GB Data,RAID6: 1.00GB
Metadata,RAID5: 1.00GB System,RAID5:
4.00MB Unallocated: 98.00GB

Quoted 6 lines

/dev/vde 100.00GB Data,RAID6: 1.00GB
Metadata,RAID5: 1.00GB System,RAID5:
4.00MB Unallocated: 98.00GB

More or less are the same information above, only grouped by disk.

Ie. it's only a variant of the 'filesystem usage' where it is grouped by blockgroup type.

Why doesn't 'btrfs device usage' take a device instead of the whole filesystem? This seems counterintuitive. It should be possible to ask for a device by it or path.

Also, I'd like to see all useful information about the device:

* id, path, uuid, ... whatever * physical device size * size visible by the filesystem * space allocated in chunks * space actually used

Quoted 2 lines

Unfortunately I don't have any information about the chunk usage per disk basis.

And I'm missing it. Is it a fundamental problem or just not addressed in this patchset?

Quoted 12 lines

Finally I have to point out that the command btrfs fi df previous didn't require the root capability, now with my patches it is required, because I need to know some info about the chunks so I need to use the "BTRFS_IOC_TREE_SEARCH".

I think that there are the following possibilities: 1) accept this regresssion 2) remove the command "btrfs fi df" and leave only "btrfs fi disk-usage" and "btrfs dev disk-usage" 3) adding a new ioctl which could be used without root capability. Of course this ioctl would return only a subset of the BTRFS_IOC_TREE_SEARCH info

Quoted 3 lines

I think that the 3) would be the "long term" solution. I am not happy about the 1), so as "short term solution" I think that we should go with the 2). What do you think ?

No sorry, 1) is not acceptable. We can live with this limitation only during development so we're not blocked by some new ioctl development.

Quoted 6 lines

No for 2), 'fi df' is useful as and widely used in existing scripts.

Yes for 3), we may also export the information through the existing ioctls if possible (eg. IOC_FS_INFO).

For _right now_ I'd say just not do the raid56 stuff if we don't notice any raid56 chunks from the normal load_space_info, and then if there are raid56 we try and run the tree search ioctl and notice if we get back EPERM or whatever you get when you don't have permissions. Then just spit out as much information that you can about the fs with a little note at the bottom that available calculation isn't 100% and you need to run as root if you want that info.

Then what we could do is add another flag type for the existing SPACE_INFO ioctl to spit out the information you need about the raid5/6 chunks and then just test for those flags and make the adjustment necessary. This way we avoid adding yet another ioctl and stuff will still work nicely for old kernels that don't have the updated ioctl. Thanks,

Josef
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQIcBAEBAgAGBQJTBko8AAoJEANb+wAKly3B8bgP+wU2yWBhtUWE9gFycBeW0gk7 gKexgV/8OV93FQbhQcPburhkzGixS1XMLbVYnJsYfwfRLKZl4l+M2mOJHyDu93M3 HGfSSuKoTJ8cQ8Fa2qdFIbZNFWSBeHpkOSmOl0NxyFLq/uXLaDP+ANRSE94K54d2 jdT9ncKdQcgiqRRaEIwgUkOUKdqRdRytm3BDr2otuvatWeKRtSVsSE7hG9uyxCBP CxwE8SM7I4OByEqfrm7MJ2beQiOJoYIAKfVRjC1h+rUMd6IhjkfQGL1CBBrkB732 JWsOupRPPZkGQLMpD1Cbxy7BebzagKwC7oTubaWr8NnTFTbRX1+URJJFkPSEKmNy 9iYK4P17/w30GIfHJ5G4oEr2dl+41iCCaOJM0G57/yTMn0ostfRtdv3gSXHDHGYr o39sDymTIX+H8w5Jd+ItTFKiP2B8NE9o6k9ZvLCF5gS38SdisyOiMwUu85jiAjHA 7YcGP2+Z2s1OlYJLpNrASt/JoXMtzGnXr7IinuLxjyA9RbnQk6K0Vzoen8eF+mtU Ge/2D5zDdILrdlPwf/gaffNglstH2MgkiY9uTeAxlN90i9zW2BylJW+WR3o3aDq+ ajZfIoerQkk16EUvXxOap+vbkBpOfaEkzfWOOj7JU8CdkMF4L2g0FUUSan9vDc5v meYQ/s2qy0eKDeRTcj3m
=tF0W
-----END PGP SIGNATURE-----

#29 lore raw
In reply to David Sterba

Hi David,

below my comments

Quoted 6 lines · David Sterba
Quoted 5 lines · Goffredo Baroncelli

space (if the next chunk are allocated as SINGLE) or the minimum one ( if the next chunks are allocated as DUP/RAID1/RAID10).

The other two commands show the chunks in the disks.

Quoted 8 lines
Quoted 3 lines
$ sudo btrfs filesystem disk-usage /mnt/btrfs1/
Data,Single: Size:8.00MB, Used:0.00
   /dev/vdb	    8.00MB

The information about per-device usage can be enhanced and there's enough space to print that:

* allocated in chunks (the number above)

This is the value already written.

Quoted 1 line

* actually used (simiar to what 'btrfs fi show' prints as 'used')

The "used" is a value returned by the ioctl BTRFS_IOC_SPACE_INFO, which returns this value per group-block basis. See below.

Quoted 21 lines

I don't see a reason why it would not fit here, nor any other place where this can be obtained.

There is the cumulative number of 'Used' for all devices, but I'd like to see it per-device as well.

Quoted 14 lines

or in tabular format

$ sudo ./btrfs filesystem disk-usage -t /mnt/btrfs1/
         Data   Data    Metadata Metadata System System             
         Single RAID6   Single   RAID5    Single RAID5   Unallocated
/dev/vdb 8.00MB  1.00GB   8.00MB   1.00GB 4.00MB  4.00MB     97.98GB
/dev/vdc      -  1.00GB        -   1.00GB      -  4.00MB     98.00GB
/dev/vdd      -  1.00GB        -   1.00GB      -  4.00MB     98.00GB
/dev/vde      -  1.00GB        -   1.00GB      -  4.00MB     98.00GB
         ====== ======= ======== ======== ====== ======= ===========

Total 8.00MB 2.00GB 8.00MB 3.00GB 4.00MB 12.00MB 391.97GB
Used 0.00 11.25MB 0.00 36.00KB 0.00 4.00KB

Quoted 5 lines
Quoted 2 lines

These are the most complete output, where it is possible to know which disk a chunk uses and the usage of every chunk.

Though not per-device, similar to the above, but the tabular output is limited compared to the sequential output. Not sure what to do here.

The tabular is to have a "friendly" summary of how the filesystem is span on the different disks. I suggest to not add more info in the tabular

Quoted 31 lines
Quoted 30 lines

Finally the last command shows which chunks a disk hosts:

$ sudo ./btrfs device disk-usage /mnt/btrfs1/
/dev/vdb	  100.00GB
   Data,Single:              8.00MB
   Data,RAID6:               1.00GB
   Metadata,Single:          8.00MB
   Metadata,RAID5:           1.00GB
   System,Single:            4.00MB
   System,RAID5:             4.00MB
   Unallocated:             97.98GB

/dev/vdc 100.00GB

   Data,RAID6:               1.00GB
   Metadata,RAID5:           1.00GB
   System,RAID5:             4.00MB

Unallocated: 98.00GB

/dev/vdd 100.00GB

   Data,RAID6:               1.00GB
   Metadata,RAID5:           1.00GB
   System,RAID5:             4.00MB

Unallocated: 98.00GB

/dev/vde 100.00GB

   Data,RAID6:               1.00GB
   Metadata,RAID5:           1.00GB
   System,RAID5:             4.00MB

Unallocated: 98.00GB

Quoted 8 lines
Quoted 1 line

More or less are the same information above, only grouped by disk.

Ie. it's only a variant of the 'filesystem usage' where it is grouped by blockgroup type.

Why doesn't 'btrfs device usage' take a device instead of the whole filesystem? This seems counterintuitive. It should be possible to ask for a device by it or path.

If a device is passed, what you would expect as output: the list of all the devices involved,or only the one(s) passed ? In other terms, the device passed has to act also as filter, or it is only a different way to indicate the path ?

Quoted 4 lines

Also, I'd like to see all useful information about the device:

* id, path, uuid, ... whatever

ok, we can add a "-v" switch to add these further information

Quoted 1 line

* physical device size

ok, it is already written

Quoted 1 line

* size visible by the filesystem

Could you be more explicit ? a) For each chunk (in each disk) or b) for each disk ? And how this information per disk basis would be useful ?

Example (supposing 4 disks)
a)
/dev/vde 100.00GB

    Data,RAID6:               Disk size:  1.00GB, FS size: 512.00MB
    Metadata,RAID5:           Disk size:  1.00GB, FS size: 750.00MB
    System,RAID5:             Disk size:  4.00MB, FS size;   3.00MB
    Unallocated:              Disk size: 98.00GB

b)
/dev/vde Disk size: 100.00GB, FS size: 1.28GB

    Data,RAID6:               1.00GB
    Metadata,RAID5:           1.00GB
    System,RAID5:             4.00MB

Unallocated: 98.00GB

Quoted 1 line

* space allocated in chunks

ok, it is already written

Quoted 1 line

* space actually used

See my other comments

I am not against to add further information to btrfs dev [disk-]usage, but I liked the idea to add the minimum information to avoid ambiguities:
- the number on the left of the device is the device size
- the number on the left of the block group is the space allocated on the disk All the number are in terms of disk space.

Quoted 5 lines
Quoted 2 lines

Unfortunately I don't have any information about the chunk usage per disk basis.

And I'm missing it. Is it a fundamental problem or just not addressed in this patchset?

As above, the "used" is a value returned by the ioctl BTRFS_IOC_SPACE_INFO, which returns this value per group-block basis. I don't know how calculates this values per device basis. If you give me some hints on how we can get these information, I can improve
the output.

Quoted 13 lines
Quoted 13 lines

Finally I have to point out that the command btrfs fi df previous didn't require the root capability, now with my patches it is required, because I need to know some info about the chunks so I need to use the "BTRFS_IOC_TREE_SEARCH".

I think that there are the following possibilities:
1) accept this regresssion
2) remove the command "btrfs fi df" and leave only "btrfs fi disk-usage" and "btrfs dev disk-usage"
3) adding a new ioctl which could be used without root capability. Of course this ioctl would return only a subset of the BTRFS_IOC_TREE_SEARCH info

Quoted 8 lines
Quoted 3 lines

I think that the 3) would be the "long term" solution. I am not happy about the 1), so as "short term solution" I think that we should go with the 2). What do you think ?

No sorry, 1) is not acceptable. We can live with this limitation only during development so we're not blocked by some new ioctl development.

No for 2), 'fi df' is useful as and widely used in existing scripts.

Of course I meant "from my patchset". I would leave the actual "fi df" implementation. Anyway if you are talking about "existing scripts", this means that we cannot touch the actual btrfs fi df at all, but we have to add a another command (btrfs fi free ?)

Quoted 3 lines

Yes for 3), we may also export the information through the existing ioctls if possible (eg. IOC_FS_INFO).

Of course this would be the best. But what if we do "btrfs fi df" in an old kernel ? It is acceptable to fall-back to the old output ?

Quoted 1 line
Signature · 2 lines
gpg @keyserver.linux.it: Goffredo Baroncelli (kreijackATinwind.it>
Key fingerprint BBF5 1610 0B64 DAC6 5F7D  17B2 0EDA 9B37 8B82 E0B5
#27 lore raw
In reply to Josef Bacik

Hi Josef,

Quoted 11 lines · Josef Bacik
Quoted 3 lines
Quoted 3 lines

Yes for 3), we may also export the information through the existing ioctls if possible (eg. IOC_FS_INFO).

For _right now_ I'd say just not do the raid56 stuff if we don't notice any raid56 chunks from the normal load_space_info, and then if there are raid56 we try and run the tree search ioctl and notice if we get back EPERM or whatever you get when you don't have permissions. Then just spit out as much information that you can about the fs with a little note at the bottom that available calculation isn't 100% and you need to run as root if you want that info.

Quoted 8 lines

Then what we could do is add another flag type for the existing SPACE_INFO ioctl to spit out the information you need about the raid5/6 chunks and then just test for those flags and make the adjustment necessary. This way we avoid adding yet another ioctl and stuff will still work nicely for old kernels that don't have the updated ioctl. Thanks,

Josef

Good suggestion ! I will take it

Signature · 2 lines
gpg @keyserver.linux.it: Goffredo Baroncelli (kreijackATinwind.it>
Key fingerprint BBF5 1610 0B64 DAC6 5F7D  17B2 0EDA 9B37 8B82 E0B5
#28 lore raw
In reply to Josef Bacik
Quoted 11 lines · Josef Bacik
Quoted 3 lines

Yes for 3), we may also export the information through the existing ioctls if possible (eg. IOC_FS_INFO).

For _right now_ I'd say just not do the raid56 stuff if we don't notice any raid56 chunks from the normal load_space_info, and then if there are raid56 we try and run the tree search ioctl and notice if we get back EPERM or whatever you get when you don't have permissions. Then just spit out as much information that you can about the fs with a little note at the bottom that available calculation isn't 100% and you need to run as root if you want that info.

Works for me.

Quoted 6 lines

Then what we could do is add another flag type for the existing SPACE_INFO ioctl to spit out the information you need about the raid5/6 chunks and then just test for those flags and make the adjustment necessary. This way we avoid adding yet another ioctl and stuff will still work nicely for old kernels that don't have the updated ioctl. Thanks,

Agreed. Extending SPACE_INFO looks suitable for this.