Skip to contentSkip to conversation

Same size drive has less usable space

DISCUSSION Luciano ES 5 participants 12 messages Dec 20, 2018 – Dec 22, 2018

lore
Share & tools
lore

Luciano ES

author
#0 lore raw

I have a new drive for backups.
I copied everything over and now I have this problem:

Filesystem Size Used Avail Use% Mounted on

/dev/sda1   	931G  914G   18G  	99% 	/xx
/dev/sdb1   	931G  920G   11G  	99% 	/xxbkp

So 914GB from the old drive expand and become 920GB. The new drive is supposed to be the same size, but for some reason it can't really hold it all. I will be forced to waste precious gigabytes.

I tried to format the new one exactly like the old one, but it was not possible:

$ xfs_info /xx
meta-data=/dev/sda1     isize=256    agcount=8, agsize=30506944 blks
         =              sectsz=4096  attr=2, projid32bit=1
         =              crc=0        finobt=0 spinodes=0 rmapbt=0
         =              reflink=0
data     =              bsize=4096   blocks=244055552, imaxpct=25
         =              sunit=0      swidth=0 blks
naming   =version 2     bsize=4096   ascii-ci=0 ftype=0
log      =internal      bsize=4096   blocks=119167, version=2
         =              sectsz=4096  sunit=1 blks, lazy-count=1
realtime =none          extsz=4096   blocks=0, rtextents=0
$ xfs_info /xxbkp
meta-data=/dev/sdb1     isize=512    agcount=8, agsize=61013888 blks
         =              sectsz=512   attr=2, projid32bit=1
         =              crc=1        finobt=1 spinodes=1 rmapbt=0
         =              reflink=0
data     =              bsize=2048   blocks=488111104, imaxpct=25
         =              sunit=0      swidth=0 blks
naming   =version 2     bsize=4096   ascii-ci=0 ftype=1
log      =internal      bsize=2048   blocks=238335, version=2
         =              sectsz=512   sunit=0 blks, lazy-count=1
realtime =none          extsz=4096   blocks=0, rtextents=0

I guess at least part of the problem is CRC enabled in the second one. So, is there anything I can do to make all the data fit in the new drive?

TIA

Signature · 1 line
Luciano ES
Quoted 1 line
Quoted 1 line
#1 lore raw
Quoted 40 lines · Luciano ES

I have a new drive for backups.
I copied everything over and now I have this problem:

Filesystem Size Used Avail Use% Mounted on

/dev/sda1   	931G  914G   18G  	99% 	/xx
/dev/sdb1   	931G  920G   11G  	99% 	/xxbkp

So 914GB from the old drive expand and become 920GB. The new drive is supposed to be the same size, but for some reason it can't really hold it all. I will be forced to waste precious gigabytes.

I tried to format the new one exactly like the old one, but it was not possible:

$ xfs_info /xx
meta-data=/dev/sda1     isize=256    agcount=8, agsize=30506944 blks
         =              sectsz=4096  attr=2, projid32bit=1
         =              crc=0        finobt=0 spinodes=0 rmapbt=0
         =              reflink=0
data     =              bsize=4096   blocks=244055552, imaxpct=25
         =              sunit=0      swidth=0 blks
naming   =version 2     bsize=4096   ascii-ci=0 ftype=0
log      =internal      bsize=4096   blocks=119167, version=2
         =              sectsz=4096  sunit=1 blks, lazy-count=1
realtime =none          extsz=4096   blocks=0, rtextents=0
$ xfs_info /xxbkp
meta-data=/dev/sdb1     isize=512    agcount=8, agsize=61013888 blks
         =              sectsz=512   attr=2, projid32bit=1
         =              crc=1        finobt=1 spinodes=1 rmapbt=0
         =              reflink=0
data     =              bsize=2048   blocks=488111104, imaxpct=25
         =              sunit=0      swidth=0 blks
naming   =version 2     bsize=4096   ascii-ci=0 ftype=1
log      =internal      bsize=2048   blocks=238335, version=2
         =              sectsz=512   sunit=0 blks, lazy-count=1
realtime =none          extsz=4096   blocks=0, rtextents=0
Quoted 3 lines

I guess at least part of the problem is CRC enabled in the second one. So, is there anything I can do to make all the data fit in the new drive?

Well, you have larger inodes on xxbkp for starters. If you want,

# mkfs.xfs -m crc=0,finobt=1 -i sparse=0 -n ftype=0

should get you the same geometry. This is all documented in the mfks.xfs manpage, btw.

But beware that copying sparse files w/o maintaining sparseness (for example) will also consume more space. If you want a nothing less than a bit-for-bit copy, use dd. ;)

-Eric

#2 lore raw
Quoted 47 lines · Eric Sandeen
Quoted 40 lines · Luciano ES

I have a new drive for backups.
I copied everything over and now I have this problem:

Filesystem Size Used Avail Use% Mounted on

/dev/sda1   	931G  914G   18G  	99% 	/xx
/dev/sdb1   	931G  920G   11G  	99% 	/xxbkp

So 914GB from the old drive expand and become 920GB. The new drive is supposed to be the same size, but for some reason it can't really hold it all. I will be forced to waste precious gigabytes.

I tried to format the new one exactly like the old one, but it was not possible:

$ xfs_info /xx
meta-data=/dev/sda1     isize=256    agcount=8, agsize=30506944 blks
         =              sectsz=4096  attr=2, projid32bit=1
         =              crc=0        finobt=0 spinodes=0 rmapbt=0
         =              reflink=0
data     =              bsize=4096   blocks=244055552, imaxpct=25
         =              sunit=0      swidth=0 blks
naming   =version 2     bsize=4096   ascii-ci=0 ftype=0
log      =internal      bsize=4096   blocks=119167, version=2
         =              sectsz=4096  sunit=1 blks, lazy-count=1
realtime =none          extsz=4096   blocks=0, rtextents=0
$ xfs_info /xxbkp
meta-data=/dev/sdb1     isize=512    agcount=8, agsize=61013888 blks
         =              sectsz=512   attr=2, projid32bit=1
         =              crc=1        finobt=1 spinodes=1 rmapbt=0
         =              reflink=0
data     =              bsize=2048   blocks=488111104, imaxpct=25
         =              sunit=0      swidth=0 blks
naming   =version 2     bsize=4096   ascii-ci=0 ftype=1
log      =internal      bsize=2048   blocks=238335, version=2
         =              sectsz=512   sunit=0 blks, lazy-count=1
realtime =none          extsz=4096   blocks=0, rtextents=0
Quoted 3 lines

I guess at least part of the problem is CRC enabled in the second one. So, is there anything I can do to make all the data fit in the new drive?

Well, you have larger inodes on xxbkp for starters. If you want,

Quoted 8 lines
# mkfs.xfs -m crc=0,finobt=1 -i sparse=0 -n ftype=0

should get you the same geometry. This is all documented in the mfks.xfs manpage, btw.

But beware that copying sparse files w/o maintaining sparseness (for example) will also consume more space. If you want a nothing less than a bit-for-bit copy, use dd. ;)

Why not xfs_copy?

--D

Quoted 1 line

-Eric

#3 lore raw
Quoted 50 lines · Darrick J. Wong
Quoted 49 lines · Eric Sandeen
Quoted 40 lines · Luciano ES

I have a new drive for backups.
I copied everything over and now I have this problem:

Filesystem Size Used Avail Use% Mounted on

/dev/sda1   	931G  914G   18G  	99% 	/xx
/dev/sdb1   	931G  920G   11G  	99% 	/xxbkp

So 914GB from the old drive expand and become 920GB. The new drive is supposed to be the same size, but for some reason it can't really hold it all. I will be forced to waste precious gigabytes.

I tried to format the new one exactly like the old one, but it was not possible:

$ xfs_info /xx
meta-data=/dev/sda1     isize=256    agcount=8, agsize=30506944 blks
         =              sectsz=4096  attr=2, projid32bit=1
         =              crc=0        finobt=0 spinodes=0 rmapbt=0
         =              reflink=0
data     =              bsize=4096   blocks=244055552, imaxpct=25
         =              sunit=0      swidth=0 blks
naming   =version 2     bsize=4096   ascii-ci=0 ftype=0
log      =internal      bsize=4096   blocks=119167, version=2
         =              sectsz=4096  sunit=1 blks, lazy-count=1
realtime =none          extsz=4096   blocks=0, rtextents=0
$ xfs_info /xxbkp
meta-data=/dev/sdb1     isize=512    agcount=8, agsize=61013888 blks
         =              sectsz=512   attr=2, projid32bit=1
         =              crc=1        finobt=1 spinodes=1 rmapbt=0
         =              reflink=0
data     =              bsize=2048   blocks=488111104, imaxpct=25
         =              sunit=0      swidth=0 blks
naming   =version 2     bsize=4096   ascii-ci=0 ftype=1
log      =internal      bsize=2048   blocks=238335, version=2
         =              sectsz=512   sunit=0 blks, lazy-count=1
realtime =none          extsz=4096   blocks=0, rtextents=0
Quoted 3 lines

I guess at least part of the problem is CRC enabled in the second one. So, is there anything I can do to make all the data fit in the new drive?

Well, you have larger inodes on xxbkp for starters. If you want,

# mkfs.xfs -m crc=0,finobt=1 -i sparse=0 -n ftype=0
Quoted 8 lines
Quoted 6 lines

should get you the same geometry. This is all documented in the mfks.xfs manpage, btw.

But beware that copying sparse files w/o maintaining sparseness (for example) will also consume more space. If you want a nothing less than a bit-for-bit copy, use dd. ;)

Why not xfs_copy?

Or that :)

-Eric

author
#4 lore raw
In reply to Eric Sandeen
Quoted 1 line · Eric Sandeen

Well, you have larger inodes on xxbkp for starters.

I can't make smaller inodes on this disk.

"specified blocksize 2048 is less than device physical sector size 4096 switching to logical sector size 512"

another attempt:

"specified blocksize 2048 is less than device physical sector size 4096 switching to logical sector size 512
Minimum inode size for CRCs is 512 bytes"

So yes, it's the CRC feature.

Quoted 1 line · Eric Sandeen
# mkfs.xfs -m crc=0,finobt=1 -i sparse=0 -n ftype=0

Your command line doesn't work. It says:

finobt not supported without CRC support

I have confirmed that disabling CRC allows me to have smaller inodes, but I wish I could keep that feature. It sounds like something I want to have, to improve reliability.

Is there another way?

Quoted 1 line · Darrick J. Wong

Why not xfs_copy?

Good question. But then the copy would not have CRC enabled either because the original doesn't.

Looks like I an going to have to commit to having CRC always on from now on. And I am going to lose about 7GB of space for that.

Is it worth it?

Signature · 1 line
Luciano ES
Quoted 1 line
Quoted 1 line
#5 lore raw
Quoted 8 lines · Luciano ES
Quoted 1 line · Eric Sandeen

Well, you have larger inodes on xxbkp for starters.

I can't make smaller inodes on this disk.

"specified blocksize 2048 is less than device physical sector size 4096 switching to logical sector size 512"

1) that's a warning not an error
2) that's blocksize not inode size

but ok, I missed that you had 2k blocks on your backup disk.... I don't really know what you're trying to do here, TBH.

Quoted 5 lines

another attempt:

"specified blocksize 2048 is less than device physical sector size 4096 switching to logical sector size 512
Minimum inode size for CRCs is 512 bytes"

So, I don't think you turned off crcs here.

Quoted 1 line

So yes, it's the CRC feature.

So turn it off.

Quoted 8 lines · Eric Sandeen
Quoted 2 lines
# mkfs.xfs -m crc=0,finobt=1 -i sparse=0 -n ftype=0

Your command line doesn't work. It says:

finobt not supported without CRC support

yeah I fat-fingered that, make it finobt=0 of course.

Quoted 5 lines

I have confirmed that disabling CRC allows me to have smaller inodes, but I wish I could keep that feature. It sounds like something I want to have, to improve reliability.

Is there another way?

You're over-constraining yourself here; crcs require larger inodes which requires more space, by definition. Sorry, but you can't get something for nothing.

If you are trying to consume identical space you're going to need to use an identical format.

-Eric

author
#6 lore raw
Quoted 2 lines · Eric Sandeen

but ok, I missed that you had 2k blocks on your backup disk.... I don't really know what you're trying to do here, TBH.

Smaller blocks waste less space. In ext3/4, I've always used 2k or even 1k blocks. It does save space. Doesn't it in XFS too?

Signature · 1 line
Luciano ES
Quoted 1 line
Quoted 1 line
#7 lore raw
Quoted 7 lines · Luciano ES
Quoted 2 lines · Eric Sandeen

but ok, I missed that you had 2k blocks on your backup disk.... I don't really know what you're trying to do here, TBH.

Smaller blocks waste less space. In ext3/4, I've always used 2k or even 1k blocks. It does save space. Doesn't it in XFS too?

Depends. Not always "smaller blocks waste less space". It depends on the mix of filesizes. If most files are big, then no.

Consider, for instance, that in a filesystem with a fixed number of inodes (like ext2/3/4), when using small blocks the structures that list the blocks are larger, simply because there are more blocks.

Signature · 4 lines
Cheers / Saludos,

		Carlos E. R.
		(from 42.3 x86_64 "Malachite" at Telcontar)
#8 lore raw
In reply to Luciano ES
Quoted 2 lines · Luciano ES

I have a new drive for backups.
I copied everything over and now I have this problem:

...

Quoted 3 lines

I guess at least part of the problem is CRC enabled in the second one. So, is there anything I can do to make all the data fit in the new drive?

Clone it... dd, for instance, but then you have the problem of changing the uuid.

Signature · 4 lines
Cheers / Saludos,

		Carlos E. R.
		(from 42.3 x86_64 "Malachite" at Telcontar)
#9 lore raw
In reply to Luciano ES
Quoted 27 lines · Luciano ES

I have a new drive for backups.
I copied everything over and now I have this problem:

Filesystem Size Used Avail Use% Mounted on

/dev/sda1   	931G  914G   18G  	99% 	/xx
/dev/sdb1   	931G  920G   11G  	99% 	/xxbkp

So 914GB from the old drive expand and become 920GB. The new drive is supposed to be the same size, but for some reason it can't really hold it all. I will be forced to waste precious gigabytes.

I tried to format the new one exactly like the old one, but it was not possible:

$ xfs_info /xx
meta-data=/dev/sda1     isize=256    agcount=8, agsize=30506944 blks
         =              sectsz=4096  attr=2, projid32bit=1
         =              crc=0        finobt=0 spinodes=0 rmapbt=0
         =              reflink=0
data     =              bsize=4096   blocks=244055552, imaxpct=25
         =              sunit=0      swidth=0 blks
naming   =version 2     bsize=4096   ascii-ci=0 ftype=0
log      =internal      bsize=4096   blocks=119167, version=2
         =              sectsz=4096  sunit=1 blks, lazy-count=1
realtime =none          extsz=4096   blocks=0, rtextents=0
Quoted 6 lines
$ xfs_info /xxbkp
meta-data=/dev/sdb1     isize=512    agcount=8, agsize=61013888 blks
         =              sectsz=512   attr=2, projid32bit=1
         =              crc=1        finobt=1 spinodes=1 rmapbt=0
         =              reflink=0
data     =              bsize=2048   blocks=488111104, imaxpct=25

Just FYI you'll get a lot better performance from the filesystem if the blocksize is the same as cpu page size (4k I think) and if the filesystem isn't totally full.

But yeah, either unmount /xx and use xfs_copy one disk to the other, or format the backup drive with the same parameters (I'll leave that to the other parts of this thread).

--D

Quoted 16 lines
         =              sunit=0      swidth=0 blks
naming   =version 2     bsize=4096   ascii-ci=0 ftype=1
log      =internal      bsize=2048   blocks=238335, version=2
         =              sectsz=512   sunit=0 blks, lazy-count=1
realtime =none          extsz=4096   blocks=0, rtextents=0

I guess at least part of the problem is CRC enabled in the second one. So, is there anything I can do to make all the data fit in the new drive?

TIA

Signature · 1 line
Luciano ES
Quoted 1 line
Quoted 1 line
#10 lore raw
In reply to Luciano ES

Le Thu, 20 Dec 2018 13:13:05 -0200
Luciano ES <[email protected]> écrivait:

Quoted 3 lines

I guess at least part of the problem is CRC enabled in the second one. So, is there anything I can do to make all the data fit in the new drive?

Your new drive is using 4K blocks, while the old one uses 512b blocks. That's why they can't match.

Signature · 6 lines
------------------------------------------------------------------------
Emmanuel Florac     |   Direction technique
                    |   Intellique
                    |	<[email protected]>
                    |   +33 1 78 94 84 02
------------------------------------------------------------------------
author
#11 lore raw
Quoted 9 lines · Emmanuel Florac

Le Thu, 20 Dec 2018 13:13:05 -0200
Luciano ES écrivait:

Quoted 3 lines

I guess at least part of the problem is CRC enabled in the second one. So, is there anything I can do to make all the data fit in the new drive?

Your new drive is using 4K blocks, while the old one uses 512b blocks. That's why they can't match.

**************************

What 512b? There is no 512 in the old drive.

$ xfs_info /xx
meta-data=/dev/sda1     isize=256    agcount=8, agsize=30506944 blks
         =              sectsz=4096  attr=2, projid32bit=1
         =              crc=0        finobt=0 spinodes=0 rmapbt=0
         =              reflink=0
data     =              bsize=4096   blocks=244055552, imaxpct=25
         =              sunit=0      swidth=0 blks
naming   =version 2     bsize=4096   ascii-ci=0 ftype=0
log      =internal      bsize=4096   blocks=119167, version=2
         =              sectsz=4096  sunit=1 blks, lazy-count=1
realtime =none          extsz=4096   blocks=0, rtextents=0
$ xfs_info /xxbkp
meta-data=/dev/sdb1     isize=512    agcount=8, agsize=61013888 blks
         =              sectsz=512   attr=2, projid32bit=1
         =              crc=1        finobt=1 spinodes=1 rmapbt=0
         =              reflink=0
data     =              bsize=2048   blocks=488111104, imaxpct=25
         =              sunit=0      swidth=0 blks
naming   =version 2     bsize=4096   ascii-ci=0 ftype=1
log      =internal      bsize=2048   blocks=238335, version=2
         =              sectsz=512   sunit=0 blks, lazy-count=1
realtime =none          extsz=4096   blocks=0, rtextents=0
Signature · 1 line
Luciano ES
Quoted 1 line
Quoted 1 line