Skip to contentSkip to conversation

The value displayed by 'ls -s' command is strange.

DISCUSSION Tsutomu Itoh 5 participants 11 messages Dec 7, 2010 – Dec 9, 2010 Btrfs file system

lore
Share & tools
lore
#0 lore raw

Hi,

I think that the disk allocation size of each file becomes a monotone increase when the file is made.
But, it sometimes return to 0. Is it correct?

The result of the test at 2.6.37-rc4 is shown below. (see inode no. 291)

    # df -T /test14
    Filesystem    Type   1K-blocks      Used Available Use% Mounted on
    /dev/sdd14   btrfs     4162560      8736   3709440   1% /test14
    # dd if=/dev/zero of=/test14/dir/as001.26603 bs=1M count=100
    # dd if=/dev/zero of=/test14/dir/as002.26603 bs=1M count=200
    # dd if=/dev/zero of=/test14/dir/sy001.26603 bs=1M count=300 oflag=direct
    # dd if=/dev/zero of=/test14/dir/as003.26603 bs=1M count=400
    # ls -lis /test14/dir
    total 406528
    288      0 -rw-r--r-- 1 root root 104857600 Dec  7 15:07 as001.26603
    289      0 -rw-r--r-- 1 root root 209715200 Dec  7 15:07 as002.26603
 -> 291  99328 -rw-r--r-- 1 root root 419430400 Dec  7 15:08 as003.26603
    290 307200 -rw-r--r-- 1 root root 314572800 Dec  7 15:08 sy001.26603
    # sleep 3
    # ls -lis /test14/dir
    total 406528
    288      0 -rw-r--r-- 1 root root 104857600 Dec  7 15:07 as001.26603
    289      0 -rw-r--r-- 1 root root 209715200 Dec  7 15:07 as002.26603
 -> 291  99328 -rw-r--r-- 1 root root 419430400 Dec  7 15:08 as003.26603
    290 307200 -rw-r--r-- 1 root root 314572800 Dec  7 15:08 sy001.26603
    # sleep 3
    # ls -lis /test14/dir
    total 307200
    288      0 -rw-r--r-- 1 root root 104857600 Dec  7 15:07 as001.26603
    289      0 -rw-r--r-- 1 root root 209715200 Dec  7 15:07 as002.26603
 -> 291      0 -rw-r--r-- 1 root root 419430400 Dec  7 15:08 as003.26603
    290 307200 -rw-r--r-- 1 root root 314572800 Dec  7 15:08 sy001.26603
    # sleep 3
    # ls -lis /test14/dir
    total 409600
    288 102400 -rw-r--r-- 1 root root 104857600 Dec  7 15:07 as001.26603
    289      0 -rw-r--r-- 1 root root 209715200 Dec  7 15:07 as002.26603
 -> 291      0 -rw-r--r-- 1 root root 419430400 Dec  7 15:08 as003.26603
    290 307200 -rw-r--r-- 1 root root 314572800 Dec  7 15:08 sy001.26603
    # sync
    # ls -lis /test14/dir
    total 1024000
    288 102400 -rw-r--r-- 1 root root 104857600 Dec  7 15:07 as001.26603
    289 204800 -rw-r--r-- 1 root root 209715200 Dec  7 15:07 as002.26603
 -> 291 409600 -rw-r--r-- 1 root root 419430400 Dec  7 15:08 as003.26603
    290 307200 -rw-r--r-- 1 root root 314572800 Dec  7 15:08 sy001.26603

The trace result of btrfs_getattr() is shown below.

 Dec  7 15:08:03 luna kernel: ino:291 blocks:198656 i_blocks:0 i_bytes:0 delalloc_bytes:101711872
 Dec  7 15:08:06 luna kernel: ino:291 blocks:198656 i_blocks:0 i_bytes:0 delalloc_bytes:101711872
 Dec  7 15:08:09 luna kernel: ino:291 blocks:0 i_blocks:0 i_bytes:0 delalloc_bytes:0
 Dec  7 15:08:12 luna kernel: ino:291 blocks:0 i_blocks:0 i_bytes:0 delalloc_bytes:0
 Dec  7 15:08:18 luna kernel: ino:291 blocks:819200 i_blocks:819200 i_bytes:0 delalloc_bytes:0

Regards,
Itoh

#1 lore raw
Quoted 6 lines · Tsutomu Itoh

Hi,

I think that the disk allocation size of each file becomes a monotone increase when the file is made.
But, it sometimes return to 0. Is it correct?

The # of blocks is:

        stat->blocks = (inode_get_bytes(inode) +
                        BTRFS_I(inode)->delalloc_bytes) >> 9;

So I think after sub(delalloc_bytes) and before inode_add_bytes(), you may see 0 value.

Quoted 58 lines

The result of the test at 2.6.37-rc4 is shown below. (see inode no. 291)

    # df -T /test14
    Filesystem    Type   1K-blocks      Used Available Use% Mounted on
    /dev/sdd14   btrfs     4162560      8736   3709440   1% /test14
    # dd if=/dev/zero of=/test14/dir/as001.26603 bs=1M count=100
    # dd if=/dev/zero of=/test14/dir/as002.26603 bs=1M count=200
    # dd if=/dev/zero of=/test14/dir/sy001.26603 bs=1M count=300 oflag=direct
    # dd if=/dev/zero of=/test14/dir/as003.26603 bs=1M count=400
    # ls -lis /test14/dir
    total 406528
    288      0 -rw-r--r-- 1 root root 104857600 Dec  7 15:07 as001.26603
    289      0 -rw-r--r-- 1 root root 209715200 Dec  7 15:07 as002.26603
 -> 291  99328 -rw-r--r-- 1 root root 419430400 Dec  7 15:08 as003.26603
    290 307200 -rw-r--r-- 1 root root 314572800 Dec  7 15:08 sy001.26603
    # sleep 3
    # ls -lis /test14/dir
    total 406528
    288      0 -rw-r--r-- 1 root root 104857600 Dec  7 15:07 as001.26603
    289      0 -rw-r--r-- 1 root root 209715200 Dec  7 15:07 as002.26603
 -> 291  99328 -rw-r--r-- 1 root root 419430400 Dec  7 15:08 as003.26603
    290 307200 -rw-r--r-- 1 root root 314572800 Dec  7 15:08 sy001.26603
    # sleep 3
    # ls -lis /test14/dir
    total 307200
    288      0 -rw-r--r-- 1 root root 104857600 Dec  7 15:07 as001.26603
    289      0 -rw-r--r-- 1 root root 209715200 Dec  7 15:07 as002.26603
 -> 291      0 -rw-r--r-- 1 root root 419430400 Dec  7 15:08 as003.26603
    290 307200 -rw-r--r-- 1 root root 314572800 Dec  7 15:08 sy001.26603
    # sleep 3
    # ls -lis /test14/dir
    total 409600
    288 102400 -rw-r--r-- 1 root root 104857600 Dec  7 15:07 as001.26603
    289      0 -rw-r--r-- 1 root root 209715200 Dec  7 15:07 as002.26603
 -> 291      0 -rw-r--r-- 1 root root 419430400 Dec  7 15:08 as003.26603
    290 307200 -rw-r--r-- 1 root root 314572800 Dec  7 15:08 sy001.26603
    # sync
    # ls -lis /test14/dir
    total 1024000
    288 102400 -rw-r--r-- 1 root root 104857600 Dec  7 15:07 as001.26603
    289 204800 -rw-r--r-- 1 root root 209715200 Dec  7 15:07 as002.26603
 -> 291 409600 -rw-r--r-- 1 root root 419430400 Dec  7 15:08 as003.26603
    290 307200 -rw-r--r-- 1 root root 314572800 Dec  7 15:08 sy001.26603

The trace result of btrfs_getattr() is shown below.

 Dec  7 15:08:03 luna kernel: ino:291 blocks:198656 i_blocks:0 i_bytes:0 delalloc_bytes:101711872
 Dec  7 15:08:06 luna kernel: ino:291 blocks:198656 i_blocks:0 i_bytes:0 delalloc_bytes:101711872
 Dec  7 15:08:09 luna kernel: ino:291 blocks:0 i_blocks:0 i_bytes:0 delalloc_bytes:0
 Dec  7 15:08:12 luna kernel: ino:291 blocks:0 i_blocks:0 i_bytes:0 delalloc_bytes:0
 Dec  7 15:08:18 luna kernel: ino:291 blocks:819200 i_blocks:819200 i_bytes:0 delalloc_bytes:0

Regards,
Itoh

#2 lore raw
Quoted 8 lines · Li Zefan
Quoted 6 lines · Tsutomu Itoh

Hi,

I think that the disk allocation size of each file becomes a monotone increase when the file is made.
But, it sometimes return to 0. Is it correct?

Quoted 7 lines

The # of blocks is:

        stat->blocks = (inode_get_bytes(inode) +
                        BTRFS_I(inode)->delalloc_bytes) >> 9;

So I think after sub(delalloc_bytes) and before inode_add_bytes(), you may see 0 value.

Yes, I also think so.
But, I think that such a state is too long for only the update timing...

Quoted 65 lines
Quoted 56 lines

The result of the test at 2.6.37-rc4 is shown below. (see inode no. 291)

    # df -T /test14
    Filesystem    Type   1K-blocks      Used Available Use% Mounted on
    /dev/sdd14   btrfs     4162560      8736   3709440   1% /test14
    # dd if=/dev/zero of=/test14/dir/as001.26603 bs=1M count=100
    # dd if=/dev/zero of=/test14/dir/as002.26603 bs=1M count=200
    # dd if=/dev/zero of=/test14/dir/sy001.26603 bs=1M count=300 oflag=direct
    # dd if=/dev/zero of=/test14/dir/as003.26603 bs=1M count=400
    # ls -lis /test14/dir
    total 406528
    288      0 -rw-r--r-- 1 root root 104857600 Dec  7 15:07 as001.26603
    289      0 -rw-r--r-- 1 root root 209715200 Dec  7 15:07 as002.26603
 -> 291  99328 -rw-r--r-- 1 root root 419430400 Dec  7 15:08 as003.26603
    290 307200 -rw-r--r-- 1 root root 314572800 Dec  7 15:08 sy001.26603
    # sleep 3
    # ls -lis /test14/dir
    total 406528
    288      0 -rw-r--r-- 1 root root 104857600 Dec  7 15:07 as001.26603
    289      0 -rw-r--r-- 1 root root 209715200 Dec  7 15:07 as002.26603
 -> 291  99328 -rw-r--r-- 1 root root 419430400 Dec  7 15:08 as003.26603
    290 307200 -rw-r--r-- 1 root root 314572800 Dec  7 15:08 sy001.26603
    # sleep 3
    # ls -lis /test14/dir
    total 307200
    288      0 -rw-r--r-- 1 root root 104857600 Dec  7 15:07 as001.26603
    289      0 -rw-r--r-- 1 root root 209715200 Dec  7 15:07 as002.26603
 -> 291      0 -rw-r--r-- 1 root root 419430400 Dec  7 15:08 as003.26603
    290 307200 -rw-r--r-- 1 root root 314572800 Dec  7 15:08 sy001.26603
    # sleep 3
    # ls -lis /test14/dir
    total 409600
    288 102400 -rw-r--r-- 1 root root 104857600 Dec  7 15:07 as001.26603
    289      0 -rw-r--r-- 1 root root 209715200 Dec  7 15:07 as002.26603
 -> 291      0 -rw-r--r-- 1 root root 419430400 Dec  7 15:08 as003.26603
    290 307200 -rw-r--r-- 1 root root 314572800 Dec  7 15:08 sy001.26603
    # sync
    # ls -lis /test14/dir
    total 1024000
    288 102400 -rw-r--r-- 1 root root 104857600 Dec  7 15:07 as001.26603
    289 204800 -rw-r--r-- 1 root root 209715200 Dec  7 15:07 as002.26603
 -> 291 409600 -rw-r--r-- 1 root root 419430400 Dec  7 15:08 as003.26603
    290 307200 -rw-r--r-- 1 root root 314572800 Dec  7 15:08 sy001.26603

The trace result of btrfs_getattr() is shown below.

 Dec  7 15:08:03 luna kernel: ino:291 blocks:198656 i_blocks:0 i_bytes:0 delalloc_bytes:101711872
 Dec  7 15:08:06 luna kernel: ino:291 blocks:198656 i_blocks:0 i_bytes:0 delalloc_bytes:101711872
 Dec  7 15:08:09 luna kernel: ino:291 blocks:0 i_blocks:0 i_bytes:0 delalloc_bytes:0
 Dec  7 15:08:12 luna kernel: ino:291 blocks:0 i_blocks:0 i_bytes:0 delalloc_bytes:0
 Dec  7 15:08:18 luna kernel: ino:291 blocks:819200 i_blocks:819200 i_bytes:0 delalloc_bytes:0
Quoted 3 lines

Regards,
Itoh

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

#3 lore raw
Quoted 7 lines · Tsutomu Itoh
Quoted 7 lines
Quoted 4 lines

I think that the disk allocation size of each file becomes a monotone increase when the file is made.
But, it sometimes return to 0. Is it correct?

The # of blocks is:

Quoted 8 lines
Quoted 5 lines
         stat->blocks = (inode_get_bytes(inode) +
                         BTRFS_I(inode)->delalloc_bytes)>>  9;

So I think after sub(delalloc_bytes) and before inode_add_bytes(), you may see 0 value.

Yes, I also think so.
But, I think that such a state is too long for only the update timing...

Several months ago, some one posted a patch to get the allocated size of the compressed file,

    http://marc.info/?l=linux-btrfs&m=128109745012238&w=2

this patch may help you to implement what you need.

Regards
Miao

Quoted 71 lines
Quoted 56 lines
Quoted 56 lines

The result of the test at 2.6.37-rc4 is shown below. (see inode no. 291)

     # df -T /test14
     Filesystem    Type   1K-blocks      Used Available Use% Mounted on
     /dev/sdd14   btrfs     4162560      8736   3709440   1% /test14
     # dd if=/dev/zero of=/test14/dir/as001.26603 bs=1M count=100
     # dd if=/dev/zero of=/test14/dir/as002.26603 bs=1M count=200
     # dd if=/dev/zero of=/test14/dir/sy001.26603 bs=1M count=300 oflag=direct
     # dd if=/dev/zero of=/test14/dir/as003.26603 bs=1M count=400
     # ls -lis /test14/dir
     total 406528
     288      0 -rw-r--r-- 1 root root 104857600 Dec  7 15:07 as001.26603
     289      0 -rw-r--r-- 1 root root 209715200 Dec  7 15:07 as002.26603
  ->  291  99328 -rw-r--r-- 1 root root 419430400 Dec  7 15:08 as003.26603
     290 307200 -rw-r--r-- 1 root root 314572800 Dec  7 15:08 sy001.26603
     # sleep 3
     # ls -lis /test14/dir
     total 406528
     288      0 -rw-r--r-- 1 root root 104857600 Dec  7 15:07 as001.26603
     289      0 -rw-r--r-- 1 root root 209715200 Dec  7 15:07 as002.26603
  ->  291  99328 -rw-r--r-- 1 root root 419430400 Dec  7 15:08 as003.26603
     290 307200 -rw-r--r-- 1 root root 314572800 Dec  7 15:08 sy001.26603
     # sleep 3
     # ls -lis /test14/dir
     total 307200
     288      0 -rw-r--r-- 1 root root 104857600 Dec  7 15:07 as001.26603
     289      0 -rw-r--r-- 1 root root 209715200 Dec  7 15:07 as002.26603
  ->  291      0 -rw-r--r-- 1 root root 419430400 Dec  7 15:08 as003.26603
     290 307200 -rw-r--r-- 1 root root 314572800 Dec  7 15:08 sy001.26603
     # sleep 3
     # ls -lis /test14/dir
     total 409600
     288 102400 -rw-r--r-- 1 root root 104857600 Dec  7 15:07 as001.26603
     289      0 -rw-r--r-- 1 root root 209715200 Dec  7 15:07 as002.26603
  ->  291      0 -rw-r--r-- 1 root root 419430400 Dec  7 15:08 as003.26603
     290 307200 -rw-r--r-- 1 root root 314572800 Dec  7 15:08 sy001.26603
     # sync
     # ls -lis /test14/dir
     total 1024000
     288 102400 -rw-r--r-- 1 root root 104857600 Dec  7 15:07 as001.26603
     289 204800 -rw-r--r-- 1 root root 209715200 Dec  7 15:07 as002.26603
  ->  291 409600 -rw-r--r-- 1 root root 419430400 Dec  7 15:08 as003.26603
     290 307200 -rw-r--r-- 1 root root 314572800 Dec  7 15:08 sy001.26603

The trace result of btrfs_getattr() is shown below.

  Dec  7 15:08:03 luna kernel: ino:291 blocks:198656 i_blocks:0 i_bytes:0 delalloc_bytes:101711872
  Dec  7 15:08:06 luna kernel: ino:291 blocks:198656 i_blocks:0 i_bytes:0 delalloc_bytes:101711872
  Dec  7 15:08:09 luna kernel: ino:291 blocks:0 i_blocks:0 i_bytes:0 delalloc_bytes:0
  Dec  7 15:08:12 luna kernel: ino:291 blocks:0 i_blocks:0 i_bytes:0 delalloc_bytes:0
  Dec  7 15:08:18 luna kernel: ino:291 blocks:819200 i_blocks:819200 i_bytes:0 delalloc_bytes:0
Quoted 9 lines
Quoted 3 lines

Regards,
Itoh

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

#4 lore raw
In reply to Tsutomu Itoh

Excerpts from Tsutomu Itoh's message of 2010-12-07 02:59:52 -0500:

Quoted 5 lines

Hi,

I think that the disk allocation size of each file becomes a monotone increase when the file is made.
But, it sometimes return to 0. Is it correct?

Well, there's a window during the processing of delayed allocation where we don't have the bytes recorded as delalloc and we don't have the bytes recorded in the inode yet. That's why they are showing up as zero.

We don't call inode_add_bytes() until after we insert the extent, but we drop the delalloc byte count on the file before the IO is done.

Fixing it will be a little tricky because all the extent accounting assumes the inode_add_bytes happens at extent insertion time.

-chris

Quoted 59 lines

The result of the test at 2.6.37-rc4 is shown below. (see inode no. 291)

    # df -T /test14
    Filesystem    Type   1K-blocks      Used Available Use% Mounted on
    /dev/sdd14   btrfs     4162560      8736   3709440   1% /test14
    # dd if=/dev/zero of=/test14/dir/as001.26603 bs=1M count=100
    # dd if=/dev/zero of=/test14/dir/as002.26603 bs=1M count=200
    # dd if=/dev/zero of=/test14/dir/sy001.26603 bs=1M count=300 oflag=direct
    # dd if=/dev/zero of=/test14/dir/as003.26603 bs=1M count=400
    # ls -lis /test14/dir
    total 406528
    288      0 -rw-r--r-- 1 root root 104857600 Dec  7 15:07 as001.26603
    289      0 -rw-r--r-- 1 root root 209715200 Dec  7 15:07 as002.26603
 -> 291  99328 -rw-r--r-- 1 root root 419430400 Dec  7 15:08 as003.26603
    290 307200 -rw-r--r-- 1 root root 314572800 Dec  7 15:08 sy001.26603
    # sleep 3
    # ls -lis /test14/dir
    total 406528
    288      0 -rw-r--r-- 1 root root 104857600 Dec  7 15:07 as001.26603
    289      0 -rw-r--r-- 1 root root 209715200 Dec  7 15:07 as002.26603
 -> 291  99328 -rw-r--r-- 1 root root 419430400 Dec  7 15:08 as003.26603
    290 307200 -rw-r--r-- 1 root root 314572800 Dec  7 15:08 sy001.26603
    # sleep 3
    # ls -lis /test14/dir
    total 307200
    288      0 -rw-r--r-- 1 root root 104857600 Dec  7 15:07 as001.26603
    289      0 -rw-r--r-- 1 root root 209715200 Dec  7 15:07 as002.26603
 -> 291      0 -rw-r--r-- 1 root root 419430400 Dec  7 15:08 as003.26603
    290 307200 -rw-r--r-- 1 root root 314572800 Dec  7 15:08 sy001.26603
    # sleep 3
    # ls -lis /test14/dir
    total 409600
    288 102400 -rw-r--r-- 1 root root 104857600 Dec  7 15:07 as001.26603
    289      0 -rw-r--r-- 1 root root 209715200 Dec  7 15:07 as002.26603
 -> 291      0 -rw-r--r-- 1 root root 419430400 Dec  7 15:08 as003.26603
    290 307200 -rw-r--r-- 1 root root 314572800 Dec  7 15:08 sy001.26603
    # sync
    # ls -lis /test14/dir
    total 1024000
    288 102400 -rw-r--r-- 1 root root 104857600 Dec  7 15:07 as001.26603
    289 204800 -rw-r--r-- 1 root root 209715200 Dec  7 15:07 as002.26603
 -> 291 409600 -rw-r--r-- 1 root root 419430400 Dec  7 15:08 as003.26603
    290 307200 -rw-r--r-- 1 root root 314572800 Dec  7 15:08 sy001.26603

The trace result of btrfs_getattr() is shown below.

 Dec  7 15:08:03 luna kernel: ino:291 blocks:198656 i_blocks:0 i_bytes:0 delalloc_bytes:101711872
 Dec  7 15:08:06 luna kernel: ino:291 blocks:198656 i_blocks:0 i_bytes:0 delalloc_bytes:101711872
 Dec  7 15:08:09 luna kernel: ino:291 blocks:0 i_blocks:0 i_bytes:0 delalloc_bytes:0
 Dec  7 15:08:12 luna kernel: ino:291 blocks:0 i_blocks:0 i_bytes:0 delalloc_bytes:0
 Dec  7 15:08:18 luna kernel: ino:291 blocks:819200 i_blocks:819200 i_bytes:0 delalloc_bytes:0

Regards,
Itoh

#5 lore raw

On Tue, Dec 7, 2010 at 10:44 AM, Chris Mason <[email protected]> w= rote:

Quoted 4 lines

Excerpts from Tsutomu Itoh's message of 2010-12-07 02:59:52 -0500:

Quoted 3 lines

Hi,

I think that the disk allocation size of each file becomes a monoton=

e increase

Quoted 4 lines
Quoted 2 lines

when the file is made.
But, it sometimes return to 0. =C2=A0Is it correct?

Well, there's a window during the processing of delayed allocation wh=

ere

Quoted 1 line

we don't have the bytes recorded as delalloc and we don't have the by=

tes

Quoted 1 line

recorded in the inode yet. =C2=A0That's why they are showing up as ze=

ro.

Quoted 2 lines

We don't call inode_add_bytes() until after we insert the extent, but=

we

Quoted 5 lines

drop the delalloc byte count on the file before the IO is done.

Fixing it will be a little tricky because all the extent accounting assumes the inode_add_bytes happens at extent insertion time.

How does opening the inode with O_APPEND during this window know where to write the bytes? If it's a pointer/cursor to the EOF then that size could be used during the window. Is that right?

Quoted 7 lines
Quoted 7 lines

The result of the test at 2.6.37-rc4 is shown below. (see inode no. 291)

=C2=A0 =C2=A0 # df -T /test14
=C2=A0 =C2=A0 Filesystem =C2=A0 =C2=A0Type =C2=A0 1K-blocks =C2=A0 =C2=

=A0 =C2=A0Used Available Use% Mounted on

Quoted 1 line
Quoted 1 line

=C2=A0 =C2=A0 /dev/sdd14 =C2=A0 btrfs =C2=A0 =C2=A0 4162560 =C2=A0 =C2=

=A0 =C2=A08736 =C2=A0 3709440 =C2=A0 1% /test14

Quoted 1 line
Quoted 1 line

=C2=A0 =C2=A0 # dd if=3D/dev/zero of=3D/test14/dir/as001.26603 bs=3D=

1M count=3D100

Quoted 1 line
Quoted 1 line

=C2=A0 =C2=A0 # dd if=3D/dev/zero of=3D/test14/dir/as002.26603 bs=3D=

1M count=3D200

Quoted 1 line
Quoted 1 line

=C2=A0 =C2=A0 # dd if=3D/dev/zero of=3D/test14/dir/sy001.26603 bs=3D=

1M count=3D300 oflag=3Ddirect

Quoted 1 line
Quoted 1 line

=C2=A0 =C2=A0 # dd if=3D/dev/zero of=3D/test14/dir/as003.26603 bs=3D=

1M count=3D400

Quoted 3 lines
Quoted 3 lines

=C2=A0 =C2=A0 # ls -lis /test14/dir
=C2=A0 =C2=A0 total 406528
=C2=A0 =C2=A0 288 =C2=A0 =C2=A0 =C2=A00 -rw-r--r-- 1 root root 10485=

7600 Dec =C2=A07 15:07 as001.26603

Quoted 1 line
Quoted 1 line

=C2=A0 =C2=A0 289 =C2=A0 =C2=A0 =C2=A00 -rw-r--r-- 1 root root 20971=

5200 Dec =C2=A07 15:07 as002.26603

Quoted 1 line
Quoted 1 line

=C2=A0-> 291 =C2=A099328 -rw-r--r-- 1 root root 419430400 Dec =C2=A0=

7 15:08 as003.26603

Quoted 1 line
Quoted 1 line

=C2=A0 =C2=A0 290 307200 -rw-r--r-- 1 root root 314572800 Dec =C2=A0=

7 15:08 sy001.26603

Quoted 4 lines
Quoted 4 lines

=C2=A0 =C2=A0 # sleep 3
=C2=A0 =C2=A0 # ls -lis /test14/dir
=C2=A0 =C2=A0 total 406528
=C2=A0 =C2=A0 288 =C2=A0 =C2=A0 =C2=A00 -rw-r--r-- 1 root root 10485=

7600 Dec =C2=A07 15:07 as001.26603

Quoted 1 line
Quoted 1 line

=C2=A0 =C2=A0 289 =C2=A0 =C2=A0 =C2=A00 -rw-r--r-- 1 root root 20971=

5200 Dec =C2=A07 15:07 as002.26603

Quoted 1 line
Quoted 1 line

=C2=A0-> 291 =C2=A099328 -rw-r--r-- 1 root root 419430400 Dec =C2=A0=

7 15:08 as003.26603

Quoted 1 line
Quoted 1 line

=C2=A0 =C2=A0 290 307200 -rw-r--r-- 1 root root 314572800 Dec =C2=A0=

7 15:08 sy001.26603

Quoted 4 lines
Quoted 4 lines

=C2=A0 =C2=A0 # sleep 3
=C2=A0 =C2=A0 # ls -lis /test14/dir
=C2=A0 =C2=A0 total 307200
=C2=A0 =C2=A0 288 =C2=A0 =C2=A0 =C2=A00 -rw-r--r-- 1 root root 10485=

7600 Dec =C2=A07 15:07 as001.26603

Quoted 1 line
Quoted 1 line

=C2=A0 =C2=A0 289 =C2=A0 =C2=A0 =C2=A00 -rw-r--r-- 1 root root 20971=

5200 Dec =C2=A07 15:07 as002.26603

Quoted 1 line
Quoted 1 line

=C2=A0-> 291 =C2=A0 =C2=A0 =C2=A00 -rw-r--r-- 1 root root 419430400 =

Dec =C2=A07 15:08 as003.26603

Quoted 1 line
Quoted 1 line

=C2=A0 =C2=A0 290 307200 -rw-r--r-- 1 root root 314572800 Dec =C2=A0=

7 15:08 sy001.26603

Quoted 4 lines
Quoted 4 lines

=C2=A0 =C2=A0 # sleep 3
=C2=A0 =C2=A0 # ls -lis /test14/dir
=C2=A0 =C2=A0 total 409600
=C2=A0 =C2=A0 288 102400 -rw-r--r-- 1 root root 104857600 Dec =C2=A0=

7 15:07 as001.26603

Quoted 1 line
Quoted 1 line

=C2=A0 =C2=A0 289 =C2=A0 =C2=A0 =C2=A00 -rw-r--r-- 1 root root 20971=

5200 Dec =C2=A07 15:07 as002.26603

Quoted 1 line
Quoted 1 line

=C2=A0-> 291 =C2=A0 =C2=A0 =C2=A00 -rw-r--r-- 1 root root 419430400 =

Dec =C2=A07 15:08 as003.26603

Quoted 1 line
Quoted 1 line

=C2=A0 =C2=A0 290 307200 -rw-r--r-- 1 root root 314572800 Dec =C2=A0=

7 15:08 sy001.26603

Quoted 4 lines
Quoted 4 lines

=C2=A0 =C2=A0 # sync
=C2=A0 =C2=A0 # ls -lis /test14/dir
=C2=A0 =C2=A0 total 1024000
=C2=A0 =C2=A0 288 102400 -rw-r--r-- 1 root root 104857600 Dec =C2=A0=

7 15:07 as001.26603

Quoted 1 line
Quoted 1 line

=C2=A0 =C2=A0 289 204800 -rw-r--r-- 1 root root 209715200 Dec =C2=A0=

7 15:07 as002.26603

Quoted 1 line
Quoted 1 line

=C2=A0-> 291 409600 -rw-r--r-- 1 root root 419430400 Dec =C2=A07 15:=

08 as003.26603

Quoted 1 line
Quoted 1 line

=C2=A0 =C2=A0 290 307200 -rw-r--r-- 1 root root 314572800 Dec =C2=A0=

7 15:08 sy001.26603

Quoted 4 lines
Quoted 4 lines

The trace result of btrfs_getattr() is shown below.

=C2=A0Dec =C2=A07 15:08:03 luna kernel: ino:291 blocks:198656 i_bloc=

ks:0 i_bytes:0 delalloc_bytes:101711872

Quoted 1 line
Quoted 1 line

=C2=A0Dec =C2=A07 15:08:06 luna kernel: ino:291 blocks:198656 i_bloc=

ks:0 i_bytes:0 delalloc_bytes:101711872

Quoted 1 line
Quoted 1 line

=C2=A0Dec =C2=A07 15:08:09 luna kernel: ino:291 blocks:0 i_blocks:0 =

i_bytes:0 delalloc_bytes:0

Quoted 1 line
Quoted 1 line

=C2=A0Dec =C2=A07 15:08:12 luna kernel: ino:291 blocks:0 i_blocks:0 =

i_bytes:0 delalloc_bytes:0

Quoted 1 line
Quoted 1 line

=C2=A0Dec =C2=A07 15:08:18 luna kernel: ino:291 blocks:819200 i_bloc=

ks:819200 i_bytes:0 delalloc_bytes:0

Quoted 7 lines
Quoted 5 lines

Regards,
Itoh

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs=

" in

Quoted 2 lines

the body of a message to [email protected]
More majordomo info at =C2=A0http://vger.kernel.org/majordomo-info.ht=

ml

Quoted 1 line

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" = in
the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

#6 lore raw

Excerpts from Mike Fedyk's message of 2010-12-07 14:16:55 -0500:

Quoted 1 line

On Tue, Dec 7, 2010 at 10:44 AM, Chris Mason <[email protected]>=

Quoted 4 lines · wrote:
Quoted 4 lines

Excerpts from Tsutomu Itoh's message of 2010-12-07 02:59:52 -0500:

Quoted 3 lines

Hi,

I think that the disk allocation size of each file becomes a monot=

one increase

Quoted 4 lines
Quoted 4 lines
Quoted 2 lines

when the file is made.
But, it sometimes return to 0. =C2=A0Is it correct?

Well, there's a window during the processing of delayed allocation =

where

Quoted 1 line
Quoted 1 line

we don't have the bytes recorded as delalloc and we don't have the =

bytes

Quoted 1 line
Quoted 1 line

recorded in the inode yet. =C2=A0That's why they are showing up as =

zero.

Quoted 2 lines
Quoted 2 lines

We don't call inode_add_bytes() until after we insert the extent, b=

ut we

Quoted 7 lines
Quoted 5 lines

drop the delalloc byte count on the file before the IO is done.

Fixing it will be a little tricky because all the extent accounting assumes the inode_add_bytes happens at extent insertion time.

=20
How does opening the inode with O_APPEND during this window know wher=

e

Quoted 2 lines

to write the bytes? If it's a pointer/cursor to the EOF then that size could be used during the window. Is that right?

This counter records the number of blocks allocated to the file, and reading it with ls -l or stat is somewhat racey by nature. Most of the time its fine, btrfs just has a really big window where the results fro= m
ls -l seem wrong.

But, the counter really means nothing to the btrfs internals. When we do file operations we go based on the extent pointers we find in the tree and i_size (i_size is strictly maintained).

The incorrect results are confusing but they don't hurt the metadata itself.

-chris
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" = in
the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

#7 lore raw

On Tue, Dec 7, 2010 at 11:29 AM, Chris Mason <[email protected]> w= rote:

Quoted 7 lines

Excerpts from Mike Fedyk's message of 2010-12-07 14:16:55 -0500:

Quoted 1 line

On Tue, Dec 7, 2010 at 10:44 AM, Chris Mason <[email protected]=

Quoted 4 lines · wrote:
Quoted 4 lines

Excerpts from Tsutomu Itoh's message of 2010-12-07 02:59:52 -0500:

Quoted 3 lines

Hi,

I think that the disk allocation size of each file becomes a mono=

tone increase

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 2 lines

when the file is made.
But, it sometimes return to 0. =C2=A0Is it correct?

Well, there's a window during the processing of delayed allocation=

where

Quoted 1 line
Quoted 1 line
Quoted 1 line

we don't have the bytes recorded as delalloc and we don't have the=

bytes

Quoted 1 line
Quoted 1 line
Quoted 1 line

recorded in the inode yet. =C2=A0That's why they are showing up as=

zero.

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

We don't call inode_add_bytes() until after we insert the extent, =

but we

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

drop the delalloc byte count on the file before the IO is done.

Fixing it will be a little tricky because all the extent accountin=

g

Quoted 4 lines
Quoted 4 lines
Quoted 2 lines

assumes the inode_add_bytes happens at extent insertion time.

How does opening the inode with O_APPEND during this window know whe=

re

Quoted 1 line
Quoted 1 line

to write the bytes? =C2=A0If it's a pointer/cursor to the EOF then t=

hat

Quoted 4 lines
Quoted 1 line

size could be used during the window. =C2=A0Is that right?

This counter records the number of blocks allocated to the file, and reading it with ls -l or stat is somewhat racey by nature. =C2=A0Most=

of the

Quoted 1 line

time its fine, btrfs just has a really big window where the results f=

rom

Quoted 2 lines

ls -l seem wrong.

I see. Is it using per-cpu vars or something similar?

Quoted 1 line

But, the counter really means nothing to the btrfs internals. =C2=A0W=

hen we

Quoted 3 lines

do file operations we go based on the extent pointers we find in the tree and i_size (i_size is strictly maintained).

Would it be too heavy of an operation to have stat walk the btrfs tree to get its data?

Quoted 2 lines

The incorrect results are confusing but they don't hurt the metadata itself.

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" = in
the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

#8 lore raw

Excerpts from Mike Fedyk's message of 2010-12-07 15:07:08 -0500:

Quoted 1 line

On Tue, Dec 7, 2010 at 11:29 AM, Chris Mason <[email protected]>=

Quoted 2 lines · wrote:
Quoted 2 lines

Excerpts from Mike Fedyk's message of 2010-12-07 14:16:55 -0500:

Quoted 1 line

On Tue, Dec 7, 2010 at 10:44 AM, Chris Mason <[email protected]=

Quoted 1 line · om> wrote:
Quoted 1 line
Quoted 1 line
Quoted 1 line

Excerpts from Tsutomu Itoh's message of 2010-12-07 02:59:52 -050=

0:

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

Hi,

I think that the disk allocation size of each file becomes a mo=

notone increase

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 2 lines

when the file is made.
But, it sometimes return to 0. =C2=A0Is it correct?

Well, there's a window during the processing of delayed allocati=

on where

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

we don't have the bytes recorded as delalloc and we don't have t=

he bytes

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

recorded in the inode yet. =C2=A0That's why they are showing up =

as zero.

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

We don't call inode_add_bytes() until after we insert the extent=

, but we

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

drop the delalloc byte count on the file before the IO is done.

Fixing it will be a little tricky because all the extent account=

ing

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 2 lines

assumes the inode_add_bytes happens at extent insertion time.

How does opening the inode with O_APPEND during this window know w=

here

Quoted 1 line
Quoted 1 line
Quoted 1 line

to write the bytes? =C2=A0If it's a pointer/cursor to the EOF then=

that

Quoted 3 lines
Quoted 3 lines
Quoted 1 line

size could be used during the window. =C2=A0Is that right?

This counter records the number of blocks allocated to the file, an=

d

Quoted 1 line
Quoted 1 line

reading it with ls -l or stat is somewhat racey by nature. =C2=A0Mo=

st of the

Quoted 1 line
Quoted 1 line

time its fine, btrfs just has a really big window where the results=

from

Quoted 4 lines
Quoted 2 lines

ls -l seem wrong.

=20
I see. Is it using per-cpu vars or something similar?

Our stat function returns the block count in the inode plus the number of bytes we have accounted as delayed allocation.

As we do writes to the file, the delayed allocation count goes up and then eventually we decide we need to do some IO.

Before we do the IO, we have to decide where on the disk to write the extents. Once that is decided, we decrement the count of delayed allocation bytes.

This is when stat starts returning the wrong answer.

Then we do the IO, and when the IO is done we actually insert the file extents into the file metadata. This is when stat starts returning the right answer again.

The whole setup sounds strange, but this is how btrfs implements the semantics from data=3Dordered. We don't update the file to point to the new blocks until after the IO is done, so we never have to wait on the data IO before we can do a transaction commit. It avoids all kinds of latencies with fsync and other problems.

One easy solution is to just add another counter in the in-memory inode for the number of bytes in flight that aren't accounted for in other places. But I'd rather not make the inode any bigger, so I'll have to think if we can solve this another way.

Quoted 2 lines

=20

Quoted 1 line

But, the counter really means nothing to the btrfs internals. =C2=A0=

When we

Quoted 1 line
Quoted 1 line

do file operations we go based on the extent pointers we find in th=

e

Quoted 4 lines
Quoted 2 lines

tree and i_size (i_size is strictly maintained).

=20
Would it be too heavy of an operation to have stat walk the btrfs tre=

e

Quoted 2 lines

to get its data?
=20

I'm afraid so, stat is fairly performance critical.

-chris
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" = in
the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

#9 lore raw

On Tue, Dec 7, 2010 at 12:15 PM, Chris Mason <[email protected]> w= rote:

Quoted 5 lines

Excerpts from Mike Fedyk's message of 2010-12-07 15:07:08 -0500:

Quoted 1 line

On Tue, Dec 7, 2010 at 11:29 AM, Chris Mason <[email protected]=

Quoted 2 lines · wrote:
Quoted 2 lines

Excerpts from Mike Fedyk's message of 2010-12-07 14:16:55 -0500:

Quoted 1 line

On Tue, Dec 7, 2010 at 10:44 AM, Chris Mason <chris.mason@oracle.=

Quoted 1 line · com> wrote:
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

Excerpts from Tsutomu Itoh's message of 2010-12-07 02:59:52 -05=

00:

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

Hi,

I think that the disk allocation size of each file becomes a m=

onotone increase

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 2 lines

when the file is made.
But, it sometimes return to 0. =C2=A0Is it correct?

Well, there's a window during the processing of delayed allocat=

ion where

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

we don't have the bytes recorded as delalloc and we don't have =

the bytes

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

recorded in the inode yet. =C2=A0That's why they are showing up=

as zero.

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

We don't call inode_add_bytes() until after we insert the exten=

t, but we

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

drop the delalloc byte count on the file before the IO is done.

Fixing it will be a little tricky because all the extent accoun=

ting

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 2 lines

assumes the inode_add_bytes happens at extent insertion time.

How does opening the inode with O_APPEND during this window know =

where

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

to write the bytes? =C2=A0If it's a pointer/cursor to the EOF the=

n that

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 1 line

size could be used during the window. =C2=A0Is that right?

This counter records the number of blocks allocated to the file, a=

nd

Quoted 1 line
Quoted 1 line
Quoted 1 line

reading it with ls -l or stat is somewhat racey by nature. =C2=A0M=

ost of the

Quoted 1 line
Quoted 1 line
Quoted 1 line

time its fine, btrfs just has a really big window where the result=

s from

Quoted 5 lines
Quoted 4 lines
Quoted 2 lines

ls -l seem wrong.

I see. =C2=A0Is it using per-cpu vars or something similar?

Ok, so to make sure I fully understand I'm going to make some psuedo code based on your description.

Quoted 1 line

Our stat function returns the block count in the inode plus the numbe=

r

Quoted 2 lines

of bytes we have accounted as delayed allocation.

stat =3D inode_a1.bytes + inode_a1_delayed_allocation_bytes

Quoted 5 lines

As we do writes to the file, the delayed allocation count goes up and then eventually we decide we need to do some IO.

Before we do the IO, we have to decide where on the disk to write the extents.

inode_a2 =3D inode_a1

inode_a1 and inode_a2 are the same inode, but inode_a2 has a different list of extents and is not written yet (in the case of appending, most of the extents will be the same in the two extent lists, but inode_a2 will have more extents for the newly appended data)

Quoted 5 lines

Once that is decided, we decrement the count of delayed allocation bytes.

This is when stat starts returning the wrong answer.

inode_a2.bytes +=3D inode_a1_delayed_allocation_bytes inode_a1_delayed_allocation_bytes -=3D inode_a1_delayed_allocation_byte= s
stat =3D inode_a1.bytes + inode_a1_delayed_allocation_bytes

Is it possible to have stat read from inode_a2 during this window?

So it would be instead:

stat =3D inode_a2.bytes

Quoted 1 line

Then we do the IO, and when the IO is done we actually insert the fil=

e

Quoted 1 line

extents into the file metadata. =C2=A0This is when stat starts return=

ing the

Quoted 2 lines

right answer again.

/* implicit when write completes */
inode_a1 =3D inode_a2
kfree(inode_a2)
stat =3D inode_a1.bytes + inode_a1_delayed_allocation_bytes

Quoted 2 lines

The whole setup sounds strange, but this is how btrfs implements the semantics from data=3Dordered. =C2=A0We don't update the file to poin=

t to

Quoted 1 line

the new blocks until after the IO is done, so we never have to wait o=

n

Quoted 1 line

the data IO before we can do a transaction commit. =C2=A0It avoids al=

l kinds

Quoted 3 lines

of latencies with fsync and other problems.

One easy solution is to just add another counter in the in-memory ino=

de

Quoted 2 lines

for the number of bytes in flight that aren't accounted for in other places. =C2=A0But I'd rather not make the inode any bigger, so I'll h=

ave to

Quoted 2 lines

think if we can solve this another way.

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" = in
the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

#10 lore raw
In reply to Chris Mason
Quoted 44 lines · Chris Mason

Excerpts from Mike Fedyk's message of 2010-12-07 15:07:08 -0500:

Quoted 25 lines · Chris Mason
Quoted 24 lines

Excerpts from Mike Fedyk's message of 2010-12-07 14:16:55 -0500:

Quoted 22 lines · Chris Mason
Quoted 11 lines

Excerpts from Tsutomu Itoh's message of 2010-12-07 02:59:52 -0500:

Quoted 5 lines

Hi,

I think that the disk allocation size of each file becomes a monotone increase when the file is made.
But, it sometimes return to 0. Is it correct?

Well, there's a window during the processing of delayed allocation where we don't have the bytes recorded as delalloc and we don't have the bytes recorded in the inode yet. That's why they are showing up as zero.

Quoted 6 lines

We don't call inode_add_bytes() until after we insert the extent, but we drop the delalloc byte count on the file before the IO is done.

Fixing it will be a little tricky because all the extent accounting assumes the inode_add_bytes happens at extent insertion time.

How does opening the inode with O_APPEND during this window know where to write the bytes? If it's a pointer/cursor to the EOF then that size could be used during the window. Is that right?

Quoted 7 lines
Quoted 5 lines

This counter records the number of blocks allocated to the file, and reading it with ls -l or stat is somewhat racey by nature. Most of the time its fine, btrfs just has a really big window where the results from ls -l seem wrong.

I see. Is it using per-cpu vars or something similar?

Our stat function returns the block count in the inode plus the number of bytes we have accounted as delayed allocation.

As we do writes to the file, the delayed allocation count goes up and then eventually we decide we need to do some IO.

Before we do the IO, we have to decide where on the disk to write the extents. Once that is decided, we decrement the count of delayed allocation bytes.

Quoted 5 lines

This is when stat starts returning the wrong answer.

Then we do the IO, and when the IO is done we actually insert the file extents into the file metadata. This is when stat starts returning the right answer again.

I understood.
However, I worry that the user is confused because the wrong condition is too long.

Quoted 25 lines

The whole setup sounds strange, but this is how btrfs implements the semantics from data=ordered. We don't update the file to point to the new blocks until after the IO is done, so we never have to wait on the data IO before we can do a transaction commit. It avoids all kinds of latencies with fsync and other problems.

One easy solution is to just add another counter in the in-memory inode for the number of bytes in flight that aren't accounted for in other places. But I'd rather not make the inode any bigger, so I'll have to think if we can solve this another way.

Quoted 9 lines
Quoted 5 lines

But, the counter really means nothing to the btrfs internals. When we do file operations we go based on the extent pointers we find in the tree and i_size (i_size is strictly maintained).

Would it be too heavy of an operation to have stat walk the btrfs tree to get its data?

I'm afraid so, stat is fairly performance critical.

-chris