number of hardlinks for directory in ls -lid always 1?

DISCUSSION Martin Steigerwald 4 participants 12 messages Mar 17, 2015 – Mar 27, 2015 Btrfs file system

lore

Share & tools

lore

Martin Steigerwald

author

2015-03-17 13:33 UTC #0 lore raw

Hi!

On BTRFS I see

martin@merkaba:~> ls -lid /usr/local
27138 drwxrwsr-x 1 root staff 62 Aug 15 2014 /usr/local martin@merkaba:~> ls -lid /usr/local/.
27138 drwxrwsr-x 1 root staff 62 Aug 15 2014 /usr/local/. martin@merkaba:~> ls -lid /usr/local/bin/..
27138 drwxrwsr-x 1 root staff 62 Aug 15 2014 /usr/local/bin/..

On other filesystems like Ext4 I see the actual number of hardlinks to the directory.

Is this intended behaviour of BTRFS or a bug?

With files it works:

martin@merkaba:~> cd Zeit
martin@merkaba:~/Zeit> mkdir links
martin@merkaba:~/Zeit> cd links
martin@merkaba:~/Zeit/links> LANG=C df -hT .

Filesystem             Type   Size  Used Avail Use% Mounted on
/dev/mapper/msata-home btrfs  170G  120G   48G  72% /home

martin@merkaba:~/Zeit/links> echo "hello" > file martin@merkaba:~/Zeit/links> ls -li
insgesamt 4
12158633 -rw-r--r-- 1 martin martin 6 Mär 17 14:32 file martin@merkaba:~/Zeit/links> ln file hardlink1 martin@merkaba:~/Zeit/links> ls -li
insgesamt 8
12158633 -rw-r--r-- 2 martin martin 6 Mär 17 14:32 file 12158633 -rw-r--r-- 2 martin martin 6 Mär 17 14:32 hardlink1

Ciao,

Signature · 2 lines

Martin 'Helios' Steigerwald - http://www.Lichtvoll.de
GPG: 03B0 0D6C 0040 0710 4AFA  B82F 991B EAAC A599 84C7

David Sterba

2015-03-17 16:07 UTC #1 lore raw

Quoted 9 lines · Martin Steigerwald

On BTRFS I see
martin@merkaba:~> ls -lid /usr/local
27138 drwxrwsr-x 1 root staff 62 Aug 15 2014 /usr/local martin@merkaba:~> ls -lid /usr/local/.
27138 drwxrwsr-x 1 root staff 62 Aug 15 2014 /usr/local/. martin@merkaba:~> ls -lid /usr/local/bin/..
27138 drwxrwsr-x 1 root staff 62 Aug 15 2014 /usr/local/bin/..

Quoted 4 lines

On other filesystems like Ext4 I see the actual number of hardlinks to the directory.
Is this intended behaviour of BTRFS or a bug?

Intended behaviour, this has been asked in the past, I don't have the link sorry, try searching the mailinglist archives.

Martin Steigerwald

author

2015-03-18 13:31 UTC #2 lore raw

Am Dienstag, 17. März 2015, 17:07:17 schrieb David Sterba:

Quoted 10 lines · Martin Steigerwald

Quoted 9 lines
On BTRFS I see
martin@merkaba:~> ls -lid /usr/local
27138 drwxrwsr-x 1 root staff 62 Aug 15 2014 /usr/local martin@merkaba:~> ls -lid /usr/local/.
27138 drwxrwsr-x 1 root staff 62 Aug 15 2014 /usr/local/. martin@merkaba:~> ls -lid /usr/local/bin/..
27138 drwxrwsr-x 1 root staff 62 Aug 15 2014 /usr/local/bin/..

Quoted 7 lines

Quoted 4 lines
On other filesystems like Ext4 I see the actual number of hardlinks to the directory.
Is this intended behaviour of BTRFS or a bug?
Intended behaviour, this has been asked in the past, I don't have the link sorry, try searching the mailinglist archives.

I did, yet I didn´t find anything.

Any idea on what words except "hardlinks", "hard links", "number", "ls -l" I can throw at Baloo fulltext search on BTRFS mailing list folder in KMail?

I also tried various search terms with Startpage.

Thanks,

Signature · 2 lines

Martin 'Helios' Steigerwald - http://www.Lichtvoll.de
GPG: 03B0 0D6C 0040 0710 4AFA  B82F 991B EAAC A599 84C7

David Sterba

2015-03-18 13:52 UTC #3 lore raw

Quoted 18 lines · Martin Steigerwald

Am Dienstag, 17. März 2015, 17:07:17 schrieb David Sterba:
Quoted 17 lines · Martin Steigerwald
Quoted 9 lines
On BTRFS I see
martin@merkaba:~> ls -lid /usr/local
27138 drwxrwsr-x 1 root staff 62 Aug 15 2014 /usr/local martin@merkaba:~> ls -lid /usr/local/.
27138 drwxrwsr-x 1 root staff 62 Aug 15 2014 /usr/local/. martin@merkaba:~> ls -lid /usr/local/bin/..
27138 drwxrwsr-x 1 root staff 62 Aug 15 2014 /usr/local/bin/..
Quoted 4 lines
On other filesystems like Ext4 I see the actual number of hardlinks to the directory.
Is this intended behaviour of BTRFS or a bug?
Intended behaviour, this has been asked in the past, I don't have the link sorry, try searching the mailinglist archives.

http://thread.gmane.org/gmane.comp.file-systems.btrfs/14634

"Directories always have a link count of 1 in btrfs. This tells find not to use the link count as the count of subdirectories in the directory."

http://thread.gmane.org/gmane.comp.file-systems.btrfs/29906

"As I understand it, inferring the number of directory entries from st_nlink is an optimization that isn't universally valid. If that count is 1, it must be considered invalid, and programs that don't handle this correctly are broken. Coreutils handle this, at least..."

Martin Steigerwald

author

2015-03-18 14:23 UTC #4 lore raw

Am Mittwoch, 18. März 2015, 14:52:30 schrieb David Sterba:

Quoted 28 lines · Martin Steigerwald

Quoted 20 lines
Am Dienstag, 17. März 2015, 17:07:17 schrieb David Sterba:
Quoted 19 lines · Martin Steigerwald
Quoted 9 lines
On BTRFS I see
martin@merkaba:~> ls -lid /usr/local
27138 drwxrwsr-x 1 root staff 62 Aug 15 2014 /usr/local martin@merkaba:~> ls -lid /usr/local/.
27138 drwxrwsr-x 1 root staff 62 Aug 15 2014 /usr/local/. martin@merkaba:~> ls -lid /usr/local/bin/..
27138 drwxrwsr-x 1 root staff 62 Aug 15 2014 /usr/local/bin/..
Quoted 5 lines
On other filesystems like Ext4 I see the actual number of hardlinks to
the directory.
Is this intended behaviour of BTRFS or a bug?
Intended behaviour, this has been asked in the past, I don't have the
link sorry, try searching the mailinglist archives.
http://thread.gmane.org/gmane.comp.file-systems.btrfs/14634
"Directories always have a link count of 1 in btrfs. This tells find not to use the link count as the count of subdirectories in the directory."

Quoted 6 lines

http://thread.gmane.org/gmane.comp.file-systems.btrfs/29906
"As I understand it, inferring the number of directory entries from st_nlink is an optimization that isn't universally valid. If that count is 1, it must be considered invalid, and programs that don't handle this correctly are broken. Coreutils handle this, at least..."

Okay, thanks. That was before I subscribed to BTRFS mailinglist so not in local mail archive.

It explains that having a correct hardlink number for directory is not mandatory, but it doesn´t explain why BTRFS always has 1 in there instead of the actual count of hardlinks. Is this an performance optimization for BTRFS or are there any other reasons why BTRFS does it this way?

Thanks,

Signature · 2 lines

Martin 'Helios' Steigerwald - http://www.Lichtvoll.de
GPG: 03B0 0D6C 0040 0710 4AFA  B82F 991B EAAC A599 84C7

David Sterba

2015-03-19 13:21 UTC #5 lore raw

Quoted 4 lines · Martin Steigerwald

It explains that having a correct hardlink number for directory is not mandatory, but it doesn´t explain why BTRFS always has 1 in there instead of the actual count of hardlinks. Is this an performance optimization for BTRFS or are there any other reasons why BTRFS does it this way?

I believe it's for performance reasons. New inodes do not update the parent directory metadata wrt link counts, compared to other filesystems that do that.

The real performance hit could be noticeable. The directory inode is cached in memory, so first update would be a bit slower, but the metadata block needs to be cow-ed on each new file. It's stress on b-tree locking and allocating new buffers for the metadata blocks.

Filipe David Manana

2015-03-19 21:47 UTC #6 lore raw

Quoted 9 lines · David Sterba

Quoted 4 lines · Martin Steigerwald
It explains that having a correct hardlink number for directory is not mandatory, but it doesn´t explain why BTRFS always has 1 in there instead of the actual count of hardlinks. Is this an performance optimization for BTRFS or are there any other reasons why BTRFS does it this way?
I believe it's for performance reasons. New inodes do not update the parent directory metadata wrt link counts, compared to other filesystems that do that.

Weird. Because creating a new inode implies adding the dentry to the parent directory, which implies updating the directory's i_size.

Quoted 9 lines

The real performance hit could be noticeable. The directory inode is cached in memory, so first update would be a bit slower, but the metadata block needs to be cow-ed on each new file. It's stress on b-tree locking and allocating new buffers for the metadata blocks. --
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

Signature · 5 lines

Filipe David Manana,

"Reasonable men adapt themselves to the world.
 Unreasonable men adapt the world to themselves.
 That's why all progress depends on unreasonable men."

Kai Krakow

2015-03-19 23:02 UTC #7 lore raw

Quoted 8 lines · Filipe David Manana

Quoted 7 lines · David Sterba
Quoted 5 lines · Martin Steigerwald
It explains that having a correct hardlink number for directory is not mandatory, but it doesn´t explain why BTRFS always has 1 in there instead of the actual count of hardlinks. Is this an performance optimization for BTRFS or are there any other reasons why BTRFS does it this way?

Quoted 6 lines

Quoted 3 lines
I believe it's for performance reasons. New inodes do not update the parent directory metadata wrt link counts, compared to other filesystems that do that.
Weird. Because creating a new inode implies adding the dentry to the parent directory, which implies updating the directory's i_size.

Maybe related to snapshots? Which brings us back to COW and performance:

Quoted 11 lines

Quoted 8 lines
The real performance hit could be noticeable. The directory inode is cached in memory, so first update would be a bit slower, but the metadata block needs to be cow-ed on each new file. It's stress on b-tree locking and allocating new buffers for the metadata blocks. --
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

Signature · 1 line

Replies to list only preferred.

David Sterba

2015-03-20 10:44 UTC #8 lore raw

In reply to Filipe David Manana

Quoted 7 lines · Filipe David Manana

Quoted 6 lines · David Sterba
Quoted 4 lines · Martin Steigerwald
It explains that having a correct hardlink number for directory is not mandatory, but it doesn´t explain why BTRFS always has 1 in there instead of the actual count of hardlinks. Is this an performance optimization for BTRFS or are there any other reasons why BTRFS does it this way?

Quoted 6 lines

Quoted 3 lines
I believe it's for performance reasons. New inodes do not update the parent directory metadata wrt link counts, compared to other filesystems that do that.
Weird. Because creating a new inode implies adding the dentry to the parent directory, which implies updating the directory's i_size.

Ah right, sorry, I've missed that.

David Sterba

2015-03-20 12:39 UTC #9 lore raw

In reply to Filipe David Manana

Quoted 7 lines · Filipe David Manana

Quoted 6 lines · David Sterba
Quoted 4 lines · Martin Steigerwald
It explains that having a correct hardlink number for directory is not mandatory, but it doesn´t explain why BTRFS always has 1 in there instead of the actual count of hardlinks. Is this an performance optimization for BTRFS or are there any other reasons why BTRFS does it this way?

Quoted 6 lines

Quoted 3 lines
I believe it's for performance reasons. New inodes do not update the parent directory metadata wrt link counts, compared to other filesystems that do that.
Weird. Because creating a new inode implies adding the dentry to the parent directory, which implies updating the directory's i_size.

I wonder why the link count is not maintained then. The directory inode is modified anyway, add a few ifs here and there will not make things worse and we could get rid of this special behaviour, plus find could use the link count reliably.

It can be turned on in a fully backward compatible way:

* new directories/subvolumes will get dir nlink set to 2
* if mkdir/rmdir/move sees a parent link count of 1, no change
* otherwise inc/dec the link count for new subdirectoris/subvolumes

Jeff Liu reported a problem, without further details
http://article.gmane.org/gmane.comp.file-systems.btrfs/14628

so I still might be missing something.

Filipe David Manana

2015-03-20 12:59 UTC #10 lore raw

Quoted 22 lines · David Sterba

Quoted 7 lines · Filipe David Manana
Quoted 6 lines · David Sterba
Quoted 4 lines · Martin Steigerwald
It explains that having a correct hardlink number for directory is not mandatory, but it doesn´t explain why BTRFS always has 1 in there instead of the actual count of hardlinks. Is this an performance optimization for BTRFS or are there any other reasons why BTRFS does it this way?
Quoted 6 lines
Quoted 3 lines
I believe it's for performance reasons. New inodes do not update the parent directory metadata wrt link counts, compared to other filesystems that do that.
Weird. Because creating a new inode implies adding the dentry to the parent directory, which implies updating the directory's i_size.
I wonder why the link count is not maintained then. The directory inode is modified anyway, add a few ifs here and there will not make things worse and we could get rid of this special behaviour, plus find could use the link count reliably.
It can be turned on in a fully backward compatible way:

Quoted 8 lines

* new directories/subvolumes will get dir nlink set to 2
* if mkdir/rmdir/move sees a parent link count of 1, no change
* otherwise inc/dec the link count for new subdirectoris/subvolumes
Jeff Liu reported a problem, without further details
http://article.gmane.org/gmane.comp.file-systems.btrfs/14628
so I still might be missing something.

Maybe this was all before delayed inodes were introduced, where updating inode items always implied touching the btrees.

Signature · 5 lines

Filipe David Manana,

"Reasonable men adapt themselves to the world.
 Unreasonable men adapt the world to themselves.
 That's why all progress depends on unreasonable men."

Martin Steigerwald

author

2015-03-27 10:13 UTC #11 lore raw

Am Freitag, 20. März 2015, 12:59:14 schrieb Filipe David Manana:

Quoted 3 lines · David Sterba

Quoted 2 lines · Filipe David Manana
Quoted 1 line
On Thu, Mar 19, 2015 at 1:21 PM, David Sterba <[email protected]>

Quoted 27 lines · wrote:

Quoted 27 lines
Quoted 7 lines
Quoted 7 lines · Martin Steigerwald
Quoted 6 lines
It explains that having a correct hardlink number for directory is not
mandatory, but it doesn´t explain why BTRFS always has 1 in there instead of the actual count of hardlinks. Is this an performance optimization for BTRFS or are there any other reasons why BTRFS does it this way?>> >
Quoted 7 lines
Quoted 4 lines
I believe it's for performance reasons. New inodes do not update the
parent directory metadata wrt link counts, compared to other filesystems that do that.
Weird. Because creating a new inode implies adding the dentry to the parent directory, which implies updating the directory's i_size.
I wonder why the link count is not maintained then. The directory inode
is modified anyway, add a few ifs here and there will not make things worse and we could get rid of this special behaviour, plus find could use the link count reliably.
It can be turned on in a fully backward compatible way:
* new directories/subvolumes will get dir nlink set to 2
* if mkdir/rmdir/move sees a parent link count of 1, no change
* otherwise inc/dec the link count for new subdirectoris/subvolumes

Quoted 7 lines

Quoted 4 lines
Jeff Liu reported a problem, without further details
http://article.gmane.org/gmane.comp.file-systems.btrfs/14628
so I still might be missing something.
Maybe this was all before delayed inodes were introduced, where updating inode items always implied touching the btrees.

Interesting.

Well this came up due to an exercise to find all hardlinks to /usr/local in a Linux training. And one participant of the training did this on a SLES 11 SP 3 VM with BTRFS.

I did some search and indeed see nothing about link count for directories related to POSIX standard. Incrementing link count only seems to be mentioned for files:

http://pubs.opengroup.org/onlinepubs/009695399/functions/link.html

Even when people tend to rely on this behavior for forensics as in:

http://digital-forensics.sans.org/blog/2009/06/19/directory-link-counts-and-hidden-directories/

Ciao,

Signature · 2 lines

Martin 'Helios' Steigerwald - http://www.Lichtvoll.de
GPG: 03B0 0D6C 0040 0710 4AFA  B82F 991B EAAC A599 84C7