Skip to contentSkip to conversation

searching checkpoints contaning an inode (or a filename)

DISCUSSION Taesoo Kim 4 participants 8 messages Jul 5, 2013 – Jul 6, 2013 4 messages not in the archive NILFS2 filesystem

lore
Share & tools
lore

Taesoo Kim

author
#0 lore raw

Hi nilfs,

I am evaluating nilfs for using it in my machine. There are a few questions regarding usages:

q1. how can I enumerate a list of cps, given a filename?
(I know I can mount each cp to check whether there is my file or not, as time-machine (in mac) does)

q2. compute diff or cat against previous files in a specific cp?
(instead of mounting and copying them individually to my /tmp)

q3. using nilfs as rootfs?
(I couldn't find any people using nilfs for rootfs. Are there any issue on that? or just matter of performance? on updates for example)

Nilfs looks conceptually nice and clean, so I can trust the implementation or jump directly into modifying code. Do you have any design notes (other than one in /Documentation and overview-nilfs1.pdf).

Thank you in advance.
Taesoo
--
To unsubscribe from this list: send the line "unsubscribe linux-nilfs" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

A message that is not in the archive (sent off-list or not yet received): [email protected]

#2 lore raw
Quoted 5 lines · Taesoo Kim

q3. using nilfs as rootfs?
(I couldn't find any people using nilfs for rootfs. Are there any issue on that? or just matter of performance? on updates for example)

I'm using NILFS as root filesystem on my work laptop for a total of 4 years. I switched to BTRFS for a year in between but looking at how fast it kills SSD, it wass not worthwhile.

Stability wise, I only had 2 problems with NILFS, the first, which was fixed promptly, caused a total data lost because of how the cleaner scrub the disk (using the time), it scrubbed data from the future after time ran backward. Then the other problem was a kernel OOPS about a file from the debian APT cache, I deleted it and it runs just fine now, I couldn't take the backtrace.

Performance wise, on SSD the filesystem is really fast and snappy, while on HDD, reads are really slow, booting takes 2 minutes instead of less than 10 seconds, however, once booted and everything is in cache, it is quite fast. Database loads, especially reads on HDD is the worst for NILFS.
--
To unsubscribe from this list: send the line "unsubscribe linux-nilfs" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

A message that is not in the archive (sent off-list or not yet received): CALJXSJrLo_5=e-M41vXyuS9SbRyeZ=zjLYv2Hs3a8Dt+BwS6ig-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org

#4 lore raw

Hi Jérôme,

Quoted 11 lines · Jérôme Poulin
Quoted 5 lines · Taesoo Kim

q3. using nilfs as rootfs?
(I couldn't find any people using nilfs for rootfs. Are there any issue on that? or just matter of performance? on updates for example)

I'm using NILFS as root filesystem on my work laptop for a total of 4 years. I switched to BTRFS for a year in between but looking at how fast it kills SSD, it wass not worthwhile.

Quoted 7 lines

Stability wise, I only had 2 problems with NILFS, the first, which was fixed promptly, caused a total data lost because of how the cleaner scrub the disk (using the time), it scrubbed data from the future after time ran backward. Then the other problem was a kernel OOPS about a file from the debian APT cache, I deleted it and it runs just fine now, I couldn't take the backtrace.

Could you share more details about the issue? I need to understand the reproducing path of the issue more clear. Can you reproduce it again?

Thanks,
Vyacheslav Dubeyko.

Quoted 9 lines

Performance wise, on SSD the filesystem is really fast and snappy, while on HDD, reads are really slow, booting takes 2 minutes instead of less than 10 seconds, however, once booted and everything is in cache, it is quite fast. Database loads, especially reads on HDD is the worst for NILFS.
--
To unsubscribe from this list: send the line "unsubscribe linux-nilfs" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

--
To unsubscribe from this list: send the line "unsubscribe linux-nilfs" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

#5 lore raw
In reply to a message not in the archive

Hi Jérôme,

Quoted 11 lines · Jérôme Poulin
Quoted 5 lines · Taesoo Kim

q3. using nilfs as rootfs?
(I couldn't find any people using nilfs for rootfs. Are there any issue on that? or just matter of performance? on updates for example)

I'm using NILFS as root filesystem on my work laptop for a total of 4 years. I switched to BTRFS for a year in between but looking at how fast it kills SSD, it wass not worthwhile.

Quoted 7 lines

Stability wise, I only had 2 problems with NILFS, the first, which was fixed promptly, caused a total data lost because of how the cleaner scrub the disk (using the time), it scrubbed data from the future after time ran backward. Then the other problem was a kernel OOPS about a file from the debian APT cache, I deleted it and it runs just fine now, I couldn't take the backtrace.

Could you share more details about the issue? I need to understand the reproducing path of the issue more clear. Can you reproduce it again?

Thanks,
Vyacheslav Dubeyko.

Quoted 9 lines

Performance wise, on SSD the filesystem is really fast and snappy, while on HDD, reads are really slow, booting takes 2 minutes instead of less than 10 seconds, however, once booted and everything is in cache, it is quite fast. Database loads, especially reads on HDD is the worst for NILFS.
--
To unsubscribe from this list: send the line "unsubscribe linux-nilfs" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

--
To unsubscribe from this list: send the line "unsubscribe linux-nilfs" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

#6 lore raw
In reply to a message not in the archive

Hi Taesoo,

Quoted 9 lines · Taesoo Kim

Hi nilfs,

I am evaluating nilfs for using it in my machine. There are a few questions regarding usages:

q1. how can I enumerate a list of cps, given a filename? (I know I can mount each cp to check whether there is my file or not, as time-machine (in mac) does)

You can use dumpseg fot this goal. To be honest this way not so simple. You can get inode id from ls output ("ls -i"), for example. And then you need to get dumpseg for all segments and select output for searching inode. Of course, it needs to implement special feature for easy retrieving such data.

Quoted 3 lines

q2. compute diff or cat against previous files in a specific cp? (instead of mounting and copying them individually to my /tmp)

As I understand, currently, we haven't this as feature of any NILFS2's tool or internal file system technique.

Quoted 5 lines

q3. using nilfs as rootfs?
(I couldn't find any people using nilfs for rootfs. Are there any issue on that? or just matter of performance? on updates for example)

Many people are using NILFS2 as rootfs file system. Currently, we have several reports about serious bugs in NILFS2 from such people. I am investigating these issues.

Quoted 4 lines

Nilfs looks conceptually nice and clean, so I can trust the implementation or jump directly into modifying code. Do you have any design notes (other than one in /Documentation and overview-nilfs1.pdf).

I can share my version of design document. But I don't end it and this document contains some mistakes in description.

http://dubeyko.com/development/FileSystems/NILFS/nilfs2-design.pdf.

So, as I understand, you are going to make some implementation in NILFS2. Could you share what feature you want to implement? Maybe, it make sense to implement above-mentioned by you features. Anyway, it makes sense to distribute tasks.

PS: I am on vacation now. So, sometimes I haven't access to my e-mail.

Thanks,
Vyacheslav Dubeyko.

Quoted 6 lines

Thank you in advance.
Taesoo
--
To unsubscribe from this list: send the line "unsubscribe linux-nilfs" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

--
To unsubscribe from this list: send the line "unsubscribe linux-nilfs" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

A message that is not in the archive (sent off-list or not yet received): B16ADF79-9EF9-46E8-95F8-9D4815E86C64-yeENwD64cLxBDgjK7y7TUQ@public.gmane.org

#8 lore raw
Quoted 13 lines · Vyacheslav Dubeyko

Hi Taesoo,

Quoted 9 lines · Taesoo Kim

Hi nilfs,

I am evaluating nilfs for using it in my machine. There are a few questions regarding usages:

q1. how can I enumerate a list of cps, given a filename? (I know I can mount each cp to check whether there is my file or not, as time-machine (in mac) does)

I think it's hard to implement this feature efficiently because nilfs doesn't have an efficient lookup method of past inodes from a file name.

Quoted 5 lines

You can use dumpseg fot this goal. To be honest this way not so simple. You can get inode id from ls output ("ls -i"), for example. And then you need to get dumpseg for all segments and select output for searching inode. Of course, it needs to implement special feature for easy retrieving such data.

dumpseg is a debug tool and it doesn't provide a solid way since it directly scans on-disk blocks from userland.

We should adds an ioctl for that if there is a reasonable purpose, but I don't know it can be implemented nicely.

Quoted 6 lines
Quoted 3 lines

q2. compute diff or cat against previous files in a specific cp? (instead of mounting and copying them individually to my /tmp)

As I understand, currently, we haven't this as feature of any NILFS2's tool or internal file system technique.

An experimental patchset and a tool exist:

http://www.mail-archive.com/[email protected]/msg00827.html

nilfs-diff:

git://github.com/konis/nilfs-utils.git (diff branch)

diff API:

git://github.com/konis/nilfs2.git (diffapi branch)

But I have never sent this to the mainline because the patchset has important limitations for practical use in addition to needing a disk format change.

Regards,
Ryusuke Konishi
--
To unsubscribe from this list: send the line "unsubscribe linux-nilfs" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

A message that is not in the archive (sent off-list or not yet received): 20130706.185038.523614087.konishi.ryusuke-Zyj7fXuS5i5L9jVzuh4AOg@public.gmane.org

author
#10 lore raw

Hi Ryusuke and Vyacheslav,

Quoted 15 lines
Quoted 4 lines
Quoted 4 lines

q1. how can I enumerate a list of cps, given a filename? (I know I can mount each cp to check whether there is my file or not, as time-machine (in mac) does)

I think it's hard to implement this feature efficiently because nilfs doesn't have an efficient lookup method of past inodes from a file name.

Quoted 5 lines

You can use dumpseg fot this goal. To be honest this way not so simple. You can get inode id from ls output ("ls -i"), for example. And then you need to get dumpseg for all segments and select output for searching inode. Of course, it needs to implement special feature for easy retrieving such data.

Quoted 6 lines

dumpseg is a debug tool and it doesn't provide a solid way since it directly scans on-disk blocks from userland.

We should adds an ioctl for that if there is a reasonable purpose, but I don't know it can be implemented nicely.

I don't know how difficult to embed a backward link (to previous cp) in each inode? So given a current inode, I can walk inode-btree of cp again to find the previous one, possibly with ioctl().

But, searching inodes of a deleted path seems still difficult and hard to implement nicely, without walking all cps to figure out.

Quoted 15 lines
Quoted 6 lines
Quoted 3 lines

q2. compute diff or cat against previous files in a specific cp? (instead of mounting and copying them individually to my /tmp)

As I understand, currently, we haven't this as feature of any NILFS2's tool or internal file system technique.

An experimental patchset and a tool exist:

http://www.mail-archive.com/[email protected]/msg00827.html

nilfs-diff:

git://github.com/konis/nilfs-utils.git (diff branch)

Quoted 9 lines

diff API:

git://github.com/konis/nilfs2.git (diffapi branch)

But I have never sent this to the mainline because the patchset has important limitations for practical use in addition to needing a disk format change.

I will definitely check the patch set, and thank you for the design doc as well.

Thank you!
Taesoo

Quoted 3 lines

Regards,
Ryusuke Konishi

--
To unsubscribe from this list: send the line "unsubscribe linux-nilfs" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

#11 lore raw
In reply to a message not in the archive

Hi Taesoo,

Quoted 9 lines · Taesoo Kim

Hi nilfs,

I am evaluating nilfs for using it in my machine. There are a few questions regarding usages:

q1. how can I enumerate a list of cps, given a filename? (I know I can mount each cp to check whether there is my file or not, as time-machine (in mac) does)

You can use dumpseg fot this goal. To be honest this way not so simple. You can get inode id from ls output ("ls -i"), for example. And then you need to get dumpseg for all segments and select output for searching inode. Of course, it needs to implement special feature for easy retrieving such data.

Quoted 3 lines

q2. compute diff or cat against previous files in a specific cp? (instead of mounting and copying them individually to my /tmp)

As I understand, currently, we haven't this as feature of any NILFS2's tool or internal file system technique.

Quoted 5 lines

q3. using nilfs as rootfs?
(I couldn't find any people using nilfs for rootfs. Are there any issue on that? or just matter of performance? on updates for example)

Many people are using NILFS2 as rootfs file system. Currently, we have several reports about serious bugs in NILFS2 from such people. I am investigating these issues.

Quoted 4 lines

Nilfs looks conceptually nice and clean, so I can trust the implementation or jump directly into modifying code. Do you have any design notes (other than one in /Documentation and overview-nilfs1.pdf).

I can share my version of design document. But I don't end it and this document contains some mistakes in description.

http://dubeyko.com/development/FileSystems/NILFS/nilfs2-design.pdf.

So, as I understand, you are going to make some implementation in NILFS2. Could you share what feature you want to implement? Maybe, it make sense to implement above-mentioned by you features. Anyway, it makes sense to distribute tasks.

PS: I am on vacation now. So, sometimes I haven't access to my e-mail.

Thanks,
Vyacheslav Dubeyko.

Quoted 6 lines

Thank you in advance.
Taesoo
--
To unsubscribe from this list: send the line "unsubscribe linux-nilfs" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

--
To unsubscribe from this list: send the line "unsubscribe linux-nilfs" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html