Skip to contentSkip to conversation

time-shifting

DISCUSSION Evert Mouw 2 participants 5 messages Aug 6, 2011 – Aug 8, 2011 4 messages not in the archive NILFS2 filesystem

lore
Share & tools
lore

Evert Mouw

author
#0 lore raw

Hi,

Yesterday I played a bit with NILFS2 on ubuntu 11.04. It was quite easy to use, but finding previous versions of a file is still a bit complicated from a user point of view.

I read a bit about ext3cow, which is currently not actively maintained anymore. In one of the papers on ext3cow, I read the following:

Other disk file systems provide coarse-grained access to versions through
the creation of namespace tunnels [Hitz et al. 1994] or via mounting separate logical volumes [Strunk et al. 2000; Soules et al. 2003]. Some disk file systems provide no interface to versions, restricting versioning to internal use only [Rosenblum and Ousterhout 1992; Seltzer et al. 1993].

In time-shifting, ext3cow introduces an interface to versioning that presents a continuous view of time. Users or applications specify a file name and any point in time, which ext3cow scopes to the appropriate snapshot or file version. The time-shifting interface allows user-space tools to create snapshots and access versions.

Ext3cow: A Time-Shifting File System for Regulatory Compliance ZACHARY N. J. PETERSON and RANDAL BURNS
ACM Transactions on Storage, 1(2), May, 2005
http://www.ext3cow.com/ext3cow/Publications.html

A usage scenario is given in a screenshot:
http://www.ext3cow.com/ext3cow/Welcome_files/example1.jpg

I don't think using the "@" symbol for such thinks is a great idea (I sometimes use it in my filenames) but the ability to use regular tools to access old versions of files is great.

I think it would be really useful to have access to previous versions of files without having to go through checkpoints or snapshots of the whole filesystem / partition. So having something like lshistory or lsversions for specific files would be great. It could output the versions, the checkpoints where those versions are available, etc. Maybe direct retrieval of old versions (cathistory?) could also be possible using such a path.

Of course the architecture of NILFS is different from ext3cow so I don't know whether this is even possible. Something like a data structure linking the different verions of some inode must exist before this can happen. Just an idea, though.

Thanks for your cool stuff!

Evert

--
To unsubscribe from this list: send the line "unsubscribe linux-nilfs" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

#2 lore raw
In reply to a message not in the archive

Hi,

Quoted 46 lines · Evert Mouw

Hi,

Yesterday I played a bit with NILFS2 on ubuntu 11.04. It was quite easy to use, but finding previous versions of a file is still a bit complicated from a user point of view.

I read a bit about ext3cow, which is currently not actively maintained anymore. In one of the papers on ext3cow, I read the following:

Other disk file systems provide coarse-grained access to versions through the creation of namespace tunnels [Hitz et al. 1994] or via mounting separate logical volumes
[Strunk et al. 2000; Soules et al. 2003]. Some disk file systems provide no interface
to versions, restricting versioning to internal use only [Rosenblum and Ousterhout 1992;
Seltzer et al. 1993].

In time-shifting, ext3cow introduces an interface to versioning that presents a continuous
view of time. Users or applications specify a file name and any point in time, which
ext3cow scopes to the appropriate snapshot or file version. The time-shifting interface
allows user-space tools to create snapshots and access versions.

Ext3cow: A Time-Shifting File System for Regulatory Compliance ZACHARY N. J. PETERSON and RANDAL BURNS
ACM Transactions on Storage, 1(2), May, 2005
http://www.ext3cow.com/ext3cow/Publications.html

A usage scenario is given in a screenshot:
http://www.ext3cow.com/ext3cow/Welcome_files/example1.jpg

I don't think using the "@" symbol for such thinks is a great idea (I sometimes use it in my filenames) but the ability to use regular tools to access old versions of files is great.

I think it would be really useful to have access to previous versions of files without having to go through checkpoints or snapshots of the whole filesystem / partition. So having something like lshistory or lsversions for specific files would be great. It could output the versions, the checkpoints where those versions are available, etc. Maybe direct retrieval of old versions (cathistory?) could also be possible using such a path.

Quoted 4 lines

Of course the architecture of NILFS is different from ext3cow so I don't know whether this is even possible. Something like a data structure linking the different verions of some inode must exist before this can happen. Just an idea, though.

If memory serves me right, ext3cow allocates an inode every time its file is changed and a pointer to the previous inode is stored in the new inode.

At present, nilfs does not have such per-file pointer to past versions, so actually it's not optimal for file history lookups.

We may be able to do indirectly by adding a new field to on-disk inode to keep a checkpoint number in which the inode lastly changed. However, the new field must be updated every time garbage collector thins out the previous version, -- this may complicate things unacceptably because the current nilfs is designed so that it never overwrites in-use blocks except super blocks.

Apart from these implementation issues, I don't feel the "@" symbol semantics is odd, though it can come under fire from related kernel developers as a violation of namespace semantics.

With regards,
Ryusuke Konishi

Quoted 3 lines

Thanks for your cool stuff!

Evert

--
To unsubscribe from this list: send the line "unsubscribe linux-nilfs" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

author
#4 lore raw
In reply to a message not in the archive

Hi,

Op 2011-08-07 16:19, Ryusuke Konishi schreef:

Quoted 9 lines

At present, nilfs does not have such per-file pointer to past versions, so actually it's not optimal for file history lookups.

We may be able to do indirectly by adding a new field to on-disk inode to keep a checkpoint number in which the inode lastly changed. However, the new field must be updated every time garbage collector thins out the previous version, -- this may complicate things unacceptably because the current nilfs is designed so that it never overwrites in-use blocks except super blocks.

Very interesting, thanks for your explanation. I agree that updating such a field to a previous checkpoint should not be overwritten. A solution would be to not update the field when the garbage collector runs. The result is that a pointer to a previous checkpoint could point to a non-existing checkpoint. That should not be a problem, because it is known which checkpoints do exist, so a simple "if ( ( previous-checkpoint == X ) and ( X not element of available checkpoints ) ) then previous-checkpoint = NULL" would do. That would enable easier checks for previous versions, although if there is some chechpoint series like "1 2 3 4" and some file exists in all four of those, and if you remove checkpoint 2, then this system will never find the oldest version in checkpoint 1. But such a scenario is unlikely because the garbage collector removes the oldest checkpoints first. Also, this solution doesn't need to be perfect to be better than nothing at all.

Just my 2 cents, for I don't have any filesystem development experience.

Regards, Evert

--
To unsubscribe from this list: send the line "unsubscribe linux-nilfs" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

#6 lore raw
In reply to a message not in the archive
Quoted 12 lines · Evert Mouw

Hi,

Op 2011-08-07 16:19, Ryusuke Konishi schreef:

Quoted 9 lines

At present, nilfs does not have such per-file pointer to past versions, so actually it's not optimal for file history lookups.

We may be able to do indirectly by adding a new field to on-disk inode to keep a checkpoint number in which the inode lastly changed. However, the new field must be updated every time garbage collector thins out the previous version, -- this may complicate things unacceptably because the current nilfs is designed so that it never overwrites in-use blocks except super blocks.

Quoted 9 lines

Very interesting, thanks for your explanation. I agree that updating such a field to a previous checkpoint should not be overwritten. A solution would be to not update the field when the garbage collector runs. The result is that a pointer to a previous checkpoint could point to a non-existing checkpoint. That should not be a problem, because it is known which checkpoints do exist, so a simple "if ( ( previous-checkpoint == X ) and ( X not element of available checkpoints ) ) then previous-checkpoint = NULL" would do. That would enable easier checks for previous versions,

I agree with this let-it-be approach.

Quoted 6 lines

although if there is some chechpoint series like "1 2 3 4" and some file exists in all four of those, and if you remove checkpoint 2, then this system will never find the oldest version in checkpoint
1. But such a scenario is unlikely because the garbage collector removes the oldest checkpoints first. Also, this solution doesn't need to be perfect to be better than nothing at all.

Okay, garbage collector may delete newer checkpoints first, and also intermediate checkpoints can be deleted manually with rmcp command. But, the let-it-be approach seems to go well in most cases. If the previous checkpoint number is lost, we only have to track back to the last valid checkpoint younger than it and confirm that the inode contained in the checkpoint is alive with the same generation number.

Regards,
Ryusuke Konishi
--
To unsubscribe from this list: send the line "unsubscribe linux-nilfs" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html

author
#8 lore raw
In reply to a message not in the archive

Hi,

Op 2011-08-08 14:28, Ryusuke Konishi schreef:

Quoted 8 lines

[...]
I agree with this let-it-be approach. [...] If the previous checkpoint number is lost, we only have to track back to the last valid checkpoint younger than it and confirm that the inode contained in the checkpoint is alive with the same generation number.

Regards,
Ryusuke Konishi

Totally cool.

Nice that it would be possible and workable. Maybe the idea can be put in the todo list? I saw a one points on the todo list that might benefit from this approach:

- Time slider

The time slider for a single file could be implemented by using the proposed extra inode field "previous". (Another approach would be the one currently employed by the TimeBrowse program.)

Keep up the good work!

Regards, Evert

--
To unsubscribe from this list: send the line "unsubscribe linux-nilfs" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html