Allow setting file birth time with utimensat()

From: Omar Sandoval <[email protected]>

Hi,

Since statx was added in 4.11, userspace has had an interface for reading btime (file creation time), but no way to set it. This RFC patch series adds support for changing btime with utimensat(). Patch 1 adds the VFS infrastructure, patch 2 adds the support to utimensat() with a new flag, and the rest of the patches add filesystem support; I excluded CIFS for now because I don't have a CIFS setup to test it on.

Updating btime is useful for at least a couple of use cases:

- Backup/restore programs (my motivation for this feature is btrfs send)
- File servers which interoperate with operating systems that allow updating file creation time, including Mac OS [1] and Windows [2]

Quoted 11 lines · Omar Sandoval

From: Omar Sandoval <[email protected]>

Hi,

Since statx was added in 4.11, userspace has had an interface for reading btime (file creation time), but no way to set it. This RFC patch series adds support for changing btime with utimensat(). Patch 1 adds the VFS infrastructure, patch 2 adds the support to utimensat() with a new flag, and the rest of the patches add filesystem support; I excluded CIFS for now because I don't have a CIFS setup to test it on.

Quoted 5 lines

Updating btime is useful for at least a couple of use cases:

- Backup/restore programs (my motivation for this feature is btrfs send)
- File servers which interoperate with operating systems that allow updating file creation time, including Mac OS [1] and Windows [2]

So you're adding an interface that allows users to change the create time of files without needing any privileges?

Inode create time is forensic metadata in XFS - information we use for sequence of event and inode lifetime analysis during examination of broken filesystem images and systems that have been broken into. Just because it's exposed to userspace via statx(), it doesn't mean that it is information that users should be allowed to change. i.e. allowing users to be able to change the create time on files makes it completely useless for the purpose it was added to XFS for...

And allowing root to change the create time doesn't really help, because once you've broken into a system, this makes it really easy to cover tracks

(e.g. we can't find files that were created and unlinked during the break in window anymore) and lay false trails....

Quoted 18 lines · Dave Chinner

Quoted 16 lines · Omar Sandoval

From: Omar Sandoval <[email protected]>

Hi,

Since statx was added in 4.11, userspace has had an interface for reading btime (file creation time), but no way to set it. This RFC patch series adds support for changing btime with utimensat(). Patch 1 adds the VFS infrastructure, patch 2 adds the support to utimensat() with a new flag, and the rest of the patches add filesystem support; I excluded CIFS for now because I don't have a CIFS setup to test it on.

Updating btime is useful for at least a couple of use cases:

- Backup/restore programs (my motivation for this feature is btrfs send)
- File servers which interoperate with operating systems that allow updating file creation time, including Mac OS [1] and Windows [2]

Quoted 2 lines

So you're adding an interface that allows users to change the create time of files without needing any privileges?

I think it'd be reasonable to make this a privileged operation. I didn't for this initial submission for a couple of reasons:

1. The precedent on Mac OS and Windows is that this isn't a privileged operation.

2. I knew there would be different opinions on this either way I went.

Quoted 8 lines

Inode create time is forensic metadata in XFS - information we use for sequence of event and inode lifetime analysis during examination of broken filesystem images and systems that have been broken into. Just because it's exposed to userspace via statx(), it doesn't mean that it is information that users should be allowed to change. i.e. allowing users to be able to change the create time on files makes it completely useless for the purpose it was added to XFS for...

Quoted 3 lines

And allowing root to change the create time doesn't really help, because once you've broken into a system, this makes it really easy to cover tracks

If the threat model is that the attacker has root, then they can overwrite the timestamp on disk anyways, no?

Quoted 3 lines

(e.g. we can't find files that were created and unlinked during the break in window anymore) and lay false trails....

Fair point, although there's still ctime during the break-in window,

which I assume you'd be looking for anyways since files modified during the break-in window are also of interest.

I see a few options, none of which are particularly nice:

1. Filesystems like XFS could choose not to support setting btime even if they support reading it.
2. XFS could add a second, writeable btime which is used for statx/utimes when available (it would fit in di_pad2...).
3. We could add a btime_writable sysctl/mount option/mkfs option.

From: Omar Sandoval <[email protected]>

Since statx was added in 4.11, userspace has had an interface for reading btime (file creation time), but no way to set it. This RFC patch series adds support for changing btime with utimensat(). Patch 1 adds the VFS infrastructure, patch 2 adds the support to utimensat() with a new flag, and the rest of the patches add filesystem support; I excluded CIFS for now because I don't have a CIFS setup to test it on.

Updating btime is useful for at least a couple of use cases:

- Backup/restore programs (my motivation for this feature is btrfs send)

- File servers which interoperate with operating systems that allow updating file creation time, including Mac OS [1] and Windows [2]

I've also included a man page patch, xfs_io support, and an xfstest.

Thoughts on the implementation or the interface?

Thanks!

1: https://developer.apple.com/library/archive/documentation/System/Conceptual/ManPages_iPhoneOS/man2/setattrlist.2.html
2: https://docs.microsoft.com/en-us/windows/desktop/api/fileapi/nf-fileapi-setfiletime

Omar Sandoval (6):
fs: add btime to struct iattr
fs: add AT_UTIME_BTIME for utimensat()
Btrfs: add support for setting btime
ext4: add support for setting btime
f2fs: add support for setting btime
xfs: add support for setting btime

 fs/attr.c                      |  6 +++
 fs/btrfs/inode.c               |  2 +
 fs/btrfs/super.c               |  4 +-
 fs/ext4/inode.c                | 15 +++++-
 fs/ext4/super.c                |  2 +-
 fs/f2fs/file.c                 | 19 ++++++--
 fs/f2fs/super.c                |  2 +-
 fs/utimes.c                    | 86 +++++++++++++++++++++-------------
 fs/xfs/libxfs/xfs_format.h     |  2 +-
 fs/xfs/libxfs/xfs_log_format.h |  2 +-
 fs/xfs/xfs_iops.c              | 11 ++++-
 fs/xfs/xfs_super.c             |  2 +-
 include/linux/fs.h             |  4 ++
 include/uapi/linux/fcntl.h     |  2 +
 14 files changed, 111 insertions(+), 48 deletions(-)

Hi,

Quoted 9 lines · Omar Sandoval

From: Omar Sandoval <[email protected]>

Since statx was added in 4.11, userspace has had an interface for reading btime (file creation time), but no way to set it. This RFC patch series adds support for changing btime with utimensat(). Patch 1 adds the VFS infrastructure, patch 2 adds the support to utimensat() with a new flag, and the rest of the patches add filesystem support; I excluded CIFS for now because I don't have a CIFS setup to test it on.

Quoted 3 lines

Updating btime is useful for at least a couple of use cases:

- Backup/restore programs (my motivation for this feature is btrfs send)

Can you give an example of such usefulness? What's the thing you run into that you can't do without having this?

Quoted 19 lines · Omar Sandoval

Quoted 18 lines · Dave Chinner

Quoted 16 lines · Omar Sandoval

From: Omar Sandoval <[email protected]>

Hi,

Since statx was added in 4.11, userspace has had an interface for reading btime (file creation time), but no way to set it. This RFC patch series adds support for changing btime with utimensat(). Patch 1 adds the VFS infrastructure, patch 2 adds the support to utimensat() with a new flag, and the rest of the patches add filesystem support; I excluded CIFS for now because I don't have a CIFS setup to test it on.

Updating btime is useful for at least a couple of use cases:

- Backup/restore programs (my motivation for this feature is btrfs send)
- File servers which interoperate with operating systems that allow updating file creation time, including Mac OS [1] and Windows [2]

Quoted 8 lines

Quoted 2 lines

So you're adding an interface that allows users to change the create time of files without needing any privileges?

I think it'd be reasonable to make this a privileged operation. I didn't for this initial submission for a couple of reasons:

1. The precedent on Mac OS and Windows is that this isn't a privileged operation.

Don't really care about them. Interop file servers that support these operations on other OSs will need to be storing this info in xattrs because they have to work on filesystems that don't support btime.

Quoted 1 line

2. I knew there would be different opinions on this either way I went.

Yup.

Quoted 8 lines

Quoted 8 lines

Inode create time is forensic metadata in XFS - information we use for sequence of event and inode lifetime analysis during examination of broken filesystem images and systems that have been broken into. Just because it's exposed to userspace via statx(), it doesn't mean that it is information that users should be allowed to change. i.e. allowing users to be able to change the create time on files makes it completely useless for the purpose it was added to XFS for...

Quoted 6 lines

Quoted 3 lines

And allowing root to change the create time doesn't really help, because once you've broken into a system, this makes it really easy to cover tracks

If the threat model is that the attacker has root, then they can overwrite the timestamp on disk anyways, no?

Modifying the block devicee under an active filesystem is fraught with danger, and there's no guarantee it will work if the metadata being modified is still active in memory. Corrupting the filesystem is a sure way to get noticed....

Quoted 5 lines

Quoted 3 lines

(e.g. we can't find files that were created and unlinked during the break in window anymore) and lay false trails....

Fair point, although there's still ctime during the break-in window,

Unless you're smart enough to know how to trigger S_NOCMTIME or FMODE_NOCMTIME....

Quoted 2 lines

which I assume you'd be looking for anyways since files modified during the break-in window are also of interest.

... and then that also can't be guaranteed. :/

Quoted 7 lines

I see a few options, none of which are particularly nice:

1. Filesystems like XFS could choose not to support setting btime even if they support reading it.
2. XFS could add a second, writeable btime which is used for statx/utimes when available (it would fit in di_pad2...).
3. We could add a btime_writable sysctl/mount option/mkfs option.

4. create time remains a read-only field, and btrfs grows its own special interface to twiddle it in btrfs-recv if it really is necessary.

I'm still not convinced that even backup/restore should be doing this, because there's so much other metadata that is unique even on restored files that it doesn't really make any sense to me to lie about it being created in the past....

Quoted 2 lines · Dave Chinner

Quoted 1 line · Omar Sandoval

On Fri, Feb 15, 2019 at 09:06:26AM +1100, Dave Chinner wrote:

Quoted 24 lines

Quoted 8 lines

Quoted 8 lines

Inode create time is forensic metadata in XFS - information we use for sequence of event and inode lifetime analysis during examination of broken filesystem images and systems that have been broken into. Just because it's exposed to userspace via statx(), it doesn't mean that it is information that users should be allowed to change. i.e. allowing users to be able to change the create time on files makes it completely useless for the purpose it was added to XFS for...

Quoted 6 lines

Quoted 3 lines

And allowing root to change the create time doesn't really help, because once you've broken into a system, this makes it really easy to cover tracks

If the threat model is that the attacker has root, then they can overwrite the timestamp on disk anyways, no?

Modifying the block devicee under an active filesystem is fraught with danger, and there's no guarantee it will work if the metadata being modified is still active in memory. Corrupting the filesystem is a sure way to get noticed....

Quoted 4 lines

Quoted 3 lines

(e.g. we can't find files that were created and unlinked during the break in window anymore) and lay false trails....

Quoted 7 lines

Quoted 1 line

Fair point, although there's still ctime during the break-in window,

Unless you're smart enough to know how to trigger S_NOCMTIME or FMODE_NOCMTIME....

Quoted 2 lines

which I assume you'd be looking for anyways since files modified during the break-in window are also of interest.

Quoted 10 lines

... and then that also can't be guaranteed. :/

Quoted 7 lines

I see a few options, none of which are particularly nice:

1. Filesystems like XFS could choose not to support setting btime even if they support reading it.
2. XFS could add a second, writeable btime which is used for statx/utimes when available (it would fit in di_pad2...).
3. We could add a btime_writable sysctl/mount option/mkfs option.

Quoted 3 lines

4. create time remains a read-only field, and btrfs grows its own special interface to twiddle it in btrfs-recv if it really is necessary.

I'm curious to hear what the ext4/f2fs/CIFS developers think. If no one else wants btime to be mutable, then I might as well make it Btrfs-specific. That is, assuming we reach consensus on the Btrfs side that btrfs receive should set btime.

Quoted 12 lines · Hans van Kranenburg

Hi,

Quoted 9 lines · Omar Sandoval

From: Omar Sandoval <[email protected]>

Since statx was added in 4.11, userspace has had an interface for reading btime (file creation time), but no way to set it. This RFC patch series adds support for changing btime with utimensat(). Patch 1 adds the VFS infrastructure, patch 2 adds the support to utimensat() with a new flag, and the rest of the patches add filesystem support; I excluded CIFS for now because I don't have a CIFS setup to test it on.

Quoted 6 lines

Quoted 3 lines

Updating btime is useful for at least a couple of use cases:

- Backup/restore programs (my motivation for this feature is btrfs send)

Can you give an example of such usefulness? What's the thing you run into that you can't do without having this?

That boils down to what's useful about having the file creation time, and it's really just another tidbit of information which you may or may not care about. Maybe you have a document that you've been editing for awhile, and you want to know when you started working on it. Or, you want to know when a user created some directory that they keep adding files to.

If the file creation time is useful to you, then you likely want it preserved if you have to restore from backups. If I had to restore from backups yesterday and I'm trying to figure out when I started that document, I don't care that I restored that file yesterday, I want the real creation date.

If you've never wondered when a file was created, then I'm sure you won't care whether btrfs send/receive preserves it :)

Quoted 9 lines · Dave Chinner

Quoted 7 lines · Omar Sandoval

I see a few options, none of which are particularly nice:

1. Filesystems like XFS could choose not to support setting btime even if they support reading it.
2. XFS could add a second, writeable btime which is used for statx/utimes when available (it would fit in di_pad2...).
3. We could add a btime_writable sysctl/mount option/mkfs option.

Quoted 3 lines

4. create time remains a read-only field, and btrfs grows its own special interface to twiddle it in btrfs-recv if it really is necessary.

I'm curious to hear what the ext4/f2fs/CIFS developers think. If no one else wants btime to be mutable, then I might as well make it Btrfs-specific. That is, assuming we reach consensus on the Btrfs side that btrfs receive should set btime.

Quoted 4 lines

I'm still not convinced that even backup/restore should be doing this, because there's so much other metadata that is unique even on restored files that it doesn't really make any sense to me to lie about it being created in the past....

I suppose it depends on how you interpret btime: if it's strictly filesystem metadata, then it makes sense that it should be immutable; if it's metadata for the user's own purposes, then we should allow setting it.

Quoted 11 lines · Omar Sandoval

From: Omar Sandoval <[email protected]>

Hi,

Since statx was added in 4.11, userspace has had an interface for reading btime (file creation time), but no way to set it. This RFC patch series adds support for changing btime with utimensat(). Patch 1 adds the VFS infrastructure, patch 2 adds the support to utimensat() with a new flag, and the rest of the patches add filesystem support; I excluded CIFS for now because I don't have a CIFS setup to test it on.

Quoted 2 lines

Updating btime is useful for at least a couple of use cases:

Quoted 3 lines

- Backup/restore programs (my motivation for this feature is btrfs send)
- File servers which interoperate with operating systems that allow updating file creation time, including Mac OS [1] and Windows [2]

So you're adding an interface that allows users to change the create time of files without needing any privileges?

Inode create time is forensic metadata in XFS - information we use for sequence of event and inode lifetime analysis during examination of broken filesystem images and systems that have been broken into. Just because it's exposed to userspace via statx(), it doesn't mean that it is information that users should be allowed to change. i.e. allowing users to be able to change the create time on files makes it completely useless for the purpose it was added to XFS for...

Quoted 6 lines · Dave Chinner

So you're adding an interface that allows users to change the create time of files without needing any privileges?

Inode create time is forensic metadata in XFS - information we use for sequence of event and inode lifetime analysis during examination of broken filesystem images and systems that have been broken into.

I think the difference in opinion here is that there are two totally different BTIme out in the world. For two somewhat opposite motivations and it seems they both try to be crammed into the same on disk space.

One - Author creation time
Two - Local creation time

So it looks like both sides are correct trying to preserve their own guy?

Quoted 8 lines · Boaz Harrosh

Quoted 6 lines · Dave Chinner

So you're adding an interface that allows users to change the create time of files without needing any privileges?

Inode create time is forensic metadata in XFS - information we use for sequence of event and inode lifetime analysis during examination of broken filesystem images and systems that have been broken into.

Quoted 8 lines

I think the difference in opinion here is that there are two totally different BTIme out in the world. For two somewhat opposite motivations and it seems they both try to be crammed into the same on disk space.

One - Author creation time
Two - Local creation time

So it looks like both sides are correct trying to preserve their own guy?

I'd say that [2] is too easily gameable to be worth the effort. You can just change it on the disk. That right now it'd take some skill to find the right place to edit doesn't matter -- a tool to update the btime against your wishes would need to be written just once. Unlike btrfs, XFS doesn't even have a chain of checksums all the way to the root.

On the other hand, [1] has a lot of uses. It can also be preserved in backups and version control (svnt and git-restore-mtime could be easily extended).

I'd thus go with [2] -- any uses for [1] are better delegated to filesystem specific tools.

While it may be a bit of a stretch to call this "forensic evidence", making

it hard to change from except via total root compromise by a skilled hacker is very useful.

If this were to go in (which I'm not in favour of), then there would need to be a CONFIG and/or runtime knob to turn it off (or better to only turn it on), similar to how FIPS and other security options can only go in one direction.

So I'm highly in favor of this patch. If XFS wants to disallow writing the birth time, fine, but I think that behavior should be overridable.

Quoted 3 lines · Andy Lutomirski

So I'm highly in favor of this patch. If XFS wants to disallow writing the birth time, fine, but I think that behavior should be overridable.

Please, no. We need to have consistent behaviour between at least Linux local filesystems. Not "Chris thinks this is a good idea, while Dave and Ted think its a bad idea, so btrfs supports it and XFS and ext4 disallow it".

Quoted 12 lines · Dave Chinner

Quoted 11 lines · Omar Sandoval

From: Omar Sandoval <[email protected]>

Hi,

Since statx was added in 4.11, userspace has had an interface for reading btime (file creation time), but no way to set it. This RFC patch series adds support for changing btime with utimensat(). Patch 1 adds the VFS infrastructure, patch 2 adds the support to utimensat() with a new flag, and the rest of the patches add filesystem support; I excluded CIFS for now because I don't have a CIFS setup to test it on.

Quoted 2 lines

Quoted 2 lines

Updating btime is useful for at least a couple of use cases:

[1]

Quoted 7 lines

Quoted 3 lines

- Backup/restore programs (my motivation for this feature is btrfs send)
- File servers which interoperate with operating systems that allow updating file creation time, including Mac OS [1] and Windows [2]

So you're adding an interface that allows users to change the create time of files without needing any privileges?

[2]

Quoted 8 lines

Inode create time is forensic metadata in XFS - information we use for sequence of event and inode lifetime analysis during examination of broken filesystem images and systems that have been broken into. Just because it's exposed to userspace via statx(), it doesn't mean that it is information that users should be allowed to change. i.e. allowing users to be able to change the create time on files makes it completely useless for the purpose it was added to XFS for...

<snap>

I think the difference in opinion here is that there are two totally different BTIme out in the world. For two somewhat opposite motivations and it seems they both try to be crammed into the same on disk space.

One - Author creation time
This is a Windows originated creature and later MAC (and all vendors who make a living by serving cifs (hint see my email address))

This is a tag carried globally on the globe denoting the time of the original creator of the file. copy, download, backup-restore and so on preserve it from the very first original creation.
This creature is a user oriented information. That needs to be carefully orchestrated by all parties.

From: Omar Sandoval <[email protected]>
Since statx was added in 4.11, userspace has had an interface for reading btime (file creation time), but no way to set it. This RFC patch series adds support for changing btime with utimensat(). Patch 1 adds the VFS infrastructure, patch 2 adds the support to utimensat() with a new flag, and the rest of the patches add filesystem support; I excluded CIFS for now because I don't have a CIFS setup to test it on.

Updating btime is useful for at least a couple of use cases:

- Backup/restore programs (my motivation for this feature is btrfs send)
- File servers which interoperate with operating systems that allow updating file creation time, including Mac OS [1] and Windows [2]

Quoted 1 line · Andreas Dilger

While it may be a bit of a stretch to call this "forensic evidence", making

We do forensic analysis of corrupt filesystems looking for evidence of what went wrong, not just looking for evidence of what happened on systems that have been broken into.

Quoted 2 lines

it hard to change from except via total root compromise by a skilled hacker is very useful.

*nod*.

Quoted 3 lines

If this were to go in (which I'm not in favour of), then there would need to be a CONFIG and/or runtime knob to turn it off (or better to only turn it on), similar to how FIPS and other security options can only go in one direction.

The problem here is that "inode birth time" is being conflated with "user document creation time". These two things are very different.

i.e. One is filesystem internal information and is not related to when the original copy of the data in the file was created, the other is user specified metadata that is related to the file data contents and needs to travel with the data, not the filesystem.

IMO, trying to make one on-disk field hold two different types of information defeats one or the other purpose, and nobody knows which one the field stores for any given file.

I'd suggest that "authored date" should be a generic system xattr so most filesystems support it, not just those that have a birth time field on disk. Sure, modify it through utimesat() and expose it through statx() (as authored time, not birth time), but store it a system xattr rather than an internal filesystem metadata field that requires was never intended to be user modifiable.

Quoted 29 lines · Dave Chinner

Quoted 1 line · Andreas Dilger

While it may be a bit of a stretch to call this "forensic evidence",

We do forensic analysis of corrupt filesystems looking for evidence of what went wrong, not just looking for evidence of what happened on systems that have been broken into.

Quoted 2 lines

making it hard to change from except via total root compromise by a skilled hacker is very useful.

*nod*.

Quoted 4 lines

If this were to go in (which I'm not in favour of), then there would need to be a CONFIG and/or runtime knob to turn it off (or better to only turn it on), similar to how FIPS and other security options can only go in one direction.

The problem here is that "inode birth time" is being conflated with "user document creation time". These two things are very different.

i.e. One is filesystem internal information and is not related to when the original copy of the data in the file was created, the other is user specified metadata that is related to the file data contents and needs to travel with the data, not the filesystem.

IMO, trying to make one on-disk field hold two different types of information defeats one or the other purpose, and nobody knows which one the field stores for any given file.

Quoted 6 lines

I'd suggest that "authored date" should be a generic system xattr so most filesystems support it, not just those that have a birth time field on disk. Sure, modify it through utimesat() and expose it through statx() (as authored time, not birth time), but store it a system xattr rather than an internal filesystem metadata field that requires was never intended to be user modifiable.

It seems that this is the general consensus, so I'll look into implementing this functionality as an xattr.