ntfs: add ioctl support for named data streams
Subsystems (3)
Namjae Jeon
maintainerPATCH 1/1 ntfs: add ioctl support for named data streams
Sebastian and Cedric requested NTFS driver support for an interface that WINE can use to access named data streams.
NTFS supports multiple named $DATA attributes, commonly known as alternate data streams, but they are not directly accessible through the existing file operations. Add NTFS-specific ioctl commands to let userspace list, read, write, and remove named data streams on an opened file.
The new UAPI provides separate commands for each operation:
- NTFS_IOC_STREAM_READ
- NTFS_IOC_STREAM_WRITE
- NTFS_IOC_STREAM_REMOVE
- NTFS_IOC_LIST_STREAMS
The read and write commands use an offset and length based request format. Writes create the named stream when it does not already exist. The list command returns variable-length stream entries containing stream size, allocated size, and stream name.
Stream names use the mounted filesystem NLS by default, or raw UTF-16LE when the UTF-16 flag is requested. Read and write transfers are limited to 16 MiB per request. If the list buffer is too small, the list command returns -ENOSPC while reporting the required size and stream count.
--- .../userspace-api/ioctl/ioctl-number.rst | 1 + fs/ntfs/file.c | 509 ++++++++++++++++++ include/uapi/linux/ntfs.h | 115 ++++ 3 files changed, 625 insertions(+) create mode 100644 include/uapi/linux/ntfs.h
Documentation/userspace-api/ioctl/ioctl-number.rst +1 -0
@@ -399,6 +399,7 @@ Code Seq# Include File Comments 0xE5 00-3F linux/fuse.h 0xEC 00-01 drivers/platform/chrome/cros_ec_dev.h ChromeOS EC driver 0xEE 00-09 uapi/linux/pfrut.h Platform Firmware Runtime Update and Telemetry +0xEF 00-0F uapi/linux/ntfs.h NTFS 0xF3 00-3F drivers/usb/misc/sisusbvga/sisusb.h sisfb (in development) <mailto:[email protected]> 0xF6 all LTTng Linux Trace Toolkit Next Generation
fs/ntfs/file.c +509 -0
533 lines not on this page · raw message
include/uapi/linux/ntfs.h new file +115 -0
115 lines not on this page · raw message
Signature · 1 line
2.25.1
Hyunchul Lee
maintainerfs/ntfs/file.c static int ntfs_ioctl_fitrim(struct ntfs_volume *vol, unsigned long arg) · line 997
Quoted 5 lines
fs/ntfs/file.c +5
-0+ + bytes =ntfs_inode_attr_pwrite (attr_vi, offset, len, data_stream, + false); +if (bytes <0 ) + err = bytes;
offset is u64, but ntfs_inode_attr_pwrite takes s64. If offset is U64_MAX and len = 1, pos becomes -1. A resident stream can the reach memcpy(addr + pos, ...), causing an out-of-bounds write.
fs/ntfs/file.c static int ntfs_ioctl_fitrim(struct ntfs_volume *vol, unsigned long arg) · line 1028
Quoted 7 lines
fs/ntfs/file.c +7
-0+static int ntfs_remove_named_stream (struct ntfs_inode *ni,__le16 *uname, +u32 uname_len) +{ +if (!ni || !uname || uname_len ==0 ) +return -EINVAL ; + +return ntfs_attr_remove (ni,AT_DATA , uname, uname_len);
Dirty folios are not guaranteed to be evicted before ntfs_attr_rm() frees clusters. So invalidating page caches seem to be needed.
And we have to serialize ntfs_attr_remove() against other mft record modifications.
Quoted 7 lines
fs/ntfs/file.c +7
-0+case NTFS_STREAM_OP_WRITE : + err =mnt_want_write_file (filp); +if (err) +break ; + err = ntfs_write_named_stream(ni, sname, sname_len, + req->stream_offset, req->io_len, + kdata,
fs/ntfs/file.c static int ntfs_ioctl_fitrim(struct ntfs_volume *vol, unsigned long arg) · line 1169
Quoted 10 lines
fs/ntfs/file.c +10
-0+ &req->bytes_returned); +mnt_drop_write_file (filp); +if (!err) { +if (copy_to_user (&ureq->bytes_returned, + &req->bytes_returned, +sizeof (req->bytes_returned))) + err = -EFAULT ; + } +break ; +
We have to check NVolShutdown().
Quoted 9 lines · Namjae Jeon
Sebastian and Cedric requested NTFS driver support for an interface that WINE can use to access named data streams.
NTFS supports multiple named $DATA attributes, commonly known as alternate data streams, but they are not directly accessible through the existing file operations. Add NTFS-specific ioctl commands to let userspace list, read, write, and remove named data streams on an opened file.
Quoted 5 lines
The new UAPI provides separate commands for each operation:
-
NTFS_IOC_STREAM_READ
-NTFS_IOC_STREAM_WRITE
-NTFS_IOC_STREAM_REMOVE
Wouldn't it be better to just provide an API to get a fd, relative to a filename? Win32 named streams can be sparse, and it is common to punch holes to deallocate sections of streams too. Streams do have their own timestamps, too.
Quoted 1 line
-
NTFS_IOC_LIST_STREAMS
NTFS_IOC_LIST_STREAMS looks OK.
Lionel
Namjae Jeon
maintainerQuoted 13 lines · Lionel Cons
Quoted 11 lines · Namjae Jeon
Sebastian and Cedric requested NTFS driver support for an interface that WINE can use to access named data streams.
NTFS supports multiple named $DATA attributes, commonly known as alternate data streams, but they are not directly accessible through the existing file operations. Add NTFS-specific ioctl commands to let userspace list, read, write, and remove named data streams on an opened file.
The new UAPI provides separate commands for each operation:
Quoted 8 lines
Quoted 3 lines
-
NTFS_IOC_STREAM_READ
-NTFS_IOC_STREAM_WRITE
-NTFS_IOC_STREAM_REMOVEWouldn't it be better to just provide an API to get a fd, relative to a filename? Win32 named streams can be sparse, and it is common to punch holes to deallocate sections of streams too. Streams do have their own timestamps, too.
I believe the current ioctl interface is the best fit for this support. It operates on an already-open base-file fd, avoids pathname ambiguity with :, supports UTF-16 stream names, and does not require a new system call or VFS changes. If WINE requires hole punching for sparse streams, I can extend the ioctl interface to support it.
Quoted 4 lines
Quoted 2 lines
-
NTFS_IOC_LIST_STREAMS
NTFS_IOC_LIST_STREAMSlooks OK.
Thanks!
Quoted 19 lines · Namjae Jeon
Quoted 17 lines · Lionel Cons
Quoted 14 lines · Namjae Jeon
Sebastian and Cedric requested NTFS driver support for an interface that WINE can use to access named data streams.
NTFS supports multiple named $DATA attributes, commonly known as alternate data streams, but they are not directly accessible through the existing file operations. Add NTFS-specific ioctl commands to let userspace list, read, write, and remove named data streams on an opened file.
The new UAPI provides separate commands for each operation:
-
NTFS_IOC_STREAM_READ
-NTFS_IOC_STREAM_WRITE
-NTFS_IOC_STREAM_REMOVE
Quoted 7 lines
Quoted 4 lines
Wouldn't it be better to just provide an API to get a fd, relative to a filename? Win32 named streams can be sparse, and it is common to punch holes to deallocate sections of streams too. Streams do have their own timestamps, too.
I believe the current ioctl interface is the best fit for this support. It operates on an already-open base-file fd, avoids pathname ambiguity with :,
Nope, NFS already fixed this by allowing "unnamed" fd for deleted files. Not having a name is OK. Win32 uses "filename:streamname:$DATA", if any resolving is needed. That works, because the Win32 API and NTFS forbit ':' within file and dir names.
Quoted 3 lines
supports UTF-16 stream names, and does not require a new system call or VFS changes. If WINE requires hole punching for sparse streams, I can extend the ioctl interface to support it.
Wearing my hat as the one who has to do the WINE implementation side:
I think a fd with all normal file operations available is the better option, because we need to do per stream: read(), write(), fsync(), fdatasync(), sync_file_range(), file locking for each stream separately, mmap(), getting/setting timestamps, futimes(), fstat(), ftruncate(), lseek() SEEK_HOLE,SEEK_DATA, fallocate() for space allocation/deallocation, unlink(), ... and likely more. Pretty much every syscall, except mkdir(),mknod(),rmdir().
Sebi
Signature · 1 line
Sebastian Feld - IT security consultant