Skip to contentSkip to conversation

ntfs: add ioctl support for named data streams

PATCH In review Namjae Jeon 4 participants 5 messages Jul 29 – Aug 4

Subsystems (3)
lore
Share & tools
lore

Namjae Jeon

author maintainer
#0 lore raw

PATCH 1/1 ntfs: add ioctl support for named data streams

Sebastian and Cedric requested NTFS driver support for an interface that WINE can use to access named data streams.

NTFS supports multiple named $DATA attributes, commonly known as alternate data streams, but they are not directly accessible through the existing file operations. Add NTFS-specific ioctl commands to let userspace list, read, write, and remove named data streams on an opened file.

The new UAPI provides separate commands for each operation:

- NTFS_IOC_STREAM_READ
- NTFS_IOC_STREAM_WRITE
- NTFS_IOC_STREAM_REMOVE
- NTFS_IOC_LIST_STREAMS

The read and write commands use an offset and length based request format. Writes create the named stream when it does not already exist. The list command returns variable-length stream entries containing stream size, allocated size, and stream name.

Stream names use the mounted filesystem NLS by default, or raw UTF-16LE when the UTF-16 flag is requested. Read and write transfers are limited to 16 MiB per request. If the list buffer is too small, the list command returns -ENOSPC while reporting the required size and stream count.

---
 .../userspace-api/ioctl/ioctl-number.rst      |   1 +
 fs/ntfs/file.c                                | 509 ++++++++++++++++++
 include/uapi/linux/ntfs.h                     | 115 ++++
 3 files changed, 625 insertions(+)
 create mode 100644 include/uapi/linux/ntfs.h
Documentation/userspace-api/ioctl/ioctl-number.rst +1 -0
index 3f0ef1e27eb0..2b9869aeb811 100644
@@ -399,6 +399,7 @@ Code  Seq#    Include File                                             Comments
 0xE5  00-3F  linux/fuse.h
 0xEC  00-01  drivers/platform/chrome/cros_ec_dev.h                     ChromeOS EC driver
 0xEE  00-09  uapi/linux/pfrut.h                                        Platform Firmware Runtime Update and Telemetry
+0xEF  00-0F  uapi/linux/ntfs.h                                         NTFS
 0xF3  00-3F  drivers/usb/misc/sisusbvga/sisusb.h                       sisfb (in development)
                                                                        <mailto:[email protected]>
 0xF6  all                                                              LTTng Linux Trace Toolkit Next Generation
fs/ntfs/file.c +509 -0

533 lines not on this page · raw message

include/uapi/linux/ntfs.h new file +115 -0

115 lines not on this page · raw message

Signature · 1 line
2.25.1
maintainer
#1 lore raw

fs/ntfs/file.c static int ntfs_ioctl_fitrim(struct ntfs_volume *vol, unsigned long arg) · line 997

Quoted 5 lines
fs/ntfs/file.c +5 -0
+
+       bytes = ntfs_inode_attr_pwrite(attr_vi, offset, len, data_stream,
+                                      false);
+       if (bytes < 0)
+               err = bytes;

offset is u64, but ntfs_inode_attr_pwrite takes s64. If offset is U64_MAX and len = 1, pos becomes -1. A resident stream can the reach memcpy(addr + pos, ...), causing an out-of-bounds write.

fs/ntfs/file.c static int ntfs_ioctl_fitrim(struct ntfs_volume *vol, unsigned long arg) · line 1028

Quoted 7 lines
fs/ntfs/file.c +7 -0
+static int ntfs_remove_named_stream(struct ntfs_inode *ni, __le16 *uname,
+               u32 uname_len)
+{
+       if (!ni || !uname || uname_len == 0)
+               return -EINVAL;
+
+       return ntfs_attr_remove(ni, AT_DATA, uname, uname_len);

Dirty folios are not guaranteed to be evicted before ntfs_attr_rm() frees clusters. So invalidating page caches seem to be needed.

And we have to serialize ntfs_attr_remove() against other mft record modifications.

Quoted 7 lines
fs/ntfs/file.c +7 -0
+       case NTFS_STREAM_OP_WRITE:
+               err = mnt_want_write_file(filp);
+               if (err)
+                       break;
+               err = ntfs_write_named_stream(ni, sname, sname_len,
+                                             req->stream_offset, req->io_len,
+                                             kdata,

fs/ntfs/file.c static int ntfs_ioctl_fitrim(struct ntfs_volume *vol, unsigned long arg) · line 1169

Quoted 10 lines
fs/ntfs/file.c +10 -0
+                                             &req->bytes_returned);
+               mnt_drop_write_file(filp);
+               if (!err) {
+                       if (copy_to_user(&ureq->bytes_returned,
+                                        &req->bytes_returned,
+                                        sizeof(req->bytes_returned)))
+                               err = -EFAULT;
+               }
+               break;
+

We have to check NVolShutdown().

#2 lore raw
In reply to Namjae Jeon
Quoted 9 lines · Namjae Jeon

Sebastian and Cedric requested NTFS driver support for an interface that WINE can use to access named data streams.

NTFS supports multiple named $DATA attributes, commonly known as alternate data streams, but they are not directly accessible through the existing file operations. Add NTFS-specific ioctl commands to let userspace list, read, write, and remove named data streams on an opened file.

Quoted 5 lines

The new UAPI provides separate commands for each operation:

- NTFS_IOC_STREAM_READ
- NTFS_IOC_STREAM_WRITE
- NTFS_IOC_STREAM_REMOVE

Wouldn't it be better to just provide an API to get a fd, relative to a filename? Win32 named streams can be sparse, and it is common to punch holes to deallocate sections of streams too. Streams do have their own timestamps, too.

Quoted 1 line

- NTFS_IOC_LIST_STREAMS

NTFS_IOC_LIST_STREAMS looks OK.

Lionel

author maintainer
#3 lore raw
Quoted 13 lines · Lionel Cons
Quoted 11 lines · Namjae Jeon

Sebastian and Cedric requested NTFS driver support for an interface that WINE can use to access named data streams.

NTFS supports multiple named $DATA attributes, commonly known as alternate data streams, but they are not directly accessible through the existing file operations. Add NTFS-specific ioctl commands to let userspace list, read, write, and remove named data streams on an opened file.

The new UAPI provides separate commands for each operation:

Quoted 8 lines
Quoted 3 lines

- NTFS_IOC_STREAM_READ
- NTFS_IOC_STREAM_WRITE
- NTFS_IOC_STREAM_REMOVE

Wouldn't it be better to just provide an API to get a fd, relative to a filename? Win32 named streams can be sparse, and it is common to punch holes to deallocate sections of streams too. Streams do have their own timestamps, too.

I believe the current ioctl interface is the best fit for this support. It operates on an already-open base-file fd, avoids pathname ambiguity with :, supports UTF-16 stream names, and does not require a new system call or VFS changes. If WINE requires hole punching for sparse streams, I can extend the ioctl interface to support it.

Quoted 4 lines
Quoted 2 lines

- NTFS_IOC_LIST_STREAMS

NTFS_IOC_LIST_STREAMS looks OK.

Thanks!

#4 lore raw
Quoted 19 lines · Namjae Jeon
Quoted 17 lines · Lionel Cons
Quoted 14 lines · Namjae Jeon

Sebastian and Cedric requested NTFS driver support for an interface that WINE can use to access named data streams.

NTFS supports multiple named $DATA attributes, commonly known as alternate data streams, but they are not directly accessible through the existing file operations. Add NTFS-specific ioctl commands to let userspace list, read, write, and remove named data streams on an opened file.

The new UAPI provides separate commands for each operation:

- NTFS_IOC_STREAM_READ
- NTFS_IOC_STREAM_WRITE
- NTFS_IOC_STREAM_REMOVE

Quoted 7 lines
Quoted 4 lines

Wouldn't it be better to just provide an API to get a fd, relative to a filename? Win32 named streams can be sparse, and it is common to punch holes to deallocate sections of streams too. Streams do have their own timestamps, too.

I believe the current ioctl interface is the best fit for this support. It operates on an already-open base-file fd, avoids pathname ambiguity with :,

Nope, NFS already fixed this by allowing "unnamed" fd for deleted files. Not having a name is OK. Win32 uses "filename:streamname:$DATA", if any resolving is needed. That works, because the Win32 API and NTFS forbit ':' within file and dir names.

Quoted 3 lines

supports UTF-16 stream names, and does not require a new system call or VFS changes. If WINE requires hole punching for sparse streams, I can extend the ioctl interface to support it.

Wearing my hat as the one who has to do the WINE implementation side:
I think a fd with all normal file operations available is the better option, because we need to do per stream: read(), write(), fsync(), fdatasync(), sync_file_range(), file locking for each stream separately, mmap(), getting/setting timestamps, futimes(), fstat(), ftruncate(), lseek() SEEK_HOLE,SEEK_DATA, fallocate() for space allocation/deallocation, unlink(), ... and likely more. Pretty much every syscall, except mkdir(),mknod(),rmdir().

Sebi

Signature · 1 line
Sebastian Feld - IT security consultant