Skip to contentSkip to conversation

stream_open bits for Linux 5.2

PATCH In review Kirill Smelkov 2 participants 7 messages May 6, 2019 – May 8, 2019

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

cover stream_open bits for Linux 5.2

Linus,

Please consider applying the following stream_open related patches:

- The first one removes unnecessary double nonseekable_open from drivers/char/dtlk.c as noticed by Pavel Machek while reviewing nonseekable_open -> stream_open mass conversion.

- The second one is the mass conversion patch promised in 10dce8af3422 ("fs:
stream_open - opener for stream-like files so that read and write can run simultaneously without deadlock") and is automatically generated by running

	$ make coccicheck MODE=patch COCCI=scripts/coccinelle/api/stream_open.cocci

I've verified each generated change manually - that it is correct to convert - and each other nonseekable_open instance left - that it is either not correct to convert there, or that it is not converted due to current stream_open.cocci limitations. More details on this in the patch.

- The third patch changes VFS to pass ppos=NULL into .read/.write for files that declare themselves streams. It was suggested by Rasmus Villemoes and makes sure that if ppos starts to be erroneously used in a stream file, such bug won't go unnoticed and will produce an oops instead of creating illusion of position change being taken into account.

Note: this patch does not conflict with "fuse: Add FOPEN_STREAM to use stream_open()" that will be hopefully coming via FUSE tree, because fs/fuse/ uses new-style .read_iter/.write_iter, and for these accessors position is still passed as non-pointer kiocb.ki_pos .

I was hoping for the patches to be picked up into VFS tree, but since that did not happenned for some reason I'm sending them to you directly.

Maybe it will help: the patches can be also pulled from here:

git pull https://lab.nexedi.com/kirr/linux.git y/stream_open-5.2

Thanks beforehand,
Kirill

Kirill Smelkov (3):
dtlk: remove double call to nonseekable_open
*: convert stream-like files from nonseekable_open -> stream_open
vfs: pass ppos=NULL to .read()/.write() of FMODE_STREAM files

 arch/powerpc/platforms/52xx/mpc52xx_gpt.c |   2 +-
 arch/powerpc/platforms/cell/spufs/file.c  |   2 +-
 arch/um/drivers/harddog_kern.c            |   2 +-
 arch/x86/kernel/cpu/microcode/core.c      |   2 +-
 drivers/char/ds1620.c                     |   2 +-
 drivers/char/dtlk.c                       |   3 +-
 drivers/char/ipmi/ipmi_watchdog.c         |   2 +-
 drivers/char/pcmcia/cm4000_cs.c           |   2 +-
 drivers/char/pcmcia/scr24x_cs.c           |   2 +-
 drivers/char/tb0219.c                     |   2 +-
 drivers/firewire/nosy.c                   |   2 +-
 drivers/gnss/core.c                       |   2 +-
 drivers/hid/uhid.c                        |   2 +-
 drivers/hwmon/fschmd.c                    |   2 +-
 drivers/hwmon/w83793.c                    |   2 +-
 drivers/infiniband/core/ucm.c             |   2 +-
 drivers/infiniband/core/ucma.c            |   2 +-
 drivers/infiniband/core/user_mad.c        |   2 +-
 drivers/infiniband/core/uverbs_main.c     |   2 +-
 drivers/input/evdev.c                     |   2 +-
 drivers/input/joydev.c                    |   2 +-
 drivers/input/misc/uinput.c               |   2 +-
 drivers/isdn/capi/capi.c                  |   2 +-
 drivers/leds/uleds.c                      |   2 +-
 drivers/media/rc/lirc_dev.c               |   2 +-
 drivers/pci/switch/switchtec.c            |   2 +-
 drivers/platform/chrome/cros_ec_debugfs.c |   2 +-
 drivers/rtc/rtc-ds1374.c                  |   2 +-
 drivers/rtc/rtc-m41t80.c                  |   2 +-
 drivers/s390/char/fs3270.c                |   2 +-
 drivers/s390/char/tape_char.c             |   2 +-
 drivers/s390/char/zcore.c                 |   2 +-
 drivers/s390/crypto/zcrypt_api.c          |   2 +-
 drivers/spi/spidev.c                      |   2 +-
 drivers/staging/pi433/pi433_if.c          |   2 +-
 drivers/usb/misc/ldusb.c                  |   2 +-
 drivers/watchdog/acquirewdt.c             |   2 +-
 drivers/watchdog/advantechwdt.c           |   2 +-
 drivers/watchdog/alim1535_wdt.c           |   2 +-
 drivers/watchdog/alim7101_wdt.c           |   2 +-
 drivers/watchdog/ar7_wdt.c                |   2 +-
 drivers/watchdog/at91rm9200_wdt.c         |   2 +-
 drivers/watchdog/ath79_wdt.c              |   2 +-
 drivers/watchdog/bcm63xx_wdt.c            |   2 +-
 drivers/watchdog/cpu5wdt.c                |   2 +-
 drivers/watchdog/cpwd.c                   |   2 +-
 drivers/watchdog/eurotechwdt.c            |   2 +-
 drivers/watchdog/f71808e_wdt.c            |   2 +-
 drivers/watchdog/gef_wdt.c                |   2 +-
 drivers/watchdog/geodewdt.c               |   2 +-
 drivers/watchdog/ib700wdt.c               |   2 +-
 drivers/watchdog/ibmasr.c                 |   2 +-
 drivers/watchdog/indydog.c                |   2 +-
 drivers/watchdog/intel_scu_watchdog.c     |   2 +-
 drivers/watchdog/iop_wdt.c                |   2 +-
 drivers/watchdog/it8712f_wdt.c            |   2 +-
 drivers/watchdog/ixp4xx_wdt.c             |   2 +-
 drivers/watchdog/ks8695_wdt.c             |   2 +-
 drivers/watchdog/m54xx_wdt.c              |   2 +-
 drivers/watchdog/machzwd.c                |   2 +-
 drivers/watchdog/mixcomwd.c               |   2 +-
 drivers/watchdog/mtx-1_wdt.c              |   2 +-
 drivers/watchdog/mv64x60_wdt.c            |   2 +-
 drivers/watchdog/nuc900_wdt.c             |   2 +-
 drivers/watchdog/nv_tco.c                 |   2 +-
 drivers/watchdog/pc87413_wdt.c            |   2 +-
 drivers/watchdog/pcwd.c                   |   4 +-
 drivers/watchdog/pcwd_pci.c               |   4 +-
 drivers/watchdog/pcwd_usb.c               |   4 +-
 drivers/watchdog/pika_wdt.c               |   2 +-
 drivers/watchdog/pnx833x_wdt.c            |   2 +-
 drivers/watchdog/rc32434_wdt.c            |   2 +-
 drivers/watchdog/rdc321x_wdt.c            |   2 +-
 drivers/watchdog/riowd.c                  |   2 +-
 drivers/watchdog/sa1100_wdt.c             |   2 +-
 drivers/watchdog/sb_wdog.c                |   2 +-
 drivers/watchdog/sbc60xxwdt.c             |   2 +-
 drivers/watchdog/sbc7240_wdt.c            |   2 +-
 drivers/watchdog/sbc8360.c                |   2 +-
 drivers/watchdog/sbc_epx_c3.c             |   2 +-
 drivers/watchdog/sbc_fitpc2_wdt.c         |   2 +-
 drivers/watchdog/sc1200wdt.c              |   2 +-
 drivers/watchdog/sc520_wdt.c              |   2 +-
 drivers/watchdog/sch311x_wdt.c            |   2 +-
 drivers/watchdog/scx200_wdt.c             |   2 +-
 drivers/watchdog/smsc37b787_wdt.c         |   2 +-
 drivers/watchdog/w83877f_wdt.c            |   2 +-
 drivers/watchdog/w83977f_wdt.c            |   2 +-
 drivers/watchdog/wafer5823wdt.c           |   2 +-
 drivers/watchdog/watchdog_dev.c           |   2 +-
 drivers/watchdog/wdrtas.c                 |   4 +-
 drivers/watchdog/wdt.c                    |   4 +-
 drivers/watchdog/wdt285.c                 |   2 +-
 drivers/watchdog/wdt977.c                 |   2 +-
 drivers/watchdog/wdt_pci.c                |   4 +-
 drivers/xen/evtchn.c                      |   2 +-
 fs/open.c                                 |   5 +-
 fs/read_write.c                           | 113 +++++++++++++---------
 net/batman-adv/icmp_socket.c              |   2 +-
 net/batman-adv/log.c                      |   2 +-
 net/rfkill/core.c                         |   2 +-
 sound/core/control.c                      |   2 +-
 sound/core/rawmidi.c                      |   2 +-
 sound/core/seq/seq_clientmgr.c            |   2 +-
 sound/core/timer.c                        |   2 +-
 105 files changed, 179 insertions(+), 158 deletions(-)
Signature · 1 line
2.20.1
#1 lore raw

PATCH 1/3 dtlk: remove double call to nonseekable_open

dtlk_open currently has 2 calls to nonseekable_open which are both executed on success path. It was not hurting to make the extra call as nonseekable_open is only changing file->f_flags in idempotent way. However the first nonseekable_open is indeed both unneeded and looks suspicious.

The first nonseekable_open was added in 6244f13c51 ("Fix up a couple of drivers - notable sg - for nonseekability."; 2004-Aug-7). The second nonseekable_open call was introduced in dc5c724584 ("Remove ESPIPE logic from drivers, letting the VFS layer handle it instead.; 2004-Aug-8). The latter patch being mass change probably missed to remove nonseekable_open that was introduced into dtlk_open the day before.

Fix it: remove the extra/unneeded nonseekable_open call and leave the call to nonseekable_open only on the path where we are actually opening the file.

---
 drivers/char/dtlk.c | 1 -
 1 file changed, 1 deletion(-)
drivers/char/dtlk.c +0 -1
index f882460b5a44..669c3311adc4 100644
@@ -298,7 +298,6 @@ static int dtlk_open(struct inode *inode, struct file *file)
 {
 	TRACE_TEXT("(dtlk_open");
 
-	nonseekable_open(inode, file);
 	switch (iminor(inode)) {
 	case DTLK_MINOR:
 		if (dtlk_busy)
Signature · 1 line
2.20.1
#2 lore raw

PATCH 3/3 vfs: pass ppos=NULL to .read()/.write() of FMODE_STREAM files

In reply to Kirill Smelkov

This amends commit 10dce8af3422 ("fs: stream_open - opener for stream-like files so that read and write can run simultaneously without deadlock") in how position is passed into .read()/.write() handler for stream-like files:

Rasmus noticed that we currently pass 0 as position and ignore any position change if that is done by a file implementation. This papers over bugs if ppos is used in files that declare themselves as being stream-like as such bugs will go unnoticed. Even if a file implementation is correctly converted into using stream_open, its read/write later could be changed to use ppos and even though that won't be working correctly, that bug might go unnoticed without someone doing wrong behaviour analysis. It is thus better to pass ppos=NULL into read/write for stream-like files as that don't give any chance for ppos usage bugs because it will oops if ppos is ever used inside .read() or .write().

Note 1: rw_verify_area, new_sync_{read,write} needs to be updated because they are called by vfs_read/vfs_write & friends before file_operations .read/.write .

Note 2: if file backend uses new-style .read_iter/.write_iter, position is still passed into there as non-pointer kiocb.ki_pos . Currently stream_open.cocci (semantic patch added by 10dce8af3422) ignores files whose file_operations has *_iter methods.

---
 fs/open.c       |   5 ++-
 fs/read_write.c | 113 ++++++++++++++++++++++++++++--------------------
 2 files changed, 70 insertions(+), 48 deletions(-)
fs/open.c +3 -2
index a00350018a47..9c7d724a6f67 100644
@@ -1219,8 +1219,9 @@ EXPORT_SYMBOL(nonseekable_open);
 /*
  * stream_open is used by subsystems that want stream-like file descriptors.
  * Such file descriptors are not seekable and don't have notion of position
- * (file.f_pos is always 0). Contrary to file descriptors of other regular
- * files, .read() and .write() can run simultaneously.
+ * (file.f_pos is always 0 and ppos passed to .read()/.write() is always NULL).
+ * Contrary to file descriptors of other regular files, .read() and .write()
+ * can run simultaneously.
  *
  * stream_open never fails and is marked to return int so that it could be
  * directly used as file_operations.open .
fs/read_write.c +67 -46

189 lines not on this page · raw message

Signature · 1 line
2.20.1
#3 lore raw
In reply to Kirill Smelkov
Quoted 4 lines · Kirill Smelkov

Maybe it will help: the patches can be also pulled from here:

git pull https://lab.nexedi.com/kirr/linux.git y/stream_open-5.2

I'll take this, but I generally *really* want a signed tag for non-kernel.org git tree sources. The gpg key used for signing doesn't necessarily even have to be signed by others yet, but just the fact that there's a pgp key means that then future pulls at least verify that it's the sam,e controlling entity, and we can get the signatures later.

For something one-time where I will then look through the details of each commit it's not like I absolutely require it, which is why I'm pulling it, but just in general I wanted to point this out.

Linus

#5 lore raw
In reply to Kirill Smelkov
Quoted 4 lines · Kirill Smelkov

I've pushed corresponding gpg-signed tag (stream_open-5.2) to my tree. I did not go the gpg way initially because we do not have a gpg-trust relation established and so I thought that signing was useless.

Ok, since I hadn't pushed out my pull yet, I just re-did it with your signature, so that the key is visible in the git tree.

Linus

#4 lore raw
In reply to Linus Torvalds
Quoted 13 lines · Linus Torvalds
Quoted 4 lines · Kirill Smelkov

Maybe it will help: the patches can be also pulled from here:

git pull https://lab.nexedi.com/kirr/linux.git y/stream_open-5.2

I'll take this, but I generally *really* want a signed tag for non-kernel.org git tree sources. The gpg key used for signing doesn't necessarily even have to be signed by others yet, but just the fact that there's a pgp key means that then future pulls at least verify that it's the sam,e controlling entity, and we can get the signatures later.

Quoted 5 lines

For something one-time where I will then look through the details of each commit it's not like I absolutely require it, which is why I'm pulling it, but just in general I wanted to point this out.

Linus

Thanks a lot.

I've pushed corresponding gpg-signed tag (stream_open-5.2) to my tree. I did not go the gpg way initially because we do not have a gpg-trust relation established and so I thought that signing was useless.

Just for the record, here is the key that was used to make the
signature: https://pgp.key-server.io/search/[email protected]
(fingerprint: 0955B024250EEFFCFE42365B66CA788413F67549)

Kirill

#6 lore raw
In reply to Linus Torvalds
Quoted 8 lines · Linus Torvalds
Quoted 4 lines · Kirill Smelkov

I've pushed corresponding gpg-signed tag (stream_open-5.2) to my tree. I did not go the gpg way initially because we do not have a gpg-trust relation established and so I thought that signing was useless.

Ok, since I hadn't pushed out my pull yet, I just re-did it with your signature, so that the key is visible in the git tree.

Thanks.