Skip to contentSkip to conversation

VFS locking: f_pos thread-safe ?

DISCUSSION Werner Almesberger 6 participants 10 messages Feb 6, 2004 – Feb 7, 2004

lore
Share & tools
lore
#0 lore raw

I'm trying to figure out how all the locking in VFS and friends works, and I can't quite explain to myself how f_pos is kept consistent with concurrent readers.

In fact, there might be a violation of atomicity requirements:
e.g. if we take the route sys_read -> vfs_read -> generic_file_read -> __generic_file_aio_read -> do_generic_file_read -> do_generic_mapping_read, we don't seem to be holding any locks. So if I have two threads that start reading the same fd at the same time, they could retrieve the same data.

Section 2.9.7 of the "Austin" draft of IEEE Std. 1003.1-200x, 28-JUL-2000, says:

"[...] read( ) [...] shall be atomic with respect to each other
in the effects specified in IEEE Std. 1003.1-200x when they operate on regular files. If two threads each call one of these functions, each call shall either see all of the specified effects of the other call, or none of them."

I've written a little test program with concurrent readers that seems to support this observation, i.e. given the following pseudo-code:

static void *reader(...)

{
    while (read(0,buffer,PAGE_SIZE));
    ...
}

...

    for (...)
	pthread_create(...,reader...);

...

More than one reader may obtain a given page.
The full test program is at
http://www.almesberger.net/misc/tt.tar.gz

Is this a real bug or am I just confused ?

- Werner

Signature · 3 lines
  _________________________________________________________________________
 / Werner Almesberger, Buenos Aires, Argentina         [email protected] /
/_http://www.almesberger.net/____________________________________________/
#1 lore raw
Quoted 6 lines · Werner Almesberger

"[...] read( ) [...] shall be atomic with respect to each other
in the effects specified in IEEE Std. 1003.1-200x when they operate on regular files. If two threads each call one of these functions, each call shall either see all of the specified effects of the other call, or none of them."

Whichever thread finishes its read last gets to update f_pos.

I'm struggling a bit to understand what they're calling for there. If thread A enters a read and then shortly afterwards thread B enters the read, does thread B see an f_pos which starts out at the beginning of A's read, or the end of it?

Similar questions apply as the threads exit their read()s.

Either way, there's no way in which we should serialise concurrent readers. That would really suck for sensible apps which are using pread64().

#2 lore raw
In reply to Werner Almesberger
Quoted 8 lines · Werner Almesberger

Section 2.9.7 of the "Austin" draft of IEEE Std. 1003.1-200x, 28-JUL-2000, says:

"[...] read( ) [...] shall be atomic with respect to each other
in the effects specified in IEEE Std. 1003.1-200x when they operate on regular files. If two threads each call one of these functions, each call shall either see all of the specified effects of the other call, or none of them."

This reads: "all of the specified effects of the other call,
or none of them." If I read that correctly, if f_pos is at N, and threads A and B concurrently read M bytes, then each thread's read() must either start at f_pos = N or f_pos = N+M, but never at N < f_pos < N+M. So as long as our code doesn't partially update f_pos, it is valid.
Of course, that doesn't change the possible race updating
f_pos at the end of each thread's call.

Joel

Signature · 9 lines
"In a crisis, don't hide behind anything or anybody. They're going
 to find you anyway."
	- Paul "Bear" Bryant

Joel Becker
Senior Member of Technical Staff
Oracle Corporation
E-mail: [email protected]
Phone: (650) 506-8127
#3 lore raw
Quoted 6 lines · Joel Becker

This reads: "all of the specified effects of the other call, or none of them." If I read that correctly, if f_pos is at N, and threads A and B concurrently read M bytes, then each thread's read() must either start at f_pos = N or f_pos = N+M, but never at N < f_pos < N+M. So as long as our code doesn't partially update f_pos, it is valid.

Umm, strictly speaking there are three possible valid cases:

thread A reads M @N   thread B reads M @N     file pointer ends up as N+M
thread A reads M @N   thread B reads M @M+N   file pointer ends up as N+2M
thread A reads M @M+N thread B reads M @N     file pointer ends up as N+2M

With your description,
thread A reads M @M+N thread B reads M @M+N file pointer ends up as N+2M

would be equally valid, which I'd declare buggy ^W non-conforming.

Signature · 1 line
Matthias Urlichs
#4 lore raw
Quoted 6 lines · Matthias Urlichs
thread A reads M @N   thread B reads M @N     file pointer ends up as N+M
thread A reads M @N   thread B reads M @M+N   file pointer ends up as N+2M
thread A reads M @M+N thread B reads M @N     file pointer ends up as N+2M

With your description,
thread A reads M @M+N thread B reads M @M+N file pointer ends up as N+2M

I didn't mean the last. I was just writing English, and figured
everyone would know what I meant. Obviously, the first read() in starts @M.

Joel

Signature · 9 lines
Life's Little Instruction Book #198

	"Feed a stranger's expired parking meter."

Joel Becker
Senior Member of Technical Staff
Oracle Corporation
E-mail: [email protected]
Phone: (650) 506-8127
#5 lore raw
In reply to Joel Becker
Quoted 2 lines · Joel Becker

This reads: "all of the specified effects of the other call,
or none of them."

[...]

Quoted 2 lines

Of course, that doesn't change the possible race updating
f_pos at the end of each thread's call.

Hmm, "all, but the f_pos read-modify-write" sounds more like how an insurance company would define "all" :-)

What's puzzling here is that the standard would introduce such an important concept in the discussion of threads.

- Werner

Signature · 3 lines
  _________________________________________________________________________
 / Werner Almesberger, Buenos Aires, Argentina         [email protected] /
/_http://www.almesberger.net/____________________________________________/
#6 lore raw
Quoted 5 lines · Werner Almesberger

Hmm, "all, but the f_pos read-modify-write" sounds more like how an insurance company would define "all" :-)

What's puzzling here is that the standard would introduce such an important concept in the discussion of threads.

Well... it's the sort of problem that's very Schrodenger unless you have another thread/process observing.

#7 lore raw
Quoted 2 lines · [email protected] wrote:

Well... it's the sort of problem that's very Schrodenger unless you have another thread/process observing.

In a way :-) What I meant was that the section on read() is worded carefully to avoid making such assurances, but then we find such a broad statement at the end of a section discussing specifically threads (and not even concurrency in general, and after pointing out that threads are optional).

- Werner

Signature · 3 lines
  _________________________________________________________________________
 / Werner Almesberger, Buenos Aires, Argentina         [email protected] /
/_http://www.almesberger.net/____________________________________________/
#8 lore raw
In reply to Werner Almesberger
Quoted 4 lines · Werner Almesberger

I'm trying to figure out how all the locking in VFS and friends works, and I can't quite explain to myself how f_pos is kept consistent with concurrent readers.

Quoted 8 lines

Section 2.9.7 of the "Austin" draft of IEEE Std. 1003.1-200x, 28-JUL-2000, says:

"[...] read( ) [...] shall be atomic with respect to each other
in the effects specified in IEEE Std. 1003.1-200x when they operate on regular files. If two threads each call one of these functions, each call shall either see all of the specified effects of the other call, or none of them."

The 2003 version can be found at

http://www.opengroup.org/onlinepubs/007904975/toc.htm
http://www.opengroup.org/onlinepubs/007904975/functions/xsh_chap02_09.html

Andries

#9 lore raw
Quoted 4 lines · Andries Brouwer

The 2003 version can be found at

http://www.opengroup.org/onlinepubs/007904975/toc.htm
http://www.opengroup.org/onlinepubs/007904975/functions/xsh_chap02_09.html

Thanks a lot !

The wording is the same, so it seems that we don't comply with that part, at least not in the letter :-(

- Werner

Signature · 3 lines
  _________________________________________________________________________
 / Werner Almesberger, Buenos Aires, Argentina         [email protected] /
/_http://www.almesberger.net/____________________________________________/