Skip to contentSkip to conversation

sysmips call and glibc atomic set

DISCUSSION Joe deBlaquiere 4 participants 6 messages Dec 25, 2000 – Dec 28, 2000 MIPS

lore
Share & tools
lore
#0 lore raw

I'm working with a vr4181 target and started digging into the atomic test and set stuff in the kernel and glibc. The first problem I had was that the glibc code assumes that all mips III targets implement the mips III ISA (funny assumption, no?) but the vr4181 doesn't include the miltiprocessor oriented LL/SC operations for atomic test and set.

So I started looking at the glibc code (yes, I know this is the kernel list... I'm getting there I promise) and notice the following operations:

__asm__ __volatile__

    (".set	mips2\n\t"
     "/* Inline spinlock test & set */\n\t"
     "1:\n\t"
     "ll	%0,%3\n\t"
     ".set	push\n\t"
     ".set	noreorder\n\t"
     "bnez	%0,2f\n\t"
     " li	%1,1\n\t"
     ".set	pop\n\t"
     "sc	%1,%2\n\t"
     "beqz	%1,1b\n"
     "2:\n\t"
     "/* End spinlock test & set */"
     : "=&r" (ret), "=&r" (temp), "=m" (*spinlock)
     : "m" (*spinlock)
     : "memory");

The significant code here being the 'll' and 'sc' operations which are supposed to ensure that the operation is atomic.

QUESTION 1) Will this _ALWAYS_ work from user land? I realize the operations are temporally close, but isn't there the possibility that an interrupt occurs in the meantime?

Of course none of this code applies to my case anyway, since the vr4181 doesn't implement these ops. So once I hack^H^H^H^Hadjust glibc to use the 'mips1' implementation, it uses the sysmips system call. regard :

_test_and_set (int *p, int v) __THROW
{
return sysmips (MIPS_ATOMIC_SET, (int) p, v, 0);
}

So then I looked at the kernel and find the code below. The system I'm working with is expressedly uniprocessor and doesn't have any swap, so it looks like the initial caveats are met, but it looks to me like there could be some confusion if the value of *arg1 at entry looks like -ENOSYS or something like that.

QUESTION 2) Wouldn't it be better to pass back the initial value of *arg1 in *arg3 and return zero or negative error code?

	case MIPS_ATOMIC_SET: {
		/* This is broken in case of page faults and SMP ...
		    Risc/OS faults after maximum 20 tries with EAGAIN.  */
		unsigned int tmp;

		p = (int *) arg1;
		errno = verify_area(VERIFY_WRITE, p, sizeof(*p));
		if (errno)
			return errno;
		errno = 0;
		save_and_cli(flags);
		errno |= __get_user(tmp, p);
		errno |= __put_user(arg2, p);
		restore_flags(flags);

		if (errno)
			return tmp;

		return tmp;             /* This is broken ...  */
        }

QUESTION 3) I notice that the code for this particular case of sysmips has changed recently. The old code looked more like the 'll/sc' version of glibc above. I would think that the 'll/sc' code would be better on SMP systems. Is there a good reason why this reverted?

Sorry For the Long Post (tm)! Thanks In Advance! Merry Xmas!

Signature · 10 lines
Joe deBlaquiere
Red Hat, Inc.
307 Wynn Drive
Huntsville AL, 35805


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [email protected]
Please read the FAQ at http://www.tux.org/lkml/
#1 lore raw
Quoted 5 lines · Joe deBlaquiere

I'm working with a vr4181 target and started digging into the atomic test and set stuff in the kernel and glibc. The first problem I had was that the glibc code assumes that all mips III targets implement the mips III ISA (funny assumption, no?) but the vr4181 doesn't include the miltiprocessor oriented LL/SC operations for atomic test and set.

Ok, but since the kernel disables MIPS III you're limited to MIPS II anyway ...

Quoted 21 lines

So I started looking at the glibc code (yes, I know this is the kernel list... I'm getting there I promise) and notice the following operations:

   __asm__ __volatile__
     (".set	mips2\n\t"
      "/* Inline spinlock test & set */\n\t"
      "1:\n\t"
      "ll	%0,%3\n\t"
      ".set	push\n\t"
      ".set	noreorder\n\t"
      "bnez	%0,2f\n\t"
      " li	%1,1\n\t"
      ".set	pop\n\t"
      "sc	%1,%2\n\t"
      "beqz	%1,1b\n"
      "2:\n\t"
      "/* End spinlock test & set */"
      : "=&r" (ret), "=&r" (temp), "=m" (*spinlock)
      : "m" (*spinlock)
      : "memory");
Quoted 6 lines

The significant code here being the 'll' and 'sc' operations which are supposed to ensure that the operation is atomic.

QUESTION 1) Will this _ALWAYS_ work from user land? I realize the operations are temporally close, but isn't there the possibility that an interrupt occurs in the meantime?

Read the ISA manual; sc will fail if the LL-bit in c0_status is cleared which will be cleared when the interrupt returns using the eret instruction.

Quoted 9 lines

Of course none of this code applies to my case anyway, since the vr4181 doesn't implement these ops. So once I hack^H^H^H^Hadjust glibc to use the 'mips1' implementation, it uses the sysmips system call. regard :

_test_and_set (int *p, int v) __THROW

{
   return sysmips (MIPS_ATOMIC_SET, (int) p, v, 0);
}
Quoted 5 lines

So then I looked at the kernel and find the code below. The system I'm working with is expressedly uniprocessor and doesn't have any swap, so it looks like the initial caveats are met, but it looks to me like there could be some confusion if the value of *arg1 at entry looks like -ENOSYS or something like that.

Not having swap doesn't mean you're safe. Think of any kind of previously unmapped page.

Quoted 2 lines

QUESTION 2) Wouldn't it be better to pass back the initial value of *arg1 in *arg3 and return zero or negative error code?

The semantics of this syscall were previously defined by Risc/OS and later on continued to be used by IRIX.

Quoted 21 lines
	case MIPS_ATOMIC_SET: {
		/* This is broken in case of page faults and SMP ...
		    Risc/OS faults after maximum 20 tries with EAGAIN.  */
		unsigned int tmp;

		p = (int *) arg1;
		errno = verify_area(VERIFY_WRITE, p, sizeof(*p));
		if (errno)
			return errno;
		errno = 0;
		save_and_cli(flags);
		errno |= __get_user(tmp, p);
		errno |= __put_user(arg2, p);
		restore_flags(flags);

		if (errno)
			return tmp;

		return tmp;             /* This is broken ...  */
         }
Quoted 4 lines

QUESTION 3) I notice that the code for this particular case of sysmips has changed recently. The old code looked more like the 'll/sc' version of glibc above. I would think that the 'll/sc' code would be better on SMP systems.

Don't think about SMP without ll/sc. There's algorithems available for that but their complexity leaves them a unpractical, theoretical construct.

Quoted 1 line

Is there a good reason why this reverted?

Above code will break if the old content of memory has bit 31 set or you take pagefaults. The latter problem is a problem even on UP - think multi- threading.

Finally, post such things to one of the MIPS-related mailing lists. If you're unlucky nobody of the MIPS'ers might see your posting on l-k.

Ralf

#2 lore raw
Quoted 2 lines · Ralf Baechle

The semantics of this syscall were previously defined by Risc/OS and later on continued to be used by IRIX.

Ralf, could you please provide me a copy of a man page for the call? I
don't have access to either of the systems and a search of the Net returned nothing.

Quoted 2 lines

Don't think about SMP without ll/sc. There's algorithems available for that but their complexity leaves them a unpractical, theoretical construct.

For SMP there is a simple kernel solution available. It suitable for a
syscall or a ll/sc emulation. There is no easy userland-only solution AFAIK.

Quoted 3 lines

Above code will break if the old content of memory has bit 31 set or you take pagefaults. The latter problem is a problem even on UP - think multi- threading.

If the code is written carefully you don't ever get a pagefault that
would break consistency.

Signature · 3 lines
+  Maciej W. Rozycki, Technical University of Gdansk, Poland   +
+--------------------------------------------------------------+
+        e-mail: [email protected], PGP key available        +
#3 lore raw
In reply to Joe deBlaquiere

Ralf, firstly, thank you for the answers :)

Quoted 3 lines · Ralf Baechle

Ok, but since the kernel disables MIPS III you're limited to MIPS II anyway ...

This makes sense...

Quoted 5 lines

Read the ISA manual; sc will fail if the LL-bit in c0_status is cleared which will be cleared when the interrupt returns using the eret instruction.

I tried to find a MIPSIII manual from mips.com but all I could find was mips32 and mips64 (which are not the same as MIPSII/MIPSIII/MIPSIV).

Quoted 5 lines

Not having swap doesn't mean you're safe. Think of any kind of previously unmapped page.

Is there a reason why it doesn't just force that page to be mapped first?

Quoted 26 lines
Quoted 2 lines

QUESTION 2) Wouldn't it be better to pass back the initial value of *arg1 in *arg3 and return zero or negative error code?

The semantics of this syscall were previously defined by Risc/OS and later on continued to be used by IRIX.

Quoted 18 lines
	case MIPS_ATOMIC_SET: {
		/* This is broken in case of page faults and SMP ...
		    Risc/OS faults after maximum 20 tries with EAGAIN.  */
		unsigned int tmp;

		p = (int *) arg1;
		errno = verify_area(VERIFY_WRITE, p, sizeof(*p));
		if (errno)
			return errno;
		errno = 0;
		save_and_cli(flags);
		errno |= __get_user(tmp, p);
		errno |= __put_user(arg2, p);
		restore_flags(flags);

		if (errno)
			return tmp;
Quoted 12 lines
Quoted 4 lines
		return tmp;             /* This is broken ...  */
         }

QUESTION 3) I notice that the code for this particular case of sysmips has changed recently. The old code looked more like the 'll/sc' version of glibc above. I would think that the 'll/sc' code would be better on SMP systems.

Don't think about SMP without ll/sc. There's algorithems available for that but their complexity leaves them a unpractical, theoretical construct.

Quoted 1 line

Is there a good reason why this reverted?

Looking at 2.4.0-test5 I see the ll/sc code, but -test12 doesn't use it. I was just curious at why it was taken out.

Quoted 9 lines

Above code will break if the old content of memory has bit 31 set or you take pagefaults. The latter problem is a problem even on UP - think multi- threading.

Finally, post such things to one of the MIPS-related mailing lists. If you're unlucky nobody of the MIPS'ers might see your posting on l-k.

Ralf

Signature · 6 lines
Joe deBlaquiere
Red Hat, Inc.
307 Wynn Drive
Huntsville AL, 35805
voice : (256)-704-9200
fax   : (256)-837-3839
#4 lore raw
Quoted 5 lines · Joe deBlaquiere
Quoted 2 lines

Read the ISA manual; sc will fail if the LL-bit in c0_status is cleared which will be cleared when the interrupt returns using the eret instruction.

I tried to find a MIPSIII manual from mips.com but all I could find was mips32 and mips64 (which are not the same as MIPSII/MIPSIII/MIPSIV).

Get "IDT MIPS Microprocessor Software Reference Manual" from
http://decstation.unix-ag.org/docs/ic_docs/3715.pdf (the original is no longer available from IDT servers).

Signature · 3 lines
+  Maciej W. Rozycki, Technical University of Gdansk, Poland   +
+--------------------------------------------------------------+
+        e-mail: [email protected], PGP key available        +
#5 lore raw
In reply to Joe deBlaquiere

Hi!

Quoted 6 lines
Quoted 3 lines

Not having swap doesn't mean you're safe. Think of any kind of previously unmapped page.

Is there a reason why it doesn't just force that page to be mapped first?

You can map it in... But background daemon can map it out in the meantime :-). You'd have to map in and pagelock.
Pavel

Signature · 2 lines
I'm [email protected]. "In my country we have almost anarchy and I don't care."
Panos Katsaloulis describing me w.r.t. patents at [email protected]