Pagesize is different between IA32 and IA64
Hello all.
If this problem has already known, I am sorry.
I am investigating the problem that calloc returns no cleared area.
It was made on IA32 system and executes on IA64. I think calloc expects to clear the memories sbrk returned. But When the program made on IA32 is executing on IA64 system, sbrk sometimes returns no cleared area.
Because the Kernel on IA64 alignes the memories 16K bytes. However, when the progmam made on IA32 is executing on IA64, the function __getpagesize returns 4Kbytes. At this case, if the memories that calloc requests to return sbrk(-xxx) is smaller than 16Kbytes, the kernel do not unmap this memory.
(I think that the time that the memories clears 0 by the kernel is when the kernel maps memories by the signal)
How do you think?
--
Regards,
Hideki Yamamoto (V).v.(V) # Empowered by Innovation
Hideki-
A couple of points:
1) It turns out that IA32 programs have all sorts of hidden assumptions that a page size is 4K. Changing `__getpagsize' to return the true page size might fix your problem but I predict that such a change would break many other programs. I'll look at your test case and see if there is anything we can do.
2) If all else fails you can run your IA64 machine with a 4K page size. We've always said that, for true complete IA32 support, you may have to run with 4K pages. I've put a lot of effort into allowing larger page sizes for `normal' prgrams, yours might be one that requires a 4K kernel.
3) I'm just curious, is this a real application or is this just a test case?
Quoted 32 lines · Hideki Yamamoto
Hello all.
If this problem has already known, I am sorry.
I am investigating the problem that calloc returns no cleared area.
It was made on IA32 system and executes on IA64. I think calloc expects to clear the memories sbrk returned. But When the program made on IA32 is executing on IA64 system, sbrk sometimes returns no cleared area.
Because the Kernel on IA64 alignes the memories 16K bytes. However, when the progmam made on IA32 is executing on IA64, the function__getpagesizereturns 4Kbytes. At this case, if the memories that calloc requests to return sbrk(-xxx) is smaller than 16Kbytes, the kernel do not unmap this memory.
(I think that the time that the memories clears 0 by the kernel is when the kernel maps memories by the signal)How do you think?
--
Regards,
Hideki Yamamoto (V).v.(V) # Empowered by Innovation_______________________________________________
Linux-IA64 mailing list
[email protected]
http://lists.linuxia64.org/lists/listinfo/linux-ia64
Signature · 4 lines
Don Dugger "Censeo Toto nos in Kansa esse decisse." - D. Gale [email protected] Ph: 303/652-0870x117
But I think the page size problem with brk() is real and can be fixed quite easily. If I understood correctly, the problem is that doing something along the lines of:
x = ALIGN_TO_4K(sbrk(8192)); memset(x, 0xff, 4096); brk(x); brk(x+4096);
might preserve the contents of the page at X on under the ia32 subsystem of ia64 when in fact it should be cleared to zero.
Would you be able/interested into looking into this?
--david
ポセイドンの...
Hi Don.
Thank you for responding sooner.
It is a real application. And I think that it may occur all IA32 executable applications on IA64 because the problem is that calloc always expected clean memories sbrk returned. and calloc believes that the returned memories(4K) to sbrk is unmaped by sbrk.
Do not you think so?
The following code is a part of calloc functions in glibc 2.2.4.
/* Check if expand_top called, in which case there may be
no need to clear. */
#if MORECORE_CLEARS
oldtop = top(ar_ptr);
oldtopsize = chunksize(top(ar_ptr));
#if MORECORE_CLEARS < 2
/* Only newly allocated memory is guaranteed to be cleared. */ if (ar_ptr = &main_arena &&
oldtopsize < sbrk_base + max_sbrked_mem - (char *)oldtop)
oldtopsize = (sbrk_base + max_sbrked_mem - (char *)oldtop);
#endif
#endif
p = chunk_alloc (ar_ptr, sz);
At Mon, 12 Nov 2001 08:28:52 -0700,
Quoted 61 lines · [email protected] wrote:
Hideki-
A couple of points:
1) It turns out that IA32 programs have all sorts of hidden assumptions that a page size is 4K. Changing `__getpagsize' to return the true page size might fix your problem but I predict that such a change would break many other programs. I'll look at your test case and see if there is anything we can do.
2) If all else fails you can run your IA64 machine with a 4K page size. We've always said that, for true complete IA32 support, you may have to run with 4K pages. I've put a lot of effort into allowing larger page sizes for `normal' prgrams, yours might be one that requires a 4K kernel.
3) I'm just curious, is this a real application or is this just a test case?
Quoted 32 lines · Hideki Yamamoto
Hello all.
If this problem has already known, I am sorry.
I am investigating the problem that calloc returns no cleared area.
It was made on IA32 system and executes on IA64. I think calloc expects to clear the memories sbrk returned. But When the program made on IA32 is executing on IA64 system, sbrk sometimes returns no cleared area.
Because the Kernel on IA64 alignes the memories 16K bytes. However, when the progmam made on IA32 is executing on IA64, the function__getpagesizereturns 4Kbytes. At this case, if the memories that calloc requests to return sbrk(-xxx) is smaller than 16Kbytes, the kernel do not unmap this memory.
(I think that the time that the memories clears 0 by the kernel is when the kernel maps memories by the signal)How do you think?
--
Regards,
Hideki Yamamoto (V).v.(V) # Empowered by Innovation_______________________________________________
Linux-IA64 mailing list
[email protected]
http://lists.linuxia64.org/lists/listinfo/linux-ia64Signature · 4 lines
Don Dugger "Censeo Toto nos in Kansa esse decisse." - D. Gale [email protected] Ph: 303/652-0870x117
David-
Yeah, as I said to Hideki I'll look into this. It might take a day or so, I can only do it in the evenings and I spent all last night configuring a `named' server (talk about a painfull experience :-(
Quoted 15 lines · David Mosberger
But I think the page size problem with
brk()is real and can be fixed quite easily. If I understood correctly, the problem is that doing something along the lines of:x = ALIGN_TO_4K(sbrk(8192)); memset(x, 0xff, 4096); brk(x); brk(x+4096);might preserve the contents of the page at X on under the ia32 subsystem of ia64 when in fact it should be cleared to zero.
Would you be able/interested into looking into this?
--david
Signature · 4 lines
Don Dugger "Censeo Toto nos in Kansa esse decisse." - D. Gale [email protected] Ph: 303/652-0870x117
I'm a little confused. From Hideki's test program the problem actually seems
to be using `sbrk' to return memory and then allocating memory again. The second
time the re-allocated memory is not zeroed, e.g.:
p = sbrk(4096);
q = sbrk(-4096);
r = sbrk(4096);and the memory pointed to by `r' is not zeroed. I just got through reading the
man page for brk' and sbrk' and neither one specifies that newly allocated memory is zeroed and therefore I interpret this issue as undefined behavior. Any program that depends upon the contents of newly allocated memory is broken. In fact, if you compile the test program, `d.c', for IA64 and run it it
fails. Also, if you compile the test program for IA32 and use 1K allocation blocks
the program fails on an IA32 machine also.
Someone correct me if I'm wrong about this.
PS: I know that `calloc' is defined to zero out the allocated memory but from
what I can tell it does this by explicitly calling `memset'. I'm trying to verify this
but I haven't found the calloc' code in the glibc' tree yet. (The `glibc' tree always
confuses me, if anyone can tell me exactly where to find the `calloc' code I'd
appreciate it :-)
--
Don Dugger
[email protected]
----- Original Message -----
From: "David Mosberger" <[email protected]>
To: <[email protected]>
Cc: "Hideki Yamamoto" <[email protected]>; <[email protected]>
Sent: Monday, November 12, 2001 4:58 PM
Subject: Re: [Linux-ia64] Pagesize is different between IA32 and IA64
Quoted 16 lines
But I think the page size problem with
brk()is real and can be fixed quite easily. If I understood correctly, the problem is that doing something along the lines of:x = ALIGN_TO_4K(sbrk(8192));
memset(x, 0xff, 4096);
brk(x);
brk(x+4096);might preserve the contents of the page at X on under the ia32 subsystem of ia64 when in fact it should be cleared to zero.
Would you be able/interested into looking into this?
--david
Never mind, I found the calloc' code and I'm very depressed because it does indeed assume that brk' zeros out newly allocated memory which is obviously going to create problems if `calloc' thinks the kernel has 4K pages when it really has bigger ones.
Unfortunately, this means I'll have to make an IA32 specific `brk' call that zero's out the last part of the last page currently allocated to a process. Oh well, patch to follow later.
--
Don Dugger
[email protected]
----- Original Message -----
From: "Dugger's" <[email protected]>
To: <[email protected]>; <[email protected]>
Cc: "Hideki Yamamoto" <[email protected]>; <[email protected]>
Sent: Tuesday, November 13, 2001 11:16 PM
Subject: Re: [Linux-ia64] Pagesize is different between IA32 and IA64
Quoted 11 lines
I'm a little confused. From Hideki's test program the problem actually seems
to be using `sbrk' to return memory and then allocating memory again. The second
time the re-allocated memory is not zeroed, e.g.:p = sbrk(4096); q = sbrk(-4096); r = sbrk(4096);and the memory pointed to by `r' is not zeroed. I just got through
reading
Quoted 2 lines
the
man page forbrk' andsbrk' and neither one specifies that newly
allocated
Quoted 1 line
memory is zeroed and therefore I interpret this issue as undefined
behavior.
Quoted 4 lines
Any program that depends upon the contents of newly allocated memory is broken. In fact, if you compile the test program, `d.c', for IA64 and run it it
fails. Also, if you compile the test program for IA32 and use 1K
allocation
Quoted 8 lines
blocks
the program fails on an IA32 machine also.Someone correct me if I'm wrong about this.
PS: I know that `calloc' is defined to zero out the allocated memory but from
what I can tell it does this by explicitly calling `memset'. I'm trying
to
Quoted 2 lines
verify this
but I haven't found thecalloc' code in theglibc' tree yet. (The
`glibc'
Quoted 33 lines
tree always
confuses me, if anyone can tell me exactly where to find the `calloc' code I'd
appreciate it :-)
--
Don Dugger
[email protected]
----- Original Message -----
From: "David Mosberger" <[email protected]>
To: <[email protected]>
Cc: "Hideki Yamamoto" <[email protected]>; <[email protected]>
Sent: Monday, November 12, 2001 4:58 PM
Subject: Re: [Linux-ia64] Pagesize is different between IA32 and IA64Quoted 16 lines
But I think the page size problem with
brk()is real and can be fixed quite easily. If I understood correctly, the problem is that doing something along the lines of:x = ALIGN_TO_4K(sbrk(8192));
memset(x, 0xff, 4096);
brk(x);
brk(x+4096);might preserve the contents of the page at X on under the ia32 subsystem of ia64 when in fact it should be cleared to zero.
Would you be able/interested into looking into this?
--david
Hi Don.
Yes, you are correct.
If my test program is run on IA64 system(not IA32 simulation), you should change the size from 4K to 16K. because the pagesize on IA64 is 16K. And I have heard if I use the sbrk function directly, The paramater for srbk must be the pagesize on running machine(except simulated machine, as a IA32 on IA64).
Please confirm the return value from __getpagesize function.
main() {
printf("0x%x\n", __getpagesize());
} BTW, calloc is included in malloc.c.
You would like to open the file glibc-2.2.4/malloc/malloc.c, and then you will be looking for cALLOc. It is calloc function.
At Tue, 13 Nov 2001 23:16:09 -0700,
Quoted 62 lines · Dugger's
I'm a little confused. From Hideki's test program the problem actually seems
to be using `sbrk' to return memory and then allocating memory again. The second
time the re-allocated memory is not zeroed, e.g.:p = sbrk(4096); q = sbrk(-4096); r = sbrk(4096);and the memory pointed to by `r' is not zeroed. I just got through reading the
man page forbrk' andsbrk' and neither one specifies that newly allocated memory is zeroed and therefore I interpret this issue as undefined behavior. Any program that depends upon the contents of newly allocated memory is broken. In fact, if you compile the test program, `d.c', for IA64 and run it it
fails. Also, if you compile the test program for IA32 and use 1K allocation blocks
the program fails on an IA32 machine also.Someone correct me if I'm wrong about this.
PS: I know that `calloc' is defined to zero out the allocated memory but from
what I can tell it does this by explicitly calling `memset'. I'm trying to verify this
but I haven't found thecalloc' code in theglibc' tree yet. (The `glibc' tree always
confuses me, if anyone can tell me exactly where to find the `calloc' code I'd
appreciate it :-)
--
Don Dugger
[email protected]
----- Original Message -----
From: "David Mosberger" <[email protected]>
To: <[email protected]>
Cc: "Hideki Yamamoto" <[email protected]>; <[email protected]>
Sent: Monday, November 12, 2001 4:58 PM
Subject: Re: [Linux-ia64] Pagesize is different between IA32 and IA64Quoted 16 lines
But I think the page size problem with
brk()is real and can be fixed quite easily. If I understood correctly, the problem is that doing something along the lines of:x = ALIGN_TO_4K(sbrk(8192));
memset(x, 0xff, 4096);
brk(x);
brk(x+4096);might preserve the contents of the page at X on under the ia32 subsystem of ia64 when in fact it should be cleared to zero.
Would you be able/interested into looking into this?
--david
Hi Don.
Thank you very much! I greatly appreciate your kindness.
At Tue, 13 Nov 2001 23:43:55 -0700,
Quoted 95 lines · Dugger's
Never mind, I found the
calloc' code and I'm very depressed because it does indeed assume thatbrk' zeros out newly allocated memory which is obviously going to create problems if `calloc' thinks the kernel has 4K pages when it really has bigger ones.Unfortunately, this means I'll have to make an IA32 specific `brk' call that zero's out the last part of the last page currently allocated to a process. Oh well, patch to follow later.
--
Don Dugger
[email protected]
----- Original Message -----
From: "Dugger's" <[email protected]>
To: <[email protected]>; <[email protected]>
Cc: "Hideki Yamamoto" <[email protected]>; <[email protected]>
Sent: Tuesday, November 13, 2001 11:16 PM
Subject: Re: [Linux-ia64] Pagesize is different between IA32 and IA64Quoted 11 lines
I'm a little confused. From Hideki's test program the problem actually seems
to be using `sbrk' to return memory and then allocating memory again. The second
time the re-allocated memory is not zeroed, e.g.:p = sbrk(4096); q = sbrk(-4096); r = sbrk(4096);and the memory pointed to by `r' is not zeroed. I just got through
reading
Quoted 2 lines
the
man page forbrk' andsbrk' and neither one specifies that newlyallocated
Quoted 1 line
memory is zeroed and therefore I interpret this issue as undefined
behavior.
Quoted 4 lines
Any program that depends upon the contents of newly allocated memory is broken. In fact, if you compile the test program, `d.c', for IA64 and run it it
fails. Also, if you compile the test program for IA32 and use 1Kallocation
Quoted 8 lines
blocks
the program fails on an IA32 machine also.Someone correct me if I'm wrong about this.
PS: I know that `calloc' is defined to zero out the allocated memory but from
what I can tell it does this by explicitly calling `memset'. I'm tryingto
Quoted 2 lines
verify this
but I haven't found thecalloc' code in theglibc' tree yet. (The`glibc'
Quoted 6 lines
tree always
confuses me, if anyone can tell me exactly where to find the `calloc' code I'd
appreciate it :-)
--
Don DuggerQuoted 27 lines
[email protected]
----- Original Message -----
From: "David Mosberger" <[email protected]>
To: <[email protected]>
Cc: "Hideki Yamamoto" <[email protected]>; <[email protected]>
Sent: Monday, November 12, 2001 4:58 PM
Subject: Re: [Linux-ia64] Pagesize is different between IA32 and IA64Quoted 16 lines
But I think the page size problem with
brk()is real and can be fixed quite easily. If I understood correctly, the problem is that doing something along the lines of:x = ALIGN_TO_4K(sbrk(8192));
memset(x, 0xff, 4096);
brk(x);
brk(x+4096);might preserve the contents of the page at X on under the ia32 subsystem of ia64 when in fact it should be cleared to zero.
Would you be able/interested into looking into this?
--david
_______________________________________________
Linux-IA64 mailing list
[email protected]
http://lists.linuxia64.org/lists/listinfo/linux-ia64
David-
OK, here's the patch to fix this. I guess the only issue is that we have to have Yet Another IA32 System call shim but other than that the change is pretty minor.
PS: Hideki, please note that this is a different, slightly better patch than the one I sent you last night. You should use this one.
Quoted 93 lines · Dugger's
Never mind, I found the
calloc' code and I'm very depressed because it does indeed assume thatbrk' zeros out newly allocated memory which is obviously going to create problems if `calloc' thinks the kernel has 4K pages when it really has bigger ones.Unfortunately, this means I'll have to make an IA32 specific `brk' call that zero's out the last part of the last page currently allocated to a process. Oh well, patch to follow later.
--
Don Dugger
[email protected]
----- Original Message -----
From: "Dugger's" <[email protected]>
To: <[email protected]>; <[email protected]>
Cc: "Hideki Yamamoto" <[email protected]>; <[email protected]>
Sent: Tuesday, November 13, 2001 11:16 PM
Subject: Re: [Linux-ia64] Pagesize is different between IA32 and IA64Quoted 11 lines
I'm a little confused. From Hideki's test program the problem actually seems
to be using `sbrk' to return memory and then allocating memory again. The second
time the re-allocated memory is not zeroed, e.g.:p = sbrk(4096); q = sbrk(-4096); r = sbrk(4096);and the memory pointed to by `r' is not zeroed. I just got through
reading
Quoted 2 lines
the
man page forbrk' andsbrk' and neither one specifies that newlyallocated
Quoted 1 line
memory is zeroed and therefore I interpret this issue as undefined
behavior.
Quoted 4 lines
Any program that depends upon the contents of newly allocated memory is broken. In fact, if you compile the test program, `d.c', for IA64 and run it it
fails. Also, if you compile the test program for IA32 and use 1Kallocation
Quoted 8 lines
blocks
the program fails on an IA32 machine also.Someone correct me if I'm wrong about this.
PS: I know that `calloc' is defined to zero out the allocated memory but from
what I can tell it does this by explicitly calling `memset'. I'm tryingto
Quoted 2 lines
verify this
but I haven't found thecalloc' code in theglibc' tree yet. (The`glibc'
Quoted 6 lines
tree always
confuses me, if anyone can tell me exactly where to find the `calloc' code I'd
appreciate it :-)
--
Don DuggerQuoted 27 lines
[email protected]
----- Original Message -----
From: "David Mosberger" <[email protected]>
To: <[email protected]>
Cc: "Hideki Yamamoto" <[email protected]>; <[email protected]>
Sent: Monday, November 12, 2001 4:58 PM
Subject: Re: [Linux-ia64] Pagesize is different between IA32 and IA64Quoted 16 lines
But I think the page size problem with
brk()is real and can be fixed quite easily. If I understood correctly, the problem is that doing something along the lines of:x = ALIGN_TO_4K(sbrk(8192));
memset(x, 0xff, 4096);
brk(x);
brk(x+4096);might preserve the contents of the page at X on under the ia32 subsystem of ia64 when in fact it should be cleared to zero.
Would you be able/interested into looking into this?
--david
_______________________________________________
Linux-IA64 mailing list
[email protected]
http://lists.linuxia64.org/lists/listinfo/linux-ia64
Signature · 55 lines
Don Dugger "Censeo Toto nos in Kansa esse decisse." - D. Gale [email protected] Ph: 303/652-0870x117 diff -aur kernel-bigsur-ref/arch/ia64/ia32/ia32_entry.S kernel-bigsur/arch/ia64/ia32/ia32_entry.S --- kernel-bigsur-ref/arch/ia64/ia32/ia32_entry.S Tue Sep 25 19:40:29 2001 +++ kernel-bigsur/arch/ia64/ia32/ia32_entry.S Wed Nov 14 21:07:28 2001 @@ -209,7 +209,7 @@ data8 sys32_pipe data8 sys32_times data8 sys32_ni_syscall /* old prof syscall holder */ - data8 sys_brk /* 45 */ + data8 sys32_brk /* 45 */ data8 sys_setgid /* 16-bit version */ data8 sys_getgid /* 16-bit version */ data8 sys32_signal diff -aur kernel-bigsur-ref/arch/ia64/ia32/sys_ia32.c kernel-bigsur/arch/ia64/ia32/sys_ia32.c --- kernel-bigsur-ref/arch/ia64/ia32/sys_ia32.c Tue Sep 25 19:40:29 2001 +++ kernel-bigsur/arch/ia64/ia32/sys_ia32.c Thu Nov 15 07:29:24 2001 @@ -68,6 +68,7 @@ /* forward declaration: */ asmlinkage long sys32_mprotect (unsigned int, unsigned int, int); +asmlinkage unsigned long sys_brk(unsigned long); static int nargs (unsigned int arg, char **ap) @@ -2123,6 +2124,7 @@ default: return -EINVAL; } + return -EINVAL; } /* @@ -3376,6 +3378,19 @@ ret = PER_LINUX; return ret; } + +asmlinkage unsigned long +sys32_brk(unsigned int brk) +{ + unsigned long ret, obrk; + struct mm_struct *mm = current->mm; + + obrk = mm->brk; + ret = sys_brk(brk); + if (ret < obrk) + clear_user(ret, PAGE_ALIGN(ret) - ret); + return(ret); +} #ifdef NOTYET /* UNTESTED FOR IA64 FROM HERE DOWN */
Hi,
Thank you for your patch!
Hideki tested it and it seems to work fine.
BTW, how about the following optimization?
IA32-glibc's calloc can call sbrk with 4K boundary.
When we do 4k sbrk, we do not need fully zero-ing up to 16K boundary, but just up to obrk.
(Think about four sbrk(-0x1000) case)
Thanks.
+asmlinkage unsigned long
+sys32_brk(unsigned int brk)
+{
+ unsigned long ret, obrk;
+ struct mm_struct *mm = current->mm;
+
+ obrk = mm->brk;
+ ret = sys_brk(brk);
+ if (ret < obrk){
+ unsigned long to;
+ to = (obrk < PAGE_ALIGN(ret) ? obrk : PAGE_ALIGN(ret));
+ clear_user(ret, to-ret);
+ }
+ return(ret);
+}
Quoted 167 lines
David-
OK, here's the patch to fix this. I guess the only issue is that we have to have Yet Another IA32 System call shim but other than that the change is pretty minor.
PS: Hideki, please note that this is a different, slightly better patch than the one I sent you last night. You should use this one.
Quoted 93 lines · Dugger's
Never mind, I found the
calloc' code and I'm very depressed because it does indeed assume thatbrk' zeros out newly allocated memory which is obviously going to create problems if `calloc' thinks the kernel has 4K pages when it really has bigger ones.Unfortunately, this means I'll have to make an IA32 specific `brk' call that zero's out the last part of the last page currently allocated to a process. Oh well, patch to follow later.
--
Don Dugger
[email protected]
----- Original Message -----
From: "Dugger's" <[email protected]>
To: <[email protected]>; <[email protected]>
Cc: "Hideki Yamamoto" <[email protected]>; <[email protected]>
Sent: Tuesday, November 13, 2001 11:16 PM
Subject: Re: [Linux-ia64] Pagesize is different between IA32 and IA64Quoted 11 lines
I'm a little confused. From Hideki's test program the problem actually seems
to be using `sbrk' to return memory and then allocating memory again. The second
time the re-allocated memory is not zeroed, e.g.:p = sbrk(4096); q = sbrk(-4096); r = sbrk(4096);and the memory pointed to by `r' is not zeroed. I just got through
reading
Quoted 2 lines
the
man page forbrk' andsbrk' and neither one specifies that newlyallocated
Quoted 1 line
memory is zeroed and therefore I interpret this issue as undefined
behavior.
Quoted 4 lines
Any program that depends upon the contents of newly allocated memory is broken. In fact, if you compile the test program, `d.c', for IA64 and run it it
fails. Also, if you compile the test program for IA32 and use 1Kallocation
Quoted 8 lines
blocks
the program fails on an IA32 machine also.Someone correct me if I'm wrong about this.
PS: I know that `calloc' is defined to zero out the allocated memory but from
what I can tell it does this by explicitly calling `memset'. I'm tryingto
Quoted 2 lines
verify this
but I haven't found thecalloc' code in theglibc' tree yet. (The`glibc'
Quoted 6 lines
tree always
confuses me, if anyone can tell me exactly where to find the `calloc' code I'd
appreciate it :-)
--
Don DuggerQuoted 27 lines
[email protected]
----- Original Message -----
From: "David Mosberger" <[email protected]>
To: <[email protected]>
Cc: "Hideki Yamamoto" <[email protected]>; <[email protected]>
Sent: Monday, November 12, 2001 4:58 PM
Subject: Re: [Linux-ia64] Pagesize is different between IA32 and IA64Quoted 16 lines
But I think the page size problem with
brk()is real and can be fixed quite easily. If I understood correctly, the problem is that doing something along the lines of:x = ALIGN_TO_4K(sbrk(8192));
memset(x, 0xff, 4096);
brk(x);
brk(x+4096);might preserve the contents of the page at X on under the ia32 subsystem of ia64 when in fact it should be cleared to zero.
Would you be able/interested into looking into this?
--david
_______________________________________________
Linux-IA64 mailing list
[email protected]
http://lists.linuxia64.org/lists/listinfo/linux-ia64Signature · 6 lines
Don Dugger "Censeo Toto nos in Kansa esse decisse." - D. Gale [email protected] Ph: 303/652-0870x117 diff -aur kernel-bigsur-ref/arch/ia64/ia32/ia32_entry.S kernel-bigsur/arch/ia64/ia32/ia32_entry.Sarch/ia64/ia32/ia32_entry.S +1
-1@@ -209,7 +209,7 @@ data8 sys32_pipe data8 sys32_times data8 sys32_ni_syscall /* old prof syscall holder */ - data8 sys_brk /* 45 */ + data8 sys32_brk /* 45 */ data8 sys_setgid /* 16-bit version */ data8 sys_getgid /* 16-bit version */ data8 sys32_signaldiff -aur
kernel-bigsur-ref/arch/ia64/ia32/sys_ia32.ckernel-bigsur/arch/ia64/ia32/sys_ia32.carch/ia64/ia32/sys_ia32.c +15
-0@@ -68,6 +68,7 @@/* forward declaration: */ asmlinkage long sys32_mprotect (unsigned int ,unsigned int ,int ); +asmlinkage unsigned long sys_brk (unsigned long );static int nargs (unsigned int arg,char **ap)@@ -2123,6 +2124,7 @@default :return -EINVAL ; } +return -EINVAL ; }/* @@ -3376,6 +3378,19 @@ ret =PER_LINUX ;return ret; } + +asmlinkage unsigned long +sys32_brk (unsigned int brk) +{ +unsigned long ret, obrk; +struct mm_struct *mm = current->mm; + + obrk = mm->brk; + ret =sys_brk (brk); +if (ret < obrk) +clear_user (ret,PAGE_ALIGN (ret) - ret); +return (ret); +}#ifdef NOTYET /* UNTESTED FOR IA64 FROM HERE DOWN */ _______________________________________________
Linux-IA64 mailing list
[email protected]
http://lists.linuxia64.org/lists/listinfo/linux-ia64
Miyoshi-
Glad to hear that.
I thought about doing that but that would have required an extra check, zeroing to the smaller of `obrk' or the end of the current page. I prefer the simplicity of just zeroing the page unless someone can demonstrate a program that shows a measurable performance gain from this change.
Quoted 207 lines · On Fri, Nov 16, 2001 at 03:17:09PM +0900, [email protected] wrote:
Hi,
Thank you for your patch!
Hideki tested it and it seems to work fine.BTW, how about the following optimization?
IA32-glibc's calloc can call sbrk with 4K boundary.
When we do 4k sbrk, we do not need fully zero-ing up to 16K boundary, but just up to obrk.(Think about four sbrk(-0x1000) case)
Thanks.
(quoted diff) +15
-0+asmlinkage unsigned long +sys32_brk(unsigned int brk) +{ + unsigned long ret, obrk; + struct mm_struct *mm = current->mm; + + obrk = mm->brk; + ret = sys_brk(brk); + if (ret < obrk){ + unsigned long to; + to = (obrk < PAGE_ALIGN(ret) ? obrk : PAGE_ALIGN(ret)); + clear_user(ret, to-ret); + } + return(ret); +}Quoted 167 lines
David-
OK, here's the patch to fix this. I guess the only issue is that we have to have Yet Another IA32 System call shim but other than that the change is pretty minor.
PS: Hideki, please note that this is a different, slightly better patch than the one I sent you last night. You should use this one.
Quoted 93 lines · Dugger's
Never mind, I found the
calloc' code and I'm very depressed because it does indeed assume thatbrk' zeros out newly allocated memory which is obviously going to create problems if `calloc' thinks the kernel has 4K pages when it really has bigger ones.Unfortunately, this means I'll have to make an IA32 specific `brk' call that zero's out the last part of the last page currently allocated to a process. Oh well, patch to follow later.
--
Don Dugger
[email protected]
----- Original Message -----
From: "Dugger's" <[email protected]>
To: <[email protected]>; <[email protected]>
Cc: "Hideki Yamamoto" <[email protected]>; <[email protected]>
Sent: Tuesday, November 13, 2001 11:16 PM
Subject: Re: [Linux-ia64] Pagesize is different between IA32 and IA64Quoted 11 lines
I'm a little confused. From Hideki's test program the problem actually seems
to be using `sbrk' to return memory and then allocating memory again. The second
time the re-allocated memory is not zeroed, e.g.:p = sbrk(4096); q = sbrk(-4096); r = sbrk(4096);and the memory pointed to by `r' is not zeroed. I just got through
reading
Quoted 2 lines
the
man page forbrk' andsbrk' and neither one specifies that newlyallocated
Quoted 1 line
memory is zeroed and therefore I interpret this issue as undefined
behavior.
Quoted 4 lines
Any program that depends upon the contents of newly allocated memory is broken. In fact, if you compile the test program, `d.c', for IA64 and run it it
fails. Also, if you compile the test program for IA32 and use 1Kallocation
Quoted 8 lines
blocks
the program fails on an IA32 machine also.Someone correct me if I'm wrong about this.
PS: I know that `calloc' is defined to zero out the allocated memory but from
what I can tell it does this by explicitly calling `memset'. I'm tryingto
Quoted 2 lines
verify this
but I haven't found thecalloc' code in theglibc' tree yet. (The`glibc'
Quoted 6 lines
tree always
confuses me, if anyone can tell me exactly where to find the `calloc' code I'd
appreciate it :-)
--
Don DuggerQuoted 27 lines
[email protected]
----- Original Message -----
From: "David Mosberger" <[email protected]>
To: <[email protected]>
Cc: "Hideki Yamamoto" <[email protected]>; <[email protected]>
Sent: Monday, November 12, 2001 4:58 PM
Subject: Re: [Linux-ia64] Pagesize is different between IA32 and IA64Quoted 16 lines
But I think the page size problem with
brk()is real and can be fixed quite easily. If I understood correctly, the problem is that doing something along the lines of:x = ALIGN_TO_4K(sbrk(8192));
memset(x, 0xff, 4096);
brk(x);
brk(x+4096);might preserve the contents of the page at X on under the ia32 subsystem of ia64 when in fact it should be cleared to zero.
Would you be able/interested into looking into this?
--david
_______________________________________________
Linux-IA64 mailing list
[email protected]
http://lists.linuxia64.org/lists/listinfo/linux-ia64Signature · 6 lines
Don Dugger "Censeo Toto nos in Kansa esse decisse." - D. Gale [email protected] Ph: 303/652-0870x117 diff -aur kernel-bigsur-ref/arch/ia64/ia32/ia32_entry.S kernel-bigsur/arch/ia64/ia32/ia32_entry.Sarch/ia64/ia32/ia32_entry.S +1
-1@@ -209,7 +209,7 @@ data8 sys32_pipe data8 sys32_times data8 sys32_ni_syscall /* old prof syscall holder */ - data8 sys_brk /* 45 */ + data8 sys32_brk /* 45 */ data8 sys_setgid /* 16-bit version */ data8 sys_getgid /* 16-bit version */ data8 sys32_signaldiff -aur
kernel-bigsur-ref/arch/ia64/ia32/sys_ia32.ckernel-bigsur/arch/ia64/ia32/sys_ia32.carch/ia64/ia32/sys_ia32.c +15
-0@@ -68,6 +68,7 @@/* forward declaration: */ asmlinkage long sys32_mprotect (unsigned int ,unsigned int ,int ); +asmlinkage unsigned long sys_brk (unsigned long );static int nargs (unsigned int arg,char **ap)@@ -2123,6 +2124,7 @@default :return -EINVAL ; } +return -EINVAL ; }/* @@ -3376,6 +3378,19 @@ ret =PER_LINUX ;return ret; } + +asmlinkage unsigned long +sys32_brk (unsigned int brk) +{ +unsigned long ret, obrk; +struct mm_struct *mm = current->mm; + + obrk = mm->brk; + ret =sys_brk (brk); +if (ret < obrk) +clear_user (ret,PAGE_ALIGN (ret) - ret); +return (ret); +}#ifdef NOTYET /* UNTESTED FOR IA64 FROM HERE DOWN */ Quoted 5 lines
_______________________________________________
Linux-IA64 mailing list
[email protected]
http://lists.linuxia64.org/lists/listinfo/linux-ia64_______________________________________________
Linux-IA64 mailing list
[email protected]
http://lists.linuxia64.org/lists/listinfo/linux-ia64
Signature · 4 lines
Don Dugger "Censeo Toto nos in Kansa esse decisse." - D. Gale [email protected] Ph: 303/652-0870x117
Hi,
Sorry for late reply.
Quoted 7 lines
Glad to hear that.
I thought about doing that but that would have required an extra check, zeroing to the smaller of `obrk' or the end of the current page. I prefer the simplicity of just zeroing the page unless someone can demonstrate a program that shows a measurable performance gain from this change.
Hmm... Simplicity also appeals to me :-)
I just did not like unnecessary copy, because copying takes most of the CPU cycles.
But your code clears just when brk is issued with minus offset and it seems to be rare (perhaps never with glibc's calloc?)
So it seems OK that simplicity takes priority.
Thanks.