Skip to contentSkip to conversation

bootmem allocator

DISCUSSION Cyrill Gorcunov 6 participants 16 messages Apr 7, 2008 – Apr 9, 2008

lore
Share & tools
lore
#0 lore raw

Hi Ingo, Peter,

small question. It was a patch recently posted which removes memset(x, 0, x) after __alloc_bootmem call. There are a few another code snippets who still call memset(x, 0, x). And who is responsible for memory clearing? bootmem allocator or caller?

- Cyrill -

#1 lore raw
Quoted 6 lines · Cyrill Gorcunov

Hi Ingo, Peter,

small question. It was a patch recently posted which removes memset(x, 0, x) after __alloc_bootmem call. There are a few another code snippets who still call memset(x, 0, x). And who is responsible for memory clearing? bootmem allocator or caller?

hm, bootmem allocator is supposed to clear memory. We have a couple of places that rely on that.

Ingo

#2 lore raw
Quoted 11 lines · Ingo Molnar
Quoted 6 lines · Cyrill Gorcunov

Hi Ingo, Peter,

small question. It was a patch recently posted which removes memset(x, 0, x) after __alloc_bootmem call. There are a few another code snippets who still call memset(x, 0, x). And who is responsible for memory clearing? bootmem allocator or caller?

hm, bootmem allocator is supposed to clear memory. We have a couple of places that rely on that.

I was actually considering to change that for the GB pages hugetlbfs patchkit, because memset for 1G is a little slow and not needed (will be cleared later anyways) and it might be a problem for very large systems with a lot of such pages at boot.

-Andi

#3 lore raw
Quoted 12 lines · Andi Kleen
Quoted 9 lines · Ingo Molnar
Quoted 6 lines · Cyrill Gorcunov

Hi Ingo, Peter,

small question. It was a patch recently posted which removes memset(x, 0, x) after __alloc_bootmem call. There are a few another code snippets who still call memset(x, 0, x). And who is responsible for memory clearing? bootmem allocator or caller?

Quoted 7 lines
Quoted 2 lines

hm, bootmem allocator is supposed to clear memory. We have a couple of places that rely on that.

I was actually considering to change that for the GB pages hugetlbfs patchkit, because memset for 1G is a little slow and not needed (will be cleared later anyways) and it might be a problem for very large systems with a lot of such pages at boot.

add another zalloc_bootmem?

YH

#4 lore raw
Quoted 17 lines · Yinghai Lu
Quoted 15 lines · Andi Kleen
Quoted 11 lines · Ingo Molnar
Quoted 6 lines · Cyrill Gorcunov

Hi Ingo, Peter,

small question. It was a patch recently posted which removes memset(x, 0, x) after __alloc_bootmem call. There are a few another code snippets who still call memset(x, 0, x). And who is responsible for memory clearing? bootmem allocator or caller?

hm, bootmem allocator is supposed to clear memory. We have a couple of places that rely on that.

Quoted 9 lines
Quoted 4 lines

I was actually considering to change that for the GB pages hugetlbfs patchkit, because memset for 1G is a little slow and not needed (will be cleared later anyways) and it might be a problem for very large systems with a lot of such pages at boot.

add another zalloc_bootmem?

YH

I think it would be a good idea ;) Btw maybe would be better to call memset on the code witch relies on "clear" memory explicitly? So we will clear memory allocated *only* if we really need this.

Cyrill

#5 lore raw
Quoted 5 lines · Cyrill Gorcunov

I think it would be a good idea ;) Btw maybe would be better to call memset on the code witch relies on "clear" memory explicitly? So we will clear memory allocated *only* if we really need this.

Are there any users of bootmem which will allocate a significant amount of memory and don't need it zeroed?

-hpa

#6 lore raw
Quoted 12 lines · H. Peter Anvin
Quoted 4 lines · Cyrill Gorcunov

I think it would be a good idea ;) Btw maybe would be better to call memset on the code witch relies on "clear" memory explicitly? So we will clear memory allocated *only* if we really need this.

Are there any users of bootmem which will allocate a significant amount of
memory and don't need it zeroed?

-hpa

Actually, I don't know without reading the code... will take a look today evening
(have no access to sources in my office).

#7 lore raw
In reply to H. Peter Anvin

[H. Peter Anvin - Mon, Apr 07, 2008 at 09:33:23PM -0700]

Quoted 10 lines
Quoted 3 lines · Cyrill Gorcunov

I think it would be a good idea ;) Btw maybe would be better to call memset on the code witch relies on "clear" memory explicitly? So we will clear memory allocated *only* if we really need this.

Are there any users of bootmem which will allocate a significant amount of memory and don't need it zeroed?

-hpa

Well, the only really significant allocation I found is in arch/ia64/kernel/iosapi.c:

---[...]---
static struct iosapic_rte_info * __init_refok iosapic_alloc_rte (void)

{
	int i;
	struct iosapic_rte_info *rte;
	int preallocated = 0;

	if (!iosapic_kmalloc_ok && list_empty(&free_rte_list)) {
--->		rte = alloc_bootmem(sizeof(struct iosapic_rte_info) *
				    NR_PREALLOCATE_RTE_ENTRIES);
		if (!rte)
			return NULL;
		for (i = 0; i < NR_PREALLOCATE_RTE_ENTRIES; i++, rte++)
			list_add(&rte->rte_list, &free_rte_list);
	}

	if (!list_empty(&free_rte_list)) {
		rte = list_entry(free_rte_list.next, struct iosapic_rte_info,
				 rte_list);
		list_del(&rte->rte_list);
		preallocated++;
	} else {
		rte = kmalloc(sizeof(struct iosapic_rte_info), GFP_ATOMIC);
		if (!rte)
			return NULL;
	}

	memset(rte, 0, sizeof(struct iosapic_rte_info));
	if (preallocated)
		rte->flags |= RTE_PREALLOCATED;

	return rte;
}

---[...]---

but it requires zeroed memory too. So, no, I didn't found any large number of bytes allocated by bootmem scheme without needing of its clearing.

- Cyrill -

#8 lore raw
In reply to Andi Kleen
Quoted 7 lines · Andi Kleen
Quoted 2 lines

hm, bootmem allocator is supposed to clear memory. We have a couple of places that rely on that.

I was actually considering to change that for the GB pages hugetlbfs patchkit, because memset for 1G is a little slow and not needed (will be cleared later anyways) and it might be a problem for very large systems with a lot of such pages at boot.

changing the default behavior of bootmem alloc to be non-clearing is a really bad idea that will only cause unrobustness. The proper approach is to add an _opt-in_ API that does not clear memory (bootmem_alloc_dontclear() or whatever), available to callers that know it for sure that they dont need the clearing.

Ingo

#9 lore raw
Quoted 6 lines · Ingo Molnar
Quoted 3 lines · Andi Kleen
Quoted 2 lines

hm, bootmem allocator is supposed to clear memory. We have a couple of places that rely on that.

Quoted 8 lines
Quoted 4 lines

I was actually considering to change that for the GB pages hugetlbfs patchkit, because memset for 1G is a little slow and not needed (will be cleared later anyways) and it might be a problem for very large systems with a lot of such pages at boot.

changing the default behavior of bootmem alloc to be non-clearing is a really bad idea that will only cause unrobustness. The proper approach is to add an _opt-in_ API that does not clear memory

I was considering that too, but we have so many weird variants of bootmem with opt in and opt out and even combinations of both now that the whole thing is starting to look really pear shaped (I admit I added some of them in the past myself but I'm not proud). Would be a great project for someone to consolidate that all a bit.

-Andi

#10 lore raw

[Andi Kleen - Tue, Apr 08, 2008 at 10:12:06AM +0200]

| On Tue, Apr 08, 2008 at 10:04:46AM +0200, Ingo Molnar wrote:
| > 
| > * Andi Kleen <[email protected]> wrote:
| > 
| > > > hm, bootmem allocator is supposed to clear memory. We have a couple 
| > > > of places that rely on that.
| > > 
| > > I was actually considering to change that for the GB pages hugetlbfs 
| > > patchkit, because memset for 1G is a little slow and not needed (will 
| > > be cleared later anyways) and it might be a problem for very large 
| > > systems with a lot of such pages at boot.
| > 
| > changing the default behavior of bootmem alloc to be non-clearing is a 
| > really bad idea that will only cause unrobustness. The proper approach 
| > is to add an _opt-in_ API that does not clear memory 
| 
| I was considering that too, but we have so many weird variants of bootmem
| with opt in and opt out and even combinations of both now that the whole thing 
| is starting to look really pear shaped (I admit I added some of them
| in the past myself but I'm not proud). Would be a great project for
| someone to consolidate that all a bit.
| 
| -Andi
| 

OK, thanks to all of you

- Cyrill -

#11 lore raw
In reply to Andi Kleen

[Andi Kleen - Tue, Apr 08, 2008 at 10:12:06AM +0200]

| On Tue, Apr 08, 2008 at 10:04:46AM +0200, Ingo Molnar wrote:
| > 
| > * Andi Kleen <[email protected]> wrote:
| > 
| > > > hm, bootmem allocator is supposed to clear memory. We have a couple 
| > > > of places that rely on that.
| > > 
| > > I was actually considering to change that for the GB pages hugetlbfs 
| > > patchkit, because memset for 1G is a little slow and not needed (will 
| > > be cleared later anyways) and it might be a problem for very large 
| > > systems with a lot of such pages at boot.
| > 
| > changing the default behavior of bootmem alloc to be non-clearing is a 
| > really bad idea that will only cause unrobustness. The proper approach 
| > is to add an _opt-in_ API that does not clear memory 
| 
| I was considering that too, but we have so many weird variants of bootmem
| with opt in and opt out and even combinations of both now that the whole thing 
| is starting to look really pear shaped (I admit I added some of them
| in the past myself but I'm not proud). Would be a great project for
| someone to consolidate that all a bit.
| 
| -Andi
| 

Andi, could you a bit clarify what exactly do you mean?

- Cyrill -

#12 lore raw
In reply to Andi Kleen
Quoted 12 lines · Andi Kleen
Quoted 5 lines · Ingo Molnar
Quoted 2 lines · Andi Kleen
Quoted 1 line

hm, bootmem allocator is supposed to clear memory. We have a couple of places that rely on that.

I was actually considering to change that for the GB pages hugetlbfs patchkit, because memset for 1G is a little slow and not needed (will be cleared later anyways) and it might be a problem for very large systems with a lot of such pages at boot.

changing the default behavior of bootmem alloc to be non-clearing is a really bad idea that will only cause unrobustness. The proper approach is to add an _opt-in_ API that does not clear memory

I was considering that too, but we have so many weird variants of bootmem
with opt in and opt out and even combinations of both now that the whole thing is starting to look really pear shaped (I admit I added some of them in the past myself but I'm not proud). Would be a great project for
someone to consolidate that all a bit.

True, but if the gbpages hugetlbfs case is the *only* case which wants a nonclearing interface (which it sounds like it is), then it seems like a really bad idea to change the default.

hugetlbfs is a pretty special case.

-hpa

#13 lore raw
In reply to Ingo Molnar
Quoted 9 lines · Ingo Molnar
Quoted 6 lines · Andi Kleen
Quoted 2 lines

hm, bootmem allocator is supposed to clear memory. We have a couple of places that rely on that.

I was actually considering to change that for the GB pages hugetlbfs patchkit, because memset for 1G is a little slow and not needed (will be cleared later anyways) and it might be a problem for very large systems with a lot of such pages at boot.

Quoted 5 lines

changing the default behavior of bootmem alloc to be non-clearing is a really bad idea that will only cause unrobustness. The proper approach is to add an _opt-in_ API that does not clear memory (bootmem_alloc_dontclear() or whatever), available to callers that know it for sure that they dont need the clearing.

Yes, changing the default of bootmem_alloc is a bad idea. I just changed a bunch of static arrays to bootmem alloc's and it was pointed out early that not only does bootmem_alloc clear memory, but also panics if it's not available.

Quoted 4 lines

Also take into account that alloc_bootmem_low() :
- calls panic() if not enough memory
- already clears allocated memory

A specialized call to bootmem_alloc probably needs it's own API... ;-)

Thanks,
Mike

#14 lore raw
Quoted 16 lines · Mike Travis
Quoted 9 lines · Ingo Molnar
Quoted 6 lines · Andi Kleen
Quoted 2 lines

hm, bootmem allocator is supposed to clear memory. We have a couple of places that rely on that.

I was actually considering to change that for the GB pages hugetlbfs patchkit, because memset for 1G is a little slow and not needed (will be cleared later anyways) and it might be a problem for very large systems with a lot of such pages at boot.

Quoted 5 lines

changing the default behavior of bootmem alloc to be non-clearing is a really bad idea that will only cause unrobustness. The proper approach is to add an _opt-in_ API that does not clear memory (bootmem_alloc_dontclear() or whatever), available to callers that know it for sure that they dont need the clearing.

Quoted 4 lines

Yes, changing the default of bootmem_alloc is a bad idea. I just changed a bunch of static arrays to bootmem alloc's and it was pointed out early that not only does bootmem_alloc clear memory, but also panics if it's not available.

There are more and more bootmem calls that don't want the panic actually. That is why _nopanic was invented (and gets more and more variants) At some point the default could be even switched.

I think the right way would be to survey the callers (there are not that many) and then come up with a sane single API that caters to the majority of them by default and passes flags for the special cases.

-Andi

#15 lore raw
Quoted 24 lines · Andi Kleen
Quoted 14 lines · Mike Travis
Quoted 8 lines · Ingo Molnar
Quoted 6 lines · Andi Kleen
Quoted 2 lines

hm, bootmem allocator is supposed to clear memory. We have a couple of places that rely on that.

I was actually considering to change that for the GB pages hugetlbfs patchkit, because memset for 1G is a little slow and not needed (will be cleared later anyways) and it might be a problem for very large systems with a lot of such pages at boot.

Quoted 5 lines

changing the default behavior of bootmem alloc to be non-clearing is a really bad idea that will only cause unrobustness. The proper approach is to add an _opt-in_ API that does not clear memory (bootmem_alloc_dontclear() or whatever), available to callers that know it for sure that they dont need the clearing.

Quoted 4 lines

Yes, changing the default of bootmem_alloc is a bad idea. I just changed a bunch of static arrays to bootmem alloc's and it was pointed out early that not only does bootmem_alloc clear memory, but also panics if it's not available.

There are more and more bootmem calls that don't want the panic actually. That is why _nopanic was invented (and gets more and more variants) At some point the default could be even switched.

Quoted 5 lines

I think the right way would be to survey the callers (there are not that many) and then come up with a sane single API that caters to the majority of them by default and passes flags for the special cases.

-Andi

Hi Andi,

I really don't care(*), but there's lot's of code that expects a certain behavior. Either all the source calls have to be modified en masse (and you well know that's difficult given the _zillion_ source trees), or you have to introduce the new API transparently. That means leaving a backdoor for old calls:

     #define bootmem_alloc_low(...) \
	 __new_bootmem_alloc(..., FLAGS_FOR_OLD_BOOTMEM_ALLOC_LOW);

Then I think you're free to optimize away... d;-) [happy face with a baseball cap]

Cheers,
Mike

(*) As long as I don't have to debug problems as a result of the change...! ;-)