Skip to contentSkip to conversation

kernel stack size

DISCUSSION Manfred Spraul 2 participants 5 messages Apr 2, 2005 – Apr 3, 2005

lore
Share & tools
lore
#0 lore raw
Quoted 6 lines · Steven Rostedt

I admit you really need to know what you're doing to use this method. If I believe that a kmalloc would be too expensive, then I use the locking of static variables. But each situation is different and I try to use the best method for the occasion.

Have you benchmarked your own memory manager?
kmalloc(1024, GFP_KERNEL) is something like 17 instructions on i386 uniprocessor.

--
Manfred

#1 lore raw
Quoted 11 lines · Manfred Spraul
Quoted 6 lines · Steven Rostedt

I admit you really need to know what you're doing to use this method. If I believe that a kmalloc would be too expensive, then I use the locking of static variables. But each situation is different and I try to use the best method for the occasion.

Have you benchmarked your own memory manager? kmalloc(1024, GFP_KERNEL) is something like 17 instructions on i386 uniprocessor.

Where did you get that? I'm looking at the assembly of it right now and it's much larger than 17 instructions. Not to mention that it calls the slab functions which might have to invoke the buddy system.

Also, I don't use my own memory manager. My memory manager would be the statically allocated globals (allocated automatically when the kernel loads at boot up) and spin_locks (which are much smaller than kmalloc) or sems. Now if kmalloc didn't have a free slab available, and needed to go to the buddy list, this gets expensive, especially if you have to contend with other processes doing the same.

With the static global variable method, you only have to worry about processes (and interrupts) that are contending for your data. This can be very efficient, especially if the data IS shared with an interrupt handler. And if you want to be more efficient, just use the normal spin_lock after disabling just your interrupt. Now you don't stop other interrupts coming in, and still can work with your own global data.

Since the original poster was talking about local data, and I'm talking about global, I sometimes use global variables for just local use, but you need to lock the data so that on SMP, or PREEMPT you don't worry about reentry. I haven't clocked the speed of sem compared to kmalloc. But I would think that the sem functions are still quicker.

Like I mentioned before, each case is different. I do use kmalloc when I find that there will be too much contention with the data, or that I would need to lock the data for long periods of time. Then again, a sem may work too.

-- Steve

#2 lore raw
Quoted 11 lines · Steven Rostedt
Quoted 4 lines

Have you benchmarked your own memory manager?
kmalloc(1024, GFP_KERNEL) is something like 17 instructions on i386 uniprocessor.

Where did you get that? I'm looking at the assembly of it right now and it's much larger than 17 instructions. Not to mention that it calls the slab functions which might have to invoke the buddy system.

Have you looked at kmem_cache_alloc? kmalloc(1024, GFP_KERNEL) is compile-time replaced with the appropriate kmem_cache_alloc call. And the fast path within kmem_cache_alloc is 17 instructions long. Best case: uniprocessor, no regparams. Unfortunately with cli and popfd, thus something like 35 cpu cycles on an Athlon 64.

Quoted 5 lines

I haven't clocked the speed of sem compared to kmalloc. But I would think that the sem functions are still quicker.

Yes - sem or spin locks are quicker as long as no cache line transfers are necessary. If the semaphore is accessed by multiple cpus, then kmalloc would be faster: slab tries hard to avoid taking global locks. I'm not speaking about contention, just the cache line ping pong for acquiring a free semaphore.

--
Manfred

#3 lore raw
Quoted 5 lines · Manfred Spraul

Yes - sem or spin locks are quicker as long as no cache line transfers are necessary. If the semaphore is accessed by multiple cpus, then kmalloc would be faster: slab tries hard to avoid taking global locks. I'm not speaking about contention, just the cache line ping pong for acquiring a free semaphore.

Without contention, is there still a problem with cache line ping pong of acquiring a free semaphore?

I mean, say only one task is using a given semaphore. Is there still going to be cache line transfers for acquiring it? Even if the task in question stays on a CPU. Is the "LOCK" on an instruction that expensive even if the other CPUs haven't accessed that location of memory.

Sorry for my ignorance, I don't know all the interworkings of the Cache on SMP systems. Is there any good references on the Internet? I definitely want to know so that my coding practices for SMP improve.

Thanks,

-- Steve

#4 lore raw
Quoted 8 lines · Steven Rostedt
Quoted 3 lines · Manfred Spraul

Yes - sem or spin locks are quicker as long as no cache line transfers are necessary. If the semaphore is accessed by multiple cpus, then kmalloc would be faster: slab tries hard to avoid taking global locks. I'm not speaking about contention, just the cache line ping pong for acquiring a free semaphore.

Quoted 10 lines

Without contention, is there still a problem with cache line ping pong of acquiring a free semaphore?

I mean, say only one task is using a given semaphore. Is there still going to be cache line transfers for acquiring it? Even if the task in question stays on a CPU. Is the "LOCK" on an instruction that expensive even if the other CPUs haven't accessed that location of memory.

No. If everything is cpu-local, then there are obviously no cache line transfers. LOCK is not that expensive. On a Pentium 3, it was 20 cpu cycles. On an Athlon 64, it's virtually free.

--
Manfred