parallel directory operations
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 2 lines · Alex Tomas
dynamic locks. supports exclusive and shared locks. exclusive lock may be taken several times by first owner.
What's the difference between these locks and the existing rw semaphores?
-Andi
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Andi Kleen (AK) writes:
Quoted 2 lines · AK> Alex Tomas <[email protected]> writes:
Quoted 2 lines
dynamic locks. supports exclusive and shared locks. exclusive lock may be taken several times by first owner.
AK> What's the difference between these locks and the existing rw semaphores?
dynlocks implements 'lock namespace', so you can lock A for namepace N1 and lock B for namespace N1 and so on. we need this because we want to take lock on _part_ of directory.
Quoted 3 lines · On Tue, 08 Jul 2003 15:28:27 +0000 [email protected] wrote:
dynlocks implements 'lock namespace', so you can lock A for namepace N1 and lock B for namespace N1 and so on. we need this because we want to take lock on _part_ of directory.
Ok, a mini database lock manager. Wouldn't it be better to use a small hash table and lock escalation on overflow for this? Otherwise you could have quite a lot of entries queued up in the list if the server is slow.
-Andi
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Andi Kleen (AK) writes:
AK> On Tue, 08 Jul 2003 15:28:27 +0000
Quoted 3 lines · AK> [email protected] wrote:
Quoted 3 lines
dynlocks implements 'lock namespace', so you can lock A for namepace N1 and lock B for namespace N1 and so on. we need this because we want to take lock on _part_ of directory.
AK> Ok, a mini database lock manager. Wouldn't it be better to use a small hash AK> table and lock escalation on overflow for this? Otherwise you could AK> have quite a lot of entries queued up in the list if the server is slow.
well, it makes sense. AFAIU, only problem with this solution is that we need very well-tuned hash function. BTW, dynlocks are taken for operation time only. so, in most often case, for dir entry creation/lookup we need two locks: one for dcache locking and another for htree's leaf locking.
Quoted 2 lines · On Tue, 08 Jul 2003 15:50:27 +0000 [email protected] wrote:
well, it makes sense. AFAIU, only problem with this solution is that we need very well-tuned hash function.
A small rbtree or similar would work too. Linux already has the utility code for this. And a fast path to avoid the overhead when it isn't needed (e.g. first locker uses a preallocated lock node, which is cheap to queue)
-Andi
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Andi Kleen (AK) writes:
AK> On Tue, 08 Jul 2003 15:50:27 +0000
Quoted 2 lines · AK> [email protected] wrote:
Quoted 2 lines
well, it makes sense. AFAIU, only problem with this solution is that we need very well-tuned hash function.
AK> A small rbtree or similar would work too. Linux already has the utility code for this. AK> And a fast path to avoid the overhead when it isn't needed (e.g. first locker uses a AK> preallocated lock node, which is cheap to queue)
hmm. interesting! thanks for review.
Quoted 5 lines · Alex Tomas
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Andi Kleen (AK) writes:
AK> On Tue, 08 Jul 2003 15:50:27 +0000
AK> [email protected] wrote:
Quoted 9 lines
Quoted 2 lines
Quoted 2 lines
well, it makes sense. AFAIU, only problem with this solution is that we need very well-tuned hash function.
AK> A small rbtree or similar would work too. Linux already has the utility code for this. AK> And a fast path to avoid the overhead when it isn't needed (e.g. first locker uses a AK> preallocated lock node, which is cheap to queue)
hmm. interesting! thanks for review.
By taking this approach one step further you may just add a semaphore into struct dentry and take it instead of directory ->i_sem. I think Alexander Viro tried something similar in the past, but it slowed down common case when there is no concurrent access to the directory.
Quoted 1 line
Nikita.
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Nikita Danilov (ND) writes:
ND> By taking this approach one step further you may just add a semaphore ND> into struct dentry and take it instead of directory ->i_sem. I think ND> Alexander Viro tried something similar in the past, but it slowed down ND> common case when there is no concurrent access to the directory.
it's not semaphore. locks are created on demand to protect part of directory. we can't have a lots of semaphores allocated statically for each part of each directory.
BTW, all of this have to be enabled by 'pdirops' ext3's mount option. so, things don't slow down w/o this option.
Quoted 8 lines · [email protected] writes:
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Nikita Danilov (ND) writes:
ND> By taking this approach one step further you may just add a semaphore ND> into struct dentry and take it instead of directory ->i_sem. I think ND> Alexander Viro tried something similar in the past, but it slowed down ND> common case when there is no concurrent access to the directory.
Quoted 6 lines
it's not semaphore. locks are created on demand to protect part of directory. we can't have a lots of semaphores allocated statically for each part of each directory.
BTW, all of this have to be enabled by 'pdirops' ext3's mount option. so, things don't slow down w/o this option.
I am talking about "dynamic locks" taken in fs/namei.c. Have you measured how your patch affects single-threaded case?
Quoted 1 line
Nikita.
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Nikita Danilov (ND) writes:
ND> I am talking about "dynamic locks" taken in fs/namei.c. Have you ND> measured how your patch affects single-threaded case?
yep.
creation of 250K hardlinks by single process:
before: 0m16.913s after: 0m17.423s
of course, it costs, but I believe some optimization are still possible (proposed by Andi, at least)
thanks for review!
Quoted 8 lines · Andi Kleen
Quoted 3 lines · On Tue, 08 Jul 2003 15:28:27 +0000 [email protected] wrote:
dynlocks implements 'lock namespace', so you can lock A for namepace N1 and lock B for namespace N1 and so on. we need this because we want to take lock on _part_ of directory.
Ok, a mini database lock manager. Wouldn't it be better to use a small hash table and lock escalation on overflow for this? Otherwise you could have quite a lot of entries queued up in the list if the server is slow.
That was my initial thought also, but the number of locks that are in existence at one time are very small (i.e. number of threads active in a directory at one time). Having a "more scalable" locking setup will, I think, hurt performance for the common case.
Cheers, Andreas
--
Andreas Dilger
http://sourceforge.net/projects/ext2resize/
http://www-mddsp.enel.ucalgary.ca/People/adilger/
Quoted 6 lines · On Tue, 8 Jul 2003 10:22:25 -0700 Andreas Dilger <[email protected]> wrote:
Quoted 5 lines · Andi Kleen
Quoted 3 lines · On Tue, 08 Jul 2003 15:28:27 +0000 [email protected] wrote:
dynlocks implements 'lock namespace', so you can lock A for namepace N1 and lock B for namespace N1 and so on. we need this because we want to take lock on _part_ of directory.
Quoted 8 lines
Quoted 3 lines
Ok, a mini database lock manager. Wouldn't it be better to use a small hash table and lock escalation on overflow for this? Otherwise you could have quite a lot of entries queued up in the list if the server is slow.
That was my initial thought also, but the number of locks that are in existence at one time are very small (i.e. number of threads active in a directory at one time). Having a "more scalable" locking setup will, I think, hurt performance for the common case.
I don't see how it will be slower than the current patch.
The main advantage of lock escalation is that the worst case is bounded very closely (worst it is just like currently with the single lock)
-Andi