Are clone/checkout operations deterministic?
Hey Git Devs,
I have a bit of an odd question: do git clone/checkout operations have a deterministic ordering?
That is: are files guaranteed to be laid down onto disk in any specific (and deterministic) order as a clone and/or checkout operations occurs? (And if so, is it alphabetical order? Depth-first? Something else?)
In case the answer is different (and I'd guess that it might be?), I'm mostly interested in the initial clone case... but it would be great to know if, indeed, the answer is different for just-checkouts too.
I did some cursory googling, but nothing hopped out at me as an answer to this question.
Thanks!
-preed
Signature · 4 lines
J. Paul Reed PGP: 0xDF8708F8 ========================================================================= I've never seen an airplane yet that can read the type ratings on your pilot's license. -- Chuck Boedecker
Quoted 9 lines · J. Paul Reed
Hey Git Devs,
I have a bit of an odd question: do git clone/checkout operations have a deterministic ordering?
That is: are files guaranteed to be laid down onto disk in any specific (and deterministic) order as a clone and/or checkout operations occurs? (And if so, is it alphabetical order? Depth-first? Something else?)
Quoted 6 lines
In case the answer is different (and I'd guess that it might be?), I'm mostly interested in the initial clone case... but it would be great to know if, indeed, the answer is different for just-checkouts too.
I did some cursory googling, but nothing hopped out at me as an answer to this question.
In practice I think clone, checkout, reset etc. always work in the same order you see with git ls-tree -r --name-only HEAD, but as far as I know this has never been guaranteed or documented, and shouldn't be relied on.
E.g. there's probably cases where writing files in parallel is going to be faster than writing them sequentially. We don't have such a mode just because nobody's written a patch for it, but having that patch would break any assumptions of our current order.
Quoted 17 lines · Ævar Arnfjörð Bjarmason
Quoted 13 lines · J. Paul Reed
Hey Git Devs,
I have a bit of an odd question: do git clone/checkout operations have a deterministic ordering?
That is: are files guaranteed to be laid down onto disk in any specific (and deterministic) order as a clone and/or checkout operations occurs? (And if so, is it alphabetical order? Depth-first? Something else?)
In case the answer is different (and I'd guess that it might be?), I'm mostly interested in the initial clone case... but it would be great to know if, indeed, the answer is different for just-checkouts too.
Quoted 7 lines
Quoted 2 lines
I did some cursory googling, but nothing hopped out at me as an answer to this question.
In practice I think clone, checkout, reset etc. always work in the same order you see with
git ls-tree -r --name-only HEAD, but as far as I know this has never been guaranteed or documented, and shouldn't be relied on.
The transmission of packfiles is non-deterministic, as the packfiles (which are packed for each clone separately when using core git as a server) are not packed in a deterministic fashion, but in a threaded environment which allows different packing orders.
If you clone from a server that gives you exactly the same pack at all times (assuming the remote repo doesn't change refs), then checkout is currently deterministic in unpacking files to the working tree.
Quoted 5 lines
E.g. there's probably cases where writing files in parallel is going to be faster than writing them sequentially. We don't have such a mode just because nobody's written a patch for it, but having that patch would break any assumptions of our current order.
+cc Ben who is looking into that, but hasn't spoken up on the mailing list yet.
Quoted 8 lines · Ævar Arnfjörð Bjarmason
In practice I think clone, checkout, reset etc. always work in the same order you see with
git ls-tree -r --name-only HEAD, but as far as I know this has never been guaranteed or documented, and shouldn't be relied on.E.g. there's probably cases where writing files in parallel is going to be faster than writing them sequentially. We don't have such a mode just because nobody's written a patch for it, but having that patch would
Well I did have some patches [1] but as usually I did not follow through. Interestingly there's actually concern about timestamp order
[2] even back then
[1] https://public-inbox.org/git/20160415095139.GA3985@lanh/
[2] https://public-inbox.org/git/CAP8UFD0WZHriY340eh3K6ygzb0tXnoT+XaY8+c2k+N2x9UBYxA@mail.gmail.com/
Quoted 1 line
break any assumptions of our current order.
Signature · 1 line
Duy
Quoted 4 lines · Ævar Arnfjörð Bjarmason
In practice I think clone, checkout, reset etc. always work in the same order you see with
git ls-tree -r --name-only HEAD, but as far as I know this has never been guaranteed or documented, and shouldn't be relied on.
I think this paragraph is correct in general (and I agree with the sentiment that this is subject to change in future versions).
There is one concrete case I know that has non-deterministic order in current versions: long-lived clean/smudge filters can defer their response. The LFS filter uses this to tell Git "no, I'm still downloading the content", at which point Git will proceed with checking out other local files (or even other LFS files that happen to arrive sooner).
Depending on what one wants to do with the determinism, it may be OK to ignore that case. ;)
-Peff