Core and Not-So Core
I have been experimenting with pure-Java implementation of GIT concepts with a goal of eventually providing plugins to Eclipse to allow the Eclipse GUI to interact with GIT repositories.
One thing I noticed when doing this is that the present index/cache structure is rather arbitrary and the optimum index structure is determined by the structure of the tools that use a GIT repository rather than the structure of the GIT repository itself.
To give a concrete example: the cache currently contains most of the posix stat structure primarily to allow quick change detection. In the Java world, most of the posix stat structure is not directly accessible via the pure-Java file system abstractions. However, for most purposes detecting changes to files modification time and file size would be enough. Given this is the case, a Java-GIT client doesn't need to bother getting access to a posix stat structure and could therefore get away with a simpler index structure, provided it doesn't need to interoperate with a 'C'-GIT client that shared the same workspace. A Java-GIT client might also choose to represent an index cache as a complex serialized Java object graph or (perhaps) an XML document.
Another example: I can imagine a variant of the index file structure that recorded all the parents which have been merged into the cache and automatically include this information when performing the commit.
The point is that many different index file structures are possible and will be determined in part by the tooling created in the porcelain layer - there really is no one true index file format as there is a one true repository format. Different tools can use different index file formats and still interoperate at the repository level because only the repository format needs to have a solid, unchanging definition.
Currently the GIT stack is structured as follows:
cogito
git-core
I think it would be worthwhile if care was taken to draw a distinction between the repository and the cache aspects of the git core, perhaps even going to the extreme of moving all knowledge of the cache into cogito itself. By clearly drawing this distinction, we will more easily enable the creation of different kind of tools sets atop the foundation of the GIT repository format.
e.g., either:
cogito
git-cache
git-respository
or:
cogito-tools
cogito-cache
git-repository
Anyway, I offer this as food for thought - chew or flame away as appropriate!
jon.
Signature · 2 lines
homepage: http://www.zeta.org.au/~jon/ blog: http://orwelliantremors.blogspot.com/
Quoted 3 lines · Jon Seymour
I have been experimenting with pure-Java implementation of GIT concepts with a goal of eventually providing plugins to Eclipse to allow the Eclipse GUI to interact with GIT repositories.
It's not April 1st. Why would you want to reimplement it in Java instead of just using the existing implementation? Is this a religious issue?
Signature · 1 line
dwmw2
He want's to make an eclipce plugin...
This must be done in Java.
Quoted 15 lines · Jon Seymour
Quoted 3 lines
I have been experimenting with pure-Java implementation of GIT concepts with a goal of eventually providing plugins to Eclipse to allow the Eclipse GUI to interact with GIT repositories.
It's not April 1st. Why would you want to reimplement it in Java instead of just using the existing implementation? Is this a religious issue?
--
dwmw2-
To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Signature · 5 lines
---------------------------- Eduardo Teixeira Dias Tendencies Consultoria Tel/Fax: +55 11 3828-1281 http://www.tendencies.com.br
Quoted 1 line · Eduardo Teixeira Dias
He want's to make an eclipce plugin...
And there's no way for him to call execve()?
Hell, even the lisp nutters manage to make emacs invoke external programs occasionally. What makes Java programmers so much less sane?
Signature · 1 line
dwmw2
Quoted 11 lines · Eduardo Teixeira Dias
Quoted 1 line
He want's to make an eclipce plugin...
And there's no way for him to call
execve()?Hell, even the lisp nutters manage to make emacs invoke external programs occasionally. What makes Java programmers so much less sane?
--
dwmw2
Good point!
It can be done using execxxx().
But a Java version is better for this particular use (Eclipce Plugin):
- Just a .jar download
- Installation without external dependencies
Quoted 2 lines · Eduardo Teixeira Dias
- Just a .jar download
- Installation without external dependencies
Someone who is going to hack the kernel can very well install more things. And anyway, git is "the linux SCM tool" so all distros will package it. Also, people who hacks the linux kernel usually runs it, so "git is not ported to win32" is not a big problem.
...and I don't think people who use eclipse wants to have the fastest tool on earth, so "java-invoking-C is slower than pure java" is not a great excuse either...
Code reuse == good. Java programmers should know that. Anyway, people is free to do whatever they want with their time :)
I'm a Java programmer and I was thinking like one (Culture thing).
The reuse argument is very Strong.
The git-core are ready and it's a very good idea to reuse it. Any change in the core or in the internal representation of git repository will benefit the Git-Java implementation...
I think that a growing utilization of GCJ and ClassPath will contribute to a Gradual Culture Change in the Java Land and promote a beter reutilization among all GCC languages...
Write Once Run Everywhere that GCJ Runs (Sounds Good).
Eclipse will be compiled with GCJ in Fedora 4. Let's see how it works...
Cheers!
Quoted 18 lines
Quoted 2 lines · Eduardo Teixeira Dias
- Just a .jar download
- Installation without external dependenciesSomeone who is going to hack the kernel can very well install more things. And anyway, git is "the linux SCM tool" so all distros will package it. Also, people who hacks the linux kernel usually runs it, so "git is not ported to win32" is not a big problem.
...and I don't think people who use eclipse wants to have the fastest tool on earth, so "java-invoking-C is slower than pure java" is not a great excuse either...
Code reuse == good. Java programmers should know that. Anyway, people is free to do whatever they want with their time :)
Dear diary, on Tue, May 10, 2005 at 11:45:01PM CEST, I got a letter where Diego Calleja <[email protected]> told me that...
Quoted 4 lines
Someone who is going to hack the kernel can very well install more things. And anyway, git is "the linux SCM tool" so all distros will package it. Also, people who hacks the linux kernel usually runs it, so "git is not ported to win32" is not a big problem.
It's not like everything git is ever going to be used for is kernel and only the kernel.
Signature · 3 lines
Petr "Pasky" Baudis Stuff: http://pasky.or.cz/ C++: an octopus made by nailing extra legs onto a dog. -- Steve Taylor
Quoted 9 lines · Petr Baudis
Dear diary, on Tue, May 10, 2005 at 11:45:01PM CEST, I got a letter where Diego Calleja <[email protected]> told me that...
Quoted 4 lines
Someone who is going to hack the kernel can very well install more things. And anyway, git is "the linux SCM tool" so all distros will package it. Also, people who hacks the linux kernel usually runs it, so "git is not ported to win32" is not a big problem.
It's not like everything git is ever going to be used for is kernel and only the kernel.
So does this mean I can keep working on my wxWidgets implementation? :)
Signature · 2 lines
James Purser http://ksit.dynalias.com
Btw, I rewrote about 90% of git in python, just out of curiousity. Performance is pretty competitive, and I can optimize certain operations by keeping the index in memory temporarily. It also saves a lot of output parsing, simplifies error-handling (if the frontend is written in python too, which bit now is), and the best of all: its about 600 lines of python code. I am all for a Java implementation. After its done we will see whether it works and how well it works. We can still throw it away if it sucks.
Andreas
Quoted 19 lines · Petr Baudis
Dear diary, on Tue, May 10, 2005 at 11:45:01PM CEST, I got a letter where Diego Calleja <[email protected]> told me that...
Quoted 4 lines
Someone who is going to hack the kernel can very well install more things. And anyway, git is "the linux SCM tool" so all distros will package it. Also, people who hacks the linux kernel usually runs it, so "git is not ported to win32" is not a big problem.
It's not like everything git is ever going to be used for is kernel and only the kernel.
Signature · 7 lines
Petr "Pasky" Baudis Stuff: http://pasky.or.cz/ C++: an octopus made by nailing extra legs onto a dog. -- Steve Taylor - To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html
I'm a Java programmer and I was thinking like one (Culture thing).
The reuse argument is very Strong.
The git-core are ready and it's a very good idea to reuse it. Any change in the core or in the internal representation of git repository will benefit the Git-Java implementation...
I think that a growing utilization of GCJ and ClassPath will contribute to a Gradual Culture Change in the Java Land and promote a beter reutilization among all GCC languages...
Write Once Run Everywhere that GCJ Runs (Sounds Good).
Eclipse will be compiled with GCJ in Fedora 4. Let's see how it works...
Cheers!
Quoted 18 lines
Quoted 2 lines · Eduardo Teixeira Dias
- Just a .jar download
- Installation without external dependenciesSomeone who is going to hack the kernel can very well install more things. And anyway, git is "the linux SCM tool" so all distros will package it. Also, people who hacks the linux kernel usually runs it, so "git is not ported to win32" is not a big problem.
...and I don't think people who use eclipse wants to have the fastest tool on earth, so "java-invoking-C is slower than pure java" is not a great excuse either...
Code reuse == good. Java programmers should know that. Anyway, people is free to do whatever they want with their time :)
[somewhat OT]
Quoted 1 line · Eduardo Teixeira Dias
Write Once Run Everywhere that GCJ Runs (Sounds Good).
GCJ in fact supports more architectures than Sun's java VM so with GCJ you _really_ can run your java code everywhere.
Quoted 6 lines · Diego Calleja
Quoted 1 line · Eduardo Teixeira Dias
Write Once Run Everywhere that GCJ Runs (Sounds Good).
GCJ in fact supports more architectures than Sun's java VM so with GCJ you _really_ can run your java code everywhere.
But then again, a half decent C implementation is even more ubicuous, so...
Signature · 4 lines
Dr. Horst H. von Brand User #22616 counter.li.org Departamento de Informatica Fono: +56 32 654431 Universidad Tecnica Federico Santa Maria +56 32 654239 Casilla 110-V, Valparaiso, Chile Fax: +56 32 797513
I'm a Java programmer and I was thinking like one (Culture thing).
The reuse argument is very Strong.
The git-core are ready and it's a very good idea to reuse it. Any change in the core or in the internal representation of git repository will benefit the Git-Java implementation...
I think that a growing utilization of GCJ and ClassPath will contribute to a Gradual Culture Change in the Java Land and promote a beter reutilization among all GCC languages...
Write Once Run Everywhere that GCJ Runs (Sounds Good).
Eclipse will be compiled with GCJ in Fedora 4. Let's see how it works...
Cheers!
Quoted 18 lines
Quoted 2 lines · Eduardo Teixeira Dias
- Just a .jar download
- Installation without external dependenciesSomeone who is going to hack the kernel can very well install more things. And anyway, git is "the linux SCM tool" so all distros will package it. Also, people who hacks the linux kernel usually runs it, so "git is not ported to win32" is not a big problem.
...and I don't think people who use eclipse wants to have the fastest tool on earth, so "java-invoking-C is slower than pure java" is not a great excuse either...
Code reuse == good. Java programmers should know that. Anyway, people is free to do whatever they want with their time :)
Quoted 3 lines · Diego Calleja
...and I don't think people who use eclipse wants to have the fastest tool on earth, so "java-invoking-C is slower than pure java" is not a great excuse either...
Note that it's also not true. At least if you use the gcj-specific CNI mechanisms instead of Sun's braindead JNI.
Quoted 9 lines · Christoph Hellwig
Quoted 3 lines · Diego Calleja
...and I don't think people who use eclipse wants to have the fastest tool on earth, so "java-invoking-C is slower than pure java" is not a great excuse either...
Note that it's also not true. At least if you use the gcj-specific CNI mechanisms instead of Sun's braindead JNI.
What I actually claimed (contrary to Diego's misquote) is that "Java-invokes-C-executable" would be slower than pure-Java. I didn't make a claim one way or the other about the performance of a Java implementation invoking some form of native interface.
jon.
Signature · 2 lines
homepage: http://www.zeta.org.au/~jon/ blog: http://orwelliantremors.blogspot.com/
Quoted 8 lines · David Woodhouse
Quoted 3 lines · Jon Seymour
I have been experimenting with pure-Java implementation of GIT concepts with a goal of eventually providing plugins to Eclipse to allow the Eclipse GUI to interact with GIT repositories.
It's not April 1st. Why would you want to reimplement it in Java instead of just using the existing implementation? Is this a religious issue?
Not really - in the Java world, things are simpler if you don't have to carry around a JNI library - that way, you can just run it wherever a Java interpreter exists and not worry about
having someone recompiling the JNI part.
A pure-Java implementation will perform better than a Java-invokes-C-executable approach [ though not, of course, a C-only approach ].
Another benefit of playing around with a Java abstraction is that I can more easily experiment with abstractions than I can in C since the abstraction-facilities of Java more directly support such playing than does C. So, for example, I can easily create a virtual repository which layers a local repository over a remote repository and transparently populates the local repository from the remote repository "on-demand". Of course, this sort of thing can be done in C, but it requires much more "work" to set up the abstractions.
You could perhaps argue that the existing 'C' tools in some way encapsulate access to the repository format. Maybe, but the fact is the rapid adoption of GIT has already effectively fixed the GIT repository format in practice so any change in repository format will require considerable planning anyway - planning that will allow sufficient time for implementations in other languages to catch up.
So, no, it's not a religious issue. If anything, it is being dogmatic to insist that the sacred GIT repository structure only be manipulated by 'C' tools blessed by the hands of Linus.
The concepts in GIT are bigger than the programming language its toolsets are implemented in.
jon.
Signature · 2 lines
homepage: http://www.zeta.org.au/~jon/ blog: http://orwelliantremors.blogspot.com/
Quoted 3 lines · Jon Seymour
So, no, it's not a religious issue. If anything, it is being dogmatic to insist that the sacred GIT repository structure only be manipulated by 'C' tools blessed by the hands of Linus.
Given the volatility of the structure -- at least the details if not the fundamentals -- it seems bizarre to want to reimplement it rather than just using the existing tools.
This is the same mentality which gives Eclipse a half-arsed SSH reimplementation which doesn't behave like normal SSH is configured to behave either, right?
Signature · 1 line
dwmw2
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 8 lines · David Woodhouse
Quoted 3 lines · Jon Seymour
So, no, it's not a religious issue. If anything, it is being dogmatic to insist that the sacred GIT repository structure only be manipulated by 'C' tools blessed by the hands of Linus.
Given the volatility of the structure -- at least the details if not the fundamentals -- it seems bizarre to want to reimplement it rather than just using the existing tools.
I did consider wrapping it - I really did. But after thinking about it for a couple of weeks
I eventually came to the conclusion it would be a sub-optimal solution.
And I don't agree that the structure is volatile. The actual structure of GIT repository has been rock-solid since Linus reversed the compression-signature order. Yes, there has been the excellent delta work that Nicholas has been working on, but a Java version doesn't need to use that anymore than Linus himself does [ and last time I checked (the mailing list), Linus was being rather conservative about that ].
What _has_ been changing at a great rate recently is the behaviour of the tooling layer.
So, if I want a stable foundation to build my stuff on, basing it on the output of the C tools would be a huge mistake. No, I think it would be far safer for me to build my tooling using assumptions that have proven to be rock-solid over the last few weeks - the structure of the GIT repository format itself.
Quoted 5 lines
This is the same mentality which gives Eclipse a half-arsed SSH reimplementation which doesn't behave like normal SSH is configured to behave either, right?
David, I have nothing whatsoever to do with the Eclipse implementation of SSH, so what exactly is your point? All Java programmers are fundamentally brain-damaged? Grow up, please.
jon.
Signature · 2 lines
homepage: http://www.zeta.org.au/~jon/ blog: http://orwelliantremors.blogspot.com/
Quoted 2 lines · Jon Seymour
David, I have nothing whatsoever to do with the Eclipse implementation of SSH, so what exactly is your point?
That all programmers of _any_ persuasion who want to reimplement the world in their own language instead of using existing tools are fundamentally brain-damaged.
I'd say the same if the KDevelop people wanted to rewrite git and ssh in C++, too.
I don't think that's such a childish observation, surely? Piecing together tools which do one thing, and do it well, is part of the UNIX philosophy.
Signature · 1 line
dwmw2
Quoted 8 lines · David Woodhouse
Quoted 2 lines · Jon Seymour
David, I have nothing whatsoever to do with the Eclipse implementation of SSH, so what exactly is your point?
That all programmers of _any_ persuasion who want to reimplement the world in their own language instead of using existing tools are fundamentally brain-damaged.
Quoted 6 lines
I'd say the same if the KDevelop people wanted to rewrite git and ssh in C++, too.
I don't think that's such a childish observation, surely? Piecing together tools which do one thing, and do it well, is part of the UNIX philosophy.
This might surprise you, but everything in the world is not a unix command line. Using tools designed for use on a unix command line in a highly interactive environment like Eclipse would produce an abominably clunky result.
Presumably you think this is the Unix way.
I have already explained all the pragmatic reasons for doing a GIT implementation in Java but you are prepared to ignore all of those reasons. You have ignored all these reasons rather than lift a finger to compose a well-reasoned rebuttal.
Instead you cling to a dogmatic insistence that GIT respositories only be accessed via the C toolsets.
David, you are being extremely childish and extremely dogmatic. Why bother?
jon
Signature · 2 lines
homepage: http://www.zeta.org.au/~jon/ blog: http://orwelliantremors.blogspot.com/
Quoted 4 lines · Jon Seymour
This might surprise you, but everything in the world is not a unix command line. Using tools designed for use on a unix command line in a highly interactive environment like Eclipse would produce an abominably clunky result.
The unix commandline is an highly interactive enviroment aswell, and supports scripting in addition :)
Quoted 4 lines
I have already explained all the pragmatic reasons for doing a GIT implementation in Java but you are prepared to ignore all of those reasons. You have ignored all these reasons rather than lift a finger to compose a well-reasoned rebuttal.
You tried to argue for re-inventing the wheel. Fortunately you are allowed to reinvent the wheel here (which isn't given anymore these days). Just don't expect any support from people who have been burnt by that before. And the Java world is re-inventing the wheel far to often - I suspect that'll be cured when the community gets more mature in a few years..
Quoted 2 lines
Instead you cling to a dogmatic insistence that GIT respositories only be accessed via the C toolsets.
The C toolchain is a well-defined interface. One that makes a lot of sense.
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 6 lines · Christoph Hellwig
Quoted 4 lines · Jon Seymour
I have already explained all the pragmatic reasons for doing a GIT implementation in Java but you are prepared to ignore all of those reasons. You have ignored all these reasons rather than lift a finger to compose a well-reasoned rebuttal.
Quoted 7 lines
You tried to argue for re-inventing the wheel. Fortunately you are allowed to reinvent the wheel here (which isn't given anymore these days). Just don't expect any support from people who have been burnt by that before. And the Java world is re-inventing the wheel far to often - I suspect that'll be cured when the community gets more mature in a few years..
I am _not_ re-inventing any wheel. Merely building a road through different territory upon which the GIT wheel may roll more freely.
Don't get me wrong. I appreciate the UNIX mindset of building small tools and composing them freely. But the UNIX way isn't the only way to interact with the world and it is being hopelessly dogmatic to insist otherwise.
jon.
--
homepage: http://www.zeta.org.au/~jon/
blog: http://orwelliantremors.blogspot.com/
Quoted 15 lines · Jon Seymour
Quoted 6 lines · Christoph Hellwig
Quoted 4 lines · Jon Seymour
I have already explained all the pragmatic reasons for doing a GIT implementation in Java but you are prepared to ignore all of those reasons. You have ignored all these reasons rather than lift a finger to compose a well-reasoned rebuttal.
Quoted 7 lines
You tried to argue for re-inventing the wheel. Fortunately you are allowed to reinvent the wheel here (which isn't given anymore these days). Just don't expect any support from people who have been burnt by that before. And the Java world is re-inventing the wheel far to often - I suspect that'll be cured when the community gets more mature in a few years..
Quoted 2 lines
I am _not_ re-inventing any wheel. Merely building a road through different territory upon which the GIT wheel may roll more freely.
Would you mind fixing your mailer before, so that your messages won't appear as new Subject: every time?
- Davide
Quoted 6 lines · Jon Seymour
I did consider wrapping it - I really did. But after thinking about it for a couple of weeks I eventually came to the conclusion it would be a sub-optimal solution.
So, if I want a stable foundation to build my stuff on, basing it on the output of the C tools would be a huge mistake.
Can Java use a library that's implemented in C, like Python and Perl can? If that is the case, the C implementation of git simply needs the ability to be called as a library and you can implement Java bindings for it.
That should also make it easier to create front-ends in other languages...
Signature · 3 lines
"Debugging is twice as hard as writing the code in the first place. Therefore, if you write the code as cleverly as possible, you are, by definition, not smart enough to debug it." - Brian W. Kernighan
Quoted 6 lines · Rik van Riel
Quoted 3 lines · Jon Seymour
I did consider wrapping it - I really did. But after thinking about it for a couple of weeks I eventually came to the conclusion it would be a sub-optimal solution.
Quoted 8 lines
Quoted 2 lines
So, if I want a stable foundation to build my stuff on, basing it on the output of the C tools would be a huge mistake.
Can Java use a library that's implemented in C, like Python and Perl can? If that is the case, the C implementation of git simply needs the ability to be called as a library and you can implement Java bindings for it.
It can in principle, yes. This is the so-called Java-Native-Interface (JNI).
I think in the longer term it would make sense to write a JNI layer but the GIT source code is probably a little too unstable for that now. What needs to happen first, I think, is that a solid and stable C API (e.g. a libgit) be proposed and developed.
One outcome of the work I am doing (a long time away, probably will never happen) might eventually be to propose an C API for GIT based, in part, on the prototyping work I do in Java. However, I suspect its Java-tainted heritage would mean instant death to it for purely dogmatic reasons so I think it I'll leave it to others to propose a C-API and then write a JNI wrapper for that when/if it ever happens.
Quoted 3 lines
That should also make it easier to create front-ends in other languages...
Agreed.
jon.
Dear diary, on Wed, May 11, 2005 at 04:09:49AM CEST, I got a letter where Jon Seymour <[email protected]> told me that...
Quoted 16 lines · Rik van Riel
Quoted 6 lines · Jon Seymour
Quoted 4 lines
I did consider wrapping it - I really did. But after thinking about it for a couple of weeks I eventually came to the conclusion it would be a sub-optimal solution.
Quoted 8 lines
Quoted 2 lines
So, if I want a stable foundation to build my stuff on, basing it on the output of the C tools would be a huge mistake.
Can Java use a library that's implemented in C, like Python and Perl can? If that is the case, the C implementation of git simply needs the ability to be called as a library and you can implement Java bindings for it.
Quoted 6 lines
It can in principle, yes. This is the so-called Java-Native-Interface (JNI).
I think in the longer term it would make sense to write a JNI layer but the GIT source code is probably a little too unstable for that now. What needs to happen first, I think, is that a solid and stable C API (e.g. a libgit) be proposed and developed.
FWIW, this is actually happening now (see Brad Roberts' posts in nearby thread).
Signature · 3 lines
Petr "Pasky" Baudis Stuff: http://pasky.or.cz/ C++: an octopus made by nailing extra legs onto a dog. -- Steve Taylor
Dear diary, on Tue, May 10, 2005 at 05:00:33PM CEST, I got a letter where Jon Seymour <[email protected]> told me that...
Quoted 4 lines
One thing I noticed when doing this is that the present index/cache structure is rather arbitrary and the optimum index structure is determined by the structure of the tools that use a GIT repository rather than the structure of the GIT repository itself.
Yes. And that's how it should be - the directory cache is just that - a _cache_. It does not hold any permanent information, merely serves to record git tools' working state relative to the given working tree. So unlike the objects database which has well-defined format and is supposed to be "public", you should view the directory cache as internal git tools' structure. If you want to mess with it too, either use the proper level of abstraction and call the git tools, or don't mess with it at all. And you need to care about it only if you want the git tools working on the same tree properly too - so in that case use the git tools too.
From your arguments, it's not clear to me what really is the big
problem with the git tools. They are _designed_ for automatic use instead of human interaction - you can perceive them just as methods with funny (but actually friendly to your programs) calling convention.
Kind regards,
Signature · 3 lines
Petr "Pasky" Baudis Stuff: http://pasky.or.cz/ C++: an octopus made by nailing extra legs onto a dog. -- Steve Taylor
Quoted 5 lines · Petr Baudis
Dear diary, on Tue, May 10, 2005 at 05:00:33PM CEST, I got a letter where Jon Seymour <[email protected]> told me that...
Yes. And that's how it should be - the directory cache is just that - a _cache_.
No argument there.
Quoted 7 lines
So unlike the objects database which has well-defined format and is
supposed to be "public", you should view the directory cache as internal git tools' structure. If you want to mess with it too, either use the proper level of abstraction and call the git tools, or don't mess with it at all. And you need to care about it only if you want the git tools working on the same tree properly too - so in that case use the git tools too.
I agree in principle, though I'd like users to be able to easily switch between the Eclipse and git tools view of the workspace if they want to - who am I to say how a user should work? Eclipse does this kind of thing quite well with CVS precisely because it shares the workspace structures with the CVS command line tools rather than "re-inventing" the wheel. Yes, separation of concerns has been lost (two implementations of a CVS client), but the big win is that the tools behave like the user wants them to behave.
The trickiest case here is when the user switches between toolsets mid-merge. I guess what I can do is this:
user switch from eclipse -> to git-tools:
1. blow away existing git tools index
2. use git-read-tree to repeat the merge executed in eclipse (my
workspace will track parents)
3. use git-update-cache --add/--remove to reflect merge actions
that have occurred since the workspace deviated from the HEAD.
Alternatively, I can just make Eclipse reflect cache changing actions out onto the git-tools, via an exec of those tools, as and when they occur.
Making use of the git tools index going the other way isn't so easy to achieve because the git tools workspace as it stands doesn't track the merges that have occurred (i.e. which parents were used to form the current cache). However, that's not necessarily a big problem. I just rebuild my "cache" from scratch based on the merges I know about and treat every other difference from the HEAD as a user edit to the workspace.
Quoted 6 lines
From your arguments, it's not clear to me what really is the big problem with the git tools. They are _designed_ for automatic use instead of human interaction - you can perceive them just as methods with funny (but actually friendly to your programs) calling convention.
I am not really arguing that there is a big problem with the existing git tools. However, what I am arguing is that the existing workspace tools are just one way to manage the workspace (Eclipse might be another, as an example) and it would be helpful to keep this in mind, particularly when/if libification ever happens.
jon.
Quoted 21 lines · Jon Seymour
Quoted 5 lines · Petr Baudis
Dear diary, on Tue, May 10, 2005 at 05:00:33PM CEST, I got a letter where Jon Seymour <[email protected]> told me that...
Yes. And that's how it should be - the directory cache is just that - a _cache_.
No argument there.
Quoted 7 lines
So unlike the objects database which has well-defined format and is supposed to be "public", you should view the directory cache as internal git tools' structure. If you want to mess with it too, either use the proper level of abstraction and call the git tools, or don't mess with it at all. And you need to care about it only if you want the git tools working on the same tree properly too - so in that case use the git tools too.
Quoted 6 lines
I agree in principle, though I'd like users to be able to easily switch between the Eclipse and git tools view of the workspace if they want to - who am I to say how a user should work? Eclipse does this kind of thing quite well with CVS precisely because it shares the workspace structures with the CVS command line tools rather than "re-inventing" the wheel.
I disagree here. My experience is that Eclipse does not work well with command line use of CVS within one of its workspaces. This is because Eclipse keeps too much "state" information internally instead of relying on CVS for all information about the state of the CVS playground. This is one of the reasons that you can't attach Eclipse to an already checked out CVS playground. The problem should be easily avoidable.
In spite of the above, Eclipse's SCM interface is the best that I've stumbled across and is the main reason that I use Eclipse. I particularly like the ability to compare a file in the workspace with those in different versions, branches, etc.
So I'm looking forward to a git plug-in.
Peter
Signature · 4 lines
Peter Williams [email protected] "Learning, n. The kind of ignorance distinguishing the studious." -- Ambrose Bierce
Quoted 9 lines · Jon Seymour
Quoted 7 lines · Petr Baudis
So unlike the objects database which has well-defined format and is supposed to be "public", you should view the directory cache as internal git tools' structure. If you want to mess with it too, either use the proper level of abstraction and call the git tools, or don't mess with it at all. And you need to care about it only if you want the git tools working on the same tree properly too - so in that case use the git tools too.
Quoted 6 lines
I agree in principle, though I'd like users to be able to easily switch between the Eclipse and git tools view of the workspace if they want to - who am I to say how a user should work? Eclipse does this kind of thing quite well with CVS precisely because it shares the workspace structures with the CVS command line tools rather than "re-inventing" the wheel.
In this case you just need to manage to use the current index format somehow.
Honestly I don't think Linus might be interested into changing the index format just to make a Java implementation happier. So your best bet is probably to use it as is or ignore it entirely.
Nicolas
Quoted 8 lines · Nicolas Pitre
Quoted 6 lines · Jon Seymour
I agree in principle, though I'd like users to be able to easily switch between the Eclipse and git tools view of the workspace if they want to - who am I to say how a user should work? Eclipse does this kind of thing quite well with CVS precisely because it shares the workspace structures with the CVS command line tools rather than "re-inventing" the wheel.
Quoted 7 lines
In this case you just need to manage to use the current index format somehow.
Honestly I don't think Linus might be interested into changing the index format just to make a Java implementation happier. So your best bet is probably to use it as is or ignore it entirely.
Agreed - I am not proposing a change to the index format, merely highlighting that the index format is closely bound to the workspace-oriented toolset and that the workspace-oriented toolset is distinct from the repository-oriented toolset; different workspace-oriented toolsets will have different requirements for their index files and so their index files might quite rightly differ in both form and substance.
That said, their is no harm making my set of tools interoperable with the git tools, where that is possible so that the user can decide to use which ever toolset fits the job immediately at hand. With this in mind, a published specification of the index format would help, but I am not going insist on that. I am certainly not going to ask for changes to it! Life is too short :-)
jon.
Quoted 8 lines · Jon Seymour
I have been experimenting with pure-Java implementation of GIT concepts with a goal of eventually providing plugins to Eclipse to allow the Eclipse GUI to interact with GIT repositories.
One thing I noticed when doing this is that the present index/cache structure is rather arbitrary and the optimum index structure is determined by the structure of the tools that use a GIT repository rather than the structure of the GIT repository itself.
I'd guess that an Eclipse plugin wouldn't have a on-disk cache at all, since it expects to keep the same address space through a whole day's work.
Quoted 4 lines
To give a concrete example: the cache currently contains most of the posix stat structure primarily to allow quick change detection. In the Java world, most of the posix stat structure is not directly accessible via the pure-Java file system abstractions. However, for
Quoted 8 lines
most purposes detecting changes to files modification time and file size would be enough. Given this is the case, a Java-GIT client doesn't need to bother getting access to a posix stat structure and could therefore get away with a simpler index structure, provided it doesn't need to interoperate with a 'C'-GIT client that shared the same workspace. A Java-GIT client might also choose to represent an index cache as a complex serialized Java object graph or (perhaps) an XML document.
The two clients should be able to interoperate trivially; the C version would just treat the Java one as a text editor whose modifications happen to match commits going into the same head; as soon as the C user does either read-tree or update-cache, everything will be seen to be consistent again.
Quoted 11 lines
Currently the GIT stack is structured as follows:
cogito
git-coreI think it would be worthwhile if care was taken to draw a distinction between the repository and the cache aspects of the git core, perhaps even going to the extreme of moving all knowledge of the cache into cogito itself. By clearly drawing this distinction, we will more easily enable the creation of different kind of tools sets atop the foundation of the GIT repository format.
I think this is nonsensical. The cache format is tied to the way in which the repository accessing code is written, so a git-core separate from the cache wouldn't have a useful set of code.
It might be worthwhile to produce a separate document describing the repository format, such that it could be accessed by different code, however. Of course, the index file wouldn't be part of this documentation, for the same reason that the public git repositories don't include index files.
-Daniel
*This .sig left intentionally blank*
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 8 lines
Quoted 7 lines
I think it would be worthwhile if care was taken to draw a distinction between the repository and the cache aspects of the git core, perhaps even going to the extreme of moving all knowledge of the cache into cogito itself. By clearly drawing this distinction, we will more easily enable the creation of different kind of tools sets atop the foundation of the GIT repository format.
Quoted 4 lines
I think this is nonsensical. The cache format is tied to the way in which the repository accessing code is written, so a git-core separate from the cache wouldn't have a useful set of code.
I guess I agree it is somewhat nonsensical if one considers the current git toolset as a collection of programs - the exercise now might simply reduce to classifying git tools as index-using/non-index-using and nothing more. However, it might be worth keeping in mind when/if the "libification" of git happens so that there is a clean separation of layers in the API between the repository API and index/cache/workspace API.
Quoted 6 lines
It might be worthwhile to produce a separate document describing the repository format, such that it could be accessed by different code, however. Of course, the index file wouldn't be part of this documentation, for the same reason that the public git repositories don't include index files.
And thanks for your constructive criticism.
jon.
--
homepage: http://www.zeta.org.au/~jon/
blog: http://orwelliantremors.blogspot.com/
Dear diary, on Wed, May 11, 2005 at 01:05:55AM CEST, I got a letter where Jon Seymour <[email protected]> told me that...
Quoted 13 lines
Quoted 8 lines
Quoted 7 lines
I think it would be worthwhile if care was taken to draw a distinction between the repository and the cache aspects of the git core, perhaps even going to the extreme of moving all knowledge of the cache into cogito itself. By clearly drawing this distinction, we will more easily enable the creation of different kind of tools sets atop the foundation of the GIT repository format.
Quoted 4 lines
I think this is nonsensical. The cache format is tied to the way in which the repository accessing code is written, so a git-core separate from the cache wouldn't have a useful set of code.
Quoted 7 lines
I guess I agree it is somewhat nonsensical if one considers the current git toolset as a collection of programs - the exercise now might simply reduce to classifying git tools as index-using/non-index-using and nothing more. However, it might be worth keeping in mind when/if the "libification" of git happens so that there is a clean separation of layers in the API between the repository API and index/cache/workspace API.
In that case the repository API's input/output is data structure equivalent to the index, and workspace API's input/output is the index too. That is what it really is - and it is kept on the disk only since the commands are invoked separately so it needs to keep the state around somewhere.
Signature · 3 lines
Petr "Pasky" Baudis Stuff: http://pasky.or.cz/ C++: an octopus made by nailing extra legs onto a dog. -- Steve Taylor
Quoted 11 lines · Petr Baudis
Dear diary, on Wed, May 11, 2005 at 01:05:55AM CEST, I got a letter where Jon Seymour <[email protected]> told me that...
Quoted 8 lines
I guess I agree it is somewhat nonsensical if one considers the current git toolset as a collection of programs - the exercise now might simply reduce to classifying git tools as index-using/non-index-using and nothing more. However, it might be worth keeping in mind when/if the "libification" of git happens so that there is a clean separation of layers in the API between the repository API and index/cache/workspace API.
Quoted 6 lines
In that case the repository API's input/output is data structure equivalent to the index, and workspace API's input/output is the index too. That is what it really is - and it is kept on the disk only since the commands are invoked separately so it needs to keep the state around somewhere.
Not sure I agree...
The repository API would contain functionality equivalent to cat-file, ls-tree, most of fsck-cache, rev-list, rev-tree, diff-tree, most of the transport code - things that don't involve use of the index.
The workspace API would contain read-tree, write-tree, commit-tree, etc - things that do involve use of the the index.
jon.
Quoted 6 lines · Jon Seymour
The repository API would contain functionality equivalent to cat-file, ls-tree, most of fsck-cache, rev-list, rev-tree, diff-tree, most of the transport code - things that don't involve use of the index.
The workspace API would contain read-tree, write-tree, commit-tree, etc - things that do involve use of the the index.
Unfortunately for this idea, you can't actually check files into or out of the repository using the git tools without the index (in memory at least, if not on disk). This is a bit like having a libc with all the system calls except read and write. Sure, there are a number of programs that would be fine that way, but it makes the API unintuitive, and most serious programs need the extension anyway.
-Daniel
*This .sig left intentionally blank*
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 9 lines · Daniel Barkalow
Quoted 6 lines · Jon Seymour
The repository API would contain functionality equivalent to cat-file, ls-tree, most of fsck-cache, rev-list, rev-tree, diff-tree, most of the transport code - things that don't involve use of the index.
The workspace API would contain read-tree, write-tree, commit-tree, etc - things that do involve use of the the index.
Quoted 7 lines
Unfortunately for this idea, you can't actually check files into or out of the repository using the git tools without the index (in memory at least, if not on disk). This is a bit like having a libc with all the system calls except read and write. Sure, there are a number of programs that would be fine that way, but it makes the API unintuitive, and most serious programs need the extension anyway.
Daniel,
Checking in/checking out is definitely a workspace API function that involves an index (almost certainly) an on-disk index, but reading and writing from and to the repository is purely a repository API function.
There is no reason why the workspace API can't make use of "lower-level" functions in the repository API. I guess I would argue that the repository API really shouldn't need much, if any knowledge, of the workspace API.
The repository API knows about the on-disk structure of repository objects - it knows how to write writing compressed, SHA1 signed objects, it also knows how to pack and unpack the various object types from their on-disk representation into structures manipulated by the workspace API.
On the otherhand the workspace API knows about managing index (workspace) state. It knows how to determine whether the pre-conditions for a commit have been reached. It knows how to read the workspace structures and coordinate read and writes from and to the repository.
I really don't think this makes the API unintuitive, but then I don't really understand why you think this is the case.
jon.
--
homepage: http://www.zeta.org.au/~jon/
blog: http://orwelliantremors.blogspot.com
Hi
Note that eclipse in particular has a fairly complicated repository provider interface. The subversion plugin developers (the subclipse project) took quite a while to implement their stuff. Basing your stuff off of their code would be a good idea.
Also, they worked in 2 stages - in the first stage, they created something called the JavaHL interface, behind which they talked to the subversion C libraries using JNI. Then they created a pure-java implemenation of the subversion C libraries which also implemented the JavaHL interface, allowing them to compare and contrast the behaviour.
Regards,
Noel Grandin
Quoted 54 lines · Jon Seymour
I have been experimenting with pure-Java implementation of GIT concepts with a goal of eventually providing plugins to Eclipse to allow the Eclipse GUI to interact with GIT repositories.
One thing I noticed when doing this is that the present index/cache structure is rather arbitrary and the optimum index structure is determined by the structure of the tools that use a GIT repository rather than the structure of the GIT repository itself.
To give a concrete example: the cache currently contains most of the posix stat structure primarily to allow quick change detection. In the Java world, most of the posix stat structure is not directly accessible via the pure-Java file system abstractions. However, for most purposes detecting changes to files modification time and file size would be enough. Given this is the case, a Java-GIT client doesn't need to bother getting access to a posix stat structure and could therefore get away with a simpler index structure, provided it doesn't need to interoperate with a 'C'-GIT client that shared the same workspace. A Java-GIT client might also choose to represent an index cache as a complex serialized Java object graph or (perhaps) an XML document.
Another example: I can imagine a variant of the index file structure that recorded all the parents which have been merged into the cache and automatically include this information when performing the commit.
The point is that many different index file structures are possible and will be determined in part by the tooling created in the porcelain layer - there really is no one true index file format as there is a one true repository format. Different tools can use different index file formats and still interoperate at the repository level because only the repository format needs to have a solid, unchanging definition.
Currently the GIT stack is structured as follows:
cogito
git-coreI think it would be worthwhile if care was taken to draw a distinction between the repository and the cache aspects of the git core, perhaps even going to the extreme of moving all knowledge of the cache into cogito itself. By clearly drawing this distinction, we will more easily enable the creation of different kind of tools sets atop the foundation of the GIT repository format.
e.g., either:
cogito
git-cache
git-respositoryor:
Quoted 9 lines
cogito-tools
cogito-cache
git-repositoryAnyway, I offer this as food for thought - chew or flame away as appropriate!
jon.
NOTICE: Please note that this email, and the contents thereof, are subject to the standard Peralex email disclaimer, which may be found at: http://www.peralex.com/disclaimer.html
If you cannot access the disclaimer through the URL attached and you wish to receive a copy thereof please send an email to [email protected]
Quoted 8 lines · Noel Grandin
Hi
Note that eclipse in particular has a fairly complicated repository provider interface.
The subversion plugin developers (the subclipse project) took quite a while to implement their stuff.
Basing your stuff off of their code would be a good idea.
Noel,
Thanks for the info. I'll certainly have a look at what the subclipse folks did as I am sure it will be helpful to understand the strategy. I do think we are in a slightly different situation here as we don't quite have a stable library interface to git yet - Brad Roberts work notwithstanding.
The repository API in pure Java is almost a no-brainer since Linus has done such a good job in keeping the repository specification simple and unambiguous.
A Java workspace API can take advantage of the abstraction and GUI facilities that Java and Eclipse afford so will naturally be different in form to the existing command line tools for manipulating the git index. Certainly, there will be similarities - aspects of the 3-way merge, for example, but there will be differences too - workspace change detection will be somewhat assisted by the change notification framework in Eclipse and won't require as much manual intervention.
Quoted 7 lines
Also, they worked in 2 stages - in the first stage, they created something called the JavaHL interface, behind which they talked to the subversion C libraries using JNI.
Then they created a pure-java implemenation of the subversion C libraries which also implemented the JavaHL interface, allowing them to compare and contrast the behaviour.
At this stage, my thoughts are to implement a listener pattern to keep the git index up to date, and I may well implement that by calling out the the C git-update-cache executable. This can run in a background thread so it needn't be a huge drag on interactive user performance.
Anyway, thanks for your input.
jon.
Quoted 6 lines
To give a concrete example: the cache currently contains most of the posix stat structure primarily to allow quick change detection. In the Java world, most of the posix stat structure is not directly accessible via the pure-Java file system abstractions. However, for most purposes detecting changes to files modification time and file size would be enough.
I've got exactly this problem in Darcs-git; and I ignore all of the cached data except the file size, mtime and sha1. I don't currently ever write to the cache.
Quoted 4 lines
I think it would be worthwhile if care was taken to draw a distinction between the repository and the cache aspects of the git core, perhaps even going to the extreme of moving all knowledge of the cache into cogito itself.
There's nothing that prevents you from ignoring the Git cache and using your own cache instead.
Juliusz