Skip to contentSkip to conversation

RFC: Support for R/W git in bitbake

DISCUSSION Ulf Samuelsson 5 participants 7 messages Apr 17, 2012 – Apr 19, 2012

lore
Share & tools
lore
#0 lore raw

Today bitbake supports read only git access in recipes. For various reasons, I would like to be able to do recipes which would check out in a read/write mode.

I.E: have bitbake do "git clone [email protected]:myproject"

Did a prototype extension to bitbake that assumes the following URI
wgit://emagii.com/myproject

Any objections to such an extension?
Suggestions for an alternative implementation?

Signature · 3 lines
Best Regards
Ulf Samuelsson
[email protected]
#1 lore raw
Quoted 3 lines · On Wednesday 18 April 2012 00:35:29 Ulf Samuelsson wrote:

Today bitbake supports read only git access in recipes. For various reasons, I would like to be able to do recipes which would check out in a read/write mode.

Could you elaborate on "various reasons"?

Would the new externalsrc bbclass be useful in your case?

Cheers,
Paul

Signature · 2 lines
Paul Eggleton
Intel Open Source Technology Centre
#2 lore raw
Quoted 11 lines · Paul Eggleton
Quoted 3 lines · On Wednesday 18 April 2012 00:35:29 Ulf Samuelsson wrote:

Today bitbake supports read only git access in recipes. For various reasons, I would like to be able to do recipes which would check out in a read/write mode.

Could you elaborate on "various reasons"?

Would the new externalsrc bbclass be useful in your case?

Cheers,
Paul

1. If I am busy working on an application, then it simplifies the development process. I can modify the code in the tree and push.
This is mainly for kernel development.

2. If I work on a prerelease of some S/W drivers/Applications under NDA, then I cannot make that code publicly available
but I still want to put that on my Internet accessible git server. Typicailly this is before the release of a new chip and info about
the chip should not be
made public before the chip is released.

3. I want to be able to ship something similar to the Angstrom setup scripts to someone else, and have them build an image, but it should not
be available
to anyone not accepted (without public key at the git server).

There are other uses for such a functionality, but those are my immediate needs.

As you see, this is mostly for development.
Once the code is released, then the recipe would be changed to the normal git access.

Didn't know anything about the externalsrc bbclass, but after checking, I would say no. It won't do the two things above. I do see the use of it though.

Signature · 4 lines
Best Regards
Ulf Samuelsson
[email protected]
+46 722 427437
#3 lore raw
Quoted 9 lines · On Wed, 18 Apr 2012 03:34:27 +0200 Ulf Samuelsson <[email protected]>…
Quoted 8 lines · Paul Eggleton
Quoted 3 lines · On Wednesday 18 April 2012 00:35:29 Ulf Samuelsson wrote:

Today bitbake supports read only git access in recipes. For various reasons, I would like to be able to do recipes which would check out in a read/write mode.

Could you elaborate on "various reasons"?

Would the new externalsrc bbclass be useful in your case?

Quoted 9 lines
Quoted 3 lines

Cheers,
Paul

1. If I am busy working on an application, then it simplifies the development process.
I can modify the code in the tree and push.
This is mainly for kernel development.

for that you can pass "protocol=file" to make the git fetcher use the "file" transport, see

http://ao2.it/en/blog/2010/05/27/neat-compilerun-cycle-git-and-openembedded

basically, in the .bb recipe, you can fetch from a local clone where you do your normal work.

Quoted 7 lines

2. If I work on a prerelease of some S/W drivers/Applications under NDA, then I cannot make that code publicly available
but I still want to put that on my Internet accessible git server. Typicailly this is before the release of a new chip and info about
the chip should not be
made public before the chip is released.

Quoted 6 lines

3. I want to be able to ship something similar to the Angstrom setup scripts
to someone else, and have them build an image, but it should not
be available
to anyone not accepted (without public key at the git server).

maybe you can pass "protocol=ssh" as well to the git fetcher as well, but I haven't tried that, when you say "R/W mode" you are basically saying "ssh transport" for git, right? Let us know if that works.

Quoted 11 lines

There are other uses for such a functionality, but those are my immediate needs.

As you see, this is mostly for development.
Once the code is released, then the recipe would be changed to the normal git access.

Didn't know anything about the externalsrc bbclass, but after checking, I would say no.
It won't do the two things above. I do see the use of it though.

Ciao,
Antonio

Signature · 6 lines
Antonio Ospite
http://ao2.it

A: Because it messes up the order in which people normally read text.
   See http://en.wikipedia.org/wiki/Posting_style
Q: Why is top-posting such a bad thing?
#5 lore raw
In reply to Ulf Samuelsson
Quoted 21 lines · On Wednesday 18 April 2012 03:34:27 Ulf Samuelsson wrote:

1. If I am busy working on an application, then it simplifies the development process.
I can modify the code in the tree and push.
This is mainly for kernel development.

2. If I work on a prerelease of some S/W drivers/Applications under NDA, then I cannot make that code publicly available
but I still want to put that on my Internet accessible git server. Typicailly this is before the release of a new chip and info about
the chip should not be
made public before the chip is released.

3. I want to be able to ship something similar to the Angstrom setup scripts
to someone else, and have them build an image, but it should not
be available
to anyone not accepted (without public key at the git server).

There are other uses for such a functionality, but those are my immediate needs.

Quoted 7 lines

As you see, this is mostly for development.
Once the code is released, then the recipe would be changed to the normal git access.

Didn't know anything about the externalsrc bbclass, but after checking, I would say no.
It won't do the two things above. I do see the use of it though.

externalsrc should handle everything except automatically fetching the source; for that you need to have your own local git clone (which of course can be r/w). I guess it depends on whether you expect to be sending such recipes to non-developers; for developers it ought not to be too much of a hassle to have their own local git clone.

The only problem with having an r/w checkout you are doing development in under WORKDIR is that if you bitbake -c clean the recipe you will lose whatever you are working on - externalsrc avoids this.

Cheers,
Paul

Signature · 2 lines
Paul Eggleton
Intel Open Source Technology Centre
#6 lore raw
Quoted 35 lines · Paul Eggleton
Quoted 21 lines · On Wednesday 18 April 2012 03:34:27 Ulf Samuelsson wrote:

1. If I am busy working on an application, then it simplifies the development process.
I can modify the code in the tree and push.
This is mainly for kernel development.

2. If I work on a prerelease of some S/W drivers/Applications under NDA, then I cannot make that code publicly available
but I still want to put that on my Internet accessible git server. Typicailly this is before the release of a new chip and info about
the chip should not be
made public before the chip is released.

3. I want to be able to ship something similar to the Angstrom setup scripts
to someone else, and have them build an image, but it should not
be available
to anyone not accepted (without public key at the git server).

There are other uses for such a functionality, but those are my immediate needs.

Quoted 7 lines

As you see, this is mostly for development.
Once the code is released, then the recipe would be changed to the normal git access.

Didn't know anything about the externalsrc bbclass, but after checking, I would say no.
It won't do the two things above. I do see the use of it though.

externalsrc should handle everything except automatically fetching the source; for that you need to have your own local git clone (which of course can be r/w). I guess it depends on whether you expect to be sending such recipes to non-developers; for developers it ought not to be too much of a hassle to have their own local git clone.

Quoted 7 lines

The only problem with having an r/w checkout you are doing development in under WORKDIR is that if you bitbake -c clean the recipe you will lose whatever you are working on - externalsrc avoids this.

Cheers,
Paul

I am working with a company, which OEMs their solution to other companies. They get tons of questions from those customers, and based on that I would like to avoid anything which requires manual intervention of the developer.

As for cleaning out changes, it might happen, but I am prepared to take the risk. Today I am initializing a git after the extract anyway.

I am sure I will use externalsrc for something.

Signature · 4 lines
Best Regards
Ulf Samuelsson
[email protected]
+46 722 427437
#4 lore raw
In reply to Antonio Ospite
Quoted 44 lines · Antonio Ospite
Quoted 9 lines · On Wed, 18 Apr 2012 03:34:27 +0200 Ulf Samuelsson <[email protected]>…
Quoted 8 lines · Paul Eggleton
Quoted 3 lines · On Wednesday 18 April 2012 00:35:29 Ulf Samuelsson wrote:

Today bitbake supports read only git access in recipes. For various reasons, I would like to be able to do recipes which would check out in a read/write mode.

Could you elaborate on "various reasons"?

Would the new externalsrc bbclass be useful in your case?

Quoted 9 lines
Quoted 3 lines

Cheers,
Paul

1. If I am busy working on an application, then it simplifies the development process.
     I can modify the code in the tree and push.
     This is mainly for kernel development.

for that you can pass "protocol=file" to make the git fetcher use the "file" transport, see

http://ao2.it/en/blog/2010/05/27/neat-compilerun-cycle-git-and-openembedded

basically, in the .bb recipe, you can fetch from a local clone where you do your normal work.

Quoted 7 lines

2. If I work on a prerelease of some S/W drivers/Applications under NDA, then I cannot make that code publicly available
     but I still want to put  that on my Internet accessible git server. Typicailly this is before the release of a new chip and info about the chip should not be
     made public before the chip is released.

Quoted 6 lines

3.  I want to be able to ship something similar to the Angstrom setup scripts
      to someone else, and have them build an image, but it should not be available
      to anyone not accepted (without public key at the git server).

Quoted 3 lines

maybe you can pass "protocol=ssh" as well to the git fetcher as well, but I haven't tried that, when you say "R/W mode" you are basically saying "ssh transport" for git, right? Let us know if that works.

Ulf, any comments on Antonios ideas?

I've just checked our own git repos and recipes; we're using SRC_URI's like:
SRC_URI = "git://gitosis@server-url/linux;protocol=ssh "
Then it is possible to work in that git clone as usual. Though, I've to mention that this is using oe-core, so there might be some differences in fetchers etc.

/Anders