Skip to contentSkip to conversation

gitmodules: recurse by default

PATCH In review Michael S. Tsirkin 4 participants 15 messages Oct 6, 2022 – Oct 12, 2022

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 gitmodules: recurse by default

The most commmon complaint about submodules is that they don't follow when one switches branches in the main repo. Enable recursing into submodules by default to address that.

---
 .gitmodules | 23 +++++++++++++++++++++++
 1 file changed, 23 insertions(+)
.gitmodules +23 -0
index aedd9a03d4..5f32332aff 100644
@@ -1,69 +1,92 @@
 [submodule "roms/seabios"]
 	path = roms/seabios
+	recurse = true
 	url = https://gitlab.com/qemu-project/seabios.git/
 [submodule "roms/SLOF"]
 	path = roms/SLOF
+	recurse = true
 	url = https://gitlab.com/qemu-project/SLOF.git
 [submodule "roms/ipxe"]
 	path = roms/ipxe
+	recurse = true
 	url = https://gitlab.com/qemu-project/ipxe.git
 [submodule "roms/openbios"]
 	path = roms/openbios
+	recurse = true
 	url = https://gitlab.com/qemu-project/openbios.git
 [submodule "roms/qemu-palcode"]
 	path = roms/qemu-palcode
+	recurse = true
 	url = https://gitlab.com/qemu-project/qemu-palcode.git
 [submodule "roms/sgabios"]
 	path = roms/sgabios
+	recurse = true
 	url = https://gitlab.com/qemu-project/sgabios.git
 [submodule "dtc"]
 	path = dtc
+	recurse = true
 	url = https://gitlab.com/qemu-project/dtc.git
 [submodule "roms/u-boot"]
 	path = roms/u-boot
+	recurse = true
 	url = https://gitlab.com/qemu-project/u-boot.git
 [submodule "roms/skiboot"]
 	path = roms/skiboot
+	recurse = true
 	url = https://gitlab.com/qemu-project/skiboot.git
 [submodule "roms/QemuMacDrivers"]
 	path = roms/QemuMacDrivers
+	recurse = true
 	url = https://gitlab.com/qemu-project/QemuMacDrivers.git
 [submodule "ui/keycodemapdb"]
 	path = ui/keycodemapdb
+	recurse = true
 	url = https://gitlab.com/qemu-project/keycodemapdb.git
 [submodule "roms/seabios-hppa"]
 	path = roms/seabios-hppa
+	recurse = true
 	url = https://gitlab.com/qemu-project/seabios-hppa.git
 [submodule "roms/u-boot-sam460ex"]
 	path = roms/u-boot-sam460ex
+	recurse = true
 	url = https://gitlab.com/qemu-project/u-boot-sam460ex.git
 [submodule "tests/fp/berkeley-testfloat-3"]
 	path = tests/fp/berkeley-testfloat-3
+	recurse = true
 	url = https://gitlab.com/qemu-project/berkeley-testfloat-3.git
 [submodule "tests/fp/berkeley-softfloat-3"]
 	path = tests/fp/berkeley-softfloat-3
+	recurse = true
 	url = https://gitlab.com/qemu-project/berkeley-softfloat-3.git
 [submodule "roms/edk2"]
 	path = roms/edk2
+	recurse = true
 	url = https://gitlab.com/qemu-project/edk2.git
 [submodule "slirp"]
 	path = slirp
+	recurse = true
 	url = https://gitlab.com/qemu-project/libslirp.git
 [submodule "roms/opensbi"]
 	path = roms/opensbi
+	recurse = true
 	url = 	https://gitlab.com/qemu-project/opensbi.git
 [submodule "roms/qboot"]
 	path = roms/qboot
+	recurse = true
 	url = https://gitlab.com/qemu-project/qboot.git
 [submodule "meson"]
 	path = meson
+	recurse = true
 	url = https://gitlab.com/qemu-project/meson.git
 [submodule "roms/vbootrom"]
 	path = roms/vbootrom
+	recurse = true
 	url = https://gitlab.com/qemu-project/vbootrom.git
 [submodule "tests/lcitool/libvirt-ci"]
 	path = tests/lcitool/libvirt-ci
+	recurse = true
 	url = https://gitlab.com/libvirt/libvirt-ci.git
 [submodule "subprojects/libvfio-user"]
 	path = subprojects/libvfio-user
+	recurse = true
 	url = https://gitlab.com/qemu-project/libvfio-user.git
Signature · 1 line
MST
#1 lore raw
Quoted 5 lines · Michael S. Tsirkin

The most commmon complaint about submodules is that they don't follow when one switches branches in the main repo. Enable recursing into submodules by default to address that.

Just to check, because the git docs are a bit unclear to me, does this retain the existing behaviour that if a submodule isn't checked out at all then it remains not-checked-out ? (That is, we don't want to force developers to have checked out submodule sources for all the edk2 and other rom blob sources which aren't needed for day-to-day QEMU development.)

thanks
-- PMM

#2 lore raw
Quoted 14 lines · Peter Maydell
Quoted 5 lines · Michael S. Tsirkin

The most commmon complaint about submodules is that they don't follow when one switches branches in the main repo. Enable recursing into submodules by default to address that.

Just to check, because the git docs are a bit unclear to me, does this retain the existing behaviour that if a submodule isn't checked out at all then it remains not-checked-out ? (That is, we don't want to force developers to have checked out submodule sources for all the edk2 and other rom blob sources which aren't needed for day-to-day QEMU development.)

Quoted 2 lines

thanks
-- PMM

I think so - by default submodules are considered inactive, git only recurses into active modules.
I just tried the following:

Quoted 3 lines

git clone qemu qemu-tst
cd qemu-tst
ls -l $(grep submodule .gitmodules |sed -e 's/[^"]*"//' -e 's/".//')

dtc:
total 0

meson:
total 0

roms/edk2:
total 0

roms/ipxe:
total 0

roms/openbios:
total 0

roms/opensbi:
total 0

roms/qboot:
total 0

roms/QemuMacDrivers:
total 0

roms/qemu-palcode:
total 0

roms/seabios:
total 0

roms/seabios-hppa:
total 0

roms/sgabios:
total 0

roms/skiboot:
total 0

roms/SLOF:
total 0

roms/u-boot:
total 0

roms/u-boot-sam460ex:
total 0

roms/vbootrom:
total 0

slirp:
total 0

subprojects/libvfio-user:
total 0

tests/fp/berkeley-softfloat-3:
total 0

tests/fp/berkeley-testfloat-3:
total 0

tests/lcitool/libvirt-ci:
total 0

ui/keycodemapdb:
total 0

HTH

Signature · 1 line
MST
#3 lore raw
In reply to Michael S. Tsirkin

Hi Michael,

Quoted 1 line · Michael S. Tsirkin

The most commmon complaint about submodules is that

Typo "common"

Quoted 8 lines

they don't follow when one switches branches in the main repo. Enable recursing into submodules by default to address that.

---
  .gitmodules | 23 +++++++++++++++++++++++
  1 file changed, 23 insertions(+)

We have 3 kinds of submodule (or more?):
1/ required to *build* QEMU
2/ required to *test* QEMU
3/ only here for satisfies license of firmware blobs shipped with QEMU.

IIUC, long-term we want to move 3/ in another repository.

Could we only set "recurse=true" for 1/ ?

What is your use-case? Do we need it also for 2/ ?

Quoted 97 lines
.gitmodules +23 -0
index aedd9a03d4..5f32332aff 100644
@@ -1,69 +1,92 @@
  [submodule "roms/seabios"]
  	path = roms/seabios
+	recurse = true
  	url = https://gitlab.com/qemu-project/seabios.git/
  [submodule "roms/SLOF"]
  	path = roms/SLOF
+	recurse = true
  	url = https://gitlab.com/qemu-project/SLOF.git
  [submodule "roms/ipxe"]
  	path = roms/ipxe
+	recurse = true
  	url = https://gitlab.com/qemu-project/ipxe.git
  [submodule "roms/openbios"]
  	path = roms/openbios
+	recurse = true
  	url = https://gitlab.com/qemu-project/openbios.git
  [submodule "roms/qemu-palcode"]
  	path = roms/qemu-palcode
+	recurse = true
  	url = https://gitlab.com/qemu-project/qemu-palcode.git
  [submodule "roms/sgabios"]
  	path = roms/sgabios
+	recurse = true
  	url = https://gitlab.com/qemu-project/sgabios.git
  [submodule "dtc"]
  	path = dtc
+	recurse = true
  	url = https://gitlab.com/qemu-project/dtc.git
  [submodule "roms/u-boot"]
  	path = roms/u-boot
+	recurse = true
  	url = https://gitlab.com/qemu-project/u-boot.git
  [submodule "roms/skiboot"]
  	path = roms/skiboot
+	recurse = true
  	url = https://gitlab.com/qemu-project/skiboot.git
  [submodule "roms/QemuMacDrivers"]
  	path = roms/QemuMacDrivers
+	recurse = true
  	url = https://gitlab.com/qemu-project/QemuMacDrivers.git
  [submodule "ui/keycodemapdb"]
  	path = ui/keycodemapdb
+	recurse = true
  	url = https://gitlab.com/qemu-project/keycodemapdb.git
  [submodule "roms/seabios-hppa"]
  	path = roms/seabios-hppa
+	recurse = true
  	url = https://gitlab.com/qemu-project/seabios-hppa.git
  [submodule "roms/u-boot-sam460ex"]
  	path = roms/u-boot-sam460ex
+	recurse = true
  	url = https://gitlab.com/qemu-project/u-boot-sam460ex.git
  [submodule "tests/fp/berkeley-testfloat-3"]
  	path = tests/fp/berkeley-testfloat-3
+	recurse = true
  	url = https://gitlab.com/qemu-project/berkeley-testfloat-3.git
  [submodule "tests/fp/berkeley-softfloat-3"]
  	path = tests/fp/berkeley-softfloat-3
+	recurse = true
  	url = https://gitlab.com/qemu-project/berkeley-softfloat-3.git
  [submodule "roms/edk2"]
  	path = roms/edk2
+	recurse = true
  	url = https://gitlab.com/qemu-project/edk2.git
  [submodule "slirp"]
  	path = slirp
+	recurse = true
  	url = https://gitlab.com/qemu-project/libslirp.git
  [submodule "roms/opensbi"]
  	path = roms/opensbi
+	recurse = true
  	url = 	https://gitlab.com/qemu-project/opensbi.git
  [submodule "roms/qboot"]
  	path = roms/qboot
+	recurse = true
  	url = https://gitlab.com/qemu-project/qboot.git
  [submodule "meson"]
  	path = meson
+	recurse = true
  	url = https://gitlab.com/qemu-project/meson.git
  [submodule "roms/vbootrom"]
  	path = roms/vbootrom
+	recurse = true
  	url = https://gitlab.com/qemu-project/vbootrom.git
  [submodule "tests/lcitool/libvirt-ci"]
  	path = tests/lcitool/libvirt-ci
+	recurse = true
  	url = https://gitlab.com/libvirt/libvirt-ci.git
  [submodule "subprojects/libvfio-user"]
  	path = subprojects/libvfio-user
+	recurse = true
  	url = https://gitlab.com/qemu-project/libvfio-user.git
#4 lore raw
Quoted 21 lines · Philippe Mathieu-Daudé

Hi Michael,

Quoted 1 line · Michael S. Tsirkin

The most commmon complaint about submodules is that

Typo "common"

Quoted 8 lines

they don't follow when one switches branches in the main repo. Enable recursing into submodules by default to address that.

---
  .gitmodules | 23 +++++++++++++++++++++++
  1 file changed, 23 insertions(+)

We have 3 kinds of submodule (or more?):
1/ required to *build* QEMU
2/ required to *test* QEMU
3/ only here for satisfies license of firmware blobs shipped with QEMU.

Quoted 5 lines

IIUC, long-term we want to move 3/ in another repository.

Could we only set "recurse=true" for 1/ ?

What is your use-case? Do we need it also for 2/ ?

The use case is avoiding git diff output simply by switching branches. to serve that purpose all submodules need to be recurse=true unless they never change.

Quoted 97 lines
Quoted 97 lines
.gitmodules +23 -0
index aedd9a03d4..5f32332aff 100644
@@ -1,69 +1,92 @@
  [submodule "roms/seabios"]
  	path = roms/seabios
+	recurse = true
  	url = https://gitlab.com/qemu-project/seabios.git/
  [submodule "roms/SLOF"]
  	path = roms/SLOF
+	recurse = true
  	url = https://gitlab.com/qemu-project/SLOF.git
  [submodule "roms/ipxe"]
  	path = roms/ipxe
+	recurse = true
  	url = https://gitlab.com/qemu-project/ipxe.git
  [submodule "roms/openbios"]
  	path = roms/openbios
+	recurse = true
  	url = https://gitlab.com/qemu-project/openbios.git
  [submodule "roms/qemu-palcode"]
  	path = roms/qemu-palcode
+	recurse = true
  	url = https://gitlab.com/qemu-project/qemu-palcode.git
  [submodule "roms/sgabios"]
  	path = roms/sgabios
+	recurse = true
  	url = https://gitlab.com/qemu-project/sgabios.git
  [submodule "dtc"]
  	path = dtc
+	recurse = true
  	url = https://gitlab.com/qemu-project/dtc.git
  [submodule "roms/u-boot"]
  	path = roms/u-boot
+	recurse = true
  	url = https://gitlab.com/qemu-project/u-boot.git
  [submodule "roms/skiboot"]
  	path = roms/skiboot
+	recurse = true
  	url = https://gitlab.com/qemu-project/skiboot.git
  [submodule "roms/QemuMacDrivers"]
  	path = roms/QemuMacDrivers
+	recurse = true
  	url = https://gitlab.com/qemu-project/QemuMacDrivers.git
  [submodule "ui/keycodemapdb"]
  	path = ui/keycodemapdb
+	recurse = true
  	url = https://gitlab.com/qemu-project/keycodemapdb.git
  [submodule "roms/seabios-hppa"]
  	path = roms/seabios-hppa
+	recurse = true
  	url = https://gitlab.com/qemu-project/seabios-hppa.git
  [submodule "roms/u-boot-sam460ex"]
  	path = roms/u-boot-sam460ex
+	recurse = true
  	url = https://gitlab.com/qemu-project/u-boot-sam460ex.git
  [submodule "tests/fp/berkeley-testfloat-3"]
  	path = tests/fp/berkeley-testfloat-3
+	recurse = true
  	url = https://gitlab.com/qemu-project/berkeley-testfloat-3.git
  [submodule "tests/fp/berkeley-softfloat-3"]
  	path = tests/fp/berkeley-softfloat-3
+	recurse = true
  	url = https://gitlab.com/qemu-project/berkeley-softfloat-3.git
  [submodule "roms/edk2"]
  	path = roms/edk2
+	recurse = true
  	url = https://gitlab.com/qemu-project/edk2.git
  [submodule "slirp"]
  	path = slirp
+	recurse = true
  	url = https://gitlab.com/qemu-project/libslirp.git
  [submodule "roms/opensbi"]
  	path = roms/opensbi
+	recurse = true
  	url = 	https://gitlab.com/qemu-project/opensbi.git
  [submodule "roms/qboot"]
  	path = roms/qboot
+	recurse = true
  	url = https://gitlab.com/qemu-project/qboot.git
  [submodule "meson"]
  	path = meson
+	recurse = true
  	url = https://gitlab.com/qemu-project/meson.git
  [submodule "roms/vbootrom"]
  	path = roms/vbootrom
+	recurse = true
  	url = https://gitlab.com/qemu-project/vbootrom.git
  [submodule "tests/lcitool/libvirt-ci"]
  	path = tests/lcitool/libvirt-ci
+	recurse = true
  	url = https://gitlab.com/libvirt/libvirt-ci.git
  [submodule "subprojects/libvfio-user"]
  	path = subprojects/libvfio-user
+	recurse = true
  	url = https://gitlab.com/qemu-project/libvfio-user.git
#5 lore raw
In reply to Philippe Mathieu-Daudé via
Quoted 22 lines · Philippe Mathieu-Daudé via

Hi Michael,

Quoted 1 line · Michael S. Tsirkin

The most commmon complaint about submodules is that

Typo "common"

Quoted 8 lines

they don't follow when one switches branches in the main repo. Enable recursing into submodules by default to address that.

---
  .gitmodules | 23 +++++++++++++++++++++++
  1 file changed, 23 insertions(+)

We have 3 kinds of submodule (or more?):
1/ required to *build* QEMU
2/ required to *test* QEMU
3/ only here for satisfies license of firmware blobs shipped with QEMU.

Quoted 5 lines

IIUC, long-term we want to move 3/ in another repository.

Could we only set "recurse=true" for 1/ ?

What is your use-case? Do we need it also for 2/ ?

I think if we're going to change the git handling of submodules from the default, it's probably going to be less confusing if all our submodules behave the same way, rather than some being the git-default and some the recurse=true behaviour. Assuming there's no reason why we would need some submodules to keep the recurse=false behaviour, of course...

-- PMM

#6 lore raw
In reply to Michael S. Tsirkin
Quoted 9 lines · Michael S. Tsirkin

The most commmon complaint about submodules is that they don't follow when one switches branches in the main repo. Enable recursing into submodules by default to address that.

---
 .gitmodules | 23 +++++++++++++++++++++++
 1 file changed, 23 insertions(+)

What am I missing, as I tried something equiv to this change with one submodule and it didn't appear to have any effect. I started from a clean slate:

$ git submodule  deinit --all --force

The .gitmodules config has 'recurse = true' for the ui/keycodemapdb module:

$ grep keycodemap --after 1 .gitmodules 
[submodule "ui/keycodemapdb"]
	path = ui/keycodemapdb
	url = https://gitlab.com/qemu-project/keycodemapdb.git
	recurse = true

I have a change on master setting the submodule commit:

$ git show master | grep +Sub
+Subproject commit 7381b9bfadd31c4c9e9a10b5bb5032f9189d4352

and check it out:

$ git submodule update --init ui/keycodemapdb
Submodule 'ui/keycodemapdb' (https://gitlab.com/qemu-project/keycodemapdb.git) registered for path 'ui/keycodemapdb'
Submodule path 'ui/keycodemapdb': checked out '7381b9bfadd31c4c9e9a10b5bb5032f9189d4352'

In another branch I have a different commit:

$ git show sub-foo  | grep +Sub
+Subproject commit 57ba70da5312170883a3d622cd2aa3fd0e2ec7ae

Now I switch branches and nothing happens, the submodule is marked as dirty:

$ git checkout sub-foo
M	ui/keycodemapdb
Switched to branch 'sub-foo'
$ (cd ui/keycodemapdb && git show | head -1)
commit 7381b9bfadd31c4c9e9a10b5bb5032f9189d4352

From your description (and indeed that of the man page) it sounded like ui/keycodemapdb should have got updated to commit 57ba70da5312170883a3d622cd2aa3fd0e2ec7ae when I did 'git checkout', but that didn't happen

Quoted 101 lines
.gitmodules +23 -0
index aedd9a03d4..5f32332aff 100644
@@ -1,69 +1,92 @@
 [submodule "roms/seabios"]
 	path = roms/seabios
+	recurse = true
 	url = https://gitlab.com/qemu-project/seabios.git/
 [submodule "roms/SLOF"]
 	path = roms/SLOF
+	recurse = true
 	url = https://gitlab.com/qemu-project/SLOF.git
 [submodule "roms/ipxe"]
 	path = roms/ipxe
+	recurse = true
 	url = https://gitlab.com/qemu-project/ipxe.git
 [submodule "roms/openbios"]
 	path = roms/openbios
+	recurse = true
 	url = https://gitlab.com/qemu-project/openbios.git
 [submodule "roms/qemu-palcode"]
 	path = roms/qemu-palcode
+	recurse = true
 	url = https://gitlab.com/qemu-project/qemu-palcode.git
 [submodule "roms/sgabios"]
 	path = roms/sgabios
+	recurse = true
 	url = https://gitlab.com/qemu-project/sgabios.git
 [submodule "dtc"]
 	path = dtc
+	recurse = true
 	url = https://gitlab.com/qemu-project/dtc.git
 [submodule "roms/u-boot"]
 	path = roms/u-boot
+	recurse = true
 	url = https://gitlab.com/qemu-project/u-boot.git
 [submodule "roms/skiboot"]
 	path = roms/skiboot
+	recurse = true
 	url = https://gitlab.com/qemu-project/skiboot.git
 [submodule "roms/QemuMacDrivers"]
 	path = roms/QemuMacDrivers
+	recurse = true
 	url = https://gitlab.com/qemu-project/QemuMacDrivers.git
 [submodule "ui/keycodemapdb"]
 	path = ui/keycodemapdb
+	recurse = true
 	url = https://gitlab.com/qemu-project/keycodemapdb.git
 [submodule "roms/seabios-hppa"]
 	path = roms/seabios-hppa
+	recurse = true
 	url = https://gitlab.com/qemu-project/seabios-hppa.git
 [submodule "roms/u-boot-sam460ex"]
 	path = roms/u-boot-sam460ex
+	recurse = true
 	url = https://gitlab.com/qemu-project/u-boot-sam460ex.git
 [submodule "tests/fp/berkeley-testfloat-3"]
 	path = tests/fp/berkeley-testfloat-3
+	recurse = true
 	url = https://gitlab.com/qemu-project/berkeley-testfloat-3.git
 [submodule "tests/fp/berkeley-softfloat-3"]
 	path = tests/fp/berkeley-softfloat-3
+	recurse = true
 	url = https://gitlab.com/qemu-project/berkeley-softfloat-3.git
 [submodule "roms/edk2"]
 	path = roms/edk2
+	recurse = true
 	url = https://gitlab.com/qemu-project/edk2.git
 [submodule "slirp"]
 	path = slirp
+	recurse = true
 	url = https://gitlab.com/qemu-project/libslirp.git
 [submodule "roms/opensbi"]
 	path = roms/opensbi
+	recurse = true
 	url = 	https://gitlab.com/qemu-project/opensbi.git
 [submodule "roms/qboot"]
 	path = roms/qboot
+	recurse = true
 	url = https://gitlab.com/qemu-project/qboot.git
 [submodule "meson"]
 	path = meson
+	recurse = true
 	url = https://gitlab.com/qemu-project/meson.git
 [submodule "roms/vbootrom"]
 	path = roms/vbootrom
+	recurse = true
 	url = https://gitlab.com/qemu-project/vbootrom.git
 [submodule "tests/lcitool/libvirt-ci"]
 	path = tests/lcitool/libvirt-ci
+	recurse = true
 	url = https://gitlab.com/libvirt/libvirt-ci.git
 [submodule "subprojects/libvfio-user"]
 	path = subprojects/libvfio-user
+	recurse = true
 	url = https://gitlab.com/qemu-project/libvfio-user.git
Signature · 1 line
MST

With regards,
Daniel

Signature · 3 lines
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
#7 lore raw
Quoted 51 lines · Daniel P. Berrangé
Quoted 9 lines · Michael S. Tsirkin

The most commmon complaint about submodules is that they don't follow when one switches branches in the main repo. Enable recursing into submodules by default to address that.

---
 .gitmodules | 23 +++++++++++++++++++++++
 1 file changed, 23 insertions(+)

What am I missing, as I tried something equiv to this change with one submodule and it didn't appear to have any effect. I started from a clean slate:

$ git submodule  deinit --all --force

The .gitmodules config has 'recurse = true' for the ui/keycodemapdb module:

$ grep keycodemap --after 1 .gitmodules 
[submodule "ui/keycodemapdb"]
	path = ui/keycodemapdb
	url = https://gitlab.com/qemu-project/keycodemapdb.git
	recurse = true

I have a change on master setting the submodule commit:

$ git show master | grep +Sub
+Subproject commit 7381b9bfadd31c4c9e9a10b5bb5032f9189d4352

and check it out:

$ git submodule update --init ui/keycodemapdb
Submodule 'ui/keycodemapdb' (https://gitlab.com/qemu-project/keycodemapdb.git) registered for path 'ui/keycodemapdb'
Submodule path 'ui/keycodemapdb': checked out '7381b9bfadd31c4c9e9a10b5bb5032f9189d4352'

In another branch I have a different commit:

$ git show sub-foo  | grep +Sub
+Subproject commit 57ba70da5312170883a3d622cd2aa3fd0e2ec7ae

Now I switch branches and nothing happens, the submodule is marked as dirty:

$ git checkout sub-foo
M	ui/keycodemapdb
Switched to branch 'sub-foo'
Quoted 8 lines
$ (cd ui/keycodemapdb && git show | head -1)
commit 7381b9bfadd31c4c9e9a10b5bb5032f9189d4352
Quoted 1 line

From your description (and indeed that of the man page) it

sounded like ui/keycodemapdb should have got updated to commit 57ba70da5312170883a3d622cd2aa3fd0e2ec7ae when I did 'git checkout', but that didn't happen

I think .gitmodules got overwritten when you checked out the branch. Possible?

Quoted 109 lines
Quoted 102 lines
.gitmodules +23 -0
index aedd9a03d4..5f32332aff 100644
@@ -1,69 +1,92 @@
 [submodule "roms/seabios"]
 	path = roms/seabios
+	recurse = true
 	url = https://gitlab.com/qemu-project/seabios.git/
 [submodule "roms/SLOF"]
 	path = roms/SLOF
+	recurse = true
 	url = https://gitlab.com/qemu-project/SLOF.git
 [submodule "roms/ipxe"]
 	path = roms/ipxe
+	recurse = true
 	url = https://gitlab.com/qemu-project/ipxe.git
 [submodule "roms/openbios"]
 	path = roms/openbios
+	recurse = true
 	url = https://gitlab.com/qemu-project/openbios.git
 [submodule "roms/qemu-palcode"]
 	path = roms/qemu-palcode
+	recurse = true
 	url = https://gitlab.com/qemu-project/qemu-palcode.git
 [submodule "roms/sgabios"]
 	path = roms/sgabios
+	recurse = true
 	url = https://gitlab.com/qemu-project/sgabios.git
 [submodule "dtc"]
 	path = dtc
+	recurse = true
 	url = https://gitlab.com/qemu-project/dtc.git
 [submodule "roms/u-boot"]
 	path = roms/u-boot
+	recurse = true
 	url = https://gitlab.com/qemu-project/u-boot.git
 [submodule "roms/skiboot"]
 	path = roms/skiboot
+	recurse = true
 	url = https://gitlab.com/qemu-project/skiboot.git
 [submodule "roms/QemuMacDrivers"]
 	path = roms/QemuMacDrivers
+	recurse = true
 	url = https://gitlab.com/qemu-project/QemuMacDrivers.git
 [submodule "ui/keycodemapdb"]
 	path = ui/keycodemapdb
+	recurse = true
 	url = https://gitlab.com/qemu-project/keycodemapdb.git
 [submodule "roms/seabios-hppa"]
 	path = roms/seabios-hppa
+	recurse = true
 	url = https://gitlab.com/qemu-project/seabios-hppa.git
 [submodule "roms/u-boot-sam460ex"]
 	path = roms/u-boot-sam460ex
+	recurse = true
 	url = https://gitlab.com/qemu-project/u-boot-sam460ex.git
 [submodule "tests/fp/berkeley-testfloat-3"]
 	path = tests/fp/berkeley-testfloat-3
+	recurse = true
 	url = https://gitlab.com/qemu-project/berkeley-testfloat-3.git
 [submodule "tests/fp/berkeley-softfloat-3"]
 	path = tests/fp/berkeley-softfloat-3
+	recurse = true
 	url = https://gitlab.com/qemu-project/berkeley-softfloat-3.git
 [submodule "roms/edk2"]
 	path = roms/edk2
+	recurse = true
 	url = https://gitlab.com/qemu-project/edk2.git
 [submodule "slirp"]
 	path = slirp
+	recurse = true
 	url = https://gitlab.com/qemu-project/libslirp.git
 [submodule "roms/opensbi"]
 	path = roms/opensbi
+	recurse = true
 	url = 	https://gitlab.com/qemu-project/opensbi.git
 [submodule "roms/qboot"]
 	path = roms/qboot
+	recurse = true
 	url = https://gitlab.com/qemu-project/qboot.git
 [submodule "meson"]
 	path = meson
+	recurse = true
 	url = https://gitlab.com/qemu-project/meson.git
 [submodule "roms/vbootrom"]
 	path = roms/vbootrom
+	recurse = true
 	url = https://gitlab.com/qemu-project/vbootrom.git
 [submodule "tests/lcitool/libvirt-ci"]
 	path = tests/lcitool/libvirt-ci
+	recurse = true
 	url = https://gitlab.com/libvirt/libvirt-ci.git
 [submodule "subprojects/libvfio-user"]
 	path = subprojects/libvfio-user
+	recurse = true
 	url = https://gitlab.com/qemu-project/libvfio-user.git
Signature · 1 line
MST

With regards,
Daniel

Signature · 3 lines
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
#8 lore raw
Quoted 56 lines · Michael S. Tsirkin
Quoted 55 lines · Daniel P. Berrangé
Quoted 9 lines · Michael S. Tsirkin

The most commmon complaint about submodules is that they don't follow when one switches branches in the main repo. Enable recursing into submodules by default to address that.

---
 .gitmodules | 23 +++++++++++++++++++++++
 1 file changed, 23 insertions(+)

What am I missing, as I tried something equiv to this change with one submodule and it didn't appear to have any effect. I started from a clean slate:

$ git submodule  deinit --all --force

The .gitmodules config has 'recurse = true' for the ui/keycodemapdb module:

$ grep keycodemap --after 1 .gitmodules 
[submodule "ui/keycodemapdb"]
	path = ui/keycodemapdb
	url = https://gitlab.com/qemu-project/keycodemapdb.git
	recurse = true

I have a change on master setting the submodule commit:

$ git show master | grep +Sub
+Subproject commit 7381b9bfadd31c4c9e9a10b5bb5032f9189d4352

and check it out:

$ git submodule update --init ui/keycodemapdb
Submodule 'ui/keycodemapdb' (https://gitlab.com/qemu-project/keycodemapdb.git) registered for path 'ui/keycodemapdb'
Submodule path 'ui/keycodemapdb': checked out '7381b9bfadd31c4c9e9a10b5bb5032f9189d4352'

In another branch I have a different commit:

$ git show sub-foo  | grep +Sub
+Subproject commit 57ba70da5312170883a3d622cd2aa3fd0e2ec7ae

Now I switch branches and nothing happens, the submodule is marked as dirty:

$ git checkout sub-foo
M	ui/keycodemapdb
Switched to branch 'sub-foo'
$ (cd ui/keycodemapdb && git show | head -1)
commit 7381b9bfadd31c4c9e9a10b5bb5032f9189d4352
Quoted 7 lines
Quoted 4 lines
Quoted 1 line

From your description (and indeed that of the man page) it

sounded like ui/keycodemapdb should have got updated to commit 57ba70da5312170883a3d622cd2aa3fd0e2ec7ae when I did 'git checkout', but that didn't happen

I think .gitmodules got overwritten when you checked out the branch. Possible?

No, it the same in both branches, since they share a common ancestor commit which added the recurse tag

$ git checkout master
Switched to branch 'master'
$ git grep keycodemapdb --after 1 .gitmodules 
[submodule "ui/keycodemapdb"]
	path = ui/keycodemapdb
	url = https://gitlab.com/qemu-project/keycodemapdb.git
	recurse = true
$ git checkout sub-foo 
M	ui/keycodemapdb
Switched to branch 'sub-foo'
$ grep keycodemapdb --after 1 .gitmodules 
[submodule "ui/keycodemapdb"]
	path = ui/keycodemapdb
	url = https://gitlab.com/qemu-project/keycodemapdb.git
	recurse = true

With regards,
Daniel

Signature · 3 lines
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
#9 lore raw
Quoted 85 lines · Daniel P. Berrangé
Quoted 56 lines · Michael S. Tsirkin
Quoted 55 lines · Daniel P. Berrangé
Quoted 9 lines · Michael S. Tsirkin

The most commmon complaint about submodules is that they don't follow when one switches branches in the main repo. Enable recursing into submodules by default to address that.

---
 .gitmodules | 23 +++++++++++++++++++++++
 1 file changed, 23 insertions(+)

What am I missing, as I tried something equiv to this change with one submodule and it didn't appear to have any effect. I started from a clean slate:

$ git submodule  deinit --all --force

The .gitmodules config has 'recurse = true' for the ui/keycodemapdb module:

$ grep keycodemap --after 1 .gitmodules 
[submodule "ui/keycodemapdb"]
	path = ui/keycodemapdb
	url = https://gitlab.com/qemu-project/keycodemapdb.git
	recurse = true

I have a change on master setting the submodule commit:

$ git show master | grep +Sub
+Subproject commit 7381b9bfadd31c4c9e9a10b5bb5032f9189d4352

and check it out:

$ git submodule update --init ui/keycodemapdb
Submodule 'ui/keycodemapdb' (https://gitlab.com/qemu-project/keycodemapdb.git) registered for path 'ui/keycodemapdb'
Submodule path 'ui/keycodemapdb': checked out '7381b9bfadd31c4c9e9a10b5bb5032f9189d4352'

In another branch I have a different commit:

$ git show sub-foo  | grep +Sub
+Subproject commit 57ba70da5312170883a3d622cd2aa3fd0e2ec7ae

Now I switch branches and nothing happens, the submodule is marked as dirty:

$ git checkout sub-foo
M	ui/keycodemapdb
Switched to branch 'sub-foo'
$ (cd ui/keycodemapdb && git show | head -1)
commit 7381b9bfadd31c4c9e9a10b5bb5032f9189d4352
Quoted 7 lines
Quoted 4 lines
Quoted 1 line

From your description (and indeed that of the man page) it

sounded like ui/keycodemapdb should have got updated to commit 57ba70da5312170883a3d622cd2aa3fd0e2ec7ae when I did 'git checkout', but that didn't happen

I think .gitmodules got overwritten when you checked out the branch. Possible?

No, it the same in both branches, since they share a common ancestor commit which added the recurse tag

$ git checkout master
Switched to branch 'master'
$ git grep keycodemapdb --after 1 .gitmodules 
[submodule "ui/keycodemapdb"]
	path = ui/keycodemapdb
	url = https://gitlab.com/qemu-project/keycodemapdb.git
	recurse = true
$ git checkout sub-foo 
M	ui/keycodemapdb
Switched to branch 'sub-foo'
$ grep keycodemapdb --after 1 .gitmodules 
[submodule "ui/keycodemapdb"]
	path = ui/keycodemapdb
	url = https://gitlab.com/qemu-project/keycodemapdb.git
	recurse = true
Quoted 6 lines

With regards,
Daniel

Signature · 3 lines
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

I just retested and it's not working for me either :(
I was sure it worked but I guess the testing wasn't done properly. Back to the drawing board sorry.

Signature · 1 line
MST
#10 lore raw
Quoted 86 lines · Michael S. Tsirkin
Quoted 68 lines · Daniel P. Berrangé
Quoted 56 lines · Michael S. Tsirkin
Quoted 55 lines · Daniel P. Berrangé
Quoted 9 lines · Michael S. Tsirkin

The most commmon complaint about submodules is that they don't follow when one switches branches in the main repo. Enable recursing into submodules by default to address that.

---
 .gitmodules | 23 +++++++++++++++++++++++
 1 file changed, 23 insertions(+)

What am I missing, as I tried something equiv to this change with one submodule and it didn't appear to have any effect. I started from a clean slate:

$ git submodule  deinit --all --force

The .gitmodules config has 'recurse = true' for the ui/keycodemapdb module:

$ grep keycodemap --after 1 .gitmodules 
[submodule "ui/keycodemapdb"]
	path = ui/keycodemapdb
	url = https://gitlab.com/qemu-project/keycodemapdb.git
	recurse = true

I have a change on master setting the submodule commit:

$ git show master | grep +Sub
+Subproject commit 7381b9bfadd31c4c9e9a10b5bb5032f9189d4352

and check it out:

$ git submodule update --init ui/keycodemapdb
Submodule 'ui/keycodemapdb' (https://gitlab.com/qemu-project/keycodemapdb.git) registered for path 'ui/keycodemapdb'
Submodule path 'ui/keycodemapdb': checked out '7381b9bfadd31c4c9e9a10b5bb5032f9189d4352'

In another branch I have a different commit:

$ git show sub-foo  | grep +Sub
+Subproject commit 57ba70da5312170883a3d622cd2aa3fd0e2ec7ae

Now I switch branches and nothing happens, the submodule is marked as dirty:

$ git checkout sub-foo
M	ui/keycodemapdb
Switched to branch 'sub-foo'
$ (cd ui/keycodemapdb && git show | head -1)
commit 7381b9bfadd31c4c9e9a10b5bb5032f9189d4352
Quoted 7 lines
Quoted 4 lines
Quoted 1 line

From your description (and indeed that of the man page) it

sounded like ui/keycodemapdb should have got updated to commit 57ba70da5312170883a3d622cd2aa3fd0e2ec7ae when I did 'git checkout', but that didn't happen

I think .gitmodules got overwritten when you checked out the branch. Possible?

No, it the same in both branches, since they share a common ancestor commit which added the recurse tag

Quoted 16 lines
$ git checkout master
Switched to branch 'master'
$ git grep keycodemapdb --after 1 .gitmodules 
[submodule "ui/keycodemapdb"]
	path = ui/keycodemapdb
	url = https://gitlab.com/qemu-project/keycodemapdb.git
	recurse = true
$ git checkout sub-foo 
M	ui/keycodemapdb
Switched to branch 'sub-foo'
$ grep keycodemapdb --after 1 .gitmodules 
[submodule "ui/keycodemapdb"]
	path = ui/keycodemapdb
	url = https://gitlab.com/qemu-project/keycodemapdb.git
	recurse = true
Quoted 3 lines

I just retested and it's not working for me either :(
I was sure it worked but I guess the testing wasn't done properly. Back to the drawing board sorry.

I think the problem is that this setting doesn't apply in the context of .gitmodules. Various commands take a '--recurse-submodules' parameter, and like many params this can be set in the .git/config file. The problem is .git/config isn't a file we can influence automatically, it is upto the dev to set things for every clone they do :-(

With regards,
Daniel

Signature · 3 lines
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
#11 lore raw
Quoted 14 lines · Daniel P. Berrangé
Quoted 13 lines · Michael S. Tsirkin
Quoted 12 lines · Daniel P. Berrangé
Quoted 11 lines · Michael S. Tsirkin
Quoted 10 lines · Daniel P. Berrangé
Quoted 9 lines · Michael S. Tsirkin

The most commmon complaint about submodules is that they don't follow when one switches branches in the main repo. Enable recursing into submodules by default to address that.

---
 .gitmodules | 23 +++++++++++++++++++++++
 1 file changed, 23 insertions(+)

snip

Quoted 9 lines
Quoted 3 lines

I just retested and it's not working for me either :(
I was sure it worked but I guess the testing wasn't done properly. Back to the drawing board sorry.

I think the problem is that this setting doesn't apply in the context of .gitmodules. Various commands take a '--recurse-submodules' parameter, and like many params this can be set in the .git/config file. The problem is .git/config isn't a file we can influence automatically, it is upto the dev to set things for every clone they do :-(

With the correct setting in my .git/config, I've just discovered an unexpected & undesirable consequence of using recurse=true. It affects the 'push' command. If your submodule contains a hash that is not present in the upstream of the submodule, then when you try to push, it will also try to push the submodule change.

eg, I have a qemu.git branch 'work' and i made a change to ui/keycodemapdb. If I try to push to my gitlab fork, whose remote I called 'gitlab', then it will also try to push ui/keycodemapdb to a fork called 'gitlab'. Except I don't have any such fork existing, so my attempt to push my qemu.git changes fails because of the submodule.

This is going to be annoying to people who are working on branches with updates to the git submodules if we were to set recurse=true by default, as they'll have to also setup remotes for submodules they work on.

With regards,
Daniel

Signature · 3 lines
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
#12 lore raw
Quoted 41 lines · Daniel P. Berrangé
Quoted 14 lines · Daniel P. Berrangé
Quoted 13 lines · Michael S. Tsirkin
Quoted 12 lines · Daniel P. Berrangé
Quoted 11 lines · Michael S. Tsirkin
Quoted 10 lines · Daniel P. Berrangé
Quoted 9 lines · Michael S. Tsirkin

The most commmon complaint about submodules is that they don't follow when one switches branches in the main repo. Enable recursing into submodules by default to address that.

---
 .gitmodules | 23 +++++++++++++++++++++++
 1 file changed, 23 insertions(+)

snip

Quoted 9 lines
Quoted 3 lines

I just retested and it's not working for me either :(
I was sure it worked but I guess the testing wasn't done properly. Back to the drawing board sorry.

I think the problem is that this setting doesn't apply in the context of .gitmodules. Various commands take a '--recurse-submodules' parameter, and like many params this can be set in the .git/config file. The problem is .git/config isn't a file we can influence automatically, it is upto the dev to set things for every clone they do :-(

With the correct setting in my .git/config, I've just discovered an unexpected & undesirable consequence of using recurse=true. It affects the 'push' command. If your submodule contains a hash that is not present in the upstream of the submodule, then when you try to push, it will also try to push the submodule change.

eg, I have a qemu.git branch 'work' and i made a change to ui/keycodemapdb. If I try to push to my gitlab fork, whose remote I called 'gitlab', then it will also try to push ui/keycodemapdb to a fork called 'gitlab'. Except I don't have any such fork existing, so my attempt to push my qemu.git changes fails because of the submodule.

Quoted 7 lines

This is going to be annoying to people who are working on branches with updates to the git submodules if we were to set recurse=true by default, as they'll have to also setup remotes for submodules they work on.

With regards,
Daniel

Well this seems like a reasonable thing to do, no?

If you push qemu commit referring to hash 0xABC, you want that 0xABC to be available in the remote, no? Otherwise how will people fetching your tree check it out?

Quoted 4 lines
Signature · 3 lines
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
#13 lore raw
Quoted 48 lines · Michael S. Tsirkin
Quoted 41 lines · Daniel P. Berrangé
Quoted 14 lines · Daniel P. Berrangé
Quoted 13 lines · Michael S. Tsirkin
Quoted 12 lines · Daniel P. Berrangé
Quoted 11 lines · Michael S. Tsirkin
Quoted 10 lines · Daniel P. Berrangé
Quoted 9 lines · Michael S. Tsirkin

The most commmon complaint about submodules is that they don't follow when one switches branches in the main repo. Enable recursing into submodules by default to address that.

---
 .gitmodules | 23 +++++++++++++++++++++++
 1 file changed, 23 insertions(+)

snip

Quoted 9 lines
Quoted 3 lines

I just retested and it's not working for me either :(
I was sure it worked but I guess the testing wasn't done properly. Back to the drawing board sorry.

I think the problem is that this setting doesn't apply in the context of .gitmodules. Various commands take a '--recurse-submodules' parameter, and like many params this can be set in the .git/config file. The problem is .git/config isn't a file we can influence automatically, it is upto the dev to set things for every clone they do :-(

With the correct setting in my .git/config, I've just discovered an unexpected & undesirable consequence of using recurse=true. It affects the 'push' command. If your submodule contains a hash that is not present in the upstream of the submodule, then when you try to push, it will also try to push the submodule change.

eg, I have a qemu.git branch 'work' and i made a change to ui/keycodemapdb. If I try to push to my gitlab fork, whose remote I called 'gitlab', then it will also try to push ui/keycodemapdb to a fork called 'gitlab'. Except I don't have any such fork existing, so my attempt to push my qemu.git changes fails because of the submodule.

Quoted 5 lines

This is going to be annoying to people who are working on branches with updates to the git submodules if we were to set recurse=true by default, as they'll have to also setup remotes for submodules they work on.

Quoted 5 lines

Well this seems like a reasonable thing to do, no?

If you push qemu commit referring to hash 0xABC, you want that 0xABC to be available in the remote, no? Otherwise how will people fetching your tree check it out?

Don't assume I'm making it available for other people. I push to remotes simply for moving code around for myself between machines. I still have the submodule code I need elsewhere, so forcing me to push the submodule & main repos so the same named remote is getting in the way.

With regards,
Daniel

Signature · 3 lines
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
#14 lore raw
Quoted 55 lines · Daniel P. Berrangé
Quoted 48 lines · Michael S. Tsirkin
Quoted 41 lines · Daniel P. Berrangé
Quoted 14 lines · Daniel P. Berrangé
Quoted 13 lines · Michael S. Tsirkin
Quoted 12 lines · Daniel P. Berrangé
Quoted 11 lines · Michael S. Tsirkin
Quoted 10 lines · Daniel P. Berrangé
Quoted 9 lines · Michael S. Tsirkin

The most commmon complaint about submodules is that they don't follow when one switches branches in the main repo. Enable recursing into submodules by default to address that.

---
 .gitmodules | 23 +++++++++++++++++++++++
 1 file changed, 23 insertions(+)

snip

Quoted 9 lines
Quoted 3 lines

I just retested and it's not working for me either :(
I was sure it worked but I guess the testing wasn't done properly. Back to the drawing board sorry.

I think the problem is that this setting doesn't apply in the context of .gitmodules. Various commands take a '--recurse-submodules' parameter, and like many params this can be set in the .git/config file. The problem is .git/config isn't a file we can influence automatically, it is upto the dev to set things for every clone they do :-(

With the correct setting in my .git/config, I've just discovered an unexpected & undesirable consequence of using recurse=true. It affects the 'push' command. If your submodule contains a hash that is not present in the upstream of the submodule, then when you try to push, it will also try to push the submodule change.

eg, I have a qemu.git branch 'work' and i made a change to ui/keycodemapdb. If I try to push to my gitlab fork, whose remote I called 'gitlab', then it will also try to push ui/keycodemapdb to a fork called 'gitlab'. Except I don't have any such fork existing, so my attempt to push my qemu.git changes fails because of the submodule.

Quoted 5 lines

This is going to be annoying to people who are working on branches with updates to the git submodules if we were to set recurse=true by default, as they'll have to also setup remotes for submodules they work on.

Quoted 5 lines

Well this seems like a reasonable thing to do, no?

If you push qemu commit referring to hash 0xABC, you want that 0xABC to be available in the remote, no? Otherwise how will people fetching your tree check it out?

Quoted 8 lines

Don't assume I'm making it available for other people. I push to remotes simply for moving code around for myself between machines. I still have the submodule code I need elsewhere, so forcing me to push the submodule & main repos so the same named remote is getting in the way.

With regards,
Daniel

So you are pushing this main tree but the submodule won't be needed there? Sounds like an unusual enough thing that it's reasonable to ask user to tweak config to enable it.

Quoted 4 lines
Signature · 3 lines
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|