Skip to contentSkip to conversation

golang/xenlight: Add libxl_utils support

PATCH In review George Dunlap 2 participants 13 messages Jul 18, 2019 – Jul 19, 2019

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 golang/xenlight: Add libxl_utils support

The Go bindings for libxl miss functions from libxl_utils, let's start with the simple libxl_domid_to_name and its counterpart libxl_name_to_domid.

NB that C.GoString() will return "" if it's passed a NULL; see
https://github.com/golang/go/issues/32734#issuecomment-506835432

---
v3:
- Wire into build system
- Add reference to C.GoString() handling NULL to commit message

Nicolas, could you test to see if this actually works for you?

It would be really good also if we could get something that would do basic unit testing on a live system, and get that running in osstest.

CC Nicolas Belouin <[email protected]>
CC Ian Jackson <[email protected]>
CC Wei Liu <[email protected]>
---
 tools/golang/xenlight/Makefile          |  2 +-
 tools/golang/xenlight/xenlight_utils.go | 55 +++++++++++++++++++++++++
 2 files changed, 56 insertions(+), 1 deletion(-)
 create mode 100644 tools/golang/xenlight/xenlight_utils.go
tools/golang/xenlight/Makefile +1 -1
index 0987305224..65923df59a 100644
@@ -8,7 +8,7 @@ GOXL_PKG_DIR = /src/$(XEN_GOCODE_URL)/xenlight/
 GOXL_INSTALL_DIR = $(GOCODE_DIR)$(GOXL_PKG_DIR)
 
 # PKGSOURCES: Files which comprise the distributed source package
-PKGSOURCES = xenlight.go
+PKGSOURCES = xenlight.go xenlight_utils.go
 
 GO ?= go
 
tools/golang/xenlight/xenlight_utils.go new file +55 -0

55 lines not on this page · raw message

Signature · 7 lines
2.20.1


_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
#1 lore raw
Quoted 7 lines · George Dunlap

The Go bindings for libxl miss functions from libxl_utils, let's start with the simple libxl_domid_to_name and its counterpart libxl_name_to_domid.

NB that C.GoString() will return "" if it's passed a NULL; see
https://github.com/golang/go/issues/32734#issuecomment-506835432

Quoted 8 lines

---
v3:
- Wire into build system
- Add reference to C.GoString() handling NULL to commit message

Nicolas, could you test to see if this actually works for you?

Tested it, it works.

I must confess I do not use that import path as the new modules mechanism introduced in Go1.11 downloads and compile a versioned copy of every dependency per project, and this behavior is incompatible with the build system used here.

Nicolas

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

#2 lore raw
Quoted 21 lines · Nicolas Belouin
Quoted 14 lines · George Dunlap

The Go bindings for libxl miss functions from libxl_utils, let's start with the simple libxl_domid_to_name and its counterpart libxl_name_to_domid.

NB that C.GoString() will return "" if it's passed a NULL; see
https://github.com/golang/go/issues/32734#issuecomment-506835432

---
v3:
- Wire into build system
- Add reference to C.GoString() handling NULL to commit message

Quoted 7 lines
Quoted 1 line

Nicolas, could you test to see if this actually works for you?

Tested it, it works.

I must confess I do not use that import path as the new modules mechanism introduced in Go1.11 downloads and compile a versioned copy of every dependency per project, and this behavior is incompatible with the build system used here.

It’s possible that something fundamentally has changed, but I suspect that rather you don’t quite understand how the current build system is supposed to work. (In which case a write-up in the tree would probably be useful.)

Go has always insisted that there be no binary compatibility between versions; so it’s always been necessary to re-compile all your libraries when upgrading from (say) 1.8 to 1.9. Which means that any useable distribution must also include all the source files necessary to recompile when you bump the version number.

So the core mechanism of the “install” is actually to copy all the source files necessary into the right local directory such that the go compiler can find them; ATM this is /usr/share/gocode/golang.xenproject.org/xenlight (as documented here: https://github.com/golang/go/wiki/FileTreeDocumentation). The files which are copied are specified by the PGKSOURCES variable in the makefile.

The purpose of building a binary during the normal “make” process is not actually to have a useable binary; it’s to detect any potential issues when you do the Xen build, rather than whenever you actually build a Go program for the first time (potentially on some end-user’s system). And “detect any issues” attempts to use “go build” in the situation you expect to use it at run time (i.e., it copies PKGSOURCES to a local directory, then points GO_PATH at them, points to the locally-built libraries, and then runs a plain go build). In this way it should, theoretically, catch any of the following errors:

* Syntax / whatever errors in the Go bindings themselves
* Mismatches between the Go bindings and libxl &c
* Missing system libraries
* Something not right in the installation path system

All that to say — the “real” way to enable a new file in the xenlight package is to add it to PKGSOURCES.

-George

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

#3 lore raw
Quoted 33 lines · George Dunlap
Quoted 22 lines · Nicolas Belouin
Quoted 14 lines · George Dunlap

The Go bindings for libxl miss functions from libxl_utils, let's start with the simple libxl_domid_to_name and its counterpart libxl_name_to_domid.

NB that C.GoString() will return "" if it's passed a NULL; see
https://github.com/golang/go/issues/32734#issuecomment-506835432

---
v3:
- Wire into build system
- Add reference to C.GoString() handling NULL to commit message

Quoted 7 lines
Quoted 1 line

Nicolas, could you test to see if this actually works for you?

Tested it, it works.

I must confess I do not use that import path as the new modules mechanism introduced in Go1.11 downloads and compile a versioned copy of every dependency per project, and this behavior is incompatible with the build system used here.

Quoted 5 lines

It’s possible that something fundamentally has changed, but I suspect that rather you don’t quite understand how the current build system is supposed to work. (In which case a write-up in the tree would probably be useful.)

Go has always insisted that there be no binary compatibility between versions; so it’s always been necessary to re-compile all your libraries when upgrading from (say) 1.8 to 1.9. Which means that any useable distribution must also include all the source files necessary to recompile when you bump the version number.

So the core mechanism of the “install” is actually to copy all the source files necessary into the right local directory such that the go compiler can find them; ATM this is /usr/share/gocode/golang.xenproject.org/xenlight

Nit: This of course should have a src/ between gocode/ and golang.xenproject.org/.

NB also that this naming scheme was designed so that at some point in the future, we could actually host the xenlight packages at the URL provided.

-George

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

#4 lore raw
Quoted 33 lines · George Dunlap
Quoted 33 lines · George Dunlap
Quoted 22 lines · Nicolas Belouin
Quoted 14 lines · George Dunlap

The Go bindings for libxl miss functions from libxl_utils, let's start with the simple libxl_domid_to_name and its counterpart libxl_name_to_domid.

NB that C.GoString() will return "" if it's passed a NULL; see
https://github.com/golang/go/issues/32734#issuecomment-506835432

---
v3:
- Wire into build system
- Add reference to C.GoString() handling NULL to commit message

Quoted 7 lines
Quoted 1 line

Nicolas, could you test to see if this actually works for you?

Tested it, it works.

I must confess I do not use that import path as the new modules mechanism introduced in Go1.11 downloads and compile a versioned copy of every dependency per project, and this behavior is incompatible with the build system used here.

It’s possible that something fundamentally has changed, but I suspect that rather you don’t quite understand how the current build system is supposed to work. (In which case a write-up in the tree would probably be useful.)

Quoted 9 lines
Quoted 3 lines

Go has always insisted that there be no binary compatibility between versions; so it’s always been necessary to re-compile all your libraries when upgrading from (say) 1.8 to 1.9. Which means that any useable distribution must also include all the source files necessary to recompile when you bump the version number.

So the core mechanism of the “install” is actually to copy all the source files necessary into the right local directory such that the go compiler can find them; ATM this is /usr/share/gocode/golang.xenproject.org/xenlight

Nit: This of course should have a src/ between gocode/ and golang.xenproject.org/.

NB also that this naming scheme was designed so that at some point in the future, we could actually host the xenlight packages at the URL provided.

-George

This new mechanism of modules is described here:
https://golang.org/cmd/go/#hdr-Modules__module_versions__and_more

The module system is intended to supersede the GOPATH approach and provide a way to get versioned dependencies, as such
it does not rely on GOPATH at all and doesn't use sources or compiled packages present in GOPATH elements such as /usr/share/gocode and systematically fetch (at the asked version) and compile a copy of the dependency as it might be a different version from the one in GOPATH.

As far as I tried, I have been unable to build my module even with the library installed.
I have to use xenbits.xen.org/git-http/xen.git/tools/golang/xenlight (or one of its mirror) in order to build the module using the new mechanism (the golang.xenproject.org/xenlight works when building with modules mode disabled).

Nicolas

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

#5 lore raw
Quoted 57 lines · Nicolas Belouin
Quoted 33 lines · George Dunlap
Quoted 33 lines · George Dunlap
Quoted 22 lines · Nicolas Belouin
Quoted 14 lines · George Dunlap

The Go bindings for libxl miss functions from libxl_utils, let's start with the simple libxl_domid_to_name and its counterpart libxl_name_to_domid.

NB that C.GoString() will return "" if it's passed a NULL; see
https://github.com/golang/go/issues/32734#issuecomment-506835432

---
v3:
- Wire into build system
- Add reference to C.GoString() handling NULL to commit message

Quoted 7 lines
Quoted 1 line

Nicolas, could you test to see if this actually works for you?

Tested it, it works.

I must confess I do not use that import path as the new modules mechanism introduced in Go1.11 downloads and compile a versioned copy of every dependency per project, and this behavior is incompatible with the build system used here.

It’s possible that something fundamentally has changed, but I suspect that rather you don’t quite understand how the current build system is supposed to work. (In which case a write-up in the tree would probably be useful.)

Quoted 9 lines
Quoted 3 lines

Go has always insisted that there be no binary compatibility between versions; so it’s always been necessary to re-compile all your libraries when upgrading from (say) 1.8 to 1.9. Which means that any useable distribution must also include all the source files necessary to recompile when you bump the version number.

So the core mechanism of the “install” is actually to copy all the source files necessary into the right local directory such that the go compiler can find them; ATM this is /usr/share/gocode/golang.xenproject.org/xenlight

Nit: This of course should have a src/ between gocode/ and golang.xenproject.org/.

NB also that this naming scheme was designed so that at some point in the future, we could actually host the xenlight packages at the URL provided.

-George

This new mechanism of modules is described here:
https://golang.org/cmd/go/#hdr-Modules__module_versions__and_more

The module system is intended to supersede the GOPATH approach and provide a way to get versioned dependencies, as such
it does not rely on GOPATH at all and doesn't use sources or compiled packages present in GOPATH elements such as /usr/share/gocode and systematically fetch (at the asked version) and compile a copy of the dependency as it might be a different version from the one in GOPATH.

Quoted 6 lines

As far as I tried, I have been unable to build my module even with the library installed.
I have to use xenbits.xen.org/git-http/xen.git/tools/golang/xenlight (or one of its mirror) in order to build the module using the new mechanism (the golang.xenproject.org/xenlight works when building with modules mode disabled).

I took a look at the module stuff when it came out, and I was never able to make sense of how it was supposed to work.

<rant>On the whole, it seems they basically hate the idea of distro packages, and seem intent on breaking them whenever people manage to start to get them working.</rant>

-George

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

#6 lore raw
Quoted 55 lines · George Dunlap
Quoted 54 lines · Nicolas Belouin
Quoted 32 lines · George Dunlap
Quoted 32 lines · George Dunlap
Quoted 22 lines · Nicolas Belouin
Quoted 14 lines · George Dunlap

The Go bindings for libxl miss functions from libxl_utils, let's start with the simple libxl_domid_to_name and its counterpart libxl_name_to_domid.

NB that C.GoString() will return "" if it's passed a NULL; see
https://github.com/golang/go/issues/32734#issuecomment-506835432

---
v3:
- Wire into build system
- Add reference to C.GoString() handling NULL to commit message

Quoted 7 lines
Quoted 1 line

Nicolas, could you test to see if this actually works for you?

Tested it, it works.

I must confess I do not use that import path as the new modules mechanism introduced in Go1.11 downloads and compile a versioned copy of every dependency per project, and this behavior is incompatible with the build system used here.

It’s possible that something fundamentally has changed, but I suspect that rather you don’t quite understand how the current build system is supposed to work. (In which case a write-up in the tree would probably be useful.)

Quoted 9 lines
Quoted 3 lines

Go has always insisted that there be no binary compatibility between versions; so it’s always been necessary to re-compile all your libraries when upgrading from (say) 1.8 to 1.9. Which means that any useable distribution must also include all the source files necessary to recompile when you bump the version number.

So the core mechanism of the “install” is actually to copy all the source files necessary into the right local directory such that the go compiler can find them; ATM this is /usr/share/gocode/golang.xenproject.org/xenlight

Nit: This of course should have a src/ between gocode/ and golang.xenproject.org/.

NB also that this naming scheme was designed so that at some point in the future, we could actually host the xenlight packages at the URL provided.

-George

This new mechanism of modules is described here:
https://golang.org/cmd/go/#hdr-Modules__module_versions__and_more

The module system is intended to supersede the GOPATH approach and provide a way to get versioned dependencies, as such
it does not rely on GOPATH at all and doesn't use sources or compiled packages present in GOPATH elements such as /usr/share/gocode and systematically fetch (at the asked version) and compile a copy of the dependency as it might be a different version from the one in GOPATH.

Quoted 8 lines
Quoted 6 lines

As far as I tried, I have been unable to build my module even with the library installed.
I have to use xenbits.xen.org/git-http/xen.git/tools/golang/xenlight (or one of its mirror) in order to build the module using the new mechanism (the golang.xenproject.org/xenlight works when building with modules mode disabled).

I took a look at the module stuff when it came out, and I was never able to make sense of how it was supposed to work.

Basically it is the same idea than a python virtualenv with |include-system-site-packages set to false: never use what is provided by the system and download everything in the exact version the manifest tells you to.
|

Quoted 3 lines

<rant>On the whole, it seems they basically hate the idea of distro packages, and seem intent on breaking them whenever people manage to start to get them working.</rant>

Actually yes because they don't want to be bound to the version provided by the distro (I will not enter the debate of whether it is a good thing or not)

Nicolas

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

#7 lore raw
Quoted 71 lines · Nicolas Belouin
Quoted 55 lines · George Dunlap
Quoted 54 lines · Nicolas Belouin
Quoted 32 lines · George Dunlap
Quoted 32 lines · George Dunlap
Quoted 22 lines · Nicolas Belouin
Quoted 14 lines · George Dunlap

The Go bindings for libxl miss functions from libxl_utils, let's start with the simple libxl_domid_to_name and its counterpart libxl_name_to_domid.

NB that C.GoString() will return "" if it's passed a NULL; see
https://github.com/golang/go/issues/32734#issuecomment-506835432

---
v3:
- Wire into build system
- Add reference to C.GoString() handling NULL to commit message

Quoted 7 lines
Quoted 1 line

Nicolas, could you test to see if this actually works for you?

Tested it, it works.

I must confess I do not use that import path as the new modules mechanism introduced in Go1.11 downloads and compile a versioned copy of every dependency per project, and this behavior is incompatible with the build system used here.

It’s possible that something fundamentally has changed, but I suspect that rather you don’t quite understand how the current build system is supposed to work. (In which case a write-up in the tree would probably be useful.)

Quoted 9 lines
Quoted 3 lines

Go has always insisted that there be no binary compatibility between versions; so it’s always been necessary to re-compile all your libraries when upgrading from (say) 1.8 to 1.9. Which means that any useable distribution must also include all the source files necessary to recompile when you bump the version number.

So the core mechanism of the “install” is actually to copy all the source files necessary into the right local directory such that the go compiler can find them; ATM this is /usr/share/gocode/golang.xenproject.org/xenlight

Nit: This of course should have a src/ between gocode/ and golang.xenproject.org/.

NB also that this naming scheme was designed so that at some point in the future, we could actually host the xenlight packages at the URL provided.

-George

This new mechanism of modules is described here:
https://golang.org/cmd/go/#hdr-Modules__module_versions__and_more

The module system is intended to supersede the GOPATH approach and provide a way to get versioned dependencies, as such
it does not rely on GOPATH at all and doesn't use sources or compiled packages present in GOPATH elements such as /usr/share/gocode and systematically fetch (at the asked version) and compile a copy of the dependency as it might be a different version from the one in GOPATH.

Quoted 8 lines
Quoted 6 lines

As far as I tried, I have been unable to build my module even with the library installed.
I have to use xenbits.xen.org/git-http/xen.git/tools/golang/xenlight (or one of its mirror) in order to build the module using the new mechanism (the golang.xenproject.org/xenlight works when building with modules mode disabled).

I took a look at the module stuff when it came out, and I was never able to make sense of how it was supposed to work.

Basically it is the same idea than a python virtualenv with |include-system-site-packages set to false: never use what is provided by the system and download everything in the exact version the manifest tells you to.
|

Quoted 6 lines
Quoted 3 lines

<rant>On the whole, it seems they basically hate the idea of distro packages, and seem intent on breaking them whenever people manage to start to get them working.</rant>

Actually yes because they don't want to be bound to the version provided by the distro (I will not enter the debate of whether it is a good thing or not)

If that's a requirement, the distro can provide multiple concurrent versions.

There are lots of places where build systems aren't allowed to access the internet. And distro packages provides lots of useful things:
discoverability, filtering (some level of review has been done to make sure this code us useful / safe), maintenance (local patches / fixes can be applied if upstream disappears), decentralization (code is still available even if upstream goes down / deletes his repositories).

I like Go as a language, but in this particular aspect, the core developers just seem to be insane.

-George

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

#8 lore raw
In reply to Nicolas Belouin
Quoted 58 lines · Nicolas Belouin
Quoted 55 lines · George Dunlap
Quoted 54 lines · Nicolas Belouin
Quoted 32 lines · George Dunlap
Quoted 32 lines · George Dunlap
Quoted 22 lines · Nicolas Belouin
Quoted 14 lines · George Dunlap

The Go bindings for libxl miss functions from libxl_utils, let's start with the simple libxl_domid_to_name and its counterpart libxl_name_to_domid.

NB that C.GoString() will return "" if it's passed a NULL; see
https://github.com/golang/go/issues/32734#issuecomment-506835432

---
v3:
- Wire into build system
- Add reference to C.GoString() handling NULL to commit message

Quoted 7 lines
Quoted 1 line

Nicolas, could you test to see if this actually works for you?

Tested it, it works.

I must confess I do not use that import path as the new modules mechanism introduced in Go1.11 downloads and compile a versioned copy of every dependency per project, and this behavior is incompatible with the build system used here.

It’s possible that something fundamentally has changed, but I suspect that rather you don’t quite understand how the current build system is supposed to work. (In which case a write-up in the tree would probably be useful.)

Quoted 9 lines
Quoted 3 lines

Go has always insisted that there be no binary compatibility between versions; so it’s always been necessary to re-compile all your libraries when upgrading from (say) 1.8 to 1.9. Which means that any useable distribution must also include all the source files necessary to recompile when you bump the version number.

So the core mechanism of the “install” is actually to copy all the source files necessary into the right local directory such that the go compiler can find them; ATM this is /usr/share/gocode/golang.xenproject.org/xenlight

Nit: This of course should have a src/ between gocode/ and golang.xenproject.org/.

NB also that this naming scheme was designed so that at some point in the future, we could actually host the xenlight packages at the URL provided.

-George

This new mechanism of modules is described here:
https://golang.org/cmd/go/#hdr-Modules__module_versions__and_more

The module system is intended to supersede the GOPATH approach and provide a way to get versioned dependencies, as such
it does not rely on GOPATH at all and doesn't use sources or compiled packages present in GOPATH elements such as /usr/share/gocode and systematically fetch (at the asked version) and compile a copy of the dependency as it might be a different version from the one in GOPATH.

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

As far as I tried, I have been unable to build my module even with the library installed.
I have to use xenbits.xen.org/git-http/xen.git/tools/golang/xenlight (or one of its mirror) in order to build the module using the new mechanism

This will break as soon as we have support in golang/xenlight for libxl features not in the version of Xen you're using.

E.g., suppose you're on Xen 4.12. Someone introduces a new feature in Xen 4.13, and plumbs it all the way from Xen, through libxl, *and* golang/xenlight. Now when *you* do a build, it will fail, because your version of golang will expect libxl features which your system doesn't have.

I had always planned on getting golang.xenproject.org set up such that it could interact with the "normal" go get thing. If you want to help us figure out how to get that set up, that would be helpful.

What would be *really* ideal is if we didn't have to link golang against one particular hypervisor. Maybe we need to use plugins?
https://golang.org/pkg/plugin/

-George

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

#9 lore raw
Quoted 62 lines · George Dunlap
Quoted 56 lines · Nicolas Belouin
Quoted 54 lines · George Dunlap
Quoted 53 lines · Nicolas Belouin
Quoted 32 lines · George Dunlap
Quoted 32 lines · George Dunlap
Quoted 22 lines · Nicolas Belouin
Quoted 14 lines · George Dunlap

The Go bindings for libxl miss functions from libxl_utils, let's start with the simple libxl_domid_to_name and its counterpart libxl_name_to_domid.

NB that C.GoString() will return "" if it's passed a NULL; see
https://github.com/golang/go/issues/32734#issuecomment-506835432

---
v3:
- Wire into build system
- Add reference to C.GoString() handling NULL to commit message

Quoted 7 lines
Quoted 1 line

Nicolas, could you test to see if this actually works for you?

Tested it, it works.

I must confess I do not use that import path as the new modules mechanism introduced in Go1.11 downloads and compile a versioned copy of every dependency per project, and this behavior is incompatible with the build system used here.

It’s possible that something fundamentally has changed, but I suspect that rather you don’t quite understand how the current build system is supposed to work. (In which case a write-up in the tree would probably be useful.)

Quoted 9 lines
Quoted 3 lines

Go has always insisted that there be no binary compatibility between versions; so it’s always been necessary to re-compile all your libraries when upgrading from (say) 1.8 to 1.9. Which means that any useable distribution must also include all the source files necessary to recompile when you bump the version number.

So the core mechanism of the “install” is actually to copy all the source files necessary into the right local directory such that the go compiler can find them; ATM this is /usr/share/gocode/golang.xenproject.org/xenlight

Nit: This of course should have a src/ between gocode/ and golang.xenproject.org/.

NB also that this naming scheme was designed so that at some point in the future, we could actually host the xenlight packages at the URL provided.

-George

This new mechanism of modules is described here:
https://golang.org/cmd/go/#hdr-Modules__module_versions__and_more

The module system is intended to supersede the GOPATH approach and provide a way to get versioned dependencies, as such
it does not rely on GOPATH at all and doesn't use sources or compiled packages present in GOPATH elements such as /usr/share/gocode and systematically fetch (at the asked version) and compile a copy of the dependency as it might be a different version from the one in GOPATH.

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

As far as I tried, I have been unable to build my module even with the library installed.
I have to use xenbits.xen.org/git-http/xen.git/tools/golang/xenlight (or one of its mirror) in order to build the module using the new mechanism

Quoted 7 lines

This will break as soon as we have support in golang/xenlight for libxl features not in the version of Xen you're using.

E.g., suppose you're on Xen 4.12. Someone introduces a new feature in Xen 4.13, and plumbs it all the way from Xen, through libxl, *and* golang/xenlight. Now when *you* do a build, it will fail, because your version of golang will expect libxl features which your system doesn't have.

I know of that, and that can be overcome using modules as you can specify a branch version of the module you depends on (e.g you can set your dependency as being xxx/[email protected]).

Quoted 4 lines

I had always planned on getting golang.xenproject.org set up such that it could interact with the "normal" go get thing. If you want to help us figure out how to get that set up, that would be helpful.

As far as I looked into vanity URLs, you can't serve a subdirectory of a repository directly, but you can trick the system using a go-proxy. To do that you need two things. First, you need
https://golang.xenproject.org/xenlight?go-get=1
<https://example.org/pkg/foo?go-get=1> to point to a page containing a <meta name="go-import" content="golang.xenproject.org mod
https://golang.xenproject.org/moduleproxy <https://code.org/moduleproxy>">
And have golang.xenproject.org/moduleproxy to follow the specifications of module proxies by proposing '.zip' files containing the different versions of the module. This part for sure can be scripted to do the hard work of dynamically packaging the versions from the git repository on demand.

Quoted 4 lines

What would be *really* ideal is if we didn't have to link golang against one particular hypervisor. Maybe we need to use plugins?
https://golang.org/pkg/plugin/

This is the Go equivalent of ldopen, I'm not fond of this approach in general, but something like this can indeed do the trick to keep from linking against libxl directly, it would need a split in two modules to be easily usable though.

Nicolas

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

#10 lore raw
Quoted 68 lines · Nicolas Belouin
Quoted 65 lines · George Dunlap
Quoted 56 lines · Nicolas Belouin
Quoted 54 lines · George Dunlap
Quoted 53 lines · Nicolas Belouin
Quoted 32 lines · George Dunlap
Quoted 32 lines · George Dunlap
Quoted 22 lines · Nicolas Belouin
Quoted 14 lines · George Dunlap

The Go bindings for libxl miss functions from libxl_utils, let's start with the simple libxl_domid_to_name and its counterpart libxl_name_to_domid.

NB that C.GoString() will return "" if it's passed a NULL; see
https://github.com/golang/go/issues/32734#issuecomment-506835432

---
v3:
- Wire into build system
- Add reference to C.GoString() handling NULL to commit message

Quoted 7 lines
Quoted 1 line

Nicolas, could you test to see if this actually works for you?

Tested it, it works.

I must confess I do not use that import path as the new modules mechanism introduced in Go1.11 downloads and compile a versioned copy of every dependency per project, and this behavior is incompatible with the build system used here.

It’s possible that something fundamentally has changed, but I suspect that rather you don’t quite understand how the current build system is supposed to work. (In which case a write-up in the tree would probably be useful.)

Quoted 9 lines
Quoted 3 lines

Go has always insisted that there be no binary compatibility between versions; so it’s always been necessary to re-compile all your libraries when upgrading from (say) 1.8 to 1.9. Which means that any useable distribution must also include all the source files necessary to recompile when you bump the version number.

So the core mechanism of the “install” is actually to copy all the source files necessary into the right local directory such that the go compiler can find them; ATM this is /usr/share/gocode/golang.xenproject.org/xenlight

Nit: This of course should have a src/ between gocode/ and golang.xenproject.org/.

NB also that this naming scheme was designed so that at some point in the future, we could actually host the xenlight packages at the URL provided.

-George

This new mechanism of modules is described here:
https://golang.org/cmd/go/#hdr-Modules__module_versions__and_more

The module system is intended to supersede the GOPATH approach and provide a way to get versioned dependencies, as such
it does not rely on GOPATH at all and doesn't use sources or compiled packages present in GOPATH elements such as /usr/share/gocode and systematically fetch (at the asked version) and compile a copy of the dependency as it might be a different version from the one in GOPATH.

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

As far as I tried, I have been unable to build my module even with the library installed.
I have to use xenbits.xen.org/git-http/xen.git/tools/golang/xenlight (or one of its mirror) in order to build the module using the new mechanism

This will break as soon as we have support in golang/xenlight for libxl features not in the version of Xen you're using.

Quoted 7 lines
Quoted 4 lines

E.g., suppose you're on Xen 4.12. Someone introduces a new feature in Xen 4.13, and plumbs it all the way from Xen, through libxl, *and* golang/xenlight. Now when *you* do a build, it will fail, because your version of golang will expect libxl features which your system doesn't have.

I know of that, and that can be overcome using modules as you can specify a branch version of the module you depends on (e.g you can set your dependency as being xxx/[email protected]).

That's not terrible I guess.

Quoted 4 lines
Quoted 3 lines

I had always planned on getting golang.xenproject.org set up such that it could interact with the "normal" go get thing. If you want to help us figure out how to get that set up, that would be helpful.

Quoted 12 lines

As far as I looked into vanity URLs, you can't serve a subdirectory of a repository directly, but you can trick the system using a go-proxy. To do that you need two things. First, you need
https://golang.xenproject.org/xenlight?go-get=1
<https://example.org/pkg/foo?go-get=1> to point to a page containing a <meta name="go-import" content="golang.xenproject.org mod
https://golang.xenproject.org/moduleproxy <https://code.org/moduleproxy>">
And have golang.xenproject.org/moduleproxy to follow the specifications of module proxies by proposing '.zip' files containing the different versions of the module. This part for sure can be scripted to do the hard work of dynamically packaging the versions from the git repository on demand.

Yes, that's sort of what I had in mind -- although it might be better to statically update the content with a crontab or something, rather than regenerating it from scratch every connection.

Quoted 8 lines
Quoted 3 lines

What would be *really* ideal is if we didn't have to link golang against one particular hypervisor. Maybe we need to use plugins?
https://golang.org/pkg/plugin/

This is the Go equivalent of ldopen, I'm not fond of this approach in general, but something like this can indeed do the trick to keep from linking against libxl directly, it would need a split in two modules to be easily usable though.

It's certainly a lot of work, but it seems a lot nicer than having to recompile your application every time you upgrade your hypervisor. :-)

But obviously all that requires work and testing, which requires someone who has enough time to make it happen.

-George

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

#11 lore raw
In reply to Nicolas Belouin
Quoted 17 lines · Nicolas Belouin
Quoted 14 lines · George Dunlap

The Go bindings for libxl miss functions from libxl_utils, let's start with the simple libxl_domid_to_name and its counterpart libxl_name_to_domid.

NB that C.GoString() will return "" if it's passed a NULL; see
https://github.com/golang/go/issues/32734#issuecomment-506835432

---
v3:
- Wire into build system
- Add reference to C.GoString() handling NULL to commit message

Quoted 7 lines
Quoted 1 line

Nicolas, could you test to see if this actually works for you?

Tested it, it works.

I must confess I do not use that import path as the new modules mechanism introduced in Go1.11 downloads and compile a versioned copy of every dependency per project, and this behavior is incompatible with the build system used here.

Also -- I can't check this in without somebody acking my changes; and you're the most obvious person to do so. :-)

-George

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

#12 lore raw
Quoted 20 lines · George Dunlap
Quoted 19 lines · Nicolas Belouin
Quoted 7 lines · George Dunlap

The Go bindings for libxl miss functions from libxl_utils, let's start with the simple libxl_domid_to_name and its counterpart libxl_name_to_domid.

NB that C.GoString() will return "" if it's passed a NULL; see
https://github.com/golang/go/issues/32734#issuecomment-506835432

Quoted 8 lines

---
v3:
- Wire into build system
- Add reference to C.GoString() handling NULL to commit message

Nicolas, could you test to see if this actually works for you?

Tested it, it works.

Quoted 8 lines
Quoted 4 lines

I must confess I do not use that import path as the new modules mechanism introduced in Go1.11 downloads and compile a versioned copy of every dependency per project, and this behavior is incompatible with the build system used here.

Also -- I can't check this in without somebody acking my changes; and you're the most obvious person to do so. :-)

-George

I successfully tested your changes so consider them as acked.

Nicolas

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel