About ftc requests
Hi Tim,
There are several functions in ftc associated to what I think is the prototype of a fuego web app that you showed us during an ELC BoF session.
- put-request
- put-test
- list-requests
- run-request
- package-run
- put-run
Are there any news or roadmap for this effort?
I am currently working on the integration of Fuego with Squad (and KernelCI) so I might touch some code there. Squad supports very similar functionality and can use LAVA to run the requests. However, for now I think that supporting put-run (publishing the results) will be enough for most Fuego users. What do you think?.
Thanks,
Daniel
Quoted 15 lines
-----Original Message-----
From: Daniel Sangorrin
Hi Tim,There are several functions in ftc associated to what I think is the prototype of a fuego web app that you showed us during an ELC BoF session.
- put-request
- put-test
- list-requests
- run-request
- package-run
- put-run
Quoted 1 line
Are there any news or roadmap for this effort?
Unfortunately, it's been on my backburner. The main reason it got stalled was that I wanted to add security for all these transfers, but that turns it into a much bigger project to get this working.
What we have now is actually very close to sufficient, for an insecure (unencrypted, unauthenticated) transfer of materials.
There's a bug right now on the server with handling json with Unicode (non-ASCII) characters, but it shouldn't be too hard to fix that.
The end goal is to automate some of this, but with very little effort (IMHO), we do things like the following workflow manually:
1) create a test
2) put the test on the server (ftc put-test)
3) add a request to run the test on another person's board (ftc put-request)
we would need a list of host:boards that were available,
but to start with we could just know each other's host and board names.
4) notify the person to run the test (e.g. by e-mail or slack). They would do:
5) ftc run-request
6) ftc put-run (maybe make it an option to do this automatically for runs based on remote requests)
7) the original user can download the run (we don't have ftc get-run yet)
8) the original user can analyze the results, looking at the logs, collateral material, etc.
This would allow anyone in the Fuego "network" to run a test on another person's board. For example, Wang could update his busybox test, and make requests for the tests to run on boards in my lab, so he can test how the test works on more than just his board. I could test new tests on boards in multiple labs (yours, Liu's, Wang's, Khiem's, Dhinakar's), which would help a lot with test development and quality.
One thing I'd like to discuss at the Fuego Jamboree is whether someone is willing to be the test subject for this, and actually try it out soon after the meeting (the following week).
Quoted 7 lines
I am currently working on the integration of Fuego with Squad (and KernelCI) so I might touch some code there.
Squad supports very similar functionality and can use LAVA to run the requests. However, for now I think that
supporting put-run (publishing the results) will be enough for most Fuego users. What do you think?.
I have no problem with an alternate implementation of put-run. Mine is very simple - basically just creating a tar file and sending it to a server. Since mine's not being actively used, you can do what you want. Just sideline my code so if I get serious about this and restart the work, the code is still there to integrate. That is, don't delete the code, but just put it in a dead routine or something. We could have the code autodetect what type of server it's talking to, but to start with we could just have a flag indicating it's a kernelci, squad, or fuego server. Or, we could add a configuration item indicating the server type in the fuego.conf file.
We can talk more about this next week.
-- Tim
Quoted 20 lines
-----Original Message-----
From: [email protected] [mailto:[email protected]]Quoted 13 lines
-----Original Message-----
From: Daniel Sangorrin
Hi Tim,There are several functions in ftc associated to what I think is the prototype of a fuego web app that you showed us during an ELC BoF session.
- put-request
- put-test
- list-requests
- run-request
- package-run
- put-runQuoted 1 line
Are there any news or roadmap for this effort?
Unfortunately, it's been on my backburner. The main reason it got stalled was that I wanted to add security for all these transfers, but that turns it into a much bigger project to get this working.
Quoted 5 lines
What we have now is actually very close to sufficient, for an insecure (unencrypted, unauthenticated) transfer of materials.
There's a bug right now on the server with handling json with Unicode (non-ASCII) characters, but it shouldn't be too hard to fix that.
Is this code public already?
I think the standard method for authentication is to provide a token in the headers. Of course, this should be passed over https when run remotely. KernelCI and Squad already have support for it.
Quoted 15 lines
The end goal is to automate some of this, but with very little effort (IMHO), we do things like the following workflow manually:
1) create a test
2) put the test on the server (ftc put-test)
3) add a request to run the test on another person's board (ftc put-request) we would need a list of host:boards that were available,
but to start with we could just know each other's host and board names.
4) notify the person to run the test (e.g. by e-mail or slack). They would do:
5) ftc run-request
6) ftc put-run (maybe make it an option to do this automatically for runs based on remote requests)
7) the original user can download the run (we don't have ftc get-run yet)
8) the original user can analyze the results, looking at the logs, collateral material, etc.
I think we should separate visualization from request management and run them as two separated "micro" services. This way we can substitute the implementation of each micro service and we can install only the visualization service locally.
When working with a Visualization service (kernelci, squad, how about "vision" for fuego's?) a user would use functions such as. ftc put-run <-- with ability to notify by e-mail in case it was a requested run
ftc get-run
When working with the Request management a user would use functions such as:
ftc put-board <-- announce that people can test on my board
ftc rm-board <-- remove the board from the fuego network
ftc list-boards <-- add a --remote flag to list also remote boards?
ftc put-request <-- puts the request. I think ftc put-test is not strictly needed because we can pass the test's git repo url or standard fuego test name on the request. ftc put-test/get-test would be convenient for uploading tarballs though to reduce storage needs. ftc get-request <-- get next request (can be used for polling also)
ftc run-request <-- downloads (and clears) request and executes ftc run-test
[Note] how about using push/pull instead of put/get?
[Note] A lab would run ftc get-request, ftc run-request, ftc put-run automatically using a simple polling script.
Quoted 2 lines
This would allow anyone in the Fuego "network" to run a test on another person's board.
About "network": Squad supports the concept of teams/groups and group user permissions. Having something similar would be nice. We also need to be flexible in regards of how we schedule the requests.
For example, we should be able to put requests like:
"run this on a bbb board, I don't care which one"
"run this on a board with a cortex-a9, I don't care which one"
"run this on a board with architecture powerpc, I don't care which one"
Quoted 8 lines
For example, Wang could update his busybox test, and make requests for the tests to
run on boards in my lab, so he can test how the test works on more than just his board.
I could test new tests on boards in multiple labs (yours, Liu's, Wang's, Khiem's, Dhinakar's),
which would help a lot with test development and quality.
Quoted 4 lines
One thing I'd like to discuss at the Fuego Jamboree is whether someone is willing to be the test subject for this, and actually try it out soon after the meeting (the following
week).
I volunteer.
Quoted 18 lines
Quoted 6 lines
I am currently working on the integration of Fuego with Squad (and KernelCI) so I might touch some code there.
Squad supports very similar functionality and can use LAVA to run the requests. However, for now I think that
supporting put-run (publishing the results) will be enough for most Fuego users. What do you think?.I have no problem with an alternate implementation of put-run. Mine is very simple - basically just creating a tar file and sending it to a server. Since mine's not being actively used, you can do what you want. Just sideline my code
so if I get serious about this and restart the work, the code is still there to integrate. That is, don't delete the code, but just put it in a dead routine or something. We could have the code autodetect what type of server it's talking to, but to start with we could just have a flag indicating it's a kernelci, squad, or fuego server. Or, we could add a configuration item indicating the server type in the fuego.conf file.
Quoted 1 line
We can talk more about this next week.
OK, thanks.
Daniel
Quoted 20 lines · Daniel Sangorrin
Quoted 20 lines
-----Original Message-----
From: [email protected] [mailto:[email protected]]Quoted 13 lines
-----Original Message-----
From: Daniel Sangorrin
Hi Tim,There are several functions in ftc associated to what I think is the prototype of a fuego web app that you showed us during an ELC BoF session.
- put-request
- put-test
- list-requests
- run-request
- package-run
- put-runQuoted 1 line
Are there any news or roadmap for this effort?
Unfortunately, it's been on my backburner. The main reason it got stalled was that I wanted to add security for all these transfers, but that turns it into a much bigger project to get this working.
Quoted 7 lines
Quoted 5 lines
What we have now is actually very close to sufficient, for an insecure (unencrypted, unauthenticated) transfer of materials.
There's a bug right now on the server with handling json with Unicode (non-ASCII) characters, but it shouldn't be too hard to fix that.
Is this code public already?
No. I need to do that. This is based on my tbwiki software, which has heretofore been unpublished. But I should do that, and add it to the Fuego project.
Quoted 2 lines
I think the standard method for authentication is to provide a token in the headers. Of course, this should be passed over https when run remotely. KernelCI and Squad already have support for it.
Yes, but the way kernelCI is doing it is quite insecure, if I recall correctly. I was going to use a similar approach, but using different cryptography.
Quoted 17 lines
Quoted 4 lines
The end goal is to automate some of this, but with very little effort (IMHO), we do things like the following workflow manually:
Quoted 12 lines
1) create a test
2) put the test on the server (ftc put-test)
3) add a request to run the test on another person's board (ftc put-request) we would need a list of host:boards that were available,
but to start with we could just know each other's host and board names.
4) notify the person to run the test (e.g. by e-mail or slack). They would do:
5) ftc run-request
6) ftc put-run (maybe make it an option to do this automatically for runs based on remote requests)
7) the original user can download the run (we don't have ftc get-run yet)
8) the original user can analyze the results, looking at the logs, collateral material, etc.
Quoted 1 line
I think we should separate visualization from request management and run them as two separated "micro" services. This way we can substitute the implementation of each micro service and we can install only the visualization service locally.
I agree with this.
Quoted 4 lines
When working with a Visualization service (kernelci, squad, how about "vision" for fuego's?) a user would use functions such as. ftc put-run <-- with ability to notify by e-mail in case it was a requested run
ftc get-run
I'm not sure I understand the 'get-run'. do kernelci and squad allow you to download data? I thought they were upload-only?
Quoted 6 lines
When working with the Request management a user would use functions such as:
ftc put-board <-- announce that people can test on my board
ftc rm-board <-- remove the board from the fuego network
ftc list-boards <-- add a --remote flag to list also remote boards?
ftc put-request <-- puts the request. I think ftc put-test is not strictly needed because we can pass the test's git repo url or standard fuego test name on the request. ftc put-test/get-test would be convenient for uploading tarballs though to reduce storage needs.
I agree that is the test is in the fuego-core repository, no put-test should be needed.
But having put-test allows someone to make an ad-hoc or under-development test, and use
it in the test network (put-test, then put-request), without having to have it in the fuego-core repo.
One use case I want to support is:
- user finds bug
- user notifies upstream developer
- upstream developer needs more data, or wants to try something, so
writes a test
- upstream developer does: put-test, put-request
- user does: run-request, put-run
- upstream developer does: get-run
- upstream developer uses run data to find the bug, possibly
repeating if more data is needed,or
the test needs to be refined.
Quoted 2 lines
ftc get-request <-- get next request (can be used for polling also) ftc run-request <-- downloads (and clears) request and executes ftc run-test
Yes.
Quoted 1 line
[Note] how about using push/pull instead of put/get?
I prefer get/put.
Quoted 1 line
[Note] A lab would run ftc get-request, ftc run-request, ftc put-run automatically using a simple polling script.
Agreed.
Quoted 10 lines
Quoted 2 lines
This would allow anyone in the Fuego "network" to run a test on another person's board.
About "network": Squad supports the concept of teams/groups and group user permissions. Having something similar would be nice. We also need to be flexible in regards of how we schedule the requests.
For example, we should be able to put requests like:
"run this on a bbb board, I don't care which one"
"run this on a board with a cortex-a9, I don't care which one"
"run this on a board with architecture powerpc, I don't care which one"
Yes. And boards should list their attributes, and maybe include lots of info, including their kernel config
(with put-board). So we could also do:
"run this on a board with a synposis USB phy"
"run this on a board with a CAN bus"
"run this on a board with an nvidia GPU"
and
"run this on all boards with ethernet chipset X"
and pick up data from all the runs.
and also describe things external to the board itself, that require special hardware setups:
"run this in a lab that has audio test (capture through external cable) capability"
"run this in a lab that has off-board power measurement capability"
Quoted 8 lines
Quoted 8 lines
For example, Wang could update his busybox test, and make requests for the tests to
run on boards in my lab, so he can test how the test works on more than just his board.
I could test new tests on boards in multiple labs (yours, Liu's, Wang's, Khiem's, Dhinakar's),
which would help a lot with test development and quality.
Quoted 6 lines
Quoted 4 lines
One thing I'd like to discuss at the Fuego Jamboree is whether someone is willing to be the test subject for this, and actually try it out soon after the meeting (the following
week).I volunteer.
Great! Let's talk.
-- Tim Bird
Senior Staff Software Engineer, Sony Corporation Architecture Group Chair, Core Embedded Linux Project, Linux Foundation