DPDK: Proposal for a patch patch-test integration tree
This is a proposal to create a patch-test dpdk tree. I welcome feedback from the dpdk community on this proposal.
This tree will consist of pre-integrated patches submitted to dpdk-dev mailing list. The purpose of this tree is to help reduce the effort in reviewing patches and providing feedback to submitters of patches. Also integrators of dpdk can benefit by having a "linux-next" like tree to clone and or try new features for performance and breadth of hardware support. It is the hope that the patch-test tree will provide early feedback to patch reviewers as well as an assurance of quality to the project maintainer.
Work Flow and Process:
All patches will be taken from from public submissions to dpdk-dev.org scraped from dpdk patchwork. Patches will be applied to the patch-test tree and tested against HEAD as they are received. The feedback from the testing will be provided to the community. The patch-test tree will periodically be git pull'ed from dpdk.
Longer term goal:
Initially, the patches will be applied along with some simple smoke tests. The longer term goal is to automate this process, apply more extensive tests and post the results in dpdk patchwork, http://dpdk.org/dev/patchwork/project/dpdk/list/ which would have an accompanying mailing list for distribution of a results summary of the tests.
Submissions of test suites will be encouraged from the community including testing against compilers, architectures, devices, options etc. The purpose is to try to evolve to widen test coverage for submitted patches.
--Thomas Herbert
Quoted 8 lines · Thomas F Herbert
Work Flow and Process:
All patches will be taken from from public submissions to dpdk-dev.org scraped from dpdk patchwork. Patches will be applied to the patch-test tree and tested against HEAD as they are received. The feedback from the testing will be provided to the community. The patch-test tree will periodically be git pull'ed from dpdk.
Quoted 8 lines
Longer term goal:
Initially, the patches will be applied along with some simple smoke tests. The longer term goal is to automate this process, apply more extensive tests and post the results in dpdk patchwork,
http://dpdk.org/dev/patchwork/project/dpdk/list/ which would have an accompanying mailing list for distribution of a results summary of the tests.
thanks for helping.
It could be broken into two parts:
- patch-test-net-next,
http://dpdk.org/browse/dpdk/tree/MAINTAINERS#n190
http://dpdk.org/browse/dpdk/tree/drivers/net
- patch-test-other-next,
all, excepted drivers
Best regards,
Vincent
Quoted 20 lines · Vincent JARDIN
Quoted 8 lines · Thomas F Herbert
Work Flow and Process:
All patches will be taken from from public submissions to dpdk-dev.org scraped from dpdk patchwork. Patches will be applied to the patch-test tree and tested against HEAD as they are received. The feedback from the testing will be provided to the community. The patch-test tree will periodically be git pull'ed from dpdk.
Quoted 8 lines
Longer term goal:
Initially, the patches will be applied along with some simple smoke tests. The longer term goal is to automate this process, apply more extensive tests and post the results in dpdk patchwork,
http://dpdk.org/dev/patchwork/project/dpdk/list/ which would have an accompanying mailing list for distribution of a results summary of the tests.thanks for helping.
Quoted 7 lines
It could be broken into two parts:
- patch-test-net-next,
http://dpdk.org/browse/dpdk/tree/MAINTAINERS#n190
http://dpdk.org/browse/dpdk/tree/drivers/net- patch-test-other-next,
all, excepted drivers
Vincent, I don't understand what is in patch-test-next vs patch-test-other-next. Is patch-test-other-next to be for pmd drivers that are not included in the standard dpdk release?
Quoted 3 lines
Best regards,
Vincent
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 9 lines · Matthew Hall
Quoted 3 lines · Thomas F Herbert
Submissions of test suites will be encouraged from the community including testing against compilers, architectures, devices, options etc. The purpose is to try to evolve to widen test coverage for submitted patches.
Hi Thomas,
This overall idea is extremely awesome and will help me a lot so I won't have to try and sit there yanking in random patches from email threads.
Thanks!
In the meantime, you can yank patches from http://dpdk.org/dev/patchwork/project/dpdk/list/ which might be easier then sorting through the emails depending on your email client.
Quoted 4 lines
I am not sure if it will be possible or not, but I'd like to see some test suites against some applications as well. They do more complex operations in my experience than the operations the DPDK tests do.
I will accept submitted test suites toward this goal if you would like to write a test suite.
Quoted 6 lines
Perhaps something like an L4-L7 firewall app or an IDS / IPS app or an L3-L4 cloud switch / router app. Etc.
Matthew.
I spot a can of worms to be opened here :-)
Quoted 8 lines · Thomas F Herbert
Work Flow and Process:
All patches will be taken from from public submissions to dpdk-dev.org scraped from dpdk patchwork. Patches will be applied to the patch-test tree and tested against HEAD as they are received. The feedback from the testing will be provided to the community. The patch-test tree will periodically be git pull'ed from dpdk.
Quoted 8 lines
Longer term goal:
Initially, the patches will be applied along with some simple smoke tests. The longer term goal is to automate this process, apply more extensive tests and post the results in dpdk patchwork,
http://dpdk.org/dev/patchwork/project/dpdk/list/ which would have an accompanying mailing list for distribution of a results summary of the tests.
Actually, github and services such as travis-ci and coveralls already provide this functionality (with very little setup). So when someone sends a pull request, the continuous integration service travis-ci will notice it, and start a build and (possibly) run a test suite on the code
- with the patches applied.
If code coverage is collected in the process [1], it's uploaded to the coveralls site. Both travis-ci and coveralls will add a note to the pull request saying something like "Build failed with this patch, be careful" or "Build OK, everything is fine" and "Coverage decreased with 5% with this patch" etc etc.
Of course, github provides an API so it's entirely possible to add your own continuous integration support with the same functionality as travis-ci (customized for DPDK).
So before venturing into implementing something like this, I think the DPDK project should at least consider the existing alternatives.
And with that, I close the can of worms again. I hope no worms were hurt in the process! :-)
// Simon
[1] https://github.com/SimonKagstrom/kcov - yes my personal project
Quoted 32 lines · Simon Kågström
I spot a can of worms to be opened here :-)
Quoted 8 lines · Thomas F Herbert
Work Flow and Process:
All patches will be taken from from public submissions to dpdk-dev.org scraped from dpdk patchwork. Patches will be applied to the patch-test tree and tested against HEAD as they are received. The feedback from the testing will be provided to the community. The patch-test tree will periodically be git pull'ed from dpdk.
Quoted 8 lines
Longer term goal:
Initially, the patches will be applied along with some simple smoke tests. The longer term goal is to automate this process, apply more extensive tests and post the results in dpdk patchwork,
http://dpdk.org/dev/patchwork/project/dpdk/list/ which would have an accompanying mailing list for distribution of a results summary of the tests.Actually, github and services such as travis-ci and coveralls already provide this functionality (with very little setup). So when someone sends a pull request, the continuous integration service travis-ci will notice it, and start a build and (possibly) run a test suite on the code
- with the patches applied.If code coverage is collected in the process [1], it's uploaded to the coveralls site. Both travis-ci and coveralls will add a note to the pull request saying something like "Build failed with this patch, be careful" or "Build OK, everything is fine" and "Coverage decreased with 5% with this patch" etc etc.
Quoted 8 lines
Of course, github provides an API so it's entirely possible to add your own continuous integration support with the same functionality as travis-ci (customized for DPDK).
So before venturing into implementing something like this, I think the DPDK project should at least consider the existing alternatives.
Using the above process seems very reasonable and we already have github account we can mirror the code and run these processes. This gives us a more standard method instead of a home grown method, but as Simon states we could enhance the process via the API.
I may have bruised a few worms here, so very sorry :-)
Quoted 7 lines
And with that, I close the can of worms again. I hope no worms were hurt in the process! :-)
// Simon
[1] https://github.com/SimonKagstrom/kcov - yes my personal project
2015-05-27 16:48, Thomas F Herbert:
Quoted 11 lines
This is a proposal to create a patch-test dpdk tree. I welcome feedback from the dpdk community on this proposal.
This tree will consist of pre-integrated patches submitted to dpdk-dev mailing list. The purpose of this tree is to help reduce the effort in reviewing patches and providing feedback to submitters of patches. Also integrators of dpdk can benefit by having a "linux-next" like tree to clone and or try new features for performance and breadth of hardware support. It is the hope that the patch-test tree will provide early feedback to patch reviewers as well as an assurance of quality to the project maintainer.
Thanks for the proposal. This tree may be hosted here:
http://dpdk.org/browse
You just have to give the name of the repository and a SSH key.
When it will be done, we could discuss which tests should be passed.
A message that is not in the archive (sent off-list or not yet received): [email protected]
2015-05-28 10:39, Thomas F Herbert:
Quoted 14 lines · Thomas Monjalon
Quoted 13 lines
2015-05-27 16:48, Thomas F Herbert:
Quoted 11 lines
This is a proposal to create a patch-test dpdk tree. I welcome feedback from the dpdk community on this proposal.
This tree will consist of pre-integrated patches submitted to dpdk-dev mailing list. The purpose of this tree is to help reduce the effort in reviewing patches and providing feedback to submitters of patches. Also integrators of dpdk can benefit by having a "linux-next" like tree to clone and or try new features for performance and breadth of hardware support. It is the hope that the patch-test tree will provide early feedback to patch reviewers as well as an assurance of quality to the project maintainer.
Quoted 5 lines
Quoted 3 lines
Thanks for the proposal. This tree may be hosted here:
http://dpdk.org/browse
You just have to give the name of the repository and a SSH key.Shall we call the repo patch-test-next?
Why not dpdk-next?
With this description? Apply and test every patches before acceptance
Quoted 1 line
I am attaching two public keys to this email
Received, thanks
Quoted 15 lines · Thomas Monjalon
2015-05-28 10:39, Thomas F Herbert:
Quoted 14 lines · Thomas Monjalon
Quoted 13 lines
2015-05-27 16:48, Thomas F Herbert:
Quoted 11 lines
This is a proposal to create a patch-test dpdk tree. I welcome feedback from the dpdk community on this proposal.
This tree will consist of pre-integrated patches submitted to dpdk-dev mailing list. The purpose of this tree is to help reduce the effort in reviewing patches and providing feedback to submitters of patches. Also integrators of dpdk can benefit by having a "linux-next" like tree to clone and or try new features for performance and breadth of hardware support. It is the hope that the patch-test tree will provide early feedback to patch reviewers as well as an assurance of quality to the project maintainer.
Quoted 8 lines
Quoted 5 lines
Quoted 3 lines
Thanks for the proposal. This tree may be hosted here:
http://dpdk.org/browse
You just have to give the name of the repository and a SSH key.Shall we call the repo patch-test-next?
Why not dpdk-next?
With this description? Apply and test every patches before acceptance
Sounds good to me.
Quoted 5 lines
Quoted 2 lines
I am attaching two public keys to this email
Received, thanks