Skip to contentSkip to conversation

kunit: split resource API from test.h into new resource.h

PATCH Superseded Daniel Latypov 3 participants 7 messages Mar 16, 2022 – Mar 25, 2022

Subsystems (4)

v1 of 3 Superseded Latest is v3 →

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 kunit: split resource API from test.h into new resource.h

Background:
Currently, a reader looking at kunit/test.h will find the file is quite long, and the first meaty comment is a doc comment about struct kunit_resource.

Most users will not ever use the KUnit resource API directly. They'll use kunit_kmalloc() and friends, or decide it's simpler to do cleanups via labels (it often can be) instead of figuring out how to use the API.

It's also logically separate from everything else in test.h. Removing it from the file doesn't cause any compilation errors (since struct kunit has struct list_head resources to store them).

This commit:
Let's move it into a kunit/resource.h file and give it a separate page in the docs, kunit/api/resource.rst.

We include resource.h at the bottom of test.h since
* don't want to force existing users to add a new include if they use the API
* it accesses lock inside struct kunit in a inline func
* so we can't just forward declare, and the alternatives require uninlining the func, adding hepers to lock/unlock, or other more invasive changes.

Now the first big comment in test.h is about kunit_case, which is a lot more relevant to what a new user wants to know.

A side effect of this is git blame won't properly track history by default, users need to run

$ git blame -L ,1 -C17 include/kunit/resource.h

---

NOTE: this file doesn't split out code from test.c to a new resource.c file.
I'm primarily concerned with users trying to read the headers, so I didn't think messing up git blame (w/ default settings) was worth it. But I can make that change if it feels appropriate (it might also be messier).

---
 Documentation/dev-tools/kunit/api/index.rst   |   5 +
 .../dev-tools/kunit/api/resource.rst          |  13 +
 include/kunit/resource.h                      | 319 ++++++++++++++++++
 include/kunit/test.h                          | 301 +----------------
 4 files changed, 339 insertions(+), 299 deletions(-)
 create mode 100644 Documentation/dev-tools/kunit/api/resource.rst
 create mode 100644 include/kunit/resource.h
Documentation/dev-tools/kunit/api/index.rst +5 -0
index 3006cadcf44a..45ce04823f9f 100644
@@ -6,6 +6,7 @@ API Reference
 .. toctree::
 
 	test
+	resource
 
 This section documents the KUnit kernel testing API. It is divided into the
 following sections:
@@ -13,3 +14,7 @@ following sections:
 Documentation/dev-tools/kunit/api/test.rst
 
  - documents all of the standard testing API
+
+Documentation/dev-tools/kunit/api/resource.rst
+
+ - documents the KUnit resource API
Documentation/dev-tools/kunit/api/resource.rst new file +13 -0
new file mode 100644
index 000000000000..0a94f831259e
@@ -0,0 +1,13 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+============
+Resource API
+============
+
+This file documents the KUnit resource API.
+
+Most users won't need to use this API directly, power users can use it to store
+state on a per-test basis, register custom cleanup actions, and more.
+
+.. kernel-doc:: include/kunit/resource.h
+   :internal:
include/kunit/resource.h new file +319 -0

319 lines not on this page · raw message

include/kunit/test.h +2 -299

317 lines not on this page · raw message

base-commit: 09688c0166e76ce2fb85e86b9d99be8b0084cdf9

Signature · 1 line
2.35.1.723.g4982287a31-goog
Reviewed-by on v1 1/1
#1 lore raw
Quoted 11 lines · Daniel Latypov

Background:
Currently, a reader looking at kunit/test.h will find the file is quite long, and the first meaty comment is a doc comment about struct kunit_resource.

Most users will not ever use the KUnit resource API directly. They'll use kunit_kmalloc() and friends, or decide it's simpler to do cleanups via labels (it often can be) instead of figuring out how to use the API.

A depressing (but probably not untrue) thought. I think that, even if most people were to use the resource API, having it in test.h makes it harder, as having the resource functions separate makes it easier to understand as well.

Quoted 7 lines

It's also logically separate from everything else in test.h. Removing it from the file doesn't cause any compilation errors (since struct kunit has struct list_head resources to store them).

This commit:
Let's move it into a kunit/resource.h file and give it a separate page in the docs, kunit/api/resource.rst.

Yay! This makes a lot of sense to me, as I've wasted a lot of time scrolling through test.h.

Quoted 7 lines

We include resource.h at the bottom of test.h since
* don't want to force existing users to add a new include if they use the API
* it accesses lock inside struct kunit in a inline func
* so we can't just forward declare, and the alternatives require uninlining the func, adding hepers to lock/unlock, or other more invasive changes.

I don't like this, but still think it's an improvement on what we have now. Ultimately, I think adding helpers to lock/unlock or similar and making users include this separately is probably the right thing to do, as nesting the headers like this is a bit ugly, but I won't lose sleep over leaving it till later.

Quoted 7 lines

Now the first big comment in test.h is about kunit_case, which is a lot more relevant to what a new user wants to know.

A side effect of this is git blame won't properly track history by default, users need to run

$ git blame -L ,1 -C17 include/kunit/resource.h

This is a pain, but is probably worth it. Thanks for including the command in the commit message, which should mitigate it slightly.

Quoted 3 lines

---

This was starting to annoy me, too, as it was a pain to read through everything in test.h. It'll be a bit of short-term pain, merge-conflict wise if we have other changes to the resource system (which I fear is likely), but is worth it.

-- David

Quoted 7 lines

NOTE: this file doesn't split out code from test.c to a new resource.c file.
I'm primarily concerned with users trying to read the headers, so I didn't think messing up git blame (w/ default settings) was worth it. But I can make that change if it feels appropriate (it might also be messier).

Personally, I think it's probably worth splitting this out as well. And the sooner we do it, the less history we'll obscure. :-)

But I agree, it's less of an issue as it only directly affects people working on KUnit itself. Though making it easier for users to find and read the implementation of these functions could help them understand API "gotchas", so I think it's worthwhile.

Quoted 10 lines
---
 Documentation/dev-tools/kunit/api/index.rst   |   5 +
 .../dev-tools/kunit/api/resource.rst          |  13 +
 include/kunit/resource.h                      | 319 ++++++++++++++++++
 include/kunit/test.h                          | 301 +----------------
 4 files changed, 339 insertions(+), 299 deletions(-)
 create mode 100644 Documentation/dev-tools/kunit/api/resource.rst
 create mode 100644 include/kunit/resource.h

<...snip...>

#2 lore raw
Quoted 8 lines · David Gow
Quoted 6 lines · Daniel Latypov

Background:
Currently, a reader looking at kunit/test.h will find the file is quite long, and the first meaty comment is a doc comment about struct kunit_resource.

Quoted 7 lines
Quoted 5 lines

Most users will not ever use the KUnit resource API directly. They'll use kunit_kmalloc() and friends, or decide it's simpler to do cleanups via labels (it often can be) instead of figuring out how to use the API.

A depressing (but probably not untrue) thought. I think that, even if

I'm not sure it's that depressing.
Without some compiler support (e.g. GCC's cleanup), I can see there being a number of one-off things that don't warrant formalizing into a resource.

More detail:
It works OK when there's one pointer parameter, e.g. [1], but I feel like you'd normally need to capture at least one more local variable. So then you need to define a new struct to hold all the values, which is where I'd draw the line personally.

[1] https://elixir.bootlin.com/linux/v5.17-rc8/source/lib/kunit/executor_test.c#L182

Quoted 23 lines

most people were to use the resource API, having it in test.h makes it harder, as having the resource functions separate makes it easier to understand as well.

Quoted 7 lines

It's also logically separate from everything else in test.h. Removing it from the file doesn't cause any compilation errors (since struct kunit has struct list_head resources to store them).

This commit:
Let's move it into a kunit/resource.h file and give it a separate page in the docs, kunit/api/resource.rst.

Yay! This makes a lot of sense to me, as I've wasted a lot of time scrolling through test.h.

Quoted 7 lines

We include resource.h at the bottom of test.h since
* don't want to force existing users to add a new include if they use the API
* it accesses lock inside struct kunit in a inline func
* so we can't just forward declare, and the alternatives require uninlining the func, adding hepers to lock/unlock, or other more invasive changes.

Quoted 2 lines

I don't like this, but still think it's an improvement on what we have now. Ultimately, I think adding helpers to lock/unlock or similar and

Yes, I can see us maybe needing this in the future.
Right now, outside of test.c, there's only one callsite for each (in resource.h).

Quoted 3 lines

making users include this separately is probably the right thing to do, as nesting the headers like this is a bit ugly, but I won't lose sleep over leaving it till later.

Ack, I can add a TODO to indicate we want to clean this up?
It's a bit annoying right now, but it'll only get more annoying in the future.

Quoted 33 lines
Quoted 8 lines

Now the first big comment in test.h is about kunit_case, which is a lot more relevant to what a new user wants to know.

A side effect of this is git blame won't properly track history by default, users need to run

$ git blame -L ,1 -C17 include/kunit/resource.h

This is a pain, but is probably worth it. Thanks for including the command in the commit message, which should mitigate it slightly.

Quoted 3 lines

---

This was starting to annoy me, too, as it was a pain to read through everything in test.h. It'll be a bit of short-term pain, merge-conflict wise if we have other changes to the resource system (which I fear is likely), but is worth it.

-- David

Quoted 7 lines

NOTE: this file doesn't split out code from test.c to a new resource.c file.
I'm primarily concerned with users trying to read the headers, so I didn't think messing up git blame (w/ default settings) was worth it. But I can make that change if it feels appropriate (it might also be messier).

Quoted 2 lines

Personally, I think it's probably worth splitting this out as well. And the sooner we do it, the less history we'll obscure. :-)

Yeah, that was my thought.
But if you think this would help users, then I think we have a case to make this change.

Should I send a v2 with resource.c split out?
Brendan (and any others who have an opinion), what's your preference?

Quoted 17 lines

But I agree, it's less of an issue as it only directly affects people working on KUnit itself. Though making it easier for users to find and read the implementation of these functions could help them understand API "gotchas", so I think it's worthwhile.

Quoted 10 lines
---
 Documentation/dev-tools/kunit/api/index.rst   |   5 +
 .../dev-tools/kunit/api/resource.rst          |  13 +
 include/kunit/resource.h                      | 319 ++++++++++++++++++
 include/kunit/test.h                          | 301 +----------------
 4 files changed, 339 insertions(+), 299 deletions(-)
 create mode 100644 Documentation/dev-tools/kunit/api/resource.rst
 create mode 100644 include/kunit/resource.h

<...snip...>

#3 lore raw
Quoted 16 lines · Daniel Latypov
Quoted 14 lines · David Gow
Quoted 11 lines · Daniel Latypov

Background:
Currently, a reader looking at kunit/test.h will find the file is quite long, and the first meaty comment is a doc comment about struct kunit_resource.

Most users will not ever use the KUnit resource API directly. They'll use kunit_kmalloc() and friends, or decide it's simpler to do cleanups via labels (it often can be) instead of figuring out how to use the API.

Quoted 6 lines
Quoted 1 line

A depressing (but probably not untrue) thought. I think that, even if

I'm not sure it's that depressing.
Without some compiler support (e.g. GCC's cleanup), I can see there being a number of one-off things that don't warrant formalizing into a resource.

True, though I do think that the resources API could use a bit of polish to reduce the friction involved in figuring out how to use the API.
(And this patch is a good start!)

Quoted 38 lines

More detail:
It works OK when there's one pointer parameter, e.g. [1], but I feel like you'd normally need to capture at least one more local variable. So then you need to define a new struct to hold all the values, which is where I'd draw the line personally.

[1] https://elixir.bootlin.com/linux/v5.17-rc8/source/lib/kunit/executor_test.c#L182

Quoted 23 lines

most people were to use the resource API, having it in test.h makes it harder, as having the resource functions separate makes it easier to understand as well.

Quoted 7 lines

It's also logically separate from everything else in test.h. Removing it from the file doesn't cause any compilation errors (since struct kunit has struct list_head resources to store them).

This commit:
Let's move it into a kunit/resource.h file and give it a separate page in the docs, kunit/api/resource.rst.

Yay! This makes a lot of sense to me, as I've wasted a lot of time scrolling through test.h.

Quoted 7 lines

We include resource.h at the bottom of test.h since
* don't want to force existing users to add a new include if they use the API
* it accesses lock inside struct kunit in a inline func
* so we can't just forward declare, and the alternatives require uninlining the func, adding hepers to lock/unlock, or other more invasive changes.

Quoted 2 lines

I don't like this, but still think it's an improvement on what we have now. Ultimately, I think adding helpers to lock/unlock or similar and

Yes, I can see us maybe needing this in the future.
Right now, outside of test.c, there's only one callsite for each (in resource.h).

Quoted 6 lines
Quoted 3 lines

making users include this separately is probably the right thing to do, as nesting the headers like this is a bit ugly, but I won't lose sleep over leaving it till later.

Ack, I can add a TODO to indicate we want to clean this up?
It's a bit annoying right now, but it'll only get more annoying in the future.

Yeah, let's get this in largely as-is first, then we can start adding direct includes of "resource.h" where necessary before making it required.

Quoted 36 lines
Quoted 33 lines
Quoted 8 lines

Now the first big comment in test.h is about kunit_case, which is a lot more relevant to what a new user wants to know.

A side effect of this is git blame won't properly track history by default, users need to run

$ git blame -L ,1 -C17 include/kunit/resource.h

This is a pain, but is probably worth it. Thanks for including the command in the commit message, which should mitigate it slightly.

Quoted 3 lines

---

This was starting to annoy me, too, as it was a pain to read through everything in test.h. It'll be a bit of short-term pain, merge-conflict wise if we have other changes to the resource system (which I fear is likely), but is worth it.

-- David

Quoted 7 lines

NOTE: this file doesn't split out code from test.c to a new resource.c file.
I'm primarily concerned with users trying to read the headers, so I didn't think messing up git blame (w/ default settings) was worth it. But I can make that change if it feels appropriate (it might also be messier).

Quoted 2 lines

Personally, I think it's probably worth splitting this out as well. And the sooner we do it, the less history we'll obscure. :-)

Quoted 6 lines

Yeah, that was my thought.
But if you think this would help users, then I think we have a case to make this change.

Should I send a v2 with resource.c split out?
Brendan (and any others who have an opinion), what's your preference?

I think it's a separate enough thing that this patch could go in as-is, and resource.c could be split in a separate one if you preferred. But doing it in a v2 is fine as well.

Quoted 17 lines
Quoted 17 lines

But I agree, it's less of an issue as it only directly affects people working on KUnit itself. Though making it easier for users to find and read the implementation of these functions could help them understand API "gotchas", so I think it's worthwhile.

Quoted 10 lines
---
 Documentation/dev-tools/kunit/api/index.rst   |   5 +
 .../dev-tools/kunit/api/resource.rst          |  13 +
 include/kunit/resource.h                      | 319 ++++++++++++++++++
 include/kunit/test.h                          | 301 +----------------
 4 files changed, 339 insertions(+), 299 deletions(-)
 create mode 100644 Documentation/dev-tools/kunit/api/resource.rst
 create mode 100644 include/kunit/resource.h

<...snip...>

#4 lore raw
In reply to Daniel Latypov
Quoted 54 lines · Daniel Latypov' via KUnit Development
Quoted 8 lines · David Gow
Quoted 6 lines · Daniel Latypov

Background:
Currently, a reader looking at kunit/test.h will find the file is quite long, and the first meaty comment is a doc comment about struct kunit_resource.

Quoted 7 lines
Quoted 5 lines

Most users will not ever use the KUnit resource API directly. They'll use kunit_kmalloc() and friends, or decide it's simpler to do cleanups via labels (it often can be) instead of figuring out how to use the API.

A depressing (but probably not untrue) thought. I think that, even if

I'm not sure it's that depressing.
Without some compiler support (e.g. GCC's cleanup), I can see there being a number of one-off things that don't warrant formalizing into a resource.

More detail:
It works OK when there's one pointer parameter, e.g. [1], but I feel like you'd normally need to capture at least one more local variable. So then you need to define a new struct to hold all the values, which is where I'd draw the line personally.

[1] https://elixir.bootlin.com/linux/v5.17-rc8/source/lib/kunit/executor_test.c#L182

Quoted 23 lines

most people were to use the resource API, having it in test.h makes it harder, as having the resource functions separate makes it easier to understand as well.

Quoted 7 lines

It's also logically separate from everything else in test.h. Removing it from the file doesn't cause any compilation errors (since struct kunit has struct list_head resources to store them).

This commit:
Let's move it into a kunit/resource.h file and give it a separate page in the docs, kunit/api/resource.rst.

Yay! This makes a lot of sense to me, as I've wasted a lot of time scrolling through test.h.

Quoted 7 lines

We include resource.h at the bottom of test.h since
* don't want to force existing users to add a new include if they use the API
* it accesses lock inside struct kunit in a inline func
* so we can't just forward declare, and the alternatives require uninlining the func, adding hepers to lock/unlock, or other more invasive changes.

Quoted 6 lines
Quoted 2 lines

I don't like this, but still think it's an improvement on what we have now. Ultimately, I think adding helpers to lock/unlock or similar and

Yes, I can see us maybe needing this in the future.
Right now, outside of test.c, there's only one callsite for each (in resource.h).

Another alternative workaround is to split even more stuff out into other header files.

Personally I would prefer not to wrap the lock/unlock functions; I like seeing the kind of locking that's happening. Plus, such a helper would be pretty gross:

void kunit_lock(struct kunit *test, unsigned long* flags) {...}

It wouldn't actually clean up the call site, just facilitate breaking out code into a header.

Quoted 5 lines
Quoted 3 lines

making users include this separately is probably the right thing to do, as nesting the headers like this is a bit ugly, but I won't lose sleep over leaving it till later.

Ack, I can add a TODO to indicate we want to clean this up?

I am fine with this.

Quoted 38 lines

It's a bit annoying right now, but it'll only get more annoying in the future.

Quoted 33 lines
Quoted 8 lines

Now the first big comment in test.h is about kunit_case, which is a lot more relevant to what a new user wants to know.

A side effect of this is git blame won't properly track history by default, users need to run

$ git blame -L ,1 -C17 include/kunit/resource.h

This is a pain, but is probably worth it. Thanks for including the command in the commit message, which should mitigate it slightly.

Quoted 3 lines

---

This was starting to annoy me, too, as it was a pain to read through everything in test.h. It'll be a bit of short-term pain, merge-conflict wise if we have other changes to the resource system (which I fear is likely), but is worth it.

-- David

Quoted 7 lines

NOTE: this file doesn't split out code from test.c to a new resource.c file.
I'm primarily concerned with users trying to read the headers, so I didn't think messing up git blame (w/ default settings) was worth it. But I can make that change if it feels appropriate (it might also be messier).

Quoted 2 lines

Personally, I think it's probably worth splitting this out as well. And the sooner we do it, the less history we'll obscure. :-)

Quoted 6 lines

Yeah, that was my thought.
But if you think this would help users, then I think we have a case to make this change.

Should I send a v2 with resource.c split out?
Brendan (and any others who have an opinion), what's your preference?

I personally don't think test.c is so huge that it is a problem to understand, but I can see it getting there.

If it's going to happen, sooner is probably better.

Quoted 17 lines
Quoted 17 lines

But I agree, it's less of an issue as it only directly affects people working on KUnit itself. Though making it easier for users to find and read the implementation of these functions could help them understand API "gotchas", so I think it's worthwhile.

Quoted 10 lines
---
 Documentation/dev-tools/kunit/api/index.rst   |   5 +
 .../dev-tools/kunit/api/resource.rst          |  13 +
 include/kunit/resource.h                      | 319 ++++++++++++++++++
 include/kunit/test.h                          | 301 +----------------
 4 files changed, 339 insertions(+), 299 deletions(-)
 create mode 100644 Documentation/dev-tools/kunit/api/resource.rst
 create mode 100644 include/kunit/resource.h

<...snip...>

#5 lore raw

On Wed, Mar 23, 2022 at 6:51 PM Brendan Higgins
<[email protected]> wrote:

<snip>

Quoted 9 lines

Another alternative workaround is to split even more stuff out into other header files.

Personally I would prefer not to wrap the lock/unlock functions; I like seeing the kind of locking that's happening. Plus, such a helper would be pretty gross:

void kunit_lock(struct kunit *test, unsigned long* flags) {...}

That's exactly why I didn't bother to try and wrap it, yeah.

Quoted 11 lines

It wouldn't actually clean up the call site, just facilitate breaking out code into a header.

Quoted 5 lines
Quoted 3 lines

making users include this separately is probably the right thing to do, as nesting the headers like this is a bit ugly, but I won't lose sleep over leaving it till later.

Ack, I can add a TODO to indicate we want to clean this up?

I am fine with this.

To clarify, are you saying you're fine w/ the nested header as-is, or fine with it + a TODO?

Quoted 43 lines
Quoted 43 lines

It's a bit annoying right now, but it'll only get more annoying in the future.

Quoted 33 lines
Quoted 8 lines

Now the first big comment in test.h is about kunit_case, which is a lot more relevant to what a new user wants to know.

A side effect of this is git blame won't properly track history by default, users need to run

$ git blame -L ,1 -C17 include/kunit/resource.h

This is a pain, but is probably worth it. Thanks for including the command in the commit message, which should mitigate it slightly.

Quoted 3 lines

---

This was starting to annoy me, too, as it was a pain to read through everything in test.h. It'll be a bit of short-term pain, merge-conflict wise if we have other changes to the resource system (which I fear is likely), but is worth it.

-- David

Quoted 7 lines

NOTE: this file doesn't split out code from test.c to a new resource.c file.
I'm primarily concerned with users trying to read the headers, so I didn't think messing up git blame (w/ default settings) was worth it. But I can make that change if it feels appropriate (it might also be messier).

Quoted 2 lines

Personally, I think it's probably worth splitting this out as well. And the sooner we do it, the less history we'll obscure. :-)

Yeah, that was my thought.
But if you think this would help users, then I think we have a case to make this change.

Quoted 7 lines
Quoted 2 lines

Should I send a v2 with resource.c split out?
Brendan (and any others who have an opinion), what's your preference?

I personally don't think test.c is so huge that it is a problem to understand, but I can see it getting there.

If it's going to happen, sooner is probably better.

Ack.
I can work on adding a second patch on to this series that splits out resource.c?

That causes more churn for the other in-flight patches, but we already have some since David has some changes in test.h.

Reviewed-by on v1 1/1
#6 lore raw
In reply to Daniel Latypov
Quoted 26 lines · Daniel Latypov

Background:
Currently, a reader looking at kunit/test.h will find the file is quite long, and the first meaty comment is a doc comment about struct kunit_resource.

Most users will not ever use the KUnit resource API directly. They'll use kunit_kmalloc() and friends, or decide it's simpler to do cleanups via labels (it often can be) instead of figuring out how to use the API.

It's also logically separate from everything else in test.h. Removing it from the file doesn't cause any compilation errors (since struct kunit has struct list_head resources to store them).

This commit:
Let's move it into a kunit/resource.h file and give it a separate page in the docs, kunit/api/resource.rst.

We include resource.h at the bottom of test.h since
* don't want to force existing users to add a new include if they use the API
* it accesses lock inside struct kunit in a inline func
* so we can't just forward declare, and the alternatives require uninlining the func, adding hepers to lock/unlock, or other more invasive changes.

Quoted 8 lines

Now the first big comment in test.h is about kunit_case, which is a lot more relevant to what a new user wants to know.

A side effect of this is git blame won't properly track history by default, users need to run

$ git blame -L ,1 -C17 include/kunit/resource.h

Aside from the discussion below,