Skip to contentSkip to conversation

GEM features into feat_profile.json

RFC Joonas Lahtinen 3 participants 5 messages May 31, 2017 – Jun 1, 2017

lore
Share & tools
lore
#0 lore raw

Hello,

I went through the gem_* tests from intel-gpu-tools and categorized them into roughly categories "X | X robustness | X performance" ready to be added to the feat_profile.json.

Lets open a discussion which ones should go where. I tried to place a single test to under only one category and I'm kind of hopeful that we'll have the ability to add "depends_on" to create super features in the future, instead of placing a single test under multiple categories.

I didn't check all the subtests nor wildcard matching with other tests, this is just all the test names placed under some categories.

Regards, Joonas

PS. I'm still bit unsure where to place gem_spin_batch.c while tests
gem_pipe_control_store_loop, gem_bad_address, gem_bad_batch,
gem_bad_blit, gem_bad_length and gem_bad_reloc seem more like hardware
sanity checks.
---
    "Driver API" : {
        "include_tests": "gem_basic",
    },
    "Driver API robustness" : {
        "include_tests": "gem_close_race|gem_fd_exhaustion|gem_eio",
    },
    "Basic IOCTLs": {
        "include_tests": "gem_create|gem_busy|gem_unref_active_buffers",
    },
    "R/W contention" : {
       "include_tests": "gem_gtt_cpu_tlb|gem_pread_after_blit|gem_pwrite_snooped|gem_readwrite|gem_caching|gem_exec_flush|gem_stress",
    },
    "R/W concurrency" : {
        "include_tests": "gem_concurrent_all|gem_concurrent_blit|gem_exec_parallel|gem_partial_pwrite_pread",
    },
    "Relocations" : {
        "include_tests": "gem_exec_reloc|gem_cpu_reloc|gem_persistent_relocs",
    },
    "Relocations robustness" : {
        "include_tests": "gem_exec_faulting_reloc|gem_exec_bad_domains|gem_reloc_overflow|gem_reloc_vs_gpu",
    },
    "Command streamer robustness" : {
        "include_tests": "gem_cs_tlb|gem_cs_prefetch",
    },
    "Contexts" : {
        "include_tests": "gem_ctx_basic|gem_ctx_exec|gem_ctx_param",
    },
    "Context robustness" : {
       "include_tests": "gem_ctx_create|gem_ctx_bad_destroy|gem_ctx_bad_exec|gem_ctx_switch|gem_ctx_thrash",
    },
    "Interrupts robustness" : {
       "include_tests": "gem_double_irq_loop|gem_ring_sync_loop|gem_sync",
    },
    "Eviction" : {
       "include_tests": "gem_evict|gem_exec_gttfill|gem_shrink",
    },
    "Execbuf IOCTL" : {
       "include_tests": "gem_exec_basic|gem_exec_alignment|gem_exec_lut_handle|gem_lut_handle",
    },
    "Execbuf IOCTL performance" : {
       "include_tests": "gem_exec_blt|gem_exec_create|gem_exec_latency|gem_exec_nop|gem_exec_reuse",
    },
    "Execbuf IOCTL robustness" : {
       "include_tests": "gem_exec_params|gem_fenced_exec_thrash|gem_exec_whisper",
    },
    "Suspend and resume" : {
       "include_tests": "gem_exec_suspend",
    },
    "Scheduler" : {
       "include_tests": "gem_exec_schedule",
    },
    "Asynchronous execution" : {
       "include_tests": "gem_exec_async",
    },
    "Command parser" : {
       "include_tests": "gem_exec_parse",
    },
    "Fences, implicit" : {
       "include_tests": "gem_exec_await",
    },
    "Fences, explicit" : {
       "include_tests": "gem_exec_fence",
    },
    "Tiling" : {
       "include_tests": "gem_fence_upload|gem_set_tiling_vs_blt|gem_set_tiling_vs_gtt|gem_set_tiling_vs_pwrite|gem_tiled_pread_basic|gem_tiled_pread_pwrite|gem_tiled_wb|gem_tiled_wc|gem_tiling_max_stride", },
    "Tiling robustness" : {
       "include_tests": "gem_fence_thrash|gem_gtt_hog|gem_render_tiled_blits|gem_threaded_access_tiled|gem_tiled_blits|gem_tiled_fence_blits|gem_tiled_swapping|gem_tiled_partial_pwrite_pread|gem_unfence_active_buffers", },
    "GGTT contention" : {
       "include_tests": "gem_gtt_hog|gem_linear_blits",
    },
    "Global objects" : {
       "include_tests": "gem_flink_basic",
    },
    "Global objects robustness" : {
       "include_tests": "gem_flink_race",
    },
    "Workloads" : {
       "include_tests": "gem_storedw_batches_loop|gem_storedw_loop",
    },
    "Workloads, GPGPU" : {
       "include_tests": "gem_gpgpu_fill",
    },
    "Workloads, media" : {
       "include_tests": "gem_media_fill",
    },
    "Workloads, render" : {
       "include_tests": "gem_render_copy",
    },
    "Workloads robustness, render" : {
       "include_tests": "gem_render_copy_redux|gem_render_linear_blits",
    },
    "Workloads robustness" : {
       "include_tests": "gem_ringfill",
    },
    "GGTT performance" : {
       "include_tests": "gem_gtt_speed",
    },
    "Hang recovery" : {
       "include_tests": "gem_hang|gem_hangcheck_forcewake|gem_exec_capture",
    },
    "Large object support" : {
       "include_tests": "gem_exec_big|gem_largeobject",
    },
    "Purging objects" : {
       "include_tests": "gem_madvise",
    },
    "mmap IOCTL" : {
       "include_tests": "gem_mmap|gem_mmap_gtt|gem_mmap_wc",
    },
    "mmap IOCTL robustness" : {
       "include_tests": "gem_mmap_offset_exhaustion",
    },
    "MOCS" : {
       "include_tests": "gem_mocs_settings",
    },
    "Secure batches robustness" : {
       "include_tests": "gem_non_secure_batch",
    },
    "Pinning" : {
       "include_tests": "gem_pin",
    },
    "PPGTT" : {
       "include_tests": "gem_ppgtt",
    },
    "R/W performance" : {
       "include_tests": "gem_pread|gem_pwrite|gem_pwrite_pread|gem_read_read_speed",
    },
    "Register read IOCTL" : {
       "include_tests": "gem_reg_read|gem_workarounds",
    },
    "Request retirement" : {
       "include_tests": "gem_request_retire",
    },
    "Reset statistics IOCTL" : {
       "include_tests": "gem_reset_stats",
    },
    "Sequence number wrapping" : {
       "include_tests": "gem_seqno_wrap",
    },
    "Soft pinning API" : {
       "include_tests": "gem_softpin",
    },
    "Stolen memory" : {
       "include_tests": "gem_stolen",
    },
    "Streaming writes" : {
       "include_tests": "gem_streaming_writes",
    },
    "User pointers API" : {
       "include_tests": "gem_userptr_blits",
    },
    "Wait IOCTL" : {
       "include_tests": "gem_wait",
    },
    "Wait IOCTL" : {
       "include_tests": "gem_wait",
    },
    "Semaphores robustness" : {
       "include_tests": "gem_write_read_ring_switch",
    }

Signature · 7 lines
Joonas Lahtinen
Open Source Technology Center
Intel Corporation
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#1 lore raw
Quoted 6 lines · Joonas Lahtinen

Hello,

I went through the gem_* tests from intel-gpu-tools and categorized them into roughly categories "X | X robustness | X performance" ready to be added to the feat_profile.json.

Quoted 7 lines

Lets open a discussion which ones should go where. I tried to place a single test to under only one category and I'm kind of hopeful that we'll have the ability to add "depends_on" to create super features in the future, instead of placing a single test under multiple categories.

I didn't check all the subtests nor wildcard matching with other tests, this is just all the test names placed under some categories.

You seem to have assigned them exclusively to one category or another, most tests belong to a few of these categories. More when you consider a subtest may be targetting a completely different aspect. -Chris

Signature · 5 lines
Chris Wilson, Intel Open Source Technology Centre
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#2 lore raw
Quoted 13 lines · Chris Wilson
Quoted 12 lines · Joonas Lahtinen

Hello,

I went through the gem_* tests from intel-gpu-tools and categorized them into roughly categories "X | X robustness | X performance" ready to be added to the feat_profile.json.

Lets open a discussion which ones should go where. I tried to place a single test to under only one category and I'm kind of hopeful that we'll have the ability to add "depends_on" to create super features in the future, instead of placing a single test under multiple categories.

Quoted 6 lines
Quoted 2 lines

I didn't check all the subtests nor wildcard matching with other tests, this is just all the test names placed under some categories.

You seem to have assigned them exclusively to one category or another, most tests belong to a few of these categories. More when you consider a subtest may be targetting a completely different aspect.

Yes, that's what I meant to say :) Subtests should probably be matched by another pattern like "\btiled\b", "\bflink\b" etc.

Ultimately there would be a resolver which would re-assign the subtests:

"Global objects" would then get:

"include_subtests": "flink",

Which would steal subtests with /\bflink\b/ from tests. Do we agree that one subtest would be assigned to one category only, or do you want to see duplication even at that level?

Regards, Joonas
--
Joonas Lahtinen
Open Source Technology Center
Intel Corporation
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

#3 lore raw
Quoted 27 lines · Joonas Lahtinen
Quoted 13 lines · Chris Wilson
Quoted 12 lines · Joonas Lahtinen

Hello,

I went through the gem_* tests from intel-gpu-tools and categorized them into roughly categories "X | X robustness | X performance" ready to be added to the feat_profile.json.

Lets open a discussion which ones should go where. I tried to place a single test to under only one category and I'm kind of hopeful that we'll have the ability to add "depends_on" to create super features in the future, instead of placing a single test under multiple categories.

Quoted 6 lines
Quoted 2 lines

I didn't check all the subtests nor wildcard matching with other tests, this is just all the test names placed under some categories.

You seem to have assigned them exclusively to one category or another, most tests belong to a few of these categories. More when you consider a subtest may be targetting a completely different aspect.

Yes, that's what I meant to say :) Subtests should probably be matched by another pattern like "\btiled\b", "\bflink\b" etc.

Ultimately there would be a resolver which would re-assign the subtests:

Quoted 7 lines

"Global objects" would then get:

"include_subtests": "flink",

Which would steal subtests with /\bflink\b/ from tests. Do we agree that one subtest would be assigned to one category only, or do you want to see duplication even at that level?

I see duplication everywhere. It's more a concept of tags as opposed to categories.

The use of such a system would be as
"give me all the tests that exercise relocation"
"give me all the tests that use a context"
"give me all the tests that exercise contention on $mutex" "give me all the tests that exercise file.c:line / this patch set"

The last one especially.
-Chris

Signature · 5 lines
Chris Wilson, Intel Open Source Technology Centre
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#4 lore raw
Quoted 36 lines · Chris Wilson
Quoted 27 lines · Joonas Lahtinen
Quoted 13 lines · Chris Wilson
Quoted 12 lines · Joonas Lahtinen

Hello,

I went through the gem_* tests from intel-gpu-tools and categorized them into roughly categories "X | X robustness | X performance" ready to be added to the feat_profile.json.

Lets open a discussion which ones should go where. I tried to place a single test to under only one category and I'm kind of hopeful that we'll have the ability to add "depends_on" to create super features in the future, instead of placing a single test under multiple categories.

Quoted 6 lines
Quoted 2 lines

I didn't check all the subtests nor wildcard matching with other tests, this is just all the test names placed under some categories.

You seem to have assigned them exclusively to one category or another, most tests belong to a few of these categories. More when you consider a subtest may be targetting a completely different aspect.

Yes, that's what I meant to say :) Subtests should probably be matched by another pattern like "\btiled\b", "\bflink\b" etc.

Ultimately there would be a resolver which would re-assign the subtests:

Quoted 7 lines

"Global objects" would then get:

"include_subtests": "flink",

Which would steal subtests with /\bflink\b/ from tests. Do we agree that one subtest would be assigned to one category only, or do you want to see duplication even at that level?

Quoted 8 lines

I see duplication everywhere. It's more a concept of tags as opposed to categories.

The use of such a system would be as
"give me all the tests that exercise relocation"
"give me all the tests that use a context"
"give me all the tests that exercise contention on $mutex" "give me all the tests that exercise file.c:line / this patch set"

file/line is probably out of scope for this, for that we'd need to have coverage information for each testcase, and then we could map a diff to the relevant set of testcases. Nifty, but probably an awful lot of work to automate ...

Quoted 1 line

The last one especially.

Yeah that was kinda the idea of this, the entries wouldn't be only an exclusive list of tests only (piglit does that grouping already), but match patterns for a given feature. And a given test can easily show up in multiple features (e.g. a testcase that tests gpu hangs vs. suspend/resume obviously needs to be both a hangcheck test and an s/r test).

I think we can just extend stuff though.

Anyway, thanks a lot for kickstarting this, unfortunately my vacations start in like 4h so can't really digg into this more before heading back.

Cheers, Daniel

Signature · 7 lines
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx