perf record: adapt NUMA awareness to machines with #CPUs > 1K
Subsystems (2)
cover perf record: adapt NUMA awareness to machines with #CPUs > 1K
Current implementation of cpu_set_t type by glibc has internal cpu mask size limitation of no more than 1024 CPUs. This limitation confines NUMA awareness of Perf tool in record mode, thru --affinity option, to the first 1024 CPUs on machines with larger amount of CPUs.
This patch set enables Perf tool to overcome 1024 CPUs limitation by using a dedicated struct mmap_cpu_mask type and applying tool's bitmap API operations to manipulate affinity masks of the tool's thread and the mmaped data buffers.
tools bitmap API has been extended with bitmap_equal() operation and its implementation is derived from the kernel one.
---
Alexey Budankov (3):
tools bitmap: extend bitmap API with bitmap_equal() perf mmap: declare type for cpu mask of arbitrary length perf record: adapt affinity to machines with #CPUs > 1K
tools/include/linux/bitmap.h | 21 +++++++++++++++++++++ tools/lib/bitmap.c | 15 +++++++++++++++ tools/perf/builtin-record.c | 28 ++++++++++++++++++++++------ tools/perf/util/mmap.c | 28 ++++++++++++++++++++++------ tools/perf/util/mmap.h | 11 ++++++++++- 5 files changed, 90 insertions(+), 13 deletions(-)
---
Testing:
$ tools/perf/perf record -v --affinity=cpu -- ls thread mask[8]: empty Using CPUID GenuineIntel-6-5E-3 ... mmap size 528384B 0x7f95f8f85010: mmap mask[8]: 0 0x7f95f8f950d8: mmap mask[8]: 1 0x7f95f8fa51a0: mmap mask[8]: 2 0x7f95f8fb5268: mmap mask[8]: 3 0x7f95f8fc5330: mmap mask[8]: 4 0x7f95f8fd53f8: mmap mask[8]: 5 0x7f95f8fe54c0: mmap mask[8]: 6 0x7f95f8ff5588: mmap mask[8]: 7 ... thread mask[8]: 0 thread mask[8]: 1 thread mask[8]: 2 thread mask[8]: 3 arch copy Documentation init kernel MAINTAINERS modules.builtin.modinfo perf.data scripts System.map vmlinux block COPYING drivers ipc lbuild Makefile modules.order perf.data.old security tools vmlinux.o certs CREDITS fs Kbuild lib mm Module.symvers README sound usr config-5.2.7-100.fc29.x86_64 crypto include Kconfig LICENSES modules.builtin net samples stdio virt thread mask[8]: 4 thread mask[8]: 5 thread mask[8]: 6 thread mask[8]: 7 thread mask[8]: 0 thread mask[8]: 1 thread mask[8]: 2 thread mask[8]: 3 thread mask[8]: 4 thread mask[8]: 5 thread mask[8]: 6 thread mask[8]: 7 [ perf record: Woken up 0 times to write data ] thread mask[8]: 0 thread mask[8]: 1 thread mask[8]: 2 thread mask[8]: 3 thread mask[8]: 4 thread mask[8]: 5 thread mask[8]: 6 thread mask[8]: 7 ... [ perf record: Captured and wrote 0.014 MB perf.data (11 samples) ]
Signature · 1 line
2.20.1
PATCH 1/3 tools bitmap: implement bitmap_equal() operation at bitmap API
Extend tools bitmap API with bitmap_equal() implementation. The implementation has been derived from the kernel.
--- tools/include/linux/bitmap.h | 21 +++++++++++++++++++++ tools/lib/bitmap.c | 15 +++++++++++++++ 2 files changed, 36 insertions(+)
tools/include/linux/bitmap.h +21 -0
@@ -15,6 +15,8 @@ void __bitmap_or(unsigned long *dst, const unsigned long *bitmap1, const unsigned long *bitmap2, int bits);int __bitmap_and (unsigned long *dst,const unsigned long *bitmap1,const unsigned long *bitmap2,unsigned int bits); +int __bitmap_equal (const unsigned long *bitmap1, +const unsigned long *bitmap2,unsigned int bits);void bitmap_clear (unsigned long *map,unsigned int start,int len);#define BITMAP_FIRST_WORD_MASK (start) (~0UL << ((start) & (BITS_PER_LONG - 1)))
@@ -148,4 +150,23 @@ static inline int bitmap_and(unsigned long *dst, const unsigned long *src1, return __bitmap_and(dst, src1, src2, nbits); } +#ifdef __LITTLE_ENDIAN +#define BITMAP_MEM_ALIGNMENT 8 +#else +#define BITMAP_MEM_ALIGNMENT (8 * sizeof(unsigned long)) +#endif +#define BITMAP_MEM_MASK (BITMAP_MEM_ALIGNMENT - 1) +#define IS_ALIGNED (x, a) (((x) & ((typeof(x))(a) - 1)) == 0) + +static inline int bitmap_equal (const unsigned long *src1, +const unsigned long *src2,unsigned int nbits) +{ +if (small_const_nbits (nbits)) +return !((*src1 ^ *src2) &BITMAP_LAST_WORD_MASK (nbits)); +if (__builtin_constant_p (nbits &BITMAP_MEM_MASK ) && +IS_ALIGNED (nbits,BITMAP_MEM_ALIGNMENT )) +return !memcmp (src1, src2, nbits /8 ); +return __bitmap_equal (src1, src2, nbits); +} +#endif /* _PERF_BITOPS_H */
tools/lib/bitmap.c +15 -0
@@ -71,3 +71,18 @@ int __bitmap_and(unsigned long *dst, const unsigned long *bitmap1, BITMAP_LAST_WORD_MASK(bits));return result !=0 ; } + +int __bitmap_equal (const unsigned long *bitmap1, +const unsigned long *bitmap2,unsigned int bits) +{ +unsigned int k, lim = bits/BITS_PER_LONG ; +for (k =0 ; k < lim; ++k) +if (bitmap1[k] != bitmap2[k]) +return 0 ; + +if (bits %BITS_PER_LONG ) +if ((bitmap1[k] ^ bitmap2[k]) &BITMAP_LAST_WORD_MASK (bits)) +return 0 ; + +return 1 ; +}
Signature · 1 line
2.20.1
PATCH 2/3 perf mmap: declare type for cpu mask of arbitrary length
In reply to Alexey BudankovDeclare a dedicated struct map_cpu_mask type for cpu masks of arbitrary length. Mask is available thru bits pointer and the mask length is kept in nbits field. mmap_cpu_mask_bytes() macro returns mask storage size in bytes.
--- tools/perf/util/mmap.h | 9 +++++++++ 1 file changed, 9 insertions(+)
tools/perf/util/mmap.h +9 -0
@@ -15,6 +15,15 @@#include "event.h" struct aiocb ; + +struct mmap_cpu_mask { +unsigned long *bits; +size_t nbits; +}; + +#define mmap_cpu_mask_bytes (m) \ + (BITS_TO_LONGS(((struct mmap_cpu_mask *)m)->nbits) * sizeof(unsigned long)) +/** * struct mmap - perf's ring buffer mmap details *
Signature · 1 line
2.20.1
Re: [PATCH v1 2/3] perf mmap: declare type for cpu mask of arbitrary length
Quoted 16 lines · Alexey Budankov
Declare a dedicated struct
map_cpu_masktype for cpu masks of arbitrary length. Mask is available thru bits pointer and the mask length is kept in nbits field.mmap_cpu_mask_bytes()macro returns mask storage size in bytes.Signed-off-by Alexey Budankov <[email protected]>--- tools/perf/util/mmap.h | 9 +++++++++ 1 file changed, 9 insertions(+)tools/perf/util/mmap.h +0
-0
tools/perf/util/mmap.h line 24
Quoted 10 lines
tools/perf/util/mmap.h +7
-0@@ -15,6 +15,15 @@#include "event.h" struct aiocb ; + +struct mmap_cpu_mask { +unsigned long *bits; +size_t nbits; +}; + +#define mmap_cpu_mask_bytes (m) \
we try to have all macros upper case
tools/perf/util/mmap.h line 25
Quoted 1 line
tools/perf/util/mmap.h +1
-0+ (BITS_TO_LONGS (((struct mmap_cpu_mask *)m)->nbits) *sizeof (unsigned long ))
we have BITS_TO_BYTES
thanks,
jirka
Quoted 7 lines
tools/perf/util/mmap.h +1
-0+/** * struct mmap - perf's ring buffer mmap details * Signature · 1 line
2.20.1
Re: [PATCH v1 2/3] perf mmap: declare type for cpu mask of arbitrary length
Quoted 19 lines · Jiri Olsa
Quoted 18 lines · Alexey Budankov
Declare a dedicated struct
map_cpu_masktype for cpu masks of arbitrary length. Mask is available thru bits pointer and the mask length is kept in nbits field.mmap_cpu_mask_bytes()macro returns mask storage size in bytes.Signed-off-by Alexey Budankov <[email protected]>--- tools/perf/util/mmap.h | 9 +++++++++ 1 file changed, 9 insertions(+)tools/perf/util/mmap.h +0
-0@@ -15,6 +15,15 @@#include "event.h"
Quoted 10 lines
tools/perf/util/mmap.h line 24
Quoted 8 lines
tools/perf/util/mmap.h +7
-0struct aiocb ; + +struct mmap_cpu_mask { +unsigned long *bits; +size_t nbits; +}; + +#define mmap_cpu_mask_bytes (m) \we try to have all macros upper case
In v2.
Quoted 4 lines
tools/perf/util/mmap.h line 25
Quoted 2 lines
tools/perf/util/mmap.h +1
-0+ (BITS_TO_LONGS (((struct mmap_cpu_mask *)m)->nbits) *sizeof (unsigned long ))we have
BITS_TO_BYTES
In v2.
Quoted 4 lines
thanks,
jirka
~Alexey
Quoted 9 lines
Quoted 7 lines
tools/perf/util/mmap.h +1
-0+/** * struct mmap - perf's ring buffer mmap details * Signature · 1 line
2.20.1
Re: [PATCH v1 2/3] perf mmap: declare type for cpu mask of arbitrary length
Quoted 3 lines · Alexey Budankov
Quoted 2 lines · Jiri Olsa
Quoted 1 line · Alexey Budankov
<SNIP>
Quoted 4 lines
Quoted 3 lines
Quoted 1 line
+#define
mmap_cpu_mask_bytes(m)\we try to have all macros upper case
Quoted 8 lines
In v2.
Quoted 4 lines
tools/perf/util/mmap.h line 25
Quoted 2 lines
tools/perf/util/mmap.h +1
-0+ (BITS_TO_LONGS (((struct mmap_cpu_mask *)m)->nbits) *sizeof (unsigned long ))we have
BITS_TO_BYTESIn v2.
Avoided BITS_TO_BYTES() in this particular case because the storage is allocated in unsigned long chunks, thus the possibly unused tail bytes at the last chunk would be accounted in the returned size, and for compatibility with cpu_set_t which is also allocated in unsigned longs.
~Alexey
PATCH 3/3 perf record: adapt affinity to machines with #CPUs > 1K
In reply to Alexey BudankovUse struct mmap_cpu_mask type for tool's thread and mmap data buffers to overcome current 1024 CPUs mask size limitation of cpu_set_t type.
Currently glibc cpu_set_t type has internal mask size limit of 1024 CPUs. Moving to struct mmap_cpu_mask type allows overcoming that limit. tools bitmap API is used to manipulate objects of struct mmap_cpu_mask type.
--- tools/perf/builtin-record.c | 28 ++++++++++++++++++++++------ tools/perf/util/mmap.c | 28 ++++++++++++++++++++++------ tools/perf/util/mmap.h | 2 +- 3 files changed, 45 insertions(+), 13 deletions(-)
tools/perf/builtin-record.c +22 -6
@@ -62,6 +62,7 @@#include <linux/string.h> #include <linux/time64.h> #include <linux/zalloc.h> +#include <linux/bitmap.h> struct switch_output {bool enabled;
@@ -93,7 +94,7 @@ struct record {bool timestamp_boundary;struct switch_output switch_output;unsigned long long samples; -cpu_set_t affinity_mask; +struct mmap_cpu_mask affinity_mask;unsigned long output_max_size;/* = 0: unlimited */ };
@@ -951,13 +952,21 @@ static struct perf_event_header finished_round_event = { .type = PERF_RECORD_FINISHED_ROUND, }; +#define MASK_SIZE 1023static void record__adjust_affinity (struct record *rec,struct mmap *map) { +char mask[MASK_SIZE +1 ] = {0 }; +if (rec->opts.affinity !=PERF_AFFINITY_SYS && - !CPU_EQUAL (&rec->affinity_mask, &map->affinity_mask)) { -CPU_ZERO (&rec->affinity_mask); -CPU_OR (&rec->affinity_mask, &rec->affinity_mask, &map->affinity_mask); -sched_setaffinity (0 ,sizeof (rec->affinity_mask), &rec->affinity_mask); + !bitmap_equal (rec->affinity_mask.bits, map->affinity_mask.bits, + rec->affinity_mask.nbits)) { +bitmap_zero (rec->affinity_mask.bits, rec->affinity_mask.nbits); +bitmap_or (rec->affinity_mask.bits, rec->affinity_mask.bits, + map->affinity_mask.bits, rec->affinity_mask.nbits); +sched_setaffinity (0 ,mmap_cpu_mask_bytes (&rec->affinity_mask), + (cpu_set_t *)rec->affinity_mask.bits); +bitmap_scnprintf (rec->affinity_mask.bits, rec->affinity_mask.nbits, mask,MASK_SIZE ); +pr_debug ("thread mask[%ld]: %s\n" , rec->affinity_mask.nbits, mask); } }
@@ -2389,7 +2398,6 @@ int cmd_record(int argc, const char **argv)# undef REASON#endif -CPU_ZERO (&rec->affinity_mask); rec->opts.affinity =PERF_AFFINITY_SYS ; rec->evlist =evlist__new ();
@@ -2405,6 +2413,14 @@ int cmd_record(int argc, const char **argv)if (quiet)perf_quiet_option (); + rec->affinity_mask.nbits =cpu__max_cpu (); + rec->affinity_mask.bits =bitmap_alloc (rec->affinity_mask.nbits); +if (!rec->affinity_mask.bits) { +pr_err ("Failed to allocate thread mask for %ld cpus\n" , rec->affinity_mask.nbits); +return -ENOMEM ; + } +pr_debug ("thread mask[%ld]: empty\n" , rec->affinity_mask.nbits); +/* Make system wide (-a) the default target. */ if (!argc &&target__none (&rec->opts.target)) rec->opts.target.system_wide = true;
tools/perf/util/mmap.c +22 -6
@@ -23,6 +23,7 @@#include "mmap.h" #include "../perf.h" #include <internal/lib.h> /* page_size */ +#include <linux/bitmap.h> size_t mmap__mmap_len (struct mmap *map) {
@@ -215,7 +216,7 @@ void mmap__munmap(struct mmap *map)auxtrace_mmap__munmap (&map->auxtrace_mmap); } -static void build_node_mask (int node,cpu_set_t *mask) +static void build_node_mask (int node,struct mmap_cpu_mask *mask) {int c, cpu, nr_cpus;const struct perf_cpu_map *cpu_map =NULL ;
@@ -228,28 +229,43 @@ static void build_node_mask(int node, cpu_set_t *mask)for (c =0 ; c < nr_cpus; c++) { cpu = cpu_map->map[c];/* map c index to online cpu index */ if (cpu__get_node (cpu) == node) -CPU_SET (cpu, mask); +set_bit (cpu, mask->bits); } } -static void perf_mmap__setup_affinity_mask (struct mmap *map,struct mmap_params *mp) +static int perf_mmap__setup_affinity_mask (struct mmap *map,struct mmap_params *mp) { -CPU_ZERO (&map->affinity_mask); + map->affinity_mask.nbits =cpu__max_cpu (); + map->affinity_mask.bits =bitmap_alloc (map->affinity_mask.nbits); +if (!map->affinity_mask.bits) +return 1 ; +if (mp->affinity ==PERF_AFFINITY_NODE &&cpu__max_node () >1 )build_node_mask (cpu__get_node (map->core.cpu), &map->affinity_mask);else if (mp->affinity ==PERF_AFFINITY_CPU ) -CPU_SET (map->core.cpu, &map->affinity_mask); +set_bit (map->core.cpu, map->affinity_mask.bits); + +return 0 ; } +#define MASK_SIZE 1023int mmap__mmap (struct mmap *map,struct mmap_params *mp,int fd,int cpu) { +char mask[MASK_SIZE +1 ] = {0 }; +if (perf_mmap__mmap (&map->core, &mp->core, fd, cpu)) {pr_debug2 ("failed to mmap perf event ring buffer, error %d\n" , errno);return -1 ; } -perf_mmap__setup_affinity_mask (map, mp); +if (perf_mmap__setup_affinity_mask (map, mp)) { +pr_debug2 ("failed to alloc mmap affinity mask, error %d\n" , + errno); +return -1 ; + } +bitmap_scnprintf (map->affinity_mask.bits, map->affinity_mask.nbits, mask,MASK_SIZE ); +pr_debug ("%p: mmap mask[%ld]: %s\n" , map, map->affinity_mask.nbits, mask); map->core.flush = mp->flush;
tools/perf/util/mmap.h +1 -1
@@ -40,7 +40,7 @@ struct mmap { int nr_cblocks; } aio;#endif -cpu_set_t affinity_mask; +struct mmap_cpu_mask affinity_mask;void *data;int comp_level; };
Signature · 1 line
2.20.1
Re: [PATCH v1 3/3] perf record: adapt affinity to machines with #CPUs > 1K
On Wed, Nov 20, 2019 at 12:38:57PM +0300, Alexey Budankov wrote:
SNIP
Quoted 28 lines
tools/perf/util/mmap.c +3
-2@@ -23,6 +23,7 @@#include "mmap.h" #include "../perf.h" #include <internal/lib.h> /* page_size */ +#include <linux/bitmap.h> size_t mmap__mmap_len (struct mmap *map) {@@ -215,7 +216,7 @@ void mmap__munmap(struct mmap *map)auxtrace_mmap__munmap (&map->auxtrace_mmap); } -static void build_node_mask (int node,cpu_set_t *mask) +static void build_node_mask (int node,struct mmap_cpu_mask *mask) {int c, cpu, nr_cpus;const struct perf_cpu_map *cpu_map =NULL ;@@ -228,28 +229,43 @@ static void build_node_mask(int node, cpu_set_t *mask)for (c =0 ; c < nr_cpus; c++) { cpu = cpu_map->map[c];/* map c index to online cpu index */ if (cpu__get_node (cpu) == node) -CPU_SET (cpu, mask); +set_bit (cpu, mask->bits); }
tools/perf/util/mmap.c static void build_node_mask(int node, cpu_set_t *mask) · line 241
Quoted 10 lines
tools/perf/util/mmap.c +5
-2} -static void perf_mmap__setup_affinity_mask (struct mmap *map,struct mmap_params *mp) +static int perf_mmap__setup_affinity_mask (struct mmap *map,struct mmap_params *mp) { -CPU_ZERO (&map->affinity_mask); + map->affinity_mask.nbits =cpu__max_cpu (); + map->affinity_mask.bits =bitmap_alloc (map->affinity_mask.nbits); +if (!map->affinity_mask.bits) +return 1 ;
I guess this works, but please return < 0 on error
thanks,
jirka
Re: [PATCH v1 3/3] perf record: adapt affinity to machines with #CPUs > 1K
Quoted 5 lines · Jiri Olsa
On Wed, Nov 20, 2019 at 12:38:57PM +0300, Alexey Budankov wrote:
SNIP
Quoted 1 line
tools/perf/util/mmap.c +0
-0
<SNIP>
Quoted 10 lines
tools/perf/util/mmap.c static void build_node_mask(int node, cpu_set_t *mask) · line 241
Quoted 8 lines
tools/perf/util/mmap.c +5
-2-static void perf_mmap__setup_affinity_mask (struct mmap *map,struct mmap_params *mp) +static int perf_mmap__setup_affinity_mask (struct mmap *map,struct mmap_params *mp) { -CPU_ZERO (&map->affinity_mask); + map->affinity_mask.nbits =cpu__max_cpu (); + map->affinity_mask.bits =bitmap_alloc (map->affinity_mask.nbits); +if (!map->affinity_mask.bits) +return 1 ;I guess this works, but please return < 0 on error
In v2.
Quoted 5 lines
thanks,
jirka
~Alexey
Quoted 11 lines · Alexey Budankov
Current implementation of
cpu_set_ttype by glibc has internal cpu mask size limitation of no more than 1024 CPUs. This limitation confines NUMA awareness of Perf tool in record mode, thru --affinity option, to the first 1024 CPUs on machines with larger amount of CPUs.This patch set enables Perf tool to overcome 1024 CPUs limitation by using a dedicated struct
mmap_cpu_masktype and applying tool's bitmap API operations to manipulate affinity masks of the tool's thread and the mmaped data buffers.
Quoted 8 lines
tools bitmap API has been extended with
bitmap_equal()operation and its implementation is derived from the kernel one.---
Alexey Budankov (3):
tools bitmap: extend bitmap API withbitmap_equal()perf mmap: declare type for cpu mask of arbitrary length perf record: adapt affinity to machines with #CPUs > 1K
looks good to me, I sent some minor comments
Quoted 11 lines
tools/include/linux/bitmap.h | 21 +++++++++++++++++++++ tools/lib/bitmap.c | 15 +++++++++++++++ tools/perf/builtin-record.c | 28 ++++++++++++++++++++++------ tools/perf/util/mmap.c | 28 ++++++++++++++++++++++------ tools/perf/util/mmap.h | 11 ++++++++++- 5 files changed, 90 insertions(+), 13 deletions(-)---
Testing:
Quoted 13 lines
$ tools/perf/perf record -v --affinity=cpu -- ls thread mask[8]: empty Using CPUID GenuineIntel-6-5E-3 ... mmap size 528384B 0x7f95f8f85010: mmap mask[8]: 0 0x7f95f8f950d8: mmap mask[8]: 1 0x7f95f8fa51a0: mmap mask[8]: 2 0x7f95f8fb5268: mmap mask[8]: 3 0x7f95f8fc5330: mmap mask[8]: 4 0x7f95f8fd53f8: mmap mask[8]: 5 0x7f95f8fe54c0: mmap mask[8]: 6 0x7f95f8ff5588: mmap mask[8]: 7
could we add this to -vv? -v is poluted already
perhaps we should make some effort and try to consolidate -v output for some really basic verbose, the rest would be under -vv or specialized --debug variable .. not in scope of this patchset of course ;-)
thanks,
jirka
Quoted 48 lines · Jiri Olsa
Quoted 11 lines · Alexey Budankov
Current implementation of
cpu_set_ttype by glibc has internal cpu mask size limitation of no more than 1024 CPUs. This limitation confines NUMA awareness of Perf tool in record mode, thru --affinity option, to the first 1024 CPUs on machines with larger amount of CPUs.This patch set enables Perf tool to overcome 1024 CPUs limitation by using a dedicated struct
mmap_cpu_masktype and applying tool's bitmap API operations to manipulate affinity masks of the tool's thread and the mmaped data buffers.Quoted 8 lines
tools bitmap API has been extended with
bitmap_equal()operation and its implementation is derived from the kernel one.---
Alexey Budankov (3):
tools bitmap: extend bitmap API withbitmap_equal()perf mmap: declare type for cpu mask of arbitrary length perf record: adapt affinity to machines with #CPUs > 1Klooks good to me, I sent some minor comments
Quoted 11 lines
tools/include/linux/bitmap.h | 21 +++++++++++++++++++++ tools/lib/bitmap.c | 15 +++++++++++++++ tools/perf/builtin-record.c | 28 ++++++++++++++++++++++------ tools/perf/util/mmap.c | 28 ++++++++++++++++++++++------ tools/perf/util/mmap.h | 11 ++++++++++- 5 files changed, 90 insertions(+), 13 deletions(-)---
Testing:Quoted 13 lines
$ tools/perf/perf record -v --affinity=cpu -- ls thread mask[8]: empty Using CPUID GenuineIntel-6-5E-3 ... mmap size 528384B 0x7f95f8f85010: mmap mask[8]: 0 0x7f95f8f950d8: mmap mask[8]: 1 0x7f95f8fa51a0: mmap mask[8]: 2 0x7f95f8fb5268: mmap mask[8]: 3 0x7f95f8fc5330: mmap mask[8]: 4 0x7f95f8fd53f8: mmap mask[8]: 5 0x7f95f8fe54c0: mmap mask[8]: 6 0x7f95f8ff5588: mmap mask[8]: 7
Quoted 1 line
could we add this to -vv? -v is poluted already
In v2.
Thanks,
Alexey
Quoted 9 lines
perhaps we should make some effort and try to consolidate -v output for some really basic verbose, the rest would be under -vv or specialized --debug variable .. not in scope of this patchset of course ;-)
thanks,
jirka