Skip to contentSkip to conversation

lttng cli: Accept human readable sizes for --subbuf-size

PATCH Christian Babeux 3 participants 5 messages Apr 3, 2013 4 messages not in the archive

lore
Share & tools
lore

A message that is not in the archive (sent off-list or not yet received): [email protected]

#1 lore raw

Hi Simon,

Quoted 2 lines

I wanted to add some unit tests for parse_human_size, but the current test system does not make it easy. They will come later.

Could you describe the issues you had on a separate thread so we can further discuss it?

Here a few comments on your patch:

Review

Quoted 8 lines
(quoted diff) +8 -0
+       char *buffer = malloc(length);
+
+       if (buffer) {
+               /* Get and print error message */
+               regerror(errcode, regex, buffer, length);
+               ERR("%s\n", buffer);
+               free(buffer);
+       }

What if malloc() fails? Printing an error message would be appropriate to handle that case.

Review

Quoted 2 lines
(quoted diff) +2 -0
+       /* Compile regex */
+       ret = regcomp(&regex, "^[0-9][0-9]*\\([kmg]\\{0,1\\}\\)$", REG_ICASE);

This regex could be simplified to: "^[:digit:]+\\([kmg]\\)?$"

Quoted 1 line

+ *size = atol(str);

atol() can return negative numbers and the result is stored in an unsigned (size_t). This could lead to enormous subbuffers allocated by accident. A check should be added to allow only positive numbers.

Also, atol() does not detect conversion errors, returning 0 in error cases [1]. Perhaps using strtol() would be more appropriate?

Quoted 7 lines
(quoted diff) +7 -0
+               /* Check if there is a suffix */
+               regmatch_t suffix_match = matches[1];
+               if (suffix_match.rm_eo - suffix_match.rm_so > 0) {
+                       switch (*(str + suffix_match.rm_so)) {
+                       case 'k':
+                       case 'K':
+                               *size = *size * 1024;

Review

Quoted 10 lines
(quoted diff) +10 -0
+                               break;
+                       case 'm':
+                       case 'M':
+                               *size = *size * 1024 * 1024;
+                               break;
+                       case 'g':
+                       case 'G':
+                               *size = *size * 1024 * 1024 * 1024;
+                               break;
+                       }

I think you could split the computation of the human size in two separate variables (size, computed_size perhaps?).

Also, you could add checks for possible overflows/truncation in the computation/store (e.g. a user could specify 4194304K (SIZE_MAX/1024
+ 1), quite unlikely but still possible!). This might be a bit overkill though...

Could you also add a default case with perhaps ERR() or assert()?

Review

Quoted 4 lines
(quoted diff) +3 -0
+               ret = 1;
[...]
+               regfree(&regex);
+               return 1;

ret seems to be unused here. Maybe you meant return ret?

You might also want to add negative numbers and large number testcases to your unit tests.

Looking forward to the corrected patch + unit tests :).

Thanks!

Christian

P.S.: This new feature seems in part related to bug #228 [1].

[1] - http://stackoverflow.com/questions/3792663/atol-v-s-strtol
[2] - http://bugs.lttng.org/issues/228

Quoted 193 lines · Simon Marchi

--subbuf-size accepts sizes such as:

- 123 -> 123
- 123k or 123K -> 123 * 1024
- 123m or 123M -> 123 * 1024 * 1024
- 123g or 123G -> 123 * 1024 * 1024 * 1024

It uses the new parse_human_size function, which could probably be used at other places, such as tracefile size.

I wanted to add some unit tests for parse_human_size, but the current test system does not make it easy. They will come later.

---
 src/bin/lttng/commands/enable_channels.c |   18 +++++-
 src/bin/lttng/utils.c                    |   96 +++++++++++++++++++++++++++++-
 src/bin/lttng/utils.h                    |    2 +
 3 files changed, 113 insertions(+), 3 deletions(-)
src/bin/lttng/commands/enable_channels.c +16 -2
index d026af4..f71bda5 100644
@@ -289,6 +289,7 @@ int cmd_enable_channels(int argc, const char **argv)
        int opt, ret = CMD_SUCCESS;
        static poptContext pc;
        char *session_name = NULL;
+       char *opt_arg = NULL;
 
        init_channel_config();
 
@@ -309,8 +310,21 @@ int cmd_enable_channels(int argc, const char **argv)
                        DBG("Channel set to overwrite");
                        break;
                case OPT_SUBBUF_SIZE:
-                       /* TODO Replace atol with strtol and check for errors */
-                       chan.attr.subbuf_size = atol(poptGetOptArg(pc));
+                       /* Parse the size */
+                       opt_arg = poptGetOptArg(pc);
+                       if (!parse_human_size(opt_arg, &chan.attr.subbuf_size)) {
+                               ERR("Wrong format for subbuf size parameter: %s", opt_arg);
+                               ret = CMD_ERROR;
+                               goto end;
+                       }
+
+                       /* Check if power of 2 */
+                       if ((chan.attr.subbuf_size - 1) & chan.attr.subbuf_size) {
+                               ERR("The subbuf size is not a power of 2: %" PRIu64 " (%s)", chan.attr.subbuf_size, opt_arg);
+                               ret = CMD_ERROR;
+                               goto end;
+                       }
+
                        DBG("Channel subbuf size set to %" PRIu64, chan.attr.subbuf_size);
                        break;
                case OPT_NUM_SUBBUF:
src/bin/lttng/utils.c +95 -1
index b7f5170..a21f00f 100644
@@ -16,9 +16,10 @@
  */
 
 #define _GNU_SOURCE
-#include <stdlib.h>
 #include <ctype.h>
 #include <limits.h>
+#include <regex.h>
+#include <stdlib.h>
 
 #include <common/error.h>
 
@@ -77,3 +78,96 @@ void list_cmd_options(FILE *ofp, struct poptOption *options)
                }
        }
 }
+
+/**
+ * Prints the error message corresponding to a regex error code.
+ *
+ * @param errcode      The error code.
+ * @param regex                The regex object that produced the error code.
+ */
+void regex_print_error(int errcode, regex_t *regex)
+{
+       /* Get length of error message and allocate accordingly */
+       size_t length = regerror(errcode, regex, NULL, 0);
+       char *buffer = malloc(length);
+
+       if (buffer) {
+               /* Get and print error message */
+               regerror(errcode, regex, buffer, length);
+               ERR("%s\n", buffer);
+               free(buffer);
+       }
+}
+
+/**
+ * Parse a string that represents a size in human readable format. It
+ * support decimal integers suffixed by 'k', 'K', 'm', 'M', g' or 'G'.
+ *
+ * The suffix multiply the integer by:
+ * 'k' or 'K': 1024
+ * 'm' or 'M': 1024 * 1024
+ * 'g' or 'G': 1024 * 1024 * 1024
+ *
+ * @param str  The string to parse.
+ * @param size Pointer to a size_t that will be filled with the
+ *             resulting size.
+ *
+ * @return     1 on success, 0 on failure.
+ */
+uint64_t parse_human_size(char *str, size_t *size)
+{
+       regex_t regex;
+       int ret;
+
+       if (!str)
+               return 0;
+
+       /* Compile regex */
+       ret = regcomp(&regex, "^[0-9][0-9]*\\([kmg]\\{0,1\\}\\)$", REG_ICASE);
+
+       /* Check for regex error */
+       if (ret != 0) {
+               regex_print_error(ret, &regex);
+
+               regfree(&regex);
+               return 0;
+       }
+
+       const int nmatch = 2;
+       regmatch_t matches[nmatch];
+
+       /* Match regex */
+       ret = regexec(&regex, str, nmatch, matches, 0);
+       if (ret == 0) {
+               /* There is a match ! */
+               ret = 1;
+
+               *size = atol(str);
+
+               /* Check if there is a suffix */
+               regmatch_t suffix_match = matches[1];
+               if (suffix_match.rm_eo - suffix_match.rm_so > 0) {
+                       switch (*(str + suffix_match.rm_so)) {
+                       case 'k':
+                       case 'K':
+                               *size = *size * 1024;
+                               break;
+                       case 'm':
+                       case 'M':
+                               *size = *size * 1024 * 1024;
+                               break;
+                       case 'g':
+                       case 'G':
+                               *size = *size * 1024 * 1024 * 1024;
+                               break;
+                       }
+               }
+
+               regfree(&regex);
+               return 1;
+       } else {
+               /* There is no match (or error) */
+               regfree(&regex);
+               return 0;
+       }
+}
src/bin/lttng/utils.h +2 -1
index 9f7bfcc..20ebd68 100644
@@ -19,8 +19,10 @@
 #define _LTTNG_UTILS_H
 
 #include <popt.h>
+#include <stdint.h>
 
 char *get_session_name(void);
 void list_cmd_options(FILE *ofp, struct poptOption *options);
+uint64_t parse_human_size(char *str, size_t *size);
 
 #endif /* _LTTNG_UTILS_H */
--

1.7.1

_______________________________________________
lttng-dev mailing list
[email protected]
http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

#2 lore raw
In reply to a message not in the archive
Quoted 6 lines · Simon Marchi

--subbuf-size accepts sizes such as:

- 123 -> 123
- 123k or 123K -> 123 * 1024
- 123m or 123M -> 123 * 1024 * 1024
- 123g or 123G -> 123 * 1024 * 1024 * 1024

I don't really like using non-SI prefixes such as "K", "m" or "g" (well, in fact "m" is an SI prefix but for milli, not mega).
I think that would be better to only use "k", "M" and "G".

_______________________________________________
lttng-dev mailing list
[email protected]
http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev

A message that is not in the archive (sent off-list or not yet received): CAGDH53ks4zAi7YfQZfFA1d3r=2ZEUvX9_gZYUk9YyPZHwT6uuA@mail.gmail.com

#4 lore raw
Quoted 7 lines · Christian Babeux

Hi Simon,

Quoted 2 lines

I wanted to add some unit tests for parse_human_size, but the current test system does not make it easy. They will come later.

Could you describe the issues you had on a separate thread so we can further discuss it?

Will do.

Quoted 11 lines

Here a few comments on your patch:

Review

Quoted 8 lines
(quoted diff) +8 -0
+       char *buffer = malloc(length);
+
+       if (buffer) {
+               /* Get and print error message */
+               regerror(errcode, regex, buffer, length);
+               ERR("%s\n", buffer);
+               free(buffer);
+       }
Quoted 2 lines

What if malloc() fails? Printing an error message would be appropriate to handle that case.

Ok.

Quoted 4 lines

Review

Quoted 2 lines
(quoted diff) +2 -0
+       /* Compile regex */
+       ret = regcomp(&regex, "^[0-9][0-9]*\\([kmg]\\{0,1\\}\\)$", REG_ICASE);

This regex could be simplified to: "^[:digit:]+\\([kmg]\\)?$"

I preferred not to use extended regex. + and ? is included in the extended posix regexes [1]. I also prefer [0-9] over [:digit:], because it is as clear and less characters, but I don't mind :).

Quoted 8 lines
Quoted 1 line

+ *size = atol(str);

atol() can return negative numbers and the result is stored in an unsigned (size_t). This could lead to enormous subbuffers allocated by accident. A check should be added to allow only positive numbers.

Also, atol() does not detect conversion errors, returning 0 in error cases [1]. Perhaps using strtol() would be more appropriate?

True. But here, we are safe because the regex test passed We know for sure that we have a positive number that atol will correctly parse.

Quoted 18 lines
Quoted 7 lines
(quoted diff) +7 -0
+               /* Check if there is a suffix */
+               regmatch_t suffix_match = matches[1];
+               if (suffix_match.rm_eo - suffix_match.rm_so > 0) {
+                       switch (*(str + suffix_match.rm_so)) {
+                       case 'k':
+                       case 'K':
+                               *size = *size * 1024;

Review

Quoted 10 lines
(quoted diff) +10 -0
+                               break;
+                       case 'm':
+                       case 'M':
+                               *size = *size * 1024 * 1024;
+                               break;
+                       case 'g':
+                       case 'G':
+                               *size = *size * 1024 * 1024 * 1024;
+                               break;
+                       }
Quoted 2 lines

I think you could split the computation of the human size in two separate variables (size, computed_size perhaps?).

Do you want to return both values to the caller, or simply use two intermediate variables and multiply them at the end ?

Quoted 4 lines

Also, you could add checks for possible overflows/truncation in the computation/store (e.g. a user could specify 4194304K (SIZE_MAX/1024
+ 1), quite unlikely but still possible!). This might be a bit overkill though...

Do you have an idea in mind on ho to do this ?

Quoted 1 line

Could you also add a default case with perhaps ERR() or assert()?

Like earlier, we are safe because we passed the regex test. But it's still good idea to add this default to catch eventual dev mistakes.

Quoted 6 lines

Review

Quoted 4 lines
(quoted diff) +3 -0
+               ret = 1;
[...]
+               regfree(&regex);
+               return 1;

ret seems to be unused here. Maybe you meant return ret?

Oops.

Quoted 2 lines

You might also want to add negative numbers and large number testcases to your unit tests.

Of course.

Quoted 10 lines

Looking forward to the corrected patch + unit tests :).

Thanks!

Christian

P.S.: This new feature seems in part related to bug #228 [1].

[1] - http://stackoverflow.com/questions/3792663/atol-v-s-strtol
[2] - http://bugs.lttng.org/issues/228

Thanks for the comments.

Simon

[1] http://en.wikipedia.org/wiki/Regular_expression#POSIX_Basic_Regular_Expressions

A message that is not in the archive (sent off-list or not yet received): CAFXXi0=GHXVvAmx7DZ44rSQ2SK6cXUjDhOyA2PLBB2HG7kHjSQ@mail.gmail.com

#6 lore raw

Hi Simon,

Quoted 3 lines

I preferred not to use extended regex. + and ? is included in the extended posix regexes [1]. I also prefer [0-9] over [:digit:], because it is as clear and less characters, but I don't mind :).

Alright, let's go with the one using basic POSIX regexes.

Quoted 9 lines
Quoted 6 lines

atol() can return negative numbers and the result is stored in an unsigned (size_t). This could lead to enormous subbuffers allocated by accident. A check should be added to allow only positive numbers.

Also, atol() does not detect conversion errors, returning 0 in error cases [1]. Perhaps using strtol() would be more appropriate?

True. But here, we are safe because the regex test passed We know for sure that we have a positive number that atol will correctly parse.

Yes, indeed. Maybe adding an assert(*size > 0) could warn developers to pay attention when eventually moving code around.

But the latter point is still unadressed: an error condition cannot be distinguished from the value "0". I would recommend using strtol() instead.

Quoted 2 lines

Do you want to return both values to the caller, or simply use two intermediate variables and multiply them at the end ?

Use two intermediate variables in a similar fashion:
---

size_t user_size, computed_size;

[...]

user_size = atol(str);

[...]

switch (*(str + suffix_match.rm_so)) {
    case 'k':
    case 'K':
        computed_size = user_size * 1024;
        /* Maybe add overflow checks on computed_size here */
        ret = 1;

[...]

/* If no error occurred */
*size = computed_size;
return ret;

---

Quoted 6 lines
Quoted 4 lines

Also, you could add checks for possible overflows/truncation in the computation/store (e.g. a user could specify 4194304K (SIZE_MAX/1024
+ 1), quite unlikely but still possible!). This might be a bit overkill though...

Do you have an idea in mind on ho to do this ?

You could probably detect the overflow of the multiplication with this:

if (user_size != 0 && computed_size / user_size != 1024) {
    /* Overflow, impendent doom, abort mission */
}

Here a couple more comments on the patch:

- Instead of hardcoding the constants 1024, 1024*1024, 1024*1024*1024, etc. you could add some defines for them.

- Perhaps in a separate patch, the lttng(1) manpage should be updated to reflect the new options added to the cli (e.g. --subbuf-size SIZE[KMG])

Thanks!

Christian

Quoted 114 lines · Simon Marchi
Quoted 7 lines · Christian Babeux

Hi Simon,

Quoted 2 lines

I wanted to add some unit tests for parse_human_size, but the current test system does not make it easy. They will come later.

Could you describe the issues you had on a separate thread so we can further discuss it?

Will do.

Quoted 11 lines

Here a few comments on your patch:

Review

Quoted 8 lines
(quoted diff) +8 -0
+       char *buffer = malloc(length);
+
+       if (buffer) {
+               /* Get and print error message */
+               regerror(errcode, regex, buffer, length);
+               ERR("%s\n", buffer);
+               free(buffer);
+       }
Quoted 2 lines

What if malloc() fails? Printing an error message would be appropriate to handle that case.

Ok.

Quoted 4 lines

Review

Quoted 2 lines
(quoted diff) +2 -0
+       /* Compile regex */
+       ret = regcomp(&regex, "^[0-9][0-9]*\\([kmg]\\{0,1\\}\\)$", REG_ICASE);

This regex could be simplified to: "^[:digit:]+\\([kmg]\\)?$"

I preferred not to use extended regex. + and ? is included in the extended posix regexes [1]. I also prefer [0-9] over [:digit:], because it is as clear and less characters, but I don't mind :).

Quoted 8 lines
Quoted 1 line

+ *size = atol(str);

atol() can return negative numbers and the result is stored in an unsigned (size_t). This could lead to enormous subbuffers allocated by accident. A check should be added to allow only positive numbers.

Also, atol() does not detect conversion errors, returning 0 in error cases [1]. Perhaps using strtol() would be more appropriate?

True. But here, we are safe because the regex test passed We know for sure that we have a positive number that atol will correctly parse.

Quoted 18 lines
Quoted 7 lines
(quoted diff) +7 -0
+               /* Check if there is a suffix */
+               regmatch_t suffix_match = matches[1];
+               if (suffix_match.rm_eo - suffix_match.rm_so > 0) {
+                       switch (*(str + suffix_match.rm_so)) {
+                       case 'k':
+                       case 'K':
+                               *size = *size * 1024;

Review

Quoted 10 lines
(quoted diff) +10 -0
+                               break;
+                       case 'm':
+                       case 'M':
+                               *size = *size * 1024 * 1024;
+                               break;
+                       case 'g':
+                       case 'G':
+                               *size = *size * 1024 * 1024 * 1024;
+                               break;
+                       }
Quoted 2 lines

I think you could split the computation of the human size in two separate variables (size, computed_size perhaps?).

Do you want to return both values to the caller, or simply use two intermediate variables and multiply them at the end ?

Quoted 4 lines

Also, you could add checks for possible overflows/truncation in the computation/store (e.g. a user could specify 4194304K (SIZE_MAX/1024
+ 1), quite unlikely but still possible!). This might be a bit overkill though...

Do you have an idea in mind on ho to do this ?

Quoted 1 line

Could you also add a default case with perhaps ERR() or assert()?

Like earlier, we are safe because we passed the regex test. But it's still good idea to add this default to catch eventual dev mistakes.

Quoted 6 lines

Review

Quoted 4 lines
(quoted diff) +3 -0
+               ret = 1;
[...]
+               regfree(&regex);
+               return 1;

ret seems to be unused here. Maybe you meant return ret?

Oops.

Quoted 2 lines

You might also want to add negative numbers and large number testcases to your unit tests.

Of course.

Quoted 10 lines

Looking forward to the corrected patch + unit tests :).

Thanks!

Christian

P.S.: This new feature seems in part related to bug #228 [1].

[1] - http://stackoverflow.com/questions/3792663/atol-v-s-strtol
[2] - http://bugs.lttng.org/issues/228

Thanks for the comments.

Simon

[1] http://en.wikipedia.org/wiki/Regular_expression#POSIX_Basic_Regular_Expressions

A message that is not in the archive (sent off-list or not yet received): [email protected]

#8 lore raw
Quoted 11 lines · Raphaël Beamonte
Quoted 6 lines · Simon Marchi

--subbuf-size accepts sizes such as:

- 123 -> 123
- 123k or 123K -> 123 * 1024
- 123m or 123M -> 123 * 1024 * 1024
- 123g or 123G -> 123 * 1024 * 1024 * 1024

I don't really like using non-SI prefixes such as "K", "m" or "g" (well, in fact "m" is an SI prefix but for milli, not mega).
I think that would be better to only use "k", "M" and "G".

Agreed. I'll do that.

Simon