Skip to contentSkip to conversation

makedumpfile: print spinner in progress information

PATCH In review HATAYAMA Daisuke 3 participants 8 messages Oct 25, 2013 – Oct 29, 2013

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 makedumpfile: print spinner in progress information

On system with huge memory, percentage in progress information is updated at very slow interval, because 1 percent on 1 TiB memory is about 10 GiB, which looks like as if system has freezed. Then, confused users might get tempted to push a reset button to recover the system. We want to avoid such situation as much as possible.

To address the issue, this patch adds spinner that rotates in the order of /, |, \ and - next to the progress indicator in percentage, which helps users to get aware that system is still active and crash dump process is still in progress now.

This code is borrowed from diskdump code.

The example is like this:

Copying data                       : [  0 %] /
Copying data                       : [  8 %] |
Copying data                       : [ 11 %] \
Copying data                       : [ 14 %] -
Copying data                       : [ 16 %] /

...
Copying data : [ 99 %] /
Copying data : [100 %] |

---
 print_info.c | 11 +++++++----
 1 file changed, 7 insertions(+), 4 deletions(-)
print_info.c +7 -4
index 3527970..879fa4b 100644
@@ -286,6 +286,8 @@ print_progress(const char *msg, unsigned long current, unsigned long end)
 	int progress;
 	time_t tm;
 	static time_t last_time = 0;
+	static unsigned int lapse = 0;
+	const char *spinner = "/|\\-";
 
 	if (current < end) {
 		tm = time(NULL);
@@ -297,13 +299,14 @@ print_progress(const char *msg, unsigned long current, unsigned long end)
 		progress = 100;
 
 	if (flag_ignore_r_char) {
-		PROGRESS_MSG("%-" PROGRESS_MAXLEN "s: [%3d %%]\n",
-			     msg, progress);
+		PROGRESS_MSG("%-" PROGRESS_MAXLEN "s: [%3d %%] %c\n",
+			     msg, progress, spinner[lapse & 3]);
 	} else {
 		PROGRESS_MSG("\r");
-		PROGRESS_MSG("%-" PROGRESS_MAXLEN "s: [%3d %%] ",
-			     msg, progress);
+		PROGRESS_MSG("%-" PROGRESS_MAXLEN "s: [%3d %%] %c",
+			     msg, progress, spinner[lapse & 3]);
 	}
+	lapse++;
 }
 
 void
Signature · 7 lines
1.8.3.1


_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
#1 lore raw
Quoted 34 lines · HATAYAMA Daisuke

On system with huge memory, percentage in progress information is updated at very slow interval, because 1 percent on 1 TiB memory is about 10 GiB, which looks like as if system has freezed. Then, confused users might get tempted to push a reset button to recover the system. We want to avoid such situation as much as possible.

To address the issue, this patch adds spinner that rotates in the order of /, |, \ and - next to the progress indicator in percentage, which helps users to get aware that system is still active and crash dump process is still in progress now.

This code is borrowed from diskdump code.

The example is like this:

Copying data                       : [  0 %] /
Copying data                       : [  8 %] |
Copying data                       : [ 11 %] \
Copying data                       : [ 14 %] -
Copying data                       : [ 16 %] /

...
Copying data : [ 99 %] /
Copying data : [100 %] |

---
  print_info.c | 11 +++++++----
  1 file changed, 7 insertions(+), 4 deletions(-)
print_info.c +0 -0
index 3527970..879fa4b 100644

print_info.c print_progress(const char *msg, unsigned long current, unsigned long end) · line 290

Quoted 5 lines
print_info.c +2 -0
@@ -286,6 +286,8 @@ print_progress(const char *msg, unsigned long current, unsigned long end)
  	int progress;
  	time_t tm;
  	static time_t last_time = 0;
+	static unsigned int lapse = 0;
+	const char *spinner = "/|\\-";

I believe your spinner should be:
const char *spinner = "/|\-/|\-";

Quoted 22 lines
print_info.c +5 -4
  	if (current<  end) {
  		tm = time(NULL);
@@ -297,13 +299,14 @@ print_progress(const char *msg, unsigned long current, unsigned long end)
  		progress = 100;
 
  	if (flag_ignore_r_char) {
-		PROGRESS_MSG("%-" PROGRESS_MAXLEN "s: [%3d %%]\n",
-			     msg, progress);
+		PROGRESS_MSG("%-" PROGRESS_MAXLEN "s: [%3d %%] %c\n",
+			     msg, progress, spinner[lapse&  3]);
  	} else {
  		PROGRESS_MSG("\r");
-		PROGRESS_MSG("%-" PROGRESS_MAXLEN "s: [%3d %%] ",
-			     msg, progress);
+		PROGRESS_MSG("%-" PROGRESS_MAXLEN "s: [%3d %%] %c",
+			     msg, progress, spinner[lapse&  3]);
  	}
+	lapse++;
  }
 
  void
Signature · 7 lines
Thanks,
Jingbai Ma

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
#2 lore raw
Quoted 30 lines · Jingbai Ma
Quoted 29 lines · HATAYAMA Daisuke

On system with huge memory, percentage in progress information is updated at very slow interval, because 1 percent on 1 TiB memory is about 10 GiB, which looks like as if system has freezed. Then, confused users might get tempted to push a reset button to recover the system. We want to avoid such situation as much as possible.

To address the issue, this patch adds spinner that rotates in the order of /, |, \ and - next to the progress indicator in percentage, which helps users to get aware that system is still active and crash dump process is still in progress now.

This code is borrowed from diskdump code.

The example is like this:

Copying data : [ 0 %] /
Copying data : [ 8 %] |
Copying data : [ 11 %] \
Copying data : [ 14 %] -
Copying data : [ 16 %] /
...
Copying data : [ 99 %] /
Copying data : [100 %] |

---
print_info.c | 11 +++++++----

1 file changed, 7 insertions(+), 4 deletions(-)
Quoted 15 lines

print_info.c print_progress(const char *msg, unsigned long · line 291

Quoted 11 lines
print_info.c +2 -0
index 3527970..879fa4b 100644
@@ -286,6 +286,8 @@ print_progress(const char *msg, unsigned long
 current, unsigned long end)
 int progress;
 time_t tm;
 static time_t last_time = 0;
+ static unsigned int lapse = 0;
+ const char *spinner = "/|\\-";

I believe your spinner should be:
const char *spinner = "/|\-/|\-";

Oh, it seems not a full round, this one should be better:
const char *spinner = "/|\-";

Quoted 7 lines
Quoted 7 lines
print_info.c +0 -0
 if (current< end) {
 tm = time(NULL);
@@ -297,13 +299,14 @@ print_progress(const char *msg, unsigned long
 current, unsigned long end)
 progress = 100;
 
Quoted 5 lines

print_info.c print_progress(const char *msg, unsigned long current, unsigned long end)

Quoted 5 lines
print_info.c +2 -2
 if (flag_ignore_r_char) {
- PROGRESS_MSG("%-" PROGRESS_MAXLEN "s: [%3d %%]\n",
- msg, progress);
+ PROGRESS_MSG("%-" PROGRESS_MAXLEN "s: [%3d %%] %c\n",
+ msg, progress, spinner[lapse& 3]);

And then, this line should be:
msg, progress, spinner[lapse % 4]);

Quoted 6 lines

print_info.c print_progress(const char *msg, unsigned long current, unsigned long end)

Quoted 6 lines
print_info.c +2 -2
 } else {
 PROGRESS_MSG("\r");
- PROGRESS_MSG("%-" PROGRESS_MAXLEN "s: [%3d %%] ",
- msg, progress);
+ PROGRESS_MSG("%-" PROGRESS_MAXLEN "s: [%3d %%] %c",
+ msg, progress, spinner[lapse& 3]);

This line too:
msg, progress, spinner[lapse % 4]);

Quoted 7 lines
Quoted 5 lines
print_info.c +1 -0
 }
+ lapse++;
 }
 

void

Signature · 7 lines
Thanks,
Jingbai Ma

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
#4 lore raw
In reply to HATAYAMA Daisuke

Hello HATAYAMA-san,

Quoted 15 lines · HATAYAMA Daisuke

On system with huge memory, percentage in progress information is updated at very slow interval, because 1 percent on 1 TiB memory is about 10 GiB, which looks like as if system has freezed. Then, confused users might get tempted to push a reset button to recover the system. We want to avoid such situation as much as possible.

To address the issue, this patch adds spinner that rotates in the order of /, |, \ and - next to the progress indicator in percentage, which helps users to get aware that system is still active and crash dump process is still in progress now.

This code is borrowed from diskdump code.

The example is like this:

Quoted 8 lines
Copying data                       : [  0 %] /
Copying data                       : [  8 %] |
Copying data                       : [ 11 %] \
Copying data                       : [ 14 %] -
Copying data                       : [ 16 %] /

...
Copying data : [ 99 %] /
Copying data : [100 %] |

I like it, but have a comment.

    6109 int
    6110 write_kdump_pages_cyclic(struct cache_data *cd_header, struct cache_data *cd_page,
    6111                          struct page_desc *pd_zero, off_t *offset_data)
    6112 {
    ...
    6156         per = info->num_dumpable / 100;
    ...
    6178         for (pfn = start_pfn; pfn < end_pfn; pfn++) {
    6179
    6180                 if ((num_dumped % per) == 0)
    6181                         print_progress(PROGRESS_COPY, num_dumped, info->num_dumpable);

The interval of calling print_progress() looks still long if num_dumpable is huge.
So how about fix this, e.g., by changing the interval to time based ?

Thanks
Atsushi Kumagai

Quoted 28 lines
---
  print_info.c | 11 +++++++----
  1 file changed, 7 insertions(+), 4 deletions(-)
print_info.c +4 -2
index 3527970..879fa4b 100644
@@ -286,6 +286,8 @@ print_progress(const char *msg, unsigned long current, unsigned long end)
  	int progress;
  	time_t tm;
  	static time_t last_time = 0;
+	static unsigned int lapse = 0;
+	const char *spinner = "/|\\-";
 
  	if (current < end) {
  		tm = time(NULL);
@@ -297,13 +299,14 @@ print_progress(const char *msg, unsigned long current, unsigned long end)
  		progress = 100;
 
  	if (flag_ignore_r_char) {
-		PROGRESS_MSG("%-" PROGRESS_MAXLEN "s: [%3d %%]\n",
-			     msg, progress);
+		PROGRESS_MSG("%-" PROGRESS_MAXLEN "s: [%3d %%] %c\n",
+			     msg, progress, spinner[lapse & 3]);
  	} else {

print_info.c print_progress(const char *msg, unsigned long current, unsigned long end) · line 312

Quoted 10 lines
print_info.c +3 -2
  		PROGRESS_MSG("\r");
-		PROGRESS_MSG("%-" PROGRESS_MAXLEN "s: [%3d %%] ",
-			     msg, progress);
+		PROGRESS_MSG("%-" PROGRESS_MAXLEN "s: [%3d %%] %c",
+			     msg, progress, spinner[lapse & 3]);
  	}
+	lapse++;
  }
 
  void

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec

#3 lore raw
In reply to Jingbai Ma
Quoted 47 lines · Jingbai Ma
Quoted 30 lines · Jingbai Ma
Quoted 29 lines · HATAYAMA Daisuke

On system with huge memory, percentage in progress information is updated at very slow interval, because 1 percent on 1 TiB memory is about 10 GiB, which looks like as if system has freezed. Then, confused users might get tempted to push a reset button to recover the system. We want to avoid such situation as much as possible.

To address the issue, this patch adds spinner that rotates in the order of /, |, \ and - next to the progress indicator in percentage, which helps users to get aware that system is still active and crash dump process is still in progress now.

This code is borrowed from diskdump code.

The example is like this:

Copying data : [ 0 %] /
Copying data : [ 8 %] |
Copying data : [ 11 %] \
Copying data : [ 14 %] -
Copying data : [ 16 %] /
...
Copying data : [ 99 %] /
Copying data : [100 %] |

---
print_info.c | 11 +++++++----

1 file changed, 7 insertions(+), 4 deletions(-)
Quoted 15 lines

print_info.c print_progress(const char *msg, unsigned long · line 291

Quoted 11 lines
print_info.c +2 -0
index 3527970..879fa4b 100644
@@ -286,6 +286,8 @@ print_progress(const char *msg, unsigned long
 current, unsigned long end)
 int progress;
 time_t tm;
 static time_t last_time = 0;
+ static unsigned int lapse = 0;
+ const char *spinner = "/|\\-";

I believe your spinner should be:
const char *spinner = "/|\-/|\-";

Quoted 3 lines

Oh, it seems not a full round, this one should be better:
const char *spinner = "/|\-";

It's escape sequence.

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
print_info.c +0 -0
 if (current< end) {
 tm = time(NULL);
@@ -297,13 +299,14 @@ print_progress(const char *msg, unsigned long
 current, unsigned long end)
 progress = 100;
 
Quoted 9 lines
Quoted 5 lines

print_info.c print_progress(const char *msg, unsigned long current, unsigned long end)

Quoted 5 lines
print_info.c +2 -2
 if (flag_ignore_r_char) {
- PROGRESS_MSG("%-" PROGRESS_MAXLEN "s: [%3d %%]\n",
- msg, progress);
+ PROGRESS_MSG("%-" PROGRESS_MAXLEN "s: [%3d %%] %c\n",
+ msg, progress, spinner[lapse& 3]);

And then, this line should be:
msg, progress, spinner[lapse % 4]);

OK, though I think either is fine.

Quoted 10 lines
Quoted 6 lines

print_info.c print_progress(const char *msg, unsigned long current, unsigned long end)

Quoted 6 lines
print_info.c +2 -2
 } else {
 PROGRESS_MSG("\r");
- PROGRESS_MSG("%-" PROGRESS_MAXLEN "s: [%3d %%] ",
- msg, progress);
+ PROGRESS_MSG("%-" PROGRESS_MAXLEN "s: [%3d %%] %c",
+ msg, progress, spinner[lapse& 3]);

This line too:
msg, progress, spinner[lapse % 4]);

OK.

Quoted 9 lines
Quoted 7 lines
Quoted 5 lines
print_info.c +1 -0
 }
+ lapse++;
 }
 

void

Signature · 8 lines
Thanks.
HATAYAMA, Daisuke


_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
#5 lore raw
In reply to Atsushi Kumagai
Quoted 41 lines · Atsushi Kumagai

Hello HATAYAMA-san,

Quoted 15 lines · HATAYAMA Daisuke

On system with huge memory, percentage in progress information is updated at very slow interval, because 1 percent on 1 TiB memory is about 10 GiB, which looks like as if system has freezed. Then, confused users might get tempted to push a reset button to recover the system. We want to avoid such situation as much as possible.

To address the issue, this patch adds spinner that rotates in the order of /, |, \ and - next to the progress indicator in percentage, which helps users to get aware that system is still active and crash dump process is still in progress now.

This code is borrowed from diskdump code.

The example is like this:

Quoted 8 lines
Copying data                       : [  0 %] /
Copying data                       : [  8 %] |
Copying data                       : [ 11 %] \
Copying data                       : [ 14 %] -
Copying data                       : [ 16 %] /

...
Copying data : [ 99 %] /
Copying data : [100 %] |

I like it, but have a comment.

     6109 int
     6110 write_kdump_pages_cyclic(struct cache_data *cd_header, struct cache_data *cd_page,
     6111                          struct page_desc *pd_zero, off_t *offset_data)
     6112 {
     ...
     6156         per = info->num_dumpable / 100;
     ...
     6178         for (pfn = start_pfn; pfn < end_pfn; pfn++) {
     6179
     6180                 if ((num_dumped % per) == 0)
     6181                         print_progress(PROGRESS_COPY, num_dumped, info->num_dumpable);
Quoted 4 lines

The interval of calling print_progress() looks still long if num_dumpable is huge.
So how about fix this, e.g., by changing the interval to time based ?

I wrote simple bench for time-based interval as below, which measures total time consumed for calling time system call with/without vDSO. It seems to me that both results are acceptable.
I'll reflect this change in next version.

$ ./bench
total: 21.059131
average: 0.000000
total: 65.558263
average: 0.000000

==bench.c
#include <stdio.h>
#include <stdlib.h>
#include <time.h>

static inline double getdtime(void)

{
        struct timeval tv;
        gettimeofday(&tv, NULL);
        return (double)tv.tv_sec + (double)tv.tv_usec * 1.0e-6;
}

int main(int argc, char **argv)

{
        unsigned long i;
        time_t t;
        double t1, t2, total;
        const int NR_time = 201;
        const unsigned long nr_repeat = (1UL << 40) / 4096;

        for (i = 0; i < nr_repeat; ++i) {
                t1 = getdtime();
                time(&t);
                t2 = getdtime();
                total += t2 - t1;
        }
        printf("total: %lf\n", total);
        printf("average: %lf\n", total / nr_repeat);

        for (i = 0; i < nr_repeat; ++i) {
                t1 = getdtime();
                syscall(NR_time, &t);
                t2 = getdtime();
                total += t2 - t1;
        }
        printf("total: %lf\n", total);
        printf("average: %lf\n", total / nr_repeat);

        return 0;
}

==

Signature · 8 lines
Thanks.
HATAYAMA, Daisuke


_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
#6 lore raw
Quoted 52 lines · HATAYAMA Daisuke
Quoted 41 lines · Atsushi Kumagai

Hello HATAYAMA-san,

Quoted 15 lines · HATAYAMA Daisuke

On system with huge memory, percentage in progress information is updated at very slow interval, because 1 percent on 1 TiB memory is about 10 GiB, which looks like as if system has freezed. Then, confused users might get tempted to push a reset button to recover the system. We want to avoid such situation as much as possible.

To address the issue, this patch adds spinner that rotates in the order of /, |, \ and - next to the progress indicator in percentage, which helps users to get aware that system is still active and crash dump process is still in progress now.

This code is borrowed from diskdump code.

The example is like this:

Quoted 8 lines
Copying data                       : [  0 %] /
Copying data                       : [  8 %] |
Copying data                       : [ 11 %] \
Copying data                       : [ 14 %] -
Copying data                       : [ 16 %] /

...
Copying data : [ 99 %] /
Copying data : [100 %] |

I like it, but have a comment.

      6109 int
      6110 write_kdump_pages_cyclic(struct cache_data *cd_header, struct cache_data *cd_page,
      6111                          struct page_desc *pd_zero, off_t *offset_data)
      6112 {
      ...
      6156         per = info->num_dumpable / 100;
      ...
      6178         for (pfn = start_pfn; pfn < end_pfn; pfn++) {
      6179
      6180                 if ((num_dumped % per) == 0)
      6181                         print_progress(PROGRESS_COPY, num_dumped, info->num_dumpable);
Quoted 4 lines

The interval of calling print_progress() looks still long if num_dumpable is huge.
So how about fix this, e.g., by changing the interval to time based ?

I wrote simple bench for time-based interval as below, which measures total time consumed for calling time system call with/without vDSO. It seems to me that both results are acceptable.
I'll reflect this change in next version.

Quoted 6 lines
$ ./bench
total: 21.059131
average: 0.000000
total: 65.558263
average: 0.000000

This conclusion was wrong. Sorry. For example on our FJ 12 TiB system we collected about 300 GiB crash dump in about 40 minutes. If removing "if ((num_dumped % per) == 0)" and calling time() in each loop in print_progress(), total time for invoking time() system call is about 65 * 12 = 780 sec = 13 min. This is about 20 % of a whole crash dump time. Obviously problematic.

Instead, I think it better to increase the number of calling print_progress() like:

per = info->num_dumpable / 10000

Signature · 8 lines
Thanks.
HATAYAMA, Daisuke


_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
#7 lore raw
Quoted 60 lines · HATAYAMA Daisuke
Quoted 52 lines · HATAYAMA Daisuke
Quoted 41 lines · Atsushi Kumagai

Hello HATAYAMA-san,

Quoted 15 lines · HATAYAMA Daisuke

On system with huge memory, percentage in progress information is updated at very slow interval, because 1 percent on 1 TiB memory is about 10 GiB, which looks like as if system has freezed. Then, confused users might get tempted to push a reset button to recover the system. We want to avoid such situation as much as possible.

To address the issue, this patch adds spinner that rotates in the order of /, |, \ and - next to the progress indicator in percentage, which helps users to get aware that system is still active and crash dump process is still in progress now.

This code is borrowed from diskdump code.

The example is like this:

Quoted 8 lines
Copying data                       : [  0 %] /
Copying data                       : [  8 %] |
Copying data                       : [ 11 %] \
Copying data                       : [ 14 %] -
Copying data                       : [ 16 %] /

...
Copying data : [ 99 %] /
Copying data : [100 %] |

I like it, but have a comment.

      6109 int
      6110 write_kdump_pages_cyclic(struct cache_data *cd_header, struct cache_data *cd_page,
      6111                          struct page_desc *pd_zero, off_t *offset_data)
      6112 {
      ...
      6156         per = info->num_dumpable / 100;
      ...
      6178         for (pfn = start_pfn; pfn < end_pfn; pfn++) {
      6179
      6180                 if ((num_dumped % per) == 0)
      6181                         print_progress(PROGRESS_COPY, num_dumped, info->num_dumpable);
Quoted 4 lines

The interval of calling print_progress() looks still long if num_dumpable is huge.
So how about fix this, e.g., by changing the interval to time based ?

I wrote simple bench for time-based interval as below, which measures total time consumed for calling time system call with/without vDSO. It seems to me that both results are acceptable.
I'll reflect this change in next version.

Quoted 6 lines
$ ./bench
total: 21.059131
average: 0.000000
total: 65.558263
average: 0.000000
Quoted 8 lines

This conclusion was wrong. Sorry. For example on our FJ 12 TiB system we collected about 300 GiB crash dump in about 40 minutes. If removing "if ((num_dumped % per) == 0)" and calling time() in each loop in print_progress(), total time for invoking time() system call is about 65 * 12 = 780 sec = 13 min. This is about 20 % of a whole crash dump time. Obviously problematic.

Instead, I think it better to increase the number of calling print_progress() like:

   per = info->num_dumpable / 10000

I agree with you, let's go with this idea.

Thanks
Atsushi Kumagai

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec