Skip to contentSkip to conversation

Cleanup migration/ram.c

PATCH In review Wei Yang 5 participants 23 messages Apr 30, 2019 – Jul 18, 2019

lore
Share & tools
lore

Wei Yang

author
#0 lore raw

cover Cleanup migration/ram.c

The *start* of migration_bitmap_sync_range is always 0, we can remove this parameter.

Since RAMBlock->offset is always *word* aligned, we can remove the check on offset and simplify the logic a little.

Wei Yang (3):
migration/ram.c: start of migration_bitmap_sync_range is always 0 migration/ram.c: start of cpu_physical_memory_sync_dirty_bitmap is
always 0
ram: RAMBlock->offset is always aligned to a word

 include/exec/ram_addr.h | 24 ++++++++++++------------
 migration/ram.c         |  8 ++++----
 2 files changed, 16 insertions(+), 16 deletions(-)
Signature · 1 line
2.19.1
author
#1 lore raw

PATCH 1/3 migration/ram.c: start of migration_bitmap_sync_range is always 0

We can eliminate to pass 0.

---
 migration/ram.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)
migration/ram.c +4 -4
index 35bd6213e9..9948b2d021 100644
@@ -1643,10 +1643,10 @@ static inline bool migration_bitmap_clear_dirty(RAMState *rs,
 }
 
 static void migration_bitmap_sync_range(RAMState *rs, RAMBlock *rb,
-                                        ram_addr_t start, ram_addr_t length)
+                                        ram_addr_t length)
 {
     rs->migration_dirty_pages +=
-        cpu_physical_memory_sync_dirty_bitmap(rb, start, length,
+        cpu_physical_memory_sync_dirty_bitmap(rb, 0, length,
                                               &rs->num_dirty_pages_period);
 }
 
@@ -1735,7 +1735,7 @@ static void migration_bitmap_sync(RAMState *rs)
     qemu_mutex_lock(&rs->bitmap_mutex);
     rcu_read_lock();
     RAMBLOCK_FOREACH_NOT_IGNORED(block) {
-        migration_bitmap_sync_range(rs, block, 0, block->used_length);
+        migration_bitmap_sync_range(rs, block, block->used_length);
     }
     ram_counters.remaining = ram_bytes_remaining();
     rcu_read_unlock();
@@ -4156,7 +4156,7 @@ static void colo_flush_ram_cache(void)
     memory_global_dirty_log_sync();
     rcu_read_lock();
     RAMBLOCK_FOREACH_NOT_IGNORED(block) {
-        migration_bitmap_sync_range(ram_state, block, 0, block->used_length);
+        migration_bitmap_sync_range(ram_state, block, block->used_length);
     }
     rcu_read_unlock();
 
Signature · 1 line
2.19.1
author
#7 lore raw

PATCH 2/3 migration/ram.c: start of cpu_physical_memory_sync_dirty_bitmap is always 0

In reply to Wei Yang

Since start of cpu_physical_memory_sync_dirty_bitmap is always 0, we can remove this parameter and simplify the calculation a bit.

---
 include/exec/ram_addr.h | 15 ++++++---------
 migration/ram.c         |  2 +-
 2 files changed, 7 insertions(+), 10 deletions(-)
include/exec/ram_addr.h +6 -9
index 9ecd911c3e..3dfb2d52fb 100644
@@ -409,18 +409,16 @@ static inline void cpu_physical_memory_clear_dirty_range(ram_addr_t start,
 
 static inline
 uint64_t cpu_physical_memory_sync_dirty_bitmap(RAMBlock *rb,
-                                               ram_addr_t start,
                                                ram_addr_t length,
                                                uint64_t *real_dirty_pages)
 {
     ram_addr_t addr;
-    unsigned long word = BIT_WORD((start + rb->offset) >> TARGET_PAGE_BITS);
+    unsigned long word = BIT_WORD(rb->offset >> TARGET_PAGE_BITS);
     uint64_t num_dirty = 0;
     unsigned long *dest = rb->bmap;
 
-    /* start address and length is aligned at the start of a word? */
-    if (((word * BITS_PER_LONG) << TARGET_PAGE_BITS) ==
-         (start + rb->offset) &&
+    /* offset and length is aligned at the start of a word? */
+    if (((word * BITS_PER_LONG) << TARGET_PAGE_BITS) == (rb->offset) &&
         !(length & ((BITS_PER_LONG << TARGET_PAGE_BITS) - 1))) {
         int k;
         int nr = BITS_TO_LONGS(length >> TARGET_PAGE_BITS);
@@ -428,14 +426,13 @@ uint64_t cpu_physical_memory_sync_dirty_bitmap(RAMBlock *rb,
         unsigned long idx = (word * BITS_PER_LONG) / DIRTY_MEMORY_BLOCK_SIZE;
         unsigned long offset = BIT_WORD((word * BITS_PER_LONG) %
                                         DIRTY_MEMORY_BLOCK_SIZE);
-        unsigned long page = BIT_WORD(start >> TARGET_PAGE_BITS);
 
         rcu_read_lock();
 
         src = atomic_rcu_read(
                 &ram_list.dirty_memory[DIRTY_MEMORY_MIGRATION])->blocks;
 
-        for (k = page; k < page + nr; k++) {
+        for (k = 0; k < nr; k++) {
             if (src[idx][offset]) {
                 unsigned long bits = atomic_xchg(&src[idx][offset], 0);
                 unsigned long new_dirty;
@@ -458,11 +455,11 @@ uint64_t cpu_physical_memory_sync_dirty_bitmap(RAMBlock *rb,
 
         for (addr = 0; addr < length; addr += TARGET_PAGE_SIZE) {
             if (cpu_physical_memory_test_and_clear_dirty(
-                        start + addr + offset,
+                        addr + offset,
                         TARGET_PAGE_SIZE,
                         DIRTY_MEMORY_MIGRATION)) {
                 *real_dirty_pages += 1;
-                long k = (start + addr) >> TARGET_PAGE_BITS;
+                long k = addr >> TARGET_PAGE_BITS;
                 if (!test_and_set_bit(k, dest)) {
                     num_dirty++;
                 }
migration/ram.c +1 -1
index 9948b2d021..1def8122e9 100644
@@ -1646,7 +1646,7 @@ static void migration_bitmap_sync_range(RAMState *rs, RAMBlock *rb,
                                         ram_addr_t length)
 {
     rs->migration_dirty_pages +=
-        cpu_physical_memory_sync_dirty_bitmap(rb, 0, length,
+        cpu_physical_memory_sync_dirty_bitmap(rb, length,
                                               &rs->num_dirty_pages_period);
 }
 
Signature · 1 line
2.19.1
author
#14 lore raw

PATCH 3/3 ram: RAMBlock->offset is always aligned to a word

In reply to Wei Yang

RAMBlock->offset is calculated by find_ram_offset, which makes sure the offset is aligned to a word.

This patch removes the alignment check on offset and unnecessary variable *word*.

---
 include/exec/ram_addr.h | 15 +++++++++------
 1 file changed, 9 insertions(+), 6 deletions(-)
include/exec/ram_addr.h +9 -6
index 3dfb2d52fb..a7c81bdb32 100644
@@ -413,18 +413,21 @@ uint64_t cpu_physical_memory_sync_dirty_bitmap(RAMBlock *rb,
                                                uint64_t *real_dirty_pages)
 {
     ram_addr_t addr;
-    unsigned long word = BIT_WORD(rb->offset >> TARGET_PAGE_BITS);
     uint64_t num_dirty = 0;
     unsigned long *dest = rb->bmap;
 
-    /* offset and length is aligned at the start of a word? */
-    if (((word * BITS_PER_LONG) << TARGET_PAGE_BITS) == (rb->offset) &&
-        !(length & ((BITS_PER_LONG << TARGET_PAGE_BITS) - 1))) {
+    /*
+     * Since RAMBlock->offset is guaranteed to be aligned to a word by
+     * find_ram_offset(), if length is aligned at the start of a word, go the
+     * fast path.
+     */
+    if (!(length & ((BITS_PER_LONG << TARGET_PAGE_BITS) - 1))) {
         int k;
         int nr = BITS_TO_LONGS(length >> TARGET_PAGE_BITS);
         unsigned long * const *src;
-        unsigned long idx = (word * BITS_PER_LONG) / DIRTY_MEMORY_BLOCK_SIZE;
-        unsigned long offset = BIT_WORD((word * BITS_PER_LONG) %
+        unsigned long idx = (rb->offset >> TARGET_PAGE_BITS) /
+                            DIRTY_MEMORY_BLOCK_SIZE;
+        unsigned long offset = BIT_WORD((rb->offset >> TARGET_PAGE_BITS) %
                                         DIRTY_MEMORY_BLOCK_SIZE);
 
         rcu_read_lock();
Signature · 1 line
2.19.1
#17 lore raw
In reply to Wei Yang
Patchew URL: https://patchew.org/QEMU/[email protected]/

Patchew URL: https://patchew.org/QEMU/[email protected]/

Hi,

This series failed the docker-mingw@fedora build test. Please find the testing commands and their output below. If you have Docker installed, you can probably reproduce it locally.

=== TEST SCRIPT BEGIN ===
#!/bin/bash
time make docker-test-mingw@fedora SHOW_ENV=1 J=14 NETWORK=1 === TEST SCRIPT END ===

The full log is available at
http://patchew.org/logs/[email protected]/testing.docker-mingw@fedora/?type=message.
---
Email generated automatically by Patchew [https://patchew.org/].
Please send your feedback to [email protected]

#18 lore raw
Quoted 18 lines · On Wed, May 01, 2019 at 03:53:01PM -0700, [email protected] wrote:

Patchew URL: https://patchew.org/QEMU/[email protected]/

Hi,

This series failed the docker-mingw@fedora build test. Please find the testing commands and their output below. If you have Docker installed, you can probably reproduce it locally.

=== TEST SCRIPT BEGIN ===
#!/bin/bash
time make docker-test-mingw@fedora SHOW_ENV=1 J=14 NETWORK=1 === TEST SCRIPT END ===

Quoted 2 lines

The full log is available at
http://patchew.org/logs/[email protected]/testing.docker-mingw@fedora/?type=message.

Where can I see the test case?

Quoted 3 lines

---
Email generated automatically by Patchew [https://patchew.org/].
Please send your feedback to [email protected]

Signature · 2 lines
Wei Yang
Help you, Help me
#19 lore raw
In reply to Wei Yang
Patchew URL: https://patchew.org/QEMU/[email protected]/

Patchew URL: https://patchew.org/QEMU/[email protected]/

Hi,

This series failed the asan build test. Please find the testing commands and their output below. If you have Docker installed, you can probably reproduce it locally.

=== TEST SCRIPT BEGIN ===
#!/bin/bash
time make docker-test-debug@fedora TARGET_LIST=x86_64-softmmu J=14 NETWORK=1 === TEST SCRIPT END ===

COPY RUNNER
RUN test-debug in qemu:fedora
container_linux.go:247: starting container process caused "process_linux.go:258: applying cgroup configuration for process caused \"The maximum number of active connections for UID 0 has been reached\"" /usr/bin/docker-current: Error response from daemon: oci runtime error: The maximum number of active connections for UID 0 has been reached.
Traceback (most recent call last):
File "./tests/docker/docker.py", line 615, in <module>

    sys.exit(main())

The full log is available at
http://patchew.org/logs/[email protected]/testing.asan/?type=message.
---
Email generated automatically by Patchew [https://patchew.org/].
Please send your feedback to [email protected]

#20 lore raw
Quoted 15 lines · On Wed, May 01, 2019 at 09:24:06PM -0700, [email protected] wrote:

Patchew URL: https://patchew.org/QEMU/[email protected]/

Hi,

This series failed the asan build test. Please find the testing commands and their output below. If you have Docker installed, you can probably reproduce it locally.

=== TEST SCRIPT BEGIN ===
#!/bin/bash
time make docker-test-debug@fedora TARGET_LIST=x86_64-softmmu J=14 NETWORK=1 === TEST SCRIPT END ===

Quoted 8 lines

COPY RUNNER
RUN test-debug in qemu:fedora
container_linux.go:247: starting container process caused "process_linux.go:258: applying cgroup configuration for process caused \"The maximum number of active connections for UID 0 has been reached\"" /usr/bin/docker-current: Error response from daemon: oci runtime error: The maximum number of active connections for UID 0 has been reached.
Traceback (most recent call last):
File "./tests/docker/docker.py", line 615, in <module>
sys.exit(main())

May I ask how I can reproduce this?

Quoted 6 lines

The full log is available at
http://patchew.org/logs/[email protected]/testing.asan/?type=message.
---
Email generated automatically by Patchew [https://patchew.org/].
Please send your feedback to [email protected]

Signature · 2 lines
Wei Yang
Help you, Help me
#21 lore raw
Quoted 25 lines · * Wei Yang ([email protected]) wrote:
Quoted 15 lines · On Wed, May 01, 2019 at 09:24:06PM -0700, [email protected] wrote:

Patchew URL: https://patchew.org/QEMU/[email protected]/

Hi,

This series failed the asan build test. Please find the testing commands and their output below. If you have Docker installed, you can probably reproduce it locally.

=== TEST SCRIPT BEGIN ===
#!/bin/bash
time make docker-test-debug@fedora TARGET_LIST=x86_64-softmmu J=14 NETWORK=1 === TEST SCRIPT END ===

Quoted 8 lines

COPY RUNNER
RUN test-debug in qemu:fedora
container_linux.go:247: starting container process caused "process_linux.go:258: applying cgroup configuration for process caused \"The maximum number of active connections for UID 0 has been reached\"" /usr/bin/docker-current: Error response from daemon: oci runtime error: The maximum number of active connections for UID 0 has been reached.
Traceback (most recent call last):
File "./tests/docker/docker.py", line 615, in <module>
sys.exit(main())

Quoted 1 line

May I ask how I can reproduce this?

To me this just looks like patchew having a problem, I don't think it's a real qemu bug.

Dave

Quoted 10 lines
Quoted 6 lines

The full log is available at
http://patchew.org/logs/[email protected]/testing.asan/?type=message.
---
Email generated automatically by Patchew [https://patchew.org/].
Please send your feedback to [email protected]

Signature · 2 lines
Wei Yang
Help you, Help me

--
Dr. David Alan Gilbert / [email protected] / Manchester, UK

#22 lore raw
Quoted 26 lines · Dr. David Alan Gilbert
Quoted 25 lines · * Wei Yang ([email protected]) wrote:
Quoted 23 lines · On Wed, May 01, 2019 at 09:24:06PM -0700, [email protected] wrote:

Patchew URL: https://patchew.org/QEMU/[email protected]/

Hi,

This series failed the asan build test. Please find the testing commands and their output below. If you have Docker installed, you can probably reproduce it locally.

=== TEST SCRIPT BEGIN ===
#!/bin/bash
time make docker-test-debug@fedora TARGET_LIST=x86_64-softmmu J=14 NETWORK=1 === TEST SCRIPT END ===

COPY RUNNER
RUN test-debug in qemu:fedora
container_linux.go:247: starting container process caused "process_linux.go:258: applying cgroup configuration for process caused \"The maximum number of active connections for UID 0 has been reached\"" /usr/bin/docker-current: Error response from daemon: oci runtime error: The maximum number of active connections for UID 0 has been reached.
Traceback (most recent call last):
File "./tests/docker/docker.py", line 615, in <module>
sys.exit(main())

Quoted 4 lines
Quoted 1 line

May I ask how I can reproduce this?

To me this just looks like patchew having a problem, I don't think it's a real qemu bug.

Ah, thanks.

Quoted 15 lines

Dave

Quoted 10 lines
Quoted 6 lines

The full log is available at
http://patchew.org/logs/[email protected]/testing.asan/?type=message.
---
Email generated automatically by Patchew [https://patchew.org/].
Please send your feedback to [email protected]

Signature · 2 lines
Wei Yang
Help you, Help me

--
Dr. David Alan Gilbert / [email protected] / Manchester, UK

Signature · 2 lines
Wei Yang
Help you, Help me
Reviewed-by on v1 1/3
#2 lore raw

Re: [Qemu-devel] [PATCH 1/3] migration/ram.c: start of migration_bitmap_sync_range is always 0

In reply to Wei Yang
Quoted 3 lines · * Wei Yang ([email protected]) wrote:

We can eliminate to pass 0.

Quoted 42 lines
---
 migration/ram.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)
migration/ram.c +4 -4
index 35bd6213e9..9948b2d021 100644
@@ -1643,10 +1643,10 @@ static inline bool migration_bitmap_clear_dirty(RAMState *rs,
 }
 
 static void migration_bitmap_sync_range(RAMState *rs, RAMBlock *rb,
-                                        ram_addr_t start, ram_addr_t length)
+                                        ram_addr_t length)
 {
     rs->migration_dirty_pages +=
-        cpu_physical_memory_sync_dirty_bitmap(rb, start, length,
+        cpu_physical_memory_sync_dirty_bitmap(rb, 0, length,
                                               &rs->num_dirty_pages_period);
 }
 
@@ -1735,7 +1735,7 @@ static void migration_bitmap_sync(RAMState *rs)
     qemu_mutex_lock(&rs->bitmap_mutex);
     rcu_read_lock();
     RAMBLOCK_FOREACH_NOT_IGNORED(block) {
-        migration_bitmap_sync_range(rs, block, 0, block->used_length);
+        migration_bitmap_sync_range(rs, block, block->used_length);
     }
     ram_counters.remaining = ram_bytes_remaining();
     rcu_read_unlock();
@@ -4156,7 +4156,7 @@ static void colo_flush_ram_cache(void)
     memory_global_dirty_log_sync();
     rcu_read_lock();
     RAMBLOCK_FOREACH_NOT_IGNORED(block) {
-        migration_bitmap_sync_range(ram_state, block, 0, block->used_length);
+        migration_bitmap_sync_range(ram_state, block, block->used_length);
     }
     rcu_read_unlock();
 
Signature · 1 line
2.19.1

--
Dr. David Alan Gilbert / [email protected] / Manchester, UK

#8 lore raw

Re: [Qemu-devel] [PATCH 2/3] migration/ram.c: start of cpu_physical_memory_sync_dirty_bitmap is always 0

In reply to Wei Yang
Quoted 4 lines · * Wei Yang ([email protected]) wrote:

Since start of cpu_physical_memory_sync_dirty_bitmap is always 0, we can remove this parameter and simplify the calculation a bit.

So I think you're right it's currently unused; however, lets ask
Paolo: Do we need to keep this parameter for flexiblity?

Dave

Quoted 77 lines
---
 include/exec/ram_addr.h | 15 ++++++---------
 migration/ram.c         |  2 +-
 2 files changed, 7 insertions(+), 10 deletions(-)
include/exec/ram_addr.h +6 -9
index 9ecd911c3e..3dfb2d52fb 100644
@@ -409,18 +409,16 @@ static inline void cpu_physical_memory_clear_dirty_range(ram_addr_t start,
 
 static inline
 uint64_t cpu_physical_memory_sync_dirty_bitmap(RAMBlock *rb,
-                                               ram_addr_t start,
                                                ram_addr_t length,
                                                uint64_t *real_dirty_pages)
 {
     ram_addr_t addr;
-    unsigned long word = BIT_WORD((start + rb->offset) >> TARGET_PAGE_BITS);
+    unsigned long word = BIT_WORD(rb->offset >> TARGET_PAGE_BITS);
     uint64_t num_dirty = 0;
     unsigned long *dest = rb->bmap;
 
-    /* start address and length is aligned at the start of a word? */
-    if (((word * BITS_PER_LONG) << TARGET_PAGE_BITS) ==
-         (start + rb->offset) &&
+    /* offset and length is aligned at the start of a word? */
+    if (((word * BITS_PER_LONG) << TARGET_PAGE_BITS) == (rb->offset) &&
         !(length & ((BITS_PER_LONG << TARGET_PAGE_BITS) - 1))) {
         int k;
         int nr = BITS_TO_LONGS(length >> TARGET_PAGE_BITS);
@@ -428,14 +426,13 @@ uint64_t cpu_physical_memory_sync_dirty_bitmap(RAMBlock *rb,
         unsigned long idx = (word * BITS_PER_LONG) / DIRTY_MEMORY_BLOCK_SIZE;
         unsigned long offset = BIT_WORD((word * BITS_PER_LONG) %
                                         DIRTY_MEMORY_BLOCK_SIZE);
-        unsigned long page = BIT_WORD(start >> TARGET_PAGE_BITS);
 
         rcu_read_lock();
 
         src = atomic_rcu_read(
                 &ram_list.dirty_memory[DIRTY_MEMORY_MIGRATION])->blocks;
 
-        for (k = page; k < page + nr; k++) {
+        for (k = 0; k < nr; k++) {
             if (src[idx][offset]) {
                 unsigned long bits = atomic_xchg(&src[idx][offset], 0);
                 unsigned long new_dirty;
@@ -458,11 +455,11 @@ uint64_t cpu_physical_memory_sync_dirty_bitmap(RAMBlock *rb,
 
         for (addr = 0; addr < length; addr += TARGET_PAGE_SIZE) {
             if (cpu_physical_memory_test_and_clear_dirty(
-                        start + addr + offset,
+                        addr + offset,
                         TARGET_PAGE_SIZE,
                         DIRTY_MEMORY_MIGRATION)) {
                 *real_dirty_pages += 1;
-                long k = (start + addr) >> TARGET_PAGE_BITS;
+                long k = addr >> TARGET_PAGE_BITS;
                 if (!test_and_set_bit(k, dest)) {
                     num_dirty++;
                 }
migration/ram.c +1 -1
index 9948b2d021..1def8122e9 100644
@@ -1646,7 +1646,7 @@ static void migration_bitmap_sync_range(RAMState *rs, RAMBlock *rb,
                                         ram_addr_t length)
 {
     rs->migration_dirty_pages +=
-        cpu_physical_memory_sync_dirty_bitmap(rb, 0, length,
+        cpu_physical_memory_sync_dirty_bitmap(rb, length,
                                               &rs->num_dirty_pages_period);
 }
 
Signature · 1 line
2.19.1

--
Dr. David Alan Gilbert / [email protected] / Manchester, UK

#3 lore raw

Re: [Qemu-devel] [PATCH 1/3] migration/ram.c: start of migration_bitmap_sync_range is always 0

In reply to Wei Yang
Quoted 3 lines · * Wei Yang ([email protected]) wrote:

We can eliminate to pass 0.

I'm going to queue just 1/3for the current pull.

Dave

Quoted 42 lines
---
 migration/ram.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)
migration/ram.c +4 -4
index 35bd6213e9..9948b2d021 100644
@@ -1643,10 +1643,10 @@ static inline bool migration_bitmap_clear_dirty(RAMState *rs,
 }
 
 static void migration_bitmap_sync_range(RAMState *rs, RAMBlock *rb,
-                                        ram_addr_t start, ram_addr_t length)
+                                        ram_addr_t length)
 {
     rs->migration_dirty_pages +=
-        cpu_physical_memory_sync_dirty_bitmap(rb, start, length,
+        cpu_physical_memory_sync_dirty_bitmap(rb, 0, length,
                                               &rs->num_dirty_pages_period);
 }
 
@@ -1735,7 +1735,7 @@ static void migration_bitmap_sync(RAMState *rs)
     qemu_mutex_lock(&rs->bitmap_mutex);
     rcu_read_lock();
     RAMBLOCK_FOREACH_NOT_IGNORED(block) {
-        migration_bitmap_sync_range(rs, block, 0, block->used_length);
+        migration_bitmap_sync_range(rs, block, block->used_length);
     }
     ram_counters.remaining = ram_bytes_remaining();
     rcu_read_unlock();
@@ -4156,7 +4156,7 @@ static void colo_flush_ram_cache(void)
     memory_global_dirty_log_sync();
     rcu_read_lock();
     RAMBLOCK_FOREACH_NOT_IGNORED(block) {
-        migration_bitmap_sync_range(ram_state, block, 0, block->used_length);
+        migration_bitmap_sync_range(ram_state, block, block->used_length);
     }
     rcu_read_unlock();
 
Signature · 1 line
2.19.1

--
Dr. David Alan Gilbert / [email protected] / Manchester, UK

author
#4 lore raw

Re: [Qemu-devel] [PATCH 1/3] migration/ram.c: start of migration_bitmap_sync_range is always 0

Quoted 6 lines · Dr. David Alan Gilbert
Quoted 3 lines · * Wei Yang ([email protected]) wrote:

We can eliminate to pass 0.

I'm going to queue just 1/3for the current pull.

In the pull request, I didn't see Patch[3]. Do I misunderstand this?

Quoted 3 lines

Dave

Signature · 2 lines
Wei Yang
Help you, Help me
#5 lore raw

Re: [Qemu-devel] [PATCH 1/3] migration/ram.c: start of migration_bitmap_sync_range is always 0

Quoted 9 lines · * Wei Yang ([email protected]) wrote:
Quoted 6 lines · Dr. David Alan Gilbert
Quoted 3 lines · * Wei Yang ([email protected]) wrote:

We can eliminate to pass 0.

I'm going to queue just 1/3for the current pull.

In the pull request, I didn't see Patch[3]. Do I misunderstand this?

Yes I meant only '[Patch 1/3]'.

Dave

Quoted 7 lines
Quoted 3 lines

Dave

Signature · 2 lines
Wei Yang
Help you, Help me

--
Dr. David Alan Gilbert / [email protected] / Manchester, UK

author
#6 lore raw

Re: [Qemu-devel] [PATCH 1/3] migration/ram.c: start of migration_bitmap_sync_range is always 0

Quoted 5 lines · Dr. David Alan Gilbert
Quoted 4 lines · * Wei Yang ([email protected]) wrote:
Quoted 3 lines · Dr. David Alan Gilbert
Quoted 2 lines · * Wei Yang ([email protected]) wrote:

We can eliminate to pass 0.

Quoted 7 lines
Quoted 5 lines
Quoted 3 lines
Quoted 1 line

I'm going to queue just 1/3for the current pull.

In the pull request, I didn't see Patch[3]. Do I misunderstand this?

Yes I meant only '[Patch 1/3]'.

Ah, thanks :-)

Quoted 12 lines

Dave

Quoted 7 lines
Quoted 3 lines

Dave

Signature · 2 lines
Wei Yang
Help you, Help me

--
Dr. David Alan Gilbert / [email protected] / Manchester, UK

Signature · 2 lines
Wei Yang
Help you, Help me
author
#9 lore raw

Re: [Qemu-devel] [PATCH 2/3] migration/ram.c: start of cpu_physical_memory_sync_dirty_bitmap is always 0

In reply to Dr. David Alan Gilbert
Quoted 9 lines · Dr. David Alan Gilbert
Quoted 4 lines · * Wei Yang ([email protected]) wrote:

Since start of cpu_physical_memory_sync_dirty_bitmap is always 0, we can remove this parameter and simplify the calculation a bit.

So I think you're right it's currently unused; however, lets ask
Paolo: Do we need to keep this parameter for flexiblity?

Hi Paolo

Do you like this one?

Quoted 81 lines

Dave

Quoted 77 lines
---
 include/exec/ram_addr.h | 15 ++++++---------
 migration/ram.c         |  2 +-
 2 files changed, 7 insertions(+), 10 deletions(-)
include/exec/ram_addr.h +6 -9
index 9ecd911c3e..3dfb2d52fb 100644
@@ -409,18 +409,16 @@ static inline void cpu_physical_memory_clear_dirty_range(ram_addr_t start,
 
 static inline
 uint64_t cpu_physical_memory_sync_dirty_bitmap(RAMBlock *rb,
-                                               ram_addr_t start,
                                                ram_addr_t length,
                                                uint64_t *real_dirty_pages)
 {
     ram_addr_t addr;
-    unsigned long word = BIT_WORD((start + rb->offset) >> TARGET_PAGE_BITS);
+    unsigned long word = BIT_WORD(rb->offset >> TARGET_PAGE_BITS);
     uint64_t num_dirty = 0;
     unsigned long *dest = rb->bmap;
 
-    /* start address and length is aligned at the start of a word? */
-    if (((word * BITS_PER_LONG) << TARGET_PAGE_BITS) ==
-         (start + rb->offset) &&
+    /* offset and length is aligned at the start of a word? */
+    if (((word * BITS_PER_LONG) << TARGET_PAGE_BITS) == (rb->offset) &&
         !(length & ((BITS_PER_LONG << TARGET_PAGE_BITS) - 1))) {
         int k;
         int nr = BITS_TO_LONGS(length >> TARGET_PAGE_BITS);
@@ -428,14 +426,13 @@ uint64_t cpu_physical_memory_sync_dirty_bitmap(RAMBlock *rb,
         unsigned long idx = (word * BITS_PER_LONG) / DIRTY_MEMORY_BLOCK_SIZE;
         unsigned long offset = BIT_WORD((word * BITS_PER_LONG) %
                                         DIRTY_MEMORY_BLOCK_SIZE);
-        unsigned long page = BIT_WORD(start >> TARGET_PAGE_BITS);
 
         rcu_read_lock();
 
         src = atomic_rcu_read(
                 &ram_list.dirty_memory[DIRTY_MEMORY_MIGRATION])->blocks;
 
-        for (k = page; k < page + nr; k++) {
+        for (k = 0; k < nr; k++) {
             if (src[idx][offset]) {
                 unsigned long bits = atomic_xchg(&src[idx][offset], 0);
                 unsigned long new_dirty;
@@ -458,11 +455,11 @@ uint64_t cpu_physical_memory_sync_dirty_bitmap(RAMBlock *rb,
 
         for (addr = 0; addr < length; addr += TARGET_PAGE_SIZE) {
             if (cpu_physical_memory_test_and_clear_dirty(
-                        start + addr + offset,
+                        addr + offset,
                         TARGET_PAGE_SIZE,
                         DIRTY_MEMORY_MIGRATION)) {
                 *real_dirty_pages += 1;
-                long k = (start + addr) >> TARGET_PAGE_BITS;
+                long k = addr >> TARGET_PAGE_BITS;
                 if (!test_and_set_bit(k, dest)) {
                     num_dirty++;
                 }
migration/ram.c +1 -1
index 9948b2d021..1def8122e9 100644
@@ -1646,7 +1646,7 @@ static void migration_bitmap_sync_range(RAMState *rs, RAMBlock *rb,
                                         ram_addr_t length)
 {
     rs->migration_dirty_pages +=
-        cpu_physical_memory_sync_dirty_bitmap(rb, 0, length,
+        cpu_physical_memory_sync_dirty_bitmap(rb, length,
                                               &rs->num_dirty_pages_period);
 }
 
Signature · 1 line
2.19.1

--
Dr. David Alan Gilbert / [email protected] / Manchester, UK

Signature · 2 lines
Wei Yang
Help you, Help me
author
#10 lore raw

Re: [Qemu-devel] [PATCH 2/3] migration/ram.c: start of cpu_physical_memory_sync_dirty_bitmap is always 0

In reply to Dr. David Alan Gilbert
Quoted 9 lines · Dr. David Alan Gilbert
Quoted 4 lines · * Wei Yang ([email protected]) wrote:

Since start of cpu_physical_memory_sync_dirty_bitmap is always 0, we can remove this parameter and simplify the calculation a bit.

So I think you're right it's currently unused; however, lets ask
Paolo: Do we need to keep this parameter for flexiblity?

Hi, Paolo,

Have time to take a look?

Quoted 81 lines

Dave

Quoted 77 lines
---
 include/exec/ram_addr.h | 15 ++++++---------
 migration/ram.c         |  2 +-
 2 files changed, 7 insertions(+), 10 deletions(-)
include/exec/ram_addr.h +6 -9
index 9ecd911c3e..3dfb2d52fb 100644
@@ -409,18 +409,16 @@ static inline void cpu_physical_memory_clear_dirty_range(ram_addr_t start,
 
 static inline
 uint64_t cpu_physical_memory_sync_dirty_bitmap(RAMBlock *rb,
-                                               ram_addr_t start,
                                                ram_addr_t length,
                                                uint64_t *real_dirty_pages)
 {
     ram_addr_t addr;
-    unsigned long word = BIT_WORD((start + rb->offset) >> TARGET_PAGE_BITS);
+    unsigned long word = BIT_WORD(rb->offset >> TARGET_PAGE_BITS);
     uint64_t num_dirty = 0;
     unsigned long *dest = rb->bmap;
 
-    /* start address and length is aligned at the start of a word? */
-    if (((word * BITS_PER_LONG) << TARGET_PAGE_BITS) ==
-         (start + rb->offset) &&
+    /* offset and length is aligned at the start of a word? */
+    if (((word * BITS_PER_LONG) << TARGET_PAGE_BITS) == (rb->offset) &&
         !(length & ((BITS_PER_LONG << TARGET_PAGE_BITS) - 1))) {
         int k;
         int nr = BITS_TO_LONGS(length >> TARGET_PAGE_BITS);
@@ -428,14 +426,13 @@ uint64_t cpu_physical_memory_sync_dirty_bitmap(RAMBlock *rb,
         unsigned long idx = (word * BITS_PER_LONG) / DIRTY_MEMORY_BLOCK_SIZE;
         unsigned long offset = BIT_WORD((word * BITS_PER_LONG) %
                                         DIRTY_MEMORY_BLOCK_SIZE);
-        unsigned long page = BIT_WORD(start >> TARGET_PAGE_BITS);
 
         rcu_read_lock();
 
         src = atomic_rcu_read(
                 &ram_list.dirty_memory[DIRTY_MEMORY_MIGRATION])->blocks;
 
-        for (k = page; k < page + nr; k++) {
+        for (k = 0; k < nr; k++) {
             if (src[idx][offset]) {
                 unsigned long bits = atomic_xchg(&src[idx][offset], 0);
                 unsigned long new_dirty;
@@ -458,11 +455,11 @@ uint64_t cpu_physical_memory_sync_dirty_bitmap(RAMBlock *rb,
 
         for (addr = 0; addr < length; addr += TARGET_PAGE_SIZE) {
             if (cpu_physical_memory_test_and_clear_dirty(
-                        start + addr + offset,
+                        addr + offset,
                         TARGET_PAGE_SIZE,
                         DIRTY_MEMORY_MIGRATION)) {
                 *real_dirty_pages += 1;
-                long k = (start + addr) >> TARGET_PAGE_BITS;
+                long k = addr >> TARGET_PAGE_BITS;
                 if (!test_and_set_bit(k, dest)) {
                     num_dirty++;
                 }
migration/ram.c +1 -1
index 9948b2d021..1def8122e9 100644
@@ -1646,7 +1646,7 @@ static void migration_bitmap_sync_range(RAMState *rs, RAMBlock *rb,
                                         ram_addr_t length)
 {
     rs->migration_dirty_pages +=
-        cpu_physical_memory_sync_dirty_bitmap(rb, 0, length,
+        cpu_physical_memory_sync_dirty_bitmap(rb, length,
                                               &rs->num_dirty_pages_period);
 }
 
Signature · 1 line
2.19.1

--
Dr. David Alan Gilbert / [email protected] / Manchester, UK

Signature · 2 lines
Wei Yang
Help you, Help me
author
#11 lore raw

Re: [Qemu-devel] [PATCH 2/3] migration/ram.c: start of cpu_physical_memory_sync_dirty_bitmap is always 0

In reply to Dr. David Alan Gilbert
Quoted 9 lines · Dr. David Alan Gilbert
Quoted 4 lines · * Wei Yang ([email protected]) wrote:

Since start of cpu_physical_memory_sync_dirty_bitmap is always 0, we can remove this parameter and simplify the calculation a bit.

So I think you're right it's currently unused; however, lets ask
Paolo: Do we need to keep this parameter for flexiblity?

Hey, Paolo

Do you have some spare time to give some insight?

Quoted 81 lines

Dave

Quoted 77 lines
---
 include/exec/ram_addr.h | 15 ++++++---------
 migration/ram.c         |  2 +-
 2 files changed, 7 insertions(+), 10 deletions(-)
include/exec/ram_addr.h +6 -9
index 9ecd911c3e..3dfb2d52fb 100644
@@ -409,18 +409,16 @@ static inline void cpu_physical_memory_clear_dirty_range(ram_addr_t start,
 
 static inline
 uint64_t cpu_physical_memory_sync_dirty_bitmap(RAMBlock *rb,
-                                               ram_addr_t start,
                                                ram_addr_t length,
                                                uint64_t *real_dirty_pages)
 {
     ram_addr_t addr;
-    unsigned long word = BIT_WORD((start + rb->offset) >> TARGET_PAGE_BITS);
+    unsigned long word = BIT_WORD(rb->offset >> TARGET_PAGE_BITS);
     uint64_t num_dirty = 0;
     unsigned long *dest = rb->bmap;
 
-    /* start address and length is aligned at the start of a word? */
-    if (((word * BITS_PER_LONG) << TARGET_PAGE_BITS) ==
-         (start + rb->offset) &&
+    /* offset and length is aligned at the start of a word? */
+    if (((word * BITS_PER_LONG) << TARGET_PAGE_BITS) == (rb->offset) &&
         !(length & ((BITS_PER_LONG << TARGET_PAGE_BITS) - 1))) {
         int k;
         int nr = BITS_TO_LONGS(length >> TARGET_PAGE_BITS);
@@ -428,14 +426,13 @@ uint64_t cpu_physical_memory_sync_dirty_bitmap(RAMBlock *rb,
         unsigned long idx = (word * BITS_PER_LONG) / DIRTY_MEMORY_BLOCK_SIZE;
         unsigned long offset = BIT_WORD((word * BITS_PER_LONG) %
                                         DIRTY_MEMORY_BLOCK_SIZE);
-        unsigned long page = BIT_WORD(start >> TARGET_PAGE_BITS);
 
         rcu_read_lock();
 
         src = atomic_rcu_read(
                 &ram_list.dirty_memory[DIRTY_MEMORY_MIGRATION])->blocks;
 
-        for (k = page; k < page + nr; k++) {
+        for (k = 0; k < nr; k++) {
             if (src[idx][offset]) {
                 unsigned long bits = atomic_xchg(&src[idx][offset], 0);
                 unsigned long new_dirty;
@@ -458,11 +455,11 @@ uint64_t cpu_physical_memory_sync_dirty_bitmap(RAMBlock *rb,
 
         for (addr = 0; addr < length; addr += TARGET_PAGE_SIZE) {
             if (cpu_physical_memory_test_and_clear_dirty(
-                        start + addr + offset,
+                        addr + offset,
                         TARGET_PAGE_SIZE,
                         DIRTY_MEMORY_MIGRATION)) {
                 *real_dirty_pages += 1;
-                long k = (start + addr) >> TARGET_PAGE_BITS;
+                long k = addr >> TARGET_PAGE_BITS;
                 if (!test_and_set_bit(k, dest)) {
                     num_dirty++;
                 }
migration/ram.c +1 -1
index 9948b2d021..1def8122e9 100644
@@ -1646,7 +1646,7 @@ static void migration_bitmap_sync_range(RAMState *rs, RAMBlock *rb,
                                         ram_addr_t length)
 {
     rs->migration_dirty_pages +=
-        cpu_physical_memory_sync_dirty_bitmap(rb, 0, length,
+        cpu_physical_memory_sync_dirty_bitmap(rb, length,
                                               &rs->num_dirty_pages_period);
 }
 
Signature · 1 line
2.19.1

--
Dr. David Alan Gilbert / [email protected] / Manchester, UK

Signature · 2 lines
Wei Yang
Help you, Help me
#12 lore raw

Re: [Qemu-devel] [PATCH 2/3] migration/ram.c: start of cpu_physical_memory_sync_dirty_bitmap is always 0

Quoted 7 lines · Wei Yang
Quoted 6 lines · Dr. David Alan Gilbert
Quoted 4 lines · * Wei Yang ([email protected]) wrote:

Since start of cpu_physical_memory_sync_dirty_bitmap is always 0, we can remove this parameter and simplify the calculation a bit.

Quoted 7 lines
Quoted 3 lines

So I think you're right it's currently unused; however, lets ask
Paolo: Do we need to keep this parameter for flexiblity?

Hey, Paolo

Do you have some spare time to give some insight?

I think it's cleaner to leave the start argument in place. However, I'll note that in migration_bitmap_sync_range the length argument is always block->used_length so that's also unnecessary (you already have block).

Paolo

Quoted 82 lines
Quoted 81 lines

Dave

Quoted 77 lines
---
 include/exec/ram_addr.h | 15 ++++++---------
 migration/ram.c         |  2 +-
 2 files changed, 7 insertions(+), 10 deletions(-)
include/exec/ram_addr.h +6 -9
index 9ecd911c3e..3dfb2d52fb 100644
@@ -409,18 +409,16 @@ static inline void cpu_physical_memory_clear_dirty_range(ram_addr_t start,
 
 static inline
 uint64_t cpu_physical_memory_sync_dirty_bitmap(RAMBlock *rb,
-                                               ram_addr_t start,
                                                ram_addr_t length,
                                                uint64_t *real_dirty_pages)
 {
     ram_addr_t addr;
-    unsigned long word = BIT_WORD((start + rb->offset) >> TARGET_PAGE_BITS);
+    unsigned long word = BIT_WORD(rb->offset >> TARGET_PAGE_BITS);
     uint64_t num_dirty = 0;
     unsigned long *dest = rb->bmap;
 
-    /* start address and length is aligned at the start of a word? */
-    if (((word * BITS_PER_LONG) << TARGET_PAGE_BITS) ==
-         (start + rb->offset) &&
+    /* offset and length is aligned at the start of a word? */
+    if (((word * BITS_PER_LONG) << TARGET_PAGE_BITS) == (rb->offset) &&
         !(length & ((BITS_PER_LONG << TARGET_PAGE_BITS) - 1))) {
         int k;
         int nr = BITS_TO_LONGS(length >> TARGET_PAGE_BITS);
@@ -428,14 +426,13 @@ uint64_t cpu_physical_memory_sync_dirty_bitmap(RAMBlock *rb,
         unsigned long idx = (word * BITS_PER_LONG) / DIRTY_MEMORY_BLOCK_SIZE;
         unsigned long offset = BIT_WORD((word * BITS_PER_LONG) %
                                         DIRTY_MEMORY_BLOCK_SIZE);
-        unsigned long page = BIT_WORD(start >> TARGET_PAGE_BITS);
 
         rcu_read_lock();
 
         src = atomic_rcu_read(
                 &ram_list.dirty_memory[DIRTY_MEMORY_MIGRATION])->blocks;
 
-        for (k = page; k < page + nr; k++) {
+        for (k = 0; k < nr; k++) {
             if (src[idx][offset]) {
                 unsigned long bits = atomic_xchg(&src[idx][offset], 0);
                 unsigned long new_dirty;
@@ -458,11 +455,11 @@ uint64_t cpu_physical_memory_sync_dirty_bitmap(RAMBlock *rb,
 
         for (addr = 0; addr < length; addr += TARGET_PAGE_SIZE) {
             if (cpu_physical_memory_test_and_clear_dirty(
-                        start + addr + offset,
+                        addr + offset,
                         TARGET_PAGE_SIZE,
                         DIRTY_MEMORY_MIGRATION)) {
                 *real_dirty_pages += 1;
-                long k = (start + addr) >> TARGET_PAGE_BITS;
+                long k = addr >> TARGET_PAGE_BITS;
                 if (!test_and_set_bit(k, dest)) {
                     num_dirty++;
                 }
migration/ram.c +1 -1
index 9948b2d021..1def8122e9 100644
@@ -1646,7 +1646,7 @@ static void migration_bitmap_sync_range(RAMState *rs, RAMBlock *rb,
                                         ram_addr_t length)
 {
     rs->migration_dirty_pages +=
-        cpu_physical_memory_sync_dirty_bitmap(rb, 0, length,
+        cpu_physical_memory_sync_dirty_bitmap(rb, length,
                                               &rs->num_dirty_pages_period);
 }
 
Signature · 1 line
2.19.1

--
Dr. David Alan Gilbert / [email protected] / Manchester, UK

#15 lore raw

Re: [Qemu-devel] [PATCH 3/3] ram: RAMBlock->offset is always aligned to a word

In reply to Wei Yang
Quoted 7 lines · Wei Yang

RAMBlock->offset is calculated by find_ram_offset, which makes sure the offset is aligned to a word.

This patch removes the alignment check on offset and unnecessary variable *word*.

I would add an assertion instead, but overall leaving the condition there is harmless. You still need the "else" part for the case where the length is unaligned.

Paolo

Quoted 37 lines
---
 include/exec/ram_addr.h | 15 +++++++++------
 1 file changed, 9 insertions(+), 6 deletions(-)
include/exec/ram_addr.h +9 -6
index 3dfb2d52fb..a7c81bdb32 100644
@@ -413,18 +413,21 @@ uint64_t cpu_physical_memory_sync_dirty_bitmap(RAMBlock *rb,
                                                uint64_t *real_dirty_pages)
 {
     ram_addr_t addr;
-    unsigned long word = BIT_WORD(rb->offset >> TARGET_PAGE_BITS);
     uint64_t num_dirty = 0;
     unsigned long *dest = rb->bmap;
 
-    /* offset and length is aligned at the start of a word? */
-    if (((word * BITS_PER_LONG) << TARGET_PAGE_BITS) == (rb->offset) &&
-        !(length & ((BITS_PER_LONG << TARGET_PAGE_BITS) - 1))) {
+    /*
+     * Since RAMBlock->offset is guaranteed to be aligned to a word by
+     * find_ram_offset(), if length is aligned at the start of a word, go the
+     * fast path.
+     */
+    if (!(length & ((BITS_PER_LONG << TARGET_PAGE_BITS) - 1))) {
         int k;
         int nr = BITS_TO_LONGS(length >> TARGET_PAGE_BITS);
         unsigned long * const *src;
-        unsigned long idx = (word * BITS_PER_LONG) / DIRTY_MEMORY_BLOCK_SIZE;
-        unsigned long offset = BIT_WORD((word * BITS_PER_LONG) %
+        unsigned long idx = (rb->offset >> TARGET_PAGE_BITS) /
+                            DIRTY_MEMORY_BLOCK_SIZE;
+        unsigned long offset = BIT_WORD((rb->offset >> TARGET_PAGE_BITS) %
                                         DIRTY_MEMORY_BLOCK_SIZE);
 
         rcu_read_lock();
author
#13 lore raw

Re: [Qemu-devel] [PATCH 2/3] migration/ram.c: start of cpu_physical_memory_sync_dirty_bitmap is always 0

In reply to Paolo Bonzini
Quoted 12 lines · Paolo Bonzini
Quoted 11 lines · Wei Yang
Quoted 9 lines · Dr. David Alan Gilbert
Quoted 4 lines · * Wei Yang ([email protected]) wrote:

Since start of cpu_physical_memory_sync_dirty_bitmap is always 0, we can remove this parameter and simplify the calculation a bit.

So I think you're right it's currently unused; however, lets ask
Paolo: Do we need to keep this parameter for flexiblity?

Quoted 9 lines
Quoted 3 lines

Hey, Paolo

Do you have some spare time to give some insight?

I think it's cleaner to leave the start argument in place. However, I'll note that in migration_bitmap_sync_range the length argument is always block->used_length so that's also unnecessary (you already have block).

That's reasonable.

Quoted 2 lines

Paolo

Signature · 2 lines
Wei Yang
Help you, Help me
author
#16 lore raw

Re: [Qemu-devel] [PATCH 3/3] ram: RAMBlock->offset is always aligned to a word

In reply to Paolo Bonzini
Quoted 4 lines · Paolo Bonzini
Quoted 3 lines · Wei Yang

RAMBlock->offset is calculated by find_ram_offset, which makes sure the offset is aligned to a word.

Quoted 9 lines
Quoted 4 lines

This patch removes the alignment check on offset and unnecessary variable *word*.

I would add an assertion instead, but overall leaving the condition there is harmless. You still need the "else" part for the case where the length is unaligned.

Thanks, then I would leave current code untouched.

Quoted 2 lines

Paolo

Signature · 2 lines
Wei Yang
Help you, Help me