Skip to contentSkip to conversation

Live Update [restart] : fork mode?

PATCH In review Zheng Chuan 2 participants 4 messages Jul 30, 2021 – Aug 5, 2021

lore
Share & tools
lore
#0 lore raw

v5 cover Live Update [restart] : fork mode?

Hi, Steve
I have saw the discussion about the fork+exec mode below:
https://www.mail-archive.com/[email protected]/msg815956.html

And I am still very curious and I want to discuss about the possibility to support both fork+exec and exec in cpr framework.

1.Why
fork+exec could have some advantages and also drawbacks versus execvp() directly.
Advantages
i) fork+exec give the chance to fallback to original process even after we do exec which is important for workload seamless if any error happens. ii) smaller downtime since we could remove the vm_start() downtime out of the frozen window.
Drawbacks
i)need more codes to handle including fork,address/ports conflict between parent and child.
ii)more complex life cycle management for the two processes.

2.How
The cpr framework is flexible and scalable, and maybe we can make use of most codes to support both execvp and fork+exec mode. However, we may need to do more work compared to execvp method.
i) do fork mode in a thread like migration thread
ii) make parent and child talk to each other through socket or anonymous pipe
iii)make use of sharing mechanism of fds in cpr framework including memfd, vfio and devices fds
iv)deal with the conflict about the socket address and port like vnc (do by reuse port and pass the different args by cprexec)
v) do life cycle managements for two qemu processes and need parent exit and reconnection for the child at last by the management service

Please tell me if I am missing something important:)

Signature · 2 lines
Regards.
Chuan
#1 lore raw

kindly ping?

Quoted 26 lines · Zheng Chuan

Hi, Steve
I have saw the discussion about the fork+exec mode below:
https://www.mail-archive.com/[email protected]/msg815956.html

And I am still very curious and I want to discuss about the possibility to support both fork+exec and exec in cpr framework.

1.Why
fork+exec could have some advantages and also drawbacks versus execvp() directly.
Advantages
i) fork+exec give the chance to fallback to original process even after we do exec which is important for workload seamless if any error happens. ii) smaller downtime since we could remove the vm_start() downtime out of the frozen window.
Drawbacks
i)need more codes to handle including fork,address/ports conflict between parent and child.
ii)more complex life cycle management for the two processes.

2.How
The cpr framework is flexible and scalable, and maybe we can make use of most codes to support both execvp and fork+exec mode. However, we may need to do more work compared to execvp method.
i) do fork mode in a thread like migration thread
ii) make parent and child talk to each other through socket or anonymous pipe
iii)make use of sharing mechanism of fds in cpr framework including memfd, vfio and devices fds
iv)deal with the conflict about the socket address and port like vnc (do by reuse port and pass the different args by cprexec)
v) do life cycle managements for two qemu processes and need parent exit and reconnection for the child at last by the management service

Please tell me if I am missing something important:)

Signature · 2 lines
Regards.
Chuan
#2 lore raw
In reply to Zheng Chuan
Quoted 24 lines · Zheng Chuan

Hi, Steve
I have saw the discussion about the fork+exec mode below:
https://www.mail-archive.com/[email protected]/msg815956.html

And I am still very curious and I want to discuss about the possibility to support both fork+exec and exec in cpr framework.

1.Why
fork+exec could have some advantages and also drawbacks versus execvp() directly.
Advantages
i) fork+exec give the chance to fallback to original process even after we do exec which is important for workload seamless if any error happens. ii) smaller downtime since we could remove the vm_start() downtime out of the frozen window.
Drawbacks
i)need more codes to handle including fork,address/ports conflict between parent and child.
ii)more complex life cycle management for the two processes.

2.How
The cpr framework is flexible and scalable, and maybe we can make use of most codes to support both execvp and fork+exec mode. However, we may need to do more work compared to execvp method.
i) do fork mode in a thread like migration thread
ii) make parent and child talk to each other through socket or anonymous pipe
iii)make use of sharing mechanism of fds in cpr framework including memfd, vfio and devices fds
iv)deal with the conflict about the socket address and port like vnc (do by reuse port and pass the different args by cprexec)
v) do life cycle managements for two qemu processes and need parent exit and reconnection for the child at last by the management service

Quoted 1 line

Please tell me if I am missing something important:)

Hi Zheng, that list sounds about right. However, additional kernel changes would be needed to change ownership of the vfio device descriptors. The vfio module records the mm of the creating process, and does not allow other processes to unmap ranges. Also, the mm->locked_vm would need to be transferred to the new process.

Fork+exec could be a new mode to the cprsave command.

- Steve

#3 lore raw

Hi, Steve.

Quoted 25 lines · Steven Sistare
Quoted 24 lines · Zheng Chuan

Hi, Steve
I have saw the discussion about the fork+exec mode below:
https://www.mail-archive.com/[email protected]/msg815956.html

And I am still very curious and I want to discuss about the possibility to support both fork+exec and exec in cpr framework.

1.Why
fork+exec could have some advantages and also drawbacks versus execvp() directly.
Advantages
i) fork+exec give the chance to fallback to original process even after we do exec which is important for workload seamless if any error happens. ii) smaller downtime since we could remove the vm_start() downtime out of the frozen window.
Drawbacks
i)need more codes to handle including fork,address/ports conflict between parent and child.
ii)more complex life cycle management for the two processes.

2.How
The cpr framework is flexible and scalable, and maybe we can make use of most codes to support both execvp and fork+exec mode. However, we may need to do more work compared to execvp method.
i) do fork mode in a thread like migration thread
ii) make parent and child talk to each other through socket or anonymous pipe
iii)make use of sharing mechanism of fds in cpr framework including memfd, vfio and devices fds
iv)deal with the conflict about the socket address and port like vnc (do by reuse port and pass the different args by cprexec)
v) do life cycle managements for two qemu processes and need parent exit and reconnection for the child at last by the management service

Quoted 7 lines
Quoted 1 line

Please tell me if I am missing something important:)

Hi Zheng, that list sounds about right. However, additional kernel changes would be needed to change ownership of the vfio device descriptors. The vfio module records the mm of the creating process, and does not allow other processes to unmap ranges. Also, the mm->locked_vm would need to be transferred to the new process.

Oh,That's right. Maybe we could update them along with VFIO_DMA_MAP_FLAG_VADDR.

Quoted 2 lines

Fork+exec could be a new mode to the cprsave command.

Yeah, I would be very happy to see it in the future:)

Quoted 3 lines

- Steve
.

Signature · 2 lines
Regards.
Chuan