Live Update [restart] : fork mode?
v5 cover Live Update [restart] : fork mode?
Hi, Steve
I have saw the discussion about the fork+exec mode below:
https://www.mail-archive.com/[email protected]/msg815956.html
And I am still very curious and I want to discuss about the possibility to support both fork+exec and exec in cpr framework.
1.Why
fork+exec could have some advantages and also drawbacks versus execvp() directly.
Advantages
i) fork+exec give the chance to fallback to original process even after we do exec which is important for workload seamless if any error happens. ii) smaller downtime since we could remove the vm_start() downtime out of the frozen window.
Drawbacks
i)need more codes to handle including fork,address/ports conflict between parent and child.
ii)more complex life cycle management for the two processes.
2.How
The cpr framework is flexible and scalable, and maybe we can make use of most codes to support both execvp and fork+exec mode. However, we may need to do more work compared to execvp method.
i) do fork mode in a thread like migration thread
ii) make parent and child talk to each other through socket or anonymous pipe
iii)make use of sharing mechanism of fds in cpr framework including memfd, vfio and devices fds
iv)deal with the conflict about the socket address and port like vnc (do by reuse port and pass the different args by cprexec)
v) do life cycle managements for two qemu processes and need parent exit and reconnection for the child at last by the management service
Please tell me if I am missing something important:)
Signature · 2 lines
Regards. Chuan
kindly ping?
Quoted 26 lines · Zheng Chuan
Hi, Steve
I have saw the discussion about the fork+exec mode below:
https://www.mail-archive.com/[email protected]/msg815956.htmlAnd I am still very curious and I want to discuss about the possibility to support both fork+exec and exec in cpr framework.
1.Why
fork+exec could have some advantages and also drawbacks versusexecvp()directly.
Advantages
i) fork+exec give the chance to fallback to original process even after we do exec which is important for workload seamless if any error happens. ii) smaller downtime since we could remove thevm_start()downtime out of the frozen window.
Drawbacks
i)need more codes to handle including fork,address/ports conflict between parent and child.
ii)more complex life cycle management for the two processes.2.How
The cpr framework is flexible and scalable, and maybe we can make use of most codes to support both execvp and fork+exec mode. However, we may need to do more work compared to execvp method.
i) do fork mode in a thread like migration thread
ii) make parent and child talk to each other through socket or anonymous pipe
iii)make use of sharing mechanism of fds in cpr framework including memfd, vfio and devices fds
iv)deal with the conflict about the socket address and port like vnc (do by reuse port and pass the different args by cprexec)
v) do life cycle managements for two qemu processes and need parent exit and reconnection for the child at last by the management servicePlease tell me if I am missing something important:)
Signature · 2 lines
Regards. Chuan
Quoted 24 lines · Zheng Chuan
Hi, Steve
I have saw the discussion about the fork+exec mode below:
https://www.mail-archive.com/[email protected]/msg815956.htmlAnd I am still very curious and I want to discuss about the possibility to support both fork+exec and exec in cpr framework.
1.Why
fork+exec could have some advantages and also drawbacks versusexecvp()directly.
Advantages
i) fork+exec give the chance to fallback to original process even after we do exec which is important for workload seamless if any error happens. ii) smaller downtime since we could remove thevm_start()downtime out of the frozen window.
Drawbacks
i)need more codes to handle including fork,address/ports conflict between parent and child.
ii)more complex life cycle management for the two processes.2.How
The cpr framework is flexible and scalable, and maybe we can make use of most codes to support both execvp and fork+exec mode. However, we may need to do more work compared to execvp method.
i) do fork mode in a thread like migration thread
ii) make parent and child talk to each other through socket or anonymous pipe
iii)make use of sharing mechanism of fds in cpr framework including memfd, vfio and devices fds
iv)deal with the conflict about the socket address and port like vnc (do by reuse port and pass the different args by cprexec)
v) do life cycle managements for two qemu processes and need parent exit and reconnection for the child at last by the management service
Quoted 1 line
Please tell me if I am missing something important:)
Hi Zheng, that list sounds about right. However, additional kernel changes would be needed to change ownership of the vfio device descriptors. The vfio module records the mm of the creating process, and does not allow other processes to unmap ranges. Also, the mm->locked_vm would need to be transferred to the new process.
Fork+exec could be a new mode to the cprsave command.
- Steve
Hi, Steve.
Quoted 25 lines · Steven Sistare
Quoted 24 lines · Zheng Chuan
Hi, Steve
I have saw the discussion about the fork+exec mode below:
https://www.mail-archive.com/[email protected]/msg815956.htmlAnd I am still very curious and I want to discuss about the possibility to support both fork+exec and exec in cpr framework.
1.Why
fork+exec could have some advantages and also drawbacks versusexecvp()directly.
Advantages
i) fork+exec give the chance to fallback to original process even after we do exec which is important for workload seamless if any error happens. ii) smaller downtime since we could remove thevm_start()downtime out of the frozen window.
Drawbacks
i)need more codes to handle including fork,address/ports conflict between parent and child.
ii)more complex life cycle management for the two processes.2.How
The cpr framework is flexible and scalable, and maybe we can make use of most codes to support both execvp and fork+exec mode. However, we may need to do more work compared to execvp method.
i) do fork mode in a thread like migration thread
ii) make parent and child talk to each other through socket or anonymous pipe
iii)make use of sharing mechanism of fds in cpr framework including memfd, vfio and devices fds
iv)deal with the conflict about the socket address and port like vnc (do by reuse port and pass the different args by cprexec)
v) do life cycle managements for two qemu processes and need parent exit and reconnection for the child at last by the management service
Quoted 7 lines
Quoted 1 line
Please tell me if I am missing something important:)
Hi Zheng, that list sounds about right. However, additional kernel changes would be needed to change ownership of the vfio device descriptors. The vfio module records the mm of the creating process, and does not allow other processes to unmap ranges. Also, the
mm->locked_vmwould need to be transferred to the new process.
Oh,That's right. Maybe we could update them along with VFIO_DMA_MAP_FLAG_VADDR.
Quoted 2 lines
Fork+exec could be a new mode to the cprsave command.
Yeah, I would be very happy to see it in the future:)
Quoted 3 lines
- Steve
.
Signature · 2 lines
Regards. Chuan