Skip to contentSkip to conversation

Direct access to GPGPU do-able?

DISCUSSION Carsten Aulbert 4 participants 4 messages Dec 1, 2009 – Feb 25, 2010 KVM

lore
Share & tools
lore
#0 lore raw

Hi,

I'll start with a one-off question here, so please cc me on the reply.

We are running a largish cluster and are currently buying GPGPU systems (Tesla and soon Fermi based). We will have at least 2 possibly 4 of these cards per box and have the problem that some codes need different CUDA kernel drivers to run. As these boxes have 4 CPU cores, 12 GB of memory and CPU-VT support we thought that this might be solvable by creating (para-) virtualized guests on the boxes and passing one GPGPU device into a guest at a time. In there we then can run any kernel/driver combo necessary.

But since my current virtualization experience only stretches to OpenVZ and VirtualBox (tinkering with Xen a couple of years back), I don't know if KVM is the right approach here. We need something which we can automatically set-up via CLI, i.e. starting and stopping the guests need to be fully automatic, we don't need a graphical environment within the guests, just plain text is good enough.

What do you think, is looking at KVM the right choice for this? Can we pass a device directly into a guest?

Cheers

Carsten

Signature · 4 lines
Dr. Carsten Aulbert - Max Planck Institute for Gravitational Physics
Callinstrasse 38, 30167 Hannover, Germany
Phone/Fax: +49 511 762-17185 / -17193
http://www.top500.org/system/9234 | http://www.top500.org/connfam/6/list/3
#1 lore raw
Quoted 10 lines · Carsten Aulbert

Hi,

I'll start with a one-off question here, so please cc me on the reply.

We are running a largish cluster and are currently buying GPGPU systems (Tesla and soon Fermi based). We will have at least 2 possibly 4 of these cards per box and have the problem that some codes need different CUDA kernel drivers to run. As these boxes have 4 CPU cores, 12 GB of memory and CPU-VT support we thought that this might be solvable by creating (para-) virtualized guests on the boxes and passing one GPGPU device into a guest at a time. In there we then can run any kernel/driver combo necessary.

But since my current virtualization experience only stretches to OpenVZ and VirtualBox (tinkering with Xen a couple of years back), I don't know if KVM is the right approach here. We need something which we can automatically set-up via CLI, i.e. starting and stopping the guests need to be fully automatic, we don't need a graphical environment within the guests, just plain text is good enough.

What do you think, is looking at KVM the right choice for this? Can we pass a device directly into a guest?

KVM already supports device assignment (e.g. NIC) to guest, but graphics card assignment is not supported yet.

Federico, you said you were porting xen graphics assignment patches to kvm, what's your progress?

Regards,
Weidong

Quoted 5 lines

Cheers

Carsten

#2 lore raw

I analyzed both XEN patches and determined where this patch must be done in KVM. It's just a matter of writing the code.

Currently I'm studying for some final exams, so I will take some time off before I continue with this port.

And because of hardware related problems, if it's not done in two or three weeks, then it will be finished in february.

Regards,
Federico

Quoted 44 lines · Weidong Han
Quoted 19 lines · Carsten Aulbert

Hi,

I'll start with a one-off question here, so please cc me on the reply. We are running a largish cluster and are currently buying GPGPU systems (Tesla and soon Fermi based). We will have at least 2 possibly 4 of these cards per box and have the problem that some codes need different CUDA kernel drivers to run. As these boxes have 4 CPU cores, 12 GB of memory and CPU-VT support we thought that this might be solvable by creating (para-) virtualized guests on the boxes and passing one GPGPU device into a guest at a time. In there we then can run any kernel/driver combo necessary.

But since my current virtualization experience only stretches to OpenVZ and VirtualBox (tinkering with Xen a couple of years back), I don't know if KVM is the right approach here. We need something which we can automatically set-up via CLI, i.e. starting and stopping the guests need to be fully automatic, we don't need a graphical environment within the guests, just plain text is good enough.

Quoted 3 lines

What do you think, is looking at KVM the right choice for this? Can we pass a device directly into a guest?

KVM already supports device assignment (e.g. NIC) to guest, but graphics card assignment is not supported yet.

Federico, you said you were porting xen graphics assignment patches to kvm, what's your progress?

Regards,
Weidong

Quoted 5 lines

Cheers

Carsten

--
To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to [email protected]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

#3 lore raw

Hi Federico,

Quoted 8 lines · Fede

I analyzed both XEN patches and determined where this patch must be done in KVM. It's just a matter of writing the code.

Currently I'm studying for some final exams, so I will take some time off before I continue with this port.

And because of hardware related problems, if it's not done in two or three weeks, then it will be finished in february.

Did you find some time to start working on the patches for KVM, so graphic cards will be initialized properly inside VMs?

Regards
André