[Chapel Merge] Support multiple GPUs in one node

Notifications Commits
1

Branch: refs/heads/main
Revision: 1a4620d
Author: e-kayrakli
Link: Support multiple GPUs in one node by e-kayrakli · Pull Request #19438 · chapel-lang/chapel · GitHub
Log Message:

Merge pull request #19438 from e-kayrakli/gpu-multi

Support multiple GPUs in one node

This PR overhauls the context creation to be able to handle multiple GPUs.

Previously, we were using cuCtxCreate to create new CUDA contexts. However,
this had a TODO for us to take a look at primary contexts. Primary contexts are
retained per device and the recommended way for interacting with devices.

With this PR, we initialize GPU layer eagerly at runtime init time. During this
initialization, we create and retain a context per device. These contexts are
stored in an array where the sublocale ID can be used to access an appropriate
context that will use an appropriate device.

We also adjust chpl_gpu_ensure_context to push/pop contexts as necessary.

Future work:
CUDA contexts are CPU-thread-private. We need to establish/understand how CPU
thread to GPU mappings work, and potentially do some pinning. If we can do that,
some of the stuff in this PR can be removed.

[Reviewed by @daviditen]

Test:

  • [x] gpu/native

    Modified Files:
    A test/gpu/native/multiGPU/multiGPU.chpl
    A test/gpu/native/multiGPU/multiGPU.execopts
    A test/gpu/native/multiGPU/multiGPU.good
    A test/gpu/native/multiGPU/multiGPU.prediff
    M runtime/src/chpl-gpu.c
    M runtime/src/chpl-init.c
    M test/gpu/native/gpuAddNums/gpuAddNums_primitive.chpl
    M test/gpu/native/threadBlockAndGridPrimitives.chpl

    Compare: Comparing 2a69071a9806...1a4620d0f66c · chapel-lang/chapel · GitHub