See CUDA kernels after compilation?

Is there a way to see the CUDA kernel code that the Chapel compiler makes?

Hi @Iainmon1,

We don't generate pure CUDA code, instead we use LLVM to generate assembly code for the GPU.

If you want to inspect the generated Chapel code, you can pass --savec <tmp dir name> to save all the generated code into a directory. To see what kind of code your Chapel kernels have generated, the following files will be of interest to you

  • chpl__gpu_module-nopt.bc: the generated LLVM IR of the GPU code, before any LLVM optimization passes are run
  • chpl__gpu_module-opt1.bc: the generated LLVM IR of the GPU code, after LLVM optimization passes are run
  • chpl__gpu.s: the PTX assembly code that will be run on the GPU.

The *.bc files are LLVM bitcode, to get the human readable version you will need to disassemble it with llvm-dis, which should be in your LLVM install (either with CHPL_LLVM=system or bundled).

Unfortunately, all of these methods require you to read either LLVM IR or PTX assembly code, neither of which can be the most human friendly.

If you are comparing against an existing CUDA kernel, you may find it interesting to pass -ptx to your nvcc command. This will change your output file to be the generated PTX assembly. Then you can compare the nvcc generated PTX next to the chpl/llvm PTX.

Hope this helps,
Jade

Hi Jade,

Thank you for your reply. That does make sense that there isn't a CUDA kernel being written by the Chapel compiler, then being compiled. But what you mentioned is certainly helpful and I will explore that functionality.

Thanks!
Iain