New Issue: [Proposal] Add a low-level 'Communication' module

17657, "e-kayrakli", "[Proposal] Add a low-level 'Communication' module", "2021-05-07T19:05:57Z"

This issue proposes addition of a low-level communication module which initially
allows point-to-point raw data copies.

Module name: Communication

  • It captures the main purpose of the module
  • It can contain collectives in the future either as top-level functions or in a
  • Alternatives I thought about:
    • Comm
      • I like this, but the user can also do import Communication as Comm
      • Consistent with CommDiagnostics module, though it is not a stable
    • Multilocale
      • Too general, doesn't tell much
    • LowLevelCommunication or LowLevelComm
      • Not sure if we need "LowLevel" in the name
    • SPMD
      • Not necessarily limited to SPMD-like idioms


I propose we only have 1 function in the first step:

proc copy(ref dst: ?t, const ref src: t, numElems: int, useCache=true)

which copies numElems*numBytes(t) bytes from the src address to dst,
regardless of whether src/dst being local/remote.


  • We don't want to have get/put as they only differ w.r.t. location
    of source/destination. Let's not make the user worry about it.
  • On similar things we use "copy":
    • c_memcpy
    • we call the compiler-driven aggregation "Automatic Copy Aggregation"
    • We have the UnorderedCopy module (which we might incorporate into this
      module in the future. See below)
    • Arkouda's aggregators have a copy method


  • I think we should have numElems not numBytes
  • dst/src is consistent with C's memcpy and sufficiently clear to me.
    • I wouldn't oppose to lhs/rhs, too. But they are too specific to
      assignments, which may not be how an MPI-minded programmer think about this
  • useCache: I think we can do some overloading tricks to check for
    --cache-remote, as we don't want useCache=true with --no-cache-remote.
    Similar to controlling the caching behavior, we may need more kinds of copys
    in the future. See below.
  • The contexts in which the need for this function came up has been about
    primitive types, so far. We may want to extend to POD/non-POD types down the
    road but current interface shouldn't block that from happening I think. See
    below for more discussion about data types.