[Chapel Merge] Optimize remote array copies by creating a local copy of the domain when possible (off by default)

Branch: refs/heads/main
Revision: ed7a3e51f5fa586d058237aa8dcabe95c0bd002b
Author: bradcray
Link: Optimize remote array copies by creating a local copy of the domain when possible (off by default) by bradcray · Pull Request #24391 · chapel-lang/chapel · GitHub
Log Message:
Optimize remote array copies by creating a local copy of the domain when possible (off by default) (#24391)

[reviewed by @jabraham17]

This PR adds an optimization that localizes an array's domain when a
copy of the array is made across locales. For example, given a pattern
like:

var D = {1..100, 1..100};
var A: [D] real;
on Locales[1] {
  var B = A;
  B[i] = …;
}

B will typically share A's domain so that if D is reassigned (resized),
B will reflect that. This has the downside that operations on 'A' may
need to refer back to 'D' in order to get its size, bounds, etc.

However, if D is declared const, it cannot be re-assigned, so rather
than referring back to the original domain, a local copy of D could be
created and B could be declared over that, permitting operations on 'B'
to be done without referring to a remote locale. Effectively, the
compiler could translate the code into:

const D = {1..100, 1..100};
var A: [D] real;
on Locales[1] {
  const localD = D;
  var B:[D] A.eltType = A;
  B[i] = …;
}

Similarly, if 'B' is const, the same optimization can be done because
we say that the domain of a const array cannot be re-assigned while
the array is alive.

This PR implements this optimization in module code and adds a few tests
of it. It also updates one existing test that locked in the old
behavior, by testing it with and without the optimization.

The optimization is currently off by default because we are close to the
2.1 deadline and want some more time to live with it and understand the
implications before merging. That said, it passed a full paratest GASNet
run. I've forked off Turn "localize array's remote domain" optimization on by default · Issue #25238 · chapel-lang/chapel · GitHub
to capture the intention to enable this by default and some of the
things we'd discussed checking before doing so.

Thanks to @jabraham17 for the conversation that led to the insight that
this might be a simple optimization (it was) after years of having
people hit this issue and be surprised by it (see issue #13213, e.g.).
And to @e-kayrakli and others on the @chapel-lang/perf-team for feedback
on the initial draft.

Compare: Comparing bd65c6dc3993127df1c35ca16033fabe05ee9c1c...2317ed8728a2bc569759e91a11020597d1b056f5 · chapel-lang/chapel · GitHub

Diff:
M modules/internal/ChapelArray.chpl
A test/arrays/localizeConstDom.chpl
A test/arrays/localizeConstDom.comm-none.good
A test/arrays/localizeConstDom.compopts
A test/arrays/localizeConstDom.good
A test/arrays/localizeConstDom.numlocales
A test/distributions/block/checkLocalizeConstBlockDom.chpl
A test/distributions/block/checkLocalizeConstBlockDom.compopts
A test/distributions/block/checkLocalizeConstBlockDom.good
A test/multilocale/diten/localBlock/needMultiLocales/localBlock5-localize.good
A test/multilocale/diten/localBlock/needMultiLocales/localBlock5.compopts
https://github.com/chapel-lang/chapel/pull/24391.diff