[Chapel Merge] Fix an MCM issue between AMs and PUTs in ofi msg-o

Notifications Commits
1

Branch: refs/heads/release/1.26
Revision: 06b1198
Author: ronawho
Link: Unavailable
Log Message:

Merge pull request #19578 from ronawho/fix-ofi-fenced-am-cp

Fix an MCM issue between AMs and PUTs in ofi msg-order-fence (cherry-pick #19575)

Fix a race where an AM wasn't waiting for outstanding PUTs to complete
for ofi message-order-fence. For a case like:

var a = 0;
on Locales[numLocales-1] {
  a = 1;
  on Locales[0] {
    assert(a == 1);
  }
}

the AM (on Locales[0]) wasn't waiting for the pending PUT (a = 1) to
complete. The comments in the code indicate we thought we were handling
this correctly by issuing our AM with FI_FENCE to wait for any pending
operations, but that does not appear to be happening.

Here I'm applying a hotfix to extend how we force message visibility to
stop skipping the target node. We want to understand what's going on
here better, but for now I just want to get a base fix in to address
failures we're seeing.

Also add a CHANGES.md entry since we're tying to land this hotfix in the
1.26 release.

Modified Files:
M CHANGES.md

M runtime/src/comm/ofi/comm-ofi.c

Compare: https://github.com/chapel-lang/chapel/compare/b558a9ef18f8...06b1198a721e