[Chapel Merge] Fence the cache on calls to chpl_comm_barrier

Branch: refs/heads/main
Revision: 04af203
Author: ronawho
Link: Fence the cache on calls to chpl_comm_barrier by ronawho · Pull Request #18802 · chapel-lang/chapel · GitHub
Log Message:

Merge pull request #18802 from ronawho/fence-cache-chpl_comm_barrier

Fence the cache on calls to chpl_comm_barrier

[reviewed by @mppf and @gbtitus]

chpl_comm_barrier is a full synchronization point so we need to fence
the cache when hitting it. We speculated this may be needed in
Cray/chapel-private#1406, but didn't think it was actually a problem
since the AllLocalesBarrier that uses this from module code will use
atomics before calling the comm barrier. However, we recently discovered
that this was causing stale values to be read from the cache during
start up in chpl_setMemFlags. That code has a GET when memlog is
non-empty, but the GET was using a stale cached value from earlier in
locale model init. This mem flag code is surrounded by barriers but
since we weren't fencing on barrier calls before we ran into problems.
Adjust the barrier to fence now to correct this issue.

Some barriers are now calling the cache before the cache_init is done.
This exposed bugs in the cache code to make pre-init operations no-ops
when using non-native thread-local-storage, so work around that here.

Resolves Cray/chapel-private#1406
Resolves Cray/chapel-private#2763
Resolves Cray/chapel-private#2768

Modified Files:
M runtime/include/chpl-cache.h

M runtime/include/chpl-comm.h
M runtime/src/chpl-cache.c
M runtime/src/comm/gasnet/comm-gasnet.c
M runtime/src/comm/none/comm-none.c
M runtime/src/comm/ofi/comm-ofi.c
M runtime/src/comm/ugni/comm-ugni.c
M test/memory/gbt/test-memLog.skipif

Compare: https://github.com/chapel-lang/chapel/compare/e19ef3575812...04af2038c032