Branch: refs/heads/master
Revision: 2f5c4f3
Author: ronawho
Log Message:
Merge pull request #17505 from ronawho/upgrade-gasnet-ex-2012.3.0
Upgrade to GASNet-EX 2021.3.0
This includes significant performance and scalability improvements for
InfiniBand networks when using dynamic registration (our default mode),
a powerpc race fix (previously patched in #17083), and improvements to
the ssh spawner. There's also initial support for "Memory Kinds" to
support communication using non-host memory (e.g. GPUs). The ofi
conduit has also been re-added though is currently experimental.
For the full changelog see: https://gasnet.lbl.gov/dist-ex/ChangeLog
InfiniBand performance is significantly improved by increasing the
number of memory registration entries and increasing the size of the
hash table used to track registration entries. This will improve some
long-standing performance issues for applications that communicate a
large fraction of memory like ISx and NPB-FT as well as applications
that increase the amount of communicated regions as scale increases like
Arkouda. https://bitbucket.org/berkeleylab/gasnet/pull-requests/433 has
more detail on this specific effort. I'm expecting large improvements
for array copies and ISx/NPB-FT. 16-node-cs-hdr performance results:
Array copy performance (PUTs)
config 1 MiB 1 GiB 48 GiB before ~10500 MiB/s ~4000 MiB/s ~1450 MiB/s after ~10500 MiB/s ~11700 MiB/s ~3250 MiB/s
ISx/NPB-FT:
config ISx NPB-FT before 50s 115 GFLOPS after 10s 430 GFLOPS
Resolves https://github.com/Cray/chapel-private/issues/1411
Resolves https://github.com/Cray/chapel-private/issues/1532
Helps Improve gasnet-ibv performance · Issue #14438 · chapel-lang/chapel · GitHub
Helps Tune gasnet-ibv-large/firehose bulk-comm performance · Issue #14439 · chapel-lang/chapel · GitHub
Helps https://github.com/Cray/chapel-private/issues/1792
Helps https://github.com/Cray/chapel-private/issues/1819
Modified Files:
A third-party/gasnet/gasnet-src/docs/memory_kinds.md
A third-party/gasnet/gasnet-src/other/hwloc/gasnet_hwloc.c
A third-party/gasnet/gasnet-src/other/hwloc/gasnet_hwloc_internal.h
A third-party/gasnet/gasnet-src/other/kinds/gasnet_cuda_uva.c
A third-party/gasnet/gasnet-src/other/kinds/gasnet_kinds_internal.h
A third-party/gasnet/gasnet-src/other/kinds/gasnet_mk.h
A third-party/gasnet/gasnet-src/other/kinds/gasnet_refkinds.c
A third-party/gasnet/gasnet-src/tests/testcudauva.c
A third-party/gasnet/gasnet-src/tests/testsegment.c
A third-party/gasnet/gasnet-src/tests/testtmpair.c
M third-party/gasnet/README
M third-party/gasnet/gasnet-src/ChangeLog
M third-party/gasnet/gasnet-src/Makefile.am
M third-party/gasnet/gasnet-src/Makefile.in
M third-party/gasnet/gasnet-src/README
M third-party/gasnet/gasnet-src/README-tools
M third-party/gasnet/gasnet-src/acconfig.h
M third-party/gasnet/gasnet-src/acinclude.m4
M third-party/gasnet/gasnet-src/aries-conduit/Makefile.am
M third-party/gasnet/gasnet-src/aries-conduit/Makefile.in
M third-party/gasnet/gasnet-src/aries-conduit/README
M third-party/gasnet/gasnet-src/aries-conduit/contrib/Makefile.in
M third-party/gasnet/gasnet-src/aries-conduit/gasnet_aries.c
M third-party/gasnet/gasnet-src/aries-conduit/gasnet_aries.h
M third-party/gasnet/gasnet-src/aries-conduit/gasnet_core.c
M third-party/gasnet/gasnet-src/aries-conduit/gasnet_core.h
M third-party/gasnet/gasnet-src/aries-conduit/gasnet_core_fwd.h
M third-party/gasnet/gasnet-src/aries-conduit/gasnet_core_internal.h
M third-party/gasnet/gasnet-src/aries-conduit/gasnet_extended.c
M third-party/gasnet/gasnet-src/aries-conduit/gasnet_extended_fwd.h
M third-party/gasnet/gasnet-src/aries-conduit/gasnet_ratomic.c
M third-party/gasnet/gasnet-src/aries-conduit/gasnet_ratomic_fwd.h
M third-party/gasnet/gasnet-src/config-aux/Makefile.in
M third-party/gasnet/gasnet-src/config-aux/config.guess
M third-party/gasnet/gasnet-src/config-aux/config.sub
M third-party/gasnet/gasnet-src/configure
M third-party/gasnet/gasnet-src/configure.in
M third-party/gasnet/gasnet-src/docs/GASNet-EX.txt
M third-party/gasnet/gasnet-src/extended-ref/coll/gasnet_autotune.c
M third-party/gasnet/gasnet-src/extended-ref/coll/gasnet_coll.h
M third-party/gasnet/gasnet-src/extended-ref/coll/gasnet_coll_internal.h
M third-party/gasnet/gasnet-src/extended-ref/coll/gasnet_reduce.c
M third-party/gasnet/gasnet-src/extended-ref/coll/gasnet_refcoll.c
M third-party/gasnet/gasnet-src/extended-ref/coll/gasnet_refcoll.h
M third-party/gasnet/gasnet-src/extended-ref/coll/gasnet_team.c
M third-party/gasnet/gasnet-src/extended-ref/gasnet_extended.h
M third-party/gasnet/gasnet-src/extended-ref/gasnet_extended_help.h
M third-party/gasnet/gasnet-src/extended-ref/gasnet_extended_refbarrier.c
M third-party/gasnet/gasnet-src/extended-ref/ratomic/gasnet_ratomic.h
M third-party/gasnet/gasnet-src/extended-ref/ratomic/gasnet_ratomic_fwd.h
M third-party/gasnet/gasnet-src/extended-ref/ratomic/gasnet_ratomic_internal.h
M third-party/gasnet/gasnet-src/extended-ref/ratomic/gasnet_refratomic.c
M third-party/gasnet/gasnet-src/extended-ref/vis/gasnet_indexed.c
M third-party/gasnet/gasnet-src/extended-ref/vis/gasnet_vector.c
M third-party/gasnet/gasnet-src/extended-ref/vis/gasnet_vis.h
M third-party/gasnet/gasnet-src/extended-ref/vis/gasnet_vis_internal.h
M third-party/gasnet/gasnet-src/gasnet.h
M third-party/gasnet/gasnet-src/gasnet_am.c
M third-party/gasnet/gasnet-src/gasnet_am.h
M third-party/gasnet/gasnet-src/gasnet_atomic_bits.h
M third-party/gasnet/gasnet-src/gasnet_atomic_fwd.h
M third-party/gasnet/gasnet-src/gasnet_basic.h
M third-party/gasnet/gasnet-src/gasnet_config.h.in
M third-party/gasnet/gasnet-src/gasnet_diagnostic.c
M third-party/gasnet/gasnet-src/gasnet_fwd.h
M third-party/gasnet/gasnet-src/gasnet_help.h
M third-party/gasnet/gasnet-src/gasnet_internal.c
M third-party/gasnet/gasnet-src/gasnet_internal.h
M third-party/gasnet/gasnet-src/gasnet_legacy.c
M third-party/gasnet/gasnet-src/gasnet_membar.h
M third-party/gasnet/gasnet-src/gasnet_mmap.c
M third-party/gasnet/gasnet-src/gasnet_pshm.c
M third-party/gasnet/gasnet-src/gasnet_pshm.h
M third-party/gasnet/gasnet-src/gasnet_syncops.h
M third-party/gasnet/gasnet-src/gasnet_timer.h
M third-party/gasnet/gasnet-src/gasnet_tm.c
M third-party/gasnet/gasnet-src/gasnet_toolhelp.h
M third-party/gasnet/gasnet-src/gasnet_tools.c
M third-party/gasnet/gasnet-src/gasnet_tools.h
M third-party/gasnet/gasnet-src/gasnet_trace.c
M third-party/gasnet/gasnet-src/gasnet_trace.h
M third-party/gasnet/gasnet-src/gasnetex.h
M third-party/gasnet/gasnet-src/ibv-conduit/Makefile.am
M third-party/gasnet/gasnet-src/ibv-conduit/Makefile.in
M third-party/gasnet/gasnet-src/ibv-conduit/README
M third-party/gasnet/gasnet-src/ibv-conduit/conduit.mak.in
M third-party/gasnet/gasnet-src/ibv-conduit/contrib/Makefile.in
M third-party/gasnet/gasnet-src/ibv-conduit/firehose_fwd.h
M third-party/gasnet/gasnet-src/ibv-conduit/gasnet_core.c
M third-party/gasnet/gasnet-src/ibv-conduit/gasnet_core.h
M third-party/gasnet/gasnet-src/ibv-conduit/gasnet_core_connect.c
M third-party/gasnet/gasnet-src/ibv-conduit/gasnet_core_fwd.h
M third-party/gasnet/gasnet-src/ibv-conduit/gasnet_core_help.h
M third-party/gasnet/gasnet-src/ibv-conduit/gasnet_core_internal.h
M third-party/gasnet/gasnet-src/ibv-conduit/gasnet_core_sndrcv.c
M third-party/gasnet/gasnet-src/ibv-conduit/gasnet_extended_fwd.h
M third-party/gasnet/gasnet-src/license.txt
M third-party/gasnet/gasnet-src/mpi-conduit/Makefile.am
M third-party/gasnet/gasnet-src/mpi-conduit/Makefile.in
M third-party/gasnet/gasnet-src/mpi-conduit/contrib/Makefile.in
M third-party/gasnet/gasnet-src/mpi-conduit/gasnet_core.c
M third-party/gasnet/gasnet-src/mpi-conduit/gasnet_core.h
M third-party/gasnet/gasnet-src/mpi-conduit/gasnet_core_fwd.h
M third-party/gasnet/gasnet-src/mpi-conduit/gasnet_core_internal.h
M third-party/gasnet/gasnet-src/ofi-conduit/Makefile.am
M third-party/gasnet/gasnet-src/ofi-conduit/Makefile.in
M third-party/gasnet/gasnet-src/ofi-conduit/README
M third-party/gasnet/gasnet-src/ofi-conduit/contrib/Makefile.in
M third-party/gasnet/gasnet-src/ofi-conduit/gasnet_core.c
M third-party/gasnet/gasnet-src/ofi-conduit/gasnet_core.h
M third-party/gasnet/gasnet-src/ofi-conduit/gasnet_core_fwd.h
M third-party/gasnet/gasnet-src/ofi-conduit/gasnet_core_internal.h
M third-party/gasnet/gasnet-src/ofi-conduit/gasnet_ofi.c
M third-party/gasnet/gasnet-src/ofi-conduit/gasnet_ofi.h
M third-party/gasnet/gasnet-src/other/Makefile-conduit.mak.in
M third-party/gasnet/gasnet-src/other/Makefile-libgasnet.mak.in
M third-party/gasnet/gasnet-src/other/Makefile.am
M third-party/gasnet/gasnet-src/other/Makefile.in
M third-party/gasnet/gasnet-src/other/ammpi/Makefile.in
M third-party/gasnet/gasnet-src/other/ammpi/license.txt
M third-party/gasnet/gasnet-src/other/amudp/Makefile.in
M third-party/gasnet/gasnet-src/other/amudp/README
M third-party/gasnet/gasnet-src/other/amudp/amudp_const.h
M third-party/gasnet/gasnet-src/other/amudp/amudp_internal.h
M third-party/gasnet/gasnet-src/other/amudp/amudp_spmd.cpp
M third-party/gasnet/gasnet-src/other/amudp/amudp_spmd.h
M third-party/gasnet/gasnet-src/other/amudp/license.txt
M third-party/gasnet/gasnet-src/other/amx/testam.h
M third-party/gasnet/gasnet-src/other/contrib/Makefile.in
M third-party/gasnet/gasnet-src/other/firehose/Makefile.in
M third-party/gasnet/gasnet-src/other/firehose/firehose.c
M third-party/gasnet/gasnet-src/other/firehose/firehose_hash.c
M third-party/gasnet/gasnet-src/other/firehose/firehose_internal.h
M third-party/gasnet/gasnet-src/other/firehose/firehose_page.c
M third-party/gasnet/gasnet-src/other/firehose/firehose_region.c
M third-party/gasnet/gasnet-src/other/fragment-body.mak.in
M third-party/gasnet/gasnet-src/other/gasnet_portable_platform.h
M third-party/gasnet/gasnet-src/other/ssh-spawner/gasnet_bootstrap_ssh.c
M third-party/gasnet/gasnet-src/other/valgrind/Makefile.in
M third-party/gasnet/gasnet-src/smp-conduit/Makefile.am
M third-party/gasnet/gasnet-src/smp-conduit/Makefile.in
M third-party/gasnet/gasnet-src/smp-conduit/gasnet_core.c
M third-party/gasnet/gasnet-src/smp-conduit/gasnet_core.h
M third-party/gasnet/gasnet-src/smp-conduit/gasnet_core_fwd.h
M third-party/gasnet/gasnet-src/smp-conduit/gasnet_core_internal.h
M third-party/gasnet/gasnet-src/tests/Makefile.in
M third-party/gasnet/gasnet-src/tests/test.h
M third-party/gasnet/gasnet-src/tests/testcore2.c
M third-party/gasnet/gasnet-src/tests/testcxx.cc
M third-party/gasnet/gasnet-src/tests/testgasnet.c
M third-party/gasnet/gasnet-src/tests/testlarge.c
M third-party/gasnet/gasnet-src/tests/testlegacy.c
M third-party/gasnet/gasnet-src/tests/testmpi.c
M third-party/gasnet/gasnet-src/tests/testratomic.c
M third-party/gasnet/gasnet-src/tests/testsmall.c
M third-party/gasnet/gasnet-src/tests/testsplit.c
M third-party/gasnet/gasnet-src/tests/testteam.c
M third-party/gasnet/gasnet-src/tests/testteambcast.c
M third-party/gasnet/gasnet-src/tests/testvis.c
M third-party/gasnet/gasnet-src/tests/upcr-harness/external-legion/Makefile
M third-party/gasnet/gasnet-src/tests/upcr-harness/external-legion/harness.conf
M third-party/gasnet/gasnet-src/tests/upcr-harness/external-upcxx/Makefile
M third-party/gasnet/gasnet-src/tests/upcr-harness/external-upcxx/harness.conf
M third-party/gasnet/gasnet-src/tests/upcr-harness/gasnet-tests/harness.conf
M third-party/gasnet/gasnet-src/tests/upcr-harness/libgasnet/harness.conf
M third-party/gasnet/gasnet-src/ucx-conduit/Makefile.am
M third-party/gasnet/gasnet-src/ucx-conduit/Makefile.in
M third-party/gasnet/gasnet-src/ucx-conduit/contrib/Makefile.in
M third-party/gasnet/gasnet-src/ucx-conduit/gasnet_core.c
M third-party/gasnet/gasnet-src/ucx-conduit/gasnet_core.h
M third-party/gasnet/gasnet-src/ucx-conduit/gasnet_core_fwd.h
M third-party/gasnet/gasnet-src/ucx-conduit/gasnet_core_internal.h
M third-party/gasnet/gasnet-src/ucx-conduit/gasnet_core_sndrcv.c
M third-party/gasnet/gasnet-src/ucx-conduit/gasnet_ratomic.c
M third-party/gasnet/gasnet-src/ucx-conduit/gasnet_ratomic_fwd.h
M third-party/gasnet/gasnet-src/udp-conduit/Makefile.am
M third-party/gasnet/gasnet-src/udp-conduit/Makefile.in
M third-party/gasnet/gasnet-src/udp-conduit/README
M third-party/gasnet/gasnet-src/udp-conduit/gasnet_core.c
M third-party/gasnet/gasnet-src/udp-conduit/gasnet_core.h
M third-party/gasnet/gasnet-src/udp-conduit/gasnet_core_fwd.h
M third-party/gasnet/gasnet-src/udp-conduit/gasnet_core_internal.h
M third-party/gasnet/gasnet-src/version.git
Compare: https://github.com/chapel-lang/chapel/compare/132e6a4ef649...2f5c4f3e6b41