[Chapel Merge] For SHOC correctness tests only do one pass

stonea · August 24, 2022, 12:40am

Branch: refs/heads/main
Revision: 4f019a8
Author: stonea
Link: For SHOC correctness tests only do one pass by stonea · Pull Request #20494 · chapel-lang/chapel · GitHub
Log Message:

Merge pull request #20494 from stonea/shoc_one_pass

For SHOC correctness tests only do one pass

We're getting timeouts on some of our implementations of the SHOC benchmarks when our nightly cray-cs-gpu-native-gasnet job runs.

Looking at our performance graphs I'm thinking this is fallout from this PR:
#20460

We should investigate the cause of the performance change but I don't think it's worth having our correctness tests continue to fail in the meantime.

Currently we're running "5 passes" of each benchmark regardless of whether we're doing so for checking correctness or measuring performance. I don't think it's necessary to run it 5 times for correctness so this PR changes it to only run 1 time under this configuration.

Ultimately, we might want to move away from having separate correctness and performance jobs for these tests (it's wasting resources) and there is a way to check correctness for performance tests here:

github.com

chapel-lang/chapel/blob/main/doc/rst/developer/bestPractices/TestSystem.rst#validating-performance-test-output

.. _readme-testsystem:

=====================
Chapel Testing System
=====================

The Chapel testing system is a key piece of technology for the Chapel
developer.  We use it as a harness for doing test-driven development,
for performing sanity checks on code before committing it, for bug and
issue tracking, and for nightly correctness and performance regression
testing.  Getting really comfortable with it is one of the most
important things a developer can do early in the development cycle.

The tests for the testing system are located in ``$CHPL_HOME/test``.
The main script that drives the test system itself is
``$CHPL_HOME/util/start_test``, though it relies on several helper scripts
located in ``$CHPL_HOME/util/test``.

This document provides only a high-level introduction to the testing
system.  For further details, ask a core Chapel developer for

This file has been truncated. show original

[Reviewed by nobody: test update; though I'll bring this up in gpu subteam meeting]

Modified Files:
M test/gpu/native/studies/shoc/EXECOPTS

M test/gpu/native/studies/shoc/sort.good
M test/gpu/native/studies/shoc/triad.good
M test/gpu/native/studies/shoc/triadalt1.good
M test/gpu/native/studies/shoc/triadalt2.good
M test/gpu/native/studies/shoc/triadchpl.good

Compare: https://github.com/chapel-lang/chapel/compare/c3a87f158998...4f019a8aa332