[Chapel Merge] Enable sub-jobs for large test dirs in GA paratest

Notifications Commits
1

Branch: refs/heads/main
Revision: 91bbe9b47d836d29c9d48e44230047f68fa317cb
Author: arifthpe
Link: Enable sub-jobs for large test dirs in GA paratest by arifthpe · Pull Request #29343 · chapel-lang/chapel · GitHub
Log Message:
Enable sub-jobs for large test dirs in GA paratest (#29343)

Enable the well-known slow directory split-up feature that was added but
commented out in Add ability to run entire test suite in GA by jabraham17 · Pull Request #29212 · chapel-lang/chapel · GitHub.

Currently nothing is getting through the merge queue, as
linux64-gasnet-everything paratesting on large test dirs is consistently
hitting the 90 minute timeout since its introduction in
Add GA paratest for gasnet-everything by jabraham17 · Pull Request #29316 · chapel-lang/chapel · GitHub. Although splitting
large dirs will blow up the number of jobs, it may be necessary if we
want paratests in MQ without a slower MQ.

This job also modifies which directories are split up (from those in the existing, commented-out list), as explained below.

I enumerated the test directories with the largest number of files,
which though not a 1:1 correspondence should be a rough indication of
which ones will take the longest to complete:

rift in ~/chapel on main λ for f in test/*; do echo $(find $f -type f | wc -l) $f; done | sort -rh
> head -n 10
5452 test/library
5085 test/types
4836 test/classes
3738 test/studies
3443 test/functions
1983 test/arrays
1825 test/visibility
1549 test/users
1243 test/parallel
1180 test/interop

Of these, looking at a recent gasnet-everything
run, the
longest runtimes (rounded to minutes) were:

  1. classes, 147m
  2. library, 121m
  3. types, 73m
  4. functions, 56m
  5. studies, 47m

Given building Chapel pre-paratest takes about 15m, and our merge queue
time limit is 90m, 75m is the approximate limit for how long a test dir
can take (assuming no contention for runners causes further delays). The
top 3 items on this list are over that limit, and the rest are under it
with a buffer of 15+m, so IMO we ought to split exactly these 3.

[reviewed by @jabraham17 , thanks!]

Compare: Comparing af00fd47f05ab1286c1c5f77d6846fb624fce97b...fd927104f528aa12c928011ab0fe1e07e518da1e · chapel-lang/chapel · GitHub

Diff:
M .github/workflows/ga-paratest.yml
https://github.com/chapel-lang/chapel/pull/29343.diff