[Chapel Merge] Add max wait time for slurm tests

Branch: refs/heads/main
Revision: 5ae2ae24dff5532106ce6c2187458657b59fbb2b
Author: jabraham17
Link: Add max wait time for slurm tests by jabraham17 · Pull Request #28701 · chapel-lang/chapel · GitHub
Log Message:
Add max wait time for slurm tests (#28701)

Adds a new environment variable to set a max time for waiting in a slurm
queue before canceling the job

This will hopefully prevent 1 test from waiting for nodes forever and
crashing the entire test run

This PR sets a conservative 1 hour max wait time, this may still be too
long and should be decreased if needed

[Reviewed by @arifthpe]

Compare: Comparing 2c1e1864115ef3e88e3193111f8ccb506936c22e...db0c94aca25550640f4ba8f29ad2d10634b0baa0 · chapel-lang/chapel · GitHub

Diff:
M util/cron/test-hpe-cray-ex-ofi.bash
M util/test/sub_test.py
https://github.com/chapel-lang/chapel/pull/28701.diff