[Chapel Merge] QIO non-buffered write optimization

Branch: refs/heads/main
Revision: e1f3c33b1172da3356c1fc9f94c634727701c743
Author: jeremiah-corrado
Link: https://github.com/chapel-lang/chapel/pull/21786
Log Message:
QIO non-buffered write optimization (#21786)

This PR implements an optimization that avoids buffering in a fileWriter
for large write operations. Buffering is typically useful for a
series of small writes, as it avoids the latency/overhead associated
with a large number of system calls; however, for sufficiently large
write operations, buffering imposes unneeded overhead so it can be
avoided.

This optimization only takes effect when:

  • the number of bytes to be written is larger than the threshold
    specified by qio_write_unbuffered_threshold
  • the channel's method is read/write, fread/fwrite, or pread/pwrite (no
    mmap)
  • there are no pending commits/reverts (the mark stack only contains
    "right_mark_start")
  • there is no IO plugin associated with the channel

Tests are included to validate the correctness of this change.

A bug where writeBinary(c_ptr) only wrote some of the requested
values was also fixed.

[ Reviewed by @mppf ] - Thanks!

  • passing paratest

Compare: Comparing 3bb40dce60440539700af7b030cff2e350f0ce92...e231ce23db3e33528c410d19ca08a0cf2ceffbd9 · chapel-lang/chapel · GitHub

Diff:
M modules/standard/IO.chpl
M runtime/include/qio/qio.h
M runtime/src/qio/qbuffer.c
M runtime/src/qio/qio.c
A test/io/corrado/write_buffering_optimization/COMPOPTS
A test/io/corrado/write_buffering_optimization/dynamic_buffering.chpl
A test/io/corrado/write_buffering_optimization/dynamic_buffering.cleanfiles
A test/io/corrado/write_buffering_optimization/dynamic_buffering.execopts
A test/io/corrado/write_buffering_optimization/dynamic_buffering.good
A test/io/corrado/write_buffering_optimization/growing_buff_write.chpl
A test/io/corrado/write_buffering_optimization/growing_buff_write.good
A test/io/corrado/write_buffering_optimization/large_write.chpl
A test/io/corrado/write_buffering_optimization/large_write.good
M test/studies/shootout/reverse-complement/bharshbarg/revcomp-buf.chpl
https://github.com/chapel-lang/chapel/pull/21786.diff