Branch: refs/heads/main
Revision: e1f3c33b1172da3356c1fc9f94c634727701c743
Author: jeremiah-corrado
Link: https://github.com/chapel-lang/chapel/pull/21786
Log Message:
QIO non-buffered write optimization (#21786)
This PR implements an optimization that avoids buffering in a fileWriter
for large write operations. Buffering is typically useful for a
series of small writes, as it avoids the latency/overhead associated
with a large number of system calls; however, for sufficiently large
write operations, buffering imposes unneeded overhead so it can be
avoided.
This optimization only takes effect when:
- the number of bytes to be written is larger than the threshold
specified byqio_write_unbuffered_threshold
- the channel's method is read/write, fread/fwrite, or pread/pwrite (no
mmap) - there are no pending commits/reverts (the mark stack only contains
"right_mark_start") - there is no IO plugin associated with the channel
Tests are included to validate the correctness of this change.
A bug where writeBinary(c_ptr)
only wrote some of the requested
values was also fixed.
[ Reviewed by @mppf ] - Thanks!
- passing paratest
Diff:
M modules/standard/IO.chpl
M runtime/include/qio/qio.h
M runtime/src/qio/qbuffer.c
M runtime/src/qio/qio.c
A test/io/corrado/write_buffering_optimization/COMPOPTS
A test/io/corrado/write_buffering_optimization/dynamic_buffering.chpl
A test/io/corrado/write_buffering_optimization/dynamic_buffering.cleanfiles
A test/io/corrado/write_buffering_optimization/dynamic_buffering.execopts
A test/io/corrado/write_buffering_optimization/dynamic_buffering.good
A test/io/corrado/write_buffering_optimization/growing_buff_write.chpl
A test/io/corrado/write_buffering_optimization/growing_buff_write.good
A test/io/corrado/write_buffering_optimization/large_write.chpl
A test/io/corrado/write_buffering_optimization/large_write.good
M test/studies/shootout/reverse-complement/bharshbarg/revcomp-buf.chpl
https://github.com/chapel-lang/chapel/pull/21786.diff