18127, "ShreyasKhandekar", "Apache Arrow Module in Chapel", "2021-07-27T18:25:22Z"
This issue will act like a base to connect all the Issues related to the design and development of the Apache Arrow Module in Chapel.
It will link to other issues which contain what the design ideas for how the high level Chapel interface for Arrow might look like.
Much of this is inspired from the pyarrow interface.
Arrays
Ways we should be able to create arrow arrays. See Design for Apache Arrow Module In Chapel · Issue #18074 · chapel-lang/chapel · GitHub
Status: Design Agreed Upon and Prototyped
Record batches
Ways we create and handle record batches. See Design for Apache Arrow Module In Chapel · Issue #18074 · chapel-lang/chapel · GitHub
Status: Design Agreed Upon and Prototyped
Parquet Files
Ways we read from and write to Parquet files. See (yet to create issue)
Memory and Buffers
How we deal with buffers in Arrow. See (yet to create issue)
IO
File IO Dealing primarily with Arrow Files but also Input Streams and other mmaped streams.
See (yet to create issue)
IPC
Inter Process Communication
See (yet to create issue)
Note
There are more topics to talk about in the development of Apache Arrow, but I believe these are the ones we should approach first.