Importing files from subfolders

[lucaferranti]

Hello, I would like to structure my mason package roughly as follows

src/
|--MainFile.chpl
|--fizz.chpl
|--folder/
  |--foo.chpl
  |--bar.chpl

and my MainFile.chpl is basically just

module MainFile {
  public use fizz;
  public use foo;
  public use bar;
}

the problem is that if foo.chpl and bar.chpl are in a subfolder, it seems they cannot be found. It works by having all files in src but this could backfire for bigger projects. Is there a work-around for this?

edit: the question probably overlaps significantly with this other topic

How to structure classes as files

Hi Chapel developers. Lately I have started writing chapel code with classes. So if I want to group a bunch of related classes together, like I am used to with c++ namespaces, It looks like modules is the right way. In chapel every file becomes a module. Quoting the documentation I find on Modules — Chapel Documentation 1.26 it says. If all the code of a file is not enclosed in an explicit module, defined using the module keyword, then the file itself is treated as a module with the same n…

Reading that discussion, I understand there is currently no work-around?

[lucaferranti]

For the records, I know about the include submodules functionality, the problem is that that requires all subfiles to be inside a folder called like the main module (MainFile in my example above). I want to be able to name my folders freely and also possibly use multiple subfolders.

[vass]

Hi Luca,

Please check if it works if you add these statements to the body of your MainFile module:

require "folder/foo.chpl";
require "folder/bar.chpl";

References: Fix support for requires of .chpl files within implicitly used modules by bradcray · Pull Request #18912 · chapel-lang/chapel · GitHub   and   $CHPL_HOME/test/modules/require/implUseReqChain/mixImplUsesAndRequires.chpl   esp. B.chpl in that directory.

[lucaferranti]

the following worked (in the sense I could run mason test successfully)

require "src/folder/foo.chpl";
require "src/folder/bar.chpl";

so one needed the src folder as prefix.

Is the require keyword documented? I could not find it, I could open a PR (to primers/modules maybe) adding a note about this.

[mppf]

Hi Luca -

The require keyword is documented for a different purpose C Interoperability — Chapel Documentation 1.32 but the use with a .chpl file is totally undocumented, as far as I know.

But, going back to your original post:

@lucaferranti

Hello, I would like to structure my mason package roughly as follows

src/
|--MainFile.chpl
|--fizz.chpl
|--folder/
  |--foo.chpl
  |--bar.chpl

and my MainFile.chpl is basically just

module MainFile {
  public use fizz;
  public use foo;
  public use bar;
}

Mason packages that are libraries are expected to provide a single top-level module with the same name as the mason package. With the above structure, the code is providing 4 top-level modules (MainFile, fizz, foo, and bar). In Chapel programs, the top-level modules all share a single namespace. I know your real application does not have a module named foo, and that is a stand-in for another name. Keeping with that name, If you later want to use another library that also has a top-level module named foo, that would cause problems. However, I don't think mason checks for this and there might be cases where a mason package really does want to provide multiple top-level modules. I have not seen one yet, though.

The solution to this problem that we recommend is to use module include.

@lucaferranti

I want to be able to name my folders freely and also possibly use multiple subfolders.

Perhaps you could use symbolic links in order to structure your files the way you like but to also provide the structure that module include needs.

Now, the above answer is mostly irrelevant if you are using mason to write an application (vs a library). If you are writing an application, you can use mason build -- -M folder/ and that will just add the -M folder/ argument to the chpl command invocation to add your folder to the module search path. Something you will notice about that command is the -- and, in Mason, that is how you can pass flags to the chpl command, rather than passing them to Mason itself. That should allow your example to build as you have described it.

In the future, we plan to look at allowing such chpl arguments to be provided in a mason .toml file, but we are not there yet.

[lucaferranti]

Thank you very much for you very detailed answer.

Maybe this was an XY-problem, I guess my real question is: what is the idiomatic way to structure a bigger library in Chapel? (it's only a matter of time before Chapel gains popularity and several people start writing libraries in it )

Based on your answer, I understand the proper way is to use submodules. Is the following a legit structure?

src/
|--MyLibrary.chpl
|--MyLibrary/
  |--SubModule1.chpl
  |--SubModule1/
    |--file1.chpl
    |--file2.chpl
  |--SubModule2.chpl
  |--SubModule2.chpl
    |--file3.chpl
    |--file4.chpl
  |--SubModule3.chpl

I guess there are two issues I'm trying to solve

1. The library has several subfunctionalities, in which case submodules are perfect (cfr. submodules in python libraries).

2. The module is simply too large to readably fit one file, in which case I simply want to span the module over multiple files (in my latest example, this could be SubModule1, which is scattered across file1.chpl and file2.chpl).

I'll try the above approach, meanwhile, happy to hear if you have other comments

[mppf]

@lucaferranti

I guess there are two issues I'm trying to solve

1. The library has several subfunctionalities, in which case submodules are perfect (cfr. submodules in python libraries).
2. The module is simply too large to readably fit one file, in which case I simply want to span the module over multiple files (in my latest example, this could be SubModule1, which is scattered across file1.chpl and file2.chpl).

Right, submodules are great for 1. I think we have been imagining that they will work OK for 2. as well, because you can choose how to divide the code, and then use re-export functionality to make it appear as if they were all implemented in a single file. (Where public use SubModule; is the way to re-export everything).