Following the instructions under the "Using Chapel in its Preferred Configuration" heading on the Quickstart page in the documentation, set the CHPL_LLVM environment variable to the version of LLVM installed on your system.
export CHPL_LLVM=system
Use one of the provided scripts to set other variables.
source util/setchplenv.bash
Make.
make -j 6
Check.
make check
The check fails with the output below.
[Info] Running minimal test script: $CHPL_HOME/util/test/checkChplInstall
[Info] Found executable chpl in /home/jan/Packages/chapel/bin/linux64-x86_64/chpl.
[Info] Found $CHPL_HOME directory: /home/jan/Packages/chapel
[Info] /home/jan/.chpl does not exist. Creating it.
[Info] Temporary test job directory: /home/jan/.chpl/chapel-test-yFawbT
[Info] Compiling $CHPL_HOME/test/release/examples/hello6-taskpar-dist.chpl
[Info] Compiling with CHPL_TARGET_COMPILER=llvm
[Fail] Test job failed to compile - Chapel is not installed correctly
[Fail] Compilation output:
/usr/bin/ld: /tmp/chpl-jan.deleteme-SMB4h7/chpl__module.o: relocation R_X86_64_32S against `.bss' can not be used when making a PIE object; recompile with -fPIE
/usr/bin/ld: failed to set dynamic section sizes: bad value
clang-13: error: linker command failed with exit code 1 (use -v to see invocation)
error: Make Binary - Linking
make: *** [Makefile:202: check] Error 1
Second attempt
Seeing the -fPIE in the error output, my crude instinct was to try to set the -fPIC compiler flag to see what that would do. I didn't immediately find a way to do that, but I did discover the CHPL_LIB_PIC environment variable in the Chapel documentation. So I decided to experiment with it.
Set the CHPL_LIB_PIC variable.
export CHPL_LIB_PIC=pic
Make
make -j 6
Check
make check
The check appears to pass with the following output.
[Info] Running minimal test script: $CHPL_HOME/util/test/checkChplInstall
[Info] Found executable chpl in /home/jan/Packages/chapel/bin/linux64-x86_64/chpl.
[Info] Found $CHPL_HOME directory: /home/jan/Packages/chapel
[Info] /home/jan/.chpl does not exist. Creating it.
[Info] Temporary test job directory: /home/jan/.chpl/chapel-test-bN1Tkv
[Info] Compiling $CHPL_HOME/test/release/examples/hello6-taskpar-dist.chpl
[Info] Compiling with CHPL_TARGET_COMPILER=llvm
[Info] Test job compiled into /home/jan/.chpl/chapel-test-bN1Tkv/hello6-taskpar-dist
[Info] $CHPL_LAUNCHER=none is compatible with test script.
[Info] Running test job.
[Info] Test job complete.
SUCCESS: 'make check' passed!
Question
When is it appropriate/necessary to set the CHPL_LIB_PIC environment variable?
Some operating systems include a compiler that builds position independent code by default. The Chapel build system sometimes has problems when it does not operate in a consistent manner with that choice. In particular I think that the problem we are running into here is that gcc/the system linker/system libraries assume PIC compilation but that our integration with clang is not picking up on that.
I'm looking at getting an archlinux VM going in order to reproduce the issue.
Thanks a lot! I don't know why this didn't show up when I was searching different combinations of words from the error message, but looking up the --enable-default-pie flag you mentioned led me to the "right" StackOverflow post. It turns out that configuration was made standard for gcc in ArchLinux circa 2017, and that change caused some headaches.
I appreciate the pull request for Arch support, even if the intersection of Arch users and Chapel users is just a handful. Another alternative might be to submit a Chapel package to the Arch User Repository (AUR). I've never submitted a package before. It could be a fun project.
I just ran into this issue building from the 1.26.0 release, but I'm delighted to see that it's already been fixed on main (thanks @mppf!)
@jfkiviaho There are dozens of us, dozens. I actually just reached out to the maintainer of the chapel AUR package earlier today to get added as a co-maintainer after which I plan to bump the version from 1.23.0 to 1.26.0. The chapel package should track major releases, but we could create a chapel-git AUR package that tracks chapel-lang/chapel directly (so would have the PIC fix automatically in this case, rather than having to add the variable define in the build() function for the chapel AUR package). Thoughts?
Awesome! I'm ashamed to admit that, at the time I'd posted my comment about submitting a package to AUR, I hadn't even checked if somebody had already done it. It's Arch. I shouldn't have assumed otherwise.
And, yes, I think that a chapel-git package would be a good idea. This may not be the only occasion when an Arch user might want to get ahead of the latest release. Are you thinking of adding the variable definition for the version 1.26.0 package and then removing it when the next version makes it obsolete?
I agree, having a git version is a good idea in general.
I just got added as a co-maintainer on chapel. Planning to update to 1.26 with the fix (yep, I'm just setting the bash variable manually now, but I'll remove on next release) tomorrow night. I'm still deciding whether I want llvm and clang as dependencies (such that llvm-clang is always backend) or to have them as optional dependencies at AUR install time. The former might be preferable for newer users so they automatically get the most complete version without hassle, but the latter allows more control. Any preference?
Once the chapel package is updated, we can take the script and make chapel-git with minor changes.
@jfkiviaho@ghbrown - I am now seeing a better strategy than the workaround in Add a workaround for arch linux pic support by mppf · Pull Request #19785 · chapel-lang/chapel · GitHub . But it might involve adjusting the clang package. In particular, I suspect that Linux::isPIEDefault is returning false for arch even though the gcc is configured to default to position independent executables. It looks like CLANG_DEFAULT_PIE_ON_LINUX is a cmake option that perhaps the Arch clang package should be setting when building the clang headers & libraries. Would you be able to look at the Arch clang package to see if it is or isn't passing this cmake option?
I was curious and took a peek. Looks like there's a patch that the PKGBUILD is applying that modifies the behavior of Linux::isPIEDefault? I'm not smart enough to make sense of what it means though. Is that patched code returning false when it should return true?
The change in the patch looks benign to me as far as erroneously returning false goes. It appears it can only set Linux::isPIEDefault=false when the architecture is MIPS or the system is running Android. I suspect these are both false for both of us (they are for me), so it should just always return true I think.
As for CLANG_DEFAULT_PIE_ON_LINUX, the PKGBUILD that @jfkiviaho linked doesn't mention it in the CMake flags.
I'm also trying to see if CLANG_DEFAULT_PIE_ON_LINUX makes a difference empirically. I'm using the packaged LLVM, either PKGBUILD Clang with no edits or PKGBUILD Clang with -DCLANG_DEFAULT_PIE_ON_LINUX=ON, and Chapel 1.26 from releases.
When using the unedited PKGBUILD for Clang, and make; sudo make install; make check I get the PIE error given in the first post of the thread. This is just a sanity check to make sure I'm doing everything correctly and can recreate the behavior of the clang binary from the official repositories.
I'm building Clang from the edited PKGBUILD with extra CMake flags now, but bulding Clang from source takes quite a while (as I should have expected). I'll post that result when it comes in.
Adding -DCLANG_DEFAULT_PIE=ON to the PKGBUILD cmake_args didn't seem to fix the linker error, unfortunately, but it's always possible I did something wrong.
You've probably already seen this discussion about default PIE for packaged Clang, but I link it just in case.
Thanks @ghbrown and @jfkiviaho . Sounds like it will need more investigation in order to figure out why Chapel wasn't building pic/pie code by default. But anyway, at least we have a workaround for now.
On a different note, regarding the Arch package for Chapel, we have been trying to bring in scripts to create and test Chapel packages into the Chapel repository. We have recently added chapel/util/packaging at main · chapel-lang/chapel · GitHub . So far, it has a Homebrew formula and some instructions for how to use the formula to do a manual build of the package. It's certainly the case that the real Homebrew formula is elsewhere, but having a version of it in our own repository has several advantages:
we can adjust it without releasing a new Homebrew formula (say, when preparing for a new release of Chapel)
we can perform automated testing of the Homebrew formula with main to make sure that our changes to the Chapel repository have not broken the Homebrew formula and the package it builds
we can share instructions and scripts to manually build and test our Homebrew formula in case somebody needs to do that in order to address an issue with Chapel on Homebrew
If there are changes to the Homebrew formula in the Homebrew repository, it's our intention to mirror them in util/packaging/homebrew/chapel-release.rb and in util/packaging/homebrew/chapel-main.rb if they are relevant for the next release.
There is a lot we haven't yet figured out about how to manage packaging Chapel for different operating systems. However, I think it would be helpful to have the relevant Arch instructions and scripts in util/packaging/arch in order to achieve the same benefits I listed above for Homebrew. (Note that the instructions are especially important because many of us have almost zero experience with Arch). Is that something you are able to prepare a PR to do?
Thanks in any case for helping with the Arch package!
@mppf How does the linked repo compare to the official packaging repo? I had stumbled across that first, and was thinking it would be the right place to put the AUR scripts we'll be working on once they're finished.
The one thing that gave me pause is that the AUR scripts themselves are part of a git repository hosted by Arch, and I'm not sure of the elegant way for your packaging git repository to subsume a smaller git repository. In principle maintainers could just copy and paste the files every time the AUR packages get updated (but I'd like to avoid this if possible). Perhaps we could have a script in the packaging repo to pull down only the build files from AUR. Are the other distros doing anything like this?
Regardless of the above, yes, I'm happy to make a PR on whichever repo you'd like for Arch build files, instructions, etc.
Right, we have been thinking it better to include the packaging information in the main repo in order to keep everything synchronized (since we might need package script changes for particular revisions of main, etc). Most of the stuff in chapel-packaging is pretty stale at this point.
I agree this is an issue. I have been imagining we could use a manual process to update util/packaging when the Arch scripts change in the Arch repo for now. In the future I'm sure we could have some scripting to check for changes to such repositories (since it also comes up for Homebrew).
Are the other distros doing anything like this?
I don't know. For Chapel we don't yet have Fedora, Debian, or Ubuntu packages (although we would like those, if anybody wants to volunteer to work with those communities). There is a Spack package but we would like to get it better maintained (and I think putting the scripts and instructions in util/packaging would help with that). Also, I am pretty sure I have seen other open source projects with their own repository for packaging - which necessarily probably duplicates the distro's repo - but I don't remember where I saw that, off-hand.
That makes sense to me. Generalized package formats like flatpaks could also be nice, since we could hit all the distros at once. I don't have any experience in this, but I could look into making one at some point in the future.
As for the present, I just finished bumping the chapel AUR package to 1.26.0. It works well for me, but I'd appreciate it if you would give it a test if you have a second.
For @mppf it should be sufficient to sudo pacman -S yay; yay -S chapel, which gets an AUR helper called yay (a one time thing) then uses said helper to get chapel. Perhaps you could comment on the built config of the compiler and what options we should or shouldn't set for the user. Right now I just use ./configure; make; make check; make install, and since LLVM and Clang are present they get used automatically.
As for working the Arch packaging into the chapel repo in a PR, I plan to look at that tomorrow.
Hm, it appears that while it passes make check it fails to compile an actual Chapel program for me, with error:
/usr/bin/ld: /home/ghbrown/Temporary/chpl-ghbrown.deleteme-lqaDoZ/chpl__module.o: in function `chpl_task_reset_spawn_order':
root:(.text+0x73175): undefined reference to `qthread_chpl_reset_spawn_order'
/usr/bin/ld: /home/ghbrown/Temporary/chpl-ghbrown.deleteme-lqaDoZ/chpl__module.o: in function `chpl_qthread_get_tasklocal':
root:(.text+0x747f0): undefined reference to `qthread_get_tasklocal'
---- many more lines ----
/usr/bin/ld: /usr//lib/chapel/1.26/runtime/lib/linux64/llvm/x86_64/cpu-native/loc-flat/comm-none/tasks-qthreads/tmr-generic/unwind-none/mem-jemalloc/atomics-cstdlib/hwloc-bundled/re2-bundled/fs-none/lib_pic-pic/san-none/libchpl.a(topo-hwloc.o): in function `chpl_hwloc_get_next_obj_by_depth':
(.text+0x1753): undefined reference to `chpl_hwloc_get_obj_by_depth'
clang-13: error: linker command failed with exit code 1 (use -v to see invocation)
error: Make Binary - Linking
I am hoping this is caused by my own source builds of Chapel interfering with the new AUR build. Even if this is the case, I'm not sure exactly how to fix it. I've done all of the make cleans and clobbers, but it doesn't appear there is an uninstall. I deleted the binary in /usr/local/bin manually, but it seems there's more.