[1.29.0] Cannot deref() a c_ptrTo(unmanaged class instance): comm-none: Assertion `node==0` failed

Developers
1

Reproducer:

use CTypes;

class foo {
  var bar : int(64);
};

proc main() {
  var myFoo = new unmanaged foo();
  myFoo.bar = 8675309;
  writeln("myFoo: ", myFoo);
  var myPtr = c_ptrTo(myFoo);
  writeln("myPtr: ", myPtr);
  //Assertion fails on this deref:
  var myDeref = myPtr.deref();
  writeln("myDeref: ", myDeref);
  delete myFoo;
}

Compiles fine with flags -g --devel --verify, when run produces:

myFoo: {bar = 8675309}
myPtr: 0x7fff867fee10
 comm-none.c:171: void chpl_comm_get(void *, c_nodeid_t, void *, size_t, int32_t, int, int32_t): Assertion `node==0' failed.

chpl is compiled with DEBUG=1 OPTIMIZE=0 CHPL_LLVM=system CHPL_LOCALE_MODEL=gpu CC=clang CXX=clang++. System Clang/LLVM is version 14

Backtrace:

#0  __pthread_kill_implementation (no_tid=0, signo=6, threadid=140737307996160) at ./nptl/pthread_kill.c:44
#1  __pthread_kill_internal (signo=6, threadid=140737307996160) at ./nptl/pthread_kill.c:78
#2  __GI___pthread_kill (threadid=140737307996160, signo=signo@entry=6) at ./nptl/pthread_kill.c:89
#3  0x00007ffff5df0476 in __GI_raise (sig=sig@entry=6) at ../sysdeps/posix/raise.c:26
#4  0x00007ffff5dd67f3 in __GI_abort () at ./stdlib/abort.c:79
#5  0x00007ffff5dd671b in __assert_fail_base (fmt=0x7ffff5f8b150 "%s%s%s:%u: %s%sAssertion `%s' failed.\n%n", assertion=0x55555579d665 "node==0", file=0x55555579d366 "comm-none.c", line=171,
    function=<optimized out>) at ./assert/assert.c:92
#6  0x00007ffff5de7e96 in __GI___assert_fail (assertion=0x55555579d665 "node==0", file=0x55555579d366 "comm-none.c", line=171,
    function=0x55555579d58f "void chpl_comm_get(void *, c_nodeid_t, void *, size_t, int32_t, int, int32_t)") at ./assert/assert.c:101
#7  0x0000555555677550 in chpl_comm_get ()
#8  0x000055555566462c in chpl_gen_comm_get(void*, int, void*, unsigned long, int, int, int) ()
#9  0x00005555555fe7db in writeThisFieldsDefaultImpl_chpl2 (writer=<optimized out>, x=..., first=<optimized out>, error_out=<optimized out>)
    at /home/psath/chapelWorkspace/tool-installs/chapel-1.29.0-build.gdb-talos/share/chapel/1.29/modules/standard/ChapelIO.chpl:187
#10 0x0000555555600203 in writeThisDefaultImpl_chpl (writer=<optimized out>, x=..., error_out=<optimized out>)
    at /home/psath/chapelWorkspace/tool-installs/chapel-1.29.0-build.gdb-talos/share/chapel/1.29/modules/standard/ChapelIO.chpl:261
#11 0x0000555555663bb1 in _auto_foo_write_chpl (this=..., f=<optimized out>, error_out=<optimized out>) at .//repro.chpl:3
#12 0x000055555564e1ee in _write_one_internal_chpl11 (_channel_internal=<optimized out>, x=..., loc=<optimized out>, error_out=<optimized out>)
    at /home/psath/chapelWorkspace/tool-installs/chapel-1.29.0-build.gdb-talos/share/chapel/1.29/modules/standard/IO.chpl:3978
#13 0x000055555564a12a in _writeOne_chpl7 (this=<optimized out>, x=..., loc=<optimized out>, error_out=<optimized out>)
    at /home/psath/chapelWorkspace/tool-installs/chapel-1.29.0-build.gdb-talos/share/chapel/1.29/modules/standard/IO.chpl:3794
#14 0x0000555555653ad7 in on_fn_chpl101 (this=<optimized out>, args=..., origLocale=<optimized out>, error_out=...)
    at /home/psath/chapelWorkspace/tool-installs/chapel-1.29.0-build.gdb-talos/share/chapel/1.29/modules/standard/IO.chpl:5921
#15 0x0000555555657efd in writeln_chpl4 (this=<optimized out>, _e0_args=<optimized out>, _e1_args=..., error_out=<optimized out>)
    at /home/psath/chapelWorkspace/tool-installs/chapel-1.29.0-build.gdb-talos/share/chapel/1.29/modules/standard/IO.chpl:5974
#16 0x0000555555600da8 in writeln_chpl2 (_e0_args=<optimized out>, _e1_args=...) at /home/psath/chapelWorkspace/tool-installs/chapel-1.29.0-build.gdb-talos/share/chapel/1.29/modules/standard/ChapelIO.chpl:717
#17 0x000055555566409c in chpl_user_main () at .//repro.chpl:14
#18 0x0000555555663d26 in chpl_gen_main () at .//repro.chpl:7
#19 0x000055555566ebb5 in chpl_executable_init ()
#20 0x000055555567618d in chapel_wrapper ()
#21 0x00005555556d4f91 in qthread_wrapper ()
#22 0x0000000000000000 in ?? ()
2

Hi Paul —

With a non-gpu build, this seems to be working as expected, so I'm tagging @e-kayrakli , @stonea , and @diten here, as the GPU experts to see what they think.

I'm also curious about what you want c_ptrTo() to be pointing to here, as I haven't thought much about how it should behave w.r.t. classes. I.e., if you think of a class pointer as being both a pointer to an object and the object itself, are you interested in getting a pointer to the pointer, or a pointer to the object? Tagging @annarift who's been working on c_ptrTo() recently, so is likely to be interested (Anna, I could imagine saying that c_ptrTo() should point to the object (or maybe just not be supported on classes), but that c_addrOf() should point to the pointer).

-Brad

3

Currently in the GPU locale model we force "wide pointers" to be used for everything outside of a local block. You can think of wide pointers as being Chapel's representation of a pointer that might point to data on another locale (node). As an immediate workaround I'm able to get your program to compile and work by embedding everything inside of a local block (I'll paste the code below).

I'm not sure what the expected behavior of c_ptr is in regards to wide pointers is, certainly it shouldn't crash so there's definitely a bug on our end here. It's less clear to me if this is something strictly related to the GPU programming model or if it's something that could be reproduced outside of it so I'll have to do more digging there.

Anyway I'm not sure if this is an acceptable workaround for your use case but I was able to get this to work:

use CTypes;

class foo {
  var bar : int(64);
};

proc main() {
  local {
    var myFoo = new unmanaged foo();
    myFoo.bar = 8675309;
    writeln("myFoo: ", myFoo);
    var myPtr = c_ptrTo(myFoo);
    writeln("myPtr: ", myPtr);
    //Assertion fails on this deref:
    var myDeref = myPtr.deref();
    writeln("myDeref: ", myDeref);
    delete myFoo;
  }
}
4

Interesting, we are planning to use the GPU for kernels, but this particular object won't be transferred to the sublocale. Only the individual arrays members of the class. (elided here for simplicity of reproducer)

For unmanaged instances of a class I would expect c_ptrTo to be a pointer to the heap-allocated, lifetime-unbounded-by-lexical-scope raw data of the instance. (Whatever an instance in Chapel entails, members, vtable, etc... sufficient for deref to produce a fully-functioning Chapel instance on the other end.)

I'm intending to use this as an opaque handle, via cast to c_void_ptr, alongside a type descriptor record to do a poor-man's RTTI. Foo is generic, but has a small number of permutations. I have concrete allocators/readers for all permutations in one module, and writers/deallocators for all permutations in another module. Everything in between has no need to know what's inside the pointer, and can't know what its type should be until runtime, because that's read off disk, hence the opaque handle

I'm not following your "both a pointer to an object and the object itself"... I would guess in the above example the value of "myFoo" is mapped to a pointer of some kind (i.e. C++ foo *), but you could also take the stack address of myFoo (i.e. C++ foo **)? I wouldn't expect to do much with the address of myFoo -- which in C would become invalid once we left the function -- but what it points to is valuable outside the local scope, because it is unmanaged. Are you saying that in Chapel its some superposition of the two?

I didn't see addrOf in https://chapel-lang.org/docs/modules/standard/CTypes.html or C Interoperability — Chapel Documentation 1.29, is that available in 1.29.0?

Otherwise, I haven't thought much about what I would expect from the other management schema. I am only using a class for the unmanaged instance lifetime, and -- please correct me if I'm wrong -- this seems the only way to get flexible-lifetime heap objects in Chapel? Which is what I really want. Everything else is deleted once it goes out of scope.

5

Thanks @stonea, I'll give it a try tomorrow.

Right now we are only doing 1-node, 1-gpu, but eventually will want to do X-node, Y-gpu. We haven't considered partitioning yet, so local should be fine for now.

6

This is what you get with myUnmanagedClassInstance : c_void_ptr. If you say c_ptrTo(myUnmanagedClassInstance) today, you are asking for a pointer to a pointer, because a Chapel unmanaged class is already a pointer.

Here is a concrete example you can run to try to explain things:

use CTypes;

class MyClass {
  var field: int;
}

proc main() {
  var obj = new unmanaged MyClass(1);

  writeln("Address of obj.field is ", c_ptrTo(obj.field):c_void_ptr);

  writeln("obj:c_void_ptr is ", obj:c_void_ptr);

  writeln("c_ptrTo(obj) is ", c_ptrTo(obj):c_void_ptr);
}

Example output

Address of obj.field is 0x7fed04002968
obj:c_void_ptr is 0x7fed04002960
c_ptrTo(obj) is 0x7fed0b1fcdc0

Note that the address of the field is 8 bytes into the object (there is a bit of metadata to support virtual dispatch). Then, obj:c_void_ptr is pointing to the object data. But c_ptrTo(obj) is pointing to stack memory somewhere.

In any case, I think it's reasonable to ask for c_ptrTo(someOwnedClass) or c_ptrTo(someUnmanagedClass) to give a pointer to the object data (just before the first field). But, it does not do that now.

1 Like
7

Ahhh thank you for the detailed response, that was a subtlety I hadn't noticed. So directly casting to/from void pointer is what I need for my use case.

I think there's a case for both pointer-to-object and pointer-to-pointer interpretations of c_ptrTo. With the direct cast to c_void_ptr supporting the former, I don't think there's anything categorically wrong with the latter. Especially since that's consistent with how I'd expect it to treat primitive vars -- i.e. address of the identifier.

Perhaps the above example can go in one of the C Types docs for posterity?

8

c_addrOf is a planned addition by Chapel 2.0, and not available in 1.29.0 or anywhere else yet. The idea is that c_ptrTo will be "smart" and return the actual data buffer of things like arrays which start with some Chapel descriptor, while c_addrOf will just point to the start of the variable in memory which could be the descriptor. This proposal is discussed in a lot of detail over at https://github.com/chapel-lang/chapel/issues/18016 -- probably more information than one would want to digest for the issue at hand but I'm linking it for completeness.

This feels very analogous to the issue at hand. However, to my knowledge we have not discussed implementing any special "smart" behavior for c_ptrTo when given a Chapel class variable. (edit: I totally spaced Brad's suggestion to do this in this very thread, oops.) If we were going to do so it would probably give a pointer-to-object like casting to c_void_ptr currently does, and c_addrOf would naively give a pointer-to-pointer. Doing so would make a lot of sense to me, so I'm going to go ahead and propose it to the team on the above issue.

We will definitely be clarifying and improving CTypes docs in this area when putting in the c_ptrTo and c_addrOf changes, and I will make sure it covers the distinction between pointer-to-pointer and pointer-to-object distinctions in the case of class types.

1 Like
9

That delineation between the c_addrOf and c_ptrTo functionalities for classes makes sense to me as well. Thanks for the additional info and teaser on 2.0 features!

2 Likes