Aha, I see what's going on now. I was on the right track with the updateFluff() thought last night, but there was one other issue I wasn't noticing.
First, to describe what you're seeing: Since you are using a serial loop to inspect the elements of your array, it is running completely on the current locale (locale 0) and using the following logic: "Do, I store this element locally? If so, access it and print it out. If not, communicate with the locale that owns it in order to get a copy and print it out." Since your fluff value is 2, there are two extra elements that are local to locale 0, but for which it doesn't own the main value, and similarly, two that are local to locale 1, but for which it doesn't own the main value. So my first thought was that you simply weren't calling updateFluff(). But then, checking your code, you obviously are.
That deepened the mystery slightly, but with some investigation and thought, here's what's going on: For an owned class object, only one variable referring to the object can own it at any given time. One of the impacts of this is that if an owned class variable is assigned to another, the original is set to nil to avoid duplicate ownership. So what's going on in this code is that when updateFluff() is running, it's not just updating our local fluff values to point to the original class object (as I was describing yesterday), but actually transferring ownership of the object from the original locale to our fluff value, setting the original value to nil.
This means everything is working as expected, though there are some changes you could make to resolve the issue. Before getting there, though, here's a standalone example that demonstrates this:
use StencilDist;
config const update = true;
class C {
var x: int;
}
var D = {1..10} dmapped new stencilDist({1..10}, fluff=(2,));
var A: [D] owned C?;
forall i in D do
A[i] = new C(i);
// If A's element type is owned, when we update, we transfer the class
// ownership from one locale to the other, making the original 'nil';
// if it's shared, then we copy the reference.
//
if update then A.updateFluff();
writeln(A);
for i in D do
writeln(A[i]);
When this is run with --update=false (which does not call updateFluff()), I get the following output:
{x = 1} {x = 2} {x = 3} {x = 4} {x = 5} {x = 6} {x = 7} {x = 8} {x = 9} {x = 10}
{x = 1}
{x = 2}
{x = 3}
{x = 4}
{x = 5}
nil
nil
{x = 8}
{x = 9}
{x = 10}
The first print shows no nil values because when I print the entire array, we'll have each locale print out its own values, such that none of the fluff values are printed. However, in the case of the loop, locale 0 is printing out everything, so when we get to elements 6 and 7 we see nil because it owns local copies of those elements, but since updateFluff() hasn't been called, they've never been updated from their default values.
So, when we run with --update=true, we get:
{x = 1} {x = 2} {x = 3} nil nil nil nil {x = 8} {x = 9} {x = 10}
{x = 1}
{x = 2}
{x = 3}
nil
nil
{x = 6}
{x = 7}
{x = 8}
{x = 9}
{x = 10}
Here, what's happening is that though we are the original/logical owner of elements 1-5, because we've made copies of our values into locale 1's fluff, we've transferred the ownership of those classes, which has caused ours to be nil.
Here are some things you could do to remedy this situation:
-
rather than using owned C? as your element type, you could use shared C?, which would permit multiple variables to refer to the object without setting any of them to nil. This makes my test program behave as expected once updateFluff() is called, and I suspect would for your code as well (though I didn't try it). It does have the downside mentioned last night that the original object will continue to live on locale 1 (say), so we'll refer to it remotely whenever we access it from locale 1 (say).
-
you could change your class types stored in the stencil array from classes to records. Since record variables are the object where class variables refer to an object, this would cause the fluff values to be copied locally such that each locale would have a copy of the object whenever you call updateFluff(). As mentioned last night, this could also have performance benefits in that any access to the fluff values would be local (which is typically the goal when storing cached values like this in the Stencil distribution).
-
you could switch from the stencil distribution to the block distribution, which would eliminate these fluff values, the need to update them, and any confusion over ownership. Normally, for a code doing stencils, this wouldn't be recommended for performance reasons; but if your array values are classes anyway, such that accessing those fluff values still requires going across the network, it may not be that big of a loss while simplifying the code. In any case, it may make sense to develop your code using the block distribution first before switching to the stencil distribution to get things working before working on performance (which would be the main incentive for using the stencil distribution).
Of these, I'd probably recommend using the record-based implementation of cell and wall, assuming that there aren't other reasons that they make more sense to represent as classes. If there are, then for me it's a toss-up at this point between using the block distribution or the shared array elements.
Hope this helps. Give a shout if anything is unclear.
-Brad