Creating a New Range Method

Users
1

When I sent this originally as part of an early email to discourse, Chapel chopped it off. Odd.

Anyway, .....

There seem to be some methods which return a new range for which the name is a verb

     expand(d : idxType)
     translate(d : idxType)

There are also two other methods that return a related range

     interior(d : idxType)
     exterior(d : idxType)

but their names are nouns.

The mix of nouns and verbs for methods which both return ranges seems out of place and it would be good to know the logic behind the choice of a mix. Then again, I cannot see for what I would use the 'interior' or the 'exterior' method so maybe they are useless.

I ask this because I want to add a method for a range and would like to follow the existing semantics as long as they make sense (and maybe even if they do not).

Anyway, I need a range method which returns a new range which is a subset of the original but where only one end is affected. A haircut for the head of the range OR the equivalent for the tail, not both as yielded by expand with a negative argument.

If I choose a verb, say "cut", "trim", "snip", "crop", "scrape", "pare" "lop", "prune", "shrink" or "shave" and I follow Python to chose a positive integer be associated with the lower bound and negative one with the upper bound, then given a range

      const r = 1 .. n;

I want to

     assert(r.shave(+1) == 2 .. n);
// AND
     assert(r.shave(-1) == 1 .. n - 1);

Note that I am only interested in subsets of the original range, not a superset. The absolute value of the argument will normally be 1 or 2. Now others may have different needs so the generalization of what I am doing will need extra thought.

I wrote:

     proc range.shave(i : integral) return if this.stridable
         then
         (
             if i > 0:i.type then
                 this.low + this.stride * i:this.idxType .. this.high
                    by this.stride
             else
                 this.low .. this.high + this.stride * i:this.idxType
                    by this.stride
         )
         else
         (
             if i > 0:i.type then
                 this.low + i:this.idxType .. this.high
             else
                 this.low .. this.high + i:this.idxType
         );

I do not understand how this works because it returns both stridable and non-stridable ranges (whatever stridable really means - see below) about which the compiler complains in other scenarios.

So, how do I write this properly and how optimal is this? Certainly the first part of the 'if' is done at compile time because 'stridable' is a param. What overhead does it have? I want minimal overhead. The only other examples I can find are in ChapelRange.chpl. These are internal methods of a range and work with internal variables of the range and often then play with the Chapel 'new' operator. So they are not much use to help me write my own method outside of this file.

What does the current concept of stridable mean? Also, why is a range with an implied stride (i.e. of 1) not stridable when it clearly is, i.e. with a stride of 1? On first reading, I thought the 'stridable' method meant

     ! (the range has a (known at compile time) unit-stride)

but that is not the case because ranges

    const r0 = 1 .. 10;
    const r1 = 1 .. 10 by 1;

are different even that the compiler should be able to treat them identically, i.e. the stride of '1' in the case of 'r1' is known at compile time.

It almost looks like it means

     ! (the range has a implicit (and known at compile time) unit-stride)

Is this correct? Either way, whatever it means, the method/concept needs a new name with at least a semblance of semantic correctness. As it stands it is confusing because every range is stridable, i.e. can be strode, irrespective of the return value of the Chapel 'stridable' method.

Why does the documentation say stridable is a 'proc' when in ChapelRange.chpl is is clearn a param?

2

Hi Damian,

There seem to be some methods which return a new range for which the name is a verb
There are also two other methods that return a related range but their names are nouns.

The mix of nouns and verbs for methods which both return ranges seems out of place and it would be good to know the logic behind the choice of a mix. Then again, I cannot see for what I would use the 'interior' or the 'exterior' method so maybe they are useless.

I am not aware of a deep reason for this. However, we are looking for better naming etc. for these methods. Please contribute to the discussion on Issue #17127 .

I ask this because I want to add a method for a range and would like to follow the existing semantics as long as they make sense (and maybe even if they do not).

Anyway, I need a range method which returns a new range which is a subset of the original but where only one end is affected. A haircut for the head of the range OR the equivalent for the tail, not both as yielded by expand with a negative argument.

If I choose a verb, say "cut", "trim", "snip", "crop", "scrape", "pare" "lop", "prune", "shrink" or "shave" and I follow Python to chose a positive integer be associated with the lower bound and negative one with the upper bound, then given a range

The name "chop" jumps at me. What does Python use for this?

Could you please clarify whether you'd like to make is a standard method of Chapel's range type or you'd like to add it to your own code. If the former, a good way to proceed is to open an issue on github proposing this and asking for feedback. Or -- given the small size of the proposal -- make a PR that adds the code you sketched out and discuss it there.

I wrote: [...]

I do not understand how this works because it returns both stridable and non-stridable ranges (whatever stridable really means - see below) about which the compiler complains in other scenarios.

So, how do I write this properly and how optimal is this? Certainly the first part of the 'if' is done at compile time because 'stridable' is a param. What overhead does it have? I want minimal overhead. The only other examples I can find are in ChapelRange.chpl. These are internal methods of a range and work with internal variables of the range and often then play with the Chapel 'new' operator. So they are not much use to help me write my own method outside of this file.

I think your code works well in common cases. Where I see it can hit corner cases is (a) idxType is an unsigned integer or an enum or bool, and (b) with some unusual or ambiguous alignment. If you'd like your method to be a standard range method, then you get to use the magic that's in ChapelRange.chpl. Otherwise maybe you don't need to cater to those, especially when you do not expect ambiguous alignment or non-integer idxType in your code.

What does the current concept of stridable mean? Also, why is a range with an implied stride (i.e. of 1) not stridable when it clearly is, i.e. with a stride of 1?
[...]
It almost looks like it means

     ! (the range has a implicit (and known at compile time) unit-stride)

Is this correct?

Yes it is.

The reasoning is this. Ranges that are !stridable have more efficient implementation, whereas ranges that are stridable provide more expressive power.

Either way, whatever it means, the method/concept needs a new name with at least a semblance of semantic correctness. As it stands it is confusing because every range is stridable, i.e. can be strode, irrespective of the return value of the Chapel 'stridable' method.

I can see it being confusing. If you have suggestions, please contribute on github issue 17131 . To our defense, if it is a range that can be strode only with its own stride of 1, it is not very useful to call it "stridable".

Why does the documentation say stridable is a 'proc' when in ChapelRange.chpl is clearly a param?

This is clearly a bug in the documentation. Thank you for pointing it out.

Let us know what remains unclear!

3

Just touching on a few things here:

Let me double down on what Vass says here. We are in the process of reviewing all standard library and library-like features in Chapel, which includes methods on ranges, so making sure the supported operations are clear and well-defined is important, and feedback like this is valuable.

It's not Chapel chopping things off, it's Discourse. Though Discourse has been in almost every respect a breath of fresh air compared to our old mailing lists, its choice to truncate incoming email messages at things that it identifies as potentially indicating the start of a "reply-to" chain is a major hassle for those of us who interact with it a lot via email. The best bets to avoid this are to post through the web interface, get out of the habit of using --- style separators, or to mix alphanumeric characters within them like --- details follow ---.

-Brad

4

Let me double down on what Vass says here. We are in the process of
reviewing all standard library and library-like features in Chapel, which
includes methods on ranges, so making sure the supported operations are
clear and well-defined is important, and feedback like this is valuable.

Happy to help. Whether my Aussie twist on English matches others ideas of
the language is another question!

  Chapel chopped it off

It's not Chapel chopping things off, it's Discourse.

Sorry. I meant Discourse. My brain waves got scrambled on their way to my
typing fingers. Or maybe my brain waves are scrambled at the best of times

Though Discourse has been in almost every respect a breath of fresh air
compared to our old mailing lists, its choice to truncate incoming email
messages at things that it identifies as potentially indicating the
start of a "reply-to" chain is a major hassle for those of us who
interact with it a lot via email. The best bets to avoid this are to
post through the web interface, get out of the habit of using --- style
separators, or to mix alphanumeric characters within them like ---
details follow ---.

I will get out of my old habits as I only use the web interface when I
have a big screen in front of me.

Regards - Damian

Pacific Engineering Systems International, 277-279 Broadway, Glebe NSW 2037
Ph:+61-2-8571-0847 .. Fx:+61-2-9692-9623 | unsolicited email not wanted here
Views & opinions here are mine and not those of any past or present employer

5

I am not aware of a deep reason for this. However, we are looking for better
naming etc. for these methods. Please contribute to the discussion on Issue
#17127 .

I will. I went fishing for issues but my choice of search keywords must
have been sub-optimal. Then again, I only just learnt how to search for a
PR on GitHub so my search skills are sub-par anyway.

Given a list/1D-array say 'x' in Python, then

x[+1:]

returns a vector the same as 'x' but with the first element removed and

x[:-1]

returns a vector the same as x' but with the last element removed.

The name "chop" jumps at me. What does Python use for this?

Chop sounds a bit strong. It conjures up images of somebody taking out
their frustrations on a poor little defenceless block of wood with a long
handled axe. I use this word for a Binary Chop. An axe will chop, but I
want the concept of giving something a haircut or using a surgical knife!
Generally I just want to remove the first or last index from the range,
i.e.

(1..20).shave(+1) -> 2..20

or
(1..20).shave(-1) -> 1..19

Could you please clarify whether you'd like to make is a standard method
of Chapel's range type or you'd like to add it to your own code. If the
former, a good way to proceed is to open an issue on github proposing
this and asking for feedback. Or -- given the small size of the proposal
-- make a PR that adds the code you sketched out and discuss it there.

Thanks for the guidelines. I had forgotten about non-integral subscripts
which I have used less than 100 times in my life. As this may be relevant
to others, it probably should be a standard method. I might open an issue.
In the meantime, to address the urgency of the underlying problem, I might
just charge on and do my own thing.

I think your code works well in common cases. Where I see it can hit
corner cases is (a) idxType is an unsigned integer or an enum or bool,
and (b) with some unusual or ambiguous alignment. If you'd like your
method to be a standard range method, then you get to use the magic
that's in ChapelRange.chpl. Otherwise maybe you don't need to cater to
those, especially when you do not expect ambiguous alignment or
non-integer idxType in your code.

I will create an issue on GitHub because I clearly need some guidance.

Yes it is.

The reasoning is this. Ranges that are !stridable have more efficient
implementation, whereas ranges that are stridable provide more expressive
power.

I understand the former but don't understand the latter.

I can see it being confusing.

Very. Especially to users who come from Fortran/Matlab backgrounds which
is where most of my colleagues and clients are/were at some stage.

If you have suggestions, please contribute on github issue 17131 .

I will. But nomenclature is often hard, especially when you are seeking a
name that is crisp and not long-winded. I certainly do not have all the
answers. My point was that the word stridable is very confusing. Even I
got confused when using it initially.

To our defense, if it is a range that can be strode only with its own
stride of 1, it is not very useful to call it "stridable".

Even that is confusing. Every range can be strode with its own stride. In
fact, it cannot be strode by anything else. Only a subrange of a range can
be strode by a stride which is different to that of the original range!

Aren't you seeking be able to ascertain at compile time whether a range
has minimal complexity, i.e. a stride of +1. Only those sorts of ranges
are friendly to implementation or even vectorization. Then again, you
would need to start identifying

const t = 1 .. 20 by 1;

My 2c.

Regards - Damian

Pacific Engineering Systems International, 277-279 Broadway, Glebe NSW 2037
Ph:+61-2-8571-0847 .. Fx:+61-2-9692-9623 | unsolicited email not wanted here
Views & opinions here are mine and not those of any past or present employer

6

Apologies for using tabs in the last posting. Discourse got confused and some things that should have been indented were not.

I think what is currently wrongly described as not stridable, i.e.

!stridable

is really just a simple range, i.e.

const r = lo .. hi;

as opposed to one which is far more complex. Hmmm, simple, simplistic, naive, facile, shallow, ..., but certainly not not stridable

How do I overload a method name such that the same method name can return either a range which is simple or a range which is not simple? It seems like a method cannot return both.

7

If the condition that decides between the two cases is a param value/expression (that is, can be computed at compile-time) then you can do this in several ways:

overloads:

proc foo(): t1 where myParamExpr { ... }
proc foo(): t2 where !myParamExpr { ... }

(optionally, either of the where clauses above could be left off, and it would be the "default" case if the other where clause didn't apply)

inferred return type with a param conditional:

proc foo() {
  if myParamExpr then
    return t1Expr;
  else
    return t2Expr;
}

explicit return type computed using a param conditional, a param expr, a type function, etc.:

proc foo(): if myParamExpr then t1 else t2 { ... }
proc foo(): typeExpr(myParamExpr) { ... }
proc foo(): typeFn(myParamExpr) { ... }

If the condition can't be computed at compile-time, then such overloading isn't possible because the compiler wouldn't be able to reason about the type being returned. You'd instead have to return a type that was sufficient to represent both conditions.

-Brad