Branch prediction with foreknowledge

The standard branching which occurs in an if .. then else, or worse, multi-level, variants of the same, can result in unnecessary branching.

A lot of the time, a programmer knows that a given branch will be executed most of the time.

Conventional programming languages, including Chapel,. have no way of the programmer passing that knowledge to the program.

Extensions to C have a likely/unlikely feature which allows a program to nominate which branch is more or less likely to occur. This allows branching to be avoided for the most commonly executed chunk of code.

How do we get that feature in Chapel. One way would be to provide adjuncts to the generic if keyword, namely the ifl and ifu which would respectively imply what in C could be done as

if ( likely (boolean expression) )    // ifl
if ( unlikely (bolean expression) )   // ifu

Would that fly as an extension or would it instead fly in the face of the fundamentals behind Chapel?

Without it, performance cn almost double.

Hi @damianmoz

Sorry, I'm only just now seeing/remembering this query and realizing that it didn't get a response.

Speaking for myself, while this is something that I agree can be valuable to communicate to the compiler, it isn't something that I'd personally want to add a keyword for within the language proper since it doesn't affect the correctness or behavior of the program, just its implementation and performance.

As a counterproposal, we've recently been discussing adding an annotation/attribute/tag feature to the language as a means of expressing things about the program outside of the language. There's still ongoing discussion about where the line between language features and these tags should be drawn, but my personal favorite option is that things which are meant to communicate directly to tools (where I consider the compiler itself to be a tool) and don't impact the program's behavior (modulo performance or implementation details) are fair game for these.

To that end, I might imagine expressing these as something like:

@likely true
if boolean-expression then

@likely false
if boolean-expression then

That said, I'm surprised by your comment that you think this could as much as double performance, as I'd imagine that if a given conditional were that predictably true/false and executed enough times to matter, the hardware branch predictors would do a good job of picking up on the pattern...?

-Brad

Speaking for myself, while this is something that I agree can be valuable to
communicate to compiler,

The word is more like in-valuable.

it isn't something that I'd personally want to add a keyword for within the language
proper since it doesn't affect the correctness or behavior of the program, just its
implementation and performance.

To me behavior is performance. In trying to convince Fortran users to try Chapel, its
performance and the language were one and the same thing.

.... we've recently been discussing adding an annotation/attribute/tag feature to the
language as a means of expressing things about the program outside of the language.

Annotations destroy readability in every case I have seen, not just in Chapel. I hope no numerical
code of mine ever needs annotations.

I am not sure what you mean by attributes and tags. If you mean attribute along the lines of the concepts of pure and const functions, then those attributes will be a part of the language.

... ongoing discussion about where the line between language features and these
tags should be drawn, but my personal favorite option is that things which are meant
to communicate directly to tools (where I consider the compiler itself to be a tool) and
don't impact the program's behavior (modulo performance or implementation details)
are fair game for these.

To that end, I might imagine expressing these as something like:

     @likely true
     if boolean-expression then

     @likely false
     if boolean-expression then

Two lines instead of one is bad. The if statement is not as obvious in the way the above is written.

Also, the above is not as readable as

if likely(<boolean-expression>) || unlikely(<boolean-expression>) then
{
   // block of code
}
else
{
   // alternative block of code
}

I am not convinced that my ifl and ifu are an optimal solution.

The likely and unlikely are inherent in the mathematical algorithm and its expression
in a language.

... I'm surprised by your comment that you think this could as much as double performance,

Proven. I discovered that in tests on replacement rounding routines. I was surprised too.

I was trying to write inline rounding routines in Chapel to replace the C library routines that handled all the edge cases and were IEEE 754 correct. It did not work. I got funny results. In the end I rewrote things in C, and got that right. But the code needed the concept of likely and unlikely conditions to perform well, i.e. boolean expressions. But at least I know the concept works. And yes, I know that some RISC chips have machine instructions for individual rounding modes making that work irrelevant on some newer architectures.

I'd imagine that if a given conditional were that predictably true/false and executed enough times
to matter, the hardware branch predictors would do a good job of picking up on the pattern...?

Well they did not. And my tests ran through every possible real(32), i.e. from min(real(32)) to max(real(32)) where two consecutive test scenarios differed only in their least significant bit. And
besides, why rely on run-time hardware branch predictions when I can predict it at compile time.

While I generally agree that annotations are uglier than first-class language features, here are some reasons I feel unconvinced that this specific feature warrants support in the language proper rather than via some language-adjacent capability like a tag or annotation:

  • It makes it hard to know where to stop: What is the line between adding a new feature for something that could benefit performance vs. using some other language-adjacent capability like an annotation? When do we stop reserving words for everything we can think of that a compiler might benefit from if only it knew it?

  • It's a difficult case to say precisely what a compliant compiler must do with this keyword in order to conform to the language specification. Must it perform some optimization if it sees the feature, and issue an error message if it is unable to do an optimization? That will make code non-portable if a given compiler or architecture doesn't have any way to take advantage of the information. Is it merely a hint that a compiler can make use of if it's able to and may freely ignore if not? That makes it feel pretty weak and non-binding for a language feature (in imperative languages, I don't like features that are legal to treat as no-ops because it removes any sort of "guaranteed behavior" that you'd hope to get by using the feature.

  • Related: It's a feature that doesn't really have any observable impact (apart from performance), which makes it difficult to define in a language standard. I get that for you "performance == behavior" but note that our language spec, and most language specs, don't really define their language features in terms of any type of performance guarantees that you'll get from them. It's usually more in terms of behavior where adjacent documents will talk about performance tips for particular or typical implementations.

-Brad