24583, "mppf", "How should we adjust 'proc sgn' and 'proc bigint.sgn'?", "2024-03-11T21:23:12Z"
PR #24582 made the proc sgn overloads as well as proc bigint.sgn unstable. This is largely a result of the discussion in What is the value of sgn(NaN) or sgn(nan) which occured just before the 2.0 release. The discussion there is focused on being able to get a floating-point NaN result from sgn(NaN), which is not possible with the current return type of int(8) for the real overload.
That PR makes the integer and bigint overloads unstable in addition to the real overload in order to leave room for reconsideration of the entire family of functions. This might include changing the name or making the return type always match the argument type.
Reconsidering the name
My understanding is that the name of the function proc sgn came from GMP via the bigint module (in GMP it is mpz_sgn.
It's likely that we'll want to make proc sgn(x: real(?w)) return a real(w) based upon the discussion in What is the value of sgn(NaN) or sgn(nan) . In particular, it would be useful for this function to be able to propagate NaN values and to differentiate between positive and negative zeros.
The mpz_sgn probably returns a regular integer (and not a mpz integer) because it avoids an allocation and the result will necessarily be very small. That seems reasonable to continue with as a kind of performance optimization for bigint.
The Rust signum function always returns the same type as the result. This seems to be a useful consistency as well.
One way to navigate these constraints would be:
proc sgn(x: real(?w)) returns a real(w)
proc sgn(x: integral) returns x.type
proc bigint.sgn() returns an int (to avoid allocation)