28163, "e-kayrakli", "Mason console output contents and how it should be tested", "2025-12-09T20:41:03Z"
Before my #27857, I had found mason's default output to be a bit random, where some of the messages printed out on the console felt too developer-oriented, while I felt some important information to be lacking.
As part of that PR, I added a relatively more advanced capability to print out more information on the console with more control on verbosity. That PR switches to that new strategy only partially. However, that still caused many tests failures as existing mason testing locks in the console output from mason.
In my follow up PR #28139, I added MASON_QUIET environment variable to quiet down the new logger as a band aid. That caused some output to be removed from the good files, which I think was appropriate in any case (to me, it was just cluttering the output). However, to be able to quickly quiet down nightly failures, I also reverted some of #27857 to go back to writelns such that good files match. It seems like, we still need to make some more reversions as of today.
This issue is to discuss a more principled approach for mason's console output and how mason testing works.
My opinions on the matter, in rough order of how strong I feel about it:
Mason needs multiple levels of verbosity in its output.
That control should not be based on subcommands, but on mason itself.
Except for a limited set of testing specifically for console output, we should not lock in console output as part of our correctness testing. Especially considering mason not being very mature, having many good files with console output feels like a recipe for slowing down development to me.
We can and should lock in the output from applications (e.g. with mason run) and testing (e.g. with mason test), though.
The band-aid I put in with MASON_QUIET should not be a long-term solution, however, we can imagine multiple levels of verbosity to be "passed" via environment variables.
More mason testing should use shell-based launches so that verbosity can be controlled more easily.