Rob Holt (rjmholt) · GitHub

More meta notes :)

As such, personally I find your tone insulting and does not further this discussion.
Previously you called me out for my use of language which seems to only seek to antagonize. This is exactly what you are doing here. Please heed some of your own advice.
You shouldn't need to coerce people into your point of view by trying to make it sound like a vote for the status quo doesn't make sense.
Second: it's rude to come to a product with 12 years of history and say things like:

(Below I'm referring to my previous post's meta note at the beginning; Mark Kraus (@markekraus), you may have misconstrued my note about "sensible" behavior: all I wanted to say there was that, divorced from its history, the name and description of -OutputVariable to me suggest that it should be on par with regular variable assignment - simply restating the intent of this RFC. No insult intended there at all.)

It's a valid point, and I apologize - I reacted to what I perceived as haughty dismissals. I will say that I am no stranger to the pitfalls I've described myself, and as such they are reminder to me as much as anyone else. In the particular context of PowerShell, it just so happens that I may need a reminder of the opposite - that, given my relative inexperience, I should be more mindful of history and real-world impact. I will do that in the future and defer to others for assessing backward-compatibility risks.

I absolutely do believe that a vote for the status makes sense, but it's important to have clarity why.
I think it is important to strive for a shared understanding of problematic existing behavior even if it is retained, so as to properly frame it in the documentation, potentially fix it down the line, and inform the design of related, future features.

In short: my (regrettable) tone aside, I wanted to have a separate discussion about the merits of the proposed change in the abstract, and, this particular discussion aside, I think the conversation actually has moved in that direction, thankfully, and let me try to continue in that spirit below.


Honestly, I feel like you jumped in here in an effort to make something easier to use, but did so without fully understanding the feature and its many benefits

Yes, I was coming from my personal use of the feature, which was like Tee-Object -Variable, as a more concise and convenient per-cmdlet alternative that also supports appending, and was then surprised to find that their behavior differed, motivating me to try to spare others the same surprise.

(i.e. the fact that these are for use within the pipeline, not just at the end).

They currently can be use that way, but it isn't obvious at all (except with the non-collecting, always-single-item -PipelineVariable).
This is evidenced by the fact that even the PowerShell committee did not consider it when they approved PowerShell/PowerShell#3154 and that expecting $var = ... and ... -OutVariable var to act the same is reasonable, based on naming and existing documentation.
Note how even the language spec. doesn't address intra-pipeline use at all.

If the consensus is that the existing behavior should be retained as-is (but see my suggestion below), the solution is therefore to improve the documentation (as Chris Gardner (@ChrisLGardner) suggested):

  • Describe how collecting values in an *Variable target variable differs from regular variable assignment (and Tee-Object -Variable).

  • Motivate the difference by explaining the intra-pipeline use of -OutVariable and its benefits (and of the other *Variable parameters, potentially, though less likely).

  • While it is definitely worth mentioning that the collection type used is not a regular PowerShell array ([object[]]), that aspect of the behavior is the least problematic one in practice, given that you may not even notice that you're talking to a [System.Collections.ArrayList] instead of [object[]].

That said, even a documented surprise will remain a surprise - both to newcomers making intuitive assumptions and to seasoned users who forget about the difference.

One of the most common complaints I get from novice users is how hard it is to comprehend just how the pipeline produces values and why you sometimes get back an Object[] vs a String, vs others. As advanced users of the language, we take that behavior for granted and to some extent we may even like it and think it is desirable, But the pipeline is already a very sour experience and a high hurdle to jump to gain basic competency of the language.

I think what you describe was true until v2, but went away in v3 with the ability to index scalars and their having a .Count property with value 1 (or 0, in the case of $null).
There is one notably wrinkle, though: PowerShell/PowerShell#2798

Understanding pipeline output may be nontrivial, but it is crucial, given PowerShell's pervasive use of pipelines. Once you have that understanding, it applies wherever pipelines are involved.
Selective deviation in places where users are likely to expect pipeline-output behavior (I certainly did) is problematic. (And there are certainly other teaching tools for explaining the pipeline).

IMO, "for the sake of parity with the pipeline" is not a convincing argument for this change from the perspective of someone new to the language.

I think the symmetry of $var = ... and ... -OutVariable var is important - you are collecting pipeline output after all (what the cmdlet at hand writes to the pipeline).

The simplest use case is to output to the screen while also capturing the output in a variable, for later use. That use case is currently broken.

(Similarly, the simplest use case for -ErrorVariable is to collect errors for later inspection; whether single-item unwrapping should occur there (and with -WarningVariable and Information-Variable) is less clear-cut, but my vote is for consistency.)

That you can currently also modify this collection as it is being built is to me a secondary consideration (and quite possibly an accidental feature) that shouldn't violate the fundamental assumption of the capture behavior.

So it sounds like the primary philosophical difference is over how to fundamentally frame the primary purpose of the -*Variable parameters and what their perceived primary use is.

Note that even advanced uses such as gci -ov items -Directory | gci -ov +items -File do not require intra-pipeline access to the collection; it is only explicit manipulations in user code - based on knowledge of the collection type - that do, such as gci -ov items -Directory | % -Process { $items.AddRange( @(dir $_ -File) ) }.

And I do wonder if the latter is worth supporting, given that it gets in the way of the simpler capture-and-inspect-later use case.

If the consensus is to grandfather in this usage, perhaps the following compromise is an option - though it ain't too pretty:

  • With multi-item output, keep the existing behavior; as stated, in terms of element access [System.Collections.ArrayList] behaves like [object[]], and users may not even notice.

  • With single-item output, unwrap it - yes, that means changing the type after the fact, but it solves the problem of the $var = ... / ... -OutVariable var symmetry.

    • New users and basic-use-case users get the behavior they expect.
    • Existing code that uses .Count and indexing to access the variable should continue to work.

Read the original on github.com ↗