Why Doesn’t My Pipeline Work?

talkbubble I saw a little discussion thread on Twitter this morning which I felt needed a little more room to explain. Plus since we’re in ScriptingGames season beginners might like a few pointers. I always talk about PowerShell, objects and the pipeline. But sometimes what looks like a pipelined expression in the PowerShell ISE doesn’t behave the way you might expect.

Here’s an example.

If you run this, you’ll see numbers 1 to 5 written to the pipeline. But if you try something like this it will fail.

You’ll get an error about an empty pipe. In fact, in the PowerShell ISE you’ll get a red squiggle under the | indicating this is not going to work. That’s because PowerShell isn’t writing to pipeline at the end of the scriptblock, but rather within in. Another way to think about it is at the While operator is not a cmdlet so the only thing writing objects to the pipeline is whatever commands are within the While loop.

What you can do is something like this:

Here, I’m capturing the pipeline output from the scriptblock and saving it to a variable. Then I have objects I can use. Or if you wanted to be clever, you could use a subexpression.

This same behavior also applies to Do and the ForEach enumerator. The latter trips people up all the time.

You think you’ll get the output of ForEach saved to the file, but you’ll run into the empty pipeline again. You could use a variable and then pipe the variable to the file or use a subexpression. Even better, use a pipelined expression.

Here I’m using the cmdlet ForEach-Object, which unfortunately has an alias of ForEach which confuses PowerShell beginners. So don’t assume that just because you see a set of { } that you get pipelined output. Remember, cmdlets write objects to the pipeline, not operators.

Pipeline Power

Last week I came across a blog post that had a decent example using PowerShell and PowerCLI to get the disk location for all virtual machines. The posted code works and does display the information you might be after.

$myVMs = get-vm

foreach($vm in $myVMs){
$myDisks = @($vm | get-harddisk)
foreach ($disk in $myDisks) {
write-host $vm.Name, ( $disk|select -ExpandProperty Filename)

But I saw an teaching opportunity. Because the code works I can’t say it is “wrong”, but it really doesn’t adopt the PowerShell paradigm. The first issue is that using Write-Host only writes content to the console. There is no way with this command to do anything else with the results such as sorting, grouping or sending to a text file.

The other issue is the need to use ForEach. This is what we had to do in VBScript but in PowerShell we can take advantage of the pipeline.

get-vm | Select Name,@{Name="Disk";Expression= {$_ | get-harddisk | Select -ExpandProperty Filename }}

But now I can do something with this such as sorting by disk:

PS S:\> get-vm | Select Name,@{Name="Disk";Expression= {$_ | get-harddisk | Select -ExpandProperty Filename }} | sort Disk,Name

Name Disk
---- ----
Cluster Alpha {[datastore1] Cluster Alpha/Cluster ...
Globomantics Mail [datastore1] globomantics mail/Win2K...
MyCompany Exchange 2007 {[datastore1] MyCompany Exchange 200...
MyCompany XP {[datastore1] MyCompany XP/MyCompany...
MyCompany Windows 2008 [datastore1] MyCompany2008/Windows S...
MyCompanyDC 2K3R2 {[datastore1] MyCompanyDC 2K3R2/MyCo...
R2 Server Core -DEL [datastore1] Research Core DC/R2 Ser...
Cluster Bravo {[datastore2] Cluster Bravo/Cluster ...
MyCompany Vista {[datastore2] MyCompany Vista/Vista ...

Or if there are multiple disks, it is much easier to work with them. Write-Host can’t.

PS S:\> $vminfo=get-vm | Select Name,@{Name="Disk";Expression= {$_ | get-harddisk | Select -ExpandProperty Filename }}
PS S:\> $vminfo[1].disk
[datastore2] MyCompany Vista/Vista Baseline.vmdk
[datastore2] MyCompany Vista/MyCompany Vista.vmdk
PS S:\> $vminfo | Export-Clixml c:\work\vminfo.xml

The tricky part here I realize is pulling up a value from a nested object, in this case the Filename and adding it to the VM object. I totally get that this is not something a beginner would necessarily discover on their own, which is why I write stuff like this. But the big difference is that I know have an object written to the pipeline that I can do something with and I didn’t have to resort to keep track of what goes where in some foreach loops.

The other advantage, although not universal, is performance. Running the ForEach code against my 23 VMs took almost 6 seconds. My PowerShell one line took a tad over 3 seconds.

I don’t want you to think you can never use Write-Host or ForEach. Sometimes they make sense and may even perform better. But always ask yourself if you are thinking the PowerShell way and pushing objects through the pipeline or are you writing something that could be mistaken for VBScript.

By the way, I have posted most of this on the blog as a comment that is awaiting moderation. But I figured I would share it with my readers as well.

ForEach or ForEach-Object

I came across a post the other day that explained differences when using the ForEach enumerator and the ForEach-Object cmdlet. They both essentially do the same thing but as the post mentions there are potential performance differences. One other difference I want to highlight is that the ForEach enumerator doesn’t write to the pipeline at the end.

By that, I mean a command like this will fail:

[cc lang=”DOS”]
PS C:\> foreach ($i in (1..10)) {$i*2} | out-file $env:temp\data.txt
An empty pipe element is not allowed.
At line:1 char:33
+ foreach ($i in (1..10)) {$i*2} | <<<< $env:temp\out-file data.txt + CategoryInfo : ParserError: (:) [], ParentContainsErrorRecordException + FullyQualifiedErrorId : EmptyPipeElement [/cc] Of course there are workarounds but you have to apply them to every pipelined object in the {}. Compare that to using ForEach-Object. [cc lang="DOS"] PS C:\> 1..10 | foreach-object {$_*2} | out-file $env:temp\data.txt

With the enumerator you get to name your own variable. With ForEach-Object you use the $_ placeholder. Personally, I think of the ForEach enumerator as a legacy construct from the VBScript days. That’s not to say you should never use it, but you need to look at the bigger picture of what you are attempting to accomplish and make sure you are doing it the “PowerShell” way.

Friday Fun Create Numbered File

I was working on my guest commentary for the upcoming Scripting Games and started thinking I would need a line numbered version of my solution to help explain. Turns out I didn’t go down that road, but in the process I put together a little PowerShell to take a text file and create a line numbered version. Continue reading