Pipeline Power

Last week I came across a blog post that had a decent example using PowerShell and PowerCLI to get the disk location for all virtual machines. The posted code works and does display the information you might be after.

$myVMs = get-vm

foreach($vm in $myVMs){
$myDisks = @($vm | get-harddisk)
foreach ($disk in $myDisks) {
write-host $vm.Name, ( $disk|select -ExpandProperty Filename)

But I saw an teaching opportunity. Because the code works I can’t say it is “wrong”, but it really doesn’t adopt the PowerShell paradigm. The first issue is that using Write-Host only writes content to the console. There is no way with this command to do anything else with the results such as sorting, grouping or sending to a text file.

The other issue is the need to use ForEach. This is what we had to do in VBScript but in PowerShell we can take advantage of the pipeline.

get-vm | Select Name,@{Name="Disk";Expression= {$_ | get-harddisk | Select -ExpandProperty Filename }}

But now I can do something with this such as sorting by disk:

PS S:\> get-vm | Select Name,@{Name="Disk";Expression= {$_ | get-harddisk | Select -ExpandProperty Filename }} | sort Disk,Name

Name Disk
---- ----
Cluster Alpha {[datastore1] Cluster Alpha/Cluster ...
Globomantics Mail [datastore1] globomantics mail/Win2K...
MyCompany Exchange 2007 {[datastore1] MyCompany Exchange 200...
MyCompany XP {[datastore1] MyCompany XP/MyCompany...
MyCompany Windows 2008 [datastore1] MyCompany2008/Windows S...
MyCompanyDC 2K3R2 {[datastore1] MyCompanyDC 2K3R2/MyCo...
R2 Server Core -DEL [datastore1] Research Core DC/R2 Ser...
Cluster Bravo {[datastore2] Cluster Bravo/Cluster ...
MyCompany Vista {[datastore2] MyCompany Vista/Vista ...

Or if there are multiple disks, it is much easier to work with them. Write-Host can’t.

PS S:\> $vminfo=get-vm | Select Name,@{Name="Disk";Expression= {$_ | get-harddisk | Select -ExpandProperty Filename }}
PS S:\> $vminfo[1].disk
[datastore2] MyCompany Vista/Vista Baseline.vmdk
[datastore2] MyCompany Vista/MyCompany Vista.vmdk
PS S:\> $vminfo | Export-Clixml c:\work\vminfo.xml

The tricky part here I realize is pulling up a value from a nested object, in this case the Filename and adding it to the VM object. I totally get that this is not something a beginner would necessarily discover on their own, which is why I write stuff like this. But the big difference is that I know have an object written to the pipeline that I can do something with and I didn’t have to resort to keep track of what goes where in some foreach loops.

The other advantage, although not universal, is performance. Running the ForEach code against my 23 VMs took almost 6 seconds. My PowerShell one line took a tad over 3 seconds.

I don’t want you to think you can never use Write-Host or ForEach. Sometimes they make sense and may even perform better. But always ask yourself if you are thinking the PowerShell way and pushing objects through the pipeline or are you writing something that could be mistaken for VBScript.

By the way, I have posted most of this on the blog as a comment that is awaiting moderation. But I figured I would share it with my readers as well.

Using Types with Imported CSV Data in PowerShell

The Import-CSV cmdlet in PowerShell is incredibly useful. You can take any CSV file and pump objects to the pipeline. The cmdlet uses the CSV header as properties for the custom object.

PS S:\> import-csv .\testdata.csv

Date : 1/18/2012 6:45:30 AM
Name : Data_1
Service : ALG
Key : 1
Size : 25

Date : 1/18/2012 2:17:30 AM
Name : Data_2
Service : AppIDSvc
Key : 2
Size : -30

But there is a downside: all of the properties are strings.

PS S:\> import-csv .\testdata.csv | get-member

TypeName: System.Management.Automation.PSCustomObject

Name MemberType Definition
---- ---------- ----------
Equals Method bool Equals(System.Object obj)
GetHashCode Method int GetHashCode()
GetType Method type GetType()
ToString Method string ToString()
Date NoteProperty System.String Date=1/18/2012 6:45:30 AM
Key NoteProperty System.String Key=1
Name NoteProperty System.String Name=Data_1
Service NoteProperty System.String Service=ALG
Size NoteProperty System.String Size=25

The means some tasks such sorting or filtering will fail. But there are ways to get around this limitation. One way is to use an expression to cast a property to a different type. For example, I want to sort my test data on the Date property, but it needs to be a [DateTime] object to sort properly. Here’s how:

PS S:\> import-csv testdata.csv | sort @{expression={$_.date -as [datetime]}} | Select Date,Name,Size

Date Name Size
---- ---- ----
1/9/2012 6:28:30 PM Data_25 26
1/11/2012 11:13:30 AM Data_20 44
1/11/2012 6:28:30 PM Data_23 33
1/13/2012 12:13:30 AM Data_16 42
1/13/2012 4:45:30 PM Data_24 47

My output object properties are all still strings. All I did was cast the Date property in the Sort expression. Here’s an example using filtering.

PS S:\> import-csv testdata.csv | where {($_.date -as [datetime]) -le ("1/12/2012" -as [datetime])} | Select Date,Name,Size

Date Name Size
---- ---- ----
1/11/2012 11:13:30 AM Data_20 44
1/11/2012 6:28:30 PM Data_23 33
1/9/2012 6:28:30 PM Data_25 26

These examples are only producing results. More likely I want to import the CSV file as typed objects. Assuming you know in advance the property names and what types you want to use, here’s how you could achieve this.

PS S:\> $data=import-csv testdata.csv | Select @{Name="Date";Expression={[datetime]$_.Date}}, Name,Service,@{Name="Key";Expression={[int32]$_.Key}},@{Name="Size";Expression={[int32]$_.Size}}

I imported my CSV file and piped it to Select-Object, using hash tables to redefine the properties with appropriate types. Import-CSV writes a PSCustomObject to the pipeline anyway so using Select-Object has no effect other than giving me typed properties.

PS S:\> $data | get-member

TypeName: Selected.System.Management.Automation.PSCustomObject

Name MemberType Definition
---- ---------- ----------
Equals Method bool Equals(System.Object obj)
GetHashCode Method int GetHashCode()
GetType Method type GetType()
ToString Method string ToString()
Date NoteProperty System.DateTime Date=1/18/2012 6:45:30 AM
Key NoteProperty System.Int32 Key=1
Name NoteProperty System.String Name=Data_1
Service NoteProperty System.String Service=ALG
Size NoteProperty System.Int32 Size=25

Now I can use $data objects anyway I want.

PS S:\> $data | where {$_.size -ge 40 -AND $_.key -le 10}

Date : 1/17/2012 11:57:30 PM
Name : Data_3
Service : Appinfo
Key : 3
Size : 42

I’m working on something that takes this idea to the next level but it isn’t quite ready for prime time. But I hope this will help manage imported objects a bit more efficiently and let you really take advantage of the PowerShell pipeline.

Friday Fun What’s My Variable

I use scriptblocks quite a bit in my PowerShell work, often saved as variables. These are handy for commands you want to run again, but don’t necessarily need to turn into permanent functions.

$freec={(get-wmiobject win32_logicaldisk -filter "deviceid='c:'" -property Freespace).FreeSpace/1mb}

Now in PowerShell I can invoke the scriptblock.

PS S:\> &$freec

Ok then. I have a number of these defined. I decided I wanted an easy way to identify them when I run Get-Variable. For example, if I remembered all the variable names I could just do this:

PS S:\> get-variable freec,dirt

Name Value
---- -----
freec (gwmi win32_logicaldisk -filter "deviceid='c:...
dirt Param([string]$Path=$env:temp) Get-ChildItem ...

But needless to say that’s asking too much. When I first looked at this problem I went down the path of trying to parse values I saw with Get-Variable to identify potential script blocks. Then I realized this was a rookie mistake. PowerShell is all about the objects. Now a variable is also an object with a value property. This value could be a string, and integer or a pscredential. So my task then was to identify each value type.

Every object in PowerShell has a built in method called GetType().

PS S:\> $s=get-service spooler
PS S:\> $s.GetType()

IsPublic IsSerial Name BaseType
-------- -------- ---- --------
True False ServiceController System.ComponentM...

This is actually another object with a Name property.

PS S:\> $s.GetType().name

Aha! Let’s look at this with my variable.

PS S:\> (get-variable freec).value.GetType().Name

This is a one-line shortcut that gets the Value property of the Freec variable and then runs the GetType() method followed by retrieving just the Name property. This is promising. Here’s one way I can use this:

get-variable | Where {$_.value.GetType().Name -eq "ScriptBlock"}

As you can see there is still an issue with variables with no values.

I’ll just add another condition to my Where expression.

get-variable | Where {$_.value -AND $_.value.GetType().Name -eq "ScriptBlock"}


These are in fact all of the scriptblocks in my current session. But now I can take this a step further and look at my other variables and their type.

get-variable | select Name,@{Name="Type";Expression={$_.value.GetType().Name}}

Or I might try grouping.

get-variable | select Name,@{Name="Type";Expression={$_.value.GetType().Name}} | where {$_.type} | Group Type | Sort Count -Descending

I wanted to filter out empty values so I’m only keeping objects that have a defined type in my grouped output.

The bottom line is never forget about the object!

Friday Fun – Get Ranked Object

Earlier this week on Google Plus, Hal Rottenberg posted a PowerShell idea he had. His goal was to identify a group of objects that would be statistically significant. For example, given a collection of files, group the files by size but ranked by size so that you might have a group for the largest files, then big files, then average files and finally small files. His idea was to calculate a rank using a logarithm of the file size. His code was quick and dirty so I took it, as I usually do, and ran with it. The result is a function that will analyse any group of objects and rank them based on a numeric property. Continue reading

Friday Fun – Get Number Object

You most likely know that I’m all about the object and the PowerShell pipeline. Everything in PowerShell is an object. Pipe something to Get-Member and you can discover all of the object’s properties and methods (ie its members). Some objects, like strings, have many methods but very few properties. Some objects, like numbers have very little of either. I mean, what can you really do with a number? Well, I can have a bit of fun with one and maybe teach a few PowerShell concepts along the way. Continue reading