Using Optimized Text Files in PowerShell

documentIf you are like many IT Pros that I know, you often rely on text files in your PowerShell work. How many times have you used a text file of computernames with Get-Content and then piped to other PowerShell commands only to have errors. Text files are convenient, but often messy. Your text file might have blank lines. Your list of computernames might have a extra spaces after each name. These types of issues will break your PowerShell pipeline.

You can use Where-Object and filter out these types of problems. Or you can use a new function I wrote called Optimize-Text.

I’ve tried to add features to this command for what I think are common issues with text files. The core features are to remove any blank lines and trim leading and trailing spaces from each line of text. I’m working under the assumption that you will be using a command like Get-Content with some sort of list that you want to use with another cmdlet. You can insert Optimize-Text, in between.

For example, perhaps you want a quick and dirty ping test for a list of computers:

In this example, computers.txt is pretty mangled. There are blank lines and some names have leading and/or trailing spaces. Using Optimize-Text, fixes those issues. But wait…there’s more!

The advantage to using PowerShell is that everything is an object. Often, it helps to take advantage of pipeline binding. For example, the Computername parameter for Test-Connection accepts pipeline input by property name. So if the incoming object has a property that matches the parameter name, it will use it. Optimize-Text allows you to specify a property name. When you do, each line is turned into a custom object with a single property name.

This means I can run a command like:

And if you download this function today, I’ll throw in parameters to convert each line of text to upper case, to ignore lines with a specified comment character and to do additional filtering with a regular expression pattern!

By the way here’s what the text file looks like:

While it would be nice to think that all of the text files you use in PowerShell and neat and tidy, there’s no guarantee someone else might not come along and mess it up again. Hopefully, this function will help.

Please let me know how this works for you in the real world, or what other common problems you run into with text files. Be sure to look at full help and examples. Enjoy!

Scraping Sysinternals

download-thumbRecently I was conversing with someone about my PowerShell code that downloads tools from the live Sysinternals site. If you search the Internet, you’ll find plenty of ways to achieve the same goal. But we were running into a problem where PowerShell was failing to get information from the site. From my testing and research I’m guessing there was a timing issue when the site is too busy. So I started playing around with some alternatives.

I knew that I could easily get the html content from through a few different commands. I chose to use Invoke-RestMethod for the sake of simplicity.

Because $html is one long string with predictable patterns, I realized I could use my script to convert text to objects using named regular expression patterns. In my test script, I dot source this script.

Now for the fun part. I had to build a regular expression pattern. Eventually I arrived at this:

If you have used the Sysinternals site, you’ll know there is also a Tools “subfolder” that appears to be essentially the same as the top level site. My pattern is for the top level site. Armed with this pattern, it wasn’t difficult to create an array of objects for each tool.


Next, I check my local directory and get the most recent file.

Then I can test for files that don’t exist locally or are newer on the site.

If there are needed files, then I create a System.Net.WebClient object and download the files.

The end result is that I can update my local Sysinternals folder very quickly and not worry about timing problems using the Webclient service.

More Fun with String Properties

The other day I posted an article about converting string properties that you might get from running a command line tool into a PowerShell named property. I was continuing to experiment with it. Here’s some code on how I could use it.

I end up with output like this:

It would be nice to clean up the values and remove non word characters like the “<“. But I wouldn’t want to remove the period from the image property. So I came up with this:

Each data element is cleaned up using the Replace operator to replace any character in the regular expression pattern with nothing. Then I started experimenting with command line output that display as a series of lines like ipconfig /displaydns. First, I have to decide what to process so I want to get all lines that have ‘. . .’ in it.

Looking at the output I can see that basically I get an “object” for every group of 5 lines.

This code counts in groups of 5 and gets each group of elements using a range of index numbers. Each line in the range is then split into 2 strings on the first : character. The first item will be the property name which is converted into a string property and the second item will be the data. This gives me a custom object:

Which lead to the next challenge. The last line is being interpreted as two separate properties which is a bit of a mess. So I need a way to rename that line. Or actually any line. Maybe instead of TimeToLive I want to use TTL. Here’s the next step.

I have to know in advance what line number each property will be on. But since I do I’ll create a hashtable that uses the line number (starting at 0) as the key. The hashtable value will be the replacement property name. In my process scriptblock I’m keeping track of how many lines I’ve worked on. If the counter matches a key in the hashtable I do a replacement. Now I get this:

At this point I recognize that I have another repeatable code that I could turn this into a function.

This function relies on Convert-StringProperty. At some point I should package this and few other things I have into a module. But for now you’ll have to make sure the other function is loaded into your shell. The default processing assumes you have tabular output like you get from arp -a. You can specify input that is in a list but then you also need to specify how many lines are grouped together. You can also specify a hashtable of replacement property names for either input type. To keep things easier, the hashtable keys start counting at 1.

You can’t pipe into the command. You have to specify the input like this:


Here are some other examples using tabular output including a property rename.


Using my IPCONFIG example from earlier, here’s I can do it with my new function.

In the grid view I can further filter or sort.


One remaining issue here is that everything is a string. But in that regard it really isn’t that much different than when you import a CSV file. I guess this gives me something else to work on. In the meantime, try this out and let me know what you think.

Convert a String to a PowerShell Property Name

talkbubbleOver the last few years I’ve written and presented a bit on the idea of turning command line tools into PowerShell tools. We have a lot of great CLI based tools that are still worth using. What I’ve done is come up with tools and techniques for turning their output into an object that can be used in the PowerShell pipeline. Often all I need to do is parse and clean up command line output. But one thing that has always nagged me is what to use for property names.

For example, I can pretty easily turn output from the ARP.EXE command into objects. Here’s what I start with.

What I want to do is take the column headings and turn them into properties. The problem is I don’t like spaces in property names. Plus, I would need to know in advance the command line heading so I could use something like a custom hashtable to rename. I was after something a bit more convenient and something that would work with almost any command line output, although I think tabular output works best. Thus I came up with a short function I call Convert-StringProperty.

Here’s how it works. I’ll take the raw ARP output and skip the first couple of lines.

The $raw variable has the data I want to turn into objects.

The first line contains the property names but I want them without the spaces. As a separate step, outside of the function, I need to split the first line. I’m going to do that with a regular expression pattern that matches 2 or more white spaces.

I can take the first line, item [0], remove leading and trailing spaces and split it. This will give me three strings: Internet Address, Physical Address, and Type. Each of these is then piped to my Convert-StringProperty.

The function will look at each string and split it again based on a delimiter, which by default is a space. But you can specify something different if you run into CLI names like INTERNET_ADDRESS. Each word is then processed with a capital first letter. The end result is camel case so “Internet Address” becomes “InternetAddress”.

Once I know what my property names will be, I can continue parsing the command line output and create a custom object.

You still need to come up with code to process your command line tool, but you can use this function to define proper PowerShell properties. Here’s one more example.

This takes command output like this:

And turns it into PowerShell output like this:

In another article I’ll share with you another tool that takes advantage of this function. Enjoy.

Friday Fun: Find File Locking Process with PowerShell

I was asked on Twitter this morning about a way to find out what process has a lock on a given file. I’m not aware of any PowerShell cmdlet that can do that but I figured there had to be a .NET way and if I could find a code sample I could put something together in PowerShell. After some research I came to the conclusion that programmatic approaches were way above my pay grade but there is still an option I came up with.

I’m a big believer in the right tool for the job and just because you can do something in PowerShell, doesn’t mean you should. The best tool for this job is Handle.exe from the Sysinternals suite which you can download for free. You can specify part of a file name and it will show you the process that has a handle to that file.


But why not have your cake and eat it too? I can take this output and turn it into a PowerShell object. I posted a function recently to convert command line output to objects using regular expressions and named captures. All I need is a regular expression pattern.


Or you can create something specific to this task.

This requires Handle.exe. I’ve hardcoded the path which you need to adjust or make sure the utility is in your PATH. But now look what I can do!

I know some people get hung up on external dependencies but if it gets the job done I don’t see the issue. Especially when the solution is free to begin with!

Enjoy your weekend.