Friday Fun: Creating Sample Files

documentNot too long ago I came across a PowerShell snippet, probably via Twitter, that showed how to use a few classes from the System.IO namespace to create dummy files. Sadly I forgot the record where I saw this first mentioned. What I liked about the code I saw was the ability to create a dummy file of any size. You’re probably wondering why you would ever want to do this. Way, way back in the day when storage was expensive I used to create a few 100MB of dummy files using an old command line utility which I think was called creatfil.exe. The reasoning was that if the server started running out of disk space unexpectedly, I could delete the dummy files and buy myself a little time and breathing room. Fortunately, I think we’ve advanced where that technique is no longer necessary.

But, where you might want some dummy or sample files, is to simulate a user folder with a variety of files. You could use this test directory to hone your PowerShell scripts. Any you may have your own reasons for needing a bunch of sample files. So I took the original code and ran with it, as I usually do. I came up with a function called New-SampleFile.

The function will create a randomly sized file in the specified folder. The default is the current location. As you look through the code you’ll see I’ve added a test to verify the path and to make sure it is a FileSystem path and not something like the registry. Also by default the function will use a randomly generated file name which will be something like rcstwqyg.34e. Personally, I’m going to specify a filename. The function will create the file of a random size between 5 bytes and the value of the MaximumSize parameter, which has a default of 100KB.

The function will not overwrite existing files unless you use –Force. Nor will you see any output unless you use –Passthru. Even those are common parameters, I had to write the PowerShell expressions to implement them. The same is true of –WhatIf. You’ll see in the cmdletbinding attribute that the function supports ShouldProcess. However, the .NET classes I’m calling have no concept of WhatIf. I had to handle that myself

Here’s the function in action:

This also makes it easy to create multiple files.

Or how about this? I want to create a variety of files with different names.

Now I have some test files I can work with

Normally I prefer to use cmdlets wherever possible. But since there is no cmdlet from Microsoft to create dummy files, I have no problem resorting to the .NET Framework. However, I think it is still worth the time to wrap the .NET commands in a user-friendly and easy to use advanced function as I’ve done today.

If you find a way to use this function, I hope you’ll let me know.

Find and Replace Text with PowerShell

magnifying-glass I’ve just finished up a series of tweets with a follower who had a question about finding and replacing a bit of data in a text file. In his case it was a web.config file but it really could be any text file that you can view in PowerShell. In PowerShell 3.0 and later this isn’t too difficult. Although I haven’t tested with really large files to determine if there are any limitations. Before you try anything I’m going to show you, please have a backup copy of of the original file.

Let’s say I have a file like this:

I need to change the IP source address in the connection string. As you probably guessed Get-Content is going to come into play. Normally when you use Get-Content you get an array of strings which adds a little complexity. But starting in v3 Get-Content has a -Raw parameter which writes the entire file as a single string. If we have a single string then we can easily use -Replace.

Your replacement pattern could even be a regular expression pattern. This could even be done as a one-liner:

For many of you, especially with unstructured text files, this technique should get the job done. But in this case, because the text file is also an XML document we could tackle the problem a different way. This is especially useful if you have several changes you want to make to the file.

First, create an XML document from the file.

We can navigate the document until we find the entry we need to change.

xmlconfig

We can assign a new value with revised connection string:

All that remains is to save the file back to disk.

Or, if you didn’t necessarily know the structure of your document, you could use an XPath query with Select-XML to find the node and then change the value.

The node properties will vary depending on your XML file so you will need to look at the object.
xmlnode

The XML approach is a bit more involved but is predictable and you don’t have to save the file until you have the XML just the way you want it.

One thing I will say about both techniques is that the more you understand how to use regular expressions in PowerShell the more you can accomplish. If you are feeling a little wobbly on the topic, there is an entire chapter on regular expressions in PowerShell in Depth.

Renaming Files with PowerShell

I am not a big fan of file names with spaces. I realize they are easy to read and obviously much more “natural”, but they are simply a royal pain to deal with, especially when working from a command prompt or PowerShell. So naturally the solution is to rename these files and replace the space with something else like an underscore (_). Here’s how I’ll tackle this using Windows PowerShell. Continue reading “Renaming Files with PowerShell”

Get File Utilization by Extension

In the past I’ve posted a few PowerShell functions that provide all types of file and folder information. The other day I had a reason to revisit one of them and I spent a little time revising and expanding. This new function, Get-Extension will search a given folder and create a custom object for each file extension showing the total number of files, the total size, the average size, the maximum size and the largest size. At it’s core, the function takes output from Get-ChildItem and pipes it to Measure-Object. But I’ve incorporated features such as filtering and the ability to run the entire function as a background job.

By default, the function searches the top level of your $ENV:Temp folder and returns a custom object for each file type.

Here’s how this works.

The function uses a few parameters from Get-ChildItem, like -Include, -Exclude and -Force. If you use one of the filtering parameters, then you also need to use -Recurse. You can specify it, or the function will automatically enable it if it detects -Include or -Exclude.

Obviously (I hope), this only works on the file system. But I went ahead and added some code to verify that the specified path is from the FileSystem provider.

Normally, I’m not a big fan of Return. But in this situation it is exactly what I want since I want to terminate the pipeline. I could have also thrown an exception here but decided not to get that wild. Assuming the path is valid, the function builds a command string based on the specified parameters.

The function will invoke this string using Invoke-Expression and filter out any folders since all I care about are files.

The results are then grouped using Group-Object. Each extension group is piped to Measure-Object to calculate the statistics based on the file’s length property.

Lastly, the function creates a custom object representing each file extension using the New-Object cmdlet.

Because I’m writing an object tot he pipeline you can further sort, filter, export or whatever. This is what makes PowerShell so flexible and valuable to IT Pros.

One thing I quickly realized, was that scanning a large folder such as Documents folder or a file share UNC, could take a long time. I could use Start-Job with my original function, but it was a bit awkward. So I decided to include -AsJob as a parameter and move the job command into the function itself. This works because I take the entire core command and wrap it in a script block.

Because of scope the scriptblock needs parameters so I can pass it my command string and the Path variable which are used within the scriptblock. After $sb has been defined, if -AsJob was specified, the function uses Start-Job to create a background job. Otherwise, it uses Invoke-Command to execute it interactively.

Use the normal job cmdlets to get the results and manage the job. But now I can run something like this:

As always I hope you’ll let me know how this works for you. The complete script has comment based help and an optional line to uncomment at the end to create an alias for the function.

Download Get-Extension.

Content Redirection

Here’s another item I see in some submisstions of the 2010 Scripting Games that I felt I should address: the use of legacy console redirection. While technically not illegal or wrong, an example like this demonstrates (at least in my opinion) that the scripter hasn’t fully adopted the PowerShell paradigm.

While this will work and create a text file, it is not fully written in the PowerShell spirit. At the very least, you should use the cmdlets Out-File, Add-Content or Set-Content in place of the legacy console redirection operators, > and >>.   I’ll admit that 99 times out of 100 you’ll have no issues with the resulting text file.  However, there is the potential for encoding issues. The cmdlets are better designed to handle encoding as well as a few other PowerShell bells and whistles like –noclobber.

Another reason I suggest using cmdlets is that the > and >> operators are actually part of the underlying command session that PowerShell.exe is running on. It’s conceivable that some future version of PowerShell or some PowerShell hosted application won’t support these operators which means your script will fail and need revision. Stick with cmdlets from the beginning to protect your scripting investment.

Here’s a better PowerShell implementation that uses cmdlets and takes better advantage of the pipeline.

In this example I’m using Tee-Object which not only displays the properties in the console but also redirects to the text file. I’m sure you’ll agree this is better than using > and >>.  Or am I making too much out of this?