If you are like many IT Pros that I know, you often rely on text files in your PowerShell work. How many times have you used a text file of computernames with Get-Content and then piped to other PowerShell commands only to have errors. Text files are convenient, but often messy. Your text file might have blank lines. Your list of computernames might have a extra spaces after each name. These types of issues will break your PowerShell pipeline.
ManageEngine ADManager Plus - Download Free Trial
Exclusive offer on ADManager Plus for US and UK regions. Claim now!
You can use Where-Object and filter out these types of problems. Or you can use a new function I wrote called Optimize-Text.
#requires -version 2.0
Function Optimize-Text {
<#
.Synopsis
Clean and optimize text input.
.Description
Use this command to clean and optimize content from text files. Sometimes text files have blank lines or the content has trailing spaces. These sorts of issues can cause problems when passing the content to other commands.
This command will strip out any lines that are blank or have nothing by white space, and trim leading and trailing spaces. The optimized text is then written back to the pipeline.
Optionally, you can specify a property name. This can be useful when your text file is a list of computernames and you want to take advantage of pipeline binding. See examples.
If your text file has commented lines, use the ignore parameter. As long as the character is the first non-whitespace character in the line, the line will be treated as a comment and ignored.
Finally, you can use the -Filter parameter to specify a regular expression pattern to further filter what text is written to the pipeline. The filter is applied after leading and trailing spaces have been removed and before any text is converted to upper case.
.Parameter ToUpper
Write text output as upper case.
.Parameter Ignore
Specify a character that will be interpreted as a comment character. It must be the first word character in a line. These lines will be ignored. This parameter has an alias of 'comment'.
.Parameter PropertyName
Assign each line of text a property name. This has the effect of turning your text file into an array of objects with a single property.
.Parameter Filter
Use a regular expression pattern to filter. The filtering is applied after leading and trailing spaces have been trimmed and before text can be converted to upper case.
.Example
PS C:\> get-content c:\scripts\computers.txt
win81-ent-01
serenity
quark
jdhit-dc01
novo8
PS C:> get-content c:\scripts\computers.txt | optimize-text
win81-ent-01
serenity
quark
jdhit-dc01
novo8
PS C:> get-content c:\scripts\computers.txt | optimize-text -property computername
computername
------------
win81-ent-01
serenity
quark
jdhit-dc01
novo8
.Example
PS C:\scripts> get-content computers.txt | optimize-text -prop computername | where { test-connection $_.computername -count 1 -erroraction silentlycontinue} | get-service bits | select Name,Status,Machinename
Name Status MachineName
---- ------ -----------
bits Running win81-ent-01
bits Running jdhit-dc01
bits Running novo8
Optimize the computernames in computers.txt and add a Computername property. Test each computer, ignoring those that fail, and get the Bits service on the ones that can be pinged.
.Example
PS C:\Scripts> get-content .\ChicagoServers.txt | optimize-text -Ignore "#" -Property ComputerName
ComputerName
------------
chi-fp01
chi-fp02
chi-core01
chi-test
chi-dc01
chi-dc02
chi-dc04
chi-db01
.Example
PS C:\scripts> get-content .\ChicagoServers.txt | optimize-text -filter "dc\d{2}" -ToUpper -PropertyName Computername | test-connection -count 1
Source Destination IPV4Address IPV6Address Bytes Time(ms)
------ ----------- ----------- ----------- ----- --------
WIN81-ENT-01 CHI-DC01 172.16.30.200 32 0
WIN81-ENT-01 CHI-DC02 172.16.30.201 32 0
WIN81-ENT-01 CHI-DC04 172.16.30.203 32 0
Get names from text file that match the pattern, turn into an object with a property name and pipe to Test-Connection.
.Notes
Last Updated: Sept. 8, 2014
Version : 1.0
Learn more:
PowerShell in Depth: An Administrator's Guide (http://www.manning.com/jones6/)
PowerShell Deep Dives (http://manning.com/hicks/)
Learn PowerShell in a Month of Lunches (http://manning.com/jones3/)
Learn PowerShell Toolmaking in a Month of Lunches (http://manning.com/jones4/)
****************************************************************
* DO NOT USE IN A PRODUCTION ENVIRONMENT UNTIL YOU HAVE TESTED *
* THOROUGHLY IN A LAB ENVIRONMENT. USE AT YOUR OWN RISK. IF *
* YOU DO NOT UNDERSTAND WHAT THIS SCRIPT DOES OR HOW IT WORKS, *
* DO NOT USE IT OUTSIDE OF A SECURE, TEST SETTING. *
****************************************************************
.Inputs
System.String
.Link
https://jdhitsolutions.com/blog/2014/09/using-optimized-text-files-in-powershell
#>
[cmdletbinding(DefaultParameterSetName="Default")]
[OutputType([string],ParameterSetName="Default")]
[OutputType([psobject],ParameterSetName="object")]
Param(
[Parameter(Position=0,HelpMessage="Enter some text",ValueFromPipeline=$True)]
[string[]]$Text,
[regex]$Filter,
[Parameter(ParameterSetName="object")]
[string]$PropertyName,
[Alias("comment")]
[string]$Ignore,
[switch]$ToUpper
)
Begin {
Write-Verbose "Starting $($MyInvocation.Mycommand)"
Write-Verbose "Using parameters: `n $($PSBoundParameters.GetEnumerator() | format-table | out-string)"
} #begin
Process {
foreach ($item in $text) {
#filter out items that don't have at least one non-whitespace character
if ($item -match "\S") {
#trim spaces
Write-Verbose "Trimming: $item"
$output = $item.Trim()
if ($Filter) {
Write-Verbose "Filtering"
$output = $output | where {$_ -match $filter}
}
#only continue if there is output
if ($output) {
Write-Verbose "Post processing $output"
if ($ToUpper) {
$output = $output.toUpper()
} #if to upper
#filter out if using the comment character
if ($Ignore -AND ($output -match "^[\s+]?$Ignore")) {
Write-Verbose "Ignoring comment $output"
} #if ignore
else {
if ($PropertyName) {
#create a custom object with the specified property
New-Object -TypeName PSObject -Property @{$PropertyName = $Output}
}
else {
#just write the output to the pipeline
$output
}
} #else not ignoring
} #if output
} #if item matches non-whitespce
} #foreach
} #process
End {
Write-Verbose "Ending $($MyInvocation.Mycommand)"
} #end
} #end function
#define an alias
Set-Alias -Name ot -Value Optimize-Text
I've tried to add features to this command for what I think are common issues with text files. The core features are to remove any blank lines and trim leading and trailing spaces from each line of text. I'm working under the assumption that you will be using a command like Get-Content with some sort of list that you want to use with another cmdlet. You can insert Optimize-Text, in between.
For example, perhaps you want a quick and dirty ping test for a list of computers:
get-content .\computers.txt | optimize-text | where {Test-Connection $_ -count 1 -ea 0}
In this example, computers.txt is pretty mangled. There are blank lines and some names have leading and/or trailing spaces. Using Optimize-Text, fixes those issues. But wait...there's more!
The advantage to using PowerShell is that everything is an object. Often, it helps to take advantage of pipeline binding. For example, the Computername parameter for Test-Connection accepts pipeline input by property name. So if the incoming object has a property that matches the parameter name, it will use it. Optimize-Text allows you to specify a property name. When you do, each line is turned into a custom object with a single property name.
PS C:\scripts> get-content .\computers.txt | optimize-text -PropertyName Computername Computername ------------ win81-ent-01 serenity quark jdhit-dc01 novo8
This means I can run a command like:
PS C:\scripts> get-content .\computers.txt | optimize-text -PropertyName Computername | test-connection -count 1 -erroraction SilentlyContinue Source Destination IPV4Address IPV6Address Bytes Time(ms) ------ ----------- ----------- ----------- ----- -------- WIN81-ENT-01 win81-ent-01 172.16.30.127 fe80::bd30:b56e:c9fa:87e1%18 32 0 WIN81-ENT-01 jdhit-dc01 172.16.10.1 32 0
And if you download this function today, I'll throw in parameters to convert each line of text to upper case, to ignore lines with a specified comment character and to do additional filtering with a regular expression pattern!
PS C:\scripts> get-content .\ChicagoServers.txt | optimize-Text -ToUpper -Ignore "#" -filter "dc\d{2}" -PropertyName Com
putername | get-ciminstance win32_logicaldisk -filter "deviceid='c:'" | Select DeviceID,Size,Freespace,PSComputername
DeviceID Size Freespace PSComputerName
-------- ---- --------- --------------
C: 15999168512 1005551616 CHI-DC01
C: 12777943040 2873114624 CHI-DC02
C: 42946523136 25388650496 CHI-DC04
By the way here's what the text file looks like:
#Globomantics chicago servers chi-fp01 chi-fp02 chi-core01 chi-test #these are the DCs chi-dc01 chi-dc02 chi-dc04 chi-db01 chi-hvr2 #end of file
While it would be nice to think that all of the text files you use in PowerShell and neat and tidy, there's no guarantee someone else might not come along and mess it up again. Hopefully, this function will help.
Please let me know how this works for you in the real world, or what other common problems you run into with text files. Be sure to look at full help and examples. Enjoy!
