Friday Fun with PowerShell and Alternate Data Streams

I have always had a peculiar fascination with alternate data streams. This is an NTFS feature that has been around for a long time. The feature, also referred to as ADS, allows a user to write data to a hidden fork of a file. You can store practically anything in an alternate data stream without affecting the reported file size. As with anything, there is always the chance of abuse or worse. But I'm going to assume you have adequate security in place. You cannot disable ADS, so maybe we can do something useful with it.

Finding Streams

The first step is to learn how to identify alternate data streams in a file. You can use Get-Item and the Streams parameter. Fortunately, the parameter accepts wildcards.

The stream :$DATA is the default stream for the file contents. You'll find this on every file. Here's a file that includes a second data stream.

I put together this simple PowerShell script to make it easier to identify extra alternate data streams.

#requires -version 5.1

Param([string]$Path = "*.*")

Get-Item -Path $path -stream * | Where-Object {$_.stream -ne ':$DATA'} |
Select-Object @{Name="Path";Expression = {$_.filename}},
Stream,@{Name="Size";Expression={$_.length}}

Getting Stream Content

I can see that these files have alternate data streams. But what is in them? Let's look.

The Stream parameter does not accept wildcards, so you need to know the name in advance. No problem. I'll use my script.

C:\scripts\Get-AlternateDataStream.ps1 .\*.db | Select Path,Stream,@{Name="Content";Expression={Get-Content $_.path -stream $_.stream}}

You could easily turn my code into a PowerShell function.

Creating Alternate Data Streams

To create your own alternate data streams, you can use the other Content cmdlets.

dir *.zip | Set-Content -Stream "source" -value "over the rainbow"

What kind of information could we store that would be independent of the file contents? How about a version number?

set-content .\Get-AboutOnline.ps1 -Stream version -Value "0.9.0"

Or maybe author information?

set-content .\Get-AboutOnline.ps1 -Stream author -Value "Jeff Hicks"

Adding this information does not alter the file contents nor change the file size. But it will update the LastModfiedTime.

Clearing Streams

You can easily clear the content of a stream.

clear-content .\vm.db -Stream secret

The stream itself remains with nothing in it. To delete the stream completely, use Remove-Item.

remove-item .\vm.db -Stream secret

Scripting with Alternate Data Streams

With this information, why not build some tools to use alternate data streams with our PowerShell scripting. I wrote a wrapper function around Set-Content to add a version alternate data stream.

Function Set-VersionStream {
    [cmdletbinding(SupportsShouldProcess)]
    Param(
        [Parameter(
            Position = 0,
            Mandatory,
            ValueFromPipeline,
            ValueFromPipelineByPropertyName,
            HelpMessage = "Specify a file path."
        )]
        [ValidateScript({ (Test-Path $_) -AND ((Get-Item $_).psprovider.name -eq 'FileSystem') })]
        [Alias("fullname")]
        [string[]]$Path,
        [parameter(Mandatory, HelpMessage = "Specify a version string")]
        [alias("version")]
        [string]$Value,
        [Parameter(HelpMessage = "Specify the name of the version stream. The default is 'versionInfo'.")]
        [string]$Stream
    )
    Begin {
        Write-Verbose "[$((Get-Date).TimeofDay) BEGIN  ] Starting $($myinvocation.mycommand)"
        if (-Not ($psboundparameters.containskey["Stream"])) {
            $psboundparameters.Add("Stream", "versionInfo")
        }
    } #begin

    Process {
        Write-Verbose "[$((Get-Date).TimeofDay) PROCESS] Adding version $value to stream $($psboundparameters['Stream']) in $Path"
        Set-Content @psboundparameters
    } #process

    End {
        Write-Verbose "[$((Get-Date).TimeofDay) END    ] Ending $($myinvocation.mycommand)"
    } #end

} #close Set-VersionStream

The default stream name is versionInfo, but you can specify something else or modify the code.

I can do something similar with author information.

Function Set-AuthorStream {
    [cmdletbinding(SupportsShouldProcess)]
    Param(
        [Parameter(
            Position = 0,
            Mandatory,
            ValueFromPipeline,
            ValueFromPipelineByPropertyName,
            HelpMessage = "Specify a file path."
        )]
        [ValidateScript({ (Test-Path $_) -AND ((Get-Item $_).psprovider.name -eq 'FileSystem') })]
        [Alias("fullname")]
        [string[]]$Path,
        [parameter(Mandatory, HelpMessage = "Specify an author string")]
        [alias("author")]
        [string]$Value,
        [Parameter(HelpMessage = "Specify the name of the author stream. The default is 'authorInfo'")]
        [string]$Stream
    )
    Begin {
        Write-Verbose "[$((Get-Date).TimeofDay) BEGIN  ] Starting $($myinvocation.mycommand)"
        if (-Not ($psboundparameters.containskey["Stream"])) {
            $psboundparameters.Add("Stream", "authorInfo")
        }
    } #begin

    Process {
        Write-Verbose "[$((Get-Date).TimeofDay) PROCESS] Adding author $value to stream $($psboundparameters['Stream']) in $Path"
        Set-Content @psboundparameters
    } #process

    End {
        Write-Verbose "[$((Get-Date).TimeofDay) END    ] Ending $($myinvocation.mycommand)"
    } #end

} #close Set-AuthorStream

The function accepts pipeline input, so I can quickly set this on multiple files.

dir *.ps1 | Set-AuthorStream -Value "Jeff Hicks"

Since I have a command to set the value, it might be handy to have a command to get the value and do more than giving me a simple string.

Function Get-AuthorStream {
    [cmdletbinding()]
    Param(
        [Parameter(
            Position = 0,
            Mandatory,
            ValueFromPipeline,
            ValueFromPipelineByPropertyName,
            HelpMessage = "Specify a file path."
        )]
        [ValidateScript({ (Test-Path $_) -AND ((Get-Item $_).psprovider.name -eq 'FileSystem') })]
        [Alias("fullname")]
        [string[]]$Path,

        [Parameter(HelpMessage = "Specify the name of the author stream. The default is 'authorInfo'")]
        [string]$Stream
    )
    Begin {
        Write-Verbose "[$((Get-Date).TimeofDay) BEGIN  ] Starting $($myinvocation.mycommand)"
        $splat = @{
            ErrorAction = "Stop"
        }

        if ($psboundparameters.containskey["Stream"]) {
            $splat.Add("Stream", $psboundparameters.containskey["Stream"])
        }
        else {
            $splat.Add("Stream", "authorInfo")
        }
    } #begin

    Process {
        foreach ($item in $path) {
            $splat['Path'] = Convert-Path $item
            Write-Verbose "[$((Get-Date).TimeofDay) PROCESS] Getting author stream $($psboundparameters['Stream']) from $item"
            Try {
                $info = Get-Content @splat
                [pscustomobject]@{
                    Path       = $splat.path
                    AuthorInfo = $info
                }
            }
            Catch {
                Write-Warning "The stream $Stream was not found on $path"
            }
        }
    } #process

    End {
        Write-Verbose "[$((Get-Date).TimeofDay) END    ] Ending $($myinvocation.mycommand)"
    } #end

} #close get-AuthorStream

However, since there isn't a limitation to what I can store in an alternate data stream, why not store something richer? Something that looks like this:

I wrote a proof-of-concept function to store this information as a JSON in an alternate data string.

Function Set-AuthorStreamJson {
    [cmdletbinding(SupportsShouldProcess)]
    Param(
        [Parameter(
            Position = 0,
            Mandatory,
            ValueFromPipeline,
            ValueFromPipelineByPropertyName,
            HelpMessage = "Specify a file path."
        )]
        [ValidateScript({ (Test-Path $_) -AND ((Get-Item $_).psprovider.name -eq 'FileSystem') })]
        [Alias("fullname")]
        [string[]]$Path,
        [parameter(Mandatory, HelpMessage = "Specify an author name")]
        [string]$Author,
        [parameter(Mandatory)]
        [string]$Company,
        [parameter(Mandatory)]
        [string]$Version,
        [string]$Comment,
        [Parameter(HelpMessage = "Enter the creation tile. It will saved as a UTC formatted string")]
        [datetime]$Created = (Get-Date),
        [Parameter(HelpMessage = "Specify the name of the author stream. The default is 'authorInfo'")]
        [string]$Stream
    )
    Begin {
        Write-Verbose "[$((Get-Date).TimeofDay) BEGIN  ] Starting $($myinvocation.mycommand)"
        $splat = @{
            ErrorAction = "Stop"
        }

        if ($psboundparameters.containskey["Stream"]) {
            $splat.Add("Stream", $psboundparameters["Stream"])
        }
        else {
            $splat.Add("Stream", "authorInfo")
        }

        $json = [PSCustomObject]@{
            Author  = $author
            Company = $company
            Version = $version
            Created = "{0:u}" -f $created
            Comment = $comment
        } | ConvertTo-Json

    } #begin

    Process {
        foreach ($item in $Path) {
            $splat["Path"] = Convert-Path $item
            $splat["Value"] = $json
            Write-Verbose "[$((Get-Date).TimeofDay) PROCESS] Adding author information for $author to stream $($splat['Stream']) in $item"
            write-verbose $json
            Set-Content @splat
        }

    } #process

    End {
        Write-Verbose "[$((Get-Date).TimeofDay) END    ] Ending $($myinvocation.mycommand)"
    } #end

} #close Set-AuthorStreamJson

The stream is a JSON string that I can convert into an object. Or I can use a PowerShell function.

Function Get-AuthorStreamJson {
    [cmdletbinding()]
    Param(
        [Parameter(
            Position = 0,
            Mandatory,
            ValueFromPipeline,
            ValueFromPipelineByPropertyName,
            HelpMessage = "Specify a file path."
        )]
        [ValidateScript({ (Test-Path $_) -AND ((Get-Item $_).psprovider.name -eq 'FileSystem') })]
        [Alias("fullname")]
        [string[]]$Path,

        [Parameter(HelpMessage = "Specify the name of the author stream. The default is 'authorInfo'")]
        [string]$Stream
    )
    Begin {
        Write-Verbose "[$((Get-Date).TimeofDay) BEGIN  ] Starting $($myinvocation.mycommand)"
        $splat = @{
            ErrorAction = "Stop"
        }

        if ($psboundparameters.containskey["Stream"]) {
            $splat.Add("Stream", $psboundparameters["Stream"])
        }
        else {
            $splat.Add("Stream", "authorInfo")
        }
    } #begin

    Process {
        foreach ($item in $path) {
            $splat['Path'] = Convert-Path $item
            Write-Verbose "[$((Get-Date).TimeofDay) PROCESS] Getting author stream $($splat.stream) from $item"
            Try {
                $info = Get-Content @splat
                $out = $info | ConvertFrom-Json
                $out | Add-Member -MemberType NoteProperty -Name Path -Value $splat.path -PassThru
            }
            Catch {
                Write-Warning "The stream $($splat.stream) was not found on $path"
            }
        }
    } #process

    End {
        Write-Verbose "[$((Get-Date).TimeofDay) END    ] Ending $($myinvocation.mycommand)"
    } #end

} #close get-AuthorStreamJson

The function will display a warning for files that lack the stream, so I'm suppressing the warning messages in this example.

Finally, how about adding tags?

Function Add-TagStream {
    [cmdletbinding(SupportsShouldProcess)]
    Param(
        [Parameter(
            Position = 0,
            Mandatory,
            ValueFromPipeline,
            ValueFromPipelineByPropertyName,
            HelpMessage = "Specify a file path."
        )]
        [ValidateScript({ (Test-Path $_) -AND ((Get-Item $_).psprovider.name -eq 'FileSystem') })]
        [Alias("fullname")]
        [string[]]$Path,
        [parameter(Mandatory, HelpMessage = "Specify a set of tags")]
        [alias("tag")]
        [string[]]$Value,
        [Parameter(HelpMessage = "Specify the name of the Tags stream. The default is 'tags'.")]
        [string]$Stream
    )
    Begin {
        Write-Verbose "[$((Get-Date).TimeofDay) BEGIN  ] Starting $($myinvocation.mycommand)"
        if (-Not ($psboundparameters.containskey["Stream"])) {
            $psboundparameters.Add("Stream", "tags")
        }
    } #begin

    Process {
        Write-Verbose "[$((Get-Date).TimeofDay) PROCESS] Adding Tags $($value -join ',') to stream $($psboundparameters['Stream']) in $Path"
        Set-Content @psboundparameters
    } #process

    End {
        Write-Verbose "[$((Get-Date).TimeofDay) END    ] Ending $($myinvocation.mycommand)"
    } #end

} #close Add-Tagstream

This command makes it a snap to add tags.

dir *.csv,*.db | Add-TagStream -tag "data","company"

Of course, I want an easy way to get the tag stream.

Function Get-TagStream {
    [cmdletbinding()]
    Param(
        [Parameter(
            Position = 0,
            Mandatory,
            ValueFromPipeline,
            ValueFromPipelineByPropertyName,
            HelpMessage = "Specify a file path."
        )]
        [ValidateScript({ (Test-Path $_) -AND ((Get-Item $_).psprovider.name -eq 'FileSystem') })]
        [Alias("fullname")]
        [string[]]$Path,

        [Parameter(HelpMessage = "Specify the name of the tag stream. The default is 'tags'")]
        [string]$Stream
    )
    Begin {
        Write-Verbose "[$((Get-Date).TimeofDay) BEGIN  ] Starting $($myinvocation.mycommand)"
        $splat = @{
            ErrorAction = "Stop"
        }

        if ($psboundparameters.containskey["Stream"]) {

            $splat.Add("Stream", $psboundparameters["Stream"])
        }
        else {
            $splat.Add("Stream", "tags")
        }
    } #begin

    Process {
        foreach ($item in $path) {
            $splat['Path'] = Convert-Path $item
            Write-Verbose "[$((Get-Date).TimeofDay) PROCESS] Getting tags stream $($splat['Stream']) from $item"
            Try {
                $info = Get-Content @splat
                [pscustomobject]@{
                    Path       = $splat.path
                    Tags = $info
                }
            }
            Catch {
                Write-Warning "The stream $($splat['Stream']) was not found on $path"
            }
        }
    } #process

    End {
        Write-Verbose "[$((Get-Date).TimeofDay) END    ] Ending $($myinvocation.mycommand)"
    } #end

} #close Get-TagStream

With this command, it is easy to find files based on a tag.

There's no end to how you could use alternate data streams, and you are welcome to use my code as a framework. My functions should be considered proof-of-concept and not production-ready.

Limitations

But before you get too excited, there are limitations to alternate data streams. This feature is only supported on Windows and NTFS formatted drives. If you copy a file with alternate data streams from one NTFS drive to another, the streams should also copy. But if you copy the file to a non-NTFS drive, you will lose the streams.

If you back up or archive files, you also might lose the alternate data streams. However, it is worth looking into settings. I use WinRar, and the application has a setting to save file streams.

Finally, if you synchronize files with a cloud service like OneDrive or DropBox, expect to lose the alternate data streams. I have not been able to find any way to configure these services to include alternate data streams. If you know of a way, I'd love to hear it.

Summary

I hope you found this intriguing and got your gears churning to figure out how you can take advantage of this feature. As I hope you've seen, building PowerShell tooling around alternate data streams is not that difficult. If you find a use for ADS, I hope you'll let me know.

By the way, in addition to PowerShell, you can use the streams.exe utility from SysInternals to list and delete alternate data streams.

Have fun with this, and please test with non-production files. Enjoy.