More Fun Getting PowerShell User Groups

A few days ago I posted a PowerShell function to retrieve information about PowerShell user groups. That function returned basic group information like this.

Each group on the site has its own page which is what that Link property is for. So it didn't take much work to use the same techniques as my original post to scrape information from that page. Again, I needed to analyze the source code to determine what classes and properties to use. But the final function, isn't that much different than the first one.

Function Get-PSUserGroupDetail {
<#
.Synopsis
Get PowerShell user group detail
.Description
This command will get detailed information about a PowerShell user group from Powershellgroup.org. You will need the link for the group which you can get with Get-PSUserGroup.
.Example
PS C:> $g = Get-PSUserGroup

This gets basic user group information for all groups.
PS C:\> $g | where {$_.name -match 'charlotte'} | Get-PSUserGroupDetail


Name     : Charlotte PowerShell Users Group
Email    : [email protected]
Meeting  : First Thursday of the month starting Jan 2012
Location : Charlotte
Region   : NC
Country  : United States
Mission  : The Charlotte PowerShell Users Group offers learning and support
           opportunities for IT administrators and developers working with
           PowerShell. While some meetings include guest speakers, most
           meetings are designated as "Script Club" sessions, where group
           members can bring in their own PowerShell scripts and receive
           advice and assistance from fellow group members. In addition, the
           group actively participates in the annual Scripting Games and holds
           it own mini-competitions to foster PowerShell knowledge in the
           Charlotte area.
Link     : http://powershellgroup.org/charlotte.nc

Filtering for a specific group and then getting detail.
.Link
Get-PSUserGroup
Invoke-Webrequest
#>
[cmdletbinding()]
Param(
[Parameter(Position=0,Mandatory,
ValueFromPipeline,
ValueFromPipelineByPropertyName,
HelpMessage="The URL for detailed user group information")]
[ValidateNotNullorEmpty()]
[Alias("link")]
[string]$URL 
)

Process {
write-Verbose "Retrieving group data from $url"

Try {
    $detail = Invoke-WebRequest $url -ErrorAction Stop
}
Catch {
    Throw
}

if ($detail ) {
    Write-Verbose "Processing results"

    $class = $detail.AllElements | group class -AsHashTable -AsString
    #suppress any errors for non-defined values which will leave 
    #corresponding property names with a null value
    $script:ErrorActionPreference = "SilentlyContinue"

    $email = ($class.'field field-type-email field-field-email'.innerText).Split(":")[1].Trim()
    $mission = $class.'og-mission'.innertext.Trim()
    $meeting = $class.'field field-type-text field-field-meetingtime'.innertext.Split(":")[1].Trim()
    $location = $class.locality[0].innertext
    $region = $class.region[0].innertext
    $country = $class.'country-name'[0].innerText
    $groupname = $class.title.where({$_.tagname -eq "H1"}).innerText
    
    Write-Verbose "Creating results for $groupname"

    #write a custom object to the pipeline
    [pscustomobject]@{
    Name = $groupname
    Email = $email
    Meeting = $meeting
    Location = $Location 
    Region = $region
    Country = $Country
    Mission = $mission
    Link = $Url
    }

    #reset variables in case there were errors
    Remove-variable -Name groupname,email,meeting,location,region,mission
} #if $r
} #process

} #end function

Now I can get the group detail directly from PowerShell.

If you have both commands, you can even combine them.

get-psusergroup | where {$_.name -match 'charlotte'} | Get-PSUserGroupDetail

This isn't too bad.

You could use PowerShell to get details for every single group but that can be time consuming as processing is done sequentially. One way you might improve performance is my taking advantage of the parallel foreach feature in a PowerShell workflow. I wrote another function, really more as a proof of concept that defines a nested workflow. Within this workflow, it processes a collection of links in parallel in batches of 8.

Foreach -parallel -throttlelimit 8 ($url in $links) {
Sequence {
Try {
$detail = Invoke-WebRequest $url -ErrorAction Stop
}
Catch {
Throw
}
…

Because workflows are intended to run isolated, I had to incorporate code from Get-PSUserGroupDetail, instead of trying to call it directly. Here's the complete function.

Function Get-PSUserGroupDetailAll {
<#
.Synopsis
Get details for all PowerShell user groups.
.Description
This command is similar to Get-PSUserGroupDetail, except it will use a PowerShell workflow to retrieve all user group detail in parallel. While this aids in performance this command could still take a few minutes to complete. There are no parameters.
.Example
PS C:\> Get-PSUserGroupDetailAll

Name     : Pac IT Pros - PowerShell - Sacramento
Email    : [email protected]
Meeting  : First Tuesday of the month 6-9 pm Pacific time and simulcast online
Location : Sacramento
Region   : CA
Country  : United States
Mission  : Pac IT Pros - PowerShell - Sacramento, CA
           PowerShell users group meeting in Sacramento, CA and online.
           Meetings are on
           the first Tuesday of the month, 6-9 pm (Pacific) and simulcast on
           the web.
Link     : http://powershellgroup.org/Sacramento
...
.Link
Get-PSUserGroupDetail
#>

[cmdletbinding()]
Param()

Write-Verbose "Starting $($MyInvocation.Mycommand)"
#define a nested workflow using the core code from Get-PSUsergroupdetail
Workflow GetDetail {

Param([string[]]$Links)
Foreach -parallel -throttlelimit 8 ($url in $links) {
    Sequence {
    Try {
        $detail = Invoke-WebRequest $url -ErrorAction Stop
    }
    Catch {
        Throw
    }

if ($detail ) {
    Write-Verbose "Processing results"

    #reset variables in case there were errors
    $groupname = $null
    $email = $null
    $meeting = $null
    $location = $null
    $region = $null
    $country = $null
    $mission = $null

    $classElements = $detail.AllElements | group class -AsHashTable -AsString

    Try {
        $email = ($classElements.'field field-type-email field-field-email'.innerText).Split(":")[1].Trim()
    } Catch { $email = $null }
    Try {
        $mission = $classElements.'og-mission'.innertext.Trim()
    } Catch { $mission = $null }
    Try {
        $meeting = $classElements.'field field-type-text field-field-meetingtime'.innertext.Split(":")[1].Trim()
    } Catch { $meeting = $null }
    Try {
        $location = $classElements.locality[0].innertext
    } Catch { $location = $null }
    Try {
        $region = $classElements.region[0].innertext
    } Catch { $region = $null }
    Try {
        $country = $classElements.'country-name'[0].innerText
    } Catch { $country = $null }
    Try {
        $groupname = $classElements.title.where({$_.tagname -eq "H1"}).innerText
    } Catch { $groupname = $null }

    Write-Verbose "Creating results for $groupname"

    #write a custom object to the pipeline
    [pscustomobject]@{
    Name = $groupname
    Email = $email
    Meeting = $meeting
    Location = $Location 
    Region = $region
    Country = $Country
    Mission = $mission
    Link = $Url
    }

} #if $detail
    
    } #sequence
} #foreach

} #end workflow
  
write-Verbose "Retrieving group data"
$links = Get-PSUserGroup | Select -ExpandProperty Link
Write-Verbose "Processing $($links.count) links"
#invoke the workflow
$results = GetDetail $links

#write results to pipeline without Workflow properties
$Results | Select * -ExcludeProperty PS*
Write-Verbose "Ending $($MyInvocation.Mycommand)"

} #end function

But even with parallel processing, this is still not a speedy process. Running the command on my Windows 8.1 box with 8GB of RAM and a very fast FiOS connection still took about 2 minutes to complete. But I suppose if you don't mind waiting here's what you can expect.

I will say, that having all of this information is fun to play with.

Or you could do something like this.

$us = $all | where {$_.country -eq 'United States'} | Group Region -AsHashTable –AsString

My last function on the topic is called Show-PSUserGroup. The central command runs my original Get-PSUserGroup function which pipes the results to Out-Gridview. From there you can select one or more groups and each group's link will open up in your browser.

Function Show-PSUserGroup {

<#
.SYNOPSIS
Graphically display PowerShell user groups.
.DESCRIPTION
This command will display PowerShell user groups using Out-Gridview. You can select one or more groups which should open the corresponding page in your web browser.
.NOTES
NAME        :  Show-PSUserGroup
VERSION     :  1.0   
LAST UPDATED:  4/8/2015
.LINK
Get-PSUserGroup 
.INPUTS
None
.OUTPUTS
None
#>

[cmdletbinding()]
Param()

Get-PSUserGroup | 
Out-GridView -Title "Select one or more user groups" -OutputMode Multiple |
Foreach {
    #write each result to the pipeline
    $_
    #open each link
    Start $_.link
}

} #end function

Clicking OK opens each link in my browser.

If you've collected all of my functions, I recommend creating a module file. I have all of them in a module file called PSUsergroups.psm1. All you need at the end is an export command.

Export-ModuleMember -Function *

Save the file in the necessary module location and your commands are ready when you are.

NOTE: If you run a PowerShell User Group and you are not registered on this site, I strongly encourage you to do so. Otherwise you are making it very hard for people to find you.

---

---