Horizontal Logic

Friday, January 4, 2013

[math]::IEEERemainder((100/$PSItem),($PSItem/100))

Tonight's intellectual curiosity produces the charts below.

function XYZ {
-100..100 | % {
try {foreach ($i in [array[]]([math]::IEEERemainder((100/$PSItem),($PSItem/100)))) {[ordered]@{$PSItem=$($i.item(0))}}} `
catch [System.Management.Automation.RuntimeException] {}}
}

A defect for 'get-random'

In PS 3.0, 'Get-random' doesn't return an error value for a $null variable without the use of '-inputobject'. I came across this writing a function so my third grader could pick up her arithmetic skills. The function has '$Mixed' as such:

[array]$Mixed= % {
0..($questions_per_worksheet - 1) | % {write "2 $operator1 $(get-random -max $random_max -min $random_min) = "}
0..($questions_per_worksheet - 1) | % {write "4 $operator2 $(get-random -max $random_max -min $random_min) = "}
0..($questions_per_worksheet - 1) | % {write "5 $operator3 $(get-random -max $random_max -min $random_min) = "}
0..($questions_per_worksheet - 1) | % {write "10 $operator4 $(get-random -max $random_max -min $random_min) = "}
}

Data Analytics with Powershell Part I

Below is some cruft I am using to analyze PDC data for candidates in my local WA elections. This is part of a larger project I am working on to use Powershell 3.0 as a data analytic solution. I find working with data in Powershell 3.0 somewhat tricky, sometimes limited, but also no less straightforward than SQL or R. Although, Powershell can sometimes be a little frustrating to work with , I rather like being able to craft my own 'data analytic' solution from the console. I found myself close to the data while working with PS 3.0. Certain techniques produced surprisingly rapid and illuminating results. Take the query below. After having imported to a variable ('$NM') a candidates data from CSV, in one line of code I am able to exclude all WA state contributions; then use 'group-object' to list all 'out of state' contributors,state,amounts in a sort.

$NM | Where State -ne WA | group -property Contributor,State,Amount -noelement | sort -desc Count,Name | ft -auto -wrap

Count Name
----- ----
1 WEBB LISA, MT, 100
1 VASKAS JANET, PA, 100
1 VASKAS ALAN, PA, 100
1 TURNER ZACHARY M, CO, 900
1 TERESA JUDITH, WV, 50
1 SOTO JLEANA, CA, 100
1 MCCLENDON SUSAN, GA, 500
1 ATU, DC, 900

Memory Stats in the Title Bar (Part II)

Two different attempts to update memory stats in the title bar in the console background. The 'start-job' script won't work because it updates the new Powershell instance the job is running in. The 'runspace' script works because it updates the existing console.

# with Start-Job
# Doesn't Work because it updates a hidden Powershell Job window
start-job -InitializationScript {

function Global:Set-title {
$PSID=([System.Diagnostics.Process]::GetCurrentProcess()).Id
$MemStats=ps -id $PSID | Select `
@{Name='ThreadCount';Expression={($_.Threads).count}}, `
@{Name='WorkSetMB';Expression={[int](($_.WorkingSet64)/1MB)}}, `
@{Name='VirMemMB';Expression={[int](($_.VirtualMemorySize64)/1MB)}}, `
@{Name='PriMemMB';Expression={[int](($_.PrivateMemorySize64)/1MB)}}, `
@{Name='PagedMemMB';Expression={[int](($_.PagedMemorySize64)/1MB)}}, `
@{Name='NonPagedMemKB';Expression={[int](($_.NonPagedSystemMemorySize64)/1KB)}}

$Title=write "Last_Title_Stats: Time: $([datetime]::now) Version: $((get-host).Version.Major) SessionHours: $([int]([datetime]::now - (ps -id $PSID).Starttime).totalhours) Memory: $($Memstats) GC_MB: $([int]([GC]::gettotalmemory(1)/1MB))"
[console]::set_title($Title)
}

} `

-scriptblock {while(1) {set-title;sleep -s 5}}

# With Runspace
# Works because the new runspace refers to the existing Powershell window
$set_title=

{

function Global:Set-title {
$PSID=([System.Diagnostics.Process]::GetCurrentProcess()).Id
$MemStats=ps -id $PSID | Select `
@{Name='ThreadCount';Expression={($_.Threads).count}}, `
@{Name='WorkSetMB';Expression={[int](($_.WorkingSet64)/1MB)}}, `
@{Name='VirMemMB';Expression={[int](($_.VirtualMemorySize64)/1MB)}}, `
@{Name='PriMemMB';Expression={[int](($_.PrivateMemorySize64)/1MB)}}, `
@{Name='PagedMemMB';Expression={[int](($_.PagedMemorySize64)/1MB)}}, `
@{Name='NonPagedMemKB';Expression={[int](($_.NonPagedSystemMemorySize64)/1KB)}}

$Title=write "Last_Title_Stats: Time: $([datetime]::now) Version: $((get-host).Version.Major) SessionHours: $([int]([datetime]::now - (ps -id $psid).Starttime).totalhours) Memory: $($Memstats) GC_MB: $([int]([GC]::gettotalmemory(1)/1MB))"
[console]::set_title($Title)
}

while(1) {set-title;sleep -s 5}

}

$ST_Runspace = [PowerShell]::Create().AddScript($set_title)
$Begin_Set_Title = $ST_Runspace.BeginInvoke()

# Commands to query,stop,dispose the titlebar update:
# $ST_Runspace.runspace
# $Begin_Set_Title
# $Stop_Set_Title = $ST_Runspace.Stop()
# $Dispose_Set_Title = $ST_Runspace.Dispose()

Monday, August 27, 2012

Memory Stats in the Title Bar (Part I)

#This function puts memory stats for the first instance of Powershell in your title bar.
function Global:Set-title {
$MemStats=(ps powershell)[0] | Select `
@{Name='ThreadCount';Expression={($_.Threads).count}}, `
@{Name='WorkSetMB';Expression={[int](($_.WorkingSet64)/1MB)}}, `
@{Name='VirMemMB';Expression={[int](($_.VirtualMemorySize64)/1MB)}}, `
@{Name='PriMemMB';Expression={[int](($_.PrivateMemorySize64)/1MB)}}, `
@{Name='PagedMemMB';Expression={[int](($_.PagedMemorySize64)/1MB)}}, `
@{Name='NonPagedMemKB';Expression={[int](($_.NonPagedSystemMemorySize64)/1KB)}}
# one line below...
$Title=write "Last_Title_Stats: Time: $([datetime]::now) Version: $((get-host).Version.Major) SessionHours: $([int]([datetime]::now - (ps powershell)[0].Starttime).totalhours) Memory: $($Memstats) GC_MB: $([int]([GC]::gettotalmemory(1)/1MB))"
[console]::set_title($Title)
}

# To see...click on this image to enlarge.:

Monday, August 20, 2012

Clearing up variables and memory in Powershell 3.0

Some GC lines can help you check and clear up memory in Powershell 3.0:

# Find out how much memory is being consumed by your Sesssion:
[System.gc]::gettotalmemory("forcefullcollection") /1MB
# Force a collection of memory by the garbage collector:
[System.gc]::collect()
# Dump all variables not locked by the system:
foreach ($i in (ls variable:/*)) {rv -ea 0 -verbose $i.Name}
#Check memory usage again and force another collection:
[System.gc]::gettotalmemory("forcefullcollection") /1MB
[System.gc]::collect()
#Check Memory once more:
[System.gc]::gettotalmemory("forcefullcollection") /1MB

#You can examine the difference before and after with:
ps powershell* | Select *memory*

or with something more complicated:

ps powershell* | Select *memory* | ft -auto `
@{Name='VirMemMB';Expression={($_.VirtualMemorySize64)/1MB}}, `
@{Name='PriMemMB';Expression={($_.PrivateMemorySize64)/1MB}}

VirMemMB PriMemMB
-------- --------
1548.3671875 432.43359375
603.9765625 58.5234375

Sunday, August 5, 2012

Some notes on querying variables in Powershell 3.0

PS C:\ps1> ls variable:/*

Name Value
---- -----
null
false False
true True
MaximumErrorCount 256
MaximumVariableCount 4096
MaximumFunctionCount 4096
MaximumAliasCount 4096
MaximumDriveCount 4096
Error {Cannot find drive. A drive with the name 'variablwe' does not exist., System.Management.Automation.P...
PSDefaultParameterValues {}
$ variablwe:/*
^ ls
StackTrace at System.Management.Automation.Internal.PipelineProcessor.SynchronousExecuteEnumerate(Object inpu...
ErrorView NormalView
LASTEXITCODE 0
PSEmailServer
....

PS C:\Windows\system32> ((ls variable:/) | gm *).typename | get-unique
System.Management.Automation.PSVariable
System.Management.Automation.QuestionMarkVariable
System.Management.Automation.LocalVariable
System.Management.Automation.SessionStateCapacityVariable
System.Management.Automation.NullVariable

"Big Math"[1] : Solver Foundation and F#

It would be powerful for 'big math' users of Powershell if we could integrate F# and Microsoft's Solver Foundation into Powershell 3.0. Microsoft's Solver Foundation provides for "Programming in the Optimization Modeling Language (OML), in C# imperatively, in F# functionally, or in any .NET language".
So will it work with a DLR language like Powershell 3.0 Beta? I don't know yet but here is some exploratory script.:

"Big Math"[0] : [BigInt] to the rescue!

BigInt to the rescue. Only with .NET 4.0. From PS 3.0 Beta:

Normally:

${2^1023}=[math]::POW(2,1023)
${2^1023}
8.98846567431158E+307
${2^1024}=[math]::POW(2,1024)
${2^1024}
Infinity
${2^1023} * 1.99999999999999988
1.79769313486232E+308
$AMAX=${2^1023} * 1.99999999999999988
[Double]::MaxValue
1.79769313486232E+308
[Double]::MaxValue - $AMAX
0

...and thus integer factorial is limited to 170:

Switch Array Automation

I am working with processing IP Address arrays for some automation I am doing with procmon exported results in Powershell.

#arrays
[array[]]$IParray="RMFHOPE","239.255.255.250"
[array[]]$IPAddress = "239.255.255.250"

# works as expected
$IPAddress = "239.255.255.250"
switch ($IPAddress){
RMFHOPE {write "0"}
239.255.255.250 {write "1"}
}

# now working; required string conversion
[array[]]$IParray="RMFHOPE","239.255.255.250"
[array[]]$IPAddress = "239.255.255.250"
$IPC= ($IParray.count)- 1

0..$IPC | % -process {
[string]$IP=$IPArray[$_] #needed string conversion here
switch ($IPAddress){
($IP) {write $IPAddress}
}
}

# prints IP Address
foreach ($i in (0..$IPC)) {switch ($IPAddress) {$IPArray[$i] {write $IPAddress}}}

# prints array item place
foreach ($i in (0..$IPC)) {switch ($IPAddress) {$IPArray[$i] {$i}}}

#stores and prints all $IPArray then all $IPArray item place
#arrays
[array[]]$IParray="RMFHOPE","239.255.255.250"
[array[]]$IPAddress = "239.255.255.250"
$IPC= ($IParray.count) - 1

#All $IPArray
$IPArray | % -process {
[array[]]$IPAddress = $_
$a=foreach ($i in (0..$IPC)) {switch ($IPAddress) {$IPArray[$i] {$IPArray[$i]}}}
$a
}

#All $IPArray item place
$IPArray | % -process {
[array[]]$IPAddress = $_
$a=foreach ($i in (0..$IPC)) {switch ($IPAddress) {$IPArray[$i] {$i}}}
$a
}

Friday, June 1, 2012

Subtraction worksheet

A somewhat irrelevant post. I had to quickly produce a sheet of 100 random subtraction drills for my 2nd grader. I can think of 10 or 20 ways to make this more compact and useful.:

0..19 | % {write "$(get-random -max 21 -min 14) - $(get-random -max 6 -min 0)"} | out-file mixed100.txt
0..19 | % {write "$(get-random -max 16 -min 9) - $(get-random -max 6 -min 0) = "} | out-file -append mixed100.txt
0..19 | % {write "$(get-random -max 11 -min 6) - $(get-random -max 6 -min 0) = "} | out-file -append mixed100.txt
0..19 | % {write "$(get-random -max 21 -min 14) - $(get-random -max 11 -min 6) = "} | out-file -append mixed100.txt
0..19 | % {write "$(get-random -max 15 -min 9) - $(get-random -max 9 -min 3) = "} | out-file -append mixed100.txt

PS C:\Windows\system32> more mixed100.txt
19 - 3 =
16 - 5 =
19 - 1 =
18 - 5 =
18 - 2 =
17 - 1 =
16 - 0 =
14 - 0 =
...

Monday, May 14, 2012

Creating hashtables with a timer for Key

Powershell 3.0 lets you unroll the elapsed seconds from a current start time in three lines:

$start = [datetime]::Now
function Time {([datetime]::Now - $start)}
function Timer {(Time).TotalSeconds}

while(1) {Timer;sleep 1}

PS C:\Windows\system32> while(1) {Timer;sleep 1}
1.6690955
2.6851536
3.7002116
4.7132696
5.7273276
6.7423856

You can use a new notation to create hash table with the elapsed time as the Key to almost any assigned Value. The notation is [object type]@{Name=Value}. I use the [array] object type below:

PS C:\Windows\system32> while(1) {[array]@{$(Timer)=$(get-random)};sleep 1}

Name Value
---- -----
0.2260129 1128651864
1.2440711 883153077
2.2581291 181563121
3.2711871 609183359
4.2852451 1733733473
5.2993031 1678081962
6.3133611 504757582

(Look Ma! No Value):
PS C:\Windows\system32> while(1) {[array]@{$(Timer)=$()};sleep 1}

Name Value
---- -----
24.6504099
25.6584675
26.6705254
27.7005843
28.7146423
29.7297004
30.7427583
31.7668169

PS C:\Windows\system32> while(1) {[array]@{$(Timer)=$((ps -ea 0 -module).count)};sleep 10}

Name Value
---- -----
43.4584856 3876
53.8810818 3876
64.811707 3876
75.2513041 3876
85.7329036 3876
96.1705006 3876
106.6060975 3876

PS C:\Windows\system32> while(1) {[array]@{$(Timer)=$((Timer)-(Timer))};sleep 1}

Name Value
---- -----
121.6359571 0
122.6440148 0
123.6580728 0
124.6721308 0
125.7001896 -0.000999999999990564
126.7142476 -0.00100000000000477
127.7283056 0
128.7433637 0
129.7574217 -0.00100009999999884
130.7864805 0

Wednesday, May 2, 2012

Automated Hash Generation in Powershell 3.0

After reading Shay Levy's excellent post on casting hashtables from object types in Powershell 3.0, I came up with some automated array/hashtable generation routines:

Function global:gen-num_array_hash
{
[array[]]$num_array=1..1000
$global:num_array_hash=$num_array | % {
[array[]]$rand_out=$(get-random); foreach ($i in $rand_out)` {[array]@{$_.getvalue(0)=$i.item(0)}}};break
}

Function global:gen-alpha_array_hash
{
[array[]]$alpha_array="A","B","C","D","E"
$global:alpha_array_hash=$alpha_array | % {
[array[]]$rand_out=$(get-random); foreach ($i in $rand_out)` {[array]@{$_.getvalue(0)=$i.item(0)}}};break
}

Results look like this:

The demonstration run is:

PS C:>[array[]]$names="Peter","Paul","Mary"

PS C:\> foreach ($i in $names) {[array]@{$i.getvalue(0)=1}}

Name Value

---- -----

Peter 1

Paul 1

Mary 1

PS C:\> foreach ($i in $names) {[hashtable]@{$i.getvalue(0)=1}}

Name Value

---- -----

Peter 1

Paul 1

Mary 1

More notes on how I got there...

On using BinarySearch and IComparer in Powershell:Part I

The Binary Search algorithm is exposed in Powershell v3.0 . It is not clear to me how to implement System.Collections.IComparer as of yet. However, even without IComparer, Powershell's BinarySearch has some usefulness. Here are the overloads:

[Array]::BinarySearch.OverloadDefinitions
static int BinarySearch(array array, System.Object value)
static int BinarySearch(array array, int index, int length, System.Object value)
static int BinarySearch(array array, System.Object value, System.Collections.IComparer comparer)
static int BinarySearch(array array, int index, int length, System.Object value, System.Collections.IComparer comparer)
static int BinarySearch[T](T[] array, T value)
static int BinarySearch[T](T[] array, T value, System.Collections.Generic.IComparer[T] comparer)
static int BinarySearch[T](T[] array, int index, int length, T value)
static int BinarySearch[T](T[] array, int index, int length, T value, System.Collections.Generic.IComparer[T] comparer)

Code Scratch: Querying the Registry with Powershell (Part I of many to come)

Below, some nearly incomprehensible code scratch on querying registry values and subkeys with gci and gp. Querying the registry with Powershell will deserve more narrative than this in the future:

HKLM:\system\CurrentControlSet\Services

$a=Get-ChildItem hklm:\system\CurrentControlSet\Services
$b=Get-ChildItem hklm:\system\CurrentControlSet\Services | ForEach-Object {Get-ItemProperty $_.pspath}
$b| export-csv C:\ps1\CCS_gp.csv
$a.count
$b.count

opens all profile locations

$a=(($profile | get-member -type noteproperty | % {$_.Definition}) -split("="))[1,3,5,7]
foreach ($i in $a) {notepad $i}

Thursday, January 5, 2012

Two methods to query psdrives

Both the methods below will query remote and local drives of all types:

Wednesday, January 4, 2012

Parse TCP and UDP ports from services file

# Parses TCP and UDP ports from services file on Windows 7

$a=gc C:\Windows\System32\drivers\etc\services
$tcp=$a | sls tcp
$udp=$a | sls udp

[array[]]$tcp=0..(($tcp.count) -1) | % { (($tcp.GetValue($_) -split("/"))[0])}
[array[]]$udp=0..(($udp.count) -1) | % { (($udp.GetValue($_) -split("/"))[0])}

[array[]]$tcp_service_ports=0..(($tcp.count) -1) | % { (($tcp.GetValue($_) -split(" "))[-1])}
[array[]]$udp_service_ports=0..(($udp.count) -1) | % { (($udp.GetValue($_) -split(" "))[-1])}

Count the number of binaries in your path

Powershell 3.0 CTP2

Count the number of binaries in your path:

([System.Environment]::GetEnvironmentVariables().Path -split(";") |% { ls $_ *.exe}).count

Saturday, September 3, 2011

Compound variable assignment as [array[]] based storage

Below is a function I have written to demonstrate a function which recursively checks ownership and access for files modified within a given time span. What I call to your attention is the ability to use the assignment operator ('+=') to store data recursively as highlighted in salmon. This is mentioned in 'about_assignment_operators' in the help for Powershell V2:

" When the value of the variable is an array, the += operator appends the
values on the right side of the operator to the array. Unless the array is
explicitly typed by casting, you can append any type of value to the array..."

In the second part of the script below, I pump the results of a foreach loop into an explicitly typed compound assignment variable ("$RecurseList"):