"Get-Winevent" PS scripts to help convert all your Archive-Security*.evtx logs to csv

These (Powershell 3.0) scripts will convert archived Security (auditing) logs. . If you run these at night, configure the advanced settings of your laptop to forbid automatic sleep/hibernation. Note that the field names for an evtx file are different than those you would use to query an existing (working) event log. Of the four scripts I test 'Function Convert-Logs3' has the best combination of optimized resource use and speed.

Finding Whole Squares for a given number range in Powershell

I have never been very good at math.  My daughter, however, is a whiz and I find myself upgrading my skill sets to make sure I stay ahead of her enough to qualify as a mentor(!).  As part of her math team efforts, she needed to find the number of whole squares from 0..1000 inclusive.  This is a pretty simple problem  which can be done on a blackboard, calculator or in Powershell:

[math]::sqrt(1000)
31.6227766016838

There are 31 whole squares in the number range 0..1000.  You can find 999 of the squares from 0..1000 and filter out all the whole squares from that group like as in the truncated result below:

rv -ea 0 a
for ($i=1; $i -lt 1000; $i++) {[array]$a+=$([math]::sqrt($i))}
0..($a.count - 1) | % {$a.item($PSItem)} | ? {([float]$PSItem - [int]$PSItem) -eq 0}
1
2
3
...
29
30
31

With not too much more effort you can create a hashtable of all these squares and their square roots:

foreach ($i in $a) {$c+=[ordered]@{$i=$([math]::pow([double]$i,2));}}


Name                           Value
----                           -----
1                              1
1.4142135623731                2
1.73205080756888               3
2                              4
2.23606797749979               5
2.44948974278318               6
2.64575131106459               7
2.82842712474619               8
3                              9
3.16227766016838               10
...

Name, Version, CreationTime file info

This snippet is simple but I thought it was worth the post:

PS C:\tools\SysinternalsSuite> $name=(ls *.exe)
PS C:\tools\SysinternalsSuite> $FileInfo=foreach ($i in $name) {new-object psobject -property @{name=$i.name; Version=($i).VersionInfo.ProductVersion; CreationTime=$i.C
reationTime} | Select Name,Version,Creationtime}

PS C:\tools\SysinternalsSuite> $FileInfo | ft -auto

name                Version     CreationTime
----                -------     ------------
accesschk.exe       5.11        8/1/2012 1:27:52 PM
AccessEnum.exe      1.32        11/1/2006 1:06:36 PM
ADExplorer.exe      1.44        11/14/2012 10:22:40 AM
ADInsight.exe       1.01        11/20/2007 12:25:34 PM
adrestore.exe                   11/1/2006 1:05:44 PM
Autologon.exe       3.01        2/22/2011 2:18:54 PM
autoruns.exe        11.60       9/10/2012 9:16:28 AM
autorunsc.exe       11.60       9/10/2012 9:16:28 AM
Bginfo.exe          4, 16, 0, 0 9/30/2009 1:31:54 AM
Cacheset.exe                    11/1/2006 1:06:08 PM
Clockres.exe        2.0         6/3/2009 10:36:40 PM
...

Random notes on [System.Collections...]

These are some very random notes on [System.Collections...]  7:07 PM 9/16/2013
Most of us know you can create a hashtable from syntax like this:

$ps=foreach ($i in $(ps)) {@{$i.id=$i.name}}
$ps | gm -s

   TypeName: System.Collections.Hashtable

But did you know you can use the same syntax to create a SortedList?

$ps=foreach ($i in $(ps)){[System.Collections.SortedList]@{$i.id=$i.name}}

$ps | gm -s

   TypeName: System.Collections.SortedList

'SortedList' is one of some number of data collections found in System.Collections. It creates a sorted list of values based on a unique key. Once loaded 'SortedList' has a faster retrieval speed than other members of System.Collections. Here's an example:

rv -ea 0 SortedNames; rv -ea 0 SortedSDDL
measure-command {
$SortedNames = New-Object System.Collections.SortedList
$SortedSDDL = New-Object System.Collections.SortedList
$index=0
foreach ($i in $(ls)) { $indx = $index++; $SortedNames.Add($indx,$i.name); $SortedSDDL.Add($indx,$i.getaccesscontrol().SDDL)  }
}

PS C:\ps1> rv -ea 0 SortedNames; rv -ea 0 SortedSDDL
PS C:\ps1> measure-command {
>> $SortedNames = New-Object System.Collections.SortedList
>> $SortedSDDL = New-Object System.Collections.SortedList
>> $index=0
>> foreach ($i in $(ls)) { $indx = $index++; $SortedNames.Add($indx,$i.name); $SortedSDDL.Add($indx,$i.getaccesscontrol().SDDL)  }
>> }

PS C:\ps1> $SortedNames.count
935

Days              : 0
Hours             : 0
Minutes           : 0
Seconds           : 0
Milliseconds      : 820
Ticks             : 8200868
TotalDays         : 9.49174537037037E-06
TotalHours        : 0.000227801888888889
TotalMinutes      : 0.0136681133333333
TotalSeconds      : 0.8200868
TotalMilliseconds : 820.0868

The following trick I picked up from Powershell.com : http://powershell.com/cs/blogs/tips/archive/2013/09/11/adding-new-type-accelerators-in-powershell.aspx . You can enumerate "Type Accelerators" in Powershell 3.0 with:

[PSObject].Assembly.GetType("System.Management.Automation.TypeAccelerators")::Get |Sort-Object -Property Value 

The property 'ImplementedInterfaces' allows you to view the interfaces for various Collections. Some Collections have more interfaces than others:

$PSAGet=[PSObject].Assembly.GetType("System.Management.Automation.TypeAccelerators")::Get
$PSAhashtable = ($PSAGet).hashtable

($PSAhashtable).ImplementedInterfaces

IsPublic IsSerial Name                                     BaseType
-------- -------- ----                                     --------
True     False    IDictionary
True     False    ICollection
True     False    IEnumerable
True     False    ISerializable
True     False    IDeserializationCallback
True     False    ICloneable

$PSAArray = ($PSAGet).array
PS C:\> ($PSAArray).ImplementedInterfaces

IsPublic IsSerial Name                                     BaseType
-------- -------- ----                                     --------
True     False    ICloneable
True     False    IList
True     False    ICollection
True     False    IEnumerable
True     False    IStructuralComparable
True     False    IStructuralEquatable

Unlike [hashtable] and [array], SortedList, ArrayList, IDictionary, IList will need to be added to Powershell Type Accelerator list in order to be used as [SortedList], [ArrayList], etc.

SortedList Collection and IP Address generation

I spent this morning working with the SortedList Collection and IP Address generation. SortedList maintains an IDictionary interface to a Key/Value pair collection (see  Krivayakov ) The advantage is a simple and direct reference to the last octet for Class C subnet generation and reference: 


rv -ea 0 SN
$SN = new-object System.Collections.SortedList
foreach ($i in (0..254)) {$SN.add($i,[IPAddress]"192.168.0.$i")}

foreach ($i in (0..254)) {$SN.add($i,[IPAddress]"192.168.0.$i")}
PS C:\> $SN

Name Value
---- -----
0 192.168.0.0
1 192.168.0.1
2 192.168.0.2
3 192.168.0.3
...

PS C:\> ($SN[0])

Address : 43200
AddressFamily : InterNetwork
ScopeId :
IsIPv6Multicast : False
IsIPv6LinkLocal : False
IsIPv6SiteLocal : False
IsIPv6Teredo : False
IsIPv4MappedToIPv6 : False
IPAddressToString : 192.168.0.0


This makes collecting arbitrary IP ranges a simple reference to their Name/Key:

PS C:\Powershell> $b = ($SN[0,8,23]).IPAddressToString + ($SN[23..27]).IPAddressToString
PS C:\Powershell> $b
192.168.0.0
192.168.0.8
192.168.0.23
192.168.0.23
192.168.0.24
192.168.0.25
192.168.0.26
192.168.0.27

A little more complicated for multiple subnets:

rv -EA 0 SN0;rv -EA 0 SN1;rv -EA 0 SN2;
$SN0 = new-object System.Collections.SortedList
$SN1 = new-object System.Collections.SortedList
$SN2 = new-object System.Collections.SortedList
for ($i = 0; $i -ile 254;$i++){$SN0.add($i,[IPAddress]"192.168.0.$i")}
for ($i = 0; $i -ile 254;$i++){$SN1.add($i,[IPAddress]"192.168.1.$i")}
for ($i = 0; $i -ile 254;$i++){$SN2.add($i,[IPAddress]"192.168.2.$i")}

$c = ($SN0[0,8,23]).IPAddressToString + ($SN1[23..27]).IPAddressToString + ($SN2[148..154]).IPAddressToString

PS C:\> $c
192.168.0.0
192.168.0.8
192.168.0.23
192.168.1.23
192.168.1.24
192.168.1.25
192.168.1.26
192.168.1.27
192.168.2.148
192.168.2.149
192.168.2.150
192.168.2.151
192.168.2.152
192.168.2.153
192.168.2.154

Processing Snort Logs with Powershell 3.0

I had a group of snort logs (total about 144 MB) I wanted to process in batch. To do this I embedded a a new PSOBJECT inside a foreach loop. 'Select-string' ('sls') isn't the speediest search, but fast enough.

Function Process-SnortLogs {
$SelectPorts = foreach ($i in ($(ls snort.log.*)))
{
$a=.\snort -qr $i.Name;
New-Object PSObject -Property  @{
LogName = $i.Name;
LastWrite = $i.LastWriteTime;
Total = $($a.count);
P445 = (($a | sls -allmatches ':445').matches).count;
P443 = (($a | sls -allmatches ':443').matches).count;
P80 =   (($a | sls -allmatches ':80').matches).count;
P53 =   (($a | sls -allmatches ':53').matches).count;
}
}
}

Process-SnortLogs
$SelectPorts | Select LogName,LastWrite,P53,P80,P443,P445,Total | ft -auto

LogName              LastWrite               P53    P80  P443 P445  Total
-------              ---------               ---    ---  ---- ----  -----
snort.log.1304098553 4/29/2011 10:36:45 AM     0      0     2    0     10
snort.log.1304098783 4/29/2011 10:39:45 AM     0      0     3    0     15
snort.log.1304098850 4/29/2011 10:43:57 AM     0      4    17    0    105
snort.log.1339265058 6/9/2012 11:44:35 AM   2706  10429  7052  110 114395
snort.log.1339278740 6/9/2012 3:10:55 PM    1415    898  1232    7  26019
snort.log.1349466038 10/5/2012 1:57:33 PM   2489   6365  9465   70 149126
snort.log.1349554671 10/6/2012 1:18:00 PM     25      0     0    0    325
snort.log.1349554686 10/6/2012 3:52:18 PM   9820  39956 27617  175 482248
snort.log.1370643770 6/7/2013 5:20:12 PM    2138      8  6482    0  43290
snort.log.1373764041 7/13/2013 7:08:13 PM  30095 101367 26708  239 683162
snort.log.1373770657 7/13/2013 8:01:32 PM     21      0     5    0    545

Here I get the percentage of ports 53,80,443,445 for each snort log:

$SelectPorts | Select LogName, LastWrite, Total,`
 @{Label='%P53';Expression={[LONG]([float]($_.P53 /$_.Total) * 100)}},`
 @{Label='%P80';Expression={[LONG]([float]($_.P80 /$_.Total) * 100)}},`
 @{Label='%P443';Expression={[LONG]([float]($_.P443 /$_.Total) * 100)}},`
 @{Label='%P445';Expression={[LONG]([float]($_.P445 /$_.Total) * 100)}} |  sort -desc Total |  ft * -auto

LogName              LastWrite              Total %P53 %P80 %P443 %P445
-------              ---------              ----- ---- ---- ----- -----
snort.log.1373764041 7/13/2013 7:08:13 PM  683162    4   15     4     0
snort.log.1349554686 10/6/2012 3:52:18 PM  482248    2    8     6     0
snort.log.1349466038 10/5/2012 1:57:33 PM  149126    2    4     6     0
snort.log.1339265058 6/9/2012 11:44:35 AM  114395    2    9     6     0
snort.log.1370643770 6/7/2013 5:20:12 PM    43290    5    0    15     0
snort.log.1339278740 6/9/2012 3:10:55 PM    26019    5    3     5     0
snort.log.1373770657 7/13/2013 8:01:32 PM     545    4    0     1     0
snort.log.1349554671 10/6/2012 1:18:00 PM     325    8    0     0     0
snort.log.1304098850 4/29/2011 10:43:57 AM    105    0    4    16     0
snort.log.1304098783 4/29/2011 10:39:45 AM     15    0    0    20     0
snort.log.1304098553 4/29/2011 10:36:45 AM     10    0    0    20     0

Current Process Memory

updated  08/04/2013 -RMF

Below is an example of the (clumsy) scripting workaround I will use because I always have a difficult time getting 'add-member' to work with membertype ScriptMethod. What I want here is WS, PM, VM for each current process, but also the differences (e.g. VM - PM, PM - WS):

Function CurrentMem {
[PSObject]$Memory=
ps * | Select Name, ID,`
@{Name='WS_MB';Expression={[INT](($_.WorkingSet)/1MB)}},`
@{Name='PriMemMB';Expression={[INT](($_.PrivateMemorySize64)/1MB)}}, `
@{Name='VirMemMB';Expression={[INT](($_.VirtualMemorySize64)/1MB)}}
$Memory | Sort -desc VirMemMB | 
Select *,  @{Name='VM-PM';Expression={[INT]($_.VirMemMB - $_.PriMemMB)}}, `
@{Name='PM-WS';Expression={[INT]($_.PriMemMB - $_.WS_MB)}}
}

So then  I have:

PS C:\> $CurrentMem=CurrentMem
PS C:\> ($CurrentMem | sort -desc VM-PM)[0..10]  | ft -auto

Name         Id WS_MB PriMemMB VirMemMB VM-PM PM-WS
----         -- ----- -------- -------- ----- -----
powershell 7300    74       66      608   542    -8
powershell 5408     3       63      603   540    60
svchost     536    39       39      575   536     0
chrome     4748   192      194      658   464     2
bash       6348     0        8      451   443     8
powershell 1536   176      185      620   435     9
syslogd    2112     1        5      432   427     4
cygrunsrv  1948     0        6      427   421     6
chrome     2956   265      323      742   419    58
taskhost   3896     6       15      394   379     9
svchost    1288    23       26      386   360     3

A more advanced version allows for a more granular look at processes:


Function Get-CM {
[PSObject]$Memory=
ps * | Select Name, ID, Company, HandleCount,`
@{Name='WorkSetMB';Expression={[LONG](($_.WorkingSet)/1MB)}},`
@{Name='PMS64Bytes';Expression={[LONG](($_.PagedMemorySize64))}},`
@{Name='NPMS64Bytes';Expression={[LONG](($_.NonpagedSystemMemorySize64))}},`
@{Name='PriMem64MB';Expression={[LONG](($_.PrivateMemorySize64)/1MB)}},`
@{Name='VirMem64MB';Expression={[LONG](($_.VirtualMemorySize64)/1MB)}}
$Memory | Sort -desc VirMem64MB | 
Select *,  @{Name='VMem-PMem.MB';Expression={[LONG]($_.VirMem64MB - $_.PriMem64MB)}},`
@{Name='PMem-NPMem.MB';Expression={[LONG](($_.PMS64Bytes - $_.NPMS64Bytes)/1MB)}},`
@{Name='PriMem-WorkSet.MB';Expression={[LONG]($_.PriMem64MB - $_.WorkSetMB)}}
}

So that here I can look at all process with over 500 handles sorted by a decreasing amount of PagedMemorySystem64 :

PS C:\ps1> get-cm | where HandleCount -gt 500 | sort -desc PMS64Bytes | fl *


Name              : chrome
Id                : 6636
Company           : The Chromium Authors
HandleCount       : 2769
WorkSetMB         : 180
PMS64Bytes        : 268677120
NPMS64Bytes       : 141712
PriMem64MB        : 256
VirMem64MB        : 672
VMem-PMem.MB      : 416
PMem-NPMem.MB     : 256
PriMem-WorkSet.MB : 76

Name              : chrome
Id                : 3532
Company           : The Chromium Authors
HandleCount       : 796
WorkSetMB         : 63
PMS64Bytes        : 220286976
NPMS64Bytes       : 69632
PriMem64MB        : 210
VirMem64MB        : 699
VMem-PMem.MB      : 489
PMem-NPMem.MB     : 210
PriMem-WorkSet.MB : 147

Name              : svchost
Id                : 456
Company           : Microsoft Corporation
HandleCount       : 727
WorkSetMB         : 137
PMS64Bytes        : 162627584
NPMS64Bytes       : 31784
PriMem64MB        : 155
VirMem64MB        : 315
VMem-PMem.MB      : 160
PMem-NPMem.MB     : 155
PriMem-WorkSet.MB : 18

...

or (click to enlarge)

Powershell 3.0 CIM snippet : Properties of IPEnabled Adapters

$count = ((Get-CimInstance -class Win32_NetworkAdapterConfiguration).count - 1)

$Adapters = 0..$count | % {(Get-CimInstance -class Win32_NetworkAdapterConfiguration)[$PSItem]}
$Adapters
$IPEnabled=($Adapters | ? {$_.IPEnabled -eq "True"})
$IPEnabled | fl *

Functions for sum of directory or file size

# file
function file.sum($x) {
rv -ea 0 total;
foreach ($i in ((ls $x -Force) | Select Length)) {[int64]$total+=$i[0].Length};
$total /1MB;
}

#dir
function dir.sum {
rv -ea 0 total;
foreach ($i in ((ls -Force) | Select Length)) {[int64]$total+=$i[0].Length};
$total /1GB;
}

#dir recurse
# really chews through memory for directories of any size
function dir.sum.recurse {
rv -ea 0 total;
foreach ($i in ((ls -Force -Recurse) | Select Length)) {[int64]$total+=$i[0].Length};
$total /1GB;
}

Can you see the pattern....?

Function global:gen-num_array_hash
{
[array[]]$num_array=1..10000
$global:num_array_hash=$num_array | %  {
[array[]]$rand_out=$(get-random); foreach ($i in $rand_out) {
[array]@{$_.getvalue(0)=$i.item(0)}}};break
}
rv -ea 0 hashdata
$hashdata=$num_array_hash
Chart-Hashdata fastpoint

[math]::IEEERemainder((100/$PSItem),($PSItem/100))

Tonight's intellectual curiosity produces the charts below.

function XYZ {
-100..100 | % {
try {foreach ($i in [array[]]([math]::IEEERemainder((100/$PSItem),($PSItem/100)))) {[ordered]@{$PSItem=$($i.item(0))}}} `
catch [System.Management.Automation.RuntimeException] {}}
}

A defect for 'get-random'

In PS 3.0, 'Get-random' doesn't return an error value for a $null variable without the use of  '-inputobject'.  I came across this writing a  function so my third grader could pick up her arithmetic skills. The function has '$Mixed' as such:

[array]$Mixed= % {
0..($questions_per_worksheet - 1) | % {write "2 $operator1 $(get-random -max $random_max -min $random_min) = "} 
0..($questions_per_worksheet - 1) | % {write "4 $operator2 $(get-random -max $random_max -min $random_min) = "} 
0..($questions_per_worksheet - 1) | % {write "5 $operator3 $(get-random -max $random_max -min $random_min) = "} 
0..($questions_per_worksheet - 1) | % {write "10 $operator4 $(get-random -max $random_max -min $random_min) = "} 
} 

Data Analytics with Powershell Part I

Below is some cruft I am using to analyze PDC data for candidates in my local WA elections. This is part of a larger project I am working on to use Powershell 3.0 as a data analytic solution.  I find working with data in Powershell 3.0 somewhat tricky,  sometimes limited, but also no less straightforward than SQL or R.  Although, Powershell can sometimes be a little frustrating to work with , I rather like being able to craft my own 'data analytic'  solution from the console. I found myself close to the data while working with PS 3.0. Certain techniques produced surprisingly rapid and illuminating results. Take the query below. After having imported to a variable ('$NM') a candidates data from CSV, in one line of code I am able to exclude all WA state contributions; then use 'group-object' to list all 'out of state' contributors,state,amounts in a sort.


$NM | Where State -ne WA | group -property Contributor,State,Amount -noelement | sort -desc Count,Name | ft -auto -wrap

Count Name
----- ----
    1 WEBB LISA, MT, 100
    1 VASKAS JANET, PA, 100
    1 VASKAS ALAN, PA, 100
    1 TURNER ZACHARY M, CO, 900
    1 TERESA JUDITH, WV, 50
    1 SOTO JLEANA, CA, 100
    1 MCCLENDON SUSAN, GA, 500
    1 ATU, DC, 900

Memory Stats in the Title Bar (Part II)

Two different attempts to update memory stats in the title bar in the console background. The 'start-job' script won't work because it updates the new Powershell instance the job is running in. The 'runspace' script works because it updates the existing console.

# with Start-Job
# Doesn't Work because it updates a hidden Powershell Job window
start-job -InitializationScript {

function Global:Set-title {
$PSID=([System.Diagnostics.Process]::GetCurrentProcess()).Id
$MemStats=ps -id $PSID | Select `
@{Name='ThreadCount';Expression={($_.Threads).count}}, `
@{Name='WorkSetMB';Expression={[int](($_.WorkingSet64)/1MB)}}, `
@{Name='VirMemMB';Expression={[int](($_.VirtualMemorySize64)/1MB)}}, `
@{Name='PriMemMB';Expression={[int](($_.PrivateMemorySize64)/1MB)}}, `
@{Name='PagedMemMB';Expression={[int](($_.PagedMemorySize64)/1MB)}}, `
@{Name='NonPagedMemKB';Expression={[int](($_.NonPagedSystemMemorySize64)/1KB)}}

$Title=write "Last_Title_Stats: Time: $([datetime]::now) Version: $((get-host).Version.Major) SessionHours: $([int]([datetime]::now - (ps -id $PSID).Starttime).totalhours) Memory: $($Memstats) GC_MB: $([int]([GC]::gettotalmemory(1)/1MB))"
[console]::set_title($Title)
}

} `

-scriptblock {while(1) {set-title;sleep -s 5}}


# With Runspace
# Works because the new runspace refers to the existing Powershell window
$set_title=

{

function Global:Set-title {
$PSID=([System.Diagnostics.Process]::GetCurrentProcess()).Id
$MemStats=ps -id $PSID | Select `
@{Name='ThreadCount';Expression={($_.Threads).count}}, `
@{Name='WorkSetMB';Expression={[int](($_.WorkingSet64)/1MB)}}, `
@{Name='VirMemMB';Expression={[int](($_.VirtualMemorySize64)/1MB)}}, `
@{Name='PriMemMB';Expression={[int](($_.PrivateMemorySize64)/1MB)}}, `
@{Name='PagedMemMB';Expression={[int](($_.PagedMemorySize64)/1MB)}}, `
@{Name='NonPagedMemKB';Expression={[int](($_.NonPagedSystemMemorySize64)/1KB)}}

$Title=write "Last_Title_Stats: Time: $([datetime]::now) Version: $((get-host).Version.Major) SessionHours: $([int]([datetime]::now - (ps -id $psid).Starttime).totalhours) Memory: $($Memstats) GC_MB: $([int]([GC]::gettotalmemory(1)/1MB))"
[console]::set_title($Title)
}

while(1) {set-title;sleep -s 5}

}

$ST_Runspace = [PowerShell]::Create().AddScript($set_title)
$Begin_Set_Title = $ST_Runspace.BeginInvoke()

# Commands to query,stop,dispose the titlebar update:
# $ST_Runspace.runspace
# $Begin_Set_Title
# $Stop_Set_Title = $ST_Runspace.Stop()
# $Dispose_Set_Title = $ST_Runspace.Dispose()

Memory Stats in the Title Bar (Part I)

#This function puts memory stats for the first instance of Powershell in your title bar.
function Global:Set-title {
$MemStats=(ps powershell)[0] | Select `
@{Name='ThreadCount';Expression={($_.Threads).count}}, `
@{Name='WorkSetMB';Expression={[int](($_.WorkingSet64)/1MB)}}, `
@{Name='VirMemMB';Expression={[int](($_.VirtualMemorySize64)/1MB)}}, `
@{Name='PriMemMB';Expression={[int](($_.PrivateMemorySize64)/1MB)}}, `
@{Name='PagedMemMB';Expression={[int](($_.PagedMemorySize64)/1MB)}}, `
@{Name='NonPagedMemKB';Expression={[int](($_.NonPagedSystemMemorySize64)/1KB)}}
# one line below...
$Title=write "Last_Title_Stats: Time: $([datetime]::now) Version: $((get-host).Version.Major) SessionHours: $([int]([datetime]::now - (ps powershell)[0].Starttime).totalhours) Memory: $($Memstats) GC_MB: $([int]([GC]::gettotalmemory(1)/1MB))"
[console]::set_title($Title)
}

# To see...click on this image to enlarge.:

Clearing up variables and memory in Powershell 3.0

Some GC lines can help you check and clear up memory  in Powershell 3.0:

# Find out how much memory is being consumed by your Sesssion:
[System.gc]::gettotalmemory("forcefullcollection") /1MB
# Force a collection of memory by the garbage collector:
[System.gc]::collect()
# Dump all variables not locked by the system:
foreach ($i in (ls variable:/*)) {rv -ea 0 -verbose $i.Name}
#Check memory usage again and force another collection:
[System.gc]::gettotalmemory("forcefullcollection") /1MB
[System.gc]::collect()
#Check Memory once more:
[System.gc]::gettotalmemory("forcefullcollection") /1MB

#You can examine the difference before and after with:
ps powershell* | Select *memory*

or with something more complicated:

ps powershell* | Select *memory* | ft -auto `
@{Name='VirMemMB';Expression={($_.VirtualMemorySize64)/1MB}}, `
@{Name='PriMemMB';Expression={($_.PrivateMemorySize64)/1MB}}

    VirMemMB     PriMemMB
    --------     --------
1548.3671875 432.43359375
 603.9765625   58.5234375

Some notes on querying variables in Powershell 3.0

PS C:\ps1> ls variable:/*


Name                           Value
----                           -----
null
false                          False
true                           True
MaximumErrorCount              256
MaximumVariableCount           4096
MaximumFunctionCount           4096
MaximumAliasCount              4096
MaximumDriveCount              4096
Error                          {Cannot find drive. A drive with the name 'variablwe' does not exist., System.Management.Automation.P...
PSDefaultParameterValues       {}
$                              variablwe:/*
^                              ls
StackTrace                        at System.Management.Automation.Internal.PipelineProcessor.SynchronousExecuteEnumerate(Object inpu...
ErrorView                      NormalView
LASTEXITCODE                   0
PSEmailServer
....


PS C:\Windows\system32> ((ls variable:/) | gm *).typename | get-unique
System.Management.Automation.PSVariable
System.Management.Automation.QuestionMarkVariable
System.Management.Automation.LocalVariable
System.Management.Automation.SessionStateCapacityVariable
System.Management.Automation.NullVariable

"Big Math"[1] : Solver Foundation and F#

It would be powerful for 'big math' users of Powershell if we could integrate F# and Microsoft's Solver Foundation into Powershell 3.0. Microsoft's Solver Foundation provides for "Programming in the Optimization Modeling Language (OML), in C# imperatively, in F# functionally, or in any .NET language".  
So will it work with a DLR language like Powershell 3.0 Beta?  I don't know yet but here is some exploratory script.:

"Big Math"[0] : [BigInt] to the rescue!

 BigInt to the  rescue.  Only with .NET 4.0. From PS 3.0 Beta:


Normally:


${2^1023}=[math]::POW(2,1023)
${2^1023}
8.98846567431158E+307
${2^1024}=[math]::POW(2,1024)
${2^1024}
Infinity
${2^1023} * 1.99999999999999988
1.79769313486232E+308
$AMAX=${2^1023} * 1.99999999999999988
[Double]::MaxValue
1.79769313486232E+308
[Double]::MaxValue - $AMAX
0


...and thus integer factorial is limited to 170:

Switch Array Automation

I am working with processing IP Address arrays for some automation I am doing with procmon exported results in Powershell.

#arrays
[array[]]$IParray="RMFHOPE","239.255.255.250"
[array[]]$IPAddress = "239.255.255.250"

# works as expected
$IPAddress = "239.255.255.250"
switch ($IPAddress){
RMFHOPE  {write "0"}
239.255.255.250 {write "1"}
}

# now working; required string conversion
[array[]]$IParray="RMFHOPE","239.255.255.250"
[array[]]$IPAddress = "239.255.255.250"
$IPC= ($IParray.count)- 1

0..$IPC | % -process {
[string]$IP=$IPArray[$_] #needed string conversion here
switch ($IPAddress){
($IP) {write $IPAddress}
}
}

# prints IP Address
foreach ($i in (0..$IPC)) {switch ($IPAddress) {$IPArray[$i] {write $IPAddress}}}

# prints array item place
foreach ($i in (0..$IPC)) {switch ($IPAddress) {$IPArray[$i] {$i}}}

#stores and prints all $IPArray then all $IPArray item place
#arrays
[array[]]$IParray="RMFHOPE","239.255.255.250"
[array[]]$IPAddress = "239.255.255.250"
$IPC= ($IParray.count) - 1

#All $IPArray
$IPArray | % -process {
[array[]]$IPAddress = $_
$a=foreach ($i in (0..$IPC)) {switch ($IPAddress) {$IPArray[$i] {$IPArray[$i]}}}
$a
}

#All $IPArray item place
$IPArray | % -process {
[array[]]$IPAddress = $_
$a=foreach ($i in (0..$IPC)) {switch ($IPAddress) {$IPArray[$i] {$i}}}
$a
}