Horizontal Logic: Parsing Event logs

Showing posts with label Parsing Event logs. Show all posts

Tuesday, January 27, 2009

This is my third update to this script. I think this script will finally push me into "programming like a real man" (e.g. with functions, params, hash arrays, throws, traps other optimizations) e.g. http://rmfdevelopment.com/PowerShell_Scripts/QueryEventFunction.ps1

Interestingly, I found System Event ID 36 breaks my script. It is a failure of the Windows Time Service and I think I have found a defect in Time/Date formatting:

## Takes Event Log queries...and finds elapsed time from event

# Default queries localhost system shutdown (EventID 6009)

# E.G. .\EventLogQueries.ps1 System 4072

# E.G . 6005..6009 | %{.\EventLogQueries.ps1 System $_ }

if ($args[0] -eq $Null) {$Log_Type = "System"} else {$Log_Type = $args[0]}

if ($args[1] -eq $Null) {$Event_ID = 6009} else {$Event_ID = $args[1]}

write $args[1]

## TODO for remote and other properties:

## if ($args[3] = $Null) {$Computer = localhost} else {$args[3] = $Computer}

# query Event Log

$EventLog = get-eventlog -log $Log_Type | Select Message,EventID,TimeGenerated

$Event = $EventLog | ?{$_.eventID -eq $Event_ID}

$EventID = $Event | %{$_.eventID}

$Message = $Event | %{$_.Message}

$TimeGenerated = $Event | %{$_.TimeGenerated}

## TODO: Needs Trap or Throw for bad date or time format from Microsoft like for Event ID 36

## EventID 36 will break this script because....(??)

## if EventID is null, discard query

if ($LogType = "System" -and $EventID -eq 36) {$EventID = $NULL;write "Skip System EventID 36 because it breaks this script"}

if ($EventID -ne $NULL)

{

# Find elapsed time, total restarts, restarts/days and generate some arrays

## DBG::$test_EventID = $EventID[0]

## DBG::$test_Args_1 = $Args[1]

## DBG::write "Args[1]:$test_Args_1 -- Date/Time -- Elapsed Time (D.H.M.S)"

write "EventID -- Message -- Date/Time -- Elapsed Time (D.H.M.S)"

$array_count = ($Event).count - 1

$total_events = ($Event).count

$curr_date = get-date

$first_event_date = $TimeGenerated[$array_count]

$last_event_date = $TimeGenerated[0]

$event_time_span =($curr_date - $first_event_date)

$elapsed = $TimeGenerated[0..$array_count] | %{($curr_date - $_)}

$AverageDaysBetweenEvents = $event_time_span.Days%$total_events

## Report Data

## What happens if data field is null?

0..$array_count | %{

$days = $elapsed[$_].days;

$hours = $elapsed[$_].hours;

$minutes = $elapsed[$_].minutes;

$seconds = $elapsed[$_].seconds;

$EventIDPrint = $EventID[$_];

$MessagePrint = $Message[$_];

$TimeGeneratedPrint = $TimeGenerated[$_];

write "$EventIDPrint,$MessagePrint,$TimeGeneratedPrint,$days.$hours.$minutes.$seconds" }

}

if ($EventID -ne $NULL)

{

write "Number of Events:$total_events First Occurrence:$first_event_date Last Occurrence:$last_event_date Average Days Between Events:$AverageDaysBetweenEvents"

}

Friday, January 23, 2009

# Takes Event Log queries...and finds elapsed time from event

# E.G . 6005..6009 | %{.\EventLogQueries.ps1 System $_ }

if ($args[0] -eq $Null) {$Log_Type = "System"} else {$Log_Type = $args[0]}

if ($args[1] -eq $Null) {$Event_ID = 6009} else {$Event_ID = $args[1]}

# query Event Log

$EventLog = get-eventlog -log $Log_Type

$EventID = $EventLog | ?{$_.eventID -eq $Event_ID}

# If EventID is null, discard query

if ($EventID -ne $NULL)

{

# Find Elapsed Time and Generate Array

write "Event ID -- Date/Time -- Elapsed Time (D.H.M.S)"

$LogType_EventID_MsgProperty = $EventID | %{$_.TimeGenerated}

$count = ($LogType_EventID_MsgProperty).count - 1

$curr_date = get-date

$array = $LogType_EventID_MsgProperty[0..$count] | %{($curr_date - $_)}

# Report Data

0..$count | %{

$days = $array[$_].days;

$hours = $array[$_].hours;

$minutes = $array[$_].minutes;

$seconds = $array[$_].seconds;

$date = $LogType_EventID_MsgProperty[$_];

write "$Event_ID -- $date -- $days.$hours.$minutes.$seconds";}

}

Output:

PS >6005..6009 | %{.\EventLogQueries.ps1 System $_ }

Event ID -- Date/Time -- Elapsed Time (D.H.M.S)

6005 -- 01/15/2009 22:40:17 -- 7.15.0.2

6005 -- 01/09/2009 09:52:51 -- 14.3.47.28

6005 -- 01/07/2009 17:39:39 -- 15.20.0.40

6005 -- 01/05/2009 18:30:06 -- 17.19.10.13

6005 -- 01/05/2009 11:01:59 -- 18.2.38.20

6005 -- 12/24/2008 11:20:21 -- 30.2.19.58

6005 -- 12/21/2008 10:01:15 -- 33.3.39.4

6005 -- 12/19/2008 09:23:52 -- 35.4.16.27

6005 -- 12/11/2008 08:04:59 -- 43.5.35.20

6005 -- 12/03/2008 08:10:23 -- 51.5.29.56

Event ID -- Date/Time -- Elapsed Time (D.H.M.S)

6006 -- 01/15/2009 09:28:14 -- 8.4.12.6

6006 -- 01/09/2009 09:52:04 -- 14.3.48.16

6006 -- 01/07/2009 17:38:05 -- 15.20.2.15

6006 -- 01/05/2009 11:00:36 -- 18.2.39.44

6006 -- 12/19/2008 09:22:42 -- 35.4.17.38

6006 -- 12/11/2008 08:03:46 -- 43.5.36.34

Event ID -- Date/Time -- Elapsed Time (D.H.M.S)

6009 -- 01/15/2009 22:40:17 -- 7.15.0.5

6009 -- 01/09/2009 09:52:51 -- 14.3.47.31

6009 -- 01/07/2009 17:39:39 -- 15.20.0.43

6009 -- 01/05/2009 18:30:06 -- 17.19.10.16

6009 -- 01/05/2009 11:01:59 -- 18.2.38.23

6009 -- 12/24/2008 11:20:21 -- 30.2.20.1

6009 -- 12/21/2008 10:01:15 -- 33.3.39.7

6009 -- 12/19/2008 09:23:52 -- 35.4.16.30

6009 -- 12/11/2008 08:04:59 -- 43.5.35.23

6009 -- 12/03/2008 08:10:23 -- 51.5.29.59

Monday, January 12, 2009

Parsing Logs again....Following up from a few posts ago. I would like to figure out how to get multiple log types working together with the simplest syntax in Powershell. I think the goal would be to take events that happen from whatever handlers (Event Logs, Application Verifier, Windbg, NetMonitor, Syslogd, Firewall, IDS) and parse them into "congruent datetime stamped events" as objects (??). To get today's dump from Windows Firewall Audit (set this up in auditing...) messaging out of my Security Event Log, I do something like this:

$now = [System.DateTime]::get_now()

$NowSDS = $now.ToShortDateString()

$SEL = get-eventlog -logname Security | where-object {($_.timegenerated -match "$NowSDS") -and ($_.message -match "Windows Firewall")} | fl *

$SEL | out-file $pwd\SEL.txt

Select-string SEL.txt -pattern "Process Identifier","Path","Port number" -allmatches

## or

Select-string SEL.txt -pattern "Process Identifier","Path","Port number" -allmatches | out-file SEL_SR.txt

$sr = [System.IO.StreamReader]("$pwd\SEL_SR.txt")

$sr.readToEnd()

Results are like:

...

SEL.txt:11669: Path: C:\WINDOWS\system32\svchost.exe

SEL.txt:11671: Process identifier: 1588

SEL.txt:11685: Port number: 50386

SEL.txt:11712: Path: C:\WINDOWS\system32\svchost.exe

SEL.txt:11714: Process identifier: 1588

SEL.txt:11728: Port number: 59453

...

Ungainly. Not parsing an object in the end but eventually text! So here we are parsing the Message descriptions from the Security Event log MSG field (which is text!) into an object:

$now = [System.DateTime]::get_now()

$NowSDS = $now.ToShortDateString()

$SEL = get-eventlog -logname Security | where-object {($_.timegenerated -match "$NowSDS") -and ($_.message -match "Windows Firewall")}

$SEL_MSG = $SEL | %{$_.message}

Select-string -inputobject $SEL_MSG -pattern "Process Identifier","Path","Port number" -allmatches

Okay, an object but not what I want yet....And Select-String isn't helping any here:

Name: -

Path: C:\WINDOWS\system32\svchost.exe

Process identifier: 1588

User account: NETWORK SERVICE

User domain: NT AUTHORITY

Service: Yes

RPC server: No

IP version: IPv4

IP protocol: UDP

Port number: 55033

Allowed: No

User notified: No The Windows Firewall has detected an application listening for incoming traffic.

## This doesn't work

## $SR = [System.IO.StreamReader]($SEL)

## $sr.readToEnd()

to be continued....

Wednesday, September 10, 2008

Creates a columnar listing of Ports to which the Windows Firewall has denied access. No doubt there is a simpler way....This uses Lee Holmes 'Convert-TextObject.ps1' from the "The Windows Powershell Cookbook". To get around parsing the message fields in the Event Log which aren't objects, I used findstr.exe with "MessageFilters.txt" as far below.

$now = [System.DateTime]::get_now()

$nowshort = ($now.ToShortDateString()).ToString()

$TodaysFA = ( ( get-eventlog -logname security | where {$_.EntryType -eq "FailureAudit" -and $_.TimeGenerated -match "$nowshort" } )| Select TimeGenerated,Message )

$TodaysFA_Delimited = ($TodaysFA | fl * | findstr /g:MessageFilters.txt) | .\Convert-TextObject.ps1 -Delimiter ":"

$TodaysFA_Ports = $TodaysFA_Delimited | where-object {$_.Property1 -match "Port"} | sort-object {$_.Property2}

$TodaysFA_PortNumber = $TodaysFA_Ports | Select {$_.Property2}

MessageFilters.txt

TimeGenerated :

Message:

Process identifier:

User account:

User domain:

Service:

RPC server:

IP version:

IP protocol:

Port number:

Allowed:

User notified: