Weevil

• What is Weevil?
• Key Features
  • Filtering
  • Navigation
  • Analysis
  • Extensible Architecture
• Software Development
  • WPF Application
  • NuGet Packages
• Development
  • Guidelines
  • Compiling
  • Verification
• Recognition
  • Open Source Projects
  • Contributors

What is Weevil?

Weevil is an open-source .NET project that is used by analysts to extract valuable insights from log files. It's all about "boring log files for tasty bytes".

A complete list of features can be found in the release notes.

Key Features

1. File and Record Level Notes
   • Capture high-level observations as remarks, or low-level details as record comments.
2. Persisted State
   • Automatically load filter history, record comments, and file level comments when opening a log file.
   • Share the application's state as an XML sidecar with colleagues.
3. Non-Destructive Operations
   • The Weevil application ensures that the original log file is never modified.
4. Simplified Callstacks
   • When a record includes an exception call stack, Weevil simplifies the call stack by only displaying business logic references.
5. Clear Operations
   • This operation removes records from memory, thus reducing the RAM footprint and speeding up the filtering process.

Filtering

One or more filter criteria can be used to show or hide log file records.

1. Inclusive and Exclusive Filters
   • Display records matching the inclusive filter while hiding those matching the exclusive filter.
2. Filter Criteria
   1. Plain Text
   2. Regular Expressions
   3. Aliases
      • Frequently used or complex filters can be assigned a unique key that can be used to speed up the filtering process.
      • For example, the #IpAddress key could be assigned to the following filter criteria ^(?:[0-9]{1,3}\.){3}[0-9]{1,3}$.
   4. Monikers
      • Monikers are built-in keys that can be used to query metadata collected by Weevil.
        • For example, the @Comment can be used to identify records that have a user comment.
3. Multiple Criteria
   • Multiple filter criteria can be combined together using a logical "OR" operator (||).
4. Pinned Records
   • Pinned records are guaranteed to be included in the filter results.

Navigation

1. Find
   • Search for text within filtered results.
2. Go To
   • Jump to specific line numbers or timestamps.
3. Pinned Records
   • Effortlessly navigate between important records.
4. Flagged Records
   • Move between records flagged during prior analysis.
5. Record Comments
   • Navigate between records containing user comments.

Analysis

Utilize Regular expression named groups to identify key data in log files. Leverage Weevil's analysis tools to then extract data and identify trends.

Each analysis tool updates the Comments fields with the values that match the provided named group(s), and the recor's Flagged field is set.

1. Detect Data
   • For example: extracting URLs from a log file
2. Detect Data Transitions
   • For example: when a hardware serial number changes
3. Detect Rising Edges
   • For example: detecting peek CPU usage
4. Detect Falling Edges
   • For example: detect when a firmware's uptime has reset
5. Detect Temporal Anomalies
   • For example: detect when records are logged out of order

Furthermore, Weevil includes the ability to generated graphs based on the extracted data.

Extensible Architecture

Maximize potential by developing domain-specific extensions tailored to your business' needs. Weevil can be enhanced by custom plugins:

1. Log File Parsers
   • Create tailored parsers to accurately interpret log files from various sources and formats, ensuring seamless integration with Weevil.
2. Log File Analyzers
   • Design specialized analyzers to process and extract valuable insights from the parsed log data, optimizing the analysis for your specific business domain.
3. Dashboard Insights
   • Develop custom dashboard visualizations and insights that highlight the most relevant information, enabling efficient decision-making and improved understanding of your log data.

Software Development

WPF Application

• Installation Guide
• Help Manual

NuGet Packages

Latest Release	NuGet Package
	BlueDotBrigade.Weevil.Common.nupkg
	BlueDotBrigade.Weevil.Core.nupkg
	BlueDotBrigade.Weevil.Windows.nupkg

A .NET application can use Weevil's feature set by directly referencing the BlueDotBrigade.Weevil.Core NuGet package.

For example, one could determine when equipment was changed using the following sample code:

var engine = Engine
   .UsingPath(@"C:\Temp\hardware.log")
   .Open();

// The `UniqueId` regular expression named group is used to
// capture serial hardware serial numbers.
engine.Filter.Apply(
   FilterType.RegularExpression,
   new FilterCriteria(@"Received hardware message. ID=(?<UniqueId>[a-zA-Z0-9]+)"));

// This type of analysis compares the captured serial numbers,
// and flags the record when a value changes.
engine.Analyzer.Analyze(AnalysisType.DetectDataTransition);

foreach (var record in engine.Filter.Results.Where(r => r.Metadata.IsFlagged == true))
{
   Console.WriteLine(
   $"{record.CreatedAt} {record.Metadata.Comment}");
}

Development

Attribute	Description
	The list of features & bug fixes for the latest Weevil release.
	Source code for the most stable version of Weevil.
	The most up-to-date source code. This branch includes features that are still under development.
	A value of passing indicates that the main branch is compiling & that the automated tests have passed.
	Total size of Weevil's Git repository.
	Total number of lines of code in the Git repository.
	Indicates when the Git repository was last updated.

Guidelines

• When working on the WPF application, please be sure to follow the Style Guide for the user interface.

Compiling

The following steps outline how to build Weevil's WPF application:

1. Download the latest stable release source code.
2. If you have implemented a custom Weevil plugin:
   • Prior to starting Visual Studio, create the following Windows [environment variable][EnvironmentVariable]:
     • %WEEVIL_PLUGINS_PATH% which refers to the directory where the Weevil plugin assembly (*.dll) can be found.
3. Using Visual Studio, compile the WPF project: BlueDotBrigade.Weevil.Gui [EnvironmentVariable]: https://en.wikipedia.org/wiki/Environment_variable#Windows

Verification

Software integrity is verified through a number of automated tests which can be found in the /Weevil/Tst/ directory:

• UnitTests
• FunctionalTests

Recognition

• PostSharp
  • PostSharp`s aspect oriented library helps to simplify a code base by reducing boilerplate. Special thanks to the PostSharp team for donating a license.
• GitHub
  • Free Git repository hosting platform for this project & many others like it.

Open Source Projects

• Live Charts
  • Beto Rodriguez et al. have developed an impressive WPF charting library. Am looking forward to future releases.
• Material Design in XAML
  • An excellent WPF library that helps to standardize themes & improve the overall quality of an application's user interface.
• Cocona
  • Mayuki Sawatari et al. have created an interesting library for creating .NET Core command-line applications.

Contributors

A special thanks to all of those who have contributed to this project.