logChecker

The idea was born because of the need for a simple tool that could do pre-post check analysis automatically.

Table of Contents

• Setup
  • System Libraries
  • Installation
• Usage
  • Configuration Options
  • Templates
  • Optional Features
    • Filtering Variables of Templates
    • Using plugins
• Results
  • Tabulation of Logs - Assessment
  • Comparision of Logs - Pre/Post Check

Setup

System Libraries

These libraries have been tested under Ubuntu 20.04 and 22.04, Windows10, Python 3.8~3.10.

Installation

Make sure that you have Python and PIP installed.

Note

For Windows users, select "Add Python to PATH" during the installation.

For Ubuntu users:

pip3 install logChecker

For Windows users:

py -m pip install logChecker

The source code is hosted on GitHub at https://github.com/laimaretto/logchecker

Installer for the latest released version available at Python Package Index (PyPI) - LogChecker

Go to Table of Contents

Usage

LogChecker reads the logs stored in folders. These logs are generally obtained by taskAutom. Though not mandatory, taskAutom is suggested as a way of obtaining the logs, because these will be stored in a json file automatically.

LogChecker organizes the outputs from several show commands on different tabs in an Excel file, helping the verification of information. The data parsing is performed using templates.

There are two ways of execution: assesment or pre-post comparison.

Configuration Options

LogChecker can be configured through CLI as shown below. It is also possible to run logchecker -h directly to show this information.

Parameter	Description
-h	Show all the parameters and exit.
-pre	Folder with Pre logs. Must end in "/"
-post	Folder with Post logs. Must end in "/"
-json	Logs in json format: yes or no. Default = yes.
-tf	Folder where templates reside. Used both for Pre and Post logs. Default = Templates/
-tf-post	If set, use this folder of templates for Post logs
-te	Engine for parsing. Default = textFSM
-ri	Router ID to be used within the tables in Excel report: name, ip or both. Default = name
-ga	Generate ATP document in .docx format, based on contents of json files from taskAutom. Default = no
-ic	Adds new column (Idx Pre/Post) in changes detected table, when running comparision. Default = no
-ug	Using generic template. If -ug=no, logChecker only use the templates indicated in the -tf folder (and `-tf-post, if applicable). Default = yes
-up	Additional plugins for manipulation of parsed information, creating new sheets. One plugin, use -up plugin1.py . For indicate a folder containing all the plugins: -up plugins/ . Default=''
-v	Show version

Templates

The parsing templates are looked for, by default, at the folder Templates/. logChecker reads the content of the folder to extract the several parsing templates.

If the CLI parameter -tf is missing or if a command/log doesn't match any of the available templates, the parsing operation will be done with a general best-effort basic template, which is already defined within the tool.

There are situations in which a comparison of logs must be done when different version of TIMOS have been used. For such cases it's possible to use different template folder: -tf to specify the template folder for the -pre logs, and -tf-post to specify the template folder for the -post logs.

Note

When different TIMOS versions have been used, the variables of the several templates must be the same for the comparison to be succesful.

Optional Features

Filtering Variables of Templates

Filtering of variables is possible. This can be done by simply adding the comments #filterAction: as exclude or include-only, and listing the variables in #filterColumns:, within the comment section of the templates at the very top.

When using #filterAction: exclude, all the variables listed under #filterColumns will be removed from the final generated dataFrame.

Similarly, if using #filterAction: include-only, the specified variables in #filterColumns are kept, while all the others are removed.

Go to Table of Contents

Using Plugins

It is possible to use plugins to create new tabs in the Excel file. These plugins interact with the parsed data and should be customized as needed.

If a plugin is used, specify it as -up pluginName.py. For multiple plugins, specify a folder, for example, using -up pluginFolder/. LogChecker will go through the folder's contents and use all .py files as plugins.

The plugin structure must follow:

pluginName.py

def usePlugin(dict_parsed):
    '''Plugin function that interacts with the dictionary containing the parsed data.

    Args:
        dict_parsed (dict): Dictionary with the parsed information.

    Returns:
        df_plg (dataFrame): Dataframe with the structure expected to be saved in new tab in Excel. In some cases, it is necessary to add the NAME or IP column, depending on the execution mode in -ri
        valueKeys_plg (list): It is necessary to identify at least one column from the dataframe df_plg to be considered as valueKeys when performing the comparison task.
        
        Or use df_plg = None and valueKeys_plg = None to use a plugin without save a new sheet in Excel.

    Notes:
       - For example, to access the parsed data dataframe of a specific template, use: dict_parsed['sh_port.template']['dfResultDatos']
    '''

    return df_plg, valueKeys_plg

If df_plg (DataFrame) and valueKeys_plg (list) are returned, the plugin will save the information in a new sheet in Excel. If return None, None , the plugin will not save the information in the Excel.

Go to Table of Contents

Results

Here are two simpler examples of how to run logChecker.

For more options, see the available parameters, as well as how to filter template variable and how to use plugins.

Tabulation of logs - Assessment

When doing an assesment, logChecker only needs an input folder with the stored logs. This can be done by using the -pre parameter.

logchecker -pre logs_pre/
##### Successfully Loaded Templates from folder Templates\ #####
##### Logs Loaded Successfully from folder logs_pre/ #####

Saving Excel
# 0 sh_srv_srv_using.template
# 1 sh_srv_sdp_using.template
# 2 sh_srv_sap_using.template
# 3 sh_srv_sdp.template
# 4 sh_port.template
# 5 sh_rtr_iface.template

Total running time: 0.45 seconds

Below is the flowchart of the use case where taskAutom is used to collect the data and logChecker is used to organize the collected information.

Use case: taskAutom and logChecker to make an assessment

Comparision of logs - Pre and post check

To do a comparison check of pre and post logs, an additional folder is needed. This can be done by using the -post parameter.

logchecker -pre logs_pre/ -post logs_post/
##### Successfully Loaded Templates from folder Templates\ #####
##### Successfully Loaded Templates from folder Templates\ #####
##### Logs Loaded Successfully from folder logs_pre/ #####
##### Logs Loaded Successfully from folder logs_post/ #####

Saving Excel
# 0 sh_srv_srv_using.template
# 1 sh_srv_sdp_using.template
# 2 sh_srv_sap_using.template
# 3 sh_srv_sdp.template
# 4 sh_port.template
# 5 sh_rtr_iface.template

Total running time: 0.46 seconds

Below is the flowchart of the use case in which taskAutom is used to collect the data and logChecker is used to compare the pre and post check.

Use case: taskAutom and logChecker to pre and post Check comparision

Go to Table of Contents