Skip to content

Commit

Permalink
Updated docs and fixed typos.
Browse files Browse the repository at this point in the history
  • Loading branch information
javihern98 committed Sep 30, 2024
1 parent 595a313 commit 847fdb2
Show file tree
Hide file tree
Showing 2 changed files with 32 additions and 23 deletions.
4 changes: 2 additions & 2 deletions docs/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,8 @@ VTL Engine Documentation

The VTL Engine is a Python library that allows you to validate and run VTL scripts.
It is a Python-based library around the `VTL Language 2.0 <http://sdmx.org/?page_id=5096>`_
VTL Language 2.1 will be supported soon.

*VTL Language 2.1 will be supported soon.*

Installation
************
Expand All @@ -29,7 +30,6 @@ To install the VTL Engine on any Operating System, you can use pip:
.. toctree::

index
walkthrough
api


51 changes: 30 additions & 21 deletions src/vtlengine/API/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -72,7 +72,7 @@ def semantic_analysis(script: Union[str, Path],
This vtl script can be a string with the actual expression or a filepath to the folder
that contains the vtl file.
Also, the data structure can be a dictionary or a filepath to the folder that contains it.
Moreover, the data structure can be a dictionary or a filepath to the folder that contains it.
If there are any value domains or external routines, this data is taken into account.
Both can be loaded the same way as data structures or vtl scripts are.
Expand All @@ -82,6 +82,7 @@ class takes all of this information and checks it with the ast generated to
return the semantic analysis result.
Concepts you may know:
- Vtl script: The expression that shows the operation to be done.
- Data Structure: Json file that contains the structure and the name for the dataset(s) \
Expand All @@ -97,11 +98,11 @@ class takes all of this information and checks it with the ast generated to
:param script: String or Path of the vtl expression.
:param data_structures: Dict or Path (file or folder), \
or List of Dicts or Paths with the data_structures json files.
or List of Dicts or Paths with the data structures JSON files.
:param value_domains: Dict or Path of the value_domains json files. (default: None)
:param value_domains: Dict or Path of the value domains JSON files. (default: None)
:param external_routines: String or Path of the external routines sql files. (default: None)
:param external_routines: String or Path of the external routines SQL files. (default: None)
:return: The computed datasets.
"""
Expand Down Expand Up @@ -135,19 +136,24 @@ def run(script: Union[str, Path], data_structures: Union[dict, Path, List[Union[
return_only_persistent=False,
output_folder: Optional[Union[str, Path]] = None):
"""
Run is the main function of the ``API``, which mission is to ensure the vtl operation is ready to be performed. When the vtl expression is given,
an AST object is created. This vtl script can be given as a string or a path with the folder or file that contains it.
Run is the main function of the ``API``, which mission is to ensure the vtl operation is ready
to be performed.
When the vtl expression is given, an AST object is created.
This vtl script can be given as a string or a path with the folder or file that contains it.
At the same time, data structures are loaded with its datapoints.
The data structure information is contained in the json file given, and establish the datatype (string, integer or number),
The data structure information is contained in the JSON file given,
and establish the datatype (string, integer or number),
and the role that each component is going to have (Identifier or Measure).
It can be a dictionary or a path to the json file or folder that contains it.
It can be a dictionary or a path to the JSON file or folder that contains it.
Moreover, a csv file with the data to operate with is going to be loaded.
It can be given with a dictionary (dataset name : pandas Dataframe),
a path or S3 URI to the folder, path or S3 to the csv file that contains the data.
.. important:: The data structure and the data points must have the same dataset name to be loaded correctly.
.. important::
The data structure and the data points must have the same dataset
name to be loaded correctly.
.. important::
If pointing to a Path or an S3 URI, dataset_name will be taken from the file name.
Expand All @@ -168,18 +174,22 @@ def run(script: Union[str, Path], data_structures: Union[dict, Path, List[Union[
Before the execution, the DAG analysis reviews if the VTL script is a direct acyclic graphs.
This information is taken by the Interpreter class, to analyze if the operation correlates with the AST object.
Consequently, if value domain data or external routines are required, the function loads this information and integrates
them into the :obj:`Interpreter <vtl-engine-spark.Interpreter.InterpreterAnalyzer>` class. Moreover,
if any component has a Time_Period component, the external representation is passed to the Interpreter class.
If value domain data or external routines are required, the function loads this information
and integrates them into the
:obj:`Interpreter <vtl-engine-spark.Interpreter.InterpreterAnalyzer>` class.
Moreover, if any component has a Time Period component, the external representation
is passed to the Interpreter class.
Concepts you may need to know:
- Vtl script: The expression that shows the operation to be done.
- Data Structure: \
Json file that contains the structure and the name for the dataset(s) (and/or scalar) \
about the datatype (String, integer or number) and the role (Measure or Identifier) each data has.
JSON file that contains the structure and the name for the dataset(s) (and/or scalar) \
about the datatype (String, integer or number) and the role (Identifier, Attribute or Measure)
each component has.
- Data point: Pointer to the data. It will be loaded as a `Pandas Dataframe \
<https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.html>`_.
Expand All @@ -194,11 +204,11 @@ def run(script: Union[str, Path], data_structures: Union[dict, Path, List[Union[
:param data_structures: Dict, Path or a List of Dicts or Paths with the data structures.
:param datapoints: Dict, Path, S3 URI or List of S3URIs or Paths with data.
:param datapoints: Dict, Path, S3 URI or List of S3 URIs or Paths with data.
:param value_domains: Dict or Path of the value_domains json files. (default:None)
:param value_domains: Dict or Path of the value domains JSON files. (default:None)
:param external_routines: String or Path of the external routines sql files. (default: None)
:param external_routines: String or Path of the external routines SQL files. (default: None)
:param time_period_output_format: String with the possible values \
("sdmx_gregorian", "sdmx_reporting", "vtl") for the representation of the \
Expand All @@ -209,7 +219,7 @@ def run(script: Union[str, Path], data_structures: Union[dict, Path, List[Union[
:param output_folder: Path or S3 URI to the output folder. (default: None)
:return: The datasets produced, without data if output_folder is defined.
:return: The datasets are produced without data if the output folder is defined.
"""
# AST generation
Expand All @@ -233,11 +243,10 @@ def run(script: Union[str, Path], data_structures: Union[dict, Path, List[Union[
# VTL Efficient analysis
ds_analysis = DAGAnalyzer.ds_structure(ast)

# Checking output path to be a Path object to a directory
# Checking the output path to be a Path object to a directory
if output_folder is not None:
_check_output_folder(output_folder)


# Running the interpreter
interpreter = InterpreterAnalyzer(datasets=datasets, value_domains=vd,
external_routines=ext_routines,
Expand Down

0 comments on commit 847fdb2

Please sign in to comment.