Home

ExeTeraEval

This repository contains the scripts used to benchmark ExeTera, Pandas, Dask and PostgreSQL relative to each other. It contains scripts for generating artificial data, and running benchmarks on both artificial and real data.

Import scripts

These scripts are used to import data from csv to hdf5. The data used in official benchmarks is the Covid Symptom Study, a copy of which (with the exception of a few fields containing identifiable information) can be obtained by through the Health Data Gateway by searching for 'Covid Symptom Study'.

Importing is benchmarked on the patients, assessments, and tests tables.

ExeTera import

ExeTera import is performed using ExeTera's import command, details of which can be found on the ExeTera wiki.

Pandas import

Pandas import can be carried out using the import_patients_pandas.py script. Despite its name, it is used to import the three tables described above, as follows:

python import_patients_pandas.py <csv_file_name> <hdf5_file_name>