User Guide

Minerva can work in either of two modes:

• Dry mode
  In dry mode you can run or train the pipeline to make sure that everything is working correctly.
• Submit mode
  Submit mode is the main Minerva mode where you exchange one step from our pipeline with your own solution and train and evaluate the modified model.

Also, three types of Neptune support are available:

• No Neptune
  Choose this option if you want to work without Neptune support.
• Neptune locally
  Choose this option if you want to run the pipeline locally and use Neptune to visualize the results.
• Neptune's cloud
  Choose this option if you want to run the pipeline on the cloud available through Neptune.

Dry mode

In dry mode you can run or train the pipeline to make sure that everything is working correctly. We provided two dry sub-modes:

• Dry eval
  In dry eval sub-mode you can run an existing pipeline and evaluate it.
• Dry train
  In dry train sub-mode you can train a new instance of the pipeline and then evaluate it.

Dry eval

In dry run mode you can run our pipeline to make sure that everything is working correctly.

No Neptune

1. Download data (once):

   fashion mnist	whales
   Data are downloaded automatically.	Download file imgs.zip from Right Whale Recognition challenge site on kaggle (you must be logged in to kaggle to do that). Extract imgs.zip to resources/whales/data/. After that, folder resources/whales/data/ should contain two elements: file metadata.csv and folder imgs with images.

2. In neptune.yaml file:

   • Comment pip-requirements-file line.

   • Uncomment Local setup paths and set them as follows:

     	fashion mnist	whales
     data_dir	doesn't matter	resources/whales/data/
     solution_dir	resources/fashion_mnist/solution	resources/whales/solution/

     Note: you can also set another solution_dir if you previously trained another pipeline instance with use of dry train sub-mode.

   • Comment Cloud setup paths.

3. Type:

   fashion mnist	whales
   python main.py -- dry_eval --problem fashion_mnist	python main.py -- dry_eval --problem whales
   Run time with GPU: less than 1 minute.	Run time with GPU: about 3 minutes.

If you want to run our pipeline locally and use Neptune to visualize the results:

1. Edit neptune.yaml file in the same way like in point 1.
2. Type:

neptune run -- dry_run --problem fashion_mnist

Run time with GPU: less than 1 minute. (15 - train)

If you want to run our pipeline on the cloud available through Neptune:

1. In neptune.yaml file: - uncomment pip-requirements-file line, - comment Local setup paths, - uncomment Cloud setup paths, - data_dir path doesn't matter in fashion_mnist problem, - set solution_dir as /public/minerva/resources/fashion_mnist/solution.
2. Type:

neptune send \
--environment keras-2.0-gpu-py3 \
--worker gcp-gpu-medium \
-- dry_run --problem fashion_mnist

Run time: TODO

Dry train

Submit mode

Submit mode is the main Minerva mode where you exchange one step from our pipeline with your own solution and train and evaluate the modified model.