Tutorial: Fine-Tuning a Model on Your Own Data -> any ideas for example evaluation?

[JayBroe]

Hi!

I'm looking for help escpecially for tutorial with trainng of the custom model
(https://haystack.deepset.ai/tutorials/02_finetune_a_model_on_your_data).

My question is: could anyone provide a simple example of the evaluation in this case?
I've tried doing it by myself, but figured out connecting it with different tutorials
was not a good idea.

[bogdankostic]

Hi @JayBroe! When calling reader.train, you can specify the parameters dev_filename and test_filename. When you specify dev_filename, you will get a performance report on your development test every evaluate_every steps. When you specify test_filename, you will get a performance report on your test set at the end of the training run.

To evaluate a Reader after training has already completed, this section of our evaluation tutorial might be interesting for you.